shapepipe.pipeline.file_io

shapepipe.pipeline.file_io

FILE IO.

This file contains methods for file I/O handling.

Author

Marc Gentile and Axel Guinot

class BaseCatalogue(fullpath)[source]

Bases: object

Base catalogue.

Base catalogue management class.

Parameters

fullpath (str) – Full path to catalogue

Notes

This class is not meant to be used directly

property fullpath

Set Full Path.

Get the full path of the catalogue file

property directory

Set Directory.

Get the directory of the catalogue file

property filename

Set File Name.

Get the name of the catalogue file

property format

Format.

Get the default input/output format of the catalogue (e.g. Text, SExtractor, FITS)

get_nb_rows()[source]

Get Number of Rows.

Get the number of rows in the catalogue

Returns

Number of rows

Return type

int

get_nb_cols()[source]

Get Number of Columns.

Get the number of columns in the catalogue

Returns

Number of columns

Return type

int

get_col_names()[source]

Get Column Names.

Get the list of column names in the catalogue

Returns

list of column names

Return type

list

get_col_formats()[source]

Get Column Formats.

Get the list of column formats in the order of columns

add_col(col_name, col_format=None, col_comment=None, col_data=None)[source]

Add Column.

Add a Column to the catalogue

Parameters

  • col_name (str) – Column name

  • col_format (str) – Column python format

  • col_comment (str) – Column comment (ignored in for class)

  • col_data (numpy.ndarray) – Column data as a numpy array

_file_exists(filepath)[source]

File Exists.

Check if input file path is a valid file.

Parameters

filepath (str) – Path to file

class InputFormat[source]

Bases: object

Supported input catalogue formats.

Undefined = 0
TabulatedText = 1
SExtractor = 2
FITS = 4
FITS_LDAC = 5
class OpenMode[source]

Bases: object

Supported input catalogue open modes.

ReadOnly = 'readonly'
ReadWrite = 'update'
Append = 'append'
class Column[source]

Bases: object

Column.

Represents a column in the catalogue.

property name

Set Name.

Get the name of the column

property format

Format.

Get the format of the column

property data

Set Data.

Get the data associated with the column

get_nb_rows()[source]

Get Number of Rows.

Retrieve the number of rows of the column.

get_info()[source]

Get Information.

Retrieve information about the column.

get_type()[source]

Get Type.

Get the data type of the column

exception FeatureNotImplemented(msg)[source]

Bases: NotImplementedError

Feature Not Implemented.

Parameters

msg (str) – Message

exception catalogueNotOpen(filepath)[source]

Bases: Exception

Catalogue Not Open.

Catalogue has not been open yet.

Parameters

filepath (str) – Path to file

exception DataNotFound(filepath, hdu)[source]

Bases: Exception

Data Not Found.

No data found (at given hdu).

Parameters

  • filepath (str) – Path to file

  • hdu (int) – HDU number

exception catalogueFileNotFound(filepath)[source]

Bases: Exception

Catalogue File Not Found.

Exception thrown when a catalogue file is not found on disk.

Parameters

filepath (str) – Path to file

exception ColumnNotFound(col_name)[source]

Bases: Exception

Column Not Found.

Exception thrown when a named catalogue column could not be found

Parameters

col_name (str) – Column name

exception catalogueNotCreated(filepath)[source]

Bases: Exception

Catalogue Not Created.

Catalogue could not be created.

Parameters

filepath (str) – Path to file

exception OpenModeNotSupported(filepath, open_mode)[source]

Bases: Exception

Open Mode Not Supported.

Opening mode not supported by this version.

Parameters

  • filepath (str) – Path to file

  • open_mode (OpenMode) – File opening mode

exception OpenModeConflict(open_mode, open_mode_needed)[source]

Bases: Exception

Open Mode Conflict.

Opening mode is in conflict with an action.

Parameters

  • open_mode (OpenMode) – File opening mode

  • open_mode_needed (OpenMode) – File opening mode needed

class FITSCatalogue(fullpath, hdu_no=None, open_mode='readonly', memmap=False, SEx_catalogue=False)[source]

Bases: shapepipe.pipeline.file_io.BaseCatalogue

FITS Catalogue.

Catalogues management in .FITS format.

Parameters

  • fullpath (str) – Full path to file

  • hdu_no (int) – HDU number

  • open_mode (OpenMode) – File opening mode

  • memmap (Bool) – Option to use memory mapping

  • SEx_catalogue (bool) – Option to specify if the input is a SExtractor catalogue

property hdu_no

Set HDU Number.

Get the HDU index of the table.

property open_mode

Set Open Mode.

Return the catalogue opening mode.

See also

FITSCatalogue.OpenMode

property use_memmap

Use Memory Map.

If True, use memory mapping.

open()[source]

Open.

Open an existing catalogue.

create(ext_name=None, s_hdu=True, sex_cat_path=None)[source]

Create.

Create an empty catalogue in FITS format.

Parameters

  • ext_name (str) – Extension name or number

  • s_hdu (bool) – If true add a secondary HDU

  • sex_cat_path (str) – Path to SEXtractor catalogue

copy_hdu(fits_file=None, hdu_no=None, hdu_name=None)[source]

Copy HDU.

Copy an HDU from a FITS file.

Parameters

  • fits_file (str) – Catalogue containing the HDU

  • hdu_no (int) – HDU number

  • hdu_name (str) – HDU name

apply_mask(fits_file=None, hdu_no=None, mask=None, hdu_name=None)[source]

Apply Mask.

Apply a mask to data for a specified HDU.

Parameters

  • fits_file (str) – FITS file containing the data to be masked

  • hdu_no (int) – HDU number

  • mask (numpy.ndarray) – Array of booleans or an array of indices

  • hdu_name (str) – HDU name

Notes

This will create a new HDU containing data[mask]

save_as_fits(data=None, names=None, ext_name=None, sex_cat_path=None, image=False, image_header=None, overwrite=False)[source]

Save as FITS.

Save data into an existing FITS file or into a new one.

Save data from dict, list, numpy.ndarray, numpy.recarray or astropy.io.fits.fitsrec.FITS_rec (data format in an astropy fits file) When creating a new FITS to store BinTable data it create a PrimaryHDU. When creating a new FITS to store Image there is no PrimaryHDU. You can create a SExtractor format FITS by specifying a SExtractor catalogue from where data come from.

Parameters

  • data (numpy.ndarray) – Data to be stored

  • names (list) – List of column names

  • ext_name (str) – Name of the HDU where data are stored

  • sex_cat_path (str) – Path of the existing SExtractor catalogue to mimic

  • image (bool) – If true create a fits image

  • image_header (astropy.io.fits.header) – Header to use when saving an image

  • overwrite (bool) – Option to overwrite an existing image, only used when creating a FITS image

Notes

To create a SExtractor-like FITS file you need to specify SEx_catalogue=True when declaring the FITSCatalogue object.

create_from_numpy(matrix, col_names, ext_name=None, ext_ver=None, header=None)[source]

Create from Numpy.

Create a new catalogue from a two-dimensional numpy array.

Parameters

  • matrix (numpy.ndarray) – Two-dimensional numpy array

  • col_names (list) – List of column names to use as the header

  • ext_name (str) – Extension name or number

  • ext_ver (str) – Extension version

  • header (list) – List of dictionaries with keys: ‘card’, name’, ‘value’, ‘value_orig’, ‘comment’

close()[source]

Close.

save()[source]

Save.

get_nb_rows(hdu_no=None)[source]

Get Number of Rows.

Get the number of rows in the catalogue.

Parameters

hdu_no (int) – HDU index

Returns

Number of rows or 0 if None

Return type

int

get_nb_cols(hdu_no=None)[source]

Get Number of Columns.

Get the number of columns in the catalogue.

Parameters

hdu_no (int) – HDU index

Returns

Number of columns or 0 if None

Return type

int

get_col_names(hdu_no=None)[source]

Get Column Names.

Get the list of column names in the catalogue.

Parameters

hdu_no (int) – HDU index

Returns

list of column names

Return type

list

get_info()[source]

Get Information.

Retrieve some information about the catalogue.

Returns

A dictionary with detailed information

Return type

dict

Notes

See the fitsio documentation of the info() function for the details

get_ext_name(hdu_no=None)[source]

Get Extension Name.

Return the name of an extansion or all of them.

Parameters

hdu_no (int) – index of the hdu to return, if None return all of them

Returns

The hdu name or a list of strings

Return type

str or list

col_exists(col_name, hdu_no=None)[source]

Column Exists.

Check whether a named column exists.

Parameters

  • col_name (str) – Column name

  • hdu_no (int) – HDU index

Returns

True if column exists, False otherwise

Return type

bool

get_col_index(col_name, hdu_no=None)[source]

Get Column Index.

Get the column index from a column name.

Parameters

  • col_name (str) – Column name

  • hdu_no (int) – HDU index

Returns

Column index (zero-based)

Return type

int

get_col_data(col_index, hdu_no=None)[source]

Get Column Data.

Return the data of a column from its index.

Parameters

  • col_name (str) – Column name

  • hdu_no (int) – HDU index

Returns

Data associated with the column

Return type

numpy.ndarray

get_named_col_data(col_name, hdu_no=None)[source]

Get Named Column Data.

Return the data of a column from its index (zero-based).

Parameters

  • col_name (str) – Column name

  • hdu_no (int) – HDU index

Returns

Data associated with the column

Return type

numpy.ndarray

get_data(hdu_no=None)[source]

Get Data.

Return data of the specified hdu.

Parameters

hdu_no (int) – HDU index

Returns

Data associated with the HDU

Return type

numpy.ndarray

get_header(hdu_no=None)[source]

Get Header.

Return the catalogue header as a list of dictionaries.

Parameters

hdu_no (int) – HDU index

Returns

FITS header

Return type

astropy.io.fits.header

Notes

See astropy documentation

get_header_value(request, hdu_no=None)[source]

Get Header Value.

Return the value of a parameters or a linear combination of parameters and/or numbers.

Parameters

  • request (str) – Request parameter or a linear combination of parameters

  • hdu_no (int) – HDU index

Returns

Result of the request

Return type

float

add_header_card(key, value=None, comment=None, hdu_no=None)[source]

Add Header Card.

Add a card in the header of the specified HDU.

Parameters

  • key (str) – The key to add

  • value (any) – The value of the key

  • comment (str) – Comment for the key

  • hdu_no (int) – HDU index

get_headers()[source]

Get Headers.

Return the catalogue header as a list of dictionaries.

Returns

list of headers

Return type

list

get_comments(hdu_no=None)[source]

Get Comments.

Return the list catalogue comments.

Parameters

hdu_no (int) – HDU index

Returns

List of catalogue comments

Return type

list

get_col_comments(hdu_no=None)[source]

Get Column Comments.

Return the list of column comments.

Parameters

hdu_no (int) – HDU index

Returns

List of column comments

Return type

list

get_col_formats(hdu_no=None)[source]

Get Column Formats.

Get the list of python column formats in the order of columns.

Parameters

hdu_no (int) – HDU index

Returns

List of column formats

Return type

list

add_col(col_name, col_data, hdu_no=None, ext_name=None, new_cat=False, new_cat_inst=None)[source]

Add Column.

Add a Column to the catalogue.

Parameters

  • col_name (str) – Column name

  • col_data (numpy.ndarray) – Column data

  • hdu_no (int) – HDU index

  • ext_name (str, optional) – Change the name of the extansion

  • new_cat (bool, optional) – If true will save the changes into a new catalogue

  • new_cat_inst (io.FITSCatalogue) – New catalogue object

remove_col(col_index)[source]

Remove Column.

Delete a column from its index.

Parameters

col_index (int) – Index of the column to delete

remove_named_col(col_name)[source]

Remove Named Column.

Delete a column from its index.

Parameters

col_name (str) – Name of the column to delete

_append_col(column, hdu_no=None)[source]

Append Column.

Append a Column object.

Parameters

  • column – An object derived from BaseCatalogue.Column

  • hdu_no (int) – HDU index

_get_fits_col_type(col_data)[source]

Get FITS Column Type.

Get the FITS data type of a given column.

Parameters

col_data (any) – Column data

Returns

Column FITS data type

Return type

str

_get_python_col_type(col_type)[source]

Get Python Column Type.

Get the Python data type of a given column.

Parameters

col_data (any) – Column data

Returns

Column Python data type

Return type

str

_save_to_fits(data, names, it, ext_name=None, sex_cat_path=None, overwrite=False)[source]

Save to FITS.

Save array of data as fits with their associated column names.

Parameters

  • data (numpy.ndarray) – Array with the data

  • names (list) – List of the column names

  • it (iterator) –

    ?

  • ext_name (str) – Name of the HDU where data are stored

  • sex_cat_path (str) – Path of the existing SExtractor catalogue to mimic

  • overwrite (bool) – Option to overwrite an existing catalogue

_save_from_recarray(data=None, ext_name=None, sex_cat_path=None, overwrite=False)[source]

Save From Record Array.

Save a numpy.recarray or astropy.io.fits.fitsrec.FITS_rec into a FITS file.

Parameters

  • data (numpy.ndarray) – Array with the data

  • ext_name (str) – Name of the HDU where data are stored

  • sex_cat_path (str) – Path of the existing SExtractor catalogue to mimic

  • overwrite (bool) – Option to overwrite an existing catalogue

_save_image(data=None, header=None, overwrite=False)[source]

Save Image.

Save an image into a FITS file.

Parameters

  • data (numpy.ndarray) – Array with the data

  • header (astropy.io.fits.header) – FITS header

  • overwrite (bool) – Option to overwrite an existing catalogue

class Column(name, format=None, comment=None, data=None)[source]

Bases: shapepipe.pipeline.file_io.BaseCatalogue.Column

Column.

Represents a column in the catalogue.

Parameters

  • name (str) – Name name of the column

  • format (str) – Python format of the column

  • comment (str) – Comment strin

  • data (numpy.ndarray) – Associated column data

property name

Set Name.

Get the name of the column.

property format

Format.

Get the format of the column.

property comment

Comment.

Get the comment string associated with the column.

property data

Set Data.

Get the data associated with the column.

get_unit_from_fits_header(header, key)[source]

Get Unit From FITS Header.

Return coordinate unit corresponding to column with name key.

Parameters

  • header (FITS header) – Header information

  • key (str) – Column name

Raises

IndexError – If key not in header

Returns

Unit object

Return type

astropy.units.Unit

previous

shapepipe.pipeline.file_handler

next

shapepipe.pipeline.job_handler