mccd.auxiliary_fun

AUXILIARY FUNCTIONS.

Essential MCCD helper functions. They are needed to:

  • Generate simulated datasets.

  • Preprocess, fit and validate MCCD PSF models.

  • Parse the configuration file.

  • Run automatically the MCCD algortithm from the config file parameters.

Authors

Tobias Liaudat <tobias.liaudat@cea.fr>

class GenerateSimDataset(input_pos_path, input_ccd_path, output_path, e1_analytic_fun=None, e2_analytic_fun=None)[source]

Bases: object

Generate simulated dataset for training and validating PSF models.

Parameters

  • input_pos_path (str) – Path to the global positions of the PSF that will be used for the training.

  • input_ccd_path (str) – Path to the corresponding CCDs of the global positions.

  • output_path (str) – Path to the folder to save the simulated datasets.

  • e1_analytic_fun (function) – The analytic e1 ellipticity function that will define an ellipticity e1 for each position in the focal plane.

  • e2_analytic_fun (function) – The analytic e2 ellipticity function that will define an ellipticity e2 for each position in the focal plane.

Notes

The simulated PSFs are based on the Moffat profile and we are using Galsim to generate them. We base ourselves on two analytic functions that have to output an ellipticity for each position in the focal plane.

load_data()[source]

Load data from input paths.

generate_train_data(sigma=1.6, image_size=51, psf_flux=1.0, beta_psf=4.8, pix_scale=0.187, desired_SNR=30, catalog_id=2086592)[source]

Generate the training dataset and saves it in fits format.

Parameters

  • sigma (float) – Size of the PSF in sigma’s. (Sigma from Galsim’s HSM adaptive moments). Default is 1.6.

  • image_size (int) – Dimension of the squared image stamp. (image_size x image_size) Default is 51.

  • psf_flux (float) – Total PSF photometric flux. Default is 1.

  • beta_psf (float) – Moffat beta parameter. Default is 4.8.

  • pix_scale (float) – Pixel scale. Default is 0.187.

  • desired_SNR (float) – Desired SNR Default is 30.

  • catalog_id (int) – Catalog identifier number. Default is 2086592.

generate_test_data(x_grid=5, y_grid=10, n_ccd=40)[source]

Generate the test dataset and save it into a fits file.

Parameters

  • x_grid (int) – Horizontal number of elements of the testing grid in one CCD.

  • y_grid (int) – Vertical number of elements of the testing grid in one CCD.

  • n_ccd (int) – Number of CCDs in the instrument.

Notes

n_ccd should be coherent with the corresponding functions on mccd.mccd_utils that do the change of coordiante system.

static e1_catalog_fun(x, y)[source]

Define an e1 ellipticity per position.

Analytic function for defining the e1 ellipticity as a function of the global position (MegaCam).

static e2_catalog_fun(x, y)[source]

Define an e2 ellipticity per position.

Analytic function for defining the e2 ellipticity as a function of the global position (MegaCam).

static scale_coordinates(x, y, coor_min, coor_max, offset=None)[source]

Scale global coordinates.

static bessel_generator(x, y, coor_min, coor_max, max_order, scale_factor, circular_symetry=False, exp_decay_alpha=None, offset=None)[source]

Generate a type of Bessel function response.

static handle_SExtractor_mask(stars, thresh)[source]

Handle SExtractor masks.

Reads SExtracted star stamps, generates MCCD-compatible masks (that is, binary weights), and replaces bad pixels with 0s - they will not be used by MCCD, but the ridiculous numerical values can otherwise still lead to problems because of convolutions.

mccd_fit(starcat, mccd_inst_kw, mccd_fit_kw, output_dir='./', catalog_id=1234567, sex_thresh=- 100000.0, use_SNR_weight=False, verbose=False, saving_name='fitted_model')[source]

Fits (train) the MCCD model.

Then saves it.

Parameters

  • starcat (fits table) – Opened fits file containing a fits table, ie astropy.io.fits(path)[1]. Should contain the training dataset with the columns: Mandatory: CCD_ID_LIST, VIGNET_LIST, GLOB_POSITION_IMG_LIST, MASK_LIST. Optional: SNR_WIN_LIST.

  • mccd_inst_kw (dict) – Parameters for the MCCD class initialization.

  • mccd_fit_kw (dict) – Parameters for the MCCD class fit() function.

  • output_dir (str) – Path to the directory to save the fitted model.

  • catalog_id (int) – Id of the catalog being trained. Default is 1234567.

  • sex_thresh (float) – Masking threshold, specially for SExtractor catalogs. Default is -1e5.

  • use_SNR_weight (bool) – Boolean to decide to use the SNR weight strategy. The columns SNR_WIN_LIST should be available on the input fits table.

  • verbose (bool) – Verbose mode. Default is False.

  • saving_name (str) – Name of the fitted model file. Default is fitted_model.

Notes

Saves the fitted model intro a .npy element on output_dir.

mccd_validation(mccd_model_path, testcat, apply_degradation=True, mccd_debug=False, global_pol_interp=False, sex_thresh=- 100000.0, loc2glob=None)[source]

Validate a MCCD model.

Parameters

  • mccd_model_path (str) – Path to the saved trained MCCD model.

  • testcat (fits table) – Opened fits file containing a fits table, ie astropy.io.fits(path)[1]. Should contain the testing dataset with the columns: Mandatory: CCD_ID_LIST, VIGNET_LIST, GLOB_POSITION_IMG_LIST, MASK_LIST. Optional: RA_LIST, DEC_LIST.

  • apply_degradation (bool) – Boolean determining if the returned PSFs should be matched to the observed stars by the application of a degradation which consists of an intra-pixel shift, flux matching, etc. Default is True.

  • mccd_debug (bool) – Boolean to determine if the validation will be run in debug mode. Default is False.

  • global_pol_interp (bool) – Boolean to determine if the global model interpolation is done with a new position matrix (True) or if it is done with the RBF kernel interpolation (False). Default is False.

  • sex_thresh (float) – Masking threshold, specially for SExtractor catalogs. Default is -1e5.

  • loc2glob (object) – The object that allows to do the coordinate conversion from local to global. It is specific for each instrument’s focal plane geometry. Default is None that uses the CFIS MegaCam geometry.

Returns

star_dict – Dictionary containing the data needed for validation purposes. Keys: PSF_VIGNET_LIST, PSF_MOM_LIST, STAR_MOM_LIST, GLOB_POSITION_IMG_LIST, VIGNET_LIST, MASK_LIST, CCD_ID_LIST. Optional keys: RA_LIST, DEC_LIST, PSF_GLOB_VIGNET_LIST, PSF_LOC_VIGNET_LIST.

Return type

dict

Notes

The parameter apply_degradation needs to be set to True if a pixel validation of the model is going to be done.

mccd_preprocessing(input_folder_path, output_path, min_n_stars=20, file_pattern='sexcat-*-*.fits', separator='-', CCD_id_filter_list=None, outlier_std_max=100.0, save_masks=True, save_name='train_star_selection', save_extension='.fits', verbose=True, loc2glob=None, fits_tb_pos=2)[source]

Preprocess input catalog.

Parameters

  • input_folder_path (str) – Path to the folder containing the files to preprocess.

  • output_path (str) – Path to the folder where to save the preprocessed files.

  • min_n_stars (int) – Minimum number of stars in order to preprocess the CCD. Default is 20.

  • file_pattern (str) – Input file pattern as a regex expression. Only the files matching the file_pattern in input_folder_path will be treated. Default is sexcat-*-*.fits. Where the first * corresponds to the catalog_id, the second * corresponds to the CCD id and they are separated by the separator.

  • separator (str) – Separator string that separates the catalog id and the CCD id. Default is '-'.

  • CCD_id_filter_list (list of int or None) – A list that correspond to the CCDs that should be preprocessed. If it is None, all the CCDs are preprocessed. (Current version: Hardcoded for the MegaCam scenario). Default is None.

  • outlier_std_max (float) – Parameter regulating the shape outlier removal. Default is very high so as it is not done at all. A decent number would be 10. Default is 100..

  • save_masks (bool) – If masks should be saved in the new file. Default is True.

  • save_name (str) – Name to save the preprocessed file. Default is 'train_star_selection'.

  • save_extension (str) – Extension of the saved file. Default is .fits.

  • verbose (bool) – Verbose mode. Default is True.

  • loc2glob (object) – The object that allows to do the coordinate conversion from local to global. It is specific for each instrument’s focal plane geometry. Default is None that uses the CFIS MegaCam’s geometry.

  • fits_tb_pos (int) – Position in the fits file of the useful table. Default is 2 that is the default for CFIS fits files.

Returns

mccd_inputs – An instance of MccdInputs class used for the input preprocessing.

Return type

class

class MCCDParamsParser(file_path)[source]

Bases: object

Parse MCCD config file.

Set up a parser for the MCCD parameters.

Parameters

file_path (str) – Path to the config file.

Raises

IOError – For non existent configuration file.

parse_document()[source]

Parse config file.

get_extra_kw(param_name)[source]

Get parameter from extra arguments.

Returns

param_name – Name of the parameter

Return type

str

get_fit_kw()[source]

Get fit parameter dictionary.

Returns

mccd_fit_kw – MCCD fit parameter dictionary.

Return type

dict

get_instance_kw()[source]

Get instace parameter dictionary.

Returns

mccd_inst_kw – MCCD instance parameter dictionary.

Return type

dict

get_inputs_kw()[source]

Get paths parameter dictionary.

Returns

mccd_inputs_kw – MCCD input parameter dictionary.

Return type

dict

get_val_prepro_kw()[source]

Get preprocessing validation input dictionary.

Returns

mccd_val_prepro_kw – MCCD preprocessing validation input dictionary.

Return type

dict

get_val_kw()[source]

Get validaiton parameter dictionary.

Returns

mccd_val_kw – MCCD validation dictionary.

Return type

dict

class RunMCCD(config_file_path, fits_table_pos=1, verbose=True)[source]

Bases: object

Run the MCCD method.

This class allows to run the MCCD method using the paramters present on the configuration file. Saves the fitted model on the output directory found in the MCCD configuration file.

Parameters

  • config_file_path (str) – Path to the configuration file.

  • fits_table_pos (int) – Position of the Table in the fits file. Default is 1.

  • verbose (bool) – Verbose mode. Default is True.

Notes

Missing: - Method including the validation. - Erase the preprocessed files (?) - Handling different geometries

parse_config_file()[source]

Parse configuration file and recover parameters.

preprocess_inputs()[source]

Preprocess the input data.

preprocess_val_inputs()[source]

Preprocess validation input data.

fit_models()[source]

Build and save the models to the catalgos found.

validate_models()[source]

Validate MCCD models.

fit_MCCD_models()[source]

Fit MCCD models.

validate_MCCD_models()[source]

Validate MCCD models.

run_MCCD()[source]

Run the MCCD routines.

recover_MCCD_PSFs(mccd_model_path, positions, ccd_id, local_pos=False)[source]

Recover MCCD PSFs at required positions.

Parameters

  • mccd_model_path (str) – Path pointing to the saved fitted MCCD model to be used.

  • positions (numpy.ndarray) – Array containing the positions where the PSF should be recovered. The shape of the array should be (n,2) [x,y].

  • ccd_id (int) – Id of the CCD from where the positions where taken.

  • local_pos (bool) – If the positions passed are local to the CCD. If False, the positions are considered to be in the same format (coordinate system, units, etc.) as the obs_pos fed to MCCD.fit(). Default is False.

Returns

rec_PSFs – Array containing the recovered PSFs. Array dimensions: (n_psf, n_im, n_im).

Return type

numpy.ndarray

Raises

  • OSError – For non-existent fitted model.

  • ValueError – For ccd_id not being an integer.