mccd.auxiliary_fun
AUXILIARY FUNCTIONS.
Essential MCCD helper functions. They are needed to:
Generate simulated datasets.
Preprocess, fit and validate MCCD PSF models.
Parse the configuration file.
Run automatically the MCCD algortithm from the config file parameters.
- Authors
Tobias Liaudat <tobias.liaudat@cea.fr>
- class GenerateSimDataset(input_pos_path, input_ccd_path, output_path, e1_analytic_fun=None, e2_analytic_fun=None)[source]
-
Bases:
object
Generate simulated dataset for training and validating PSF models.
- Parameters
input_pos_path (str) – Path to the global positions of the PSF that will be used for the training.
input_ccd_path (str) – Path to the corresponding CCDs of the global positions.
output_path (str) – Path to the folder to save the simulated datasets.
e1_analytic_fun (function) – The analytic e1 ellipticity function that will define an ellipticity e1 for each position in the focal plane.
e2_analytic_fun (function) – The analytic e2 ellipticity function that will define an ellipticity e2 for each position in the focal plane.
Notes
The simulated PSFs are based on the Moffat profile and we are using Galsim to generate them. We base ourselves on two analytic functions that have to output an ellipticity for each position in the focal plane.
- load_data()[source]
-
Load data from input paths.
- generate_train_data(sigma=1.6, image_size=51, psf_flux=1.0, beta_psf=4.8, pix_scale=0.187, desired_SNR=30, catalog_id=2086592)[source]
-
Generate the training dataset and saves it in fits format.
- Parameters
sigma (float) – Size of the PSF in sigma’s. (Sigma from Galsim’s HSM adaptive moments). Default is
1.6
.image_size (int) – Dimension of the squared image stamp. (image_size x image_size) Default is
51
.psf_flux (float) – Total PSF photometric flux. Default is
1
.beta_psf (float) – Moffat beta parameter. Default is
4.8
.pix_scale (float) – Pixel scale. Default is
0.187
.desired_SNR (float) – Desired SNR Default is
30
.catalog_id (int) – Catalog identifier number. Default is
2086592
.
- generate_test_data(x_grid=5, y_grid=10, n_ccd=40)[source]
-
Generate the test dataset and save it into a fits file.
- Parameters
Notes
n_ccd
should be coherent with the corresponding functions onmccd.mccd_utils
that do the change of coordiante system.
- static e1_catalog_fun(x, y)[source]
-
Define an e1 ellipticity per position.
Analytic function for defining the e1 ellipticity as a function of the global position (MegaCam).
- static e2_catalog_fun(x, y)[source]
-
Define an e2 ellipticity per position.
Analytic function for defining the e2 ellipticity as a function of the global position (MegaCam).
- static scale_coordinates(x, y, coor_min, coor_max, offset=None)[source]
-
Scale global coordinates.
- static bessel_generator(x, y, coor_min, coor_max, max_order, scale_factor, circular_symetry=False, exp_decay_alpha=None, offset=None)[source]
-
Generate a type of Bessel function response.
- static handle_SExtractor_mask(stars, thresh)[source]
-
Handle SExtractor masks.
Reads SExtracted star stamps, generates MCCD-compatible masks (that is, binary weights), and replaces bad pixels with 0s - they will not be used by MCCD, but the ridiculous numerical values can otherwise still lead to problems because of convolutions.
- mccd_fit(starcat, mccd_inst_kw, mccd_fit_kw, output_dir='./', catalog_id=1234567, sex_thresh=- 100000.0, use_SNR_weight=False, verbose=False, saving_name='fitted_model')[source]
-
Fits (train) the MCCD model.
Then saves it.
- Parameters
starcat (fits table) – Opened fits file containing a fits table, ie
astropy.io.fits(path)[1]
. Should contain the training dataset with the columns: Mandatory:CCD_ID_LIST
,VIGNET_LIST
,GLOB_POSITION_IMG_LIST
,MASK_LIST
. Optional:SNR_WIN_LIST
.mccd_inst_kw (dict) – Parameters for the MCCD class initialization.
mccd_fit_kw (dict) – Parameters for the MCCD class fit() function.
output_dir (str) – Path to the directory to save the fitted model.
catalog_id (int) – Id of the catalog being trained. Default is
1234567
.sex_thresh (float) – Masking threshold, specially for SExtractor catalogs. Default is
-1e5
.use_SNR_weight (bool) – Boolean to decide to use the SNR weight strategy. The columns
SNR_WIN_LIST
should be available on the input fits table.verbose (bool) – Verbose mode. Default is
False
.saving_name (str) – Name of the fitted model file. Default is
fitted_model
.
Notes
Saves the fitted model intro a
.npy
element onoutput_dir
.
- mccd_validation(mccd_model_path, testcat, apply_degradation=True, mccd_debug=False, global_pol_interp=False, sex_thresh=- 100000.0, loc2glob=None)[source]
-
Validate a MCCD model.
- Parameters
mccd_model_path (str) – Path to the saved trained MCCD model.
testcat (fits table) – Opened fits file containing a fits table, ie
astropy.io.fits(path)[1]
. Should contain the testing dataset with the columns: Mandatory:CCD_ID_LIST
,VIGNET_LIST
,GLOB_POSITION_IMG_LIST
,MASK_LIST
. Optional:RA_LIST
,DEC_LIST
.apply_degradation (bool) – Boolean determining if the returned PSFs should be matched to the observed stars by the application of a degradation which consists of an intra-pixel shift, flux matching, etc. Default is
True
.mccd_debug (bool) – Boolean to determine if the validation will be run in debug mode. Default is
False
.global_pol_interp (bool) – Boolean to determine if the global model interpolation is done with a new position matrix (True) or if it is done with the RBF kernel interpolation (False). Default is
False
.sex_thresh (float) – Masking threshold, specially for SExtractor catalogs. Default is
-1e5
.loc2glob (object) – The object that allows to do the coordinate conversion from local to global. It is specific for each instrument’s focal plane geometry. Default is
None
that uses the CFIS MegaCam geometry.
- Returns
star_dict – Dictionary containing the data needed for validation purposes. Keys:
PSF_VIGNET_LIST
,PSF_MOM_LIST
,STAR_MOM_LIST
,GLOB_POSITION_IMG_LIST
,VIGNET_LIST
,MASK_LIST
,CCD_ID_LIST
. Optional keys:RA_LIST
,DEC_LIST
,PSF_GLOB_VIGNET_LIST
,PSF_LOC_VIGNET_LIST
.- Return type
Notes
The parameter
apply_degradation
needs to be set to True if a pixel validation of the model is going to be done.
- mccd_preprocessing(input_folder_path, output_path, min_n_stars=20, file_pattern='sexcat-*-*.fits', separator='-', CCD_id_filter_list=None, outlier_std_max=100.0, save_masks=True, save_name='train_star_selection', save_extension='.fits', verbose=True, loc2glob=None, fits_tb_pos=2)[source]
-
Preprocess input catalog.
- Parameters
input_folder_path (str) – Path to the folder containing the files to preprocess.
output_path (str) – Path to the folder where to save the preprocessed files.
min_n_stars (int) – Minimum number of stars in order to preprocess the CCD. Default is
20
.file_pattern (str) – Input file pattern as a regex expression. Only the files matching the
file_pattern
ininput_folder_path
will be treated. Default issexcat-*-*.fits
. Where the first * corresponds to the catalog_id, the second * corresponds to the CCD id and they are separated by theseparator
.separator (str) – Separator string that separates the catalog id and the CCD id. Default is
'-'
.CCD_id_filter_list (list of int or None) – A list that correspond to the CCDs that should be preprocessed. If it is None, all the CCDs are preprocessed. (Current version: Hardcoded for the MegaCam scenario). Default is
None
.outlier_std_max (float) – Parameter regulating the shape outlier removal. Default is very high so as it is not done at all. A decent number would be
10
. Default is100.
.save_masks (bool) – If masks should be saved in the new file. Default is
True
.save_name (str) – Name to save the preprocessed file. Default is
'train_star_selection'
.save_extension (str) – Extension of the saved file. Default is
.fits
.verbose (bool) – Verbose mode. Default is
True
.loc2glob (object) – The object that allows to do the coordinate conversion from local to global. It is specific for each instrument’s focal plane geometry. Default is
None
that uses the CFIS MegaCam’s geometry.fits_tb_pos (int) – Position in the fits file of the useful table. Default is
2
that is the default for CFIS fits files.
- Returns
mccd_inputs – An instance of
MccdInputs
class used for the input preprocessing.- Return type
class
- class MCCDParamsParser(file_path)[source]
-
Bases:
object
Parse MCCD config file.
Set up a parser for the MCCD parameters.
- Parameters
file_path (str) – Path to the config file.
- Raises
IOError – For non existent configuration file.
- parse_document()[source]
-
Parse config file.
- get_extra_kw(param_name)[source]
-
Get parameter from extra arguments.
- Returns
param_name – Name of the parameter
- Return type
- get_fit_kw()[source]
-
Get fit parameter dictionary.
- Returns
mccd_fit_kw – MCCD fit parameter dictionary.
- Return type
- get_instance_kw()[source]
-
Get instace parameter dictionary.
- Returns
mccd_inst_kw – MCCD instance parameter dictionary.
- Return type
- get_inputs_kw()[source]
-
Get paths parameter dictionary.
- Returns
mccd_inputs_kw – MCCD input parameter dictionary.
- Return type
- get_val_prepro_kw()[source]
-
Get preprocessing validation input dictionary.
- Returns
mccd_val_prepro_kw – MCCD preprocessing validation input dictionary.
- Return type
- class RunMCCD(config_file_path, fits_table_pos=1, verbose=True)[source]
-
Bases:
object
Run the MCCD method.
This class allows to run the MCCD method using the paramters present on the configuration file. Saves the fitted model on the output directory found in the MCCD configuration file.
- Parameters
Notes
Missing: - Method including the validation. - Erase the preprocessed files (?) - Handling different geometries
- parse_config_file()[source]
-
Parse configuration file and recover parameters.
- preprocess_inputs()[source]
-
Preprocess the input data.
- preprocess_val_inputs()[source]
-
Preprocess validation input data.
- fit_models()[source]
-
Build and save the models to the catalgos found.
- validate_models()[source]
-
Validate MCCD models.
- fit_MCCD_models()[source]
-
Fit MCCD models.
- validate_MCCD_models()[source]
-
Validate MCCD models.
- run_MCCD()[source]
-
Run the MCCD routines.
- recover_MCCD_PSFs(mccd_model_path, positions, ccd_id, local_pos=False)[source]
-
Recover MCCD PSFs at required positions.
- Parameters
mccd_model_path (str) – Path pointing to the saved fitted MCCD model to be used.
positions (numpy.ndarray) – Array containing the positions where the PSF should be recovered. The shape of the array should be (n,2) [x,y].
ccd_id (int) – Id of the CCD from where the positions where taken.
local_pos (bool) – If the positions passed are local to the CCD. If False, the positions are considered to be in the same format (coordinate system, units, etc.) as the
obs_pos
fed toMCCD.fit()
. Default isFalse
.
- Returns
rec_PSFs – Array containing the recovered PSFs. Array dimensions: (n_psf, n_im, n_im).
- Return type
- Raises
OSError – For non-existent fitted model.
ValueError – For ccd_id not being an integer.