sp_validation.run_joint_cat

RUN JOINT CAT.

This module implements classes to create, mask, and calibrate joint comprehensive catalogues.

Author:

Martin Kilbinger

class BaseCat

Bases: object

Base_Cat.

Basic catalogue class.

set_params_from_command_line(args)

Set Params From Command Line.

Only use when calling using python from command line. Does not work from ipython or jupyter.

read_config_set_params(fpath)

Read Config Set Params.

Read configuration file and sets class parameters.

Parameters:

fpath (str) – inpput file path

Returns:

configuration

Return type:

dict

read_cat(load_into_memory=False, mode='r', hdu=1, name='data')

Read Cat.

Read input catalogue, either FITS or HDF5.

Parameters:

• load_into_memory (bool, optional) – load data into memory (potentially slow) of True; default is False

• mode (bool, optional) – HDF5 read mode, default is “r”

• hdu (int, optional) – HDU number (for FITS file); default is 1

• name (str, optional) – dataset name, default is ‘data’

Returns:

Catalogue data

Return type:

list

Raises:

IOError – If file extension is not .fits or .hd5

write_hdf5_header(hd5file)

Write HDF5 Header.

Write basic header information to HDF5 file.

Parameters:

hd5file (h5py.File) – input HDF5 file

get_header(path=None)

Get Header.

Return header of hd5 file.

Parameters:

path (str, optional) – input path; if not None (default) use output_path of self._params dict

Returns:

header

Return type:

dict

write_hdf5_file(dat, output_path=None)

Write HDF5 File.

Write HDF5 data to file.

Parameters:

• dat (numpy.ndarray) – input data

• output_path (str, optional) – output file path; when None (default) use self._params[‘output_path’]

close_hd5()

Close HD5.

Close HDF5 file.

class JointCat

Bases: BaseCat

Joint Cat.

Class to create joint weak-lensing catalogues.

set_params_from_command_line(args)

Set Params From Command Line.

Only use when calling using python from command line. Does not work from ipython or jupyter.

params_default()

Params Default.

Set default parameter values.

get_patches()

Get Patches.

Return list of patches according to option parameter value.

Returns:

patches, list of str

Return type:

list

get_n_obj(patches, base_path, input_sub_path)

Get N Obj.

Get number of objects from FITS file headers.

Parameters:

• patches (list) – input patches, type is str

• base_path (str) – input base directory, root dir of patches

• input_sub_path (str) – input file name; input path is base_path/patch/input_sub_path

• Raises – ValueError: if input file canont be read

• Returns –

  list

  HDUs

  list

  number of objects per file

  int

  total number of objects

get_col_info(dat)

Get Col Info.

Return information of input columns.

Parameters:

dat (numpy.ndarray) – input data

Returns:

• list – column names

• list – column formats

• int – number of columns

dtype_out(name, dtype_in)

Set output dtype.

Parameters:

• name (str) – column name

• dtype_in (np.dtype) – input dtype

Returns:

output dtype

Return type:

np.dtype

init_data(n_col, n_obj, ndim, dat)

Init Data.

Initialize empty structured data.

Parameters:

• n_col (int) – number of columns

• n_obj (int) – number of objects (rows)

• ndim (dict) – dimension of input columns

• dat (numpy.ndarray) – example data

Returns:

combined structure data, (n_col x n_obj) array

Return type:

numpy.ndarray

write_hdf5_file(dat_all, patches)

Write HDF5 File.

Write data to HDF5 file.

Parameters:

• dat_all (numpy.ndarray) – input data

• patches (list) – input patches, list of str

write_hdf5_header(hd5file, patches=None)

Write HDF5 Header.

Write header information to HDF5 file.

Parameters:

• hd5file (h5py.File) – input HDF5 file

• patches (list, optional) – input patches, list of str, default is None

merge_catalogues(patches, base_path='.')

Merge Catalogues.

Merge individual patch-based catalogues.

Parameters:

• patches (list) – input patches; list of str

• base_path (str, optional) – input base directory path; default is “.”

run()

Run.

Main processing function.

class ApplyHspMasks

Bases: BaseCat

Apply Hsp Masks.

_labels_struct = {1: 'Faint_star_halos', 2: 'Bright_star_halos', 4: 'Stars', 8: 'Manual', 16: 'u', 32: 'g', 64: 'r', 128: 'i', 256: 'z', 512: 'Tile_RA_DEC_cut', 1024: 'Maximask', 2048: 'z2'}

classmethod get_label_struct(bit)

Get Label Struct.

Return label of bit-coded mask.

Parameters:

bit (int) – input bit

Returns:

label

Return type:

str

classmethod get_mask_col_name(bit)

Get Mask Col Name.

Return column name of mask corresponding to input bit.

Parameters:

bit (int) – input bit

Returns:

column name

Return type:

str

params_default()

Params Default.

Set default parameter values.

check_params()

Check Params.

Check whether parameter values are valid.

Raises:

ValueError – if a parameter value is not valid

update_params()

Update Params.

Update and transform parameter values.

reverse_bit_list()

Reverse Bit List.

Split bit-coded integer into bits.

Parameters:

bit (int) – Bit-coded integer

Returns:

List of bits

Return type:

list

get_paths_bit_masks()

Get Paths Bit Masks.

Return paths of bit-coded mask files.

Returns:

Dictionary with bit as key and path as value.

Return type:

dict

get_mask(path)

Get Mask.

Read from file and return healsparse mask.

Parameters:

path (str) – input path

Returns:

mask

Return type:

hsp.HealSparseMap

apply_mask(ra, dec, hsp_mask, label)

Apply Mask.

Apply mask to input coordinates.

Parameters:

• hsp_mask (hsp.HealSparseMap) – input mask

• ra (numpy.ndarray) – input right ascension

• dec (numpy.ndarray) – input declination

Returns:

mask values

Return type:

numpy.ndarray

get_masks(dat=None)

Get Masks.

Returns per-object masks for all bits.

Parameters:

dat (numpy.ndarray, optional) – input data; if not given (default), data will be read from input file

Returns:

masks

Return type:

dict

run()

Run.

Main processing function.

append_masks(dat, masks)

Append Masks.

Add mask information as columns to data.

Parameters:

• dat (numpy.ndarray) – input data

• masks (dict) – mask information

Returns:

updated data

Return type:

numpy.ndarray

write_hdf5_file(dat, dat_new=None, masks=None)

Write HDF5 File.

Save data to a hdf5 file on disk.

Parameters:

• dat (h5py dataset) – input dataset

• dat_new (h5py dataset, optional) – second dataset; unused if None

• masks (list, optional) – masks, to be added to header information

write_hdf5_header(hd5file)

Write HDF5 Header.

Write header information to HDF5 file.

Parameters:

• hd5file (h5py.File) – input HDF5 file

• patches (list, optional) – input patches, list of str, default is None

class CalibrateCat

Bases: BaseCat

Calibrate Cat.

Class to calibrate joint catalogue.

params_default()

Params Default.

Set default parameter values.

read_cat(load_into_memory=False)

Read Cat.

Read input HDF5 catalogue.

Parameters:

load_into_memory (bool, optional) – load data into memory (potentially slow) of True; default is False

Returns:

• list – Catalogue data

• list_ext – Extended catalogue data if exists in input file

add_params_to_FITS_header(header, cm=None)

run()

Run.

Main processing function.

correlation_matrix(masks, confidence_level=0.9)

confusion_matrix(prediction, observation)

class Mask(col_name, label, kind=None, value=0, dat=None, verbose=False)

Bases: object

Mask.

Class to handle masking of catalogues.

Parameters:

• col_name (str) – name of column to use for mask

• label (str) – mask label

• kind (str) – operation type, allowed are “equal”, “not_equal, “”greater_equal”, “smaller_equal”, “range”

• value (float or list) – value(s) to be used in mask operation

• dat (numpy.ndarray, optional) – input data, default is None; apply mask if given

• verbose (bool, optional) – verbose output if True; default is False

classmethod from_list(masks, label='combined', verbose=False)

apply(dat)

to_bool(hsp_mask)

classmethod print_strings(coln, lab, num, fnum, f_out=None)

print_stats(num_obj, f_out=None)

get_sign(latex=False)

print_condition(f_out, latex=False)

print_summary(f_out)

create_descr()

Create Descr.

Create description of mask for later use in output file header.

Returns:

description

Return type:

str

add_summary_to_FITS_header(header)

print_mask_stats(num_obj, masks, mask_combined)

Print Mask Stats.

Print mask statistics.

Parameters:

num_obj

class ReadCat

Bases: object

params_default()

Params Default.

Set default parameter values.

run()

get_masks_from_config(config, dat, dat_ext, masks_to_apply=None, verbose=False)

Get Masks From Config.

Return mask information from yaml config structure.

Parameters:

• config (dict) – config information

• dat (numpy.ndarray) – input data

• det_ext (numpy.ndarray) – input extended data

• masks_to_apply (list, optional) – masks to apply exclusively; if None (default), use all masks

• verbose (bool, optional) – verbose output if True; default is False

Returns:

• list – list of masks

• dict – list of indices for given mask column name (label)

compute_weights_gatti(cat_gal, g_uncorr, gal_metacal, dat, mask_combined, mask_metacal, num_bins=20, snr_min=10, snr_max=500, size_ratio_min=0.707, size_ratio_max=3)

Compute Weights Gatti.

Compute Gatti et al. (2021) DES-like weights.

compute_PSF_leakage(cat_gal, g_corr_mc, dat, mask_combined, mask_metacal, num_bins=20)

Compute PSF Leakage.

run_joint_comprehensive_cat(*args)

Run Joint Comprehensive Cat.

Run class to create joint comprehensive catalogue from command line.

run_calibrate_comprehensive_cat(*args)

Run Calibrate Comprehensive Cat.

Run class to calibrate joint comprehensive catalogue from command line.

run_apply_hsp_masks(*args)

Run Apply Healsparse Masks.

Run class to apply healsparse masks.