sp_validation.catalog

CATALOG.

Name:

catalog.py

Description:

Catalogue data layer — the read/write, column-access, and object-matching free functions that operate directly on shape catalogues. These are the low-level primitives; the catalogue construction pipeline (runner classes that orchestrate them) lives in catalog_builders.

Author:

Martin Kilbinger

print_mean_ellipticity(dd, ell_col_name, ell_n_comp, n_tot, stats_file, invalid=-10, verbose=False)[source]

Print Mean Ellipticity.

Output mean ellipticity from a catalogue.

Parameters:

  • dd (dict) – galaxy catalog

  • ell_col_name (array of string) – ellipticity column name(s)

  • ell_n_comp (int) – dimension (= number of components) of ellipticity column. Should be 1 or 2.

  • n_tot (int) – number of total objects

  • stats_file (file handler) – summary statistics output file handler

  • invalid (float, optional, default -10) – flag objects with ellipticity value = invalid

  • verbose (bool, optional, default=False) – verbose output if True

print_some_quantities(dd, stats_file, verbose=False)[source]

Print some quantities.

Output some summary statistics from a catalogue.

Parameters:

  • dd (dict) – galaxy catalog

  • stats_file (file handler) – summary statistics output file handler

  • verbose (bool, optional, default=False) – verbose output if True

Returns:

n_tot – number of objects

Return type:

int

check_matching(d1, d2, keys_1, keys_2, thresh, stats_file, name=None, verbose=False)[source]

Check matching.

Check matching between two catalogues.

Parameters:

  • d1 (dict) – catalogs

  • d2 (dict) – catalogs

  • keys_1 (list) – column keys for d1, d2, corresponding to x, y

  • keys_2 (list) – column keys for d1, d2, corresponding to x, y

  • thres (float) – threshold for matching, in deg

  • stats_file (file handler) – summary statistics output file handler

  • verbose (bool, optional, default=False) – verbose output if True

Returns:

  • ind (array of int) – index list of d2 of objects that were matched to d1

  • mask_area_tiles (array of int) – index list of tiles in footprint

check_invalid(dd, key, val, stats_file, name=None, verbose=False)[source]

Check invalid objects.

Check whether objects have invalid values.

Parameters:

  • dd (dict) – catalog

  • key (list) – key names of (scalar) columns to check

  • val (array of float) – values for above columns indicating invalid entries

  • stats_file (file handler) – summary statistics output file handler

  • name (list, optional, default=None) – for output message. If None, key strings are used

  • verbose (bool, optional, default=False) – verbose output if True

match_subsample(dd, ind, mask, pos_key, g1_key, g2_key, n_ref, stats_file, verbose=False)[source]

Match subsample.

Match subsamples of catalogues.

Parameters:

  • dd (dict) – catalog

  • ind (array of int) – index list of d2 of objects that were matched to d1

  • mask (array of bool) – boolean mask

  • pos_key (list) – key names for position columns

  • g1_key (str) – key names for the two scalar ellipticity components

  • g2_key (str) – key names for the two scalar ellipticity components

  • n_ref (int) – reference number of objects

  • stats_file (file handler) – summary statistics output file handler

  • verbose (bool, optional, default=False) – verbose output if True

Returns:

  • ra, dec (array of float) – positions

  • g (array(2) of float) – ellipticities

match_catalogs_radec(ra1, dec1, ra2, dec2, thresh_deg=0.0002)[source]

Match two catalogues by RA/Dec.

Match each object in catalogue 2 to the nearest in catalogue 1 within a threshold.

Parameters:

  • ra1 (array_like) – coordinates of reference catalogue [deg]

  • dec1 (array_like) – coordinates of reference catalogue [deg]

  • ra2 (array_like) – coordinates of catalogue to match [deg]

  • dec2 (array_like) – coordinates of catalogue to match [deg]

  • thresh_deg (float, optional) – maximum separation [deg], default 0.0002

Returns:

  • idx1 (ndarray of int) – indices into catalogue 1 of matched objects

  • idx2 (ndarray of int) – indices into catalogue 2 of matched objects

match_stars2(ra_gal, dec_gal, ra_star, dec_star, thresh=0.0002)[source]

Add docstring.

…

read_shape_catalog(input_path, w_name='w')[source]

Read Shape Catalog.

Read catalogue with galaxy shapes = shear estimates.

Parameters:

  • input_path (str) – input file path

  • w_name (str, optional) – name of weight column, default is “w”

Returns:

  • ra (array of float) – right ascension in degrees

  • dec (array of float) – declination in degrees

  • g1 (array of float) – uncalibrated shear estimate component 1

  • g2 (array of float) – uncalibrated shear estimate component 2

  • w (array of float) – weight

  • mag (array of float) – magnitude

  • snr (array of float) – signal-to-noise ratio

write_shape_catalog(output_path, ra, dec, w, mag=None, snr=None, g=None, g1_uncal=None, g2_uncal=None, R_g11=None, R_g22=None, R_g12=None, R_g21=None, R=None, R_shear=None, R_select=None, c=None, c_err=None, alpha_leakage=None, sigma_epsilon=None, w_type='iv', add_cols=None, add_cols_format=None, add_header=None)[source]

Write Shape Catalog.

Write catalogue with galaxy shapes = shear estimates.

Parameters:

  • output_path (str) – output file path

  • ra (arrays(ngal) of float) – coordinates in deg

  • dec (arrays(ngal) of float) – coordinates in deg

  • w (np.ndarray) – inverse-variance weights

  • mag (array(ngal) of float, optional) – magnitude, signal-to-noise ratio

  • snr (array(ngal) of float, optional) – signal-to-noise ratio, default is None

  • g (np.ndarray, optional) – calibrated reduced shear estimate components, corrected for multiplicative and additive bias, g = R^-1 g_uncal - c; expected type is arrays(2, ngal) of float; default is None (no calibrated shears written)

  • g1_uncal (np.ndarray, optional) – uncalibrated shear estimates; expected types are arrays(ngal) of float default is None (no uncalibrated shears written)

  • g2_uncal (np.ndarray, optional) – uncalibrated shear estimates; expected types are arrays(ngal) of float default is None (no uncalibrated shears written)

  • R_g11 (np.ndarray, optional) –

    shear response matrix elements per galaxy; expected format is arrays(ngal) of float;

    default is None

  • R_g22 (np.ndarray, optional) –

    shear response matrix elements per galaxy; expected format is arrays(ngal) of float;

    default is None

  • R_g12 (np.ndarray, optional) –

    shear response matrix elements per galaxy; expected format is arrays(ngal) of float;

    default is None

  • R_g21 (np.ndarray, optional) –

    shear response matrix elements per galaxy; expected format is arrays(ngal) of float;

    default is None

  • R (2x2 matrix of float, optional) – Mean full response matrix, default is None

  • R_shear (2x2 matrix of float, optional) – Mean shear response matrix, default is None

  • R_select (2x2 matrix of float, optional) – Global selection response matrix, default is None

  • c (array(2) of float, optional, default is None) – additive shear bias

  • c_err (array(2) of float, optional, default is None) – error of c

  • alpha_leakage (float, optional) – Mean scale-dependent PSF leakage, default is None

  • sigma_epsilon (float, optional) – shape noise, default is None

  • w_type (str, optional) – weight type, allowed are “iv” (default), “des”

  • add_cols (dict, optional, default is None) – data for n additional columns to add

  • add_cols_format (dict, optional) – format for n additional columns to add, default is None, for which float format is used

  • add_header (fits.header.Header, optional) – additional header information; default is None

write_galaxy_cat(output_path, ra, dec, tile_id)[source]

Write Galaxy Cat.

Write catalogue with position information only, no shapes. E.g. random object catalogue.

Parameters:

  • output_path (string) – output file path

  • ra (arrays(ngal) of float) – coordinates in deg

  • dec (arrays(ngal) of float) – coordinates in deg

  • tile_id (array(ngal) of float) – tile ID of objects

write_PSF_cat(output_path, ra, dec, e1, e2)[source]

Write PSF Cat.

Write PSF catalogue to file.

Parameters:

  • output_path (string) – output file path

  • ra (list of float) – coordinates in deg

  • dec (list of float) – coordinates in deg

  • e1 (list of float) – first ellipticity component

  • e2 (list of float) – second ellipticity component

read_param_file(path, verbose=False)[source]

Read Param File. MKDEBUG TODO: Move to cs_util. Also used in sp/create_final_cat.

Return parameter list read from file.

Parameters:

  • path (str) – input file name

  • verbose (bool, optional, default=False) – verbose output if True

Returns:

parameter names

Return type:

list of str

read_hdf5_file(file_path, name, stats_file, check_only=False, param_path=None)[source]

Read HDF5 File.

Read hdf5 file and return contained data.

Parameters:

  • file_path (str) – input file path

  • name (str) – patch name

  • stats_file (file handler) – summary statistics output file handler

  • check_only (bool, optional) – If True only check, not return data

Returns:

data

Return type:

dict

get_maked_col(dat, col, mask)[source]

Get Masked Col.

Retrieve a specific column from the data with a mask.

Parameters:

  • dat (dict) – Input data

  • col (str) – Key of the column to be returned

  • mask (array-like) – Boolean mask used for selection

Returns:

Requested column from the data, filtered by the mask.

Return type:

array-like

get_col(dat, col, m_sel=None, m_flg=None)[source]

Get Col.

Retrieve a specific column from the data with optional selection and flag masks.

Parameters:

  • dat (dict) – Input data

  • col (str) – Key of the column to be returned

  • m_sel (array-like, optional) – Boolean mask used for selection. If specified, m_flg must also be specified; default is None

  • m_flg (array-like, optional) – Boolean mask used as a flag. If specified, m_sel must also be specified. default is None

Returns:

Requested column from the data, optionally filtered by the selection and flag masks.

Return type:

array-like

See also

get_maked_col

More efficient if masks have been combined beforehand.

Raises:

ValueError – If only one of m_sel or m_flg is specified without the other.

get_snr(sh, dat, m_sel, m_flg)[source]

Get SNR.

Return signal-to-noise ratio.

Parameters:

  • sh (str) – shape method identified, e.g. “ngmix”

  • dat (dict) – Input data

  • m_sel (array-like, optional) – Boolean mask used for selection. If specified, m_flg must also be specified; default is None

  • m_flg (array-like, optional) – Boolean mask used as a flag. If specified, m_sel must also be specified. default is None

Returns:

signal-to-noise ratios

Return type:

array-like