Skip to content

shine.validation.statistics

Statistical analysis for bias measurement.

Computes multiplicative and additive shear bias, coverage tests, and simulation-based calibration (SBC) diagnostics.

statistics

Statistical analysis for bias measurement.

BiasResult(m, m_err, c, c_err, method, component) dataclass

Result of a bias measurement.

Attributes:

Name Type Description
m float

Multiplicative bias.
m_err float

Uncertainty on m.
c float

Additive bias.
c_err float

Uncertainty on c.
method str

Method used to compute bias.
component str

Shear component ("g1" or "g2").

CoverageResult(alpha, observed, n_total, n_covered) dataclass

Result of a coverage test.

Attributes:

Name Type Description
alpha float

Nominal coverage level (e.g., 0.68, 0.95).
observed float

Observed coverage fraction.
n_total int

Total number of realizations tested.
n_covered int

Number of realizations where truth is within credible interval.

SBCResult(ranks, ks_pvalue, param) dataclass

Result of Simulation-Based Calibration.

Attributes:

Name Type Description
ranks NDArray[int64]

Array of rank statistics.
ks_pvalue float

p-value from KS test of rank uniformity.
param str

Parameter name.

compute_bias_single_point(g_true, g_est_mean, g_est_std, component)

Compute multiplicative and additive bias from a single shear point.

For a single g_true value, the bias model is: g_est = (1 + m) * g_true + c

For Level 0 (single point), this simplifies to: m = (g_est / g_true) - 1 (when g_true != 0) c = g_est - (1 + m) * g_true = 0 by construction for single-point

Parameters:

Name Type Description Default
g_true float

True shear value.

 required
g_est_mean float

Estimated (posterior mean) shear value.

 required
g_est_std float

Posterior standard deviation of shear estimate.

 required
component str

Shear component name ("g1" or "g2").

 required

Returns:

Type Description
BiasResult

BiasResult with multiplicative and additive bias.

Raises:

Type Description
ValueError

If g_true is zero (cannot compute multiplicative bias).
Source code in shine/validation/statistics.py 
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
def compute_bias_single_point(
    g_true: float,
    g_est_mean: float,
    g_est_std: float,
    component: str,
) -> BiasResult:
    """Compute multiplicative and additive bias from a single shear point.

    For a single g_true value, the bias model is:
        g_est = (1 + m) * g_true + c

    For Level 0 (single point), this simplifies to:
        m = (g_est / g_true) - 1  (when g_true != 0)
        c = g_est - (1 + m) * g_true = 0 by construction for single-point

    Args:
        g_true: True shear value.
        g_est_mean: Estimated (posterior mean) shear value.
        g_est_std: Posterior standard deviation of shear estimate.
        component: Shear component name ("g1" or "g2").

    Returns:
        BiasResult with multiplicative and additive bias.

    Raises:
        ValueError: If g_true is zero (cannot compute multiplicative bias).
    """
    if g_true == 0.0:
        raise ValueError(
            "Cannot compute multiplicative bias for g_true=0. "
            "Use compute_bias_regression() with multiple shear values."
        )

    m = (g_est_mean / g_true) - 1.0
    m_err = g_est_std / abs(g_true)

    # For single-point, c = g_est - (1+m)*g_true = 0 by construction
    c = 0.0
    c_err = g_est_std

    return BiasResult(
        m=m,
        m_err=m_err,
        c=c,
        c_err=c_err,
        method="single_point",
        component=component,
    )

compute_bias_regression(g_true_values, g_est_means, weights=None)

Compute bias via weighted linear regression over multiple shear values.

Fits the model: g_est = (1 + m) * g_true + c

Parameters:

Name Type Description Default
g_true_values NDArray[float64]

Array of true shear values.

 required
g_est_means NDArray[float64]

Array of estimated (posterior mean) shear values.

 required
weights Optional[NDArray[float64]]

Optional inverse-variance weights for regression.

 None

Returns:

Type Description
BiasResult

BiasResult with regression-fitted m and c.

Raises:

Type Description
NotImplementedError

This function is a stub for Level 1+.
Source code in shine/validation/statistics.py 
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
def compute_bias_regression(
    g_true_values: NDArray[np.float64],
    g_est_means: NDArray[np.float64],
    weights: Optional[NDArray[np.float64]] = None,
) -> BiasResult:
    """Compute bias via weighted linear regression over multiple shear values.

    Fits the model: g_est = (1 + m) * g_true + c

    Args:
        g_true_values: Array of true shear values.
        g_est_means: Array of estimated (posterior mean) shear values.
        weights: Optional inverse-variance weights for regression.

    Returns:
        BiasResult with regression-fitted m and c.

    Raises:
        NotImplementedError: This function is a stub for Level 1+.
    """
    raise NotImplementedError(
        "compute_bias_regression() is planned for Level 1+. "
        "Use compute_bias_single_point() for Level 0."
    )

compute_paired_response(g_est_plus, g_est_minus, g_true)

Compute shear response from paired +g/-g observations.

R_i = (g_est(+g) - g_est(-g)) / (2 * g_true)

Parameters:

Name Type Description Default
g_est_plus NDArray[float64]

Shear estimates from +g realizations.

 required
g_est_minus NDArray[float64]

Shear estimates from -g realizations.

 required
g_true float

Absolute value of true shear.

 required

Returns:

Type Description
NDArray[float64]

Array of response values R_i.

Raises:

Type Description
NotImplementedError

This function is a stub for Level 1+.
Source code in shine/validation/statistics.py 
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
def compute_paired_response(
    g_est_plus: NDArray[np.float64],
    g_est_minus: NDArray[np.float64],
    g_true: float,
) -> NDArray[np.float64]:
    """Compute shear response from paired +g/-g observations.

    R_i = (g_est(+g) - g_est(-g)) / (2 * g_true)

    Args:
        g_est_plus: Shear estimates from +g realizations.
        g_est_minus: Shear estimates from -g realizations.
        g_true: Absolute value of true shear.

    Returns:
        Array of response values R_i.

    Raises:
        NotImplementedError: This function is a stub for Level 1+.
    """
    raise NotImplementedError(
        "compute_paired_response() is planned for Level 1+."
    )

jackknife_bias(g_true_values, g_est_means, weights=None, n_groups=10)

Compute bias with delete-one jackknife error estimation.

Parameters:

Name Type Description Default
g_true_values NDArray[float64]

Array of true shear values.

 required
g_est_means NDArray[float64]

Array of estimated shear means.

 required
weights Optional[NDArray[float64]]

Optional inverse-variance weights.

 None
n_groups int

Number of jackknife groups.

 10

Returns:

Type Description
BiasResult

BiasResult with jackknife uncertainty estimates.

Raises:

Type Description
NotImplementedError

This function is a stub for Level 1+.
Source code in shine/validation/statistics.py 
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
def jackknife_bias(
    g_true_values: NDArray[np.float64],
    g_est_means: NDArray[np.float64],
    weights: Optional[NDArray[np.float64]] = None,
    n_groups: int = 10,
) -> BiasResult:
    """Compute bias with delete-one jackknife error estimation.

    Args:
        g_true_values: Array of true shear values.
        g_est_means: Array of estimated shear means.
        weights: Optional inverse-variance weights.
        n_groups: Number of jackknife groups.

    Returns:
        BiasResult with jackknife uncertainty estimates.

    Raises:
        NotImplementedError: This function is a stub for Level 1+.
    """
    raise NotImplementedError(
        "jackknife_bias() is planned for Level 1+."
    )

compute_coverage(g_true_values, g_est_means, g_est_stds, alpha_levels=None)

Compute credible interval coverage.

Parameters:

Name Type Description Default
g_true_values NDArray[float64]

Array of true shear values.

 required
g_est_means NDArray[float64]

Array of posterior means.

 required
g_est_stds NDArray[float64]

Array of posterior standard deviations.

 required
alpha_levels Optional[List[float]]

Coverage levels to test (default: [0.68, 0.95]).

 None

Returns:

Type Description
List[CoverageResult]

List of CoverageResult for each alpha level.

Raises:

Type Description
NotImplementedError

This function is a stub for Level 1+.
Source code in shine/validation/statistics.py 
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
def compute_coverage(
    g_true_values: NDArray[np.float64],
    g_est_means: NDArray[np.float64],
    g_est_stds: NDArray[np.float64],
    alpha_levels: Optional[List[float]] = None,
) -> List[CoverageResult]:
    """Compute credible interval coverage.

    Args:
        g_true_values: Array of true shear values.
        g_est_means: Array of posterior means.
        g_est_stds: Array of posterior standard deviations.
        alpha_levels: Coverage levels to test (default: [0.68, 0.95]).

    Returns:
        List of CoverageResult for each alpha level.

    Raises:
        NotImplementedError: This function is a stub for Level 1+.
    """
    raise NotImplementedError(
        "compute_coverage() is planned for Level 1+."
    )

compute_sbc_ranks(g_true_values, posterior_samples, param)

Compute Simulation-Based Calibration rank statistics.

For each realization, the rank is the number of posterior samples less than the true value. Under correct calibration, ranks should be uniformly distributed.

Parameters:

Name Type Description Default
g_true_values NDArray[float64]

Array of true parameter values.

 required
posterior_samples NDArray[float64]

Array of posterior samples (n_realizations, n_samples).

 required
param str

Parameter name.

 required

Returns:

Type Description
SBCResult

SBCResult with rank histogram and KS test p-value.

Raises:

Type Description
NotImplementedError

This function is a stub for Level 1+.
Source code in shine/validation/statistics.py 
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
def compute_sbc_ranks(
    g_true_values: NDArray[np.float64],
    posterior_samples: NDArray[np.float64],
    param: str,
) -> SBCResult:
    """Compute Simulation-Based Calibration rank statistics.

    For each realization, the rank is the number of posterior samples
    less than the true value. Under correct calibration, ranks should
    be uniformly distributed.

    Args:
        g_true_values: Array of true parameter values.
        posterior_samples: Array of posterior samples (n_realizations, n_samples).
        param: Parameter name.

    Returns:
        SBCResult with rank histogram and KS test p-value.

    Raises:
        NotImplementedError: This function is a stub for Level 1+.
    """
    raise NotImplementedError(
        "compute_sbc_ranks() is planned for Level 1+."
    )