API: calibrators

base

BaseCalibrator: the common fit / predict_proba / interpret contract.

UnattainableTargetError

Bases: ValueError

The requested calibrated interval does not intersect the calibrator's output range (or was emptied by buffer_logit). Raised instead of silently clamping — spec §10.

BaseCalibrator

Bases: ABC

Common contract for all probcal calibrators.

Subclasses implement _fit (estimation on validated arrays), _predict (the fitted map on clipped scores), and interpret. Everything else — validation, sklearn-style parameter handling without an sklearn import, the 2-D probability helper — lives here.

ATTRIBUTE	DESCRIPTION
is_monotone_	Whether the fitted map is guaranteed non-decreasing. Class-level default True; non-monotone calibrators (e.g. ENIR) override it. TYPE: bool
fitted_	Set by :meth:fit. TYPE: bool

affine_logit_coeffs_ property

affine_logit_coeffs_: tuple[float, float] | None

Coefficients (a, b) of logit g(s) = a * logit(s) + b, if affine.

None for calibrators that are not affine on the logit scale. Consumed by the attribution adjustment (spec §9).

fit

fit(s: object, y: object, sample_weight: object = None) -> Self

Fit the calibration map on scores and binary outcomes.

PARAMETER	DESCRIPTION
s	Raw scores/probabilities in [0, 1] (clipped to [1e-12, 1 - 1e-12]). Users holding raw logits convert with :func:probcal.expit first. TYPE: array_like
y	Binary outcomes in {0, 1}; both classes must be present. TYPE: array_like
sample_weight	Positive observation weights. TYPE: array_like or None DEFAULT: None

RETURNS	DESCRIPTION
Self	The fitted calibrator.

Source code in src/probcal/base.py

def fit(self, s: object, y: object, sample_weight: object = None) -> Self:
    """Fit the calibration map on scores and binary outcomes.

    Parameters
    ----------
    s : array_like
        Raw scores/probabilities in ``[0, 1]`` (clipped to
        ``[1e-12, 1 - 1e-12]``). Users holding raw logits convert with
        :func:`probcal.expit` first.
    y : array_like
        Binary outcomes in ``{0, 1}``; both classes must be present.
    sample_weight : array_like or None
        Positive observation weights.

    Returns
    -------
    Self
        The fitted calibrator.
    """
    s_arr = validate_scores(s)
    y_arr = validate_binary_y(y)
    if s_arr.shape[0] != y_arr.shape[0]:
        raise ValueError(
            f"s and y must have equal length, got {s_arr.shape[0]} and {y_arr.shape[0]}"
        )
    w_arr = validate_weights(sample_weight, s_arr.shape[0])
    self._fit(s_arr, y_arr, w_arr)
    self.fitted_ = True
    return self

predict_proba

predict_proba(s: object) -> ndarray

Calibrated probabilities P(y = 1) for new scores.

PARAMETER	DESCRIPTION
s	Raw scores/probabilities in [0, 1]. TYPE: array_like

RETURNS	DESCRIPTION
numpy.ndarray of shape (n,)	Calibrated probabilities.

Source code in src/probcal/base.py

def predict_proba(self, s: object) -> np.ndarray:
    """Calibrated probabilities ``P(y = 1)`` for new scores.

    Parameters
    ----------
    s : array_like
        Raw scores/probabilities in ``[0, 1]``.

    Returns
    -------
    numpy.ndarray of shape (n,)
        Calibrated probabilities.
    """
    self._check_fitted()
    return self._predict(validate_scores(s))

predict_proba_2d

predict_proba_2d(s: object) -> ndarray

Sklearn-style (n, 2) probability matrix [P(y=0), P(y=1)].

Source code in src/probcal/base.py

def predict_proba_2d(self, s: object) -> np.ndarray:
    """Sklearn-style ``(n, 2)`` probability matrix ``[P(y=0), P(y=1)]``."""
    p = self.predict_proba(s)
    return np.column_stack([1.0 - p, p])

interpret abstractmethod

interpret() -> Interpretation

Fitted parameters with a plain-language, domain-aware reading.

Source code in src/probcal/base.py

@abstractmethod
def interpret(self) -> Interpretation:
    """Fitted parameters with a plain-language, domain-aware reading."""

interval_inverse

interval_inverse(lo: float, hi: float, *, space: str = 'probability', buffer_logit: float = 0.0) -> tuple[float, float]

Generalized-inverse preimage (raw_lo, raw_hi) of a calibrated interval.

For a non-decreasing fitted map g: raw_lo = inf{s : g(s) >= lo} and raw_hi = sup{s : g(s) <= hi}.

PARAMETER	DESCRIPTION
lo	Calibrated-probability bounds; lo=0/hi=1 map to the full raw range (−inf/+inf on the logit scale). TYPE: float
hi	Calibrated-probability bounds; lo=0/hi=1 map to the full raw range (−inf/+inf on the logit scale). TYPE: float
space	Scale of the returned raw bounds. "logit" is what a SIGMOID-link raw-margin consumer (e.g. a counterfactual engine's Target.raw) expects. TYPE: (probability, logit) DEFAULT: "probability"
buffer_logit	Shrink the calibrated interval by this margin in logit space before inverting — robustness against future recalibration drift (a central-tendency update of magnitude <= buffer cannot invalidate the result). TYPE: float DEFAULT: 0.0

RAISES	DESCRIPTION
UnattainableTargetError	If the (buffered) interval does not intersect the output range — never silently clamped.
NotImplementedError	For non-monotone calibrators (is_monotone_ = False), whose preimage may be a union of intervals.

Source code in src/probcal/base.py

def interval_inverse(
    self,
    lo: float,
    hi: float,
    *,
    space: str = "probability",
    buffer_logit: float = 0.0,
) -> tuple[float, float]:
    """Generalized-inverse preimage ``(raw_lo, raw_hi)`` of a calibrated interval.

    For a non-decreasing fitted map ``g``:
    ``raw_lo = inf{s : g(s) >= lo}`` and ``raw_hi = sup{s : g(s) <= hi}``.

    Parameters
    ----------
    lo, hi : float
        Calibrated-probability bounds; ``lo=0``/``hi=1`` map to the full
        raw range (−inf/+inf on the logit scale).
    space : {"probability", "logit"}
        Scale of the returned raw bounds. ``"logit"`` is what a
        SIGMOID-link raw-margin consumer (e.g. a counterfactual engine's
        ``Target.raw``) expects.
    buffer_logit : float
        Shrink the calibrated interval by this margin in logit space
        *before* inverting — robustness against future recalibration
        drift (a central-tendency update of magnitude <= buffer cannot
        invalidate the result).

    Raises
    ------
    UnattainableTargetError
        If the (buffered) interval does not intersect the output range —
        never silently clamped.
    NotImplementedError
        For non-monotone calibrators (``is_monotone_ = False``), whose
        preimage may be a union of intervals.
    """
    self._check_fitted()
    if not self.is_monotone_:
        raise NotImplementedError(
            f"{type(self).__name__} is not monotone (is_monotone_=False); its preimage "
            "may be a union of intervals. Use a monotone calibrator for thresholding "
            "and recourse."
        )
    if not 0.0 <= lo <= hi <= 1.0:
        raise ValueError(f"need 0 <= lo <= hi <= 1, got lo={lo}, hi={hi}")
    if space not in ("probability", "logit"):
        raise ValueError(f"space must be 'probability' or 'logit', got {space!r}")
    lo_b, hi_b = float(lo), float(hi)
    if buffer_logit > 0.0:
        if lo > 0.0:
            lo_b = float(expit(np.array([logit(np.array([lo]))[0] + buffer_logit]))[0])
        if hi < 1.0:
            hi_b = float(expit(np.array([logit(np.array([hi]))[0] - buffer_logit]))[0])
        if lo_b > hi_b:
            raise UnattainableTargetError(
                f"buffer_logit={buffer_logit} empties the calibrated interval " f"[{lo}, {hi}]"
            )
    gmin, gmax = self._output_range()
    if lo_b > gmax or hi_b < gmin:
        raise UnattainableTargetError(
            f"calibrated target [{lo_b:.6g}, {hi_b:.6g}] does not intersect the "
            f"calibrator's output range [{gmin:.6g}, {gmax:.6g}]"
        )
    raw_lo = 0.0 if lo_b <= gmin else float(self._inverse_left(lo_b))
    raw_hi = 1.0 if hi_b >= gmax else float(self._inverse_right(hi_b))
    if space == "logit":
        lo_out = -np.inf if raw_lo <= 0.0 else float(logit(np.array([raw_lo]))[0])
        hi_out = np.inf if raw_hi >= 1.0 else float(logit(np.array([raw_hi]))[0])
        return lo_out, hi_out
    return raw_lo, raw_hi

get_params

get_params(deep: bool = True) -> dict[str, object]

Constructor parameters as a dict (manual sklearn-compatible clone info).

Source code in src/probcal/base.py

def get_params(self, deep: bool = True) -> dict[str, object]:
    """Constructor parameters as a dict (manual sklearn-compatible clone info)."""
    sig = inspect.signature(type(self).__init__)
    return {
        name: getattr(self, name)
        for name in sig.parameters
        if name not in ("self", "args", "kwargs")
    }

set_params

set_params(**params: object) -> Self

Set constructor parameters; unknown names raise ValueError.

Source code in src/probcal/base.py

def set_params(self, **params: object) -> Self:
    """Set constructor parameters; unknown names raise ``ValueError``."""
    valid = self.get_params()
    for key, value in params.items():
        if key not in valid:
            raise ValueError(
                f"unknown parameter {key!r} for {type(self).__name__}; "
                f"valid: {sorted(valid)}"
            )
        setattr(self, key, value)
    return self

parametric

Parametric calibrators: Platt, temperature, and beta calibration.

Theory, derivations, and parameter interpretation: docs/concepts/methods-parametric.md.

References

Platt (1999); Lin, Lin & Weng (2007); Guo et al. (2017); Kull, Silva Filho & Flach (2017, AISTATS and EJS) — full records in the documentation.

PlattCalibrator

Bases: BaseCalibrator

Logistic recalibration on the logit scale (Platt scaling).

Fits logit g(s) = a * logit(s) + b by IRLS with Lin–Lin–Weng smoothed targets (N+ + 1)/(N+ + 2) and 1/(N- + 2) for stability on small samples. The identity map is (a, b) = (1, 0).

ATTRIBUTE	DESCRIPTION
a_	Fitted slope — spread correction: a < 1 shrinks overconfident scores toward the base rate, a > 1 sharpens underconfident ones. TYPE: float
b_	Fitted intercept — calibration-in-the-large shift in log-odds. TYPE: float

References

Platt (1999); Lin, Lin & Weng (2007). The logistic family fitted on raw SVM outputs (Platt's original setting) does not contain the identity; on logits it does — see the parametric-methods chapter.

affine_logit_coeffs_ property

affine_logit_coeffs_: tuple[float, float] | None

(a, b): Platt scaling is affine on the logit scale.

interpret

interpret() -> Interpretation

Read the fitted slope and intercept against the identity (1, 0).

Source code in src/probcal/parametric.py

def interpret(self) -> Interpretation:
    """Read the fitted slope and intercept against the identity ``(1, 0)``."""
    self._check_fitted()
    if self.a_ < 1.0:
        slope_msg = (
            f"slope a = {self.a_:.3f} < 1: scores were overconfident (too spread out); "
            "predictions are shrunk toward the base rate"
        )
    else:
        slope_msg = (
            f"slope a = {self.a_:.3f} >= 1: scores were underconfident (too flat); "
            "predictions are sharpened"
        )
    int_msg = (
        f"intercept b = {self.b_:.3f}: base-rate (calibration-in-the-large) shift of "
        f"{self.b_:+.3f} log-odds, odds factor {np.exp(self.b_):.3f}"
    )
    return Interpretation(
        method=type(self).__name__,
        param_names=("a", "b"),
        param_values=(self.a_, self.b_),
        messages=(slope_msg, int_msg, "identity map corresponds to (a, b) = (1, 0)"),
    )

TemperatureCalibrator

Bases: BaseCalibrator

Temperature scaling: g(s) = sigma(logit(s) / T).

T minimizes the calibration-set negative log-likelihood via a safeguarded 1-D Newton iteration (bisection fallback) on u = 1/T.

ATTRIBUTE	DESCRIPTION
T_	Fitted temperature. T > 1: the model was overconfident and is softened; T < 1: underconfident and sharpened. Temperature cannot fix base-rate error — s = 0.5 is a fixed point for every T; use Platt scaling or LogitOffset for level shifts. TYPE: float

References

Guo, Pleiss, Sun & Weinberger (2017).

affine_logit_coeffs_ property

affine_logit_coeffs_: tuple[float, float] | None

(1/T, 0): temperature scaling is affine on the logit scale.

interpret

interpret() -> Interpretation

Read the fitted temperature against the identity T = 1.

Source code in src/probcal/parametric.py

def interpret(self) -> Interpretation:
    """Read the fitted temperature against the identity ``T = 1``."""
    self._check_fitted()
    if self.T_ > 1.0:
        msg = (
            f"T = {self.T_:.3f} > 1: the model was overconfident; logits are divided "
            "by T (softening toward 1/2)"
        )
    else:
        msg = (
            f"T = {self.T_:.3f} <= 1: the model was underconfident; logits are divided "
            "by T (sharpening away from 1/2)"
        )
    return Interpretation(
        method=type(self).__name__,
        param_names=("T",),
        param_values=(self.T_,),
        messages=(
            msg,
            "temperature cannot fix base-rate error: s = 0.5 maps to 0.5 for every T "
            "(use PlattCalibrator or LogitOffset for level shifts)",
        ),
    )

BetaCalibrator

BetaCalibrator(variant: str = 'abm')

Bases: BaseCalibrator

Beta calibration: logit g(s) = a·ln s − b·ln(1 − s) + c.

Variants (spec §6; DECISIONS entry 27): "abm" fits (a, b, c); "ab" ties a = b (equivalent to Platt scaling on logits); "a" additionally fixes c = 0 (a single-parameter map, the temperature family in a different parameterization). The monotonicity constraint a, b >= 0 is enforced by the betacal refit strategy: a negative exponent drops its feature and refits.

ATTRIBUTE	DESCRIPTION
a_	Sensitivity near s -> 0 — governs the low-probability tail (critical for low-PD credit portfolios). TYPE: float
b_	Sensitivity near s -> 1. TYPE: float
c_	Base-rate shift in log-odds. TYPE: float
constraint_active_	Whether the a, b >= 0 constraint forced a refit. TYPE: bool

References

Kull, Silva Filho & Flach (2017), AISTATS 54 and EJS 11(2). The identity is (a, b, c) = (1, 1, 0): beta calibration cannot un-calibrate an already calibrated model. a != b captures asymmetric tail distortion; temperature is the special case a = b = 1/T, c = 0.

Source code in src/probcal/parametric.py

def __init__(self, variant: str = "abm") -> None:
    self.variant = variant

affine_logit_coeffs_ property

affine_logit_coeffs_: tuple[float, float] | None

(a, c) for the tied variants; None for "abm".

With a = b the map reduces to logit g = a * logit(s) + c, which is affine on the logit scale; the full three-parameter map is not (see the shap-calibration chapter).

interpret

interpret() -> Interpretation

Read the fitted exponents and intercept against the identity (1, 1, 0).

Source code in src/probcal/parametric.py

def interpret(self) -> Interpretation:
    """Read the fitted exponents and intercept against the identity (1, 1, 0)."""
    self._check_fitted()
    messages = [
        (
            f"a = {self.a_:.3f}: sensitivity near s -> 0; a < 1 raises the smallest "
            "probabilities (model was overconfident in the low tail), a > 1 deepens them"
        ),
        (
            f"b = {self.b_:.3f}: sensitivity near s -> 1; the mirrored reading for the "
            "high tail"
        ),
        (
            f"c = {self.c_:.3f}: base-rate shift of {self.c_:+.3f} log-odds, odds factor "
            f"{np.exp(self.c_):.3f}"
        ),
        "identity map corresponds to (a, b, c) = (1, 1, 0)",
    ]
    if abs(self.a_ - self.b_) > 0.1:
        messages.append(
            f"a != b (gap {self.a_ - self.b_:+.3f}): asymmetric tail distortion that no "
            "symmetric (Platt/temperature) map could express"
        )
    if self.constraint_active_:
        messages.append(
            "monotonicity constraint a, b >= 0 was active: a negative exponent was "
            "dropped and the model refitted (betacal strategy)"
        )
    return Interpretation(
        method=type(self).__name__,
        param_names=("a", "b", "c"),
        param_values=(self.a_, self.b_, self.c_),
        messages=tuple(messages),
    )

isotonic

Isotonic calibrators: PAVA-based isotonic and centered isotonic regression (CIR).

Theory and worked example: docs/concepts/methods-nonparametric.md.

References

Barlow, Bartholomew, Bremner & Brunk (1972); Zadrozny & Elkan (2002); Oron & Flournoy (2017) — full records in the documentation.

IsotonicCalibrator

IsotonicCalibrator(interpolation: str = 'none')

Bases: BaseCalibrator

Isotonic calibration: the PAVA step function.

Fits the least-squares non-decreasing map of outcomes on scores. The fitted map is a right-continuous step function with one level per pooled block; scores outside the calibration range clamp to the first/last level. interpolation="linear" instead joins block midpoints, removing the discontinuities.

ATTRIBUTE	DESCRIPTION
n_blocks_	Number of pooled blocks — the effective complexity estimated from the data. TYPE: int
block_mean_	Event rate of each pooled block (the step levels). TYPE: ndarray
block_first_s_, block_last_s_	Score range covered by each block. TYPE: ndarray
block_center_s_	Weight-centered score coordinate of each block (used by CIR). TYPE: ndarray

References

Barlow et al. (1972) for PAVA; Zadrozny & Elkan (2002) for its use in classifier calibration.

Source code in src/probcal/isotonic.py

def __init__(self, interpolation: str = "none") -> None:
    self.interpolation = interpolation

interpret

interpret() -> Interpretation

Read the block structure as effective complexity and local event rates.

Source code in src/probcal/isotonic.py

def interpret(self) -> Interpretation:
    """Read the block structure as effective complexity and local event rates."""
    self._check_fitted()
    flats = int(np.sum(np.isclose(np.diff(self.block_mean_), 0.0)))
    messages = [
        (
            f"{self.n_blocks_} pooled blocks: each step level is the empirical event "
            "rate of a score region the data could not subdivide further"
        ),
        (
            f"block count = effective complexity actually estimated from the data "
            f"(range of levels: {self.block_mean_[0]:.4g} to {self.block_mean_[-1]:.4g})"
        ),
    ]
    if flats:
        messages.append(
            f"{flats} adjacent block pairs share a level: expect tied predictions there"
        )
    messages.append(
        "output range is limited to the span of block levels; targets outside it are "
        "unattainable (relevant for interval_inverse)"
    )
    return Interpretation(
        method=type(self).__name__,
        param_names=("n_blocks",),
        param_values=(float(self.n_blocks_),),
        messages=tuple(messages),
    )

CenteredIsotonicCalibrator

CenteredIsotonicCalibrator()

Bases: IsotonicCalibrator

Centered isotonic regression (CIR): strictly increasing where data permit.

Post-processes the PAVA solution by collapsing each block to its weight-centered score coordinate and interpolating linearly through the points (Oron & Flournoy, 2017). Removes the step function's tied predictions — preferred when downstream ranking must be strict.

References

Oron & Flournoy (2017).

Source code in src/probcal/isotonic.py

def __init__(self) -> None:
    super().__init__(interpolation="none")

interpret

interpret() -> Interpretation

Isotonic reading plus the strictness property CIR adds.

Source code in src/probcal/isotonic.py

def interpret(self) -> Interpretation:
    """Isotonic reading plus the strictness property CIR adds."""
    base = super().interpret()
    return Interpretation(
        method=type(self).__name__,
        param_names=base.param_names,
        param_values=base.param_values,
        messages=base.messages
        + (
            "centered isotonic interpolation is strictly increasing wherever block "
            "levels differ: distinct scores keep distinct predictions (no ties)",
        ),
    )

binning

Binning calibrators: histogram binning and scaling-binning.

Theory: docs/concepts/methods-nonparametric.md.

References

Zadrozny & Elkan (2001); Kumar, Liang & Ma (2019) — full records in the documentation.

HistogramBinningCalibrator

HistogramBinningCalibrator(n_bins: int = 10, strategy: str = 'mass', shrinkage: str | None = 'jeffreys')

Bases: BaseCalibrator

Histogram binning: per-bin event rates with optional Jeffreys shrinkage.

PARAMETER	DESCRIPTION
n_bins	Requested number of bins B — the bias–variance dial. TYPE: int DEFAULT: 10
strategy	"mass" (equal-count, recommended default: lower estimator bias, no empty bins) or "width" (equal-width over [0, 1]). TYPE: (mass, width) DEFAULT: "mass"
shrinkage	"jeffreys" replaces the raw rate k/n with (k + 1/2)/(n + 1) — the posterior mean under the Beta(1/2, 1/2) prior — keeping small bins away from 0 and 1. TYPE: (jeffreys, None) DEFAULT: "jeffreys"

ATTRIBUTE	DESCRIPTION
bin_rate_	Calibrated value per (non-degenerate) bin. TYPE: ndarray
is_monotone_	Computed after fitting: binning does not assume monotonicity, so the flag reports whether the fitted rates happen to be non-decreasing. TYPE: bool

References

Zadrozny & Elkan (2001).

Source code in src/probcal/binning.py

def __init__(
    self, n_bins: int = 10, strategy: str = "mass", shrinkage: str | None = "jeffreys"
) -> None:
    self.n_bins = n_bins
    self.strategy = strategy
    self.shrinkage = shrinkage

interpret

interpret() -> Interpretation

Read bin rates as local event frequencies and B as the complexity dial.

Source code in src/probcal/binning.py

def interpret(self) -> Interpretation:
    """Read bin rates as local event frequencies and B as the complexity dial."""
    self._check_fitted()
    messages = [
        (
            f"{len(self.bin_rate_)} bins ({self.strategy} strategy): each calibrated "
            "value is the (shrunken) empirical event rate of its score bin"
        ),
        "B controls bias-variance: few bins are stable but coarse, many are sharp but noisy",
    ]
    if self.shrinkage == "jeffreys":
        messages.append("Jeffreys shrinkage (k+1/2)/(n+1) keeps sparse bins away from 0 and 1")
    if not self.is_monotone_:
        messages.append(
            "fitted bin rates are not monotone: binning does not enforce ranking "
            "preservation — read inversions as noise, not signal"
        )
    return Interpretation(
        method=type(self).__name__,
        param_names=("n_bins",),
        param_values=(float(len(self.bin_rate_)),),
        messages=tuple(messages),
    )

ScalingBinningCalibrator

ScalingBinningCalibrator(n_bins: int = 10)

Bases: BaseCalibrator

Scaling-binning (Kumar–Liang–Ma): Platt stage, then bin the fitted values.

Fits Platt scaling first, then forms equal-mass bins of the fitted function values and outputs the mean of the fitted values within each bin. Achieves measurable calibration error with O(1/eps^2 + B) samples versus O(B/eps^2) for histogram binning.

References

Kumar, Liang & Ma (2019).

Source code in src/probcal/binning.py

def __init__(self, n_bins: int = 10) -> None:
    self.n_bins = n_bins

interpret

interpret() -> Interpretation

Two-stage reading: Platt map, then the error-measurability discretization.

Source code in src/probcal/binning.py

def interpret(self) -> Interpretation:
    """Two-stage reading: Platt map, then the error-measurability discretization."""
    self._check_fitted()
    platt_interp = self.platt_.interpret()
    return Interpretation(
        method=type(self).__name__,
        param_names=platt_interp.param_names + ("n_bins",),
        param_values=platt_interp.param_values + (float(len(self.bin_value_)),),
        messages=platt_interp.messages
        + (
            (
                f"binning stage: {len(self.bin_value_)} equal-mass bins of the fitted "
                "Platt values; outputs are bin means, which makes the residual "
                "calibration error estimable with O(1/eps^2 + B) samples "
                "(vs O(B/eps^2) for histogram binning)"
            ),
        ),
    )

bayesian

Bayesian-ensemble calibrators: BBQ and ENIR.

Theory: docs/concepts/methods-nonparametric.md.

References

Naeini, Cooper & Hauskrecht (2015); Naeini & Cooper (2016); Tibshirani, Hoefling & Tibshirani (2011) — full records in the documentation.

BBQCalibrator

BBQCalibrator(min_bins: int | None = None, max_bins: int | None = None)

Bases: BaseCalibrator

Bayesian Binning into Quantiles: model averaging over equal-mass binnings.

Considers equal-mass binning models over a range of bin counts, scores each by its Beta-Binomial log marginal likelihood under a per-bin Jeffreys Beta(1/2, 1/2) prior, and predicts with the posterior-weighted average of the models' (posterior-mean) bin rates.

PARAMETER	DESCRIPTION
min_bins	Range of candidate bin counts; defaults to [2, ceil(sqrt(n))] capped at 50 (DECISIONS entry). TYPE: int or None DEFAULT: None
max_bins	Range of candidate bin counts; defaults to [2, ceil(sqrt(n))] capped at 50 (DECISIONS entry). TYPE: int or None DEFAULT: None

ATTRIBUTE	DESCRIPTION
bins_grid_	Candidate bin counts. TYPE: ndarray
weights_	Posterior weights over the candidates (sum to 1). TYPE: ndarray

References

Naeini, Cooper & Hauskrecht (2015).

Source code in src/probcal/bayesian.py

def __init__(self, min_bins: int | None = None, max_bins: int | None = None) -> None:
    self.min_bins = min_bins
    self.max_bins = max_bins

interpret

interpret() -> Interpretation

Read the posterior weights as uncertainty about the data's resolution.

Source code in src/probcal/bayesian.py

def interpret(self) -> Interpretation:
    """Read the posterior weights as uncertainty about the data's resolution."""
    self._check_fitted()
    top = np.argsort(self.weights_)[::-1][:3]
    top_txt = ", ".join(
        f"B={int(self.bins_grid_[i])} (weight {self.weights_[i]:.3f})" for i in top
    )
    return Interpretation(
        method=type(self).__name__,
        param_names=("n_models",),
        param_values=(float(len(self.bins_grid_)),),
        messages=(
            f"top-3 binning models by posterior weight: {top_txt}",
            "concentrated weight = the data speak clearly about their own resolution; "
            "diffuse weight = the averaging is doing real work",
        ),
    )

ENIRCalibrator

Bases: BaseCalibrator

Ensemble of near-isotonic regressions (ENIR).

Computes the full nearly-isotonic solution path (modified PAVA of Tibshirani, Hoefling & Tibshirani, 2011) from the raw data (lambda = 0) to the fully isotonic fit, then averages the breakpoint solutions with BIC weights. The combined map may be non-monotone: is_monotone_ is False and consumers requiring order preservation should prefer a monotone calibrator.

ATTRIBUTE	DESCRIPTION
path_lambdas_	Breakpoints of the penalty parameter, starting at 0. TYPE: ndarray
path_solutions_	Fitted values on the tie-aggregated score grid at each breakpoint. TYPE: numpy.ndarray of shape (T, m)
weights_	BIC weights over the path solutions (sum to 1). TYPE: ndarray

References

Naeini & Cooper (2016); Tibshirani, Hoefling & Tibshirani (2011).

interpret

interpret() -> Interpretation

Read the path length and BIC weights; warn about non-monotonicity.

Source code in src/probcal/bayesian.py

def interpret(self) -> Interpretation:
    """Read the path length and BIC weights; warn about non-monotonicity."""
    self._check_fitted()
    top = np.argsort(self.weights_)[::-1][:3]
    top_txt = ", ".join(
        f"lambda={self.path_lambdas_[i]:.4g} (weight {self.weights_[i]:.3f})" for i in top
    )
    return Interpretation(
        method=type(self).__name__,
        param_names=("n_path_solutions",),
        param_values=(float(len(self.path_lambdas_)),),
        messages=(
            f"top-3 path solutions by BIC weight: {top_txt}",
            "lambda trades monotonicity strictness against fit; BIC weights are model "
            "plausibility along the path",
            "the ensemble output may be non-monotone (is_monotone_ = False): consumers "
            "requiring order preservation should use a monotone calibrator",
        ),
    )

vennabers

Venn–Abers calibrators: inductive (IVAP) and cross (CVAP).

Theory, validity guarantee scope, and the scalarization caveat: docs/concepts/methods-distribution-free.md. The guarantee attaches to the interval returned by :meth:VennAbersCalibrator.predict_interval; the scalar from predict_proba is the log-loss-minimax merger and is not itself covered by the validity theorem.

References

Vovk & Petej (2014) — full record in the documentation.

VennAbersCalibrator

Bases: BaseCalibrator

Inductive Venn–Abers predictor (IVAP).

For a query score, two isotonic fits on the calibration set augmented with the query labeled 0 (resp. 1) yield the interval [p0, p1]; predict_proba scalarizes it as p1 / (1 - p0 + p1).

Batch prediction deduplicates query scores and runs two PAVA fits per unique score (DECISIONS entry: the O((n+m)log(n+m)) precomputation of Vovk & Petej is a planned optimization, not yet implemented).

predict_interval

predict_interval(s: object) -> ndarray

Venn–Abers intervals [p0, p1] for new scores.

PARAMETER	DESCRIPTION
s	Raw scores/probabilities in [0, 1]. TYPE: array_like

RETURNS	DESCRIPTION
numpy.ndarray of shape (n, 2)	Columns p0 (lower) and p1 (upper). The distribution-free validity guarantee attaches to this pair, not to the scalarized predict_proba output.

Source code in src/probcal/vennabers.py

def predict_interval(self, s: object) -> np.ndarray:
    """Venn–Abers intervals ``[p0, p1]`` for new scores.

    Parameters
    ----------
    s : array_like
        Raw scores/probabilities in ``[0, 1]``.

    Returns
    -------
    numpy.ndarray of shape (n, 2)
        Columns ``p0`` (lower) and ``p1`` (upper). The distribution-free
        validity guarantee attaches to this pair, not to the scalarized
        ``predict_proba`` output.
    """
    self._check_fitted()
    from ._validation import validate_scores

    arr = validate_scores(s)
    uniq, inverse = np.unique(arr, return_inverse=True)
    pairs = np.array([self._pair_at(float(x)) for x in uniq])
    return pairs[inverse]

interpret

interpret() -> Interpretation

Report interval widths over the calibration scores — where to trust the map.

Source code in src/probcal/vennabers.py

def interpret(self) -> Interpretation:
    """Report interval widths over the calibration scores — where to trust the map."""
    self._check_fitted()
    if self._widths_cache is None:
        intervals = self.predict_interval(self._s)
        widths = intervals[:, 1] - intervals[:, 0]
        self._widths_cache = (float(widths.mean()), float(widths.max()))
    mean_w, max_w = self._widths_cache
    return Interpretation(
        method=type(self).__name__,
        param_names=("mean_width", "max_width"),
        param_values=(mean_w, max_w),
        messages=(
            f"mean Venn–Abers interval width {mean_w:.4f}, maximum {max_w:.4f} over the "
            "calibration scores: width is per-score calibration uncertainty",
            "the validity guarantee holds for the interval [p0, p1] from "
            "predict_interval(); the scalar from predict_proba() is the log-loss-minimax "
            "merger p1/(1-p0+p1) and is not itself covered by the guarantee",
        ),
    )

CrossVennAbersCalibrator

CrossVennAbersCalibrator(cv: int = 5, random_state: int = 42)

Bases: BaseCalibrator

Cross Venn–Abers predictor (CVAP): fold-wise IVAPs, geometric-mean merge.

Splits the calibration data into cv stratified folds; each fold's IVAP is fitted on the remaining folds. The scalar output merges the fold-wise pairs by the log-loss rule of Vovk & Petej: GM(p1) / (GM(1 - p0) + GM(p1)). predict_interval returns the conservative envelope [min_k p0_k, max_k p1_k] (DECISIONS entry — the paper defines only the scalar merge).

Source code in src/probcal/vennabers.py

def __init__(self, cv: int = 5, random_state: int = 42) -> None:
    self.cv = cv
    self.random_state = random_state

predict_interval

predict_interval(s: object) -> ndarray

Conservative fold envelope [min_k p0_k, max_k p1_k] (see class docs).

Source code in src/probcal/vennabers.py

def predict_interval(self, s: object) -> np.ndarray:
    """Conservative fold envelope ``[min_k p0_k, max_k p1_k]`` (see class docs)."""
    self._check_fitted()
    from ._validation import validate_scores

    arr = validate_scores(s)
    p0, p1 = self._fold_pairs(arr)
    return np.column_stack([p0.min(axis=0), p1.max(axis=0)])

interpret

interpret() -> Interpretation

Report fold count and envelope widths over a probe grid.

Source code in src/probcal/vennabers.py

def interpret(self) -> Interpretation:
    """Report fold count and envelope widths over a probe grid."""
    self._check_fitted()
    probe = np.linspace(0.01, 0.99, 99)
    env = self.predict_interval(probe)
    widths = env[:, 1] - env[:, 0]
    return Interpretation(
        method=type(self).__name__,
        param_names=("cv", "mean_envelope_width"),
        param_values=(float(self.cv), float(widths.mean())),
        messages=(
            f"{self.cv} stratified folds, one IVAP per fold; scalar output is the "
            "geometric-mean merge GM(p1)/(GM(1-p0)+GM(p1))",
            "predict_interval() returns the conservative fold envelope "
            "[min p0, max p1]; per-fold IVAP intervals carry the validity guarantee",
        ),
    )

spline

Spline calibrator: penalized natural cubic splines on the logit scale.

Theory: docs/concepts/methods-nonparametric.md.

References

Lucena (2018); Hastie, Tibshirani & Friedman (2009), §5.2.1 — full records in the documentation.

SplineCalibrator

SplineCalibrator(n_knots: int | None = None, lambdas: object = None, cv: int = 5, random_state: int = 42)

Bases: BaseCalibrator

Natural cubic spline calibration on the logit scale.

Models logit g(s) = sum_k theta_k N_k(logit s) with the natural cubic basis (linear beyond the boundary knots), fitted by penalized IRLS with a second-difference roughness penalty. The penalty weight is chosen by K-fold cross-validated log loss within the calibration set.

PARAMETER	DESCRIPTION
n_knots	Number of knots (placed at equally spaced quantiles of the logit scores); defaults to clip(ceil(n^(1/3)), 4, 12) (DECISIONS entry). TYPE: int or None DEFAULT: None
lambdas	Candidate penalty weights; defaults to logspace(-4, 4, 17). TYPE: array_like or None DEFAULT: None
cv	Inner fold count for the lambda search. TYPE: int DEFAULT: 5
random_state	Seed for the stratified fold assignment. TYPE: int DEFAULT: 42

ATTRIBUTE	DESCRIPTION
lambda_	Selected penalty weight. TYPE: float
edof_	Effective degrees of freedom — trace of the smoother matrix at the fitted solution; the honest complexity measure. TYPE: float
n_knots_	Number of knots actually used. TYPE: int
is_monotone_	Checked on a dense grid after fitting; the penalty does not enforce monotonicity, and a rare non-monotone fit is flagged with a warning. TYPE: bool

References

Lucena (2018); Hastie, Tibshirani & Friedman (2009), §5.2.1.

Source code in src/probcal/spline.py

def __init__(
    self,
    n_knots: int | None = None,
    lambdas: object = None,
    cv: int = 5,
    random_state: int = 42,
) -> None:
    self.n_knots = n_knots
    self.lambdas = lambdas
    self.cv = cv
    self.random_state = random_state

interpret

interpret() -> Interpretation

Read effective degrees of freedom as the honest complexity measure.

Source code in src/probcal/spline.py

def interpret(self) -> Interpretation:
    """Read effective degrees of freedom as the honest complexity measure."""
    self._check_fitted()
    messages = [
        (
            f"effective degrees of freedom {self.edof_:.2f} (trace of the smoother): "
            "values near 2 mean a parametric family would have sufficed; larger values "
            "mean the curvature is real"
        ),
        (
            f"penalty lambda = {self.lambda_:.4g} chosen by {self.cv}-fold "
            f"cross-validated log loss over {len(self.lambdas_grid_)} candidates; "
            f"{self.n_knots_} knots at logit-score quantiles"
        ),
        (
            "regions where the fitted curve runs steeper than the identity are locally "
            "underconfident score regions; shallower, locally overconfident"
        ),
    ]
    if not self.is_monotone_:
        messages.append("fitted curve is NOT monotone on the probe grid (warned at fit)")
    return Interpretation(
        method=type(self).__name__,
        param_names=("edof", "lambda", "n_knots"),
        param_values=(self.edof_, self.lambda_, float(self.n_knots_)),
        messages=tuple(messages),
    )