probcal

Post-hoc probability calibration and calibration governance for binary classifiers: the evidence a regulated PD model needs alongside the map itself. If you have binary scores that feed a threshold, a price, or a report, this applies to you; PD models are the archetype because they are the most audited, and the glossary maps their vocabulary to the generic one. The core is numpy-only; everything else is an optional extra.

One deployed calibration runs a loop, and probcal covers all of it: calibrate → evaluate → invert and decide → monitor → act → report, then back to re-anchoring. The How it works page draws that loop once, in one view.

Calibrate. Thirteen calibrators: Platt, temperature, beta, isotonic, centered isotonic, histogram binning, scaling-binning, BBQ, ENIR, Venn–Abers (IVAP and CVAP), spline, and segmented. With them come a rigid, auditable logit offset, automatic selection under nested cross-validation, and prefit / cross-validation data flows. Choose a calibrator is the catalog.

Evaluate. Proper scores, binned and binning-free calibration errors, the CORP decomposition, per-grade supervisory backtests, calibration belts, reliability curves on both probability and logit scales, grouped evaluation by segment, and conservatism tooling for low-default portfolios (Pluto–Tasche most-prudent PDs, Jeffreys upper bands, margin-of-conservatism offsets).

Decide. Exact and generalized inverses turn a policy PD into a raw-score cutoff or a whole masterscale, refusing unattainable targets instead of clamping. Set cutoffs and invert maps.

Monitor and act. Anytime-valid monitoring runs on an e-process whose alarm keeps its type-I guarantee at every look, under stated assumptions. It localizes the onset of drift and issues a recommendation that can be applied as a new offset, on the record. Monitor and act.

Prove it. JSON serialization (never pickle) with fingerprints and a golden-file compatibility promise, self-contained HTML reports, and one page tying the artifacts to what each of them actually proves: Auditability.

Integrate. scikit-learn (on ≥ 1.6 a bare calibrator is accepted as an estimator without any adapter; the adapter covers the probability-matrix API from 1.4, see the support matrix), optbinning scorecards (calibrated PDs carried back to the points scale), and treecf counterfactuals bound to a named calibrator.

pip install probcal

Where to start, by what you are doing:

• New to calibration → Install and quickstart, then the 20-minute PD calibration walkthrough (diagnose, select, fit, backtest). The longer full lifecycle notebook adds offsetting, treecf recourse, monitoring, and JSON round-trips on a rare-event portfolio.
• Validating someone's model → Auditability and Build a validation report.
• Coming from scikit-learn → the three-tier sklearn guide.
• Unsure what a word means → the glossary.

Status: beta on PyPI. Until 1.0, an API-breaking change bumps the minor version; a change that only alters computed numbers may ship in any release. Both are listed in the changelog with the reasoning and keep an escape hatch where the old behaviour had legitimate uses. A deprecated symbol warns with a DeprecationWarning for at least one minor release before removal, naming its replacement. Serialized artifacts carry a stronger promise: every 0.x release reads schema 1, pinned by golden files in CI.

The full policy, the public surface, and the support matrix: API stability.