optbinning scorecards

Data these snippets assume

Blocks on this page use a fixed set of names for held-out data instead of re-deriving it each time. To run them, define the names first:

# docs: no-run — the definitions the snippets on this page assume
import numpy as np
from probcal import BetaCalibrator, Masterscale, make_pd_portfolio
from probcal.monitor import CalibrationMonitor

cal_set, new_set = make_pd_portfolio(n=3000, random_state=0), make_pd_portfolio(n=1000, random_state=1)
s_cal, y_cal = cal_set.scores, cal_set.y          # held-out calibration scores and outcomes
w_cal = np.ones_like(y_cal)                       # uniform sample weights
s_new = new_set.scores                            # scores of new obligors
ms = Masterscale.from_edges([0.01, 0.05], names=["G1", "G2", "G3"])   # the masterscale
p_cal = BetaCalibrator().fit(s_cal, y_cal).predict_proba(s_cal)         # calibrated PDs
grades = ms.assign(p_cal)                                              # rating labels
segments = np.array(["seg-a", "seg-b", "seg-c"])[np.arange(len(s_cal)) % 3]  # segment labels
model = ...   # any object with predict_proba(X); the docs use a stub that reads X[:, 0]
mon = CalibrationMonitor(alpha=0.05)              # with a few batches of a calibrated forecast applied

Every block also names, in its first comment line, which of these it uses.

probcal.integrations.optbinning (extra: pip install "probcal[optbinning]", optbinning ≥ 0.21) calibrates a fitted Scorecard without touching the deployed points, and translates calibrated PD policies to the points scale exactly. Scorecard.score is affine in the logistic regression's log-odds unless rounding=True, and the integration recovers and verifies that affine map to machine precision.

Scorecard → calibration → masterscale → monitoring, end to end:

# s_cal, y_cal, s_new: held-out calibration scores, outcomes, scores of new obligors
import pandas as pd
from optbinning import BinningProcess, Scorecard
from sklearn.linear_model import LogisticRegression

from probcal import Masterscale
from probcal._math import logit
from probcal.integrations.optbinning import calibrate_scorecard
from probcal.monitor import CalibrationMonitor

# X_train, y_train: training split; X_cal, y_cal: held-out calibration
# split; X_new: new applicants — built here from s_cal/s_new (one numeric
# feature, the model's own logit) since the walkthrough carries no raw
# tabular features.
X_train = pd.DataFrame({"z": logit(s_cal)})
y_train = y_cal.astype(int)
X_cal = X_train
X_new = pd.DataFrame({"z": logit(s_new)})

# 1. A fitted scorecard (train split).
sc = Scorecard(
    binning_process=BinningProcess(variable_names=list(X_train.columns)),
    estimator=LogisticRegression(),
    scaling_method="pdo_odds",
    scaling_method_params={"pdo": 20, "odds": 50, "scorecard_points": 600},
).fit(X_train, y_train)

# 2. Calibrate on held-out data. Points are untouched; predict_proba is
#    the calibrated PD; the fitted probcal object is one attribute away.
cs = calibrate_scorecard(sc, X_cal, y_cal)
print(cs.interpret())
pd_cal = cs.predict_proba(X_new)          # calibrated PD
points = cs.score(X_new)                  # unchanged deployed points

# 3. Masterscale: calibrated PD bands -> exact point cut-offs.
ms = Masterscale.from_edges([0.005, 0.02, 0.08], names=list("ABCD"))
print(cs.masterscale(ms))                 # cut-offs on the points scale
# -> {'A': (638.68, inf), 'B': (598.96, 638.68), ...}

# 4. Provenance: the JSON names both layers.
cs.to_json("scorecard-calibration.json")   # calibrator envelope +
cs.fingerprint()                           # scorecard-table fingerprint

# 5. Monitoring: matured cohorts through the anytime-valid monitor.
cohorts = {"m0": (X_cal.iloc[:1500], y_cal[:1500]), "m1": (X_cal.iloc[1500:], y_cal[1500:])}
mon = CalibrationMonitor(alpha=0.05)
for label, (X_b, y_b) in cohorts.items():
    mon.update(y_b, cs.predict_proba(X_b), label=label)
print(mon.report().recommendation)

With rounding=True the points are no longer affine in log-odds: calibrate_scorecard warns, points_affine_coeffs_ is None, and masterscale refuses. Use interval_inverse on the raw model probability instead (the exact escape hatch; no silent approximation).

Reattaching after a reload: persist the scorecard with optbinning's own save/load and the calibration layer as JSON, then CalibratedScorecard.from_json("scorecard-calibration.json", scorecard=sc) (or from_dict(d, scorecard=sc)). The stored scorecard-table fingerprint is checked, so a calibration layer can never be silently attached to a different scorecard. A scorecard whose probability is constant on the calibration data gets the same treatment as rounding=True: a warning, points_affine_coeffs_ = None, and a refusing masterscale.