Skip to content

treecf: counterfactuals on calibrated targets

treecf computes exact counterfactuals on tree ensembles; its Target.calibrated accepts any object with the probcal calibrator protocol (is_monotone_, interval_inverse). Every probcal calibrator, LogitOffset, Chain, CalibratedModel, probcal.sklearn.CalibratedClassifier, and CalibratedScorecard conform (pinned by tests/test_calibrator_protocol.py). treecf never imports probcal at runtime; the coupling is one duck-typed protocol. Extra: pip install "probcal[treecf]" (treecf ≥ 0.2.4); the joint smoke test runs whenever both are installed.

Case 1, parametric: "below 2% calibrated PD"

# docs: no-run — needs a real tree-ensemble model and feature matrix;
# see tests/test_treecf_integration.py for a runnable fixture
from treecf import Explainer, Target
from probcal import BetaCalibrator

cal = BetaCalibrator().fit(model_scores_cal, y_cal)     # held-out data
exp = Explainer(model=model, background=X_train_sample)

res = exp.explain(x, target=Target.calibrated(cal, op="<=", value=0.02),
                  seed=0, backend="exact")
cal.predict_proba(model.predict_proba(res.x_cf[None])[:, 1])  # <= 0.02, certified

treecf resolves the calibrated target once, through cal.interval_inverse(..., space="logit"), which gives bounds on the raw margin, and optimizes there; the calibrated read-out is exact because the inverse is.

Case 2, isotonic: a target inside a plateau

Step calibrators map whole raw regions to one level. Ask for "at most the plateau level" and the counterfactual lands at the block boundary, the cheapest raw score whose level qualifies, because probcal's generalized inverse returns the largest raw score inside the preimage (one float below the next block's edge), so treecf's closed-bound treatment can never overshoot into the next block. The full contract, with a worked example: the Inverse maps chapter, "The step-calibrator contract".

# docs: no-run — continues the Case 1 setup (exp, x); see that block's note
iso = IsotonicCalibrator().fit(model_scores_cal, y_cal)
res = exp.explain(x, target=Target.calibrated(iso, op="<=", value=plateau_level),
                  seed=0, backend="exact")
# res lands on the boundary of the block whose level == plateau_level

Case 3, after a macro shift: Chain

After a LogitOffset re-anchoring, recourse must invert offset ∘ calibrator; inverting the calibrator alone answers yesterday's policy. Chain composes the stages exactly, and CalibratedModel.chain_ hands you the composed map without the model:

# docs: no-run — continues the Case 1 setup (cal, exp, x); see that block's note
from probcal import Chain, LogitOffset

off = LogitOffset(target_mean=0.031).fit(cal.predict_proba(model_scores_now))
chain = Chain([cal, off])                       # or wrapped.chain_
res = exp.explain(x, target=Target.calibrated(chain, op="<=", value=0.02),
                  seed=0, backend="exact")

Grade recourse and buffer_logit

Target.bands(ms.bands, space="calibrated", calibrator=cal) solves a whole rating ladder in one call: one counterfactual (or certified infeasibility) per grade band. treecf takes the band dict, which a Masterscale exposes as ms.bands; accepting the object directly is a one-line follow-up on the treecf side, not something this package needs.

buffer_logit shrinks the calibrated interval in logit space before inversion, so a future central-tendency update of at most that magnitude cannot invalidate the counterfactual. The principled value is the monitor's offset confidence-sequence half-width: with mon = CalibrationMonitor(...), take 0.5 * (delta_ci[1] - delta_ci[0]) from the latest MonitorStep. A re-offset anywhere inside the anytime-valid sequence then leaves buffered recourse certificates intact.

Pitfall: Target.probability is not the calibrated probability

Target.probability inverts the model's sigmoid, giving the raw margin's probability, not the calibrator's output. A "2% PD" policy defined on calibrated probabilities but requested through Target.probability(0.02) silently targets the wrong quantity whenever the calibrator is not the identity. Calibrated policies go through Target.calibrated (or Target.bands(space="calibrated")), always.

What treecf provides for calibrated targets

treecf ≥ 0.2.4, the version the probcal[treecf] extra pins, closes the loop from its side:

  • Provenance: the certificate's calibrated-target block carries a structured calibrator sub-block {embedded, fingerprint, type, buffer_logit}, where the fingerprint comes from the duck-typed fingerprint() (available on every probcal object) and is null when absent. Every BatchRecord carries calibrator_fingerprint. Explainer.check_certificate(cert, calibrator=...) re-checks the fingerprint and re-inverts the stored lo/hi against the stored raw interval.
  • Calibrated read-out: Counterfactual.score_calibrated and the certificate factual block's score_calibrated carry the calibrator's probability at the counterfactual and factual points (presentational; optimization stays on the raw margin).
  • Plateau-aware feasibility tests: an engine-level plateau suite against brute force, run twice, once with a counting stub implementing probcal's generalized-inverse step contract and once with real probcal isotonic/centered-isotonic fits, asserting identical feasibility decisions on closed intervals.
  • Test matrix: a matrix test module in treecf with probcal as an optional test dependency: {Platt, Temperature, Beta abm, Isotonic, CenteredIsotonic, Chain(Beta + offset), CalibratedModel.chain_} × {op="<=", op=">=", range, bands} × {buffer_logit 0, 0.2} on sklearn and LightGBM ensembles, every plan re-verified through the model and calibrator.
  • Bit-exact boundary routing: sklearn tree_ ensembles route float32(x) <= float64(threshold), and a counterfactual coordinate placed exactly on a split threshold (the natural optimum of a smallest-change search) must route the same way in treecf as in the model. treecf re-expresses each threshold as the exact float64 boundary of the float32 cast, so routing is bit-exact for every input (treecf#21). XGBoost also casts features to float32 natively; the analogous handling is a treecf follow-up.
  • Docs: treecf's concepts/calibration.md documents provenance and the read-out, and pin tests count exactly one interval_inverse call per explain_batch (one per band for ladders).

What the closed loop gives a validator: a certificate plus the calibrator's probcal JSON is a self-contained, independently verifiable pair: from_json the calibrator, match its fingerprint() against the certificate's calibrator block, and let check_certificate(cert, calibrator=...) re-invert the stored calibrated bounds against the stored raw interval. For buffer_logit, take the value from the monitor's offset confidence-sequence half-width (see above).