TRIAD Guidebook — synthetic credit portfolio¶

TRIAD splits every feature's contribution to a model score into three signed channels that sum back exactly: I (information — supported by training data), D (density / epistemic — where the model is guessing) and M (missingness — routed through missing-value handling). The point is a distinction adverse-action letters need: "denied because of bad information" (Group A) versus "denied because we lack information" (Group B).

This notebook walks the full workflow on synthetic credit data with known ground truth:

1. Setup — a thin-file portfolio with informative (MNAR) missingness.
2. Exact mode — fit an additive LightGBM, decompose, verify reconciliation.
3. Epistemic scores — ES, ES_signed and knowledge ratio KR.
4. Reason codes — Group A vs Group B adverse-action reasons.
5. Three-channel waterfalls.
6. Density gap — the D channel finds where training data is absent.
7. Approximate mode — any LightGBM booster via TreeSHAP reallocation.
8. Audit payload — shrinkage parameters per term.
9. Summary and API pointers.

1. Setup¶

make_missingness generates an additive logistic DGP with three features and masks 30% of x0 under MNAR: the masking probability rises with x0 itself, and masked rows get +1.0 planted on their default logit. That is informative missingness — the thin-file situation where a lender must separate derogatory evidence from absent evidence. Higher score means higher default risk throughout.

In [1]:

import numpy as np
import pandas as pd

from triadxai import TriadExplainer, fit_lgbm_gam
from triadxai.synthetic import make_missingness

%matplotlib inline

import numpy as np import pandas as pd from triadxai import TriadExplainer, fit_lgbm_gam from triadxai.synthetic import make_missingness %matplotlib inline

In [2]:

data = make_missingness(n=6000, mechanism="MNAR", missing_rate=0.3, missing_effect=1.0, seed=0)
X, y = data.X, data.y
print(f"n={len(X)}, base rate={y.mean():.3f}, x0 missing share={X['x0'].isna().mean():.3f}")
X.head(5)

data = make_missingness(n=6000, mechanism="MNAR", missing_rate=0.3, missing_effect=1.0, seed=0) X, y = data.X, data.y print(f"n={len(X)}, base rate={y.mean():.3f}, x0 missing share={X['x0'].isna().mean():.3f}") X.head(5)

n=6000, base rate=0.550, x0 missing share=0.292

Out[2]:

	x0	x1	x2
0	NaN	-1.351865	0.321473
1	-0.132105	-0.662572	-0.811584
2	0.640423	-0.081800	0.448092
3	0.104900	0.879421	-1.567154
4	-0.535669	1.604094	-1.089273

2. Exact mode: fit and decompose¶

fit_lgbm_gam constrains every tree to split on a single feature, so the booster is an additive model and the TRIAD decomposition is exact; n_bags=8 bootstrap replicas supply per-bin epistemic variance. TriadExplainer infers exact mode, and explain returns a long channels frame: one row per (instance, feature) with the signed channels, the support weight w and an out-of-view flag oov.

In [3]:

result = fit_lgbm_gam(X, y, n_bags=8, seed=0)
explainer = TriadExplainer(result)

X_eval = X.head(400)
exp = explainer.explain(X_eval)
print(f"mode={exp.mode}, approximate={exp.approximate}")
exp.channels.head(6)

result = fit_lgbm_gam(X, y, n_bags=8, seed=0) explainer = TriadExplainer(result) X_eval = X.head(400) exp = explainer.explain(X_eval) print(f"mode={exp.mode}, approximate={exp.approximate}") exp.channels.head(6)

mode=exact, approximate=False

Out[3]:

	instance	feature	I	D	M	w	oov
0	0	x0	0.000000	0.002474	0.981076	0.997484	False
1	0	x1	-0.289743	-0.174455	0.000000	0.624180	False
2	0	x2	-0.272074	-0.548795	0.000000	0.331447	False
3	1	x0	-0.741672	-0.125090	0.000000	0.855681	False
4	1	x1	-0.608558	-0.294440	0.000000	0.673931	False
5	1	x2	-0.036273	-0.112302	0.000000	0.244139	False

Reconciliation and channel mass¶

C2 requires channel totals plus the intercept to reproduce the model's raw score to machine precision — explain guards this internally, and reconciliation_error verifies it externally. channel_mass reports the portfolio share of mean |I|, |D| and |M|.

In [4]:

from triadxai.metrics import channel_mass, reconciliation_error

print(f"C2 reconciliation error: {reconciliation_error(exp, result, X_eval):.2e}")
channel_mass(exp).rename("share").to_frame()

from triadxai.metrics import channel_mass, reconciliation_error print(f"C2 reconciliation error: {reconciliation_error(exp, result, X_eval):.2e}") channel_mass(exp).rename("share").to_frame()

C2 reconciliation error: 6.22e-15

Out[4]:

	share
I	0.622966
D	0.258555
M	0.118479

3. Epistemic scores¶

Per instance: ES = sum_j |D_j| + |M_j| (how much of the score rests on weak support), ES_signed = sum_j D_j + M_j (its net direction) and knowledge ratio KR = sum|I| / (sum|I| + ES). Thin-file rows should show lower KR, and their M on x0 should recover the planted +1.0 MNAR effect in sign.

In [5]:

es = exp.epistemic_score()
es.head()

es = exp.epistemic_score() es.head()

Out[5]:

	ES	ES_signed	KR
instance
0	1.706800	0.260301	0.247648
1	0.531832	-0.531832	0.722764
2	0.211561	-0.184487	0.646600
3	0.328773	0.269838	0.574013
4	0.661873	-0.661873	0.710798

In [6]:

thin = X_eval["x0"].isna().to_numpy()
m_x0 = exp.channels.query("feature == 'x0'").sort_values("instance")["M"].to_numpy()
print(f"mean M on x0 — thin files: {m_x0[thin].mean():+.3f}, complete: {m_x0[~thin].mean():+.3f}")
es.groupby(np.where(thin, "thin file (x0 missing)", "complete")).mean()

thin = X_eval["x0"].isna().to_numpy() m_x0 = exp.channels.query("feature == 'x0'").sort_values("instance")["M"].to_numpy() print(f"mean M on x0 — thin files: {m_x0[thin].mean():+.3f}, complete: {m_x0[~thin].mean():+.3f}") es.groupby(np.where(thin, "thin file (x0 missing)", "complete")).mean()

mean M on x0 — thin files: +0.981, complete: +0.000

Out[6]:

	ES	ES_signed	KR
complete	0.578023	-0.027096	0.720253
thin file (x0 missing)	1.340810	0.986830	0.296707

4. Reason codes¶

reasons() maps each instance's channels to ranked adverse-action reasons: Group A ("derogatory information", carried by I) versus Group B ("insufficient information", carried by D + M and split into missing vs density subtypes). Because a higher score means higher default risk here, we pass orientation="lower_is_better" so score-raising contributions count as adverse. First a thin-file applicant — expect Group B on x0:

In [7]:

from dataclasses import asdict

reasons = exp.reasons(orientation="lower_is_better", k=4)
i_thin = int(np.flatnonzero(thin)[0])
pd.DataFrame([asdict(code) for code in reasons[i_thin]])

from dataclasses import asdict reasons = exp.reasons(orientation="lower_is_better", k=4) i_thin = int(np.flatnonzero(thin)[0]) pd.DataFrame([asdict(code) for code in reasons[i_thin]])

Out[7]:

	feature	group	subtype	contribution	text
0	x0	B	insufficient_missing	-0.983551	No or limited information on file: x0

The highest-scoring complete applicant, by contrast, is declined on derogatory information — Group A reasons carried by the I channel:

In [8]:

complete_idx = np.flatnonzero(~thin)
i_complete = int(complete_idx[np.argmax(exp.score[complete_idx])])
pd.DataFrame([asdict(code) for code in reasons[i_complete]])

complete_idx = np.flatnonzero(~thin) i_complete = int(complete_idx[np.argmax(exp.score[complete_idx])]) pd.DataFrame([asdict(code) for code in reasons[i_complete]])

Out[8]:

	feature	group	subtype	contribution	text
0	x0	A	derogatory	-2.632419	Unfavorable information: x0
1	x1	A	derogatory	-1.275024	Unfavorable information: x1

5. Three-channel waterfall¶

plot_waterfall renders one instance's decomposition as stacked bars per feature: I solid, D hatched, M in its own colour; features outside the training view are starred. The thin-file applicant's score is carried by the M segment on x0; the derogatory applicant's by solid I.

In [9]:

exp.plot_waterfall(i_thin);

exp.plot_waterfall(i_thin);

In [10]:

exp.plot_waterfall(i_complete);

exp.plot_waterfall(i_complete);

6. Density gap: where D lives¶

make_density_gap carves the training mass out of x0 ∈ (1.0, 2.5) (2% kept). The model still extrapolates through that region, and TRIAD should reallocate those contributions from I to D. We profile |D| on an x0 grid for two training sizes: D should concentrate inside the gap and shrink as data grows (invariant C3).

In [11]:

from triadxai.synthetic import make_density_gap

GAP = (1.0, 2.5)
grid = np.linspace(-3.0, 4.0, 141)
X_grid = pd.DataFrame({"x0": grid, "x1": np.zeros_like(grid), "x2": np.zeros_like(grid)})


def d_profile(n: int) -> np.ndarray:
    gap_data = make_density_gap(n, gap=GAP, keep_frac=0.02, seed=0)
    gap_result = fit_lgbm_gam(gap_data.X, gap_data.y, n_bags=6, seed=0, n_estimators=200)
    rows = TriadExplainer(gap_result).explain(X_grid).channels.query("feature == 'x0'")
    return rows.sort_values("instance")["D"].abs().to_numpy()


d_4k, d_16k = d_profile(4_000), d_profile(16_000)

from triadxai.synthetic import make_density_gap GAP = (1.0, 2.5) grid = np.linspace(-3.0, 4.0, 141) X_grid = pd.DataFrame({"x0": grid, "x1": np.zeros_like(grid), "x2": np.zeros_like(grid)}) def d_profile(n: int) -> np.ndarray: gap_data = make_density_gap(n, gap=GAP, keep_frac=0.02, seed=0) gap_result = fit_lgbm_gam(gap_data.X, gap_data.y, n_bags=6, seed=0, n_estimators=200) rows = TriadExplainer(gap_result).explain(X_grid).channels.query("feature == 'x0'") return rows.sort_values("instance")["D"].abs().to_numpy() d_4k, d_16k = d_profile(4_000), d_profile(16_000)

In [12]:

import matplotlib.pyplot as plt

fig, ax = plt.subplots(figsize=(8, 4))
ax.plot(grid, d_4k, label="n = 4,000")
ax.plot(grid, d_16k, label="n = 16,000")
ax.axvspan(*GAP, alpha=0.15, color="grey", label="carved-out region")
ax.set_xlabel("x0")
ax.set_ylabel("|D| on x0")
ax.set_title("D concentrates where training data is absent and shrinks with n (C3)")
ax.legend();

import matplotlib.pyplot as plt fig, ax = plt.subplots(figsize=(8, 4)) ax.plot(grid, d_4k, label="n = 4,000") ax.plot(grid, d_16k, label="n = 16,000") ax.axvspan(*GAP, alpha=0.15, color="grey", label="carved-out region") ax.set_xlabel("x0") ax.set_ylabel("|D| on x0") ax.set_title("D concentrates where training data is absent and shrinks with n (C3)") ax.legend();

7. Approximate mode: any LightGBM booster¶

For an unconstrained booster (the DGP now adds an x0·x1 interaction inside the same density gap), exact term contributions do not exist. TRIAD instead reallocates native TreeSHAP attributions (pred_contrib) into the same channels: bag replicas from fit_bagged provide the per-instance variance signal, and X_ref anchors the τ² estimate. approximate=True, waterfalls carry an "approximate" badge, and C2 still reconciles to the raw score exactly.

In [13]:

import lightgbm as lgb

from triadxai.bagging import fit_bagged
from triadxai.synthetic import make_interacting

inter = make_interacting(n=6000, gap=GAP, seed=0)


def fit_fn(X: pd.DataFrame, y: np.ndarray, seed: int) -> lgb.Booster:
    return lgb.train(
        {"objective": "binary", "verbosity": -1, "seed": seed},
        lgb.Dataset(X, label=y),
        num_boost_round=300,
    )


booster = fit_fn(inter.X, inter.y, 0)
bags = fit_bagged(fit_fn, inter.X, inter.y, n_bags=8, seed=0)

shap_explainer = TriadExplainer(booster, bag_boosters=bags, X_ref=inter.X)
shap_exp = shap_explainer.explain(inter.X.head(400))
print(f"mode={shap_exp.mode}, approximate={shap_exp.approximate}")
print(f"C2 reconciliation error: {reconciliation_error(shap_exp, booster, inter.X.head(400)):.2e}")
channel_mass(shap_exp).rename("share").to_frame()

import lightgbm as lgb from triadxai.bagging import fit_bagged from triadxai.synthetic import make_interacting inter = make_interacting(n=6000, gap=GAP, seed=0) def fit_fn(X: pd.DataFrame, y: np.ndarray, seed: int) -> lgb.Booster: return lgb.train( {"objective": "binary", "verbosity": -1, "seed": seed}, lgb.Dataset(X, label=y), num_boost_round=300, ) booster = fit_fn(inter.X, inter.y, 0) bags = fit_bagged(fit_fn, inter.X, inter.y, n_bags=8, seed=0) shap_explainer = TriadExplainer(booster, bag_boosters=bags, X_ref=inter.X) shap_exp = shap_explainer.explain(inter.X.head(400)) print(f"mode={shap_exp.mode}, approximate={shap_exp.approximate}") print(f"C2 reconciliation error: {reconciliation_error(shap_exp, booster, inter.X.head(400)):.2e}") channel_mass(shap_exp).rename("share").to_frame()

mode=shap, approximate=True
C2 reconciliation error: 3.55e-14

Out[13]:

	share
I	0.716072
D	0.283928
M	0.000000

In [14]:

X_grid_i = pd.DataFrame({"x0": grid, "x1": np.ones_like(grid), "x2": np.zeros_like(grid)})
rows = shap_explainer.explain(X_grid_i).channels.query("feature == 'x0'").sort_values("instance")

fig, ax = plt.subplots(figsize=(8, 3.5))
ax.plot(grid, rows["D"].abs().to_numpy(), color="tab:red")
ax.axvspan(*GAP, alpha=0.15, color="grey", label="carved-out region")
ax.set_xlabel("x0")
ax.set_ylabel("|D| on x0")
ax.set_title("Approximate mode: bag disagreement lights up the same gap")
ax.legend();

X_grid_i = pd.DataFrame({"x0": grid, "x1": np.ones_like(grid), "x2": np.zeros_like(grid)}) rows = shap_explainer.explain(X_grid_i).channels.query("feature == 'x0'").sort_values("instance") fig, ax = plt.subplots(figsize=(8, 3.5)) ax.plot(grid, rows["D"].abs().to_numpy(), color="tab:red") ax.axvspan(*GAP, alpha=0.15, color="grey", label="carved-out region") ax.set_xlabel("x0") ax.set_ylabel("|D| on x0") ax.set_title("Approximate mode: bag disagreement lights up the same gap") ax.legend();

In [15]:

shap_exp.plot_waterfall(0);

shap_exp.plot_waterfall(0);

8. Audit payload¶

The audit payload required by spec §4.4: per-term shrinkage parameters — signal variance τ², average noise σ², bin count k — plus the per-feature support weight w already present in the channels frame. In approximate mode, shrinkage reports the per-feature τ² instead.

In [16]:

explainer.shrinkage

explainer.shrinkage

Out[16]:

	term	features	tau2	sigma2	k
0	0	x0	1.391379	5.937838	4.267591
1	1	x1	0.387370	6.345807	16.381777
2	2	x2	0.064032	4.365137	68.170940

9. Summary¶

• Exact mode reconciled to the model score at machine precision (C2), with the channel-mass split printed in §2.
• Thin-file rows showed lower KR, and the planted +1.0 MNAR effect surfaced as positive M on x0 (§3).
• Reason codes separated Group A (derogatory, I-driven) from Group B (insufficient information, D+M-driven) applicants (§4).
• The D channel concentrated inside the carved-out density gap and shrank with more training data (§6, C3).
• Approximate mode told the same story for an interacting booster via TreeSHAP reallocation, with C2 intact (§7).

API reference for everything used above: local · lgbm · bagging · reasons · waterfall · metrics · synthetic · shrinkage.