API reference¶
Full auto-generated API documentation from source code docstrings.
SWIFTMonitor¶
The main entry point — the scikit-learn compatible monitor orchestrating all five stages.
SWIFTMonitor
¶
SWIFTMonitor(model: object = None, order: int = 1, n_permutations: int = 1000, alpha: float = 0.05, correction: CorrectionMethod | str = BH, n_synthetic: int = 10, max_samples: int | None = None, random_state: int = 42)
Bases: MonitorPlottingMixin, BaseEstimator, TransformerMixin
SHAP-Weighted Impact Feature Testing monitor.
Orchestrates the 5-stage SWIFT pipeline:
1. Extract decision points from trained model
2. Build buckets from decision points
3. Compute bucket-level mean SHAP (SHAP normalization σ_j)
4. Compute Wasserstein distance on SHAP-transformed distributions
5. Permutation-based significance testing with MTC
The transformation σ_j is computed ONCE during fit() and applied
identically to all monitoring samples via transform().
| PARAMETER | DESCRIPTION |
|---|---|
model
|
Trained tree-ensemble model (LightGBM
TYPE:
|
order
|
Wasserstein order (1 → W₁, 2 → W₂).
TYPE:
|
n_permutations
|
Number of permutations for p-value estimation in
TYPE:
|
alpha
|
Significance level for multiple testing correction.
TYPE:
|
correction
|
Multiple testing correction method. Accepts enum members or
strings (
TYPE:
|
n_synthetic
|
Number of synthetic observations to create for empty buckets
during
TYPE:
|
max_samples
|
Maximum total pool size (n_ref + n_mon) for the permutation test.
If exceeded, subsample proportionally.
TYPE:
|
random_state
|
Seed for the random number generator, ensuring reproducibility.
TYPE:
|
| ATTRIBUTE | DESCRIPTION |
|---|---|
bucket_sets_ |
Per-feature bucket sets with
TYPE:
|
X_ref_ |
Copy of the reference DataFrame (stored for permutation testing).
TYPE:
|
shap_values_ |
SHAP values computed on the reference data.
TYPE:
|
importance_weights_ |
Normalized mean |SHAP| importance weight per feature (sums to 1);
used for the
TYPE:
|
feature_names_in_ |
Feature names inferred from
TYPE:
|
n_features_in_ |
Number of features seen during
TYPE:
|
Examples:
>>> monitor = SWIFTMonitor(model=lgb_model, n_permutations=200)
>>> monitor.fit(X_ref)
SWIFTMonitor(...)
>>> result = monitor.test(X_mon)
>>> result.drifted_features
('feature_3',)
Source code in src/swift/pipeline.py
152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 | |
fit
¶
fit(X: DataFrame, y: None = None) -> SWIFTMonitor
Fit the SWIFT monitor on reference data.
Executes stages 1–3: extraction → bucketing → SHAP normalization.
| PARAMETER | DESCRIPTION |
|---|---|
X
|
Reference DataFrame. Feature names are inferred from
TYPE:
|
y
|
Not used; present for API compatibility.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
self
|
|
Source code in src/swift/pipeline.py
176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 | |
transform
¶
transform(X: DataFrame) -> DataFrame
Apply the SHAP transformation σ_j to every feature.
Each value x_ij is mapped to the mean SHAP of its bucket:
σ_j(x_ij) = mean_shap_j^{bucket(x_ij)}.
| PARAMETER | DESCRIPTION |
|---|---|
X
|
Input data.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
DataFrame
|
SHAP-transformed DataFrame (same shape and column names). |
Source code in src/swift/pipeline.py
252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 | |
score
¶
score(X: DataFrame, X_compare: DataFrame | None = None) -> dict[str, float]
Compute per-feature SWIFT scores (stage 4 only, no testing).
| PARAMETER | DESCRIPTION |
|---|---|
X
|
Monitoring DataFrame. When X_compare is
TYPE:
|
X_compare
|
Optional second sample. When provided, SWIFT scores are
computed between X and X_compare instead of between
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
dict[str, float]
|
Feature name → SWIFT score (Wasserstein distance on SHAP-transformed distributions). |
Source code in src/swift/pipeline.py
276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 | |
test
¶
test(X: DataFrame, X_compare: DataFrame | None = None) -> SWIFTResult
Run the full SWIFT pipeline: score + test + aggregate.
Stages 4–5 + aggregation:
- Compute per-feature SWIFT scores (Wasserstein on SHAP-transformed).
- Permutation test for p-values.
- Multiple testing correction.
- Model-level aggregation.
All hyperparameters (order, n_permutations, alpha,
correction, max_samples) are taken from instance attributes
set in the constructor. Use set_params() to override for
individual calls (e.g. different random_state per experiment
repetition).
| PARAMETER | DESCRIPTION |
|---|---|
X
|
Monitoring DataFrame. When X_compare is
TYPE:
|
X_compare
|
Optional second sample. When provided, the test compares
X against X_compare instead of
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
SWIFTResult
|
Per-feature and model-level results. |
Source code in src/swift/pipeline.py
311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 | |
Types¶
Data types used across the pipeline: buckets, bucket sets, result records, and enums.
Bucket
dataclass
¶
Bucket(bucket_type: BucketType, index: int, lower: float = float('-inf'), upper: float = float('inf'), mean_shap: float | None = None)
A single bucket for a feature.
For numeric features: defined by [lower, upper) interval. For null bucket: lower=upper=NaN.
contains
¶
contains(value: float | None) -> bool
Check if a value falls into this bucket.
Source code in src/swift/types.py
103 104 105 106 107 108 109 110 111 112 113 114 115 116 | |
BucketSet
dataclass
¶
BucketSet(feature_name: str, buckets: tuple[Bucket, ...] = tuple(), decision_points: ndarray = (lambda: array([]))())
Collection of buckets for a single feature.
| ATTRIBUTE | DESCRIPTION |
|---|---|
feature_name |
Name of the feature.
TYPE:
|
buckets |
List of Bucket objects (including null bucket).
TYPE:
|
decision_points |
Sorted array of decision points (split thresholds).
TYPE:
|
assign_bucket
¶
assign_bucket(value: float | str | None) -> int
Return the index of the bucket that contains the given value.
| RAISES | DESCRIPTION |
|---|---|
ValueError
|
If the value does not fall into any bucket. |
Source code in src/swift/types.py
139 140 141 142 143 144 145 146 147 148 149 150 151 152 | |
get_mean_shap
¶
get_mean_shap(bucket_index: int) -> float
Return the mean SHAP value for the given bucket index.
| RAISES | DESCRIPTION |
|---|---|
KeyError
|
If the bucket index does not exist. |
Source code in src/swift/types.py
154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 | |
BucketType
¶
Bases: Enum
Type of bucket.
WassersteinOrder
¶
Bases: Enum
Order of Wasserstein distance.
CorrectionMethod
¶
Bases: Enum
Multiple testing correction method.
Supports construction from strings via CorrectionMethod.resolve(),
so users can write correction="benjamini-hochberg" instead of
importing the enum.
resolve
classmethod
¶
resolve(value: CorrectionMethod | str) -> CorrectionMethod
Return a CorrectionMethod from a member, its value, or an alias.
| PARAMETER | DESCRIPTION |
|---|---|
value
|
Enum member, canonical string (
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
CorrectionMethod
|
|
| RAISES | DESCRIPTION |
|---|---|
ValueError
|
If value cannot be resolved. |
Source code in src/swift/types.py
39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 | |
FeatureSWIFTResult
dataclass
¶
FeatureSWIFTResult(feature_name: str, swift_score: float, wasserstein_order: WassersteinOrder = W1, p_value: float | None = None, is_drifted: bool | None = None, num_buckets: int = 0)
SWIFT result for a single feature.
| ATTRIBUTE | DESCRIPTION |
|---|---|
feature_name |
Name of the feature.
TYPE:
|
swift_score |
SWIFT score (Wasserstein distance on SHAP-transformed distributions).
TYPE:
|
wasserstein_order |
Which Wasserstein order was used (W1 or W2).
TYPE:
|
p_value |
p-value from permutation or bootstrap test (None if not computed).
TYPE:
|
is_drifted |
Whether the feature is flagged as drifted after correction.
TYPE:
|
num_buckets |
Number of buckets used.
TYPE:
|
SWIFTResult
dataclass
¶
SWIFTResult(feature_results: tuple[FeatureSWIFTResult, ...], swift_max: float, swift_mean: float, swift_weighted: float | None = None, alpha: float = 0.05, correction_method: CorrectionMethod | None = None, max_feature: str | None = None)
Aggregate SWIFT result across all features.
| ATTRIBUTE | DESCRIPTION |
|---|---|
feature_results |
Per-feature results.
TYPE:
|
swift_max |
Maximum SWIFT score across features.
TYPE:
|
swift_mean |
Mean SWIFT score across features.
TYPE:
|
swift_weighted |
Importance-weighted SWIFT score (optional).
TYPE:
|
alpha |
Significance level used for testing.
TYPE:
|
correction_method |
Multiple testing correction method used.
TYPE:
|
max_feature |
Name of the feature with the highest SWIFT score.
TYPE:
|
Extraction¶
Stage 1 — decision point extraction from trained tree models.
extraction
¶
Stage 1: Decision point extraction from trained models.
Extracts the set of unique split thresholds (decision points) per feature from a trained model. These decision points define the bucket boundaries for the SWIFT pipeline.
Supported model types: - LightGBM Booster (extract_decision_points_lgb) - XGBoost Booster (extract_decision_points_xgb)
extract_decision_points
¶
extract_decision_points(model: object, feature_names: Sequence[str]) -> dict[str, ndarray]
Auto-dispatch extraction based on model type.
| PARAMETER | DESCRIPTION |
|---|---|
model
|
A trained LightGBM Booster or XGBoost Booster.
TYPE:
|
feature_names
|
List of feature names (must match model's feature order).
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
dict[str, ndarray]
|
Dict mapping feature name -> sorted 1-D array of unique split thresholds. |
| RAISES | DESCRIPTION |
|---|---|
TypeError
|
If the model type is not supported. |
Source code in src/swift/extraction.py
24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 | |
extract_decision_points_lgb
¶
extract_decision_points_lgb(model: Booster, feature_names: Sequence[str]) -> dict[str, ndarray]
Extract unique, sorted split thresholds per feature from a LightGBM Booster.
For each feature, collects every split threshold used across all trees in the ensemble, deduplicates, and returns them in ascending order.
| PARAMETER | DESCRIPTION |
|---|---|
model
|
A trained LightGBM Booster.
TYPE:
|
feature_names
|
List of feature names (must match model's feature order).
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
dict[str, ndarray]
|
Dict mapping feature name -> sorted 1-D array of unique split thresholds. Features never used in any split get an empty array. |
Source code in src/swift/extraction.py
65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 | |
extract_decision_points_xgb
¶
extract_decision_points_xgb(model: object, feature_names: Sequence[str]) -> dict[str, ndarray]
Extract unique, sorted split thresholds per feature from an XGBoost Booster.
Uses get_dump(dump_format='json') to obtain JSON-serialised trees,
then recursively collects every numeric split threshold per feature.
| PARAMETER | DESCRIPTION |
|---|---|
model
|
A trained
TYPE:
|
feature_names
|
List of feature names (must match model's feature order).
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
dict[str, ndarray]
|
Dict mapping feature name -> sorted 1-D array of unique split thresholds. Features never used in any split get an empty array. |
Source code in src/swift/extraction.py
119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 | |
Bucketing¶
Stage 2 — bucket construction from decision points.
bucketing
¶
Stage 2: Bucket construction from decision points.
Given sorted, unique decision points (split thresholds) per feature, constructs a BucketSet with: - A null bucket for missing values (index 0) - Numeric buckets: (-inf, t1), [t1, t2), ..., [tm, inf)
build_buckets
¶
build_buckets(decision_points: ndarray, feature_name: str) -> BucketSet
Construct a BucketSet from sorted decision points for a single feature.
| PARAMETER | DESCRIPTION |
|---|---|
decision_points
|
Sorted 1-D array of unique split thresholds.
TYPE:
|
feature_name
|
Name of the feature.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
BucketSet
|
A BucketSet containing:
Total: len(decision_points) + 2 buckets. |
Source code in src/swift/bucketing.py
20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 | |
build_all_buckets
¶
build_all_buckets(decision_points_dict: dict[str, ndarray]) -> dict[str, BucketSet]
Construct BucketSets for all features.
| PARAMETER | DESCRIPTION |
|---|---|
decision_points_dict
|
Dict mapping feature name -> sorted decision points.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
dict[str, BucketSet]
|
Dict mapping feature name -> BucketSet. |
Source code in src/swift/bucketing.py
115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 | |
Normalization¶
Stage 3 — bucket-level mean SHAP and the feature transformation.
normalization
¶
Stage 3: SHAP normalization — bucket-level mean SHAP and feature transformation.
Computes the mean SHAP value per bucket on the reference sample, then provides a transformation function that maps any feature value to its bucket's mean SHAP (the "SHAP normalization" step).
For empty buckets (no reference observations): creates synthetic observations by sampling real rows and placing the feature value inside the empty bucket, then computes SHAP on those synthetic observations.
compute_bucket_shap
¶
compute_bucket_shap(bucket_sets: dict[str, BucketSet], X_ref: DataFrame, shap_values: ndarray, model: object | None = None, n_synthetic: int = 10, rng: Generator | None = None) -> dict[str, BucketSet]
Compute mean SHAP per bucket for all features.
For each feature j and bucket k, computes: mean_shap_j^k = mean(shap_j(x_i) for all i where x_ij in bucket k)
If a bucket has zero observations in X_ref: - If model is provided: create n_synthetic observations by sampling real rows and setting the feature value to fall in the empty bucket, then compute SHAP on those synthetic observations. - If model is None: assign mean_shap = 0.0 with a warning.
| PARAMETER | DESCRIPTION |
|---|---|
bucket_sets
|
Dict of feature_name -> BucketSet (from build_all_buckets).
TYPE:
|
X_ref
|
Reference DataFrame (n_ref x p).
TYPE:
|
shap_values
|
SHAP values array of shape (n_ref, p).
TYPE:
|
model
|
Trained model for computing SHAP on synthetic observations. If None, empty buckets get mean_shap = 0.0.
TYPE:
|
n_synthetic
|
Number of synthetic observations to create for empty buckets.
TYPE:
|
rng
|
Random number generator for reproducibility.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
dict[str, BucketSet]
|
Dict of feature_name -> BucketSet with mean_shap populated on each Bucket. |
Source code in src/swift/normalization.py
25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 | |
assign_bucket_indices
¶
assign_bucket_indices(values: ndarray | Series, bucket_set: BucketSet) -> ndarray
Assign a bucket index to every value, vectorized.
Bucket layout: null bucket (index 0), then numeric buckets in interval order: bucket 1: (-inf, dp[0]) bucket 2: [dp[0], dp[1]) ... bucket k+1: [dp[k-1], inf)
np.searchsorted(dp, val, side='right') gives the number of decision points <= val, which maps to bucket index offset by 1 (since bucket 0 is null). NaN values map to the null bucket in O(n log k) time.
| PARAMETER | DESCRIPTION |
|---|---|
values
|
1-D array or Series of feature values (may contain NaN).
TYPE:
|
bucket_set
|
BucketSet with sorted decision_points.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
ndarray
|
1-D integer array of bucket indices (same length as input). |
Source code in src/swift/normalization.py
103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 | |
transform_feature
¶
transform_feature(values: ndarray | Series, bucket_set: BucketSet) -> ndarray
Map feature values to their bucket's mean SHAP value.
This is the SHAP transformation sigma_j defined in the paper: sigma_j(x_ij) = mean_shap_j^{bucket(x_ij)}
| PARAMETER | DESCRIPTION |
|---|---|
values
|
1-D array or Series of feature values (may contain NaN).
TYPE:
|
bucket_set
|
BucketSet with mean_shap populated on each bucket.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
ndarray
|
1-D numpy array of transformed values (same length as input). |
Source code in src/swift/normalization.py
166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 | |
transform_frame
¶
transform_frame(X: DataFrame, bucket_sets: dict[str, BucketSet]) -> dict[str, ndarray]
Apply transform_feature to every feature in bucket_sets.
| PARAMETER | DESCRIPTION |
|---|---|
X
|
DataFrame containing at least the features in bucket_sets.
TYPE:
|
bucket_sets
|
Dict of feature_name -> fitted BucketSet.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
dict[str, ndarray]
|
Dict of feature_name -> 1-D array of transformed values. |
Source code in src/swift/normalization.py
196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 | |
Distance¶
Stage 4 — Wasserstein distance on SHAP-transformed distributions.
distance
¶
Stage 4: Wasserstein distance on SHAP-transformed distributions.
Computes the Wasserstein distance (W₁ or W₂) between two 1-D empirical distributions of SHAP-transformed feature values. The transformation σ_j is computed ONCE from D_ref (Stage 3) and applied identically to both reference and monitoring samples.
Functions: wasserstein_1d: W_p distance between two 1-D arrays. compute_swift_scores: Per-feature SWIFT scores for ref vs mon.
wasserstein_1d
¶
wasserstein_1d(u: ndarray, v: ndarray, order: int = 1) -> float
Compute the p-th Wasserstein distance between two 1-D empirical distributions.
For order=1 this is the Earth Mover's Distance (L₁ area between CDFs). For order=2 this is the root of the integral of squared CDF differences.
Both are computed via the sorted-quantile formula: W_p^p = (1/N) Σ |F⁻¹_u(i/N) - F⁻¹_v(i/N)|^p with the step empirical quantile functions evaluated on the merged grid of both samples' CDF levels (grid spacings as weights), so that unequal sample sizes are handled correctly.
| PARAMETER | DESCRIPTION |
|---|---|
u
|
1-D array of samples from distribution P.
TYPE:
|
v
|
1-D array of samples from distribution Q.
TYPE:
|
order
|
Wasserstein order (1 or 2).
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
float
|
Non-negative float W_p(P, Q). |
| RAISES | DESCRIPTION |
|---|---|
ValueError
|
If order is not 1 or 2, or arrays are empty. |
Source code in src/swift/distance.py
26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 | |
compute_swift_scores
¶
compute_swift_scores(X_ref: DataFrame, X_mon: DataFrame, bucket_sets: dict[str, BucketSet], order: int = 1) -> dict[str, float]
Compute per-feature SWIFT scores: W_p on SHAP-transformed distributions.
For each feature j: 1. Apply σ_j (from bucket_sets) to both ref and mon columns. 2. Compute W_p between the two transformed arrays.
The transformation σ_j was fitted on D_ref (Stage 3) and is not recomputed here — it is applied identically to both samples.
| PARAMETER | DESCRIPTION |
|---|---|
X_ref
|
Reference DataFrame (n_ref × p).
TYPE:
|
X_mon
|
Monitoring DataFrame (n_mon × p).
TYPE:
|
bucket_sets
|
Dict of feature_name → BucketSet with mean_shap already computed (output of compute_bucket_shap).
TYPE:
|
order
|
Wasserstein order (1 or 2).
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
dict[str, float]
|
Dict of feature_name → SWIFT score (non-negative float). |
Source code in src/swift/distance.py
108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 | |
Threshold¶
Stage 5 — permutation test, bootstrap thresholds, and multiple testing correction.
threshold
¶
Stage 5: Threshold calibration — permutation test, bootstrap, multiple testing correction.
Provides three functions: permutation_test: Per-feature p-values via permutation test. bootstrap_threshold: Per-feature absolute thresholds via bootstrap. correct_pvalues: Multiple testing correction (Bonferroni / BH).
Key invariant: the SHAP transformation σ_j is computed ONCE from D_ref (Stage 3) and reused for every permutation / bootstrap draw. SHAP is never recomputed inside the permutation loop.
permutation_test
¶
permutation_test(X_ref: DataFrame, X_mon: DataFrame, bucket_sets: dict[str, BucketSet], order: int = 1, n_permutations: int = 1000, max_samples: int | None = None, rng: Generator | None = None) -> dict[str, float]
Compute per-feature p-values via permutation test.
Under H₀ (no drift), reference and monitoring samples are exchangeable. We pool them, draw random splits, and compute SWIFT scores under the null.
The pre-computed SHAP transformation σ_j (stored in bucket_sets) is applied identically to every permutation — it is NOT recomputed.
The p-value uses the conservative formula: p_j = (1 + #{b : SWIFT_j^(b) ≥ SWIFT_j^obs}) / (1 + B)
When max_samples is set and the pooled data exceeds it, both
reference and monitoring data are randomly subsampled (preserving
the ref/mon ratio) before running the permutation loop. This
provides a significant speedup on large datasets with negligible
impact on statistical power.
| PARAMETER | DESCRIPTION |
|---|---|
X_ref
|
Reference DataFrame (n_ref × p).
TYPE:
|
X_mon
|
Monitoring DataFrame (n_mon × p).
TYPE:
|
bucket_sets
|
Dict of feature → BucketSet with mean_shap populated.
TYPE:
|
order
|
Wasserstein order (1 or 2).
TYPE:
|
n_permutations
|
Number of permutations B.
TYPE:
|
max_samples
|
Maximum total pool size. If the pool (n_ref + n_mon) exceeds this, subsample proportionally. None = no limit.
TYPE:
|
rng
|
Random number generator for reproducibility.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
dict[str, float]
|
Dict of feature_name → p-value ∈ (0, 1]. |
Source code in src/swift/threshold.py
27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 | |
bootstrap_threshold
¶
bootstrap_threshold(X_ref: DataFrame, bucket_sets: dict[str, BucketSet], n_mon: int, order: int = 1, alpha: float = 0.05, n_bootstrap: int = 1000, rng: Generator | None = None) -> dict[str, float]
Compute per-feature absolute thresholds via bootstrap.
Draws bootstrap samples of size n_mon from X_ref, computes SWIFT scores against X_ref, and returns the (1 − α) quantile as the threshold for each feature.
This provides a principled alternative to PSI's ad-hoc 0.10 / 0.25.
| PARAMETER | DESCRIPTION |
|---|---|
X_ref
|
Reference DataFrame (n_ref × p).
TYPE:
|
bucket_sets
|
Dict of feature → BucketSet with mean_shap populated.
TYPE:
|
n_mon
|
Size of monitoring sample (bootstrap sample size).
TYPE:
|
order
|
Wasserstein order (1 or 2).
TYPE:
|
alpha
|
Significance level (threshold is (1-α) quantile).
TYPE:
|
n_bootstrap
|
Number of bootstrap iterations B.
TYPE:
|
rng
|
Random number generator for reproducibility.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
dict[str, float]
|
Dict of feature_name → threshold (non-negative float). |
Source code in src/swift/threshold.py
149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 | |
correct_pvalues
¶
correct_pvalues(pvalues: dict[str, float], method: CorrectionMethod, alpha: float = 0.05) -> dict[str, bool]
Apply multiple testing correction and return rejection decisions.
Bonferroni: reject if p_j < α / p (controls FWER). Benjamini-Hochberg: sort p-values, find largest k s.t. p_(k) ≤ k·α / p, then reject all with rank ≤ k (controls FDR).
Bonferroni uses strict inequality (p < threshold); BH uses p ≤ threshold.
| PARAMETER | DESCRIPTION |
|---|---|
pvalues
|
Dict of feature_name → p-value.
TYPE:
|
method
|
CorrectionMethod.BONFERRONI or CorrectionMethod.BH.
TYPE:
|
alpha
|
Significance level.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
dict[str, bool]
|
Dict of feature_name → bool (True = reject H₀ / flagged as drifted). |
Source code in src/swift/threshold.py
226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 | |
Aggregation¶
Model-level score aggregation (SWIFT_max, SWIFT_mean, SWIFT_weighted).
aggregation
¶
Stage 6: Aggregation — feature-level to model-level SWIFT scores.
Provides three aggregation strategies from the paper (§3.6): - Maximum SWIFT: most-drifted feature (SWIFT_max) - Mean SWIFT: average feature drift (SWIFT_mean) - Weighted SWIFT: importance-weighted (SWIFT_weighted)
Functions: aggregate_scores: Compute model-level summary statistics. compute_importance_weights: Normalized mean |SHAP| weights w_j.
AggregatedScores
dataclass
¶
AggregatedScores(swift_max: float, swift_mean: float, max_feature: str, swift_weighted: float | None = None)
Model-level SWIFT aggregation.
| ATTRIBUTE | DESCRIPTION |
|---|---|
swift_max |
Maximum per-feature SWIFT score.
TYPE:
|
swift_mean |
Unweighted mean of per-feature scores.
TYPE:
|
swift_weighted |
Importance-weighted score (None if no weights).
TYPE:
|
max_feature |
Name of the feature with the highest score.
TYPE:
|
aggregate_scores
¶
aggregate_scores(scores: dict[str, float], weights: dict[str, float] | None = None) -> AggregatedScores
Aggregate per-feature SWIFT scores to model level.
| PARAMETER | DESCRIPTION |
|---|---|
scores
|
Dict of feature_name → SWIFT score.
TYPE:
|
weights
|
Optional dict of feature_name → importance weight. If provided, weights should sum to 1 (or be normalizable). If None, swift_weighted is set to None.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
AggregatedScores
|
AggregatedScores with max, mean, and optionally weighted score. |
Source code in src/swift/aggregation.py
45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 | |
compute_importance_weights
¶
compute_importance_weights(shap_values: ndarray, feature_names: list[str]) -> dict[str, float]
Compute normalized mean absolute SHAP importance weights.
w_j = |φ̄_j| / Σ_k |φ̄_k|
where φ̄_j = mean(|SHAP_j(x_i)|) across all reference observations.
This generalizes SSI's IV-based weights with model-aware SHAP-based feature importance.
| PARAMETER | DESCRIPTION |
|---|---|
shap_values
|
SHAP values array of shape (n, p).
TYPE:
|
feature_names
|
List of p feature names.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
dict[str, float]
|
Dict of feature_name → weight (non-negative, sums to 1). |
Source code in src/swift/aggregation.py
88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 | |
Plotting¶
Visualization functions behind plot_buckets and plot_swift_scores.
plotting
¶
Visualization utilities for SWIFT monitoring results.
Provides two public plotting functions:
plot_bucket_profile— SHAP response curve + density per bucket for a feature.plot_feature_swift_scores— Bar chart of SWIFT scores per feature.
Both return (Figure, Axes) tuples for further customization.
plot_bucket_profile
¶
plot_bucket_profile(bucket_set: BucketSet, feature_values: ndarray, shap_values: ndarray, compare_values: ndarray | None = None, primary_values: ndarray | None = None, labels: tuple[str, str] = ('Reference', 'Comparison'), figsize: tuple[float, float] = (10, 5), title: str | None = None, max_label_buckets: int = 20, x_axis: str = 'bucket') -> tuple[Figure, Axes]
Plot the bucketing profile for a single feature.
Shows mean SHAP per bucket (line + error band) and observation density (filled line). Optionally overlays a comparison sample's density.
| PARAMETER | DESCRIPTION |
|---|---|
bucket_set
|
Fitted bucket set for the feature.
TYPE:
|
feature_values
|
Raw feature values from the reference sample (used for the SHAP
curve and, when primary_values is
TYPE:
|
shap_values
|
SHAP values for this feature on the reference sample.
TYPE:
|
compare_values
|
Raw feature values from a comparison sample. If provided, shows a second density line.
TYPE:
|
primary_values
|
If provided, these values are used for the primary density instead of feature_values. Useful for showing density of an arbitrary sample while the SHAP curve stays anchored to the reference.
TYPE:
|
labels
|
Legend labels for (primary, comparison) densities.
TYPE:
|
figsize
|
Figure size.
TYPE:
|
title
|
Custom title. Defaults to
TYPE:
|
max_label_buckets
|
Switch from interval notation to bucket indices if exceeded.
TYPE:
|
x_axis
|
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
(Figure, Axes)
|
|
Source code in src/swift/plotting/bucket_profile.py
26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 | |
plot_feature_swift_scores
¶
plot_feature_swift_scores(result: SWIFTResult, result_compare: SWIFTResult | None = None, labels: tuple[str, str] = ('Result A', 'Result B'), threshold: float | None = None, sort_by: str = 'score', feature_order: list[str] | None = None, figsize: tuple[float, float] = (12, 5), title: str | None = None) -> tuple[Figure, Axes]
Plot SWIFT scores per feature as a bar chart with reference lines.
Optionally compare two SWIFTResult objects side by side.
| PARAMETER | DESCRIPTION |
|---|---|
result
|
Primary result from
TYPE:
|
result_compare
|
Optional second result for side-by-side comparison.
TYPE:
|
labels
|
Legend labels for the two results in comparison mode.
TYPE:
|
threshold
|
Optional detection threshold horizontal line.
TYPE:
|
sort_by
|
Feature ordering on x-axis.
TYPE:
|
feature_order
|
Original feature order (from
TYPE:
|
figsize
|
Figure size.
TYPE:
|
title
|
Custom title.
TYPE:
|
| RETURNS | DESCRIPTION |
|---|---|
(Figure, Axes)
|
|
Source code in src/swift/plotting/swift_scores.py
19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 | |