Skip to content

Missing values

SWIFT never imputes and never drops rows. Missing values get their own bucket, their own SHAP value, and therefore their own voice in the drift comparison — a shift in how often a feature is missing is detected exactly like a shift in its observed values.

The null bucket

Every feature's bucket set reserves index 0 for the null bucket (buckets). Assignment is by pd.isna: a NaN lands in bucket 0, an observed value is routed by np.searchsorted over the decision points. This happens identically during fit() (on the reference sample) and during test() (on monitoring data), so a monitoring batch with NaN needs no preprocessing.

What a missing value is worth

During normalization, the null bucket gets its mean SHAP like any other bucket: the average SHAP value over the reference rows where the feature is missing. That number is what every NaN is transformed into —

\[ \sigma_j(\text{NaN}) \;=\; \overline{\phi_j}\bigl(\{\,i : x_{ij}\ \text{missing}\,\}\bigr) \]

If the model treats missingness as informative (tree libraries route NaN through dedicated default branches), the null bucket's mean SHAP will sit far from the observed buckets' values, and missingness shifts will move the transformed distribution accordingly. If missingness is neutral to the model, the null bucket's SHAP is close to zero and missingness churn is — correctly — muted.

When the reference has no missing values

If no reference row is missing the feature, the null bucket is empty and its mean SHAP is produced by the synthetic-fill mechanism: n_synthetic (default 10) reference rows are resampled with replacement, the feature is set to NaN in those copies, and shap.TreeExplainer scores them; the null bucket gets the mean of those SHAP values. Without a model to score with, the fill degrades to 0.0 with a logged warning. This means SWIFT can still price a first appearance of missingness in production, even when the reference sample was fully observed.

What this buys in monitoring

Because NaN maps to a real SHAP value, the SWIFT score sees a missingness surge as probability mass moving to the null bucket's SHAP position, and the permutation test judges it with the same machinery as any other shift. There is no separate missingness alarm to configure — and no way for a silent upstream-ETL change that floods a feature with NaN to hide from the score, as long as the model itself reacts to that feature's missingness.