Loading exercise…
Feature drift monitor
Build the drift check behind an ML monitoring dashboard: compare live traffic with the training sample, feature by feature.
PythonTask
Build a feature drift monitor that compares live model inputs with the training sample and flags the features that have shifted.
Requirements
- features maps each feature name to "numeric" or "categorical"; anything else, or an empty mapping, raises ValueError containing feature. reference and live are nonempty lists of row objects (ValueError containing sample otherwise). Every row must have every feature (ValueError containing feature). Other keys are ignored.
- Numeric values must be finite numbers, not booleans (ValueError containing numeric). Bin them with edges from the REFERENCE sample only: sort it, take the values at positions n*1//4, n*2//4 and n*3//4, and drop duplicate edges. A value goes in the bin numbered by how many edges are at or below it, so values outside the reference range fall in the first or last bin.
- Categorical values must be strings (ValueError containing categorical). The bins are the reference categories plus one bin for every live category the reference never saw.
- PSI = sum over bins of (live% - ref%) * ln(live% / ref%), where each % is the bin count divided by that sample's size and a 0% is replaced by 0.0001. Round PSI to four decimals; status is "stable" below 0.1, "watch" below 0.25, otherwise "drift". Return [{feature, psi, status}] sorted by psi descending, then feature ascending. Do not mutate input.