Fraud detection platform on IEEE-CIS, stage 9 of 10

Two clocks and a gate

A fraud model fails in two ways on two clocks. Its inputs can shift the day a batch closes, before a single label exists; its accuracy can only be measured once the chargebacks arrive, weeks later. This page replays one recorded run of the platform's lifecycle: a drift job that reads no labels, a decay job that waits for them, a retrain trigger that reads both, a challenger fitted through the same leakage-safe pipeline, a promotion gate with a margin derived from measured noise, and a rollback. Every number on the page is read from the committed artifact of one seed-pinned run.

The run, one cycle a week

Fourteen cycles over a stream of real consecutive weeks. From day 141 every raw row carries a change the model never saw. Step through, or press play. The charts reveal what each job had seen by that cycle; the console shows what each job wrote.

  • score PSI against the champion's training window
  • score alert edge, shipped model
  • matured batch, PR-AUC with its 95% interval
  • decay: the drop clears the tolerance
  • the champion's reference
  • reference less the tolerance
  • promoted
  • refused
  • retrain called, no challenger fitted

The gate, four times

A challenger is promoted when its paired advantage on the gate window clears the margin and the interval lies above zero. Both refusals and both promotions are in the record, with the reason the gate wrote.

Set before the run

The edges and margins are derived from measured quantities in earlier stages; the clocks and the injected shift are choices, labelled as such in the artifact.

What kind of drift this file has on its own

Before the injection, one classifier was told to separate the training window from the test window on the shipped model's input columns. Which columns it leans on says what kind of shift the file carries.

What this run does not establish

    Where every number comes from

    Each artifact is committed with the command that regenerates it. The page is built from them by one script and a test compares the committed page with a fresh build.