Each iteration of the ongoing evaluation
Each iteration checks data drift, concept drift and other drift, records the plan for each and watches for reinforcement of external bias.
What clause 9.3.4.2 asks
Every cycle looks for three kinds of drift and records a plan for each, even if the plan is to do nothing. Data drift, also called feature, population or covariate drift, is a change in live data that lowers accuracy; the response may be retraining, a change to a business or bias measure, or a change to how often the model is retrained and checked. Concept drift is a move in the decision boundary that lowers accuracy even when the data is stable; the response may be retraining, a change to the design measures or a different algorithm. Other drift is identified and watched, with a plan to deal with the bias. Where there is evidence that bias in the world is being reinforced, that loop is watched too.
Why it matters
The three drift types need different tests and can produce different responses. A team that monitors only data drift will miss a concept change, and a team that retrains on new data will not fix a decision boundary that has moved for social reasons. The reinforcement loop is the most serious: the system's outputs change the world, and the changed world becomes the next input. That is a bias amplifier, and it needs its own monitor.
What good looks like
Run the three checks each iteration and record the result for each. For data drift, compare the production distribution with the training distribution. For concept drift, compare the input-output relation with the relation at training. For other drift, use the list from the set-up action. Record the plan for each finding, including a decision to take no action. Then check the reinforcement loop against the external indicators named in the plan.
Evidence an assessor asks for
The assessor will ask which drift type a finding belongs to and why the response fits. A retraining response to a concept drift can make the problem worse.
- Data drift check The production and training distributions, with the test and the result
- Concept drift check The input-output relation, with the test and the result
- Other drift record The other types monitored, the result and the plan
- Reinforcement check The external indicators and the evidence of a feedback loop
Where it commonly fails
- Only data drift is monitored
- A drift is recorded with no plan and no decision
- The reinforcement loop is assumed to be someone else's problem
What to do before publication
Use a different test for each drift type and record the response separately.
Read across
← 9.3.4.1 Set up the ongoing procedure9.3.4.3 End of each iteration →