Set up the ongoing procedure
The procedure lists what to monitor, how each item is evaluated, the rationale for recurrence, a post-mitigation cycle and stakeholder feedback.
What clause 9.3.4.1 asks
Setting up the procedure starts with a list of what to watch. The document's examples include the data, the system's decisions, the effect on people, risks and impacts, the tendency to accept a system's answer without question, feedback loops in recommendation, the interface, culture, and the effect of deployment on bias in the world. For each item the team writes how it will be checked, using the way the system runs to decide what fits, and reviews the earlier evaluation. It gives a reason for how often the check repeats, by time or by a signal such as volume, throughput, alarms or unusual conditions. A treatment in one part of the system calls for one more cycle. Where it fits, the team gathers feedback from affected people and uses it.
Why it matters
The list is deliberately wider than a model dashboard. Complacency bias, feedback loops and culture are system-level effects that no accuracy metric will catch. The post-mitigation cycle is the same lesson as the end review in clause 9.2.4.4: a change made to reduce bias can create another. The stakeholder feedback action is a correction to the risk that the team monitors only what it already measures.
What good looks like
Build the list with the operations team, the user researcher and the risk lead. For each item, write the procedure and the indicator. Set the recurrence with a reason. Add the post-mitigation cycle as a standing rule. Then design the feedback route, including who reads it, how it is verified and how it reaches the evaluation.
Evidence an assessor asks for
The assessor will look for at least one item that is not a model metric. A list that contains only accuracy and latency has not used the clause's examples.
- Monitoring list The items, procedures and indicators, including social and cultural items
- Recurrence rationale The interval or indicator and the reason
- Post-mitigation rule The extra cycle after a mitigation and who triggers it
- Feedback route The collection, verification and use of impacted stakeholder feedback
Where it commonly fails
- The list is a copy of the model dashboard
- There is no post-mitigation cycle
- Stakeholder feedback is collected but never used
What to do before publication
Draft the list with people who see the system in operation, not only the model team.
Read across
- 9.3.1 Ongoing evaluation: purpose
- 9.3.4.2 Each iteration of the ongoing evaluation
- 9.3.4.3 End of each iteration
- 9.2.4.4 End of the evaluation
← 9.3.3 Ongoing evaluation: outputs9.3.4.2 Each iteration of the ongoing evaluation →