End of the evaluation
The end review compares human and system decisions, tests bias added by mitigation and optimization, and checks the UI and UX against every stakeholder attribute.
What clause 9.2.4.4 asks
The end of the design and output evaluation has four actions. Where appropriate, compare the decisions made only by people with the decisions made by the system, and an expert panel can be used. Examine the bias that mitigation procedures add, including each trade-off where reducing one bias adds another, and record each new bias that appears after the procedures. Examine the bias that computational performance optimization adds by applying the metrics before and after the optimization and comparing the results. Compare the UI and UX design with the stakeholder identification, and require the design to reflect the attributes of all identified stakeholders, including a control that does not list all the attributes and a design that is not accessible to persons with disabilities.
Why it matters
The end review catches the biases that the team's own remedies create. A mitigation that improves one group's outcome can worsen another's. An optimization that improves latency can change the ranking. A user interface that omits an attribute can force people into the wrong category. These are not edge cases; they are the predictable side effects of changing a system. The clause asks the team to look for them deliberately and to record the trade-off.
What good looks like
Run the end review as a separate session with the design, evaluation and accessibility leads. Compare the human and system decisions on a sample, and record the disagreements. Apply the metrics before and after each optimization and record the difference. Walk through the UI and UX against the stakeholder attribute list, control by control, and record every omission and accessibility barrier. Then decide whether the trade-off is acceptable and who accepts it.
Evidence an assessor asks for
The assessor will ask what the mitigation made worse. A team that reports only improvements has probably not looked.
- Human comparison The sample, the human decisions and the system decisions, with the disagreements
- Mitigation trade-off record Each mitigation, the bias it reduced, the bias it added and the decision
- Optimization comparison The metrics before and after each optimization, by group
- UI and UX review Every stakeholder attribute checked against the design, with accessibility barriers recorded
Where it commonly fails
- The end review is a demo, not a comparison
- The optimization is not tested for bias
- The UI review checks colours but not whether the attributes can be entered
What to do before publication
Schedule the end review after the mitigation and before release. It is the last chance to see a side effect.
Read across
- 6.3 Outputs
- 9.2.3 Evaluation of design and outputs: outputs
- 9.2.4 Evaluation of design and outputs: actions
- 9.2.5 Evaluation of design and outputs: outcomes
← 9.2.4.3 Testing9.2.5 Evaluation of design and outputs: outcomes →