Evaluation of design and outputs: outputs
The outputs are an updated profile and an evaluation document with nine required items, from metrics and scenarios to mitigation and optimization results.
What clause 9.2.3 asks
Two things come out: a new profile version and an evaluation report. The report covers nine points: the measures chosen; the test situations; what the tests produced; an assessment of those results with the method and the reason for it; how the findings relate to the business requirement; a comparison with an expert panel where that applies; bias in the interface and experience design and its effect on data bias where that applies; the biases examined and, where possible, where they came from; and what happened when treatments and performance tuning were tested.
Why it matters
The nine items are a completeness test for the evaluation report. Most reports cover the metrics and the test outputs. Fewer cover the relation to the business requirements, the expert panel, the UI and UX, the sources of the biases, and the effect of optimization. The last item is particularly important: a performance optimization can reintroduce bias that a mitigation had removed, and the report has to show that the interaction was tested.
What good looks like
Use the nine items as the report template. For each item, write the answer or record why it does not apply. Keep the evidence links beside the item, not in an appendix. Where an expert panel was used, record its composition and the comparison method. Where an optimization was applied, record the metrics before and after and the effect on the bias findings.
Evidence an assessor asks for
The assessor will pick one of the nine items at random. A report that has six of nine is a partial evaluation, however good the six are.
- Evaluation report The nine items, each answered or marked not applicable with a reason
- Metric evidence The metrics, scenarios and test outputs, with the raw results
- Expert panel record The panel composition and the comparison method, where used
- Optimization record The metrics before and after optimization and the effect on bias
Where it commonly fails
- The report covers metrics and outputs only
- The optimization effect is not tested
- The expert panel is mentioned but its composition is not recorded
What to do before publication
Adopt the nine-item template before the first evaluation. It shapes the evidence the team collects.
Read across
- 5.2.5 Evaluation in the life cycle
- 9.2.2 Evaluation of design and outputs: inputs
- 9.2.4 Evaluation of design and outputs: actions
- 9.2.5 Evaluation of design and outputs: outcomes
- D Measuring Bias
← 9.2.2 Evaluation of design and outputs: inputs9.2.4 Evaluation of design and outputs: actions →