Appendix 3: Equality and fairness measures in classification systems

Classification models are often evaluated using a set of standard metrics derived from the confusion matrix, which is a table that compares the predictions of the classification model against the actual outcomes to measure accuracy and other related performance indicators. Some of these metrics may apply to generative AI models, however, this section is specific to classification problems.

Indicators include:

  • Prevalence: The proportion of actual positive cases in each group.
  • Predicted prevalence: The proportion of positive predictions in each group.
  • True positive rate (TPR): The proportion of actual positive cases correctly identified by the model.
  • False positive rate (FPR): The proportion of negative cases incorrectly classified as positive.
  • Precision (positive predictive value): The proportion of positive predictions that are correct.
  • Negative predictive value (NPV): The proportion of negative predictions that are correct.

Fairness can be assessed in several ways:

  • Disparate treatment or equality of opportunity: Requires that groups have similar true positive rates, focusing on equal chances for correct positive outcomes.
  • Disparate impact or distributive justice: Looks at precision, aiming for similar accuracy of positive predictions across groups.
  • Statistical parity (demographic parity): Demands that the proportion of positive predictions is the same for all groups, regardless of differences in actual outcomes.

Other group fairness metrics include:

  • Equalised odds: Requires both TPR and FPR to be equal across groups, or that false alarms and correct identifications occur at similar rates for all groups.
  • Sufficiency (predictive rate parity): Seeks equal precision and negative predictive value for all groups. For example, when the model predicts reoffending, the likelihood of being correct is the same across groups.

It is important to note that, in practice, no imperfect model can satisfy all fairness criteria at once, especially when the underlying rates of positive outcomes differ between groups. Auditors should consider these limitations and focus on the practical impact of any disparities.

Other fairness concepts include:

  • Fairness through unawareness: Simply omitting sensitive attributes does not guarantee fairness, as other variables may be correlated.
  • Individual fairness: Similar individuals should receive similar predictions, though this is often difficult to measure.
  • Counterfactual fairness: The model’s prediction should not change if a sensitive attribute is altered, holding all else equal.

A good overview of these and more fairness concepts are given in the literature.68 It is important to note that the list of metrics included here is not exhaustive and are included as a starting point for the auditor.

  1. See for example: S. Verma and J. Rubin (2018): Fairness Definitions Explained.↩︎