4 Audit Catalogue
The audit catalogue described in this chapter consists of guidelines for auditing AI systems, including both the suggested content of audit topics based on risks, as well as expected controls or mitigations that the auditor might expect to see.
AI systems tend to be embedded in a wider IT infrastructure. While AI applications may present challenges that differ from those associated with traditional software or IT systems, the audit of AI systems and machine learning models can still be structured using frameworks familiar to IT auditors—such as the Cross-Industry Standard Process for Data Mining (CRISP-DM).56 This paper focuses on issues specific to AI applications, with a particular focus on the safe, responsible and ethical use of AI. Key considerations include transparency and explainability of an AI system’s decisions, the equality and fairness of these decisions, and system security. These aspects should be integrated throughout the AI development process according to the principles of fairness by design, transparency by design and privacy by design. Accordingly, this audit catalogue is not intended as a linear sequence of single steps. Instead, it represents a continuous loop of interacting stages.
See appendix 1 for a description of AI audit structure along the CRISP-DM framework.↩︎