ISACA Member Tutorial · 2026

AI-Assisted
Auditing:
A Complete Field Guide

Everything an information systems auditor needs to understand, deploy, govern, and critically evaluate artificial intelligence in the modern audit function. Prepared by Edzai Zvobwo, AI Scientist & CEO of , for the ISACA SA Johannesburg Chapter.

12
Core Topics
6
Panel Questions Answered
100%
AI Population Coverage vs 1–5% Sampling
∞
Human Judgement Required
Scroll to explore
Foundational Principle

The Causal Inference Frame

Before exploring any specific AI application, every auditor must internalise one governing idea that shapes everything else in this guide.

Audit is fundamentally a causal inference problem. Auditors ask: What caused the anomaly? What control failed? What counterfactual should have happened? AI dramatically increases the ability to detect signals — but human auditors remain solely responsible for interpreting cause and making final judgement.
Step 01
Detect Signal

AI scans 100% of population data for statistical anomalies

Step 02
Identify Anomaly

Pattern recognition flags deviations from expected behaviour

Step 03
Root Cause

Auditor investigates why the anomaly exists

Step 04
Control Failure

Which specific control broke down or was bypassed?

Step 05
Counterfactual

What should have happened under proper controls?

Step 06
Human Judgement

Auditor forms conclusion — AI never does this alone

💡

Why this matters for ISACA members

ISACA's IS Audit Standards (specifically Standard 1005 – Due Professional Care) require that auditors apply professional scepticism and independent judgement. No AI system can fulfil this standard on its own. Understanding where the human-machine boundary lies is the most important competency you can develop in the AI era.

Question 01 · Where AI Delivers Value

Tangible Impact in Audit Practice

AI creates value in audit where scale and pattern recognition exceed human capability. The single most transformative shift is moving from statistical sampling to full-population analysis.

1–5%
Data covered by traditional sampling
→
100%
Population coverage with AI monitoring
|
Real-time
From periodic point-in-time to continuous
📊

Anomaly Detection

ML models scan entire transaction ledgers rather than samples. Outliers in vendor payments, expense claims, or journal entries are surfaced instantly and prioritised by risk score.

High Impact Mature Technology
🔄

Continuous Control Monitoring

Instead of periodic audits, AI monitors systems continuously and flags control breaches in real time — transforming audit from retrospective to preventive.

High Impact Strategic Shift
🕸

Graph Fraud Analytics

Graph analytics identifies suspicious relationships between employees, vendors, and bank accounts. Network anomalies often reveal collusion schemes that are impossible to detect through traditional sampling.

Fraud Focus High Impact
🎯

Predictive Risk Scoring

Models estimate the probability that a process, department, or supplier will produce a control failure — enabling dynamic, risk-based audit planning rather than static annual schedules.

Emerging Strategic
📄

Document Intelligence

NLP systems extract structured information from contracts, invoices, emails, and policy documents. This automates evidence gathering and contract compliance review at scale.

Emerging Efficiency Gain
💸

AML & Financial Crime

Banks deploy AI for real-time transaction monitoring under Anti-Money Laundering frameworks — flagging suspicious patterns across millions of transactions that no human team could review manually.

Regulatory High Impact
📌

ISACA Connection: CRISC Domain 2 — IT Risk Assessment

CRISC practitioners should note that AI-based anomaly detection directly enhances IT risk assessment by providing continuous, data-driven risk signals rather than point-in-time snapshots. The predictive risk scoring use case aligns directly with CRISC's risk identification and analysis domain.

Question 02 · Prerequisites

Maturity Required Before Deployment

AI in audit fails when data foundations are weak. The guiding principle: AI amplifies existing system quality. Good systems become powerful. Poor systems become dangerous.

Data Layer
Centralised data warehouse or lake Clean, consistently labelled transactional records Consistent identifiers across all systems Preserved audit trails with timestamps Data lineage documentation Data classification policy
Systems Layer
API or structured access to ERP systems Comprehensive system event logging Model version control Integrated HRMS, CRM, ERP data Secure data pipelines
Governance Layer
Data ownership clearly defined Model validation procedures documented Access controls and segregation of duties Model monitoring and drift detection Model Risk Register maintained Third-party AI vendor risk assessment process
People Layer
AI literacy in audit team Data steward roles defined AI ethics training completed Change management plan in place Executive sponsorship secured

Maturity Benchmark

A useful baseline is CMMI Level 3 or equivalent — processes are defined, documented, and applied consistently across the organisation.

Below Level 3, organisations should invest in data foundations before AI tooling. Deploying AI on immature data produces unreliable, potentially dangerous outputs.

The First-Principles Test

Before any AI audit deployment, ask three questions:

  • Can we trust the data this model will train on?
  • Do we have the governance to respond when the model is wrong?
  • Do our auditors understand the model's limitations?

If any answer is "no", fix that first.

Question 03 · Model Governance

How to Audit the Algorithm Itself

Auditing AI requires treating the model as a control system. The goal is not blind trust in the algorithm — it is explainable, testable machine reasoning.

Layer 01
📐

Logic Review

Document model purpose and assumptions. Confirm inputs are legitimate risk indicators. Validate feature engineering.

Layer 02
📈

Performance Validation

Precision and recall metrics. False positive and negative rates. Back-testing against historical cases.

Layer 03
🌡

Stability Testing

Data drift monitoring. Sensitivity testing. Scenario stress tests. Reproducibility audits.

Layer 04
🗂

Operational Controls

Independent validation. Full model documentation. Audit logs of all model decisions and outputs.

Key Technical Concepts for Auditors

🔬
Explainability: SHAP & LIME
Understanding why a model made a specific decision
+

SHAP (SHapley Additive exPlanations) assigns each input feature a contribution score for each individual prediction. This tells you: "The model flagged this transaction primarily because the vendor was registered 3 days before the payment, and the amount exceeded the usual range for this department."

LIME (Local Interpretable Model-agnostic Explanations) works by slightly perturbing the input and observing how the model's output changes — building a simpler, interpretable model that approximates the black box locally.

⚡

Auditor Takeaway

Both tools allow auditors to interrogate black-box models without needing access to the model's internal weights. Any AI audit tool procured should support SHAP or LIME output to enable explainable findings in working papers.

🎯
Precision, Recall & the False Positive Problem
Why traditional control testing metrics don't apply
+

Precision

Of all the anomalies the model flagged, how many were real?

Low precision = many false positives = auditor workload explodes investigating non-issues.

Recall

Of all the real anomalies that existed, how many did the model catch?

Low recall = missed fraud / control failures = false assurance to management.

The audit profession's tolerance for false positives differs by context. An AML model should favour high recall (catch everything, even with false positives). An internal control model should favour high precision (only flag what's worth investigating).

The F1 Score balances both. Auditors should require vendors to disclose F1 scores on relevant test datasets, not just headline accuracy figures.

📉
Model Drift & Stability
Why a well-performing model can degrade silently over time
+

Data drift occurs when the statistical properties of real-world data change after the model was trained — making the model's learned patterns outdated. Example: a fraud model trained on pre-COVID purchase patterns will perform poorly post-COVID as spending behaviours changed fundamentally.

Concept drift is subtler — the relationship between inputs and the outcome changes. Example: what constitutes a "suspicious" journal entry changes as accounting standards evolve.

⚠️

Audit Risk

An AI audit tool that performed well in Year 1 may silently degrade by Year 3. Auditors must establish model performance monitoring as a control in itself — not assume past performance predicts future reliability.

🗃
The Model Risk Register
Governing AI tools like any other critical control
+

Every AI model used in the audit function should be catalogued in a Model Risk Register containing:

  • Model name and version
  • Business purpose and use case
  • Training data sources and vintage
  • Validation methodology and results
  • Known limitations and failure modes
  • Model owner and approval authority
  • Last validation date and next scheduled review
  • Performance thresholds and alert triggers
  • Third-party vendor details
  • Data access permissions granted
Question 04 · Cybersecurity Risks

AI Expands the Attack Surface

When audit AI tools receive read access to financial systems and sensitive records, they become high-value targets. AI must be treated as critical infrastructure, not just software.

⚠️ Data Exposure
Training datasets used to build AI audit models may contain the entire population of sensitive financial records — far beyond what any human auditor would access. A breach of the training environment exposes everything.
Secure, isolated training environments. Data masking and anonymisation where possible. Least-privilege access for all AI processes.
⚠️ Model Poisoning
An attacker with access to training data can corrupt it specifically to make the model overlook certain transaction types, vendor relationships, or user accounts — creating a blind spot in your audit coverage that persists silently.
Immutable audit trails on training data. Cryptographic signing of approved datasets. Independent validation on separate held-out data.
⚠️ Prompt Injection
LLM-based audit tools that process documents (contracts, invoices, emails) can be manipulated by malicious content embedded in those documents — causing the AI to ignore suspicious elements or produce false assurance.
Input sanitisation pipelines. Restricted output modes. Human review of LLM summaries on high-risk documents. No autonomous LLM conclusions.
⚠️ Model Inversion
Sophisticated adversaries can query a deployed model repeatedly to reconstruct sensitive training data. If payroll or HR records were in the training set, they may be recoverable through model outputs alone.
Rate limiting on model queries. Differential privacy techniques during training. No public-facing API access to audit models.
⚠️ Supply Chain Risk
Third-party AI vendors with weak security postures become an indirect attack vector. A compromised AI vendor with access to your ERP has effective access to your most sensitive financial data.
SOC 2 Type II review of all AI vendors. Contractual data handling obligations. Regular vendor security reassessment. Right-to-audit clauses.
⚠️ Adversarial Inputs
Deliberately crafted transactions designed to fool AI anomaly detectors — exploiting the model's decision boundaries. Sophisticated fraudsters can probe an AI system's blind spots if they know it is deployed.
Keep AI model details confidential internally. Layer AI with traditional controls. Rotate models periodically. Monitor for probing behaviour patterns.
🔐

Zero Trust Principles for AI Audit Systems

Apply zero-trust architecture: no AI tool should have standing access to production data. Connections should be ephemeral, scoped to the minimum data required, logged comprehensively, and revocable in real time. The AI audit tool's access logs should themselves be subject to audit.

📋

ISACA Connection: CISM Domain 2 — Information Risk Management

CISM-certified members are well-positioned to lead AI cybersecurity risk assessments. The threat vectors above map directly to CISM's information risk identification and assessment competencies. AI tools should appear in your organisation's risk register as distinct assets with their own threat models.

Question 05 · Bias Governance

Detecting & Mitigating AI Bias

Bias in audit AI is primarily a data problem. A model trained on historically biased audit outcomes will perpetuate and amplify those biases at scale — affecting which vendors get scrutinised, which departments get flagged, and which risks get ignored.

Phase 01
🔍

Detect

Dataset auditing · Historical bias review · Feature importance analysis · Outcome distribution analysis

Phase 02
🧪

Test

Counterfactual testing · Outcome parity tests · Disparate impact analysis · Subgroup performance breakdowns

Phase 03
🛡

Mitigate

Balanced training datasets · Remove proxy variables · In-process fairness constraints · Post-model threshold adjustment

Phase 04
📡

Monitor

Continuous bias tracking · Periodic fairness audits · Human review of borderline flags · Bias metrics in model dashboard

🔄
Types of Bias in Audit AI
Understanding the root causes
+

Historical Bias

The training data reflects past human decisions that were themselves biased. If auditors historically scrutinised certain vendor types more than others, the model learns to replicate that pattern.

Proxy Bias

A seemingly neutral variable (e.g., geography or business unit) correlates strongly with a protected attribute (e.g., ownership demographics), creating indirect discrimination in model outputs.

Survivorship Bias

Training on "known fraud" cases only captures fraud that was previously detected. The model misses novel fraud patterns precisely because no one has seen them before.

🧮
Fairness Metrics Auditors Should Know
Quantifying and reporting bias
+
MetricWhat It MeasuresWhen to Use
Demographic ParityEqual flag rate across subgroups regardless of actual riskScreening processes where equal treatment is mandated
Equalized OddsEqual true positive AND false positive rates across subgroupsWhen both missed positives and false positives carry costs
CalibrationPredicted risk scores match actual outcome rates per subgroupRisk scoring models used for prioritisation
Disparate Impact RatioRatio of selection rates between most and least selected groupsRegulatory compliance (80% rule in US EEOC context)

Tools: IBM AI Fairness 360 · Google What-If Tool · Scikit-learn fairness utilities · Aequitas (open source)

🔧
Mitigation Strategies
Pre-, in-, and post-processing approaches
+

Pre-processing

Rebalance or re-sample training data. Remove or transform biased features before training. Apply re-weighting to underrepresented classes.

In-processing

Build fairness constraints into the model's objective function during training. Use adversarial debiasing — train a secondary model to penalise bias.

Post-processing

Adjust decision thresholds per subgroup after training. Calibrate outputs so risk scores are equally meaningful across groups.

⚠️

Key Warning

No single mitigation approach is sufficient. Bias governance must be a continuous programme — not a one-time compliance exercise before deployment.

Question 06 · In Practice

Tools, Platforms & Real Examples

AI adoption in audit is already operational across multiple sectors. Here is a curated landscape of what's being used, where, and by whom.

🏦

Financial Services Deployment

  • Real-time transaction anomaly detection across millions of daily transactions
  • Anti-Money Laundering (AML) monitoring with network graph analysis
  • Continuous internal control testing replacing periodic reviews
  • Counterparty risk profiling using alternative data sources
  • Automated regulatory reporting validation (IFRS, Basel III)
🏢

Large Enterprise Internal Audit

  • ERP log analysis at scale (SAP, Oracle) for unusual posting patterns
  • Vendor master file monitoring for duplicate or ghost vendors
  • Procurement cycle analytics — detecting bid rigging patterns
  • Travel & expense claim anomaly detection
  • Payroll ghost employee and salary manipulation detection
🔬

External Audit Firms

  • PwC: GL Outlier — AI-powered journal entry testing at 100% population
  • EY Helix: Continuous controls monitoring platform across client systems
  • Deloitte Omnia: ML-driven risk assessment and audit planning
  • KPMG Clara: NLP-based contract review and policy compliance checking
  • Process mining for internal control mapping and gap identification
🛠

Technology Stack

  • Audit Analytics: MindBridge Ai, ACL/Galvanize, IDEA
  • Fraud Analytics: SAS Fraud Management, FICO Falcon
  • Process Mining: Celonis, UiPath Process Mining
  • GRC Platforms: ServiceNow, MetricStream, RSA Archer
  • ML Frameworks: Python (Scikit-learn, PyTorch), R, Azure ML
ISACA Deep Dive · AI Governance

AI Governance, COBIT & Frameworks

ISACA's COBIT 2019 and the emerging global AI governance landscape provide the structural backbone for responsible AI deployment in audit. Every ISACA member should understand how these frameworks interlock.

Framework Relevance to AI Auditing Key Principles / Domains ISACA Certification Alignment
COBIT 2019 Governance and management objectives for AI systems as IT components APO12 (Risk Mgmt), APO14 (Data Mgmt), BAI09 (Asset Mgmt), DSS05 (Security) CGEIT, CISA
ISO/IEC 42001 International standard for AI Management Systems — the "ISO 27001 for AI" AI system lifecycle, impact assessment, transparency, bias governance CISA (emerging)
EU AI Act (2024) Classifies AI systems by risk; "high-risk" AI in financial services subject to audit requirements Risk classification, conformity assessment, human oversight, explainability CRISC, CGEIT
NIST AI RMF Voluntary framework for managing AI risk — widely adopted by US financial institutions Govern, Map, Measure, Manage CRISC, CISM
ISACA AI Audit Guidance ISACA-specific guidance on auditing AI systems and AI-assisted auditing Model validation, data governance, transparency, continuous monitoring CISA, all ISACA certs
IIA Global IIA Standards 2024 Updated standards explicitly address technology-enabled auditing and AI use Independence, objectivity, professional scepticism, technology proficiency Complementary to CISA
🏛

COBIT APO14: Data Management

The most directly relevant COBIT domain for AI auditing. APO14 requires organisations to define data policies, ensure data quality, and govern data throughout its lifecycle — exactly the foundation AI needs to function reliably.

⚖️

Three Lines Model & AI

AI governance maps naturally to the Three Lines Model. First line: business units operating AI tools. Second line: risk and compliance functions validating models. Third line: internal audit independently assessing the entire AI risk framework.

🌍

EU AI Act: High-Risk AI

AI systems used in credit scoring, insurance, employment decisions, and critical infrastructure are classified "high-risk" and subject to mandatory conformity assessments, transparency requirements, and ongoing monitoring — a new audit universe.

ISACA Deep Dive · Ethical Boundaries

Ethics & the Human Boundary in AI Audit

The most contested question in AI auditing: where must human judgement always remain? And how do we protect professional scepticism from eroding when AI produces confident-looking outputs?

Professional scepticism is not a personality trait — it is a disciplined cognitive stance that requires actively challenging assumptions, seeking disconfirming evidence, and resisting anchoring to the first explanation. AI systems do not possess this. Auditors who defer to AI outputs without challenge abandon the core of their profession.
🚫

Where AI Must Never Have Final Authority

  • Audit conclusions and opinions — always human
  • Materiality judgements and significance assessments
  • Disclosure of control weaknesses to management
  • Decisions that affect individuals' employment or access
  • Regulatory findings and enforcement referrals
  • Engagement risk assessments and acceptance decisions
✅

Where AI Can Operate With Light Oversight

  • Transaction-level anomaly flagging for human review
  • Automated data extraction and normalisation
  • First-pass document classification and routing
  • Real-time threshold breach alerts
  • Workpaper population and evidence organisation
  • Scheduling and resource allocation recommendations

The Automation Bias Problem

🧠

Automation Bias in Audit Contexts

Automation bias is the tendency to over-rely on automated systems and to discount information that contradicts them. Research shows that when AI produces confident numerical outputs (e.g., "97.3% probability of fraud"), human reviewers dramatically reduce their independent investigation. For auditors, this is a professional ethics issue — not just an operational one.

🔮

Safeguard: Structured Challenge

Require auditors to document at least one reason why the AI's finding might be wrong before accepting it. This protocol counteracts anchoring bias and forces independent thought.

🎲

Safeguard: Blind Comparison

Periodically run traditional sampling alongside AI coverage. Compare results. Discrepancies reveal model blind spots and maintain auditor skill levels that might otherwise atrophy.

📚

Safeguard: AI Literacy Training

Auditors who understand how models work — and how they fail — are less susceptible to automation bias. The antidote is knowledge, not restriction.

ISACA Deep Dive · Continuous Auditing

From Periodic to Continuous Assurance

AI makes continuous auditing technically feasible for the first time. But technology is the easy part. The harder question is what governance, resourcing, and professional standards changes are needed to make it a reality.

Traditional
📅

Point-in-Time

Annual or quarterly audit. 1–5% sampling. Retrospective. Results delivered months after period end.

Transitional
🔁

Periodic Analytics

Monthly or quarterly data analytics. Full population. Still retrospective but faster. Exception-based reporting.

Advanced
📡

Continuous Monitoring

Always-on automated control testing. Real-time alerts. Human triage of exceptions. Near-real-time assurance.

Future State
🔮

Predictive Assurance

AI predicts control failures before they occur. Preventive interventions. Audit role shifts to model governance.

What Needs to Change for Continuous Auditing

  • Auditing standards must define the scope and requirements for continuous assurance engagements
  • Professional liability frameworks must address real-time conclusions vs periodic opinions
  • Organisations must invest in always-on data feeds and monitoring infrastructure
  • Audit teams need real-time triage capabilities and clear escalation protocols
  • Clients and boards must understand what continuous monitoring does and doesn't guarantee

What Continuous Auditing Does NOT Mean

  • It does not eliminate the need for periodic, substantive audit procedures
  • It does not reduce the required level of professional judgement
  • It does not mean the AI can flag without human review
  • It does not provide real-time audit opinions (standards don't support this yet)
  • It does not replace risk-based audit planning with algorithm-only planning
ISACA Deep Dive · Regulatory Landscape

Standards, Regulation & What's Coming

The regulatory environment for AI in financial services and audit is evolving rapidly. Here is the current landscape and what ISACA members should be preparing for.

🇪🇺

EU AI Act (in force 2024–2027)

The world's first comprehensive AI regulation. Financial services AI is largely "high-risk" — subject to conformity assessments, transparency requirements, human oversight mandates, and mandatory audit logging. Auditors need to understand this to assess client compliance.

🇿🇦

South Africa: AI Governance

South Africa's Department of Communications published the National AI Policy Framework. The SARB has issued guidance on model risk management for financial institutions. POPIA (Protection of Personal Information Act) governs training data use — auditors must assess POPIA compliance of AI systems using personal data.

🏦

Financial Sector Regulators

Basel Committee's principles for operational risk include AI model risk. IOSCO has published AI guidance for securities markets. Most central banks now expect model risk management policies that explicitly cover AI/ML models used in risk management and compliance.

📋

Auditing Standards Gap

Current ISAs (International Standards on Auditing) were written before AI-assisted auditing existed. The IAASB is actively developing ISA 315 revisions and technology standards guidance. ISACA's own standards are ahead of many statutory frameworks in this area.

🔮

What's Coming in 2026–2028

  • IAASB technology-specific auditing standards
  • EU AI Act enforcement regime begins
  • PCAOB guidance on AI use in public company audits
  • ISACA AI audit certification (proposed)
  • South Africa AI regulatory framework finalisation
🎯

Auditor's Regulatory Checklist

  • Is the AI tool classified under EU AI Act risk tiers?
  • Does training data use comply with POPIA / GDPR?
  • Is model documentation sufficient for regulatory review?
  • Does the model governance satisfy SARB / FSCA expectations?
  • Are bias assessments documented and current?
ISACA Deep Dive · Career Development

The Future of the Audit Profession

Will AI shrink audit teams or create demand for more specialised skills? The evidence points clearly in one direction: the auditor who understands AI will be more valuable, not less.

AI will not replace auditors. AI will replace auditors who don't use AI — and create a shortage of auditors who can govern, validate, and critically interpret AI outputs.

Competency Development Roadmap

Data literacy & SQL fundamentals
Essential now
Understanding ML model types
Essential now
AI governance frameworks
Essential now
Model validation methodology
High priority
Python / R for data analysis
High priority
Bias detection & fairness testing
High priority
LLM prompting & evaluation
Emerging need
EU AI Act & regulatory compliance
Emerging need
Causal inference methods
Advanced
Adversarial ML & red-teaming
Specialist
📉

Roles That Will Shrink

  • Manual transaction testing at sample level
  • Routine document review and classification
  • Basic control walkthroughs on standardised processes
  • Manual data extraction and normalisation
📈

Roles That Will Grow

  • AI model auditor — validating AI systems used by clients
  • Data audit specialist — governing training data quality
  • Continuous assurance lead — managing real-time monitoring
  • AI ethics and bias assessor
  • Technology risk specialist in AI/ML domains
🎓

ISACA Certifications to Prioritise

  • CISA — remains foundational; evolving AI content
  • CRISC — AI risk is technology risk
  • CGEIT — AI governance at enterprise level
  • CSX-P — cybersecurity for AI systems
  • Watch for ISACA's forthcoming AI-specific certification
Reference · Terminology

AI Glossary for Auditors

Plain-language definitions of the technical terms every ISACA member will encounter when working with AI systems.

Algorithm
A set of rules or statistical instructions a computer follows to solve a problem or make a prediction from data.
Machine Learning (ML)
A type of AI where the system learns patterns from data rather than following explicitly programmed rules. The model improves through exposure to more examples.
Large Language Model (LLM)
An AI model trained on vast amounts of text that can generate, summarise, and interpret natural language. Examples: GPT-4, Claude. Used for contract review, policy analysis.
Training Data
The historical dataset the model learns from. Its quality, size, and representativeness directly determine the model's reliability and fairness.
Precision
Of all the positives the model predicted, what fraction were actually positive? High precision = few false alarms.
Recall (Sensitivity)
Of all the actual positives that existed, what fraction did the model detect? High recall = few missed cases.
F1 Score
The harmonic mean of precision and recall. A balanced metric that penalises models that sacrifice one for the other.
Anomaly Detection
Identifying observations that deviate significantly from the expected pattern. The core technique for transaction-level fraud and control monitoring.
SHAP Values
A mathematical method (SHapley Additive exPlanations) for explaining individual model predictions by quantifying each feature's contribution to the output.
Model Drift
The degradation of a model's performance over time as real-world data patterns change from what was seen in training. Requires ongoing monitoring.
Prompt Injection
A cyberattack on LLM systems where malicious instructions are embedded in content the AI processes, causing it to behave contrary to its intended purpose.
Automation Bias
The human tendency to over-rely on automated systems and reduce independent verification. A key professional risk for auditors using AI tools.
Process Mining
Analysing event log data from IT systems to reconstruct actual process flows, identify bottlenecks, and detect control deviations. Tools: Celonis, UiPath.
Graph Analytics
Analysing relationships between entities (people, accounts, vendors) as a network to detect unusual connections — particularly powerful for collusion and money laundering detection.
Counterfactual Testing
Changing one variable in an input and observing how the model's output changes. Used to test bias: "Would the model flag this differently if only the vendor's geography changed?"
Differential Privacy
A mathematical technique for training models on sensitive data while providing provable guarantees that individual records cannot be reconstructed from the trained model.
Model Inversion Attack
A cyberattack where an adversary uses a model's outputs to reconstruct its training data — potentially exposing confidential financial records used to train audit AI.
Zero Trust Architecture
A security model that assumes no entity — human or AI system — should be trusted by default and requires continuous verification, least-privilege access, and real-time monitoring.
Reference · Self-Assessment

AI Readiness Checklist

Use this checklist to assess your organisation's readiness to deploy AI-assisted auditing. Click each item to mark it complete.

📊 Data Foundations
We have a centralised data warehouse or lake with clean, consistent transactional records
Audit trails are preserved with timestamps and cannot be altered without detection
Data ownership is defined for all major data assets used in audit processes
A data classification policy exists and is actively enforced
POPIA / relevant data protection compliance has been assessed for AI training data use
🔬 Model Governance
A Model Risk Register exists cataloguing all AI tools used in the audit function
Formal model validation procedures are documented and applied before deployment
Model performance is monitored continuously with defined alert thresholds
Explainability (SHAP/LIME) is required for any model producing audit findings
False positive and false negative rates are tracked and reported to management
🛡 Cybersecurity Controls
AI audit tools operate under least-privilege access — no standing connections to production data
Third-party AI vendors have been assessed for security posture (SOC 2 Type II or equivalent)
Training environments are isolated from production systems
AI tool access logs are themselves subject to audit and monitoring
⚖️ Bias & Ethics
A bias assessment has been completed for all AI models used in audit
Continuous bias monitoring is in place — not just a pre-deployment check
Human auditors are required to document independent analysis before accepting AI findings
No AI system has final authority over audit conclusions or regulatory findings
🎓 Team Capability
All audit team members have completed basic AI literacy training
At least one team member can interpret model outputs including SHAP values
The team understands the relevant regulatory requirements (EU AI Act, POPIA, SARB model risk guidance)
Score: 0 / 22 items completed
Knowledge Check

Test Your Understanding

Ten questions covering the key concepts in this guide. Aligned to CISA and CRISC exam topics.

Question 1 of 10

What is the primary advantage of AI over traditional sampling in transaction-level audit testing?

A AI is faster and cheaper than human auditors
B AI can analyse 100% of the population rather than a 1–5% sample
C AI produces legally binding audit conclusions
D AI eliminates the need for professional judgement
✓ Correct. The shift from 1–5% sampling to 100% population coverage is the most transformative advantage of AI in transaction testing, revealing anomalies that would statistically never appear in a sample.
✗ Not quite. The key advantage is coverage — AI analyses the entire transaction population (100%), not just a sample. Cost and speed are secondary benefits.
Question 2 of 10

Which of the following BEST describes "model drift" in the context of AI audit tools?

A The model moving to a different cloud server provider
B Auditors changing their interpretation of the model's outputs
C The gradual degradation of model performance as real-world data changes from training data
D Regulatory requirements changing faster than the model can be updated
✓ Correct. Model drift occurs when the statistical properties of real-world data diverge from training data over time, causing performance to silently degrade. This is why ongoing monitoring is essential.
✗ Model drift refers to the performance degradation that happens when real-world data patterns change after the model was trained. A COVID-era fraud model trained on pre-COVID data is a classic example.
Question 3 of 10

An AI audit model flags 200 transactions. Investigation reveals 160 are genuine anomalies and 40 are false alarms. What is the model's precision?

A 20%
B 40%
C 80%
D 100%
✓ Correct. Precision = True Positives / (True Positives + False Positives) = 160 / (160 + 40) = 160/200 = 80%. The model is right 80% of the time when it flags something.
✗ Precision = True Positives ÷ Total Flagged = 160 ÷ 200 = 80%. Of the 200 flagged items, 160 (80%) were genuine anomalies.
Question 4 of 10

Which attack vector involves an adversary deliberately corrupting training data to make an AI model overlook specific transaction types?

A Prompt injection
B Model inversion
C Model poisoning
D Data exposure
✓ Correct. Model poisoning involves corrupting training data to create specific blind spots in the trained model — allowing a fraudster to conduct activities that the AI will never flag.
✗ This is model poisoning — corrupting training data to bias the model's outputs. Prompt injection targets LLM inputs; model inversion reconstructs training data; data exposure is a confidentiality breach.
Question 5 of 10

SHAP values are primarily used in AI-assisted auditing to:

A Encrypt training data against unauthorised access
B Measure a model's overall accuracy on test data
C Explain why a model produced a specific prediction by quantifying each feature's contribution
D Detect adversarial inputs in real time
✓ Correct. SHAP (SHapley Additive exPlanations) assigns contribution scores to each input feature for a specific prediction — answering "why did the model flag this particular transaction?"
✗ SHAP values explain individual model predictions by quantifying each feature's contribution. This is essential for audit working papers — auditors must be able to explain why a finding was made.
Question 6 of 10

According to ISACA's IS Audit Standards, who is responsible for the final audit conclusion in an AI-assisted audit engagement?

A The AI vendor whose tool produced the findings
B The AI model, provided its accuracy exceeds 95%
C The human auditor, regardless of what tools were used
D The audit committee that authorised use of AI tools
✓ Correct. The human auditor retains full professional responsibility for audit conclusions — this is a fundamental principle that no AI tool changes. Standard 1005 (Due Professional Care) requires independent judgement.
✗ The human auditor always retains final responsibility. No level of AI accuracy transfers professional responsibility. This is a cornerstone principle of ISACA's audit standards.
Question 7 of 10

Which of the following represents a "pre-processing" approach to AI bias mitigation?

A Adjusting the model's decision thresholds after training is complete
B Adding fairness constraints to the model's objective function during training
C Rebalancing training data to remove historical bias before the model is trained
D Monitoring model outputs for disparate impact after deployment
✓ Correct. Pre-processing techniques intervene before training — rebalancing datasets, removing biased features, or re-weighting samples. In-processing happens during training; post-processing after training.
✗ The three categories are: pre-processing (before training — data rebalancing), in-processing (during training — fairness constraints in objective function), and post-processing (after training — threshold adjustment).
Question 8 of 10

The EU AI Act classifies AI systems used in credit scoring and financial risk assessment as:

A Minimal risk — no specific requirements apply
B Limited risk — transparency requirements only
C High-risk — subject to conformity assessments and mandatory human oversight
D Prohibited — banned from use in the EU
✓ Correct. The EU AI Act classifies financial AI as high-risk, requiring conformity assessments, transparency, explainability, human oversight, and ongoing monitoring — creating a significant new audit domain.
✗ Financial services AI is generally "high-risk" under the EU AI Act — not prohibited, but subject to significant requirements including conformity assessments and mandatory human oversight.
Question 9 of 10

What is "automation bias" and why is it particularly dangerous for auditors?

A AI tools that are biased toward finding more audit exceptions than actually exist
B The tendency of AI models to produce biased results when the input data is automated
C The human tendency to over-rely on automated outputs and reduce independent verification
D Bias introduced when audit processes are automated without human review
✓ Correct. Automation bias describes how humans reduce independent thinking when AI produces confident-looking results. For auditors, this directly violates the professional scepticism requirements in ISACA's standards.
✗ Automation bias is the human tendency to over-rely on automated systems. Auditors who accept AI findings without independent challenge abandon professional scepticism — a core ISACA standard requirement.
Question 10 of 10

A Model Risk Register should contain which of the following? (Select the MOST complete answer)

A Model name and deployment date only
B Model name, vendor, and last validation date
C Model name, training data sources, validation results, and known limitations
D All of the above plus performance thresholds, data access permissions, model owner, and next review schedule
✓ Correct. A comprehensive Model Risk Register covers identity, provenance, validation, limitations, governance, access controls, and monitoring cadence — treating AI as managed critical infrastructure.
✗ A comprehensive Model Risk Register covers far more than basic identification. It must include training data, validation results, known limitations, access controls, performance monitoring thresholds, and scheduled review dates.
—
out of 10 correct
Question 1 of 10