There is a moment in every audit engagement when the auditor looks at the population of transactions in front of them and makes a choice. It is always the same choice, and it is never comfortable: which 5% of this data am I going to actually look at?
For decades, that choice, sampling, has been the defining constraint of the audit profession. Not because auditors lack rigour or skill, but because the volume of modern financial data long ago outpaced any human team's capacity to review it. A mid-sized manufacturer processes millions of journal entries a year. A bank processes hundreds of millions of transactions a day. Audit, by necessity, became a discipline of intelligent selection rather than comprehensive review.
Artificial intelligence changes that constraint for the first time in the profession's history. And with it, every assumption auditors have built around what is possible, what is sufficient, and what constitutes professional coverage is now up for renegotiation.
This guide is for the professional who wants to understand that renegotiation honestly, not the breathless version where AI solves everything, and not the fearful version where AI threatens everything, but the practical version where AI is a powerful and genuinely useful tool that comes with its own serious risks, governance requirements, and ethical boundaries.
The One Statistic That Reframes Everything
Before getting into technique, there is one number worth sitting with.
Traditional audit sampling typically covers between 1 and 5 percent of a transaction population. AI-powered continuous monitoring can cover 100 percent, not as a theoretical future state, but as current operational reality in organisations that have deployed it.
That is not an incremental improvement. It is a categorical shift. A fraud scheme that operates in the 95% of transactions your sample never reaches is invisible to traditional audit. It is detectable to AI.
But that detection is only the beginning of the story. The harder questions, Why did this anomaly occur? Which control failed? What should have happened instead?, those remain irreducibly human. Audit is, at its core, a causal inference problem. AI dramatically extends the auditor's ability to find signals. It does not, and cannot, replace the judgement required to interpret them.
Where AI Is Actually Delivering Value Right Now
The AI-in-audit conversation is cluttered with speculation. Here is what is actually operational.
Anomaly Detection at Full Population Scale
Machine learning models can scan entire general ledgers rather than samples, flagging outliers in vendor payments, expense claims, payroll, and journal entries. These models learn what "normal" looks like for a given organisation, patterns by day of week, by posting user, by account combination, by amount range, and surface deviations that break those patterns.
The practical impact is significant. A payments anomaly that statistically appears in 0.1% of transactions would almost never appear in a 5% sample. It appears in every AI scan.
Graph Analytics for Collusion Detection
Standard anomaly detection looks at transactions in isolation. Graph analytics looks at the relationships between entities, between employees, vendors, bank accounts, and approval chains. Shell vendor schemes, kickback arrangements, and collusion between purchasing officers and suppliers all leave network fingerprints that individual transaction review cannot detect.
Graph-based fraud detection doesn't ask "is this transaction unusual?" It asks "is this relationship unusual?" That is a fundamentally different and often more powerful question.
Continuous Control Monitoring
The traditional model is periodic: the audit team arrives, reviews a period, produces a report, and leaves. By the time a control failure is identified, it may have been operating for twelve months.
Continuous monitoring replaces the periodic cadence with always-on surveillance. Control breaches generate real-time alerts. The audit function shifts from retrospective reviewer to near-real-time assurance provider.
Predictive Risk Scoring
Rather than applying a static annual risk assessment to decide which areas to audit, machine learning models can continuously update risk scores based on emerging signals, changes in transaction volumes, personnel movements, system access anomalies, supplier behaviour shifts. The audit plan becomes dynamic rather than fixed, responsive to actual risk rather than last year's assumptions.
Document Intelligence
Large language models (LLMs) can extract structured information from contracts, invoices, policy documents, and emails at scale. Evidence gathering, previously one of the most time-consuming phases of audit fieldwork, becomes partially automated. Contract compliance review, which once required reading hundreds of documents, can be performed across thousands simultaneously.
The Prerequisite Nobody Wants to Talk About
Here is the uncomfortable truth about AI-assisted auditing: it only works if your data foundations are solid. And for most organisations, they are not.
AI amplifies existing system quality. In practice this means: good systems become powerful; poor systems become dangerous.
An anomaly detection model trained on dirty, inconsistently coded, poorly maintained transaction data will produce unreliable outputs. Worse, it will produce confident-looking unreliable outputs, numbers, percentages, risk scores, that carry an air of algorithmic authority they have not earned.
Before any AI audit deployment is worth considering, three layers of maturity need to be in place:
Data maturity. A centralised data warehouse or lake with clean, consistently labelled transactional records. Consistent identifiers across systems. Preserved audit trails. Documented data lineage. If your ERP uses different cost centre codes from your HR system and no one has mapped the relationship, your AI model will be learning patterns from fragmented data.
Systems maturity. API or structured access to ERP and source systems. Comprehensive event logging. Version control for models. The AI needs a reliable, repeatable pipeline to the data it needs to learn from.
Governance maturity. Defined data ownership. Documented model validation procedures. Access controls and segregation of duties applied to the AI systems themselves. A Model Risk Register cataloguing every AI tool used in the audit function.
A useful benchmark is CMMI Level 3 or its equivalent, processes that are defined, documented, and applied consistently. Below that level, the investment should go into data foundations first, AI tooling second.
Auditing the Auditor: How to Validate an AI Model
If an AI model is going to influence audit conclusions, it must itself be subject to audit. Treating AI as a black box that produces correct answers because it is a computer is a governance failure waiting to happen.
Model validation covers three layers.
Layer One: Logic Review
Does the model do what it claims to do? Are the input variables legitimate risk indicators, or are some of them proxies for protected characteristics that introduce bias? Has the feature engineering been reviewed by someone who understands both the audit domain and the model's mechanics?
Explainability tools, specifically SHAP (SHapley Additive exPlanations) and LIME (Local Interpretable Model-agnostic Explanations), allow auditors to interrogate black-box models without needing access to internal weights. SHAP answers the question: which variables drove this particular flag, and by how much? Any AI audit tool worth procuring should support SHAP or equivalent output, because audit working papers must explain why a finding was made.
Layer Two: Performance Validation
Two metrics matter above all others: precision and recall.
Precision asks: of all the anomalies the model flagged, what fraction were genuine? Low precision means the audit team is wading through false alarms, with the real risk of genuine issues being buried in noise, and of professional cynicism developing toward the AI's outputs.
Recall asks: of all the genuine anomalies that existed, what fraction did the model catch? Low recall is the more dangerous failure mode for auditors. It means the model is producing false assurance, flagging some issues while systematically missing others.
The right balance depends on context. AML systems should optimise for high recall (catch everything, accept false positives). Internal control testing models should optimise for high precision (only surface what is worth investigating). The F1 score, the harmonic mean of both, provides a single balanced metric. Vendors who quote headline accuracy figures without F1 scores are not giving you the number you need.
Layer Three: Stability Testing
A model that performed well in Year 1 may silently degrade by Year 3. Model drift, the erosion of model performance as real-world data patterns diverge from training patterns, is one of the most under-appreciated risks in AI deployment.
The COVID-19 pandemic is the most vivid recent example. Fraud models trained on pre-2020 purchasing patterns were effectively useless in 2021. Transaction behaviours, approval patterns, supplier relationships, and spending volumes all changed fundamentally. Any model that wasn't retrained struggled to distinguish pandemic-related anomalies from genuine control failures.
Ongoing drift monitoring, comparing current input distributions to training distributions, must be a standard operational control, not an afterthought.
The Cybersecurity Risk Nobody in Audit Is Talking About Enough
When an AI audit tool is granted access to an organisation's complete transaction history in order to train a fraud detection model, it becomes one of the highest-value attack targets in the organisation. The entire financial record is now, in effect, in the model's training environment.
Four attack vectors deserve specific attention:
Model poisoning. An adversary with access to training data can corrupt specific records to create deliberate blind spots in the trained model. The model learns to ignore certain transaction types, vendor relationships, or user account patterns, and then continues ignoring them throughout its operational life, with no visible sign that anything is wrong. This is the most dangerous attack vector because it is silent and self-concealing.
Prompt injection. LLM-based audit tools that process documents, contracts, invoices, emails, can be manipulated by malicious content embedded in those documents. An invoice with a carefully crafted payload can cause the AI to ignore suspicious line items, misclassify the document, or produce a false assurance summary. Every document an LLM processes is a potential attack surface.
Model inversion. Sophisticated adversaries can query a deployed model repeatedly to reconstruct its training data. If your AI model was trained on payroll records or customer financial data, that data may be recoverable through patient exploitation of model outputs alone.
Supply chain compromise. Third-party AI vendors with access to your ERP are indirect access points to your most sensitive data. A vendor's security posture is now your risk.
The governance implication is clear: AI audit tools must be governed as critical infrastructure. Zero-trust access architecture, least-privilege principles, isolated training environments, comprehensive logging, and rigorous third-party security assessments are not optional controls, they are the baseline.
The Bias Problem Is an Audit Integrity Problem
Bias in AI audit systems is, at its root, a data problem. A model trained on historically biased audit outcomes will perpetuate those biases at scale, and it will do so with algorithmic confidence that can be harder to challenge than human bias.
The most insidious form is what might be called survivorship bias in fraud training data. A model trained on "known fraud cases" can only learn the patterns of fraud that was previously detected. Novel fraud schemes, precisely the ones most dangerous and hardest to catch, are by definition absent from the training set. The model becomes excellent at finding the old way of doing things wrong, and systematically blind to the new way.
The bias management cycle has four phases:
- Detect — audit the training dataset itself for historical bias, and use feature importance analysis to identify which variables are doing the most work in the model's decisions.
- Test — run counterfactual tests (change one variable and observe whether the outcome changes disproportionately) and outcome parity tests (does the model flag the same proportion of anomalies across different business units, geographies, or supplier types?).
- Mitigate — through pre-processing (rebalancing training data), in-processing (fairness constraints in the objective function), or post-processing (threshold calibration per subgroup).
- Monitor — continuously, as an ongoing programme. Bias management is not a pre-deployment checkbox. It is a permanent operational responsibility.
Tools including IBM AI Fairness 360, Google's What-If Tool, and Scikit-learn fairness utilities provide quantitative frameworks for this work. Their outputs should appear in model validation documentation and, where AI audit findings are consequential, in audit working papers.
The Governance Framework: COBIT, ISO 42001, and the EU AI Act
The regulatory and governance landscape for AI in audit is evolving rapidly. ISACA members need a working map.
COBIT 2019 provides the most immediately actionable framework. Several management objectives are directly relevant: APO12 (Risk Management) for AI model risk, APO14 (Data Management) for training data governance, BAI09 (Asset Management) for cataloguing AI tools, and DSS05 (Security Services) for AI cybersecurity controls. COBIT also maps naturally to the Three Lines Model: the first line operates AI tools, the second line validates models, and the third line, internal audit, independently assesses the entire AI risk framework.
ISO/IEC 42001:2023 — sometimes described as the "ISO 27001 for AI", establishes requirements for AI Management Systems. It addresses AI system lifecycle governance, impact assessment, transparency obligations, and bias governance. Organisations deploying AI in material financial processes should be working toward conformance.
The EU AI Act, which entered force in 2024 with a phased implementation timeline, classifies AI systems used in credit scoring, insurance underwriting, and financial risk management as "high-risk." This classification triggers mandatory conformity assessments, explainability requirements, human oversight obligations, and ongoing monitoring standards. For auditors in organisations operating in or selling into the EU, assessing client AI compliance is becoming a standard engagement scope item.
South African context: POPIA (the Protection of Personal Information Act) governs training data that includes personal information, relevant for any AI model trained on employee or customer records. SARB's model risk management guidance for financial institutions addresses ML models in risk management. FSCA oversight is expanding to cover algorithmic systems in financial services.
The gap between where current auditing standards stand and where AI-assisted auditing is heading is real. The IAASB is actively developing technology-specific guidance. ISACA's own standards are, in most respects, ahead of statutory frameworks. But the profession is in a transitional period, and members who understand both the technical realities and the governance requirements will be the ones defining what professional standards look like on the other side.
The Ethics Question That Cannot Be Delegated to an Algorithm
There is a question that comes up in every serious conversation about AI in audit, and it deserves a direct answer rather than a diplomatic one.
Should AI ever be permitted to make audit conclusions autonomously?
No.
Not because AI isn't capable of producing confident, well-reasoned outputs. It clearly is. But because audit conclusions are not purely technical determinations, they are professional judgements that carry legal, regulatory, and ethical weight, and that weight must rest with a human being who can be held accountable for it.
ISACA's IS Audit Standard 1005 (Due Professional Care) requires that auditors apply professional scepticism and independent judgement. No AI system satisfies this requirement on its own, not because the technology is insufficient, but because professional accountability is inherently human.
The subtler risk is automation bias: the documented human tendency to reduce independent investigation when an automated system produces confident-looking outputs. Research consistently shows that when AI gives a high-confidence numerical verdict, human reviewers skip steps they would otherwise take. For auditors, this is not just a cognitive quirk, it is a professional ethics issue.
The practical safeguard is straightforward but requires discipline: require auditors to document at least one substantive reason why an AI finding might be wrong before accepting it. Not as bureaucratic friction, but as a structured mechanism for preserving the cognitive habit of challenge that the profession calls professional scepticism.
The Future of the Profession: A Realistic View
The question everyone asks is whether AI will replace auditors. The honest answer is that it will replace some of what auditors currently do — and create strong demand for capabilities that barely exist in the profession today.
Manual transaction testing at sample level will become less economically justifiable. Routine document classification and basic control walkthroughs on standardised processes will be automated. These are not the interesting parts of audit work, and most auditors will not mourn their passing.
What will grow, and what is already growing faster than the talent supply can keep up with, includes:
- AI model auditor: validating the AI systems that clients use in financial risk management, fraud detection, and compliance monitoring. This is a new audit universe that the profession is only beginning to map.
- Data audit specialist: governing the quality, lineage, and governance of training data for AI systems.
- Continuous assurance lead: managing real-time monitoring operations and triaging alerts in near-real-time.
- AI ethics and bias assessor: providing independent assessment of fairness, transparency, and accountability in deployed AI systems.
- Technology risk specialist in AI/ML domains: understanding the specific threat landscape of machine learning systems.
The competencies that matter most right now, that most audit professionals can start building today, are data literacy, a working understanding of how machine learning models function and fail, fluency with AI governance frameworks including COBIT and ISO 42001, and the ability to interpret model validation outputs including SHAP values, precision and recall metrics, and drift detection results.
Python and R are increasingly useful. Regulatory knowledge of the EU AI Act and local equivalents is becoming essential. But the foundational competency is older than all of these: the professional scepticism to ask, when an AI tells you something, why do I believe this, and how would I know if it were wrong?
What This Means for Your Next Engagement
If you are an ISACA member preparing for an audit in an AI-enabled environment, or an organisation preparing for one, the practical starting points are these.
First, treat the AI tools your client uses as audit objects in their own right. Do they have model documentation? A validation history? A Model Risk Register? Drift monitoring in place? If not, that is a finding.
Second, assess the data foundations before forming any view on the AI outputs. Ask for training data provenance, data quality metrics, and evidence of bias testing. An AI system is only as trustworthy as the data it learned from.
Third, understand the cybersecurity perimeter that the AI tools have created. Which systems do they access? Under what access controls? Have third-party AI vendors been assessed? Are training environments isolated?
Fourth, look for the human governance layer. Who owns the models? Who reviews their outputs? What is the escalation path when an AI flag is disputed? Is there a policy governing what decisions AI tools may and may not make autonomously?
And fifth, perhaps most importantly, audit your own team's relationship with the AI tools you are using. Are auditors genuinely challenging AI outputs, or accepting them because they carry the authority of numbers? The most sophisticated AI governance failure in the next decade will not be a cyberattack or a biased model. It will be a profession that slowly stopped thinking because a machine was thinking for them.
Conclusion: The Causal Inference Frame
Audit has always been a causal inference discipline. Auditors ask: What caused this anomaly? Which control failed? What would have happened under proper controls?
AI dramatically expands the auditor's ability to find the anomaly in the first place, scanning populations that were previously impossible to review, surfacing patterns that no human would have spotted, flagging relationships that no sample would have revealed.
But the inference, the explanation, the conclusion, the judgement, remains stubbornly, necessarily, irreplaceably human.
The auditors who will define the profession over the next decade are the ones who understand both sides of that boundary clearly: what AI can do, what it cannot do, how to govern it rigorously, and how to use it as the powerful tool it is without surrendering the professional scepticism that gives the audit opinion its meaning.
That is not a modest ambition. It is the right one.
Edzai Zvobwo is an AI Scientist and the CEO of Acalytica, a digital growth platform for creators, SMEs, and agencies across Africa. He presented on AI-assisted auditing at the ISACA SA Johannesburg Chapter in-person event on 4 March 2026, hosted by SKX Protiviti.
The complete interactive tutorial, knowledge quiz, readiness checklist, and glossary referenced in this article are available at acalytica.com/ai-assisted-auditing.
Related Resources
- AI-Assisted Auditing: Complete Interactive Guide for ISACA Members
- ISACA IS Audit Standards — Standard 1005: Due Professional Care
- COBIT 2019 Framework — APO12, APO14, DSS05
- ISO/IEC 42001:2023 — AI Management Systems
- EU Artificial Intelligence Act (2024)
- NIST AI Risk Management Framework
- POPIA — Protection of Personal Information Act (South Africa)
