AI Risk Assessment Framework: Step-by-Step for Australian Organisations
An AI risk assessment isn’t a one-time audit—it’s a disciplined process that identifies what can go wrong with your AI systems and what it’ll cost if it does. In 2024, 69% of Australian data breaches involved malicious attacks; many exploited weaknesses in AI systems that weren’t properly assessed. This six-step framework is structured for Australian organisations and aligns with Privacy Act 2024, APRA CPS 230, and OAIC guidance.
Step 1: Identify Your AI Assets
Create an inventory of every AI system your organisation uses: commercial tools (ChatGPT, Copilot), vendor-managed AI (marketing automation, recruitment screening), in-house models (demand forecasting, customer segmentation), and AI-enabled processes (chatbots, content generation). Document ownership, purpose, data inputs, and business criticality. Many organisations overlook Step 1 and regret it later: regulators often ask “show me all your AI” and organisations can’t. Start now—this is where 80% of assessment value comes from.
Step 2: Identify Threats and Failure Modes
For each AI system, ask: What could go wrong? Model accuracy could degrade. Data quality could corrupt outputs. A vendor could have a security breach. Your model could inherit bias from training data. Regulations could shift. Document threats across six categories: model (performance, drift, bias), data (quality, privacy, accuracy), operational (vendor failure, downtime), legal (Privacy Act, sector rules), reputational (bad decisions, public perception), and strategic (competitive, talent, dependency). Involve technical teams, compliance, and business owners—threats look different from different angles.
Step 3: Assess Likelihood and Impact
For each threat, estimate likelihood (1=rare, 5=almost certain) and impact (1=negligible, 5=catastrophic). Use your organisation’s risk tolerance and historical data where available. If you’ve had three data breaches in five years, breach likelihood is moderate-to-high. If your model has never drifted but the vendor documentation flags it as possible, it’s low-to-moderate. Don’t overthink it; aim for directional accuracy, not precision.
Risk Matrix (5×5): Plot likelihood (x-axis) and impact (y-axis). Multiply them for a risk score (1–25). Score 20+: red (unacceptable without strong controls). Score 10–19: amber (tolerable with controls). Score 1–9: green (low risk, monitor). This visual helps prioritise.
Step 4: Identify Existing Controls
What mitigations do you have? Model performance monitoring? Data validation? Vendor security audits? Privacy policies? Human review gates? Incident response plans? For each threat, document the control(s) reducing its impact or likelihood. Be honest: if you *think* you’re validating model accuracy but no one owns it, that’s not a control. Controls must be documented, assigned to someone, and regularly tested.
Step 5: Calculate Risk Ratings
Residual risk = (likelihood × impact) – (control effectiveness). If a threat has likelihood 4 and impact 4 (score 16) but you have strong controls (say, 60% effective), residual risk is roughly 6–7 (low). If your controls are weak, residual risk stays high. Document your assumptions. If a regulator later asks why you accepted a 15-rated risk, you need to explain your control confidence and your risk appetite. Guesswork doesn’t survive scrutiny.
Step 6: Treatment Planning and Prioritisation
For red and amber risks, plan treatment: reduce likelihood (better data, vendor diversification), reduce impact (human oversight, staged rollout), transfer (insurance, vendor indemnity), or accept (if business value justifies it, board must approve). Prioritise by residual risk—highest first. Assign owners, set deadlines, and track progress monthly. After treatment, re-assess and confirm risk rating dropped to acceptable levels.
Review Cadence and Continuous Improvement
Conduct full assessments quarterly or after major changes (new AI system, regulation shift, incident). Monthly check-ins on high-risk systems. Annual refreshes of threat landscape. Each incident should trigger a rapid re-assessment: What did we miss? How does it change our risk profile? Feed these learnings back into your next assessment cycle. Make assessment a rhythm, not an event.
Integration with Privacy Act and APRA CPS 230
Link your AI risk assessment to Privacy Impact Assessment (PIA) requirements under Privacy Act 2024. If your AI system processes personal information, a PIA is mandatory if the processing is likely to result in serious harm. If you’re APRA-regulated and your AI system is material to operations, your assessment must inform your CPS 230 vendor risk and operational resilience framework. Assessments serve double duty when aligned properly.
Frequently Asked Questions
Q: Should we use a numerical risk matrix or qualitative assessment?
A: Start with numerical (5×5 matrix). It forces precision and makes comparisons easier. As you mature, you can use qualitative language, but numbers are more defensible in front of regulators or boards.
Q: How do we know if our controls are effective?
A: Test them. Run scenarios. Audit logs. Performance data. If you claim model monitoring is a control but don’t have proof of weekly reviews, it’s not a control. Build evidence trails.
Q: What if we can’t quantify impact?
A: Use descriptors: negligible, minor, moderate, major, catastrophic. Then assign numbers (1–5) based on your risk appetite. Consistency matters more than absolute accuracy.
Call to Action
Assessment is foundational but time-consuming. If your team lacks capacity or expertise, contact Anitech to discuss a guided assessment, or book a consultation to review your current approach. We’ll help you prioritise high-risk areas and build an assessment rhythm tailored to your business size and risk profile.