Building an AI Risk Assessment Process: A Practical Guide

AI risk assessment is the process that helps an organization understand what risks an AI system it is about to use or build carries, whether it works as expected, and whether it aligns with the organization's values. This article walks through how to build that assessment process step by step.

Why You Need an AI Assessment Process

A good AI assessment process helps you identify risks before they become problems, check the fairness and transparency of the AI, comply with relevant laws and standards, and build trust with every stakeholder involved.

"AI risk assessment is both a science and an art." It takes data, tools, and expert judgment working together.

The 4 Tier AI Risk Framework (EU Model)

• Unacceptable Risk: Prohibited outright, such as social scoring systems or real-time facial recognition in public spaces.
• High Risk: Permitted but requires strict controls, such as AI in healthcare, employment, or credit lending.
• Limited Risk: Users must be informed they are interacting with AI.
• Minimal Risk: Permitted under voluntary standards, such as spam filters.

Steps to Build an AI Assessment Process

Step 1: Gather Stakeholders

Identify who needs to be part of the assessment, such as the business team, technical team, legal, and risk functions.

Step 2: Define the AI's Business Objective

Clearly state what the AI will be used for, for whom, and what outcome is expected.

Step 3: Identify Potential Harms

Analyze every potential harm, including false positives/negatives and predictive bias.

Step 4: Evaluate the Training Data

Check the data's source, quality, completeness, and personal data protection.

Step 5: Compare Against Alternatives

Benchmark the AI against the existing process to confirm it is actually an improvement.

Step 6: Calculate the Risk Level

Use the formula: Likelihood of harm × Impact magnitude = Risk level.

Step 7: Choose a Standard and Assessment Framework

Reference NIST AI RMF, ISO 42001, or the EU AI Act depending on your context.

Step 8: Document the Results and Communicate to Leadership

Prepare a summary report and connect it with your existing Privacy Review process.

Tip: Don't build an AI assessment process entirely from scratch. Start by adapting your existing Privacy Review or Risk Assessment process for AI. It saves significant time and resources.