AI Agents in Action Foundations for Evaluation and Governance 2025

Page 22 of 34 · WEF_AI_Agents_in_Action_Foundations_for_Evaluation_and_Governance_2025.pdf

Risk assessment life cycle for AI agents TABLE 1 Step Objective Example activities Example outputs 1. Define context Establish the scope of the assessment, system boundaries, objectives and criteria for managing risk –Determine internal and external context (strategic goals, legal framework, stakeholders) –Define boundaries, intended use, assumptions –Establish risk criteria (likelihood, impact scales, acceptance threshold) –Context definition –Risk management plan –Risk evaluation criteria 2. Identify risks Identify potential technical, organizational and ecosystem risks, harms and affected partiesBrainstorm, workshops, risk identification (e.g. hazard identification, threat identification, etc.), identification of sources of risk, causes, failure mode analysis –Risk register listing risks, causes, impacts 3. Analyse risks Understand the nature, likelihood and consequence of each risk and quantify them –Assess probability and impact (considering, for example, characteristics like autonomy and authority, predictability and operational context) –Identify existing controls or guardrails; apply qualitative or quantitative methods for risk estimation; use evaluation results to inform likelihood and impact –Risk analysis scores showing likelihood impact ratings and rationale 4. Evaluate risks Compare analysis results with risk criteria to determine priority and tolerability –Rank and prioritize risks –Use evaluation results for quantifying and prioritizing risks –Use performance metrics and test confidence to inform risk thresholds –Risk ranking summary –Risk acceptance evaluations 5. Manage risks Implement risk response actions (avoid, mitigate, transfer, accept) and monitor risks –Assign owners of preventive, detective and response controls –Evidence these controls through evaluation results –Address emerging risks as systems evolve or context changes –Integrate feedback loops for continuing monitoring –Coordinate incident response and impact mitigation –Update governance and controls based on lessons learned –Control actions –Implementation plan –Residual risk profile –Risk assessment report –Evidence logs –Monitoring reports –Revised frameworks –Improved processes Defining clear risk criteria and tolerability thresholds, and applying them consistently to prioritize and evaluate risks, remains a central challenge in AI risk management. The identification, analysis and evaluation of risks are directly linked to the classification dimensions introduced earlier, allowing organizations to understand how factors such as autonomy, authority, predictability and environmental complexity shape overall risk levels for AI agents. Inherent risk combines likelihood and impact, while residual risk reflects the effectiveness of applied mitigations, informed by evaluation evidence such as system reliability, robustness and observed error rates. This relationship establishes a clear connection between how an agent is designed, how it performs and how risks are managed, providing the basis for proportionate governance and oversight. Applying this approach in practice helps demonstrate how structured risk assessment translates classification and evaluation evidence into measurable controls. The following example illustrates the risk assessment process in the context of an autonomous vehicle. AI Agents in Action: Foundations for Evaluation and Governance 22
Ask AI what this page says about a topic: