AI Agents in Action Foundations for Evaluation and Governance 2025
Page 22 of 34 · WEF_AI_Agents_in_Action_Foundations_for_Evaluation_and_Governance_2025.pdf
Risk assessment life cycle for AI agents TABLE 1
Step Objective Example activities Example outputs
1. Define context Establish the scope of the
assessment, system boundaries,
objectives and criteria for
managing risk –Determine internal and external context
(strategic goals, legal framework,
stakeholders)
–Define boundaries, intended use,
assumptions
–Establish risk criteria (likelihood, impact
scales, acceptance threshold) –Context definition
–Risk management plan
–Risk evaluation criteria
2. Identify risks Identify potential technical,
organizational and ecosystem
risks, harms and affected partiesBrainstorm, workshops, risk identification
(e.g. hazard identification, threat
identification, etc.), identification
of sources of risk, causes, failure
mode analysis –Risk register listing risks,
causes, impacts
3. Analyse risks Understand the nature, likelihood
and consequence of each risk
and quantify them –Assess probability and impact
(considering, for example, characteristics
like autonomy and authority,
predictability and operational context)
–Identify existing controls or guardrails;
apply qualitative or quantitative
methods for risk estimation; use
evaluation results to inform likelihood
and impact –Risk analysis scores
showing likelihood impact
ratings and rationale
4. Evaluate risks Compare analysis results with
risk criteria to determine priority
and tolerability –Rank and prioritize risks
–Use evaluation results for quantifying
and prioritizing risks
–Use performance metrics and test
confidence to inform risk thresholds –Risk ranking summary
–Risk acceptance
evaluations
5. Manage risks Implement risk response actions
(avoid, mitigate, transfer, accept)
and monitor risks –Assign owners of preventive, detective
and response controls
–Evidence these controls through
evaluation results
–Address emerging risks as systems
evolve or context changes
–Integrate feedback loops for
continuing monitoring
–Coordinate incident response and
impact mitigation
–Update governance and controls
based on lessons learned –Control actions
–Implementation plan
–Residual risk profile
–Risk assessment report
–Evidence logs
–Monitoring reports
–Revised frameworks
–Improved processes
Defining clear risk criteria and tolerability thresholds,
and applying them consistently to prioritize and
evaluate risks, remains a central challenge in AI
risk management.
The identification, analysis and evaluation of risks
are directly linked to the classification dimensions
introduced earlier, allowing organizations to
understand how factors such as autonomy, authority,
predictability and environmental complexity shape
overall risk levels for AI agents. Inherent risk
combines likelihood and impact, while residual risk
reflects the effectiveness of applied mitigations,
informed by evaluation evidence such as system reliability, robustness and observed error rates.
This relationship establishes a clear connection
between how an agent is designed, how it performs
and how risks are managed, providing the basis for
proportionate governance and oversight.
Applying this approach in practice helps
demonstrate how structured risk assessment
translates classification and evaluation evidence
into measurable controls. The following example
illustrates the risk assessment process in the
context of an autonomous vehicle.
AI Agents in Action: Foundations for Evaluation and Governance
22
Ask AI what this page says about a topic: