AI Agents in Action Foundations for Evaluation and Governance 2025

CASE STUDY 3 Autonomous vehicle – risk assessment Agent characteristics 1. Function 3. Predictability 5. AuthorityPerforms the complete driving task without human control Deterministic Non-deterministic2. Role Specialist Generalist Low High Low HighOperational context 6. Use case An autonomous vehicle operates in the transportation domain, navigating public or private road environments to transport passengers or goods safely and efficiently without direct human control. 7. Environment Simple Complex 4. AutonomyAutonomous vehicle Autonomous vehicle – risk assessment Risk assessment focuses on identifying and mitigating possible failures across perception, decision-making and control systems. Key risk areas include sensor malfunction, data drift, adversarial interference and coordination failures with other vehicles or infrastructure that could lead, for example, to loss of steering or braking control and eventual collisions. Each risk is analysed for its likelihood (for example, the frequency of sensor failure leading to braking failure) and its impact (for example, the severity of injury, fatality or legal consequence). Quantitative scoring combines these factors and is weighted according to the vehicle’s autonomy and authority levels. Mitigation measures may include redundancy and diversity in critical sensors, reduction of autonomy or authority thresholds, anomaly detection mechanisms and real-time incident reporting. Residual risk is evaluated after these safeguards are applied, drawing on evidence from controlled testing, field trials and continuous monitoring. The results determine whether the system can safely progress to wider deployment or requires additional control layers. AI Agents in Action: Foundations for Evaluation and Governance 23