AI Agents in Action Foundations for Evaluation and Governance 2025

Baseline governance mechanisms for AI agents TABLE 2 Governance area Foundational mechanism Purpose Access controlEnforce least-privilege access; define task boundaries.Prevent each agent from accessing unnecessary data, systems, or tools; reduce risk of misuse or accidental harm. Legal and complianceConduct a data protection impact assessment (DPIA); perform privacy and regulation compliance checks, such as General Data Protection Regulation or the California Consumer Privacy Act (CCPA).Ensure data handling and processing complies with relevant laws and regulations. Testing and validationPerform sandbox runs or controlled pilots with non-production data; install input-output filters; perform third-party audits.Validate expected behaviour, detect errors and prevent untested code from affecting live systems, conduct audits (code, red teaming, etc.). Monitoring and loggingImplement logging for all agent actions; set up anomaly alerts or dashboards.Maintain traceability for accountability; enable early detection, incident response and post- incident analysis. Human oversightDefine and assign oversight models, including HITL/HOTL. Require policy review before deployment and set supervisory triggers for exceptions.Ensure accountable human control for material decisions, keep behaviour aligned with organizational policies and provide escalation paths when the agent acts unexpectedly. Traceability and identityAssign unique agent identifiers; tag outputs to the responsible agent instance.Attribute actions and outcomes to specific agents; enable forensic review and performance tracking. Long-term management Establish protocols for ongoing monitoring, updates and eventual decommissioning.Ensure continued alignment, performance and relevance throughout the agent’s life cycle. Trustworthiness and explainability Implement explainability tools; establish trust metrics.Ensure agent behaviour is interpretable and measurable; build user confidence. Manual redundancyEstablish manual redundancy procedures to ensure the sustained continuity of critical business use cases.Preserve data integrity and plan for human resources to take over.The example illustrates that an agent’s overall impact emerges from the interaction of multiple dimensions across function, role, predictability, autonomy, authority and context. As these dimensions shift, so does the risk profile, reinforcing the need for governance frameworks that are both progressive and adaptive. Effective governance requires maintaining an appropriate level of human oversight in relation to the agent’s autonomy, authority and operational context. In high-risk or less predictable settings, a human-in-the-loop (HITL) configuration ensures that agents can suggest or prepare actions, but final decisions remain subject to explicit human approval. In more stable or clearly defined environments, a human-on-the-loop (HOTL) configuration allows agents to act within defined boundaries, while humans monitor behaviour, receive alerts and retain the ability to intervene or override when necessary. Integrating these oversight models into governance structures helps maintain accountability and human judgment as agents operate with greater independence and scale. AI Agents in Action: Foundations for Evaluation and Governance 26