AI Agents in Action Foundations for Evaluation and Governance 2025

The orchestration layer (framework layer) governs how the agent interprets inputs, invokes tools and coordinates tasks. While some LLM providers5 have integrated tools directly into their solutions, this can create rigid and vendor-locked systems. Agentic frameworks overcome this by standardizing tools and systems integration, remaining LLM-agnostic and spanning multiple workloads across cloud and edge. This enables AI agents to employ a range of reasoning strategies and support features, such as code execution or search, and use protocols like MCP to connect with enterprise resources, including databases and customer relationship management (CRM) systems. Most agents also include specialized sub-agents that handle distinct tasks, which makes them functionally part of a multi-agent system. The orchestration layer is critical in this regard, as it coordinates sub-agents, assigns responsibilities and manages dependencies between them. It also enables model switching, allowing organizations to assign different models to various tasks based on their complexity, cost or performance. Importantly, AI agents have a unique architecture that can be extended beyond the organization’s security perimeter. Their ability to invoke external tools and communicate with other agents enables them to operate beyond traditional network boundaries, introducing novel cybersecurity concerns. The reasoning layer underpins the agent’s ability to generate, predict, classify or apply rules in pursuit of its goals. Depending on the task, the reasoning layer can draw on a range of models, including deterministic, rule-based approaches and classical machine learning, as well as small or large language models and other generative architectures. The choice of model shapes how the agent processes information, adapts to context and ultimately carries out its assigned role. Figure 2 illustrates this layered architecture, showing how internal components across application, orchestration and reasoning work together to support dynamic agent behaviour while maintaining secure boundaries across organizational systems. In combination, these layers constitute the technical backbone that governs agent functionality. For organizations implementing AI agents, understanding this architecture is key to anticipating how agents will engage with users and systems, coordinate workflows and make context-aware decisions. Software architecture of an AI agent FIGURE 2 Internal organization resources Third-party resources Al agent boundaryEnvironment IT applications Al agentAl agentEvent User Application Orchestration ReasoningCRM Messaging Database Application Orchestration ReasoningAPI UI CodeInput/output Non-generative Generative MechanisticModelsAgentic framework Memory Planning Tools WorkflowMCP A2AActions Percepts Understanding this architecture is key to anticipating how agents will engage with users and systems, coordinate workflows and make context- aware decisions. AI Agents in Action: Foundations for Evaluation and Governance 8