Scaling Managed Agents: Decoupling Brain From Hands

Lisa Ernst · 12.04.2026 · Artificial Intelligence · 11 min

Architecting for Agility: Scaling AI Agentic Systems in the Enterprise

As a technology journalist, I’ve witnessed countless shifts, but few feel as transformative as the rise of artificial intelligence agents. These sophisticated systems, blending advanced reasoning with practical action, are poised to redefine how enterprises operate. Yet, the journey from powerful concept to production-ready solution is fraught with challenges, particularly when scaling these intelligent entities.

Source: The "more agents are better" approach often encounters limitations, potentially hindering performance if not precisely aligned with specific task characteristics.

The "more agents are better" approach often encounters limitations, potentially hindering performance if not precisely aligned with specific task characteristics, as detailed in a recent arXiv paper. Multi-agent coordination demonstrably improves performance for parallelizable tasks, yet it can degrade performance significantly in sequential tasks, as also noted in the same research. A predictive model can identify the optimal architecture for 87% of unforeseen tasks, highlighted in the paper.

Quick Summary

Here’s a brief overview of key insights on scaling AI agents:

• AI Agents Defined: AI agents combine large language models (LLMs) or multimodal models (VLMs) with reasoning, planning, memory, and tool use to execute complex tasks.
• Architectural Choices Matter: The effectiveness of multi-agent systems depends heavily on the chosen architecture (Single-Agent, Independent, Centralized, Decentralized, Hybrid) and task type.
• Task Specialization: For parallelizable tasks like financial analysis, centralized coordination significantly boosts performance. For sequential tasks, multi-agent approaches can hinder performance due to communication overhead.
• Tool Coordination Trade-off: More tools mean greater coordination overhead, which can amplify errors in independent multi-agent systems. Orchestrators in centralized systems can mitigate this.
• Enterprise Solutions: Major tech providers (Google Cloud, Databricks, Atos, Microsoft, Camunda) offer frameworks and architectures for deploying and scaling AI agents, emphasizing modularity, governance, and secure operations.
• Challenges: Scaling agents involves hurdles like ensuring reliability, reproducibility, managing non-determinism, and addressing new security risks like prompt injection.
• Learning Frameworks: Advanced learning frameworks, such as Google’s Forge, tackle the trilemma of system throughput, training stability, and agent flexibility to enable large-scale deployment.

The Anatomy of an AI Agent

AI agents function as systems that think, plan, and act, leveraging Large Language Models (LLMs) or multimodal models (VLMs). They serve as a practical bridge between natural language and real-world computation. These systems combine foundational models with capabilities such as reasoning, planning, memory, and tool utilization. They act as controllers, translating intentions into procedures executed in the external environment. Modern digital work, often fragmented across interfaces and APIs, underscores the growing necessity for AI agents.

Pure conversational systems frequently struggle with hallucinations, a lack of grounding, and an inability to perform or verify actions, highlighting the need for more robust, agentic approaches. Tool- and retrieval-augmented designs significantly enhance reliability by anchoring claims to evidence and making intermediate artifacts verifiable.

The execution cycle of an AI agent typically involves observation, memory retrieval, action proposal, validation, and execution. These "Agent Transformers," as they are sometimes called, are Transformer-based policy models embedded within structured control cycles. Their behavior is modeled as sequence modeling across interaction traces, encompassing observations, thoughts, tool calls, and results. Decision policies can branch based on risk, for instance, by implementing additional verification steps for high-risk actions. A ReAct-style trace proves invaluable for both performance and governance, as it binds decisions to concrete evidence and tool outputs.

Scaling AI Agents: Architectural Paradigms

Scaling AI agentic systems within an enterprise demands careful architectural consideration. Five standard agent architectures have emerged: Single-Agent-System (SAS), Independent, Centralized, Decentralized, and Hybrid, as discussed in an arXiv paper.

A Single-Agent-System (SAS) executes all thought and action steps sequentially using a unified memory stream, according to the same source. Independent multi-agent systems operate in parallel on sub-tasks without communication, aggregating results only at the final stage, as detailed in the research. Centralized systems employ a central orchestrator which delegates tasks to worker agents and synthesizes their outputs, as described in the paper. Decentralized systems form a peer-to-peer network where agents communicate directly to exchange information and reach consensus, also mentioned in the arXiv article. Hybrid systems merge hierarchical oversight with peer-to-peer coordination, balancing centralized control with flexible execution, as also outlined in the paper.

The performance of multi-agent systems varies significantly with configuration and can improve or degrade depending on the model family being used, such as OpenAI GPT, Google Gemini, or Anthropic Claude, as noted in the study. For parallelizable tasks, like financial analysis, centralized coordination led to an 80.9% performance increase over a single agent, as referenced in the arXiv paper and Vals.ai benchmarks. Conversely, tasks demanding strict sequential reasoning, such as planning (e.g., in PlanCraft), saw multi-agent variants worsen performance by 39-70%, as the communication overhead fragmented the sequential thought process, according to this research.

A notable “tool-coordination trade-off” exists: as the number of required tools increases, the coordination overhead for multiple agents disproportionately rises. Independent multi-agent systems amplified errors by 17.2 times due to the absence of a work verification mechanism, as found in the arXiv paper. Centralized systems with an orchestrator mitigated error amplification to 4.4 times, with the orchestrator functioning as a critical validation bottleneck, also according to the study. The existence of a predictive model with an R-squared value of 0.513 indicates that task characteristics, such as tool count and decomposability, can forecast the optimal architectural choice, as highlighted in the research. This model accurately identifies the best coordination strategy for 87% of unseen task configurations, as reported in the paper.

Enterprise-Grade Agentic Systems

Several major technology providers are developing robust frameworks and architectures to facilitate the adoption and scaling of AI agents in enterprise settings. For example, Google Cloud offers design patterns for scalable and cost-effective AI agent solutions, even on rival platforms like Azure. Their Google Cloud Agent Development Kit (ADK) provides scaffolding, tools, and patterns for developing enterprise agents, while the Agent Engine offers a secure runtime environment for managing their lifecycle, orchestrating tools, and controlling their reasoning. The Model Context Protocol (MCP) standardizes how applications provide context to LLMs, and the Agent-to-Agent Protocol (A2A) enables communication between agents at scale.

Source: the-decoder.com

Google Cloud’s supply chain architecture utilizes a "Root" agent and specialized sub-agents, powered by Gemini, for tracking and order management.

A real-world illustration of this approach can be seen in Google Cloud's proposed supply chain architecture, which involves an orchestrating "Root" agent and specialized sub-agents for tracking, distribution, and order management. These agents, powered by Gemini, can be tuned with domain-specific data to optimize responses and reduce costs. The Root agent directs requests to the appropriate sub-agent; for instance, the Order agent, using MCP, retrieves inventory details from a BigQuery data warehouse. Similarly, a Distributor agent leverages MCP server connections to external APIs to place orders.

Databricks, in collaboration with BASF Coatings, implemented a production-ready multi-agent solution to automate coordination and analysis tasks previously requiring intensive manual effort. Their Marketmind project, focused on the Sales & Marketing division, utilizes specialized "Genie" agents and function-calling agents interacting with a Databricks Vector Store Retrieval Tool to handle structured and unstructured data. Genie agents make structured data accessible via natural language interfaces, constructing SQL queries from user questions by leveraging Unity Catalog metadata. This system employs a supervisor pattern to coordinate these specialized agents, integrating with Microsoft Teams for real-time execution. Marketmind, set to be used by over 1,000 sales representatives globally, aims to transform field interactions into timely, AI-driven actions, demonstrating the tangible business impact of multi-agent architectures characterized by scalability, modularity, and enhanced user experience.

Atos also provides a blueprint for "Services as Software," aiming to transform workflows into software-delivered outcomes. Their approach emphasizes a control plane for trust at scale, including runtime governance, zero-trust access, behavioral security, kill switches, observability, and immutable audit trails. This "Sovereign Agentic AI" framework ensures deliberate control over critical data, decisions, and AI behavior across the entire tech stack.

Microsoft also offers a reference architecture for adaptive, scalable, and secure multi-agent systems, focusing on orchestrating, governing, and scaling specialized agents. Camunda 8.8 introduces standard-based, enterprise-grade agents, allowing for the design and execution of high-quality processes. Camunda's Agentic BPMN facilitates building AI agents, connecting them to LLMs, and deploying them. The AI Agent Connector empowers agents to act autonomously within end-to-end processes and integrate with LLM providers, extending orchestration beyond individual tasks into dynamic process execution. Camunda can orchestrate both its own agents and external ones, maintaining process state and continuity, which supports larger toolsets and greater scalability.

Source: stickpng.com

Databricks’ Marketmind project integrates specialized agents with Microsoft Teams for real-time execution, transforming sales interactions into AI-driven actions.

Challenges in Scaling and Learning

The journey towards robust, production-scale AI agent systems still presents significant hurdles. Evaluating AI agents demands end-to-end measurements that reflect real interaction trajectories. Key metrics include end-to-end task performance, efficiency and cost, correctness of tool usage, trajectory quality, robustness, and security.

Reliability, reproducibility, and governance pose substantial limits to agent systems at scale. Long-running tasks amplify cumulative errors, and non-determinism complicates evaluation and debugging. Tool-centric agents introduce new security risks, such as prompt injection and side-effect actions. System-level trade-offs—latency versus accuracy, autonomy versus controllability, capability versus reliability—remain poorly understood.

Google's internal reinforcement learning (RL) framework, Forge, addresses the fundamental trilemma of system throughput, training stability, and agent flexibility when scaling RL for complex, real-world agents. Forge's holistic approach combines flexible system architecture, algorithmic design, optimized asynchronous scheduling, and extreme training-inference efficiency. It supports training arbitrary agent scaffolds through standardized interaction protocols, having processed over a hundred thousand distinct real-world agent scaffolds and environments. The system achieved a daily throughput of millions of samples and consistent reward convergence.

Source: swishdata.com

Google’s Forge framework offers a holistic approach to address the trilemma of system throughput, training stability, and agent flexibility for scaling RL in complex settings.

Forge's RL system comprises an agent side, middleware abstraction layer, and training/inference side. The agent side abstracts the general agent and its operational environment, while the middleware isolates it from the training/inference side. A Gateway server handles completion requests between the agent and LLM, and a distributed Data Pool asynchronously gathers rollout trajectories and reports. The training and inference side manages computational load, featuring an LLM Engine for high-throughput token generation and a Train Engine to update policies based on processed token sequences. Forge enables varied scaffold training without internal agent modifications, ensuring consistent optimization for both white-box and black-box agents.

Conclusion

The advent of AI agents represents a significant leap in enterprise automation, offering practical interfaces between natural language and complex computations. However, effective deployment at scale demands strategic architectural choices, robust governance models, and sophisticated learning frameworks. Solutions from major tech innovators highlight a clear focus on modularity, specialized coordination, and comprehensive control planes to overcome inherent scaling challenges. As these intelligent systems become more pervasive, the emphasis will remain on ensuring their reliability, security, and efficiency, continuously refining the balance between autonomy and human oversight.