In part one of this series, we discussed the importance of MCP (Model Context Protocol), central tool registry, proxied access, and the controller/worker hierarchy. Now in part two we provide code examples and delve deeper into dynamic tool registration, remote tool execution, and our take on the chat processing loop.
Implementations and Code Examples
To illustrate how MCP proxies work in practice, let’s examine a simplified example of a proxy server that connects to remote MCP services. This approach demonstrates the core architectural patterns that emerge when accessing multiple MCP servers through a single point of control.
Understanding the MCP Server Flow
Before diving into the proxy implementation, it’s essential to understand how MCP servers operate. A standard MCP server exposes three main capabilities: tools (functions that the AI can call), resources (data sources that the AI can read), and prompts (templates that the AI can use). The server handles capability discovery through a negotiation process where clients ask, “What can you do?” and servers respond with their available capabilities.
The challenge with direct MCP connections is that each server requires its own connection management, authentication, and error handling. When you have multiple teams providing different MCP servers—database tools from the data team, email tools from communications, analytics tools from business intelligence—clients end up managing dozens of connections with different authentication schemes and reliability characteristics.
Proxy Architecture
A proxy server addresses this complexity by presenting a single MCP interface that routes requests to multiple backend servers. The proxy handles three key responsibilities: capability aggregation (discovering and exposing tools from all backend servers), request routing (directing tool calls to the appropriate backend), and connection management (handling authentication and error recovery for backend connections).
This connection pattern demonstrates a stateless proxy approach where the MCP proxy establishes temporary connections to remote MCP servers for tool discovery and execution. The proxy fetches available tools during each connection session and creates local representations, allowing it to present a stateless HTTP interface to clients while periodically connecting to backend MCP servers. However, this implementation lacks connection persistence and failure recovery mechanisms—each interaction requires establishing a new connection to the remote server.
Dynamic Tool Registration
The most complex part of implementing a proxy is dynamically registering tools from backend servers. MCP tools have schemas that define their parameters and behavior; the proxy must discover these schemas and create local proxy functions that match the original tool signatures:
A dynamic registration approach is necessary because at development time, you don’t know what tools will be available from backend servers. The proxy discovers capabilities at runtime and creates matching local functions that maintain the original tool signatures while adding proxy logic for routing and error handling.
Remote Tool Execution
Tool execution demonstrates how proxies handle the complexity of maintaining connections to backend servers while providing a synchronous-looking interface to clients:
This implementation shows how proxies must handle the inherent unreliability of network connections while providing predictable behavior to AI agents. Timeout handling, connection recovery, and error normalization are essential considerations for production deployments where tool failures shouldn’t break AI conversations.
Chat Processing Loop
Now let’s transition to the client. We don’t delve into tool-limiting strategies here, but we do want to take a quick look at the chat processing loop. We spent a considerable amount of time talking about how to manage a large number of tools and how to set your enterprise up for governance. Now, let’s look at how client applications make use of tools and the main chat processing logic that handles conversation flow and tool execution:
This client implementation demonstrates the fundamental architecture of an MCP-enabled conversational AI system, showcasing how to bridge the gap between language models and distributed tool ecosystems. Let’s review of the entire client application and point out several important factors to consider when deploying MCP in enterprise applications.
Connection and Tool Discovery
The client establishes a connection to the MCP server and dynamically discovers available tools, converting MCP tool schemas into OpenAI’s function-calling format. This abstraction allows the same client to work with any MCP-compliant tool server without modification.
Conversation State Management
The implementation maintains a complete conversation history, preserving context across multiple tool invocations. Conversation State Management is crucial for complex, multi-step workflows where later tool calls depend on earlier results, as it involves properly handling the three-way conversation between the user, assistant, and tools.
Tool Execution Flow
The core loop implements sophisticated orchestration: The LLM decides which tools to call and when, the client executes those calls against MCP servers, and results feed back into the conversation context. The loop limit prevents infinite recursion while allowing flexible multi-step reasoning.
Error Handling and Resilience
The current error handling needs significant enhancement, including retry logic with exponential backoff, connection failure recovery, and graceful degradation when tools become unavailable. Production systems require sophisticated error categorization to distinguish between retriable failures and permanent errors.
Performance Optimization
Several critical optimizations are missing tool definition caching to avoid repeated discovery calls, parallel execution of independent tool calls, and streaming responses for a better user experience.
Resource Management
The implementation lacks proper resource controls, including conversation history truncation strategies, configurable timeouts for tool calls, memory management for long-running sessions, and concurrency limits to prevent overwhelming backend services.
Advanced Tool Management
While fundamental tool discovery works for demos, enterprise systems need dynamic tool filtering based on user permissions, conversation context, and system load. Tool selection strategies become critical when dealing with hundreds or thousands of available capabilities.
Scalability and Reliability
Production deployments require sophisticated connection management, circuit breaker patterns for failing services, load balancing across multiple MCP server instances, and backup strategies when primary tool providers become unavailable.
Context Optimization
Long conversations require intelligent context management, including the automatic summarization of distant history, selective retention of relevant information, and compression of tool results to maintain performance while preserving essential context.
This implementation provides a solid foundation that demonstrates the essential patterns of MCP integration, including dynamic tool discovery, conversation state management, and LLM-driven tool orchestration. These building blocks enable AI agents to scale across distributed tool ecosystems while maintaining clean architectural separation. However, the gap between this functional prototype and production-ready enterprise systems is substantial, requiring significant investment in reliability, security, and operational excellence.
The Model Context Protocol isn’t just another integration standard—it’s the infrastructure foundation that will determine whether your AI initiatives scale beyond demos and proof-of-concepts into genuine business value. Organizations that understand this distinction and invest accordingly will build sustainable competitive advantages. Those that don’t will find themselves trapped in a maze of brittle, ungovernable AI integrations that become more expensive and riskier with each passing quarter.
The governance patterns we’ve explored (central registries and proxied access) aren’t theoretical constructs. They’re battle-tested approaches to managing distributed systems, adapted for the unique requirements of AI agents. Organizations already operating large-scale microservices architectures have a significant advantage here, as they understand the operational complexity that emerges when simple protocols meet real-world deployment requirements.
The FactSet authors of this article are Tony Piazza, Principal Software Architect, and Telmuun Enkhbold, Machine Learning Operation Engineer.
This blog post is for informational purposes only. The information contained in this blog post is not legal, tax, or investment advice. FactSet does not endorse or recommend any investments and assumes no liability for any consequence relating directly or indirectly to any action or inaction taken based on the information contained in this article.