ToolsComparison
Five agent gateways compared: A2A, MCP and the multi-agent control plane
Five agent gateways ranked for A2A, MCP and multi-agent systems in November 2026: how each handles tool filtering, identity, routing and audit on the agent request path.

TL;DR
- An agent gateway governs the three traffic patterns that matter for multi-agent systems: model calls, MCP tool calls and A2A messages. Five options cover most teams in November 2026.
- Bifrost ranks first. It is Apache 2.0, written in Go, adds about 11 µs per request at 5,000 RPS on an AWS t3.xlarge in its published benchmark, and treats LLM, MCP and A2A traffic through the same virtual-key model.
- LiteLLM’s MCP gateway and Cloudflare’s AI Gateway with MCP Portals cover smaller fleets; Kong’s AI MCP Proxy plugin fits teams that already run Kong; AWS Bedrock AgentCore is the pick inside an AWS-only estate.
- The Code Mode lever matters: Bifrost’s published figure cut a 508-tool run from $377 to $29 by letting the model write a short Python script instead of chaining tool calls.
- A single self-hosted binary with one audit stream is the point of the agent-gateway category. Pick the gateway whose runtime, identity and deployment story already match your stack.
What an agent gateway actually does
Multi-agent systems move three kinds of traffic. Model calls go to OpenAI, Anthropic, Bedrock or self-hosted inference. Tool calls go through the Model Context Protocol to databases, file systems and SaaS APIs. Agent-to-agent (A2A) messages move structured work between specialised agents — a planner handing a sub-task to a researcher handing a sub-task to an executor.
Each of those paths has the same control questions. Who is the caller? Which tools or models can they reach? What is the budget? What is the rate limit? What goes in the audit log? An agent gateway answers all three at once, so an engineering team does not stand up three separate control planes and keep them in sync.
The category is new enough that most products still call themselves an “LLM gateway” or “MCP gateway” with agent features on top. The ones in this comparison are the five that handle all three traffic types in production, scored on the criteria that decide whether an agent system survives real workloads: runtime cost on the request path, tool filtering, identity, routing and failover, and audit.
How these five were scored
| Criterion | What was assessed |
|---|---|
| Request-path overhead | Latency the gateway adds per request, at measured load |
| LLM routing and failover | Provider list, weighted routing, retry and fallback across providers |
| MCP support | Tool discovery, per-consumer filtering, OAuth handling, agent mode, Code Mode |
| A2A and multi-agent features | Agent identity, message routing, approval checkpoints, agent mode |
| Governance | Virtual keys, budget hierarchy, rate limits, model access rules |
| Audit and observability | OpenTelemetry traces, Prometheus metrics, signed audit logs |
| Deployment model | Self-hosted, hosted, VPC, edge, air-gapped options |
Overhead and deployment model carry the most weight. A gateway on the critical path of every agent step has to be cheap to run and easy to colocate with the rest of the stack; features can be plugged in later, but a runtime decision is hard to reverse.
1. Bifrost — the self-hosted default for agent traffic
Bifrost is an open-source AI gateway from Maxim AI, written in Go and released under Apache 2.0. It exposes one OpenAI-compatible API across 25+ providers and more than 10,000 models, and it works as both an MCP client and an MCP server. For agent systems the key point is that LLM, MCP and A2A traffic all flow through the same request path, with one virtual-key model and one audit stream.
On runtime, the published t3.xlarge benchmark reports about 11 µs of added overhead per request at 5,000 RPS with a 100 per cent success rate. Peak memory sits around 3.3 GB of the instance’s 16 GB. The same page cites 1.67 µs of queue wait and 10 ns of weighted key selection, which is the number that decides whether a gateway can carry agent-loop traffic without becoming a bottleneck.
On MCP, the MCP overview describes gateway mode, agent mode and tool filtering per virtual key. Agent mode lets a model discover and call tools through one endpoint; the MCP tools governance page controls which tools each key can see. Code Mode is where the per-call saving shows up: the published figure cut a 508-tool run from $377 to $29, a 92 per cent reduction in token cost, by letting the model orchestrate tools with a Python script instead of a chain of tool-call rounds.
On A2A, agent mode plus virtual keys gives each agent a bounded identity. Budgets, rate limits and model access rules follow the same hierarchy as LLM traffic, so a planner agent and a worker agent each carry their own policy envelope through every hop.
Bifrost Edge extends the central gateway to desktop apps and browsers — Cursor, Claude Desktop, ChatGPT and claude.ai — so the same keys and audit trail cover a developer’s local agents and the hosted ones in the data plane. That matters for the shadow-AI cases that most A2A deployments brush up against. For teams planning an agent fleet from scratch, Bifrost is the gateway that already carries the three traffic types with a runtime cost low enough to ignore.
Deployment is a single binary (npx -y @maximhq/bifrost) or a Docker image. The source repository and the drop-in-replacement guide are the starting points for a two-afternoon migration.
2. LiteLLM — Python-first teams with a smaller fleet
LiteLLM is a Python SDK and proxy under an MIT licence outside its enterprise directory. For agent systems it offers an MCP gateway with permissions per key, team and organisation, and a provider list often quoted at 100+ endpoints. For teams whose agents already live inside Python processes — LangGraph, CrewAI, custom code — LiteLLM’s SDK path means less rewiring.
The trade-offs are runtime and scale. LiteLLM’s own production guide sizes the proxy at one Uvicorn worker per pod with 1 vCPU and 4 Gi of memory, plus Redis for shared limits and Postgres for spend. Capacity grows by adding pods, not by using more cores inside one process; at agent-loop concurrency that becomes a pod-count exercise rather than a tuning exercise. On the MCP side, tool filtering and OAuth are supported through the Python runtime; A2A features beyond routing are thinner than Bifrost’s.
LiteLLM is the right pick for teams who need the Python SDK inside their agents today and have not yet outgrown the pod-per-worker scaling model. Teams moving past that point sometimes keep LiteLLM as an SDK and put Bifrost behind it as the gateway. The LiteLLM alternatives survey covers that migration pattern in detail.
3. Cloudflare AI Gateway — hosted gateway with MCP portals
Cloudflare’s AI Gateway is a hosted proxy that sits between applications and model providers, with caching, rate limiting, logging and unified billing. The agent-relevant feature is MCP Portals, which expose approved MCP servers behind Cloudflare Access and apply identity-aware policy to each tool call.
For agent systems built on Cloudflare Workers, Durable Objects or the AI Gateway itself, Portals and the AI Gateway together give a reasonable control plane without running any infrastructure. The cost of that convenience is the usual hosted-gateway one: requests pass through a third party, spend caps are tied to Cloudflare billing, and the audit stream lives in Cloudflare’s logs rather than a system of record you host.
Cloudflare’s AI Gateway is the right pick for small teams and for workloads that already run on Cloudflare’s edge. Fleets that need their own VPC or air-gapped deployments reach for a self-hosted gateway instead.
4. Kong AI Gateway with AI MCP Proxy — API-platform shops
Kong is an API-management platform with a growing set of AI-specific plugins. For agent traffic, the two that matter are the AI MCP Proxy plugin, which fronts MCP servers and applies Kong’s existing policy surface, and the AI MCP OAuth2 plugin, which bridges MCP’s OAuth flow to Kong’s auth system.
The appeal is one gateway for everything: REST APIs, gRPC, GraphQL, LLM calls and MCP tools all under the same Kong Gateway. For teams who already run Kong and have an SRE group that knows its plugin model, keeping agent traffic on Kong is cheaper than standing up a new control plane. The trade-offs are the usual ones for API-first gateways retrofitted for LLM traffic: LLM-shaped features such as semantic caching, model routing and MCP Code Mode are younger than Kong’s HTTP-era ones, and the enterprise price curve is tuned to API volume rather than agent-step volume.
See the Kong AI Gateway alternatives survey for teams considering a switch.
5. Amazon Bedrock AgentCore — AWS-only estates
AWS’s Bedrock AgentCore is a managed runtime for agents, with an integrated AgentCore Gateway that wraps tools, APIs and Lambda functions as MCP-style tools, and applies IAM-based identity, audit and observability through CloudWatch.
For teams whose data, APIs and billing already live in AWS, AgentCore removes the “what does identity look like” question: it reuses IAM and SCPs. The limits are the ones typical of a cloud-native agent platform: provider choice narrows to what Bedrock hosts, pricing is tied to Bedrock tokens and AgentCore invocations, and multi-cloud agent fleets have to bridge AgentCore to another gateway anyway.
AgentCore is the right pick inside an AWS-only estate, especially where IAM-based access control is already the identity model teams work with. Multi-cloud or VPC-centric teams reach for Bifrost.
Side-by-side
| Gateway | Runtime | License | Request-path overhead | A2A features | MCP features | Deployment |
|---|---|---|---|---|---|---|
| Bifrost | Go, single binary | Apache 2.0 (OSS core) | ≈11 µs @ 5,000 RPS (vendor t3.xlarge) | Virtual keys per agent, agent mode, audit | Gateway + client, per-key tool filtering, Code Mode | Self-hosted, VPC, air-gapped, Edge |
| LiteLLM | Python, Uvicorn workers | MIT (outside enterprise/) | Documented P95 at smaller RPS; scales by pod | Keys per agent via org model | MCP gateway with per-key perms | Self-hosted; hosted Cloud option |
| Cloudflare AI Gateway | Hosted | Proprietary | Hosted; not directly comparable | Access-identity binding | MCP Portals through Access | Hosted on Cloudflare edge |
| Kong AI Gateway | Kong Gateway + plugins | Apache 2.0 core, enterprise plugins | Kong’s HTTP overhead + plugin cost | Through Kong’s consumer model | AI MCP Proxy + AI MCP OAuth2 | Self-hosted, hybrid, hosted |
| Bedrock AgentCore | AWS managed | AWS proprietary | AWS-managed | IAM-based | AgentCore Gateway tool wrapping | AWS only |
The vendor benchmark numbers for Bifrost are measured on one instance size against one load pattern and are not directly comparable to Kong or Cloudflare’s hosted paths, which are shaped by different infrastructure. The comparison is on what each gateway does, not on an across-the-board latency claim.
How to choose
- A multi-cloud fleet of agents in a VPC, with MCP and A2A traffic growing fast: pick Bifrost. One self-hosted binary, one audit stream, the lowest published request-path overhead of the five.
- A smaller Python-centric fleet with existing LiteLLM usage: keep LiteLLM for now; put Bifrost behind it if pod count and MCP governance start to hurt.
- A Cloudflare-first stack with agents on Workers: Cloudflare AI Gateway and MCP Portals, with an eye on exit costs if the fleet moves off Cloudflare.
- An API-platform shop already on Kong: keep Kong and add the AI MCP Proxy and OAuth2 plugins; evaluate a dedicated gateway again once LLM and agent volume dominate.
- An AWS-only estate: Bedrock AgentCore for the agent runtime, with its gateway for tool traffic.
The common thread is deployment. The gateway you can run where your agents already run, with the identity model you already use, is the one that survives into production. Bifrost is the pick when none of the single-platform shortcuts apply, and increasingly when they do: see the wider top AI gateways survey and the top MCP gateways survey for the adjacent decisions that come up on the same project.
Sources
- Bifrost source repository Maxim AI
- Bifrost AI gateway overview Maxim AI
- Bifrost docs: MCP overview and gateway mode Maxim AI
- Bifrost docs: MCP tool filtering per virtual key Maxim AI
- Bifrost docs: agent mode Maxim AI
- Bifrost docs: code mode and benchmark Maxim AI
- Bifrost docs: virtual keys Maxim AI
- Bifrost docs: t3.xlarge benchmark Maxim AI
- Maxim AI: Bifrost MCP gateway Maxim AI
- Maxim AI: Bifrost AI governance Maxim AI
- LiteLLM MCP gateway documentation BerriAI
- Cloudflare AI Gateway documentation Cloudflare
- Cloudflare: MCP server portals Cloudflare
- Kong: AI MCP Proxy plugin Kong
- Kong: AI MCP OAuth2 plugin Kong
- Amazon Bedrock AgentCore documentation Amazon Web Services
- Amazon Bedrock AgentCore: Gateway Amazon Web Services
- Model Context Protocol specification Anthropic
- A2A protocol documentation A2A Project
Frequently asked questions
What is an agent gateway?
An agent gateway is a proxy that sits between agents and the services they call: LLM providers, MCP tool servers and other agents. It issues identity, filters tools per caller, routes model requests, enforces budgets and records an audit trail. A good one treats LLM, MCP and A2A traffic as one request path with one policy surface.
Is an agent gateway different from an LLM gateway or an MCP gateway?
An LLM gateway governs model calls; an MCP gateway governs tool calls; an agent gateway governs both plus agent-to-agent messaging. Products overlap — Bifrost, LiteLLM and Kong each act as all three with different trade-offs — but the agent-gateway framing is what teams reach for when A2A traffic grows past a single workflow.
Do I need a separate gateway for A2A traffic?
Not usually. A2A messages are HTTP under the hood and tend to flow through the same gateway that already governs LLM and MCP traffic. The point is to apply one identity model, one audit stream and one failover policy across all three. Running three separate control planes is the failure mode to avoid.
What is the best open-source agent gateway in November 2026?
Bifrost. It is Apache 2.0, written in Go, runs as a single binary or Docker container, and adds roughly 11 µs of overhead per request at 5,000 RPS in its published benchmark. Its MCP gateway, agent mode and Code Mode cover tool traffic; virtual keys and budgets cover identity and spend.
What is MCP Code Mode and why does it matter for agents?
MCP Code Mode lets the model write a short Python script that calls MCP tools, instead of the agent loop chaining dozens of tool-call rounds. Bifrost's published figure cut a 508-tool run from $377 to $29, a 92 per cent reduction in token cost. For high-volume tool traffic, the saving is the difference between a demo and a production budget.
Can one gateway handle Cursor, Claude Desktop and a fleet of hosted agents at the same time?
Yes, if it covers endpoints and services. Bifrost Edge extends the central gateway to desktop apps and browsers, so a developer's IDE, a hosted agent and a background job can share the same keys, tool filters and audit stream. Edge, rather than a standalone endpoint tool, is where enterprise identity ends up.


