Top 5 AI gateways for multimodal workloads in 2026

A multimodal LLM is a model that takes or produces more than text: images in, images out, recorded audio, synthesised speech, live voice and vector embeddings. Teams adopting these models quickly discover that their AI gateway was designed for chat completions, and that the other request types travel different paths: multipart file uploads, binary audio responses, WebSocket and WebRTC sessions, and batch jobs that finish hours later. This comparison is for platform and AI engineers who need one gateway in front of all of that traffic, not only the chat endpoint.

The question is narrower than “which gateway is best”. It is which gateways proxy speech, transcription, image generation, embeddings, realtime audio, files and batch across providers, and how they handle the parts that differ from chat: payload size, streaming audio, per-modality cost and caching, and fallbacks between providers that do not share an API shape. Every capability below comes from each vendor’s own documentation, read between 8 and 10 October 2026.

How the ranking was built

The ranking weights six criteria. A gateway scores well when its documentation states, per request type, what it supports, rather than implying that anything OpenAI-compatible will work.

Request-type coverage and cost tracking carry the most weight. Without the first, the gateway is not in the path; without the second, the spend reports it produces are wrong. Readers new to the category may want to start with the site’s explainer on what an AI gateway is; the broader ranking of AI gateways for 2026 covers governance, guardrails and MCP in more depth.

The five gateways at a glance

The five contenders cover the three ways a team can run a gateway: a self-hosted purpose-built service, a plugin set on an API gateway, and a hosted endpoint. The table summarises request-type coverage as each vendor documents it.

Figure 1: “Multimodal” is five request shapes, and a gateway built around the first one can quietly mishandle the other four.

1. Bifrost

Bifrost is an open-source AI gateway written in Go and released under Apache 2.0 by Maxim AI, with a paid enterprise tier. It exposes 25+ providers and 10,000+ models through one OpenAI-compatible API, and it also accepts native OpenAI, Anthropic, Bedrock, Google GenAI and Cohere SDK calls by changing the base URL. Bifrost’s published benchmark reports 11 µs of added overhead per request at 5,000 RPS on an AWS t3.xlarge.

It gets the longest entry here for a reason a reader can check: its documentation publishes a provider support matrix that lists, for each of roughly 30 providers, whether text, chat, responses, images, image edits, image variations, embeddings, text-to-speech, speech-to-text, files, batch, count tokens, rerank, OCR, video and passthrough are supported, each with its streaming variant. No other gateway in this comparison documents multimodal coverage at that granularity, and that is what the first criterion rewards.

Request-type coverage. The multimodal guide walks through vision input on /v1/chat/completions, image generation on /v1/images/generations, audio input to gpt-4o-audio-preview, text-to-speech on /v1/audio/speech and transcription on /v1/audio/transcriptions, including word- and segment-level timestamps. The matrix adds image edits and variations, video generation and remix, OCR, rerank and containers. OpenAI is supported on nearly every operation; Gemini covers images, embeddings, streamed TTS and STT, files, batch and video; Azure covers images, embeddings, streamed TTS, STT, files, batch and video.

Audio specialists. Speech traffic often goes to vendors that do not offer chat at all. The ElevenLabs provider guide maps speech (streaming and non-streaming), sound effects and transcription, voice settings such as stability and similarity boost, character-level timestamps, diarisation and pronunciation dictionaries. Groq, Mistral (with streamed transcription and OCR), Sarvam AI for Indic speech, Hugging Face and vLLM add further speech-to-text routes, and Runway and Runware handle video and image generation.

Realtime. Bifrost proxies realtime sessions over WebSocket and performs the WebRTC SDP exchange on /v1/realtime/calls. Its realtime API reference states that it “applies governance, observability, and key selection on connect”, and it can mint short-lived client secrets for browser clients, mapping each token back to the virtual key that requested it. That matters for voice agents, where the browser should never hold a provider key.

Caching. The semantic caching plugin caches “chat completions, text completions, the Responses API (including WebSocket), embeddings, transcriptions, speech, and image generation”, including streaming variants, which are replayed chunk by chunk. It runs in direct (hash) mode with no embedding provider, or adds similarity search on a miss. Direct mode is the right setting for generated media, where only exactly repeated prompts should replay.

Cost tracking. The model catalog prices each modality in its native unit: character-, token- and duration-based pricing for speech and transcription, per-image, per-pixel or token pricing for images, and per-second rates for video banded by resolution. Custom pricing overrides can be scoped to one request type, such as speech or image_generation, and budgets on virtual keys draw on those costs.

Files, batch and large payloads. Bifrost supports the OpenAI Files and Batch APIs with cross-provider routing to OpenAI, Anthropic, Bedrock (with S3 storage configured) and Gemini, and an async inference endpoint returns a job ID for every inference type except batches, files and containers, with signed webhooks on completion. Retries and fallbacks rotate keys on 429s, back off on 5xx errors and then move to the next provider in the chain, each with its own retry budget. Log exports can offload request and response bodies, including streamed chunks, to S3 or GCS while metadata stays searchable.

Enterprise tier. Clustering, guardrails (including AWS Bedrock Guardrails with image content analysis), SSO, RBAC, audit logs and log exports to object storage come with Bifrost Enterprise.

Best for: teams that want speech, transcription, images, embeddings, realtime voice and batch jobs governed, cached and costed by one self-hosted gateway.

2. LiteLLM

LiteLLM is a Python SDK and proxy from BerriAI with an MIT-licensed core and a separately licensed enterprise directory. Its supported-endpoints index is the longest in the category: audio transcriptions and speech, image generations, edits and variations, embeddings, files, batches, videos, OCR, rerank, realtime (including WebRTC) and pass-through routes, alongside chat and responses.

Strengths. Provider breadth per modality is the main one. The text-to-speech page lists OpenAI, Azure OpenAI, Azure AI Speech, AWS Polly, Vertex AI, Gemini, ElevenLabs and MiniMax, plus a bridge that exposes chat-only TTS models such as Gemini’s through /audio/speech. Transcription reaches OpenAI, Azure, Vertex AI, Gemini, Deepgram, Groq, Fireworks AI, OVHcloud and Mistral’s Voxtral, and image generation adds Bedrock, Black Forest Labs, Recraft and others. Each of those pages marks cost tracking, logging, fallbacks and load balancing as supported, and the realtime proxy routes OpenAI, Azure and xAI voice models.

Trade-offs. Feature depth thins outside chat. Guardrails apply to input text on speech and to output text on transcription, in both cases “non-streaming only”. The caching docs enable caching by default for chat, completions, embeddings, transcriptions, rerank, responses and messages, which leaves speech and image generation out. Setting a maximum upload size for transcription is marked as an enterprise feature, and batch cost tracking is “LiteLLM Enterprise only”. Multi-instance deployments need PostgreSQL and Redis, and the site’s LiteLLM alternatives guide covers the operational record in more detail.

Best for: Python-first teams that need the widest list of speech and image providers and are prepared to run the proxy’s supporting databases.

3. Vercel AI Gateway

Vercel AI Gateway is a hosted gateway callable from any infrastructure. Its modalities page covers text, image generation, video generation, realtime voice, speech-to-text, text-to-speech, embeddings, reranking and typed decisions through one API, and a separate inputs page covers images, PDFs, audio and video sent to models.

Strengths. It is the simplest way to put realtime voice behind a gateway. The realtime docs pair a server-side token helper, so the API key never reaches the browser, with a React hook that handles the WebSocket, microphone capture and playback. Streaming transcription connects to the model over a WebSocket and returns transcript updates as audio arrives. Model fallbacks use a models array that, according to the fallback docs, “works across every AI Gateway API format”, and the pricing page states that the gateway charges no markup on tokens, including with bring-your-own-key.

Trade-offs. It cannot be self-hosted. Speech-to-text and text-to-speech are both labelled beta with access “rolling out gradually”, and text-to-speech lists Google, Microsoft and OpenAI voices rather than audio specialists. Realtime sessions close after 25 minutes or 5 idle minutes, do not accept image input and do not resume on reconnect. Response caching for media is not published; the caching it documents is provider prompt caching. Files and Batch API support is not published.

Best for: application teams on any stack that want voice agents, transcription and image generation behind one hosted endpoint without running anything.

4. Kong AI Gateway

Kong AI Gateway is a set of AI plugins on Kong Gateway, managed self-hosted or through Konnect. The AI Proxy plugin translates requests to and from an OpenAI-style format using route types, and its table is explicit about which modalities exist and from which gateway version.

Strengths. Coverage is documented per route type: llm/v1/embeddings, llm/v1/files and llm/v1/batches from 3.11, audio/v1/audio/speech, audio/v1/audio/transcriptions and audio/v1/audio/translations from 3.11, image/v1/images/generations and image/v1/images/edits from 3.11, and video/v1/videos/generations from 3.13. In native-format mode, the docs say the gateway “will still provide analytics, logging, and cost calculation”. For organisations that already run Kong, media traffic picks up the same consumers, rate limits and logging plugins as every other API.

Trade-offs. Realtime is where the enterprise line falls. The realtime/v1/realtime route type appears in AI Proxy Advanced, which, like the AI Semantic Cache plugin, is “only available as part of our AI Gateway Enterprise offering”. Load balancing across targets and semantic routing also sit in AI Proxy Advanced. Kong does not publish how cost calculation handles character- or duration-priced audio, and semantic caching needs a vector database alongside the gateway.

Best for: organisations already running Kong Gateway or Konnect that want speech, image and batch traffic under the policies they already operate, with an enterprise licence if realtime voice is in scope.

5. Cloudflare AI Gateway

Cloudflare AI Gateway runs on Cloudflare’s network and takes a different approach from the other four: rather than translating every modality into one schema, it proxies each provider’s native endpoint and adds caching, analytics, logging and rate limiting around it.

Strengths. Audio and image specialists are first-class providers. The provider list includes ElevenLabs, Deepgram and Cartesia for speech, and Ideogram, Fal AI and Replicate for generated media, alongside OpenAI, Google AI Studio, Vertex AI, Bedrock and Workers AI. The realtime WebSockets API supports “multimodal interactions such as text, audio, and video” for OpenAI, Google AI Studio, Cartesia, ElevenLabs, Fal AI and Deepgram on Workers AI, with browser authentication through sec-websocket-protocol. Its caching works for image responses as well as text.

Trade-offs. It is hosted only. Caching is “supported only for text and image responses, and it applies only to identical requests”, and the limits page caps cacheable requests at 25 MB. The costs page states that cost metrics “are only available for endpoints where the models return token data”, which leaves character- and duration-priced audio outside its spend figures. Fallbacks are documented for the deprecated Universal endpoint and for Dynamic Routing, and Files and Batch API proxying is not published.

Best for: teams already on Cloudflare that want caching, analytics and realtime proxying in front of audio and image specialists without changing provider SDKs.

What actually differs for multimodal traffic

All five proxy more than chat. They differ on five points that only show up once speech, images and voice reach production.

Request shapes the gateway understands

A chat request is a small JSON body with a small JSON or SSE response. Transcription is a multipart upload; OpenAI’s speech-to-text guide accepts files “up to 25 MB” in seven formats. Text-to-speech returns binary audio. Realtime is a long-lived socket. Batch is a file upload followed by a job that, per the OpenAI Batch API guide, “completes within 24 hours” at a 50 per cent discount, with up to 50,000 requests and a 200 MB input file. Bifrost, LiteLLM and Kong translate all of these into one OpenAI-style schema; Vercel does so for everything except files and batch; Cloudflare passes native shapes through.

The translation approach lets one client library call ElevenLabs, Gemini and OpenAI speech the same way, and lets a fallback chain cross providers. Passthrough keeps provider-specific features intact but means a fallback from one speech vendor to another needs two request formats in application code.

Streaming audio and realtime sessions

Streaming matters in three places: synthesised speech that starts playing before generation finishes, transcription that updates while someone is still talking, and two-way voice. OpenAI’s Realtime API guide describes sessions that connect “over WebRTC in the browser or WebSocket on the server”, with ephemeral client secrets minted server-side. A gateway in that path must forward the upgrade, keep the socket alive, and authorise the session without exposing provider keys.

Teams running a proxy such as NGINX in front of any self-hosted gateway need to disable response buffering and pass upgrade headers; Bifrost’s deployment guides include streaming-safe NGINX settings for SSE and WebSocket traffic.

Cost in the right units

Media is not billed in tokens alone. Speech is priced by characters or by duration, transcription by audio minutes or audio tokens, image generation per image at a given size and quality, and Claude prices image input by visual tokens, where each 28 by 28 pixel patch is one token according to its vision documentation. A gateway that estimates cost from token counts in the response will record a speech request as nearly free.

Bifrost’s catalogue carries per-character, per-second, per-image and per-pixel fields and lets overrides target a single request type. LiteLLM and Kong state that cost tracking works for these endpoints without publishing the unit model, and LiteLLM gates batch cost tracking behind its enterprise licence. Cloudflare limits cost metrics to token-returning endpoints. Vercel publishes per-model pricing and zero markup. For a vendor-side view of how per-request cost ties into tracing and spend dashboards, see this overview of gateway-level AI observability.

Caching media responses

Caching generated images and speech saves the most money per hit, because each miss is slow and expensive, but it is correct only for exactly repeated inputs: a product thumbnail regenerated on every page view, or a fixed IVR prompt spoken in a fixed voice. Semantic similarity is a poor fit for media, because two prompts that are close in meaning should usually produce different images.

The site’s explainer on semantic caching for LLMs covers thresholds and invalidation; for media, exact-match mode is the safer default.

Large payloads, logs and fallbacks

Media payloads are measured in megabytes. Claude accepts up to 10 MB per base64 image and 32 MB per standard request, and its documentation warns that resending base64 images on every turn “can significantly increase request size and latency”. The Gemini Files API should be used when a request exceeds 100 MB, and stores files for 48 hours with a 2 GB per-file limit. Gateways inherit those limits, and add their own when they log or cache the body: Cloudflare’s legacy logs store 10 MB per log and its Logpush jobs 1 MB, while Bifrost Enterprise can offload request and response bodies to object storage.

Fallbacks are the other half. A fallback from one transcription provider to another works only if the gateway can rebuild the multipart request in the second provider’s format and the second provider accepts the same file. LiteLLM marks fallbacks as supported on its speech, transcription and image pages, Vercel says its models fallback option works across every API format, and Bifrost’s API reference states that requests to its unified API can include fallback models tried when the primary fails; Kong applies failover inside AI Proxy Advanced; Cloudflare’s documented fallbacks run through Dynamic Routing. Whichever gateway is chosen, keep fallback models compatible with the input type, a point Vercel’s own file-input docs make explicitly.

Figure 2: For media requests, the metering and logging steps are where chat-first gateways most often get the numbers wrong.

Recommendations keyed to constraints

Choose by the request types in scope and where the gateway is allowed to run, then check the beta and enterprise labels on the specific features needed.

• Every modality, self-hosted. Bifrost. It is the one gateway here whose documentation covers speech, transcription, images, video, embeddings, files, batch and realtime per provider, with caching and per-modality pricing on media requests, under an Apache 2.0 licence. Clustering, guardrails, SSO and log offload are in the enterprise tier.
• The widest list of speech and image providers. LiteLLM, if the team is comfortable with Python, PostgreSQL and Redis, and does not need speech or image caching by default.
• Voice agents with zero operations. Vercel AI Gateway. Plan around the 25-minute realtime session limit and the beta status of speech and transcription.
• Kong already in production. Extend it. Speech, transcription, images and batch are covered by route types; budget for AI Gateway Enterprise if realtime voice or semantic caching is required.
• Cloudflare already in production. Cloudflare AI Gateway for analytics, caching and realtime proxying in front of ElevenLabs, Deepgram, Cartesia and similar providers, with media cost tracked on the provider side.
• Large batch and file workloads. Use a gateway that proxies the Files and Batch APIs (Bifrost, LiteLLM or Kong), so asynchronous jobs share keys, budgets and logs with synchronous traffic.

Figure 3: The deployment constraint and the platform already in place settle most multimodal shortlists.

For readers comparing these tools on chat routing and failover alone, Maxim AI’s production-ready comparison of LLM gateways covers that ground from the vendor’s side.

The verdict

Multimodal support in AI gateways has moved from “OpenAI-compatible, so it probably works” to documented request types, but depth still varies by modality. Most gateways in this comparison proxy speech, transcription and image generation; far fewer cache those responses, meter them in their native units, or carry realtime sessions with the same governance as chat. We think that gap, more than raw provider counts, is what separates a gateway that handles multimodal LLM traffic from one that merely forwards it.

Bifrost ranks first because its documentation answers every one of the six criteria per provider and per request type, and because media caching, per-modality pricing, realtime with key selection on connect, and cross-provider files and batch all sit in the open-source gateway. LiteLLM is the broadest alternative, Vercel the easiest hosted path to voice, and Kong and Cloudflare the right extensions for teams already on those platforms. Before committing, list the request types the product will send in the next year and check each against the vendor’s own matrix; the gateway that is right for chat is not always right for speech.

Feature claims are drawn from public documentation as of October 2026; check current docs before deciding.