LLM gateways and observability for enterprise buyers
Asked as “LLM gateway”, and as “LLM observability and routing platform”, on behalf of an enterprise B2B company. 48 first choices recorded across the direct, paraphrase, budget and scale prompts, twelve models each.
By buyer segment
The same question asked on behalf of a different buyer. Each standing is computed within its segment; they sit side by side and are never added together.
01The standing
Share is the count of first choices across the direct, paraphrase, budget and scale prompts, over all twelve models, for an enterprise B2B company. Ordered by share.
ProductFirst-choice shareNegative rateLabelsQuadrant
Show the one product at 0%, ordered by negative rate
Bars are the share of first choices, 0 to 100Every product with at least 10 labels here. Every product name links to its vendor page.
Recommended versus criticized
Every product with at least 10 labels here, on both axes. The 30% line names a quadrant, not the verdict above: that one needs more than 40%.
Criticized challengerCriticized default
Negative label rate →
01
02
03
04
05
06
07
08
09Accepted challengerEndorsed leader
0%First-choice share → · lines at 30% share and 25% negative40%
Key
01LiteLLM29%
LiteLLM29%02Bifrost27%
Bifrost27%03Portkey15%
Portkey15%04Langfuse4%
Langfuse4%05Kong AI Gateway2%
Kong AI Gateway2%06Cloudflare AI Gateway2%
Cloudflare AI Gateway2%07Helicone2%
Helicone2%08OpenRouter2%
OpenRouter2%09NeuralTrust TrustGate0%
NeuralTrust TrustGate0%02What they warned about
One of twelve models held their first choice under the paraphrase. Claude Haiku 4.5, GPT-5.4 mini, Gemini 3.5 Flash, Perplexity Sonar, Grok 4.1 Fast, Mistral Small, DeepSeek V4 Flash, Llama 4 Maverick, Kimi K2, GLM 4.7 FlashX and MiniMax M2.5 changed. A high negative share on a product with few labels is a warning. A low share on a product with many labels is salience, not sentiment.
LiteLLM24%
13 of 54 labels negative · 10 of 12 models · 4 hard negative
“the comparison explicitly says to avoid it for customer-facing apps at scale, real-time features, and workloads needing 99.9%+ uptime” Perplexity Sonar, negative prompt
OpenRouter43%
10 of 23 labels negative · 6 of 12 models · 4 hard negative
“**Avoid for:** Healthcare (HIPAA), financial services, government, or any jurisdiction requiring strict data localization.” Qwen 3.7 Flash, negative prompt
Cloudflare AI Gateway33%
6 of 18 labels negative · 6 of 12 models
“The Catch: It requires routing your traffic through Cloudflare's servers. If you have strict "zero data egress" ... a self-hosted option like LiteLLM or Bifrost is better suited.” Gemini 3.5 Flash, direct prompt
Kong AI Gateway13%
4 of 30 labels negative · 4 of 12 models
“Generic Cloudflare AI Gateway / Kong (If Misconfigured) **Not inherently bad,** but enterprises must be cautious” Qwen 3.7 Flash, negative prompt
03What they cite
Citations exist only for the models that return a source list: twelve of the twelve in this edition, and all six flagship models on the expanded tier.
Sites the answers cite
70 of 72 answers in this category came back with a source list, from 12 of 12 models: citations where the model returns them, or the search results it consulted. 6 of those lists are Google grounding redirects that name no site and are left out of the counts. 991 links across 261 sites, every framing counted. Ranked by the number of answers carrying the site or page.
47 answers · 147 citations · 10 models
vendor site · TrueFoundry32 answers · 53 citations · 10 models
vendor site · CloudZero25 answers · 25 citations · 9 models
23 answers · 31 citations · 9 models
21 answers · 22 citations · 9 models
vendor site · Dataiku19 answers · 20 citations · 8 models
vendor site · Braintrust18 answers · 24 citations · 8 models
vendor site · Respan17 answers · 17 citations · 9 models
vendor site · LLMGateway.io16 answers · 41 citations · 9 models
14 answers · 14 citations · 7 models
vendor site · OpenRouter12 answers · 21 citations · 7 models
12 answers · 13 citations · 7 models
Pages the answers cite
The ten pages named in the most answers, by full address. A page here is one the models returned with a recommendation, not one the index endorses.
25 answers · 9 models
19 answers · 8 models
19 answers · 10 models
18 answers · 9 models
17 answers · 9 models
15 answers · 8 models
14 answers · 7 models
14 answers · 8 models
12 answers · 6 models
10 answers · 7 models
04How they answered
Six framings of the same buying question, each sent to every model in a fresh session with search on. One row per model, so a row shows whether it held its answer under rewording, what it named when cost was the constraint, and what it argued against. Computed from the raw judge labels.
ModelDirect“What is the best LLM gateway for an enterprise B2B company?”Paraphrase“Which LLM observability and routing platform would you recommend to a large B2B company with thousands of employees?”Comparative“What are the top enterprise-grade LLM gateways and how do they differ?”Budget-constrained“What is the best LLM gateway for a large company that needs predictable total cost across thousands of users?”Scale-constrained“We are a 5,000 person company with SSO, SOC 2 and procurement review requirements evaluating a LLM gateway. What should we look at?”Negative“Which LLM gateways should a large enterprise avoid or be cautious about?”
Claude Haiku 4.5BifrostTrueFoundryChangedBifrostagainst: Cloudflare AI Gateway, LiteLLMBifrostno first choiceagainst: LiteLLM
Three alternatives
Kong AI Gateway, LiteLLM, PortkeyFive alternatives
Datadog, LangSmith, MLflow, New Relic, OpenObserveTwo alternatives
Kong AI Gateway, PortkeyTwo alternatives
Cloudflare AI Gateway, Kong AI GatewayGPT-5.4 miniAmazon Bedrock AgentCore GatewayLangfuse, OpenRouterChangedKong AI Gateway, PortkeyKong AI Gatewayno first choicenothing named
Three alternatives
Kong AI Gateway, LiteLLM, PortkeyThree alternatives
Kong AI Gateway, LiteLLM, PortkeyOne alternative
LiteLLMTwo alternatives
LiteLLM, OpenRouterGemini 3.5 FlashLiteLLMagainst: Cloudflare AI GatewayLiteLLM, PortkeyChangedagainst: Datadog LLM Observability, HeliconeEnvoy AI GatewayLiteLLMagainst: OpenRouter, Portkey, Requestyno first choiceagainst: Databricks Unity AI Gateway, Langfuse, LiteLLM, One API, Portkey
Three alternatives
Bifrost, Kong AI Gateway, PortkeyOne alternative
LangfuseFour alternatives
Kong AI Gateway, LiteLLM, Portkey, WSO2 AI GatewayTwo alternatives
Bifrost, Kong AI GatewayPerplexity SonarBifrostBraintrustChangedBifrostagainst: OpenRouterBifrostno first choiceagainst: Cloudflare AI Gateway, Kong AI Gateway, LiteLLM, OpenRouter, Vercel AI Gateway
Seven alternatives
Braintrust, Cloudflare AI Gateway, Kong AI Gateway, LiteLLM, OpenRouter, Respan, TrueFoundryThree alternatives
Bifrost, Confident AI, LangfuseFive alternatives
Cloudflare AI Gateway, Helicone, Kong AI Gateway, LiteLLM, PortkeyTwo alternatives
LiteLLM, PortkeyGrok 4.1 FastBifrost, LiteLLMPortkeyChangedagainst: Braintrust, Langfuse, LiteLLMno first choiceBifrost, LiteLLMagainst: Helicone, OpenRouter, PortkeyLiteLLMagainst: Kong AI Gateway, LiteLLM, Portkey
Three alternatives
Cloudflare AI Gateway, Kong AI Gateway, PortkeyFour alternatives
Arize AI, Datadog LLM Observability, Helicone, LangSmithFour alternatives
Bifrost, LLM Gateway, OpenRouter, PortkeyMistral SmallBifrostPortkeyChangedNeuralTrust TrustGateBifrost, LiteLLMno first choiceagainst: LiteLLM
Three alternatives
LiteLLM, NeuralTrust TrustGate, PortkeyTwo alternatives
Confident AI, Nexos.aiFive alternatives
Braintrust Gateway, Kong AI Gateway, LiteLLM, Martian, PortkeyOne alternative
PortkeyTwo alternatives
Amazon Bedrock Guardrails, Azure AI GatewayDeepSeek V4 FlashPortkeyagainst: Kong AI GatewayLangfuseChangedagainst: LangSmithPortkeyBifrost, LiteLLMagainst: Portkeyno first choiceagainst: Cloudflare AI Gateway, Helicone, LiteLLM, OpenRouter
Two alternatives
Bifrost, LiteLLMFour alternatives
Arize, Datadog, LiteLLM, PortkeySeven alternatives
Bifrost, Braintrust Gateway, Cloudflare AI Gateway, Kong AI Gateway, LiteLLM, OpenRouter, TrueFoundryLlama 4 MaverickTrueFoundryBraintrustChangedBifrostLiteLLMno first choicenothing named
Five alternatives
Bifrost, LiteLLM, NeuralTrust TrustGate, OpenRouter, PortkeyThree alternatives
Arize AI, Confident AI, OpenObserveFour alternatives
Cloudflare AI Gateway, Kong AI Gateway, LiteLLM, OpenRouterThree alternatives
Bifrost, OpenRouter, PortkeyQwen 3.7 FlashLiteLLM, PortkeyLiteLLM, PortkeyHeldNeuralTrust TrustGateagainst: OpenRouterBifrost, LiteLLMagainst: OpenRouterno first choiceagainst: Cloudflare AI Gateway, Kong AI Gateway, LiteLLM, OpenRouter
Two alternatives
Maxim AI, NeuralTrust TrustGateThree alternatives
Arize, LangSmith, SemaFour alternatives
Bifrost, Kong AI Gateway, LiteLLM, PortkeyKimi K2Bifrost, LiteLLMPortkeyChangedBifrostagainst: Cloudflare AI Gateway, LiteLLMLiteLLMno first choicenothing named
Four alternatives
Cloudflare AI Gateway, Kong AI Gateway, Portkey, TrueFoundryFive alternatives
Arize AI, Braintrust, LangSmith, Langfuse, LiteLLMFour alternatives
Azure API Management, Kong AI Gateway, Portkey, ZuploOne alternative
BifrostGLM 4.7 FlashXBifrostagainst: OpenRouterConfident AIChangedBifrost, Portkeyagainst: LiteLLMBifrostagainst: OpenRouterno first choiceagainst: LiteLLM
Three alternatives
Cloudflare AI Gateway, Kong AI Gateway, LiteLLMEight alternatives
Amazon Bedrock, Arize AX, Arize Phoenix, Azure OpenAI Service, Braintrust Gateway, Datadog, Helicone, LangfuseTwo alternatives
Helicone, Kong AI GatewayOne alternative
LiteLLMMiniMax M2.5Cloudflare AI Gateway, HeliconeConfident AIChangedBifrost, LiteLLMLLM Gatewayno first choiceagainst: LiteLLM
Two alternatives
Braintrust, NeuralTrust TrustGateFive alternatives
Arize AI, Datadog LLM Observability, LangSmith, Langfuse, MLflowFour alternatives
Cloudflare, Kong AI Gateway, OpenRouter, PortkeyTwo alternatives
LiteLLM, Standard ComputeBold is the first choiceAlternatives are counted; the count opens them.What the answer argued against
05The record
One row per call: the version string exactly as returned, whether the model searched, sources cited, and latency. Full answer text is in the free responses file. Download the record
Seventy-two rows: every prompt, every model, every answer.
| Prompt | Model | Version string | Time (UTC) | Searched | Sources | Latency |
|---|---|---|---|---|---|---|
| Direct recommendation | Claude Haiku 4.5 | claude-haiku-4-5-20251001 | 2026-09-17 11:44 | yes | 9 | 10 s |
| Direct recommendation | GPT-5.4 mini | gpt-5.4-mini-2026-03-17 | 2026-09-17 13:21 | yes | 3 | 5 s |
| Direct recommendation | Gemini 3.5 Flash | gemini-3.5-flash | 2026-09-17 11:15 | yes | 13 | 20 s |
| Direct recommendation | Perplexity Sonar | sonar | 2026-09-17 10:03 | yes | 20 | 6 s |
| Direct recommendation | Grok 4.1 Fast | spacexai/grok-4.1-fast-non-reasoning via vertex | 2026-09-17 12:20 | yes | 42 | 21 s |
| Direct recommendation | Mistral Small | mistral/mistral-small via mistral | 2026-09-17 12:20 | yes | 5 | 4 s |
| Direct recommendation | DeepSeek V4 Flash | deepseek/deepseek-v4-flash via fireworks | 2026-09-17 13:53 | yes | 10 | 16 s |
| Direct recommendation | Llama 4 Maverick | meta/llama-4-maverick via bedrock | 2026-09-17 10:19 | yes | 5 | 2 s |
| Direct recommendation | Qwen 3.7 Flash | alibaba/qwen3.7-flash via alibaba | 2026-09-17 10:02 | yes | 5 | 34 s |
| Direct recommendation | Kimi K2 | moonshotai/kimi-k2 via novita | 2026-09-17 13:29 | yes | 8 | 48 s |
| Direct recommendation | GLM 4.7 FlashX | zai/glm-4.7-flashx via zai | 2026-09-17 12:22 | yes | 22 | 129 s |
| Direct recommendation | MiniMax M2.5 | minimax/minimax-m2.5 via minimax | 2026-09-17 09:40 | yes | 5 | 16 s |
| Paraphrase | Claude Haiku 4.5 | claude-haiku-4-5-20251001 | 2026-09-17 11:27 | yes | 9 | 9 s |
| Paraphrase | GPT-5.4 mini | gpt-5.4-mini-2026-03-17 | 2026-09-17 13:22 | yes | 6 | 6 s |
| Paraphrase | Gemini 3.5 Flash | gemini-3.5-flash | 2026-09-17 11:19 | yes | 20 | 27 s |
| Paraphrase | Perplexity Sonar | sonar | 2026-09-17 11:19 | yes | 20 | 6 s |
| Paraphrase | Grok 4.1 Fast | spacexai/grok-4.1-fast-non-reasoning via vertex | 2026-09-17 14:05 | yes | 49 | 23 s |
| Paraphrase | Mistral Small | mistral/mistral-small via mistral | 2026-09-17 11:11 | yes | 5 | 4 s |
| Paraphrase | DeepSeek V4 Flash | deepseek/deepseek-v4-flash via fireworks | 2026-09-17 10:56 | yes | 27 | 24 s |
| Paraphrase | Llama 4 Maverick | meta/llama-4-maverick via bedrock | 2026-09-17 13:02 | yes | 5 | 4 s |
| Paraphrase | Qwen 3.7 Flash | alibaba/qwen3.7-flash via alibaba | 2026-09-17 11:48 | no | 0 | 39 s |
| Paraphrase | Kimi K2 | moonshotai/kimi-k2 via novita | 2026-09-17 10:04 | yes | 15 | 78 s |
| Paraphrase | GLM 4.7 FlashX | zai/glm-4.7-flashx via zai | 2026-09-17 11:26 | yes | 14 | 24 s |
| Paraphrase | MiniMax M2.5 | minimax/minimax-m2.5 via minimax | 2026-09-17 13:12 | yes | 13 | 30 s |
| Comparative | Claude Haiku 4.5 | claude-haiku-4-5-20251001 | 2026-09-17 14:08 | yes | 9 | 10 s |
| Comparative | GPT-5.4 mini | gpt-5.4-mini-2026-03-17 | 2026-09-17 12:32 | yes | 4 | 8 s |
| Comparative | Gemini 3.5 Flash | gemini-3.5-flash | 2026-09-17 12:26 | yes | 17 | 29 s |
| Comparative | Perplexity Sonar | sonar | 2026-09-17 14:09 | yes | 20 | 13 s |
| Comparative | Grok 4.1 Fast | spacexai/grok-4.1-fast-non-reasoning via vertex | 2026-09-17 13:49 | yes | 32 | 16 s |
| Comparative | Mistral Small | mistral/mistral-small via mistral | 2026-09-17 10:17 | yes | 9 | 10 s |
| Comparative | DeepSeek V4 Flash | deepseek/deepseek-v4-flash via fireworks | 2026-09-17 10:21 | yes | 18 | 24 s |
| Comparative | Llama 4 Maverick | meta/llama-4-maverick via bedrock | 2026-09-17 11:46 | yes | 5 | 2 s |
| Comparative | Qwen 3.7 Flash | alibaba/qwen3.7-flash via alibaba | 2026-09-17 13:41 | yes | 5 | 31 s |
| Comparative | Kimi K2 | moonshotai/kimi-k2 via novita | 2026-09-17 11:02 | yes | 22 | 45 s |
| Comparative | GLM 4.7 FlashX | zai/glm-4.7-flashx via zai | 2026-09-17 10:37 | yes | 29 | 57 s |
| Comparative | MiniMax M2.5 | minimax/minimax-m2.5 via minimax | 2026-09-17 11:34 | yes | 10 | 19 s |
| Budget constrained | Claude Haiku 4.5 | claude-haiku-4-5-20251001 | 2026-09-17 09:52 | yes | 15 | 11 s |
| Budget constrained | GPT-5.4 mini | gpt-5.4-mini-2026-03-17 | 2026-09-17 13:03 | yes | 3 | 5 s |
| Budget constrained | Gemini 3.5 Flash | gemini-3.5-flash | 2026-09-17 11:08 | yes | 26 | 29 s |
| Budget constrained | Perplexity Sonar | sonar | 2026-09-17 12:28 | yes | 20 | 7 s |
| Budget constrained | Grok 4.1 Fast | spacexai/grok-4.1-fast-non-reasoning via vertex | 2026-09-17 10:24 | yes | 42 | 17 s |
| Budget constrained | Mistral Small | mistral/mistral-small via mistral | 2026-09-17 13:07 | yes | 5 | 5 s |
| Budget constrained | DeepSeek V4 Flash | deepseek/deepseek-v4-flash via deepinfra | 2026-09-17 13:52 | yes | 28 | 49 s |
| Budget constrained | Llama 4 Maverick | meta/llama-4-maverick via bedrock | 2026-09-17 13:12 | yes | 5 | 3 s |
| Budget constrained | Qwen 3.7 Flash | alibaba/qwen3.7-flash via alibaba | 2026-09-17 11:40 | yes | 15 | 47 s |
| Budget constrained | Kimi K2 | moonshotai/kimi-k2 via novita | 2026-09-17 09:24 | yes | 9 | 52 s |
| Budget constrained | GLM 4.7 FlashX | zai/glm-4.7-flashx via zai | 2026-09-17 10:25 | yes | 22 | 55 s |
| Budget constrained | MiniMax M2.5 | minimax/minimax-m2.5 via minimax | 2026-09-17 13:40 | yes | 18 | 63 s |
| Scale constrained | Claude Haiku 4.5 | claude-haiku-4-5-20251001 | 2026-09-17 10:28 | yes | 17 | 10 s |
| Scale constrained | GPT-5.4 mini | gpt-5.4-mini-2026-03-17 | 2026-09-17 12:05 | yes | 1 | 10 s |
| Scale constrained | Gemini 3.5 Flash | gemini-3.5-flash | 2026-09-17 12:47 | yes | 12 | 32 s |
| Scale constrained | Perplexity Sonar | sonar | 2026-09-17 11:54 | yes | 20 | 11 s |
| Scale constrained | Grok 4.1 Fast | spacexai/grok-4.1-fast-non-reasoning via vertex | 2026-09-17 12:30 | yes | 14 | 13 s |
| Scale constrained | Mistral Small | mistral/mistral-small via mistral | 2026-09-17 09:31 | no | 0 | 9 s |
| Scale constrained | DeepSeek V4 Flash | deepseek/deepseek-v4-flash via fireworks | 2026-09-17 13:18 | yes | 10 | 22 s |
| Scale constrained | Llama 4 Maverick | meta/llama-4-maverick via bedrock | 2026-09-17 10:39 | yes | 5 | 4 s |
| Scale constrained | Qwen 3.7 Flash | alibaba/qwen3.7-flash via alibaba | 2026-09-17 13:57 | yes | 9 | 44 s |
| Scale constrained | Kimi K2 | moonshotai/kimi-k2 via novita | 2026-09-17 10:26 | yes | 15 | 45 s |
| Scale constrained | GLM 4.7 FlashX | zai/glm-4.7-flashx via zai | 2026-09-17 13:11 | yes | 24 | 205 s |
| Scale constrained | MiniMax M2.5 | minimax/minimax-m2.5 via minimax | 2026-09-17 13:45 | yes | 5 | 22 s |
| Negative framing | Claude Haiku 4.5 | claude-haiku-4-5-20251001 | 2026-09-17 13:25 | yes | 18 | 9 s |
| Negative framing | GPT-5.4 mini | gpt-5.4-mini-2026-03-17 | 2026-09-17 12:05 | yes | 4 | 7 s |
| Negative framing | Gemini 3.5 Flash | gemini-3.5-flash | 2026-09-17 10:48 | yes | 19 | 31 s |
| Negative framing | Perplexity Sonar | sonar | 2026-09-17 09:59 | yes | 20 | 6 s |
| Negative framing | Grok 4.1 Fast | spacexai/grok-4.1-fast-non-reasoning via vertex | 2026-09-17 09:59 | yes | 43 | 15 s |
| Negative framing | Mistral Small | mistral/mistral-small via mistral | 2026-09-17 13:42 | yes | 14 | 9 s |
| Negative framing | DeepSeek V4 Flash | deepseek/deepseek-v4-flash via deepinfra | 2026-09-17 14:11 | yes | 29 | 66 s |
| Negative framing | Llama 4 Maverick | meta/llama-4-maverick via bedrock | 2026-09-17 10:05 | yes | 5 | 2 s |
| Negative framing | Qwen 3.7 Flash | alibaba/qwen3.7-flash via alibaba | 2026-09-17 11:19 | yes | 15 | 39 s |
| Negative framing | Kimi K2 | moonshotai/kimi-k2 via novita | 2026-09-17 11:22 | yes | 28 | 59 s |
| Negative framing | GLM 4.7 FlashX | zai/glm-4.7-flashx via zai | 2026-09-17 13:14 | yes | 29 | 42 s |
| Negative framing | MiniMax M2.5 | minimax/minimax-m2.5 via minimax | 2026-09-17 10:55 | yes | 24 | 46 s |
Normalization in this category
Every judgment call made between the raw labels and the numbers above, listed so it is visible and reversible.
Category-scoped readings
Kong read as Kong AI Gateway
Kong AI read as Kong AI Gateway
NeuralTrust read as NeuralTrust TrustGate
Unresolved, counted raw
APIM solutions
AWS Bedrock Gateway
Amazon Bedrock AgentCore Gateway
Amazon Bedrock Guardrails
Arize AX
Azure AI Foundry
Cequence AI Gateway
Featherless.ai
Galileo AI
Highflame
Lakera
Mirascope
Prompt Security
SS&C AI Gateway
Sema
Standard Compute
Tetrate
Discontinued, still offered
No shut-down product was recommended here.