# LLM gateways and observability: what AI models recommend, September 2026

IT AI Recommendation Index, September 2026 Edition. Asked as "LLM gateway" and as "LLM observability and routing platform", six framings each, to twelve models with search on, on behalf of a mid-market B2B company. Page: https://it-ai-index.com/it-data/llm-gateways-and-observability/

**Standing:** LiteLLM leads with 38% of first choices; verdict contested. 45 first choices across the direct, paraphrase, budget and scale prompts.

## First-choice share

| # | Product | Share | Negative rate | Labels |
|---|---|---|---|---|
| 1 | LiteLLM | 38% | 18% | 51 |
| 2 | Portkey | 27% | 2% | 43 |
| 3 | Helicone | 9% | 13% | 30 |
| 4 | Langfuse | 9% | 0% | 11 |
| 5 | Cloudflare AI Gateway | 2% | 4% | 23 |
| 6 | OpenRouter | 2% | 23% | 31 |
| 7 | Kong AI Gateway | 2% | 17% | 18 |
| 8 | Braintrust | 2% | 0% | 11 |
| 9 | Bifrost | 0% | 0% | 21 |

## Each model's first choice on the direct prompt

- Claude Haiku 4.5: Portkey; alternatives LiteLLM, OpenRouter
- GPT-5.4 mini: Portkey / Prisma AIRS AI Gateway; alternatives Kong AI Gateway, LiteLLM
- Gemini 3.5 Flash: LiteLLM; alternatives Bifrost, Cloudflare AI Gateway, Portkey, TrueFoundry
- Perplexity Sonar: LiteLLM, Portkey; alternatives Bifrost, Cloudflare AI Gateway, Kong AI Gateway
- Grok 4.1 Fast: Portkey; alternatives LiteLLM
- Mistral Small: Helicone, LiteLLM; alternatives Braintrust, OpenRouter
- DeepSeek V4 Flash: LiteLLM; alternatives Cloudflare AI Gateway, Portkey
- Llama 4 Maverick: no first choice
- Qwen 3.7 Flash: Portkey; alternatives Bifrost, Cloudflare AI Gateway, LiteLLM
- Kimi K2: LiteLLM; alternatives Helicone, Portkey, Requesty
- GLM 4.7 FlashX: Portkey; alternatives Helicone, LiteLLM
- MiniMax M2.5: Merge Gateway, Portkey; alternatives LiteLLM

## Sources the answers cite

65 of 72 answers came back with a source list, from 12 of 12 models. Sites named in the most answers:

- braintrust.dev: 27 answers, 37 citations
- getmaxim.ai: 26 answers, 52 citations
- respan.ai: 26 answers, 30 citations
- cloudzero.com: 26 answers, 26 citations
- dev.to: 25 answers, 46 citations
- dataiku.com: 21 answers, 24 citations
- orq.ai: 21 answers, 23 citations
- techsy.io: 19 answers, 19 citations

Pages named in the most answers:

- https://cloudzero.com/blog/llm-gateways (26 answers)
- https://dataiku.com/blog/best-llm-gateways (21 answers)
- https://orq.ai/blog/best-llm-gateways (21 answers)
- https://respan.ai/articles/best-llm-gateways (20 answers)
- https://techsy.io/en/blog/best-llm-gateway-tools (19 answers)
- https://braintrust.dev/articles/best-llm-gateways-2026 (14 answers)
- https://dev.to/varshithvhegde/top-5-llm-gateways-in-2026-a-deep-dive-comparison-for-production-teams-34d2 (11 answers)
- https://opper.ai/blog/best-llm-gateways (11 answers)
- https://getmaxim.ai/articles/best-enterprise-llm-gateways-in-2026-a-comparative-guide (9 answers)
- https://getmaxim.ai/articles/top-5-llm-gateways-in-2026-a-production-ready-comparison (9 answers)

## Warned against

- LiteLLM: 9 of 51 labels negative. "Be cautious with self-hosted or open-source LLM gateways like LiteLLM and Ollama, especially if internet-exposed, due to documented security risks... actively exploited and CISA KEV-listed" (Grok 4.1 Fast, negative prompt)
- OpenRouter: 7 of 31 labels negative. "routing sensitive enterprise or user data through a third-party hosted cloud gateway introduces strict compliance and privacy risks" (Gemini 3.5 Flash, negative prompt)
- Helicone: 4 of 30 labels negative. "Avoid depending on Helicone — it's now in maintenance mode" (DeepSeek V4 Flash, budget prompt)
- Kong AI Gateway: 3 of 18 labels negative. "the operational weight is hard to justify for LLM routing alone if you're not already a Kong shop" (Claude Haiku 4.5, comparative prompt)

## Record

- Method: https://it-ai-index.com/methodology/
- Raw judge labels and full responses: https://it-ai-index.com/data/
- License: CC BY 4.0. Cite as IT AI Recommendation Index, September 2026 Edition, it-ai-index.com.