LLM gateways and observability · September 2026 Edition
LiteLLM vs Portkey
Five of twelve models named LiteLLM first on the direct prompt; six named Portkey. Both were named by all twelve models and LiteLLM carries 51 labels and Portkey 43, so the shares are not directly comparable.
Named in two categories this edition.
Named in two categories this edition.
First-choice share38%27%Of first choices across the direct, paraphrase, budget and scale prompts, 0 to 100.
Negative rate18%2%Negative labels as a share of the product's labels, 0 to 100.
Rank in category#1#2A position in a field of 9; printed, not drawn.
Labels5143A count; the two differ.
The two percentage rows are drawn on one 0 to 100 track, LiteLLM reading right to left. Rank and label count are printed, not drawn.LiteLLM vs Helicone · Portkey vs Helicone
Share is the count of first choices across the direct, paraphrase, budget and scale prompts over all twelve models, for a mid-market B2B company; rank is within the category; every quote names the model and the prompt it came from. Both figures come from the LLM gateways and observability page.
By framing
How many of the twelve models made each the first choice, per way of asking, and how many argued against it.
LiteLLMFirst choices, of twelve modelsPortkey
Direct56
Paraphrase05
Comparative61
Budget-constrained110
Scale-constrained11
Negative019 against LiteLLM · 1 against Portkey
Bars are first choices, 0 to 12 each sideModels that argued againstA model can name both, so the two sides of a row do not sum to twelve.
The direct prompt
The plain question, one answer per model, grouped by where LiteLLM and Portkey stood in it.
Both were the first choice
1 of 12 modelsThe answer named them together, and the judge labeled each a first choice.Perplexity SonarLiteLLM, Portkey alternatives: Bifrost, Cloudflare AI Gateway, Kong AI Gateway
LiteLLM first, Portkey an alternative
4 of 12 modelsPortkey was named in the answer but not as the choice, or not at all.Gemini 3.5 FlashLiteLLM alternatives: Bifrost, Cloudflare AI Gateway, Portkey, TrueFoundry
Mistral SmallHelicone, LiteLLM alternatives: Braintrust, OpenRouter
DeepSeek V4 FlashLiteLLM alternatives: Cloudflare AI Gateway, Portkey
Kimi K2LiteLLM alternatives: Helicone, Portkey, Requesty
Portkey first, LiteLLM an alternative
5 of 12 modelsLiteLLM was named in the answer but not as the choice, or not at all.Claude Haiku 4.5Portkey alternatives: LiteLLM, OpenRouter
Grok 4.1 FastPortkey alternatives: LiteLLM
Qwen 3.7 FlashPortkey alternatives: Bifrost, Cloudflare AI Gateway, LiteLLM
GLM 4.7 FlashXPortkey alternatives: Helicone, LiteLLM
MiniMax M2.5Merge Gateway, Portkey alternatives: LiteLLM
Neither was the first choice, one was named
1 of 12 modelsThe answer put something else first and named one of the two as an alternative.GPT-5.4 miniPortkey / Prisma AIRS AI Gateway alternatives: Kong AI Gateway, LiteLLM
Neither was named
1 of 12 modelsThe answer made no first choice from these two in this category.Llama 4 Maverickno first choice
Bold names in an answer are the products the judge labeled a first choice; a model naming several gives each of them that label. The full answer text for every row is in the record.
By buyer segment
The same question asked on behalf of a different buyer. Each standing is computed within its segment and they are never added together. The figures above are the mid-market standing, which is the one the category orders by.
Small business
LiteLLM leads by thirty-one points.
LiteLLM46%#1 of 6Portkey15%#2 of 6Mid-marketThe figures above
Enterprise
What the models said about LiteLLM
Every negative label with a quote, up to three, then the highest-weighted positives, up to three. Six of eight in this category shown.
“Be cautious with self-hosted or open-source LLM gateways like LiteLLM and Ollama, especially if internet-exposed, due to documented security risks... actively exploited and CISA KEV-listed” Grok 4.1 Fast · negative prompt · hard negative
“LiteLLM (Critical Risk) ... Why it is dangerous: Recent research has shown that compromised LiteLLM instances can lead to Remote Code Execution (RCE) and full privilege escalation.” Qwen 3.7 Flash · negative prompt · hard negative
“Avoid or exercise extreme caution with LiteLLM and LMDeploy in production or security-sensitive environments” Mistral Small · negative prompt · hard negative
“LiteLLM (Best Self-Hosted / Open-Source): The most popular open-source, Python-based gateway... Excellent choice if you want to deploy a gateway inside your own Kubernetes cluster.” Gemini 3.5 Flash · scale prompt · first choice
“It is the standard choice for building internal developer platforms (IDPs) because it is open-source and fits into any existing architecture.” Qwen 3.7 Flash · comparative prompt · first choice
“LiteLLM (for flexibility and cost control) or Helicone (for observability and ease of use) are the best LLM gateway choices” Mistral Small · direct prompt · first choice
What the models said about Portkey
Every negative label with a quote, up to three, then the highest-weighted positives, up to three. Four of five in this category shown.
“Strong governance/compliance features, but one comparison says you should expect extra latency overhead” Perplexity Sonar · negative prompt · soft negative
“For most mid-sized B2B companies looking for a balance of velocity and control, starting with a Portkey pilot is highly recommended.” Gemini 3.5 Flash · paraphrase prompt · first choice
“Portkey is widely considered the best all-around solution for mid-market businesses due to its specialized governance features” Qwen 3.7 Flash · direct prompt · first choice
“Portkey (Best Managed / Enterprise Governance): ... a premium, developer-friendly gateway. It offers top-tier reliability features” Gemini 3.5 Flash · scale prompt · first choice
Comparisons are drawn for the top three products in each category. The output is the models' output; nothing here is a recommendation by the index.