Static application security testing · October 2026 Edition
Snyk vs CodeQL
Two of fourteen models named Snyk first on the direct prompt; zero named CodeQL. Snyk was named by seven of the fourteen models and CodeQL by eleven and Snyk carries 11 labels and CodeQL 27, so the shares are not directly comparable.
Named in eleven categories this edition.
CodeQL
accepted challengerNamed in one category this edition.
First-choice share8%4%Of first choices across the direct, paraphrase, budget and scale prompts, 0 to 100.
Negative rate9%11%Negative labels as a share of the product's labels, 0 to 100.
Rank in category#4#5A position in a field of 10; printed, not drawn.
Labels1127A count; the two differ.
The two percentage rows are drawn on one 0 to 100 track, Snyk reading right to left. Rank and label count are printed, not drawn.Semgrep was named alongside these two in twelve of the fourteen direct answers. Semgrep vs Snyk · Semgrep vs CodeQL · SonarQube vs Snyk
Share is the count of first choices across the direct, paraphrase, budget and scale prompts over all fourteen models, for a mid-market B2B company; rank is within the category; every quote names the model and the prompt it came from. Both figures come from the static application security testing page.
By framing
How many of the fourteen models made each the first choice, per way of asking, and how many argued against it.
SnykFirst choices, of fourteen modelsCodeQLDirect20
Paraphrase21
Comparative00
Budget-constrained011 against CodeQL
Scale-constrained00
Negative011 against Snyk · 2 against CodeQL
Bars are first choices, 0 to 14 each sideModels that argued againstA model can name both, so the two sides of a row do not sum to fourteen.
Every model, every framing
The eighty-four answers behind the chart above, one cell each: where Snyk and CodeQL stood in it.
| Model | DirectCO | ParaphraseCO | ComparativeCO | Budget-constrainedCO | Scale-constrainedCO | NegativeCO |
|---|---|---|---|---|---|---|
| Claude Haiku 4.5 | | | ||||
| GPT-5.4 mini | CO | CO | ||||
| Gemini 3.5 Flash | | CO | CO | |||
| Perplexity Sonar | CO | |||||
| Grok 4.1 Fast | CO | CO | CO | |||
| Mistral Small | | CO | ||||
| DeepSeek V4 Flash | CO | CO | CO | CO | ||
| Llama 4 Maverick | ||||||
| Qwen 3.7 Flash | CO | CO | ||||
| Kimi K2 | CO | CO | ||||
| GLM 4.7 FlashX | ||||||
| MiniMax M2.5 | | |||||
| GPT-6 Luna | | CO | CO | CO | CO | |
| Muse Glimmer 30B | CO | CO |
SnykCO CodeQL first choice named as an alternative argued againstblank: not namedEach cell is one answer, Snyk on the left and CodeQL on the right.The direct prompt
The plain question, one answer per model, grouped by where Snyk and CodeQL stood in it.
Snyk first, CodeQL not the choice
2 of 14 modelsCodeQL was named in the answer but not as the choice, or not at all.Mistral SmallSnyk alternatives: Aikido Security, Semgrep
MiniMax M2.5Snyk alternatives: Semgrep, Veracode Static Analysis
Neither was the first choice, one was named
1 of 14 modelsThe answer put something else first and named one of the two as an alternative.GPT-6 LunaSemgrep alternatives: GitHub Advanced Security, Snyk
Neither was named
11 of 14 modelsThe answer made no first choice from these two in this category.Claude Haiku 4.5no first choice
GPT-5.4 miniSemgrep alternatives: Checkmarx One, Snyk Code, Veracode Static Analysis
Gemini 3.5 FlashAikido Security alternatives: GitHub Advanced Security, GitLab SAST, Semgrep, Snyk Code
Perplexity SonarSemgrep alternatives: Checkmarx One, Snyk Code
Grok 4.1 FastSnyk Code alternatives: Semgrep, SonarQube
DeepSeek V4 FlashSnyk Code alternatives: CodeAnt.ai, Semgrep, SonarQube
Llama 4 MaverickSnyk Code
Qwen 3.7 FlashSonarQube alternatives: Semgrep, Snyk Code
Kimi K2Semgrep, Snyk Code alternatives: SonarQube
GLM 4.7 FlashXSonarQube alternatives: Checkmarx One, Codacy, DeepSource, GitHub Advanced Security, OpenText Fortify, Semgrep, Snyk Code, Veracode Static Analysis
Muse Glimmer 30BSonarQube alternatives: Semgrep
Bold names in an answer are the products the judge labeled a first choice; a model naming several gives each of them that label. The full answer text for every row is in the record.
By buyer segment
The same question asked on behalf of a different buyer. Each standing is computed within its segment and they are never added together. The figures above are the mid-market standing, which is the one the category orders by.
Small business
Mid-marketThe figures above
Enterprise
The order flips: CodeQL leads at enterprise.
CodeQL2%#7 of 10
Snyk0%#10 of 10What the models said about Snyk
Every negative label with a quote, up to three, then the highest-weighted positives, up to three. Two of two in this category shown.
“"We are forced to use Snyk–it's basically infuriating support and false positives... it's also insanely expensive"” Claude Haiku 4.5 · negative prompt · soft negative
“Snyk appears to be the most frequently recommended option for mid-market B2B companies running modern stacks” MiniMax M2.5 · direct prompt · first choice
What the models said about CodeQL
Every negative label with a quote, up to three, then the highest-weighted positives, up to three. Six of seven in this category shown.
“CodeQL requires a subscription to GitHub Advanced Security (GHAS), which is highly enterprise-focused and expensive” Gemini 3.5 Flash · budget prompt · soft negative
“CodeQL (A "Proceed with Caution" for Closed-Source Teams) ... GHAS licensing, which can be prohibitively expensive” Gemini 3.5 Flash · negative prompt · soft negative
“CodeQL if your stack or plan doesn't fit: Check language support before adopting it” GPT-6 Luna · negative prompt · soft negative
“GitHub CodeQL – Free for public repositories ... It provides deep code analysis and is widely used for finding vulnerabilities in multiple languages.” Mistral Small · budget prompt · first choice
“my default recommendation would be GitHub CodeQL if you already use GitHub” GPT-5.4 mini · paraphrase prompt · first choice
“Lower noise: CodeQL (semantic, low FPs)” Grok 4.1 Fast · negative prompt · first choice
Also compared
Semgrep vs SnykSemgrep vs CodeQLSonarQube vs SnykSonarQube vs CodeQLSnyk Code vs SnykSnyk Code vs CodeQLSnyk vs GitHub Advanced SecuritySnyk vs Veracode Static AnalysisSnyk vs Checkmarx OneCodeQL vs GitHub Advanced SecurityCodeQL vs Veracode Static AnalysisCodeQL vs Checkmarx OneAll twenty-eight head-to-head pages and the full SAST standing
Comparisons are drawn for the top eight products in each category, each against each. The output is the models' output; nothing here is a recommendation by the index.