Static application security testing · October 2026 Edition
Semgrep vs CodeQL
Four of fourteen models named Semgrep first on the direct prompt; zero named CodeQL. Semgrep was named by fourteen of the fourteen models and CodeQL by eleven and Semgrep carries 63 labels and CodeQL 27, so the shares are not directly comparable.
Named in three categories this edition.
CodeQL
accepted challengerNamed in one category this edition.
First-choice share36%4%Of first choices across the direct, paraphrase, budget and scale prompts, 0 to 100.
Negative rate5%11%Negative labels as a share of the product's labels, 0 to 100.
Rank in category#1#5A position in a field of 10; printed, not drawn.
Labels6327A count; the two differ.
The two percentage rows are drawn on one 0 to 100 track, Semgrep reading right to left. Rank and label count are printed, not drawn.Snyk Code was named alongside these two in nine of the fourteen direct answers. Semgrep vs SonarQube · Semgrep vs Snyk Code · Semgrep vs Snyk
Share is the count of first choices across the direct, paraphrase, budget and scale prompts over all fourteen models, for a mid-market B2B company; rank is within the category; every quote names the model and the prompt it came from. Both figures come from the static application security testing page.
By framing
How many of the fourteen models made each the first choice, per way of asking, and how many argued against it.
SemgrepFirst choices, of fourteen modelsCodeQLDirect40
Paraphrase41
Comparative40
Budget-constrained911 against CodeQL
Scale-constrained10
Negative413 against Semgrep · 2 against CodeQL
Bars are first choices, 0 to 14 each sideModels that argued againstA model can name both, so the two sides of a row do not sum to fourteen.
Across every category in the October 2026 Edition, Semgrep and CodeQL were named in the same answer seventy times, of the 208 answers naming Semgrep and the 74 naming CodeQL. In those answers CodeQL took the first choice three times and Semgrep thirty-two.
Every model, every framing
The eighty-four answers behind the chart above, one cell each: where Semgrep and CodeQL stood in it.
| Model | DirectCO | ParaphraseCO | ComparativeCO | Budget-constrainedCO | Scale-constrainedCO | NegativeCO |
|---|---|---|---|---|---|---|
| Claude Haiku 4.5 | | | ||||
| GPT-5.4 mini | | CO | CO | |||
| Gemini 3.5 Flash | | | | CO | CO | |
| Perplexity Sonar | | CO | ||||
| Grok 4.1 Fast | | CO | CO | | CO | |
| Mistral Small | | | CO | |||
| DeepSeek V4 Flash | | CO | CO | CO | CO | |
| Llama 4 Maverick | ||||||
| Qwen 3.7 Flash | | | CO | CO | ||
| Kimi K2 | | | CO | CO | | |
| GLM 4.7 FlashX | | | | | ||
| MiniMax M2.5 | | | | |||
| GPT-6 Luna | | CO | CO | CO | CO | |
| Muse Glimmer 30B | | | | CO | CO |
SemgrepCO CodeQL first choice named as an alternative argued againstblank: not namedEach cell is one answer, Semgrep on the left and CodeQL on the right.The direct prompt
The plain question, one answer per model, grouped by where Semgrep and CodeQL stood in it.
Semgrep first, CodeQL not the choice
4 of 14 modelsCodeQL was named in the answer but not as the choice, or not at all.GPT-5.4 miniSemgrep alternatives: Checkmarx One, Snyk Code, Veracode Static Analysis
Perplexity SonarSemgrep alternatives: Checkmarx One, Snyk Code
Kimi K2Semgrep, Snyk Code alternatives: SonarQube
GPT-6 LunaSemgrep alternatives: GitHub Advanced Security, Snyk
Neither was the first choice, one was named
8 of 14 modelsThe answer put something else first and named one of the two as an alternative.Gemini 3.5 FlashAikido Security alternatives: GitHub Advanced Security, GitLab SAST, Semgrep, Snyk Code
Grok 4.1 FastSnyk Code alternatives: Semgrep, SonarQube
Mistral SmallSnyk alternatives: Aikido Security, Semgrep
DeepSeek V4 FlashSnyk Code alternatives: CodeAnt.ai, Semgrep, SonarQube
Qwen 3.7 FlashSonarQube alternatives: Semgrep, Snyk Code
GLM 4.7 FlashXSonarQube alternatives: Checkmarx One, Codacy, DeepSource, GitHub Advanced Security, OpenText Fortify, Semgrep, Snyk Code, Veracode Static Analysis
MiniMax M2.5Snyk alternatives: Semgrep, Veracode Static Analysis
Muse Glimmer 30BSonarQube alternatives: Semgrep
Neither was named
2 of 14 modelsThe answer made no first choice from these two in this category.Claude Haiku 4.5no first choice
Llama 4 MaverickSnyk Code
Bold names in an answer are the products the judge labeled a first choice; a model naming several gives each of them that label. The full answer text for every row is in the record.
By buyer segment
The same question asked on behalf of a different buyer. Each standing is computed within its segment and they are never added together. The figures above are the mid-market standing, which is the one the category orders by.
Small business
Semgrep leads by fifty-three points.
Semgrep53%#1 of 13CodeQL0%#8 of 13
The full small business standing →Mid-marketThe figures above
Enterprise
What the models said about Semgrep
Every negative label with a quote, up to three, then the highest-weighted positives, up to three. Six of seven in this category shown.
“Semgrep OSS and Bandit are fast and low false-positive for obvious patterns, but miss vulnerabilities that span multiple functions... insufficient as sole nightly security scanners” Muse Glimmer 30B · negative prompt · soft negative
“Semgrep Community Edition as your only deep security scanner: It can be useful for fast checks and custom rules, but its analysis is limited to a file/function context” GPT-6 Luna · negative prompt · soft negative
“Checkmarx, Veracode, and Semgrep have all struggled with this because their rule-based engines flag anything that could be a vulnerability” Claude Haiku 4.5 · negative prompt · soft negative
“Semgrep as the primary scanner across all languages... For most budget-constrained companies, Semgrep OSS + SonarQube Community Edition... is the most commonly recommended starting point” Muse Glimmer 30B · budget prompt · first choice
“Tools like Semgrep are generally preferred for modern CI/CD because they are lightweight, customizable, and run significantly faster.” GLM 4.7 FlashX · negative prompt · first choice
“look for developer-friendly alternatives like Semgrep, where rules are written in YAML and read like standard code” Gemini 3.5 Flash · negative prompt · first choice
What the models said about CodeQL
Every negative label with a quote, up to three, then the highest-weighted positives, up to three. Six of seven in this category shown.
“CodeQL requires a subscription to GitHub Advanced Security (GHAS), which is highly enterprise-focused and expensive” Gemini 3.5 Flash · budget prompt · soft negative
“CodeQL (A "Proceed with Caution" for Closed-Source Teams) ... GHAS licensing, which can be prohibitively expensive” Gemini 3.5 Flash · negative prompt · soft negative
“CodeQL if your stack or plan doesn't fit: Check language support before adopting it” GPT-6 Luna · negative prompt · soft negative
“GitHub CodeQL – Free for public repositories ... It provides deep code analysis and is widely used for finding vulnerabilities in multiple languages.” Mistral Small · budget prompt · first choice
“my default recommendation would be GitHub CodeQL if you already use GitHub” GPT-5.4 mini · paraphrase prompt · first choice
“Lower noise: CodeQL (semantic, low FPs)” Grok 4.1 Fast · negative prompt · first choice
Also compared
Semgrep vs SonarQubeSemgrep vs Snyk CodeSemgrep vs SnykSemgrep vs GitHub Advanced SecuritySemgrep vs Veracode Static AnalysisSemgrep vs Checkmarx OneSonarQube vs CodeQLSnyk Code vs CodeQLSnyk vs CodeQLCodeQL vs GitHub Advanced SecurityCodeQL vs Veracode Static AnalysisCodeQL vs Checkmarx OneAll twenty-eight head-to-head pages and the full SAST standing
Comparisons are drawn for the top eight products in each category, each against each. The output is the models' output; nothing here is a recommendation by the index.