Static application security testing · October 2026 Edition
CodeQL vs GitHub Advanced Security
Zero of fourteen models named CodeQL first on the direct prompt; zero named GitHub Advanced Security. CodeQL was named by eleven of the fourteen models and GitHub Advanced Security by eleven and CodeQL carries 27 labels and GitHub Advanced Security 16, so the shares are not directly comparable.
CodeQL
accepted challengerNamed in one category this edition.
Named in five categories this edition.
First-choice share4%2%Of first choices across the direct, paraphrase, budget and scale prompts, 0 to 100.
Negative rate11%6%Negative labels as a share of the product's labels, 0 to 100.
Rank in category#5#6A position in a field of 10; printed, not drawn.
Labels2716A count; the two differ.
The two percentage rows are drawn on one 0 to 100 track, CodeQL reading right to left. Rank and label count are printed, not drawn.Semgrep was named alongside these two in twelve of the fourteen direct answers. Semgrep vs CodeQL · Semgrep vs GitHub Advanced Security · SonarQube vs CodeQL
Share is the count of first choices across the direct, paraphrase, budget and scale prompts over all fourteen models, for a mid-market B2B company; rank is within the category; every quote names the model and the prompt it came from. Both figures come from the static application security testing page.
By framing
How many of the fourteen models made each the first choice, per way of asking, and how many argued against it.
CodeQLFirst choices, of fourteen modelsGitHub Advanced Security
Direct00
Paraphrase111 against GitHub Advanced Security
Comparative00
Budget-constrained101 against CodeQL
Scale-constrained00
Negative102 against CodeQL
Bars are first choices, 0 to 14 each sideModels that argued againstA model can name both, so the two sides of a row do not sum to fourteen.
Every model, every framing
The eighty-four answers behind the chart above, one cell each: where CodeQL and GitHub Advanced Security stood in it.
| Model | DirectCO | ParaphraseCO | ComparativeCO | Budget-constrainedCO | Scale-constrainedCO | NegativeCO |
|---|---|---|---|---|---|---|
| Claude Haiku 4.5 | ||||||
| GPT-5.4 mini | CO | CO | ||||
| Gemini 3.5 Flash | | | | CO | CO | |
| Perplexity Sonar | CO | |||||
| Grok 4.1 Fast | | CO | CO | CO | ||
| Mistral Small | CO | |||||
| DeepSeek V4 Flash | CO | CO | CO | CO | ||
| Llama 4 Maverick | ||||||
| Qwen 3.7 Flash | | CO | CO | |||
| Kimi K2 | | CO | CO | | ||
| GLM 4.7 FlashX | | |||||
| MiniMax M2.5 | | | ||||
| GPT-6 Luna | | CO | CO | CO | CO | |
| Muse Glimmer 30B | | CO | CO |
CO CodeQL GitHub Advanced SecurityCO first choiceCO named as an alternativeCO argued againstblank: not namedEach cell is one answer, CodeQL on the left and GitHub Advanced Security on the right.
GitHub Advanced SecurityCO first choiceCO named as an alternativeCO argued againstblank: not namedEach cell is one answer, CodeQL on the left and GitHub Advanced Security on the right.The direct prompt
The plain question, one answer per model, grouped by where CodeQL and GitHub Advanced Security stood in it.
Neither was the first choice, one was named
3 of 14 modelsThe answer put something else first and named one of the two as an alternative.Gemini 3.5 FlashAikido Security alternatives: GitHub Advanced Security, GitLab SAST, Semgrep, Snyk Code
GLM 4.7 FlashXSonarQube alternatives: Checkmarx One, Codacy, DeepSource, GitHub Advanced Security, OpenText Fortify, Semgrep, Snyk Code, Veracode Static Analysis
GPT-6 LunaSemgrep alternatives: GitHub Advanced Security, Snyk
Neither was named
11 of 14 modelsThe answer made no first choice from these two in this category.Claude Haiku 4.5no first choice
GPT-5.4 miniSemgrep alternatives: Checkmarx One, Snyk Code, Veracode Static Analysis
Perplexity SonarSemgrep alternatives: Checkmarx One, Snyk Code
Grok 4.1 FastSnyk Code alternatives: Semgrep, SonarQube
Mistral SmallSnyk alternatives: Aikido Security, Semgrep
DeepSeek V4 FlashSnyk Code alternatives: CodeAnt.ai, Semgrep, SonarQube
Llama 4 MaverickSnyk Code
Qwen 3.7 FlashSonarQube alternatives: Semgrep, Snyk Code
Kimi K2Semgrep, Snyk Code alternatives: SonarQube
MiniMax M2.5Snyk alternatives: Semgrep, Veracode Static Analysis
Muse Glimmer 30BSonarQube alternatives: Semgrep
Bold names in an answer are the products the judge labeled a first choice; a model naming several gives each of them that label. The full answer text for every row is in the record.
By buyer segment
The same question asked on behalf of a different buyer. Each standing is computed within its segment and they are never added together. The figures above are the mid-market standing, which is the one the category orders by.
Small business
GitHub Advanced Security leads by two points.
GitHub Advanced Security2%#6 of 13CodeQL0%#8 of 13
The full small business standing →Mid-marketThe figures above
The order flips: CodeQL leads at mid-market.
CodeQL4%#5 of 10
GitHub Advanced Security2%#6 of 10Enterprise
Level: the same share of first choices.
CodeQL2%#7 of 10
GitHub Advanced Security2%#8 of 10What the models said about CodeQL
Every negative label with a quote, up to three, then the highest-weighted positives, up to three. Six of seven in this category shown.
“CodeQL requires a subscription to GitHub Advanced Security (GHAS), which is highly enterprise-focused and expensive” Gemini 3.5 Flash · budget prompt · soft negative
“CodeQL (A "Proceed with Caution" for Closed-Source Teams) ... GHAS licensing, which can be prohibitively expensive” Gemini 3.5 Flash · negative prompt · soft negative
“CodeQL if your stack or plan doesn't fit: Check language support before adopting it” GPT-6 Luna · negative prompt · soft negative
“GitHub CodeQL – Free for public repositories ... It provides deep code analysis and is widely used for finding vulnerabilities in multiple languages.” Mistral Small · budget prompt · first choice
“my default recommendation would be GitHub CodeQL if you already use GitHub” GPT-5.4 mini · paraphrase prompt · first choice
“Lower noise: CodeQL (semantic, low FPs)” Grok 4.1 Fast · negative prompt · first choice
What the models said about GitHub Advanced Security
Every negative label with a quote, up to three, then the highest-weighted positives, up to three. Three of three in this category shown.
“GHAS is sold as an add-on license that can be quite expensive for a mid-sized B2B budget.” Gemini 3.5 Flash · paraphrase prompt · soft negative
“Top Recommendation: GitHub Advanced Security (GHAS)” Kimi K2 · paraphrase prompt · first choice
“GitHub Advanced Security / CodeQL if your company is deeply GitHub-native and wants SAST in the same platform.” Muse Glimmer 30B · comparative prompt · alternative
Also compared
Semgrep vs CodeQLSemgrep vs GitHub Advanced SecuritySonarQube vs CodeQLSonarQube vs GitHub Advanced SecuritySnyk Code vs CodeQLSnyk Code vs GitHub Advanced SecuritySnyk vs CodeQLSnyk vs GitHub Advanced SecurityCodeQL vs Veracode Static AnalysisCodeQL vs Checkmarx OneGitHub Advanced Security vs Veracode Static AnalysisGitHub Advanced Security vs Checkmarx OneAll twenty-eight head-to-head pages and the full SAST standing
Comparisons are drawn for the top eight products in each category, each against each. The output is the models' output; nothing here is a recommendation by the index.