IT AI Index
September 2026 Edition · The permanent record of this edition. The unqualified address always carries the latest edition.
Index › Security operations › PTaaS › Enterprise › September 2026 Edition

Penetration testing as a service for enterprise buyers

Asked as “penetration testing service”, and as “pentest-as-a-service provider”, on behalf of an enterprise B2B company. 48 first choices recorded across the direct, paraphrase, budget and scale prompts, twelve models each.
Standing
NetSPI
Contested
40% of first choices, contested.
ShowinginAll 67 categories in the September 2026 Edition →

By buyer segment

The same question asked on behalf of a different buyer. Each standing is computed within its segment; they sit side by side and are never added together.
Small businessBreachLock28%ContestedStanding →Mid-marketCobalt44%Clear leaderStanding →EnterpriseNetSPI40%ContestedThis page

01The standing

Share is the count of first choices across the direct, paraphrase, budget and scale prompts, over all twelve models, for an enterprise B2B company. Ordered by share.
ProductFirst-choice shareNegative rateLabelsQuadrant
01NetSPI40%0%36endorsed leader
02Synack23%0%42accepted challenger
03Cobalt15%6%35accepted challenger
04Bishop Fox4%7%27accepted challenger
05BreachLock4%0%13accepted challenger
06HackerOne4%14%21accepted challenger
07NCC Group2%6%17accepted challenger
Show the two products at 0%, ordered by negative rate
09Bugcrowd0%25%12criticized challenger
08Mandiant0%0%13accepted challenger
Bars are the share of first choices, 0 to 100Every product with at least 10 labels here. Every product name links to its vendor page.
Head to head
Cobalt vs Astra SecurityCobalt vs BreachLockAstra Security vs BreachLock

Recommended versus criticized

Every product with at least 10 labels here, on both axes. The 30% line names a quadrant, not the verdict above: that one needs more than 40%.

Criticized challengerCriticized default
Negative label rate →
01
02
03
04
05
06
07
08
09
Accepted challengerEndorsed leader
0%First-choice share → · lines at 30% share and 25% negative50%
Key
01NetSPI40%
02Synack23%
03Cobalt15%
04Bishop Fox4%
05BreachLock4%
06HackerOne4%
07NCC Group2%
08Mandiant0%
09Bugcrowd0%

02What they warned about

Two of twelve models held their first choice under the paraphrase. Claude Haiku 4.5, GPT-5.4 mini, Gemini 3.5 Flash, Grok 4.1 Fast, Mistral Small, DeepSeek V4 Flash, Llama 4 Maverick, Qwen 3.7 Flash, Kimi K2 and MiniMax M2.5 changed. A high negative share on a product with few labels is a warning. A low share on a product with many labels is salience, not sentiment.
Bugcrowd
25%
3 of 12 labels negative · 2 of 12 models · 2 hard negative
“Avoid pure crowdsourced (HackerOne/Bugcrowd) due to variable bounties/credits” Grok 4.1 Fast, budget prompt
Bishop Fox
7%
2 of 27 labels negative · 2 of 12 models · 1 hard negative
“What to Avoid for Predictable Cost ... Costs scale unpredictably as scope expands.” DeepSeek V4 Flash, budget prompt
HackerOne
14%
3 of 21 labels negative · 1 of 12 models · 2 hard negative
“Avoid pure crowdsourced (HackerOne/Bugcrowd) due to variable bounties/credits” Grok 4.1 Fast, budget prompt
AttackIQ Flex
100%
1 of 1 labels negative · 1 of 12 models · 1 hard negative
“avoid pure pay-as-you-go** services like AttackIQ Flex unless your testing volume is tightly controlled” GPT-5.4 mini, budget prompt

03What they cite

Citations exist only for the models that return a source list: twelve of the twelve in this edition, and all six flagship models on the expanded tier.

Sites the answers cite

64 of 72 answers in this category came back with a source list, from 12 of 12 models: citations where the model returns them, or the search results it consulted. 6 of those lists are Google grounding redirects that name no site and are left out of the counts. 878 links across 208 sites, every framing counted. Ranked by the number of answers carrying the site or page.

deepstrike.io
vendor site · DeepStrike40 answers · 78 citations · 10 models
synack.com
vendor site · Synack40 answers · 58 citations · 11 models
stingrai.io
vendor site · Stingrai29 answers · 46 citations · 8 models
underdefense.com
vendor site · UnderDefense25 answers · 27 citations · 8 models
cybri.com
vendor site · Cybri24 answers · 30 citations · 10 models
softwaresecured.com
vendor site · Software Secured21 answers · 27 citations · 7 models
getastra.com
15 answers · 17 citations · 6 models
brightdefense.com
vendor site · Bright Defense14 answers · 17 citations · 7 models
redbotsecurity.com
vendor site · Redbot Security14 answers · 14 citations · 7 models
techrepublic.com
14 answers · 14 citations · 7 models
soc2auditors.org
13 answers · 15 citations · 6 models
lorikeetsecurity.com
12 answers · 13 citations · 7 models

Pages the answers cite

The ten pages named in the most answers, by full address. A page here is one the models returned with a recommendation, not one the index endorses.

synack.com/blog/best-penetration-testing-companies
29 answers · 10 models
underdefense.com/blog/penetration-testing-services
25 answers · 8 models
softwaresecured.com/post/top-10-penetration-testing-vendors
18 answers · 7 models
cybri.com/blog/best-penetration-testing-vendors
17 answers · 9 models
techrepublic.com/article/best-penetration-testing-companies
14 answers · 7 models
deepstrike.io/blog/top-penetration-testing-as-a-service-ptaas
13 answers · 7 models
redbotsecurity.com/penetration-testing-companies
13 answers · 7 models
deepstrike.io/blog/top-penetration-testing-companies
12 answers · 6 models
terra.security/blog/top-10-penetration-testing-as-a-service-ptaas-providers
12 answers · 8 models
lorikeetsecurity.com/blog/top-10-pentesting-companies-2026
11 answers · 7 models

04How they answered

Six framings of the same buying question, each sent to every model in a fresh session with search on. One row per model, so a row shows whether it held its answer under rewording, what it named when cost was the constraint, and what it argued against. Computed from the raw judge labels.
ModelDirect“What is the best penetration testing service for an enterprise B2B company?”Paraphrase“Which pentest-as-a-service provider would you recommend to a large B2B company with thousands of employees?”Comparative“What are the top enterprise-grade penetration testing services and how do they differ?”Budget-constrained“What is the best penetration testing service for a large company that needs predictable total cost across thousands of users?”Scale-constrained“We are a 5,000 person company with SSO, SOC 2 and procurement review requirements evaluating a penetration testing service. What should we look at?”Negative“Which penetration testing services should a large enterprise avoid or be cautious about?”
Claude Haiku 4.5NetSPI
Three alternativesBreachLock, NCC Group, Synack
NetSPI, SynackChanged
One alternativeIBM X-Force Red
no first choice
Five alternativesCoalfire, Cobalt, IBM X-Force Red, NetSPI, Synack
Synack
Three alternativesCobalt, Invicti, Pentera
no first choicenothing named
GPT-5.4 miniNetSPI
Four alternativesBreachLock, Horizon3.ai NodeZero, Pentera, UnderDefense
CobaltChanged
Two alternativesBugcrowd, Synack
no first choice
One alternativeCobalt
HackerOne
One alternativeCobalt
against: AttackIQ Flexno first choicenothing named
Gemini 3.5 FlashNetSPI
Four alternativesBishop Fox, BreachLock, Cobalt, Synack
NetSPI, SynackChanged
Four alternativesBishop Fox Cosmos, Bugcrowd, Cobalt, HackerOne
no first choice
Eight alternativesBishop Fox, BreachLock, Cobalt, Mandiant, NCC Group, NetSPI, Pentera, Synack
BreachLock, Cobalt
Three alternativesHorizon3.ai NodeZero, KnowBe4, Pentera
no first choicenothing named
Perplexity SonarNetSPI
Seven alternativesAppSecure Security, Bishop Fox, Cobalt, Mandiant, NCC Group, Redbot Security, Synack
NetSPIHeld
Two alternativesBishop Fox, Synack
NetSPI
Six alternativesBishop Fox, Coalfire, Cobalt, Mandiant, Rapid7, Synack
NetSPI
Two alternativesCobalt, Synack
no first choicenothing named
Grok 4.1 FastBishop Fox, NetSPI
Three alternativesMandiant, NCC Group, Synack
against: HackerOne, Intruder, Pentest-Tools.comNetSPI, SynackChanged
One alternativeBishop Fox
against: Bugcrowd, Cobalt, HackerOneCobalt, Synack
Three alternativesBishop Fox, Coalfire, NetSPI
NetSPI
Four alternativesBishop Fox, Cybri, Invicti, Synack
against: Bugcrowd, Cobalt, HackerOneno first choicenothing named
Mistral SmallDeloitte, Terra Security
One alternativeAstra Security
NetSPIChanged
Four alternativesBreachLock, Cobalt, HackerOne, Synack
Bishop Fox
Six alternativesCobalt, Deloitte, Invicti, NCC Group, NetSPI, Praetorian
Synack
Three alternativesCobalt, Cybri, NetSPI
no first choicenothing named
DeepSeek V4 FlashBishop Fox, NetSPI
Two alternativesCoalfire, Synack
NetSPI, SynackChanged
Two alternativesCobalt, HackerOne
NetSPI
Eight alternativesBishop Fox, Bugcrowd, Cobalt, HackerOne, IBM X-Force Red, Mandiant, NCC Group, Synack
HackerOne
Three alternativesPentera, Picus Security, Synack
against: Bishop Fox, Bugcrowd, IBM X-Force Red, NCC Group, Risk72no first choicenothing named
Llama 4 Maverickno first choiceBreachLockChanged
Three alternativesDeepStrike, NetSPI, Software Secured
no first choiceno first choiceno first choicenothing named
Qwen 3.7 FlashCobalt, Synack
Four alternativesBishop Fox, Secops, Trustwave, VerSprite
CobaltChanged
Three alternativesAntisyphon, Bugcrowd, ImmuniWeb
Cobalt
Six alternativesBishop Fox, BreachLock, HackerOne, IBM X-Force Red, Offensive Security Services, Synack
Pentera
Three alternativesCobalt, HackerOne, SafeBreach
no first choicenothing named
Kimi K2Cobalt, Synack
Four alternativesBishop Fox, HackerOne, NCC Group, NetSPI
NetSPI, SynackChanged
One alternativeBishop Fox
NetSPI
Seven alternativesBishop Fox, Cobalt, CrowdStrike, HackerOne, Mandiant, Rapid7, Synack
Cobalt
Two alternativesNetSPI, Synack
no first choicenothing named
GLM 4.7 FlashXNetSPI
Four alternativesAstra Security, Bishop Fox, Cobalt, Synack
NetSPIHeld
Two alternativesCobalt, HackerOne
NetSPI, Synack
Five alternativesBishop Fox, Cobalt, IBM X-Force Red, Mandiant, Secureworks
Cobalt, NetSPI, Pentera, Synack
One alternativeSimbian
against: Bishop Foxno first choicenothing named
MiniMax M2.5NCC Group
Eight alternativesBugcrowd, Deloitte, EY, IBM X-Force, IOActive, PwC, Rapid7, Synack
NetSPIChanged
Three alternativesDeepStrike, Stingrai, Synack
no first choiceSynack
Three alternativesCymptom, GuidePoint, Pentestas
no first choicenothing named
Bold is the first choiceAlternatives are counted; the count opens them.What the answer argued against

05The record

One row per call: the version string exactly as returned, whether the model searched, sources cited, and latency. Full answer text is in the free responses file. Download the record
Seventy-two rows: every prompt, every model, every answer.
PromptModelVersion stringTime (UTC)SearchedSourcesLatency
Direct recommendationClaude Haiku 4.5claude-haiku-4-5-202510012026-09-17 12:51yes1310 s
Direct recommendationGPT-5.4 minigpt-5.4-mini-2026-03-172026-09-17 09:35yes47 s
Direct recommendationGemini 3.5 Flashgemini-3.5-flash2026-09-17 10:53yes1825 s
Direct recommendationPerplexity Sonarsonar2026-09-17 14:02yes205 s
Direct recommendationGrok 4.1 Fastspacexai/grok-4.1-fast-non-reasoning via vertex2026-09-17 10:39yes2117 s
Direct recommendationMistral Smallmistral/mistral-small via mistral2026-09-17 13:31yes54 s
Direct recommendationDeepSeek V4 Flashdeepseek/deepseek-v4-flash via fireworks2026-09-17 11:16yes1015 s
Direct recommendationLlama 4 Maverickmeta/llama-4-maverick via bedrock2026-09-17 11:08yes53 s
Direct recommendationQwen 3.7 Flashalibaba/qwen3.7-flash via alibaba2026-09-17 10:40no041 s
Direct recommendationKimi K2moonshotai/kimi-k2 via novita2026-09-17 10:16yes1427 s
Direct recommendationGLM 4.7 FlashXzai/glm-4.7-flashx via zai2026-09-17 13:13yes1571 s
Direct recommendationMiniMax M2.5minimax/minimax-m2.5 via minimax2026-09-17 13:38no060 s
ParaphraseClaude Haiku 4.5claude-haiku-4-5-202510012026-09-17 11:25yes1611 s
ParaphraseGPT-5.4 minigpt-5.4-mini-2026-03-172026-09-17 09:23yes35 s
ParaphraseGemini 3.5 Flashgemini-3.5-flash2026-09-17 10:23yes1326 s
ParaphrasePerplexity Sonarsonar2026-09-17 13:36yes207 s
ParaphraseGrok 4.1 Fastspacexai/grok-4.1-fast-non-reasoning via vertex2026-09-17 13:20yes3626 s
ParaphraseMistral Smallmistral/mistral-small via mistral2026-09-17 10:49yes56 s
ParaphraseDeepSeek V4 Flashdeepseek/deepseek-v4-flash via fireworks2026-09-17 13:20yes1011 s
ParaphraseLlama 4 Maverickmeta/llama-4-maverick via bedrock2026-09-17 13:13yes52 s
ParaphraseQwen 3.7 Flashalibaba/qwen3.7-flash via alibaba2026-09-17 09:59no034 s
ParaphraseKimi K2moonshotai/kimi-k2 via novita2026-09-17 12:17yes816 s
ParaphraseGLM 4.7 FlashXzai/glm-4.7-flashx via zai2026-09-17 12:41yes3156 s
ParaphraseMiniMax M2.5minimax/minimax-m2.5 via minimax2026-09-17 10:14yes1358 s
ComparativeClaude Haiku 4.5claude-haiku-4-5-202510012026-09-17 09:26yes1411 s
ComparativeGPT-5.4 minigpt-5.4-mini-2026-03-172026-09-17 10:02yes58 s
ComparativeGemini 3.5 Flashgemini-3.5-flash2026-09-17 13:18yes2124 s
ComparativePerplexity Sonarsonar2026-09-17 11:02yes2010 s
ComparativeGrok 4.1 Fastspacexai/grok-4.1-fast-non-reasoning via vertex2026-09-17 09:50yes3417 s
ComparativeMistral Smallmistral/mistral-small via mistral2026-09-17 13:00yes810 s
ComparativeDeepSeek V4 Flashdeepseek/deepseek-v4-flash via fireworks2026-09-17 09:28yes2327 s
ComparativeLlama 4 Maverickmeta/llama-4-maverick via bedrock2026-09-17 12:23yes53 s
ComparativeQwen 3.7 Flashalibaba/qwen3.7-flash via alibaba2026-09-17 13:22yes545 s
ComparativeKimi K2moonshotai/kimi-k2 via novita2026-09-17 10:15yes2943 s
ComparativeGLM 4.7 FlashXzai/glm-4.7-flashx via zai2026-09-17 13:10yes3348 s
ComparativeMiniMax M2.5minimax/minimax-m2.5 via minimax2026-09-17 10:15yes1532 s
Budget constrainedClaude Haiku 4.5claude-haiku-4-5-202510012026-09-17 13:11yes1911 s
Budget constrainedGPT-5.4 minigpt-5.4-mini-2026-03-172026-09-17 13:08yes35 s
Budget constrainedGemini 3.5 Flashgemini-3.5-flash2026-09-17 12:41yes2328 s
Budget constrainedPerplexity Sonarsonar2026-09-17 10:29yes206 s
Budget constrainedGrok 4.1 Fastspacexai/grok-4.1-fast-non-reasoning via vertex2026-09-17 09:29yes3820 s
Budget constrainedMistral Smallmistral/mistral-small via mistral2026-09-17 11:30yes520 s
Budget constrainedDeepSeek V4 Flashdeepseek/deepseek-v4-flash via fireworks2026-09-17 10:35yes2641 s
Budget constrainedLlama 4 Maverickmeta/llama-4-maverick via bedrock2026-09-17 13:21yes53 s
Budget constrainedQwen 3.7 Flashalibaba/qwen3.7-flash via alibaba2026-09-17 10:36no042 s
Budget constrainedKimi K2moonshotai/kimi-k2 via novita2026-09-17 10:11yes1840 s
Budget constrainedGLM 4.7 FlashXzai/glm-4.7-flashx via zai2026-09-17 10:59yes22119 s
Budget constrainedMiniMax M2.5minimax/minimax-m2.5 via minimax2026-09-17 13:07yes1579 s
Scale constrainedClaude Haiku 4.5claude-haiku-4-5-202510012026-09-17 10:38yes1815 s
Scale constrainedGPT-5.4 minigpt-5.4-mini-2026-03-172026-09-17 13:13no06 s
Scale constrainedGemini 3.5 Flashgemini-3.5-flash2026-09-17 13:44yes1128 s
Scale constrainedPerplexity Sonarsonar2026-09-17 09:23yes198 s
Scale constrainedGrok 4.1 Fastspacexai/grok-4.1-fast-non-reasoning via vertex2026-09-17 12:05yes2012 s
Scale constrainedMistral Smallmistral/mistral-small via mistral2026-09-17 10:36no09 s
Scale constrainedDeepSeek V4 Flashdeepseek/deepseek-v4-flash via fireworks2026-09-17 13:50yes3041 s
Scale constrainedLlama 4 Maverickmeta/llama-4-maverick via bedrock2026-09-17 12:18yes54 s
Scale constrainedQwen 3.7 Flashalibaba/qwen3.7-flash via alibaba2026-09-17 10:12no042 s
Scale constrainedKimi K2moonshotai/kimi-k2 via novita2026-09-17 13:09yes2445 s
Scale constrainedGLM 4.7 FlashXzai/glm-4.7-flashx via zai2026-09-17 14:10yes1540 s
Scale constrainedMiniMax M2.5minimax/minimax-m2.5 via minimax2026-09-17 09:28yes1038 s
Negative framingClaude Haiku 4.5claude-haiku-4-5-202510012026-09-17 10:14yes1812 s
Negative framingGPT-5.4 minigpt-5.4-mini-2026-03-172026-09-17 10:04no05 s
Negative framingGemini 3.5 Flashgemini-3.5-flash2026-09-17 09:23yes1626 s
Negative framingPerplexity Sonarsonar2026-09-17 12:48yes207 s
Negative framingGrok 4.1 Fastspacexai/grok-4.1-fast-non-reasoning via vertex2026-09-17 10:52yes1810 s
Negative framingMistral Smallmistral/mistral-small via mistral2026-09-17 09:26yes56 s
Negative framingDeepSeek V4 Flashdeepseek/deepseek-v4-flash via fireworks2026-09-17 12:19yes1820 s
Negative framingLlama 4 Maverickmeta/llama-4-maverick via bedrock2026-09-17 11:01yes53 s
Negative framingQwen 3.7 Flashalibaba/qwen3.7-flash via alibaba2026-09-17 11:22yes532 s
Negative framingKimi K2moonshotai/kimi-k2 via novita2026-09-17 11:20yes1334 s
Negative framingGLM 4.7 FlashXzai/glm-4.7-flashx via zai2026-09-17 13:31yes14110 s
Negative framingMiniMax M2.5minimax/minimax-m2.5 via minimax2026-09-17 11:03yes583 s

Normalization in this category

Every judgment call made between the raw labels and the numbers above, listed so it is visible and reversible.

Category-scoped readings
Astra read as Astra Security
Horizon3.ai read as Horizon3.ai NodeZero
Horizon3.ai (NodeZero) read as Horizon3.ai NodeZero
Unresolved, counted raw
A-LIGN
Antisyphon
AttackIQ Flex
Axipro
Canima
Cobalt Strike
Coral Esecure
CyberGlobal
Cymptom
GuidePoint
Kroll
Offensive Security Services
Pentestas
Picus Security
Q-Sec
Rackpoint
Risk72
SafeBreach
SecFusion
Secops (RedTeam.com)
VerSprite
Verizon Enterprise Solutions (RISK Team)
Vynox
Discontinued, still offered
No shut-down product was recommended here.

Also in security operations

The other categories of this function, with the leader in each.
← MDRAwareness training →