Google Mantis: An Agentic Vulnerability Scanning Harness for Reducing False Positives

Sergio De Simone · rss · 2026-09-06 Reader 🖥️

Agent Security AI-Driven Vulnerability Discovery Model Routing Multi-Agent Orchestration Retrieval & RAG Sandboxing Token Efficiency

Google built Mantis, an open-source multi-agent harness that combines critic/reviewer agents, hierarchical repository context, and sandboxed reproduction of vulnerabilities, because conventional AI-powered vulnerability scanning is unreliable — plagued by hallucinated bugs and true-positive rates under 7% — and false positives can only be brought under control through disciplined agentic engineering, not by trusting a single LLM's judgment.

practical · expository — A short InfoQ news report on a newly open-sourced Google tool. It explains what Mantis does and how it is architected, largely through paraphrase plus two direct quotes from Google's own blog posts; it makes no independent argument of its own beyond organizing and summarizing Google's claims. There is no narrative arc, no counter-evidence, and no author-original argument — the piece is a practical technology writeup, not a theoretical treatise.

Structure

Part	Role
Opening framing (what Mantis is and why it exists)	thesis statement
Google's quoted rationale (<7% true-positive rate; critic/review + sandboxed reproduction)	evidence / authority citation
Context-analysis description (repo history, architecture, threat models, hierarchical tree summarization, 85% token reduction)	mechanism detail
Agent roster (critic, reviewer, strategist, research agents)	mechanism detail
Modular skill-suite description (15+ tools, shared on-disk state)	mechanism detail
Model-selection guidance (flash/lite vs. frontier models per stage)	mechanism detail / secondary theme
mantis-review negative filter, with Google's caution against over-broad filtering	qualification / self-critique
Availability and pointer to full agent reference guide	closing / call to action

Key terms

hallucinated vulnerabilities
Findings an LLM-based scanner reports as real security bugs that do not actually exist in the code — a failure mode distinct from merely low-severity or low-confidence findings.
vulnerability lifecycle
The full sequence from identifying a candidate bug, through validating it, to reproducing it in a controlled environment, to fixing it — Mantis is scoped to automate all four stages, not just detection.
agentic harness
The orchestration layer wrapping multiple specialized AI agents (critic, reviewer, strategist, researcher) and 15+ modular tools coordinating through shared on-disk state — the engineering scaffold, not any single model call.
grounding Retrieval & RAG
Providing verifiable, sandboxed-execution evidence that a flagged vulnerability is real, rather than relying solely on an LLM's textual judgment about the code.
critic and reviewer agents
Distinct agent roles: a critic agent adversarially evaluates candidate findings, and a separate reviewer agent (via the mantis-review stage's negative filter) screens out likely false positives before findings reach a human.
negative filter
A rule-based filtering mechanism in the mantis-review stage that eliminates candidate findings judged to be false positives; explicitly flagged by Google as risky if made too broad, since low-risk-looking findings should not be automatically discarded.

Propositions

explicitcorroboratedGoogle open-sourced Mantis, an AI-agent framework designed to automate the full software vulnerability lifecycle — identifying, validating, reproducing, and fixing bugs.conf: high
supports ←Agentic AI in the Software Development Lifecycle (SDLC) · p5 — Mantis is a concrete, specified example of exactly the kind of AI-driven vulnerability-scanning agent this memo asserts Telenor is 'already leveraging' in more general terms.
explicitGoogle built Mantis specifically to address the high false-positive rate and hallucinated vulnerabilities produced by conventional AI-powered code scanning, whose true-positive rate falls under 7%.conf: high
extendsProject Glasswing: what Mythos showed us · p4 — Mythos coverage quantifies false-positive rate as dominated by target-language memory safety and model choice; Mantis proposes a harness-engineering answer to the same false-positive problem — critic/reviewer agents plus sandboxed grounding — applicable regardless of which underlying model or language is involved.
explicitMantis achieves its accuracy improvement by combining critic and review agents with sandboxed reproduction of vulnerabilities for grounding, rather than relying solely on the LLM's own judgment.conf: high
supportsUsing AI to write better code more slowly · p5 — Both treat independent, structured multi-stage verification — sandboxed reproduction here, independent research there — as the mechanism that makes a false-positive-filtering claim trustworthy rather than an unverified LLM assertion.
explicitMantis analyzes repository history, previous security fixes, architecture, and threat models before scanning, rather than brute-force scanning files blindly.conf: high
explicitMantis summarizes analyzed files into a hierarchical directory/repository-level context tree, cutting token usage by 85% while retaining important structural information.conf: high
explicitMantis is organized as a modular skill suite of 15+ tools (e.g. mantis-summarize, mantis-review, mantis-critic) executable sequentially or in parallel, communicating through shared on-disk state.conf: high
explicitMantis's guidance recommends pairing lightweight 'flash'/'lite' models with low-depth tasks (rapid classification, deduplication) and reserving heavier frontier models for tasks needing deep contextual understanding (crash reproduction, side-effect-free patch generation).conf: high
explicitThe mantis-review stage applies a rule-based negative filter to eliminate likely false positives, but Google cautions the filter should be used carefully because an overly broad filter could reduce the system's ability to detect genuine vulnerabilities.conf: high
supportsWhen Guardrails Go Wrong · p3 — Google's own caution about the mantis-review negative filter (avoid over-broad filtering or true positives get lost) is a concrete instance of the general no-binary-classifier-achieves-zero-both-errors trade-off this proposition states abstractly.
implicitBecause Mantis requires sandboxed reproduction as evidence rather than trusting a critic agent's textual assessment alone, Google treats raw LLM judgment about vulnerabilities as structurally insufficient on its own, not merely as something to be improved with better prompting.conf: medium
supportsArticle: Multi-Agent AI for Production Security Operations: An A2A and MCP Architecture in a 5G Core · p3 — Both pieces embed the same underlying distrust of raw LLM judgment in a security-agent context: Mantis requires sandboxed-execution evidence rather than trusting a critic agent's text output, just as the reviewer agent there is required to call out to version-controlled deterministic policy rather than reason freely.
implicitThe division of labor between a strategist agent (high-level structure, threat models, dependency graphs) and research agents (source-level data-flow, control-flow, and sanitization-logic tracing) implies Mantis's architecture deliberately separates broad contextual reasoning from narrow code-level verification, rather than asking one agent to do both.conf: medium
implicitThe article itself never states any quantified accuracy figure for Mantis's own output (its true-positive or false-positive rate after the review/critic stages) — only the problem baseline it addresses (<7% true-positive rate for 'conventional' scanning) is quantified.conf: high

Arguments

Mantis's harness-engineering approach — a multi-agent pipeline combining critic/reviewer agents with sandboxed reproduction for grounding — is presented as the solution to the false-positive and hallucination problem endemic to AI-powered vulnerability scanning. strength: medium

Explicit premises:

Conventional AI code scanning produces hallucinated bugs and true-positive rates under 7%.
Critic and reviewer agents filter out false positives and prioritize meaningful findings.
Sandboxed reproduction provides evidence that a finding is real rather than relying solely on LLM judgment.

Implicit premises:

False positives and hallucinations in AI vulnerability scanning are primarily an architecture/engineering problem, solvable by structured multi-stage verification, rather than solely a raw-model-capability problem that only a better base model can fix.
A negative filter, used cautiously, removes more false positives than true positives — the article assumes this balance is achievable without stating how it is measured or enforced.
Structured, multi-stage agentic verification is inherently more reliable than a single LLM call making an end-to-end judgment, even though each stage is still LLM-driven and therefore still fallible.

Inferences:

Therefore a well-engineered agentic harness can make AI vulnerability scanning practically trustworthy despite the underlying unreliability of any single LLM judgment.

Cost and speed should be optimized by matching model weight-class to task difficulty within the same vulnerability-scanning pipeline, rather than running every stage on the most capable available model. strength: medium