Project Glasswing: what Mythos showed us

Grant Bourzikas · article · 2026-05-18 source ↗ Reader 🖥️

Agent Harness AI-Driven Vulnerability Discovery Context Rot & Context Limits Multi-Agent Orchestration Mythos Project Glasswing

Frontier security-focused LLMs (exemplified by Mythos Preview) have made a real qualitative jump in exploit-chain construction and proof generation, but their value at production scale is unlocked not by faster patching or smarter single-agent use, but by wrapping them in a multi-stage harness of narrow, parallel, adversarially-validated agents and by hardening the architecture around vulnerabilities so that the gap between disclosure and patch matters less.

practical · mixed — Argumentative-prescriptive with case-study reporting. Bourzikas reports empirical observations from running Mythos Preview against Cloudflare code, but the post is ultimately in service of prescriptions: how to build a harness around security LLMs, and how to architect security around vulnerabilities rather than just patching faster. The post also functions as ecosystem positioning (Cloudflare as Anthropic partner, and as the natural defensive layer when LLM-enabled attacks proliferate).

Structure

Part	Role
Opening framing: Cloudflare has been testing security-focused LLMs on its own infrastructure for months; Mythos Preview from Anthropic, accessed via Project Glasswing, was pointed at more than fifty Cloudflare repositories.	context and credentialing
What Mythos does well: exploit chain construction and proof generation (with compile/run loop).	evidence (qualitative empirical claim)
Comparison to other frontier models: similar bug-finding, but they stop at the chain-stitching step.	evidence (comparative)
Safety/refusals: Mythos lacks the safeguards of GA models; emergent guardrails exist but are inconsistent and prompt-framing-sensitive.	qualified caveat / policy claim
Triage and noise: programming language (C/C++ vs Rust) and model hedging bias drive false-positive rate; Mythos reduces hedged findings.	evidence + diagnostic
Failure of the naive approach: point a coding agent at a repo and ask for vulns. Reasons: context window / compaction (covers ~0.1% of a 100k-line repo usefully), and throughput / single-stream serial work.	argument (negative case)
Four lessons: narrow scope, adversarial review, splitting the chain across agents, parallel narrow tasks.	prescriptive principles
Harness stages: Recon, Hunt (~50 concurrent hunters), Validate, Gapfill, Dedupe, Trace, Feedback, Report — applied to runtime, edge data path, protocol stack, control plane, and OSS dependencies.	operational illustration of the harness principles
Speed-fetish critique: a two-hour CVE-to-patch SLA is the wrong frame; skipping regression testing produces worse bugs; letting the model write patches has already broken things.	counter-argument against industry instinct
Architectural principles: defenses in front of the application, blast-radius isolation, instantaneous global rollout.	prescription / product positioning
Dual-use coda: the same capabilities accelerate attackers; Cloudflare sits in front of millions of applications.	framing / ecosystem positioning

Key terms

Mythos Preview Mythos
An Anthropic security-focused LLM (a 'cyber frontier model') made available to Cloudflare through Project Glasswing, without the standard safeguards present in GA models like 'Opus 4.7 or GPT-5.5'. Distinguished from general-purpose frontier models by its ability to chain attack primitives and generate compiled, executed proofs of exploitability.
Project Glasswing Project Glasswing
An Anthropic program through which select partners (including Cloudflare) were invited to use the Mythos Preview model on their own code in a controlled research context.
Security-focused LLM / cyber frontier model AI-Driven Vulnerability Discovery
A frontier model whose capabilities are oriented toward vulnerability research — finding bugs, reasoning about exploitability, and constructing exploits — as distinct from general-purpose coding assistants tuned for feature work.
Live code
Code actually running in Cloudflare's production environment — runtime, edge data path, protocol stack, control plane, and the open-source projects they depend on — as opposed to test corpora or synthetic benchmarks.
Harness Agent Harness
The orchestration layer around the model — a multi-stage pipeline of narrow, parallel, adversarially-validated agents (Recon, Hunt, Validate, Gapfill, Dedupe, Trace, Feedback, Report) that converts raw model output into actionable findings. Not a chat interface; closer to a distributed system in which the model is a component.
Exploit chain construction AI-Driven Vulnerability Discovery
Taking several low-severity attack primitives and reasoning about how to combine them into a working exploit (e.g., use-after-free -> arbitrary read/write -> control-flow hijack -> ROP chain). Mythos's distinctive ability.
Proof generation AI-Driven Vulnerability Discovery
Closing the gap between 'suspected bug' and 'proven exploit' by writing code that triggers the bug, compiling it in a scratch environment, running it, and iterating on failure. The loop, not the bugs, is the point.
Adversarial review Multi-Agent Orchestration
Putting a second agent — with a different prompt, a different model, and no ability to emit its own findings — between the initial finding and the queue, deliberately disagreeing with the first agent.
Wander problem AI-Driven Vulnerability Discovery
The failure mode of generic coding agents pointed at a repo with 'find vulnerabilities': they spread shallowly across the codebase rather than going deep on a narrow hypothesis.
Context anxiety (implicit) Context Rot & Context Limits
Not used verbatim. Bourzikas describes the analogous phenomenon: as a single-agent session approaches its context-window limit, compaction kicks in 'potentially discarding earlier findings that would have mattered.' The harness exists in part to avoid this.

Propositions

explicitMythos Preview was tested against more than fifty Cloudflare repositories spanning runtime, edge data path, protocol stack, control plane, and OSS dependencies.conf: high
explicitMythos's Hunt stage runs approximately fifty hunter agents concurrently, each fanning out to a handful of exploration subagents.conf: high
explicitA single coding-agent session against a 100,000-line repository can usefully cover at most about a tenth of a percent of the surface before context compaction discards earlier findings.conf: high
supportsHarness design for long-running application development · p1 — Quantifies the context-coherence failure mode in the field: a single session on a 100k-line repo covers ~0.1% of the surface before compaction discards earlier findings.
explicitdevelopedFalse-positive rate from security LLMs is dominated by two factors: programming-language memory safety (C/C++ produce more false positives than Rust) and model hedging bias.conf: high
extends ←Google Mantis: An Agentic Vulnerability Scanning Harness for Reducing False Positives · p2 — Mythos coverage quantifies false-positive rate as dominated by target-language memory safety and model choice; Mantis proposes a harness-engineering answer to the same false-positive problem — critic/reviewer agents plus sandboxed grounding — applicable regardless of which underlying model or language is involved.
explicitMythos Preview produces noticeably higher-quality triage output than prior frontier models: fewer hedged findings, clearer reproduction steps, less work to fix-or-dismiss.conf: high
explicitcontestedMythos Preview's distinguishing capability over other frontier models is chain-stitching: turning several low-severity primitives into a single, more severe exploit, where prior frontier models would identify the bug but stop short of finishing the chain.conf: high
contradictsThree reasons to think that the Claude Mythos announcement from Anthropic was overblown · p3 — Cloudflare's firsthand evaluation finds chain-stitching a qualitatively distinguishing capability, contradicting the incremental-only assessment.
explicitMythos Preview's emergent guardrails are inconsistent: the same task framed differently, or run again, can produce opposite refuse/comply outcomes.conf: high
explicitAny GA cyber-capable frontier model must include additional safeguards on top of emergent baseline behavior; emergent refusals alone are not a complete safety boundary.conf: high
explicitGeneric coding agents are the wrong shape for vulnerability research because coding agents hold a single hypothesis at a time while vulnerability research is narrow-and-parallel by nature.conf: high
explicitSplitting the question 'is this code buggy?' from 'can an attacker actually reach this bug from outside?' across separate agents produces better reasoning on each than asking them together.conf: high
supportsDesigning a Multi-Agent System for Engineering Support at Scale: A Case Study From Grab · p1 — Both independently find that splitting one hard question across separate specialized agents (buggy-vs-reachable; diagnose-vs-write) produces better reasoning than asking it whole.
explicitcorroboratedAdversarial review (a second, finding-suppressing agent with a different prompt and model) is more effective than asking the original agent to check its own work.conf: high
supportsHarness design for long-running application development · p3 — Independent Cloudflare corroboration: a separate adversarial reviewer with a different prompt and model beats asking the original agent to check its own work.
extendsHarness design for long-running application development · p3 — Cloudflare extends the standalone-skeptical-evaluator claim by showing adversarial review works best with a different prompt AND different model, in production security triage.
supports ←Have your agent record video demos of its work with shot-scraper video · p9 — Corroborates that a reviewer with a different model (and prompt) is more effective than asking the original agent to check its own work.
explicitReachability tracing (Stage: Trace) — determining whether attacker-controlled input actually reaches a bug from outside the system — is the harness stage that matters most.conf: high
explicitestablishedIndustry teams targeting a two-hour SLA from CVE release to patch in production are pursuing a goal that will mostly fail or backfire, because patching faster does not change the pipeline that produces the patch.conf: high
supports ←IBM and Red Hat Commit $5 Billion to Redefine the Future of Open Source in the AI Era · p2 — Aligns with the argument that fixing OSS security requires changing the patch-production pipeline itself rather than just chasing faster incident-response SLAs — Lightwell frames itself as exactly that kind of pipeline-level intervention (upstream fixes, validated patches, lifecycle management).
supports ←Predict, Don't Enumerate · p6 — Independently argues that faster remediation without local prioritization increases churn rather than reducing risk, matching Cloudflare's case against two-hour patch SLAs.
explicitCloudflare has, on at least some occasions, let the model write its own patches and shipped patches that fixed the original bug while breaking other things the code depended on.conf: high
explicitThe correct response to AI-accelerated vulnerability discovery is architectural hardening (front-of-application defenses, blast-radius isolation, instantaneous global rollout), not faster patching.conf: high
explicitThe same security-LLM capabilities Cloudflare used defensively will be used by attackers against every Internet-exposed application.conf: high
implicitCloudflare's product suite already implements the architectural principles needed to mitigate LLM-accelerated attacks (implicit: 'so our customers don't need to rebuild their architecture, they need us').conf: high
explicitFindings carrying a working PoC are categorically different (and far cheaper to triage) than hedged findings without one.conf: high
explicitHarnesses can be bootstrapped by asking the model itself to help improve the harness ('we used Mythos Preview to build on, tailor, and improve our original harnesses to suit its strengths').conf: high
explicitHedging bias is a 'reasonable bias for an exploratory tool' but 'a ruinous one for a triage queue' because every speculative finding spends human attention and tokens to dismiss.conf: high
explicitCloudflare's harnesses are deliberately tuned to over-report (more noise, less missed), implying the harness must do downstream noise removal rather than upstream precision.conf: high

Arguments

Faster patching is not a sufficient defensive response to AI-accelerated vulnerability discovery. strength: high

Explicit premises:

Attacker timelines are shortening because of cyber frontier models.
Patching faster does not change the shape of the pipeline that produces the patch.
Skipping pipeline stages like regression testing produces worse bugs than the ones being patched.
Cloudflare empirically shipped model-written patches that broke unrelated functionality.

Implicit premises:

The marginal value of a faster patch is bounded by the reliability of the patching pipeline.
A worse-quality patch deployed faster has negative expected value compared to a higher-quality patch deployed later (a Beck/Cutler-style flow-vs-batch claim).
Architectural defenses are independently deployable and do not require touching the per-bug pipeline.

Inferences:

Two-hour CVE-to-patch SLAs are a category error rather than an aggressive target.
Investment dollars currently chasing patch-speed should be reallocated to in-front defenses and blast-radius isolation.
Cloudflare's edge products are well-positioned to capture that reallocated investment (unstated by author but follows directly).

The right unit of analysis for production security LLMs is the harness, not the model. strength: high

Explicit premises:

Generic coding agents fail at vulnerability research because they are single-hypothesis-at-a-time.
Vulnerability research is intrinsically narrow-and-parallel.
Context windows + compaction cap single-session coverage at roughly 0.1% of a 100k-line repo.
Adversarial review by a second agent catches noise the first agent misses.
Splitting 'is this buggy?' from 'is this reachable?' improves both.
Many narrow parallel tasks plus deduplication beats one exhaustive agent.

Implicit premises:

Model capabilities improve faster than the harness around them does, so leverage comes from harness work.
Coverage is the right success metric for production vulnerability research (vs. e.g. bug severity, attacker-realism, or mean-time-to-exploit-detection).
The shape of vulnerability research generalizes — narrow-and-parallel is intrinsic to the task, not just to Cloudflare's codebase.

Inferences:

Vendors selling 'AI SAST as a chatbot' or 'one agent that scans your repo' are mis-shaped for the workload.
Investment in harness engineering compounds across model upgrades in a way investment in prompt engineering does not.

GA frontier models with cyber capabilities require external safeguards beyond emergent guardrails. strength: medium

Explicit premises:

Mythos Preview's emergent refusals exist but are inconsistent.
Semantically equivalent tasks produce opposite outcomes across framings and runs.
Probabilistic decoding means even identical requests can differ across runs.

Implicit premises:

Inconsistency is the binding constraint on using emergent guardrails as a safety boundary (rather than e.g. severity of failure modes or attacker adaptability).
External safeguards (the form of which is unspecified) can be made more consistent than emergent ones.

Inferences:

Pre-deployment red-teaming alone is insufficient because it can find one framing that refuses and miss the framing that complies.
Anthropic (and peer labs) bear a positive duty to add safeguards before broader release — this is partly an industry-policy argument dressed as an empirical observation.

Cloudflare's experience with Mythos Preview validates Anthropic's bet on Mythos Preview as a meaningful capability step.

Explicit premises:

Mythos achieves exploit chain construction and proof generation that prior frontier models did not finish.
Mythos's outputs require less triage work, with clearer reproduction steps.
Cloudflare voluntarily rebuilt its harness around Mythos's strengths.

Implicit premises:

Cloudflare's reporting is unbiased despite being an invited Glasswing partner.
The capability jump observed at Cloudflare generalizes beyond Cloudflare's stack and beyond Cloudflare's harness investment.
Anthropic's safety story (emergent refusals + external safeguards) is the right governance frame for these capabilities.

Inferences:

Cloudflare is signaling preferred-partner status with Anthropic.
The post is a soft co-marketing artifact as well as a research report.

Narratives

Mythos refused vulnerability research on a project, then agreed to perform the same research on the same code after an unrelated change to the project's environment — nothing about the analyzed code had changed.
→ Concretizes the proposition that emergent guardrails are not a sufficient safety boundary; without this anecdote, the claim is theoretical.
Mythos found and confirmed several serious memory bugs in a codebase, then refused to write a demonstration exploit; the same request, framed differently, got a different answer.
→ Demonstrates that the refusal is sensitive to surface framing rather than to actual harm — supports the call for external safeguards.
Cloudflare let the model write its own patches and watched a few go out that fixed the original bug while quietly breaking something else the code depended on.
→ Concrete empirical evidence for the broader argument against compressing the response cycle by automating patch generation. This is the single most consequential admission in the post.
When Cloudflare first started AI-assisted vulnerability research a year ago, they pointed a generic coding agent at an arbitrary repository and asked it to discover vulnerabilities; this 'works' in that the model produces findings, but does not produce meaningful coverage of value.
→ Origin story for the harness, supporting the claim that the harness is necessary rather than a refinement.
Exploit chain narrative: turning a use-after-free into arbitrary read/write, hijacking control flow, and using ROP chains for full system control — described as the canonical thing Mythos can now do end-to-end.
→ Makes the abstract 'exploit chain construction' capability concrete and assesses Mythos as comparable to a senior researcher rather than an automated scanner.

Questions addressed

✅ What can current security-focused LLMs actually do on production code?
Mythos Preview can construct multi-primitive exploit chains and autonomously generate compile/run proofs of exploitability; prior frontier models find similar bugs but stop short of chaining.
✅ What are the production failure modes of LLM-based vulnerability research?
Hedging bias inflates triage queues; memory-unsafe languages amplify false positives; single-agent context limits cap useful coverage at ~0.1% of a large repo.
✅ How should an organization architect around these models to extract production value?
Build a multi-stage harness (Recon, Hunt, Validate, Gapfill, Dedupe, Trace, Feedback, Report) of narrow, parallel, adversarially-validated agents; bootstrap with the model's help.
✅ Are emergent model refusals a sufficient safety boundary for GA cyber-capable models?
No — they are real but inconsistent; external safeguards must be added.
✅ What is the correct strategic response to attacker use of these same capabilities?
Architectural hardening (front-of-app defenses, blast-radius isolation, instantaneous global rollout) rather than patch-speed races.
⭕ Does the Cloudflare-specific harness design generalize to other organizations' stacks (different languages, less homogeneity, less mature trust-boundary documentation)?
Not addressed.
⭕ What is the cost (in tokens, agent-hours, dollars) per confirmed reachable vulnerability, and how does that compare to traditional SAST/DAST or human pentesting?
Not addressed.
⭕ What is Mythos's false-negative rate? Are there bug classes the harness systematically misses?
Not addressed.
⭕ How does the harness behave against adversarially seeded vulnerabilities — code written knowing an LLM is in the loop?
Not addressed.

Gaps

incompleteNo base rate or comparison to non-LLM SAST/DAST. The post claims Mythos produces 'noticeably higher quality' triage output and that the harness produces 'meaningful coverage', but provides no quantitative baseline against either prior frontier models or traditional static analysis. The reader has to take Cloudflare's qualitative judgment on faith.
Read next: Read the Anthropic harness paper (planner/generator/evaluator) and any forthcoming Mythos technical report for quantified eval data; cross-reference with industry SAST benchmarks (e.g. OWASP Benchmark) to triangulate.
incompleteVendor-incentive analysis is absent. Cloudflare was an invited Glasswing partner and benefits from being perceived as an Anthropic-grade security research org. The post is simultaneously a research report, a co-marketing artifact, and a strategic positioning piece for Cloudflare's edge product suite ('our products are built to apply [these principles] on behalf of customers'). None of this is acknowledged.
Read next: Read Schneier on Security or Marcus on AI safety for an external skeptic's read; look for independent reproductions of Cloudflare's claims by orgs without an Anthropic partnership.
incompleteNo adversarial robustness data. If attackers know LLM-based scanners are in the loop, they can seed code with patterns that exploit known model biases — false-negative attractors, hedging triggers, or refusal-inducing framings on the defender's side. The post discusses defender adversariality (adversarial review between agents) but not attacker adversariality.
Read next: Anthropic's own work on adversarial robustness; the broader literature on prompt injection and indirect prompt injection in agentic systems.
incompleteGeneralizability across stacks is asserted by silence. Cloudflare's codebase — heavily Rust, well-documented, with explicit trust boundaries and the discipline that comes from running a global edge network — is unusually well-suited to the harness as described. Whether the same harness pays off on a typical enterprise Java monorepo with sparse documentation and ambiguous trust boundaries is an open empirical question.
Read next: Look for similar reports from non-Cloudflare practitioners (Google Project Zero, Microsoft, GitHub Advanced Security with Copilot Autofix at scale) using comparable harness architectures; pair with Böckeler on variety reduction as the precondition for any harness to work.
incompleteThe 'context anxiety / compaction discards earlier findings' claim is offered as a one-line motivation for the harness but is the most theoretically interesting failure mode in the post — and the one that connects most directly to the user's harness-engineering reading thread. Bourzikas leaves the mechanism opaque (how often does compaction discard a real finding? does Mythos behave differently from general frontier models on this dimension?).
Read next: Re-read the Anthropic harness paper's 'context anxiety' discussion; pair with Böckeler's variety-reduction framing and the smallcode failure-mode catalog; consider an extraction comparing the three on a single axis: 'what is the model's relationship to its own context, and what does the harness do about it?'
incompleteThe model-written-patches anecdote is mentioned in one sentence and then abandoned. How many patches went out? How were the regressions detected — by Cloudflare's own pipeline or by customer impact? What was rolled back? Given that this is a public commitment that Cloudflare shipped broken patches authored by an LLM, the lack of detail is itself a finding.
Read next: Watch for follow-up Cloudflare posts; pair with research on LLM-generated patch quality (e.g. recent academic work on Copilot Autofix regression rates).
illogicalThe two-hour SLA argument elides a continuum. Bourzikas treats 'two-hour SLA' as the strawman and 'architectural hardening' as the alternative, but the realistic question is whether to push patch-speed to four hours, twelve hours, or 24 hours while also investing in architecture. The dichotomy is rhetorically convenient and conveniently routes attention toward Cloudflare's product moat.
Read next: Read existing CVE-to-patch SLA literature (e.g. industry reports on time-to-remediate) to understand the actual distribution; question whether Cloudflare's customers are the ones being warned-off the two-hour target.
uninformed (potentially)No engagement with formal-methods or non-LLM approaches to reachability analysis (the 'Trace' stage's job). Static analyzers, taint trackers, and symbolic execution engines have been doing reachability for decades; the post implicitly treats LLM tracer agents as a novel solution without engaging with what they replace.
Read next: Compare against CodeQL, Semgrep, and formal taint-tracking literature; ask whether LLM tracer agents complement or substitute for these.

Closes gaps elsewhere

Harness design for long-running application development · g4 — Answers the same-base-model evaluator-bias question with a working intervention: Cloudflare's adversarial review uses a second finding-suppressing agent with a different prompt and a different model (p11).
Three reasons to think that the Claude Mythos announcement from Anthropic was overblown · g1 — Supplies the missing independent evaluation: Cloudflare's testing across 50+ repositories finds qualitatively distinct chain-stitching and triage quality, contradicting the unevidenced 'incremental' claim.
Backlash builds over NHS plan to hide source code from AI hacking risk · g2 — Provides the constructive alternative the article lacked: the proportionate response to AI-accelerated vulnerability discovery is architectural hardening (front-of-application defenses, blast-radius isolation, instant rollout), not hiding code or faster patching.

Project Glasswing: what Mythos showed us

Structure

Key terms

Propositions

Arguments

Narratives

Questions addressed

Gaps

Closes gaps elsewhere

Related documents