AI Companies Aren't Evil. But They Are Reckless.

Julie Guirado (Persuasion / Center for Humane Technology) · email · 2026-05-04 source ↗ Reader 🖥️

AI Regulation & Governance (External)AI-Driven Vulnerability Discovery Mythos

The absence of external AI governance is not a policy gap but a structural failure: a broken incentive system rewards reckless development, and only mandatory independent testing, transparency requirements, and legal liability can realign private incentives with public safety — as has been done with every other consequential technology.

practical · persuasive essay / policy argument — An op-ed making the case for AI regulation. Uses the Anthropic Mythos incident as a concrete hook, then generalizes to a structural argument about incentives and governance. Primarily prescriptive — it diagnoses a problem and proposes a regulatory framework.

Structure

Part	Role
Opening narrative: the Mythos incident	Establishes the concrete case — Anthropic discovered its model could find zero-day vulnerabilities across all major operating systems, handled it unilaterally, and the model was subsequently accessed by unauthorized users
Diagnosis: broken incentive structure, not evil intent	Reframes the problem from individual bad actors to systemic incentives — the 'if I don't build it, someone else will' logic combined with genuine techno-optimism
Catalogue of harms	Lists documented AI harms (job loss, manipulative engagement, cognitive offloading, AI psychosis, AI-assisted suicide and murder) to establish that risks are not hypothetical
Rejection of the 'too complex to govern' objection	Turns the complexity argument on its head — because AI is complex and consequential, governance is more necessary, not less
Regulatory vacuum: federal inaction and state patchwork	Documents the current state — 73 AI laws across 27 states, but federal regulation absent and Trump executive order blocking 'excessive state regulation', while industry lobbies against oversight
Proposed regulatory framework (Mythos as test case)	Prescribes four concrete measures: mandatory independent pre-deployment testing, standardized public risk reporting, whistleblower protections, and legal liability for foreseeable harm
Historical analogy: automobiles, aviation, pharmaceuticals, nuclear energy	Provides precedent that regulation of consequential technologies did not kill innovation

Key terms

broken incentive structure AI Regulation & Governance (External)
The systemic dynamic in which AI companies are rationally compelled to prioritize speed over safety because competitive pressure rewards first-movers and punishes caution
Mythos Mythos
Anthropic's AI model that autonomously discovered zero-day vulnerabilities across every major OS and browser — the article's primary exhibit of ungoverned AI risk
zero-day vulnerabilities AI-Driven Vulnerability Discovery
Security flaws unknown to software makers, which Mythos could autonomously identify — used as the specific mechanism of civilizational risk
duty of care AI Regulation & Governance (External)
Legal and ethical obligation of AI companies to the public, enforced through liability — one of two foundational principles the author proposes for AI governance

Propositions

explicitAI companies are not acting out of malice but are rationally operating within a system that rewards chasing progress now and worrying about consequences later.
explicitThe Mythos incident demonstrates that self-governance by AI companies is insufficient — Anthropic unilaterally decided how to handle a civilizational-scale risk, and the model was subsequently breached.
explicitThe argument that AI is 'too complex, too consequential, or too powerful to govern' is a defense of the broken incentive structure, not a genuine epistemic claim.
explicitRegulation of consequential technologies (automobiles, aviation, pharmaceuticals, nuclear energy) did not kill innovation in those industries.
explicitLegal liability is the mechanism that aligns private incentives with public safety.
explicitcontestedNo meaningful federal regulation of AI currently exists in the United States.
contradictsThe Week AI Grew Up · p5 — Claims no meaningful federal AI regulation exists in the US, in tension with the claim that the US government is already operating an informal licensing regime through compute gatekeeping.
implicitGovernment is the only legitimate manifestation of public will capable of governing AI.
explicitdevelopedThe AI industry actively works to prevent regulation through lobbying and political spending.
refines ←What OpenAI and Anthropic Think Happens Next With AI · p7 — Shifts the picture from industry lobbying to prevent regulation to labs actively co-authoring the regulatory regime that will govern them.

Arguments

AI requires external government regulation

The 'too complex to govern' objection is invalid

Liability is the key regulatory mechanism for AI

Narratives

Questions addressed

✅ Why are AI companies building dangerous systems — are they evil?
Not evil but reckless — they are rationally responding to a broken incentive structure that rewards speed over safety.
✅ Can AI companies govern themselves?
No — the Mythos incident demonstrates that self-governance fails; Anthropic's private containment was breached.
✅ Is AI too complex or too important to regulate?
The opposite — complexity and consequence increase the obligation to govern. Historical precedents prove it is possible.
✅ What would meaningful AI regulation look like?
Independent pre-deployment testing, standardized public risk reporting, whistleblower protections, and legal liability for foreseeable harm.
✅ Do we have historical precedent for regulating transformative technologies without killing innovation?
Yes — automobiles, aviation, pharmaceuticals, and nuclear energy all operate inside hard-won accountability systems.
⭕ How would US-only regulation interact with international AI competition?
⭕ How can regulatory frameworks keep pace with AI's rapid capability growth?

Gaps

International dimension entirely absent
Regulatory capture risk unexamined
closed byWhat OpenAI and Anthropic Think Happens Next With AI — Directly examines the regulatory-capture risk the essay left unexplored: labs co-authoring the regime that will govern them, government equity entanglement, and cross-spectrum critics of lab-led governance.
Speed-of-development vs. speed-of-governance mismatch
closed byThe Trump administration's AI doomer moment — Documents the speed mismatch the essay omits: AI capability advancement outpaced the political cycle, forcing a mid-course governance correction the administration did not anticipate.
No engagement with open-source or decentralized AI development
closed byHow AI Swarms Are Disrupting Democracy — Engages the open-source/decentralized dimension the essay ignores: open-weight models run on local servers outside any jurisdiction structurally escape both regulation and detection, limiting what liability-based governance can reach.
The auto-safety analogy is weaker than presented