Study finds AI assistants help developers produce code that's more likely to be buggy
Thomas Claburn · article · 2022-12-13 Reader 🖥️
A controlled Stanford user study found that developers given access to an AI coding assistant (OpenAI's codex-davinci-002, as used in GitHub Copilot) produced measurably less secure code than a control group working without it — and, more strikingly, were more likely to believe their own AI-assisted code was secure than the control group was about theirs — indicating AI assistance can degrade both code security and developers' self-assessment of that security simultaneously.
theoretical · expository — A Register news report summarizing a Stanford user study (Perry, Srivastava, Kumar, Boneh, 'Do Users Write More Insecure Code with AI Assistants?,' arXiv:2211.03622) and situating it against two prior NYU studies on GitHub Copilot's code-security properties. The article itself makes no independent argument; it reports and lightly contextualizes the paper's findings. Published 2022-12-13, so 'previously' and prior studies cited (Aug 2021, Oct 2022) are anchored to that date, not today's.
Structure
| Part | Role |
|---|---|
| Headline finding stated: AI-assisted participants produced more security vulnerabilities and were more likely to believe their code was secure | conclusion (stated upfront) |
| Positioning against prior work: an Aug 2021 NYU-adjacent study ('Asleep at the Keyboard?') found ~40% of Copilot-aided solutions across 89 scenarios had exploitable vulnerabilities, but was limited to 25 vulnerability types and three languages (Python, C, Verilog) | context |
| Positioning against a second prior study ('Security Implications of Large Language Model Code Assistants'): used a weaker Codex model, focused only on C, and reported inconclusive results — which the Stanford authors attribute to C being the one language in their own broader study with mixed results | context |
| Study design: 47 participants (undergrads, grad students, industry professionals) wrote code for 5 prompts in a monitored Electron app, with/without AI assistance | evidence |
| Prompt-by-prompt results: AI-assisted group had a lower correct-answer rate (67% vs 79%) and significantly more insecure solutions and trivial-cipher use on the encryption task; similar pattern on prompts 2-4; mixed/inconclusive but still negative signal (more integer-overflow mistakes) on the C-language prompt 5 | evidence |
| Authors' stated conclusion: AI assistants should be used cautiously because they can mislead inexperienced developers and introduce vulnerabilities, while also potentially improving productivity, lowering entry barriers, and making development more accessible than hostile forums like early Stack Overflow | conclusion |
Key terms
- more likely to believe they wrote secure code
Participants self-reported higher confidence in the security of their AI-assisted solutions than the control group reported for their unassisted solutions, even though the AI-assisted solutions were objectively less secure — a confidence/competence mismatch specifically induced by AI assistance.
Propositions
- explicitcontestedProgrammers given access to an AI coding assistant produced less secure code overall than a control group working without one, with particularly significant effects on string-encryption and SQL-injection tasks.conf: highcontradictsWe Studied 150 Developers Using AI (Here's What's Actually Changed...) · p3 — This 2022 study found AI assistance produced measurably less secure/buggier code; the later, larger, pre-registered Farley-reported study found no significant difference in objective code quality or maintenance cost between AI-assisted and non-AI-assisted code — a direct empirical tension worth reading together rather than assuming either superseded the other.
- explicitParticipants who used the AI assistant were more likely to believe they had written secure code than participants in the control group, even though the AI-assisted code was in fact less secure — i.e. AI assistance both degraded output security and inflated developers' confidence in that output.conf: highsupportsTo Adopt AI at Scale, Employees Need to Trust Agents · p13 — This study's 'AI assistance makes developers more confident in objectively worse code' is a concrete, early (2022) empirical instance of the 'trust excess' half of the later essay's general trust-miscalibration framework.
- explicitOn the string-encryption task specifically, AI-assisted participants were both less likely to produce a correct answer (67% vs. 79% for the control group) and significantly more likely to produce an insecure solution, to use a trivial cipher such as substitution, and to omit an authenticity check.conf: high
- explicitA prior study's inconclusive finding about AI-assisted code security is attributable to its narrow single-language (C) focus, since C was the only language in the Stanford study's broader, multi-language design to show mixed rather than clearly negative results.conf: medium
- explicitAI coding assistants carry genuine productivity, accessibility, and barrier-lowering benefits (including a friendlier alternative to hostile programming forums) even as they pose a security risk, so the appropriate response is caution and design improvement rather than rejection.conf: medium
Arguments
AI coding assistants should be used cautiously, because they can simultaneously degrade code security and inflate developers' confidence in that code's security. strength: medium
Explicit premises:
- A controlled experiment (47 participants, 5 prompts, monitored environment) found AI-assisted participants produced measurably less secure code on most tasks.
- The same participants were more likely than the control group to believe their code was secure.
Implicit premises:
- Self-reported confidence about code security, measured immediately after writing it, is a reasonable proxy for the overconfidence risk that matters in real-world deployment (where a developer's belief about their own code's security affects whether they seek further review).
- A 47-participant lab study with short, templated prompts (symmetric-string-encryption, SQL-related tasks, a C integer-to-string function) generalizes to the security properties of AI-assisted code in real, larger-scale production codebases.
- The specific model tested (codex-davinci-002, an earlier, less capable model than later-generation coding assistants) is informative about today's AI coding assistants' security properties, despite substantial capability differences since 2022.
Inferences:
- If overconfidence from AI assistance generalizes beyond this lab setting, it implies review processes cannot rely on developer self-assessment as a filter for AI-assisted code and need an independent check — exactly the kind of structural review-gate argument made in later AI-era essays about validation overhead.
Questions addressed
- ✅ Does AI coding assistance make developers write less secure code, and if so, do developers notice?
Yes to both halves, and in the worse possible combination: less secure code, paired with higher (not lower) confidence that the code is secure. - ⭕ Does this 2022-era finding (using codex-davinci-002, an early Copilot-adjacent model) still hold for later-generation coding assistants and agentic coding tools?
Not addressed by the article or, at the time of publication, by any available follow-up study — the article's own comparisons are limited to two other studies from the same 2021-2022 window.
Gaps
- incompleteNeither the paper as reported nor the article examines whether the overconfidence effect is specific to older, less capable assistants (which may produce plausible-looking but subtly wrong code more often) or would persist, worsen, or disappear with later-generation coding agents that have explicit self-critique, test-running, or verification steps built into their workflow.
Read next: extract-farley-studied-150-developers-using-ai (a later, larger, pre-registered study) found no significant difference in downstream code quality or maintenance cost between AI-assisted and non-AI-assisted code, suggesting the picture may have shifted with later tools and more disciplined usage patterns — worth reading the two findings against each other rather than assuming this early result still holds. - incompleteThe study measures self-reported confidence immediately after writing code in a lab setting; it does not examine whether this overconfidence propagates into real-world consequences — e.g. whether overconfident developers skip code review, merge faster, or escalate less — which is the actual mechanism by which miscalibrated trust would cause harm in production teams.
Read next: extract-to-adopt-ai-at-scale-employees-need-to-trust-agents takes up trust miscalibration (both deficit and excess) as a general organizational-adoption problem and would be the natural next read for the downstream consequences this study doesn't measure.