Bugs in Large Language Models Generated Code: An Empirical Study

Florian Tambon, Arghavan Moradi Dakhel, Amin Nikanjam, Foutse Khomh, Michel C. Desmarais, Giuliano Antoniol · pdf · 2024-03-13 source ↗ Reader 🖥️

Coding Agents Test-Driven Development

Bugs in LLM-generated code are not simply a noisier version of human bugs but form a distinct, empirically-grounded taxonomy of 10 patterns — and roughly two-thirds of observed bugs (Wrong Attribute, Hallucinated Object, Prompt-biased Code, Non-Prompted Consideration) stem from the model's generative/stochastic nature rather than from prompt underspecification, meaning no amount of better prompting will eliminate them and tooling built on human-bug assumptions (like mutation-based program repair) will systematically underperform on AI-generated code.

theoretical · mixed — Primarily a theoretical/empirical contribution (a validated taxonomy of bug patterns), but closes with explicit practical recommendations for both LLM users and researchers, so the piece argues both 'this is the structure of the phenomenon' and 'here is what to do about it.'

Structure

Part	Role
Motivating example (the 'parser_flags' function that silently adds an unrequested sort) establishing that LLM code can look correct while being subtly, non-obviously wrong.	context / motivation
Methodology: 333 buggy Python samples manually open-coded from 1,997 CoderEval-flagged buggy outputs of three LLMs (CodeGen, PanGu-Coder, Codex), yielding a 10-category taxonomy with 78.2% initial inter-rater agreement.	evidence / method
RQ1 results: the taxonomy itself, each pattern's frequency, and distribution across tasks and dependency ('runnable') levels.	evidence
RQ2 results: a 34-respondent practitioner/researcher survey validating that all 10 patterns are real, frequent, and vary in diagnosing/fixing difficulty; correlation analysis against CoderEval's distribution (strongest for Codex, ρ=0.47).	evidence / validation
Lessons learned for LLM users (prompt engineering, more testing, calibrating generation vs. completion vs. manual coding) and for researchers (repair tooling, benchmarks, interpretability).	practical recommendation
Related work comparison, specifically rebutting Fan et al.'s claim that LLM bugs share human-like repair patterns, attributing the discrepancy to CoderEval's more realistic, GitHub-mined tasks versus LeetCode-style benchmarks.	counter-evidence / positioning

Key terms

Misinterpretation
The generated code deviates from the prompt's intent because the model failed to capture the task's specification — the single most frequent bug pattern (20.77%).
Non-Prompted Consideration (NPC)
Code that adds statements or behavior unrelated to, and not requested by, the task specification (e.g. silently sorting a list no one asked to be sorted).
runnable level
CoderEval's classification of how much external dependency a task requires to execute (self / stdlib (slib) / public-library (plib) runnable) — used here to test whether bug patterns are an artifact of task difficulty rather than genuine LLM behavior.
Hallucinated Object
The model uses an object, function, or variable that neither exists nor has been defined anywhere in the available context — a code-specific instance of the broader 'hallucination' phenomenon in generative AI.

Propositions

explicitcontestedLLM-generated code bugs form a distinct, empirically-derived taxonomy of 10 recurring patterns (Misinterpretation, Syntax Error, Silly Mistake, Prompt-biased Code, Missing Corner Case, Wrong Input Type, Hallucinated Object, Wrong Attribute, Incomplete Generation, Non-Prompted Consideration), built via open coding of 333 manually-labeled buggy Python samples across three LLMs on real-world GitHub tasks (CoderEval).conf: high
contradictsNo Vibes Allowed: Solving Hard Problems in Complex Codebases – Dex Horthy, HumanLayer · p10 — This paper's emphasis on correctness-focused testing to catch specific, named bug categories sits in tension with the claim that code review's real function is 'mental alignment,' not correctness — the taxonomy implies some LLM-specific error categories genuinely need correctness-focused verification that mental-alignment review would not catch.
explicitMisinterpretation (20.77%) and Missing Corner Case (15.27%) are the two most frequent bug patterns across all three models studied.conf: high
explicitAt least four bug patterns — Wrong Attribute, Hallucinated Object, Prompt-biased Code, and Non-Prompted Consideration — are uncommon or absent in taxonomies of human-authored bugs and originate from the model's stochastic generative process rather than from limited or ambiguous prompt information; together with related patterns these account for a non-trivial 64% of the sampled bugs that cannot be attributed to prompt underspecification.conf: high
supportsState Of AI-Assisted Software Development · p4 — A concrete mechanism for why AI adoption increases delivery instability even as it improves throughput: a non-trivial share of AI-generated bugs (the generatively-caused categories) are not addressable by better prompting and will recur regardless of how skilled the prompting team becomes.
explicitA validation survey of 34 LLM practitioners and researchers (8.9% response rate) confirmed all 10 taxonomy categories as real and recognized; Wrong Attribute, Prompt-biased Code, and Missing Corner Case were the most frequently encountered (weighted average >3/5), while Syntax Error and Silly Mistake were the least encountered.conf: high
explicitThe survey-reported frequency of bug patterns correlates moderately with the frequency observed in the CoderEval sample specifically for Codex (Spearman ρ=0.47), weakly for PanGu-Coder (ρ=0.28), and not at all for CodeGen (ρ=-0.18) — the taxonomy generalizes best to the Codex/ChatGPT-class models practitioners actually report using.conf: high
explicitMinor textual noise that a human reader would tolerate or infer past — an extra newline in a prompt, an undocumented expectation about which library function to call — can derail an LLM into a specific, predictable bug pattern (notably Missing Corner Case or Prompt-biased Code), because the model lacks the tacit inference a human applies to ambiguous instructions.conf: high
explicitUsers unfamiliar with the specific task, especially novices, are the most likely to over-trust LLM-generated code that 'looks correct,' which is precisely the failure mode the most common taxonomy categories (Misinterpretation, Missing Corner Case) produce.conf: high
explicitAutomatic Program Repair (APR) tools built on patterns learned from human-authored bugs will systematically underperform on LLM-generated bugs, because a meaningful share of LLM bug patterns (e.g. Wrong Attribute, Hallucinated Object) have no equivalent 'mutant operator' in human-bug taxonomies at all.conf: high
supportsAgentic AI and Code Reviews · p2 — If a meaningful share of LLM bugs require semantic judgment rather than pattern-matching against known human mutant operators, this reinforces the claim that review's value (and now testing's value) lies beyond simple defect-pattern matching — in catching maintainability, intent-fit, and non-obvious category errors.
explicitAn earlier study's conclusion that LLM bugs share repair-pattern similarity with human bugs (Fan et al., tested on LeetCode-style tasks) does not hold on CoderEval's more realistic, GitHub-mined tasks, where non-human-like categories like Wrong Attribute and Hallucinated Object appear even from the more advanced Codex model.conf: high
implicitBecause categories like Hallucinated Object and Wrong Attribute are not primarily caused by prompt underspecification, no amount of prompt engineering alone will eliminate them — some bug patterns are an intrinsic cost of the generative mechanism itself, not a solvable input problem.conf: medium
supportsAgentic AI and Code Reviews · p9 — The taxonomy's split between prompt-caused categories (Misinterpretation, Missing Corner Case — failures of ambiguous intent) and generatively-caused categories (Hallucinated Object, Wrong Attribute) gives empirical grounding to the claim that the decisive human skill is writing precise specifications and judging ambiguity, not mechanically re-checking code.
explicitPractitioners should calibrate how they use an LLM to the task: full generation works best as a 'kickstart' when the user has no clear idea how to solve the task; code completion (providing partial code as context) works better when the user has a specific target in mind, because the added context constrains and clarifies generation; manual coding is preferable when correctly specifying and then verifying/debugging generated output would cost more than writing the code directly.conf: high

Arguments

LLM-generated code bugs are categorically different from human-authored bugs — not just more frequent or sloppier — which undercuts the common framing of LLMs as merely a faster, slightly-less-careful 'pair programmer' producing human-style mistakes. strength: medium

Explicit premises:

Roughly 64% of the sampled bugs (Wrong Attribute, Hallucinated Object, Prompt-biased Code, Non-Prompted Consideration) cannot be attributed to limited prompt information.
These categories are rare or absent in existing human-bug taxonomies and in a prior LLM-bug study based on simpler LeetCode tasks.
An independent practitioner/researcher survey corroborates these categories as real, frequent, and in several cases hard to diagnose and fix.

Implicit premises:

A taxonomy built from realistic, GitHub-mined tasks (CoderEval) surfaces failure modes that simpler, well-specified competitive-programming benchmarks (LeetCode/HumanEval) mask, because the latter's dense, example-rich prompts hide ambiguity- and library-misuse-driven failures that real-world tasks expose.

Inferences:

Tooling and processes designed around human-bug assumptions (mutation-based automatic program repair, code-review habits tuned to catch human-style mistakes) will systematically miss or mishandle a meaningful share of AI-generated bugs.
Since some bug categories are generatively intrinsic rather than prompt-caused, teams cannot 'prompt their way out' of them — they need detection/verification mechanisms that don't depend on better prompting.

Narratives

The 'parser_flags' example: Codex generates a function that looks correct at a glance, but silently adds an unrequested sorted() call on the last line, producing output that fails the actual task.
→ Illustrates both Non-Prompted Consideration as a category and the paper's broader warning that LLM code can look right while being subtly wrong — motivates the 'more testing, less trusting' recommendation.
CodeGen calling Python's built-in min() on a list of class objects (minimalBases), a mistake the paper says no reasonable human developer would make regardless of how complete the prompt was.
→ Supports the central claim that a meaningful share of LLM bugs are not explainable by prompt underspecification — they are errors a competent human simply wouldn't produce.

Questions addressed

✅ What are the characteristics of bugs occurring in code generated by LLMs for real-world programming tasks?
A validated 10-category taxonomy derived from manual open-coding of 333 buggy samples, with frequency, diagnosing-difficulty, and complexity data per category.
✅ To what extent are these bug patterns relevant to practitioners' and researchers' actual experience with LLMs, rather than artifacts of this specific dataset?
Confirmed via a 34-respondent survey; frequency correlates moderately with the CoderEval sample for Codex specifically.
⭕ Will these bug patterns persist as LLMs are embedded in iterative, self-testing agent loops (write code, run tests, see failures, revise) rather than used for single-shot generation from a static prompt?

Gaps

incompleteThe entire study examines single-shot code generation from a static prompt — none of the three LLMs (CodeGen, PanGu-Coder, Codex) operate in an iterative agent loop that runs the generated code, observes failures, and self-corrects. The paper never asks which of its 10 bug patterns would be caught and fixed automatically by such a loop (plausibly Syntax Error, already run through a compiler/interpreter) versus which would persist because they require understanding intent rather than running code (Misinterpretation, Missing Corner Case, Hallucinated Object, Wrong Attribute) — exactly the distinction that matters for today's agentic coding tools.
Read next: extract-agentic-ai-and-code-reviews.json and extract-horthy-no-vibes-allowed-complex-codebases.json both grapple with what review/verification should target once agents run tests themselves; this taxonomy gives empirical substance to which bug categories are mechanical (test-catchable) versus semantic (requiring judgment on intent-fit).
uninformedThe three models studied (CodeGen, PanGu-Coder, Codex) are all pre-2022-era, non-agentic code models; the paper itself flags (Threats to Validity) that the taxonomy's long-term relevance depends on untested assumptions about which patterns persist as models improve. It offers no data on whether 2025-26 frontier coding models or agent harnesses have reduced, eliminated, or introduced new bug categories.
Read next: extract-dora-2025-state-of-ai-assisted-software-development.json reports delivery-instability effects of current-generation AI adoption at scale, which this paper's taxonomy could help explain mechanistically if re-run on current models.
incompleteThe 'knowing how and when to use LLMs' recommendation (generation vs. completion vs. manual coding) is framed entirely as an individual developer's moment-to-moment choice. The paper never addresses team-level process design — code review policy, CI gating, pairing, or testing discipline — for catching these specific bug patterns at the scale of many AI-authored changes landing across a team, which is the organizational question the rest of this corpus's 2025-26 material is working through.
Read next: extract-agentic-ai-and-code-reviews.json's layered, risk-tiered review system is the organizational-scale answer this paper's individual-developer recommendations don't attempt.