LLMs Reward Expertise

Sean Goedecke · rss · 2026-09-12 source ↗ Reader 🖥️

Capacity to Verify AI Output Code Was Never the Bottleneck Domain Expertise as AI Force-Multiplier Expert Generalist Prompting as Communication

The single most important variable in getting valuable output from an LLM is the user's pre-existing domain expertise — not prompting technique — because expertise is what lets a person signal sophistication to the model, evaluate and redirect its output, and supply the human judgment the model itself cannot originate, which means the human, not the model, is increasingly the bottleneck.

mixed — primarily theoretical (an explanatory claim about what determines the value a person extracts from an LLM) with a practical undertow: it implicitly counsels investing in domain mastery rather than prompt-technique tricks, and reassures technical readers that expertise remains valuable · argumentative blog essay (republished from the author's own blog) built around a single extended case study — Terence Tao's public ChatGPT conversation about a Jacobian-conjecture counterexample — plus a personal-experience analogy from the author's own programming practice, closing with a postscript that engages Hacker News pushback — Short (~950 words), highly compressed op-ed form; the argument rests on one exceptional case (Tao) generalized to a broad claim about all LLM use, plus one first-person anecdote. Bolded sentences mark the two load-bearing claims (thesis and its restatement).

Structure

Part	Role
Opening: LLMs as generalist-makers	Sets up the naive view the essay will complicate — LLMs let anyone produce sort-of-okay output outside their expertise (e.g. CSS), which seems to imply prompting requires no skill.
Thesis statement	States, in bold, that domain expertise — not prompting skill in the abstract — is the most important variable.
Case study: Terence Tao's ChatGPT conversation	Provides the evidentiary centerpiece: a named, exceptional expert visibly out-performing ordinary users of 'the same' model, with four specific behavioral observations (brevity, register-matching, indirect pushback, self-generated leaps).
Anti-imitation move	Explicitly blocks the reader from concluding that copying Tao's stylistic tips would replicate his results; relocates the causal factor from style to substantive mathematical understanding.
Personal analogy: 'theory of the codebase'	Extends the single exceptional case (world-class mathematician) to the author's own ordinary domain (software engineering), linking to two of the author's own prior essays.
Concession and qualification	Acknowledges that a person without domain knowledge can still get real value ('cling onto the LLM'), and that most people are a mix of expert-in-some-areas / novice-in-others.
Closing claim: the human is the bottleneck	Generalizes from the case studies to a claim about the trajectory of AI capability — as models improve, the constraint shifts to human communication of intent.
Postscript: responding to Hacker News comments	Engages a specific counterexample (OpenAI's own math prompts were reportedly 'inexpert') and explains it away via a hidden mechanism — expert human filtering downstream of the model.

Key terms

expertise / domain knowledge
A structural, internalized understanding of a subject that lets a person evaluate an LLM's output, redirect it, spot what 'looks weird,' and supply the judgment calls the model can't originate — not just factual recall.
skilled prompters
Used ironically/in scare quotes to name a category the author argues doesn't really exist as a distinct thing — people who are good at LLM interaction independent of what they know about the subject.
theory of your codebase
A borrowed concept (linked to the author's own prior essay, in Naur's 'programming as theory building' lineage) meaning the internal mental model of a specific system that lets an engineer judge whether a proposed change fits — not general programming knowledge.
the human is the bottleneck
A reversal of the common 'AI removes bottlenecks' narrative: as models get more capable, the binding constraint on value shifts to the human's ability to communicate exactly what solution is wanted, since the model already 'contains' the answer.

Propositions

explicitThe most important skill in prompting an LLM is expertise in the domain you're prompting about — not prompting technique in the abstract.conf: high
supportsEight years of wanting, three months of building with AI · p4 — Both claims name domain expertise as the (or a) primary determinant of how much value a person gets from an LLM; this doc isolates and elaborates the expertise axis with a detailed mechanism (register-matching, correction, self-generated direction) that the other extract states more tersely.
supportsIs Your AI Training the Competition? · p3 — Both claims locate AI output value in the blend of model capability and the specific human expertise steering it, rather than in the raw prompt/output text itself.
explicitcorroboratedLLMs let non-experts produce passable output in domains they don't understand (e.g. 'sort-of-okay CSS'), which creates the appearance that no skill is involved in working with LLMs at all.conf: high
supports ←Agentic Skill Decay · p3 — Both describe the same effect from different angles — agents producing passable output that masks the underlying skill gap of the person directing them.
explicitSignaling domain expertise (via brevity, register, and specificity) shifts the model into a more concise, expert-register mode of response, rather than an explanatory, amateur-register mode.conf: high
explicitGenuine domain understanding — not the surface stylistic habits of an expert prompter — is what makes expert-level prompting reproducible; copying Tao's message style without his mathematical understanding would not reproduce his results.conf: high
explicitDeep familiarity with a specific codebase ('a good theory of your codebase') lets an engineer push an LLM harder and extract more value than familiarity with general software-engineering principles would.conf: high
explicitSystem design problems are dominated by concrete, local specifics rather than generic principles, so codebase-specific familiarity is worth more (for directing an LLM) than general software-systems knowledge.conf: high
explicitA person with no domain knowledge in an area can still get real value from an LLM by relying on it directly, though this yields far less value than the same LLM used by someone with domain expertise steering it.conf: high
extendsEight years of wanting, three months of building with AI · p4 — Extends the two-axis (expertise + verifiability) framing into a forward-looking structural claim: as models improve, the expertise axis becomes relatively more binding because the information gap closes while the communication/elicitation gap does not.
explicitAs models get stronger, the binding constraint on value for many tasks shifts to the human's ability to communicate exactly what kind of solution is wanted, rather than to the model's raw capability — the needed information is already 'in the model,' and it takes a smart human to extract it.conf: high
explicitdevelopedThe usefulness of domain knowledge for prompting implies that human technical expertise will remain valuable as a structural matter even as models get more capable, not merely as a transitional artifact of current model limitations.conf: high
extends ←Agentic Skill Decay · p6 — Sharpens the general claim that expertise remains structurally valuable into a specific mechanism: expertise functions as the floor (verification capacity) and ceiling (imaginative specification) that together bound what a person can safely ship with an agent.
extends ←How Coders Can Survive—and Thrive—in a ChatGPT World · p2 — Montani's and Solar-Lezama's 2023 claim that structural/problem-solving judgment (not tool knowledge) is what remains distinctively human is an early version of the later, more developed thesis that human technical expertise remains structurally valuable for getting good results from LLMs, not just as a transitional artifact.
explicitThe objection that OpenAI's own math-discovery prompts were 'inexpert' does not refute the expertise thesis, because OpenAI's expert mathematicians checked and filtered the model's suggested discoveries before they counted as real results — expert filtering downstream can substitute for expert prompting upstream.conf: high

Arguments

Domain expertise, not generic prompting technique, is the most important variable in getting valuable output from an LLM.

Explicit premises:

Terence Tao gets dramatically better mathematical results from ChatGPT than the author does, despite (implicitly) using 'the same' model.
Tao's observable prompting behaviors (brevity, register-matching, indirect pushback, self-generated leaps) differ from typical prompting style.
Copying those stylistic behaviors without Tao's mathematical understanding would not reproduce his results — the substance, not the style, is doing the causal work.
The author's own analogous experience (a 'theory of the codebase' in programming) shows the same pattern in a second, unrelated domain.

Implicit premises:

Prompting technique and domain expertise are cleanly separable causal factors, such that a single case study can attribute the outcome gap to one and not the other — rather than, e.g., Tao's fame/authority causing the model to behave differently, or Tao simply being a better communicator independent of mathematical knowledge.
The model's register-shift (concise vs. explanatory mode) is genuinely instrumental to getting a better answer, rather than merely a stylistic artifact that could equally mislead a non-expert who successfully mimics the register into false confidence.
Two adjacent knowledge-work domains with relatively clear correctness criteria (formal mathematics, a specific codebase) are representative enough to generalize the claim to 'prompting' in general, including domains with more contested or subjective correctness criteria.

Inferences:

G
e
n
e
r
a
l
i
z
e
s
f
r
o
m
o
n
e
e
x
t
r
a
o
r
d
i
n
a
r
i
l
y
e
x
c
e
p
t
i
o
n
a
l
p
u
b
l
i
c
c
a
s
e
(
a
r
g
u
a
b
l
y
t
h
e
m
o
s
t
e
m
i
n
e
n
t
l
i
v
i
n
g
m
a
t
h
e
m
a
t
i
c
i
a
n
)
p
l
u
s
o
n
e
f
i
r
s
t
-
p
e
r
s
o
n
a
n
e
c
d
o
t
e
t
o
a
g
e
n
e
r
a
l
c
l
a
i
m
a
b
o
u
t
w
h
a
t
d
e
t
e
r
m
i
n
e
s
L
L
M
o
u
t
p
u
t
v
a
l
u
e
f
o
r
e
v
e
r
y
o
n
e
.

For many tasks, the human, not the model, is now the bottleneck to getting value from AI. strength: medium