Teaching Coding When AI Can Write the Code

Eric Freeman · rss · 2026-07-28 Reader 🖥️

Cognitive Surrender Desirable Difficulty & Productive Struggle Learning Culture & Slack Systemic Apprenticeship & Skill Transmission

Because AI has severed the link between a finished piece of code and evidence of the student's understanding, coding education must shift its center of gravity from grading the finished artifact to making the process of thinking — explained, revised, and performed in public — the thing that is observed and assessed.

practical · mixed — Argues for specific pedagogical action (three concrete assessment practices) grounded in a diagnostic claim (AI breaks code's function as a window into thinking). Expository/argumentative frame carrying narrative case studies (the Molnár chatbot, the algorave) as evidence.

Structure

Part	Role
Diagnosis: code used to be a window into student thinking; generative AI now makes a finished program reveal more about the prompts than the mind behind it, and the better it looks, the less it proves.	problem statement
Rejection of detection as the answer: AI detectors are unreliable (false-flag nonnative speakers, OpenAI retired its own classifier) and, even if perfect, would answer the wrong question.	counter-argument / reframing
Borrowing from studio-art pedagogy: the arts never treated the finished piece as the whole story, and already built critique, rehearsal, and revision into assessment.	analogical grounding
Three concrete practices at AET/UT Austin: (1) make the work public in a studio-style classroom, (2) invert AI's role into a conversational teacher-assessor (the Vera Molnár chatbot), (3) make understanding performative via live coding (algorave).	evidence / proposed solution
Candid limits: results are based on unstudied, uncontrolled observation, not a controlled comparison.	qualification
Generalization: the same artifact-no-longer-proves-process problem applies beyond school to code review, hiring, and performance management.	conclusion / extension

Key terms

window into thinking
The historical property of student code — its errors, structure, and awkward solutions — that let a teacher infer how the student reasoned and where they got stuck.
sketch (in creative coding)
A program treated as temporary, exploratory, and open to change — made, tested, revised, and shared, borrowing the studio-art sense of a preparatory work rather than a final piece.
the process is the proof
The author's summary principle: once a finished artifact (code, memo, PR) can be produced cheaply by AI, only observing how someone thought, chose, revised, and recovered from failure can certify understanding or skill.
algorave
A live, public performance event (short for 'algorithmic rave') in which students write and modify code in real time on a projected screen while an audience watches the editor and the resulting audiovisuals simultaneously.

Propositions

explicitFor as long as programming has been taught, a student's code has functioned as a window into their thinking — errors, structure, and awkward working solutions reveal how they reasoned and where they got stuck.conf: high
extendsWhat we lose when AI writes for us · p2 — Generalizes the 'the medium is the evidence of thinking, and AI severs that link' claim from writing (Franco: writing is thinking) to code (Freeman: code was a window into thinking).
explicitGenerative AI breaks this window: a finished program now reveals more about the student's prompts than their ideas, and the better the AI-assisted code looks, the less it reveals about what the student actually learned.conf: high
extendsCognitive Surrender · p6 — Extends the claim that generated code's strong surface-correctness signal hides comprehension debt from professional software engineering into the classroom: the better AI-assisted student code looks, the less it proves about learning.
explicitAI-detection tools are not a viable primary defense: Stanford researchers found popular detectors falsely flagged 61.22% of TOEFL essays by nonnative English speakers as AI-generated, and OpenAI retired its own classifier in 2023 as insufficiently accurate.conf: high
explicitEven a hypothetically perfect AI detector would not solve the real problem, because the operative question is not 'how do we stop students from using AI' but 'how do we teach coding in a world with AI while still being able to see if students are learning.'conf: high
explicitStudio-based arts pedagogy (sketches, critique, rehearsal, revision, performance) already solves the problem of a finished piece not revealing process, because in the arts the final piece was never assumed to be the whole story.conf: high
explicitRunning the classroom as a studio — students showing false starts, revisions, and choices, and being required to explain their own code aloud — prevents AI from being used as a hiding place, since a student who needs the AI to explain their own submission has failed the requirement regardless of how the code looks.conf: high
explicitRequiring students to talk through their work in public also addresses a separate, non-AI problem — a perceived rise in classroom silence and reluctance to speak since 2020 — because both problems share the same remedy: mandatory, structured verbal participation.conf: medium
explicitInverting the usual AI relationship — having an AI conversational agent question and assess the student, rather than answer the student's questions — produces a transcript that is a different, harder-to-fake kind of proof of understanding than a submitted artifact.conf: high
explicitThis inverted-role assessment design has research precedent predating ChatGPT: teachable-agent systems like Betty's Brain showed that explaining material — even to a software agent — forces students to organize their knowledge, make connections explicit, and expose gaps.conf: high
explicitLive, on-stage coding performance (the algorave format) makes real-time faking structurally impossible, because a student cannot paste their way through public debugging while the projected visuals fail in front of an audience — what is displayed is fluency, not memorized or copied code.conf: high
supportsDet skal være vanskelig · p15 — Independently arrives at the same 'process, not product, is the locus of learning' claim and supplies three concrete coding-education implementations of it.
extendsDet skal være vanskelig · p13 — Concretizes the 'visible-thinking assessment is a system responsibility' prescription into three specific mechanisms: public studio work, AI-as-assessor conversation, and live performance.
explicitAcross all three practices, the underlying shift is the same: as finished work becomes trivially easy to generate, assessment must move from evaluating the artifact to evaluating the process — explanation, revision, choice, and recovery from failure.conf: high
supportsResearch: Why Some Junior Employees Work Well with AI—and Others Don’t · p7 — Corroborates, in an educational setting, the workplace finding that once AI can independently produce competent output, evaluation must shift from grading the deliverable to observing the process of interaction with AI.
explicitThis shift is not confined to education: a polished memo no longer proves real thinking behind it, a working prototype no longer proves product sense, and a passing pull request no longer proves the developer changed the code carefully — so evaluation in code review, hiring, and performance management must also move toward observing how people think, choose, revise, and recover.conf: high
explicitThe evidence for all three practices is the author's own unstudied classroom observation — no controlled comparison group, no measured learning-outcome data — so the claimed effectiveness is early and anecdotal, not established.conf: high
implicitThe three practices work especially well because creative coding already has a public, visual, and performative culture (studios, critique, live-coding tradition) — an implicit acknowledgment that they may not transfer cleanly to standard, non-performative CS instruction (e.g., large lecture-hall CS1 courses without an arts-department studio culture).conf: medium
implicitThe author trusts AI as a reliable evaluator of student understanding (grading the Molnár conversation and assisting the OOP-project rubric) even while arguing, earlier in the same piece, that AI's own text-classification judgments (detectors) cannot be trusted — an unexamined asymmetry in how much epistemic authority AI is granted depending on which side of the assessment it sits on.conf: medium

Arguments

Institutions should not make AI-authorship detection the primary defense against AI undermining coding assessment. strength: high

Explicit premises:

Popular AI detectors have high false-positive rates against nonnative English writers (61.22% of TOEFL essays in one study).
OpenAI itself retired its AI Text Classifier in 2023 for inaccuracy.

Implicit premises:

A detector's job is only to flag that AI was used, not to establish whether the student learned anything — so even a perfectly accurate detector would leave the actual pedagogical question (did the student learn?) unanswered.
Basing high-stakes honor-code decisions on an unreliable and ultimately beside-the-point signal produces false accusations without producing better learning outcomes.

Inferences:

Schools investing primarily in surveillance/lockdown/detection infrastructure are misallocating effort relative to the actual assessment problem.

Assessment of coding ability should shift from the finished program to the visible process (explanation, revision, live performance) that produced it. strength: medium

Explicit premises:

In the arts, the finished piece was never treated as the whole story — process has always been assessed there.
A finished AI-assisted program now reveals the student's prompts more than their reasoning.

Implicit premises:

A coding assignment's purpose was always to develop and demonstrate the student's own understanding, not to produce a working program as an end in itself — i.e., the artifact was always instrumental, not terminal, to the pedagogical goal.
Process signals (public explanation, live on-stage debugging, sustained conversational interrogation) are harder to outsource wholesale to AI than a single static file is.

Inferences:

The same reasoning extends beyond the classroom to any domain where AI can now cheaply produce a polished final artifact — code review, hiring, performance management — which the author draws explicitly in the closing section.

An AI conversational agent that questions and assesses a student (rather than answering the student's questions) can produce trustworthy, previously-unavailable evidence of understanding. strength: low

Explicit premises:

Prior teachable-agent research (Betty's Brain) shows that explaining material to a software agent forces students to organize knowledge and exposes gaps.
In small trials, the AI assessor's rubric-based evaluations of the Molnár conversations closely matched independent teaching-assistant evaluations.

Implicit premises:

An AI system can reliably distinguish genuine understanding from fluent-sounding mimicry when following a rubric — the same kind of trust in an AI's judgment that the article's earlier detector argument explicitly withheld from AI when the question was 'did a human write this.'
The bias risk raised against detectors (systematically penalizing nonnative or atypical communication styles) does not apply, or applies less, to an AI grading spoken/conversational explanations — a claim the article does not examine.

Inferences:

If the same fairness scrutiny applied to detectors were applied to the AI-as-assessor role, it might surface analogous bias risks (e.g., against less fluent or nonnative speakers in a conversational assessment) that the article does not consider.

Narratives

The Vera Molnár chatbot: a simple prompt-engineered avatar of algorithmic-art pioneer Vera Molnár converses with students about randomness, computation, and creative choice, follows up on unclear answers, and reviews the transcript against a rubric to assess understanding.
→ Concretely demonstrates the 'invert the roles: AI as teacher and assessor' practice and supports the claim that a conversational transcript can reveal understanding a finished submission cannot.
The AudioPixel Collider algorave: students from different disciplines stage a live, public performance where code is written and modified in real time on a projected editor while audio/visual output responds live, with the explicit risk of the whole projection failing ('turning into a beige rectangle') if the coder falters.
→ Shows 'make understanding performative' in action and grounds the proposition that live coding makes real-time faking impossible ('it is very hard to plagiarize panic').
The end-of-term object-oriented game assignment, where teaching assistants with limited OOP experience used an AI-generated rubric-based analysis of each submission to structure their own grading, with the author available as an OOP expert for unresolved questions.
→ Illustrates AI used to scale substantive process-aware feedback despite student-to-TA ratio constraints, but also implicitly shows TAs leaning on AI's analysis to compensate for their own domain gaps — a case the article treats as a success story without examining the risk that TAs might defer to a plausible-looking AI analysis they cannot independently verify.

Questions addressed

⭕ If AI can write code, should schools still teach coding at all?
The author states his own answer is yes, 'at least for some students and situations,' but explicitly defers the justification: 'that's another topic.'
✅ If coding is still taught in a world where AI can write code, how can teachers know whether students are actually learning?
Answered with three practices — public studio-style work, AI-as-assessor conversation, and live performative coding — that shift assessment from the finished artifact to the visible process.
⭕ Do these three practices actually improve or reliably measure learning, compared to conventional assessment?
Explicitly left open by the author: 'we haven't conducted a controlled study... these efforts are experiments, not final answers.'
⭕ Do these creative-coding-specific, small-class, performance-culture practices generalize to standard computer-science instruction that lacks a studio/arts tradition and operates at much larger scale?
Not resolved; the author notes the practices 'work especially well in creative coding' but does not address transfer to conventional CS1-style courses.

Gaps

incompleteThe three practices are validated only by the author's own uncontrolled classroom observation — no comparison group, no measured learning-outcome data, and no discussion of cost/scale requirements (small classes, an arts-department studio culture, custom-built chatbot avatars, TA availability for public critique) that make the approach hard to generalize to large, non-arts-adjacent CS programs.
Read next: Look for controlled or larger-N studies of process-based/performative assessment in programming education, and for follow-on evaluations of teachable-agent-as-assessor systems (extending Betty's Brain-style research to the assessor role rather than the tutee role).
illogicalThe article distrusts AI's reliability as a detector of whether a human wrote a piece of text (citing bias against nonnative English speakers and OpenAI's own retired classifier) but does not extend the same scrutiny to AI's reliability as an assessor of student understanding — despite grading spoken/conversational explanation being exactly the kind of task where fluency-related bias (the same failure mode documented for detectors) could plausibly recur.
Read next: Research on fairness and bias in AI-as-grader / conversational-assessment systems, particularly regarding nonnative-speaker or atypical-communication-style students, given the article's own evidence that AI text-judgment tools already show this exact bias in a different context.
incompleteThe article's proposed replacement signal for 'code as reliable evidence of thinking' is the process trace itself (public explanation, live performance, AI-mediated conversation) — but it does not establish how durable this new signal is against its own future gaming: a rehearsed AI-coached explanation, a memorized live-coding routine, or answers effectively drafted by AI and read aloud in the chatbot conversation. It swaps one hard-to-fake signal for one whose own gaming-resistance is asserted, not demonstrated, and the reading direction implied by the source brief (what replaces code once code stops proving understanding) is left as an open empirical question rather than a settled design.
Read next: History and literature on the gaming and eventual erosion of other 'process' evidence forms (oral exams, viva voce, live demonstrations, in-person coding interviews) — do process-based signals have their own track record of being coached, memorized, or otherwise gamed over time?

Closes gaps elsewhere

Det skal være vanskelig · g1 — Offers a concrete, structural (not self-discipline-dependent) answer to how a 'simulator' can be made gaming-resistant at the point of use: public studio critique makes AI-only work visible to the room, an AI-as-assessor conversational agent follows up until it gets independent evidence of understanding rather than accepting a first answer, and live on-stage coding performance makes real-time paste-and-pray structurally impossible. This is a partial closure only — Freeman's own evidence is uncontrolled, and he concedes the practices work 'especially well' in creative coding's public/performative culture, not shown to generalize to standard, large-scale CS instruction, which is exactly the 'at scale' qualifier the original gap raised.

Teaching Coding When AI Can Write the Code

Structure

Key terms

Propositions

Arguments

Narratives

Questions addressed

Gaps

Closes gaps elsewhere

Related documents