Presentation: Getting Rid of LeetCode Interviews in the World of AI
Daniel Doubrovkine · rss · 2026-07-29 source ↗ Reader 🖥️
Because algorithmic whiteboard interviews measure luck and recent rehearsal rather than genuine engineering ability — as the speaker's own repeatedly inconsistent interview record proves — companies should replace them with open-ended, AI-assisted system-design exercises that reveal the judgment and human skills AI cannot substitute.
practical · mixed — A first-person conference talk: roughly 80% autobiographical anecdote (career history across Microsoft, a New York bank, a startup, Artsy, AWS, and Shopify) deployed as evidence, culminating in an explicit practical prescription for redesigning technical interviews for the AI era, followed by a Q&A.
Structure
| Part | Role |
|---|---|
| Opening hook: the speaker, a veteran principal engineer, recently failed a 'trivial' diameter-of-binary-tree interview question. | motivating problem / credibility-undermining anecdote |
| Extended career-history narrative (Microsoft whiteboard interviews, a NY bank's isolation-room interview, a startup where he administered deliberately torturous interviews), including the LRU-cache episodes at AWS and Shopify. | evidence that interview outcomes are contingent on luck/recent exposure rather than stable signal |
| Case study of the interview process he designed at Artsy: discussion of prior work and open-source contributions, references from past collaborators, systems-design conversation, and a values-fit round — deliberately excluding algorithmic puzzles. | counter-example / positive alternative model |
| Diagnosis of why LeetCode-style interviews persist: companies claim they want a 'consistent, fair, objective' process, and the speaker rebuts that this framing mistakes puzzle-solving ability for job-relevant signal. | rebuttal of the status-quo justification |
| Prescription: redefine the interview loop for the age of AI — have candidates build/design systems live while prompting AI, and judge their system-design thinking, iteration, and ability to get unstuck; frame this as necessary for any company that claims to want to 'be an AI company.' | conclusion / call to action |
| Q&A: clarifies the role of take-home projects for candidates without a track record, and offers informal red/green-flag heuristics for keeping AI-assisted evaluation consistent across interviewers. | qualification / partial resolution of an open implementation question |
Key terms
- LeetCode interview
Shorthand for the entire genre of closed-form algorithmic coding puzzles (e.g., implement the diameter of a binary tree, an LRU cache) administered under artificial, timed, whiteboard-style conditions — used even when the literal LeetCode website isn't involved. - signal
Reliable information a hiring process yields about whether a candidate can actually perform the job, as distinct from information about how good they are at a specific isolated puzzle or how lucky their recent preparation happened to be. - AI slop AI Slop
Self-deprecating, ambivalent term for code he personally produced by directing an AI assistant — used here without the usual pejorative charge, since he argues his AI-assisted GitHub output is functionally comparable in volume/quality to his earlier hand-written work. - T-shaped people
Candidates with a broad range of interests alongside deep expertise in a narrower specialty — the hiring profile Artsy deliberately selected for instead of narrow algorithm specialists.
Propositions
- explicitWhiteboard/LeetCode-style algorithmic interviews do not reliably signal whether an experienced engineer can actually do the job.conf: highsupportsThe Last Technical Interview · p2 — Independent personal-experience evidence for interviewer judgments being poorly correlated with actual on-the-job ability — his own repeated interview failures despite deep, directly relevant expertise mirror the interviewer-disagreement/poor-correlation finding.
- explicitPerformance on a given whiteboard coding puzzle is driven largely by chance recent exposure to that specific problem type, not by a candidate's general programming ability.conf: high
- explicitThe common justification that whiteboard coding interviews provide a 'consistent, fair, objective' evaluation is not supported by what the process actually measures.conf: high
- explicitDiscussing a candidate's actual prior work (open-source contributions, a build prototype, take-home projects) is a more valid hiring signal than synthetic algorithmic puzzles.conf: highsupportsThe Last Technical Interview · p3 — The Artsy hiring model (discuss actual prior work, take-homes, references) is a concrete instance of assessment quality scaling with the realism of observed work, landing between 'work samples/credentials' and 'on-site interviews' on that ladder.
- explicitReferences from a candidate's past collaborators — effectively an NPS-style measure of how many people who worked with them would recommend them — are an underused but strong predictor of hiring success.conf: medium
- explicitAs AI increasingly performs the mechanical work of writing code, interviews should shift to evaluating a candidate's system-design thinking and their ability to direct, correct, and get unstuck when collaborating with AI, rather than testing memorized algorithms.conf: high
- explicitAs AI absorbs more of the coding itself, distinctly human capacities — empathy, ethics, collaboration, judgment — increasingly become the main differentiator of an engineer's productivity, more than the code they personally write.conf: low
- implicitCompanies that claim they want to 'be an AI company' but do not adapt their hiring process to assess AI-collaboration skills are acting inconsistently with that stated ambition.conf: medium
- implicitWhiteboard-style coding interviews persist largely due to institutional inertia inherited from an earlier technological era rather than because they were ever validated as producing accurate hiring signal.conf: medium
Arguments
Companies should stop relying on LeetCode-style algorithmic whiteboard interviews to assess senior engineering candidates.
Explicit premises:
- The speaker himself, despite a 25+ year career substantially built on implementing tree structures, repeatedly failed and passed similar puzzles seemingly at random.
- At AWS, more than half of principal-engineer candidates bombed their coding interviews and were hired anyway, meaning interviewers themselves routinely override the interview's own failure signal.
Implicit premises:
- A hiring signal is only valuable if it reliably predicts on-the-job performance; a process that yields inconsistent, luck-dependent results for the same person across trials does not meet that bar.
- If experienced hiring managers already know internally that failed technical-interview results should often be overridden, the interview's stated purpose (screening for ability) is not actually what it is being used for.
Inferences:
- If the same engineer's outcome flips between 'fail' and 'ace' depending on which specific puzzle is asked and what they happened to review beforehand, the interview is measuring recent rehearsal rather than underlying competence — undermining the claim that it is 'objective.'
Interview processes should be redesigned so candidates build and design systems live with AI, revealing system-design judgment and AI-collaboration skill instead of memorized algorithms.
Explicit premises:
- AI can already perform most of the mechanical coding that whiteboard puzzles test.
- A 45-minute system-design interview previously couldn't reach implementation because writing the code took too long; AI removes that time constraint, so design and implementation can now be assessed together.
Implicit premises:
- The skills needed to direct and correct an AI collaborator (judgment, iteration, breaking down ambiguity) are the same skills that matter on the job, so testing them in an AI-assisted interview format transfers validly to predicting performance.
- Candidates who already use AI fluently in daily work will not be meaningfully disadvantaged by having to demonstrate that skill under interview conditions.
Inferences:
- If evaluators penalize candidates for merely reading and retyping whatever the AI outputs (per the speaker's Q&A answer) rather than working through an abstraction layer, the graded behavior is really 'good AI-orchestration practice' — confirming that the object of assessment has shifted from code production to direction and verification of an agent.
Narratives
- Despite a 25+ year career substantially built on implementing tree structures (a commercial C++ search engine, a Microsoft memory-allocation library, cross-platform base-class libraries), he froze and failed a diameter-of-binary-tree question his own interviewer called a trivial warm-up.
→ Shows that failing a puzzle-style interview carries no information about actual competence, even for a domain expert in the exact tested subject matter — supports p1. - He happened to review an LRU cache on a plane the night before an AWS interview and passed when asked to implement one; later, at Shopify, he was asked to implement an LRU cache again and 'aced' it.
→ Shows interview success is driven by contingent, recent exposure rather than stable underlying skill — the narrative form of p2. - As an interviewer, he watched a VP-of-engineering candidate claim to 'write code all day, every day,' then refuse to write a sort function on the whiteboard and walk out when challenged to prove it.
→ Shows the interview ritual can become theater/power-play, and that self-reported claims about coding fluency go untested by the very process meant to test them. - At an earlier startup, he devised and administered deliberately torturous interview techniques (an insultingly trivial warm-up for experts, isolating candidates alone in a room, demanding a second solution method, leaving mid-problem for a smoke) — and the company's software was fine, but the business failed anyway.
→ Implicitly argues that the toughness/rigor of the interview process was irrelevant to the company's actual outcome, undercutting the idea that harsh coding gatekeeping protects business success. - At Artsy, an engineer with an MFA in painting, hired after soldering art-tech hardware for the first time (blowing the circuit, then fixing it by trial and error), went on to become a strong, long-tenured hire who now runs a large consultancy.
→ Supports the work-sample/reference-based 'T-shaped' hiring model — a candidate who likely would not have passed a LeetCode filter became a durable, high-value hire.
Questions addressed
- ✅ Does failing a LeetCode-style coding interview mean a candidate can't do the job?
No — demonstrated via the speaker's own repeated failures despite deep, directly relevant expertise, and via the AWS pattern of principal engineers bombing coding rounds yet being hired anyway. - ✅ What should companies test for once AI can write the code itself?
System-design ability and AI-collaboration judgment — whether the candidate can direct AI, evaluate its output, and get unstuck when it fails — assessed through open-ended, extended build exercises (autoscalers, rate limiters, data pipelines, an AI chat system) rather than closed algorithmic puzzles. - ⭕ How do you keep evaluation consistent across many interviewers once the interview format becomes open-ended and AI-assisted?
Only partially addressed: offers informal red/green-flag heuristics (e.g., writing a test before checking whether the AI's code passes it is a green flag; blindly reading and retyping AI output without keeping an abstraction layer is a red flag) rather than a validated, calibrated rubric. - ⭕ Will the industry actually abandon LeetCode-style interviews soon?
He explicitly doubts a widely-circulated prediction that these interviews would vanish by 2026, framing AI-native interviewing as a slow, contested transition led so far by only a few companies (e.g. Shopify).
Gaps
- incompleteThe case for replacing algorithmic interviews with AI-assisted system-design interviews rests entirely on personal anecdote and a joking, admittedly fabricated 'survey' — no comparative data is offered on whether AI-collaboration-based interviews actually predict on-the-job performance better than the interviews they would replace.
Read next: Look for controlled or longitudinal studies comparing hiring-outcome validity between AI-assisted/system-design interview formats and traditional algorithmic interviews. - incompleteThe proposed AI-assisted interview format (watch the candidate prompt an AI and evaluate their judgment) is not examined for gameability — e.g., a remote candidate secretly using a stronger AI assistant, a human collaborator, or pre-rehearsed prompts off-screen — despite the speaker noting elsewhere that he himself could have used a hidden AI to solve the original binary-tree question undetected.
Read next: Proctoring and anti-cheating practices for remote technical interviews in the AI era. - incompleteThe talk's alternative model (discuss prior work, take-homes, references, AI-assisted system design) is implicitly built around candidates who already have a track record — open-source contributions, past jobs, people who can vouch for them. It briefly touches, in Q&A, on candidates fresh out of college, but doesn't seriously address how a portfolio/reference-based hiring model evaluates candidates with no employment or open-source history.
Read next: Hiring-signal design specifically for early-career or no-track-record candidates. - uninformedThe claim that human skills (empathy, ethics, collaboration, judgment) now matter more than code output is attributed secondhand to a single executive recruiter's remark in conversation, not examined against counter-evidence or defined more precisely — it is asserted as received wisdom rather than argued.
Read next: Primary research on which specific human capacities remain predictive of engineering performance as AI absorbs code production.
Closes gaps elsewhere
- The Last Technical Interview · g2 — Argues explicitly for the stronger, more specific thesis that gap flags as missing: that AI changes *what* needs to be assessed in an interview — shifting evaluation from from-scratch algorithmic coding to system-design judgment and the ability to direct/correct an AI collaborator — rather than treating AI merely as an accelerant of pre-existing interview noise and fraud.
Related documents
- The Last Technical Interview
- The best response to AI slop, infinite advice, and online noise is from Robin Williams
- Podcast: Formal Methods for Every Engineer in an AI-Powered Future
- How to turn your AI into a world-class designer
- Adam Mosseri: AI is a tailwind for authenticity
- Notes from FOSE Europe
- We've Stopped Arguing About Whether AI Changes Software
- SWLW #701: Gazing at the misty AI Frontier, From Vibe Coding to Agentic Engineering, and more.
- SWLW #710: Revised rules of Eng Leadership, When AI costs more than the engineer, and more.
- You can't unit test for taste