Design AI Systems That Actually Strengthen Human Reasoning
Melchior Tamisier-Fayard, Theodoros Evgeniou, Anne-Laure Fayard · article · 2026-07-20 source ↗ Reader 🖥️
AI-as-Coach Effect (Learning-Environment Quality)Cognitive LoadCognitive SurrenderEnterprise AI Adoption & ReadinessAgency
Because AI use erodes critical thinking by decontextualizing expertise and homogenizing ideas, organizations must stop treating AI as a chatbot tool to be prompted better and instead deliberately design AI-enabled work systems — Socratic AI interactions, protected AI-free space, and non-chatbot interfaces — that preserve human agency and epistemic pluralism.
practical · mixed — Practical/prescriptive HBR piece (three concrete design strategies for managers), but it opens with a theoretical/expository diagnostic section synthesizing management, cognitive-science and HCI research, then argues its recommendations through named organizational narratives rather than pure argument.
Structure
| Part | Role |
|---|---|
| Framing: two organizational risks (loss of expertise, loss of epistemic pluralism) and the claim that technology is not neutral | thesis statement |
| 'How AI Shapes Thinking' — black boxing precedent, cognitive offloading/surrender (MIT Media Lab 2025, Wharton 2026), decontextualization of expertise (Bar-exam example), homogeneous thinking (LLM averaging/fixation) | evidence / diagnosis |
| 'How Humans Can Shape AI' — reframing AI as a shapeable system, not just a chatbot to prompt better | pivot from diagnosis to prescription |
| 'Turn AI on its head' (Socratic AI) — AstraZeneca Prompt with Me Challenge, BUILD2GETHER hackathon | prescription 1, with narrative evidence |
| 'Understand the right place and time for using AI' — AI-free space (Intel email analogy), centering human agency (Callen Anthony's banker study), processing in parallel (With Company) | prescription 2, with narrative evidence |
| 'Try different interfaces' — critique of chatbot UX, Google's AI Co-mathematician | prescription 3, with narrative evidence |
| Closing exhortation: go beyond the hype, engage in strategic experimentation, AI should empower not replace people | conclusion / call to action |
Key terms
- black boxing
Workers trusting an analytic tool's output without understanding how it works, documented by Callen Anthony (2021) among investment bankers as a precursor mechanism to expertise loss — the pre-genAI ancestor of what the authors later call cognitive surrender. - cognitive offloading / cognitive surrender
The compromised ability to question information one receives from AI — used somewhat interchangeably in the piece, with 'surrender' as the more severe endpoint where independent verification stops entirely. - decontextualization of expertise
AI can access facts and data but cannot contextually interpret and apply knowledge the way people with intuition, emotion and lived, multi-domain experience can — so professional reliance on AI outputs without a human validator risks context-blind errors. - epistemic pluralism
The organizational presence of diverse views, experiences and perspectives, which the authors argue is undermined by AI's tendency toward homogeneous, 'averaged' idea generation. - technology as a system to be shaped (vs. tool to be prompted)
A framing borrowed from the work-and-technology literature: AI is not a neutral chatbot interrogated for answers, but a contextual, interpreted part of a work environment that humans can and should actively redesign. - gated interfaces
AI interfaces that 'unlock assistance only after the user deposits their own context' — attributed to Chengwei Liu and colleagues — a deliberate-friction design pattern cited as one route to preserving human agency. - processing in parallel
Having AI and a human team perform the same task independently and simultaneously, then combining the two outputs into a hybrid result, rather than a sequential AI-then-human or human-then-AI workflow.
Propositions
- explicitThe two main organizational risks from AI-driven erosion of critical thinking are the loss of expertise and the loss of epistemic pluralism.conf: high
- explicitTechnology, including AI, is not neutral and can shape ways of thinking and interacting in negative ways if organizations take its features for granted.conf: high
- explicit"Black boxing" — trusting analytic tools without understanding them — predates generative AI and was already documented (Callen Anthony's 2021 investment-banker study) as a mechanism by which professionals lose expertise.conf: high
- explicitPeople using AI are disinclined to question and research AI output, per a 2025 MIT Media Lab study and a 2026 Wharton study on introducing AI as a 'third cognitive system.'conf: highsupportsCognitive Surrender · p1 — Both describe AI users' disinclination to independently verify AI output; this article corroborates the mechanism with two additional named 2025/2026 studies (MIT Media Lab, Wharton).
- explicitAI can readily access facts and data but cannot contextually interpret and apply knowledge the way people with intuition, emotion, and lived, multi-domain experience can.conf: highsupportsTBM 432: Bundling & Unbundling Capabilities (and AI) · p7 — Both articulate a ceiling on how much of contextual/tacit expertise AI can diffuse or substitute for, framed via Polanyi-style tacit knowledge in tbm-432 and via lived/embodied experience here.
- explicitPassing a professional qualifying exam (e.g., the Bar) does not indicate an AI understands the contextual nuances of the profession it operates in.conf: high
- implicitDecontextualized reliance on AI expertise increases the risk that professionals (e.g., lawyers) misinterpret evidence and make mistakes.conf: medium
- explicitCutting entry-level positions and middle-management headcount to reduce costs risks eliminating the future pool of people with the contextual understanding needed to validate AI outputs.conf: high
- explicitcorroboratedLLMs tend to reduce the diversity of generated ideas because they fixate on the first idea generated and are biased toward averaging.conf: highsupportsTBM 418: Campfires, Trails, and Quests · p23 — Both name AI-induced homogenization/epistemic narrowing of ideas as a distinct organizational risk, independent of individual output quality.supports ←AI Is Undermining Leaders’ Judgment. Here’s What to Do About It. · p4 — 'Organizational monoculture' is the leadership-level, systemic manifestation of the idea-homogenization mechanism Reganti documents at the individual AI-output level (LLMs fixate on the first idea and average toward the mean).
- implicitBecause unexpected/surprising information triggers creative insight, AI's tendency to homogenize ideas will reduce organizational innovation capacity and the ability to notice environmental signals that call for change.conf: medium
- explicitdevelopedFraming generative AI as a chatbot tool that mainly needs better prompting will not materially reduce the risks of expertise erosion or homogenization.conf: highextends ←AI Is Undermining Leaders’ Judgment. Here’s What to Do About It. · p1 — Explicitly cited in the text: this article picks up where the design-literature piece leaves off, taking up 'a prior question' — how leaders cultivate judgment no interface redesign can supply.
- explicitOrganizations should treat AI as part of a system that humans can shape, designing AI-enabled work environments that preserve human agency and leadership.conf: high
- explicitStructuring AI interactions to Socratically challenge users' assumptions (rather than simply answering) helps develop critical thinking.conf: high
- explicitAI feedback that challenges the novelty/redundancy of an idea (versus feedback that only helps an idea fit a predefined goal) produces a wider range of more innovative outcomes.conf: high
- explicitDeliberately creating AI-free time blocks, or 'gated interfaces' that require users to deposit their own context before AI assists, helps preserve independent judgment.conf: highextendsThe Psychological Costs of Adopting AI · p13 — psychological-costs-of-adopting-ai proposes cognitive friction (e.g. requiring hypothesis formation) abstractly as an intervention; this article supplies concrete organizational mechanisms (AI-free sessions, gated interfaces) implementing that same friction principle.
- explicitRequiring entry-level employees to complete tasks without AI tools builds the contextual expertise and error-spotting ability that senior employees who engage deeply with (rather than just consume output from) a technology retain and even deepen.conf: highsupportsTBM 432: Bundling & Unbundling Capabilities (and AI) · p11 — Both describe the human role shifting toward direction/assessment/judgment as a genuinely different (and valuable) skill from the skill AI substitutes for — Anthony's senior bankers who engage with the technology vs. tbm-432's 'direction, assessment, and oversight' framing.
- explicitRunning AI and a human team on the same task independently in parallel, then combining outputs, can reveal each side's comparative strengths — AI for exploration/breadth, humans for framing, prioritization and judgment.conf: high
- explicitThe dominant chatbot interface (a wall of text the user must sift through) is poorly suited to some tasks and can devolve into a messy loop of misinterpretation and reinforcement.conf: high
- explicitReplacing the 'frictionless' chatbot interface with one that shares raw data and competing evidence, forcing deliberation, works well specifically in domains where users are already experts.conf: high
- implicitAI need not be designed as a generic chatbot; interface design itself is a lever for preserving or eroding human reasoning, independent of underlying model capability.conf: medium
Arguments
Widespread AI use is eroding organizational critical-thinking capacity, not just individual task performance. strength: medium
Explicit premises:
- Black boxing (trusting tools without understanding them) was already documented pre-genAI as eroding expertise (Anthony 2021)
- Named 2025 MIT Media Lab and 2026 Wharton studies show AI users are disinclined to question or research AI output
Implicit premises:
- Critical thinking is constituted by the habit of actively questioning information sources, not just an innate trait
- A habit that erodes at the individual level compounds into an organizational-level capability loss when adopted broadly
- The long-term cognitive impact is not yet established (the authors concede this), yet they treat the directional risk as actionable now
Inferences:
- If the mechanism is habitual rather than purely cognitive, then interventions that reintroduce friction into the habit (not just model improvements) should be able to counteract it — which is exactly the move the article's second half makes
Reliance on AI outputs without contextual/tacit expertise increases the risk of professional error. strength: medium
Explicit premises:
- AI accesses facts and data but does not experience data directly, so it overlooks nuances
- Passing a Bar exam does not mean an AI understands the justice system's stakeholders, case variation, or how other expertise relates to law
Implicit premises:
- Genuine professional expertise requires lived, embodied, multi-domain experience — a form of tacit knowledge not reducible to propositional facts or exam performance
- Nuance recognition is a distinctly human capacity rather than a matter of more data or better models
- Organizations that cut the entry-level and middle-management roles that build this contextual understanding are trading a short-term cost saving for a longer-term validation-capacity loss
Inferences:
- If organizations use AI to eliminate the very entry-level positions that build contextual expertise, they may run out of people qualified to validate AI output within a generation — a pipeline problem the article names but does not quantify
Broad AI adoption will reduce organizational innovation and agility via homogenization of ideas. strength: medium
Explicit premises:
- Several studies show LLMs reduce the diversity of generated ideas, via first-idea fixation and averaging bias
- Unexpected/surprising information is what triggers creative insight
Implicit premises:
- If many employees independently converge on the same AI-mediated 'average' idea, the pool of divergent raw material available to the organization as a whole shrinks — an aggregate, collective-action effect invisible from any single AI interaction
- Organizational agility depends on noticing weak/unexpected signals, which a homogenized thinking style is less likely to register
Inferences:
- The homogenization risk is a negative externality that scales with the number of employees using AI independently — meaning it may be invisible to (and unmeasurable by) any individual manager evaluating a single AI deployment, which is itself an argument for organization-level rather than tool-level governance
Structuring AI interactions to challenge users (Socratic questioning, redundancy-flagging feedback) produces better outcomes than structuring them to simply answer. strength: medium
Explicit premises:
- AstraZeneca's Prompt with Me Challenge participants reported the approach 'surfaced new angles, challenged my assumptions, and inspired self-reflection'
- In the BUILD2GETHER hackathon, teams that received AI feedback identifying redundancy with existing projects (rather than feedback on fitting predefined goals) produced a wider range of more innovative ideas
Implicit premises:
- The value of an AI interaction is primarily a function of interaction/prompt design, not raw model capability
- A single controlled comparison (two feedback conditions in one hackathon) generalizes to broader organizational design choices about how AI should be deployed
Inferences:
- If interaction design matters more than model choice for this outcome, procurement and adoption decisions that focus mainly on which model/vendor to buy are targeting the wrong lever — a claim with direct relevance to enterprise AI-adoption governance, though the article does not draw it out explicitly
For expert domains, replacing the frictionless chatbot with a deliberative, evidence-surfacing interface improves human-AI collaboration; this does not necessarily generalize to novice users. strength: medium
Explicit premises:
- The default chatbot interface produces a 'wall of text' the user must sift through, and can devolve into a messy, self-reinforcing loop of misinterpretation
- Google's AI Co-mathematician was built with a flexible, annotated interface because chat interfaces did not support mathematicians' iterative, multi-dimensional research process
- The authors themselves state the deliberative-interface approach 'works well in a context such as mathematics or coding, where users are already experts'
Implicit premises:
- Interface design is itself a lever for preserving or eroding human reasoning, independent of the underlying model
- Deliberative/friction-heavy interfaces impose a cognitive cost that only pays off when the user already has the expertise to make use of the surfaced complexity
Inferences:
- By the article's own logic, novice or entry-level users — the population it elsewhere identifies as most at risk of losing the chance to build contextual expertise — are the group for whom this specific interface fix is least validated, an unresolved tension the authors do not address
Narratives
- AstraZeneca's 'Prompt with Me Challenge' (launched August 2025): a 10-day program of brief exercises for 1,500+ participants teaching not just prompting but how to prompt in ways that deepen thinking; won a silver Brandon Hall award; a year later adopted by 50+ teams/functions with thousands of completions.
→ Supports the claim that structuring AI interaction Socratically (challenging assumptions rather than just answering) builds critical thinking, and that such programs can scale across a large organization. - The BUILD2GETHER AI–SDG Hackathon: ETH Zürich / NTU Singapore researchers ran a 24-hour hackathon (~2,000 participants, ~500 teams) where half the teams got AI feedback on fitting predefined SDG goals and half got feedback flagging redundancy with existing projects; the redundancy-challenged group produced a wider range of more innovative ideas.
→ Supports the same 'turn AI on its head' claim with a controlled (two-condition) comparison, giving it more evidentiary weight than the AstraZeneca case, though methodology/evaluation criteria for 'innovative' are not detailed. - Callen Anthony's ethnographic study of investment bankers: junior bankers who practiced the underlying process themselves (rather than trusting the black-boxed tool) were better able to spot mistakes and suggest alternatives; among senior bankers, those who made sense of the technology (rather than treating it as just an output-producing tool) deepened their expertise and could meaningfully challenge junior analysts.
→ Supports both the 'decontextualization of expertise' risk (via the black-boxing junior contrast) and the 'center human agency' prescription (via the senior-banker contrast). - With Company's 2024 parallel-processing branding experiment: for an early-stage venture's branding challenge, one team worked traditionally and another relied heavily on AI, generating ~1,700 images. The AI-heavy team explored faster and more broadly; the human-led team produced fewer, more coherent, better-framed routes and was ultimately selected. With Company continued using parallel processing on subsequent projects.
→ Supports the 'process in parallel' prescription and the claim that AI and humans have complementary strengths (AI for exploration, humans for framing/judgment) rather than AI simply substituting for human creative work. - Google's 'AI Co-mathematician': built with a flexible, non-chatbot interface featuring an annotation system that visualizes reasoning steps, because standard chat interfaces did not support mathematicians' iterative, multi-dimensional research process; it helps develop workstreams and verify proofs.
→ Supports the claim that AI need not be a generic chatbot and that deliberate interface redesign can better support expert reasoning processes. - Intel's 2007 no-email-day pilot for a group of engineers (following similar experiments at Deloitte and U.S. Cellular).
→ Used as a historical analogy legitimizing the 'AI-free space' prescription — framing deliberate AI abstinence blocks as a natural successor to prior information-overload interventions, rather than a novel or untested idea.
Questions addressed
- ✅ How can organizations mitigate AI-driven erosion of critical thinking, expertise, and epistemic pluralism?
Three prescriptions: turn AI on its head (Socratic/challenging interaction design), understand the right place and time for AI (AI-free space, centering human agency, parallel processing), and try different (non-chatbot) interfaces. - ⭕ What is the long-term cognitive impact of AI use on human reasoning?
The authors explicitly state 'the long-term impact on cognition remains to be determined,' while still treating the directional risk as actionable. - ✅ Should AI governance frameworks focus only on where AI can be implemented, or also on where/when it should not be used?
The authors argue explicitly for the latter — companies should consider where and when AI should not be used, not just where it can add value. - ✅ Can chatbot interfaces be redesigned to better support deliberative human reasoning?
Partially — yes for expert domains (math, coding), illustrated by Google's AI Co-mathematician; the article does not resolve whether this generalizes beyond expert users. - ⭕ What interface or interaction design should protect the reasoning of novice/entry-level employees — the group the article itself identifies as most exposed to expertise erosion — given that the deliberative-interface fix is validated only for domains where 'users are already experts'?
- ⭕ How do these interventions (AI-free sessions, gated interfaces, Socratic prompting programs) scale and survive organizational resistance, incentive misalignment, or the productivity pressure that drove AI adoption in the first place?
Gaps
- incompleteThe narrative examples (AstraZeneca's Prompt with Me Challenge, With Company's parallel-processing experiment) report adoption/engagement metrics — participant counts, teams onboarded, an industry award — but no outcome data tied to measured critical-thinking, decision-quality, or business performance. No control group, no before/after cognition measure, no longitudinal retention check is reported for either case, despite the article's central claim being about a durable organizational capability.
Read next: The MIT Media Lab 2025 and Wharton 2026 studies (cited but not named or detailed beyond one sentence) would need to be read directly to assess whether they measure organizational-level outcomes or only individual-session behavior; also worth checking whether AstraZeneca or ETH Zürich/NTU Singapore have published any follow-up evaluation beyond the award and participation counts. - incompleteSourcing quality is uneven: alongside two well-identified, dated studies (2025 MIT Media Lab, 2026 Wharton), the article repeatedly leans on unnamed attribution — 'several studies have shown,' 'one recent study suggests,' 'another group of researchers reported,' 'many researchers worry' — for claims (LLM idea-homogenization, decontextualization risk, the Australian telecom AI-free-session finding) that are just as load-bearing as the named studies but cannot be independently checked.
Read next: Track down Chengwei Liu and colleagues' HBR article on 'gated interfaces' (cited by name) and the study of the unnamed Australian telecommunications carrier's AI-free strategy sessions for primary sourcing before treating those specific claims as established. - illogicalThe BUILD2GETHER hackathon's causal claim — that redundancy-challenging AI feedback produced 'more innovative' ideas than goal-fit feedback — is drawn from a single ~24-hour event with no reported statistical test, no stated evaluation methodology (who judged 'innovative' and by what rubric), and no effect size. The article presents it with the confidence of a controlled experiment rather than a single-event field observation.
Read next: Look for a published paper from the ETH Zürich / NTU Singapore research team behind BUILD2GETHER, which would specify the evaluation methodology and any statistical analysis omitted from this summary. - incompleteThe piece proposes concrete organizational interventions (AI-free sessions, gated interfaces, parallel processing) but never addresses adoption cost, employee resistance, or whether an 'AI-free' mandate is enforceable or verifiable at scale — a gap made sharper by other corpus evidence that employees actively route around AI mandates and governance friction under productivity pressure.
Read next: extract-enterprise-ai-leadership-problem (executives planning to lay off non-adopters while employees actively sabotage AI initiatives) is directly relevant background this article does not engage with. - incompleteThe 'different interfaces' section concedes that deliberative, evidence-surfacing interfaces 'work well ... where users are already experts,' but never addresses what interface design should look like for non-expert or entry-level users — precisely the population the article elsewhere identifies as most at risk of losing the chance to build contextual expertise in the first place. The prescription and the diagnosis point at two different populations without the article noticing the mismatch.
Read next: Research on scaffolded/adaptive interfaces for novices (e.g., the 'expertise reversal effect' in cognitive-load literature) would be the natural next read; see extract-precision-proactivity-measuring-cognitive-load#p27 in this corpus, which names that exact effect.
Related documents
- TBM 418: Campfires, Trails, and Quests
- TBM 432: Bundling & Unbundling Capabilities (and AI)
- Cognitive Surrender
- How People Are Really Using AI in 2026
- Research: Why You Shouldn't Treat AI Agents Like Employees
- Designing the AI-native engineering organization
- The best response to AI slop, infinite advice, and online noise is from Robin Williams
- Finding Comfort in the Uncertainty
- The Hidden Realities of AI Adoption
- What is the harder human-capital problem beneath token capital?