When Employees Are Held Accountable for AI-Generated Decisions

Anne-Sophie Mayer, Elmira van den Broek, Tomislav Karačić · article · 2026-07-22 source ↗ Reader 🖥️

practical · mixed — Grounded in a multi-year qualitative field study (banking, recruitment, biotechnology) but written for a managerial audience: expository framing, three narrative case studies, and an argumentative closing section deriving prescriptive lessons for leaders.

Structure

Key terms

• masking (AI's role)
  Substituting the AI's actual (often opaque or counter-intuitive) rationale with familiar, plausible-sounding professional-expert explanations that do not reflect how the decision was actually made, in order to preserve one's perceived expertise.
• embracing (AI)
  Actively leveraging AI outputs (scores, graphs, traits) to bolster one's own credibility with a skeptical audience, potentially to the point of automating away one's own explanatory role.
• complementing (AI)
  Building durable, visible interpretive expertise that translates an AI's granular outputs into operationally meaningful explanations, without competing with the AI for the underlying decision.
• explainable AI / accountability
  Reframed as a social and organizational problem — whether an employee positioned between the AI and a stakeholder can produce an explanation the stakeholder finds credible — rather than a purely technical property of the model.
• interpretive expertise
  A new form of professional value built around interpreting, contextualizing, and connecting AI outputs to business outcomes, distinct from the expertise of making the decision oneself.
• critical engagement
  The organizationally sanctioned ability of an employee to question, investigate, and push back on an AI decision rather than merely being expected to defend it.

Propositions

• explicitEmployees held accountable for explaining AI-generated decisions do not merely relay AI outputs as-is; they reinterpret, reshape, and sometimes hide them to protect their professional role and credibility.conf: high
• explicitHow employees respond to being accountable for AI decisions depends largely on whom they are accountable to and must convince, and on the organizational structures that support or constrain their ability to make sense of AI outputs — not on the underlying AI technology itself.conf: high
  supportsTBM 420: The AI Playbook Puzzle · p2 — Supports the claim that identity threat (not technical or tactical gaps) is the central obstacle to effective AI adoption, by showing identity/credibility protection as the primary driver of employee response across three separate case studies.
• explicitWhen loan officers could not see or override a fully automated loan-approval AI's actual rationale, they masked its role by citing familiar expert-sounding reasons (e.g., weak credit history) that did not reflect the AI's real basis, in order to preserve their perceived expertise.conf: high
  extendsThe Psychological Costs of Adopting AI · p8 — Extends the abstract claim that mandated AI use damages professional identity when it conflicts with professional-group membership by supplying a concrete mechanism (masking familiar expert language over an unrecognizable AI rationale) and its downstream organizational cost (customer attrition) in a real field case.
• explicitMasking AI decisions behind plausible but inaccurate expert explanations eroded rather than built customer trust: customers detected the mismatch through follow-up questions, doubted the officer's understanding, and some left for a competitor bank.conf: high
  supportsThe Psychological Costs of Adopting AI · p7 — The loan officers' credibility-protecting masking behavior, and its detection and punishment by customers, is a concrete field instance of the credibility-debt dynamic identified in the psychological-costs survey.
• explicitWhen recruiters redesigned an AI recruiting tool's outputs to make them self-explanatory to hiring managers, hiring managers no longer needed recruiters as translators — making the recruiters' substantial ongoing interpretive work invisible even though it remained functionally essential.conf: high
• explicitEmployees build durable, valued new interpretive expertise around AI outputs when they have direct access to the AI's underlying data, dedicated time and space to study it (e.g., a shared imaging lab), and regular structured interaction with both the AI's developers and the downstream stakeholders who rely on their explanations.conf: high
• explicitAccountability for an AI decision becomes problematic — pressure to accept rather than genuine accountability — when employees are expected to defend decisions they are not permitted to question.conf: high
• explicitOrganizations that create recurring, structured dialogue between employees and the people to whom AI decisions must be explained (rather than one-off, terminal exchanges) enable employees to progressively improve their explanatory practice.conf: high
• implicitcorroboratedTechnical improvements to an AI system's explainability (better model transparency) are insufficient on their own to solve the employee-accountability problem; the organizational and relational structures around who must explain the decision to whom matter at least as much.conf: medium
  supports ←Presentation: Decision Models in Agentic Architectures: From Production to Agent Skills · p11 — Porcelli's audit-trail-first framing (built for regulators/auditors) supplies exactly the technical explainability infrastructure the prior extraction treats as necessary but, on its own, insufficient — the two are compatible and about different audiences (auditor vs. customer-facing employee), not opposed; together they suggest a technical audit trail is necessary but the organizational/relational explaining-to-people work still has to happen on top of it.
  refines ←To Adopt AI at Scale, Employees Need to Trust Agents · p4 — This article shows disclosing limitations raises interpersonal trust/effectiveness; the other piece argues technical explainability alone is insufficient for the deeper accountability problem — together they suggest transparency helps trust but doesn't resolve who bears justificatory responsibility for an agent's decisions.
• implicitcorroboratedFully automating a decision (stripping employees of override authority) transfers the social and justificatory labor of explaining it onto employees without necessarily transferring the epistemic access needed to perform that labor well, creating a structural mismatch between formal decision authority (the AI) and social accountability (the human).conf: high
  supports ←Everyone I recommend on AI is optimistic. Except Cory Doctorow. · p6 — Svenson's 'accountability sink' (radiologist/lawyer signs, liability transfers) is the same structural mismatch that proposition names directly: formal decision authority sits with the AI while social/legal accountability is transferred to the human, without transferring the epistemic access needed to bear it.

Arguments

Narratives

• A large German bank (followed 2019–2025) fully automated loan-approval decisions with a predictive AI; loan officers could not override the AI but remained responsible for explaining decisions to customers. Officers found the AI's technical, rule-based explanations (e.g., 'unstable financial situation' for a customer with no debt and steady income) didn't match their expert standards, so they substituted familiar banking-expert justifications that didn't reflect the AI's actual logic. Customers detected officers' confusion, lost trust, and some left for competitors.
  → Illustrates the 'masking' response and its cost: preserving individual expert authority at the price of organizational trust.
• A global consumer-goods company's internal recruiters (studied 2018–2022) enthusiastically adopted an AI candidate-scoring tool to bolster their credibility with skeptical hiring managers. When managers struggled to interpret raw scores, recruiters redesigned the AI's outputs (benchmarks, highlighted traits, interview questions) to be self-explanatory. This succeeded at making the tool usable — but also made recruiters' substantial behind-the-scenes interpretive work invisible, since managers came to attribute the value to the AI rather than to the recruiters who shaped it.
  → Illustrates the 'embracing' response and its paradox: automating your own explanatory role can erase the visibility of your expertise.
• A biotechnology company (studied 2021–2023) used an AI system to automatically sort seeds by quality. Because seed experts operated the sorting machine, they could inspect the underlying image data and work closely with developers. Given a dedicated imaging lab, time to study individual seeds, weekly meetings with developers, and daily consultation with supply-chain managers, they built durable new expertise translating the AI's classifications into operationally meaningful explanations — and could identify defects the model had never been trained on, triggering funded joint research.
  → Illustrates the 'complementing' response as the most functional outcome: an organizationally supported partnership between employee and AI that keeps expertise visible, valued, and improving.

Questions addressed

• ✅ What happens to employees who must explain and defend AI-generated decisions they did not make and do not fully understand?
  They diverge into (at least) three response patterns — masking, embracing, or complementing the AI — depending on their accountability relationships and organizational supports.
• ✅ Why do some employees mask AI's role while others embrace or complement it?
  The divergence tracks whom the employee must convince (one-off customers vs. recurring internal stakeholders) and whether the organization provides data access, dialogue opportunities, and recognition for interpretive work.
• ✅ Does improving an AI system's technical explainability, by itself, resolve the accountability problem for the employees who must explain its decisions?
  Implicitly resolved as 'no' — the article treats organizational and relational structure, not model transparency, as the operative variable across all three cases.
• ⭕ Why did the biotech company invest in the organizational structures (imaging lab, weekly developer meetings, funded joint research) that produced the successful 'complementing' response, while the bank and recruiting firm did not build comparable structures?
• ⭕ Are the organizational investments modeled by the biotech case (dedicated lab space, weekly cross-functional meetings, funded research into edge cases) economically feasible in high-volume, thin-margin decision contexts like consumer lending or high-volume recruiting — or is the 'complement' response only available in labor contexts with the biotech case's cost and volume structure?

Gaps

• incompleteThe article never explains why the biotech company invested in the structures that enabled the 'complement' response (dedicated imaging lab, weekly developer meetings, funded joint research) while the bank and recruiting firm did not build comparable structures. Is the difference driven by decision stakes, regulatory pressure, per-decision cost, or organizational culture?
  Read next: Literature on why organizations invest differently in human-AI collaboration design — regulatory-stakes and decision-criticality frameworks (e.g., EU AI Act risk tiers) and studies of resourcing decisions in AI deployment.
• incompleteNo discussion of cost or scale feasibility: the biotech-style organizational investment (dedicated lab, weekly meetings, funded research) may not be economically viable in high-volume, thin-margin decision contexts like consumer lending or mass recruiting, where the 'complement' response may simply not be affordable regardless of leadership intent.
  Read next: Operations and cost-of-service literature on human-AI collaboration at scale; comparisons of per-decision cost across banking, recruiting, and biotech contexts.
• uninformedThe article does not engage the regulatory dimension of the banking case at all: in consumer lending, banks in many jurisdictions have legal obligations (e.g., US Regulation B adverse-action notices, EU consumer-credit rules, and the EU AI Act's explanation rights for high-risk AI) to provide accurate, specific reasons for credit denials. Masking the AI's actual rationale with plausible but inaccurate reasons could carry compliance and legal risk the article never mentions.
  Read next: US Regulation B / ECOA adverse-action-notice requirements and case law; EU AI Act Article 86 right to explanation for high-risk AI systems.
• illogicalThe three responses are presented as roughly comparable strategic choices employees and organizations make, but the cases differ from the outset in how automated/black-box the AI was by design: loan officers and recruiters faced systems they could not inspect, while seed experts could directly examine underlying image data because they operated the machine. This conflates 'which response an employee chose' with 'which response was structurally available given the AI's design' — understating how much of the outcome variance is explained by initial system design (level of automation and data access) rather than by post-hoc organizational choices about dialogue and recognition.
  Read next: Literature on levels of automation (e.g., the Sheridan-Verplank scale) and how automation level constrains downstream human sense-making options, independent of organizational culture.

Closes gaps elsewhere

• The Psychological Costs of Adopting AI · g1 — That gap calls out the survey's failure to engage the harder case of genuinely substitutive AI, where identity threat is an accurate perception of real displacement rather than 'debt to be paid down.' This article supplies exactly that harder case: three field studies of AI systems that fully removed employee decision authority, showing that the resulting psychological/behavioral responses (masking, embracing, complementing) are structurally shaped coping strategies with real, divergent organizational costs and benefits — not simply debt to be mitigated with reframing language.

Related documents

• TBM 420: The AI Playbook Puzzle
• Your AI agent just blamed the network team. Now what?
  Agent Blame Attribution & Accountability Explainable AI Human-in-the-Loop Trust
• To Adopt AI at Scale, Employees Need to Trust Agents
  Explainable AI Human-in-the-Loop Trust
• Lenny Rachitsky's Post
  Professional Identity Psychological Debt
• Can an AI-Powered Scribe Curb Physician Burnout?
  Human-in-the-Loop Trust
• From AI Code to Trusted Software: Harness Engineering in Practice
  Human-in-the-Loop Trust
• The Hidden Realities of AI Adoption
  Professional Identity Trust
• The Kubernetes Approach to AI-Assisted Maintainership Prioritises Human Accountability
  Human-in-the-Loop Trust
• How tech workers actually feel about AI in 2026 | Annual AI sentiment survey (Noam Segal)
  Professional Identity Psychological Debt
• The Psychological Costs of Adopting AI
  Explainable AI Psychological Debt