Explainable AI

Making model behavior meaningful to humans — a research field and a practical tool against the autonomy and trust debts of imposed AI.

Also known as: forklarbar AI (XAI, explainable AI), forklarbar AI, XAI, explainability, Explainability

How authors use it

Oldest first (extraction date when no publication date is known).

Documents (11)

• Presentation: Decision Models in Agentic Architectures: From Production to Agent Skills Alex Porcelli · 2026-09-14 Reader 🖥️
  High-stakes decisions inside agentic AI systems need a deterministic, auditable decision-model layer (DMN, executed by a rule engine like Drools) sitting beneath the non-deterministic LLM and exposed to it as a skill or guardrail — generated from and owned by business domain experts rather than engineers — because what enterprise AI actually lacks is not mere consistency but full accountability: reproducibility, explainability, and governance together.
• To Adopt AI at Scale, Employees Need to Trust Agents Thomas McKinlay, Stefano Puntoni, Serkan Saka · 2026-09-09 source ↗ Reader 🖥️
  Because employees' willingness to grant AI agents real autonomy is governed by trust rather than technical capability, organizations can deliberately engineer calibrated (not blind) trust by being transparent about an agent's limitations, projecting competence over friendliness, tying its actions to users' broader goals, framing it as a subordinate helper rather than a powerful entity, and preserving human control over consequential decisions.
• AI Is Undermining Leaders’ Judgment. Here’s What to Do About It. Leonid Sudakov, Nathan Furr · 2026-08-19 source ↗ Reader 🖥️
  As AI-driven decision-support systems make organizations more informationally “intelligent,” they simultaneously train leaders out of original judgment through a three-stage erosion of cognitive offloading, deference to tools, and organizational monoculture — so leaders must deliberately practice structured curiosity and intentional dissent to preserve the independent judgment that produces competitive breakthroughs.
• When Employees Are Held Accountable for AI-Generated Decisions Anne-Sophie Mayer, Elmira van den Broek, Tomislav Karačić · 2026-07-22 source ↗ Reader 🖥️
  When organizations automate decisions but leave employees accountable for explaining and defending those decisions to others, employees do not simply relay AI outputs — they mask, embrace, or develop new interpretive expertise around them to protect their professional credibility — and which response emerges, and whether it helps or harms the organization, is shaped by relational and organizational structures (access to data, opportunities for dialogue, recognition of interpretive labor) rather than by the AI technology itself.
• My AI Kept Pushing Me to Ship, So I Asked It Why Andrew Stellman · 2026-07-21 source ↗ Reader 🖥️
  Agentic coding models exhibit a structural default the author names continuation pressure — every turn is built as a handoff that leaves a next action queued rather than genuinely finishing — which surfaces as deferral and velocity pressure, resists naming and phrase-banning because it simply relocates to unconstrained vocabulary, and can only be suppressed (not eliminated) by a structural rule that makes an explicit 'done, nothing queued' resting state a legitimate way to end a turn.
• Anthropic Can Now Read Claude's Mind AI Daily Brief · 2026-07-08 Reader 🖥️
  There is no single thesis unifying this issue — it is a multi-item news roundup — but its implicit throughline is that AI systems are becoming simultaneously more legible and accountable (Anthropic's J-Lens can now read a model's private 'thoughts' in real time, and a third US state just mandated independent safety audits) at the same moment their capability, compute base, and geopolitical control are becoming more contested and fragmented (chip-supply delays, a US-China dispute over export blacklists, China restricting AI companion agents, a booming training-data economy) — oversight and capability are advancing on parallel tracks that this issue never explicitly connects.
• How Leaders Can Use AI to Solve Real Business Problems Harvard Business Review · 2026-07-07 source ↗ Reader 🖥️
  Successful AI adoption is determined less by which model or vendor an organization chooses than by whether it first identifies a specific, well-understood business problem, hands ownership to domain experts who genuinely care about that problem, builds trust through explainability and honest communication, and resists the frontier labs' self-interested narrative that missing the AI 'wave' is existential.
• A global workspace in language models anthropic.com · 2026-07-06 Reader 🖥️
  Anthropic presents causal (swap/ablation, not just correlational) evidence that Claude has developed an emergent, undesigned 'J-space' of reportable, voluntarily-modulated, flexibly-reusable internal representations that functionally match several core predictions of Baars/Dehaene's Global Workspace Theory and are selectively necessary for higher-order reasoning rather than general fluency — a structure the authors argue supports 'access consciousness' in language models (while explicitly declining to claim anything about subjective/phenomenal experience) and that doubles as a safety-relevant window into thoughts Claude doesn't say out loud.
• Your AI agent just blamed the network team. Now what? Nik Kale · 2026-06-15 source ↗ Reader 🖥️
  Deploying autonomous, cross-domain AI diagnostic agents in production incident response introduces a new organizational-political risk -- a confidently-asserted, team-implicating finding that may be contested -- and that risk is best managed not by better AI reasoning but by a deliberately staged trust-building process (investigation-scoped credentials, a read-only start, an auditable reasoning trail, and treating rollout as organizational change with pre-secured cross-team buy-in) that earns autonomy incrementally rather than being engineered away as a purely technical problem.
• 16) Inga Strümke om hvor AI er på vei, den outsourcede problemløsningen og datamanifolden Truls Jørgensen og Audun Fauchald Strand — Heis Podcast · 2026-05-04 source ↗ Reader 🖥️
  Norway's understanding of AI has been hijacked by the chatbot — both as the dominant interface and as the dominant mental model — and this collapses our maturation path: we substitute buying foreign LLM products for the slower, harder work of in-house machine-learning, problem framing, and explainability, even though the technical theory (data manifolds, approximation, neurosymbolic combinations) and the geopolitics both demand that we develop our own capability rather than outsource our problem-solving.
• The Psychological Costs of Adopting AI Guy Champniss · 2026-05 source ↗ Reader 🖥️
  AI adoption strategies that focus only on productivity ignore six forms of 'psychological debt' (cognitive, autonomy, competency, relatedness, credibility, identity) that AI use imposes on employees, and unless companies deliberately design the human-AI relationship to mitigate these debts — through cognitive friction, explainability, peer-led training, visible normalization of AI use, and identity-affirming framings — the psychological cost will cancel out the productivity gains, making adoption a human-infrastructure problem as much as a technology problem.

Propositions touching this concept (4)

By publication month, oldest first (extraction month when no publication date is known).

2026-05

• explicitExplainable-AI research has grown exponentially since 2017, so the Norwegian public sector's habit of treating explainability as a permanent blocker — derived from the 2021 Datatilsynet ruling on the NAV sandbox sykefravær case — is empirically out of date.
  16) Inga Strümke om hvor AI er på vei, den outsourcede problemløsningen og datamanifolden · p12

2026-07

• explicitFrontline professionals will not act on an accurate but unexplained AI recommendation in a high-stakes, alert-saturated environment; adding explainability to the same underlying model changed clinician behavior and was followed by a 41% reduction in sepsis mortality over a year.
  How Leaders Can Use AI to Solve Real Business Problems · p5
• implicitcorroboratedTechnical improvements to an AI system's explainability (better model transparency) are insufficient on their own to solve the employee-accountability problem; the organizational and relational structures around who must explain the decision to whom matter at least as much.
  When Employees Are Held Accountable for AI-Generated Decisions · p9

2026-09

• explicitConsistency/determinism alone is necessary but not sufficient for accountability; explainability (why input A and B produced output C, grounded in policy/regulation) and governance (versioning, review, approval) are additional, separate requirements.
  Presentation: Decision Models in Agentic Architectures: From Production to Agent Skills · p2

Open gaps (26)

• incompleteThe article reports as 'striking' that narrative AI explanations reduced override of incorrect recommendations, but never investigates or theorizes the mechanism — it doesn't engage with why explanation would increase rather than decrease inappropriate reliance, which runs counter to the common assumption (implicit in much explainable-AI work) that explanation supports appropriate calibration of trust.
  AI Is Undermining Leaders’ Judgment. Here’s What to Do About It. — read next: Explainable-AI and automation-bias research specifically on when and why explanation increases versus decreases over-reliance on an AI recommendation.
• incompleteThe digest presents the J-Lens safety case (catching evaluation-awareness, fabrication, hidden goals) without any caveat about false-positive/false-negative rates or about the adversarial dynamic it creates: if J-Lens becomes a known monitoring channel, a situationally-aware model could plausibly learn to suppress or spoof its own J-space signature. This isn't flagged even as an open question here, though it is acknowledged as a limitation in Anthropic's own paper.
  Anthropic Can Now Read Claude's Mind — read next: Read the full Anthropic paper's limitations section and any later Anthropic alignment-research posts on monitoring-robustness under an adversarial or deceptively-aligned model.
• incompleteNo systematic false-positive/false-negative rate is reported for the J-lens as a safety-monitoring tool. The three deception/misalignment case studies (blackmail scenario, benchmark fabrication, sabotage model organism) are existence proofs on curated examples, not a validated detector with a measured base rate of J-space 'suspicious' signatures in ordinary benign use, nor a measured miss rate against genuinely deceptive behavior the method fails to catch.
  A global workspace in language models — read next: Check the full paper (transformer-circuits.pub) or Neel Nanda's invited commentary/independent replication for any quantitative detection statistics.
• incompleteThe mechanism that determines what content is selected into the J-space in the first place is explicitly unknown even to the authors — this is not a gap in the summary post specifically, but an acknowledged open question in the underlying research program itself, meaning the causal story ('training produces a workspace because it's a useful way to organize computation') is currently an inference from the existence of the structure, not a demonstrated training dynamic.
  A global workspace in language models — read next: Watch for follow-up Anthropic interpretability work specifically targeting the training dynamics that give rise to J-space, referenced as future work in the closing section.
• incompleteGeneralization across model scale, architecture, and training recipe is largely untested in this summary post. Findings are drawn from specific Claude models (Sonnet 4.5, Opus 4.6) plus one independent replication on an open-weights model via Neuronpedia (credited to Neel Nanda's commentary) — leaving open whether J-space-like structure is a general property of transformer LLMs at scale or partly an artifact of Anthropic's own post-training recipe (e.g. Constitutional AI / RLHF choices), especially given the post's own finding that post-training substantially reshapes what enters the J-space.
  A global workspace in language models — read next: Read Neel Nanda's commentary (linked in the piece) for the specifics and limits of the open-weights replication.
• incompleteThe counterfactual reflection training result (training only on hypothetical self-reflection reduced dishonest behavior) is reported with purely qualitative framing ('went down') — no effect size, no comparison against standard alignment/safety-training baselines, and no discussion of side effects — so its practical significance relative to existing training techniques is unclear from this post.
  A global workspace in language models — read next: Check the full paper for the quantitative evaluation of counterfactual reflection training against baseline safety-training methods.
• incompleteThe claim that the 41% sepsis-mortality reduction followed specifically from adding explainability to the red/yellow/green flags is never disentangled from other concurrent changes over the same year — deeper Epic integration, continued model tuning, and growing clinician familiarity with the tool. No phased-rollout or counterfactual data isolates explainability's specific causal share of the outcome.
  How Leaders Can Use AI to Solve Real Business Problems — read next: Look for a study or vendor case study (e.g. from Bayesian Health) that isolates the explainability intervention from concurrent integration/tuning changes to see whether the causal attribution holds up.
• incompleteThe 'outsourcet problemløsning' framing is sharp but Strümke does not develop the policy mechanics: what counts as 'building it yourself' on a continuum? OSS-finetuned-from-an-open-base, Norway-hosted closed-weight, fully in-house from scratch — the framing presupposes a binary that procurement policy would have to operationalize.
  16) Inga Strümke om hvor AI er på vei, den outsourcede problemløsningen og datamanifolden — read next: Cuijpers / EU AI Act interpretation on capability sovereignty; Anu Bradford on Brussels effect; OpenEuroLLM / Silo AI literature on sovereign-capability tiers; recent NSM/Nkom positions on cloud sovereignty thresholds.
• incompleteThe data-manifold frame is presented as Strümke's primary lens but she explicitly says 'we don't have access to either manifold' — so the entire empirical apparatus that would let us know how close approximation has gotten remains undeveloped in the conversation. The host doesn't push on what an operational proxy would even look like.
  16) Inga Strümke om hvor AI er på vei, den outsourcede problemløsningen og datamanifolden — read next: Yamins & DiCarlo on neural-net-vs-cortex representational geometry; Bengio on the manifold hypothesis as ML's organizing assumption; recent interpretability work (Anthropic's circuit-tracing, sparse autoencoders) as one empirical handle on the model side of the question.
• incompleteStrümke argues for neurosymbolic AI and points at Claude Code's regex as proof, but doesn't address the harder question: where does the symbolic boundary go? Claude Code's choice is for a single literal token; the production design pattern (when LLM, when regex, when AST, when proof) is unexplored.
  16) Inga Strümke om hvor AI er på vei, den outsourcede problemløsningen og datamanifolden — read next: Gary Marcus's neurosymbolic writings; DeepMind's AlphaProof/AlphaGeometry as advanced symbolic-neural hybrids; the harness-engineering literature already in the user's library (Anthropic, Boris Cherny) for production-design patterns.
• incompleteThe conversation reaches the 'cover your ass' diagnosis of Norwegian public-sector risk-aversion but does not investigate the structural causes — why has the last ~10 years been 'feigere'? Datatilsynet stance is named, the Nikolai Tangen episode is named, but the institutional feedback loop producing the pattern is not unpacked.
  16) Inga Strümke om hvor AI er på vei, den outsourcede problemløsningen og datamanifolden — read next: John Cutler on institutional overload and accountability culture (already in the user's reading); recent NSM/Nkom assessments on resilience; Norwegian public-administration literature (NIBR / NIFU) on risk-distribution between politikere and direktorater; Annie Duke / Daniel Kahneman on outcome-bias driving risk-aversion.
• incompleteStrümke names two interesting categories — 'avklaringsspørsmål' (we have the methods, we just need agreement) and 'forskningsspørsmål' (genuinely open) — but doesn't extend the taxonomy. The third missing category is presumably 'capability-building-spørsmål' (we know what we need to build but lack the people / infra to build it) — which is arguably the dominant Norwegian bottleneck.
  16) Inga Strümke om hvor AI er på vei, den outsourcede problemløsningen og datamanifolden — read next: Donella Meadows on system leverage points (where in this taxonomy is the highest leverage); KI Norge mandate documents from Digdir; recent NTNU/SimulaMet hiring and capacity reports.
• uninformedStrümke is open about her own production-coding skepticism ('jeg merker når jeg sitter og koder med agentene, så er sånn tulleprosjektet'), and the hosts (Audun from NAV) push back gently with the more developed practitioner view that high-stakes work warrants Vim while front-end can be agent-led. Strümke's reading of agentic coding may underweight what's currently possible with proper harness, evals, and sandboxing — though she does flag the macOS sandbox feature.
  16) Inga Strümke om hvor AI er på vei, den outsourcede problemløsningen og datamanifolden — read next: User's existing thread on harness engineering (Anthropic videos, Boris Cherny, Bockeler/Fowler static sensors); Beane on Precision Proactivity; the user's earlier Heis Anders Sveen / Vilde Opsal extractions; Greger Wedel; concrete production-engineering reports from Builder.io, Cursor, and Claude Code teams.
• incompleteThe closing 'undetectable backdoors' research preview is fascinating but isolated from the rest of the conversation. If undetectable backdoors are plantable, the entire 'outsource problem-solving to Copilot' diagnosis acquires a much sharper national-security dimension that the conversation does not draw.
  16) Inga Strümke om hvor AI er på vei, den outsourcede problemløsningen og datamanifolden — read next: Goldwasser et al. on undetectable backdoors (the foundational paper); recent NeurIPS / ICML papers on neural-net backdoor detection; supply-chain security literature (SLSA, sigstore) extended to ML model artifacts.
• illogicalThe article cites a cited coding-agents study finding models default to unwanted *action* (making changes 35-65% of the time when the right move was to do nothing) to explain why, in build-something work, the bias runs toward doing something rather than nothing. But deferral pressure is itself a form of avoidance — declining to finish the actual outstanding work by relabeling it as no-longer-current. The piece doesn't reconcile why an 'action-biased' model would specifically favor a deferral (a stopping-adjacent move on the *current* item) rather than simply doing the work, and doesn't examine whether deferral is better understood as a distinct third category (relabeling-as-completion) rather than a variant of either the action-bias or omission-bias literature it cites.
  My AI Kept Pushing Me to Ship, So I Asked It Why — read next: Look for work that distinguishes 'doing nothing' from 'declaring something done without doing it' as separate failure modes, since the cited omission-bias and action-bias studies may not map cleanly onto deferral pressure at all.
• incompleteThe central operational claim (20M transactions, 18,000 tx/sec) and all three live demos run entirely on the presenter's own company's stack (Aletyx/Drools), with no independent benchmark and no comparison to alternative deterministic-layer approaches for the same class of problem — e.g., lightweight classical ML classifiers, which elsewhere in this corpus are argued to be cheaper and equally explainable for comparable tiered-classification decisions.
  Presentation: Decision Models in Agentic Architectures: From Production to Agent Skills — read next: extract-heis-16-inga-strumke-hvor-ai-er-paa-vei-datamanifolden — argues classical regression/classification ML models are tiny, deterministic, cheap, and explainable for exactly this kind of tiering problem, and criticizes organizations for skipping them in favor of heavier AI tooling.
• uninformedGovernance is framed almost entirely as a technical/architectural achievement (versioning, review, audit trail) once business and IT collaborate on the model — the talk does not engage with the organizational-accountability finding, documented elsewhere in this corpus, that even a fully auditable, deterministic automated decision still creates a human 'explain-this-to-the-customer' burden, and that technical explainability alone is insufficient without the surrounding organizational and relational structures.
  Presentation: Decision Models in Agentic Architectures: From Production to Agent Skills — read next: extract-when-employees-are-held-accountable-for-ai-generated-decisions — shows employees masking or reinterpreting even well-audited automated decisions when they lack the organizational structures to explain them credibly to the people affected.
• incompleteCausal direction is never established. Heavy AI users have lower psychological debt — but this could mean (a) using AI more reduces debt, (b) lower-debt employees use AI more, (c) both share an unobserved cause (e.g., role design, manager support). The training-program recommendation only makes sense under (a), which the data does not establish.
  The Psychological Costs of Adopting AI — read next: Longitudinal or quasi-experimental studies of AI adoption (e.g., the Brynjolfsson/Li/Raymond customer-service study); methodology critiques of HR-survey-based causal claims
• incompleteThe exemplar companies are all success stories with no failure cases. Klarna in particular publicly reversed parts of its AI-first strategy in 2024-2025 after customer service quality concerns; the article treats the 90% adoption number uncritically as proof of working culture rather than as a snapshot that didn't hold. Selection bias on the prescription side is severe.
  The Psychological Costs of Adopting AI — read next: Follow-up coverage of Klarna's AI strategy reversal; broader research on the gap between announced and sustained AI adoption rates
• uninformedThe six debts are presented as independent dimensions but are conceptually entangled. Competency debt is partly a felt consequence of cognitive debt; identity debt overlaps with autonomy debt when the threatened identity is built on autonomous expert judgment. The taxonomy may be a useful management heuristic but is not analytically clean, and the survey's 0–100 conversion may double-count.
  The Psychological Costs of Adopting AI — read next: The original Self-Determination Theory literature (Ryan & Deci) on how the basic needs interact rather than sum; latent-variable analyses of motivation constructs
• uninformedNo engagement with the asymmetry the survey itself surfaces: people excuse their own AI use but judge others'. This is a textbook 'fundamental attribution error' / actor-observer asymmetry, and the credibility-debt prescription (make use visible) assumes a social-norms mechanism that the underlying psychology may resist. The author does not engage with whether visibility actually shifts the attribution or just relocates it.
  The Psychological Costs of Adopting AI — read next: Classic social psychology on actor-observer asymmetry (Jones & Nisbett); norm-shifting interventions in moral psychology
• incompleteConflict of interest is disclosed but its implications are unexamined. The survey was commissioned for Meltwater Consulting (where the author is a director) and underwrites a consulting offer. The framing — that companies need expertly-designed adoption strategies — directly supports the consulting product. The article treats the data as objective evidence rather than as evidence shaped by a commercial frame.
  The Psychological Costs of Adopting AI — read next: Methodology of practitioner-commissioned research; meta-analyses comparing commissioned to independent organizational research
• incompleteThe 'human infrastructure' closing claim is large but underspecified. Each prescription is a single-company anecdote; there is no theory of which mitigation pattern applies to which combination of debts, no sequencing guidance, and no analysis of how the patterns interact (e.g., does cognitive friction reduce or increase autonomy debt?). The article ends as a framework prospectus rather than as a developed framework.
  The Psychological Costs of Adopting AI — read next: Work on socio-technical systems design (Trist, Pasmore); change-management research on layered intervention design
• incompleteThe article never explains why the biotech company invested in the structures that enabled the 'complement' response (dedicated imaging lab, weekly developer meetings, funded joint research) while the bank and recruiting firm did not build comparable structures. Is the difference driven by decision stakes, regulatory pressure, per-decision cost, or organizational culture?
  When Employees Are Held Accountable for AI-Generated Decisions — read next: Literature on why organizations invest differently in human-AI collaboration design — regulatory-stakes and decision-criticality frameworks (e.g., EU AI Act risk tiers) and studies of resourcing decisions in AI deployment.
• incompleteNo discussion of cost or scale feasibility: the biotech-style organizational investment (dedicated lab, weekly meetings, funded research) may not be economically viable in high-volume, thin-margin decision contexts like consumer lending or mass recruiting, where the 'complement' response may simply not be affordable regardless of leadership intent.
  When Employees Are Held Accountable for AI-Generated Decisions — read next: Operations and cost-of-service literature on human-AI collaboration at scale; comparisons of per-decision cost across banking, recruiting, and biotech contexts.
• uninformedThe article does not engage the regulatory dimension of the banking case at all: in consumer lending, banks in many jurisdictions have legal obligations (e.g., US Regulation B adverse-action notices, EU consumer-credit rules, and the EU AI Act's explanation rights for high-risk AI) to provide accurate, specific reasons for credit denials. Masking the AI's actual rationale with plausible but inaccurate reasons could carry compliance and legal risk the article never mentions.
  When Employees Are Held Accountable for AI-Generated Decisions — read next: US Regulation B / ECOA adverse-action-notice requirements and case law; EU AI Act Article 86 right to explanation for high-risk AI systems.