Precision Proactivity: Measuring Cognitive Load in Real-World AI-Assisted Work

Brandon Lepine; Juho Kim; Pamela Mishkin; Matthew Beane · pdf · 2026-03-10 Reader 🖥️

theoretical · mixed — Primarily theoretical (extending Cognitive Load Theory into conversational AI as a state-dependent, path-dependent property) layered on top of expository methodology (transcript-based computational measurement framework) and quantitative empirical reporting. Has a clear practical undertone: the paper closes with three design implications for 'precision proactivity'. Argumentative in pushing against a simple 'AI augmentation is positive' narrative and against treating proactive interventions as discrete, evaluable events.

Structure

Key terms

• Precision Proactivity Cognitive Load
  Authors' own framing: 'systems that can detect entrenched high-load trajectories, constrain scope expansion, and actively reimpose structure to dampen rather than perpetuate cognitive burden'. Three operational principles: (1) detect/respond to elevated cognitive load (treat high-load prompts as signals for structural support, not opportunities for maximal elaboration); (2) constrain system-initiated scope expansion via subtask focus stack + explicit scope elicitation; (3) enable structured disclosure via user-directed navigation. The 'precision' is about WHEN to intervene and WHAT scope to maintain — not about model output accuracy.
• Cognitive Load (intrinsic / extraneous) Cognitive Load
  Adopted from Sweller's classical Cognitive Load Theory. ICL: complexity inherent to task itself, arising from element interactivity. ECL: arises from presentation of information — avoidable working-memory burden imposed by information poorly structured, mistimed, redundant, or misaligned with goals. CRITICAL terminological note: the authors operationalize these as computational estimates derived from conversational transcripts, NOT physiological or self-reported states — 'they should be understood as theory-grounded proxies — not direct measurements of neural or subjective load'.
• Prompt-anchored Subtask Coherence (PSC) / EL-PSC
  Primary measure of extraneous cognitive load at participant-subtask level. Bidirectionally assesses whether subtasks invoked by user are addressed by system AND whether proactive content connects back to user goals. Anchored against 101-subtask decomposition + FinanceKG knowledge graph. Reported as complement of coherence so higher = more burden.
• Task switching (LLM behavior) Cognitive Load
  Partial interruptions in which attention is redirected to a different subtask without fully abandoning the original goal — specifically when the system proactively introduces subtasks the user did not request, or when conversational focus oscillates across subtasks. The single most damaging behavioral feature (β=−0.271, p<0.01 in joint models). The paper's signature operational finding: it's not depth that hurts, it's unsolicited scope expansion.
• AI-Generated Content Usage (AIGCU) Measuring AI Impact
  Content uptake: degree to which model-generated material appears in participants' final valuation reports. Two components: coverage (fraction of report attributable to model responses) + structural continuity (whether matches appear in coherent runs vs scattered fragments), combined via PCA. Critically measures incorporation regardless of objective accuracy — what matters is whether participants USED the content and how it correlates with expert quality grades.
• Within-speaker persistence / path-dependent load trajectories
  Headline RQ2 finding: EL in human-LLM dialogue is dominated by within-speaker momentum, not cross-speaker propagation. prompt→prompt β=0.268, response→response β=0.170 (both p<0.001). Once either speaker enters a high-load pattern it tends to persist; the interaction provides weak endogenous mechanisms for restoring coherence.
• Contextual inertia (users') Context Rot & Context Limits
  User-side counterpart to LLMs getting lost in multi-turn. 'A growing reluctance to reset a conversation because accumulated context carries both informational value and coordination costs.' Preserving context avoids restating objectives but increases the likelihood that responses reflect organizational patterns that increase extraneous load over time.
• Expertise reversal effect
  From Kalyuga et al. (2003): instructional supports beneficial for novices can become redundant or counterproductive for experts. Used here to argue that uniformly dampening output for all users risks imposing unnecessary simplification on experts.

Propositions

• empirical findingAI-generated content usage (AIGCU) is positively associated with output quality: β≈0.27 to 0.35, p<0.01 across mixed-effects specifications (N=1,178 obs, N=33 participants).
• empirical findingExtraneous cognitive load (EL-PSC) exhibits the largest negative association with output quality: β≈−0.47 to −0.49, p<0.01 — roughly 3× the magnitude of intrinsic load.
• empirical findingIntrinsic cognitive load (ICL) has a smaller but reliable negative association with output quality: β≈−0.15, p<0.01.
• empirical findingAIGCU×EL and AIGCU×ICL interactions are consistently null — no evidence that AI usage moderates or buffers the marginal load-quality relationship. Pattern is additive, not multiplicative.
• empirical finding (moderation)Experience moderates the EL-quality association: less experienced professionals (below-median, N=657 obs, 20 participants) show EL→quality β=−0.607 (p<0.001) versus β=−0.220 (n.s.) for more experienced (N=521, 13 participants). Joint specification: Experience×EL β=0.208 (p<0.01).
• empirical finding (moderation)Less experienced professionals derive larger marginal quality gains per unit of AIGCU (b-path β=0.618 at −1SD experience vs 0.192 at +1SD; Experience×AIGCU β=−0.159, p<0.01).
• empirical finding (moderated mediation)More experienced professionals increase AIGCU uptake more sharply as extraneous load rises (a-path β=0.896 at +1SD vs 0.308 at −1SD; moderation β=0.294, p<0.001). The dissociation: those who'd benefit most don't uptake more under load.
• empirical finding (mediation)Mediation pathway through AIGCU attenuates ~37% of intrinsic-load penalty (0.060/0.164) and ~29% of extraneous-load penalty (0.222/0.762) — partial but incomplete compensation.
• empirical findingWithin-speaker persistence dominates utterance-level extraneous load: prompt→prompt β=0.268 (p<0.001), response→response β=0.170 (p<0.001).
• empirical findingCross-speaker spillover is asymmetric and modest: prompt→response β=0.089 (p<0.001); response→prompt β=0.015 (n.s.). Apparent contemporaneous speaker asymmetry collapses to β=−0.041 (n.s.) after AR(2) controls.
• empirical findingTask switching is the strongest behavioral predictor of quality decline (β=−0.271, p<0.01 in joint models), absorbing the effects of information sprawl and truncation when modeled together.
• empirical findingAsymmetric prompt-vs-response structural effects: prompt-side element interactivity is POSITIVELY associated with quality (β=0.166, p<0.01); response-side is not. Prompt-side dependency debt is negatively associated (β=−0.185, p<0.05); response-side is not. Disruption hurts when it interferes with the user's active task representation, not when the response is internally complex.
• empirical finding (incidental)Professional experience (years) has a negative direct association with output quality in controlled models: β≈−0.45 to −0.53 (p<0.01). The paper does not interpret this counterintuitive finding but it is stable across specifications.
• empirical findingContent usage exhibits direct persistence across consecutive responses (β=0.115, p=0.004) — AIGCU follows its own temporal trajectory independently of load dynamics.
• theoretical contributionExtraneous load in human-LLM interaction is not a static property of information presentation at any single moment but a path-dependent, jointly constructed feature of the interaction that evolves with each speaker's own momentum.
• theoretical contributionProactivity should be reconceptualized as a state-dependent phenomenon rather than a sequence of isolated, independently evaluable system actions — its effects depend on the cognitive trajectory it enters and reinforces or disrupts.
• theoretical contributionTask-structure alignment is the central theoretical mediator of when AI assistance helps vs harms: AI supports performance when it deepens/elaborates within an existing task schema, but imposes extraneous load when it forces users to reorganize that schema mid-stream.
• theoretical contributionUsers may exhibit 'contextual inertia' — reluctance to reset a conversation because accumulated context carries both informational value and coordination costs, increasing the likelihood that responses reflect organizational patterns that increase extraneous load over time.
• theoretical interpretationCurrent RLHF objectives reward relevance/responsiveness to the immediate prompt in ways that produce models which mirror and elaborate user organizational structure rather than restructuring or simplifying it — creating a self-reinforcing cycle.
• methodological claimCognitive load can be estimated from conversational transcripts at utterance-level resolution using theory-grounded computational indicators: task decomposition for intrinsic load, prompt-anchored subtask coherence (PSC) augmented with a domain-specific knowledge graph for extraneous load.
• methodological claimTranscript-based measurement addresses three limitations of NASA-TLX/dual-task paradigms in naturalistic multi-hour AI-assisted work: differentiating load by speaker, tracking moment-to-moment evolution, avoiding ecological intrusion.
• methodological claimThe framework is portable across domains but requires rebuilding domain-specific components (task decomposition, knowledge graph) for each new domain.
• methodological claim (robustness)Direction and significance of EL→quality, ICL→quality, and AIGCU→quality associations are robust across 86 parameter combinations in a specification curve analysis spanning subtask-memory dynamics, anchoring thresholds, coherence weights, and attribution hyperparameters.
• design implicationProactive systems should detect conversational signals of elevated cognitive load and respond by dampening — prioritizing simplification, sequencing, clarification — rather than amplifying through maximal elaboration.
• design implicationSystems should constrain system-initiated scope expansion by default. Architectural pattern: a subtask focus stack inferred from recent user turns; out-of-scope candidate content triggers boundary-signaling mode (tight in-scope answer + optional deferred topics). Complementary pattern: explicit scope elicitation via clarifying questions before generation.
• design implicationResponses should be structured as navigable information architectures with progressive disclosure (expandable sections, in-line annotations, branching) so depth coexists with focused presentation — decoupling availability from imposition.
• design implicationLoad-dampening interventions must be calibrated to user expertise but dynamically, because the expertise reversal effect means novice-helpful scaffolding becomes counterproductive for experts.
• design implication / future workBecause extraneous load impairs not only immediate performance but schema acquisition (CLT's original prediction), self-sustaining load patterns may impair the learning that would eventually reduce dependence on AI — affecting durable skill formation. This warrants longitudinal investigation.
• implication / reframeFindings refine prevailing narratives about AI augmentation: AI assistance operates ALONGSIDE — not in interaction with — the cognitive conditions of the conversation. The determining factor in mixed prior results may be parallel accumulation of extraneous processing demands, not the presence of assistance per se.

Arguments

Narratives

• Figure 1 paper opener: turn-level annotation of a user-LLM conversation showing intrinsic and extraneous load estimated for each prompt-response pair, with interaction-level indicators of structural misalignment (task switching, sprawl, phase spread).
  → Operationalizes the measurement framework visually before technical detail. Anchors what 'cognitive load looks like in a transcript' for readers who'd expect dual-task or physiological data. Implicit claim: cognitive burden is legible in dialogue structure itself.
• The 34-participant valuation task: finance professionals (median 4 years experience) given a fictional tire manufacturer in merger talks with synthetic financial data (to evade GPT-4o training), instructed to complete a valuation in ≤10 hours, paid $75/hour, using ChatGPT GPT-4o (v2024-04-01) freely, producing a final report graded by independent experts (median 18 years) on a 101-subtask rubric.
  → Embodies 'real-world AI-assisted work' as a specific operationalization. Time pressure + professional task + expert grading = ecological-validity claim. Implicit claim: what happens here generalizes meaningfully to knowledge work broadly (though limitations qualify).
• Interpretive vignette in §7.2: 'The model does not reorganize a user's sprawling prompt into a more coherent response, and the user does not restructure their approach in response to model output. Each party continues along its own trajectory.'
  → Translates the within-speaker persistence statistics into an interactional image. Implicit claim: human-LLM dialogue lacks the grounding adjustments human dialogue uses to recover from misalignment (Clark & Brennan 1991) — it is structurally weaker as a coordination medium.
• Figure 2 computational pipeline: Top — FinanceKG construction (document collection → concept extraction → embedding → graph). Middle — participant-subtask pipeline computing utterance-level IL, EL, and behavioral features. Bottom — attribution pipeline estimating AIGCU via capacitated thresholded similarity matching of transcript sentences to report sentences.
  → Makes the measurement pipeline auditable. Implicit claim: methodological rigor compensates for the absence of physiological/self-report ground truth — the construct validity argument rests on transparent pipeline construction plus specification curves.

Questions addressed

• ✅ RQ1: How do extraneous cognitive load dynamics from LLM interactions affect users' ability to benefit from AI-generated content?
  Additively, not multiplicatively — both AIGCU and load matter independently and the interaction terms are null. EL ~3× ICL in magnitude.
• ✅ RQ2: How do cognitive load dynamics propagate between users and proactive LLMs during extended task-oriented conversations?
  Dominated by within-speaker persistence; the only reliable cross-speaker channel runs from prompts to responses. Apparent contemporaneous role asymmetry is autoregressive accumulation, not inherent to speaker role.
• ✅ RQ3: What specific LLM behaviors and dynamics contribute most to the effects of extraneous cognitive load during task completion?
  Unsolicited task switching is the dominant behavioral predictor; sprawl and truncation are absorbed once switching is controlled. Disruption to the user's active task representation matters more than response-side complexity.
• ✅ Does professional experience moderate these load and AIGCU dynamics?
  Novices carry larger direct EL penalty and derive larger marginal AIGCU benefit; experts uptake more AI under load but extract less per unit. Skewed moderation patterns establish 'precision proactivity must calibrate to expertise'.
• ⭕ Do transcript-based load indicators converge with NASA-TLX, secondary-task performance, or physiological measures?
  Acknowledged as future-work triangulation; only readability/text-complexity convergent validity provided.
• ⭕ Do findings generalize beyond financial valuation, beyond N=34, beyond GPT-4o?
  Explicitly flagged as a limitation. Authors speculate dynamics may intensify with more capable models (citing Storey 2026 on cognitive debt and Ranganathan & Ye 2026 HBR).
• ⭕ Are observed associations causal?
  Observational design precludes causal claims; the authors are explicit about this throughout, framing everything as 'associated with'.
• ⭕ Do precision-proactivity design principles serve durable skill formation, not just immediate performance?
  Proposed as longitudinal study target — most productive incomplete line, given Beane's prior work on skill acquisition.
• ⭕ Can the broad bundle of 'proactive behaviors' be parsed into more precise sub-behaviors via controlled experimental manipulation?
  Future-work direction explicitly flagged in §8.
• ⭕ Is 'precision' always normatively right — or is there value in imprecise exploratory proactivity that surfaces ideas the user wouldn't have queried?
  Not engaged. The paper conflates 'precision' with 'good' without considering serendipitous surfacing or desirable difficulty as productive categories.
• ⭕ What are the cost, latency, and vendor-economic implications of subtask focus stacks, explicit scope elicitation, and navigable architectures?
  Entirely absent. Typical HCI blind spot but load-bearing because 'restrict the rate of information transmission' is a deliberate worse-is-better UX move.
• ⭕ What are the privacy and surveillance implications of real-time modeling of user cognitive state and expertise to calibrate proactivity?
  Not engaged. Notable absence given Mishkin's policy background.
• ⭕ How does intra-dialogue extraneous load combine with ambient attentional load (notifications, multitasking, fragmented focus)?
  Not engaged. Cognitive load is treated as task-internal and conversation-internal — the broader attention infrastructure (Bailey, Hyperfocus, Stolen Focus) is not part of the frame.
• ⭕ Why does professional experience have a NEGATIVE direct association with output quality (β≈−0.45 to −0.53)?
  Passing observation in the tables, never interpreted. Could reflect Upwork selection effects, the specific valuation task, or generational differences in finance training.

Gaps

• replication and ecological generalizationN=34 finance professionals on one 10-hour valuation task with GPT-4o. Whether within-speaker persistence and task-switching dominance generalize to software engineering, healthcare, legal, or shorter sub-hour tasks is open. The paper's portability claim (rebuild task decomposition and knowledge graph) is undemonstrated outside finance.
  Read next: Pu et al. 2025 'Assistance or disruption?' on proactive programming assistants (cited ref. 26) — closest existing analog in software domain. Also Kazemitabaar et al. 2024 (ref. 61) on interactive task decomposition in data analysis.
• normative-direction assumption'Precision proactivity' assumes calibrated scope-bounded proactivity is normatively superior to indiscriminate proactivity. But CLT has a 'desirable difficulty' tradition (intentional friction supporting learning), and serendipitous unsolicited surfacing — exploration that helps users find what they wouldn't have queried — is a real category. The paper conflates 'precision' with 'good' without engaging the exploratory case.
  Read next: van Merriënboer & Sweller 2005 on germane load (cited ref. 125) — the paper's exclusion of germane load from its operationalization is itself a theoretical commitment worth examining.
• construct validity vs alternative load measuresThe paper relies entirely on transcript-derived computational proxies. NASA-TLX, dual-task secondary performance, physiological measures (fNIRS, pupillometry) are listed as future-work triangulation. The 3× EL:ICL ratio is a strong claim resting on whether the two composites have comparable measurement reliability.
  Read next: Babaei et al. 2025 on NASA-TLX limitations (cited ref. 54); Hirshfield et al. 2024 on fNIRS for cognitive load in multiple resources (cited ref. 69).
• cost / latency / model-economics blind spotSubtask focus stacks, explicit scope elicitation, and navigable architectures all imply additional compute, latency, and interaction friction. The paper does not engage with vendor cost models, latency-vs-quality tradeoffs, or the operational realities of deploying precision-proactive systems.
  Read next: Xiao & Yang 2025 'Streaming, fast and slow: cognitive load-aware streaming for efficient LLM serving' (cited ref. 154) — the only operational-economics paper cited.
• attention infrastructure absent (hidden-layer thread)The paper treats cognitive load as intra-dialogue. It does not engage with the broader attention environment users bring to the work (notifications, multitasking, fragmented focus — Bailey, Stolen Focus, Hyperfocus). Extraneous load in the conversation arrives on top of ambient extraneous load in the environment, and the paper's instruments cannot see the latter.
  Read next: Mark, Gudith & Klocke 2008 'cost of interrupted work' (cited ref. 36) is used only for intra-dialogue framing — Chris Bailey's Hyperfocus, Stolen Focus, and the broader attention-infrastructure literature would be the right complement.
• skill acquisition under sustained AI use (Beane signature)The paper raises in §7.4.1 the long-term schema-acquisition concern: 'if the self-sustaining load patterns we document prevent novice users from constructing coherent task schemas during AI-assisted work, the long-term consequence may extend beyond diminished output quality to impaired skill development.' Proposes longitudinal study as future work but does not test. Given Beane is co-author and The Skill Code's central concern is preserving conditions for skill acquisition, this is the most productive incomplete line in the paper. The one-shot 10-hour design is structurally unable to address it.
  Read next: Matthew Beane, The Skill Code; van Merriënboer & Sweller 2005 on cognitive load and complex learning (cited ref. 125).
• governance / policy framing latentMishkin's policy lens does not surface — proactivity is treated as a design problem, not a policy or organizational-adoption problem. The 'precision proactivity' implications could be read as soft regulatory guidance for high-stakes professional contexts but the paper does not develop this. The latent policy angle (should regulators require scope-elicitation in high-stakes professional AI?) is not posed.
  Read next: Appel et al. 2026 Anthropic Economic Index report (cited ref. 52) for the policy framing Mishkin and co-authors have produced elsewhere.
• RLHF interpretation as mechanism vs analogyThe Discussion proposes that prompt→response spillover reflects RLHF optimizing for relevance and helpfulness. This is plausible but mechanistic only by analogy — the paper does not actually instrument the model or correlate with known RLHF training details. Newer post-training paradigms (Constitutional AI, RLAIF, weighted preference learning, reasoning models) may produce different mirroring behaviors.
  Read next: Examine whether reasoning-trained models (o-series, Claude 3.7+ extended thinking, GPT-5 reasoning modes) exhibit different prompt-mirroring spillover under the same measurement framework. Stiennon et al. 2020 and Ouyang et al. 2022 (refs. 110-111) are the cited RLHF foundations.
• extension vs relabel of classical CLTThe paper claims to extend CLT by reframing extraneous load as dynamic and path-dependent. But classical CLT has dynamic formulations (effective intrinsic load varies with expertise via schema automation — Kalyuga et al. 2003 expertise reversal). The contribution is sharper if read as: importing CLT's static-design-feature framing into the dynamic conversational setting required computational instruments classical CLT didn't have, which then revealed the static framing was insufficient. Whether this is 'extension' or 'translation' is not foregrounded.
  Read next: Sweller 1994 (ref. 6); Kalyuga et al. 2003 expertise reversal (ref. 124); van Merriënboer & Sweller 2005 (ref. 125).
• harness-engineering syntopical bridgeThe paper does not engage with the agent-harness literature (compaction, sub-agents, context anxiety) but its within-speaker persistence finding is precisely the empirical grounding that literature needs. Each compaction event in a harness can be read as an attempt to break the self-sustaining load trajectory documented here.
  Read next: Anthropic harness paper on long-running application development; Turing Post 5 patterns for long-running agents; Cloudflare compaction-loss anecdotes.

Related documents

• The productivity paradox of AI coding assistants
  Context Rot & Context Limits Measuring AI Impact
• The Acceleration Whiplash
  Cognitive Load Measuring AI Impact
• How to measure AI developer productivity in 2025 | Nicole Forsgren
  Cognitive Load Measuring AI Impact