To Adopt AI at Scale, Employees Need to Trust Agents

Thomas McKinlay, Stefano Puntoni, Serkan Saka · article · 2026-09-09 source ↗ Reader 🖥️

Agentic AI Calibrated Trust (in AI Agents)Enterprise AI Adoption & Readiness Explainable AI Human-in-the-Loop Trust

Because employees' willingness to grant AI agents real autonomy is governed by trust rather than technical capability, organizations can deliberately engineer calibrated (not blind) trust by being transparent about an agent's limitations, projecting competence over friendliness, tying its actions to users' broader goals, framing it as a subordinate helper rather than a powerful entity, and preserving human control over consequential decisions.

practical · mixed — A practical, action-oriented piece (five concrete techniques for managers) built on a narrative hook, quantitative evidence, and cited experimental findings — expository/argumentative in service of an implied 'you should do X' rather than a theoretical claim about what trust is.

Structure

Part	Role
Opening scenario: a manager confronted with an AI agent's permission requests denies them, the agent then gets stuck on trivial tasks, and the manager abandons it	narrative hook that makes the cost of low trust concrete and relatable before any data is introduced
Enterprise adoption statistics (59% of enterprises say they use agentic AI but only 9% have autonomous workflows; only 27% of middle managers see real ROI; ~$202M average planned AI investment)	quantifies the scale and cost of the trust gap at the industry level
A study of over 3,000 people adopting an AI financial adviser, breaking down adoption concerns by category (trust-related concerns dominant; technological performance second; privacy 31%, fear of unwanted actions 23%, incomprehension 11%)	evidence that trust specifically, not raw capability, is the binding constraint
Methodology statement: a review of human-AI interaction research plus interviews with Wharton researchers and practitioners at Google, ServiceNow and Zapier, published as the Wharton Blueprint for AI Agent Adoption	establishes the evidentiary authority behind the five techniques that follow
Five named techniques, each paired with a quantified experimental finding and a named practitioner example (Concentrix, Cleo, Microsoft Copilot, ServiceNow, Workato)	the practical core of the piece — each technique is an independently argued, actionable lever
Closing reframe: the goal is calibrated trust, not blind trust; excessive trust and insufficient trust are both named as failure modes	conclusion that bounds and qualifies the whole argument against over-application

Key terms

calibrated trust Calibrated Trust (in AI Agents)
An appropriately proportioned level of trust in which employees understand exactly where an agent is capable, where it is likely to fail, and when they retain control — the explicit design target, contrasted with both blind trust and blanket distrust.
moderate autonomy
A specific middle level of agent independence — more than minimal (fully human-gated, high-friction) but less than total (fully autonomous, unsupervised) — empirically associated with successful adoption.
control tower
ServiceNow's specific governance layer for agent workflows: guardrails set in advance, real-time monitoring of agent activity, and the ability to override or intervene — a concrete architecture that operationalizes 'preserved human control' without eliminating agent autonomy.
glorified chatbots
AI agents that are technically capable of autonomous, multi-step action but get used only for narrow, low-stakes, conversational tasks because employees don't trust them enough to grant real autonomy — the specific failure mode all five techniques exist to prevent.

Propositions

explicitTrust, not technical performance, is the central obstacle to employees granting AI agents meaningful autonomy at work.conf: high
explicitMost enterprises report using agentic AI, but very few have converted that into genuine autonomous workflows (59% claim usage vs. 9% with autonomous workflows), and only a minority of middle managers report real ROI (27%) despite heavy planned investment (~$202M).conf: high
explicitIn a large study of AI financial-adviser adoption, trust-related concerns (privacy, fear of unwanted actions, incomprehension of the AI's actions) outweighed concerns about the AI's technological performance.conf: high
supportsThe Hidden Realities of AI Adoption · p9 — Both identify trust (not raw capability or accuracy) as the leading determinant of AI-adoption success.
explicitBeing upfront about an AI agent's specific limitations increases how transparent it is perceived to be (up to 14.6% more) and how effectively people work with it (up to 7.2% more), even when the underlying AI is identical.conf: high
refinesWhen Employees Are Held Accountable for AI-Generated Decisions · p9 — This article shows disclosing limitations raises interpersonal trust/effectiveness; the other piece argues technical explainability alone is insufficient for the deeper accountability problem — together they suggest transparency helps trust but doesn't resolve who bears justificatory responsibility for an agent's decisions.
explicitMost mainstream AI tools (e.g., Gemini, Claude) only provide generic, blanket error disclaimers rather than task-specific failure warnings, leaving individual organizations to build their own internal guidance.conf: high
explicitPeople are less willing to use an AI agent that comes across as friendly and warm than one that comes across as competent.conf: high
explicitAn agent projects competence by explaining what it did and why it made the choices it did, rather than by simply producing outputs.conf: medium
explicitEmployees accept an AI agent's recommendations substantially more often (54% more) when the agent explicitly ties its recommendation to the employee's broader, longer-term goals rather than surfacing an isolated suggestion.conf: high
explicitFraming an AI agent as a subordinate 'helper' rather than a powerful, autonomous, decision-making entity reduces perceived risk; reminding people that the agent has real power (e.g. acting on their behalf) increases perceived risk above that of an equivalent human performing the same task.conf: high
refinesPodcast: Will Agentic AI Bring Fantasia's Sorcerer's Apprentice to Life?: A Conversation with Tracy Bannon · p5 — This article assumes agent trust is built through framing/design; Bannon's point complicates that by showing some agent trust is pre-existing institutional trust inherited from the underlying platform, not newly earned through interaction design.
explicitcorroboratedSuccessful agent adoption depends on keeping autonomy at a moderate level — neither minimal nor total — supported by mechanisms that let humans set guardrails in advance, monitor activity, and override agent decisions.conf: high
extendsYour AI agent just blamed the network team. Now what? · p11 — This article's single 'moderate autonomy via control tower' prescription is a compressed version of that piece's fuller three-phase autonomy-expansion model (shadow mode -> read-only human-in-the-loop -> limited automated response).
supports ←Presentation: Decision Models in Agentic Architectures: From Production to Agent Skills · p4 — Porcelli's business-owned, IT-governed decision model is a concrete instance of the 'moderate autonomy with guardrails set in advance, monitoring, and override' design the trust-adoption research prescribes.
explicitcorroboratedAgents should be required to ask for confirmation before any consequential action (e.g., sending an email, executing a payment or inventory move) rather than being trusted with full autonomous execution by default.conf: high
supports ←Podcast: Securing AI Agents: Identity, Authorization, and the DPACT Framework · p7 — Both independently converge on requiring explicit human confirmation before an agent takes a consequential, hard-to-reverse action.
explicitdevelopedThe design goal is calibrated trust — employees understanding where an agent is capable, where it will likely fail, and when they remain in control — not blind or maximal trust.conf: high
extends ←How Leaders Can Use AI to Solve Real Business Problems · p5 — Supplies a concrete before/after mechanism (unexplained flags ignored, explained flags acted on, mortality fell 41%) for the abstract 'calibrated trust' design goal.
extends ←Podcast: Securing AI Agents: Identity, Authorization, and the DPACT Framework · p14 — Agarwal frames the same 'calibrated trust' design goal specifically as an infrastructure/identity failure mode, not primarily a UX or communication failure as the trust-adoption piece emphasizes.
implicitcorroboratedBoth trust deficits (avoidance, treating agents as 'glorified chatbots') and trust excesses (consequential errors from unchecked autonomy) are costs of the same underlying problem — trust miscalibration — rather than opposite, independent risks.conf: medium
supports ←Study finds AI assistants help developers produce code that's more likely to be buggy · p2 — This study's 'AI assistance makes developers more confident in objectively worse code' is a concrete, early (2022) empirical instance of the 'trust excess' half of the later essay's general trust-miscalibration framework.
implicitThe five trust-building techniques generalize across AI agent types and organizational contexts (finance, customer service, engineering, HR) without meaningful boundary conditions.conf: medium

Arguments

Organizations should proactively disclose an agent's specific, task-level failure modes rather than relying on the AI vendor's generic error disclaimers. strength: medium

Explicit premises:

People informed of an AI's known weaknesses trust it more and work with it more effectively than people who aren't told, holding the AI constant.
Mainstream AI tools only provide blanket, non-specific disclaimers about possible errors.

Implicit premises:

In the absence of specific failure information, employees default to maximal uncertainty/distrust rather than assuming competence (ambiguity aversion).
Employees who are told where an agent is likely to fail will reliably act on that information (e.g., double-checking output) rather than becoming fatalistic or ignoring the warning.

Inferences:

Because major vendors do not supply task-specific failure guidance, the burden — and the associated liability if it's done poorly — falls on the deploying organization, not the AI vendor.

Agents should be designed and framed as competent-but-subordinate helpers, not friendly, autonomous, or powerful entities. strength: medium

Explicit premises:

Competence beats friendliness/warmth in driving willingness to use an AI.
Framing an agent's actions as 'assisting' rather than 'deciding' measurably reduces perceived privacy/power risk.
Microsoft's 'Copilot' naming is cited as a successful instance of this framing.

Implicit premises:

Perceived warmth/friendliness and perceived competence trade off against each other rather than varying independently, so an agent cannot easily be both.
Framing an agent as a 'colleague' (used to motivate technique 1's disclosure of limitations) can be done without implying the peer-level status or power that technique 4 explicitly warns against — the piece never explains how to invoke colleague-like candor without colleague-like status.

Human control over consequential actions must be preserved through confirmation gates and continuous monitoring ('control tower'), with agent autonomy kept moderate rather than maximized. strength: medium

Explicit premises:

Moderate autonomy is empirically associated with successful adoption; neither minimal nor total autonomy is.
ServiceNow's control-tower pattern (advance guardrails, monitoring, override) is offered as a working example.
Workato's CTO describes explicit action-level permission splits ('draft but not send').

Implicit premises:

The correct threshold for what counts as a 'consequential' action requiring confirmation is knowable and can be set consistently in advance — the piece never explains how an organization should determine that threshold.
Adding confirmation gates does not meaningfully erode the productivity gains that justified deploying an autonomous agent in the first place.

Inferences:

Because confirmation gates constrain exactly the kind of end-to-end automation the article's own opening statistic says only 9% of enterprises have achieved, the fifth technique is in some tension with closing the article's own headline adoption gap — an inference the authors don't draw or address.

Narratives

A manager sets up a new AI agent 'assistant,' is confronted with broad permission requests (file access, email, payments), denies them out of discomfort, then finds the crippled agent constantly getting stuck and abandons it, concluding 'I could never make my team use this.'
→ Implicitly supports the unity by making the mechanism concrete: distrust leads to permission denial, which leads to a crippled agent, which confirms and deepens the initial distrust — a vicious cycle the rest of the article's techniques are meant to break.
Cleo, a personal-finance assistant, ties every recommendation to the user's daily goal and long-term financial roadmap instead of surfacing isolated suggestions.
→ Supports proposition p8 (goal-alignment increases recommendation acceptance) by giving a concrete, named product example of the abstract technique.
Microsoft's choice of the name 'Copilot' rather than language like 'AI can take this over for you.'
→ Supports proposition p9 (subordinate/helper framing reduces perceived risk) via a well-known branding example.

Questions addressed

✅ Why does trust act as the central obstacle to agentic AI adoption, rather than the AI's raw technical capability?
Cited survey/study evidence (the 3,000-person financial-adviser study) shows trust-related concerns outweigh performance concerns among adopters.
✅ What concrete, actionable techniques can managers use to build appropriate trust in AI agents?
Five named techniques, each backed by a cited experimental finding and a practitioner example.
⭕ How should an organization tell whether trust in a given agent is correctly calibrated (versus still too low, or already too high)?
The article names calibrated trust as the goal and names both under-trust and over-trust as failure modes, but gives no measurement, test, or diagnostic for where an organization currently sits, or when it has reached calibration.
✅ Why do so few organizations progress from experimenting with agentic AI to genuine autonomous workflows (the 59%-vs-9% gap cited at the top)?
The article implicitly treats this gap as a trust problem, resolved by the five techniques — but never explicitly reconnects the gap statistic to the techniques at the end, and doesn't consider or rule out non-trust explanations (e.g., organizational readiness/legibility, process design) that the corpus elsewhere treats as a distinct, possibly more fundamental blocker.

Gaps

illogicalThe article's two central metaphors for the agent are in unaddressed tension: technique 1 ("think of the agent as a new colleague who's open about which parts of the job they're still learning") implies peer-like candor and status, while technique 4 explicitly warns against exactly that ("make sure your team knows your agent is just a tool," avoiding language that implies it is an equal or powerful entity). The piece never explains how an agent discloses limitations with colleague-like openness without triggering the peer-status framing technique 4 says to avoid.
Read next: Look for HCI/anthropomorphism research on how self-disclosure of limitations and perceived-status framing interact — whether a 'tool' can disclose weaknesses without being perceived as gaining agency or intentions.
incompleteThe article gives no guidance on how to define or calibrate which actions count as 'consequential' enough to require human confirmation, even though that exact threshold sits at the crux of both the opening statistic (only 9% of enterprises have turned agentic-AI usage into autonomous workflows) and the closing prescription (moderate autonomy, confirmation gates).
Read next: extract-the-unsexy-truth-of-ai-adoption argues the harder blocker is organizational legibility (the company can't describe its own workflows), not trust per se; extract-ai-agent-blame-network-team offers a concrete three-phase framework (shadow mode -> read-only human-in-the-loop -> limited automated response) for expanding autonomy in defined steps, which is a more operational version of this article's undeveloped 'moderate autonomy' prescription.
incompleteThe five techniques are illustrated with single-company anecdotes (Cleo, ServiceNow, Workato, Concentrix, Microsoft) chosen by authors who co-produced the underlying Wharton report; the article never discusses contexts where these techniques might trade off against speed or reintroduce friction — e.g., confirmation-gate-heavy designs risk recreating exactly the 'agent keeps getting stuck' frustration described in the opening scenario.
Read next: extract-psychological-costs-of-adopting-ai catalogs competing costs (approval fatigue, autonomy debt) that confirmation-heavy, trust-preserving designs could themselves generate — a tension this article doesn't engage with.
incompleteThe article treats trust in an agent as something built fresh through interaction design (transparency, framing, control), but doesn't engage with the possibility that much real-world agent trust is inherited from pre-existing institutional/vendor relationships rather than earned through the agent's own behavior.
Read next: extract-agentic-ai-sorcerers-apprentice-tracy-bannon argues hyperscalers (Google, Microsoft) don't need to newly earn trust for agentic workflows because they already hold latent access to email/drive/calendar from before any agent existed — meaning this article's interpersonal-design levers may matter far less for hyperscaler-embedded agents than for standalone or novel-vendor agents.

To Adopt AI at Scale, Employees Need to Trust Agents

Structure

Key terms

Propositions

Arguments

Narratives

Questions addressed

Gaps

Related documents