What Is a Token? A Developer's Guide to Every Type

Janet Wagner · rss · 2026-07-30 source ↗ Reader 🖥️

Agent Governance (Internal)Agent Security AI Unit Economics API Products & Sprawl Context Rot & Context Limits Token Efficiency

The word 'token' names unrelated technical artifacts across four domains -- identity/authorization state, numerical text representation for LLMs, blockchain asset ownership, and single-purpose application security -- so developers must learn each domain's specific token vocabulary and the engineering tradeoff each token type encodes (statelessness vs. revocability, cost vs. capability, security scope) rather than treating 'token' as one concept.

practical · expository — Primarily a categorized glossary/reference guide across four unrelated token domains (authentication, LLM, crypto, application/API). Genuine argumentative or evaluative content is limited to scattered comparative and best-practice asides (stateless vs. session-based tradeoffs, why tokens beat passwords, LLM cost/context-window implications) rather than a sustained thesis-driven argument.

Structure

Part	Role
Authentication Tokens (token-based, session-based, passwordless, OATH)	Catalogs the identity/authorization token types (access, ID, refresh, bearer, JWT, opaque, SAML, session, magic link, OATH software/hardware tokens) and states why token-based auth became the default over passwords.
AI and LLM Tokens	Defines input/output/reasoning/cache tokens as the numerical units LLMs process, and explains their cost and context-window implications.
Crypto Tokens	Catalogs fungible, non-fungible, and semi-fungible blockchain tokens as digital representations of financial, physical, digital, or service assets.
Application and API Tokens (CSRF, password reset, idempotency)	Catalogs narrow, single-purpose security tokens that don't fit the other three categories.

Key terms

bearer token
An opaque string that grants access to whoever holds it, without requiring additional proof of identity; the predominant access-token type used with OAuth 2.0. The article notes bearer tokens are often implemented as JWTs in practice.
opaque token
Also called a 'reference token': a unique, random alphanumeric string generated by the authorization server that carries no readable or actionable information itself (implying it requires server-side interpretation, unlike a self-contained JWT).
JSON Web Token (JWT)
An open standard (RFC 7519) for representing claims transferred between two parties as a compact, signable and/or encryptable JSON object.
access token vs. ID token
The author explicitly distinguishes the two despite their similar issuance context: an access token is proof of authorization (what you can do), an ID token is proof of authentication (who you are).
session token
A unique identifier, created at login and usually stored in a cookie or locally, that authenticates a user and lets the app manage session activity -- the sole token of session-based authentication, where state lives server-side (contrast to token-based auth's client-side state).
refresh token
Enables obtaining a new access token without forcing the user to re-authenticate once the current access token expires.
context window
An LLM's 'working memory,' measured in tokens, with a limit that varies per model; all input, output, and (where applicable) extended-thinking tokens count toward it.
OATH vs. OAuth
Explicitly distinguished: OATH (Initiative for Open Authentication, founded 2004) is an industry collaboration standardizing one-time-password authenticator tokens (software/hardware OTP generators); OAuth is an open authorization protocol. The similar names are flagged as a real source of confusion.

Propositions

explicitToken-based authentication uses a stateless, self-contained model in which token state lives on the client side and does not have to be stored on the server.conf: high
explicitAn access token serves as proof of authorization, while an ID token serves as proof of authentication -- the two are functionally distinct even though both are typically issued during the same login flow.conf: high
explicitA bearer token grants access to whoever holds it (typically as an opaque string) without requiring further proof of identity, and is the predominant access-token type used with OAuth 2.0; in practice it is often implemented as a JWT.conf: high
supportsWhy Token Handling Must Evolve for AI Agents · E2 — This article's description of a bearer token as granting access 'to the party holding the token' matches and grounds the other piece's characterization of traditional token auth: once granted, the holder has broad access for the token's/session's duration -- the very property that piece argues is unsuited to autonomous agents.
explicitOpaque tokens carry no readable or actionable information themselves -- they are just a random string generated by the authorization server, unlike a JWT whose claims are inspectable in the token.conf: medium
explicitToken-based authentication has become the popular default for application developers because of its high scalability, compatibility with APIs, and the proliferation of industry standards such as OAuth 2.0 and OpenID Connect.conf: high
supportsWhat Happens After Every Company Becomes a Data Company? · p6 — This article's account of why OAuth 2.0 and OpenID Connect became the standard, scalable basis of token-based auth is the premise the other piece complicates: those same standards, designed in 2012 for human sign-in, are argued to now need adaptation (just-in-time authorization, Rich Authorization Requests) for autonomous non-human API consumers.
explicitDespite the popularity of token-based authentication, session-based authentication is often adequate for many web applications.conf: medium
explicitThe tech industry has been moving toward passwordless authentication (passkeys, biometrics, magic links) in recent years.conf: medium
explicitOATH and OAuth are different things despite the similarity of the names -- OATH tokens produce one-time passwords, while OAuth is an authorization protocol -- and this is a real point of confusion among developers.conf: high
explicitAuthentication tokens are more secure than password-only verification because they are typically short-lived, encrypted, and specific to a user's session, application, or device, which makes them difficult to reuse if stolen.conf: medium
supportsHow to Manage Privilege Drift in Multi-Agent Systems · p6 — Both identify short-lived, narrowly-scoped tokens as the mechanism that reduces theft/reuse risk, though this piece frames it as a human-vs-password security benefit and the other as a multi-agent least-privilege design principle.
explicitLLM token usage directly determines both the cost of using a model and the quality of its output.conf: high
explicitA model's context window functions as its working memory; exceeding the token limit produces an error, and approaching the limit can degrade response quality because the model has less room to work with.conf: high
explicitCache read tokens are usually less expensive than input tokens and cache write tokens, making prompt caching a cost-saving strategy for repetitive or consistent prompt content.conf: medium
explicitA CSRF token serves as a first line of defense against cross-site request forgery attacks by embedding an unpredictable value in a web page that is difficult to forge.conf: high
explicitAn idempotency token ensures that if the same API request is sent multiple times, it is only processed once -- a common component of payment APIs.conf: high

Arguments

Token-based authentication has become the dominant authentication method for application developers. strength: medium

Explicit premises:

The stateless, client-side model is highly scalable.
It is compatible with APIs.
Industry standards such as OAuth 2.0 and OpenID Connect have proliferated.

Implicit premises:

Standardization lowers integration cost and interoperability risk enough to outweigh the operational cost of managing token lifecycles on the client side.

Inferences:

Scalability + API compatibility + standard proliferation -> widespread developer adoption.

Authentication tokens provide stronger identity-verification security than a password-only approach. strength: medium

Explicit premises:

Tokens are usually short-lived.
Tokens are encrypted.
Tokens are specific to a user's session, application, or device.

Implicit premises:

An attacker who intercepts or steals a token has a narrower window and narrower scope of exploitation than one who steals a reusable password.
Passwords are frequently weak or reused, so any layer that isn't a static shared secret raises the practical bar for attackers.

Inferences:

Short-lived + encrypted + scoped -> difficult to reuse if stolen -> stronger security than passwords alone.

Developers building LLM-backed applications must actively track which token types their application uses and how many. strength: high

Explicit premises:

LLM pricing is usage-based on tokens.
Every interaction with the model generates tokens.
More tokens used means higher cost.
Context window limits are measured in tokens and vary per model.
Exceeding the limit causes an error; approaching it can degrade response quality.

Implicit premises:

Cost and output quality are both operationally significant enough to justify active token management as a distinct engineering practice.

Narratives

A screenshot of Tiktokenizer, an app that visualizes how an LLM breaks text down into numeric tokens.
→ Makes the otherwise abstract 'tokenization' process concrete and visible for readers unfamiliar with how LLMs process text.

Questions addressed

✅ What does the term 'token' mean across different developer contexts (authentication, LLMs, blockchain, application security)?
Answered by categorizing tokens into four domains and defining each subtype within them.
⭕ Which authentication method or token type should a developer actually choose for a given use case?
The article describes the popularity and general tradeoffs of token-based, session-based, and passwordless authentication but never gives selection criteria tied to specific use cases (e.g., mobile vs. server-rendered app, need for immediate revocation, multi-service SSO).
✅ Why do LLM tokens matter for developers building AI applications?
Answered via the cost-per-token and context-window-limit explanation.

Gaps

incompleteNo discussion of token revocation strategies -- how access/refresh tokens are invalidated on logout, compromise, or password change -- despite this being central to the very JWT-vs-opaque-token tradeoff the article gestures at (opaque tokens support server-side revocation via lookup; self-contained JWTs generally don't without extra infrastructure like a blocklist).
Read next: Nordic APIs' own linked piece, 'JWT vs Opaque Tokens: Choosing the Right Token for API Security'; general OAuth 2.0 revocation (RFC 7009) literature.
incompleteThe article places authentication 'tokens' and LLM 'tokens' side by side under one umbrella term without ever flagging that they are entirely unrelated concepts sharing only a name -- a real source of confusion for developers moving into AI-augmented applications who may conflate 'token limit' (LLM context) with 'token expiry' (auth) or assume LLM tokens carry any security property at all.
Read next: Any introductory LLM tokenization explainer paired with an OAuth/JWT primer, read side by side to make the non-relationship explicit.
incompleteNo treatment of token binding or proof-of-possession mechanisms (DPoP, mTLS-bound tokens) that address the exact weakness the article itself names in bearer tokens -- that a bearer token grants access 'to the party holding the token' with no additional proof of identity, so token theft is equivalent to full impersonation.
Read next: OAuth 2.0 DPoP (RFC 9449) and mutual-TLS-bound access tokens (RFC 8705).
incompleteJWT alternatives designed to avoid well-known JWT signature/algorithm pitfalls (e.g., the 'alg: none' vulnerability, RS256/HS256 confusion) such as PASETO are not mentioned, even though the article recommends JWT as the default bearer-token implementation.
Read next: PASETO specification and comparative JWT-vulnerability writeups.
incompleteClient-side token storage (localStorage vs. httpOnly/secure cookies) -- a major practical security decision for the 'stateless, client-side state' model the article describes -- is never discussed, despite being one of the most common real-world vectors for token theft (XSS).
Read next: OWASP guidance on token storage for SPAs and mobile clients.

What Is a Token? A Developer's Guide to Every Type

Structure

Key terms

Propositions

Arguments

Narratives

Questions addressed

Gaps

Related documents