I benchmarked CC's caveman plugin against 'be brief'. The token math is wild.
TroyHay6677 · article · 2026-04-30 source ↗ Reader 🖥️
Extreme token compression (via the Caveman plugin or equivalent system prompts) delivers 70-75% output token savings on routine coding tasks but degrades reasoning on complex problems, so practitioners need a hybrid approach that toggles compression based on task difficulty.
practical · mixed — Primarily expository/argumentative. The author reports benchmark results (expository) and argues for a hybrid approach to token management (argumentative). Includes narrative elements via personal testing experience.
Structure
| Part | Role |
|---|---|
| Problem statement: CC burns tokens on conversational fluff | Establishes the pain point — RLHF-trained politeness wastes output tokens in high-volume coding sessions |
| The Caveman plugin phenomenon | Introduces the trending solution — forcing broken English to bypass the helpful-assistant persona |
| Benchmark methodology and results | Core evidence — three setups tested over 20-turn sessions showing ~70-75% token reduction for both Caveman and primal system prompts |
| Performance degradation on complex tasks | Counter-evidence — Caveman fails on a Prisma circular dependency migration where verbose mode succeeds |
| Hybrid approach recommendation | Synthesis — use Caveman for 80% of routine work, toggle off for complex reasoning |
| Ecosystem commentary and product mentions | Contextualizes the finding within a broader tooling trend (Code Burn, Evo discover) |
Key terms
- Caveman plugin Claude Code
An open-source Claude Code plugin by JuliusBrussee that forces the model to output primitive, broken-English responses to minimize output tokens - Primal system prompt Token Efficiency
A manually injected system prompt using broken English ('Me talk short. No explain. Tool first.') that achieves the same effect as the Caveman plugin - Chain-of-thought amputation Token Efficiency
The removal of intermediate reasoning steps that the model normally produces via output tokens, which degrades complex problem-solving - Token burn Token Throughput
The rate at which output tokens are consumed, treated as a resource management problem analogous to CPU profiling
Propositions
- explicitThe Caveman plugin and a manually crafted primal system prompt achieve practically identical token savings (~70-75% reduction in output tokens).conf: medium
- explicitcorroboratedSimply telling the model to 'be brief' does not significantly reduce output tokens because RLHF-trained politeness is too deeply ingrained.conf: mediumsupports ←The Post-training Process OpenAI Used for ChatGPT · p6 — The KL penalty's job is precisely to keep the RL policy close to the SFT-shaped reference model, which is the mechanism that would make RLHF-trained stylistic habits like excessive politeness resistant to being prompted away.
- explicitForcing broken English in the system prompt short-circuits the model's helpful-assistant persona more effectively than polite instructions to be concise.conf: medium
- explicitdevelopedOutput tokens serve as the mechanism for reasoning in LLMs, so compressing them degrades complex problem-solving.conf: highextends ←How Do AI Agents Spend Your Money? Analyzing and Predicting Token Consumption in Agentic Coding Tasks · p7 — Caveman's finding is scoped to output-token compression in a single-turn/chat-style setting ('output tokens are the mechanism for reasoning'). This paper shows that in long-horizon agentic coding specifically, the dominant cost driver is cache-read *input* accumulation, not output -- extending token-economics analysis to a task regime where the caveman finding's premise (output tokens are the main lever) does not hold.
- explicitExtreme token compression causes failures on tasks requiring multi-step reasoning (e.g., circular dependency resolution).conf: medium
- implicitThe Caveman plugin is essentially just a convenient packaging of the primal system prompt trick — it adds no unique technical capability.conf: high
- explicitcorroboratedToken management is becoming a first-class developer skill comparable to writing good code.conf: lowsupports ←The Almedra Token-Efficiency Benchmark, Updated For GLM 5.2 · p5 — extract-cc-caveman-benchmark#p7 claims token management is becoming a first-class developer skill; this benchmark corroborates it from the harness-selection angle, showing a large, reproducible-in-kind cost spread among six coding-agent products running the identical model and task.
- implicitcorroboratedModels with extended thinking / separate reasoning tokens would not suffer the same chain-of-thought amputation from compressed output.conf: highsupports ←Claude Code: Best practices for agentic coding · p7 — Documenting a keyword-controlled thinking budget as a lever distinct from visible output supports the caveman analysis that models with separate reasoning tokens would not suffer chain-of-thought amputation from output compression.
Arguments
Developers should use a hybrid approach: Caveman mode for routine tasks, full verbosity for complex reasoning. strength: medium
Explicit premises:
- Caveman/primal prompts cut output tokens by 70-75% on routine tasks.
- Caveman mode fails on complex tasks like circular dependency resolution.
- Output tokens are the mechanism for model reasoning.
Implicit premises:
- Developers can reliably predict which tasks are routine vs complex before starting them.
- The cost of toggling modes is low enough to be practical.
Inferences:
- Since compression helps on simple tasks but hurts on complex ones, an adaptive strategy dominates a fixed strategy.
Broken English prompts are more effective than polite brevity instructions at reducing token output.
Explicit premises:
- 'Be brief' still produces preambles and summaries.
- Broken English ('Me talk short') short-circuits the helpful-assistant persona.
Implicit premises:
- The RLHF training creates a 'persona' that responds differently to register/style cues than to explicit instructions.
- The benchmark adequately controlled for other variables.
Inferences:
- Stylistic persona disruption is a more effective steering mechanism than direct instruction for output compression.
Narratives
- Author ran a Prisma schema migration with circular dependency under both standard and Caveman modes. Standard Opus explained the relationships in three paragraphs and generated a correct schema; Caveman produced an invalid schema by skipping reasoning steps.
→ Serves as the critical counter-example demonstrating that token compression has a real cost on reasoning quality. - Author describes a week of testing across a dozen workflows — CSS tweaks to deep refactors — measuring token savings.
→ Establishes credibility and the empirical basis for the 70-75% token savings claim.
Questions addressed
- ✅ Is the Caveman plugin worth installing, or can you achieve the same effect with a system prompt?
They are functionally equivalent. The plugin is a convenience wrapper around the same technique. - ✅ Does extreme token compression degrade model reasoning quality?
Yes, on complex multi-step tasks. The author demonstrates failure on a Prisma circular dependency migration. - ✅ What is the optimal strategy for managing output token consumption?
A hybrid approach: Caveman for routine work (~80%), full verbosity for complex reasoning. - ✅ Does 'be brief' work as a token reduction strategy?
Not well — the model's RLHF training overrides polite brevity instructions. - ⭕ Does the chain-of-thought amputation affect smaller/local models differently than Opus?
Author raises this question but does not answer it.
Gaps
- uninformedThe author does not distinguish between visible output tokens and extended thinking tokens. Models like Opus 4.7 with extended thinking have a separate, hidden reasoning budget that is NOT affected by output compression. The 'chain-of-thought amputation' thesis may be largely irrelevant for models with extended thinking enabled, since the reasoning happens in the thinking block, not the visible output.
Read next: Anthropic documentation on extended thinking; investigate whether Caveman mode affects thinking token budget or only visible output tokens. - incompleteThe benchmark methodology is described only in broad strokes. No raw numbers, no statistical measures, no task descriptions beyond category labels. We cannot evaluate reproducibility or effect sizes beyond the author's summary claim of '70-75%'.
Read next: Look for the actual benchmark data or reproduction attempts by others. - incompleteThe 'be brief' baseline is mentioned as ineffective but no quantitative comparison is shown. The three-way benchmark claims to test vanilla, Caveman, and primal prompt — but 'be brief' is discussed separately without data.
Read next: A proper controlled comparison of graduated brevity instructions vs persona disruption prompts. - misinformedThe claim that 'output tokens are literally the mechanism for reasoning' conflates text generation with all model computation. The model performs substantial reasoning in forward passes and (with extended thinking) in hidden thinking tokens. Visible output verbosity is a proxy for reasoning, not the mechanism itself.
Read next: Research on chain-of-thought prompting vs internal model representations; literature on extended thinking / scratchpad approaches. - incompleteThe post promotes Code Burn and Evo discover without disclosing any affiliation. Commenters flag this as potential astroturfing. The product mentions weaken the post's credibility as disinterested benchmarking.
Read next: Check author's post history for patterns of product promotion.