GitHub - Doorman11991/smallcode: AI coding agent optimized for small LLMs. 87% benchmark with 4B-active model.

https://github.com/Doorman11991/ · article · 2026-05-21 source ↗ Reader 🖥️

Agent Harness Agent Planning Artifacts Agent Tool Management Ashby's Law (Requisite Variety)Coding Agents Context Engineering Local AI Model Routing Riding Model Improvements

A well-engineered harness — context budgeting, forgiving tool-call parsing, patch-first editing, two-stage tool routing, decomposed planning, and per-model profiles — can extract production-grade agentic coding work from small (7B-20B), locally-hosted LLMs that would otherwise be considered too weak for the task, making local, private, frontier-free coding agents viable.

practical · mixed — A GitHub README that is primarily practical (argues for adoption: install it, use it) but mixes expository description of architecture with an implicit theoretical claim that harness design can compensate for small model capability. The titular benchmark number ('87% with 4B-active model') is a marketing hook used to make the practical argument but is never substantiated in the body.

Structure

Part	Role
Title and lead claim (87% benchmark with 4B-active model)	Marketing hook intended to establish credibility before the reader evaluates the architecture
OpenCode vs SmallCode comparison table	Frames the design as an oppositional reaction to frontier-model-assuming agents; sets up each architectural choice as a small-model accommodation
Quick Start / Install / Requirements	Reduces adoption friction; emphasizes one-line install, prebuilt binaries, optional native compilation
Configuration section	Demonstrates local-first design (SMALLCODE_BASE_URL points to localhost), with optional cloud escalation
Architecture file tree	Establishes engineering legitimacy by showing modular structure; lists ~18 tools and named subsystems (governor, escalation, model_client)
Key Features section (MarrowScript, BoneScript, context budget engine, 2-stage tool routing, early-stop detection, forgiving parser, patch-first editing, TODO planning, model profiles, working memory)	Enumerates the specific harness components that compensate for small-model weaknesses; each feature is paired with a small-model failure mode
Model escalation list (Claude 4.5/4.6, GPT-5.4, DeepSeek V4)	Cloud-model fallback option for hard failures, opt-in only
Observability section (token monitor, traces, /eval, trace-to-test)	Shows the project takes measurement seriously
Programmatic API and tool list	Library-mode usage and capability surface
License (MIT)	Lowest-friction license choice

Key terms

harness (implicit) Agent Harness
The entire surrounding engineering layer — context manager, tool parser, planning system, model profiles, governor, escalation — that the project provides on top of a raw local LLM endpoint. Never named explicitly in the README, but every feature is justified as part of this layer.
4B-active
Used only in the title; never defined in the body. Most plausibly refers to an MoE model where only ~4B parameters activate per token (e.g. a sparse model like a hypothetical gemma-4-e4b variant, which appears in the example code), but could also refer to a small dense model.
small LLM Local AI
7B-20B parameter models, running on consumer hardware via a local OpenAI-compatible endpoint (LM Studio, Ollama)
frontier model Riding Model Improvements
Models like Claude and GPT-5 with 128k+ context and 'perfect tool calling' — the implicit comparison class
context budget Context Engineering
Active management of context-window usage: tool results capped at 4k chars, mid-turn eviction of old results, semantic compression of history rather than truncation
2-stage tool routing Agent Tool Management
Model first selects a tool category (read/write/search/run/plan), then receives only that category's schemas. Halves schema-context overhead.
forgiving tool-call parser
Parser that accepts tool calls in JSON, YAML, XML, Hermes format, or plain text; auto-repairs param-name and type mismatches
patch-first editing
Search-and-replace as the primary edit primitive, on the premise that small models 'can't reliably reproduce entire files — they truncate, hallucinate, or drift'
TODO-driven planning Agent Planning Artifacts
Complex tasks decomposed into atomic steps written to a TODO file the model reads each turn; each step validated (lint/compile) before moving on
early-stop detection
Pattern-matching for repetition loops, 'patch spirals' (model stuck repeatedly patching a corrupted file), and 'greeting regression' (model loses task context and reverts to greeting). Triggers rewrite or context re-injection.
model profile
Per-model configuration capturing context length, tool-call format (native/Hermes/JSON/XML/text), chat template, and strengths/weaknesses
MarrowScript
A DSL (by the same author) for declaring 'cognition' — prompts with caching, retry, validation, traces, and budget enforcement. 50 lines of .marrow allegedly compile to 1400+ lines of TypeScript.
BoneScript
Another DSL (by the same author) for declaring Node.js/TypeScript backends in a single .bone file; the agent compiles it to a full project. Claimed to reduce 8-15 tool calls to 1-2.
escalation Model Routing
Opt-in fallback from local model to a cloud model (Claude/OpenAI/DeepSeek) on hard failure, session-limited to prevent runaway costs

Propositions

explicitFrontier-model-assuming agents (e.g. OpenCode) systematically fail when paired with small local models because they assume reliable JSON tool calling and dump full context.
explicitA SmallCode-style harness allows a 4B-active model to achieve 87% on some unspecified coding benchmark.
supportsHugging Face's Clem Delangue: Stop Comparing Engines to Cars · p3 — Concrete demonstration of Delangue's claim: a harness designed for small local models lifts a 4B-active model to 87% on a coding benchmark.
explicitcorroboratedSmall models (7B-20B) produce tool calls in heterogeneous, often malformed formats (JSON, YAML, XML, Hermes, plain text).
supports ←Viability of local models for coding · p10 — Identifies inconsistent tool-call parameter naming across harnesses as an additional, concrete source of the malformed/heterogeneous tool calls SmallCode's harness is engineered to absorb.
explicitSmall models cannot reliably regenerate entire files — they truncate, hallucinate, or drift.
explicitLimiting the per-turn tool schema surface (via 2-stage routing) is necessary for small models with 8-16k context windows.
explicitSmall models exhibit specific failure modes — repetition loops, patch spirals, greeting regression — that can be detected by pattern matching and repaired by structural interventions (rewrite, context re-injection).
explicitExternalizing planning (TODO files) and memory (persistent scratchpad) compensates for small models' limited reasoning depth.
explicitNarrowing the agent's output space — via domain-specific DSLs like BoneScript that compile to full projects — dramatically improves reliability with small models (reduces 8-15 tool calls to 1-2).
explicitA declarative cognition layer (MarrowScript) compiled to runtime code provides caching, retry, validation, traces, and budget enforcement for free.
explicitA local-only agent (no network calls) is a meaningful privacy and sovereignty advantage over cloud-API agents.
explicitdevelopedEven the best harness will sometimes hit hard failures with small models, requiring cloud escalation as a fallback.
extends ←AlloyDB Ships Proxy Models That Replace LLM Calls with Local Inference Inside the Database · p5 — Same escalate-to-the-frontier-when-the-cheap-local-model-fails safety pattern, ported from coding harnesses (small model with cloud escalation fallback) to a database query engine (proxy with frontier fallback).
implicitdevelopedThe harness, not the model, is the dominant lever in agentic coding performance — at least at the small-model end of the spectrum.
refinesAgent Harness Engineering · p1 — Confirms harness-beats-model but conditions it: the harness is the dominant lever specifically at the small-model end of the capability spectrum.
extendsHarness engineering for coding agent users · p1 — Generalizes the outer-harness thesis to its extreme: at the small-model end the harness, not the model, is the dominant performance lever.
refines ←Experiences with local models for coding · p2 — SmallCode claims the harness (not the model) is the dominant lever for small-model agentic coding. Böckeler's field experience refines this: task selection is at least as dominant, and even hardware/RAM matters — swapping harness (OpenCode vs Pi) 'did not make much of a difference' on her hardest tasks, and the same model swung wildly in quality between a 48GB and a 64GB machine.
implicitAgent quality is reducible to or substantially measurable by coding benchmarks.
implicitdevelopedSelf-hostable, locally-run coding agents are a viable adoption path for serious software work, not just a hobbyist demo.
refines ←Experiences with local models for coding · p3 — SmallCode presents locally-run coding agents as a viable adoption path for serious software work. Böckeler refines the scope: viable today only for small, well-scoped, often pre-planned changes; more complex logic 'fell apart', and the overall experience is 'not plug-and-play' and 'very far away' from frontier models.
implicitdevelopedCoding tasks are bounded enough that a 7B-20B model with a sufficiently constrained harness can complete them — i.e. the relevant agentic 'reasoning' is mostly about navigation, format compliance, and decomposition, not about latent intelligence.
refines ←Portal by Spotify cut my Claude Code token usage by 90% · p8 — Cuts against the optimistic edge of that claim (that most agentic 'reasoning' in coding is bounded navigation/format/decomposition doable by small models): here the worker model was given a genuine reasoning task without smallcode's heavier scaffolding and missed a subtle thread-safety bug, which is why the piece explicitly excludes reasoning and debugging from delegation rather than trying to scaffold a small model into handling them.
implicitCoupling the agent to an author-controlled DSL stack (MarrowScript, BoneScript, budget-aware-mcp) is an acceptable price for adoption.
implicitThe dominant cost geometry to optimize for is upfront hardware cost + privacy + control, not marginal API spend + frontier capability.

Arguments

A small local model with the SmallCode harness can do real coding work that would otherwise require a frontier model. strength: low

Explicit premises:

Small models fail in specific, characterizable ways (malformed tool calls, file regeneration drift, context overflow, repetition loops)
Each failure mode has a corresponding harness component (forgiving parser, patch-first edits, context budget, early-stop detection)
A 4B-active model with the harness achieves 87% on some benchmark

Implicit premises:

The 87% benchmark number is on a meaningful benchmark for agentic coding work (not, e.g., a cherry-picked easy subset)
The benchmark generalizes to the kinds of coding tasks readers actually want to do
The small-model failure modes named are the *complete* set of small-model failure modes — not just the visible ones
Compensating for the visible failure modes is sufficient to reach frontier-comparable quality
Latent reasoning depth is not the bottleneck for typical coding tasks

Inferences:

If true at the 4B-active scale, the harness-over-model thesis would imply that frontier-API spending is largely unnecessary for many coding workflows — a far-reaching economic claim the README does not draw explicitly.

Locality (no network calls) is a sufficient reason to prefer SmallCode for users with privacy or sovereignty concerns. strength: medium

Explicit premises:

SmallCode runs against a local LLM endpoint with no cloud calls by default
Frontier-API agents send code and prompts to third-party servers

Implicit premises:

The user can actually run a 7B-20B model on their own hardware at acceptable latency
The capability gap to frontier models is small enough that locality is worth the trade-off
Optional escalation (which does call the cloud) will not be triggered often enough to undermine the privacy claim

Inferences:

For Norwegian/EU public-sector contexts with data-sovereignty rules, this argument has higher weight than the README acknowledges.

Narrowing the agent's output space (via DSLs like BoneScript) is a generalizable strategy for getting small models to work. strength: medium

Explicit premises:

BoneScript reduces 8-15 tool calls to 1-2 for backend generation
Fewer tool calls means fewer opportunities for tool-call format errors and context overflow
Small models perform better on narrower output spaces

Implicit premises:

Domain-specific DSLs are an acceptable layer for users to adopt
The set of tasks reachable through BoneScript is large enough to matter (effectively an Ashby's-Law variety-reduction move, though never named)
DSL maintenance cost does not exceed the agent-reliability gains

Inferences:

This is the same conceptual move as Böckeler's 'harness templates' — pre-defined topologies as variety reduction. SmallCode arrives at the conclusion without naming the principle.

Heterogeneity of small-model tool-call formats is best handled at the parser, not the prompt. strength: high

Explicit premises:

Different small models emit tool calls in different formats (JSON, YAML, XML, Hermes, plain text)
Trying to force a uniform format via prompting is unreliable
A multi-format parser with auto-repair handles the variation gracefully

Implicit premises:

Format variation is an irreducible property of small models, not something that better fine-tuning or chat templates would fix
Auto-repair (guessing the intended call from a malformed one) is safer than rejecting and retrying

Inferences:

If small-model format heterogeneity is irreducible, this is an argument against ever assuming 'tool calling JSON mode' will hold at this end of the model spectrum — independent of which agent framework you use.

Narratives

The OpenCode vs SmallCode comparison table — six rows of contrasts (target, context, tool calling, planning, editing, privacy) — implicitly tells the story of a developer trying frontier-targeted tooling on a small local model, watching it fail row by row, and building SmallCode in response.
→ Carries the entire argument for the project's existence as a small-model-specific design. Acts as the project's origin story without ever stating it as one.
The example programmatic API code shows `model: 'gemma-4-e4b'` — likely a Google Gemma variant with ~4B active parameters in an MoE architecture (the 'e4b' suggesting expert-4B). This is the implicit reference model behind the title claim.
→ Acts as the only concrete instantiation of the '4B-active' claim — and the reader has to infer it from a code sample.
The 50-line .marrow declaration that generates 1400+ lines of TypeScript with caching, retry, validation, traces, and budget enforcement is presented as evidence that the cognition layer is engineered, not hacked together.
→ Argues that SmallCode is not a thin wrapper but a serious system, and indirectly argues that MarrowScript itself is worth adopting.
The Requirements section narrates a fallback chain: prebuilt binaries → native compilation → JSON-based memory fallback. 'If build fails, SmallCode still works.'
→ Tells a story of graceful degradation — supporting the implicit claim that the project is production-aware, not just a research demo.

Questions addressed

✅ Can small, locally-hosted LLMs do real agentic coding work?
Author claims yes, with the right harness — citing an unspecified 87% benchmark at 4B-active scale.
✅ What does a small-model-optimized harness look like in concrete terms?
Context budgeting, 2-stage tool routing, forgiving multi-format parser, patch-first editing, TODO planning, early-stop detection, model profiles, working memory, opt-in escalation.
✅ Which specific failure modes does small-model agentic coding exhibit?
Malformed tool calls, file-regeneration drift, context overflow, repetition loops, patch spirals, greeting regression — each named with a counter-measure.
⭕ On which benchmark is 87% achieved, with what methodology, and against what baseline?
Not addressed in the README.
⭕ How does SmallCode compare to other small-model-friendly agents (Aider, Cline + local model, Continue, OpenHands with local models)?
Only OpenCode is named as a comparison point; no comparison to other small-model-aware tools.
⭕ What is the lock-in cost of adopting SmallCode plus its co-developed dependencies (BoneScript, MarrowScript, budget-aware-mcp)?
Not addressed; the one-author ecosystem is framed as a feature, not a risk.
⭕ How mature is the project — production-ready, beta, demo?
Not addressed. No version history, no users-in-production, no maturity signal beyond MIT licensing.

Gaps

incompleteThe headline '87% benchmark with 4B-active model' is the entire credibility hook and is completely unspecified in the body. No benchmark name (SWE-Bench? Aider polyglot? HumanEval? a custom suite?), no subset, no scoring methodology, no comparison baseline (what does a frontier model score on this same eval? what does the same 4B model score without SmallCode?), no link to results. For a rigorous adopter this is the single most disqualifying gap.
Read next: Check the GitHub repo itself for an evals/ or results/ directory; check the project's `/eval` command output; check whether any third party has reproduced. Compare against published SWE-Bench / Aider / HumanEval+ leaderboards for similar-sized models. Look for the actual model named in the example (gemma-4-e4b) and find its baseline scores.
incompleteNo comparison to other small-model-aware agentic coding tools — Aider (with local models), Cline + Ollama, Continue, OpenHands with local backends, Roo. The only named comparator is OpenCode, which is set up as a strawman ('assumes frontier models').
Read next: Read Aider's local-model documentation and benchmark notes (Paul Gauthier publishes Aider polyglot scores for many models). Compare with Cline's local-model guidance and OpenHands' SWE-Bench results with open models.
incompleteNo failure-mode analysis: what *can't* this do? Where does the harness break? At what task complexity does the small model fall off? The README enumerates failure modes the harness handles, but not failure modes it doesn't.
Read next: Look for issue tracker discussions, particularly around long-horizon tasks, large refactors, and tasks requiring deep type-system reasoning. Run the agent on a known-hard task (e.g. a non-trivial SWE-Bench instance) and observe.
closed byExperiences with local models for coding — Partially. SmallCode's g3 asks the general question of at what task complexity a small model falls off and where the harness breaks. Böckeler supplies concrete, hands-on failure boundaries — cumulative/derived computations, continued-conversation runtime crashes, the 'text wall of doom', and complex game logic falling apart — from OpenCode/Pi/LM Studio setups (not SmallCode's own harness), so it answers the general question, not SmallCode's specifics.
incompleteNo discussion of total cost of ownership: hardware to run a 7B-20B model at usable latency (GPU memory, electricity), engineer time to maintain a local model server, escalation cost when small model fails. The privacy/local row in the comparison table treats locality as pure upside.
Read next: Map total cost (hardware amortization + electricity + maintenance + escalation API spend) against frontier-API consumption rates for a comparable workload to compute break-even.
closed byExperiences with local models for coding — Partially. SmallCode's g4 flags missing total-cost-of-ownership discussion (hardware, latency, RAM). Böckeler reports concrete hardware (M3 Max 48GB, M5 Pro 64GB), RAM pressure (48GB 'stretches' the 35B model, must close other apps), and latency anecdotes (2.5 min success, ~6 min, 'gave up after 11 minutes'). Still qualitative and specific to her setups rather than SmallCode's, so it narrows but does not close the accounting.
incompleteOne-author ecosystem risk: SmallCode depends on BoneScript, MarrowScript, and budget-aware-mcp — all by the same GitHub user. If the author stops maintaining, four projects rot together. This single-maintainer concentration is not flagged anywhere.
Read next: Check commit history and contributor counts on all four repos; check npm download stats for smallcode and budget-aware-mcp; check whether MarrowScript / BoneScript have any external adopters.
misinformedThe escalation target list cites 'Claude Sonnet 4.5 / 4.6, Haiku 4.5' and 'GPT-5.4 Mini / Nano' and 'DeepSeek V4 / V4 Pro / V4 Flash'. As of the document's apparent date, these specific named version numbers may be aspirational, outdated, or fabricated. The presence of confidently-named future/non-existent model versions weakens credibility on other un-checkable claims.
Read next: Cross-check the named model versions against vendor product pages; if they are not real, treat the rest of the README's confident numbers (87%, '8-15 → 1-2 tool calls', '1400+ lines from 50 lines') with proportional skepticism.
illogicalThe marketing claim ('87% with 4B-active model') and the body text ('7B-20B local models') are inconsistent. The title advertises a 4B-active model; the body describes the project as targeting 7B-20B. If 4B-active means MoE active params with a larger total, both can be true — but the README never reconciles them, leaving the reader to guess.
Read next: Identify the actual model used to produce the headline number; if it is MoE (e.g. gemma-4-e4b with ~4B active), check whether 7B-20B framing in the body is meant as total-parameter range, active-parameter range, or both — the project's positioning depends on which.
incompleteNo discussion of how the harness itself is engineered, tested, or evaluated. The Observability section mentions traces and `/eval` but provides no example evaluation results, no harness-coverage metrics, no failure-mode regression tests beyond 'trace-to-test'. The harness is presented as fait accompli, not as something the reader could audit.
Read next: Connect to Birgitta Böckeler's harness-engineering article (already in the user's library): her steering-loop concept asks how a team iterates the harness based on observed failures. Apply that lens here — what is SmallCode's own steering loop?
incompleteThe cybernetic / control-theory framing latent in the design (variety reduction via BoneScript topologies, feedback via early-stop detection, model profiles as adaptive control) is never made explicit. Connecting this project to the broader harness-engineering conversation (Anthropic's effective harness paper, Böckeler's framework) would let it argue from theory rather than just engineering instinct.
Read next: Read SmallCode against Böckeler's 'Harness engineering for coding agent users' (in this batch) — map SmallCode's components to her feedforward/feedback and computational/inferential axes. Read against the Anthropic harness-design materials the user is also tracking.

Closes gaps elsewhere

Harness design for long-running application development · g6 — Supplies the smaller-model-substitution and vendor-independence perspective the Anthropic piece omits: a local-only harness for 7B-20B models with cloud escalation only as fallback (p2, p10, p11).

GitHub - Doorman11991/smallcode: AI coding agent optimized for small LLMs. 87% benchmark with 4B-active model.

Structure

Key terms

Propositions

Arguments

Narratives

Questions addressed

Gaps

Closes gaps elsewhere

Related documents