Presentation: Accelerating Performance by Incrementally Integrating Rust Into Existing Codebase
Lily Mara · rss · 2026-09-10 source ↗ Reader 🖥️
Changeability & OptionalityCode QualityIncremental Rewrite GranularityInfrastructure EfficiencyLegacy & Organizational ForgettingSoftware Design Fundamentals
Rewriting an existing system for performance is safer and more tractable at the individual-function level — via a C FFI boundary connecting a fast language (Rust) to the existing slow one (Python, via PyO3) — than at the whole-system or even microservice level, because the smaller unit of change lets teams keep the old implementation, tests, and process contracts intact while swapping only the internals that are actually the aggregate cost driver.
practical · mixed — A conference-talk transcript: opens with an argumentative case against full rewrites, moves into expository walkthrough of a worked Python-to-Rust example (with narrated code), closes with a Q&A that surfaces caveats the prepared talk elided. The load-bearing claim is a practical recommendation about technique and granularity of migration, not a theoretical model.
Structure
| Part | Role |
|---|---|
| Critique of full rewrites | opening argument establishing the problem the talk's technique solves |
| Definition of 'FFI refactoring' as a technique more fine-grained than microservice extraction | names and scopes the proposed alternative |
| Feasibility: tradeoffs (deployment complexity, dev-environment complexity, added-bug risk) | honest cost accounting for the technique |
| Two-axis selection framework (Rust's relative speed vs. its FFI tooling maturity, by source language) | decision heuristic for when the technique applies |
| Worked example: Flask stats endpoint reimplemented as a PyO3-backed Rust crate (rstats) | concrete demonstration of mechanics (pymodule, pyfunction, pyclass, cdylib, Maturin) |
| Divergent results between the Python and Rust statistics libraries (quartile mismatch) | case study in a realistic failure mode of ecosystem-swapping rewrites |
| Testing strategy: reusing existing pytest/HTTP-contract tests, dependency injection to compare old vs. new, property-based cross-checks | evidence that the technique preserves existing verification investment |
| Performance results: microbenchmark (~100x at the function level) vs. macrobenchmark (~15%, then ~20% with JSON serialization moved to Rust) | quantifies the payoff and corrects for the seduction of the headline function-level number |
| Real-world validation: Discord's own production use of this pattern | external corroboration that this is not merely a conference-talk technique |
| Q&A: zero-copy/GIL mechanics, exception/logging bridging, Go FFI comparison, Rust's async-runtime-outside-stdlib design philosophy | audience-driven elaboration and caveats the prepared talk did not cover |
Key terms
- FFI refactoring Incremental Rewrite Granularity
Mara's own coinage for rewriting an individual function (not a service, not a whole system) in a faster language and connecting it back to the original codebase across the C Foreign Function Interface, so the migration unit is smaller than a microservice. - aggregate (thinking in aggregates)
Choosing what to rewrite by looking at total time spent across all calls (frequency x per-call cost), not by intuition about which code 'feels slow' — e.g. cheap-but-constant request-verification code can be a bigger total cost than rare expensive operations. - PyO3 / pymodule / pyfunction / pyclass
PyO3 is the Rust crate that generates Python-callable wrappers from Rust code; #[pymodule] turns a Rust module into an importable Python module, #[pyfunction] wraps a Rust function as a Python-callable one, #[pyclass] generates a Python class wrapper for a Rust struct so it can cross the FFI boundary. - Option (Rust's optional/nullable type)
An enum with two variants (something-with-a-value, or nothing) that makes it a compile-time error to use a possibly-absent value without first handling the absent case, letting checks be centralized at a library's edge rather than repeated throughout. - microbenchmark vs. macrobenchmark
A microbenchmark isolates the rewritten function itself (~100x speedup observed); a macrobenchmark measures the whole HTTP request/response cycle including Flask, JSON (de)serialization, and everything else Python still does (~15-20% speedup observed) — the macrobenchmark is the number that actually matters for infrastructure cost.
Propositions
- explicitdevelopedFull system rewrites driven by 'let's just rewrite everything in Rust because it's fast' are error-prone in practice: they blow past deadlines, turn out more complicated than expected, and often reintroduce bugs the old system had already fixed because a lot of institutional knowledge is encoded in old code, not just cruft.conf: highsupportsPresentation: The Future of Engineering: Mindsets That Matter When Code Isn't Enough · p2 — Both invoke the same underlying claim (Gall's Law, in the other doc's terms) that working complex systems evolve from working simple ones and that discarding that lineage via a from-scratch rewrite is what makes rewrites fail.supportsPresentation: The Future of Engineering: Mindsets That Matter When Code Isn't Enough · p4 — Both treat a full/expensive rewrite as the fallback when a team can't (or won't) build genuine incremental comprehension-preserving change into an existing system.refines ←Bun Rewrites 535K Lines of Zig into Rust in Four Months, Eliminates Numerous Memory Leaks · p6 — p1 there claims full-system rewrites are generically error-prone and blow past deadlines; this case reports the opposite outcome for an AI-orchestrated, test-suite-anchored full rewrite — though the 19 post-merge regressions this same document reports suggest the underlying risk was reduced, not eliminated.
- explicitOptimizing for performance by picking a faster programming language alone, without also addressing architecture, database schema, query/caching patterns, and microservice layering, is an incomplete strategy.conf: high
- explicitFFI refactoring — rewriting at the individual-function level and connecting old and new code via the C FFI — is a viable unit of migration that is more fine-grained than microservice extraction.conf: highextendsThe Hard-Stop Rule: From 3 HCM Monoliths to 120 Domain Microservices · p1 — The Hard-Stop Rule doc shows incremental migration at the service-extraction granularity funded as a byproduct of ordinary roadmap work (pull-based-migration); this talk pushes the same 'smaller unit of change, preserve the old contract' logic one level deeper, to the individual function via FFI.
- explicitThis technique carries real costs: more complex deployments (native binaries instead of simple dynamic-code deploys), more complex developer environments (compilers or matched native binaries on every dev machine, or maintaining duplicate implementations), and some non-zero risk of introducing new bugs.conf: high
- explicitThe bug-introduction risk is lower for function-level FFI refactoring than for a full system rewrite because the change is more constrained/scoped.conf: medium
- explicitThe best candidates for this technique are dynamic languages with strong Rust FFI tooling and a large speed gap versus Rust (Ruby, Python, Node.js, Lua); Go is a poor candidate despite being slower than Rust because its runtime makes C FFI expensive and awkward; C/C++ have good tooling but too small a speed gap over Rust to be worth the trouble for this purpose.conf: highrefinesWhy Rust is different, with Alice Ryhl · p1 — Where the Rust/Alice Ryhl doc frames Rust's advantage as a compound feature-set (memory safety + no GC + type system etc.), this talk narrows the practical adoption question to a specific, scoped decision procedure: which source language and which functions are actually worth touching.
- explicitBecause the interaction happens in-process (same memory space, low call overhead) rather than across a network hop, it is practical to keep some sub-behavior in the old language deliberately (e.g. to preserve exact numeric output) — an option that would not exist as cleanly if the rewrite unit were a separate microservice.conf: high
- explicitRewriting even a small, well-scoped piece of functionality into a different ecosystem can silently change output (e.g. exact vs. estimated quartile algorithms produced different results for a 9-element dataset), so behavioral parity cannot be assumed and must be checked.conf: high
- explicitBecause FFI refactoring preserves the existing function/HTTP contract, existing test suites (pytest, integration tests against the HTTP handler) continue to validate the new implementation with no additional service to stand up, and dependency injection or property-based testing can directly compare old vs. new implementations.conf: highsupportsAgents don't know what good looks like. And that's exactly the problem. · p7 — Reusing the existing pytest/HTTP-contract test suite unmodified to validate the new Rust implementation is a concrete instance of the 'iterative migration preserves the team's existing verification/learning apparatus' pattern the other doc names abstractly for strangler-fig migrations.
- explicitThe headline function-level speedup (~100x in this example) substantially overstates the real-world payoff; the macrobenchmark of the full HTTP request (~15%, later ~20% after also moving JSON serialization to Rust) is the number that should drive investment decisions, because most of the request's time is spent in Flask/JSON/Python overhead unrelated to the rewritten function.conf: high
- explicitThis is not a purely theoretical technique: Discord's real-time engineering team has used a similar Python/Rust FFI pattern in production to cut CPU/infrastructure costs on critical systems.conf: high
- explicitPyO3 mostly avoids copying data across the Python/Rust boundary by handing Rust pointers into Python-owned memory, but working on that data from a new Rust-spawned thread (rather than the calling thread) requires either a copy or explicit interaction with Python's GIL.conf: high
- explicitRust's lack of an exception mechanism is well-suited to FFI boundary code: errors-as-values naturally bubble up to a single conversion point where they become a Python exception or log entry, rather than needing to be caught at arbitrary points.conf: high
- implicitSelecting what to FFI-refactor should be judged by an aggregate cost lens (frequency x per-call cost summed across the system), not by which function feels most computationally expensive in isolation.conf: high
- implicitThe talk implicitly treats a full ecosystem swap (even at function scope) as introducing an unavoidable verification burden — you cannot assume drop-in numeric/behavioral equivalence between libraries in different languages, so a diffing/comparison step against the old implementation is a required part of any such migration, not an optional nicety.conf: high
Arguments
For performance-motivated rewrites, function-level FFI refactoring is a better default unit of change than either a full system rewrite or a microservice extraction. strength: high
Explicit premises:
- Full rewrites are error-prone, frequently over-run, and tend to reintroduce previously-fixed bugs because they discard institutional knowledge encoded in old code.
- Microservice extraction adds network hops and makes it hard to fall back to old behavior when the new implementation diverges.
- FFI refactoring keeps the existing contract (function signature / HTTP handler) unchanged, so existing tests keep validating the system with no new service to run.
- Same-process, same-memory-space FFI calls have low enough overhead that mixed old/new implementations are practical to run side by side indefinitely if needed.
Implicit premises:
- Most performance-critical hot paths in a legacy monolith are small enough in surface area (a function or a handful of functions) that they can be meaningfully isolated and swapped.
- The team retains ownership of, and comfort maintaining, both the old and new language on an ongoing basis (dual toolchain cost is acceptable).
- A smaller unit of change is intrinsically easier to verify and revert than a larger one, all else equal.
Inferences:
- Teams whose performance bottleneck is spread diffusely across many small operations (not concentrated in a few functions) will get less benefit from this technique and may need architectural rather than per-function changes.
- The technique is a form of Kent-Beck-style incrementalism applied to performance work specifically: make the smallest safe change that captures most of the value, rather than reaching for the biggest, riskiest lever.
The headline 'Rust is 100x faster' framing is a rhetorical trap that this technique's own benchmarking discipline must actively correct for. strength: high
Explicit premises:
- The isolated function microbenchmark showed roughly 100x speedup.
- The full-request macrobenchmark showed roughly 15%, then 20%, speedup.
- Most of the HTTP request's time is spent in Python/Flask machinery (JSON parsing, request handling) that the rewrite did not touch.
Implicit premises:
- Decision-makers (e.g. a boss evaluating infrastructure savings) reason in macro, end-to-end cost terms, not micro function-level ones.
- Reporting only the microbenchmark number would overstate expected savings and could lead to over-investment in this technique relative to its actual payoff.
Inferences:
- Any pitch for this technique to management should lead with macrobenchmark/aggregate infrastructure-cost numbers, not the flashier function-level multiplier.
- Iteratively FFI-refactoring more of the request path (as with the JSON-serialization follow-up) is the correct lever for closing the gap between the micro and macro numbers, rather than distrusting the technique.
Narratives
- A worked example: a Flask endpoint doing simple statistics (range, quartiles, mean, standard deviation) over a JSON list of numbers is reimplemented as a Rust crate (rstats) using PyO3, Cargo, and Maturin — walked through dependency setup, pymodule/pyfunction/pyclass annotations, and the resulting Python-callable function.
→ Demonstrates that the FFI-refactoring technique requires only a modest, learnable amount of new code and tooling, grounding the abstract argument in a concrete, reproducible mechanism. - When the Python and Rust implementations' outputs are compared, the quartiles disagree (one library computes exact quartiles, the other estimates — good for huge datasets, bad for five numbers), forcing a decision about whether exact behavioral parity is required.
→ Illustrates a realistic risk of ecosystem-swap rewrites and uses the small, contract-preserving unit of change to show a menu of graceful remediation options (find an equivalent library, reimplement the exact algorithm, or deliberately keep that one calculation in Python) that a larger-scoped rewrite would not offer as easily. - Discord's real-time engineering team publicly documented using this same Python/Rust FFI pattern in production on systems handling user state, to cut costs and improve scalability for large guilds.
→ External corroboration that the technique is production-proven at scale, not just a conference-talk toy example.
Questions addressed
- ✅ Why are full-system rewrites risky, and what should you do instead if you want a faster system?
Full rewrites lose institutional knowledge and run over budget; the recommended alternative is to rewrite only the aggregate-cost-driving functions via FFI, keeping everything else unchanged. - ✅ How do you decide which source language/runtime is a good FFI-refactoring candidate?
A two-axis framework: relative Rust speedup vs. the source language, and Rust's FFI tooling maturity for that language — dynamic languages (Python, Ruby, Node, Lua) score well on both; Go scores poorly on tooling despite being slower than Rust; C/C++ score well on tooling but too small a speed gap to bother. - ✅ How do you actually build and wire up a Rust extension callable from an existing Python codebase?
Via PyO3 (pymodule/pyfunction/pyclass, cdylib compilation target) and the Maturin build tool. - ✅ What do you do when the rewritten implementation's output doesn't exactly match the original?
It depends: find an equivalent library, reimplement the exact algorithm if business-critical, or deliberately keep that sub-behavior in the old language and call across languages as needed. - ✅ How do you avoid unnecessary copying across the Python/Rust boundary, and what limits that?
PyO3 typically hands over pointers into Python-owned memory rather than copying; copying (or GIL interaction) becomes necessary only when processing that data from a separately-spawned Rust thread. - ✅ How do error handling and logging work across the language boundary?
Rust's errors-as-values naturally bubble to the FFI edge for conversion into a Python exception; logging/tracing facades on the Rust side can be wired to forward into Python's logger. - ✅ Is this a purely theoretical/academic technique, or does it hold up in production?
Discord's real-time engineering team reportedly used the same pattern in production for cost and scalability gains. - ⭕ What is the total, honest cost of maintaining a mixed Python+Rust codebase long-term (hiring, onboarding, ongoing dual-toolchain maintenance, CI/build complexity growth as more functions get FFI-refactored)?
Only gestured at (deployments/dev-environments 'more complicated') without concrete numbers or a long-horizon account of what happens as the number of FFI-refactored functions grows. - ⭕ How do you decide when to stop FFI-refactoring — i.e. what is the exit condition or steady state of this incremental strategy?
Not addressed; the talk shows two increments (stats computation, then JSON serialization) but does not discuss a stopping rule, a point of diminishing returns, or what an organization does once most 'hot' functions have been migrated.
Gaps
- incompleteNo accounting of how this strategy scales organizationally as more and more functions get FFI-refactored: at what point does 'a bit of Rust here and there' become its own maintenance burden (build complexity, CI matrix growth across OS/architecture combinations, code-review expertise requirements, on-call knowledge split across two languages)? The talk treats each FFI refactor as an isolated decision without discussing the compounding cost of many such decisions.
Read next: Read Discord's own engineering blog posts on their Rust adoption (referenced but not linked in the talk) and any postmortems on maintaining polyglot monoliths at scale, to see whether the aggregate-cost lens the talk applies to individual functions also holds once dozens of functions have been migrated. - incompleteThe talk never states a stopping rule or steady-state vision for this incremental strategy — unlike a strangler-fig microservice migration (which has a natural endpoint: the monolith is fully decomposed), it's unclear whether FFI refactoring is meant to converge on some end state or continue indefinitely as new hot spots are discovered.
Read next: Compare against Kent Beck's framing of incremental design as a continuous discipline rather than a project with an end date (Tidy First?) — does 'no defined end state' matter less than it would for a rewrite-style migration, precisely because each increment is independently low-risk?closed byBrownfield Agentic Engineering — Supplies the general stopping rule that talk was criticized for lacking: a migration unit converges when the new path works and the old dependency is demonstrably, verifiably removed — giving incremental migration a defined endpoint per unit, analogous to a strangler-fig migration's natural endpoint, rather than an indefinite process. - illogicalThe talk asserts the bug-introduction risk is lower at function scale than for a full rewrite (proposition p5) but never actually demonstrates this — no comparative data, incident rates, or even an anecdote of a bug this technique introduced. The claim rests entirely on 'smaller scope, therefore safer' intuition, which the talk does not test against the possibility that FFI boundaries introduce their own novel bug class (memory/lifetime mismatches, GIL misuse, silent numeric divergence as actually shown) that a same-language refactor would not.
Read next: Read practitioner postmortems specifically about FFI-boundary bugs (use-after-free across a Python/Rust boundary, GIL deadlocks, ABI mismatches) to test whether 'smaller unit of change' actually implies 'lower risk' once the risk profile of the boundary itself is counted. - uninformedDeployment and dev-environment complexity are named as costs but never quantified or walked through concretely (e.g. what a CI pipeline that cross-compiles Rust wheels for every supported OS/architecture actually looks like, how long that adds to build times, what happens when a contributor's machine can't match a required target triple).
Read next: Read maturin/PyO3's own packaging documentation and case studies (e.g. cryptography, polars, pydantic-core) on multi-platform wheel-building pain, to fill in the concrete operational cost this talk only names in the abstract.
Closes gaps elsewhere
- Why Rust is different, with Alice Ryhl · g1 — That gap flags the Rust/Alice-Ryhl conversation's one-sided pitch and its missing honest cost accounting for adopting Rust. This talk supplies exactly that from a practitioner doing a real (if smaller-scope) Rust adoption: named, concrete costs — deployment complexity from shipping native binaries, developer-environment complexity (compilers or matched binaries per dev machine, or maintaining duplicate implementations), and residual bug risk — plus a worked example where the rewrite's output didn't match the original. It is not the requested postmortem-with-metrics (no hiring/training/compile-time data), but it is the honest, non-hype-driven cost accounting the other extraction found missing.
Related documents
- Agents don't know what good looks like. And that's exactly the problem.
- The Hard-Stop Rule: From 3 HCM Monoliths to 120 Domain Microservices
- Presentation: The Future of Engineering: Mindsets That Matter When Code Isn't Enough
- Code maintainability plummets in the AI coding era
- CannotMeasureProductivity
- Fragments: July 6
- Software Design in the Agentic Age: Placing Your Bets
- Software Design in the Agentic Age: Place Your Bets
- What good software looks like at REA
- Applying Best Simple System for Now for Software Design