The op log is the source of truth, so stop keeping everything resident

I imported 6 years of my Roam graph into outl. About 80,000 blocks. On the laptop it opens. Heavy on load, a beat of lag, then it’s there and you scroll.

On the iPhone the app just dies.

It loads the workspace in the background, memory climbs, and iOS jetsam kills the process around 400MB RSS. Reopen, same thing. The vault is unopenable on the phone and there’s no lever for the user to pull. No setting, no “load less mode”, nothing.

The interesting part isn’t that the phone is tight on memory. It’s that the phone has an out-of-memory killer that does not negotiate. The laptop has slack and a swap file, so it forgives a lazy decision. The phone just shoots the process. So mobile isn’t the hard case of the same problem, it’s a different kind of constraint. It turns “wastes memory” into “does not run”. And a constraint that hard is useful, because it points straight at the one decision you got away with everywhere else.

two CRDTs, not one

To see the bug you need to know how outl stores a document. There are two separate CRDTs doing two separate jobs.

The first is the tree. Outl is an outliner, so the document is a tree of blocks, and moving a block under a new parent has to converge across devices without a merge conflict. That’s the move-operation CRDT from Kleppmann’s 2022 paper. It is purely structural. A node in that tree is a parent pointer and a fractional position between its siblings, and that’s it. Roughly 50 bytes. It knows where a block sits and nothing about what the block says.

The second is the text. Each block’s content is its own little text CRDT, so two people editing the same block on two devices merge cleanly instead of clobbering each other. I use Yrs for that. The text deliberately lives outside the tree. The tree moves blocks around, the text CRDT owns the characters.

Hold onto that separation, because it’s the whole reason the fix is small. The expensive thing and the structural thing were never the same object.

what was actually resident

The bug: every block that ever got edited kept its own live Yrs document resident in memory for the entire session. One per block. Each one carrying its own internal edit history, its own struct store, its own state. Nothing was lazy. Open the vault and the full text CRDT of every block you’d ever touched was sitting in RAM.

The tree, the cheap part, was never the issue. A live Yrs doc is. Empty it’s already a few hundred bytes of machinery, and with real edit history behind it you’re at one to two kilobytes minimum, often more. Multiply by hundreds of thousands of blocks and you land between half a gig and a full gig of RAM, just for block text, most of it for blocks you opened once two years ago and never looked at again. The laptop shrugs. iOS sends jetsam and you’re done.

the temptation I didn’t take

The obvious fixes are all the wrong shape. Page the docs to disk and fault them back in. Compress them in memory. Add a snapshot format so you don’t replay history every time.

Every one of those treats the symptom. They keep the model (“a doc per block, somewhere”) and bolt on a tier of machinery to make that model affordable. More code, more state to keep coherent, and the dangerous one: most of those schemes invent a second place where a block’s text lives, and the moment text has two homes that sync independently you’ve reintroduced exactly the cross-device divergence the CRDT existed to kill. I’d be spending complexity to protect a decision I hadn’t questioned yet.

So I questioned it. Why is the doc resident at all?

a projection, and why the rebuild is exact

Outl is event-sourced. Every mutation is an append-only operation in a log, and the log is the only thing that’s authoritative. The materialised tree you navigate is not the truth, it’s a fold over the log. Replay the operations and the tree falls out.

A Yrs document is the same kind of thing. It is a projection of the log, built by replaying that one block’s edit operations. It does not need to stay resident any more than the tree needs a second copy on disk. If I drop it, I can rebuild it from the block’s edits whenever someone actually wants to edit that block again.

The part that makes this safe rather than hopeful is the CRDT itself. A Yrs document’s final state is a function of the set of updates applied to it, not the order they arrive in. That’s the convergence property, the same one that lets two phones sync in the first place. So a doc I rebuild by replaying a block’s edits in log order is byte-for-byte the same document as the one I’d been holding resident, with the same internal state vector. Which matters, because when you do edit the block, the delta you ship to peers is computed against that state vector. Rebuild has to land on the identical state or the next edit’s delta is wrong. It does, and it’s not luck, it’s the property the whole system is already built on. I’m not adding a cache I have to reason about invalidating. I’m recomputing a value the algorithm guarantees is deterministic.

Once you see the text as a projection, holding all of it resident looks absurd. You’re pinning a derived value in memory that you can recompute, exactly, from a log that’s already on disk.

materialise or recompute

But you can’t take that too literally either. A pure projection recomputes from the log every time you read it, and rebuilding a Yrs doc to read one string on every render would be its own disaster, the CPU version of the same mistake. The other extreme is what we had: materialise everything and pin it forever, and you pay in memory.

That’s the real axis. Memory against recompute. Every projection sits somewhere on it, and “keep it all resident” and “recompute it all on demand” are just the two ends, both wrong here for opposite reasons. The fix is to put each block at the right point on that axis instead of one global choice for all of them.

Reads don’t need the CRDT. They need the string.

two tiers

So the store has two tiers now.

A hot tier: a plain map from block id to the materialised text string. That’s what every read in the app actually wants. Rendering markdown, drawing a block on screen, computing backlinks, none of it needs a live CRDT, it needs the current text. A string is cheap, roughly the size of the text itself, and it’s a flat read with no transaction and no replay.

A cold tier: a small bounded cache of live documents, capped, evicted least-recently-used, holding only the handful of blocks you’re editing right now. Edit a block that isn’t in it and its doc gets rebuilt from the log, mutated, the new string written back to the hot tier, and the doc left in the cache in case you keep typing. Stop touching it and it ages out. You only ever pay for live CRDT machinery on the blocks under your cursor. The other 79,950 are strings.

The worst case is reassuring rather than scary. Edit more distinct blocks than the cache holds and docs get evicted, but the materialised string never leaves the hot tier, and an evicted doc is just one rebuild away from the log. You can’t lose text by overflowing the cache. You can only make the next edit of an evicted block do a rebuild.

the cost I took on purpose

That rebuild isn’t free, and I want to be straight about it. Rebuilding a cold block scans the op log for that block’s edits. Today that’s a linear pass over the log. On a vault with a couple hundred thousand operations, every edit of an evicted block walks all of them to pick out the few that belong to it.

I left it linear on purpose. Edits are paced by a human hand, one block at a time, and a rebuild only happens on the rare path where you re-edit a block that already fell out of the cache. The obvious optimisation, an index from block id to its edit operations, buys you very little on a workload this cold and costs you correctness headaches: the log reorders itself when a late operation arrives from another device, which invalidates any positional index you were keeping, so you’d be maintaining and invalidating a structure to speed up something that happens between keystrokes. Not worth it until a profile says otherwise. The linear scan is the honest default, not an oversight.

the peak nobody thinks about

Here’s the part I’d have missed a year ago. Steady-state memory was only half the bug.

The thing actually killing the phone was the peak during replay on open, not the resting footprint. When the workspace opens it replays the whole log to rebuild state. In the old design every block’s doc came alive during that replay and they were all alive at the same time, so even if steady state had somehow been fine, the open spike alone sailed past jetsam. And against an OOM killer, peak is the only number that matters. A system that swaps degrades. A system that gets killed is binary. It either fits under the line for the entire replay or it dies, and an average doesn’t save you.

So replay is two passes now. The first applies every operation to the structural tree, where a text edit is a no-op, because the tree genuinely doesn’t care what a block says, only where it sits. The second pass groups the edit operations per block, and for each block it builds one doc, materialises the string, and drops the doc before moving to the next one. Peak memory during open is one live document. One. Not one per block. That flat peak is what actually lets the phone open the vault, more than the steady-state win does, and it’s the kind of thing that hides from you until something with a hard ceiling forces you to look. Any import, any replay, any migration that brings a whole dataset to life at once has this trap in it.

the numbers

80,000 blocks.

Before: roughly 80,000 live docs at one to two kilobytes each. Half a gig to a full gig, and a replay spike on top of it.

After: 80,000 short strings at around 60 bytes each, plus at most a few hundred live docs in the cache. Five to ten megabytes, and a replay peak of a single doc.

Two orders of magnitude on the resident side, and the open spike gone. The phone opens the vault and sits well under the jetsam line with room to spare.

it lands everywhere, and the kernel never moved

The change lives in the core, in the workspace type in the shared Rust crate every client is built on. Mobile, desktop, the terminal UI, the CLI, they all read a block’s text through the same call, and that call now reads the hot tier instead of holding a doc. Nobody wrote a line of client code. The phone was the only place that was dying, but the waste was everywhere, and fixing it in the core fixed it for all of them at once.

And the CRDT kernel, the move-operation algorithm that has to match the paper line for line, I never touched it. A text edit was already a no-op on the structural tree, so the part with the proof behind it didn’t move at all. That’s the separation from the top paying off. Because text was never tangled into the tree, the memory fix stayed entirely on the text side and the structural correctness nobody wants to re-litigate was never in the blast radius.

what I left out on purpose

There’s still a second copy of history resident, and I want to name it so this doesn’t read as a clean sweep. The op log itself holds every edit operation’s bytes in memory. That’s a cheaper copy than the live docs were, but it’s real, and on a big enough vault it becomes the next ceiling. Shrinking it is a different project, sharding the log per page so you only hold the operations for pages you’ve opened, and it’s only worth building when a vault hits the ten-thousand-page wall. Cutting the live docs already drops my 80k vault comfortably under jetsam, so I stopped there. Ship the fix that clears the binding constraint. Don’t gold-plate the one that isn’t biting yet.

the actual credit

I want to be honest about why this was cheap, because that’s the real lesson and it isn’t “look at my clever patch”.

It was cheap because the architecture had already done the hard part. “The op log is the source of truth, everything else is a projection” is a decision made before any of this code existed, back when the tree being a fold over the log was the whole model. The fix didn’t invent anything. It noticed that the rule I’d written for the tree applied just as well to the text, and took it one step further: if the structure is a projection you can rebuild, so is the content. That’s the entire idea. The thinking that made it possible happened a year earlier.

That’s the takeaway I’d hand anyone building local-first software, or anything event-sourced. Spend your hardest thinking up front on the one question of what is authoritative and what is derived, and hold that line without exceptions. Get it right and the performance work downstream stops being clever. A memory problem stops being “design a cache” and turns into “this is a projection, why am I holding it”. The cache, the eviction, the two passes, all of it is mechanical once the model is honest about what’s truth and what’s a copy.

Issue and discussion: https://github.com/avelino/outl/issues/108