Project

The reasoning invariants

The reasoning invariants

Root CLAUDE.md states every invariant and keeps 1-7 in full: those are short, and each is a flat rule about the op log or the CRDT.

Invariants 8 through 15 are different in kind. Each one is a question to ask before merging, each was bought by an incident with measurements, and each carries the alternatives that were rejected and what the fix made worse. They live here because that reasoning is what makes them usable, and because CLAUDE.md is loaded into context on every session in this repo — past ~40k characters the budget spent re-reading the rules comes out of the task.

The short form in CLAUDE.md is the rule and the question. This file is the why, and the two are one fact: change a rule here and change it there in the same commit.

How they stack

#AsksBought by
8Did these bytes come from the op log, or only from us?233 pages / 1,426 lines deleted by doctor --repair
9Where does the problem live now, and what does that place require?A device actor store with no isolation and no GC
10Who was standing on the thing you moved?A lease that locked every process out of the endpoint
11Is the cost you are avoiding even caused by this change?A second daemon proposed over a cost a flag fixed
12Who does not have what you built, and does anything fail if you don’t say?Three hand-maintained copies of one fact, each stale differently
13Does this name mean the same fact everywhere it appears?The OS appearance setting changing block elevation
14Can the user say “not this”, and does the same code answer it?Two of six filters negatable, each with its own matcher
15What does it cost, how often does it run, and is anyone reading it?131% of a core, p90 at 30%, on a converged device
  1. A sidecar hash match is NOT evidence that the .md came from the op log. sidecar.last_synced_hash == file_hash(disk) answers exactly one question: did outl write these bytes last? It does not answer did these bytes come from the op log? Those are different questions, and a page can answer yes to the first and no to the second — that is precisely the state a reconcile_md leaves behind when it rewrites the sidecar without emitting ops covering everything it read. So never overwrite a .md on the strength of the hash gate alone. Before re-projecting, ask outl_actions::content_lines_missing_from(disk, &sidecar.blocks) and refuse when it returns anything (ActionError::PageMarkdownAheadOfLog). Ask it against the sidecar’s blocks, never against a fresh render. The sidecar is what the log held when the two last agreed, so it answers “does the log know this line”; a render answers “do disk and tree disagree”, and that is also yes for every remote edit, remote delete and reorder. The first version of this guard compared against the render and therefore refused to re-project any page a peer had touched — issue #166 reintroduced, with the blame moved. That function is the single owner of the verdict; a second opinion about which pages are safe to overwrite is how a read-only listing promises a repair the writing pass then refuses. A refusal has to reach the user. Refusing freezes the page in both directions until outl reconcile --ahead-of-log runs. A client that swallows the error into a log line therefore ships a page that silently stopped syncing — the same “silence is the defect” failure this invariant exists to prevent, moved one layer up. Every surface names it: docs/clients.md → Surfacing a page that stopped syncing.

    Why this is an invariant and not a code comment. Issue #166 fixed a real bug (tree ahead of .md, page renders empty) by re-projecting whenever the tree outran a faithful projection, where faithful meant the hash matched. That traded one divergence direction for its mirror image, and the mirror is worse: .md ahead of tree is content deleted, while tree ahead of .md was only content hidden — the op log still had it. On a real 2,560-page workspace that cost 233 pages holding 1,426 lines of work that existed in no op. They were deleted by doctor --repair (which printed 708 fixed) and by every GUI page open, with the rebuilt sidecar making the loss undetectable afterwards. Full reasoning, rejected alternatives and what this change makes worse: RFC 0210.

    The producer is fixed too, not just the guard. The state above only exists because render → parse was not a roundtrip for a block whose text held a blank line or its own indentation. outl-md’s permissive parser used to close continuation on the first blank line (dropping everything after it) and silently skip an unplaceable line at any depth below 0. Both are fixed in crates/outl-md/src/parse.rs; see crates/outl-md/CLAUDE.md → “What this crate owns” and its own invariant 8 entry. reconcile_md now enforces invariant 8 in code — it will not advance last_synced_hash over content it could not emit an op for. The matching side has the mirrored guard: reconcile_md_with_guard refuses a bulk delete instead of trashing an oversized orphan list (crates/outl-md/src/matching/guard.rs). outl recover (op log, a truncating Op::Edit’s still-live predecessor) and outl reconcile --ahead-of-log (the .md, content the log never saw) are the two routes to recover the 1,426 lines this incident already produced. See docs/cli.md → “outl recover”. Full status: issue #210.

    The general rule this is an instance of: when you fix one direction of a .md ↔ tree divergence, state what happens in the opposite direction before merging. Reconciliation bugs come in mirrored pairs, and the pair that deletes is never the one being reported.

    The verdict is about the bytes on disk, not about the sidecar’s shape. The gate is outl_md::unlogged::sidecar_can_vouch_for; sidecar_can_answer is one arm of it and answers a narrower question. Asking only the second conflates a pre-0.11 sidecar with a page nobody has written in yet — both carry text: "" on every block, and the second has nothing to lose. That froze every fresh journal day and every unfilled [[link]] page the moment a peer wrote into one, silently, since Ok(None) is what reproject_stale_md reads as success (issue #332). Measured: one such page and zero genuine pre-0.11 sidecars on a 2,874-page workspace. It narrows which pages are refused, never which content is protected — a line the log does not hold still refuses, phrased by the same unlogged_content_error.

    The regression net. These tests exist to fail if someone re-simplifies the gate back to a hash comparison — do not delete or relax them: if_stale_refuses_when_the_md_carries_content_the_log_lacks, if_stale_still_reprojects_when_the_md_holds_no_unlogged_content, if_stale_ignores_whitespace_only_differences_when_deciding, if_stale_declines_when_the_sidecar_cannot_answer (an empty verdict from a reference that cannot answer is not permission to write — that is how a peer on an older binary re-arms the loss), if_stale_still_declines_when_an_unanswerable_sidecar_covers_real_text (its other half: the decline is decided by what is on disk, so the narrowing above cannot reach the pre-0.11 case), if_stale_reprojects_a_page_whose_only_block_is_empty (the #332 direction: a page holding one bare bullet has nothing to lose and must be re-projected), if_stale_still_projects_a_page_whose_sidecar_has_no_blocks (the opposite case: nothing on disk to lose) (crates/outl-actions/src/journal/tests/if_stale.rs), plus the_sweep_reprojects_a_page_whose_only_block_is_empty and the_sweep_still_declines_when_an_unanswerable_sidecar_covers_real_text (crates/outl-actions/src/journal/survey/tests.rs — the same pair through the classifier outl serve’s sweep and doctor read, because fixing the write path alone leaves them blind to it), plus crates/outl-md/tests/unlogged_vouching.rs (the predicate itself, both directions), plus recovery_does_not_reproject_over_text_the_log_never_saw (crates/outl-actions/tests/desync_recovery.rs, the same defect reached through the desync recovery’s re-projection), plus a_torn_op_log_never_lets_repair_overwrite_a_good_md (crates/outl-cli/src/cmd/doctor/tests/safety.rs), which pins the precedence order: a damaged op log is reported as the damaged log, not as 2,000 pages of “unlogged content”. The producer side has its own net, pinning that a line is never dropped rather than just refused-to-overwrite: crates/outl-md/tests/multiline_block_roundtrip.rs (blank lines, indentation and unplaceable lines round-trip, never truncating the op log on the next reconcile). crates/outl-actions/src/recover/tests.rs (the truncation signature and the additive-only restore).

  2. When state crosses a boundary, say what its new home requires. Moving state out of the workspace, out of a file, out of the op log, or into a process global does not delete its problems — it hands them to a place with different rules. The RFC that moves state must answer four questions about the destination: who can write it, who can read it, how does a test get its own copy, and what cleans it up?

    Not theory. 0.11.0 moved the write actor out of .outl/config.toml into a device-local store, correctly — the file rode every sync transport except iCloud, so two devices resolved the same actor and appended to one ops-<actor>.jsonl. Question three went unanswered, so the test suite wrote into the developer’s real ~/.config/outl (64 entries, 15 of them orphaned TempDir paths) and three doctor tests went flaky. Question four went unanswered for longer — nothing pruned a workspace the user deleted, and actors/ reached 1,208 records on one machine, 1,166 of them orphaned. outl-core’s device/gc.rs answers it now, reported by outl doctor and dropped by --repair. What that took is the part worth carrying. The obvious rule (“delete the ones whose directory is gone”) is wrong: an unplugged drive and a deleted folder are the same observation, and a wrong delete forks the workspace’s actor — the failure the store exists to prevent. A cleanup answer is only as good as what it refuses, so the rule is three conditions, not one, and every reading it cannot make faithfully keeps the entry. RFC 0211, issue #211.

    The general rule, of which invariant 8 is one instance: a fix relocates a problem far more often than it removes one. After “did I fix it?” comes “where does the problem live now, and what does that place require that the old one did not?” Three separate defects in this codebase came from skipping that question — a divergence fixed in one direction only (invariant 8), state moved without an isolation story (this one), and a size guard with no escape hatch that turned into a wall.

  3. When you change who decides, enumerate who was standing on the old decision. Replacing a hardcoded policy with a runtime one is usually the right fix, and it is also the change most likely to break something that policy was quietly holding up. A policy has beneficiaries that never had to declare themselves, because under the old rule they always won.

    Not theory. Issue #220: “only the GUI binds the iroh endpoint” was a policy that fixed a real collision and assumed a GUI exists, so a headless outl mcp serve machine synced with nobody, silently. Replacing it with a lease was correct. What the change missed was everyone leaning on “the GUI always wins”:

    • Desktop pairing read the transport straight out of app state, because a GUI that always held the endpoint always had one. Losing the election made adding a device impossible.
    • outl peer status was documented as exempt from “never bind a second endpoint”, on the grounds that the CLI has no running transport to conflict with. True under the old policy. False the moment the MCP server could hold the endpoint — and status is the command a user runs to diagnose sync.
    • $OUTL_DEVICE_DIR already meant “throwaway actor”. Giving it a second job (moving the iroh identity) silently rotated the node id for anyone already exporting it.

    Then the guard itself. The lease was stored on the transport rather than on the endpoint thread, so a failed bind() killed the thread while the transport kept the claim, locking every process on the device out of an endpoint. That is the #220 bug again, this time with a padlock. Releasing it any earlier reopens the mirror image: two endpoints on one node id while the first is still closing.

    Before merging a change that moves authority, answer:

    1. Who consumed the winner? Grep the consumers of the resource, not just its producers. Every caller that assumed a particular process would own it is a caller you just broke.
    2. What does each loser do now? “Degrades gracefully” is a claim, not a design — name the path, and check it is reachable.
    3. Does the guard die with the thing it authorises? Too late strands the resource forever, too early admits a second owner, and both fail silently.
    4. Which exemptions existed because the old rule held? An exception is an argument and arguments have premises. Re-check them; the exemption may have outlived its reason.
    5. Did the name already mean something? A flag, env var or file you are giving a second job still has its first one, and somebody is depending on it.

    The general rule: invariant 9 asks where the problem moved to. This one asks who was standing on the thing you moved.

  4. A cost only argues against a design if that design causes it. Rejecting an approach because it is expensive is sound only after you have attributed the expense. A cost the alternative pays too is not a tiebreaker — it is a separate problem, and folding it into an unrelated decision buys a worse design and leaves the real problem unfixed.

    Not theory. Issue #244 asked for a background sync daemon. outl serve was already the long-lived background process and already had no P2P at all, so the obvious move was to give it the transport. That got rejected on the grounds that a permanently-running serve holds the device write actor, which pushes every later GUI and TUI launch onto a fresh ephemeral actor and a fresh ops-<ulid>.jsonl. 16 such files, 1 to 221 ops each, had already accumulated in a real 20-actor workspace. The conclusion drawn was “so sync needs its own command”.

    The attribution was wrong. The actor cost comes from running any process permanently, not from what that process does while it runs. outl serve under launchd holds the write actor whether or not it also syncs. The proposed second command only dodged the cost because it dropped the file watcher. A flag drops the watcher just as well, at the price of one boolean instead of a second daemon, a second launchd job, and a second thing that can die quietly.

    Before a cost rules a design out:

    1. Would the alternative pay it too? If yes, it cannot choose between them, and quoting it as though it could hides that you have no argument yet.
    2. Does the cost track the feature, or the lifetime? “Runs forever” and “does X” are different properties, and only one of them is the thing you are deciding.
    3. What is the smallest thing that avoids it? A flag, a parameter and a whole new entry point are not equivalent. The reflex to reach for the largest is strongest exactly when the argument feels settled.
    4. Did it already exist? A pre-existing cost you happened to discover while investigating is a finding that deserves its own issue, not evidence about the change in front of you.

    The general rule: invariant 9 asks where a problem moved to, invariant 10 asks who was standing on what you moved. This one asks whether the problem you are avoiding is even caused by the thing you are changing.

  5. A capability without a per-client verdict is a gap nobody recorded. Every client does not do everything, and that is fine. What is not fine is that the difference has no owner, because then it is discovered by a user pressing a key that does nothing.

    Not theory. “Does the desktop bind y r?” had three answers in three places and they disagreed. docs/shortcuts.md listed y r (CopyBlockRef) and : (OpenCommandPalette) as desktop chords — neither had a handler, so both were dead keys that logged to a console the user never opens. The same doc listed mobile undo / redo as “toolbar” when mobile had neither, and Journal.tsx carried a comment saying so — closed now by RFC 0254 (#14). The desktop dispatcher’s own comment called its console.warn something “the user sees”. Three hand-maintained copies of one fact, each stale in a different direction, and nothing that could fail.

    outl_shortcuts::support owns it now: one exhaustive match, so a new Action variant does not compile until all three clients have declared their verdict. docs/client-parity.md is generated from it and pinned by the_parity_doc_matches_the_code; shortcuts.support.test.ts pins the desktop’s handler map against it in both directions. A capability with no chord — page history, the plugin marketplace, a calendar grid — is invisible to that match. outl_shortcuts::capability_support is a second exhaustive match, over Capability instead of Action, reusing the same Support states and the same generated, pinned doc table (RFC 0253).

    Two things that took a second pass and are worth carrying:

    • The reason text belongs in the catalog, not the client. A client that writes its own wording is a fourth copy of the fact. The desktop’s charCursorNudge was already the right shape — one shared message for ten actions — and it still lived on the wrong side of the boundary.
    • “Reachable” and “has a handler” are different questions. Backspace on an empty textarea works on the desktop and has no handler, because the platform does it. A boolean would have forced that row to lie in one direction or the other, so Support::Native is its own state: reachable, no handler, no nudge.

    The general rule (9 → 10 → 11 above): this one asks who does not have what you just built, and does anything fail if you don’t say?

  6. A colour token has exactly one meaning, on every client. A token name that resolves to bg in one client and bg_elev in another is not a naming inconsistency — it is two definitions of one fact, and the wrong one gets selected by whatever signal happens to be wired to it.

    Not theory. applyPaletteToRoot() mapped --color-ios-bg to palette.bg but --color-iosd-bg to palette.bg_elev, so on the desktop iosd meant elevated. Mobile’s stylesheet used the same prefix for something else entirely: dark. MarkdownInline.tsx — one shared component — read both namespaces and reached the iosd set through Tailwind’s dark: variant, which resolves off prefers-color-scheme. So the operating system appearance setting changed the elevation of markdown blocks on the desktop, with no relationship to the theme the user picked.

    The sharpest part: the regression test caught a real bug on its first run — a smaller one than this invariant used to claim. The desktop’s @theme boot block declared --color-outl-ref-link where the Palette field is ref_link_fg, so that token had no boot value. That is not Tailwind dropping the class: Tailwind v4’s bg-(--var) / text-(--var) arbitrary-property shorthand resolves a CSS custom property at paint time whether or not @theme declares it. --color-outl-status-normal-bg, --color-outl-help-title-fg and --color-outl-status-insert-bg are three tokens absent from @theme today and still render correctly in the built desktop bundle — proof the class was never silently dropped. What the missing boot value actually cost: the first painted frame, before applyPaletteToRoot() runs over the wire, had no value for --color-outl-ref-link-fg. Refs, tags, markdown links, inline code and TODO markers rendered with that one property unset (inherited/transparent) for that single frame, then repainted correctly the instant the palette landed. @theme’s job is supplying that boot default so the first frame is already branded, not making the utility resolve at all — the utility resolves either way, with or without @theme.

    Colours come from outl_theme::Palette and reach a client through applyPaletteToRoot(). A hex literal in a client stylesheet is a second definition. The one exception is a client’s @theme boot block, which exists so the first painted frame is branded before the palette arrives over the wire. the_theme_tokens_match_the_palette pins those names to real Palette fields, which is exactly what caught the missing boot value above.

    The regression net: no_client_references_the_legacy_ios_namespace, the_theme_tokens_match_the_palette (crates/outl-theme/tests/tokens.rs). RFC 0022.

    The general rule (9 → 12 above): this one asks does this name mean the same fact everywhere it appears?

  7. A filter that cannot be negated is half a filter, and the negation is never a second implementation. Every predicate a user can write — a ```query directive, an outl query flag, an MCP filter argument — ships with its complement in the same change. The complement is ! over the positive’s own matcher, applied by one generic wrapper, not a NotFoo sibling that re-answers the question.

    Not theory. The ```query DSL shipped with six positive filters and no way to say “not”. “open work tasks, minus anything parked under #someday” was unexpressible, which is a feature gap. The first fix was worse than the gap: two hand-written variants, NotTag and NotProp, giving two of the six filters a negative and leaving four without one — and the two that existed each carried their own copy of “does this block have tag x”.

    Two copies of one predicate drift, and the direction a negative filter drifts is always the one that silently removes results. That asymmetry is the whole reason this is an invariant. An over-inclusive positive shows the user an extra row they can see and ignore. An over-inclusive negative deletes a row from the answer, and a user cannot notice a block that is not there. tag: matched by substring, harmlessly, for months; negated, that same substring would have hidden every #workflow block behind a not-tag: work written to hide something else.

    So the mechanism, not the discipline:

    • not-<key> parses <key> and wraps the result in Filter::Not(Box<Filter>) (crates/outl-exec/src/runtimes/query/dsl.rs). The engine’s only negative arm is Filter::Not(inner) => !matches(inner, …).
    • A new directive is therefore negatable the moment its variant and match arm exist. Adding a hand-written NotFoo variant is the thing to refuse in review — it reintroduces the second opinion this wrapper removed.
    • What is not a filter has no negative: sort and limit are rejected as unknown keys under a not- prefix, rather than read as some reversed ordering.
    • The same pairing holds on surfaces that cannot share the enum. outl query’s flags are hand-paired, so each pair is pinned by a test instead: query_every_filter_and_its_negation_return_nothing.

    A filter whose complement reads oddly still gets the honest complement. since: 7d means “a journal dated within 7 days”, so not-since: 7d matches every ordinary page too — they were never journals. Inventing a friendlier meaning (“older than”) would be a second matcher wearing a helpful name, and tag: x + not-tag: x returning rows is exactly the bug that buys. Document the surprise; do not paper over it.

    The regression net: no_filter_and_its_negation_can_both_match (crates/outl-exec/src/runtimes/query/engine.rs) iterates the whole Filter set rather than the pairs that shipped first, so a new variant with a broken negation fails there. every_filter_key_has_a_negative, sort_and_limit_have_no_negative, a_double_negative_is_an_unknown_key_not_a_positive (dsl.rs), every_directive_has_a_working_negative (crates/outl-exec/tests/query_negative_filters.rs), query_every_filter_and_its_negation_return_nothing (crates/outl-cli/tests/cli_machine.rs).

    A filter that can only match nothing fails loudly. not-tag: with no name, not-text: with no needle, not-prop: status: with a dangling colon, a tag name outside the tokenizer’s alphabet — each parses to a predicate no block can satisfy, so the negative quietly excludes nothing and returns the rows the user asked to hide. Every one is a parse error.

    The general rule (9 → 13 above): this one asks can the user say “not this”, and is that answered by the same code that says “this”?

  8. Being fast is the product, so a repeated question is answered once and a cheap answer is never bought with an expensive read. Speed is not a bar applied after a feature works. It is the feature, and it fails in a shape no test catches: every function is reasonable on its own, everything is correct, and the app cooks.

    Not theory. The sync census (outl_sync_iroh::oplog_census) was already the cheap read, optimised once when the iOS watchdog killed the app over the full open it replaced. That fix cut the constant and left the shape — O(total ops) per call, nothing memoising it, no caller aware another had just asked. One exchange asked six times and opened storage twice: ~710 ms of CPU per peer per exchange on a 748,684-op workspace, every 8 s, to move zero ops on a converged device. Measured on a real machine: 30 % of a core at p90, 131 % at peak, and a window that stopped taking keystrokes. No test failed.

    Four questions, and the first was skipped three times:

    1. Who else is asking this, in the same breath? Grep the read’s callers, not just the one you are writing — six call sites with one answer is one read, passed along.
    2. Is anyone looking at this answer? An expensive column computed behind a closed panel is waste, and a polling loop is where waste multiplies.
    3. What does it cost at the size a real graph reaches? Not the fixture’s nine ops: 750k ops, 170 MB of sidecar, 23 actors. An O(total ops) read gets slower every day the user keeps using the app.
    4. Does the cheap path stay cheap on the branch that does something? An early return is not a budget.

    Measure on a real graph, on the device. A process sample found this while the whole suite stayed green.

    The net is structural, because a timing test is flaky and a structural one is not: the no-open tests delete the sidecars and fail if either comes back. The pre-existing fast-path test passed throughout, because it only ever reached the early return — a fast-path test that skips the working branch is how this hid.

    And the net is run against the defect it refuses, or it is not a net. The second incident’s first ratchet matched the source string tree().properties_of(, which rustfmt had already broken across three lines, so it scored zero on the file it existed to refuse while reporting itself verified. Its other assertion — that the file reaches the shared pass — was satisfied by a doc comment naming it. So the pre-fix file is committed verbatim as a fixture and the ratchet is pointed at it, and the ratchet reads idents off a token stream rather than text, because a formatter decides where the lines go and a comment is not a call.

    A second incident added one more question, and it is about the diagnosis rather than the code. “Saving a backlink edit is extremely slow” was reported with a four-item cause already attached. All four were real, summed to ~85 ms, and none was the problem: the reply every mutation returns read the whole workspace property map once per block, because Tree::properties_of does that and its doc comment said it does not. 2,588 ms to project one 688-block page; the in-page edit_block nobody complained about was equally slow on the same page, which is what should have been measured first. So: measure the path that is supposedly fine, and treat an accessor’s doc comment as a claim, not a measurement.

    The incidents, the measurements, the rejected cache design and the per-client checklist: docs/performance.md.

    The general rule (9 → 14 above): those ask whether a change is correct. This one asks what it costs, how often it runs, and whether anyone reads the result — because here a correct answer delivered slowly is a wrong answer.