Nothing to lose and nothing to go on are the same empty string

On the phone, one day of the journal was a single empty bullet.

Every other day rendered fine. That day had been written on a laptop, synced over a Linux outl serve, and the blocks had arrived: the op log held them and the materialised tree held them. Only the .md that the view reads was stale. Typing into the empty bullet made the whole day appear at once.

the tree was ahead of the file, which is the bug that already has a fix

A page whose tree has moved past its .md is issue #166, and the repair for it is apply_page_md_with_sidecar_if_stale in crates/outl-actions/src/journal/apply.rs. Every GUI open path calls it before reading a view off the file. It re-renders the page from the op log and writes, unless one of the invariant 8 questions says no.

Invariant 8 exists because the first version of that repair cost 233 pages holding 1,426 lines of my own notes that existed in no operation. A matching sidecar hash proves outl wrote a file’s bytes last. It does not prove those bytes came from the op log. So before re-projecting, the gate asks whether the file holds content no operation can account for, and the reference it asks is the sidecar’s block list, because that list is what the log held when the two last agreed.

The reference has to be able to answer. crates/outl-md/src/unlogged.rs owns that predicate, and until this change it owned the whole verdict:

pub fn sidecar_can_answer(blocks: &[SidecarBlock]) -> bool {
    blocks.is_empty() || blocks.iter().any(|b| !b.text.is_empty())
}

SidecarBlock::text did not exist before 0.11. Every sidecar written by an older binary carries text: "" on every entry, so it records which blocks the page had and nothing about what they said. Comparing the file against that reference reports every line on disk as unknown. On the workspace that produced RFC 0210, answering anyway would have flagged 615 pages and 35,261 lines against the 233 and 1,426 that were genuinely unlogged. So the comparison stands down, and the caller that is about to overwrite a file declines rather than reading an empty verdict as permission.

That is correct, and it is the whole of the bug.

a page nobody has written in has the identical shape

Here is what outl leaves on disk for a page that exists and has never been typed into:

title:: Notes

-

One block. Its text is the empty string. Its sidecar entry therefore carries text: "", which is byte for byte what a pre-0.11 sidecar carries, and sidecar_can_answer returns false for both.

One shape, two facts that point in opposite directions. A pre-0.11 sidecar means I cannot check. A fresh page means there is nothing to check. The first is a reason to refuse. The second is a page with no bytes at stake, and refusing it freezes it.

The set of pages in that shape is not a corner. A journal day the first time it is opened. Every page created because somebody wrote a [[link]] and never filled it. Both pages outl init leaves behind. Each of them is refused for good the moment a peer’s blocks reach the tree, which is exactly when you would want the projection to catch up.

the refusal is invisible, because Ok(None) is what success looks like

The declining line in apply_page_md_with_sidecar_if_stale returns Ok(None), and reproject_stale_md in crates/outl-tauri-shared/src/helpers.rs reads Ok as “nothing to do”. It is built to surface exactly one failure, ActionError::PageMarkdownAheadOfLog, because that one freezes a page until outl reconcile --ahead-of-log runs. A quiet decline is not that error. No banner reached the view, no line reached the log, and the page rendered whatever the stale .md held.

outl serve’s sweep folded the page into the arm in crates/outl-actions/src/journal/survey/sweep.rs commented “nothing here is a page that silently stopped converging”. That comment was false for two releases.

And outl doctor named it, in a way that was worse than silence. The wording said the sidecar was “written before 0.11”, which is a claim about provenance that check cannot make: all it read was an empty text field. It now says what it actually observed, that the sidecar records no text for any block, and leaves where the sidecar came from out of it.

The quiet was not an oversight. It was argued for, and the argument was good. A text-less sidecar necessarily carries a stale pipeline_version, so the orphan scan has the page queued, and the reconcile that runs on it rewrites the sidecar with text, which arms the real check for the next open. Nothing to tell the user, nothing for them to do.

That argument holds for a pre-0.11 sidecar and it cannot reach an empty page. Being queued was never the difference. The page observed on the real workspace carried pipeline_version 4 against a current 5, so it was queued. Its blocks are genuinely empty, so every reconcile rewrote the same text: "", and the next open refused it again. The self-healing is structurally unable to fix a page whose problem is that it has nothing in it.

the question is about the bytes, so ask it about the bytes

The fix is five lines in the same file:

pub fn sidecar_can_vouch_for(disk: &str, blocks: &[SidecarBlock]) -> bool {
    sidecar_can_answer(blocks)
        || content_lines_missing_from(disk, blocks)
            .iter()
            .all(String::is_empty)
}

A caller about to overwrite a file does not have the question “can these blocks say what the log held”. It has the question “is anything in this file at risk”. The second clause answers that directly, and it answers it through the same comparison the write it guards would make, so the two cannot disagree about what counts as a content line.

Three callers asked the narrow question. One of them already paired it correctly: collect_ahead, behind outl reconcile --ahead-of-log in crates/outl-cli/src/cmd/reconcile.rs, had written “is anything actually on disk” inline next to it. It was the only one of the three getting it right, and the difference between it and the other two was recorded nowhere. sidecar_can_vouch_for is the single owner now, and the write path, the survey’s classifier and collect_ahead all gate on it.

Measured on a 2,874-page workspace: one page frozen in that state, and zero genuine pre-0.11 sidecars. On that machine, every refusal the narrow gate ever produced was a page with nothing at risk. The narrow arm still has to exist, because a peer on an older binary can rewrite a sidecar without text at any time, but on the workspace that reported the bug its only observable effect was wrong.

The other half of why this took so long to notice: the two write gates have deliberately opposite policies here. apply_page_md_with_sidecar_guarded, the one that runs after a real mutation, treats an unanswerable sidecar as permission to write, because there is a genuine edit to project and refusing every older page would freeze the app. So typing into the stuck bullet took the gate that writes. The user’s own keystroke was the workaround, and it looked like the page had merely been slow.

fixing the gate would have left the reporter’s machine broken

The review caught the part that mattered more. The same conflation sat one layer below the gate, in the counting.

content_lines_missing_from normalises a disk line down to the text a sidecar block would hold. The marker and the indent come from the renderer’s layout, so they are stripped, and there is one more case:

        if let Some(body) = t.strip_prefix("- ") {
            return Some(body.trim_start());
        }
        if t == "-" {
            return Some("");
        }

A bare - is an empty block, a first-class state that every Enter in the TUI produces, and it normalises to the empty string its sidecar entry carries. Correct. But then four places counted the resulting entries as content.

lines_removed_by in crates/outl-actions/src/journal/survey/mod.rs is the expensive one. It measures what a re-projection would remove by comparing the file against the new render, and a bare - on disk matches nothing in a reference whose blocks all hold text. So it was reported as a line this write would delete: a line holding no bytes. And in sweep.rs, that count is read before anything else:

            PageProjectionState::Stale { lines_removed } if lines_removed > 0 => {
                sweep.withheld.push(WithheldPage {
                    path: page.path,
                    lines_removed,
                });
            }

The withheld arm sits above the write arm, so the page never reaches the call that would have fixed it. outl serve re-refused the same page every 30 seconds, for good, while outl doctor reported a content loss that could not physically happen.

That is the page shape where a peer types into the journal’s seed bullet rather than appending beside it, which is what the reporter’s laptop did. It is the commonest form of this bug there is. Fixing only the gate would have shipped a release that closed the issue on paper with half of it alive on the machine that reported it.

classify, in the same file, had it too, and there it made the read-only listing contradict the writing pass: AheadOfLog { sample: "" }, sending the user to outl reconcile --ahead-of-log for a page apply.rs was about to write. collect_ahead had it in its own verdict, putting a row with no content behind it into that command’s pick list. And the writer’s own refusal, unlogged_content_error in crates/outl-actions/src/journal/guard.rs, had it as well: an answerable sidecar under a .md carrying one surplus bare bullet, two presses of Enter in an external editor, was refused with an empty sample, for a page both the listing and the recovery command reported clean.

All four count only non-empty lines now:

    let unlogged: Vec<String> = content_lines_missing_from(disk, blocks)
        .into_iter()
        .filter(|l| !l.is_empty())
        .collect();
    let sample = unlogged.first()?;

the_sweep_converges_a_page_whose_only_empty_block_a_peer_typed_into pins the first half, asserting Stale { lines_removed: 0 } for exactly that page, and the_sweep_still_counts_a_real_line_a_reprojection_would_remove pins that a peer genuinely deleting a line still lands in withheld, where outl doctor --repair backs the file up before touching it. Both are in crates/outl-actions/src/journal/survey/tests.rs.

narrowing which pages are refused, never which content is protected

Every change in this area is one mistake away from being the #210 loss again, so the net is two property tests in crates/outl-md/tests/block_text_roundtrip_properties.rs, over the generator that already walks arbitrary block text, and each catches the mutation the other misses.

Reverting the gate to sidecar_can_answer fails a_rendered_page_is_always_vouched_for_by_its_own_log, shrunk to a page whose only block is empty, which is this issue at its smallest. Stubbing the second clause to true fails vouching_on_a_text_less_reference_implies_nothing_to_lose, on the input ["a"], while the first property still passes. One guards against re-narrowing the gate, the other against widening it, and widening is the direction that deletes bytes. crates/outl-md/tests/unlogged_vouching.rs carries the named shapes next to them: a text-less sidecar over real text still refuses, one real line among empty ones is still enough to refuse, and property lines do not block vouching because they ride a different channel.

The frontmatter channel was traced separately and survives both ways. A YAML fence the log does not know is still refused with PageMarkdownAheadOfLog, and one the log does know is re-emitted from the log, which is the guard a block append could satisfy by putting the fence into the log as bullets. A block list cannot answer for a fence, so neither of these two clauses touches it.

what it cost

The gate is still two readings of one field. A sidecar carrying text: "" over a .md holding real text is refused, as it must be, and nothing on that page tells the user why beyond a doctor warning naming it. That decline is still quiet in the clients, and the justification is still the orphan scan’s reconcile arming the real check. It holds for that class. I now know it holds for that class only, which is a narrower claim than the one I was running on.

lines_removed_by and the gate ask different questions of the same comparison, deliberately: one asks whether the log knows a line, the other whether the line survives the write. Both now filter empty entries, and they filter them at four separate call sites rather than inside the owner. That is four places a future change has to find. Pushing the filter down into content_lines_missing_from was the tempting move and it is wrong, because an empty entry is a real line of the file and a caller counting lines of a file should see it.

And the measurement that makes this post’s central number sharp is also its limit. One page, on one workspace, with zero pre-0.11 sidecars on it. The class is large and reachable by construction, and the count of how often it actually bites somebody is a sample of one graph.

the question to ask of an empty result

A guard refuses what it cannot vouch for. That is the right instinct, and the thing it misses is that cannot vouch for has two causes and they usually share a representation. An empty comparison means nothing is at risk when the reference could speak, and means nothing was checked when it could not, and if the shape of a blank reference is also the shape of a blank subject then no caller downstream can tell those apart.

So when a predicate comes back false, do not stop at whether the answer is correct. Ask what the absence of evidence is made of, and whether the thing you are protecting can be built out of the same bytes. Here it could: text: "" was both the sidecar that knew nothing and the page that held nothing, and one of them had everything to lose while the other had nothing at all.

The predicate lives in crates/outl-md/src/unlogged.rs, the two write gates in crates/outl-actions/src/journal/apply.rs with their verdicts in journal/guard.rs, the classifier and the sweep in journal/survey/, and the issue is #332 in the outl repository. The root cause, the reproduction and the shape of the fix are @davclark’s.