Transcripts

The agent writes JSONL to a file. Zimmer polls that file, normalizes it, and streams it to your browser. That’s the whole loop. Each step has a wrinkle.

The pipeline

Where a transcript is stored

A transcript is one JSONL document, and session.transcript still hands you the whole thing as a String. Underneath, it lives in session_transcript_chunks — line-aligned slices of at most 256 KiB, ordered by seq, whose concatenation is the document byte for byte.

It used to be a single sessions.transcript column, and that is what made a poll expensive. Postgres cannot update a TOASTed value in place: UPDATE sessions SET transcript = … writes a new tuple, re-compresses and re-stores the whole document, and WALs it. So recording ten new lines cost a function of everything the session had already said — every five seconds, for the life of a session, on exactly the long autonomous runs Zimmer exists for (#110).

Writing through the chunk table costs what was added instead:

The write	What it touches
The incoming value extends what is stored (every ordinary poll)	The open tail chunk, plus new chunks for the overflow. Nothing already closed is rewritten.
The incoming value is byte-identical (a poll with nothing new)	Nothing.
The incoming value diverges (a rotation carryover, a recovery merge, a fork’s truncation)	The whole chunk set is replaced, and the byte counts on both sides are logged.

“Extends” is decided without reading the transcript back. sessions.transcript_byte_size and sessions.transcript_digest (SHA-256 of the whole document) live on the session row, so the check is a hash of a value the caller already has in memory. sessions.transcript_line_count is on the row for the same reason: the regression guard below compares line counts, and computing that used to mean detoasting megabytes to count newlines.

“Byte-identical” is decided the same way, by Session#transcript_matches?. Most polls find nothing new, so this check runs more than any other. The agent loop reloads the session before every poll, which drops whatever was read last time, so comparing transcript != incoming would read the whole conversation out of the chunk table on every poll of every running session. With twenty-odd sessions running, that saturated production Postgres on 2026-09-12 and wedged the default lane behind it (GlitchTip #99). The check compares the size and SHA-256 of the normalised incoming value against the row instead. It reads the stored text back only for a row that cannot vouch for its bytes: one the backfill has not reached, or one with no digest. The same holds on the poller’s other per-poll paths. A Codex session still on the rollout it was on last poll, with no history carried over, does not read the stored transcript. Neither does an idle poll on a followed re-keyed branch.

A write from a stale copy of the session cannot desynchronise the two either. If a writer re-saves the value it last read after another writer has committed a newer transcript, nothing on the row changes, and the chunk set is left describing the newer transcript the row’s digest vouches for.

Two invariants the writer maintains, both relied on elsewhere:

Every chunk but the last ends at a line break. Content search matches per chunk, and a boundary inside a JSON event would make a phrase silently unfindable. A boundary between events cannot, because that newline is a record separator, not something anybody typed.
Chunk line counts sum to the document’s line count, so transcript_line_count is exact.

One byte does not survive the move: a raw NUL. The old json column stored a transcript as a JSON string, so Rails encoded 0x00 as the escape \u0000 and Postgres never saw the byte; text does see it and rejects it. Session.normalize_transcript strips NULs on the way in — every write goes through that one funnel, so the chunks, the digest and the byte count always describe the same bytes — because the alternative is a poll that raises inside its own transaction and freezes that session’s transcript for good. Nothing readable is lost, and the line count is unaffected.

OpenTranscripts — the normalization layer

Claude, Codex and Pi write completely different JSONL. Rather than teaching the UI three dialects, Zimmer normalizes into OpenTranscripts v0.1 (app/services/open_transcript.rb), a vendor-neutral schema vendored from pulsemcp/ai-artifacts. Nine event types:

UserMessage · AssistantMessage · Thinking · ToolCall · ToolResult · SubagentSpawn · Compaction · Error · SystemEvent

Every one renders through a single partial (app/views/timeline_items/_item.html.erb), keyed on type. One raw JSONL line can fan out into several events.

Zimmer’s copy diverges from upstream on purpose: per-line normalization with no cross-line timestamp carry-forward, several fields hardcoded to null, Zimmer-internal adornments on every event (sort_time, transcript_index, event_order), and redaction applied one layer up rather than during conversion. The divergences are listed in vendor/open_transcripts/README.md.

Keeping the copy honest

A hand-written mirror drifts, and a mirror that drifts silently is how an upstream fix never reaches Zimmer. Two checks, one per link in the chain:

Link	Checked by	When
Zimmer’s Ruby ↔ the pinned snapshot	`test/services/open_transcript_drift_test.rb`	every CI run, offline
The snapshot ↔ upstream `main`	`.github/workflows/open-transcripts-drift.yml`	daily, on demand, and on PRs touching the vendored files

vendor/open_transcripts/ holds a byte-for-byte snapshot of the upstream files Zimmer mirrors, plus UPSTREAM.json pinning the commit and a SHA-256 per file. The offline test re-derives those digests and asserts that OpenTranscript still declares the nine event-type discriminators and the schema version the snapshotted spec does. The scheduled workflow re-fetches upstream and fails when the bytes have moved; alert-ci-failure.yml listens on every workflow in the repo, so that failure lands in Slack. Refreshing the snapshot is a deliberate act with a diff to read — the procedure is in vendor/open_transcripts/README.md.

Who wrote this line

Claude Code writes lines into its own transcript that carry type: "user" but were never typed by a person. Two of them show up constantly in Zimmer:

Flag on the JSONL line	What the CLI is recording
`interruptedByShutdown: true`	a turn killed mid-tool-use, by a process shutdown rather than a keyboard interrupt
`isMeta: true`	context the CLI injected rather than a person typing — its resume scaffolding (“Continue from where you left off.”), and the whole body of a skill when one fires

In Zimmer the shutdown behind the first one is usually Zimmer: Sessions::InterruptService SIGTERMs the CLI to deliver an enqueued message ahead of the running turn. Rendering the resulting line as a UserMessage told the reader a human had acted — the opposite of what Zimmer knows, and it cost the deployment’s owner a full trace across the logs, the raw JSONL and the normalizer source to establish that he had not interrupted a session he had never touched.

ClaudeTranscriptNormalizer routes both to SystemEvent under the subtype OpenTranscript::SystemEventSubtypes::RUNTIME_NOTICE, and the timeline draws them as a Runtime Notice — the cyan system glyph, not the indigo user one — above a line naming the flag that marked the line machine-written. The CLI’s own wording inside the text is left alone: “[Request interrupted by user for tool use]” says “by user” and is Claude Code’s string, not Zimmer’s. The attribution Zimmer wraps around it is Zimmer’s, and that is what carries the correction.

The flags are matched against a literal true. Anything looser — truthiness, key presence — would relabel a turn a person really did type as machine-written, which is the same misattribution with the sign flipped, so both directions are covered by tests.

One line wears type: "assistant" and belongs in the same bucket: the stand-in reply Claude Code inserts on resume when the history it loaded ends on a user message, as it does after its own “Continue from where you left off.”. It is model: "<synthetic>" with one text block, “No response requested.”, and the model never wrote it. Drawn as an assistant turn, it reads as the agent declining to act on each resume. That misled an investigation of Slack routers stuck through a 529 outage: the model had never been reached at all. The normalizer matches both the synthetic model and the exact text, and marks the notice model: <synthetic>. That is a matched field value, not a JSONL flag. Session#formatted_conversation drops the stub, so the push-notification summary built from the last assistant message never quotes it. Every other synthetic entry, including the “API Error: 529 …” lines the CLI writes the same way, still renders as an assistant turn.

A runtime notice stays in the message filter bucket and keeps its fork affordance. It is not a message, but it sits in the conversational slot the CLI wrote it into, and “the turn was cut off here” is context a reader on the minimal filter still needs. A flagged line with no text is not made into a notice at all — there is nothing to attribute — so it stays on the message path, where a content-less message is already suppressed at render and an image-only one still shows its image.

The timeline is not the only surface that was asserting a person. TranscriptTextRenderer (the plain-text export behind GET /api/v1/sessions/:id/transcript and the get_session MCP tool’s transcript_format: "text") and the session page’s own copy-to-clipboard both read parsed_transcript — raw JSONL, never normalized — so both apply the same discriminator directly, via ClaudeTranscriptNormalizer.runtime_notice_markers. Fixing only the normalizer would have left an agent reading the text export being told a human typed the line.

Codex gets no equivalent treatment: its rollout response_item message payloads carry only role and content, with no per-line marker that distinguishes a machine-written user turn from a typed one. There is nothing unambiguous to key off, so CodexTranscriptNormalizer is left alone rather than given an invented discriminator.

Large notices render collapsed

isMeta carries more than the resume stub. It is also the flag on the line Claude Code writes when a skill fires, and that line is an entire SKILL.md — the smallest one on a production Zimmer host is 3.4k characters, the largest 690k. Printed in full, two or three of them push the conversation off the page.

So the timeline folds them away. A collapsed notice draws one muted line — the skill’s name, an approximate token count, and a disclosure control — and the body renders unchanged as soon as it is opened. Nothing is removed: the full text is still in the row, still what the copy button copies, and the plain-text export never comes through this path at all.

Two things trigger the fold, and both live in SessionsHelper#ot_runtime_notice_digest:

Trigger	Header label
a first line reading `Base directory for this skill: <path>`	the skill’s name, from the path’s last segment
anything else longer than `RUNTIME_NOTICE_COLLAPSE_CHARS` (2,000)	the notice’s own first line, truncated

Both are floored at RUNTIME_NOTICE_FLOOR_CHARS (400): folding a short notice away behind an accordion is worse than leaving it alone, and that holds for a short skill dump too. The thresholds sit far above the scaffolding the same flag carries — “Continue from where you left off.” is 33 characters. The token count is a character-count estimate at four characters per token, labelled approx. in the UI: Zimmer runs no tokenizer at render time and the number is not a billing figure.

It is a <details>, not a Stimulus controller, for the reason the session hierarchy is: no JS to load, and it works under touch and keyboard. The one controller it has to tell is auto-scroll, whose ResizeObserver watches the container this row sits in — expanding a body worth tens of thousands of pixels is indistinguishable from new content arriving, and a reader who was tailing would be thrown to the bottom of the transcript, past what they just opened. The toggle clears tailing, the same as scrolling up does.

Secret redaction

Zimmer hands its agents real credentials. MCP ${VAR} values are interpolated into .mcp.json inside the clone, a Claude session’s OAuth access token is in its environment as CLAUDE_CODE_OAUTH_TOKEN and its MCP tokens in its own .credentials.json, git pushes over an authenticated remote. An agent that cats one of those files, echoes an environment variable, or pastes a curl -H "Authorization: Bearer …" into its own reasoning puts that credential in the transcript — and the transcript is stored, rendered, and downloadable through the transcript-archive API.

TranscriptRedactor runs inside TranscriptSource#read, where transcript bytes are pulled off disk. That is deliberately on write, not on read: Zimmer stores the whole raw transcript string, so redacting at the read boundary is what keeps a credential out of the database rather than only out of the rendered page.

Reading through that method is a requirement on callers, not a property of the code — Zimmer has one other place that re-reads a transcript and persists it: the manual refresh, Sessions::RefreshTranscript, which the web UI, the REST API and the action_session MCP tool all call, each in a single and a bulk form. It resolves its reader through TranscriptRuntime.source_for(session).read for exactly this reason; a bare File.read there writes an unredacted transcript over the redacted one the poller stored. test/contracts/transcript_redaction_contract_test.rb pins that structurally, so a new refresh path cannot quietly reintroduce the bypass.

It works in two tiers:

Known values. Every ${VAR} the AIR catalog’s MCP servers reference, resolved through the same SecretProviders chain that injects them into a session, plus the OAuth tokens Zimmer holds in its own database. Exact string matches: no false positives, and they cover Zimmer-specific credential formats no regex could know about.
Shapes. High-confidence patterns — sk-ant-oat01…, ghp_…, xoxb-…, JWTs, PEM blocks, Authorization: Bearer …, credentials embedded in a URL, and a value sitting immediately after a name that says “credential”. Ported from the upstream reference redactor and extended with what this system actually handles.

Redaction preserves line count exactly (the regression and rotation guards below compare line counts) and leaves no reversible fragment of the original.

What a marker tells you

A marker has to answer two questions for whoever is reading the transcript: why did this fire, and how much did it take out. It carries the tier, what fired, and the length of the span it replaced.

Marker	What it means
`[REDACTED:ENV:SLACK_BOT_TOKEN:56ch]`	Tier 1. The text was the value Zimmer resolves for that catalog variable.
`[REDACTED:OAUTH_TOKEN:88ch]`	Tier 1. It was an OAuth token Zimmer stores in its own database.
`[REDACTED:MATCH:GITHUB_TOKEN:40ch]`	Tier 2. It matched a shape. Nobody confirmed it was a live credential.
`[REDACTED:MATCH:ENV_SECRET:61ch]`	Tier 2, the generic name-then-value rule — the loosest one here.
`[REDACTED:UNSCANNABLE_LINE:9412ch]`	No pattern pass finished on this line, so the whole line went.

MATCH: is the load-bearing part. Everything carrying it is a guess, so a reader who recognises the length and the framing can tell “that was my GITHUB_TOKEN” from “something here looked like one”. The character count is disclosed on purpose: redaction is defense in depth over material already treated as secret, most labels imply the length anyway, and the count is what lets a reader recognise their own value in the gap.

Resource names are not credentials

The generic name-then-value rule fires on ⟨a name that says credential⟩⟨:|=⟩⟨a long value⟩. Two of the names it knows — the bare nouns secret and token — are also the nouns every cloud CLI uses for the resource, so --secret=NAME, ?secret=NAME and secret: NAME put a public identifier exactly where the rule expects a credential.

That cost a human a runnable command: an orchestrator session handed over gcloud secrets versions access latest --secret=… and the argument came back as [REDACTED:ENV_SECRET], so the instruction identified no secret (session 6512). A Secret Manager secret’s name is what gcloud secrets list prints to anyone with read access; the credential is the version’s payload, which was never in the message.

After those two nouns only, the rule now declines a value that is shaped like a public identifier: lowercase and digits, in at least three separator-joined segments, no segment longer than 12 characters, and more than half the segments carrying a letter outside the hex alphabet. That last condition is what keeps a UUID (550e8400-e29b-41d4-a716-446655440000) and a hyphen-grouped hex key redacted — both are real session-token formats and neither has a letter in [g-z]. Compound names (client_secret, api_key, password, passphrase) are untouched by any of this, so password=correct-horse-battery-staple still redacts.

Placeholders are not credentials

Every rule that redacts a value first checks whether that value is a fill-in-the-blank. An agent writing setup steps for a human puts the blank exactly where those rules look: http://<proxy-username>:<proxy-password>@p.webshare.io:80 came back with the password slot as [REDACTED:MATCH:URL_CREDENTIALS:37ch], which made the step harder to follow and hid nothing (session 24427).

A value is left alone when the whole of it is one of these, with only a name inside the delimiters: <proxy-password> (or its JSON-escaped \u003c…\u003e form), ${PROXY_PASSWORD}, a bare $PROXY_PASSWORD or $proxy_password, {{proxy_password}}, %PROXY_PASSWORD%, or one character repeated (xxxxxxxx, ********). In practice the delimited forms matter for URL credentials and database URLs; the Bearer, X-API-Key and name-then-value rules never capture <, $, { or %, so only the repeated-character filler reaches them.

Some lookalikes still redact on purpose. ${VAR:-default} does, because a shell or compose default is routinely the real password. So does a “name” that mixes digits with both letter cases (<Xk9mQ2vL8pR4tZ>), a mixed-case bare $Var, and anything where the placeholder is only part of the value (abc<def>ghi). The prefixed-token rules run first, so a full-length ghp_… token wrapped in angle brackets is still caught by its own shape. The same check covers a prompt a human pastes into a session: that prompt is a record in the same transcript and goes through the same redactor.

Why the patterns carry their own regexp timeout

config.load_defaults 8.0 sets Regexp.timeout = 1 for the whole process. That cap is sized for request-scoped strings, and it applies to a single search rather than to a gsub as a whole — so a transcript reaches it in two ordinary ways once it grows into the tens of megabytes. A gsub is a sequence of searches, one per stretch between matches, and a rule with no literal to seek scans a whole transcript that holds none of its shape as one search. And a single line carrying a multi-megabyte base64 tool result is one uninterrupted run of the generic ENV_SECRET rule’s value class, which its greedy repeat consumes in one match.

Both were measured on a real 32 MB session transcript, at 1.0 s and 2.2 s. The resulting Regexp::TimeoutError escaped TranscriptSource#read into TranscriptPollerService, which logged Error polling transcript for session N: regexp match timeout and dropped the whole transcript update — every poll, for as long as the session stayed alive (#472).

Every pattern that can be handed transcript-scale input therefore carries its own TranscriptRedactor::SCAN_TIMEOUT (10 s), which takes precedence over the global cap. Like the cap it replaces it bounds one search, not a whole pass, so the figure it should be read against is the slowest single search: 2.2 s, four times under it. It is a backstop against one runaway match rather than a latency budget for redact, which takes ~7.6 s on that 32 MB transcript and is not a quantity the timeout ever measures.

If it is reached anyway, the pattern pass retries line by line and replaces the single line it cannot finish scanning with [REDACTED:UNSCANNABLE_LINE], so a pathological line costs that line rather than the whole update. Retrying that way finds exactly what the whole-string pass would have found, because no pattern can match across a newline.

Why a poll only redacts the newly appended bytes

Redaction costs roughly 245 ms per megabyte, and the poller re-reads the transcript every few seconds for as long as the session lives. That used to mean ~7.6 s of CPU per poll on a 32 MB transcript, to re-derive a result byte-identical to the last poll’s for everything but the few kilobytes the agent had appended — a cost that tracked session length rather than session activity, and the poller’s dominant expense once several long sessions were alive at once (#477).

TranscriptRedactionCache sits between TranscriptSource#read and the redactor and keeps the already-redacted prefix of each path, so a poll pays only for the new bytes. Measured against real session transcripts on this host: a 22.4 MB one went from 8.5 s to 34 ms on a warm poll, a 17.9 MB one from 2.8 s to 43 ms, and in every case the incremental output’s SHA-256 is identical to the full re-scan’s.

That is sound because redaction is line-decomposable: splitting the text at a newline and redacting the halves separately gives the same bytes as redacting the whole. No pattern carries /m or uses . at all, every value and gap class either excludes \s or is an explicit [ \t], the known-value tier admits only values matching \A\S+\z, and a UTF-8 sequence never contains a \n byte. It is the same invariant the line-count guarantee and the line-by-line degradation above already rest on.

The one exception is the multi-line PEM walk, which spans newlines by construction. So the commit point never crosses a line that could open a block: the committed prefix contains no opener, so no block can straddle the cut. A transcript that carries PEM armor simply stops advancing its commit point and re-scans from the opener — never worse than re-scanning everything, which is what every poll used to do.

Invalidation is explicit, because a transcript is not always append-only — it can be truncated, rotated to a new Codex rollout, replaced by a new session reusing the path, or rewritten by the resume restore. The cache re-checks three things on every read: the file is no shorter than the committed prefix, the commit point still lands immediately after a newline in the bytes being read now, and 16 KB fingerprints of the file’s head and of the bytes just before the commit point still match. Any mismatch falls back to a full re-scan.

The middle check is the one that carries the safety weight. A stale prefix is not a leak — the cached prefix is itself redactor output, so re-emitting one produces a wrong transcript, never an unredacted one. The only way an incremental scan can under-redact is if the tail begins mid-line, which is exactly what re-asserting the newline rules out.

A re-clone lands where the old one was

Claude Code names its transcript directory after the cwd it was spawned from. That makes the working directory the identity of a conversation’s transcript directory — and clone directory names carry a timestamp and a random suffix ({repo}-{branch}-{timestamp}-{random}), so a conversation that gets a new clone gets a new slug and the whole JSONL is written out again underneath it. The previous copy stays behind at full size.

That is not a rare event. A clone is reaped once its session is archived or has failed — and an unarchive, a trigger following up, or a recovery then brings the same conversation back. Production held one conversation in 23 clone directories, 18 MB each, one per day, and 286 conversations in more than one copy — 595 MiB of pure redundancy (#576).

So a re-clone goes back to the path the session already occupied. SessionClonePath.for_recreate answers that question for both callers that rebuild a clone under an existing conversation — AgentSessionJob’s follow-up path and UnarchiveSessionService — and GitCloneService generates a fresh path only when it declines.

It declines unless the stored clone_path is a direct child of ClonesDirectory.base and there is nothing at it. That second condition is what makes reuse safe rather than merely tidy:

git clone needs an absent or empty destination, and create_clone’s rollback deletes whatever is standing at the path it was given. Only an absent path is ever handed back.
There is exactly one reason a conversation gets a fresh clone mid-life: the old one is gone from disk. Nothing re-clones because a tree is dirty or a branch moved — AgentSessionJob reuses a surviving clone exactly as it finds it, uncommitted changes and all, and UnarchiveSessionService takes its quick path whenever the clone and working directory are both still there. So there is no stale checkout to inherit at a reused path, because the reason the code is there at all is that there is no checkout.
A session whose branch moved still clones session.branch; only the directory name spells the branch it was first cut for, which is cosmetic. A session whose agent-root subdirectory moved in the catalog still adopts the new subdirectory (#921), so its cwd — and its transcript directory — legitimately move with it.

One consequence worth naming: a transcript directory can now come back to life. Before this, deleting the clone at a path killed the transcript directory named after it for good, which is why CloneReaper reaps the two together. AtomicCloneRemoval renames the clone aside and then rm -rfs a whole working tree, and a session resumed inside that window re-clones at the same path. So CloneReaper re-asks who owns the path after the delete and before the transcript reap, and leaves the transcript alone if the answer changed.

This fixes the generation of duplicate transcript directories, not the ones already on disk. Those are #434’s half — OrphanTranscriptDirectoryCleanupJob — which now works through a backlog that stops growing.

Writing a transcript back to disk

The stored transcript is the durable record; the file on disk is what the runtime actually resumes from. Three paths re-materialize the former as the latter — resuming a session whose clone was recreated (AgentSessionJob#write_transcript_to_clone), unarchiving (UnarchiveSessionService), and forking (ForkSessionService).

None of them compute a path. They all ask the session’s TranscriptSource:

TranscriptRuntime.source_for(session, file_system: file_system)
  .resume_transcript_path(session: session, working_directory: working_directory)

For Claude Code that is ~/.claude/projects/<sanitized-cwd>/<session_id>.jsonl — the same file locate prefers, so the runtime resumes from exactly what the poller reads.

Pi has one too, and gets it for a different reason. Pi resolves --session-id against the id recorded inside a session file rather than against the filename, so Zimmer writes the stored bytes to one fixed name — <clone>/.pi/sessions/zimmer_session.jsonl — and Pi picks the session up and continues appending to its leaf. That is why PiTranscriptSource can support single-file restore while CodexTranscriptSource cannot.

The same rule holds for reading: the manual refresh (Sessions::RefreshTranscript) and the four process-recovery services take the directory and the file inside it from the source — transcript_directory then find_main_transcript. Pairing one runtime’s directory with another’s file-picker is how a Codex session ends up searched with Claude’s flat <session_id>.jsonl rule: it finds nothing at best, and at worst adopts an unrelated rollout that happens to sit at the top of ~/.codex/sessions.

nil means “this runtime has no single-file restore”, and it is not an error. Codex is the only runtime that returns it: its rollouts are date-partitioned, UUID-named and possibly Zstandard-compressed, so there is no one deterministic path to write stored bytes to. Every caller skips the write on nil and carries on; a Codex fork and a Codex unarchive both succeed, having written nothing. Writing a Claude-shaped file the runtime will never read would not have helped, and gating the operation on that write turned a no-op into a failure.

A write that raises is still a failure, and fork and unarchive still abort on it — they distinguish “nothing to do” from “the write did not land”.

A re-keyed transcript, and why the name is only a preference

sessions.session_id is the id Zimmer supplies at spawn, and Claude Code honors it — so <session_id>.jsonl is normally the file, for the life of the session. Normally. A transcript can also be re-keyed: a file opens with a byte-identical copy of this session’s conversation and then carries on under a different uuid. Preferring the recorded name when the copy is the file being written means polling a conversation that stopped (#1047).

Zimmer produces that shape itself, and knowing so is half the rule. ForkSessionService writes the source session’s stored transcript to the fork’s <session_id>.jsonl, so every fork’s file opens under the source’s id and then re-keys to the fork’s own. That is a different conversation owned by a different Session row — following it would show a session its child’s work. What is left after excluding it is the same shape with no Session row behind it: a runtime that re-keys its own transcript in place, against which selection by name has no defence at all, and whose failure is total and silent.

So the recorded name is a preference. TranscriptFileLocator lets it compete with any sibling in the same directory proved to be this same conversation, and the most recently written of them wins:

Both halves of that middle box matter:

The head, not the mtime. A candidate qualifies only if the first sessionId in its opening lines is this session’s. That is the evidence #1047 used to tie a branch to its session, and requiring it keeps this from decaying into “newest .jsonl wins” — the rule the pre-session_id fallback is deliberately narrow to avoid, because a working directory can hold a previous occupant’s transcript. Only a file written more recently than the recorded one can displace it, so mtime prunes the candidates before any of them is opened: the ordinary session, where the recorded file is also the newest, costs a glob and a stat per sibling and opens nothing.
Not a fork. A fork’s transcript is copied verbatim from its source, so its early lines carry the source session’s id — the same shape a re-key has. A uuid another Session row already holds is excluded outright, so a fork that ran in this working directory can never be mistaken for its source’s continuation. This is the same hazard that keeps capture_runtime_session_id! switched off for Claude, and it is why nothing here rewrites sessions.session_id: that column stays the durable, uniquely-indexed identity, and the live branch is represented separately.

Following the branch is only half the repair, because the branch is not a superset of what Zimmer stored. The copy was taken at a point in the conversation; whatever the abandoned file recorded after that point exists nowhere else. RekeyedTranscriptBranch merges the two — the stored transcript, extended by whatever the branch adds — and reads as three terms in order:

Term	What it is
`H`	the head the branch copied, shared with the stored transcript
`A`	what the abandoned file recorded after the copy — held nowhere but `sessions.transcript`
`T`	the branch’s own tail

stored is H + A the first time and H + A + T on every pass after that, so the work is to find how much of T is already at the end of stored and append only the rest. That is recomputed from the two texts every time, never remembered. A remembered split point is wrong the moment the stored transcript stops being prefix-shaped — which is precisely what the first merge does to it — and the error compounds on each further re-key. Recomputing also absorbs the two things that go wrong in practice: a read taken mid-flush, whose trailing partial line is dropped and re-supplied from the branch, and a poll that could not open the branch at all.

In the ordinary re-key the branch was seeded with the whole recorded file, so it already starts with everything stored and the merge is one string comparison.

Every writer of sessions.transcript goes through it, not just the poller: the manual refresh, Sessions::RefreshTranscript — one implementation behind the web UI’s, the REST API’s and the action_session MCP tool’s single and bulk refreshes — starts from find_main_transcript and then guards the write with Session.transcript_regression?, which compares line counts, so a branch longer than the stored transcript but missing its tail passes and takes A with it. A writer that skipped the merge would undo it for all of them — which is what the REST API’s bulk refresh did while it kept its own copy of the sequence.

metadata["transcript_branch_session_id"] records which file a session is being polled on. It is a breadcrumb rather than state the merge depends on — #1047’s complaint is that the failure was invisible — and it is retired only on an affirmative “the recorded file is the one being read”, which is what a resume produces: AgentSessionJob#restore_regressed_transcript_if_needed re-materializes the stored transcript at <session_id>.jsonl, now including everything the branch contributed, before the runtime reads it.

The regression guard

If the clone is recreated, the agent starts a fresh transcript file. Naively overwriting the stored transcript with it would wipe the session’s history.

So Session.transcript_regression? refuses to overwrite a stored transcript with a shorter one, and logs it once via metadata["transcript_regression_detected"]. If a resume hits an unrepairable regression, the resume is refused outright — because resuming would silently drop the user’s prompt.

Codex has one extra recovery case. A failed codex exec resume can be recovered by starting a fresh Codex process for the same Zimmer session, which creates a new rollout beginning at line 1 with a new runtime thread id. When the poller sees that recovery rollout and it does not already include the stored recovery boundary, it appends the new segment instead of treating it as destructive replacement, records metadata["transcript_recovery_segment_appended"], and broadcasts from the old transcript boundary. The same poll also repairs a stale broadcast_message_count that points past the new transcript, so the recovered messages are not silently skipped.

That guard exists because the underlying condition happens.

Append-only storage does not retire it

It would be tidy if moving to an append-only chunk table made truncation structurally impossible, and it does not. The guard is not an artifact of how the transcript was stored — it encodes a policy: a file on disk that is shorter than what Zimmer recorded means the clone was recreated and the runtime started over, not that the conversation got shorter. Only the caller knows which reading applies, and both readings occur — the Codex recovery merge two paragraphs up is a deliberate rewrite to a shorter document, and it is correct.

What the chunk store changes is the shape of the risk rather than its existence. A truncating write is now an explicit delete_all of a session’s chunks, logged with the byte counts on both sides, instead of an UPDATE indistinguishable from every other one — and the guard itself got cheaper, because session.transcript_regression?(incoming) reads the stored side’s line count off the session row instead of loading the conversation to count newlines in it.

Rotation: when a shorter file is new, not lost

The guard above assumes one canonical transcript file per session, which is true for Claude Code and for Pi, and false for Codex. Codex rollouts are append-only and immutable, and codex exec mints a new rollout UUID for every run that is not a resume — so when a resume fails and Zimmer fresh-starts the turn, the conversation continues in a brand-new, initially tiny file.

TranscriptSource#rotates_transcript_files? is what tells them apart: false for Claude and Pi (shorter means lost, refuse and repair), true for Codex (shorter means next file, carry history forward). Pi never rotates because --session-id continues an existing session tree rather than starting a new file, and a Pi resume whose transcript vanished is repaired from Zimmer’s stored bytes before the spawn.

On a rotation the poller folds the whole stored transcript into an immutable prefix and records its length in metadata["transcript_carryover_event_count"], alongside metadata["transcript_source_path"]. Later polls re-derive the prefix as the first N lines of what is stored — the stored transcript is always carryover + live — so the timeline reads as one continuous conversation across any number of rotations. Requiring the path to change is what keeps a partially flushed read of the same file from duplicating history.

Rotation and the recovery marker overlap

A failed-resume recovery rollout is usually both things at once: a recovery segment and a rotation to a new file. The two mechanisms are keyed on different signals, so which one re-attaches the history depends on the shape of the recovery rollout:

Shorter than the stored transcript — that is a regression on a new path, so rotation sees it and carries the history forward. It gets there first, in the same poll.
Longer than the stored transcript — not a regression, so rotation re-attaches nothing (it still replays the prefix an earlier rotation recorded, and still stamps the new transcript_source_path). Only the recovery marker knows the stored history has to survive, so the recovery path rebuilds the transcript as recovery boundary + live rollout. Rebuilding from the live rollout rather than from what rotation handed it is what keeps an already-carried prefix from being counted twice.

Either way the poll records metadata["transcript_recovery_segment_appended"]. The marker is a statement about the session — a recovery happened and its segment is in the stored transcript — not about which of the two mechanisms happened to fire. Unlike transcript_recovery_expected and transcript_recovery_base_line_count, which are turn-scoped and cleared when the turn exits, this one is sticky and diagnostic: it stays set for the life of the session, and nothing in the app reads it.

The other half of the same failure is the id the poller looks the file up by. Codex mints its own rollout UUID, so after a fresh start the stored session_id names an abandoned rollout; ProcessLifecycleManager#release_stale_runtime_session_id! drops it so CodexTranscriptSource#find_main_transcript can find the live rollout instead. Without that, the locator kept returning the dead file and capture_runtime_session_id! could never learn the new UUID — it reads that UUID from a file the locator would never hand it.

Codex is told its UUID; it no longer has to be inferred

That circle — the locator needs the UUID, the UUID comes from a file only the locator can find — is why the fallback existed at all. codex exec --json closes it. Zimmer was already passing the flag and discarding stdout; the adapter now captures that stream into codex_events.jsonl inside the working directory, and CodexEventStream reads the thread UUID off its first line ({"type":"thread.started","thread_id":"…"}), printed before the model is even contacted (#109).

Two consumers, and both of them are earlier than they used to be:

Consumer	What it does with it
`TranscriptPollerService#capture_runtime_session_id_from_stream!`	Runs at the top of `poll_and_broadcast`, before the transcript is located, so `sessions.session_id` holds the real `codex exec resume` target from the first poll — including on a poll that finds no rollout at all
`CodexTranscriptSource#find_main_transcript`	Globs the rollout by that UUID instead of inferring it

The clone-path inference survives as the fallback for the window before the first line is flushed, and for a rollout Zimmer adopted without having spawned it. capture_runtime_session_id! likewise survives as the second capture point. Both are now backstops rather than the primary path.

The seam is TranscriptSource#runtime_session_id, which defaults to nil — “no side channel, use the transcript”, which is the right answer for Claude and Pi, whose stored session_id Zimmer supplied in the first place. ForkSessionService sheds the event log along with the stderr log (RuntimeCliAdapter.spawn_artifact_paths), because it names the source session’s thread.

Rotation repairs the timeline, not the agent’s memory

Carryover keeps Zimmer’s view of the conversation whole. It does nothing for the runtime’s own view: a fresh start is a new conversation, and the new process is handed only the prompt that triggered it. So the two halves of a lost rollout fail differently — the timeline still reads correctly, while the agent answers as though the conversation never happened.

That makes the durability of the runtime home a correctness property, not an ops nicety. resume resolves a conversation by its file on disk — ~/.claude for Claude Code, CODEX_HOME for Codex — so a runtime home on the container’s writable layer is destroyed by every deploy, and the following turn cannot resume. Both homes are mounted as durable named volumes (see Deploying). Pi is the exception that proves the rule: its conversation is not in its home at all, because --session-dir points inside the clone, so what PI_CODING_AGENT_DIR holds is credential and provider state rather than a transcript. Its home is mounted on the same terms anyway — losing auth.json every deploy is milder than losing a conversation, and just as invisible until a session cannot authenticate. test/config/runtime_home_volumes_test.rb pins all three for every role, because the symptom of getting it wrong is not an error — it is an agent with amnesia and a timeline that still looks fine.

The transcript loads tail first

The Transcript disclosure on the session detail screen starts collapsed, like every disclosure on the page. Collapsed is not unloaded. Its body is an eager <turbo-frame> pointing at GET /sessions/:id/transcript_panel?filter=<level>, fetched in the background as soon as the page is on screen. A reader who opens the panel finds the rows already there, not a skeleton.

That response is the tail and nothing else: the newest TRANSCRIPT_TAIL_LIMIT (10) items, because the end of a transcript is what a reader almost always opens it for. It is read backwards from the end of the stored transcript through Session#each_transcript_line, which fetches one stored chunk at a time and stops once the page is full. The newest ten items of a 14 MB transcript therefore cost the last chunk or two, not the document. That makes the frame’s cost flat in session length.

Everything older pages in through GET /sessions/:id/timeline_items, and the first page is prefetched. As soon as the tail lands, infinite-scroll#prefetchEarlier fetches the 40 items before it in an idle callback, so a reader who scrolls up finds them already in place. Later pages are 100 items, loaded as the reader scrolls toward the top. Each page carries two cursors:

before_timestamp is the boundary: only items strictly older than the oldest one shown. A page is never cut between two items with the same sort_time, because the next page could not ask for the other half of a tie. A log and a transcript line stamped the same second is the ordinary case.
before_line is a hint: the transcript line the next backward read starts below, so a page does not re-read lines the reader already has. The read runs BACKWARD_READ_OVERLAP (20) lines past the page boundary in both directions. Transcript lines are written in time order but not strictly, and the page is cut by sort_time.

At show-logs and verbose, logs are a second source, cut by the same timestamp in SQL (at show-logs, with verbose logs left out of the query so they cannot fill a page’s limit) and merged before the page is taken.

Lines with no timestamp of their own (Claude’s last-prompt, mode and cost-state bookkeeping records, visible at show-logs and verbose) sort at the session’s created_at, which a full build puts at the very top of the timeline. They cannot take part in a backward read: counted against a page they would fill it and drag the cursor to created_at, ending the history early. So they are left out of every page but the last, which collects all of them in one full pass of the transcript, the one full read paging ever does, and only for a reader who reaches the start of the history.

The total arrives separately. compute_filtered_count is the one figure that cannot be read from a tail: an exact count of what the active filter would show means normalizing every transcript entry. So the tail says “Showing latest 10 items”, and once the prefetched page is in, infinite-scroll#fetchTotal asks timeline_items?total_only=1 for the count and writes “Showing 50 of 1,288 items”. The transcript half of the count is cached against transcript_digest and the line and byte counts, so a session that has stopped writing pays for it once. A tail with nothing older is the whole timeline, and states its own size.

A #message-N link into older history reads back to it. The tail holds ten rows, so a link from the Status panel usually names a row that is not on the page yet. transcript-panel#reveal asks infinite-scroll#loadThrough(N), which sends through_index=N. The server keeps reading backwards until that transcript index is in the page, and returns everything between it and the oldest row shown, in one request. A fragment that still matches nothing (an index past the end, or a row the active log level hides) leaves the panel open on the newest row.

On a session with a 5,002-line, 14 MB transcript, measured against main on a development server:

	before	after
Detail page response	248 KB, ~390 ms, 35 queries	248 KB, ~256 ms, 36 queries
Transcript frame	621 KB, ~790 ms (100 rows + full count)	65 KB, ~46 ms (10 rows)
Newest rows visible	after a click, 1.4–2.5 s from navigation	without a click, ~0.8–1.1 s from navigation

The detail page itself got faster because two checks on it had been materializing the whole transcript on every render. One is the Status panel’s “could Regenerate run?” check, now RuntimeConversationPresence.stored_transcript_conversation?, which stops at the first conversation record. The other is Session#never_ran?, which now asks transcript_present?.

Three things stay outside the frame, and each is load-bearing:

The append target. #session_<id>_timeline is rendered on the page, just after the frame, and starts empty. Turbo Stream appends target it by id (see Broadcast bookkeeping), so a message broadcast while the frame is in flight still has somewhere to land, and lands below the batch the frame brings, which is where the newest items belong.
The log-level select, which lives in the sticky header and has to be there to be usable.
The disclosure itself, so the reader can see there is a transcript to open.

Both containers are live at once, so one of them has to give a row up. A message broadcast while the frame is in flight lands in the append target; the frame then renders the server’s current tail, which can contain that same message, just above it. Nothing dedupes across the two by itself — Turbo’s append only compares direct children of its target, and a frame swap does no id reconciliation at all — so transcript-panel#dropStreamedDuplicates runs on every turbo:frame-load and removes any row from the append target whose id the frame has just brought its own copy of. It is keyed on presence inside the frame, which is what makes the in-flight case safe: a row broadcast after the server rendered its response is not in the batch, is not a duplicate, and stays where it landed. Both copies carry the id SessionsHelper#timeline_item_dom_id derives, which is deliberately identical across the broadcast and render paths — that shared id is what makes the check possible at all.

The reconnect backfill needs the same treatment from the other side: a fetched copy of the page has an unloaded frame and so cannot carry the rows. The recovery fetches the panel’s own URL for any deferred frame that has loaded — for the transcript, the wider data-backfill-src (a 100-item tail), because a reader away for a while can have missed more than ten rows — grafts its batch onto the append target’s id before reconciling — which keeps a recovered row in the same container, and therefore the same order, as one that arrived over the socket — and sweeps the panel’s empty-state placeholder if the server has stopped rendering it, which the region reconcile cannot see because it sits beside the append target rather than inside it. See The reopen backfill.

Broadcast bookkeeping

The poller only broadcasts new_messages[broadcast_count..], where broadcast_count comes from metadata["broadcast_message_count"] (recomputed from the stored transcript when nil). This is what prevents the entire transcript from replaying into your browser on every poll.

Note the sharp edge in that expression: Ruby returns nil, not [], when the index is past the end of the array. Any change that can make the parsed transcript shorter than broadcast_count therefore stops the timeline dead rather than degrading — which is exactly how the rotation bug above stayed invisible.

There is also an ownership guard: the poller skips if session.running_job_id != job_id, which is what stops two monitoring jobs from double-broadcasting the same session.

Subagent transcripts

When an agent spawns subagents (Claude’s Task tool), each writes its own agent-*.jsonl. The poller discovers them, stores each as a SubagentTranscript row, and links it back to the parent Task tool call by matching tool_use_id against toolUseResult.agentId — filling in subagent_type, description, status, duration_ms, total_tokens, and tool_use_count.

They render as a nested, collapsible accordion inside the parent’s timeline row.

Transcript hooks

A small Ruby plugin system that runs inside Zimmer (not inside the agent) whenever new transcript messages are broadcast. Sequential, error-isolated per hook, run after the transcript is saved. Each hook writes into session.custom_metadata.

GithubPrUrlHook records the pull requests a session opened — read out of create output (gh pr create, a REST POST to repos/OWNER/REPO/pulls, or an MCP create_pull_request tool call) or the agent’s own “opened PR <url>” prose — into custom_metadata["github_pull_request_urls"]. That list is what the GitHub PR poller, the comment poller, and the merge-conflict poller all key off, so a PR missing from it is invisible to Zimmer, and a PR wrongly on it sends another session’s comments here. GithubCommentAuthorshipHook ships alongside it, recording the comments a session posted so the comment poller never reads one back as if a human wrote it.

→ Transcript hooks for the contract and how to write one.

Searching what was said

GET /api/v1/sessions/search?search_contents=true and the quick_search_sessions MCP tool (search_contents: true) both match the stored transcript, so a session can be found by a phrase from its conversation rather than by its title. A multi-word query is matched as one literal substring — the words have to be adjacent and in order, against the transcript’s stored JSON.

There is no index a leading-wildcard substring match can use, so the scan is bounded rather than best-effort: SessionContentSearch walks candidates newest-first in chunks, stops at the result limit or a wall-clock budget under the proxy’s timeout, and always returns. What it hands back with the results is how far it got — complete, scanned_sessions, candidate_sessions and a next_cursor to resume from. An empty result with complete: false means “not found yet”, which is a different claim from “not there”, and both surfaces say which one it is. See Searching transcript contents.

Archive and download

TranscriptArchiveJob rebuilds a latest.zip of all transcripts every 10 minutes (temp file + atomic rename). It’s served by GET /api/v1/transcript_archive/download.

The rebuild is incremental in both directions. It reads sessions one at a time rather than loading every changed session at once — a transcript is a single large payload, so holding a whole corpus of them is what used to run the worker out of memory (#719) — and it archives at most MAX_SESSIONS_PER_RUN sessions per tick, recording what it finished before it stops. A backlog therefore drains over several ticks instead of being retried whole. Steady state is a handful of changed sessions per tick, where neither bound is reached.

A run is bounded in time as well as memory. Between sessions it checks a RUN_DEADLINE of ten minutes, one cron interval, and once that has passed it stops, writes the archive with what it did, and defers the rest the same way the cap does. It always archives at least one session, so the frontier keeps moving. The deadline exists because SingletonSweep refuses every tick while a run is in progress, so a run with no bound stops the schedule: on 2026-10-09 one held the slot for over two hours, past the maintenance lane’s 90-minute ceiling, and transcript_archive paged as a stale cron (#1264).

Each run writes the archive once. The old latest.zip is opened read-only, the changed sessions, removals and manifest are applied in memory, and the result is streamed into the temp file, with unchanged entries copied still compressed. The job does not copy latest.zip first and does not commit through rubyzip, which would rewrite the whole file again. A removed session costs nothing extra: its entries are simply not copied.

That makes the job’s peak a function of the largest single transcript rather than of the corpus. Not one copy of it: reassembling a transcript from its chunks holds the chunk contents and the joined string at once, and serializing it builds a third copy before the write. Budget about three times the largest transcript.

What counts as “changed” is the whole of a session’s zip entry, not just the session row. An entry carries the session’s subagent transcripts alongside its own, and SubagentTranscript does not touch: its parent — so writing one leaves sessions.updated_at exactly where it was. The job therefore compares the sidecar against the later of sessions.updated_at and the session’s newest subagent-transcript updated_at, read for the whole corpus in one GROUP BY. Before that, a subagent transcript written after its session’s last archive never reached latest.zip until some unrelated write happened to bump the session row (#720).

A draining backlog is visible in two places, and it has to be, because a catching-up archive is rewritten on every tick and so is never stale — staleness and completeness are different questions. The job logs a deferred to the next tick line at WARN, the severity production ships to its log store; and the metadata sidecar records deferred_count, which surfaces as complete, deferred_count and incomplete_reason on GET /api/v1/transcript_archive/status and as a Complete: line in the get_transcript_archive MCP tool. A reader that must not mistake a partial export for the whole corpus should check that rather than the age.

It writes under ~/.zimmer/transcript_archives — the zimmer_data named volume, mounted at the same path in both the web and worker containers. That matters because the writer and the readers are in different containers: cron runs only in worker (GoodJob starts a cron capsule only in an :async webserver process, and production is :external), while /api/v1/transcript_archive/* and the get_transcript_archive MCP tool are served by Puma in web. While the archive lived under Rails.root/storage — a container overlay layer no deploy config mounts — the reader looked in its own empty copy and could never succeed, and every deploy destroyed the writer’s copy too (#714).

Every tick starts by reclaiming the temp files earlier runs were killed before they could delete. build_archive removes its own temp in an ensure, which covers an exception but not a SIGKILL — and this job is killed routinely, by the OOM killer and by the container swap on every deploy. Each run picks a fresh random name, so nothing ever reclaimed the previous run’s file: production accumulated 219 orphans holding 233.3 GiB, which took the 309 GB root filesystem to 100% and stopped CloneDiskGuard letting the fleet clone (#1160). The sweep matches latest_<hex>.zip.tmp and anything suffixed onto it — rubyzip’s Zip::File#commit writes its own same-sized temp beside the job’s, so one killed run leaks up to two multi-GB files rather than one — and it takes only what nothing has written to for an hour, the same bar the archive is judged stale by, so a build in flight keeps its own. latest.zip and latest_metadata.json cannot match that pattern at all. A tick that reclaims something logs it at WARN; a tick that finds nothing is silent. The sweep runs before the change detection, so it happens on a quiet tick that rebuilds nothing too.

The sweep covers the archive directory and nothing else. rubyzip also stages each entry it writes in its own temp file under the container’s /tmp until the zip is committed, so a killed run leaves up to MAX_SESSIONS_PER_RUN of those behind as well. Those sit on the container’s overlay layer, which every deploy recreates, not on the zimmer_data volume — a different footprint with a different lifetime, and out of this sweep’s reach on purpose.

The archive is a bulk export, not a search index: it is hundreds of megabytes and up to ten minutes stale. To find a session by something said in it, use the content search above.

Rendering to plain text

TranscriptTextRenderer turns a parsed transcript into a reading copy. One class serves both surfaces that need one — GET /api/v1/sessions/:id/transcript and the get_session MCP tool’s transcript_format: "text" — because they previously carried separate copies of the same case and drifted apart.

user, assistant, tool_use and tool_result get a labelled section each — and a user line carrying one of the runtime-notice flags gets --- Runtime Notice (agent runtime, not a person) --- instead of --- User ---, as does the synthetic “No response requested.” assistant line instead of --- Assistant ---. Every other entry type is labelled and dumped rather than dropped, so the text never quietly disagrees with the raw transcript — this matters most for Codex, whose rollout lines are all session_meta / response_item / event_msg / turn_context and would otherwise render as nothing at all.

Content that arrives as an array of blocks is rendered block by block: text, thinking, image, tool_use, tool_result, and pretty JSON for anything unrecognized.

Two truncations keep this a summary rather than a second copy of the file: tool results at 500 characters, unrecognized entries at 1,000.