Clickable artifact links across the transcript (markdown anchors, prose
file mentions, web source/fetch links, produced-file chips, workflow
member links) now share one language: a dedicated --dsw-alias-link color
(light deepseek-500, dark deepseek-400) decoupled from
state-business-primary, font-weight 500, no underline at rest, and a
dotted 3px-offset underline on hover/focus. A new ui-primitives LinkIcon
leads each link with a currentColor category glyph (url / folder / code /
image / document / other) derived from the path extension by
classifyLinkPath. Produced-file chips drop the grey pill and 96px cap and
render at natural width, shrinking with ellipsis only on row overflow.
ToolRow's grey dotted file links and the grey show-in-folder action stay
as they were. The inline-code chip tint moves to neutral-50 (dark
neutral-800) with its 0.5px border kept.
Coverage: LinkIcon unit spec, refreshed markdown-dom fixtures, and the
clickable-links-gallery web e2e registered in the host compiler face.
Closes#3546
Stop re-exporting the verbatim file-store helpers from attachment-local, move
the raw-byte upload route handler into a non-entry module of file-upload, and
drop the redundant hook type re-exports from the Client entry. The route test
moves next to the module it exercises.
Write handles now hold a durable lease (session.lock.json beside the
log): a random owner token, diagnostic pid, and an expiry. Acquisition
wins by exclusive create; a second process's create or write open rejects
while the record is unrenewed for less than leaseTtlMs (default 5 min),
and takes over after that — a crashed holder is waited out, never
reclaimed by pid. The holder renews every leaseRenewIntervalMs (default
4 min); a renewal that finds a foreign, vanished, or expired record — or
fails outright — marks the lease lost permanently, so every later
append/flush rejects with SessionOwnershipLostError while reads continue.
Close releases the record; read handles never touch it. Takeover of an
expired record is eventually exclusive: a replaced holder stops within
one renewal interval.
Refs #3245
The dsh-family version rule lived only in the packages/ block of
checkWorkspaceManifest, so apps/ members and the root-named CLI carried the
shared version with no static check of their own. The name-based rule now
covers every scanned manifest; regression coverage names the stale-version
merge that shipped packages/util/http-proxy at 0.1.2-alpha.5 behind a
0.1.2-rc.1 family.
`SESSION_FORMAT_VERSION` is the only value that names the Session format
generation. Two other version fields had moved to 2 alongside it:
- The `dsh_session_log` request extension's outer schema `version` returns
to 1 and its Session header projection keeps `seedLength`, derived from
the logical inherited cut, so the external wire is unchanged in this PR;
`sessionFormatVersion` alone identifies the embedded generation.
- The `sessionStats` projection unit's `stateVersion` returns to 1: the
projection cache binds every checkpoint to the format generation, so a
generation change discards old rows without a unit version bump.
Test doubles and the standalone Client fixture that build a current
header now spell `SESSION_FORMAT_VERSION` instead of the literal.
The README file table listed four of the six source modules and omitted
the two largest, so the package read as an oversized identity conversion.
The summary now states that most of the source is the frozen released
v0/v1 event vocabulary, why the edge refuses a malformed historical log
before the installed current restorer runs, and that later edges reuse
those shapes; the table gains `payload-validation.ts` and
`relationships.ts` rows.
The `start` frame and the reconnect baseline attempt carried a wall-clock
`startedTime` that no Host or Client consumer read: the Host accumulator
and the Client reconciler copied it into their attempt maps and nothing
looked at it again. Presentation frames now carry only the coordinates
consumers use (attempt, revision, turn, step, chunk index, and v1 seq
provenance); the type-equivalence block, event graph, READMEs, and the
live-stream Agent Note follow the type.
`session[.vN].jsonl` was assembled independently by the JSONL provider
(twice), the session-log export archive, and the recorded-session fixture
helpers. `dsh-session-format` now owns `sessionFormatLogFilename` and
`parseSessionFormatLogFilename`; the three consumers append only their
compression suffix, and the migration note names the owner.
`snapshotSessionFormatJson` re-implemented the lossless JSON walk that
`dsh-util-values` already publishes as `snapshotJsonValue` + `deepFreeze`,
and the frozen v0 relationship validator carried a third structural JSON
comparison next to `deepEqualJson`. Both now delegate; the format package
reports one `is not lossless JSON` diagnostic per labelled subject instead
of ten member-specific ones, and enumerable accessors whose values survive
a JSON round trip are accepted like `JSON.stringify` accepts them.
`assertFixtureInventory` folds every retained generation of one parent or
child fixture into a single role before comparing, so an older generation kept
beside the current one is not an inventory drift. The generation spec writes
headers that match each filename, since fixture selection validates the chosen
generation, and asserts that an absent parent resolves only for an
override-only replay. The `dsh_session_log` composition expectation follows the
extension's version 2 payload.
The continuable-child headless expectation and the Python `advanced` and
`restart` scenarios pin the current runtime, whose subagent children carry no
`session/end-seed` marker and whose assistant messages embed their streams:
the child fixture drops the marker and renumbers its references, `advanced/`
re-records `result.json`, and both scenarios gain their v2 Session fixtures.
Unit tests cover the `tool-call-delta` id and name validation, embedded
stream members without timing arrays, and the versionless-header refusal.
The JSONL backend rethrows storage errno failures and abort signals from the
source read of a pending migration unwrapped; a test now drives both through
the stat seam so the classification stays covered. The navigation-panes Web
e2e derives the exported archive entry from `SESSION_FORMAT_VERSION`, which
`session-log-export` uses for its `session.v<N>.jsonl` name. The module graph
records the `session-log-export` → `session` peer edge.
`session-format-v1-to-v2` takes the `0.1.2-rc.1` root version that master
requires, and `defineReleasedPayloadDisposition` drops the `@param` tag for a
parameter it no longer has.
`session-log-export` names its archive entry after `SESSION_FORMAT_VERSION`,
a runtime value of the shared `dsh-session` instance, so `dsh-session` becomes
a peer dependency and the export is classified as peer-required in the package
dependency policy. The three Session format packages take the `0.1.2-rc.1`
root version that master now requires.
The repository-only benchmark, its unit-suite spec, the root script, and the
README section are removed; the accepted acceptance run's figures stay in the
embedded-assistant-streams note as recorded facts. The note also records why
the migration edge reuses the `dsh-llm` stream helpers instead of frozen
copies and why its target validation re-checks message/stream agreement
itself. A validation test that recomputed `RELEASED_V2_EVENT_TYPES` from the
same expression as its definition is dropped.
`ensureJsonlGenerationCurrent` returned a live Zstandard body iterator that
every production caller disposed unread before decoding the same bytes again;
the header read now decodes only the first frame and owns no decoder past its
return. The JSONL backend reuses the generation module's stable-revision read
instead of keeping a second copy, `encodeCurrent` encodes the validated
current artifact it is documented to receive instead of re-running the
migration chain on it, and the test-only `inspectVersion` catalog method and
the `migrateSessionSnapshotFixture` duplicate of
`prepareSessionSnapshotFixtureForComparison` are removed.
Second review round (ds-review-bot v4p/v5/v6): the exit cleanup claim that
collectors unlink their spill files on dispose was wrong — completed spill
files are retained (seal() only closes, discardSpill() is the only unlink),
so the exit removal only ever applies to directories that never spilled.
Reword the JSDoc, README, and Agent Note accordingly, drop the redundant
readdirSync precheck (rmdirSync ENOTEMPTY is caught anyway), and recount the
v8-ignore window. The spawn.spec default-spill test now records the default
per-process directory it created and removes it in afterAll, so a completed
spill file is never left behind by the unit suite.
The Agent Note now links the 2026-07-17 local-spill-startup-cleanup note it
builds on, and the privateSpillDir JSDoc documents the empty-dir exit
removal alongside the directory's creation contract.
Review (ds-review-bot) found the exit-time deletion of per-process spill
roots conflicts with the documented retention decision: spill artifacts are
model-visible locators that resumed or forked sessions may still reference
(2026-07-17-local-spill-startup-cleanup), and an exception thrown from an
'exit' listener can change the process exit code.
- dsh-spill-local: revert the exit handler entirely; its default root is
already owned by the package's 30-day startup sweep.
- dsh-subprocess-local/spawn: remove the per-process spill dir at a
JavaScript-observable exit only while it is EMPTY (collectors unlink
their spill files on dispose), best-effort with a named swallow so a
Windows-held handle never changes the exit code; dirs holding spill
files keep them for external cleanup.
- Document the empty-dir exit removal in the package README and the Agent
Note, and re-record both bilingual pairs.
The exit listener runs after the coverage dump, so its body can never be
measured by the unit coverage lane; mark it v8-ignore with the reason, as
with other process-exit-only code paths.
Full-template residue histogram on the CI host surfaced four more normal-exit
leaks below the earlier cutoff: tool-bash/tool-pwsh tools.spec (module spill
dir / per-test homes), bash-sandbox sandbox.spec (module spill dir and the
read-only denial root), and fs-sandbox fs-sandbox.spec (inline tmp dir). All
four now remove what they create at the same teardown points as their
neighbors.
Spec files that create /tmp/dsh-* directories via mkdtemp now track and
delete them in afterEach/afterAll; module-scope fixture dirs (executor
spill dirs) are removed in afterAll. The file list came from the
observed-residue inventory on the self-hosted CI host: only specs whose
dirs actually accumulated were leak sources (issue #3134), superseding
the kept-but-unmerged CI sweep branch per the #3233 review decision.
Product per-process spill roots (dsh-subprocess-local spawn,
dsh-spill-local store) register a process-exit handler that removes the
memoized dir, so processes that used the spawn/spill path clean up on
normal exit. A SIGKILLed process cannot run in-process teardown; the
machine-side timer remains the backstop for that path.
Agent Note: .agents/notes/implemented/process/2026-08-28-test-temp-dir-self-cleanup.md
Review feedback: the dsh-terminal-bash product default is 30s, not 300s;
the raised value bounds one send plus the complete startup sequence, so it
must cover the same cold start the tool deadline does. The vitest case
budget stays at its pre-existing 120s: the case-level timeout overrides
the lane --testTimeout, so syncing it to the plugin deadline would make a
stalled partition wait 300s instead of 120s.
Replies captured on 2026-09-02 from OpenRouter, models.dev, and DeepSeek
replay through the discovery parser, and OpenRouter's nested
top_provider.max_completion_tokens now feeds the output-token cap.
Model requests hand the configured baseURL to pi-ai unchanged. Only the
discovery listing URL accepts the Anthropic API root with or without a
trailing /v1, because gateway documentation publishes both spellings.
The new keyless sdk-spawn-node scenario drives the platform shell tool
through a command starting with node and requires the machine's own Node
version in the tool result with no PKG_EXECPATH in the child environment,
so a pkg upgrade that re-records the child-process patch cannot silently
restore the hijack. Runs on every target through --scenario all.
@yao-pkg/pkg's SEA bootstrap rewrites child_process commands named node --
including the string after a -c flag, exactly the Bash tool's bash -c form --
to the executable itself and stamps PKG_EXECPATH into every child
environment, so a model-issued 'node --version' silently booted the dsh CLI
instead of the machine's Node. Pin the packager as an exact root
devDependency invoked through pnpm exec and patch out the single
patchChildProcess call from the SEA bootstrap bundle; packaged children now
resolve node through PATH like any other process. The third-party notices
drop the build-time tools section: the packager is now a declared, patched
devDependency, so the manifest and patch tiers disclose it.
master released 0.1.2-alpha.5 after this package was created, so the
release bump never reached it; the workspace constraints gate requires
every package version to match the root.
Prefer gh v2.99.0's repeatable --attach flag for publishing demonstration
GIFs: it uploads the artifact and rewrites the body's local-path
reference in place, keeping media out of git history without an orphan
assets branch. The assets-branch workflow remains the fallback when the
GIF exceeds 10 MB, gh is older than v2.99.0, or the repository is not on
github.com.
Review follow-ups: the Agent Note triplet moves to implemented/ rewritten as
shipped state (Decision/Consequences/Testing, present tense), cross-linked
both ways with the 2026-07-28 storage recovery proposal whose projcache
reset/destroy path it supersedes (that proposal stays live for authoritative
and whole-medium damage). The fixtures spec header and the note state the
fixture provenance as recorded facts of the released builds instead of
citing local tooling, and the spec JSDoc points at the note's final home.
The proposed Agent Note records the three shipped on-disk generations of
session_projcache, the read-compat and backup-and-skip decisions, the
upgrade matrix, and the rejected alternatives. The package README documents
the upgrade guarantees and requires every future schema or domain-version
change to land with archived fixtures and tests proving its upgrade story.
The storage subsystem page and the generated cordis catalog pick up the new
DomainSpec fields.
The session_projcache domain declares compatibleVersions: [3, 4] and
invalidRecords: 'backup-and-skip'. The two lineage identity fields become
optional — records admitted from older versions predate them, and the single
reader (identityMatches) interprets absence as the unseeded lineage: exact
for unseeded sessions, while a seeded caller fails the match and refolds
cold, so the lineage binding keeps its protection. Upgraded homes therefore
boot and serve their cached listing titles immediately, including homes
whose new tree already holds current-stamped documents without lineage
fields, and a record failing validation anyway is backed up and skipped
instead of refusing the plugin tree.
tests/fixtures/ archives the real on-disk media of every shipped generation
(v3 whole-unit file, v4 and v5 per-record documents, and the lineage-less
current-stamped shape); fixtures.spec.ts proves each recovers through the
real storage stack, rewrites to the current format on the next live write,
and that a hopeless record is salvaged without costing the boot.
A DomainSpec may declare compatibleVersions: older domain versions whose
stored records the current record schemas still accept. The json backend's
per-record reads admit documents stamped with a declared version (writes
always stamp the current one), and the legacy whole-unit bootstrap migrates
only a file whose stored version is in the accepted set — previously it
migrated any version and stamped the records current, turning a discardable
stale cache into invalid-record failures that refused the whole domain at
open and permanently poisoned the new tree on first boot.
A DomainSpec may also declare invalidRecords: 'backup-and-skip' for domains
whose records are disposable derived data: a stored record failing its zod
schema is moved aside through the new optional KvUnit.backupRecord
(<key>.json.bak.<YYYYMMDDHHmm> under the json backend), logged with its
cause, and skipped, instead of rejecting the open. The default stays
fail-loud, and so do backends without backupRecord.
The first call pays the full pwsh cold-start latency (spawn + .NET +
PSReadLine + Defender) inside the tool deadline. A 60s bound on the fully
loaded self-hosted Windows pool is exceeded often enough to reset the
session mid-test: two master CI runs (2026-09-01, runs 33524764567 and
33534262413) each failed at ~62s with the second call observing a fresh
session (cwd back at root, env empty). 300s matches the product default
so cold start no longer races the budget.
Windows folds `https_proxy` and `HTTPS_PROXY` into one variable, so the
exported spelling shadowed the file's lowercase one and the case failed
there. Each spelling now gets its own name: the file supplies a name the
shell did not export, and the shell outranks the file for the one it did.
The 0.1.2-alpha.4 release on master touched every manifest that existed
there; this package did not, so the merge left it at alpha.3 and
`check-workspace-constraints` refused the tree.
Refusing a response whose tool call never receives an identity needed a new
failure code, a change to the default retryable set, and a `[DONE]` gate that
overrode the finish reason a provider had already sent — turning a safe
`max-tokens` truncation into up to five retries. The lenient wire it guarded
against is hypothetical: no report describes a stream that omits identity
entirely, and the pre-existing test for it is labelled as such.
Only `acceptIdentity` and the widened wire types remain. They close the
reported erasure and cannot reach a worse outcome than the previous
assignment, because the set of inputs that assign only narrows.
`proxyEnvironmentForChild()` hands a child the proxy values the user
exported, including one this package refused — a SOCKS URL kept because
`curl` reads it — and sets `NODE_USE_ENV_PROXY=1` so a child Node honors
them. Node parses `HTTP_PROXY` and `HTTPS_PROXY` under that flag before
running the program and exits on any scheme other than `http:` or
`https:`. So a user with a usable `HTTP_PROXY` and `HTTPS_PROXY=socks4://…`
lost every Node child — stdio MCP servers, subagent CLIs, `npm` in the
bash tool — before its first line, while this process had reported only
that the scheme stayed direct. Measured on Node 24.17: `socks4://`,
`ftp://`, and a malformed value all exit 1; `socks5://` is accepted there
and only there.
The flag is now withheld whenever a value under the names Node parses is
one `isSupportedProxyUrl` refuses. Such a child connects directly, which
is what this process already said about that scheme, and `curl` still
reads the value it was kept for. Node does not read `ALL_PROXY`, so a
refused value there alone changes nothing.
The socks5 case in `install.spec.ts` now asserts the flag absent; the
ALL_PROXY fill case asserts it present; a new case spawns a real child
Node under the overlay for each refused shape and asserts it starts.
A direct policy installed over a proxied one swapped the global
dispatcher but left `process.env` holding the outer install's published
normalization. `proxyEnvironmentForChild()` returns nothing under a
direct policy, and `scrubbedParentEnv()` copies `process.env` as it is,
so a child spawned in that window inherited values no active policy
stood behind: an `HTTPS_PROXY` derived from `HTTP_PROXY` the user never
set, or the loss of a SOCKS value they set for `curl` and this package
had refused.
The direct branch now writes the user's own values — the record the
outermost install keeps — back into `process.env` for the window, and
re-applies the outer install's published values when it ends. An
install underneath that proxied nothing published nothing, so there is
nothing to put back. Publishing and restoring share one `writeProxyEnv`;
the empty-string special case it replaced was unreachable, since a
resolved bypass list always carries the loopback entries and an
accepted proxy URL is never empty.
The nesting is reachable only from tests since the plugin was removed;
the fix keeps the disposer symmetric for whoever layers installs next.
`LlmRuntime.adapterStream` normalizes a thrown `LlmError` into the same error
`finish` the loop routes to `agent/request-error`, so throwing would reach
retry too. The Note claimed otherwise. Yielding is chosen because it reports
the attempt's billed usage first and matches the neighbouring `EMPTY_RESPONSE`
refusal.
The rejection comment repeats the corrected durability wording, and the
assembler's delta-only fallback carries a TODO for the empty name it still
invents for adapters that never close a tool-call block.
The proxy guide told users a proxy could live in a project or
`$DSH_HOME` `.env`. It could not: `loadLayeredEnv` refuses the four
proxy names from any discovered file, as it refuses `PATH` and
`NODE_OPTIONS`, and the launch fails with a pointer to `export`.
That refusal is right for the invoking directory's file — it arrives
with a clone, and a repository must not choose where the harness sends
its traffic — and wrong for the user's own `$DSH_HOME/.env`, which
already holds their API key. `readEnvLayer` now accepts `HTTP_PROXY`,
`HTTPS_PROXY`, `ALL_PROXY`, and `NO_PROXY` from the directory that is
the Harness home, and nowhere else. `DSH_HOME` is itself bootstrap-only,
so no `.env` can relocate the exemption; the CA and TLS names in the
same group stay refused everywhere, since they change what is trusted
rather than where traffic goes. A project `.env` that sets a proxy name
still fails the launch, and its message now names the home file as the
second way out. Launching from inside the home directory reads that one
file as the project layer; the exemption follows the directory.
The seven existing refusal cases all write to the project layer and
pass unchanged. Four new cases cover the home layer accepting both
casings below an exported value, the home layer still refusing
`SSL_CERT_FILE`, the project layer's new message, and the same-directory
launch. The guide, both package READMEs, and the two Agent Notes that
stated the old rule now state this one.
The `MALFORMED_TOOL_CALL` JSDoc claimed nothing durable is written, but the
loop appends an `assistant/chunk` for every yielded chunk; only the assistant
message and tool result are withheld. The bounded-recovery Note still listed
a five-code transient set, and neither Note linked the other.
A keyless `malformed-tool-call-retry` scenario now records the refusal, the
retry, and the absence of a `tool/call` for the failed attempt. A translator
case pins that an already closable block also withholds its `block-end`.
Review found the package's prose still describing a plugin that an
earlier revision removed, and three factual slips about behavior.
- policy.ts, install.ts, install.spec.ts, and the Agent Note named a
`Config` surface, a `cordis.yml` source, a mountable plugin, and a
`plugin.spec.ts` that no longer exist; each now describes the
environment-only resolution the launcher actually runs.
- `NO_PROXY=example.com` bypasses `api.example.com` as well — the matcher
accepts the host and every subdomain under it, and a leading `.` or
`*.` means the same thing. The guide, README, and JSDoc claimed a bare
entry matched only the exact host, which would let a reader believe a
subdomain was proxied when it went direct.
- A rejection diagnostic names the variable and never its value, so no
username is shown; the guide said the username was shown with the rest
masked. The README's source map claimed a "redaction" step that does
not exist.
- The `node:https` worker placeholder was added for a `node:http` agent
factory this PR later removed; nothing imports `node:https` now, so the
stub, its VFS mapping, and its test return to their state on master.
Adding `MALFORMED_TOOL_CALL` to the default retryable set changes two
expected outputs the session snapshot lane does not own: the shipped Web
composition's inline snapshot, and the canonical packed layout of the
refreshed `empty-response-retry` fixture.
The persistence seam is now create/open/stat/list returning per-session
SessionHandles (read/append/flush/close); every log read and write flows
through the owning handle. The seam package exports only the service and
handle contracts, consumer-visible errors, and pure durable-data
validation helpers; each backend owns its complete storage runtime, and
the shared contract suites pin equivalent observable behavior. The
backend routes published sessions' live events by id into the active
write handle; agent-loop only acquires, seeds, and closes the handle.
Resume appends interruptedTurnClosers through its write handle;
session-query owns the revision-keyed cold cache. Legacy-only surfaces
are removed in the same swap: locate/readRaw/supportsRawArtifacts, the
legacy event-shape read migration, zstd torn-frame salvage,
DSH_SESSION_JSONL, and hook transcript_path population; a torn final
zstd frame is discarded whole; the session-list cold blank probe returns
on stat metadata (eventCount derived from the last physical row,
sizeBytes). The WebUI ZIP export serializes the logical log from a read
handle, so both backends export identically.
Refs #3245
A continuation SSE delta that repeats a tool call's `id` or `name` as an
empty string — or as `null`, which some OpenAI-compatible gateways send —
erased the identity established by the call's first delta. The assembled
block reached the loop with an empty name and failed as `unknown tool ""`,
and the empty `callId` persisted into `tool/result`, which the session
reader refuses on reopen.
`acceptIdentity` accepts only a non-empty string, so a repeated empty or
null field means "no update". A tool call still missing `id` or `name` at
`[DONE]` ends the response with the new retryable `MALFORMED_TOOL_CALL`
code instead of closing an unusable block.
Reverting telemetry to its own transport dropped the package's dependency
on the proxy library, and the generated graph still recorded that edge.
The gate that catches this is `verify-module-graph`, which runs in
`check:ci:static` and not in `doc-sync`; the revert was checked with the
latter alone.
Telemetry was the only call site this PR could not cover without changing
the SDK transport underneath it, and both ways of doing that cost more
than the channel is worth.
Routing an `http.Agent` needs Node's `proxyEnv`, added in 22.21 and 24.5
— inside the engines range, so three supported runtimes stayed direct
anyway, and the proxy package had to keep a `createNodeHttpAgent` export
for a path that only sometimes worked. Replacing the transport with the
SDK's `fetch` delegate covered every runtime but has no compression,
while the shipped `base` bundle enables gzip and a realistic OTLP batch
is 6.4x smaller with it; keeping both meant gzipping at the serializer,
which put transport code inside a telemetry plugin.
Telemetry is the one outbound channel whose loss costs the user nothing:
no tool, model request, or session depends on it, and an export that
cannot connect is already dropped silently. A user behind a mandatory
proxy is left where they were rather than regressed.
`src/index.ts`, `otel.spec.ts`, and `tsconfig.json` return to their state
on master; the package keeps only a dev dependency on the proxy library.
`egress.spec.ts` inverts: it installs a policy and asserts the fake proxy
saw nothing, so an SDK upgrade that moved the exporter onto `fetch` would
surface as a failing test rather than silently routing telemetry.
The previous commit moved the OTLP exporter to the SDK's `fetch` delegate
and refused `exporter.compression` at load, on the belief that nothing
shipped enabled it. `packages/bundle/base/cordis.patch.yml` does, so the
refusal broke every test that boots the shipped bundle — the snapshot,
e2e, and Windows observational jobs all failed on that one load error.
Dropping gzip was the wrong trade anyway: a realistic OTLP batch measures
6.4x smaller with it, so trading it for proxy support would have charged
every deployment to fix one. The `fetch` transport has no compression
hook, but serialization is the seam before the body reaches it — the
plugin now gzips there and declares `Content-Encoding` itself.
`keepAlive` and `httpAgentOptions` have no such seam, since they
configure a connection pool `fetch` does not expose, so those two stay
refused at load rather than accepted and ignored. `compression` is typed
as the two values this package can actually apply rather than the SDK's
wider enum, and a third value fails loud at load.
The code-mode → ptc rename rewrote Cloudflare's product name in the
link text and the external URL path, leaving "PTC mode" pointing at
https://blog.cloudflare.com/ptc/ (404). Restore Cloudflare's own name
and the working https://blog.cloudflare.com/code-mode/ link in both
the English and Chinese notes.
The package exported six functions, four of them shaped by one SDK's
transport each: a dispatcher factory, a `node:http` agent factory, a
proxy-URL lookup, and a policy accessor. Review asked whether the call
sites could converge instead of the package growing an export per SDK.
They could, and each removal took a whole shape with it:
- The OTLP exporter moves to the SDK's `fetch` delegate, retiring
`createNodeHttpAgent`. Its Node-version floor goes too: `proxyEnv` on
an `http.Agent` needs 22.21 or 24.5, inside the engines range, so
telemetry was direct on 22.19, 22.20, and 24.0-24.4. The cost is
`compression`, a Node-transport option; the plugin now refuses it,
`keepAlive`, and `httpAgentOptions` at load instead of ignoring them.
- `web-fetch-http` builds its own address-pinning agent under an
annotated `proxy-exempt:` exemption, retiring `createDispatcher`.
Pinning is per-request state a process-wide dispatcher cannot hold.
- E2B reads `route.proxy`, retiring `proxyUrlFor`.
What remains is `installProxyFromEnvironment`, `proxyRouteFor`,
`proxyEnvironmentForChild`, and `clearedProxyEnv` — one per way a caller
can need the policy. Installation absorbs resolution and diagnostic
reporting, which no caller needed apart.
`proxyRouteFor` also closes a defect the old accessor made expressible:
`web-fetch-http` read the policy to decide whether to pin, then read it
again to build a transport, so an unmount between the two returned a
direct, unpinned agent for a URL the first read had cleared as proxied.
A route carries the answer and the transport that answer assumed.
Every egress spec now installs through `installProxyFromEnvironment`, so
no test asserts a policy object a real launch could not produce.
The two styling notes name the surveyed product; the mechanism facts
(superellipse token, guard, full-round opt-out, stroke-in-shadow
elevation, 0.5px hairline) stand alone, so the notes now state them
without the product reference. Pairing records re-recorded.
The composer hairline moves one step up from the menus' l1; the seven
menu-fill dropdown cards grow from 16px to 20px corners. Notes and the
token comment record the new layering.
Re-declare the derived elevation tokens on body * so per-surface
--dsw-elevation-stroke-color rebinds reach the consuming shadow (custom
properties inherit with var() already substituted); pin that mechanism
and add synthetic rejection cases to the stylesheet scans; take the
ring-track basenames through node:path so the exemption matches on
Windows; update the ModelsSection row-card spec to the hairline recipe;
align the elevation note with the shipped l1 menu rebind, refresh the
feedback-popover note's surface recipe, and document the soft tier.
Apply global visual polish across the web client: corner-shape:
superellipse(1.5) with corner-shape: round pairing for full circles,
elevation tokens that draw 0.5px stroke outlines inside box-shadow for
floating surfaces, 0.5px hairline borders and divider lines for
neutral-token strokes, and tuned stroke contrast plus larger radii for
menus, settings panels, and cards. Stylesheet-scan specs in ui-theme
reject unpaired circles, border+shadow mixes, and 1px neutral hairlines
repo-wide.
Closes#3287
Master moved 212 commits. Conflicts were eight `package.json` and nine
`tsconfig.json` files, all dependency-and-reference unions: master dropped the
`runtime-diagnostics/invariants` reference across packages while this branch
added `util/http-proxy`. The lockfile, `tsconfig.base.json`, and the module
graph were regenerated rather than merged by hand.
Master also brought `verify-package-invariants`, which rejects an empty
invariant companion. This package's companion was empty by design, so it is
gone with its publication wiring — the export, the `files` entry, the project
reference, the `dsh-invariants` peer, and the test. Master did the same across
`util/`. The README carries the reason sentence that gate requires.
Carried in the same commit, because the review arrived while the merge was open:
`e2bApiUrl` moves out of the `dsh-e2b` entry into `src/api-url.ts`. Code Entropy
flagged it as a public name with no consumer outside its own package; the seam
keeps its URL-precedence test, which is worth having directly — getting that
order wrong hands a proxy the control-plane traffic and its API key.
Review asked why this is a plugin and why it exports so much. The design note
this branch shipped answered the second question itself — "a pure resolution
function plus an installation function" — and the code drifted to seventeen
exports and a Cordis plugin nobody approved or mounted.
The plugin is gone. Transport policy has one answer per process: nothing to
swap, and no scope narrower than the process to give one. Its `Config` was also
the only supplier of a configuration branch, so resolution now reads the
environment and nothing else — `mode`, the config-sourced fields, and the
`config` policy source were unreachable the moment the plugin left.
Four exports nothing outside the package used are internal again, and
`currentProxyPolicy` answers with the direct policy instead of `undefined`, so
`DIRECT_POLICY` no longer needs a public face. Nine functions remain, one per
way a caller can need the policy; two is not reachable with six consumer seams.
The package moves to `util/`. The note claimed a dependency on `undici`
disqualified it from that group; the charter governs harness dependencies, not
external ones, and the process note that says so predates this branch. The real
blocker was the harness dependency: resolution needed one method of
`LaunchEnvironmentSnapshot`, so it names a structural `EnvLookup` and the
launcher passes its snapshot unchanged. `net/` is dissolved.
Dropping the group's line from the repository layout also returns `AGENTS.md`
to its original ceiling, so the raise the merge needed is reverted.
The hostFrameParseCeiling JSDoc and one load-gate test comment still quoted
the pre-16x derivation (~29 MiB for a ~300 MiB heap, ~14 MiB for a 128 MiB
old space). With HOST_PARSE_WORST_CASE_MULTIPLE = 16 the same hosts derive
~14 MiB and ~7 MiB (floor((176-64)/16)); protocol.spec.ts pins the 304 MiB
case at 15 MiB. Comment-only correction, no behavior change.
The #3289 merge-forward dropped four typescript/no-unnecessary-condition
suppressions from index.ts (boot-write-failure fake child stdin, the
admit()-closure logsTruncated recheck, and both settled rechecks whose
guards flip mid-wait), turning the lint:contracts-ready gate red. Re-add
them with their reasons. The merge also carried a ptc-python-turn session
fixture that was not in canonical packed layout; migrate-packed-session-fixtures
re-writes it so session-fixture-layout passes.
Review findings on the CPython backend: a child that never reads fd 3 leaves
the reply pipe full forever, so the drain loop waits on 'drain' while every
call frame it keeps sending resolves a binding and queues another reply —
the backlog (and the binding results it pins) would grow until the wall
clock. sendReply now caps the pending backlog at MAX_PENDING_REPLIES and
settles the run as worker-exit past it, mirroring the frame cap; a child
flooding calls against a binding that never settles would otherwise bypass
that cap (pendingReplies grows only after the await), so the dispatcher
counts in-flight binding calls before dispatch and releases the slot in the
async body's finally, capping outstanding closures at the same bound. The
drain also compacts its consumed prefix (replyQueue.splice(0, head)) once
head reaches the bound, so a drain that stays alive without emptying cannot
grow the backing store linearly with cumulative throughput.
The completion-value meter counted lone surrogates with
_SURROGATE.findall(folded), materializing one single-character string per
surrogate: a surrogate-dense value near the budget (millions of surrogates,
each serializing to six bytes) allocated millions of objects before the meter
returned, defeating the meter's counting-without-building contract. The count
is now the length difference between folded and the without string the meter
already computes; a standalone equivalence check confirms it matches findall
across lone-high, lone-low, paired, astral, and mixed cases.
validateBindings read namespace.global/errorClass.name/memberNameProperty
several times and retained the original errorClass object for the boot
frame, whose JSON.stringify re-read it after validation: a stateful getter
could throw or change between the two stages, turning the seam-misuse
rejection into a worker-exit or injecting an unvalidated name. Each field is
now read once into a plain value and the bindings map stores a plain
{ name, memberNameProperty } copy, so validation and the boot frame see
identical values.
Regression tests: a hostile child floods 5000 sequential valid calls without
reading fd 3 and the run settles worker-exit with the reply-queue message
before maxWallMs; a 3,000,000-surrogate completion succeeds at an
18,000,002-byte budget and reports output-limit one byte under; a 5000-call
flood against a never-settling binding settles worker-exit with the
call-backlog message; getter-backed namespace metadata that throws or
changes on a second read boots and runs with each field read exactly once; a
two-wave flood whose replies exceed the writable high-water mark drives the
drain past the compaction bound mid-delivery and verifies all 1524 replies
arrive. README Known Limitations gains the reply-backlog and call-backlog
bounds (en/zh, pairing re-recorded); a new Agent Note registers the findings.
The capability-seams graph derives its implementation lists from
SERVICE_ROLES in scripts/gen-doc-graphs.ts, which still listed only the
worker-thread backend. Add experimental-code-runtime-python so the
generated graph and table match the registered ctx.codeRuntime
implementations; regenerate docs/capability-seams.md, sync the zh pair,
and re-record the i18n pairing.
Review findings on the CPython backend: an explicit pythonBin path bypassed
the load-time checks (missing/non-executable/directory paths surfaced only
as a run-time worker-exit); a throwing binding member accessor escaped the
fd-3 data callback and terminated the host; the reply drain waited on
'drain' alone, so a pipe destroyed under the wait hung forever; and two
staging-leak assertions diffed a global tmpdir that parallel workers can
perturb.
resolvePythonBin now applies the same accessSync(X_OK) + isFile check to
explicit paths (resolved against the host CWD), and the load error message
distinguishes 'is not an executable regular file' from 'does not resolve on
PATH'. validateBindings snapshots callables into a plain record during run()'s
synchronous validation, turning an accessor throw into the seam-misuse
rejection and fixing the key set the boot frame and dispatch share. The reply
drain waits on drain/close/error together and short-circuits on
proto.destroyed. The staging-leak assertions check the exact paths this test
file staged (recorded by the mocked mkdtempSync) instead of a tmpdir diff.
docs(code-runtime-python): add the alternatives section to the hardening note
docs(config-catalog): refresh the code-runtime-python Config source line
test(code-runtime-python): cover the async spawn-error worker-exit path
The review's items: the experimental group README's Packages table and Summary
now list code-runtime-python (CPython subprocess backend, ctx.codeRuntime),
paired; the portable-identifier note's Scope drops the 'has since shipped …
now lists' change narration in favor of the current state, removing the
apparent contradiction with 'the worker is the only shipped backend'.
The review's final wording items: the portable-identifier note's Scope said the
backend 'has since shipped' without noting it is experimental/private; the
RESERVED_WORDS JSDoc said backends 'ship for both languages'. Both now name the
TypeScript backend as released and the CPython backend as experimental and
private. The package README also records that the truncation-marker text and
tempdir prefix keep the pre-rename short names (byte-anchored by tests,
independent of the npm name).
The review's carry-over: 'each has a published backend' in the CodeRuntime
JSDoc and its projections (tool-cordis api-catalog, subsystems page) plus
'both shipped'/'backends ship' in the code-runtime README all claimed the
Python backend is released; it is private and experimental, excluded from the
release family. The wording now states the TypeScript backend is released and
the Python backend is experimental and private (not published), in the JSDoc
(api-catalog regenerated to match), the READMEs (paired), and the subsystems
page (paired).
The fd-3 note's Status line moved off line 3 when the experimental-location
fact was added; it is back as the sole line-3 status. The zh portable-identifier
note's merged Scope paragraph lost its blank-line separator, which the
md-wrap gate read as one hard-wrapped paragraph — the blank line is restored.
The review's move-follow-ups: the Windows test exclude now points at
packages/experimental/code-runtime-python (the constructor throws by design on
Windows, so the suite must stay excluded); the invariant companion and
@module annotations use the new npm name; the truncation marker text and
tmpdir prefix stay as-is (tests anchor them); the package JSDoc and READMEs no
longer call the private experimental backend 'published'/'shipped'; the
code-runtime README row describes the package as protocol AND runtime; the
fd-3 note records the package's experimental location.
The move broke two generated/derived surfaces: (1) the package invariant
companion still registered the old name
@deepseek-ai/dsh-code-runtime-python, so the exhaustive-topology test found
the new name unreserved — it now registers
@deepseek-ai/dsh-experimental-code-runtime-python; (2) the tsconfig.base.json
alias for the renamed package sat inside the generated region, so
gen-tsconfig-paths dropped it (a package named after something other than its
directory needs a hand-written alias before the BEGIN marker) — the alias is
moved out and the config is current again.
The CPython code runtime's complete public contract is experimental, so it
moves to packages/experimental per the experimental-packages rules: npm name
@deepseek-ai/dsh-experimental-code-runtime-python, private: true, no
publishConfig. All references updated (code-runtime READMEs, config-catalog
and module-graph regenerated with zh alignment, tsconfig paths, doc-standard
and workspace-constraints scripts, the fd-3 and settlement Agent Notes, and
the package README links); md-links and translation pairing pass, and the
suite still runs green.
The review's item: the Problem section counts eleven no-fail-before fixes, but
the Consequences section still said 'the ten called out in the Problem section'
(zh: '那十处') and omitted the new unknown-binding preview cap. Both sides now
say eleven and name the cap, paired and re-recorded.
The review's warning: the settlement note's Problem paragraph says ten fixes
have no fail-before test, but the unknown-binding preview cap (a transient
whole-target JSON.stringify peak, unmeasurable through the seam) is the
eleventh. The count and the item are now recorded, paired.
The review's suggestion: the settlement note enumerates each review fix in this
PR, so the unknown-binding preview cap (escaped from a 1 KiB prefix,
capMessage still enforces the reply budget) gets its own short section, paired
and re-recorded.
The reviewer's standing item: the unknown-binding reply ran JSON.stringify on
the WHOLE capped target (global + '.' + name, each up to maxValueBytes code
units), allocating the escaped form — up to ~6x under control-heavy input, a
multi-hundred-MB spike near the maxValueBytes ceiling that no hostile-peer
bound would have admitted. The escaped preview is now built from a 1 KiB
prefix of the target (enough to identify the binding); capMessage still
enforces the reply budget. A forged huge-name case drives the path.
The review's wording item: the layer-5 bullet ended with 'not by this PR'
(zh: 'not borne by this PR'), which references PR context in durable prose.
The sentence now ends with the current-state fact ('not by this package's
suite'), paired and re-recorded.
The review's open suggestion: the Known Limitations now records that the
real-Loader assembly snapshot is deferred to issue #1182 layer 5 (this package
is exercised through ctx.plugin and real-subprocess tests; the full application
composition is covered by a tracked assembly test in that layer), paired.
The review's two remaining non-blocking items: (1) a case where the run ends
with a SEALED open hold (past MAX_PENDING_CHUNKS) — finish() must commit the
sealed prefix, verified to fail if finish drops openSealed. (2) openSealed is
now a block ARRAY (one joined block per seal) matching the fd-3 reader's
blocks and the stray capture's seal, instead of one repeated string concat
that leaned on V8 ConsString amortization.
The review's follow-ups on the open-seal fix: (1) the settlement note's seal
section now records the HOST-side open hold seal (openParts -> openSealed,
mirroring the child _LogStream and stray-capture seals), paired. (2) the
first-fragment guard comment notes the empty-first-frame case (bills cost + 1 =
3, establishes no hold, bounded over-charge in the safe direction). (3) a
regression case commits a SEALED open hold before the truncation marker —
verified to fail if truncateLogs drops openSealed.
The review's warning: each held open fragment is a distinct array slot plus
string object header (~30x overhead the byte cap cannot see), and a
budget-sized single-character open flood is honest-child reachable
(print('x', end='', flush=True) in a loop). With maxLogBytes near its ~67 MB
load ceiling that was up to ~2 GB of host auxiliary heap. The hold now seals
into one block past MAX_PENDING_CHUNKS, mirroring the fd-3 reader's blocks and
the stray capture's seal; the merge, truncateLogs, and the finish residual all
read sealed + current fragments, and a within-budget flood regression asserts
the merged entry is byte-identical.
The review's wording items: the load-check comment still referenced the
resolvePythonBin JSDoc's old ENOENT promise; the spawn-site comment called the
type assertion a non-null assertion; and two comments claimed the
'logs serialize to maxLogBytes + marker + envelope' bound is recorded in the
README's Known Limitations, which has no such entry — the cross-references are
dropped, the bound stays stated inline.
The review's carry-over: detachResidual (a test seam for the settled run's
resource cleanup) is re-exported from the '.' entry but was not in the README's
declared public surface; the list now names it alongside resolvePythonBin and
readProcessStart, paired.
The empty-open continuation skip (5b61a8fe6) made the Known Limitations entry
stale — the held fragment array no longer grows per empty frame — so the entry
is removed on both sides. The public-surface list now declares
resolvePythonBin and readProcessStart, which the '.' entry re-exports for the
test suite.
The skip branch (an empty open continuation is not pushed into the hold) needs
coverage; a case drives an empty continuation between a first fragment and the
closing frame and asserts the merged entry is unchanged.
The review's items: a zero-content open continuation bills 0 but still pushed
'' into the held fragment array, so a forged empty-open flood grew host memory
without touching the ledger — the push is now skipped (an empty fragment
contributes nothing to the merged entry). The spawn-site comment said a PATH
change between load and run would fail with ENOENT; it actually makes spawn
throw synchronously, which the surrounding try settles as worker-exit.
The review's warning: the pythonBin load-rejection is a product-visible change
(unresolvable basename now fails at load instead of a run-time worker-exit),
but the READMEs (en + zh) only said the basename is resolved against PATH, and
the load-check comment still described the old fallback. The README pythonBin
entries and the load-check comment now state the rejection; pairing
re-recorded.
The review's follow-ups: (1) the product-visible change (an unresolvable
basename pythonBin now fails at load instead of a run-time ENOENT worker-exit)
is registered in the settlement note, paired. (2) PYABS falls back to the bare
name when python3 is not resolvable, instead of interpolating the literal
'undefined' into the wrappers.
The review's follow-ups on the pythonBin change: (1) the JSDoc and the two
call-site comments still described the old fallback-to-bare-name contract;
they now state the load-rejection behavior. (2) the ExceptionGroup case's
version guard raised a skip message on Python < 3.11 but the assertion still
required the truncation marker unconditionally — the assertion now matches
either the truncation marker (3.11+) or the skip message (3.10). (3) the shell
wrappers quote the resolved interpreter path.
The review's two non-blocking items: (1) resolvePythonBin returned the bare
basename when PATH had no hit, and spawn (env:{}) would silently fall to
execvp's platform default PATH and could start a system interpreter the caller
never asked for. It now returns undefined for an unresolvable basename and the
load check rejects it (absolute paths pass through), so the failure is loud at
configuration time instead of silent at spawn; the case that expected a
run-time worker-exit now asserts the load rejection, consistent with the
empty/NUL pythonBin cases. (2) the over-cap exception-group case skipped on
Python < 3.11 (ExceptionGroup is a 3.11+ builtin), matching the TaskGroup
case's version guard.
The review's portability warning: the six shell wrappers exec'd a bare
'python3', which /bin/sh resolves against its compiled-in default PATH while
the runtime spawns with env:{} — in environments where python3 is reachable
only through the caller's PATH (Nix, pyenv) every wrapper run would fail as
worker-exit. The wrappers now bake the resolved absolute interpreter path
(module-level resolvePythonBin, which the product spawn already uses), and
resolvePythonBin is exported for the tests.
The reviewer's residual timing item: the inherited-SIGXCPU reset ran AFTER
setrlimit(RLIMIT_CPU) and the boot-namespace construction, so a huge namespace
under an inherited ignore/block could burn past the soft limit inside that
window and be misclassified as worker-exit. The reset now happens at the very
top of _run, before the resource-limit setup and namespace construction.
The review's two follow-ups on the inherited-SIGXCPU fix: (1) a discriminating
case — pythonBin points at a wrapper that ignores SIGXCPU before exec'ing
python3, so the child genuinely inherits the ignore; with cpuSeconds: 1 the
busy loop must end as timeout (the bootstrap reset restored SIG_DFL), and
reverting the reset leaves it running to the wall — verified red. (2) The zh
README's OUTER wire section now carries the truncation-exception sentence
(the previous commit had duplicated it in the inner section instead); the
duplicate is removed, and the settlement note registers the inherited-SIGXCPU
reset.
The reviewer's standing issue: the child inherits the host's SIGXCPU
disposition and signal mask — if the host ignores or blocks SIGXCPU, the soft
RLIMIT_CPU fires but cannot stop the child, and the hard limit's SIGKILL then
classifies a definite CPU overrun as worker-exit instead of a timeout. The
bootstrap now resets SIGXCPU to SIG_DFL and unblocks it before any model code
runs (the settle-time enforcer already restores SIG_DFL for a program that
traps or masks the signal mid-run; this closes the inherited-state gap). The
zh README's outer wire section also gains the truncation-exception sentence to
match the en side.
The review's warning: the READMEs (outer and inner wire sections, en + zh) and
the fd-3 protocol note still claimed the next log frame always merges into an
open entry, while truncateLogs commits the already-billed prefix as its own
entry before the marker. The one exception (truncation) is now stated in both
READMEs and the owning note, paired and re-recorded. The prefix-commit case's
parenthetical describing the pre-fix implementation is removed per the
comment-does-not-record-review-history rule.
The reviewer's standing issue: line.toString('utf8') silently replaces illegal
bytes with U+FFFD, so a forged frame could land a corrupted completion value
(the honest child's lossless encoder never emits non-UTF-8, so such a frame is
hostile traffic). The fd-3 frame decode now uses a fatal UTF-8 decoder: an
illegal byte throws and the frame is dropped, same treatment as the
unsafe-integer check. A forged illegal-UTF-8 done frame is verified to be
dropped (the run settles on the program's real return), and reverting to
toString makes the case fail.
The review's warning: a flushed unterminated line is billed and committed
(README wire contract says so), but every truncation arm — the child truncated
frame, an over-budget open frame, an over-budget closing frame, and admit's two
budget arms — pushed only the marker, dropping the held prefix: the ledger
charged for output that vanished. All arms now funnel through truncateLogs(),
which pushes the (already billed) held prefix before the marker and clears
openParts, so the prefix survives and only the marker stays last; the finish()
guard drops the now-dead !logsTruncated check (a truncated run has an empty
hold). A regression case asserts [prefix, marker]; the forged-flood and
closing-overflow cases now expect the committed prefix plus the marker.
The review's suggestion: an empty open continuation frame bills zero and holds
one host slot, so a forged empty-open flood grows the held fragment array
without touching logBudget. Accepted as a residual (per-frame host cost far
below its ~30-byte fd-3 wire cost, bounded by pipe throughput, model-code trust
level equal to bash) and now registered in the README's Known Limitations on
both sides, paired and re-recorded.
The review's revision: the buffered-chunks pre-check's open-aware overhead is
observationally inert — when the +3 form trips and the open-aware form does not
(pending + newline in [remaining - 2, remaining]), _push_bounded_prefix
re-slices the same newline-free line text and _push_locked admits it under the
same open-aware billing, byte for byte. The comment now states that the
open-aware form keeps _push_bounded_prefix's 'certain to reject' precondition
true, contrasting with the scan pre-check whose slice carries the newline and
therefore genuinely truncates.
The review's warning: the en README's inner 'Wire contract' section still
described only the truncated flag while the zh counterpart (and the outer 'The
wire' section) described open. The inner en section now matches. The exact-fit
closing-line case comment described the buffered-chunks pre-check recipe while
the program actually drives the scan pre-check; the comment now states the
actual arithmetic and path (and the buffered variant was dropped — its writes
coalesce into one call in the test environment, so it did not discriminate).
The review's warning: while an open entry accumulates, the newline pre-checks in
the write path still charged a NEW entry's +3 cheap-bound overhead (quotes +
separator), so an exact-fit merged TAIL was truncated (or the pre-check
over-rejected it and flushed a truncated prefix). Both pre-checks now charge
the overhead only when no open entry is in progress, matching _push_locked's
open-aware bound. A regression case (the review's recipe: flush an open
fragment, then write one exact-fit newline-terminated line) is verified to
truncate when the +3 is restored.
The zh README's wire-contract section now describes the open flag like the en
side (the fd-3 Agent Note holds the split-billing arithmetic; a cross-doc link
was omitted to keep the bilingual link sequence aligned).
The review's warning: the one-byte overflow case ran through the CHILD ledger
(print path), so the host's first-fragment cap (logBudget - 1) never executed,
and the sub-2-byte guard test does not discriminate logBudget from
logBudget - 1 (a reverted cap still trips the guard). The frame is now forged
on fd 3, so a reverted cap of logBudget admits it and flushes it at settlement
— verified to turn the test red.
The review's dedupe suggestion: the split-billing arithmetic was stated in both
notes; the settlement note's Decision paragraph now links to the fd-3 protocol
note's wire-contract section (one home per fact), paired and re-recorded.
The review's suggestion: the settlement note's Decision section now states the
shipped split-billing fact (first fragment pays quotes+separator, continuations
and the closing frame pay content only; host caps logBudget-1 / logBudget+2;
child keys off _open_started), paired and re-recorded.
The review's suggestion: the open-merge mechanism (incremental split billing on
both sides, host caps logBudget-1/logBudget+2, the sub-2-byte walk guard, the
child's _open_started-keyed billing) lived only in code comments. The wire
contract section of the note now states it, paired and re-recorded.
The new jsonStringCostUpTo guard (returns undefined below a 2-byte cap) was
uncovered: forged open frames drive the host ledger down to one byte, and a new
open entry's first-fragment cap (logBudget - 1 = 0) trips the guard and
truncates to the marker, asserted as the merged entry plus the marker.
The review's arithmetic checks: the closing-frame walk used cap
logBudget - openCost, so a compliant merged entry (58-byte wire cost under a
64-byte budget) could see a negative cap and truncate; the first-fragment cap
used logBudget instead of the ledger's logBudget - 1, so an open frame costing
63 was admitted with a bill of 64, pushing the ledger negative and letting a
subsequent empty frame ride in one byte past the configured cap; and the child
billed a closing frame as a fresh entry (quotes+separator again) instead of the
merged tail, truncating an exact-fit 30+30 entry.
Fixes: first-fragment cap logBudget - 1 (matching admit), continuation and
closing-frame cap logBudget + 2 (billed without quotes), jsonStringCostUpTo
returns undefined below 2 bytes, and the child's split billing keys off
_open_started alone (a closing frame pays content only) with the cheaper bound
len(text) while a merge is open. Regression cases cover all three arithmetic
paths.
The review's critical: the open-merge branch re-joined and re-walked the whole
held text per frame, so k tiny open frames cost O(k * budget) (thousands of
1-byte frames against a near-64 MiB budget would re-traverse hundreds of GB and
block the host event loop). The host now holds a fragment ARRAY with an
incrementally billed cost — each fragment's jsonStringCostUpTo walks only its
own text — and the closing frame bills only its own content, so the merged
entry's wire cost is charged exactly once, split across the fragments. The
child bills symmetrically: the first open fragment pays quotes+separator, each
continuation pays only its content, matching the host ledger (the review's
warning: per-fragment full billing truncated a 16-char merged entry under
maxLogBytes: 64 that costs only 19 bytes as one entry).
Regression cases: 16 single-character flushes merge to one whole entry; a
closing frame that overflows the remaining budget truncates to the marker; a
closing frame after an open flood already truncated the ledger is a no-op; and
a forged open-frame flood stays bounded by the ledger. The closing-frame
post-truncation guard is an invariant-false branch (an open frame that would
trip the ledger resets openParts, so a non-empty hold implies no truncation)
and carries a v8 ignore with that reason.
The review's critical: the open-merge branch accumulated the held fragment
before any ledger check, so a forged open flood could grow host memory without
touching logBudget. The held fragment is now bounded by the exact-cost walk
(jsonStringCostUpTo against the remaining budget; the closing frame's admit()
still bills the merged entry once), and the open field is registered in the
README wire-contract section and the fd-3 protocol note (en + zh). A forged
open-flood case asserts truncation to the marker under a 64-byte budget.
The review's remaining warning: an explicit flush of an unterminated line
(print(..., end='', flush=True)) pushed a full log frame, so the following
print() landed in a second entry and logs.join('\n') rendered 'a\nb' for what
the program printed as one line — a model-visible output defect. The flush
frame now carries an flag (LogMessage gains the optional field on both
sides and in the mirror test), the host holds it and appends the next log frame
to the same entry, and finish() admits the residual if the run ends with it
still open. The settlement note registers the decimal-context fix from the
previous commit.
The review's critical: Decimal(repr(value)).normalize() read the process-global
decimal context, so a legitimate program setting getcontext().prec = 2 silently
rounded the completion value's digits and traps[Inexact] = True made the encode
raise, misclassifying a successful run as an exception. A fixed module-level
Context(prec=28) makes the spelling decision context-independent; a regression
case mutates both context knobs and asserts the float round-trips exactly.
The binding-reply README entry now states the fact (no seam-level cap;
maxValueBytes meters only the done frame; a wide reply is rebuilt and encoded
whole, bounded by process memory), matching the earlier reviewer wording.
The review showed the added dispose case was a placebo (dispose in the same
tick as run means SIGTERM hits the group before the program body runs; the
group-emptied arm is already deterministically covered by the same-group
survivor case, which this removes the v8 ignore for). The test is deleted; the
stale silently-discards comment in the boundary test now says rejects; the
README Known Limitations gains the late-log-frame-drop and host-side
binding-value-memory entries. Pairing re-recorded.
The review's premise that the group-emptied arm could not be pinned was
incorrect; the same-group reap case already exercises it. This adds the missing
seam-observable case: dispose() while a setsid orphan holds the pipes and the
run is unresolved — settle kills the child, the group empties (the orphan is in
its own session), and the poll finalizes promptly instead of waiting out the
60 s grace. The v8 ignore on that arm is removed.
The review's remaining coverage gap: the group-emptied arm of pollGroup depends
on the close-driven settle winning the race against the grace SIGKILL, a timing
interleaving no seam-observable test pins deterministically (the same-group
cases assert the settle and the reap, not this exact interleaving) — the arm
now carries a v8 ignore with that reason. The load-check comment and the
FRAME_ENVELOPE_BYTES JSDoc say rejects-as-worker-exit instead of drops.
The forged-second-boot-ack regression makes the re-entry guard covered, so its
v8 ignore is removed. Doc drift: the python README and run() JSDoc state the
resolve-with-value/resolve-with-error contract without inversion; the README
Known Limitations gains the setsid-escaped-orphan entry (the settlement note
referenced it); the settlement note drops the stale drops/discard phrasing and
the two 256 MiB references; the fd-3 protocol zh note no longer claims the
codec is undelivered; the code-runtime seam README (en + zh) says both
backends ship. Pairings re-recorded.
The review rejected the v8-ignore defense for the ack gate: a forged second
boot-ack is deterministically constructible (one os.write on fd 3) and the
run-write failure is deterministically constructible with the boot-write-failure
mock pattern. A program that forges an extra boot-ack asserts the run still
completes once (the gate does not re-send the run frame); a mocked child whose
fd-3 pipe accepts the boot frame but rejects the run write resolves a
worker-exit.
The resolvePythonBin directory branch now has a regression: a PATH whose first
entry is an executable DIRECTORY named python3 is skipped for a later real
interpreter (fail-before: without the isFile guard the directory would be
chosen and spawn would fail). The boot-ack gate's forged-second-ack re-entry
guard and its write-failure branch are covered by v8 ignore comments (the
honest child sends exactly one ack; the write failure needs the child to exit
between ack and write).
The review's two behavior items: the run frame was written back-to-back with
the boot frame (the seam contract puts run after boot-ack, which confirms the
namespaces were accepted); it now goes out from the boot-ack handler, so a
boot failure cannot race the run frame. resolvePythonBin now requires the
candidate to be a regular file — a directory passes X_OK and would otherwise
shadow a later real interpreter. Doc spots: the load-time overflow message
says worker-exit (not stranding to the wall clock), the run JSDoc spells out
the resolve-with-error contract, the PATH-stub test removes the stale v8
ignore, and the README's binding-value bullet names serialization cost.
The local machine's node_modules still links claude-agent-sdk 0.3.220, so a
local gen-third-party-notices run rewrites the file to that version; CI's
fresh install resolves the lockfile's 0.3.241 and gen expects it. The branch
adds no third-party dependencies (the schemastery workspace link is already
covered), so the notices file adopts master's 0.3.241 content.
The earlier merge had adopted master's protocol-only package.json (peer/dev
limited to invariants and cordis, no dependencies), but src/index.ts imports
@deepseek-ai/dsh-code-runtime, dsh-session, dsh-timeout, and schemastery at
runtime — a published lib/index.js could not resolve those bare specifiers.
The manifest now mirrors code-runtime-worker-thread (the five peers, the
schemastery dependency, and the matching dev set); the lockfile, module graph,
and third-party notices are regenerated, and the module-graph zh pair is
re-synced.
The branch's sdk-runtime manifest had carried a code-runtime-python workspace
peer that master's lockfile does not record, so a frozen install failed on the
mismatched specifier. The branch changes no sdk-runtime code, so it adopts
master's manifest verbatim.
The earlier merge had kept the branch's older package.json while taking
master's lockfile, so a frozen install failed on mismatched specifiers for the
code-runtime-python package (master added dsh-code-runtime, dsh-session, and
dsh-timeout peers). The branch changes no dependencies, so it adopts master's
manifest verbatim.
The merge conflict on pnpm-lock.yaml had kept the branch's older dependency
resolutions; the coverage gate's notices check then failed because CI's frozen
install resolved the master lockfile's versions while the committed notices
still named the branch's older ones. The branch adds no dependencies, so it
adopts master's lockfile and notices verbatim.
The audited library registry still listed dsh-code-runtime-python as a plain
protocol library, but the shipped package's src/index.ts has a plugin default
export; the entry is removed from PACKAGE_LIBRARIES and both READMEs declare
kind: package-reference. The zh README heading is 概述 per the standard.
The merge pulled master's README rewrite (front-matter, Summary, TOC, section
anchors, details-folding); its content described the pre-delivery protocol-only
package, contradicting the shipped backend. The README (en + zh) now follows
that structure with the delivered facts: PythonCodeRuntime, the fd-3 wire, the
load-validated caps, the 64 MiB frame parse cap (worker-exit settlement), and
the known limitations. Pairing re-recorded.
The review's three stale-comment items in index.ts: the orphan JSDoc above
FRAME_PARSE_CAP_BYTES (left over from the deleted receive ceiling), the
pre-join comment's change narration and its reference to a no-longer-existing
higher ceiling, and the first-frame comment's mention of a per-line cap check
that no longer exists. Test comments for the pythonBin and sealing-threshold
cases are weakened to their observable claims (both orders reject an over-cap
frame; the pythonBin case pins the contract, not a worker-exit distinction).
The fd-3 protocol note (en + zh) drops the 'future provider/runtime' staging
language (the runtime is delivered and its real-subprocess suite owns the
field-type gap), and the settlement note's Testing paragraph records the
frame-cap, multi-frame, sealing-threshold, and pythonBin resolution cases now
in the suite. Pairings re-recorded.
The review's doc drift items: the orphan receive-ceiling JSDoc, the frame-ceiling
references in index.ts/bootstrap.py/tests, and the README's 'dropped, stranding
to the wall clock' phrasing (the run now settles as a worker-exit) are all
updated to the 64 MiB FRAME_PARSE_CAP_BYTES semantics; the README notes the
>64 MiB binding-argument residual as a worker-exit trip of the same cap. A
regression case resolves a basename pythonBin against a PATH whose first entry
is relative ('.') and asserts the absolute entry is used.
The review's remaining code items:
- resolvePythonBin now skips RELATIVE PATH segments (a bare 'bin' or '.'): the
returned candidate must be absolute, because spawn() resolves a relative
pythonBin against the host CWD, outside the seam contract.
- A deterministic-ish regression pins the sealing-threshold corner: 64 MiB of
4 KiB (<= PIPE_BUF, atomic) newline-free writes plus 12289 more A's before
the first newline make the first frame exceed FRAME_PARSE_CAP_BYTES; the
newline-bearing chunk reaches the first-frame check (sealing is the ELSE
half of the newline branch), so the run reports worker-exit with the
protocol-frame-exceeded message.
The review's sealing corner: the fragment-count seal ran before the newline
branch and did not exclude a newline-bearing chunk, so the 1024th chunk (the
first to carry a newline) was concatenated into a sealed block, pendingChunks
was emptied, sawNewline stayed false, and the first-frame check was skipped for
a join that then contained the newline. Sealing now runs as the ELSE half of
the newline branch, so a newline-bearing chunk always reaches the join and its
first-frame check, and the invariant 'sealed blocks hold newline-free prefixes
only' is true — which is what makes the removed per-line check genuinely dead.
The pre-join counter (single unframed line) and the first-frame check
(newline-bearing chunk) reject any frame past FRAME_PARSE_CAP_BYTES before the
join, so every line reaching this loop is within the cap by construction — the
per-line check was dead code and its continue branch could never fire, failing
the per-file 100% coverage gate.
The pre-join check charged the whole unframed buffer, which legitimately holds
several frames each within FRAME_PARSE_CAP_BYTES: a first frame of exactly the
cap followed by a second frame crossed the counter and was misreported as a
worker-exit. The pre-join rejection now fires only while the held bytes are a
single unframed line (this chunk carries no newline); once a newline arrives,
a FIRST-FRAME check measures the bytes up to the first newline across the held
chunks (including sealed blocks) and rejects only that frame before the join —
keeping the peak at one copy of its wire bytes — while later frames in the
same buffer are handled by the restored per-line check. Regression cases: a
72 MiB newline-free buffer is rejected pre-join (fail-before: joining would
have doubled it); two within-cap frames whose combined buffer crosses the cap
both survive (fail-before: the unconditional counter check turns it red).
The unframed-buffer counter guard runs before every join and guarantees each
line is within FRAME_PARSE_CAP_BYTES, so the line-loop cap check was dead code
(its continue branch could never fire, failing the per-file 100% coverage gate
on index.ts). Removed with a comment explaining the invariant.
The pendingBytes guard now trips at FRAME_PARSE_CAP_BYTES (64 MiB) instead of
the 256 MiB wire ceiling, so the three tests that flood/pin frames against the
guard assert the 67108864 message and write a 64 MiB-based workload.
The review's remaining critical: the fd-3 data handler checked the unframed
counter against the 256 MiB wire ceiling, so a single 64-256 MiB frame was
fully Buffer.concat-joined (a second copy) and only then dropped in the line
loop — the peak-memory doubling the pre-join check exists to prevent, for a
frame the parser is guaranteed to discard. The counter is now checked against
FRAME_PARSE_CAP_BYTES before the join; the regression case asserts a worker-exit
with 'protocol frame exceeded' (fail-before: reverting to the ceiling turns it
green, proving the join path). FRAME_CEILING_BYTES is removed.
The rejection-cap fix now has its regression: a completion value whose class
name is 70 MiB of Ns asserts invalid-output, not worker-exit (fail-before:
uncapping the diagnostic turns it red).
The settlement note (en + zh) updates the remaining stale bound text, and the
fd-3 protocol note (en + zh) no longer claims protocol-only exports or a
missing Python codec. Pairings re-recorded.
The review's remaining items:
- _done_with_value's rejection branch now caps the _check_done_value diagnostic
through _cap_message (a reason embedding a hostile class name could otherwise
push the done frame past the host's 64 MiB parse cap, misreporting an
invalid-output run as a worker-exit).
- The settlement note (en + zh) updates three stale facts (load bound is now
parse-cap minus envelope at 67108800; the sink goes directly through the
bound primitives); the fd-3 protocol note (en + zh) no longer claims the
package ships protocol without the runtime; FRAME_ENVELOPE_BYTES' JSDoc and
_cap_message's docstring follow the new bound.
Pairings re-recorded.
The review's remaining warning: the _run binding comment claimed the log sink
went 'through the bound send', contradicting the sink's actual direct use of the
bound encode+write primitives. The comment now states that; the settlement note
(en + zh) registers FRAME_PARSE_CAP_BYTES and the 65 MiB-frame regression case.
Pairing re-recorded.
The review found the 64 MiB parse cap contradicted the load-time budget bound:
maxLogBytes/maxValueBytes could be configured up to ceiling - envelope (~256 MiB),
but the receive path silently dropped any frame past the 64 MiB parser cap, so an
honest child's budget-internal done frame under such a config would be discarded
and the run stranded to the wall clock. The load bound is now parse-cap -
envelope, so a configured budget always fits through the parser; the boundary
test moves to 64 MiB - 64. The >64 MiB model-constructed binding-argument drop
is registered as an accepted residual in the README (en + zh).
Addresses the review's remaining two items:
- FRAME_PARSE_CAP_BYTES (64 MiB) drops an fd-3 frame whose raw length exceeds
it BEFORE toString/JSON.parse: the 256 MiB wire ceiling bounds the bytes, not
the decoded structure, and a compact wide frame near that ceiling could decode
to far more host memory. A regression test writes a 65 MiB log frame plus a
normal one and asserts the oversized frame is dropped while the trailing frame
still lands in logs (fail-before: without the cap the oversized text is parsed
and admitted, truncating the ledger so the trailing frame is dropped). The
forged-oversized lower-bound test's frame is reduced to stay under the cap
while still exercising the truncation path.
- The log sink writes through the def-time bound encode+write primitives (not
send_sync, whose body resolves _encode_json_plain and self.write_encoded at
call time), so a rebind cannot break a log frame.
The review's remaining functional item: send_sync's body resolves
_encode_json_plain (module global) and self.write_encoded (class attribute) at
call time, so a program rebinding either before the first binding call could
turn a legitimate call into an exception. dispatch now writes the call frame
through def-time bound write_encoded+_encode_json_plain, and the log sink goes
through the bound send; the dispatch rebind test also rebinds those two names
(verified fail-before by reverting to send_sync). The annotation test title
matches its assertion direction, and the note (en + zh) registers the
error-class constructor, dispatch primitives, and dont_inherit mechanisms.
Pairing re-recorded.
The review required regression cases for the two cfb35bef6 fixes:
- Rebinding __main__.Exception/__main__.setattr must not break the minted
error class: a host rejection still surfaces as ToolCallError with the member
property readable.
- Rebinding __main__._lossless_json_violation/__main__.asyncio/
__main__.ProtocolChannel.send_sync must not break dispatch: a legitimate
binding call still round-trips.
bootstrap.py imports from __future__ import annotations; compile(wrapped) was
inheriting that PEP 563 flag, stringifying the program's type annotations and
changing the semantics of a legal program that reads f.__annotations__ at
runtime. compile(..., dont_inherit=True) stops the leak; a regression test
defines an annotated function and asserts the annotation is the live int class,
verified fail-before by removing dont_inherit (the test turns red).
The review's remaining items:
- _make_error_class captures Exception and setattr as def-time defaults, so a
rebind of __main__.Exception/__main__.setattr cannot break the rejection
constructor.
- dispatch binds _lossless_json_violation, asyncio.get_event_loop, and the
channel's send method into _run locals before the program runs, so a rebind
cannot turn a legitimate binding call into an exception or a wall-clock
timeout.
- The note (en + zh) corrects the stdin coverage phrasing: d3f9f57f5's direct
EOF-observing case is the in-tree pin, not an approximation.
- Collapse two stray double blank lines in the test file.
Pairing re-recorded.
The review's three remaining items:
- A regression test rebinds __main__.ProtocolChannel.read_frame_async and asserts
a binding reply still round-trips (the pump's reader is a bound method
captured by _run before the program runs).
- The settlement note (en + zh) records that send_done's frame-shape check uses
_run's bound _str/_isinstance.
- The staging-removal comment no longer claims teardown retries tracked state:
teardown deliberately does not sweep staging, so a removal failure is the one
case the gone-by-settlement contract degrades on.
Pairing re-recorded.
The review flagged that the implemented note's capture-family enumeration had
not followed c0ca236b5: read_frame/read_frame_async now also capture len (and
asyncio.get_event_loop on the async reader), _decode_json_plain captures
isinstance/str/list, and the reply pump's frame reader is injected as a bound
method captured by _run before the program runs. Note (en + zh) updated;
pairing re-recorded.
The stdin destroy (child.stdin?.destroy() right after spawn) previously had no
in-tree coverage. A program that reads fd 0 now sees EOF immediately; without
the destroy it blocks and the run would hang to maxWallMs as a timeout —
verified fail-before by disabling the destroy (the test turns red at the wall
ceiling) and restoring it (green). The _str rebind regression was attempted but
is not viable: the success path's done-frame serialization reaches str
transitively through _encode_json_plain, which the README Known Limitations
already records as the accepted success-to-exception residual, so any rebind
test trips that documented residual before send_done's bound _str.
The review's completeness check found the def-time capture pattern was not yet
applied to every name the reply/settlement paths resolve at call time:
- _decode_json_plain now also captures isinstance/str/list.
- read_frame/read_frame_async capture len; read_frame_async captures
asyncio.get_event_loop.
- send_done uses _run's bound _str/_isinstance for its frame-shape check.
- The reply pump's frame reader is a bound method captured by _run BEFORE the
program runs and passed into _pump_replies, so a rebind of the class
attribute cannot redirect it.
The decode-rebind regression test still pins the _decode_json_plain rebind;
rebinding builtins (len/isinstance/list/str) in a test is not viable because
the Python runtime itself resolves them implicitly.
The boot-write-failure fake child carries no stdin at runtime, so the optional
call is the documented guard; the static type (ChildProcessWithoutNullStreams)
says stdin is non-null, which trips the no-unnecessary-condition lint.
The boot-write-failure path's fake child carries no stdin handle, so the
unconditional destroy threw inside the spawn error handler and mislabeled the
worker-exit. Use the optional-call form; the no-stdin branch is exercised by
that same test.
Addresses the review's two remaining items:
- The host closes the child's stdin write handle immediately after spawn. The
program is an async body that reads nothing from fd 0; a live pipe would hold
a host-side handle open past the run, so a setsid-escaped descendant
inheriting fd 0 could keep the host process from exiting even after the
closeDeadline forced settlement. The child (and any descendant) reads EOF on
fd 0 and no host handle survives.
- read_frame/read_frame_async bind their decode primitives (_decode_json_plain,
os.read, _READ_CHUNK_BYTES, bytes) as def-time default arguments, and
_decode_json_plain itself captures json.loads, its two regexes, and len the
same way, so a __main__ rebind cannot kill the reply pump and strand every
pending Future to the wall clock. _decode_json_plain and its regexes moved
before the ProtocolChannel class so the defaults resolve at class-definition
time. A regression test rebinds _decode_json_plain and asserts a binding reply
still round-trips.
Note (en + zh) registers both mechanisms; pairings re-recorded.
The review's remaining non-blocking suggestion: dispatch's call_failure(str(exc))
resolved the builtin str at call time, so a program rebinding __main__.str could
run a hostile callable when the binding-rejection message is formatted. Bind
_str into _run locals and use it in dispatch.
dispatch's call_failure and its except clause resolved the module globals at
call time, so a program rebinding __main__._BindingRejection = ValueError let
the internal marker type leak into model code. Bind _RuntimeError_cls and
_BindingRejection_cls into _run locals before the program runs (names distinct
from the module globals so the assignment RHS resolves the global, not an
unbound local); dispatch now uses the locals. A regression test rebinds
_BindingRejection and asserts a host rejection still surfaces as RuntimeError.
The sys.__stdout__ flush test now reconfigures the streams back to block
buffering (write_through=False) so the settlement drain path is what the case
pins — verified fail-before: binding the stream objects instead of their flush
methods turns the test red.
The settlement drain iterated the bound stream OBJECTS, which are not
callable — every _flush() raised TypeError and was swallowed by the loop's
except, so the drain never ran and only the -u flag carried the behavior.
Bind sys.__stdout__.flush/sys.__stderr__.flush (bound methods, capturing the
stream at binding time, immune to a later sys.__stdout__ rebind; None-guarded).
Verified by removing -u temporarily: the sys.__stdout__ regression test still
passes, so the drain is a genuine backstop, not a documented-but-dead layer.
Addresses the review's two carried warnings and the comment suggestion:
- Once the ledger truncates, every arm that marks it (admit()'s two ceilings and
the child-marker frame arm) now clears both stray pipes' buffered output
wholesale, so the end-path flushStray sees empty buffers instead of
concat+decoding doomed data near a 256 MiB maxLogBytes; captureStray's newline
loop re-checks the flag before re-retaining the residual.
- The child runs with -u (unbuffered), so sys.__stdout__/sys.__stderr__ writes
are visible to stray capture immediately; the settlement flush still drains
the original std streams before the done frame as a guard. A regression test
writes through sys.__stdout__/sys.__stderr__ without an explicit flush and
asserts both bytes land in logs. C-ext stdio remains an accepted residual,
recorded in the README Known Limitations (en + zh).
- The ledger-comment arithmetic now states the exact boundary (serializes to
exactly maxLogBytes; without the reserved byte it would be maxLogBytes + 1)
in both host and child.
Note (en + zh) registers the stray-clear and -u/settlement-drain mechanisms and
the new test; pairings re-recorded; corpus passes 1029.
The review flagged the change-narrative wording 'degrades to the pre-existing
behavior' (prohibited by docs/AGENTS.md) in four spots — README en/zh, the
readProcessStart JSDoc, and the test comment — and the incomplete :77 residual
sentence ('can still' with no verb complement). Reword the four to a direct
statement of current behavior (killGroup signals the pgid without the identity
re-check on macOS), complete the residual sentence with the actual consequence,
and re-record both pairings. Corpus-wide verify-translation-pairing passes 1029.
The master merge brought a stale generated config-catalog that omitted the
dsh-code-runtime-python config section and mislisted the package. Regenerate
docs/config-catalog.md (verify-config-catalog passes), translate the python
config section into zh, keep the ts config-catalog code blocks verbatim
(untranslated, per the pairing rule), and drop the stray zh Library-packages
line. Corpus-wide verify-translation-pairing passes 1029.
The review found the 62 floor off by two (the marker's fixed prefix is 51
characters counting both square brackets, so marker(62) serializes to 63) and
the constructor error over-claiming a bound the marker-as-envelope design does
not deliver. Fixes:
- MIN_LOG_BYTES is 64 (marker-only serialization fits with one byte of room);
the JSDoc arithmetic counts the brackets; the rejection test pins 63; the
forged-frame test uses 11 NULs (69 escaped) at 64.
- The constructor error now states the marker-only guarantee, and the README
Known Limitations (en + zh) records the real bound: a truncated run with
admitted entries serializes its logs to maxLogBytes + marker + envelope.
- The SIGXCPU-mask tests burn with time.process_time() instead of wall-clock
perf_counter, so a contended CI runner cannot under-burn the budget.
- The settlement note (en + zh) records the 64 floor and the marker envelope
bound, including the zh pre-encode section that the earlier pass missed.
- The README constructor-rejection list names the maxLogBytes floor.
Pairings re-recorded; corpus-wide verify-translation-pairing passes 1004.
The review flagged four mechanism changes shipped without note registration:
- Log ledgers start one byte below the budget (outer-array envelope reservation)
and the constructor floors maxLogBytes at 62 (the smallest budget that can
serialize its own truncation marker plus the envelope).
- die_if_cpu_exhausted restores SIG_DFL before unblocking a program-masked
SIGXCPU, so a trap+mask program cannot run a re-masking handler at the unblock.
- ast.parse passes filename="<model>" so parse-time syntax diagnostics share the
compile/runtime source label.
Decision and Testing (en + zh) now record all four with their fail-before cases
(exact-limit, budget rejection, syntax label, SIGXCPU-mask, trap+mask); pairing
re-recorded and consistent.
Addresses the review's two code warnings and one suggestion:
- die_if_cpu_exhausted now restores SIG_DFL BEFORE unblocking SIGXCPU: a program
that installed a custom handler AND masked the signal would otherwise have
that pending handler run at the unblock (in model code, re-masking or raising)
and escape the re-raise; with SIG_DFL first the pending signal kills inside
the kernel with no bytecode window. A trap+mask combined regression test pins
it (the mask-only case was already covered).
- The constructor rejects budgets too small to honor: maxLogBytes must fit the
truncation marker plus the serialized outer-array envelope (floor 64), and
maxValueBytes must at least represent the smallest JSON completion (floor 4,
matching the worker backend). The exact-limit test moves to the 64 floor and
a rejection test pins the floors.
- The pthread_sigmask None-guard comment cites the real rationale (defensive
against stripped CPython builds; win32 is refused at construction), not the
unreachable Windows path.
The residual bullets listed the encoder's transitive deps as an exhaustive set
but disagreed with each other and omitted io. Mark the list as a non-exhaustive
example (e.g. _dump_scalar/_dump_string/json/io) in the README (en + zh) and the
settlement note (en + zh); pairings re-recorded and consistent.
Addresses the review's two remaining code warnings and the three suggestions:
- Log ledgers (host and child) start one byte below the budget, reserving the
serialized outer-array envelope (two brackets and n-1 commas over n entries'
separators); the exact-zero test moves to maxLogBytes 104 and a new exact-limit
case pins that maxLogBytes 5 admits ['a'] (5 bytes) while 4 truncates to the
marker alone.
- die_if_cpu_exhausted unblocks SIGXCPU (pthread_sigmask SIG_UNBLOCK, captured at
import, None-guarded for Windows) before re-delivering it, so a program that
masks SIGXCPU, burns past the soft limit, and returns is still classified as a
timeout; a regression test pins the masked path.
- ast.parse passes filename="<model>" so parse-time syntax diagnostics carry the
same source label as compile and runtime tracebacks; the syntax-error test
asserts the label.
- The NUL-escape test comments use the true six-byte JSON escape \u0000 instead
of the caret notation; the README Known Limitations (en + zh) records that
PID-reuse protection is inert on macOS; a combined-rebind regression test pins
BaseException plus the traceback reporter rebinding together.
Addresses the review's registration-text accuracy findings:
- _run binds _done_with_value into a local (done_with_value_bound) before the
program runs, closing the __main__._done_with_value = boom success-rewrite
vector; a regression test rebinds it and returns a legitimate value, asserting
the success survives.
- README (en + zh): the CPU-recheck bullet now states the recheck runs
unconditionally after the program returns (a pre-return overrun dies there as
a timeout) and the false-success window is only a trap-SIGXCPU program that
passes the recheck and overruns during the settlement flush/encode; the
encoder-deps residual rationale is replaced with the actual one (bash-equivalent
trust, verdict still delivered via the send_done fallback frame) and names the
now-bound entry; the t.join() deadlock bullet fixes the subject/object (the
main coroutine joins the worker, blocking the pump's main event loop).
- The portable-identifier-seam architecture note no longer claims the Python
backend does not exist.
- Settlement note (en + zh) registers the entry-name binding and the new test.
- All pairings re-recorded; corpus-wide verify-translation-pairing passes 1004.
Addresses the review's two registration-text accuracy findings:
- The cross-thread t.join() deadlock is a process-isolation-backend property (the
pump runs on the child's main event loop), so it is split out of the wide-binding
REPLY bullet into its own Known Limitations entry with the correct attribution
(fix belongs in this backend, not packages/core/session); the zh half-width
space is removed.
- The settlement note's _done_with_value def-time default-arg sentence is
qualified: it guards a rebind of _check_done_value/_encode_json_plain, while a
transitive encoder dep (_dump_scalar/io) rebind can still downgrade, which is
registered as an accepted residual in the package README.
Pairing re-recorded; corpus-wide verify-translation-pairing passes 1004.
Document the two remaining keep-current residuals in the python package README
Known Limitations (en + zh), per the review's accepted-resolution path:
- A trap-SIGXCPU program can exceed the soft CPU limit during settlement encoding
and still report success (containment holds via hard +1s and wall clock; only
the classification is degraded, because the recheck cannot meter mid-encode).
- The encoder's direct deps (_dump_scalar/_dump_string/json) resolve at call
time, so a __main__ rebind after a legit return can downgrade success to
exception; the value path's top-level deps are def-time bound, the transitive
ones are an accepted residual.
Pairing re-recorded and consistent.
Addresses the bot's keep-current findings:
- The settlement note distinguishes the BaseException (lost done frame) and
RuntimeError (pump killed -> replies stranded to the wall clock) consequences;
registers the _done_with_value def-time default-arg capture and the new
RuntimeError-rebind closed-loop test; zh:95 half-width space fixed.
- The python package README Known Limitations records the cross-thread binding +
sync t.join() deadlock (en + zh).
- The code-runtime Service Definition README no longer claims only the
worker-thread backend ships: the Python (process) backend is acknowledged,
with 'container' as future work (en + zh).
- All pairings re-recorded; corpus-wide verify-translation-pairing passes 1002.
A body-local X = X binding in _pump_replies is too late: _run reaches the
model's top-level statements (which run first, since there is no suspension
point between create_task and await __dsh_main__) before the pump's first step,
so a __main__.RuntimeError rebind there would be captured by the body local and
a closed-loop failure would escape the except, killing the pump. Bind
_RuntimeError, _BindingRejection, str, and bool as DEF-TIME default arguments of
_pump_replies (evaluated at import, before any model code runs). Add a regression
test that rebinds __main__.RuntimeError as the first program statement and drives
the closed-loop worker pattern, asserting the pump survives and delivers the
later binding. Update the settlement note (en + zh) to describe the default-arg
capture; pairing re-recorded and consistent.
- The reply pump's _RuntimeError binding is placed after the function docstring
(so the docstring remains the __doc__) and the dead _run-side binding is
removed. _done_with_value binds _check_done_value/_encode_json_plain as
default arguments so a __main__ rebind after model execution cannot rewrite a
success into an exception.
The _str/_bool/_BindingRejection pump bindings were attempted but break the
closed-loop pump test (the self-referential _BindingRejection local interferes
with the closure), so they are left unbound; rebinding those names (builtins and
one internal class) is outside the practical threat model.
The previous commit bound _RuntimeError in _run, but _pump_replies is a separate
module-level function, so its except _RuntimeError referenced an out-of-scope
local and raised NameError instead of catching the closed-loop failure — killing
the pump and timing out the run. Bind _RuntimeError at the top of _pump_replies
too. The closed-loop pump test now passes.
The reply pump's except RuntimeError resolved the module global at runtime, so a
__main__.RuntimeError rebind could make a closed-loop scheduling failure escape
the catch, killing the pump and stranding every later reply. Bind RuntimeError
into a _run local alongside BaseException and catch the local. The settlement
note Decision now records that the exception classes the settlement-path except
clauses catch are bound into locals / a closure cell before model code runs
(en + zh); pairing re-recorded and consistent.
The rebindable-BaseException vector the bot flagged existed in every except
clause of the settlement path, not just the _run outer catch: safe_model_traceback
(three guards) and the post-done flush swallow resolved the module-global
BaseException at runtime, so a __main__.BaseException rebind plus a throwing
__str__ could let a render-time exception escape and lose the done frame. Bind
BaseException into a _run local (at the top) and a closure cell in
_make_failure_reporter, and change every such except clause to catch the local
— immune to a one-line rebind.
The zh Consequences section counted ten but enumerated only nine; add the
log-fragment seal as the 10th no-fail-before item. Also unify the term to
'封存' (matching the Decision/Testing sections) instead of '封口'. Pairing
re-recorded and consistent.
The _run outer try/except used the module-global BaseException, which the
program (running as __main__) can rebind: __main__.BaseException = RuntimeError
made the except resolve to RuntimeError, so a subsequent ValueError escaped _run
with no done frame and misreported the run as worker-exit. Bind BaseException
into a _run local before the program runs so the catch is immune; a regression
test rebinds BaseException and raises, asserting an exception, not a worker-exit.
Also correct the NUL-escape comment text: the JSON escape-result side is \^@ (6
bytes, the valid JSON NUL escape), not \x00, so the 6x-budget arithmetic in the
comments is self-consistent. Register the BaseException-rebind case in the
settlement note Testing (en + zh) and re-record the pairing.
Addresses the bot's keep-current review findings:
- The module-level fallback comment now states the mechanism truthfully: the
module globals are RAW primitives bound into _run LOCALS before the program
runs (the immunity lives in the frame-local binding, not the module global);
and the fallback literal <unrenderable> is distinguished from the failure
reporter's _UNRENDERABLE_DIAGNOSTIC text.
- The settlement note's fallback mechanism wording, the transitive-name rebind
case (now listing the three fallback primitives), and the no-fail-before count
are aligned en/zh; the zh Problem paste damage is fixed and the Consequences
count is ten with the 10th item.
- Pairing re-recorded and consistent.
Addresses the keep-current review findings:
- The settlement note's Problem/Consequences count is nine -> ten, adding the
log-fragment seal to the no-fail-before enumeration (its 25 M-scale OOM is not
deterministically constructible in CI); the new Decision section title now
names all four mechanisms and the double blank line is removed.
- README Known Limitations (en + zh) documents the 1-second dual-limit
ulimit -t 1 CPU overrun being reported as worker-exit (the hard >= 2 guard
cannot lower a 1-second soft to 0); pairings re-recorded and consistent.
The done-frame fallback read _os_write/_memoryview/_FALLBACK_DONE_FRAME as module
globals at call time, so a single-line rebind of any of them reopened the
rebind hole the fallback exists to close. Bind them into _run locals before the
program runs, and use a bare except (which catches everything without naming
BaseException, so a rebind of that name cannot defeat the handler). The
transitive-name rebind test now also rebinds _os_write/_memoryview/
_FALLBACK_DONE_FRAME to pin the fallback's immunity.
Keep the settlement note current with the latest code-review fixes:
- New Decision section for the _LogStream fragment seal, the _clamped
RLIMIT_CPU soft-lowering (and its hard==1 blind spot), the send_done
fallback frame, and the reply-queue slot release.
- Testing registers the fragment-cap drip (no-fail-before), the dual-limit CPU
overrun, and the transitive-name rebind cases.
- zh mirrored; settlement-fixes.i18n.yaml re-recorded and consistent.
Addresses the bot's follow-up review findings on the settlement-path fixes:
- The _LogStream seal joined the WHOLE accumulated buffer past the fragment cap,
re-copying the growing block O(B^2/cap) times for a large drip. It now seals
only the current fragments into a _pending_blocks entry (character count
unchanged), so a 25 M single-character drip stays O(B); the newline/flush/
_push_bounded_prefix consumers join blocks + fragments once.
- The _clamped soft==hard lowering is scoped to RLIMIT_CPU: for RLIMIT_AS a
one-byte soft differential would only misalign the child's applied limit with
the host-side budget gate, with no signal to preserve. The hard == 1 blind
spot is documented.
- send_done's fallback captures memoryview at import (_memoryview) alongside
os.write, so a one-line rebind of the name cannot change the fallback write;
the comment now states the module-level-captured mechanism.
The dual-limit CPU test used ulimit -t 1 (hard == 1), which the _clamped
soft-lowering guard (hard >= 2) intentionally does not lower, and trapped
SIGXCPU (which defeats the fix). Use ulimit -t 2 (hard == 2, so the soft is
lowered to 1) and leave SIGXCPU unhandled; the run then classifies as a timeout.
The message is the CPU-time-exhausted diagnostic, not the literal 'SIGXCPU'.
Addresses the bot's v16 review on the settlement-path code:
- critical: _LogStream._pending now seals the fragment list past a chunk cap
(like the host captureStray seal), so a newline-free single-character drip no
longer accumulates one list slot per write and OOMs on its own accounting.
- _clamped lowers a soft==hard result by one unit (when hard >= 2) so a
dual-limit ulimit -t leaves SIGXCPU a window to fire and a definite CPU
overrun is reported as a timeout, not a worker-exit.
- send_done wraps its encode+write in a try and, on any throw from a rebound
transitive name (_dump_scalar/os), writes a fixed pre-encoded done frame via
the import-time captured os.write, so a settled exception verdict is never
downgraded to worker-exit.
- drainReplies clears the consumed replyQueue slot so a wide written payload is
released immediately, bounding host memory to the current backlog under
sustained fd-3 backpressure.
Tests added for each (fragment cap drip, dual-limit CPU overrun, transitive-name
rebind done frame).
The Testing sentence's subject attached the three rebinds to 'the fix' rather than
to the fixture that performs them; reword to 'pinned by a case that rebinds' and
mirror zh ('由一个…用例钉住'), re-recording the pairing.
The rebinds-every-name fixture previously only rebound ProtocolChannel.send_sync,
which a bound method object ignores and the shipped send_done no longer calls —
so it did not actually guard the call-time-lookup shape. Rebind write_encoded
and _encode_json_plain too (the names send_done would resolve late if it looked
them up at call time) and state that in the settlement note's Testing section
(en + zh), re-recording the pairing.
Keep the agent note current with the recently landed code-review fixes:
- six -> nine no-fail-before cases, adding the done-value TOCTOU pre-encoding,
the stray-UTF-8 budget-flush retention, and the late-rejection settled guard,
each with its reason for not carrying a fail-before test.
- New Decision sections for the pre-encode + send_done binding and the stray
flush retention; Testing lists the binding-all-names case as a tested fix.
- zh mirrored; settlement-fixes.i18n.yaml re-recorded and consistent.
Each shift() re-slices the remaining array, so draining a large gather of
wide bindings awaiting fd 3's drain was O(n^2). Reading by a head index into
the array keeps the drain linear; the finally still discards everything.
Addresses the follow-up review findings on the settlement-path fixes:
- send_done now routes both the pre-encoded VALUE frame and the dict ERROR
frame through a bound _encode_json_plain + bound write_encoded, never through
channel.send_sync (whose body re-resolves self.write_encoded and the module
_encode_json_plain at call time) — a program rebinding ProtocolChannel.
write_encoded or __main__._encode_json_plain no longer skips the done frame.
- flushStray retention re-accrues the withheld multibyte tail from a FRESH
utf8 state (previously metering the carried lead against the post-flush
expected>0 state charged it as an illegal continuation), and skips admitting
when the whole residual drained into the retained tail so no bogus empty
entry is pushed.
Corrections to the settlement-path review fixes:
- send_done was invoking channel.send_sync / channel.write_encoded via a late
method look-up, which a program running as __main__ could rebind through
__main__.ProtocolChannel.send_sync before the failure path ran — a rebound
send that raises then skipped the done frame and downgraded a settled
exception to worker-exit. Bind both channel methods into locals before the
program runs, mirroring the pre-existing binding of flush_out/flush_err/
safe_model_traceback.
- Restructure flushStray so the mid-sequence budget-flush retention arm is a
self-contained v8-ignored branch and the covered default path decodes the
full residual (not schedulable-through-the-seam boundary).
Pace-free completion framing, stray UTF-8 flush, and late-rejection guards:
- Pre-encode the completion value at its validation point so send_done never
re-walks a live value a mutating daemon thread could change (TOCTOU); a
mutation-induced encode throw is then classified as 'exception', not a
host-side worker-exit.
- Budget-triggered stray flush retains an incomplete multibyte UTF-8 tail
(<=3 bytes) as residual instead of decoding a legal, split character to
U+FFFD in an admitted entry; the end/closeDeadline paths still full-decode.
- Check 'settled' before formatting a late binding rejection's message, so a
hostile message getter cannot stall or exhaust a run that already settled.
- Document _check_done_value's first-to-trip ruling in its docstring.
- Rewrite ProtocolChannel.send_sync around a shared write_encoded that the
done frame's pre-encoded string path uses.
The drain loop's `if (settled) break` needs the run to settle in the window
between two queued frames. A file probe on the concurrent-replies case shows the
queue does reach depth 11, but the wall clock never lands inside that window, so
the branch is not schedulable from a test; a case written to force it passed
without ever executing the line, so it is removed rather than left as coverage it
does not provide. The branch carries a v8 ignore naming what is unreachable.
`sendReply` ignored `proto.write`'s `false` return, so a program resolving
several large values in one `asyncio.gather` round encoded every reply in the
same turn and queued all of them in fd 3's writable buffer. Binding resolution
carries no seam-level byte cap to bound that, and the failure kills the host
process rather than failing the run: measured on a 64 KiB-highWaterMark pipe,
eight 4 MiB replies buffered 32.0 MiB at once against 0.0 MiB once paced.
Replies now go through a queue that encodes and writes one frame at a time,
awaiting `drain` when the pipe is full. The encode happens inside the loop, so a
queued reply the run no longer needs is dropped by the `settled` check without
ever being serialized.
This was previously deferred on the grounds that serializing would narrow the
seam's concurrency contract. That reasoning was wrong: the child matches each
reply to its `call` by id from a pump that reads fd 3 continuously, so arrival
order was never observable, and the bindings still run concurrently. Only the
host's peak memory and the flush timing change. The README entry recording the
deferral is removed and the Agent Note records the mechanism instead.
`sendReply` already refuses to write after the run settled, but only after
`snapshotJsonValue` walked and copied the resolution. Binding resolution carries
no seam-level byte cap, so a binding resolving a wide value after `maxWallMs`,
an abort, or dispose settled the run spent host heap building a frame that was
then discarded. The check moves ahead of the snapshot.
Also in this change:
- `readProcessStart` moved after `messageOf`. Inserting it between `messageOf`'s
JSDoc and its body left that function undocumented and the orphaned block
reading as a second doc for the reader; `verify-export-jsdoc` does not catch it
because `messageOf` is not exported.
- The README pair adds the disposed-runtime rejection to `run()`'s public
contract, which `src/index.ts` has enforced all along.
- Known Limitations records three deferred constraints that until now existed
only in review discussion: the combined log-and-value peak the load gate does
not model, the host-side per-member expansion of a wide binding reply (owned by
`packages/core/session`, and shared with the worker-thread backend), and the
absence of fd-3 backpressure for concurrent replies.
- The Agent Note's same-group section records the teardown identity guard and its
two rulings, including why an ABSENT start-time reading proceeds rather than
withholding the signal, and that reading it as a mismatch is what turned the
three same-group heartbeat cases red on Linux.
The directive carried its whole justification inline at 203 characters, past the
140 the @stylistic/max-len rule allows (imports and template-literal messages
are exempt; a line comment is not). The reasoning moves to the lines above and
the directive keeps a short pointer, since a v8 ignore must stay on one line.
The PID-reuse guard has two arms no single OS can execute: the non-Linux early
return in readProcessStart (the Linux coverage lane always takes the read path)
and the refusal arm, which needs a real pid recycled into a new group leader
between spawn and teardown -- no test can schedule that. The coverage lane
reported 99.53% statements / 99.14% branches on src/index.ts for exactly these
two.
Both carry a v8 ignore naming what cannot be reached and why, the convention
this file and subprocess-local already use for platform defenses. The reader
itself stays covered by the process-identity test rather than being exempted
wholesale.
The PID-reuse guard refused to signal whenever the current reading differed
from the one taken at spawn, including when it was ABSENT. On Linux a reaped
leader has no /proc/<pid>/stat, so every teardown after the leader exited
skipped SIGTERM/SIGKILL while the group it led still held survivors -- the
exact case the process-group teardown exists to reap. Three same-group survivor
tests went red on the coverage lane; they pass on Darwin because the reader
always returns undefined there, leaving the guard inert.
Only a present-and-different reading now blocks the signal. Verified on the
self-hosted Linux box: a reaped leader with live survivors allows the signal, a
pid whose start time differs still blocks it, and a live matching process is
signalled.
The README pair described `run()` as rejecting "a malformed binding namespace or
non-positive config", which understated and misplaced the configuration
failures: a non-Unix platform, a non-integer budget, a timer value setTimeout
would clamp, a budget larger than one fd-3 frame, and an incompatible
addressSpaceMb/output-budget pair all throw from the CONSTRUCTOR, so they fail
when the plugin loads rather than on a later run. Both sides now separate the
load-time platform/configuration errors from the run-result contract.
The Chinese README's Model Experience and KV Cache effect sections were still
untranslated English; the pairing record only tracks hashes, so it could not
show that. Both are now translated.
Adding a published Python backend and reordering `flush_line` left several
owning documents stating things that are no longer true.
`src/invariant.ts` justified its empty installer with "ships only the fd-3
wire-protocol codec", which the subprocess execution path contradicts. The
reason now states the actual one: every relation this backend maintains lives
in the CPython child or on the fd-3 wire, so no same-process event sequence is
observable from a listener -- the same shape the sibling worker-thread backend
uses.
The seam's `PORTABLE_RESERVED_WORDS` and `language` JSDoc, the code-runtime
README pair, and docs/subsystems/code-runtime both said only TypeScript has a
published backend. Corrected in all four, with the generated cordis catalog
regenerated for the `language` change.
The note attributed the 12x multiple to the settlement flush holding three
copies. That stopped being true when `flush_line` was reordered to drop the
pending chunks before its push: the binding worst case is the newline path's
single near-budget write. Corrected in the note (both sides) and in the test
comment that repeated it.
The note's Testing section now registers the cases this stack added, and the
Chinese side receives the O(depth) entry it never got plus the new ones -- it
had drifted from the English.
`INTERPRETER_BASELINE_BYTES` argued 64 MiB from a RESIDENT set while RLIMIT_AS
bounds address space. It now cites the bootstrap's own measurement (30.23 MiB
of mappings for `python3 -I`), making 64 MiB roughly twice the measured
baseline.
Also: a hardcoded `(:232-235)` comment reference becomes a reference by name,
a "which now walks in O(depth) too" change narrative becomes a current-state
statement, and a stray double blank line is removed.
Four independent corrections in the run lifecycle.
`killGroup` signalled `-child.pid` with a raw `process.kill`. Node keeps the
numeric `child.pid` after the leader is reaped and only clears its internal
handle, so `child.kill()` refuses while the raw call does not; `close` can
trail `exit` by seconds when a pipe-holding descendant keeps the streams open.
A recycled pgid could therefore receive this run's SIGTERM and armed SIGKILL.
`groupEmpty()` does not cover it: it reports whether the group has members, not
whether they are ours, and it first runs after the signal. The leader's start
time is now read at spawn and re-checked before each signal, matching the
position packages/subprocess/subprocess-local already states
("ProcessIdentity ... preventing teardown escalation after PID reuse"). Kept
local rather than depending on that package, which would add an architectural
edge. Linux reads /proc; Darwin has no /proc, so the reader reports undefined
and the guard degrades to the previous behavior instead of forking `ps` on a
teardown path.
`_push_bounded_prefix` built `(*self._pending, extra)`, copying every pending
reference into a same-size tuple before the bounded loop. For a
single-character drip that is a second pointer array as large as the list:
measured +80 MiB of tuple over a 40 MiB list for 5.2M chunks, the allocation
the bounded prefix exists to avoid. It now iterates the list in place and
handles `extra` in the loop's `else`; 4000 randomized inputs produce byte-identical
prefixes.
The settlement `flush_out()`/`flush_err()` ran outside any guard while `done`
was already decided, so a flush raising under memory pressure skipped
`send_done` and downgraded a child-classified `exception` into a host-side
`worker-exit`. Both are now wrapped, swallowing only the log tail.
The boot re-check's `if effective_soft != RLIM_INFINITY` was dead: `_clamped`
is asked for a finite `addr_bytes` on both sides and each branch returns that
value or a `min` with an inherited bound, so RLIM_INFINITY is unreachable. The
guard could only ever have skipped the re-check it claimed to protect.
Three separate paths in the CPython child allocated state proportional to a
value's width or a string's length, so a legitimate input the byte budgets
admit could die as the program's own MemoryError.
`_lossless_json_violation` enqueued one traversal tuple per member while
running, in `dispatch`, over MODEL-CONSTRUCTED binding arguments that no
child-side byte budget bounds first. It now uses the same (kind, container,
iterator) cursor the other two walks already had, checking dict keys as the
cursor pulls each entry. Measured over `[0] * 6_000_000` (~17 MB of JSON):
459.1 MiB of traversal tuples before, 0.0 MiB after.
`_decode_json_plain` matched JSON strings with a `(?:[^"\\]|\\.)*` repetition,
which makes CPython's engine retain backtracking state proportional to the
string's width: 146 MiB for a 1 MiB string, 557.8 MiB for 4 MiB. A legitimate
multi-megabyte binding reply raised MemoryError inside `_pump_replies`, and
because that pump is the only settler of the call's future, the run stranded
until the wall clock reported `timeout`. Strings now scan chunk-to-chunk over a
character class, which the engine matches without backtracking state; the same
4 MiB decode peaks at the 4.0 MiB result.
`_check_done_value` charged strings and dict keys what
`_dump_string(...).encode()` returned, building the escaped copy plus its
encode to MEASURE it -- ~6x the original each for control-heavy text, so
metering a value the budget then rejects could itself breach RLIMIT_AS and
report `exception` where the seam promises `output-limit`. The new
`_json_str_cost` counts instead, reusing `_json_string_cost`'s C-level passes
and reproducing `_dump_string`'s exact surrogate rules (fold spelled-out pairs,
charge six ASCII bytes per lone surrogate). Identical values, 228.9 MiB -> 19.1
MiB of peak on a 20M-NUL string.
Each fix ships a regression test. The two RLIMIT_AS repros are Linux-only:
Darwin does not apply the limit, so the peaks above are measured directly and
recorded in the test comments.
The O(depth) wide-value regression test ran under `maxWallMs: 20_000`, but the
cursor pulls 6M elements one at a time through Python-level frames: ~11s on an
idle machine, and more under the coverage lane's V8 instrumentation with several
workers sharing a runner. CI reported `timeout` instead of the round-trip.
Raise the run's ceiling to 60s inside a 90s vitest timeout, so the runtime's own
wall clock still fires first on a genuine hang. The assertion is unchanged and
still discriminates: restoring the O(width) `stack.extend` enqueue fails the test
with a child-side MemoryError in ~2.6s.
`_check_done_value` and `_encode_json_plain` pushed one stack entry per child
(plus a separator marker, and `dict.items()` materialized as a list), so the
bookkeeping scaled with the value's WIDTH rather than its depth. A value the
byte meter admits could then die on the walk's own frames: a flat
`[0] * 2_000_000` serializes to 4.0 MB, but measured peaks were 145.2 MB in the
meter and 114.7 MB in the encoder — 28.7x the serialized size, far past the 12x
the load-time address-space gate reserves.
Each container now pushes ONE cursor frame that pulls its children one at a
time and writes into a shared `io.StringIO`, so the output string is the only
width-proportional allocation and the caller already metered its size. Measured
on the same value: 0.0 MB in the meter and 9.0 MB in the encoder (2.3x), with
identical verdicts.
The load gate bounds maxLogBytes and maxValueBytes independently against the
address space, but the child framed the completion value (materializing its
escaped form to meter it, then encoding the frame) while a newline-free log tail
still sat unflushed in _pending. Those two peaks added, so two budgets each
admitted alone could together breach RLIMIT_AS and die as worker-exit instead of
settling. The success path now flushes both log streams before _done_with_value
runs; the trailing flush stays for the exception path and is an idempotent no-op
after a successful settle. A combined-peak regression test (32 MiB each against
512 MiB) asserts the over-budget value reports output-limit rather than OOMing.
Also corrects the worst-case-multiple JSDoc and Agent Note: after 1088d6f03d
made flush_line drop pending before its push, the settlement-flush path holds
two copies, not three, so the newline path is the sole 12x worst case. The
reorder is recorded as a called-out untested fix (the 12x gate already admits
only configs safe under both flush orders).
The load-time output-budget/addressSpaceMb gate used a worst-case multiple of 8,
assuming two simultaneous ~4x astral copies (the built string and its encode).
Three are live at the peak: on the newline path a single write holds the caller's
text argument, the line slice handed to push, and push's encode copy; the
settlement flush_line path held the pending chunks, their join, and that encode
copy. A budget admitted at 8x (e.g. maxLogBytes 48 MiB against addressSpaceMb 512)
could still OOM the child. The multiple is now 12, the strict `>` is `>=` so a
budget whose peak exactly equals the room left after the interpreter baseline is
rejected (that peak plus the baseline is the whole address space), and flush_line
drops the pending chunks before its push to match the newline path's
join-clear-push order. The child re-check mirror and both note sides move in step;
config-catalog is regenerated from the updated field JSDoc.
The boot re-check raises inside bootstrap's setrlimit-phase handler, which
classifies every resource-limit-application failure as kind 'exception'. The
test asserted 'worker-exit'; align it to the actual class and keep the message
assertion so the case still discriminates a config rejection from a generic
setrlimit error. The Agent Note's two references to the reported kind are
corrected on both language sides and the pair re-recorded.
The output-budget/address-space gate's 8x multiple had no room for the
interpreter's own footprint, so a budget sized right at addressSpaceMb/8 was
admitted while its worst-case peak plus the interpreter overran RLIMIT_AS
(e.g. 15 MiB maxLogBytes against 128 MiB). Reserve a fixed
INTERPRETER_BASELINE_BYTES (64 MiB) before the multiple claims the rest, so each
budget times 8 must fit the room LEFT after the baseline.
The host gate validates against the CONFIGURED addressSpaceMb, but a launch
environment can inherit a stricter RLIMIT_AS (a ulimit -v wrapper below
addressSpaceMb) that _clamped lowers the effective limit to, leaving the budgets
sized for a ceiling the child never gets. bootstrap.py now re-checks both budgets
against the effective clamped soft limit after applying it, mirroring the host
gate's multiple and baseline, and raises at boot rather than letting a
near-budget output OOM mid-run.
Add regression tests for both (the load gate against a 256 MiB address space
covering both budgets, and a ulimit -v wrapper for the inherited-limit re-check);
register the tail-copy test in the note Testing section; sync the zh pair. Merges
origin/feat/code-runtime-python-protocol to resolve the DIRTY base.
The tail-copy regression built `"first\n" + "A" * 200 MiB`, whose construction
alone peaks near 400 MiB (the string plus the concat temporary) and OOMs under
the 384 MiB addressSpaceMb before the log path under test runs — a MemoryError in
the model, not the defect. Build the tail in a variable and concatenate only the
newline (peak ~2x150 MiB = 300 MiB, under the address space), so the model's own
allocation fits; the pre-fix code then buffered the whole 150 MiB tail again,
pushing past 384 MiB, while the sliced prefix does not.
The load-time addressSpaceMb gate used a 1/8 fraction derived for ASCII, but the
child ledgers trigger on character count against a serialized-byte budget: an
astral character is one character yet ~4 bytes stored and ~4 encoded, live at
once, so the true worst-case peak is ~8x the budget, not ~2x. Replace the
fraction with an explicit OUTPUT_BUDGET_WORST_CASE_ADDRESS_SPACE_MULTIPLE (8)
and a strict `>`, and gate maxValueBytes the same way as maxLogBytes — the value
path builds and encodes a near-budget completion under the same RLIMIT_AS, so
the incompatible pair was previously admitted there too.
Slice the newline branch's unterminated tail to a budget-sized prefix: it
buffered the whole text[pos:] before the flush trigger could bound it, so an
early newline plus a huge tail made a second full copy of the model's string —
an RLIMIT_AS death the config gate cannot cover since the tail can far exceed
maxLogBytes.
Disclose the cross-field constraint in the maxLogBytes/maxValueBytes/addressSpaceMb
JSDoc (regenerating config-catalog); refresh the note's stale
Buffer.byteLength(JSON.stringify) reference; reconcile the arrival-order rebuttal
with the seam's "in order" logs JSDoc (within-stream, cross-stream best-effort).
Extend the load-rejection test to both budgets and add a tail-copy regression;
sync the zh pair.
The child log ledger encodes an admitted entry to UTF-8 once to charge its
serialized cost, so a maxLogBytes approaching addressSpaceMb lets a legitimate
near-budget log entry breach RLIMIT_AS and die as worker-exit instead of
truncating. Two runtime fixes were tried and both traded one resource bound for
another: an exact serialized-cost check is either a full encode (the allocation
being avoided) or a per-character Python loop that burns the CPU budget (a 10 MB
write hits SIGXCPU under cpuSeconds:1). The breach is a property of the
maxLogBytes/addressSpaceMb pair, not any write, so reject the incompatible pair
at load — maxLogBytes must stay within one eighth of the addressSpaceMb byte
count — and revert _LogStream to its original character-count buffering, which
is memory-safe once the budget fits the address space. The check runs on every
platform since the incompatibility is a config-value property, not a runtime one.
Replace the child-flood regression tests (which asserted the reverted runtime
behavior) with a load-rejection test. The host-side accrueStrayCost UTF-8
per-lead validation and its tests are unaffected. Update the note and zh pair.
accrueStrayCost accepted any 0x80-0xBF continuation, so a CESU-8 surrogate
(ED A0 80) or overlong (E0 80 80) — structurally well-formed but illegal, and
as cheap to flood as 0xFF — was charged its structural width 3 while
toString('utf8') renders each byte as its own U+FFFD (cost 9). Validate each
lead's first-continuation range (WHATWG E0/ED/F0/F4 bounds) and charge 3 per
byte of any sequence outside it, folding a broken prefix to one U+FFFD.
The child _LogStream newline path had the same char-vs-serialized gap the
newline-free trigger had: its per-line fit checks (first reconstructed line and
each subsequent line) compared character count against the serialized-byte
budget, so a control-char line passed and _logs.push encoded it whole, breaching
RLIMIT_AS. Route every check through _fragment_cost_upto, which sums per-char
costs from _json_char_cost over a start/end sub-range without slicing or
encoding and stops at the budget.
Decline arrival-order stray flushing: the two pipes' data events interleave
nondeterministically and logs carries no cross-pipe ordering guarantee, so a
fixed drain order is as valid as any and an arrival-tick branch could not be
covered without a flaky test.
Add CESU-8/overlong, newline-path-flood, and all-lead-class reassembly
regression tests; fix the note's now-inaccurate CESU/illegal-byte claims and a
fixture byte-count comment; sync the zh pair.
The prior child-flush fix measured each fragment with chunk.encode('utf-8'),
which copies the whole write — under a tight addressSpaceMb a single 340 MiB
write died on that encode (the exact allocation _push_bounded_prefix exists to
avoid), and re-scanning the whole pending list per write was quadratic under a
daemon-thread flood (the concurrent-write test timed out at 28s). Compute each
fragment's serialized cost with _fragment_cost_upto, which walks the str via a
new _json_char_cost (code point to escaped width, no encode) and stops once the
running total passes the budget, and accumulate it into _pending_cost once per
write. The early-flush trigger reads that accumulator: still charges control
chars their full serialized width (a NUL is 6 bytes), but never encodes a whole
write and never re-scans the buffer, so the 340 MiB single-write and
daemon-thread tests pass alongside the NUL-flood one.
Rework the child NUL-flood regression to write in 1 MiB chunks under a 512 MiB
address space so its own argument construction is not the allocation under test.
The child-log-flood regression built one 30M-char argument string, which under
the 64 MB addressSpaceMb died on RLIMIT_AS during construction (exit 120) before
the flush trigger under test could run, so it failed on Linux CI. Write the
flood in 1 MiB chunks under a 512 MiB address space instead: the argument str is
never itself the allocation under test, the fixed serialized-cost trigger keeps
the pending tail bounded to a few MiB, and the run completes at the marker; the
pre-fix char-count trigger accumulates the whole ~200 MiB and its ~1.2 GiB
settlement encode breaches RLIMIT_AS. Mirrors the addressSpaceMb budget the
existing oversized-completion tests use.
The host stray-capture cost function charged illegal UTF-8 bytes (0x80-0xC1,
0xF5-0xFF, and orphaned multibyte leads) the raw 1, but toString('utf8')
renders each as U+FFFD (3 serialized bytes). A b"\xff" flood was undercounted
threefold, so the residual grew to a full budget's worth of raw bytes before
flushing and, near a large maxLogBytes, expanded toward a ~1 GiB peak in the
flush's concat plus toString. Replace serializedBufferCost with accrueStrayCost,
a cross-chunk UTF-8 walker that charges each byte its decoded serialized width;
carry its sequence state on each StrayBuffer.
The child _LogStream had the same-family bug: its early-flush trigger compared
_pending_chars (character count) against remaining (a serialized-byte budget),
so a 30M-NUL newline-free flood stayed under a 50 MB char trigger yet encoded to
~180 MB at settlement, breaching RLIMIT_AS as worker-exit. Track _pending_cost
via the _JSON_BYTE_COST table and trigger on it; keep _pending_chars for the
char-based slice bounds.
Correct the note's surrogate claim (only the string-walking jsonStringCostUpTo
charges a lone surrogate six bytes; the byte walker never sees one). Shrink the
post-truncation fixture below PIPE_BUF for a deterministic single callback. List
the shared stdout/stderr budget as a third honest fail-before exception
(cross-pipe arrival timing is nondeterministic). Add illegal-UTF-8,
broken-multibyte, and child-log-flood regression tests; sync the zh pair.
The stray-sealing regression test asserted copied < 2 MiB — about 4x the
defended sealed shape, so reverting the seal to a re-merge (or removing it)
left the test green. Measured both shapes as the fd-3 sibling does: the sealed
shape copies ~120 KB, the re-merge shape ~538 KB. Tighten the bound to 256 KiB,
which sits between them, and record the measurements in the comment and the
Agent Note so the fail-before claim holds.
stdout and stderr each checked their pending serialized cost against the full
logBudget independently, so both could retain nearly a budget's worth of
newline-free residual at once — double the intended peak, up to ~512 MiB near
the ceiling. The flush threshold now reads the COMBINED cost of both pipes and
flushes both when it crosses, since they share one ledger.
Remove the post-truncation admit() v8-ignore: captureStray's per-line loop
makes that branch deterministically reachable within one data callback (a chunk
whose first newline-terminated line exhausts the budget hits it on the second),
so it is measured by a new regression test rather than ignored.
Refresh two stray-output test comments that still named the removed
StringDecoder; the raw-chunk buffer reassembles a split multibyte sequence by
concatenating before it decodes, and the end flush renders a stranded partial
as U+FFFD via toString('utf8').
Three follow-ups the review caught in the stray-capture rewrite, plus a cost
undercount shared with the log ledger.
Seal the stray fragment list into blocks past MAX_PENDING_CHUNKS, mirroring the
fd-3 reader: a program pacing single-byte os.write(1, ...) calls otherwise
accumulates one live Buffer per write, and the per-object overhead no byte
count sees exhausts the host heap far below the budget.
Flush the residual by its running SERIALIZED cost (serializedBufferCost, a
per-byte lower bound) rather than raw byte count: a control-char-dense
newline-free flood serializes several-fold, so a raw-byte threshold let it grow
to a full budget's worth of raw bytes — up to ~6x what the ledger admits —
before flushStray concat/decoded the whole ~256 MiB residual at once.
Charge a lone surrogate its full six escaped bytes (\uXXXX under ES2019
well-formed JSON.stringify) in both jsonStringCostUpTo and serializedBufferCost,
not the three bytes Buffer.byteLength reports for U+FFFD: a forged log frame
flooding \ud800 escapes was undercharged by half and admitted ~2x maxLogBytes.
Key the sync-spawn leak assertion off the exact bootstrap path from the mocked
spawn's argv, immune to a sibling worker's concurrent staging. Refresh the
stale load-check comment that named the replaced JSON.stringify mechanism.
Add lone-surrogate, stray-sealing, and companion regression tests (per-file
100% coverage); update the Agent Note and zh pair.
The line-aggregating stray capture from the previous round regressed three
ways the review caught. Rewrite it on the fd-3 reader's raw-Buffer-chunk
shape: accumulate chunks with a byte counter and split on the raw 0x0a byte,
so a large newline-free write no longer re-copies the residual and re-scans
from index 0 per chunk (both O(N^2)). Meter each admitted entry by serialized
cost through a new jsonStringCostUpTo that walks to the cap and stops, so a
near-budget control-char-dense line never allocates the sixfold-inflated
JSON.stringify result the old ledger did (the critical: ~1.6 GiB transient
under a large maxLogBytes). Flush the residual explicitly in the closeDeadline
handler before it destroys the streams, so a setsid escapee's path (which
fires no end) does not drop a leader's final newline-free diagnostic.
Harden the sync-spawn leak assertion to a set difference against a pre-run
snapshot, immune to a parallel worker's concurrent tmpdir create/delete.
Decline the round-2 request to enforce the fd-3 ceiling per-frame: the counter
check must precede Buffer.concat to prevent ~2x memory doubling (two
regression tests assert this), and the batch-edge false reject it would fix is
reachable only at a maxLogBytes/maxValueBytes configured within one pipe read
of the 256 MiB ceiling, far past the defaults. Documented at the check and in
the note Alternatives.
Add flood, NUL-flood, short-escape, and closeDeadline-flush regression tests
(restoring per-file 100% coverage); update the Agent Note and zh pair.
The line-aggregating stray capture added two branches — the post-truncation
early return and the residual-overflow admit — that the aggregation and
split tests did not exercise, so per-file coverage dropped below 100%. A
2 MB newline-free native write under a 4 KiB maxLogBytes drives the residual
across the budget (admit-and-truncate) and then short-circuits later chunks,
asserting the captured output ends at the truncation marker and stays under
budget rather than buffering the whole flood.
Wrap spawn and the fd-3 narrowing so a synchronous throw (ENAMETOOLONG on
an over-PATH_MAX pythonBin, EMFILE) removes the run's staging directory and
resolves the same worker-exit class as the async error event, instead of
rejecting run() and leaking the directory.
Aggregate native stdout/stderr by real newline rather than by Node data
chunk: logs entries are joined with "\n" downstream, so a newline-free
write larger than one pipe read no longer reads back with spurious breaks.
The ledger still bounds a newline-free flood.
Track a running scan offset in both frame readers so a large frame
accumulated across chunks is scanned once, not re-scanned from 0 per chunk.
Reword the deadline hard-bound v8-ignore to state its real environment
dependence (PID-1-doesn't-reap container, zombie survivor) and cross-ref
the note's rejected signal-0 alternative; fix settle comments that quoted
the pre-qualification teardown contract; document the capMessage vs
_cap_message billing split on both sides; guard the dispose-after-resolve
heartbeat assertion against a vacuous 0===0 pass; reuse
_TRUNCATION_MARKER_BYTES; note the abandoned-call pending-entry bound.
Update the Agent Note Decision/Testing/Alternatives/Consequences for the
above and record the confirmed-empty finalize as a second honest
fail-before exception; sync the zh pair.
The reap-poll deadline arm sent SIGKILL then finalized immediately, declaring
quiescence on mere signal delivery while the group was still dying. It now keeps
polling for the group to actually empty (bounded by one more reap margin) after
its self-sent SIGKILL, so `finished` resolves only on a confirmed-empty group.
ProtocolChannel.read_frame read the boot/run handshake frames through
FileIO.readline() on the unbuffered fd — one os.read(1) per byte, so a
multi-megabyte program burned CPU (RLIMIT_CPU already in force for the run frame)
in millions of syscalls before ast.parse. It now reads in chunks into the same
_pending buffer the async reader uses; the wrapping os.fdopen is gone. read_frame
is this PR's own code (e7f22ed3), not the protocol layer. The chunked read is a
syscall-count improvement with no cross-platform-deterministic failure to assert,
noted as such in the Agent Note.
The closed-loop reply-pump guard now ships with a deterministic regression test:
a worker thread abandons a binding so its loop closes, the host answers that call
before a later binding, and the pump must survive the closed-loop
call_soon_threadsafe to deliver the later reply (host-gated ordering makes it
deterministic; unguarding the pump hangs the later binding to the wall clock).
Align the quiescence self-description with the shipped setsid limitation:
teardown()'s JSDoc and the Agent Note's Problem line now qualify "no subprocess
outlives the fiber" to subprocesses that stay in the child's process group, with
a setsid()-escape exception pointing at the README. Tighten the setsid-orphan
fixture's self-timeout to 5s and its upper-bound assertion to <4000ms so a failed
deadline backstop is a sharper red. Register the new regression tests in the note.
The child reads these byte budgets through int(...), which silently floors a
float, so maxLogBytes: 3.5 would truncate at 3 bytes child-side while the host
meters and marks at 3.5 — the two sides enforcing different public config. Gate
them to integers at load, as the worker backend does; correct the stale comment
that claimed the int()-truncated caps needed no gate. Adds a regression test.
The SIGXCPU timeout message changed from "CPU budget (Ns) exhausted" to name the
configured value as a ceiling; two existing timeout tests asserted the old text.
Assert "CPU time exhausted" to match.
Raising maxValueBytes' load bound to ceiling-envelope assumed both budgets are
metered in serialized (JSON-escaped) bytes, which held for completion values and
logs but not the diagnostic: _cap_message capped by raw UTF-8, so a control-heavy
message near maxValueBytes could serialize sixfold and breach the fd-3 frame
ceiling — the silent worker-exit inversion the load check prevents. _cap_message
now accumulates per-byte serialized cost (new _JSON_BYTE_COST table) and cuts the
prefix that fits. Also reword the host SIGXCPU timeout message to name cpuSeconds
as the configured ceiling rather than a budget a stricter inherited RLIMIT_CPU
soft may undercut. Adds a control-heavy-diagnostic regression test.
The group-reap poll folded its deadline arm into the empty-group arm, so a host
event loop blocked past graceMs + CLOSE_REAP_MARGIN_MS would run the overdue
poll before the grace SIGKILL timer: the group is still non-empty, the deadline
has passed, and the shared arm cancelled the never-fired SIGKILL and finalized —
releasing a SIGTERM-ignoring same-group survivor for good. Split the arms: empty
group cancels the moot timer and finalizes; deadline-with-non-empty-group sends
SIGKILL itself (idempotent if the timer already ran) before finalizing. Adds a
regression test that busy-blocks the loop past both timers and asserts the
survivor's heartbeat freezes.
The group-reap poll's deadline arm (Date.now() >= deadline) is a backstop that
SIGKILL emptying the reachable group never reaches, leaving one uncovered branch
under the per-file 100% gate. Mark it v8-ignore with the reason and drop the
always-true graceTimer-defined guard inside pollGroup (it runs only when killing
is set, so kill() has armed the timer).
settle() dropped the run from `live` eagerly, before the grace-window SIGKILL
reaped a same-group survivor. A dispose() racing a just-resolved run() then
snapshotted an empty `live` and returned while the descendant was still alive,
so teardown's "no subprocess outlives the fiber" (and its JSDoc) was false for
that window. The run now stays in `live` until the process-group poll confirms
the group empty, at which point it is both dropped from `live` and its finished
promise resolved. Adds a regression test asserting dispose() of a completed run
with a same-group survivor returns only after the survivor stops executing.
The class docstring still credited the GIL plus per-frame PIPE_BUF atomicity for
serializing writes, which _write_lock's full-write loop already superseded. State
the current contract (writers serialized by _write_lock around a full-write loop)
and drop the double blank line under the binding-replies note heading.
Record that die_if_cpu_exhausted compares against the effective clamped cpu_soft
in the rlimit section, and add the recheck-timeout test to Testing; re-record pair.
The settlement-time CPU recheck compared spent CPU against the configured
cpuSeconds, but _clamped may have lowered the effective soft limit to a stricter
inherited value. A program that traps SIGXCPU, burns past the inherited soft,
and returns inside the soft-to-hard gap was checked against the configured value
and falsely reported successful, bypassing the inherited limit. The recheck now
uses the clamped cpu_soft. Adds a regression test that inherits a 1s soft CPU
limit and asserts a SIGXCPU-trapping over-burn is a timeout, not a success.
Record the call_soon_threadsafe-onto-a-closed-loop guard in the binding-reply
section of the settlement-fixes Agent Note; re-record the bilingual pair.
A binding called from a worker thread records that thread's loop for its reply.
If the thread finished and closed its loop before the host reply arrived,
_pump_replies' call_soon_threadsafe onto the closed loop raises RuntimeError;
unguarded, that ends the pump task and strands every later reply. Wrap the
schedule in a try/except that drops the moot reply (nothing awaits it) and keeps
the pump serving.
A descendant that calls setsid()/start_new_session leaves the child's process
group, so kill(-pid) teardown cannot reach it; if it also releases the inherited
pipes the run still settles and the fiber goes quiescent while the orphan runs.
This is the containment boundary (model code has bash-equivalent trust), not a
guarantee; reaching such an orphan needs descendant-pid tracking and is deferred.
Two further review findings on the CPython backend:
- The grace-window SIGKILL timer was left armed after settlement, so on a
normal completion a kill(-pid) could fire up to graceMs later and strike a
recycled pgid once the kernel reused the leader's pid. settle() now clears
the timer the moment the process group is confirmed empty (the normal path
and when the poll sees the survivor gone), bounding the reuse window to the
genuine-survivor case where the group cannot be empty to reuse.
- _clamped bounded rlimits by the inherited hard limit only, silently raising
an inherited soft limit stricter than the request (loosening RLIMIT_AS or
deferring RLIMIT_CPU SIGXCPU). It now clamps each side against its own
inherited counterpart and pins soft under hard, keeping the strictest of
configured and inherited. Adds an inherited-soft-limit regression test.
Agent Note expanded to seven fixes with the two new rejected alternatives;
zh pair re-recorded.
Two review findings on the CPython backend:
- Disposal could return while a same-group descendant that ignores SIGTERM
but releases the inherited pipes was still alive: the leader's close fired
and the previous fix relied on an unref'd SIGKILL timer that a short-lived
host never fires, reparenting the survivor to init. settle() now withholds
the run's finished promise on a ref'd process-group poll until the SIGKILL
has emptied the group (bounded by graceMs + margin, zero-cost when already
empty), so teardown's "await each child's exit" holds.
- A binding called from a model worker thread via asyncio.run created its
reply Future on that thread's loop, but _pump_replies completed it directly
from the main loop; asyncio.Future is not thread-safe across loops, so the
call hung to the wall clock. Replies now complete via the owning loop's
call_soon_threadsafe, and a lock serializes the id claim/write/advance.
Tests: the same-group reap case now asserts a heartbeat file stops (robust
whether the killed descendant is reaped or a zombie, so it holds where PID 1
does not wait() orphans); a cross-loop case runs a binding from a worker
thread and asserts the reply round-trips instead of timing out. Agent Note
expanded to all six fixes with rejected alternatives; zh pair re-recorded.
The frame-ceiling cap test asserted the old (ceiling-envelope)/6 bound and
its 44739232 message. The load bound is now ceiling-envelope because both
budgets are metered in already-escaped bytes; assert 268435392.
Address review findings on the CPython backend:
- CRITICAL: a model program could leave a descendant in the child's own
process group that ignores SIGTERM but releases the inherited pipes, so
the leader's `close` fired and settle() cancelled the pending SIGKILL
before it escalated — run()/dispose() returned while that child lived.
kill() now unrefs the grace timer and settle() no longer clears it, so
the SIGKILL reaches the whole group; killGroup swallows ESRCH when the
group is already gone (the normal case). Adds a real-subprocess
regression test.
- WARNING: the maxLogBytes/maxValueBytes load bound divided the frame
ceiling by 6 for escape expansion, but both budgets are metered in
already-escaped serialized bytes, so a payload occupies at most
cap+envelope on the wire. Bound is now ceiling-envelope; drop the unused
escape constant.
- Narrow the runtime.spec.ts header to "no subprocess mocks" (it mocks
node:fs.copyFileSync for staging-failure cases).
- Use full-width punctuation in the README.zh.md prose per translation
rules; re-record the pair.
The package README (both languages) still described this layer as
protocol-only with the PythonCodeRuntime implementation deferred to a
later PR, contradicting the shipped code. Rewrite the intro to describe
the registered runtime, add a Configuration section for every Config cap,
and drop the "implementation not in this layer" limitation. Also pin the
residual-detach fixture's size invariant: the byteLength assertion only
holds above Node's Buffer pool threshold.
The manifest omitted @deepseek-ai/dsh-code-runtime although src/index.ts
imports CodeRuntime and the portable-identifier constants from it and the
tsconfig references ../code-runtime. A three-way package.json merge over
the protocol-layer stub dropped the entry; restore it in peer and dev
dependencies so the declaration matches the import.
Unwrap the English note to one physical line per paragraph (verify-md-wrap)
and retarget the backend link to the fd-3 protocol architecture note that
this stack actually ships (verify-md-links); re-record the bilingual pair.
Land the PythonCodeRuntime implementation on top of the fd-3 protocol
seam: python3 -I per run, binding namespace over fd 3, RLIMIT_CPU/AS,
wall-clock timer, and SIGTERM->grace->SIGKILL process-group teardown,
with the real-subprocess integration suite.
Fixes three defects surfaced on the source PR's review before they ship:
- boot-write failure resolved a worker-exit through finish()/settle()
that read wallTimer/onAbort/live in their TDZ, rejecting run() instead;
the boot write now runs after those bindings and the v8-ignore that hid
the branch is removed.
- log capture serialized against settlement with no lock while model
daemon threads keep writing; LogBuffer now owns one shared re-entrant
lock taken by write/flush_line/push.
- the fd-3 line residual was a subarray view pinning the whole joined
frame; it is copied into a right-sized Buffer via detachResidual so
pendingBytes measures what is retained.
`AGENTS.md` sat at exactly its 1950-word ceiling on master, so the one line this
branch adds to the repository layout — a new top-level package group — cannot
fit at any length: even a one-word description overflows. The description is
condensed to five words and the ceiling raised by ten, the smallest change that
keeps every package group listed. Omitting only `net/` from a list that names
every other group was the alternative.
One new test drove an IPv4-mapped literal through a full fetch. An IPv6 literal
sends `resolvePublicAddresses` looking for a NAT64 prefix before it refuses
anything, and that is a real DNS query — 5s under load, 6ms here, which is why
it passed alone and timed out in the full suite. The three IPv4 cases already
prove the branch end to end without touching the network, so the mapped form is
asserted on the predicate instead.
Review found `127.0.0.2` routed through the proxy. The bypass list carries four
literal loopback entries because that is all a consumer reading an environment
can match, and `proxyForUrl` matched only those — leaving the rest of
`127.0.0.0/8`, `0.0.0.0`, and the IPv4-mapped spellings routed through a proxy
that could then reach them. Loopback is now recognised structurally, which a
list entry cannot express; the published entries stay for the environment
readers.
The same review case exposed a wider one. `web_fetch` skips its address checks
on a proxied hop, because the proxy resolves the origin — but a literal needs no
resolution, so the skip bought nothing and let a proxy on this machine reach
every private range those checks refuse, `169.254.169.254` included. A literal
the checks would refuse now takes the validated path, where the existing
refusal already covers it.
Tests that proved a tunnelled hop used a loopback origin, which no policy can
route through a proxy any more. They name a host only the proxy can answer for
instead — closer to what a proxied request actually looks like.
The user guide promised the proxy carried every outbound request including
telemetry. It carries neither on an older Node, nor anything a model-authored
script sends, so the promise is narrowed and the exceptions listed. A password
in a proxy URL reaching every tool DSH runs is documented there too: it is how
the variable already behaves, and worth knowing before putting one in.
A settled top-level `read_image` call printed its raw attachment object as
literal text in the tool card — `{"type":"image","attachment":{…}}` —
instead of the image, because no presentation metadata told a client card
how to present the reference and the tool-card layer had no image concept.
Host: `read_image` declares an `output.presentationMeta` persisting
`{ path }` only. The attachment reference deliberately lives in the
settled result content — the single record a `tools/post-execute`
replacement rewrites — not in `meta`; the id is opaque and
provider-owned, checked for existence only.
Client: `imageCardModel` derives the card from the call head, the meta
path, the result's own image block, and a shape-matched envelope. ToolRow
gains an `image` card slot; the `read_image` toolview declares the
Tool-owned `tool.call.images` slot as its child and dispatches the
gallery through it. ui-chat down-threads the session-authorized loader
(`ChatNodeOwnerProps.loadImage`), so the tool layer supplies only derived
references plus the loader and never imports an attachment
implementation; ui-attachment fills the slot with its message gallery
renderer. The card keeps the envelope text below the gallery for the
no-attachment-plugin deployment. An image-bearing tool registers a keyed
toolview; the generic fallback keeps its flattened text. `read_image`
joins the read variant with its own locale title key; both rows share
`read-family-row.tsx`.
Verification: `read-image.spec.ts` (metadata projection, envelope by
shape, reference narrowing, real-execution round trip, rejection
branches incl. non-digest ids), `image-card.client.spec.tsx` (derivation,
row render site dispatching the slot, keyed registration with the
child-slot declaration, empty-slot fallbacks, media-type enum), keyless
snapshots (`read-image-gif` added; read-image/-dimension/-reencode
updated to the `{path}` meta), five injected-defect negative controls,
and a demo GIF recorded from this PR's head through the official
image-capable model.
The Windows coverage lane failed on four cases that assume `http_proxy` and
`HTTP_PROXY` are separate variables. They are one variable there: `process.env`
is case-insensitive, and the launch snapshot folds names for the same reason. A
scenario built on "the user set only the lowercase name" cannot exist.
Two of them assert a contract rather than a spelling, so they now hold either
way: what reaches a child for a scheme the user named is the user's own value
and never the derived one, and a nested install carries the active policy's
value rather than the outer install's published one. Both read over the pair of
names instead of one.
The other two are about the case distinction itself. Neither can hold on a
folded environment, so each asserts what that platform does instead of skipping:
the later entry wins where a preference cannot be expressed, and a diagnostic
names the spelling resolution asked for. Both were verified against the folding
arm rather than reasoned about.
A developer's Clash and a CI runner's squid both export HTTP_PROXY and its
siblings. Now that the harness honors them, an ambient value decides test
outcomes: this PR has already recorded a proxy's 502 page as a snapshot's
expected output, and let a runner's own export stand in for "what the user
exported" in an assertion about inherited names.
A Vitest setup file clears the proxy names before any suite runs, wired into
every configuration that declares a setup. Real-API e2e is cleared too: before
proxy support existed every request connected directly and that suite passed, so
direct is the environment it is known to work in.
`NODE_USE_ENV_PROXY` cannot be cleared this way — Node samples the proxy
environment at process start — and the module says so. A proxy application never
exports it, and the eight names one does export are fully handled: with all of
them set, the affected suites pass.
The wiring is what regresses, so that is what the test pins. Configurations are
discovered rather than listed, because the web suites carry no setup today and a
hand-written list would let one of them gain a setup without gaining this one.
`plugin.spec.ts` kept its own copy of the eight names to guard against the
machine; it never sets a proxy variable itself, so the setup replaces that
entirely. `install.spec.ts` had one assertion waiting on a DNS miss with no
deadline, which timed out once under load.
`installGlobalProxy` claimed each worker thread calls it with a policy its host
passed through `workerData`. Nothing does: the two workers this repository ships
evaluate model-authored scripts and are deliberately left without a proxy URL
that may carry credentials.
`agentBindings` now walks the whole tree to reach a dynamic `import('undici')`,
which pushed the repository-wide scan past the 5s default under the coverage
lane's instrumentation.
Both violations name one of two words in source: an agent construction needs a
binding from the undici module, and the option is a property called
`dispatcher`. Skipping a file that mentions neither leaves 21 of 1597 files to
parse, so the scan runs in ~60ms instead of ~500ms — well clear of the timeout
even instrumented.
Second review pass on the outbound proxy work.
The installed dispatcher was undici's EnvHttpProxyAgent, which reuses the HTTP
proxy for `https:` whenever no HTTPS proxy is present. That is exactly the state
this package resolves after refusing a SOCKS or malformed URL the user named for
`https:`, so the scheme the diagnostic reported as direct was tunnelled anyway.
The dispatcher is now an Agent whose per-origin factory calls `proxyForUrl`, so
routing and `proxyForUrl` cannot disagree by parsing the same list twice.
`childProxyEnv` returned only the names the user exported, which left a child
Node direct whenever the proxy came from `ALL_PROXY` or from cordis.yml — Node's
`NODE_USE_ENV_PROXY` reads neither — and stripped the merged loopback bypass so
the child sent its own localhost traffic to the proxy. A scheme the user named
in either casing still reaches the child exactly as written; one they named in
neither now carries the resolved value, and the bypass list is always the merged
one.
A nested install (the plugin mounted over the launcher's policy) recorded the
outer policy's published values as the user's, then cleared the record on
disposal, so every later child inherited the normalization instead. The record
now belongs to the outermost install and is restored, not dropped.
`web_fetch` read the active policy twice — once to skip address pinning, again
inside the transport — so a disposal landing between the two reads produced an
unpinned direct connection to a host nothing validated. One snapshot now decides
both.
Also: the node:http proxy test asserted a route the engines range does not always
have, and the gate could not see undici bound through `await import('undici')`,
the form this repository actually uses.
The case builds a user environment and asserts a child receives it verbatim,
so a runner that exports its own proxy supplied half the "user" values and
decided the assertion.
The workflow worker no longer receives proxy configuration: it executes the
model-authored script body, and a proxy URL may carry credentials. A child
process now inherits the values the user exported rather than this process's
normalization, so a SOCKS proxy set for curl survives and no HTTPS_PROXY is
invented. `mode: 'off'` installs a direct dispatcher instead of recording a
policy the global dispatcher ignores, and the environment snapshot is taken
before any write so Windows restores the user's values.
E2B picks its proxy from the control-plane URL the SDK will really call, the
OTLP agent honors `exporter.keepAlive`, and a scheme whose own value was
refused stays direct instead of borrowing another scheme's proxy.
verify-no-bare-dispatcher parses the TypeScript AST as scripts/AGENTS.md
requires; it immediately found the `{ dispatcher }` shorthand the regex missed.
A replay must not depend on the runner's network policy, the same reason it
pins its home and sessions root. The harness now honors the proxy environment,
so a runner exporting one sent the web-fetch scenario's fixture request to a
proxy that could not resolve the fixture host and recorded that proxy's error
page. Clear the proxy names in both test-support spawners, from the one list
dsh-http-proxy owns.
Both adapters' inference requests were argued from a code read and, for
DeepSeek, from one product smoke. Drive each shipping adapter at an
unresolvable endpoint through a fake proxy instead, and cover pi-ai's provider
stream rather than only its model discovery.
The VFS packer sweeps module requests statically, so dsh-http-proxy's agent
factory made the preview image unpackable: it names node:https for the SDKs
that post through Node's core HTTP modules, a path the worker never takes.
Mock it the way node:net is mocked rather than hiding the request.
NODE_USE_ENV_PROXY reaches Node 24.0+ and 22.21+, while engines admits 22.19.
Assert the direct connection on an older runtime instead of only the proxied
one, so the seam is executable rather than prose.
Node's built-in fetch ignores HTTP_PROXY, so every harness request connected
directly regardless of what the user exported. Resolve one policy from the
launch environment and install it as undici's global dispatcher, then wire the
four surfaces a global dispatcher cannot reach: web_fetch's pinned transport,
the OTLP exporter's node:http agent, the E2B SDK's own proxy option, and the
environment a child process or worker thread is given.
Each outbound call site carries an egress test that drives its real code path
through a fake proxy; that measurement is what found the OTLP and E2B gaps.
# Agent Note: Make packed chunk rows the default JSONL layout
Status: implemented
Archived: 2026-09-01
English | [中文](2026-07-26-packed-chunk-rows-by-default.zh.md)
@@ -18,7 +19,7 @@ Reading is unconditional and layout-blind. Packed, unpacked, and mixed files loa
### Logical events and physical rows
The JSONL packing path stays at the `dsh-session` storage seam through `packChunkRuns()` and `decodeStorageRecord()`. The encoder recognizes exact delta-event shapes, preserves unrecognized events verbatim, and packs only runs of at least three. A packed row is encoding vocabulary, not a `SessionEventMap` member: it never enters `Session.events` or fires `session/event`. The [packed session-history transport decision](2026-08-15-packed-session-history-transport.md) reuses this vocabulary for a bounded lossless wire interval without changing those event semantics.
The JSONL packing path stays at the `dsh-session` storage seam through `packChunkRuns()` and `decodeStorageRecord()`. The encoder recognizes exact delta-event shapes, preserves unrecognized events verbatim, and packs only runs of at least three. A packed row is encoding vocabulary, not a `SessionEventMap` member: it never enters the Session log or fires `session/event`. The [packed session-history transport decision](2026-08-15-packed-session-history-transport.md) reuses this vocabulary for a bounded lossless wire interval without changing those event semantics.
The JSONL backend packs each durable append batch. Raw `compression: 'none'` and default Zstandard framing carry the same logical storage records; selecting raw mode for reviewable fixtures does not disable packing. Repository replay readers and normalizers decode the shared row format instead of maintaining snapshot-specific codecs.
English | [中文](2026-09-03-session-search-result-reveal.zh.md)
## Problem
Selecting a Session search result opened its conversation while leaving the sidebar in the filtered search view. The user could not see where the Session belonged in the normal Workspace hierarchy. Clearing search alone was insufficient because the owning Workspace could be closed, the Session could be hidden beyond the five-row fold, and either grouped or flat navigation could place the row outside the scrollport.
## Decision
[`WorkspaceBrowser`](../../../../packages/client/ui-workspace/README.md) treats result selection as a transition back to normal browsing. It records the target Session id, clears the query, collapses search, and opens the Session. In grouped browsing, `SessionTree` waits until the current Workspace stream has a complete Host baseline, derives and opens the owning Workspace or Ungrouped group, and transiently reveals the hidden remainder only when the target is behind the five-row fold. Flat browsing needs no fold override.
The normal Session row owns completion of the one-shot reveal. A matching mounted row scrolls itself into the nearest visible position and acknowledges the target id, preventing later renders from repeating the scroll. Starting another non-empty search cancels an unacknowledged reveal. Metadata and content matches use the same transition because both result kinds resolve to a Session id.
## Alternatives considered
**Preserve the query after opening the Session.** This keeps the discovery context but leaves the user in the temporary result list and does not identify the Session's normal location.
**Clear search without opening or unfolding the owning group.** The conversation would open while its selected row could remain hidden, reproducing the missing-location problem in a different sidebar state.
**Persist an expanded-all preference for the group.** One navigation would permanently replace the bounded five-row presentation. The reveal instead expands the remainder only for the current tree mount.
**Scroll from the browser parent.** The parent cannot complete the operation before a folded target row mounts. The row that owns the DOM element performs and acknowledges the scroll.
## Consequences
Selecting a result discards the current query and returns the sidebar to normal browsing. The owning group stays open, and its hidden remainder is visible for that tree mount when required, so the selected row supplies both hierarchy context and an on-screen location. Waiting for the current Workspace baseline prevents a reconnect's retained membership from acknowledging the reveal before replacement state arrives. A later search or ordinary render does not repeat the scroll after acknowledgment.
The target row is the only completion signal. If another client archives or moves the Session between result selection and row mount, the reveal remains armed; the row will scroll if it mounts later, unless a new non-empty search cancels the reveal or the browser unmounts.
## Testing
UI tests cover a content-only hit in the sixth position of a closed Workspace, pending and reconnecting Workspace baselines, cancellation by a new search, transient group expansion and scroll acknowledgment, and the same one-shot scroll in flat mode. The assembled Web navigation test verifies that one click clears search and leaves exactly one selected Session row in the normal tree. The long-conversation browser test opens its seeded Session with that single-click transition.
English | [中文](2026-08-31-pr-opened-issue-start-dates.zh.md)
## Problem
The Issue Project records planned work in a `Start date` field, but adding or linking an Issue does not provide a date value. A pull request can identify both Issues it resolves and Issues that supply related implementation context, and either relationship marks the start of repository work.
The organization-level `Start date` Issue field records when work begins, but adding an Issue to the Issue Project or linking it from a pull request does not provide a date value. A pull request can identify both Issues it resolves and Issues that supply related implementation context, and either relationship marks the start of repository work.
Updating the field on every pull-request event would assign dates to existing work after edits, pushes, or reopenings. Replacing an existing date would also discard a manually planned date or a date recorded by an earlier pull request.
## Decision
The Issue lifecycle workflow initializes `Start date` only for `pull_request.opened`. It reads the pull request's live body, retains every same-repository reference that resolves to an Issue, converts `created_at` to a calendar date in the configured Project time zone, ensures the Issue is a Project item, and writes the configured Date field only when the current value is empty.
The Issue lifecycle workflow initializes `Start date` only for `pull_request.opened`. It reads the pull request's live body, retains every same-repository reference that resolves to an Issue, converts `created_at` to a calendar date in the configured Project time zone, ensures the Issue is a Project item, and writes the configured organization Issue Date field only when the current value is empty.
The configuration names the Project field and time zone. Missing configuration fails when the policy module loads; a missing field, a non-Date field, an invalid timestamp, or a failed API request fails the workflow at the first relevant pull request.
The configuration names the field exposed in the Project and the time zone. The Project field must resolve to an organization Issue Date field; the workflow reads its Issue value and updates it through `updateIssueFieldValue`. Missing configuration fails when the policy module loads; a missing field, a non-Date or Project-local field, an invalid timestamp, or a failed API request fails the workflow at the first relevant pull request.
[Event-directed PR review status commands](2026-08-10-event-directed-pr-review-status.md) continue to own Status transitions. Date initialization includes resolving and informational Issue references, runs for Draft and automated pull requests, and does not depend on PR policy enforcement.
## Verification
[Issue-management tests](../../../../.github/issue-management/policy.test.mjs) cover the Shanghai date boundary, opened-only dispatch, all retained Issue references, empty-value writes, existing-value preservation, missing Project items, invalid field configuration, and the GraphQL mutation variables. [Workflow tests](../../../../scripts/ci-workflow.spec.ts) require the `pull_request.opened` subscription.
[Issue-management tests](../../../../.github/issue-management/policy.test.mjs) cover the Shanghai date boundary, opened-only dispatch, all retained Issue references, Issue-field discovery, empty-value writes, existing-value preservation, missing Project items, invalid field configuration, and the `updateIssueFieldValue` variables. [Workflow tests](../../../../scripts/ci-workflow.spec.ts) require the `pull_request.opened` subscription.
## Alternatives considered
**Use a built-in Project workflow.** The built-in workflows own fixed Project item and Status transitions; the repository workflow already owns authenticated GraphQL mutations and can supply the PR creation date.
**Use a Project-local Date field.** A Project field would allow different dates for the same Issue in different Projects and would not appear on the Issue itself. Work begins for the Issue rather than for one Project membership, so the organization Issue field owns the value.
**Process every subscribed PR event or run a reconciler.** Later events would fill dates for existing pull requests and references added after creation, but they would make the field a repair projection instead of a record created with the pull request and would add repeated Project reads.
**Update only resolving Issue references.** Informational references also identify Issues whose implementation work begins with the pull request, so the date initializer uses the existing all-reference set while Status transitions retain resolving-only semantics.
@@ -34,6 +37,6 @@ The configuration names the Project field and time zone. Missing configuration f
## Consequences
Only pull requests opened after the workflow ships initialize dates. References added after creation and existing open pull requests remain unchanged, and the workflow does not scan existing Project items or pull requests.
Only pull requests opened after the workflow ships initialize dates. References added after creation and existing open pull requests remain unchanged, and the workflow does not scan existing Project items or pull requests. The date follows the Issue across organization Projects that expose the field.
The empty-value read makes retries idempotent in ordinary operation. ProjectV2 has no conditional field update, so simultaneous pull requests that reference the same empty Issue can both write; per-PR concurrency does not serialize that Issue, and the last mutation can win.
The empty-value read makes retries idempotent in ordinary operation. The Issue-field mutation has no compare-and-set precondition, so simultaneous pull requests that reference the same empty Issue can both write; per-PR concurrency does not serialize that Issue, and the last mutation can win.
@@ -22,7 +22,7 @@ Responsibility is split between an always-on storage boundary and optional devel
`Session` accepts an event only after one recursive pass has materialized a lossless JSON snapshot. That pass rejects unsupported values and produces the exact detached record that enters the log, so validation and storage cannot observe different values from a stateful getter or retain caller-owned nested references.
The accepted event and all of its descendants are deep-frozen before publication. `append()` returns that owned frozen event, `session/event` observers receive the same record, and `session.events` returns a frozen array snapshot. A previously returned array does not grow after a later append. Seed records pass through the same validation, snapshot, and freeze boundary before construction succeeds.
The accepted event and all of its descendants are deep-frozen before publication. `append()` returns that owned frozen event, and`session/event` observers and `eventAt(seq)` receive the same record. `snapshotEvents(fromSeq?, toSeqExclusive?)` returns a frozen array snapshot; a previously returned array does not grow after a later append.`seq` and `eventAt()` avoid array materialization when a caller needs only the current length or one event. Seed records pass through the same validation, snapshot, and freeze boundary before construction succeeds.
This guarantee belongs in `Session`, not in an optional listener, because every composition relies on trustworthy history. A production deployment, a focused test, or a custom embedding receives the same storage semantics whether or not development support plugins are registered.
@@ -32,7 +32,7 @@ This guarantee belongs in `Session`, not in an optional listener, because every
`dsh-invariants` registers the configurable `ctx.invariants` service and contains no product checks. Every package publishes a `./invariant` ownership companion;`dsh-session`, `dsh-agent`, `dsh-scope`, and `dsh-agent-loop`currently add the rules that require trace state or observation of another seam: monotonic sequence numbers, turn and step nesting, tool-call/result pairing, legal agent-status transitions, subject-correct scoped dispatch, and equality between a loop-built request and the request reconstructed from its session-log prefix. Global enablement and package-name regex filters belong to the service ([package-owned invariant service](2026-07-19-package-owned-invariant-service.md)).
`dsh-invariants` registers the configurable `ctx.invariants` service and contains no product checks. A package publishes a `./invariant` ownership companion only for an independently observable runtime relationship; packages without one omit the companion and record the reason in their README.`dsh-session`, `dsh-agent`, `dsh-scope`, and `dsh-agent-loop`provide the initial rules that require trace state or observation of another seam: monotonic sequence numbers, turn and step nesting, tool-call/result pairing, legal agent-status transitions, subject-correct scoped dispatch, and equality between a loop-built request and the request reconstructed from its session-log prefix. Global enablement and package-name regex filters belong to the service ([package-owned invariant service](2026-07-19-package-owned-invariant-service.md); [omission decision](../simplification/2026-08-28-omit-unneeded-invariant-companions.md)).
When the session companion attaches to an existing or seeded session, it replays the immutable log to rebuild trace state. The service gives each contribution a disposable child fiber, so hot reload is safe in the middle of a turn without giving diagnostics ownership of session storage.
@@ -48,12 +48,12 @@ Freezing history only when an invariants plugin is installed would make the core
### Clone only when deriving messages
Detaching `deriveMessages()` would protect the most common request path but leave other readers of `session.events`, append return values, and session-event observers able to mutate durable history. The log must protect its own boundary; derived projections are an additional isolation boundary, not a substitute.
Detaching `deriveMessages()` would protect the most common request path but leave other readers of `snapshotEvents()`, `eventAt()`, append return values, and session-event observers able to mutate durable history. The log must protect its own boundary; derived projections are an additional isolation boundary, not a substitute.
## Consequences
- Every accepted live or seeded session event is detached from caller-owned inputs and deeply immutable before any observer can receive it.
-`session.events` exposes stable immutable snapshots instead of the private growing array.
-`snapshotEvents()` exposes stable immutable snapshots instead of the private growing array; `seq` and `eventAt()` serve scalar reads without copying that array.
- Request-side mutation cannot reach stored history through derived messages.
- Development builds can enable relational assertions without changing storage behavior, and disposing or filtering a companion does not weaken log immutability.
-`dsh-invariants` configures global enablement plus package allow/block regex lists; each check remains owned and tested by its product package.
@@ -14,23 +14,23 @@ The [event-sourced model](2026-06-11-event-sourced-sessions.md) makes the append
Persistence is a **capability seam** with an abstract Service Definition ([capability seams](2026-06-13-capability-seams.md), the `dsh-shell` template), not loop or core logic:
1.**Interface** (`dsh-session-persistence`, `ctx.sessionPersistence`) — an abstract `SessionPersistence` service: `locate`/`create`/`append`/`prepare`/`load`/`inspect`/`readFrom`/`list`/`listSnapshots`. Its persisted unit IS the existing `SessionEvent` (`{ type, seq, time, data }`), reused verbatim — no conversion type.
2.**Implementation** (`dsh-session-persistence-jsonl`) — an append-only logical JSONL log per session: a `SessionHeader` line followed by storage records that losslessly represent the contiguous `SessionEvent` stream. Eligible `assistant/chunk` delta runs use packed rows by default; [checksummed Zstandard frames](2026-07-19-zstandard-jsonl-session-logs.md) are the default physical encoding, with raw lines configurable.
1.**Interface** (`dsh-session-persistence`, `ctx.sessionPersistence`) — an abstract `SessionPersistence` service: `create`/`open`/`stat`/`list`/`flush`, with `create`/`open` returning per-session `SessionHandle`s that carry `read`/`append`/`flush`/`close` ([handle-based seam](2026-08-27-handle-based-session-persistence.md)). Its persisted unit IS the existing `SessionEvent` (`{ type, seq, time, data }`), reused verbatim — no conversion type.
2.**Implementation** (`dsh-session-persistence-jsonl`) — an append-only logical JSONL log per session: a `SessionHeader` line followed by storage records that losslessly represent the contiguous `SessionEvent` stream. Current v2 writes one event per row; frozen v0 and v1 readers retain their historical packed-delta representation. [Checksummed Zstandard frames](2026-07-19-zstandard-jsonl-session-logs.md) are the default physical encoding, with raw lines configurable.
Key durable, contested choices:
- **The canonical durable log persists every `SessionEvent` losslessly, including `assistant/chunk`.** JSONL storage may encode a consecutive delta run as one packed row, but logical readers reconstruct the exact event boundaries, sequence numbers, and timestamps. `deriveMessages()` skips chunks, and a chunk-filtered rollout (Codex's `policy.rs`) is tempting — but`seq = log.length` and validation of `events[i].seq === i` require a *contiguous* logical log; filtering chunks out would leave holes and break both the contract and resume. A chunk-filtered projection is possible later as a derived view with its own renumbering, but it is NOT the canonical log.
- **Append-only; a crashed turn is closed, never truncated.** Flushed events are never rewritten. The [semantic checkpoint policy](../bug-fix/2026-07-21-semantic-session-checkpoints.md) drains the request before model dispatch, a recorded top-level call before tool dispatch, and the complete response/result batch after a step; the loop drains the final turn boundary. Because one interrupted turn may contain substantial valid work, cold inspection preserves its contiguous, parseable events and adds risk-classified error results for unanswered assistant calls, a missing `step/end`, and `turn/end` with `{ kind: 'interrupted' }`to the in-memory logical view. `prepare` or `load` commits those closers before returning a recoverable view; the synthetic results keep resumed provider transcripts valid. Only an incomplete final record is discarded during committed repair; a parse error or sequence gap at or before the last real `turn/end` is corruption and makes the session unloadable.
- **The file backend is canonical while the service remains extensible.** `dsh-session-persistence-jsonl` is the sole first-party provider and passes `runPersistenceContract`; the abstract service and coordinator remain available to out-of-tree providers. The [JSONL-only persistence decision](../simplification/2026-08-30-jsonl-only-session-persistence.md) owns removal of the first-party database provider and its deliberate compatibility cut.
- **The canonical durable log persists every current `SessionEvent` losslessly.** In v2, one `assistant/message` or `assistant/attempt` embeds the exact timed provider stream for an attempt; `deriveMessages()` projects only the surface message. Dropping embedded stream members is tempting, but it loses replay, timing, usage, partial-failure, and diagnostic facts. Removing a complete event likewise requires dense renumbering because`seq = log.length` and `events[i].seq === i`; the [v1-to-v2 migration](2026-09-01-v2-embedded-assistant-streams.md) performs that rewrite explicitly rather than filtering the canonical log.
- **Append-only; a crashed turn is closed, never truncated.** Flushed events are never rewritten. The [semantic checkpoint policy](../bug-fix/2026-07-21-semantic-session-checkpoints.md) drains the request before model dispatch, a recorded top-level call before tool dispatch, and the complete response/result batch after a step; the loop drains the final turn boundary. Because one interrupted turn may contain substantial valid work, persistence returns its contiguous, parseable events unmodified; the reader owns balancing — resume computes risk-classified error results for unanswered assistant calls, a missing `step/end`, and `turn/end` with `{ kind: 'interrupted' }`(`interruptedTurnClosers`) and appends them through its write handle, while read-only observers add the same closers in memory. The synthetic results keep resumed provider transcripts valid. Only the incomplete fragment of a torn final append is discarded — complete records recovered from it are durably rewritten by the write path before its first new append; a parse error or sequence gap in the committed prefix is corruption and makes the session unloadable.
- **The file backend is canonical while the service remains extensible.** `dsh-session-persistence-jsonl` is the sole first-party provider and passes `runPersistenceContract`; the abstract service remains available to out-of-tree providers. The [JSONL-only persistence decision](../simplification/2026-08-30-jsonl-only-session-persistence.md) owns removal of the first-party database provider and its deliberate compatibility cut.
- **Metadata is out-of-log.** Format version, cwd, and lineage are storage concerns, not replayable conversation state, so they live in a `SessionHeader` owned by `dsh-session` and attached to a `Session` via a new readonly `session.header` — never in `SessionEventMap`, never reaching `deriveMessages()`. `createdAt` is non-negative safe-integer Unix epoch milliseconds: live creation and persistence registration reject fractional values, and JSONL validates the decoded header. The alternative (a merge-extensible `session/meta` event as log line 0) was rejected: an in-log event would ride along with a seeded/forked session for free, but metadata is not replayable state, so the explicit out-of-log header boundary is the cleaner cost. (The header was originally split into an immutable `SessionHeader` plus a mutable `SessionSummary` whose union was `SessionMeta`; the mutable summary was later removed as dead state — see [Drop the mutable session summary](../simplification/2026-06-19-drop-mutable-session-summary.md).)
- **`ctx.agents.create()` and `ctx.agents.resume()` are async factories; resume additionally crosses the persistence boundary.** `ctx.agents.resume({ resumeSessionId })` obtains the exact unpublished Session through `ctx.sessionPersistence.prepare()`, publishes it under the persisted id, and continues its projections. The [Session preparation decision](2026-08-05-session-preparation.md) owns reuse between history inspection and resume. The agent-loop does NOT hard-inject `sessionPersistence` (that would pend non-persistent demos forever); `resume` rejects with a clear error when it is absent.
- **`ctx.agents.create()` and `ctx.agents.resume()` are async factories; resume additionally crosses the persistence boundary.** `ctx.agents.resume({ resumeSessionId })` opens the session's write handle, reads the stored log, and publishes the prepared Session under the persisted id, continuing its projections. The [Session preparation decision](2026-08-05-session-preparation.md) owns the unpublished-Session ownership window. The agent-loop does NOT hard-inject `sessionPersistence` (that would pend non-persistent demos forever); `resume` rejects with a clear error when it is absent.
## Alternatives considered
Each key choice above records its rejected alternative where the choice is stated: a **chunk-filtered canonical log**(Codex's `policy.rs` shape) — breaks the contiguous-seq contract; **truncating a crashed turn** — silently destroys a long autonomous run's real work; an **in-log `session/meta` event as log line 0** — metadata is not replayable state; **finite fractional `createdAt` values** — have no producer and diverge from integer Unix-millisecond storage; **hard-injecting `sessionPersistence` into the loop** — would pend non-persistent demos forever.
Each key choice above records its rejected alternative where the choice is stated: a **stream-filtered canonical log**— loses attempt evidence, while removing events without an explicit migration breaks contiguoussequence numbers; **truncating a crashed turn** — silently destroys a long autonomous run's real work; an **in-log `session/meta` event as log line 0** — metadata is not replayable state; **finite fractional `createdAt` values** — have no producer and diverge from integer Unix-millisecond storage; **hard-injecting `sessionPersistence` into the loop** — would pend non-persistent demos forever.
Format versioning: the header carries a `version`; cold reads reject any non-current version. The pre-release session format stays pinned at`SESSION_FORMAT_VERSION = 0` and carries no broad compatibility promise, while the coordinator may own an explicit narrow import upgrade when persisted user data requires it ([pre-identity message recovery](../bug-fix/2026-07-28-load-pre-identity-session-messages.md)). Append-only + flush is robust to partial trailing writes tolerated during cold preparation; a future provider or write-ahead log needs its own power-loss and recovery contract.
Format versioning: the header carries a `version`; handles expose only`SESSION_FORMAT_VERSION = 2`. JSONL event-body reads compose the static v0-to-v1 and v1-to-v2 adjacent migration chain before returning a handle; the first edge owns bounded legacy normalization, while the second owns Assistant stream embedding and dense reference remapping. V0 remains at suffixless `session.jsonl[.zstd]`, while positive versions use immutable lowercase `session.vN.jsonl[.zstd]` names ([released Session migration](2026-08-31-released-session-format-migrations.md)). Current-generation append and flush are robust to partial trailing writes; a future provider or write-ahead log needs its own power-loss and recovery contract.
## Consequences
The Service Definition, JSONL provider, and metadata contract in `dsh-session` (`session.header`, the `create(id?, options?)` signature) buy durable resume/fork, a read/replay path, crash tolerance, and host-side session access over the existing event-sourced log. The reusable `runPersistenceContract` suite holds the provider and future implementations to the same append-only, contiguous-seq, lazy-materialization, logical-recovery, integer-metadata, and serializability semantics. Persisting the full logical log also settles event fidelity: every `assistant/chunk` survives exactly even when JSONL packs several into one storage row.
The Service Definition, JSONL provider, and metadata contract in `dsh-session` (`session.header`, the `create(header, options?)` signature) buy durable resume/fork, a read/replay path, crash tolerance, and host-side session access over the existing event-sourced log. The reusable `runPersistenceContract` suite holds the provider and future implementations to the same append-only, contiguous-seq, lazy-materialization, logical-recovery, integer-metadata, and serializability semantics. Persisting the full logical log also settles event fidelity: every Assistant attempt retains its exact compact timed stream in one durable settlement.
持久化是一个具有抽象 Service Definition 的**能力 seam**（[能力 seam](2026-06-13-capability-seams.zh.md)，`dsh-shell` 模板），而非循环或核心逻辑：
1.**接口**（`dsh-session-persistence`，`ctx.sessionPersistence`）：一个抽象的 `SessionPersistence` 服务，提供 `locate`/`create`/`append`/`prepare`/`load`/`inspect`/`readFrom`/`list`/`listSnapshots`。其持久化单元就是现有的 `SessionEvent`（`{ type, seq, time, data }`），原样复用，无转换类型。
@@ -16,7 +16,7 @@ Add a **surface** — a derived, cached order of event sequences (the subset of
Every `SessionEvent` gains two optional fields (structural metadata, like `seq`/`time`):
- **`sourceEventSeqs?: number[]`** — seq numbers of earlier events cited as sources (e.g., the `assistant/chunk` seqs that built an `assistant/message`, or the surface nodes shadowed by a compaction marker). A present `[]` is valid only on `assistant/message` and records a known empty provider stream; when the field is absent, a legacy or foreign event does not record which earlier events produced the message. Other surface events require a non-empty list when the field is present. Without these cited seqs, replay cannot validate that a replace-range operation names every event it removed.
- **`sourceEventSeqs?: number[]`** — seq numbers of earlier events cited as sources, such as a `tool/call` cited by its result or surface nodes shadowed by a compaction marker. A present list is non-empty, unique, earlier, and known. V2 `assistant/message` embeds its provider stream and cannot carry this field. Without cited seqs, replay cannot validate that a replace-range operation names every event it removed.
- **`surfaceOp?: SurfaceOp`** — how this event entered the surface. Absent for non-surface events.
1.**Append** — add the new event seq to the tail. Used by `user/message`, `assistant/message`, `tool/result`, `context/message`. The loop passes `surfaceOp: 'append'` on all such appends and records `sourceEventSeqs` where applicable: every successful `assistant/message` records its complete `assistant/chunk` source set, including `[]`, while `tool/result` records its `tool/call` source.
1.**Append** — add the new event seq to the tail. Used by `user/message`, `assistant/message`, `tool/result`, `context/message`. The loop passes `surfaceOp: 'append'` on all such appends and records `sourceEventSeqs` where applicable: `tool/result` records its `tool/call` source, while `assistant/message` owns its embedded stream directly.
2.**Replace** — remove entries from `start` through `end` (both inclusive) and insert the new event seq in their place. Both `start` and `end` must be present in the current surface; `start === end` replaces one entry. The event's `sourceEventSeqs` must contain every shadowed surface seq. The shadowed events remain in the log but are no longer on the surface.
@@ -41,7 +41,7 @@ Delta processing is O(1) when no new events and O(new events) when new events ar
### Persistence
The new fields are serialized as top-level JSON properties. JSONL storage requires no separate column mapping: its lossless JSON boundary preserves both values. The session format `version` is pinned at `SESSION_FORMAT_VERSION = 0`; the optional surface fields are absorbed without bumping it.
The new fields are serialized as top-level JSON properties. JSONL storage requires no separate column mapping: its lossless JSON boundary preserves both values. Released v0 and v1 share this surface representation, and the identity v0-to-v1 edge preserves it exactly; a future structural representation change increments `SESSION_FORMAT_VERSION` and owns an adjacent migration.
### Crash recovery
@@ -49,9 +49,9 @@ The `repair.ts` module synthesizes `tool/result` closers for orphaned tool calls
### Invariants
`Session` validates `sourceEventSeqs` and `surfaceOp` at the always-on seed/append boundary: only `assistant/message` may use an empty source-event list; references are unique, earlier, and known; replacement endpoints exist in surface order; and `sourceEventSeqs` covers every shadowed node. These are single-record acceptance and storage-projection rules, not optional invariant-service contributions.
`Session` validates `sourceEventSeqs` and `surfaceOp` at the always-on seed/append boundary: source lists are non-empty, unique, earlier, and known; `assistant/message` carries no source list; replacement endpoints exist in surface order; and `sourceEventSeqs` covers every shadowed node. These are single-record acceptance and storage-projection rules, not optional invariant-service contributions.
Every surface-eligible event must carry `surfaceOp` or it would disappear from derived history. Typed `append` overloads enforce this for literal event types; runtime checks in `append` and the seed constructor cover widened unions and loaded logs. Invalid seeds are rejected rather than upgraded under the pre-release format policy.
Every surface-eligible event must carry `surfaceOp` or it would disappear from derived history. Typed `append` overloads enforce this for literal event types; runtime checks in `append` and the seed constructor cover widened unions and current loaded logs. Historical v0 validation and normalization belong to the v0-to-v1 edge rather than generic Session code.
@@ -51,7 +51,7 @@ Kept deliberately narrow per the "not every string needs a brand" policy. Each o
- **`ModelId`** (`GenerateOptions.model`, the `LlmRuntime` adapter-registry key) — a real cross-package lookup key (config → agent → llm → adapter); a reasonable next brand, left out only to keep this decision's blast radius focused.
- **`ToolName`** (the `ToolRuntime` key) — author-defined, human-readable, and rarely confused with another id; the weakest candidate, likely not worth a brand.
- **`ErrorCode`** (`HarnessError.code`) — a closed vocabulary (`ABORTED`, `NO_ADAPTER`, …), not a per-instance id; better served by a string-literal union than a brand, if anything.
- **Numeric ordinals** — turn number, step number, and the event `seq` are `number`, not `string`, so `Branded<string>` does not apply; a parallel `number & { readonly [BRAND]: B }` variant could brand them, but they are positional ordinals rarely passed across boundaries, so the payoff is low.
- **Other numeric ordinals** — the [Session sequence and log-offset decision](2026-08-31-session-sequence-and-log-offset-brands.md) brands event identities and log gaps because they cross persistence and reference seams. Turn and step numbers remain plain numbers: they are payload-local ordinals and are not interchangeable with Session event positions.
- **Validated construction** — `brandString<T>()` performs no runtime check, and every boundary (ACP `sessionId`, provider-issued `call.id`, the empty-string fallback in `dsh-llm-deepseek`) trusts the raw string. A `SessionId.parse()` / `isValid()` companion that throws on malformed input at boundaries is a genuine gap, but it is a runtime-behavior change with its own design (what is "malformed"? what happens on failure?) and belongs in its own decision.
@@ -10,7 +10,7 @@ The [per-provider request retry policy](../feature/2026-07-24-provider-retry-pol
Provider adapters can fail by throwing during dispatch or iteration or by ending with `finish { kind: 'error' | 'aborted' }`. The final adapter boundary normalizes thrown values to that terminal finish protocol before `dsh-agent-loop` receives them; middleware and result-processing defects remain thrown. The loop offers a terminal model-request failure to `agent/request-error`. An unhandled failure is terminal; a handling listener repairs policy-owned state, returns `{ kind: 'retry' }`, and stops waterfall delegation. The [retry-action decision](../simplification/2026-07-27-request-error-retry-action.md) owns this return contract.
That boundary is already safe for another request attempt. Raw `assistant/chunk` events carry the failed `turn` and `step`, message derivation ignores them unless a successful `assistant/message` cites them, tool calls are dispatched only after a successful terminal finish and assembly, and a retry reconstructs its next attempt from the durable log. The harness therefore does not need a second response lifecycle or tentative-output protocol to keep two attempts separate.
That boundary is already safe for another request attempt. Each failed stream commits one log-only `assistant/attempt` with its exact compact stream, message derivation ignores it, tool calls are dispatched only after a successful terminal finish and assembled `assistant/message`, and a retry reconstructs its next attempt from the durable surface. The harness therefore does not need a second response lifecycle or tentative-output protocol to keep two attempts separate.
The prior boundary left three narrower gaps.
@@ -38,7 +38,7 @@ interface LlmFailure {
}
```
`code` remains the provider-neutral machine-routing taxonomy established by `HarnessError`; the new fields are observations from the provider boundary. `ProviderRequestId` is owned and constructed by `dsh-llm`, then serializes as its provider-issued string. The payload deliberately has no `retryable`, `failover`, `partialOutput`, provider, model, phase, or route id fields. Retryability belongs to policy, provider/model are already in the durable request header, and partial output is derived from the failed step's `assistant/chunk` events.
`code` remains the provider-neutral machine-routing taxonomy established by `HarnessError`; the new fields are observations from the provider boundary. `ProviderRequestId` is owned and constructed by `dsh-llm`, then serializes as its provider-issued string. The payload deliberately has no `retryable`, `failover`, `partialOutput`, provider, model, phase, or route id fields. Retryability belongs to policy, provider/model are already in the durable request header, and partial output is preserved by the failed attempt's embedded stream.
`LlmError` carries `failure: LlmFailure` and preserves `failure.code === error.code`. `FinishReasonMap.error` and `FinishReasonMap.aborted` carry the same payload instead of parallel failure shapes. The final adapter boundary detaches those facts from adapter-thrown values and emits the appropriate terminal finish; unknown SDK exceptions receive an `UNKNOWN` payload. Exact thrown-object identity does not cross the LLM stream seam.
@@ -82,7 +82,7 @@ Boundary tests prove termination at both actual transports. The hand-written ada
### Keep attempts separate in the existing log
A failed attempt may leave `assistant/chunk` events in its step, but it never appends `assistant/message` and never dispatches a tool. A retry continues inside the failing turn and step, reconstructs the request from the durable surface, and produces its own chunks; only the final outcome closes the turn. UIs may render livechunks while a step is open, then mark or clear that transient view when `llm/retry` identifies the failed attempt or `turn/end` records failure. Web validates the complete retry payload contract, clears the failed partial at `llm/retry`, projects each producer-correlated `retryId` chain into one stable row updated to the latest attempt, and derives scheduled, started, or cancelled status from `llm/retry-started` and the owning turn and step boundaries' closure. Its countdown anchors the scheduled delay to browser receipt rather than the Host event clock, uses ceiling-rounded seconds with a one-second floor, animates only while unresolved, and keeps exact latest failure details collapsed behind the row. Retry nodes anchor their own trajectory turn even when the failed attempt has no assistant node. Message derivation continues to ignore the failed chunks, and Web applies the same projection during history rebuild so refreshing cannot resurrect discarded partials or duplicate retry rows.
A failed attempt appends `assistant/attempt` with its embedded stream, but never appends a surface `assistant/message` or dispatches a tool. A retry continues inside the failing turn and step, reconstructs the request from the durable surface, and produces its own settlement; only the final outcome closes the turn. UIs may render transient `assistant/live-chunk` updates while a step is open, then settle the failed attempt when `llm/retry` identifies it or `turn/end` records failure. Web validates the complete retry payload contract, projects each producer-correlated `retryId` chain into one stable row updated to the latest attempt, and derives scheduled, started, or cancelled status from `llm/retry-started` and the owning turn and step boundaries' closure. Its countdown anchors the scheduled delay to browser receipt rather than the Host event clock, uses ceiling-rounded seconds with a one-second floor, animates only while unresolved, and keeps exact latest failure details collapsed behind the row. Retry nodes anchor their own trajectory turn even when the failed attempt has no surface Assistant node. Message derivation ignores `assistant/attempt`, and Web applies the same projection during history rebuild so refreshing cannot promote failed partials into model history or duplicate retry rows.
If recovery is exhausted, the final failure is stored once on `turn/end.reason` with the structured facts. Web derives one `turn-error` node at that sequence position and renders its display-safe message and optional code inline; AUTH projections replace provider copy that may echo credential fragments with `API key is invalid`, while the raw diagnostic remains in the session log. The same fold runs for live events and history replay. While transient recovery continues, `llm/retry` is the durable home for each intermediate failure and delay; the terminal row exists only once `turn/end` records the error, and because exhausted recovery shares the failing turn, the turn's retry history never suppresses that row — the settled retry chain and the terminal error render side by side. No standalone final-error event or response-id vocabulary is added.
@@ -21,7 +21,7 @@ This vocabulary is the foundation for interception decisions, the durable `hook/
**Three domains, one job each, with a single boundary rule.**
- **`session/*` — the durable, replayable FACT log.** Owns `SessionEventMap`; every entry is JSON-only (no live objects). One `session/event` emit per append, plus the `session/flush` parallel durability checkpoint. It is also the live transcript feed: a consumer that wants to render or react to what happened subscribes here, so live rendering and replay projections share one path.
- **`agent/*` — the LIVE runtime surface.** Always carries the live `Agent`. Interception waterfalls (`agent/pre-step`, `agent/request`, `agent/request-error`) transform, reject, or recover; awaited `agent/turn-stopping` observes the stop boundary; transient emits report lifecycle, status, inbox insertion/claim/discard, and errors. Turn and step BOUNDARIES are NOT here — they are durable session events read off `session/event`, as are the token stream (`assistant/chunk`) and mid-turn steering (a`user/message`).
- **`agent/*` — the LIVE runtime surface.** Always carries the live `Agent`. Interception waterfalls (`agent/pre-step`, `agent/request`, `agent/request-error`) transform, reject, or recover; awaited `agent/turn-stopping` observes the stop boundary; transient emits report lifecycle, status, inbox insertion/claim/discard, errors, and process-local `agent/assistant-stream` frames. Turn and step BOUNDARIES are NOT here — they are durable session events read off `session/event`; Assistant stream evidence becomes durable only inside one `assistant/message` or `assistant/attempt` settlement, and mid-turn steering is a durable`user/message`.
- **`tools/*` — the tool registry and execution pipeline.**
**The boundary rule:** a durable, replayable fact is a `SessionEvent`; a live interception or a transient/live-object signal is an `agent`/`tools` Cordis event. A turn or step boundary is a durable fact, so it lives in the session log and is read off the `session/event` feed — it is NOT mirrored as an `agent/*` emit.
@@ -53,5 +53,5 @@ Like MiniCode, the conversation advances append-only and resets only when model-
-`agent/pre-step` is the current-request message channel; direct inbox mutation is the eventual later-request channel.
- Tool-result trimming needs no new mechanism: a logged single-entry surface replace (`start === end`) carrying a trimmed `tool/result` under the same `callId` — compaction-family, replay-correct, cache-bust batched by the same pressure logic.
- Unreadable referenced attachment objects still fail model requests; [automatic attachment quarantine](../../proposed/bug-fix/2026-08-20-attachment-read-quarantine.md) records the proposed recovery without weakening byte-exact reconstruction.
- Session logs grow one `request/header` snapshot per loop instance, real change, and later model-message series. Repeating the full system prompt and tool catalog is larger than a delta codec but small beside chunk-heavy logs and retains one self-contained replay representation. `SESSION_FORMAT_VERSION` stays `0`; legacy delta events are rejected rather than migrated.
- Session logs grow one `request/header` snapshot per loop instance, real change, and later model-message series. Repeating the full system prompt and tool catalog is larger than a delta codec but small beside chunk-heavy logs and retains one self-contained replay representation. Current v1 retains this single representation; the frozen v0-to-v1 edge explicitly refuses legacy delta events before current Session construction.
- Snapshot fixtures include each repeated series header. Keyless refresh owns those deterministic log changes, while the snapshot harness pins prompt and tool sidecars only for the initial and actual change revisions and reuses the current revision for `series` snapshots. Filesystem-writing fixtures remain in normalized authored form with cwd-relative tool arguments because replay only round-trips cwd-independent argument paths.
@@ -21,6 +21,8 @@ The exe is packaged with the **`--sea` (enhanced SEA) mode** of [@yao-pkg/pkg](h
`--sea` requires target ≥ node22; the exe uniformly targets node24. One pkg invocation packages exactly one target; multi-platform builds invoke it once per platform.
`@yao-pkg/pkg` is an exact-pinned root `devDependency` invoked as `pnpm exec pkg`, with [`patches/@yao-pkg__pkg@6.21.0.patch`](../../../../patches/@yao-pkg__pkg@6.21.0.patch) removing the SEA bootstrap's `patchChildProcess` call. Unpatched, pkg rewrites spawned commands named `node` — including the string after a `-c`/`/c` flag, exactly the Bash tool's `bash -c` form — to the executable itself and stamps `PKG_EXECPATH` into every child environment, so a model-issued `node --version` silently boots the dsh CLI; Node's own SEA layer performs no such rewrite, and a SEA binary cannot impersonate plain Node because it always boots its embedded app. With the call removed, children resolve `node` through PATH like any other process (a machine without Node reports command-not-found honestly), no `PKG_EXECPATH` reaches children, absolute `process.execPath` spawns still re-enter the app, worker threads never applied the hook, and `process.pkg` sidecar selection is untouched.
Terminology reminder: pkg's `/snapshot` VFS has nothing to do with this repo's testing-system "snapshot" (ACP replay expected outputs, `$DSH_SNAPSHOT`); this document says "VFS" for the former.
### The serving interface is a plugin inside the dsh application
@@ -44,13 +46,13 @@ The deploy root includes `@deepseek-ai/dsh-mcp-client` as an explicitly supporte
[`scripts/build-exe-for-python-sdk.ts`](../../../../scripts/build-exe-for-python-sdk.ts): runtime closure verification → `pnpm run build` → (after clearing) `pnpm --filter dsh-python-runtime-closure deploy --legacy --prod --config.node-linker=hoisted --config.auto-install-peers=false --config.link-workspace-packages=true`**directly into**`python/sdk-runtime/src/deepseek_harness_runtime/runtime/node/` → restore direct workspace packages omitted by legacy deploy and reject any remaining manifest gap → replace staged dependency symlinks with their target bytes, remove package-manager `.bin` links, and fail if any symlink remains → inject pkg configuration whose bin is `node_modules/@deepseek-ai/dsh/lib/bin.js` and whose assets cover dynamic profile, bundle, frontend, preset, native-library, and configuration reads → stage the target `node-pty` addon → invoke `pkg --sea` once per target → write `deepseek-harness-sdk-runtime-<platform>-<arch>` under `dist-exe/` and copy it into the runtime directory. Linux CI rebuilds `pty.node` inside the matching manylinux 2.28 container because legacy deploy omits that install side effect. Every target copies its native `@vscode/ripgrep` binary beside the executable as the required `-rg` sidecar; pkg runtimes select that sidecar through `process.pkg`, while ordinary Node execution uses `@vscode/ripgrep` directly. macOS uses its target prebuild and also emits the required `-spawn-helper`. All four deploy flags are grounded in measurement: `--legacy` is the mandatory path with inject-workspace-packages off; hoisted gives pkg a stable single-instance layout that the explicit materialization pass makes symlink-free; disabling automatic peer installation prevents undeclared peers from expanding the closure; link-workspace-packages selects direct workspace dependencies. [`pnpm-workspace.yaml`](../../../../pnpm-workspace.yaml) overrides the transitive `@deepseek-ai/cosmokit` and `@deepseek-ai/schemastery` semver requests to the pinned vendor sources so legacy deploy never resolves those unpublished names from a registry.
CI: [`.github/workflows/build-exe-for-python-sdk.yml`](../../../../.github/workflows/build-exe-for-python-sdk.yml) is called for all four targets by the [installed-wheel Python runtime pull-request validation](../testing/2026-08-23-installed-python-wheel-black-box-ci.md) and the [public publication workflow](../process/2026-08-11-python-publication-workflow.md); `workflow_dispatch` can still select a subset. Native builds run on linux-x64 / linux-arm64 (`ubuntu-24.04-arm`) / macos-arm64 / win-x64 (`windows-2025`), with `~/.pkg-cache` cached where applicable, and pkg handles macOS ad-hoc signing. Each leg installs the release-shaped SDK and runtime wheels into a clean venv outside the checkout, proves their package and executable provenance, then drives the complete keyless scenario set through the public SDK and direct NDJSON JSON-RPC. Trusted pull requests additionally run a real DeepSeek two-turn tool smoke on every target; fork and Dependabot heads receive no key. Linux inspects the executable and native addon's GLIBC requirements and runs an additional manylinux 2.28 smoke, while macOS verifies that the executable's deployment target fits the wheel tag. A full four-target run retains five artifacts, each containing one release file: the platform-independent SDK wheel and four native runtime wheels; a subset dispatch retains the SDK wheel and selected runtime wheels. Bare executables and source bundles remain intermediate test inputs. [`.gitlab-ci.yml`](../../../../.gitlab-ci.yml) accepts `python-v<repository-version>` tag pipelines whose version matches the root `package.json`, builds one SDK wheel and four native runtime wheels, then a single serialized job checks and publishes all five to the project PyPI registry. The [Windows x64 runtime decision](2026-08-23-python-sdk-windows-x64-runtime.md) owns the fourth target and the explicit exclusion of Windows arm64.
CI: [`.github/workflows/build-exe-for-python-sdk.yml`](../../../../.github/workflows/build-exe-for-python-sdk.yml) is called for all five targets by the [installed-wheel Python runtime pull-request validation](../testing/2026-08-23-installed-python-wheel-black-box-ci.md) and the [public publication workflow](../process/2026-08-11-python-publication-workflow.md); `workflow_dispatch` can still select a subset. Native builds run on linux-x64 / linux-arm64 (`ubuntu-24.04-arm`) / macos-arm64 / macos-x64 (`macos-15-intel`) / win-x64 (`windows-2025`), with `~/.pkg-cache` cached where applicable, and pkg handles macOS ad-hoc signing. Each leg installs the release-shaped SDK and runtime wheels into a clean venv outside the checkout, proves their package and executable provenance, then drives the complete keyless scenario set through the public SDK and direct NDJSON JSON-RPC. Trusted pull requests additionally run a real DeepSeek two-turn tool smoke on every target; fork and Dependabot heads receive no key. Linux inspects the executable and native addon's GLIBC requirements and runs an additional manylinux 2.28 smoke, while macOS checks the runtime, ripgrep, and PTY helper architectures and verifies that all three deployment targets fit the wheel tag. A full five-target run retains six artifacts, each containing one release file: the platform-independent SDK wheel and five native runtime wheels; a subset dispatch retains the SDK wheel and selected runtime wheels. Bare executables and source bundles remain intermediate test inputs. [`.gitlab-ci.yml`](../../../../.gitlab-ci.yml) accepts `python-v<repository-version>` tag pipelines whose version matches the root `package.json`, builds one SDK wheel and five native runtime wheels, then a single serialized job checks and publishes all six to the project PyPI registry. The [Windows x64 runtime decision](2026-08-23-python-sdk-windows-x64-runtime.md) owns the Windows target and the explicit exclusion of Windows arm64.
### Python SDK distribution: two carriers, exe for production, node for development
The Python SDK lives at [`python/`](../../../../python/README.md): `python/sdk` is the client and `python/sdk-runtime` is the runtime carrier package. The runtime package's data directory holds the build-injected platform executable with its required `-rg` sidecar and optional macOS helper, plus the build-injected `runtime/node/` closure tree for repository development. `resolve_bundled_launch_args()` selects the executable by default; explicit `DSH_RUNTIME_MODE=node` runs `runtime/node/node_modules/@deepseek-ai/dsh/lib/bin.js` on system Node 22.19 or newer. The node carrier never enters wheel distributions, and neither carrier uses a checked-in complete `cordis.yml`.
[`scripts/build-python-release.py`](../../../../scripts/build-python-release.py) reads the authoritative `X.Y.Z` or prerelease version from the repository root `package.json`, converts prereleases to their PEP 440 spelling, and stages both packages at that wheel version, with `deepseek-harness-sdk` depending exactly on the matching `deepseek-harness-runtime-bin`. An optional `python-v<repository-version>` release tag is a consistency assertion and is rejected when it differs from the repository version; the source `pyproject.toml` development sentinel never determines a release version. Staging also carries the repository license into both wheels and the third-party notices into the bundled runtime wheel. The SDK is a `py3-none-any` wheel; each wheel-only runtime package contains one exe and its architecture-matched ripgrep sidecar, and the macOS wheel also contains its architecture-matched spawn helper. Runtime wheels use `py3-none-manylinux_2_28_x86_64`, `py3-none-manylinux_2_28_aarch64`, the conservative`py3-none-macosx_14_0_arm64` tag for the Node 24 executable's macOS 13.5 deployment target, or `py3-none-win_amd64`; the Hatch hook rejects sdists, universal tags, mixed-platform payloads, missing or extra sidecars, and unsupported platforms.
[`scripts/build-python-release.py`](../../../../scripts/build-python-release.py) reads the authoritative `X.Y.Z` or prerelease version from the repository root `package.json`, converts prereleases to their PEP 440 spelling, and stages both packages at that wheel version, with `deepseek-harness-sdk` depending exactly on the matching `deepseek-harness-runtime-bin`. An optional `python-v<repository-version>` release tag is a consistency assertion and is rejected when it differs from the repository version; the source `pyproject.toml` development sentinel never determines a release version. Staging also carries the repository license into both wheels and the third-party notices into the bundled runtime wheel. The SDK is a `py3-none-any` wheel; each wheel-only runtime package contains one exe and its architecture-matched ripgrep sidecar, and the macOS wheel also contains its architecture-matched spawn helper. Runtime wheels use `py3-none-manylinux_2_28_x86_64`, `py3-none-manylinux_2_28_aarch64`, `py3-none-macosx_14_0_arm64`, `py3-none-macosx_14_0_x86_64`, or `py3-none-win_amd64`; the Hatch hook rejects sdists, universal tags, mixed-platform payloads, missing or extra sidecars, and unsupported platforms. Both macOS tags deliberately declare a conservative 14.0 installation floor: the packaged Node 24 executables declare macOS 13.5, and the x64 PTY helper declares 10.7, but release validation proves the complete payload only against the 14.0 wheel claim rather than promising each observed component minimum as a supported host. The two macOS wheels remain architecture-specific; no universal2 wheel is published.
The Python client launches the packaged `dsh` command with the selected profile (`sdk` by default), ordered patch files, and an explicit Harness home. The profile owns JSON-RPC serving and application composition; missing homes, profiles, bundles, patches, and server rows fail without an external complete-config fallback.
@@ -64,7 +66,7 @@ The Python client launches the packaged `dsh` command with the selected profile
## Testing
The verification surface has three tiers. Mechanism tier: the measured conclusions for the `--sea` chain are embedded in the Decision sections (ESM dynamic import inside the VFS, single cordis instance, fail-loud config chain, `node:sqlite`, macOS ad-hoc signing runs). SDK tier: the complete keyless pytest suite covers the client protocol against a fake runtime peer, subprocess cleanup, absolute cwd propagation, dual-carrier launch, and carrier resolution; root CI runs it on Python 3.10. End-to-end tier: every platform build installs both wheels into a clean venv outside the checkout, proves matching versions and installed module/executable locations, then completes turns against a mock endpoint through the default SDK path, a custom config, the checked-in standalone minimal composition, and the direct binary protocol, with final text and JSONL checked. The minimal run asserts its exact system prompt and two-tool catalog, retains Bash state across calls, and invokes the editor. The custom config additionally drives `run_code` and a zero-agent `workflow` through their real worker files inside the packaged VFS. The filesystem-search scenario requires the model to call both `glob` and `grep` through the target-native `-rg` sidecar. The MCP scenario starts a temporary external stdio server, deliberately delays its initial `tools/list` response, then immediately starts the first SDK prompt; the prompt must see and call the discovered tool, proving that `initialize` is a real Loader-settlement readiness boundary rather than a timing sleep. The same installed run compares a committed executable-specific snapshot through the Python SDK: a keyless scripted model mounts a Cordis plugin that registers a tool, invokes that tool from `run_code`, runs a direct spawn subagent and a workflow that starts a second spawn child, then unmounts the plugin. The fixture explicitly disables its unused bundled Bash and local skill discovery so its tool set does not depend on repository-external state, and the comparison normalizes opaque message, agent, workflow-run, and session IDs across the SDK result and notification stream plus the parent and two child JSONL logs. Trusted pull requests add a real-provider two-turn file write/read whose external bytes, tool calls, completed reasons, and persisted log must agree. This harness stays separate from ACP's `pnpm run test:snapshot` because the protocols and build artifacts differ.
The verification surface has three tiers. Mechanism tier: the measured conclusions for the `--sea` chain are embedded in the Decision sections (ESM dynamic import inside the VFS, single cordis instance, fail-loud config chain, `node:sqlite`, macOS ad-hoc signing runs). SDK tier: the complete keyless pytest suite covers the client protocol against a fake runtime peer, subprocess cleanup, absolute cwd propagation, dual-carrier launch, and carrier resolution; root CI runs it on Python 3.10. End-to-end tier: every platform build installs both wheels into a clean venv outside the checkout, proves matching versions and installed module/executable locations, then completes turns against a mock endpoint through the default SDK path, a custom config, the checked-in standalone minimal composition, and the direct binary protocol, with final text and JSONL checked. The minimal run asserts its exact system prompt and two-tool catalog, retains Bash state across calls, and invokes the editor. The custom config additionally drives `run_code` and a zero-agent `workflow` through their real worker files inside the packaged VFS. The filesystem-search scenario requires the model to call both `glob` and `grep` through the target-native `-rg` sidecar. The spawn-node scenario drives the platform shell tool through a command starting with `node` and requires the machine's own Node version in the tool result with no `PKG_EXECPATH` in the child environment, pinning the packaged runtime against a pkg upgrade that re-records the child-process patch. The MCP scenario starts a temporary external stdio server, deliberately delays its initial `tools/list` response, then immediately starts the first SDK prompt; the prompt must see and call the discovered tool, proving that `initialize` is a real Loader-settlement readiness boundary rather than a timing sleep. The same installed run compares a committed executable-specific snapshot through the Python SDK: a keyless scripted model mounts a Cordis plugin that registers a tool, invokes that tool from `run_code`, runs a direct spawn subagent and a workflow that starts a second spawn child, then unmounts the plugin. The fixture explicitly disables its unused bundled Bash and local skill discovery so its tool set does not depend on repository-external state, and the comparison normalizes opaque message, agent, workflow-run, and session IDs across the SDK result and notification stream plus the parent and two child JSONL logs. Trusted pull requests add a real-provider two-turn file write/read whose external bytes, tool calls, completed reasons, and persisted log must agree. This harness stays separate from ACP's `pnpm run test:snapshot` because the protocols and build artifacts differ.
Manual-driving caveat: the bin treats stdin EOF as "the client is gone" and disposes immediately, so a short-lived pipe aborts an in-flight turn — pipe-driven runs must keep stdin open until the turn ends.
@@ -84,4 +86,4 @@ Manual-driving caveat: the bin treats stdin EOF as "the client is gone" and disp
**Bought**: zero-dependency single-file distribution on target platforms; plugin semantics strictly identical to running from source (the same real package tree, no transpilation, no registry); the serving interface, the plugin set, and the configuration all converge on two sources of truth — `cordis.yml` plus one dependency manifest; the exe and node carriers share one tree and one semantics, so development verification never waits for packaging; official Node binaries remove the patched-binary supply-chain concern.
**Paid**: artifacts on the order of 174MB with source entering the blob as-is (no bytecode obfuscation; a closed-source distribution requirement needs a separate evaluation); pkg's VFS/module-hook layer remains community-maintained (the build script pins `@yao-pkg/pkg@6.21.0`; upgrading is an explicit change); `--sea` is one invocation per target (matching CI's one leg per platform; local multi-platform builds are serial).
**Paid**: artifacts on the order of 174MB with source entering the blob as-is (no bytecode obfuscation; a closed-source distribution requirement needs a separate evaluation); pkg's VFS/module-hook layer remains community-maintained (`@yao-pkg/pkg` is an exact-pinned, pnpm-patched root devDependency; upgrading re-records the patch and is an explicit change); `--sea` is one invocation per target (matching CI's one leg per platform; local multi-platform builds are serial).
The JSON-RPC runtime receives provider and model explicitly. Its convenience fallback mounts `dsh-llm-deepseek` only for provider `deepseek` when that provider has no registered owner; other missing providers fail without guessing an adapter.
The on-disk session format remains the pre-release pinned version `0`, with no compatibility promise. Seed/load validation rejects request headers and assistant messages that omit required provider/model fields instead of accepting an old shape that can no longer reconstruct the request.
Current v1 seed/load validation rejects request headers and assistant messages that omit required provider/model fields. The frozen v0-to-v1 edge requires the same reconstructable routing identity before migration; it never guesses a missing provider or model, and malformed shapes refuse before publication.
## Alternatives considered
@@ -78,7 +78,7 @@ The on-disk session format remains the pre-release pinned version `0`, with no c
- pi-ai credentials, transport knobs, SDK timeouts, and the five-minute-default `streamIdleTimeoutMs` watchdog are scoped per provider profile. Hidden provider retries are disabled; bounded retries belong to the separately composed agent recovery policy.
-`dsh-llm-pi-ai` rejects stop sequences because pi-ai's common stream API cannot express them; the native DeepSeek adapter retains its stop support.
- Replay state is portable only within the adapter instance that owns both the historical and target providers. Cross-provider and cross-model restoration is an adapter responsibility, and another adapter receives provider-neutral history without the opaque state.
- Current pre-release session JSONL requires provider/model on request headers and assistant messages. Older shapes remain version `0` but are rejected rather than migrated.
- Current v1 Session JSONL requires provider/model on request headers and assistant messages. The v0 edge migrates only frozen shapes that already carry reconstructable request identity.
## Testing
@@ -88,4 +88,4 @@ The on-disk session format remains the pre-release pinned version `0`, with no c
## Risks
This is a repo-wide pre-release API break: model-only request construction, adapter registration, app protocols, fixtures, and persisted version-0 event shapes all change together, with no compatibility aliases. The provider exclusivity rule deliberately prevents two implementations of the same upstream from coexisting in one context. A pi-ai dependency update can change the accepted provider/model catalog, so the lockfile and adapter e2e matrix define the tested set. Custom `baseURL` endpoints inherit the chosen catalog model's protocol assumptions and cannot repair an incompatible proxy. Catalog-external model descriptors and multimodal content remain unsupported. pi-ai replay state may contain opaque encrypted reasoning signatures; it is persisted because the provider requires it for continuity, but it is never rendered or logged outside the existing session record.
This was a repo-wide API break when introduced: model-only request construction, adapter registration, app protocols, fixtures, and persisted v0 event shapes changed together, with no compatibility aliases. Released historical recovery now belongs to the adjacent Session-format edge. The provider exclusivity rule deliberately prevents two implementations of the same upstream from coexisting in one context. A pi-ai dependency update can change the accepted provider/model catalog, so the lockfile and adapter e2e matrix define the tested set. Custom `baseURL` endpoints inherit the chosen catalog model's protocol assumptions and cannot repair an incompatible proxy. Catalog-external model descriptors and multimodal content remain unsupported. pi-ai replay state may contain opaque encrypted reasoning signatures; it is persisted because the provider requires it for continuity, but it is never rendered or logged outside the existing session record.
@@ -14,20 +14,20 @@ Some packages genuinely own no continuously observable relation. Pure utilities,
## Decision
### Registration is exhaustive; assertions must be meaningful
### Published assertions must be meaningful
Every workspace package publishes a separately built `./invariant` companion and registers its exact npm package name. A companion does one of two things:
A workspace package publishes a separately built `./invariant` companion only when it owns an independently observable runtime relationship. A published companion:
- installs a package-owned check over an event stream or relevant mutable data structure and reports violations through its bound `fail(message)` reporter; or
-uses an empty installer whose declaration has an owner-specific `No runtime invariant:` comment explaining why the package has no plausible runtime relation to observe.
- installs a package-owned check over an event stream or relevant mutable data structure and reports violations through its bound `fail(message)` reporter; and
-registers the package's exact npm name while keeping diagnostics outside the root entrypoint.
The empty form is an explicit architectural conclusion, not a generated placeholder. A future package change that introduces mutable state or an event protocol must replace the explanation with the corresponding check.
When no plausible relationship exists, the package omits the companion and publication wiring and records its package-specific reason in the README. A future change that introduces an independently observable relationship must replace the explanation with the corresponding check. The omission mechanics and current audit are owned by the [omit-unneeded-companions decision](../simplification/2026-08-28-omit-unneeded-invariant-companions.md).
The central `dsh-invariants` service owns only configuration, registration uniqueness, child-fiber lifecycle, rollback, disposal, and package-attributed failure. It exposes no generic plugin-shape, service-shape, or startup-assertion helpers and imports no product package.
### Implemented checks
### Representative implemented checks
The current 103-package workspace has 21 executable companions and 82 justified empty companions.
Published companions are enumerated mechanically by `verify-package-invariants`; the current audit count is recorded in the [omit-unneeded-companions decision](../simplification/2026-08-28-omit-unneeded-invariant-companions.md). The table below samples representative runtime relationships rather than listing every companion.
| Owner | Runtime relationship |
|---|---|
@@ -57,13 +57,13 @@ Session-backed companions validate existing durable events when they load, using
### Repository gate and tests
`verify-package-invariants` discovers every workspace package and enforces companion source, exact-name registration, named-only Loader shape, `./invariant` exports, publication files, dependencies, TypeScript references, and bundle entries. Its AST rule rejects generated markers, default exports, and unexplained empty installers. A non-empty installer must accept and use the failure reporter, and registration must pass that checked local `install` function. The gate deliberately does not infer semantic quality from method names or helper calls.
`verify-package-invariants` discovers every workspace package. It accepts clean omission, rejects stale or partial companion wiring, and enforces exact-name registration, named-only Loader shape, `./invariant` exports, publication files, dependencies, TypeScript references, and bundle entries for published companions. Its AST rule rejects generated markers, default exports, and empty installers. Every installer must accept and use the failure reporter, and registration must pass that checked local `install` function. The gate deliberately does not infer semantic quality from method names or helper calls.
Vitest mounts `InvariantRegistry` with `{ enabled: true }` for every package test topology and loads the owning companion. The invariant subpath path mapping resolves source companions instead of stale built output. Focused suites cover every executable companion's valid and invalid observations, and the exhaustive topology runs every source companion through the real Loader namespace normalization. After the structural gate validates each publication map, an artifact gate stages its manifest-declared `lib/` files, imports the compiled `./invariant` self-reference under plain Node, and repeats that Loader-shape check, so a companion that imports an undeclared runtime chunk fails before release. Tests that synthesize event streams must produce a valid surrounding lifecycle unless the test is intentionally asserting a violation.
Vitest mounts `InvariantRegistry` with `{ enabled: true }` for every package test topology and loads the owning companion when one is published. The invariant subpath path mapping resolves source companions instead of stale built output. Focused suites cover every published companion's valid and invalid observations, and the exhaustive topology runs every source companion through real Loader namespace normalization. After the structural gate validates each publication map, an artifact gate stages its manifest-declared `lib/` files, imports the compiled `./invariant` self-reference under plain Node, and repeats that Loader-shape check, so a companion that imports an undeclared runtime chunk fails before release. Tests that synthesize event streams must produce a valid surrounding lifecycle unless the test is intentionally asserting a violation.
## Alternatives considered
- **Keep generated empty companions.** Rejected because an unexplained placeholder can survive after a package gains a meaningful runtime relation.
- **Keep explained empty companions.** Rejected because source, publication, dependency, and test wiring are disproportionate machinery for a negative conclusion that belongs in the package README.
- **Require an assertion from every package.** Rejected because method-presence, plugin-shape, and fixed-example assertions duplicate stronger type, load, and unit-test contracts without checking runtime consistency.
- **Keep generic shape helpers in the service.** Rejected because they blur compile-time API validation with runtime invariants and encourage centrally defined product assumptions.
- **Move the product checks into the service.** Rejected because product vocabulary, dependencies, tests, and change ownership belong with the package that emits the data.
@@ -71,8 +71,8 @@ Vitest mounts `InvariantRegistry` with `{ enabled: true }` for every package tes
## Consequences
-Every package has visible ownership and publication wiring, but only packages with a plausible runtime relation add listeners or trace state.
- Empty companions remain reviewable decisions with package-specific explanations and fail the gate if the explanation is removed.
-Packages with a plausible runtime relation have visible ownership and publication wiring; packages without one record the omission reason in their README.
- Empty companions fail the gate, and partial omission wiring fails before build or release.
- Type declarations, Cordis loadability, plugin metadata, service method APIs, and pure algebra remain covered by their owning compile, load, unit, or integration gates.
- Runtime failures identify the owning npm package and point to an inconsistent observation rather than restating a required API shape.
- The original selection, blocklist precedence, duplicate ownership, rollback, disposal, and HMR service contracts remain unchanged.
Deployments that opt into diagnostics need more than presence or absence of one plugin. Such a composition carries the known invariant contributions while permitting a global off switch and package-selective diagnostics. Selection must remain stable when a package loads later or reloads under HMR, and disabled contributions must not allow two plugins to claim the same package name silently.
Package ownership must also be exhaustive. Without a mechanical repository rule, a new package can omit the companion, dependency, or publication wiring and remain invisible to diagnostics until a maintainer notices the gap.
Published ownership must be mechanically complete. Without a repository rule, a package can expose a partial companion, dependency, or publication map and remain broken until a maintainer notices the gap; packages that publish none must keep their reason reviewable in the README.
## Decision
@@ -18,7 +18,7 @@ Package ownership must also be exhaustive. Without a mechanical repository rule,
`@deepseek-ai/dsh-invariants` is a product-independent Cordis service plugin that registers `ctx.invariants`. It owns configuration, registration uniqueness, child-fiber lifecycle, and package-attributed failures. It imports no session, agent, scope, or agent-loop package and contains none of their checks.
Every workspace package publishes a `./invariant` companion plugin that registers its exact full npm name. A companion checks a meaningful event or mutable-data relationship when its owner has one; otherwise it carries an owner-specific explanation for its empty installer. Generated ownership placeholders and synthetic API-shape assertions are forbidden by the follow-up [runtime-contract Agent Note](2026-07-19-package-invariant-runtime-contracts.md). Package root entrypoints do not import or register diagnostics implicitly, so loading a root package does not change runtime checking or require the invariant service.
A workspace package publishes a `./invariant` companion plugin only when it owns an independently observable event or mutable-data relationship. The companion registers its exact full npm name. Packages without such a relationship omit the companion and publication wiring and record the reason in their README; generated placeholders, empty installers, and synthetic API-shape assertions are forbidden by the [runtime-contract Agent Note](2026-07-19-package-invariant-runtime-contracts.md) and [omission decision](../simplification/2026-08-28-omit-unneeded-invariant-companions.md). Package root entrypoints do not import or register diagnostics implicitly, so loading a root package does not change runtime checking or require the invariant service.
### Configuration and selection
@@ -64,9 +64,9 @@ The former functional-plugin entry point and one-argument `InvariantError` const
These four owners supplied the initial stateful checks. The follow-up runtime-contract decision adds checks for seventeen more owners with real event or mutable-data relationships and records justified empty companions for the rest. Every companion is a separately bundled `./invariant` export with its own declarations and Loader-safe namespace plugin shape; the service package's own companion imports its local service type to avoid a self-dependency.
These four owners supplied the initial stateful checks. Later owners add companions for real event or mutable-data relationships, while packages without one omit the companion and document why. Every published companion is a separately bundled `./invariant` export with its own declarations and Loader-safe namespace plugin shape.
`verify-package-invariants` discovers every workspace package and rejects missing companion source, generated markers, unexplained empty installers, non-empty installers that omit or ignore the reporter, foreign or unresolved registration names, missing `./invariant` exports or published files, missing invariant peer/development dependencies and project references, and bundle overrides that omit the companion entry.
`verify-package-invariants` discovers every workspace package, accepts clean omission, and rejects partial companion wiring, generated markers, empty installers, installers that omit or ignore the reporter, foreign or unresolved registration names, missing `./invariant` exports or published files, missing invariant peer/development dependencies and project references, and bundle overrides that omit a published companion entry.
### Scoped-event semantic map
@@ -84,7 +84,7 @@ Service tests cover defaults, global disablement, allow/block selection, blockli
Composition tests cover standard-spine forwarding and generated SDK entries. Loader tests preserve each companion namespace, while built plain-Node smokes exercise the compiled subpath exports. The scoped-event freshness gate reruns its semantic Program analysis.
Every Vitest configuration loads a test host that mounts an explicitly enabled service before an ordinary Cordis root's first plugin and adds the current test package's companion. One exhaustive topology mounts all package companions once; focused service and owner tests construct their own invariant topology so they can exercise disablement, filtering, rollback, and reload without duplicate ownership. Gate tests also execute every companion's `apply` function and verify that it calls `register` with its manifest name, rather than accepting source text alone.
Every Vitest configuration loads a test host that mounts an explicitly enabled service before an ordinary Cordis root's first plugin and adds the current test package's companion when one exists. One exhaustive topology mounts all published companions once; focused service and owner tests construct their own invariant topology so they can exercise disablement, filtering, rollback, and reload without duplicate ownership. Gate tests also execute every published companion's `apply` function and verify that it calls `register` with its manifest name, rather than accepting source text alone.
## Alternatives considered
@@ -96,10 +96,10 @@ Every Vitest configuration loads a test host that mounts an explicitly enabled s
## Consequences
- Product packages own and test their relational assertions while the service stays product-independent.
-Every package pays the publication and dependency cost of a companion; only owners with a meaningful runtime relationship add listener or trace-state cost.
-Only owners with a meaningful runtime relationship pay the publication, dependency, listener, or trace-state cost of a companion; other packages record the omission reason in their README.
- Compositions that mount the diagnostics can disable all checks or select package names without changing their plugin tree.
- Explicit companion entries make diagnostic cost and ownership visible in Cordis config and package exports.
- One selected executable contribution adds one child fiber and its listener/state cost; a selected empty contribution has no listener or trace-state cost, while filtered registrations retain only name ownership.
- One selected contribution adds one child fiber and its listener/state cost, while filtered registrations retain only name ownership.
- Regex sources are deployment configuration and remain fixed until the service reloads.
- Ordinary Vitest roots install the owning test package's selected companion; one exhaustive topology pays the full child-fiber cost once for repository-wide registration coverage.
- Ordinary Vitest roots install the owning test package's selected companion when published; one exhaustive topology pays the full child-fiber cost once for repository-wide registration coverage.
- Session storage validation, snapshotting, freezing, cited source-event validation, and surface acceptance remain always on and are not affected by invariant selection.
@@ -6,7 +6,7 @@ English | [中文](2026-07-19-zstandard-jsonl-session-logs.zh.md)
## Problem
The JSONL persistence backend keeps every `SessionEvent` verbatim, including high-volume `assistant/chunk` records. Raw text makes logs inspectable but spends storage and I/O on repeated JSON keys and model text. Compression must retain the existing append/fsync commit boundary, collision-safe first materialization, crash repair, and metadata-only listing; rewriting a whole compressed file after every turn would discard those properties.
The JSONL persistence backend keeps every `SessionEvent` verbatim, including Assistant settlements with embedded model streams. Raw text makes logs inspectable but spends storage and I/O on repeated JSON keys and model text. Compression must retain the existing append/fsync commit boundary, collision-safe first materialization, crash repair, and metadata-only listing; rewriting a whole compressed file after every turn would discard those properties.
The encoding also has to remain explicit at the deployment boundary. Snapshot fixtures and external line readers require raw JSONL, while a backend cannot safely guess between compressed and raw artifacts in one root or silently migrate pre-release session data.
@@ -14,9 +14,9 @@ The encoding also has to remain explicit at the deployment boundary. Snapshot fi
### Configuration and suffix ownership
`dsh-session-persistence-jsonl` accepts `compression?: 'zstd' | 'none'` and explicitly resolves omission to `'zstd'`. Zstandard artifacts end in `.jsonl.zstd`; `'none'` retains the original newline-delimited UTF-8 `.jsonl` representation. `SessionLocation.kind` remains `'jsonl'`, because both encodings carry the same logical record format, and `SESSION_FORMAT_VERSION` remains `0` under the repository's pre-release reject-without-migration policy.
`dsh-session-persistence-jsonl` accepts `compression?: 'zstd' | 'none'` and explicitly resolves omission to `'zstd'`. Zstandard artifacts end in `.jsonl.zstd`; `'none'` retains the newline-delimited UTF-8 `.jsonl` representation. Within either configured suffix, v0 uses suffixless `session.jsonl[.zstd]` and every positive format generation uses lowercase `session.vN.jsonl[.zstd]`. `SessionLocation.kind` remains `'jsonl'`, because both encodings carry the same logical record format. Session-format migration uses the configured full suffix and one shared logical chain, so compression does not branch generation selection or publication.
Each persistence root belongs to one encoding. A one-time discovery preflight rejects any opposite suffix, and targeted load, live-adoption, listing, and materialization paths repeat the relevant suffix check after an initially empty preflight. The error names the incompatible artifact and directs the deployment to the matching configuration or a separate root. There is no migration, dual read, dual write, or extension-based fallback.
Each persistence root belongs to one encoding. A one-time discovery preflight rejects any opposite suffix, and targeted load, live-adoption, listing, and materialization paths repeat the relevant suffix check after an initially empty preflight. The error names the incompatible artifact and directs the deployment to the matching configuration or a separate root. There is no compression conversion, dual read, dual write, or extension-based fallback; logical version migration stays within the configured suffix, preserves the source generation, and exclusively publishes the final version-named successor.
### Frame and write path
@@ -32,7 +32,7 @@ A frame-boundary scanner reads the standard magic, variable header fields, block
Listing reads in bounded chunks only until the first complete frame is available, validates and decompresses that header frame, and never reads an event frame. The dedicated header frame therefore preserves metadata-only listing even for very large session logs.
EOF inside the final frame is a recoverable torn tail. After the scanner establishes that boundary, a dedicated prefix decoder uses `finishFlush: ZSTD_e_flush` so Node emits available plaintext without requiring frame or checksum completion; every complete newline-terminated event it emits is retained. Repair truncates from that frame's starting byte and appends one new checksummed frame containing the recovered complete events followed by the coordinator's synthetic tool, step, and turn closers. If the tear occurs before any complete event is decodable, repair drops the partial frame and retains all prior complete frames.
EOF inside the final frame is a torn tail. The frame belongs to an append that never resolved, so none of its records were acknowledged durable: repair truncates from that frame's starting byte, retains all prior complete frames, and appends the coordinator's synthetic tool, step, and turn closers as one new checksummed frame ([export and pre-release trims](../simplification/2026-08-27-persistence-export-and-pre-release-trims.md) owns dropping the earlier partial-plaintext salvage).
@@ -29,7 +29,7 @@ Case-insensitive filesystems can also make differently cased project keys refer
The configured root remains a deployment choice. The layout neither selects a global root nor requires projects to share one. When a deployment does centralize storage, project paths remain recognizable; a project-local root uses the same deterministic structure.
The encoded session id names an ownership directory rather than the transcript itself. `SessionPersistence.locate()`continues to return the fixed transcript path, preserving hook `transcript_path` and `DSH_SESSION_JSONL` semantics. Discovery ignores other entries inside the session directory so the backend can add session-owned artifacts without another layout change.
The encoded session id names an ownership directory rather than the transcript itself. The backend's diagnostics-only `locate`hook resolves the fixed transcript path inside it for format-refusal messages ([export and pre-release trims](../simplification/2026-08-27-persistence-export-and-pre-release-trims.md) owns removing the consumer-facing path query). Discovery ignores other entries inside the session directory so the backend can add session-owned artifacts without another layout change.
Lazy materialization remains tied to the transcript: `create()` performs no filesystem I/O, and the first append creates the project/session directories before collision-safe transcript publication. Empty directories are not listed as sessions. The backend rejects flat `<project>/<id>.jsonl*` artifacts with an explicit layout error; the pre-release format provides no automatic data migration.
A session "materialized but with no first prompt" is governed by the summary-derived bit `blank` (a derived column, not a header field; SessionHeader stays immutable):
- The host criterion: `session.events.length === 0` (zero log events = no user message yet). A live session reads `summarize()` straight from memory; a cold session is always `false` — the JSONL provider's lazy-create contract guarantees a never-appended Session never enters `persistence.list()`, so blank never touches disk.
- The host criterion: `session.seq === 0` (zero log events = no user message yet). A live session reads `summarize()` straight from memory; a cold session is always `false` — the JSONL provider's lazy-create contract guarantees a never-appended session never enters `persistence.list()`, so blank never touches disk.
- The wire carries it in two places: the required `SessionSummary.blank` column, and the required `blank` field on the `host/session-added` frame (always true at creation, letting other tabs enter the same blank-session state into their mirrors).
- The client mirror only lowers, never raises (monotonic), flipped from three sources, all reusing existing wire signals:
- The sender's own tab: the **successful response** to the first `prompt()` flips false (acceptance proves the user/message is already in the host log — this flip is confirmation, not optimism; `onEngaged` synchronously updates the list mirror, converting the current `New Session` row in place to an ordinary title, adding no list row). A rejected first prompt keeps the session blank: aligned with host authority, still shown as `New Session`, keeping its connectWorkspace reuse eligibility while it remains a Workspace member.
@@ -55,7 +55,7 @@ A trigger/menu/pick pipeline with zero knowledge of "commands":
- The hub (trigger/decoration registries + send orchestration) takes the slash/command services as optional `ctx.get()` dependencies: without ui-input-trigger or the command surfaces, input still sends and receives normally — graceful degradation.
- Each materialized Session has exactly one `SessionInputShell` (the facade), created and torn down with the session scope; with no session, no input machine is built. `ConversationRoot` is itself the `session-maybe` resident shell, holding HeroShell, the Workspace picker, the composer stack, and the chain-fallback frame. It always owns the same scrollport and composer seat; separate strict-session header and body outlets fill those fixed regions after a Session appears.
- The composer bar is one `session-maybe` slot entry rendered unconditionally: with no session the same InputBar renders inert (machine faces absent, `disabled` owner prop), and once `connectWorkspace` returns a blank session the same instance goes live — the composer surface DOM survives the no-session → blank transition and every later phase flip; `ConversationRoot`, the Hero, and the layout skeleton hold throughout.
- The composer bar is one `session-maybe` slot entry rendered unconditionally: with no session the same InputBar renders inert (machine faces absent, `disabled` owner prop), and once `connectWorkspace` returns a blank session the same instance goes live — the composer surface DOM survives the no-session → blank transition and every later phase flip; `ConversationRoot`, the Hero, and the layout skeleton hold throughout. The memoized InputBar renders its overlay, left, right, and dock child slots after the renderer has bound their standard props; `ConversationRoot` passes only scalar data and callbacks, so an unrelated shell render does not create fresh ReactNode owner props or invalidate the bar.
- ConversationRoot's Hero criterion is `sessionId === undefined || (composerPhase === 'blank' && (openState === 'open' || summaryBlank === true))`: a summary-proven blank Session remains Hero in every open state, while an unproven Session settles during loading. The first submit enters engaging synchronously, and a failure keeps the composer and the error context rather than falling back to the blank Hero; the sidebar's blank bit flips false only after a prompt is successfully accepted.
- Sending unifies in the hub defaultSink: after an optimistic draft clear it goes only through `session.prompt` with `mode:'queue'` (the Web UI has no steer entry; host-wire `mode:'steer'` remains outside this machine); backfill happens only when it fails and the live draft is still empty — a user who has kept typing is never overwritten. No Draft materialize or attach transaction exists.
- When the blank Hero re-picks the Workspace, the shell calls `connectWorkspace`; if the target session differs, the non-empty draft moves from the current shell to the target shell before the new id is opened, and the old blank session survives but is no longer current.
- PickOutcome gains a `{text}` arm; the new scoped bail event `slash/input-insert-text``{text, span}` (the same contract as the other three: draftRev CAS, returning true ⟺ an actual rewrite); facade.insertText goes through setDraft concatenation — zero machine changes.
- Sources get an optional `lexicon?(session)` hook: a synchronous hot-snapshot name roster, with `undefined` = data not warm — zero decoration, never triggering a fetch (the render path stays synchronous and side-effect-free); the paired optional `subscribeLexicon?(session, listener)` hook is the invalidation channel for rolls that change after warm (catalog settles, children spawn/exit). The controller aggregates the rolls into its `lexicon` snapshot store (re-polling on each source notification); sources registered after scope birth are warmed and folded in via the service's live-controller broadcast.
- `decorations.scanTextRefs`: a word-boundary scan of the draft (`/name`, `@name` at line start / after whitespace; `x/name` never hits) against the roster; a hit becomes a `TextRefNode` entity in the Lexical tree (the claim decoration has precedence on the leading-token seat — [the Lexical composer note](2026-08-20-web-composer-lexical-editor.md)); an edit breaking the match shape reverts the entity to plain text.
- Sending is the literal text (no more `<skill>` serialization); on the bubble side MessageItem decorates both shapes (the legacy `<skill>` tag + plain-text tokens).
- `decorations.scanTextRefs`: a word-boundary scan of the draft (`/name`, `@name` at line start / after whitespace; `x/name` never hits; a `/name` token also ends at whitespace or the draft end — the whitespace-bounded shape of the host skill gesture, so `/nfs-hg/xxx` is a path and `/plan。` is prose; the sent-text projection `projectUserText` in ui-primitives applies the same shape) against the roster; a hit becomes a `TextRefNode` entity in the Lexical tree (the claim decoration has precedence on the leading-token seat — [the Lexical composer note](2026-08-20-web-composer-lexical-editor.md)); an edit breaking the match shape reverts the entity to plain text.
- Sending is the literal text (no more `<skill>` serialization); on the bubble side `projectUserText` decorates a plain-text `/name` token only when the same step logged a `skill-invocation` injection for that name — ui-chat's `SkillNameProjector` attaches the step's injected names to the direct message Node, the way the recall projector attaches session labels — so `/123` or a stray `/word` stays plain; a command-input bubble (ui-goal) names its executed command the same way and renders the token as a `command` chip; `@name` tokens still decorate by shape.
- Decoration reactivity: the shell subscribes to the controller's lexicon store and re-scans the document on each roll change, so a roll that settles after the scope-birth prewarm lights existing draft tokens up without any menu interaction or unrelated re-render.
### Per-session provide contributions and the private keyboard surface
@@ -105,6 +105,7 @@ The state machine's entire behavior is covered by pure-JS unit tests (event sequ
| Dual draft persistence {text, occurrences} | The mirror writing the clipboard projection adds zero new concepts; chip degradation across refresh is acceptable |
| The native textarea undo stack | Unreliable under controlled + programmatic writes; the paste two-step undo semantics can only be self-managed — both sides retired with the textarea itself; Lexical's history owns undo now |
| The InputBar receiving a 16-member wiring-callback bundle | The consumption matrix proved 11 members InputBar-exclusive and 1 a dead member; the standard-kit channel lets components fetch their own, with the keyboard surface passed privately in-package |
| `ConversationRoot` rendering InputBar's child slots into owner props | Fresh React elements defeat the bar's memo boundary; the bar already receives `renderSlot` and owns the exact positions |
| Space adjudication also claiming execute-kind commands | The misfire defense: after a space the whole line is an ordinary prompt; irreversible side effects keep explicit entry points only |
| A generic tokenPattern decoration mechanism | Structured occurrence records replace pattern scanning |
| A placeholder select resident in the tool row | Named seats stay empty until registration; a placeholder clashing with the real implementation is two sources of truth |
@@ -14,7 +14,7 @@ Context occupancy needs a numerator and a denominator that no existing surface c
Both values are ordinary durable session-projection state. `@deepseek-ai/dsh-token-meter` registers two units when `ctx.sessionProjections` is present.
`tokenUsage` folds the complete durable log into uncached input, output, cache-read, and cache-write buckets. An `assistant/chunk` usage sample survives a later failed request; an `assistant/message` usage value replaces the earlier sample from the same model attempt instead of double-counting it. A matching `llm/retry-started` boundary ends that replacement scope, so a retry with the same `(turn, step)` contributes a new attempt. Reasoning stays an output subdivision. Compaction and surface replacement do not erase earlier billing.
`tokenUsage` folds the complete durable log into uncached input, output, cache-read, and cache-write buckets. It expands each `assistant/message` or `assistant/attempt` stream and takes the last usage sample; a message's top-level usage takes precedence over its embedded sample instead of double-counting it. `assistant/attempt` therefore preserves usage from failed requests. A matching `llm/retry-started` boundary opens a new attempt, so a retry with the same `(turn, step)` contributes separately. Reasoning stays an output subdivision. Compaction and surface replacement do not erase earlier billing.
Token-meter also owns the shared pure attempt/Turn fold over durable events. It applies the same retry boundary while adding the stricter completeness and exact-total checks required by an exact per-Turn disclosure. A presentation consumer may select a complete Turn window and invoke that fold, but does not own or duplicate the accounting semantics.
@@ -26,7 +26,7 @@ Capacity deliberately stays out of `EpochHeader`. That type is the reconstructio
Both units ride the standard projection lifecycle: history tail baselines, `session/projection` live frames, higher-seq-wins client storage, JSON checkpoints, cache recovery, and unit unload. There is no token-specific history field, mux frame, projector, revision counter, or client fence.
The Web `StatsLine` reads both through the standard `useProjection` seat. Window nodes still supply turn and step counts plus LLM and tool wall times — those answer "what is on screen" and are correctly window-scoped. Durable token and context groups remain when compaction leaves no visible assistant step. Cache writes count in billed input and in the cache-hit denominator. A deployment without token-meter drops the token groups; occupancy stays hidden until both pressure and capacity are known.
The Web `StatsLine` reads both through the standard `useProjection` seat. Window nodes still supply turn and step counts plus LLM and tool wall times — those answer "what is on screen" and are correctly window-scoped. Durable token and context groups remain when compaction leaves no visible assistant step. Cache writes count in billed input and in the cache-hit denominator. A deployment without token-meter drops the token groups; occupancy stays hidden until both pressure and capacity are known. The exact-overflow tooltip mounts its measuring child only for a non-empty line and retains one `ResizeObserver` while values change; text changes perform one direct measurement without replacing the observer.
## Context occupancy is approximate, and that is the decision
@@ -58,4 +58,4 @@ Token totals stay stable across pagination, compaction, replay, restart, and rec
Occupancy is approximate in the ways documented above. It is available immediately after restore or reconnect, since both fields are durable, at the cost of describing the last recorded request rather than an exact current boundary.
Each session log gains one small `request/context` record per route or advertised-capacity change. Token-meter is the canonical owner of durable usage semantics, including retry-attempt separation in the cumulative projection and the reusable exact attempt/Turn fold; Web Chat only selects a complete loaded Turn and renders the fold result. The TUI retains its live per-step map because it does not mount the generic projection seam, and the standalone browser fixture mirrors the unit. Connection and API Gateway carry no token-specific code, own no per-session metrics cache, and perform no measurement. The browser keeps two generic projection values and no connection-local telemetry, and streaming text deltas still do not force the stats line to recompute.
Each session log gains one small `request/context` record per route or advertised-capacity change. Token-meter is the canonical owner of durable usage semantics, including retry-attempt separation in the cumulative projection and the reusable exact attempt/Turn fold; Web Chat only selects a complete loaded Turn and renders the fold result. The TUI retains its live per-step map because it does not mount the generic projection seam, and the standalone browser fixture mirrors the unit. Connection and API Gateway carry no token-specific code, own no per-session metrics cache, and perform no measurement. The browser keeps two generic projection values and no connection-local telemetry; streaming text deltas do not force the stats line to recompute or churn layout-observer subscriptions.
@@ -40,7 +40,7 @@ The predicate holds for a bracket *this* session inherited, not as a liveness si
**A boundary appended at loop start.** The loop calls `resumeWith`, so it covers the resume paths, but it misses `fork()` and `adopt()` entirely, and the event would have to fire on `'startup'` — the source a fork child publishes — so `SessionStartSource` would stop discriminating. It also publishes the session before the marker is appended, so a `session/created` listener could observe a seeded log with no boundary.
**Reusing `header.seedLength`.** It is the durable *fork-lineage*boundary and deliberately keeps the original fork value across a resume, where the constructor seed is the whole stored log. The two facts differ and conflating them would lose both.
**Reusing `Session.inheritedEventCount`.** It is the durable *fork-lineage*cut and deliberately keeps the original fork value across a resume, where the constructor seed is the whole stored log. The two facts differ and conflating them would lose both.
**Crash repair closing `compaction/*` alongside turn boundaries.** Rejected: it moves every plugin's bracket semantics into core's repair pass, and core cannot know what closing another package's bracket should record.
@@ -48,8 +48,8 @@ The predicate holds for a bracket *this* session inherited, not as a liveness si
Bought: one boundary, written in one place, correct for all six seeded-start paths — including the fork gap the persistence-layer version could not reach. The persistence packages keep a pure read path. `firstLiveSeq` gains a durable twin rather than a second, competing notion of the same boundary.
Cost: a seeded session's log is one event longer, including an empty resumed log. Seq expectations move with that boundary. Two updates are load-bearing rather than mechanical: telemetry's adoption tests assert the boundary IS exported, because it is this lifecycle's own write, and the property suite's replay invariant is "seed reproduced verbatim, plus one log-only boundary" with idempotence as its own property.
Cost: a seeded session's log is one event longer, including an empty resumed log. Seq expectations move with that boundary. Two updates are load-bearing rather than mechanical: telemetry's adoption tests assert that capture begins with the current lifecycle's newly appended boundary and excludes the constructor seed, and the property suite's replay invariant is "seed reproduced verbatim, plus one log-only boundary" with idempotence as its own property.
`session/end-seed` joins the on-disk vocabulary. Under the pre-release stance (`SESSION_FORMAT_VERSION` pinned at `0`, no compatibility promise) older logs simply lack it, and a log without a boundary correctly classifies nothing as constructor-seed history.
`session/end-seed` joins the on-disk vocabulary. Current v1 requires the validated marker semantics owned by Session; the frozen v0 codec and migration edge own which historical v0 seed layouts remain admissible. The exact inherited cut stays separate from the logical header and is available after a body read.
The [queued manual compaction decision](../feature/2026-07-30-queued-manual-compaction.md) now supplies the first consumer. Its tail scan independently finds the unmatched `compaction/start` and newest end-seed, treats only a start after that boundary as live, and clears the invariant trace on the same replay transition. The predicate remains in the compaction package rather than becoming a generic core helper.
@@ -22,7 +22,7 @@ The request-level configuration seam made LLM adapter configuration restart-free
**A hand-written editor over a schema model layer.** `ctx.settingsSchema`, provided by `dsh-client-ui-settings`, rehydrates the wire's `toJSON()` envelope into live schemastery nodes for validation, path resolution, and immutable draft editing — but no generic rendering: the first cut shipped a full schema-driven form renderer, and the resulting page was an unstyled schema dump (every advanced field flattened onto the card, raw field names as labels, the `retryPolicy` unsupported-fallback in the main flow). The hand-written direction won over adding a hint/grouping system, and a further simplification removed the reference input entirely: the card's primary field is one **API key** input, a whole-section provider without a configured key opens as its setup card, and the collapsed 自定义设置 fold carries the curated per-family extras (`baseURL` for both families, `reasoningEffort` for deepseek / `reasoning` for pi-ai, plus direct DeepSeek model rows with `id`, `name`, and `contextWindow`). Existing model fields outside that visible set survive array edits; retry policy, timeouts, and other fields remain owned by `settings.yaml`. Validation still runs the rehydrated schema before writing, while adapter-specific checks reject catalog invariants that the serialized schema cannot express. The card's colors resolve through the `--dsw-alias-*` design tokens; it had named `--border`/`--surface`/`--text-*`, which nothing in this app defines, so it rendered their light-mode fallbacks and stayed light under the dark theme. The model catalog takes the row shape the pi-ai provider form introduces: one bordered entry per model, id and display name on the row, and the capacities behind the row's own disclosure, so the two editors read as one design rather than diverging. Every field keeps the indexed `aria-label` that names it. Both capacities are text fields reading a decimal `K`/`M` suffix (`1M` is 1000K, matching how capacities are quoted) and storing the plain count: a field holds the typed text while it has focus, because re-deriving it from the parsed count on every keystroke would rewrite `1000` to `1K` mid-word, and text that does not parse stays on screen so the save-time rejection names a row the user can still see. The shared class names carry only declared token spellings: `--dsw-alias-border-subtle`, `--dsw-alias-text-tertiary`, and `--dsw-alias-text-primary` are undeclared, so naming them resolves to the light-mode literals in their fallback slots. A styles test now rejects any `--dsw-*` name the token sheet does not declare, so the next editor to name one fails rather than shipping a light-only surface.
**The Models page is a three-domain join with service-shaped apply semantics.** Rows are configured providers; the add card's select is the dormant directory remainder. Route liveness still gates readiness and invalidates the join, but the page does not render it as provider status because configuration presence and runtime availability are distinct. The key path stays reference-shaped without ever showing a reference: a typed key stores **write-only** through `credentials.set` under the profile's `apiKeyEnv`, deriving `<ROUTE>_API_KEY` when none exists (the pi-ai profile records the derivation only when a key is entered), so `settings.yaml` never carries a key value; a blank pi-ai key materializes a reference-free profile and preserves provider-native authentication. Profile edits and removals land as minimal path-addressed `settings.mutate` operations against the redacted user section, which never names a secret the page did not receive. Removing a user-layer provider first opens a localized confirmation dialog whose row actions, title, description, and final action identify the same provider; confirmation removes an exact configured+writable derived credential before the profile, while custom, environment, and unidentified targets remain untouched. Both stages are idempotent and a partial failure stays in the dialog for retry. DeepSeek's model list is array-replace configuration: inherited effective rows remain visible until the first edit materializes the complete list in the user layer, and reset unsets the list override. `llm.discoverModels` results stay in picker-local state until **Add selected**; configured ids start unchecked, while**Select all** / **Deselect all** changes only that local set, so bulk selection preserves the same capacity-protection rule. The partial-commit and credential-ownership rationale lives in the [provider credential lifecycle note](../bug-fix/2026-08-06-provider-credential-lifecycle.md).
**The Models page is a three-domain join with service-shaped apply semantics.** Rows are configured providers; the add card's select is the dormant directory remainder. Route liveness still gates readiness and invalidates the join, but the page does not render it as provider status because configuration presence and runtime availability are distinct. The key path stays reference-shaped without ever showing a reference: a typed key stores **write-only** through `credentials.set` under the profile's `apiKeyEnv`, deriving `<ROUTE>_API_KEY` when none exists (the pi-ai profile records the derivation only when a key is entered), so `settings.yaml` never carries a key value; a blank pi-ai key materializes a reference-free profile and preserves provider-native authentication. Profile edits and removals land as minimal path-addressed `settings.mutate` operations against the redacted user section, which never names a secret the page did not receive. Removing a user-layer provider first opens a localized confirmation dialog whose row actions, title, description, and final action identify the same provider; confirmation removes an exact configured+writable derived credential before the profile, while custom, environment, and unidentified targets remain untouched. Both stages are idempotent and a partial failure stays in the dialog for retry. DeepSeek's model list is array-replace configuration: inherited effective rows remain visible until the first edit materializes the complete list in the user layer, and reset unsets the list override. `llm.discoverModels` results stay in picker-local state until **Add selected**; configured ids start unchecked, and a localized search filters ids and optional display names without changing hidden selections.**Select all** / **Deselect all** changes only the visible candidates in that local set, so filtered bulk selection preserves the same capacity-protection rule. The partial-commit and credential-ownership rationale lives in the [provider credential lifecycle note](../bug-fix/2026-08-06-provider-credential-lifecycle.md).
## Alternatives considered
@@ -36,4 +36,4 @@ The request-level configuration seam made LLM adapter configuration restart-free
## Consequences
The whole loop is pinned keyless in the browser lane (`apps/web/tests/models-settings.e2e.ts`): the add card offers the dormant pi-ai catalog, adding `minimax-cn` with a typed key writes the reference-only profile into `settings.yaml`, stores the value into the harness home's `.env` under the derived `MINIMAX_CN_API_KEY`, registers the route live on the topology frame, and the customized fold merges `reasoning` beside the reference — zero model calls, ARIA goldens for the add-card, configured, model-picker, and identified delete-confirmation states, plus a scaffold `harnessHome` so tests never touch a real `~/.dsh` (the provider under test is one whose derived reference cannot collide with a developer's exported keys). The component suite pins configured-id exclusions and both directions of the bulk toggle. The settings-shell scenario intercepts the pathless native intent; Service Definition, provider, wire, React, and native-opener tests separately pin provider absence, custom-path resolution, absent-file materialization, owner-only permissions, hidden remote/unavailable states, duplicate-click collapse, localized failure, macOS text-editor dispatch, and Linux/Windows desktop dispatch. The removal scenario proves cancellation leaves both profile and key intact, then confirmation removes both the profile and its identified managed credential. The DeepSeek onboarding fixture edits the default catalog into a user-owned list, persists an arbitrary model id/name/context window, removes the active row, and observes the model selector's empty-selection fallback. The rename touched 239 files (fixtures, goldens, docs, python) with no compatibility alias. The renderer replacement needed no wire change: apply semantics, redaction, and the directory join were renderer-agnostic all along. Deferred: a per-row models preview (the picker already lists models) and a page address for live routes that never declared configurability.
The whole loop is pinned keyless in the browser lane (`apps/web/tests/models-settings.e2e.ts`): the add card offers the dormant pi-ai catalog, adding `minimax-cn` with a typed key writes the reference-only profile into `settings.yaml`, stores the value into the harness home's `.env` under the derived `MINIMAX_CN_API_KEY`, registers the route live on the topology frame, and the customized fold merges `reasoning` beside the reference — zero model calls, ARIA goldens for the add-card, configured, model-picker, and identified delete-confirmation states, plus a scaffold `harnessHome` so tests never touch a real `~/.dsh` (the provider under test is one whose derived reference cannot collide with a developer's exported keys). The model-picker path filters the catalog, clears only the visible selection, restores hidden picks when the query clears, and records the localized search control in its ARIA golden; the component suite also pins matching by id and optional name, the no-results state, and visible-only bulk selection. The settings-shell scenario intercepts the pathless native intent; Service Definition, provider, wire, React, and native-opener tests separately pin provider absence, custom-path resolution, absent-file materialization, owner-only permissions, hidden remote/unavailable states, duplicate-click collapse, localized failure, macOS text-editor dispatch, and Linux/Windows desktop dispatch. The removal scenario proves cancellation leaves both profile and key intact, then confirmation removes both the profile and its identified managed credential. The DeepSeek onboarding fixture edits the default catalog into a user-owned list, persists an arbitrary model id/name/context window, removes the active row, and observes the model selector's empty-selection fallback. The rename touched 239 files (fixtures, goldens, docs, python) with no compatibility alias. The renderer replacement needed no wire change: apply semantics, redaction, and the directory join were renderer-agnostic all along. Deferred: a per-row models preview (the picker already lists models) and a page address for live routes that never declared configurability.
@@ -25,7 +25,7 @@ The constants live in the Service Definition even though the worker is the only
## Scope
This decision delivers only the Service Definition extension and the worker's adoption of it. The `py-types` renderer and PTC mode language dispatch are owned by the [language-dispatch note](../feature/2026-07-31-ptc-language-dispatch.md); a Python backend does not exist yet. The Service Definition README keeps its worker-only wording for that reason: linking to a `dsh-code-runtime-python` README that does not exist would break the dead-link gate.
This decision delivers the Service Definition extension and the worker-thread backend's adoption of it. The `py-types` renderer and PTC mode language dispatch are owned by the [language-dispatch note](../feature/2026-07-31-ptc-language-dispatch.md). The private experimental CPython subprocess backend (`dsh-experimental-code-runtime-python`) adopts the same portable-identifier contract.
`RESERVED_BINDING_GLOBALS` encodes the Python bootstrap's concrete design ahead of the backend itself: it seeds exactly `__builtins__`/`__name__` and wraps the program under `__dsh_main__`. A Python backend that seeds any additional module global (`__doc__`, `__loader__`, `__spec__`, `__file__`, `__package__`, …) MUST widen this set in the same change, exactly as adding a language widens `PORTABLE_RESERVED_WORDS` — a name the bootstrap seeds but the set omits is the portability split this contract exists to prevent.
The CPython code runtime now lives at `packages/experimental/code-runtime-python` (private, npm name `@deepseek-ai/dsh-experimental-code-runtime-python`); promotion to a released package follows the experimental-packages decision.
English | [中文](2026-07-31-code-runtime-python-fd3-protocol.zh.md)
## Problem
`@deepseek-ai/dsh-code-runtime-python` owns the wire protocol intended for a CPython code-runtime provider. Such a provider runs each model program in a fresh `python3 -I` subprocess and bridges binding calls and completion values over the child's fd 3. The host cannot trust that channel: model code has full access to fd 3 and can forge any frame, so every inbound frame is hostile input that the host must validate and rebuild before reading. The protocol also has to carry lossless JSON without the depth limit `JSON.stringify` and `json.dumps` impose, because the seam's `CodeJsonValue` is depth-unbounded.
`@deepseek-ai/dsh-experimental-code-runtime-python` owns the wire protocol intended for a CPython code-runtime provider. Such a provider runs each model program in a fresh `python3 -I` subprocess and bridges binding calls and completion values over the child's fd 3. The host cannot trust that channel: model code has full access to fd 3 and can forge any frame, so every inbound frame is hostile input that the host must validate and rebuild before reading. The protocol also has to carry lossless JSON without the depth limit `JSON.stringify` and `json.dumps` impose, because the seam's `CodeJsonValue` is depth-unbounded.
The package ships the protocol independently from a runtime implementation. It exports no`PythonCodeRuntime`, subprocess path, or Python-side JSON codec; those remain work for a future provider. The protocol builds on the [portable identifier seam](2026-07-31-code-runtime-portable-identifier-seam.md).
The private experimental package contains both the protocol and runtime implementation:`PythonCodeRuntime` (the plugin's default export), the `python3 -I` subprocess path, and the Python-side JSON codec all live in `@deepseek-ai/dsh-experimental-code-runtime-python`. The protocol builds on the [portable identifier seam](2026-07-31-code-runtime-portable-identifier-seam.md).
## Decision
@@ -20,24 +22,24 @@ The package ships the protocol independently from a runtime implementation. It e
`py/protocol.py` mirrors the message shapes as `TypedDict`s and re-declares the two surfaces both sides EXECUTE against — `PROTOCOL_FD = 3` and `log_truncation_marker` — with byte-identical text.
The package remains independently buildable with protocol-only exports. `check-workspace-constraints` reads every `packages/<group>/<pkg>/package.json` unconditionally, while the coverage and invariant-topology checks exercise the package as soon as its directory exists.
The package ships the runtime alongside the protocol; it remains independently buildable. `check-workspace-constraints` reads every `packages/<group>/<pkg>/package.json` unconditionally, while the coverage and invariant-topology checks exercise the package as soon as its directory exists.
## Wire contract
Frames are JSON-lines on fd 3, one object per line, leaving stdout/stderr free for the program's own output. Child → host: `boot-ack`, `call`, `log`, `done`. Host → child: `boot` (first frame), `run` (after `boot-ack`), and one `reply` per `call`. The `log` frame's `truncated` flag marks the frame that IS the child ledger's own truncation marker, so the host stops capturing at the same point the child did instead of inferring it from its own budget. `done.error.kind` is one of `exception`, `invalid-output`, `output-limit`; wall/CPU budgets, aborts, and substrate death are observed host-side, not carried as frames.
Frames are JSON-lines on fd 3, one object per line, leaving stdout/stderr free for the program's own output. Child → host: `boot-ack`, `call`, `log`, `done`. Host → child: `boot` (first frame), `run` (after `boot-ack`), and one `reply` per `call`. The `log` frame's `truncated` flag marks the frame that IS the child ledger's own truncation marker, so the host stops capturing at the same point the child did instead of inferring it from its own budget. The `log` frame's `open` flag marks an unterminated line committed by an explicit flush: the host holds it and appends the next frame to the same entry, so an explicit flush followed by more text reads back as one line rather than a fake newline. The one exception is truncation: when a later over-budget frame trips the ledger, the already-billed prefix is committed as its own entry and the truncation marker follows it (marker last, no re-charge). The merged entry's wire cost is billed exactly once, split incrementally across its fragments on both sides (O(k) for k fragments, never a re-walk of the whole hold): the FIRST fragment pays the full JSON-string cost plus the separator, each continuation and the closing frame pay only their content; the host's exact-cost caps are `logBudget - 1` for a first fragment (the ledger's reserved byte, matching `admit`) and `logBudget + 2` for a continuation or closing frame (billed without the two quotes), and `jsonStringCostUpTo` returns `undefined` below a 2-byte cap; the child keys its split billing off `_open_started` alone, so a closing frame bills as the merged tail.`done.error.kind` is one of `exception`, `invalid-output`, `output-limit`; wall/CPU budgets, aborts, and substrate death are observed host-side, not carried as frames.
## Mirror alignment
`py/protocol.py` and `src/protocol.ts` agree that `LogMessage` carries `truncated`, `DoneMessage.error` carries `kind`, and `Namespace` may carry `errorClass`. `tests/protocol-mirror.e2e.ts` spawns a real `python3` and asserts `PROTOCOL_FD`, `log_truncation_marker`, and each `TypedDict`'s required and optional wire field sets against `src/protocol.ts`. A renamed or dropped field, or a required/optional mismatch, fails the test. Field *types* are not compared across the language boundary; review and a future provider's real-subprocess suite own that gap.
`py/protocol.py` and `src/protocol.ts` agree that `LogMessage` carries `truncated`, `DoneMessage.error` carries `kind`, and `Namespace` may carry `errorClass`. `tests/protocol-mirror.e2e.ts` spawns a real `python3` and asserts `PROTOCOL_FD`, `log_truncation_marker`, and each `TypedDict`'s required and optional wire field sets against `src/protocol.ts`. A renamed or dropped field, or a required/optional mismatch, fails the test. Field *types* are not compared across the language boundary; review and the runtime's real-subprocess suite (`runtime.spec.ts`) own that gap.
## Alternatives considered
**Require a future Python JSON codec (`_encode_json_plain` / `_decode_json_plain`) to live in `py/protocol.py` for cross-side symmetry with `protocol.ts`.** Rejected. The repository's "prefer symmetry for parallel values" rule points at genuinely parallel values; these are not. The host-side codec in `protocol.ts` validates hostile input and is self-contained. A child-side codec would produce trusted output and belong with bootstrap-owned emission and cost accounting; forcing only its entry points into `protocol.py` would couple the vocabulary mirror to runtime internals or create an import cycle. `protocol.py` remains a pure wire-vocabulary mirror. No Python codec ships in this package.
**Require a future Python JSON codec (`_encode_json_plain` / `_decode_json_plain`) to live in `py/protocol.py` for cross-side symmetry with `protocol.ts`.** Rejected. The repository's "prefer symmetry for parallel values" rule points at genuinely parallel values; these are not. The host-side codec in `protocol.ts` validates hostile input and is self-contained. A child-side codec would produce trusted output and belong with bootstrap-owned emission and cost accounting; forcing only its entry points into `protocol.py` would couple the vocabulary mirror to runtime internals or create an import cycle. `protocol.py` remains a pure wire-vocabulary mirror; the codec (`_encode_json_plain` / `_decode_json_plain`) lives in `bootstrap.py` with the runtime it serves.
**Keep the protocol files outside a buildable package until a runtime ships.** Rejected: the workspace-constraint, coverage, and invariant-topology checks require every directory under `packages/<group>/<pkg>` to be a buildable package, and the protocol has independent tests and a public wire vocabulary.
## Consequences
Bought: the fd-3 protocol and its hostile-input codec form a self-contained, fully unit-covered layer, with an executing guard against TypeScript/Python field-set drift. A future runtime can consume a reviewed wire contract.
Bought: the fd-3 protocol and its hostile-input codec form a self-contained, fully unit-covered layer, with an executing guard against TypeScript/Python field-set drift. The runtime built on it (`bootstrap.py`) consumes the reviewed wire contract.
Cost: the package name denotes a Python runtime family while`src/index.ts` exports only the protocol vocabulary. The mirror e2e compares field names and required/optional status across the two sides but not field types; comparing type declarations across TypeScript and Python has no mechanical equivalent, so review and the future runtime's real-subprocess suite retain that responsibility.
Cost: the package name denotes a Python runtime family and`src/index.ts` exports the full `PythonCodeRuntime` implementation, so the protocol vocabulary is only one part of the package surface. The mirror e2e compares field names and required/optional status across the two sides but not field types; comparing type declarations across TypeScript and Python has no mechanical equivalent, so review and the runtime's real-subprocess suite retain that responsibility.
@@ -42,7 +42,7 @@ The launching environment wins because `DEEPSEEK_API_KEY=… dsh`, a CI secret,
**The project the harness is launched in is trusted, by default and without a prompt.** A checkout may carry its own endpoint, its own ordinary variables, and its own key; the key ranks below the managed store, so a key stored through the Models page is never displaced by one a checkout happens to contain. `LaunchEnvironmentSnapshot.getFrom(name, sources)` still searches only the layers a caller names, and omitting one is a refusal rather than a demotion — the mechanism exists for decisions where a layer must be unreachable; this decision includes the project layer.
**Trust does not extend to changing the harness itself.** `loadLayeredEnv` rejects, at load and before anything is materialized, any `.env` that sets a variable governing how a process launches (`PATH`, `SHELL`, `NODE_OPTIONS`, `LD_PRELOAD`), which ambient program handles an operation (`EDITOR`, `PAGER`, `BROWSER`), what code a runtime executes before the program it was asked to run (`BASH_ENV`, `PERL5OPT`, `PYTHONSTARTUP`, `RUBYOPT`, `JAVA_TOOL_OPTIONS`, the Git hook commands), where model-visible instructions load from (the whole `DSH_*` namespace, `HOME`, `XDG_*`), or how the network is reached and trusted (proxy and CA variables). Matching is case-insensitive, so `https_proxy` is not a bypass.
**Trust does not extend to changing the harness itself.** `loadLayeredEnv` rejects, at load and before anything is materialized, any `.env` that sets a variable governing how a process launches (`PATH`, `SHELL`, `NODE_OPTIONS`, `LD_PRELOAD`), which ambient program handles an operation (`EDITOR`, `PAGER`, `BROWSER`), what code a runtime executes before the program it was asked to run (`BASH_ENV`, `PERL5OPT`, `PYTHONSTARTUP`, `RUBYOPT`, `JAVA_TOOL_OPTIONS`, the Git hook commands), where model-visible instructions load from (the whole `DSH_*` namespace, `HOME`, `XDG_*`), or how the network is reached and trusted (proxy and CA variables). Matching is case-insensitive, so `https_proxy` is not a bypass. One exemption, recorded in [the proxy policy note](2026-08-27-outbound-proxy-policy.md): the four proxy names are accepted from `$DSH_HOME/.env`, which no `.env` can relocate, and still refused from the invoking directory's file.
The line is that these take effect with no user action, before any turn, outside the permission policy and the sandbox. `DSH_PERMISSION_MODE` would switch off the approvals that make trusting a project meaningful at all, and `BASH_ENV` runs a file of the project's choosing on every single `bash -c` the bash tool issues — the project's code running under the agent's policy is the deal; the project rewriting that policy is not. Enumerating these is a losing game one variable at a time, which is why the whole `DSH_*` namespace is denied rather than an audited subset, and why the list is organised by what a variable *does* rather than by which runtime owns it. There is no opt-out: an escape hatch would have to be readable from somewhere, and anything a discovered file could set is the hole itself.
@@ -53,7 +53,7 @@ The line is that these take effect with no user action, before any turn, outside
## Consequences
- The web credential form now takes effect against an older key in the user's `.env`; only a key exported in the launching shell still makes it read-only, and the diagnostic says so.
- A `.env` holding `DSH_*`, `PATH`, `BROWSER`, or a proxy variable fails the launch instead of being applied. Developers keeping switches in a repository `.env` move them to their shell — a deliberate, loud break.
- A `.env` holding `DSH_*`, `PATH`, `BROWSER`, or — in the invoking directory — a proxy variable fails the launch instead of being applied. Developers keeping switches in a repository `.env` move them to their shell — a deliberate, loud break.
- Composition is no longer overridable by a stale shell endpoint. It is still overridable by a user's stored `settings.yaml`, which is the settings seam's layering and not something this note changes; the product CLI offers no flag above it, so a deployment that must win against stored settings owns its own bin or loader tree.
- Not solved: the layers are still materialized into `process.env`, so ordinary project variables continue to reach child processes under the subprocess scrub. Bootstrap variables cannot come from a file at all; the environment package records the remaining subprocess reach as a limitation.
- Exa and Perplexity still capture their key at load time rather than through the credential seam. They no longer read raw `process.env` — they resolve through the trusted layers — but converting them to per-request credential resolution is separate work.
@@ -6,7 +6,7 @@ English | [中文](2026-08-04-draft-provider-endpoint-interrogation.zh.md)
## Problem
Once a pi-ai route became [a declaration rather than a catalog lookup](2026-08-03-pi-ai-declared-provider-catalog.md), a person adding an OpenAI-compatible gateway had to know its model ids before they could configure it. The adapter no longer constrains them to an installed catalog, which is the point, but it also means nothing tells the user what the endpoint actually serves — and most of these endpoints do publish that list at `GET /models`.
Once a pi-ai route became [a declaration rather than a catalog lookup](2026-08-03-pi-ai-declared-provider-catalog.md), a person adding acompatible gateway had to know its model ids before they could configure it. The adapter no longer constrains them to an installed catalog, which is the point, but it also means nothing tells the user what the endpoint actually serves — and OpenAI- and Anthropic-compatible endpoints publish that list through protocol-specific model-listing routes.
The obvious answer, a dynamic runtime catalog refreshed in the background, was rejected with the layer below it: it makes a route's model list external mutable state needing a cache, an invalidation story, and an offline path, while the product need is narrower. What is needed is a *question asked once*, whose answer the user adopts into `settings.yaml` — so `settings.yaml` remains the only thing deciding what a route serves.
@@ -17,11 +17,11 @@ The awkward part is that the question is about something that does not exist yet
Interrogation is keyed by **settings namespace**, not by provider route:
- `ctx.llm.registerModelDiscovery(settingsNs, discover)` lets an adapter plugin offer to interrogate endpoints for the namespace it owns, and `ctx.llm.discoverModels(settingsNs, request)` asks. There is no way to enumerate which namespaces registered: a surface that cannot interrogate learns it from the refusal, and a list nothing consumed would be a required wire field doing nothing. The namespace is the right key because a configuration surface already holds it from the configurable-provider directory, and because a provider being added has no route to name.
- `LlmModelDiscoveryRequest` carries the draft — an optional `provider`, an optional `baseURL`, an optional `api`, an optional `apiKey`, and a signal — and needs at least one of `provider` or `baseURL` to have anything to answer about. `provider` exists because a route the adapter already describes is answered from its own registry with no network call at all; only a route it does not describe reaches an endpoint. Nothing in this path writes settings or credentials. The one read is the credential of a route the request names: a configuration surface holds a redacted descriptor rather than the stored secret, so the draft's `apiKey` is present only while the user is typing one, and without that read an already-configured route would be interrogated unauthenticated and answer 401. The typed key wins, being the one under test.
- `LlmModelDiscoveryRequest` carries the draft — an optional `provider`, an optional `baseURL`, an optional `api`, an optional `apiKey`, and a signal — and needs at least one of `provider` or `baseURL` to have anything to answer about. `provider` exists because a route the adapter already describes is answered from its own registry with no network call at all; only a route it does not describe reaches an endpoint. Nothing in this path writes settings or credentials. A named configured route reads its stored credential and deployment-owned profile `headers` inside the Host: the credential is write-only and the curated Models page does not edit headers, so neither can be reconstructed from that page's draft. The typed key wins over the stored credential, while the profile headers still accompany the request.
- `LlmDiscoveredModel` makes every field but `id` optional, because most listings disclose an id and nothing else. The reply is candidates, not a catalog: a surface adopting one still owes the capacities the adapter requires.
- `llm.discoverModels` carries the same draft over the wire. Its `apiKey` is the third and last payload on which a secret may ride, alongside `settings.update`/`mutate` and `credentials.set`, and it is never stored or echoed back. It does ride the client's outgoing envelope like every other secret-bearing payload, where a `subscribeEnvelopes()` observer can see it; redacting that tap is a configuration-plane-wide change, not this method's to make alone. Connection authenticates the method with the complete Host API: it makes the host issue a GET to a caller-chosen URL and reports the outcome, which an anonymous caller must not receive. Every refusal folds into `model-discovery-failed`, whose message is the adapter's own text and whose details name the endpoint asked but never the credential offered.
`dsh-llm-pi-ai`implements the wire path as a plain `GET {baseURL}/models`, reading `openai-completions` and `openai-responses`: their `GET /models` shape with bearer auth is the one a gateway, a self-hosted server, and the official endpoints all agree on. Azure is excluded despite its OpenAI lineage — it authenticates with an `api-key` header and requires an `api-version` query — and Codex uses OAuth; both would have reported an authentication failure as a provider with no models. Every other protocol answers `DISCOVERY_UNSUPPORTED`, so the surface falls back to hand-entry rather than reporting a guessed response shape as an empty provider. `baseURL` is treated as a prefix rather than a URL to resolve against, so a deployment path such as `https://gateway.example/openai/v1` keeps its segments. The reply is read under a four-megabyte ceiling enforced on the bytes actually received — the endpoint is a URL the user typed, so a declared `content-length` is checked first as a courtesy but never trusted as the bound, matching `dsh-web-fetch`'s two-stage shape for its own caller-supplied URLs.
`dsh-llm-pi-ai`applies the protocol-specific listing routes, authentication, URL normalization, response formats, and metadata rules recorded by [protocol-specific model listing discovery](2026-09-02-protocol-specific-model-listing-discovery.md). Profile resolution rejects names and values Fetch cannot represent, so a malformed deployment header is reported as a configuration error before interrogation. Configured profile headers are installed first; fixed protocol headers, a typed-or-stored protocol credential, and Harness attribution then win their case-insensitive collisions. A protocol without a documented listing contract answers `DISCOVERY_UNSUPPORTED`, so the surface falls back to hand-entry rather than reporting guessed response fields as an empty provider. `baseURL` is treated as a prefix rather than a URL to resolve against, so deployment path segments remain intact. The reply is read under a four-megabyte ceiling enforced on the bytes actually received — the endpoint is a URL the user typed, so a declared `content-length` is checked first as a courtesy but never trusted as the bound, matching `dsh-web-fetch`'s two-stage pattern for its own caller-supplied URLs.
### Why not pi-ai's own refresh machinery
@@ -33,18 +33,18 @@ pi-ai supplies `createProvider({ fetchModels })` plus `Models.refresh()` and a `
**Put the capability on `LlmAdapter`.** Adapters are reached through a route registration, so this has the same problem, plus it would make an adapter instance answer questions about endpoints it does not serve.
**Have the host read the stored profile instead of accepting a draft.** No secret would cross the wire for an already-configured provider. But adding a provider would then require saving an unusable configuration first, and a form whose endpoint was edited but not yet saved would silently interrogate the old one. Accepting the draft keeps what the user sees and what is asked identical — with the credential as the one exception, because it is the one field a surface is never shown and so can never put in the draft.
**Have the host read the entire stored profile instead of accepting a draft.** No secret would cross the wire for an already-configured provider. But adding a provider would then require saving an unusable configuration first, and a form whose endpoint was edited but not yet saved would silently interrogate the old one. The draft remains authoritative for the endpoint and protocol. The narrow Host-side exceptions are the stored credential, which is write-only, and profile headers, which remain deployment configuration rather than Models-page fields.
**Interrogate every pi-ai protocol.** Anthropic's listing happens to share OpenAI's envelope, and Google's does not. Supporting the ones that are easy would make coverage arbitrary and, worse, make a wrong guess at a response shape indistinguishable from a provider with no models. A protocol that says it cannot be interrogated sends the user to hand-entry, which is the documented fallback.
**Interrogate every pi-ai protocol.** Coverage based on convenient response similarities would be arbitrary and would make a wrong guess indistinguishable from a provider with no models. Anthropic is included only through its documented native listing contract, as the [protocol-specific extension](2026-09-02-protocol-specific-model-listing-discovery.md) records; Google's field set and Azure's request contract differ, while Codex uses OAuth. An unsupported protocol sends the user to hand-entry, which remains the documented fallback.
**Buffer the reply with `response.text()` and check its length.** Simpler, but the bound would arrive after the bytes did, and the endpoint is whatever URL the user typed.
## Consequences
A person adding a gateway can ask it what it serves instead of hunting through its documentation, and the answer arrives as candidates they choose from rather than as configuration written behind their back. The seam gained a registry that is deliberately small: one offer per namespace, no storage, no lifecycle beyond the fiber.
A person adding a gateway can ask it what it serves instead of hunting through its documentation, and the answer arrives as candidates they choose from rather than as configuration written behind their back. When an endpoint discloses richer metadata, adopting a candidate fills its id, name, context window, and output-token cap into the editable Web row. Search preserves hidden selections, selecting all adds the visible results, and deselecting all clears every result so a filtered picker cannot submit hidden models accidentally. An already-configured enterprise gateway uses the same deployment headers and Harness `User-Agent` for interrogation and model requests without adding a header injection field to the browser protocol. The seam gained a registry that is deliberately small: one offer per namespace, no storage, no lifecycle beyond the fiber.
What it costs: the wire gained a third secret-carrying payload, so the configuration plane's write-only surface is now three methods rather than two. Discovery coverage is protocol-shaped rather than provider-shaped — an Anthropic-compatible gateway must be filled in by hand even though its listing would parse. And because nothing re-runs the question, a model list is still only as current as its last edit; that is the same trade the layer below made deliberately.
What it costs: the wire gained a third secret-carrying payload, so the configuration plane's write-only surface is now three methods rather than two. Discovery coverage remains protocol-shaped rather than provider-shaped, and an endpoint using an unsupported request contract must be filled in by hand. Because nothing re-runs the question, a model list is still only as current as its last edit; that is the same trade the layer below made deliberately.
## Testing
`packages/llm/llm/tests/topology.spec.ts` covers the registry: one offer per namespace, disposal with the fiber, normalization that drops duplicate and unusable ids without inventing capacities, the `NO_DISCOVERY`/`INVALID_DISCOVERY` refusals, and the `model-discovery-failed` Remote mapping. `packages/llm/llm-pi-ai/tests/discovery.spec.ts` drives the probe against local HTTP servers — a listing with and without disclosed capacities, a preserved deployment path, an absent credential, a configured route supplying its own where the draft has none and a typed key winning over it, a catalog route answering without resolving one at all, dropped rows, 401/403 versus a server fault, a non-listing and a non-JSON body, an unreachable endpoint, caller cancellation, an unsupported protocol, and the size ceiling in both its declared-length and streamed forms. `packages/client/connection/tests/node-half.host.spec.ts` pins the `llm/discoverModels``/api` carrier registration, while `packages/client/ui-settings-models/tests/provider-form.client.spec.tsx` verifies that the draft reaches the Remote whole, absent fields stay absent, and no settings namespace or credential is written before selection.
`packages/llm/llm/tests/topology.spec.ts` covers the registry: one offer per namespace, disposal with the fiber, normalization that drops duplicate and unusable ids without inventing capacities, the `NO_DISCOVERY`/`INVALID_DISCOVERY` refusals, and the `model-discovery-failed` Remote mapping. `packages/llm/llm-pi-ai/tests/discovery.spec.ts` drives the probe against local HTTP servers — standard arrays and enriched objects with every accepted metadata spelling, Anthropic's native path, headers, and capacity fields, route keys that differ from nested canonical ids, name fallback, a preserved deployment path, an absent credential, a configured route supplying its stored credential and headers while a typed key wins without resolving the stored one, a catalog route answering without resolving one at all, dropped rows, 401/403 versus a server fault, a non-listing and a non-JSON body, an unreachable endpoint, caller cancellation, an unsupported protocol, and the size ceiling in both its declared-length and streamed forms. `packages/llm/llm-pi-ai/tests/loader-composition.spec.ts` boots settings and credentials through the Loader and proves settings-only headers reach `GET /models` with request-owned headers winning collisions. `packages/llm/llm-pi-ai/tests/adapter.spec.ts` rejects profile headers Fetch cannot represent, and `packages/llm/llm-pi-ai/tests/dynamic-config.spec.ts` proves a settings write reports that configuration error while its last good routes keep serving. `packages/client/connection/tests/node-half.host.spec.ts` pins the `llm/discoverModels``/api` carrier registration, while the component and built-Web settings tests verify that the complete draft reaches the Remote, absent fields stay absent, selected metadata fills all four editable model fields, tuned rows win over rediscovery, filtered deselection clears hidden candidates, and no settings namespace or credential is written before selection.
@@ -30,7 +30,7 @@ The scanner stops retaining events at the first unparsable row or sequence gap b
### Restore admission
Persistence transfers freshly materialized JSON values to `Session.fromRestore`. These values are detached, acyclic trees, and packed chunk rows expand into newly allocated events, so the restore-only path validates the fixed event envelope with one `for...in` and `switch`, dispatches current-shape checks by event discriminant, and iteratively freezes the owned graph with an explicit `pending` array and no cycle-tracking set. Surface validation records one transition plan and commits that plan when the exact candidate enters the log instead of planning the same event twice.
Persistence transfers freshly materialized current JSON values to `Session.fromRestore`. These values are detached, acyclic trees; historical packed rows and adjacent migrations have already produced newly allocated v2 settlements. The restore-only path validates the fixed event envelope with one `for...in` and `switch`, dispatches current-shape checks by event discriminant, and iteratively freezes the owned graph with an explicit `pending` array and no cycle-tracking set. Surface validation records one transition plan and commits that plan when the exact candidate enters the log instead of planning the same event twice.
Borrowed seeds used by ordinary creation and fork paths still take a JSON snapshot and use the generic cycle-safe deep freeze. The specialization therefore changes only durable restoration; it does not weaken acceptance for caller-owned values.
@@ -41,7 +41,7 @@ Borrowed seeds used by ordinary creation and fork paths still take a JSON snapsh
- **Concatenate all plaintext before scanning** — rejected because it retains the compressed input, complete plaintext, whole-log UTF-8 string, line metadata, and parsed rows at the same time, and it rescans a torn-frame prefix.
- **Implement a streaming JSON parser** — rejected because JSONL already provides record boundaries; native newline search plus `JSON.parse` removes the large intermediates without owning another parser or changing JSON semantics.
- **Use a shared `WeakSet` while freezing restored events** — rejected because JSON materialization cannot produce cycles, and the set adds a lookup per object while retaining the complete graph during traversal.
- **Skip validation or freezing for restored values** — rejected because durable storage is a runtime boundary and `Session.events` promises immutable accepted history. The optimized path specializes those operations around stronger ownership facts instead of removing them.
- **Skip validation or freezing for restored values** — rejected because durable storage is a runtime boundary and Session read methods promise immutable accepted history. The optimized path specializes those operations around stronger ownership facts instead of removing them.
@@ -6,63 +6,40 @@ English | [中文](2026-08-05-session-preparation.zh.md)
## Problem
Cold history inspection and Agent resume independently materialized the same persisted session log. For a large compressed log, each operation repeated the full read, decompression, parse, validation, freezing, and Session construction. Pagination could therefore pay the cold-read cost again, while making a history query activate an Agent would couple a read lifecycle to a live Agent with no natural retirement point.
Fresh creation and persisted resume reached the same publication boundary through different construction flows. This obscured the invariant that setup must finish against one unpublished Session before that exact Session and its Agent become visible together.
Fresh creation and persisted resume also reached the same publication boundary through different construction flows. This obscured the invariant that setup must finish against one unpublishedSession before that exact Session and its Agent become visible together.
Cold history inspection and Agent resume also independently materialized the same persisted session log, which this note originally answered with a persistence-side prepared-Session cache; that half is superseded below.
## Decision
`SessionPreparation` owns one exact unpublished `Session` until publication or rollback. It is a Session lifecycle object, not an Agent lifecycle or activation object. Fresh creation wraps the result of `SessionStore.prepare()`; persisted resume obtains a preparation from `SessionPersistence.prepare()`.
`SessionPreparation` owns one exact unpublished `Session` until publication or rollback. It is a Session lifecycle object, not an Agent lifecycle or activation object. Fresh creation wraps the result of `SessionStore.prepare()`; persisted resume reads the stored log through the session's write handle, appends `interruptedTurnClosers`, and wraps `SessionStore.prepare(id, { seed, meta, seedSource: 'persistence' })` — the restoration branch that validates and freezes the transferred graphs in place.
The Agent loop consumes both forms through one setup-and-publication pipeline: it acquires the preparation, builds the private Agent context around `preparation.session`, awaits optional setup, publishes that exact Session and Agent, and disposes the preparation on every exit. Publication transfers the live lifecycle to the existing Session and Agent stores; `SessionPreparation` itself owns no Agent behavior.
This refines the publication boundary from the [Agent lifecycle and ownership decision](2026-06-18-agent-lifecycle-and-ownership-contracts.md) without replacing its ownership model.
## Persisted preparation lifecycle
## Superseded: the persistence-side preparation lifecycle
A coordinator-backed persistence implementation loads one cold source into a prepared Session. The backend transfers fresh, mutually unaliased metadata and events together with the source-qualified revision that identifies those exact values; the Session restore path validates and freezes the graphs in place instead of cloning them. The coordinator computes interrupted-turn closers and constructs the exact unpublished Session once. Its immutable header and balanced logical event log form the `SessionInspection` borrowed by readers, while the revision remains internal to persistence.
`inspect(id, signal?)` does not mutate storage. Synthetic closers exist only in the prepared in-memory view, and a torn physical tail remains untouched. Same-id callers share an in-flight cold read. Once ready, the preparation may remain in a per-coordinator LRU whose capacity defaults to five and is configurable by first-party backends. Before reusing a retained source, the coordinator reads that id's current revision; a mismatch evicts a ready source and repeats the cold materialization. A source already committing or reserved for resume remains exclusively owned, so concurrent inspection borrows that immutable view until publication or release.
`prepare(id, signal?)` exclusively reserves the prepared Session. It confirms the retained revision before committing any torn-tail and interrupted-turn repair, establishes the durable cursor, then returns a disposable preparation. A stale source is discarded and reloaded instead of being repaired or published. A successful repair also discards the pre-repair source and materializes the committed log again before reservation, so a newer revision is never associated with an older event graph. Another same-id preparation waits until the reservation is published or released. Publication accepts only the exact reserved Session and attaches the committed cursor without rebuilding its history. Failed setup or cancellation returns an unchanged unpublished Session to the LRU; mutation or attachment consumes the reservation.
The legacy `load(id)` API uses the same preparation and repair machinery, then discards its reservation and returns the immutable logical view. It remains a compatibility API, not the history-to-resume reuse path. This lifecycle extends the [shared persistence coordinator](2026-06-18-shared-persistence-write-coordinator.md) while preserving the storage and recovery rules owned by the [session persistence decision](2026-06-14-session-persistence.md).
## History and resume reuse
History reads use `inspect()`, so repeated pages borrow the same immutable prepared state without activating an Agent. A later resume uses `prepare()` and receives the exact Session retained by inspection; it does not read, decompress, parse, clone, validate, or freeze the complete log again.
If the durable log changes after inspection, its revision changes. The next history read or resume discards a retained ready Session and materializes the new log, so an old event graph cannot be associated with a newer snapshot revision. A source already claimed by an in-flight resume is not evicted: its exclusive owner keeps it through publication or release, and concurrent history may borrow the same immutable view.
Cold continuable-subagent access follows the same path. Descriptor authorization first inspects the child, then `ctx.agents.resume()` reserves and publishes the retained Session. This preserves the lifecycle and authorization rules in the [continuable subagent conversation decision](../feature/2026-07-28-continuable-subagent-conversations.md) while removing its duplicate cold read.
This note originally also gave persistence a `prepare(id)`/`inspect(id)` lifecycle: a coordinator-backed bounded LRU of cold unpublished Sessions with exclusive reservations, revision-checked reuse, and repair committed inside `prepare`/`load`, so history pagination and a later resume shared one cold materialization. The [handle-based persistence seam](2026-08-27-handle-based-session-persistence.md) deletes all of it: persistence exposes handles only, resume reads the log through its write handle and owns repair, and read-only observers (session-query) own their cold-Session cache keyed by the `stat().revision` change token. The read-reuse goal survives in that cache; the exclusive-reservation machinery does not, because the write handle's single-writer ownership is the exclusion resume actually needs. Resume pays one whole-log read through the handle where the prepared cache sometimes served a warm Session — an accepted cost recorded in the handle note.
## Boundaries
- `readFrom()` remains a detached physical-suffix API. It neither creates nor consumes a preparation, synthesizes logical closers, or joins the LRU.
- HMR adoption keeps the live Session authoritative and reads the stored prefix directly. It may truncate a torn physical fragment but never closes the live open turn as interrupted.
- The cache belongs to one persistence coordinator, not a process-global Session map. Live Sessions are owned by the existing stores and never occupy preparation capacity.
- A fresh create never claims a cold persisted preparation with the same id. Persistence collisions continue to reject.
- Third-party persistence implementations retain the abstract `prepare()` fallback through `load()`. They receive the same publication interface but gain exact-object reuse only when they override preparation.
- Revision validation establishes freshness at the reuse and repair-commit points; it does not add cross-process writer exclusion to a backend. Retries converge after the durable log remains unchanged for one read/check round trip, so continuous external writers can delay preparation.
- The preparation is one disposable ownership window, not a cache: disposal is synchronous and idempotent, and publication accepts only the exact prepared Session.
- A fresh create never claims a persisted identity implicitly. Persistence collisions continue to reject (`SessionAlreadyExistsError`, `SessionAlreadyOwnedError`).
- Live Sessions are owned by the existing stores; preparations hold only unpublished ones.
## Verification
The shared persistence contract pins non-mutating balanced cold inspection and later repair. `persistence.spec.ts` and `preparations.spec.ts` pin same-id in-flight sharing, exact Session reuse across inspect and prepare, revision-triggered refresh before history and resume, single repair commit, exclusive reservation, release after failed setup, ready-entry LRU eviction, append rejection during reservation, and publication of only the reserved Session. Backend tests pin that full and lightweight reads use the same revision identity. Agent-loop and continuable-subagent tests pin the common publication pipeline and inspection-to-resume path across cancellation and teardown.
Agent-loop tests pin the common publication pipeline across create, `createAgent`, and resume, including rollback on setup failure, cancellation, and teardown, and that disposal releases the write handle (reopening for write succeeds). Session-store tests pin the restoration branch's validate-and-freeze-in-place transfer.
## Alternatives considered
**Activate an Agent for history reads.** Rejected because pagination would keep query-only Agents live and transfer cache retirement into the Agent lifecycle.
**Activate an Agent for history reads.** Rejected because pagination would keep query-only Agents live and transfer cache retirement into the Agent lifecycle. This rationale still guards the session-query cold cache: observation never creates an Agent.
**Cache only `{ meta, events }`.** Rejected because resume would still reconstruct, validate, freeze, and copy a Session from the cached values. The exact unpublished Session is the reusable unit.
**Cache only `{ meta, events }`.** Rejected at the time because resume would still reconstruct a Session from the cached values. Under the handle seam this is exactly what the read side does — session-query caches a cold Session per revision for reads only — while resume rebuilds from the handle read, trading the warm-Session reuse for a single write-ownership door.
**Keep a process-global Session map.** Rejected because it would cross backend and runtime ownership boundaries, retain unbounded identities, and duplicate the live Session store.
**Add a restore transaction or coordinator to the Agent loop.** Rejected because cold reading, repair, reservation, and cursor attachment are persistence and Session concerns. The Agent loop only needs the uniform `SessionPreparation` ownership boundary.
**Turn `readFrom()` into logical preparation.** Rejected because watermark consumers need a detached physical suffix and, on seek-capable backends, a bounded read. Recovery balancing and whole-Session reuse have different semantics.
**Add a restore transaction or coordinator to the Agent loop.** Rejected because cold reading and Session construction are persistence and Session concerns. The Agent loop only needs the uniform `SessionPreparation` ownership boundary; the handle seam kept that split while moving repair to the loop's resume path.
## Consequences
One cold materialization can serve history pagination, subagent descriptor inspection, and a later resume. Ownership transfer removes redundant restoration clones, while the bounded per-coordinator LRU limits memory and avoids creating live Agents for queries. Create and resume share one publication protocol without merging Agent and Session responsibilities.
The first cold inspection now pays the complete validation and Session-construction cost and may retain that unpublished Session until eviction. Persistence must coordinate reservation, append, repair, and publication, and callers must treat inspection values as immutable borrowed state. Backends that rely on the default `prepare()` remain correct but do not receive the reuse optimization.
Create and resume share one publication protocol without merging Agent and Session responsibilities, and every exit path disposes exactly one preparation. The persistence-side reuse consequencesoriginally recorded here (shared cold materialization, LRU bounds, reservation coordination) now belong to the [handle note](2026-08-27-handle-based-session-persistence.md) and the session-query cache that replaced them.
存量 `load(id)` API 使用相同的准备和修复机制，随后丢弃其预留并返回不可变逻辑视图。它保留为兼容 API，不承担历史到恢复的复用路径。该生命周期扩展了[共享持久化协调器](2026-06-18-shared-persistence-write-coordinator.zh.md)，同时继续遵循[会话持久化决策](2026-06-14-session-persistence.zh.md)所规定的存储与恢复规则。
@@ -14,16 +14,16 @@ The root cause is that the [durable-subagent-catalog decision](../feature/2026-0
## Decision
mode and label are folded by the new `subagent` projection unit (pure identity, two arms), and the unit is the sole authority over the fold rules; `listChildren` no longer depends on session-query — enumeration is a subagent-owned live-preferred merge, and value retrieval walks a three-rung compute-and-discard ladder: a live child synchronously reads the registry's existing watermark cache (zero log reads); a cold child first asks the optional `sessionProjectionCache` checkpoint, and a served identity that passes the seq gate is final; otherwise it pays one full `persistence.inspect` read plus a fold through the registered `subagent` unit. No index, no cache of its own, no write-back.
mode and label are folded by the `subagent` projection unit (pure identity, two arms), and the unit is the sole authority over the fold rules. Enumeration uses the shared Sessionquery corpus, while value retrieval walks a three-rung compute-and-discard ladder: a live child synchronously reads the registry's existing watermark cache (zero log reads); an unseeded cold child may use the optional `sessionProjectionCache` checkpoint because its exact inherited cut is known to be zero; every seeded child and every cache miss pays one body-bearing Session observation plus a fold through the registered `subagent` unit. No index, no cache of its own, no list-side write-back.
There are three families of escape from the per-child scan: promote mode/label into the header (the write path pays); build a durable derivation for the projection (a checkpoint ladder, or values landed during query-index rebuild with read-side reconciliation); or compute at read time (live from the watermark cache, cold from one full read). This note takes the third. "Values landed with the query index" was retired wholesale: query infrastructure was forced to learn domain vocabulary while the sole consumer is satisfied by read-time computation — the live child's zero reads come for free from session-projection's existing watermark cache, and the cold child's single full read is explicitly accepted as compute-and-discard. The first two routes and the retirement rationale are detailed under Alternatives considered.
Key points:
- **The subagent list does not depend on session-query**: enumeration is completed by a subagent-owned live-preferred merge, and mode/label is retrieved through `ctx.sessionProjections`; deployments without a query backend list as usual.
- **Value retrieval is a three-rung compute-and-discard ladder**: a live child reads `sessionProjections.snapshot(session, ['subagent'])` (the registry's existing watermark cache, zero log reads); a cold child first reads the optional`sessionProjectionCache.cachedSnapshot(header, ['subagent'])`, using the non-null identity directly when it passes the seq gate (`seq >= seedLength ?? 0`); otherwise it pays one full Session observation plus a fold through the registered `subagent` unit; beyond that, absent is absent — no cache of its own, no write-back, no index.
- **The subagent list uses the Sessionquery corpus for enumeration and body-bearing observations**: mode/label still comes through `ctx.sessionProjections`, and the list owns no descriptor parser or domain index.
- **Value retrieval is a three-rung compute-and-discard ladder**: a live child reads `sessionProjections.snapshot(session, ['subagent'])` (the registry's existing watermark cache, zero log reads); an unseeded cold child may read`sessionProjectionCache.cachedSnapshot(header, SessionLogOffset(0), ['subagent'])`; a seeded child or cache miss pays one Session observation carrying `inheritedEventCount` plus a fold through the registered `subagent` unit. Beyond that, absent is absent — no cache of its own, no list-side write-back, no index.
- **The `subagent` projection unit is the sole authority over the fold rules**: live and cold snapshots both run the one registered unit; no second copy of descriptor-interpretation logic exists.
- **The header, the descriptor (v2), session-persistence, session-projection(-cache), and session-query(-sqlite) are all untouched**; pre-existing data acquires exact values through one `inspect` computation the first time it is listed — no degraded unknown state, no migration.
- **The descriptor (v2) remains untouched**. Session, persistence, projectioncache, and query now carry the exact inherited cut separately from the logical header; pre-existing data acquires exact values through one body-bearing observation when listing cannot prove a zero cut — no degraded unknown state and no durable format migration.
Relationship to existing notes:
@@ -36,8 +36,8 @@ It hangs beside the existing `subagentTiming` ([projection.ts](../../../../packa
```ts ignore-check
export type SubagentIdentityProjection =
| { mode: 'one-shot'; label?: string; seq: number }
| { mode: 'continuable'; label: string; seq: number }
- The projection is pure identity, and **the projection system has no failure channel**: a unit never throws; a corrupt payload or an unrecognized version folds exactly like a log with no descriptor at all. The host checkpoint state is the serializable wrapper `{ identity?: SubagentIdentityProjection }`; absence is `{}`. Its client view is the non-optional `SubagentIdentityProjection | null` entry. `null` passes JSON losslessly, so a pushed reset replaces a stale identity instead of being dropped by stringify. The judging discipline: consuming surfaces treat null and an absent client key alike as no value. How "computed to nothing" is presented is the consumer's own business (see the `listChildren` four-state mapping below).
- Label strength is decided by the descriptor schema: a continuable's label is mandatory at parse, a one-shot's was always optional; the mode/label discriminant matches the child row's strong contract below exactly (the row carries no `seq` — it is the projection's internal own-suffix proof).
- The identity carries `seq`: the seq of the `subagent/descriptor` event it was folded from, mandatory on both arms and absent on the null sentinel — `seq >= header.seedLength ?? 0` proves the identity was folded from the child's own suffix rather than a fork seed's replayed ancestor descriptor. The unit maps the wrapper's validated identity to its client wire view and is checkpointed like every unit (the `persist` opt-in is gone); its `stateVersion` is 2, bumped when `seq` was added. Existing older checkpoint rows are invalidated by version mismatch per the registry contract, falling to the authoritative refold.
- The identity carries branded `seq`: the seq of the `subagent/descriptor` event it was folded from, mandatory on both arms and absent on the null sentinel. A live Session checks it through `isOwnSeq()`; a cold body-bearing observation compares it with `inheritedEventCount`. Header-only seeded candidates skip the cache because the header intentionally exposes no integer cut; unseeded candidates know the cut is zero. The unit maps the wrapper's validated identity to its client wire view and is checkpointed like every unit (the `persist` opt-in is gone); its `stateVersion` is 2, bumped when `seq` was added. Existing older checkpoint rows are invalidated by version mismatch per the registry contract, falling to the authoritative refold.
- Fold rule: `subagent/descriptor` is last-wins, under the same descriptor-reset discipline as `subagentTiming` — ancestor descriptors in the fork prefix are overridden by the session's own descriptor. A corrupt or unrecognized-version payload is last-wins all the same: it resets to the null sentinel rather than keeping the prior identity, so a fork of a healthy ancestor does not inherit an identity its own descriptor cannot stand up.
`listChildren`'s ([list-children.ts](../../../../packages/subagent/subagent/src/list-children.ts)) enumeration goes through no query service: the two sources `ctx.sessions.list()` and `ctx.get('sessionPersistence')?.list()` merge by id, with a live record overriding the same-id persisted record wholesale and no header consistency check. Everything enumeration needs is header facts:
`listChildren` ([list-children.ts](../../../../packages/subagent/subagent/src/list-children.ts)) asks `sessionQuery.listSessions()` for the canonical live-preferred corpus, then pairs each listed id with `ctx.sessions.get(id)` when a live Session exists. The live header overrides the listed header for that id. Everything enumeration needs is header facts:
- `hasChildren`: the same merged material, looked at one level down — a direct descendant exists with `origin === 'subagent'` whose `parentSession` is that child.
- `activity`: a live record is `running`; one present only in persistence is `inactive`.
- Ordering: `createdAt` ascending, then child id ascending (matching the old contract).
- **Absent persistence degrades to live-only enumeration, not an error**: in a deployment without persistence, a cold child could not be resumed anyway, and listing live children remains meaningful. (Contrast: the old implementation rejected wholesale when sessionQuery was missing.)
- A persistence listing failure fails the whole enumeration; per-child isolation applies only to the per-child cold reads.
- An absent `sessionQuery` service fails with `SUBAGENT_CONTROL_QUERY_UNAVAILABLE`; the shared query corpus owns whether a deployment can enumerate live-only or persisted Sessions.
- A query-corpus failure fails the whole enumeration; per-child isolation applies only to per-child cold observations.
### Value retrieval: the three-rung compute-and-discard ladder
@@ -73,20 +73,20 @@ For each enumerated child, mode/label retrieval walks a three-rung ladder — co
| Rung | Read | Cost |
| --- | --- | --- |
| 1: live child | `ctx.sessionProjections.snapshot(session, ['subagent'])` | Zero log reads — the registry's existing watermark cache, synchronous retrieval |
| 2: cold child, cache hit | The optional `sessionProjectionCache.cachedSnapshot(header, ['subagent'])`, used directly only when the non-null identity satisfies `identity.seq >= header.seedLength ?? 0` — an own descriptor is immutable once appended, and the seq gate proves the value was folded from the child's own suffix, regardless of the row's watermark | Zero log reads |
| 3: cold child, fallback | One full `persistence.inspect(id)` read + a fold through the registered `subagent` unit | One full read computed per listing |
| 2: unseeded cold child, cache hit | The optional `sessionProjectionCache.cachedSnapshot(header, SessionLogOffset(0), ['subagent'])`; every valid seq is owned when the exact cut is zero | Zero log reads |
| 3: seeded child or cold fallback | One body-bearing `sessionQuery.observeSession(id)` + the registered `subagent` projection, with `inheritedEventCount` available for the own-suffix check | One full read computed per listing |
- Error contract: `sessionProjections` is a required injection — `SubagentRuntime` declares it in its inject set, so a deployment without the registry never activates the service (or the loop), and `listChildren` is unreachable rather than served degraded rows ([mandatory-seam note](2026-08-19-session-projection-mandatory-seam.md)); the loud runtime check and `SUBAGENT_CONTROL_PROJECTIONS_UNAVAILABLE` are deleted with it. The session store keeps the explicit posture: an absent `ctx.get('sessions')` (a strict global read, never the caller-scope-bound property proxy) fails with`SUBAGENT_CONTROL_SESSION_STORE_UNAVAILABLE`. apiproxy's dedicated `PROJECTIONS_UNAVAILABLE` wire face is deleted along with the code; `SESSION_STORE_UNAVAILABLE` goes through the generic internal fallback — apiproxy's composition injects `sessions` itself, so that error is unreachable in its deployment, and a dedicated mapping would violate the need principle. `SUBAGENT_CONTROL_SESSION_QUERY_UNAVAILABLE` is deleted along with the session-query dependency.
- The cache is a purely optional acceleration layer: an absent service is skipped on a null check — no error code, no part in configuration validation (in contrast to `sessionProjections`, a required injection). Anything the second rung throws (including a poisoned unit row in the cache detonating `viewCheckpoint`) silently falls to the third rung — the cache is derived data, so its faults never produce a `corrupt` verdict; the final judgment belongs to the authoritative refold. A row whose checkpoint cut predates the descriptor naturally lacks the `subagent` key and falls through automatically, with no special-casing; a null sentinel in the row does not count either — it falls to the third rung for the authoritative refold's verdict. A count/interval checkpoint inside the creation window can land a fork seed's replayed ancestor identity in the row — the ancestor's seq falls inside the seed range, the seq gate rejects it, and it likewise falls to the third rung's verdict.
- Error contract: `sessionProjections`, the Session store, and `sessionQuery` are required runtime services for listing. Their explicit failures are `SUBAGENT_CONTROL_PROJECTIONS_UNAVAILABLE`,`SUBAGENT_CONTROL_SESSION_STORE_UNAVAILABLE`, and `SUBAGENT_CONTROL_QUERY_UNAVAILABLE`; no empty result disguises a missing classification or corpus capability.
- The cache is a purely optional acceleration layer: an absent service is skipped on a null check — no error code, no part in configuration validation (in contrast to `sessionProjections`, a required injection). A seeded header skips this rung because it cannot supply the cache identity's exact cut without a body read. For an unseeded child, anything the second rung throws (including a poisoned unit row detonating `viewCheckpoint`) silently falls to the third rung — the cache is derived data, so its faults never produce a `corrupt` verdict; the final judgment belongs to the authoritative refold. A row whose checkpoint cut predates the descriptor, an absent key, or a null sentinel likewise falls through.
- Per-child isolation: a single child's failed cold full read only turns that row into an `unavailable` diagnostic, naturally retried on the next listing, without affecting siblings (see the four-state mapping).
- The cold path's lifecycle witness: preparation's result must still point at the lifecycle that was enumerated — the witness field set is the same seven fields as the old SOURCE_CONFLICT check (version, id, createdAt, cwd, parentSession, seedLength, delegationDepth); a session deleted and republished under the same id degrades to a `corrupt` row in the old parent's catalog, leaking nothing of the new owner's child.
- The cold path's lifecycle witness: the observation must still point at the lifecycle that was enumerated. The witness fields are version, id, createdAt, cwd, parentSession, isSeeded, delegationDepth, origin, and agentPreset; a Session deleted and republished under the same id degrades to a `corrupt` row in the old parent's catalog, leaking nothing of the new owner's child.
- Cold-read concurrency is bounded by the constant 4 — it constrains a read-only scan of local media, not deployment behavior; when a networked persistence backend appears, it is promoted to a validated `Config` field.
- The cold-read cost, recorded honestly: only with the cache unmounted or missed does a cold child pay one full read per listing, at a cost proportional to its transcript size; the settled stance is compute-and-discard, and no cache of its own is built. The full read goes through `inspect()` into the [Session preparation](2026-08-05-session-preparation.md) cold read, so short-term repeated reads of the same id can hit its LRU for reuse, but listing does not depend on this. A live child reads zero log throughout.
- The cold-read cost, recorded honestly: every seeded child and every unseeded cache miss pays one full query observation per listing, at a cost proportional to its transcript size; the settled stance is compute-and-discard, and no cache of its own is built. The observation may reuse the query/persistence preparation layer, but listing does not depend on that optimization. A live child reads zero log throughout.
- Cancellation: the caller's signal is checked before and after each persistence read, and a read that settles only after abort is rejected, normalized to the stable error code `CANCELLED`.
### Authority model
- The session log is the sole authority; this design adds no derived persistence of any kind — no index values, no checkpoints of its own, no in-process memo; the `sessionProjectionCache` checkpoint the second rung reads is an existing composition item's derived data, which this design only reads and never writes. Values are computed on read and discarded, and a value's freshness is exactly the live state or persisted revision at the moment of the read (an own descriptor is immutable once appended — a cached identity past the seq gate has no staleness problem; the gate guards against seed-replayed ancestor identities).
- The session log is the sole authority; this design adds no domain index, checkpoint of its own, or in-process memo. The `sessionProjectionCache` checkpoint the second rung reads is an existing composition item's derived data, which the list only reads. Values are computed on read and discarded. Seeded candidates use a body-bearing observation to classify the identity against the exact cut; unseeded cached identities need no seq gate because every valid seq is owned.
- The Session and persistence write paths are entirely unaware of listing and projection consumption: no event-listener write-back, no fold-on-write.
- Enumeration and value retrieval constitute no second authorization source and make no unpublished child visible — the two sources see only published live records and durably written persisted records, consistent with the rule the durable-subagent-catalog note laid down for derived read surfaces.
@@ -127,25 +127,24 @@ For each enumerated child, the ladder's result maps to a row through four states
Known boundary deviations (deliberately accepted, recorded with this note):
- A fork child that died in its publication window, with an ancestor descriptor in its seed, gets the ancestor identity from last-wins and wrongly surfaces as a child row; resume still fails against the own-suffix fold authority (`NOT_RESUMABLE`). The old implementation omitted it via `seedLength` filtering; the projection unit cannot see the header, and this debris-grade deviation is accepted (`subagentTiming` has the same kind of pre-existing exposure).
- Multiple descriptors in the own suffix: the old implementation judged corrupt; last-wins now takes the final one (the provider contract guarantees exactly one anyway).
- A live/persisted header conflict: the old implementation made it per-child corrupt; enumeration now prefers live with no consistency check, the conflict goes unnoticed, and the live record forms the row.
- A source-read failure on damaged storage (e.g. a bad surface rejected by the cold full read): the old implementation mapped it to per-child `corrupt`; it is now uniformly an `unavailable` row (the read side cannot tell the causes apart).
- An unknown parent: the old implementation threw not-found through session-query ('parent session … was not found'); the subagent-owned merge now yields an empty subset for a nonexistent parent, enumeration returns an empty list, and later operations on the wire land as child-level subagent-not-found — a silent change of semantics and wording, recorded as explicitly accepted.
- Rung 2's later-event window: a cache row lands right after the first own descriptor, the log then appends a second own descriptor (or a malformed payload setting the null sentinel), and the process crashes before the next checkpoint — from then on a cold listing's rung 2, admitted by the seq≥seedLength gate, keeps serving the row's old identity (the first own descriptor's value), diverging from the authoritative refold (last-wins, the second), and a rung-2 hit triggers no refold, so nothing notices. Three boundaries: ① the precondition is a second own descriptor on the same child, violating the establishing provider's append-exactly-once contract — corruption-class data, same family and source as the multi-descriptor deviation; ② it takes both "corruption + a crash missing every checkpoint (the two mandatory points, turn/end and disposal, and the count/interval throttle points all unmet)" at once; ③ a healthy child (exactly one own descriptor) is unaffected — what the seq gate admits is precisely the only true identity. Self-healing: any live run of that child (the turn/end mandatory checkpoint) or any moment that triggers cache.write overwrites the whole row with a fresh fold (whole-record replace), and rung 2 serves correctly from then on; the authoritative paths (the rung-3 refold, the live snapshot, the resume fold) are correct from the start, and the divergence exists only in listing reads while the child stays cold and the row is never rewritten. The mechanical fixes were not taken: gate reconciliation would need the log-end seq, unavailable to a zero-read cold path; a cache row carrying the revision is an opaque token, incomparable and a cross-domain schema change — filed as accepted under the "the cache is never authoritative" doctrine.
- Rung 2's later-event window applies only to an unseeded child: a cache row lands right after the first descriptor, the log then appends a second descriptor (or a malformed payload setting the null sentinel), and the process crashes before the next checkpoint. Cold listing can keep serving the old identity until a live run or cache write replaces the row. The precondition violates the provider's append-exactly-once contract and also requires missing every mandatory checkpoint; healthy children are unaffected. Seeded children never take rung 2 without the body-owned cut.
Consuming surfaces: diagnostic handling across wire, tool, and GUI **stays entirely as it was, zero changes** (the `list_agents` description and output schema are untouched; the plugin's load requirement changes — `sessionQuery` dropped from inject, `sessionProjections` added as a required injection). The only behavioral changes are in apiproxy: on the route segment, the `hasSubagentDescriptor()` scan is deleted and `hasSubagentOwner` looks only at `header.origin` — pre-#1569 data without `origin` is no longer recognized as a subagent owner; it never entered the catalog anyway, and the pre-release stance accepts this; and `subagents.history` is aligned with `session.history`'s source — a live child served from in-memory events and the registry's watermark snapshot, a cold child from `inspectServable` reading persistence directly with a detached fold, no query service involved, the SESSION_QUERY_* error arms retired with it, and the wire shape unchanged (the `history` JSDoc wording becomes the live in-memory snapshot / cold persisted log dual arm).
Consuming surfaces keep the same row and diagnostic wire shape. `list_agents` reaches the required query corpus plus projection registry; live identities come from the registry snapshot and cold identities from cache or query observation. Host ownership still uses `header.origin`, and history uses the shared live/cold Session query sources; no consumer parses descriptor events independently.
### Change footprint
| Area | Files | Change |
| --- | --- | --- |
| subagent | projection.ts, projection-types.ts, index.ts | New client-visible `subagent` unit and its registration |
| subagent | list-children.ts and its types | Rewritten as subagent-owned enumeration plus the projection-ladder four-state mapping; the session-query dependency, per-child event reads, and in-place classification machinery deleted; error code `SUBAGENT_CONTROL_SESSION_QUERY_UNAVAILABLE` deleted, and `sessionProjections` becomes a required injection (no projection error code remains); new optional dependency dsh-session-projection-cache (pure read acceleration, skipped when absent) |
| host/apiproxy | api-proxy.ts | `hasSubagentDescriptor` deleted; the owner check looks only at `header.origin`; `subagents.history` shares `session.history`'s source — live from in-memory events and the registry's watermark snapshot, cold from `inspectServable` reading persistence directly with a detached fold, no query service, the SESSION_QUERY_* error arms and the dedicated `PROJECTIONS_UNAVAILABLE` wire face retired with it |
| tool | tool-subagent-control/list-agents.ts | Load requirement narrowed (`sessionQuery` dropped from inject); model-visible schema, description, and rendering unchanged |
| subagent | list-children.ts and its types | Query-corpus enumeration plus the projection-ladder four-state mapping; required projections/query services and optional projection-cache acceleration |
| host/apiproxy | Session controller/query integration | Owner checks use `header.origin`; live/cold history and listing consume the shared query and projection sources |
| wire/client | api/subagents.ts, runtime sessions/service.ts, GUI | Types, row shape, and diagnostic handling **unchanged**; api/subagents.ts only reworded the `history` JSDoc to the dual arm |
| core/session, session-persistence, session-projection(-cache), session-query(-sqlite) | body-bearing cut and branded seq plumbing | Logical headers expose `isSeeded`; Session, persistence observations, cache identity, and query records carry exact `inheritedEventCount` separately |
## Alternatives considered
@@ -169,20 +168,20 @@ Consuming surfaces: diagnostic handling across wire, tool, and GUI **stays entir
## Verification
`packages/subagent/subagent/tests/list-children.spec.ts`is rewritten to this contract: live-only listing without persistence, query services, or the continuation runtime; without the registry the service never activates (the mandatory seam — a `setup` variant asserting `ctx.get('subagents')` stays undefined); a live child incurs zero `inspect` throughout while a cold child incurs exactly one per listing; multiple descriptors resolve last-wins to the final one; corrupt payloads and unknown versions fold to `corrupt`; a cold-read failure maps to `unavailable` and retries on the next listing; the ancestor descriptor in a fork seed forms a row under that identity (pinning deviation one); ordinary forks and descendants without a subagent origin neither enter the list nor count toward `hasChildren`; `createdAt`-then-id ordering; an unmounted provider does not affect listing; compacted and uncompacted twins list identically; the three cases of pre-abort, persistence listing, and cold-read cancellation all normalize to `CANCELLED`; the empty list and stable error codes (`SUBAGENT_CONTROL_SESSION_STORE_UNAVAILABLE` for an absent store). Second-rung cases: an own-seq identity used directly with zero `inspect`, a fork seed's ancestor identity (seq inside the seed range) rejected by the gate and falling through, an in-row identity absence (null sentinel or absent key) falling through, an absent cacheservice falling through, and a poisoned cache row silently falling through to the refold; cold-path lifecycle tampering degrades to `corrupt`field by witness field (`it.each` over the seven). The `tool-subagent-control` list-agents tests are updated for the narrowed load requirement; `optional-session-query.spec.ts` is deleted with the dependency it guarded; the existing keyless snapshots (`subagent-list-agents` among others) are unchanged, pinning that the healthy path's wire and model-visible surfaces did not move; a new keyless snapshot, `subagent-diagnostic` (examples/headless-agent), pins the four-state mapping's diagnostic classification — the model-visible changes such as descriptor-less settled debris becoming a `corrupt` row.
`packages/subagent/subagent/tests/list-children.spec.ts`pins this contract: live identity checks use `Session.isOwnSeq()`; an unseeded cold identity may use the cache at cut zero; seeded candidates skip that cache rung and use an observation carrying `inheritedEventCount`; ancestor identities fail the own-suffix check; absent, null, poisoned, and unavailable cache/observation cases fall through or produce the documented diagnostic; lifecycle tampering degrades to `corrupt`across the complete witness field set. The existing keyless snapshots keep the healthy wire and model-visible surfaces fixed, while `subagent-diagnostic` pins diagnostic classification.
## Consequences
- Listing a live child reads zero log throughout; with the cache unmounted or missed, a cold child pays one full `inspect` read per listing, at a cost proportional to its transcript size and repeated with listing frequency — compute-and-discard is the settled stance: no cache of its own is built, nothing is written back, and short-term repeated full reads of the same id can hit the preparation-phase LRU, though listing does not depend on it.
- The subagent list no longer requires a query backend: both pure-live and persistence-less deployments can list; `SUBAGENT_CONTROL_SESSION_QUERY_UNAVAILABLE` is gone, loading the `list_agents` plugin no longer requires `sessionQuery`, and `sessionProjections` becomes a required injection of `SubagentRuntime` — a deployment without the projection registry never activates the service (the mandatory seam).
- The subagent list requires the Session query corpus and projection registry; missing services fail explicitly instead of producing incomplete rows. The optional projection cache changes only the number of body reads.
- Identity interpretation exists only in the single unit registered with the registry: the list's three-rung ladder and GUI history's cold read use its live, cached, or observed wire snapshots, and no hand-written bypass fold exists; if some future consuming surface bypasses the unit with a hand-written fold, values will drift across read faces — a discipline this design requires be maintained, not a mechanical guarantee.
- Per-child isolation is back: a single child's cold-read failure loses only that row and healthy siblings are unaffected; a persistence listing failure still fails the whole enumeration.
- The diagnostic and enumeration semantics leaves six boundary deviations (a stillborn fork surfacing under its ancestor's identity, multiple descriptors resolving to the last, header conflicts going unnoticed, damaged-source read failures shifting from `corrupt` to `unavailable`, an unknown parent yielding an empty list instead of not-found, and rung 2's later-event window); the full semantics is in the known-boundary-deviations list; the first four are display or classification deviations on debris-grade data, the unknown-parent one is a silent query-semantics change, and the rung-2 window is a self-healing cache-serving divergence under the double condition of corruption plus a crash; resume authorization is unaffected throughout, all explicitly accepted.
- The diagnostic and enumeration semantics leaves five boundary deviations (multiple descriptors resolving to the last, header conflicts going unnoticed, damaged-source read failures changing classification, an unknown parent yielding an empty list instead of not-found, and the unseeded rung-2 later-event window). Seeded ancestor identities are no longer a deviation because body-bearing reads compare them with `inheritedEventCount`; resume authorization remains unaffected.
- Pre-#1569 data without `origin` is no longer recognized as a subagent owner; it never entered the catalog anyway, and pre-release carries no compatibility promise.
## Related
- [Durable subagent catalog and list_agents](../feature/2026-07-22-durable-subagent-catalog-and-list-agents.md) — partially superseded by this note: the descriptor remains the durable authority for mode/label and the fold input, while the list's enumeration and value retrieval move to the subagent-owned merge plus the projection ladder.
- [Durable subagent catalog and list_agents](../feature/2026-07-22-durable-subagent-catalog-and-list-agents.md) — partially superseded by this note: the descriptor remains the durable authority for mode/label and the fold input, while value retrieval moves to the projection ladder over the shared query corpus.
- [Session projections and command lifecycle logging](../../proposed/architecture/2026-07-27-session-projection-and-command-log.md) — the authority for the registry contract; this note adds the `subagent` identity unit and consumes its live and cold wire snapshots.
- [Session projection state and client views](2026-08-19-session-projection-state-and-client-views.md) — the state/client split; both `subagent` and `subagentTiming` provide client wire views.
- [Session projections as a required seam](2026-08-19-session-projection-mandatory-seam.md) — `sessionProjections` becomes a required injection; the list's error contract follows it (registry absence is an activation-time failure, and the projection error code is deleted).
`listChildren`（[list-children.ts](../../../../packages/subagent/subagent/src/list-children.ts)）的枚举不经任何查询服务：`ctx.sessions.list()` 与`ctx.get('sessionPersistence')?.list()` 两个来源按 id 合并，live 记录整条覆盖同 id 持久化记录、不做 header 一致性校验。枚举所需全部是 header 事实：
`listChildren`（[list-children.ts](../../../../packages/subagent/subagent/src/list-children.ts)）通过 `sessionQuery.listSessions()` 取得 canonical live-preferred corpus，再把每个 listed id 与可能存在的`ctx.sessions.get(id)` 配对；同 id 存在 live Session 时使用 live header。枚举所需全部是 header 事实：
@@ -6,13 +6,13 @@ English | [中文](2026-08-08-bounded-session-persistence-write-batching.zh.md)
## Problem
Streaming responses can emit many `assistant/chunk` events in a short interval. The persistence coordinator previously scheduled a provider append as soon as an idle queue received one event. Events arriving while that append was active shared a follow-up batch, but a fast provider could still produce many small durable appends. Each JSONL append creates and syncs a Zstandard frame or raw suffix.
One agent step can emit several durable events in a short interval: request metadata, one Assistant settlement, tool lifecycles, plugin facts, and execution boundaries. Scheduling a provider append as soon as an idle queue receives one event can therefore produce many small durable appends. Each JSONL append creates and syncs a Zstandard frame or raw suffix.
Dropping chunk events or replacing them with assembled messages would reduce logical storage, but it would also change the event log, replay, sequence numbers, timestamps, and the chunk seqs cited by assistant messages. The write-amplification problem does not require that larger semantic change.
Assistant stream embedding reduces one high-volume event family, but write cadence remains a provider-neutral lifecycle concern for every other burst and for historical generations. The batching decision does not change event semantics or storage encoding.
### Quantified baseline
Repository fixtures make the logical volume concrete. Decoding the current packed rows in [`goal-multi-turn-actions`](../../../../snapshots/web/goal-multi-turn-actions/session.jsonl) yields 2,098 events: 2,017 chunks (96.1%). Their unpacked JSONL lines occupy 332,647 of 379,225 event bytes (87.7%), while chunk packing reduces the committed file to 89,176 bytes and 182 storage rows, including 23 packed chunk rows. [`permission-policy-context`](../../../../snapshots/web/permission-policy-context/session.jsonl) yields 813 events: 746 chunks (91.8%) and 118,935 of 184,821 unpacked event bytes (64.4%); its packed file is 84,917 bytes and 123 storage rows, including 14 packed rows. These are tracked deterministic fixtures, not a production workload distribution, but they demonstrate why deleting chunks would reduce logical volume and why the existing packed-row layout already removes much of their JSON envelope cost.
Released-v1 repository fixtures established the original logical volume. Decoding the packed `goal-multi-turn-actions` generation yielded 2,098 events, including 2,017 chunks (96.1%); unpacked chunk lines occupied 332,647 of 379,225 event bytes, while the packed file used 89,176 bytes and 182 rows. The packed `permission-policy-context` generation yielded 813 events, including 746 chunks (91.8%); unpacked chunk lines occupied 118,935 of 184,821 event bytes, while the packed file used 84,917 bytes and 123 rows. These deterministic historical measurements explain why v2 embeds streams, but they are not a production workload distribution or a current-format size claim.
JSONL writes one Zstandard frame and fsync per durable append batch. Runtime files do not record former append boundaries, so fixture row counts cannot honestly be presented as fsync counts.
@@ -20,40 +20,40 @@ The scheduling bound is deterministic. With an immediately resolving sink, the f
## Decision
The JSONL provider exposes `writeBatchMaxDelayMs`, a positive integer no greater than Node's timer limit. Its default is `200`. The provider resolves the value at load and passes it to `PersistenceCoordinator`; the coordinator remains the single owner of batching behavior.
The fixed window is the JSONL provider's constant `LIVE_WRITE_BATCH_MAX_DELAY_MS` (200 ms), an internal scheduling policy rather than configuration: the backend's own session listeners route live events by id into the active write handle's buffer, so batching never crosses the package boundary ([handle note](2026-08-27-handle-based-session-persistence.md)).
Each live Session receives a package-private `SessionWriteBehind`. When its pending queue changes from empty to non-empty, the controller starts one fixed window. Later events join that batch without resetting the deadline: this is bounded coalescing, not debounce. When the deadline expires, the controller hands the complete pending prefix to the existing per-id serialization and `appendBatch` path. At most one write for a Session is active. Events admitted during that write form a new pending prefix with their own fixed deadline; if that deadline expires before the active write completes, the new prefix starts immediately after it.
Each active write handle owns its buffer directly. A routed event lands in the handle's pending array, and the first event of an idle buffer arms one fixed timer. Later events join that batch without resetting the deadline: this is bounded coalescing, not debounce. When the deadline expires, a single-flight drain persists the pending prefix through the handle's mutation chain, which already serializes it against explicit appends. Events admitted during a drain pass coalesce into the next chained batch, in order.
`writeBatchMaxDelayMs` bounds only the controller's intentional batching wait. Event-loop scheduling, initialization, an earlier serialized operation, and backend I/O can delay durable completion, so the option is not a hard fsync or crash-loss SLA.
The window bounds only the controller's intentional batching wait. Event-loop scheduling, initialization, an earlier serialized operation, and backend I/O can delay durable completion, so the option is not a hard fsync or crash-loss SLA.
`session/flush` cancels any remaining wait and becomes a shared quiescence barrier. It drains the active attempt and every event admitted while the barrier is running before it resolves. Session retirement and backend disposal use that same barrier, so lifecycle teardown never waits for the batching timer. The checkpoint policy continues to place mandatory barriers before model requests and top-level tool side effects.
`session/flush` cancels any remaining wait and becomes a shared quiescence barrier. It drains the active attempt and every event admitted while the barrier is running before it resolves. Session retirement (`session/disposed`), the handle's close, and backend teardown's close sweep use that same barrier, so lifecycle teardown never waits for the batching timer. The checkpoint policy continues to place mandatory barriers before model requests and top-level tool side effects.
Every event remains durable in its original order and shape. The controller copies each event on admission; no `assistant/chunk`, `seq`, `time`, surface metadata, or storage record is removed or rewritten. JSONL can therefore encode more events in one append frame without changing its on-disk format.
Every admitted event remains durable in its original order and representation. The controller copies each event on admission; batching removes or rewrites no sequence, timestamp, surface metadata, embedded Assistant stream, or storage record. JSONL can therefore encode more events in one append frame without changing the Session format.
A failed background append restores its complete batch before any newer pending events, reports the failure once, and pauses automatic retry. The next newly admitted event opens a fresh fixed window; an explicit flush, retirement, or disposal retries immediately and surfaces a repeated failure to its caller. This avoids a timer-driven failure loop while preserving the existing recoverable flush boundary.
A failed background drain retains its complete batch in order ahead of newer pending events, reports the failure once, and pauses the automatic timer. The next explicit drain — a `session/flush` barrier, service-level `flush()`, or close — retries immediately and surfaces a repeated failure to its caller. This avoids a timer-driven failure loop while preserving the existing recoverable flush boundary.
This decision supersedes only the immediate scheduling cadence in [Collapse live persistence into one flush controller](../simplification/2026-07-23-collapse-persistence-flush-state.md). That note remains authoritative for one controller per live Session, retained failed batches, per-id serialization, retirement, and quiescent disposal. The [shared persistence coordinator](2026-06-18-shared-persistence-write-coordinator.md) remains the owner of the backend hook boundary.
This decision supersedes only the immediate scheduling cadence in [Collapse live persistence into one flush controller](../simplification/2026-07-23-collapse-persistence-flush-state.md). That note remains authoritative for one buffer owner per live Session, retained failed batches, retirement, and quiescent disposal. The coordinator and the separate write-behind controller that first hosted this behavior are deleted; the buffer, timer, and drain live on the provider's handle, and the [handle-based seam](2026-08-27-handle-based-session-persistence.md) owns the storage boundary they write through.
## Alternatives considered
**Do not persist streaming chunk events.** Rejected here: it changes the event-sourced authority and recovery semantics rather than only physical write cadence. The existing [assembled-message rejection](../../rejected/simplification/2026-06-20-assembled-assistant-messages-only.md) remains the guardrail until a no-information-loss replacement defines replay, fork, cited source-event links, sequence, and crash behavior independently. The [packed-row decision](2026-07-26-packed-chunk-rows-by-default.md) remains the complementary JSONL storage-size optimization.
**Use one settlement per Assistant attempt instead of batching writes.** The [v2 Assistant stream decision](2026-09-01-v2-embedded-assistant-streams.md) provides that no-information-loss event model and reduces Assistant event cardinality. It does not replace bounded batching for other adjacent events, historical-generation publication, or providers with the same append interface.
**Write only at semantic checkpoints.** Rejected: it maximizes batching but makes the ordinary crash-loss window depend on a separately mounted policy. Bounded background writes preserve progress between checkpoints while mandatory flushes keep their stronger ordering contract.
**Debounce from the latest event.** Rejected: a continuously streaming response could postpone its first write indefinitely. A fixed window from the first pending event provides a real upper bound on intentional coalescing wait.
**Implement the timer inside JSONL.** Rejected: scheduling, failure retention, flush races, and teardown are provider-neutral lifecycle concerns that belong in `PersistenceCoordinator`; an out-of-tree provider can reuse the same behavior.
**A shared provider-neutral controller component.** Rejected after one iteration shipped it: the handle's mutation chain already serializes writes, so a separate controller duplicated that ordering machinery. Each provider implements the buffer on its own handle, and the shared live-write contract suite pins the equivalent observable behavior for any provider.
## Verification
The controller tests use a fake clock to prove the fixed, non-resetting 200 ms window; immediate and shared flush barriers; events admitted during a barrier; an over-budget tail behind an active write; ordered failure retention; paused automatic retry; and explicit retry of an overlapping background failure. Coordinator tests run the controller through Session notifications, retirement, collision reclamation, and teardown. The JSONL suite retains storage-format, recovery, and shared persistence-contract coverage.
The shared live-write contract suite (`runLiveWritePathContract`) uses a fake clock to prove the fixed, non-resetting 200 ms window; the `session/flush` barrier and its loud failure surfacing; ordered failure retention with exactly-once recovery; the service-level `flush()` sweep with per-session failure aggregation; and the disposed/close/teardown drains. The JSONL suite retains its storage-format, recovery, and shared persistence-contract coverage.
## Consequences
High-frequency event bursts normally produce fewer durable append operations while preserving the exact logical event count. The reduction depends on arrival rate and backend latency: a burst inside one 200 ms window becomes one batch, while mandatory flushes and sparse events can still produce small batches.
High-frequency event bursts normally produce fewer durable append operations while preserving the exact admitted event sequence. The reduction depends on arrival rate and backend latency: a burst inside one 200 ms window becomes one batch, while mandatory flushes and sparse events can still produce small batches.
This decision does not cap pending event count or bytes behind a slow provider, and it does not reduce the decoded logical log. A demonstrated memory bound or logical-retention policy would require its own failure and replay contract rather than another hidden timer rule.
An admitted event can remain only in memory during the configured window, and then while scheduling or backend work is outstanding. Deployments choose a smaller value for a narrower ordinary loss window or a larger value for stronger batching. Explicit durability boundaries remain unchanged and bypass the wait.
An admitted event can remain only in memory during the fixed window, and then while scheduling or backend work is outstanding. Explicit durability boundaries remain unchanged and bypass the wait.
The deep module gives the timer, active write, pending prefix, retry pause, and barrier one owner. `PersistenceCoordinator` retains initialization and identity serialization; the provider retains only durable storage primitives.`SESSION_FORMAT_VERSION` remains unchanged.
The handle gives the timer, active drain, pending prefix, retry pause, and barrier one owner; the backend's listeners own routing and lifecycle-driven drains. Batching itself never changes`SESSION_FORMAT_VERSION`.
@@ -51,7 +51,7 @@ Each `(kind, id)` has at most one start Match. A second start fails immediately;
#### `match(event)`
`match(event)` reads only the current `SessionEventLike` and returns `{ id, role: 'start' | 'update' }` or `null`. It cannot access a Context, history, a Reader, a Location, or the view envelope. A `chunkrow/*` event can only be an update; the Assembler rejects it as a start, and `start()` receives a `ConversationStartMatch` containing a standard`SessionEvent`.
`match(event)` reads only the current `SessionEventLike` and returns `{ id, role: 'start' | 'update' }` or `null`. It cannot access a Context, history, a Reader, a Location, or the view envelope. A Client-only `assistant/live-chunk` event can only be an update; the Assembler rejects every transient start, and `start()` receives a `ConversationStartMatch` containing a durable`SessionEvent`.
This restriction makes one scalar event or packed run's routing cost depend only on the number of registered Definitions. The Assembler never scans a Definition's historical Contexts to decide which one owns an update.
@@ -110,7 +110,7 @@ Dependencies point strictly from earlier starts to later starts, so transitive r
#### `update(context, match)`
`update()` handles a post-start scalar or packed Match that `match()` has already routed exactly to the current `(kind, id)`. It does not decide which Context owns the input. A Definition that consumes Assistant deltas folds each matching `chunkrow/*` value as one batch without constructing member events.
`update()` handles a post-start durable or transient Match that `match()` has already routed exactly to the current `(kind, id)`. It does not decide which Context owns the input. An Assistant Definition folds each `assistant/live-chunk` update directly and expands an embedded `assistant/message` or `assistant/attempt` stream during history replay.
The Assembler invokes `update()` in ascending `seq` order. A live tail update can apply incrementally; any non-tail insertion, newly loaded start, or invalidated dependency causes a complete replay from `start()`.
@@ -125,16 +125,16 @@ The Assembler does not use State reference equality to decide publication or pro
| Return value | Behavior |
|---|---|
| `immediate` | Request a notification and flush in the current microtask |
| `animation-frame` | Coalesce high-frequency updates into materialization on the next frame |
| `animation-frame` | Coalesce high-frequency updates into materialization after three browser animation frames |
| `none` | Do not schedule a flush for this Match; retain its State and dirty marker |
Omitting `publication()` means `immediate`. Assistant token deltas and packed runs use `animation-frame`, invisible Inbox Contexts use `none`, and finals, dependency replays, and Location boundaries publish the latest result through an immediate path.
Every live delta within a frame still executes `update()`, while one historical packed run executes one batch `update()`. Only`buildViewNode()`, View Builder work, and React snapshot notification are coalesced; no fragments are lost.
Every live delta during the three-frame interval still executes `update()`, while one historical packed run executes one batch `update()`. Location-data publication,`buildViewNode()`, View Builder work, and React snapshot notification are coalesced; no fragments are lost. An immediate publication cancels a pending frame interval and flushes the latest State without delay.
#### `buildLocationData(context, scope)`
`buildLocationData()` lets a Definition publish a read-only value derived from its State onto an engine-owned Step or Turn without exposing another business's mutable State. The Assembler always materializes `step` before `turn`, so Turn-level aggregation can read Step data updated in the same flush; it calls `buildViewNode()` only after all Location data is ready.
`buildLocationData()` lets a Definition publish a read-only value derived from its State onto an engine-owned Step or Turn without exposing another business's mutable State. The Assembler passes the preceding publication back to its owner, which returns that exact value when its business data is unchanged. The Assembler always materializes `step` before `turn`, so Turn-level aggregation can read Step data updated in the same flush; it calls `buildViewNode()` only after all Location data is ready.
A Definition receives the `step` and `turn` scopes separately and may return one value or `null` in either phase. A value must identify the exact turn/step coordinates and use the Definition's `kind` as its key. The Assembler owns replacement and removal and rejects another Context that claims the same Location key.
@@ -262,7 +262,7 @@ Page size, record packing, the number of history loads, and RAF coalescing affec
| Next-step Inbox / `inbox-next-step` | Splice Event seq | Each `agent/inbox/spliced` targeting next-step | None | Append message IDs to persistent splice state; materialize once per claim and expose the shared current claimed batch to Message |
| Message / `input-message` | Message ID | Append-surface `user/message` | None | Use source for a context message, or read the nearest next-step Inbox to distinguish user from steering |
| Request Prompt / `request-prompt` | Header Event seq | Each `request/header` | None | Read the preceding Request Prompt through Reader, retain the full prompt state, and classify system/tool changes |
| Assistant / `assistant-step` | `turn:step` | `step/start` | Scalar or packed`assistant/chunk`, final`assistant/message`, and same-step Retry | Aggregate blocks, usage, first-token time, final evidence, and retry-hidden state, then publish same-key Step data |
| Assistant / `assistant-step` | `turn:step` | `step/start` | Live`assistant/live-chunk`, durable`assistant/message` or `assistant/attempt`, and same-step Retry | Aggregate blocks, usage, first-token time, settlement evidence, and retry-hidden state, then publish same-key Step data |
| Tool / `tool-call` | Root call ID | Root `tool/call` | Root result and Code Dispatch start/result | Aggregate the root, children, and parent Map; Dispatch Events route exactly through `rootCallId` |
| Command / `command` | Command ID | `command/run` | `command/done` and compact lifecycle/checkpoint Events carrying a source command ID | Aggregate command outcome and manual-compaction evidence |
| Automatic Compaction / `compaction` | Compaction ID | `compaction/start` without a source command ID | Summary, end, and replacement checkpoint | Aggregate summary/checkpoint; sufficient checkpoint evidence supports fallback without a start |
@@ -315,21 +315,21 @@ The shell synchronously resolves the persisted selection when a Session binding
Ordinary prepend and append flushes call `apply({ upserts, timeline })` only for active targets. Complete window replacement and Registry rebuild call `replace()` only for active targets. Unsubscription does not remove a target, so returning to an opened View does not rebuild it.
[`ChatSnapshotBuilder`](../../../../packages/client/ui-chat/src/client/conversation-nodes/chat-snapshot-builder.ts) maintains `order`, a keyed `nodes` store, the turn/step `locations` index, `timeline`, and the `legacy` slice used by StatsLine and mirrored into top-level public compatibility fields.
[`ChatSnapshotBuilder`](../../../../packages/client/ui-chat/src/client/conversation-nodes/chat-snapshot-builder.ts) maintains `order`, a keyed `nodes` store with identity-stable Node and Turn-process sources, the turn/step `locations` index, `timeline`, and the `legacy` slice used by StatsLine and mirrored into top-level public compatibility fields.
Only a new key or a change to `anchorSeq`, visibility, or Location identity makes a Chat update structural. An ordinary content change does not rebuild `order`; the keyed Node store replaces only that key's value.
Only a new key or a change to `anchorSeq`, visibility, or Location identity makes a Chat update structural. An ordinary content change does not rebuild `order`; the keyed Node store replaces that key's value and publishes only its source. The Turn-process projector recalculates cross-Node presentation only for a Turn whose structure, specification, or status changed, then publishes only that Turn's process sources.
For a structural change, the Builder computes visible order from current store values and reuses unchanged index arrays by reference. Prepend may add earlier history keys, append may add a key at the tail or its business anchor, and ordering never renames existing keys.
[`ChatView`](../../../../packages/client/ui-chat/src/client/chat/ChatView.tsx) only traverses `order`. Each [`ChatNodeSeat`](../../../../packages/client/ui-chat/src/client/chat/ChatNodeSeat.tsx) remains in the same parent list under its Context key and dispatches the `'conversation.chat.node'` keyed slot by `node.kind`.
[`ChatView`](../../../../packages/client/ui-chat/src/client/chat/ChatView.tsx) only traverses `order` and resolves the two stable sources for each key. Each [`ChatNodeSeat`](../../../../packages/client/ui-chat/src/client/chat/ChatNodeSeat.tsx) remains in the same parent list under its Context key, subscribes only to its Node and Turn-process sources, and dispatches the `'conversation.chat.node'` keyed slot by `node.kind`.
[`ChatNodeDataMap`](../../../../packages/client/ui-chat/src/client/contract/chat-nodes.ts) is a declaration-merged renderer payload registry. Each business module registers its own Definition and keyed renderer; `registerConversationNodes()` and `registerChatNodeRenderers()` only assemble those independent contributions and do not interpret business through a closed union or central switch. Built-ins live in `ui-chat`, and this type and registration boundary allows a business to move into an independent package without changing the Chat dispatcher.
The Chat entry in `conversation.view` registers `ChatNodeTurnDataInjected` once when it declares the `conversation.chat.node` child slot. `ChatNodeSeat` passes only the stable Node key as `hookContext`; the Slot renderer combines that key with `useSession` from the official standard props to construct`useTurnData(businessKey)`. Every keyed Chat renderer therefore reads strongly typed, read-only data from its own Node's Turn, and the Assistant renderer has no special injection authority.
The Chat entry in `conversation.view` registers `ChatNodeTurnDataInjected` once when it declares the `conversation.chat.node` child slot. `ChatNodeSeat` passes the Node's stable Turn data store as `hookContext`; the Slot renderer binds`useTurnData(businessKey)` directly to that store. Every keyed Chat renderer therefore reads strongly typed, read-only data from its own Node's Turn, and the Assistant renderer has no special injection authority.
Slot-level contextual Hooks and entry-owned `inject.hooks` remain independent paths. The latter continues to bind only registration-owned Observables. The former caches definitions by stable slot-inject-face identity and binds its factory and Hook per stable render occurrence. The selector inside `useTurnData()` returns only the current Node's`turn.data.get(key)`, so selector equality filters unrelated Session publications.
Slot-level contextual Hooks and entry-owned `inject.hooks` remain independent paths. The latter continues to bind only registration-owned Observables. The former caches definitions by stable slot-inject-face identity and binds its factory and Hook per stable render occurrence. `useTurnData()` subscribes to`turn.data.source(key)`, so another Location-data key or Session snapshot publication does not notify it.
The standard `useSession` remains available to every session-scoped slot renderer. `useTurnData()` narrows the common read path rather than acting as a permission sandbox. Whole-window statistics or arbitrary object indexes may still read the Session snapshot explicitly, but they are not modeled as current-Node Turn data.
The standard `useSession` remains available to every session-scoped slot renderer, although `ChatNodeSeat` needs neither it nor aggregate `useChat`. `useTurnData()` narrows the common read path rather than acting as a permission sandbox. Whole-window statistics or arbitrary object indexes may still read the Session snapshot explicitly, but they are not modeled as current-Node Turn data.
Assistant streaming to final and Tool running to settled stay in one Seat while updating its data and necessary ordering properties. Settlement therefore does not reset component-local State through a parent move.
**Define a reverse State fold for backward history scanning.** Rejected: every business would maintain two inverse algorithms, and deletion, non-invertible aggregation, and cross-Context dependencies would be difficult to keep equivalent. Ordered Matches followed by forward replay from start preserve one business meaning.
**Add a separate chunk-run matcher and update lifecycle.** Rejected: a second Definition path would duplicate dispatch, replay, publication, and Context types. `ChunkRowEvent` uses the existing `match(event)` and `update(context, match)` lifecycle while making packed handling explicit through its `chunkrow/*` discriminant.
**Add a separate live-stream matcher and update lifecycle.** Rejected: a second Definition path would duplicate dispatch, replay, publication, and Context types. Client-only `assistant/live-chunk` and durable settlements use the existing `match(event)` and `update(context, match)` lifecycle; only the event discriminator and stream expansion differ.
**Make Inbox a first-class engine concept or one window-wide Context.** Rejected: Inbox is ordinary business State and does not belong in the generic engine. Per-splice instantaneous State plus a strictly backward Reader supports prepend, append, and Message lookup together.
**Let a Location-data consumer read the provider's Context State directly.** Rejected: the consumer would depend on another business's mutable internal shape and could not express which Turn/Step owns the value. Declaration-merged data maps expose only the provider-selected read-only value and engine-owned coordinates.
**Cache every Definition's Location data by State identity.** Rejected because a Definition may mutate and return the same State object, and its Location data may also depend on Match Locations or values published by another Definition. Each Definition instead decides whether its business value changed and returns the preceding publication unchanged when it did not.
**Add generic `end()`, prepared, or window-reset lifecycles.** Rejected: businesses have different completion conditions, and a pagination gap is not a business lifecycle. Business Events update State, Location close triggers replay/build, and Reader dependencies own pagination invalidation.
**Reuse one Event Definition across Chat and Trajectory by branching in `buildViewNode(target)`.** Rejected: the views require different business State and intermediate records, so a shared Definition would make each package carry the other's conditions and payloads. Separate target-owned Definitions keep those choices local while sharing the Assembler's ingestion and lifecycle contracts.
@@ -412,11 +414,11 @@ Initial tail, older prepend, and live append share one set of Context invariants
Append does not scan historical Contexts; prepend replays only Contexts whose Matches, Locations, or Reader answers actually changed. A structural Chat change may still recompute visible order and indexes, but does not rerun unrelated business folds or replace unchanged Node identity.
Separating State updates from publication cadence folds every live Assistant delta and each historical packed run while materializing at most once per animation frame. Step or Turn close and final Events can immediately publish the latest State.
Separating State updates from publication cadence folds every live Assistant delta and each historical packed run while materializing at most once per three animation frames. The Assistant view reads the same projection that the preceding Step Location phase installed. Turn Process returns its existing open data and Node for continuing Assistant chunks without deriving or encoding them again, and Turn Tail defers its complete-Match scan until `turn/end`. Step or Turn close and final Events immediately publish the latest State.
An inactive target retains Definition State and a target Context index but no builder, materialized Nodes, or snapshot. The mounted built-in or third-party View activates its own target through normal subscription; previously opened targets continue receiving incremental updates.
Steps and Turns are stable homes for cross-business aggregates. Turn Tail and Deliverables derive their values without renderer scans of global Nodes; slot-level `useTurnData()` narrows common reads to the current Node's Turn and uses selector equality to isolate unrelated updates.
Steps and Turns are stable homes for cross-business aggregates. Turn Tail and Deliverables derive their values without renderer scans of global Nodes; slot-level `useTurnData()` narrows common reads to the current Node's Turn, and keyed Location sources isolate unrelated updates.
Inbox Context retention grows with splice count and claimed message count rather than their cumulative prefixes. This removes duplicate state growth but does not deduplicate message content in durable Session events or bound the loaded event window.
@@ -6,23 +6,23 @@ English | [中文](2026-08-10-cancelled-stream-prefix-finalize.zh.md)
## Problem
A cancelled stream can leave `assistant/chunk` events that clients continue rendering while `deriveMessages()` excludes them because no `assistant/message` records the delivered prefix. A follow-up such as "expand on your second point" then lacks text the user read, and a fork at the cancelled turn inherits the same gap.
A cancelled stream can leave transientchunks that clients have rendered while `deriveMessages()` excludes them because no `assistant/message` records the delivered prefix. A follow-up such as "expand on your second point" then lacks text the user read, and a fork at the cancelled turn inherits the same gap.
The model history must contain assistant content that remains visible to the user after cancellation.
## Decision
`ReactLoopAgent.step()` catches cancellation while consuming a model stream, when its `BlockAssembler`, logged chunk seqs, and provider route identify the delivered prefix. It appends that prefix as the step's `assistant/message` with `interrupted: true`, `surfaceOp: 'append'`, and `sourceEventSeqs` containing exactly the logged chunks. The append precedes`step/end` and the aborted `turn/end`.
`ReactLoopAgent.step()` catches cancellation while consuming a model stream, when its `BlockAssembler`, compact stream accumulator, and provider route identify the delivered prefix. It appends that prefix as the step's `assistant/message` with `interrupted: true`, `surfaceOp: 'append'`, and the exact embedded timed stream. The append precedes the committed `agent/assistant-stream` end frame,`step/end`, and the aborted `turn/end`.
`BlockAssembler.interruptedBlocks()` returns closed and open `text` and `reasoning` blocks with non-whitespace content in stream order. It omits tool calls because interruption precedes dispatch and no real result exists; it also omits empty blocks and open unknown block types. An empty result appends no assistant message. Provider `error` and `aborted` finishes leave the stream-consumption scope before `agent/request-error`, so provider failures and cancellation during recovery commit no content from the failedrequest.
`BlockAssembler.interruptedBlocks()` returns closed and open `text` and `reasoning` blocks with non-whitespace content in stream order. It omits tool calls because interruption precedes dispatch and no real result exists; it also omits empty blocks and open unknown block types. An empty result appends `assistant/attempt` instead of a surface message. Provider `error` and `aborted` finishes also commit `assistant/attempt` before `agent/request-error`, so their streams remain durable without contributing failed-request content to model history.
Chat and Trajectory Conversation Definitions read `interrupted` from the durable message. Chat renders the Stopped marker, while Trajectory keeps the provider request in the error lifecycle after `step/end` and retains the durable result seq and provenance. Cancellation during tool execution follows the tool scheduler contract because the assistant message has already committed: started calls produce real results, and undispatched calls receive `ABORTED_BEFORE_DISPATCH` results.
Chat and Trajectory Conversation Definitions read `interrupted` from the durable message. Chat renders the Stopped marker, while Trajectory keeps the provider request in the error lifecycle after `step/end` and retains the durable result seq and provider information. Cancellation during tool execution follows the tool scheduler contract because the assistant message has already committed: started calls produce real results, and undispatched calls receive `ABORTED_BEFORE_DISPATCH` results.
## Alternatives considered
**Always discard the prefix.** This avoids a new durable marker but makes every cancel-then-follow-up and fork omit assistant content that remains visible to the user.
**Assemble the prefix from chunks during projection.** `deriveMessages()` and client Conversation Definitions would each need interruption assembly rules, and the log would have no authoritative assistant message for the prefix. This also expands model history beyond the three `SurfaceEventType` events.
**Assemble the prefix from the embedded attempt during projection.** `deriveMessages()` and Client Conversation Definitions would each need interruption assembly rules, and the log would have no authoritative surface message for the prefix. This also expands model history beyond the three `SurfaceEventType` events.
**Retain complete tool calls with synthetic aborted results.** These calls never dispatched, so synthetic results would claim an execution outcome that did not occur and add content the user did not receive as a tool result.
@@ -32,8 +32,8 @@ Chat and Trajectory Conversation Definitions read `interrupted` from the durable
Post-cancel follow-ups and forks include the delivered prefix. The ACP bridge drains ordered assistant output before settling the prompt, so the final `agent_message_chunk` update precedes the cancelled stop reason.
Terminal provider errors still discard their streamed prefix. That asymmetry remains because an error turn ends without the user's cancellation decision and requires its own retention policy.
Terminal provider errors retain their stream in `assistant/attempt` but keep its content out of model history. Only the user's cancellation decision turns visible delivered text into an interrupted surface message.
## Testing
`packages/core/agent-loop/tests/cancel.spec.ts` covers content, cited seqs, event order, next-request parity, reasoning-only output, tool-call omission, recovery cancellation, and the empty-prefix case. `packages/llm/llm/tests/assembler.spec.ts` covers `interruptedBlocks()`. `packages/client/ui-chat/tests/conversation-node-definitions.client.spec.ts` and `packages/client/ui-trajectory/tests/conversation-definitions.client.spec.ts` cover both client projections. The keyless `cancel` ACP snapshot and `goal-round-driver` goal snapshot cover assembled applications.
`packages/core/agent-loop/tests/cancel.spec.ts` covers content, embedded streams, event order, next-request parity, reasoning-only output, tool-call omission, recovery cancellation, and the empty-prefix attempt. `packages/llm/llm/tests/assembler.spec.ts` covers `interruptedBlocks()`. `packages/client/ui-chat/tests/conversation-node-definitions.client.spec.ts` and `packages/client/ui-trajectory/tests/conversation-definitions.client.spec.ts` cover both Client projections. The keyless `cancel` ACP snapshot and `goal-round-driver` goal snapshot cover assembled applications.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Yichen Jiang