The child log ledger encodes an admitted entry to UTF-8 once to charge its
serialized cost, so a maxLogBytes approaching addressSpaceMb lets a legitimate
near-budget log entry breach RLIMIT_AS and die as worker-exit instead of
truncating. Two runtime fixes were tried and both traded one resource bound for
another: an exact serialized-cost check is either a full encode (the allocation
being avoided) or a per-character Python loop that burns the CPU budget (a 10 MB
write hits SIGXCPU under cpuSeconds:1). The breach is a property of the
maxLogBytes/addressSpaceMb pair, not any write, so reject the incompatible pair
at load — maxLogBytes must stay within one eighth of the addressSpaceMb byte
count — and revert _LogStream to its original character-count buffering, which
is memory-safe once the budget fits the address space. The check runs on every
platform since the incompatibility is a config-value property, not a runtime one.
Replace the child-flood regression tests (which asserted the reverted runtime
behavior) with a load-rejection test. The host-side accrueStrayCost UTF-8
per-lead validation and its tests are unaffected. Update the note and zh pair.
accrueStrayCost accepted any 0x80-0xBF continuation, so a CESU-8 surrogate
(ED A0 80) or overlong (E0 80 80) — structurally well-formed but illegal, and
as cheap to flood as 0xFF — was charged its structural width 3 while
toString('utf8') renders each byte as its own U+FFFD (cost 9). Validate each
lead's first-continuation range (WHATWG E0/ED/F0/F4 bounds) and charge 3 per
byte of any sequence outside it, folding a broken prefix to one U+FFFD.
The child _LogStream newline path had the same char-vs-serialized gap the
newline-free trigger had: its per-line fit checks (first reconstructed line and
each subsequent line) compared character count against the serialized-byte
budget, so a control-char line passed and _logs.push encoded it whole, breaching
RLIMIT_AS. Route every check through _fragment_cost_upto, which sums per-char
costs from _json_char_cost over a start/end sub-range without slicing or
encoding and stops at the budget.
Decline arrival-order stray flushing: the two pipes' data events interleave
nondeterministically and logs carries no cross-pipe ordering guarantee, so a
fixed drain order is as valid as any and an arrival-tick branch could not be
covered without a flaky test.
Add CESU-8/overlong, newline-path-flood, and all-lead-class reassembly
regression tests; fix the note's now-inaccurate CESU/illegal-byte claims and a
fixture byte-count comment; sync the zh pair.
The prior child-flush fix measured each fragment with chunk.encode('utf-8'),
which copies the whole write — under a tight addressSpaceMb a single 340 MiB
write died on that encode (the exact allocation _push_bounded_prefix exists to
avoid), and re-scanning the whole pending list per write was quadratic under a
daemon-thread flood (the concurrent-write test timed out at 28s). Compute each
fragment's serialized cost with _fragment_cost_upto, which walks the str via a
new _json_char_cost (code point to escaped width, no encode) and stops once the
running total passes the budget, and accumulate it into _pending_cost once per
write. The early-flush trigger reads that accumulator: still charges control
chars their full serialized width (a NUL is 6 bytes), but never encodes a whole
write and never re-scans the buffer, so the 340 MiB single-write and
daemon-thread tests pass alongside the NUL-flood one.
Rework the child NUL-flood regression to write in 1 MiB chunks under a 512 MiB
address space so its own argument construction is not the allocation under test.
The host stray-capture cost function charged illegal UTF-8 bytes (0x80-0xC1,
0xF5-0xFF, and orphaned multibyte leads) the raw 1, but toString('utf8')
renders each as U+FFFD (3 serialized bytes). A b"\xff" flood was undercounted
threefold, so the residual grew to a full budget's worth of raw bytes before
flushing and, near a large maxLogBytes, expanded toward a ~1 GiB peak in the
flush's concat plus toString. Replace serializedBufferCost with accrueStrayCost,
a cross-chunk UTF-8 walker that charges each byte its decoded serialized width;
carry its sequence state on each StrayBuffer.
The child _LogStream had the same-family bug: its early-flush trigger compared
_pending_chars (character count) against remaining (a serialized-byte budget),
so a 30M-NUL newline-free flood stayed under a 50 MB char trigger yet encoded to
~180 MB at settlement, breaching RLIMIT_AS as worker-exit. Track _pending_cost
via the _JSON_BYTE_COST table and trigger on it; keep _pending_chars for the
char-based slice bounds.
Correct the note's surrogate claim (only the string-walking jsonStringCostUpTo
charges a lone surrogate six bytes; the byte walker never sees one). Shrink the
post-truncation fixture below PIPE_BUF for a deterministic single callback. List
the shared stdout/stderr budget as a third honest fail-before exception
(cross-pipe arrival timing is nondeterministic). Add illegal-UTF-8,
broken-multibyte, and child-log-flood regression tests; sync the zh pair.
Wrap spawn and the fd-3 narrowing so a synchronous throw (ENAMETOOLONG on
an over-PATH_MAX pythonBin, EMFILE) removes the run's staging directory and
resolves the same worker-exit class as the async error event, instead of
rejecting run() and leaking the directory.
Aggregate native stdout/stderr by real newline rather than by Node data
chunk: logs entries are joined with "\n" downstream, so a newline-free
write larger than one pipe read no longer reads back with spurious breaks.
The ledger still bounds a newline-free flood.
Track a running scan offset in both frame readers so a large frame
accumulated across chunks is scanned once, not re-scanned from 0 per chunk.
Reword the deadline hard-bound v8-ignore to state its real environment
dependence (PID-1-doesn't-reap container, zombie survivor) and cross-ref
the note's rejected signal-0 alternative; fix settle comments that quoted
the pre-qualification teardown contract; document the capMessage vs
_cap_message billing split on both sides; guard the dispose-after-resolve
heartbeat assertion against a vacuous 0===0 pass; reuse
_TRUNCATION_MARKER_BYTES; note the abandoned-call pending-entry bound.
Update the Agent Note Decision/Testing/Alternatives/Consequences for the
above and record the confirmed-empty finalize as a second honest
fail-before exception; sync the zh pair.
The reap-poll deadline arm sent SIGKILL then finalized immediately, declaring
quiescence on mere signal delivery while the group was still dying. It now keeps
polling for the group to actually empty (bounded by one more reap margin) after
its self-sent SIGKILL, so `finished` resolves only on a confirmed-empty group.
ProtocolChannel.read_frame read the boot/run handshake frames through
FileIO.readline() on the unbuffered fd — one os.read(1) per byte, so a
multi-megabyte program burned CPU (RLIMIT_CPU already in force for the run frame)
in millions of syscalls before ast.parse. It now reads in chunks into the same
_pending buffer the async reader uses; the wrapping os.fdopen is gone. read_frame
is this PR's own code (e7f22ed3), not the protocol layer. The chunked read is a
syscall-count improvement with no cross-platform-deterministic failure to assert,
noted as such in the Agent Note.
Raising maxValueBytes' load bound to ceiling-envelope assumed both budgets are
metered in serialized (JSON-escaped) bytes, which held for completion values and
logs but not the diagnostic: _cap_message capped by raw UTF-8, so a control-heavy
message near maxValueBytes could serialize sixfold and breach the fd-3 frame
ceiling — the silent worker-exit inversion the load check prevents. _cap_message
now accumulates per-byte serialized cost (new _JSON_BYTE_COST table) and cuts the
prefix that fits. Also reword the host SIGXCPU timeout message to name cpuSeconds
as the configured ceiling rather than a budget a stricter inherited RLIMIT_CPU
soft may undercut. Adds a control-heavy-diagnostic regression test.
The class docstring still credited the GIL plus per-frame PIPE_BUF atomicity for
serializing writes, which _write_lock's full-write loop already superseded. State
the current contract (writers serialized by _write_lock around a full-write loop)
and drop the double blank line under the binding-replies note heading.
The settlement-time CPU recheck compared spent CPU against the configured
cpuSeconds, but _clamped may have lowered the effective soft limit to a stricter
inherited value. A program that traps SIGXCPU, burns past the inherited soft,
and returns inside the soft-to-hard gap was checked against the configured value
and falsely reported successful, bypassing the inherited limit. The recheck now
uses the clamped cpu_soft. Adds a regression test that inherits a 1s soft CPU
limit and asserts a SIGXCPU-trapping over-burn is a timeout, not a success.
A binding called from a worker thread records that thread's loop for its reply.
If the thread finished and closed its loop before the host reply arrived,
_pump_replies' call_soon_threadsafe onto the closed loop raises RuntimeError;
unguarded, that ends the pump task and strands every later reply. Wrap the
schedule in a try/except that drops the moot reply (nothing awaits it) and keeps
the pump serving.
Two further review findings on the CPython backend:
- The grace-window SIGKILL timer was left armed after settlement, so on a
normal completion a kill(-pid) could fire up to graceMs later and strike a
recycled pgid once the kernel reused the leader's pid. settle() now clears
the timer the moment the process group is confirmed empty (the normal path
and when the poll sees the survivor gone), bounding the reuse window to the
genuine-survivor case where the group cannot be empty to reuse.
- _clamped bounded rlimits by the inherited hard limit only, silently raising
an inherited soft limit stricter than the request (loosening RLIMIT_AS or
deferring RLIMIT_CPU SIGXCPU). It now clamps each side against its own
inherited counterpart and pins soft under hard, keeping the strictest of
configured and inherited. Adds an inherited-soft-limit regression test.
Agent Note expanded to seven fixes with the two new rejected alternatives;
zh pair re-recorded.
Two review findings on the CPython backend:
- Disposal could return while a same-group descendant that ignores SIGTERM
but releases the inherited pipes was still alive: the leader's close fired
and the previous fix relied on an unref'd SIGKILL timer that a short-lived
host never fires, reparenting the survivor to init. settle() now withholds
the run's finished promise on a ref'd process-group poll until the SIGKILL
has emptied the group (bounded by graceMs + margin, zero-cost when already
empty), so teardown's "await each child's exit" holds.
- A binding called from a model worker thread via asyncio.run created its
reply Future on that thread's loop, but _pump_replies completed it directly
from the main loop; asyncio.Future is not thread-safe across loops, so the
call hung to the wall clock. Replies now complete via the owning loop's
call_soon_threadsafe, and a lock serializes the id claim/write/advance.
Tests: the same-group reap case now asserts a heartbeat file stops (robust
whether the killed descendant is reaped or a zombie, so it holds where PID 1
does not wait() orphans); a cross-loop case runs a binding from a worker
thread and asserts the reply round-trips instead of timing out. Agent Note
expanded to all six fixes with rejected alternatives; zh pair re-recorded.
Land the PythonCodeRuntime implementation on top of the fd-3 protocol
seam: python3 -I per run, binding namespace over fd 3, RLIMIT_CPU/AS,
wall-clock timer, and SIGTERM->grace->SIGKILL process-group teardown,
with the real-subprocess integration suite.
Fixes three defects surfaced on the source PR's review before they ship:
- boot-write failure resolved a worker-exit through finish()/settle()
that read wallTimer/onAbort/live in their TDZ, rejecting run() instead;
the boot write now runs after those bindings and the v8-ignore that hid
the branch is removed.
- log capture serialized against settlement with no lock while model
daemon threads keep writing; LogBuffer now owns one shared re-entrant
lock taken by write/flush_line/push.
- the fd-3 line residual was a subarray view pinning the whole joined
frame; it is copied into a right-sized Buffer via detachResidual so
pendingBytes measures what is retained.
- Cover the log-frame `truncated` rebuild branch: assert a literal-true flag
rides along and any other value (1, string, false) is dropped, closing the
protocol.ts branch the coverage gate flagged.
- Correct encodeJsonPlain's JSDoc: it matches compact JSON.stringify EXCEPT on
a beyond-safe-range integral double, where it emits the exact BigInt digits
(`...846976`) rather than the rounded `...847000` — the divergence the
"emits exact digits" test pins.
- Declare py/protocol.py's `global`-bearing frames (Namespace, CallMessage)
with functional TypedDict syntax so they carry the real wire key instead of
a `global_` attribute the wire never sends, and split optional-field messages
(Namespace/LogMessage/DoneMessage) into a required base plus a total=False
subclass so `type` and other required fields cannot be dropped. Widen
HostToChild to include the boot and run frames the host sends before replies.
- Reword the mirror e2e's py/ directory assertion to describe the source-tree
layout it actually checks.
Introduce @deepseek-ai/dsh-code-runtime-python with the versionless
JSON-lines protocol between the Node host and the CPython subprocess:
the host-side hostile-frame codec (validateChildFrame, encodeJsonPlain,
checkDoneValue, hasUnsafeIntegerToken, hasNonLosslessNumber,
logTruncationMarker) and the Python-side wire-vocabulary mirror
(py/protocol.py).
This is the protocol layer of the code-runtime-python stack, split from
#436 and based on the multi-language seam extension. The PythonCodeRuntime
implementation and its Python JSON codec land in the backend-core PR on
top of this branch.
Ship the minimal buildable package skeleton (package.json, tsconfig,
tsdown, barrel index, invariant companion, bilingual README) because the
workspace-constraint, coverage, and invariant-topology gates require the
package to exist and build the moment its directory does; the backend-core
PR extends those files rather than creating them.
Align py/protocol.py with src/protocol.ts (the round-12 review of #436
found LogMessage.truncated, DoneMessage.error.kind, and Namespace.errorClass
stale) and guard the two runtime-executed surfaces (PROTOCOL_FD and the log
truncation marker) with a real-python3 cross-language mirror e2e test.
Chinesezjc