The review's doc drift items: the orphan receive-ceiling JSDoc, the frame-ceiling
references in index.ts/bootstrap.py/tests, and the README's 'dropped, stranding
to the wall clock' phrasing (the run now settles as a worker-exit) are all
updated to the 64 MiB FRAME_PARSE_CAP_BYTES semantics; the README notes the
>64 MiB binding-argument residual as a worker-exit trip of the same cap. A
regression case resolves a basename pythonBin against a PATH whose first entry
is relative ('.') and asserts the absolute entry is used.
The review's remaining items:
- _done_with_value's rejection branch now caps the _check_done_value diagnostic
through _cap_message (a reason embedding a hostile class name could otherwise
push the done frame past the host's 64 MiB parse cap, misreporting an
invalid-output run as a worker-exit).
- The settlement note (en + zh) updates three stale facts (load bound is now
parse-cap minus envelope at 67108800; the sink goes directly through the
bound primitives); the fd-3 protocol note (en + zh) no longer claims the
package ships protocol without the runtime; FRAME_ENVELOPE_BYTES' JSDoc and
_cap_message's docstring follow the new bound.
Pairings re-recorded.
The review's remaining warning: the _run binding comment claimed the log sink
went 'through the bound send', contradicting the sink's actual direct use of the
bound encode+write primitives. The comment now states that; the settlement note
(en + zh) registers FRAME_PARSE_CAP_BYTES and the 65 MiB-frame regression case.
Pairing re-recorded.
Addresses the review's remaining two items:
- FRAME_PARSE_CAP_BYTES (64 MiB) drops an fd-3 frame whose raw length exceeds
it BEFORE toString/JSON.parse: the 256 MiB wire ceiling bounds the bytes, not
the decoded structure, and a compact wide frame near that ceiling could decode
to far more host memory. A regression test writes a 65 MiB log frame plus a
normal one and asserts the oversized frame is dropped while the trailing frame
still lands in logs (fail-before: without the cap the oversized text is parsed
and admitted, truncating the ledger so the trailing frame is dropped). The
forged-oversized lower-bound test's frame is reduced to stay under the cap
while still exercising the truncation path.
- The log sink writes through the def-time bound encode+write primitives (not
send_sync, whose body resolves _encode_json_plain and self.write_encoded at
call time), so a rebind cannot break a log frame.
The review's remaining functional item: send_sync's body resolves
_encode_json_plain (module global) and self.write_encoded (class attribute) at
call time, so a program rebinding either before the first binding call could
turn a legitimate call into an exception. dispatch now writes the call frame
through def-time bound write_encoded+_encode_json_plain, and the log sink goes
through the bound send; the dispatch rebind test also rebinds those two names
(verified fail-before by reverting to send_sync). The annotation test title
matches its assertion direction, and the note (en + zh) registers the
error-class constructor, dispatch primitives, and dont_inherit mechanisms.
Pairing re-recorded.
bootstrap.py imports from __future__ import annotations; compile(wrapped) was
inheriting that PEP 563 flag, stringifying the program's type annotations and
changing the semantics of a legal program that reads f.__annotations__ at
runtime. compile(..., dont_inherit=True) stops the leak; a regression test
defines an annotated function and asserts the annotation is the live int class,
verified fail-before by removing dont_inherit (the test turns red).
The review's remaining items:
- _make_error_class captures Exception and setattr as def-time defaults, so a
rebind of __main__.Exception/__main__.setattr cannot break the rejection
constructor.
- dispatch binds _lossless_json_violation, asyncio.get_event_loop, and the
channel's send method into _run locals before the program runs, so a rebind
cannot turn a legitimate binding call into an exception or a wall-clock
timeout.
- The note (en + zh) corrects the stdin coverage phrasing: d3f9f57f5's direct
EOF-observing case is the in-tree pin, not an approximation.
- Collapse two stray double blank lines in the test file.
Pairing re-recorded.
The review's completeness check found the def-time capture pattern was not yet
applied to every name the reply/settlement paths resolve at call time:
- _decode_json_plain now also captures isinstance/str/list.
- read_frame/read_frame_async capture len; read_frame_async captures
asyncio.get_event_loop.
- send_done uses _run's bound _str/_isinstance for its frame-shape check.
- The reply pump's frame reader is a bound method captured by _run BEFORE the
program runs and passed into _pump_replies, so a rebind of the class
attribute cannot redirect it.
The decode-rebind regression test still pins the _decode_json_plain rebind;
rebinding builtins (len/isinstance/list/str) in a test is not viable because
the Python runtime itself resolves them implicitly.
Addresses the review's two remaining items:
- The host closes the child's stdin write handle immediately after spawn. The
program is an async body that reads nothing from fd 0; a live pipe would hold
a host-side handle open past the run, so a setsid-escaped descendant
inheriting fd 0 could keep the host process from exiting even after the
closeDeadline forced settlement. The child (and any descendant) reads EOF on
fd 0 and no host handle survives.
- read_frame/read_frame_async bind their decode primitives (_decode_json_plain,
os.read, _READ_CHUNK_BYTES, bytes) as def-time default arguments, and
_decode_json_plain itself captures json.loads, its two regexes, and len the
same way, so a __main__ rebind cannot kill the reply pump and strand every
pending Future to the wall clock. _decode_json_plain and its regexes moved
before the ProtocolChannel class so the defaults resolve at class-definition
time. A regression test rebinds _decode_json_plain and asserts a binding reply
still round-trips.
Note (en + zh) registers both mechanisms; pairings re-recorded.
The review's remaining non-blocking suggestion: dispatch's call_failure(str(exc))
resolved the builtin str at call time, so a program rebinding __main__.str could
run a hostile callable when the binding-rejection message is formatted. Bind
_str into _run locals and use it in dispatch.
dispatch's call_failure and its except clause resolved the module globals at
call time, so a program rebinding __main__._BindingRejection = ValueError let
the internal marker type leak into model code. Bind _RuntimeError_cls and
_BindingRejection_cls into _run locals before the program runs (names distinct
from the module globals so the assignment RHS resolves the global, not an
unbound local); dispatch now uses the locals. A regression test rebinds
_BindingRejection and asserts a host rejection still surfaces as RuntimeError.
The sys.__stdout__ flush test now reconfigures the streams back to block
buffering (write_through=False) so the settlement drain path is what the case
pins — verified fail-before: binding the stream objects instead of their flush
methods turns the test red.
The settlement drain iterated the bound stream OBJECTS, which are not
callable — every _flush() raised TypeError and was swallowed by the loop's
except, so the drain never ran and only the -u flag carried the behavior.
Bind sys.__stdout__.flush/sys.__stderr__.flush (bound methods, capturing the
stream at binding time, immune to a later sys.__stdout__ rebind; None-guarded).
Verified by removing -u temporarily: the sys.__stdout__ regression test still
passes, so the drain is a genuine backstop, not a documented-but-dead layer.
Addresses the review's two carried warnings and the comment suggestion:
- Once the ledger truncates, every arm that marks it (admit()'s two ceilings and
the child-marker frame arm) now clears both stray pipes' buffered output
wholesale, so the end-path flushStray sees empty buffers instead of
concat+decoding doomed data near a 256 MiB maxLogBytes; captureStray's newline
loop re-checks the flag before re-retaining the residual.
- The child runs with -u (unbuffered), so sys.__stdout__/sys.__stderr__ writes
are visible to stray capture immediately; the settlement flush still drains
the original std streams before the done frame as a guard. A regression test
writes through sys.__stdout__/sys.__stderr__ without an explicit flush and
asserts both bytes land in logs. C-ext stdio remains an accepted residual,
recorded in the README Known Limitations (en + zh).
- The ledger-comment arithmetic now states the exact boundary (serializes to
exactly maxLogBytes; without the reserved byte it would be maxLogBytes + 1)
in both host and child.
Note (en + zh) registers the stray-clear and -u/settlement-drain mechanisms and
the new test; pairings re-recorded; corpus passes 1029.
Addresses the review's two code warnings and one suggestion:
- die_if_cpu_exhausted now restores SIG_DFL BEFORE unblocking SIGXCPU: a program
that installed a custom handler AND masked the signal would otherwise have
that pending handler run at the unblock (in model code, re-masking or raising)
and escape the re-raise; with SIG_DFL first the pending signal kills inside
the kernel with no bytecode window. A trap+mask combined regression test pins
it (the mask-only case was already covered).
- The constructor rejects budgets too small to honor: maxLogBytes must fit the
truncation marker plus the serialized outer-array envelope (floor 64), and
maxValueBytes must at least represent the smallest JSON completion (floor 4,
matching the worker backend). The exact-limit test moves to the 64 floor and
a rejection test pins the floors.
- The pthread_sigmask None-guard comment cites the real rationale (defensive
against stripped CPython builds; win32 is refused at construction), not the
unreachable Windows path.
Addresses the review's two remaining code warnings and the three suggestions:
- Log ledgers (host and child) start one byte below the budget, reserving the
serialized outer-array envelope (two brackets and n-1 commas over n entries'
separators); the exact-zero test moves to maxLogBytes 104 and a new exact-limit
case pins that maxLogBytes 5 admits ['a'] (5 bytes) while 4 truncates to the
marker alone.
- die_if_cpu_exhausted unblocks SIGXCPU (pthread_sigmask SIG_UNBLOCK, captured at
import, None-guarded for Windows) before re-delivering it, so a program that
masks SIGXCPU, burns past the soft limit, and returns is still classified as a
timeout; a regression test pins the masked path.
- ast.parse passes filename="<model>" so parse-time syntax diagnostics carry the
same source label as compile and runtime tracebacks; the syntax-error test
asserts the label.
- The NUL-escape test comments use the true six-byte JSON escape \u0000 instead
of the caret notation; the README Known Limitations (en + zh) records that
PID-reuse protection is inert on macOS; a combined-rebind regression test pins
BaseException plus the traceback reporter rebinding together.
Addresses the review's registration-text accuracy findings:
- _run binds _done_with_value into a local (done_with_value_bound) before the
program runs, closing the __main__._done_with_value = boom success-rewrite
vector; a regression test rebinds it and returns a legitimate value, asserting
the success survives.
- README (en + zh): the CPU-recheck bullet now states the recheck runs
unconditionally after the program returns (a pre-return overrun dies there as
a timeout) and the false-success window is only a trap-SIGXCPU program that
passes the recheck and overruns during the settlement flush/encode; the
encoder-deps residual rationale is replaced with the actual one (bash-equivalent
trust, verdict still delivered via the send_done fallback frame) and names the
now-bound entry; the t.join() deadlock bullet fixes the subject/object (the
main coroutine joins the worker, blocking the pump's main event loop).
- The portable-identifier-seam architecture note no longer claims the Python
backend does not exist.
- Settlement note (en + zh) registers the entry-name binding and the new test.
- All pairings re-recorded; corpus-wide verify-translation-pairing passes 1004.
A body-local X = X binding in _pump_replies is too late: _run reaches the
model's top-level statements (which run first, since there is no suspension
point between create_task and await __dsh_main__) before the pump's first step,
so a __main__.RuntimeError rebind there would be captured by the body local and
a closed-loop failure would escape the except, killing the pump. Bind
_RuntimeError, _BindingRejection, str, and bool as DEF-TIME default arguments of
_pump_replies (evaluated at import, before any model code runs). Add a regression
test that rebinds __main__.RuntimeError as the first program statement and drives
the closed-loop worker pattern, asserting the pump survives and delivers the
later binding. Update the settlement note (en + zh) to describe the default-arg
capture; pairing re-recorded and consistent.
- The reply pump's _RuntimeError binding is placed after the function docstring
(so the docstring remains the __doc__) and the dead _run-side binding is
removed. _done_with_value binds _check_done_value/_encode_json_plain as
default arguments so a __main__ rebind after model execution cannot rewrite a
success into an exception.
The _str/_bool/_BindingRejection pump bindings were attempted but break the
closed-loop pump test (the self-referential _BindingRejection local interferes
with the closure), so they are left unbound; rebinding those names (builtins and
one internal class) is outside the practical threat model.
The previous commit bound _RuntimeError in _run, but _pump_replies is a separate
module-level function, so its except _RuntimeError referenced an out-of-scope
local and raised NameError instead of catching the closed-loop failure — killing
the pump and timing out the run. Bind _RuntimeError at the top of _pump_replies
too. The closed-loop pump test now passes.
The reply pump's except RuntimeError resolved the module global at runtime, so a
__main__.RuntimeError rebind could make a closed-loop scheduling failure escape
the catch, killing the pump and stranding every later reply. Bind RuntimeError
into a _run local alongside BaseException and catch the local. The settlement
note Decision now records that the exception classes the settlement-path except
clauses catch are bound into locals / a closure cell before model code runs
(en + zh); pairing re-recorded and consistent.
The rebindable-BaseException vector the bot flagged existed in every except
clause of the settlement path, not just the _run outer catch: safe_model_traceback
(three guards) and the post-done flush swallow resolved the module-global
BaseException at runtime, so a __main__.BaseException rebind plus a throwing
__str__ could let a render-time exception escape and lose the done frame. Bind
BaseException into a _run local (at the top) and a closure cell in
_make_failure_reporter, and change every such except clause to catch the local
— immune to a one-line rebind.
The _run outer try/except used the module-global BaseException, which the
program (running as __main__) can rebind: __main__.BaseException = RuntimeError
made the except resolve to RuntimeError, so a subsequent ValueError escaped _run
with no done frame and misreported the run as worker-exit. Bind BaseException
into a _run local before the program runs so the catch is immune; a regression
test rebinds BaseException and raises, asserting an exception, not a worker-exit.
Also correct the NUL-escape comment text: the JSON escape-result side is \^@ (6
bytes, the valid JSON NUL escape), not \x00, so the 6x-budget arithmetic in the
comments is self-consistent. Register the BaseException-rebind case in the
settlement note Testing (en + zh) and re-record the pairing.
Addresses the bot's keep-current review findings:
- The module-level fallback comment now states the mechanism truthfully: the
module globals are RAW primitives bound into _run LOCALS before the program
runs (the immunity lives in the frame-local binding, not the module global);
and the fallback literal <unrenderable> is distinguished from the failure
reporter's _UNRENDERABLE_DIAGNOSTIC text.
- The settlement note's fallback mechanism wording, the transitive-name rebind
case (now listing the three fallback primitives), and the no-fail-before count
are aligned en/zh; the zh Problem paste damage is fixed and the Consequences
count is ten with the 10th item.
- Pairing re-recorded and consistent.
The done-frame fallback read _os_write/_memoryview/_FALLBACK_DONE_FRAME as module
globals at call time, so a single-line rebind of any of them reopened the
rebind hole the fallback exists to close. Bind them into _run locals before the
program runs, and use a bare except (which catches everything without naming
BaseException, so a rebind of that name cannot defeat the handler). The
transitive-name rebind test now also rebinds _os_write/_memoryview/
_FALLBACK_DONE_FRAME to pin the fallback's immunity.
Addresses the bot's follow-up review findings on the settlement-path fixes:
- The _LogStream seal joined the WHOLE accumulated buffer past the fragment cap,
re-copying the growing block O(B^2/cap) times for a large drip. It now seals
only the current fragments into a _pending_blocks entry (character count
unchanged), so a 25 M single-character drip stays O(B); the newline/flush/
_push_bounded_prefix consumers join blocks + fragments once.
- The _clamped soft==hard lowering is scoped to RLIMIT_CPU: for RLIMIT_AS a
one-byte soft differential would only misalign the child's applied limit with
the host-side budget gate, with no signal to preserve. The hard == 1 blind
spot is documented.
- send_done's fallback captures memoryview at import (_memoryview) alongside
os.write, so a one-line rebind of the name cannot change the fallback write;
the comment now states the module-level-captured mechanism.
Addresses the bot's v16 review on the settlement-path code:
- critical: _LogStream._pending now seals the fragment list past a chunk cap
(like the host captureStray seal), so a newline-free single-character drip no
longer accumulates one list slot per write and OOMs on its own accounting.
- _clamped lowers a soft==hard result by one unit (when hard >= 2) so a
dual-limit ulimit -t leaves SIGXCPU a window to fire and a definite CPU
overrun is reported as a timeout, not a worker-exit.
- send_done wraps its encode+write in a try and, on any throw from a rebound
transitive name (_dump_scalar/os), writes a fixed pre-encoded done frame via
the import-time captured os.write, so a settled exception verdict is never
downgraded to worker-exit.
- drainReplies clears the consumed replyQueue slot so a wide written payload is
released immediately, bounding host memory to the current backlog under
sustained fd-3 backpressure.
Tests added for each (fragment cap drip, dual-limit CPU overrun, transitive-name
rebind done frame).
Addresses the follow-up review findings on the settlement-path fixes:
- send_done now routes both the pre-encoded VALUE frame and the dict ERROR
frame through a bound _encode_json_plain + bound write_encoded, never through
channel.send_sync (whose body re-resolves self.write_encoded and the module
_encode_json_plain at call time) — a program rebinding ProtocolChannel.
write_encoded or __main__._encode_json_plain no longer skips the done frame.
- flushStray retention re-accrues the withheld multibyte tail from a FRESH
utf8 state (previously metering the carried lead against the post-flush
expected>0 state charged it as an illegal continuation), and skips admitting
when the whole residual drained into the retained tail so no bogus empty
entry is pushed.
Corrections to the settlement-path review fixes:
- send_done was invoking channel.send_sync / channel.write_encoded via a late
method look-up, which a program running as __main__ could rebind through
__main__.ProtocolChannel.send_sync before the failure path ran — a rebound
send that raises then skipped the done frame and downgraded a settled
exception to worker-exit. Bind both channel methods into locals before the
program runs, mirroring the pre-existing binding of flush_out/flush_err/
safe_model_traceback.
- Restructure flushStray so the mid-sequence budget-flush retention arm is a
self-contained v8-ignored branch and the covered default path decodes the
full residual (not schedulable-through-the-seam boundary).
Pace-free completion framing, stray UTF-8 flush, and late-rejection guards:
- Pre-encode the completion value at its validation point so send_done never
re-walks a live value a mutating daemon thread could change (TOCTOU); a
mutation-induced encode throw is then classified as 'exception', not a
host-side worker-exit.
- Budget-triggered stray flush retains an incomplete multibyte UTF-8 tail
(<=3 bytes) as residual instead of decoding a legal, split character to
U+FFFD in an admitted entry; the end/closeDeadline paths still full-decode.
- Check 'settled' before formatting a late binding rejection's message, so a
hostile message getter cannot stall or exhaust a run that already settled.
- Document _check_done_value's first-to-trip ruling in its docstring.
- Rewrite ProtocolChannel.send_sync around a shared write_encoded that the
done frame's pre-encoded string path uses.
Adding a published Python backend and reordering `flush_line` left several
owning documents stating things that are no longer true.
`src/invariant.ts` justified its empty installer with "ships only the fd-3
wire-protocol codec", which the subprocess execution path contradicts. The
reason now states the actual one: every relation this backend maintains lives
in the CPython child or on the fd-3 wire, so no same-process event sequence is
observable from a listener -- the same shape the sibling worker-thread backend
uses.
The seam's `PORTABLE_RESERVED_WORDS` and `language` JSDoc, the code-runtime
README pair, and docs/subsystems/code-runtime both said only TypeScript has a
published backend. Corrected in all four, with the generated cordis catalog
regenerated for the `language` change.
The note attributed the 12x multiple to the settlement flush holding three
copies. That stopped being true when `flush_line` was reordered to drop the
pending chunks before its push: the binding worst case is the newline path's
single near-budget write. Corrected in the note (both sides) and in the test
comment that repeated it.
The note's Testing section now registers the cases this stack added, and the
Chinese side receives the O(depth) entry it never got plus the new ones -- it
had drifted from the English.
`INTERPRETER_BASELINE_BYTES` argued 64 MiB from a RESIDENT set while RLIMIT_AS
bounds address space. It now cites the bootstrap's own measurement (30.23 MiB
of mappings for `python3 -I`), making 64 MiB roughly twice the measured
baseline.
Also: a hardcoded `(:232-235)` comment reference becomes a reference by name,
a "which now walks in O(depth) too" change narrative becomes a current-state
statement, and a stray double blank line is removed.
Four independent corrections in the run lifecycle.
`killGroup` signalled `-child.pid` with a raw `process.kill`. Node keeps the
numeric `child.pid` after the leader is reaped and only clears its internal
handle, so `child.kill()` refuses while the raw call does not; `close` can
trail `exit` by seconds when a pipe-holding descendant keeps the streams open.
A recycled pgid could therefore receive this run's SIGTERM and armed SIGKILL.
`groupEmpty()` does not cover it: it reports whether the group has members, not
whether they are ours, and it first runs after the signal. The leader's start
time is now read at spawn and re-checked before each signal, matching the
position packages/subprocess/subprocess-local already states
("ProcessIdentity ... preventing teardown escalation after PID reuse"). Kept
local rather than depending on that package, which would add an architectural
edge. Linux reads /proc; Darwin has no /proc, so the reader reports undefined
and the guard degrades to the previous behavior instead of forking `ps` on a
teardown path.
`_push_bounded_prefix` built `(*self._pending, extra)`, copying every pending
reference into a same-size tuple before the bounded loop. For a
single-character drip that is a second pointer array as large as the list:
measured +80 MiB of tuple over a 40 MiB list for 5.2M chunks, the allocation
the bounded prefix exists to avoid. It now iterates the list in place and
handles `extra` in the loop's `else`; 4000 randomized inputs produce byte-identical
prefixes.
The settlement `flush_out()`/`flush_err()` ran outside any guard while `done`
was already decided, so a flush raising under memory pressure skipped
`send_done` and downgraded a child-classified `exception` into a host-side
`worker-exit`. Both are now wrapped, swallowing only the log tail.
The boot re-check's `if effective_soft != RLIM_INFINITY` was dead: `_clamped`
is asked for a finite `addr_bytes` on both sides and each branch returns that
value or a `min` with an inherited bound, so RLIM_INFINITY is unreachable. The
guard could only ever have skipped the re-check it claimed to protect.
Three separate paths in the CPython child allocated state proportional to a
value's width or a string's length, so a legitimate input the byte budgets
admit could die as the program's own MemoryError.
`_lossless_json_violation` enqueued one traversal tuple per member while
running, in `dispatch`, over MODEL-CONSTRUCTED binding arguments that no
child-side byte budget bounds first. It now uses the same (kind, container,
iterator) cursor the other two walks already had, checking dict keys as the
cursor pulls each entry. Measured over `[0] * 6_000_000` (~17 MB of JSON):
459.1 MiB of traversal tuples before, 0.0 MiB after.
`_decode_json_plain` matched JSON strings with a `(?:[^"\\]|\\.)*` repetition,
which makes CPython's engine retain backtracking state proportional to the
string's width: 146 MiB for a 1 MiB string, 557.8 MiB for 4 MiB. A legitimate
multi-megabyte binding reply raised MemoryError inside `_pump_replies`, and
because that pump is the only settler of the call's future, the run stranded
until the wall clock reported `timeout`. Strings now scan chunk-to-chunk over a
character class, which the engine matches without backtracking state; the same
4 MiB decode peaks at the 4.0 MiB result.
`_check_done_value` charged strings and dict keys what
`_dump_string(...).encode()` returned, building the escaped copy plus its
encode to MEASURE it -- ~6x the original each for control-heavy text, so
metering a value the budget then rejects could itself breach RLIMIT_AS and
report `exception` where the seam promises `output-limit`. The new
`_json_str_cost` counts instead, reusing `_json_string_cost`'s C-level passes
and reproducing `_dump_string`'s exact surrogate rules (fold spelled-out pairs,
charge six ASCII bytes per lone surrogate). Identical values, 228.9 MiB -> 19.1
MiB of peak on a 20M-NUL string.
Each fix ships a regression test. The two RLIMIT_AS repros are Linux-only:
Darwin does not apply the limit, so the peaks above are measured directly and
recorded in the test comments.
`_check_done_value` and `_encode_json_plain` pushed one stack entry per child
(plus a separator marker, and `dict.items()` materialized as a list), so the
bookkeeping scaled with the value's WIDTH rather than its depth. A value the
byte meter admits could then die on the walk's own frames: a flat
`[0] * 2_000_000` serializes to 4.0 MB, but measured peaks were 145.2 MB in the
meter and 114.7 MB in the encoder — 28.7x the serialized size, far past the 12x
the load-time address-space gate reserves.
Each container now pushes ONE cursor frame that pulls its children one at a
time and writes into a shared `io.StringIO`, so the output string is the only
width-proportional allocation and the caller already metered its size. Measured
on the same value: 0.0 MB in the meter and 9.0 MB in the encoder (2.3x), with
identical verdicts.
The load gate bounds maxLogBytes and maxValueBytes independently against the
address space, but the child framed the completion value (materializing its
escaped form to meter it, then encoding the frame) while a newline-free log tail
still sat unflushed in _pending. Those two peaks added, so two budgets each
admitted alone could together breach RLIMIT_AS and die as worker-exit instead of
settling. The success path now flushes both log streams before _done_with_value
runs; the trailing flush stays for the exception path and is an idempotent no-op
after a successful settle. A combined-peak regression test (32 MiB each against
512 MiB) asserts the over-budget value reports output-limit rather than OOMing.
Also corrects the worst-case-multiple JSDoc and Agent Note: after 1088d6f03d
made flush_line drop pending before its push, the settlement-flush path holds
two copies, not three, so the newline path is the sole 12x worst case. The
reorder is recorded as a called-out untested fix (the 12x gate already admits
only configs safe under both flush orders).
The load-time output-budget/addressSpaceMb gate used a worst-case multiple of 8,
assuming two simultaneous ~4x astral copies (the built string and its encode).
Three are live at the peak: on the newline path a single write holds the caller's
text argument, the line slice handed to push, and push's encode copy; the
settlement flush_line path held the pending chunks, their join, and that encode
copy. A budget admitted at 8x (e.g. maxLogBytes 48 MiB against addressSpaceMb 512)
could still OOM the child. The multiple is now 12, the strict `>` is `>=` so a
budget whose peak exactly equals the room left after the interpreter baseline is
rejected (that peak plus the baseline is the whole address space), and flush_line
drops the pending chunks before its push to match the newline path's
join-clear-push order. The child re-check mirror and both note sides move in step;
config-catalog is regenerated from the updated field JSDoc.
The output-budget/address-space gate's 8x multiple had no room for the
interpreter's own footprint, so a budget sized right at addressSpaceMb/8 was
admitted while its worst-case peak plus the interpreter overran RLIMIT_AS
(e.g. 15 MiB maxLogBytes against 128 MiB). Reserve a fixed
INTERPRETER_BASELINE_BYTES (64 MiB) before the multiple claims the rest, so each
budget times 8 must fit the room LEFT after the baseline.
The host gate validates against the CONFIGURED addressSpaceMb, but a launch
environment can inherit a stricter RLIMIT_AS (a ulimit -v wrapper below
addressSpaceMb) that _clamped lowers the effective limit to, leaving the budgets
sized for a ceiling the child never gets. bootstrap.py now re-checks both budgets
against the effective clamped soft limit after applying it, mirroring the host
gate's multiple and baseline, and raises at boot rather than letting a
near-budget output OOM mid-run.
Add regression tests for both (the load gate against a 256 MiB address space
covering both budgets, and a ulimit -v wrapper for the inherited-limit re-check);
register the tail-copy test in the note Testing section; sync the zh pair. Merges
origin/feat/code-runtime-python-protocol to resolve the DIRTY base.
The load-time addressSpaceMb gate used a 1/8 fraction derived for ASCII, but the
child ledgers trigger on character count against a serialized-byte budget: an
astral character is one character yet ~4 bytes stored and ~4 encoded, live at
once, so the true worst-case peak is ~8x the budget, not ~2x. Replace the
fraction with an explicit OUTPUT_BUDGET_WORST_CASE_ADDRESS_SPACE_MULTIPLE (8)
and a strict `>`, and gate maxValueBytes the same way as maxLogBytes — the value
path builds and encodes a near-budget completion under the same RLIMIT_AS, so
the incompatible pair was previously admitted there too.
Slice the newline branch's unterminated tail to a budget-sized prefix: it
buffered the whole text[pos:] before the flush trigger could bound it, so an
early newline plus a huge tail made a second full copy of the model's string —
an RLIMIT_AS death the config gate cannot cover since the tail can far exceed
maxLogBytes.
Disclose the cross-field constraint in the maxLogBytes/maxValueBytes/addressSpaceMb
JSDoc (regenerating config-catalog); refresh the note's stale
Buffer.byteLength(JSON.stringify) reference; reconcile the arrival-order rebuttal
with the seam's "in order" logs JSDoc (within-stream, cross-stream best-effort).
Extend the load-rejection test to both budgets and add a tail-copy regression;
sync the zh pair.
The child log ledger encodes an admitted entry to UTF-8 once to charge its
serialized cost, so a maxLogBytes approaching addressSpaceMb lets a legitimate
near-budget log entry breach RLIMIT_AS and die as worker-exit instead of
truncating. Two runtime fixes were tried and both traded one resource bound for
another: an exact serialized-cost check is either a full encode (the allocation
being avoided) or a per-character Python loop that burns the CPU budget (a 10 MB
write hits SIGXCPU under cpuSeconds:1). The breach is a property of the
maxLogBytes/addressSpaceMb pair, not any write, so reject the incompatible pair
at load — maxLogBytes must stay within one eighth of the addressSpaceMb byte
count — and revert _LogStream to its original character-count buffering, which
is memory-safe once the budget fits the address space. The check runs on every
platform since the incompatibility is a config-value property, not a runtime one.
Replace the child-flood regression tests (which asserted the reverted runtime
behavior) with a load-rejection test. The host-side accrueStrayCost UTF-8
per-lead validation and its tests are unaffected. Update the note and zh pair.
accrueStrayCost accepted any 0x80-0xBF continuation, so a CESU-8 surrogate
(ED A0 80) or overlong (E0 80 80) — structurally well-formed but illegal, and
as cheap to flood as 0xFF — was charged its structural width 3 while
toString('utf8') renders each byte as its own U+FFFD (cost 9). Validate each
lead's first-continuation range (WHATWG E0/ED/F0/F4 bounds) and charge 3 per
byte of any sequence outside it, folding a broken prefix to one U+FFFD.
The child _LogStream newline path had the same char-vs-serialized gap the
newline-free trigger had: its per-line fit checks (first reconstructed line and
each subsequent line) compared character count against the serialized-byte
budget, so a control-char line passed and _logs.push encoded it whole, breaching
RLIMIT_AS. Route every check through _fragment_cost_upto, which sums per-char
costs from _json_char_cost over a start/end sub-range without slicing or
encoding and stops at the budget.
Decline arrival-order stray flushing: the two pipes' data events interleave
nondeterministically and logs carries no cross-pipe ordering guarantee, so a
fixed drain order is as valid as any and an arrival-tick branch could not be
covered without a flaky test.
Add CESU-8/overlong, newline-path-flood, and all-lead-class reassembly
regression tests; fix the note's now-inaccurate CESU/illegal-byte claims and a
fixture byte-count comment; sync the zh pair.
The prior child-flush fix measured each fragment with chunk.encode('utf-8'),
which copies the whole write — under a tight addressSpaceMb a single 340 MiB
write died on that encode (the exact allocation _push_bounded_prefix exists to
avoid), and re-scanning the whole pending list per write was quadratic under a
daemon-thread flood (the concurrent-write test timed out at 28s). Compute each
fragment's serialized cost with _fragment_cost_upto, which walks the str via a
new _json_char_cost (code point to escaped width, no encode) and stops once the
running total passes the budget, and accumulate it into _pending_cost once per
write. The early-flush trigger reads that accumulator: still charges control
chars their full serialized width (a NUL is 6 bytes), but never encodes a whole
write and never re-scans the buffer, so the 340 MiB single-write and
daemon-thread tests pass alongside the NUL-flood one.
Rework the child NUL-flood regression to write in 1 MiB chunks under a 512 MiB
address space so its own argument construction is not the allocation under test.
The host stray-capture cost function charged illegal UTF-8 bytes (0x80-0xC1,
0xF5-0xFF, and orphaned multibyte leads) the raw 1, but toString('utf8')
renders each as U+FFFD (3 serialized bytes). A b"\xff" flood was undercounted
threefold, so the residual grew to a full budget's worth of raw bytes before
flushing and, near a large maxLogBytes, expanded toward a ~1 GiB peak in the
flush's concat plus toString. Replace serializedBufferCost with accrueStrayCost,
a cross-chunk UTF-8 walker that charges each byte its decoded serialized width;
carry its sequence state on each StrayBuffer.
The child _LogStream had the same-family bug: its early-flush trigger compared
_pending_chars (character count) against remaining (a serialized-byte budget),
so a 30M-NUL newline-free flood stayed under a 50 MB char trigger yet encoded to
~180 MB at settlement, breaching RLIMIT_AS as worker-exit. Track _pending_cost
via the _JSON_BYTE_COST table and trigger on it; keep _pending_chars for the
char-based slice bounds.
Correct the note's surrogate claim (only the string-walking jsonStringCostUpTo
charges a lone surrogate six bytes; the byte walker never sees one). Shrink the
post-truncation fixture below PIPE_BUF for a deterministic single callback. List
the shared stdout/stderr budget as a third honest fail-before exception
(cross-pipe arrival timing is nondeterministic). Add illegal-UTF-8,
broken-multibyte, and child-log-flood regression tests; sync the zh pair.
Wrap spawn and the fd-3 narrowing so a synchronous throw (ENAMETOOLONG on
an over-PATH_MAX pythonBin, EMFILE) removes the run's staging directory and
resolves the same worker-exit class as the async error event, instead of
rejecting run() and leaking the directory.
Aggregate native stdout/stderr by real newline rather than by Node data
chunk: logs entries are joined with "\n" downstream, so a newline-free
write larger than one pipe read no longer reads back with spurious breaks.
The ledger still bounds a newline-free flood.
Track a running scan offset in both frame readers so a large frame
accumulated across chunks is scanned once, not re-scanned from 0 per chunk.
Reword the deadline hard-bound v8-ignore to state its real environment
dependence (PID-1-doesn't-reap container, zombie survivor) and cross-ref
the note's rejected signal-0 alternative; fix settle comments that quoted
the pre-qualification teardown contract; document the capMessage vs
_cap_message billing split on both sides; guard the dispose-after-resolve
heartbeat assertion against a vacuous 0===0 pass; reuse
_TRUNCATION_MARKER_BYTES; note the abandoned-call pending-entry bound.
Update the Agent Note Decision/Testing/Alternatives/Consequences for the
above and record the confirmed-empty finalize as a second honest
fail-before exception; sync the zh pair.
The reap-poll deadline arm sent SIGKILL then finalized immediately, declaring
quiescence on mere signal delivery while the group was still dying. It now keeps
polling for the group to actually empty (bounded by one more reap margin) after
its self-sent SIGKILL, so `finished` resolves only on a confirmed-empty group.
ProtocolChannel.read_frame read the boot/run handshake frames through
FileIO.readline() on the unbuffered fd — one os.read(1) per byte, so a
multi-megabyte program burned CPU (RLIMIT_CPU already in force for the run frame)
in millions of syscalls before ast.parse. It now reads in chunks into the same
_pending buffer the async reader uses; the wrapping os.fdopen is gone. read_frame
is this PR's own code (e7f22ed3), not the protocol layer. The chunked read is a
syscall-count improvement with no cross-platform-deterministic failure to assert,
noted as such in the Agent Note.
Raising maxValueBytes' load bound to ceiling-envelope assumed both budgets are
metered in serialized (JSON-escaped) bytes, which held for completion values and
logs but not the diagnostic: _cap_message capped by raw UTF-8, so a control-heavy
message near maxValueBytes could serialize sixfold and breach the fd-3 frame
ceiling — the silent worker-exit inversion the load check prevents. _cap_message
now accumulates per-byte serialized cost (new _JSON_BYTE_COST table) and cuts the
prefix that fits. Also reword the host SIGXCPU timeout message to name cpuSeconds
as the configured ceiling rather than a budget a stricter inherited RLIMIT_CPU
soft may undercut. Adds a control-heavy-diagnostic regression test.
The class docstring still credited the GIL plus per-frame PIPE_BUF atomicity for
serializing writes, which _write_lock's full-write loop already superseded. State
the current contract (writers serialized by _write_lock around a full-write loop)
and drop the double blank line under the binding-replies note heading.
The settlement-time CPU recheck compared spent CPU against the configured
cpuSeconds, but _clamped may have lowered the effective soft limit to a stricter
inherited value. A program that traps SIGXCPU, burns past the inherited soft,
and returns inside the soft-to-hard gap was checked against the configured value
and falsely reported successful, bypassing the inherited limit. The recheck now
uses the clamped cpu_soft. Adds a regression test that inherits a 1s soft CPU
limit and asserts a SIGXCPU-trapping over-burn is a timeout, not a success.
A binding called from a worker thread records that thread's loop for its reply.
If the thread finished and closed its loop before the host reply arrived,
_pump_replies' call_soon_threadsafe onto the closed loop raises RuntimeError;
unguarded, that ends the pump task and strands every later reply. Wrap the
schedule in a try/except that drops the moot reply (nothing awaits it) and keeps
the pump serving.
Two further review findings on the CPython backend:
- The grace-window SIGKILL timer was left armed after settlement, so on a
normal completion a kill(-pid) could fire up to graceMs later and strike a
recycled pgid once the kernel reused the leader's pid. settle() now clears
the timer the moment the process group is confirmed empty (the normal path
and when the poll sees the survivor gone), bounding the reuse window to the
genuine-survivor case where the group cannot be empty to reuse.
- _clamped bounded rlimits by the inherited hard limit only, silently raising
an inherited soft limit stricter than the request (loosening RLIMIT_AS or
deferring RLIMIT_CPU SIGXCPU). It now clamps each side against its own
inherited counterpart and pins soft under hard, keeping the strictest of
configured and inherited. Adds an inherited-soft-limit regression test.
Agent Note expanded to seven fixes with the two new rejected alternatives;
zh pair re-recorded.
Two review findings on the CPython backend:
- Disposal could return while a same-group descendant that ignores SIGTERM
but releases the inherited pipes was still alive: the leader's close fired
and the previous fix relied on an unref'd SIGKILL timer that a short-lived
host never fires, reparenting the survivor to init. settle() now withholds
the run's finished promise on a ref'd process-group poll until the SIGKILL
has emptied the group (bounded by graceMs + margin, zero-cost when already
empty), so teardown's "await each child's exit" holds.
- A binding called from a model worker thread via asyncio.run created its
reply Future on that thread's loop, but _pump_replies completed it directly
from the main loop; asyncio.Future is not thread-safe across loops, so the
call hung to the wall clock. Replies now complete via the owning loop's
call_soon_threadsafe, and a lock serializes the id claim/write/advance.
Tests: the same-group reap case now asserts a heartbeat file stops (robust
whether the killed descendant is reaped or a zombie, so it holds where PID 1
does not wait() orphans); a cross-loop case runs a binding from a worker
thread and asserts the reply round-trips instead of timing out. Agent Note
expanded to all six fixes with rejected alternatives; zh pair re-recorded.
Land the PythonCodeRuntime implementation on top of the fd-3 protocol
seam: python3 -I per run, binding namespace over fd 3, RLIMIT_CPU/AS,
wall-clock timer, and SIGTERM->grace->SIGKILL process-group teardown,
with the real-subprocess integration suite.
Fixes three defects surfaced on the source PR's review before they ship:
- boot-write failure resolved a worker-exit through finish()/settle()
that read wallTimer/onAbort/live in their TDZ, rejecting run() instead;
the boot write now runs after those bindings and the v8-ignore that hid
the branch is removed.
- log capture serialized against settlement with no lock while model
daemon threads keep writing; LogBuffer now owns one shared re-entrant
lock taken by write/flush_line/push.
- the fd-3 line residual was a subarray view pinning the whole joined
frame; it is copied into a right-sized Buffer via detachResidual so
pendingBytes measures what is retained.
Chinesezjc