From 9a8663cc4c87420125154739e763fdb29b508e4b Mon Sep 17 00:00:00 2001 From: Chinesezjc Date: Mon, 3 Aug 2026 14:43:56 +0800 Subject: [PATCH] fix(code-runtime-python): flush logs before framing the completion value The load gate bounds maxLogBytes and maxValueBytes independently against the address space, but the child framed the completion value (materializing its escaped form to meter it, then encoding the frame) while a newline-free log tail still sat unflushed in _pending. Those two peaks added, so two budgets each admitted alone could together breach RLIMIT_AS and die as worker-exit instead of settling. The success path now flushes both log streams before _done_with_value runs; the trailing flush stays for the exception path and is an idempotent no-op after a successful settle. A combined-peak regression test (32 MiB each against 512 MiB) asserts the over-budget value reports output-limit rather than OOMing. Also corrects the worst-case-multiple JSDoc and Agent Note: after 1088d6f03d made flush_line drop pending before its push, the settlement-flush path holds two copies, not three, so the newline path is the sole 12x worst case. The reorder is recorded as a called-out untested fix (the 12x gate already admits only configs safe under both flush orders). --- ...-runtime-python-settlement-fixes.i18n.yaml | 4 +-- ...31-code-runtime-python-settlement-fixes.md | 8 ++--- ...code-runtime-python-settlement-fixes.zh.md | 8 ++--- .../code-runtime-python/py/bootstrap.py | 14 +++++++- .../code-runtime-python/src/index.ts | 11 +++--- .../code-runtime-python/tests/runtime.spec.ts | 36 +++++++++++++++++++ 6 files changed, 65 insertions(+), 16 deletions(-) diff --git a/.agents/notes/implemented/bug-fix/2026-07-31-code-runtime-python-settlement-fixes.i18n.yaml b/.agents/notes/implemented/bug-fix/2026-07-31-code-runtime-python-settlement-fixes.i18n.yaml index 2b5234531f..04e908dd25 100644 --- a/.agents/notes/implemented/bug-fix/2026-07-31-code-runtime-python-settlement-fixes.i18n.yaml +++ b/.agents/notes/implemented/bug-fix/2026-07-31-code-runtime-python-settlement-fixes.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/bug-fix/2026-07-31-code-runtime-python-settlement-fixes.md -2026-07-31-code-runtime-python-settlement-fixes.md: 088b26397765b908cfbf3514fe020301a0a19235 -2026-07-31-code-runtime-python-settlement-fixes.zh.md: 1d45ceffda823f8cc1fb15f6cfb0a3bcef1688ad +2026-07-31-code-runtime-python-settlement-fixes.md: 0614d4f40c03c01b63c0ef3ae5584e667d94dddc +2026-07-31-code-runtime-python-settlement-fixes.zh.md: c1320696c6a8c2316befd79396618157299a9484 diff --git a/.agents/notes/implemented/bug-fix/2026-07-31-code-runtime-python-settlement-fixes.md b/.agents/notes/implemented/bug-fix/2026-07-31-code-runtime-python-settlement-fixes.md index 088b263977..0614d4f40c 100644 --- a/.agents/notes/implemented/bug-fix/2026-07-31-code-runtime-python-settlement-fixes.md +++ b/.agents/notes/implemented/bug-fix/2026-07-31-code-runtime-python-settlement-fixes.md @@ -6,7 +6,7 @@ English | [中文](2026-07-31-code-runtime-python-settlement-fixes.zh.md) ## Problem -The CPython subprocess backend for Code Mode, built on the [fd-3 frame protocol](../architecture/2026-07-31-code-runtime-python-fd3-protocol.md), resolves every program outcome as a `CodeRunResult`, rejects `run()` only for seam misuse, and disposes to quiescence so no subprocess that stays in the child's process group outlives the fiber (a descendant that escapes the group with `setsid()` is the documented exception — see the package README's Known Limitations). A sequence of review passes surfaced defects that broke those contracts in ways unit coverage did not catch — each hid behind a `/* v8 ignore */`, a captured-callable that read as a fix but was not, a memory effect invisible through the seam, a load-time bound that double-counted, a process-group escalation that a survivor could outlast, a cross-event-loop completion that silently deadlocked, a synchronous throw outside the settlement path, or a transport boundary rendered as a log boundary. Most behavioral fixes ship with a test that fails without them; three do not, and are called out as such — the chunked frame read (a syscall-count improvement with no cross-platform-deterministic failure), the confirmed-empty finalize (its only seam-observable effect, a frozen heartbeat, freezes the instant SIGKILL is delivered, which the pre-fix finalize-on-delivery code also produced, and the discriminating probe is the signal-0 check the Alternatives reject as cross-environment-unreliable), and the shared stdout/stderr budget (its only seam-observable difference is which entry boundary a mid-stream flush lands on, and that depends on the relative arrival timing of two independent OS pipes, which `os.sched_yield` does not make deterministic; the per-pipe memory bounds it strengthens ARE covered by the single-pipe flood tests). +The CPython subprocess backend for Code Mode, built on the [fd-3 frame protocol](../architecture/2026-07-31-code-runtime-python-fd3-protocol.md), resolves every program outcome as a `CodeRunResult`, rejects `run()` only for seam misuse, and disposes to quiescence so no subprocess that stays in the child's process group outlives the fiber (a descendant that escapes the group with `setsid()` is the documented exception — see the package README's Known Limitations). A sequence of review passes surfaced defects that broke those contracts in ways unit coverage did not catch — each hid behind a `/* v8 ignore */`, a captured-callable that read as a fix but was not, a memory effect invisible through the seam, a load-time bound that double-counted, a process-group escalation that a survivor could outlast, a cross-event-loop completion that silently deadlocked, a synchronous throw outside the settlement path, or a transport boundary rendered as a log boundary. Most behavioral fixes ship with a test that fails without them; four do not, and are called out as such — the chunked frame read (a syscall-count improvement with no cross-platform-deterministic failure), the confirmed-empty finalize (its only seam-observable effect, a frozen heartbeat, freezes the instant SIGKILL is delivered, which the pre-fix finalize-on-delivery code also produced, and the discriminating probe is the signal-0 check the Alternatives reject as cross-environment-unreliable), the shared stdout/stderr budget (its only seam-observable difference is which entry boundary a mid-stream flush lands on, and that depends on the relative arrival timing of two independent OS pipes, which `os.sched_yield` does not make deterministic; the per-pipe memory bounds it strengthens ARE covered by the single-pipe flood tests), and the `flush_line` join-clear-push reorder (it lowers the settlement-flush peak from three copies to two, but the 12x load gate already covers the three-copy newline path, so every gate-admitted config stays within the address space under both orders and no seam-observable difference exists — the memory effect is inside the Python child, unmeasurable through the seam like the shared-budget case). ## Decision @@ -58,7 +58,7 @@ Also in `src/index.ts`, native stdout/stderr bytes (C-extension writes, `os.writ ### An incompatible output-budget/addressSpaceMb pair is rejected at load -The child ([`py/bootstrap.py`](../../../../packages/code-runtime/code-runtime-python/py/bootstrap.py)) builds, charges, and frames a `maxLogBytes` log entry or a `maxValueBytes` completion value under `RLIMIT_AS`, and both ledgers trigger on CHARACTER count against a serialized-BYTE budget. An astral character is one character but four bytes of CPython `str` storage and four UTF-8 bytes, and THREE such copies are live at the peak: on the newline path a single `sys.stdout.write(line + "\n")` holds the caller's `text` argument (alive for the whole `write` call, ~4×), the line slice handed to `LogBuffer.push` (~4×), and the `text.encode("utf-8")` copy `_push_locked` takes to charge and ship it (~4×); the settlement `flush_line` path holds the pending chunks, their `"".join(...)`, and that same encode copy — a peak of ~12× the budget. When a budget approaches `addressSpaceMb`, a LEGITIMATE near-budget output breaches the address space during that build-and-encode and dies as `worker-exit` instead of truncating (log) or failing as `output-limit` (value). Metering every child write against the address space at runtime is the wrong fix: an exact serialized-cost check on the hot path is either a full `encode` (the allocation being avoided) or a per-character Python loop (which burns the CPU budget — a 10 MB legitimate write hits SIGXCPU under `cpuSeconds: 1`). Both trade one resource bound for another. Instead [`src/index.ts`](../../../../packages/code-runtime/code-runtime-python/src/index.ts) rejects the incompatible pair at LOAD: each budget times `OUTPUT_BUDGET_WORST_CASE_ADDRESS_SPACE_MULTIPLE` (twelve — the three simultaneous ~4× copies) must fit the address space LEFT after a fixed `INTERPRETER_BASELINE_BYTES` reservation for the interpreter's own footprint, with a `>=` so a budget whose worst-case peak exactly equals that room is rejected (that peak plus the reserved baseline is the whole address space, the `RLIMIT_AS` edge). `flush_line` was also made to drop the pending chunks BEFORE its push, matching the newline path's join-clear-push order, so that path holds at most the join and its encode rather than three copies. The baseline is reserved SEPARATELY from the multiple because it is a fixed cost, not one that scales with the budget: folding it into the multiple would leave a budget sized right at `addressSpaceMb / 12` admitted while its peak plus the interpreter still overran. Both `maxLogBytes` and `maxValueBytes` are gated symmetrically; the value path's build-and-encode is the same shape. The check runs on every platform, not just where `RLIMIT_AS` is enforced: the incompatibility is a property of the config values, so a Linux deployment OOMs regardless of the host that assembled the config, and a uniform load-time rejection is the fail-loud contract (Darwin skips only the runtime `setrlimit`). This eliminates the class at the config seam rather than patching the write path, so `_LogStream` keeps its original character-count buffering (a valid lower bound on serialized cost, memory-safe once the budget fits the address space). +The child ([`py/bootstrap.py`](../../../../packages/code-runtime/code-runtime-python/py/bootstrap.py)) builds, charges, and frames a `maxLogBytes` log entry or a `maxValueBytes` completion value under `RLIMIT_AS`, and both ledgers trigger on CHARACTER count against a serialized-BYTE budget. An astral character is one character but four bytes of CPython `str` storage and four UTF-8 bytes, and the heaviest path holds THREE such copies at once: a single `sys.stdout.write(line + "\n")` keeps the caller's `text` argument (alive for the whole `write` call, ~4×), the line slice handed to `LogBuffer.push` (~4×), and the `text.encode("utf-8")` copy `_push_locked` takes to charge and ship it (~4×) — a peak of ~12× the budget. The settlement `flush_line` path holds only two (its `"".join(...)` and that encode copy — it drops the pending chunks before pushing), so the newline path is the binding worst case. When a budget approaches `addressSpaceMb`, a LEGITIMATE near-budget output breaches the address space during that build-and-encode and dies as `worker-exit` instead of truncating (log) or failing as `output-limit` (value). Metering every child write against the address space at runtime is the wrong fix: an exact serialized-cost check on the hot path is either a full `encode` (the allocation being avoided) or a per-character Python loop (which burns the CPU budget — a 10 MB legitimate write hits SIGXCPU under `cpuSeconds: 1`). Both trade one resource bound for another. Instead [`src/index.ts`](../../../../packages/code-runtime/code-runtime-python/src/index.ts) rejects the incompatible pair at LOAD: each budget times `OUTPUT_BUDGET_WORST_CASE_ADDRESS_SPACE_MULTIPLE` (twelve — the three simultaneous ~4× copies of the newline path) must fit the address space LEFT after a fixed `INTERPRETER_BASELINE_BYTES` reservation for the interpreter's own footprint, with a `>=` so a budget whose worst-case peak exactly equals that room is rejected (that peak plus the reserved baseline is the whole address space, the `RLIMIT_AS` edge). `flush_line` was also made to drop the pending chunks BEFORE its push, matching the newline path's join-clear-push order, so it holds at most the join and its encode rather than three copies. The baseline is reserved SEPARATELY from the multiple because it is a fixed cost, not one that scales with the budget: folding it into the multiple would leave a budget sized right at `addressSpaceMb / 12` admitted while its peak plus the interpreter still overran. Both `maxLogBytes` and `maxValueBytes` are gated symmetrically; the value path's build-and-encode is the same shape. The check runs on every platform, not just where `RLIMIT_AS` is enforced: the incompatibility is a property of the config values, so a Linux deployment OOMs regardless of the host that assembled the config, and a uniform load-time rejection is the fail-loud contract (Darwin skips only the runtime `setrlimit`). This eliminates the class at the config seam rather than patching the write path, so `_LogStream` keeps its original character-count buffering (a valid lower bound on serialized cost, memory-safe once the budget fits the address space). This eliminates the class at the config seam rather than patching the write path, so `_LogStream` keeps its original character-count buffering (a valid lower bound on serialized cost, memory-safe once the budget fits the address space). The host gate validates against the CONFIGURED `addressSpaceMb`, but a launch environment can inherit a STRICTER `RLIMIT_AS` (a `ulimit -v` wrapper below `addressSpaceMb`), which the bootstrap's `_clamped` correctly lowers the EFFECTIVE limit to — leaving the budgets sized for a ceiling the child never gets. So `bootstrap.py` re-checks both budgets against the effective clamped soft limit after applying it, mirroring the host gate's multiple and baseline, and raises at boot (caught by the setrlimit-phase handler and reported as `exception`, the same class as any other resource-limit-application failure) rather than letting a near-budget output OOM mid-run. The two child constants are kept in step with the host's by the shared reasoning, not a wire field. @@ -68,7 +68,7 @@ One residual write-path copy is fixed alongside, independent of the config gate: - `tests/boot-write-failure.spec.ts` mocks `spawn` so the fd-3 pipe throws on the boot write — the one path a real subprocess cannot be coerced into — and asserts `run()` resolves a `worker-exit` rather than rejecting. A sibling case makes the mocked `spawn` throw SYNCHRONOUSLY and asserts `run()` still resolves a `worker-exit` and removes its staging directory, keyed off the exact bootstrap path the mocked `spawn` received in its argv so a sibling worker's concurrent staging cannot flake it. Both are isolated in this spec so the real-subprocess suite is untouched. - `tests/residual-detach.spec.ts` unit-tests `detachResidual`: the carried copy equals the residual, owns a backing store sized to its own length (fixture kept above Node's Buffer pool threshold), and does not share the source frame's `ArrayBuffer`. -- `tests/runtime.spec.ts` — the output-cap case asserts the `ceiling - envelope` bound (268435392) and its message. A daemon-thread case drives four threads emitting unterminated writes through settlement's flush. A native-write case writes 200 KiB with no newline via `os.write` under a raised `maxLogBytes` and asserts it reads back as EXACTLY one log entry (proving stray output is aggregated by line, not split at pipe-chunk boundaries); a companion writes `b"one\ntwo\nthree"` and asserts three entries (proving real newlines still delimit). A newline-free-flood case writes 2 MiB under a 4 KiB `maxLogBytes` and asserts the capture ends at the truncation marker and stays under budget (proving the residual is bounded by the ledger, not buffered whole); a NUL-flood companion writes 4000 newline-free NULs under the same budget and asserts truncation (proving the residual is charged by SERIALIZED cost, ~6× raw, measured without allocating the escaped copy); an illegal-UTF-8 case paces single-byte `\xff` writes under a 3072-byte budget with `Buffer.concat` wrapped to measure the peak merged buffer, asserting it stays under 2048 (charged at the U+FFFD width 3 the residual flushes near 1024 raw bytes; a raw-byte undercount would let it reach ~3072, so the bound discriminates); a CESU-8/overlong case paces the structurally-well-formed but illegal `ED A0 80` one byte at a time and asserts the same peak bound (charged at the true 9 per sequence it flushes early; charging the structural width 3 triples the peak, so reverting the per-lead range check turns it red); a broken-multibyte case writes a 3-byte lead then a fresh ASCII byte in separate chunks and asserts both a captured `A` and a U+FFFD (exercising `accrueStrayCost`'s cross-chunk broken-sequence branch); a post-truncation case writes a 108-byte payload (under the smallest PIPE_BUF, so one atomic write) whose first line exhausts a 64-byte budget and asserts the second line is dropped (exercising the post-truncation admit no-op in one `data` callback, no v8-ignore); a short-escape case writes a line mixing a tab, quote, backslash, a `\uXXXX` control, a multibyte character, and ASCII, asserting it round-trips verbatim (exercising every branch of `jsonStringCostUpTo`); a reassembly case writes a payload spanning every valid multibyte lead class (E0-range, plain 3-byte, F0, and F4) past the pipe buffer and asserts it round-trips with no U+FFFD (exercising `accrueStrayCost`'s per-lead ranges and cross-chunk reassembly); a lone-surrogate case forges an fd-3 `log` frame flooding 1000 `\ud800` escapes under a 4 KiB budget and asserts truncation (the count sits in the window where charging 3 bytes would admit and 6 bytes truncates, proving the surrogate is charged its full escaped width); a stray-sealing case paces 60000 single-byte newline-free `os.write(1, …)` calls under a raised budget with `Buffer.concat` wrapped to measure copy volume, asserting the trickle coalesces to one entry and the cumulative copy stays under a measured 256 KiB threshold (the sealed shape copies ~120 KB, the re-merge shape ~538 KB, so reverting the seal to a re-merge turns the assertion red — proving the fragment list seals into blocks past `MAX_PENDING_CHUNKS`). A closeDeadline-flush case has the leader write a newline-free diagnostic then spawn a `setsid` orphan holding the pipes open, and asserts the diagnostic survives in `logs` (proving the residual is flushed before the deadline destroys the streams). The same-group reap case spawns a SIGTERM-ignoring same-group descendant that releases the pipes and bumps a heartbeat file; the test asserts the heartbeat STOPS after the grace-window SIGKILL — an assertion robust whether the killed descendant is reaped or lingers as a zombie, so it holds where PID 1 does not wait() orphans. A dispose-after-resolve case asserts `dispose()` of a completed run with a same-group survivor returns only after the survivor stops executing (proving the run stays in `live` until its group is reaped), with an `expect(afterDispose).toBeGreaterThan(0)` guard so the frozen-heartbeat assertion cannot pass vacuously when the file was never written. A deadline case busy-blocks the event loop past both timers and asserts the survivor's heartbeat freezes (proving the poll's deadline arm sends SIGKILL itself rather than cancelling the unfired escalation). The cross-loop case runs a binding from a worker thread's own `asyncio.run` loop while the main coroutine yields with `await asyncio.sleep`, asserting the reply round-trips instead of timing out; a companion case abandons a thread's call so its loop closes, then answers it before a later binding — asserting the pump survives the closed-loop `call_soon_threadsafe` (host-gated ordering makes it deterministic, fail-before hangs the later binding to the wall clock). The inherited-soft-limit case runs the interpreter through a `ulimit -S -t` wrapper that sets a CPU soft limit below `cpuSeconds` and asserts the applied `RLIMIT_CPU` soft is the inherited value, not the configured one (CPU rather than address space, since macOS ignores `ulimit -v`); a companion inherits a 1 s CPU soft, has the program trap SIGXCPU and busy-loop past it, and asserts the settlement recheck reports a timeout — proving the recheck uses the effective soft, not the configured `cpuSeconds`. A control-heavy-diagnostic case raises a NUL-flood exception under a small `maxValueBytes` and asserts the serialized frame fits (proving the diagnostic is metered by serialized cost). A tail-copy case (`maxLogBytes: 256`, `addressSpaceMb: 384`) has the program build a tail in a variable and write `"\n" + tail` where `tail` is 150 MiB — construction peaks at ~2× (~300 MiB, within the address space, so the model's own allocation succeeds and any OOM belongs to the defect path), and the pre-fix whole-tail re-buffer added a third ~150 MiB copy past 384 MiB; the sliced prefix lets the run truncate and complete (Linux-only RLIMIT_AS repro, macOS happy path — the fixture's own construction must fit the address space, a general rule for these RLIMIT_AS cases). An output-budget/address-space case asserts a `maxLogBytes` of 50 MB AND a `maxValueBytes` of 50 MB each reject at load against a 256 MiB `addressSpaceMb` (past the room left after the interpreter baseline when multiplied by the worst-case 12) while the default caps against 512 MiB load, gating both budgets symmetrically; a discriminating case asserts a 48 MiB `maxLogBytes` against a 512 MiB `addressSpaceMb` rejects — 48×8 = 384 MiB fits the 448 MiB budgetable (the old 8× multiple wrongly admitted it) but 48×12 = 576 MiB does not, and that config is exactly the one whose settlement flush holds the pending chunks, their join, and the encode copy at ~12×. An inherited-RLIMIT_AS case runs the interpreter through a `ulimit -v 131072` wrapper with a 32 MiB `maxLogBytes` the configured 512 MiB `addressSpaceMb` admits, and asserts the boot re-check rejects it as an `exception` whose message names the inherited RLIMIT_AS (the 128 MiB inherited limit leaves too little after the baseline; Linux-only, macOS ignores `ulimit -v` and the run proceeds). A non-integer-budget case asserts a fractional `maxLogBytes`/`maxValueBytes` rejects at load. +- `tests/runtime.spec.ts` — the output-cap case asserts the `ceiling - envelope` bound (268435392) and its message. A daemon-thread case drives four threads emitting unterminated writes through settlement's flush. A native-write case writes 200 KiB with no newline via `os.write` under a raised `maxLogBytes` and asserts it reads back as EXACTLY one log entry (proving stray output is aggregated by line, not split at pipe-chunk boundaries); a companion writes `b"one\ntwo\nthree"` and asserts three entries (proving real newlines still delimit). A newline-free-flood case writes 2 MiB under a 4 KiB `maxLogBytes` and asserts the capture ends at the truncation marker and stays under budget (proving the residual is bounded by the ledger, not buffered whole); a NUL-flood companion writes 4000 newline-free NULs under the same budget and asserts truncation (proving the residual is charged by SERIALIZED cost, ~6× raw, measured without allocating the escaped copy); an illegal-UTF-8 case paces single-byte `\xff` writes under a 3072-byte budget with `Buffer.concat` wrapped to measure the peak merged buffer, asserting it stays under 2048 (charged at the U+FFFD width 3 the residual flushes near 1024 raw bytes; a raw-byte undercount would let it reach ~3072, so the bound discriminates); a CESU-8/overlong case paces the structurally-well-formed but illegal `ED A0 80` one byte at a time and asserts the same peak bound (charged at the true 9 per sequence it flushes early; charging the structural width 3 triples the peak, so reverting the per-lead range check turns it red); a broken-multibyte case writes a 3-byte lead then a fresh ASCII byte in separate chunks and asserts both a captured `A` and a U+FFFD (exercising `accrueStrayCost`'s cross-chunk broken-sequence branch); a post-truncation case writes a 108-byte payload (under the smallest PIPE_BUF, so one atomic write) whose first line exhausts a 64-byte budget and asserts the second line is dropped (exercising the post-truncation admit no-op in one `data` callback, no v8-ignore); a short-escape case writes a line mixing a tab, quote, backslash, a `\uXXXX` control, a multibyte character, and ASCII, asserting it round-trips verbatim (exercising every branch of `jsonStringCostUpTo`); a reassembly case writes a payload spanning every valid multibyte lead class (E0-range, plain 3-byte, F0, and F4) past the pipe buffer and asserts it round-trips with no U+FFFD (exercising `accrueStrayCost`'s per-lead ranges and cross-chunk reassembly); a lone-surrogate case forges an fd-3 `log` frame flooding 1000 `\ud800` escapes under a 4 KiB budget and asserts truncation (the count sits in the window where charging 3 bytes would admit and 6 bytes truncates, proving the surrogate is charged its full escaped width); a stray-sealing case paces 60000 single-byte newline-free `os.write(1, …)` calls under a raised budget with `Buffer.concat` wrapped to measure copy volume, asserting the trickle coalesces to one entry and the cumulative copy stays under a measured 256 KiB threshold (the sealed shape copies ~120 KB, the re-merge shape ~538 KB, so reverting the seal to a re-merge turns the assertion red — proving the fragment list seals into blocks past `MAX_PENDING_CHUNKS`). A closeDeadline-flush case has the leader write a newline-free diagnostic then spawn a `setsid` orphan holding the pipes open, and asserts the diagnostic survives in `logs` (proving the residual is flushed before the deadline destroys the streams). The same-group reap case spawns a SIGTERM-ignoring same-group descendant that releases the pipes and bumps a heartbeat file; the test asserts the heartbeat STOPS after the grace-window SIGKILL — an assertion robust whether the killed descendant is reaped or lingers as a zombie, so it holds where PID 1 does not wait() orphans. A dispose-after-resolve case asserts `dispose()` of a completed run with a same-group survivor returns only after the survivor stops executing (proving the run stays in `live` until its group is reaped), with an `expect(afterDispose).toBeGreaterThan(0)` guard so the frozen-heartbeat assertion cannot pass vacuously when the file was never written. A deadline case busy-blocks the event loop past both timers and asserts the survivor's heartbeat freezes (proving the poll's deadline arm sends SIGKILL itself rather than cancelling the unfired escalation). The cross-loop case runs a binding from a worker thread's own `asyncio.run` loop while the main coroutine yields with `await asyncio.sleep`, asserting the reply round-trips instead of timing out; a companion case abandons a thread's call so its loop closes, then answers it before a later binding — asserting the pump survives the closed-loop `call_soon_threadsafe` (host-gated ordering makes it deterministic, fail-before hangs the later binding to the wall clock). The inherited-soft-limit case runs the interpreter through a `ulimit -S -t` wrapper that sets a CPU soft limit below `cpuSeconds` and asserts the applied `RLIMIT_CPU` soft is the inherited value, not the configured one (CPU rather than address space, since macOS ignores `ulimit -v`); a companion inherits a 1 s CPU soft, has the program trap SIGXCPU and busy-loop past it, and asserts the settlement recheck reports a timeout — proving the recheck uses the effective soft, not the configured `cpuSeconds`. A control-heavy-diagnostic case raises a NUL-flood exception under a small `maxValueBytes` and asserts the serialized frame fits (proving the diagnostic is metered by serialized cost). A tail-copy case (`maxLogBytes: 256`, `addressSpaceMb: 384`) has the program build a tail in a variable and write `"\n" + tail` where `tail` is 150 MiB — construction peaks at ~2× (~300 MiB, within the address space, so the model's own allocation succeeds and any OOM belongs to the defect path), and the pre-fix whole-tail re-buffer added a third ~150 MiB copy past 384 MiB; the sliced prefix lets the run truncate and complete (Linux-only RLIMIT_AS repro, macOS happy path — the fixture's own construction must fit the address space, a general rule for these RLIMIT_AS cases). An output-budget/address-space case asserts a `maxLogBytes` of 50 MB AND a `maxValueBytes` of 50 MB each reject at load against a 256 MiB `addressSpaceMb` (past the room left after the interpreter baseline when multiplied by the worst-case 12) while the default caps against 512 MiB load, gating both budgets symmetrically; a discriminating case asserts a 48 MiB `maxLogBytes` against a 512 MiB `addressSpaceMb` rejects — 48×8 = 384 MiB fits the 448 MiB budgetable (the old 8× multiple wrongly admitted it) but 48×12 = 576 MiB does not, and that config is exactly the one whose settlement flush holds the pending chunks, their join, and the encode copy at ~12×. An inherited-RLIMIT_AS case runs the interpreter through a `ulimit -v 131072` wrapper with a 32 MiB `maxLogBytes` the configured 512 MiB `addressSpaceMb` admits, and asserts the boot re-check rejects it as an `exception` whose message names the inherited RLIMIT_AS (the 128 MiB inherited limit leaves too little after the baseline; Linux-only, macOS ignores `ulimit -v` and the run proceeds). A non-integer-budget case asserts a fractional `maxLogBytes`/`maxValueBytes` rejects at load. A combined-peak case (`maxLogBytes: 32 MiB`, `maxValueBytes: 32 MiB`, `addressSpaceMb: 512` — each budget admitted alone at 12×) writes ~33M newline-free astral characters (buffered, unflushed) then returns ~33M astral characters, and asserts the run settles as `output-limit` (the value is itself over its 32 MiB budget); pre-fix the unflushed log pending plus the value's build-and-encode peak added past the 512 MiB address space and OOM'd, so flushing the logs before framing the value is what lets the value check complete (Linux-only RLIMIT_AS repro; on macOS the over-budget value reports output-limit under both orders). ## Alternatives considered @@ -102,4 +102,4 @@ One residual write-path copy is fixed alongside, independent of the config gate: ## Consequences -The seam's resolve-don't-reject contract holds on the boot-write path and the synchronous-spawn-failure path, both with measured coverage, and neither strands a staging directory. Log capture is thread-safe at the cost of one re-entrant lock acquisition per write and flush, and stray native output is delimited by its own newlines rather than by transport chunks. Fd-3 residual memory is bounded by the actual retained bytes, and both frame readers scan an accumulating frame once rather than quadratically. The output caps admit every value a frame can carry and reject a non-integer budget at load. Disposal is genuinely quiescent against a same-group survivor — bounded by `graceMs + 2 * CLOSE_REAP_MARGIN_MS`, zero-cost when the group is already empty, with the SIGKILL timer cleared once the group empties so a stale kill cannot strike a recycled pgid — RLIMIT enforcement keeps the strictest of configured and inherited on both soft and hard (and the SIGXCPU diagnostic no longer names a budget the host cannot guarantee), bindings called from model-created threads complete instead of timing out, and the handshake frame reader no longer burns the CPU budget on a large program. Every behavioral fix carries a test that fails without it, except the three called out in the Problem section — the chunked frame read (a syscall-count improvement), the confirmed-empty finalize (whose only seam-observable effect freezes at signal delivery, which the pre-fix code also produced), and the shared stdout/stderr budget (whose only seam-observable difference turns on nondeterministic cross-pipe arrival timing) — so a future regression on the rest goes red. +The seam's resolve-don't-reject contract holds on the boot-write path and the synchronous-spawn-failure path, both with measured coverage, and neither strands a staging directory. Log capture is thread-safe at the cost of one re-entrant lock acquisition per write and flush, and stray native output is delimited by its own newlines rather than by transport chunks. Fd-3 residual memory is bounded by the actual retained bytes, and both frame readers scan an accumulating frame once rather than quadratically. The output caps admit every value a frame can carry and reject a non-integer budget at load. Disposal is genuinely quiescent against a same-group survivor — bounded by `graceMs + 2 * CLOSE_REAP_MARGIN_MS`, zero-cost when the group is already empty, with the SIGKILL timer cleared once the group empties so a stale kill cannot strike a recycled pgid — RLIMIT enforcement keeps the strictest of configured and inherited on both soft and hard (and the SIGXCPU diagnostic no longer names a budget the host cannot guarantee), bindings called from model-created threads complete instead of timing out, and the handshake frame reader no longer burns the CPU budget on a large program. Every behavioral fix carries a test that fails without it, except the four called out in the Problem section — the chunked frame read (a syscall-count improvement), the confirmed-empty finalize (whose only seam-observable effect freezes at signal delivery, which the pre-fix code also produced), the shared stdout/stderr budget (whose only seam-observable difference turns on nondeterministic cross-pipe arrival timing), and the `flush_line` reorder (whose lowered peak stays within what the 12x gate already admits, so no config behaves differently) — so a future regression on the rest goes red. diff --git a/.agents/notes/implemented/bug-fix/2026-07-31-code-runtime-python-settlement-fixes.zh.md b/.agents/notes/implemented/bug-fix/2026-07-31-code-runtime-python-settlement-fixes.zh.md index 1d45ceffda..c1320696c6 100644 --- a/.agents/notes/implemented/bug-fix/2026-07-31-code-runtime-python-settlement-fixes.zh.md +++ b/.agents/notes/implemented/bug-fix/2026-07-31-code-runtime-python-settlement-fixes.zh.md @@ -6,7 +6,7 @@ Status: implemented ## Problem -用于 Code Mode 的 CPython 子进程后端建立在 [fd-3 帧协议](../architecture/2026-07-31-code-runtime-python-fd3-protocol.md)之上，把每个程序结果都 resolve 成一个 `CodeRunResult`，仅在 seam 被误用时才 reject `run()`，并且会 dispose 到完全停稳，从而没有任何留在子进程自己进程组内的子进程存活得比 fiber 更久（一个用 `setsid()` 逃出该进程组的后代是有文档记载的例外——见该包 README 的 Known Limitations）。一连串审查暴露出一些缺陷，它们以单元测试覆盖率无法捕获的方式破坏了这些契约：每一个都藏在一处 `/* v8 ignore */` 之后、一个读起来像修复但实际并非修复的捕获可调用对象之后、一处透过 seam 不可见的内存效应之后、一处重复计数的加载期上界之后、一处存活者能够熬过的进程组升级之后、一处静默死锁的跨事件循环完成之后、一处位于结算路径之外的同步抛出之后，或者一处被当作日志边界处理的传输边界之后。大多数行为修复都附带一个在缺少它时会失败的测试；有三处没有，并被如此标注——分块读取帧（一处系统调用次数的改进，没有可跨平台确定性断言的失败）、确认为空后的收尾（它唯一透过 seam 可观测的效应，即一个冻结的心跳，会在 SIGKILL 被投递的瞬间冻结，而修复前"投递即收尾"的代码也会产生同样的结果，用于区分的探测手段是 Alternatives 以跨环境不可靠为由否决的 signal-0 检查），以及共享的 stdout／stderr 预算（它唯一透过 seam 可观测的差异，是一次流中冲刷落在哪条条目边界上，而这取决于两条相互独立的 OS 管道的相对到达时机，`os.sched_yield` 并不能使其确定；它所强化的按管道计的内存界限确实由单管道洪泛测试覆盖）。 +用于 Code Mode 的 CPython 子进程后端建立在 [fd-3 帧协议](../architecture/2026-07-31-code-runtime-python-fd3-protocol.md)之上，把每个程序结果都 resolve 成一个 `CodeRunResult`，仅在 seam 被误用时才 reject `run()`，并且会 dispose 到完全停稳，从而没有任何留在子进程自己进程组内的子进程存活得比 fiber 更久（一个用 `setsid()` 逃出该进程组的后代是有文档记载的例外——见该包 README 的 Known Limitations）。一连串审查暴露出一些缺陷，它们以单元测试覆盖率无法捕获的方式破坏了这些契约：每一个都藏在一处 `/* v8 ignore */` 之后、一个读起来像修复但实际并非修复的捕获可调用对象之后、一处透过 seam 不可见的内存效应之后、一处重复计数的加载期上界之后、一处存活者能够熬过的进程组升级之后、一处静默死锁的跨事件循环完成之后、一处位于结算路径之外的同步抛出之后，或者一处被当作日志边界处理的传输边界之后。大多数行为修复都附带一个在缺少它时会失败的测试；有四处没有，并被如此标注——分块读取帧（一处系统调用次数的改进，没有可跨平台确定性断言的失败）、确认为空后的收尾（它唯一透过 seam 可观测的效应，即一个冻结的心跳，会在 SIGKILL 被投递的瞬间冻结，而修复前"投递即收尾"的代码也会产生同样的结果，用于区分的探测手段是 Alternatives 以跨环境不可靠为由否决的 signal-0 检查）、共享的 stdout／stderr 预算（它唯一透过 seam 可观测的差异，是一次流中冲刷落在哪条条目边界上，而这取决于两条相互独立的 OS 管道的相对到达时机，`os.sched_yield` 并不能使其确定；它所强化的按管道计的内存界限确实由单管道洪泛测试覆盖），以及 `flush_line` 的 join-清空-push 重排序（它把结算期冲刷的峰值从三份副本降到两份，但 12× 加载门本就覆盖了换行路径的三副本峰值，因此每个被门放行的配置在两种顺序下都落在地址空间之内、不存在透过 seam 可观测的差异——该内存效应在 Python 子进程内部，与共享预算那处一样无法透过 seam 度量）。 ## Decision @@ -58,7 +58,7 @@ Status: implemented ### An incompatible output-budget/addressSpaceMb pair is rejected at load -子进程（[`py/bootstrap.py`](../../../../packages/code-runtime/code-runtime-python/py/bootstrap.py)）在 `RLIMIT_AS` 之下构建、计费并分帧一条 `maxLogBytes` 的日志条目或一个 `maxValueBytes` 的完成值，而两个账本都是按字符计数、对照一个序列化字节预算触发的。一个星芒面字符是一个字符，但占 CPython `str` 存储的四个字节以及四个 UTF-8 字节，且峰值时有三份这样的副本同时存活：换行路径上一次 `sys.stdout.write(line + "\n")` 会持有调用方的 `text` 实参（在整个 `write` 调用期间存活，约 4 倍）、交给 `LogBuffer.push` 的行切片（约 4 倍）、以及 `_push_locked` 为计费和发送而取的 `text.encode("utf-8")` 副本（约 4 倍）；结算期的 `flush_line` 路径则持有 pending 分块、它们的 `"".join(...)` 以及同一份 encode 副本——峰值约为预算的 12 倍。当一项预算逼近 `addressSpaceMb` 时，一次合法的、接近预算的输出会在那次构建加编码期间突破地址空间，并作为 `worker-exit`（日志）而不是截断而终止，或作为 `output-limit`（值）而失败。在运行时对每次子进程写入按地址空间计量是错误的修复：热路径上一次精确的序列化开销检查，要么是一次完整的 `encode`（正是要避免的那次分配），要么是一个逐字符的 Python 循环（它会烧掉 CPU 预算——一次 10 MB 的合法写入会在 `cpuSeconds: 1` 之下触发 SIGXCPU）。两者都是拿一种资源界限换另一种。取而代之，[`src/index.ts`](../../../../packages/code-runtime/code-runtime-python/src/index.ts) 在加载期（LOAD）拒绝这个不兼容的组合：每项预算乘以 `OUTPUT_BUDGET_WORST_CASE_ADDRESS_SPACE_MULTIPLE`（十二——三份同时存在的约 4 倍副本）必须放得进为解释器自身占用预留一份固定的 `INTERPRETER_BASELINE_BYTES` 之后剩下的地址空间，并用一个 `>=`，使得一项其最坏情况峰值恰好等于那片余量的预算也会被拒绝（该峰值加上预留的基线正好是整个地址空间，即 `RLIMIT_AS` 边界）。`flush_line` 也被改为在 push 之前先丢弃 pending 分块，与换行路径的 join-清空-push 顺序一致，使该路径至多只持有 join 及其 encode 副本，而非三份副本。该基线是与倍数分开（SEPARATELY）预留的，因为它是一项固定开销，而非随预算伸缩的开销：把它折进倍数会让一项恰好为 `addressSpaceMb / 12` 大小的预算被放行，而其峰值加上解释器仍会越界。`maxLogBytes` 和 `maxValueBytes` 都被对称地门控；值路径的构建加编码是同一形态。该检查在每个平台上都运行，而不仅在强制 `RLIMIT_AS` 的平台上：这种不兼容是那些配置值的属性，因此一个 Linux 部署无论由哪个宿主组装配置都会 OOM，而一致的加载期拒绝正是 fail-loud 契约（Darwin 仅跳过运行时的 `setrlimit`）。这在配置 seam 处消除了这一类问题，而不是给写入路径打补丁，因此 `_LogStream` 保留它原有的按字符计数的缓冲（一个对序列化开销有效的下界，一旦预算放进地址空间就是内存安全的）。 +子进程（[`py/bootstrap.py`](../../../../packages/code-runtime/code-runtime-python/py/bootstrap.py)）在 `RLIMIT_AS` 之下构建、计费并分帧一条 `maxLogBytes` 的日志条目或一个 `maxValueBytes` 的完成值，而两个账本都是按字符计数、对照一个序列化字节预算触发的。一个星芒面字符是一个字符，但占 CPython `str` 存储的四个字节以及四个 UTF-8 字节，且最重的路径峰值时有三份这样的副本同时存活：一次 `sys.stdout.write(line + "\n")` 会持有调用方的 `text` 实参（在整个 `write` 调用期间存活，约 4 倍）、交给 `LogBuffer.push` 的行切片（约 4 倍）、以及 `_push_locked` 为计费和发送而取的 `text.encode("utf-8")` 副本（约 4 倍）——峰值约为预算的 12 倍。结算期的 `flush_line` 路径只持有两份（它的 `"".join(...)` 与那份 encode 副本——它在 push 之前先丢弃 pending 分块），因此换行路径才是起约束作用的最坏情况。当一项预算逼近 `addressSpaceMb` 时，一次合法的、接近预算的输出会在那次构建加编码期间突破地址空间，并作为 `worker-exit`（日志）而不是截断而终止，或作为 `output-limit`（值）而失败。在运行时对每次子进程写入按地址空间计量是错误的修复：热路径上一次精确的序列化开销检查，要么是一次完整的 `encode`（正是要避免的那次分配），要么是一个逐字符的 Python 循环（它会烧掉 CPU 预算——一次 10 MB 的合法写入会在 `cpuSeconds: 1` 之下触发 SIGXCPU）。两者都是拿一种资源界限换另一种。取而代之，[`src/index.ts`](../../../../packages/code-runtime/code-runtime-python/src/index.ts) 在加载期（LOAD）拒绝这个不兼容的组合：每项预算乘以 `OUTPUT_BUDGET_WORST_CASE_ADDRESS_SPACE_MULTIPLE`（十二——换行路径三份同时存在的约 4 倍副本）必须放得进为解释器自身占用预留一份固定的 `INTERPRETER_BASELINE_BYTES` 之后剩下的地址空间，并用一个 `>=`，使得一项其最坏情况峰值恰好等于那片余量的预算也会被拒绝（该峰值加上预留的基线正好是整个地址空间，即 `RLIMIT_AS` 边界）。`flush_line` 也被改为在 push 之前先丢弃 pending 分块，与换行路径的 join-清空-push 顺序一致，使它至多只持有 join 及其 encode 副本，而非三份副本。该基线是与倍数分开（SEPARATELY）预留的，因为它是一项固定开销，而非随预算伸缩的开销：把它折进倍数会让一项恰好为 `addressSpaceMb / 12` 大小的预算被放行，而其峰值加上解释器仍会越界。`maxLogBytes` 和 `maxValueBytes` 都被对称地门控；值路径的构建加编码是同一形态。该检查在每个平台上都运行，而不仅在强制 `RLIMIT_AS` 的平台上：这种不兼容是那些配置值的属性，因此一个 Linux 部署无论由哪个宿主组装配置都会 OOM，而一致的加载期拒绝正是 fail-loud 契约（Darwin 仅跳过运行时的 `setrlimit`）。这在配置 seam 处消除了这一类问题，而不是给写入路径打补丁，因此 `_LogStream` 保留它原有的按字符计数的缓冲（一个对序列化开销有效的下界，一旦预算放进地址空间就是内存安全的）。 宿主门控是对照配置的（CONFIGURED）`addressSpaceMb` 校验的，但一个启动环境可能继承一个更严格的（STRICTER）`RLIMIT_AS`（一个低于 `addressSpaceMb` 的 `ulimit -v` 包装层），而 bootstrap 的 `_clamped` 会正确地把有效（EFFECTIVE）限制降到该值——从而让这些预算是按一个子进程永远得不到的上限来定尺寸的。因此 `bootstrap.py` 在应用该有效被夹紧的软限制之后，会对照它重新检查两项预算，镜像宿主门控的倍数与基线，并在引导期抛出（被 setrlimit 阶段的处理器捕获，并作为 `exception` 上报——与任何其他资源限制应用失败同属一类），而不是任由一次接近预算的输出在运行途中 OOM。这两个子进程侧常量与宿主侧的保持一致，靠的是共享的推理，而不是一个 wire 字段。 @@ -68,7 +68,7 @@ Status: implemented - `tests/boot-write-failure.spec.ts` 对 `spawn` 做 mock，使 fd-3 管道在引导写入时抛出异常（这是真实子进程无法被迫进入的唯一路径），并断言 `run()` resolve 出一个 `worker-exit` 而非 reject。一个同级用例让被 mock 的 `spawn` 同步抛出，并断言 `run()` 仍然 resolve 出一个 `worker-exit`，且会移除它的暂存目录——以被 mock 的 `spawn` 在其 argv 中收到的确切引导路径为准，因此一个同级 worker 的并发暂存不会让它变得不稳定。两者都被隔离在这个 spec 中，因此真实子进程测试套件不受影响。 - `tests/residual-detach.spec.ts` 对 `detachResidual` 做单元测试：向前传递的副本与残余数据相等、拥有一个大小与其自身长度一致的底层存储（fixture 保持在 Node 的 Buffer 池阈值之上），并且不与源帧的 `ArrayBuffer` 共享。 -- `tests/runtime.spec.ts`：output-cap 用例断言 `ceiling - envelope` 上界（268435392）及其消息。一个 daemon 线程用例驱动四个线程穿过结算的 flush 发出未结束的写入。一个 native-write 用例在抬高后的 `maxLogBytes` 之下，通过 `os.write` 写入 200 KiB 且不含换行符，断言它回读时恰好是一条日志条目（证明散逸输出是按行聚合的，而不是在管道分片边界处被切开）；一个配套用例写入 `b"one\ntwo\nthree"`，断言得到三条条目（证明真正的换行符仍然起分隔作用）。一个 newline-free-flood 用例在一个 4 KiB 的 `maxLogBytes` 之下写入 2 MiB，断言捕获终止于截断标记且保持在预算之内（证明残余数据受账本约束，而不是被整体缓冲）；一个 NUL-flood 配套用例在同一预算之下写入 4000 个不含换行符的 NUL，断言发生截断（证明残余数据是按序列化开销计费的，约为原始的 6 倍，且在度量时不分配转义后的副本）；一个 illegal-UTF-8 用例在一个 3072 字节的预算之下控速发出单字节 `\xff` 写入，并对 `Buffer.concat` 做包装以度量峰值合并缓冲区，断言它保持在 2048 之下（按 U+FFFD 宽度 3 计费时残余数据在约 1024 原始字节处冲刷；一次原始字节的少计会让它达到约 3072，因此该界限具有区分力）；一个 CESU-8/overlong 用例把结构良构但非法的 `ED A0 80` 一次一个字节地控速发出，断言同样的峰值界限（按每序列真实的 9 计费时它提前冲刷；按结构宽度 3 计费会使峰值增至三倍，因此把逐前导字节范围检查回退会使它变红）；一个 broken-multibyte 用例在分开的分片里先写入一个 3 字节的前导字节、再写入一个新的 ASCII 字节，断言同时捕获到一个 `A` 和一个 U+FFFD（覆盖 `accrueStrayCost` 的跨分片断裂序列分支）；一个 post-truncation 用例写入一个 108 字节的载荷（小于最小的 PIPE_BUF，因此是一次原子写入），其首行耗尽一个 64 字节的预算，断言第二行被丢弃（覆盖单次 `data` 回调中的截断后准入空操作，无需 v8-ignore）；一个 short-escape 用例写入一行混合了制表符、引号、反斜杠、一个 `\uXXXX` 控制字符、一个多字节字符和 ASCII 的内容，断言它原样完成往返（覆盖 `jsonStringCostUpTo` 的每一条分支）；一个 reassembly 用例写入一个跨越每个合法多字节前导字节类别（E0 范围、普通 3 字节、F0 和 F4）、越过管道缓冲区的载荷，断言它原样完成往返且不含 U+FFFD（覆盖 `accrueStrayCost` 的逐前导字节范围与跨分片重组）；一个 lone-surrogate 用例在一个 4 KiB 预算之下伪造一个以 1000 个 `\ud800` 转义洪泛的 fd-3 `log` 帧，断言发生截断（该计数正落在计 3 字节会放行、计 6 字节则截断的窗口内，证明该代理项是按其完整转义宽度计费的）；一个 stray-sealing 用例在抬高后的预算之下控速发出 60000 次单字节、不含换行符的 `os.write(1, …)` 调用，并对 `Buffer.concat` 做包装以度量复制量，断言这股细流合并为一条条目、且累积复制量保持在一个实测的 256 KiB 阈值之下（封存后的形态复制约 120 KB，重新合并的形态复制约 538 KB，因此把封存回退成重新合并会使该断言变红——证明分片列表在越过 `MAX_PENDING_CHUNKS` 后封存为块）。一个 closeDeadline-flush 用例让 leader 写入一段不含换行符的诊断，随后 spawn 一个持有管道不放的 `setsid` 孤儿进程，断言该诊断在 `logs` 中存留下来（证明残余数据在截止时间销毁流之前被冲刷）。same-group 回收用例 spawn 一个忽略 SIGTERM 的同进程组后代，它释放管道并递增一个心跳文件；该测试断言在宽限窗口的 SIGKILL 之后心跳停止：无论被杀死的后代是被回收还是作为僵尸进程滞留，这个断言都成立，因此它在 PID 1 不 wait() 孤儿进程的环境下同样成立。一个 dispose-after-resolve 用例断言，对一个已完成、且存在同进程组存活者的运行调用 `dispose()`，只有在该存活者停止执行之后才返回（证明该运行会一直留在 `live` 中，直到它的进程组被回收），并带有一个 `expect(afterDispose).toBeGreaterThan(0)` 守卫，使得当心跳文件从未被写入时，冻结心跳的断言不会被空洞地通过。一个 deadline 用例忙阻塞事件循环越过两个定时器，断言该存活者的心跳冻结（证明轮询的截止时间分支自身发送 SIGKILL，而不是取消尚未触发的升级）。cross-loop 用例在主协程通过 `await asyncio.sleep` 让出时，从一个工作线程自己的 `asyncio.run` 事件循环运行一个绑定，断言该回复完成往返而不是超时；一个配套用例放弃某个线程的调用，使其事件循环关闭，随后在一个后续绑定之前回答它——断言 pump 在关闭事件循环上的 `call_soon_threadsafe` 之后仍然存活（由宿主门控的顺序使其具有确定性，未修复时会把后续绑定拖到墙钟上挂起）。inherited-soft-limit 用例通过一个 `ulimit -S -t` 包装脚本运行解释器，将 CPU 软限制设为低于 `cpuSeconds`，并断言实际应用的 `RLIMIT_CPU` 软限制是继承来的值，而不是配置的值（用 CPU 而非地址空间，因为 macOS 忽略 `ulimit -v`）。一个配套用例继承 1 秒的 CPU 软限制，让程序捕获 SIGXCPU 并忙循环越过它，断言结算复查报告 timeout——证明复查用的是实际生效的软限制，而不是配置的 `cpuSeconds`。一个 control-heavy-diagnostic 用例在一个较小的 `maxValueBytes` 之下抛出一个 NUL 洪泛异常，断言序列化后的帧能放得下（证明该诊断是按序列化开销计量的）。一个 tail-copy 用例（`maxLogBytes: 256`、`addressSpaceMb: 384`）让程序在一个变量里构建一个尾部并写入 `"\n" + tail`，其中 `tail` 为 150 MiB——构建峰值约 2 倍（约 300 MiB，落在地址空间之内，因此模型自身的分配会成功，任何 OOM 都属于缺陷路径），而修复前的整尾重新缓冲会加上第三份约 150 MiB 的副本、越过 384 MiB；切片后的前缀让该次运行得以截断并完成（仅 Linux 的 RLIMIT_AS 复现，macOS 走顺利路径——fixture 自身的构建必须放进地址空间，这是这些 RLIMIT_AS 用例的一条通用规则）。一个 output-budget/address-space 用例断言一个 50 MB 的 `maxLogBytes` 和一个 50 MB 的 `maxValueBytes` 各自对照一个 256 MiB 的 `addressSpaceMb` 在加载期被拒绝（乘以最坏情况的 12 之后超过解释器基线之后剩下的余量），而默认的各项上限对照 512 MiB 则加载成功，对两项预算对称地门控；一个具区分力的用例断言一个 48 MiB 的 `maxLogBytes` 对照一个 512 MiB 的 `addressSpaceMb` 被拒绝——48×8 = 384 MiB 放得进 448 MiB 的可预算余量（旧的 8× 倍数会错误放行），但 48×12 = 576 MiB 放不进，而正是这个配置的结算期 flush 会以约 12× 同时持有 pending 分块、它们的 join 与 encode 副本。一个 inherited-RLIMIT_AS 用例通过一个 `ulimit -v 131072` 包装层运行解释器，配以一个配置的 512 MiB `addressSpaceMb` 所允许的 32 MiB `maxLogBytes`，断言引导期的重新检查把它作为 `exception` 拒绝、且其消息点名了继承的 RLIMIT_AS（128 MiB 的继承限制在基线之后剩下的太少；仅 Linux，macOS 忽略 `ulimit -v`，该次运行会继续）。一个 non-integer-budget 用例断言一个小数的 `maxLogBytes`／`maxValueBytes` 在加载期被拒绝。 +- `tests/runtime.spec.ts`：output-cap 用例断言 `ceiling - envelope` 上界（268435392）及其消息。一个 daemon 线程用例驱动四个线程穿过结算的 flush 发出未结束的写入。一个 native-write 用例在抬高后的 `maxLogBytes` 之下，通过 `os.write` 写入 200 KiB 且不含换行符，断言它回读时恰好是一条日志条目（证明散逸输出是按行聚合的，而不是在管道分片边界处被切开）；一个配套用例写入 `b"one\ntwo\nthree"`，断言得到三条条目（证明真正的换行符仍然起分隔作用）。一个 newline-free-flood 用例在一个 4 KiB 的 `maxLogBytes` 之下写入 2 MiB，断言捕获终止于截断标记且保持在预算之内（证明残余数据受账本约束，而不是被整体缓冲）；一个 NUL-flood 配套用例在同一预算之下写入 4000 个不含换行符的 NUL，断言发生截断（证明残余数据是按序列化开销计费的，约为原始的 6 倍，且在度量时不分配转义后的副本）；一个 illegal-UTF-8 用例在一个 3072 字节的预算之下控速发出单字节 `\xff` 写入，并对 `Buffer.concat` 做包装以度量峰值合并缓冲区，断言它保持在 2048 之下（按 U+FFFD 宽度 3 计费时残余数据在约 1024 原始字节处冲刷；一次原始字节的少计会让它达到约 3072，因此该界限具有区分力）；一个 CESU-8/overlong 用例把结构良构但非法的 `ED A0 80` 一次一个字节地控速发出，断言同样的峰值界限（按每序列真实的 9 计费时它提前冲刷；按结构宽度 3 计费会使峰值增至三倍，因此把逐前导字节范围检查回退会使它变红）；一个 broken-multibyte 用例在分开的分片里先写入一个 3 字节的前导字节、再写入一个新的 ASCII 字节，断言同时捕获到一个 `A` 和一个 U+FFFD（覆盖 `accrueStrayCost` 的跨分片断裂序列分支）；一个 post-truncation 用例写入一个 108 字节的载荷（小于最小的 PIPE_BUF，因此是一次原子写入），其首行耗尽一个 64 字节的预算，断言第二行被丢弃（覆盖单次 `data` 回调中的截断后准入空操作，无需 v8-ignore）；一个 short-escape 用例写入一行混合了制表符、引号、反斜杠、一个 `\uXXXX` 控制字符、一个多字节字符和 ASCII 的内容，断言它原样完成往返（覆盖 `jsonStringCostUpTo` 的每一条分支）；一个 reassembly 用例写入一个跨越每个合法多字节前导字节类别（E0 范围、普通 3 字节、F0 和 F4）、越过管道缓冲区的载荷，断言它原样完成往返且不含 U+FFFD（覆盖 `accrueStrayCost` 的逐前导字节范围与跨分片重组）；一个 lone-surrogate 用例在一个 4 KiB 预算之下伪造一个以 1000 个 `\ud800` 转义洪泛的 fd-3 `log` 帧，断言发生截断（该计数正落在计 3 字节会放行、计 6 字节则截断的窗口内，证明该代理项是按其完整转义宽度计费的）；一个 stray-sealing 用例在抬高后的预算之下控速发出 60000 次单字节、不含换行符的 `os.write(1, …)` 调用，并对 `Buffer.concat` 做包装以度量复制量，断言这股细流合并为一条条目、且累积复制量保持在一个实测的 256 KiB 阈值之下（封存后的形态复制约 120 KB，重新合并的形态复制约 538 KB，因此把封存回退成重新合并会使该断言变红——证明分片列表在越过 `MAX_PENDING_CHUNKS` 后封存为块）。一个 closeDeadline-flush 用例让 leader 写入一段不含换行符的诊断，随后 spawn 一个持有管道不放的 `setsid` 孤儿进程，断言该诊断在 `logs` 中存留下来（证明残余数据在截止时间销毁流之前被冲刷）。same-group 回收用例 spawn 一个忽略 SIGTERM 的同进程组后代，它释放管道并递增一个心跳文件；该测试断言在宽限窗口的 SIGKILL 之后心跳停止：无论被杀死的后代是被回收还是作为僵尸进程滞留，这个断言都成立，因此它在 PID 1 不 wait() 孤儿进程的环境下同样成立。一个 dispose-after-resolve 用例断言，对一个已完成、且存在同进程组存活者的运行调用 `dispose()`，只有在该存活者停止执行之后才返回（证明该运行会一直留在 `live` 中，直到它的进程组被回收），并带有一个 `expect(afterDispose).toBeGreaterThan(0)` 守卫，使得当心跳文件从未被写入时，冻结心跳的断言不会被空洞地通过。一个 deadline 用例忙阻塞事件循环越过两个定时器，断言该存活者的心跳冻结（证明轮询的截止时间分支自身发送 SIGKILL，而不是取消尚未触发的升级）。cross-loop 用例在主协程通过 `await asyncio.sleep` 让出时，从一个工作线程自己的 `asyncio.run` 事件循环运行一个绑定，断言该回复完成往返而不是超时；一个配套用例放弃某个线程的调用，使其事件循环关闭，随后在一个后续绑定之前回答它——断言 pump 在关闭事件循环上的 `call_soon_threadsafe` 之后仍然存活（由宿主门控的顺序使其具有确定性，未修复时会把后续绑定拖到墙钟上挂起）。inherited-soft-limit 用例通过一个 `ulimit -S -t` 包装脚本运行解释器，将 CPU 软限制设为低于 `cpuSeconds`，并断言实际应用的 `RLIMIT_CPU` 软限制是继承来的值，而不是配置的值（用 CPU 而非地址空间，因为 macOS 忽略 `ulimit -v`）。一个配套用例继承 1 秒的 CPU 软限制，让程序捕获 SIGXCPU 并忙循环越过它，断言结算复查报告 timeout——证明复查用的是实际生效的软限制，而不是配置的 `cpuSeconds`。一个 control-heavy-diagnostic 用例在一个较小的 `maxValueBytes` 之下抛出一个 NUL 洪泛异常，断言序列化后的帧能放得下（证明该诊断是按序列化开销计量的）。一个 tail-copy 用例（`maxLogBytes: 256`、`addressSpaceMb: 384`）让程序在一个变量里构建一个尾部并写入 `"\n" + tail`，其中 `tail` 为 150 MiB——构建峰值约 2 倍（约 300 MiB，落在地址空间之内，因此模型自身的分配会成功，任何 OOM 都属于缺陷路径），而修复前的整尾重新缓冲会加上第三份约 150 MiB 的副本、越过 384 MiB；切片后的前缀让该次运行得以截断并完成（仅 Linux 的 RLIMIT_AS 复现，macOS 走顺利路径——fixture 自身的构建必须放进地址空间，这是这些 RLIMIT_AS 用例的一条通用规则）。一个 output-budget/address-space 用例断言一个 50 MB 的 `maxLogBytes` 和一个 50 MB 的 `maxValueBytes` 各自对照一个 256 MiB 的 `addressSpaceMb` 在加载期被拒绝（乘以最坏情况的 12 之后超过解释器基线之后剩下的余量），而默认的各项上限对照 512 MiB 则加载成功，对两项预算对称地门控；一个具区分力的用例断言一个 48 MiB 的 `maxLogBytes` 对照一个 512 MiB 的 `addressSpaceMb` 被拒绝——48×8 = 384 MiB 放得进 448 MiB 的可预算余量（旧的 8× 倍数会错误放行），但 48×12 = 576 MiB 放不进，而正是这个配置的结算期 flush 会以约 12× 同时持有 pending 分块、它们的 join 与 encode 副本。一个 inherited-RLIMIT_AS 用例通过一个 `ulimit -v 131072` 包装层运行解释器，配以一个配置的 512 MiB `addressSpaceMb` 所允许的 32 MiB `maxLogBytes`，断言引导期的重新检查把它作为 `exception` 拒绝、且其消息点名了继承的 RLIMIT_AS（128 MiB 的继承限制在基线之后剩下的太少；仅 Linux，macOS 忽略 `ulimit -v`，该次运行会继续）。一个 non-integer-budget 用例断言一个小数的 `maxLogBytes`／`maxValueBytes` 在加载期被拒绝。一个 combined-peak 用例（`maxLogBytes: 32 MiB`、`maxValueBytes: 32 MiB`、`addressSpaceMb: 512`——每项预算单独都被 12× 门放行）写入约 33M 个不含换行符的星芒面字符（缓冲、未冲刷）后返回约 33M 个星芒面字符，断言该次运行以 `output-limit` 结算（该值本身就超过它 32 MiB 的预算）；修复前未冲刷的日志 pending 加上值的构建加编码峰值会一起越过 512 MiB 地址空间而 OOM，因此在分帧值之前先冲刷日志正是让值检查得以完成的原因（仅 Linux 的 RLIMIT_AS 复现；在 macOS 上超预算的值在两种顺序下都报 output-limit）。 ## Alternatives considered @@ -102,4 +102,4 @@ Status: implemented ## Consequences -seam 的"只 resolve、不 reject"契约在引导写入路径和同步 spawn 失败路径上都得以成立，两者的覆盖率都是被度量的，且两者都不会遗留一个暂存目录。日志捕获是线程安全的，代价是每次写入和 flush 都要获取一次可重入锁，并且散逸的原生输出由它自己的换行符来分隔，而不是由传输分片来分隔。fd-3 残余数据的内存受实际保留的字节数约束，并且两个帧读取器都以一次而非平方级的方式扫描一个不断累积的帧。输出上限放行一个帧所能承载的每一个值，并在加载期拒绝一个非整数的预算。dispose 面对同进程组存活者是真正完全停稳的（以 `graceMs + 2 * CLOSE_REAP_MARGIN_MS` 为界，在进程组已为空时代价为零，并且一旦进程组清空就清除 SIGKILL 定时器，从而一次滞留的 kill 无法击中一个被回收的 pgid），RLIMIT 强制在 soft 和 hard 两者上都保持配置值与继承值中的最严格者（并且 SIGXCPU 诊断不再把一个宿主无法保证的预算说出来），并且从模型创建的线程调用的绑定会完成而不是超时，而且握手帧读取器不再在一个大程序上烧掉 CPU 预算。每处行为修复都附带一个在缺少它时会失败的测试，除了 Problem 一节点出的那三处——分块读取帧（一处系统调用次数的改进）、确认为空后的收尾（它唯一透过 seam 可观测的效应会在信号投递时冻结，而修复前的代码也会产生同样的结果），以及共享的 stdout／stderr 预算（它唯一透过 seam 可观测的差异取决于不确定的跨管道到达时机）——因此其余各处未来若发生回归都会变红。 +seam 的"只 resolve、不 reject"契约在引导写入路径和同步 spawn 失败路径上都得以成立，两者的覆盖率都是被度量的，且两者都不会遗留一个暂存目录。日志捕获是线程安全的，代价是每次写入和 flush 都要获取一次可重入锁，并且散逸的原生输出由它自己的换行符来分隔，而不是由传输分片来分隔。fd-3 残余数据的内存受实际保留的字节数约束，并且两个帧读取器都以一次而非平方级的方式扫描一个不断累积的帧。输出上限放行一个帧所能承载的每一个值，并在加载期拒绝一个非整数的预算。dispose 面对同进程组存活者是真正完全停稳的（以 `graceMs + 2 * CLOSE_REAP_MARGIN_MS` 为界，在进程组已为空时代价为零，并且一旦进程组清空就清除 SIGKILL 定时器，从而一次滞留的 kill 无法击中一个被回收的 pgid），RLIMIT 强制在 soft 和 hard 两者上都保持配置值与继承值中的最严格者（并且 SIGXCPU 诊断不再把一个宿主无法保证的预算说出来），并且从模型创建的线程调用的绑定会完成而不是超时，而且握手帧读取器不再在一个大程序上烧掉 CPU 预算。每处行为修复都附带一个在缺少它时会失败的测试，除了 Problem 一节点出的那四处——分块读取帧（一处系统调用次数的改进）、确认为空后的收尾（它唯一透过 seam 可观测的效应会在信号投递时冻结，而修复前的代码也会产生同样的结果）、共享的 stdout／stderr 预算（它唯一透过 seam 可观测的差异取决于不确定的跨管道到达时机），以及 `flush_line` 重排序（它降低后的峰值仍落在 12× 门本就放行的范围内，因此没有配置会有不同表现）——因此其余各处未来若发生回归都会变红。 diff --git a/packages/code-runtime/code-runtime-python/py/bootstrap.py b/packages/code-runtime/code-runtime-python/py/bootstrap.py index 6bce4ee014..8ccd2991c6 100644 --- a/packages/code-runtime/code-runtime-python/py/bootstrap.py +++ b/packages/code-runtime/code-runtime-python/py/bootstrap.py @@ -890,6 +890,16 @@ async def _run(channel: ProtocolChannel) -> None: exec(code, ns) # noqa: S102 -- defines __dsh_main__; executing model code is the point value = await ns["__dsh_main__"]() die_if_cpu_exhausted(cpu_seconds) + # Flush the log buffers BEFORE metering and framing the completion value. + # `_done_with_value` materializes the value's escaped JSON form to meter + # it, and `send_done` encodes the frame — several copies of a near-budget + # value live at once (see OUTPUT_BUDGET_WORST_CASE_ADDRESS_SPACE_MULTIPLE). + # Any unflushed log pending would add its own bytes to that peak, so a + # `maxLogBytes` and a `maxValueBytes` each admitted alone by the load gate + # could together breach RLIMIT_AS. Flushing first frees the log pending so + # the value frame's peak stands alone against the address space. + flush_out() + flush_err() done = _done_with_value(value, max_value_bytes) except BaseException as exc: # noqa: BLE001 -- report every failure to host done = { @@ -911,7 +921,9 @@ async def _run(channel: ProtocolChannel) -> None: # Flush any print output not terminated by a newline (a traceback always # ends in one, but `print(x, end="")` or a bare write may not), so the - # final partial line is not silently dropped. + # final partial line is not silently dropped. The success path already + # flushed before framing the value; this is an idempotent no-op there and + # the flush the exception path needs. flush_out() flush_err() reply_task.cancel() diff --git a/packages/code-runtime/code-runtime-python/src/index.ts b/packages/code-runtime/code-runtime-python/src/index.ts index e8239df720..723fda695b 100644 --- a/packages/code-runtime/code-runtime-python/src/index.ts +++ b/packages/code-runtime/code-runtime-python/src/index.ts @@ -240,13 +240,14 @@ const CLOSE_REAP_MARGIN_MS = 2_000 * as a multiple of the budget. The child's ledgers trigger on CHARACTER count * against a serialized-BYTE budget, and an astral character is one character but * four bytes of CPython `str` storage and four UTF-8 bytes — so a budget's worth - * of astral characters is ~4x the budget in each string that holds it. THREE - * such copies are live at the peak: on the newline path a single - * `sys.stdout.write(line + "\n")` holds the caller's `text` argument (alive for + * of astral characters is ~4x the budget in each string that holds it. The + * heaviest path holds THREE such copies at once: a single + * `sys.stdout.write(line + "\n")` keeps the caller's `text` argument (alive for * the whole `write` call, ~4x), the line slice `text[pos:newline]` handed to * `LogBuffer.push` (~4x), and the `text.encode("utf-8")` copy `_push_locked` - * takes to charge and ship it (~4x); the settlement `flush_line` path holds the - * pending chunks, their `"".join(...)`, and that same encode copy. Twelve covers + * takes to charge and ship it (~4x). The settlement `flush_line` path holds only + * two (its `"".join(...)` and that encode copy — it drops the pending chunks + * before pushing), so the newline path is the binding worst case. Twelve covers * those three simultaneous ~4x copies. The interpreter baseline is NOT in this * multiple — it is reserved separately as {@link INTERPRETER_BASELINE_BYTES} — * because it is a fixed cost, not one that scales with the budget. Used to bound diff --git a/packages/code-runtime/code-runtime-python/tests/runtime.spec.ts b/packages/code-runtime/code-runtime-python/tests/runtime.spec.ts index 946fd84aa2..7b0572ab1a 100644 --- a/packages/code-runtime/code-runtime-python/tests/runtime.spec.ts +++ b/packages/code-runtime/code-runtime-python/tests/runtime.spec.ts @@ -3649,6 +3649,42 @@ describe('PythonCodeRuntime — hostile peer', () => { expect(result.logs.some(line => line.includes('log capture truncated'))).toBe(true) }, 30_000) + it('flushes logs before framing the value so their peaks do not add against RLIMIT_AS', async () => { + // The load gate bounds maxLogBytes and maxValueBytes INDEPENDENTLY against the + // address space, each at the 12x worst case. But the child framed the + // completion value (materializing its escaped form to meter it, then encoding + // the frame) while a newline-free log tail still sat unflushed in _pending. + // Those two peaks added: two budgets each admitted alone could together breach + // RLIMIT_AS, dying as worker-exit instead of settling. The flush now runs + // before the value is framed, so the log pending is freed first. + // + // Config: 32 MiB each against 512 MiB (each 32*12 = 384 MiB < 448 MiB + // budgetable, so both load). The program writes ~33M astral chars with no + // newline (buffered ~132 MB, under the char-count flush trigger) then returns + // ~33M astral chars — a ~132 MB serialized value that is itself OVER the 32 MiB + // maxValueBytes, so the correct outcome is `output-limit`. Pre-fix the + // unflushed 132 MB plus the value's build-and-encode (~396 MB) exceeded 512 MiB + // and OOM'd (reported as exception/worker-exit); flushing first lets the value + // check complete (~460 MB alone) and report output-limit. On Darwin (no + // RLIMIT_AS) the value is over budget too, so output-limit holds either way; + // the OOM the reorder prevents is the Linux-only failure. + const { runtime } = await setup({ + maxLogBytes: 32 * 1024 * 1024, + maxValueBytes: 32 * 1024 * 1024, + addressSpaceMb: 512, + maxWallMs: 20_000, + }) + const result = await runtime.run({ + program: [ + 'import sys', + 'sys.stdout.write("\\U0001F600" * 33_000_000)', + 'return "\\U0001F600" * 33_000_000', + ].join('\n'), + bindings: [], + }) + expect(result.error?.kind).toBe('output-limit') + }, 30_000) + it('bounds a flood of zero-byte log lines through the per-entry separator charge', async () => { // Blank print() lines carry zero content bytes; without the +1 separator // charge they would bypass maxLogBytes entirely and grow the retained