fix(code-runtime-python): pace concurrent binding replies against fd 3

`sendReply` ignored `proto.write`'s `false` return, so a program resolving several large values in one `asyncio.gather` round encoded every reply in the same turn and queued all of them in fd 3's writable buffer. Binding resolution carries no seam-level byte cap to bound that, and the failure kills the host process rather than failing the run: measured on a 64 KiB-highWaterMark pipe, eight 4 MiB replies buffered 32.0 MiB at once against 0.0 MiB once paced. Replies now go through a queue that encodes and writes one frame at a time, awaiting `drain` when the pipe is full. The encode happens inside the loop, so a queued reply the run no longer needs is dropped by the `settled` check without ever being serialized. This was previously deferred on the grounds that serializing would narrow the seam's concurrency contract. That reasoning was wrong: the child matches each reply to its `call` by id from a pump that reads fd 3 continuously, so arrival order was never observable, and the bindings still run concurrently. Only the host's peak memory and the flush timing change. The README entry recording the deferral is removed and the Agent Note records the mechanism instead.
2026-09-11 04:00:38 +00:00 · 2026-08-31 14:33:31 +08:00
parent 2a9a917853
commit 6f58f9c336
11 changed files with 146 additions and 14 deletions
@@ -2,5 +2,5 @@
 # side as of the last confirmed-consistent state. Both languages carry equal authority;
 # after editing either side, bring the other along and re-record with:
 #   pnpm run verify-translation-pairing --write packages/code-runtime/code-runtime-python/README.md
-README.md: e6f78893e32e03760a9d62bae701eb0e93776fd1
-README.zh.md: f4876cf4e13719de4e446bdd28ba58ae041be0a3
+README.md: 3a719c875a39cd2f19b481b8087b20a5b4c884a7
+README.zh.md: 3423763a04c5c18ef02ca52c1cc5aa1bce9ab586
@@ -39,4 +39,3 @@ No direct invalidation; the named consumer owns any request-prefix changes.
 - **A descendant that calls `setsid()` / `start_new_session=True` escapes teardown.** Termination signals the child's process group with `kill(-pid)`; a descendant that moves itself into a fresh session is no longer in that group and no signal reaches it. If it also releases the inherited stdout/stderr/fd-3 pipes, the leader's `close` still settles the run, and after the `closeDeadline` bound the fiber goes quiescent while that orphan keeps running. This is the containment boundary, not a security one — model code has bash-equivalent trust, and a bash tool can `setsid` away just the same. Reaching such an orphan would require tracking every descendant pid (as the bash-local backend's process-inspector does) and is deferred; the process-group teardown reaps everything that stays in the group.
 - **A combined log-and-value peak is not modelled by the load gate.** Each budget is checked against `addressSpaceMb` on its own. A model daemon thread that keeps writing while the completion value is metered and framed can refill the log pending toward `maxLogBytes` during that window, so the two peaks add in a way no gate admits or rejects. A gate over `(maxLogBytes + maxValueBytes)` was considered and deferred: its discriminating case cannot be scheduled deterministically under `RLIMIT_AS`, so the gate would only prove its own arithmetic. When the combined peak is reached the run dies as `worker-exit` -- containment holds and only the failure classification is degraded.
 - **A wide binding REPLY expands host-side state per member.** Resolutions cross through `snapshotJsonValue` in [`@deepseek-ai/dsh-session`](../../core/session/README.md), whose `walkJsonValue` pushes one task frame per member, and binding resolution carries no seam-level byte cap. A legitimate reply of several million elements can therefore exhaust the host heap. The property belongs to that shared walk, not to this backend -- the worker-thread backend consumes the same function -- so the fix belongs in `packages/core/session` where every consumer benefits.
- **Concurrent binding replies are not paced against fd 3.** `proto.write` returns `false` once the pipe's buffer is full and this backend does not wait for `drain`, so several bindings resolving large values in one `asyncio.gather` round encode and queue together in host memory. Serializing the replies would bound it, at the cost of changing the concurrency the seam currently allows; the sibling worker-thread backend has no equivalent (it posts structured clones, which carry no stream backpressure), so there is no in-repo precedent to copy.
@@ -39,4 +39,3 @@ host 与 CPython 子进程在子进程的 fd 3 上交换一个无版本号的 JS
 - **调用 `setsid()` ／ `start_new_session=True` 的后代会逃出 teardown。** 终止是用 `kill(-pid)` 向子进程的进程组发信号；一个把自己移入新会话的后代已不在该进程组内，任何信号都到不了它。若它同时释放了继承而来的 stdout／stderr／fd-3 管道，leader 的 `close` 仍会结算该次运行，在 `closeDeadline` 到界之后 fiber 变为完全停稳，而那个孤儿仍在运行。这是 containment 边界，而非安全边界——模型代码具有等同 bash 的信任级别，一个 bash 工具同样能 `setsid` 逃逸。要够到这样的孤儿需要追踪每一个后代 pid（如 bash-local 后端的 process-inspector 所做），此项已推迟；进程组 teardown 会回收所有留在组内的进程。
 - **日志与完成值的叠加峰值未被加载门建模。** 每项预算都是各自对照 `addressSpaceMb` 检查的。模型的 daemon 线程可以在完成值被计量并分帧的窗口内持续写入、把日志 pending 重填到接近 `maxLogBytes`，于是两个峰值以任何门都不曾放行也不曾拒绝的方式相加。对 `(maxLogBytes + maxValueBytes)` 设门的方案经评估后推迟：它的判别用例无法在 `RLIMIT_AS` 之下确定性地构造出来，因此该门只能证明自己的算术。叠加峰值被触及时该次运行死为 `worker-exit`——containment 仍然成立，只是失败分类失真。
 - **宽 binding 回复会按成员展开宿主侧状态。** 回复经由 [`@deepseek-ai/dsh-session`](../../core/session/README.md) 的 `snapshotJsonValue` 穿越，其 `walkJsonValue` 为每个成员压入一个任务帧，而 binding 回复在 seam 层没有字节上限。因此一个数百万元素的合法回复可以耗尽宿主堆。该性质属于那个共享遍历，而不属于本后端——worker-thread 后端消费同一个函数——所以修复应落在 `packages/core/session`，让所有消费方一并受益。
- **并发 binding 回复没有对 fd 3 做节流。** 管道缓冲写满后 `proto.write` 返回 `false`，而本后端不等待 `drain`，因此在一轮 `asyncio.gather` 中多个 binding 同时返回大值时，它们会一起编码并排入宿主内存。把回复串行化可以给它设界，代价是改变 seam 当前允许的并发度；同类的 worker-thread 后端没有等价物（它投递结构化克隆，不存在流背压），因此仓库内没有可照抄的先例。
@@ -13,6 +13,7 @@
 */

 import { spawn, type ChildProcessWithoutNullStreams } from 'node:child_process'
+import { once } from 'node:events'
 import { accessSync, copyFileSync, constants as fsConstants, mkdtempSync, readFileSync, rmSync } from 'node:fs'
 import { tmpdir } from 'node:os'
 import { delimiter, dirname, isAbsolute, join } from 'node:path'
@@ -1382,14 +1383,47 @@ export class PythonCodeRuntime extends CodeRuntime {
      // strings/numbers), which is encodeJsonPlain's precondition. A closed
      // pipe (child already gone) is swallowed since the close path settles
      // the run.
+      //
+      // Replies are encoded and written ONE AT A TIME, waiting for `drain`
+      // whenever fd 3's buffer is full. Binding resolution carries no
+      // seam-level byte cap, so a program that resolves several large values in
+      // one `asyncio.gather` round would otherwise encode them all in the same
+      // turn and queue every frame in the writable stream's buffer -- measured
+      // to exhaust a 256 MiB Node heap, which kills the whole host process
+      // rather than failing this one run. Pacing changes no model-visible
+      // behavior: the child matches each reply to its `call` by id from a pump
+      // that reads fd 3 continuously, so arrival order was never observable,
+      // and the bindings themselves still run concurrently. Only the host's peak
+      // memory and the flush timing change.
+      const replyQueue: ReplyMessage[] = []
+      let draining = false
+      const drainReplies = async (): Promise<void> => {
+        if (draining) return
+        draining = true
+        try {
+          while (replyQueue.length > 0) {
+            if (settled) break
+            const payload = replyQueue.shift() as ReplyMessage
+            // Encode inside the loop, not up front: a queued reply the run no
+            // longer needs is dropped by the `settled` check above without ever
+            // being serialized.
+            if (!proto.write(`${encodeJsonPlain(payload)}\n`)) {
+              await once(proto, 'drain')
+            }
+          }
+        } catch {
+          // Pipe closed under us (child exited), or `drain` never arrives because
+          // the child died. The close path settles the run either way.
+        } finally {
+          draining = false
+          replyQueue.length = 0
+        }
+      }
      const sendReply = (payload: ReplyMessage): void => {
        /* v8 ignore next -- `settled` covers a race where the child exits between decision and write. */
        if (settled) return
-        try {
-          proto.write(`${encodeJsonPlain(payload)}\n`)
-        } catch {
-          // Pipe closed under us (child exited). The close path finishes the run.
-        }
+        replyQueue.push(payload)
+        void drainReplies()
      }

      // Escalate SIGTERM → grace → SIGKILL on the entire process group. Idempotent
@@ -3863,6 +3863,35 @@ describe('PythonCodeRuntime — hostile peer', () => {
    expect(resolvedLate).toBe(true)
  }, 90_000)

+  it('paces concurrent binding replies instead of queueing every frame at once', async () => {
+    // Binding resolution carries no seam-level byte cap. Before pacing, a program
+    // resolving several large values in one `asyncio.gather` round encoded them
+    // all in the same turn and queued every frame in fd 3's writable buffer,
+    // which exhausted the host heap and killed the whole process rather than
+    // failing the run. Replies are now encoded one at a time, waiting for
+    // `drain` when the pipe is full.
+    //
+    // Eight concurrent 4 MiB replies (32 MiB of frames) must all round-trip. The
+    // program sums the lengths, so the assertion proves every reply arrived and
+    // was matched to its own call -- pacing must not drop or misroute any. What
+    // this case cannot show is the peak itself, which lives in the stream's
+    // buffer: measured directly on a 64 KiB-highWaterMark pipe with this same
+    // 8x4 MiB shape, the unpaced writes buffered 32.0 MiB while the paced ones
+    // peaked at 0.0 MiB.
+    const chunk = 'A'.repeat(4 * 1024 * 1024)
+    const { runtime } = await setup({ maxWallMs: 60_000 })
+    const result = await runtime.run({
+      program: [
+        'import asyncio',
+        'parts = await asyncio.gather(*[tools.chunk({}) for _ in range(8)])',
+        'return sum(len(p) for p in parts)',
+      ].join('\n'),
+      bindings: [{ global: 'tools', functions: { chunk: async () => chunk } }],
+    })
+    expect(result.error).toBeUndefined()
+    expect(result.value).toBe(8 * chunk.length)
+  }, 90_000)
+
  it('bounds a flood of zero-byte log lines through the per-entry separator charge', async () => {
    // Blank print() lines carry zero content bytes; without the +1 separator
    // charge they would bypass maxLogBytes entirely and grow the retained