fix(code-runtime-python): drop a late binding resolution before snapshotting it

`sendReply` already refuses to write after the run settled, but only after `snapshotJsonValue` walked and copied the resolution. Binding resolution carries no seam-level byte cap, so a binding resolving a wide value after `maxWallMs`, an abort, or dispose settled the run spent host heap building a frame that was then discarded. The check moves ahead of the snapshot. Also in this change: - `readProcessStart` moved after `messageOf`. Inserting it between `messageOf`'s JSDoc and its body left that function undocumented and the orphaned block reading as a second doc for the reader; `verify-export-jsdoc` does not catch it because `messageOf` is not exported. - The README pair adds the disposed-runtime rejection to `run()`'s public contract, which `src/index.ts` has enforced all along. - Known Limitations records three deferred constraints that until now existed only in review discussion: the combined log-and-value peak the load gate does not model, the host-side per-member expansion of a wide binding reply (owned by `packages/core/session`, and shared with the worker-thread backend), and the absence of fd-3 backpressure for concurrent replies. - The Agent Note's same-group section records the teardown identity guard and its two rulings, including why an ABSENT start-time reading proceeds rather than withholding the signal, and that reading it as a mismatch is what turned the three same-group heartbeat cases red on Linux.
2026-09-11 04:00:38 +00:00 · 2026-08-31 14:31:48 +08:00
parent 0a46bb3414
commit 2a9a917853
8 changed files with 66 additions and 17 deletions
@@ -2,5 +2,5 @@
 # side as of the last confirmed-consistent state. Both languages carry equal authority;
 # after editing either side, bring the other along and re-record with:
 #   pnpm run verify-translation-pairing --write packages/code-runtime/code-runtime-python/README.md
-README.md: b643330ecc427ae04439d658b7b3f7b084010d4c
-README.zh.md: 4b656d9735a52b5473658b00fe6210325d819eff
+README.md: e6f78893e32e03760a9d62bae701eb0e93776fd1
+README.zh.md: f4876cf4e13719de4e446bdd28ba58ae041be0a3
@@ -9,7 +9,7 @@ English | [中文](README.zh.md)

 CPython-subprocess implementation of the [`@deepseek-ai/dsh-code-runtime`](../code-runtime/README.md) seam. Companion to [`@deepseek-ai/dsh-code-runtime-worker-thread`](../code-runtime-worker-thread/README.md); trades the Node worker thread for a fresh `python3` subprocess so model code is Python instead of TypeScript.

-The package owns the wire protocol for that seam: the host-side frame codec and the Python-side mirror of the same message vocabulary. On top of that protocol it ships `PythonCodeRuntime` (the plugin's default export), which registers as `codeRuntime` with `language: 'python'` and `isolation: 'process'`. Each `run()` spawns a fresh `python3 -I` process, sends a boot frame and the program over fd 3, and resolves a `CodeRunResult` for every program outcome — `run()` rejects only for seam misuse, such as a malformed binding namespace. Configuration is rejected earlier, when the plugin loads: a non-Unix platform, a non-positive or non-integer budget, a timer value `setTimeout` would clamp, a budget larger than one fd-3 frame can carry, and an `addressSpaceMb`/output-budget pair whose worst-case peak would breach `RLIMIT_AS` all throw from the constructor, so a misconfiguration fails at assembly rather than on a later run. The child runs the program as the body of an async function, so top-level `await` and `return` both work; binding calls travel back over fd 3 as JSON-lines. Containment (not a security boundary — model code has bash-equivalent trust) comes from an empty environment, `RLIMIT_CPU`/`RLIMIT_AS`, a wall-clock ceiling, and a `SIGTERM`→grace→`SIGKILL` teardown on the child's process group.
+The package owns the wire protocol for that seam: the host-side frame codec and the Python-side mirror of the same message vocabulary. On top of that protocol it ships `PythonCodeRuntime` (the plugin's default export), which registers as `codeRuntime` with `language: 'python'` and `isolation: 'process'`. Each `run()` spawns a fresh `python3 -I` process, sends a boot frame and the program over fd 3, and resolves a `CodeRunResult` for every program outcome — `run()` rejects only for seam misuse, such as a malformed binding namespace or a call on a runtime whose fiber was already disposed. Configuration is rejected earlier, when the plugin loads: a non-Unix platform, a non-positive or non-integer budget, a timer value `setTimeout` would clamp, a budget larger than one fd-3 frame can carry, and an `addressSpaceMb`/output-budget pair whose worst-case peak would breach `RLIMIT_AS` all throw from the constructor, so a misconfiguration fails at assembly rather than on a later run. The child runs the program as the body of an async function, so top-level `await` and `return` both work; binding calls travel back over fd 3 as JSON-lines. Containment (not a security boundary — model code has bash-equivalent trust) comes from an empty environment, `RLIMIT_CPU`/`RLIMIT_AS`, a wall-clock ceiling, and a `SIGTERM`→grace→`SIGKILL` teardown on the child's process group.

 ## Wire protocol

@@ -37,3 +37,6 @@ No direct invalidation; the named consumer owns any request-prefix changes.
 - **The cross-language guard covers executed values and frame field sets, not field types** — `tests/protocol-mirror.e2e.ts` compares `PROTOCOL_FD`, the log truncation marker, and each `TypedDict`'s required and optional fields against a real `python3`. Comparing field types across TypeScript and Python has no mechanical equivalent here, so review plus the backend's real-subprocess suite owns type-level drift.
 - **`RLIMIT_AS` is not enforced on macOS** — the dyld shared cache mapped into every process at exec exceeds any practical address-space cap, and the kernel rejects the `setrlimit` call, so `addressSpaceMb` is skipped there. `cpuSeconds` and `maxWallMs` still bound every run.
 - **A descendant that calls `setsid()` / `start_new_session=True` escapes teardown.** Termination signals the child's process group with `kill(-pid)`; a descendant that moves itself into a fresh session is no longer in that group and no signal reaches it. If it also releases the inherited stdout/stderr/fd-3 pipes, the leader's `close` still settles the run, and after the `closeDeadline` bound the fiber goes quiescent while that orphan keeps running. This is the containment boundary, not a security one — model code has bash-equivalent trust, and a bash tool can `setsid` away just the same. Reaching such an orphan would require tracking every descendant pid (as the bash-local backend's process-inspector does) and is deferred; the process-group teardown reaps everything that stays in the group.
+- **A combined log-and-value peak is not modelled by the load gate.** Each budget is checked against `addressSpaceMb` on its own. A model daemon thread that keeps writing while the completion value is metered and framed can refill the log pending toward `maxLogBytes` during that window, so the two peaks add in a way no gate admits or rejects. A gate over `(maxLogBytes + maxValueBytes)` was considered and deferred: its discriminating case cannot be scheduled deterministically under `RLIMIT_AS`, so the gate would only prove its own arithmetic. When the combined peak is reached the run dies as `worker-exit` -- containment holds and only the failure classification is degraded.
+- **A wide binding REPLY expands host-side state per member.** Resolutions cross through `snapshotJsonValue` in [`@deepseek-ai/dsh-session`](../../core/session/README.md), whose `walkJsonValue` pushes one task frame per member, and binding resolution carries no seam-level byte cap. A legitimate reply of several million elements can therefore exhaust the host heap. The property belongs to that shared walk, not to this backend -- the worker-thread backend consumes the same function -- so the fix belongs in `packages/core/session` where every consumer benefits.
+- **Concurrent binding replies are not paced against fd 3.** `proto.write` returns `false` once the pipe's buffer is full and this backend does not wait for `drain`, so several bindings resolving large values in one `asyncio.gather` round encode and queue together in host memory. Serializing the replies would bound it, at the cost of changing the concurrency the seam currently allows; the sibling worker-thread backend has no equivalent (it posts structured clones, which carry no stream backpressure), so there is no in-repo precedent to copy.
@@ -9,7 +9,7 @@ kind: "package-reference"

 [`@deepseek-ai/dsh-code-runtime`](../code-runtime/README.zh.md) seam 的 CPython 子进程实现。与 [`@deepseek-ai/dsh-code-runtime-worker-thread`](../code-runtime-worker-thread/README.zh.md) 配套；以全新的 `python3` 子进程取代 Node worker 线程，让模型代码从 TypeScript 换成 Python。

-本包持有该 seam 的 wire protocol：host 侧的帧编解码，以及 Python 侧对同一套消息词汇的镜像。在该协议之上，本包交付 `PythonCodeRuntime`（插件的默认导出），它以 `language: 'python'`、`isolation: 'process'` 注册为 `codeRuntime`。每次 `run()` 启动一个全新的 `python3 -I` 进程，通过 fd 3 发送 boot 帧和程序，并为每个程序结果 resolve 一个 `CodeRunResult`——`run()` 仅在 seam 被误用时才 reject，例如 binding 命名空间不合法。配置错误在更早的插件加载期被拒绝：非 Unix 平台、非正或非整数的预算、会被 `setTimeout` 截断的定时器值、超过单个 fd-3 帧承载能力的预算，以及最坏峰值会突破 `RLIMIT_AS` 的 `addressSpaceMb`／输出预算组合，都从构造器抛出，因此配置错误在装配时就失败，而不是等到之后某次运行。子进程把程序作为 async 函数体运行，因此顶层 `await` 与 `return` 都可用；binding 调用经 fd 3 以 JSON-lines 回传。containment 不是安全边界——模型代码具有等同 bash 的信任级别；空环境、`RLIMIT_CPU`／`RLIMIT_AS`、墙钟上限与对子进程进程组的 `SIGTERM`→grace→`SIGKILL` 拆卸共同提供 containment。
+本包持有该 seam 的 wire protocol：host 侧的帧编解码，以及 Python 侧对同一套消息词汇的镜像。在该协议之上，本包交付 `PythonCodeRuntime`（插件的默认导出），它以 `language: 'python'`、`isolation: 'process'` 注册为 `codeRuntime`。每次 `run()` 启动一个全新的 `python3 -I` 进程，通过 fd 3 发送 boot 帧和程序，并为每个程序结果 resolve 一个 `CodeRunResult`——`run()` 仅在 seam 被误用时才 reject，例如 binding 命名空间不合法，或对 fiber 已被 dispose 的 runtime 发起调用。配置错误在更早的插件加载期被拒绝：非 Unix 平台、非正或非整数的预算、会被 `setTimeout` 截断的定时器值、超过单个 fd-3 帧承载能力的预算，以及最坏峰值会突破 `RLIMIT_AS` 的 `addressSpaceMb`／输出预算组合，都从构造器抛出，因此配置错误在装配时就失败，而不是等到之后某次运行。子进程把程序作为 async 函数体运行，因此顶层 `await` 与 `return` 都可用；binding 调用经 fd 3 以 JSON-lines 回传。containment 不是安全边界——模型代码具有等同 bash 的信任级别；空环境、`RLIMIT_CPU`／`RLIMIT_AS`、墙钟上限与对子进程进程组的 `SIGTERM`→grace→`SIGKILL` 拆卸共同提供 containment。

 ## Wire protocol

@@ -37,3 +37,6 @@ host 与 CPython 子进程在子进程的 fd 3 上交换一个无版本号的 JS
 - **跨语言 guard 覆盖执行值与帧字段集，但不覆盖字段类型** —— `tests/protocol-mirror.e2e.ts` 使用真实 `python3` 比较 `PROTOCOL_FD`、日志截断标记，以及每个 `TypedDict` 的必填和可选字段。跨 TypeScript 与 Python 比较字段类型在此没有机械等价物，因此类型级漂移由 review 加后端真子进程套件负责。
 - **`RLIMIT_AS` 在 macOS 上不施加** —— 在 exec 时映射进每个进程的 dyld 共享缓存超过任何实际的地址空间上限，内核会拒绝该 `setrlimit` 调用，故 `addressSpaceMb` 在那里被跳过。`cpuSeconds` 与 `maxWallMs` 仍约束每一次运行。
 - **调用 `setsid()` ／ `start_new_session=True` 的后代会逃出 teardown。** 终止是用 `kill(-pid)` 向子进程的进程组发信号；一个把自己移入新会话的后代已不在该进程组内，任何信号都到不了它。若它同时释放了继承而来的 stdout／stderr／fd-3 管道，leader 的 `close` 仍会结算该次运行，在 `closeDeadline` 到界之后 fiber 变为完全停稳，而那个孤儿仍在运行。这是 containment 边界，而非安全边界——模型代码具有等同 bash 的信任级别，一个 bash 工具同样能 `setsid` 逃逸。要够到这样的孤儿需要追踪每一个后代 pid（如 bash-local 后端的 process-inspector 所做），此项已推迟；进程组 teardown 会回收所有留在组内的进程。
+- **日志与完成值的叠加峰值未被加载门建模。** 每项预算都是各自对照 `addressSpaceMb` 检查的。模型的 daemon 线程可以在完成值被计量并分帧的窗口内持续写入、把日志 pending 重填到接近 `maxLogBytes`，于是两个峰值以任何门都不曾放行也不曾拒绝的方式相加。对 `(maxLogBytes + maxValueBytes)` 设门的方案经评估后推迟：它的判别用例无法在 `RLIMIT_AS` 之下确定性地构造出来，因此该门只能证明自己的算术。叠加峰值被触及时该次运行死为 `worker-exit`——containment 仍然成立，只是失败分类失真。
+- **宽 binding 回复会按成员展开宿主侧状态。** 回复经由 [`@deepseek-ai/dsh-session`](../../core/session/README.md) 的 `snapshotJsonValue` 穿越，其 `walkJsonValue` 为每个成员压入一个任务帧，而 binding 回复在 seam 层没有字节上限。因此一个数百万元素的合法回复可以耗尽宿主堆。该性质属于那个共享遍历，而不属于本后端——worker-thread 后端消费同一个函数——所以修复应落在 `packages/core/session`，让所有消费方一并受益。
+- **并发 binding 回复没有对 fd 3 做节流。** 管道缓冲写满后 `proto.write` 返回 `false`，而本后端不等待 `drain`，因此在一轮 `asyncio.gather` 中多个 binding 同时返回大值时，它们会一起编码并排入宿主内存。把回复串行化可以给它设界，代价是改变 seam 当前允许的并发度；同类的 worker-thread 后端没有等价物（它投递结构化克隆，不存在流背压），因此仓库内没有可照抄的先例。
@@ -316,6 +316,17 @@ const GROUP_REAP_POLL_MS = 50
 * @returns The value's message or string form; a fixed placeholder when its own
 *   conversion throws.
 */
+function messageOf(error: unknown): string {
+  try {
+    return String(error instanceof Error ? error.message : error)
+  } catch {
+    // Swallows only a throw from the value's own `message` getter or string
+    // conversion. Nothing else runs inside the try, and the placeholder is a
+    // literal, so this cannot throw again.
+    return '<unrenderable rejection value>'
+  }
+}
+
 /**
 * A process's start time, as the identity half of (pid, started).
 *
@@ -347,17 +358,6 @@ export function readProcessStart(pid: number): string | undefined {
  }
 }

-function messageOf(error: unknown): string {
-  try {
-    return String(error instanceof Error ? error.message : error)
-  } catch {
-    // Swallows only a throw from the value's own `message` getter or string
-    // conversion. Nothing else runs inside the try, and the placeholder is a
-    // literal, so this cannot throw again.
-    return '<unrenderable rejection value>'
-  }
-}
-
 /**
 * Resolve `pythonBin` to an absolute path against the CURRENT process `PATH`,
 * BEFORE the child spawns with an empty environment. A basename (the default
@@ -1347,6 +1347,14 @@ export class PythonCodeRuntime extends CodeRuntime {
            void (async () => {
              try {
                const resolved = await fn(message.args)
+                // Drop a reply the run no longer needs BEFORE snapshotting it.
+                // `sendReply` also checks `settled`, but only after this value has
+                // been walked and copied: a binding that resolves a wide value
+                // after `maxWallMs`, an abort, or dispose already settled the run
+                // would spend host heap on a frame that is then discarded, and
+                // binding resolution carries no seam-level byte cap to bound it.
+                // oxlint-disable-next-line typescript/no-unnecessary-condition -- the run can settle while this binding is awaited.
+                if (settled) return
                // The seam requires a lossy resolution to REJECT descriptively,
                // not silently coerce: a raw JSON.stringify would turn NaN/
                // Infinity into null and drop undefined fields. Snapshot through
@@ -3832,6 +3832,37 @@ describe('PythonCodeRuntime — hostile peer', () => {
    expect(result.value).toBe(reply.length)
  }, 90_000)

+  it('drops a late binding resolution before snapshotting it', async () => {
+    // `sendReply` checks `settled`, but only after the resolution has been walked
+    // and copied by `snapshotJsonValue`. Binding resolution carries no seam-level
+    // byte cap, so a binding that resolves a wide value AFTER the run already
+    // settled (here on `maxWallMs`) spent host heap building a frame that is then
+    // discarded. The check now runs before the snapshot.
+    //
+    // The binding resolves well after the 1s wall clock with a 2M-element array;
+    // the run must still report `timeout`, and the late value must not appear.
+    let resolvedLate = false
+    const { runtime } = await setup({ maxWallMs: 1_000 })
+    const result = await runtime.run({
+      program: 'return await tools.slow({})',
+      bindings: [{
+        global: 'tools',
+        functions: {
+          slow: async () => {
+            await new Promise(resolve => setTimeout(resolve, 2_500))
+            resolvedLate = true
+            return Array.from({ length: 2_000_000 }, () => 0)
+          },
+        },
+      }],
+    })
+    expect(result.error?.kind).toBe('timeout')
+    expect(result.value).toBeUndefined()
+    // Pin that the late path actually ran, so the assertion above is not vacuous.
+    await new Promise(resolve => setTimeout(resolve, 2_000))
+    expect(resolvedLate).toBe(true)
+  }, 90_000)
+
  it('bounds a flood of zero-byte log lines through the per-entry separator charge', async () => {
    // Blank print() lines carry zero content bytes; without the +1 separator
    // charge they would bypass maxLogBytes entirely and grow the retained