diff --git a/packages/code-runtime/code-runtime-python/README.i18n.yaml b/packages/code-runtime/code-runtime-python/README.i18n.yaml index 3dfef67e47..901ddd90d5 100644 --- a/packages/code-runtime/code-runtime-python/README.i18n.yaml +++ b/packages/code-runtime/code-runtime-python/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/code-runtime/code-runtime-python/README.md -README.md: 507c1d78a1e25636792a077f657326f4e884ac46 -README.zh.md: e4284e852be26b0d19dc4b8bd484b1d72bac9129 +README.md: 26c4db82f0caf8a1d4333480426621e75e7610b9 +README.zh.md: ee754d03e747b01fa11443d22e156d1b5ebde9fd diff --git a/packages/code-runtime/code-runtime-python/README.md b/packages/code-runtime/code-runtime-python/README.md index 507c1d78a1..26c4db82f0 100644 --- a/packages/code-runtime/code-runtime-python/README.md +++ b/packages/code-runtime/code-runtime-python/README.md @@ -1,32 +1,103 @@ --- -description: "CPython subprocess implementation of the DeepSeek Harness code-execution seam, with fd-3 bindings, resource limits, log capture, and process-group teardown." -kind: "package-reference" +description: "CPython-subprocess code runtime: the dsh-code-runtime seam implementation for Python model code, with the fd-3 wire protocol it speaks." +kind: "package-library" --- # @deepseek-ai/dsh-code-runtime-python English | [中文](README.zh.md) -CPython-subprocess implementation of the [`@deepseek-ai/dsh-code-runtime`](../code-runtime/README.md) seam. Companion to [`@deepseek-ai/dsh-code-runtime-worker-thread`](../code-runtime-worker-thread/README.md); trades the Node worker thread for a fresh `python3` subprocess so model code is Python instead of TypeScript. +## Summary -The package owns the wire protocol for that seam: the host-side frame codec and the Python-side mirror of the same message vocabulary. On top of that protocol it ships `PythonCodeRuntime` (the plugin's default export), which registers as `codeRuntime` with `language: 'python'` and `isolation: 'process'`. Each `run()` spawns a fresh `python3 -I` process, sends a boot frame and the program over fd 3, and resolves a `CodeRunResult` for every program outcome — `run()` rejects only for seam misuse, such as a malformed binding namespace or a call on a runtime whose fiber was already disposed. Configuration is rejected earlier, when the plugin loads: a non-Unix platform, a non-positive or non-integer budget, a `maxLogBytes` below the truncation-marker floor (64), a timer value `setTimeout` would clamp, a budget larger than one fd-3 frame can carry, and an `addressSpaceMb`/output-budget pair whose worst-case peak would breach `RLIMIT_AS` all throw from the constructor, so a misconfiguration fails at assembly rather than on a later run. The child runs the program as the body of an async function, so top-level `await` and `return` both work; binding calls travel back over fd 3 as JSON-lines. Containment (not a security boundary — model code has bash-equivalent trust) comes from an empty environment, `RLIMIT_CPU`/`RLIMIT_AS`, a wall-clock ceiling, and a `SIGTERM`→grace→`SIGKILL` teardown on the child's process group. +`dsh-code-runtime-python` ships `PythonCodeRuntime`, the CPython-subprocess implementation of the [`dsh-code-runtime`](../code-runtime/README.md) seam: it registers as `codeRuntime` with `language: 'python'` and `isolation: 'process'`, spawning a fresh `python3 -I` child per `run()` and executing the program as an async function body over a versionless JSON-lines protocol on the child's fd 3 (stdout/stderr stay free for the program's own output). The host side (`src/protocol.ts`) treats every inbound frame as hostile and rebuilds it before reading; the Python side (`py/protocol.py`) mirrors the message vocabulary. Containment — not a security boundary, model code has bash-equivalent trust — comes from an empty environment, `RLIMIT_CPU`/`RLIMIT_AS`, a wall-clock ceiling, and `SIGTERM`→grace→`SIGKILL` process-group teardown, with all caps validated at plugin load. -## Wire protocol +## Table of Contents -The host and the CPython subprocess exchange a versionless, JSON-lines protocol on the child's fd 3 — one JSON object per line, leaving stdout/stderr free for the program's own output. `src/protocol.ts` is the host side; `py/protocol.py` mirrors its message shapes and the shared truncation-marker text on the Python side. +- [Use this package](#use-this-package) +- [Understand the implementation](#understand-the-implementation) +- [Further Exploration](#further-exploration) +- [Model Experience](#model-experience) +- [Known Limitations and Deferred Work](#known-limitations-and-deferred-work) +- [Dev Note](#dev-note) -- **fd 3, not stdout** — Node pins the channel positionally with `stdio: ['pipe','pipe','pipe','pipe']`; the Python bootstrap reads the same `PROTOCOL_FD` constant. JSON-lines framing. -- **Host treats every inbound frame as hostile** — model code has full access to fd 3 and can post anything through it, so `validateChildFrame` shape-validates and REBUILDS each frame before the host reads it: forged extra fields never ride along, a non-number call id can never be echoed into a reply, and junk drops to `undefined` rather than throwing in the host's message handler. The Python side trusts host replies (the host is not model-controlled). -- **Lossless-JSON crossing** — completion values and binding arguments cross as exact JSON. `encodeJsonPlain` serializes a `JSON.parse`-produced value without recursion, so a deep value below the byte budget crosses intact instead of dying on `JSON.stringify`'s stack limit; `checkDoneValue` meters a forged completion value's byte length AND number losslessness in one bounded traversal that rejects an over-budget payload before enqueuing its children; `hasUnsafeIntegerToken` reads the raw frame text to catch an integer token that `JSON.parse` would silently round; `hasNonLosslessNumber` rejects a non-finite or negative-zero number in unbounded `call.args`. Beyond-safe-range integral doubles serialize through `BigInt` digits so the exact integer crosses, not the rounded `String()` form. -- **Shared truncation marker** — `logTruncationMarker(maxBytes)` produces byte-identical text on both sides, so a truncated log run reads the same however the cap was hit. The `log` frame's `truncated` flag distinguishes the child ledger's own marker from program output. +----- -## Configuration + +## Use this package -Every cap is a validated `Config` field with a default, changeable from `cordis.yml` (no hardcoded tunables). `cpuSeconds` (default 60) is the `RLIMIT_CPU` whole-second budget; the child sets the soft limit to `cpuSeconds` and the hard limit to `cpuSeconds + 1`, so the kernel's `SIGXCPU` at the soft limit classifies as a `timeout` while the +1s hard limit is a `SIGKILL` backstop. `maxWallMs` (default 600000) is the wall-clock ceiling that backstops CPU time for a program awaiting a promise nobody resolves. `addressSpaceMb` (default 512) is the `RLIMIT_AS` cap, not applied on Darwin (the dyld shared cache mapped into every process exceeds any practical cap there; `cpuSeconds` and `maxWallMs` still bound the run). `maxLogBytes` (default 65536) is the shared captured-log byte budget; `maxValueBytes` (default 32768) caps the completion value; `graceMs` (default 3000) is the `SIGTERM`→`SIGKILL` grace window; `pythonBin` (default `python3`) is the interpreter, resolved against `PATH` before the child spawns with an empty environment. +Choose this package to run Python model code through the code-runtime seam: register `PythonCodeRuntime` with `dsh-tools` and `run()` executes each program in a fresh `python3 -I` subprocess, resolves with an `error` FIELD for every program outcome (the orthogonal `CodeRunFailure.kind` taxonomy classifies parse failures, thrown exceptions, invalid completions, output overflows, budget expiry, aborts, and substrate death), and rejects only for seam misuse — a malformed binding namespace, or a call after disposal. Configuration is rejected at load: a non-Unix platform, a non-positive or non-integer budget, a `maxLogBytes` below the truncation-marker floor (64), a timer value `setTimeout` would clamp, a budget larger than one fd-3 frame can carry, and an `addressSpaceMb`/output-budget pair whose worst-case peak would breach `RLIMIT_AS`. +### What you get + +The package's default export is the `PythonCodeRuntime` plugin. Its public surface also re-exports the host-side protocol vocabulary: `validateChildFrame` (rebuilds every inbound frame), the lossless-JSON codec and meters (`encodeJsonPlain`, `checkDoneValue`, `hasUnsafeIntegerToken`, `hasNonLosslessNumber`), and `logTruncationMarker` (the shared truncation-marker text). Every cap is a validated `Config` field with a default: `cpuSeconds` (60), `maxWallMs` (600000), `addressSpaceMb` (512, not applied on Darwin), `maxLogBytes` (65536), `maxValueBytes` (32768), `graceMs` (3000), and `pythonBin` (`python3`, resolved against `PATH` before the child spawns with an empty environment). + +### The wire + +Frames travel on the child's fd 3 as JSON-lines — one object per line — so stdout/stderr stay clear for the program's own output. Child → host: `boot-ack`, `call`, `log`, `done`. Host → child: `boot` (first frame, carrying every cap and the namespace declarations), `run` (after `boot-ack`, carrying only the program body), and one `reply` per `call`. A forged frame can carry both `value` and `error` on `done`, so a consumer must check `error` first and ignore `value` when it is set. + +### What can go wrong + +Host-side validation drops junk without throwing, so a malformed or forged frame never crashes the host process: `validateChildFrame` returns `undefined` for anything that does not rebuild cleanly, a non-number call id can never be echoed into a reply, and forged extra fields never ride along. A completion value that is not lossless JSON, or that exceeds the configured byte budget, is rejected explicitly (`non-lossless` / `over-budget`) rather than silently rounded or truncated. An fd-3 frame whose raw length exceeds 64 MiB settles the run as a `worker-exit` (the receive path caps raw frames before `toString`/`JSON.parse` so a compact wide frame cannot decode to far more host memory than its wire bytes admitted). + +----- + + +## Understand the implementation + +
+Implementation internals — click to expand + +This section explains the design behind the backend; observable behavior is fully covered in [Use this package](#use-this-package). + +### Design concept + +One direction of trust: the host treats every inbound frame as hostile (model code can forge anything on fd 3) and REBUILDS it field by field before reading; the Python side trusts host replies. The bootstrap (`py/bootstrap.py`) runs the program as the body of an async function, so top-level `await` and `return` work; binding calls travel over fd 3 as JSON-lines and replies are paced across the pump so a flood of large replies cannot pin the host's fd-3 write buffer. + +### Wire contract + +The frames are `boot` / `run` (host → child) and `boot-ack` / `call` / `log` / `done` plus one `reply` per call (child → host). The `log` frame's `truncated` flag marks the frame that IS the child ledger's truncation marker, so the host stops capturing at the same point the child did instead of inferring it from its own budget. `done.error.kind` is one of `exception`, `invalid-output`, `output-limit`; wall/CPU budgets, aborts, and substrate death are observed host-side, not carried as frames. + +### Lossless JSON crossing + +Completion values and binding arguments cross as exact JSON: values serialize without recursion, so a deep payload below the byte budget survives instead of dying on `JSON.stringify`'s stack limit, and integral doubles beyond the safe range cross as exact digits rather than silently rounded tokens; the meters in `src/protocol.ts` enforce byte budgets and number losslessness before anything else reads the payload. + +### Mirror alignment + +`tests/protocol-mirror.e2e.ts` spawns a real `python3` and asserts, against `src/protocol.ts`, both `PROTOCOL_FD` / the truncation-marker text and each `TypedDict`'s required/optional wire field set in `py/protocol.py`, so a renamed or dropped field — or one side making a field optional the other requires — fails the test. Field *types* are not compared across the language boundary; that residue stays with review plus the backend's real-subprocess suite (`tests/runtime.spec.ts`). + +### Source map + +| File | Role | +|---|---| +| [`src/index.ts`](src/index.ts) | Plugin entry: `PythonCodeRuntime` — spawn, frame pump, budgets, containment, teardown; re-exports the protocol vocabulary | +| [`src/protocol.ts`](src/protocol.ts) | Host side: frame codec, hostile-frame validators, lossless-JSON meters, shared marker text | +| [`py/bootstrap.py`](py/bootstrap.py) | Child side: fd-3 channel, program execution, binding dispatch, ledger and settlement | +| [`py/protocol.py`](py/protocol.py) | Python side: `PROTOCOL_FD`, `TypedDict` frame mirrors, `log_truncation_marker` | +| [`tests/runtime.spec.ts`](tests/runtime.spec.ts) | Real-subprocess suite: budgets, containment, hostile frames, name rebinding | +| [`tests/protocol-mirror.e2e.ts`](tests/protocol-mirror.e2e.ts) | Cross-language mirror test against a real `python3` | +| [`src/invariant.ts`](src/invariant.ts) | Invariant companion (no runtime invariant; the package registers no mutable data relation) | + +
+ +----- + + +## Further Exploration + +Read these when the runtime contract is not enough. They move from the seam definition to the design record and the companion backend. + +- [Code runtime seam](../code-runtime/README.md) — the abstract contract this backend implements. +- [fd-3 protocol Agent Note](../../../.agents/notes/implemented/architecture/2026-07-31-code-runtime-python-fd3-protocol.md) — design rationale and wire contract. +- [Settlement-fixes Agent Note](../../../.agents/notes/implemented/bug-fix/2026-07-31-code-runtime-python-settlement-fixes.md) — settlement, metering, and containment fixes and their regression cases. +- [Worker-thread backend](../code-runtime-worker-thread/README.md) — the shipped TypeScript sibling. +- [Code runtime subsystem reference](../../../docs/subsystems/code-runtime.md) — request/result vocabulary, bindings, and failure taxonomy. + +----- + + ## Model Experience -Indirectly, through Code Mode in [`dsh-tools`](../../core/tools/README.md), which renders this backend's exact completion value when it fits (or an explicit `invalid-output` / `output-limit` failure), plus the exact `[dsh-code-runtime-python] log capture truncated at bytes` log marker, into a retained `run_code` result. +Indirectly, through Code Mode in `dsh-tools`, which renders the program's completion value or failure into a retained `run_code` result. #### KV Cache effect @@ -34,18 +105,25 @@ No direct invalidation; the named consumer owns any request-prefix changes. ## Known Limitations and Deferred Work -- **The cross-language guard covers executed values and frame field sets, not field types** — `tests/protocol-mirror.e2e.ts` compares `PROTOCOL_FD`, the log truncation marker, and each `TypedDict`'s required and optional fields against a real `python3`. Comparing field types across TypeScript and Python has no mechanical equivalent here, so review plus the backend's real-subprocess suite owns type-level drift. -- **`RLIMIT_AS` is not enforced on macOS** — the dyld shared cache mapped into every process at exec exceeds any practical address-space cap, and the kernel rejects the `setrlimit` call, so `addressSpaceMb` is skipped there. `cpuSeconds` and `maxWallMs` still bound every run. -- **PID-reuse protection is inert on macOS** — `readProcessStart` reads `/proc//stat`, which Darwin does not provide, so the identity re-check that guards `killGroup` against signalling a recycled pgid always passes there; `killGroup` signals the pgid without the identity re-check on macOS rather than paying a `ps` fork on a teardown path. The process-group teardown and the `closeDeadline` bound still contain the run. -- **C-ext stdio buffers are not drained at settlement.** The child runs with `-u` (unbuffered), so `sys.__stdout__`/`sys.__stderr__` and `os.write` bytes are visible to the host's stray capture immediately; but a C extension's private C-stdio (`FILE*`) buffering is outside the interpreter, and its unwritten bytes are lost when the host SIGTERMs the child after the done frame. Model code should flush C-level stdio explicitly before returning if it must survive. + -- **An fd-3 frame whose raw length exceeds 64 MiB settles the run as a worker-exit.** The receive path caps raw frames at `FRAME_PARSE_CAP_BYTES` before `toString`/`JSON.parse` (a compact wide frame could decode to far more host memory than the wire bytes admitted). `maxLogBytes`/`maxValueBytes` are load-bounded to that parser cap so an honest child's frames always fit; a model-constructed binding ARGUMENT above 64 MiB (a value with no seam-level budget) trips the same cap — an accepted residual of the OOM guard. -- **A truncated log's serialized array runs to `maxLogBytes` plus the marker.** The truncation marker is envelope, not payload — it rides uncharged so it can always be emitted — and the outer-array envelope is reserved one byte in the ledger. A truncated run with admitted entries therefore serializes its `logs` array to at most `maxLogBytes + marker + 1`; the marker alone fits any admissible budget (the 64-byte floor guarantees it). -- **A descendant that calls `setsid()` / `start_new_session=True` escapes teardown.** Termination signals the child's process group with `kill(-pid)`; a descendant that moves itself into a fresh session is no longer in that group and no signal reaches it. If it also releases the inherited stdout/stderr/fd-3 pipes, the leader's `close` still settles the run, and after the `closeDeadline` bound the fiber goes quiescent while that orphan keeps running. This is the containment boundary, not a security one — model code has bash-equivalent trust, and a bash tool can `setsid` away just the same. Reaching such an orphan would require tracking every descendant pid (as the bash-local backend's process-inspector does) and is deferred; the process-group teardown reaps everything that stays in the group. -- **A combined log-and-value peak is not modelled by the load gate.** Each budget is checked against `addressSpaceMb` on its own. A model daemon thread that keeps writing while the completion value is metered and framed can refill the log pending toward `maxLogBytes` during that window, so the two peaks add in a way no gate admits or rejects. A gate over `(maxLogBytes + maxValueBytes)` was considered and deferred: its discriminating case cannot be scheduled deterministically under `RLIMIT_AS`, so the gate would only prove its own arithmetic. When the combined peak is reached the run dies as `worker-exit` -- containment holds and only the failure classification is degraded. -- **A 1-second dual-limit `ulimit -t 1` CPU overrun is reported as `worker-exit`, not a timeout.** When the host starts under a hard CPU limit equal to the soft (`ulimit -t N` sets both) and that limit is 1, `_clamped` cannot lower the soft to 0, so the kernel SIGKILLs the busy loop in the same tick and SIGXCPU is never delivered. The host classifies a CPU overrun only on `signal === 'SIGXCPU'`, so the overrun is reported as `worker-exit`. For a dual limit of 2 or more the soft is lowered by one unit, SIGXCPU fires, and the run is a timeout. Containment holds in both cases; only the classification is degraded. -- **A program that traps SIGXCPU can exceed the soft CPU limit during settlement encoding and still report success.** The settlement CPU recheck (`die_if_cpu_exhausted`) runs unconditionally after the program returns and before the log flush and completion encode; a program that exceeded the soft limit before returning is caught there and dies on the re-delivered SIGXCPU, classified as a timeout. The only false-success window is a program that PASSES the recheck and then, with SIGXCPU trapped, exceeds the soft limit during the settlement flush/encode window. A post-encode recheck is not done because it would charge the settlement encode's own CPU to the program, misclassifying a legitimate near-limit program. Containment holds — the hard limit (soft + 1s) and the wall clock still bound it — and only the classification is degraded. -- **The encoder's direct dependencies resolve at call time.** `_encode_json_plain` reaches `_dump_scalar`/`_dump_string`/`json` via module-global lookup, so a program running as `__main__` that rebinds one of those names (e.g. `__main__._dump_scalar = boom`) after returning a legitimate value can make the encode throw and downgrade a success to `exception`. The value path's entry name (`_done_with_value`) is bound into `_run` locals and its top-level `_check_done_value`/`_encode_json_plain` are def-time defaults, but the encoder's transitive deps (e.g. `_dump_scalar`/`_dump_string`/`json`/`io` — a non-exhaustive set) still resolve at call time. This is an accepted residual: under the bash-equivalent trust model a rebind here only harms the model's own run, and the verdict still reaches the host — `send_done`'s fixed fallback frame delivers a done frame even when the error-path encode/write throws. -- **A wide binding REPLY expands host-side state per member.** Resolutions cross through `snapshotJsonValue` in [`@deepseek-ai/dsh-session`](../../core/session/README.md), whose `walkJsonValue` pushes one task frame per member, and binding resolution carries no seam-level byte cap. A legitimate reply of several million elements can therefore exhaust the host heap. The property belongs to that shared walk, not to this backend -- the worker-thread backend consumes the same function -- so the fix belongs in `packages/core/session` where every consumer benefits. -- **A cross-thread binding that the program joins with a synchronous `t.join()` can deadlock.** This is specific to the `process` isolation backend: the reply pump runs on the child's main event loop, so when the program's main coroutine calls `t.join()` on a worker thread that is still awaiting a binding reply, the join blocks the main thread's event loop — the loop the pump needs to deliver that reply — and the worker's `await` never resumes until the wall clock. The worker-thread backend does not share this structure, so the fix belongs here, not in `packages/core/session`. +These limits define what the package does and does not cover; they are current package constraints, not a task backlog. + +- **The cross-language guard covers the executed surfaces and the frame field shapes, not the field types** — the mirror e2e compares required/optional field sets, not that `cpuSeconds` is an `int` on both sides; a type-level drift is caught by review plus the backend's real-subprocess suite. +- **`run()` is one-shot** — `logs` become available only after `CodeRunResult` resolves; there is no streaming-log or progress interface for output produced by a running program. +- **No state persists across runs** — every request executes in a fresh subprocess; a persistent REPL-style kernel stays deferred until a backend brings its own logging scheme. +- **An fd-3 frame whose raw length exceeds 64 MiB settles the run as a worker-exit** — `maxLogBytes`/`maxValueBytes` are load-bounded to the same parser cap so an honest child's frames always fit; a model-constructed binding ARGUMENT above 64 MiB (a value with no seam-level budget) trips the same cap — an accepted residual of the OOM guard. +- **A combined log-and-value peak is not modelled by the load gate** — a model daemon thread that keeps writing while the completion value is metered and framed can add the two peaks in a way no gate admits or rejects; the run dies as `worker-exit`, containment holds, and only the failure classification is degraded. +- **A 1-second dual-limit `ulimit -t 1` CPU overrun is reported as `worker-exit`, not a timeout** — when the host starts under a hard CPU limit equal to the soft and that limit is 1, `_clamped` cannot lower the soft, so the kernel SIGKILLs the busy loop and SIGXCPU is never delivered; containment holds, only the classification is degraded. +- **No byte cap on intermediate binding values** — the implementation remains bounded by structured-clone cost and process memory, and a provider or executor may apply its own fetch cap. + + +### Dev Note + +
+Working context for maintainers — click to expand + +None. + +
diff --git a/packages/code-runtime/code-runtime-python/README.zh.md b/packages/code-runtime/code-runtime-python/README.zh.md index e4284e852b..ee754d03e7 100644 --- a/packages/code-runtime/code-runtime-python/README.zh.md +++ b/packages/code-runtime/code-runtime-python/README.zh.md @@ -1,51 +1,129 @@ --- -description: "DeepSeek Harness 代码执行 seam 的 CPython 子进程实现，提供 fd-3 binding、资源限制、日志捕获与进程组拆卸。" -kind: "package-reference" +description: "CPython 子进程代码 runtime：为 Python 模型代码实现 dsh-code-runtime seam，及其使用的 fd-3 wire 协议。" +kind: "package-library" --- # @deepseek-ai/dsh-code-runtime-python [English](README.md) | 中文 -[`@deepseek-ai/dsh-code-runtime`](../code-runtime/README.zh.md) seam 的 CPython 子进程实现。与 [`@deepseek-ai/dsh-code-runtime-worker-thread`](../code-runtime-worker-thread/README.zh.md) 配套；以全新的 `python3` 子进程取代 Node worker 线程，让模型代码从 TypeScript 换成 Python。 +## 摘要 -本包持有该 seam 的 wire protocol：host 侧的帧编解码，以及 Python 侧对同一套消息词汇的镜像。在该协议之上，本包交付 `PythonCodeRuntime`（插件的默认导出），它以 `language: 'python'`、`isolation: 'process'` 注册为 `codeRuntime`。每次 `run()` 启动一个全新的 `python3 -I` 进程，通过 fd 3 发送 boot 帧和程序，并为每个程序结果 resolve 一个 `CodeRunResult`——`run()` 仅在 seam 被误用时才 reject，例如 binding 命名空间不合法，或对 fiber 已被 dispose 的 runtime 发起调用。配置错误在更早的插件加载期被拒绝：非 Unix 平台、非正或非整数的预算、低于截断标记下限（64）的 `maxLogBytes`、会被 `setTimeout` 截断的定时器值、超过单个 fd-3 帧承载能力的预算，以及最坏峰值会突破 `RLIMIT_AS` 的 `addressSpaceMb`／输出预算组合，都从构造器抛出，因此配置错误在装配时就失败，而不是等到之后某次运行。子进程把程序作为 async 函数体运行，因此顶层 `await` 与 `return` 都可用；binding 调用经 fd 3 以 JSON-lines 回传。containment 不是安全边界——模型代码具有等同 bash 的信任级别；空环境、`RLIMIT_CPU`／`RLIMIT_AS`、墙钟上限与对子进程进程组的 `SIGTERM`→grace→`SIGKILL` 拆卸共同提供 containment。 +`dsh-code-runtime-python` 交付 `PythonCodeRuntime`——[`dsh-code-runtime`](../code-runtime/README.zh.md) seam 的 CPython 子进程实现：它以 `language: 'python'`、`isolation: 'process'` 注册为 `codeRuntime`，每次 `run()` 启动一个全新的 `python3 -I` 子进程，把程序作为 async 函数体执行，通过子进程 fd 3 上的无版本 JSON-lines 协议通信（stdout/stderr 留给程序自己的输出）。宿主侧（`src/protocol.ts`）把每条入站帧都视为敌意并逐字段重建后才读取；Python 侧（`py/protocol.py`）镜像消息词汇。隔离（不是安全边界——模型代码与 bash 同等的信任）来自空环境、`RLIMIT_CPU`/`RLIMIT_AS`、墙钟上限与 `SIGTERM`→宽限→`SIGKILL` 进程组拆卸，所有上限都在插件加载期校验。 -## Wire protocol +## 目录 -host 与 CPython 子进程在子进程的 fd 3 上交换一个无版本号的 JSON-lines 协议——每行一个 JSON 对象，让 stdout/stderr 空出给程序自己的输出。`src/protocol.ts` 是 host 侧；`py/protocol.py` 在 Python 侧镜像其帧词汇与共享的截断标记文本。 +- [使用本包](#use-this-package) +- [理解实现](#understand-the-implementation) +- [进一步探索](#further-exploration) +- [模型体验](#model-experience) +- [已知限制与延期工作](#known-limitations-and-deferred-work) +- [开发备注](#dev-note) -- **fd 3，而非 stdout** —— Node 通过 `stdio: ['pipe','pipe','pipe','pipe']` 按位置钉住通道；Python bootstrap 读取相同的 `PROTOCOL_FD` 常量。JSON-lines 帧。 -- **host 把每个入站帧当作敌意输入** —— 模型代码对 fd 3 有完全访问权、可通过它发送任意内容，所以 `validateChildFrame` 在 host 读取前对每个帧做形状校验并重建：伪造的额外字段绝不随行，非数字的 call id 绝不会被回显进 reply，垃圾降为 `undefined` 被丢弃，而不是在 host 的 message handler 里抛错。Python 侧信任 host 回复（host 不受模型控制）。 -- **lossless-JSON 穿越** —— 完成值与 binding 参数以精确 JSON 穿越。`encodeJsonPlain` 无递归地序列化一个 `JSON.parse` 产出的值，使低于字节预算的深层值能完整穿越，而不是死在 `JSON.stringify` 的栈限制上；`checkDoneValue` 在一次有界遍历中同时计量伪造完成值的字节长度与数字无损性，在把子节点入栈之前就拒绝超预算 payload；`hasUnsafeIntegerToken` 读取原始帧文本，捕获 `JSON.parse` 会静默舍入的整数 token；`hasNonLosslessNumber` 拒绝无字节上限的 `call.args` 中的非有限数或负零。超出安全范围的整数型 double 通过 `BigInt` 数字序列化，穿越的是精确整数而非 `String()` 的舍入形式。 -- **共享截断标记** —— `logTruncationMarker(maxBytes)` 在两侧产出逐字节一致的文本，使被截断的日志运行无论从哪侧触达上限都读起来一致。`log` 帧的 `truncated` 标志把子进程 ledger 自身的标记与程序输出区分开。 +----- -## Configuration + +## 使用本包 -每个上限都是带默认值的、经校验的 `Config` 字段，可从 `cordis.yml` 修改（无硬编码可调项）。`cpuSeconds`（默认 60）是 `RLIMIT_CPU` 的整秒预算；子进程把软限设为 `cpuSeconds`、硬限设为 `cpuSeconds + 1`，因此内核在软限处发出的 `SIGXCPU` 被归类为 `timeout`，而 +1 秒的硬限是 `SIGKILL` 兜底。`maxWallMs`（默认 600000）是墙钟上限，为一个在等待无人 resolve 的 promise 的程序兜住 CPU 时间。`addressSpaceMb`（默认 512）是 `RLIMIT_AS` 上限，在 Darwin 上不施加（那里映射进每个进程的 dyld 共享缓存超过任何实际上限；`cpuSeconds` 与 `maxWallMs` 仍约束运行）。`maxLogBytes`（默认 65536）是共享的捕获日志字节预算；`maxValueBytes`（默认 32768）为完成值设上限；`graceMs`（默认 3000）是 `SIGTERM`→`SIGKILL` 的 grace 窗口；`pythonBin`（默认 `python3`）是解释器，在子进程以空环境启动前先对 `PATH` 解析。 +在需要通过 code-runtime seam 运行 Python 模型代码时选择本包：向 `dsh-tools` 注册 `PythonCodeRuntime`，`run()` 就在全新的 `python3 -I` 子进程中执行每个程序，并对每种程序结果都通过 resolve 结果的 error 字段报告（正交的 `CodeRunFailure.kind` 分类涵盖解析失败、抛出异常、无效完成值、输出溢出、预算到期、中止与执行基底终止）；只有 seam 误用才 reject——绑定命名空间畸形，或已释放后仍调用。配置在加载期被拒绝：非 Unix 平台、非正或非整数的预算、低于截断标记下限（64）的 `maxLogBytes`、`setTimeout` 会收敛的定时器值、超过单个 fd-3 帧可承载的预算，以及最坏峰值会突破 `RLIMIT_AS` 的 `addressSpaceMb`/输出预算组合。 -## Model Experience +### 你得到什么 -间接触达：经由 [`dsh-tools`](../../core/tools/README.md) 中的 Code Mode——它把本后端精确的完成值（在放得下时）或一个明确的 `invalid-output` ／ `output-limit` 失败，连同精确的 `[dsh-code-runtime-python] log capture truncated at bytes` 日志标记，一并渲染进一条被保留的 `run_code` 结果。 +包的默认导出是 `PythonCodeRuntime` 插件。其公开面还重新导出宿主侧协议词汇：`validateChildFrame`（重建每条入站帧）、无损 JSON codec 与计量器（`encodeJsonPlain`、`checkDoneValue`、`hasUnsafeIntegerToken`、`hasNonLosslessNumber`）以及 `logTruncationMarker`（共享截断标记文本）。每个上限都是带默认值并经校验的 `Config` 字段：`cpuSeconds`（60）、`maxWallMs`（600000）、`addressSpaceMb`（512，Darwin 上不生效）、`maxLogBytes`（65536）、`maxValueBytes`（32768）、`graceMs`（3000）与 `pythonBin`（`python3`，在子进程以空环境启动前对照 `PATH` 解析）。 -#### KV Cache effect +### wire -不直接造成失效；任何对请求前缀的改动由上述具名 Consumer 负责。 +帧在子进程 fd 3 上以 JSON-lines 传输——每行一个对象——因此 stdout/stderr 留给程序自己的输出。子进程 → 宿主：`boot-ack`、`call`、`log`、`done`。宿主 → 子进程：`boot`（首帧，携带全部上限与命名空间声明）、`run`（`boot-ack` 之后，只携带程序体）与每个 `call` 一个 `reply`。伪造帧可在 `done` 上同时携带 `value` 与 `error`，因此消费方必须先检查 `error`，在它存在时忽略 `value`。 -## Known Limitations and Deferred Work +### 可能出错的地方 -- **跨语言 guard 覆盖执行值与帧字段集，但不覆盖字段类型** —— `tests/protocol-mirror.e2e.ts` 使用真实 `python3` 比较 `PROTOCOL_FD`、日志截断标记，以及每个 `TypedDict` 的必填和可选字段。跨 TypeScript 与 Python 比较字段类型在此没有机械等价物，因此类型级漂移由 review 加后端真子进程套件负责。 -- **`RLIMIT_AS` 在 macOS 上不施加** —— 在 exec 时映射进每个进程的 dyld 共享缓存超过任何实际的地址空间上限，内核会拒绝该 `setrlimit` 调用，故 `addressSpaceMb` 在那里被跳过。`cpuSeconds` 与 `maxWallMs` 仍约束每一次运行。 -- **PID 复用防护在 macOS 上失效** —— `readProcessStart` 读取 `/proc//stat`，Darwin 不提供它，因此防止 `killGroup` 对已回收的 pgid 发信号的同一性复检在那里恒通过；`killGroup` 在 macOS 上不经同一性复检直接对 pgid 发信号，而非在拆卸路径上付出一次 `ps` fork。进程组拆卸与 `closeDeadline` 上界仍约束该次运行。 -- **C 扩展的 stdio 缓冲在结算时不被排空。** 子进程以 `-u`（无缓冲）运行，因此 `sys.__stdout__`/`sys.__stderr__` 与 `os.write` 的字节立即可见；但 C 扩展私有的 C-stdio（`FILE*`）缓冲在解释器之外，其未写出的字节会在宿主于 done 帧后 SIGTERM 子进程时丢失。模型代码若需保留，应在返回前显式 flush C 层 stdio。 +宿主侧校验在不抛异常的情况下丢弃垃圾，因此畸形或伪造帧永远不会让宿主进程崩溃：`validateChildFrame` 对任何不能干净重建的内容返回 `undefined`，非数字的 call id 永远不会被回显进 reply，伪造的额外字段永远不会被带走。非无损 JSON 或超过配置字节预算的完成值会被显式拒绝（`non-lossless`／`over-budget`），而不是被静默取整或截断。原始长度超过 64 MiB 的 fd-3 帧会让本次运行以 `worker-exit` 结算（接收路径在 `toString`/`JSON.parse` 之前限制原始帧，紧凑宽帧不能解码出远超其线上字节的宿主内存）。 -- **原始长度超过 64 MiB 的 fd-3 帧会让本次运行以 worker-exit 结算。** 接收路径在 `toString`/`JSON.parse` 之前把原始帧限制在 `FRAME_PARSE_CAP_BYTES`（紧凑宽帧解码后可能占用远超线上字节的宿主内存）。`maxLogBytes`/`maxValueBytes` 在加载期被限制到该解析器上限，因此诚实子进程的帧总能放得下；模型构造的超过 64 MiB 的 binding 实参（一个在 seam 层没有预算的值）会触发同一上限——这是该 OOM 防护的已接受残余。 +----- -- **截断日志的序列化数组会到 `maxLogBytes` 加标记为止。** 截断标记是 envelope 而非 payload——它不计费地随行，因此总能发出——而外层数组外壳在账本中预留了一字节。因此带已放行条目的截断运行，其 `logs` 数组序列化后至多为 `maxLogBytes + marker + 1`；标记单独能放进任何可接受的预算（64 字节下限保证这一点）。 -- **调用 `setsid()` ／ `start_new_session=True` 的后代会逃出 teardown。** 终止是用 `kill(-pid)` 向子进程的进程组发信号；一个把自己移入新会话的后代已不在该进程组内，任何信号都到不了它。若它同时释放了继承而来的 stdout／stderr／fd-3 管道，leader 的 `close` 仍会结算该次运行，在 `closeDeadline` 到界之后 fiber 变为完全停稳，而那个孤儿仍在运行。这是 containment 边界，而非安全边界——模型代码具有等同 bash 的信任级别，一个 bash 工具同样能 `setsid` 逃逸。要够到这样的孤儿需要追踪每一个后代 pid（如 bash-local 后端的 process-inspector 所做），此项已推迟；进程组 teardown 会回收所有留在组内的进程。 -- **日志与完成值的叠加峰值未被加载门建模。** 每项预算都是各自对照 `addressSpaceMb` 检查的。模型的 daemon 线程可以在完成值被计量并分帧的窗口内持续写入、把日志 pending 重填到接近 `maxLogBytes`，于是两个峰值以任何门都不曾放行也不曾拒绝的方式相加。对 `(maxLogBytes + maxValueBytes)` 设门的方案经评估后推迟：它的判别用例无法在 `RLIMIT_AS` 之下确定性地构造出来，因此该门只能证明自己的算术。叠加峰值被触及时该次运行死为 `worker-exit`——containment 仍然成立，只是失败分类失真。 -- **1 秒双限 `ulimit -t 1` 下的 CPU 超限会被报告为 `worker-exit`，而非超时。** 当宿主在一个硬 CPU 限制等于软限制（`ulimit -t N` 同时设置两者）且该限制为 1 的环境下启动时，`_clamped` 无法把软限制降到 0，因此内核在同一 tick 直接 SIGKILL 忙循环，SIGXCPU 永不送达。宿主只在 `signal === 'SIGXCPU'` 时把 CPU 超限分类为超时，因此该超限被报告为 `worker-exit`。当双限为 2 或更大时，软限制会被降低一个单位，SIGXCPU 触发，该次运行成为超时。两种情况 containment 都成立；只是分类被降级。 -- **一个 trap SIGXCPU 的程序可以在结算编码期间超过软 CPU 限制并仍报告成功。** 结算时的 CPU 复查（`die_if_cpu_exhausted`）在程序返回后、日志 flush 与完成值编码之前无条件运行；一个在返回前已超过软限制的程序会在这里死于重投递的 SIGXCPU，被归类为超时。唯一的误报窗口是一个通过复查后、trap 住 SIGXCPU 并在结算 flush/编码窗口内越过软限制的程序。不做编码后复查，是因为那会把结算编码自身消耗的 CPU 记到程序头上、误分类一个合法的近限程序。containment 成立——硬限制（软限制 + 1s）与墙钟仍会约束它——只是分类被降级。 -- **编码器的直接依赖在调用时解析。** `_encode_json_plain` 通过模块全局查找到达 `_dump_scalar`/`_dump_string`/`json`，因此以 `__main__` 运行的程序在返回合法值后重绑这些名字之一（例如 `__main__._dump_scalar = boom`）可以让编码抛出、把成功降级为 `exception`。值路径的入口名（`_done_with_value`）被绑定进 `_run` 局部、其顶层的 `_check_done_value`/`_encode_json_plain` 是 def 期默认值，但编码器的传递依赖（例如 `_dump_scalar`/`_dump_string`/`json`/`io`——非穷举清单）仍在调用时解析。这是已接受的残余：在 bash-equivalent 信任模型下，这里的重绑只会伤害模型自身的运行，且判决仍必达宿主——`send_done` 的固定兜底帧即使在错误路径编码/写入抛出时也能送达一帧 done。 -- **宽 binding 回复会按成员展开宿主侧状态。** 回复经由 [`@deepseek-ai/dsh-session`](../../core/session/README.zh.md) 的 `snapshotJsonValue` 穿越，其 `walkJsonValue` 为每个成员压入一个任务帧，而 binding 回复在 seam 层没有字节上限。因此一个数百万元素的合法回复可以耗尽宿主堆。该性质属于那个共享遍历，而不属于本后端——worker-thread 后端消费同一个函数——所以修复应落在 `packages/core/session`，让所有消费方一并受益。 -- **程序用同步的 `t.join()` 连接一个跨线程 binding 会死锁。** 这是 `process` 隔离后端特有的：回复泵运行在子进程的主事件循环上，因此当程序的主协程对一个仍在等待 binding 回复的 worker 线程调用 `t.join()` 时，join 会阻塞承载泵的主线程事件循环——正是泵投递该回复所需的循环——该 worker 的 `await` 直到墙钟才会恢复。worker-thread 后端不共享此结构，所以修复应落在这里，而非 `packages/core/session`。 + +## 理解实现 + +
+实现内部——点击展开 + +本节解释后端背后的设计；可观察行为在[使用本包](#use-this-package)中完整覆盖。 + +### 设计概念 + +单向信任：宿主把每条入站帧都视为敌意（模型代码可以在 fd 3 上伪造任何内容）并逐字段重建后才读取；Python 侧信任宿主回复。bootstrap（`py/bootstrap.py`）把程序作为 async 函数体执行，因此顶层 `await` 与 `return` 都可用；binding 调用经 fd 3 以 JSON-lines 往返，回复在 pump 中限速，以免大量大回复钉住宿主的 fd-3 可写缓冲。 + +### wire 契约 + +帧为 `boot`／`run`（宿主 → 子进程）与 `boot-ack`／`call`／`log`／`done` 加每个 call 一个 `reply`（子进程 → 宿主）。`log` 帧的 `truncated` 标志标记的就是子进程账本自己的截断标记帧，因此宿主在与子进程相同的点停止捕获，而不是从自己的预算推断。`done.error.kind` 为 `exception`、`invalid-output`、`output-limit` 之一；墙钟／CPU 预算、中止与基底死亡在宿主侧观察，不以帧形式携带。 + +### 无损 JSON 跨越 + +完成值与 binding 实参以精确 JSON 跨越：值无递归序列化，因此低于字节预算的深层载荷存活，而不会死在 `JSON.stringify` 的栈上限；超出安全范围的整型 double 以精确数字跨越，而不是被静默取整的 token；`src/protocol.ts` 中的计量器在任何其他代码读取载荷之前强制字节预算与数字无损性。 + +### 镜像对齐 + +`tests/protocol-mirror.e2e.ts` 启动真实 `python3`，对照 `src/protocol.ts` 断言 `PROTOCOL_FD`／截断标记文本以及 `py/protocol.py` 中每个 `TypedDict` 的必填／可选 wire 字段集，因此字段改名、删除或一侧把另一侧必填的字段变成可选都会使测试失败。字段*类型*不跨语言边界比较；该残留由评审加后端的真实子进程套件（`tests/runtime.spec.ts`）负责。 + +### 源码地图 + +| 文件 | 职责 | +|---|---| +| [`src/index.ts`](src/index.ts) | 插件入口：`PythonCodeRuntime`——spawn、帧 pump、预算、隔离、拆卸；重新导出协议词汇 | +| [`src/protocol.ts`](src/protocol.ts) | 宿主侧：帧 codec、敌意帧校验器、无损 JSON 计量器、共享标记文本 | +| [`py/bootstrap.py`](py/bootstrap.py) | 子进程侧：fd-3 通道、程序执行、binding 分发、账本与结算 | +| [`py/protocol.py`](py/protocol.py) | Python 侧：`PROTOCOL_FD`、`TypedDict` 帧镜像、`log_truncation_marker` | +| [`tests/runtime.spec.ts`](tests/runtime.spec.ts) | 真实子进程套件：预算、隔离、敌意帧、名称重绑 | +| [`tests/protocol-mirror.e2e.ts`](tests/protocol-mirror.e2e.ts) | 对照真实 `python3` 的跨语言镜像测试 | +| [`src/invariant.ts`](src/invariant.ts) | 不变式伴生（无运行时不变式；本包不注册可变数据关系） | + +
+ +----- + + +## 进一步探索 + +当 runtime 契约不够时阅读这些。它们从 seam 定义走向设计记录与配套后端。 + +- [Code runtime seam](../code-runtime/README.zh.md) — 本后端实现的抽象契约。 +- [fd-3 协议 Agent Note](../../../.agents/notes/implemented/architecture/2026-07-31-code-runtime-python-fd3-protocol.zh.md) — 设计理由与 wire 契约。 +- [结算修复 Agent Note](../../../.agents/notes/implemented/bug-fix/2026-07-31-code-runtime-python-settlement-fixes.zh.md) — 结算、计量与隔离修复及其回归用例。 +- [Worker 线程后端](../code-runtime-worker-thread/README.zh.md) — 已发布的 TypeScript 兄弟。 +- [Code runtime 子系统参考](../../../docs/subsystems/code-runtime.zh.md) — 请求／结果词汇、binding 与失败分类。 + +----- + + +## 模型体验 + +间接地，通过 `dsh-tools` 中的 Code Mode，它把程序的完成值或失败渲染成保留的 `run_code` 结果。 + +#### KV Cache 效应 + +无直接失效；指定的消费方拥有任何请求前缀变化。 + +## 已知限制与延期工作 + + + + +这些限制定义本包覆盖与不覆盖的内容；它们是当前包约束，不是任务积压。 + +- **跨语言 guard 覆盖执行的表面与帧字段形状，而非字段类型**——mirror e2e 比较必填／可选字段集，而非 `cpuSeconds` 在两侧是否都是 `int`；类型级漂移由评审加后端的真实子进程套件捕获。 +- **`run()` 是一次性的**——`logs` 只有在 `CodeRunResult` resolve 后才能获得；没有为运行中程序产生的输出提供流式日志或进度接口。 +- **运行之间不保留状态**——每次请求都在全新子进程中执行；持久 REPL 风格内核在某个后端带来自己的日志方案之前保持延期。 +- **原始长度超过 64 MiB 的 fd-3 帧会让本次运行以 worker-exit 结算**——`maxLogBytes`/`maxValueBytes` 在加载期被限制到同一解析器上限，因此诚实子进程的帧总能放得下；模型构造的超过 64 MiB 的 binding 实参（一个在 seam 层没有预算的值）会触发同一上限——这是该 OOM 防护的已接受残余。 +- **组合日志与值的峰值不被加载门建模**——持续写入的模型 daemon 线程与完成值计量、分帧相加的峰值没有任何门会放行或拒绝；运行以 `worker-exit` 告终，隔离成立，只有失败分类降级。 +- **1 秒双限 `ulimit -t 1` CPU 超限被报告为 `worker-exit` 而非 timeout**——当宿主在一个与软限相等的硬 CPU 限下启动且该限为 1 时，`_clamped` 无法下调软限，内核在同一 tick SIGKILL 忙循环，SIGXCPU 永远不会送达；隔离成立，只有分类降级。 +- **中间 binding 值没有字节上限**——实现仍受 structured-clone 成本与进程内存约束，提供方或执行器可能应用自己的获取上限。 + + +### 开发备注 + +
+维护者的工作上下文——点击展开 + +无。 + +