mirror of
https://github.com/deepseek-ai/deepseek-harness.git
synced 2026-09-11 04:00:38 +00:00
fix(code-runtime): settle Python provider contracts
This commit is contained in:
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/experimental/code-runtime-python/README.md
|
||||
README.md: 1009c150a320e23811bae01e989e82cefeb9b907
|
||||
README.zh.md: b3dd8803855b9f579f2d1cfdd155ff3691b4573b
|
||||
README.md: 8b596ec5e8bcb7a0d458fe11e61c3742b6efc823
|
||||
README.zh.md: a0035beec49a531d7ff37363921ecefa869877a6
|
||||
|
||||
@@ -9,7 +9,7 @@ English | [中文](README.zh.md)
|
||||
|
||||
## Summary
|
||||
|
||||
`dsh-experimental-code-runtime-python` ships `PythonCodeRuntime`, the CPython-subprocess implementation of the [`dsh-code-runtime`](../../code-runtime/code-runtime/README.md) seam: it registers as `codeRuntime` with `language: 'python'` and `isolation: 'process'`, spawning a fresh `python3 -I` child per `run()` and executing the program as an async function body over a versionless JSON-lines protocol on the child's fd 3 (stdout/stderr stay free for the program's own output). The host side (`src/protocol.ts`) treats every inbound frame as hostile and rebuilds it before reading; the Python side (`py/protocol.py`) mirrors the message vocabulary. Containment — not a security boundary, model code has bash-equivalent trust — comes from an empty environment, `RLIMIT_CPU`/`RLIMIT_AS`, a wall-clock ceiling, and `SIGTERM`→grace→`SIGKILL` process-group teardown, with all caps validated at plugin load.
|
||||
`dsh-experimental-code-runtime-python` provides the private source-checkout `PythonCodeRuntime`, a CPython-subprocess implementation of the [`dsh-code-runtime`](../../code-runtime/code-runtime/README.md) seam. It registers as `codeRuntime` with `language: 'python'` and `isolation: 'process'`, spawning a fresh CPython 3.10+ child per `run()` and executing the program as an async function body over a versionless JSON-lines protocol on the child's fd 3 (stdout/stderr stay free for the program's own output). The host side (`src/protocol.ts`) treats every inbound frame as hostile and rebuilds it before reading; the Python side (`py/protocol.py`) mirrors the message vocabulary. Containment — not a security boundary, model code has bash-equivalent trust — comes from a tempdir-only environment, `RLIMIT_CPU`/`RLIMIT_AS`, a wall-clock ceiling, and `SIGTERM`→grace→`SIGKILL` process-group teardown, with all caps validated at plugin load.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
@@ -25,11 +25,11 @@ English | [中文](README.zh.md)
|
||||
<a id="use-this-package"></a>
|
||||
## Use this package
|
||||
|
||||
Choose this package to run Python model code through the code-runtime seam: register `PythonCodeRuntime` with `dsh-tools` and `run()` executes each program in a fresh `python3 -I` subprocess, resolving with `result.value` on success and `result.error` on failure (the orthogonal `CodeRunFailure.kind` taxonomy classifies parse failures, thrown exceptions, invalid completions, output overflows, budget expiry, aborts, and substrate death). It rejects only for seam misuse — a malformed binding namespace, or a call after disposal. Configuration is rejected at load: a non-Unix platform, a non-positive or non-integer budget, a `maxLogBytes` below the truncation-marker floor (64), a timer value `setTimeout` would clamp, a budget larger than one fd-3 frame can carry, an `addressSpaceMb`/output-budget pair whose worst-case peak would breach `RLIMIT_AS`, and a `pythonBin` that is not an executable regular file — an explicit path (absolute or containing `/`) is judged directly, a bare name is judged against `PATH`.
|
||||
Choose this private experimental package only in an explicit source-checkout composition. Register `PythonCodeRuntime` beside `dsh-tools` and `run()` executes each program in a fresh CPython 3.10+ subprocess, resolving with `result.value` on success and `result.error` on failure (the orthogonal `CodeRunFailure.kind` taxonomy classifies parse failures, thrown exceptions, invalid completions, output overflows, budget expiry, aborts, and substrate death). It rejects only for seam misuse — a malformed binding namespace, or a call after disposal. Configuration is rejected at load: a non-Unix platform; an explicit `pythonBin` that is not an executable regular file or a bare name that does not resolve on `PATH`; a non-CPython, pre-3.10, or probe-failing interpreter; a non-positive or non-integer budget; a `maxLogBytes` below the truncation-marker floor (64); a timer value `setTimeout` would clamp; a budget larger than one fd-3 frame can carry; or an `addressSpaceMb`/output-budget pair whose worst-case peak would breach `RLIMIT_AS`.
|
||||
|
||||
### What you get
|
||||
|
||||
The package's default export is the `PythonCodeRuntime` plugin. Its public surface also re-exports the host-side protocol vocabulary: `validateChildFrame` (rebuilds every inbound frame), the lossless-JSON codec and meters (`encodeJsonPlain`, `checkDoneValue`, `hasUnsafeIntegerToken`, `hasNonLosslessNumber`), `logTruncationMarker` (the shared truncation-marker text), plus `resolvePythonBin` (interpreter lookup against the current `PATH`), `readProcessStart` (process-start statistics for tests), and `detachResidual` (a test seam for the settled run's resource cleanup). Every cap is a validated `Config` field with a default: `cpuSeconds` (60), `maxWallMs` (600000), `addressSpaceMb` (512, not applied on Darwin), `maxLogBytes` (65536), `maxValueBytes` (32768), `graceMs` (3000), and `pythonBin` (`python3`, resolved before the child spawns with an empty environment: an explicit path must be an executable regular file, a bare name must resolve on `PATH`; either failure is rejected at load, distinguishing 'is not an executable regular file' from 'does not resolve on PATH' instead of silently falling to the platform default `PATH`).
|
||||
The package's default export is the `PythonCodeRuntime` plugin. Its public surface also re-exports the host-side protocol vocabulary: `validateChildFrame` (rebuilds every inbound frame), the lossless-JSON codec and meters (`encodeJsonPlain`, `checkDoneValue`, `hasUnsafeIntegerToken`, `hasNonLosslessNumber`), `logTruncationMarker` (the shared truncation-marker text), plus `resolvePythonBin` (interpreter lookup against the current `PATH`), `readProcessStart` (process-start statistics for tests), and `detachResidual` (a test seam for the settled run's resource cleanup). Every cap is a validated `Config` field with a default: `cpuSeconds` (60), `maxWallMs` (600000), `addressSpaceMb` (512, not applied on Darwin), `maxLogBytes` (65536), `maxValueBytes` (32768), `graceMs` (3000), and `pythonBin` (`python3`, resolved, executable-checked, version-probed, and frozen at load). Each child receives only `TMPDIR`; ambient credentials, `PATH`, `HOME`, and other host state stay unavailable.
|
||||
|
||||
### The wire
|
||||
|
||||
@@ -97,7 +97,7 @@ Read these when the runtime contract is not enough. They move from the seam defi
|
||||
<a id="model-experience"></a>
|
||||
## Model Experience
|
||||
|
||||
Indirectly, through Code Mode in `dsh-tools`, which renders the program's completion value or failure into a retained `run_code` result.
|
||||
Indirectly, through PTC mode in `dsh-tools` when an explicit source-checkout composition mounts this provider; it renders the program's completion value or failure into a retained `run_code` result, and no shipped profile mounts this private package.
|
||||
|
||||
#### KV Cache effect
|
||||
|
||||
@@ -114,7 +114,9 @@ These limits define what the package does and does not cover; they are current p
|
||||
- **A descendant that escapes the child's process group with `setsid()` is not reaped by the group teardown** — `kill(-pid)` cannot reach it; the run still settles on the value the done frame decided, and the close-deadline backstop forces settlement if the orphan holds the pipes open, but the orphan itself outlives the fiber until it exits on its own.
|
||||
- **A `log` frame that arrives after settlement is dropped** — once the run has settled, host-side capture is closed; a late fd-3 `log` frame (from a thread that outlived the done frame) is discarded rather than appended to `logs`.
|
||||
- **A binding REPLY value has no seam-level byte or depth cap** — `maxValueBytes` meters only the done frame's completion value; a wide binding reply is rebuilt host-side (`snapshotJsonValue` traversal) and encoded whole, bounded on both sides only by process memory (like a binding argument, which has no child-side budget either).
|
||||
- **A real-Loader assembly snapshot is deferred to issue #1182 layer 5** — this package is exercised through `ctx.plugin(...)` and real-subprocess tests; the full dsh application composition (codeRuntime registered through a real Loader) is covered by a tracked assembly test in that layer, not by this package's suite.
|
||||
- **No shipped profile mounts this provider** — the keyless `ptc-python-turn` snapshot replaces the headless PTC runtime through the real Loader; released profiles continue to use the worker-thread backend.
|
||||
- **Cross-channel log interleaving is backend-dependent** — Python stdout, stderr, and fd-3 log frames travel independently; each channel preserves its own order, while their total order in `result.logs` may differ.
|
||||
- **CPython 3.10 or newer is required** — the configured executable is resolved and version-probed at load; unsupported interpreters fail before `ctx.codeRuntime` is registered.
|
||||
- **The truncation-marker text and the tempdir prefix keep the pre-rename short names** — the marker `[dsh-code-runtime-python] log capture truncated at <N> bytes` and the `dsh-code-runtime-python-` tempdir prefix are byte-anchored by tests and are independent of the npm package name; promotion (dropping the `experimental-` prefix) does not rename them.
|
||||
- **`run()` is one-shot** — `logs` become available only after `CodeRunResult` resolves; there is no streaming-log or progress interface for output produced by a running program.
|
||||
- **No state persists across runs** — every request executes in a fresh subprocess; a persistent REPL-style kernel stays deferred until a backend brings its own logging scheme.
|
||||
|
||||
@@ -9,7 +9,7 @@ kind: "package-reference"
|
||||
|
||||
## 概述
|
||||
|
||||
`dsh-experimental-code-runtime-python` 交付 `PythonCodeRuntime`——[`dsh-code-runtime`](../../code-runtime/code-runtime/README.zh.md) seam 的 CPython 子进程实现:它以 `language: 'python'`、`isolation: 'process'` 注册为 `codeRuntime`,每次 `run()` 启动一个全新的 `python3 -I` 子进程,把程序作为 async 函数体执行,通过子进程 fd 3 上的无版本 JSON-lines 协议通信(stdout/stderr 留给程序自己的输出)。宿主侧(`src/protocol.ts`)把每条入站帧都视为敌意并逐字段重建后才读取;Python 侧(`py/protocol.py`)镜像消息词汇。隔离(不是安全边界——模型代码与 bash 同等的信任)来自空环境、`RLIMIT_CPU`/`RLIMIT_AS`、墙钟上限与 `SIGTERM`→宽限→`SIGKILL` 进程组拆卸,所有上限都在插件加载期校验。
|
||||
`dsh-experimental-code-runtime-python` 提供私有的源码 checkout `PythonCodeRuntime`,即 [`dsh-code-runtime`](../../code-runtime/code-runtime/README.zh.md) seam 的 CPython 子进程实现。它以 `language: 'python'`、`isolation: 'process'` 注册为 `codeRuntime`,每次 `run()` 启动一个全新的 CPython 3.10+ 子进程,把程序作为 async 函数体执行,通过子进程 fd 3 上的无版本 JSON-lines 协议通信(stdout/stderr 留给程序自己的输出)。宿主侧(`src/protocol.ts`)把每条入站帧都视为敌意并逐字段重建后才读取;Python 侧(`py/protocol.py`)镜像消息词汇。隔离(不是安全边界——模型代码与 bash 同等的信任)来自仅含临时目录的环境、`RLIMIT_CPU`/`RLIMIT_AS`、墙钟上限与 `SIGTERM`→宽限→`SIGKILL` 进程组拆卸,所有上限都在插件加载期校验。
|
||||
|
||||
## 目录
|
||||
|
||||
@@ -25,11 +25,11 @@ kind: "package-reference"
|
||||
<a id="use-this-package"></a>
|
||||
## 使用本包
|
||||
|
||||
在需要通过 code-runtime seam 运行 Python 模型代码时选择本包:向 `dsh-tools` 注册 `PythonCodeRuntime`,`run()` 就在全新的 `python3 -I` 子进程中执行每个程序,成功时以 `result.value` resolve、失败时以 `result.error` resolve(正交的 `CodeRunFailure.kind` 分类涵盖解析失败、抛出异常、无效完成值、输出溢出、预算到期、中止与执行基底终止);只有 seam 误用才 reject——绑定命名空间畸形,或已释放后仍调用。配置在加载期被拒绝:非 Unix 平台、非正或非整数的预算、低于截断标记下限(64)的 `maxLogBytes`、`setTimeout` 会收敛的定时器值、超过单个 fd-3 帧可承载的预算、最坏峰值会突破 `RLIMIT_AS` 的 `addressSpaceMb`/输出预算组合,以及不是可执行普通文件的 `pythonBin`——显式路径(绝对或含 `/`)直接判定,裸名对照 `PATH` 判定。
|
||||
仅在显式源码检出组合中选择这个私有实验包。将 `PythonCodeRuntime` 与 `dsh-tools` 一起注册后,`run()` 会在全新的 CPython 3.10+ 子进程中执行每个程序;成功时以 `result.value` resolve,失败时以 `result.error` resolve(正交的 `CodeRunFailure.kind` 分类涵盖解析失败、抛出异常、无效完成值、输出溢出、预算到期、中止与执行基底终止)。仅有 seam 误用会 reject——binding 命名空间不合法,或在 dispose 后调用。配置在加载期拒绝:非 Unix 平台;不是可执行普通文件的显式 `pythonBin`,或无法在 `PATH` 上解析的裸名;非 CPython、低于 3.10 或探测失败的解释器;非正或非整数预算;低于截断标记下限(64)的 `maxLogBytes`;会被 `setTimeout` 截断的定时器值;超过单个 fd-3 帧承载能力的预算;或最坏峰值会突破 `RLIMIT_AS` 的 `addressSpaceMb`/输出预算组合。
|
||||
|
||||
### 你得到什么
|
||||
|
||||
包的默认导出是 `PythonCodeRuntime` 插件。其公开面还重新导出宿主侧协议词汇:`validateChildFrame`(重建每条入站帧)、无损 JSON codec 与计量器(`encodeJsonPlain`、`checkDoneValue`、`hasUnsafeIntegerToken`、`hasNonLosslessNumber`)、`logTruncationMarker`(共享截断标记文本),以及 `resolvePythonBin`(对照当前 `PATH` 的解释器查找)、`readProcessStart`(供测试用的进程启动统计)和 `detachResidual`(已结算运行的资源清理测试 seam)。每个上限都是带默认值并经校验的 `Config` 字段:`cpuSeconds`(60)、`maxWallMs`(600000)、`addressSpaceMb`(512,Darwin 上不生效)、`maxLogBytes`(65536)、`maxValueBytes`(32768)、`graceMs`(3000)与 `pythonBin`(`python3`,在子进程以空环境启动前解析:显式路径必须是可执行普通文件,裸名必须在 `PATH` 上可解析;任一失败都在加载期被拒绝,区分『is not an executable regular file』与『does not resolve on PATH』,而不是静默回退到平台默认 `PATH`)。
|
||||
包的默认导出是 `PythonCodeRuntime` 插件。其公开面还重新导出宿主侧协议词汇:`validateChildFrame`(重建每条入站帧)、无损 JSON codec 与计量器(`encodeJsonPlain`、`checkDoneValue`、`hasUnsafeIntegerToken`、`hasNonLosslessNumber`)、`logTruncationMarker`(共享截断标记文本),以及 `resolvePythonBin`(对照当前 `PATH` 的解释器查找)、`readProcessStart`(供测试用的进程启动统计)和 `detachResidual`(已结算运行的资源清理测试 seam)。每个上限都是带默认值并经校验的 `Config` 字段:`cpuSeconds`(60)、`maxWallMs`(600000)、`addressSpaceMb`(512,Darwin 上不生效)、`maxLogBytes`(65536)、`maxValueBytes`(32768)、`graceMs`(3000)与 `pythonBin`(`python3`,在加载期解析、检查可执行性、探测版本并固定)。每个子进程只接收 `TMPDIR`;环境中的凭证、`PATH`、`HOME` 与其他宿主状态均不可见。
|
||||
|
||||
### wire
|
||||
|
||||
@@ -97,7 +97,7 @@ kind: "package-reference"
|
||||
<a id="model-experience"></a>
|
||||
## 模型体验
|
||||
|
||||
间接地,通过 `dsh-tools` 中的 Code Mode,它把程序的完成值或失败渲染成保留的 `run_code` 结果。
|
||||
间接地,通过 `dsh-tools` 中的 PTC mode;当显式的源码 checkout 组合挂载本提供方时,它会把程序的完成值或失败渲染成保留的 `run_code` 结果,且已发布 profile 均不挂载这个私有包。
|
||||
|
||||
#### KV Cache 效应
|
||||
|
||||
@@ -114,6 +114,9 @@ kind: "package-reference"
|
||||
- **以 `setsid()` 逃出子进程组后代不被组拆卸回收**——`kill(-pid)` 够不到它;运行仍按 done 帧决定的值结算,若该孤儿持有管道,close 截止兜底会强制结算,但孤儿本身在自行退出前一直存活到 fiber 之外。
|
||||
- **结算后到达的 `log` 帧被丢弃**——运行一旦结算,宿主侧捕获即关闭;迟到的 fd-3 `log` 帧(来自比 done 帧存活更久的线程)会被丢弃,而不是追加到 `logs`。
|
||||
- **binding 回复值没有 seam 级字节或深度上限**——`maxValueBytes` 只计量 done 帧的完成值;宽 binding 回复在宿主侧重建(`snapshotJsonValue` 遍历)并整帧编码,两侧都只受进程内存约束(与没有子进程侧预算的 binding 实参一样)。
|
||||
- **已发布 profile 均不挂载本提供方**——keyless `ptc-python-turn` 快照通过真实 Loader 替换 headless PTC 运行时;已发布 profile 继续使用 Worker 线程后端。
|
||||
- **跨通道日志交错由后端决定**——Python stdout、stderr 与 fd-3 日志帧彼此独立传输;每个通道保留自身顺序,但它们在 `result.logs` 中的总顺序可能不同。
|
||||
- **需要 CPython 3.10 或更高版本**——配置的可执行文件会在加载期完成解析与版本探测;不受支持的解释器会在 `ctx.codeRuntime` 注册前失败。
|
||||
- **`run()` 是一次性的**——`logs` 只有在 `CodeRunResult` resolve 后才能获得;没有为运行中程序产生的输出提供流式日志或进度接口。
|
||||
- **运行之间不保留状态**——每次请求都在全新子进程中执行;持久 REPL 风格内核在某个后端带来自己的日志方案之前保持延期。
|
||||
- **原始长度超过 64 MiB 的 fd-3 帧会让本次运行以 worker-exit 结算**——`maxLogBytes`/`maxValueBytes` 在加载期被限制到同一解析器上限,因此诚实子进程的帧总能放得下;模型构造的超过 64 MiB 的 binding 实参(一个在 seam 层没有预算的值)会触发同一上限——这是该 OOM 防护的已接受残余。
|
||||
@@ -122,7 +125,6 @@ kind: "package-reference"
|
||||
- **组合日志与值的峰值不被加载门建模**——持续写入的模型 daemon 线程与完成值计量、分帧相加的峰值没有任何门会放行或拒绝;运行以 `worker-exit` 告终,隔离成立,只有失败分类降级。
|
||||
- **1 秒双限 `ulimit -t 1` CPU 超限被报告为 `worker-exit` 而非 timeout**——当宿主在一个与软限相等的硬 CPU 限下启动且该限为 1 时,`_clamped` 无法下调软限,内核在同一 tick SIGKILL 忙循环,SIGXCPU 永远不会送达;隔离成立,只有分类降级。
|
||||
- **中间 binding 值没有字节上限**——实现仍受无损 JSON 序列化成本与进程内存约束,提供方或执行器可能应用自己的获取上限。
|
||||
- **真实 Loader 装配态快照推迟到 issue #1182 layer 5**——本包通过 `ctx.plugin(...)` 与真实子进程测试得到验证;完整的 dsh 应用组合(codeRuntime 经真实 Loader 注册)由该层一个受跟踪的装配测试覆盖,不由本包的测试套件承担。
|
||||
- **截断标记文本与临时目录前缀保留改名前的短名**——标记 `[dsh-code-runtime-python] log capture truncated at <N> bytes` 与 `dsh-code-runtime-python-` 临时目录前缀被测试逐字节锚定,且独立于 npm 包名;promotion(去掉 `experimental-` 前缀)不会重命名它们。
|
||||
|
||||
<a id="dev-note"></a>
|
||||
|
||||
@@ -7,8 +7,8 @@ the completion), and posts a terminal :class:`DoneMessage`. The program calls
|
||||
host functions through the ``tools`` (or other namespace) proxy, whose attribute
|
||||
and subscript access return awaitables that ride binding messages over fd 3.
|
||||
|
||||
This module runs under ``python3 -I`` with an empty environment and
|
||||
``sys.path`` containing only its own directory.
|
||||
This module runs under ``python3 -I`` with only ``TMPDIR`` in its environment
|
||||
and ``sys.path`` containing only its own directory.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -223,7 +223,7 @@ class _LogStream(io.TextIOBase):
|
||||
Installed as ``sys.stdout`` / ``sys.stderr`` before executing the model
|
||||
program. ``print(...)`` calls ``write`` once per argument, separator, and
|
||||
newline, so a raw one-push-per-write stream would emit
|
||||
``["a", " ", "b", "\\n"]`` for ``print("a", "b")`` — and Code Mode renders
|
||||
``["a", " ", "b", "\\n"]`` for ``print("a", "b")`` — and PTC mode renders
|
||||
``logs`` with ``join('\\n')``, turning that into spurious blank lines. This
|
||||
stream instead buffers writes and pushes one LogBuffer entry per completed
|
||||
LINE (the text up to each ``\\n``, newline stripped), so the rendered join
|
||||
@@ -1137,7 +1137,7 @@ async def _run(channel: ProtocolChannel) -> None:
|
||||
error_class = error_classes.get(global_name)
|
||||
|
||||
def call_failure(message: str) -> BaseException:
|
||||
# The namespace's declared rejection contract (e.g. Code Mode's
|
||||
# The namespace's declared rejection contract (e.g. PTC mode's
|
||||
# ToolCallError with .toolName) when present; RuntimeError keeps
|
||||
# the pre-errorClass behavior for namespaces that declared none.
|
||||
if error_class is not None:
|
||||
@@ -2124,7 +2124,7 @@ def _make_cpu_enforcer() -> Any:
|
||||
:func:`_run`'s frame and reads its locals, so a program determined to
|
||||
tamper still can — consistent with this backend's documented posture, where
|
||||
the in-process interpreter is containment rather than a security boundary
|
||||
(§Trust posture in the Code Mode RFC). The bounds that model code cannot
|
||||
(§Trust posture in the PTC mode Agent Note). The bounds that model code cannot
|
||||
forge are outside the interpreter: the RLIMIT_CPU HARD limit at
|
||||
``cpuSeconds + 1``, whose SIGKILL is undeliverable to a handler and
|
||||
unraisable by a process that cannot raise its own hard limit, and the
|
||||
|
||||
@@ -2,8 +2,8 @@
|
||||
* CPython subprocess code runtime: a fresh `python3` process runs each model program under an
|
||||
* asyncio event loop with top-level ``await``. Binding calls travel on fd 3 as JSON-lines,
|
||||
* leaving stdout/stderr free for the program's own output. This is containment, not a security
|
||||
* boundary: model code has bash-equivalent trust, contained by an empty environment, RLIMIT_CPU
|
||||
* + RLIMIT_AS, wall-clock timeout, and SIGTERM→grace→SIGKILL on the process group.
|
||||
* boundary: model code has bash-equivalent trust, contained by a tempdir-only environment,
|
||||
* RLIMIT_CPU + RLIMIT_AS, wall-clock timeout, and SIGTERM→grace→SIGKILL on the process group.
|
||||
*
|
||||
* The package owns the versionless fd-3 wire protocol between the Node host and
|
||||
* the CPython subprocess. The protocol's host-side codec and hostile-frame
|
||||
@@ -12,7 +12,7 @@
|
||||
* @module @deepseek-ai/dsh-experimental-code-runtime-python
|
||||
*/
|
||||
|
||||
import { spawn, type ChildProcessWithoutNullStreams } from 'node:child_process'
|
||||
import { execFileSync, spawn, type ChildProcessWithoutNullStreams } from 'node:child_process'
|
||||
import { accessSync, copyFileSync, constants as fsConstants, mkdtempSync, readFileSync, rmSync, statSync } from 'node:fs'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { delimiter, dirname, isAbsolute, join, resolve } from 'node:path'
|
||||
@@ -84,8 +84,8 @@ export interface Config {
|
||||
/** SIGTERM→SIGKILL grace period on kill, matching bash-local's default. */
|
||||
graceMs?: number
|
||||
/**
|
||||
* Absolute path or basename of the CPython interpreter to spawn. Resolved
|
||||
* through `PATH` when a basename is given.
|
||||
* Absolute path, relative path, or basename of a CPython 3.10+ interpreter.
|
||||
* Resolved and validated once at plugin load; a basename searches `PATH`.
|
||||
*/
|
||||
pythonBin?: string
|
||||
}
|
||||
@@ -393,41 +393,31 @@ export function readProcessStart(pid: number): string | undefined {
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve `pythonBin` to an absolute path against the CURRENT process `PATH`,
|
||||
* BEFORE the child spawns with an empty environment. A basename (the default
|
||||
* `python3`) would otherwise fail: `env: {}` drops `PATH`, so Node's own lookup
|
||||
* falls back to the platform default (`/usr/bin:/bin`) and misses interpreters
|
||||
* Resolve `pythonBin` to one executable absolute path at plugin load. A basename
|
||||
* (the default `python3`) searches the current process `PATH`; the child receives
|
||||
* no `PATH`, so Node's own lookup would otherwise fall back to the platform
|
||||
* default (`/usr/bin:/bin`) and miss interpreters
|
||||
* that live only on the caller's `PATH` (Nix, pyenv, Homebrew, conda). An
|
||||
* absolute or explicitly relative path is validated directly: it must exist,
|
||||
* be executable, and be a regular file — a missing, non-executable, or
|
||||
* directory path is a self-contained configuration error that must fail at
|
||||
* load, not at the first run (the child spawns with an empty environment, so
|
||||
* execvp's platform default would otherwise silently mask the mistake). A
|
||||
* relative explicit path resolves against the host CWD, mirroring where
|
||||
* `spawn` would have looked for it. When no `PATH` entry holds an executable
|
||||
* match, `undefined` is returned and the LOAD check rejects the configuration:
|
||||
* falling back to the bare name would let spawn's `env: {}` execvp silently
|
||||
* start a system interpreter from the platform default PATH that the caller
|
||||
* never asked for.
|
||||
* @param bin - the configured interpreter (absolute or relative path, or bare command).
|
||||
* absolute path is verified in place, and an explicitly relative path is first
|
||||
* resolved against the load-time working directory. When no candidate is an
|
||||
* executable regular file, `undefined` is returned and the load check rejects
|
||||
* the configuration: falling back to the bare name would let spawn's scrubbed env
|
||||
* execvp silently start a system interpreter from the platform default PATH
|
||||
* that the caller never asked for.
|
||||
* @param bin - the configured interpreter (absolute path, relative path, or bare command).
|
||||
* @returns an absolute path when resolvable, else `undefined`.
|
||||
*/
|
||||
export function resolvePythonBin(bin: string): string | undefined {
|
||||
if (isAbsolute(bin) || bin.includes('/')) {
|
||||
// An explicit path is used as given (resolved against the host CWD when
|
||||
// relative), but only when it is a real executable regular file. The same
|
||||
// checks as the PATH branch below: `accessSync(X_OK)` admits directories,
|
||||
// so `isFile` narrows further, and a path that fails either is not a
|
||||
// usable interpreter.
|
||||
const candidate = resolve(bin)
|
||||
const executableFile = (candidate: string): string | undefined => {
|
||||
try {
|
||||
accessSync(candidate, fsConstants.X_OK)
|
||||
if (!statSync(candidate).isFile()) return undefined
|
||||
return candidate
|
||||
return statSync(candidate).isFile() ? candidate : undefined
|
||||
} catch {
|
||||
return undefined
|
||||
}
|
||||
}
|
||||
if (isAbsolute(bin)) return executableFile(bin)
|
||||
if (bin.includes('/')) return executableFile(resolve(bin))
|
||||
const path = process.env.PATH
|
||||
/* v8 ignore next -- PATH is set in every environment the runtime boots in; the guard is defensive. */
|
||||
if (path === undefined) return undefined
|
||||
@@ -438,21 +428,55 @@ export function resolvePythonBin(bin: string): string | undefined {
|
||||
// path — spawn() resolves a relative pythonBin against the host CWD, which
|
||||
// is outside the seam contract.
|
||||
if (dir === '' || !isAbsolute(dir)) continue
|
||||
const candidate = join(dir, bin)
|
||||
try {
|
||||
accessSync(candidate, fsConstants.X_OK)
|
||||
// A directory passes X_OK too, so require a regular file: a PATH entry
|
||||
// named like the interpreter (e.g. a `python3` directory) must not be
|
||||
// chosen over a later real interpreter.
|
||||
if (!statSync(candidate).isFile()) continue
|
||||
return candidate
|
||||
} catch {
|
||||
// Not executable here; try the next PATH entry.
|
||||
}
|
||||
const executable = executableFile(join(dir, bin))
|
||||
if (executable !== undefined) return executable
|
||||
}
|
||||
return undefined
|
||||
}
|
||||
|
||||
/** Lowest CPython version supported by the bootstrap and its traceback behavior. */
|
||||
const MIN_CPYTHON = { major: 3, minor: 10 } as const
|
||||
|
||||
/** Fixed load-time probe bound; a configured executable must not hang plugin activation. */
|
||||
const PYTHON_PROBE_TIMEOUT_MS = 5_000
|
||||
|
||||
/** The only host environment fact exposed to the child. */
|
||||
function pythonEnvironment(): NodeJS.ProcessEnv {
|
||||
return { TMPDIR: tmpdir() }
|
||||
}
|
||||
|
||||
/** Fail load unless `bin` is a responsive CPython 3.10+ interpreter. */
|
||||
function validatePythonBin(bin: string): void {
|
||||
let output: string
|
||||
try {
|
||||
output = execFileSync(bin, [
|
||||
'-I',
|
||||
'-c',
|
||||
'import sys; print(sys.implementation.name, sys.version_info.major, sys.version_info.minor, sys.version_info.micro)',
|
||||
], {
|
||||
encoding: 'utf8',
|
||||
env: pythonEnvironment(),
|
||||
timeout: PYTHON_PROBE_TIMEOUT_MS,
|
||||
maxBuffer: 1_024,
|
||||
}).trim()
|
||||
} catch (error: unknown) {
|
||||
throw new Error(`dsh-code-runtime-python: config.pythonBin ${JSON.stringify(bin)} failed the CPython version probe: ${messageOf(error)}`)
|
||||
}
|
||||
const match = /^(\S+) (\d+) (\d+) (\d+)$/.exec(output)
|
||||
if (match === null) {
|
||||
throw new Error(`dsh-code-runtime-python: config.pythonBin ${JSON.stringify(bin)} did not report a CPython version`)
|
||||
}
|
||||
const [, implementation, majorText, minorText, patchText] = match
|
||||
const major = Number(majorText)
|
||||
const minor = Number(minorText)
|
||||
if (implementation !== 'cpython') {
|
||||
throw new Error(`dsh-code-runtime-python: config.pythonBin ${JSON.stringify(bin)} must be CPython, got ${implementation}`)
|
||||
}
|
||||
if (major < MIN_CPYTHON.major || (major === MIN_CPYTHON.major && minor < MIN_CPYTHON.minor)) {
|
||||
throw new Error(`dsh-code-runtime-python: config.pythonBin ${JSON.stringify(bin)} must be CPython ${MIN_CPYTHON.major}.${MIN_CPYTHON.minor} or newer, got ${implementation} ${majorText}.${minorText}.${patchText}`)
|
||||
}
|
||||
}
|
||||
|
||||
/** The marker appended when a diagnostic message is byte-capped host-side. */
|
||||
const TRUNCATION_MARKER = '… [truncated]'
|
||||
|
||||
@@ -731,6 +755,7 @@ export class PythonCodeRuntime extends CodeRuntime {
|
||||
readonly isolation = 'process'
|
||||
|
||||
private readonly config: ResolvedConfig
|
||||
private readonly pythonBin: string
|
||||
private readonly live = new Set<LiveRun>()
|
||||
private disposed = false
|
||||
|
||||
@@ -785,26 +810,13 @@ export class PythonCodeRuntime extends CodeRuntime {
|
||||
// throws `ERR_INVALID_ARG_TYPE` — both from inside `run()`, so the method
|
||||
// REJECTS instead of resolving the `worker-exit` the seam promises for a
|
||||
// child that cannot start. A basename with no `PATH` match would silently
|
||||
// fall to execvp's platform default `PATH` under the empty spawn
|
||||
// fall to execvp's platform default `PATH` under the minimal spawn
|
||||
// environment (see the resolvePythonBin JSDoc), so it is rejected here
|
||||
// too. All three are self-contained configuration errors that fail at
|
||||
// load.
|
||||
if (this.config.pythonBin === '' || this.config.pythonBin.includes('\0')) {
|
||||
throw new Error(`dsh-code-runtime-python: config.pythonBin must be a non-empty path without NUL bytes, got ${JSON.stringify(this.config.pythonBin)}`)
|
||||
}
|
||||
// An explicit path that is not an executable regular file must fail at load
|
||||
// like any other self-contained configuration error (the empty/NUL cases
|
||||
// above); a basename that is not on PATH must fail at load, not silently
|
||||
// fall to execvp's platform default PATH (spawn runs with an EMPTY
|
||||
// environment, so execvp would resolve /usr/bin:/bin and could start a
|
||||
// system interpreter the caller never asked for). resolvePythonBin applies
|
||||
// the executable-regular-file check to both forms and returns undefined for
|
||||
// either failure; the message distinguishes the two so the fix is obvious.
|
||||
const resolvedBin = resolvePythonBin(this.config.pythonBin)
|
||||
if (resolvedBin === undefined) {
|
||||
const explicit = isAbsolute(this.config.pythonBin) || this.config.pythonBin.includes('/')
|
||||
throw new Error(`dsh-code-runtime-python: config.pythonBin ${JSON.stringify(this.config.pythonBin)} ${explicit ? 'is not an executable regular file' : 'does not resolve on PATH'}`)
|
||||
}
|
||||
// `maxWallMs` and `graceMs` are armed with setTimeout, which clamps any
|
||||
// delay past MAX_TIMER_DELAY_MS to 1 ms without a word — turning a
|
||||
// generous ceiling into an instant timeout and a generous grace period into
|
||||
@@ -905,6 +917,19 @@ export class PythonCodeRuntime extends CodeRuntime {
|
||||
throw new Error(`dsh-code-runtime-python: config.${key} times the ${OUTPUT_BUDGET_WORST_CASE_ADDRESS_SPACE_MULTIPLE}x worst-case Unicode expansion must fit within the ${budgetableBytes} bytes left after the ${INTERPRETER_BASELINE_BYTES}-byte interpreter baseline within the ${addressSpaceBytes}-byte addressSpaceMb, so a near-budget output truncates rather than breaching RLIMIT_AS as worker-exit; got ${String(this.config[key])} against a limit of ${admissibleBudget}`)
|
||||
}
|
||||
}
|
||||
// Resolve and validate the executable ONCE, after the pure config checks.
|
||||
// Re-resolving a basename in each run would let a later PATH change silently
|
||||
// switch interpreters, while an unchecked explicit path would turn
|
||||
// self-contained misconfiguration into a late worker-exit. A missing or
|
||||
// unsupported interpreter is a load failure. Later filesystem mutation is
|
||||
// outside config validation; a missing executable settles as worker-exit.
|
||||
const pythonBin = resolvePythonBin(this.config.pythonBin)
|
||||
if (pythonBin === undefined) {
|
||||
const explicit = isAbsolute(this.config.pythonBin) || this.config.pythonBin.includes('/')
|
||||
throw new Error(`dsh-code-runtime-python: config.pythonBin ${JSON.stringify(this.config.pythonBin)} ${explicit ? 'is not an executable regular file' : 'does not resolve on PATH'}`)
|
||||
}
|
||||
validatePythonBin(pythonBin)
|
||||
this.pythonBin = pythonBin
|
||||
ctx.effect(() => () => this.teardown(), 'python code-runtime teardown')
|
||||
}
|
||||
|
||||
@@ -1061,8 +1086,8 @@ export class PythonCodeRuntime extends CodeRuntime {
|
||||
// This run's own staging directory, removed at settlement.
|
||||
const bootstrapDir = dirname(bootstrapPath)
|
||||
// Explicit pipe count of 4 puts the framed-JSON channel at fd 3 in the child.
|
||||
// Resolve the interpreter against the current PATH first: the child's empty
|
||||
// env would otherwise strip PATH and miss a basename python3 (see resolvePythonBin).
|
||||
// The constructor resolved and validated the interpreter once; runs keep that
|
||||
// exact path even if the host later changes PATH.
|
||||
// `spawn` can throw SYNCHRONOUSLY — a descriptor-exhausted host (EMFILE) or a
|
||||
// libuv-level failure surfaces here, before the Promise executor and its
|
||||
// settlement path exist. Left uncaught it would REJECT run() (the seam
|
||||
@@ -1080,15 +1105,11 @@ export class PythonCodeRuntime extends CodeRuntime {
|
||||
// right after the done frame, before any finalization-time flush could
|
||||
// run. The `_LogStream` replacement of `sys.stdout`/`sys.stderr` is
|
||||
// unaffected (it is a Python object, not the C-level stdio buffer).
|
||||
// Load validated that the configured interpreter resolves to an
|
||||
// executable regular file (basename through PATH, explicit path
|
||||
// directly). The type assertion is the load-time contract (see the
|
||||
// pythonBin load checks); a PATH change between load and run would make
|
||||
// this undefined and spawn throws synchronously, which the surrounding
|
||||
// try settles as worker-exit like any other spawn failure.
|
||||
const resolvedPythonBin = resolvePythonBin(this.config.pythonBin) as string
|
||||
child = spawn(resolvedPythonBin, ['-u', '-I', bootstrapPath], {
|
||||
env: {},
|
||||
child = spawn(this.pythonBin, ['-u', '-I', bootstrapPath], {
|
||||
// Preserve only the platform temp directory. macOS system Python emits a
|
||||
// startup warning when TMPDIR is absent; ambient credentials, PATH, HOME,
|
||||
// and other host state remain unavailable to model code.
|
||||
env: pythonEnvironment(),
|
||||
detached: true, // Own process group — kill(-pid, sig) reaches subprocesses the model program spawns.
|
||||
stdio: ['pipe', 'pipe', 'pipe', 'pipe'],
|
||||
})
|
||||
@@ -1107,7 +1128,6 @@ export class PythonCodeRuntime extends CodeRuntime {
|
||||
// inheriting fd 0 would keep the host process from exiting even after the
|
||||
// closeDeadline forced settlement. The child (and any descendant) reads
|
||||
// EOF on fd 0 instead, and no host handle survives.
|
||||
// oxlint-disable-next-line typescript/no-unnecessary-condition -- the boot-write-failure fake child has no stdin.
|
||||
child.stdin?.destroy()
|
||||
} catch (error: unknown) {
|
||||
try {
|
||||
@@ -1229,7 +1249,7 @@ export class PythonCodeRuntime extends CodeRuntime {
|
||||
// (native prints, C-extension writes) still counts against the ledger.
|
||||
//
|
||||
// Output is admitted per LINE, not per transport chunk. `logs` entries
|
||||
// are joined with `\n` downstream (Code Mode), so each entry must be one
|
||||
// are joined with `\n` downstream (PTC mode), so each entry must be one
|
||||
// line: pushing a raw `data` chunk would turn every arbitrary pipe-read
|
||||
// boundary into a model-visible newline, so a single 200 KiB native write
|
||||
// split across pipe reads would read back with spurious line breaks. The
|
||||
@@ -1289,7 +1309,6 @@ export class PythonCodeRuntime extends CodeRuntime {
|
||||
// A line admitted inside the loop may have exhausted the ledger and
|
||||
// cleared this pipe (see clearStray); the re-retain below must not
|
||||
// resurrect the doomed residual.
|
||||
// oxlint-disable-next-line typescript/no-unnecessary-condition -- admit() (a closure) sets it.
|
||||
if (logsTruncated) return
|
||||
stray.chunks = detachResidual(buffered)
|
||||
stray.utf8 = { expected: 0, width: 0, lowerFirst: 0, upperFirst: 0 }
|
||||
@@ -1784,7 +1803,6 @@ export class PythonCodeRuntime extends CodeRuntime {
|
||||
// after `maxWallMs`, an abort, or dispose already settled the run
|
||||
// would spend host heap on a frame that is then discarded, and
|
||||
// binding resolution carries no seam-level byte cap to bound it.
|
||||
// oxlint-disable-next-line typescript/no-unnecessary-condition -- the run can settle while this binding is awaited.
|
||||
if (settled) return
|
||||
// The seam requires a lossy resolution to REJECT descriptively,
|
||||
// not silently coerce: a raw JSON.stringify would turn NaN/
|
||||
@@ -1804,13 +1822,8 @@ export class PythonCodeRuntime extends CodeRuntime {
|
||||
// before `sendReply` peeks at `settled`. Dropping the framed
|
||||
// reply early spares the host heap and time for a run whose
|
||||
// outcome is already fixed.
|
||||
// (oxlint block-disable so both `v8 ignore next` and the rule
|
||||
// suppression land on the `if`: `settled` flips true mid-wait,
|
||||
// invisible to the type-aware lint, which narrows it to false.)
|
||||
/* oxlint-disable typescript/no-unnecessary-condition */
|
||||
/* v8 ignore next -- a rejection arriving after settlement is not schedulable from a test. */
|
||||
if (settled) return
|
||||
/* oxlint-enable typescript/no-unnecessary-condition */
|
||||
sendReply({ type: 'reply', id: message.id, ok: false, message: messageOf(error) })
|
||||
} finally {
|
||||
// Release the in-flight slot on every exit — reply written,
|
||||
|
||||
@@ -1,19 +1,16 @@
|
||||
import { existsSync, realpathSync, rmSync, statSync, writeFileSync } from 'node:fs'
|
||||
import { mkdtemp, writeFile } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { basename, dirname, join } from 'node:path'
|
||||
import { basename, dirname, join, relative } from 'node:path'
|
||||
import { describe, expect, it, vi } from 'vitest'
|
||||
import { Context } from '@deepseek-ai/cordis'
|
||||
import { PythonCodeRuntime, readProcessStart, resolvePythonBin } from '../src/index.ts'
|
||||
import { logTruncationMarker } from '../src/protocol.ts'
|
||||
import type { Config } from '../src/index.ts'
|
||||
|
||||
// Absolute interpreter path for the shell wrappers: the runtime spawns the
|
||||
// child with env:{} (an empty environment by design), so a bare 'python3' in a
|
||||
// wrapper resolves against /bin/sh's compiled-in default PATH, which misses
|
||||
// interpreters only reachable through the caller's PATH (Nix, pyenv). Baking
|
||||
// the resolved absolute path mirrors what resolvePythonBin does for the product
|
||||
// spawn.
|
||||
// Absolute supported interpreter path for shell wrappers. The runtime gives a
|
||||
// child only TMPDIR, so a bare `python3` inside a wrapper would resolve against
|
||||
// /bin/sh's default PATH rather than the caller's selected interpreter.
|
||||
const PYABS = resolvePythonBin('python3') ?? 'python3'
|
||||
import type { CodeBindingFunction, CodeJsonValue, CodeRunResult } from '@deepseek-ai/dsh-code-runtime'
|
||||
|
||||
@@ -201,6 +198,44 @@ describe('PythonCodeRuntime — seam descriptors and misuse', () => {
|
||||
}
|
||||
})
|
||||
|
||||
it('rejects a non-CPython, outdated, or probe-failing interpreter at load', async () => {
|
||||
const nonPython = new Context()
|
||||
await expect(nonPython.plugin(PythonCodeRuntime, { pythonBin: '/bin/echo' }))
|
||||
.rejects.toThrow(/did not report a CPython version/)
|
||||
|
||||
const dir = await mkdtemp(join(tmpdir(), 'dsh-python-probe-'))
|
||||
const oldMajor = join(dir, 'python-old-major')
|
||||
const old = join(dir, 'python-old')
|
||||
const future = join(dir, 'python-future')
|
||||
const pypy = join(dir, 'pypy')
|
||||
const failed = join(dir, 'python-failed')
|
||||
await writeFile(oldMajor, '#!/bin/sh\nprintf \'cpython 2 99 0\\n\'\n', { mode: 0o755 })
|
||||
await writeFile(old, '#!/bin/sh\nprintf \'cpython 3 9 6\\n\'\n', { mode: 0o755 })
|
||||
await writeFile(future, '#!/bin/sh\nprintf \'cpython 4 0 0\\n\'\n', { mode: 0o755 })
|
||||
await writeFile(pypy, '#!/bin/sh\nprintf \'pypy 3 10 0\\n\'\n', { mode: 0o755 })
|
||||
await writeFile(failed, '#!/bin/sh\nexit 7\n', { mode: 0o755 })
|
||||
try {
|
||||
expect(resolvePythonBin(relative(process.cwd(), old))).toBe(old)
|
||||
const obsolete = new Context()
|
||||
await expect(obsolete.plugin(PythonCodeRuntime, { pythonBin: oldMajor }))
|
||||
.rejects.toThrow(/must be CPython 3\.10 or newer, got cpython 2\.99\.0/)
|
||||
const outdated = new Context()
|
||||
await expect(outdated.plugin(PythonCodeRuntime, { pythonBin: old }))
|
||||
.rejects.toThrow(/must be CPython 3\.10 or newer, got cpython 3\.9\.6/)
|
||||
const forwardCompatible = new Context()
|
||||
const fiber = await forwardCompatible.plugin(PythonCodeRuntime, { pythonBin: future })
|
||||
await fiber.dispose()
|
||||
const alternative = new Context()
|
||||
await expect(alternative.plugin(PythonCodeRuntime, { pythonBin: pypy }))
|
||||
.rejects.toThrow(/must be CPython, got pypy/)
|
||||
const probeFailure = new Context()
|
||||
await expect(probeFailure.plugin(PythonCodeRuntime, { pythonBin: failed }))
|
||||
.rejects.toThrow(/failed the CPython version probe/)
|
||||
} finally {
|
||||
rmSync(dir, { recursive: true, force: true })
|
||||
}
|
||||
})
|
||||
|
||||
it('keeps an explicit executable pythonBin working through load and run', async () => {
|
||||
// The same validation that rejects bad explicit paths must admit a good
|
||||
// one: an absolute path to the real interpreter (or a wrapper around it)
|
||||
@@ -274,6 +309,32 @@ describe('PythonCodeRuntime — seam descriptors and misuse', () => {
|
||||
await fiber.dispose()
|
||||
})
|
||||
|
||||
it('resolves pythonBin once so a later PATH change cannot switch interpreters', async () => {
|
||||
const firstDir = await mkdtemp(join(tmpdir(), 'dsh-python-first-'))
|
||||
const secondDir = await mkdtemp(join(tmpdir(), 'dsh-python-second-'))
|
||||
const wrapper = (marker: string): string => `#!/bin/sh\nDSH_TEST_PYTHON=${marker}\nexport DSH_TEST_PYTHON\nexec "${PYABS}" "$@"\n`
|
||||
await writeFile(join(firstDir, 'python3'), wrapper('first'), { mode: 0o755 })
|
||||
await writeFile(join(secondDir, 'python3'), wrapper('second'), { mode: 0o755 })
|
||||
vi.stubEnv('PATH', firstDir)
|
||||
let fiber: Awaited<ReturnType<typeof setup>>['fiber'] | undefined
|
||||
try {
|
||||
const mounted = await setup({ pythonBin: 'python3' })
|
||||
fiber = mounted.fiber
|
||||
vi.stubEnv('PATH', secondDir)
|
||||
const result = await mounted.runtime.run({
|
||||
program: 'import os\nreturn os.environ.get("DSH_TEST_PYTHON")',
|
||||
bindings: [],
|
||||
})
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.value).toBe('first')
|
||||
} finally {
|
||||
await fiber?.dispose()
|
||||
vi.unstubAllEnvs()
|
||||
rmSync(firstDir, { recursive: true, force: true })
|
||||
rmSync(secondDir, { recursive: true, force: true })
|
||||
}
|
||||
})
|
||||
|
||||
it('skips relative PATH entries when resolving a basename pythonBin', async () => {
|
||||
// resolvePythonBin must return an absolute path: a RELATIVE PATH entry
|
||||
// ('.' here) would otherwise resolve the basename against the host CWD.
|
||||
@@ -451,7 +512,8 @@ describe('PythonCodeRuntime — seam descriptors and misuse', () => {
|
||||
const entry = await entryOf()
|
||||
expect(entry.endsWith('/bootstrap.py')).toBe(true)
|
||||
const dir = dirname(entry)
|
||||
expect(dir.startsWith(realpathSync(tmpdir()))).toBe(true)
|
||||
expect(realpathSync(dirname(dir))).toBe(realpathSync(tmpdir()))
|
||||
expect(basename(dir)).toMatch(/^dsh-code-runtime-python-/)
|
||||
expect(dir).not.toContain('/packages/')
|
||||
// Staging is per RUN and removed at settlement, so by the time `run()`
|
||||
// resolved the directory is already gone — nothing survives to be rewritten
|
||||
@@ -621,7 +683,7 @@ describe('PythonCodeRuntime — process identity', () => {
|
||||
})
|
||||
|
||||
describe('PythonCodeRuntime — inherited resource limits', () => {
|
||||
it('runs under an inherited hard limit tighter than addressSpaceMb', async () => {
|
||||
it.skipIf(process.platform === 'darwin')('runs under an inherited hard limit tighter than addressSpaceMb', async () => {
|
||||
// An unprivileged process may lower a hard rlimit but never raise it. Under
|
||||
// a harness started with `ulimit -v` below `addressSpaceBytes`, requesting
|
||||
// the configured cap made `setrlimit` raise `ValueError` and every run
|
||||
@@ -684,13 +746,21 @@ describe('PythonCodeRuntime — inherited resource limits', () => {
|
||||
const result = await runtime.run({
|
||||
// `getrlimit` returns a tuple, which the lossless-JSON completion check
|
||||
// rejects; the pair is listed explicitly rather than converted.
|
||||
program: 'import resource\ncpu = resource.getrlimit(resource.RLIMIT_CPU)\nreturn [cpu[0], cpu[1], resource.getrlimit(resource.RLIMIT_AS)[1]]',
|
||||
program: [
|
||||
'import resource, sys',
|
||||
'cpu = resource.getrlimit(resource.RLIMIT_CPU)',
|
||||
'address_space = None if sys.platform == "darwin" else resource.getrlimit(resource.RLIMIT_AS)[1]',
|
||||
'return {"cpu": [cpu[0], cpu[1]], "addressSpace": address_space}',
|
||||
].join('\n'),
|
||||
bindings: [],
|
||||
})
|
||||
expect(result.error).toBeUndefined()
|
||||
// Soft at cpuSeconds, hard at +1 (the SIGKILL backstop), address space at
|
||||
// the configured megabytes — exactly what the unclamped path applied.
|
||||
expect(result.value).toEqual([42, 43, 400 * 1024 * 1024])
|
||||
// Darwin deliberately skips RLIMIT_AS; every other Unix host applies the
|
||||
// configured bytes alongside the CPU soft/hard pair.
|
||||
expect(result.value).toEqual({
|
||||
cpu: [42, 43],
|
||||
addressSpace: process.platform === 'darwin' ? null : 400 * 1024 * 1024,
|
||||
})
|
||||
}, 15_000)
|
||||
|
||||
it('preserves an inherited soft limit stricter than the configured cap', async () => {
|
||||
@@ -875,6 +945,25 @@ describe('PythonCodeRuntime — programs and bindings', () => {
|
||||
// the 5s default alone; later tests reuse the warm page cache.
|
||||
}, 15_000)
|
||||
|
||||
it('exposes only the platform temp directory from the host environment', async () => {
|
||||
const { runtime } = await setup()
|
||||
const result = await runtime.run({
|
||||
program: [
|
||||
'import os',
|
||||
'return {',
|
||||
' "tmpdir": os.environ.get("TMPDIR"),',
|
||||
' "path": os.environ.get("PATH"),',
|
||||
' "home": os.environ.get("HOME"),',
|
||||
' "token": os.environ.get("DEEPSEEK_API_KEY"),',
|
||||
'}',
|
||||
].join('\n'),
|
||||
bindings: [],
|
||||
})
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.value).toEqual({ tmpdir: tmpdir(), path: null, home: null, token: null })
|
||||
expect(result.logs).toEqual([])
|
||||
})
|
||||
|
||||
it('bridges binding calls both ways and rejects the program-side call on a host rejection', async () => {
|
||||
const { runtime } = await setup()
|
||||
const calls: unknown[] = []
|
||||
@@ -1139,7 +1228,7 @@ describe('PythonCodeRuntime — programs and bindings', () => {
|
||||
|
||||
it('coalesces print arguments into one log line, not per-write fragments', async () => {
|
||||
// print("a","b") calls write() per arg/sep/newline; the stream must emit
|
||||
// one logical line "a b" so Code Mode's join(newline) does not insert
|
||||
// one logical line "a b" so PTC mode's join(newline) does not insert
|
||||
// spurious blank lines. Two prints → exactly two entries, no empties.
|
||||
const { runtime } = await setup()
|
||||
const result = await runtime.run({
|
||||
@@ -1189,6 +1278,24 @@ describe('PythonCodeRuntime — programs and bindings', () => {
|
||||
expect(result.logs).toEqual(['one', 'two', 'three'])
|
||||
})
|
||||
|
||||
it('preserves each native stream order while allowing backend-dependent interleaving', async () => {
|
||||
const { runtime } = await setup()
|
||||
const result = await runtime.run({
|
||||
program: [
|
||||
'import os',
|
||||
'os.write(1, b"stdout-one\\n")',
|
||||
'os.write(2, b"stderr-one\\n")',
|
||||
'os.write(1, b"stdout-two\\n")',
|
||||
'os.write(2, b"stderr-two\\n")',
|
||||
'return None',
|
||||
].join('\n'),
|
||||
bindings: [],
|
||||
})
|
||||
expect(result.error).toBeUndefined()
|
||||
expect(result.logs.indexOf('stdout-one')).toBeLessThan(result.logs.indexOf('stdout-two'))
|
||||
expect(result.logs.indexOf('stderr-one')).toBeLessThan(result.logs.indexOf('stderr-two'))
|
||||
})
|
||||
|
||||
it('bounds a newline-free native flood by the ledger instead of buffering it whole', async () => {
|
||||
// A newline-free write far larger than maxLogBytes must not accumulate in
|
||||
// the host-side residual: when the pending residual would cross the budget
|
||||
@@ -2439,7 +2546,7 @@ describe('PythonCodeRuntime — programs and bindings', () => {
|
||||
})
|
||||
|
||||
it('raises the declared errorClass with the member name on rejection', async () => {
|
||||
// Code Mode declares { name: ToolCallError, memberNameProperty: toolName };
|
||||
// PTC mode declares { name: ToolCallError, memberNameProperty: toolName };
|
||||
// a host rejection must surface as that class, carrying the failed tool.
|
||||
const { runtime } = await setup()
|
||||
const result = await runtime.run({
|
||||
@@ -2618,7 +2725,7 @@ describe('PythonCodeRuntime — programs and bindings', () => {
|
||||
// asked for.
|
||||
const ctx = new Context()
|
||||
await expect(ctx.plugin(PythonCodeRuntime, { pythonBin: 'definitely-no-such-python-xyz' }))
|
||||
.rejects.toThrow(/does not resolve on PATH/)
|
||||
.rejects.toThrow(/does not resolve to an executable file/)
|
||||
})
|
||||
|
||||
it('rejects a memberNameProperty naming a constrained BaseException attribute', async () => {
|
||||
@@ -2969,28 +3076,19 @@ describe('PythonCodeRuntime — budgets, termination, disposal', () => {
|
||||
expect(['abort', 'worker-exit']).toContain(result.error?.kind)
|
||||
}, 5000)
|
||||
|
||||
it('reports a spawn failure via a bogus python binary as worker-exit', async () => {
|
||||
// An explicit path that does not exist at LOAD is a configuration error and
|
||||
// is rejected by the constructor (see the seam-misuse block). A path that
|
||||
// is valid at load but gone by run time is a SUBSTRATE failure and must
|
||||
// resolve as worker-exit: stage a real executable wrapper, load the runtime
|
||||
// against it, then delete it before run() — the spawn then fails exactly
|
||||
// like a child that cannot start.
|
||||
const nodePath = await import('node:path')
|
||||
const { mkdtempSync, rmSync, writeFileSync, chmodSync } = await import('node:fs')
|
||||
const dir = mkdtempSync(nodePath.join(tmpdir(), 'dsh-spawn-fail-'))
|
||||
const wrapper = nodePath.join(dir, 'python-wrapper')
|
||||
const pyAbs = resolvePythonBin('python3') ?? 'python3'
|
||||
writeFileSync(wrapper, `#!/bin/sh\nexec ${pyAbs} "$@"\n`, { mode: 0o755 })
|
||||
chmodSync(wrapper, 0o755)
|
||||
const { runtime } = await setup({ pythonBin: wrapper, maxWallMs: 3000 })
|
||||
rmSync(wrapper)
|
||||
rmSync(dir, { recursive: true, force: true })
|
||||
const result = await runtime.run({
|
||||
program: 'return 1',
|
||||
bindings: [],
|
||||
})
|
||||
expect(result.error?.kind).toBe('worker-exit')
|
||||
it('reports an interpreter removed after load as worker-exit', async () => {
|
||||
const dir = await mkdtemp(join(tmpdir(), 'dsh-python-removed-'))
|
||||
const pythonBin = join(dir, 'python3')
|
||||
await writeFile(pythonBin, `#!/bin/sh\nexec "${PYABS}" "$@"\n`, { mode: 0o755 })
|
||||
const { runtime, fiber } = await setup({ pythonBin, maxWallMs: 3000 })
|
||||
rmSync(pythonBin)
|
||||
try {
|
||||
const result = await runtime.run({ program: 'return 1', bindings: [] })
|
||||
expect(result.error?.kind).toBe('worker-exit')
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
rmSync(dir, { recursive: true, force: true })
|
||||
}
|
||||
}, 8000)
|
||||
|
||||
it('applies the strictest of the configured and inherited resource limits', async () => {
|
||||
|
||||
Reference in New Issue
Block a user