fix(code-runtime-python): validate explicit pythonBin at load, snapshot bindings, and settle the reply drain

Review findings on the CPython backend: an explicit pythonBin path bypassed
the load-time checks (missing/non-executable/directory paths surfaced only
as a run-time worker-exit); a throwing binding member accessor escaped the
fd-3 data callback and terminated the host; the reply drain waited on
'drain' alone, so a pipe destroyed under the wait hung forever; and two
staging-leak assertions diffed a global tmpdir that parallel workers can
perturb.

resolvePythonBin now applies the same accessSync(X_OK) + isFile check to
explicit paths (resolved against the host CWD), and the load error message
distinguishes 'is not an executable regular file' from 'does not resolve on
PATH'. validateBindings snapshots callables into a plain record during run()'s
synchronous validation, turning an accessor throw into the seam-misuse
rejection and fixing the key set the boot frame and dispatch share. The reply
drain waits on drain/close/error together and short-circuits on
proto.destroyed. The staging-leak assertions check the exact paths this test
file staged (recorded by the mocked mkdtempSync) instead of a tmpdir diff.

docs(code-runtime-python): add the alternatives section to the hardening note

docs(config-catalog): refresh the code-runtime-python Config source line

test(code-runtime-python): cover the async spawn-error worker-exit path
This commit is contained in:
Chinesezjc
2026-08-31 15:25:26 +08:00
committed by Tianyi Cui
parent 2c8d545524
commit 2df28fd249
12 changed files with 486 additions and 48 deletions
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/experimental/code-runtime-python/README.md
README.md: 1fe1996719dd4d9aac413fe8bbca8d730db7cdd7
README.zh.md: ae4c89a347a750fc31d503e1c769586fdabbe0f9
README.md: 11f845626fe7aa06e7725c70dcab764482eb9552
README.zh.md: 13f60ebc191fb5ed82566ec145f526c34dfe0714
@@ -25,11 +25,11 @@ English | [中文](README.zh.md)
<a id="use-this-package"></a>
## Use this package
Choose this package to run Python model code through the code-runtime seam: register `PythonCodeRuntime` with `dsh-tools` and `run()` executes each program in a fresh `python3 -I` subprocess, resolving with `result.value` on success and `result.error` on failure (the orthogonal `CodeRunFailure.kind` taxonomy classifies parse failures, thrown exceptions, invalid completions, output overflows, budget expiry, aborts, and substrate death). It rejects only for seam misuse — a malformed binding namespace, or a call after disposal. Configuration is rejected at load: a non-Unix platform, a non-positive or non-integer budget, a `maxLogBytes` below the truncation-marker floor (64), a timer value `setTimeout` would clamp, a budget larger than one fd-3 frame can carry, and an `addressSpaceMb`/output-budget pair whose worst-case peak would breach `RLIMIT_AS`.
Choose this package to run Python model code through the code-runtime seam: register `PythonCodeRuntime` with `dsh-tools` and `run()` executes each program in a fresh `python3 -I` subprocess, resolving with `result.value` on success and `result.error` on failure (the orthogonal `CodeRunFailure.kind` taxonomy classifies parse failures, thrown exceptions, invalid completions, output overflows, budget expiry, aborts, and substrate death). It rejects only for seam misuse — a malformed binding namespace, or a call after disposal. Configuration is rejected at load: a non-Unix platform, a non-positive or non-integer budget, a `maxLogBytes` below the truncation-marker floor (64), a timer value `setTimeout` would clamp, a budget larger than one fd-3 frame can carry, an `addressSpaceMb`/output-budget pair whose worst-case peak would breach `RLIMIT_AS`, and a `pythonBin` that is not an executable regular file — an explicit path (absolute or containing `/`) is judged directly, a bare name is judged against `PATH`.
### What you get
The package's default export is the `PythonCodeRuntime` plugin. Its public surface also re-exports the host-side protocol vocabulary: `validateChildFrame` (rebuilds every inbound frame), the lossless-JSON codec and meters (`encodeJsonPlain`, `checkDoneValue`, `hasUnsafeIntegerToken`, `hasNonLosslessNumber`), `logTruncationMarker` (the shared truncation-marker text), plus `resolvePythonBin` (interpreter lookup against the current `PATH`), `readProcessStart` (process-start statistics for tests), and `detachResidual` (a test seam for the settled run's resource cleanup). Every cap is a validated `Config` field with a default: `cpuSeconds` (60), `maxWallMs` (600000), `addressSpaceMb` (512, not applied on Darwin), `maxLogBytes` (65536), `maxValueBytes` (32768), `graceMs` (3000), and `pythonBin` (`python3`, resolved against `PATH` before the child spawns with an empty environment; a basename with no `PATH` match is rejected at load rather than silently falling to the platform default `PATH`).
The package's default export is the `PythonCodeRuntime` plugin. Its public surface also re-exports the host-side protocol vocabulary: `validateChildFrame` (rebuilds every inbound frame), the lossless-JSON codec and meters (`encodeJsonPlain`, `checkDoneValue`, `hasUnsafeIntegerToken`, `hasNonLosslessNumber`), `logTruncationMarker` (the shared truncation-marker text), plus `resolvePythonBin` (interpreter lookup against the current `PATH`), `readProcessStart` (process-start statistics for tests), and `detachResidual` (a test seam for the settled run's resource cleanup). Every cap is a validated `Config` field with a default: `cpuSeconds` (60), `maxWallMs` (600000), `addressSpaceMb` (512, not applied on Darwin), `maxLogBytes` (65536), `maxValueBytes` (32768), `graceMs` (3000), and `pythonBin` (`python3`, resolved before the child spawns with an empty environment: an explicit path must be an executable regular file, a bare name must resolve on `PATH`; either failure is rejected at load, distinguishing 'is not an executable regular file' from 'does not resolve on PATH' instead of silently falling to the platform default `PATH`).
### The wire
@@ -25,11 +25,11 @@ kind: "package-reference"
<a id="use-this-package"></a>
## 使用本包
在需要通过 code-runtime seam 运行 Python 模型代码时选择本包:向 `dsh-tools` 注册 `PythonCodeRuntime``run()` 就在全新的 `python3 -I` 子进程中执行每个程序,成功时以 `result.value` resolve、失败时以 `result.error` resolve(正交的 `CodeRunFailure.kind` 分类涵盖解析失败、抛出异常、无效完成值、输出溢出、预算到期、中止与执行基底终止);只有 seam 误用才 reject——绑定命名空间畸形,或已释放后仍调用。配置在加载期被拒绝:非 Unix 平台、非正或非整数的预算、低于截断标记下限(64)的 `maxLogBytes``setTimeout` 会收敛的定时器值、超过单个 fd-3 帧可承载的预算,以及最坏峰值会突破 `RLIMIT_AS``addressSpaceMb`/输出预算组合。
在需要通过 code-runtime seam 运行 Python 模型代码时选择本包:向 `dsh-tools` 注册 `PythonCodeRuntime``run()` 就在全新的 `python3 -I` 子进程中执行每个程序,成功时以 `result.value` resolve、失败时以 `result.error` resolve(正交的 `CodeRunFailure.kind` 分类涵盖解析失败、抛出异常、无效完成值、输出溢出、预算到期、中止与执行基底终止);只有 seam 误用才 reject——绑定命名空间畸形,或已释放后仍调用。配置在加载期被拒绝:非 Unix 平台、非正或非整数的预算、低于截断标记下限(64)的 `maxLogBytes``setTimeout` 会收敛的定时器值、超过单个 fd-3 帧可承载的预算最坏峰值会突破 `RLIMIT_AS``addressSpaceMb`/输出预算组合,以及不是可执行普通文件的 `pythonBin`——显式路径(绝对或含 `/`)直接判定,裸名对照 `PATH` 判定
### 你得到什么
包的默认导出是 `PythonCodeRuntime` 插件。其公开面还重新导出宿主侧协议词汇:`validateChildFrame`(重建每条入站帧)、无损 JSON codec 与计量器(`encodeJsonPlain``checkDoneValue``hasUnsafeIntegerToken``hasNonLosslessNumber`)、`logTruncationMarker`(共享截断标记文本),以及 `resolvePythonBin`(对照当前 `PATH` 的解释器查找)、`readProcessStart`(供测试用的进程启动统计)和 `detachResidual`(已结算运行的资源清理测试 seam)。每个上限都是带默认值并经校验的 `Config` 字段:`cpuSeconds`60)、`maxWallMs`600000)、`addressSpaceMb`512Darwin 上不生效)、`maxLogBytes`65536)、`maxValueBytes`32768)、`graceMs`3000)与 `pythonBin``python3`,在子进程以空环境启动前对照 `PATH` 解析;在 `PATH` 上无命中的裸名会在加载期被拒绝,而不是静默回退到平台默认 `PATH`)。
包的默认导出是 `PythonCodeRuntime` 插件。其公开面还重新导出宿主侧协议词汇:`validateChildFrame`(重建每条入站帧)、无损 JSON codec 与计量器(`encodeJsonPlain``checkDoneValue``hasUnsafeIntegerToken``hasNonLosslessNumber`)、`logTruncationMarker`(共享截断标记文本),以及 `resolvePythonBin`(对照当前 `PATH` 的解释器查找)、`readProcessStart`(供测试用的进程启动统计)和 `detachResidual`(已结算运行的资源清理测试 seam)。每个上限都是带默认值并经校验的 `Config` 字段:`cpuSeconds`60)、`maxWallMs`600000)、`addressSpaceMb`512Darwin 上不生效)、`maxLogBytes`65536)、`maxValueBytes`32768)、`graceMs`3000)与 `pythonBin``python3`,在子进程以空环境启动前解析:显式路径必须是可执行普通文件,裸名必须在 `PATH` 上可解析;任一失败都在加载期被拒绝,区分『is not an executable regular file』与『does not resolve on PATH』,而不是静默回退到平台默认 `PATH`)。
### wire
@@ -13,10 +13,9 @@
*/
import { spawn, type ChildProcessWithoutNullStreams } from 'node:child_process'
import { once } from 'node:events'
import { accessSync, copyFileSync, constants as fsConstants, mkdtempSync, readFileSync, rmSync, statSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { delimiter, dirname, isAbsolute, join } from 'node:path'
import { delimiter, dirname, isAbsolute, join, resolve } from 'node:path'
import { fileURLToPath } from 'node:url'
import type { Duplex } from 'node:stream'
import { Context } from 'cordis'
@@ -386,16 +385,36 @@ export function readProcessStart(pid: number): string | undefined {
* `python3`) would otherwise fail: `env: {}` drops `PATH`, so Node's own lookup
* falls back to the platform default (`/usr/bin:/bin`) and misses interpreters
* that live only on the caller's `PATH` (Nix, pyenv, Homebrew, conda). An
* absolute or explicitly relative path is used verbatim. When no `PATH` entry
* holds an executable match, `undefined` is returned and the LOAD check rejects
* the configuration: falling back to the bare name would let spawn's `env: {}`
* execvp silently start a system interpreter from the platform default PATH
* that the caller never asked for.
* @param bin - the configured interpreter (absolute path or bare command).
* absolute or explicitly relative path is validated directly: it must exist,
* be executable, and be a regular file — a missing, non-executable, or
* directory path is a self-contained configuration error that must fail at
* load, not at the first run (the child spawns with an empty environment, so
* execvp's platform default would otherwise silently mask the mistake). A
* relative explicit path resolves against the host CWD, mirroring where
* `spawn` would have looked for it. When no `PATH` entry holds an executable
* match, `undefined` is returned and the LOAD check rejects the configuration:
* falling back to the bare name would let spawn's `env: {}` execvp silently
* start a system interpreter from the platform default PATH that the caller
* never asked for.
* @param bin - the configured interpreter (absolute or relative path, or bare command).
* @returns an absolute path when resolvable, else `undefined`.
*/
export function resolvePythonBin(bin: string): string | undefined {
if (isAbsolute(bin) || bin.includes('/')) return bin
if (isAbsolute(bin) || bin.includes('/')) {
// An explicit path is used as given (resolved against the host CWD when
// relative), but only when it is a real executable regular file. The same
// checks as the PATH branch below: `accessSync(X_OK)` admits directories,
// so `isFile` narrows further, and a path that fails either is not a
// usable interpreter.
const candidate = resolve(bin)
try {
accessSync(candidate, fsConstants.X_OK)
if (!statSync(candidate).isFile()) return undefined
return candidate
} catch {
return undefined
}
}
const path = process.env.PATH
/* v8 ignore next -- PATH is set in every environment the runtime boots in; the guard is defensive. */
if (path === undefined) return undefined
@@ -760,12 +779,18 @@ export class PythonCodeRuntime extends CodeRuntime {
if (this.config.pythonBin === '' || this.config.pythonBin.includes('\0')) {
throw new Error(`dsh-code-runtime-python: config.pythonBin must be a non-empty path without NUL bytes, got ${JSON.stringify(this.config.pythonBin)}`)
}
// A basename that is not on PATH must fail at load, not silently fall to
// execvp's platform default PATH (spawn runs with an EMPTY environment, so
// execvp would resolve /usr/bin:/bin and could start a system interpreter
// the caller never asked for). Absolute paths pass through.
if (resolvePythonBin(this.config.pythonBin) === undefined) {
throw new Error(`dsh-code-runtime-python: config.pythonBin ${JSON.stringify(this.config.pythonBin)} does not resolve on PATH`)
// An explicit path that is not an executable regular file must fail at load
// like any other self-contained configuration error (the empty/NUL cases
// above); a basename that is not on PATH must fail at load, not silently
// fall to execvp's platform default PATH (spawn runs with an EMPTY
// environment, so execvp would resolve /usr/bin:/bin and could start a
// system interpreter the caller never asked for). resolvePythonBin applies
// the executable-regular-file check to both forms and returns undefined for
// either failure; the message distinguishes the two so the fix is obvious.
const resolvedBin = resolvePythonBin(this.config.pythonBin)
if (resolvedBin === undefined) {
const explicit = isAbsolute(this.config.pythonBin) || this.config.pythonBin.includes('/')
throw new Error(`dsh-code-runtime-python: config.pythonBin ${JSON.stringify(this.config.pythonBin)} ${explicit ? 'is not an executable regular file' : 'does not resolve on PATH'}`)
}
// `maxWallMs` and `graceMs` are armed with setTimeout, which clamps any
// delay past MAX_TIMER_DELAY_MS to 1 ms without a word — turning a
@@ -971,7 +996,31 @@ export class PythonCodeRuntime extends CodeRuntime {
}
claimGlobal(errorClass.name, 'errorClass.name')
}
bindings.set(namespace.global, { functions: namespace.functions, ...errorClass ? { errorClass } : {} })
// Snapshot the callables into a plain own-property record before the
// child can dispatch. `namespace.functions` is caller-supplied, so it may
// expose members through getters or a Proxy; reading one of them inside
// the fd-3 `data` callback would throw OUTSIDE the dispatcher's try and
// terminate the host (defensive-patterns contain-callback-exceptions).
// Reading every member here, in run()'s synchronous validation segment,
// turns that throw into the seam-misuse rejection run() reserves for
// malformed bindings. The snapshot is also the single key set the boot
// frame advertises AND dispatch reads, so a getter whose keys differ
// between reads cannot desynchronize the child's allowed names from what
// the host will actually call. The record is null-prototype: the seam
// contract treats member names like `__proto__` or `constructor` as
// ordinary own properties, and a plain `{}` assignment of `__proto__`
// would hit the prototype setter instead of creating the own property.
const functions = Object.create(null) as Record<string, CodeBindingFunction>
for (const name of Object.keys(namespace.functions)) {
// Only callables enter the snapshot: a getter exposing a non-function
// member would otherwise assign a value the dispatcher's `typeof fn
// !== 'function'` check rejects anyway, and keeping it out of the
// snapshot keeps the boot frame's name list and the dispatch key set
// one and the same.
const fn = namespace.functions[name]
if (typeof fn === 'function') functions[name] = fn
}
bindings.set(namespace.global, { functions, ...errorClass ? { errorClass } : {} })
}
return bindings
}
@@ -1004,11 +1053,12 @@ export class PythonCodeRuntime extends CodeRuntime {
// right after the done frame, before any finalization-time flush could
// run. The `_LogStream` replacement of `sys.stdout`/`sys.stderr` is
// unaffected (it is a Python object, not the C-level stdio buffer).
// Load validated that a basename resolves; absolute paths pass through.
// The type assertion is the load-time contract (see the pythonBin load
// checks); a PATH change between load and run would make this undefined
// and spawn throws synchronously, which the surrounding try settles as
// worker-exit like any other spawn failure.
// Load validated that the configured interpreter resolves to an
// executable regular file (basename through PATH, explicit path
// directly). The type assertion is the load-time contract (see the
// pythonBin load checks); a PATH change between load and run would make
// this undefined and spawn throws synchronously, which the surrounding
// try settles as worker-exit like any other spawn failure.
const resolvedPythonBin = resolvePythonBin(this.config.pythonBin) as string
child = spawn(resolvedPythonBin, ['-u', '-I', bootstrapPath], {
env: {},
@@ -1751,6 +1801,24 @@ export class PythonCodeRuntime extends CodeRuntime {
// memory and the flush timing change.
const replyQueue: ReplyMessage[] = []
let draining = false
// Resolve when fd 3 can take another frame, OR when it is gone: a pipe
// destroyed under the drain (child exited, close-deadline teardown) never
// emits 'drain' again, so waiting on that event alone would hang the
// drain forever — `draining` stays true and the unconsumed queue is
// pinned with the closure. `once` plus the manual detach removes every
// listener whichever event wins, so a long backpressure wait leaves none
// behind.
const waitForDrain = (): Promise<void> => new Promise<void>((resolvePromise) => {
const finish = (): void => {
proto.off('drain', finish)
proto.off('close', finish)
proto.off('error', finish)
resolvePromise()
}
proto.once('drain', finish)
proto.once('close', finish)
proto.once('error', finish)
})
const drainReplies = async (): Promise<void> => {
if (draining) return
draining = true
@@ -1761,6 +1829,10 @@ export class PythonCodeRuntime extends CodeRuntime {
// depths reach 11 without the wall clock landing inside that window.
/* v8 ignore next -- see above; not schedulable from a test. */
if (settled) break
// A pipe destroyed under us (child exited, close deadline) will
// never emit 'drain' again; short-circuit before the write so the
// remaining frames are dropped by the `finally` below.
if (proto.destroyed) break
// Read by index, not `shift()`: a large `asyncio.gather` of wide
// bindings awaiting fd 3's `drain` can queue many frames, and each
// `shift()` re-slices the remaining array (O(n) per pop, O(n²) over
@@ -1779,7 +1851,7 @@ export class PythonCodeRuntime extends CodeRuntime {
// longer needs is dropped by the `settled` check above without ever
// being serialized.
if (!proto.write(`${encodeJsonPlain(payload)}\n`)) {
await once(proto, 'drain')
await waitForDrain()
}
}
} catch {
@@ -47,6 +47,26 @@ afterEach(() => {
spawnMock.mockReset()
})
/** A child that emits an async `error` (an ENOENT-style spawn failure). */
function fakeChildWithAsyncSpawnError(): EventEmitter {
const child = new EventEmitter() as EventEmitter & {
pid?: number
stdout: PassThrough
stderr: PassThrough
stdio: unknown[]
}
child.stdout = new PassThrough()
child.stderr = new PassThrough()
const proto = new PassThrough()
child.stdio = [new PassThrough(), child.stdout, child.stderr, proto]
// `spawn` reports an async failure via the child's `error` event; the run
// settles on it as a worker-exit without waiting for `close`.
setImmediate(() => {
child.emit('error', Object.assign(new Error('ENOENT: no such file or directory, spawn python3'), { code: 'ENOENT' }))
})
return child
}
/** A child whose fd-3 pipe accepts the boot write, then rejects the run write. */
function fakeChildWithAckThenThrowingFd3(): EventEmitter {
const child = new EventEmitter() as EventEmitter & {
@@ -71,6 +91,41 @@ function fakeChildWithAckThenThrowingFd3(): EventEmitter {
return child
}
/**
* A child whose fd-3 pipe backpressures every write and is then destroyed
* while the host waits for `drain`. The reply-drain loop must settle on the
* pipe's `close` (or destroyed state) rather than hanging forever waiting for
* a `drain` that can never arrive. Returns the pipe as well so the test can
* assert the drain wait left no listener behind.
*/
function fakeChildBackpressuredThenDestroyed(): { child: EventEmitter; proto: PassThrough } {
const child = new EventEmitter() as EventEmitter & {
pid?: number
stdout: PassThrough
stderr: PassThrough
stdio: unknown[]
}
child.stdout = new PassThrough()
child.stderr = new PassThrough()
const proto = new PassThrough()
// Every write reports backpressure (never a `drain` event): the only way the
// reply drain can proceed is the pipe being destroyed under it.
proto.write = () => false
child.stdio = [new PassThrough(), child.stdout, child.stderr, proto]
// Boot-ack → run frame → two binding calls whose replies backpressure, then
// destroy the pipe while the host still waits for `drain`: the drain loop
// resumes with a queued reply left and must break on the destroyed pipe.
setImmediate(() => {
proto.emit('data', Buffer.from('{"type":"boot-ack"}\n'))
setImmediate(() => {
proto.emit('data', Buffer.from('{"type":"call","id":0,"global":"tools","name":"f","args":[]}\n'))
proto.emit('data', Buffer.from('{"type":"call","id":1,"global":"tools","name":"f","args":[]}\n'))
setImmediate(() => proto.destroy())
})
})
return { child, proto }
}
describe('PythonCodeRuntime — boot-write failure', () => {
it('resolves a worker-exit when the fd-3 boot write throws (no TDZ ReferenceError)', async () => {
// Before the fix, the boot-write block ran BEFORE `wallTimer`, `onAbort`,
@@ -138,4 +193,58 @@ describe('PythonCodeRuntime — boot-write failure', () => {
expect(result.error?.message).toContain('failed to boot python subprocess')
await fiber.dispose()
})
it('resolves a worker-exit when spawn reports an async error', async () => {
// A spawn that fails asynchronously (ENOENT for an interpreter removed
// after load, or a libuv-level failure) surfaces through the child's
// `error` event, not a synchronous throw. The run must settle as a
// worker-exit from that event.
spawnMock.mockImplementation(() => fakeChildWithAsyncSpawnError())
const ctx = new Context()
const fiber = await ctx.plugin(PythonCodeRuntime)
const runtime = ctx.codeRuntime as InstanceType<typeof PythonCodeRuntime>
const result = await runtime.run({ program: 'return 1', bindings: [] })
expect(result.error?.kind).toBe('worker-exit')
expect(result.error?.message).toContain('python spawn error')
await fiber.dispose()
})
it('does not hang the reply drain when the pipe is destroyed mid-backpressure', async () => {
// The reply drain waits for `drain` when fd 3's buffer is full. A pipe
// destroyed under that wait never emits `drain` again; the drain must
// settle on `close` instead, or `draining` stays true and the queued reply
// (here a 4 MiB string) is pinned with the closure forever. The fake child
// backpressures every write and destroys fd 3 right after the binding
// call, so the host is mid-drain when the pipe dies. No `done` frame ever
// arrives, so the run settles on the wall clock — the drain wait must have
// removed its listeners by then (a `once('drain')` wait would leave one
// attached to the destroyed pipe forever).
let proto: PassThrough | undefined
spawnMock.mockImplementation(() => {
const fake = fakeChildBackpressuredThenDestroyed()
proto = fake.proto
return fake.child
})
const ctx = new Context()
const fiber = await ctx.plugin(PythonCodeRuntime, { maxWallMs: 3000 })
const runtime = ctx.codeRuntime as InstanceType<typeof PythonCodeRuntime>
const result = await runtime.run({
program: 'return 1',
bindings: [{ global: 'tools', functions: { f: async () => 'x'.repeat(4 * 1024 * 1024) } }],
})
expect(result.error?.kind).toBe('timeout')
// The drain wait settled on `close` and cleaned up after itself. The
// discriminating listener is `drain`: a `once('drain')` wait would leave
// its wrapper attached to the destroyed pipe forever (the event never
// fires again), while the fixed wait removes it. (`error` is not asserted:
// the runtime's own `silenceStreamError` occupies one slot.)
expect(proto).toBeDefined()
expect(proto?.listenerCount('drain')).toBe(0)
expect(proto?.listenerCount('close')).toBe(0)
await fiber.dispose()
})
})
@@ -1,4 +1,4 @@
import { existsSync, readdirSync, realpathSync, rmSync, statSync, writeFileSync } from 'node:fs'
import { existsSync, realpathSync, rmSync, statSync, writeFileSync } from 'node:fs'
import { mkdtemp, writeFile } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { basename, dirname, join } from 'node:path'
@@ -21,8 +21,19 @@ import type { CodeBindingFunction, CodeJsonValue, CodeRunResult } from '@deepsee
* Names one `py/` script whose `copyFileSync` must fail, for the partial-staging
* case. A real disk-full or missing-asset failure mid-copy cannot be produced
* from a test, and the leak only shows when `mkdtempSync` has already succeeded.
*
* `stagedDirs` records every staging directory THIS test file creates, so the
* leak assertions check the exact paths instead of a global tmpdir diff: a
* parallel vitest worker running the same prefix could create or remove
* `dsh-code-runtime-python-*` directories inside the sampling window, which a
* readdir diff would misattribute to this test. `boot-write-failure.spec.ts`
* records the same race and solves it with argv-based identity; recording the
* mkdtempSync results is the fs-mock equivalent.
*/
const { failNextCopyOf } = vi.hoisted(() => ({ failNextCopyOf: { value: undefined as string | undefined } }))
const { failNextCopyOf, stagedDirs } = vi.hoisted(() => ({
failNextCopyOf: { value: undefined as string | undefined },
stagedDirs: [] as string[],
}))
vi.mock('node:fs', async (importOriginal) => {
const actual = await importOriginal<typeof import('node:fs')>()
return {
@@ -34,6 +45,11 @@ vi.mock('node:fs', async (importOriginal) => {
}
actual.copyFileSync(source, destination)
},
mkdtempSync(prefix: string): string {
const dir = actual.mkdtempSync(prefix)
if (basename(prefix).startsWith('dsh-code-runtime-python-')) stagedDirs.push(dir)
return dir
},
}
})
@@ -149,6 +165,115 @@ describe('PythonCodeRuntime — seam descriptors and misuse', () => {
.rejects.toThrow(/pythonBin must be a non-empty path without NUL bytes/)
})
it('rejects an explicit pythonBin that is not an executable regular file, at load', async () => {
// An explicit path (absolute, or containing a slash) bypasses PATH lookup,
// so it must be validated directly: missing, non-executable, or directory
// paths are self-contained configuration errors that used to slip through
// load and surface only at the first run() as a misleading worker-exit.
// The message distinguishes the explicit-path failure from a basename that
// simply does not resolve on PATH.
const nodePath = await import('node:path')
const { mkdtempSync, writeFileSync, mkdirSync } = await import('node:fs')
const dir = mkdtempSync(nodePath.join(tmpdir(), 'dsh-bad-bin-'))
const notExecutable = nodePath.join(dir, 'not-executable')
writeFileSync(notExecutable, '#!/bin/sh\nexit 0\n') // Regular file, but no X bit.
const directory = nodePath.join(dir, 'is-a-directory')
mkdirSync(directory)
try {
const missing = new Context()
await expect(missing.plugin(PythonCodeRuntime, { pythonBin: nodePath.join(dir, 'missing') }))
.rejects.toThrow(/is not an executable regular file/)
const noX = new Context()
await expect(noX.plugin(PythonCodeRuntime, { pythonBin: notExecutable }))
.rejects.toThrow(/is not an executable regular file/)
const isDir = new Context()
await expect(isDir.plugin(PythonCodeRuntime, { pythonBin: directory }))
.rejects.toThrow(/is not an executable regular file/)
// A relative explicit path fails the same way, resolved against the host
// CWD: `dir` is absolute, so a slash-containing relative form of it is
// the dirname prefix plus the file, which does not exist as such.
const rel = new Context()
await expect(rel.plugin(PythonCodeRuntime, { pythonBin: './definitely-not-there-python' }))
.rejects.toThrow(/is not an executable regular file/)
} finally {
const { rmSync } = await import('node:fs')
rmSync(dir, { recursive: true, force: true })
}
})
it('keeps an explicit executable pythonBin working through load and run', async () => {
// The same validation that rejects bad explicit paths must admit a good
// one: an absolute path to the real interpreter (or a wrapper around it)
// is the deployment form the validation exists to serve.
const pyAbs = resolvePythonBin('python3') ?? 'python3'
const { runtime, fiber } = await setup({ pythonBin: pyAbs, maxWallMs: 30_000 })
const result = await runtime.run({ program: 'return 1', bindings: [] })
expect(result.error).toBeUndefined()
expect(result.value).toBe(1)
await fiber.dispose()
})
it('rejects a binding member accessor that throws, as seam misuse', async () => {
// `namespace.functions` is caller-supplied, so its members may come from a
// getter or Proxy. Reading one of them inside the fd-3 `data` callback used
// to throw OUTSIDE the dispatcher's try and terminate the host; the
// validation now snapshots the callables synchronously, so the throw
// surfaces as the seam-misuse rejection run() reserves for malformed
// bindings — the child is never spawned.
const { runtime } = await setup()
const exploding = {
get explode(): CodeBindingFunction {
throw new Error('getter blew up')
},
}
await expect(runtime.run({
program: 'return 1',
bindings: [{ global: 'tools', functions: exploding }],
})).rejects.toThrow(/getter blew up/)
})
it('snapshots binding callables once, so a getter is read exactly once', async () => {
// The snapshot also fixes the key set the boot frame advertises: the child
// learns the namespace names from the SAME record dispatch reads, so a
// getter whose keys differ between reads cannot desynchronize the two.
let reads = 0
const countReads = {
get first(): CodeBindingFunction {
reads += 1
return async () => 1
},
}
const { runtime, fiber } = await setup()
const result = await runtime.run({
program: 'return 1',
bindings: [{ global: 'tools', functions: countReads }],
})
expect(result.error).toBeUndefined()
// One read for the validation snapshot; the boot frame and every dispatch
// read the snapshot, not the getter.
expect(reads).toBe(1)
await fiber.dispose()
})
it('keeps a __proto__ binding member dispatchable', async () => {
// The seam contract treats member names like `__proto__` or `constructor`
// as ordinary own properties (null-prototype construction). The binding
// snapshot must preserve that: a plain `{}` record would hit the prototype
// setter on assignment and drop the member, so the child would never learn
// the name and a call to it would fail with KeyError.
const { runtime, fiber } = await setup()
const result = await runtime.run({
program: 'return await tools["__proto__"]({})',
bindings: [{
global: 'tools',
functions: { ['__proto__']: async () => 'proto-callable' },
}],
})
expect(result.error).toBeUndefined()
expect(result.value).toBe('proto-callable')
await fiber.dispose()
})
it('skips relative PATH entries when resolving a basename pythonBin', async () => {
// resolvePythonBin must return an absolute path: a RELATIVE PATH entry
// ('.' here) would otherwise resolve the basename against the host CWD.
@@ -371,12 +496,12 @@ describe('PythonCodeRuntime — seam descriptors and misuse', () => {
// `dispose()` is called in the same synchronous turn as `run()`, with no
// `await` between them, so it lands exactly in that window.
//
// The leak assertion compares before and after rather than requiring an
// empty tmpdir: other tests in this file build runtimes they never dispose,
// so only the directories this test adds are its own evidence.
const staged = (): string[] =>
readdirSync(realpathSync(tmpdir())).filter(name => name.startsWith('dsh-code-runtime-python-'))
const before = new Set(staged())
// The leak assertion checks the EXACT paths this test file staged (recorded
// by the mocked mkdtempSync) rather than diffing a global tmpdir: a
// parallel vitest worker can create or remove same-prefix directories
// inside the sampling window, which a readdir diff would misattribute to
// this test (boot-write-failure.spec.ts records the same race).
const stagedBefore = stagedDirs.length
const { fiber, runtime } = await setup({ maxWallMs: 8_000 })
const pending = runtime.run({ program: 'import time\nwhile True: time.sleep(0.1)', bindings: [] })
const disposed = fiber.dispose()
@@ -385,9 +510,9 @@ describe('PythonCodeRuntime — seam descriptors and misuse', () => {
// Whatever the run reports, it must be terminal and must not be a success.
expect(result.value).toBeUndefined()
expect(['abort', 'worker-exit', 'timeout']).toContain(result.error?.kind)
// Disposal is to quiescence, so this run's directory is gone once it
// resolves, and nothing recreated it afterwards.
expect(staged().filter(name => !before.has(name))).toEqual([])
// Disposal is to quiescence, so every directory this run staged is gone.
const created = stagedDirs.slice(stagedBefore)
for (const dir of created) expect(existsSync(dir)).toBe(false)
}, 15_000)
it('settles as abort when the signal fires in the same turn as the first run', async () => {
@@ -445,9 +570,9 @@ describe('PythonCodeRuntime — seam descriptors and misuse', () => {
//
// Only `copyFileSync` is stubbed, and only for the second script, so
// `mkdtempSync` really runs and the directory under assertion is real.
const staged = (): string[] =>
readdirSync(realpathSync(tmpdir())).filter(name => name.startsWith('dsh-code-runtime-python-'))
const before = new Set(staged())
// The assertion checks the exact paths this test staged (see the sibling
// disposal-race test for why a global tmpdir diff races parallel workers).
const stagedBefore = stagedDirs.length
failNextCopyOf.value = 'protocol.py'
try {
const { runtime } = await setup()
@@ -455,7 +580,7 @@ describe('PythonCodeRuntime — seam descriptors and misuse', () => {
expect(result.error?.kind).toBe('worker-exit')
expect(result.error?.message).toContain('failed to stage the python bootstrap')
// The partial directory is gone, so nothing accumulates across retries.
expect(staged().filter(name => !before.has(name))).toEqual([])
for (const dir of stagedDirs.slice(stagedBefore)) expect(existsSync(dir)).toBe(false)
} finally {
failNextCopyOf.value = undefined
}
@@ -2787,7 +2912,22 @@ describe('PythonCodeRuntime — budgets, termination, disposal', () => {
}, 5000)
it('reports a spawn failure via a bogus python binary as worker-exit', async () => {
const { runtime } = await setup({ pythonBin: '/nonexistent/python-binary', maxWallMs: 3000 })
// An explicit path that does not exist at LOAD is a configuration error and
// is rejected by the constructor (see the seam-misuse block). A path that
// is valid at load but gone by run time is a SUBSTRATE failure and must
// resolve as worker-exit: stage a real executable wrapper, load the runtime
// against it, then delete it before run() — the spawn then fails exactly
// like a child that cannot start.
const nodePath = await import('node:path')
const { mkdtempSync, rmSync, writeFileSync, chmodSync } = await import('node:fs')
const dir = mkdtempSync(nodePath.join(tmpdir(), 'dsh-spawn-fail-'))
const wrapper = nodePath.join(dir, 'python-wrapper')
const pyAbs = resolvePythonBin('python3') ?? 'python3'
writeFileSync(wrapper, `#!/bin/sh\nexec ${pyAbs} "$@"\n`, { mode: 0o755 })
chmodSync(wrapper, 0o755)
const { runtime } = await setup({ pythonBin: wrapper, maxWallMs: 3000 })
rmSync(wrapper)
rmSync(dir, { recursive: true, force: true })
const result = await runtime.run({
program: 'return 1',
bindings: [],
@@ -4902,6 +5042,35 @@ describe('PythonCodeRuntime — hostile peer', () => {
expect(result.value).toBe(8 * chunk.length)
}, 90_000)
it('drops queued binding replies when the child dies mid-drain, without hanging', async () => {
// drainReplies waits for `drain` when fd 3's buffer is full. If the child
// exits while a reply is queued, the pipe never emits `drain` again — the
// wait must also settle on `close`/`error`/destroyed, or `draining` stays
// true and the queue is pinned with the closure forever. The program fills
// the pipe with a wide binding reply and then exits without reading it, so
// the host is blocked mid-drain when the child dies; the run must still
// settle promptly (worker-exit from the close) rather than hanging on the
// drain wait.
const chunk = 'A'.repeat(4 * 1024 * 1024)
const { runtime } = await setup({ maxWallMs: 10_000 })
const result = await runtime.run({
program: [
'import asyncio',
// Resolve a reply big enough to backpressure fd 3, then exit without
// reading it: the child's `close` lands while the host still waits for
// `drain`, exercising the destroyed-pipe branch of the reply drain.
'pending = asyncio.create_task(tools.chunk({}))',
'await asyncio.sleep(0.05)',
'return "done"',
].join('\n'),
bindings: [{ global: 'tools', functions: { chunk: async () => chunk } }],
})
// The program returned, so the completion wins over the mid-flight reply;
// whatever the result, the run must settle (no hang on the drain wait).
expect(result.error).toBeUndefined()
expect(result.value).toBe('done')
}, 30_000)
it('bounds a flood of zero-byte log lines through the per-entry separator charge', async () => {
// Blank print() lines carry zero content bytes; without the +1 separator
// charge they would bypass maxLogBytes entirely and grow the retained