mirror of
https://github.com/deepseek-ai/deepseek-harness.git
synced 2026-08-29 04:26:38 +00:00
feat(code-runtime-python): add the fd-3 frame protocol
Introduce @deepseek-ai/dsh-code-runtime-python with the versionless JSON-lines protocol between the Node host and the CPython subprocess: the host-side hostile-frame codec (validateChildFrame, encodeJsonPlain, checkDoneValue, hasUnsafeIntegerToken, hasNonLosslessNumber, logTruncationMarker) and the Python-side wire-vocabulary mirror (py/protocol.py). This is the protocol layer of the code-runtime-python stack, split from #436 and based on the multi-language seam extension. The PythonCodeRuntime implementation and its Python JSON codec land in the backend-core PR on top of this branch. Ship the minimal buildable package skeleton (package.json, tsconfig, tsdown, barrel index, invariant companion, bilingual README) because the workspace-constraint, coverage, and invariant-topology gates require the package to exist and build the moment its directory does; the backend-core PR extends those files rather than creating them. Align py/protocol.py with src/protocol.ts (the round-12 review of #436 found LogMessage.truncated, DoneMessage.error.kind, and Namespace.errorClass stale) and guard the two runtime-executed surfaces (PROTOCOL_FD and the log truncation marker) with a real-python3 cross-language mirror e2e test.
This commit is contained in:
Chinesezjc
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/code-runtime/code-runtime-python/README.md
|
||||
README.md: 8a394f18f8e27addf0f4a7530cbdb31b9d629bb9
|
||||
README.zh.md: 1c246c952492eb574b6fc6c6bc6c76fffcabe01e
|
||||
@@ -0,0 +1,24 @@
|
||||
# @deepseek-ai/dsh-code-runtime-python
|
||||
|
||||
English | [中文](README.zh.md)
|
||||
|
||||
CPython-subprocess implementation of the [`@deepseek-ai/dsh-code-runtime`](../code-runtime/README.md) seam. Companion to [`@deepseek-ai/dsh-code-runtime-worker`](../code-runtime-worker/README.md); trades the Node worker thread for a fresh `python3` subprocess so model code is Python instead of TypeScript.
|
||||
|
||||
This package is built up across the code-runtime-python PR stack. This layer ships the wire protocol; the `PythonCodeRuntime` implementation that drives a `python3 -I` process over it lands on top of it.
|
||||
|
||||
## Wire protocol
|
||||
|
||||
The host and the CPython subprocess exchange a versionless, JSON-lines protocol on the child's fd 3 — one JSON object per line, leaving stdout/stderr free for the program's own output. `src/protocol.ts` is the host side; `py/protocol.py` mirrors its message shapes and the shared truncation-marker text on the Python side.
|
||||
|
||||
- **fd 3, not stdout** — Node pins the channel positionally with `stdio: ['pipe','pipe','pipe','pipe']`; the Python bootstrap reads the same `PROTOCOL_FD` constant. JSON-lines framing.
|
||||
- **Host treats every inbound frame as hostile** — model code has full access to fd 3 and can post anything through it, so `validateChildFrame` shape-validates and REBUILDS each frame before the host reads it: forged extra fields never ride along, a non-number call id can never be echoed into a reply, and junk drops to `undefined` rather than throwing in the host's message handler. The Python side trusts host replies (the host is not model-controlled).
|
||||
- **Lossless-JSON crossing** — completion values and binding arguments cross as exact JSON. `encodeJsonPlain` serializes a `JSON.parse`-produced value without recursion, so a deep value below the byte budget crosses intact instead of dying on `JSON.stringify`'s stack limit; `checkDoneValue` meters a forged completion value's byte length AND number losslessness in one bounded traversal that rejects an over-budget payload before enqueuing its children; `hasUnsafeIntegerToken` reads the raw frame text to catch an integer token that `JSON.parse` would silently round; `hasNonLosslessNumber` rejects a non-finite or negative-zero number in unbounded `call.args`. Beyond-safe-range integral doubles serialize through `BigInt` digits so the exact integer crosses, not the rounded `String()` form.
|
||||
- **Shared truncation marker** — `logTruncationMarker(maxBytes)` produces byte-identical text on both sides, so a truncated log run reads the same however the cap was hit. The `log` frame's `truncated` flag distinguishes the child ledger's own marker from program output.
|
||||
|
||||
## Model Experience
|
||||
|
||||
Indirectly, through Code Mode in [`dsh-tools`](../../core/tools/README.md), which renders this backend's exact completion value when it fits (or an explicit `invalid-output` / `output-limit` failure), plus the exact `[dsh-code-runtime-python] log capture truncated at <maxLogBytes> bytes` log marker, into a retained `run_code` result.
|
||||
|
||||
#### KV Cache effect
|
||||
|
||||
No direct invalidation; the named consumer owns any request-prefix changes.
|
||||
@@ -0,0 +1,24 @@
|
||||
# @deepseek-ai/dsh-code-runtime-python
|
||||
|
||||
[English](README.md) | 中文
|
||||
|
||||
[`@deepseek-ai/dsh-code-runtime`](../code-runtime/README.md) seam 的 CPython 子进程实现。与 [`@deepseek-ai/dsh-code-runtime-worker`](../code-runtime-worker/README.md) 配套；以全新的 `python3` 子进程取代 Node worker 线程，让模型代码从 TypeScript 换成 Python。
|
||||
|
||||
本包分多个 code-runtime-python PR 逐层搭建。本层交付 wire protocol；在其之上驱动 `python3 -I` 进程的 `PythonCodeRuntime` 实现随后落地。
|
||||
|
||||
## Wire protocol
|
||||
|
||||
host 与 CPython 子进程在子进程的 fd 3 上交换一个无版本号的 JSON-lines 协议——每行一个 JSON 对象，让 stdout/stderr 空出给程序自己的输出。`src/protocol.ts` 是 host 侧；`py/protocol.py` 在 Python 侧镜像其帧词汇与共享的截断标记文本。
|
||||
|
||||
- **fd 3，而非 stdout** —— Node 通过 `stdio: ['pipe','pipe','pipe','pipe']` 按位置钉住通道；Python bootstrap 读取相同的 `PROTOCOL_FD` 常量。JSON-lines 帧。
|
||||
- **host 把每个入站帧当作敌意输入** —— 模型代码对 fd 3 有完全访问权、可通过它发送任意内容，所以 `validateChildFrame` 在 host 读取前对每个帧做形状校验并重建：伪造的额外字段绝不随行，非数字的 call id 绝不会被回显进 reply，垃圾降为 `undefined` 被丢弃，而不是在 host 的 message handler 里抛错。Python 侧信任 host 回复（host 不受模型控制）。
|
||||
- **lossless-JSON 穿越** —— 完成值与 binding 参数以精确 JSON 穿越。`encodeJsonPlain` 无递归地序列化一个 `JSON.parse` 产出的值，使低于字节预算的深层值能完整穿越，而不是死在 `JSON.stringify` 的栈限制上；`checkDoneValue` 在一次有界遍历中同时计量伪造完成值的字节长度与数字无损性，在把子节点入栈之前就拒绝超预算 payload；`hasUnsafeIntegerToken` 读取原始帧文本，捕获 `JSON.parse` 会静默舍入的整数 token；`hasNonLosslessNumber` 拒绝无字节上限的 `call.args` 中的非有限数或负零。超出安全范围的整数型 double 通过 `BigInt` 数字序列化，穿越的是精确整数而非 `String()` 的舍入形式。
|
||||
- **共享截断标记** —— `logTruncationMarker(maxBytes)` 在两侧产出逐字节一致的文本，使被截断的日志运行无论从哪侧触达上限都读起来一致。`log` 帧的 `truncated` 标志把子进程 ledger 自身的标记与程序输出区分开。
|
||||
|
||||
## Model Experience
|
||||
|
||||
Indirectly, through Code Mode in [`dsh-tools`](../../core/tools/README.md), which renders this backend's exact completion value when it fits (or an explicit `invalid-output` / `output-limit` failure), plus the exact `[dsh-code-runtime-python] log capture truncated at <maxLogBytes> bytes` log marker, into a retained `run_code` result.
|
||||
|
||||
#### KV Cache effect
|
||||
|
||||
No direct invalidation; the named consumer owns any request-prefix changes.
|
||||
@@ -0,0 +1,39 @@
|
||||
{
|
||||
"name": "@deepseek-ai/dsh-code-runtime-python",
|
||||
"description": "CPython subprocess implementation of the DeepSeek Harness code-execution seam",
|
||||
"version": "0.0.1",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"main": "lib/index.js",
|
||||
"types": "lib/types/index.d.ts",
|
||||
"exports": {
|
||||
".": {
|
||||
"types": "./lib/types/index.d.ts",
|
||||
"default": "./lib/index.js"
|
||||
},
|
||||
"./invariant": {
|
||||
"types": "./lib/types/invariant.d.ts",
|
||||
"default": "./lib/invariant.js"
|
||||
},
|
||||
"./package.json": "./package.json"
|
||||
},
|
||||
"files": [
|
||||
"lib/index.js",
|
||||
"lib/invariant.js",
|
||||
"py/**/*.py",
|
||||
"lib/types/**/*.d.ts",
|
||||
"lib/types/**/*.d.ts.map",
|
||||
"src"
|
||||
],
|
||||
"license": "BSD-3-Clause",
|
||||
"peerDependencies": {
|
||||
"@deepseek-ai/dsh-code-runtime": "^0.0.1",
|
||||
"@deepseek-ai/dsh-invariants": "^0.0.1",
|
||||
"cordis": "^4.0.0-rc.7"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@deepseek-ai/dsh-code-runtime": "workspace:^",
|
||||
"@deepseek-ai/dsh-invariants": "workspace:^",
|
||||
"cordis": "^4.0.0-rc.7"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,126 @@
|
||||
"""Wire protocol vocabulary for the Python side of dsh-code-runtime-python.
|
||||
|
||||
Mirrors ``src/protocol.ts``. Frames travel on fd 3 as JSON-lines (one JSON
|
||||
object per line). The host validates every inbound frame; this side trusts
|
||||
host replies.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any, Literal, TypedDict, Union
|
||||
|
||||
# The protocol fd from the child's perspective. Node passes
|
||||
# ``stdio: [pipe, pipe, pipe, pipe]`` so the fourth entry (fd 3) is the
|
||||
# framed-JSON channel; stdout/stderr stay clear for the program's own output.
|
||||
PROTOCOL_FD = 3
|
||||
|
||||
|
||||
class BootMessage(TypedDict):
|
||||
"""Host → child, first frame on fd 3. Carries every cap and the namespaces."""
|
||||
|
||||
type: Literal["boot"]
|
||||
cpuSeconds: int
|
||||
addressSpaceBytes: int
|
||||
maxLogBytes: int
|
||||
maxValueBytes: int
|
||||
namespaces: list["Namespace"]
|
||||
|
||||
|
||||
class ErrorClass(TypedDict):
|
||||
"""A namespace's program-visible exception class: rejected calls raise its
|
||||
instances carrying the failed member name on ``memberNameProperty``."""
|
||||
|
||||
name: str
|
||||
memberNameProperty: str
|
||||
|
||||
|
||||
class Namespace(TypedDict, total=False):
|
||||
"""One binding namespace declaration: the global name, its function names,
|
||||
and an optional program-visible ``errorClass`` for rejected calls."""
|
||||
|
||||
global_: str # required; renamed on the wire: JSON field is ``global`` (Python keyword collision)
|
||||
names: list[str] # required
|
||||
errorClass: ErrorClass # optional — mirrors the TS `errorClass?`
|
||||
|
||||
|
||||
class RunMessage(TypedDict):
|
||||
"""Host → child, sent after ``boot-ack``. Carries only the program body."""
|
||||
|
||||
type: Literal["run"]
|
||||
program: str
|
||||
|
||||
|
||||
class BootAckMessage(TypedDict):
|
||||
"""Child → host: resource limits applied, ready for the run message."""
|
||||
|
||||
type: Literal["boot-ack"]
|
||||
|
||||
|
||||
class CallMessage(TypedDict):
|
||||
"""Child → host: one bridged binding call from the model program."""
|
||||
|
||||
type: Literal["call"]
|
||||
id: int
|
||||
global_: str # wire field is ``global``
|
||||
name: str
|
||||
args: Any
|
||||
|
||||
|
||||
class LogMessage(TypedDict, total=False):
|
||||
"""Child → host: one captured text chunk, streamed eagerly.
|
||||
|
||||
``truncated`` is set only on the frame that IS the child ledger's truncation
|
||||
marker (not program output), so the host stops capturing at the same point
|
||||
the child did — mirrors the TS `truncated?`.
|
||||
"""
|
||||
|
||||
type: Literal["log"] # required
|
||||
text: str # required
|
||||
truncated: bool # optional
|
||||
|
||||
|
||||
class DoneErrorField(TypedDict):
|
||||
"""Child → host: the failure carried on a ``done`` frame. ``kind`` is one of
|
||||
the three the host validates; ``message`` is the traceback or diagnostic."""
|
||||
|
||||
kind: Literal["exception", "invalid-output", "output-limit"]
|
||||
message: str
|
||||
|
||||
|
||||
class DoneMessage(TypedDict, total=False):
|
||||
"""Child → host: the program settled. ``value`` and ``error`` are optional per the TS mirror."""
|
||||
|
||||
type: Literal["done"] # required — TypedDict(total=False) allows this via a required subclass in Py 3.11+; MVP keeps it flat
|
||||
value: Any
|
||||
error: DoneErrorField
|
||||
|
||||
|
||||
ChildToHost = Union[BootAckMessage, CallMessage, LogMessage, DoneMessage]
|
||||
|
||||
|
||||
class ReplyOk(TypedDict):
|
||||
type: Literal["reply"]
|
||||
id: int
|
||||
ok: Literal[True]
|
||||
value: Any
|
||||
|
||||
|
||||
class ReplyErr(TypedDict):
|
||||
type: Literal["reply"]
|
||||
id: int
|
||||
ok: Literal[False]
|
||||
message: str
|
||||
|
||||
|
||||
ReplyMessage = Union[ReplyOk, ReplyErr]
|
||||
HostToChild = ReplyMessage
|
||||
|
||||
|
||||
def log_truncation_marker(max_bytes: int) -> str:
|
||||
"""Return the in-band marker for a log ledger that exhausted its budget.
|
||||
|
||||
Byte-identical text on both sides of the wire so a truncated run reads the
|
||||
same however the cap was hit.
|
||||
"""
|
||||
|
||||
return f"[dsh-code-runtime-python] log capture truncated at {max_bytes} bytes"
|
||||
@@ -0,0 +1,20 @@
|
||||
/**
|
||||
* CPython subprocess code runtime for the DeepSeek Harness code-execution seam.
|
||||
*
|
||||
* This layer of the package ships the versionless fd-3 wire protocol between the
|
||||
* Node host and the CPython subprocess; the `PythonCodeRuntime` implementation
|
||||
* that drives a `python3 -I` process over it lands on top of this seam. The
|
||||
* protocol's host-side codec and hostile-frame validators are re-exported so the
|
||||
* runtime and its tests share one wire vocabulary.
|
||||
* @module @deepseek-ai/dsh-code-runtime-python
|
||||
*/
|
||||
|
||||
export type { BootMessage, ChildToHost, ReplyMessage } from './protocol.ts'
|
||||
export {
|
||||
checkDoneValue,
|
||||
encodeJsonPlain,
|
||||
hasNonLosslessNumber,
|
||||
hasUnsafeIntegerToken,
|
||||
logTruncationMarker,
|
||||
validateChildFrame,
|
||||
} from './protocol.ts'
|
||||
@@ -0,0 +1,30 @@
|
||||
/**
|
||||
* Package-owned invariant companion for `@deepseek-ai/dsh-code-runtime-python`.
|
||||
* @module @deepseek-ai/dsh-code-runtime-python/invariant
|
||||
*/
|
||||
|
||||
/* jscpd:ignore-start */
|
||||
import type { Context } from 'cordis'
|
||||
import type { InvariantInstaller } from '@deepseek-ai/dsh-invariants'
|
||||
|
||||
const PACKAGE_NAME = '@deepseek-ai/dsh-code-runtime-python'
|
||||
|
||||
/** Cordis companion plugin name. */
|
||||
export const name = 'code-runtime-python-invariant'
|
||||
/** Service required before the companion can reserve package ownership. */
|
||||
export const inject = ['invariants']
|
||||
|
||||
/**
|
||||
* No runtime invariant: this process-boundary implementation exposes no same-process event relation;
|
||||
* the fd-3 protocol and real-subprocess integration tests cover it.
|
||||
*/
|
||||
const install: InvariantInstaller = () => {}
|
||||
|
||||
/**
|
||||
* Register this package's invariant companion.
|
||||
* @param ctx - Cordis context carrying the invariant service.
|
||||
* @returns the installed registration's disposer after setup succeeds.
|
||||
*/
|
||||
export const apply = (ctx: Context): Promise<() => void> =>
|
||||
Promise.resolve(ctx.invariants.register(PACKAGE_NAME, install))
|
||||
/* jscpd:ignore-end */
|
||||
@@ -0,0 +1,420 @@
|
||||
/**
|
||||
* Versionless, JSON-lines wire protocol between the Node host and the CPython subprocess. Frames
|
||||
* travel on the child's fd 3 (one JSON object per line), leaving stdout/stderr free for the
|
||||
* program's own output. Host treats every inbound frame as hostile because model code can post
|
||||
* anything through the same fd; the Python bootstrap trusts host replies.
|
||||
* @module @deepseek-ai/dsh-code-runtime-python/src/protocol
|
||||
*/
|
||||
|
||||
// The protocol channel is fd 3 from the child's perspective — the host pins it
|
||||
// positionally via `stdio: ['pipe','pipe','pipe','pipe']` (index.ts), and the
|
||||
// Python bootstrap reads the same constant from its own protocol.py.
|
||||
|
||||
/**
|
||||
* What the host sends immediately after spawn, as the first line on fd 3. The
|
||||
* Python bootstrap reads this, applies resource limits, then waits for the
|
||||
* subsequent run frame. Separated from the run so the run message stays
|
||||
* pure model input.
|
||||
*/
|
||||
export interface BootMessage {
|
||||
type: 'boot'
|
||||
/** RLIMIT_CPU seconds; the Python bootstrap sets this on itself before executing model code. */
|
||||
cpuSeconds: number
|
||||
/** RLIMIT_AS bytes; caps address space so a runaway allocation fails cleanly. */
|
||||
addressSpaceBytes: number
|
||||
/** Shared byte budget for captured log text (Python-side ledger). */
|
||||
maxLogBytes: number
|
||||
/** Byte cap for the rendered completion value. */
|
||||
maxValueBytes: number
|
||||
/**
|
||||
* The namespaces to materialize inside the program (globals + names;
|
||||
* functions stay host-side). `errorClass` asks the bootstrap to mint a
|
||||
* program-visible exception class under that global: rejected calls raise
|
||||
* its instances carrying the member name on `memberNameProperty`.
|
||||
*/
|
||||
namespaces: { global: string; names: string[]; errorClass?: { name: string; memberNameProperty: string } }[]
|
||||
}
|
||||
|
||||
// The run request `{ type: 'run', program }` follows BootMessage once the
|
||||
// child acknowledges with `boot-ack`; the host sends it as an inline literal
|
||||
// (it carries only the model's program body — caps and bindings crossed on boot).
|
||||
|
||||
/** Python → host: acknowledges boot completed and resource limits are in place. */
|
||||
interface BootAckMessage {
|
||||
type: 'boot-ack'
|
||||
}
|
||||
|
||||
/** Python → host: one bridged binding call (`await tools.name(args)` inside the program). */
|
||||
interface CallMessage {
|
||||
type: 'call'
|
||||
/** Python-issued correlation id; the host answers each id at most once and ignores duplicates. */
|
||||
id: number
|
||||
/** The namespace global the call targets. */
|
||||
global: string
|
||||
/** The function name within the namespace. */
|
||||
name: string
|
||||
/** The JSON-safe argument the model program passed. */
|
||||
args: unknown
|
||||
}
|
||||
|
||||
/**
|
||||
* Python → host: captured text, streamed eagerly so output survives a
|
||||
* mid-run termination (RLIMIT_CPU, SIGTERM/SIGKILL, host wall-timeout).
|
||||
*/
|
||||
interface LogMessage {
|
||||
type: 'log'
|
||||
text: string
|
||||
/**
|
||||
* Set when this frame IS the child ledger's truncation marker rather than
|
||||
* program output. The two ledgers can exhaust at different points — one
|
||||
* child entry larger than `maxLogBytes` sends only the marker while the host
|
||||
* ledger is still nearly empty — so the host cannot infer the child's state
|
||||
* from its own budget, and comparing the text against the marker string
|
||||
* would also honour a program that printed that string itself. Carrying it
|
||||
* as a field lets the host stop capturing at the same point the child did
|
||||
* and keeps exactly one marker in `logs`.
|
||||
*/
|
||||
truncated?: boolean
|
||||
}
|
||||
|
||||
/**
|
||||
* Python → host: the program settled. `error` carries a program exception
|
||||
* (traceback text), an `invalid-output` (completion value was not lossless
|
||||
* JSON), or an `output-limit` (serialized completion exceeded the configured
|
||||
* cap); wall/CPU budgets, aborts, and substrate death are observed host-side.
|
||||
* `value` is present only on a clean completion that produced one, and crosses
|
||||
* as exact lossless JSON — never substituted or truncated.
|
||||
*/
|
||||
interface DoneMessage {
|
||||
type: 'done'
|
||||
value?: unknown
|
||||
error?: { kind: 'exception' | 'invalid-output' | 'output-limit'; message: string }
|
||||
}
|
||||
|
||||
/**
|
||||
* Every message the Python side sends. The member interfaces stay module-
|
||||
* private: consumers match on the union's discriminant; the host sends the
|
||||
* boot and run frames as inline literals.
|
||||
*/
|
||||
export type ChildToHost = BootAckMessage | CallMessage | LogMessage | DoneMessage
|
||||
|
||||
/** Host → Python: the answer to one {@link CallMessage}. */
|
||||
export type ReplyMessage =
|
||||
| { type: 'reply'; id: number; ok: true; value: unknown }
|
||||
| { type: 'reply'; id: number; ok: false; message: string }
|
||||
|
||||
/**
|
||||
* The in-band marker text announcing that log capture stopped at the byte
|
||||
* budget. Shared wire vocabulary: the Python-side LogBuffer emits it when ITS
|
||||
* ledger exhausts, and the host emits identical text when its own ledger drops
|
||||
* a frame first (forged fd-3 traffic, stray stdout bytes) — a truncated run
|
||||
* reads the same however the cap was hit.
|
||||
* @param maxBytes - the configured `maxLogBytes` the marker names.
|
||||
* @returns the marker line.
|
||||
*/
|
||||
export function logTruncationMarker(maxBytes: number): string {
|
||||
return `[dsh-code-runtime-python] log capture truncated at ${maxBytes} bytes`
|
||||
}
|
||||
|
||||
/**
|
||||
* Serialize one JSON-parse-produced value without recursion. `JSON.stringify`
|
||||
* recurses per nesting level and throws `RangeError` a few thousand levels
|
||||
* deep, but the seam's `CodeJsonValue` has no depth limit — an honest deep
|
||||
* completion or binding resolution below the byte budget must cross intact
|
||||
* (the worker backend's wire is equally stack-safe). Callers must pass a value
|
||||
* produced by `JSON.parse` (or equally JSON-plain): only `null`, finite
|
||||
* numbers, booleans, strings, dense arrays, and plain objects — this encoder
|
||||
* validates nothing. Output is byte-identical to compact `JSON.stringify`.
|
||||
* @param value - a JSON-plain value (e.g. straight from `JSON.parse`).
|
||||
* @returns the compact JSON encoding.
|
||||
*/
|
||||
export function encodeJsonPlain(value: unknown): string {
|
||||
type Task = { text: string } | { value: unknown }
|
||||
const chunks: string[] = []
|
||||
const tasks: Task[] = [{ value }]
|
||||
for (let task = tasks.pop(); task !== undefined; task = tasks.pop()) {
|
||||
if ('text' in task) {
|
||||
chunks.push(task.text)
|
||||
continue
|
||||
}
|
||||
const current = task.value
|
||||
if (typeof current === 'string') {
|
||||
chunks.push(JSON.stringify(current))
|
||||
} else if (Array.isArray(current)) {
|
||||
chunks.push('[')
|
||||
tasks.push({ text: ']' })
|
||||
for (let index = current.length - 1; index >= 0; index--) {
|
||||
if (index < current.length - 1) tasks.push({ text: ',' })
|
||||
tasks.push({ value: current[index] })
|
||||
}
|
||||
} else if (typeof current === 'object' && current !== null) {
|
||||
const record = current as Record<string, unknown>
|
||||
chunks.push('{')
|
||||
tasks.push({ text: '}' })
|
||||
const keys = Object.keys(record)
|
||||
for (let index = keys.length - 1; index >= 0; index--) {
|
||||
const key = keys[index] as string
|
||||
if (index < keys.length - 1) tasks.push({ text: ',' })
|
||||
tasks.push({ value: record[key] })
|
||||
tasks.push({ text: `${JSON.stringify(key)}:` })
|
||||
}
|
||||
} else {
|
||||
chunks.push(scalarJson(current))
|
||||
}
|
||||
}
|
||||
return chunks.join('')
|
||||
}
|
||||
|
||||
/**
|
||||
* One scalar (null, boolean, finite number) as JSON text. A beyond-safe-range
|
||||
* integral double needs BigInt digits: `String(2 ** 60)` emits the ROUNDED
|
||||
* `...847000` form, and echoing that to the child would silently change the
|
||||
* integer the seam promised to carry losslessly — `BigInt(2 ** 60)` prints the
|
||||
* exact `...846976` the double actually holds.
|
||||
* @param current - a JSON-plain scalar (JSON.parse emits nothing else).
|
||||
* @returns its JSON encoding.
|
||||
*/
|
||||
function scalarJson(current: unknown): string {
|
||||
if (typeof current === 'number' && Number.isInteger(current) && !Number.isSafeInteger(current)) {
|
||||
return BigInt(current).toString()
|
||||
}
|
||||
return String(current)
|
||||
}
|
||||
|
||||
/**
|
||||
* Meter a forged done value's compact-JSON byte length AND its number
|
||||
* losslessness in one bounded traversal, stopping the instant `maxBytes` is
|
||||
* crossed. A forged `done.value` arrives straight off fd 3 and can sit anywhere
|
||||
* below the 256 MiB frame ceiling while `maxValueBytes` defaults to 32 KiB. The
|
||||
* previous split — an unbounded `hasNonLosslessNumber` scan in
|
||||
* {@link validateChildFrame} followed by a separate byte meter — pushed every
|
||||
* member of a wide flat payload onto a scan stack before any cap check ran, so
|
||||
* a below-ceiling forgery could still force a hundreds-of-megabytes host
|
||||
* allocation. Folding both jobs here rejects over-budget BEFORE enqueuing an
|
||||
* array's or object's children, keeping the traversal O(cap). A non-lossless
|
||||
* number (non-finite, negative zero) is caught only when the value fits the
|
||||
* budget — an over-budget value is rejected regardless, so the distinction is
|
||||
* moot. Same JSON-plain precondition and traversal shape as
|
||||
* {@link encodeJsonPlain}; per-scalar encoding delegates to `JSON.stringify`.
|
||||
* @param value - a JSON-plain value (e.g. straight from `JSON.parse`).
|
||||
* @param maxBytes - the completion-value budget in bytes.
|
||||
* @returns `{ ok: true, bytes }` with the exact serialized size, or
|
||||
* `{ ok: false, reason }` — `over-budget` once the size exceeds `maxBytes`,
|
||||
* `non-lossless` on a non-finite or negative-zero number.
|
||||
*/
|
||||
export function checkDoneValue(value: unknown, maxBytes: number): { ok: true; bytes: number } | { ok: false; reason: 'over-budget' | 'non-lossless' } {
|
||||
let bytes = 0
|
||||
const stack: unknown[] = [value]
|
||||
while (stack.length > 0) {
|
||||
const current = stack.pop()
|
||||
if (typeof current === 'number') {
|
||||
if (!Number.isFinite(current) || Object.is(current, -0)) return { ok: false, reason: 'non-lossless' }
|
||||
bytes += Buffer.byteLength(scalarJson(current), 'utf8')
|
||||
} else if (typeof current === 'string') {
|
||||
// Lower-bound BEFORE materializing the escaped form: every UTF-16 code
|
||||
// unit is at least one UTF-8 byte plus the two quotes, so a huge or
|
||||
// control-heavy forged string (whose escaped copy expands severalfold)
|
||||
// is rejected without allocating that copy.
|
||||
if (bytes + current.length + 2 > maxBytes) return { ok: false, reason: 'over-budget' }
|
||||
bytes += Buffer.byteLength(JSON.stringify(current), 'utf8')
|
||||
} else if (Array.isArray(current)) {
|
||||
// Brackets plus one comma per gap; elements add themselves. Reject
|
||||
// BEFORE enqueuing children: every element serializes to at least one
|
||||
// byte, so a forged flat array below the frame ceiling but far above
|
||||
// the budget fails here without growing the host stack by millions of
|
||||
// entries first.
|
||||
bytes += 2 + (current.length > 1 ? current.length - 1 : 0)
|
||||
if (bytes + current.length > maxBytes) return { ok: false, reason: 'over-budget' }
|
||||
for (const item of current) stack.push(item)
|
||||
} else if (typeof current === 'object' && current !== null) {
|
||||
const record = current as Record<string, unknown>
|
||||
// Count own keys WITHOUT Object.entries/Object.keys: either would
|
||||
// allocate one slot (entries: one pair array) per member before the
|
||||
// bound below could run, recreating the spike the bound exists to stop.
|
||||
let count = 0
|
||||
for (const key in record) if (Object.hasOwn(record, key)) count += 1
|
||||
bytes += 2 + (count > 1 ? count - 1 : 0)
|
||||
// Same pre-enqueue bound: each entry contributes its quoted key (>= 2
|
||||
// bytes), the colon, and a >= 1-byte value.
|
||||
if (bytes + count * 4 > maxBytes) return { ok: false, reason: 'over-budget' }
|
||||
for (const key in record) {
|
||||
if (!Object.hasOwn(record, key)) continue
|
||||
// The same string lower bound, before escaping the key.
|
||||
if (bytes + key.length + 3 > maxBytes) return { ok: false, reason: 'over-budget' }
|
||||
bytes += Buffer.byteLength(JSON.stringify(key), 'utf8') + 1
|
||||
stack.push(record[key])
|
||||
}
|
||||
} else {
|
||||
bytes += Buffer.byteLength(scalarJson(current), 'utf8')
|
||||
}
|
||||
if (bytes > maxBytes) return { ok: false, reason: 'over-budget' }
|
||||
}
|
||||
return { ok: true, bytes }
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether a raw JSON line contains an integer token that would lose precision
|
||||
* as a JavaScript number. `JSON.parse` silently rounds such a token
|
||||
* (`9007199254740993` becomes `...992`) BEFORE any validation can see it, so
|
||||
* the check must read the source text; a beyond-safe-range token whose double
|
||||
* parse round-trips exactly (`2**53`, `2**60`) is lossless and passes. The scan walks the line skipping string literals (a digit run
|
||||
* inside a string is data, not a number token) and tests every number token
|
||||
* in plain integer form — no fraction or exponent, which parse as doubles by
|
||||
* intent. A reviver cannot do this job: the reviver walk recurses per nesting
|
||||
* level and would reintroduce the depth limit `encodeJsonPlain` removes.
|
||||
* @param line - the raw UTF-8 text of one JSON-lines frame.
|
||||
* @returns true when an unsafe integer token is present outside strings.
|
||||
*/
|
||||
export function hasUnsafeIntegerToken(line: string): boolean {
|
||||
for (let index = 0; index < line.length; index++) {
|
||||
const char = line[index]
|
||||
if (char === '"') {
|
||||
// Skip the string literal, honoring backslash escapes.
|
||||
for (index++; index < line.length; index++) {
|
||||
if (line[index] === '\\') index++
|
||||
else if (line[index] === '"') break
|
||||
}
|
||||
continue
|
||||
}
|
||||
if (char === '-' || (char !== undefined && char >= '0' && char <= '9')) {
|
||||
let end = index + 1
|
||||
while (end < line.length) {
|
||||
const c = line[end] as string
|
||||
if ((c >= '0' && c <= '9') || c === '.' || c === 'e' || c === 'E' || c === '+' || c === '-') end++
|
||||
else break
|
||||
}
|
||||
const token = line.slice(index, end)
|
||||
// Beyond the safe range an integer token is still lossless IFF the
|
||||
// double parse round-trips exactly (2**53 does; 2**53+1 rounds) — the
|
||||
// canonical boundary accepts every JS-double-exact value, so only a
|
||||
// genuinely rounding token marks the frame as forged.
|
||||
if (/^-?\d+$/.test(token)) {
|
||||
const parsed = Number(token)
|
||||
// A token that parses to Infinity is trivially lossy; a finite
|
||||
// beyond-safe-range one is lossy only when the BigInt round-trip
|
||||
// disagrees.
|
||||
if (!Number.isFinite(parsed)) return true
|
||||
if (!Number.isSafeInteger(parsed) && BigInt(token) !== BigInt(parsed)) return true
|
||||
}
|
||||
index = end - 1
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
/**
|
||||
* Lazily yield one plain object's own enumerable property values. A generator
|
||||
* (not `Object.values`/`Object.entries`) because {@link hasNonLosslessNumber}
|
||||
* traverses breadth it cannot bound: those helpers copy the whole member list
|
||||
* up front, so a wide forged object would cost a second full-breadth
|
||||
* allocation before a single value is examined.
|
||||
* @param record - a JSON-parse-produced object.
|
||||
* @yields each own enumerable property value, in key order.
|
||||
*/
|
||||
function* ownValues(record: object): Generator {
|
||||
for (const key in record) {
|
||||
if (Object.hasOwn(record, key)) yield (record as Record<string, unknown>)[key]
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether a JSON.parse-produced value contains a number outside lossless
|
||||
* JSON: non-finite (`1e400` parses to `Infinity`) or negative zero (`-0.0`
|
||||
* parses to JS `-0`, whose sign bit a re-serialization drops). The honest
|
||||
* child's validator rejects these before sending, so a frame carrying one is
|
||||
* forged.
|
||||
*
|
||||
* Runs on `call.args`, which — unlike a completion value — has NO seam byte
|
||||
* cap, so there is no budget to reject a wide payload against the way
|
||||
* {@link checkDoneValue} does. The traversal therefore holds ONE cursor per
|
||||
* NESTING LEVEL (an array or {@link ownValues} iterator) instead of one entry
|
||||
* per member: a forged flat `args` just below the 256 MiB frame ceiling would
|
||||
* otherwise push tens of millions of stack entries — and `Object.values` would
|
||||
* copy each object's full breadth — allocating hundreds of megabytes beyond
|
||||
* what `JSON.parse` already holds. Iterative either way, so a deep frame
|
||||
* cannot overflow the host stack.
|
||||
* @param value - a JSON-parse-produced value from an fd-3 frame.
|
||||
* @returns true when any contained number is non-finite or negative zero.
|
||||
*/
|
||||
export function hasNonLosslessNumber(value: unknown): boolean {
|
||||
const cursors: Iterator<unknown>[] = [[value].values()]
|
||||
while (cursors.length > 0) {
|
||||
// The loop condition guarantees a top cursor.
|
||||
const cursor = cursors.at(-1) as Iterator<unknown>
|
||||
const step = cursor.next()
|
||||
if (step.done === true) {
|
||||
cursors.pop()
|
||||
continue
|
||||
}
|
||||
const current = step.value
|
||||
if (typeof current === 'number') {
|
||||
if (!Number.isFinite(current) || Object.is(current, -0)) return true
|
||||
} else if (Array.isArray(current)) {
|
||||
cursors.push((current as unknown[]).values())
|
||||
} else if (typeof current === 'object' && current !== null) {
|
||||
cursors.push(ownValues(current))
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
/**
|
||||
* Runtime shape gate for inbound fd-3 traffic. Model code has full access to
|
||||
* fd 3 and can post anything — `null`, primitives, poisoned fields — so the
|
||||
* compile-time union means nothing here: every field is validated and REBUILT
|
||||
* before the host reads it (forged extras never ride along; a non-number id
|
||||
* can never be echoed into a reply). Junk returns `undefined` and is dropped
|
||||
* so a throw in the host's `message` handler cannot crash the host process.
|
||||
* @param raw - one JSON-parsed frame from fd 3.
|
||||
* @returns the rebuilt frame, or `undefined` to drop it silently.
|
||||
*/
|
||||
export function validateChildFrame(raw: unknown): ChildToHost | undefined {
|
||||
if (typeof raw !== 'object' || raw === null) return undefined
|
||||
const m = raw as Record<string, unknown>
|
||||
switch (m.type) {
|
||||
case 'boot-ack':
|
||||
return { type: 'boot-ack' }
|
||||
case 'log':
|
||||
if (typeof m.text !== 'string') return undefined
|
||||
// Rebuilt, not passed through: a forged `truncated` of any other type
|
||||
// would reach the host as a truthy value and silence capture for the
|
||||
// rest of the run. Only the literal `true` counts.
|
||||
return { type: 'log', text: m.text, ...m.truncated === true ? { truncated: true } : {} }
|
||||
case 'call': {
|
||||
// The id must be a finite number: it is echoed verbatim into the reply
|
||||
// frame, and a forged `1e400` id (Infinity after JSON.parse) would make
|
||||
// the reply unencodable as strict JSON.
|
||||
if (typeof m.id !== 'number' || !Number.isFinite(m.id) || typeof m.global !== 'string' || typeof m.name !== 'string') return undefined
|
||||
// A forged frame can omit `args` entirely; rebuilding it as `undefined`
|
||||
// would invoke the binding with a non-JSON value, bypassing the
|
||||
// lossless-JSON argument boundary. Any PRESENT value is JSON-plain by
|
||||
// construction (the frame came from JSON.parse), so presence is the
|
||||
// whole check.
|
||||
if (!Object.hasOwn(m, 'args')) return undefined
|
||||
// JSON.parse yields Infinity for 1e400 and preserves -0; both are
|
||||
// outside lossless JSON, and the honest child never sends them.
|
||||
if (hasNonLosslessNumber(m.args)) return undefined
|
||||
return { type: 'call', id: m.id, global: m.global, name: m.name, args: m.args }
|
||||
}
|
||||
case 'done': {
|
||||
// The value passes through untouched here: scanning it for non-lossless
|
||||
// numbers would push every member of a wide forged payload before any
|
||||
// byte cap runs. The done handler's bounded `checkDoneValue` folds the
|
||||
// losslessness check into the metered traversal, rejecting over-budget
|
||||
// before it enqueues children.
|
||||
const err = m.error
|
||||
if (err === undefined) {
|
||||
return m.value === undefined ? { type: 'done' } : { type: 'done', value: m.value }
|
||||
}
|
||||
if (typeof err !== 'object' || err === null) return undefined
|
||||
const { kind, message } = err as Record<string, unknown>
|
||||
if (typeof message !== 'string') return undefined
|
||||
if (kind !== 'exception' && kind !== 'invalid-output' && kind !== 'output-limit') return undefined
|
||||
return m.value === undefined
|
||||
? { type: 'done', error: { kind, message } }
|
||||
: { type: 'done', value: m.value, error: { kind, message } }
|
||||
}
|
||||
default:
|
||||
return undefined
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,60 @@
|
||||
import { execFile } from 'node:child_process'
|
||||
import { existsSync } from 'node:fs'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
import { promisify } from 'node:util'
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { logTruncationMarker } from '../src/protocol.ts'
|
||||
|
||||
/**
|
||||
* Cross-language mirror check for the two protocol surfaces the host and the
|
||||
* CPython subprocess share at runtime, spawning a real `python3` to read them
|
||||
* from `py/protocol.py`. `src/protocol.ts` and `py/protocol.py` declare the same
|
||||
* frame vocabulary on two sides of the wire; the only values both sides EXECUTE
|
||||
* against are `PROTOCOL_FD` (the fd the channel is pinned to) and the log
|
||||
* truncation marker text (emitted verbatim by whichever ledger exhausts first),
|
||||
* so a drift there silently corrupts a live run. Self-skips when no `python3` is
|
||||
* on PATH — CI provides one; the pure-TS `protocol.spec.ts` covers the host
|
||||
* codec unconditionally.
|
||||
*/
|
||||
|
||||
const execFileAsync = promisify(execFile)
|
||||
const pyDir = fileURLToPath(new URL('../py', import.meta.url))
|
||||
|
||||
async function hasPython3(): Promise<boolean> {
|
||||
try {
|
||||
await execFileAsync('python3', ['--version'])
|
||||
return true
|
||||
} catch {
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
const python3Available = await hasPython3()
|
||||
|
||||
describe.skipIf(!python3Available)('protocol.py mirrors protocol.ts at runtime', () => {
|
||||
it('agrees on PROTOCOL_FD and the log truncation marker across byte budgets', async () => {
|
||||
const budgets = [1, 65536, 1048576]
|
||||
const probe = [
|
||||
'import json, sys',
|
||||
`sys.path.insert(0, ${JSON.stringify(pyDir)})`,
|
||||
'from protocol import PROTOCOL_FD, log_truncation_marker',
|
||||
`budgets = ${JSON.stringify(budgets)}`,
|
||||
'print(json.dumps({',
|
||||
' "fd": PROTOCOL_FD,',
|
||||
' "markers": [log_truncation_marker(b) for b in budgets],',
|
||||
'}))',
|
||||
].join('\n')
|
||||
const { stdout } = await execFileAsync('python3', ['-I', '-c', probe])
|
||||
const seen = JSON.parse(stdout) as { fd: number; markers: string[] }
|
||||
// fd 3 is the wire contract, not a tunable: index.ts pins it positionally.
|
||||
expect(seen.fd).toBe(3)
|
||||
expect(seen.markers).toEqual(budgets.map(budget => logTruncationMarker(budget)))
|
||||
})
|
||||
})
|
||||
|
||||
it('names the py/ directory that ships with the package', () => {
|
||||
// The package.json `files` list ships `py/**/*.py`; the mirror test resolves
|
||||
// the marker source relative to the built package, so the directory must exist
|
||||
// beside the tests even when python3 is absent from the runner.
|
||||
expect(existsSync(pyDir)).toBe(true)
|
||||
})
|
||||
@@ -0,0 +1,239 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { checkDoneValue, encodeJsonPlain, hasNonLosslessNumber, hasUnsafeIntegerToken, logTruncationMarker, validateChildFrame } from '../src/index.ts'
|
||||
|
||||
describe('logTruncationMarker', () => {
|
||||
it('names the configured byte budget', () => {
|
||||
expect(logTruncationMarker(65536)).toBe('[dsh-code-runtime-python] log capture truncated at 65536 bytes')
|
||||
expect(logTruncationMarker(1)).toBe('[dsh-code-runtime-python] log capture truncated at 1 bytes')
|
||||
})
|
||||
})
|
||||
|
||||
describe('validateChildFrame', () => {
|
||||
it('rebuilds boot-ack frames without extra fields', () => {
|
||||
expect(validateChildFrame({ type: 'boot-ack' })).toEqual({ type: 'boot-ack' })
|
||||
// Forged extras never ride along.
|
||||
expect(validateChildFrame({ type: 'boot-ack', extra: 'x' })).toEqual({ type: 'boot-ack' })
|
||||
})
|
||||
|
||||
it('rebuilds log frames when the text field is a string', () => {
|
||||
expect(validateChildFrame({ type: 'log', text: 'hi' })).toEqual({ type: 'log', text: 'hi' })
|
||||
// Non-string text drops.
|
||||
expect(validateChildFrame({ type: 'log', text: 42 })).toBeUndefined()
|
||||
expect(validateChildFrame({ type: 'log' })).toBeUndefined()
|
||||
})
|
||||
|
||||
it('rebuilds call frames with a numeric id, string global, and string name', () => {
|
||||
expect(validateChildFrame({ type: 'call', id: 1, global: 'tools', name: 'echo', args: { x: 1 } }))
|
||||
.toEqual({ type: 'call', id: 1, global: 'tools', name: 'echo', args: { x: 1 } })
|
||||
// A frame with NO args key drops whole: rebuilding it as `undefined`
|
||||
// would invoke the binding with a non-JSON value, bypassing the
|
||||
// lossless-JSON argument boundary. Any present value is JSON-plain by
|
||||
// construction (frames arrive via JSON.parse), so null passes.
|
||||
expect(validateChildFrame({ type: 'call', id: 2, global: 'tools', name: 'echo' })).toBeUndefined()
|
||||
expect(validateChildFrame({ type: 'call', id: 2, global: 'tools', name: 'echo', args: null }))
|
||||
.toEqual({ type: 'call', id: 2, global: 'tools', name: 'echo', args: null })
|
||||
// A missing/mistyped required field drops.
|
||||
expect(validateChildFrame({ type: 'call', id: '1', global: 'tools', name: 'echo' })).toBeUndefined()
|
||||
expect(validateChildFrame({ type: 'call', id: 1, global: 7, name: 'echo' })).toBeUndefined()
|
||||
expect(validateChildFrame({ type: 'call', id: 1, global: 'tools' })).toBeUndefined()
|
||||
})
|
||||
|
||||
it('rebuilds done frames with optional value/error', () => {
|
||||
expect(validateChildFrame({ type: 'done' })).toEqual({ type: 'done' })
|
||||
expect(validateChildFrame({ type: 'done', value: 42 })).toEqual({ type: 'done', value: 42 })
|
||||
expect(validateChildFrame({ type: 'done', error: { kind: 'exception', message: 'boom' } }))
|
||||
.toEqual({ type: 'done', error: { kind: 'exception', message: 'boom' } })
|
||||
expect(validateChildFrame({ type: 'done', error: { kind: 'invalid-output', message: 'lossy' } }))
|
||||
.toEqual({ type: 'done', error: { kind: 'invalid-output', message: 'lossy' } })
|
||||
expect(validateChildFrame({ type: 'done', error: { kind: 'output-limit', message: 'big' } }))
|
||||
.toEqual({ type: 'done', error: { kind: 'output-limit', message: 'big' } })
|
||||
expect(validateChildFrame({ type: 'done', value: 1, error: { kind: 'exception', message: 'boom' } }))
|
||||
.toEqual({ type: 'done', value: 1, error: { kind: 'exception', message: 'boom' } })
|
||||
// A `value: undefined` field is dropped (JSON never carries it, but a forged
|
||||
// shape might; the rebuild coalesces to the absent case).
|
||||
expect(validateChildFrame({ type: 'done', value: undefined })).toEqual({ type: 'done' })
|
||||
// A missing or unrecognized kind drops the frame: the child always sends
|
||||
// one of the three, so anything else is a forgery.
|
||||
expect(validateChildFrame({ type: 'done', error: { message: 'boom' } })).toBeUndefined()
|
||||
expect(validateChildFrame({ type: 'done', error: { kind: 'timeout', message: 'x' } })).toBeUndefined()
|
||||
})
|
||||
|
||||
it('rejects malformed done frames', () => {
|
||||
// error must be an object.
|
||||
expect(validateChildFrame({ type: 'done', error: 'boom' })).toBeUndefined()
|
||||
expect(validateChildFrame({ type: 'done', error: null })).toBeUndefined()
|
||||
// error.message must be a string.
|
||||
expect(validateChildFrame({ type: 'done', error: {} })).toBeUndefined()
|
||||
expect(validateChildFrame({ type: 'done', error: { message: 42 } })).toBeUndefined()
|
||||
})
|
||||
|
||||
it('drops non-object inputs and unknown types silently', () => {
|
||||
expect(validateChildFrame(null)).toBeUndefined()
|
||||
expect(validateChildFrame(undefined)).toBeUndefined()
|
||||
expect(validateChildFrame(42)).toBeUndefined()
|
||||
expect(validateChildFrame('str')).toBeUndefined()
|
||||
expect(validateChildFrame({})).toBeUndefined()
|
||||
expect(validateChildFrame({ type: 'unknown' })).toBeUndefined()
|
||||
})
|
||||
|
||||
it('drops CALL frames whose args are non-finite or negative zero', () => {
|
||||
// JSON.parse turns 1e400 into Infinity and preserves -0; the honest child
|
||||
// rejects both before sending, so a call frame carrying one is forged.
|
||||
expect(validateChildFrame({ type: 'call', id: 1, global: 'tools', name: 'x', args: { n: Infinity } })).toBeUndefined()
|
||||
expect(validateChildFrame({ type: 'call', id: Infinity, global: 'tools', name: 'x', args: null })).toBeUndefined()
|
||||
// Plain zero and ordinary floats pass.
|
||||
expect(validateChildFrame({ type: 'call', id: 1, global: 'tools', name: 'x', args: [0, 1.5] }))
|
||||
.toEqual({ type: 'call', id: 1, global: 'tools', name: 'x', args: [0, 1.5] })
|
||||
})
|
||||
|
||||
it('passes DONE values through untouched — losslessness is metered later', () => {
|
||||
// validateChildFrame no longer scans done.value: an unbounded scan would
|
||||
// push every member of a wide forged payload before any byte cap ran. The
|
||||
// done handler's checkDoneValue folds losslessness into the metered walk.
|
||||
expect(validateChildFrame({ type: 'done', value: Infinity })).toEqual({ type: 'done', value: Infinity })
|
||||
expect(validateChildFrame({ type: 'done', value: [{ x: -0 }] })).toEqual({ type: 'done', value: [{ x: -0 }] })
|
||||
expect(validateChildFrame({ type: 'done', value: [0, 1.5] })).toEqual({ type: 'done', value: [0, 1.5] })
|
||||
})
|
||||
})
|
||||
|
||||
describe('lossless-number scan', () => {
|
||||
it('finds non-finite and negative-zero numbers at any depth, iteratively', () => {
|
||||
expect(hasNonLosslessNumber(Infinity)).toBe(true)
|
||||
expect(hasNonLosslessNumber(-Infinity)).toBe(true)
|
||||
expect(hasNonLosslessNumber(NaN)).toBe(true)
|
||||
expect(hasNonLosslessNumber(-0)).toBe(true)
|
||||
expect(hasNonLosslessNumber({ a: [1, { b: -0 }] })).toBe(true)
|
||||
expect(hasNonLosslessNumber({ a: [0, 1.5, 'x', null, true] })).toBe(false)
|
||||
// Deep nesting must not overflow the stack.
|
||||
let deep: unknown = 0
|
||||
for (let i = 0; i < 100000; i++) deep = [deep]
|
||||
expect(hasNonLosslessNumber(deep)).toBe(false)
|
||||
})
|
||||
|
||||
it('walks wide arrays and objects one member at a time', () => {
|
||||
// `call.args` carries no seam byte cap, so a wide forged payload has no
|
||||
// budget to be rejected against — the walk must hold one cursor per
|
||||
// NESTING LEVEL, not one entry per member, or a flat payload just below
|
||||
// the 256 MiB frame ceiling would allocate tens of millions of stack
|
||||
// entries (and `Object.values` a second full-breadth copy). Observable
|
||||
// through the boundary: a wide payload whose per-member cost the old shape
|
||||
// would have paid still scans, and a violation ANYWHERE in it is found
|
||||
// wherever it sits.
|
||||
const wideArray = new Array(2_000_000).fill(0) as unknown[]
|
||||
expect(hasNonLosslessNumber(wideArray)).toBe(false)
|
||||
// Last element, so the cursor must run the whole breadth lazily.
|
||||
wideArray[wideArray.length - 1] = -0
|
||||
expect(hasNonLosslessNumber(wideArray)).toBe(true)
|
||||
const wideObject: Record<string, unknown> = {}
|
||||
for (let i = 0; i < 200_000; i++) wideObject[`k${i}`] = i
|
||||
expect(hasNonLosslessNumber(wideObject)).toBe(false)
|
||||
wideObject.last = Infinity
|
||||
expect(hasNonLosslessNumber(wideObject)).toBe(true)
|
||||
// Interleaved nesting: a per-level cursor must resume its parent after a
|
||||
// child level ends, so a violation after a nested container is still seen.
|
||||
expect(hasNonLosslessNumber([[1], { a: 2 }, NaN])).toBe(true)
|
||||
})
|
||||
|
||||
it('scans only own enumerable properties', () => {
|
||||
// The per-level cursor filters own keys (a prototype-carrying frame is
|
||||
// impossible off JSON.parse, but the filter is what keeps the walk equal
|
||||
// to what the encoder would serialize).
|
||||
const withProto = Object.create({ inherited: -0 }) as Record<string, unknown>
|
||||
withProto.own = 1
|
||||
expect(hasNonLosslessNumber(withProto)).toBe(false)
|
||||
})
|
||||
})
|
||||
|
||||
describe('unsafe-integer token scan', () => {
|
||||
it('flags integer tokens outside the safe range, skipping strings and float forms', () => {
|
||||
expect(hasUnsafeIntegerToken('{"v":9007199254740993}')).toBe(true)
|
||||
// Exact beyond-safe-range tokens are lossless and pass (2**53, 2**64).
|
||||
expect(hasUnsafeIntegerToken('{"v":9007199254740992}')).toBe(false)
|
||||
expect(hasUnsafeIntegerToken('{"v":18446744073709551616}')).toBe(false)
|
||||
// A token that parses to Infinity is trivially lossy.
|
||||
expect(hasUnsafeIntegerToken(`{"v":${'9'.repeat(400)}}`)).toBe(true)
|
||||
expect(hasUnsafeIntegerToken('{"v":-9007199254740993}')).toBe(true)
|
||||
expect(hasUnsafeIntegerToken('{"v":9007199254740991}')).toBe(false)
|
||||
expect(hasUnsafeIntegerToken('{"v":"9007199254740993"}')).toBe(false)
|
||||
expect(hasUnsafeIntegerToken(String.raw`{"v":"esc\"9007199254740993"}`)).toBe(false)
|
||||
expect(hasUnsafeIntegerToken('{"v":9007199254740993.0}')).toBe(false)
|
||||
expect(hasUnsafeIntegerToken('{"v":9e99}')).toBe(false)
|
||||
})
|
||||
})
|
||||
|
||||
describe('checkDoneValue', () => {
|
||||
it('matches the exact encoded size and rejects one byte over', () => {
|
||||
const cases: unknown[] = [null, true, false, 0, -1.5, 'a"b\\', [], {}, [1, 'x', null], { a: [1, 2], b: { c: 'd' } }]
|
||||
for (const value of cases) {
|
||||
const exact = Buffer.byteLength(JSON.stringify(value), 'utf8')
|
||||
expect(checkDoneValue(value, exact), JSON.stringify(value)).toEqual({ ok: true, bytes: exact })
|
||||
expect(checkDoneValue(value, exact - 1), JSON.stringify(value)).toEqual({ ok: false, reason: 'over-budget' })
|
||||
expect(encodeJsonPlain(value)).toBe(JSON.stringify(value))
|
||||
}
|
||||
})
|
||||
|
||||
it('stops early on a huge value instead of measuring it whole', () => {
|
||||
const huge = { data: 'x'.repeat(1_000_000), tail: 'y' }
|
||||
expect(checkDoneValue(huge, 1024)).toEqual({ ok: false, reason: 'over-budget' })
|
||||
// A forged flat array below the frame ceiling must fail BEFORE its
|
||||
// elements are enqueued — the pre-enqueue bound keeps the walk O(cap).
|
||||
const flat = new Array(10_000_000).fill(0)
|
||||
expect(checkDoneValue(flat, 1024)).toEqual({ ok: false, reason: 'over-budget' })
|
||||
// Same bound for a wide object: braces+commas fit the cap, but the
|
||||
// per-entry lower bound (quoted key + colon + value) does not, so it fails
|
||||
// before any key is metered or any value enqueued.
|
||||
const wide: Record<string, number> = {}
|
||||
for (let i = 0; i < 10; i++) wide[`k${i}`] = i
|
||||
expect(checkDoneValue(wide, 12)).toEqual({ ok: false, reason: 'over-budget' })
|
||||
})
|
||||
|
||||
it('rejects an over-budget string on its length before escaping it', () => {
|
||||
// A control-heavy forged string escapes to ~6x its length; the walk must
|
||||
// refuse it on the cheap `length + 2` lower bound so the escaped copy is
|
||||
// never allocated. Observable through the boundary: a string whose LENGTH
|
||||
// already exceeds the cap fails even though every character is 1 byte.
|
||||
expect(checkDoneValue(' '.repeat(4096), 1024)).toEqual({ ok: false, reason: 'over-budget' })
|
||||
// The bound is a lower bound, never a false rejection: a string that fits
|
||||
// exactly still passes with its exact escaped size.
|
||||
expect(checkDoneValue(' ', 8)).toEqual({ ok: true, bytes: 8 })
|
||||
expect(checkDoneValue(' ', 7)).toEqual({ ok: false, reason: 'over-budget' })
|
||||
// Same lower bound for keys, checked before the key is escaped.
|
||||
expect(checkDoneValue({ [' '.repeat(4096)]: 1 }, 1024)).toEqual({ ok: false, reason: 'over-budget' })
|
||||
})
|
||||
|
||||
it('meters only own enumerable keys', () => {
|
||||
// The walk counts keys with a `for...in` + hasOwn pass rather than
|
||||
// Object.keys/entries (which allocate per member before the bound). A
|
||||
// prototype-carrying forgery is impossible off JSON.parse, but the own-key
|
||||
// filter is what keeps the count equal to the encoder's.
|
||||
const withProto = Object.create({ inherited: 'x' }) as Record<string, unknown>
|
||||
withProto.own = 1
|
||||
expect(checkDoneValue(withProto, 1024)).toEqual({ ok: true, bytes: Buffer.byteLength('{"own":1}', 'utf8') })
|
||||
})
|
||||
|
||||
it('rejects non-finite and negative-zero numbers at any depth as non-lossless', () => {
|
||||
expect(checkDoneValue(Infinity, 1024)).toEqual({ ok: false, reason: 'non-lossless' })
|
||||
expect(checkDoneValue(-Infinity, 1024)).toEqual({ ok: false, reason: 'non-lossless' })
|
||||
expect(checkDoneValue(NaN, 1024)).toEqual({ ok: false, reason: 'non-lossless' })
|
||||
expect(checkDoneValue(-0, 1024)).toEqual({ ok: false, reason: 'non-lossless' })
|
||||
expect(checkDoneValue({ a: [1, { b: -0 }] }, 1024)).toEqual({ ok: false, reason: 'non-lossless' })
|
||||
// An ordinary finite value within budget passes with its exact byte count.
|
||||
const clean = { a: [0, 1.5, 'x', null, true] }
|
||||
expect(checkDoneValue(clean, 1024)).toEqual({ ok: true, bytes: Buffer.byteLength(JSON.stringify(clean), 'utf8') })
|
||||
})
|
||||
|
||||
it('meters deep nesting iteratively without overflowing the stack', () => {
|
||||
let deep: unknown = 0
|
||||
for (let i = 0; i < 100_000; i++) deep = [deep]
|
||||
// 100000 '[' + '0' + 100000 ']' = 200001 bytes.
|
||||
expect(checkDoneValue(deep, 1_000_000)).toEqual({ ok: true, bytes: 200_001 })
|
||||
})
|
||||
|
||||
it('emits exact digits for beyond-safe integral doubles', () => {
|
||||
// String(2**60) prints the ROUNDED ...847000; echoing that to the child
|
||||
// would change the integer. BigInt digits give the exact ...846976.
|
||||
const v = JSON.parse('[1152921504606846976]') as unknown
|
||||
expect(encodeJsonPlain(v)).toBe('[1152921504606846976]')
|
||||
expect(checkDoneValue(v, 100)).toEqual({ ok: true, bytes: Buffer.byteLength('[1152921504606846976]', 'utf8') })
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,21 @@
|
||||
{
|
||||
"extends": "../../../tsconfig.base.json",
|
||||
"compilerOptions": {
|
||||
"rootDir": "src",
|
||||
"outDir": "lib/types"
|
||||
},
|
||||
"include": [
|
||||
"src"
|
||||
],
|
||||
"references": [
|
||||
{
|
||||
"path": "../../../vendor/cosmokit"
|
||||
},
|
||||
{
|
||||
"path": "../../../vendor/cordis"
|
||||
},
|
||||
{
|
||||
"path": "../../support/invariants"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,16 @@
|
||||
import { defineConfig } from 'tsdown'
|
||||
|
||||
/**
|
||||
* Single ESM bundle. The Python-side code is not TypeScript and ships verbatim
|
||||
* under `py/` (whitelisted in package.json `files`) — no build step needed.
|
||||
*/
|
||||
export default defineConfig({
|
||||
entry: ['lib/types/index.js', 'lib/types/invariant.js'],
|
||||
outDir: 'lib',
|
||||
format: ['esm'],
|
||||
platform: 'node',
|
||||
target: 'es2024',
|
||||
fixedExtension: false,
|
||||
dts: false,
|
||||
clean: false,
|
||||
})
|
||||
Reference in New Issue
Block a user