mirror of
https://github.com/deepseek-ai/deepseek-harness.git
synced 2026-08-29 04:26:38 +00:00
docs(code-runtime-python): name the roster aliases by subset direction, align metering prose
Rename UnionCoversRoster/RosterCoversUnion to UnionSubsetOfRoster/RosterSubsetOfUnion so the names read in the same direction as their extends clauses, share the python3 -I -B flags between the two mirror probes, and align the README and Agent Note prose with the checkDoneValue JSDoc: the escaped-size scan is the metering itself, not deferred work. Regenerate docs/module-graph.md, which listed code-runtime-python twice.
This commit is contained in:
Chinesezjc
+2
-2
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-31-code-runtime-python-fd3-protocol.md
|
||||
2026-07-31-code-runtime-python-fd3-protocol.md: c497ebdcd26f174f3b8bfa325ef404b057d83bb8
|
||||
2026-07-31-code-runtime-python-fd3-protocol.zh.md: b4747199a584d62ee4b61df98e487d60b0028d78
|
||||
2026-07-31-code-runtime-python-fd3-protocol.md: 5f9600a3f658df907d68ae695d42154009947fbd
|
||||
2026-07-31-code-runtime-python-fd3-protocol.zh.md: dc3ae7cdfe1daf6e2ab1326e353c1bdbf9833175
|
||||
|
||||
+1
-1
@@ -15,7 +15,7 @@ This layer of the stack delivers only that protocol, so the large `PythonCodeRun
|
||||
`src/protocol.ts` is the host side of the wire vocabulary and its hostile-frame codec:
|
||||
|
||||
- **`validateChildFrame`** shape-validates and REBUILDS every inbound frame. The compile-time union means nothing on fd 3 — a forged frame can carry `null`, poisoned fields, or omit required ones — so each accepted frame is reconstructed field by field: forged extras never ride along, a non-finite call id can never be echoed into a reply, and junk returns `undefined` to be dropped rather than throwing in the host's message handler.
|
||||
- **`encodeJsonPlain` / `checkDoneValue` / `hasUnsafeIntegerToken` / `hasNonLosslessNumber`** are the lossless-JSON codec and meters. They traverse iteratively (an explicit stack, not recursion) so a deep value below the byte budget crosses intact; `checkDoneValue` folds byte-metering and number-losslessness into one walk that rejects an over-budget payload before the INCREMENTAL work it would otherwise add — a non-allocating escaped-size scan (`jsonStringBytesUpTo`) and the enqueued children. It does not re-bound the frame's own width: `done.value` is already `JSON.parse`'d when the check runs, so the payload's size is paid upstream and capped there by the host's fixed fd-3 receive buffer (a later stack layer), not here. Beyond-safe-range integral doubles serialize through `BigInt` digits so the exact integer crosses, not `String()`'s rounded form.
|
||||
- **`encodeJsonPlain` / `checkDoneValue` / `hasUnsafeIntegerToken` / `hasNonLosslessNumber`** are the lossless-JSON codec and meters. They traverse iteratively (an explicit stack, not recursion) so a deep value below the byte budget crosses intact; `checkDoneValue` folds byte-metering and number-losslessness into one walk that rejects an over-budget payload before the INCREMENTAL work it would otherwise add — the enqueued children; strings and keys are metered by a non-allocating escaped-size scan (`jsonStringBytesUpTo`), so the escaped copy is never materialized. It does not re-bound the frame's own width: `done.value` is already `JSON.parse`'d when the check runs, so the payload's size is paid upstream and capped there by the host's fixed fd-3 receive buffer (a later stack layer), not here. Beyond-safe-range integral doubles serialize through `BigInt` digits so the exact integer crosses, not `String()`'s rounded form.
|
||||
- **`logTruncationMarker`** produces the in-band marker text a log ledger emits when it exhausts its byte budget.
|
||||
|
||||
`py/protocol.py` mirrors the message shapes as `TypedDict`s and re-declares the two surfaces both sides EXECUTE against — `PROTOCOL_FD = 3` and `log_truncation_marker` — with byte-identical text.
|
||||
|
||||
+1
-1
@@ -15,7 +15,7 @@ CPython code-runtime 后端（`@deepseek-ai/dsh-code-runtime-python`，分多个
|
||||
`src/protocol.ts` 是 wire vocabulary 的 host 侧及其敌意帧编解码：
|
||||
|
||||
- **`validateChildFrame`** 对每个入站帧做形状校验并重建。编译期 union 在 fd 3 上毫无意义——伪造帧可携带 `null`、被污染的字段，或省略必需字段——所以每个被接受的帧都逐字段重建：伪造的额外字段绝不随行，非有限的 call id 绝不会被回显进 reply，垃圾返回 `undefined` 被丢弃，而不是在 host 的 message handler 里抛错。
|
||||
- **`encodeJsonPlain` / `checkDoneValue` / `hasUnsafeIntegerToken` / `hasNonLosslessNumber`** 是 lossless-JSON 编解码器与计量器。它们迭代遍历（显式栈，非递归），使低于字节预算的深层值能完整穿越；`checkDoneValue` 把字节计量和数字无损性折进一次遍历，在它本会新增的 INCREMENTAL 工作之前就拒绝超预算 payload——先做非分配的转义尺寸扫描（`jsonStringBytesUpTo`），再入栈子节点。它不会重新约束帧自身的宽度：`done.value` 在检查运行时已被 `JSON.parse`，故 payload 的尺寸是上游代价，由 host 固定的 fd-3 接收缓冲（后续 stack 层）在那里封顶，而非本函数。超出安全范围的整数型 double 通过 `BigInt` 数字序列化，穿越的是精确整数而非 `String()` 的舍入形式。
|
||||
- **`encodeJsonPlain` / `checkDoneValue` / `hasUnsafeIntegerToken` / `hasNonLosslessNumber`** 是 lossless-JSON 编解码器与计量器。它们迭代遍历（显式栈，非递归），使低于字节预算的深层值能完整穿越；`checkDoneValue` 把字节计量和数字无损性折进一次遍历，在它本会新增的 INCREMENTAL 工作之前就拒绝超预算 payload——即入栈子节点；字符串与 key 由非分配的转义尺寸扫描（`jsonStringBytesUpTo`）计量，从不物化转义副本。它不会重新约束帧自身的宽度：`done.value` 在检查运行时已被 `JSON.parse`，故 payload 的尺寸是上游代价，由 host 固定的 fd-3 接收缓冲（后续 stack 层）在那里封顶，而非本函数。超出安全范围的整数型 double 通过 `BigInt` 数字序列化，穿越的是精确整数而非 `String()` 的舍入形式。
|
||||
- **`logTruncationMarker`** 产出日志 ledger 耗尽字节预算时发出的带内标记文本。
|
||||
|
||||
`py/protocol.py` 用 `TypedDict` 镜像消息形状，并重新声明两侧都会 EXECUTE 的两个面——`PROTOCOL_FD = 3` 与 `log_truncation_marker`——文本逐字节一致。
|
||||
|
||||
@@ -1150,7 +1150,6 @@ flowchart TD
|
||||
| [`client-test-runtime`](../packages/client/test-runtime) | `client` | [`client-runtime`](../packages/client/runtime), [`client-ui-slots`](../packages/client/ui-slots), [`client-web-react`](../packages/client/web-react), [`host-apiproxy`](../packages/host/apiproxy), [`invariants`](../packages/support/invariants) |
|
||||
| [`client-ui-settings`](../packages/client/ui-settings) | `client` | [`client-runtime`](../packages/client/runtime), [`client-ui-primitives`](../packages/client/ui-primitives), [`client-ui-slots`](../packages/client/ui-slots), [`invariants`](../packages/support/invariants) |
|
||||
| [`client-ui-trajectory`](../packages/client/ui-trajectory) | `client` | [`client-runtime`](../packages/client/runtime), [`client-ui-primitives`](../packages/client/ui-primitives), [`invariants`](../packages/support/invariants) |
|
||||
| [`code-runtime-python`](../packages/code-runtime/code-runtime-python) | `code-runtime` | [`code-runtime`](../packages/code-runtime/code-runtime), [`invariants`](../packages/support/invariants) |
|
||||
| [`credentials`](../packages/credentials/credentials) | `credentials` | [`brand`](../packages/util/brand), [`invariants`](../packages/support/invariants) |
|
||||
| [`frontend-static`](../packages/host/frontend-static) | `host` | [`host-webserver`](../packages/host/webserver), [`invariants`](../packages/support/invariants) |
|
||||
| [`helper`](../packages/sdk/helper) | `sdk` | [`brand`](../packages/util/brand), [`invariants`](../packages/support/invariants), [`subprocess`](../packages/subprocess/subprocess) |
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/code-runtime/code-runtime-python/README.md
|
||||
README.md: 606d153eb899925ae274133b3728d317607a17e2
|
||||
README.zh.md: f4387d6b20994781b8e64d31da7319badb56aacd
|
||||
README.md: e7ca08e3e42d368e1b46f4dd52ae7dc1040e6d76
|
||||
README.zh.md: df0488b469daca6f33254ce2233b10e0c7138ed5
|
||||
|
||||
@@ -12,7 +12,7 @@ The host and the CPython subprocess exchange a versionless, JSON-lines protocol
|
||||
|
||||
- **fd 3, not stdout** — Node pins the channel positionally with `stdio: ['pipe','pipe','pipe','pipe']`; the Python bootstrap reads the same `PROTOCOL_FD` constant. JSON-lines framing.
|
||||
- **Host treats every inbound frame as hostile** — model code has full access to fd 3 and can post anything through it, so `validateChildFrame` shape-validates and REBUILDS each frame before the host reads it: forged extra fields never ride along, a non-number call id can never be echoed into a reply, and junk drops to `undefined` rather than throwing in the host's message handler. The Python side trusts host replies (the host is not model-controlled).
|
||||
- **Lossless-JSON crossing** — completion values and binding arguments cross as exact JSON. `encodeJsonPlain` serializes a `JSON.parse`-produced value without recursion, so a deep value below the byte budget crosses intact instead of dying on `JSON.stringify`'s stack limit; `checkDoneValue` meters a forged completion value's byte length AND number losslessness in one traversal that rejects an over-budget payload before the incremental work it would add (a non-allocating escaped-size scan, then enqueued children) — the frame's own width is already parsed and capped upstream by the host's fd-3 receive buffer, not re-bounded here; `hasUnsafeIntegerToken` reads the raw frame text to catch an integer token that `JSON.parse` would silently round; `hasNonLosslessNumber` rejects a non-finite or negative-zero number in unbounded `call.args`. Beyond-safe-range integral doubles serialize through `BigInt` digits so the exact integer crosses, not the rounded `String()` form.
|
||||
- **Lossless-JSON crossing** — completion values and binding arguments cross as exact JSON. `encodeJsonPlain` serializes a `JSON.parse`-produced value without recursion, so a deep value below the byte budget crosses intact instead of dying on `JSON.stringify`'s stack limit; `checkDoneValue` meters a forged completion value's byte length AND number losslessness in one traversal that rejects an over-budget payload before the incremental work it would add (the enqueued children; strings and keys are metered by a non-allocating escaped-size scan, so the escaped copy is never materialized) — the frame's own width is already parsed and capped upstream by the host's fd-3 receive buffer, not re-bounded here; `hasUnsafeIntegerToken` reads the raw frame text to catch an integer token that `JSON.parse` would silently round; `hasNonLosslessNumber` rejects a non-finite or negative-zero number in unbounded `call.args`. Beyond-safe-range integral doubles serialize through `BigInt` digits so the exact integer crosses, not the rounded `String()` form.
|
||||
- **Shared truncation marker** — `logTruncationMarker(maxBytes)` produces byte-identical text on both sides, so a truncated log run reads the same however the cap was hit. The `log` frame's `truncated` flag distinguishes the child ledger's own marker from program output.
|
||||
|
||||
## Model Experience
|
||||
|
||||
@@ -12,7 +12,7 @@ host 与 CPython 子进程在子进程的 fd 3 上交换一个无版本号的 JS
|
||||
|
||||
- **fd 3，而非 stdout** —— Node 通过 `stdio: ['pipe','pipe','pipe','pipe']` 按位置钉住通道；Python bootstrap 读取相同的 `PROTOCOL_FD` 常量。JSON-lines 帧。
|
||||
- **host 把每个入站帧当作敌意输入** —— 模型代码对 fd 3 有完全访问权、可通过它发送任意内容，所以 `validateChildFrame` 在 host 读取前对每个帧做形状校验并重建：伪造的额外字段绝不随行，非数字的 call id 绝不会被回显进 reply，垃圾降为 `undefined` 被丢弃，而不是在 host 的 message handler 里抛错。Python 侧信任 host 回复（host 不受模型控制）。
|
||||
- **lossless-JSON 穿越** —— 完成值与 binding 参数以精确 JSON 穿越。`encodeJsonPlain` 无递归地序列化一个 `JSON.parse` 产出的值，使低于字节预算的深层值能完整穿越，而不是死在 `JSON.stringify` 的栈限制上；`checkDoneValue` 在一次遍历中同时计量伪造完成值的字节长度与数字无损性，在它本会新增的增量工作之前就拒绝超预算 payload（先做非分配的转义尺寸扫描，再入栈子节点）——帧自身的宽度已被上游 `JSON.parse` 支付、由 host 的 fd-3 接收缓冲封顶，并非在此重新约束；`hasUnsafeIntegerToken` 读取原始帧文本，捕获 `JSON.parse` 会静默舍入的整数 token；`hasNonLosslessNumber` 拒绝无字节上限的 `call.args` 中的非有限数或负零。超出安全范围的整数型 double 通过 `BigInt` 数字序列化，穿越的是精确整数而非 `String()` 的舍入形式。
|
||||
- **lossless-JSON 穿越** —— 完成值与 binding 参数以精确 JSON 穿越。`encodeJsonPlain` 无递归地序列化一个 `JSON.parse` 产出的值，使低于字节预算的深层值能完整穿越，而不是死在 `JSON.stringify` 的栈限制上；`checkDoneValue` 在一次遍历中同时计量伪造完成值的字节长度与数字无损性，在它本会新增的增量工作之前就拒绝超预算 payload（即入栈子节点；字符串与 key 由非分配的转义尺寸扫描计量，从不物化转义副本）——帧自身的宽度已被上游 `JSON.parse` 支付、由 host 的 fd-3 接收缓冲封顶，并非在此重新约束；`hasUnsafeIntegerToken` 读取原始帧文本，捕获 `JSON.parse` 会静默舍入的整数 token；`hasNonLosslessNumber` 拒绝无字节上限的 `call.args` 中的非有限数或负零。超出安全范围的整数型 double 通过 `BigInt` 数字序列化，穿越的是精确整数而非 `String()` 的舍入形式。
|
||||
- **共享截断标记** —— `logTruncationMarker(maxBytes)` 在两侧产出逐字节一致的文本，使被截断的日志运行无论从哪侧触达上限都读起来一致。`log` 帧的 `truncated` 标志把子进程 ledger 自身的标记与程序输出区分开。
|
||||
|
||||
## Model Experience
|
||||
|
||||
@@ -205,12 +205,12 @@ type RosterMessageFrames = Exclude<WireFrameShapes[keyof WireFrameShapes], Names
|
||||
* `false`, failing the assignment below. Type-only; the `const`s emit nothing
|
||||
* meaningful at runtime.
|
||||
*/
|
||||
type UnionCoversRoster = [MessageFrames] extends [RosterMessageFrames] ? true : false
|
||||
type RosterCoversUnion = [RosterMessageFrames] extends [MessageFrames] ? true : false
|
||||
const _unionCoversRoster: UnionCoversRoster = true
|
||||
const _rosterCoversUnion: RosterCoversUnion = true
|
||||
void _unionCoversRoster
|
||||
void _rosterCoversUnion
|
||||
type UnionSubsetOfRoster = [MessageFrames] extends [RosterMessageFrames] ? true : false
|
||||
type RosterSubsetOfUnion = [RosterMessageFrames] extends [MessageFrames] ? true : false
|
||||
const _unionSubsetOfRoster: UnionSubsetOfRoster = true
|
||||
const _rosterSubsetOfUnion: RosterSubsetOfUnion = true
|
||||
void _unionSubsetOfRoster
|
||||
void _rosterSubsetOfUnion
|
||||
|
||||
/**
|
||||
* Each frame's wire fields tagged by required/optional, keyed by field name so
|
||||
|
||||
@@ -20,6 +20,9 @@ import { logTruncationMarker, PROTOCOL_FD, WIRE_FRAME_FIELDS } from '../src/prot
|
||||
|
||||
const execFileAsync = promisify(execFile)
|
||||
const pyDir = fileURLToPath(new URL('../py', import.meta.url))
|
||||
// `-B` blocks bytecode writes into the source tree (`py/__pycache__/*.pyc`);
|
||||
// `-I` isolates the interpreter but does not imply it.
|
||||
const python3Flags = ['-I', '-B']
|
||||
|
||||
async function hasPython3(): Promise<boolean> {
|
||||
try {
|
||||
@@ -45,7 +48,7 @@ describe.skipIf(!python3Available)('protocol.py mirrors protocol.ts at runtime',
|
||||
' "markers": [log_truncation_marker(b) for b in budgets],',
|
||||
'}))',
|
||||
].join('\n')
|
||||
const { stdout } = await execFileAsync('python3', ['-I', '-B', '-c', probe])
|
||||
const { stdout } = await execFileAsync('python3', [...python3Flags, '-c', probe])
|
||||
const seen = JSON.parse(stdout) as { fd: number; markers: string[] }
|
||||
// Assert against the TS-side PROTOCOL_FD export (the value the host wires),
|
||||
// not a bare literal, so a drift on either side of the wire is caught here.
|
||||
@@ -75,7 +78,7 @@ describe.skipIf(!python3Available)('protocol.py mirrors protocol.ts at runtime',
|
||||
+ ' if not n.startswith("_") and hasattr(v, "__required_keys__")}',
|
||||
'print(json.dumps(frames))',
|
||||
].join('\n')
|
||||
const { stdout } = await execFileAsync('python3', ['-I', '-B', '-c', probe])
|
||||
const { stdout } = await execFileAsync('python3', [...python3Flags, '-c', probe])
|
||||
const seen = JSON.parse(stdout) as Record<string, { required: string[]; optional: string[] }>
|
||||
// Normalize the TS source of truth to the same sorted shape Python reports.
|
||||
const expected = Object.fromEntries(
|
||||
|
||||
Reference in New Issue
Block a user