mirror of
https://github.com/deepseek-ai/deepseek-harness.git
synced 2026-08-30 04:40:37 +00:00
feat(code-runtime-python): make the TypedDict wire mirror an executable gate
Address the two standing review suggestions in this layer rather than deferring them to PR #4: - Extend tests/protocol-mirror.e2e.ts to read each py/protocol.py TypedDict's required/optional key set and assert it against the wire field names src/protocol.ts declares (global included, via functional TypedDict). The round-12 class of drift — a renamed/dropped field, or one side making a field optional the other requires — now fails a test instead of relying on review. Field types remain review-guarded (no mechanical TS/Python equivalent). - Drop the forward references to PR #4's internal mechanisms from this layer's prose: the "256 MiB frame ceiling" figure and the "(index.ts)" fd-3 pinning citation become an abstract "host-side inbound frame-size cap" so the JSDoc, spec, README, and Agent Note describe only what this layer owns. Update both README sides and the Agent Note (both languages) to state the mirror is now executable, and re-record their i18n pairings.
This commit is contained in:
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/code-runtime/code-runtime-python/README.md
|
||||
README.md: 62e899941ac8eadb2046b1bffae0a3c9c6e14308
|
||||
README.zh.md: 5a7aa880830bccc4ebd647d7b009ec7e4f0d6307
|
||||
README.md: d0491c478d04a8436199bc23fd79917e8c019b1c
|
||||
README.zh.md: 63d7d38ee05b561e60a3adf387c5c1292c37a7a2
|
||||
|
||||
@@ -25,5 +25,5 @@ No direct invalidation; the named consumer owns any request-prefix changes.
|
||||
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
- **The cross-language guard covers only the two runtime-executed surfaces** — `PROTOCOL_FD` and the log truncation marker. The `TypedDict` frame shapes in `py/protocol.py` mirror `src/protocol.ts` by review, not by an automated check: comparing type declarations across TypeScript and Python has no mechanical equivalent here, so a future shape drift is caught by review plus the backend's real-subprocess suite rather than this package's tests.
|
||||
- **The cross-language guard covers the runtime-executed surfaces and the frame field shapes** — `tests/protocol-mirror.e2e.ts` spawns a real `python3` and asserts, against `src/protocol.ts`, both `PROTOCOL_FD` / the log truncation marker text AND each `TypedDict`'s required/optional wire field set in `py/protocol.py`. What it does not compare is the field *types* (e.g. that `cpuSeconds` is an `int` on both sides): comparing type declarations across TypeScript and Python has no mechanical equivalent here, so a type-level drift is still caught by review plus the backend's real-subprocess suite rather than this package's tests.
|
||||
- **The `PythonCodeRuntime` implementation and its Python-side JSON codec are not in this layer** — they ship in the backend-core PR on top of this branch; `src/index.ts` re-exports only the protocol vocabulary until then.
|
||||
|
||||
@@ -25,5 +25,5 @@ host 与 CPython 子进程在子进程的 fd 3 上交换一个无版本号的 JS
|
||||
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
- **跨语言 guard 只覆盖两个运行时执行的面** —— `PROTOCOL_FD` 与日志截断标记。`py/protocol.py` 中的 `TypedDict` 帧形状靠 review 而非自动化检查来镜像 `src/protocol.ts`:跨 TypeScript 与 Python 比较类型声明在此无机械等价物,故未来的形状漂移由 review 加后端真子进程套件捕获,而非本包的测试。
|
||||
- **跨语言 guard 覆盖运行时执行的面与帧字段形状** —— `tests/protocol-mirror.e2e.ts` 启动一个真实 `python3`,对照 `src/protocol.ts` 断言 `PROTOCOL_FD` / 日志截断标记文本,以及 `py/protocol.py` 中每个 `TypedDict` 的必填/可选 wire 字段集。它不比较字段的*类型*(例如 `cpuSeconds` 两侧都是 `int`):跨 TypeScript 与 Python 比较类型声明在此无机械等价物,故类型级漂移仍由 review 加后端真子进程套件捕获,而非本包的测试。
|
||||
- **`PythonCodeRuntime` 实现与 Python 侧 JSON codec 不在本层** —— 它们在基于本分支的 backend-core PR 中交付;在那之前 `src/index.ts` 只 re-export 协议词汇。
|
||||
|
||||
@@ -7,8 +7,9 @@
|
||||
*/
|
||||
|
||||
// The protocol channel is fd 3 from the child's perspective — the host pins it
|
||||
// positionally via `stdio: ['pipe','pipe','pipe','pipe']` (index.ts), and the
|
||||
// Python bootstrap reads the same constant from its own protocol.py.
|
||||
// positionally via `stdio: ['pipe','pipe','pipe','pipe']` when it spawns the
|
||||
// child, and the Python bootstrap reads the same constant from its own
|
||||
// protocol.py.
|
||||
|
||||
/**
|
||||
* What the host sends immediately after spawn, as the first line on fd 3. The
|
||||
@@ -194,12 +195,13 @@ function scalarJson(current: unknown): string {
|
||||
* crossed. This bounds the INCREMENTAL allocation the check itself would add on
|
||||
* top of the already-parsed value — the escaped-string copy, the enqueued
|
||||
* children, the per-key `JSON.stringify` — not the parse that produced `value`.
|
||||
* That upstream width is bounded separately: the host reads fd 3 into a fixed
|
||||
* 256 MiB receive buffer (a later stack layer), so `value` cannot already be
|
||||
* larger than that when it reaches here, while `maxValueBytes` defaults to
|
||||
* 32 KiB. The traversal rejects over-budget BEFORE materializing a string's
|
||||
* escaped form or enqueuing an array's/object's children, so a below-ceiling
|
||||
* forgery cannot force those secondary allocations. Object key COUNTING is
|
||||
* That upstream width is bounded separately, by the host-side cap on inbound
|
||||
* fd-3 frame size before `JSON.parse` runs (owned by the runtime that reads the
|
||||
* channel), so `value` cannot be arbitrarily large when it reaches here, while
|
||||
* `maxValueBytes` defaults to 32 KiB. The traversal rejects over-budget BEFORE
|
||||
* materializing a string's escaped form or enqueuing an array's/object's
|
||||
* children, so a forgery within that frame cap cannot force those secondary
|
||||
* allocations. Object key COUNTING is
|
||||
* unavoidably O(keys) — JS has no lazy own-key iterator, and the parse already
|
||||
* built the key set — but the check still refuses the per-entry work before the
|
||||
* enqueue loop. A non-lossless number (non-finite, negative zero) is caught only
|
||||
@@ -353,7 +355,8 @@ function* ownValues(record: object): Generator {
|
||||
* cap, so there is no budget to reject a wide payload against the way
|
||||
* {@link checkDoneValue} does. The traversal therefore holds ONE cursor per
|
||||
* NESTING LEVEL (an array or {@link ownValues} iterator) instead of one entry
|
||||
* per member: a forged flat `args` just below the 256 MiB frame ceiling would
|
||||
* per member: a forged flat `args` at the top of the host's inbound frame-size
|
||||
* cap would
|
||||
* otherwise push tens of millions of stack entries — and `Object.values` would
|
||||
* copy each object's full breadth — allocating hundreds of megabytes beyond
|
||||
* what `JSON.parse` already holds. Iterative either way, so a deep frame
|
||||
|
||||
@@ -6,13 +6,14 @@ import { describe, expect, it } from 'vitest'
|
||||
import { logTruncationMarker } from '../src/protocol.ts'
|
||||
|
||||
/**
|
||||
* Cross-language mirror check for the two protocol surfaces the host and the
|
||||
* CPython subprocess share at runtime, spawning a real `python3` to read them
|
||||
* from `py/protocol.py`. `src/protocol.ts` and `py/protocol.py` declare the same
|
||||
* frame vocabulary on two sides of the wire; the only values both sides EXECUTE
|
||||
* against are `PROTOCOL_FD` (the fd the channel is pinned to) and the log
|
||||
* truncation marker text (emitted verbatim by whichever ledger exhausts first),
|
||||
* so a drift there silently corrupts a live run. Self-skips when no `python3` is
|
||||
* Cross-language mirror check between `src/protocol.ts` and `py/protocol.py`,
|
||||
* spawning a real `python3` to read the Python side. Two things are asserted:
|
||||
* the runtime surfaces both sides EXECUTE against — `PROTOCOL_FD` and the log
|
||||
* truncation marker text, where a drift silently corrupts a live run — and the
|
||||
* per-frame wire field sets (required/optional keys of each `TypedDict`), which
|
||||
* turns the otherwise review-only shape mirror into an executable check that
|
||||
* catches the round-12 kind of drift (a renamed/dropped field, or one side
|
||||
* making a field optional the other requires). Self-skips when no `python3` is
|
||||
* on PATH — CI provides one; the pure-TS `protocol.spec.ts` covers the host
|
||||
* codec unconditionally.
|
||||
*/
|
||||
@@ -46,10 +47,53 @@ describe.skipIf(!python3Available)('protocol.py mirrors protocol.ts at runtime',
|
||||
].join('\n')
|
||||
const { stdout } = await execFileAsync('python3', ['-I', '-c', probe])
|
||||
const seen = JSON.parse(stdout) as { fd: number; markers: string[] }
|
||||
// fd 3 is the wire contract, not a tunable: index.ts pins it positionally.
|
||||
// fd 3 is the wire contract, not a tunable: the host pins it positionally
|
||||
// when it spawns the child.
|
||||
expect(seen.fd).toBe(3)
|
||||
expect(seen.markers).toEqual(budgets.map(budget => logTruncationMarker(budget)))
|
||||
})
|
||||
|
||||
it('agrees on every frame type\'s wire field set between the TS and Python declarations', async () => {
|
||||
// Turn the TypedDict mirror from a review-only obligation into an executable
|
||||
// check: read each Python TypedDict's required/optional key sets and assert
|
||||
// them against the wire field names the TS side declares. `global` is the
|
||||
// reserved-keyword key the Python side carries via functional TypedDict —
|
||||
// catching exactly the round-12 kind of drift (a renamed/dropped field, an
|
||||
// optional field the other side made required).
|
||||
const probe = [
|
||||
'import json, sys',
|
||||
`sys.path.insert(0, ${JSON.stringify(pyDir)})`,
|
||||
'import protocol as p',
|
||||
'def keys(td): return {"required": sorted(td.__required_keys__), "optional": sorted(td.__optional_keys__)}',
|
||||
'print(json.dumps({',
|
||||
' "BootMessage": keys(p.BootMessage),',
|
||||
' "Namespace": keys(p.Namespace),',
|
||||
' "RunMessage": keys(p.RunMessage),',
|
||||
' "BootAckMessage": keys(p.BootAckMessage),',
|
||||
' "CallMessage": keys(p.CallMessage),',
|
||||
' "LogMessage": keys(p.LogMessage),',
|
||||
' "DoneErrorField": keys(p.DoneErrorField),',
|
||||
' "DoneMessage": keys(p.DoneMessage),',
|
||||
' "ErrorClass": keys(p.ErrorClass),',
|
||||
'}))',
|
||||
].join('\n')
|
||||
const { stdout } = await execFileAsync('python3', ['-I', '-c', probe])
|
||||
const seen = JSON.parse(stdout) as Record<string, { required: string[]; optional: string[] }>
|
||||
// The wire field sets each frame carries, mirroring src/protocol.ts. `global`
|
||||
// is the JSON key `CallMessage`/`Namespace` send (a Python keyword, declared
|
||||
// functionally on the Python side).
|
||||
expect(seen).toEqual({
|
||||
BootMessage: { required: ['addressSpaceBytes', 'cpuSeconds', 'maxLogBytes', 'maxValueBytes', 'namespaces', 'type'], optional: [] },
|
||||
Namespace: { required: ['global', 'names'], optional: ['errorClass'] },
|
||||
RunMessage: { required: ['program', 'type'], optional: [] },
|
||||
BootAckMessage: { required: ['type'], optional: [] },
|
||||
CallMessage: { required: ['args', 'global', 'id', 'name', 'type'], optional: [] },
|
||||
LogMessage: { required: ['text', 'type'], optional: ['truncated'] },
|
||||
DoneErrorField: { required: ['kind', 'message'], optional: [] },
|
||||
DoneMessage: { required: ['type'], optional: ['error', 'value'] },
|
||||
ErrorClass: { required: ['memberNameProperty', 'name'], optional: [] },
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
it('names the py/ directory that ships with the package', () => {
|
||||
|
||||
@@ -135,8 +135,8 @@ describe('lossless-number scan', () => {
|
||||
it('walks wide arrays and objects one member at a time', () => {
|
||||
// `call.args` carries no seam byte cap, so a wide forged payload has no
|
||||
// budget to be rejected against — the walk must hold one cursor per
|
||||
// NESTING LEVEL, not one entry per member, or a flat payload just below
|
||||
// the 256 MiB frame ceiling would allocate tens of millions of stack
|
||||
// NESTING LEVEL, not one entry per member, or a flat payload at the top of
|
||||
// the host's inbound frame-size cap would allocate tens of millions of stack
|
||||
// entries (and `Object.values` a second full-breadth copy). Observable
|
||||
// through the boundary: a wide payload whose per-member cost the old shape
|
||||
// would have paid still scans, and a violation ANYWHERE in it is found
|
||||
|
||||
Reference in New Issue
Block a user