Agent Note: the code-runtime-python fd-3 frame protocol

Status: implemented

English | 中文

Problem

@deepseek-ai/dsh-code-runtime-python owns the wire protocol intended for a CPython code-runtime provider. Such a provider runs each model program in a fresh python3 -I subprocess and bridges binding calls and completion values over the child's fd 3. The host cannot trust that channel: model code has full access to fd 3 and can forge any frame, so every inbound frame is hostile input that the host must validate and rebuild before reading. The protocol also has to carry lossless JSON without the depth limit JSON.stringify and json.dumps impose, because the seam's CodeJsonValue is depth-unbounded.

The package ships the protocol independently from a runtime implementation. It exports no PythonCodeRuntime, subprocess path, or Python-side JSON codec; those remain work for a future provider. The protocol builds on the portable identifier seam.

Decision

src/protocol.ts is the host side of the wire vocabulary and its hostile-frame codec:

validateChildFrame shape-validates and REBUILDS every inbound frame. The compile-time union means nothing on fd 3 — a forged frame can carry null, poisoned fields, or omit required ones — so each accepted frame is reconstructed field by field: forged extras never ride along, a non-finite call id can never be echoed into a reply, and junk returns undefined to be dropped rather than throwing in the host's message handler.
encodeJsonPlain / checkDoneValue / hasUnsafeIntegerToken / hasNonLosslessNumber are the lossless-JSON codec and meters. They traverse iteratively (an explicit stack, not recursion) so a deep value below the byte budget crosses intact; checkDoneValue folds byte-metering and number-losslessness into one walk that rejects an over-budget payload before the incremental work it would otherwise add — the enqueued children; strings and keys are metered by a non-allocating escaped-size scan (jsonStringBytesUpTo), so the escaped copy is never materialized. It does not re-bound the frame's own width: done.value is already JSON.parse'd when the check runs, so a consuming runtime must cap fd-3 bytes before parsing. Beyond-safe-range integral doubles serialize through BigInt digits so the exact integer crosses, not String()'s rounded form.
logTruncationMarker produces the in-band marker text a log ledger emits when it exhausts its byte budget.

py/protocol.py mirrors the message shapes as TypedDicts and re-declares the two surfaces both sides EXECUTE against — PROTOCOL_FD = 3 and log_truncation_marker — with byte-identical text.

The package remains independently buildable with protocol-only exports. check-workspace-constraints reads every packages/<group>/<pkg>/package.json unconditionally, while the coverage and invariant-topology checks exercise the package as soon as its directory exists.

Wire contract

Frames are JSON-lines on fd 3, one object per line, leaving stdout/stderr free for the program's own output. Child → host: boot-ack, call, log, done. Host → child: boot (first frame), run (after boot-ack), and one reply per call. The log frame's truncated flag marks the frame that IS the child ledger's own truncation marker, so the host stops capturing at the same point the child did instead of inferring it from its own budget. done.error.kind is one of exception, invalid-output, output-limit; wall/CPU budgets, aborts, and substrate death are observed host-side, not carried as frames.

Mirror alignment

py/protocol.py and src/protocol.ts agree that LogMessage carries truncated, DoneMessage.error carries kind, and Namespace may carry errorClass. tests/protocol-mirror.e2e.ts spawns a real python3 and asserts PROTOCOL_FD, log_truncation_marker, and each TypedDict's required and optional wire field sets against src/protocol.ts. A renamed or dropped field, or a required/optional mismatch, fails the test. Field types are not compared across the language boundary; review and a future provider's real-subprocess suite own that gap.

Alternatives considered

Require a future Python JSON codec (_encode_json_plain / _decode_json_plain) to live in py/protocol.py for cross-side symmetry with protocol.ts. Rejected. The repository's "prefer symmetry for parallel values" rule points at genuinely parallel values; these are not. The host-side codec in protocol.ts validates hostile input and is self-contained. A child-side codec would produce trusted output and belong with bootstrap-owned emission and cost accounting; forcing only its entry points into protocol.py would couple the vocabulary mirror to runtime internals or create an import cycle. protocol.py remains a pure wire-vocabulary mirror. No Python codec ships in this package.

Keep the protocol files outside a buildable package until a runtime ships. Rejected: the workspace-constraint, coverage, and invariant-topology checks require every directory under packages/<group>/<pkg> to be a buildable package, and the protocol has independent tests and a public wire vocabulary.

Consequences

Bought: the fd-3 protocol and its hostile-input codec form a self-contained, fully unit-covered layer, with an executing guard against TypeScript/Python field-set drift. A future runtime can consume a reviewed wire contract.

Cost: the package name denotes a Python runtime family while src/index.ts exports only the protocol vocabulary. The mirror e2e compares field names and required/optional status across the two sides but not field types; comparing type declarations across TypeScript and Python has no mechanical equivalent, so review and the future runtime's real-subprocess suite retain that responsibility.

5.8 KiB Raw Permalink Blame History