Tianyi Cui
3c1c6a89b1
* test(python): exercise installed wheels as black boxes Add an installed-wheel mode that refuses source/editable imports, repository working directories, mismatched SDK/runtime versions, unpinned runtime dependencies, and executables outside the installed runtime distribution. The mode resolves the wheel-owned executable itself, so callers cannot accidentally prove an explicit checkout artifact. Add a real-API scenario that drives two tool-using turns through the public synchronous SDK, verifies the file bytes outside the agent, checks completed turn/tool events and persistence, and projects provider failures without retaining credential-bearing error text. The existing deterministic scenario set remains the keyless behavior oracle. Refs #2952. * ci(python): require installed-wheel checks on every release target Move the complete deterministic runtime scenarios behind construction and clean installation of the SDK and matching runtime wheels. Each native leg runs outside the checkout with source-resolution environment variables removed; Linux manylinux smokes assert the same installed provenance. Expand the required pull-request call from Linux x64 to Linux x64, Linux arm64, and macOS arm64. Trusted heads receive only DEEPSEEK_API_KEY_EXTERNAL for a fail-loud live two-turn smoke on each carrier, while fork and Dependabot heads retain the full keyless path without exposing secrets. Pin the reusable secret declaration, matrix call, aggregate dependency, untrusted-head condition, and live/keyless commands in the workflow contract test. Refs #2952. * docs(testing): make installed wheels the Python CI authority Record the clean-wheel provenance boundary, complete keyless scenario set, trusted real-API contract, secret handling, and three-target required topology in a new implemented testing decision. Update the SEA distribution and portable-CI authorities plus the Python contributor reference to describe the same current state. Archive the fully superseded Linux-x64-only decision after consolidating its rationale and alternatives into the new owner. Preserve its bilingual triplet as a sealed historical snapshot and redirect every active current-state reference. Refs #2952.
3.8 KiB
Agent Note: Required Python runtime pull-request validation
Status: implemented Archived: 2026-08-23
English | 中文
Problem
Ordinary pull-request CI runs the complete Python SDK pytest suite against fake runtime peers, while Node snapshots exercise different clients and expected outputs. The real Python client, packaged JSON-RPC executable, executable-specific snapshot, release-shaped wheels, and clean installation meet only in the optional single-executable or Python release workflows. A runtime event change or closure change can therefore merge with a stale Python projection or broken wheel path and fail only when someone later builds a Python release candidate.
Decision
Every pull request has a required python-runtime job in CI. It calls the shared single-executable builder for node24-linux-x64 without a path filter and participates in all checks passed. The called workflow builds the real executable, runs all keyless Python full-turn and direct-binary scenarios including both committed snapshots, builds the SDK and runtime wheels, installs them into a clean virtual environment, checks the executable and native addon's GLIBC requirements, and runs the installed wheels in a manylinux 2.28 container.
The required job and the Python publication workflow use the same builder. Its concurrency key includes the caller workflow, so required CI and an explicit full release validation for the same ref do not cancel each other. The complete linux-x64, linux-arm64, and macos-arm64 matrix remains a release validation because platform-independent runtime, SDK, and snapshot behavior needs one merge-blocking native carrier, while architecture-specific executable, addon, wheel-tag, and deployment-target behavior still needs all release targets before publication.
The advanced executable snapshot normalizes opaque session, message, subagent, and workflow-run identifiers before comparison. A newly persisted workflow event therefore changes the reviewed expected output without making a random run identifier part of that output. The minimal scenario's model-visible snapshot covers the assembled system prompt, tool schemas, and message list that this one tokenizes.
Alternatives considered
Run the complete native matrix on every pull request. This duplicates platform-independent full-turn and snapshot behavior across three jobs and consumes ARM64 Linux and macOS capacity on every change. The publication workflow retains that evidence at the point where all three artifacts are required.
Run the snapshot against the development Node carrier. This catches protocol and event projection drift but does not prove pkg assembly, the deployed runtime closure, native addon staging, wheel construction, exact dependency pins, or clean installation. The required Linux executable path covers the published path directly.
Select the job with path filters or labels. Python behavior depends on shared agent, session, workflow, subagent, plugin-loading, and packaging code outside python/. An incomplete dependency filter recreates the delayed failure, and a label leaves the evidence optional.
Consequences
Every pull request pays for one standard-hosted Linux executable and wheel build, and all checks passed waits for it. This makes the first-party Python distribution a merge-time contract and reuses the release implementation instead of maintaining a smaller substitute pipeline.
One required architecture cannot detect macOS or Linux ARM64 packaging regressions. Explicit full release validation remains mandatory before publication and owns those platform-specific results.