mirror of
https://github.com/deepseek-ai/deepseek-harness.git
synced 2026-08-29 04:26:38 +00:00
test(python): pin the minimal composition's model-visible surface
The Python lane never compared what the minimal composition shows the model: the mock model only asserted system-role messages, and the advanced snapshot tokenizes the assembled system prompt and tool schemas. The sdk-minimal scenario now records model-visible.json — every model request's advertised tool schemas verbatim and its message list, with system and user text kept and assistant/tool payloads reduced to call identity so the expected output replays on macOS and Linux. It excludes the dynamic runtime-context snapshot, which the same composition emits on macOS and not on Linux (#2488). AGENTS.md and the testing policy name both SDKs as independent projections of the agent loop, session lifecycle, and SessionEventMap.
This commit is contained in:
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write python/development.md
|
||||
development.md: 31dc254b58c05c2a19c7c4cd5dc1e53207517902
|
||||
development.zh.md: dbb85a0cffc5e06c7ca781b2b01f6993395204e3
|
||||
development.md: fe62a109f2643afe0b9be1ed51b86be0b9fa731f
|
||||
development.zh.md: d4ab9850d6c83dc17a740240f97cef89d61faaa0
|
||||
|
||||
@@ -27,6 +27,15 @@ uv run --project python/sdk pytest
|
||||
|
||||
`python/sdk/tests/test_bundled_runtime.py` exercises available bundled carriers and skips a carrier when its artifact has not been built. For repository-wide test policy, see [Testing](../docs/testing.md).
|
||||
|
||||
That suite drives fake runtime peers. `scripts/smoke-python-runtime.py` drives the real packaged runtime instead, and the required `python-runtime` CI job runs every scenario against a freshly built executable:
|
||||
|
||||
```sh
|
||||
uv run --project python/sdk python scripts/smoke-python-runtime.py \
|
||||
--scenario sdk-minimal --exe dist-exe/dsh-jsonrpc-agent-pkg-macos-arm64
|
||||
```
|
||||
|
||||
Two scenarios compare committed expected output under `scripts/snapshots/python-sdk-single-exe/`. `minimal/model-visible.json` pins the checked-in minimal composition's assembled system prompts, advertised tool schemas, and model-visible messages, so a plugin that contributes an unintended system section or user message fails the job; it drops the dynamic runtime-context snapshot, which the same composition emits on macOS and not on Linux ([#2488](https://github.com/deepseek-harness/deepseek-harness/issues/2488)). `advanced/` pins the SDK result and the persisted session logs. Rerun the owning scenario with `--update-snapshots` and review that diff before committing it.
|
||||
|
||||
An interactive smoke test needs `DEEPSEEK_API_KEY` in the environment or repository-root `.env`:
|
||||
|
||||
```python
|
||||
|
||||
@@ -27,6 +27,15 @@ uv run --project python/sdk pytest
|
||||
|
||||
`python/sdk/tests/test_bundled_runtime.py` 会运行可用的内置载体;某个载体的产物尚未构建时,会跳过该载体。仓库级测试政策见 [测试](../docs/testing.md)。
|
||||
|
||||
该套件面向的是伪造的运行时对端。`scripts/smoke-python-runtime.py` 面向真实的打包运行时;必需的 `python-runtime` CI 任务会用新构建的可执行文件运行全部场景:
|
||||
|
||||
```sh
|
||||
uv run --project python/sdk python scripts/smoke-python-runtime.py \
|
||||
--scenario sdk-minimal --exe dist-exe/dsh-jsonrpc-agent-pkg-macos-arm64
|
||||
```
|
||||
|
||||
其中两个场景会比对 `scripts/snapshots/python-sdk-single-exe/` 下已提交的期望输出。`minimal/model-visible.json` 固定了签入的极简组合所组装的系统提示词、对外公布的工具 schema 以及模型可见消息,因此插件一旦贡献出计划外的系统分段或 user 消息,该任务即失败;它会丢弃动态运行时上下文快照——同一组合在 macOS 上会发出它,在 Linux 上不会([#2488](https://github.com/deepseek-harness/deepseek-harness/issues/2488))。`advanced/` 固定 SDK 结果与持久化的会话日志。重新运行对应场景时加上 `--update-snapshots`,并在提交前审阅该差异。
|
||||
|
||||
交互式冒烟测试需要环境变量或仓库根目录 `.env` 中存在 `DEEPSEEK_API_KEY`:
|
||||
|
||||
```python
|
||||
|
||||
Reference in New Issue
Block a user