mirror of
https://github.com/deepseek-ai/deepseek-harness.git
synced 2026-08-29 04:26:38 +00:00
test(python): pin the minimal composition's model-visible surface
The Python lane never compared what the minimal composition shows the model: the mock model only asserted system-role messages, and the advanced snapshot tokenizes the assembled system prompt and tool schemas. The sdk-minimal scenario now records model-visible.json — every model request's advertised tool schemas verbatim and its message list, with system and user text kept and assistant/tool payloads reduced to call identity so the expected output replays on macOS and Linux. It excludes the dynamic runtime-context snapshot, which the same composition emits on macOS and not on Linux (#2488). AGENTS.md and the testing policy name both SDKs as independent projections of the agent loop, session lifecycle, and SessionEventMap.
This commit is contained in:
Yichen Jiang
+2
-2
@@ -2,5 +2,5 @@
|
|||||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||||
# after editing either side, bring the other along and re-record with:
|
# after editing either side, bring the other along and re-record with:
|
||||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/testing/2026-08-12-required-python-runtime-pull-request-ci.md
|
# pnpm run verify-translation-pairing --write .agents/notes/implemented/testing/2026-08-12-required-python-runtime-pull-request-ci.md
|
||||||
2026-08-12-required-python-runtime-pull-request-ci.md: 2f5dcac17262bd885049221620a4d9708b083faa
|
2026-08-12-required-python-runtime-pull-request-ci.md: 61b1e832be6d29eafe5cb304d2bca3f0a59e3d84
|
||||||
2026-08-12-required-python-runtime-pull-request-ci.zh.md: 66c81f70d1bf25355425030883ccc4307efbb1ce
|
2026-08-12-required-python-runtime-pull-request-ci.zh.md: 92bf80688d8d152b26fdd893fe0f0b96553868e9
|
||||||
|
|||||||
+2
-2
@@ -10,11 +10,11 @@ Ordinary pull-request CI runs the complete Python SDK pytest suite against fake
|
|||||||
|
|
||||||
## Decision
|
## Decision
|
||||||
|
|
||||||
Every pull request has a required `python-runtime` job in [CI](../../../../.github/workflows/ci.yml). It calls the shared [single-executable builder](../../../../.github/workflows/build-exe-for-python-sdk.yml) for `node24-linux-x64` without a path filter and participates in `all checks passed`. The called workflow builds the real executable, runs all keyless Python full-turn and direct-binary scenarios including the committed executable snapshot, builds the SDK and runtime wheels, installs them into a clean virtual environment, checks the executable and native addon's GLIBC requirements, and runs the installed wheels in a manylinux 2.28 container.
|
Every pull request has a required `python-runtime` job in [CI](../../../../.github/workflows/ci.yml). It calls the shared [single-executable builder](../../../../.github/workflows/build-exe-for-python-sdk.yml) for `node24-linux-x64` without a path filter and participates in `all checks passed`. The called workflow builds the real executable, runs all keyless Python full-turn and direct-binary scenarios including both committed snapshots, builds the SDK and runtime wheels, installs them into a clean virtual environment, checks the executable and native addon's GLIBC requirements, and runs the installed wheels in a manylinux 2.28 container.
|
||||||
|
|
||||||
The required job and the [Python publication workflow](../process/2026-08-11-python-publication-workflow.md) use the same builder. Its concurrency key includes the caller workflow, so required CI and an explicit full release validation for the same ref do not cancel each other. The complete linux-x64, linux-arm64, and macos-arm64 matrix remains a release validation because platform-independent runtime, SDK, and snapshot behavior needs one merge-blocking native carrier, while architecture-specific executable, addon, wheel-tag, and deployment-target behavior still needs all release targets before publication.
|
The required job and the [Python publication workflow](../process/2026-08-11-python-publication-workflow.md) use the same builder. Its concurrency key includes the caller workflow, so required CI and an explicit full release validation for the same ref do not cancel each other. The complete linux-x64, linux-arm64, and macos-arm64 matrix remains a release validation because platform-independent runtime, SDK, and snapshot behavior needs one merge-blocking native carrier, while architecture-specific executable, addon, wheel-tag, and deployment-target behavior still needs all release targets before publication.
|
||||||
|
|
||||||
The executable snapshot normalizes opaque session, message, subagent, and workflow-run identifiers before comparison. A newly persisted workflow event therefore changes the reviewed expected output without making a random run identifier part of that output.
|
The advanced executable snapshot normalizes opaque session, message, subagent, and workflow-run identifiers before comparison. A newly persisted workflow event therefore changes the reviewed expected output without making a random run identifier part of that output. The minimal scenario's [model-visible snapshot](2026-08-13-python-minimal-model-visible-snapshot.md) covers the assembled system prompt, tool schemas, and message list that this one tokenizes.
|
||||||
|
|
||||||
## Alternatives considered
|
## Alternatives considered
|
||||||
|
|
||||||
|
|||||||
+2
-2
@@ -10,11 +10,11 @@ Status: implemented
|
|||||||
|
|
||||||
## 决策
|
## 决策
|
||||||
|
|
||||||
每个拉取请求都在 [CI](../../../../.github/workflows/ci.yml) 中运行必需的 `python-runtime` 作业。该作业不使用路径过滤，调用共享的[单文件可执行程序构建器](../../../../.github/workflows/build-exe-for-python-sdk.yml)构建 `node24-linux-x64`，并参与 `all checks passed`。被调用的工作流会构建真实可执行文件，运行全部无密钥 Python 完整轮次和直接二进制场景（包括检入的 exe 快照），构建 SDK 与运行时 wheel 包，将二者安装进干净的虚拟环境，检查可执行文件与原生 addon 的 GLIBC 依赖，并在 manylinux 2.28 容器中运行已安装的 wheel 包。
|
每个拉取请求都在 [CI](../../../../.github/workflows/ci.yml) 中运行必需的 `python-runtime` 作业。该作业不使用路径过滤，调用共享的[单文件可执行程序构建器](../../../../.github/workflows/build-exe-for-python-sdk.yml)构建 `node24-linux-x64`，并参与 `all checks passed`。被调用的工作流会构建真实可执行文件，运行全部无密钥 Python 完整轮次和直接二进制场景（包括两份检入的快照），构建 SDK 与运行时 wheel 包，将二者安装进干净的虚拟环境，检查可执行文件与原生 addon 的 GLIBC 依赖，并在 manylinux 2.28 容器中运行已安装的 wheel 包。
|
||||||
|
|
||||||
必需作业与 [Python 发布工作流](../process/2026-08-11-python-publication-workflow.md)共用同一构建器。其并发键包含调用方工作流，因此同一 ref 上的必需 CI 与显式完整发布验证不会互相取消。完整的 linux-x64、linux-arm64 和 macos-arm64 矩阵仍属于发布验证：平台无关的运行时、SDK 与快照行为只需要一个阻断合并的原生载体，而架构相关的可执行文件、addon、wheel 包标签与部署目标行为在发布前仍需要全部发布目标验证。
|
必需作业与 [Python 发布工作流](../process/2026-08-11-python-publication-workflow.md)共用同一构建器。其并发键包含调用方工作流，因此同一 ref 上的必需 CI 与显式完整发布验证不会互相取消。完整的 linux-x64、linux-arm64 和 macos-arm64 矩阵仍属于发布验证：平台无关的运行时、SDK 与快照行为只需要一个阻断合并的原生载体，而架构相关的可执行文件、addon、wheel 包标签与部署目标行为在发布前仍需要全部发布目标验证。
|
||||||
|
|
||||||
exe 快照会在比较前规范化不透明的会话、消息、subagent 和工作流运行标识符。因此，新增的持久化工作流事件会改变经过审阅的预期输出，但不会把随机运行标识符写入其中。
|
进阶 exe 快照会在比较前规范化不透明的会话、消息、subagent 和工作流运行标识符。因此，新增的持久化工作流事件会改变经过审阅的预期输出，但不会把随机运行标识符写入其中。极简场景的[模型可见快照](2026-08-13-python-minimal-model-visible-snapshot.md)覆盖了这份快照所占位化的已组装系统提示词、工具 schema 与消息列表。
|
||||||
|
|
||||||
## 曾考虑的替代方案
|
## 曾考虑的替代方案
|
||||||
|
|
||||||
|
|||||||
+6
@@ -0,0 +1,6 @@
|
|||||||
|
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||||
|
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||||
|
# after editing either side, bring the other along and re-record with:
|
||||||
|
# pnpm run verify-translation-pairing --write .agents/notes/implemented/testing/2026-08-13-python-minimal-model-visible-snapshot.md
|
||||||
|
2026-08-13-python-minimal-model-visible-snapshot.md: 66cfa1ed667d9a60579b0d27ddca2667614d7e1c
|
||||||
|
2026-08-13-python-minimal-model-visible-snapshot.zh.md: 40f596208f07532e68e382013b36e0d7ec46de3d
|
||||||
@@ -0,0 +1,33 @@
|
|||||||
|
# Agent Note: Python minimal-composition model-visible snapshot
|
||||||
|
|
||||||
|
Status: implemented
|
||||||
|
|
||||||
|
English | [中文](2026-08-13-python-minimal-model-visible-snapshot.zh.md)
|
||||||
|
|
||||||
|
## Problem
|
||||||
|
|
||||||
|
The Python lane never compared what the minimal composition actually shows the model. Dynamic runtime context reaches history as a user message, so the mock model's assertion that system-role messages equal the deployment persona could not see it, and the advanced executable snapshot replaces each request header's assembled system prompt with a token and each tool schema with its name. The sandbox-policy runtime-context message therefore rode along in the checked-in [minimal composition](../../../../examples/jsonrpc-agent/minimal.cordis.yml) while `python-runtime` stayed green, and any plugin that adds a system section, a tool, or another context message could do the same.
|
||||||
|
|
||||||
|
## Decision
|
||||||
|
|
||||||
|
The `sdk-minimal` scenario in [the packaged-runtime smoke](../../../../scripts/smoke-python-runtime.py) records `scripts/snapshots/python-sdk-single-exe/minimal/model-visible.json`: for every model request of the turn, the advertised tool schemas verbatim and the message list. System and user messages keep their full text with the scenario's temporary directory tokenized; assistant and tool messages keep only call identity, because their PTY and filesystem text differs across the platforms the expected output replays on.
|
||||||
|
|
||||||
|
One model-visible message is excluded: the agent loop's dynamic runtime-context snapshot. The same composition emits it on macOS and not on Linux, which the required lane runs, so no single expected output can carry it. That difference is a defect in its own right ([#2488](https://github.com/deepseek-harness/deepseek-harness/issues/2488)) — this expected output covers every other model-visible message rather than waiting for it.
|
||||||
|
|
||||||
|
The mock model no longer asserts the minimal scenario's tools and system prompts — the snapshot owns that surface and reports a complete diff instead of the first mismatch. Snapshot comparison takes its directory and file set as arguments, so the `minimal` and `advanced` expected outputs use one implementation, and `--update-snapshots` accepts `sdk-minimal`.
|
||||||
|
|
||||||
|
## Alternatives considered
|
||||||
|
|
||||||
|
**Snapshot the minimal session log, like the advanced scenario.** The minimal turn drives a real PTY and editor, so persisted tool results carry platform-dependent text. The expected output would go red for reasons unrelated to model-visible assembly, and normalizing that text away leaves the log carrying little the model-visible file does not.
|
||||||
|
|
||||||
|
**Extend the mock model's inline assertions.** Every new model-visible contribution would need another hand-written expectation, and a failure names one mismatch rather than the whole surface. Tool descriptions would also be duplicated from the composition into the script.
|
||||||
|
|
||||||
|
**Rely on the TypeScript SDK snapshot.** Its `persistent-tools` scenario pins the same composition's system prompt, tool schemas, and runtime context, but through replayed model responses and a source or `lib` runtime, in a different required job. It cannot show what the deployed executable's closure assembles for a Python caller.
|
||||||
|
|
||||||
|
## Consequences
|
||||||
|
|
||||||
|
A change to the minimal composition's model-visible surface — a system section, a tool, a tool description, or an added user message — now fails `python-runtime` with the exact diff, and landing it means rerunning `--scenario sdk-minimal --update-snapshots` and reviewing that diff. The minimal composition's tool descriptions become reviewed expected output.
|
||||||
|
|
||||||
|
Assistant and tool message text is no longer compared, and the runtime-context snapshot is not compared at all. The scenario's own assertions continue to own persistent-shell state, editor output, and the final response; [#2488](https://github.com/deepseek-harness/deepseek-harness/issues/2488) owns the excluded message until its platform difference is resolved.
|
||||||
|
|
||||||
|
[AGENTS.md](../../../../AGENTS.md) and [the testing policy](../../../../docs/testing.md) now name both SDKs as independent projections of the agent loop, session lifecycle, and `SessionEventMap`, so a change to any of those carries updating both expected outputs rather than only the one a contributor happens to run.
|
||||||
+33
@@ -0,0 +1,33 @@
|
|||||||
|
# Agent Note：Python 极简组合的模型可见快照
|
||||||
|
|
||||||
|
Status: implemented
|
||||||
|
|
||||||
|
[English](2026-08-13-python-minimal-model-visible-snapshot.md) | 中文
|
||||||
|
|
||||||
|
## 问题
|
||||||
|
|
||||||
|
Python 通道从未比对极简组合实际展示给模型的内容。动态运行时上下文以 user 消息进入历史，因此 mock 模型"system 角色消息等于部署 persona"的断言看不见它；而进阶可执行文件快照会把每个请求头中已组装的系统提示词换成占位符、把每个工具 schema 换成其名称。于是 sandbox-policy 的运行时上下文消息一直搭车留在签入的[极简组合](../../../../examples/jsonrpc-agent/minimal.cordis.yml)里，而 `python-runtime` 始终是绿的；任何新增系统分段、工具或其他上下文消息的插件都能照此蒙混过关。
|
||||||
|
|
||||||
|
## 决策
|
||||||
|
|
||||||
|
[打包运行时冒烟测试](../../../../scripts/smoke-python-runtime.py)的 `sdk-minimal` 场景会录制 `scripts/snapshots/python-sdk-single-exe/minimal/model-visible.json`：对该回合的每个模型请求，逐字记录对外公布的工具 schema 与消息列表。system 与 user 消息保留全文，仅将场景的临时目录替换为占位符；assistant 与 tool 消息只保留调用标识，因为它们的 PTY 与文件系统文本在期望输出需要重放的各平台上并不相同。
|
||||||
|
|
||||||
|
有一条模型可见消息被排除在外：agent loop 的动态运行时上下文快照。同一组合在 macOS 上会发出它，在必需车道所用的 Linux 上不会，因此任何单一期望输出都无法承载它。该差异本身就是缺陷（[#2488](https://github.com/deepseek-harness/deepseek-harness/issues/2488)）——这份期望输出覆盖其余全部模型可见消息，而不是等它先被修复。
|
||||||
|
|
||||||
|
mock 模型不再断言极简场景的工具与系统提示词——该面由快照拥有，并给出完整差异而非首个不匹配项。快照比对以目录与文件集合为参数，因此 `minimal` 与 `advanced` 两份期望输出共用一套实现，且 `--update-snapshots` 接受 `sdk-minimal`。
|
||||||
|
|
||||||
|
## 曾考虑的替代方案
|
||||||
|
|
||||||
|
**像进阶场景那样对极简会话日志做快照。** 极简回合驱动真实 PTY 与编辑器，持久化的工具结果带有平台相关文本。期望输出会因与模型可见组装无关的原因变红；而把这些文本归一化掉之后，日志所承载的内容也就所剩无几。
|
||||||
|
|
||||||
|
**扩展 mock 模型中的内联断言。** 每新增一项模型可见贡献都要再手写一条期望，且失败只会指出一处不匹配而非整个面。工具描述还会从组合复制进脚本，形成重复。
|
||||||
|
|
||||||
|
**依赖 TypeScript SDK 快照。** 其 `persistent-tools` 场景固定了同一组合的系统提示词、工具 schema 与运行时上下文，但走的是重放的模型响应与 source 或 `lib` 运行时，且位于另一个必需任务中。它无法体现已部署可执行文件的闭包为 Python 调用方组装出什么。
|
||||||
|
|
||||||
|
## 后果
|
||||||
|
|
||||||
|
极简组合模型可见面的改动——系统分段、工具、工具描述或新增的 user 消息——现在会让 `python-runtime` 带着精确差异失败；要让它落地，就必须重新运行 `--scenario sdk-minimal --update-snapshots` 并审阅该差异。极简组合的工具描述由此成为经过审阅的期望输出。
|
||||||
|
|
||||||
|
assistant 与 tool 消息文本不再参与比对，运行时上下文快照则完全不参与比对。持久 shell 状态、编辑器输出与最终响应仍由该场景自身的断言拥有；被排除的那条消息由 [#2488](https://github.com/deepseek-harness/deepseek-harness/issues/2488) 负责，直到其平台差异得到解决。
|
||||||
|
|
||||||
|
[AGENTS.md](../../../../AGENTS.md) 与[测试政策](../../../../docs/testing.md)现已点明两个 SDK 都是 agent loop、会话生命周期与 `SessionEventMap` 的独立投影，因此改动其中任何一项都要连带更新两侧的期望输出，而不只是贡献者恰好会运行的那一侧。
|
||||||
@@ -123,6 +123,7 @@ Real-API tests and demos read `DEEPSEEK_API_KEY`, optional `DEEPSEEK_BASE_URL`,
|
|||||||
- **Testing policy** — [docs/testing.md](docs/testing.md). Every non-trivial model- or product-user-visible behavior change adds or updates a keyless snapshot through a real runnable example in the same PR; package tests, e2e-only assertions, and mock-only fixtures do not substitute for the assembled application transcript. Fixtures must replay on macOS/Linux; fix fixtures, not normalizers.
|
- **Testing policy** — [docs/testing.md](docs/testing.md). Every non-trivial model- or product-user-visible behavior change adds or updates a keyless snapshot through a real runnable example in the same PR; package tests, e2e-only assertions, and mock-only fixtures do not substitute for the assembled application transcript. Fixtures must replay on macOS/Linux; fix fixtures, not normalizers.
|
||||||
- **A tool's UI render intent is part of its design**, decided up front (`generic`/`terminal`/`diff`, `locations`); presentation methods are pure functions of `args` ([cookbook](docs/cookbook/adding-a-tool.md)).
|
- **A tool's UI render intent is part of its design**, decided up front (`generic`/`terminal`/`diff`, `locations`); presentation methods are pure functions of `args` ([cookbook](docs/cookbook/adding-a-tool.md)).
|
||||||
- **Plan unit, e2e, and snapshot coverage** for capability seams, lifecycle paths, and transcript output; include missing snapshot-harness support in the same change.
|
- **Plan unit, e2e, and snapshot coverage** for capability seams, lifecycle paths, and transcript output; include missing snapshot-harness support in the same change.
|
||||||
|
- **Both SDKs project the loop.** Agent-loop, session-lifecycle, and `SessionEventMap` changes update the TypeScript and Python SDK expected outputs in the same PR; `pnpm run test` covers neither ([surfaces](docs/testing.md#when-a-snapshot-test-is-required)).
|
||||||
- **Choose PR history deliberately.** Split independent changes; fix the introducing PR before propagation. Standalone PRs and official stacks may merge-forward or rebase after review. Rewrites use `--force-with-lease`, abort on remote movement, never raw `--force`; an in-progress merge-forward preserves its checkpoint before taking a newer base ([rationale](.agents/notes/implemented/process/2026-08-02-native-github-stacks-and-optional-rebases.md)).
|
- **Choose PR history deliberately.** Split independent changes; fix the introducing PR before propagation. Standalone PRs and official stacks may merge-forward or rebase after review. Rewrites use `--force-with-lease`, abort on remote movement, never raw `--force`; an in-progress merge-forward preserves its checkpoint before taking a newer base ([rationale](.agents/notes/implemented/process/2026-08-02-native-github-stacks-and-optional-rebases.md)).
|
||||||
- **Labels:** one PR `kind/*`, all material `area/*`, and native Issue Type ([taxonomy](.agents/notes/implemented/process/2026-08-08-unified-github-label-taxonomy.md)).
|
- **Labels:** one PR `kind/*`, all material `area/*`, and native Issue Type ([taxonomy](.agents/notes/implemented/process/2026-08-08-unified-github-label-taxonomy.md)).
|
||||||
- TODO markers: `FIXME`/`TODO`/`XXX` by urgency ([semantics](docs/development.md)).
|
- TODO markers: `FIXME`/`TODO`/`XXX` by urgency ([semantics](docs/development.md)).
|
||||||
|
|||||||
@@ -2,5 +2,5 @@
|
|||||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||||
# after editing either side, bring the other along and re-record with:
|
# after editing either side, bring the other along and re-record with:
|
||||||
# pnpm run verify-translation-pairing --write docs/testing.md
|
# pnpm run verify-translation-pairing --write docs/testing.md
|
||||||
testing.md: 8ed81412a954bcd1d4b5d84eb4133ed081092764
|
testing.md: bef73983e48365f6655e4d4242f4f73223d971ed
|
||||||
testing.zh.md: 5662db51847ba2f4e297d0bd3d4af0d024fa29fb
|
testing.zh.md: 7a5935165a335b4f3885092ef7b7b29f08f53c51
|
||||||
|
|||||||
+1
-1
@@ -46,4 +46,4 @@ An e2e assertion re-runs the command or re-reads the file externally; a keyword
|
|||||||
|
|
||||||
## When a snapshot test is required
|
## When a snapshot test is required
|
||||||
|
|
||||||
Every non-trivial model-, protocol-, or human-visible change adds or updates a keyless scenario in the same PR through a runnable example's owning snapshot suite. Package tests, e2e assertions, mock/test-only compositions, and PR rationale do not replace the assembled transcript; extend the harness when needed. ACP automation scenarios use `examples/<name>/tests/snapshots/`, a scenario table over the [`dsh-acp-snapshot`](../packages/test-support/acp-snapshot/README.md) suite factory (`examples/acp-agent` is primary); `examples/headless-agent` owns the internal canonical-event JSONL snapshots and replay fixtures. The `pwsh-tool-turn` ACP scenario boots real `pwsh` and skips where it is absent. Completed interactive-terminal journeys use JSONL-driven scenarios under `apps/cli/tests/snapshots/`; transient presentation uses the package-local semantic matrix, with a PTY case when input, Loader selection, or terminal teardown changes. Browser-rendered web GUI journeys use `apps/web/tests/snapshots/`. New capability seams, lifecycle variants, or transcript surfaces name every coverage tier at plan time and verify the harness can express it before implementation.
|
Every non-trivial model-, protocol-, or human-visible change adds or updates a keyless scenario in the same PR through a runnable example's owning snapshot suite. Package tests, e2e assertions, mock/test-only compositions, and PR rationale do not replace the assembled transcript; extend the harness when needed. ACP automation scenarios use `examples/<name>/tests/snapshots/`, a scenario table over the [`dsh-acp-snapshot`](../packages/test-support/acp-snapshot/README.md) suite factory (`examples/acp-agent` is primary); `examples/headless-agent` owns the internal canonical-event JSONL snapshots and replay fixtures. The `pwsh-tool-turn` ACP scenario boots real `pwsh` and skips where it is absent. Completed interactive-terminal journeys use JSONL-driven scenarios under `apps/cli/tests/snapshots/`; transient presentation uses the package-local semantic matrix, with a PTY case when input, Loader selection, or terminal teardown changes. Browser-rendered web GUI journeys use `apps/web/tests/snapshots/`. The two SDKs project the agent loop, session lifecycle, and `SessionEventMap` independently, so changing any of those updates both: `examples/jsonrpc-agent/tests/snapshots/` owns the TypeScript client; `scripts/snapshots/python-sdk-single-exe/` owns the Python client, which only the required `python-runtime` CI job runs. New capability seams, lifecycle variants, or transcript surfaces name every coverage tier at plan time and verify the harness can express it before implementation.
|
||||||
|
|||||||
+1
-1
@@ -46,4 +46,4 @@ e2e 断言应重新运行命令或从外部重新读取文件；对 agent 自身
|
|||||||
|
|
||||||
## 何时需要快照测试
|
## 何时需要快照测试
|
||||||
|
|
||||||
每项非平凡的模型可见、协议可见或人类可见变更，都必须在同一 PR 中，通过可运行示例所属的快照套件添加或更新无密钥场景。包测试、e2e 断言、mock 与仅测试组合、PR 理由都不能取代组装后的 transcript；必要时应扩展 harness。ACP 自动化场景使用 `examples/<name>/tests/snapshots/`，即基于 [`dsh-acp-snapshot`](../packages/test-support/acp-snapshot/README.md) 套件工厂的场景表（`examples/acp-agent` 为主套件）；`examples/headless-agent` 拥有内部规范事件 JSONL 快照与回放 fixture。`pwsh-tool-turn` ACP 场景启动真实 `pwsh`，在无 `pwsh` 的主机上跳过。已完成的交互式终端旅程使用 `apps/cli/tests/snapshots/` 下由 JSONL 驱动的场景；瞬态呈现使用包内语义矩阵，输入、Loader 选择或终端清理发生变化时还要添加 PTY 用例。浏览器渲染的 Web GUI 旅程使用上述 Web 应用快照套件。新的能力 seam、生命周期变体或 transcript 呈现接口在计划阶段就要列出每个覆盖层级，并在实现前验证 harness 能够表达它们。
|
每项非平凡的模型可见、协议可见或人类可见变更，都必须在同一 PR 中，通过可运行示例所属的快照套件添加或更新无密钥场景。包测试、e2e 断言、mock 与仅测试组合、PR 理由都不能取代组装后的 transcript；必要时应扩展 harness。ACP 自动化场景使用 `examples/<name>/tests/snapshots/`，即基于 [`dsh-acp-snapshot`](../packages/test-support/acp-snapshot/README.md) 套件工厂的场景表（`examples/acp-agent` 为主套件）；`examples/headless-agent` 拥有内部规范事件 JSONL 快照与回放 fixture。`pwsh-tool-turn` ACP 场景启动真实 `pwsh`，在无 `pwsh` 的主机上跳过。已完成的交互式终端旅程使用 `apps/cli/tests/snapshots/` 下由 JSONL 驱动的场景；瞬态呈现使用包内语义矩阵，输入、Loader 选择或终端清理发生变化时还要添加 PTY 用例。浏览器渲染的 Web GUI 旅程使用上述 Web 应用快照套件。两个 SDK 各自独立地投影 agent loop、会话生命周期与 `SessionEventMap`，因此改动其中任何一项都要同时更新两者：`examples/jsonrpc-agent/tests/snapshots/` 拥有 TypeScript 客户端；`scripts/snapshots/python-sdk-single-exe/` 拥有 Python 客户端，且只有必需的 `python-runtime` CI 作业会运行它。新的能力 seam、生命周期变体或 transcript 呈现接口在计划阶段就要列出每个覆盖层级，并在实现前验证 harness 能够表达它们。
|
||||||
|
|||||||
@@ -2,5 +2,5 @@
|
|||||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||||
# after editing either side, bring the other along and re-record with:
|
# after editing either side, bring the other along and re-record with:
|
||||||
# pnpm run verify-translation-pairing --write python/development.md
|
# pnpm run verify-translation-pairing --write python/development.md
|
||||||
development.md: 31dc254b58c05c2a19c7c4cd5dc1e53207517902
|
development.md: fe62a109f2643afe0b9be1ed51b86be0b9fa731f
|
||||||
development.zh.md: dbb85a0cffc5e06c7ca781b2b01f6993395204e3
|
development.zh.md: d4ab9850d6c83dc17a740240f97cef89d61faaa0
|
||||||
|
|||||||
@@ -27,6 +27,15 @@ uv run --project python/sdk pytest
|
|||||||
|
|
||||||
`python/sdk/tests/test_bundled_runtime.py` exercises available bundled carriers and skips a carrier when its artifact has not been built. For repository-wide test policy, see [Testing](../docs/testing.md).
|
`python/sdk/tests/test_bundled_runtime.py` exercises available bundled carriers and skips a carrier when its artifact has not been built. For repository-wide test policy, see [Testing](../docs/testing.md).
|
||||||
|
|
||||||
|
That suite drives fake runtime peers. `scripts/smoke-python-runtime.py` drives the real packaged runtime instead, and the required `python-runtime` CI job runs every scenario against a freshly built executable:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
uv run --project python/sdk python scripts/smoke-python-runtime.py \
|
||||||
|
--scenario sdk-minimal --exe dist-exe/dsh-jsonrpc-agent-pkg-macos-arm64
|
||||||
|
```
|
||||||
|
|
||||||
|
Two scenarios compare committed expected output under `scripts/snapshots/python-sdk-single-exe/`. `minimal/model-visible.json` pins the checked-in minimal composition's assembled system prompts, advertised tool schemas, and model-visible messages, so a plugin that contributes an unintended system section or user message fails the job; it drops the dynamic runtime-context snapshot, which the same composition emits on macOS and not on Linux ([#2488](https://github.com/deepseek-harness/deepseek-harness/issues/2488)). `advanced/` pins the SDK result and the persisted session logs. Rerun the owning scenario with `--update-snapshots` and review that diff before committing it.
|
||||||
|
|
||||||
An interactive smoke test needs `DEEPSEEK_API_KEY` in the environment or repository-root `.env`:
|
An interactive smoke test needs `DEEPSEEK_API_KEY` in the environment or repository-root `.env`:
|
||||||
|
|
||||||
```python
|
```python
|
||||||
|
|||||||
@@ -27,6 +27,15 @@ uv run --project python/sdk pytest
|
|||||||
|
|
||||||
`python/sdk/tests/test_bundled_runtime.py` 会运行可用的内置载体；某个载体的产物尚未构建时，会跳过该载体。仓库级测试政策见 [测试](../docs/testing.md)。
|
`python/sdk/tests/test_bundled_runtime.py` 会运行可用的内置载体；某个载体的产物尚未构建时，会跳过该载体。仓库级测试政策见 [测试](../docs/testing.md)。
|
||||||
|
|
||||||
|
该套件面向的是伪造的运行时对端。`scripts/smoke-python-runtime.py` 面向真实的打包运行时；必需的 `python-runtime` CI 任务会用新构建的可执行文件运行全部场景：
|
||||||
|
|
||||||
|
```sh
|
||||||
|
uv run --project python/sdk python scripts/smoke-python-runtime.py \
|
||||||
|
--scenario sdk-minimal --exe dist-exe/dsh-jsonrpc-agent-pkg-macos-arm64
|
||||||
|
```
|
||||||
|
|
||||||
|
其中两个场景会比对 `scripts/snapshots/python-sdk-single-exe/` 下已提交的期望输出。`minimal/model-visible.json` 固定了签入的极简组合所组装的系统提示词、对外公布的工具 schema 以及模型可见消息，因此插件一旦贡献出计划外的系统分段或 user 消息，该任务即失败；它会丢弃动态运行时上下文快照——同一组合在 macOS 上会发出它，在 Linux 上不会（[#2488](https://github.com/deepseek-harness/deepseek-harness/issues/2488)）。`advanced/` 固定 SDK 结果与持久化的会话日志。重新运行对应场景时加上 `--update-snapshots`，并在提交前审阅该差异。
|
||||||
|
|
||||||
交互式冒烟测试需要环境变量或仓库根目录 `.env` 中存在 `DEEPSEEK_API_KEY`：
|
交互式冒烟测试需要环境变量或仓库根目录 `.env` 中存在 `DEEPSEEK_API_KEY`：
|
||||||
|
|
||||||
```python
|
```python
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
{
|
{
|
||||||
"AGENTS.md": 1900,
|
"AGENTS.md": 1950,
|
||||||
"docs/AGENTS.md": 1320,
|
"docs/AGENTS.md": 1320,
|
||||||
"docs/architecture.md": 2400,
|
"docs/architecture.md": 2400,
|
||||||
"docs/cordis-primer.md": 600,
|
"docs/cordis-primer.md": 600,
|
||||||
|
|||||||
+118
-31
@@ -28,7 +28,6 @@ WORKFLOW_WORKER_TEXT = "workflow worker smoke ok"
|
|||||||
MINIMAL_PROMPT = "Exercise the packaged minimal agent's persistent Bash and string-replacement editor."
|
MINIMAL_PROMPT = "Exercise the packaged minimal agent's persistent Bash and string-replacement editor."
|
||||||
MINIMAL_TEXT = "minimal agent smoke ok"
|
MINIMAL_TEXT = "minimal agent smoke ok"
|
||||||
MINIMAL_EDITOR_PATH_PREFIX = "Editor path: "
|
MINIMAL_EDITOR_PATH_PREFIX = "Editor path: "
|
||||||
MINIMAL_SYSTEM_PROMPT = "You are a helpful software engineer assistant."
|
|
||||||
MINIMAL_CORDIS = (
|
MINIMAL_CORDIS = (
|
||||||
Path(__file__).resolve().parent.parent / "examples" / "jsonrpc-agent" / "minimal.cordis.yml"
|
Path(__file__).resolve().parent.parent / "examples" / "jsonrpc-agent" / "minimal.cordis.yml"
|
||||||
)
|
)
|
||||||
@@ -65,10 +64,18 @@ SNAPSHOT_WORKFLOW_SCRIPT = (
|
|||||||
f"const reply = await agent('{SNAPSHOT_WORKFLOW_CHILD_PROMPT}', {{ label: 'workflow-child' }})\n"
|
f"const reply = await agent('{SNAPSHOT_WORKFLOW_CHILD_PROMPT}', {{ label: 'workflow-child' }})\n"
|
||||||
"return { reply }"
|
"return { reply }"
|
||||||
)
|
)
|
||||||
SNAPSHOT_DIRECTORY = (
|
ADVANCED_SNAPSHOT_DIRECTORY = (
|
||||||
Path(__file__).resolve().parent / "snapshots" / "python-sdk-single-exe" / "advanced"
|
Path(__file__).resolve().parent / "snapshots" / "python-sdk-single-exe" / "advanced"
|
||||||
)
|
)
|
||||||
SNAPSHOT_FILENAMES = ("result.json", "session.jsonl", "session.1.jsonl", "session.2.jsonl")
|
ADVANCED_SNAPSHOT_FILENAMES = ("result.json", "session.jsonl", "session.1.jsonl", "session.2.jsonl")
|
||||||
|
MINIMAL_SNAPSHOT_DIRECTORY = (
|
||||||
|
Path(__file__).resolve().parent / "snapshots" / "python-sdk-single-exe" / "minimal"
|
||||||
|
)
|
||||||
|
MINIMAL_SNAPSHOT_FILENAMES = ("model-visible.json",)
|
||||||
|
# The agent loop's dynamic runtime-context snapshot is the one model-visible message this
|
||||||
|
# expected output cannot carry: the same composition emits it on macOS and not on Linux
|
||||||
|
# (deepseek-harness#2488), and the file must replay on both. Everything else is compared.
|
||||||
|
RUNTIME_CONTEXT_PREFIX = "Current runtime context"
|
||||||
CUSTOM_CORDIS = """\
|
CUSTOM_CORDIS = """\
|
||||||
- id: sdk-jsonrpc-server
|
- id: sdk-jsonrpc-server
|
||||||
name: '@deepseek-ai/dsh-sdk-jsonrpc-server'
|
name: '@deepseek-ai/dsh-sdk-jsonrpc-server'
|
||||||
@@ -170,17 +177,9 @@ def completion_chunks(body: dict[str, object]) -> list[dict[str, object]]:
|
|||||||
),
|
),
|
||||||
None,
|
None,
|
||||||
)
|
)
|
||||||
|
# The minimal composition's assembled system prompt, advertised tool schemas, and
|
||||||
|
# model-visible messages are pinned by its snapshot, not asserted here.
|
||||||
if minimal_prompt is not None:
|
if minimal_prompt is not None:
|
||||||
names = advertised_tool_names(body)
|
|
||||||
if names != {"bash", "str_replace_editor"}:
|
|
||||||
raise AssertionError(f"minimal agent smoke advertised unexpected tools: {names}")
|
|
||||||
system_prompts = [
|
|
||||||
message_text(message.get("content"))
|
|
||||||
for message in messages
|
|
||||||
if isinstance(message, dict) and message.get("role") == "system"
|
|
||||||
]
|
|
||||||
if system_prompts != [MINIMAL_SYSTEM_PROMPT]:
|
|
||||||
raise AssertionError(f"minimal agent smoke assembled unexpected system prompts: {system_prompts}")
|
|
||||||
return tool_call_chunks(
|
return tool_call_chunks(
|
||||||
"minimal-bash-1",
|
"minimal-bash-1",
|
||||||
"bash",
|
"bash",
|
||||||
@@ -485,8 +484,8 @@ def main() -> None:
|
|||||||
args = parser.parse_args()
|
args = parser.parse_args()
|
||||||
if args.scenario in {"all", "sdk-custom", "sdk-minimal", "sdk-snapshot", "direct"} and args.exe is None:
|
if args.scenario in {"all", "sdk-custom", "sdk-minimal", "sdk-snapshot", "direct"} and args.exe is None:
|
||||||
parser.error("--exe is required for custom, minimal, snapshot, and direct scenarios")
|
parser.error("--exe is required for custom, minimal, snapshot, and direct scenarios")
|
||||||
if args.update_snapshots and args.scenario not in {"all", "sdk-snapshot"}:
|
if args.update_snapshots and args.scenario not in {"all", "sdk-minimal", "sdk-snapshot"}:
|
||||||
parser.error("--update-snapshots requires --scenario sdk-snapshot or all")
|
parser.error("--update-snapshots requires --scenario sdk-minimal, sdk-snapshot, or all")
|
||||||
if args.exe is not None and not args.exe.is_file():
|
if args.exe is not None and not args.exe.is_file():
|
||||||
parser.error(f"runtime executable does not exist: {args.exe}")
|
parser.error(f"runtime executable does not exist: {args.exe}")
|
||||||
|
|
||||||
@@ -498,7 +497,7 @@ def main() -> None:
|
|||||||
smoke_sdk_custom(model.url, args.exe.resolve())
|
smoke_sdk_custom(model.url, args.exe.resolve())
|
||||||
if args.scenario in {"all", "sdk-minimal"}:
|
if args.scenario in {"all", "sdk-minimal"}:
|
||||||
assert args.exe is not None
|
assert args.exe is not None
|
||||||
smoke_sdk_minimal(model.url, args.exe.resolve())
|
smoke_sdk_minimal(model.url, args.exe.resolve(), args.update_snapshots)
|
||||||
if args.scenario in {"all", "sdk-snapshot"}:
|
if args.scenario in {"all", "sdk-snapshot"}:
|
||||||
assert args.exe is not None
|
assert args.exe is not None
|
||||||
smoke_sdk_snapshot(model.url, args.exe.resolve(), args.update_snapshots)
|
smoke_sdk_snapshot(model.url, args.exe.resolve(), args.update_snapshots)
|
||||||
@@ -558,10 +557,12 @@ def smoke_sdk_custom(base_url: str, executable: Path) -> None:
|
|||||||
assert_session_log(sessions, root, EXPECTED_TEXT, CODE_WORKER_TEXT, WORKFLOW_WORKER_TEXT)
|
assert_session_log(sessions, root, EXPECTED_TEXT, CODE_WORKER_TEXT, WORKFLOW_WORKER_TEXT)
|
||||||
|
|
||||||
|
|
||||||
def smoke_sdk_minimal(base_url: str, executable: Path) -> None:
|
def smoke_sdk_minimal(base_url: str, executable: Path, update_snapshots: bool) -> None:
|
||||||
"""Exercise the checked-in minimal composition through the packaged executable."""
|
"""Exercise the checked-in minimal composition through the packaged executable."""
|
||||||
from deepseek_harness import DeepSeekHarness
|
from deepseek_harness import DeepSeekHarness
|
||||||
|
|
||||||
|
# One mock model serves every scenario of a run, so the snapshot takes this turn's slice.
|
||||||
|
first_request = len(MockModelHandler.requests)
|
||||||
with tempfile.TemporaryDirectory(prefix="dsh-sdk-minimal-") as temporary:
|
with tempfile.TemporaryDirectory(prefix="dsh-sdk-minimal-") as temporary:
|
||||||
root = Path(temporary).resolve()
|
root = Path(temporary).resolve()
|
||||||
editor_path = root / "created.txt"
|
editor_path = root / "created.txt"
|
||||||
@@ -587,6 +588,11 @@ def smoke_sdk_minimal(base_url: str, executable: Path) -> None:
|
|||||||
raise AssertionError(f"packaged editor wrote unexpected content: {editor_path.read_text()!r}")
|
raise AssertionError(f"packaged editor wrote unexpected content: {editor_path.read_text()!r}")
|
||||||
assert_session_log(sessions, root, MINIMAL_TEXT, "COUNT=1", "COUNT=2 CWD=/tmp")
|
assert_session_log(sessions, root, MINIMAL_TEXT, "COUNT=1", "COUNT=2 CWD=/tmp")
|
||||||
|
|
||||||
|
files = build_minimal_snapshot_files(MockModelHandler.requests[first_request:], root)
|
||||||
|
compare_snapshot_files(
|
||||||
|
files, update_snapshots, MINIMAL_SNAPSHOT_DIRECTORY, MINIMAL_SNAPSHOT_FILENAMES,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def smoke_sdk_snapshot(base_url: str, executable: Path, update_snapshots: bool) -> None:
|
def smoke_sdk_snapshot(base_url: str, executable: Path, update_snapshots: bool) -> None:
|
||||||
"""Drive and compare the advanced SDK/executable behavioral snapshot."""
|
"""Drive and compare the advanced SDK/executable behavioral snapshot."""
|
||||||
@@ -628,7 +634,9 @@ def smoke_sdk_snapshot(base_url: str, executable: Path, update_snapshots: bool)
|
|||||||
raise AssertionError("second advanced child log has no workflow-subagent result")
|
raise AssertionError("second advanced child log has no workflow-subagent result")
|
||||||
|
|
||||||
files = build_snapshot_files(result, logs, child_ids, root)
|
files = build_snapshot_files(result, logs, child_ids, root)
|
||||||
compare_snapshot_files(files, update_snapshots)
|
compare_snapshot_files(
|
||||||
|
files, update_snapshots, ADVANCED_SNAPSHOT_DIRECTORY, ADVANCED_SNAPSHOT_FILENAMES,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
def smoke_direct(base_url: str, executable: Path) -> None:
|
def smoke_direct(base_url: str, executable: Path) -> None:
|
||||||
@@ -802,6 +810,79 @@ def snapshot_child_ids(result: "RunResult") -> list[str]:
|
|||||||
return child_ids
|
return child_ids
|
||||||
|
|
||||||
|
|
||||||
|
def build_minimal_snapshot_files(
|
||||||
|
requests: list[dict[str, object]],
|
||||||
|
cwd: Path,
|
||||||
|
) -> dict[str, str]:
|
||||||
|
"""Render the minimal composition's model-visible surface as expected output.
|
||||||
|
|
||||||
|
Every assembled system prompt, advertised tool schema, and system or user message is
|
||||||
|
kept verbatim: they carry what the deployment actually shows the model, so a plugin
|
||||||
|
that contributes an unintended system section or user message cannot pass unnoticed.
|
||||||
|
Assistant and tool payloads keep only their call identity, and the dynamic
|
||||||
|
runtime-context snapshot is dropped, because their text differs across the platforms
|
||||||
|
this expected output must replay on.
|
||||||
|
"""
|
||||||
|
snapshot = []
|
||||||
|
for body in requests:
|
||||||
|
messages = body.get("messages")
|
||||||
|
if not isinstance(messages, list):
|
||||||
|
raise AssertionError(f"minimal model request has no messages: {body}")
|
||||||
|
snapshot.append({
|
||||||
|
"tools": minimal_snapshot_text(body.get("tools"), cwd),
|
||||||
|
"messages": [
|
||||||
|
minimal_snapshot_message(message, cwd)
|
||||||
|
for message in messages
|
||||||
|
if not is_runtime_context_message(message)
|
||||||
|
],
|
||||||
|
})
|
||||||
|
return {"model-visible.json": json.dumps(snapshot, indent=2, ensure_ascii=False) + "\n"}
|
||||||
|
|
||||||
|
|
||||||
|
def is_runtime_context_message(message: object) -> bool:
|
||||||
|
"""Identify the agent loop's dynamic runtime-context snapshot, current or cleared."""
|
||||||
|
return (
|
||||||
|
isinstance(message, dict)
|
||||||
|
and message.get("role") == "user"
|
||||||
|
and message_text(message.get("content")).startswith(RUNTIME_CONTEXT_PREFIX)
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def minimal_snapshot_message(message: object, cwd: Path) -> dict[str, object]:
|
||||||
|
"""Reduce one model-visible message to its stable, behavior-carrying parts."""
|
||||||
|
if not isinstance(message, dict):
|
||||||
|
raise AssertionError(f"minimal model request has an invalid message: {message}")
|
||||||
|
role = message.get("role")
|
||||||
|
if role in ("system", "user"):
|
||||||
|
return {"role": role, "text": minimal_snapshot_text(message_text(message.get("content")), cwd)}
|
||||||
|
if role == "assistant":
|
||||||
|
calls = message.get("tool_calls")
|
||||||
|
if not isinstance(calls, list):
|
||||||
|
raise AssertionError(f"minimal assistant message has no tool calls: {message}")
|
||||||
|
return {
|
||||||
|
"role": role,
|
||||||
|
"toolCalls": [
|
||||||
|
{"id": call.get("id"), "name": (call.get("function") or {}).get("name")}
|
||||||
|
for call in calls
|
||||||
|
if isinstance(call, dict)
|
||||||
|
],
|
||||||
|
}
|
||||||
|
if role == "tool":
|
||||||
|
return {"role": role, "toolCallId": message.get("tool_call_id"), "text": "{{tool-result}}"}
|
||||||
|
raise AssertionError(f"minimal model request has an unexpected message role: {message}")
|
||||||
|
|
||||||
|
|
||||||
|
def minimal_snapshot_text(value: object, cwd: Path) -> object:
|
||||||
|
"""Replace the scenario's temporary working directory everywhere it appears."""
|
||||||
|
if isinstance(value, str):
|
||||||
|
return value.replace(str(cwd), "{{cwd}}")
|
||||||
|
if isinstance(value, list):
|
||||||
|
return [minimal_snapshot_text(item, cwd) for item in value]
|
||||||
|
if isinstance(value, dict):
|
||||||
|
return {key: minimal_snapshot_text(item, cwd) for key, item in value.items()}
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
def build_snapshot_files(
|
def build_snapshot_files(
|
||||||
result: "RunResult",
|
result: "RunResult",
|
||||||
logs: dict[str, list[dict[str, object]]],
|
logs: dict[str, list[dict[str, object]]],
|
||||||
@@ -838,8 +919,6 @@ def build_snapshot_files(
|
|||||||
files[f"session.{index}.jsonl"] = render_jsonl(
|
files[f"session.{index}.jsonl"] = render_jsonl(
|
||||||
[normalize_snapshot_value(record, replacements) for record in logs[child_id]]
|
[normalize_snapshot_value(record, replacements) for record in logs[child_id]]
|
||||||
)
|
)
|
||||||
if tuple(files) != SNAPSHOT_FILENAMES:
|
|
||||||
raise AssertionError(f"advanced snapshot file set drifted: {tuple(files)}")
|
|
||||||
return files
|
return files
|
||||||
|
|
||||||
|
|
||||||
@@ -930,27 +1009,35 @@ def render_jsonl(records: list[object]) -> str:
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
def compare_snapshot_files(files: dict[str, str], update: bool) -> None:
|
def compare_snapshot_files(
|
||||||
"""Write or exactly compare the advanced executable snapshot files."""
|
files: dict[str, str],
|
||||||
|
update: bool,
|
||||||
|
directory: Path,
|
||||||
|
filenames: tuple[str, ...],
|
||||||
|
) -> None:
|
||||||
|
"""Write or exactly compare one scenario's expected snapshot files."""
|
||||||
|
scenario = directory.name
|
||||||
|
if tuple(files) != filenames:
|
||||||
|
raise AssertionError(f"{scenario} snapshot builder produced {tuple(files)}, expected {filenames}")
|
||||||
if update:
|
if update:
|
||||||
SNAPSHOT_DIRECTORY.mkdir(parents=True, exist_ok=True)
|
directory.mkdir(parents=True, exist_ok=True)
|
||||||
for name, content in files.items():
|
for name, content in files.items():
|
||||||
(SNAPSHOT_DIRECTORY / name).write_text(content, encoding="utf-8")
|
(directory / name).write_text(content, encoding="utf-8")
|
||||||
print(f"smoke-python-runtime: updated snapshots in {SNAPSHOT_DIRECTORY}")
|
print(f"smoke-python-runtime: updated snapshots in {directory}")
|
||||||
|
|
||||||
existing = {
|
existing = {
|
||||||
path.name
|
path.name
|
||||||
for path in SNAPSHOT_DIRECTORY.iterdir()
|
for path in directory.iterdir()
|
||||||
if path.is_file()
|
if path.is_file()
|
||||||
} if SNAPSHOT_DIRECTORY.is_dir() else set()
|
} if directory.is_dir() else set()
|
||||||
expected = set(SNAPSHOT_FILENAMES)
|
expected = set(filenames)
|
||||||
if existing != expected:
|
if existing != expected:
|
||||||
raise AssertionError(
|
raise AssertionError(
|
||||||
"advanced snapshot files differ: "
|
f"{scenario} snapshot files differ: "
|
||||||
f"missing={sorted(expected - existing)}, unexpected={sorted(existing - expected)}"
|
f"missing={sorted(expected - existing)}, unexpected={sorted(existing - expected)}"
|
||||||
)
|
)
|
||||||
for name, actual in files.items():
|
for name, actual in files.items():
|
||||||
expected_text = (SNAPSHOT_DIRECTORY / name).read_text(encoding="utf-8")
|
expected_text = (directory / name).read_text(encoding="utf-8")
|
||||||
if actual == expected_text:
|
if actual == expected_text:
|
||||||
continue
|
continue
|
||||||
diff = "".join(difflib.unified_diff(
|
diff = "".join(difflib.unified_diff(
|
||||||
@@ -960,7 +1047,7 @@ def compare_snapshot_files(files: dict[str, str], update: bool) -> None:
|
|||||||
tofile=f"actual/{name}",
|
tofile=f"actual/{name}",
|
||||||
))
|
))
|
||||||
raise AssertionError(
|
raise AssertionError(
|
||||||
f"advanced executable snapshot mismatch in {name}; "
|
f"{scenario} executable snapshot mismatch in {name}; "
|
||||||
"rerun with --update-snapshots after reviewing the behavior\n"
|
"rerun with --update-snapshots after reviewing the behavior\n"
|
||||||
f"{diff}"
|
f"{diff}"
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -0,0 +1,430 @@
|
|||||||
|
[
|
||||||
|
{
|
||||||
|
"tools": [
|
||||||
|
{
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "bash",
|
||||||
|
"description": "Run commands in a bash shell\n* When invoking this tool, the contents of the \"command\" parameter does NOT need to be XML-escaped.\n* You don't have access to the internet via this tool.\n* You do have access to a mirror of common linux and python packages via apt and pip.\n* State is persistent across command calls and discussions with the user.\n* To inspect a particular line range of a file, e.g. lines 10-25, try 'sed -n 10,25p /path/to/the/file'.\n* Please avoid commands that may produce a very large amount of output.\n* Please run long lived commands in the background, e.g. 'sleep 10 &' or start a server in the background.",
|
||||||
|
"parameters": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"command": {
|
||||||
|
"type": "string",
|
||||||
|
"description": "The bash command to run. Relative path is preferred in the command."
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"required": [
|
||||||
|
"command"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "str_replace_editor",
|
||||||
|
"description": "Custom editing tool for viewing, creating and editing files\n* State is persistent across command calls and discussions with the user\n* If `path` is a file, `view` displays the result of applying `cat -n`. If `path` is a directory, `view` lists non-hidden files and directories up to 2 levels deep\n* The `create` command cannot be used if the specified `path` already exists as a file\n* If a `command` generates a long output, it will be truncated and marked with `<response clipped>`\n\nNotes for using the `str_replace` command:\n* The `old_str` parameter should match EXACTLY one or more consecutive lines from the original file. Be mindful of whitespaces!\n* If the `old_str` parameter is not unique in the file, the replacement will not be performed. Make sure to include enough context in `old_str` to make it unique\n* The `new_str` parameter should contain the edited lines that should replace the `old_str`",
|
||||||
|
"parameters": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"command": {
|
||||||
|
"type": "string",
|
||||||
|
"description": "The commands to run. Allowed options are: `view`, `create`, `str_replace`, `insert`.",
|
||||||
|
"enum": [
|
||||||
|
"view",
|
||||||
|
"create",
|
||||||
|
"str_replace",
|
||||||
|
"insert"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"path": {
|
||||||
|
"type": "string",
|
||||||
|
"description": "Absolute path to file or directory, e.g. `/repo/file.py` or `/repo`."
|
||||||
|
},
|
||||||
|
"file_text": {
|
||||||
|
"type": "string",
|
||||||
|
"description": "Required parameter of `create` command, with the content of the file to be created."
|
||||||
|
},
|
||||||
|
"insert_line": {
|
||||||
|
"type": "integer",
|
||||||
|
"description": "Required parameter of `insert` command. The `new_str` will be inserted AFTER the line `insert_line` of `path`."
|
||||||
|
},
|
||||||
|
"new_str": {
|
||||||
|
"type": "string",
|
||||||
|
"description": "Optional parameter of `str_replace` command containing the new string (if not given, no string will be added). Required parameter of `insert` command containing the string to insert."
|
||||||
|
},
|
||||||
|
"old_str": {
|
||||||
|
"type": "string",
|
||||||
|
"description": "Required parameter of `str_replace` command containing the string in `path` to replace."
|
||||||
|
},
|
||||||
|
"view_range": {
|
||||||
|
"type": "array",
|
||||||
|
"description": "Optional parameter of `view` command when `path` points to a file. If none is given, the full file is shown. If provided, the file will be shown in the indicated line number range, e.g. [11, 12] will show lines 11 and 12. Indexing at 1 to start. Setting `[start_line, -1]` shows all lines from `start_line` to the end of the file.",
|
||||||
|
"items": {
|
||||||
|
"type": "integer"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"required": [
|
||||||
|
"command",
|
||||||
|
"path"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"messages": [
|
||||||
|
{
|
||||||
|
"role": "system",
|
||||||
|
"text": "You are a helpful software engineer assistant."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "user",
|
||||||
|
"text": "Exercise the packaged minimal agent's persistent Bash and string-replacement editor.\nEditor path: {{cwd}}/created.txt"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"tools": [
|
||||||
|
{
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "bash",
|
||||||
|
"description": "Run commands in a bash shell\n* When invoking this tool, the contents of the \"command\" parameter does NOT need to be XML-escaped.\n* You don't have access to the internet via this tool.\n* You do have access to a mirror of common linux and python packages via apt and pip.\n* State is persistent across command calls and discussions with the user.\n* To inspect a particular line range of a file, e.g. lines 10-25, try 'sed -n 10,25p /path/to/the/file'.\n* Please avoid commands that may produce a very large amount of output.\n* Please run long lived commands in the background, e.g. 'sleep 10 &' or start a server in the background.",
|
||||||
|
"parameters": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"command": {
|
||||||
|
"type": "string",
|
||||||
|
"description": "The bash command to run. Relative path is preferred in the command."
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"required": [
|
||||||
|
"command"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "str_replace_editor",
|
||||||
|
"description": "Custom editing tool for viewing, creating and editing files\n* State is persistent across command calls and discussions with the user\n* If `path` is a file, `view` displays the result of applying `cat -n`. If `path` is a directory, `view` lists non-hidden files and directories up to 2 levels deep\n* The `create` command cannot be used if the specified `path` already exists as a file\n* If a `command` generates a long output, it will be truncated and marked with `<response clipped>`\n\nNotes for using the `str_replace` command:\n* The `old_str` parameter should match EXACTLY one or more consecutive lines from the original file. Be mindful of whitespaces!\n* If the `old_str` parameter is not unique in the file, the replacement will not be performed. Make sure to include enough context in `old_str` to make it unique\n* The `new_str` parameter should contain the edited lines that should replace the `old_str`",
|
||||||
|
"parameters": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"command": {
|
||||||
|
"type": "string",
|
||||||
|
"description": "The commands to run. Allowed options are: `view`, `create`, `str_replace`, `insert`.",
|
||||||
|
"enum": [
|
||||||
|
"view",
|
||||||
|
"create",
|
||||||
|
"str_replace",
|
||||||
|
"insert"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"path": {
|
||||||
|
"type": "string",
|
||||||
|
"description": "Absolute path to file or directory, e.g. `/repo/file.py` or `/repo`."
|
||||||
|
},
|
||||||
|
"file_text": {
|
||||||
|
"type": "string",
|
||||||
|
"description": "Required parameter of `create` command, with the content of the file to be created."
|
||||||
|
},
|
||||||
|
"insert_line": {
|
||||||
|
"type": "integer",
|
||||||
|
"description": "Required parameter of `insert` command. The `new_str` will be inserted AFTER the line `insert_line` of `path`."
|
||||||
|
},
|
||||||
|
"new_str": {
|
||||||
|
"type": "string",
|
||||||
|
"description": "Optional parameter of `str_replace` command containing the new string (if not given, no string will be added). Required parameter of `insert` command containing the string to insert."
|
||||||
|
},
|
||||||
|
"old_str": {
|
||||||
|
"type": "string",
|
||||||
|
"description": "Required parameter of `str_replace` command containing the string in `path` to replace."
|
||||||
|
},
|
||||||
|
"view_range": {
|
||||||
|
"type": "array",
|
||||||
|
"description": "Optional parameter of `view` command when `path` points to a file. If none is given, the full file is shown. If provided, the file will be shown in the indicated line number range, e.g. [11, 12] will show lines 11 and 12. Indexing at 1 to start. Setting `[start_line, -1]` shows all lines from `start_line` to the end of the file.",
|
||||||
|
"items": {
|
||||||
|
"type": "integer"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"required": [
|
||||||
|
"command",
|
||||||
|
"path"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"messages": [
|
||||||
|
{
|
||||||
|
"role": "system",
|
||||||
|
"text": "You are a helpful software engineer assistant."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "user",
|
||||||
|
"text": "Exercise the packaged minimal agent's persistent Bash and string-replacement editor.\nEditor path: {{cwd}}/created.txt"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"toolCalls": [
|
||||||
|
{
|
||||||
|
"id": "minimal-bash-1",
|
||||||
|
"name": "bash"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"toolCallId": "minimal-bash-1",
|
||||||
|
"text": "{{tool-result}}"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"tools": [
|
||||||
|
{
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "bash",
|
||||||
|
"description": "Run commands in a bash shell\n* When invoking this tool, the contents of the \"command\" parameter does NOT need to be XML-escaped.\n* You don't have access to the internet via this tool.\n* You do have access to a mirror of common linux and python packages via apt and pip.\n* State is persistent across command calls and discussions with the user.\n* To inspect a particular line range of a file, e.g. lines 10-25, try 'sed -n 10,25p /path/to/the/file'.\n* Please avoid commands that may produce a very large amount of output.\n* Please run long lived commands in the background, e.g. 'sleep 10 &' or start a server in the background.",
|
||||||
|
"parameters": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"command": {
|
||||||
|
"type": "string",
|
||||||
|
"description": "The bash command to run. Relative path is preferred in the command."
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"required": [
|
||||||
|
"command"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "str_replace_editor",
|
||||||
|
"description": "Custom editing tool for viewing, creating and editing files\n* State is persistent across command calls and discussions with the user\n* If `path` is a file, `view` displays the result of applying `cat -n`. If `path` is a directory, `view` lists non-hidden files and directories up to 2 levels deep\n* The `create` command cannot be used if the specified `path` already exists as a file\n* If a `command` generates a long output, it will be truncated and marked with `<response clipped>`\n\nNotes for using the `str_replace` command:\n* The `old_str` parameter should match EXACTLY one or more consecutive lines from the original file. Be mindful of whitespaces!\n* If the `old_str` parameter is not unique in the file, the replacement will not be performed. Make sure to include enough context in `old_str` to make it unique\n* The `new_str` parameter should contain the edited lines that should replace the `old_str`",
|
||||||
|
"parameters": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"command": {
|
||||||
|
"type": "string",
|
||||||
|
"description": "The commands to run. Allowed options are: `view`, `create`, `str_replace`, `insert`.",
|
||||||
|
"enum": [
|
||||||
|
"view",
|
||||||
|
"create",
|
||||||
|
"str_replace",
|
||||||
|
"insert"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"path": {
|
||||||
|
"type": "string",
|
||||||
|
"description": "Absolute path to file or directory, e.g. `/repo/file.py` or `/repo`."
|
||||||
|
},
|
||||||
|
"file_text": {
|
||||||
|
"type": "string",
|
||||||
|
"description": "Required parameter of `create` command, with the content of the file to be created."
|
||||||
|
},
|
||||||
|
"insert_line": {
|
||||||
|
"type": "integer",
|
||||||
|
"description": "Required parameter of `insert` command. The `new_str` will be inserted AFTER the line `insert_line` of `path`."
|
||||||
|
},
|
||||||
|
"new_str": {
|
||||||
|
"type": "string",
|
||||||
|
"description": "Optional parameter of `str_replace` command containing the new string (if not given, no string will be added). Required parameter of `insert` command containing the string to insert."
|
||||||
|
},
|
||||||
|
"old_str": {
|
||||||
|
"type": "string",
|
||||||
|
"description": "Required parameter of `str_replace` command containing the string in `path` to replace."
|
||||||
|
},
|
||||||
|
"view_range": {
|
||||||
|
"type": "array",
|
||||||
|
"description": "Optional parameter of `view` command when `path` points to a file. If none is given, the full file is shown. If provided, the file will be shown in the indicated line number range, e.g. [11, 12] will show lines 11 and 12. Indexing at 1 to start. Setting `[start_line, -1]` shows all lines from `start_line` to the end of the file.",
|
||||||
|
"items": {
|
||||||
|
"type": "integer"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"required": [
|
||||||
|
"command",
|
||||||
|
"path"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"messages": [
|
||||||
|
{
|
||||||
|
"role": "system",
|
||||||
|
"text": "You are a helpful software engineer assistant."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "user",
|
||||||
|
"text": "Exercise the packaged minimal agent's persistent Bash and string-replacement editor.\nEditor path: {{cwd}}/created.txt"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"toolCalls": [
|
||||||
|
{
|
||||||
|
"id": "minimal-bash-1",
|
||||||
|
"name": "bash"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"toolCallId": "minimal-bash-1",
|
||||||
|
"text": "{{tool-result}}"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"toolCalls": [
|
||||||
|
{
|
||||||
|
"id": "minimal-bash-2",
|
||||||
|
"name": "bash"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"toolCallId": "minimal-bash-2",
|
||||||
|
"text": "{{tool-result}}"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"tools": [
|
||||||
|
{
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "bash",
|
||||||
|
"description": "Run commands in a bash shell\n* When invoking this tool, the contents of the \"command\" parameter does NOT need to be XML-escaped.\n* You don't have access to the internet via this tool.\n* You do have access to a mirror of common linux and python packages via apt and pip.\n* State is persistent across command calls and discussions with the user.\n* To inspect a particular line range of a file, e.g. lines 10-25, try 'sed -n 10,25p /path/to/the/file'.\n* Please avoid commands that may produce a very large amount of output.\n* Please run long lived commands in the background, e.g. 'sleep 10 &' or start a server in the background.",
|
||||||
|
"parameters": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"command": {
|
||||||
|
"type": "string",
|
||||||
|
"description": "The bash command to run. Relative path is preferred in the command."
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"required": [
|
||||||
|
"command"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"type": "function",
|
||||||
|
"function": {
|
||||||
|
"name": "str_replace_editor",
|
||||||
|
"description": "Custom editing tool for viewing, creating and editing files\n* State is persistent across command calls and discussions with the user\n* If `path` is a file, `view` displays the result of applying `cat -n`. If `path` is a directory, `view` lists non-hidden files and directories up to 2 levels deep\n* The `create` command cannot be used if the specified `path` already exists as a file\n* If a `command` generates a long output, it will be truncated and marked with `<response clipped>`\n\nNotes for using the `str_replace` command:\n* The `old_str` parameter should match EXACTLY one or more consecutive lines from the original file. Be mindful of whitespaces!\n* If the `old_str` parameter is not unique in the file, the replacement will not be performed. Make sure to include enough context in `old_str` to make it unique\n* The `new_str` parameter should contain the edited lines that should replace the `old_str`",
|
||||||
|
"parameters": {
|
||||||
|
"type": "object",
|
||||||
|
"properties": {
|
||||||
|
"command": {
|
||||||
|
"type": "string",
|
||||||
|
"description": "The commands to run. Allowed options are: `view`, `create`, `str_replace`, `insert`.",
|
||||||
|
"enum": [
|
||||||
|
"view",
|
||||||
|
"create",
|
||||||
|
"str_replace",
|
||||||
|
"insert"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"path": {
|
||||||
|
"type": "string",
|
||||||
|
"description": "Absolute path to file or directory, e.g. `/repo/file.py` or `/repo`."
|
||||||
|
},
|
||||||
|
"file_text": {
|
||||||
|
"type": "string",
|
||||||
|
"description": "Required parameter of `create` command, with the content of the file to be created."
|
||||||
|
},
|
||||||
|
"insert_line": {
|
||||||
|
"type": "integer",
|
||||||
|
"description": "Required parameter of `insert` command. The `new_str` will be inserted AFTER the line `insert_line` of `path`."
|
||||||
|
},
|
||||||
|
"new_str": {
|
||||||
|
"type": "string",
|
||||||
|
"description": "Optional parameter of `str_replace` command containing the new string (if not given, no string will be added). Required parameter of `insert` command containing the string to insert."
|
||||||
|
},
|
||||||
|
"old_str": {
|
||||||
|
"type": "string",
|
||||||
|
"description": "Required parameter of `str_replace` command containing the string in `path` to replace."
|
||||||
|
},
|
||||||
|
"view_range": {
|
||||||
|
"type": "array",
|
||||||
|
"description": "Optional parameter of `view` command when `path` points to a file. If none is given, the full file is shown. If provided, the file will be shown in the indicated line number range, e.g. [11, 12] will show lines 11 and 12. Indexing at 1 to start. Setting `[start_line, -1]` shows all lines from `start_line` to the end of the file.",
|
||||||
|
"items": {
|
||||||
|
"type": "integer"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"required": [
|
||||||
|
"command",
|
||||||
|
"path"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"messages": [
|
||||||
|
{
|
||||||
|
"role": "system",
|
||||||
|
"text": "You are a helpful software engineer assistant."
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "user",
|
||||||
|
"text": "Exercise the packaged minimal agent's persistent Bash and string-replacement editor.\nEditor path: {{cwd}}/created.txt"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"toolCalls": [
|
||||||
|
{
|
||||||
|
"id": "minimal-bash-1",
|
||||||
|
"name": "bash"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"toolCallId": "minimal-bash-1",
|
||||||
|
"text": "{{tool-result}}"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"toolCalls": [
|
||||||
|
{
|
||||||
|
"id": "minimal-bash-2",
|
||||||
|
"name": "bash"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"toolCallId": "minimal-bash-2",
|
||||||
|
"text": "{{tool-result}}"
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "assistant",
|
||||||
|
"toolCalls": [
|
||||||
|
{
|
||||||
|
"id": "minimal-editor",
|
||||||
|
"name": "str_replace_editor"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"role": "tool",
|
||||||
|
"toolCallId": "minimal-editor",
|
||||||
|
"text": "{{tool-result}}"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
]
|
||||||
Reference in New Issue
Block a user