mengzhx/deepseek-harness
1
0
Fork 0
mirror of https://github.com/deepseek-ai/deepseek-harness.git synced 2026-09-11 04:00:38 +00:00
Code Issues Packages Projects Releases Wiki Activity

test(sdk): record in-history system prompt updates in both clients

Browse Source 
Cover the missing SDK projections through shipped dsh profiles. TypeScript reuses the headless read-completion composition and records SDK notifications. Python all includes a focused minimal-profile variant with an ordinary prompt section updated after shell completion, unchanged tools, persisted/subscribed append events, and independently checked editor output.

Observed: focused TS refresh and built replay passed (1 test); Python sdk-minimal-in-history and sdk-minimal passed against apps/cli/lib/bin.js using uv; unchanged-prompt negative control failed as expected then restored scenario passed; corpus 3 tests, doc-sync 33 gates, and lint:contracts-ready passed. Host-only build initially left SDK built startup incomplete; build:lib:client completed the prerequisites. Native packaged/wheel and Windows validation remain pending parent integration.
This commit is contained in:
Tianyi Cui
2026-09-07 11:09:39 +08:00
parent 8d0bf14eb6
commit 9c8884284f
13 changed files with 248 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files
python/development.md
+1 -1
View File
@@ -34,7 +34,7 @@ uv run --project python/sdk python scripts/smoke-python-runtime.py \
--scenario sdk-minimal --exe dist-exe/deepseek-harness-sdk-runtime-macos-arm64
```
Three scenarios compare committed expected output under `scripts/snapshots/python-sdk-single-exe/`. `minimal/model-visible.json` pins the Linux/macOS `sdk-minimal` profile's assembled system prompts, advertised tool schemas, and model-visible messages; `minimal/win-x64/model-visible.json` pins its PowerShell counterpart. A plugin that contributes an unintended system section or user message therefore fails the job, and every message the profile emits is compared. `advanced/` pins one complex process's SDK result and parent/child session logs across every target. `restart/` launches two complete SDK runtime processes against one persistence root and snapshots their isolated model histories, high-level results, and separate durable logs across every target. Rerun the owning scenario with `--update-snapshots` and review that diff before committing it.
Four scenarios compare committed expected output under `scripts/snapshots/python-sdk-single-exe/`. `minimal/model-visible.json` pins the Linux/macOS `sdk-minimal` profile's assembled system prompts, advertised tool schemas, and model-visible messages; `minimal/win-x64/model-visible.json` pins its PowerShell counterpart. A plugin that contributes an unintended system section or user message therefore fails the job, and every message the profile emits is compared. `advanced/` pins one complex process's SDK result and parent/child session logs across every target. `restart/` launches two complete SDK runtime processes against one persistence root and snapshots their isolated model histories, high-level results, and separate durable logs across every target. `sdk-minimal-in-history` reuses the persistent-shell and editor scenario with a section that changes after the first successful shell call. `minimal-in-history/prompt-history.json` pins both prompt versions, the unchanged leading prompt in later requests, appended SDK system-message events, and `request/context.systemPromptUpdate`; tool schemas stay fixed and the editor file is checked independently. Rerun the owning scenario with `--update-snapshots` and review that diff before committing it.
Trusted pull requests also run `--scenario sdk-live --installed-wheel` on every native target. That scenario performs two tool-using turns against `https://api.deepseek.com`, verifies the created file externally, and fails when the repository secret is absent instead of self-skipping. Fork and Dependabot pull requests run the complete keyless installed-wheel path but receive no key.