Commit Graph
3200 Commits
Author SHA1 Message Date
Tianyi Cui a4e445236f Merge latest master prompt placement into in-history updates
Keep prepared-route in-history append, clear-all, and normalization semantics while adopting named persona prefix/suffix and late environment facts from master. Resolve documentation against the derived system-message history and retain the existing strict migration behavior.
2026-09-07 13:39:12 +08:00
Tianyi Cui d3ed6d6f88 Merge remote-tracking branch 'origin/master' into feat/system-prompt-surface-node 2026-09-07 13:35:06 +08:00
Tianyi Cui 25b28a14cd Merge branch 'feat/system-prompt-surface-node' into feat/system-prompt-in-history 2026-09-07 13:12:53 +08:00
Tianyi Cui 76cf592234 Merge commit 'd1e5b724d71e9a6f675232caa760395710826b81' into feat/system-prompt-surface-node 2026-09-07 13:12:46 +08:00
Tianyi Cui f68c4239e4 Merge remote-tracking branch 'origin/master' into release/session-log-v3 2026-09-07 13:09:36 +08:00
Tianyi Cui 3d54bd6d58 Merge branch 'master' into fix/ci-node-compile-cache-data-disk 2026-09-07 13:06:30 +08:00
Tianyi Cui ac4fa3d6f5 docs: clarify compatibility runner scope and verification 2026-09-07 12:42:02 +08:00
Tianyi Cui a137256f80 ci: set compatibility toolcache after runner environment export 2026-09-07 12:41:23 +08:00
Tianyi Cui f7a18f49ff ci: isolate compatibility Node jobs on self-hosted Linux 2026-09-07 12:41:23 +08:00
Tianyi Cui c379894af8 fix(ci): isolate routing test setup and correct scheduling docs 2026-09-07 12:40:40 +08:00
Tianyi Cui ab1ee99685 ci: defer macOS ARM runtime and Wine checks to master 2026-09-07 12:40:40 +08:00
Tianyi Cui 691861091f test(python): record upper V3 native event ordering
Regenerate current Python Session expectations through the real macOS arm64 SEA executable built from fdd44667b4b496093e765af57d3751dfb917d834. The native writer emits session/title after request/header and request/context. Update only five current V3 logs; result.json, minimal/model-visible.json, and minimal-in-history/prompt-history.json regenerate identically. All five V2 predecessors remain byte-identical to b2b300f9.

Build evidence: LEFTHOOK=0 pnpm install --frozen-lockfile --offline; pnpm exec tsx scripts/build-exe-for-python-sdk.ts --targets node24-macos-arm64 (full build, no skip-build): passed. file dist-exe/deepseek-harness-sdk-runtime-macos-arm64: Mach-O 64-bit executable arm64.

Smoke evidence: export PYTHONDONTWRITEBYTECODE=1 PYTHONPATH=python/sdk/src:python/sdk-runtime/src; PY=/Users/cty/dsh-deploy/.agents/worktrees/sp-kv-sdk-evidence/tmp/py-sdk-venv/bin/python. For each scenario in sdk-snapshot sdk-restart sdk-minimal sdk-minimal-in-history, ran "$PY" scripts/smoke-python-runtime.py --exe dist-exe/deepseek-harness-sdk-runtime-macos-arm64 --scenario "$scenario" --update-snapshots, then the same command without --update-snapshots: all eight passed.

Helpers: PYTHONDONTWRITEBYTECODE=1 PYTHONPATH=python/sdk/src:python/sdk-runtime/src uv run --offline --no-project --python /Users/cty/.local/share/uv/python/cpython-3.14.5-macos-aarch64-none/bin/python3 --with pytest --with "pydantic>=2.12,<3" python -m pytest python/sdk/tests/test_smoke_model.py -q: 28 passed. git diff --quiet b2b300f9 -- "scripts/snapshots/python-sdk-single-exe/**/*.v2.jsonl" and git diff --cached --check passed. No source, migration, test, or launcher changes.
2026-09-07 11:09:40 +08:00
Tianyi Cui 42955a346a test(python): record final admitted-input notification order
Refresh advanced and restart expectations through the final native packaged runtime. Synchronous request finalization places deferred fallback-title events after header/context in parent and child logs; expected SDK notifications must reflect that actual lifecycle instead of forcing the old incidental scheduling order.

Validation: the advanced mismatch was reproduced against the built macOS ARM64 executable, both owning scenarios refreshed, then the complete keyless --scenario all passed, including the new in-history prompt evidence and independent shell/file effects. No SDK runtime or normalizer behavior changes in this commit.
2026-09-07 11:09:40 +08:00
Tianyi Cui 627747d8fa docs: synchronize prompt admission contracts and repair lifecycle diagram
Copy the current SessionEventMap prompt and route JSDoc into both subsystem references so type-equiv validates the prepared-route and clear-all semantics. Remove the Mermaid statement delimiter from the generated retry note while retaining the same lifecycle meaning; regenerate graph artifacts and confirm both bilingual pairs.

Validation: verify-type-equiv, verify-mermaid, verify-doc-graphs, scoped verify-translation-pairing, and git diff --check.
2026-09-07 11:09:40 +08:00
Tianyi Cui 9c8884284f test(sdk): record in-history system prompt updates in both clients
Cover the missing SDK projections through shipped dsh profiles. TypeScript reuses the headless read-completion composition and records SDK notifications. Python all includes a focused minimal-profile variant with an ordinary prompt section updated after shell completion, unchanged tools, persisted/subscribed append events, and independently checked editor output.

Observed: focused TS refresh and built replay passed (1 test); Python sdk-minimal-in-history and sdk-minimal passed against apps/cli/lib/bin.js using uv; unchanged-prompt negative control failed as expected then restored scenario passed; corpus 3 tests, doc-sync 33 gates, and lint:contracts-ready passed. Host-only build initially left SDK built startup incomplete; build:lib:client completed the prerequisites. Native packaged/wheel and Windows validation remain pending parent integration.
2026-09-07 11:09:39 +08:00
Tianyi Cui 425a0a55e3 fix(agent-loop): reconcile prompts after retry compaction
Every same-step attempt resolves its bound route and reconciles the accepted rendered prompt against the current surface before deriving history. Preserve assembly, pre-step admission and entered users exactly once; compaction retries must not resurrect an older surviving prompt.

Normalize a broken request series to the current head plus dormant empty later system nodes, even if later versions survive or the effective text is unchanged. This removes the later-survivor exception without a delete operation and avoids appending instructions after already-admitted users on retry. Capture surface generation after reconciliation so subsequent ordinary retries do not emit phantom series headers.

Tests: both compaction cases fail before the fix (stale head, or stale later survivor), then pass with real loop/MockAdapter, pi-ai conversion and source-event reconstruction. Assert two retries keep one assembly/pre-step/user admission, preserve rendered text across section changes, and log one series boundary. Add unchanged-text explicit/tool series regressions. 391 loop tests with exact agent.ts/runtime-context.ts coverage pass at 100%; final admission spec has 11 passing cases. Source tsc and focused lint pass. Update EN/ZH implementation and owning rationale plus lifecycle retry documentation.
2026-09-07 11:09:39 +08:00
Tianyi Cui 2ce9738564 fix(agent-loop): admit prompts under the prepared route
Resolve agent/request and prepareCall inside the accepted open step before committing system and user input. Preserve prompt-assembly model selection, bind capability and dispatch to the same prepared adapter, then log the envelope and derive the request synchronously. Preparation cancellation leaves a balanced empty step with no admitted input.

Normalize incapable-route retained prompt versions with logged per-node empty replacements and a current head, including unchanged rendering and resumed history. Dormant empty tails do not override the effective prompt. No new event or delete operation.

Tests: 389 agent-loop cases pass with exact agent.ts/runtime-context.ts coverage at 100% for all metrics; tsc -b packages/core/agent-loop/tsconfig.json passes. Seven actual-loop MockAdapter cases exercise pi-ai conversion, source-event reconstruction, both route transitions, resume, cancellation barriers, and bound selection. Stale-capability negative control fails all three transition regressions. Update EN/ZH request visibility, lifecycle generator, and owning rationale. Broad catalogs, recorded SDK/GUI and aggregate gates remain parent-owned.
2026-09-07 11:09:39 +08:00
Tianyi Cui c988a6796f feat(agent-loop): append system prompt changes on capable routes
Consolidate the existing in-history capability, loop, UI and artifacts into one baseline. Preserve the reviewed tree so each independently verified correction has a subsequent rationale-rich commit.
2026-09-07 11:09:39 +08:00
Tianyi Cui 71b0d83822 Merge current V3 PTC vocabulary into system-prompt representation
Compose the shared release branch PTC rename with the structural system-head migration in the existing V2-to-V3 stage. Audit source payloads and remap local coordinates before renaming dispatch vocabulary and owned plugin attribution; retain original message identities and generation-qualified references. Native V3 validation combines the same system and PTC lifecycle semantics through a private released-validation view.

Reuse the V3 row-admission owner in current JSONL scanning before recoverable-tail suppression, so malformed rows cannot hide retired required PTC tags or header.system. The new corruption-prefix regressions and all173 existing JSONL scanner tests pass. Combined migration/catalog/persistence suite:300 tests; executable V3 migration source has100% coverage. No released generation, prior migration implementation, shared release branch, benchmark or CI workflow is modified by this integration.
2026-09-06 21:33:35 +08:00
Tianyi Cui 5c8e1b53de Merge latest master into fix/ci-node-compile-cache-data-disk 2026-09-06 21:26:37 +08:00
Tianyi Cui a4dcff874e fix(scripts): validate historical fixture layout with source codecs 2026-09-06 21:18:49 +08:00
Tianyi Cui e9957a2671 test(python): record native V3 delivery provenance
Regenerate advanced and restart expectations through the shipped macOS arm64 SEA runtime built from 0521815d45. Real delivery-accepted events now record sessionFormatVersion 3 in the five V3 logs and advanced SDK notifications. Restart result.json and requests.json regenerate identically. Preserve all five V2 predecessors byte-for-byte from b2b300f9; no production, migration, launcher, or test changes.

Evidence (macOS arm64): pnpm exec tsx scripts/build-exe-for-python-sdk.ts --targets node24-macos-arm64 passed, including the full pnpm build. file dist-exe/deepseek-harness-sdk-runtime-macos-arm64 reports Mach-O 64-bit executable arm64.

For each SCENARIO=sdk-snapshot,sdk-restart: PYTHONDONTWRITEBYTECODE=1 PYTHONPATH=python/sdk/src:python/sdk-runtime/src /Users/cty/dsh-deploy/.agents/worktrees/sp-kv-sdk-evidence/tmp/py-sdk-venv/bin/python scripts/smoke-python-runtime.py --exe dist-exe/deepseek-harness-sdk-runtime-macos-arm64 --scenario SCENARIO --update-snapshots passed; the same two commands without --update-snapshots passed.

PYTHONDONTWRITEBYTECODE=1 PYTHONPATH=python/sdk/src:python/sdk-runtime/src uv run --no-project --python /Users/cty/.local/share/uv/python/cpython-3.14.5-macos-aarch64-none/bin/python3 --with pytest --with pydantic>=2.12,<3 python -m pytest python/sdk/tests/test_smoke_model.py -q: 28 passed. Earlier helper invocations lacked pytest or SDK imports; corrected environment passed without source changes. git diff --quiet b2b300f9 -- scripts/snapshots/python-sdk-single-exe/**/*.v2.jsonl and git diff --cached --check passed.
2026-09-06 20:58:57 +08:00
Tianyi Cui 1a9bef60c2 test: separate native V3 writer expectations from historical replay 2026-09-06 20:45:00 +08:00
Tianyi Cui 8cdc6eba02 test(session): preserve released fixtures with corrected V3 successors
Restore all 147 historical canonical fixtures changed by the lower branch to exact shared base 220ff708e3628a9be46d9747d09af6d8cd742f0d bytes (145 V2 and two V0). All 180 committed V0/V1/V2 canonical files now have zero diff against that base, including unchanged Python, test-support and preview sources.

Preserve corrected lower writer bodies in 146 V3 successors across 119 owner directories: replace 135 unreleased V3 counterparts and add 11 siblings. Change only each copied Session header version; preserve all body bytes, raw model streams, inline images, IDs, references and historical delivery generation markers. Remove two unshipped V2 additions after creating their V3 sibling: record-suite/rec-pin and empty-response-retry-current.

Keep six explicitly historical snapshot owners pinned with no V3 sibling. Restore the two pre-step V0 record child sources without synthesizing current output. Fix seven embedded Session header versions in exactly five test-support behavior owners whose output filenames already select V3. Leave protocol expectations, shared references, sidecars, consumer scripts and production migration/normalization code untouched.

Validation: purpose-built temporary Python inventory and verifier checked 337 canonical headers, 3714 successor JSON rows, message roles, contiguous child roles, exact successor bodies, all 180 historical base bytes and zero git diff, noncanonical JSONL immutability, six retained owners and six unchanged preview fixtures. git diff --cached --check passed. Temporary generators removed. Strict migration, native refresh and replay remain pending parent integration; unsupported pre-step sources are reported separately.
2026-09-06 20:45:00 +08:00
Tianyi Cui c29b97ad94 test(python): refresh system-node and packed restart expectations
The representation PR left child and restart recordings in the old header-system representation while the packaged writer emits system/message before entered user messages. Restart result expectations also retained standalone assistant/chunk notifications after the writer moved stream records into assistant/message. These are stale expected artifacts, not fields to erase in normalization.

Regenerate the owning advanced and restart scenarios through the native macOS ARM64 packaged dsh runtime. Retain messages, tool effects, typed feedback and packed streams; update only the missing system nodes and their sequence references, and remove obsolete standalone chunk notifications. The advanced parent result and parent session already match the writer after the rebase, so this commit changes only two child logs and the restart result/logs.

Validation: pnpm run build; pnpm exec tsx scripts/build-exe-for-python-sdk.ts --skip-build --targets=node24-macos-arm64; uv run --project python/sdk python scripts/smoke-python-runtime.py --scenario sdk-snapshot --exe dist-exe/deepseek-harness-sdk-runtime-macos-arm64 and the equivalent sdk-restart command each reproduced the mismatch, then passed with --update-snapshots and again without it. Both read-only reruns pass. Other native targets remain covered by exact-head CI.
2026-09-06 20:42:43 +08:00
Tianyi Cui ee956c720d refactor(session): represent the system prompt as surface node zero
Consolidate the representation-change PR and its rebase reconciliations into one baseline. Preserve the exact tree and keep the in-history feature in the dependent PR. Follow-up fixes remain separate.
2026-09-06 20:42:43 +08:00
Tianyi Cui 320354a8e0 Merge commit 'adb605ba17936187320cac20bb145d4ee1edb387' into session-v3/ptc-durable-vocabulary 2026-09-06 20:22:48 +08:00
Tianyi Cui 6d11955852 Merge remote-tracking branch 'origin/master' into release/session-log-v3 2026-09-06 20:20:16 +08:00
Tianyi Cui 88e4c3c235 Merge master and preserve hosted benchmark documentation 2026-09-06 19:58:19 +08:00
Tianyi Cui 6f21b112da fix: wait for the Python console runtime on Windows 2026-09-06 19:53:23 +08:00
Tianyi Cui faa33c0ffa revert(ci): run Windows Python runtime CI on GitHub-hosted Windows
Windows x64 runtime builds resolve their hosted matrix.runner unconditionally
again (windows-2025 for pull-request CI). Remove the DSH_CI_FAILOVER_WINDOWS
selector, job-private Python toolchain, self-hosted dependency install and
post-step cleanup, the private setup script, and the routing spec introduced
in #3629. The Windows failover switch again covers only the native Windows
jobs in ci.yml.
2026-09-06 19:17:27 +08:00
Tianyi Cui 368ec64489 Merge corrected Session V3 base into PTC vocabulary 2026-09-06 18:29:14 +08:00
Tianyi Cui e53df8b64a fix(session): guard V3 delivery activation and smoke writer generation 2026-09-06 17:39:15 +08:00
Tianyi Cui f75aabcb16 test(ci): run required benchmarks on standard hosted Linux 2026-09-06 17:10:30 +08:00
Tianyi Cui 000cd7e762 Merge master and reconcile platform failover documentation 2026-09-06 15:35:49 +08:00
Tianyi Cui d15aaee271 Merge pull request #3627 from deepseek-harness/optimize/release-rehearsal-selfhosted
ci: route trusted release rehearsals to self-hosted Linux
2026-09-06 15:22:44 +08:00
Tianyi Cui 37d3ec1681 ci: address Windows runtime routing and isolation review 2026-09-06 15:07:17 +08:00
Tianyi Cui a1188bbf3f ci: contain release temporary installs and document shared routing 2026-09-06 14:57:11 +08:00
Tianyi Cui ceb3136bef ci: size PR previews on measured standard hosted runners (#3628)
* ci: measure hosted preview runner sizes [preview-sizing-benchmark]

* ci: type benchmark job keys explicitly [preview-sizing-benchmark]

* ci: size PR previews on measured standard hosted runners
2026-09-06 14:56:30 +08:00
Tianyi Cui bad4254d71 Rename durable PTC vocabulary with identity-preserving v2 migration 2026-09-06 14:53:16 +08:00
Tianyi Cui 705f84f952 docs(session): document format upgrades and advance current V3 fixtures 2026-09-06 14:01:01 +08:00
Tianyi Cui 3a79abedec ci: use UTF-8 Python text on self-hosted Windows 2026-09-06 13:49:19 +08:00
Tianyi Cui 69b1e315b4 ci: use isolated self-hosted Windows Python runtime builds 2026-09-06 13:21:26 +08:00
Tianyi Cui f66dd78ff7 ci: route trusted release rehearsals to self-hosted Linux 2026-09-06 13:01:55 +08:00
imccyu 9b78f99dec perf(session): transfer frozen persistence reads into restore 2026-09-06 02:10:23 +08:00
imccyu ec2f63dbdb perf(session-persistence): stream migration publication and verification 2026-09-06 02:05:40 +08:00
imccyu 46196d6f95 perf(session-format): stream released v0-to-v2 migrations 2026-09-06 02:05:27 +08:00
imccyu a548150f86 test(perf): isolate benchmark dependencies 2026-09-06 01:01:04 +08:00
Tianyi Cui 74e27c97d0 test(perf): list the bench config among proxy-setup Vitest inventories
scripts/test-proxy-environment.spec.ts pins the Vitest configs that declare
setupFiles so the proxy isolation setup is never dropped; the benchmark
lane declares it too.
2026-09-06 01:01:03 +08:00
Tianyi Cui 2395f112d8 test(perf): gate opening large Sessions with synthetic CI benchmarks
Add a benchmark lane (`vitest.bench.config.ts`, `pnpm run test:bench`,
gate mode `ci-bench`) and a required `node 24 / benchmarks` CI job that
runs it alone. Benchmarks synthesize their input in-process from fixed
parameters and fail on documented budgets:

- `open-generation.bench.ts`: a 200-turn released-v0 log with 500 text
  and 125 reasoning deltas per reply (127,400 events, ~2.8 MB) encoded
  through the frozen v0 codec; the migrating first `open()` must finish
  within 2,000 ms in a child process capped at 128 MB of old space, and a
  fresh process must open the published current generation within 500 ms.
- `conversation-fold.bench.client.ts`: 200 replies whose compact streams
  hold 2,000 text + 500 reasoning deltas each, folded through every Chat
  Definition by the real assembler; the fold must finish within 150 ms
  and stay within 3x the fold of the same window with 100 deltas per
  reply.

On this commit both gates fail: the migration exhausts the 128 MB heap
(4.8 s and 696 MB peak RSS without the cap; the pre-stack decode of the
same bytes took 34 ms and 168 MB) and the fold scales 11x with the delta
count. The stacked fixes bring both paths to O(records).
2026-09-06 01:01:03 +08:00