diff --git a/.agents/notes/implemented/bug-fix/2026-09-06-windows-python-console-spawn-wait.i18n.yaml b/.agents/notes/implemented/bug-fix/2026-09-06-windows-python-console-spawn-wait.i18n.yaml new file mode 100644 index 0000000000..652b921960 --- /dev/null +++ b/.agents/notes/implemented/bug-fix/2026-09-06-windows-python-console-spawn-wait.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write .agents/notes/implemented/bug-fix/2026-09-06-windows-python-console-spawn-wait.md +2026-09-06-windows-python-console-spawn-wait.md: 92443bcf8a6e4e5609dc469efa4ebd1d82ab127f +2026-09-06-windows-python-console-spawn-wait.zh.md: dba2f324b29955580fc11e7cea7a0525a8bc8c86 diff --git a/.agents/notes/implemented/bug-fix/2026-09-06-windows-python-console-spawn-wait.md b/.agents/notes/implemented/bug-fix/2026-09-06-windows-python-console-spawn-wait.md new file mode 100644 index 0000000000..92443bcf8a --- /dev/null +++ b/.agents/notes/implemented/bug-fix/2026-09-06-windows-python-console-spawn-wait.md @@ -0,0 +1,27 @@ +# Agent Note: Wait for the Windows Python console runtime + +Status: implemented + +English | [中文](2026-09-06-windows-python-console-spawn-wait.zh.md) + +## Problem + +The installed Python `dsh.exe` console command intermittently exits with Windows access violation `0xc0000005` before initializing a profile. Its smoke assertion omitted the process status and reported only empty streams. A [native faulthandler probe](https://github.com/deepseek-harness/deepseek-harness/actions/runs/34030851888) captures the fault in Python 3.10 `os._execvpe`, called by the runtime console entry, rather than in the bundled Node executable. Direct executable controls pass. + +## Decision + +The [Python console entry](../../../../python/sdk-runtime/src/deepseek_harness_runtime/__init__.py) uses `subprocess.run` on Windows, inherits standard streams and environment, waits for runtime completion, and exits with the runtime status. POSIX retains `os.execvpe` process replacement. Windows CRT exec is not POSIX process replacement; the explicit spawn-and-wait path avoids the observed native exec operation. + +The [installed-wheel smoke](../../../../scripts/smoke-python-runtime.py) reports decimal and unsigned 32-bit hexadecimal status alongside captured streams when profile installation fails. This preserves the distinction between ordinary command failure and native process exceptions. + +## Alternatives considered + +**Disable Node compile caching.** Not selected: cache environment changes correlated with early probes, but cold-cache controls also passed and Python faulthandler locates the actual fault at the native exec call. Cache configuration remains unchanged. + +**Retry or bypass the installed console command.** Rejected because either masks the shipped command failure instead of repairing its process launch. The keyless installed-wheel assertion remains required. + +## Consequences + +Windows keeps a Python parent until the runtime exits; it no longer depends on CRT overlay behavior. The standard synchronous subprocess implementation owns waiting and interruption cleanup. No custom process-tree manager or global host setting is added. + +[Runtime-resolution tests](../../../../python/sdk/tests/test_runtime_resolution.py) retain POSIX forwarding and cover Windows argument/environment forwarding, statuses 0/37/513, real child completion, Unicode streams and arguments with spaces. Native Windows owns the wide exit-status case because POSIX truncates process statuses to eight bits. The [native fixed-count comparison](https://github.com/deepseek-harness/deepseek-harness/actions/runs/34031142773) passes all four patched launches with compile caching enabled; all four unpatched controls also pass in that batch, so it is not a same-batch reproduction. Full installed-wheel CI must validate the final artifact separately from local branch-level tests. diff --git a/.agents/notes/implemented/bug-fix/2026-09-06-windows-python-console-spawn-wait.zh.md b/.agents/notes/implemented/bug-fix/2026-09-06-windows-python-console-spawn-wait.zh.md new file mode 100644 index 0000000000..dba2f324b2 --- /dev/null +++ b/.agents/notes/implemented/bug-fix/2026-09-06-windows-python-console-spawn-wait.zh.md @@ -0,0 +1,27 @@ +# Agent Note: 等待 Windows Python 控制台运行时 + +Status: implemented + +[English](2026-09-06-windows-python-console-spawn-wait.md) | 中文 + +## 问题 + +Python 安装的 `dsh.exe` 控制台命令会在初始化 profile 前间歇性地以 Windows 访问冲突 `0xc0000005` 退出。其冒烟断言遗漏进程状态，只报告空标准流。[原生 faulthandler 探测](https://github.com/deepseek-harness/deepseek-harness/actions/runs/34030851888) 将故障定位在运行时控制台入口调用的 Python 3.10 `os._execvpe`，而非打包的 Node 可执行文件。直接启动可执行文件的对照通过。 + +## 决策 + +[Python 控制台入口](../../../../python/sdk-runtime/src/deepseek_harness_runtime/__init__.py) 在 Windows 上使用 `subprocess.run`，继承标准流与环境，等待运行时结束，再以运行时状态退出。POSIX 保留 `os.execvpe` 进程替换。Windows CRT exec 并非 POSIX 进程替换；显式启动并等待的路径避开观测到的原生 exec 操作。 + +[安装后 wheel 冒烟测试](../../../../scripts/smoke-python-runtime.py) 在 profile 安装失败时，同时报告十进制、无符号 32 位十六进制状态与捕获的标准流。这保留普通命令失败和原生进程异常的区别。 + +## 已考虑的替代方案 + +**禁用 Node 编译缓存。** 未采用：早期探测中缓存环境变化与结果相关，但冷缓存对照也能通过，且 Python faulthandler 将实际故障定位在原生 exec 调用。缓存配置保持不变。 + +**重试或绕过已安装的控制台命令。** 拒绝，因为二者都会掩盖已发布命令的失败，而不是修复进程启动。keyless 安装后 wheel 断言仍为必需检查。 + +## 后果 + +Windows 保留 Python 父进程直到运行时退出，不再依赖 CRT overlay 行为。标准同步子进程实现负责等待和中断清理。不添加自定义进程树管理器或全局主机设置。 + +[运行时解析测试](../../../../python/sdk/tests/test_runtime_resolution.py) 保留 POSIX 转发验证，并覆盖 Windows 参数／环境转发、状态 0/37/513、真实子进程完成、Unicode 标准流和带空格的参数。宽退出状态由原生 Windows 验证，因为 POSIX 会将进程状态截断为八位。[原生固定次数对照](https://github.com/deepseek-harness/deepseek-harness/actions/runs/34031142773) 中，启用编译缓存的四次修复后启动全部通过；该批次四次未修复对照也全部通过，因此它不是同批次复现。完整安装后 wheel CI 必须独立于本地分支级测试，验证最终产物。 diff --git a/.agents/notes/implemented/process/2026-07-26-ci-failover-runbook.i18n.yaml b/.agents/notes/implemented/process/2026-07-26-ci-failover-runbook.i18n.yaml index 8d50d7a5f9..eef19973a4 100644 --- a/.agents/notes/implemented/process/2026-07-26-ci-failover-runbook.i18n.yaml +++ b/.agents/notes/implemented/process/2026-07-26-ci-failover-runbook.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/process/2026-07-26-ci-failover-runbook.md -2026-07-26-ci-failover-runbook.md: b24996a4ba4dfaa4b26f88519a61f45c81efb5b5 -2026-07-26-ci-failover-runbook.zh.md: ee7339d70e4796c367f97490ea57468687464b3f +2026-07-26-ci-failover-runbook.md: d5c12492671941c45cf3ccab255dd76fb53773bd +2026-07-26-ci-failover-runbook.zh.md: b42a14dc9a3e4f40c59b27633752f6969173777e diff --git a/.agents/notes/implemented/process/2026-07-26-ci-failover-runbook.md b/.agents/notes/implemented/process/2026-07-26-ci-failover-runbook.md index b24996a4ba..d5c1249267 100644 --- a/.agents/notes/implemented/process/2026-07-26-ci-failover-runbook.md +++ b/.agents/notes/implemented/process/2026-07-26-ci-failover-runbook.md @@ -6,11 +6,11 @@ English | [中文](2026-07-26-ci-failover-runbook.zh.md) ## Problem -The three required Linux worker jobs in [CI](../../../../.github/workflows/ci.yml) (`node 24 / static`, `node 24 / coverage`, `node 24 / snapshots and artifacts`) run on the hosted enterprise 32-core pools; the required verdict job that aggregates them (`all checks passed`) runs on standard `ubuntu-latest`; the independent native Windows job (`windows node 24 / native complete`) runs on the hosted `dsh-windows-2025-16core` larger runner. When the enterprise pools degrade — jobs queue indefinitely or the enterprise labels vanish — every open pull request becomes unmergeable, and the ordinary recovery of merging a fix is itself deadlocked behind the very required checks that cannot run. **Scope: two independent switches, one per platform.** `DSH_CI_FAILOVER_LINUX` recovers an enterprise Linux-pool outage (the three required Linux workers plus the `all checks passed` verdict); `DSH_CI_FAILOVER_WINDOWS` recovers a hosted Windows-pool outage (the native Windows job). A Linux-pool outage need not retarget the native Windows job and vice versa. The verdict's other required dependencies (`node-compat`, `python-sdk`, `windows`) stay on standard hosted runners by design (the portable boundary); in a broader GitHub-hosted capacity failure that also takes out the standard pools, those dependencies still block `all checks passed`. An outage therefore needs a switch any responder with repository write access can throw without merging anything. +The three required Linux worker jobs in [CI](../../../../.github/workflows/ci.yml) (`node 24 / static`, `node 24 / coverage`, `node 24 / snapshots and artifacts`) run on the hosted enterprise 32-core pools; the required verdict job that aggregates them (`all checks passed`) runs on standard `ubuntu-latest`; the [native Windows jobs](2026-08-08-native-windows-pull-request-ci.md) run on the hosted `dsh-windows-2025-16core` larger runner. When the enterprise pools degrade — jobs queue indefinitely or the enterprise labels vanish — every open pull request becomes unmergeable, and the ordinary recovery of merging a fix is itself deadlocked behind the very required checks that cannot run. **Scope: two independent switches, one per platform.** `DSH_CI_FAILOVER_LINUX` recovers an enterprise Linux-pool outage (the three required Linux workers plus the `all checks passed` verdict); `DSH_CI_FAILOVER_WINDOWS` recovers a hosted Windows-pool outage (the native Windows jobs). A Linux-pool outage need not retarget Windows jobs and vice versa. The verdict's other required dependencies (`node-24-bench`, `node-compat`, `python-sdk`, `windows`) stay on standard hosted runners by design (the portable boundary); in a broader GitHub-hosted capacity failure that also takes out the standard pools, those dependencies still block `all checks passed`. An outage therefore needs a switch any responder with repository write access can throw without merging anything. ## Decision -Each of the three required Linux worker jobs, the independent native Windows job, and the `all checks passed` verdict job — which would otherwise stay queued on the failed pool even after every worker passed — resolves its runner pool through a repository variable, and the switch is split by platform so an outage on one platform does not retarget the other. The three Linux workers and the `all checks passed` verdict (whose `needs` are the required Linux workers and which runs on the `vm-backup` pool) resolve through `DSH_CI_FAILOVER_LINUX`; the native Windows job resolves through `DSH_CI_FAILOVER_WINDOWS`. Unset (normal), they run on the hosted enterprise pools. Set to `selfhosted` by any repository writer, the corresponding jobs retarget onto the in-house self-hosted pool: under `DSH_CI_FAILOVER_LINUX`, the Linux jobs and verdict move onto the `vm-backup` pool, snapshot concurrency drops to the shared-VM bound, and the hosted-path pnpm cache restores are skipped; under `DSH_CI_FAILOVER_WINDOWS`, the native Windows job moves onto the `dsh-win-ci` pool. Each switch is writer-manageable repository state, not a merge, so it works while every check is red. The in-house pools' readiness is continuously re-proven by the `serial / linux (self-hosted standby)` and `serial / windows (self-hosted standby)` lanes, which run the complete unsharded aggregates on every master push. +Each of the three required Linux worker jobs, the native Windows jobs, and the `all checks passed` verdict job — which would otherwise stay queued on the failed pool even after every worker passed — resolves its runner pool through a repository variable, and the switch is split by platform so an outage on one platform does not retarget the other. The three Linux workers and the `all checks passed` verdict (whose `needs` are the required Linux workers and which runs on the `vm-backup` pool) resolve through `DSH_CI_FAILOVER_LINUX`; the native Windows jobs resolve through `DSH_CI_FAILOVER_WINDOWS`. Unset, they default to their hosted pools; selecting `selfhosted` is an explicit operator choice. Set to `selfhosted` by any repository writer, the corresponding jobs retarget onto the in-house self-hosted pool: under `DSH_CI_FAILOVER_LINUX`, the Linux jobs and verdict move onto the `vm-backup` pool, snapshot concurrency drops to the shared-VM bound, and the hosted-path pnpm cache restores are skipped; under `DSH_CI_FAILOVER_WINDOWS`, the native Windows jobs move onto the `dsh-win-ci` pool. Each switch is writer-manageable repository state, not a merge, so it works while every check is red. The in-house pools' readiness is continuously re-proven by the `serial / linux (self-hosted standby)` and `serial / windows (self-hosted standby)` lanes, which run the complete unsharded aggregates on every master push. `ci-master.yml` exempts exactly one event from `cancel-in-progress` (`${{ github.event_name != 'push' }}`), so one master push does not cancel the drill still running from the previous one. Each drill runs its complete unsharded aggregate with one gate worker, which takes longer than the interval between master merges; under unconditional cancellation a drill is superseded before reaching a verdict and the lane yields no readiness evidence for a responder to check. @@ -18,13 +18,17 @@ The exemption is narrower than "a drill always finishes", in two ways. GitHub ke The decision belongs at workflow level because cancellation applies to the whole superseded run: a job-level `concurrency` group does not exempt its job. The negated form is load-bearing rather than cosmetic: naming `pull_request` alone would also stop cancelling `workflow_dispatch`, and each runner benchmark fans out to twelve larger runners for up to fifteen minutes inside this same group on master, so a re-dispatch would queue ahead of a drill instead of replacing a stale measurement. What bounds the cost is that a master push in `ci-master.yml` carries only `wine-apt-cache` and these two drills; the pull-request jobs live in the separate `ci.yml` (which does not see `push`), and the benchmarks are `workflow_dispatch`-gated within `ci-master.yml`. `scripts/ci-workflow.spec.ts` pins that push-reachable set — classifying by exact condition, since a negated event test mentions the event it excludes — so a new push-reachable job cannot quietly start accumulating uncancelled runs. +### Release rehearsals share the Linux switch + +`DSH_CI_FAILOVER_LINUX=selfhosted` also routes the credential-free dependency-layout job and both dsh/vendor pack jobs onto `vm-backup` for eligible same-repository PRs and master pushes. Their [release rehearsal decision](2026-09-06-release-rehearsal-selfhosted.md) owns the stricter event eligibility and hosted manual dispatch. This coupling is intentional: keeping the variable set to save release minutes also keeps the eligible main-CI Linux jobs self-hosted. Clearing it returns both workloads to their hosted targets for subsequent runs; publication stays hosted regardless. + ### What the in-house pool is -`vm-backup`: one 64-core VM, six always-on systemd-managed runner instances. Its image must preinstall Playwright Chromium's Linux system packages; CI downloads the lockfile-selected browser but never runs `apt` on this persistent shared host. Check the latest `serial / linux (self-hosted standby)` run before switching: its aggregate includes browser replay, so a green standby verifies both ordinary capacity and this browser prerequisite. +`vm-backup`: one shared VM with multiple always-on systemd-managed runner instances. Registrations share its CPU, memory, and disk; their count is not a count of independent machines. Its image must preinstall Playwright Chromium's Linux system packages; CI downloads the lockfile-selected browser but never runs `apt` on this persistent shared host. Check the latest `serial / linux (self-hosted standby)` run before switching: its aggregate includes browser replay, so a green standby verifies both ordinary capacity and this browser prerequisite. #### Windows pool -`dsh-win-ci`: 32 always-on runner instances (scheduled tasks `GH-Runner-01`…`GH-Runner-32`) on the in-house Windows CI server (one 96-core / 580 GB machine). Labels: `[self-hosted, dsh-win-ci, windows]`. The image must preinstall Node 24, pnpm, Git (with Git Bash on `PATH`, i.e. `C:\Program Files\Git\bin` — the `bash` tool spawns `bash` by name), PowerShell 7, and enable Developer Mode for symlink support. The workspaces and the pnpm store must both live on a ReFS volume (`F:`): the Windows installs pass `--package-import-method=clone` on ReFS, which needs that volume layout and the `@reflink/reflink` native module that the system corepack pnpm carries (see [the Windows ReFS store note](../../archived/process/2026-08-30-windows-refs-store-block-clone-install.md)); a rebuilt runner without this layout fails the Windows build gates with TS6231. Check the latest `serial / windows (self-hosted standby)` run before switching: a green standby verifies the pool can execute `check:ci:windows-complete` end-to-end. +`dsh-win-ci`: 32 always-on runner instances (scheduled tasks `GH-Runner-01`…`GH-Runner-32`) on the in-house Windows CI server (one 96-core / 580 GB machine). Labels: `[self-hosted, dsh-win-ci, windows]`. The image must preinstall Node 24, pnpm, Git (with Git Bash on `PATH`, i.e. `C:\Program Files\Git\bin` — the `bash` tool spawns `bash` by name), PowerShell 7, and enable Developer Mode for symlink support. The general-purpose Windows workspaces and pnpm store must both live on a ReFS volume (`F:`): those installs pass `--package-import-method=clone` on ReFS, which needs that volume layout and the `@reflink/reflink` native module that the system corepack pnpm carries (see [the Windows ReFS store note](../../archived/process/2026-08-30-windows-refs-store-block-clone-install.md)); a rebuilt runner without this layout fails the Windows build gates with TS6231. Check the latest `serial / windows (self-hosted standby)` run before switching: a green standby verifies the pool can execute `check:ci:windows-complete` end-to-end. ### Switch (any repository writer, ~1 minute, no merge) @@ -32,7 +36,7 @@ The two switches are independent: flip only the one whose platform is degraded. 1. Repository **Settings → Secrets and variables → Actions → Variables → New repository variable**: name `DSH_CI_FAILOVER_LINUX` (Linux pool outage) or `DSH_CI_FAILOVER_WINDOWS` (Windows pool outage), value `selfhosted`. 2. Retrigger the required jobs so they re-resolve their pool. Jobs already **queued** for the hosted labels do not retarget and cannot be re-run in place, so for the documented indefinite-queue outage, cancel the stuck run and re-run all jobs, or push a new commit; "Re-run failed jobs" only helps once a job has actually failed rather than queued. -3. That is the entire switch. Under Linux failover the workflow also drops `DSH_SNAPSHOT_MAX_CONCURRENCY` to 12 for the shared VM and skips the hosted-path pnpm cache restores because the VM's persistent store serves warm installs. Coverage uses the same four single-worker instrumented partitions and two exempt workers on both Linux pools. The Windows switch has no concurrency or cache branches; it only retargets the native Windows job's pool. +3. That is the entire switch. Under Linux failover the workflow also drops `DSH_SNAPSHOT_MAX_CONCURRENCY` to 12 for the shared VM and skips the hosted-path pnpm cache restores because the VM's persistent store serves warm installs. Coverage uses the same four single-worker instrumented partitions and two exempt workers on both Linux pools. The Windows switch has no concurrency or cache branches; it only retargets the native Windows jobs' pool. #**Dependabot exception.** Both switches' selectors deliberately exclude `dependabot[bot]`: under failover, Dependabot PRs stay queued for the hosted pool rather than executing dependency-supplied code on the persistent VMs. A Dependabot PR that remains queued during an outage is expected behavior, not a failed switch; it completes when the hosted pool recovers. @@ -40,12 +44,12 @@ The two switches are independent: flip only the one whose platform is degraded. ## Capacity during failover -Six always-on instances absorb normal PR traffic (the pool's steady-state load is one serial standby job per master push, so failover capacity is effectively the full pool). If queues still build, register additional instances with an org registration token (org Settings → Actions → Runners → New runner). Clone an existing runner directory **excluding its identity files** — `rsync -a --exclude '.runner*' --exclude '.credentials*' --exclude '_diag' --exclude '_work' / /` (the globs also catch `.runner_migrated`/`.credentials_migrated`, which GitHub writes on migrated runners and which equally trigger the already-configured refusal) — then run `config.sh` (copying `.runner`/`.credentials` verbatim makes it refuse with "already configured"), and **start the listener**: `sudo ./svc.sh install ubuntu && sudo ./svc.sh start`. Registration alone leaves the runner offline; only a started service adds capacity. About a minute per instance. +Capacity includes the master standby, main-CI jobs, and three release-rehearsal jobs for each eligible PR or master push while the Linux switch is set. The release workflows do not cancel running rehearsals when another run arrives, so overlapping refs can add sustained build, pack, and install load. Check current CPU, memory, disk, and queue pressure before extending self-hosted operation; extra registrations on this VM add scheduling slots, not machine resources. Do not infer spare capacity from the standby alone. When host resources permit extra registrations, use an org registration token (org Settings → Actions → Runners → New runner). Clone an existing runner directory **excluding its identity files** — `rsync -a --exclude '.runner*' --exclude '.credentials*' --exclude '_diag' --exclude '_work' / /` (the globs also catch `.runner_migrated`/`.credentials_migrated`, which GitHub writes on migrated runners and which equally trigger the already-configured refusal) — then run `config.sh` (copying `.runner`/`.credentials` verbatim makes it refuse with "already configured"), and **start the listener**: `sudo ./svc.sh install ubuntu && sudo ./svc.sh start`. Registration alone leaves the runner offline; a started service adds a scheduling slot, not CPU or memory. ### Switch back -Delete the `DSH_CI_FAILOVER_LINUX` or `DSH_CI_FAILOVER_WINDOWS` variable (or set it to anything other than `selfhosted`). New runs resolve back to the hosted enterprise pools. Remove any extra instances that were registered during the incident. +Delete the `DSH_CI_FAILOVER_LINUX` or `DSH_CI_FAILOVER_WINDOWS` variable (or set it to anything other than `selfhosted`). New runs resolve back to their hosted pools. Remove any extra instances that were registered during the incident. ### Trust boundary @@ -55,7 +59,7 @@ The variables are writer-manageable repository state; a pull request event itsel **Merge a workflow change to switch pools.** Rejected because the outage that motivates the switch is exactly the state in which no PR can merge: the required checks are the ones failing. A repository variable is writer-manageable state that takes effect on re-run without a merge. -**Keep the self-hosted pool always in the required path.** Rejected because it trades hosted-pool availability for the in-house VM's, moving a single point of failure rather than adding a fallback. The variables keep the hosted pools primary and the self-hosted pools proven, one-action standbys; splitting them by platform means an outage on one platform does not retarget the other. +**Keep the self-hosted pool always in the required path.** Rejected because it trades hosted-pool availability for the in-house VM's, moving a single point of failure rather than adding a fallback. The unset defaults retain hosted targets and the switches provide a reversible, operator-selected self-hosted path; splitting them by platform means an outage on one platform does not retarget the other. ## Consequences diff --git a/.agents/notes/implemented/process/2026-07-26-ci-failover-runbook.zh.md b/.agents/notes/implemented/process/2026-07-26-ci-failover-runbook.zh.md index ee7339d70e..b42a14dc9a 100644 --- a/.agents/notes/implemented/process/2026-07-26-ci-failover-runbook.zh.md +++ b/.agents/notes/implemented/process/2026-07-26-ci-failover-runbook.zh.md @@ -6,11 +6,11 @@ Status: implemented ## 问题 -[CI](../../../../.github/workflows/ci.yml) 中三个必需的 Linux 工作作业（`node 24 / static`、`node 24 / coverage`、`node 24 / snapshots and artifacts`）运行在托管的企业级 32 核池上；聚合它们的必需判定作业（`all checks passed`）运行在标准 `ubuntu-latest` 上；独立的原生 Windows 作业（`windows node 24 / native complete`）运行在托管的 `dsh-windows-2025-16core` 大型运行器上。当企业池发生故障——作业无限排队或企业标签消失——所有开启的拉取请求都无法合并，而"合并一个修复"这一常规恢复手段本身正被那些无法运行的必需检查死锁。**适用范围：两个独立开关，每个平台一个。**`DSH_CI_FAILOVER_LINUX` 恢复企业级 Linux 池故障（三个必需的 Linux 工作作业加 `all checks passed` 判定作业）；`DSH_CI_FAILOVER_WINDOWS` 恢复托管 Windows 池故障（原生 Windows 作业）。Linux 池故障无需重定向原生 Windows 作业，反之亦然。判定作业的其余必需依赖（`node-compat`、`python-sdk`、`windows`）按设计留在标准托管运行器上（可移植边界）；若更大范围的 GitHub 托管容量故障连标准池一并击倒，这些依赖仍会阻塞 `all checks passed`。因此故障需要一个任何具备仓库写权限的响应者都能在不合并任何代码的情况下触发的开关。 +[CI](../../../../.github/workflows/ci.yml) 中三个必需的 Linux 工作作业（`node 24 / static`、`node 24 / coverage`、`node 24 / snapshots and artifacts`）运行在托管的企业级 32 核池上；聚合它们的必需判定作业（`all checks passed`）运行在标准 `ubuntu-latest` 上；[原生 Windows 作业](2026-08-08-native-windows-pull-request-ci.zh.md)运行在托管的 `dsh-windows-2025-16core` 大型运行器上。当企业池发生故障——作业无限排队或企业标签消失——所有开启的拉取请求都无法合并，而"合并一个修复"这一常规恢复手段本身正被那些无法运行的必需检查死锁。**适用范围：两个独立开关，每个平台一个。**`DSH_CI_FAILOVER_LINUX` 恢复企业级 Linux 池故障（三个必需的 Linux 工作作业加 `all checks passed` 判定作业）；`DSH_CI_FAILOVER_WINDOWS` 恢复托管 Windows 池故障（原生 Windows 作业）。Linux 池故障无需重定向 Windows 作业，反之亦然。判定作业的其余必需依赖（`node-24-bench`、`node-compat`、`python-sdk`、`windows`）按设计留在标准托管运行器上（可移植边界）；若更大范围的 GitHub 托管容量故障连标准池一并击倒，这些依赖仍会阻塞 `all checks passed`。因此故障需要一个任何具备仓库写权限的响应者都能在不合并任何代码的情况下触发的开关。 ## 决策 -三个必需的 Linux 工作作业、独立的原生 Windows 作业，以及 `all checks passed` 判定作业（若不随切换，即使全部工作作业通过，它仍会滞留在故障池的队列中）——各自通过仓库变量解析运行器池，且开关按平台拆分，使一个平台的故障不会重定向另一个平台。三个 Linux 工作作业与 `all checks passed` 判定作业（其 `needs` 是必需的 Linux 工作作业，且运行在 `vm-backup` 池上）通过 `DSH_CI_FAILOVER_LINUX` 解析；原生 Windows 作业通过 `DSH_CI_FAILOVER_WINDOWS` 解析。变量不存在（正常）时它们运行在托管企业池上；由任何具备写权限的协作者设为 `selfhosted` 时，对应作业切换到公司自有的自托管池：`DSH_CI_FAILOVER_LINUX` 下，Linux 作业与判定作业切到 `vm-backup` 池，快照并发降到共享虚拟机上限，并跳过托管路径的 pnpm 缓存恢复；`DSH_CI_FAILOVER_WINDOWS` 下，原生 Windows 作业切到 `dsh-win-ci` 池。每个开关都是写者可管理的仓库状态而非一次合并，因此在所有检查都是红色时仍然有效。自有池的就绪状态由 `serial / linux (self-hosted standby)` 与 `serial / windows (self-hosted standby)` 通道持续验证——每次 master 推送都在其上运行完整的未分片聚合流程。 +三个必需的 Linux 工作作业、原生 Windows 作业，以及 `all checks passed` 判定作业（若不随切换，即使全部工作作业通过，它仍会滞留在故障池的队列中）——各自通过仓库变量解析运行器池，且开关按平台拆分，使一个平台的故障不会重定向另一个平台。三个 Linux 工作作业与 `all checks passed` 判定作业（其 `needs` 是必需的 Linux 工作作业，且运行在 `vm-backup` 池上）通过 `DSH_CI_FAILOVER_LINUX` 解析；原生 Windows 作业通过 `DSH_CI_FAILOVER_WINDOWS` 解析。未设置变量时默认使用各自的托管池；选择 `selfhosted` 是运维人员的明确操作；由任何具备写权限的协作者设为 `selfhosted` 时，对应作业切换到公司自有的自托管池：`DSH_CI_FAILOVER_LINUX` 下，Linux 作业与判定作业切到 `vm-backup` 池，快照并发降到共享虚拟机上限，并跳过托管路径的 pnpm 缓存恢复；`DSH_CI_FAILOVER_WINDOWS` 下，原生 Windows 作业切到 `dsh-win-ci` 池。每个开关都是写者可管理的仓库状态而非一次合并，因此在所有检查都是红色时仍然有效。自有池的就绪状态由 `serial / linux (self-hosted standby)` 与 `serial / windows (self-hosted standby)` 通道持续验证——每次 master 推送都在其上运行完整的未分片聚合流程。 `ci-master.yml` 只豁免一个事件不做取消（`${{ github.event_name != 'push' }}`），因此一次 master 推送不会取消上一次推送留下的、仍在运行的演练。每次演练以单门禁工作进程执行完整的未分片聚合流程，耗时长于 master 合并的间隔；在无条件取消下，演练会在得出结论前被后续运行取代，该通道无法产出供响应者查看的就绪证据。 @@ -18,13 +18,17 @@ Status: implemented 这个决定必须放在工作流级：取消作用于被取代的整个运行，作业级 `concurrency` 组并不能豁免其所属作业。采用否定式写法而非仅指名 `pull_request`，是有实质作用的：后者会连 `workflow_dispatch` 一起停止取消，而每次运行器基准测试会在 master 上的同一并发组内同时占用 12 台大规格运行器、最长 15 分钟，届时重复派发会排在演练之前，而不是替换掉已过时的测量。成本之所以可控，是因为 `ci-master.yml` 中一次 master 推送只承载 `wine-apt-cache` 和这两条演练；拉取请求作业位于独立的 `ci.yml`（不监听 `push`），而基准测试在 `ci-master.yml` 内受 `workflow_dispatch` 门控。`scripts/ci-workflow.spec.ts` 会锁定这个推送可达集合——按条件精确匹配，因为否定式事件判断会包含它所排除的事件名——使新的推送可达作业无法悄悄开始累积未取消的运行。 +### 发布演练共用 Linux 开关 + +`DSH_CI_FAILOVER_LINUX=selfhosted` 还会将符合条件的同仓库 PR 和 master 推送中的无凭据依赖布局作业与 dsh/vendor 两个打包作业路由到 `vm-backup`。[发布演练决策](2026-09-06-release-rehearsal-selfhosted.zh.md) 负责更严格的事件准入规则及保留托管的手动触发。这种耦合是有意的：持续设置变量来节省发布分钟，也会让符合条件的主 CI Linux 作业持续使用自托管。清除变量会让两类负载的后续运行返回各自的托管目标；发布操作始终保留托管。 + ### 自有池是什么 -`vm-backup`：一台 64 核虚拟机，6 个常驻 systemd 管理的运行器实例。其镜像必须预装 Playwright Chromium 的 Linux 系统软件包；CI 会下载锁文件选定的浏览器，但绝不在这台持久化共享主机上运行 `apt`。切换前先看 `serial / linux (self-hosted standby)` 最近一次运行：其聚合流程包含浏览器回放，因此绿色热备同时验证常规容量和这项浏览器先决条件。 +`vm-backup`：一台共享虚拟机，运行多个常驻 systemd 管理的运行器实例。注册实例共享 CPU、内存和磁盘；实例数量不代表独立机器数量。其镜像必须预装 Playwright Chromium 的 Linux 系统软件包；CI 会下载锁文件选定的浏览器，但绝不在这台持久化共享主机上运行 `apt`。切换前先看 `serial / linux (self-hosted standby)` 最近一次运行：其聚合流程包含浏览器回放，因此绿色热备同时验证常规容量和这项浏览器先决条件。 #### Windows 池 -`dsh-win-ci`：公司内部 Windows CI 服务器（一台 96 核 / 580 GB 机器）上 32 个常驻运行器实例（计划任务 `GH-Runner-01`…`GH-Runner-32`）。标签：`[self-hosted, dsh-win-ci, windows]`。镜像必须预装 Node 24、pnpm、Git（Git Bash 在 `PATH` 上，即 `C:\Program Files\Git\bin`——`bash` 工具按名称 spawn `bash`）、PowerShell 7，并为符号链接支持启用开发人员模式。工作区与 pnpm store 必须都位于 ReFS 卷（`F:`）上：Windows 安装步骤在 ReFS 上传递 `--package-import-method=clone`，这需要该卷布局以及系统 corepack pnpm 携带的 `@reflink/reflink` 原生模块（见 [Windows ReFS store note](../../archived/process/2026-08-30-windows-refs-store-block-clone-install.md)）；没有此布局的重建运行器会在 Windows 构建门禁阶段以 TS6231 失败。切换前先看 `serial / windows (self-hosted standby)` 最近一次运行：绿色热备验证该池能端到端执行 `check:ci:windows-complete`。 +`dsh-win-ci`：公司内部 Windows CI 服务器（一台 96 核 / 580 GB 机器）上 32 个常驻运行器实例（计划任务 `GH-Runner-01`…`GH-Runner-32`）。标签：`[self-hosted, dsh-win-ci, windows]`。镜像必须预装 Node 24、pnpm、Git（Git Bash 在 `PATH` 上，即 `C:\Program Files\Git\bin`——`bash` 工具按名称 spawn `bash`）、PowerShell 7，并为符号链接支持启用开发人员模式。通用 Windows 通道的工作区与 pnpm store 必须都位于 ReFS 卷（`F:`）上：这些安装步骤在 ReFS 上传递 `--package-import-method=clone`，这需要该卷布局以及系统 corepack pnpm 携带的 `@reflink/reflink` 原生模块（见 [Windows ReFS store note](../../archived/process/2026-08-30-windows-refs-store-block-clone-install.md)）；没有此布局的重建运行器会在 Windows 构建门禁阶段以 TS6231 失败。切换前先看 `serial / windows (self-hosted standby)` 最近一次运行：绿色热备验证该池能端到端执行 `check:ci:windows-complete`。 ### 切换步骤（任何具备写权限的协作者，约 1 分钟，无需合并） @@ -40,12 +44,12 @@ Status: implemented ## 切换期间的容量 -6 个常驻实例可承接正常 PR 流量（该池平时唯一的稳态负载是每次 master 推送一个串行热备作业，故障切换时几乎全池可用）。若仍出现排队，用组织级注册 token（组织 Settings → Actions → Runners → New runner）追加注册实例。复制现有 runner 目录时**必须排除身份文件**——`rsync -a --exclude '.runner*' --exclude '.credentials*' --exclude '_diag' --exclude '_work' / /`（通配同时排除 `.runner_migrated`/`.credentials_migrated`——GitHub 会在迁移过的运行器上写入这些文件，它们同样会触发 already-configured 拒绝）——再跑 `config.sh`（原样拷贝 `.runner`/`.credentials` 会使其以 "already configured" 拒绝），然后**启动监听器**：`sudo ./svc.sh install ubuntu && sudo ./svc.sh start`。仅注册不会上线；只有启动了服务的 runner 才会增加容量。每个约一分钟。 +Linux 开关启用期间，容量需覆盖 master 热备、主 CI 作业，以及每个符合条件的 PR 或 master 推送的三个发布演练作业。发布工作流不会因为新运行到来而取消正在执行的演练，因此不同引用的重叠运行会增加持续的构建、打包和安装负载。延长自托管运行前，检查当前 CPU、内存、磁盘和队列压力；同一虚拟机上新增注册只增加调度槽位，不增加机器资源。不能只依据热备负载推断空闲容量。主机资源允许增加注册实例时，使用组织级注册 token（组织 Settings → Actions → Runners → New runner）。复制现有 runner 目录时**必须排除身份文件**——`rsync -a --exclude '.runner*' --exclude '.credentials*' --exclude '_diag' --exclude '_work' / /`（通配同时排除 `.runner_migrated`/`.credentials_migrated`——GitHub 会在迁移过的运行器上写入这些文件，它们同样会触发 already-configured 拒绝）——再跑 `config.sh`（原样拷贝 `.runner`/`.credentials` 会使其以 "already configured" 拒绝），然后**启动监听器**：`sudo ./svc.sh install ubuntu && sudo ./svc.sh start`。仅注册不会上线；启动服务增加的是调度槽位，而非 CPU 或内存。 ### 切回 -删除 `DSH_CI_FAILOVER_LINUX` 或 `DSH_CI_FAILOVER_WINDOWS` 变量（或改为 `selfhosted` 以外的任何值），新的运行即解析回托管企业池。若故障期间追加注册过实例，将其移除。 +删除 `DSH_CI_FAILOVER_LINUX` 或 `DSH_CI_FAILOVER_WINDOWS` 变量（或改为 `selfhosted` 以外的任何值），新的运行即解析回各自的托管池。若故障期间追加注册过实例，将其移除。 ### 信任边界 @@ -55,7 +59,7 @@ Status: implemented **通过合并一次工作流改动来切换池。** 否决，因为触发切换的故障状态恰恰是任何 PR 都无法合并的状态：必需检查正是失败的那些。仓库变量是写者可管理的状态，重跑即生效，无需合并。 -**让自托管池长期处于必需路径中。** 否决，因为这是拿托管池的可用性去换自有虚拟机的可用性，只是搬移了单点故障而非增加回退。这些变量让托管池保持主路径，自托管池作为一个经过验证、一步即可启用的热备；按平台拆分意味着一个平台的故障不会重定向另一个平台。 +**让自托管池长期处于必需路径中。** 否决，因为这是拿托管池的可用性去换自有虚拟机的可用性，只是搬移了单点故障而非增加回退。未设置变量时默认保留托管目标，开关提供由运维人员选择、可逆的自托管路径；按平台拆分意味着一个平台的故障不会重定向另一个平台。 ## 后果 diff --git a/.agents/notes/implemented/process/2026-09-06-evidence-driven-performance-skill.i18n.yaml b/.agents/notes/implemented/process/2026-09-06-evidence-driven-performance-skill.i18n.yaml new file mode 100644 index 0000000000..53de712240 --- /dev/null +++ b/.agents/notes/implemented/process/2026-09-06-evidence-driven-performance-skill.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write .agents/notes/implemented/process/2026-09-06-evidence-driven-performance-skill.md +2026-09-06-evidence-driven-performance-skill.md: 5b15cce1adbd7ff47e5668f7332cba8d1b59e5fe +2026-09-06-evidence-driven-performance-skill.zh.md: c1fbbd76740badd87ed0a95218e2c4082f2c0e8b diff --git a/.agents/notes/implemented/process/2026-09-06-evidence-driven-performance-skill.md b/.agents/notes/implemented/process/2026-09-06-evidence-driven-performance-skill.md new file mode 100644 index 0000000000..5b15cce1ad --- /dev/null +++ b/.agents/notes/implemented/process/2026-09-06-evidence-driven-performance-skill.md @@ -0,0 +1,48 @@ +# Agent Note: Evidence-driven performance optimization workflow + +Status: implemented + +English | [中文](2026-09-06-evidence-driven-performance-skill.zh.md) + +## Problem + +Performance work can improve an isolated phase while moving cost into another phase, retaining more data, or skipping required behavior. Historical PR descriptions also retain abandoned implementations and estimates, so copying their apparent solution can restore a rejected design instead of addressing a current bottleneck. + +## Decision + +The [dsh-speed-up-perf skill](../../../skills/dsh-speed-up-perf/SKILL.md) guides broad surveys toward bounded, measured user paths. It combines focused attribution with independently timed backend and browser endpoints, synthetic workload distributions, comparable cold/warm and retained-memory conditions, and negative controls for tightened budgets. The historical evidence below distinguishes merged implementations, superseded proposals, author-reported measurements, and estimates. + +The workflow requires behavior evidence independently of timing: model-visible logs, durable generation and publication rules, stream ordering, cancellation, and disposal remain obligations. Authorized private corpus inspection yields only aggregate workload inspiration; committed inputs and published artifacts contain synthetic material. Optimization PRs carry their tighter budgets, while a preceding benchmark layer can protect the measured baseline and remain independently mergeable. + +The [Session-opening performance-gate decision](../testing/2026-09-04-session-open-performance-gate.md) retains ownership of lane mechanics and calibration. The [simplification skill](../../../skills/dsh-find-simplifications/SKILL.md) retains ownership of deletion-oriented surveys. Neither is superseded: this workflow adds performance-specific candidate selection, measurement comparability, and stopping criteria rather than replacing their decisions. + +## Historical evidence + +These are author-reported historical measurements, not benchmarks rerun for this workflow. Final merged diffs and owning source take precedence over original PR descriptions. The rejected intermediate proposal is retained only to explain why identity registries are not a general prescription. + +| Evidence | Measured path and result | Reusable lesson | +|---|---|---| +| [#3535](https://github.com/deepseek-harness/deepseek-harness/pull/3535), merged | The [final benchmark design](https://github.com/deepseek-harness/deepseek-harness/pull/3535#issuecomment-5552779119) reports a 4,394 ms first-open negative control against 550 ms, first-history 4,452 against 550, resume 4,333 against 450, and 128 MB heap failures. Client fold: 123.9 ms / 10.84× against 40 ms / 3.125×. | Built-JS user-path gates and positive/negative controls matter more than an earlier PR-body design. | +| [#3536](https://github.com/deepseek-harness/deepseek-harness/pull/3536), closed unmerged | Repeated snapshot/freeze work occupied about 70% of profiled CPU; synthetic open improved from 4,734–4,921 to 707–823 ms. | Streaming migration superseded this identity-registry proposal. Do not revive it without current ownership evidence. | +| [#3585](https://github.com/deepseek-harness/deepseek-harness/pull/3585), merged | Historical physical decode: 7.527 s / 7,219 MB peak RSS to 1.467 s / 908 MB; streaming migration with serial publication: 6.241 s, 2.107 GB peak, 477 MB retained. Settled 500,000-delta Client fold: 3.2 ms. | Keep representations compact across consumers; bound intermediate state. Attribution estimates overlap and cannot be added. | +| [#3586](https://github.com/deepseek-harness/deepseek-harness/pull/3586), merged | Current-v2 opening snapshot: 2,011.4→1,027.9 ms; restore: 598.5→16 ms; retained heap: 1,025.3→478.7 MB. | Separate read-only preparation from awaited write publication; share immutable ownership with revision-keyed preparation and caller-local cancellation. | +| [#3537](https://github.com/deepseek-harness/deepseek-harness/pull/3537), merged | Synthetic 200-turn projection: 28→5.4 ms; total: 76.9→50 ms; peak RSS: 137.2→94.9 MB. | Read stats, usage, text and image references per compact record. Expanded-stream caching retains unnecessary representation cost. Chat/Trajectory belong to the preceding migration change. | +| [#2587](https://github.com/deepseek-harness/deepseek-harness/pull/2587), merged | Historical 416,756 events represented by 696 records: client history 4,682→276 ms; sampled additional V8 peak 612.5→199.4 MB. | Preserve compactness through validation and folding; [baseline review](https://github.com/deepseek-harness/deepseek-harness/pull/2587#discussion_r3803082730) requires equal validation and retained output, not parse-and-discard. | +| [#3331](https://github.com/deepseek-harness/deepseek-harness/pull/3331), merged | 10,000 collapsed tool rows: 22.5→7.5 ms, retained 12.2→1.6 MiB; inactive Trajectory flushes: 4,082→15.5 ms. | Defer unused parsing and materialization; first activation and retained Context still cost work. | +| [#3391](https://github.com/deepseek-harness/deepseek-harness/pull/3391) and [#3383](https://github.com/deepseek-harness/deepseek-harness/pull/3383), merged | Narrow subscriptions, stable identities, batched publication, and viewport-triggered highlighting. The 10,000-node timing table is estimated, not browser measurement. | Deferral is not virtualization: visited token DOM remains retained. | +| [#3292](https://github.com/deepseek-harness/deepseek-harness/pull/3292), merged | Two-million-item FIFO drain: 9.656 ms median, excluding enqueue. | A deque removes shift copying, not queue admission or backpressure obligations. | +| [#1161](https://github.com/deepseek-harness/deepseek-harness/pull/1161), merged | Keyless 100,000-chunk browser stress at 128 chunks per 16 ms. | [Producer catch-up](https://github.com/deepseek-harness/deepseek-harness/pull/1161#discussion_r3699970161) and [final heartbeat stalls](https://github.com/deepseek-harness/deepseek-harness/pull/1161#discussion_r3699970162) can distort measurements; scheduled events are not trusted keyboard/pointer input. | + +The [cancellation review](https://github.com/deepseek-harness/deepseek-harness/pull/3586#discussion_r3940578092), [source-revision review](https://github.com/deepseek-harness/deepseek-harness/pull/3586#discussion_r3940569241), and [typed-reader review](https://github.com/deepseek-harness/deepseek-harness/pull/3537#discussion_r3942974015) illustrate why removing repeated work does not authorize deleting validation or publication obligations. A [standby-runner review](https://github.com/deepseek-harness/deepseek-harness/pull/3535#discussion_r3927945561) distinguishes a dedicated job from an isolated physical host. + +## Alternatives considered + +**Optimize suspicious code before measuring.** Rejected because local complexity does not identify dominant user cost and cannot establish improvement or regression protection. + +**Treat historical speedups as reusable prescriptions.** Rejected because representation, ownership, and lifecycle requirements change. Historical evidence generates hypotheses; current production paths and fresh measurements decide whether a change applies. + +**Use only microbenchmarks or only end-to-end timing.** Rejected because isolated phases can omit moved work, while aggregate timing alone cannot locate its cause. Both are required at the scope appropriate to the selected problem. + +## Consequences + +The skill adds no runtime behavior, benchmark implementation, or new CI policy. Its validation is document/link consistency and skill metadata; each future optimization supplies executable measurements and functional evidence at its owner. The finite scenario/fix scope prevents a broad performance request from becoming an unrelated architectural rewrite. diff --git a/.agents/notes/implemented/process/2026-09-06-evidence-driven-performance-skill.zh.md b/.agents/notes/implemented/process/2026-09-06-evidence-driven-performance-skill.zh.md new file mode 100644 index 0000000000..c1fbbd7674 --- /dev/null +++ b/.agents/notes/implemented/process/2026-09-06-evidence-driven-performance-skill.zh.md @@ -0,0 +1,48 @@ +# Agent Note: 以证据驱动的性能优化工作流 + +Status: implemented + +[English](2026-09-06-evidence-driven-performance-skill.md) | 中文 + +## 问题 + +性能工作可能改善某个独立阶段，却把成本转移到另一阶段、保留更多数据，或跳过必要行为。历史 PR（Pull Request）描述也可能保留已放弃的实现和估计值，因此照搬其表面方案可能恢复已否决的设计，而不是解决当前瓶颈。 + +## 决定 + +[dsh-speed-up-perf skill](../../../skills/dsh-speed-up-perf/SKILL.md)（技能）引导广泛调查收敛到范围明确、可测量的用户路径。它结合聚焦的成本归因与独立计时的后端和浏览器端点、合成负载分布、可比较的冷态／热态与保留内存条件，以及收紧预算的负向对照。下方历史证据区分已合并实现、已被替代的提案、作者报告的测量值和估计值。 + +该工作流要求独立于计时的行为证据：模型可见日志、持久化代际和发布规则、流顺序、取消及 dispose（资源释放）仍是必须满足的要求。获授权的私有语料检查仅提供聚合负载启发；提交的输入和发布的产物包含合成材料。优化 PR 携带收紧后的预算，而前置基准测试层可以保护已测基线并保持独立可合并。 + +[会话打开性能门禁决策](../testing/2026-09-04-session-open-performance-gate.zh.md)继续负责测试通道机制与校准。[简化 skill](../../../skills/dsh-find-simplifications/SKILL.md)继续负责以删除为目标的调查。两者均未被替代：本工作流增加面向性能的候选选择、测量可比性和停止条件，而不替换它们的决策。 + +## 历史证据 + +这些是作者报告的历史测量，并非为本工作流重新运行的基准测试。最终合并差异与所属源码优先于最初 PR 描述。保留已否决的中间提案，仅用于解释为何身份注册表不是通用处方。 + +| 证据 | 测量路径与结果 | 可复用经验 | +|---|---|---| +| [#3535](https://github.com/deepseek-harness/deepseek-harness/pull/3535)，已合并 | [最终基准设计](https://github.com/deepseek-harness/deepseek-harness/pull/3535#issuecomment-5552779119)报告首次打开负向对照 4,394 ms，预算 550 ms；首屏历史 4,452，预算 550；恢复 4,333，预算 450；128 MB 堆检查失败。Client fold：123.9 ms / 10.84×，预算 40 ms / 3.125×。 | built-JS 用户路径门禁与正／负向对照比早期 PR 正文设计更重要。 | +| [#3536](https://github.com/deepseek-harness/deepseek-harness/pull/3536)，关闭未合并 | 重复 snapshot／freeze 工作占采样 CPU 的约 70%；合成打开从 4,734–4,921 改善为 707–823 ms。 | 流式迁移替代了该身份注册表提案。没有当前所有权证据时，不恢复它。 | +| [#3585](https://github.com/deepseek-harness/deepseek-harness/pull/3585)，已合并 | 历史物理解码：7.527 s / 7,219 MB 峰值 RSS 降至 1.467 s / 908 MB；流式迁移加串行发布：6.241 s，2.107 GB 峰值，477 MB 保留。已结算的 500,000-delta Client fold：3.2 ms。 | 跨消费者保持紧凑表示；限制中间状态。归因估计重叠，不能相加。 | +| [#3586](https://github.com/deepseek-harness/deepseek-harness/pull/3586)，已合并 | 当前 v2 打开快照：2,011.4→1,027.9 ms；恢复：598.5→16 ms；保留堆：1,025.3→478.7 MB。 | 分离只读准备与必须等待的写发布；通过按修订号共享准备和调用方局部取消共享不可变所有权。 | +| [#3537](https://github.com/deepseek-harness/deepseek-harness/pull/3537)，已合并 | 合成 200 轮投影：28→5.4 ms；总计：76.9→50 ms；峰值 RSS：137.2→94.9 MB。 | 按紧凑记录读取统计、usage、文本和图像引用。展开流缓存保留不必要的表示成本。Chat／Trajectory 属于前置迁移改动。 | +| [#2587](https://github.com/deepseek-harness/deepseek-harness/pull/2587)，已合并 | 历史 416,756 事件由 696 记录表示：Client 历史 4,682→276 ms；采样额外 V8 峰值 612.5→199.4 MB。 | 验证和折叠过程保持紧凑；[基线审查](https://github.com/deepseek-harness/deepseek-harness/pull/2587#discussion_r3803082730)要求相同验证与保留输出，而不是解析后丢弃。 | +| [#3331](https://github.com/deepseek-harness/deepseek-harness/pull/3331)，已合并 | 10,000 个折叠工具行：22.5→7.5 ms，保留 12.2→1.6 MiB；非活动 Trajectory 刷新：4,082→15.5 ms。 | 延迟未使用的解析和实体化；首次激活与保留 Context 仍有成本。 | +| [#3391](https://github.com/deepseek-harness/deepseek-harness/pull/3391) 和 [#3383](https://github.com/deepseek-harness/deepseek-harness/pull/3383)，已合并 | 缩小订阅范围、稳定身份、批量发布和视口触发高亮。10,000 节点计时表是估计，不是浏览器测量。 | 延迟不等于虚拟化：访问过的 token DOM 仍被保留。 | +| [#3292](https://github.com/deepseek-harness/deepseek-harness/pull/3292)，已合并 | 两百万条 FIFO 排空：中位数 9.656 ms，不含入队。 | deque 删除 shift 复制，不删除队列准入或背压义务。 | +| [#1161](https://github.com/deepseek-harness/deepseek-harness/pull/1161)，已合并 | 无密钥的 100,000-chunk 浏览器压力测试，每 16 ms 推送 128 个 chunk。 | [生产者追赶](https://github.com/deepseek-harness/deepseek-harness/pull/1161#discussion_r3699970161)和[最后一次心跳停顿](https://github.com/deepseek-harness/deepseek-harness/pull/1161#discussion_r3699970162)可能扭曲测量；定时派发事件不是真实键盘／指针输入。 | + +[取消审查](https://github.com/deepseek-harness/deepseek-harness/pull/3586#discussion_r3940578092)、[源修订审查](https://github.com/deepseek-harness/deepseek-harness/pull/3586#discussion_r3940569241)和[类型化读取器审查](https://github.com/deepseek-harness/deepseek-harness/pull/3537#discussion_r3942974015)说明删除重复工作不等于允许删除验证或发布义务。[备用 runner 审查](https://github.com/deepseek-harness/deepseek-harness/pull/3535#discussion_r3927945561)区分独立 job 与隔离的物理主机。 + +## 考虑过的替代方案 + +**先优化可疑代码，再测量。** 否决，因为局部复杂度不能确定主要用户成本，也无法证明改善或防止回归。 + +**把历史加速方案当作可复用处方。** 否决，因为表示方式、所有权和生命周期要求会变化。历史证据用于产生假设；当前生产路径与新的测量决定改动是否适用。 + +**只使用微基准测试，或只使用端到端计时。** 否决，因为独立阶段可能遗漏被转移的工作，而总计时无法定位原因。两者都需要在与所选问题相符的范围内使用。 + +## 后果 + +该 skill 不增加运行时行为、基准测试实现或新的 CI 策略。其验证涵盖文档／链接一致性和 skill 元数据；后续每项优化在其所属位置提供可执行测量与功能证据。有限的场景／修复范围防止广泛性能请求演变成无关的架构重写。 diff --git a/.agents/notes/implemented/process/2026-09-06-preview-hosted-runner-sizing.i18n.yaml b/.agents/notes/implemented/process/2026-09-06-preview-hosted-runner-sizing.i18n.yaml new file mode 100644 index 0000000000..797bffd756 --- /dev/null +++ b/.agents/notes/implemented/process/2026-09-06-preview-hosted-runner-sizing.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write .agents/notes/implemented/process/2026-09-06-preview-hosted-runner-sizing.md +2026-09-06-preview-hosted-runner-sizing.md: 87298e94f11aa7e483afde31e0963a56f523febc +2026-09-06-preview-hosted-runner-sizing.zh.md: 285b21d60d755e76db582e4e55c5cde913f7fc2e diff --git a/.agents/notes/implemented/process/2026-09-06-preview-hosted-runner-sizing.md b/.agents/notes/implemented/process/2026-09-06-preview-hosted-runner-sizing.md new file mode 100644 index 0000000000..87298e94f1 --- /dev/null +++ b/.agents/notes/implemented/process/2026-09-06-preview-hosted-runner-sizing.md @@ -0,0 +1,46 @@ +# Agent Note: Measured GitHub-hosted PR preview sizing + +Status: implemented + +English | [中文](2026-09-06-preview-hosted-runner-sizing.zh.md) + +## Problem + +PR previews build the full workspace and browser-worker VFS image. A lower per-minute runner price does not guarantee lower job cost because GitHub rounds each job upward to whole minutes. Moving previews to persistent self-hosted machines also changes isolation and is outside this decision. + +## Decision + +The [preview workflow](../../../../.github/workflows/build-preview-cloudflare.yml) uses standard GitHub-hosted `ubuntu-24.04`. Build, cache, deployment, protected-image verification, and comment semantics remain unchanged. The [sizing reference](../../../../.github/preview-sizing/README.md) owns comparison requirements. The separate CI [failover runbook](2026-07-26-ci-failover-runbook.md) retains its independent runner-switch decision; previews do not use those switches. + +### Measurements + +[Experiment 34012729982](https://github.com/deepseek-harness/deepseek-harness/actions/runs/34012729982) succeeds for all eight size/cache combinations plus one cache seed. Every measured job checks out SHA `9149d7e7ef945b5601711badd3cf63d58ab384f5`, uses Node 24.19.0 and pnpm 11.7.0, and executes immutable install, full workspace build, preview/VFS packing, and local upload shaping with gzip integrity verification. Warm jobs restore one exact run-private pnpm cache; cold jobs skip restoration but contain pnpm bootstrap files. No compiled outputs are restored. + +| Runner | Cold / warm job seconds | Rounded minutes each | USD each | Workspace seconds cold / warm | Preview seconds cold / warm | +|---|---:|---:|---:|---:|---:| +| standard, 2 vCPU | 202 / 203 | 4 | 0.024 | 138.92 / 147.21 | 12.65 / 12.94 | +| larger, 4 vCPU | 177 / 162 | 3 | 0.036 | 124.21 / 114.86 | 10.77 / 9.88 | +| larger, 8 vCPU | 154 / 154 | 3 | 0.066 | 110.51 / 110.77 | 9.20 / 9.21 | +| larger, 16 vCPU | 124 / 125 | 3 | 0.126 | 90.57 / 84.99 | 7.62 / 7.33 | + +Using [published rates](https://docs.github.com/en/billing/reference/actions-runner-pricing), measured jobs total $0.504; the 60-second standard seed adds $0.006. The $0.510 gross compute estimate includes setup, restoration, measurement upload, and cleanup, but excludes storage and account discounts. Standard costs 80.95% less than 16-core and 33.33% less than 4-core in each sampled cache state. It adds 78 seconds against the corresponding 16-core job. + +Standard jobs expose two vCPUs and 7.75 GiB RAM. Workspace maximum process RSS is 2.86 / 2.76 GiB; preview maximum process RSS is 0.76 / 0.74 GiB. Both complete without an OOM or timeout. GNU time RSS is not simultaneous process-tree memory. These samples establish successful execution, not a permanent memory guarantee. + +The comparison fixes source, lockfile, commands, and runtime versions, not physical CPUs or image release: standard and 4-core use image 20260831.293.1; 8-core and 16-core use 20260823.283.1. CPUs vary among AMD EPYC 9V74/7763 and Intel Xeon 8370C/8573C. One sample per cache state measures the offered labels, not isolated CPU scaling or statistical repeatability. + +The experiment does not deploy or access Cloudflare credentials. Measurement upload takes zero to one second; warm-cache restore takes six to ten seconds. For context, [production job 101428009994](https://github.com/deepseek-harness/deepseek-harness/actions/runs/34011495156/job/101428009994) spends 14 seconds uploading, one second verifying, and two seconds commenting on a different SHA. Adding that overhead to this experiment is a projection, not a measured standard-runner publication result. The actual PR preview workflow owns deployment confirmation. + +## Alternatives considered + +**Keep 16-core.** It provides the shortest measured job, but costs $0.102 more per sample for a 78-second improvement. Preview builds do not justify that premium for this cost-focused decision. + +**Select 4-core or 8-core.** Both succeed and shorten builds, but their rounded sample costs exceed standard Ubuntu. Four-core retains more RAM and disk headroom if future workloads exhaust standard capacity; such a change requires new measurements. + +**Move to self-hosted.** Rejected by scope: previews remain on GitHub CI. The existing Linux and Windows registrations can share persistent hosts; their dependency, store-volume, and cleanup assumptions do not apply to fresh hosted VMs. No failover or trust condition changes. + +## Consequences + +Previews trade approximately 78 seconds of sampled build-job latency for lower compute cost. Production Cloudflare latency, image rollout variance, future build growth, and broader success rates remain observable limitations. No hourly or monthly savings are extrapolated from this single experiment. The temporary benchmark workflow and its safety test are absent from the final tree; the experiment commits and linked run preserve the method and evidence. + +The executed [focused regression](../../../../scripts/preview-workflow.spec.ts) pins hosted routing, PR triggers and permissions, immutable full builds, restore-only caching, publication shaping, protected-image checks, and idempotent comments. A physical self-hosted routing mutation fails its routing assertion; restoration passes all three tests. No model-visible runtime behavior changes, so no Session snapshot changes are required. diff --git a/.agents/notes/implemented/process/2026-09-06-preview-hosted-runner-sizing.zh.md b/.agents/notes/implemented/process/2026-09-06-preview-hosted-runner-sizing.zh.md new file mode 100644 index 0000000000..285b21d60d --- /dev/null +++ b/.agents/notes/implemented/process/2026-09-06-preview-hosted-runner-sizing.zh.md @@ -0,0 +1,46 @@ +# Agent Note: 基于测量的 GitHub 托管 PR 预览规格 + +Status: implemented + +[English](2026-09-06-preview-hosted-runner-sizing.md) | 中文 + +## 问题 + +PR（Pull Request）预览构建完整工作区及浏览器 worker VFS 镜像。较低的每分钟运行器价格不能保证较低的作业成本，因为 GitHub 将每个作业向上取整至整分钟。将预览移至持久化自托管机器还会改变隔离方式，不属于本决策范围。 + +## 决策 + +[预览工作流](../../../../.github/workflows/build-preview-cloudflare.yml) 使用标准 GitHub 托管 `ubuntu-24.04`。构建、缓存、部署、受保护镜像验证及评论语义保持不变。[规格参考](../../../../.github/preview-sizing/README.zh.md) 负责比较要求。独立的 CI [故障切换手册](2026-07-26-ci-failover-runbook.zh.md) 保留其运行器切换决策；预览不使用这些开关。 + +### 测量 + +[实验 34012729982](https://github.com/deepseek-harness/deepseek-harness/actions/runs/34012729982) 的八种规格/缓存组合及一个缓存预热作业均成功。每个测量作业检出 SHA `9149d7e7ef945b5601711badd3cf63d58ab384f5`，使用 Node 24.19.0 与 pnpm 11.7.0，并执行不可变安装、完整工作区构建、预览/VFS 打包，以及含 gzip 完整性验证的本地上传内容整理。热作业恢复同一个运行私有精确 pnpm 缓存；冷作业跳过恢复，但包含 pnpm 引导安装文件。不恢复编译产物。 + +| 运行器 | 冷 / 热作业秒数 | 各自取整分钟数 | 各自美元费用 | 冷 / 热工作区秒数 | 冷 / 热预览秒数 | +|---|---:|---:|---:|---:|---:| +| 标准，2 vCPU | 202 / 203 | 4 | 0.024 | 138.92 / 147.21 | 12.65 / 12.94 | +| 大型，4 vCPU | 177 / 162 | 3 | 0.036 | 124.21 / 114.86 | 10.77 / 9.88 | +| 大型，8 vCPU | 154 / 154 | 3 | 0.066 | 110.51 / 110.77 | 9.20 / 9.21 | +| 大型，16 vCPU | 124 / 125 | 3 | 0.126 | 90.57 / 84.99 | 7.62 / 7.33 | + +按[公开费率](https://docs.github.com/en/billing/reference/actions-runner-pricing)，测量作业合计 $0.504；60 秒标准预热作业增加 $0.006。$0.510 总计算费用估算包含设置、恢复、测量上传及清理，但不含存储和账户折扣。在每种采样缓存状态下，标准运行器比 16 核低 80.95%，比 4 核低 33.33%。相比对应的 16 核作业增加 78 秒。 + +标准作业提供两个 vCPU 与 7.75 GiB 内存。工作区最大进程 RSS 为 2.86 / 2.76 GiB；预览最大进程 RSS 为 0.76 / 0.74 GiB。两者均未发生 OOM 或超时并完成。GNU time RSS 不是进程树同时占用的内存总量。这些样本证明成功执行，而非永久内存保证。 + +比较固定源代码、锁文件、命令和运行时版本，但不固定物理 CPU 或镜像版本：标准与 4 核使用镜像 20260831.293.1；8 核与 16 核使用 20260823.283.1。CPU 包括 AMD EPYC 9V74/7763 与 Intel Xeon 8370C/8573C。每种缓存状态的单个样本测量所提供的标签，而非独立 CPU 扩展性或统计可重复性。 + +实验不部署，也不访问 Cloudflare 凭据。测量上传耗时零至一秒；热缓存恢复耗时六至十秒。作为背景，[生产作业 101428009994](https://github.com/deepseek-harness/deepseek-harness/actions/runs/34011495156/job/101428009994) 在不同 SHA 上上传耗时 14 秒、验证一秒、评论两秒。将该开销加至本实验属于推算，而非已测量的标准运行器发布结果。实际 PR 预览工作流负责部署确认。 + +## 考虑过的替代方案 + +**保留 16 核。** 它提供最短的测量作业，但为 78 秒改善使每个样本增加 $0.102。对于本次以成本为重点的决策，预览构建不值得这项溢价。 + +**选择 4 核或 8 核。** 两者均成功并缩短构建，但取整后的样本费用高于标准 Ubuntu。若未来工作负载耗尽标准容量，4 核可保留更多内存与磁盘余量；这样的变更需要新测量。 + +**移至自托管。** 因范围限制而拒绝：预览保留在 GitHub CI。现有 Linux 与 Windows 注册实例可能共享持久化主机；其依赖、store 卷及清理假设不适用于全新的托管 VM。不改变故障切换或信任条件。 + +## 影响 + +预览以约 78 秒采样构建作业延迟换取更低的计算费用。生产 Cloudflare 延迟、镜像发布差异、未来构建增长及更广泛的成功率仍是可观测限制。不从本次单一实验外推每小时或每月节省。最终文件树不包含临时基准工作流及其安全测试；实验提交与链接的运行保留方法和证据。 + +已执行的[针对性回归](../../../../scripts/preview-workflow.spec.ts) 固定托管路由、PR 触发器与权限、不可变完整构建、只恢复缓存、发布内容整理、受保护镜像检查及幂等评论。实际修改为自托管路由会使路由断言失败；恢复后全部三个测试通过。不改变模型可见运行时行为，因此不需要修改 Session 快照。 diff --git a/.agents/notes/implemented/process/2026-09-06-python-runtime-windows-hosted.i18n.yaml b/.agents/notes/implemented/process/2026-09-06-python-runtime-windows-hosted.i18n.yaml new file mode 100644 index 0000000000..c7bff1c862 --- /dev/null +++ b/.agents/notes/implemented/process/2026-09-06-python-runtime-windows-hosted.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write .agents/notes/implemented/process/2026-09-06-python-runtime-windows-hosted.md +2026-09-06-python-runtime-windows-hosted.md: ca2f02e8bac8a90be2b10bd6d7ae0b68215152ae +2026-09-06-python-runtime-windows-hosted.zh.md: e1d2ca1a65de19a6604f0848de23fe5cc100e87f diff --git a/.agents/notes/implemented/process/2026-09-06-python-runtime-windows-hosted.md b/.agents/notes/implemented/process/2026-09-06-python-runtime-windows-hosted.md new file mode 100644 index 0000000000..ca2f02e8ba --- /dev/null +++ b/.agents/notes/implemented/process/2026-09-06-python-runtime-windows-hosted.md @@ -0,0 +1,25 @@ +# Agent Note: Windows Python runtime CI stays on GitHub-hosted Windows + +Status: implemented + +English | [中文](2026-09-06-python-runtime-windows-hosted.zh.md) + +## Problem + +The Windows x64 target in [build-exe-for-python-sdk.yml](../../../../.github/workflows/build-exe-for-python-sdk.yml) started resolving through `DSH_CI_FAILOVER_WINDOWS=selfhosted` for trusted pull-request CI when #3629 added the failover selector and the job-private Windows toolchain. The shared `dsh-win-ci` pool did not make the lane more reliable. On 2026-09-06 the installed-wheel smoke passed at 09:12 on `dsh-win-ci-16` for [an earlier commit of the same pull request](https://github.com/deepseek-harness/deepseek-harness/actions/runs/34023970384), then failed at 10:06 on `dsh-win-ci-21` for [another pull request](https://github.com/deepseek-harness/deepseek-harness/actions/runs/34026500701) and at 10:46 on `dsh-win-ci-04` for [the same pull request](https://github.com/deepseek-harness/deepseek-harness/actions/runs/34028339888/job/101473395734), where `smoke_sdk_profile_plugin`'s packaged `dsh plugin add` child exited without output while the Linux and macOS cells of that run passed. The migration proposal ([#3629](https://github.com/deepseek-harness/deepseek-harness/pull/3629)) remained `proposed` because its throughput and shared-load acceptance criteria were never measured. + +## Decision + +The Windows x64 target always uses its hosted `matrix.runner` — `windows-2025` for pull-request CI — with the standard setup-python toolchain, the pnpm cache restore, and the pkg cache. The failover selector, the job-private Python setup step, the self-hosted dependency install and post-step cleanup, the private setup script, and the routing spec from #3629 are removed. `DSH_CI_FAILOVER_WINDOWS=selfhosted` again retargets only the native Windows jobs in [ci.yml](../../../../.github/workflows/ci.yml); the [failover runbook](2026-07-26-ci-failover-runbook.md) and [python/development.md](../../../../python/development.md) describe hosted-only runtime builds. The migration's UTF-8 mode exports existed because the persistent host used a GBK default code page; hosted images provide the locale the lane previously ran under. + +## Alternatives considered + +**Keep the failover routing.** Rejected: the shared pool reproduced the same silent installed-wheel child death twice in one day while the migrated inventory's throughput acceptance stayed open, and routing a correctness lane through failover state couples it to an unrelated pool-outage switch. + +**Fix the shared pool instead.** Left to pool operators: the observed failures are subprocesses dying without output, not a missing image prerequisite, and the same image serves the native Windows failover jobs. + +**Retain the job-private toolchain on hosted images.** Rejected: the private uv/Python download exists to avoid mutating a persistent shared host; disposable hosted images already provide the registered Python 3.10 toolchain the pre-migration lane used. + +## Consequences + +Every qualifying pull request again pays GitHub-hosted Windows capacity for the runtime build, and the job-private setup and cleanup machinery — including the bounded filesystem retries — is gone with the lane. In exchange each build runs on a disposable host with the proven toolchain and hosted caches, and the Windows failover switch covers only the native Windows jobs as documented before the migration. A future self-hosted attempt must re-validate throughput and failure reproducibility on the actual pool before any routing change. diff --git a/.agents/notes/implemented/process/2026-09-06-python-runtime-windows-hosted.zh.md b/.agents/notes/implemented/process/2026-09-06-python-runtime-windows-hosted.zh.md new file mode 100644 index 0000000000..e1d2ca1a65 --- /dev/null +++ b/.agents/notes/implemented/process/2026-09-06-python-runtime-windows-hosted.zh.md @@ -0,0 +1,25 @@ +# Agent Note: Windows Python runtime CI 保留在 GitHub 托管 Windows 上 + +Status: implemented + +[English](2026-09-06-python-runtime-windows-hosted.md) | 中文 + +## 问题 + +当 #3629 加入故障切换选择器与作业私有的 Windows 工具链后，[build-exe-for-python-sdk.yml](../../../../.github/workflows/build-exe-for-python-sdk.yml) 中的 Windows x64 目标开始对受信任的 PR CI 通过 `DSH_CI_FAILOVER_WINDOWS=selfhosted` 解析运行器。共享的 `dsh-win-ci` 池并未让该通道更可靠。2026-09-06，安装后 wheel 冒烟测试在 09:12 于 `dsh-win-ci-16` 上为[同一拉取请求的较早提交](https://github.com/deepseek-harness/deepseek-harness/actions/runs/34023970384)通过，随后 10:06 在 `dsh-win-ci-21` 上为[另一个拉取请求](https://github.com/deepseek-harness/deepseek-harness/actions/runs/34026500701)失败，10:46 在 `dsh-win-ci-04` 上为[同一拉取请求](https://github.com/deepseek-harness/deepseek-harness/actions/runs/34028339888/job/101473395734)失败——`smoke_sdk_profile_plugin` 打包的 `dsh plugin add` 子进程无输出即退出，而该次运行的 Linux 与 macOS 单元均通过。迁移提案（[#3629](https://github.com/deepseek-harness/deepseek-harness/pull/3629)）保持 `proposed`，因为其吞吐量与共享负载验收标准从未实测。 + +## 决策 + +Windows x64 目标始终使用托管的 `matrix.runner`——PR CI 为 `windows-2025`——配以标准 setup-python 工具链、pnpm 缓存恢复与 pkg 缓存。来自 #3629 的故障切换选择器、作业私有 Python 准备步骤、自托管依赖安装与后置清理、私有准备脚本及路由测试均被移除。`DSH_CI_FAILOVER_WINDOWS=selfhosted` 再次只重定向 [ci.yml](../../../../.github/workflows/ci.yml) 中的原生 Windows 作业；[故障切换手册](2026-07-26-ci-failover-runbook.zh.md)与 [python/development.zh.md](../../../../python/development.zh.md) 描述仅托管的 runtime 构建。迁移中的 UTF-8 模式导出之所以存在，是因为持久主机使用 GBK 默认代码页；托管镜像提供该通道此前运行的区域设置。 + +## 已考虑的替代方案 + +**保留故障切换路由。** 不采用：共享池同一天两次复现相同的安装后 wheel 子进程无声死亡，而迁移清单的吞吐量验收仍然悬置；并且把正确性通道路由进故障切换状态，会使其耦合到无关的池故障开关。 + +**改为修复共享池。** 交由池运维者处理：观测到的失败是无输出即退出的子进程，而非镜像前置条件缺失；同一镜像还服务原生 Windows 故障切换作业。 + +**在托管镜像上保留作业私有工具链。** 不采用：私有 uv/Python 下载的存在理由是不修改持久共享主机；一次性托管镜像已提供迁移前通道使用的已注册 Python 3.10 工具链。 + +## 后果 + +每个符合条件的拉取请求再次为 runtime 构建支付 GitHub 托管 Windows 容量，作业私有准备与清理机制（包括有界文件系统重试）随通道一同移除。交换来的是每次构建运行在带标准工具链与托管缓存的一次性主机上，且 Windows 故障切换开关只覆盖迁移前文档所述的原生 Windows 作业。未来的自托管尝试必须在任何路由变更前，对实际池重新验证吞吐量与失败可复现性。 diff --git a/.agents/notes/implemented/process/2026-09-06-release-rehearsal-selfhosted.i18n.yaml b/.agents/notes/implemented/process/2026-09-06-release-rehearsal-selfhosted.i18n.yaml new file mode 100644 index 0000000000..a8ce01be68 --- /dev/null +++ b/.agents/notes/implemented/process/2026-09-06-release-rehearsal-selfhosted.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write .agents/notes/implemented/process/2026-09-06-release-rehearsal-selfhosted.md +2026-09-06-release-rehearsal-selfhosted.md: 415ae4716e9bc0ae9b165afc807f6f41e8a57e04 +2026-09-06-release-rehearsal-selfhosted.zh.md: a6fa441d01e66cea998d77a9b1be588053ac60a5 diff --git a/.agents/notes/implemented/process/2026-09-06-release-rehearsal-selfhosted.md b/.agents/notes/implemented/process/2026-09-06-release-rehearsal-selfhosted.md new file mode 100644 index 0000000000..415ae4716e --- /dev/null +++ b/.agents/notes/implemented/process/2026-09-06-release-rehearsal-selfhosted.md @@ -0,0 +1,27 @@ +# Agent Note: trusted release rehearsals on persistent Linux runners + +Status: implemented + +English | [中文](2026-09-06-release-rehearsal-selfhosted.zh.md) + +## Problem + +Dependency-layout and release-pack rehearsals consume hosted Linux minutes without requiring npm or API credentials. Moving arbitrary pull-request code or credentialed publication onto a persistent shared host would weaken isolation; reusing a checkout without cleaning would also weaken the packed-payload proof. + +## Decision + +The two jobs in [release.yml](../../../../.github/workflows/release.yml) and the pack job in [release-vendor.yml](../../../../.github/workflows/release-vendor.yml) select the existing self-hosted Linux pool only with the writer-controlled `DSH_CI_FAILOVER_LINUX` repository variable set to `selfhosted`. The selector requires the canonical repository and a non-Dependabot actor, then admits only master pushes or same-repository, non-fork PRs whose author is not Dependabot. Manual dispatch always selects `ubuntu-24.04`, as do all other rejected contexts. The [failover runbook](2026-07-26-ci-failover-runbook.md) owns the platform switches and standby operation. Release rehearsals intentionally share the Linux switch with main CI: enabling or disabling it routes both workloads, not releases independently. Unset remains the hosted default; hosted-minute savings occur only while an operator selects `selfhosted`, whether for an outage or a longer-running cost choice. + +The runner labels are `[self-hosted, linux, x64, vm-backup]`. Runner registrations share one VM, not independent machine capacity. Each job uses its runner-private temporary volume for Node compile cache and node-gyp headers before pnpm setup, and a pnpm setup destination qualified by run, attempt, and job. `TMPDIR` also points to `runner.temp`, so temporary npm consumers stay outside the checkout but inside runner cleanup even when a killed process cannot execute `finally`. The persistent pnpm store stays outside checkout cleanup; only GitHub-hosted runners restore the remote store cache. Neither rehearsal workflow saves remote caches. + +Checkout explicitly cleans ignored and untracked output before immutable installation and the existing builds. Full tag history, pack concurrency, dependency checks, tarball verification, and artifact retention remain unchanged. The packed-install verifier creates a fresh consumer outside the checkout, installs tarballs with npm, removes inherited Node resolution hooks, and deletes the consumer in `finally`; a warm pnpm store cannot substitute workspace links or stale build output for a tarball payload. The [npm release decision](2026-08-10-npm-release-sequences.md) still owns release families and publication. Both manual publish workflows remain entirely hosted and gain no credentials or registry changes here. + +## Alternatives considered + +Always-hosted rehearsals avoid persistent-host risk but retain all hosted minutes. Always-self-hosted rehearsals remove the portable fallback. A scheduling job or reusable workflow adds another logical job and hides the three short setup sequences. Allowing manual dispatch on arbitrary refs gives a maintainer action broader persistent-host access than the explicit event trust rule. + +## Consequences + +Unsetting the variable or changing it away from `selfhosted` routes subsequent eligible jobs to hosted Ubuntu. This is an operator-selected fallback, not automatic runner-health detection or failover for already queued jobs. The shared VM can still contend with other trusted jobs, and repository writers remain responsible for code admitted to its persistent trust domain. No workflow provisions host packages or changes global host configuration. + +[scripts/tests/ci-release-selfhosted.spec.ts](../../../../scripts/tests/ci-release-selfhosted.spec.ts) evaluates the committed selectors with trusted events and negative controls for forks, Dependabot, other repositories, non-master pushes, dispatches, missing PR data, and disabled switches. It pins setup ordering, checkout cleanup, hosted-only remote cache access, publication isolation, and the retained commands. Real release-build and packed-install execution remains the PR CI verification owner; selector tests do not claim to reproduce those builds. diff --git a/.agents/notes/implemented/process/2026-09-06-release-rehearsal-selfhosted.zh.md b/.agents/notes/implemented/process/2026-09-06-release-rehearsal-selfhosted.zh.md new file mode 100644 index 0000000000..a6fa441d01 --- /dev/null +++ b/.agents/notes/implemented/process/2026-09-06-release-rehearsal-selfhosted.zh.md @@ -0,0 +1,27 @@ +# Agent Note: 在持久化 Linux 运行器上执行受信任的发布演练 + +Status: implemented + +[English](2026-09-06-release-rehearsal-selfhosted.md) | 中文 + +## Problem + +依赖布局检查和发布打包演练消耗托管 Linux 分钟，但不需要 npm 或 API 凭据。将任意拉取请求代码或携带凭据的发布任务放到持久化共享主机会削弱隔离；复用未经清理的检出目录也会削弱打包载荷验证。 + +## Decision + +[release.yml](../../../../.github/workflows/release.yml) 的两个作业和 [release-vendor.yml](../../../../.github/workflows/release-vendor.yml) 的打包作业仅在写权限维护者控制的仓库变量 `DSH_CI_FAILOVER_LINUX` 设为 `selfhosted` 时选择现有 Linux 自托管池。选择器要求当前仓库为正式仓库且触发者不是 Dependabot，然后只接纳 master 推送，或作者不是 Dependabot 的同仓库、非 fork PR（Pull Request）。手动触发始终选择 `ubuntu-24.04`，其他不满足条件的上下文也一样。[故障切换手册](2026-07-26-ci-failover-runbook.zh.md) 负责按平台划分的开关与热备操作。发布演练有意与主 CI 共用 Linux 开关：启用或禁用会同时路由两类负载，不能独立切换发布演练。未设置时仍默认使用托管池；只有运维人员选择 `selfhosted` 期间才节省托管分钟，无论该选择用于故障恢复还是持续的成本控制。 + +运行器标签为 `[self-hosted, linux, x64, vm-backup]`。运行器注册共享一台虚拟机，不代表独立机器容量。每个作业在 pnpm 设置前将 Node 编译缓存与 node-gyp 头文件放在运行器私有临时卷上，pnpm 设置目标路径包含运行、重试次数和作业标识。`TMPDIR` 也指向 `runner.temp`，因此临时 npm 消费目录既在检出目录之外，也在运行器清理范围之内，即使进程被强杀而无法执行 `finally` 也一样。持久化 pnpm 存储位于检出清理范围之外；只有 GitHub 托管运行器恢复远端存储缓存。两个演练工作流都不保存远端缓存。 + +检出操作显式清理被忽略和未跟踪的输出，再执行锁定依赖安装与现有构建。完整标签历史、打包并发、依赖检查、压缩包验证和产物保留期均保持不变。打包安装验证器在检出目录外创建全新的消费目录，用 npm 安装压缩包，移除继承的 Node 解析钩子，并在 `finally` 中删除消费目录；预热 pnpm 存储无法用工作区链接或过期构建输出代替压缩包载荷。[npm 发布决策](2026-08-10-npm-release-sequences.zh.md) 仍负责发布族与发布操作。两个手动发布工作流全部保留在托管运行器上，本改动不增加凭据，也不改变注册表。 + +## Alternatives considered + +始终使用托管演练可以避免持久化主机风险，但会保留全部托管分钟。始终自托管则失去可移植回退。增加调度作业或可复用工作流会多出一个逻辑作业，并隐藏三个简短的设置序列。允许任意引用的手动触发，会让维护者操作获得比明确事件信任规则更广的持久化主机访问权限。 + +## Consequences + +取消变量或将其改为非 `selfhosted` 值，会将后续符合条件的作业路由到托管 Ubuntu。这是运维人员选择的回退，不会自动探测运行器健康，也不会切换已排队的作业。共享虚拟机仍可能与其他受信任作业竞争资源；仓库写权限维护者仍对进入持久化信任域的代码负责。工作流不安装主机系统包，也不修改全局主机配置。 + +[scripts/tests/ci-release-selfhosted.spec.ts](../../../../scripts/tests/ci-release-selfhosted.spec.ts) 使用受信任事件和 fork、Dependabot、其他仓库、非 master 推送、手动触发、缺失 PR 数据、禁用开关等负向对照求值已提交的选择器。测试固定设置顺序、检出清理、仅托管运行器访问远端缓存、发布隔离和保留命令。真实发布构建与打包安装执行仍由 PR CI 验证；选择器测试不声称重现这些构建。 diff --git a/.agents/notes/implemented/testing/2026-09-04-session-open-performance-gate.i18n.yaml b/.agents/notes/implemented/testing/2026-09-04-session-open-performance-gate.i18n.yaml index 395f28c0f0..22cf5283e3 100644 --- a/.agents/notes/implemented/testing/2026-09-04-session-open-performance-gate.i18n.yaml +++ b/.agents/notes/implemented/testing/2026-09-04-session-open-performance-gate.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/testing/2026-09-04-session-open-performance-gate.md -2026-09-04-session-open-performance-gate.md: 30e65eb52dea48426cf87cf91053a9133ab42763 -2026-09-04-session-open-performance-gate.zh.md: 3e202c966ea4764e135c79e1d238af707dca1f8d +2026-09-04-session-open-performance-gate.md: 2820c9d7d0e5b7d9382c7f8d6540154440175f26 +2026-09-04-session-open-performance-gate.zh.md: 965b9035074504870bcb2f1ca8166962c264d75a diff --git a/.agents/notes/implemented/testing/2026-09-04-session-open-performance-gate.md b/.agents/notes/implemented/testing/2026-09-04-session-open-performance-gate.md index 30e65eb52d..2820c9d7d0 100644 --- a/.agents/notes/implemented/testing/2026-09-04-session-open-performance-gate.md +++ b/.agents/notes/implemented/testing/2026-09-04-session-open-performance-gate.md @@ -12,7 +12,7 @@ Measuring only `SessionPersistence.open()` does not stably describe the result f ## Decision -Linux pull requests run a required `node 24 / benchmarks` job that executes `pnpm run check:ci:bench` → `pnpm run test:bench`. The private `@deepseek-ai/dsh-benchmarks` workspace owns benchmark-only dependencies. The command first builds workspace libraries and dedicated workers under `benchmarks/.dsh-build/`, then invokes `vitest.bench.config.ts`. The job runs the benchmark lane alone; Vitest runs one file at a time and only prepares input, starts measurement children, aggregates results, and enforces budgets. Every timed CPU path executes compiled JavaScript under plain Node with `NODE_OPTIONS` removed and no TypeScript loader; bare workspace imports therefore resolve from `benchmarks/node_modules` through package exports to built `lib/` entries. +Linux pull requests run a required `node 24 / benchmarks` job that executes `pnpm run check:ci:bench` → `pnpm run test:bench`. The private `@deepseek-ai/dsh-benchmarks` workspace owns benchmark-only dependencies. The command first builds workspace libraries and dedicated workers under `benchmarks/.dsh-build/`, then invokes `vitest.bench.config.ts`. The [standard hosted runner decision](2026-09-06-standard-hosted-benchmark-runner.md) owns runner selection and the outer job timeout. The job runs the benchmark lane alone; Vitest runs one file at a time and only prepares input, starts measurement children, aggregates results, and enforces budgets. Every timed CPU path executes compiled JavaScript under plain Node with `NODE_OPTIONS` removed and no TypeScript loader; bare workspace imports therefore resolve from `benchmarks/node_modules` through package exports to built `lib/` entries. Required performance gates live under top-level `benchmarks/`, grouped by measured user path rather than package ownership. Host files use `*.bench.ts`, Client-face files use `*.bench.client.ts`, and scenario-specific workers and fixtures stay beside their benchmark without a benchmark suffix. Package-local `.perf.ts` files remain non-gating diagnostics; `scripts/` owns orchestration rather than benchmark cases. @@ -37,7 +37,7 @@ Normal-heap mode performs a fixed pair of explicit garbage collections after Hos The performance gate does not duplicate semantic assertions owned by functional tests; it requires only that the target call completes and reaches its measured endpoint. The Client-fold benchmark continues to use the real `ConversationNodeAssembler` and every Chat Definition, and requires both the large window's absolute time and its scaling relative to the small window to remain below fixed budgets. -Budgets are calibrated per measured endpoint. Two repeated Node 24.19 x64 CI runs differ by at most 5.2% in their medians; their CPU-heavy wall times are 1.95–2.06× the Node 24.18 arm64 reference run. Source constants record expected reference-machine durations; `ciTimeBudget()` multiplies them by the measured 2× CI time scale and 1.25× variance headroom. The retained-heap and Client-fold scaling budgets use only the 1.25× headroom because neither is a wall-clock duration. The 128 MB completion check remains an independent transient-allocation limit. The resulting first-open time limits, constrained-heap checks, and Client-fold limits all reject the known regressions. Pre-stack commit `0d7ea53743e273930a31e9e2b6ca682f21dd4ca5` is the fixed calibration and review reference; CI does not check out or execute the historical repository. Budgets are reviewed source constants and have no environment-variable override. +Budgets are calibrated per measured endpoint. Two repeated Node 24.19 x64 CI runs differ by at most 5.2% in their medians; their CPU-heavy wall times are 1.95–2.06× the Node 24.18 arm64 reference run. Except for current-generation `open`, source constants record expected reference-machine durations; `ciTimeBudget()` multiplies them by the measured 2× CI time scale and 1.25× variance headroom. Current-generation `open` uses a directly measured standard-runner expectation of 50 ms with only the 1.25× headroom, rounded up to a 63 ms budget. The retained-heap and Client-fold scaling budgets use only the 1.25× headroom because neither is a wall-clock duration. The 128 MB completion check remains an independent transient-allocation limit. The resulting first-open time limits, constrained-heap checks, and Client-fold limits all reject the known regressions. Pre-stack commit `0d7ea53743e273930a31e9e2b6ca682f21dd4ca5` is the fixed calibration and review reference; CI does not check out or execute the historical repository. Budgets are reviewed source constants and have no environment-variable override. ## Calibration evidence @@ -54,12 +54,14 @@ Five-sample medians on the same Node 24 reference machine establish the positive The pre-stack implementation keeps V0 as its current format, so first open does not change its on-disk representation; its native V0 first-history and Agent-resume measurements therefore apply to both lifecycle rows. +The [standard two-CPU run](https://github.com/deepseek-harness/deepseek-harness/actions/runs/34023970384/job/101461539961) at `ca3ffe95dac2c55eefeb16ed9b61067bbd19ee90` uses Node 24.20.0 x64 and Ubuntu image `20260831.293.1`. Its five current-generation `open` samples are 49.2, 47.4, 49.1, 48.6, and 48.1 ms: median 48.6 ms, maximum 49.2 ms. The rounded 50 ms CI expectation gives a 63 ms limit without reapplying the 2× machine scale. The log identifies two available CPUs but not their model; it does not isolate hardware from the Node-version change. This is endpoint-specific runner calibration, not evidence of an application optimization or a new reference-machine measurement. Every other benchmark passes its existing budget. Deterministic controls reject the observed median at the historical 30 ms limit, accept it at 63 ms, reject a synthetic 75 ms reopen median, and reject a synthetic 4,000 ms first-open duration at its unchanged 550 ms limit. These controls verify budget enforcement, not a measured new regression. + The calibrated source budgets are: | Measurement | Reference expectation | CI budget | |---|---:|---:| | First-open `open` | 220 ms | 550 ms | -| Current-generation `open` | 12 ms | 30 ms | +| Current-generation `open` | 12 ms (historical reference; CI expectation: 50 ms) | 63 ms | | Complete read | 8 ms | 20 ms | | Session restore | 24 ms | 60 ms | | Projection | 14 ms | 35 ms | diff --git a/.agents/notes/implemented/testing/2026-09-04-session-open-performance-gate.zh.md b/.agents/notes/implemented/testing/2026-09-04-session-open-performance-gate.zh.md index 3e202c966e..965b903507 100644 --- a/.agents/notes/implemented/testing/2026-09-04-session-open-performance-gate.zh.md +++ b/.agents/notes/implemented/testing/2026-09-04-session-open-performance-gate.zh.md @@ -12,7 +12,7 @@ Session format v2 的推出改变了两条成本随模型输出增长的路径 ## 决定 -Linux pull request 运行必需的 `node 24 / benchmarks` job，执行 `pnpm run check:ci:bench` → `pnpm run test:bench`。私有 `@deepseek-ai/dsh-benchmarks` workspace 拥有 benchmark 专属依赖。该命令先构建 workspace library 和 `benchmarks/.dsh-build/` 下的专用 worker，再调用 `vitest.bench.config.ts`。该 job 单独运行 benchmark lane；Vitest 逐文件运行，只负责准备输入、启动测量子进程、汇总结果和执行预算断言。每条被计时的 CPU 路径都以纯 Node 执行编译后的 JavaScript，并移除 `NODE_OPTIONS` 且不加载 TypeScript runtime；workspace 裸导入因此从 `benchmarks/node_modules` 通过 package exports 解析到构建后的 `lib/` 入口。 +Linux pull request 运行必需的 `node 24 / benchmarks` job，执行 `pnpm run check:ci:bench` → `pnpm run test:bench`。私有 `@deepseek-ai/dsh-benchmarks` workspace 拥有 benchmark 专属依赖。该命令先构建 workspace library 和 `benchmarks/.dsh-build/` 下的专用 worker，再调用 `vitest.bench.config.ts`。[标准托管运行器决策](2026-09-06-standard-hosted-benchmark-runner.zh.md)拥有运行器选择及外层 job 超时。该 job 单独运行 benchmark lane；Vitest 逐文件运行，只负责准备输入、启动测量子进程、汇总结果和执行预算断言。每条被计时的 CPU 路径都以纯 Node 执行编译后的 JavaScript，并移除 `NODE_OPTIONS` 且不加载 TypeScript runtime；workspace 裸导入因此从 `benchmarks/node_modules` 通过 package exports 解析到构建后的 `lib/` 入口。 必需性能 gate 位于顶层 `benchmarks/`，按被测用户路径而非 package 归属组织。Host 文件使用 `*.bench.ts`，Client 面文件使用 `*.bench.client.ts`，场景专属 worker 与 fixture 留在对应 benchmark 旁且不带 benchmark 后缀。包内 `.perf.ts` 文件仍是非门禁诊断；`scripts/` 负责编排而不承载 benchmark case。 @@ -37,7 +37,7 @@ Session benchmark 使用固定参数合成 released-v0 输入：200 轮，每轮 性能 gate 不重复功能测试的内容断言，只要求目标调用完成并到达对应的可观察终点。Client fold benchmark 继续使用真实 `ConversationNodeAssembler` 与全部 Chat Definition，要求大窗口的绝对时间和相对小窗口的缩放比均低于固定预算。 -预算按各测量终点分别校准。两次 Node 24.19 x64 CI 运行的中位数最大相差 5.2%；其 CPU 密集型壁钟时间是 Node 24.18 arm64 参考运行的 1.95–2.06 倍。源码常量记录参考机器上的预期耗时；`ciTimeBudget()` 将其乘以实测的 2 倍 CI 时间系数和 1.25 倍波动余量。GC 后增量堆与 Client fold 缩放预算不属于壁钟时间，因此只使用 1.25 倍余量。128 MB 完成性检查仍是独立的瞬时分配限制。由此得到的 first-open 时间上限、受限堆检查与 Client fold 上限都会拒绝已知退化。栈前参考提交固定为 `0d7ea53743e273930a31e9e2b6ca682f21dd4ca5`，只用于校准和评审预算；CI 不 checkout 或执行历史仓库。预算是源码中的受评审常量，不由环境变量覆盖。 +预算按各测量终点分别校准。两次 Node 24.19 x64 CI 运行的中位数最大相差 5.2%；其 CPU 密集型壁钟时间是 Node 24.18 arm64 参考运行的 1.95–2.06 倍。除当前 generation `open` 外，源码常量记录参考机器上的预期耗时；`ciTimeBudget()` 将其乘以实测的 2 倍 CI 时间系数和 1.25 倍波动余量。当前 generation `open` 使用标准运行器直接测得的 50 ms 预期值，仅乘以 1.25 倍余量，向上取整得到 63 ms 预算。GC 后增量堆与 Client fold 缩放预算不属于壁钟时间，因此只使用 1.25 倍余量。128 MB 完成性检查仍是独立的瞬时分配限制。由此得到的 first-open 时间上限、受限堆检查与 Client fold 上限都会拒绝已知退化。栈前参考提交固定为 `0d7ea53743e273930a31e9e2b6ca682f21dd4ca5`，只用于校准和评审预算；CI 不 checkout 或执行历史仓库。预算是源码中的受评审常量，不由环境变量覆盖。 ## 校准证据 @@ -54,12 +54,14 @@ Session benchmark 使用固定参数合成 released-v0 输入：200 轮，每轮 栈前实现以 V0 作为当前格式，因此 first open 不改变磁盘表示；它的原生 V0 首屏历史与 Agent resume 测量同时适用于两个生命周期行。 +`ca3ffe95dac2c55eefeb16ed9b61067bbd19ee90` 上的[标准双 CPU 运行](https://github.com/deepseek-harness/deepseek-harness/actions/runs/34023970384/job/101461539961)使用 Node 24.20.0 x64 和 Ubuntu 镜像 `20260831.293.1`。当前 generation `open` 的五次样本为 49.2、47.4、49.1、48.6 和 48.1 ms：中位数 48.6 ms，最大值 49.2 ms。取整后的 50 ms CI 预期值给出 63 ms 上限，不重复乘以 2 倍机器系数。日志标明两个可用 CPU，但未记录型号；它无法区分硬件变化与 Node 版本变化的影响。这是端点专属的运行器校准，不是应用优化或参考机器新测量的证据。其他每项 benchmark 均通过既有预算。确定性正反例在历史 30 ms 上限下拒绝实测中位数，在 63 ms 下接受它，拒绝合成的 75 ms reopen 中位数，并以未改变的 550 ms 上限拒绝合成的 4,000 ms 首次打开耗时。这些正反例验证预算执行，不代表测得新的退化。 + 校准后的源码预算如下： | 测量项 | 参考机预期 | CI 预算 | |---|---:|---:| | First-open `open` | 220 ms | 550 ms | -| 当前 generation `open` | 12 ms | 30 ms | +| 当前 generation `open` | 12 ms（历史参考值；CI 预期值：50 ms） | 63 ms | | 完整 read | 8 ms | 20 ms | | Session restore | 24 ms | 60 ms | | Projection | 14 ms | 35 ms | diff --git a/.agents/notes/implemented/testing/2026-09-06-standard-hosted-benchmark-runner.i18n.yaml b/.agents/notes/implemented/testing/2026-09-06-standard-hosted-benchmark-runner.i18n.yaml new file mode 100644 index 0000000000..feddeffb9f --- /dev/null +++ b/.agents/notes/implemented/testing/2026-09-06-standard-hosted-benchmark-runner.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write .agents/notes/implemented/testing/2026-09-06-standard-hosted-benchmark-runner.md +2026-09-06-standard-hosted-benchmark-runner.md: af95af6ee8cef1128a5475863ea7a1aaf66f30b9 +2026-09-06-standard-hosted-benchmark-runner.zh.md: 47095a5b2e91318c2f3ec5ac3cc9f6df3bce04d6 diff --git a/.agents/notes/implemented/testing/2026-09-06-standard-hosted-benchmark-runner.md b/.agents/notes/implemented/testing/2026-09-06-standard-hosted-benchmark-runner.md new file mode 100644 index 0000000000..af95af6ee8 --- /dev/null +++ b/.agents/notes/implemented/testing/2026-09-06-standard-hosted-benchmark-runner.md @@ -0,0 +1,26 @@ +# Agent Note: Standard hosted runner for required benchmarks + +Status: implemented + +English | [中文](2026-09-06-standard-hosted-benchmark-runner.zh.md) + +## Problem + +Wall-clock performance checks need an isolated execution lane and a consistent runner class. Routing them through the enterprise Linux failover switch makes their measurements depend on either larger hosted capacity or a shared self-hosted VM, while also consuming capacity needed by parallel correctness checks. + +## Decision + +The required benchmark job in [ci.yml](../../../../.github/workflows/ci.yml) uses the standard GitHub-hosted `ubuntu-24.04` runner independently of Linux failover. It always attempts to restore the pnpm store cache and retains a standalone benchmark lane. The complete job has a 15-minute timeout covering setup, installation, builds, and measurements. This bounds infrastructure execution, not an individual performance assertion. + +The [Session performance decision](2026-09-04-session-open-performance-gate.md) continues to own workloads, timing and memory budgets, worker isolation, and calibration. Only current-generation `open` uses an endpoint-specific 50 ms standard-runner expectation with the existing 1.25× headroom, giving a 63 ms limit. All other performance budgets and the worker, test, and hook deadlines remain unchanged. Successful raw measurements remain in the Actions log through step-local `DSH_GATE_VERBOSE=1`. The hardware-comparison workflows retain their deliberately different runner sizes. + +## Alternatives considered + +- Enterprise or shared self-hosted routing retains more build capacity but ties the measurement environment to unrelated failover operations. +- Increasing performance thresholds without endpoint measurements conflates a bounded CI execution with a regression allowance. Threshold changes require measured calibration and positive and negative controls. + +## Consequences + +A standard runner trades parallel build capacity for a fixed measurement class without removing the required verdict. Cache misses and runner variation can still affect total duration. Each runner change needs an actual hosted benchmark run before its job timeout is treated as validated; local workflow assertions alone cannot establish execution time. + +The owning [workflow tests](../../../../scripts/ci-workflow.spec.ts) pin runner routing, unconditional cache restoration, required status, and the job timeout. Negative controls reject failover routing, a cache condition, and the former 30-minute job bound. diff --git a/.agents/notes/implemented/testing/2026-09-06-standard-hosted-benchmark-runner.zh.md b/.agents/notes/implemented/testing/2026-09-06-standard-hosted-benchmark-runner.zh.md new file mode 100644 index 0000000000..47095a5b2e --- /dev/null +++ b/.agents/notes/implemented/testing/2026-09-06-standard-hosted-benchmark-runner.zh.md @@ -0,0 +1,26 @@ +# Agent Note: 必需 benchmark 使用标准托管运行器 + +Status: implemented + +[English](2026-09-06-standard-hosted-benchmark-runner.md) | 中文 + +## 问题 + +壁钟性能检查需要独立执行的 lane 和一致的运行器类别。通过企业 Linux 故障转移开关路由这些检查，会让测量取决于大型托管运行器或共享自托管虚拟机，同时占用并行正确性检查所需的容量。 + +## 决定 + +[ci.yml](../../../../.github/workflows/ci.yml) 中的必需 benchmark job 使用标准 GitHub 托管 `ubuntu-24.04` 运行器，不受 Linux 故障转移影响。它始终尝试恢复 pnpm 存储缓存，并保留独立的 benchmark lane。整个 job 的超时为 15 分钟，覆盖准备、安装、构建和测量。这限制的是基础设施执行时间，而非单项性能断言。 + +[Session 性能决策](2026-09-04-session-open-performance-gate.zh.md) 继续拥有工作负载、时间和内存预算、worker 隔离及校准。仅当前 generation `open` 使用端点专属的 50 ms 标准运行器预期值，乘以既有 1.25 倍余量后得到 63 ms 上限。其他性能预算以及 worker、测试和钩子的截止时间均保持不变。步骤级 `DSH_GATE_VERBOSE=1` 使成功运行的原始测量保留在 Actions 日志中。硬件比较工作流保留有意设置的不同运行器规格。 + +## 考虑过的替代方案 + +- 企业或共享自托管路由保留更多构建容量，但使测量环境受无关故障转移操作影响。 +- 没有端点测量就提高性能阈值，会混淆有界 CI 执行与退化容许量。阈值调整需要实测校准及正反例。 + +## 后果 + +标准运行器以并行构建容量换取固定测量类别，不移除必需判定。缓存未命中和运行器波动仍会影响总耗时。每次更换运行器都需要实际托管 benchmark 运行，才能认定 job 超时经过验证；本地工作流断言无法单独证明执行耗时。 + +所属[工作流测试](../../../../scripts/ci-workflow.spec.ts) 固定运行器路由、无条件缓存恢复、必需状态及 job 超时。反例验证拒绝故障转移路由、缓存条件和原来的 30 分钟 job 上限。 diff --git a/.agents/notes/proposed/architecture/2026-09-02-system-prompt-as-surface-node.i18n.yaml b/.agents/notes/proposed/architecture/2026-09-02-system-prompt-as-surface-node.i18n.yaml new file mode 100644 index 0000000000..ebc503e4f4 --- /dev/null +++ b/.agents/notes/proposed/architecture/2026-09-02-system-prompt-as-surface-node.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write .agents/notes/proposed/architecture/2026-09-02-system-prompt-as-surface-node.md +2026-09-02-system-prompt-as-surface-node.md: 56577a7199235e95f4a7c6500140c8a8841d48dc +2026-09-02-system-prompt-as-surface-node.zh.md: da864fd300c93cae0210e758562b823c0a61679a diff --git a/.agents/notes/proposed/architecture/2026-09-02-system-prompt-as-surface-node.md b/.agents/notes/proposed/architecture/2026-09-02-system-prompt-as-surface-node.md new file mode 100644 index 0000000000..56577a7199 --- /dev/null +++ b/.agents/notes/proposed/architecture/2026-09-02-system-prompt-as-surface-node.md @@ -0,0 +1,78 @@ +# Agent Note: The system prompt is surface node 0 + +Status: proposed + +English | [中文](2026-09-02-system-prompt-as-surface-node.zh.md) + +## Problem + +The system prompt has a different durable representation from every other message the model reads. Conversation messages are surface events (`user/message`, `assistant/message`, `tool/result`) folded in seq order by `Session.deriveMessages()`; the system prompt is the `system` field of the log-only `request/header` snapshot, and each DeepSeek serializer prepends it as wire message 0 (`serializeRequest`, `serializeRequestWithImages`). The [reconstructable-requests Agent Note](../../implemented/architecture/2026-07-05-reconstructable-requests.md) made both halves durable, but it left one model-visible fact with two homes: the surface owns the messages, the header owns the message in front of them. + +That split forces every reader of "what did the model see" to join two sources. The compaction summarizer (`buildSummarizationInput`) copies `header.system` in front of the region's derived messages; `dsh-token-meter` estimates the system prompt from the header while pricing every other message from the surface; the Web request-prompt card, the trajectory view, and the snapshot normalizer's `{{system}}` placeholder each read the header on their own. The loop's change detection is also split: `headerEquals` compares `system` byte-for-byte beside `config` and `tools`, so a prompt change and a tool change are indistinguishable in the log (`request/header` reason `change`) even though they are different operations on the conversation. + +The split also blocks the next step. A model that accepts a mid-conversation `system` message as a prompt replacement needs the harness to append a system-role message to history; with the prompt living in the header there is no surface representation to append, and the header would have to be frozen by special case. The [in-history replacement proposal](../feature/2026-09-02-in-history-system-prompt-replacement.md) depends on this note. + +## Proposal + +Move the system prompt onto the surface. It becomes an ordinary surface event, `system/message`, and every prompt lifecycle operation is one of the two existing `SurfaceOp` variants applied to that event type. The wire request does not change: the surface fold yields the same message list the serializers already build today, with the system message first. + +### The event + +`system/message` joins `SurfaceEventType` beside `user/message`, `assistant/message`, and `tool/result`. Its payload mirrors `tool/result`: `{ turn, step, message }`, where `message` is a `Message` with `role: 'system'`, exactly one text block holding the rendered prompt, and source `{ kind: 'plugin', plugin: '@deepseek-ai/dsh-system-prompt' }`. `deriveEventMessage` projects it verbatim, so `deriveMessages()` returns the system message at its surface position and both DeepSeek serializers, which already pass a `role: 'system'` history message through unchanged, emit it as wire message 0. `EpochHeader.system` is removed; the header keeps `config`, `adapterDefaults`, and `tools`. + +### The operations + +| Situation | Surface operation | +|---|---| +| First request of a session with a non-empty rendered prompt | append `system/message` as surface node 0, before the first `user/message` of the step | +| Rendered prompt differs from the prompt at node 0 | replace node 0: `surfaceOp: { op: 'replace', start: , end: }`, `sourceEventSeqs: []` | +| Rendered prompt is empty on the first request | no system node; a later non-empty prompt appends node 0 when the surface has no system node yet | + +Replacing node 0 is today's head rewrite expressed on the surface: the provider prefix changes from the first token, the log records the shadowed node through `sourceEventSeqs`, and `replaceGeneration` advances exactly as it does for a compaction replacement, so the loop's existing `startsSeries` detection (`requestSurfaceGeneration !== surfaceGeneration`) covers the prompt change without a `system` comparison in `headerEquals`. `request/header` keeps reasons `initial`, `resume`, `change`, and `series`; `change` now means config or tools changed. + +### Ownership in the loop + +`dsh-agent-loop` owns a `SystemPromptProjection` beside `RuntimeContextProjection` in `runtime-context.ts`. It restores the current system node from the log (the latest surviving `system/message` on the surface), follows `session/event` for new system nodes and for replacements whose `sourceEventSeqs` shadow the retained one, and returns the uncommitted append or replace intent when the rendered prompt differs. `turn()` commits that intent immediately before the step's `user/message` events, so the log order is the wire order. `step()` no longer passes `system` to `buildRequest`; the request is `header.config` plus `deriveMessages()` plus `header.tools`. The `dsh-agent-loop/invariant` companion keeps comparing the rebuilt request against the frozen one, now with the system message inside `messages`. `docs/architecture.md` records the new loop step order: claim, assemble, project system prompt, project runtime context, pre-step, commit system node, commit user messages, build request. + +### Consumers retargeted + +| Consumer | Today | After | +|---|---|---| +| DeepSeek serializers (`serializeRequest`, `serializeRequestWithImages`) | prepend `options.system` | serialize `options.messages` only; `GenerateOptions.system` remains for direct one-shot callers such as the summarizer and title providers | +| `compaction-basic` `buildSummarizationInput` | `header.system` + region messages | node 0's derived message + region messages, still a genuine prefix of the routed request | +| `compaction-basic` `selectCompactableRange` | head-anchored at `surfaceNodes[0]` | anchored at the first non-system node; node 0 is never inside a compaction range | +| `dsh-token-meter` system estimate | `header.system` length | the system node is priced like every other surface node; the context breakdown labels it by its source plugin | +| Web request-prompt card, trajectory request-header node, request inspection | read `header.system` | read the `system/message` node; the card keeps its collapsed inspectable presentation and is never a chat bubble | +| Snapshot normalizer `{{system}}` placeholder, plan-mode tests asserting `header.system` | header | the system node's text | +| TypeScript and Python SDK expected outputs | no system event | include the `system/message` event | +| Human transcript projections (`isAppendSurfaceEvent` readers) | no system events | skip `system/message`; it is model history, not conversation | + +`RuntimeContextProjection` and `SystemPromptProjection` are symmetric: both watch owned surface nodes and their shadowing through `sourceEventSeqs`, and both hand the loop an uncommitted message that `turn()` commits. The difference is the role and the operation set — runtime context appends user-role snapshots only, the system prompt appends once and then replaces. + +## Alternatives considered + +**Keep `header.system` and add `system/message` only for updates.** Two homes for one fact: every consumer above would read the header for message 0 and the surface for later messages, and the loop would need a special case that ignores `system` in `headerEquals` while a surface system node exists. Rejected because the point of the change is one representation. + +**A dedicated log-only `system-prompt/change` event that rewrites the header.** Preserves the header as the home of the prompt and records changes as their own event kind, but still cannot express a system message inside history, so the in-history proposal would need a second mechanism anyway. Rejected. + +**Synthesize the system message inside the adapter from consecutive headers.** The adapter is stateless per request and never sees the log; a wire history that depends on adapter state is not reconstructable from the surface fold. Rejected. + +**Express the prompt as a `user/message` snapshot like runtime context.** Reuses an existing event type but sends the wrong role, so a model that treats a system message as authoritative would not. Rejected. + +## Acceptance criteria + +- `SurfaceEventType` contains `system/message`; `deriveEventMessage` projects it; `Session.append('system/message', …)` requires a `SurfaceIntent` like the other surface events. +- `EpochHeader` has no `system` field; `headerEquals` compares `config`, `adapterDefaults`, and `tools` only. +- A first request with a non-empty rendered prompt appends `system/message` as surface node 0 before the step's first `user/message`; a changed prompt replaces node 0 with `sourceEventSeqs` naming the shadowed node; an unchanged prompt appends nothing. +- The DeepSeek wire request for every loop step is byte-identical to today's for the same session history: system first, then the folded conversation. +- Compaction never selects node 0; the summarizer's replayed prefix starts with node 0's derived message. +- `dsh-token-meter`, the Web request-prompt card, trajectory and inspection views, the snapshot normalizer, plan-mode tests, and both SDK expected outputs read the system node; the `dsh-agent-loop/invariant` companion rebuilds requests with the system message inside `messages`. +- Keyless recorded snapshots that exercise a mid-session prompt change (plan mode entering and leaving) show a replaced node 0 instead of a `request/header` `change`. +- `docs/architecture.md`, the `dsh-agent-loop`, `dsh-session`, `dsh-system-prompt`, `dsh-compaction-basic`, and `dsh-token-meter` READMEs, and the reconstructable-requests Agent Note describe the surface node as the home of the system prompt. + +## Risks + +- Every reader of `header.system` moves in one change; a missed reader fails at compile time because the field is gone, which is the intended failure mode. +- Compaction region selection gains an invariant (node 0 is never compacted). A compaction provider other than `compaction-basic` that anchors at `surfaceNodes[0]` would shadow the prompt; the `dsh-session` surface manager rejects a replacement whose range covers surface node 0 while node 0 is a `system/message` unless the replacing event is itself a `system/message` covering exactly that node, so the invariant is enforced where the operation happens, not only in the shipped provider. System nodes at later positions carry no such protection: a compaction range may shadow them. +- Replacing node 0 advances `replaceGeneration`, which today means "compaction happened" to some readers; those readers switch to inspecting the replacement event's type. +- Recorded snapshot fixtures whose logs contain `header.system` are re-recorded; the fixtures, not the normalizer, change. diff --git a/.agents/notes/proposed/architecture/2026-09-02-system-prompt-as-surface-node.zh.md b/.agents/notes/proposed/architecture/2026-09-02-system-prompt-as-surface-node.zh.md new file mode 100644 index 0000000000..da864fd300 --- /dev/null +++ b/.agents/notes/proposed/architecture/2026-09-02-system-prompt-as-surface-node.zh.md @@ -0,0 +1,78 @@ +# Agent Note: 系统提示词是 surface 的第 0 号节点 + +Status: proposed + +[English](2026-09-02-system-prompt-as-surface-node.md) | 中文 + +## Problem + +系统提示词的持久化表示与模型读到的其他所有消息都不同。对话消息是 surface 事件（`user/message`、`assistant/message`、`tool/result`），由 `Session.deriveMessages()` 按 seq 顺序折叠；系统提示词则是仅记日志的 `request/header` 快照中的 `system` 字段，每个 DeepSeek 序列化器把它前置为协议消息 0（`serializeRequest`、`serializeRequestWithImages`）。[可重建请求 Agent Note](../../implemented/architecture/2026-07-05-reconstructable-requests.zh.md) 让两半都成为持久数据，却让一个模型可见的事实拥有两个归属：surface 拥有消息，header 拥有排在这些消息之前的那条消息。 + +这种拆分迫使每个想知道「模型看到了什么」的读取方都要合并两个来源。压缩摘要器（`buildSummarizationInput`）把 `header.system` 复制到区域派生消息之前；`dsh-token-meter` 从 header 估算系统提示词，却从 surface 为其他每条消息计价；Web 请求提示词卡片、轨迹视图和快照归一化器的 `{{system}}` 占位符各自单独读取 header。循环的变更检测同样被拆开：`headerEquals` 在 `config` 和 `tools` 旁边逐字节比较 `system`，因此提示词变更与工具变更在日志中无法区分（`request/header` 的 reason 都是 `change`），尽管它们是对对话的两种不同操作。 + +这种拆分还阻塞了下一步。一个把对话中途的 `system` 消息当作提示词替换来接受的模型，需要 harness 向历史追加一条 system 角色消息；当提示词住在 header 里时，没有可追加的 surface 表示，header 也只能靠特例被冻结。[历史内替换提案](../feature/2026-09-02-in-history-system-prompt-replacement.zh.md) 依赖本 Agent Note。 + +## Proposal + +把系统提示词搬到 surface 上。它成为一个普通的 surface 事件 `system/message`，提示词生命周期中的每个操作都是对该事件类型施加现有两种 `SurfaceOp` 变体之一。协议请求不变：surface 折叠产出的消息列表与序列化器今天构建的完全相同，系统消息在最前面。 + +### 事件 + +`system/message` 加入 `SurfaceEventType`，与 `user/message`、`assistant/message`、`tool/result` 并列。它的载荷与 `tool/result` 对称：`{ turn, step, message }`，其中 `message` 是 `role: 'system'` 的 `Message`，恰好一个文本块承载渲染后的提示词，source 为 `{ kind: 'plugin', plugin: '@deepseek-ai/dsh-system-prompt' }`。`deriveEventMessage` 逐字投影它，因此 `deriveMessages()` 在其 surface 位置返回系统消息，而两个 DeepSeek 序列化器本已原样透传 `role: 'system'` 的历史消息，会把它作为协议消息 0 发出。`EpochHeader.system` 被移除；header 保留 `config`、`adapterDefaults` 和 `tools`。 + +### 操作 + +| 情形 | surface 操作 | +|---|---| +| 会话首个请求且渲染后的提示词非空 | 追加 `system/message` 作为 surface 第 0 号节点，位于该步骤首条 `user/message` 之前 | +| 渲染后的提示词与第 0 号节点不同 | 替换第 0 号节点：`surfaceOp: { op: 'replace', start: <第 0 号节点的 seq>, end: <同一值> }`，`sourceEventSeqs: [<第 0 号节点的 seq>]` | +| 首个请求时渲染后的提示词为空 | 没有系统节点；之后出现非空提示词且 surface 尚无系统节点时，追加为第 0 号节点 | + +替换第 0 号节点就是今天的头部重写在 surface 上的表达：提供方前缀从第一个 token 起改变，日志通过 `sourceEventSeqs` 记录被遮蔽的节点，`replaceGeneration` 与压缩替换时一样推进，因此循环现有的 `startsSeries` 检测（`requestSurfaceGeneration !== surfaceGeneration`）无需在 `headerEquals` 中比较 `system` 即可覆盖提示词变更。`request/header` 保留 `initial`、`resume`、`change`、`series` 四种 reason；`change` 现在表示 config 或 tools 变更。 + +### 循环中的归属 + +`dsh-agent-loop` 在 `runtime-context.ts` 中与 `RuntimeContextProjection` 并列拥有一个 `SystemPromptProjection`。它从日志恢复当前系统节点（surface 上最新存活的 `system/message`），跟随 `session/event` 观察新的系统节点以及 `sourceEventSeqs` 遮蔽了所保留节点的替换，并在渲染后的提示词不同时返回未提交的追加或替换意图。`turn()` 紧接在该步骤的 `user/message` 事件之前提交该意图，因此日志顺序即协议顺序。`step()` 不再向 `buildRequest` 传递 `system`；请求由 `header.config`、`deriveMessages()` 和 `header.tools` 构成。`dsh-agent-loop/invariant` 伴随组件继续把重建的请求与冻结的请求比较，只是系统消息现在位于 `messages` 内。`docs/architecture.md` 记录新的循环步骤顺序：领取、装配、投影系统提示词、投影运行时上下文、pre-step、提交系统节点、提交用户消息、构建请求。 + +### 消费方迁移 + +| 消费方 | 现状 | 变更后 | +|---|---|---| +| DeepSeek 序列化器（`serializeRequest`、`serializeRequestWithImages`） | 前置 `options.system` | 只序列化 `options.messages`；`GenerateOptions.system` 为摘要器、标题提供方等直接单次调用方保留 | +| `compaction-basic` 的 `buildSummarizationInput` | `header.system` + 区域消息 | 第 0 号节点的派生消息 + 区域消息，仍是已路由请求的真实前缀 | +| `compaction-basic` 的 `selectCompactableRange` | 锚定在头部 `surfaceNodes[0]` | 锚定在首个非系统节点；第 0 号节点永不落入压缩范围 | +| `dsh-token-meter` 的系统提示词估算 | `header.system` 长度 | 系统节点与其他每个 surface 节点一样计价；上下文明细按其 source 插件标注 | +| Web 请求提示词卡片、轨迹请求 header 节点、请求检视 | 读取 `header.system` | 读取 `system/message` 节点；卡片保持折叠可检视的呈现，永不作为聊天气泡 | +| 快照归一化器的 `{{system}}` 占位符、断言 `header.system` 的 plan-mode 测试 | header | 系统节点的文本 | +| TypeScript 与 Python SDK 期望输出 | 没有系统事件 | 包含 `system/message` 事件 | +| 人类转录投影（`isAppendSurfaceEvent` 的读取方） | 没有系统事件 | 跳过 `system/message`；它是模型历史，不是对话 | + +`RuntimeContextProjection` 与 `SystemPromptProjection` 是对称的：两者都通过 `sourceEventSeqs` 观察自己拥有的 surface 节点及其被遮蔽的情况，都把一条未提交的消息交给循环由 `turn()` 提交。区别在于角色与操作集——运行时上下文只追加 user 角色快照，系统提示词追加一次之后只做替换。 + +## Alternatives considered + +**保留 `header.system`，只为更新添加 `system/message`。** 一个事实两个归属：上述每个消费方都要从 header 读消息 0、从 surface 读后续消息，循环还需要一个在 surface 存在系统节点时让 `headerEquals` 忽略 `system` 的特例。被否决，因为本次变更的目的就是单一表示。 + +**用专门的仅记日志事件 `system-prompt/change` 重写 header。** 保留 header 作为提示词归属，并把变更记录为独立事件种类，但仍无法表达历史内部的系统消息，历史内替换提案还是需要第二套机制。被否决。 + +**在适配器内根据相邻 header 合成系统消息。** 适配器逐请求无状态且从不接触日志；依赖适配器状态的协议历史无法从 surface 折叠重建。被否决。 + +**像运行时上下文那样用 `user/message` 快照表达提示词。** 复用了现有事件类型，却发送了错误的角色，因此把系统消息视为权威的模型不会这样对待它。被否决。 + +## Acceptance criteria + +- `SurfaceEventType` 包含 `system/message`；`deriveEventMessage` 投影它；`Session.append('system/message', …)` 与其他 surface 事件一样要求 `SurfaceIntent`。 +- `EpochHeader` 没有 `system` 字段；`headerEquals` 只比较 `config`、`adapterDefaults` 和 `tools`。 +- 渲染后的提示词非空的首个请求在该步骤首条 `user/message` 之前追加 `system/message` 作为 surface 第 0 号节点；提示词变更时以指明被遮蔽节点的 `sourceEventSeqs` 替换第 0 号节点；提示词不变时不追加任何内容。 +- 对同一会话历史，每个循环步骤的 DeepSeek 协议请求与今天逐字节一致：系统消息在先，随后是折叠后的对话。 +- 压缩永不选中第 0 号节点；摘要器回放的前缀以第 0 号节点的派生消息开头。 +- `dsh-token-meter`、Web 请求提示词卡片、轨迹与检视视图、快照归一化器、plan-mode 测试以及两个 SDK 的期望输出都读取系统节点；`dsh-agent-loop/invariant` 伴随组件重建请求时系统消息位于 `messages` 内。 +- 演练会话中途提示词变更（进入与退出 plan 模式）的无密钥录制快照显示被替换的第 0 号节点，而不是 `request/header` 的 `change`。 +- `docs/architecture.md`、`dsh-agent-loop`、`dsh-session`、`dsh-system-prompt`、`dsh-compaction-basic`、`dsh-token-meter` 的 README 以及可重建请求 Agent Note 都把 surface 节点描述为系统提示词的归属。 + +## Risks + +- `header.system` 的每个读取方在一次变更中迁移；遗漏的读取方因字段消失而在编译期失败，这正是预期的失败方式。 +- 压缩范围选择新增一条不变量（第 0 号节点永不被压缩）。除 `compaction-basic` 以外、锚定在 `surfaceNodes[0]` 的压缩提供方会遮蔽提示词；`dsh-session` 的 surface 管理器拒绝在第 0 号节点是 `system/message` 时覆盖第 0 号节点的替换，除非替换事件本身是恰好覆盖该节点的 `system/message`，因此不变量在操作发生处被强制，而不只在随发的提供方中。位于更后位置的系统节点没有此类保护：压缩范围可以遮蔽它们。 +- 替换第 0 号节点会推进 `replaceGeneration`，今天有些读取方把它理解为「发生了压缩」；这些读取方改为检查替换事件的类型。 +- 日志中包含 `header.system` 的录制快照 fixture 需要重新录制；改变的是 fixture，而不是归一化器。 diff --git a/.agents/notes/proposed/feature/2026-09-02-in-history-system-prompt-replacement.i18n.yaml b/.agents/notes/proposed/feature/2026-09-02-in-history-system-prompt-replacement.i18n.yaml new file mode 100644 index 0000000000..6d808e5101 --- /dev/null +++ b/.agents/notes/proposed/feature/2026-09-02-in-history-system-prompt-replacement.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write .agents/notes/proposed/feature/2026-09-02-in-history-system-prompt-replacement.md +2026-09-02-in-history-system-prompt-replacement.md: 229e92500936ec8742f341c4dc18184eb9a2fe0c +2026-09-02-in-history-system-prompt-replacement.zh.md: f776f25a43f936024564488c1c53e9728fa19827 diff --git a/.agents/notes/proposed/feature/2026-09-02-in-history-system-prompt-replacement.md b/.agents/notes/proposed/feature/2026-09-02-in-history-system-prompt-replacement.md new file mode 100644 index 0000000000..229e925009 --- /dev/null +++ b/.agents/notes/proposed/feature/2026-09-02-in-history-system-prompt-replacement.md @@ -0,0 +1,74 @@ +# Agent Note: In-history system prompt replacement for cache-stable prompt changes + +Status: proposed + +English | [中文](2026-09-02-in-history-system-prompt-replacement.zh.md) + +## Problem + +Every system prompt change costs the whole provider prefix cache. The loop renders the prompt on every step; when the bytes differ — a plan-mode section entering or leaving, a skill or tool guidance section registering, an agent-scoped persona shadow, a changed `{{model}}` variable — the request's message 0 changes and the DeepSeek context cache misses from the first token. Long agentic sessions pay this repeatedly, and the [runtime-context snapshot design](../../archived/feature/2026-07-30-current-sandbox-policy-context.md) exists precisely because moving a changing fact out of the prompt was the only way to keep the prefix stable. + +A DeepSeek model, provided as an unpublished model fact for this proposal, removes that constraint: it accepts a `system` message at any position of the conversation and treats the latest one as the complete effective system prompt, replacing the leading one. Tool schemas remain part of the cached prefix, so a tool-set change still invalidates the cache. With that model the harness can append the new prompt after the cached history instead of rewriting message 0, and the prefix stays warm. + +The harness has the representation for this only after the [system prompt is surface node 0](../architecture/2026-09-02-system-prompt-as-surface-node.md): a prompt change is then an operation on `system/message` surface nodes, and the choice between "replace node 0" and "append a new node" is a per-model decision. + +## Proposal + +For a model route that declares the capability, the loop appends a new `system/message` surface node instead of replacing node 0 when the rendered prompt changes and the prefix would otherwise survive. Everything else in the [surface-node design](../architecture/2026-09-02-system-prompt-as-surface-node.md) is unchanged: the event type, the projection owner, the serializers, and the presentation. + +### Capability + +The DeepSeek adapter's catalog model gains a validated optional field, `systemPromptUpdate`, with the single accepted value `'in-history'`; absence means the model needs message 0 rewritten. The adapter surfaces it on `LlmResolvedModelInfo` and `prepareCall()` returns it beside `context.contextWindow`, so the loop reads it from the same registration-bound metadata it already consumes. No default catalog entry declares it until the model is released; a deployment enables it through the `models` list in `cordis.yml`. Models without the field — including every current default entry and every `dsh-llm-pi-ai` route — keep the replace-node-0 behaviour exactly. + +### The decision rule + +`SystemPromptProjection` tracks the **effective prompt**: the text of the latest surviving `system/message` on the surface (node 0 when no later system node exists). When the rendered prompt differs from the effective prompt: + +| Route capability | Prefix state | Operation | +|---|---|---| +| none | any | replace node 0 | +| `in-history` | the current request series continues (no compaction since the last request, no tools or config change) | append a new `system/message` before the step's `user/message` events | +| `in-history` | a new series starts (compaction replaced the surface, or `request/header` records a `change` for tools or config) and no mid-history system node survives | replace node 0 with the current prompt | +| `in-history` | a new series starts but a mid-history system node survives | append a new `system/message`; node 0 stays as it is | + +The third row exists because a series start already costs the cache; folding the prompt back into node 0 keeps the history short. The fourth row exists because the surface has no delete operation: replacing node 0 while a later system node survives would leave the model reading the later, stale node as authoritative, so the loop appends instead. In-history mode never rewrites node 0 while any later system node survives. + +Resume follows the mid-session rule. A new loop instance restores the effective prompt from the log and, when the freshly rendered prompt differs, appends — the provider cache may still be warm across a process boundary, and the `resume` header is not a series start. + +### Presentation and accounting + +A mid-history `system/message` uses the same collapsed request-prompt inspection card as node 0, labelled as a prompt update at its position in the request; it is never a chat bubble, transcript projections skip it, and SDK projections expose it as a typed event. `dsh-token-meter` prices it like any other surface node, so the per-step context breakdown shows the accumulated cost of retained prompt versions until compaction shadows them. `cacheReadTokens` on the following `assistant/message` usage is the observable effect: for a capable route the value covers the prefix through the last cached message; for a non-capable route it drops to the shared-prefix detection floor. + +### Verification plan + +- Unit tests in `dsh-agent-loop` for the projection: append on a mid-series change, replace on a series start without surviving mid-history nodes, append on a series start with one, append on resume, no operation when unchanged, and replace-only behaviour for a route without the capability. +- Unit tests in `dsh-llm-deepseek` for catalog validation (`systemPromptUpdate` accepts `'in-history'` only) and for `prepareCall()` surfacing the field. +- A keyless recorded snapshot under `snapshots/` whose composition declares the capability on the mock route and toggles plan mode mid-session, pinning the appended `system/message` and the untouched node 0; TypeScript and Python SDK expected outputs include the appended event. +- A real-API e2e that runs two steps with a prompt change against a capable route and asserts that the second request's `cacheReadTokens` is at least the first request's prompt token count. It resolves its route from the standard credential and base-URL mechanism and self-skips when no capable route is configured. + +## Alternatives considered + +**Send only the changed sections as a delta.** The model treats the latest system message as the complete prompt, so a delta would silently drop every unchanged section. Rejected on the model contract. + +**Enable in-history mode by plugin config instead of a model capability.** A deployment flag could pair a non-capable model with appended system messages, which such a model would read as ordinary history at best. The capability belongs to the route that honours it; the adapter catalog already carries per-model capacities. Rejected. + +**Always append, never re-baseline.** One rule, but node 0 would stay stale for the life of the session and every request after compaction would carry the stale head plus the replacement. Re-baselining at a series start costs nothing extra because the cache is already lost there. Rejected. + +**Re-baseline on every resume.** Accepts one cache miss per process restart for a simpler resume path. The cache persists across restarts for hours to days, and the log already carries what resume needs. Rejected. + +**Place the system message after the step's user messages.** Both positions sit after the cached prefix, but the model then reads the instructions after the input it must apply them to; system-before-user matches the leading position's ordering. Rejected. + +## Acceptance criteria + +- `DeepSeekCatalogModel.systemPromptUpdate` is validated at load, exposed through `LlmResolvedModelInfo`, and returned by `prepareCall()`; a misspelt value fails at load. +- On a capable route a mid-series prompt change appends `system/message` before the step's `user/message` events and node 0 is unchanged; on a non-capable route the same change replaces node 0. +- On a capable route a series start with no surviving mid-history system node replaces node 0; with a surviving one it appends. +- A resumed loop instance whose rendered prompt differs appends on a capable route. +- The recorded snapshot and both SDK expected outputs pin the appended event; the e2e asserts the cache-hit inequality when a capable route is configured and skips otherwise. +- The `dsh-llm-deepseek`, `dsh-agent-loop`, and `dsh-system-prompt` READMEs document the capability, the decision rule, and the KV Cache effect; `docs/config-catalog.md` lists the field. + +## Risks + +- The model contract is unpublished; the note records it as provided. If the released model narrows it (for example, honouring only the latest system message within a bounded window), the decision rule needs a re-baseline trigger beyond series starts. +- Retained prompt versions accumulate in history until compaction shadows them. Each version costs its tokens on every request in the series; a deployment whose prompt changes on most steps would be better served by moving that fact into runtime context. +- A proxy that rewrites or reorders system messages breaks the replacement semantics silently; the e2e's cache-hit assertion is the detector. diff --git a/.agents/notes/proposed/feature/2026-09-02-in-history-system-prompt-replacement.zh.md b/.agents/notes/proposed/feature/2026-09-02-in-history-system-prompt-replacement.zh.md new file mode 100644 index 0000000000..f776f25a43 --- /dev/null +++ b/.agents/notes/proposed/feature/2026-09-02-in-history-system-prompt-replacement.zh.md @@ -0,0 +1,74 @@ +# Agent Note: 历史内系统提示词替换，实现缓存稳定的提示词变更 + +Status: proposed + +[English](2026-09-02-in-history-system-prompt-replacement.md) | 中文 + +## Problem + +每一次系统提示词变更都要付出整个提供方前缀缓存的代价。循环在每个步骤渲染提示词；一旦字节不同——plan 模式片段进入或退出、某个 skill 或工具指引片段完成注册、agent 作用域的 persona 遮蔽、`{{model}}` 变量改变——请求的消息 0 随之改变，DeepSeek 上下文缓存从第一个 token 起失效。长时间的 agent 会话反复为此付费，而[运行时上下文快照设计](../../archived/feature/2026-07-30-current-sandbox-policy-context.md)之所以存在，正是因为把会变化的事实移出提示词是保持前缀稳定的唯一办法。 + +一个 DeepSeek 模型——作为本提案所依据的未公开模型事实——移除了这一限制：它接受对话任意位置的 `system` 消息，并把最新一条视为完整的有效系统提示词，替换最前面那条。工具 schema 仍属于被缓存的前缀，因此工具集变更仍会使缓存失效。有了这样的模型，harness 可以把新提示词追加到已缓存的历史之后而不是重写消息 0，前缀就能保持热态。 + +只有在[系统提示词成为 surface 第 0 号节点](../architecture/2026-09-02-system-prompt-as-surface-node.zh.md)之后，harness 才拥有实现这一点的表示：提示词变更随之成为对 `system/message` surface 节点的操作，而「替换第 0 号节点」与「追加新节点」之间的选择是逐模型的决定。 + +## Proposal + +对于声明了该能力的模型路由，当渲染后的提示词变化且前缀本可存活时，循环追加一个新的 `system/message` surface 节点而不是替换第 0 号节点。[surface 节点设计](../architecture/2026-09-02-system-prompt-as-surface-node.zh.md)中的其他一切不变：事件类型、投影的拥有者、序列化器和呈现。 + +### 能力 + +DeepSeek 适配器的目录模型新增一个经校验的可选字段 `systemPromptUpdate`，唯一接受的值是 `'in-history'`；缺省表示该模型需要重写消息 0。适配器把它暴露在 `LlmResolvedModelInfo` 上，`prepareCall()` 在 `context.contextWindow` 旁边返回它，因此循环从它已经消费的同一份注册绑定元数据中读取。在该模型发布之前，没有默认目录条目声明它；部署方通过 `cordis.yml` 的 `models` 列表启用。没有该字段的模型——包括当前所有默认条目和所有 `dsh-llm-pi-ai` 路由——完全保持替换第 0 号节点的行为。 + +### 决策规则 + +`SystemPromptProjection` 跟踪**有效提示词**：surface 上最新存活的 `system/message` 的文本（不存在更后的系统节点时即第 0 号节点）。当渲染后的提示词与有效提示词不同时： + +| 路由能力 | 前缀状态 | 操作 | +|---|---|---| +| 无 | 任意 | 替换第 0 号节点 | +| `in-history` | 当前请求序列延续（上次请求以来没有压缩，tools 或 config 没有变更） | 在该步骤的 `user/message` 事件之前追加新的 `system/message` | +| `in-history` | 新序列开始（压缩替换了 surface，或 `request/header` 记录了 tools 或 config 的 `change`）且没有历史中途的系统节点存活 | 用当前提示词替换第 0 号节点 | +| `in-history` | 新序列开始但有历史中途的系统节点存活 | 追加新的 `system/message`；第 0 号节点保持原样 | + +第三行存在，是因为序列开始已经付出了缓存代价；把提示词折回第 0 号节点能让历史保持简短。第四行存在，是因为 surface 没有删除操作：在更后的系统节点仍存活时替换第 0 号节点，会让模型把更后、已过时的节点当作权威，所以循环改为追加。历史内模式在任何更后的系统节点存活期间永不重写第 0 号节点。 + +恢复遵循会话中途的规则。新的循环实例从日志恢复有效提示词，当新渲染的提示词不同时执行追加——提供方缓存在进程边界之后可能仍是热的，且 `resume` header 不是序列开始。 + +### 呈现与记账 + +历史中途的 `system/message` 使用与第 0 号节点相同的折叠请求提示词检视卡片，在请求中的对应位置标注为提示词更新；它永不作为聊天气泡，转录投影跳过它，SDK 投影把它暴露为带类型的事件。`dsh-token-meter` 像对待其他任何 surface 节点一样为它计价，因此逐步骤的上下文明细会显示被保留的各个提示词版本累计的开销，直到压缩遮蔽它们。随后 `assistant/message` 用量上的 `cacheReadTokens` 是可观察的效果：对具备能力的路由，该值覆盖到最后一条已缓存消息为止的前缀；对不具备能力的路由，它回落到公共前缀检测的下限。 + +### 验证计划 + +- `dsh-agent-loop` 中针对投影的单元测试：序列中途变更时追加、没有存活的历史中途节点时在序列开始处替换、有存活节点时在序列开始处追加、恢复时追加、未变更时无操作，以及不具备能力的路由只做替换。 +- `dsh-llm-deepseek` 中针对目录校验（`systemPromptUpdate` 只接受 `'in-history'`）和 `prepareCall()` 暴露该字段的单元测试。 +- `snapshots/` 下的一个无密钥录制快照，其组合在 mock 路由上声明该能力并在会话中途切换 plan 模式，钉住追加的 `system/message` 与未被触及的第 0 号节点；TypeScript 与 Python SDK 的期望输出包含追加的事件。 +- 一个真实 API 的 e2e：针对具备能力的路由运行两个步骤并夹带一次提示词变更，断言第二个请求的 `cacheReadTokens` 不小于第一个请求的提示词 token 数。它通过标准的凭据与 base-URL 机制解析路由，未配置具备能力的路由时自动跳过。 + +## Alternatives considered + +**只发送变化的片段作为增量。** 模型把最新的系统消息当作完整提示词，因此增量会静默丢掉每个未变化的片段。基于模型约定被否决。 + +**用插件配置而不是模型能力启用历史内模式。** 部署标志可能把不具备能力的模型与追加的系统消息配对，这样的模型最多把它们当作普通历史。该能力属于兑现它的路由；适配器目录已经承载逐模型的容量信息。被否决。 + +**永远追加，从不重新基线化。** 规则单一，但第 0 号节点会在会话整个生命周期内保持过时，压缩之后的每个请求都要携带过时的头部加替换消息。在序列开始处重新基线化不花额外代价，因为缓存在那里已经丢失。被否决。 + +**每次恢复都重新基线化。** 为更简单的恢复路径接受每次进程重启一次缓存未命中。缓存跨重启持续数小时到数天，而日志已经承载恢复所需的一切。被否决。 + +**把系统消息放在该步骤的用户消息之后。** 两个位置都在已缓存前缀之后，但模型会在读到必须应用指令的输入之后才读到指令；system 在 user 之前与最前位置的顺序一致。被否决。 + +## Acceptance criteria + +- `DeepSeekCatalogModel.systemPromptUpdate` 在加载时校验、通过 `LlmResolvedModelInfo` 暴露、由 `prepareCall()` 返回；拼错的值在加载时失败。 +- 在具备能力的路由上，序列中途的提示词变更在该步骤的 `user/message` 事件之前追加 `system/message`，第 0 号节点不变；在不具备能力的路由上，同样的变更替换第 0 号节点。 +- 在具备能力的路由上，没有存活的历史中途系统节点的序列开始替换第 0 号节点；有存活节点时追加。 +- 渲染后的提示词不同的已恢复循环实例在具备能力的路由上追加。 +- 录制快照与两个 SDK 的期望输出钉住追加的事件；配置了具备能力的路由时 e2e 断言缓存命中不等式，否则跳过。 +- `dsh-llm-deepseek`、`dsh-agent-loop`、`dsh-system-prompt` 的 README 记录该能力、决策规则和 KV Cache 效果；`docs/config-catalog.md` 列出该字段。 + +## Risks + +- 模型约定尚未公开；本 Agent Note 按所提供的内容记录。若发布的模型收窄了约定（例如只在有界窗口内兑现最新的系统消息），决策规则需要序列开始之外的重新基线化触发条件。 +- 被保留的提示词版本在历史中累积，直到压缩遮蔽它们。每个版本在该序列的每个请求上都要付出其 token 开销；提示词在多数步骤都变化的部署，更适合把那个事实移入运行时上下文。 +- 重写或重排系统消息的代理会静默破坏替换语义；e2e 的缓存命中断言是探测器。 diff --git a/.agents/skills/dsh-speed-up-perf/SKILL.md b/.agents/skills/dsh-speed-up-perf/SKILL.md new file mode 100644 index 0000000000..fbd5d262e1 --- /dev/null +++ b/.agents/skills/dsh-speed-up-perf/SKILL.md @@ -0,0 +1,92 @@ +--- +name: dsh-speed-up-perf +description: 'Use when investigating or optimizing DeepSeek Harness performance, designing realistic synthetic benchmarks or CI performance gates, profiling long Sessions or Web responsiveness, or turning performance PR evidence into measured behavior-preserving fixes.' +--- + +# Speed Up DeepSeek Harness + +Turn a broad “make it faster” request into reproducible user-path measurements and small, evidence-backed fixes. This is guidance, not a quota or a script: survey broadly, follow measured cost, and reject attractive changes that do not improve the workload users actually run. + +## Establish scope and current authority + +Read [AGENTS.md](../../../AGENTS.md), [architecture](../../../docs/architecture.md), [testing policy](../../../docs/testing.md), [defensive patterns](../../../docs/defensive-patterns.md), and the affected packages’ instructions and Agent Notes. Use [CI test reliability](../dsh-ci-test-reliability/SKILL.md) for processes, clocks, browser tests, and asynchronous cleanup. + +Agree on the user-visible endpoint, workload range, resource constraints, acceptable minor behavior differences, and stopping rule. Keep backend and browser end-to-end measurements separate: a fast history iterator or Client fold does not prove fast transport, paint, scrolling, or input response. Exclude model/network latency when measuring local overhead, and state that exclusion rather than calling the result complete product latency. + +Inspect the exact current base, not just the running checkout. Study final merged diffs, owning source, tests, and resolved review threads; a PR body can describe an abandoned implementation. Separate merged, closed-unmerged, superseded, estimated, and newly measured evidence. The [performance workflow decision and evidence](../../notes/implemented/process/2026-09-06-evidence-driven-performance-skill.md) supply historical leads, not authority to reintroduce their implementations. + +## Survey user paths, then rank candidates + +Delegate independent domains when breadth helps; require measurements and production call sites, not guesses. Useful domains include: + +- Cold profile startup, first historical read, current-generation reopen, and writable resume. +- Many-turn and tool-heavy history, large individual messages/results, child Session listing, and repeated navigation among Sessions. +- Initial history transport and fold, first usable browser paint, older-page loading, scrolling, tool expansion, and inactive-view activation. +- Live streaming and reconnect, including a long active attempt, interleaved tool work, settlement, cancellation, and teardown. + +Vary independent cost drivers: bytes, durable events, compact records, raw deltas, turns, tools, children, and visible DOM nodes are different quantities. Do not call a large count of tiny identical messages “realistic” without checking which user operation it stresses. Include typical and tail workloads, but avoid a combinatorial matrix with no decision value. + +Rank candidates by observed user latency, CPU/allocations, retained memory, occurrence, and confidence. For each, name the production consumer, the repeated work, the expected complexity, the smallest falsifiable intervention, and the behavior that must remain stable. A suspicious loop, unused cache, or large file alone is not evidence of a bottleneck. + +## Build realistic synthetic benchmarks first + +Follow [benchmarks/AGENTS.md](../../../benchmarks/AGENTS.md) and the [performance-gate decision](../../notes/implemented/testing/2026-09-04-session-open-performance-gate.md). Extend the existing required lane rather than creating competing calibration or reporting infrastructure. Package-local diagnostics remain beside their owner; cross-package required cases live under the measured user path in `benchmarks/`. + +If the user authorizes local corpus inspection, extract only aggregate workload characteristics. Never copy prompts, outputs, paths, identities, IDs, credentials, recordings, or recognizable snippets into fixtures, logs, screenshots, PRs, or artifacts. Generate fixed inputs from reviewed constants; no benchmark depends on the user’s home, ambient repository, network service, or private data. + +Before implementation, record a measurement card: + +| Field | Required decision | +|---|---| +| User operation | Exact action and externally observable completion condition | +| Workload | Fixed dimensions, distributions, construction seed/constants, and why they exercise ordinary and tail use | +| Entry path | Production calls/composition and built artifacts; mocked external boundaries | +| Clock | Included setup, cold/warm state, timing start/end, and excluded costs | +| Memory | Reachable endpoint objects, baseline, GC policy, retained versus transient limits | +| Verdict | Raw samples, chosen aggregate, calibrated absolute/ratio/memory limits, and negative control | +| Behavior | Owning functional tests/snapshots and permitted minor differences | + +Measure built JavaScript under plain Node for CPU workers; source-loader overhead and module resolution are not the shipped path. Browser cases use built product assets and the supported `dsh` profile through the existing test harness. Do not add a production export solely for measurement or copy the algorithm into a “benchmark implementation.” + +Use fresh children and private temporary roots for cold/process-memory samples. Warm samples explicitly retain the intended cache; never let fixture setup secretly warm a cold scenario. Keep the same input, validations, completion condition, and reachable output on both sides. A parse-and-discard baseline is not comparable with validated retained history. + +Report all samples and the aggregate that decides the result. For the Node lane, use the existing shared time calibration and reviewed variance headroom; do not scale bytes, counts, or dimensionless ratios by CPU speed. Keep manual browser diagnostics threshold-free. A required browser performance case needs an explicit lane decision and repeated measurements on its actual CI browser/runner before adopting timing budgets; the Node machine multiplier alone is not browser calibration. Budgets are source constants, not environment overrides. Serialize measured work against other owned CPU-heavy jobs; measure reference and candidate under comparable conditions. Do not widen a budget or select a lucky run to hide a regression. + +Measure end-to-end latency independently from component phases. Track retained memory with intended objects still reachable, and transient pressure separately through constrained-heap completion or an appropriate peak measurement. Faster execution with unbounded retention is not an automatic win. + +For browser responsiveness, use real browser input and observe the resulting UI update. Include the final stall in frame/input measurements, distinguish scheduled timers from actual input, and bound synthetic producers so catch-up bursts do not invent a different workload. State whether first paint, scrolling, paging, live updates, and activated-but-hidden views are covered. Node folds, fake DOMs, and custom heartbeat events alone cannot establish browser responsiveness. + +## Prove the regression, then remove work + +Run the unoptimized workload before changing production code. Save the command, revision, runtime/platform, fixture dimensions, raw measurements, and verdict. Reduce a failing scenario until it still exercises the real bottleneck, then rank falsifiable hypotheses before patching. Use profiles, allocation samples, work counts, or phase timings to distinguish them. + +Common patterns worth testing, not automatic prescriptions: + +- Keep compact representations compact through downstream readers; avoid per-delta objects when the consumer needs settled content or one aggregate. +- Remove duplicate parsing, copying, freezing, and validation only after identifying the actual ownership and trust transition. Typed same-process borrowing is not permission to weaken durable or wire parsing. +- Stream artifact transformations and bound intermediate state rather than retaining every generation. Include publication, verification, and writable-readiness obligations where the user operation requires them. +- Separate read-only preparation from write/publication work without moving awaited work past a correctness-required endpoint. +- Defer inactive-view and collapsed-detail work; measure first activation and retained state too. Deferral is not deletion, and viewport highlighting is not full virtualization. +- Stabilize identities and narrow subscriptions so one changed node does not invalidate an entire history; preserve update ordering and immediate-event behavior. +- Prefer a suitable data structure to repeated shifting, scanning, or rebuilding. Measure the whole consumer path, not just the isolated container operation. +- Use revision-keyed reuse or singleflight only with explicit invalidation, bounded retention, independent waiter cancellation, and disposal ownership. Avoid caching expanded representations merely to make repeated benchmarks look fast. + +Change one causal factor at a time. Re-run both the focused scenario and its end-to-end parent. Require a negative control: the tightened assertion fails on the original implementation or a controlled reintroduction of the targeted cost. A threshold so generous that the regression passes is not protection; a budget below a verified noise floor is not reliable either. + +## Preserve behavior and resource ownership + +Performance measurements complement functional evidence; they do not replace it. Run or add the narrow owning tests for output, ordering, paging, stream indexes, errors, cancellation, concurrency, and disposal as applicable. Preserve model-visible/logged equivalence, released-generation immutability, atomic publication, required validation, and writable readiness. Do not silently truncate history, skip tool results, disable invariants, or change lifecycle semantics to reach a number. + +State any deliberate minor visible difference and verify it through the owning keyless snapshot. For a product-visible GUI change, include the required browser evidence/GIF. Keep functional expectations independent of benchmark internals; benchmark assertions need enough evidence to reach the real endpoint, not a second semantic test suite. + +Reject an optimization when gains disappear end-to-end, a typical workload regresses materially, complexity outweighs a small gain, or cancellation/retention/durability cannot be explained and tested. Record the rejected hypothesis briefly instead of expanding scope to justify it. + +## Deliver a bounded, reviewable result + +Use [Agent Note rules](../../notes/README.md) for durable rationale, alternatives, calibration, exclusions, and remaining risks. Check relevant notes for supersession without turning performance work into a corpus-wide prose cleanup. Keep the reusable procedure here and scenario-specific truth with its benchmark or package owner. + +When the task requests stacked PRs, choose layers before editing and use official GitHub stacks and separate worktrees. Keep each layer mergeable: benchmark infrastructure can protect the measured baseline; the optimization layer carries its fix, functional coverage, and tighter budget. Independent bottlenecks may use separate stacks. Fix a finding in its owning layer before propagating upward. + +Apply [pre-push checks](../dsh-pre-push-checks/SKILL.md), report only executed evidence, and inspect CI rather than assuming local timing proves runner stability. After marking ready, evaluate review findings against code and executable evidence; reply with the reason or fix and resolve addressed threads. Do not dismiss a report merely because it came from a bot. + +Summarize each result as: workload → before/after absolute values and ratio → endpoint and memory semantics → behavior evidence → negative control → exact checks → exclusions. Separate author-reported historical numbers, fresh local measurements, and CI evidence. Stop at the agreed scenario/fix scope; retain a short ranked follow-up list instead of chasing unrelated opportunities. diff --git a/.github/preview-sizing/README.i18n.yaml b/.github/preview-sizing/README.i18n.yaml new file mode 100644 index 0000000000..cd5241c300 --- /dev/null +++ b/.github/preview-sizing/README.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write .github/preview-sizing/README.md +README.md: 142854e582523eb1ed48924a73fbe29ea6164bf6 +README.zh.md: a71325189dc7966df87e14bbb591af3c76a3d498 diff --git a/.github/preview-sizing/README.md b/.github/preview-sizing/README.md new file mode 100644 index 0000000000..142854e582 --- /dev/null +++ b/.github/preview-sizing/README.md @@ -0,0 +1,35 @@ +# PR preview runner sizing + +English | [中文](README.zh.md) + +## Summary + +The [preview workflow](../workflows/build-preview-cloudflare.yml) builds pull-request previews on standard GitHub-hosted `ubuntu-24.04`. Runner sizing compares complete job cost, not price per minute or core count alone. + +## Table of Contents + +- [Comparison requirements](#comparison-requirements) +- [Publication semantics](#publication-semantics) +- [Dev Note](#dev-note) + + + +## Comparison requirements + +A sizing experiment holds checkout SHA, lockfile, Node and pnpm versions, workspace build, and preview/VFS packing commands constant. Each runner starts without build outputs. Cold installs do not restore dependency caches; pnpm bootstrap files may already exist. Warm installs restore the same exact cache without prefix fallback. Record the actual runner image, CPU, RAM, disk, cache outcome, phase duration, exit status, and peak memory. GNU time maximum RSS reports a process maximum, not simultaneous aggregate memory across the build process tree. + +Calculate estimated gross compute as the sum of each completed job’s elapsed minutes rounded upward, multiplied by that runner’s rate. Include setup, cache restoration, cleanup, failures, and measurement-upload overhead. Report seed jobs separately. Queue delay is a latency observation, not executed job time. These estimates are not invoice totals; standard-runner included minutes and storage are separate. + +A build-only benchmark does not deploy, access Cloudflare credentials, or post pull-request comments. Its cost does not establish complete preview publication cost. Confirm the selected runner through the actual preview workflow before treating deployment latency and protected-image delivery as verified. + + + +## Publication semantics + +Runner selection does not alter pull-request events, per-PR cancellation, immutable installation, restore-only dependency caching, full workspace build, preview packing, sourcemap removal, or the preview page copied to the deployment root. Cloudflare uploads only the built site to the PR branch alias. The protected-image check requires HTTP 200, no transport content encoding, and gzip magic bytes; the URL comment remains idempotent. Dependabot and other PR authors remain on GitHub-hosted machines. + + + +## Dev Note + +The [runner decision](../../.agents/notes/implemented/process/2026-09-06-preview-hosted-runner-sizing.md) records measurements, cost estimates, and image/CPU variation. The build-only experiment does not verify production deployment. diff --git a/.github/preview-sizing/README.zh.md b/.github/preview-sizing/README.zh.md new file mode 100644 index 0000000000..a71325189d --- /dev/null +++ b/.github/preview-sizing/README.zh.md @@ -0,0 +1,35 @@ +# PR 预览运行器规格 + +[English](README.md) | 中文 + +## 摘要 + +[预览工作流](../workflows/build-preview-cloudflare.yml) 在标准 GitHub 托管 `ubuntu-24.04` 上构建 PR（Pull Request）预览。运行器规格选择比较完整作业成本，而非仅比较每分钟价格或核心数。 + +## 目录 + +- [比较要求](#comparison-requirements) +- [发布语义](#publication-semantics) +- [开发备注](#dev-note) + + + +## 比较要求 + +规格实验保持检出 SHA、锁文件、Node 与 pnpm 版本、工作区构建以及预览/VFS 打包命令一致。每个运行器启动时均无构建产物。冷安装不恢复依赖缓存，但 pnpm 引导安装文件可能已存在；热安装恢复同一个精确缓存，不使用前缀回退。记录实际运行器镜像、CPU、内存、磁盘、缓存结果、各阶段耗时、退出状态与内存峰值。GNU time 最大 RSS 表示进程最大值，而非构建进程树同时占用的内存总量。 + +估算总计算费用时，将每个已完成作业的运行分钟数向上取整，乘以对应运行器费率后求和。纳入设置、缓存恢复、清理、失败及测量数据上传的开销。单独报告缓存预热作业。排队延迟属于延迟观测，不属于作业执行时间。这些估算不是账单总额；标准运行器的套餐内分钟数及存储另行计算。 + +仅构建的基准测试不部署、不访问 Cloudflare 凭据，也不发布 PR 评论。其成本不能证明完整预览发布成本。在将部署延迟与受保护镜像交付视为已验证之前，须通过实际预览工作流确认所选运行器。 + + + +## 发布语义 + +运行器选择不改变 PR 事件、按 PR 取消、不可变安装、只恢复的依赖缓存、完整工作区构建、预览打包、sourcemap 删除，以及复制到部署根目录的预览页面。Cloudflare 仅将构建站点上传至 PR 分支别名。受保护镜像检查要求 HTTP 200、无传输内容编码及 gzip 魔数字节；URL 评论保持幂等。Dependabot 与其他 PR 作者仍使用 GitHub 托管机器。 + + + +## 开发备注 + +[运行器决策](../../.agents/notes/implemented/process/2026-09-06-preview-hosted-runner-sizing.zh.md) 记录测量、成本估算及镜像/CPU 差异。仅构建实验不验证生产部署。 diff --git a/.github/workflows/build-preview-cloudflare.yml b/.github/workflows/build-preview-cloudflare.yml index 5f67b97893..80caebdb76 100644 --- a/.github/workflows/build-preview-cloudflare.yml +++ b/.github/workflows/build-preview-cloudflare.yml @@ -32,7 +32,7 @@ env: jobs: preview: - runs-on: dsh-ubuntu-24-04-16core + runs-on: ubuntu-24.04 name: cloudflare pages preview steps: - uses: actions/checkout@v6 diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index d21df3c02a..c55aa4e58f 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -158,15 +158,11 @@ jobs: node-24-bench: if: github.event_name == 'pull_request' - runs-on: >- - ${{ vars.DSH_CI_FAILOVER_LINUX == 'selfhosted' - && github.event.pull_request.user.login != 'dependabot[bot]' - && fromJSON('["self-hosted", "linux", "x64", "vm-backup"]') - || 'dsh-ubuntu-24-04-16core' }} + runs-on: ubuntu-24.04 name: node 24 / benchmarks # Wall-clock budgets need an otherwise idle runner, so this job runs the # benchmark lane alone instead of joining a concurrent gate aggregate. - timeout-minutes: 30 + timeout-minutes: 15 steps: - uses: actions/checkout@v6 with: @@ -189,7 +185,6 @@ jobs: echo "path=$store_path" >> "$GITHUB_OUTPUT" - uses: actions/cache/restore@v4 - if: vars.DSH_CI_FAILOVER_LINUX != 'selfhosted' || github.event.pull_request.user.login == 'dependabot[bot]' with: path: ${{ steps.pnpm-store.outputs.path }} key: ${{ runner.os }}-node-${{ env.PRIMARY_NODE_VERSION }}-pnpm-${{ hashFiles('pnpm-lock.yaml') }} @@ -200,6 +195,8 @@ jobs: run: pnpm install --frozen-lockfile - name: Run performance benchmarks + env: + DSH_GATE_VERBOSE: '1' run: pnpm run check:ci:bench node-24-consumers: diff --git a/.github/workflows/release-vendor.yml b/.github/workflows/release-vendor.yml index 8e1d531a03..ddf3c3f96c 100644 --- a/.github/workflows/release-vendor.yml +++ b/.github/workflows/release-vendor.yml @@ -29,21 +29,40 @@ env: jobs: pack: name: Pack npm tarballs - runs-on: ubuntu-24.04 + # Persistent runners accept only trusted, credential-free rehearsals. + runs-on: >- + ${{ vars.DSH_CI_FAILOVER_LINUX == 'selfhosted' + && github.repository == 'deepseek-harness/deepseek-harness' + && github.actor != 'dependabot[bot]' + && ((github.event_name == 'push' && github.ref == 'refs/heads/master') + || (github.event_name == 'pull_request' + && github.event.pull_request.head.repo.full_name == github.repository + && github.event.pull_request.head.repo.fork == false + && github.event.pull_request.user.login != 'dependabot[bot]')) + && fromJSON('["self-hosted", "linux", "x64", "vm-backup"]') + || 'ubuntu-24.04' }} steps: # Complete history: the release scripts read tags. - uses: actions/checkout@v6 with: fetch-depth: 0 persist-credentials: false + clean: true + + - name: Configure runner-private caches + run: | + echo "NODE_COMPILE_CACHE=${{ runner.temp }}/node-compile-cache" >> "$GITHUB_ENV" + echo "npm_config_devdir=${{ runner.temp }}/node-gyp" >> "$GITHUB_ENV" + echo "TMPDIR=${{ runner.temp }}" >> "$GITHUB_ENV" - uses: pnpm/action-setup@v4 with: - dest: ${{ runner.temp }}/setup-pnpm + dest: ${{ runner.temp }}/setup-pnpm-${{ github.run_id }}-${{ github.run_attempt }}-${{ github.job }} - uses: actions/setup-node@v6 with: node-version: ${{ env.PRIMARY_NODE_VERSION }} + package-manager-cache: false - name: Configure pnpm store path id: pnpm-store @@ -54,6 +73,7 @@ jobs: echo "path=$store_path" >> "$GITHUB_OUTPUT" - uses: actions/cache/restore@v4 + if: runner.environment == 'github-hosted' with: path: ${{ steps.pnpm-store.outputs.path }} key: ${{ runner.os }}-node-${{ env.PRIMARY_NODE_VERSION }}-pnpm-${{ hashFiles('pnpm-lock.yaml') }} diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index ad6c64ae67..1e6662149e 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -28,19 +28,38 @@ env: jobs: dependencies: name: Dependency layout - runs-on: ubuntu-24.04 + # Persistent runners accept only trusted, credential-free rehearsals. + runs-on: >- + ${{ vars.DSH_CI_FAILOVER_LINUX == 'selfhosted' + && github.repository == 'deepseek-harness/deepseek-harness' + && github.actor != 'dependabot[bot]' + && ((github.event_name == 'push' && github.ref == 'refs/heads/master') + || (github.event_name == 'pull_request' + && github.event.pull_request.head.repo.full_name == github.repository + && github.event.pull_request.head.repo.fork == false + && github.event.pull_request.user.login != 'dependabot[bot]')) + && fromJSON('["self-hosted", "linux", "x64", "vm-backup"]') + || 'ubuntu-24.04' }} steps: - uses: actions/checkout@v6 with: persist-credentials: false + clean: true + + - name: Configure runner-private caches + run: | + echo "NODE_COMPILE_CACHE=${{ runner.temp }}/node-compile-cache" >> "$GITHUB_ENV" + echo "npm_config_devdir=${{ runner.temp }}/node-gyp" >> "$GITHUB_ENV" + echo "TMPDIR=${{ runner.temp }}" >> "$GITHUB_ENV" - uses: pnpm/action-setup@v4 with: - dest: ${{ runner.temp }}/setup-pnpm + dest: ${{ runner.temp }}/setup-pnpm-${{ github.run_id }}-${{ github.run_attempt }}-${{ github.job }} - uses: actions/setup-node@v6 with: node-version: ${{ env.PRIMARY_NODE_VERSION }} + package-manager-cache: false - name: Configure pnpm store path id: pnpm-store @@ -51,6 +70,7 @@ jobs: echo "path=$store_path" >> "$GITHUB_OUTPUT" - uses: actions/cache/restore@v4 + if: runner.environment == 'github-hosted' with: path: ${{ steps.pnpm-store.outputs.path }} key: ${{ runner.os }}-node-${{ env.PRIMARY_NODE_VERSION }}-pnpm-${{ hashFiles('pnpm-lock.yaml') }} @@ -68,21 +88,40 @@ jobs: pack: name: Pack npm tarballs - runs-on: ubuntu-24.04 + # Persistent runners accept only trusted, credential-free rehearsals. + runs-on: >- + ${{ vars.DSH_CI_FAILOVER_LINUX == 'selfhosted' + && github.repository == 'deepseek-harness/deepseek-harness' + && github.actor != 'dependabot[bot]' + && ((github.event_name == 'push' && github.ref == 'refs/heads/master') + || (github.event_name == 'pull_request' + && github.event.pull_request.head.repo.full_name == github.repository + && github.event.pull_request.head.repo.fork == false + && github.event.pull_request.user.login != 'dependabot[bot]')) + && fromJSON('["self-hosted", "linux", "x64", "vm-backup"]') + || 'ubuntu-24.04' }} steps: # Complete history: the release scripts read tags. - uses: actions/checkout@v6 with: fetch-depth: 0 persist-credentials: false + clean: true + + - name: Configure runner-private caches + run: | + echo "NODE_COMPILE_CACHE=${{ runner.temp }}/node-compile-cache" >> "$GITHUB_ENV" + echo "npm_config_devdir=${{ runner.temp }}/node-gyp" >> "$GITHUB_ENV" + echo "TMPDIR=${{ runner.temp }}" >> "$GITHUB_ENV" - uses: pnpm/action-setup@v4 with: - dest: ${{ runner.temp }}/setup-pnpm + dest: ${{ runner.temp }}/setup-pnpm-${{ github.run_id }}-${{ github.run_attempt }}-${{ github.job }} - uses: actions/setup-node@v6 with: node-version: ${{ env.PRIMARY_NODE_VERSION }} + package-manager-cache: false - name: Configure pnpm store path id: pnpm-store @@ -93,6 +132,7 @@ jobs: echo "path=$store_path" >> "$GITHUB_OUTPUT" - uses: actions/cache/restore@v4 + if: runner.environment == 'github-hosted' with: path: ${{ steps.pnpm-store.outputs.path }} key: ${{ runner.os }}-node-${{ env.PRIMARY_NODE_VERSION }}-pnpm-${{ hashFiles('pnpm-lock.yaml') }} diff --git a/benchmarks/session-open/session-open.bench.ts b/benchmarks/session-open/session-open.bench.ts index 91dde3529d..5a68eb1445 100644 --- a/benchmarks/session-open/session-open.bench.ts +++ b/benchmarks/session-open/session-open.bench.ts @@ -45,7 +45,6 @@ const SOURCE_GENERATION_BY_ACCESS = { /** Expected durations on the reference machine before CI scaling and variance headroom. */ const EXPECTED_MS = { migrationOpen: 220, - reopenOpen: 12, read: 8, sessionRestore: 24, projection: 14, @@ -56,7 +55,9 @@ const EXPECTED_MS = { } as const const MIGRATION_OPEN_BUDGET_MS = ciTimeBudget(EXPECTED_MS.migrationOpen) -const REOPEN_OPEN_BUDGET_MS = ciTimeBudget(EXPECTED_MS.reopenOpen) +/** Standard two-CPU CI reopen samples span 47.4–49.2 ms; 50 ms is the rounded expectation. */ +const EXPECTED_REOPEN_CI_MS = 50 +const REOPEN_OPEN_BUDGET_MS = Math.ceil(EXPECTED_REOPEN_CI_MS * PERFORMANCE_BUDGET_HEADROOM) const READ_BUDGET_MS = ciTimeBudget(EXPECTED_MS.read) const SESSION_RESTORE_BUDGET_MS = ciTimeBudget(EXPECTED_MS.sessionRestore) const PROJECTION_BUDGET_MS = ciTimeBudget(EXPECTED_MS.projection) @@ -270,6 +271,28 @@ const ACCESS_BENCHMARKS: readonly AccessBenchmarkSpec[] = [ }, ] +function expectOpenWithinBudget(value: number, budget: number): void { + expect(value).toBeLessThanOrEqual(budget) +} + +describe('standard hosted reopen calibration', () => { + it('accepts the recorded two-CPU samples that exceed the historical budget', () => { + const recordedMedian = median([49.2, 47.4, 49.1, 48.6, 48.1]) + + expect(recordedMedian).toBe(48.6) + expect(() => expectOpenWithinBudget(recordedMedian, ciTimeBudget(12))).toThrow() + expectOpenWithinBudget(recordedMedian, REOPEN_OPEN_BUDGET_MS) + expect(REOPEN_OPEN_BUDGET_MS).toBe(63) + }) + + it('rejects synthetic reopen and multi-second first-open regressions', () => { + const regressionMedian = median([74, 75, 76, 75, 74]) + expect(() => expectOpenWithinBudget(regressionMedian, REOPEN_OPEN_BUDGET_MS)).toThrow() + expect(MIGRATION_OPEN_BUDGET_MS).toBe(550) + expect(() => expectOpenWithinBudget(4_000, MIGRATION_OPEN_BUDGET_MS)).toThrow() + }) +}) + describe('opening a large Session for first open and post-upgrade reopen', () => { const suite = new SessionOpenBenchmarkSuite() @@ -291,7 +314,7 @@ describe('opening a large Session for first open and post-upgrade reopen', () => projection: PROJECTION_BUDGET_MS, }, })) - expect(result.openMs.median).toBeLessThanOrEqual(access.openBudgetMs) + expectOpenWithinBudget(result.openMs.median, access.openBudgetMs) expect(result.readMs.median).toBeLessThanOrEqual(READ_BUDGET_MS) expect(result.sessionRestoreMs.median).toBeLessThanOrEqual(SESSION_RESTORE_BUDGET_MS) expect(result.projectionMs.median).toBeLessThanOrEqual(PROJECTION_BUDGET_MS) diff --git a/docs/development.i18n.yaml b/docs/development.i18n.yaml index 14b86dfa27..71b047fa7a 100644 --- a/docs/development.i18n.yaml +++ b/docs/development.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write docs/development.md -development.md: 0f9e76f30e6fc65f242b7f4ce6e1bef79e3451b9 -development.zh.md: 6e98acf9d9cda5271b98c3afbf4b915673fde4d0 +development.md: 37028720ff2487a81caecb8b2e2c6e5bc26b2df5 +development.zh.md: 01e3e88605cb4f9ea0b14df4a0099f436c48c5e9 diff --git a/docs/development.md b/docs/development.md index 0f9e76f30e..37028720ff 100644 --- a/docs/development.md +++ b/docs/development.md @@ -120,7 +120,9 @@ Contributors can opt into the comprehensive local gate set with `pnpm run check: ### CI gates -The keyless [CI workflow](../.github/workflows/ci.yml) groups independent gates into broad lanes and runs a smaller compatibility signal across supported Node versions. Artifact consumers wait for one build within their lane. The separate real-API workflow runs `pnpm run test:e2e` with its configured worker bound. See [scripts/run-gates.ts](../scripts/run-gates.ts) and the workflow files for the current gate and job inventory. +The keyless [CI workflow](../.github/workflows/ci.yml) groups independent gates into broad lanes and runs a smaller compatibility signal across supported Node versions. Artifact consumers wait for one build within their lane. Required benchmarks run separately on standard GitHub-hosted Linux; the [benchmark runner decision](../.agents/notes/implemented/testing/2026-09-06-standard-hosted-benchmark-runner.md) owns routing and the job timeout. The separate real-API workflow runs `pnpm run test:e2e` with its configured worker bound. See [scripts/run-gates.ts](../scripts/run-gates.ts) and the workflow files for the current gate and job inventory. + +The credential-free dsh dependency-layout and dsh/vendor pack rehearsals use the existing Linux self-hosted pool only when `DSH_CI_FAILOVER_LINUX=selfhosted` and the event is a trusted master push or same-repository, non-fork, non-Dependabot pull request. All other cases, including manual dispatch, use `ubuntu-24.04`; manual publication stays hosted. See the [release rehearsal runner decision](../.agents/notes/implemented/process/2026-09-06-release-rehearsal-selfhosted.md) for persistent-store isolation and fallback limits. ### Daily commands diff --git a/docs/development.zh.md b/docs/development.zh.md index 6e98acf9d9..01e3e88605 100644 --- a/docs/development.zh.md +++ b/docs/development.zh.md @@ -124,7 +124,9 @@ vendor manifest 守卫检查 `vendor/*/src` 下的改动是否连同对应的 `v ### CI 门禁 -keyless [CI 工作流](../.github/workflows/ci.yml) 将独立门禁分组到若干宽粒度 lane，并在受支持的 Node 版本上运行一组较小的兼容性检查。产物消费方在各自 lane 内等待一次 build。单独的真实 API 工作流按其配置的 worker 上限运行 `pnpm run test:e2e`。当前门禁和 job 清单以 [scripts/run-gates.ts](../scripts/run-gates.ts) 和工作流文件为准。 +keyless [CI 工作流](../.github/workflows/ci.yml) 将独立门禁分组到若干宽粒度 lane，并在受支持的 Node 版本上运行一组较小的兼容性检查。产物消费方在各自 lane 内等待一次 build。必需 benchmark 在标准 GitHub 托管 Linux 上独立运行；[benchmark 运行器决策](../.agents/notes/implemented/testing/2026-09-06-standard-hosted-benchmark-runner.zh.md)拥有路由及 job 超时。单独的真实 API 工作流按其配置的 worker 上限运行 `pnpm run test:e2e`。当前门禁和 job 清单以 [scripts/run-gates.ts](../scripts/run-gates.ts) 和工作流文件为准。 + +不带凭据的 dsh 依赖布局检查与 dsh/vendor 打包演练仅在 `DSH_CI_FAILOVER_LINUX=selfhosted`，且事件为受信任的 master 推送或同仓库、非 fork、非 Dependabot 拉取请求时使用现有 Linux 自托管池。其余情况（包括手动触发）均使用 `ubuntu-24.04`；手动发布仍使用托管运行器。持久化存储隔离与回退限制见[发布演练运行器决策](../.agents/notes/implemented/process/2026-09-06-release-rehearsal-selfhosted.zh.md)。 ### 日常命令 diff --git a/packages/subagent/subagent/src/continuation.ts b/packages/subagent/subagent/src/continuation.ts index b50903fa70..c5b8782e28 100644 --- a/packages/subagent/subagent/src/continuation.ts +++ b/packages/subagent/subagent/src/continuation.ts @@ -294,7 +294,7 @@ function agentMessageSource(sender: Agent): AgentMessageSource { function agentMessage(sender: Agent, content: ContentBlock[]) { return createUserMessage({ content: [ - { type: 'text' as const, text: `Agent ${sender.id} sent a message:` }, + { type: 'text' as const, text: `Agent ${sender.id} sent a message: ` }, ...content, ], source: agentMessageSource(sender), diff --git a/packages/subagent/subagent/tests/continuation.spec.ts b/packages/subagent/subagent/tests/continuation.spec.ts index 3acaf20af1..68072264e3 100644 --- a/packages/subagent/subagent/tests/continuation.spec.ts +++ b/packages/subagent/subagent/tests/continuation.spec.ts @@ -1954,7 +1954,7 @@ describe('continuable adjacent-Agent delivery', () => { senderSessionId: started.childId, }) expect(delivered?.content).toEqual([ - { type: 'text', text: `Agent ${started.childId} sent a message:` }, + { type: 'text', text: `Agent ${started.childId} sent a message: ` }, { type: 'text', text: 'an explicit message' }, ]) diff --git a/packages/subagent/tool-subagent-control/tests/tool-subagent-control.spec.ts b/packages/subagent/tool-subagent-control/tests/tool-subagent-control.spec.ts index 021d64b6f6..da7fe7a51b 100644 --- a/packages/subagent/tool-subagent-control/tests/tool-subagent-control.spec.ts +++ b/packages/subagent/tool-subagent-control/tests/tool-subagent-control.spec.ts @@ -221,7 +221,7 @@ describe('dsh-tool-subagent-control', () => { senderSessionId: started.childId, }) expect(delivered[0]?.message.content).toEqual([ - { type: 'text', text: `Agent ${started.childId} sent a message:` }, + { type: 'text', text: `Agent ${started.childId} sent a message: ` }, { type: 'text', text: 'CHILD_FINDING' }, ]) @@ -257,7 +257,7 @@ describe('dsh-tool-subagent-control', () => { senderSessionId: parent.id, }) expect(followUp?.type === 'user/message' && followUp.data.content).toEqual([ - { type: 'text', text: `Agent ${parent.id} sent a message:` }, + { type: 'text', text: `Agent ${parent.id} sent a message: ` }, { type: 'text', text: 'and then?' }, ]) }) @@ -288,7 +288,7 @@ describe('dsh-tool-subagent-control', () => { : []) expect(prompts).toEqual([ 'long work', - `Agent ${parent.id} sent a message:`, + `Agent ${parent.id} sent a message: `, 'also consider Y', ]) }) @@ -411,9 +411,9 @@ describe('dsh-tool-subagent-control interrupt_agent', () => { : []) expect(prompts).toEqual([ 'long work', - `Agent ${parent.id} sent a message:`, + `Agent ${parent.id} sent a message: `, 'parked follow-up', - `Agent ${parent.id} sent a message:`, + `Agent ${parent.id} sent a message: `, 'wake up', ]) }) diff --git a/python/sdk-runtime/README.i18n.yaml b/python/sdk-runtime/README.i18n.yaml index 410f250dd9..62e489fa02 100644 --- a/python/sdk-runtime/README.i18n.yaml +++ b/python/sdk-runtime/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write python/sdk-runtime/README.md -README.md: 050ae85d9b0a38b82a3c66a84c3d8f34e137be6c -README.zh.md: 7066d7224752294c25b58cfe8fb6a94524a2013d +README.md: fb7478f305015e60dafd23861ab6fd91f10757f3 +README.zh.md: 7c663aba8d387fb7b1048afe28d50faee7350303 diff --git a/python/sdk-runtime/README.md b/python/sdk-runtime/README.md index 050ae85d9b..fb7478f305 100644 --- a/python/sdk-runtime/README.md +++ b/python/sdk-runtime/README.md @@ -19,7 +19,7 @@ Both carriers execute the same `dsh` grammar and shipped profiles, including the - `bundled_package_dir() -> Path` returns the installed module-data root and verifies its release metadata. - `bundled_runtime_path() -> Path` returns the current platform executable and verifies required sidecars. - `resolve_bundled_launch_args(mode=None) -> tuple[str, ...]` returns the executable argv by default. Explicit `mode="node"` or `DSH_RUNTIME_MODE=node` selects the repo-only Node carrier. -- `main()` implements the installed `dsh` console command and rejects an absent or blank `DSH_HOME` before replacing the Python process. +- `main()` implements the installed `dsh` console command and rejects an absent or blank `DSH_HOME`. On Windows it waits for the bundled process with inherited standard streams and forwards its exit status; on POSIX it replaces the Python process. Unsupported platforms and missing executables or sidecars raise `FileNotFoundError` with the build and installation routes. Unknown runtime modes raise `ValueError`. diff --git a/python/sdk-runtime/README.zh.md b/python/sdk-runtime/README.zh.md index 7066d72247..7c663aba8d 100644 --- a/python/sdk-runtime/README.zh.md +++ b/python/sdk-runtime/README.zh.md @@ -19,7 +19,7 @@ Wheel 会安装 `dsh` 控制台命令和 `deepseek_harness_runtime` Python 模 - `bundled_package_dir() -> Path` 返回已安装模块数据根目录，并校验发布元数据。 - `bundled_runtime_path() -> Path` 返回当前平台可执行程序，并校验必需伴随文件。 - `resolve_bundled_launch_args(mode=None) -> tuple[str, ...]` 默认返回可执行程序 argv。显式 `mode="node"` 或 `DSH_RUNTIME_MODE=node` 会选择仅限仓库使用的 Node 载体。 -- `main()` 实现已安装的 `dsh` 控制台命令，并在替换 Python 进程前拒绝缺失或空白的 `DSH_HOME`。 +- `main()` 实现已安装的 `dsh` 控制台命令，并拒绝缺失或空白的 `DSH_HOME`。在 Windows 上，它让打包进程继承标准流，等待其结束并转发退出状态；在 POSIX 上，它替换 Python 进程。 不支持的平台以及缺失的可执行程序或伴随文件会抛出 `FileNotFoundError`，并指出构建与安装路径。未知运行时模式会抛出 `ValueError`。 diff --git a/python/sdk-runtime/src/deepseek_harness_runtime/__init__.py b/python/sdk-runtime/src/deepseek_harness_runtime/__init__.py index 1fc5377e10..5235ff4d00 100644 --- a/python/sdk-runtime/src/deepseek_harness_runtime/__init__.py +++ b/python/sdk-runtime/src/deepseek_harness_runtime/__init__.py @@ -24,6 +24,7 @@ from __future__ import annotations import os import platform import shutil +import subprocess import sys from pathlib import Path @@ -156,7 +157,7 @@ def _node_launch_args() -> tuple[str, str]: def main() -> None: - """Execute the bundled dsh CLI with an explicitly selected Harness home.""" + """Launch the CLI with explicit DSH_HOME; wait on Windows, replace the process on POSIX.""" if not os.environ.get("DSH_HOME", "").strip(): print( "dsh: the Python runtime command requires an explicit DSH_HOME; " @@ -165,6 +166,9 @@ def main() -> None: ) raise SystemExit(2) argv = (*resolve_bundled_launch_args(), *sys.argv[1:]) + if sys.platform == "win32": + # Windows CRT exec does not replace the process; wait and preserve the runtime status. + raise SystemExit(subprocess.run(argv, env=os.environ).returncode) os.execvpe(argv[0], argv, os.environ) diff --git a/python/sdk/tests/test_runtime_resolution.py b/python/sdk/tests/test_runtime_resolution.py index 3241fd3ce1..dd520df78b 100644 --- a/python/sdk/tests/test_runtime_resolution.py +++ b/python/sdk/tests/test_runtime_resolution.py @@ -2,7 +2,11 @@ from __future__ import annotations +import os +import subprocess +import sys from pathlib import Path +from types import SimpleNamespace import deepseek_harness_runtime as runtime import pytest @@ -129,7 +133,7 @@ def test_python_dsh_command_executes_the_bundled_cli( called: dict[str, object] = {} monkeypatch.setenv("DSH_HOME", "/explicit/home") monkeypatch.setattr(runtime, "resolve_bundled_launch_args", lambda: ("/runtime",)) - monkeypatch.setattr(runtime.sys, "argv", ["dsh", "plugin", "--profile", "sdk", "list"]) + monkeypatch.setattr(runtime, "sys", SimpleNamespace(platform="linux", argv=["dsh", "plugin", "--profile", "sdk", "list"])) def execvpe(file: str, args: tuple[str, ...], env: dict[str, str]) -> None: called.update(file=file, args=args, home=env.get("DSH_HOME")) @@ -143,3 +147,51 @@ def test_python_dsh_command_executes_the_bundled_cli( "args": ("/runtime", "plugin", "--profile", "sdk", "list"), "home": "/explicit/home", } + + +@pytest.mark.parametrize("returncode", [0, 37, 513]) +def test_windows_console_waits_and_forwards_runtime_status(monkeypatch: pytest.MonkeyPatch, returncode: int) -> None: + monkeypatch.setenv("DSH_HOME", "/explicit/home") + monkeypatch.setattr(runtime, "sys", SimpleNamespace(platform="win32", argv=["dsh", "plugin", "argument with spaces", "中文"])) + monkeypatch.setattr(runtime, "resolve_bundled_launch_args", lambda: ("runtime.exe",)) + called = [] + + def run(args: tuple[str, ...], **kwargs: object) -> subprocess.CompletedProcess[str]: + called.append((args, kwargs)) + return subprocess.CompletedProcess(args, returncode) + + def forbidden_exec(*args: object) -> None: + pytest.fail("Windows console must wait instead of entering CRT exec") + + monkeypatch.setattr(subprocess, "run", run) + monkeypatch.setattr(runtime.os, "execvpe", forbidden_exec) + with pytest.raises(SystemExit) as result: + main() + assert result.value.code == returncode + assert called == [(("runtime.exe", "plugin", "argument with spaces", "中文"), {"env": os.environ})] + + +@pytest.mark.parametrize("returncode", [0, 37, pytest.param(513, marks=pytest.mark.skipif(sys.platform != "win32", reason="POSIX truncates process exit codes to eight bits"))]) +def test_windows_console_branch_preserves_real_child_io_and_completion(tmp_path: Path, returncode: int) -> None: + child = tmp_path / "child with spaces.py" + sentinel = tmp_path / "finished" + child.write_text( + "import pathlib,sys\n" + "assert sys.argv[1] == 'argument with spaces'\n" + "assert sys.argv[2] == '中文'\n" + "print('stdout-中文', flush=True)\n" + "print('stderr-中文', file=sys.stderr, flush=True)\n" + f"pathlib.Path({str(sentinel)!r}).write_text('done')\n" + f"raise SystemExit({returncode})\n", encoding="utf-8", + ) + driver = ( + "import deepseek_harness_runtime as runtime; from types import SimpleNamespace; " + f"runtime.sys = SimpleNamespace(platform='win32', argv=['dsh', 'argument with spaces', '中文']); " + f"runtime.resolve_bundled_launch_args = lambda: ({sys.executable!r}, {str(child)!r}); runtime.main()" + ) + result = subprocess.run([sys.executable, "-c", driver], capture_output=True, text=True, encoding="utf-8", + env={**os.environ, "DSH_HOME": str(tmp_path), "PYTHONIOENCODING": "utf-8"}, timeout=15) + assert result.returncode == returncode, result.stderr + assert result.stdout == "stdout-中文\n" + assert result.stderr == "stderr-中文\n" + assert sentinel.read_text() == "done" diff --git a/python/sdk/tests/test_smoke_model.py b/python/sdk/tests/test_smoke_model.py index f3e0bbfe26..4ff22908ed 100644 --- a/python/sdk/tests/test_smoke_model.py +++ b/python/sdk/tests/test_smoke_model.py @@ -2,6 +2,7 @@ from __future__ import annotations import json import runpy +import subprocess from pathlib import Path import pytest @@ -331,3 +332,17 @@ def test_snapshot_generation_filename_must_match_header(tmp_path: Path) -> None: with pytest.raises(AssertionError, match="filename declares Session format v1"): SMOKE["selected_snapshot_session_files"](tmp_path) + + +@pytest.mark.parametrize("returncode", [1, -1073741819, 3221225477]) +def test_profile_plugin_failure_reports_native_exit_status(monkeypatch: pytest.MonkeyPatch, returncode: int) -> None: + def failed_install(*args: object, **kwargs: object) -> subprocess.CompletedProcess[str]: + return subprocess.CompletedProcess(args=[], returncode=returncode, stdout="", stderr="") + + monkeypatch.setattr(subprocess, "run", failed_install) + with pytest.raises(AssertionError) as error: + SMOKE["smoke_sdk_profile_plugin"]("http://127.0.0.1:1") + message = str(error.value) + assert f"returncode={returncode}" in message + assert f"0x{returncode & 0xffffffff:08x}" in message + assert "stdout='' stderr=''" in message diff --git a/scripts/ci-workflow.spec.ts b/scripts/ci-workflow.spec.ts index 421c72b0d5..eb06172527 100644 --- a/scripts/ci-workflow.spec.ts +++ b/scripts/ci-workflow.spec.ts @@ -235,10 +235,10 @@ describe('CI workflow', () => { expect(aggregate.needs).not.toContain('windows-observational') expect(aggregate.needs).not.toContain('serial-windows') - // Linux failover is a separate switch: the four required Linux workers + // Linux failover is a separate switch: the three enterprise Linux workers // and the verdict job resolve their pool through DSH_CI_FAILOVER_LINUX, // never the Windows switch. - for (const [jobName, job] of [['node-24', node24], ['node-24-coverage', node24Coverage], ['node-24-bench', node24Bench], ['node-24-consumers', node24Consumers]] as const) { + for (const [jobName, job] of [['node-24', node24], ['node-24-coverage', node24Coverage], ['node-24-consumers', node24Consumers]] as const) { expect(typeof job['runs-on']).toBe('string') expect(job['runs-on'], `${jobName} runs-on must use the Linux failover switch`).toContain('DSH_CI_FAILOVER_LINUX') expect(job['runs-on'], `${jobName} runs-on must not use the Windows failover switch`).not.toContain('DSH_CI_FAILOVER_WINDOWS') @@ -269,6 +269,45 @@ describe('CI workflow', () => { expect(windowsObservational.env).not.toMatchObject({ DSH_GATE_FAIL_FAST: '1' }) }) + it('runs required benchmarks on standard hosted Linux independently of failover', () => { + const workflow = loadWorkflow('.github/workflows/ci.yml') + const benchmark = workflowJob(workflow, 'node-24-bench') + const aggregate = workflowJob(workflow, 'all-checks-passed') + + expect(benchmark['runs-on']).toBe('ubuntu-24.04') + expect(benchmark.if).toBe("github.event_name == 'pull_request'") + expect(benchmark.needs).toBeUndefined() + expect(benchmark['continue-on-error']).toBeUndefined() + expect(benchmark.env).toBeUndefined() + expect(aggregate.needs).toContain('node-24-bench') + }) + + it('always restores the hosted benchmark pnpm cache', () => { + const benchmark = workflowJob(loadWorkflow('.github/workflows/ci.yml'), 'node-24-bench') + if (!Array.isArray(benchmark.steps)) throw new TypeError('benchmark job must define steps') + const caches = benchmark.steps.filter(step => isRecord(step) && step.uses === 'actions/cache/restore@v4') + + expect(caches).toHaveLength(1) + expect(caches[0]).not.toHaveProperty('if') + expect(caches[0]).toMatchObject({ + with: { + path: '${{ steps.pnpm-store.outputs.path }}', + key: "${{ runner.os }}-node-${{ env.PRIMARY_NODE_VERSION }}-pnpm-${{ hashFiles('pnpm-lock.yaml') }}", + }, + }) + }) + + it('bounds the complete benchmark job to fifteen minutes', () => { + const benchmark = workflowJob(loadWorkflow('.github/workflows/ci.yml'), 'node-24-bench') + + expect(benchmark['timeout-minutes']).toBe(15) + expect(benchmark.steps).toContainEqual({ + name: 'Run performance benchmarks', + env: { DSH_GATE_VERBOSE: '1' }, + run: 'pnpm run check:ci:bench', + }) + }) + it('gives the Wine Host TypeScript compile the repository heap budget', () => { const wineGates = readFileSync(resolve(root, 'scripts/wine-windows-gates.sh'), 'utf8') diff --git a/scripts/preview-workflow.spec.ts b/scripts/preview-workflow.spec.ts new file mode 100644 index 0000000000..f3d14ce484 --- /dev/null +++ b/scripts/preview-workflow.spec.ts @@ -0,0 +1,63 @@ +import { readFileSync } from 'node:fs' +import { resolve } from 'node:path' +import * as yaml from 'js-yaml' +import { describe, expect, it } from 'vitest' + +const workflow = yaml.load(readFileSync(resolve(import.meta.dirname, '../.github/workflows/build-preview-cloudflare.yml'), 'utf8')) as { + on: unknown + permissions: unknown + concurrency: unknown + env: Record + jobs: Record<'preview', { + 'runs-on': string + steps: Array<{ name?: string; uses?: string; run?: string; with?: Record; env?: Record }> + }> +} +const preview = workflow.jobs.preview + +describe('PR preview workflow', () => { + it('keeps every PR author on the selected GitHub-hosted runner', () => { + expect(Object.keys(workflow.jobs)).toEqual(['preview']) + expect(preview['runs-on']).toBe('ubuntu-24.04') + expect(workflow.on).toEqual({ pull_request: { types: ['opened', 'synchronize', 'reopened'] } }) + expect(workflow.permissions).toEqual({ contents: 'read', 'pull-requests': 'write' }) + expect(preview.steps.find(step => step.uses === 'actions/checkout@v6')?.with).toEqual({ 'persist-credentials': false }) + }) + + it('keeps the immutable full build and restore-only dependency cache', () => { + expect(workflow.env.PRIMARY_NODE_VERSION).toBe('24') + expect(workflow.env.DSH_TELEMETRY_DISABLED).toBe('1') + const commands = preview.steps.map(step => step.run) + expect(commands).toContain('pnpm install --frozen-lockfile') + expect(commands).toContain('pnpm run build') + expect(commands).toContain('pnpm --filter @deepseek-ai/dsh-web-frontend run build:preview') + expect(commands.indexOf('pnpm run build')).toBeLessThan(commands.indexOf('pnpm --filter @deepseek-ai/dsh-web-frontend run build:preview')) + expect(preview.steps.filter(step => step.uses?.startsWith('actions/cache'))).toHaveLength(1) + expect(preview.steps.find(step => step.uses === 'actions/cache/restore@v4')?.with).toMatchObject({ + key: "${{ runner.os }}-node-${{ env.PRIMARY_NODE_VERSION }}-pnpm-${{ hashFiles('pnpm-lock.yaml') }}", + }) + }) + + it('retains per-PR deployment, protected image verification, and idempotent URL comments', () => { + expect(workflow.concurrency).toEqual({ + group: 'build-preview-cloudflare-${{ github.event.pull_request.number }}', + 'cancel-in-progress': true, + }) + expect(workflow.env.CF_PROJECT).toBe('dsh-build-preview') + const shape = preview.steps.find(step => step.name === 'Shape the upload')! + expect(shape.run).toContain("find apps/web/dist -name '*.map' -delete") + expect(shape.run).toContain('cp apps/web/dist/preview.html apps/web/dist/index.html') + const deploy = preview.steps.find(step => step.name === 'Upload to Cloudflare Pages')! + expect(deploy.run).toContain('npx --yes wrangler@4 pages deploy apps/web/dist') + expect(deploy.run).toContain('--branch "pr-${{ github.event.pull_request.number }}"') + const verify = preview.steps.find(step => step.name === 'Verify the protected deployment serves the image')! + expect(verify.run).toContain('/preview/vfs-image.tar.gz') + expect(verify.run).toContain('"$code" != "200"') + expect(verify.run).toContain('content-encoding:') + expect(verify.run).toContain('"$magic" != "1f8b"') + expect(verify.env?.CF_ACCESS_CLIENT_SECRET).toBe('${{ secrets.CF_ACCESS_CLIENT_SECRET }}') + const comment = preview.steps.find(step => step.name === 'Comment the preview URL')! + expect(comment.run).toContain('') + expect(comment.run).toContain('gh pr comment "$PR" --body-file -') + }) +}) diff --git a/scripts/smoke-python-runtime.py b/scripts/smoke-python-runtime.py index 5da6d6dc98..e4bae53fcc 100644 --- a/scripts/smoke-python-runtime.py +++ b/scripts/smoke-python-runtime.py @@ -1227,6 +1227,7 @@ def smoke_sdk_profile_plugin(base_url: str) -> None: if installed.returncode != 0: raise AssertionError( f"Python-installed dsh could not add the external profile plugin: " + f"returncode={installed.returncode} (0x{installed.returncode & 0xffffffff:08x}) " f"stdout={installed.stdout!r} stderr={installed.stderr!r}" ) manifest = json.loads((dsh_home / "profiles" / "sdk" / "package.json").read_text()) diff --git a/scripts/tests/ci-release-selfhosted.spec.ts b/scripts/tests/ci-release-selfhosted.spec.ts new file mode 100644 index 0000000000..99ce317248 --- /dev/null +++ b/scripts/tests/ci-release-selfhosted.spec.ts @@ -0,0 +1,150 @@ +/** Release rehearsal routing and persistent-runner isolation, without executing release builds. */ +import { readFileSync } from 'node:fs' +import { resolve } from 'node:path' +import { runInNewContext } from 'node:vm' +import { load } from 'js-yaml' +import { describe, expect, it } from 'vitest' + +const root = resolve(import.meta.dirname, '../..') +const repository = 'deepseek-harness/deepseek-harness' +const selfhosted = ['self-hosted', 'linux', 'x64', 'vm-backup'] +const hosted = 'ubuntu-24.04' + +interface Step { + name?: string + uses?: string + run?: string + if?: string + with?: Record +} +interface Workflow { + on: Record + permissions: Record + concurrency?: Record + jobs: Record +} + +function workflow(file: string): Workflow { + return load(readFileSync(resolve(root, '.github/workflows', file), 'utf8')) as Workflow +} + +// This canonical-case corpus has matching Actions/JavaScript comparison results. +// This is not an Actions interpreter: string case-folding and general coercion differ. +// Missing context properties use the Actions empty-string value. +function evaluate(expression: string, context: Record): unknown { + const source = expression.trim().replace(/^\$\{\{|\}\}$/g, '') + .replace(/\b(?:github|vars|runner)(?:\.[a-zA-Z_][a-zA-Z_0-9]*)+/g, + key => JSON.stringify(context[key] ?? '')) + return runInNewContext(source, { fromJSON: JSON.parse }, { timeout: 1000 }) as unknown +} + +const trustedPr = { + 'vars.DSH_CI_FAILOVER_LINUX': 'selfhosted', + 'github.repository': repository, + 'github.actor': 'maintainer', + 'github.event_name': 'pull_request', + 'github.ref': 'refs/pull/42/merge', + 'github.event.pull_request.head.repo.full_name': repository, + 'github.event.pull_request.head.repo.fork': false, + 'github.event.pull_request.user.login': 'contributor', +} +const trustedPush = { + 'vars.DSH_CI_FAILOVER_LINUX': 'selfhosted', + 'github.repository': repository, + 'github.actor': 'maintainer', + 'github.event_name': 'push', + 'github.ref': 'refs/heads/master', +} +const fallbackCases: Array<[string, Record]> = [ + ['unset switch', { ...trustedPr, 'vars.DSH_CI_FAILOVER_LINUX': '' }], + ['hosted switch', { ...trustedPr, 'vars.DSH_CI_FAILOVER_LINUX': 'hosted' }], + ['unknown switch', { ...trustedPr, 'vars.DSH_CI_FAILOVER_LINUX': 'true' }], + ['fork PR', { ...trustedPr, 'github.event.pull_request.head.repo.full_name': 'outsider/fork', 'github.event.pull_request.head.repo.fork': true }], + ['different head repository', { ...trustedPr, 'github.event.pull_request.head.repo.full_name': 'outsider/repo' }], + ['fork flag', { ...trustedPr, 'github.event.pull_request.head.repo.fork': true }], + ['Dependabot author rerun by maintainer', { ...trustedPr, 'github.event.pull_request.user.login': 'dependabot[bot]' }], + ['Dependabot PR actor', { ...trustedPr, 'github.actor': 'dependabot[bot]' }], + ['Dependabot push actor', { ...trustedPush, 'github.actor': 'dependabot[bot]' }], + ['non-master push', { ...trustedPush, 'github.ref': 'refs/heads/topic' }], + ['tag push', { ...trustedPush, 'github.ref': 'refs/tags/dsh-v1.0.0' }], + ['push in another repository', { ...trustedPush, 'github.repository': 'outsider/fork' }], + ['dispatch on master', { ...trustedPush, 'github.event_name': 'workflow_dispatch' }], + ['dispatch on topic', { ...trustedPush, 'github.event_name': 'workflow_dispatch', 'github.ref': 'refs/heads/topic' }], + ['dispatch on tag', { ...trustedPush, 'github.event_name': 'workflow_dispatch', 'github.ref': 'refs/tags/dsh-v1.0.0' }], + ['pull_request_target', { ...trustedPr, 'github.event_name': 'pull_request_target' }], + ['missing PR payload', { ...trustedPush, 'github.event_name': 'pull_request' }], +] + +for (const [file, jobIds] of [['release.yml', ['dependencies', 'pack']], ['release-vendor.yml', ['pack']]] as const) { + describe(file, () => { + const release = workflow(file) + it('preserves the logical jobs, rehearsal events and read-only permission', () => { + expect(Object.keys(release.jobs)).toEqual(jobIds) + expect(release.on).toEqual({ pull_request: null, push: { branches: ['master'] }, workflow_dispatch: null }) + expect(release.permissions).toEqual({ contents: 'read' }) + expect(release.concurrency).toEqual({ group: '${{ github.workflow }}-${{ github.ref }}', 'cancel-in-progress': false }) + }) + for (const jobId of jobIds) { + describe(jobId, () => { + const job = release.jobs[jobId]! + it('routes trusted PRs and master pushes onto the existing Linux pool', () => { + expect(evaluate(job['runs-on'], trustedPr)).toEqual(selfhosted) + expect(evaluate(job['runs-on'], trustedPush)).toEqual(selfhosted) + expect(evaluate(job['runs-on'], { ...trustedPush, 'vars.DSH_CI_FAILOVER_LINUX': '' })).toBe(hosted) + }) + it.each(fallbackCases)('keeps %s hosted', (_name, context) => { + expect(evaluate(job['runs-on'], context)).toBe(hosted) + }) + it('cleans stale checkout output and isolates setup before any pnpm invocation', () => { + expect(job.steps[0]).toMatchObject({ uses: 'actions/checkout@v6', with: { clean: true, 'persist-credentials': false } }) + const cacheIndex = job.steps.findIndex(step => step.run?.includes('NODE_COMPILE_CACHE=')) + const pnpmIndex = job.steps.findIndex(step => step.uses?.startsWith('pnpm/') || /\bpnpm\b/.test(step.run ?? '')) + expect(cacheIndex).toBeGreaterThan(0) + expect(cacheIndex).toBeLessThan(pnpmIndex) + expect(job.steps[cacheIndex]?.run).toContain('echo "NODE_COMPILE_CACHE=${{ runner.temp }}/node-compile-cache" >> "$GITHUB_ENV"') + expect(job.steps[cacheIndex]?.run).toContain('echo "npm_config_devdir=${{ runner.temp }}/node-gyp" >> "$GITHUB_ENV"') + expect(job.steps[cacheIndex]?.run).toContain('echo "TMPDIR=${{ runner.temp }}" >> "$GITHUB_ENV"') + expect(job.steps.find(step => step.uses === 'pnpm/action-setup@v4')?.with?.dest) + .toBe('${{ runner.temp }}/setup-pnpm-${{ github.run_id }}-${{ github.run_attempt }}-${{ github.job }}') + expect(job.steps.find(step => step.name === 'Install (immutable)')?.run).toBe('pnpm install --frozen-lockfile') + }) + it('uses the persistent store without remote cache reads or writes on self-hosted', () => { + expect(job.steps.find(step => step.name === 'Configure pnpm store path')?.run).toContain('store_root="$HOME/.local/share/pnpm/store"') + const caches = job.steps.filter(step => step.uses?.startsWith('actions/cache')) + expect(caches.map(step => step.uses)).toEqual(['actions/cache/restore@v4']) + for (const step of caches) { + expect(evaluate(step.if!, { 'runner.environment': 'self-hosted' })).toBe(false) + expect(evaluate(step.if!, { 'runner.environment': 'github-hosted' })).toBe(true) + } + const nodeSetup = job.steps.find(step => step.uses === 'actions/setup-node@v6') + expect(nodeSetup?.with?.cache).toBeUndefined() + expect(nodeSetup?.with?.['package-manager-cache']).toBe(false) + }) + it('retains the dependency and pack verification commands', () => { + const commands = job.steps.flatMap(step => step.run === undefined ? [] : [step.run]) + if (jobId === 'dependencies') { + expect(commands).toContain('pnpm run verify-package-dependencies') + expect(commands).toContain('pnpm run verify-npm-install-layout') + } else { + const family = file === 'release.yml' ? 'dsh' : 'vendor' + const output = family === 'dsh' ? 'dist/npm' : 'dist/npm-vendor' + expect(job.steps[0]?.with?.['fetch-depth']).toBe(0) + expect(commands).toContain('pnpm run release:verify --family ' + family) + expect(commands).toContain('pnpm run ' + (family === 'dsh' ? 'build:official' : 'build:lib:host')) + expect(commands).toContain('pnpm run release:pack --family ' + family + ' --out ' + output + ' --concurrency 8') + expect(commands).toContain('pnpm run release:verify-packed-install --family ' + family + ' --from ' + output + + (family === 'dsh' ? ' --from dist/npm-vendor --from dist/npm-landlock' : '')) + expect(job.steps.at(-1)).toMatchObject({ uses: 'actions/upload-artifact@v4', with: { path: output + '/*', 'retention-days': 7 } }) + } + expect(JSON.stringify(job)).not.toMatch(/secrets\.|release:publish|npm-publish/) + }) + }) + } + }) +} + +it.each(['release-publish.yml', 'release-vendor-publish.yml'])('keeps %s manual and entirely hosted', (file) => { + const publish = workflow(file) + expect(publish.on).toEqual({ workflow_dispatch: null }) + for (const job of Object.values(publish.jobs)) expect(job['runs-on']).toBe(hosted) +}) diff --git a/snapshots/sdk/subagent-continuable/session.1.v2.jsonl b/snapshots/sdk/subagent-continuable/session.1.v2.jsonl index 21b2ccbcc9..4212579c43 100644 --- a/snapshots/sdk/subagent-continuable/session.1.v2.jsonl +++ b/snapshots/sdk/subagent-continuable/session.1.v2.jsonl @@ -6,8 +6,8 @@ {"type":"agent/inbox/spliced","data":{"target":"next-turn","start":0,"inserted":[{"content":[{"type":"text","text":"Reply with exactly the word CHILD_OK and nothing else."},{"type":"text","text":"Your parent agent id is \"{{session:1}}\". Before you finish, send your result to that agent with send_message({ agent_id: \"{{session:1}}\", message: \"\" }). The parent shares your workspace but does not automatically receive your transcript, tool output, or reasoning. Send earlier messages as well when a finding changes what the parent should do next; sending a message does not end your turn."}],"source":{"kind":"user"},"role":"user","id":"{{message:14}}"}]}} {"type":"turn/start","data":{"turn":1}} {"type":"agent/inbox/spliced","data":{"target":"next-turn","start":0,"removedCount":1,"inserted":[]}} -{"type":"agent/inbox/spliced","data":{"target":"next-step","start":0,"inserted":[{"content":[{"type":"text","text":"Agent {{session:1}} sent a message:"},{"type":"text","text":"Now reply with exactly SECOND_OK."}],"source":{"kind":"agent-message","form":"relay","senderSessionId":"{{session:1}}"},"role":"user","id":"{{message:15}}"}]}} -{"type":"agent/inbox/spliced","data":{"target":"next-step","start":1,"inserted":[{"content":[{"type":"text","text":"Agent {{session:1}} sent a message:"},{"type":"text","text":"Now reply with exactly THIRD_OK."}],"source":{"kind":"agent-message","form":"relay","senderSessionId":"{{session:1}}"},"role":"user","id":"{{message:16}}"}]}} +{"type":"agent/inbox/spliced","data":{"target":"next-step","start":0,"inserted":[{"content":[{"type":"text","text":"Agent {{session:1}} sent a message: "},{"type":"text","text":"Now reply with exactly SECOND_OK."}],"source":{"kind":"agent-message","form":"relay","senderSessionId":"{{session:1}}"},"role":"user","id":"{{message:15}}"}]}} +{"type":"agent/inbox/spliced","data":{"target":"next-step","start":1,"inserted":[{"content":[{"type":"text","text":"Agent {{session:1}} sent a message: "},{"type":"text","text":"Now reply with exactly THIRD_OK."}],"source":{"kind":"agent-message","form":"relay","senderSessionId":"{{session:1}}"},"role":"user","id":"{{message:16}}"}]}} {"type":"step/start","data":{"turn":1,"step":1}} {"type":"user/message","data":{"content":[{"type":"text","text":"Reply with exactly the word CHILD_OK and nothing else."},{"type":"text","text":"Your parent agent id is \"{{session:1}}\". Before you finish, send your result to that agent with send_message({ agent_id: \"{{session:1}}\", message: \"\" }). The parent shares your workspace but does not automatically receive your transcript, tool output, or reasoning. Send earlier messages as well when a finding changes what the parent should do next; sending a message does not end your turn."}],"source":{"kind":"user"},"role":"user","id":"{{message:14}}"},"surfaceOp":"append"} {"type":"user/message","data":{"content":[{"type":"text","text":"Current runtime context. This snapshot supersedes earlier runtime-context snapshots.\n\nCurrent DSH file policy: danger-full-access. The DSH file sandbox does not restrict file modifications by available operations.\n\nApproval prompts are disabled in this session: actions that require approval are rejected automatically — do not request sandbox escalation (do not set `sandbox_permissions`).\n\nYou are a delegated subagent: your permission scope was fixed when you were started and cannot be widened from inside this session — operations that require approval are rejected automatically. When the task needs access beyond that scope, do not retry the denied operation; state the limitation in your reply so the delegating agent can handle it."}],"source":{"kind":"plugin","plugin":"@deepseek-ai/dsh-system-prompt","form":"snapshot","sections":[{"name":"sandbox:policy","text":"Current DSH file policy: danger-full-access. The DSH file sandbox does not restrict file modifications by available operations."},{"name":"approval:policy","text":"Approval prompts are disabled in this session: actions that require approval are rejected automatically — do not request sandbox escalation (do not set `sandbox_permissions`)."},{"name":"subagent:delegation","text":"You are a delegated subagent: your permission scope was fixed when you were started and cannot be widened from inside this session — operations that require approval are rejected automatically. When the task needs access beyond that scope, do not retry the denied operation; state the limitation in your reply so the delegating agent can handle it."}]},"role":"user","id":"{{message:17}}"},"surfaceOp":"append"} @@ -18,8 +18,8 @@ {"type":"step/end","data":{"turn":1,"step":1}} {"type":"agent/inbox/spliced","data":{"target":"next-step","start":0,"removedCount":2,"inserted":[]}} {"type":"step/start","data":{"turn":1,"step":2}} -{"type":"user/message","data":{"content":[{"type":"text","text":"Agent {{session:1}} sent a message:"},{"type":"text","text":"Now reply with exactly SECOND_OK."}],"source":{"kind":"agent-message","form":"relay","senderSessionId":"{{session:1}}"},"role":"user","id":"{{message:15}}"},"surfaceOp":"append"} -{"type":"user/message","data":{"content":[{"type":"text","text":"Agent {{session:1}} sent a message:"},{"type":"text","text":"Now reply with exactly THIRD_OK."}],"source":{"kind":"agent-message","form":"relay","senderSessionId":"{{session:1}}"},"role":"user","id":"{{message:16}}"},"surfaceOp":"append"} +{"type":"user/message","data":{"content":[{"type":"text","text":"Agent {{session:1}} sent a message: "},{"type":"text","text":"Now reply with exactly SECOND_OK."}],"source":{"kind":"agent-message","form":"relay","senderSessionId":"{{session:1}}"},"role":"user","id":"{{message:15}}"},"surfaceOp":"append"} +{"type":"user/message","data":{"content":[{"type":"text","text":"Agent {{session:1}} sent a message: "},{"type":"text","text":"Now reply with exactly THIRD_OK."}],"source":{"kind":"agent-message","form":"relay","senderSessionId":"{{session:1}}"},"role":"user","id":"{{message:16}}"},"surfaceOp":"append"} {"type":"assistant/message","data":{"turn":1,"step":2,"message":{"role":"assistant","content":[{"type":"text","text":"SECOND_OK"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash"},"id":"{{message:19}}"},"usage":{"inputTokens":10,"outputTokens":5},"stream":[{"type":"chunk","time":1788269696707,"chunk":{"type":"block-start","index":0,"blockType":"text"}},{"type":"text-chunks","time0":1788269696707,"index":0,"dt":[],"texts":["SECOND_OK"]},{"type":"chunk","time":1788269696707,"chunk":{"type":"block-end","index":0,"block":{"type":"text","text":"SECOND_OK"}}},{"type":"chunk","time":1788269696707,"chunk":{"type":"usage","usage":{"inputTokens":10,"outputTokens":5}}},{"type":"chunk","time":1788269696707,"chunk":{"type":"finish","reason":{"kind":"stop"}}}]},"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":2}} {"type":"turn/end","data":{"turn":1,"reason":{"kind":"completed"}}} diff --git a/snapshots/sdk/subagent-send-message/session.v2.jsonl b/snapshots/sdk/subagent-send-message/session.v2.jsonl index 4d9a232e60..d80daddee2 100644 --- a/snapshots/sdk/subagent-send-message/session.v2.jsonl +++ b/snapshots/sdk/subagent-send-message/session.v2.jsonl @@ -19,13 +19,13 @@ {"type":"assistant/message","data":{"turn":1,"step":2,"message":{"role":"assistant","content":[{"type":"text","text":"STARTED"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash"},"id":"{{message:5}}"},"usage":{"inputTokens":10,"outputTokens":5},"stream":[{"type":"chunk","time":1788269697354,"chunk":{"type":"block-start","index":0,"blockType":"text"}},{"type":"text-chunks","time0":1788269697354,"index":0,"dt":[],"texts":["STARTED"]},{"type":"chunk","time":1788269697354,"chunk":{"type":"block-end","index":0,"block":{"type":"text","text":"STARTED"}}},{"type":"chunk","time":1788269697354,"chunk":{"type":"usage","usage":{"inputTokens":10,"outputTokens":5}}},{"type":"chunk","time":1788269697354,"chunk":{"type":"finish","reason":{"kind":"stop"}}}]},"surfaceOp":"append"} {"type":"step/end","data":{"turn":1,"step":2}} {"type":"turn/end","data":{"turn":1,"reason":{"kind":"completed"}}} -{"type":"agent/inbox/spliced","data":{"target":"next-step","start":0,"inserted":[{"content":[{"type":"text","text":"Agent {{session:2}} sent a message:"},{"type":"text","text":"CHILD_MESSAGE_OK"}],"source":{"kind":"agent-message","form":"relay","senderSessionId":"{{session:2}}"},"role":"user","id":"{{message:6}}"}]}} +{"type":"agent/inbox/spliced","data":{"target":"next-step","start":0,"inserted":[{"content":[{"type":"text","text":"Agent {{session:2}} sent a message: "},{"type":"text","text":"CHILD_MESSAGE_OK"}],"source":{"kind":"agent-message","form":"relay","senderSessionId":"{{session:2}}"},"role":"user","id":"{{message:6}}"}]}} {"type":"agent/inbox/spliced","data":{"target":"next-turn","start":0,"inserted":[{"content":[{"type":"text","text":"Background subagent {{session:2}} finished and will do no further work unless you send it more."},{"type":"text","text":"Its closing message:"},{"type":"text","text":"Message sent."}],"source":{"kind":"subagent-settled","form":"notice","summary":"Background subagent {{session:2}} finished and will do no further work unless you send it more.","senderSessionId":"{{session:2}}"},"role":"user","id":"{{message:7}}"}]}} {"type":"turn/start","data":{"turn":2}} {"type":"agent/inbox/spliced","data":{"target":"next-step","start":0,"removedCount":1,"inserted":[]}} {"type":"agent/inbox/spliced","data":{"target":"next-turn","start":0,"removedCount":1,"inserted":[]}} {"type":"step/start","data":{"turn":2,"step":1}} -{"type":"user/message","data":{"content":[{"type":"text","text":"Agent {{session:2}} sent a message:"},{"type":"text","text":"CHILD_MESSAGE_OK"}],"source":{"kind":"agent-message","form":"relay","senderSessionId":"{{session:2}}"},"role":"user","id":"{{message:6}}"},"surfaceOp":"append"} +{"type":"user/message","data":{"content":[{"type":"text","text":"Agent {{session:2}} sent a message: "},{"type":"text","text":"CHILD_MESSAGE_OK"}],"source":{"kind":"agent-message","form":"relay","senderSessionId":"{{session:2}}"},"role":"user","id":"{{message:6}}"},"surfaceOp":"append"} {"type":"user/message","data":{"content":[{"type":"text","text":"Background subagent {{session:2}} finished and will do no further work unless you send it more."},{"type":"text","text":"Its closing message:"},{"type":"text","text":"Message sent."}],"source":{"kind":"subagent-settled","form":"notice","summary":"Background subagent {{session:2}} finished and will do no further work unless you send it more.","senderSessionId":"{{session:2}}"},"role":"user","id":"{{message:7}}"},"surfaceOp":"append"} {"type":"assistant/message","data":{"turn":2,"step":1,"message":{"role":"assistant","content":[{"type":"text","text":"SUBAGENT_SETTLED_NOTED"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash"},"id":"{{message:8}}"},"usage":{"inputTokens":10,"outputTokens":5},"stream":[{"type":"chunk","time":1788269697428,"chunk":{"type":"block-start","index":0,"blockType":"text"}},{"type":"text-chunks","time0":1788269697428,"index":0,"dt":[],"texts":["SUBAGENT_SETTLED_NOTED"]},{"type":"chunk","time":1788269697428,"chunk":{"type":"block-end","index":0,"block":{"type":"text","text":"SUBAGENT_SETTLED_NOTED"}}},{"type":"chunk","time":1788269697428,"chunk":{"type":"usage","usage":{"inputTokens":10,"outputTokens":5}}},{"type":"chunk","time":1788269697428,"chunk":{"type":"finish","reason":{"kind":"stop"}}}]},"surfaceOp":"append"} {"type":"step/end","data":{"turn":2,"step":1}}