diff --git a/.agents/notes/implemented/process/2026-07-21-serial-cross-platform-ci-reference.i18n.yaml b/.agents/notes/implemented/process/2026-07-21-serial-cross-platform-ci-reference.i18n.yaml index 3263246d05..12c5d5d1cc 100644 --- a/.agents/notes/implemented/process/2026-07-21-serial-cross-platform-ci-reference.i18n.yaml +++ b/.agents/notes/implemented/process/2026-07-21-serial-cross-platform-ci-reference.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/process/2026-07-21-serial-cross-platform-ci-reference.md -2026-07-21-serial-cross-platform-ci-reference.md: e92104cce5b726797f7b4d88c98cf3b837cba6b4 -2026-07-21-serial-cross-platform-ci-reference.zh.md: a8787006b7d44ecab94f01b771b62bfa0ae3224b +2026-07-21-serial-cross-platform-ci-reference.md: fb44f1122eae8120615ab5963d40b8fa3fb6b3bb +2026-07-21-serial-cross-platform-ci-reference.zh.md: 9a316b13912347e2bfff0fbf866a8470c7070029 diff --git a/.agents/notes/implemented/process/2026-07-21-serial-cross-platform-ci-reference.md b/.agents/notes/implemented/process/2026-07-21-serial-cross-platform-ci-reference.md index e92104cce5..fb44f1122e 100644 --- a/.agents/notes/implemented/process/2026-07-21-serial-cross-platform-ci-reference.md +++ b/.agents/notes/implemented/process/2026-07-21-serial-cross-platform-ci-reference.md @@ -16,7 +16,7 @@ Real-kernel sandbox proofs require specific hosted operating systems and archite ## Decision -[CI](../../../../.github/workflows/ci.yml) (pull-request-only) and [CI master](../../../../.github/workflows/ci-master.yml) (master-push + workflow_dispatch) give pull-request and master-push events complementary responsibilities. Pull requests run Linux, native Windows, Node compatibility, and Python checks; [platform scheduling](2026-09-06-master-only-platform-ci.md) assigns Wine and three Python runtime carriers to master pushes. On a push to `master`, the active references are `serial / linux (self-hosted standby)` on the in-house `vm-backup` pool and `serial / windows (self-hosted standby)` on the in-house `dsh-win-ci` pool — the hot-standby drills that continuously re-prove the failover targets described in the [failover runbook](2026-07-26-ci-failover-runbook.md). There is no standard-hosted `serial / linux` definition; the standard-hosted `serial / macos` remains disabled under `TODO(hosted-serial-ci)` until its portable capacity can be restored. The current `serial / windows` definition is the in-house `dsh-win-ci` standby. The separate job definitions intentionally keep their short checkout, runtime setup, and immutable install sequences visible instead of hiding operating systems behind a matrix or reusable workflow. `workflow_dispatch` is reserved for runner benchmarks. +[CI](../../../../.github/workflows/ci.yml) (pull-request-only) and [CI master](../../../../.github/workflows/ci-master.yml) (master-push + workflow_dispatch) give pull-request and master-push events complementary responsibilities. Pull requests run Linux, native Windows, [Node compatibility with isolated self-hosted routing](2026-09-06-node-compatibility-selfhosted.md), and Python checks; [platform scheduling](2026-09-06-master-only-platform-ci.md) assigns Wine and three Python runtime carriers to master pushes. On a push to `master`, the active references are `serial / linux (self-hosted standby)` on the in-house `vm-backup` pool and `serial / windows (self-hosted standby)` on the in-house `dsh-win-ci` pool — the hot-standby drills that continuously re-prove the failover targets described in the [failover runbook](2026-07-26-ci-failover-runbook.md). There is no standard-hosted `serial / linux` definition; the standard-hosted `serial / macos` remains disabled under `TODO(hosted-serial-ci)` until its portable capacity can be restored. The current `serial / windows` definition is the in-house `dsh-win-ci` standby. The separate job definitions intentionally keep their short checkout, runtime setup, and immutable install sequences visible instead of hiding operating systems behind a matrix or reusable workflow. `workflow_dispatch` is reserved for runner benchmarks. Each reference job runs `pnpm run check:ci` without any shard selector. `DSH_GATE_CONCURRENCY=1` makes the top-level aggregate execute one ready gate at a time; coverage, snapshot replay, built-bin smoke, and publication validation also receive worker counts of one. The reference jobs may run beside one another, but each host's repository gates are serial and complete. Linux installs bubblewrap before replaying snapshots, and Windows enables Developer Mode before installing the symlinked workspace. diff --git a/.agents/notes/implemented/process/2026-07-21-serial-cross-platform-ci-reference.zh.md b/.agents/notes/implemented/process/2026-07-21-serial-cross-platform-ci-reference.zh.md index a8787006b7..9a316b1391 100644 --- a/.agents/notes/implemented/process/2026-07-21-serial-cross-platform-ci-reference.zh.md +++ b/.agents/notes/implemented/process/2026-07-21-serial-cross-platform-ci-reference.zh.md @@ -16,7 +16,7 @@ Status: implemented ## 决策 -[CI](../../../../.github/workflows/ci.yml)（仅 pull request）与 [CI master](../../../../.github/workflows/ci-master.yml)（master 推送 + `workflow_dispatch`）为拉取请求事件与 master 推送事件赋予互补的职责。拉取请求运行 Linux、原生 Windows、Node 兼容性与 Python 检查；[平台调度](2026-09-06-master-only-platform-ci.zh.md)将 Wine 与三个 Python 运行时载体分配给 master 推送。向 `master` 推送时，当前启用的参考作业是公司自有 `vm-backup` 池上的 `serial / linux (self-hosted standby)` 和 `dsh-win-ci` 池上的 `serial / windows (self-hosted standby)`——这些热备演练持续验证[故障切换手册](2026-07-26-ci-failover-runbook.zh.md)所描述的切换目标。不存在标准托管的 `serial / linux` 定义；标准托管的 `serial / macos` 仍处于禁用状态，并由 `TODO(hosted-serial-ci)` 标记，直到其可移植容量恢复。当前 `serial / windows` 定义是公司自有 `dsh-win-ci` 池的 standby。各自独立的作业定义有意显式保留简短的代码检出、运行时设置和依赖锁定的安装步骤，而不是用矩阵或可复用工作流隐藏操作系统差异。`workflow_dispatch` 仅用于运行器基准测试。 +[CI](../../../../.github/workflows/ci.yml)（仅 pull request）与 [CI master](../../../../.github/workflows/ci-master.yml)（master 推送 + `workflow_dispatch`）为拉取请求事件与 master 推送事件赋予互补的职责。拉取请求运行 Linux、原生 Windows、[使用隔离自托管路由的 Node 兼容性](2026-09-06-node-compatibility-selfhosted.zh.md)与 Python 检查；[平台调度](2026-09-06-master-only-platform-ci.zh.md)将 Wine 与三个 Python 运行时载体分配给 master 推送。向 `master` 推送时，当前启用的参考作业是公司自有 `vm-backup` 池上的 `serial / linux (self-hosted standby)` 和 `dsh-win-ci` 池上的 `serial / windows (self-hosted standby)`——这些热备演练持续验证[故障切换手册](2026-07-26-ci-failover-runbook.zh.md)所描述的切换目标。不存在标准托管的 `serial / linux` 定义；标准托管的 `serial / macos` 仍处于禁用状态，并由 `TODO(hosted-serial-ci)` 标记，直到其可移植容量恢复。当前 `serial / windows` 定义是公司自有 `dsh-win-ci` 池的 standby。各自独立的作业定义有意显式保留简短的代码检出、运行时设置和依赖锁定的安装步骤，而不是用矩阵或可复用工作流隐藏操作系统差异。`workflow_dispatch` 仅用于运行器基准测试。 每个参考作业均在不设置任何分片选择器的情况下运行 `pnpm run check:ci`。`DSH_GATE_CONCURRENCY=1` 使顶层聚合每次只执行一个已经就绪的门禁；覆盖率、快照回放、built-bin 冒烟测试和发布验证的 worker 数量也设为 1。各参考作业可以彼此并行，但每台主机上的仓库门禁都串行运行且完整执行。Linux 在回放快照前安装 bubblewrap，Windows 则在安装采用符号链接的工作区前启用开发人员模式。 diff --git a/.agents/notes/implemented/process/2026-07-26-ci-failover-runbook.i18n.yaml b/.agents/notes/implemented/process/2026-07-26-ci-failover-runbook.i18n.yaml index 50e7fa9040..012d82fc53 100644 --- a/.agents/notes/implemented/process/2026-07-26-ci-failover-runbook.i18n.yaml +++ b/.agents/notes/implemented/process/2026-07-26-ci-failover-runbook.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/process/2026-07-26-ci-failover-runbook.md -2026-07-26-ci-failover-runbook.md: 9fbdd76ce3a376ea5b4e86584f14c3558bddff9e -2026-07-26-ci-failover-runbook.zh.md: fda030d9c628709c31ec53e767c74989f130a7b1 +2026-07-26-ci-failover-runbook.md: 68fcbf956410235bb245354477234fb921561d7c +2026-07-26-ci-failover-runbook.zh.md: 7f3cffe8b34250b32351cd890f7de1c129ca992b diff --git a/.agents/notes/implemented/process/2026-07-26-ci-failover-runbook.md b/.agents/notes/implemented/process/2026-07-26-ci-failover-runbook.md index 9fbdd76ce3..68fcbf9564 100644 --- a/.agents/notes/implemented/process/2026-07-26-ci-failover-runbook.md +++ b/.agents/notes/implemented/process/2026-07-26-ci-failover-runbook.md @@ -6,7 +6,7 @@ English | [中文](2026-07-26-ci-failover-runbook.zh.md) ## Problem -The three required Linux worker jobs in [CI](../../../../.github/workflows/ci.yml) (`node 24 / static`, `node 24 / coverage`, `node 24 / snapshots and artifacts`) run on the hosted enterprise 32-core pools; the required verdict job that aggregates them (`all checks passed`) runs on standard `ubuntu-latest`; the [native Windows jobs](2026-08-08-native-windows-pull-request-ci.md) run on the hosted `dsh-windows-2025-16core` larger runner. When the enterprise pools degrade — jobs queue indefinitely or the enterprise labels vanish — every open pull request becomes unmergeable, and the ordinary recovery of merging a fix is itself deadlocked behind the very required checks that cannot run. **Scope: two independent switches, one per platform.** `DSH_CI_FAILOVER_LINUX` recovers an enterprise Linux-pool outage (the three required Linux workers plus the `all checks passed` verdict); `DSH_CI_FAILOVER_WINDOWS` recovers a hosted Windows-pool outage (the native Windows jobs). A Linux-pool outage need not retarget Windows jobs and vice versa. The verdict's other required dependencies (`node-24-bench`, `node-compat`, `python-sdk`, `python-runtime`) stay on standard hosted runners by design (the portable boundary); in a broader GitHub-hosted capacity failure that also takes out the standard pools, those dependencies still block `all checks passed`. An outage therefore needs a switch any responder with repository write access can throw without merging anything. +The three required Linux worker jobs in [CI](../../../../.github/workflows/ci.yml) (`node 24 / static`, `node 24 / coverage`, `node 24 / snapshots and artifacts`) run on the hosted enterprise 32-core pools; the required verdict job that aggregates them (`all checks passed`) runs on standard `ubuntu-latest`; the [native Windows jobs](2026-08-08-native-windows-pull-request-ci.md) run on the hosted `dsh-windows-2025-16core` larger runner. When the enterprise pools degrade — jobs queue indefinitely or the enterprise labels vanish — every open pull request becomes unmergeable, and the ordinary recovery of merging a fix is itself deadlocked behind the very required checks that cannot run. **Scope: two independent switches, one per platform.** `DSH_CI_FAILOVER_LINUX` recovers an enterprise Linux-pool outage (the three required Linux workers plus the `all checks passed` verdict); `DSH_CI_FAILOVER_WINDOWS` recovers a hosted Windows-pool outage (the native Windows jobs). A Linux-pool outage need not retarget Windows jobs and vice versa. The [Node compatibility jobs](2026-09-06-node-compatibility-selfhosted.md) also follow the Linux switch with isolated setup; the verdict's `node-24-bench`, `python-sdk`, and `python-runtime` dependencies stay on standard hosted runners; in a broader GitHub-hosted capacity failure that also takes out the standard pools, those dependencies still block `all checks passed`. An outage therefore needs a switch any responder with repository write access can throw without merging anything. ## Decision diff --git a/.agents/notes/implemented/process/2026-07-26-ci-failover-runbook.zh.md b/.agents/notes/implemented/process/2026-07-26-ci-failover-runbook.zh.md index fda030d9c6..7f3cffe8b3 100644 --- a/.agents/notes/implemented/process/2026-07-26-ci-failover-runbook.zh.md +++ b/.agents/notes/implemented/process/2026-07-26-ci-failover-runbook.zh.md @@ -6,7 +6,7 @@ Status: implemented ## 问题 -[CI](../../../../.github/workflows/ci.yml) 中三个必需的 Linux 工作作业（`node 24 / static`、`node 24 / coverage`、`node 24 / snapshots and artifacts`）运行在托管的企业级 32 核池上；聚合它们的必需判定作业（`all checks passed`）运行在标准 `ubuntu-latest` 上；[原生 Windows 作业](2026-08-08-native-windows-pull-request-ci.zh.md)运行在托管的 `dsh-windows-2025-16core` 大型运行器上。当企业池发生故障——作业无限排队或企业标签消失——所有开启的拉取请求都无法合并，而"合并一个修复"这一常规恢复手段本身正被那些无法运行的必需检查死锁。**适用范围：两个独立开关，每个平台一个。**`DSH_CI_FAILOVER_LINUX` 恢复企业级 Linux 池故障（三个必需的 Linux 工作作业加 `all checks passed` 判定作业）；`DSH_CI_FAILOVER_WINDOWS` 恢复托管 Windows 池故障（原生 Windows 作业）。Linux 池故障无需重定向 Windows 作业，反之亦然。判定作业的其余必需依赖（`node-24-bench`、`node-compat`、`python-sdk`、`python-runtime`）按设计留在标准托管运行器上（可移植边界）；若更大范围的 GitHub 托管容量故障连标准池一并击倒，这些依赖仍会阻塞 `all checks passed`。因此故障需要一个任何具备仓库写权限的响应者都能在不合并任何代码的情况下触发的开关。 +[CI](../../../../.github/workflows/ci.yml) 中三个必需的 Linux 工作作业（`node 24 / static`、`node 24 / coverage`、`node 24 / snapshots and artifacts`）运行在托管的企业级 32 核池上；聚合它们的必需判定作业（`all checks passed`）运行在标准 `ubuntu-latest` 上；[原生 Windows 作业](2026-08-08-native-windows-pull-request-ci.zh.md)运行在托管的 `dsh-windows-2025-16core` 大型运行器上。当企业池发生故障——作业无限排队或企业标签消失——所有开启的拉取请求都无法合并，而"合并一个修复"这一常规恢复手段本身正被那些无法运行的必需检查死锁。**适用范围：两个独立开关，每个平台一个。**`DSH_CI_FAILOVER_LINUX` 恢复企业级 Linux 池故障（三个必需的 Linux 工作作业加 `all checks passed` 判定作业）；`DSH_CI_FAILOVER_WINDOWS` 恢复托管 Windows 池故障（原生 Windows 作业）。Linux 池故障无需重定向 Windows 作业，反之亦然。[Node 兼容性作业](2026-09-06-node-compatibility-selfhosted.zh.md)也通过隔离设置跟随 Linux 开关；判定作业的 `node-24-bench`、`python-sdk` 和 `python-runtime` 依赖仍留在标准托管运行器上；若更大范围的 GitHub 托管容量故障连标准池一并击倒，这些依赖仍会阻塞 `all checks passed`。因此故障需要一个任何具备仓库写权限的响应者都能在不合并任何代码的情况下触发的开关。 ## 决策 diff --git a/.agents/notes/implemented/process/2026-09-06-node-compatibility-selfhosted.i18n.yaml b/.agents/notes/implemented/process/2026-09-06-node-compatibility-selfhosted.i18n.yaml new file mode 100644 index 0000000000..f65f8cb889 --- /dev/null +++ b/.agents/notes/implemented/process/2026-09-06-node-compatibility-selfhosted.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write .agents/notes/implemented/process/2026-09-06-node-compatibility-selfhosted.md +2026-09-06-node-compatibility-selfhosted.md: e62be31b4383d9d13548565dce6371a4b250857d +2026-09-06-node-compatibility-selfhosted.zh.md: ca8457d160dfd5f48010eb80ee03984425e29392 diff --git a/.agents/notes/implemented/process/2026-09-06-node-compatibility-selfhosted.md b/.agents/notes/implemented/process/2026-09-06-node-compatibility-selfhosted.md new file mode 100644 index 0000000000..e62be31b43 --- /dev/null +++ b/.agents/notes/implemented/process/2026-09-06-node-compatibility-selfhosted.md @@ -0,0 +1,35 @@ +# Agent Note: Isolated Node compatibility jobs on self-hosted Linux + +Status: implemented + +English | [中文](2026-09-06-node-compatibility-selfhosted.zh.md) + +## Problem + +The Node 22.19, 24.9, and 26 compatibility jobs consume hosted Linux minutes even when the repository has selected its existing self-hosted Linux pool. Moving version installers onto a persistent shared machine can create tool-directory collisions and accumulate generated cache files outside runner cleanup. + +## Decision + +[CI](../../../../.github/workflows/ci.yml) applies the Linux failover variable to these three jobs, requiring a non-Dependabot author and a non-fork head repository matching the current repository. The standard hosted fallback remains available. [Runner documentation](../../../../docs/ci-compatible-selfhosted.md) owns setup and cleanup behavior. + +The temporary tool cache trades repeated Node downloads for isolation across concurrent runners and Node versions. pnpm keeps its existing private setup destination and persistent content-addressed store. Compile caches and node-gyp headers use runner temp before the first pnpm invocation. No global Node symlink or system package changes are introduced. Hosted jobs retain their tool and package caching. + +The [failover runbook](2026-07-26-ci-failover-runbook.md) remains the owner of repository trust and pool switching. The [serial reference decision](2026-07-21-serial-cross-platform-ci-reference.md) remains the owner of master scheduling. Neither decision is superseded beyond the compatibility jobs' runner selection; both remain active. + +## Alternatives considered + +**Keep all compatibility jobs hosted.** This avoids extra shared-host load but continues paying for Linux runtime checks that do not require a different operating system or architecture. + +**Use the shared Node installation or global version-manager links.** The jobs must run different Node releases concurrently. Mutable shared links would make the selected version depend on another job's timing. + +**Move the Python SDK job in the same change.** Its setup-python installation and global pip installation of uv need separate isolation evidence. Its short hosted job is not required for the Node optimization. + +## Consequences + +The pool receives three additional jobs per trusted PR; each retains gate concurrency one, including the build-backed Node 22 leg. The September 6 inventory reports 31 Linux registrations, not 31 independent machines. The shared VM's contention and download latency remain rollout risks; the variable preserves hosted recovery. Test inventory, check names, and master scheduling are unchanged. + +## Verification + +The focused [workflow regression](../../../../scripts/ci-compatible-selfhosted.spec.ts) executes the actual routing expressions and environment setup. A negative control removing the fork condition fails the hosted-fallback assertion. It checks Dependabot reruns by a maintainer, repository mismatch, fork flags, disabled variables, and runner-scoped cache paths. + +[Successful standby run 33984559660](https://github.com/deepseek-harness/deepseek-harness/actions/runs/33984559660) at the implementation base supplies Linux Node 24.19.0 and Windows Node 24.20.0 baseline evidence. Linux job 101359402557 uses runner-specific temporary and tool directories on the data volume. [Read-only capability probe 34012679056](https://github.com/deepseek-harness/deepseek-harness/actions/runs/34012679056/job/101431064925) reports Linux x64, 192 online logical CPUs, GCC/G++ 13.3, Make 4.3, and Python 3.12.3. Python 3.10 is absent, reinforcing the separate SDK provisioning requirement. That baseline does not prove these three exact Node versions on self-hosted Linux; their PR matrix execution is the platform verification owner. diff --git a/.agents/notes/implemented/process/2026-09-06-node-compatibility-selfhosted.zh.md b/.agents/notes/implemented/process/2026-09-06-node-compatibility-selfhosted.zh.md new file mode 100644 index 0000000000..ca8457d160 --- /dev/null +++ b/.agents/notes/implemented/process/2026-09-06-node-compatibility-selfhosted.zh.md @@ -0,0 +1,35 @@ +# Agent Note: 自托管 Linux 上隔离的 Node 兼容性作业 + +Status: implemented + +[English](2026-09-06-node-compatibility-selfhosted.md) | 中文 + +## 问题 + +即使仓库已经选择现有的自托管 Linux 池，Node 22.19、24.9 和 26 兼容性作业仍消耗托管 Linux 分钟数。将版本安装器移到持久化共享机器上可能造成工具目录冲突，并让生成的缓存文件积累在运行器清理范围之外。 + +## 决策 + +[CI](../../../../.github/workflows/ci.yml) 将 Linux 故障切换变量应用于这三个作业，要求作者不是 Dependabot，且非 fork 的头部仓库与当前仓库相同。标准托管回退仍然可用。[运行器文档](../../../../docs/ci-compatible-selfhosted.zh.md) 拥有安装与清理行为的说明。 + +临时工具缓存以重复下载 Node 为代价，换取并发运行器与 Node 版本之间的隔离。pnpm 保留现有的私有安装目录和持久化内容寻址 store。编译缓存与 node-gyp 头文件在首次调用 pnpm 前就使用运行器临时目录。不引入全局 Node 符号链接或系统软件包变更。托管作业保留其工具与软件包缓存。 + +[故障切换手册](2026-07-26-ci-failover-runbook.zh.md) 仍拥有仓库信任与池切换规则。[串行参考决策](2026-07-21-serial-cross-platform-ci-reference.zh.md) 仍拥有 master 调度规则。除兼容性作业的运行器选择外，这两个决策都未被取代；两者均保持活跃。 + +## 曾考虑的替代方案 + +**让所有兼容性作业保持托管。** 这避免额外的共享主机负载，但继续为不需要不同操作系统或架构的 Linux 运行时检查付费。 + +**使用共享 Node 安装或全局版本管理器链接。** 这些作业必须并发运行不同的 Node 版本。可变的共享链接会使选中的版本取决于另一作业的时序。 + +**在同一改动中迁移 Python SDK 作业。** 其 setup-python 安装和通过全局 pip 安装 uv 需要单独的隔离证据。这个短暂的托管作业不是 Node 优化的必需部分。 + +## 后果 + +每个可信 PR（Pull Request）会为池增加三个作业；每个作业保留门禁并发度一，包括需要构建的 Node 22 条目。9 月 6 日的清单报告了 31 个 Linux 注册实例，而不是 31 台独立机器。共享虚拟机的资源争用和下载延迟仍是上线风险；变量保留托管恢复路径。测试清单、检查名称和 master 调度保持不变。 + +## 验证 + +聚焦的[工作流回归测试](../../../../scripts/ci-compatible-selfhosted.spec.ts) 执行真实的路由表达式和环境设置。移除 fork 条件的负对照使托管回退断言失败。它检查维护者重跑 Dependabot PR、仓库不匹配、fork 标志、禁用变量以及运行器范围内的缓存路径。 + +实施基线上的[成功热备运行 33984559660](https://github.com/deepseek-harness/deepseek-harness/actions/runs/33984559660) 提供 Linux Node 24.19.0 和 Windows Node 24.20.0 基线证据。Linux 作业 101359402557 使用数据卷上运行器专属的临时目录和工具目录。[只读能力探测 34012679056](https://github.com/deepseek-harness/deepseek-harness/actions/runs/34012679056/job/101431064925) 报告 Linux x64、192 个在线逻辑 CPU、GCC/G++ 13.3、Make 4.3 和 Python 3.12.3。Python 3.10 缺失，进一步说明 SDK 需要单独配置。该基线不能证明自托管 Linux 上这三个精确 Node 版本的行为；其 PR 矩阵执行拥有平台验证责任。 diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index fa7ecfdd9e..03a0bfbeee 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -291,8 +291,14 @@ jobs: node-compat: if: github.event_name == 'pull_request' - # Each compatibility contract receives an independent standard hosted job. - runs-on: ${{ matrix.runner }} + # Only repository-owned PR code may reach the persistent shared VM. + runs-on: >- + ${{ vars.DSH_CI_FAILOVER_LINUX == 'selfhosted' + && github.event.pull_request.head.repo.full_name == github.repository + && github.event.pull_request.head.repo.fork == false + && github.event.pull_request.user.login != 'dependabot[bot]' + && fromJSON('["self-hosted", "linux", "x64", "vm-backup"]') + || matrix.runner }} name: ${{ matrix.name }} env: DSH_GATE_CONCURRENCY: ${{ matrix.gate_concurrency }} @@ -321,15 +327,28 @@ jobs: gate_concurrency: '1' steps: - uses: actions/checkout@v6 + with: + persist-credentials: false + + # Shared hosts keep version installs and generated caches inside runner temp. + - name: Isolate compatibility caches + if: runner.environment == 'self-hosted' + run: | + echo "NODE_COMPILE_CACHE=$RUNNER_TEMP/node-compile-cache" >> "$GITHUB_ENV" + echo "npm_config_devdir=$RUNNER_TEMP/node-gyp" >> "$GITHUB_ENV" + echo "PNPM_CONFIG_STORE_DIR=$HOME/.local/share/pnpm/store" >> "$GITHUB_ENV" - uses: pnpm/action-setup@v4 with: dest: ${{ runner.temp }}/setup-pnpm-${{ github.run_id }}-${{ github.run_attempt }} - uses: actions/setup-node@v6 + env: + RUNNER_TOOL_CACHE: ${{ runner.environment == 'self-hosted' && format('{0}/node-compat-toolcache', runner.temp) || runner.tool_cache }} with: node-version: ${{ matrix.node }} - cache: pnpm + cache: ${{ runner.environment == 'github-hosted' && 'pnpm' || '' }} + package-manager-cache: false - name: Install (immutable) run: pnpm install --frozen-lockfile diff --git a/docs/ci-compatible-selfhosted.i18n.yaml b/docs/ci-compatible-selfhosted.i18n.yaml new file mode 100644 index 0000000000..b127493d64 --- /dev/null +++ b/docs/ci-compatible-selfhosted.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write docs/ci-compatible-selfhosted.md +ci-compatible-selfhosted.md: b8f4efd31449c83d11c1db9802dfe1255743edb1 +ci-compatible-selfhosted.zh.md: 592f9d6b467315b3ed88b81df9fbd90a206f7f8b diff --git a/docs/ci-compatible-selfhosted.md b/docs/ci-compatible-selfhosted.md new file mode 100644 index 0000000000..b8f4efd314 --- /dev/null +++ b/docs/ci-compatible-selfhosted.md @@ -0,0 +1,39 @@ +# Node compatibility CI runners + +English | [中文](ci-compatible-selfhosted.zh.md) + +## Summary + +The three Node compatibility jobs can use the existing Linux self-hosted pool without changing their versions, required checks, or master scheduling. [CI](../.github/workflows/ci.yml) owns the runner selection; the [decision record](../.agents/notes/implemented/process/2026-09-06-node-compatibility-selfhosted.md) explains isolation and trade-offs. + +## Table of Contents + +- [Runner selection](#runner-selection) +- [Installation and cleanup](#installation-and-cleanup) +- [Verification](#verification) + + + +## Runner selection + +Node 22.19, 24.9, and 26 select `[self-hosted, linux, x64, vm-backup]` only when `DSH_CI_FAILOVER_LINUX=selfhosted` and the PR author is not Dependabot, the head repository matches the current repository, and the head repository is not a fork. All other cases select `ubuntu-latest`. The Python SDK job remains hosted. + +Each matrix entry runs one repository gate at a time. The matrix retains independent jobs and does not cancel sibling versions on failure. Runner registrations share host resources; their count is not a count of independent machines. + + + +## Installation and cleanup + +Self-hosted Node installations use a tool cache beneath `runner.temp`. pnpm setup uses its runner-and-run-private destination. Node compile caches and node-gyp headers also stay beneath runner temp; the pnpm content-addressed store remains persistent. Hosted jobs retain their normal tool cache and pnpm caching. Self-hosted jobs do not restore or upload hosted package caches. + +The runner owns temporary-directory cleanup between jobs. These jobs do not install system packages or change global Node symlinks. The shared image must already provide the compiler and Python dependencies needed by native npm packages. A cold temporary Node cache requires downloading the selected runtime again. + + + +## Verification + +`pnpm exec vitest run scripts/ci-compatible-selfhosted.spec.ts scripts/ci-workflow.spec.ts` checks routing, hosted fallback, matrix preservation, cache paths, and the executed environment setup. The actual Node matrix on the self-hosted host remains the required platform verification; local workflow tests do not prove native runtime compatibility or capacity under concurrent PR load. + +## Dev Note + +None. diff --git a/docs/ci-compatible-selfhosted.zh.md b/docs/ci-compatible-selfhosted.zh.md new file mode 100644 index 0000000000..592f9d6b46 --- /dev/null +++ b/docs/ci-compatible-selfhosted.zh.md @@ -0,0 +1,39 @@ +# Node 兼容性 CI 运行器 + +[English](ci-compatible-selfhosted.md) | 中文 + +## 摘要 + +三个 Node 兼容性作业可以使用现有的 Linux 自托管池，而不改变其版本、必需检查或 master 调度。[CI](../.github/workflows/ci.yml) 拥有运行器选择逻辑；[决策记录](../.agents/notes/implemented/process/2026-09-06-node-compatibility-selfhosted.zh.md) 解释隔离和取舍。 + +## 目录 + +- [运行器选择](#runner-selection) +- [安装与清理](#installation-and-cleanup) +- [验证](#verification) + + + +## 运行器选择 + +Node 22.19、24.9 和 26 仅在 `DSH_CI_FAILOVER_LINUX=selfhosted`，且 PR（Pull Request）作者不是 Dependabot、头部仓库与当前仓库相同、头部仓库不是 fork 时选择 `[self-hosted, linux, x64, vm-backup]`。其余情况均选择 `ubuntu-latest`。Python SDK 作业仍使用托管运行器。 + +每个矩阵条目一次运行一个仓库门禁。矩阵保留独立作业，不会因某个版本失败而取消其他版本。运行器注册实例共享主机资源；注册数量不等于独立机器数量。 + + + +## 安装与清理 + +自托管 Node 安装使用 `runner.temp` 下的工具缓存。pnpm 设置使用运行器和运行私有的目标目录。Node 编译缓存和 node-gyp 头文件也保留在运行器临时目录下；pnpm 内容寻址 store 保持持久化。托管作业保留其常规工具缓存和 pnpm 缓存。自托管作业不恢复或上传托管软件包缓存。 + +运行器负责作业之间的临时目录清理。这些作业不安装系统软件包，也不修改全局 Node 符号链接。共享镜像必须已提供原生 npm 软件包所需的编译器和 Python 依赖。冷的临时 Node 缓存需要重新下载所选运行时。 + + + +## 验证 + +`pnpm exec vitest run scripts/ci-compatible-selfhosted.spec.ts scripts/ci-workflow.spec.ts` 检查路由、托管回退、矩阵保留、缓存路径和实际执行的环境设置。自托管主机上的真实 Node 矩阵仍是必需的平台验证；本地工作流测试不能证明原生运行时兼容性或并发 PR 负载下的容量。 + +## 开发备注 + +无。 diff --git a/scripts/ci-compatible-selfhosted.spec.ts b/scripts/ci-compatible-selfhosted.spec.ts new file mode 100644 index 0000000000..8c3ddd2165 --- /dev/null +++ b/scripts/ci-compatible-selfhosted.spec.ts @@ -0,0 +1,131 @@ +import { mkdtempSync, readFileSync, rmSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { join, resolve } from 'node:path' +import { spawnSync } from 'node:child_process' +import { runInNewContext } from 'node:vm' +import * as yaml from 'js-yaml' +import { describe, expect, it } from 'vitest' + +interface Step { + name?: string + uses?: string + if?: string + run?: string + env?: Record + with?: Record +} + +interface CompatibilityJob { + 'runs-on': string + if: string + env: Record + strategy: { 'fail-fast': boolean; matrix: { include: Array<{ node: string | number; name: string; runner: string; gate_concurrency: string }> } } + steps: Step[] +} + +const workflow = yaml.load(readFileSync(resolve(import.meta.dirname, '../.github/workflows/ci.yml'), 'utf8')) as { + jobs: { 'node-compat': CompatibilityJob; 'python-sdk': { 'runs-on': string } } +} +const job = workflow.jobs['node-compat'] +const labels = ['self-hosted', 'linux', 'x64', 'vm-backup'] + +// These expressions use only boolean operators and equal-typed string/boolean +// comparisons, whose results agree in Actions and JavaScript. +function evaluate(expression: string, context: Record): unknown { + const body = expression.trim().slice(3, -2) + return runInNewContext(body, { + ...context, fromJSON: JSON.parse, format: (template: string, value: string) => template.replace('{0}', value), + }, { timeout: 1000 }) as unknown +} + +function route(options: { mode?: string; author?: string; repository?: string; fork?: boolean; actor?: string } = {}): unknown { + return evaluate(job['runs-on'], { + vars: { DSH_CI_FAILOVER_LINUX: options.mode ?? 'selfhosted' }, + github: { + repository: 'deepseek-harness/deepseek-harness', + actor: options.actor ?? 'maintainer', + event: { pull_request: { + user: { login: options.author ?? 'maintainer' }, + head: { repo: { full_name: options.repository ?? 'deepseek-harness/deepseek-harness', fork: options.fork ?? false } }, + } }, + }, + matrix: { runner: 'ubuntu-latest' }, + }) +} + +describe('Node compatibility self-hosted routing', () => { + it('uses the Linux pool only for opted-in repository-owned PRs', () => { + expect(route()).toEqual(labels) + for (const mode of ['', 'hosted', 'unexpected']) expect(route({ mode })).toBe('ubuntu-latest') + expect(route({ author: 'dependabot[bot]', actor: 'maintainer' })).toBe('ubuntu-latest') + expect(route({ repository: 'outsider/fork', fork: true })).toBe('ubuntu-latest') + expect(route({ repository: 'outsider/fork', fork: false })).toBe('ubuntu-latest') + expect(route({ fork: true })).toBe('ubuntu-latest') + expect(route({ repository: '' })).toBe('ubuntu-latest') + }) + + it('preserves all three required version jobs and their concurrency', () => { + expect(job.if).toBe("github.event_name == 'pull_request'") + expect(job.strategy['fail-fast']).toBe(false) + expect(job.strategy.matrix.include).toEqual([ + { node: '22.19', name: 'node 22.19', runner: 'ubuntu-latest', gate_concurrency: '1' }, + { node: '24.9', name: 'node 24.9', runner: 'ubuntu-latest', gate_concurrency: '1' }, + { node: 26, name: 'node 26', runner: 'ubuntu-latest', gate_concurrency: '1' }, + ]) + expect(job.env.DSH_GATE_CONCURRENCY).toBe('${{ matrix.gate_concurrency }}') + expect(job.steps.map(step => step.run)).toContain('pnpm run check:node-compat') + expect(job.steps.map(step => step.run)).toContain('pnpm exec vitest run packages/boot/app-boot/tests/loader-shape.compat.spec.ts') + expect(workflow.jobs['python-sdk']['runs-on']).toBe('ubuntu-latest') + }) + + it('isolates version installs and enables hosted package caching only on hosted runners', () => { + const setup = job.steps.find(step => step.uses === 'actions/setup-node@v6')! + expect(setup.env).toEqual({ + RUNNER_TOOL_CACHE: "${{ runner.environment == 'self-hosted' && format('{0}/node-compat-toolcache', runner.temp) || runner.tool_cache }}", + }) + expect(setup.with?.['node-version']).toBe('${{ matrix.node }}') + expect(setup.with?.['package-manager-cache']).toBe(false) + for (const [environment, cache] of [['github-hosted', 'pnpm'], ['self-hosted', '']]) { + const context = { runner: { environment, temp: '/runner/temp', tool_cache: '/runner/toolcache' } } + expect(evaluate(setup.with?.cache as string, context)).toBe(cache) + expect(evaluate(setup.env!.RUNNER_TOOL_CACHE!, context)).toBe( + environment === 'self-hosted' ? '/runner/temp/node-compat-toolcache' : '/runner/toolcache', + ) + } + expect(job.steps[0]?.with).toEqual({ 'persist-credentials': false }) + expect(job.steps.some(step => step.uses?.startsWith('actions/cache/'))).toBe(false) + }) + + it.skipIf(process.platform === 'win32')('configures generated caches before pnpm without changing HOME or global links', () => { + const index = job.steps.findIndex(step => step.name === 'Isolate compatibility caches') + const step = job.steps[index]! + expect(index).toBeGreaterThan(0) + expect(index).toBeLessThan(job.steps.findIndex(candidate => candidate.uses === 'pnpm/action-setup@v4')) + expect(step.if).toBe("runner.environment == 'self-hosted'") + const root = mkdtempSync(join(tmpdir(), 'ci-compatible-selfhosted-')) + try { + const outputs = ['runner-a', 'runner-b'].map((runner) => { + const envFile = join(root, runner + '.env') + const temp = join(root, runner) + const child = spawnSync('bash', ['-e', '-u', '-o', 'pipefail', '-c', step.run!], { + env: { PATH: process.env.PATH, HOME: join(root, 'shared home'), RUNNER_TEMP: temp, GITHUB_ENV: envFile }, + encoding: 'utf8', timeout: 10_000, + }) + expect(child.error).toBeUndefined() + expect(child.signal).toBeNull() + expect(child.status, child.stderr).toBe(0) + const output = readFileSync(envFile, 'utf8') + expect(output).toBe([ + 'NODE_COMPILE_CACHE=' + temp + '/node-compile-cache', + 'npm_config_devdir=' + temp + '/node-gyp', + 'PNPM_CONFIG_STORE_DIR=' + join(root, 'shared home') + '/.local/share/pnpm/store', + '', + ].join('\n')) + return output + }) + expect(outputs[0]).not.toBe(outputs[1]) + } finally { + rmSync(root, { recursive: true, force: true }) + } + }) +})