diff --git a/.agents/notes/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.i18n.yaml index f03f7e81ad..83638635ad 100644 --- a/.agents/notes/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.md -2026-07-10-single-file-executable-sdk-runtime-distribution.md: 8731528b9ae600bb8bfe12738669f3a84c11a06b -2026-07-10-single-file-executable-sdk-runtime-distribution.zh.md: 755b4bd7ddbe9b88b4f40b8a3ae7419f746b8dde +2026-07-10-single-file-executable-sdk-runtime-distribution.md: c152345772826ec4e2dbfd238726c429418c7897 +2026-07-10-single-file-executable-sdk-runtime-distribution.zh.md: ea5e457afd761cb5071f8b584ef10fa7ffaa8210 diff --git a/.agents/notes/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.md b/.agents/notes/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.md index 8731528b9a..c152345772 100644 --- a/.agents/notes/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.md +++ b/.agents/notes/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.md @@ -44,7 +44,7 @@ The deploy root includes `@deepseek-ai/dsh-mcp-client` as an explicitly supporte [`scripts/build-exe-for-python-sdk.ts`](../../../../scripts/build-exe-for-python-sdk.ts): runtime closure verification → `pnpm run build` → (after clearing) `pnpm --filter dsh-python-runtime-closure deploy --legacy --prod --config.node-linker=hoisted --config.auto-install-peers=false --config.link-workspace-packages=true` **directly into** `python/sdk-runtime/src/deepseek_harness_runtime/runtime/node/` → restore direct workspace packages omitted by legacy deploy and reject any remaining manifest gap → replace staged dependency symlinks with their target bytes, remove package-manager `.bin` links, and fail if any symlink remains → inject pkg configuration whose bin is `node_modules/@deepseek-ai/dsh/lib/bin.js` and whose assets cover dynamic profile, bundle, frontend, preset, native-library, and configuration reads → stage the target `node-pty` addon → invoke `pkg --sea` once per target → write `deepseek-harness-sdk-runtime--` under `dist-exe/` and copy it into the runtime directory. Linux CI rebuilds `pty.node` inside the matching manylinux 2.28 container because legacy deploy omits that install side effect. Every target copies its native `@vscode/ripgrep` binary beside the executable as the required `-rg` sidecar; pkg runtimes select that sidecar through `process.pkg`, while ordinary Node execution uses `@vscode/ripgrep` directly. macOS uses its target prebuild and also emits the required `-spawn-helper`. All four deploy flags are grounded in measurement: `--legacy` is the mandatory path with inject-workspace-packages off; hoisted gives pkg a stable single-instance layout that the explicit materialization pass makes symlink-free; disabling automatic peer installation prevents undeclared peers from expanding the closure; link-workspace-packages selects direct workspace dependencies. [`pnpm-workspace.yaml`](../../../../pnpm-workspace.yaml) overrides the transitive `@deepseek-ai/cosmokit` and `@deepseek-ai/schemastery` semver requests to the pinned vendor sources so legacy deploy never resolves those unpublished names from a registry. -CI: [`.github/workflows/build-exe-for-python-sdk.yml`](../../../../.github/workflows/build-exe-for-python-sdk.yml) is called for all four targets by the [installed-wheel Python runtime pull-request validation](../testing/2026-08-23-installed-python-wheel-black-box-ci.md) and the [public publication workflow](../process/2026-08-11-python-publication-workflow.md); `workflow_dispatch` and the `build-exe` label can still select a subset. Native builds run on linux-x64 / linux-arm64 (`ubuntu-24.04-arm`) / macos-arm64 / win-x64 (`windows-2025`), with `~/.pkg-cache` cached where applicable, and pkg handles macOS ad-hoc signing. Each leg installs the release-shaped SDK and runtime wheels into a clean venv outside the checkout, proves their package and executable provenance, then drives the complete keyless scenario set through the public SDK and direct NDJSON JSON-RPC. Trusted pull requests additionally run a real DeepSeek two-turn tool smoke on every target; fork and Dependabot heads receive no key. Linux inspects the executable and native addon's GLIBC requirements and runs an additional manylinux 2.28 smoke, while macOS verifies that the executable's deployment target fits the wheel tag. A full four-target run retains five artifacts, each containing one release file: the platform-independent SDK wheel and four native runtime wheels; a subset dispatch retains the SDK wheel and selected runtime wheels. Bare executables and source bundles remain intermediate test inputs. [`.gitlab-ci.yml`](../../../../.gitlab-ci.yml) accepts `python-v` tag pipelines whose version matches the root `package.json`, builds one SDK wheel and four native runtime wheels, then a single serialized job checks and publishes all five to the project PyPI registry. The [Windows x64 runtime decision](2026-08-23-python-sdk-windows-x64-runtime.md) owns the fourth target and the explicit exclusion of Windows arm64. +CI: [`.github/workflows/build-exe-for-python-sdk.yml`](../../../../.github/workflows/build-exe-for-python-sdk.yml) is called for all four targets by the [installed-wheel Python runtime pull-request validation](../testing/2026-08-23-installed-python-wheel-black-box-ci.md) and the [public publication workflow](../process/2026-08-11-python-publication-workflow.md); `workflow_dispatch` can still select a subset. Native builds run on linux-x64 / linux-arm64 (`ubuntu-24.04-arm`) / macos-arm64 / win-x64 (`windows-2025`), with `~/.pkg-cache` cached where applicable, and pkg handles macOS ad-hoc signing. Each leg installs the release-shaped SDK and runtime wheels into a clean venv outside the checkout, proves their package and executable provenance, then drives the complete keyless scenario set through the public SDK and direct NDJSON JSON-RPC. Trusted pull requests additionally run a real DeepSeek two-turn tool smoke on every target; fork and Dependabot heads receive no key. Linux inspects the executable and native addon's GLIBC requirements and runs an additional manylinux 2.28 smoke, while macOS verifies that the executable's deployment target fits the wheel tag. A full four-target run retains five artifacts, each containing one release file: the platform-independent SDK wheel and four native runtime wheels; a subset dispatch retains the SDK wheel and selected runtime wheels. Bare executables and source bundles remain intermediate test inputs. [`.gitlab-ci.yml`](../../../../.gitlab-ci.yml) accepts `python-v` tag pipelines whose version matches the root `package.json`, builds one SDK wheel and four native runtime wheels, then a single serialized job checks and publishes all five to the project PyPI registry. The [Windows x64 runtime decision](2026-08-23-python-sdk-windows-x64-runtime.md) owns the fourth target and the explicit exclusion of Windows arm64. ### Python SDK distribution: two carriers, exe for production, node for development diff --git a/.agents/notes/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.zh.md b/.agents/notes/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.zh.md index 755b4bd7dd..ea5e457afd 100644 --- a/.agents/notes/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-10-single-file-executable-sdk-runtime-distribution.zh.md @@ -44,7 +44,7 @@ exe 的 VFS 内是**构建产物形态的真实包树**（各包的 `lib/` + 真 [`scripts/build-exe-for-python-sdk.ts`](../../../../scripts/build-exe-for-python-sdk.ts)：运行时闭包校验 → `pnpm run build` →（清空后）`pnpm --filter dsh-python-runtime-closure deploy --legacy --prod --config.node-linker=hoisted --config.auto-install-peers=false --config.link-workspace-packages=true` **直接写入** `python/sdk-runtime/src/deepseek_harness_runtime/runtime/node/` → 恢复 legacy deploy 遗漏的直接工作区包，并拒绝剩余的 manifest 缺口 → 将暂存依赖中的符号链接替换为目标文件内容，删除包管理器的 `.bin` 链接，并在仍有任何符号链接时失败 → 注入 pkg 配置，其中 bin 为 `node_modules/@deepseek-ai/dsh/lib/bin.js`，assets 覆盖动态读取的 profile、bundle、前端、preset、原生库与配置文件 → 暂存目标平台的 `node-pty` addon → 每个构建目标调用一次 `pkg --sea` → 将 `deepseek-harness-sdk-runtime--` 写入 `dist-exe/` 并拷回运行时目录。Linux CI 会在匹配的 manylinux 2.28 容器中重新构建 `pty.node`，因为 legacy deploy 会遗漏这一安装副作用。每个目标都会把对应的原生 `@vscode/ripgrep` 二进制复制到可执行文件旁，作为必需的 `-rg` 伴随文件；pkg 运行时通过 `process.pkg` 选择该伴随文件，普通 Node 执行则直接使用 `@vscode/ripgrep`。macOS 使用对应目标的预构建产物，并额外生成所需的 `-spawn-helper`。四个部署标志都有实测依据：未启用 `inject-workspace-packages` 时必须使用 `--legacy`；`hoisted` 为 pkg 提供稳定的单实例布局，再由显式物化步骤消除符号链接；关闭对等依赖自动安装可防止未声明的对等依赖扩大闭包；`link-workspace-packages` 选择直接工作区依赖。[`pnpm-workspace.yaml`](../../../../pnpm-workspace.yaml) 将传递的 `@deepseek-ai/cosmokit` 与 `@deepseek-ai/schemastery` semver 请求覆盖到固定的 vendor 源码，使 legacy deploy 不会从注册表解析这些未发布名称。 -CI 使用 [`.github/workflows/build-exe-for-python-sdk.yml`](../../../../.github/workflows/build-exe-for-python-sdk.yml)：[安装后 wheel Python 运行时拉取请求验证](../testing/2026-08-23-installed-python-wheel-black-box-ci.zh.md)与[公开发布工作流](../process/2026-08-11-python-publication-workflow.zh.md)都会调用它构建全部四个目标；`workflow_dispatch` 与 `build-exe` 标签仍可选择部分目标。linux-x64、linux-arm64（`ubuntu-24.04-arm`）、macos-arm64 与 win-x64（`windows-2025`）分别进行原生构建，并在适用平台缓存 `~/.pkg-cache`；macOS 的 ad-hoc 签名由 pkg 处理。每个平台都把发布形态的 SDK wheel 包与运行时 wheel 包安装到 checkout 外的干净 venv，证明包与可执行文件来源，再通过公开 SDK 与直接 NDJSON JSON-RPC 运行完整 keyless 场景。可信拉取请求还会在每个目标上运行真实 DeepSeek 双轮工具冒烟测试；fork 与 Dependabot head 不会获得密钥。Linux 会检查可执行文件和原生 addon 各自的 GLIBC 依赖，并额外运行 manylinux 2.28 冒烟测试；macOS 则验证可执行文件的部署目标符合 wheel 包标签。完整构建四个目标时保留 5 个产物，每个产物只含一个发布文件：平台无关的 SDK wheel 包与 4 个原生运行时 wheel 包；手动选择部分目标时保留 SDK wheel 与所选运行时 wheel。裸 exe 与源码包只作为测试中间输入。[`.gitlab-ci.yml`](../../../../.gitlab-ci.yml) 只接受版本与根目录 `package.json` 匹配的 `python-v` 标签流水线，构建一个 SDK wheel 包和 4 个原生运行时 wheel 包，再由单个串行任务校验并将这 5 个文件发布到项目的 PyPI 注册表。[Windows x64 运行时决策](2026-08-23-python-sdk-windows-x64-runtime.zh.md)负责第四个目标及对 Windows arm64 的明确排除。 +CI 使用 [`.github/workflows/build-exe-for-python-sdk.yml`](../../../../.github/workflows/build-exe-for-python-sdk.yml)：[安装后 wheel Python 运行时拉取请求验证](../testing/2026-08-23-installed-python-wheel-black-box-ci.zh.md)与[公开发布工作流](../process/2026-08-11-python-publication-workflow.zh.md)都会调用它构建全部四个目标；`workflow_dispatch` 仍可选择部分目标。linux-x64、linux-arm64（`ubuntu-24.04-arm`）、macos-arm64 与 win-x64（`windows-2025`）分别进行原生构建，并在适用平台缓存 `~/.pkg-cache`；macOS 的 ad-hoc 签名由 pkg 处理。每个平台都把发布形态的 SDK wheel 包与运行时 wheel 包安装到 checkout 外的干净 venv，证明包与可执行文件来源，再通过公开 SDK 与直接 NDJSON JSON-RPC 运行完整 keyless 场景。可信拉取请求还会在每个目标上运行真实 DeepSeek 双轮工具冒烟测试；fork 与 Dependabot head 不会获得密钥。Linux 会检查可执行文件和原生 addon 各自的 GLIBC 依赖，并额外运行 manylinux 2.28 冒烟测试；macOS 则验证可执行文件的部署目标符合 wheel 包标签。完整构建四个目标时保留 5 个产物，每个产物只含一个发布文件：平台无关的 SDK wheel 包与 4 个原生运行时 wheel 包；手动选择部分目标时保留 SDK wheel 与所选运行时 wheel。裸 exe 与源码包只作为测试中间输入。[`.gitlab-ci.yml`](../../../../.gitlab-ci.yml) 只接受版本与根目录 `package.json` 匹配的 `python-v` 标签流水线，构建一个 SDK wheel 包和 4 个原生运行时 wheel 包，再由单个串行任务校验并将这 5 个文件发布到项目的 PyPI 注册表。[Windows x64 运行时决策](2026-08-23-python-sdk-windows-x64-runtime.zh.md)负责第四个目标及对 Windows arm64 的明确排除。 ### Python SDK 分发：双载体，exe 用于生产，`node` 用于开发 diff --git a/.agents/notes/implemented/architecture/2026-07-30-web-config-plane.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-30-web-config-plane.i18n.yaml index a345f69828..6e02fa3915 100644 --- a/.agents/notes/implemented/architecture/2026-07-30-web-config-plane.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-30-web-config-plane.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-30-web-config-plane.md -2026-07-30-web-config-plane.md: d5bd9c05e8352536c5c6f8b265db7dbd56a4fb84 -2026-07-30-web-config-plane.zh.md: 3c7f801766b1a6c197cc208a3c4a030b8aaac771 +2026-07-30-web-config-plane.md: c817071ed17554d06249aa6893ed759bea5d72d0 +2026-07-30-web-config-plane.zh.md: 0b15e329340051d0f63e8a5b1f8c70a29a2138d2 diff --git a/.agents/notes/implemented/architecture/2026-07-30-web-config-plane.md b/.agents/notes/implemented/architecture/2026-07-30-web-config-plane.md index d5bd9c05e8..c817071ed1 100644 --- a/.agents/notes/implemented/architecture/2026-07-30-web-config-plane.md +++ b/.agents/notes/implemented/architecture/2026-07-30-web-config-plane.md @@ -12,7 +12,7 @@ The request-level configuration seam made LLM adapter configuration restart-free ## Decision -**Wire domains on the compiled RPC map, rejections as codes, owner events forwarded verbatim.** `settings.describe/openDocument/update/replace/mutate`, `credentials.describe/set/unset`, `llm.providers`, and `llm.models` join `RpcMethodMap`, so the compiler-locked wiring sites keep schema, handler, and client in lockstep. Seam rejections fold into `settings-rejected {ns}` / `credential-rejected {ref}` business errors, while clients subscribe to forwarded settings, credentials, and LLM owner events and converge without polling ([forwarded Remote events](2026-08-10-remote-event-delivery.md)). Connection authenticates settings reads, native actions, writes, `pickDirectory`, `openPath`, and every other Host operation with one browser session; Host/Origin failures still return 403 before identity is checked. +**Configuration calls use their owning wire implementation, rejections as codes, and owner events forwarded verbatim.** `@deepseek-ai/dsh-api-settings-controller` owns generated Remote methods for `settings/describe`, `settings/update`, `settings/replace`, `settings/mutate`, and `credentials/describe|set|unset`; `settings.openDocument` and the `llm.*` methods remain in `RpcMethodMap`. Provider absence retains the configuration API's actionable `internal` diagnostic, while seam rejections retain `settings-rejected {ns}` / `settings-conflict {ns, expected, actual}` / `credential-rejected {ref}`. Clients subscribe to forwarded settings, credentials, and LLM owner events and converge without polling ([forwarded Remote events](2026-08-10-remote-event-delivery.md)). Connection authenticates generated Remote methods and API Proxy fallbacks with the same browser session; Host/Origin failures still return 403 before identity is checked. **`describe()` grows layers and structural secret redaction.** `SettingsDescriptor` carries `base`/`user` beside the effective value, so the form marks "overridden" by presence in the user layer, not value inequality (an override *equal* to the base is still an override). `describe({ redactSecrets: true })` — mandatory at every wire face — strips `role('secret')` subtrees from all three layers via a pure structural walk of the schema (object/dict/array containers; a secret-role subtree is one opaque leaf) and enumerates the stripped slots as `{path, set}`, so a page can render write-only inputs without ever receiving a value. diff --git a/.agents/notes/implemented/architecture/2026-07-30-web-config-plane.zh.md b/.agents/notes/implemented/architecture/2026-07-30-web-config-plane.zh.md index 3c7f801766..0b15e32934 100644 --- a/.agents/notes/implemented/architecture/2026-07-30-web-config-plane.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-30-web-config-plane.zh.md @@ -12,7 +12,7 @@ Status: implemented ## 决策 -**wire 领域挂上编译期 RPC 映射，拒绝落为错误码，owner 事件原样转发。**`settings.describe/openDocument/update/replace/mutate`、`credentials.describe/set/unset`、`llm.providers` 与 `llm.models` 一同加入 `RpcMethodMap`，由编译器锁定的接线位点让 schema、处理器与客户端保持步调一致。seam 侧拒绝折叠为业务错误，客户端则订阅转发的 settings、credentials 与 LLM owner 事件，无需轮询即可收敛（见[转发的 Remote 事件](2026-08-10-remote-event-delivery.zh.md)）。Connection 用一个浏览器会话认证 settings 读取、原生操作、写入、`pickDirectory`、`openPath` 与其他所有 Host 操作；Host/Origin 失败仍会在身份校验前返回 403。 +**配置调用使用其所属的 wire 实现，拒绝落为错误码，owner 事件原样转发。**`@deepseek-ai/dsh-api-settings-controller` 持有 `settings/describe`、`settings/update`、`settings/replace`、`settings/mutate` 与 `credentials/describe|set|unset` 的生成 Remote 方法；`settings.openDocument` 和 `llm.*` 方法仍位于 `RpcMethodMap`。provider 缺失时保留配置 API 可操作的 `internal` 诊断，seam 拒绝则保留 `settings-rejected {ns}`／`settings-conflict {ns, expected, actual}`／`credential-rejected {ref}`。Client 订阅转发的 settings、credentials 与 LLM owner 事件，无需轮询即可收敛（见[转发的 Remote 事件](2026-08-10-remote-event-delivery.zh.md)）。Connection 使用同一个浏览器会话认证生成的 Remote 方法与 API Proxy 回退；Host/Origin 失败仍会在身份校验前返回 403。 **`describe()` 增加分层与结构化 secret 脱敏。**`SettingsDescriptor` 在生效值之外携带 `base`/`user`，表单据此按「字段是否出现在用户层」来标记「已覆盖」，而非按值是否不等（与 base *相等*的覆盖仍然是覆盖）。`describe({ redactSecrets: true })`——在每个 wire 面都强制启用——经由对 schema 的纯结构遍历（object/dict/array 容器；secret 角色子树整体是一个不透明叶节点）从全部三层剥除 `role('secret')` 子树，并把剥除的槽位枚举为 `{path, set}`，页面因此不必收到任何值就能渲染只写输入框。 diff --git a/.agents/notes/implemented/architecture/2026-08-25-persistence-latency-and-page-size.i18n.yaml b/.agents/notes/implemented/architecture/2026-08-25-persistence-latency-and-page-size.i18n.yaml new file mode 100644 index 0000000000..abc20bb19c --- /dev/null +++ b/.agents/notes/implemented/architecture/2026-08-25-persistence-latency-and-page-size.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-08-25-persistence-latency-and-page-size.md +2026-08-25-persistence-latency-and-page-size.md: 27eb58cc551f01c48361a3af3224eb8b12592a00 +2026-08-25-persistence-latency-and-page-size.zh.md: 24ab1835cc313cd617d665a0c52a399d505069ea diff --git a/.agents/notes/implemented/architecture/2026-08-25-persistence-latency-and-page-size.md b/.agents/notes/implemented/architecture/2026-08-25-persistence-latency-and-page-size.md new file mode 100644 index 0000000000..27eb58cc55 --- /dev/null +++ b/.agents/notes/implemented/architecture/2026-08-25-persistence-latency-and-page-size.md @@ -0,0 +1,70 @@ +# Agent Note: Persistence compression latency and SQLite page size + +Status: implemented + +English | [中文](2026-08-25-persistence-latency-and-page-size.zh.md) + +## Problem + +The physical persistence optimizations need to reduce retained storage without moving disproportionate work into full writes, reads, or session forks. The original 105-session corpus showed that JSONL level-19 compression made full writes and forks more than twice as slow. The earlier SQLite page-size experiment predated shared-dictionary row compression and showed negligible savings, so it did not establish the best page size for the current row distribution. + +The decision needs evidence from more varied sessions, including long event streams and payloads outside the original corpus. The expanded corpus contains 501 real sessions, 16,153,332 logical events, and 2,002,145,570 bytes of serialized event data. + +## Decision + +### Storage encoding stays physical and independently decodable + +JSONL stores strictly increasing `sourceEventSeqs` as mixed scalar values and inclusive ranges; other orders remain verbatim. SQLite stores the same arrays as tagged zigzag-delta or `(start, count)` varints, choosing the smaller encoding. Both readers restore the original `number[]` before exposing an event. + +SQLite uses an internal integer `sessions.id` and keeps the public session id once in `sessions.session_key`, so event rows and their primary key do not repeat a text identifier. Each `events.data` value remains independently decodable: the writer tries level-3 Zstandard with the packaged 64 KiB raw-content dictionary and retains SQLite text when compression is not smaller. The dictionary bytes are part of schema 19 and a test pins their SHA-256 digest; replacing them requires another schema-version bump. + +### JSONL uses the standard Zstandard level + +The JSONL writer keeps one checksummed Zstandard frame per durable append batch but uses the compressor's standard level. Lossless `sourceEventSeqs` range encoding remains active. Frames stay independently decodable for suffix reads and torn-tail recovery; only the expensive level-19 search is removed. + +### New SQLite databases use 64 KiB pages + +The SQLite provider sets `page_size=65536` before initializing a pristine schema-19 database. An established schema-19 database retains its current page size because SQLite ignores the pragma after allocation. + +The page size is part of schema 19's fixed physical layout and is applied through the package's closed SQL resources like the other fixed SQLite pragmas. + +### Expanded benchmark + +Each candidate was rebuilt five times from the same 501-session corpus with 512-event append batches. Their order rotates between rounds so every candidate occupies each run position once. Each build runs three complete and suffix-read sweeps. For each displayed metric, the highest and lowest build are discarded and the remaining three values are averaged. Complete and suffix read times cover one sweep over all sessions, and fork time covers all 501 sessions. + +| Backend | Stored size | Full write | Full read | Suffix read | Fork | +| --- | ---: | ---: | ---: | ---: | ---: | +| JSONL `master` | 172.43 MB | 200.902 s | 8.033 s | 24.479 s | 72.670 s | +| JSONL with provenance ranges | 148.15 MB (-14.1%) | 197.281 s (-1.8%) | 7.799 s (-2.9%) | 24.582 s (+0.4%) | 72.308 s (-0.5%) | +| JSONL with provenance ranges and level 19 | 130.22 MB (-24.5%) | 329.442 s (+64.0%) | 7.764 s (-3.3%) | 24.454 s (-0.1%) | 166.177 s (+128.7%) | +| SQLite `master` (schema 17) | 438.31 MB | 69.632 s | 8.211 s | 0.546 s | 64.290 s | +| SQLite with all physical optimizations and 64 KiB pages | 233.18 MB (-46.8%) | 87.656 s (+25.9%) | 9.155 s (+11.5%) | 0.575 s (+5.3%) | 79.417 s (+23.5%) | + +Relative to standard-level frames with provenance ranges, level 19 saves another 12.1% of the JSONL bytes but increases full-write time by 67.0% and fork time by 129.8%. Its complete and suffix reads change by -0.4% and -0.5%. The extra search therefore benefits retained size without improving the latency-sensitive operations enough to offset its repeated encoding cost. + +An otherwise identical SQLite build isolates the page-size effect: 4 KiB pages use 256.97 MB and 64 KiB pages use 233.18 MB (-9.26%). The `events` table's unused page bytes fall from 30.25 MB to 6.95 MB, while the index changes from 5.92 MB to 6.03 MB. In the paired run, full write, full read, and suffix read change by -0.5%, -0.4%, and -3.8%; fork changes by -14.8%. The space gain therefore comes from better large-row page utilization rather than a smaller index or omitted data, without a measured latency regression. + +## Alternatives considered + +**Keep JSONL level 19.** Rejected. On the expanded corpus it saves another 12.1% relative to default-level frames but increases full-write time by 67.0% and fork time by 129.8%, while complete and suffix reads differ by less than 1%. Default-level frames plus provenance ranges retain a 14.1% size reduction relative to master without a material latency regression. + +**Compress one whole JSONL log as a single frame.** Rejected. It improves cross-batch compression but makes suffix reads decompress from the start and removes batch-local torn-tail recovery. + +**Keep 4 KiB SQLite pages.** Rejected for pristine databases. The current compressed-row distribution retains 9.26% more bytes because large compressed records leave more unusable space across 4 KiB B-tree pages. Existing databases keep their page size to avoid a historical rewrite. + +**Remove ROWID from `events`.** Rejected. The composite primary key becomes the table B-tree key and repeats through internal pages; the 105-session comparison produced a larger database than ordinary ROWID tables. + +**Deduplicate event content.** Rejected. Message restatements and tool arguments can be reconstructed only under assumptions that compaction, retries, and pruning may invalidate. Physical compression preserves every event without adding reconstruction semantics. + +**Use per-session SQLite files or DuckDB.** Rejected for the hot store. Per-session files lose cross-session queries, while DuckDB's OLAP write model fits cold batch analysis rather than durable append batches and low-latency suffix reads. + +## Consequences + +JSONL keeps the low-cost provenance optimization without the level-19 write and fork penalty. SQLite exchanges approximately 5–26% more time across the measured operations for a 46.8% retained-size reduction; its full write remains materially faster than JSONL, and its suffix read remains much faster. Its complete read and fork are slightly slower than default-level JSONL on this expanded corpus. + +New SQLite databases use 64 KiB WAL frames and cache pages. Small databases may reserve more bytes for sparsely populated schema and metadata pages, while the measured multi-session workload gains substantially better `events` page utilization. Schema 19 rejects every other schema version rather than migrating it. + +## Related + +- [sqlite-physical-chunk-row-compression](2026-08-18-sqlite-physical-chunk-row-compression.md) — owns the packed row model; its earlier page-size conclusion applies to the pre-dictionary layout. +- [zstandard-jsonl-session-logs](2026-07-19-zstandard-jsonl-session-logs.md) — owns the checksummed frame-per-batch container and the standard compressor-level policy restored here. diff --git a/.agents/notes/implemented/architecture/2026-08-25-persistence-latency-and-page-size.zh.md b/.agents/notes/implemented/architecture/2026-08-25-persistence-latency-and-page-size.zh.md new file mode 100644 index 0000000000..24ab1835cc --- /dev/null +++ b/.agents/notes/implemented/architecture/2026-08-25-persistence-latency-and-page-size.zh.md @@ -0,0 +1,70 @@ +# Agent Note: 持久化压缩延迟与 SQLite page size + +Status: implemented + +[English](2026-08-25-persistence-latency-and-page-size.md) | 中文 + +## 问题 + +物理持久化优化需要减少保留存储，同时不能把不成比例的工作转移到完整写入、读取或会话 fork。原有的 105 会话语料显示，JSONL level-19 压缩会让完整写入与 fork 耗时增加一倍以上。此前的 SQLite page-size 实验早于共享字典行压缩，所得空间收益可以忽略，因此无法确定当前行分布的最佳 page size。 + +该决策需要来自更多样会话的证据，包括长事件流与原语料之外的 payload。扩展后的语料包含 501 个真实会话、16,153,332 个逻辑事件与 2,002,145,570 字节序列化事件数据。 + +## 决策 + +### 存储编码保持为物理层行为并可独立解码 + +JSONL 把严格递增的 `sourceEventSeqs` 存为标量值与闭区间的混合数组，其他顺序保持原样。SQLite 把同一数组存为带 tag 的 zigzag-delta 或 `(start, count)` varint，并选择更小的编码。两个读取方都会在暴露事件前还原原始 `number[]`。 + +SQLite 使用内部整数 `sessions.id`，并只在 `sessions.session_key` 中保留一次公开会话 id，使事件行及其主键不再重复文本标识。每个 `events.data` 值仍可独立解码：写入方尝试用打包的 64 KiB raw-content 字典执行 level-3 Zstandard 压缩，结果不更小时保留 SQLite 文本。字典字节属于 schema 19，测试固定其 SHA-256 摘要；替换字典需要再次提升 schema 版本。 + +### JSONL 使用 Zstandard 标准级别 + +JSONL 写入方继续为每个持久 append 批次写入一个带 checksum 的 Zstandard frame，但使用压缩器的标准级别。无损 `sourceEventSeqs` 区间编码继续生效。各 frame 仍可独立解码，以支持后缀读取与撕裂尾部恢复；只移除昂贵的 level-19 搜索。 + +### 新建 SQLite 数据库使用 64 KiB page + +SQLite 提供方在初始化全新 schema-19 数据库前设置 `page_size=65536`。SQLite 在 page 已分配后会忽略该 pragma，因此已有 schema-19 数据库保留其当前 page size。 + +Page size 属于 schema 19 的固定物理布局，并与其他固定 SQLite pragma 一样通过包内封闭的 SQL 资源应用。 + +### 扩展基准 + +每个候选方案都从同一份 501 会话语料独立重建五次，每个 append 批次包含 512 个事件。各轮轮换执行顺序，使每个候选方案在每个运行位置各出现一次。每次重建执行三轮完整读取与后缀读取。下表中的每项指标都去掉最高与最低的一次重建，再平均其余三次。完整读取与后缀读取耗时覆盖对全部会话的一轮扫描，fork 耗时覆盖全部 501 个会话。 + +| 后端 | 存储大小 | 完整写入 | 完整读取 | 后缀读取 | Fork | +| --- | ---: | ---: | ---: | ---: | ---: | +| JSONL `master` | 172.43 MB | 200.902 s | 8.033 s | 24.479 s | 72.670 s | +| JSONL + 来源区间 | 148.15 MB (-14.1%) | 197.281 s (-1.8%) | 7.799 s (-2.9%) | 24.582 s (+0.4%) | 72.308 s (-0.5%) | +| JSONL + 来源区间 + level 19 | 130.22 MB (-24.5%) | 329.442 s (+64.0%) | 7.764 s (-3.3%) | 24.454 s (-0.1%) | 166.177 s (+128.7%) | +| SQLite `master`（schema 17） | 438.31 MB | 69.632 s | 8.211 s | 0.546 s | 64.290 s | +| SQLite + 全部物理优化 + 64 KiB page | 233.18 MB (-46.8%) | 87.656 s (+25.9%) | 9.155 s (+11.5%) | 0.575 s (+5.3%) | 79.417 s (+23.5%) | + +相对使用来源区间的标准级别 frame，level 19 可再减少 12.1% 的 JSONL 字节，但会让完整写入增加 67.0%、fork 增加 129.8%；完整读取与后缀读取分别变化 -0.4% 与 -0.5%。因此，更深入的搜索只改善保留体积，无法通过延迟敏感操作的收益抵消反复付出的编码成本。 + +其余条件相同的 SQLite 重建可单独观察 page-size 影响：4 KiB page 使用 256.97 MB，64 KiB page 使用 233.18 MB（-9.26%）。`events` 表的 page 内未使用字节从 30.25 MB 降至 6.95 MB，索引则从 5.92 MB 变为 6.03 MB。在该成对运行中，完整写入、完整读取与后缀读取分别变化 -0.5%、-0.4% 与 -3.8%，fork 变化 -14.8%。因此，空间收益来自更高的大记录 page 利用率，而不是索引缩小或数据省略，并且没有测得延迟退化。 + +## 考虑过的替代方案 + +**保留 JSONL level 19。** 不予采用。在扩展语料上，它相对默认级别 frame 可再减少 12.1%，却让完整写入增加 67.0%、fork 增加 129.8%，而完整读取与后缀读取的差异都不足 1%。默认级别 frame 配合来源区间后，相对 master 仍能缩小 14.1%，且没有实质性延迟退化。 + +**把整份 JSONL 日志压成单个 frame。** 不予采用。该方案可改善跨批次压缩，但后缀读取必须从头解压，也会失去按批次恢复撕裂尾部的能力。 + +**新建 SQLite 数据库继续使用 4 KiB page。** 不予采用。当前压缩行分布会在 4 KiB B-tree page 之间留下更多不可用空间，使保留字节增加 9.26%。已有数据库保留其 page size，避免改写历史数据。 + +**从 `events` 移除 ROWID。** 不予采用。复合主键会成为表 B-tree 键并在内部 page 中重复；105 会话对比所得数据库大于使用普通 ROWID 的表。 + +**对事件内容去重。** 不予采用。消息复述与工具参数只能在依赖重建假设时删除，而 compaction、重试和修剪可能让这些假设失效。物理压缩保留每个事件，不增加重建语义。 + +**使用逐会话 SQLite 文件或 DuckDB。** 不用于热存储。逐会话文件会失去跨会话查询，DuckDB 的 OLAP 写入模型则更适合冷批量分析，而不是持久 append 批次与低延迟后缀读取。 + +## 后果 + +JSONL 保留低成本来源优化，同时避开 level-19 的写入与 fork 代价。SQLite 以实测各项操作约 5–26% 的额外耗时换取 46.8% 的保留体积缩减；其完整写入仍明显快于 JSONL，后缀读取也仍快得多。在这份扩展语料上，完整读取与 fork 略慢于默认级别 JSONL。 + +新建 SQLite 数据库使用 64 KiB WAL frame 与 cache page。小型数据库可能为稀疏的 schema 与元数据 page 预留更多字节，而实测的多会话工作负载显著改善了 `events` page 利用率。Schema 19 会拒绝其他所有 schema 版本，而不是迁移它们。 + +## 相关资料 + +- [sqlite-physical-chunk-row-compression](2026-08-18-sqlite-physical-chunk-row-compression.zh.md) — 定义打包行模型；其此前的 page-size 结论适用于共享字典之前的布局。 +- [zstandard-jsonl-session-logs](2026-07-19-zstandard-jsonl-session-logs.zh.md) — 定义带 checksum 的按批次 frame 容器，以及本笔记恢复的标准压缩级别策略。 diff --git a/.agents/notes/implemented/architecture/2026-08-26-models-page-extension-slots.i18n.yaml b/.agents/notes/implemented/architecture/2026-08-26-models-page-extension-slots.i18n.yaml new file mode 100644 index 0000000000..e6e2dae0a2 --- /dev/null +++ b/.agents/notes/implemented/architecture/2026-08-26-models-page-extension-slots.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-08-26-models-page-extension-slots.md +2026-08-26-models-page-extension-slots.md: 9983c669b7838de00d49a634a54e5d115ef4945a +2026-08-26-models-page-extension-slots.zh.md: 639654ea4b912d43a95f0a9295969523f4e1c5f8 diff --git a/.agents/notes/implemented/architecture/2026-08-26-models-page-extension-slots.md b/.agents/notes/implemented/architecture/2026-08-26-models-page-extension-slots.md new file mode 100644 index 0000000000..9983c669b7 --- /dev/null +++ b/.agents/notes/implemented/architecture/2026-08-26-models-page-extension-slots.md @@ -0,0 +1,33 @@ +# Agent Note: Models-page extension slots + +Status: implemented + +English | [中文](2026-08-26-models-page-extension-slots.zh.md) + +## Problem + +Provider sign-in for the pi-ai catalog (GitHub Copilot, OpenAI accounts) is moving out of the product into an optional out-of-tree plugin for provider terms-of-service reasons. The plugin needs its sign-in button and attempt UI inside the Models page's provider cards — the surface where a user meets a provider — but `ui-settings-models` rendered its cards from closed code: the only integration path was editing this package, which an external plugin cannot do, and the page's one open seam (`settings.section`) can only add a whole separate page. + +## Decision + +`ui-settings-models` declares two SlotMap seats in `src/client/slot-contract.ts`, claims them as `children` of its `settings.section` registration, and re-exports their types from `./client` so an out-of-tree plugin can merge them with a type-only import. + +`settings.models.provider-card` is `keyed` with `entryKey = ConfigurableProviderView.settingsNs`: one registration under an adapter family's settings namespace receives every card of that family — shipped catalog routes, adopted directory rows, and hand-declared routes alike — while the section never interprets the key. The key domain stays the open string space (no `keyProps` table) because hand-declared route ids are user-chosen at runtime. The seat dispatches on every card that shows a directory row: a saved row's card, its first-run setup posture, and the add-provider draft (its dormant row, `configured: false` in practice), which is where sign-in matters most — the user has just met the provider and holds no key. The hand-declared draft card has no directory row before saving and dispatches nothing. Owner props carry the row's `ConfigurableProviderView`, its `configured` join, and its confirmed api-key credential state (`keyConfigured`, which the first consumer uses to withhold sign-in beside a stored key); nothing more has a current consumer. + +`settings.models.footer` is a `list` seat after the rows and the add controls, for section-level extension content such as orphaned-record management. + +Without registrants both seats render nothing, so the shipped page is pixel-identical to before. + +## Alternatives considered + +**A `list` seat with self-filtering registrants instead of keying.** Every registrant would render (and return null) on every card, and two plugins could silently interleave UI inside one family's cards. Keying by namespace gives one accountable extension owner per adapter family and zero wasted dispatches, and reuses the exact pairing rationale of `settings.plugin.item`. + +**Keying by provider route id.** Route ids are dynamic — hand-declared routes are named by users at runtime — so a plugin could not register ahead of the rows it wants and would have to churn registrations as the directory changes. + +**A `chain` seat replacing the whole card body.** No current consumer needs to replace the editor; the sign-in surface is additive. A takeover contract would also make the section's layout a compatibility surface. A chain can still be added later without disturbing these seats. + +**Keeping the sign-in UI wired inside `ui-settings-models` (the pre-plugin design).** Ships the terms-of-service-sensitive surface in the product, which is the outcome this extension point exists to avoid. + +## Consequences + +An out-of-tree plugin can now integrate per-family card UI into the Models page with no product edits; `llm-pi-ai-oauth` is the first consumer. The cost is a public contract: `ProviderCardExtrasOwnerProps` exposes `ConfigurableProviderView` at the `./client` boundary, and the dispatch sites (saved card, setup posture, add draft, footer) become behavior extensions rely on. Per adapter family the keyed cell renders one owner at a time: a second registration under the same namespace at the same priority is refused by the registry, while a different priority deliberately shadows it (the lowest-priority entry renders) — the slot kit's standard override channel, never a silent merge. diff --git a/.agents/notes/implemented/architecture/2026-08-26-models-page-extension-slots.zh.md b/.agents/notes/implemented/architecture/2026-08-26-models-page-extension-slots.zh.md new file mode 100644 index 0000000000..639654ea4b --- /dev/null +++ b/.agents/notes/implemented/architecture/2026-08-26-models-page-extension-slots.zh.md @@ -0,0 +1,33 @@ +# Agent Note: Models 页扩展插槽 + +Status: implemented + +[English](2026-08-26-models-page-extension-slots.md) | 中文 + +## Problem + +出于提供方服务条款的考虑，pi-ai catalog 的提供方登录（GitHub Copilot、OpenAI 账号）正从产品中移出，改由一个可选的仓库外插件承担。该插件需要把登录按钮与登录过程 UI 放进 Models 页的提供方卡片——用户与提供方相遇的界面——但 `ui-settings-models` 的卡片由封闭代码渲染：唯一的集成途径是修改本包，外部插件做不到；页面仅有的开放缝隙（`settings.section`）只能新增一整个独立页面。 + +## Decision + +`ui-settings-models` 在 `src/client/slot-contract.ts` 声明两个 SlotMap 席位，在其 `settings.section` 注册中以 `children` 认领它们，并从 `./client` 再导出其类型，使仓库外插件通过 type-only import 即可获得类型合并。 + +`settings.models.provider-card` 为 `keyed`，`entryKey = ConfigurableProviderView.settingsNs`：以某适配器家族的 settings namespace 注册一次，即可收到该家族的全部卡片——内置 catalog 路由、从目录采纳的行、手工声明的路由一视同仁——而分区从不解释这个 key。键域保持开放字符串空间（不设 `keyProps` 表），因为手工声明的路由 id 由用户在运行时命名。该席位在每张展示目录行的卡片上分发：已保存行的卡片、其首次运行 setup 形态、以及「添加提供方」草稿卡（其休眠行，实际为 `configured: false`）——草稿卡正是登录价值最大的时刻：用户刚遇到该提供方、手中还没有密钥。手工声明的草稿卡在保存前没有目录行，不分发。Owner props 携带该行的 `ConfigurableProviderView`、其 `configured` 合并结果与已确认的 api-key 凭据状态（`keyConfigured`，首个消费者用它在已存密钥旁抑制登录入口）；更多字段没有现役消费者。 + +`settings.models.footer` 为 `list` 席位，位于行列表与新增控件之后，承载孤儿记录管理这类分区级扩展内容。 + +没有注册方时两个席位均不渲染，产品页面与之前逐像素一致。 + +## Alternatives considered + +**用 `list` 席位、由注册方自行筛选，替代按键分发。**每个注册方都会在每张卡片上渲染（再返回 null），且两个插件可能在同一家族的卡片里静默交错 UI。按 namespace 分发让每个适配器家族有唯一可问责的扩展所有者、零浪费分发，并完全复用 `settings.plugin.item` 的配对理由。 + +**按提供方路由 id 分发。**路由 id 是动态的——手工声明的路由由用户在运行时命名——插件无法先于目标行注册，还得随目录变化反复重注册。 + +**用 `chain` 席位整体接管卡片。**没有现役消费者需要替换编辑器；登录界面是加法。接管契约还会让分区布局成为兼容面。将来仍可在不动这两个席位的前提下追加 chain。 + +**把登录 UI 继续织在 `ui-settings-models` 里（插件化之前的设计）。**会把服务条款敏感的界面随产品发布，而这正是本扩展点要避免的结果。 + +## Consequences + +仓库外插件现在无需改动产品即可把按家族的卡片 UI 集成进 Models 页；`llm-pi-ai-oauth` 是首个消费者。代价是一份公开契约：`ProviderCardExtrasOwnerProps` 在 `./client` 边界暴露 `ConfigurableProviderView`，且各分发位点（已保存卡片、setup 形态、新增草稿、footer）成为扩展方依赖的行为。每个适配器家族的 keyed 单元格同一时刻只渲染一个所有者：同一 namespace 下同 priority 的第二次注册会被注册表拒绝，不同 priority 则是刻意的遮蔽（最低 priority 的条目渲染）——这是 slot 套件的标准覆盖通道，绝非静默合并。 diff --git a/.agents/notes/implemented/bug-fix/2026-08-26-composer-gesture-editable-gate.i18n.yaml b/.agents/notes/implemented/bug-fix/2026-08-26-composer-gesture-editable-gate.i18n.yaml new file mode 100644 index 0000000000..2b3943b416 --- /dev/null +++ b/.agents/notes/implemented/bug-fix/2026-08-26-composer-gesture-editable-gate.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write .agents/notes/implemented/bug-fix/2026-08-26-composer-gesture-editable-gate.md +2026-08-26-composer-gesture-editable-gate.md: 9b01a79d8d8a546641eaf87040d7a19fc5b4d5bc +2026-08-26-composer-gesture-editable-gate.zh.md: b5dc8d36c7b195a152d32e92fac7ae36eea880d9 diff --git a/.agents/notes/implemented/bug-fix/2026-08-26-composer-gesture-editable-gate.md b/.agents/notes/implemented/bug-fix/2026-08-26-composer-gesture-editable-gate.md new file mode 100644 index 0000000000..9b01a79d8d --- /dev/null +++ b/.agents/notes/implemented/bug-fix/2026-08-26-composer-gesture-editable-gate.md @@ -0,0 +1,25 @@ +# Agent Note: Composer e2e gestures gate on the contenteditable attribute + +Status: implemented + +English | [中文](2026-08-26-composer-gesture-editable-gate.zh.md) + +## Problem + +Two Playwright gesture semantics silently changed when the composer became a Lexical `contenteditable` `
`, and both bit only under CI load. While the input machine is adjudicating or submitting a send — and in every locked state — the composer renders read-only by flipping `contenteditable` to `"false"` on the same element. On that element `fill()` throws immediately (`Element is not an ,