diff --git a/.agents/notes/implemented/architecture/2026-06-14-session-persistence.i18n.yaml b/.agents/notes/implemented/architecture/2026-06-14-session-persistence.i18n.yaml index 3bdde9ece6..d664537d10 100644 --- a/.agents/notes/implemented/architecture/2026-06-14-session-persistence.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-06-14-session-persistence.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-06-14-session-persistence.md -2026-06-14-session-persistence.md: 50ec79de83f0cef4a3ec94b689cc25937e334016 -2026-06-14-session-persistence.zh.md: 7b66aed6f077ac484802cfa1e23e1ba7ac3ae985 +2026-06-14-session-persistence.md: 989beb6f8cc65c8d033206a61b4408f3aecdbbc7 +2026-06-14-session-persistence.zh.md: 9dcdbffc7df89ce6fcd5341ef77f99e47643aa92 diff --git a/.agents/notes/implemented/architecture/2026-06-14-session-persistence.md b/.agents/notes/implemented/architecture/2026-06-14-session-persistence.md index 50ec79de83..989beb6f8c 100644 --- a/.agents/notes/implemented/architecture/2026-06-14-session-persistence.md +++ b/.agents/notes/implemented/architecture/2026-06-14-session-persistence.md @@ -20,7 +20,7 @@ Persistence is a **capability seam** with an abstract Service Definition ([capab Key durable, contested choices: - **The canonical durable log persists every `SessionEvent` losslessly, including `assistant/chunk`.** JSONL storage may encode a consecutive delta run as one packed row, but logical readers reconstruct the exact event boundaries, sequence numbers, and timestamps. `deriveMessages()` skips chunks, and a chunk-filtered rollout (Codex's `policy.rs`) is tempting — but `seq = log.length` and validation of `events[i].seq === i` require a *contiguous* logical log; filtering chunks out would leave holes and break both the contract and resume. A chunk-filtered projection is possible later as a derived view with its own renumbering, but it is NOT the canonical log. -- **Append-only; a crashed turn is closed, never truncated.** Flushed events are never rewritten. The [semantic checkpoint policy](../bug-fix/2026-07-21-semantic-session-checkpoints.md) drains the request before model dispatch, a recorded top-level call before tool dispatch, and the complete response/result batch after a step; the loop drains the final turn boundary. Because one interrupted turn may contain substantial valid work, cold inspection preserves its contiguous, parseable events and adds risk-classified error results for unanswered assistant calls, a missing `step/end`, and `turn/end` with `{ kind: 'interrupted' }` to the in-memory logical view. `prepare` or `load` commits those closers before returning a recoverable view; the synthetic results keep resumed provider transcripts valid. Only an incomplete final record is discarded during committed repair; a parse error or sequence gap at or before the last real `turn/end` is corruption and makes the session unloadable. +- **Ordinary writes append; a crashed turn is closed, never truncated.** Flushed current-generation events are never rewritten by normal persistence. A format migration leaves the exact physical source path, bytes, and inode unchanged, then publishes one re-encoded current successor at a previously absent canonical versioned filename after only edge-owned normalization and current crash repair. The [semantic checkpoint policy](../bug-fix/2026-07-21-semantic-session-checkpoints.md) drains the request before model dispatch, a recorded top-level call before tool dispatch, and the complete response/result batch after a step; the loop drains the final turn boundary. Because one interrupted turn may contain substantial valid work, cold inspection preserves its contiguous, parseable events and adds risk-classified error results for unanswered assistant calls, a missing `step/end`, and `turn/end` with `{ kind: 'interrupted' }` to the in-memory logical view. `prepare` or `load` commits those closers before returning a recoverable view; the synthetic results keep resumed provider transcripts valid. Only an incomplete final record is discarded during committed repair; a parse error or sequence gap at or before the last real `turn/end` is corruption and makes the session unloadable. - **The file backend is canonical while the service remains extensible.** `dsh-session-persistence-jsonl` is the sole first-party provider and passes `runPersistenceContract`; the abstract service and coordinator remain available to out-of-tree providers. The [JSONL-only persistence decision](../simplification/2026-08-30-jsonl-only-session-persistence.md) owns removal of the first-party database provider and its deliberate compatibility cut. - **Metadata is out-of-log.** Format version, cwd, and lineage are storage concerns, not replayable conversation state, so they live in a `SessionHeader` owned by `dsh-session` and attached to a `Session` via a new readonly `session.header` — never in `SessionEventMap`, never reaching `deriveMessages()`. `createdAt` is non-negative safe-integer Unix epoch milliseconds: live creation and persistence registration reject fractional values, and JSONL validates the decoded header. The alternative (a merge-extensible `session/meta` event as log line 0) was rejected: an in-log event would ride along with a seeded/forked session for free, but metadata is not replayable state, so the explicit out-of-log header boundary is the cleaner cost. (The header was originally split into an immutable `SessionHeader` plus a mutable `SessionSummary` whose union was `SessionMeta`; the mutable summary was later removed as dead state — see [Drop the mutable session summary](../simplification/2026-06-19-drop-mutable-session-summary.md).) - **`ctx.agents.create()` and `ctx.agents.resume()` are async factories; resume additionally crosses the persistence boundary.** `ctx.agents.resume({ resumeSessionId })` obtains the exact unpublished Session through `ctx.sessionPersistence.prepare()`, publishes it under the persisted id, and continues its projections. The [Session preparation decision](2026-08-05-session-preparation.md) owns reuse between history inspection and resume. The agent-loop does NOT hard-inject `sessionPersistence` (that would pend non-persistent demos forever); `resume` rejects with a clear error when it is absent. @@ -29,7 +29,7 @@ Key durable, contested choices: Each key choice above records its rejected alternative where the choice is stated: a **chunk-filtered canonical log** (Codex's `policy.rs` shape) — breaks the contiguous-seq contract; **truncating a crashed turn** — silently destroys a long autonomous run's real work; an **in-log `session/meta` event as log line 0** — metadata is not replayable state; **finite fractional `createdAt` values** — have no producer and diverge from integer Unix-millisecond storage; **hard-injecting `sessionPersistence` into the loop** — would pend non-persistent demos forever. -Format versioning: the header carries a `version`; cold reads reject any non-current version. The pre-release session format stays pinned at `SESSION_FORMAT_VERSION = 0` and carries no broad compatibility promise, while the coordinator may own an explicit narrow import upgrade when persisted user data requires it ([pre-identity message recovery](../bug-fix/2026-07-28-load-pre-identity-session-messages.md)). Append-only + flush is robust to partial trailing writes tolerated during cold preparation; a future provider or write-ahead log needs its own power-loss and recovery contract. +Format versioning: the header carries a `version`; current Session and coordinator code accept only `SESSION_FORMAT_VERSION = 1`. JSONL event-body reads compose the static adjacent migration chain before constructing a Session, and the v0-to-v1 edge owns the former narrow import upgrades such as [pre-identity message recovery](../bug-fix/2026-07-28-load-pre-identity-session-messages.md). V0 remains at suffixless `session.jsonl[.zstd]`, while v1 and later use immutable lowercase `session.vN.jsonl[.zstd]` names ([released Session migration](2026-08-31-released-session-format-migrations.md)). Current-generation append and flush are robust to partial trailing writes tolerated during cold preparation; a future provider or write-ahead log needs its own power-loss and recovery contract. ## Consequences diff --git a/.agents/notes/implemented/architecture/2026-06-14-session-persistence.zh.md b/.agents/notes/implemented/architecture/2026-06-14-session-persistence.zh.md index 7b66aed6f0..9dcdbffc7d 100644 --- a/.agents/notes/implemented/architecture/2026-06-14-session-persistence.zh.md +++ b/.agents/notes/implemented/architecture/2026-06-14-session-persistence.zh.md @@ -20,7 +20,7 @@ Status: implemented 长期有效、存在争议的关键选择： - **规范的持久日志无损保留每个 `SessionEvent`，包括 `assistant/chunk`。** JSONL 存储可以将一段连续的增量事件编码为一条打包行，但逻辑读取方会重建精确的事件边界、序号与时间戳。`deriveMessages()` 跳过分片，而过滤分片的方案（Codex 的 `policy.rs`）很有吸引力，但 `seq = log.length` 以及 `events[i].seq === i` 验证要求*连续*的逻辑日志；过滤掉分片会留下空洞，同时破坏约定和恢复功能。基于分片过滤的投影可以作为派生视图在后续实现（带有自己的重新编号），但它不是规范日志。 -- **仅追加；崩溃的轮次被关闭，而非截断。** 已刷写的事件永不被重写。[语义检查点策略](../bug-fix/2026-07-21-semantic-session-checkpoints.zh.md)会在调用模型前排空请求、在调用工具前排空已记录的顶层调用，并在步骤结束后排空完整的响应/结果批次；循环则排空最终轮次边界。由于一个被中断的轮次可能包含大量有效工作，冷检查会保留其连续、可解析的事件，并在内存逻辑视图中为未应答的 assistant 调用添加按风险分类的错误结果、补一个缺失的 `step/end`，以及带 `{ kind: 'interrupted' }` 的 `turn/end`。`prepare` 或 `load` 在返回可恢复视图前提交这些收尾事件；合成结果保证恢复后的提供方 transcript（文本记录）仍然有效。只有不完整的最后一条记录会在提交修复时被丢弃；在最后一个真实 `turn/end` 处或之前出现解析错误或序号间隙，属于数据损坏，会使该会话不可加载。 +- **普通写入仅追加；崩溃的轮次被关闭，而非截断。** 正常持久化绝不重写已刷入当前 generation 的事件。格式迁移保持精确物理源路径、字节与 inode 不变，再只经过迁移边拥有的归一化与当前崩溃修复，在此前不存在的规范具名版本文件下发布一个重新编码的当前后继。[语义检查点策略](../bug-fix/2026-07-21-semantic-session-checkpoints.zh.md)会在调用模型前排空请求、在调用工具前排空已记录的顶层调用，并在步骤结束后排空完整的响应/结果批次；循环则排空最终轮次边界。由于一个被中断的轮次可能包含大量有效工作，冷检查会保留其连续、可解析的事件，并在内存逻辑视图中为未应答的 assistant 调用添加按风险分类的错误结果、补一个缺失的 `step/end`，以及带 `{ kind: 'interrupted' }` 的 `turn/end`。`prepare` 或 `load` 在返回可恢复视图前提交这些收尾事件；合成结果保证恢复后的提供方 transcript（文本记录）仍然有效。只有不完整的最后一条记录会在提交修复时被丢弃；在最后一个真实 `turn/end` 处或之前出现解析错误或序号间隙，属于数据损坏，会使该会话不可加载。 - **文件后端为规范实现，服务保持可扩展。** `dsh-session-persistence-jsonl` 是唯一 first-party provider，并通过 `runPersistenceContract`；抽象服务与 coordinator 继续供仓库外 provider 使用。[JSONL-only 持久化决策](../simplification/2026-08-30-jsonl-only-session-persistence.zh.md)负责 first-party 数据库 provider 的删除及其明确 compatibility cut。 - **元数据在日志之外。** 格式版本、cwd 和谱系是存储关注点，不是可回放的对话状态，因此它们存放在 `dsh-session` 拥有的 `SessionHeader` 中，并通过新的只读属性 `session.header` 附加到 `Session` 上——永远不进入 `SessionEventMap`，永远不到达 `deriveMessages()`。`createdAt` 是以 Unix epoch 毫秒表示的非负安全整数：运行时创建和持久化注册会拒绝小数值，JSONL 会验证解码后的 header。替代方案（一个可合并扩展的 `session/meta` 事件作为日志第 0 行）被否决：日志内事件会自然随 seed/fork 的会话携带，但元数据不是可回放状态，因此显式的日志外 header 边界是更清晰的取舍。（header 最初被拆分为不可变的 `SessionHeader` 加可变的 `SessionSummary`，二者的联合类型为 `SessionMeta`；可变 summary 后来因属于死状态而被移除——见 [移除可变会话摘要](../simplification/2026-06-19-drop-mutable-session-summary.zh.md)。） - **`ctx.agents.create()` 和 `ctx.agents.resume()` 是异步工厂；恢复还跨越持久化边界。** `ctx.agents.resume({ resumeSessionId })` 通过 `ctx.sessionPersistence.prepare()` 取得精确的未发布 Session，以持久化 id 发布它，并继续其投影。[Session 准备阶段决策](2026-08-05-session-preparation.zh.md)定义历史检查与恢复之间的复用。agent loop（智能体循环）不会硬注入 `sessionPersistence`（那样会让非持久化的演示永远挂起）；当它不存在时，`resume` 会以明确的错误拒绝。 @@ -29,7 +29,7 @@ Status: implemented 上述每个关键选择都在陈述处记录了被否决的替代方案：**过滤分片的规范日志**（Codex 的 `policy.rs` 形式）破坏连续 seq 约定；**截断崩溃的轮次**会静默销毁长时间自主运行中的真实工作；**日志内 `session/meta` 事件作为第 0 行**——元数据不是可回放状态；**有限的非整数 `createdAt` 值**没有生产方，且与整数 Unix 毫秒存储不一致；**将 `sessionPersistence` 硬注入循环**会让非持久化的演示永远挂起。 -格式版本控制：header 携带一个 `version`；冷读取拒绝任何非当前版本。预发布阶段的会话格式仍固定为 `SESSION_FORMAT_VERSION = 0`，不承诺广泛兼容；当持久化用户数据确有需要时，协调器可以负责显式且范围受限的导入升级（[消息标识机制引入前的消息恢复](../bug-fix/2026-07-28-load-pre-identity-session-messages.zh.md)）。仅追加 + 刷写能承受冷准备时可容忍的尾部不完整写入；未来 provider 或 write-ahead log 需要自有的断电与恢复约定。 +格式版本控制：header 携带一个 `version`；当前 Session 与协调器代码只接受 `SESSION_FORMAT_VERSION = 1`。JSONL 的事件正文读取会在构造 Session 前组合静态相邻迁移链，v0-to-v1 边拥有原有的范围受限导入升级，例如[消息标识机制引入前的消息恢复](../bug-fix/2026-07-28-load-pre-identity-session-messages.zh.md)。V0 保留在无后缀 `session.jsonl[.zstd]`，v1 及后续版本使用不可变的小写 `session.vN.jsonl[.zstd]` 名称（[已发布 Session 迁移](2026-08-31-released-session-format-migrations.zh.md)）。当前 generation 的追加与 flush 能承受冷准备时可容忍的尾部不完整写入；未来 provider 或 write-ahead log 需要自有的断电与恢复约定。 ## 后果 diff --git a/.agents/notes/implemented/architecture/2026-06-18-session-surface.i18n.yaml b/.agents/notes/implemented/architecture/2026-06-18-session-surface.i18n.yaml index 46d2df95a6..1a123eb416 100644 --- a/.agents/notes/implemented/architecture/2026-06-18-session-surface.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-06-18-session-surface.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-06-18-session-surface.md -2026-06-18-session-surface.md: 95298da0e4bd16e822cb5960718d23ecda7a1b5c -2026-06-18-session-surface.zh.md: 7dd05d79f635b193b2c11cb3599264ebf2424d79 +2026-06-18-session-surface.md: e9331ca2f827edd291d6365ef1356520c8f1927a +2026-06-18-session-surface.zh.md: 26b58afe6aec3a192da94b2c29d4ec0c95d88a6b diff --git a/.agents/notes/implemented/architecture/2026-06-18-session-surface.md b/.agents/notes/implemented/architecture/2026-06-18-session-surface.md index 95298da0e4..e9331ca2f8 100644 --- a/.agents/notes/implemented/architecture/2026-06-18-session-surface.md +++ b/.agents/notes/implemented/architecture/2026-06-18-session-surface.md @@ -41,7 +41,7 @@ Delta processing is O(1) when no new events and O(new events) when new events ar ### Persistence -The new fields are serialized as top-level JSON properties. JSONL storage requires no separate column mapping: its lossless JSON boundary preserves both values. The session format `version` is pinned at `SESSION_FORMAT_VERSION = 0`; the optional surface fields are absorbed without bumping it. +The new fields are serialized as top-level JSON properties. JSONL storage requires no separate column mapping: its lossless JSON boundary preserves both values. Released v0 and v1 share this surface representation, and the identity v0-to-v1 edge preserves it exactly; a future structural representation change increments `SESSION_FORMAT_VERSION` and owns an adjacent migration. ### Crash recovery @@ -51,7 +51,7 @@ The `repair.ts` module synthesizes `tool/result` closers for orphaned tool calls `Session` validates `sourceEventSeqs` and `surfaceOp` at the always-on seed/append boundary: only `assistant/message` may use an empty source-event list; references are unique, earlier, and known; replacement endpoints exist in surface order; and `sourceEventSeqs` covers every shadowed node. These are single-record acceptance and storage-projection rules, not optional invariant-service contributions. -Every surface-eligible event must carry `surfaceOp` or it would disappear from derived history. Typed `append` overloads enforce this for literal event types; runtime checks in `append` and the seed constructor cover widened unions and loaded logs. Invalid seeds are rejected rather than upgraded under the pre-release format policy. +Every surface-eligible event must carry `surfaceOp` or it would disappear from derived history. Typed `append` overloads enforce this for literal event types; runtime checks in `append` and the seed constructor cover widened unions and current loaded logs. Historical v0 validation and normalization belong to the v0-to-v1 edge rather than generic Session code. ## Alternatives considered diff --git a/.agents/notes/implemented/architecture/2026-06-18-session-surface.zh.md b/.agents/notes/implemented/architecture/2026-06-18-session-surface.zh.md index 7dd05d79f6..26b58afe6a 100644 --- a/.agents/notes/implemented/architecture/2026-06-18-session-surface.zh.md +++ b/.agents/notes/implemented/architecture/2026-06-18-session-surface.zh.md @@ -41,7 +41,7 @@ export type SurfaceOp = ### 持久化 -新字段作为顶层 JSON 属性序列化。JSONL 存储无需单独列映射：其无损 JSON 边界会保留两个值。会话格式 `version` 固定为 `SESSION_FORMAT_VERSION = 0`；可选 surface 字段被吸收而不递增版本号。 +新字段作为顶层 JSON 属性序列化。JSONL 存储无需单独列映射：其无损 JSON 边界会保留两个值。已发布 v0 与 v1 共享该 surface 表示，恒等的 v0-to-v1 边会精确保留它；未来结构性表示变更会递增 `SESSION_FORMAT_VERSION` 并拥有一项相邻迁移。 ### 崩溃恢复 @@ -51,7 +51,7 @@ export type SurfaceOp = `Session` 在始终启用的 seed/append 边界校验 `sourceEventSeqs` 与 `surfaceOp`：只有 `assistant/message` 可以使用空的源事件列表；引用必须唯一、更早且已知；替换端点必须存在于 surface 顺序中；`sourceEventSeqs` 必须覆盖每个被遮蔽的节点。这些是单记录接纳与存储投影规则，不是由可选的不变式服务提供的规则。 -每个可进入 surface 的事件都必须携带 `surfaceOp`，否则它将从派生历史中消失。类型化的 `append` 重载对字面事件类型强制执行此规则；`append` 和种子构造函数中的运行时检查覆盖宽化联合类型和加载的日志。按照预发布格式策略，无效的种子被拒绝而非升级。 +每个可进入 surface 的事件都必须携带 `surfaceOp`，否则它将从派生历史中消失。类型化的 `append` 重载对字面事件类型强制执行此规则；`append` 和种子构造函数中的运行时检查覆盖宽化联合类型和当前已加载日志。历史 v0 的校验与规范化属于 v0-to-v1 边，而不属于通用 Session 代码。 ## 曾考虑的替代方案 diff --git a/.agents/notes/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.i18n.yaml b/.agents/notes/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.i18n.yaml index a6f873d0f8..6eebca38d8 100644 --- a/.agents/notes/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.md -2026-06-18-shared-persistence-write-coordinator.md: a61ceb9b2197a6dd8ed86c1c971373a2706607aa -2026-06-18-shared-persistence-write-coordinator.zh.md: 777d5f5972ac1096c2e3434f9e0ac5aec27e8c26 +2026-06-18-shared-persistence-write-coordinator.md: 83cfbf65cf27b79c9deef2f58943931f402feb22 +2026-06-18-shared-persistence-write-coordinator.zh.md: a7b69add8be5a96ad7b9f25484d9601200c551f3 diff --git a/.agents/notes/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.md b/.agents/notes/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.md index a61ceb9b21..83cfbf65cf 100644 --- a/.agents/notes/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.md +++ b/.agents/notes/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.md @@ -24,15 +24,14 @@ The coordinator retires a session from `session/disposed`: it waits for the cont ### The hook interface (`PersistenceBackend`) -Five required members plus optional empty-materialization and lifecycle hooks form the only boundary between the coordinator and storage: +Six required durable primitives plus optional format-fusion, seek, empty-materialization, artifact, and lifecycle hooks form the only boundary between the coordinator and storage: -- `name` — backend label for the dispose-failure `AggregateError`. -- `loadStored(id)` — read one stored prefix by id across every storage scope. Preparation, logical load/inspection, physical suffix reads, live adoption, and the create-collision probe share this lookup. The coordinator asserts the returned id and rejects a stored/live cwd mismatch before repair or state publication. -- `appendBatch(meta, events, isMaterialized)` — durably append a contiguous batch, lazily materializing the session ATOMICALLY when not yet materialized. Ordinary creation therefore cannot leave an abandoned materialized-but-empty session. -- `materializeHeader?(meta)` — explicitly persist a header-only session for `SessionPersistence.ensureMaterialized(session)`. This is reserved for a lifecycle frontend that treats an empty session itself as a resumable durable resource; [standard ACP automation controls](../feature/2026-08-22-standard-acp-automation-controls.md) are the first consumer. Backends that support that lifecycle implement the hook; lazy creation remains the default. -- `commitRepair(meta, tornMarker, closers)` — make a crash repair durable: truncate the torn tail (iff `tornMarker !== undefined`) and append `closers`. **NOT required to be atomic** — JSONL legitimately truncates then appends in two fsync'd steps. Used by `prepare`/`load` (truncate + synthetic closers) and live adoption (truncate only, `closers = []`). -- `list()` — list all stored metadata. -- `close?()` — optional lifecycle teardown for a provider with owned resources; JSONL omits it. The dispose effect awaits it after the quiescence drain so a close failure never masks a drain error. +- `name` labels an aggregate disposal failure; `loadStored(id)` reads one already-current detached prefix; and `readStoredRevision(id)` returns its cheap source identity. The coordinator asserts ids and rejects a stored/live cwd mismatch before repair or state publication. +- `ensureCurrent?(id)` lets a backend publish any supported historical generation's current successor before a current read. `loadCurrentStored?(id)` fuses that operation with prefix decoding over one stable selected-generation snapshot; otherwise the coordinator calls `ensureCurrent` and `loadStored` in order. The same split exists for raw artifacts through `readCurrentRawStored?` and `readRawStored?`. +- `loadStoredFrom?(id, fromSeq)` is an optional seek-capable current suffix read. Sequential backends omit it and reuse the full current prefix. +- `appendBatch(meta, events, isMaterialized)` durably appends a contiguous batch and atomically performs lazy first materialization. `materializeHeader?(meta)` explicitly persists an empty resumable Session for lifecycle frontends such as [standard ACP automation controls](../feature/2026-08-22-standard-acp-automation-controls.md). +- `commitRepair(meta, tornMarker, closers)` makes crash repair durable by truncating the torn tail when present and appending closers when present. It need not be atomic: JSONL legitimately truncates then appends in two synced steps. Preparation and load commit truncation plus synthetic closers; live adoption commits truncation only. +- `list()` returns one header-only descriptor for every stored artifact; `locate?()` resolves a backend-owned artifact without I/O; and `close?()` releases backend resources after the coordinator's quiescence drain. ### The opaque torn marker @@ -40,13 +39,13 @@ The single design choice that keeps the seam clean: the crash-repair "where is t ## Testing -The shared `runPersistenceContract` proves that JSONL `inspect` balances an interrupted logical view without changing storage or revisions before `prepare` or `load` commits recovery. `runCoordinatorContract` (`tests/coordinator-contract.ts`) covers adoption, HMR, collision, Session and provider disposal drains, and crash-tail repair through an in-memory reference and JSONL. `persistence.spec.ts`, `preparations.spec.ts`, and `write-behind.spec.ts` cover preparation reuse and reservation, bounded prepared-state eviction, fixed-window follow-up batches, live-controller cleanup, same-id chain-tail races, failed-batch retry, and close ordering. JSONL specs retain storage mechanics and the through-coordinator torn-tail case that exercises the opaque-marker branch. +The shared `runPersistenceContract` proves that already-current JSONL `inspect` balances an interrupted logical view without changing storage or revisions before `prepare` or `load` commits recovery; a supported historical inspection first publishes its migrated and repaired current successor beside the unchanged source. `runCoordinatorContract` (`tests/coordinator-contract.ts`) covers adoption, HMR, collision, Session and provider disposal drains, and crash-tail repair through an in-memory reference and JSONL. `persistence.spec.ts`, `preparations.spec.ts`, and `write-behind.spec.ts` cover preparation reuse and reservation, bounded prepared-state eviction, fixed-window follow-up batches, live-controller cleanup, same-id chain-tail races, failed-batch retry, and close ordering. JSONL specs retain storage mechanics and the through-coordinator torn-tail case that exercises the opaque-marker branch. ## Alternatives considered - **A base class the backends extend** — rejected for composition: a backend exposes only the hooks, cannot reach the coordinator's private orchestration state, and a third-party backend may still implement the abstract service directly without the coordinator at all. -- **A wider hook API** — each candidate hook folds away: there is no scope-specific live lookup because `loadStored` plus the coordinator's cwd check preserves the collision boundary, no storage-locator generic because validated JSONL metadata reproduces its path, no separate `materialize` hook because the first batch must commit atomically with materialization, no separate create-collision probe because it is `loadStored(id) !== undefined`, and no coordinator pass-through for `list()` because listing needs none of the orchestration. +- **Put historical formats in the coordinator or require two physical reads** — rejected because format framing, highest-generation selection, immutable successor publication, and stable-source decoding belong to the backend. Optional fused current reads preserve one coordinator lifecycle while letting JSONL classify or migrate and decode one exact snapshot; the ordinary hooks remain the fallback for other backends. ## Consequences -The coordinator adds one indirection, an opaque torn marker, detached Session-retirement tasks, and bounded prepared Session state, but centralizes correctness-heavy orchestration for the JSONL provider and future implementations. Session disposal remains an observe-only event, so the Session owner does not await persistence retirement; the coordinator contains failures, preserves pending events in the live controller, and makes provider teardown the quiescence boundary. Its hook surface stays narrow: identity, adoption, collision checks, preparation, and immutable inspection reuse `loadStored`; materialization stays atomic inside `appendBatch`; and listing bypasses the coordinator. Read models use `inspect` rather than `load`, so observing a persisted open turn does not commit interruption closers; the [Session preparation decision](2026-08-05-session-preparation.md) owns reuse, reservation, and publication. A new provider implements storage primitives rather than copy the bounded write lifecycle. +The coordinator adds one indirection, an opaque torn marker, detached Session-retirement tasks, and bounded prepared Session state, but centralizes correctness-heavy orchestration for the JSONL provider and future implementations. Session disposal remains an observe-only event, so the Session owner does not await persistence retirement; the coordinator contains failures, preserves pending events in the live controller, and makes provider teardown the quiescence boundary. Its hooks stay tied to current consumers: identity, adoption, collision checks, preparation, and immutable inspection share the serialized current-prefix path; materialization stays atomic inside `appendBatch`; and listing bypasses stateful orchestration. For already-current input, read models use `inspect` rather than `load`, so observing an open turn does not commit interruption closers; historical input first publishes a separate migrated and repaired successor. The [Session preparation decision](2026-08-05-session-preparation.md) owns reuse, reservation, and publication. A new provider implements storage primitives rather than copying the bounded write lifecycle. diff --git a/.agents/notes/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.zh.md b/.agents/notes/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.zh.md index 777d5f5972..a7b69add8b 100644 --- a/.agents/notes/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.zh.md +++ b/.agents/notes/implemented/architecture/2026-06-18-shared-persistence-write-coordinator.zh.md @@ -24,15 +24,14 @@ JSONL provider 需要在其存储原语周围执行对正确性要求很高的 ### 钩子接口（`PersistenceBackend`） -五个必需成员加可选的空会话实体化与生命周期钩子，构成协调器与存储之间唯一的边界： +六个必需的持久原语，加上可选的格式融合、seek、空会话物化、产物与生命周期钩子，构成协调器与存储之间唯一的边界： -- `name`——后端标签，用于 dispose 失败时的 `AggregateError`。 -- `loadStored(id)`——按 id 跨所有存储范围读取一个已存储前缀。准备、逻辑加载/检查、物理后缀读取、存活会话接管与创建碰撞探测共用此查找。协调器会断言返回的 id，并在修复或发布状态之前拒绝已存储记录与存活会话的 cwd 不匹配。 -- `appendBatch(meta, events, isMaterialized)`——持久追加一个连续批次，在尚未物化时原子地惰性物化会话。因此，普通创建不会留下被放弃的已物化空会话。 -- `materializeHeader?(meta)`——为 `SessionPersistence.ensureMaterialized(session)` 显式持久化仅含 header 的会话。它只供把空会话本身视为可恢复持久资源的生命周期前端使用；[标准 ACP 自动化控制](../feature/2026-08-22-standard-acp-automation-controls.zh.md)是第一个 consumer。支持该生命周期的后端实现此钩子；惰性创建仍是默认行为。 -- `commitRepair(meta, tornMarker, closers)`——使崩溃修复持久化：截断损坏的尾部（当且仅当 `tornMarker !== undefined`）并追加 `closers`。**不要求原子性**——JSONL 合理地分两步 fsync，先截断再追加。用于 `prepare`/`load`（截断 + 合成收尾事件）和存活会话接管（仅截断，`closers = []`）。 -- `list()`——列出所有已存储的元数据。 -- `close?()`——供拥有资源的 provider 使用的可选生命周期清理；JSONL 省略该钩子。dispose effect 在排空至完全停稳后 await 它，因此 close 失败不会掩盖排空错误。 +- `name` 标记聚合 dispose 失败；`loadStored(id)` 读取一个已经是当前格式的分离前缀；`readStoredRevision(id)` 返回其低成本来源标识。协调器会断言 id，并在修复或发布状态之前拒绝存储记录与活动会话的 cwd 不匹配。 +- `ensureCurrent?(id)` 让后端在当前格式读取前发布任何受支持历史 generation 的当前后继。`loadCurrentStored?(id)` 把该操作与基于一个稳定选定 generation 快照的前缀解码融合；否则协调器依次调用 `ensureCurrent` 与 `loadStored`。原始产物通过 `readCurrentRawStored?` 与 `readRawStored?` 使用同一分工。 +- `loadStoredFrom?(id, fromSeq)` 是可选的可 seek 当前格式后缀读取。顺序后端省略它并复用完整当前前缀。 +- `appendBatch(meta, events, isMaterialized)` 持久追加一个连续批次，并原子执行惰性首次物化。`materializeHeader?(meta)` 为 [标准 ACP 自动化控制](../feature/2026-08-22-standard-acp-automation-controls.zh.md)这类生命周期前端显式持久化空的可恢复 Session。 +- `commitRepair(meta, tornMarker, closers)` 在存在 torn tail 时截断它，在存在 closer 时追加它，从而持久化崩溃修复。它不要求原子性：JSONL 合理地分两步同步，先截断再追加。准备与加载提交截断和合成 closer；活动接管只提交截断。 +- `list()` 为每个存储产物返回一个仅 header descriptor；`locate?()` 不经 I/O 解析后端拥有的产物；`close?()` 在协调器排空至完全停稳后释放后端资源。 ### 不透明的 torn marker @@ -40,13 +39,13 @@ JSONL provider 需要在其存储原语周围执行对正确性要求很高的 ## 测试 -共享 `runPersistenceContract` 证明 JSONL 的 `inspect` 会配平被中断的逻辑视图但不改变存储或修订版本，随后由 `prepare` 或 `load` 提交恢复。`runCoordinatorContract`（`tests/coordinator-contract.ts`）通过内存参考实现与 JSONL 覆盖接管、HMR、碰撞、Session 与 provider dispose 排空和崩溃尾部修复。`persistence.spec.ts`、`preparations.spec.ts` 与 `write-behind.spec.ts` 覆盖准备复用与预留、有界准备状态淘汰、固定窗口后续批次、存活控制器清理、同 id 链尾竞态、失败批次重试与关闭顺序。JSONL 规格保留存储机制，以及覆盖不透明 marker 分支的经由协调器崩溃尾部用例。 +共享 `runPersistenceContract` 证明已经是当前格式的 JSONL `inspect` 会配平被中断的逻辑视图但不改变存储或 revision，随后由 `prepare` 或 `load` 提交恢复；受支持的历史检查会先在不改变源文件的情况下于其旁边发布迁移并修复后的当前后继。`runCoordinatorContract`（`tests/coordinator-contract.ts`）通过内存参考实现与 JSONL 覆盖接管、HMR、碰撞、Session 与 provider dispose 排空和崩溃尾部修复。`persistence.spec.ts`、`preparations.spec.ts` 与 `write-behind.spec.ts` 覆盖准备复用与预留、有界准备状态淘汰、固定窗口后续批次、存活 controller 清理、同 id 链尾竞态、失败批次重试与关闭顺序。JSONL spec 保留存储机制，以及覆盖不透明 marker 分支的经由协调器崩溃尾部用例。 ## 曾考虑的替代方案 - **后端继承的基类**——否决，改用组合：后端只暴露钩子，无法触及协调器的私有编排状态，且第三方后端仍可完全不使用协调器、直接实现抽象服务。 -- **更宽的钩子 API**——每个候选钩子都被折叠掉：没有限定存储范围的存活会话查找，因为 `loadStored` 加上协调器的 cwd 检查即可维持碰撞边界；没有存储定位器泛型，因为经验证的 JSONL 元数据可还原其路径；没有单独的 `materialize` 钩子，因为首批事件必须与物化原子提交；没有单独的创建碰撞探测，因为它就是 `loadStored(id) !== undefined`；`list()` 也不经由协调器透传，因为列举不需要任何编排。 +- **把历史格式放入协调器，或要求两次物理读取**——不予采用，因为格式 framing、最高 generation 选择、不可变后继发布与稳定来源解码属于后端。可选的融合当前格式读取保留一个协调器生命周期，同时让 JSONL 对同一个精确快照完成分类或迁移与解码；普通钩子继续作为其他后端的 fallback。 ## 后果 -协调器增加一层间接、一个不透明 torn marker、脱离 Session 生命周期的退役任务，以及有界的已准备 Session 状态，但为 JSONL provider 与未来实现集中管理对正确性要求很高的编排。Session dispose 仍是仅观察事件，因此 Session owner 不等待持久化退役；协调器收容失败、在存活控制器中保留待处理事件，并以 provider teardown 为完全停稳边界。其钩子面保持窄小：标识校验、接管、碰撞检查、准备与不可变检查共用 `loadStored`；物化保持在 `appendBatch` 内原子完成；列举绕过协调器。读模型使用 `inspect` 而非 `load`，因此观察已持久化但仍开放的轮次时不会提交中断收尾事件；复用、预留与发布由 [Session 准备阶段决策](2026-08-05-session-preparation.zh.md)定义。新 provider 只需实现存储原语，而无需复制有界写入生命周期。 +协调器增加一层间接、一个不透明 torn marker、脱离 Session 生命周期的退役任务，以及有界的已准备 Session 状态，但为 JSONL provider 与未来实现集中管理对正确性要求很高的编排。Session dispose 仍是仅观察事件，因此 Session owner 不等待持久化退役；协调器收容失败、在活动 controller 中保留待处理事件，并以 provider teardown 为完全停稳边界。其钩子只服务当前 consumer：标识、接管、碰撞检查、准备与不可变检查共用串行化当前前缀路径；物化保持在 `appendBatch` 内原子完成；列举绕过有状态编排。对已经是当前格式的输入，读模型使用 `inspect` 而非 `load`，因此观察开放轮次时不会提交中断 closer；历史输入会先发布一个独立的迁移并修复后继。复用、预留与发布由 [Session 准备阶段决策](2026-08-05-session-preparation.zh.md)定义。新 provider 只需实现存储原语，而无需复制有界写入生命周期。 diff --git a/.agents/notes/implemented/architecture/2026-07-05-reconstructable-requests.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-05-reconstructable-requests.i18n.yaml index a9d00bc3b9..c1a0acb3d7 100644 --- a/.agents/notes/implemented/architecture/2026-07-05-reconstructable-requests.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-05-reconstructable-requests.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-05-reconstructable-requests.md -2026-07-05-reconstructable-requests.md: 3786de02d06c0b6c094297ae89ac3f84053e408d -2026-07-05-reconstructable-requests.zh.md: 851045aca7dababd0da859f3b04b721c65382fc3 +2026-07-05-reconstructable-requests.md: bc8ba640c400b18598f18aa303f2bd1b5c5b9cdc +2026-07-05-reconstructable-requests.zh.md: de1802aac83f1e0980172d2d541a093f2d729e4a diff --git a/.agents/notes/implemented/architecture/2026-07-05-reconstructable-requests.md b/.agents/notes/implemented/architecture/2026-07-05-reconstructable-requests.md index 3786de02d0..bc8ba640c4 100644 --- a/.agents/notes/implemented/architecture/2026-07-05-reconstructable-requests.md +++ b/.agents/notes/implemented/architecture/2026-07-05-reconstructable-requests.md @@ -53,5 +53,5 @@ Like MiniCode, the conversation advances append-only and resets only when model- - `agent/pre-step` is the current-request message channel; direct inbox mutation is the eventual later-request channel. - Tool-result trimming needs no new mechanism: a logged single-entry surface replace (`start === end`) carrying a trimmed `tool/result` under the same `callId` — compaction-family, replay-correct, cache-bust batched by the same pressure logic. - Unreadable referenced attachment objects still fail model requests; [automatic attachment quarantine](../../proposed/bug-fix/2026-08-20-attachment-read-quarantine.md) records the proposed recovery without weakening byte-exact reconstruction. -- Session logs grow one `request/header` snapshot per loop instance, real change, and later model-message series. Repeating the full system prompt and tool catalog is larger than a delta codec but small beside chunk-heavy logs and retains one self-contained replay representation. `SESSION_FORMAT_VERSION` stays `0`; legacy delta events are rejected rather than migrated. +- Session logs grow one `request/header` snapshot per loop instance, real change, and later model-message series. Repeating the full system prompt and tool catalog is larger than a delta codec but small beside chunk-heavy logs and retains one self-contained replay representation. Current v1 retains this single representation; the frozen v0-to-v1 edge explicitly refuses legacy delta events before current Session construction. - Snapshot fixtures include each repeated series header. Keyless refresh owns those deterministic log changes, while the snapshot harness pins prompt and tool sidecars only for the initial and actual change revisions and reuses the current revision for `series` snapshots. Filesystem-writing fixtures remain in normalized authored form with cwd-relative tool arguments because replay only round-trips cwd-independent argument paths. diff --git a/.agents/notes/implemented/architecture/2026-07-05-reconstructable-requests.zh.md b/.agents/notes/implemented/architecture/2026-07-05-reconstructable-requests.zh.md index 851045aca7..de1802aac8 100644 --- a/.agents/notes/implemented/architecture/2026-07-05-reconstructable-requests.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-05-reconstructable-requests.zh.md @@ -53,5 +53,5 @@ Status: implemented - `agent/pre-step` 是当前请求的消息通道；直接修改 inbox 则是最终进入后续请求的通道。 - 工具结果裁剪无需新机制：一个已记录的单条目 surface replace（`start === end`），携带同一 `callId` 下裁剪后的 `tool/result`——属压缩家族，回放正确，缓存失效由相同的压力逻辑批量处理。 - 无法读取的被引用附件对象仍会让模型请求失败；[附件自动隔离](../../proposed/bug-fix/2026-08-20-attachment-read-quarantine.zh.md)记录了不削弱字节精确重建的拟议恢复方案。 -- 会话日志会为每个循环实例、真实变更和后续模型消息序列增加一个 `request/header` 快照。重复完整系统提示词与工具目录比 delta 编解码器更大，但相对分片密集型日志仍然很小，并保留一种自包含的回放表示。`SESSION_FORMAT_VERSION` 保持 `0`；旧的 delta 事件被拒绝而非迁移。 -- 快照 fixture 包含每个重复的 series header。无密钥 refresh 负责这些确定性日志变化；快照 harness 只为 initial 与真实 change 修订固定提示词和工具 sidecar，并让 `series` 快照复用当前修订。写入文件系统的 fixture 继续以规范化的撰写形式存储，工具参数使用 cwd 相对路径，因为回放只对 cwd 无关的参数路径做往返。 \ No newline at end of file +- 会话日志会为每个循环实例、真实变更和后续模型消息序列增加一个 `request/header` 快照。重复完整系统提示词与工具目录比 delta 编解码器更大，但相对分片密集型日志仍然很小，并保留一种自包含的回放表示。当前 v1 保留这一种表示；冻结的 v0-to-v1 迁移边会在构造当前 Session 前显式拒绝旧版 delta 事件。 +- 快照 fixture 包含每个重复的 series header。无密钥 refresh 负责这些确定性日志变化；快照 harness 只为 initial 与真实 change 修订固定提示词和工具 sidecar，并让 `series` 快照复用当前修订。写入文件系统的 fixture 继续以规范化的撰写形式存储，工具参数使用 cwd 相对路径，因为回放只对 cwd 无关的参数路径做往返。 diff --git a/.agents/notes/implemented/architecture/2026-07-14-provider-routed-llm-adapters.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-14-provider-routed-llm-adapters.i18n.yaml index 34523f2723..a060f0ddd9 100644 --- a/.agents/notes/implemented/architecture/2026-07-14-provider-routed-llm-adapters.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-14-provider-routed-llm-adapters.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-14-provider-routed-llm-adapters.md -2026-07-14-provider-routed-llm-adapters.md: 78c8d6788006c503b532ff2bbddd30342415f0a4 -2026-07-14-provider-routed-llm-adapters.zh.md: af8bc4fe27a50d47d7b49b51eada67afe889fc13 +2026-07-14-provider-routed-llm-adapters.md: ab4c485b79f37d9360da9a9a32f38063cae99755 +2026-07-14-provider-routed-llm-adapters.zh.md: c431d710af95352a2ecfc7b76b6393428ebedfec diff --git a/.agents/notes/implemented/architecture/2026-07-14-provider-routed-llm-adapters.md b/.agents/notes/implemented/architecture/2026-07-14-provider-routed-llm-adapters.md index 78c8d67880..ab4c485b79 100644 --- a/.agents/notes/implemented/architecture/2026-07-14-provider-routed-llm-adapters.md +++ b/.agents/notes/implemented/architecture/2026-07-14-provider-routed-llm-adapters.md @@ -54,7 +54,7 @@ Compaction configuration gains `summarizationProvider` beside `summarizationMode The JSON-RPC runtime receives provider and model explicitly. Its convenience fallback mounts `dsh-llm-deepseek` only for provider `deepseek` when that provider has no registered owner; other missing providers fail without guessing an adapter. -The on-disk session format remains the pre-release pinned version `0`, with no compatibility promise. Seed/load validation rejects request headers and assistant messages that omit required provider/model fields instead of accepting an old shape that can no longer reconstruct the request. +Current v1 seed/load validation rejects request headers and assistant messages that omit required provider/model fields. The frozen v0-to-v1 edge requires the same reconstructable routing identity before migration; it never guesses a missing provider or model, and malformed shapes refuse before publication. ## Alternatives considered @@ -78,7 +78,7 @@ The on-disk session format remains the pre-release pinned version `0`, with no c - pi-ai credentials, transport knobs, SDK timeouts, and the five-minute-default `streamIdleTimeoutMs` watchdog are scoped per provider profile. Hidden provider retries are disabled; bounded retries belong to the separately composed agent recovery policy. - `dsh-llm-pi-ai` rejects stop sequences because pi-ai's common stream API cannot express them; the native DeepSeek adapter retains its stop support. - Replay state is portable only within the adapter instance that owns both the historical and target providers. Cross-provider and cross-model restoration is an adapter responsibility, and another adapter receives provider-neutral history without the opaque state. -- Current pre-release session JSONL requires provider/model on request headers and assistant messages. Older shapes remain version `0` but are rejected rather than migrated. +- Current v1 Session JSONL requires provider/model on request headers and assistant messages. The v0 edge migrates only frozen shapes that already carry reconstructable request identity. ## Testing @@ -88,4 +88,4 @@ The on-disk session format remains the pre-release pinned version `0`, with no c ## Risks -This is a repo-wide pre-release API break: model-only request construction, adapter registration, app protocols, fixtures, and persisted version-0 event shapes all change together, with no compatibility aliases. The provider exclusivity rule deliberately prevents two implementations of the same upstream from coexisting in one context. A pi-ai dependency update can change the accepted provider/model catalog, so the lockfile and adapter e2e matrix define the tested set. Custom `baseURL` endpoints inherit the chosen catalog model's protocol assumptions and cannot repair an incompatible proxy. Catalog-external model descriptors and multimodal content remain unsupported. pi-ai replay state may contain opaque encrypted reasoning signatures; it is persisted because the provider requires it for continuity, but it is never rendered or logged outside the existing session record. +This was a repo-wide API break when introduced: model-only request construction, adapter registration, app protocols, fixtures, and persisted v0 event shapes changed together, with no compatibility aliases. Released historical recovery now belongs to the adjacent Session-format edge. The provider exclusivity rule deliberately prevents two implementations of the same upstream from coexisting in one context. A pi-ai dependency update can change the accepted provider/model catalog, so the lockfile and adapter e2e matrix define the tested set. Custom `baseURL` endpoints inherit the chosen catalog model's protocol assumptions and cannot repair an incompatible proxy. Catalog-external model descriptors and multimodal content remain unsupported. pi-ai replay state may contain opaque encrypted reasoning signatures; it is persisted because the provider requires it for continuity, but it is never rendered or logged outside the existing session record. diff --git a/.agents/notes/implemented/architecture/2026-07-14-provider-routed-llm-adapters.zh.md b/.agents/notes/implemented/architecture/2026-07-14-provider-routed-llm-adapters.zh.md index af8bc4fe27..c431d710af 100644 --- a/.agents/notes/implemented/architecture/2026-07-14-provider-routed-llm-adapters.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-14-provider-routed-llm-adapters.zh.md @@ -54,7 +54,7 @@ pi-ai 回放状态用其成功 `AssistantMessage` 的带版本最小投影填充 JSON-RPC 运行时显式接收提供方与模型。仅当 `deepseek` 提供方没有注册所有者时，其便利回退才会挂载 `dsh-llm-deepseek`；其他缺失的提供方会直接失败，不会猜测适配器。 -磁盘会话格式仍使用预发布阶段固定的版本 `0`，且不承诺兼容性。seed/load 验证会拒绝省略必需提供方/模型字段的请求头和助手消息，不会接受已无法重建请求的旧格式。 +当前 v1 的 seed/load 验证会拒绝省略必需提供方／模型字段的请求头和助手消息。冻结的 v0-to-v1 迁移边要求迁移前已具备同一套可重建路由身份；它绝不会猜测缺失的提供方或模型，畸形结构会在发布前被拒绝。 ## 考虑过的替代方案 @@ -78,7 +78,7 @@ JSON-RPC 运行时显式接收提供方与模型。仅当 `deepseek` 提供方 - pi-ai 凭据、传输选项、SDK 超时，以及默认五分钟的 `streamIdleTimeoutMs` 空闲超时机制均按提供方配置隔离。系统禁用隐藏的提供方重试；有界重试由单独组合的 agent 恢复策略负责。 - pi-ai 的通用流 API 无法表达停止序列，因此 `dsh-llm-pi-ai` 会拒绝停止序列；原生 DeepSeek 适配器仍支持停止序列。 - 仅当历史提供方与目标提供方归同一个适配器实例所有时，回放状态才可移植。适配器负责跨提供方和跨模型恢复；其他适配器只接收不含不透明状态的提供方无关历史。 -- 当前预发布会话 JSONL 要求请求头和助手消息都包含提供方/模型。旧格式仍使用版本 `0`，但会被拒绝，不执行迁移。 +- 当前 v1 Session JSONL 要求请求头和助手消息都包含提供方／模型。v0 边只迁移已经携带可重建请求身份的冻结结构。 ## 测试 @@ -88,4 +88,4 @@ JSON-RPC 运行时显式接收提供方与模型。仅当 `deepseek` 提供方 ## 风险 -这是一次覆盖全仓库的预发布 API 破坏性变更：仅模型的请求构造、适配器注册、应用协议、fixture，以及持久化版本 0 事件格式会同时变化，不提供兼容别名。提供方排他规则有意禁止同一上游的两个实现共存于同一上下文。pi-ai 依赖升级可能改变可接受的提供方/模型目录，因此锁文件与适配器 e2e 矩阵定义已验证集合。自定义 `baseURL` 端点会继承所选目录模型的协议假设，无法修复不兼容的代理。目录外模型描述符与多模态内容仍不受支持。pi-ai 回放状态可能包含不透明的加密推理签名；提供方需要该信息维持连续性，因此系统会持久化该状态，但不会在现有会话记录之外渲染或记录它。 +这项变更在引入时是覆盖全仓库的 API 破坏性变更：仅模型的请求构造、适配器注册、应用协议、fixture，以及持久化 v0 事件结构同时变化，不提供兼容别名。已发布历史恢复现在属于相邻 Session 格式边。提供方排他规则有意禁止同一上游的两个实现共存于同一上下文。pi-ai 依赖升级可能改变可接受的提供方／模型目录，因此锁文件与适配器 e2e 矩阵定义已验证集合。自定义 `baseURL` 端点会继承所选目录模型的协议假设，无法修复不兼容的代理。目录外模型描述符与多模态内容仍不受支持。pi-ai 回放状态可能包含不透明的加密推理签名；提供方需要该信息维持连续性，因此系统会持久化该状态，但不会在现有会话记录之外渲染或记录它。 diff --git a/.agents/notes/implemented/architecture/2026-07-19-zstandard-jsonl-session-logs.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-19-zstandard-jsonl-session-logs.i18n.yaml index f53ede9338..76348b2cb2 100644 --- a/.agents/notes/implemented/architecture/2026-07-19-zstandard-jsonl-session-logs.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-19-zstandard-jsonl-session-logs.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-19-zstandard-jsonl-session-logs.md -2026-07-19-zstandard-jsonl-session-logs.md: 93fc20f931c75552352834b9340e7d38680d4254 -2026-07-19-zstandard-jsonl-session-logs.zh.md: d58f89430ab91de6beabba83c2a31f43e4a7d275 +2026-07-19-zstandard-jsonl-session-logs.md: a79bc3907c4f6c02851ba1814f733684ce373898 +2026-07-19-zstandard-jsonl-session-logs.zh.md: 178420c126b68689983d5b01f4ffae29b17fe657 diff --git a/.agents/notes/implemented/architecture/2026-07-19-zstandard-jsonl-session-logs.md b/.agents/notes/implemented/architecture/2026-07-19-zstandard-jsonl-session-logs.md index 93fc20f931..a79bc3907c 100644 --- a/.agents/notes/implemented/architecture/2026-07-19-zstandard-jsonl-session-logs.md +++ b/.agents/notes/implemented/architecture/2026-07-19-zstandard-jsonl-session-logs.md @@ -14,9 +14,9 @@ The encoding also has to remain explicit at the deployment boundary. Snapshot fi ### Configuration and suffix ownership -`dsh-session-persistence-jsonl` accepts `compression?: 'zstd' | 'none'` and explicitly resolves omission to `'zstd'`. Zstandard artifacts end in `.jsonl.zstd`; `'none'` retains the original newline-delimited UTF-8 `.jsonl` representation. `SessionLocation.kind` remains `'jsonl'`, because both encodings carry the same logical record format, and `SESSION_FORMAT_VERSION` remains `0` under the repository's pre-release reject-without-migration policy. +`dsh-session-persistence-jsonl` accepts `compression?: 'zstd' | 'none'` and explicitly resolves omission to `'zstd'`. Zstandard artifacts end in `.jsonl.zstd`; `'none'` retains the newline-delimited UTF-8 `.jsonl` representation. Within either configured suffix, v0 uses suffixless `session.jsonl[.zstd]` and every positive format generation uses lowercase `session.vN.jsonl[.zstd]`. `SessionLocation.kind` remains `'jsonl'`, because both encodings carry the same logical record format. Session-format migration uses the configured full suffix and one shared logical chain, so compression does not branch generation selection or publication. -Each persistence root belongs to one encoding. A one-time discovery preflight rejects any opposite suffix, and targeted load, live-adoption, listing, and materialization paths repeat the relevant suffix check after an initially empty preflight. The error names the incompatible artifact and directs the deployment to the matching configuration or a separate root. There is no migration, dual read, dual write, or extension-based fallback. +Each persistence root belongs to one encoding. A one-time discovery preflight rejects any opposite suffix, and targeted load, live-adoption, listing, and materialization paths repeat the relevant suffix check after an initially empty preflight. The error names the incompatible artifact and directs the deployment to the matching configuration or a separate root. There is no compression conversion, dual read, dual write, or extension-based fallback; logical version migration stays within the configured suffix, preserves the source generation, and exclusively publishes the final version-named successor. ### Frame and write path diff --git a/.agents/notes/implemented/architecture/2026-07-19-zstandard-jsonl-session-logs.zh.md b/.agents/notes/implemented/architecture/2026-07-19-zstandard-jsonl-session-logs.zh.md index d58f89430a..178420c126 100644 --- a/.agents/notes/implemented/architecture/2026-07-19-zstandard-jsonl-session-logs.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-19-zstandard-jsonl-session-logs.zh.md @@ -14,9 +14,9 @@ JSONL 持久化后端会逐字保留每个 `SessionEvent`，其中包括数量 ### 配置与后缀归属 -`dsh-session-persistence-jsonl` 接受 `compression?: 'zstd' | 'none'`，并将省略值显式解析为 `'zstd'`。Zstandard 产物使用 `.jsonl.zstd` 后缀；`'none'` 保留原有的换行分隔 UTF-8 `.jsonl` 表示。`SessionLocation.kind` 仍为 `'jsonl'`，因为两种编码承载同一逻辑记录格式；按照仓库的预发布拒绝且不迁移策略，`SESSION_FORMAT_VERSION` 仍为 `0`。 +`dsh-session-persistence-jsonl` 接受 `compression?: 'zstd' | 'none'`，并将省略值显式解析为 `'zstd'`。Zstandard 产物使用 `.jsonl.zstd` 后缀；`'none'` 保留换行分隔 UTF-8 `.jsonl` 表示。在任一已配置后缀内，v0 使用无版本后缀 `session.jsonl[.zstd]`，每个正格式 generation 使用小写 `session.vN.jsonl[.zstd]`。`SessionLocation.kind` 仍为 `'jsonl'`，因为两种编码承载同一逻辑记录格式。Session 格式迁移使用配置后的完整后缀和同一条逻辑链，因此压缩不会分叉 generation 选择或发布。 -每个持久化根目录只归属于一种编码。一次性的发现预检会拒绝任何相反后缀，而针对性的加载、活跃采用、列举与物化路径会在最初空目录预检之后再次执行对应后缀检查。错误会指出不兼容产物，并要求部署选择匹配配置或单独根目录。系统不提供迁移、双重读取、双重写入或基于扩展名的兜底。 +每个持久化根目录只归属于一种编码。一次性的发现预检会拒绝任何相反后缀，而针对性的加载、活跃采用、列举与物化路径会在最初空目录预检之后再次执行对应后缀检查。错误会指出不兼容产物，并要求部署选择匹配配置或单独根目录。系统不提供压缩转换、双重读取、双重写入或基于扩展名的 fallback；逻辑版本迁移始终留在配置后缀内，保留源 generation，并排他发布最终具名版本后继。 ### 帧与写入路径 diff --git a/.agents/notes/implemented/architecture/2026-07-24-project-session-directories.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-24-project-session-directories.i18n.yaml index c20fc92181..a9bc401dd2 100644 --- a/.agents/notes/implemented/architecture/2026-07-24-project-session-directories.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-24-project-session-directories.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-24-project-session-directories.md -2026-07-24-project-session-directories.md: 0aa3f513d5a1bb3e44cf33a0ae1eb791ee3a46c2 -2026-07-24-project-session-directories.zh.md: 932b1d29c41d2a854abfc0bab0e47a0ff8c96fe9 +2026-07-24-project-session-directories.md: 199a2b273b791e1ee566e566fb59e1b83c63cdea +2026-07-24-project-session-directories.zh.md: 8b6327e349c25aa438b2ad409152cd2c8b720822 diff --git a/.agents/notes/implemented/architecture/2026-07-24-project-session-directories.md b/.agents/notes/implemented/architecture/2026-07-24-project-session-directories.md index 0aa3f513d5..199a2b273b 100644 --- a/.agents/notes/implemented/architecture/2026-07-24-project-session-directories.md +++ b/.agents/notes/implemented/architecture/2026-07-24-project-session-directories.md @@ -19,9 +19,10 @@ The JSONL backend stores sessions under a readable project key and gives every s ----/ / session.jsonl.zstd + session.v1.jsonl.zstd ``` -Raw mode uses `session.jsonl`, and sessions without a cwd use `_no-cwd`. Filesystem and drive separators become `-`, unsafe code units use `~XXXX`, and the readable name is bounded to keep the component within filesystem limits. +The two files illustrate retained v0 plus current v1; raw mode omits `.zstd`, positive versions use lowercase `.vN`, and Sessions without a cwd use `_no-cwd`. Filesystem and drive separators become `-`, unsafe code units use `~XXXX`, and the readable name is bounded to keep the component within filesystem limits. The project key intentionally has no hash suffix. This follows the common human-readable convention used by coding agents and keeps the normalized project path as the complete directory name. The normalization is lossy: paths such as `/a/b-c` and `/a-b/c`, or long paths with the same retained prefix, share one project directory. Their distinct session ids still select separate session directories; reuse of the same session id remains a storage collision and is rejected. @@ -29,9 +30,9 @@ Case-insensitive filesystems can also make differently cased project keys refer The configured root remains a deployment choice. The layout neither selects a global root nor requires projects to share one. When a deployment does centralize storage, project paths remain recognizable; a project-local root uses the same deterministic structure. -The encoded session id names an ownership directory rather than the transcript itself. `SessionPersistence.locate()` continues to return the fixed transcript path, preserving hook `transcript_path` and `DSH_SESSION_JSONL` semantics. Discovery ignores other entries inside the session directory so the backend can add session-owned artifacts without another layout change. +The encoded Session id names an ownership directory rather than one transcript. `SessionPersistence.locate(meta)` returns the canonical target for `meta.version`; header-only discovery scans canonical generation names and selects the numerically highest one, while ignoring unrelated entries. The directory can therefore retain prior generations and add other Session-owned artifacts without another layout change. -Lazy materialization remains tied to the transcript: `create()` performs no filesystem I/O, and the first append creates the project/session directories before collision-safe transcript publication. Empty directories are not listed as sessions. The backend rejects flat `/.jsonl*` artifacts with an explicit layout error; the pre-release format provides no automatic data migration. +Lazy materialization remains tied to the current generation: `create()` performs no filesystem I/O, and the first append creates the project/Session directories before no-overwrite publication at the current version's canonical name. Empty directories are not listed as Sessions. The backend rejects flat `/.jsonl*` artifacts with an explicit layout error; it provides no automatic migration from that obsolete directory layout. ## Alternatives considered @@ -47,6 +48,6 @@ Lazy materialization remains tied to the transcript: `create()` performs no file ## Consequences -Shared stores can be navigated by recognizable project names, while local and custom roots keep their existing configuration freedom. Every session has a directory available for future backend-owned artifacts, and existing transcript consumers still receive a file path. +Shared stores can be navigated by recognizable project names, while local and custom roots keep their existing configuration freedom. Every Session has a directory for immutable generation names and other future backend-owned artifacts; callers receive either a version-qualified `locate` target or the exact highest path discovered by listing. Project directory names are longer than the former 12-hex cwd hashes. Very long paths show only a bounded prefix. Moving a project usually selects a different directory, but distinct cwd strings that normalize to the same name share one project directory by design. diff --git a/.agents/notes/implemented/architecture/2026-07-24-project-session-directories.zh.md b/.agents/notes/implemented/architecture/2026-07-24-project-session-directories.zh.md index 932b1d29c4..8b6327e349 100644 --- a/.agents/notes/implemented/architecture/2026-07-24-project-session-directories.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-24-project-session-directories.zh.md @@ -19,9 +19,10 @@ JSONL 后端按可读的项目键存储会话，并为每个会话提供独立 ----/ / session.jsonl.zstd + session.v1.jsonl.zstd ``` -原始模式使用 `session.jsonl`，没有 cwd 的会话使用 `_no-cwd`。文件系统路径分隔符和驱动器分隔符会转换为 `-`，不安全的代码单元使用 `~XXXX`，可读名称则限制长度，以确保目录项不超过文件系统限制。 +这两个文件表示保留的 v0 与当前 v1；raw 模式省略 `.zstd`，正版本使用小写 `.vN`，没有 cwd 的 Session 使用 `_no-cwd`。文件系统路径分隔符和驱动器分隔符会转换为 `-`，不安全的代码单元使用 `~XXXX`，可读名称则限制长度，以确保目录项不超过文件系统限制。 项目键有意不带哈希后缀。这遵循 coding agent（智能体）常用的易读约定，使规范化后的项目路径本身就是完整的目录名。规范化过程有损：`/a/b-c` 与 `/a-b/c` 等路径，或者保留前缀相同的长路径，会共用同一个项目目录。不同的会话 id 仍会选择不同的会话目录；复用相同的会话 id 仍构成存储冲突，系统会予以拒绝。 @@ -29,9 +30,9 @@ JSONL 后端按可读的项目键存储会话，并为每个会话提供独立 根目录由部署配置决定。这种布局既不选择全局根目录，也不要求项目共享根目录。部署选择集中存储时，目录名仍能让项目路径易于辨认；使用项目本地根目录时，也采用同样的确定性结构。 -编码后的会话 id 用于命名归属目录，而不是 transcript 文件本身。`SessionPersistence.locate()` 仍返回固定的 transcript 路径，从而保持钩子 `transcript_path` 和 `DSH_SESSION_JSONL` 的语义不变。发现过程会忽略会话目录中的其他条目，因此后端以后添加会话自有产物时无需再次改变布局。 +编码后的 Session id 用于命名归属目录，而不是某一份 transcript。`SessionPersistence.locate(meta)` 返回 `meta.version` 的规范目标；仅 header 的发现会扫描规范 generation 名并选择数值最高的一项，同时忽略无关条目。因此，该目录可以保留先前 generation，也能添加其他 Session 自有产物，无需再次改变布局。 -延迟物化仍以 transcript 为界：`create()` 不执行文件系统 I/O，首次追加会先创建项目目录和会话目录，再以无冲突方式发布 transcript。空目录不会被列为会话。后端会显式报告布局错误并拒绝扁平的 `/.jsonl*` 产物；预发布格式不提供自动数据迁移。 +延迟物化仍以当前 generation 为界：`create()` 不执行文件系统 I/O，首次 append 会先创建项目目录和 Session 目录，再在当前版本的规范名称下以不覆盖方式发布。空目录不会被列为 Session。后端会显式报告布局错误并拒绝扁平的 `/.jsonl*` 产物；它不提供从该废弃目录布局自动迁移数据的能力。 ## 考虑过的替代方案 @@ -47,6 +48,6 @@ JSONL 后端按可读的项目键存储会话，并为每个会话提供独立 ## 后果 -共享存储可以通过易于辨认的项目名进行浏览，本地根目录和自定义根目录则继续保有现有的配置自由。每个会话都有一个可供后端未来存放自有产物的目录，而现有 transcript 消费方仍会收到文件路径。 +共享存储可以通过易于辨认的项目名进行浏览，本地根目录和自定义根目录则继续保有现有的配置自由。每个 Session 都有一个用于不可变 generation 名称与其他未来后端自有产物的目录；调用方会收到版本限定的 `locate` 目标，或列表发现的精确最高路径。 项目目录名比原先由 12 个十六进制字符组成的 cwd 哈希更长。路径很长时，目录名只显示长度受限的前缀。移动项目通常会选择不同的目录，但按设计，不同的 cwd 字符串如果规范化成相同名称，就会共用同一个项目目录。 diff --git a/.agents/notes/implemented/architecture/2026-07-30-session-end-seed-log-boundary.i18n.yaml b/.agents/notes/implemented/architecture/2026-07-30-session-end-seed-log-boundary.i18n.yaml index b91733813c..ebb20ae0c6 100644 --- a/.agents/notes/implemented/architecture/2026-07-30-session-end-seed-log-boundary.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-07-30-session-end-seed-log-boundary.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-30-session-end-seed-log-boundary.md -2026-07-30-session-end-seed-log-boundary.md: c6ed3911a797480804d064273922d85412664c79 -2026-07-30-session-end-seed-log-boundary.zh.md: 1e9517f9a5aed819fdaff6194ab952c322c85b82 +2026-07-30-session-end-seed-log-boundary.md: 79323c9b987bc220bc634393bc542baea409fbbb +2026-07-30-session-end-seed-log-boundary.zh.md: b51c47ff5667c90e394c42f3ef1084c7ecd6b7ba diff --git a/.agents/notes/implemented/architecture/2026-07-30-session-end-seed-log-boundary.md b/.agents/notes/implemented/architecture/2026-07-30-session-end-seed-log-boundary.md index c6ed3911a7..79323c9b98 100644 --- a/.agents/notes/implemented/architecture/2026-07-30-session-end-seed-log-boundary.md +++ b/.agents/notes/implemented/architecture/2026-07-30-session-end-seed-log-boundary.md @@ -48,8 +48,8 @@ The predicate holds for a bracket *this* session inherited, not as a liveness si Bought: one boundary, written in one place, correct for all six seeded-start paths — including the fork gap the persistence-layer version could not reach. The persistence packages keep a pure read path. `firstLiveSeq` gains a durable twin rather than a second, competing notion of the same boundary. -Cost: a seeded session's log is one event longer, including an empty resumed log. Seq expectations move with that boundary. Two updates are load-bearing rather than mechanical: telemetry's adoption tests assert the boundary IS exported, because it is this lifecycle's own write, and the property suite's replay invariant is "seed reproduced verbatim, plus one log-only boundary" with idempotence as its own property. +Cost: a seeded session's log is one event longer, including an empty resumed log. Seq expectations move with that boundary. Two updates are load-bearing rather than mechanical: telemetry's adoption tests assert the boundary is exported as part of complete canonical-log replay, including when it arrived in a resumed constructor seed, and the property suite's replay invariant is "seed reproduced verbatim, plus one log-only boundary" with idempotence as its own property. -`session/end-seed` joins the on-disk vocabulary. Under the pre-release stance (`SESSION_FORMAT_VERSION` pinned at `0`, no compatibility promise) older logs simply lack it, and a log without a boundary correctly classifies nothing as constructor-seed history. +`session/end-seed` joins the on-disk vocabulary. Current v1 requires the validated marker semantics owned by Session; the frozen v0 codec and migration edge own which historical v0 seed layouts remain admissible. The exact inherited cut stays separate from the logical header and is available after a body read. The [queued manual compaction decision](../feature/2026-07-30-queued-manual-compaction.md) now supplies the first consumer. Its tail scan independently finds the unmatched `compaction/start` and newest end-seed, treats only a start after that boundary as live, and clears the invariant trace on the same replay transition. The predicate remains in the compaction package rather than becoming a generic core helper. diff --git a/.agents/notes/implemented/architecture/2026-07-30-session-end-seed-log-boundary.zh.md b/.agents/notes/implemented/architecture/2026-07-30-session-end-seed-log-boundary.zh.md index 1e9517f9a5..b51c47ff56 100644 --- a/.agents/notes/implemented/architecture/2026-07-30-session-end-seed-log-boundary.zh.md +++ b/.agents/notes/implemented/architecture/2026-07-30-session-end-seed-log-boundary.zh.md @@ -48,8 +48,8 @@ Status: implemented 买到的：一条边界，在一处写入，对全部六条带种子启动路径都正确——包括持久化层方案触及不到的 fork 缺口。持久化各包保留纯读取路径。`firstLiveSeq` 获得一个持久孪生体，而不是关于同一边界的第二套彼此竞争的概念。 -代价：带种子会话的日志长了一个事件，空日志恢复也包括在内。seq 期望会随这条边界移动。两处更新是承重的而非机械的：telemetry 的接管测试断言该边界*会*被导出，因为它是本生命周期的自有写入；属性测试套件的回放不变式则是「种子逐字节复现，外加一个仅日志边界」，并把幂等性作为独立属性。 +代价：带种子会话的日志长了一个事件，空日志恢复也包括在内。seq 期望会随这条边界移动。两处更新是承重的而非机械的：telemetry 的接管测试断言该边界会作为完整权威日志回放的一部分导出，包括它经由 resume constructor seed 进入时；属性测试套件的回放不变式则是「种子逐字节复现，外加一个仅日志边界」，并把幂等性作为独立属性。 -`session/end-seed` 加入了落盘词汇表。在预发布立场下（`SESSION_FORMAT_VERSION` 固定为 `0`，不作兼容承诺），更旧的日志只是没有它，而没有边界的日志会正确地判定没有任何内容属于构造种子历史。 +`session/end-seed` 加入了落盘词汇表。当前 v1 要求由 Session 拥有的已校验 marker 语义；冻结的 v0 codec 与迁移边负责哪些历史 v0 seed 布局仍可接受。精确继承 cut 与逻辑 header 分离，并在读取正文后可用。 [排队手动压缩决策](../feature/2026-07-30-queued-manual-compaction.zh.md)如今提供了第一个消费方。其尾部扫描会分别查找未匹配的 `compaction/start` 与最新 end-seed，只把位于该边界之后的 start 视为存活，并在同一个回放转换上清除不变量追踪状态。该谓词仍位于压缩功能所在的包中，不会成为通用核心辅助函数。 diff --git a/.agents/notes/implemented/architecture/2026-08-05-session-preparation.i18n.yaml b/.agents/notes/implemented/architecture/2026-08-05-session-preparation.i18n.yaml index 3e3b610f73..faf19fe0c4 100644 --- a/.agents/notes/implemented/architecture/2026-08-05-session-preparation.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-08-05-session-preparation.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-08-05-session-preparation.md -2026-08-05-session-preparation.md: 50f1ea38e671c6aa7b0f4adaf2fecbf83decc23c -2026-08-05-session-preparation.zh.md: cd918d126b56b081bcc1b6aa43a10d662668102d +2026-08-05-session-preparation.md: a8adcc577993f749be3658ee952e087c6943f462 +2026-08-05-session-preparation.zh.md: abe805505be351ecb9dc5face3e152c2f820a6be diff --git a/.agents/notes/implemented/architecture/2026-08-05-session-preparation.md b/.agents/notes/implemented/architecture/2026-08-05-session-preparation.md index 50f1ea38e6..a8adcc5779 100644 --- a/.agents/notes/implemented/architecture/2026-08-05-session-preparation.md +++ b/.agents/notes/implemented/architecture/2026-08-05-session-preparation.md @@ -22,7 +22,7 @@ This refines the publication boundary from the [Agent lifecycle and ownership de A coordinator-backed persistence implementation loads one cold source into a prepared Session. The backend transfers fresh, mutually unaliased metadata and events together with the source-qualified revision that identifies those exact values; the Session restore path validates and freezes the graphs in place instead of cloning them. The coordinator computes interrupted-turn closers and constructs the exact unpublished Session once. Its immutable header and balanced logical event log form the `SessionInspection` borrowed by readers, while the revision remains internal to persistence. -`inspect(id, signal?)` does not mutate storage. Synthetic closers exist only in the prepared in-memory view, and a torn physical tail remains untouched. Same-id callers share an in-flight cold read. Once ready, the preparation may remain in a per-coordinator LRU whose capacity defaults to five and is configurable by first-party backends. Before reusing a retained source, the coordinator reads that id's current revision; a mismatch evicts a ready source and repeats the cold materialization. A source already committing or reserved for resume remains exclusively owned, so concurrent inspection borrows that immutable view until publication or release. +For already-current input, `inspect(id, signal?)` does not mutate storage: synthetic closers exist only in the prepared in-memory view, and a torn physical tail remains untouched. A supported historical body read first publishes its migrated and repaired current generation. Same-id callers share an in-flight cold read. Once ready, the preparation may remain in a per-coordinator LRU whose capacity defaults to five and is configurable by first-party backends. Before reusing a retained source, the coordinator reads that id's current revision; a mismatch evicts a ready source and repeats the cold materialization. A source already committing or reserved for resume remains exclusively owned, so concurrent inspection borrows that immutable view until publication or release. `prepare(id, signal?)` exclusively reserves the prepared Session. It confirms the retained revision before committing any torn-tail and interrupted-turn repair, establishes the durable cursor, then returns a disposable preparation. A stale source is discarded and reloaded instead of being repaired or published. A successful repair also discards the pre-repair source and materializes the committed log again before reservation, so a newer revision is never associated with an older event graph. Another same-id preparation waits until the reservation is published or released. Publication accepts only the exact reserved Session and attaches the committed cursor without rebuilding its history. Failed setup or cancellation returns an unchanged unpublished Session to the LRU; mutation or attachment consumes the reservation. @@ -38,7 +38,7 @@ Cold continuable-subagent access follows the same path. Descriptor authorization ## Boundaries -- `readFrom()` remains a detached physical-suffix API. It neither creates nor consumes a preparation, synthesizes logical closers, or joins the LRU. +- `readFrom()` remains a detached physical-suffix API. It neither creates nor consumes a preparation nor joins the LRU. Current input synthesizes no logical closers; historical input may first publish its repaired current generation. - HMR adoption keeps the live Session authoritative and reads the stored prefix directly. It may truncate a torn physical fragment but never closes the live open turn as interrupted. - The cache belongs to one persistence coordinator, not a process-global Session map. Live Sessions are owned by the existing stores and never occupy preparation capacity. - A fresh create never claims a cold persisted preparation with the same id. Persistence collisions continue to reject. @@ -47,7 +47,7 @@ Cold continuable-subagent access follows the same path. Descriptor authorization ## Verification -The shared persistence contract pins non-mutating balanced cold inspection and later repair. `persistence.spec.ts` and `preparations.spec.ts` pin same-id in-flight sharing, exact Session reuse across inspect and prepare, revision-triggered refresh before history and resume, single repair commit, exclusive reservation, release after failed setup, ready-entry LRU eviction, append rejection during reservation, and publication of only the reserved Session. Backend tests pin that full and lightweight reads use the same revision identity. Agent-loop and continuable-subagent tests pin the common publication pipeline and inspection-to-resume path across cancellation and teardown. +The shared persistence contract pins non-mutating balanced cold inspection for current input, historical migration-before-inspection, and later current repair. `persistence.spec.ts` and `preparations.spec.ts` pin same-id in-flight sharing, exact Session reuse across inspect and prepare, revision-triggered refresh before history and resume, single repair commit, exclusive reservation, release after failed setup, ready-entry LRU eviction, append rejection during reservation, and publication of only the reserved Session. Backend tests pin that full and lightweight reads use the same revision identity. Agent-loop and continuable-subagent tests pin the common publication pipeline and inspection-to-resume path across cancellation and teardown. ## Alternatives considered @@ -65,4 +65,4 @@ The shared persistence contract pins non-mutating balanced cold inspection and l One cold materialization can serve history pagination, subagent descriptor inspection, and a later resume. Ownership transfer removes redundant restoration clones, while the bounded per-coordinator LRU limits memory and avoids creating live Agents for queries. Create and resume share one publication protocol without merging Agent and Session responsibilities. -The first cold inspection now pays the complete validation and Session-construction cost and may retain that unpublished Session until eviction. Persistence must coordinate reservation, append, repair, and publication, and callers must treat inspection values as immutable borrowed state. Backends that rely on the default `prepare()` remain correct but do not receive the reuse optimization. +The first cold inspection now pays the complete validation and Session-construction cost and may retain that unpublished Session until eviction; historical input also pays one durable migration and repair publication first. Persistence must coordinate reservation, append, repair, and publication, and callers must treat inspection values as immutable borrowed state. Backends that rely on the default `prepare()` remain correct but do not receive the reuse optimization. diff --git a/.agents/notes/implemented/architecture/2026-08-05-session-preparation.zh.md b/.agents/notes/implemented/architecture/2026-08-05-session-preparation.zh.md index cd918d126b..abe805505b 100644 --- a/.agents/notes/implemented/architecture/2026-08-05-session-preparation.zh.md +++ b/.agents/notes/implemented/architecture/2026-08-05-session-preparation.zh.md @@ -22,7 +22,7 @@ agent loop（智能体循环）通过同一条设置与发布流水线消费这 使用协调器的持久化实现会将一个冷源加载为准备完成的 Session。后端转移新鲜、彼此无别名的元数据和事件，以及标识这些精确值的来源限定 revision；Session 恢复路径直接验证并冻结这些对象图，不再复制。协调器计算中断轮次的 closer，并且只构造一次精确的未发布 Session。其不可变 header 与已配平的逻辑事件日志构成读取方借用的 `SessionInspection`，revision 则保留在持久化内部。 -`inspect(id, signal?)` 不修改存储。合成 closer 只存在于准备完成的内存视图中，撕裂的物理尾部保持不变。同 id 调用方共享进行中的冷读。准备完成后，该对象可以进入每个协调器自己的 LRU；第一方后端可配置容量，默认保留五个。协调器复用保留源之前会读取该 id 的当前 revision；如果不匹配，就淘汰处于就绪阶段的源并重新完成冷实体化。已经进入提交或为恢复而预留的源仍由其所有者独占，因此并发检查会借用该不可变视图，直至发布或释放。 +对于已经是当前格式的输入，`inspect(id, signal?)` 不修改存储：合成 closer 只存在于准备完成的内存视图中，撕裂的物理尾部保持不变。受支持的历史正文读取会先发布迁移并修复后的当前 generation。同 id 调用方共享进行中的冷读。准备完成后，该对象可以进入每个协调器自己的 LRU；第一方后端可配置容量，默认保留五个。协调器复用保留源之前会读取该 id 的当前 revision；如果不匹配，就淘汰处于就绪阶段的源并重新完成冷实体化。已经进入提交或为恢复而预留的源仍由其所有者独占，因此并发检查会借用该不可变视图，直至发布或释放。 `prepare(id, signal?)` 独占预留准备完成的 Session。它先确认保留的 revision，再提交撕裂尾部和中断轮次修复、建立持久游标，最后返回可 dispose 的准备对象。陈旧源会被丢弃并重新读取，不会参与修复或发布。修复成功后也会丢弃修复前的源，并在预留前重新实体化已提交日志，以免把较新的 revision 关联到较旧的事件对象图。同 id 的另一个准备请求会等待当前预留发布或释放。发布只接受精确的预留 Session，并直接附接已提交游标，无需重建历史。设置失败或取消时，未发生变化的未发布 Session 会返回 LRU；发生变更或完成附接后，系统会消费该预留。 @@ -38,7 +38,7 @@ agent loop（智能体循环）通过同一条设置与发布流水线消费这 ## 边界 -- `readFrom()` 仍是脱离的物理后缀 API。它不会创建或消费准备对象，不会合成逻辑 closer，也不会进入 LRU。 +- `readFrom()` 仍是脱离的物理后缀 API。它不会创建或消费准备对象，也不会进入 LRU。当前输入不会合成逻辑 closer；历史输入可能先发布已修复的当前 generation。 - HMR（热模块替换）接管继续以实时 Session 为权威，并直接读取已存储前缀。它可以截断撕裂的物理碎片，但绝不把实时开放轮次关闭为中断状态。 - 缓存属于单个持久化协调器，而不是进程全局 Session map。实时 Session 由现有存储持有，绝不占用准备容量。 - 新建流程绝不认领相同 id 的冷持久化准备对象。持久化冲突仍会被拒绝。 @@ -47,7 +47,7 @@ agent loop（智能体循环）通过同一条设置与发布流水线消费这 ## 验证 -共享持久化约定规定冷检查不得修改存储且须保持配平，并覆盖后续修复。`persistence.spec.ts` 与 `preparations.spec.ts` 覆盖同 id 进行中读取共享、检查与准备之间的精确 Session 复用、在历史读取与恢复前由 revision 触发刷新、修复只提交一次、独占预留、设置失败后释放、就绪项 LRU 淘汰、预留期间拒绝 append，以及只允许发布预留 Session。后端测试覆盖完整读取与轻量读取使用同一 revision 身份。agent loop 与 continuable subagent 测试覆盖统一发布流水线，以及取消和清理期间从检查到恢复的路径。 +共享持久化约定规定当前格式冷检查不修改存储且须保持配平，并覆盖历史迁移先于检查与后续当前修复。`persistence.spec.ts` 与 `preparations.spec.ts` 覆盖同 id 进行中读取共享、检查与准备之间的精确 Session 复用、在历史读取与恢复前由 revision 触发刷新、修复只提交一次、独占预留、设置失败后释放、就绪项 LRU 淘汰、预留期间拒绝 append，以及只允许发布预留 Session。后端测试覆盖完整读取与轻量读取使用同一 revision 身份。agent loop 与 continuable subagent 测试覆盖统一发布流水线，以及取消和清理期间从检查到恢复的路径。 ## 考虑过的替代方案 @@ -65,4 +65,4 @@ agent loop（智能体循环）通过同一条设置与发布流水线消费这 一次冷实体化可以同时服务历史分页、subagent descriptor 检查和后续恢复。所有权转移去除了恢复阶段的冗余复制；每个协调器的有界 LRU 限制内存占用，也避免查询创建实时 agent。新建和恢复共享同一发布协议，同时保持 agent 与 Session 职责分离。 -首次冷检查需要承担完整验证与 Session 构造成本，并可能保留该未发布 Session 直至淘汰。持久化层必须协调预留、append、修复和发布；调用方必须把检查结果视为借用的不可变状态。依赖默认 `prepare()` 的后端仍然正确，但无法获得复用优化。 +首次冷检查需要承担完整验证与 Session 构造成本，并可能保留该未发布 Session 直至淘汰；历史输入还会先承担一次持久迁移与修复发布。持久化层必须协调预留、append、修复和发布；调用方必须把检查结果视为借用的不可变状态。依赖默认 `prepare()` 的后端仍然正确，但无法获得复用优化。 diff --git a/.agents/notes/implemented/architecture/2026-08-10-message-feedback-sidecar.i18n.yaml b/.agents/notes/implemented/architecture/2026-08-10-message-feedback-sidecar.i18n.yaml index 884aa27385..3e8040a52c 100644 --- a/.agents/notes/implemented/architecture/2026-08-10-message-feedback-sidecar.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-08-10-message-feedback-sidecar.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-08-10-message-feedback-sidecar.md -2026-08-10-message-feedback-sidecar.md: eb1f8786f9ec0363b3c98d5b80a796b9c0fc0b4c -2026-08-10-message-feedback-sidecar.zh.md: 573d3b30e3bd14b492925f4a00424db588736943 +2026-08-10-message-feedback-sidecar.md: 94c607e14704a9a41a4ff0beca084c97e982d959 +2026-08-10-message-feedback-sidecar.zh.md: dbebb77febde3a111595d828005f79c552c6799b diff --git a/.agents/notes/implemented/architecture/2026-08-10-message-feedback-sidecar.md b/.agents/notes/implemented/architecture/2026-08-10-message-feedback-sidecar.md index eb1f8786f9..94c607e147 100644 --- a/.agents/notes/implemented/architecture/2026-08-10-message-feedback-sidecar.md +++ b/.agents/notes/implemented/architecture/2026-08-10-message-feedback-sidecar.md @@ -16,7 +16,7 @@ A sidecar keyed only by `SessionId` can outlive the log lifecycle it describes w Every usable row is bound to the inspected Session header identity `{createdAt, cwd}`, not merely its `SessionId`. A lifecycle mismatch is treated as absence: `list` returns no items, and `put` may replace the stale row with one bound to the current identity. An id reused with a different header identity therefore cannot inherit stale feedback. A fork receives its own Session identity and no sidecar copy: even when the fork seed contains the same assistant messages, feedback remains attached to the Session in which the human recorded it. -`put` accepts a target only when `SessionPersistence.inspect()` observes a non-empty, append-origin `assistant/message` with that `MessageId`. Replacement-origin messages, empty usage-only assistant records, and non-assistant targets are rejected. Inspection is the cold-safe authority: it neither publishes or resumes an Agent nor commits cold-log repair merely to validate feedback. A cold `listSnapshots()` preflight classifies definite absence; inspection failure for a catalogued Session remains an infrastructure failure. A request in the narrow live-detach-to-header-materialization interval can therefore return `session-not-found`, and the caller retries after retirement materialization. +`put` accepts a target only when `SessionPersistence.inspect()` observes a non-empty, append-origin `assistant/message` with that `MessageId`. Replacement-origin messages, empty usage-only assistant records, and non-assistant targets are rejected. Inspection is the cold-safe authority: it never publishes or resumes an Agent; already-current repair stays in memory, while a supported historical body read may first publish migration and repair. A cold `listSnapshots()` preflight classifies definite absence; inspection failure for a catalogued Session remains an infrastructure failure. A request in the narrow live-detach-to-header-materialization interval can therefore return `session-not-found`, and the caller retries after retirement materialization. Before `put` commits a sidecar row, it puts the target log behind a durability barrier. A matching live Session passes through the canonical `ctx.sessions.flush` checkpoint, then both live and cold paths are physically read from sequence zero through `SessionPersistence.readFrom`. The resulting observation's header identity and target are checked again. A missing flush participant, changed identity, vanished target, or physical-read failure prevents the sidecar write, so a committed feedback item never precedes the durable assistant message it references. diff --git a/.agents/notes/implemented/architecture/2026-08-10-message-feedback-sidecar.zh.md b/.agents/notes/implemented/architecture/2026-08-10-message-feedback-sidecar.zh.md index 573d3b30e3..dbebb77feb 100644 --- a/.agents/notes/implemented/architecture/2026-08-10-message-feedback-sidecar.zh.md +++ b/.agents/notes/implemented/architecture/2026-08-10-message-feedback-sidecar.zh.md @@ -16,7 +16,7 @@ Status: implemented 每条可用记录都绑定到经检查的 Session header 身份 `{createdAt, cwd}`，而不只是其 `SessionId`。生命周期不匹配按不存在处理：`list` 返回空条目，`put` 可以用绑定当前身份的新记录替换陈旧行。因此，以不同 header 身份复用的 id 不会继承陈旧反馈。fork 拥有自己的 Session 身份，且不复制伴随记录：即使 fork 种子包含相同的 assistant 消息，反馈仍只属于人类记录它的那个 Session。 -`put` 只接受由 `SessionPersistence.inspect()` 观测到的非空、append-origin `assistant/message`，且其 `MessageId` 必须与目标相同。replacement-origin 消息、仅承载 usage 的空 assistant 记录以及非 assistant 目标都会被拒绝。检查使用 cold-safe 权威路径：它不会仅为验证反馈而发布或恢复 Agent，也不会提交 cold 日志修复。cold 路径由 `listSnapshots()` 预检明确不存在；已进入目录的 Session 若检查失败，仍按基础设施故障处理。因此，请求若恰落在 live detach 到 header materialization 的极短窗口，可能返回 `session-not-found`，调用方在 retirement materialization 后重试。 +`put` 只接受由 `SessionPersistence.inspect()` 观测到的非空、append-origin `assistant/message`，且其 `MessageId` 必须与目标相同。replacement-origin 消息、仅承载 usage 的空 assistant 记录以及非 assistant 目标都会被拒绝。检查使用 cold-safe 权威路径：它绝不发布或恢复 Agent；已经是当前格式的修复留在内存中，受支持的历史正文读取则可能先发布迁移与修复。cold 路径由 `listSnapshots()` 预检明确不存在；已进入目录的 Session 若检查失败，仍按基础设施故障处理。因此，请求若恰落在 live detach 到 header materialization 的极短窗口，可能返回 `session-not-found`，调用方在 retirement materialization 后重试。 `put` 提交伴随记录前，会先让目标日志通过 durability barrier。身份匹配的 live Session 经过权威 `ctx.sessions.flush` checkpoint，随后 live 与 cold 路径都会通过 `SessionPersistence.readFrom` 从序列零做物理复读。之后再次校验所得观测的 header 身份与目标。缺少 flush 参与方、身份变化、目标消失或物理读取失败都会阻止伴随记录写入，因此已提交反馈绝不会先于它引用的持久 assistant 消息。 diff --git a/.agents/notes/implemented/architecture/2026-08-10-session-log-version-mechanism.i18n.yaml b/.agents/notes/implemented/architecture/2026-08-10-session-log-version-mechanism.i18n.yaml index a36497c9e7..ae65d43b61 100644 --- a/.agents/notes/implemented/architecture/2026-08-10-session-log-version-mechanism.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-08-10-session-log-version-mechanism.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-08-10-session-log-version-mechanism.md -2026-08-10-session-log-version-mechanism.md: 0d4c9e73acc6abd4a67123e3d7b0e4f94e0b5a23 -2026-08-10-session-log-version-mechanism.zh.md: 6c57da618a09c1d423323940ea36dbd4caccde02 +2026-08-10-session-log-version-mechanism.md: 19650ca6a02139d6ad138c6914185f210386ce49 +2026-08-10-session-log-version-mechanism.zh.md: a54a1f30d2d30e55b7c03d529e1ca337346ce2d2 diff --git a/.agents/notes/implemented/architecture/2026-08-10-session-log-version-mechanism.md b/.agents/notes/implemented/architecture/2026-08-10-session-log-version-mechanism.md index 0d4c9e73ac..19650ca6a0 100644 --- a/.agents/notes/implemented/architecture/2026-08-10-session-log-version-mechanism.md +++ b/.agents/notes/implemented/architecture/2026-08-10-session-log-version-mechanism.md @@ -14,17 +14,17 @@ Session logs must be upgradable after release, and the runtime that ships first **The writer decides bumps, not the reader.** A bump is required exactly when an old runtime could no longer handle a new log with full semantic correctness. "Parses without error" is not the bar: silently skipping content that shapes reconstruction is a wrong read. Only structural changes qualify — header shape, event envelope, core event semantics, the surface mechanism (`SurfaceEventType` set, `SurfaceOp` variants). When unsure, bump: a near-identity upgrader is almost free, a missed bump silently corrupts old readers. -**Read rules by direction.** Equal version: read normally. Newer than the reader: refuse, name the direction ("written by a newer harness — upgrade"), and point at the raw log artifact so the user can still see the text (`SessionFormatUnsupportedError`, distinct from `SessionPersistenceCorruptionError` because nothing is damaged). Older than the reader: convert in memory through the chain of n→n+1 upgraders for viewing; persist the converted log only when the session is actually continued (atomic temp-file replace, original kept as backup). A step whose upgrader cannot be written is left empty, which cuts off every version at or below it — those degrade to raw-text viewing. +**Read rules by direction.** Equal version: read normally. Newer than the reader: refuse, name the direction ("written by a newer harness — upgrade"), and point at the raw log artifact so the user can still see the text (`SessionFormatUnsupportedError`, distinct from `SessionPersistenceCorruptionError` because nothing is damaged). Older than the reader: every event-body operation first runs the complete adjacent chain in memory, leaves the source path, bytes, and inode unchanged, exclusively publishes only the final current generation under its canonical versioned filename, and reopens it before current restoration. Header-only listing remains non-mutating and reports the numerically highest canonical generation. Catalog generation and module initialization reject a missing adjacent step, so a published first-party build never exposes a partial historical chain. Retained lower generations are not automatic fallback or a downgrade compatibility promise. **A per-event `ignorable` marker covers vocabulary growth, so ordinary event additions never bump the version.** The event vocabulary is decided by which plugins are mounted, which a single version integer cannot describe. A reader meeting an unrecognized event type refuses to interpret the log unless the event carries `ignorable: true` in its envelope. The default is *required*: forgetting the marker over-refuses a resumable session (an inconvenience), while a default of ignorable would make the same mistake silently resume a gutted one (a safety failure). The architecture makes this sound: model-visible content flows only through the three `surfaceOp`-marked surface event types plus the `request/header`/`request/context` folds, so the dangerous unknowns are exactly the non-surface events that change how the rest of the log is read (`session/end-seed` is the existing example). ## Consequences -What shipped in v0 (release 0812): direction-aware refusal with the raw-log path; the unknown-event guard against a generated known-vocabulary list (`KNOWN_SESSION_EVENT_TYPES`, emitted by `gen-persistence-catalog` from every `SessionEventMap` merge and kept fresh by `verify-persistence-catalog`); the `ignorable` envelope field accepted by seed validation, JSONL, and the BFF wire schema. The upgrader chain itself is deferred until the first real v0→v1 step exists to test it against. First-party writers do not set `ignorable` through `Session.append`, while a repository-external plugin is a current consumer; its retention and replacement condition lives in the [external-plugin retention decision](2026-08-30-retain-ignorable-external-session-events.md). An external informational event carrying the marker remains reloadable, while an unknown required event refuses resume. The unknown-type guard is read-side only: `appendCore` keeps rejecting retired legacy shapes but does not vocabulary-check new types, because an append-time refusal would stall a live session's durability mid-flight, which costs more than a loud refusal at the log's next load. The JSONL provider refuses a foreign version from the raw header line before validating this format version's header or decoding any event row, so a structurally different future format still reports the upgrade direction instead of "corrupt". +What shipped in v0 (release 0812): direction-aware refusal with the raw-log path; the unknown-event guard against a generated known-vocabulary list (`KNOWN_SESSION_EVENT_TYPES`, emitted by `gen-persistence-catalog` from every `SessionEventMap` merge and kept fresh by `verify-persistence-catalog`); the `ignorable` envelope field accepted by seed validation, JSONL, and the BFF wire schema. V1 adds the static adjacent catalog, the identity v0-to-v1 edge, header-only descriptors, exact-generation JSONL publication, and current-only restoration described in [Released Session formats](2026-08-31-released-session-format-migrations.md). First-party writers do not set `ignorable` through `Session.append`, while a repository-external plugin is a current consumer; equal-version retention lives in the [external-plugin retention decision](2026-08-30-retain-ignorable-external-session-events.md), and the stricter historical rule lives in the [alpha migration refusal decision](2026-08-31-alpha-historical-unknown-event-refusal.md). The unknown-type guard remains read-side because append-time vocabulary refusal would stall a live session's durability. JSONL classifies foreign versions from the minimal raw header before current-header or event parsing, so a structurally different future format reports the upgrade direction instead of "corrupt". ## Alternatives considered - **Major/minor versioning** — the "is it convertible" bit lives on each step's upgrader, and pre-committing it into a number shape invites wrong promises. - **Default-ignorable unknown events** — inverts the failure mode of a forgotten marker from visible over-refusal into silent corruption. -- **Auto-migrating on view** — rewriting the artifact on open turns a read into a destructive write: a converter bug corrupts logs at browse time, and a same-directory older runtime loses access because a newer one merely looked. +- **Migrating during header-only listing** — makes cheap inventory mutate storage and requires event bodies to compute facts that a header cannot prove. Listing returns descriptors; event-body reads own publication. - **Per-plugin runtime registration of known event types** — rejected because it would make the known set composition-dependent and register event names without classifying whether omission is safe. The persisted `ignorable` marker keeps that classification with each record; the [external-plugin retention decision](2026-08-30-retain-ignorable-external-session-events.md) owns the current consumer constraint. diff --git a/.agents/notes/implemented/architecture/2026-08-10-session-log-version-mechanism.zh.md b/.agents/notes/implemented/architecture/2026-08-10-session-log-version-mechanism.zh.md index 6c57da618a..a54a1f30d2 100644 --- a/.agents/notes/implemented/architecture/2026-08-10-session-log-version-mechanism.zh.md +++ b/.agents/notes/implemented/architecture/2026-08-10-session-log-version-mechanism.zh.md @@ -14,17 +14,17 @@ Session log 在发布后必须能升级格式，而最先发布的运行时决 **升不升版本由写入方决定，与读取方能力无关。**当且仅当老运行时无法在语义上完全正确地处理新日志时才必须升版本。"解析不报错"不是标准：静默跳过影响重建的内容就是读错。只有结构性变更够得上这条线：header 形状、事件信封、核心事件语义、surface 机制（`SurfaceEventType` 集合、`SurfaceOp` 变体）。拿不准就升：近似恒等的升级器几乎没有成本，漏升一次会让老读取器静默读坏。 -**读取规则按方向区分。**版本相等：正常读。比读取器新：拒绝，说明方向（"由更新的 harness 写入，请升级"），并给出原始日志文件的路径，用户仍能看到文本（`SessionFormatUnsupportedError`，与 `SessionPersistenceCorruptionError` 区分，因为数据没有损坏）。比读取器旧：查看时经 n→n+1 升级器链在内存中逐级转换；只有会话真正被继续时才把转换落盘（临时文件原子替换，原文件留备份）。写不出升级器的那一步留空，这会切断该步及更早所有版本的升级路径，它们降级为只能看原文。 +**读取规则按方向区分。**版本相等：正常读。比读取器新：拒绝，说明方向（"由更新的 harness 写入，请升级"），并给出原始日志文件的路径，用户仍能看到文本（`SessionFormatUnsupportedError`，与 `SessionPersistenceCorruptionError` 区分，因为数据没有损坏）。比读取器旧：每个事件正文操作先在内存中运行完整相邻链，保持源路径、字节与 inode 不变，只在规范具名版本文件下排他发布最终当前 generation，再在当前恢复前重新打开。仅 header 的列表保持不变更，并报告数值最高的规范 generation。catalog 生成与模块初始化会拒绝缺失的相邻步骤，因此已发布第一方 build 绝不会暴露不完整历史链。保留的低 generation 不是自动 fallback，也不构成 downgrade compatibility 承诺。 **逐事件的 `ignorable` 标记吸收词汇表增长，普通的新增事件永远不用升版本。**事件词汇表由挂载了哪些插件决定，单个版本整数描述不了它。读取器遇到不认识的事件类型时拒绝解读日志，除非该事件的信封带 `ignorable: true`。默认为必需：忘写标记的后果是把一个本可恢复的会话拒绝过头（体验问题），而默认可忽略会让同样的疏忽静默恢复出残缺会话（安全事故）。架构保证了这条规则成立：模型可见内容只经三种带 `surfaceOp` 标记的 surface 事件加 `request/header`、`request/context` 折叠进入重建，危险的未知事件恰好是那些不进 surface 但改变日志其余部分解读方式的事件（`session/end-seed` 是现存例子）。 ## 影响 -v0（0812 发布）交付的内容：分方向的拒绝并带原始日志路径；基于生成的已知词汇清单（`KNOWN_SESSION_EVENT_TYPES`，由 `gen-persistence-catalog` 从所有 `SessionEventMap` 声明合并生成，`verify-persistence-catalog` 保证新鲜）的未知事件守卫；`ignorable` 信封字段被种子校验、JSONL 和 BFF 线上 schema 接受。升级器链本身推迟到第一个真实的 v0→v1 变更出现、有真实对象可测时再建。第一方写入方不通过 `Session.append` 设置 `ignorable`，但当前有一个仓库外插件依赖该字段；其保留条件与替代机制要求由[外部插件保留决策](2026-08-30-retain-ignorable-external-session-events.zh.md)定义。带该标记的外部信息性事件可以继续重新加载，未知必需事件则会拒绝恢复。未知类型守卫只在读取侧生效：`appendCore` 继续拒绝已淘汰的 legacy 形状，但不对新类型做词汇检查，因为写入时拒绝会让活跃会话的持久化中途停摆，代价大于下次加载时的显式拒绝。JSONL provider 会在校验本格式版本的 header、解码任何事件行之前，直接从原始 header 行拒绝外来版本，因此结构完全不同的未来格式仍会报告升级方向而不是"损坏"。 +v0（0812 发布）交付的内容：分方向的拒绝并带原始日志路径；基于生成的已知词汇清单（`KNOWN_SESSION_EVENT_TYPES`，由 `gen-persistence-catalog` 从所有 `SessionEventMap` 声明合并生成，`verify-persistence-catalog` 保证新鲜）的未知事件守卫；`ignorable` 信封字段被种子校验、JSONL 和 BFF 线上 schema 接受。V1 添加静态相邻 catalog、恒等 v0-to-v1 迁移边、仅 header descriptor、精确代际 JSONL 发布与[已发布 Session 格式](2026-08-31-released-session-format-migrations.zh.md)定义的当前专用恢复。第一方 writer 不通过 `Session.append` 设置 `ignorable`，而一个仓库外插件仍依赖该字段；同版本保留由[外部插件保留决策](2026-08-30-retain-ignorable-external-session-events.zh.md)定义，更严格的历史规则由 [alpha 迁移拒绝决策](2026-08-31-alpha-historical-unknown-event-refusal.zh.md)定义。未知类型守卫仍只在读取侧生效，因为 append 时的词汇拒绝会中断活跃 Session 的持久化。JSONL 会在当前 header 或事件解析前从最小原始 header 分类外来版本，因此结构完全不同的未来格式会报告升级方向而不是"损坏"。 ## 曾考虑的替代方案 - **大小两级版本号**：能否转换这一位信息属于每一步的升级器，把它预先固化进编号形状会做出错误承诺。 - **未知事件默认可忽略**：把忘写标记的后果从可见的过度拒绝反转成静默损坏。 -- **查看时自动迁移落盘**：打开即改写把读操作变成破坏性写操作，转换器的 bug 会在浏览时损坏日志，同目录的旧版本运行时也会因为新版本只是看了一眼就失去访问能力。 +- **在仅 header 列表期间迁移**：让便宜清单改变存储，而且需要读取事件正文才能计算 header 无法证明的事实。列表返回 descriptor，事件正文读取负责发布。 - **插件运行时注册已知事件类型**：不予采用，因为该方案会让已知集依赖插件组合，而且只注册事件名称，无法判定省略事件是否安全。持久化的 `ignorable` 标记把该分类保留在每条记录中；[外部插件保留决策](2026-08-30-retain-ignorable-external-session-events.zh.md)定义当前消费方约束。 diff --git a/.agents/notes/implemented/architecture/2026-08-25-rename-code-mode-to-ptc.i18n.yaml b/.agents/notes/implemented/architecture/2026-08-25-rename-code-mode-to-ptc.i18n.yaml index c6d3883ea8..38435b4fcf 100644 --- a/.agents/notes/implemented/architecture/2026-08-25-rename-code-mode-to-ptc.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-08-25-rename-code-mode-to-ptc.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-08-25-rename-code-mode-to-ptc.md -2026-08-25-rename-code-mode-to-ptc.md: 618167516aefc54445d37cb1ce3939419e707bf5 -2026-08-25-rename-code-mode-to-ptc.zh.md: d6cf5cdea1154bd2b8cb424653b76315bb20b05d +2026-08-25-rename-code-mode-to-ptc.md: 10c81fdca8acc67ab0e5e3b29c785ab4aec1d925 +2026-08-25-rename-code-mode-to-ptc.zh.md: 881ea86030ae09f198f12ac769d5ccd07159dba2 diff --git a/.agents/notes/implemented/architecture/2026-08-25-rename-code-mode-to-ptc.md b/.agents/notes/implemented/architecture/2026-08-25-rename-code-mode-to-ptc.md index 618167516a..10c81fdca8 100644 --- a/.agents/notes/implemented/architecture/2026-08-25-rename-code-mode-to-ptc.md +++ b/.agents/notes/implemented/architecture/2026-08-25-rename-code-mode-to-ptc.md @@ -21,7 +21,7 @@ Renamed in this PR: - prompt rule `tools:code-only` → `tools:ptc-only` - prose "Code Mode" → "PTC mode" / "PTC 模式" in docs, READMEs, and the eight implemented Agent Notes whose topic names the feature (those files were renamed in place) -Deferred to the stacked persistence PR: the session-persistent vocabulary — the durable event types `tool/code-dispatch` / `tool/code-dispatch-start`, the logged plugin name `tools-code-mode`, and the sub-call id segment `:code:`. That PR is blocked until the `SESSION_FORMAT_VERSION` v0→v1 migration lands with it. +The session-persistent vocabulary remains deferred: the durable event types `tool/code-dispatch` / `tool/code-dispatch-start`, the logged plugin name `tools-code-mode`, and the sub-call id segment `:code:`. Renaming those values is a structural Session-format change and requires its own adjacent edge after the identity v0-to-v1 foundation. Kept unchanged: `run_code` and its `code` parameter (they name the program payload, not the mode), `CodeSdkLanguage`, `CodeRunFailedError`, the `dsh-code-runtime*` package family, the third-party `codex-code-mode-host` binary name, and every frozen archived note. @@ -30,8 +30,8 @@ Kept unchanged: `run_code` and its `code` parameter (they name the program paylo - **`ptc-mode` identifiers** — rejected: PTC is a tool-presentation transport, not a mode in the plan-mode sense, and the identifier should not claim that kinship. - **Surface-only rename** — rejected: the pre-release stance updates every reference together. - **Renaming `run_code` too** — rejected: the tool name describes running a program, not the mode, and is model-facing API surface. -- **Renaming the durable event vocabulary in this PR** — rejected: renaming `tool/code-dispatch*` without a format bump would make pre-rename session logs unreadable; that rename belongs to the stacked persistence PR that lands together with the v0→v1 migration. +- **Renaming the durable event vocabulary without an adjacent edge** — rejected: renaming `tool/code-dispatch*` in place would make pre-rename Session logs unreadable; that rename requires a later structural format version and explicit migration. ## Consequences -Configs with `mode: code` and preset ids `code` are unsupported on this build. The session-persistent vocabulary still says `tool/code-dispatch*`, `tools-code-mode`, and `:code:`, so existing session logs load unchanged and no `SESSION_FORMAT_VERSION` bump is needed yet. The stacked persistence PR renames that vocabulary and is blocked until the v0→v1 migration lands with it (the version mechanics are in the [session-log versioning note](2026-08-10-session-log-version-mechanism.md)). Keyless snapshot refreshes carry this PR's vocabulary; the persistence PR refreshes the dispatch-bearing fixtures. The shipped decision this note renames is [the PTC foundation note](../feature/2026-06-15-ptc.md). +Configs with `mode: code` and preset ids `code` are unsupported on this build. The session-persistent vocabulary still says `tool/code-dispatch*`, `tools-code-mode`, and `:code:`; the identity v0-to-v1 edge preserves those values, so no structural version change belongs to this rename. A later adjacent edge must rename that vocabulary and refresh the dispatch-bearing fixtures ([version mechanics](2026-08-10-session-log-version-mechanism.md)). The shipped decision this note renames is [the PTC foundation note](../feature/2026-06-15-ptc.md). diff --git a/.agents/notes/implemented/architecture/2026-08-25-rename-code-mode-to-ptc.zh.md b/.agents/notes/implemented/architecture/2026-08-25-rename-code-mode-to-ptc.zh.md index d6cf5cdea1..881ea86030 100644 --- a/.agents/notes/implemented/architecture/2026-08-25-rename-code-mode-to-ptc.zh.md +++ b/.agents/notes/implemented/architecture/2026-08-25-rename-code-mode-to-ptc.zh.md @@ -21,7 +21,7 @@ Status: implemented - 提示词规则 `tools:code-only` → `tools:ptc-only` - 文档、README 与八个以该功能命名的 implemented Agent Note 中的文案 "Code Mode" → "PTC mode"／"PTC 模式"（这些 Note 文件一并就地改名） -延后到堆叠的持久化 PR：会话持久词汇——持久事件类型 `tool/code-dispatch`／`tool/code-dispatch-start`、日志中的插件名 `tools-code-mode`、子调用 id 段 `:code:`。该 PR 被阻塞，直到 `SESSION_FORMAT_VERSION` v0→v1 迁移与其一同落地。 +会话持久词汇继续延后处理：持久事件类型 `tool/code-dispatch`／`tool/code-dispatch-start`、日志中的插件名 `tools-code-mode`、子调用 id 段 `:code:`。重命名这些值属于结构性 Session 格式变更，必须在恒等 v0-to-v1 基础之后拥有自己的相邻迁移边。 保持不变：`run_code` 及其 `code` 参数（它们描述程序载荷，而非模式）、`CodeSdkLanguage`、`CodeRunFailedError`、`dsh-code-runtime*` 包族、第三方二进制名 `codex-code-mode-host`，以及所有冻结的 archived Note。 @@ -30,8 +30,8 @@ Status: implemented - **使用 `ptc-mode` 标识符**——否决：PTC 是工具呈现传输层，不是 plan-mode 意义上的模式，标识符不应宣示这种亲缘关系。 - **仅重命名表面**——否决：预发布立场要求一次性更新所有引用。 - **连 `run_code` 一起改名**——否决：该工具名描述的是运行程序，不是模式，而且是对模型可见的 API 表面。 -- **在本 PR 中一并重命名持久事件词汇**——否决：在没有格式版本提升的情况下重命名 `tool/code-dispatch*` 会让更名前的会话日志无法读取；该重命名属于与 v0→v1 迁移一同落地的堆叠持久化 PR。 +- **不提供相邻迁移边就重命名持久事件词汇**——否决：就地重命名 `tool/code-dispatch*` 会让更名前的 Session 日志无法读取；该重命名需要后续结构格式版本与显式迁移。 ## 后果 -配置中写 `mode: code`、预设 id 为 `code`，在本构建上不再受支持。会话持久词汇仍为 `tool/code-dispatch*`、`tools-code-mode` 与 `:code:`，因此既有会话日志照常读取，无需 `SESSION_FORMAT_VERSION` 提升。堆叠的持久化 PR 负责重命名该词汇，并被阻塞到 v0→v1 迁移与其一同落地（版本机制见 [Session log 版本 Note](2026-08-10-session-log-version-mechanism.zh.md)）。无密钥的 snapshot refresh 携带本 PR 的词汇；持久化 PR 刷新包含分发的夹具。本 Note 所更名的已发布决策是 [PTC 基础 Note](../feature/2026-06-15-ptc.zh.md)。 +配置中写 `mode: code`、预设 id 为 `code`，在本构建上不再受支持。会话持久词汇仍为 `tool/code-dispatch*`、`tools-code-mode` 与 `:code:`；恒等 v0-to-v1 迁移边会保留这些值，因此本次更名不包含结构版本变更。后续相邻迁移边必须重命名该词汇并刷新包含分发的 fixture（参见[版本机制](2026-08-10-session-log-version-mechanism.zh.md)）。本 Note 所更名的已发布决策是 [PTC 基础 Note](../feature/2026-06-15-ptc.zh.md)。 diff --git a/.agents/notes/implemented/architecture/2026-08-30-retain-ignorable-external-session-events.i18n.yaml b/.agents/notes/implemented/architecture/2026-08-30-retain-ignorable-external-session-events.i18n.yaml index 8c92e95dac..33188729f5 100644 --- a/.agents/notes/implemented/architecture/2026-08-30-retain-ignorable-external-session-events.i18n.yaml +++ b/.agents/notes/implemented/architecture/2026-08-30-retain-ignorable-external-session-events.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-08-30-retain-ignorable-external-session-events.md -2026-08-30-retain-ignorable-external-session-events.md: c796a8dab1a9d127473a341fc98bc9934429fbdf -2026-08-30-retain-ignorable-external-session-events.zh.md: 0c635b1082a31a0a35d01669ff9f933a1f218bee +2026-08-30-retain-ignorable-external-session-events.md: 60d78d452e854f9c82e6edb739bdd1060b6376d3 +2026-08-30-retain-ignorable-external-session-events.zh.md: cf845ef1e6fad20ee6cc9059436d6cf7a1d75ed7 diff --git a/.agents/notes/implemented/architecture/2026-08-30-retain-ignorable-external-session-events.md b/.agents/notes/implemented/architecture/2026-08-30-retain-ignorable-external-session-events.md index c796a8dab1..60d78d452e 100644 --- a/.agents/notes/implemented/architecture/2026-08-30-retain-ignorable-external-session-events.md +++ b/.agents/notes/implemented/architecture/2026-08-30-retain-ignorable-external-session-events.md @@ -16,6 +16,8 @@ The canonical `SessionEvent` envelope retains `ignorable?: true`, and every repr The field is removable only after a replacement supports the current third-party plugin across event production, persistence, reload, and transport, with an explicit cutover for sessions already containing the marker. The [session log versioning decision](2026-08-10-session-log-version-mechanism.md) continues to own the default-required safety rule and format-version policy. +Historical format migration is deliberately stricter in the alpha implementation. The v0-to-v1 edge refuses every unknown v0 type, including an ignorable one, because an opaque payload may contain references that a format edge cannot validate. The [alpha historical-event decision](2026-08-31-alpha-historical-unknown-event-refusal.md) owns that bounded exception; equal-version append and reload continue to follow this note. + ## Alternatives considered **Require every unknown event on read.** Rejected because the current third-party plugin emits an informational event outside the repository-generated vocabulary. A first-party reload would reject that session even though omitting the event is safe. diff --git a/.agents/notes/implemented/architecture/2026-08-30-retain-ignorable-external-session-events.zh.md b/.agents/notes/implemented/architecture/2026-08-30-retain-ignorable-external-session-events.zh.md index 0c635b1082..cf845ef1e6 100644 --- a/.agents/notes/implemented/architecture/2026-08-30-retain-ignorable-external-session-events.zh.md +++ b/.agents/notes/implemented/architecture/2026-08-30-retain-ignorable-external-session-events.zh.md @@ -16,6 +16,8 @@ Status: implemented 只有替代机制在事件生产、持久化、重新加载与传输中都支持当前第三方插件，并为已包含该标记的会话提供显式切换方案后，才能删除此字段。[Session log 版本决策](2026-08-10-session-log-version-mechanism.zh.md)继续定义默认读取必需的安全规则与格式版本策略。 +Alpha 实现中的历史格式迁移有意更严格。v0-to-v1 迁移边会拒绝每个未知 v0 类型，包括 ignorable 类型，因为不透明 payload 可能包含格式迁移边无法校验的引用。[Alpha 历史事件决策](2026-08-31-alpha-historical-unknown-event-refusal.zh.md)定义该有限例外；同版本 append 与 reload 继续遵循本记录。 + ## 曾考虑的替代方案 **要求读取所有未知事件。** 不予采用，因为当前第三方插件会发出仓库生成词汇之外的信息性事件。即使省略该事件是安全的，第一方重新加载仍会拒绝该会话。 diff --git a/.agents/notes/implemented/architecture/2026-08-31-alpha-historical-unknown-event-refusal.i18n.yaml b/.agents/notes/implemented/architecture/2026-08-31-alpha-historical-unknown-event-refusal.i18n.yaml new file mode 100644 index 0000000000..fd26357a60 --- /dev/null +++ b/.agents/notes/implemented/architecture/2026-08-31-alpha-historical-unknown-event-refusal.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-08-31-alpha-historical-unknown-event-refusal.md +2026-08-31-alpha-historical-unknown-event-refusal.md: a052c76e37ff3d4649f3fdd1e172a2f569ade857 +2026-08-31-alpha-historical-unknown-event-refusal.zh.md: ac6fd0631187735e7ddbdda0ca90f12c8417291c diff --git a/.agents/notes/implemented/architecture/2026-08-31-alpha-historical-unknown-event-refusal.md b/.agents/notes/implemented/architecture/2026-08-31-alpha-historical-unknown-event-refusal.md new file mode 100644 index 0000000000..a052c76e37 --- /dev/null +++ b/.agents/notes/implemented/architecture/2026-08-31-alpha-historical-unknown-event-refusal.md @@ -0,0 +1,37 @@ +# Agent Note: Alpha Session migration refuses every unknown historical event + +Status: implemented + +English | [中文](2026-08-31-alpha-historical-unknown-event-refusal.zh.md) + +## Problem + +Equal-version Session reading can safely skip an unknown event only when its producer marked the envelope `ignorable: true`. A cardinality-preserving migration has a stricter obligation: it must prove that every preserved payload remains semantically valid in the target generation. An unknown JSON payload may contain Session sequence numbers, lifecycle facts, or model-visible state that compile-time brands cannot discover. + +Silently copying such an event can leave stale numeric references after a later edge changes event positions. Silently omitting it loses durable data. Retaining the exact immutable v0 generation does not make either transformed v1 result lossless. + +## Decision + +The alpha v0-to-v1 edge owns a frozen complete released-v0 event and payload inventory. It refuses every unknown historical event type before target staging, including an event marked `ignorable: true`, and refuses unexpected members of known payloads except fields explicitly classified as owner-opaque JSON. The diagnostic names the event type, its sequence number, and the unchanged source generation. + +The rule applies only while crossing a historical format edge. Ordinary current-format reading retains the established envelope behavior: an unknown required event refuses, while an unknown event carrying `ignorable: true` remains readable. New v1 external events therefore keep the existing equal-version extension seam, but they do not become implicitly migratable by a future format edge. + +Every first-party source event type has an executable disposition and target validator in the edge package. The catalog is build-static and profile-independent, so mounting or omitting the producer plugin cannot change whether an old artifact migrates. + +## Consequences + +Some v0 Sessions produced by repository-external informational plugins may refuse alpha migration even though the v0 codec can decode them. Refusal publishes no successor, so the suffixless v0 path, bytes, and inode remain authoritative and unchanged. Operators can identify the blocking type from the diagnostic and retain full access to its raw text. + +Community feedback will determine the next policy. A later release may add an explicit external-owner migration interface, permit omission of explicitly ignorable historical events while retaining the exact source generation, or keep strict refusal. No option is implied by the alpha marker. + +`SessionSeq` and `SessionLogOffset` make known first-party numeric fields auditable, but they cannot classify numbers inside an unknown runtime object. The migration rule therefore cannot infer safety from the absence of a recognized branded field. + +This note supersedes [Retain ignorable external Session events](2026-08-30-retain-ignorable-external-session-events.md) only for historical format migration. That decision remains current for equal-version append and reload. + +## Alternatives considered + +- **Copy unknown ignorable events verbatim** — preserves bytes but cannot prove that opaque numeric or lifecycle facts remain valid after structural edges. +- **Drop unknown ignorable events** — keeps migration available but is not lossless and makes the marker authorize data deletion. +- **Search unknown JSON for number-like field names** — heuristics cannot establish semantic identity and create false confidence. +- **Dynamically ask mounted plugins** — makes migration availability depend on one deployment composition and fails before an absent producer can mount. +- **Refuse only when the first structural edge ships** — would let v1 contain historical values whose safe interpretation was never established; the identity rehearsal is the point where the policy must become executable. diff --git a/.agents/notes/implemented/architecture/2026-08-31-alpha-historical-unknown-event-refusal.zh.md b/.agents/notes/implemented/architecture/2026-08-31-alpha-historical-unknown-event-refusal.zh.md new file mode 100644 index 0000000000..ac6fd06311 --- /dev/null +++ b/.agents/notes/implemented/architecture/2026-08-31-alpha-historical-unknown-event-refusal.zh.md @@ -0,0 +1,37 @@ +# Agent Note: Alpha Session 迁移拒绝所有未知历史事件 + +Status: implemented + +[English](2026-08-31-alpha-historical-unknown-event-refusal.md) | 中文 + +## 问题 + +同版本 Session 读取只有在 producer 把信封标记为 `ignorable: true` 时，才可以安全跳过未知事件。保持基数的迁移有更严格义务：它必须证明每个保留 payload 在目标代际中仍具备正确语义。未知 JSON payload 可能包含编译期品牌无法发现的 Session 序号、生命周期事实或模型可见状态。 + +静默复制这类事件，会在后续迁移边改变事件位置时留下陈旧数字引用。静默省略它会丢失持久数据。保留精确且不可变的 v0 generation 不能让任何一种转换后的 v1 结果变得无损。 + +## 决策 + +Alpha v0-to-v1 迁移边拥有冻结且完整的已发布 v0 事件与 payload 清单。它在目标 staging 前拒绝每个未知历史事件类型，包括标记了 `ignorable: true` 的事件；除明确分类为 owner 不透明 JSON 的字段外，它也拒绝已知 payload 的意外成员。诊断会点名事件类型、序号和保持不变的源 generation。 + +该规则只适用于跨越历史格式迁移边。普通当前格式读取保留既有信封行为：未知必需事件被拒绝，带 `ignorable: true` 的未知事件仍可读取。因此新的 v1 外部事件继续使用既有同版本扩展 seam，但不会自动获得未来格式迁移能力。 + +每个第一方源事件类型都在迁移边包中拥有可执行 disposition 与目标 validator。catalog 在构建时静态确定且与 profile 无关，因此 producer 插件是否挂载不会改变旧产物能否迁移。 + +## 后果 + +某些由仓库外信息型插件产生的 v0 Session 可能拒绝 alpha 迁移，即使 v0 codec 能解码它们。拒绝不会发布后继，因此无后缀 v0 路径、字节与 inode 仍然权威且不变。操作者可以从诊断识别阻塞类型，并完整访问其原始文本。 + +社区反馈将决定下一步策略。后续版本可以添加显式外部 owner 迁移接口、在保留精确源代际时允许省略明确 ignorable 的历史事件，或继续严格拒绝。Alpha 标记不预先承诺任何选项。 + +`SessionSeq` 与 `SessionLogOffset` 让已知第一方数字字段可审计，但无法分类未知 runtime 对象中的数字。因此迁移规则不能根据没有识别到品牌字段来推断安全。 + +本记录仅在历史格式迁移方面取代 [保留可忽略外部 Session 事件](2026-08-30-retain-ignorable-external-session-events.zh.md)。原决定对同版本 append 与 reload 仍然有效。 + +## 考虑过的替代方案 + +- **逐字复制未知 ignorable 事件**——保留字节，但不能证明不透明数字或生命周期事实在结构迁移后仍有效。 +- **丢弃未知 ignorable 事件**——让迁移保持可用，但不再无损，并让该标记授权删除数据。 +- **在未知 JSON 中搜索类似数字字段的名称**——启发式无法建立语义身份，还会制造虚假信心。 +- **动态询问已挂载插件**——让迁移可用性取决于某个部署组合，并在缺席 producer 能挂载前失败。 +- **等到第一条结构迁移边再拒绝**——会让 v1 包含从未建立安全解释的历史值；恒等演练正是把策略变成可执行规则的时点。 diff --git a/.agents/notes/implemented/architecture/2026-08-31-released-session-format-migrations.i18n.yaml b/.agents/notes/implemented/architecture/2026-08-31-released-session-format-migrations.i18n.yaml new file mode 100644 index 0000000000..e1e8c300e5 --- /dev/null +++ b/.agents/notes/implemented/architecture/2026-08-31-released-session-format-migrations.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-08-31-released-session-format-migrations.md +2026-08-31-released-session-format-migrations.md: 5fcc3d7d99abfb8e1c0b96e2056a1a13b95984c9 +2026-08-31-released-session-format-migrations.zh.md: fe5bb82bce5c838ef30675df1a35dec96ca73bb3 diff --git a/.agents/notes/implemented/architecture/2026-08-31-released-session-format-migrations.md b/.agents/notes/implemented/architecture/2026-08-31-released-session-format-migrations.md new file mode 100644 index 0000000000..5fcc3d7d99 --- /dev/null +++ b/.agents/notes/implemented/architecture/2026-08-31-released-session-format-migrations.md @@ -0,0 +1,55 @@ +# Agent Note: Released Session formats migrate on body read through adjacent pure edges + +Status: implemented + +English | [中文](2026-08-31-released-session-format-migrations.zh.md) + +## Problem + +Session format v0 shipped in an alpha release, so a structural writer change can no longer treat existing JSONL as disposable pre-release state. The runtime also has several ways to read event bodies besides explicit resume: inspect, query, export, fork, continuation, suffix reads, and raw-artifact export. Migrating only one entry path would let callers observe different logical generations or fail only when a later writer reaches the old file. + +Migration must retain the exact source path, bytes, and inode, including a torn physical tail, while giving every published format one unambiguous canonical filename. Plain JSONL and Zstandard are encoding choices for the same logical format and must not create parallel migration implementations. + +## Decision + +`SESSION_FORMAT_VERSION` is a monotonic current-writer integer. One profile-independent pure package owns each adjacent `vN -> vN+1` conversion. `@deepseek-ai/dsh-session-format` supplies only lossless snapshots, unique gap-free planning, header-only conversion, and whole-artifact composition; `@deepseek-ai/dsh-session-format-catalog` statically imports the complete chain independently of mounted Cordis plugins. Historical codecs and normalizers live in the named edge package, while current Session and persistence code accept only the latest logical types. + +Each edge freezes strict source and target semantics, while its target physical codec remains vocabulary-neutral so ordinary event growth can stay within one format version. The catalog restores the final generation through the installed peer `@deepseek-ai/dsh-session` and its current `KNOWN_SESSION_EVENT_TYPES`, preventing a frozen historical edge from becoming the current vocabulary owner. + +Every event-body operation crosses the persistence coordinator's per-Session serialization chain and completes provider-owned ensure-current work before current values escape. JSONL fuses highest-generation resolution, classification or migration, and current decoding over one physical snapshot; fallback backends retain separate `ensureCurrent` and current-read hooks. The six public reads are prepare, load, inspect, borrowSession, readFrom, and readRaw; cold append adoption uses the same body path. Header-only list and listSnapshots never migrate: they rescan each Session directory and return one descriptor for its numerically highest canonical generation, so a future, unsupported, or malformed highest file remains visible instead of silently falling back. + +For `prepare`, `inspect`, and `borrowSession`, cancellation belongs to the observing call rather than shared preparation or migration. A cancelled observer stops waiting, while already-started work may finish for another inspector or later resume; durable publication is never rolled back to satisfy observer cancellation. Detached `readFrom` and `readRaw` operations instead pass cancellation into their serialized backend read. + +The configured JSONL encoding owns one full suffix, `.jsonl` or `.jsonl.zstd`. Migration reads a stable exact source, decodes the recoverable logical prefix, composes every required edge in memory, applies current interrupted-turn repair, validates and syncs a same-directory temporary stage for only the final target, rechecks the source fingerprint, publishes that previously absent target without overwrite, syncs the namespace, and reopens it through the ordinary current reader before a Session is constructed. The source never moves or changes; only disposable temporary stages may be moved, linked, or removed. + +Canonical filenames encode the physical format generation: v0 is `session.jsonl` or `session.jsonl.zstd`; every positive generation is lowercase `session.vN.jsonl` or `session.vN.jsonl.zstd`. Publication never renames, replaces, or deletes a committed generation path. If the target already exists, it is accepted only as a regular current-format file with exactly the expected bytes; any other target refuses. Lower generations remain for operator inspection or explicit copying, but normal runtime operations select the numerically highest canonical name and never use retained predecessors as automatic fallback, restore, or downgrade support. + +The current-format fast path classifies the header from one stable source snapshot, invokes no historical converter or generation write, and passes that snapshot to current decoding without another file read. A validated current selection is cached for later opens in the same backend instance under the one-writer assumption, while listing deliberately rescans. Multiple edges leave the original generation unchanged and publish only the final target; intermediate versions exist only in memory. Same-process operations are serialized, and a source fingerprint recheck restarts the complete attempt when content changes. Cross-process writer fencing remains outside this guarantee. + +The first edge, `@deepseek-ai/dsh-session-format-v0-to-v1`, is intentionally identity-shaped: aside from the version and bounded historical normalizations already accepted by v0, it preserves logical headers, events, sequence numbers, references, timestamps, payloads, and the configured compression choice. The exact `session.jsonl[.zstd]` source remains byte- and inode-identical, while the current writer encodes the new `session.v1.jsonl[.zstd]` successor. This exercises the complete publication lifecycle before a cardinality-changing format needs it. + +## Consequences + +Reading event bodies with a newer build may durably add a higher generation. The exact old generation remains available, but the runtime thereafter selects the highest canonical filename; retention does not promise that an older build can safely downgrade or that the newer build will fall back when the successor is corrupt. A read-only filesystem reports an actionable migration failure instead of returning an in-memory current view that differs from disk. + +JSONL publication uses POSIX hard-link creation plus directory sync, and Windows uses no-overwrite `MoveFileExW` with write-through. A competing writer that wins target creation is accepted only when the committed bytes exactly match. One process-local writer per Session is the supported concurrency model. A future per-Session cross-process lock can close the remaining source-check-to-publication race without changing the format edge interface. + +Retained generations are not a live-stream write-ahead log. A future optional WAL sidecar may preserve unfinished assistant streams across a hard crash. Explicit generation inspection or copying, retention tooling, compression conversion, and streamed whole-artifact transformation are separate features; automatic fallback and downgrade compatibility are not implied future work. + +This note supersedes the continue-only persistence rule and the deferred-chain status in [Session log versioning](2026-08-10-session-log-version-mechanism.md). That note remains the authority for when to bump the version and for ordinary equal-version `ignorable` event behavior. + +## Verification + +Release verification ran the committed Session-format corpus gate over 152 versioned persisted-or-projected `session*.jsonl` fixtures under `snapshots/`, `packages/`, and `scripts/snapshots/python-sdk-single-exe/`. Fixture-only omitted envelopes and request-header tokens are materialized before the real static catalog; 150 fixtures reached the current v1 view through current restoration or historical migration. Released-v0 replay inputs remain suffixless, while fresh v1 writer outputs use `session.v1.jsonl` for a parent and `session..v1.jsonl` for children; older role generations remain beside the selected highest file. The two exact alpha refusals were `snapshots/session/agent-instructions/session.jsonl`, whose projected compaction checkpoint has no matching start, and `snapshots/web/schedule-catalog/session.jsonl`, whose title source contradicts its citations. The continuing gate discovers the corpus dynamically and fails any refusal outside that closed manifest; separate assembled JSONL tests own exact physical-byte migration. + +Current-head performance used three independent runs, each with 100 warmups and 600 alternating samples per case; a pooled 1,800-sample marginal estimator compared the immutable resolver with the same-commit dispatch-disabled baseline. Hot median/p95 deltas were raw small `-1.864%/-1.109%`, raw 100-turn `-0.711%/-0.445%`, Zstandard small `-0.880%/-5.025%`, and Zstandard 100-turn `-0.301%/-2.090%`, all within the five-percent regression ceiling. Cold enabled-path median/p95 costs were raw small `220.125/294.708 µs`, raw 100-turn `580.625/730.834 µs`, Zstandard small `248.042/960.083 µs`, and Zstandard 100-turn `636.291/1421.208 µs`. Repeated hot body reads performed zero directory scans; two listing calls performed two scans. + +The assembled headless profile test stages `session.jsonl`, resumes it through the shipped composition, observes v1 before Session construction, verifies that the exact v0 bytes and inode remain while `session.v1.jsonl` appears, and proves the next append targets v1. JSONL contract tests exercise raw and Zstandard exclusive publication, torn-tail preservation, source changes, target collisions, future-highest refusal, current-selection caching, listing rescans, temporary cleanup, committed reopen, and current-format bypass. + +## Alternatives considered + +- **Migrate only on continuation** — leaves query, export, fork, and suffix consumers on old generations and duplicates restoration policy. +- **Return a migrated in-memory view without persisting** — lets one process observe state that does not match the highest committed generation and postpones failure until a later writer. +- **Persist every intermediate version** — consumes space and creates recovery states with no runtime consumer; only the source and final generation are durable. +- **Let mounted event-owner plugins register migrations** — makes historical readability deployment-dependent; the static catalog must work before feature plugins mount. +- **Reuse one filename for every current format and relocate its predecessor** — rejected because migration would move or overwrite committed evidence, require collision and retention rules, and make the filename disagree with the stored format. Canonical immutable generation names let discovery select the highest version directly. diff --git a/.agents/notes/implemented/architecture/2026-08-31-released-session-format-migrations.zh.md b/.agents/notes/implemented/architecture/2026-08-31-released-session-format-migrations.zh.md new file mode 100644 index 0000000000..fe5bb82bce --- /dev/null +++ b/.agents/notes/implemented/architecture/2026-08-31-released-session-format-migrations.zh.md @@ -0,0 +1,55 @@ +# Agent Note: 已发布 Session 格式在读取正文时通过相邻纯迁移边升级 + +Status: implemented + +[English](2026-08-31-released-session-format-migrations.md) | 中文 + +## 问题 + +Session 格式 v0 已随 alpha 版本发布，因此结构化 writer 变更不能再把已有 JSONL 当作可丢弃的预发布状态。除显式恢复外，runtime 还有多个读取事件正文的入口：检查、查询、导出、分叉、继续、后缀读取和原始产物导出。只迁移一个入口会让调用方看到不同的逻辑 generation，或只在后续 writer 到达旧文件时失败。 + +迁移必须保留精确源路径、字节与 inode，包括撕裂的物理尾部，同时为每个已发布格式提供一个无歧义的规范文件名。普通 JSONL 与 Zstandard 是同一逻辑格式的编码选择，不能产生两套并行迁移实现。 + +## 决策 + +`SESSION_FORMAT_VERSION` 是单调递增的当前 writer 整数。每个相邻 `vN -> vN+1` 转换由一个与 profile 无关的纯包负责。`@deepseek-ai/dsh-session-format` 只提供无损快照、唯一且无缺口的规划、仅 header 转换与整产物组合；`@deepseek-ai/dsh-session-format-catalog` 静态导入完整链，不依赖已挂载的 Cordis 插件。历史 codec 和归一化器位于具名迁移边包中，而当前 Session 与持久化代码只接纳最新逻辑类型。 + +每条迁移边都会冻结严格的源与目标语义，其目标物理 codec 则保持词汇中立，使普通事件增长可以留在同一格式版本内。目录通过已安装的 peer `@deepseek-ai/dsh-session` 及其当前 `KNOWN_SESSION_EVENT_TYPES` 还原最终代，避免冻结的历史迁移边反过来成为当前词汇 owner。 + +每个事件正文操作都经过持久化协调器的逐 Session 串行链，并在当前值离开前完成 provider 拥有的 ensure-current 工作。JSONL 会基于同一个物理快照融合最高 generation 解析、分类或迁移与当前格式解码；fallback 后端保留分离的 `ensureCurrent` 与当前读取钩子。六个公开读取是 prepare、load、inspect、borrowSession、readFrom 与 readRaw；冷 append 接管使用同一正文路径。仅 header 的 list 与 listSnapshots 从不迁移：它们重新扫描每个 Session 目录，并为数值最高的规范 generation 返回一个 descriptor，因此未来、不支持或 malformed 的最高文件会保持可见，而不会静默 fallback。 + +对于 `prepare`、`inspect` 与 `borrowSession`，取消属于观察调用，而不属于共享准备或迁移。被取消的观察者会停止等待，而已经开始的工作可以为另一检查者或后续恢复继续完成；持久发布绝不会为了满足观察者取消而回滚。分离的 `readFrom` 与 `readRaw` 操作则会把取消传入其串行化后端读取。 + +配置的 JSONL 编码拥有一个完整后缀：`.jsonl` 或 `.jsonl.zstd`。迁移读取稳定的精确源，解码可恢复逻辑前缀，在内存中组合全部必需迁移边，应用当前的中断轮次修复，只为最终目标校验并同步同目录临时 stage，重新检查源 fingerprint，以不覆盖方式发布此前不存在的目标，同步 namespace，并在构造 Session 前通过普通当前 reader 重新打开。源永不移动或改变；只有可丢弃临时 stage 可以被移动、链接或移除。 + +规范文件名编码物理格式 generation：v0 是 `session.jsonl` 或 `session.jsonl.zstd`；每个正 generation 都是小写 `session.vN.jsonl` 或 `session.vN.jsonl.zstd`。发布绝不重命名、替换或删除已提交 generation 路径。目标已经存在时，只有它是普通当前格式文件且字节与预期完全相同时才接受；其他目标都会拒绝。低 generation 为 operator 检查或显式复制而保留，但普通 runtime 操作选择数值最高的规范名称，绝不把保留的前任当作自动 fallback、restore 或 downgrade 支持。 + +当前格式快速路径从一个稳定源快照分类 header，不调用历史 converter，不写 generation，并把该快照交给当前格式解码，而不再次读取文件。后端会在单 writer 假设下缓存已校验的当前选择，供同一实例后续打开使用，而列表会有意重新扫描。多条迁移边保持原 generation 不变，并只发布最终目标；中间版本只存在于内存。同一进程内的操作会串行化，源 fingerprint 重新检查会在内容变化时重启完整尝试。跨进程 writer 隔离不在此保证内。 + +第一条迁移边 `@deepseek-ai/dsh-session-format-v0-to-v1` 有意保持恒等形态：除版本和 v0 已接纳的有限历史归一化外，它保留逻辑 header、事件、序号、引用、时间戳、payload 与已配置的压缩选择。精确的 `session.jsonl[.zstd]` 源保持字节与 inode 相同，当前 writer 则编码新的 `session.v1.jsonl[.zstd]` 后继。这样可在出现改变基数的格式前先验证完整发布生命周期。 + +## 后果 + +较新 build 读取事件正文时可能持久增加一个更高 generation。精确旧 generation 仍然可用，但 runtime 此后选择最高规范文件名；保留不承诺旧 build 能安全 downgrade，也不保证新 build 在后继损坏时 fallback。只读文件系统会报告可操作的迁移失败，而不会返回与磁盘不一致的内存当前视图。 + +JSONL 发布在 POSIX 上使用硬链接创建与目录同步，在 Windows 上使用 write-through 且不覆盖的 `MoveFileExW`。竞争 writer 已先创建目标时，只有已提交字节完全匹配才接受。每个 Session 只支持一个进程内 writer。未来逐 Session 跨进程锁可以关闭剩余的源检查到发布竞态，而无需改变格式迁移边接口。 + +保留的 generation 不是实时流 WAL。未来可选 WAL sidecar 可以在硬崩溃间保留未完成 assistant 流。显式 generation 检查或复制、保留策略工具、压缩转换与流式整产物转换都是独立功能；自动 fallback 与 downgrade compatibility 并非隐含 future work。 + +本记录取代 [Session 日志版本机制](2026-08-10-session-log-version-mechanism.zh.md) 中仅在继续时持久化和迁移链仍推迟的规则。原记录继续负责何时递增版本，以及普通同版本 `ignorable` 事件行为。 + +## 验证 + +发布验证针对 `snapshots/`、`packages/` 与 `scripts/snapshots/python-sdk-single-exe/` 下 152 个带版本、来自持久化或投影的 `session*.jsonl` fixture 运行了已提交 Session 格式语料门禁。fixture 专用的缺失信封与 request-header token 会先被实体化，再进入真实静态 catalog；其中 150 个通过当前格式 restore 或历史迁移得到当前 v1 视图。Released-v0 replay 输入保持无后缀，而新鲜 v1 writer 输出对 parent 使用 `session.v1.jsonl`、对 child 使用 `session..v1.jsonl`；较旧角色 generation 保留在选定最高文件旁边。两个精确 alpha 拒绝分别是 `snapshots/session/agent-instructions/session.jsonl`（投影出的 compaction checkpoint 没有匹配 start）与 `snapshots/web/schedule-catalog/session.jsonl`（title 来源与其 citation 矛盾）。持续运行的门禁会动态发现语料，并拒绝封闭 manifest 之外的任何失败；独立组装式 JSONL 测试负责精确物理字节迁移。 + +当前 head 性能证据来自 3 次独立运行，每个 case 含 100 次 warmup 与 600 个 alternating sample；pooled 1,800-sample marginal estimator 把不可变 resolver 与同一 commit 下禁用 dispatch 的 baseline 比较。Hot median/p95 delta 分别为 raw small `-1.864%/-1.109%`、raw 100-turn `-0.711%/-0.445%`、Zstandard small `-0.880%/-5.025%`、Zstandard 100-turn `-0.301%/-2.090%`，全部满足 5% regression ceiling。Cold enabled-path median/p95 cost 分别为 raw small `220.125/294.708 µs`、raw 100-turn `580.625/730.834 µs`、Zstandard small `248.042/960.083 µs`、Zstandard 100-turn `636.291/1421.208 µs`。重复 hot body read 执行 0 次目录扫描；两次 listing 调用执行 2 次扫描。 + +组装后的 headless profile 测试会暂存 `session.jsonl`，通过随附组合恢复它，在构造 Session 前观察到 v1，验证精确 v0 字节与 inode 保持不变而 `session.v1.jsonl` 出现，并证明下一次 append 以 v1 为目标。JSONL 约定测试覆盖 raw 与 Zstandard 排他发布、撕裂尾部保留、源变化、目标冲突、最高未来版本拒绝、当前选择 cache、列表重新扫描、临时文件清理、已提交重开与当前格式直通。 + +## 考虑过的替代方案 + +- **只在继续时迁移**——让查询、导出、分叉与后缀消费者停留在旧代际，并重复恢复策略。 +- **返回迁移后的内存视图但不持久化**——让进程观察到与最高已提交 generation 不一致的状态，并把失败推迟到后续 writer。 +- **持久化每个中间版本**——消耗空间并产生没有 runtime 消费者的恢复状态；只有源与最终代际应持久。 +- **让已挂载事件 owner 插件注册迁移**——使历史可读性依赖部署；静态 catalog 必须在功能插件挂载前工作。 +- **让每个当前格式复用同一个文件名并迁走前任**——不予采用，因为迁移会移动或覆盖已提交证据，需要冲突与保留规则，并让文件名与存储格式不一致。规范不可变 generation 名让发现流程直接选择最高版本。 diff --git a/.agents/notes/implemented/bug-fix/2026-07-20-jsonl-storage-identity.i18n.yaml b/.agents/notes/implemented/bug-fix/2026-07-20-jsonl-storage-identity.i18n.yaml index 1f998e3982..84feb26de2 100644 --- a/.agents/notes/implemented/bug-fix/2026-07-20-jsonl-storage-identity.i18n.yaml +++ b/.agents/notes/implemented/bug-fix/2026-07-20-jsonl-storage-identity.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/bug-fix/2026-07-20-jsonl-storage-identity.md -2026-07-20-jsonl-storage-identity.md: e249640b1cd8900fdb7a136e9ab56abbf474ac86 -2026-07-20-jsonl-storage-identity.zh.md: 4775d6b7aa02abbd58ef89cdfa9377dc4f94b8de +2026-07-20-jsonl-storage-identity.md: 5224ebf6bebc42b4b72e14243c1af6c02f366372 +2026-07-20-jsonl-storage-identity.zh.md: 05b7150edb99cdf336b9df90bc8e3cc1df9848a3 diff --git a/.agents/notes/implemented/bug-fix/2026-07-20-jsonl-storage-identity.md b/.agents/notes/implemented/bug-fix/2026-07-20-jsonl-storage-identity.md index e249640b1c..5224ebf6be 100644 --- a/.agents/notes/implemented/bug-fix/2026-07-20-jsonl-storage-identity.md +++ b/.agents/notes/implemented/bug-fix/2026-07-20-jsonl-storage-identity.md @@ -10,7 +10,7 @@ JSONL lookup selects a physical log from the requested session id across project ## Decision -`loadStored(id)` is the coordinator's single stored-prefix lookup. The JSONL backend scans every project directory, requires at most one matching encoded session directory with a transcript, parses that file, then validates `header.id === id` and that the selected path either equals `logPath(root, header.cwd, header.id)` or filesystem canonicalization resolves both spellings to the same transcript. `list()` applies the same path validation and rejects duplicate ids across project directories. +`findLog(id)` is the JSONL provider's single physical resolver. It scans every project directory and requires at most one matching encoded session directory with a transcript. For a supported historical snapshot, the generation operation translates its header in memory, then validates `header.id === id` and that the selected path either equals `logPath(root, header.cwd, header.id)` or filesystem canonicalization resolves both spellings to the same transcript before any write. The fused `loadCurrentStored(id)` decodes the resulting current prefix without another file read; an already-current snapshot takes the no-write fast path and applies the same identity check during that decode. The ordinary current `loadStored(id)` hook retains the identity validation for coordinator fallbacks. `list()` applies the same path validation and rejects duplicate ids across project directories. The coordinator independently asserts the returned id and compares the stored cwd with a live session's cwd before repair, state publication, or suffix persistence. It keeps a detached copy of validated metadata; JSONL append and repair derive their path from that copy. The `PersistenceBackend` interface therefore needs neither a scope-specific live lookup nor a storage-locator type. diff --git a/.agents/notes/implemented/bug-fix/2026-07-20-jsonl-storage-identity.zh.md b/.agents/notes/implemented/bug-fix/2026-07-20-jsonl-storage-identity.zh.md index 4775d6b7aa..05b7150edb 100644 --- a/.agents/notes/implemented/bug-fix/2026-07-20-jsonl-storage-identity.zh.md +++ b/.agents/notes/implemented/bug-fix/2026-07-20-jsonl-storage-identity.zh.md @@ -10,7 +10,7 @@ JSONL 查找会根据请求的会话 id 在各个项目目录中选出物理日 ## 决策 -`loadStored(id)` 是协调器唯一的已存前缀查找操作。JSONL 后端扫描所有项目目录，要求名称与该 id 的编码值匹配且其中包含 transcript（文本记录）的会话目录至多有一个，解析其中的 transcript，然后验证 `header.id === id`，并验证选定路径要么等于 `logPath(root, header.cwd, header.id)`，要么经文件系统路径规范化后，两种写法解析为同一份 transcript。`list()` 执行相同的路径验证，并拒绝跨项目目录重复的 id。 +`findLog(id)` 是 JSONL provider 唯一的物理解析器。它扫描所有项目目录，并要求名称与该 id 的编码值匹配且其中包含 transcript（文本记录）的会话目录至多有一个。对于受支持的历史快照，generation 操作会在内存中转换其 header，然后验证 `header.id === id`，并验证选定路径要么等于 `logPath(root, header.cwd, header.id)`，要么经文件系统路径规范化后，两种写法解析为同一份 transcript；任何写入都必须在该检查之后。融合的 `loadCurrentStored(id)` 会在不再次读取文件的情况下解码所得当前前缀；已经是当前格式的快照采用无写入快速路径，并在该次解码中执行相同的身份检查。普通当前格式 `loadStored(id)` 钩子为协调器 fallback 保留身份校验。`list()` 执行相同的路径校验，并拒绝跨项目目录重复的 id。 协调器会独立断言返回的 id，并在修复、发布状态或持久化后缀之前比较已存 cwd 和活动会话的 cwd。协调器保留一份已验证元数据的独立副本；JSONL 的追加和修复操作根据该副本派生路径。因此，`PersistenceBackend` 接口既不需要限定范围的活动会话查找，也不需要存储定位器类型。 diff --git a/.agents/notes/implemented/bug-fix/2026-07-28-load-pre-identity-session-messages.i18n.yaml b/.agents/notes/implemented/bug-fix/2026-07-28-load-pre-identity-session-messages.i18n.yaml index a353fc98ef..35876cd934 100644 --- a/.agents/notes/implemented/bug-fix/2026-07-28-load-pre-identity-session-messages.i18n.yaml +++ b/.agents/notes/implemented/bug-fix/2026-07-28-load-pre-identity-session-messages.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/bug-fix/2026-07-28-load-pre-identity-session-messages.md -2026-07-28-load-pre-identity-session-messages.md: 6d022cb4b37345cd61cc9a89c6fc55c19ad402a5 -2026-07-28-load-pre-identity-session-messages.zh.md: 86439337b3c646a72b7584fbf4640799226fc9ca +2026-07-28-load-pre-identity-session-messages.md: 003331cd66ce0f299f1ef7c03684dbd0bee148b8 +2026-07-28-load-pre-identity-session-messages.zh.md: cf15b5434b66708ae70620326fe3e975738e68d1 diff --git a/.agents/notes/implemented/bug-fix/2026-07-28-load-pre-identity-session-messages.md b/.agents/notes/implemented/bug-fix/2026-07-28-load-pre-identity-session-messages.md index 6d022cb4b3..003331cd66 100644 --- a/.agents/notes/implemented/bug-fix/2026-07-28-load-pre-identity-session-messages.md +++ b/.agents/notes/implemented/bug-fix/2026-07-28-load-pre-identity-session-messages.md @@ -12,17 +12,17 @@ Changing the message representation without a version bump made those logs indis ## Decision -`PersistenceCoordinator` normalizes the four exact pre-identity message payloads after backend decoding and before current message validation. It wraps their existing semantic fields in the current role-specific message shape and assigns `legacy-message::` as the deterministic imported `MessageId`. A legacy `tool/result` content replacement inherits the imported id of its replacement target, preserving the current content-only rewrite invariant. +The frozen `@deepseek-ai/dsh-session-format-v0-to-v1` edge normalizes the four exact pre-identity message payloads after v0 decoding and before v1 validation. It wraps their existing semantic fields in the current role-specific message shape and assigns `legacy-message::` as the deterministic imported `MessageId`. A legacy `tool/result` content replacement inherits the imported id of its replacement target, preserving the current content-only rewrite invariant. -The same normalization runs for `load`, `inspect`, an ownerless loaded state claiming its live session, and HMR prefix adoption. Prefix comparisons therefore compare the live current-shape seed with the same normalized stored view. Current-looking wrappers with missing or invalid fields are not repaired, and unsupported event vocabulary, request headers, versions, and surface relations retain their existing rejection paths. +Every event-body operation runs the same edge through the build-static catalog before current Session construction. `load`, `inspect`, ownerless-state adoption, HMR prefix adoption, query, export, fork, and suffix reads therefore see one normalized current generation. Current-looking wrappers with missing or invalid fields are not repaired, and unsupported event vocabulary, request headers, versions, and surface relations retain their refusal paths. -The upgrade is read-only. Stored legacy records remain unchanged; a resumed session appends only current-shape events after them. Deterministic identities make repeated loads and a mixed legacy/current log reproduce the same message ids without a backend-specific rewrite transaction. +JSONL migration leaves the exact suffixless v0 artifact path, bytes, and inode unchanged and exclusively publishes `session.v1.jsonl[.zstd]` beside it. Deterministic identities make repeated restoration reproduce the same message ids, and subsequent appends target only v1. ## Alternatives considered -**Reject the logs under the pre-release compatibility stance.** This is the default for unrelated v0 churn, but it strands real first-party sessions even though every old field maps unambiguously to the current message representation. +**Reject the released logs.** This strands real first-party sessions even though every old field maps unambiguously to the current message representation. -**Rewrite the complete stored log in place.** This would canonicalize the artifact but violate the append-only storage contract, require an atomic replacement mechanism, and expand a read compatibility fix into a migration system. +**Keep a same-version importer inside the coordinator.** This avoids a format edge but leaves historical payloads in current Session code and provides neither immutable source/successor naming nor independently testable publication. The released adjacent migration system owns canonical publication instead. **Mint random ids on each load.** The messages would satisfy the type shape but lose stable identity across inspect, resume, restart, and mixed legacy/current appends. @@ -30,7 +30,7 @@ The upgrade is read-only. Stored legacy records remain unchanged; a resumed sess Pre-identity JSONL Sessions resume with their original message content, sources, assistant provider/model fields, tool correlation, errors, metadata, and surface replacements. The returned events are otherwise indistinguishable from current imported message snapshots and remain deeply frozen. -This is one explicit same-version import exception, not a general v0 compatibility layer. Adding another exception requires another complete, unambiguous mapping at the persistence boundary; malformed current data continues to fail rather than being guessed into validity. The shared coordinator contract exercises the upgrade against the in-memory reference and JSONL provider, including deterministic reload and tool-result replacement identity. +This is one explicit released-v0 normalization, not a permissive compatibility layer. Adding another normalization requires another complete, unambiguous mapping in the frozen edge; malformed current data continues to fail rather than being guessed into validity. Edge and JSONL generation tests exercise deterministic restoration and tool-result replacement identity. ## Related diff --git a/.agents/notes/implemented/bug-fix/2026-07-28-load-pre-identity-session-messages.zh.md b/.agents/notes/implemented/bug-fix/2026-07-28-load-pre-identity-session-messages.zh.md index 86439337b3..cf15b5434b 100644 --- a/.agents/notes/implemented/bug-fix/2026-07-28-load-pre-identity-session-messages.zh.md +++ b/.agents/notes/implemented/bug-fix/2026-07-28-load-pre-identity-session-messages.zh.md @@ -12,17 +12,17 @@ Status: implemented ## 决策 -`PersistenceCoordinator` 会在后端解码之后、当前消息验证之前，规范化消息标识机制引入前的四种特定消息载荷。它将载荷现有的语义字段包装进当前按角色区分的消息形状，并为其分配确定性的导入用 `MessageId`：`legacy-message::`。旧版 `tool/result` 的内容替换会继承替换目标导入后的 id，从而保持当前仅改写内容的不变量。 +冻结的 `@deepseek-ai/dsh-session-format-v0-to-v1` 迁移边会在 v0 解码之后、v1 验证之前，规范化消息标识机制引入前的四种特定消息载荷。它将载荷现有的语义字段包装进当前按角色区分的消息结构，并为其分配确定性的导入用 `MessageId`：`legacy-message::`。旧版 `tool/result` 的内容替换会继承替换目标导入后的 id，从而保持当前仅改写内容的不变量。 -同一项规范化也用于 `load`、`inspect`、无 owner 的已加载状态认领其活跃会话，以及 HMR（热模块替换）前缀接管。因此，前缀比较会将活跃会话的当前形状 seed 与同一份规范化存储视图进行比较。看似当前形状、但字段缺失或无效的包装层不会被修复；不受支持的事件词汇、请求 header、版本和 surface 关系仍沿用现有拒绝路径。 +每项事件正文操作都会在构造当前 Session 前，通过构建期静态目录运行同一条迁移边。因此，`load`、`inspect`、无 owner 状态接管、HMR（热模块替换）前缀接管、查询、导出、fork 与后缀读取都会看到同一份规范化当前代际。看似当前结构、但字段缺失或无效的包装层不会被修复；不受支持的事件词汇、请求 header、版本和 surface 关系仍沿用现有拒绝路径。 -这项升级只发生在读取时。存储中的旧版记录保持不变；会话恢复后，只会在其后追加当前形状的事件。确定性标识使重复加载以及新旧形状混合的日志无需执行后端专用的重写事务，也能复现相同的消息 id。 +JSONL 迁移会保持精确的无后缀 v0 产物路径、字节与 inode 不变，并在其旁边排他发布 `session.v1.jsonl[.zstd]`。确定性标识使重复恢复能够复现相同的消息 id，后续 append 只以 v1 为目标。 ## 考虑过的替代方案 -**按照预发布兼容性立场拒绝这些日志。** 这是处理其他 v0 形状变动的默认方式，但即使每个旧字段都能明确映射到当前消息表示，它仍会导致真实的第一方会话无法恢复。 +**拒绝这些已发布日志。** 即使每个旧字段都能明确映射到当前消息表示，这也会导致真实的第一方会话无法恢复。 -**就地重写完整的存储日志。** 这会使产物规范化，但违反仅追加存储约定，还需要原子替换机制，并将一次读取兼容性修复扩大为迁移系统。 +**在协调器中保留同版本导入器。** 这可以避免迁移边，但会把历史 payload 留在当前 Session 代码中，而且既没有不可变源／后继命名，也没有可独立测试的发布。已发布相邻迁移系统负责规范发布。 **每次加载时随机生成 id。** 这些消息会满足类型形状，却无法在检查、恢复、重启以及新旧形状混合追加之间保持稳定标识。 @@ -30,7 +30,7 @@ Status: implemented 消息标识机制引入前的 JSONL Session 可以恢复，并保留原始消息内容、来源、assistant 的 provider／model 字段、工具调用关联、错误、元数据和 surface 替换。除此之外，返回事件与当前导入的消息快照无法区分，并且仍然经过深度冻结。 -这是一个显式的同版本导入例外，而非通用的 v0 兼容层。若要增加另一个例外，必须在持久化边界提供另一套完整且无歧义的映射；当前数据若格式错误，系统仍会拒绝，而不会猜测如何将其变成有效数据。共享协调器约定会通过内存参考实现与 JSONL provider 验证这项升级，包括重新加载时的确定性，以及工具结果替换时的标识继承。 +这是一项显式的已发布 v0 规范化，而非宽松的兼容层。若要增加另一项规范化，必须在冻结迁移边中提供另一套完整且无歧义的映射；当前数据若格式错误，系统仍会拒绝，而不会猜测如何将其变成有效数据。迁移边与 JSONL 代际测试会验证恢复的确定性，以及工具结果替换时的标识继承。 ## 相关 diff --git a/.agents/notes/implemented/bug-fix/2026-08-04-load-pre-react-loop-sessions.i18n.yaml b/.agents/notes/implemented/bug-fix/2026-08-04-load-pre-react-loop-sessions.i18n.yaml index 0a513e049f..bd53685b34 100644 --- a/.agents/notes/implemented/bug-fix/2026-08-04-load-pre-react-loop-sessions.i18n.yaml +++ b/.agents/notes/implemented/bug-fix/2026-08-04-load-pre-react-loop-sessions.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/bug-fix/2026-08-04-load-pre-react-loop-sessions.md -2026-08-04-load-pre-react-loop-sessions.md: e95817ee60647ca002060a4f90c2263d4fe7ce42 -2026-08-04-load-pre-react-loop-sessions.zh.md: 98fb1f530f5168fc02b312775d1bb8e6d305b8f8 +2026-08-04-load-pre-react-loop-sessions.md: beda9c9984f37a5ee6fa4e1a5bbff07d3b1e363c +2026-08-04-load-pre-react-loop-sessions.zh.md: ace424476a96c4abc46fc8afeb3920d564bb65ae diff --git a/.agents/notes/implemented/bug-fix/2026-08-04-load-pre-react-loop-sessions.md b/.agents/notes/implemented/bug-fix/2026-08-04-load-pre-react-loop-sessions.md index e95817ee60..beda9c9984 100644 --- a/.agents/notes/implemented/bug-fix/2026-08-04-load-pre-react-loop-sessions.md +++ b/.agents/notes/implemented/bug-fix/2026-08-04-load-pre-react-loop-sessions.md @@ -12,29 +12,29 @@ The new durable inbox is not part of this compatibility problem. The base emitte ## Decision -`PersistenceCoordinator` recognizes the exact pre-react-loop shapes after backend decoding and projects them into the current read view. It removes the obsolete `turn/start.trigger`, converts `steering/message` to the same identified `user/message`, maps old failure facts into the current structured error, folds `disposed` into an aborted turn with the `disposed` cause, and represents coarse aborted records with the persistence-only `{ kind: 'legacy' }` cause because their caller is unavailable. +The frozen `@deepseek-ai/dsh-session-format-v0-to-v1` edge recognizes the exact pre-react-loop shapes after v0 decoding and projects them into v1. It removes the obsolete `turn/start.trigger`, converts `steering/message` to the same identified `user/message`, maps old failure facts into the current structured error, folds `disposed` into an aborted turn with the `disposed` cause, and represents coarse aborted records with the persistence-only `{ kind: 'legacy' }` cause because their caller is unavailable. -The coordinator applies the projection to `load`, `inspect`, adoption, HMR prefix comparison, and `readFrom`. A seek-capable `readFrom` normally reads only its suffix; when that suffix contains a legacy event needing an earlier replacement identity, the coordinator loads and normalizes the complete prefix before returning the requested seq range. +Every persistence event-body entry point first migrates the complete detached artifact through the build-static catalog. `load`, `inspect`, adoption, HMR prefix comparison, and `readFrom` therefore receive the same validated v1 view; suffix reading happens only after immutable successor publication and current restoration. -The importer does not synthesize inbox splices. A resumed pre-react-loop agent begins with empty pending lists, matching the base runtime's inability to persist pending inbox work. The stored artifact remains append-only and later events use the current format. +The edge does not synthesize inbox splices. A resumed pre-react-loop agent begins with empty pending lists, matching the base runtime's inability to persist pending inbox work. Migration leaves the exact suffixless v0 generation unchanged and publishes one `session.v1.jsonl[.zstd]` successor before later events append. ## Alternatives considered -**Treat the same-version records as unsupported.** This follows the pre-release default but strands sessions produced by the PR base even though the removed steering content and terminal facts have complete mappings. +**Treat the released records as unsupported.** This strands sessions produced by the supported first-party writer even though the removed steering content and terminal facts have complete mappings. **Replay old inbox notifications into durable splices.** Those notifications were not session events and do not provide a trustworthy pending-state snapshot. Inferring insertions without every claim and discard would re-run consumed work. **Assign coarse aborted records to an existing caller.** Mapping them to `user`, `parent`, or `hook` would invent a caller that the old record did not name. A dedicated `legacy` cause keeps the stop classification without making a false audit claim. -**Rewrite stored JSONL records.** A rewrite would violate the append-only contract and require atomic migration machinery for a read compatibility boundary. +**Keep a generic same-version importer in the coordinator.** This lets current Session code accumulate historical forms and gives no immutable physical generation naming or independently testable adjacent edge. The released migration lifecycle owns the conversion instead. ## Consequences -Sessions written in the refactor's base format resume through the current AgentLoop with their steering content, turn boundaries, error facts, and stop classification intact. The shared coordinator contract covers in-memory and JSONL `load`/`inspect`/`readFrom`; an assembled JSONL Agent resume verifies that the historical transcript is visible while both new inbox lists start empty. +Sessions written in the refactor's base format resume through the current AgentLoop with their steering content, turn boundaries, error facts, and stop classification intact. The frozen edge and JSONL generation contract cover `load`/`inspect`/`readFrom`; an assembled JSONL Agent resume verifies that the historical transcript is visible while both new inbox lists start empty. This exception supports the base format, not intermediate formats produced during development of the refactor. In particular, it defines no migration for earlier experimental `agent/inbox/spliced` payloads. Exact-shape recognition keeps malformed current-looking records on their rejection path instead of guessing them into validity. ## Related -- [Load sessions persisted before message identity](2026-07-28-load-pre-identity-session-messages.md) — owns deterministic identities and the general read-only import boundary for another same-version format change. +- [Load sessions persisted before message identity](2026-07-28-load-pre-identity-session-messages.md) — owns deterministic identities for another released v0 normalization in the same edge. - [Session persistence as an abstract service](../architecture/2026-06-14-session-persistence.md) — owns append-only backend storage and resume. diff --git a/.agents/notes/implemented/bug-fix/2026-08-04-load-pre-react-loop-sessions.zh.md b/.agents/notes/implemented/bug-fix/2026-08-04-load-pre-react-loop-sessions.zh.md index 98fb1f530f..ace424476a 100644 --- a/.agents/notes/implemented/bug-fix/2026-08-04-load-pre-react-loop-sessions.zh.md +++ b/.agents/notes/implemented/bug-fix/2026-08-04-load-pre-react-loop-sessions.zh.md @@ -12,29 +12,29 @@ react-loop 简化在保持 `SESSION_FORMAT_VERSION` 为 0 的同时更改了持 ## 决策 -`PersistenceCoordinator` 会在后端解码后识别 react-loop 重构前的确切形状，并将其投影为当前读取视图。它移除已废弃的 `turn/start.trigger`，把 `steering/message` 转换为同一条带标识的 `user/message`，将旧版失败事实映射为当前结构化错误，把 `disposed` 折叠为带 `disposed` 原因的已中止轮次，并用仅供持久化导入使用的 `{ kind: 'legacy' }` 原因表示粗粒度中止记录，因为无法获得其调用方。 +冻结的 `@deepseek-ai/dsh-session-format-v0-to-v1` 迁移边会在 v0 解码后识别 react-loop 重构前的确切结构，并将其投影为 v1。它移除已废弃的 `turn/start.trigger`，把 `steering/message` 转换为同一条带标识的 `user/message`，将旧版失败事实映射为当前结构化错误，把 `disposed` 折叠为带 `disposed` 原因的已中止轮次，并用仅供持久化导入使用的 `{ kind: 'legacy' }` 原因表示粗粒度中止记录，因为无法获得其调用方。 -协调器会把该投影应用于 `load`、`inspect`、接管、HMR（热模块替换）前缀比较和 `readFrom`。可寻址的 `readFrom` 通常只读取后缀；如果后缀包含需要更早替换标识的旧版事件，协调器会先加载并规范化完整前缀，再返回所请求的 seq 范围。 +每个持久化事件正文入口都会先通过构建期静态目录迁移完整且分离的产物。因此，`load`、`inspect`、接管、HMR（热模块替换）前缀比较和 `readFrom` 会收到同一份经过校验的 v1 视图；系统只会在不可变后继发布和当前格式恢复后读取后缀。 -导入器不会合成 inbox splice。恢复后的 react-loop 重构前 agent（智能体）从空的待处理列表开始，这与基线运行时无法持久化待处理 inbox 工作的行为一致。已存储产物仍然仅追加，后续事件使用当前格式。 +该迁移边不会合成 inbox splice。恢复后的 react-loop 重构前 agent（智能体）从空的待处理列表开始，这与基线运行时无法持久化待处理 inbox 工作的行为一致。迁移会保持精确的无后缀 v0 generation 不变，并在后续事件 append 前发布一个 `session.v1.jsonl[.zstd]` 后继。 ## 考虑过的替代方案 -**将同版本记录视为不受支持。** 这符合预发布阶段的默认立场，但会使 PR（Pull Request）基线产生的会话无法恢复，尽管已移除的 steering 内容和终止事实都有完整映射。 +**将已发布记录视为不受支持。** 这会使受支持的第一方 writer 所产生的 Session 无法恢复，尽管已移除的 steering 内容和终止事实都有完整映射。 **将旧 inbox 通知回放为持久 splice。** 这些通知不是会话事件，也无法提供可信的待处理状态快照。如果无法获知每一次领取和丢弃，就推断插入操作，会让已消费的工作再次执行。 **将粗粒度中止记录归因于现有调用方。** 将其映射到 `user`、`parent` 或 `hook` 会凭空指定旧记录未注明的调用方。专用的 `legacy` 原因既能保留停止分类，也不会产生虚假的审计事实。 -**重写已存储的 JSONL 记录。** 重写会违反仅追加约定，并要求为读取兼容边界建立原子迁移机制。 +**在协调器中保留通用同版本导入器。** 这会让当前 Session 代码不断积累历史结构，而且没有不可变物理 generation 命名或可独立测试的相邻迁移边。已发布迁移生命周期负责该转换。 ## 后果 -以重构基线格式写入的会话可以通过当前 AgentLoop 恢复，并完整保留 steering 内容、轮次边界、错误事实和停止分类。共享协调器约定覆盖内存与 JSONL 的 `load`／`inspect`／`readFrom`；组装后的 JSONL agent 恢复用例会验证历史 transcript（文本记录）可见，同时两个新 inbox 列表都从空状态开始。 +以重构基线格式写入的会话可以通过当前 AgentLoop 恢复，并完整保留 steering 内容、轮次边界、错误事实和停止分类。冻结迁移边与 JSONL 代际约定覆盖 `load`／`inspect`／`readFrom`；组装后的 JSONL agent 恢复用例会验证历史 transcript（文本记录）可见，同时两个新 inbox 列表都从空状态开始。 此例外支持基线格式，不支持重构开发期间产生的中间格式。具体而言，它没有为更早的实验性 `agent/inbox/spliced` 载荷定义迁移。通过确切形状识别，当前格式外观相似但结构错误的记录仍会走拒绝路径，不会被猜测性地转换为有效记录。 ## 相关资料 -- [加载消息标识机制引入前持久化的会话](2026-07-28-load-pre-identity-session-messages.zh.md)：负责另一项同版本格式变更的确定性标识和通用只读导入边界。 +- [加载消息标识机制引入前持久化的会话](2026-07-28-load-pre-identity-session-messages.zh.md)：负责同一迁移边中另一项已发布 v0 规范化的确定性标识。 - [以抽象服务实现会话持久化](../architecture/2026-06-14-session-persistence.zh.md)：负责仅追加后端存储和恢复。 diff --git a/.agents/notes/implemented/bug-fix/2026-08-20-turn-error-survives-same-turn-retry-history.i18n.yaml b/.agents/notes/implemented/bug-fix/2026-08-20-turn-error-survives-same-turn-retry-history.i18n.yaml index 79d3dc8ee4..61f0cc4e3e 100644 --- a/.agents/notes/implemented/bug-fix/2026-08-20-turn-error-survives-same-turn-retry-history.i18n.yaml +++ b/.agents/notes/implemented/bug-fix/2026-08-20-turn-error-survives-same-turn-retry-history.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/bug-fix/2026-08-20-turn-error-survives-same-turn-retry-history.md -2026-08-20-turn-error-survives-same-turn-retry-history.md: 68b09fc32f2d409a9060bb8e53f11837dd365657 -2026-08-20-turn-error-survives-same-turn-retry-history.zh.md: 1aa78aa405113fecc890fb2213f41cf41afec6b4 +2026-08-20-turn-error-survives-same-turn-retry-history.md: 9280d59d020a88d0f0f703d2e2adc54656687b16 +2026-08-20-turn-error-survives-same-turn-retry-history.zh.md: 1856d72e05c6261535df5f94c41cb3ff86c69362 diff --git a/.agents/notes/implemented/bug-fix/2026-08-20-turn-error-survives-same-turn-retry-history.md b/.agents/notes/implemented/bug-fix/2026-08-20-turn-error-survives-same-turn-retry-history.md index 68b09fc32f..9280d59d02 100644 --- a/.agents/notes/implemented/bug-fix/2026-08-20-turn-error-survives-same-turn-retry-history.md +++ b/.agents/notes/implemented/bug-fix/2026-08-20-turn-error-survives-same-turn-retry-history.md @@ -28,4 +28,4 @@ The Definition suite drives the real assembler through a same-turn retry chain e ## Consequences -Exhausted recovery now leaves durable, replayable feedback: the red terminal row with the display-safe message and code, plus the collapsed retry chain as recovery context. Session logs recorded under the retired new-turn retry model would render one `turn-error` row per failed turn on replay; the pre-release format stance accepts that, and no shipped log producer has emitted that shape since same-turn retries landed. +Exhausted recovery now leaves durable, replayable feedback: the red terminal row with the display-safe message and code, plus the collapsed retry chain as recovery context. Session logs recorded under the retired new-turn retry model render one `turn-error` row per failed turn on replay; the v0-to-v1 edge preserves those terminal facts, and no shipped log producer has emitted that shape since same-turn retries landed. diff --git a/.agents/notes/implemented/bug-fix/2026-08-20-turn-error-survives-same-turn-retry-history.zh.md b/.agents/notes/implemented/bug-fix/2026-08-20-turn-error-survives-same-turn-retry-history.zh.md index 1aa78aa405..1856d72e05 100644 --- a/.agents/notes/implemented/bug-fix/2026-08-20-turn-error-survives-same-turn-retry-history.zh.md +++ b/.agents/notes/implemented/bug-fix/2026-08-20-turn-error-survives-same-turn-retry-history.zh.md @@ -28,4 +28,4 @@ Definition 套件驱动真实 assembler 走完同轮次重试链并以 error 原 ## 影响 -恢复耗尽后现在留下持久、可回放的反馈：红色终态行带展示安全的消息与错误码，加上折叠的重试链作为恢复上下文。按已退役的"新轮次重试"模型录制的会话日志在回放时会为每个失败轮次各渲染一条 `turn-error` 行；预发布格式立场接受这一点，且自同轮次重试落地以来，没有任何已交付的日志生产方再发出过那种形态。 +恢复耗尽后现在留下持久、可回放的反馈：红色终态行带展示安全的消息与错误码，加上折叠的重试链作为恢复上下文。按已退役的“新轮次重试”模型录制的会话日志在回放时会为每个失败轮次各渲染一条 `turn-error` 行；v0-to-v1 迁移边会保留这些终态事实，且自同轮次重试落地以来，没有任何已交付的日志生产方再发出过那种结构。 diff --git a/.agents/notes/implemented/feature/2026-07-10-sqlite-session-query-provider.i18n.yaml b/.agents/notes/implemented/feature/2026-07-10-sqlite-session-query-provider.i18n.yaml index 4f6d006d90..8c6ae9f034 100644 --- a/.agents/notes/implemented/feature/2026-07-10-sqlite-session-query-provider.i18n.yaml +++ b/.agents/notes/implemented/feature/2026-07-10-sqlite-session-query-provider.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-10-sqlite-session-query-provider.md -2026-07-10-sqlite-session-query-provider.md: 76ebbc24e16a9429be63d0e45ec84b14c29d43f6 -2026-07-10-sqlite-session-query-provider.zh.md: 6d8518252d0a79906fd9438876d4a92ba414363e +2026-07-10-sqlite-session-query-provider.md: 46af02d39ed687a1f4b418109a1da4b071b9e409 +2026-07-10-sqlite-session-query-provider.zh.md: ccb833886a459c5a6888637c6cb8354e9e9f7977 diff --git a/.agents/notes/implemented/feature/2026-07-10-sqlite-session-query-provider.md b/.agents/notes/implemented/feature/2026-07-10-sqlite-session-query-provider.md index 76ebbc24e1..46af02d39e 100644 --- a/.agents/notes/implemented/feature/2026-07-10-sqlite-session-query-provider.md +++ b/.agents/notes/implemented/feature/2026-07-10-sqlite-session-query-provider.md @@ -34,13 +34,13 @@ Both persistent and live FTS5 tables use `unicode61`. The implementation experim The shared extractor includes message text, reasoning, nested tool-call/result content, tool names and arguments, blocked-prompt reasons, todo status/content, and error or terminal status detail. Structural boundaries, stream chunks, request headers, successful completion markers, and unknown declaration-merged event/content variants produce no document. Surface classification reuses `foldSurface()` so search agrees with model-history derivation. -One serialized operation reads the provider-neutral `SessionPersistence` snapshot listing, compares each source-qualified opaque revision with the revision stored beside the indexed session, loads only new or changed logs, reconciles rows in one transaction, and executes the query. It passes the caller's exact abort signal into snapshot listing and non-mutating inspection, directly awaits every started backend operation, and checks cancellation after each await and before starting more work. Cancellation therefore rejects only after active backend work is quiescent, starts no subsequent observation or reconciliation step, and keeps a following search serialized behind cleanup even if a backend ignores the signal. The operation never calls the backend's mutating `load()` for an id currently owned by `ctx.sessions`; the TEMP overlay records persisted availability, and the durable base refreshes after the live owner detaches. A revision identifies its backing persistence store as well as the backend-local log revision, so reopening against the same store reuses indexed rows while switching to an independent store cannot collide on a session id and local counter. Observation repeats when listing changes during a load; this incorporates a mutating load repair's refreshed revision before commit. Repeated queries and unchanged reopen load no full persisted logs. New, changed, and deleted sessions update on the next stable search. A source or extraction failure cannot mark a row current, and a transaction failure rolls back so a later search retries. +One serialized operation reads the provider-neutral `SessionPersistence` snapshot listing, compares each source-qualified opaque revision with the revision stored beside the indexed session, loads only new or changed logs, reconciles rows in one transaction, and executes the query. It passes the caller's exact abort signal into snapshot listing and inspection, awaits each observer operation, and checks cancellation after each await and before starting more work. Already-current inspection keeps recovery in memory; a supported historical body read may first publish migration and repair. Cancellation settles this query observer without starting another observation or reconciliation step; a shared persistence load or migration already admitted for another observer may continue behind its per-id chain. The operation never calls the backend's mutating `load()` for an id currently owned by `ctx.sessions`; the TEMP overlay records persisted availability, and the durable base refreshes after the live owner detaches. A revision identifies its backing persistence store as well as the backend-local log revision, so reopening against the same store reuses indexed rows while switching to an independent store cannot collide on a session id and local counter. Observation repeats when listing changes during a load; this incorporates migration or mutating load repair's refreshed revision before commit. Repeated queries and unchanged reopen load no full persisted logs. New, changed, and deleted sessions update on the next stable search. A source or extraction failure cannot mark a row current, and a transaction failure rolls back so a later search retries. Persisted documents survive restarts. Live sessions use connection-local TEMP tables, shadow the persisted base for the same id, and reveal that base on detach. Closing the database drops live rows. Unmounting persistence hides durable rows without treating absence as authoritative deletion; remounting observes and reconciles the backend again. Conflicting immutable live and durable headers fail rather than combining sources. The derived schema has its own application id and monotonic schema version. Persistent and TEMP session metadata store the integer `SessionHeader.createdAt` contract in strict `INTEGER` columns. A recognized incompatible version resets only this derived database. A database with a foreign application id or unrecognized user tables is refused before journal-mode mutation, which prevents an accidentally configured canonical session database from being changed. On POSIX filesystems, missing directories and database files are created owner-only so new SQLite sidecars inherit that mode; existing modes are preserved. One service in one process exclusively owns a derived-index path; cross-process writers are unsupported because generations and live TEMP shadow state are connection-owned. -Cancellation rejects queued operations promptly. Once asynchronous source observation starts, the caller waits for that backend promise to settle before rejection, without committing an aborted observation or starting more source/index work. Node's synchronous `DatabaseSync` metadata and MATCH calls cannot be interrupted once executing on the JavaScript thread, so the service checks the signal around those calls but does not promise mid-statement preemption. +Cancellation rejects queued operations promptly. Once source observation starts, cancellation rejects the query observer without committing its result or starting more source/index work; persistence may keep a shared cold load or migration alive for another observer. Node's synchronous `DatabaseSync` metadata and MATCH calls cannot be interrupted once executing on the JavaScript thread, so the service checks the signal around those calls but does not promise mid-statement preemption. ## Alternatives considered @@ -54,6 +54,6 @@ Cancellation rejects queued operations promptly. Once asynchronous source observ Search has a small provider-neutral API while its only backend owns every derived-index state transition. The separate database adds configuration and a lightweight snapshot read before queries, but index corruption, reset, and tokenizer changes cannot endanger canonical logs. Durable revisions avoid full-log reads and rewrites for unchanged sessions; TEMP live overlays preserve current-session truth without making uncheckpointed events durable. -The chosen tokenizer supports short tokens with a smaller index but does not promise substring recall. Literal phrases make query syntax safe and predictable at the cost of excluding boolean/full MATCH expressions. Cancellation is prompt while queued and quiescent while awaiting sources; synchronous SQLite execution remains a non-preemptible section bracketed by signal checks. +The chosen tokenizer supports short tokens with a smaller index but does not promise substring recall. Literal phrases make query syntax safe and predictable at the cost of excluding boolean/full MATCH expressions. Cancellation is prompt while queued and observer-local during shared source work; synchronous SQLite execution remains a non-preemptible section bracketed by signal checks. Unit coverage pins extraction, filters, both search scopes, all default surfaces, metadata-before-ranking, snippets, literal escaping, deterministic ties, complete pagination, scoped cursor invalidation, dynamic persistence mount/unmount, restart reconciliation, live shadow/reveal/reopen, schema safety, rollback retry, and queued/in-flight source-wait cancellation. A keyless real-Loader-path test combines the derived SQLite index with the real JSONL persistence provider. diff --git a/.agents/notes/implemented/feature/2026-07-10-sqlite-session-query-provider.zh.md b/.agents/notes/implemented/feature/2026-07-10-sqlite-session-query-provider.zh.md index 6d8518252d..ccb833886a 100644 --- a/.agents/notes/implemented/feature/2026-07-10-sqlite-session-query-provider.zh.md +++ b/.agents/notes/implemented/feature/2026-07-10-sqlite-session-query-provider.zh.md @@ -34,13 +34,13 @@ Service Definition 包还拥有共享的第一方语义提取与提供方无关 共享提取器会提取消息文本、推理（reasoning）、嵌套的工具调用/结果内容、工具名称与参数、被阻止提示词的原因、待办事项状态与内容，以及错误或结束状态详情。结构性边界、流分片、请求头、成功完成标记，以及通过声明合并扩展的未知事件/内容变体都不会产生文档。surface 分类复用 `foldSurface()`，使搜索与模型历史派生保持一致。 -一个串行化操作会读取提供方无关的 `SessionPersistence` 快照清单，将每个包含源身份的不透明修订号与同已索引会话一并存储的修订号比较，只加载新增或变更的日志，在一个事务中对齐各行，然后执行查询。它将调用方传入的原始中止信号传给快照清单查询和不会修改状态的检查，直接等待每个已启动的后端操作，并在每次等待结束后、启动更多工作之前检查取消状态。因此，即使后端忽略该信号，取消也只会在活跃的后端工作完全停稳后才拒绝，不会启动后续观察或对齐步骤；后续搜索仍会串行等待清理完成。它绝不会调用后端会修改状态的 `load()` 来处理当前由 `ctx.sessions` 拥有的 id；TEMP 覆盖层会记录持久化可用性，实时所有者分离后，持久化基础层随之刷新。修订号同时标识其底层持久化存储与后端本地日志修订版本，因此针对同一存储重新打开服务可以复用已索引行，而切换到独立存储时不会因会话 id 与本地计数器相同而发生冲突。如果加载期间清单发生变化，系统会重复观察；因此，会修改状态的加载修复所产生的新修订号会在提交前纳入结果。重复查询与针对未变更存储的重新打开都不会加载完整的持久化日志。新增、变更与删除的会话会在下一次稳定搜索中更新。源读取或提取失败时，不能将相应行标记为最新状态，事务失败则会回滚，使后续搜索能够重试。 +一个串行化操作会读取提供方无关的 `SessionPersistence` 快照清单，将每个包含源身份的不透明修订号与同已索引会话一并存储的修订号比较，只加载新增或变更的日志，在一个事务中对齐各行，然后执行查询。它将调用方传入的原始中止信号传给快照清单查询与检查，等待每项观察操作，并在每次等待结束后、启动更多工作之前检查取消状态。已经是当前格式的检查只在内存中保留恢复；受支持的历史正文读取可能先发布迁移与修复。取消会结算本次查询观察方，且不启动后续观察或对齐步骤；已经为其他观察方接纳的共享持久化加载或迁移仍可能在其逐 id 链后继续。它绝不会调用后端会修改状态的 `load()` 来处理当前由 `ctx.sessions` 拥有的 id；TEMP 覆盖层会记录持久化可用性，实时所有者分离后，持久化基础层随之刷新。修订号同时标识其底层持久化存储与后端本地日志修订版本，因此针对同一存储重新打开服务可以复用已索引行，而切换到独立存储时不会因会话 id 与本地计数器相同而发生冲突。如果加载期间清单发生变化，系统会重复观察；因此，迁移或会修改状态的加载修复所产生的新修订号会在提交前纳入结果。重复查询与针对未变更存储的重新打开都不会加载完整的持久化日志。新增、变更与删除的会话会在下一次稳定搜索中更新。源读取或提取失败时，不能将相应行标记为最新状态，事务失败则会回滚，使后续搜索能够重试。 持久化文档在重启后仍然存在。实时会话使用连接本地的 TEMP 表，遮蔽相同 id 的持久化基础行，并在实时所有者分离时重新显露该基础行。关闭数据库会删除实时行。卸载持久化服务会隐藏持久化行，但不会把缺失视为权威删除；重新挂载后，系统会再次观察并对齐后端。实时会话头与持久化会话头的不可变字段发生冲突时，系统会失败，而不会合并两个来源。 派生 schema 拥有独立的 application id 与单调递增的 schema 版本。持久化与 TEMP 会话元数据均遵循 `SessionHeader.createdAt` 的整数约定，将其存入严格的 `INTEGER` 列。系统识别到不兼容版本时，只会重置该派生数据库。如果数据库具有不属于本应用的 application id 或无法识别的用户表，系统会在修改日志模式前拒绝该数据库，防止意外配置的规范会话数据库遭到修改。在 POSIX 文件系统上，缺失的目录与数据库文件会以仅所有者可访问的权限创建，使新的 SQLite 伴随文件沿用该模式；现有权限模式保持不变。一个进程中的一个服务独占一条派生索引路径；代际与实时 TEMP 遮蔽状态都归连接所有，因此不支持跨进程写入方。 -取消会使排队中的操作及时被拒绝。异步源观察一旦开始，调用方必须等待该后端 Promise 结算后才会收到拒绝；系统不会提交已中止的观察结果，也不会启动更多源观察或索引工作。Node 的同步 `DatabaseSync` 元数据与 MATCH 调用一旦开始在 JavaScript 线程上执行就无法中断，因此服务会在这些调用前后检查信号，但不承诺在语句执行期间抢占。 +取消会使排队中的操作及时被拒绝。源观察开始后，取消会拒绝查询观察方，不提交其结果，也不启动更多源观察或索引工作；持久化层仍可能为其他观察方保留共享 cold load 或迁移。Node 的同步 `DatabaseSync` 元数据与 MATCH 调用一旦开始在 JavaScript 线程上执行就无法中断，因此服务会在这些调用前后检查信号，但不承诺在语句执行期间抢占。 ## 曾考虑的替代方案 @@ -54,6 +54,6 @@ Service Definition 包还拥有共享的第一方语义提取与提供方无关 搜索只公开精简的提供方无关 API，而唯一后端负责派生索引的全部状态转换。独立数据库增加了配置与查询前的轻量快照读取，但索引损坏、重置与分词器变更都不会危及规范日志。持久化修订号使未变更会话无需读取或重写完整日志；TEMP 实时覆盖层保留当前会话事实，同时不会让尚未经过检查点的事件具有持久性。 -选定的分词器以较小的索引体积支持短 token，但不承诺子串召回。字面短语使查询语法安全且可预测，代价是不支持布尔表达式或完整 MATCH 表达式。取消在操作排队期间会及时生效，在等待数据源期间则会等待其完全停稳；同步 SQLite 执行仍是不可抢占区段。 +选定的分词器以较小的索引体积支持短 token，但不承诺子串召回。字面短语使查询语法安全且可预测，代价是不支持布尔表达式或完整 MATCH 表达式。取消在排队期间及时生效，在共享源工作期间只影响当前观察方；同步 SQLite 执行仍是不可抢占区段。 单元测试将以下行为固化为约定：提取、过滤器、两种搜索范围、所有默认 surface、先过滤元数据再排序、摘要片段、字面量转义、确定性平局处理、完整分页、按范围的游标失效、动态挂载/卸载持久化服务、重启对齐、实时遮蔽、显露与重新打开、schema 安全、回滚重试，以及排队中或进行中的数据源等待取消。一个无需密钥的真实 Loader 路径测试会把派生 SQLite 索引与真实 JSONL 持久化 provider 组合使用。 diff --git a/.agents/notes/implemented/feature/2026-07-20-ptc-typed-tool-returns.i18n.yaml b/.agents/notes/implemented/feature/2026-07-20-ptc-typed-tool-returns.i18n.yaml index 8ef13a4638..76d1a7c3b4 100644 --- a/.agents/notes/implemented/feature/2026-07-20-ptc-typed-tool-returns.i18n.yaml +++ b/.agents/notes/implemented/feature/2026-07-20-ptc-typed-tool-returns.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-20-ptc-typed-tool-returns.md -2026-07-20-ptc-typed-tool-returns.md: a4651eef89d32cdc2330eb5a2562bd34d617a3df -2026-07-20-ptc-typed-tool-returns.zh.md: d8e68e273bd18fe60d8ba8e0bbf435dbb773a9c2 +2026-07-20-ptc-typed-tool-returns.md: cc784ca8c9b1565135bc59c11ae0af973845fe5f +2026-07-20-ptc-typed-tool-returns.zh.md: c4dd52d3c136569610beb8a65303809a1fe5fe4d diff --git a/.agents/notes/implemented/feature/2026-07-20-ptc-typed-tool-returns.md b/.agents/notes/implemented/feature/2026-07-20-ptc-typed-tool-returns.md index a4651eef89..cc784ca8c9 100644 --- a/.agents/notes/implemented/feature/2026-07-20-ptc-typed-tool-returns.md +++ b/.agents/notes/implemented/feature/2026-07-20-ptc-typed-tool-returns.md @@ -73,7 +73,7 @@ Temporary Cordis Plugins follow the same rule: `cordis_mount` returns `{ id, plu ### Persistence, metadata, and spill -Nested dispatch logs the sub-call's full rendered `content`/`isError` on `tool/code-dispatch` but does not persist canonical values. `tool/result` continues to persist only rendered content, error, and optional metadata. A successful final content sequence containing an image is also wrapped in a source-attributed user message and deferred through the outer result; the normal session event makes that model-visible input reconstructable. `SESSION_FORMAT_VERSION` remains unchanged (pre-release shape churn does not bump it) and replay cannot recreate intermediate canonical program values. +Nested dispatch logs the sub-call's full rendered `content`/`isError` on `tool/code-dispatch` but does not persist canonical values. `tool/result` continues to persist only rendered content, error, and optional metadata. A successful final content sequence containing an image is also wrapped in a source-attributed user message and deferred through the outer result; the normal session event makes that model-visible input reconstructable. This feature did not itself require a structural Session-format change; the released v0-to-v1 identity edge preserves these records, and replay still cannot recreate intermediate canonical program values. The opaque `exec.parent` token marks nested calls. Presentation metadata and generic or tool-owned spill projections skip those calls because they have no direct result card and their canonical values never enter context. The outer `run_code` call alone produces one card and may spill its final post-policy presentation; `run_code` intentionally declares neither a result presenter nor presentation metadata, so UI adapters complete the card through their generic raw-content fallback using durable `tool/result.content`. diff --git a/.agents/notes/implemented/feature/2026-07-20-ptc-typed-tool-returns.zh.md b/.agents/notes/implemented/feature/2026-07-20-ptc-typed-tool-returns.zh.md index d8e68e273b..c4dd52d3c1 100644 --- a/.agents/notes/implemented/feature/2026-07-20-ptc-typed-tool-returns.zh.md +++ b/.agents/notes/implemented/feature/2026-07-20-ptc-typed-tool-returns.zh.md @@ -73,7 +73,7 @@ PTC mode 通过运行时请求中的 `{ name: "ToolCallError", memberNamePropert ### 持久化、元数据与 spill -嵌套分发在 `tool/code-dispatch` 上记录子调用完整渲染后的 `content`/`isError`，但不会持久化规范值。`tool/result` 继续只持久化渲染后的内容、错误和可选元数据。包含图片的成功最终内容序列还会包装成带来源归属的用户消息，并经外层结果延后；普通会话事件使该模型可见输入可以重建。`SESSION_FORMAT_VERSION` 保持不变（预发布阶段的形状变动不递增版本号），回放也无法重建程序的规范中间值。 +嵌套分发在 `tool/code-dispatch` 上记录子调用完整渲染后的 `content`/`isError`，但不会持久化规范值。`tool/result` 继续只持久化渲染后的内容、错误和可选元数据。包含图片的成功最终内容序列还会包装成带来源归属的用户消息，并经外层结果延后；普通会话事件使该模型可见输入可以重建。该功能本身不要求结构性 Session 格式变更；已发布的 v0-to-v1 恒等边会保留这些记录，回放仍无法重建程序的规范中间值。 不透明的 `exec.parent` token 用于标识嵌套调用。由于这些调用没有直接对应的结果卡片，而且其规范值永远不会进入上下文，展示元数据以及通用或工具自有的 spill 投影都会跳过它们。只有外层 `run_code` 调用会生成一张卡片，并且可能对 post-policy 处理后的最终展示执行 spill；`run_code` 有意既不声明结果展示器，也不声明展示元数据，因此 UI 适配器会通过通用的原始内容回退机制，使用持久化的 `tool/result.content` 补全该卡片。 diff --git a/.agents/notes/implemented/feature/2026-07-21-instruction-load-all-dedup.i18n.yaml b/.agents/notes/implemented/feature/2026-07-21-instruction-load-all-dedup.i18n.yaml index 6c1ceceba1..4cb9a826ee 100644 --- a/.agents/notes/implemented/feature/2026-07-21-instruction-load-all-dedup.i18n.yaml +++ b/.agents/notes/implemented/feature/2026-07-21-instruction-load-all-dedup.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-21-instruction-load-all-dedup.md -2026-07-21-instruction-load-all-dedup.md: 4cf8ae025cd04d02d00eba7b5b26a661a354e11a -2026-07-21-instruction-load-all-dedup.zh.md: 2dbccc4ac6336ce48cabcfd2ab3842a2f86f152e +2026-07-21-instruction-load-all-dedup.md: fc88b4ea6fcbdcda76b9a7da763a04ead2557f50 +2026-07-21-instruction-load-all-dedup.zh.md: f74585850ddaf814fce1cb36239bc666a4142696 diff --git a/.agents/notes/implemented/feature/2026-07-21-instruction-load-all-dedup.md b/.agents/notes/implemented/feature/2026-07-21-instruction-load-all-dedup.md index 4cf8ae025c..fc88b4ea6f 100644 --- a/.agents/notes/implemented/feature/2026-07-21-instruction-load-all-dedup.md +++ b/.agents/notes/implemented/feature/2026-07-21-instruction-load-all-dedup.md @@ -34,4 +34,4 @@ Dedup is enforced during reconciliation, not only at baseline composition. Each ## Consequences -A directory with two distinct real instruction files now surfaces both; a directory whose second file merely mirrors the first still renders once, and the ubiquitous symlink case is unchanged. The visible behavior difference is confined to transition repositories that carry two distinct real files. The scope-key shape changed from a tier sentinel to a per-candidate key and `previousPath` disappeared from the durable change metadata; `dsh-session` keeps no compatibility promise for older sessions, so both are free changes. The version cache row grew a `trimmedDigest` field, and reconciliation now compares trimmed content per directory, so an unchanged file can be removed by a sibling's convergence — a transition the [state model](2026-06-24-workspace-context.md) previously could not produce. +A directory with two distinct real instruction files now surfaces both; a directory whose second file merely mirrors the first still renders once, and the ubiquitous symlink case is unchanged. The visible behavior difference is confined to transition repositories that carry two distinct real files. The released v0 baseline uses the per-candidate scope key and omits `previousPath` from durable change metadata. The version cache row carries `trimmedDigest`, and reconciliation compares trimmed content per directory, so an unchanged file can be removed by a sibling's convergence — a transition the [state model](2026-06-24-workspace-context.md) previously could not produce. diff --git a/.agents/notes/implemented/feature/2026-07-21-instruction-load-all-dedup.zh.md b/.agents/notes/implemented/feature/2026-07-21-instruction-load-all-dedup.zh.md index 2dbccc4ac6..f74585850d 100644 --- a/.agents/notes/implemented/feature/2026-07-21-instruction-load-all-dedup.zh.md +++ b/.agents/notes/implemented/feature/2026-07-21-instruction-load-all-dedup.zh.md @@ -34,4 +34,4 @@ Status: implemented ## 后果 -一个携带两个不同真实指令文件的目录现在会把两者都暴露；一个第二个文件仅仅是镜像的目录仍只渲染一次，而无处不在的符号链接场景保持不变。可见的行为差异被限定在携带两个不同真实文件的迁移期仓库中。scope 键的形态从层级哨兵改为按候选划分，`previousPath` 也从持久化的变更元数据中消失；`dsh-session` 对旧会话不作兼容承诺，因此两者都是无成本的改动。版本缓存行新增了一个 `trimmedDigest` 字段，协调过程现在按目录比较去空白后的内容，因此一个未变更的文件可以被同级文件的收敛所移除——这是[状态模型](2026-06-24-workspace-context.zh.md)此前无法产生的转换。 +一个携带两个不同真实指令文件的目录现在会把两者都暴露；一个第二个文件仅仅是镜像的目录仍只渲染一次，而无处不在的符号链接场景保持不变。可见的行为差异被限定在携带两个不同真实文件的迁移期仓库中。已发布 v0 基线使用按候选划分的 scope 键，并从持久化变更元数据中省略 `previousPath`。版本缓存行携带 `trimmedDigest` 字段，协调过程按目录比较去空白后的内容，因此一个未变更的文件可以被同级文件的收敛所移除——这是[状态模型](2026-06-24-workspace-context.zh.md)此前无法产生的转换。 diff --git a/.agents/notes/implemented/feature/2026-07-21-local-instruction-overlay.i18n.yaml b/.agents/notes/implemented/feature/2026-07-21-local-instruction-overlay.i18n.yaml index 70b3c64b7d..2f11dc6de9 100644 --- a/.agents/notes/implemented/feature/2026-07-21-local-instruction-overlay.i18n.yaml +++ b/.agents/notes/implemented/feature/2026-07-21-local-instruction-overlay.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-21-local-instruction-overlay.md -2026-07-21-local-instruction-overlay.md: 9706285f4e699bfb6db9591f9850e4f95f1d2a15 -2026-07-21-local-instruction-overlay.zh.md: b375455dcb1f7e5c157907501f342cbd29a51667 +2026-07-21-local-instruction-overlay.md: 0574cb3568ce1dc41f49f9de2b76131ce75c409b +2026-07-21-local-instruction-overlay.zh.md: cd7406374cb6f39ff3c24fb95fee5f53410e11a0 diff --git a/.agents/notes/implemented/feature/2026-07-21-local-instruction-overlay.md b/.agents/notes/implemented/feature/2026-07-21-local-instruction-overlay.md index 9706285f4e..0574cb3568 100644 --- a/.agents/notes/implemented/feature/2026-07-21-local-instruction-overlay.md +++ b/.agents/notes/implemented/feature/2026-07-21-local-instruction-overlay.md @@ -34,4 +34,4 @@ The base and local candidates in one directory must stay independent across base ## Consequences -`.local.` guidance is read by default across all products with no per-deployment configuration, matching neighboring tools. Each project directory can contribute a durable scope per existing candidate rather than one, so dynamic discovery, edits, and removals reconcile the base and local files independently. The scope key is now [per-candidate](2026-07-21-instruction-load-all-dedup.md); `dsh-session` keeps no compatibility promise for older sessions, so this is a free change. The user-global scope remains base-only, recorded as a Known Limitation in the package README. +`.local.` guidance is read by default across all products with no per-deployment configuration, matching neighboring tools. Each project directory can contribute a durable scope per existing candidate rather than one, so dynamic discovery, edits, and removals reconcile the base and local files independently. The released v0 baseline already records the [per-candidate](2026-07-21-instruction-load-all-dedup.md) scope key. The user-global scope remains base-only, recorded as a Known Limitation in the package README. diff --git a/.agents/notes/implemented/feature/2026-07-21-local-instruction-overlay.zh.md b/.agents/notes/implemented/feature/2026-07-21-local-instruction-overlay.zh.md index b375455dcb..cd7406374c 100644 --- a/.agents/notes/implemented/feature/2026-07-21-local-instruction-overlay.zh.md +++ b/.agents/notes/implemented/feature/2026-07-21-local-instruction-overlay.zh.md @@ -34,4 +34,4 @@ Status: implemented ## 后果 -`.local.` 指导在所有产品中默认被读取，无需按部署单独配置，与邻近工具保持一致。每个项目目录可以为每个存在的候选贡献一个持久 scope 而非仅一个，因此动态发现、编辑和移除会分别独立地协调基础文件与本地文件。scope 键现在[按候选划分](2026-07-21-instruction-load-all-dedup.zh.md)；`dsh-session` 对旧会话不作兼容承诺，因此这是一次无成本的改动。用户全局 scope 仍然只有基础文件，这一点作为已知限制记录在包 README 中。 +`.local.` 指导在所有产品中默认被读取，无需按部署单独配置，与邻近工具保持一致。每个项目目录可以为每个存在的候选贡献一个持久 scope 而非仅一个，因此动态发现、编辑和移除会分别独立地协调基础文件与本地文件。已发布 v0 基线已经记录[按候选划分](2026-07-21-instruction-load-all-dedup.zh.md)的 scope 键。用户全局 scope 仍然只有基础文件，这一点作为已知限制记录在包 README 中。 diff --git a/.agents/notes/implemented/feature/2026-07-22-durable-subagent-catalog-and-list-agents.i18n.yaml b/.agents/notes/implemented/feature/2026-07-22-durable-subagent-catalog-and-list-agents.i18n.yaml index 2832c8e125..9d25cc9d42 100644 --- a/.agents/notes/implemented/feature/2026-07-22-durable-subagent-catalog-and-list-agents.i18n.yaml +++ b/.agents/notes/implemented/feature/2026-07-22-durable-subagent-catalog-and-list-agents.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-22-durable-subagent-catalog-and-list-agents.md -2026-07-22-durable-subagent-catalog-and-list-agents.md: 813ad0a57bf5399cf641f8a7271671da57ce3966 -2026-07-22-durable-subagent-catalog-and-list-agents.zh.md: ab5351647da32384645c036c0c5e3e2281ba645a +2026-07-22-durable-subagent-catalog-and-list-agents.md: df7298b542a71e043cf00e0aae1201586f94321e +2026-07-22-durable-subagent-catalog-and-list-agents.zh.md: 93fe9a7d5b2728febeeebfb1f8b0dfa8c15b8719 diff --git a/.agents/notes/implemented/feature/2026-07-22-durable-subagent-catalog-and-list-agents.md b/.agents/notes/implemented/feature/2026-07-22-durable-subagent-catalog-and-list-agents.md index 813ad0a57b..df7298b542 100644 --- a/.agents/notes/implemented/feature/2026-07-22-durable-subagent-catalog-and-list-agents.md +++ b/.agents/notes/implemented/feature/2026-07-22-durable-subagent-catalog-and-list-agents.md @@ -39,7 +39,7 @@ The subagent service keeps `sessionQuery` optional so start and follow-up remain `listChildren(parentSessionId, signal?)` forwards the caller's signal to `traceSession()` and the conditional exact `readEvent()` operation. `listEvents()` has no cancellation parameter, so the listing path checks the signal before and after that await and after each candidate settles. If any query operation rejects after the signal aborts, the service normalizes the result to `SubagentError` with stable code `CANCELLED`; a backend abort error or a diagnostic-mapped query error cannot escape or become a successful partial listing. -This descriptor-read path is the correctness baseline, not a claim that work is linear only in the number of direct children. Let D be the number of direct-child candidates, C be the number of persisted sessions scanned by each persistence listing, and L_i be the size of candidate i's full log. One corpus trace is followed by `sessionQuery.listEvents(childId)` for every candidate. A candidate with no descriptor is omitted, and one with multiple descriptors is diagnosed without another read; only a candidate with exactly one descriptor is loaded again through `sessionQuery.readEvent({ sessionId: childId, seq })`. The read must return the same immutable session header observed by the trace, including the direct-parent relationship, and its target must still be the located descriptor event; a mismatch is per-child corruption. In the persisted-only worst case, each exact read repeats `persistence.list()`, loads the full child log, and clones its events, for O(D × C + Σ L_i) work up to constant factors; a candidate with exactly one descriptor pays those costs twice, while other candidates pay them once. A live candidate similarly takes one detached in-memory snapshot of its full log, or two when its descriptor is read. Session query resolves persisted candidates through the persistence seam's non-mutating `inspect()` read, which returns the valid stored prefix without repairing a torn tail or closing an interrupted turn, so listing is storage-read-only; repair remains the resume path's concern. The first version accepts these repeated reads as the no-index correctness baseline, but deployments must treat total corpus and child-log size—not only direct-child count—as the capacity constraint. Listing creates no Agent and appends no catalog, descriptor, or repair event. The model-hidden descriptor remains outside the conversation surface and survives compaction, so compacted and uncompacted children must enumerate identically. +This descriptor-read path is the correctness baseline, not a claim that work is linear only in the number of direct children. Let D be the number of direct-child candidates, C be the number of persisted sessions scanned by each persistence listing, and L_i be the size of candidate i's full log. One corpus trace is followed by `sessionQuery.listEvents(childId)` for every candidate. A candidate with no descriptor is omitted, and one with multiple descriptors is diagnosed without another read; only a candidate with exactly one descriptor is loaded again through `sessionQuery.readEvent({ sessionId: childId, seq })`. The read must return the same immutable session header observed by the trace, including the direct-parent relationship, and its target must still be the located descriptor event; a mismatch is per-child corruption. In the persisted-only worst case, each exact read repeats `persistence.list()`, loads the full child log, and clones its events, for O(D × C + Σ L_i) work up to constant factors; a candidate with exactly one descriptor pays those costs twice, while other candidates pay them once. A live candidate similarly takes one detached in-memory snapshot of its full log, or two when its descriptor is read. Session query resolves persisted candidates through `inspect()`: already-current recovery remains in memory, while a supported historical body read may first publish its migrated and repaired current generation. Listing creates no Agent and appends no catalog, descriptor, or repair event, but its first historical read is not storage-read-only. The first version accepts these repeated reads as the no-index correctness baseline, but deployments must treat total corpus and child-log size—not only direct-child count—as the capacity constraint. The model-hidden descriptor remains outside the conversation surface and survives compaction, so compacted and uncompacted children must enumerate identically. If measured scale later requires an index, that index is derived state: session headers and child descriptors remain authoritative, and rebuilding or corruption fallback must reproduce the same results. An index cannot become a second authorization source or make an unpublished child visible. @@ -86,7 +86,7 @@ The first version has no child deletion operation. If later product behavior del **Return separate child and diagnostic arrays.** Separate arrays introduce two ordering domains or require exposing another sort key to reconstruct candidate order. One discriminated entry array preserves the trace order while keeping child and diagnostic fields type-safe. -**Read candidates through the repairing `load()` path.** Reusing resume's `load()` semantics would let discovery durably close an interrupted tail early, but turns a listing query into a mutating operation and couples its failure modes to write coordination. Session query's corpus reads already use the non-mutating `inspect()` contract, so listing stays storage-read-only and leaves tail repair to the resume path that needs it. +**Read candidates through the repairing `load()` path.** For already-current input, reusing resume's `load()` semantics would durably close an interrupted tail on every discovery, while `inspect()` keeps those closers in memory. Historical inspection may already publish one migration and repair, but it does not make recurring current listings mutating or couple them to `load()` repair on every read. **Paginate or cap the query now (deferred).** This bounds one result, but makes model discovery stateful and can hide older children unless the model follows a cursor. The first version has no cursor, page arguments, or candidate-limit configuration and returns the complete stably ordered set; a service-level bound remains a later decision if measured scale requires it. @@ -103,7 +103,7 @@ The first version has no child deletion operation. If later product behavior del - Session tracing observes the complete logical corpus, then descriptor validation reads every direct-child log once and candidates with exactly one descriptor twice. In the persisted-only worst case, work is O(D × C + Σ L_i), not merely O(D), because each exact read rescans persistence and loads and clones the full candidate log. A later derived index must preserve the same authorization, per-child diagnostic, and fallback behavior. - Corpus construction is an all-or-nothing trust boundary: one live/persisted header conflict fails the initial trace and hides otherwise healthy siblings. Per-child isolation begins only after that trace succeeds. -- A torn child tail is surfaced, not repaired: the non-mutating `inspect()` read returns the valid stored prefix, so a child interrupted mid-write may list from a shorter log until the resume path's repairing load closes it. +- For already-current storage, a torn child tail is surfaced rather than repaired: `inspect()` returns the valid stored prefix, so a child interrupted mid-write may list from a shorter log until the resume path's repairing load closes it. Historical input is migrated and repaired before this current inspection behavior applies. - There is no deletion operation, so persisted children remain listed for as long as their sessions remain in persistence even though live Agent resources remain bounded by resident Activations. - The service returns every direct session-backed subagent and diagnostic without a cursor or candidate cap. Stable ordering makes the result deterministic; the model projection avoids one-shot context growth but remains unbounded in the number of continuable children. - `running` and `inactive` are process-local corpus snapshots, not outcomes or delivery promises. Another process may activate a persisted child while this process reports it as `inactive`; cross-process accuracy requires a shared lease. diff --git a/.agents/notes/implemented/feature/2026-07-22-durable-subagent-catalog-and-list-agents.zh.md b/.agents/notes/implemented/feature/2026-07-22-durable-subagent-catalog-and-list-agents.zh.md index ab5351647d..93fe9a7d5b 100644 --- a/.agents/notes/implemented/feature/2026-07-22-durable-subagent-catalog-and-list-agents.zh.md +++ b/.agents/notes/implemented/feature/2026-07-22-durable-subagent-catalog-and-list-agents.zh.md @@ -39,7 +39,7 @@ subagent 服务将 `sessionQuery` 保持为可选依赖，因此没有该服务 `listChildren(parentSessionId, signal?)` 会把调用方的取消信号转发给 `traceSession()` 和条件性精确 `readEvent()` 操作。`listEvents()` 不接受取消参数，因此列表查询路径会在等待该操作的前后，以及每个候选处理完成后检查信号。如果取消信号触发后有查询操作以拒绝结算，服务会将结果归一化为 `SubagentError`，并携带稳定错误码 `CANCELLED`；后端中止错误或可映射为 diagnostic 的查询错误均不会逃逸，也不会使调用以成功的部分列表返回。 -这条描述符读取路径是正确性基线，并不声称工作量只与直接 child 数量呈线性关系。令 D 为直接 child 候选数量，C 为每次持久化列表查询所扫描的持久化会话数量，L_i 为候选 i 的完整日志大小。一次语料追踪后，每个候选都会执行 `sessionQuery.listEvents(childId)`。没有描述符的候选会被排除；含有多个描述符的候选会直接产生 diagnostic，无需再次读取；只有恰好含有一个描述符的候选才会通过 `sessionQuery.readEvent({ sessionId: childId, seq })` 再次加载。此次读取返回的不可变会话 header 必须与追踪时观测到的相同，包括直接 parent 关系，并且读取目标仍必须是先前定位的描述符事件；任何不一致均视为该 child 损坏。对于只存在于持久化存储中的最坏情况，每次精确读取都会重复执行 `persistence.list()`、加载完整 child 日志并克隆其中的事件，因此忽略常数因子后的工作量为 O(D × C + Σ L_i)；恰好含有一个描述符的候选承担两次这类成本，其他候选只承担一次。存活候选同样会对其完整日志取得一份分离的内存快照；读取其描述符时则会取得两份。会话查询通过持久化 seam 的非变更 `inspect()` 读取解析持久化候选：它返回有效的已存储前缀，既不修复撕裂的尾部，也不关闭中断的 turn，因此列表查询是存储只读操作；修复仍是恢复路径的职责。第一版接受这些重复读取，将其作为无索引的正确性基线，但部署必须将语料总量和 child 日志大小，而不仅是直接 child 数量，视为容量约束。列表查询不会创建 Agent，也不会追加任何目录、描述符或修复事件。对模型隐藏的描述符始终位于对话 surface 之外，并且会在压缩后保留，因此经过压缩和未经压缩的 child 必须枚举出相同结果。 +这条描述符读取路径是正确性基线，并不声称工作量只与直接 child 数量呈线性关系。令 D 为直接 child 候选数量，C 为每次持久化列表查询所扫描的持久化会话数量，L_i 为候选 i 的完整日志大小。一次语料追踪后，每个候选都会执行 `sessionQuery.listEvents(childId)`。没有描述符的候选会被排除；含有多个描述符的候选会直接产生 diagnostic，无需再次读取；只有恰好含有一个描述符的候选才会通过 `sessionQuery.readEvent({ sessionId: childId, seq })` 再次加载。此次读取返回的不可变会话 header 必须与追踪时观测到的相同，包括直接 parent 关系，并且读取目标仍必须是先前定位的描述符事件；任何不一致均视为该 child 损坏。对于只存在于持久化存储中的最坏情况，每次精确读取都会重复执行 `persistence.list()`、加载完整 child 日志并克隆其中的事件，因此忽略常数因子后的工作量为 O(D × C + Σ L_i)；恰好含有一个描述符的候选承担两次这类成本，其他候选只承担一次。存活候选同样会对其完整日志取得一份分离的内存快照；读取其描述符时则会取得两份。会话查询通过 `inspect()` 解析持久化候选：已经是当前格式的恢复留在内存中，受支持的历史正文读取则可能先发布迁移并修复后的当前 generation。列表查询不会创建 Agent，也不会追加目录、描述符或修复事件，但其首次历史读取并非存储只读。第一版接受这些重复读取，将其作为无索引的正确性基线，但部署必须将语料总量和 child 日志大小，而不仅是直接 child 数量，视为容量约束。对模型隐藏的描述符始终位于对话 surface 之外，并且会在压缩后保留，因此经过压缩和未经压缩的 child 必须枚举出相同结果。 如果实测规模日后需要索引，该索引属于派生状态：会话 header 和 child 描述符仍是权威信息，重建或损坏回退必须复现相同结果。索引不能成为第二个鉴权来源，也不能让尚未发布的 child 变得可见。 @@ -86,7 +86,7 @@ diagnostic 是瞬时查询结果，不属于会话事件或目录状态。推导 **分别返回 child 和 diagnostic 数组。** 分离的数组会引入两个排序域，或者要求公开另一个排序键才能重建候选顺序。一个带判别字段的条目数组既能保留追踪顺序，也能保证 child 与 diagnostic 字段的类型安全。 -**通过会触发修复的 `load()` 路径读取候选。** 复用恢复路径的 `load()` 语义可以让发现提前持久化地关闭中断尾部，但会把列表查询变成变更操作，并使其失败模式与写协调耦合。会话查询的语料读取本就使用非变更的 `inspect()` 约定，因此列表查询保持存储只读，尾部修复留给真正需要它的恢复路径。 +**通过会触发修复的 `load()` 路径读取候选。** 对已经是当前格式的输入，复用恢复路径的 `load()` 语义会让每次发现都持久关闭中断尾部，而 `inspect()` 只在内存中保留这些 closer。历史检查可能已经发布一次迁移与修复，但这不会让重复的当前列表变成修改操作，也不会让它们每次都耦合到 `load()` 修复。 **立即为查询分页或设置上限（暂缓）。** 这可以限制一次结果的大小，但会使模型发现成为有状态操作，而且除非模型继续跟随 cursor，否则可能隐藏更早的 child。第一版没有 cursor、分页参数或候选数量上限配置，而是返回经稳定排序的完整集合；如果实测规模需要限制，服务级限制仍留待后续决策。 @@ -103,7 +103,7 @@ diagnostic 是瞬时查询结果，不属于会话事件或目录状态。推导 - 会话追踪会观察完整的逻辑语料，随后描述符校验会读取每个直接 child 的日志一次，并对恰好含有一个描述符的候选读取两次。对于只存在于持久化存储中的最坏情况，工作量为 O(D × C + Σ L_i)，而不只是 O(D)，因为每次精确读取都会重新扫描持久化存储，并加载和克隆候选的完整日志。后续的派生索引必须保持相同的鉴权、逐 child diagnostic 和回退行为。 - 语料构建是一个全有或全无的信任边界：一处存活／持久化 header 冲突就会导致初始追踪失败，并隐藏原本健康的 sibling。只有初始追踪成功后，逐 child 隔离才会生效。 -- 撕裂的 child 尾部会被呈现而非修复：非变更的 `inspect()` 读取返回有效的已存储前缀，因此写入中途被打断的 child 在恢复路径的修复加载将其关闭之前，可能以较短的日志形式出现在列表中。 +- 对已经是当前格式的存储，撕裂的 child 尾部会被呈现而非修复：`inspect()` 返回有效的已存储前缀，因此写入中途被打断的 child 在恢复路径的修复加载将其关闭之前，可能以较短的日志形式出现在列表中。历史输入会先迁移并修复，再应用这项当前检查行为。 - 没有删除操作，因此只要 child 会话仍保留在持久化存储中，它们就会继续出现在列表里，但存活 Agent 资源仍由驻留 Activation 数量限制。 - 服务会返回每个直接且由会话支撑的 subagent 和 diagnostic，不设 cursor 或候选数量上限。稳定排序可使结果确定；模型投影避免了一次性 child 带来的上下文增长，但可继续 child 的数量仍无上限。 - `running` 和 `inactive` 是进程本地语料快照，而非结果或消息送达承诺。另一个进程可能在当前进程将某个持久化 child 报告为 `inactive` 时激活它；跨进程准确性需要共享租约。 diff --git a/.agents/notes/implemented/feature/2026-07-23-session-telemetry-otel-revival.i18n.yaml b/.agents/notes/implemented/feature/2026-07-23-session-telemetry-otel-revival.i18n.yaml index 67b922e203..5350e26149 100644 --- a/.agents/notes/implemented/feature/2026-07-23-session-telemetry-otel-revival.i18n.yaml +++ b/.agents/notes/implemented/feature/2026-07-23-session-telemetry-otel-revival.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-23-session-telemetry-otel-revival.md -2026-07-23-session-telemetry-otel-revival.md: 1110db4b0dfcfd68bde97f9964383ddd041f328b -2026-07-23-session-telemetry-otel-revival.zh.md: 6c046c5f963760638deacacda51f8d94a13e0efd +2026-07-23-session-telemetry-otel-revival.md: 27e9eb9d073c3d4f726595a16bf4b63293207e22 +2026-07-23-session-telemetry-otel-revival.zh.md: 6928d7086c47460e33660dbf3ced6cea7119f456 diff --git a/.agents/notes/implemented/feature/2026-07-23-session-telemetry-otel-revival.md b/.agents/notes/implemented/feature/2026-07-23-session-telemetry-otel-revival.md index 1110db4b0d..27e9eb9d07 100644 --- a/.agents/notes/implemented/feature/2026-07-23-session-telemetry-otel-revival.md +++ b/.agents/notes/implemented/feature/2026-07-23-session-telemetry-otel-revival.md @@ -6,17 +6,17 @@ English | [中文](2026-07-23-session-telemetry-otel-revival.zh.md) ## Problem -Every deployment that wants harness sessions in an observability stack must hand-roll a session-log consumer: subscription, lifecycle handoff, and — hardest — redaction, since the raw log carries file contents and command output that may embed credentials. A telemetry seam and OTel backend shipped once on the `session-telemetry-otlp-rfc` branch (PR #222/#231) but never reached master: the proposal exported raw session events verbatim, which legal review declined. The capture-side design (backend contract, coordinator, handoff cursor, chunk projection) was sound and reviewed; the export-side stance was the blocker. +Every deployment that wants harness sessions in an observability stack must hand-roll a session-log consumer: subscription, lifecycle handoff, and — hardest — redaction, since the raw log carries file contents and command output that may embed credentials. A telemetry seam and OTel backend shipped once on the `session-telemetry-otlp-rfc` branch (PR #222/#231) but never reached master: the proposal exported raw session events verbatim, which legal review declined. The reusable capture-side design comprised the backend contract, coordinator, handoff cursor, and session-event subscription; the export-side stance was the blocker. ## Decision `packages/session/` (formerly `telemetry/`) revives the two reviewed packages under the SDK stance — the harness provides the capability, the deployment configures where records go and owns what leaves in them: -- **`@deepseek-ai/dsh-session-telemetry`** — the seam. `SessionTelemetrySink` (`emit`/`flush?`/`shutdown`), the service-registered `SessionTelemetryBackend` form, and `SessionTelemetryCoordinator` owning capture: live adoption with cursor read-back and the per-append firehose (project → `structuredClone` → redact → `emit`, zero I/O), buffer-free on-demand replay from the canonical log, the fixed first-chunk-per-(turn, step) projection, the live `agent/error` relay, and live dispose-time `shutdown` records. +- **`@deepseek-ai/dsh-session-telemetry`** — the seam. `SessionTelemetrySink` (`emit`/`flush?`/`shutdown`), the service-registered `SessionTelemetryBackend` form, and `SessionTelemetryCoordinator` own complete capture: each new Session object replays every canonical event from seq 0, then the per-append firehose deep-copies, redacts, and hands off every event with zero I/O; re-adopting the same object resumes after its module-scope cursor. Buffer-free on-demand capture uses the same one-record-per-event mapping through an optional inclusive boundary. Ledger identity includes `session.id`, `session.format_version`, and `event.seq`; live capture also relays `agent/error` and creates dispose-time `shutdown` records. - **The `session-telemetry/record` waterfall** — the delta over the branch version and the seam's redaction extension point. Every record passes it before reaching any backend; the seam ships NO rules of its own — the innermost `next()` is a pass-through, deployments mount their rules as listeners (stacking by transforming `next()`'s return value), and a throwing rule withholds the record fail-closed. Redaction applies to the exported copy only; the canonical log is never rewritten. - **`@deepseek-ai/dsh-session-telemetry-otel`** — the reference backend: OTel JS SDK log pipeline (`LoggerProvider` → `BatchLogRecordProcessor` → OTLP/HTTP exporter), configured verbatim through `exporter`/`processor` passthroughs. `DISABLED` is the default and constructs no transport; the [feedback-gated telemetry decision](2026-08-05-feedback-gated-session-telemetry.md) defines the explicit `FULL` and `FEEDBACK_ONLY` delivery modes, which require `exporter.url`, without moving the redaction or backend boundary. [Buffer-free feedback replay](../simplification/2026-08-06-buffer-free-feedback-telemetry.md) avoids a second in-memory copy of the session prefix. -The boundary axiom holds: the harness's aspect ends at `emit()`. Batching, retry, queueing, and loss policy are the reporting SDK's, configured through passthroughs — delivery is best-effort (at-most-once across a crash), which the READMEs state plainly. +The boundary axiom holds: the harness's aspect ends at `emit()`. Batching, retry, queueing, and loss policy are the reporting SDK's, configured through passthroughs. Delivery is best-effort: a crash can lose queued records, while constructing the Session again can replay records already delivered before the crash. Receivers therefore deduplicate ledger rows on `(session.id, session.format_version, event.seq)`. ## Alternatives considered @@ -28,10 +28,10 @@ The boundary axiom holds: the harness's aspect ends at `emit()`. Batching, retry **Map onto OTel spans (GenAI semantic conventions) instead of logs.** Rejected for this revival: the branch implementation's log mapping is reviewed and shipped-shaped; the span model is lossy for forkable, interruptible sessions and belongs to a future consumer with real span queries to serve. -**Full-log replay when no handoff cursor survived (re-export constructor seeds).** Shipped in the first revival round, then narrowed: adoption now replays from the session's construction boundary (`Session.firstLiveSeq`, the constructor-seed length; `Session.inheritedEventCount` cannot serve because it is the durable fork-lineage cut and a resumed session's constructor seed is its full stored log). A resumed session's history already shipped from the previous process under the same id, and a fork's inherited prefix already shipped in the parent's stream — re-exporting either re-billed every resume for its full history and doubled query-time counts on OTLP backends with no native ingest dedupe. Receivers stitch fork lineage via `session.parent_id` + `session.seed_length`. What the narrowing gives up, consistently with the at-most-once stance: a resume no longer backfills records the previous process failed to deliver (telemetry unmounted then, or queued at crash) — the full replay's only real benefit, bought at the common case's expense. A deployment that states a backfill requirement needs the deferred outbox above, not replay. The boundary also swallows the synthetic turn closers `SessionPersistence.load()` writes when repairing a crash-interrupted log (they sit below `firstLiveSeq` despite never existing in the previous process) — deliberate, not incidental: exporting a synthetic closer cannot complete the remote turn whose real tail records died in the crashed process's queue, it can only make an incomplete turn look closed. The wire stream stays faithful to what the crashed process actually shipped; receivers read a never-closed turn on a resumed stream as "the previous process died inside it" (the OTel README states the rule), and a later clean `shutdown` marker attests only to the resumed process's exit. Threading the pre-repair boundary through load/prepare so repairs export as live events would couple three packages to un-ship that signal. +**Start a new Session object at its constructor boundary and skip the seed prefix.** Rejected because this assumes another process or parent identity successfully delivered those records. It loses history when telemetry mounts after the original run, when the SDK queue dies in a crash, and when format migration or crash repair creates the current canonical seed before telemetry sees the restored object. Every new object instead starts without a handoff cursor and replays from seq 0, including a fork's inherited prefix and a resumed or migrated log. Re-adopting the same object still resumes after its cursor, so HMR does not duplicate the settled prefix. Full replay can duplicate previously delivered rows and intentionally repeats inherited events under a child session id; receiver-side deduplication uses `(session.id, session.format_version, event.seq)`, while `session.parent_id` and `session.seed_length` preserve lineage. This replay is opportunistic recovery, not an at-least-once guarantee: records can still be lost if no later Session object is constructed, so a deployment requiring guaranteed backfill needs the deferred outbox. **Forwarding the seam's turn-boundary `flush()` hint to the OTel provider's `forceFlush()`.** Shipped in the first revival round, then removed: three distinct silent-loss paths shared the wrapper state — a dispose racing an in-flight flush (the SDK's concurrent-flush guard makes shutdown's internal drain skip), overlapping hints displacing the retained promise, and the provider's fixed 30-second flush timeout rejecting while the processor still drains. Every path exists only because the forwarding made this backend the process's second flusher against undocumented SDK internals from the upstream experimental tree; with no `flush()` implemented, the batch processor is the only flusher, its `scheduledDelayMillis` (already deployment-tunable through the `processor` passthrough) governs export cadence, and `shutdown()`'s drain is complete by construction. Reinstate only if a deployment states a turn-boundary latency requirement `scheduledDelayMillis` cannot meet — and then by calling the retained `BatchLogRecordProcessor`'s own `forceFlush()`, never the provider's timeout-wrapped one. ## Consequences -A deployment adds one `cordis.yml` entry with an OTLP endpoint and explicitly selects `FULL` to stream sessions into an OTel-compatible stack or `FEEDBACK_ONLY` to replay a canonical-log prefix when feedback is recorded. `DISABLED` is the [default](2026-08-10-telemetry-default-off.md) and constructs no reporting pipeline; removing the entry remains a silent opt-out, while the disabled mode keeps the local feedback warning. A rule-free deployment exports records exactly as captured — including any credentials embedded in file contents or command output — so a deployment crossing a trust boundary must mount `session-telemetry/record` listeners, and both READMEs state this plainly. Where rules are mounted, exported bodies can differ from canonical log bytes, so receivers must not treat telemetry as a byte-exact replica; the log remains the source of truth. Crash durability is explicitly out of scope until the outbox decision above is revisited. +A deployment adds one `cordis.yml` entry with an OTLP endpoint and explicitly selects `FULL` to stream complete canonical events into an OTel-compatible stack or `FEEDBACK_ONLY` to replay a complete canonical-log prefix when feedback is recorded. `DISABLED` is the [default](2026-08-10-telemetry-default-off.md) and constructs no reporting pipeline; removing the entry remains a silent opt-out, while the disabled mode keeps the local feedback warning. A rule-free deployment exports each event body exactly as captured — including every assistant chunk and any credentials embedded in file contents or command output — so a deployment crossing a trust boundary must mount `session-telemetry/record` listeners, and both READMEs state this plainly. Where rules are mounted, exported bodies can differ from canonical log bytes, so receivers must not treat telemetry as a byte-exact replica; the log remains the source of truth. New-object replay can duplicate ledger rows, and crash durability remains out of scope until the outbox decision above is revisited. diff --git a/.agents/notes/implemented/feature/2026-07-23-session-telemetry-otel-revival.zh.md b/.agents/notes/implemented/feature/2026-07-23-session-telemetry-otel-revival.zh.md index 6c046c5f96..6928d7086c 100644 --- a/.agents/notes/implemented/feature/2026-07-23-session-telemetry-otel-revival.zh.md +++ b/.agents/notes/implemented/feature/2026-07-23-session-telemetry-otel-revival.zh.md @@ -6,18 +6,18 @@ Status: implemented ## 问题 -每个想把 harness 会话接入可观测性体系的部署方都得手写一个会话日志消费方：订阅、生命周期交接、以及最难的脱敏——原始日志携带文件内容与命令输出，可能内嵌凭据。遥测 seam 和 OTel 后端曾在 `session-telemetry-otlp-rfc` 分支（PR #222/#231）上完成过一版，但从未进入 master：该提案将原始会话事件原样导出，法务评审未予通过。捕获侧设计（后端约定、coordinator、handoff 游标、分片投影）本身合理且经过评审；导出侧的立场才是阻塞点。 +每个想把 harness 会话接入可观测性体系的部署方都得手写一个会话日志消费方：订阅、生命周期交接、以及最难的脱敏——原始日志携带文件内容与命令输出，可能内嵌凭据。遥测 seam 和 OTel 后端曾在 `session-telemetry-otlp-rfc` 分支（PR #222/#231）上完成过一版，但从未进入 master：该提案将原始会话事件原样导出，法务评审未予通过。可复用的捕获侧设计包括后端约定、coordinator、handoff 游标与会话事件订阅；导出侧的立场才是阻塞点。 ## 决策 `packages/session/`（原 `telemetry/`）以 SDK 立场复活这两个经过评审的包——harness 提供能力，部署方配置上报去向并对导出内容负责： -- **`@deepseek-ai/dsh-session-telemetry`** —— seam 本体。`SessionTelemetrySink`（`emit`/`flush?`/`shutdown`）、服务注册形态的 `SessionTelemetryBackend`、以及拥有捕获侧的 `SessionTelemetryCoordinator`：带游标回读的实时纳管与逐 append 的 firehose（投影 → `structuredClone` → 脱敏 → `emit`，零 I/O）、从权威日志进行的无缓冲按需回放、固定的每个（轮次、步骤）组合首分片投影、实时 `agent/error` 转发，以及实时 dispose（资源释放）时的 `shutdown` 记录。 +- **`@deepseek-ai/dsh-session-telemetry`** —— seam 本体。`SessionTelemetrySink`（`emit`/`flush?`/`shutdown`）、服务注册形态的 `SessionTelemetryBackend` 与 `SessionTelemetryCoordinator` 共同拥有完整捕获：每个新的 Session 对象会从 seq 0 回放全部权威事件，随后逐 append firehose 以零 I/O 深拷贝、脱敏并交接每个事件；重新收养同一对象时从模块作用域游标之后继续。无缓冲按需捕获使用同样的一事件一记录映射，直到可选的包含式边界。Ledger 身份包含 `session.id`、`session.format_version` 与 `event.seq`；实时捕获还会转发 `agent/error`，并创建 dispose（资源释放）时的 `shutdown` 记录。 - **`session-telemetry/record` waterfall（瀑布式事件）** —— 相对分支版本的增量，也是该 seam 的脱敏扩展点。每条记录抵达任何后端前必经此处；seam 自身不带任何规则——最内层 `next()` 原样透传，部署方以监听器挂载自己的规则（通过变换 `next()` 的返回值堆叠），抛异常的规则将该记录 fail-closed 扣下。脱敏只作用于导出副本；canonical log 永不改写。 - **`@deepseek-ai/dsh-session-telemetry-otel`** —— 参考后端：OTel JS SDK 日志流水线（`LoggerProvider` → `BatchLogRecordProcessor` → OTLP/HTTP exporter），经 `exporter`/`processor` passthrough 原样配置。`DISABLED` 是默认值，且不构造任何传输；[反馈门控遥测决策](2026-08-05-feedback-gated-session-telemetry.zh.md)定义了需显式启用的 `FULL` 与 `FEEDBACK_ONLY` 投递模式，这两种模式要求 `exporter.url`，且不移动脱敏或后端边界。[无缓冲反馈回放](../simplification/2026-08-06-buffer-free-feedback-telemetry.zh.md)避免在内存中创建会话前缀的第二份副本。 -边界公理保持不变：harness 的职责止于 `emit()`。批处理、重试、排队与丢失策略属于 reporting SDK，经 passthrough 配置——投递是尽力而为（崩溃时至多一次），两份 README 对此如实陈述。 +边界公理保持不变：harness 的职责止于 `emit()`。批处理、重试、排队与丢失策略属于 reporting SDK，并经 passthrough 配置。投递是尽力而为：崩溃可能丢失已排队记录，而重新构造 Session 也可能回放崩溃前已经送达的记录。因此，接收端要基于 `(session.id, session.format_version, event.seq)` 对 ledger 行去重。 ## 考虑过的替代方案 @@ -29,10 +29,10 @@ Status: implemented **映射到 OTel span（GenAI 语义约定）而非日志。** 本次复活否决：分支实现的日志映射已经过评审、形态可交付；span 模型对可 fork、可中断的会话有损，留给将来真正有 span 查询需求的消费方。 -**handoff 游标未存活时全量回放日志（重新导出构造函数种子）。** 首轮复活曾交付此方案，其后收窄：接管操作现在从会话的构造边界起回放（`Session.firstLiveSeq`，即构造函数种子长度；`Session.inheritedEventCount` 不能胜任，因为它是持久保存的 fork 谱系 cut，而恢复会话的构造函数种子是其完整的已存储日志）。恢复会话的历史已由上一个进程以同一 id 发出，fork 继承的前缀也已在父会话的流中发出；再次导出任何一者，都会让每次恢复为其完整历史重复付费，并在没有原生摄取去重的 OTLP 后端上使查询时的计数翻倍。接收端基于 `session.parent_id` + `session.seed_length` 拼接 fork 谱系。此次收窄放弃的内容与至多一次立场一致：恢复不再回填上一个进程未能投递的记录（彼时遥测未挂载，或崩溃时仍在队列中）——这本是全量回放唯一的真实收益，代价却由常见情形承担。提出回填要求的部署需要的是上文已推迟的 outbox，而不是回放。该边界同样吞掉 `SessionPersistence.load()` 修复被崩溃打断的日志时写入的合成轮次关闭事件（它们落在 `firstLiveSeq` 之前，尽管在上一个进程中从未存在过）。这是有意为之，而非附带效果：远端轮次的真实尾部记录已随崩溃进程的队列一同消亡，导出合成关闭事件无法补全该轮次，只会让一个未完成的轮次看起来已经关闭。导出的流忠实于崩溃进程实际发出的内容；接收端会把恢复后的流中一个从未关闭的轮次读作「上一个进程死在了该轮次之内」（OTel README 陈述了这条规则），其后干净的 `shutdown` 标记也只证明恢复后进程自身的退出。若为让修复以实时事件的身份导出而将修复前边界贯穿 load/prepare 传递，将使三个包相互耦合，只为抹除这一信号。 +**让新的 Session 对象从构造边界开始，并跳过 seed 前缀。** 不予采用，因为这种做法假设另一个进程或父身份已经成功投递那些记录。如果遥测在原始运行之后才挂载、SDK 队列随崩溃消失，或格式迁移与崩溃修复在遥测看到恢复对象前生成当前权威 seed，该假设都会丢失历史。每个新对象改为从没有 handoff 游标的状态开始，并从 seq 0 回放，包括 fork 继承前缀以及恢复或迁移后的日志。重新收养同一对象时仍从游标之后继续，因此 HMR 不会重复稳定前缀。全量回放可能重复此前已投递的行，也会有意在 child Session id 下重复继承事件；接收端基于 `(session.id, session.format_version, event.seq)` 去重，`session.parent_id` 与 `session.seed_length` 则保留谱系。这种回放是机会式恢复，而非 at-least-once 保证：如果之后没有构造新的 Session 对象，记录仍可能丢失，因此要求保证回填的部署需要上文推迟的 outbox。 **将 seam 的轮次边界 `flush()` 提示转发到 OTel 提供方的 `forceFlush()`。** 首轮复活曾交付此转发，其后移除：三条不同的静默丢失路径共用同一份包装层状态——dispose 与进行中的 flush 之间的竞态（SDK 的并发 flush 防护会令 shutdown 的内部排空被跳过）、相互重叠的提示顶掉留存的 promise、以及提供方固定的 30 秒 flush 超时在批处理器仍在排空时便 reject。这些路径存在的唯一原因，是该转发让这个后端成为进程内第二个执行 flush 的组件，面对的还是上游实验性（experimental）源码树中未见诸文档的 SDK 内部行为；不实现 `flush()` 时，批处理器就是唯一执行 flush 的组件，其 `scheduledDelayMillis`（已可由部署方经 `processor` passthrough 调优）决定导出节奏，`shutdown()` 的排空从构造上就是完整的。仅当某个部署提出 `scheduledDelayMillis` 无法满足的轮次边界延迟要求时才恢复此转发——且届时应调用留存的 `BatchLogRecordProcessor` 自身的 `forceFlush()`，绝不调用提供方那个带超时包装的版本。 ## 后果 -部署方在 `cordis.yml` 加一个带 OTLP endpoint 的 Cordis 配置项，并显式选择 `FULL`，即可把会话流接入任何 OTel 兼容体系；选择 `FEEDBACK_ONLY` 则会在记录反馈时回放权威日志前缀。`DISABLED` 是[默认值](2026-08-10-telemetry-default-off.zh.md)，且不构造上报流水线；删除该配置项仍是静默退出方式，而禁用模式会保留本地反馈警告。未挂载规则的部署导出的记录与捕获时完全一致，包括文件内容与命令输出中内嵌的任何凭据。因此，跨信任边界的部署必须挂载 `session-telemetry/record` 监听器，两个 README 对此如实陈述。挂载规则后，导出的 body 可能与 canonical log 字节不同，接收端不得把遥测当作字节精确副本；日志仍是真源。崩溃持久性在上述 outbox 决定重新审议前明确不在范围内。 +部署方在 `cordis.yml` 加一个带 OTLP endpoint 的 Cordis 配置项，并显式选择 `FULL`，即可把完整权威事件流接入任何 OTel 兼容体系；选择 `FEEDBACK_ONLY` 则会在记录反馈时回放完整的权威日志前缀。`DISABLED` 是[默认值](2026-08-10-telemetry-default-off.zh.md)，且不构造上报流水线；删除该配置项仍是静默退出方式，而禁用模式会保留本地反馈警告。未挂载规则的部署会按捕获原样导出每个事件 body，包括每条 assistant chunk，以及文件内容与命令输出中内嵌的任何凭据。因此，跨信任边界的部署必须挂载 `session-telemetry/record` 监听器，两份 README 对此如实陈述。挂载规则后，导出的 body 可能与 canonical log 字节不同，接收端不得把遥测当作字节精确副本；日志仍是真源。新对象回放可能重复 ledger 行，崩溃持久性则在上述 outbox 决定重新审议前继续不在范围内。 diff --git a/.agents/notes/implemented/feature/2026-07-25-subagent-policy-inheritance.i18n.yaml b/.agents/notes/implemented/feature/2026-07-25-subagent-policy-inheritance.i18n.yaml index ac6cdbcd01..d86262a56b 100644 --- a/.agents/notes/implemented/feature/2026-07-25-subagent-policy-inheritance.i18n.yaml +++ b/.agents/notes/implemented/feature/2026-07-25-subagent-policy-inheritance.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-25-subagent-policy-inheritance.md -2026-07-25-subagent-policy-inheritance.md: 6df9ad53f8588018fca53ebae3a9dfba0d4382de -2026-07-25-subagent-policy-inheritance.zh.md: f3f7328d558ea55379cb95e53bb010f9f2d2c792 +2026-07-25-subagent-policy-inheritance.md: 3fa8df405d9705a2bf145f1182e18fc0f7586b25 +2026-07-25-subagent-policy-inheritance.zh.md: 94c19d22db050441a6ce48fb507ccc428dbbe91e diff --git a/.agents/notes/implemented/feature/2026-07-25-subagent-policy-inheritance.md b/.agents/notes/implemented/feature/2026-07-25-subagent-policy-inheritance.md index 6df9ad53f8..3fa8df405d 100644 --- a/.agents/notes/implemented/feature/2026-07-25-subagent-policy-inheritance.md +++ b/.agents/notes/implemented/feature/2026-07-25-subagent-policy-inheritance.md @@ -12,7 +12,7 @@ Sandbox and approval overrides are per-session log folds. An in-process subagent The delegation boundary snapshots `sandboxPolicy.overrideOf(parent.session)` before its first await, through the shared child-agent helpers (`captureDelegatedPolicyOverrides`/`appendDelegatedPolicyOverrides` in `dsh-subagent`), which the one-shot driver and the [continuable start](2026-08-10-continuable-subagent-policy-inheritance.md) both call. A later parent switch belongs to the parent's future; cancel-and-redelegate takes a new snapshot. The sandbox-policy service is optional, and only the explicit session override is copied, never deployment defaults or one-shot grants. The approval policy is not inherited: the same capture pins every child to `'never'` — the [approvals-pinned decision](2026-08-10-subagent-approval-pinned-never.md) supersedes this note's original approval-override inheritance. -Each captured value becomes a source-tagged `sandbox/mode` or `approval/policy` event appended during the child factory's unpublished setup. The session constructor has already fixed `Session.firstLiveSeq` after the constructor seed, while `Session.inheritedEventCount` keeps the exact fork-prefix length, so the inherited facts follow fork history and reach telemetry when the child is announced without changing its lineage cut. Existing last-event-wins folds therefore make the delegation snapshot beat stale fork history and let a later child switch beat the snapshot. A grandchild folds its parent's logged state, so the rule composes without another inheritance mechanism. +Each captured value becomes a source-tagged `sandbox/mode` or `approval/policy` event appended during the child factory's unpublished setup. The session constructor has already fixed `Session.firstLiveSeq` after the constructor seed, while `Session.inheritedEventCount` keeps the exact fork-prefix length, so the inherited facts follow fork history without changing its lineage cut. Complete telemetry capture includes the entire announced child log, including both its constructor seed and these unpublished-setup events. Existing last-event-wins folds therefore make the delegation snapshot beat stale fork history and let a later child switch beat the snapshot. A grandchild folds its parent's logged state, so the rule composes without another inheritance mechanism. Ordinary session appends validate the inherited events before publication, and persistence captures the complete unpublished log when the session is announced. Any materialized child log therefore stores the inherited events with its first batch; there is no second policy store, schema field, or query index. The `source: 'delegation'` marker lets approval narration distinguish inheritance from a child-side user switch. @@ -23,7 +23,7 @@ A confined child gets the ordinary denial marker, and an escalation request is r ## Alternatives considered - **Generic `SessionHeader` policy fields** — rejected: they duplicate an event-sourced fact in metadata and require propagation through core session types, persistence backends, query indexes, collision identity, and every policy consumer. Unpublished setup events have the required ordering and reuse the existing durable store. -- **Combining new policy facts with constructor history** — rejected: `Session.firstLiveSeq` classifies the complete constructor seed as replayed history, so telemetry would skip child-only facts. Unpublished setup keeps history and new facts on their existing sides of that boundary without another session option. +- **Combining new policy facts with constructor history** — rejected because it would classify child-owned delegation policy as inherited history and blur the lifecycle ordering that makes the child snapshot override a stale fork value. Unpublished setup keeps history and new facts on their existing sides of the construction boundary without another session option; telemetry captures both sides. - **A first-prompt listener** — rejected: it introduces listener ordering and a later timing boundary even though the creation transaction already permits log appends before publication. - **Copying deployment defaults** — rejected: defaults remain operator-owned and may change; an unswitched parent stamps nothing, so its child follows the current deployment. - **Live resolution walking `parentSession` at each call** — rejected: it breaks the "two sessions never see each other's state" isolation invariant, requires the parent session to stay loaded for the child's lifetime, and makes a mid-run parent switch retroactively change a running child. Snapshot-at-delegation is the semantic: the child keeps the policy it was handed; cancel-and-respawn picks up a tightening. diff --git a/.agents/notes/implemented/feature/2026-07-25-subagent-policy-inheritance.zh.md b/.agents/notes/implemented/feature/2026-07-25-subagent-policy-inheritance.zh.md index f3f7328d55..94c19d22db 100644 --- a/.agents/notes/implemented/feature/2026-07-25-subagent-policy-inheritance.zh.md +++ b/.agents/notes/implemented/feature/2026-07-25-subagent-policy-inheritance.zh.md @@ -12,7 +12,7 @@ Status: implemented 委派边界在第一次 await 之前，经由共享的子 agent 辅助函数（`dsh-subagent` 中的 `captureDelegatedPolicyOverrides`／`appendDelegatedPolicyOverrides`）对 `sandboxPolicy.overrideOf(parent.session)` 获取快照；一次性驱动器与[可继续启动](2026-08-10-continuable-subagent-policy-inheritance.zh.md)都会调用这些辅助函数。父级后续的切换属于父级的未来；取消后重新委派会取得新快照。沙箱策略服务为可选，仅复制显式会话覆盖项，绝不复制部署默认值或一次性授权。审批策略不继承：同一次捕获会把每个子 agent 钉定为 `'never'`——[审批钉定决策](2026-08-10-subagent-approval-pinned-never.zh.md)取代了本 note 原先的审批覆盖项继承。 -每个捕获值都会成为子 agent 工厂在未发布设置阶段追加的一条带来源标记的 `sandbox/mode` 或 `approval/policy` 事件。会话构造函数已把 `Session.firstLiveSeq` 固定在 constructor seed 之后，而 `Session.inheritedEventCount` 保留精确的 fork 前缀长度，因此继承事实会排在 fork 历史之后，并在子 agent 公布时进入遥测，却不改变其谱系 cut。因此，既有的末事件胜出折叠会让委派快照压过陈旧的 fork 历史，并让子 agent 后续的切换压过该快照。孙代 agent 会折叠其父级已记录的状态，因此无需另一套继承机制即可组合此规则。 +每个捕获值都会成为子 agent 工厂在未发布设置阶段追加的一条带来源标记的 `sandbox/mode` 或 `approval/policy` 事件。会话构造函数已把 `Session.firstLiveSeq` 固定在 constructor seed 之后，而 `Session.inheritedEventCount` 保留精确的 fork 前缀长度，因此继承事实会排在 fork 历史之后，却不改变其谱系 cut。完整遥测捕获会包含已公布 child 的整份日志，其中既有 constructor seed，也有这些未发布设置事件。因此，既有的末事件胜出折叠会让委派快照压过陈旧的 fork 历史，并让子 agent 后续的切换压过该快照。孙代 agent 会折叠其父级已记录的状态，因此无需另一套继承机制即可组合此规则。 普通的会话追加会在发布前校验继承事件，持久化层则在会话公布时捕获完整的未发布日志。因此，任何已物化的子 agent 日志都会在首批数据中存下继承事件；不存在第二套策略存储、schema 字段或查询索引。`source: 'delegation'` 标记让审批叙述能够区分继承与子 agent 侧的用户切换。 @@ -23,7 +23,7 @@ Status: implemented ## 考虑过的替代方案 - **通用的 `SessionHeader` 策略字段**：不予采纳。它们会在元数据中复制一项事件溯源事实，并要求贯穿核心会话类型、持久化后端、查询索引、碰撞标识与每个策略消费方进行传播。未发布设置阶段的事件具备所需顺序，并复用现有持久化存储。 -- **将新策略事实与构造历史合并**：不予采纳。`Session.firstLiveSeq` 会把完整的构造种子归类为回放历史，因此遥测会跳过仅属于子 agent 的事实。未发布设置让历史与新事实留在该边界各自原有的一侧，无需再增加会话选项。 +- **将新策略事实与构造历史合并**：不予采纳，因为这会把 child 拥有的委派策略归类为继承历史，并模糊 child 快照压过陈旧 fork 值所依赖的生命周期顺序。未发布设置让历史与新事实留在构造边界各自原有的一侧，无需再增加会话选项；遥测会捕获两侧。 - **首个提示词监听器**：不予采纳。尽管创建事务已经允许在发布前追加日志，它仍会引入监听器顺序与更晚的时序边界。 - **复制部署默认值**：不予采纳。默认值仍由运维人员拥有且可能变化；未切换的父级不会记录任何值，因此其子 agent 跟随当前部署。 - **每次调用时沿 `parentSession` 实时解析**：不予采纳。这会打破「两个会话永远看不到彼此状态」的隔离不变量，要求父会话在子 agent 的整个生命周期内保持加载，还会让父级在子 agent 运行途中做的切换追溯性地改变一个正在运行的子 agent。委派时快照才是本设计的语义：子 agent 保持它被交付时的策略；取消后重新 spawn 即可拿到收紧后的策略。 diff --git a/.agents/notes/implemented/feature/2026-07-26-ptc-dispatch-ui-foundation.i18n.yaml b/.agents/notes/implemented/feature/2026-07-26-ptc-dispatch-ui-foundation.i18n.yaml index 4330c215e1..9775a62d15 100644 --- a/.agents/notes/implemented/feature/2026-07-26-ptc-dispatch-ui-foundation.i18n.yaml +++ b/.agents/notes/implemented/feature/2026-07-26-ptc-dispatch-ui-foundation.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-26-ptc-dispatch-ui-foundation.md -2026-07-26-ptc-dispatch-ui-foundation.md: 793735a87b734429bfdcba91b0512ce264a8aa1f -2026-07-26-ptc-dispatch-ui-foundation.zh.md: 68cd9c538d71db9f14e2bb4f8eae15049f0c9ad4 +2026-07-26-ptc-dispatch-ui-foundation.md: e44b67294a0c2c13809323b2aa42ec867bbcbbff +2026-07-26-ptc-dispatch-ui-foundation.zh.md: 4b1ae0b3ebf35982449f5941d84361c530766534 diff --git a/.agents/notes/implemented/feature/2026-07-26-ptc-dispatch-ui-foundation.md b/.agents/notes/implemented/feature/2026-07-26-ptc-dispatch-ui-foundation.md index 793735a87b..e44b67294a 100644 --- a/.agents/notes/implemented/feature/2026-07-26-ptc-dispatch-ui-foundation.md +++ b/.agents/notes/implemented/feature/2026-07-26-ptc-dispatch-ui-foundation.md @@ -28,4 +28,4 @@ Three changes, one per obstacle: ## Consequences -Session format keeps `SESSION_FORMAT_VERSION` 0 (pre-release churn does not bump; old logs with `resultSummary` simply carry an extra unread field and lack `content` — v0 makes no compatibility promise). Existing ptc snapshot fixtures were re-recorded. Model-visible surface grew: the `run_code` schema (one required parameter) and every ptc system prompt/tool-schema snapshot changed. The web UI work builds directly on the new event payload; live per-sub-call running state reshaped this event into a dispatch start/end pair ([live parallel dispatch](2026-07-26-ptc-live-parallel-dispatch.md)). +This UI change did not itself require a structural Session-format increment. The released v0 edge now freezes the accepted historical dispatch payloads and preserves them into v1; malformed mixtures refuse before publication. Existing ptc snapshot fixtures were re-recorded. Model-visible surface grew: the `run_code` schema (one required parameter) and every ptc system prompt/tool-schema snapshot changed. The web UI work builds directly on the new event payload; live per-sub-call running state reshaped this event into a dispatch start/end pair ([live parallel dispatch](2026-07-26-ptc-live-parallel-dispatch.md)). diff --git a/.agents/notes/implemented/feature/2026-07-26-ptc-dispatch-ui-foundation.zh.md b/.agents/notes/implemented/feature/2026-07-26-ptc-dispatch-ui-foundation.zh.md index 68cd9c538d..4b1ae0b3eb 100644 --- a/.agents/notes/implemented/feature/2026-07-26-ptc-dispatch-ui-foundation.zh.md +++ b/.agents/notes/implemented/feature/2026-07-26-ptc-dispatch-ui-foundation.zh.md @@ -28,4 +28,4 @@ Status: implemented ## 后果 -会话格式保持 `SESSION_FORMAT_VERSION` 为 0（预发布阶段的变动不递增版本号；携带 `resultSummary` 的旧日志只是多出一个不被读取的字段并缺少 `content`；v0 不作任何兼容性承诺）。既有的 PTC mode 快照 fixture（测试前置数据）已重新录制。模型可见范围扩大了：`run_code` 的 schema（新增一个必填参数）以及每一份 PTC mode 系统提示词／工具 schema 快照都发生了变化。Web UI 工作直接构建在新的事件载荷之上；每个子调用的实时运行状态已把本事件重塑为一对分发 start/end 事件（[实时并行分发](2026-07-26-ptc-live-parallel-dispatch.zh.md)）。 +该 UI 变更本身不要求结构性 Session 格式递增。已发布 v0 边现在冻结可接受的历史分发 payload 并将其保留到 v1；畸形混合结构会在发布前被拒绝。既有的 PTC mode 快照 fixture（测试前置数据）已重新录制。模型可见范围扩大了：`run_code` 的 schema（新增一个必填参数）以及每一份 PTC mode 系统提示词／工具 schema 快照都发生了变化。Web UI 工作直接构建在新的事件载荷之上；每个子调用的实时运行状态已把本事件重塑为一对分发 start/end 事件（[实时并行分发](2026-07-26-ptc-live-parallel-dispatch.zh.md)）。 diff --git a/.agents/notes/implemented/feature/2026-07-30-web-read-card.i18n.yaml b/.agents/notes/implemented/feature/2026-07-30-web-read-card.i18n.yaml index 89b5d305d7..9e12a4e70e 100644 --- a/.agents/notes/implemented/feature/2026-07-30-web-read-card.i18n.yaml +++ b/.agents/notes/implemented/feature/2026-07-30-web-read-card.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-30-web-read-card.md -2026-07-30-web-read-card.md: 73608391c6a8fff13f94423b07d141311fe06580 -2026-07-30-web-read-card.zh.md: 2914286c48520c3a13c5396bfddb5c10562d024f +2026-07-30-web-read-card.md: 2023b93f44b343ce8482371b1168c8486fea703e +2026-07-30-web-read-card.zh.md: 426b5cc1eb6050aeb87f8dcabd372ef6c2eb557e diff --git a/.agents/notes/implemented/feature/2026-07-30-web-read-card.md b/.agents/notes/implemented/feature/2026-07-30-web-read-card.md index 73608391c6..2023b93f44 100644 --- a/.agents/notes/implemented/feature/2026-07-30-web-read-card.md +++ b/.agents/notes/implemented/feature/2026-07-30-web-read-card.md @@ -16,7 +16,7 @@ Add a fourth `card` tag, `read`, to the [render-intent union](../architecture/20 The read tool projects the structured window through `output.presentationMeta`, the same persisted channel write/edit use for their applied-diff hunks ([canonical tool output contract](../architecture/2026-07-20-canonical-tool-output-contract.md)). `presentationMeta` runs once for a top-level surface call, returns `{ path, offset, lines, totalLines, lang? }` as JSON the session validates and stores on the result's `meta`, and `presentResult` narrows that meta back into the `ReadResultView` on both live and replay paths. `offset` (the 1-based first line the window requested) rides along because a byte cap below the first selected line yields an empty `lines` array with a positive `totalLines`; without the persisted `offset` a replayed card of such a window could not report where it starts or where a continuation resumes, and the last-line and re-parse fallbacks are both lossy. Without this channel the line array and total would be unreachable: the raw output object is not on the wire, and re-parsing the `N: text` text is lossy and fragile against the truncation footer. -`presentResult` returns `undefined` — the generic fallback — whenever the meta is absent or malformed (`readMetaFromMeta` narrows it defensively, so a replay of an older logged result never throws), whenever the result is an error, and whenever the single text block is not the read envelope. A pre-card logged result — a valid read envelope with no persisted `meta`, recorded before this card existed — takes that same `undefined` path deliberately: the client falls back to the raw `result.content`, so it shows the enveloped `//` text rather than the envelope-stripped generic card the old presenter returned. This is the accepted degradation under the [pre-release stance](../../../../AGENTS.md#pre-release-stance-foundation-over-blast-radius): reject the old on-disk format rather than add an envelope-stripping compatibility branch, since this change re-records every published fixture and the session format promises no backward compatibility. On the success path `presentResult` carries `content` (the envelope-stripped text) alongside the structured fields, so a UI without the read capability renders the file text through its generic/default card arm. The former TUI established the need for this fallback: its non-exhaustive result switch read `view.content`, while a separate dim-Markdown gate also had to admit `card: 'read'`. That frontend has since been removed, but the content fallback remains part of the view contract for any consumer without a structured read card. +`presentResult` returns `undefined` — the generic fallback — whenever the meta is absent or malformed (`readMetaFromMeta` narrows it defensively, so a replay of an older logged result never throws), whenever the result is an error, and whenever the single text block is not the read envelope. A pre-card logged result — a valid read envelope with no persisted `meta`, recorded before this card existed — takes that same `undefined` path deliberately: the client falls back to the raw `result.content`, so it shows the enveloped `//` text rather than the envelope-stripped generic card the old presenter returned. This remains the accepted degradation: the presenter keeps its generic fallback instead of adding an envelope-stripping compatibility branch. The released Session edge preserves old logged bytes, and this change re-recorded every published fixture. On the success path `presentResult` carries `content` (the envelope-stripped text) alongside the structured fields, so a UI without the read capability renders the file text through its generic/default card arm. The former TUI established the need for this fallback: its non-exhaustive result switch read `view.content`, while a separate dim-Markdown gate also had to admit `card: 'read'`. That frontend has since been removed, but the content fallback remains part of the view contract for any consumer without a structured read card. ### Language hint derivation diff --git a/.agents/notes/implemented/feature/2026-07-30-web-read-card.zh.md b/.agents/notes/implemented/feature/2026-07-30-web-read-card.zh.md index 2914286c48..426b5cc1eb 100644 --- a/.agents/notes/implemented/feature/2026-07-30-web-read-card.zh.md +++ b/.agents/notes/implemented/feature/2026-07-30-web-read-card.zh.md @@ -16,7 +16,7 @@ Status: implemented read 工具通过 `output.presentationMeta` 投影结构化窗口，这与 write/edit 用来投影其应用 diff hunk 的持久化通道相同（[规范化工具输出约定](../architecture/2026-07-20-canonical-tool-output-contract.zh.md)）。`presentationMeta` 对一次顶层 surface 调用运行一次，返回 `{ path, offset, lines, totalLines, lang? }` 作为会话校验并存储在结果 `meta` 上的 JSON，`presentResult` 在实时和回放路径上都把该 meta 收窄回 `ReadResultView`。`offset`（窗口请求的 1-based 起始行）一并携带，是因为当字节上限低于首个选中行时，窗口会返回空的 `lines` 数组而 `totalLines` 为正；没有持久化的 `offset`，这类窗口的回放 card 就无法报告它从哪行开始、或续读应从哪行继续，而末行推断与文本重解析两种兜底都有损。没有这个通道，行数组和总数就无法触及：原始输出对象不在线上，而重新解析 `N: text` 文本既有损又对截断尾注脆弱。 -`presentResult` 在以下情况返回 `undefined`——即 generic 回退：meta 缺失或畸形（`readMetaFromMeta` 防御性收窄它，因此回放旧的已记录结果永不抛错）、结果是错误、以及单个文本块不是 read 信封。本 card 出现之前记录的结果——信封合法但无持久化 `meta`——有意走同一条 `undefined` 路径：客户端回退到原始 `result.content`，因此显示带 `//` 信封的原文，而非旧展示器返回的剥信封 generic card。这是 [pre-release 立场](../../../../AGENTS.md#pre-release-stance-foundation-over-blast-radius)下接受的降级：拒绝旧的磁盘格式，而非加一个剥信封的兼容分支——本变更已重录全部已发布 fixture（测试前置数据），且会话格式不承诺向后兼容。在成功路径上，`presentResult` 在结构化字段之外携带 `content`（剥信封后的文本），因此不具备 read 能力的 UI 会通过自己的 generic/default card 分支渲染文件文本。原 TUI 证明了这条回退的必要性：它的非穷尽结果 switch 读取 `view.content`，而另一道 dim-Markdown 门控也必须接纳 `card: 'read'`。该前端随后被移除，但对任何没有结构化 read 卡片的消费方而言，content 回退仍是视图约定的一部分。 +`presentResult` 在以下情况返回 `undefined`——即 generic 回退：meta 缺失或畸形（`readMetaFromMeta` 防御性收窄它，因此回放旧的已记录结果永不抛错）、结果是错误、以及单个文本块不是 read 信封。本 card 出现之前记录的结果——信封合法但无持久化 `meta`——有意走同一条 `undefined` 路径：客户端回退到原始 `result.content`，因此显示带 `//` 信封的原文，而非旧展示器返回的剥信封 generic card。这仍是已接受的降级：展示器保留 generic 回退，而不添加剥信封兼容分支。已发布 Session 边会保留旧日志字节，本变更也重录了全部已发布 fixture（测试前置数据）。在成功路径上，`presentResult` 在结构化字段之外携带 `content`（剥信封后的文本），因此不具备 read 能力的 UI 会通过自己的 generic/default card 分支渲染文件文本。原 TUI 证明了这条回退的必要性：它的非穷尽结果 switch 读取 `view.content`，而另一道 dim-Markdown 门控也必须接纳 `card: 'read'`。该前端随后被移除，但对任何没有结构化 read 卡片的消费方而言，content 回退仍是视图约定的一部分。 ### 语言提示推导 diff --git a/.agents/notes/implemented/feature/2026-08-05-context-form-vocabulary.i18n.yaml b/.agents/notes/implemented/feature/2026-08-05-context-form-vocabulary.i18n.yaml index 062c3eff4c..b34b876c6d 100644 --- a/.agents/notes/implemented/feature/2026-08-05-context-form-vocabulary.i18n.yaml +++ b/.agents/notes/implemented/feature/2026-08-05-context-form-vocabulary.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-08-05-context-form-vocabulary.md -2026-08-05-context-form-vocabulary.md: be792256fa3375dd2621d647d6d3074b1a6d6b18 -2026-08-05-context-form-vocabulary.zh.md: f648dd1071128b879e8c36e9fba1646b6c70cda1 +2026-08-05-context-form-vocabulary.md: 12bb0132ad779d47b1090eb1d15d692032292524 +2026-08-05-context-form-vocabulary.zh.md: 899bef0892a9ec2d865d91e51869527afea40f9d diff --git a/.agents/notes/implemented/feature/2026-08-05-context-form-vocabulary.md b/.agents/notes/implemented/feature/2026-08-05-context-form-vocabulary.md index be792256fa..12bb0132ad 100644 --- a/.agents/notes/implemented/feature/2026-08-05-context-form-vocabulary.md +++ b/.agents/notes/implemented/feature/2026-08-05-context-form-vocabulary.md @@ -27,7 +27,7 @@ The vocabulary is semantic, never visual. A value states that the content is a f Entries record the published fact **unescaped**. The pseudo-XML escaping belongs to the `` frame, which exists for the model, so it is applied when rendering that frame and never stored; otherwise a consumer would have to know the frame's encoding to display a description containing `<`, and the same frame knowledge this decision removes would leak back in another shape. `escapeText` is deterministic and injective, so digesting the unescaped entries preserves republish semantics exactly, and the model-facing text stays byte-identical. -That move also relocates catalog **identity**: the republish digest now covers the durable entries rather than the rendered text, so the model-facing framing can no longer decide whether a republish is needed, and the text-slicing that recovered entries from a logged message is gone. A resumed session whose newest catalog predates this change republishes once, which the pre-release stance permits. One case does not self-heal: if that old-format catalog is the only one and the current view has no skills, the plugin sees no published catalog and emits no tombstone, so the model keeps a stale catalog nothing replaces. The pre-release stance ("backends reject old on-disk formats") permits it; it is recorded here rather than left to the optimistic path. +That move also relocates catalog **identity**: the republish digest now covers the durable entries rather than the rendered text, so the model-facing framing can no longer decide whether a republish is needed, and the text-slicing that recovered entries from a logged message is gone. The v0-to-v1 edge preserves an accepted older catalog payload, so a resumed session whose newest catalog predates this change republishes once. One case does not self-heal: if that old-form catalog is the only one and the current view has no skills, the plugin sees no published catalog and emits no tombstone, so the model keeps a stale catalog nothing replaces. This is the context feature's explicit degradation, not a Session-format refusal. **`snapshot`** — current state that a later snapshot from the same producer supersedes. The runtime-context snapshot, `time-context`, and `tmux-context` declare it. `renderContextSections()` exposes the assembly's named contributions, which `renderContextSnapshot()` already joined for the model, so the body attributes each part to the subsystem that produced it without re-splitting joined prose. The two single-contribution producers record one section each. The cleared runtime-context marker has no contributions left and declares no form. diff --git a/.agents/notes/implemented/feature/2026-08-05-context-form-vocabulary.zh.md b/.agents/notes/implemented/feature/2026-08-05-context-form-vocabulary.zh.md index f648dd1071..899bef0892 100644 --- a/.agents/notes/implemented/feature/2026-08-05-context-form-vocabulary.zh.md +++ b/.agents/notes/implemented/feature/2026-08-05-context-form-vocabulary.zh.md @@ -27,7 +27,7 @@ Status: implemented 条目记录的是**未转义**的发布事实。伪 XML 转义属于 `` 这层为模型而设的框架，因此只在渲染该框架时施加、从不存储；否则消费方要正确展示含 `<` 的描述就得知道框架的编码方式，本决策刚移除的框架知识会换一种形式泄漏回来。`escapeText` 确定且单射，故对未转义条目取 digest 与此前完全等价，重新发布语义不变，面向模型的文本逐字节不变。 -这次迁移同时挪动了目录的**身份**：重新发布用的 digest 现在覆盖持久条目而非渲染文本，于是面向模型的包装再也无法左右是否需要重新发布，那段从已记录消息里切出条目的文本切分逻辑也随之删除。若恢复的会话中最新目录早于本次改动，会重新发布一次——预发布立场允许这样做。有一种情形不会自愈：当那份旧格式目录是唯一的一份、且当前视图没有任何 skill 时，插件看不到已发布目录，也就不会发出 tombstone，模型手里会留着一份无人替换的陈旧目录。预发布立场（「后端拒绝旧的磁盘格式」）允许这一点；此处如实记录，而不是只写乐观路径。 +这次迁移同时挪动了目录的**身份**：重新发布用的 digest 现在覆盖持久条目而非渲染文本，于是面向模型的包装再也无法左右是否需要重新发布，那段从已记录消息里切出条目的文本切分逻辑也随之删除。v0-to-v1 迁移边会保留可接纳的旧目录 payload，因此恢复的 Session 若以本变更前的目录为最新目录，就会重新发布一次。有一种情形不会自愈：当那份旧结构目录是唯一的一份、且当前视图没有任何 skill 时，插件看不到已发布目录，也就不会发出 tombstone，模型手里会留着一份无人替换的陈旧目录。这是 context 功能的显式降级，不是 Session 格式拒绝。 **`snapshot`**——会被同一生产方后续快照取代的当前状态。运行时快照、`time-context`、`tmux-context` 声明它。`renderContextSections()` 暴露出装配时的具名贡献——`renderContextSnapshot()` 本来就是把它们拼给模型的——因此内容区能把每一段归属到产生它的子系统，而不必去切分已经拼好的散文。两个单贡献生产方各记录一段。运行时快照的「已清空」标记没有任何贡献可归属，因此不声明形态。 diff --git a/.agents/notes/implemented/feature/2026-08-05-feedback-gated-session-telemetry.i18n.yaml b/.agents/notes/implemented/feature/2026-08-05-feedback-gated-session-telemetry.i18n.yaml index 72e8ad9a15..ab67e951f7 100644 --- a/.agents/notes/implemented/feature/2026-08-05-feedback-gated-session-telemetry.i18n.yaml +++ b/.agents/notes/implemented/feature/2026-08-05-feedback-gated-session-telemetry.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-08-05-feedback-gated-session-telemetry.md -2026-08-05-feedback-gated-session-telemetry.md: 9dab135029bd4a5dd209157b48b6f7dca4040224 -2026-08-05-feedback-gated-session-telemetry.zh.md: 824aa1a04f3854163fdd16f4db7a7593540976e0 +2026-08-05-feedback-gated-session-telemetry.md: ecca36703e3f85f67a68348f06f35deeacc33c2a +2026-08-05-feedback-gated-session-telemetry.zh.md: 0f5f8c89223bdbce8fe5cbe838408b9634b45a68 diff --git a/.agents/notes/implemented/feature/2026-08-05-feedback-gated-session-telemetry.md b/.agents/notes/implemented/feature/2026-08-05-feedback-gated-session-telemetry.md index 9dab135029..ecca36703e 100644 --- a/.agents/notes/implemented/feature/2026-08-05-feedback-gated-session-telemetry.md +++ b/.agents/notes/implemented/feature/2026-08-05-feedback-gated-session-telemetry.md @@ -16,7 +16,7 @@ Session telemetry originally has one mounted behavior: every accepted record ent - `FEEDBACK_ONLY` reads the canonical session log when `feedback/record` is appended and hands over the unreleased prefix through that exact event. Records appended after that boundary remain local until another feedback event. - `DISABLED` is the [default](2026-08-10-telemetry-default-off.md), constructs no exporter, processor, or logger provider, and prints that nothing is shared and the feedback remains local when it observes `feedback/record`. -The generic telemetry coordinator owns `live` and `on-demand` capture. Live capture projects, clones, redacts, and hands each event to the backend on the session firehose. On-demand capture registers no continuous capture listeners; `captureSession(session, throughSeq)` reads the canonical log from the handoff cursor through an inclusive boundary, then projects, clones, redacts, and hands over that prefix. The cursor advances only for handed-over records. The [buffer-free replay decision](../simplification/2026-08-06-buffer-free-feedback-telemetry.md) owns why the on-demand path uses the canonical log instead of copied records. +The generic telemetry coordinator owns `live` and `on-demand` capture. Live capture deep-copies, redacts, and hands every canonical event to the backend on the session firehose. On-demand capture registers no continuous capture listeners; `captureSession(session, throughSeq)` reads the canonical log after the same-object handoff cursor through an inclusive boundary, then deep-copies, redacts, and hands over every event in that prefix. A new Session object starts with cursor `-1`, so its first capture includes the complete constructor seed whether the object is fresh, forked, resumed, or restored after migration; re-adopting or recapturing the same object starts after its highest handed-off seq. The [buffer-free replay decision](../simplification/2026-08-06-buffer-free-feedback-telemetry.md) owns why the on-demand path uses the canonical log instead of copied records. Mode resolution is a closed, fail-before-setup check: an unknown direct-construction value fails before transport configuration is read. Only `FULL` exposes the public service's `emit()` path to the SDK pipeline. `FEEDBACK_ONLY` gives its on-demand coordinator a private backend capability; its listener passes an event to `captureSession()` only when `session.eventAt(event.seq)` returns that exact `feedback/record` object. `Session.append` commits that object before publishing `session/event`, so replay includes the feedback but cannot extend past its boundary. `DISABLED` creates neither the capability nor the SDK pipeline and does not inspect exporter configuration. @@ -32,4 +32,4 @@ Mode resolution is a closed, fail-before-setup check: an unknown direct-construc ## Consequences -`FULL` retains the original source and wire behavior as an explicit opt-in. `FEEDBACK_ONLY` adds no telemetry-owned per-event buffer before feedback; direct service calls and non-canonical feedback events upload nothing, and a crash before feedback uploads nothing from that prefix. Replay applies the redaction policy mounted when feedback is recorded and excludes operational records that do not exist in the canonical log. Feedback-only streams therefore carry neither `agent-error` nor `shutdown` records, and shutdown absence is not a crash signal. Each later feedback captures the suffix accumulated since the previous boundary. `DISABLED` can omit `exporter.url`, does no reporting work, and keeps feedback only in the canonical session log. +`FULL` hands off every canonical event and replays the complete log of each new Session object as an explicit opt-in. `FEEDBACK_ONLY` adds no telemetry-owned per-event buffer before feedback; direct service calls and non-canonical feedback events upload nothing, and a crash before feedback uploads nothing unless a later Session object is restored and feedback releases its log. The first feedback on a new resumed or migrated object includes its complete current canonical prefix through that feedback; later feedback on the same object captures only the suffix after its cursor. Replay applies the redaction policy mounted when feedback is recorded and excludes operational records that do not exist in the canonical log, so feedback-only streams carry neither `agent-error` nor `shutdown` records and shutdown absence is not a crash signal. New-object replay and backend retries can duplicate ledger rows; receivers deduplicate on `(session.id, session.format_version, event.seq)`. `DISABLED` can omit `exporter.url`, does no reporting work, and keeps feedback only in the canonical session log. diff --git a/.agents/notes/implemented/feature/2026-08-05-feedback-gated-session-telemetry.zh.md b/.agents/notes/implemented/feature/2026-08-05-feedback-gated-session-telemetry.zh.md index 824aa1a04f..0f5f8c8922 100644 --- a/.agents/notes/implemented/feature/2026-08-05-feedback-gated-session-telemetry.zh.md +++ b/.agents/notes/implemented/feature/2026-08-05-feedback-gated-session-telemetry.zh.md @@ -16,7 +16,7 @@ Status: implemented - `FEEDBACK_ONLY` 在追加 `feedback/record` 时读取权威会话日志，并交接截至该事件的未释放前缀。该边界后追加的记录会留在本地，直到另一个反馈事件。 - `DISABLED` 是[默认值](2026-08-10-telemetry-default-off.zh.md)，不构造导出器、处理器或日志提供方，并在观察到 `feedback/record` 时输出警告，说明什么都不会共享，且反馈仍留在本地。 -通用遥测协调器拥有 `live` 与 `on-demand` 捕获。实时捕获在会话 firehose 上投影、深拷贝、脱敏每个事件，并将其交给后端。按需捕获不注册持续捕获监听器；`captureSession(session, throughSeq)` 从 handoff 游标起读取权威日志，直至含边界的指定序列号，然后投影、深拷贝、脱敏并交接该前缀。游标只为已交接记录推进。[无缓冲回放决策](../simplification/2026-08-06-buffer-free-feedback-telemetry.zh.md)说明了按需路径为何使用权威日志而非记录副本。 +通用遥测协调器拥有 `live` 与 `on-demand` 捕获。实时捕获在会话 firehose 上深拷贝、脱敏并向后端交接每个权威事件。按需捕获不注册持续捕获监听器；`captureSession(session, throughSeq)` 从同一对象 handoff 游标之后读取权威日志，直至包含式边界，然后深拷贝、脱敏并交接该前缀中的每个事件。新 Session 对象以游标 `-1` 开始，因此无论对象是全新、fork、resume 还是迁移后恢复，其首次捕获都会包含完整构造 seed；重新收养或再次捕获同一对象时，则从其已交接的最高 seq 之后开始。[无缓冲回放决策](../simplification/2026-08-06-buffer-free-feedback-telemetry.zh.md)说明了按需路径为何使用权威日志而非记录副本。 模式解析采用封闭式检查，并在设置前失败：通过直接构造传入未知值时，会在读取传输配置前失败。只有 `FULL` 向 SDK 流水线开放公共服务的 `emit()` 路径。`FEEDBACK_ONLY` 向其按需协调器提供私有后端能力；其监听器向 `captureSession()` 传递事件的唯一条件，是 `session.eventAt(event.seq)` 返回完全相同的 `feedback/record` 对象。`Session.append` 在发布 `session/event` 前已提交该对象，因此回放包含该反馈，但不会越过其边界。`DISABLED` 既不创建该能力，也不创建 SDK 流水线，并且不检查导出器配置。 @@ -32,4 +32,4 @@ Status: implemented ## 后果 -`FULL` 作为显式启用模式保留原有的源码与协议行为。`FEEDBACK_ONLY` 在反馈前不增加遥测自有的逐事件缓冲；直接服务调用与非权威反馈事件均不上传任何内容，且反馈前发生崩溃时，该前缀也不上传任何内容。回放使用记录反馈时挂载的脱敏策略，并排除权威日志中不存在的运维记录。因此，仅反馈的流既不携带 `agent-error` 记录，也不携带 `shutdown` 记录，而缺少 shutdown 不是崩溃信号。每个后续反馈都会捕获从上一个边界起累积的后缀。`DISABLED` 可省略 `exporter.url`，不执行任何上报工作，并仅在权威会话日志中保留反馈。 +`FULL` 作为显式启用模式交接每个权威事件，并回放每个新 Session 对象的完整日志。`FEEDBACK_ONLY` 在反馈前不增加遥测自有的逐事件缓冲；直接服务调用与非权威反馈事件均不上传任何内容，反馈前发生崩溃时也不上传该前缀，除非后续恢复出新的 Session 对象并由反馈释放其日志。新的 resume 或迁移对象上的首次反馈会包含截至该反馈的完整当前权威前缀；同一对象上的后续反馈只捕获游标之后的后缀。回放使用记录反馈时挂载的脱敏策略，并排除权威日志中不存在的运维记录，因此仅反馈的流既不携带 `agent-error` 也不携带 `shutdown`，且缺少 shutdown 不是崩溃信号。新对象回放与后端重试可能重复 ledger 行；接收端基于 `(session.id, session.format_version, event.seq)` 去重。`DISABLED` 可省略 `exporter.url`，不执行任何上报工作，并仅在权威会话日志中保留反馈。 diff --git a/.agents/notes/implemented/feature/2026-08-10-web-session-log-export.i18n.yaml b/.agents/notes/implemented/feature/2026-08-10-web-session-log-export.i18n.yaml index 3f5971500e..38ff1b42e2 100644 --- a/.agents/notes/implemented/feature/2026-08-10-web-session-log-export.i18n.yaml +++ b/.agents/notes/implemented/feature/2026-08-10-web-session-log-export.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-08-10-web-session-log-export.md -2026-08-10-web-session-log-export.md: df80ad4d264d835b2c11973ca61cf143576f11f3 -2026-08-10-web-session-log-export.zh.md: 2af86f371f9e7ed5255bb4e57a8a43427c745fe0 +2026-08-10-web-session-log-export.md: df54c0b042de14c94e78d22e066bc61a1768c76f +2026-08-10-web-session-log-export.zh.md: 4ed9d3839dece876a45a9af18ef3bc4609f8e3a9 diff --git a/.agents/notes/implemented/feature/2026-08-10-web-session-log-export.md b/.agents/notes/implemented/feature/2026-08-10-web-session-log-export.md index df80ad4d26..df54c0b042 100644 --- a/.agents/notes/implemented/feature/2026-08-10-web-session-log-export.md +++ b/.agents/notes/implemented/feature/2026-08-10-web-session-log-export.md @@ -10,7 +10,7 @@ The Trajectory view had no way to hand a debugging artifact to a human: the raw ## Decision -- **The export is a host-only download, not an RPC**: `GET /api/session.export?sessionId=…&includeDescendants=true` streams one ZIP attachment. Every file is a session's **stored artifact text verbatim**: `readRaw` on the persistence service reads the backend's own durable bytes (the JSONL backend decodes its physical zstd frames, or returns plaintext) — never a reconstruction from parsed events, so packed-chunk rows, key order, and line breaks survive byte-for-byte — under its original base name (`session.jsonl` at the root, `subagents//session.jsonl` for descendants). Compression runs on the host with fflate's streaming `Zip`/`ZipDeflate` API at validated `sessionExportCompressionLevel` 0–9 (default 6), letting deployments trade CPU and latency against archive size; each entry is deflated in bounded chunks as it is produced, so the response is chunked as it is generated and the host never holds the whole archive in one buffer (at most one descendant's artifact text beyond the preloaded root). At the 64 KiB response byte high-water mark, production waits for consumer pull to restore capacity; fflate's synchronous callback can add at most one bounded input push beyond that queue bound. No manifest is written — every file is byte-identical to the durable artifact and self-describing through its own header line. +- **The export is a host-only download, not an RPC**: `GET /api/session.export?sessionId=…&includeDescendants=true` streams one ZIP attachment. Every file is a Session's **selected generation text verbatim**: `readRaw` on the persistence service reads the backend's numerically highest canonical generation after any required migration (the JSONL backend decodes its physical zstd frames, or returns plaintext) — never a reconstruction from parsed events, so packed-chunk rows, key order, and line breaks survive byte-for-byte — under the backend-reported logical basename. The root entry is `session.jsonl` for v0 or `session.vN.jsonl` for a positive generation; descendants use `subagents//`. Compression runs on the host with fflate's streaming `Zip`/`ZipDeflate` API at validated `sessionExportCompressionLevel` 0–9 (default 6), letting deployments trade CPU and latency against archive size; each entry is deflated in bounded chunks as it is produced, so the response is chunked as it is generated and the host never holds the whole archive in one buffer (at most one descendant's artifact text beyond the preloaded root). At the 64 KiB response byte high-water mark, production waits for consumer pull to restore capacity; fflate's synchronous callback can add at most one bounded input push beyond that queue bound. No manifest is written — every file is byte-identical to the selected durable generation and self-describing through its own header line. - **Error vocabulary is HTTP-native**: missing services → 500, a backend without per-session raw artifacts → 501, missing root session → 404 (all decided before any byte streams), and a descendant without a stored artifact → the stream errors (fail-loud, never silent under-export). Request abort remains cancellation instead of being rewritten as 500; request and response-consumer cancellation converge on the producer signal, which reaches lineage, persistence, and attachment reads and terminates the active compressor. Connection applies the `/api` trust fence before dispatching the exact `GET`/`HEAD /api/session.export` route registered by `session-log-export`. - **The UI just downloads**: browser consumers may issue a bodyless `HEAD` preflight for preparation errors, then hand the GET endpoint to the browser's native download manager, so JavaScript never buffers the ZIP. The `session.log` RPC that an earlier iteration shipped was removed — the download endpoint is its only consumer, and the repo rule is no public interface without a current owner. The client bundle carries no archive implementation. - The current Header and `/export` consumers are defined by the [session-log export package contract](../../../../packages/session-query/session-log-export/README.md). diff --git a/.agents/notes/implemented/feature/2026-08-10-web-session-log-export.zh.md b/.agents/notes/implemented/feature/2026-08-10-web-session-log-export.zh.md index 2af86f371f..4ed9d3839d 100644 --- a/.agents/notes/implemented/feature/2026-08-10-web-session-log-export.zh.md +++ b/.agents/notes/implemented/feature/2026-08-10-web-session-log-export.zh.md @@ -10,7 +10,7 @@ Trajectory 视图没有任何方式把调试工件交到人手里：原始会话 ## 决策 -- **导出是宿主侧的下载面，不是 RPC**：`GET /api/session.export?sessionId=…&includeDescendants=true` 流式返回一个 ZIP 附件。每个文件都是会话**存储工件的逐字原文**：持久化服务新增的 `readRaw` 读取后端自己的持久化字节（jsonl 后端解码其物理 zstd 帧，或直接返回明文）——绝非从解析后事件重建，因此 chunk 打包、键序、换行全部逐字节保留——放在其原始基础文件名下（根为 `session.jsonl`，子代理为 `subagents//session.jsonl`）。压缩在宿主侧使用 fflate 流式 `Zip`/`ZipDeflate` API 和已验证的 `sessionExportCompressionLevel` 0–9（默认 6），使部署可以在 CPU／延迟与归档大小之间取舍；每个条目按有界分块边产出边压缩，响应随生成分块写出，宿主从不把整个归档放进单个缓冲区（除预载的根外，最多同时持有一条后代的工件文本）。到达 64 KiB 响应字节高水位后，生产会等待 Consumer pull 恢复容量；fflate 的同步回调最多只会在该队列界限外再增加一次有界输入 push。不写清单——每个文件都与持久化工件逐字节一致，并通过自身 header 行自描述。 +- **导出是宿主侧的下载面，不是 RPC**：`GET /api/session.export?sessionId=…&includeDescendants=true` 流式返回一个 ZIP 附件。每个文件都是 Session **选定 generation 的逐字原文**：持久化服务的 `readRaw` 会在完成所需迁移后读取后端数值最高的规范 generation（JSONL 后端解码其物理 zstd frame，或直接返回明文）——绝非从解析后事件重建，因此 chunk 打包、key 顺序与换行全部逐字节保留——并使用后端报告的逻辑 basename。根条目在 v0 下为 `session.jsonl`，正 generation 下为 `session.vN.jsonl`；后代使用 `subagents//`。压缩在宿主侧使用 fflate 流式 `Zip`/`ZipDeflate` API 和已验证的 `sessionExportCompressionLevel` 0–9（默认 6），使部署可以在 CPU／延迟与 archive 大小之间取舍；每个条目按有界分块边产出边压缩，响应随生成分块写出，宿主从不把整个 archive 放进单个 buffer（除预载的根外，最多同时持有一条后代的工件文本）。到达 64 KiB 响应字节高水位后，生产会等待 consumer pull 恢复容量；fflate 的同步回调最多只会在该队列界限外再增加一次有界输入 push。不写 manifest：每个文件都与选定持久 generation 逐字节一致，并通过自身 header 行自描述。 - **错误词汇是 HTTP 原生的**：服务缺失 → 500，后端不提供每会话原始工件 → 501，根会话缺失 → 404（三者都在任何字节流出前判定），后代缺少存储工件 → 流失败（fail-loud，绝不静默少导出）。请求中止会保持取消语义而不会改写成 500；请求取消与响应 consumer 取消汇合到生产者 signal，该 signal 会传到血缘、持久化与附件读取，并终止活跃压缩器。Connection 在分发 `session-log-export` 注册的精确 `GET`/`HEAD /api/session.export` 路由前应用 `/api` 信任围栏。 - **UI 只负责下载**：浏览器 Consumer 可以先发出不读取 body 的 `HEAD` 预检以取得准备阶段错误，再把 GET 端点交给浏览器原生下载管理器，因此 JavaScript 不会缓冲 ZIP。早先迭代发布的 `session.log` RPC 已删除——下载端点是它唯一的消费者，仓库规则是不留无当前所有者的公共接口。客户端 bundle 不包含任何归档实现。 - 当前 Header 与 `/export` Consumer 由 [Session 日志导出包约定](../../../../packages/session-query/session-log-export/README.zh.md)定义。 diff --git a/.agents/notes/implemented/feature/2026-08-25-feedback-gated-telemetry-default.i18n.yaml b/.agents/notes/implemented/feature/2026-08-25-feedback-gated-telemetry-default.i18n.yaml index 6221d9d868..454f05ad34 100644 --- a/.agents/notes/implemented/feature/2026-08-25-feedback-gated-telemetry-default.i18n.yaml +++ b/.agents/notes/implemented/feature/2026-08-25-feedback-gated-telemetry-default.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-08-25-feedback-gated-telemetry-default.md -2026-08-25-feedback-gated-telemetry-default.md: 772d134da53386292083790148dce738a98f2c0f -2026-08-25-feedback-gated-telemetry-default.zh.md: ea05d4d687bc2270f907983a71f592b880d7449c +2026-08-25-feedback-gated-telemetry-default.md: 35d79384094ff26a5d9169da9268c264e73a63d4 +2026-08-25-feedback-gated-telemetry-default.zh.md: 28268d6dda5fd734250080c203482b423a752c8e diff --git a/.agents/notes/implemented/feature/2026-08-25-feedback-gated-telemetry-default.md b/.agents/notes/implemented/feature/2026-08-25-feedback-gated-telemetry-default.md index 772d134da5..35d7938409 100644 --- a/.agents/notes/implemented/feature/2026-08-25-feedback-gated-telemetry-default.md +++ b/.agents/notes/implemented/feature/2026-08-25-feedback-gated-telemetry-default.md @@ -10,7 +10,7 @@ Diagnosing a `/feedback` report needs the session data the report describes. Wit ## Decision -The shared dsh base resolves an unset or empty `DSH_TELEMETRY_MODE` to `FEEDBACK_ONLY` instead of `DISABLED`. Nothing is uploaded before the user records `/feedback`; each recorded feedback uploads the not-yet-shared session-log records — from the last handoff through that exact event — to the configured OTLP endpoint, a resumed session shares only its current lifecycle, and the acknowledgement's sharing disclosure states that recording feedback uploads the records not yet shared. `FULL` and `DISABLED` remain explicit `DSH_TELEMETRY_MODE` overrides, any non-empty `DSH_TELEMETRY_DISABLED` remains the authoritative pre-load hard opt-out, and the plugin's own omitted-`mode` default stays `DISABLED`: the default changes only in the shared base's config expression, where deployments already override it. +The shared dsh base resolves an unset or empty `DSH_TELEMETRY_MODE` to `FEEDBACK_ONLY` instead of `DISABLED`. Nothing is uploaded before the user records `/feedback`. On a Session object already captured, each feedback uploads the suffix after the last handoff through that exact event; a newly constructed fresh, forked, resumed, or migrated object has no cursor, so its first feedback uploads the complete current canonical prefix from seq 0. The acknowledgement's sharing disclosure states that recording feedback releases the session prefix. `FULL` and `DISABLED` remain explicit `DSH_TELEMETRY_MODE` overrides, any non-empty `DSH_TELEMETRY_DISABLED` remains the authoritative pre-load hard opt-out, and the plugin's own omitted-`mode` default stays `DISABLED`: the default changes only in the shared base's config expression, where deployments already override it. This supersedes the session-backend default of the [default-off decision](2026-08-10-telemetry-default-off.md), accepting the user's explicit feedback action as the release authorization that note required a deployment setting for. That note's hard opt-out and its launcher-feed history remain current, and the [default-mount decision](2026-07-31-web-telemetry-default-mount.md) continues to own the endpoint, batching cadence, and exit-drain settings. diff --git a/.agents/notes/implemented/feature/2026-08-25-feedback-gated-telemetry-default.zh.md b/.agents/notes/implemented/feature/2026-08-25-feedback-gated-telemetry-default.zh.md index ea05d4d687..28268d6dda 100644 --- a/.agents/notes/implemented/feature/2026-08-25-feedback-gated-telemetry-default.zh.md +++ b/.agents/notes/implemented/feature/2026-08-25-feedback-gated-telemetry-default.zh.md @@ -10,7 +10,7 @@ Status: implemented ## 决定 -共享 dsh 基础配置把未设置或为空的 `DSH_TELEMETRY_MODE` 解析为 `FEEDBACK_ONLY` 而不是 `DISABLED`。用户记录 `/feedback` 之前不上传任何数据；每条已记录的反馈把尚未共享的会话日志记录——自上次交接至该事件为止——上传到已配置的 OTLP 端点，恢复的会话只共享当前生命周期，确认信息中的共享声明会说明记录反馈将上传尚未共享的记录。`FULL` 和 `DISABLED` 仍是显式的 `DSH_TELEMETRY_MODE` 覆盖值，任何非空的 `DSH_TELEMETRY_DISABLED` 仍是加载前的强制关闭开关，插件自身省略 `mode` 的默认值仍是 `DISABLED`：默认值只在共享基础配置的配置表达式中改变，部署本来就在那里覆盖它。 +共享 dsh 基础配置把未设置或为空的 `DSH_TELEMETRY_MODE` 解析为 `FEEDBACK_ONLY` 而不是 `DISABLED`。用户记录 `/feedback` 之前不上传任何数据。对于已经捕获的 Session 对象，每条反馈会上传从上次交接之后至该事件的后缀；新构造的全新、fork、resume 或迁移对象没有游标，因此其首次反馈会从 seq 0 上传完整当前权威前缀。确认信息中的共享声明会说明记录反馈将释放会话前缀。`FULL` 和 `DISABLED` 仍是显式的 `DSH_TELEMETRY_MODE` 覆盖值，任何非空的 `DSH_TELEMETRY_DISABLED` 仍是加载前的强制关闭开关，插件自身省略 `mode` 的默认值仍是 `DISABLED`：默认值只在共享基础配置的配置表达式中改变，部署本来就在那里覆盖它。 本决定取代[默认关闭决定](2026-08-10-telemetry-default-off.zh.md)中会话后端的默认值，把用户显式的反馈动作接受为该决定原本要求由部署设置提供的释放授权。该决定的强制关闭开关和 launcher 上报历史仍然有效，端点、批处理节奏和退出排空设置仍由[默认挂载决定](2026-07-31-web-telemetry-default-mount.zh.md)持有。 diff --git a/.agents/notes/implemented/simplification/2026-06-20-collapse-trace-only-session-events.i18n.yaml b/.agents/notes/implemented/simplification/2026-06-20-collapse-trace-only-session-events.i18n.yaml index 6858059c93..67ce1cb560 100644 --- a/.agents/notes/implemented/simplification/2026-06-20-collapse-trace-only-session-events.i18n.yaml +++ b/.agents/notes/implemented/simplification/2026-06-20-collapse-trace-only-session-events.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/simplification/2026-06-20-collapse-trace-only-session-events.md -2026-06-20-collapse-trace-only-session-events.md: a7a4ab3a2e4c008b59bc4cb1023e699c6fd8a76b -2026-06-20-collapse-trace-only-session-events.zh.md: 3b37f55795aa76fbdc5193e751f518c713483ccb +2026-06-20-collapse-trace-only-session-events.md: e7c40cbde6dd666542bb4022064a1be97b0e0ce8 +2026-06-20-collapse-trace-only-session-events.zh.md: b2c062fc8138a120da9467250b50772083adca75 diff --git a/.agents/notes/implemented/simplification/2026-06-20-collapse-trace-only-session-events.md b/.agents/notes/implemented/simplification/2026-06-20-collapse-trace-only-session-events.md index a7a4ab3a2e..e7c40cbde6 100644 --- a/.agents/notes/implemented/simplification/2026-06-20-collapse-trace-only-session-events.md +++ b/.agents/notes/implemented/simplification/2026-06-20-collapse-trace-only-session-events.md @@ -27,7 +27,7 @@ The user conversation log contains what is needed to render, resume, audit, and ## Verification -`SessionEventMap` carries no standalone `usage` or `error`; the loop appends no separate usage event and records durable failures through `turn/end { kind: 'error', step, message, code? }`; ACP snapshots and persistence tests assert no trace-only lines; recorded fixtures are on the new event shape with the session format version pinned at `0` (backends reject any non-`0` stored log per the pre-release format policy); and the docs state where token usage and operational errors are observed. +`SessionEventMap` carries no standalone `usage` or `error`; the loop appends no separate usage event and records durable failures through `turn/end { kind: 'error', step, message, code? }`; ACP snapshots and persistence tests assert no trace-only lines; the frozen v0 codec and identity migration preserve this released representation into current v1; and the docs state where token usage and operational errors are observed. ## Consequences @@ -35,6 +35,6 @@ A consumer can no longer filter the canonical log for standalone `usage` or step ## Implementation note -**Format version.** This changes persisted events, but the pre-release session format remains pinned at `0` and rejects any other version without migration. `dsh-session` owns the constant used by writers and load validation. Monotonic format versions begin at the first release. +**Format version.** This event simplification predates the released v0 baseline. `dsh-session` owns the current writer constant, while the static catalog and adjacent packages now own supported historical decoding and migration. The identity v0-to-v1 edge proves that lifecycle without changing this event representation. Usage is now observed on `assistant/message.usage`; an operational error's step on `turn/end.reason` for `kind: 'error'`. `agent/error` + logging are unchanged for live diagnostics. diff --git a/.agents/notes/implemented/simplification/2026-06-20-collapse-trace-only-session-events.zh.md b/.agents/notes/implemented/simplification/2026-06-20-collapse-trace-only-session-events.zh.md index 3b37f55795..b2c062fc81 100644 --- a/.agents/notes/implemented/simplification/2026-06-20-collapse-trace-only-session-events.zh.md +++ b/.agents/notes/implemented/simplification/2026-06-20-collapse-trace-only-session-events.zh.md @@ -27,7 +27,7 @@ Status: implemented ## 验证 -`SessionEventMap` 不再包含独立的 `usage` 或 `error`；agent loop（智能体循环）不再追加独立的 usage 事件，并通过 `turn/end { kind: 'error', step, message, code? }` 持久记录失败；ACP 快照和持久化测试断言不存在仅用于追踪的行；已录制的 fixture（测试前置数据）使用新事件形状，会话格式版本固定为 `0`（后端按预发布格式策略拒绝任何版本非 `0` 的已存储日志）；文档说明了 token 用量和运行错误的观测位置。 +`SessionEventMap` 不再包含独立的 `usage` 或 `error`；agent loop（智能体循环）不再追加独立的 usage 事件，并通过 `turn/end { kind: 'error', step, message, code? }` 持久记录失败；ACP 快照和持久化测试断言不存在仅用于追踪的行；冻结的 v0 codec 与恒等迁移会把该已发布表示保留到当前 v1；文档说明了 token 用量和运行错误的观测位置。 ## 后果 @@ -35,6 +35,6 @@ Status: implemented ## 实现说明 -**格式版本。** 此变更影响已持久化的事件，但预发布会话格式仍固定为 `0`，拒绝任何其他版本且不做迁移。`dsh-session` 拥有写入方和加载校验使用的常量。单调递增的格式版本从首次正式发布开始。 +**格式版本。** 该事件简化早于已发布 v0 基线。`dsh-session` 拥有当前写入方常量，静态目录与相邻包则拥有受支持历史版本的解码和迁移。恒等的 v0-to-v1 边在不改变该事件表示的情况下证明整套生命周期。 Usage 现在通过 `assistant/message.usage` 观测；运行错误的步骤编号通过 `turn/end.reason`（当 `kind: 'error'` 时）观测。`agent/error` 与日志用于实时诊断，保持不变。 diff --git a/.agents/notes/implemented/simplification/2026-07-12-simplify-session-log-representation.i18n.yaml b/.agents/notes/implemented/simplification/2026-07-12-simplify-session-log-representation.i18n.yaml index 2eb3e56677..bb00b05064 100644 --- a/.agents/notes/implemented/simplification/2026-07-12-simplify-session-log-representation.i18n.yaml +++ b/.agents/notes/implemented/simplification/2026-07-12-simplify-session-log-representation.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/simplification/2026-07-12-simplify-session-log-representation.md -2026-07-12-simplify-session-log-representation.md: 749d0da1774d74586988cbac39e32dd783b79af4 -2026-07-12-simplify-session-log-representation.zh.md: 5f320ab7d768aa682f54c3d42f67aef50c60ad7a +2026-07-12-simplify-session-log-representation.md: 9c8383d72b16ccd430a094515cbfd7227b222363 +2026-07-12-simplify-session-log-representation.zh.md: c77f97c3a5f3011cf76ee493ce97d2354a634428 diff --git a/.agents/notes/implemented/simplification/2026-07-12-simplify-session-log-representation.md b/.agents/notes/implemented/simplification/2026-07-12-simplify-session-log-representation.md index 749d0da177..9c8383d72b 100644 --- a/.agents/notes/implemented/simplification/2026-07-12-simplify-session-log-representation.md +++ b/.agents/notes/implemented/simplification/2026-07-12-simplify-session-log-representation.md @@ -20,7 +20,7 @@ The implementation retains append and replacement `sourceEventSeqs`, the `tool/c Request headers use canonical full snapshots only. Initial and resume anchors remain full snapshots even when unchanged; an in-instance change appends another full `request/header` with reason `change`; and an unchanged envelope beginning an explicitly declared message series or following a surface replacement appends a full snapshot with reason `series`. Ordinary append-only later Turns, further Steps, and retries in that model-message series inherit the latest snapshot. The delta event, codec types, diff/apply helpers, and codec-only `fallback` reason are removed. Request reconstruction selects the latest snapshot. -`SESSION_FORMAT_VERSION` remains pinned at `0`, so seed, append, and persistence-load validation explicitly reject old v0 `request/header-delta` events and full snapshots carrying the removed `fallback` reason. There is no compatibility fold or migration. JSONL tests pin this fail-loud boundary, and the ACP snapshot harness represents legitimate mid-session changes as full pinned headers and full readable prompts. +Current v1 seed and append validation accept only full pinned request headers and current reasons. The frozen v0-to-v1 edge owns the explicit refusal of historical `request/header-delta` and `fallback` forms before current Session construction; accepted historical normalizations are limited to the separately enumerated lossless shapes. The ACP snapshot harness represents legitimate mid-session changes as full pinned headers and full readable prompts. ## Alternatives considered @@ -32,4 +32,4 @@ Unit coverage pins ordered-surface append/replace behavior, tool pairing, compac ## Consequences -Full headers increase log volume, and linear replacement lookup could be slower on very large surfaces. Replacements were already linear because the prior implementation called `indexOf`; benchmarks are deferred until real traces show the simpler array is a bottleneck. The format version remains `0`, so explicit legacy-event rejection is a permanent part of the pre-release format boundary. In return, surface order and request-header state each have one representation, deleting link maintenance, maps, codec arms, round-trip fallback, and delta-aware snapshot normalization. +Full headers increase log volume, and linear replacement lookup could be slower on very large surfaces. Replacements were already linear because the prior implementation called `indexOf`; benchmarks are deferred until real traces show the simpler array is a bottleneck. Current v1 keeps exactly one representation, while the v0 edge is the only owner of the released legacy forms. In return, surface order and request-header state each have one current representation, deleting link maintenance, maps, codec arms, round-trip fallback, and delta-aware snapshot normalization. diff --git a/.agents/notes/implemented/simplification/2026-07-12-simplify-session-log-representation.zh.md b/.agents/notes/implemented/simplification/2026-07-12-simplify-session-log-representation.zh.md index 5f320ab7d7..c77f97c3a5 100644 --- a/.agents/notes/implemented/simplification/2026-07-12-simplify-session-log-representation.zh.md +++ b/.agents/notes/implemented/simplification/2026-07-12-simplify-session-log-representation.zh.md @@ -20,7 +20,7 @@ Status: implemented 请求头只使用规范的完整快照。初始与恢复锚点即使没有变化也仍是完整快照；实例内变化会追加另一个完整 `request/header`，reason 为 `change`；未变的信封显式开启消息序列或跟随 surface 替换时，会追加 reason 为 `series` 的完整快照。普通的仅追加后续 Turn、同一模型消息序列内的后续 Step 与重试沿用最新快照。delta 事件、codec 类型、diff/apply 辅助函数，以及仅供 codec 使用的 `fallback` reason 均已移除。请求重建选择最新快照。 -`SESSION_FORMAT_VERSION` 仍固定为 `0`，因此 seed、追加和持久化加载验证会显式拒绝旧 v0 `request/header-delta` 事件，以及携带已删除 `fallback` reason 的完整快照。不存在兼容性 fold 或迁移。JSONL 测试固定了这一失败即报错的边界；ACP（Agent Client Protocol）快照 harness 则把合法的会话中途变更表示为固定的完整请求头和完整可读提示词。 +当前 v1 的 seed 与追加校验只接受固定的完整请求头和当前 reason。冻结的 v0-to-v1 边会在构造当前 Session 前，独占负责对历史 `request/header-delta` 与 `fallback` 结构的显式拒绝；可接受的历史规范化仅限另行穷举的无损结构。ACP（Agent Client Protocol）快照 harness 会把合法的会话中途变更表示为固定的完整请求头和完整可读提示词。 ## 曾考虑的替代方案 @@ -32,4 +32,4 @@ Status: implemented ## 后果 -完整请求头会增加日志体积，线性替换查找在极大 surface 上也可能较慢。由于先前实现调用 `indexOf`，替换原本就是线性的；benchmark 推迟到真实 trace 表明更简单的数组成为瓶颈时再进行。格式版本仍为 `0`，因此显式拒绝旧事件是预发布格式边界的永久组成部分。作为交换，surface 顺序和请求头状态现在各自只有一种表示，删除了链接维护、map、codec 分支、往返 fallback 和针对 delta 的快照规范化。 +完整请求头会增加日志体积，线性替换查找在极大 surface 上也可能较慢。由于先前实现调用 `indexOf`，替换原本就是线性的；benchmark 推迟到真实 trace 表明更简单的数组成为瓶颈时再进行。当前 v1 只保留一种表示，v0 边则是已发布旧结构的唯一拥有者。作为交换，surface 顺序和请求头状态现在各自只有一种当前表示，删除了链接维护、map、codec 分支、往返 fallback 和针对 delta 的快照规范化。 diff --git a/.agents/notes/implemented/simplification/2026-07-22-plan-specific-collaboration-state.i18n.yaml b/.agents/notes/implemented/simplification/2026-07-22-plan-specific-collaboration-state.i18n.yaml index 0cdc45df60..f11b9827e8 100644 --- a/.agents/notes/implemented/simplification/2026-07-22-plan-specific-collaboration-state.i18n.yaml +++ b/.agents/notes/implemented/simplification/2026-07-22-plan-specific-collaboration-state.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/simplification/2026-07-22-plan-specific-collaboration-state.md -2026-07-22-plan-specific-collaboration-state.md: b477623872d8f5e1ac21836c9d43dee23a83edc6 -2026-07-22-plan-specific-collaboration-state.zh.md: 4af06f2212b6db6d4735670128299df4b9e8a3fb +2026-07-22-plan-specific-collaboration-state.md: 20648b8d6cee3e3c049aeee3022bd322d657ba1c +2026-07-22-plan-specific-collaboration-state.zh.md: 2142839d5a834abba7aac042c8b1fee7a56c56b1 diff --git a/.agents/notes/implemented/simplification/2026-07-22-plan-specific-collaboration-state.md b/.agents/notes/implemented/simplification/2026-07-22-plan-specific-collaboration-state.md index b477623872..20648b8d6c 100644 --- a/.agents/notes/implemented/simplification/2026-07-22-plan-specific-collaboration-state.md +++ b/.agents/notes/implemented/simplification/2026-07-22-plan-specific-collaboration-state.md @@ -66,6 +66,6 @@ The tool renders the submitted plan as a generic card titled by its first headin ## Consequences -The implementation has one vocabulary for one shipped feature. Adding another collaboration stance is an explicit design decision instead of a config entry, and automation clients do not acquire human mode controls through ACP. The migration intentionally rejects old `mode/set` logs and old `modes.plan.section` configuration under the repository's pre-release format policy. +The implementation has one vocabulary for one shipped feature. Adding another collaboration stance is an explicit design decision instead of a config entry, and automation clients do not acquire human mode controls through ACP. The frozen v0-to-v1 Session edge explicitly refuses old `mode/set` events; configuration parsing independently refuses the retired `modes.plan.section` form. Plan state remains reconstructable and tool schemas remain stable, but an idle pending selection is lost if the process exits before the next boundary. Entering or leaving plan mode changes the prompt from first-party order 500 onward, and a model that ignores the guidance can still mutate unless the deployment independently configures sandbox, approval, or filesystem policy. diff --git a/.agents/notes/implemented/simplification/2026-07-22-plan-specific-collaboration-state.zh.md b/.agents/notes/implemented/simplification/2026-07-22-plan-specific-collaboration-state.zh.md index 4af06f2212..2142839d5a 100644 --- a/.agents/notes/implemented/simplification/2026-07-22-plan-specific-collaboration-state.zh.md +++ b/.agents/notes/implemented/simplification/2026-07-22-plan-specific-collaboration-state.zh.md @@ -66,6 +66,6 @@ Plan mode 拥有一个 plan 专用产品包：位于 `packages/plan/plan-mode/` ## 后果 -该实现只用一套词汇描述一项已交付功能。若要添加另一种协作方式，必须显式作出设计决策，而不能只增加配置项；自动化客户端不会通过 ACP 获得面向人类的模式控制。根据仓库的预发布格式策略，本次迁移有意拒绝旧的 `mode/set` 日志与 `modes.plan.section` 配置。 +该实现只用一套词汇描述一项已交付功能。若要添加另一种协作方式，必须显式作出设计决策，而不能只增加配置项；自动化客户端不会通过 ACP 获得面向人类的模式控制。冻结的 v0-to-v1 Session 迁移边会显式拒绝旧 `mode/set` 事件；配置解析会独立拒绝已退役的 `modes.plan.section` 结构。 Plan 状态仍可重建，工具 schema 仍保持稳定，但如果进程在下一边界前退出，空闲状态下待生效的选择会丢失。进入或离开 plan mode 会改变 first-party 提示词顺序 500 处及其后的内容；如果模型忽略引导，仍可能执行修改，除非部署另行配置沙箱、审批或文件系统策略。 diff --git a/.agents/notes/implemented/simplification/2026-07-23-collapse-persistence-flush-state.i18n.yaml b/.agents/notes/implemented/simplification/2026-07-23-collapse-persistence-flush-state.i18n.yaml index 8dfb4281ed..3f3010a179 100644 --- a/.agents/notes/implemented/simplification/2026-07-23-collapse-persistence-flush-state.i18n.yaml +++ b/.agents/notes/implemented/simplification/2026-07-23-collapse-persistence-flush-state.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/simplification/2026-07-23-collapse-persistence-flush-state.md -2026-07-23-collapse-persistence-flush-state.md: dc26f760a9d74ed2a6f6ee13193701dd3357fea0 -2026-07-23-collapse-persistence-flush-state.zh.md: a26868fb0f3daaac20cc4585e30043d3177a1482 +2026-07-23-collapse-persistence-flush-state.md: d64be38a58f582a721ddf1dd22dc581107746afe +2026-07-23-collapse-persistence-flush-state.zh.md: 9114640f6d2eff1293cc50b1f253c6a16feab721 diff --git a/.agents/notes/implemented/simplification/2026-07-23-collapse-persistence-flush-state.md b/.agents/notes/implemented/simplification/2026-07-23-collapse-persistence-flush-state.md index dc26f760a9..d64be38a58 100644 --- a/.agents/notes/implemented/simplification/2026-07-23-collapse-persistence-flush-state.md +++ b/.agents/notes/implemented/simplification/2026-07-23-collapse-persistence-flush-state.md @@ -18,7 +18,7 @@ Each live `Session` has one lifecycle entry containing initialization and one pa Initialization now enters the existing per-id operation chain once and calls the unserialized core operations while it owns that turn. The chain remains separate from the live controller because detached public `create`/`append`/`load` calls can race without a `Session` object and still require identity-level serialization. -Crash repair is cold-only. For a live identity, `load(id)` snapshots the authoritative in-memory events before awaiting their flush, then returns them with `SessionState.meta`, the header actually used for durable writes; it rejects an open turn without reading or repairing storage. A cold load reserves its identity synchronously inside the per-id chain before awaiting stored-prefix reads or repair writes; the `session/created` publication boundary rejects and rolls back a same-id live session until the reservation clears. HMR adoption remains separate through `loadStored` plus the coordinator's cwd check and truncates torn storage without closing the authoritative live turn. +Crash repair is cold-only. For a live identity, `load(id)` snapshots the authoritative in-memory events before awaiting their flush, then returns them with `SessionState.meta`, the header actually used for durable writes; it rejects an open turn without reading or repairing storage. A cold load reserves its identity synchronously inside the per-id chain before awaiting current-prefix reads or repair writes; the `session/created` publication boundary rejects and rolls back a same-id live session until the reservation clears. HMR adoption reaches the current prefix through the same serialized backend path plus the coordinator's cwd check. It may publish a supported historical generation first, while adoption of current storage truncates a torn tail without closing the authoritative live turn. The live-controller map is also the retirement registry. Successful retirement drains and removes its controller; failed retirement leaves it in the map. Backend teardown stops event admission, flushes every controller still present, awaits remaining per-id operations, and closes the backend. No separate retirement set is needed to rediscover unfinished work. diff --git a/.agents/notes/implemented/simplification/2026-07-23-collapse-persistence-flush-state.zh.md b/.agents/notes/implemented/simplification/2026-07-23-collapse-persistence-flush-state.zh.md index a26868fb0f..9114640f6d 100644 --- a/.agents/notes/implemented/simplification/2026-07-23-collapse-persistence-flush-state.zh.md +++ b/.agents/notes/implemented/simplification/2026-07-23-collapse-persistence-flush-state.zh.md @@ -18,7 +18,7 @@ Status: implemented 初始化只进入现有的按 id 操作链一次，并在占有该轮执行权时调用未串行化的核心操作。该操作链与活跃控制器保持分离，因为公共 `create`、`append`、`load` 调用即使没有 `Session` 对象仍可能发生竞态，依然需要按标识串行执行。 -崩溃修复仅适用于冷态标识。对于活跃标识，`load(id)` 会在等待刷写完成前，先对内存中的权威事件生成快照，再将这些事件与 `SessionState.meta`（即持久化写入实际使用的标头）一同返回；若轮次仍打开，则在不读取或修复存储的情况下拒绝该次加载。冷态加载会先在按 id 操作链内同步占用对应标识，再等待读取已存储前缀或执行修复写入；在这项占用解除前，`session/created` 发布边界会拒绝同 id 活跃会话的发布并将其回滚。HMR（热模块替换）接管仍由 `loadStored` 与协调器的 cwd 检查独立处理，会截断撕裂的存储，但不会闭合权威的活跃轮次。 +崩溃修复仅适用于冷态标识。对于活跃标识，`load(id)` 会在等待刷写完成前，先对内存中的权威事件生成快照，再将这些事件与 `SessionState.meta`（即持久化写入实际使用的标头）一同返回；若轮次仍打开，则在不读取或修复存储的情况下拒绝该次加载。冷态加载会先在按 id 操作链内同步占用对应标识，再等待当前前缀读取或修复写入；在这项占用解除前，`session/created` 发布边界会拒绝同 id 活跃会话的发布并将其回滚。HMR（热模块替换）接管经由同一条串行化后端路径与协调器的 cwd 检查取得当前前缀。它可以先发布受支持的历史 generation；接管当前格式存储时则会截断撕裂尾部，但不会闭合权威的活跃轮次。 活跃控制器映射同时也是退役注册表。退役成功时，系统排空并移除其控制器；退役失败时，控制器保留在映射中。后端资源销毁会停止接纳事件，刷写所有仍存在的控制器，等待其余按 id 操作完成，然后关闭后端。无需另设退役集合来重新发现未完成的工作。 diff --git a/.agents/notes/implemented/simplification/2026-08-06-buffer-free-feedback-telemetry.i18n.yaml b/.agents/notes/implemented/simplification/2026-08-06-buffer-free-feedback-telemetry.i18n.yaml index 4e7f686f4f..c9f042554a 100644 --- a/.agents/notes/implemented/simplification/2026-08-06-buffer-free-feedback-telemetry.i18n.yaml +++ b/.agents/notes/implemented/simplification/2026-08-06-buffer-free-feedback-telemetry.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/simplification/2026-08-06-buffer-free-feedback-telemetry.md -2026-08-06-buffer-free-feedback-telemetry.md: a17ad586fbf94c1622a29dc158050a308147a206 -2026-08-06-buffer-free-feedback-telemetry.zh.md: 3e0a7985568f579bf671b99a38865c8827449e44 +2026-08-06-buffer-free-feedback-telemetry.md: 387342c8282072d07957054fbb7e879a9874d23b +2026-08-06-buffer-free-feedback-telemetry.zh.md: b0f5f4eadef35a6a3d59847d13029b98dae68f14 diff --git a/.agents/notes/implemented/simplification/2026-08-06-buffer-free-feedback-telemetry.md b/.agents/notes/implemented/simplification/2026-08-06-buffer-free-feedback-telemetry.md index a17ad586fb..387342c828 100644 --- a/.agents/notes/implemented/simplification/2026-08-06-buffer-free-feedback-telemetry.md +++ b/.agents/notes/implemented/simplification/2026-08-06-buffer-free-feedback-telemetry.md @@ -10,9 +10,9 @@ Feedback-only telemetry must upload the session-log prefix only after recorded f ## Decision -The telemetry coordinator provides `live` and `on-demand` capture. On-demand capture registers no session, flush, or operational-event listeners and retains no projected records. `captureSession(session, throughSeq?)` reads the canonical session log after the handoff cursor through an optional inclusive sequence boundary, applies the fixed projection, deep-copies each accepted event, runs the current `session-telemetry/record` waterfall, and hands the result to the backend. +The telemetry coordinator provides `live` and `on-demand` capture. On-demand capture registers no session, flush, or operational-event listeners and retains no copied records. `captureSession(session, throughSeq?)` reads the canonical session log after the same-object handoff cursor through an optional inclusive sequence boundary, deep-copies every event in order, runs the current `session-telemetry/record` waterfall, and hands one record per event to the backend, including every `assistant/chunk` with its complete body. A new Session object has no WeakMap entry, so its logical cursor is `-1` and capture starts at seq 0. -`FEEDBACK_ONLY` invokes that method with the `feedback/record` event's sequence. The append is already committed when `session/event` listeners run, so the replay contains the feedback event and cannot include a later suffix. The existing handoff cursor distinguishes later replays without another pending-record index. +`FEEDBACK_ONLY` invokes that method with the `feedback/record` event's sequence. The append is already committed when `session/event` listeners run, so the replay contains the feedback event and cannot include a later suffix. The object-keyed handoff cursor distinguishes later replays without another pending-record index: repeated feedback on the same object releases only a suffix, while the first feedback on a new resumed or migrated object releases its complete current canonical prefix. Because on-demand capture reads only the canonical log, it emits no `agent-error` or `shutdown` operational records. Redaction is evaluated at feedback time rather than append time. The [feedback mode decision](../feature/2026-08-05-feedback-gated-session-telemetry.md) owns the public sharing behavior; this note owns its buffer-free realization. @@ -26,4 +26,4 @@ Because on-demand capture reads only the canonical log, it emits no `agent-error ## Consequences -A no-feedback session consumes no telemetry-owned memory proportional to its event count; the canonical session log remains the only pre-feedback copy. Feedback handling performs projection, cloning, and redaction synchronously before the backend's non-blocking enqueue, so its cost scales with the unreleased prefix. A redaction-policy change before feedback affects that replay, and a crash before feedback uploads nothing. Later feedback processes only events beyond the handoff cursor. +A no-feedback session consumes no telemetry-owned memory proportional to its event count; the canonical session log remains the only pre-feedback copy. Feedback handling performs cloning and redaction synchronously before the backend's non-blocking enqueue, so its cost scales with the unreleased prefix and includes every chunk event. A redaction-policy change before feedback affects that replay, and a crash before feedback uploads nothing unless a later Session object is restored and feedback releases its log. The first capture of each new object starts at seq 0; later feedback on the same object processes only events beyond its handoff cursor. Receivers deduplicate new-object replay on `(session.id, session.format_version, event.seq)`. diff --git a/.agents/notes/implemented/simplification/2026-08-06-buffer-free-feedback-telemetry.zh.md b/.agents/notes/implemented/simplification/2026-08-06-buffer-free-feedback-telemetry.zh.md index 3e0a798556..b0f5f4eade 100644 --- a/.agents/notes/implemented/simplification/2026-08-06-buffer-free-feedback-telemetry.zh.md +++ b/.agents/notes/implemented/simplification/2026-08-06-buffer-free-feedback-telemetry.zh.md @@ -10,9 +10,9 @@ Status: implemented ## 决策 -遥测协调器提供 `live` 与 `on-demand` 捕获。按需捕获不注册会话、flush 或运维事件监听器，也不保留投影记录。`captureSession(session, throughSeq?)` 从 handoff 游标之后读取权威会话日志，直至可选的序列号边界（含边界），应用固定投影、深拷贝每个已接受事件、运行当前的 `session-telemetry/record` waterfall（瀑布式事件），并将结果交给后端。 +遥测协调器提供 `live` 与 `on-demand` 捕获。按需捕获不注册会话、flush 或运维事件监听器，也不保留记录副本。`captureSession(session, throughSeq?)` 从同一对象 handoff 游标之后读取权威会话日志，直至可选的包含式序列号边界，按顺序深拷贝每个事件、运行当前的 `session-telemetry/record` waterfall（瀑布式事件），并为每个事件向后端交接一条记录，其中包括每条 `assistant/chunk` 及其完整 body。新 Session 对象没有 WeakMap 条目，因此逻辑游标为 `-1`，捕获从 seq 0 开始。 -`FEEDBACK_ONLY` 以 `feedback/record` 事件的序列号调用该方法。`session/event` 监听器运行时，追加已经提交，因此回放包含该反馈事件，且无法包含后续后缀。现有 handoff 游标可区分后续回放，无需另一个待处理记录索引。 +`FEEDBACK_ONLY` 以 `feedback/record` 事件的序列号调用该方法。`session/event` 监听器运行时，追加已经提交，因此回放包含该反馈事件，且无法包含后续后缀。以对象为键的 handoff 游标可区分后续回放，无需另一个待处理记录索引：同一对象上的重复反馈只释放后缀，而新的 resume 或迁移对象上的首次反馈会释放其完整当前权威前缀。 按需捕获只读取权威日志，因此不会发出 `agent-error` 或 `shutdown` 运维记录。脱敏在反馈时而非追加时求值。[反馈模式决策](../feature/2026-08-05-feedback-gated-session-telemetry.zh.md)规定公开的共享行为；本记录规定其无缓冲实现。 @@ -26,4 +26,4 @@ Status: implemented ## 后果 -没有反馈的会话不会消耗随事件数量增长的遥测自有内存；权威会话日志仍是反馈前的唯一副本。反馈处理会在后端非阻塞入队前同步执行投影、深拷贝与脱敏，因此其开销随未释放前缀增长。反馈前的脱敏策略变更会影响该次回放，而反馈前发生崩溃时什么都不上传。后续反馈只处理 handoff 游标之后的事件。 +没有反馈的会话不会消耗随事件数量增长的遥测自有内存；权威会话日志仍是反馈前的唯一副本。反馈处理会在后端非阻塞入队前同步执行深拷贝与脱敏，因此其开销随未释放前缀增长，并包含每条 chunk 事件。反馈前的脱敏策略变更会影响该次回放；反馈前发生崩溃时不会上传任何内容，除非后续恢复出新的 Session 对象并由反馈释放其日志。每个新对象首次捕获时从 seq 0 开始；同一对象上的后续反馈只处理 handoff 游标之后的事件。接收端基于 `(session.id, session.format_version, event.seq)` 对新对象回放去重。 diff --git a/.agents/notes/implemented/simplification/2026-08-30-jsonl-only-session-persistence.i18n.yaml b/.agents/notes/implemented/simplification/2026-08-30-jsonl-only-session-persistence.i18n.yaml index e697040ad9..bd2498f9df 100644 --- a/.agents/notes/implemented/simplification/2026-08-30-jsonl-only-session-persistence.i18n.yaml +++ b/.agents/notes/implemented/simplification/2026-08-30-jsonl-only-session-persistence.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/simplification/2026-08-30-jsonl-only-session-persistence.md -2026-08-30-jsonl-only-session-persistence.md: f21fb89e3747dffd043d42ace2c05bbe521f3069 -2026-08-30-jsonl-only-session-persistence.zh.md: 4100e576ccdcd46443e12a22cfec6dd3d3495317 +2026-08-30-jsonl-only-session-persistence.md: e5bd18f05adeb4595c9ce155d0ac99047bea9818 +2026-08-30-jsonl-only-session-persistence.zh.md: 7ae27b0f4ff91024b5ab63b8fdc9dbff98b2a57d diff --git a/.agents/notes/implemented/simplification/2026-08-30-jsonl-only-session-persistence.md b/.agents/notes/implemented/simplification/2026-08-30-jsonl-only-session-persistence.md index f21fb89e37..e5bd18f05a 100644 --- a/.agents/notes/implemented/simplification/2026-08-30-jsonl-only-session-persistence.md +++ b/.agents/notes/implemented/simplification/2026-08-30-jsonl-only-session-persistence.md @@ -6,7 +6,7 @@ English | [中文](2026-08-30-jsonl-only-session-persistence.zh.md) ## Problem -The product ships and exercises JSONL as its authoritative Session store, while the optional SQLite Session-persistence provider duplicates the same logical service over a second physical format. Every Session contract, event-envelope change, recovery rule, package graph, platform lane, and format transition therefore carries a second implementation and test matrix even though shipped profiles do not select it. Released Session-format migration also needs an exact per-Session source artifact that can be archived before replacement; the single-database provider would require a separate publication design without serving a current deployment. +The product ships and exercises JSONL as its authoritative Session store, while the optional SQLite Session-persistence provider duplicates the same logical service over a second physical format. Every Session contract, event-envelope change, recovery rule, package graph, platform lane, and format transition therefore carries a second implementation and test matrix even though shipped profiles do not select it. Released Session-format migration also needs an exact per-Session source generation that remains untouched while a version-named successor is published; the single-database provider would require a separate immutable-generation transaction design without serving a current deployment. The SQLite full-text Session-query provider is not an alternative authoritative store. It observes persistence through `ctx.sessionPersistence` and maintains a separate disposable derived index. The generic SQLite domain-KV provider is also independent of Session logs. @@ -26,6 +26,6 @@ Existing databases written by the removed provider are not opened or migrated by ## Consequences -Session persistence has one first-party physical format and one first-party durability path. The migration stack can archive and atomically replace one per-Session JSONL artifact without implementing a parallel database transaction protocol. SQLite search remains available and its integration tests now prove that it observes JSONL rather than sharing an authoritative database. +Session persistence has one first-party physical format and one first-party durability path. The migration stack can keep one per-Session JSONL generation path, bytes, and inode unchanged while exclusively publishing a final successor, without implementing a parallel database transaction protocol. SQLite search remains available and its integration tests prove that it observes JSONL rather than sharing an authoritative database. Removing the provider is a deliberate compatibility cut for its opt-in database files. The change reduces implementation and CI surface but also removes the stronger database/WAL storage option; a future provider needs a current owner, deployment need, complete shared-contract evidence, and its own format-transition policy. diff --git a/.agents/notes/implemented/simplification/2026-08-30-jsonl-only-session-persistence.zh.md b/.agents/notes/implemented/simplification/2026-08-30-jsonl-only-session-persistence.zh.md index 4100e576cc..7ae27b0f4f 100644 --- a/.agents/notes/implemented/simplification/2026-08-30-jsonl-only-session-persistence.zh.md +++ b/.agents/notes/implemented/simplification/2026-08-30-jsonl-only-session-persistence.zh.md @@ -6,7 +6,7 @@ Status: implemented ## Problem -产品交付并实际使用 JSONL 作为权威 Session store，而可选的 SQLite Session persistence provider 用第二种物理格式重复实现同一逻辑服务。因此，每项 Session 约定、event envelope 变更、恢复规则、package graph、平台测试与格式迁移都要承担第二套实现和测试矩阵，即使交付 profile 并不选择它。已发布 Session 格式的迁移还需要一份可在替换前归档的精确逐 Session 源产物；单数据库 provider 需要另一套发布设计，却没有服务当前部署。 +产品交付并实际使用 JSONL 作为权威 Session store，而可选的 SQLite Session persistence provider 用第二种物理格式重复实现同一逻辑服务。因此，每项 Session 约定、event envelope 变更、恢复规则、package graph、平台测试与格式迁移都要承担第二套实现和测试矩阵，即使交付 profile 并不选择它。已发布 Session 格式迁移还需要保持精确逐 Session 源 generation 不变，同时发布具名版本后继；单数据库 provider 需要另一套不可变 generation transaction 设计，却没有服务当前部署。 SQLite 全文 Session-query provider 不是另一种权威 store。它通过 `ctx.sessionPersistence` 观察持久化，并维护独立、可丢弃的派生索引。通用 SQLite domain-KV provider 也与 Session 日志无关。 @@ -26,6 +26,6 @@ SQLite 全文 Session-query provider 不是另一种权威 store。它通过 `ct ## Consequences -Session persistence 只有一种 first-party 物理格式和一条 first-party durability path。迁移 stack 可以归档并原子替换逐 Session JSONL 产物，而无需实现并行的数据库 transaction protocol。SQLite search 保持可用，其 integration test 现在证明它观察 JSONL，而不是共享权威数据库。 +Session persistence 只有一种 first-party 物理格式和一条 first-party durability path。迁移 stack 可以保持逐 Session JSONL generation 的路径、字节与 inode 不变，同时排他发布最终后继，而无需实现并行的数据库 transaction protocol。SQLite search 保持可用，其 integration test 证明它观察 JSONL，而不是共享权威数据库。 删除 provider 是针对其可选数据库文件的明确 compatibility cut。该变更缩小实现与 CI surface，但也移除更强的 database/WAL 存储选项；未来 provider 需要当前 owner、部署需求、完整 shared-contract evidence，以及自身的 format-transition policy。 diff --git a/.agents/notes/implemented/testing/2026-06-19-acp-snapshot-tests.i18n.yaml b/.agents/notes/implemented/testing/2026-06-19-acp-snapshot-tests.i18n.yaml index f525829152..1e0e66f927 100644 --- a/.agents/notes/implemented/testing/2026-06-19-acp-snapshot-tests.i18n.yaml +++ b/.agents/notes/implemented/testing/2026-06-19-acp-snapshot-tests.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/testing/2026-06-19-acp-snapshot-tests.md -2026-06-19-acp-snapshot-tests.md: 900c6ece99196219c22fddb5e869334d4ca84422 -2026-06-19-acp-snapshot-tests.zh.md: 16ee8cb34752eaab66e30480061aa7e5aa0fd484 +2026-06-19-acp-snapshot-tests.md: c69f4a66aedb72d1fb8b83c3ec05ec5bc069ebf9 +2026-06-19-acp-snapshot-tests.zh.md: 0cb64a95af0391a34a1903bdcaa0111e2195c9d7 diff --git a/.agents/notes/implemented/testing/2026-06-19-acp-snapshot-tests.md b/.agents/notes/implemented/testing/2026-06-19-acp-snapshot-tests.md index 900c6ece99..c69f4a66ae 100644 --- a/.agents/notes/implemented/testing/2026-06-19-acp-snapshot-tests.md +++ b/.agents/notes/implemented/testing/2026-06-19-acp-snapshot-tests.md @@ -18,7 +18,7 @@ The [session-log snapshot corpus decision](2026-08-24-session-log-snapshot-corpu ### The fixture projects the persisted session JSONL -Each scenario's `session.jsonl` is harvested from a real run. `assistant/chunk` events reproduce the model streams; tool, message, and boundary events capture the harness behavior. One ordinary session artifact therefore serves as both replay source and behavioral expected output. +Each scenario's selected highest parent generation is harvested from a real run: `session.jsonl` for v0 or `session.vN.jsonl` for a positive generation. `assistant/chunk` events reproduce the model streams; tool, message, and boundary events capture the harness behavior. One ordinary Session generation therefore serves as both replay source and behavioral expected output. Every committed session-format fixture uses the canonical packed physical layout. The all-row-kinds scenario is mechanically derived from an independent real recording; its test requires every packed storage-row kind and exact event-for-event equality after both fixtures decode, then ordinary replay and log comparison prove that the assembled process consumes and reproduces the layout. @@ -53,7 +53,7 @@ Replay uses a `cordis.snapshot.yml` overlay that replaces the real adapter with A snapshot run asserts **two** normalized outputs, because the harness's external APIs are distinct: 1. The **stdout transcript** — the framed ACP JSON-RPC responses and committed-message updates an automation client receives. It catches regressions in the transport contract and is compared against a committed `stdout.expected.jsonl`. -2. The **re-persisted session JSONL**, normalized and compared with `session.jsonl`. The same fixture is both replay source and expected log. Prompt and tool bulk are scrubbed; one scenario per header class pins the remaining header sequence. The pin owns readable prompt and tool-schema sidecars by default, or names another pin as either source when the complete sequence is identical, so each distinct sidecar version is committed once. Fixture guards reject duplicate sidecar content, and record/refresh rejects shared claimants that generate different bytes. The original header-pinning rationale is preserved in the [header-pinning Agent Note](../../archived/testing/2026-07-06-pin-request-header-content-in-one-scenario.md). Override scenarios derive model behavior solely from their sidecar. +2. The **re-persisted Session JSONL**, normalized and compared with the selected highest parent fixture. The same generation is both replay source and expected log; a fresh current writer may produce a higher canonical filename while the logical comparison retains the older replay input. Prompt and tool bulk are scrubbed; one scenario per header class pins the remaining header sequence. The pin owns readable prompt and tool-schema sidecars by default, or names another pin as either source when the complete sequence is identical, so each distinct sidecar version is committed once. Fixture guards reject duplicate sidecar content, and record/refresh rejects shared claimants that generate different bytes. The original header-pinning rationale is preserved in the [header-pinning Agent Note](../../archived/testing/2026-07-06-pin-request-header-content-in-one-scenario.md). Override scenarios derive model behavior solely from their sidecar. The surfaces are complementary: stdout covers the minimal automation wire, while JSONL covers loop, tool, and boundary structure that the wire intentionally omits. @@ -69,7 +69,7 @@ Tool determinism comes from a generated cwd, scrubbed environment, fresh non-log ### Two subcommands, replay in the default gate -`pnpm run test:snapshot` replays committed fixtures keylessly; `test:snapshot:record` uses the real API and rewrites the projected session snapshot plus interface-specific expected output. The same keyless gate discovers repository JSONL by its `session` header and rejects any fixture that differs from the shared codec's projected canonical packed representation. Missing fixtures fail loud. Every ACP scenario carries `input.json`, `stdout.expected.jsonl`, and `session.jsonl`; no-model cases use a header-only log. Other profiles derive ordinary accepted user input from `session.jsonl` and retain only controller input that the accepted session cannot reconstruct in `snapshot.yml`. `replay.override.json` is required only for scenarios whose successful model behavior cannot be derived from the log. Fixture guards reject missing, mismatched, and orphaned files. Both commands accept scenario filters. +`pnpm run test:snapshot` replays committed fixtures keylessly; `test:snapshot:record` uses the real API and writes the projected current Session generation plus interface-specific expected output. The same keyless gate discovers canonical repository JSONL generations by filename/header agreement and rejects any fixture that differs from the shared codec's projected canonical packed representation. Missing roles fail loud. Every ACP scenario carries `input.json`, `stdout.expected.jsonl`, and one selected parent `session[.vN].jsonl`; no-model cases use a header-only log. Other profiles derive ordinary accepted user input from the selected parent generation and retain only controller input that the accepted Session cannot reconstruct in `snapshot.yml`. `replay.override.json` is required only for scenarios whose successful model behavior cannot be derived from the log. Fixture guards reject missing, mismatched, noncanonical, and orphaned files. Both commands accept scenario filters. ## Alternatives considered diff --git a/.agents/notes/implemented/testing/2026-06-19-acp-snapshot-tests.zh.md b/.agents/notes/implemented/testing/2026-06-19-acp-snapshot-tests.zh.md index 16ee8cb347..0cb64a95af 100644 --- a/.agents/notes/implemented/testing/2026-06-19-acp-snapshot-tests.zh.md +++ b/.agents/notes/implemented/testing/2026-06-19-acp-snapshot-tests.zh.md @@ -18,7 +18,7 @@ Status: implemented ### fixture 投影持久化会话 JSONL -每个场景的 `session.jsonl` 都从真实运行中采集。`assistant/chunk` 事件复现模型流；工具、消息和边界事件捕获 harness 行为。因此，一份普通会话产物同时充当回放来源和行为预期输出。 +每个场景数值最高的选定 parent generation 都从真实运行中采集：v0 为 `session.jsonl`，正 generation 为 `session.vN.jsonl`。`assistant/chunk` 事件复现模型 stream；工具、消息与边界事件捕获 harness 行为。因此，一份普通 Session generation 同时充当 replay source 与行为预期输出。 每个签入仓库的会话格式 fixture 都使用规范的打包物理布局。覆盖所有行类型的场景从一份独立的真实录制机械派生；测试要求它包含每一种打包存储行类型，并在两份 fixture 解码后逐事件精确相等；随后，普通回放与日志比较会证明组装后的进程能够消费并复现该布局。 @@ -53,7 +53,7 @@ Status: implemented 快照运行断言**两个**归一化后的表面，因为 harness 的外部表面是不同的： 1. **stdout transcript**——自动化客户端收到的、分帧后的 ACP JSON-RPC 响应与已提交的消息更新。它捕获传输约定的回归，与已提交的 `stdout.expected.jsonl` 比较。 -2. **重新持久化的会话 JSONL**，经过规范化后与 `session.jsonl` 比较。同一 fixture 同时作为回放来源和预期日志。提示词与工具的主体内容会被清理；每种请求头类别由一个场景固定余下的请求头序列。该 pin 默认拥有可读的提示词与工具 schema 伴随文件；当完整的对应序列相同时，也可将另一个 pin 指定为其中任一来源，因此每个不同的伴随文件版本只提交一次。fixture 守卫会拒绝重复的伴随文件内容，录制/刷新会拒绝生成不同字节的共享引用方。最初的请求头固定理由保留在[请求头固定 Agent Note](../../archived/testing/2026-07-06-pin-request-header-content-in-one-scenario.md)中。Override 场景仅从其伴随文件派生模型行为。 +2. **重新持久化的 Session JSONL**，规范化后与选定的最高 parent fixture 比较。同一个 generation 同时作为 replay source 与预期日志；新鲜当前 writer 可以产生更高的规范文件名，而逻辑比较会保留较旧 replay 输入。prompt 与工具 bulk 会被清理；每种 request header 类别由一个场景固定余下 header sequence。该 pin 默认拥有可读的 prompt 与工具 schema sidecar；完整对应 sequence 相同时，也可把另一个 pin 指定为任一来源，因此每个不同 sidecar 版本只提交一次。fixture guard 会拒绝重复 sidecar 内容，record/refresh 会拒绝生成不同字节的共享 claimant。最初的 request header 固定理由保留在[请求头固定 Agent Note](../../archived/testing/2026-07-06-pin-request-header-content-in-one-scenario.md)中。Override 场景只从其 sidecar 派生模型行为。 两个表面互补：stdout 覆盖精简的自动化协议格式，JSONL 覆盖协议格式有意省略的循环、工具和边界结构。 @@ -69,7 +69,7 @@ Status: implemented ### 两个子命令，回放在默认门禁中 -`pnpm run test:snapshot` 无需密钥即可回放已提交 fixture；`test:snapshot:record` 使用真实 API，并重写投影后的会话快照与接口专属预期输出。同一无密钥门禁会通过 `session` 头记录发现仓库中的 JSONL，并拒绝与共享编解码器的投影后规范打包表示不同的任何 fixture。缺少 fixture 时会明确报错。每个 ACP 场景都包含 `input.json`、`stdout.expected.jsonl` 和 `session.jsonl`；不调用模型的情况使用仅含头记录的日志。其他 profile 从 `session.jsonl` 推导普通的已受理用户输入，只在 `snapshot.yml` 中保留已受理会话无法重建的控制器输入。只有成功模型行为无法从日志推导的场景才需要 `replay.override.json`。fixture 守卫会拒绝缺失、不匹配和孤立文件。两个命令都接受场景过滤器。 +`pnpm run test:snapshot` 无需密钥即可 replay 已提交 fixture；`test:snapshot:record` 使用真实 API，并写入投影后的当前 Session generation 与接口专属预期输出。同一无密钥 gate 会通过文件名/header 一致性发现仓库中的规范 JSONL generation，并拒绝与共享 codec 的投影后规范打包表示不同的任何 fixture。缺失角色会明确失败。每个 ACP 场景都包含 `input.json`、`stdout.expected.jsonl` 与一个选定 parent `session[.vN].jsonl`；不调用模型的情况使用仅含 header 的日志。其他 profile 从选定 parent generation 推导普通 accepted user input，只在 `snapshot.yml` 中保留 accepted Session 无法重建的 controller input。只有成功模型行为无法从日志推导的场景才需要 `replay.override.json`。fixture guard 会拒绝缺失、不匹配、非规范与孤立文件。两个命令都接受场景 filter。 ## 曾考虑的替代方案 diff --git a/.agents/notes/implemented/testing/2026-06-22-subagent-snapshot-replay.i18n.yaml b/.agents/notes/implemented/testing/2026-06-22-subagent-snapshot-replay.i18n.yaml index 11cbbb229f..04ea7b6f32 100644 --- a/.agents/notes/implemented/testing/2026-06-22-subagent-snapshot-replay.i18n.yaml +++ b/.agents/notes/implemented/testing/2026-06-22-subagent-snapshot-replay.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/testing/2026-06-22-subagent-snapshot-replay.md -2026-06-22-subagent-snapshot-replay.md: 05ba69ee1927912309b94851d2812f147e81b1e2 -2026-06-22-subagent-snapshot-replay.zh.md: 5501db493ab0c29b69e8bdc08cb3c6fb166b8cfb +2026-06-22-subagent-snapshot-replay.md: e3da6ed984e2fc446556ee9f30d7ab9a2eef82ce +2026-06-22-subagent-snapshot-replay.zh.md: ee2271d5fc91cbf78c744a474764c1566ab5dc3b diff --git a/.agents/notes/implemented/testing/2026-06-22-subagent-snapshot-replay.md b/.agents/notes/implemented/testing/2026-06-22-subagent-snapshot-replay.md index 05ba69ee19..e3da6ed984 100644 --- a/.agents/notes/implemented/testing/2026-06-22-subagent-snapshot-replay.md +++ b/.agents/notes/implemented/testing/2026-06-22-subagent-snapshot-replay.md @@ -25,7 +25,7 @@ Replay is keyed **per calling session**, and the harness harvests **every** sess ### 2. Replay binds live sessions to recorded scripts by first-call order -A nested scenario records more than one log: the parent (`session.jsonl`) plus one per subagent child (`session.1.jsonl`, …). `dsh-llm-replay` loads them all, derives one script per recorded session, and orders the scripts by header `createdAt` (the parent is created before its children). +A nested scenario records more than one role: parent `session[.vN].jsonl`, then one per subagent child as `session.[.vN].jsonl`. V0 omits `.v0`; positive generations use lowercase `.vN`; the harness selects the numerically highest file per role. `dsh-llm-replay` loads that selected set, derives one script per recorded Session, and orders the scripts by role (parent then contiguous children), while persisted discovery still uses header `createdAt` to assign child ordinals. Live session ids are freshly random every run and never equal the recorded ones, so a live session cannot bind to a script by id equality. Instead it binds by **first-call order**: the first live session to make any model call claims the first ordered script (the parent — earliest `createdAt`, and necessarily the first to stream, because it must run a turn before it can delegate), the next new live session claims the next script, and so on. Each session then advances its own cursor independently. @@ -39,7 +39,7 @@ The alternative considered and rejected was a **call-ordered merge of the parent ### 3. The harness harvests every log, primary-first -`harvestSessionLogs` recursively collects every fixed `session.jsonl` transcript under the sessions root (the JSONL backend gives each parent and child its own project/session directory), parses each header, and orders them primary-first: the top-level session (no `parentSession`) leads, then each child by ascending `createdAt`. `RunResult.sessionLogs` is the plural result; the spec writes each back to its fixture on record (`session.jsonl` + `session..jsonl`) and diffs each harvested log against its fixture on replay. The normalizer already accepted plural session ids and collapses any stray UUID, so no normalizer change was needed. +`harvestSessionLogs` recursively selects the numerically highest canonical generation under each persistence Session directory, parses each header, and orders them primary-first: the top-level Session (no `parentSession`) leads, then each child by ascending `createdAt`. `RunResult.sessionLogs` is the plural result; record and refresh write each current output under `session[.vN].jsonl` or `session.[.vN].jsonl`, retain older generations for roles still produced, and diff the selected highest replay inputs against fresh current outputs. The normalizer already accepted plural Session ids and collapses any stray UUID, so no normalizer change was needed. ### 4. Scenarios diff --git a/.agents/notes/implemented/testing/2026-06-22-subagent-snapshot-replay.zh.md b/.agents/notes/implemented/testing/2026-06-22-subagent-snapshot-replay.zh.md index 5501db493a..ee2271d5fc 100644 --- a/.agents/notes/implemented/testing/2026-06-22-subagent-snapshot-replay.zh.md +++ b/.agents/notes/implemented/testing/2026-06-22-subagent-snapshot-replay.zh.md @@ -25,7 +25,7 @@ Status: implemented ### 2. 回放按首次调用顺序将活跃会话绑定到录制脚本 -嵌套场景录制多份日志：父会话（`session.jsonl`）加每个 subagent 子会话各一份（`session.1.jsonl`……）。`dsh-llm-replay` 全部加载，为每个录制会话派生一份脚本，并按 header 中的 `createdAt` 排序（父会话先于子会话创建）。 +嵌套场景录制多个角色：parent 使用 `session[.vN].jsonl`，每个 subagent child 使用 `session.[.vN].jsonl`。V0 省略 `.v0`，正 generation 使用小写 `.vN`，harness 为每个角色选择数值最高的文件。`dsh-llm-replay` 加载这个选定集合，为每个录制 Session 派生一份脚本，并按角色排序（parent 后接连续 child）；持久化发现仍按 header `createdAt` 分配 child ordinal。 活跃会话 id 每次运行都是全新随机值，永远不等于录制时的 id，因此活跃会话无法通过 id 相等绑定到脚本。取而代之的是**首次调用顺序**绑定：第一个发起任何模型调用的活跃会话认领第一份有序脚本（即父会话：`createdAt` 最早，且必然最先流式输出，因为它必须先运行一个轮次才能委派），下一个新活跃会话认领下一份脚本，依此类推。此后每个会话独立推进自己的游标。 @@ -39,7 +39,7 @@ Status: implemented ### 3. harness 收集所有日志，主会话优先 -`harvestSessionLogs` 递归收集 sessions 根目录下所有固定命名为 `session.jsonl` 的 transcript（JSONL 后端为每个父会话和子会话分别提供独立的项目/会话目录），解析各自的 header，并按主会话优先排序：顶层会话（无 `parentSession`）在前，各子会话按 `createdAt` 升序排列。`RunResult.sessionLogs` 包含多份日志；spec 在录制时将每份日志写回对应 fixture（`session.jsonl` + `session..jsonl`），在回放时将每份收集到的日志与其 fixture 做 diff。归一化器已支持多个会话 id 并会折叠任何游离 UUID，因此无需修改归一化器。 +`harvestSessionLogs` 会在每个持久化 Session 目录下递归选择数值最高的规范 generation，解析各自 header，并按 primary 优先排序：顶层 Session（无 `parentSession`）在前，各 child 按 `createdAt` 升序排列。`RunResult.sessionLogs` 包含多份日志；record 与 refresh 会把每份当前输出写入 `session[.vN].jsonl` 或 `session.[.vN].jsonl`，为仍产生的角色保留旧 generation，并把选定的最高 replay 输入与新鲜当前输出做 diff。normalizer 已支持多个 Session id 并会折叠任何游离 UUID，因此无需修改 normalizer。 ### 4. 场景 diff --git a/.agents/notes/implemented/testing/2026-07-24-web-gui-browser-e2e-lane.i18n.yaml b/.agents/notes/implemented/testing/2026-07-24-web-gui-browser-e2e-lane.i18n.yaml index 533f2c268b..1605f1d168 100644 --- a/.agents/notes/implemented/testing/2026-07-24-web-gui-browser-e2e-lane.i18n.yaml +++ b/.agents/notes/implemented/testing/2026-07-24-web-gui-browser-e2e-lane.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/testing/2026-07-24-web-gui-browser-e2e-lane.md -2026-07-24-web-gui-browser-e2e-lane.md: 642a2a36741468f2af1de55913c96f7146856ed4 -2026-07-24-web-gui-browser-e2e-lane.zh.md: 72b002c3be4a2ab1dbcdd279a1c8c485b5af3b3e +2026-07-24-web-gui-browser-e2e-lane.md: 6a73cced5e8f085ad5c6a7b9b280ce93ce2d1cf7 +2026-07-24-web-gui-browser-e2e-lane.zh.md: 7eb6a2e1facc1b38ca153eee3c649fbd66a7134e diff --git a/.agents/notes/implemented/testing/2026-07-24-web-gui-browser-e2e-lane.md b/.agents/notes/implemented/testing/2026-07-24-web-gui-browser-e2e-lane.md index 642a2a3674..6a73cced5e 100644 --- a/.agents/notes/implemented/testing/2026-07-24-web-gui-browser-e2e-lane.md +++ b/.agents/notes/implemented/testing/2026-07-24-web-gui-browser-e2e-lane.md @@ -32,7 +32,7 @@ Every scenario fails on any pageerror and on the client's connection-loss/gap-re ### Expected outputs -Scenarios with a stable owning region commit a normalized `ariaSnapshot()` for each distinct user-visible state; cross-region workspace-management states instead use semantic DOM assertions plus authoritative host-state checks. UUID, cwd, workspace basename, and duration volatility collapse to stable tokens; captures poll until consecutive normalized reads agree. Role and text anchors remain semantic guards around the reviewable expected outputs and own cross-region states directly. A scenario that owns its recorded session compares the normalized re-persisted root session with that same `session.jsonl` by default; borrowed or derived fixtures opt out explicitly. World-state assertions still use authoritative host state or an independent complete `workspace.expected/`, because a matching transcript cannot prove an external effect. `refresh` is the sole ARIA expected-output writer; a missing replay expectation fails with the regeneration command. +Scenarios with a stable owning region commit a normalized `ariaSnapshot()` for each distinct user-visible state; cross-region workspace-management states instead use semantic DOM assertions plus authoritative host-state checks. UUID, cwd, workspace basename, and duration volatility collapse to stable tokens; captures poll until consecutive normalized reads agree. Role and text anchors remain semantic guards around the reviewable expected outputs and own cross-region states directly. A scenario that owns its recorded Session compares the normalized re-persisted root generation with the selected highest parent fixture by default (`session.jsonl` for v0 or `session.vN.jsonl` for a positive generation); borrowed or derived fixtures opt out explicitly. World-state assertions still use authoritative host state or an independent complete `workspace.expected/`, because a matching transcript cannot prove an external effect. `refresh` is the sole ARIA expected-output writer; a missing replay expectation fails with the regeneration command. The typecheck plane split is structural: the host scaffold, its support module, and every web spec that boots or inspects the host composition are excluded from the client-registered `apps/web` project and included file-by-file in `tsconfig.host.json`. One program cannot hold both sides of the Cordis `Context` merges. @@ -66,7 +66,7 @@ Surveyed AI-chat/agent web UIs and mocking layers (LibreChat, vercel/ai-chatbot **A `packages/test-support/web-snapshot` package with a `defineWebSnapshotSuite` factory.** Rejected: chromium-driving source cannot honestly hold per-file 100% coverage on browserless coverage runners, and the scenario-specific interactions have not produced a stable browser-free contract beyond the helpers already exported from gated packages and the local scaffold. Reconsider when a second web-shaped consumer or demonstrably repeated lifecycle code establishes that contract. -**A separately maintained normalized-session-log golden as a second expected surface.** Rejected: duplicating the recorded fixture as another file would double refresh cost without adding an independent oracle. The corpus instead compares the normalized persisted result with the same `session.jsonl` that drives replay, while host-state and complete workspace assertions retain the independent world-verification duty. +**A separately maintained normalized-Session-log golden as a second expected surface.** Rejected: duplicating the selected recorded generation as another file would double refresh cost without adding an independent oracle. The corpus instead compares the normalized persisted result with the same highest-generation parent fixture that drives replay, while host-state and complete workspace assertions retain the independent world-verification duty. **Spawning the `dsh web` bin with a `DSH_SNAPSHOT` replay branch.** Rejected: it needs a test-only replay branch plus environment plumbing in the shipped CLI. The in-process scaffold already loads the same `apps/cli/config/base.cordis.yml` plus `apps/cli/config/web.cordis.yml`; only argv, profile JSON, and `AppCLIEntry` glue remain outside it, and the keyless CLI smokes cover those paths. diff --git a/.agents/notes/implemented/testing/2026-07-24-web-gui-browser-e2e-lane.zh.md b/.agents/notes/implemented/testing/2026-07-24-web-gui-browser-e2e-lane.zh.md index 72b002c3be..7eb6a2e1fa 100644 --- a/.agents/notes/implemented/testing/2026-07-24-web-gui-browser-e2e-lane.zh.md +++ b/.agents/notes/implemented/testing/2026-07-24-web-gui-browser-e2e-lane.zh.md @@ -32,7 +32,7 @@ Web GUI 以一条真实组装链交付——chromium 页面 → client 插件 bu ### 预期输出 -具有稳定所属区域的场景会为每个不同的用户可见状态提交一份规范化的 `ariaSnapshot()`；跨区域的工作区管理状态则使用语义 DOM 断言和权威的 host 状态检查。UUID、cwd、工作区目录名与时长等易变内容会归一为稳定 token；采集过程持续轮询，直到连续两次规范化读取结果相同。Role 与文本锚点继续充当可评审预期输出周围的语义防线，并直接覆盖跨区域状态。拥有录制会话的场景默认将规范化后的重新持久化根会话与同一份 `session.jsonl` 比较；借用或派生 fixture 的场景显式关闭该比较。世界状态断言仍使用权威 host 状态或独立、完整的 `workspace.expected/`，因为 transcript 匹配不能证明外部效果。`refresh` 是 ARIA 预期输出的唯一写入者；回放模式下缺少预期输出时，测试会连同重新生成命令一起失败。 +具有稳定所属区域的场景会为每个不同的用户可见状态提交一份规范化的 `ariaSnapshot()`；跨区域的 workspace 管理状态则使用语义 DOM 断言与权威 host 状态检查。UUID、cwd、workspace basename 与时长等易变内容会归一为稳定 token；采集过程持续轮询，直到连续两次规范化读取结果相同。Role 与文本锚点继续充当可评审预期输出周围的语义防线，并直接覆盖跨区域状态。拥有录制 Session 的场景默认将规范化后的重新持久化根 generation 与选定的最高 parent fixture 比较（v0 为 `session.jsonl`，正 generation 为 `session.vN.jsonl`）；借用或派生 fixture 的场景显式关闭该比较。世界状态断言仍使用权威 host 状态或独立、完整的 `workspace.expected/`，因为 transcript 匹配不能证明外部效果。`refresh` 是 ARIA 预期输出的唯一 writer；replay 模式下缺少预期输出时，测试会连同重新生成命令一起失败。 类型检查平面切分是结构性的：host scaffold、其支持模块，以及每个启动或检查 host 组合的 web spec 都会从注册在 client 侧的 `apps/web` 工程中排除，并逐文件纳入 `tsconfig.host.json`。一个程序不能同时持有 Cordis `Context` 合并的两侧。 @@ -66,7 +66,7 @@ Web GUI 以一条真实组装链交付——chromium 页面 → client 插件 bu **`packages/test-support/web-snapshot` 包 + `defineWebSnapshotSuite` 工厂。** 已否决：驱动 chromium 的源码在无浏览器的覆盖率 runner 上无法诚实保持逐文件 100%，且除受门禁的包已导出的辅助工具与本地 scaffold 外，这些场景专用交互尚未形成稳定的无浏览器约定。出现第二个 web 形态消费方，或被证实重复的生命周期代码确立该约定后，再重新考虑。 -**单独维护第二份规范化会话日志预期输出。** 已否决：把录制 fixture 再复制成另一个文件会使刷新成本翻倍，却不会增加独立 oracle。语料改为将规范化持久化结果与驱动回放的同一份 `session.jsonl` 比较，同时以 host 状态和完整 workspace 断言保留独立的世界验证义务。 +**单独维护第二份规范化 Session 日志预期输出。** 不予采用：把选定的录制 generation 再复制成另一个文件会使 refresh 成本翻倍，却不会增加独立 oracle。语料改为将规范化持久化结果与驱动 replay 的同一份最高 generation parent fixture 比较，同时以 host 状态和完整 workspace 断言保留独立的世界验证义务。 **以 `DSH_SNAPSHOT` 回放分支拉起 `dsh web` bin。** 已否决：它需要在交付的 CLI 中增加测试专用回放分支和环境变量管道。进程内 scaffold 已加载同一份 `apps/cli/config/base.cordis.yml` 与 `apps/cli/config/web.cordis.yml`；只剩 argv、profile JSON 和 `AppCLIEntry` 胶水不在其覆盖范围内，而这些路径已由无密钥 CLI 冒烟覆盖。 diff --git a/.agents/skills/dsh-pre-push-checks/SKILL.md b/.agents/skills/dsh-pre-push-checks/SKILL.md index 9a687e8f9f..663762c55d 100644 --- a/.agents/skills/dsh-pre-push-checks/SKILL.md +++ b/.agents/skills/dsh-pre-push-checks/SKILL.md @@ -33,7 +33,7 @@ When the outgoing change adds or changes a resource-owning or asynchronous test, - **Package or script behavior:** run the owning Vitest file or focused test name. Add adjacent package tests when a shared contract changes; leave repository-wide coverage to CI unless the change is genuinely cross-cutting or the user requests it. - **Documentation, Agent Notes, catalogs, or doc-linked comments:** run `pnpm run doc-sync`; run full lint when the documentation workflow requires it. - **Model-, editor-, CLI-, or terminal-visible output:** run the focused keyless snapshot or real runnable-example scenario that owns the output. -- **Expected-output placement:** a test whose recorded `session.jsonl` is replay input and expected persisted output belongs under top-level `snapshots/`, with `snapshot.yml` naming its shipped `dsh` profile and composition/header pin. ARIA, geometry, generator, CLI, and unit expectations without that session round trip stay beside their owning test under `tests/expected/`; do not place them in `snapshots/` or give them a `*.snapshot.ts` owner. Use the owning `test:expected`, `test:web`, or `test` lane. +- **Expected-output placement:** a test whose selected recorded Session generation is replay input and expected persisted output belongs under top-level `snapshots/`, with `snapshot.yml` naming its shipped `dsh` profile and composition/header pin. Canonical parent files are `session[.vN].jsonl`, children are `session.[.vN].jsonl`, and the harness selects the highest generation per role. ARIA, geometry, generator, CLI, and unit expectations without that Session round trip stay beside their owning test under `tests/expected/`; do not place them in `snapshots/` or give them a `*.snapshot.ts` owner. Use the owning `test:expected`, `test:web`, or `test` lane. - **Profile and configuration placement:** cross-package behavior of a shipped `dsh` profile belongs under `apps/cli/tests/profiles/`; a package-specific Loader composition belongs under that package's `tests/fixtures/`. User-facing optional overlays live under `apps/cli/config/examples/` and pair with a guide under `docs/user/`. - **Package manifests, public exports, build configuration, worker/bin entries, or built runtime paths:** run `pnpm run build`, the relevant hygiene checks, and the owning built-artifact smoke. - **Real provider or agent behavior:** run the relevant `pnpm run test:e2e` target when credentials are available; never print secrets. diff --git a/AGENTS.md b/AGENTS.md index df47d34f4d..d867db746e 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -2,9 +2,9 @@ DeepSeek Harness is an all-plugin Cordis agent harness. Read [docs/architecture.md](docs/architecture.md) before changing `packages/`; follow [docs/AGENTS.md](docs/AGENTS.md) for documentation. -## Pre-release stance: foundation over blast radius +## Pre-stable APIs and released Session data -**Remove at the first tagged release.** Until then, prefer correct foundations to compatibility shims: rename or repackage freely and update every reference. Backends reject old on-disk formats. SQLite uses monotonic `SCHEMA_VERSION`; `dsh-session` keeps `SESSION_FORMAT_VERSION` at `0` with no compatibility promise. +Public APIs are pre-stable; update every consumer. Released Session JSONL follows [adjacent migration](.agents/notes/implemented/architecture/2026-08-31-released-session-format-migrations.md): body reads may add a version-named successor but never move, overwrite, or delete committed generations; predecessors imply neither fallback nor downgrade support. SQLite domains use monotonic `SCHEMA_VERSION`. **Application launch.** Only `dsh` profiles launch supported Node apps; package bins, demos, and public SDK argv escapes are forbidden ([rule](docs/architecture.md#application-launch)). diff --git a/apps/cli/tests/profiles/acp/tests/goal-expected/goal-round-driver/session.expected.jsonl b/apps/cli/tests/profiles/acp/tests/goal-expected/goal-round-driver/session.expected.jsonl index 390b157a76..4e7a211574 100644 --- a/apps/cli/tests/profiles/acp/tests/goal-expected/goal-round-driver/session.expected.jsonl +++ b/apps/cli/tests/profiles/acp/tests/goal-expected/goal-round-driver/session.expected.jsonl @@ -1,4 +1,4 @@ -{"type":"session","version":0,"id":"{{sessionId}}","createdAt":0,"cwd":"{{cwd}}","delegationDepth":0} +{"type":"session","version":1,"id":"{{sessionId}}","createdAt":0,"cwd":"{{cwd}}","delegationDepth":0} {"type":"permission/preset","data":{"preset":"danger-full-access"}} {"type":"sandbox/mode","data":{"mode":"danger-full-access"}} {"type":"approval/policy","data":{"policy":"never"}} diff --git a/apps/cli/tests/profiles/acp/tests/goal-expected/goal-wrapup/session.expected.jsonl b/apps/cli/tests/profiles/acp/tests/goal-expected/goal-wrapup/session.expected.jsonl index b724c08ef5..d5848cf48b 100644 --- a/apps/cli/tests/profiles/acp/tests/goal-expected/goal-wrapup/session.expected.jsonl +++ b/apps/cli/tests/profiles/acp/tests/goal-expected/goal-wrapup/session.expected.jsonl @@ -1,4 +1,4 @@ -{"type":"session","version":0,"id":"{{sessionId}}","createdAt":0,"cwd":"{{cwd}}","delegationDepth":0} +{"type":"session","version":1,"id":"{{sessionId}}","createdAt":0,"cwd":"{{cwd}}","delegationDepth":0} {"type":"permission/preset","data":{"preset":"danger-full-access"}} {"type":"sandbox/mode","data":{"mode":"danger-full-access"}} {"type":"approval/policy","data":{"policy":"never"}} diff --git a/apps/cli/tests/profiles/headless/tests/expected/headless-profile/session.expected.jsonl b/apps/cli/tests/profiles/headless/tests/expected/headless-profile/session.expected.jsonl index 3ddd2377fc..8430d23f28 100644 --- a/apps/cli/tests/profiles/headless/tests/expected/headless-profile/session.expected.jsonl +++ b/apps/cli/tests/profiles/headless/tests/expected/headless-profile/session.expected.jsonl @@ -1,4 +1,4 @@ -{"type":"session","version":0,"id":"{{sessionId}}","createdAt":0,"cwd":"{{cwd}}","delegationDepth":0} +{"type":"session","version":1,"id":"{{sessionId}}","createdAt":0,"cwd":"{{cwd}}","delegationDepth":0} {"type":"permission/preset","data":{"preset":"danger-full-access"}} {"type":"sandbox/mode","data":{"mode":"danger-full-access"}} {"type":"approval/policy","data":{"policy":"never"}} diff --git a/apps/cli/tests/profiles/headless/tests/expected/semantic-checkpoint/session.expected.jsonl b/apps/cli/tests/profiles/headless/tests/expected/semantic-checkpoint/session.expected.jsonl index 221c2a522d..8dd7bd6de3 100644 --- a/apps/cli/tests/profiles/headless/tests/expected/semantic-checkpoint/session.expected.jsonl +++ b/apps/cli/tests/profiles/headless/tests/expected/semantic-checkpoint/session.expected.jsonl @@ -1,4 +1,4 @@ -{"type":"session","version":0,"id":"{{sessionId}}","createdAt":0,"cwd":"{{cwd}}","delegationDepth":0} +{"type":"session","version":1,"id":"{{sessionId}}","createdAt":0,"cwd":"{{cwd}}","delegationDepth":0} {"type":"turn/start","data":{"turn":1}} {"type":"user/message","data":{"content":[{"type":"text","text":"Perform one side-effecting remote mutation."}],"source":{"kind":"user"},"role":"user","id":"{{sessionId}}"},"surfaceOp":"append"} {"type":"step/start","data":{"turn":1,"step":1}} diff --git a/apps/cli/tests/profiles/headless/tests/expected/subagent-diagnostic/parent.expected.jsonl b/apps/cli/tests/profiles/headless/tests/expected/subagent-diagnostic/parent.expected.jsonl index 6db6d26f41..0dd94885f0 100644 --- a/apps/cli/tests/profiles/headless/tests/expected/subagent-diagnostic/parent.expected.jsonl +++ b/apps/cli/tests/profiles/headless/tests/expected/subagent-diagnostic/parent.expected.jsonl @@ -1,4 +1,4 @@ -{"type":"session","version":0,"id":"{{sessionId}}","createdAt":0,"cwd":"{{cwd}}","delegationDepth":0} +{"type":"session","version":1,"id":"{{sessionId}}","createdAt":0,"cwd":"{{cwd}}","delegationDepth":0} {"type":"turn/start","data":{"turn":1}} {"type":"user/message","data":{"content":[{"type":"text","text":"Start a background job."}],"source":{"kind":"user"},"role":"user","id":"{{sessionId}}"},"surfaceOp":"append"} {"type":"turn/end","data":{"turn":1,"reason":{"kind":"completed"}}} diff --git a/apps/cli/tests/profiles/headless/tests/expected/subagent-inheritance/child.expected.jsonl b/apps/cli/tests/profiles/headless/tests/expected/subagent-inheritance/child.expected.jsonl index 22dec79be0..49c0c5a091 100644 --- a/apps/cli/tests/profiles/headless/tests/expected/subagent-inheritance/child.expected.jsonl +++ b/apps/cli/tests/profiles/headless/tests/expected/subagent-inheritance/child.expected.jsonl @@ -1,4 +1,4 @@ -{"type":"session","version":0,"id":"{{sessionId}}","createdAt":0,"cwd":"{{cwd}}","parentSession":"{{sessionId}}","origin":"subagent","delegationDepth":1} +{"type":"session","version":1,"id":"{{sessionId}}","createdAt":0,"cwd":"{{cwd}}","parentSession":"{{sessionId}}","origin":"subagent","delegationDepth":1} {"type":"sandbox/mode","data":{"mode":"read-only","source":"delegation"}} {"type":"approval/policy","data":{"policy":"never","source":"delegation"}} {"type":"agent/inbox/spliced","data":{"target":"next-turn","start":0,"inserted":[{"content":[{"type":"text","text":"Use the write tool exactly once with file_path set to exactly the relative path inherited.txt and content escaped. If the write is denied, reply with the single word CHILD_DENIED and the denial marker line; do not retry and do not request escalation. If it succeeds, reply CHILD_WROTE."}],"source":{"kind":"user"},"role":"user","id":"{{sessionId}}"}]}} diff --git a/apps/cli/tests/profiles/headless/tests/expected/subagent-inheritance/parent.expected.jsonl b/apps/cli/tests/profiles/headless/tests/expected/subagent-inheritance/parent.expected.jsonl index c8e2424126..33d1fb0723 100644 --- a/apps/cli/tests/profiles/headless/tests/expected/subagent-inheritance/parent.expected.jsonl +++ b/apps/cli/tests/profiles/headless/tests/expected/subagent-inheritance/parent.expected.jsonl @@ -1,4 +1,4 @@ -{"type":"session","version":0,"id":"{{sessionId}}","createdAt":0,"cwd":"{{cwd}}","delegationDepth":0} +{"type":"session","version":1,"id":"{{sessionId}}","createdAt":0,"cwd":"{{cwd}}","delegationDepth":0} {"type":"turn/start","data":{"turn":1}} {"type":"user/message","data":{"content":[{"type":"text","text":"Tighten this session to read-only."}],"source":{"kind":"user"},"role":"user","id":"{{sessionId}}"},"surfaceOp":"append"} {"type":"sandbox/mode","data":{"mode":"read-only"}} diff --git a/apps/cli/tests/profiles/headless/tests/expected/subagent-settlement/child.expected.jsonl b/apps/cli/tests/profiles/headless/tests/expected/subagent-settlement/child.expected.jsonl index 18c9ac4ff4..af1a3ac71b 100644 --- a/apps/cli/tests/profiles/headless/tests/expected/subagent-settlement/child.expected.jsonl +++ b/apps/cli/tests/profiles/headless/tests/expected/subagent-settlement/child.expected.jsonl @@ -1,4 +1,4 @@ -{"type":"session","version":0,"id":"{{sessionId}}","createdAt":0,"cwd":"{{cwd}}","parentSession":"{{sessionId}}","origin":"subagent","delegationDepth":1} +{"type":"session","version":1,"id":"{{sessionId}}","createdAt":0,"cwd":"{{cwd}}","parentSession":"{{sessionId}}","origin":"subagent","delegationDepth":1} {"type":"subagent/descriptor","data":{"version":3,"mode":"continuable","provider":"spawn","label":"Return child result","agentProvider":"deepseek-official","agentModel":"deepseek-v4-flash"}} {"type":"session/end-seed","data":{}} {"type":"sandbox/mode","data":{"mode":"workspace-write","source":"delegation"}} diff --git a/apps/cli/tests/profiles/headless/tests/expected/subagent-settlement/child.replay.jsonl b/apps/cli/tests/profiles/headless/tests/expected/subagent-settlement/child.replay.jsonl index 6e1f89f9ec..fc3078d671 100644 --- a/apps/cli/tests/profiles/headless/tests/expected/subagent-settlement/child.replay.jsonl +++ b/apps/cli/tests/profiles/headless/tests/expected/subagent-settlement/child.replay.jsonl @@ -1,4 +1,6 @@ {"type":"session","version":0,"id":"subagent-settlement-child","createdAt":2,"delegationDepth":1} +{"type":"turn/start","data":{"turn":1}} +{"type":"step/start","data":{"turn":1,"step":1}} {"type":"assistant/chunk","data":{"turn":1,"step":1,"chunk":{"type":"block-start","index":0,"blockType":"text"}}} {"type":"assistant/chunk","data":{"turn":1,"step":1,"chunk":{"type":"text-delta","index":0,"text":"CHILD_RESULT"}}} {"type":"assistant/chunk","data":{"turn":1,"step":1,"chunk":{"type":"block-end","index":0,"block":{"type":"text","text":"CHILD_RESULT"}}}} diff --git a/apps/cli/tests/profiles/headless/tests/expected/workspace-context-resume/offline-edit/session.expected.jsonl b/apps/cli/tests/profiles/headless/tests/expected/workspace-context-resume/offline-edit/session.expected.jsonl index ba2e593e50..1a196c3640 100644 --- a/apps/cli/tests/profiles/headless/tests/expected/workspace-context-resume/offline-edit/session.expected.jsonl +++ b/apps/cli/tests/profiles/headless/tests/expected/workspace-context-resume/offline-edit/session.expected.jsonl @@ -1,4 +1,4 @@ -{"type":"session","version":0,"id":"{{sessionId}}","createdAt":0,"cwd":"{{cwd}}","delegationDepth":0} +{"type":"session","version":1,"id":"{{sessionId}}","createdAt":0,"cwd":"{{cwd}}","delegationDepth":0} {"type":"turn/start","data":{"turn":1}} {"type":"user/message","data":{"content":[{"type":"text","text":"Remember the workspace instruction."}],"source":{"kind":"user"},"role":"user","id":"{{sessionId}}"},"surfaceOp":"append"} {"type":"user/message","data":{"content":[{"type":"text","text":"\nThe following workspace instructions may be relevant to your work. Use them as guidance when applicable. More specific instructions take precedence over broader ones. They do not override system, developer, or direct user instructions.\n\nInstructions from: AGENTS.md\n\nOld workspace instruction.\n"}],"source":{"kind":"agent-instructions","form":"instructions","baseline":true,"baselineIdentity":"{\"projectRoot\":\"\",\"projectRootMarkers\":[\".git\"],\"maxBytes\":65536,\"maxSourceBytes\":1048576,\"instructionFileCandidates\":[\"AGENTS.md\",\"CLAUDE.md\"],\"localInstructionFileCandidates\":[\"AGENTS.local.md\",\"CLAUDE.local.md\"]}","changes":[{"action":"set","scope":".\u0000AGENTS.md","path":"AGENTS.md","digest":"ba65bdb41810f4d0129129dcbd6cadcd643c069d"}]},"role":"user","id":"{{sessionId}}"},"surfaceOp":"append"} diff --git a/apps/cli/tests/profiles/headless/tests/expected/workspace-context-resume/precedence-change/session.expected.jsonl b/apps/cli/tests/profiles/headless/tests/expected/workspace-context-resume/precedence-change/session.expected.jsonl index 0e793a4480..a9ddd628b2 100644 --- a/apps/cli/tests/profiles/headless/tests/expected/workspace-context-resume/precedence-change/session.expected.jsonl +++ b/apps/cli/tests/profiles/headless/tests/expected/workspace-context-resume/precedence-change/session.expected.jsonl @@ -1,4 +1,4 @@ -{"type":"session","version":0,"id":"{{sessionId}}","createdAt":0,"cwd":"{{cwd}}","delegationDepth":0} +{"type":"session","version":1,"id":"{{sessionId}}","createdAt":0,"cwd":"{{cwd}}","delegationDepth":0} {"type":"turn/start","data":{"turn":1}} {"type":"user/message","data":{"content":[{"type":"text","text":"Remember the workspace instruction."}],"source":{"kind":"user"},"role":"user","id":"{{sessionId}}"},"surfaceOp":"append"} {"type":"user/message","data":{"content":[{"type":"text","text":"\nThe following workspace instructions may be relevant to your work. Use them as guidance when applicable. More specific instructions take precedence over broader ones. They do not override system, developer, or direct user instructions.\n\nInstructions from: CLAUDE.md\n\nOld CLAUDE rule.\n\nInstructions from: AGENTS.md\n\nOld AGENTS rule.\n"}],"source":{"kind":"agent-instructions","form":"instructions","baseline":true,"baselineIdentity":"{\"projectRoot\":\"\",\"projectRootMarkers\":[\".git\"],\"maxBytes\":65536,\"maxSourceBytes\":1048576,\"instructionFileCandidates\":[\"CLAUDE.md\",\"AGENTS.md\"],\"localInstructionFileCandidates\":[\"AGENTS.local.md\",\"CLAUDE.local.md\"]}","changes":[{"action":"set","scope":".\u0000CLAUDE.md","path":"CLAUDE.md","digest":"b525eb8a6d3660b732dad4b0aff1b7c63ab32890"},{"action":"set","scope":".\u0000AGENTS.md","path":"AGENTS.md","digest":"3113bd093ae91976207dcef7390bdc0b2bfcfa10"}]},"role":"user","id":"{{sessionId}}"},"surfaceOp":"append"} diff --git a/apps/cli/tests/profiles/headless/tests/session-format-guard.expected.e2e.ts b/apps/cli/tests/profiles/headless/tests/session-format-guard.expected.e2e.ts index a1f5d0ff9e..3119b39aa4 100644 --- a/apps/cli/tests/profiles/headless/tests/session-format-guard.expected.e2e.ts +++ b/apps/cli/tests/profiles/headless/tests/session-format-guard.expected.e2e.ts @@ -1,12 +1,13 @@ /** - * Assembled-app regression for the session-format refusal surface: resuming a - * log written by a "newer" harness (format version ahead, or an unknown - * required event type) fails loud through the real Loader composition, and the - * error the product user sees names the direction and the raw log path. + * Assembled-app regressions for Session-format lifecycle behavior: a physical + * v0 log migrates through the real Loader composition, remains byte-for-byte + * intact beside v1, and accepts the next append through v1; a future format or unknown + * required current event refuses with the direction and raw log path. * @module session-format-guard-snapshot */ import { join, dirname } from 'node:path' +import { mkdir, readFile, readdir, stat, writeFile } from 'node:fs/promises' import { fileURLToPath } from 'node:url' import { Context } from '@deepseek-ai/cordis' import { LOADER_SMOKE_TEST_TIMEOUT_MS, runLoaderSmoke } from '@deepseek-ai/dsh-loader-smoke' @@ -18,6 +19,7 @@ import SessionStore, { type SessionHeader, } from '@deepseek-ai/dsh-session' import JsonlSessionPersistence from '@deepseek-ai/dsh-session-persistence-jsonl' +import { generationLogFilename } from '@deepseek-ai/dsh-session-persistence-jsonl/src/format.ts' import { describe, expect, it } from 'vitest' const fixtureDir = join(dirname(fileURLToPath(import.meta.url)), 'expected/workspace-context-resume/offline-edit') @@ -28,18 +30,29 @@ const tsconfigPath = fileURLToPath(new URL('../../../../../../tsconfig.json', im // The resumed-agent fixture in the shared config resumes exactly this id. const sessionId = SessionId('workspace-context-resume') -/** Persist one session with the given header version and events, returning its log path. */ +/** Stage one physical raw JSONL session without passing through the current writer. */ async function seedSession(root: string, cwd: string, version: number, events: SessionEvent[]): Promise { const ctx = new Context() await ctx.plugin(SessionStore) await ctx.plugin(JsonlSessionPersistence, { root, compression: 'none' }) - const meta: SessionHeader = { version, id: sessionId, createdAt: 1, cwd, isSeeded: false } + const meta: SessionHeader = { + version: SESSION_FORMAT_VERSION, + id: sessionId, + createdAt: 1, + cwd, + isSeeded: false, + } try { - await ctx.sessionPersistence.create(meta) - await ctx.sessionPersistence.append(sessionId, events) const location = ctx.sessionPersistence.locate(meta) if (location === undefined) throw new Error('JSONL backend did not locate the seeded session') - return location.path + const content = [ + { type: 'session', version, id: sessionId, createdAt: 1, cwd, delegationDepth: 0 }, + ...events, + ].map(record => JSON.stringify(record)).join('\n') + '\n' + const path = join(dirname(location.path), generationLogFilename(version, 'none')) + await mkdir(dirname(path), { recursive: true }) + await writeFile(path, content) + return path } finally { await ctx.fiber.dispose() } @@ -53,6 +66,42 @@ function closedTurn(): SessionEvent[] { } describe('session format guard through the assembled app', () => { + it('migrates a raw v0 log before resume, preserves exact source bytes, and appends only to v1', async () => { + let v0Path = '' + let v0 = '' + let v0Identity: { readonly dev: bigint; readonly ino: bigint } | undefined + await runLoaderSmoke({ + label: 'v0 identity migration before resume', + tempDirPrefix: 'dsh-format-migrate-v0-', + binScript, + libBinScript: binScript, + configPath, + binArgs: [configPath, 'Continue the migrated session.'], + tsconfigPath, + env: { DSH_SNAPSHOT_FILE: replayFixture }, + prepare: async (runCwd) => { + v0Path = await seedSession(join(runCwd, '.sessions'), runCwd, 0, closedTurn()) + v0 = await readFile(v0Path, 'utf8') + const identity = await stat(v0Path, { bigint: true }) + v0Identity = { dev: identity.dev, ino: identity.ino } + }, + inspect: async () => { + const v1Path = join(dirname(v0Path), generationLogFilename(SESSION_FORMAT_VERSION, 'none')) + const current = await readFile(v1Path, 'utf8') + const sourceIdentity = await stat(v0Path, { bigint: true }) + const currentIdentity = await stat(v1Path, { bigint: true }) + expect(await readFile(v0Path, 'utf8')).toBe(v0) + expect({ dev: sourceIdentity.dev, ino: sourceIdentity.ino }).toEqual(v0Identity) + expect({ dev: currentIdentity.dev, ino: currentIdentity.ino }).not.toEqual(v0Identity) + expect((JSON.parse(current.split('\n')[0] as string) as { version: number }).version) + .toBe(SESSION_FORMAT_VERSION) + expect(current).not.toBe(v0) + expect(current.trimEnd().split('\n').length).toBeGreaterThan(closedTurn().length + 1) + expect((await readdir(dirname(v0Path))).sort()).toEqual(['session.jsonl', 'session.v1.jsonl']) + }, + }) + }, LOADER_SMOKE_TEST_TIMEOUT_MS) + it('refuses to resume a newer-format log, naming the upgrade direction and the raw log path', async () => { let sessionPath = '' const result = await runLoaderSmoke({ diff --git a/apps/web/tests/cold-blank-session.e2e.ts b/apps/web/tests/cold-blank-session.e2e.ts index b9e64960d3..9036c644b4 100644 --- a/apps/web/tests/cold-blank-session.e2e.ts +++ b/apps/web/tests/cold-blank-session.e2e.ts @@ -6,6 +6,7 @@ import { join } from 'node:path' import type { Browser, Page } from 'playwright' import { chromium } from 'playwright' import { afterAll, beforeAll, describe, expect, it, onTestFailed } from 'vitest' +import { isReadableSessionPersistenceListing } from '@deepseek-ai/dsh-session-persistence' import { captureStableAria, compareOrRefreshGolden, launchWebScaffold, seedBlankSession, watchConsole, webSnapshotMode, type WebScaffold, @@ -29,10 +30,11 @@ describe('web e2e: cold blank Session visibility', () => { const cwd = join(scaffold.workspaceCwd, WORKSPACE_NAME) await mkdir(cwd, { recursive: true }) await seedBlankSession(scaffold, SESSION_ID, cwd) - const header = (await scaffold.ctx.sessionPersistence.list()) - .find(candidate => candidate.id === SESSION_ID) - if (header === undefined) throw new Error('blank Session fixture did not materialize') - const location = scaffold.ctx.sessionPersistence.locate(header) + const listing = (await scaffold.ctx.sessionPersistence.list()) + .filter(isReadableSessionPersistenceListing) + .find(candidate => candidate.header.id === SESSION_ID) + if (listing === undefined) throw new Error('blank Session fixture did not materialize') + const location = scaffold.ctx.sessionPersistence.locate(listing.header) if (location === undefined) throw new Error('JSONL fixture has no physical artifact') expect((await stat(location.path)).size).toBeLessThanOrEqual(1024) diff --git a/apps/web/tests/markdown-cjk-strong.e2e.ts b/apps/web/tests/markdown-cjk-strong.e2e.ts index da82648c50..17ec2f515c 100644 --- a/apps/web/tests/markdown-cjk-strong.e2e.ts +++ b/apps/web/tests/markdown-cjk-strong.e2e.ts @@ -75,6 +75,7 @@ function markdownFixture(): string { id: '{{sessionId}}', createdAt: 0, cwd: '{{cwd}}', + delegationDepth: 0, }), ...session.snapshotEvents().map(event => JSON.stringify({ ...event, diff --git a/apps/web/tests/markdown-images.e2e.ts b/apps/web/tests/markdown-images.e2e.ts index a888469339..78288fbb45 100644 --- a/apps/web/tests/markdown-images.e2e.ts +++ b/apps/web/tests/markdown-images.e2e.ts @@ -124,6 +124,7 @@ function markdownImageFixture(remoteUrl: string): string { id: '{{sessionId}}', createdAt: 0, cwd: '{{cwd}}', + delegationDepth: 0, } return [ JSON.stringify(header), diff --git a/apps/web/tests/markdown-inline-code-links.e2e.ts b/apps/web/tests/markdown-inline-code-links.e2e.ts index d1da29922a..338574b25c 100644 --- a/apps/web/tests/markdown-inline-code-links.e2e.ts +++ b/apps/web/tests/markdown-inline-code-links.e2e.ts @@ -72,6 +72,7 @@ function markdownFixture(linkUrl: string): string { id: '{{sessionId}}', createdAt: 0, cwd: '{{cwd}}', + delegationDepth: 0, }), ...session.snapshotEvents().map(event => JSON.stringify({ ...event, diff --git a/apps/web/tests/markdown-wide-table.e2e.ts b/apps/web/tests/markdown-wide-table.e2e.ts index 28e98a89a6..eed8d1fdd6 100644 --- a/apps/web/tests/markdown-wide-table.e2e.ts +++ b/apps/web/tests/markdown-wide-table.e2e.ts @@ -135,6 +135,7 @@ function wideTableFixture(): string { id: '{{sessionId}}', createdAt: 0, cwd: '{{cwd}}', + delegationDepth: 0, } return [ JSON.stringify(header), diff --git a/apps/web/tests/math-rendering.e2e.ts b/apps/web/tests/math-rendering.e2e.ts index 020d4b2a3a..157a7fd810 100644 --- a/apps/web/tests/math-rendering.e2e.ts +++ b/apps/web/tests/math-rendering.e2e.ts @@ -76,6 +76,7 @@ function mathFixture(): string { id: '{{sessionId}}', createdAt: 0, cwd: '{{cwd}}', + delegationDepth: 0, }), ...session.snapshotEvents().map(event => JSON.stringify({ ...event, diff --git a/apps/web/tests/message-actions.e2e.ts b/apps/web/tests/message-actions.e2e.ts index de849baea3..c91882a6fc 100644 --- a/apps/web/tests/message-actions.e2e.ts +++ b/apps/web/tests/message-actions.e2e.ts @@ -39,12 +39,18 @@ function completedTailFixture(raw: string): string { const decoded = parseSeedFixture(raw) const kept = decoded.events.filter(event => event.seq < 101).map((event) => { if (event.type === 'assistant/message' && event.seq === 64) { - const data = event.data as unknown as { content?: unknown[] } - const content = data.content + const data = event.data as unknown as { message?: { content?: unknown[] } } + const content = data.message?.content if (!Array.isArray(content)) throw new Error('borrowed step-one assistant message has no content') return { ...event, - data: { ...data, content: [...content.slice(0, 1), { type: 'text', text: MID_TURN_TEXT }, ...content.slice(1)] }, + data: { + ...data, + message: { + ...data.message, + content: [...content.slice(0, 1), { type: 'text', text: MID_TURN_TEXT }, ...content.slice(1)], + }, + }, } } return event @@ -161,7 +167,7 @@ describe('web e2e: message IconActions and clocks on settled history', () => { if (rowBox === null) throw new Error('fork source row has no layout box') const actionButton = sourceRow.locator('button[aria-label^="Session actions for "]') await sourceRow.hover({ position: { x: rowBox.width - 16, y: rowBox.height / 2 } }) - await expect.poll(() => actionButton.isVisible(), { timeout: 2_000 }).toBe(true) + await expect.poll(() => actionButton.isVisible(), { timeout: 10_000 }).toBe(true) const buttonBox = await actionButton.boundingBox() if (buttonBox === null) throw new Error('fork source row action has no layout box') await page.mouse.click(buttonBox.x + buttonBox.width / 2, buttonBox.y + buttonBox.height / 2) diff --git a/apps/web/tests/navigation-panes.e2e.ts b/apps/web/tests/navigation-panes.e2e.ts index d5ecdabc70..e216393423 100644 --- a/apps/web/tests/navigation-panes.e2e.ts +++ b/apps/web/tests/navigation-panes.e2e.ts @@ -14,7 +14,7 @@ import { chromium } from 'playwright' import { strFromU8, unzipSync } from 'fflate' import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, onTestFailed, vi } from 'vitest' import { parseSessionLog } from '@deepseek-ai/dsh-llm-replay' -import type { SessionEvent } from '@deepseek-ai/dsh-session' +import { SESSION_FORMAT_VERSION, type SessionEvent } from '@deepseek-ai/dsh-session' import { assertFixtureInventory, captureStableAria, compareOrRefreshGolden, fixtureUserPrompts, launchWebScaffold, recordFixture, seedSession, watchConsole, webSnapshotMode, type WebScaffold, @@ -28,6 +28,7 @@ const SEARCH_EXPECTED = join(SNAPSHOT_DIR, 'search-results.expected.md') const TERMINAL_EXPECTED = join(SNAPSHOT_DIR, 'terminal-card.expected.md') const MODE = webSnapshotMode() const SEED_ID = 'navigation-panes-web-e2e' +const EXPORTED_LOG_FILE = `session.v${String(SESSION_FORMAT_VERSION)}.jsonl` // Turn 1 leads with a distinctive word: the session-title fallback takes the // first words of the first message, so the sidebar-search scenario has a @@ -309,8 +310,11 @@ describe('web e2e: navigation & panes over a rich seeded session', () => { // The real host streamed the ZIP; its root entry is the persisted log // text verbatim (the assembled seam: real route, real persistence read). const files = unzipSync(await readFile(await download.path())) - expect(Object.keys(files)).toEqual(['session.jsonl']) - const content = strFromU8(files['session.jsonl'] as Uint8Array) + expect(Object.keys(files)).toEqual([EXPORTED_LOG_FILE]) + const content = strFromU8(files[EXPORTED_LOG_FILE] as Uint8Array) + expect(JSON.parse(content.split('\n')[0] ?? '')).toMatchObject({ + type: 'session', version: SESSION_FORMAT_VERSION, + }) expect(content.split('\n')[0]).toContain(SEED_ID) expect(content).toContain('FIRST_DONE') await dialog.getByText('Close', { exact: true }).click() @@ -343,7 +347,11 @@ describe('web e2e: navigation & panes over a rich seeded session', () => { const slashDownload = await slashDownloadPromise expect(slashDownload.suggestedFilename()).toBe(download.suggestedFilename()) const slashFiles = unzipSync(await readFile(await slashDownload.path())) - const slashContent = strFromU8(slashFiles['session.jsonl'] as Uint8Array) + expect(Object.keys(slashFiles)).toEqual([EXPORTED_LOG_FILE]) + const slashContent = strFromU8(slashFiles[EXPORTED_LOG_FILE] as Uint8Array) + expect(JSON.parse(slashContent.split('\n')[0] ?? '')).toMatchObject({ + type: 'session', version: SESSION_FORMAT_VERSION, + }) const slashEvents = parseSessionLog(slashContent) const exportRun = slashEvents.findLast(event => event.type === 'command/run' && event.data.name === 'export') if (exportRun?.type !== 'command/run') throw new Error('slash ZIP has no export command/run') diff --git a/apps/web/tests/produced-file-mentions.e2e.ts b/apps/web/tests/produced-file-mentions.e2e.ts index ca8fafc63e..47b1744d55 100644 --- a/apps/web/tests/produced-file-mentions.e2e.ts +++ b/apps/web/tests/produced-file-mentions.e2e.ts @@ -106,6 +106,7 @@ function mentionFixture(): string { id: '{{sessionId}}', createdAt: 0, cwd: '{{cwd}}', + delegationDepth: 0, }), ...session.snapshotEvents().map(event => JSON.stringify({ ...event, diff --git a/apps/web/tests/produced-files.e2e.ts b/apps/web/tests/produced-files.e2e.ts index 5494a934e3..6da856ea8a 100644 --- a/apps/web/tests/produced-files.e2e.ts +++ b/apps/web/tests/produced-files.e2e.ts @@ -94,7 +94,7 @@ function producedFixture(): string { return [ JSON.stringify({ type: 'session', version: SESSION_FORMAT_VERSION, id: '{{sessionId}}', - createdAt: 0, cwd: '{{cwd}}', + createdAt: 0, cwd: '{{cwd}}', delegationDepth: 0, }), ...session.snapshotEvents().map(event => JSON.stringify({ ...event, time: eventTimeOrigin + event.seq * 1_000, diff --git a/apps/web/tests/question-composer.e2e.ts b/apps/web/tests/question-composer.e2e.ts index 29fbf1e146..d4555b6680 100644 --- a/apps/web/tests/question-composer.e2e.ts +++ b/apps/web/tests/question-composer.e2e.ts @@ -13,11 +13,12 @@ import { join } from 'node:path' import type { Browser, Locator, Page } from 'playwright' import { chromium } from 'playwright' import { afterAll, beforeAll, describe, expect, it, onTestFailed } from 'vitest' -import type { SessionEvent } from '@deepseek-ai/dsh-session' +import { SESSION_FORMAT_VERSION, type SessionEvent } from '@deepseek-ai/dsh-session' import type { SessionId } from '@deepseek-ai/dsh-session/types' import { assertFixtureInventory, captureStableAria, compareOrRefreshGolden, fixtureUserPrompts, - launchWebScaffold, recordFixture, seedSession, watchConsole, webSnapshotMode, type WebScaffold, + launchWebScaffold, recordedSessionFixturePath, recordFixture, seedSession, + watchConsole, webSnapshotMode, type WebScaffold, } from './scaffold.ts' import { connectFreshWorkspace, expandTurnProcesses, newEnglishPage, saveFailureShot, @@ -107,7 +108,6 @@ function cancelledFixture(fixture: string): string { message.content[0].isError = true data.error = { name: 'UserQuestionError', - message: 'the user cancelled ask_user_question', code: 'ASK_CANCELLED', } replaced = true @@ -274,6 +274,11 @@ describe('web e2e: resident question composer round trip', () => { const sessionId = await settled if (MODE === 'record') { await recordFixture(scaffold, sessionId, FIXTURE) + const recorded = await readFile(recordedSessionFixturePath(FIXTURE, SESSION_FORMAT_VERSION), 'utf8') + const header = JSON.parse(recorded.split('\n').find(line => line.trim().length > 0) ?? '{}') as { + cwd?: unknown + } + expect(header.cwd).toBe('{{cwd}}') return } answeredSession = sessionId diff --git a/apps/web/tests/reference-composer.e2e.ts b/apps/web/tests/reference-composer.e2e.ts index 3587272a12..8e53eb82dc 100644 --- a/apps/web/tests/reference-composer.e2e.ts +++ b/apps/web/tests/reference-composer.e2e.ts @@ -4,7 +4,7 @@ import { mkdir, writeFile } from 'node:fs/promises' import { fileURLToPath } from 'node:url' import { join } from 'node:path' -import type { Browser, Page } from 'playwright' +import type { Browser, Locator, Page } from 'playwright' import { chromium } from 'playwright' import { afterAll, beforeAll, describe, expect, it, onTestFailed } from 'vitest' import { createUserMessage } from '@deepseek-ai/dsh-llm' @@ -35,6 +35,16 @@ const MODE = webSnapshotMode() const SOURCE_SESSION_ID = 'reference-source-session' const TARGET_SESSION_ID = 'reference-order-target-session' +async function settledSourceOption(menu: Locator): Promise { + await expect.poll( + () => menu.getByRole('option', { name: new RegExp(TARGET_SESSION_ID) }).count(), + { timeout: 15_000 }, + ).toBe(0) + const source = menu.getByRole('option', { name: new RegExp(SOURCE_SESSION_ID) }) + await expect.poll(() => source.count(), { timeout: 15_000 }).toBe(1) + return source +} + /** Build one closed source session with a stable title for reference discovery. */ function sourceSessionFixture(): string { const session = Session.create(SessionId(SOURCE_SESSION_ID)) @@ -58,6 +68,7 @@ function sourceSessionFixture(): string { id: '{{sessionId}}', createdAt: 0, cwd: '{{cwd}}', + delegationDepth: 0, }), ...session.snapshotEvents().map(event => JSON.stringify(event)), '', @@ -105,6 +116,7 @@ function targetSessionFixture(): string { id: '{{sessionId}}', createdAt: 0, cwd: '{{cwd}}', + delegationDepth: 0, }), ...session.snapshotEvents().map(event => JSON.stringify(event)), '', @@ -151,7 +163,7 @@ describe.skipIf(MODE === 'record')('web e2e: file and session references through const input = page.locator('[data-composer-input]').first() const menu = page.getByRole('listbox', { name: 'Trigger suggestions' }) - await input.fill('@') + await writeComposerDraft(page, input, '@') await expect.poll(() => menu.getByRole('option').count(), { timeout: 15_000 }).toBeGreaterThanOrEqual(2) // Session rows are dated from the live Host list, so their age bucket // advances while the suite runs. @@ -171,7 +183,7 @@ describe.skipIf(MODE === 'record')('web e2e: file and session references through expect(snapshot).not.toContain('Research notes') expect(snapshot).not.toContain('text: Subagents') - await input.fill('@reference') + await writeComposerDraft(page, input, '@reference') // The open menu keeps the previous query's rows while the new one loads // (stale-while-revalidate), and rows are keyed by index, so a click // resolved against a stale row lands on whatever settles into that slot. @@ -187,8 +199,8 @@ describe.skipIf(MODE === 'record')('web e2e: file and session references through await expect.poll(() => fileReference.locator('svg').count()).toBe(1) await expect.poll(() => input.textContent()).toBe('reference.txt ') - await input.fill('@reference-source') - await menu.getByRole('option', { name: new RegExp(SOURCE_SESSION_ID) }).click() + await writeComposerDraft(page, input, '@reference-source') + await (await settledSourceOption(menu)).click() const sessionReference = page.locator('[data-composer-chip]').last() await expect.poll(() => sessionReference.textContent()).toBe(SOURCE_SESSION_ID) await expect.poll(() => sessionReference.locator('svg').count()).toBe(1) @@ -203,7 +215,8 @@ describe.skipIf(MODE === 'record')('web e2e: file and session references through const input = page.locator('[data-composer-input]').first() const menu = page.getByRole('listbox', { name: 'Trigger suggestions' }) - await input.fill('@reference') + await writeComposerDraft(page, input, '@reference') + await expect.poll(() => input.locator('[data-composer-chip]').count()).toBe(0) await menu.getByRole('option', { name: /reference\.txt/ }).click() await expect.poll(() => input.locator('[data-composer-chip]').count()).toBe(1) @@ -213,7 +226,7 @@ describe.skipIf(MODE === 'record')('web e2e: file and session references through await page.keyboard.press('ControlOrMeta+A') await page.keyboard.press('ArrowLeft') await page.keyboard.type('@reference-source') - await menu.getByRole('option', { name: new RegExp(SOURCE_SESSION_ID) }).click() + await (await settledSourceOption(menu)).click() // Both chips survive the boundary insert: the session chip lands ahead of // the intact file chip. @@ -232,7 +245,8 @@ describe.skipIf(MODE === 'record')('web e2e: file and session references through const input = page.locator('[data-composer-input]').first() const menu = page.getByRole('listbox', { name: 'Trigger suggestions' }) - await input.fill('@reference') + await writeComposerDraft(page, input, '@reference') + await expect.poll(() => input.locator('[data-composer-chip]').count()).toBe(0) await menu.getByRole('option', { name: /reference\.txt/ }).click() await expect.poll(() => input.locator('[data-composer-chip]').count()).toBe(1) diff --git a/apps/web/tests/scaffold-generation.spec.ts b/apps/web/tests/scaffold-generation.spec.ts new file mode 100644 index 0000000000..6bc3af9708 --- /dev/null +++ b/apps/web/tests/scaffold-generation.spec.ts @@ -0,0 +1,195 @@ +import { afterEach, describe, expect, it } from 'vitest' +import { mkdtemp, rm, writeFile } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { + assertFixtureInventory, + fixtureIdentity, + normalizeAria, + normalizeWebSessionVolatiles, + realizeSeedFixture, + recordedSessionFixturePath, + selectedSessionFixture, + type WebScaffold, +} from './scaffold.ts' + +const roots: string[] = [] + +afterEach(async () => { + for (const root of roots.splice(0)) await rm(root, { recursive: true, force: true }) +}) + +describe('Web snapshot generation filenames', () => { + it('normalizes the live Session cwd instead of its scaffold parent on Windows', () => { + const scaffoldRoot = 'C:\\Users\\dsh\\AppData\\Local\\Temp\\dsh-web-e2e-ws-test' + const sessionCwd = `${scaffoldRoot}\\workspace` + const log = [ + JSON.stringify({ + type: 'session', version: 1, id: 'stable-session-windows', createdAt: 0, + cwd: sessionCwd, delegationDepth: 0, + }), + JSON.stringify({ + type: 'user/message', + data: { + content: [{ + type: 'text', + text: `System workspace: "${sessionCwd}". Nested file: ${sessionCwd}/notes.txt`, + }], + opaque: { + sibling: `${sessionCwd}-copy`, + parentPrefix: `${scaffoldRoot}ed`, + }, + }, + }), + '', + ].join('\n') + + const [headerLine, eventLine] = normalizeWebSessionVolatiles(log, scaffoldRoot).split('\n') + expect(JSON.parse(headerLine!) as unknown).toMatchObject({ cwd: '{{cwd}}' }) + expect(JSON.parse(eventLine!) as unknown).toMatchObject({ + data: { + content: [{ + text: 'System workspace: "{{cwd}}". Nested file: {{cwd}}/notes.txt', + }], + opaque: { + sibling: `${sessionCwd}-copy`, + parentPrefix: `${scaffoldRoot}ed`, + }, + }, + }) + }) + + it('normalizes native and browser-rendered Windows workspace paths', () => { + const cwd = 'C:\\Users\\runner\\AppData\\Local\\Temp\\dsh-web-e2e-ws-abc' + const escapedCwd = cwd.replaceAll('\\', '\\\\') + expect(normalizeAria([ + `${escapedCwd}\\\\workspace\\\\file.txt`, + `${cwd}\\workspace\\file.txt`, + `${cwd.replaceAll('\\', '/')}/workspace/file.txt`, + 'dsh-web-e2e-ws-abc', + ].join('\n'), cwd, false)).toBe([ + '{{cwd}}\\\\workspace\\\\file.txt', + '{{cwd}}\\workspace\\file.txt', + '{{cwd}}/workspace/file.txt', + '{{workspace}}', + ].join('\n')) + }) + + it('realizes Windows cwd and identity tokens as JSON string values', () => { + const recordedCwd = 'D:\\recorded\\workspace' + const workspaceCwd = 'C:\\Users\\runner\\work\\deepseek-harness' + const fixture = [ + JSON.stringify({ + type: 'session', version: 1, id: '{{session:1}}', createdAt: 0, + cwd: recordedCwd, delegationDepth: 0, + }), + JSON.stringify({ + type: 'event', + data: { + placeholderPath: '{{cwd}}\\placeholder.txt', + recordedPath: `${recordedCwd}\\recorded.txt`, + child: '{{session:2}}', + message: '{{message:1}}', + opaque: '{ "literal": "\\\\b", "keep": true }', + }, + }), + '', + ].join('\n') + const scaffold = { workspaceCwd } as WebScaffold + + const realized = realizeSeedFixture(scaffold, fixture, 'live-session') + const [headerLine, eventLine] = realized.split('\n') + const header = JSON.parse(headerLine!) as { id: string; cwd: string } + const event = JSON.parse(eventLine!) as { data: Record } + + expect(header).toMatchObject({ id: 'live-session', cwd: workspaceCwd }) + expect(event.data).toEqual({ + placeholderPath: `${workspaceCwd}\\placeholder.txt`, + recordedPath: `${workspaceCwd}\\recorded.txt`, + child: 'live-session-child-2', + message: fixtureIdentity('message', 1), + opaque: '{ "literal": "\\\\b", "keep": true }', + }) + expect(realized.endsWith('\n')).toBe(true) + expect(realizeSeedFixture(scaffold, realized, 'live-session')).toBe(realized) + }) + + it('collapses a tokenized recorded cwd before realizing its generic cwd prefix', () => { + const workspaceCwd = 'C:\\Users\\runner\\work\\deepseek-harness' + const recordedCwd = '{{cwd}}\\workspace' + const fixture = [ + JSON.stringify({ + type: 'session', version: 1, id: '{{sessionId}}', createdAt: 0, + cwd: recordedCwd, delegationDepth: 0, + }), + JSON.stringify({ + type: 'event', data: { path: `${recordedCwd}\\nav-a.md` }, + }), + '', + ].join('\n') + + const [headerLine, eventLine] = realizeSeedFixture( + { workspaceCwd } as WebScaffold, + fixture, + 'live-session', + ).split('\n') + + expect(JSON.parse(headerLine!) as unknown).toMatchObject({ cwd: workspaceCwd }) + expect(JSON.parse(eventLine!) as unknown).toMatchObject({ + data: { path: `${workspaceCwd}\\nav-a.md` }, + }) + }) + + it('selects the highest parent and child generations without counting retained inputs twice', async () => { + const root = await mkdtemp(join(tmpdir(), 'dsh-web-fixture-generations-')) + roots.push(root) + for (const name of [ + 'session.jsonl', + 'session.v2.jsonl', + 'session.1.jsonl', + 'session.1.v1.jsonl', + ]) await writeFile(join(root, name), '') + + await expect(selectedSessionFixture(join(root, 'session.jsonl'))) + .resolves.toBe(join(root, 'session.v2.jsonl')) + await expect(selectedSessionFixture(join(root, 'session.1.jsonl'))) + .resolves.toBe(join(root, 'session.1.v1.jsonl')) + await expect(selectedSessionFixture(join(root, 'replay.override.json'))) + .resolves.toBe(join(root, 'replay.override.json')) + }) + + it('leaves an absent override-only parent fixture unresolved', async () => { + const root = await mkdtemp(join(tmpdir(), 'dsh-web-fixture-generations-')) + roots.push(root) + + await expect(selectedSessionFixture(join(root, 'session.jsonl'))) + .resolves.toBe(join(root, 'session.jsonl')) + }) + + it('records beside an older generation and preserves the parent or child role', () => { + const fixtures = join('/', 'fixtures') + expect(recordedSessionFixturePath(join(fixtures, 'session.jsonl'), 1)) + .toBe(join(fixtures, 'session.v1.jsonl')) + expect(recordedSessionFixturePath(join(fixtures, 'session.2.jsonl'), 3)) + .toBe(join(fixtures, 'session.2.v3.jsonl')) + expect(recordedSessionFixturePath(join(fixtures, 'session.v1.jsonl'), 1)) + .toBe(join(fixtures, 'session.v1.jsonl')) + expect(() => recordedSessionFixturePath(join(fixtures, 'notes.jsonl'), 1)) + .toThrow('invalid Session fixture path') + }) + + it('treats retained generations as one exact inventory role', async () => { + const root = await mkdtemp(join(tmpdir(), 'dsh-web-fixture-inventory-')) + roots.push(root) + await writeFile(join(root, 'session.jsonl'), `${JSON.stringify({ + type: 'session', version: 0, id: '{{session:1}}', createdAt: 0, delegationDepth: 0, + })}\n`) + await writeFile(join(root, 'session.v1.jsonl'), `${JSON.stringify({ + type: 'session', version: 1, id: '{{session:1}}', createdAt: 0, delegationDepth: 0, + })}\n`) + await writeFile(join(root, 'ui.expected.md'), 'stable\n') + + await expect(assertFixtureInventory(root, ['session.jsonl', 'ui.expected.md'])) + .resolves.toBeUndefined() + }) +}) diff --git a/apps/web/tests/scaffold.ts b/apps/web/tests/scaffold.ts index 95a7118560..06a1c505a1 100644 --- a/apps/web/tests/scaffold.ts +++ b/apps/web/tests/scaffold.ts @@ -42,10 +42,13 @@ import { normalizedSystemPrompts, normalizedToolSchemas, parseSnapshotManifest, + parseSessionFixtureName, redactSessionSnapshotIds, normalizeSessionSnapshots, scrubRequestHeaders, scrubSessionSnapshot, + sessionFixtureFiles, + sessionFixtureName, stabilizeFixtureMessageIds, type NormalizeContext, } from '@deepseek-ai/dsh-session-snapshot' @@ -62,9 +65,14 @@ import type { LlmModelInfo, LlmProviderInfo, LlmResolvedModelInfo, RetryPolicyConfig, StreamChunk, } from '@deepseek-ai/dsh-llm' import type { ReplayHandle } from '@deepseek-ai/dsh-llm-replay' -import { installLlmReplay, parseSessionLog } from '@deepseek-ai/dsh-llm-replay' +import { + installLlmReplay, + parseSessionLog, + parseSessionLogForReplay, +} from '@deepseek-ai/dsh-llm-replay' +import type { SessionFormatEvent } from '@deepseek-ai/dsh-session-format' +import { sessionFormatCatalog } from '@deepseek-ai/dsh-session-format-catalog' import SessionStore, { - packChunkRuns, SESSION_FORMAT_VERSION, SessionId, SessionSeq, @@ -128,13 +136,44 @@ export async function assertFinalWorkspaceSnapshot(scenarioDir: string, workspac } async function ownsReplayFixture(replayFixture: string | undefined): Promise { - if (replayFixture === undefined || basename(replayFixture) !== 'session.jsonl') return false + if (replayFixture === undefined) return false + const fixture = parseSessionFixtureName(basename(replayFixture)) + if (fixture === undefined || fixture.index !== 0) return false const manifestPath = join(dirname(replayFixture), 'snapshot.yml') if (!existsSync(manifestPath)) return false const manifest = parseSnapshotManifest(await readFile(manifestPath, 'utf8'), manifestPath) return manifest.session === undefined } +/** + * Resolve one requested fixture role to its highest committed generation. + * @param path - any generation path for the requested parent or child role. + * @returns the highest canonical sibling generation, or the input for non-Session files. + */ +export async function selectedSessionFixture(path: string): Promise { + const requested = parseSessionFixtureName(basename(path)) + if (requested === undefined) return path + // An override-only scenario deliberately has no projected parent log. Keep + // the absent source path so the replay adapter can use its replacement + // script without asking the fixture-role inventory to invent a parent. + if (!existsSync(path)) return path + const selected = sessionFixtureFiles(await readdir(dirname(path))) + .find(candidate => candidate.index === requested.index) + return selected === undefined ? path : join(dirname(path), selected.name) +} + +/** + * Return the current-writer target without replacing the requested older fixture. + * @param path - any canonical fixture generation for one role. + * @param version - generation emitted by the current writer. + * @returns the canonical sibling path for that role and generation. + */ +export function recordedSessionFixturePath(path: string, version: number): string { + const fixture = parseSessionFixtureName(basename(path)) + if (fixture === undefined) throw new Error(`record harvest: invalid Session fixture path ${path}`) + return join(dirname(path), sessionFixtureName(fixture.index, version)) +} + /** The shipped composition under test: the dsh-base and dsh-web-app bundle patches over the empty profile root. */ const BASE_PATCH_PATH = join(REPO_ROOT, 'packages/bundle/base/cordis.patch.yml') const WEB_PATCH_PATH = join(REPO_ROOT, 'packages/bundle/web-app/cordis.patch.yml') @@ -364,7 +403,13 @@ async function cleanupScaffoldWorld(ctx: Context, workspaceCwd: string, persiste export async function launchWebScaffold(options: LaunchOptions = {}): Promise { requireDist() const mode = webSnapshotMode() - const compareReplaySession = options.compareReplaySession ?? await ownsReplayFixture(options.replayFixture) + const replayFixture = options.replayFixture === undefined + ? undefined + : await selectedSessionFixture(options.replayFixture) + const replayChildFixtures = options.replayChildFixtures === undefined + ? undefined + : await Promise.all(options.replayChildFixtures.map(selectedSessionFixture)) + const compareReplaySession = options.compareReplaySession ?? await ownsReplayFixture(replayFixture) const browserHost = options.remoteAuthority ?? '127.0.0.1' if (mode === 'record') { // Both owning vitest configs (web unconditionally, snapshot in record @@ -651,10 +696,10 @@ export async function launchWebScaffold(options: LaunchOptions = {}): Promise ( event.type === 'assistant/chunk' || event.type === 'request/header' || event.type === 'tool/call' )) @@ -680,15 +725,15 @@ export async function launchWebScaffold(options: LaunchOptions = {}): Promise ({ ...provider, ...(options.replayRetryPolicy === undefined ? {} : { retryPolicy: options.replayRetryPolicy }), })), ...(options.replayOverride === undefined ? {} : { overrideFile: options.replayOverride }), - ...(options.replayChildFixtures === undefined ? {} : { childFiles: options.replayChildFixtures }), + ...(replayChildFixtures === undefined ? {} : { childFiles: replayChildFixtures }), ...(options.paceMs === undefined ? {} : { paceMs: options.paceMs }), }) } else if (mode !== 'record' && options.deepSeekMissingCredential !== true) { @@ -760,13 +805,13 @@ export async function launchWebScaffold(options: LaunchOptions = {}): Promise { const failures: unknown[] = [] if (mode !== 'record' - && options.replayFixture !== undefined + && replayFixture !== undefined && options.replayProvidersOnly !== true && compareReplaySession) { try { await assertReplaySession( [...observedSessions.values()], - options.replayFixture, + replayFixture, mode, `http://${browserHost}:${port}`, ) @@ -802,39 +847,99 @@ export async function launchWebScaffold(options: LaunchOptions = {}): Promise JSON.stringify(record)), + JSON.stringify(encoded.header), + ...encoded.rows.map(record => JSON.stringify(record)), '', ].join('\n') } -function normalizeWebSessionVolatiles(log: string): string { - const normalizeValue = (value: unknown): unknown => { - if (typeof value === 'string') { - return value.replace(/Anonymous user: [^.]+(?=\. Session sharing)/g, 'Anonymous user: {{anonymousUserId}}') - } - if (Array.isArray(value)) return value.map(normalizeValue) - if (value !== null && typeof value === 'object') { - return Object.fromEntries(Object.entries(value).map(([key, item]) => [key, normalizeValue(item)])) - } - return value +function mapJsonStringValues(value: unknown, map: (value: string) => string): unknown { + if (typeof value === 'string') return map(value) + if (Array.isArray(value)) return value.map(item => mapJsonStringValues(item, map)) + if (value !== null && typeof value === 'object') { + return Object.fromEntries(Object.entries(value).map(([key, item]) => [ + key, + mapJsonStringValues(item, map), + ])) } + return value +} + +const WEB_PATH_TEXT_BOUNDARY_RE = /[\s<>'"`()\[\]{},;:!?=]/ +const WEB_FILE_URI_PATH_PREFIX_RE = /(?:^|[^a-z0-9+.-])file:\/\/\/?$/i + +function isWebCwdMatch(value: string, start: number, length: number): boolean { + const before = value[start - 1] + const after = value[start + length] + const afterPunctuation = value[start + length + 1] + const startsAtBoundary = before === undefined + || WEB_PATH_TEXT_BOUNDARY_RE.test(before) + || WEB_FILE_URI_PATH_PREFIX_RE.test(value.slice(0, start)) + const endsAtBoundary = after === undefined + || after === '/' + || after === '\\' + || WEB_PATH_TEXT_BOUNDARY_RE.test(after) + || after === '.' && (afterPunctuation === undefined || WEB_PATH_TEXT_BOUNDARY_RE.test(afterPunctuation)) + return startsAtBoundary && endsAtBoundary +} + +function replaceWebCwd(value: string, cwd: string): string { + let cursor = 0 + let normalized = '' + while (cursor < value.length) { + const match = value.indexOf(cwd, cursor) + if (match < 0) return normalized + value.slice(cursor) + const end = match + cwd.length + if (isWebCwdMatch(value, match, cwd.length)) { + normalized += value.slice(cursor, match) + '{{cwd}}' + cursor = end + } else { + normalized += value.slice(cursor, end) + cursor = end + } + } + return normalized +} + +/** + * Normalize Web-only volatile strings while preserving JSON structure and row framing. + * @param log - Raw Session JSONL. + * @param workspaceCwd - Optional scaffold parent used before a live Session selects its cwd. + * @returns Compact JSONL with run-local strings tokenized. + */ +export function normalizeWebSessionVolatiles(log: string, workspaceCwd?: string): string { + const headerLine = log.split(/\r?\n/).find(line => line.trim().length > 0) + const header = headerLine === undefined ? undefined : JSON.parse(headerLine) as { cwd?: unknown } + const sessionCwd = typeof header?.cwd === 'string' && header.cwd.length > 0 ? header.cwd : undefined + const cwdSpellings = [...new Set([sessionCwd ?? workspaceCwd] + .filter((value): value is string => typeof value === 'string' && value.length > 0) + .flatMap((value) => { + const forward = value.replaceAll('\\', '/') + const native = /^[A-Za-z]:[\\/]/.test(value) ? forward.replaceAll('/', '\\') : value + return [value, value.replaceAll('\\', '\\\\'), forward, native] + }))].sort((left, right) => right.length - left.length) return log.split(/\r?\n/).map((line) => { if (line.trim() === '') return line - const record = normalizeValue(JSON.parse(line)) as { type?: unknown; data?: { endpoint?: unknown } } + const record = mapJsonStringValues(JSON.parse(line), (value) => { + let normalized = value + .replace(/Anonymous user: [^.]+(?=\. Session sharing)/g, 'Anonymous user: {{anonymousUserId}}') + for (const cwd of cwdSpellings) normalized = replaceWebCwd(normalized, cwd) + return normalized + }) as { + type?: unknown + data?: { endpoint?: unknown } + } if (record.type === 'web/deepseek-search-llm-request' && typeof record.data?.endpoint === 'string') { record.data.endpoint = '{{webSearchEndpoint}}' } @@ -843,9 +948,8 @@ function normalizeWebSessionVolatiles(log: string): string { } function stableSessionFixture(session: Session, existing: string, workspaceCwd: string): string { - const fresh = scrubSessionSnapshot(normalizeWebSessionVolatiles(rawSessionLog(session))) + const fresh = scrubSessionSnapshot(normalizeWebSessionVolatiles(rawSessionLog(session), workspaceCwd)) .split(session.id).join('{{sessionId}}') - .split(workspaceCwd).join('{{cwd}}') const stable = redactSessionSnapshotIds(stabilizeFixtureMessageIds([fresh], [existing]))[0] if (stable === undefined) throw new Error('session harvest produced no stabilized fixture') return stable @@ -858,7 +962,7 @@ async function assertReplaySession( webUrl: string, ): Promise { let expected = await readFile(fixturePath, 'utf8') - const userPrompts = fixtureUserPrompts(expected) + const userPrompts = fixtureUserPrompts(expected, fixturePath) const candidates = sessions.filter((session) => { if (session.header.parentSession !== undefined) return false const actual = session.snapshotEvents().flatMap((event) => { @@ -887,7 +991,9 @@ async function assertReplaySession( cwd: typeof expectedHeader.cwd === 'string' ? expectedHeader.cwd : '\0no-cwd\0', } expect(normalizeSessionSnapshots([normalizeWebSessionVolatiles(actual)], actualContext)[0], `${fixturePath}: persisted replay`) - .toBe(normalizeSessionSnapshots([normalizeWebSessionVolatiles(expected)], expectedContext)[0]) + .toBe(normalizeSessionSnapshots([normalizeWebSessionVolatiles(expected)], expectedContext, { + sourcePaths: [fixturePath], + })[0]) const fixtureDir = dirname(fixturePath) const manifestPath = join(fixtureDir, 'snapshot.yml') @@ -915,23 +1021,29 @@ async function assertReplaySession( * identities with typed relationship-preserving tokens, and write the fixture. * @param scaffold - the record-mode scaffold. * @param sessionId - the driven session. - * @param fixturePath - the committed session.jsonl target. + * @param fixturePath - any committed generation for the fixture role. */ export async function recordFixture(scaffold: WebScaffold, sessionId: SessionId, fixturePath: string): Promise { const agent = scaffold.ctx.agents.get(sessionId) if (agent === undefined) throw new Error(`record harvest: no live agent for ${sessionId}`) - const existing = existsSync(fixturePath) ? await readFile(fixturePath, 'utf8') : '' - await writeFile(fixturePath, stableSessionFixture(agent.session, existing, scaffold.workspaceCwd)) + const target = recordedSessionFixturePath(fixturePath, agent.session.header.version) + const existingPath = existsSync(target) ? target : fixturePath + const existing = existsSync(existingPath) ? await readFile(existingPath, 'utf8') : '' + await writeFile(target, stableSessionFixture(agent.session, existing, scaffold.workspaceCwd)) } /** * The user prompts recorded in a fixture, in order — the single source tying * spec drive steps to recorded reality so script and fixture cannot drift. * @param fixtureText - raw session.jsonl contents. + * @param fixturePath - exact source path for the closed replay-only refusal policy. * @returns the recorded user prompt texts. */ -export function fixtureUserPrompts(fixtureText: string): string[] { - return parseSessionLog(fixtureText).flatMap((event) => { +export function fixtureUserPrompts(fixtureText: string, fixturePath?: string): string[] { + const events = fixturePath === undefined + ? parseSessionLog(fixtureText) + : parseSessionLogForReplay(fixtureText, fixturePath) + return events.flatMap((event) => { if (event.type !== 'user/message' || event.data.source.kind !== 'user') return [] const text = event.data.content.filter(block => block.type === 'text').map(block => block.text).join('') return text.length > 0 ? [text] : [] @@ -949,23 +1061,6 @@ export function fixtureIdentity( return `${hex.slice(0, 8).join('')}-${hex.slice(8, 12).join('')}-${hex.slice(12, 16).join('')}-${hex.slice(16, 20).join('')}-${hex.slice(20).join('')}` } -/** - * Seed a recorded session fixture into the scaffold's persistence root - * through the REAL backend API (throwaway Context + SessionStore + JSONL - * plugin — the semantic-checkpoint precedent), never raw file writes: no - * knowledge of bucket hashing, filename encoding, or compression, and - * malformed session events fail loud at seed time. The fixture's tokenized identity - * ({{sessionId}}/{{cwd}}) is realized for this world before parsing. Event - * times are materialized from event order against the fixture header's - * creation time, or the seeded creation time when normalization replaced the - * header value with zero. - * @param scaffold - the target scaffold. - * @param fixtureText - raw recorded session.jsonl contents. - * @param id - the seeded session id (stable for deterministic goldens). - * @param agentPreset - the preset the recorded session was composed from, - * for scenarios asserting what a resumed session reports running. - * @returns the seeded id. - */ /** * Realize a recorded seed fixture against one scaffold: substitute the * `{{sessionId}}`/`{{cwd}}` placeholders and rewrite the recorded cwd to the @@ -978,25 +1073,35 @@ export function fixtureIdentity( * @returns the realized fixture text. */ export function realizeSeedFixture(scaffold: WebScaffold, fixtureText: string, id: string): string { - const realized = fixtureText - .split('{{sessionId}}').join(id) - .split('{{session:1}}').join(id) - .replace(/\{\{session:([2-9]\d*)\}\}/g, (_token, ordinal: string) => `${id}-child-${ordinal}`) - .replace(/\{\{(message|approval|workflow|command|rpc|retry|id):([1-9]\d*)\}\}/g, (_token, kind: string, ordinal: string) => - fixtureIdentity(kind as 'message' | 'approval' | 'workflow' | 'command' | 'rpc' | 'retry' | 'id', Number(ordinal))) - .split('{{cwd}}').join(scaffold.workspaceCwd) - const fixtureCwd = (JSON.parse(realized.split('\n', 1)[0]!) as { cwd?: string }).cwd - return fixtureCwd === undefined - ? realized - : realized.split(fixtureCwd).join(scaffold.workspaceCwd) + const headerLine = fixtureText.split(/\r?\n/).find(line => line.trim().length > 0) + if (headerLine === undefined) return fixtureText + const fixtureCwd = (JSON.parse(headerLine) as { cwd?: unknown }).cwd + return fixtureText.split(/\r?\n/).map((line) => { + if (line.trim() === '') return line + const realized = mapJsonStringValues(JSON.parse(line), (value) => { + let result = typeof fixtureCwd === 'string' + ? value.split(fixtureCwd).join(scaffold.workspaceCwd) + : value + result = result + .split('{{sessionId}}').join(id) + .split('{{session:1}}').join(id) + .replace(/\{\{session:([2-9]\d*)\}\}/g, (_token, ordinal: string) => `${id}-child-${ordinal}`) + .replace(/\{\{(message|approval|workflow|command|rpc|retry|id):([1-9]\d*)\}\}/g, (_token, kind: string, ordinal: string) => + fixtureIdentity(kind as 'message' | 'approval' | 'workflow' | 'command' | 'rpc' | 'retry' | 'id', Number(ordinal))) + .split('{{cwd}}').join(scaffold.workspaceCwd) + return result + }) + return JSON.stringify(realized) + }).join('\n') } /** * Parse a committed web seed fixture through the replay reader. * @param fixtureText - session JSONL fixture contents. + * @param fixturePath - exact source path for the closed replay-only refusal policy. * @returns the original header line, parsed header, and logical events. */ -export function parseSeedFixture(fixtureText: string): { +export function parseSeedFixture(fixtureText: string, fixturePath?: string): { headerLine: string header: Record events: SessionEvent[] @@ -1005,7 +1110,10 @@ export function parseSeedFixture(fixtureText: string): { if (headerLine === undefined) throw new Error('seed fixture has no session header') const header = JSON.parse(headerLine) as Record if (header.type !== 'session') throw new Error('seed fixture must start with a session header') - return { headerLine, header, events: parseSessionLog(fixtureText) } + const events = fixturePath === undefined + ? parseSessionLog(fixtureText) + : parseSessionLogForReplay(fixtureText, fixturePath) + return { headerLine, header, events } } /** @@ -1025,13 +1133,25 @@ export function renderSeedFixture( ].join('\n') } +/** + * Seed a recorded session fixture into the scaffold's persistence root + * through the real Session and JSONL APIs. The source identity is consulted + * only for the two closed replay-only alpha refusals. + * @param scaffold - the target scaffold. + * @param fixtureText - raw recorded session.jsonl contents. + * @param id - the seeded session id. + * @param agentPreset - preset recorded by scenarios that assert resumed composition. + * @param fixturePath - exact source path for replay-only refusal policy. + * @returns the seeded id. + */ export async function seedSession( scaffold: WebScaffold, fixtureText: string, id: string, agentPreset?: string, + fixturePath?: string, ): Promise { - const decoded = parseSeedFixture(realizeSeedFixture(scaffold, fixtureText, id)) + const decoded = parseSeedFixture(realizeSeedFixture(scaffold, fixtureText, id), fixturePath) const events = decoded.events if (events.length === 0) throw new Error('seed fixture has no events') const last = events[events.length - 1]! @@ -1122,12 +1242,18 @@ async function persistSeedSession( const ARIA_AGE = /(?:now|\d+min|\d+h|\d+d|\d+mo|\d+y|刚刚|\d+分钟|\d+小时|\d+天|\d+个月|\d+年)(?=")/g -function normalizeAria(snapshot: string, workspaceCwd: string, age: boolean): string { +export function normalizeAria(snapshot: string, workspaceCwd: string, age: boolean): string { // The session heading renders the workspace's basename, not the full - // path, so both spellings must collapse to the token. - const base = workspaceCwd.split('/').pop()! + // path. Browser-visible paths may use URL slashes even when the Host cwd is + // native Windows, while serialized text doubles each backslash. Normalize + // every complete spelling longest-first before the basename. + const escapedCwd = workspaceCwd.replaceAll('\\', '\\\\') + const slashCwd = workspaceCwd.replaceAll('\\', '/') + const base = basename(slashCwd) return (age ? snapshot.replace(ARIA_AGE, '{{age}}') : snapshot) + .split(escapedCwd).join('{{cwd}}') .split(workspaceCwd).join('{{cwd}}') + .split(slashCwd).join('{{cwd}}') .split(base).join('{{workspace}}') .replace(/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/gi, '{{uuid}}') // The optional space in `\d+m ?\d+s` covers both minute spellings: the @@ -1259,15 +1385,19 @@ export async function assertFixtureInventory(dir: string, expected: string[]): P const entries = (await readdir(dir)).sort() const ownsManifest = entries.includes('snapshot.yml') const artifacts = entries.filter(name => name !== 'snapshot.yml') - expect(artifacts).toEqual([...expected].sort()) + const roleInventory = (names: readonly string[]): string[] => [...new Set(names.map((name) => { + const fixture = parseSessionFixtureName(name) + return fixture === undefined ? name : sessionFixtureName(fixture.index, 0) + }))].sort() + expect(roleInventory(artifacts)).toEqual(roleInventory(expected)) if (ownsManifest) { const manifestPath = join(dir, 'snapshot.yml') const manifest = parseSnapshotManifest(await readFile(manifestPath, 'utf8'), manifestPath) expect(manifest.profile).toBe('web') if (manifest.session === undefined) { expect( - artifacts.includes('session.jsonl'), - `${dir}: session owner must carry session.jsonl`, + artifacts.some(name => parseSessionFixtureName(name)?.index === 0), + `${dir}: session owner must carry a canonical parent Session fixture`, ).toBe(true) } else { expect(existsSync(resolve(dir, manifest.session.source)), `${dir}: session source`).toBe(true) diff --git a/apps/web/tests/schedule-after.e2e.ts b/apps/web/tests/schedule-after.e2e.ts index 28c3425941..8d560114e5 100644 --- a/apps/web/tests/schedule-after.e2e.ts +++ b/apps/web/tests/schedule-after.e2e.ts @@ -610,7 +610,7 @@ describe.skipIf(MODE === 'record')('web e2e: active Schedule catalog', () => { replayFixture: CATALOG_FIXTURE, replayProvidersOnly: true, }) - await seedSession(scaffold, fixture, CATALOG_SESSION_ID, 'standard') + await seedSession(scaffold, fixture, CATALOG_SESSION_ID, 'standard', CATALOG_FIXTURE) const workspace = await scaffold.ctx.workspaceRegistry.create(scaffold.workspaceCwd) await workspace.attachSession(CATALOG_SESSION_ID) diff --git a/apps/web/tests/workspace-management.e2e.ts b/apps/web/tests/workspace-management.e2e.ts index a8345c45d3..885cfe3e64 100644 --- a/apps/web/tests/workspace-management.e2e.ts +++ b/apps/web/tests/workspace-management.e2e.ts @@ -18,6 +18,7 @@ import type { Browser, Locator, Page } from 'playwright' import { chromium } from 'playwright' import { afterAll, beforeAll, describe, expect, it, onTestFailed } from 'vitest' import { SessionId } from '@deepseek-ai/dsh-session' +import { isReadableSessionPersistenceListing } from '@deepseek-ai/dsh-session-persistence' import { acknowledgeReloadConnectionLoss, assertFixtureInventory, captureStableAria, compareOrRefreshGolden, launchWebScaffold, seedSession, watchConsole, webSnapshotMode, type WebScaffold, @@ -211,10 +212,11 @@ describe('web e2e: workspace management (create / rename / flat view / hover aff const workspace = await scaffold.ctx.workspaceRegistry.resolveByPath(scaffold.workspaceCwd) if (workspace === undefined) throw new Error('GUI did not register the existing project directory') await workspace.attachSession(SessionId(SEED_ID)) - const header = (await scaffold.ctx.sessionPersistence.list()) - .find(candidate => candidate.id === SEED_ID) - if (header === undefined) throw new Error('seeded Session log disappeared before deletion') - const logLocation = scaffold.ctx.sessionPersistence.locate(header) + const listing = (await scaffold.ctx.sessionPersistence.list()) + .filter(isReadableSessionPersistenceListing) + .find(candidate => candidate.header.id === SEED_ID) + if (listing === undefined) throw new Error('seeded Session log disappeared before deletion') + const logLocation = scaffold.ctx.sessionPersistence.locate(listing.header) if (logLocation === undefined) throw new Error('JSONL persistence did not expose the seeded log path') expect(await readFile(join(scaffold.workspaceCwd, 'workspace', 'a.txt'), 'utf8')).toBe('alpha\n') await stat(logLocation.path) @@ -583,7 +585,9 @@ describe('web e2e: workspace management (create / rename / flat view / hover aff // Durable on the host: the registry-global set carries the id while the // session log itself stays in persistence untouched. expect([...scaffold.ctx.workspaceRegistry.archivedSessionIds]).toEqual([SessionId(SEED_ID)]) - expect((await scaffold.ctx.sessionPersistence.list()).map(header => header.id)).toContain(SessionId(SEED_ID)) + expect((await scaffold.ctx.sessionPersistence.list()) + .filter(isReadableSessionPersistenceListing) + .map(candidate => candidate.header.id)).toContain(SessionId(SEED_ID)) // Reload: the hidden state is rebuilt from the workspace.list baseline. const warningStart = tripwire.warnings.length await page.reload({ waitUntil: 'load' }) diff --git a/apps/web/tsconfig.json b/apps/web/tsconfig.json index 23760125d3..38a844cbbe 100644 --- a/apps/web/tsconfig.json +++ b/apps/web/tsconfig.json @@ -23,6 +23,7 @@ // cannot see both sides of the cordis Context merges). "exclude": [ "tests/scaffold.ts", + "tests/scaffold-generation.spec.ts", "tests/scaffold-hermetic.e2e.ts", "tests/startup-rpc-budget.e2e.ts", "tests/minimal-preset.snapshot.ts", diff --git a/docs/architecture.i18n.yaml b/docs/architecture.i18n.yaml index 9516e43c8a..35a4abb12c 100644 --- a/docs/architecture.i18n.yaml +++ b/docs/architecture.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write docs/architecture.md -architecture.md: ea1719b68e1109ca5446699a59ebb503f7f989e2 -architecture.zh.md: 3e7bbe7d51559d91858ba1062656c36bb390ec96 +architecture.md: 913567b76f74c2d964ea97da4720069034ed4a87 +architecture.zh.md: a3ac632b7811fbb03d6859a9fcfbe0ca4ba03744 diff --git a/docs/architecture.md b/docs/architecture.md index ea1719b68e..913567b76f 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -104,6 +104,8 @@ Details: the [sequence diagram](agent-lifecycle.md), the [tool pipeline](tool-ex The session log is the source of the context the model sees. `deriveMessages()` projects model history from it, and raw `assistant/chunk` events preserve replay and UI fidelity. Fork, resume, transcripts, telemetry, and persistence all derive from this stream. +Session consumers know only the current logical format. Header-only listing rescans each Session directory and classifies its numerically highest canonical generation without loading events. A cold body read selects that same highest generation, refuses a future version, or composes the static adjacent migration chain in memory, validates and repairs the final result, and exclusively publishes only that version-named successor beside the unchanged source; an already-validated current generation takes the fused no-write path and is cached for later same-process opens. JSONL v0 uses `session.jsonl[.zstd]`, v1 and later use lowercase `session.vN.jsonl[.zstd]`, and committed generation paths are never renamed, replaced, or deleted. The JSONL provider owns physical framing, compression, generation selection, and exclusive publication, while each adjacent migration package owns exactly one `vN -> vN+1` step ([decision](../.agents/notes/implemented/architecture/2026-08-31-released-session-format-migrations.md)). + **Model-visible means logged.** Anything that reaches a model request must be reconstructable from the log, and a runtime invariant asserts it. This is why a new model-visible input requires a new session event: extend `SessionEventMap` and render from the log. **Projection seam.** `dsh-session-projection` owns `ctx.sessionProjections`: registered units fold committed events incrementally, host consumers read one typed state with `stateOf()`, and carriers batch cropped client views with `snapshot()`. A host reader either requires this service during activation or fails explicitly when the registry or required key is absent. Contributors may retain `ctx.inject(['sessionProjections'], ...)` registration without silently defaulting a missing host value. The agent loop registers shared `turnBoundary` state for its readers ([decision](../.agents/notes/implemented/architecture/2026-08-19-session-projection-mandatory-seam.md)). diff --git a/docs/architecture.zh.md b/docs/architecture.zh.md index 3e7bbe7d51..a3ac632b78 100644 --- a/docs/architecture.zh.md +++ b/docs/architecture.zh.md @@ -108,6 +108,8 @@ turn/end 会话日志是模型所见上下文的来源。`deriveMessages()` 从中投影出模型历史，原始 `assistant/chunk` 事件则保证回放和 UI 保真。fork、恢复、transcript（文本记录）、遥测和持久化都派生自该事件流。 +Session 消费方只了解当前逻辑格式。仅 header 的列表会重新扫描每个 Session 目录，在不加载事件的情况下分类数值最高的规范 generation。冷正文读取选择同一个最高 generation，并拒绝未来版本；对于受支持的历史版本，它会在内存中组合静态相邻迁移链，校验并修复最终结果，再以不覆盖方式只发布该具名版本的后继文件，保持源文件不变。已经校验的当前 generation 采用融合的无写入路径，并缓存给同一进程的后续打开。JSONL v0 使用 `session.jsonl[.zstd]`，v1 及后续版本使用小写 `session.vN.jsonl[.zstd]`；已提交 generation 路径绝不重命名、替换或删除。JSONL provider 负责物理 framing、压缩、generation 选择与排他发布，每个相邻迁移包只负责一个 `vN -> vN+1` 步骤（[决策](../.agents/notes/implemented/architecture/2026-08-31-released-session-format-migrations.zh.md)）。 + **模型可见即已记录。** 抵达模型请求的一切都必须能从日志重建，并由一项运行时不变量断言这一点。因此，新增一项模型可见输入就需要新增一个会话事件：扩展 `SessionEventMap` 并从日志渲染。 **投影 seam。** `dsh-session-projection` 提供 `ctx.sessionProjections`：已注册单元增量折叠已提交事件，host 消费方通过 `stateOf()` 读取单个类型化状态，载体通过 `snapshot()` 批量取得裁剪后的客户端视图。host 读取方要么在激活时要求该服务，要么在注册表或必需 key 缺席时明确失败。贡献方可以保留 `ctx.inject(['sessionProjections'], ...)` 注册，但不能为缺失的 host 值静默提供默认值。agent loop 为读取方注册共享的 `turnBoundary` 状态（[决策](../.agents/notes/implemented/architecture/2026-08-19-session-projection-mandatory-seam.zh.md)）。 diff --git a/docs/config-catalog.i18n.yaml b/docs/config-catalog.i18n.yaml index 19563b4b27..8ad05e1f69 100644 --- a/docs/config-catalog.i18n.yaml +++ b/docs/config-catalog.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write docs/config-catalog.md -config-catalog.md: ec62912c76ad336a7bd7e966be7921c9938a211c -config-catalog.zh.md: e785cacb15652e4162066cbdae478870cf61bc4e +config-catalog.md: 7e522b215e748b393b0466b32982fa69a1d56c6e +config-catalog.zh.md: ccaca4ce39a0124b85535b1e6fcfeb53e67dff14 diff --git a/docs/config-catalog.md b/docs/config-catalog.md index ec62912c76..7e522b215e 100644 --- a/docs/config-catalog.md +++ b/docs/config-catalog.md @@ -111,7 +111,7 @@ export interface Config { Depends on: [`AgentOptions`](subsystems/core.md) · [`SessionId`](subsystems/core.md) -Source: [`packages/core/agent-loop/src/index.ts:311`](../packages/core/agent-loop/src/index.ts) +Source: [`packages/core/agent-loop/src/index.ts:312`](../packages/core/agent-loop/src/index.ts) @@ -1355,7 +1355,7 @@ export interface ReplayModelConfig { Depends on: [`ModelModality`](../packages/llm/llm/src/index.ts) · [`RetryPolicyConfig`](../packages/llm/llm/src/index.ts) -Source: [`packages/test-support/llm-replay/src/index.ts:924`](../packages/test-support/llm-replay/src/index.ts) +Source: [`packages/test-support/llm-replay/src/index.ts:1185`](../packages/test-support/llm-replay/src/index.ts) @@ -1499,7 +1499,7 @@ export interface Config { } ``` -Source: [`packages/feedback/message-feedback/src/index.ts:50`](../packages/feedback/message-feedback/src/index.ts) +Source: [`packages/feedback/message-feedback/src/index.ts:51`](../packages/feedback/message-feedback/src/index.ts) @@ -1858,7 +1858,7 @@ export interface Config { export type JsonlCompression = 'zstd' | 'none' ``` -Source: [`packages/session/session-persistence-jsonl/src/index.ts:70`](../packages/session/session-persistence-jsonl/src/index.ts) +Source: [`packages/session/session-persistence-jsonl/src/index.ts:89`](../packages/session/session-persistence-jsonl/src/index.ts) @@ -1882,7 +1882,7 @@ export interface Config { } ``` -Source: [`packages/session/session-projection-cache/src/index.ts:55`](../packages/session/session-projection-cache/src/index.ts) +Source: [`packages/session/session-projection-cache/src/index.ts:54`](../packages/session/session-projection-cache/src/index.ts) @@ -3466,6 +3466,9 @@ Imported as libraries by other packages; a `cordis.yml` cannot load them. - `@deepseek-ai/dsh-sdk-client` ([`packages/sdk/client/src/index.ts`](../packages/sdk/client/src/index.ts)) - `@deepseek-ai/dsh-sdk-minimal` ([`packages/bundle/sdk-minimal/src/index.ts`](../packages/bundle/sdk-minimal/src/index.ts)) - `@deepseek-ai/dsh-sdk-protocol` ([`packages/sdk/protocol/src/index.ts`](../packages/sdk/protocol/src/index.ts)) +- `@deepseek-ai/dsh-session-format` ([`packages/session/session-format/src/index.ts`](../packages/session/session-format/src/index.ts)) +- `@deepseek-ai/dsh-session-format-catalog` ([`packages/session/session-format-catalog/src/index.ts`](../packages/session/session-format-catalog/src/index.ts)) +- `@deepseek-ai/dsh-session-format-v0-to-v1` ([`packages/session/session-format-v0-to-v1/src/index.ts`](../packages/session/session-format-v0-to-v1/src/index.ts)) - `@deepseek-ai/dsh-session-snapshot` ([`packages/test-support/session-snapshot/src/index.ts`](../packages/test-support/session-snapshot/src/index.ts)) - `@deepseek-ai/dsh-session-telemetry` ([`packages/session/session-telemetry/src/index.ts`](../packages/session/session-telemetry/src/index.ts)) - `@deepseek-ai/dsh-session-title-llm` ([`packages/session/session-title-llm/src/index.ts`](../packages/session/session-title-llm/src/index.ts)) diff --git a/docs/config-catalog.zh.md b/docs/config-catalog.zh.md index e785cacb15..ccaca4ce39 100644 --- a/docs/config-catalog.zh.md +++ b/docs/config-catalog.zh.md @@ -113,7 +113,7 @@ export interface Config { 依赖：[`AgentOptions`](subsystems/core.zh.md) · [`SessionId`](subsystems/core.zh.md) -来源：[`packages/core/agent-loop/src/index.ts:311`](../packages/core/agent-loop/src/index.ts) +来源：[`packages/core/agent-loop/src/index.ts:312`](../packages/core/agent-loop/src/index.ts) @@ -1357,7 +1357,7 @@ export interface ReplayModelConfig { 依赖：[`ModelModality`](../packages/llm/llm/src/index.ts) · [`RetryPolicyConfig`](../packages/llm/llm/src/index.ts) -来源：[`packages/test-support/llm-replay/src/index.ts:924`](../packages/test-support/llm-replay/src/index.ts) +来源：[`packages/test-support/llm-replay/src/index.ts:1082`](../packages/test-support/llm-replay/src/index.ts) @@ -1501,7 +1501,7 @@ export interface Config { } ``` -来源：[`packages/feedback/message-feedback/src/index.ts:50`](../packages/feedback/message-feedback/src/index.ts) +来源：[`packages/feedback/message-feedback/src/index.ts:51`](../packages/feedback/message-feedback/src/index.ts) @@ -1860,7 +1860,7 @@ export interface Config { export type JsonlCompression = 'zstd' | 'none' ``` -来源：[`packages/session/session-persistence-jsonl/src/index.ts:70`](../packages/session/session-persistence-jsonl/src/index.ts) +来源：[`packages/session/session-persistence-jsonl/src/index.ts:88`](../packages/session/session-persistence-jsonl/src/index.ts) @@ -1930,7 +1930,7 @@ export type JournalMode = 'wal' | 'delete' | 'truncate' | 'persist' 依赖：[`SessionQueryConfig`](../packages/session-query/session-query/src/index.ts) -来源：[`packages/session-query/session-query-sqlite/src/index.ts:96`](../packages/session-query/session-query-sqlite/src/index.ts) +来源：[`packages/session-query/session-query-sqlite/src/index.ts:97`](../packages/session-query/session-query-sqlite/src/index.ts) @@ -3467,6 +3467,9 @@ export interface Config { - `@deepseek-ai/dsh-sdk-client`（[`packages/sdk/client/src/index.ts`](../packages/sdk/client/src/index.ts)） - `@deepseek-ai/dsh-sdk-minimal`（[`packages/bundle/sdk-minimal/src/index.ts`](../packages/bundle/sdk-minimal/src/index.ts)） - `@deepseek-ai/dsh-sdk-protocol`（[`packages/sdk/protocol/src/index.ts`](../packages/sdk/protocol/src/index.ts)） +- `@deepseek-ai/dsh-session-format`（[`packages/session/session-format/src/index.ts`](../packages/session/session-format/src/index.ts)） +- `@deepseek-ai/dsh-session-format-catalog`（[`packages/session/session-format-catalog/src/index.ts`](../packages/session/session-format-catalog/src/index.ts)） +- `@deepseek-ai/dsh-session-format-v0-to-v1`（[`packages/session/session-format-v0-to-v1/src/index.ts`](../packages/session/session-format-v0-to-v1/src/index.ts)） - `@deepseek-ai/dsh-session-snapshot`（[`packages/test-support/session-snapshot/src/index.ts`](../packages/test-support/session-snapshot/src/index.ts)） - `@deepseek-ai/dsh-session-telemetry`（[`packages/session/session-telemetry/src/index.ts`](../packages/session/session-telemetry/src/index.ts)） - `@deepseek-ai/dsh-session-title-llm`（[`packages/session/session-title-llm/src/index.ts`](../packages/session/session-title-llm/src/index.ts)） diff --git a/docs/deepseek-llm-api-wire-extensions.i18n.yaml b/docs/deepseek-llm-api-wire-extensions.i18n.yaml index 96013a27ad..92b49b9949 100644 --- a/docs/deepseek-llm-api-wire-extensions.i18n.yaml +++ b/docs/deepseek-llm-api-wire-extensions.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write docs/deepseek-llm-api-wire-extensions.md -deepseek-llm-api-wire-extensions.md: fd42609693ac6fbf91dd82b6e73b2d1d06e65a54 -deepseek-llm-api-wire-extensions.zh.md: 61af718841c8778e8943a1a1c16621a9a6618add +deepseek-llm-api-wire-extensions.md: ba4573a73a289fc8af8ecf120a44c802476593ae +deepseek-llm-api-wire-extensions.zh.md: 6d0a444796215eb723fad49d155e39f323ccd494 diff --git a/docs/deepseek-llm-api-wire-extensions.md b/docs/deepseek-llm-api-wire-extensions.md index fd42609693..ba4573a73a 100644 --- a/docs/deepseek-llm-api-wire-extensions.md +++ b/docs/deepseek-llm-api-wire-extensions.md @@ -79,8 +79,9 @@ An enabled inventory with no qualifying entries sends `packages: []`; disabling { "dsh_session_log": { "version": 1, + "sessionFormatVersion": 1, "session": { - "version": 0, + "version": 1, "id": "session-id", "createdAt": 1780000000000 }, @@ -103,20 +104,21 @@ An enabled inventory with no qualifying entries sends `packages: []`; disabling | Member | Type | Meaning | |---|---|---| | `version` | `1` | Schema version for `dsh_session_log` | -| `session` | object | Immutable canonical `SessionHeader` | +| `sessionFormatVersion` | non-negative integer | Session format generation represented by this suffix | +| `session` | object | Immutable wire projection of the current Session header and inherited cut | | `afterSeq` | integer | Greatest sequence recorded as accepted before this request, or `-1` | | `throughSeq` | non-negative integer | Greatest sequence represented by this request | | `events` | array | Contiguous events from `afterSeq + 1` through `throughSeq` | The first upload uses `afterSeq: -1` and carries the complete current log. Each later upload starts after the greatest accepted watermark for the same Session id. The sender snapshots the event array once per request; appends after that snapshot belong to a later request. -### Session header +### Wire Session header -The `session` member is the exact `Session.header`, not a complete runtime Session. The outer `dsh_session_log.version` selects this extension schema, while `session.version` selects the canonical on-disk Session format; the two version values evolve independently. +The `session` member projects `Session.header`, not a complete runtime Session or the header object itself. It copies the current header facts and replaces `isSeeded` with numeric `seedLength` derived from the exact `Session.inheritedEventCount`. The outer `dsh_session_log.version` selects this extension schema, while `session.version` selects the logical Session format; the two version values evolve independently. | Member | Presence | Meaning | |---|---|---| -| `version` | required | Canonical Session format version; currently `0` | +| `version` | required | Logical Session format version; currently `1` | | `id` | required | Exact Session id | | `createdAt` | required | Non-negative safe-integer Unix epoch milliseconds | | `cwd` | optional | Absolute working directory recorded at Session creation | @@ -141,14 +143,15 @@ After the endpoint returns HTTP 2xx, the contribution appends this canonical eve "time": 1780000000002, "data": { "sessionId": "session-id", + "sessionFormatVersion": 1, "throughSeq": 7 } } ``` -`delivery-accepted` means that the configured endpoint returned HTTP 2xx for the containing LLM request. It does not assert SSE completion or remote persistence. The event's `throughSeq` must identify an earlier event, and its `sessionId` identifies the Session whose suffix was sent. +`delivery-accepted` means that the configured endpoint returned HTTP 2xx for the containing LLM request. It does not assert SSE completion or remote persistence. The event's `throughSeq` must identify an earlier event, its `sessionId` identifies the Session whose suffix was sent, and `sessionFormatVersion` binds the watermark to that exact logical generation. Absence means historical format v0. -The sender folds the greatest matching `throughSeq`, so concurrent accepted requests cannot move the cursor backward. A resumed process rebuilds the cursor from the durable log. A fork ignores inherited watermarks that name its parent, and therefore sends its own complete inherited prefix before advancing under the child id. The watermark event itself belongs to the next unsent suffix. +The sender folds the greatest matching `throughSeq` for the current Session id and format generation, so concurrent accepted requests cannot move the cursor backward and a migrated v0 watermark cannot authorize a v1 suffix. A resumed process rebuilds the cursor from the durable log. A fork ignores inherited watermarks that name another Session, and therefore sends its own complete inherited prefix before advancing under the child id. The watermark event itself belongs to the next unsent suffix. Transport and non-2xx failures append no watermark. A crash after endpoint acceptance but before local persistence may resend an already accepted range; uncertainty produces duplicates, never a sequence gap. There is no independent upload store, size cap, or truncation path. diff --git a/docs/deepseek-llm-api-wire-extensions.zh.md b/docs/deepseek-llm-api-wire-extensions.zh.md index 61af718841..6d0a444796 100644 --- a/docs/deepseek-llm-api-wire-extensions.zh.md +++ b/docs/deepseek-llm-api-wire-extensions.zh.md @@ -79,8 +79,9 @@ { "dsh_session_log": { "version": 1, + "sessionFormatVersion": 1, "session": { - "version": 0, + "version": 1, "id": "session-id", "createdAt": 1780000000000 }, @@ -103,20 +104,21 @@ | 成员 | 类型 | 含义 | |---|---|---| | `version` | `1` | `dsh_session_log` 的 schema 版本 | -| `session` | 对象 | 不可变的权威 `SessionHeader` | +| `sessionFormatVersion` | 非负整数 | 该后缀所表示的 Session 格式 generation | +| `session` | 对象 | 当前 Session header 与继承切点的不可变协议投影 | | `afterSeq` | 整数 | 本次请求前记录为已接受的最大序号，或 `-1` | | `throughSeq` | 非负整数 | 本次请求所表示的最大序号 | | `events` | 数组 | 从 `afterSeq + 1` 到 `throughSeq` 的连续事件 | 首次上传使用 `afterSeq: -1`，并携带当前的完整日志。此后每次上传都从同一会话 id 的最大已接受水位（watermark）之后开始。发送方为每次请求仅快照一次事件数组；快照后的追加内容属于后续请求。 -### 会话头 +### Session 协议 header -`session` 成员是确切的 `Session.header`，不是完整的运行时会话。外层 `dsh_session_log.version` 选择本扩展 schema，`session.version` 则选择权威磁盘会话格式；两个版本值相互独立演进。 +`session` 成员投影 `Session.header`，既不是完整的运行时 Session，也不是 header 对象本身。它复制当前 header 事实，并把 `isSeeded` 替换为根据精确 `Session.inheritedEventCount` 得出的数值 `seedLength`。外层 `dsh_session_log.version` 选择本扩展 schema，`session.version` 则选择逻辑 Session 格式；两个版本值相互独立演进。 | 成员 | 出现条件 | 含义 | |---|---|---| -| `version` | 必需 | 权威会话格式版本；当前为 `0` | +| `version` | 必需 | 逻辑 Session 格式版本；当前为 `1` | | `id` | 必需 | 确切的会话 id | | `createdAt` | 必需 | 非负安全整数 Unix epoch 毫秒数 | | `cwd` | 可选 | 创建会话时记录的绝对工作目录 | @@ -141,14 +143,15 @@ "time": 1780000000002, "data": { "sessionId": "session-id", + "sessionFormatVersion": 1, "throughSeq": 7 } } ``` -`delivery-accepted` 表示已配置端点为包含该字段的 LLM 请求返回 HTTP 2xx。它不表示 SSE 已完整结束，也不表示远端已经持久化。该事件的 `throughSeq` 必须标识一项更早的事件，`sessionId` 则标识已发送后缀所属的会话。 +`delivery-accepted` 表示已配置端点为包含该字段的 LLM 请求返回 HTTP 2xx。它不表示 SSE 已完整结束，也不表示远端已经持久化。该事件的 `throughSeq` 必须标识一项更早的事件，`sessionId` 标识已发送后缀所属的 Session，`sessionFormatVersion` 则把水位绑定到该逻辑 generation。缺少该字段表示历史格式 v0。 -发送方会折叠最大的匹配 `throughSeq`，因此并发已接受请求无法使游标倒退。恢复后的进程会从持久日志重建游标。fork 会忽略命名其父会话的继承水位，因此先发送自身完整的继承前缀，再以子会话 id 推进。水位事件自身属于下一段未发送后缀。 +发送方只会为当前 Session id 与格式 generation 折叠最大的匹配 `throughSeq`，因此并发已接受请求无法使游标倒退，迁移后的 v0 水位也不能授权 v1 后缀。恢复后的进程会从持久日志重建游标。fork 会忽略命名其他 Session 的继承水位，因此先发送自身完整的继承前缀，再以子会话 id 推进。水位事件自身属于下一段未发送后缀。 传输失败和非 2xx 响应不会追加水位。端点接受后、本地持久化前发生崩溃时，系统可能重新发送已接受范围；不确定性只会产生重复，绝不会产生序号缺口。系统没有独立上传存储、大小上限或截断路径。 diff --git a/docs/event-producer-consumer.i18n.yaml b/docs/event-producer-consumer.i18n.yaml index 28dc25a1c8..2bdb1a813a 100644 --- a/docs/event-producer-consumer.i18n.yaml +++ b/docs/event-producer-consumer.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write docs/event-producer-consumer.md -event-producer-consumer.md: bdfb3f23e817dadebb090df154af8e1ebf018ee8 -event-producer-consumer.zh.md: 3fe1fe737cb7e99bb8e3a8641dc7bbd7b11b74a4 +event-producer-consumer.md: 783132a5062b5374f2245eb27d866af471d9cec0 +event-producer-consumer.zh.md: b2816cf2edaf123d39e8368390234bccc7d817f4 diff --git a/docs/event-producer-consumer.md b/docs/event-producer-consumer.md index bdfb3f23e8..783132a506 100644 --- a/docs/event-producer-consumer.md +++ b/docs/event-producer-consumer.md @@ -7,7 +7,7 @@ This matrix shows which packages dispatch each harness-owned event and which pac | Event | Mode | Declared in | Dispatchers | Listeners | | --- | --- | --- | --- | --- | -| `agent-loop/config-start-failed` | `emit` | [`packages/core/agent-loop/src/index.ts:239`](../packages/core/agent-loop/src/index.ts) | [`agent-loop`](../packages/core/agent-loop) (`events.dispatch`) | - | +| `agent-loop/config-start-failed` | `emit` | [`packages/core/agent-loop/src/index.ts:240`](../packages/core/agent-loop/src/index.ts) | [`agent-loop`](../packages/core/agent-loop) (`events.dispatch`) | - | | `agent-preset/selected` | `emit` | [`packages/preset/agent-presets/src/types.ts:80`](../packages/preset/agent-presets/src/types.ts) | [`agent-presets`](../packages/preset/agent-presets) (`emit`) | `remotes` | | `agent/created` | `emit` | [`packages/core/agent/src/runtime-types.ts:166`](../packages/core/agent/src/runtime-types.ts) | [`agent`](../packages/core/agent) (`events.dispatch`) | [`agent-presets`](../packages/preset/agent-presets), [`file-reference-local`](../packages/context/file-reference-local), [`goal-round-driver`](../packages/goal/goal-round-driver), [`schedule`](../packages/schedule/schedule), `tool-agent-team`, [`tool-subagent`](../packages/subagent/tool-subagent) | | `agent/disposed` | `emit` | [`packages/core/agent/src/runtime-types.ts:175`](../packages/core/agent/src/runtime-types.ts) | [`agent`](../packages/core/agent) (`events.dispatch`) | [`agent-loop`](../packages/core/agent-loop), [`file-reference-local`](../packages/context/file-reference-local), [`goal-round-driver`](../packages/goal/goal-round-driver), [`subagent`](../packages/subagent/subagent), `tool-agent-team`, [`tool-subagent`](../packages/subagent/tool-subagent) | diff --git a/docs/event-producer-consumer.zh.md b/docs/event-producer-consumer.zh.md index 3fe1fe737c..b2816cf2ed 100644 --- a/docs/event-producer-consumer.zh.md +++ b/docs/event-producer-consumer.zh.md @@ -9,7 +9,7 @@ | 事件 | 模式 | 声明位置 | 派发方 | 监听方 | | --- | --- | --- | --- | --- | -| `agent-loop/config-start-failed` | `emit` | [`packages/core/agent-loop/src/index.ts:239`](../packages/core/agent-loop/src/index.ts) | [`agent-loop`](../packages/core/agent-loop) (`events.dispatch`) | - | +| `agent-loop/config-start-failed` | `emit` | [`packages/core/agent-loop/src/index.ts:240`](../packages/core/agent-loop/src/index.ts) | [`agent-loop`](../packages/core/agent-loop) (`events.dispatch`) | - | | `agent-preset/selected` | `emit` | [`packages/preset/agent-presets/src/types.ts:80`](../packages/preset/agent-presets/src/types.ts) | [`agent-presets`](../packages/preset/agent-presets) (`emit`) | `remotes` | | `agent/created` | `emit` | [`packages/core/agent/src/runtime-types.ts:166`](../packages/core/agent/src/runtime-types.ts) | [`agent`](../packages/core/agent) (`events.dispatch`) | [`agent-presets`](../packages/preset/agent-presets), [`file-reference-local`](../packages/context/file-reference-local), [`goal-round-driver`](../packages/goal/goal-round-driver), [`schedule`](../packages/schedule/schedule), `tool-agent-team`, [`tool-subagent`](../packages/subagent/tool-subagent) | | `agent/disposed` | `emit` | [`packages/core/agent/src/runtime-types.ts:175`](../packages/core/agent/src/runtime-types.ts) | [`agent`](../packages/core/agent) (`events.dispatch`) | [`agent-loop`](../packages/core/agent-loop), [`file-reference-local`](../packages/context/file-reference-local), [`goal-round-driver`](../packages/goal/goal-round-driver), [`subagent`](../packages/subagent/subagent), `tool-agent-team`, [`tool-subagent`](../packages/subagent/tool-subagent) | diff --git a/docs/module-graph.i18n.yaml b/docs/module-graph.i18n.yaml index 3b1f274aee..a2a5e47205 100644 --- a/docs/module-graph.i18n.yaml +++ b/docs/module-graph.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write docs/module-graph.md -module-graph.md: a028d1efd12924f0592b1dba954071cebd9d3dbe -module-graph.zh.md: c4efd116e5c79aa9c0210d5ff3692d7a39cd1c46 +module-graph.md: e665a73a72c0282bcc7950c7c57553b7793241ee +module-graph.zh.md: e4d03468060b2857e8826971ccdcbd6f45351a94 diff --git a/docs/module-graph.md b/docs/module-graph.md index a028d1efd1..e665a73a72 100644 --- a/docs/module-graph.md +++ b/docs/module-graph.md @@ -283,6 +283,9 @@ flowchart TD end subgraph group_session["packages/session"] pkg_session_checkpoint_policy["session-checkpoint-policy"] + pkg_session_format["session-format"] + pkg_session_format_catalog["session-format-catalog"] + pkg_session_format_v0_to_v1["session-format-v0-to-v1"] pkg_session_log_deepseek["session-log-deepseek"] pkg_session_persistence["session-persistence"] pkg_session_persistence_jsonl["session-persistence-jsonl"] @@ -424,6 +427,7 @@ flowchart TD pkg_persona --> pkg_system_prompt pkg_sandbox --> pkg_llm pkg_sandbox --> pkg_session + pkg_session_format_catalog --> pkg_session pkg_session_log_deepseek --> pkg_deepseek_llm_api_extensions pkg_session_log_deepseek --> pkg_invariants pkg_session_log_deepseek --> pkg_session @@ -1218,6 +1222,8 @@ flowchart TD | [`host-webserver`](../packages/host/webserver) | `host` | — | | [`invariants`](../packages/runtime-diagnostics/invariants) | `runtime-diagnostics` | — | | [`sandbox-windows-acl`](../packages/sandbox/sandbox-windows-acl) | `sandbox` | — | +| [`session-format`](../packages/session/session-format) | `session` | — | +| [`session-format-v0-to-v1`](../packages/session/session-format-v0-to-v1) | `session` | — | | [`storage`](../packages/storage/storage) | `storage` | — | | [`subprocess`](../packages/subprocess/subprocess) | `subprocess` | — | | [`win32-process`](../packages/subprocess/win32-process) | `subprocess` | — | @@ -1258,6 +1264,7 @@ flowchart TD | [`code-runtime-worker-thread`](../packages/code-runtime/code-runtime-worker-thread) | `code-runtime` | [`code-runtime`](../packages/code-runtime/code-runtime), [`session`](../packages/core/session), [`timeout`](../packages/util/timeout) | | [`persona`](../packages/preset/persona) | `preset` | [`system-prompt`](../packages/core/system-prompt) | | [`sandbox`](../packages/sandbox/sandbox) | `sandbox` | [`llm`](../packages/llm/llm), [`session`](../packages/core/session) | +| [`session-format-catalog`](../packages/session/session-format-catalog) | `session` | [`session`](../packages/core/session) | | [`session-log-deepseek`](../packages/session/session-log-deepseek) | `session` | [`deepseek-llm-api-extensions`](../packages/llm/deepseek-llm-api-extensions), [`invariants`](../packages/runtime-diagnostics/invariants), [`session`](../packages/core/session) | | [`session-persistence`](../packages/session/session-persistence) | `session` | [`brand`](../packages/util/brand), [`session`](../packages/core/session), [`timeout`](../packages/util/timeout) | | [`session-projection`](../packages/session/session-projection) | `session` | [`session`](../packages/core/session) | diff --git a/docs/module-graph.zh.md b/docs/module-graph.zh.md index c4efd116e5..e4d0346806 100644 --- a/docs/module-graph.zh.md +++ b/docs/module-graph.zh.md @@ -285,6 +285,9 @@ flowchart TD end subgraph group_session["packages/session"] pkg_session_checkpoint_policy["session-checkpoint-policy"] + pkg_session_format["session-format"] + pkg_session_format_catalog["session-format-catalog"] + pkg_session_format_v0_to_v1["session-format-v0-to-v1"] pkg_session_log_deepseek["session-log-deepseek"] pkg_session_persistence["session-persistence"] pkg_session_persistence_jsonl["session-persistence-jsonl"] @@ -426,6 +429,7 @@ flowchart TD pkg_persona --> pkg_system_prompt pkg_sandbox --> pkg_llm pkg_sandbox --> pkg_session + pkg_session_format_catalog --> pkg_session pkg_session_log_deepseek --> pkg_deepseek_llm_api_extensions pkg_session_log_deepseek --> pkg_invariants pkg_session_log_deepseek --> pkg_session @@ -1220,6 +1224,8 @@ flowchart TD | [`host-webserver`](../packages/host/webserver) | `host` | — | | [`invariants`](../packages/runtime-diagnostics/invariants) | `runtime-diagnostics` | — | | [`sandbox-windows-acl`](../packages/sandbox/sandbox-windows-acl) | `sandbox` | — | +| [`session-format`](../packages/session/session-format) | `session` | — | +| [`session-format-v0-to-v1`](../packages/session/session-format-v0-to-v1) | `session` | — | | [`storage`](../packages/storage/storage) | `storage` | — | | [`subprocess`](../packages/subprocess/subprocess) | `subprocess` | — | | [`win32-process`](../packages/subprocess/win32-process) | `subprocess` | — | @@ -1260,6 +1266,7 @@ flowchart TD | [`code-runtime-worker-thread`](../packages/code-runtime/code-runtime-worker-thread) | `code-runtime` | [`code-runtime`](../packages/code-runtime/code-runtime), [`session`](../packages/core/session), [`timeout`](../packages/util/timeout) | | [`persona`](../packages/preset/persona) | `preset` | [`system-prompt`](../packages/core/system-prompt) | | [`sandbox`](../packages/sandbox/sandbox) | `sandbox` | [`llm`](../packages/llm/llm), [`session`](../packages/core/session) | +| [`session-format-catalog`](../packages/session/session-format-catalog) | `session` | [`session`](../packages/core/session) | | [`session-log-deepseek`](../packages/session/session-log-deepseek) | `session` | [`deepseek-llm-api-extensions`](../packages/llm/deepseek-llm-api-extensions), [`invariants`](../packages/runtime-diagnostics/invariants), [`session`](../packages/core/session) | | [`session-persistence`](../packages/session/session-persistence) | `session` | [`brand`](../packages/util/brand), [`session`](../packages/core/session), [`timeout`](../packages/util/timeout) | | [`session-projection`](../packages/session/session-projection) | `session` | [`session`](../packages/core/session) | diff --git a/docs/persistence-catalog.i18n.yaml b/docs/persistence-catalog.i18n.yaml index 68394c63b7..ad9d851a59 100644 --- a/docs/persistence-catalog.i18n.yaml +++ b/docs/persistence-catalog.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write docs/persistence-catalog.md -persistence-catalog.md: 2fb2b020841a42da6f7505928a87868fc43bc559 -persistence-catalog.zh.md: 4014fd2c10e20193ea68a5f6c761c99b4569b032 +persistence-catalog.md: 94a2415f92d4fccdde96553372b2536c81744e24 +persistence-catalog.zh.md: 649530fb89de98dc7c649b00d2c5bed4d9da30e8 diff --git a/docs/persistence-catalog.md b/docs/persistence-catalog.md index 2fb2b02084..94a2415f92 100644 --- a/docs/persistence-catalog.md +++ b/docs/persistence-catalog.md @@ -7,7 +7,7 @@ Every event type that can appear in a session's durable event log: the complete This file is GENERATED from source (`scripts/gen-persistence-catalog.ts`) and verified fresh by `pnpm run verify-persistence-catalog` (part of `doc-sync`) — do not edit it by hand. Declaration blocks retain the source declaration and nested property JSDoc, removing only the indentation imposed by a containing interface/module, and use a `ts persistence-catalog` fence (skipped by doc-typecheck because declarations reference types from their owning modules). Type names in a payload link to the page that documents them. See [the persistence-log-catalog Agent Note](../.agents/notes/archived/process/2026-07-04-persistence-log-catalog.md). -The envelope declarations below compose each event's `type`, monotonic `seq`, epoch-ms `time`, `data`, the optional `ignorable` unknown-type skip marker, and the conditional `surfaceOp`/`sourceEventSeqs` fields. **surface** marks a `SurfaceEventType` member: it produces an LLM message and declares how it joins the surface list. **log-only** marks everything else: a durable, replayable record with no derived-history contribution. Every payload is JSON-serializable (enforced at `Session.append`), and the whole format is pinned at `SESSION_FORMAT_VERSION = 0` — pre-release, no compatibility implied ([the version stance](subsystems/persistence.md)). Scope: the packages in this repo; a downstream plugin can merge further event types, which are outside this catalog by construction. +The envelope declarations below compose each event's `type`, monotonic `seq`, epoch-ms `time`, `data`, the optional `ignorable` unknown-type skip marker, and the conditional `surfaceOp`/`sourceEventSeqs` fields. **surface** marks a `SurfaceEventType` member: it produces an LLM message and declares how it joins the surface list. **log-only** marks everything else: a durable, replayable record with no derived-history contribution. Every payload is JSON-serializable (enforced at `Session.append`). Current writers stamp `SESSION_FORMAT_VERSION`; supported historical artifacts reach this current vocabulary through the build-static adjacent migration catalog ([the version lifecycle](subsystems/persistence.md)). Scope: the packages in this repo; a downstream plugin can merge further current-version event types, which are outside this catalog by construction and require an explicit disposition at a later format edge. ## Event envelope @@ -90,7 +90,7 @@ export type SessionEvent = { }[T] ``` -Sources: [`packages/core/session/src/types.ts:366`](../packages/core/session/src/types.ts) · [`packages/core/session/src/types.ts:373`](../packages/core/session/src/types.ts) · [`packages/core/session/src/types.ts:402`](../packages/core/session/src/types.ts) · [`packages/core/session/src/types.ts:434`](../packages/core/session/src/types.ts) +Sources: [`packages/core/session/src/types.ts:364`](../packages/core/session/src/types.ts) · [`packages/core/session/src/types.ts:371`](../packages/core/session/src/types.ts) · [`packages/core/session/src/types.ts:400`](../packages/core/session/src/types.ts) · [`packages/core/session/src/types.ts:432`](../packages/core/session/src/types.ts) ## Events @@ -215,7 +215,7 @@ Source: [`packages/interaction/user-approval/src/index.ts:33`](../packages/inter Types: [StreamChunk](subsystems/llm-streaming.md) -Source: [`packages/core/session/src/types.ts:289`](../packages/core/session/src/types.ts) +Source: [`packages/core/session/src/types.ts:287`](../packages/core/session/src/types.ts) @@ -237,7 +237,7 @@ Source: [`packages/core/session/src/types.ts:289`](../packages/core/session/src/ Types: [TokenUsage](subsystems/llm-streaming.md) -Source: [`packages/core/session/src/types.ts:300`](../packages/core/session/src/types.ts) +Source: [`packages/core/session/src/types.ts:298`](../packages/core/session/src/types.ts) ### `command/*` @@ -563,7 +563,7 @@ Source: [`packages/plan/plan-mode/src/index.ts:46`](../packages/plan/plan-mode/s 'request/context': RequestContext ``` -Source: [`packages/core/session/src/types.ts:339`](../packages/core/session/src/types.ts) +Source: [`packages/core/session/src/types.ts:337`](../packages/core/session/src/types.ts) @@ -582,7 +582,7 @@ Source: [`packages/core/session/src/types.ts:339`](../packages/core/session/src/ } ``` -Source: [`packages/core/session/src/types.ts:329`](../packages/core/session/src/types.ts) +Source: [`packages/core/session/src/types.ts:327`](../packages/core/session/src/types.ts) ### `sandbox/*` @@ -657,7 +657,7 @@ Source: [`packages/schedule/schedule/src/types.ts:219`](../packages/schedule/sch 'session/end-seed': Record ``` -Source: [`packages/core/session/src/types.ts:362`](../packages/core/session/src/types.ts) +Source: [`packages/core/session/src/types.ts:360`](../packages/core/session/src/types.ts) @@ -699,12 +699,14 @@ Source: [`packages/session/session-title-llm/src/index.ts:45`](../packages/sessi 'session-log-deepseek/delivery-accepted': { /** Session identity the accepted delivery carried; inherited fork markers retain the parent's id. */ sessionId: import('@deepseek-ai/dsh-session/types').SessionId + /** Accepted Session format generation; absence identifies version 0. */ + sessionFormatVersion?: number /** Last canonical event included in the accepted request. */ throughSeq: import('@deepseek-ai/dsh-session/types').SessionSeq } ``` -Source: [`packages/session/session-log-deepseek/src/types.ts:57`](../packages/session/session-log-deepseek/src/types.ts) +Source: [`packages/session/session-log-deepseek/src/types.ts:59`](../packages/session/session-log-deepseek/src/types.ts) ### `step/*` @@ -717,7 +719,7 @@ Source: [`packages/session/session-log-deepseek/src/types.ts:57`](../packages/se 'step/end': { turn: number; step: number } ``` -Source: [`packages/core/session/src/types.ts:279`](../packages/core/session/src/types.ts) +Source: [`packages/core/session/src/types.ts:277`](../packages/core/session/src/types.ts) @@ -728,7 +730,7 @@ Source: [`packages/core/session/src/types.ts:279`](../packages/core/session/src/ 'step/start': { turn: number; step: number } ``` -Source: [`packages/core/session/src/types.ts:277`](../packages/core/session/src/types.ts) +Source: [`packages/core/session/src/types.ts:275`](../packages/core/session/src/types.ts) ### `subagent/*` @@ -859,7 +861,7 @@ Source: [`packages/todo/tool-todo/src/types.ts:31`](../packages/todo/tool-todo/s Types: [ToolCallId](subsystems/core.md) -Source: [`packages/core/session/src/types.ts:306`](../packages/core/session/src/types.ts) +Source: [`packages/core/session/src/types.ts:304`](../packages/core/session/src/types.ts) @@ -934,7 +936,7 @@ Source: [`packages/core/tools/src/types.ts:40`](../packages/core/tools/src/types } ``` -Source: [`packages/core/session/src/types.ts:318`](../packages/core/session/src/types.ts) +Source: [`packages/core/session/src/types.ts:316`](../packages/core/session/src/types.ts) ### `tool-workflow/*` @@ -1014,7 +1016,7 @@ Source: [`packages/workflow/tool-workflow/src/types.ts:47`](../packages/workflow Types: [TurnEndReason](subsystems/session.md) -Source: [`packages/core/session/src/types.ts:275`](../packages/core/session/src/types.ts) +Source: [`packages/core/session/src/types.ts:273`](../packages/core/session/src/types.ts) @@ -1030,7 +1032,7 @@ Source: [`packages/core/session/src/types.ts:275`](../packages/core/session/src/ 'turn/start': { turn: number } ``` -Source: [`packages/core/session/src/types.ts:266`](../packages/core/session/src/types.ts) +Source: [`packages/core/session/src/types.ts:264`](../packages/core/session/src/types.ts) ### `user/*` @@ -1049,7 +1051,7 @@ Source: [`packages/core/session/src/types.ts:266`](../packages/core/session/src/ 'user/message': UserMessage ``` -Source: [`packages/core/session/src/types.ts:287`](../packages/core/session/src/types.ts) +Source: [`packages/core/session/src/types.ts:285`](../packages/core/session/src/types.ts) ### `web/*` diff --git a/docs/persistence-catalog.zh.md b/docs/persistence-catalog.zh.md index 4014fd2c10..649530fb89 100644 --- a/docs/persistence-catalog.zh.md +++ b/docs/persistence-catalog.zh.md @@ -9,7 +9,7 @@ 英文源文件根据源码生成（`scripts/gen-persistence-catalog.ts`），并由 `pnpm run verify-persistence-catalog`（`doc-sync`（文档同步门禁）的一部分）验证新鲜度；本中文文件作为经评审对侧通过双语配对维护。声明块保留源码声明和嵌套属性的 JSDoc，只移除其所在接口／模块带来的缩进，并使用 `ts persistence-catalog` 围栏（doc-typecheck 会跳过这些围栏，因为声明引用了其所属模块中的类型）。payload 中的类型名称会链接到记录该类型的页面。参见 [persistence-log-catalog Agent Note](../.agents/notes/archived/process/2026-07-04-persistence-log-catalog.md)。 -以下信封声明组合了每个事件的 `type`、单调递增的 `seq`、以 epoch 毫秒表示的 `time`、`data`、可选的未知类型跳过标记 `ignorable`，以及条件字段 `surfaceOp`／`sourceEventSeqs`。**surface** 表示 `SurfaceEventType` 成员：它会生成一条 LLM（大语言模型）消息，并声明该事件如何加入 surface 列表。**log-only** 表示其他所有事件：这类记录可持久化、可回放，但不参与派生历史。每个 payload 均可进行 JSON 序列化（在 `Session.append` 处强制执行），整个格式固定为 `SESSION_FORMAT_VERSION = 0`：这是预发布格式，不暗示任何兼容性（参见[版本立场](subsystems/persistence.zh.md)）。范围仅限本仓库中的包；下游插件可以继续合并其他事件类型，而这些类型按设计不属于本目录。 +以下信封声明组合了每个事件的 `type`、单调递增的 `seq`、以 epoch 毫秒表示的 `time`、`data`、可选的未知类型跳过标记 `ignorable`，以及条件字段 `surfaceOp`／`sourceEventSeqs`。**surface** 表示 `SurfaceEventType` 成员：它会生成一条 LLM（大语言模型）消息，并声明该事件如何加入 surface 列表。**log-only** 表示其他所有事件：这类记录可持久化、可回放，但不参与派生历史。每个 payload 均可进行 JSON 序列化（在 `Session.append` 处强制执行）。当前 writer 会写入 `SESSION_FORMAT_VERSION`；受支持的历史产物通过构建期静态相邻迁移目录进入这套当前词汇（参见[版本生命周期](subsystems/persistence.zh.md)）。范围仅限本仓库中的包；下游插件可以继续合并其他当前版本事件类型，这些类型按设计不属于本目录，并且在后续格式迁移边中需要显式 disposition。 ## 事件信封 @@ -701,12 +701,14 @@ export type SessionEvent = { 'session-log-deepseek/delivery-accepted': { /** Session identity the accepted delivery carried; inherited fork markers retain the parent's id. */ sessionId: import('@deepseek-ai/dsh-session/types').SessionId + /** Accepted Session format generation; absence identifies version 0. */ + sessionFormatVersion?: number /** Last canonical event included in the accepted request. */ throughSeq: import('@deepseek-ai/dsh-session/types').SessionSeq } ``` -来源：[`packages/session/session-log-deepseek/src/types.ts:57`](../packages/session/session-log-deepseek/src/types.ts) +来源：[`packages/session/session-log-deepseek/src/types.ts:59`](../packages/session/session-log-deepseek/src/types.ts) ### `step/*` diff --git a/docs/subsystems/feedback.i18n.yaml b/docs/subsystems/feedback.i18n.yaml index 9e6b0d6424..886c121222 100644 --- a/docs/subsystems/feedback.i18n.yaml +++ b/docs/subsystems/feedback.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write docs/subsystems/feedback.md -feedback.md: 046765b65773834b1804aa2a915625a3d6c9f099 -feedback.zh.md: 55b8e7d1b5b2c8ca5613d9888ef7f415b34ed5dc +feedback.md: 7fbb1b962802e0a53665ff1235a83d6b976524be +feedback.zh.md: b042e00d6940a49978cde37a6d9c50e3ccd380ba diff --git a/docs/subsystems/feedback.md b/docs/subsystems/feedback.md index 046765b657..7fbb1b9628 100644 --- a/docs/subsystems/feedback.md +++ b/docs/subsystems/feedback.md @@ -191,7 +191,7 @@ One Session sidecar row contains its header identity `{createdAt, cwd}` and feed ## Target and lifecycle authority -`SessionPersistence.inspect()` supplies the target Session observation without publishing or resuming an Agent and without committing cold repair. A cold `listSnapshots()` preflight classifies definite absence; inspection failure for a catalogued Session propagates as infrastructure failure. `put` accepts only a non-empty, append-origin `assistant/message` with the requested `MessageId`; replacement-origin, usage-only empty, and non-assistant records are not feedback targets. +`SessionPersistence.inspect()` supplies the target Session observation without publishing or resuming an Agent. Already-current inspection keeps cold repair in memory; a supported historical body read first publishes its migrated and repaired current generation. A cold `listSnapshots()` preflight classifies definite absence; inspection failure for a catalogued Session propagates as infrastructure failure. `put` accepts only a non-empty, append-origin `assistant/message` with the requested `MessageId`; replacement-origin, usage-only empty, and non-assistant records are not feedback targets. The stored `{createdAt, cwd}` identity must match the inspected header. A mismatch is treated as absence: `list` returns no items, while `put` may replace the stale row with one bound to the current header identity. Forks use a new Session identity and receive no sidecar copy even when their seed contains the same messages. diff --git a/docs/subsystems/feedback.zh.md b/docs/subsystems/feedback.zh.md index 55b8e7d1b5..b042e00d69 100644 --- a/docs/subsystems/feedback.zh.md +++ b/docs/subsystems/feedback.zh.md @@ -191,7 +191,7 @@ type MessageFeedbackDeleteResult = ## 目标与生命周期权威 -`SessionPersistence.inspect()` 提供目标 Session 的观测，且不会发布或恢复 Agent，也不会提交 cold repair。cold 路径先由 `listSnapshots()` 预检明确不存在；已进入目录的 Session 若检查失败，会按基础设施故障原样传播。`put` 只接受具有指定 `MessageId` 的非空、append-origin `assistant/message`；replacement-origin、仅承载 usage 的空记录和非 assistant 记录都不是反馈目标。 +`SessionPersistence.inspect()` 提供目标 Session 的观测，且不会发布或恢复 Agent。已经是当前格式的检查只在内存中保留 cold repair；受支持的历史正文读取会先发布其迁移并修复后的当前 generation。cold 路径先由 `listSnapshots()` 预检明确不存在；已进入目录的 Session 若检查失败，会按基础设施故障原样传播。`put` 只接受具有指定 `MessageId` 的非空、append-origin `assistant/message`；replacement-origin、仅承载 usage 的空记录和非 assistant 记录都不是反馈目标。 存储的 `{createdAt, cwd}` 身份必须与检查所得 header 匹配。不匹配按不存在处理：`list` 返回空条目，`put` 则可用绑定当前 header 身份的新记录替换陈旧行。fork 使用新的 Session 身份，即使种子包含相同消息，也不获得伴随记录副本。 diff --git a/docs/subsystems/persistence.i18n.yaml b/docs/subsystems/persistence.i18n.yaml index ec456511f9..6e75f6aa2d 100644 --- a/docs/subsystems/persistence.i18n.yaml +++ b/docs/subsystems/persistence.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write docs/subsystems/persistence.md -persistence.md: b61ddee8d12854b330b967a09a1a685d763f0041 -persistence.zh.md: 08cc35d987360138ec096b70f8c6b11b5cba4c55 +persistence.md: d00e3c5fcd6f2e770abe115bb5c83c06c3e903b6 +persistence.zh.md: 55aa5449d1b8c1635a3f26da3c9e52ad98cc7f65 diff --git a/docs/subsystems/persistence.md b/docs/subsystems/persistence.md index b61ddee8d1..d00e3c5fcd 100644 --- a/docs/subsystems/persistence.md +++ b/docs/subsystems/persistence.md @@ -16,11 +16,11 @@ A backend that reloads a log crashed mid-turn finds an open `turn/start` with no Repair applies only to cold sessions. For a live id, `SessionPersistence.load(id)` waits until the authoritative in-memory snapshot is durable and returns it only when balanced; an open live turn rejects rather than receiving synthetic interruption boundaries. HMR adopts a live prefix without closing its active turn. -`SessionPersistence.inspect(id)` constructs an immutable logical Session without publishing it or writing recovery. Cold inspection balances an interrupted turn in memory while leaving torn physical tails untouched; inspection of an already-live Session borrows its current immutable snapshot and may therefore contain an open turn. Coordinator-backed implementations retain the exact cold unpublished Session in a bounded LRU, so repeated history reads and a later `prepare(id)` share one read, decompression, validation, freeze, and Session construction. `prepare(id)` reserves the Session, commits pending repair, and returns a disposable publication handle; `load(id)` uses the same machinery to commit repair without publication. The [Session preparation decision](../../.agents/notes/implemented/architecture/2026-08-05-session-preparation.md) owns this lifecycle. +`SessionPersistence.inspect(id)` constructs an immutable logical Session without publishing it. For an already-current cold generation, inspection balances an interrupted turn in memory without writing recovery and leaves torn physical tails untouched. A supported historical generation is different: the serialized body-read operation first publishes its repaired current successor at a new version-qualified path while preserving the source, so inspection never exposes an old logical format. Inspection of an already-live Session borrows its current immutable snapshot and may therefore contain an open turn. Coordinator-backed implementations retain the exact cold unpublished Session in a bounded LRU, so repeated history reads and a later `prepare(id)` share one read, decompression, validation, freeze, and Session construction. `prepare(id)` reserves the Session, commits pending current-format repair, and returns a disposable publication handle; `load(id)` uses the same machinery to commit repair without publication. The [Session preparation decision](../../.agents/notes/implemented/architecture/2026-08-05-session-preparation.md) owns this lifecycle. ## `SessionLocation` — optional per-session artifact target -`SessionPersistence.locate(meta)` synchronously resolves a backend-owned independent artifact without reading, creating, or flushing it. JSONL returns the absolute transcript path inside its project/session directory; a backend without one independent artifact per session returns `undefined`. A returned path can therefore name a file that does not yet exist or lacks the current unflushed turn; it is a location hint, not authorization or a freshness guarantee. +`SessionPersistence.locate(meta)` synchronously resolves a backend-owned independent artifact target without reading, creating, or flushing it. JSONL derives the canonical filename from `meta.version`: v0 is `session.jsonl[.zstd]`, while every positive version is lowercase `session.vN.jsonl[.zstd]`. A backend without one independent artifact per Session returns `undefined`. The returned path may not exist yet and says nothing about a different highest generation already stored in the directory; it is a version-qualified location hint, not authorization, discovery, or a freshness guarantee. Header-only listing owns discovery and returns the exact selected generation in `listing.location`. ```ts type-equiv /** @@ -50,11 +50,10 @@ Source: [`packages/core/session/src/types.ts`](../../packages/core/session/src/t */ interface SessionHeader { /** - * On-disk format version, stamped from {@link SESSION_FORMAT_VERSION} when the - * session is created. A persistence backend rejects any other version on load - * (no migration — see the constant). + * Current logical format version, stamped from {@link SESSION_FORMAT_VERSION}. + * Historical physical headers are translated before entering this interface. */ - readonly version: number + readonly version: typeof SESSION_FORMAT_VERSION /** The session's id (mirrors the {@link Session}'s id). */ readonly id: SessionId /** Non-negative safe-integer Unix epoch milliseconds when the session was created. */ @@ -91,7 +90,7 @@ interface SessionHeader { ## Format refusal — logs a build cannot faithfully read -A backend refuses a log it cannot faithfully interpret with `SessionFormatUnsupportedError`, distinct from `SessionPersistenceCorruptionError` because nothing is damaged. A header `version` ahead of `SESSION_FORMAT_VERSION` names the direction ("written by a newer harness — upgrade the harness to open it"); one behind it states that this build ships no upgrade path. After legacy-shape normalization, an event type outside this build's generated vocabulary (`KNOWN_SESSION_EVENT_TYPES`, emitted by `gen-persistence-catalog`) refuses the same way unless the event's envelope carries `ignorable: true` — silently skipping an unrecognized required event could change how the rest of the log must be read. The message appends the raw log path when the backend keeps one artifact per session, so the refused text stays reachable. The JSONL backend refuses a foreign version straight from the raw header line, before validating this format version's header shape or decoding any event row — a structurally different future format still reports the upgrade direction, never "corrupt". An out-of-tree backend must enforce the equivalent direction-aware refusal at its own physical-format boundary. Design rationale and the deferred upgrader chain live in the [session-log-version-mechanism note](../../.agents/notes/implemented/architecture/2026-08-10-session-log-version-mechanism.md). +A backend refuses a log it cannot faithfully interpret with `SessionFormatUnsupportedError`, distinct from `SessionPersistenceCorruptionError` because nothing is damaged. Header-only listing rescans and classifies the numerically highest canonical generation before applying event rules. A supported older version is reported as migration-required; every event-body read runs inside the per-Session serialization chain, completes the static adjacent migration operation, leaves the source path, bytes, and inode unchanged, exclusively publishes only the final current filename, and reopens it before current restoration. A future highest generation refuses even when an older readable generation remains. Catalog generation and initialization reject a missing edge. An alpha body-migration refusal remains unsupported and publishes nothing. Equal-version reads allow an unknown event only when its envelope carries `ignorable: true`; historical v0 migration deliberately refuses every unknown type, including an ignorable one. Retention provides no automatic fallback and no downgrade compatibility promise. See the [released-format lifecycle](../../.agents/notes/implemented/architecture/2026-08-31-released-session-format-migrations.md) and [alpha historical-event rule](../../.agents/notes/implemented/architecture/2026-08-31-alpha-historical-unknown-event-refusal.md). ## `CreateSessionOptions` — seeding and metadata @@ -131,7 +130,7 @@ Plain replay is `ctx.sessions.create(id, { seed: seedEvents })`; a fork addition ## `SessionStorageMetadata` — logical header and inherited cut -Every persistence result that reads a Session body carries `SessionStorageMetadata`: the current logical header plus the separately validated inherited-event cut. Header-only listing intentionally returns only `SessionHeader`. +Every persistence result that reads a Session body carries `SessionStorageMetadata`: the current logical header plus the separately validated inherited-event cut. Header-only listing carries no inherited cut; its current and migration-required descriptors expose the latest `SessionHeader`, while unsupported and malformed descriptors do not. ```ts type-equiv /** Logical Session header paired with its exact inherited cut for body-bearing storage operations. */ @@ -145,12 +144,12 @@ interface SessionStorageMetadata { ## `SessionRawArtifact` — verbatim stored artifact text -A backend's own artifact text for one session, byte-identical to what it durably wrote (decoded from its physical encoding). `readRaw` returns it without reconstructing from parsed events, so backend-specific serialization (chunk packing, key order, line breaks) survives. Consumers first test `supportsRawArtifacts`: `false` means the backend does not provide this capability, while `readRaw(...) === undefined` means a supported backend has no materialized artifact for that session. +A backend's selected generation text for one Session, byte-identical to what it durably wrote after decoding the physical compression. `readRaw` returns it without reconstructing from parsed events, so backend-specific serialization (chunk packing, key order, line breaks) survives. JSONL sets `filename` to the selected logical basename without `.zstd`: `session.jsonl` for v0 and `session.vN.jsonl` for every positive generation. Consumers first test `supportsRawArtifacts`: `false` means the backend does not provide this capability, while `readRaw(...) === undefined` means a supported backend has no materialized generation for that Session. ```ts type-equiv /** A backend's own raw artifact text for one session, verbatim. */ interface SessionRawArtifact extends SessionStorageMetadata { - /** The artifact's base filename on disk, without any physical encoding suffix. */ + /** Selected generation basename; physical `.zstd` is omitted, while `.vN` remains. */ readonly filename: string /** The artifact's full text content, decoded from the backend's physical encoding. */ readonly content: string @@ -239,7 +238,14 @@ interface SessionEventSuffix extends SessionStorageMetadata { ## Lightweight source revisions -Consumers of derived state compare a cheap opaque revision before loading a full event log. The persistence backend owns its representation and changes it transactionally with append or mutating load repair; callers compare it only for equality. +Consumers of derived state compare a cheap opaque revision before loading a full event log. The persistence backend owns its representation and changes it transactionally with append, mutating load repair, or successor publication; callers compare it only for equality. Header-only listing rescans each Session directory and returns one tagged descriptor for its highest canonical generation: current and migration-required results expose the latest logical header, while unsupported and malformed results expose only that selected file's location and diagnosis. + +| Status | Header | Version and diagnosis facts | +|---|---|---| +| `current` | latest logical header | highest stored version, target version, exact selected location | +| `migration-required` | latest header translated in memory | highest stored version, target version, exact historical source location | +| `unsupported` | none | optional stored version, target version, required location and reason | +| `malformed` | none | target version, required location and reason | ```ts type-equiv /** @@ -249,13 +255,20 @@ Consumers of derived state compare a cheap opaque revision before loading a full type SessionPersistenceRevision = Branded<'SessionPersistenceRevision'> ``` +```ts type-equiv +/** One complete header-only listing result; event bodies are never read. */ +type SessionPersistenceListing = + | CurrentSessionPersistenceListing + | MigrationRequiredSessionPersistenceListing + | UnsupportedSessionPersistenceListing + | MalformedSessionPersistenceListing +``` + ```ts type-equiv /** Lightweight immutable source identity returned without loading a full log. */ -interface SessionPersistenceSnapshot { - /** Detached metadata for one materialized session. */ - header: SessionHeader - /** Opaque source-qualified token that changes whenever this stored log changes. */ - revision: SessionPersistenceRevision +type SessionPersistenceSnapshot = SessionPersistenceListing & { + /** Opaque source-qualified token that changes whenever this stored artifact changes. */ + readonly revision: SessionPersistenceRevision } ``` @@ -263,7 +276,7 @@ interface SessionPersistenceSnapshot { The shipped provider implements the abstract `SessionPersistence` contract (locate/create/append/prepare/load/inspect/readFrom/list/listSnapshots over `SessionEvent`, with optional cancellation on observation methods) and passes the shared `runPersistenceContract` suite: -- **[dsh-session-persistence-jsonl](../../packages/session/session-persistence-jsonl)** — an append-only logical JSONL log per session, stored as checksummed concatenated Zstandard frames by default or raw lines by configuration, with crash-safe atomic writes, interrupted-turn recovery, and a read/replay path. +- **[dsh-session-persistence-jsonl](../../packages/session/session-persistence-jsonl)** — canonical immutable generation filenames per Session, stored as checksummed concatenated Zstandard frames by default or raw lines by configuration, with append-only ordinary writes, exclusive successor publication beside retained predecessors, interrupted-turn recovery, and a read/replay path. @@ -281,9 +294,11 @@ Durable append-only session storage. Implementations preserve contiguous, lossle ```ts cordis-catalog /** - * Resolve this backend's independent local artifact for a session without - * reading, creating, flushing, or otherwise materializing it. A backend - * that does not own one artifact per Session returns `undefined`. + * Resolve this backend's current-generation target for a session without + * reading, creating, flushing, or otherwise materializing it. Historical + * generations may live at other immutable paths; listing descriptors carry + * the exact selected stored location. A backend without per-Session files + * returns `undefined`. * @param meta - the immutable session header whose artifact is requested. * @returns the backend-specific absolute location, when one exists. */ @@ -296,7 +311,9 @@ abstract locate(meta: SessionHeader): SessionLocation | undefined * reconstruction from parsed events, so it preserves backend-specific * serialization (chunk packing, key order, line breaks). Callers first test * {@link supportsRawArtifacts}; `undefined` then means only that the requested - * session has no materialized artifact. + * session has no materialized artifact. Reading a supported historical + * artifact leaves that generation untouched and exclusively publishes a + * separate repaired current successor; an already-current artifact is not rewritten. * @param _id - the persisted session to read (unused by the default: no * per-session artifact). * @param signal - optional cancellation for backend read work. @@ -343,15 +360,21 @@ abstract append(id: SessionId, events: readonly SessionEvent[]): Promise * their durable revision is still current; disposal releases an unpublished * reservation. Revision retries require the durable log to remain unchanged * for one read/check round trip; continuous external writers may delay completion. + * Preparing a supported historical artifact first persists its migration and + * current-format repair, while current input takes the no-write fast path. * @param id - persisted session to prepare. - * @param signal - optional cancellation for preparation work. + * @param signal - optional cancellation for this caller's wait. A shared + * preparation or historical migration already started for another observer + * may continue to completion. * @returns one owned unpublished Session preparation. */ async prepare(id: SessionId, signal?: AbortSignal): Promise /** - * Load an immutable balanced logical view and commit any required cold - * recovery. A complete interrupted final turn is preserved and durably + * Load an immutable balanced current logical view and commit any required cold + * recovery. A supported historical artifact remains immutable while a + * separate repaired current successor is published before restoration. A complete + * interrupted final turn is preserved and durably * closed with missing tool errors plus any open step and turn boundaries; * only a torn final record is discarded. Unknown versions and corruption in * the committed prefix reject. Implementations MUST NOT crash-repair an @@ -365,10 +388,12 @@ async prepare(id: SessionId, signal?: AbortSignal): Promise abstract load(id: SessionId): Promise /** - * Inspect an immutable logical session without committing recovery or - * publishing it. A cold complete interrupted turn receives synthetic closers - * in memory and a torn physical tail remains untouched. An already-live - * Session instead yields its current immutable snapshot, which may contain an + * Inspect an immutable current logical session without publishing a live + * Session. For an already-current cold artifact, a complete interrupted turn receives + * synthetic closers only in memory and a torn physical tail remains untouched. + * A supported historical artifact first publishes its separate repaired + * current successor, so inspection is not storage-read-only in that case. An + * already-live Session instead yields its current immutable snapshot, which may contain an * open turn and its `session/end-seed` boundary. Coordinator-backed * implementations retain the exact cold unpublished Session for bounded * reuse by a later {@link prepare}. A stale ready source is reloaded; a source @@ -376,7 +401,9 @@ abstract load(id: SessionId): Promise * may borrow its immutable view. Callers borrow only the immutable header and * log. Continuous external writers may delay revision convergence. * @param id - the persisted session to inspect. - * @param signal - optional cancellation for queued and backend read work. + * @param signal - optional cancellation for this observer. Shared cold + * preparation and an already-started historical migration may continue for + * another inspector or later resume. * @returns the validated header and current logical event log. */ abstract inspect(id: SessionId, signal?: AbortSignal): Promise @@ -385,9 +412,11 @@ abstract inspect(id: SessionId, signal?: AbortSignal): Promise @@ -397,8 +426,11 @@ abstract borrowSession(id: SessionId, signal?: AbortSignal): Promise +abstract list(signal?: AbortSignal): Promise /** * List materialized sessions with cheap per-log change tokens. * * Repeated observations of an unchanged log return the same revision. A - * successful mutating {@link load} repair changes the next listed revision. + * successful mutating {@link load} repair changes the next listed revision; + * so does migration publication from any supported historical body read. * Revisions also distinguish independently backed stores so backend-local * counters cannot compare equal across different persistence sources. * @param signal - optional cancellation for backend snapshot-listing work. - * @returns one header and opaque revision per materialized session without loading full logs. + * @returns one isolated current, migration-required, unsupported, or malformed + * descriptor plus its opaque revision per materialized artifact, without + * loading full logs. */ abstract listSnapshots(signal?: AbortSignal): Promise ``` diff --git a/docs/subsystems/persistence.zh.md b/docs/subsystems/persistence.zh.md index 08cc35d987..55aa5449d1 100644 --- a/docs/subsystems/persistence.zh.md +++ b/docs/subsystems/persistence.zh.md @@ -16,11 +16,11 @@ 修复仅适用于冷会话。对于活跃 id，`SessionPersistence.load(id)` 会等待权威内存快照完成持久化，并且只在日志平衡时返回；若活跃轮次仍未闭合，则拒绝操作，而不是添加合成的中断边界。HMR（热模块替换）会接管活跃前缀，而不会关闭其中正在进行的轮次。 -`SessionPersistence.inspect(id)` 会构造一个不可变的逻辑 Session，但不发布它，也不写入恢复内容。冷检查会在内存中配平中断的轮次，同时保持撕裂的物理尾部不变；检查已处于活跃状态的 Session 则借用其当前不可变快照，因此可能包含未闭合的轮次。使用协调器的实现会在有界 LRU 中保留这个精确的冷未发布 Session，因此重复历史读取与后续 `prepare(id)` 可复用同一次读取、解压、验证、冻结及 Session 构造。`prepare(id)` 会预留该 Session、提交待处理修复并返回可 dispose 的发布句柄；`load(id)` 使用相同机制提交修复，但不会发布 Session。该生命周期由 [Session 准备阶段决策](../../.agents/notes/implemented/architecture/2026-08-05-session-preparation.zh.md)定义。 +`SessionPersistence.inspect(id)` 会构造一个不可变的逻辑 Session，但不发布它。对于已经是当前格式的冷 generation，检查只在内存中配平中断轮次，不写入恢复内容，并保持撕裂的物理尾部不变。受支持的历史 generation 不同：串行化正文读取操作会先在保留源文件的同时，于新的版本限定路径发布已修复当前后继，因此检查永远不会暴露旧逻辑格式。检查已处于活跃状态的 Session 会借用其当前不可变快照，因此可能包含未闭合的轮次。使用协调器的实现会在有界 LRU 中保留这个精确的冷未发布 Session，因此重复历史读取与后续 `prepare(id)` 可复用同一次读取、解压、验证、冻结及 Session 构造。`prepare(id)` 会预留该 Session、提交待处理的当前格式修复并返回可 dispose 的发布句柄；`load(id)` 使用相同机制提交修复，但不会发布 Session。该生命周期由 [Session 准备阶段决策](../../.agents/notes/implemented/architecture/2026-08-05-session-preparation.zh.md)定义。 ## `SessionLocation`——可选的逐会话产物目标 -`SessionPersistence.locate(meta)` 会同步解析一个归后端所有的独立产物，而不会读取、创建或 flush 它。JSONL 返回其项目/会话目录内 transcript（文本记录）的绝对路径；不为每个会话各自拥有独立产物的后端返回 `undefined`。因此，返回的路径可能指向尚不存在的文件，或指向还不包含当前尚未 flush 轮次的文件；它是位置提示，不是授权或新鲜度保证。 +`SessionPersistence.locate(meta)` 会同步解析归后端所有的独立产物目标，而不会读取、创建或 flush 它。JSONL 根据 `meta.version` 派生规范文件名：v0 是 `session.jsonl[.zstd]`，每个正版本都是小写 `session.vN.jsonl[.zstd]`。不为每个 Session 各自拥有独立产物的后端返回 `undefined`。返回路径可能尚不存在，也不说明目录中是否已经存有另一个更高 generation；它是版本限定的位置提示，不是授权、发现或新鲜度保证。仅 header 的列表负责发现，并在 `listing.location` 中返回精确选定 generation。 ```ts type-equiv /** @@ -50,11 +50,10 @@ interface SessionLocation { */ interface SessionHeader { /** - * On-disk format version, stamped from {@link SESSION_FORMAT_VERSION} when the - * session is created. A persistence backend rejects any other version on load - * (no migration — see the constant). + * Current logical format version, stamped from {@link SESSION_FORMAT_VERSION}. + * Historical physical headers are translated before entering this interface. */ - readonly version: number + readonly version: typeof SESSION_FORMAT_VERSION /** The session's id (mirrors the {@link Session}'s id). */ readonly id: SessionId /** Non-negative safe-integer Unix epoch milliseconds when the session was created. */ @@ -91,7 +90,7 @@ interface SessionHeader { ## 格式拒绝：本构建无法可靠读取的日志 -后端用 `SessionFormatUnsupportedError` 拒绝无法可靠解读的日志，它与 `SessionPersistenceCorruptionError` 区分，因为数据没有损坏。header 的 `version` 比 `SESSION_FORMAT_VERSION` 新时，消息说明方向（"由更新的 harness 写入，请升级 harness 后打开"）；比它旧时说明本构建没有升级路径。经过 legacy 形状归一化后，本构建生成词汇表（`KNOWN_SESSION_EVENT_TYPES`，由 `gen-persistence-catalog` 生成）之外的事件类型同样被拒绝，除非该事件的信封带 `ignorable: true`：静默跳过一个不认识的必需事件可能改变日志其余部分的解读方式。后端为每个会话保留独立文件时，消息附上原始日志路径，被拒绝的文本仍然可读。JSONL 后端直接从原始 header 行拒绝外来版本，先于本格式版本的 header 形状校验和任何事件行解码，因此结构完全不同的未来格式仍会报告升级方向，绝不会报"损坏"。仓库外后端必须在自己的物理格式入口执行等价的方向感知拒绝。设计理由与推迟建设的升级器链见 [session-log 版本机制 Agent Note](../../.agents/notes/implemented/architecture/2026-08-10-session-log-version-mechanism.zh.md)。 +后端用 `SessionFormatUnsupportedError` 拒绝无法可靠解读的日志，它与 `SessionPersistenceCorruptionError` 区分，因为数据没有损坏。仅 header 的列表会重新扫描，并在应用事件规则前分类数值最高的规范 generation。受支持的旧版本会报告为 migration-required；每个事件正文读取都在逐 Session 串行链中运行，完成静态相邻迁移操作，保持源路径、字节与 inode 不变，以排他方式只发布最终当前文件名，并在当前恢复前重新打开。即使仍有可读旧 generation，最高 generation 是未来版本时也会拒绝。catalog 生成与初始化会拒绝缺失的迁移边。alpha 正文迁移拒绝仍是 unsupported，且不发布任何文件。同版本读取只有在信封带 `ignorable: true` 时才允许未知事件；历史 v0 迁移有意拒绝每个未知类型，包括 ignorable 类型。保留不提供自动 fallback，也不承诺 downgrade compatibility。见[已发布格式生命周期](../../.agents/notes/implemented/architecture/2026-08-31-released-session-format-migrations.zh.md)与 [alpha 历史事件规则](../../.agents/notes/implemented/architecture/2026-08-31-alpha-historical-unknown-event-refusal.zh.md)。 ## `CreateSessionOptions`：seed 与元数据 @@ -131,7 +130,7 @@ interface CreateSessionOptions { ## `SessionStorageMetadata`：逻辑 header 与继承 cut -每个读取 Session 正文的持久化结果都携带 `SessionStorageMetadata`：当前逻辑 header，以及单独校验的继承事件 cut。仅 header 的列表操作有意只返回 `SessionHeader`。 +每个读取 Session 正文的持久化结果都携带 `SessionStorageMetadata`：当前逻辑 header，以及单独校验的继承事件 cut。仅 header 的列表不携带继承 cut；其中 current 与 migration-required descriptor 暴露最新 `SessionHeader`，unsupported 与 malformed descriptor 则不暴露。 ```ts type-equiv /** Logical Session header paired with its exact inherited cut for body-bearing storage operations. */ @@ -145,12 +144,12 @@ interface SessionStorageMetadata { ## `SessionRawArtifact`——逐字存储工件文本 -后端为单个会话自持的工件文本，与其持久化写入的字节逐字一致（按物理编码解码）。`readRaw` 返回它而不从解析后事件重建，因此后端特定的序列化（chunk 打包、键序、换行）得以保留。Consumer 须先检查 `supportsRawArtifacts`：`false` 表示后端不提供此能力，而 `readRaw(...) === undefined` 表示受支持的后端没有该会话的已实体化工件。 +后端为一个 Session 选定的 generation 文本，在解码物理压缩后与其持久写入内容逐字节相同。`readRaw` 不通过已解析事件重建就返回该文本，因此后端特定的序列化（chunk 打包、key 顺序、换行）都会保留。JSONL 把 `filename` 设为不带 `.zstd` 的选定逻辑 basename：v0 为 `session.jsonl`，每个正 generation 为 `session.vN.jsonl`。消费方先检查 `supportsRawArtifacts`：`false` 表示后端不提供该能力，而 `readRaw(...) === undefined` 表示支持该能力的后端中不存在该 Session 的已物化 generation。 ```ts type-equiv /** A backend's own raw artifact text for one session, verbatim. */ interface SessionRawArtifact extends SessionStorageMetadata { - /** The artifact's base filename on disk, without any physical encoding suffix. */ + /** Selected generation basename; physical `.zstd` is omitted, while `.vN` remains. */ readonly filename: string /** The artifact's full text content, decoded from the backend's physical encoding. */ readonly content: string @@ -239,7 +238,14 @@ interface SessionEventSuffix extends SessionStorageMetadata { ## 轻量源修订号 -派生状态的消费方会在加载完整事件日志之前比较一个低开销的不透明修订号。其表示由持久化后端拥有，并随 append 或会修改数据的 load 修复以事务方式改变；调用方仅比较修订号是否相等。 +派生状态的消费方会在加载完整事件日志之前比较一个低开销的不透明修订号。其表示由持久化后端拥有，并随 append、会修改数据的 load 修复或后继发布以事务方式改变；调用方仅比较修订号是否相等。仅 header 的列表会重新扫描每个 Session 目录，并为其最高规范 generation 返回一个带标签 descriptor：`current` 与 `migration-required` 结果暴露最新逻辑 header，`unsupported` 与 `malformed` 结果则只暴露该选定文件的位置和诊断。 + +| 状态 | Header | 版本与诊断事实 | +|---|---|---| +| `current` | 最新逻辑 header | 最高存储版本、目标版本、精确选定位置 | +| `migration-required` | 在内存中转换后的最新 header | 最高存储版本、目标版本、精确历史源位置 | +| `unsupported` | 无 | 可选存储版本、目标版本、必需的位置与原因 | +| `malformed` | 无 | 目标版本、必需的位置与原因 | ```ts type-equiv /** @@ -249,13 +255,20 @@ interface SessionEventSuffix extends SessionStorageMetadata { type SessionPersistenceRevision = Branded<'SessionPersistenceRevision'> ``` +```ts type-equiv +/** One complete header-only listing result; event bodies are never read. */ +type SessionPersistenceListing = + | CurrentSessionPersistenceListing + | MigrationRequiredSessionPersistenceListing + | UnsupportedSessionPersistenceListing + | MalformedSessionPersistenceListing +``` + ```ts type-equiv /** Lightweight immutable source identity returned without loading a full log. */ -interface SessionPersistenceSnapshot { - /** Detached metadata for one materialized session. */ - header: SessionHeader - /** Opaque source-qualified token that changes whenever this stored log changes. */ - revision: SessionPersistenceRevision +type SessionPersistenceSnapshot = SessionPersistenceListing & { + /** Opaque source-qualified token that changes whenever this stored artifact changes. */ + readonly revision: SessionPersistenceRevision } ``` @@ -263,7 +276,7 @@ interface SessionPersistenceSnapshot { 随产品交付的 provider 实现抽象 `SessionPersistence` 约定（在 `SessionEvent` 上执行 locate/create/append/prepare/load/inspect/readFrom/list/listSnapshots，观察方法可选支持取消），并通过共享的 `runPersistenceContract` 套件： -- **[dsh-session-persistence-jsonl](../../packages/session/session-persistence-jsonl)**——逐会话仅追加的逻辑 JSONL 日志，默认存储为带 checksum 的连续 Zstandard frame，也可配置为原始行；支持崩溃安全的原子写入、被中断轮次的恢复以及读取/回放路径。 +- **[dsh-session-persistence-jsonl](../../packages/session/session-persistence-jsonl)**——逐 Session 使用规范不可变 generation 文件名，默认存储为带 checksum 的连续 Zstandard frame，也可配置为原始行；普通写入仅追加，在保留前任的同时排他发布后继，并支持中断轮次恢复与读取／回放。 @@ -281,9 +294,11 @@ Durable append-only session storage. Implementations preserve contiguous, lossle ```ts cordis-catalog /** - * Resolve this backend's independent local artifact for a session without - * reading, creating, flushing, or otherwise materializing it. A backend - * that does not own one artifact per Session returns `undefined`. + * Resolve this backend's current-generation target for a session without + * reading, creating, flushing, or otherwise materializing it. Historical + * generations may live at other immutable paths; listing descriptors carry + * the exact selected stored location. A backend without per-Session files + * returns `undefined`. * @param meta - the immutable session header whose artifact is requested. * @returns the backend-specific absolute location, when one exists. */ @@ -296,7 +311,9 @@ abstract locate(meta: SessionHeader): SessionLocation | undefined * reconstruction from parsed events, so it preserves backend-specific * serialization (chunk packing, key order, line breaks). Callers first test * {@link supportsRawArtifacts}; `undefined` then means only that the requested - * session has no materialized artifact. + * session has no materialized artifact. Reading a supported historical + * artifact leaves that generation untouched and exclusively publishes a + * separate repaired current successor; an already-current artifact is not rewritten. * @param _id - the persisted session to read (unused by the default: no * per-session artifact). * @param signal - optional cancellation for backend read work. @@ -343,15 +360,21 @@ abstract append(id: SessionId, events: readonly SessionEvent[]): Promise * their durable revision is still current; disposal releases an unpublished * reservation. Revision retries require the durable log to remain unchanged * for one read/check round trip; continuous external writers may delay completion. + * Preparing a supported historical artifact first persists its migration and + * current-format repair, while current input takes the no-write fast path. * @param id - persisted session to prepare. - * @param signal - optional cancellation for preparation work. + * @param signal - optional cancellation for this caller's wait. A shared + * preparation or historical migration already started for another observer + * may continue to completion. * @returns one owned unpublished Session preparation. */ async prepare(id: SessionId, signal?: AbortSignal): Promise /** - * Load an immutable balanced logical view and commit any required cold - * recovery. A complete interrupted final turn is preserved and durably + * Load an immutable balanced current logical view and commit any required cold + * recovery. A supported historical artifact remains immutable while a + * separate repaired current successor is published before restoration. A complete + * interrupted final turn is preserved and durably * closed with missing tool errors plus any open step and turn boundaries; * only a torn final record is discarded. Unknown versions and corruption in * the committed prefix reject. Implementations MUST NOT crash-repair an @@ -365,10 +388,12 @@ async prepare(id: SessionId, signal?: AbortSignal): Promise abstract load(id: SessionId): Promise /** - * Inspect an immutable logical session without committing recovery or - * publishing it. A cold complete interrupted turn receives synthetic closers - * in memory and a torn physical tail remains untouched. An already-live - * Session instead yields its current immutable snapshot, which may contain an + * Inspect an immutable current logical session without publishing a live + * Session. For an already-current cold artifact, a complete interrupted turn receives + * synthetic closers only in memory and a torn physical tail remains untouched. + * A supported historical artifact first publishes its separate repaired + * current successor, so inspection is not storage-read-only in that case. An + * already-live Session instead yields its current immutable snapshot, which may contain an * open turn and its `session/end-seed` boundary. Coordinator-backed * implementations retain the exact cold unpublished Session for bounded * reuse by a later {@link prepare}. A stale ready source is reloaded; a source @@ -376,7 +401,9 @@ abstract load(id: SessionId): Promise * may borrow its immutable view. Callers borrow only the immutable header and * log. Continuous external writers may delay revision convergence. * @param id - the persisted session to inspect. - * @param signal - optional cancellation for queued and backend read work. + * @param signal - optional cancellation for this observer. Shared cold + * preparation and an already-started historical migration may continue for + * another inspector or later resume. * @returns the validated header and current logical event log. */ abstract inspect(id: SessionId, signal?: AbortSignal): Promise @@ -385,9 +412,11 @@ abstract inspect(id: SessionId, signal?: AbortSignal): Promise @@ -397,8 +426,11 @@ abstract borrowSession(id: SessionId, signal?: AbortSignal): Promise +abstract list(signal?: AbortSignal): Promise /** * List materialized sessions with cheap per-log change tokens. * * Repeated observations of an unchanged log return the same revision. A - * successful mutating {@link load} repair changes the next listed revision. + * successful mutating {@link load} repair changes the next listed revision; + * so does migration publication from any supported historical body read. * Revisions also distinguish independently backed stores so backend-local * counters cannot compare equal across different persistence sources. * @param signal - optional cancellation for backend snapshot-listing work. - * @returns one header and opaque revision per materialized session without loading full logs. + * @returns one isolated current, migration-required, unsupported, or malformed + * descriptor plus its opaque revision per materialized artifact, without + * loading full logs. */ abstract listSnapshots(signal?: AbortSignal): Promise ``` diff --git a/docs/subsystems/session-query.i18n.yaml b/docs/subsystems/session-query.i18n.yaml index 6ade7bbe1b..7de15cc6bf 100644 --- a/docs/subsystems/session-query.i18n.yaml +++ b/docs/subsystems/session-query.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write docs/subsystems/session-query.md -session-query.md: 4317fa3d706dd40c596114a396ba746da67a979f -session-query.zh.md: 4b67706aea3d3f73304e4c8ad0c689a4c77a13a1 +session-query.md: a605819e846d14ca2b7eb3fafaf8675f7bd6805c +session-query.zh.md: 93262697e6471d659c22ef0249c81ba1ea887fbf diff --git a/docs/subsystems/session-query.md b/docs/subsystems/session-query.md index 4317fa3d70..a605819e84 100644 --- a/docs/subsystems/session-query.md +++ b/docs/subsystems/session-query.md @@ -8,7 +8,7 @@ Source: [`packages/session-query/session-query/src/types.ts`](../../packages/ses ## Logical records -`SessionRecord` is returned by the cross-corpus list. It exposes source availability independently from the cloned live-preferred header. `SessionEventRecord` is a lightweight raw-log projection; classification uses the same `foldSurface()` transitions as model-history derivation. +`SessionRecord` is returned by the cross-corpus list. It exposes source availability independently from the cloned live-preferred header and carries the exact selected persisted-generation location when listing provides one. `SessionEventRecord` is a lightweight raw-log projection; classification uses the same `foldSurface()` transitions as model-history derivation. ```ts type-equiv /** Whether an event is current model context, replaced context, or raw-log-only. */ @@ -20,6 +20,8 @@ type SessionEventSurface = 'current' | 'shadowed' | 'log-only' interface SessionRecord { /** Cloned session header selected from the live-preferred corpus. */ header: SessionHeader + /** Exact listed artifact location, when persistence exposes one. */ + location?: SessionLocation /** Whether the id currently exists in `ctx.sessions`. */ live: boolean /** Whether the active persistence backend currently materializes the id. */ diff --git a/docs/subsystems/session-query.zh.md b/docs/subsystems/session-query.zh.md index 4b67706aea..93262697e6 100644 --- a/docs/subsystems/session-query.zh.md +++ b/docs/subsystems/session-query.zh.md @@ -8,7 +8,7 @@ ## 逻辑记录 -`SessionRecord` 由全语料库列表返回。它除了克隆的、优先取自 live 源的 header 外，还单独公开各源的可用性。`SessionEventRecord` 是轻量的原始日志投影；分类使用与模型历史推导相同的 `foldSurface()` 状态转换。 +`SessionRecord` 由全语料库列表返回。它除了克隆的、优先取自 live 源的 header 外，还单独公开各源的可用性；listing 提供位置时，它也携带精确选定的持久 generation location。`SessionEventRecord` 是轻量的原始日志投影；分类使用与模型历史推导相同的 `foldSurface()` 状态转换。 ```ts type-equiv /** Whether an event is current model context, replaced context, or raw-log-only. */ @@ -20,6 +20,8 @@ type SessionEventSurface = 'current' | 'shadowed' | 'log-only' interface SessionRecord { /** Cloned session header selected from the live-preferred corpus. */ header: SessionHeader + /** Exact listed artifact location, when persistence exposes one. */ + location?: SessionLocation /** Whether the id currently exists in `ctx.sessions`. */ live: boolean /** Whether the active persistence backend currently materializes the id. */ diff --git a/docs/subsystems/session-reference.i18n.yaml b/docs/subsystems/session-reference.i18n.yaml index 29ba36ab67..109af5852e 100644 --- a/docs/subsystems/session-reference.i18n.yaml +++ b/docs/subsystems/session-reference.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write docs/subsystems/session-reference.md -session-reference.md: 1dd5cc1ee8c594b34015f9bf2d765f68621b86a1 -session-reference.zh.md: 75b5018a6afbd1bbe21f303013e0e7fa0d1f89ab +session-reference.md: 4921c29d25083a75c7a4fed8dd14202a4da9b2d2 +session-reference.zh.md: 7cd03ea31258eadd207a5dcbbcbf208f291021f8 diff --git a/docs/subsystems/session-reference.md b/docs/subsystems/session-reference.md index 1dd5cc1ee8..4921c29d25 100644 --- a/docs/subsystems/session-reference.md +++ b/docs/subsystems/session-reference.md @@ -68,7 +68,31 @@ interface SessionReferenceMentionCandidate extends SessionReferenceCandidate { ## Prepared messages -Preparation preserves readable current-message content and returns at most one aggregated context. +Preparation preserves readable current-message content and returns at most one aggregated context. Its durable source records keep `capturedThroughSeq` as a coordinate in the referenced Session's original generation; they never reinterpret it as a seq in the containing Session. `capturedFormatVersion` records that generation, with absence meaning released format v0. + +```ts type-equiv +/** Durable source session, cited event seqs, and snapshot facts for prepared cross-session context. */ +interface SessionReferenceSource { + kind: 'session-reference' + /** Material lifted out of another session's log (`recall` context form). */ + form: 'recall' + version: 1 + references: { + sessionId: string + label: string + /** Source Session format generation; absence identifies version 0. */ + capturedFormatVersion?: number + capturedThroughSeq: OptionalSessionSeq + compacted: boolean + originalMessages: number + retainedMessages: number + omittedMessages: number + omittedBytes: number + truncated: boolean + inputIndex: number + }[] +} +``` ```ts type-equiv /** Direct message content and optional referenced-session context. */ diff --git a/docs/subsystems/session-reference.zh.md b/docs/subsystems/session-reference.zh.md index 75b5018a6a..7cd03ea312 100644 --- a/docs/subsystems/session-reference.zh.md +++ b/docs/subsystems/session-reference.zh.md @@ -68,7 +68,31 @@ interface SessionReferenceMentionCandidate extends SessionReferenceCandidate { ## 准备后的消息 -准备过程保留可读的当前消息内容，并最多返回一个聚合上下文。 +准备过程保留可读的当前消息内容，并最多返回一个聚合上下文。其持久 source 记录会把 `capturedThroughSeq` 保留为被引用 Session 原始 generation 中的坐标，绝不会把它重新解释为所在 Session 的 seq。`capturedFormatVersion` 记录该 generation；缺失表示已发布格式 v0。 + +```ts type-equiv +/** Durable source session, cited event seqs, and snapshot facts for prepared cross-session context. */ +interface SessionReferenceSource { + kind: 'session-reference' + /** Material lifted out of another session's log (`recall` context form). */ + form: 'recall' + version: 1 + references: { + sessionId: string + label: string + /** Source Session format generation; absence identifies version 0. */ + capturedFormatVersion?: number + capturedThroughSeq: OptionalSessionSeq + compacted: boolean + originalMessages: number + retainedMessages: number + omittedMessages: number + omittedBytes: number + truncated: boolean + inputIndex: number + }[] +} +``` ```ts type-equiv /** Direct message content and optional referenced-session context. */ diff --git a/docs/subsystems/session-telemetry.i18n.yaml b/docs/subsystems/session-telemetry.i18n.yaml index 9e8a3fa318..b21cceb024 100644 --- a/docs/subsystems/session-telemetry.i18n.yaml +++ b/docs/subsystems/session-telemetry.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write docs/subsystems/session-telemetry.md -session-telemetry.md: 718fe5dabfd1c059a6a02077407480e96bdd271a -session-telemetry.zh.md: d4081664a121877e36a2cba123e9c5840108b529 +session-telemetry.md: 5f589d00940039cdfd6b7d50f8689b4280b94c83 +session-telemetry.zh.md: bb08eb348874c0c3bae4cacdec16ac954193726e diff --git a/docs/subsystems/session-telemetry.md b/docs/subsystems/session-telemetry.md index 718fe5dabf..5f589d0094 100644 --- a/docs/subsystems/session-telemetry.md +++ b/docs/subsystems/session-telemetry.md @@ -2,7 +2,7 @@ English | [中文](session-telemetry.zh.md) -Outbound session reporting is split as a [capability seam](../capability-seams.md): the Service Definition and capture coordinator ([dsh-session-telemetry](../../packages/session/session-telemetry), `ctx.sessionTelemetry`) own the capture points, fixed chunk projection, `session-telemetry/record` redaction waterfall, handoff cursor, and minimal backend contract; the Service Provider a deployment loads ([dsh-session-telemetry-otel](../../packages/session/session-telemetry-otel)) is the OpenTelemetry JS SDK's log pipeline configured verbatim. It is one optional capability, not part of the agent-loop spine, and nothing here reaches a model request. The boundary axiom — the harness's aspect ends at `emit()`; batching, retry, queueing, and loss policy belong to the reporting SDK — and the rejected alternatives are pinned in the [revival Agent Note](../../.agents/notes/implemented/feature/2026-07-23-session-telemetry-otel-revival.md); the capture points, cursor, and projection contracts live in the [Service Definition README](../../packages/session/session-telemetry/README.md). +Outbound session reporting is split as a [capability seam](../capability-seams.md): the Service Definition and capture coordinator ([dsh-session-telemetry](../../packages/session/session-telemetry), `ctx.sessionTelemetry`) own complete canonical-event capture, the `session-telemetry/record` redaction waterfall, the handoff cursor, and the minimal backend contract; the Service Provider a deployment loads ([dsh-session-telemetry-otel](../../packages/session/session-telemetry-otel)) is the OpenTelemetry JS SDK's log pipeline configured verbatim. It is one optional capability, not part of the agent-loop spine, and nothing here reaches a model request. The boundary axiom — the harness's aspect ends at `emit()`; batching, retry, queueing, and loss policy belong to the reporting SDK — and the rejected alternatives are pinned in the [revival Agent Note](../../.agents/notes/implemented/feature/2026-07-23-session-telemetry-otel-revival.md); the capture and cursor contracts live in the [Service Definition README](../../packages/session/session-telemetry/README.md). Source: [`packages/session/session-telemetry/src/index.ts`](../../packages/session/session-telemetry/src/index.ts) @@ -37,8 +37,9 @@ interface SessionTelemetryRecord { severity: SessionTelemetrySeverity /** * Identity attributes, deliberately minimal: ledger records carry - * `session.id`, `event.type`, `event.seq`, plus `session.cwd` / - * `session.parent_id` / `session.seed_length` when the header has them; + * `session.id`, `session.format_version`, `event.type`, `event.seq`, plus optional + * `session.cwd` / `session.parent_id`; a seeded Session also carries + * `session.seed_length` from its exact inherited event count; * ops records carry `telemetry.op`, `session.id`, and (for `agent-error`) * `agent.id`, `turn`, `step`, `error.name`. Anything recoverable from the * body is intentionally NOT duplicated here. @@ -54,7 +55,7 @@ interface SessionTelemetryRecord { } ``` -Only the first `assistant/chunk` of each `(turn, step)` ships — the stream-started signal; the rest drop at capture, so `seq` gaps are routine on the wire and never a loss signal. Every other [session event](session.md) type, including plugin-merged ones the seam never heard of, passes through whole. Delivery is best-effort: the cursor marks handed-off, not delivered, records can be lost (crash, reload window) and duplicated (cursor-less re-adoption, SDK retries), so receivers dedupe ledger records on `(session.id, event.seq)`; ops records deliberately omit that identity — they are signals to alert on, not entries to sum, and tolerate duplicates instead. +Every canonical [session event](session.md), including every `assistant/chunk` and plugin-merged type the seam never heard of, passes through whole as one ordered ledger record. A new Session object replays its complete log from seq 0, including constructor seed history; re-adopting the same object resumes after its handoff cursor. Delivery is best-effort: the cursor marks handed-off, not delivered, and records can be lost (crash, reload window) or duplicated (new-object replay, SDK retries), so receivers dedupe ledger records on `(session.id, session.format_version, event.seq)`; ops records deliberately omit that identity — they are signals to alert on, not entries to sum, and tolerate duplicates instead. ## The sharing disclosure @@ -122,7 +123,7 @@ interface SessionTelemetrySink { ## The redact waterfall: `session-telemetry/record` -Every record passes the `session-telemetry/record` [waterfall](../cordis-primer.md#cordis-waterfall-semantics) between projection and `emit()` ([event entry](#session-telemetryrecord--waterfall)). The seam ships NO rules of its own: with no listener mounted, records reach the backend exactly as captured, so exported data is precisely as clean as the rules a deployment mounts. Listeners stack by transforming `next()`'s return value; returning without `next()` replaces everything beneath; a throwing listener withholds that one record fail-closed inside the coordinator's containment. Redaction applies to the exported copy only — the canonical session log is never rewritten. +Every record passes the `session-telemetry/record` [waterfall](../cordis-primer.md#cordis-waterfall-semantics) between the canonical-event copy and `emit()` ([event entry](#session-telemetryrecord--waterfall)). The seam ships NO rules of its own: with no listener mounted, records reach the backend exactly as captured, so exported data is precisely as clean as the rules a deployment mounts. Listeners stack by transforming `next()`'s return value; returning without `next()` replaces everything beneath; a throwing listener withholds that one record fail-closed inside the coordinator's containment. Redaction applies to the exported copy only — the canonical session log is never rewritten. diff --git a/docs/subsystems/session-telemetry.zh.md b/docs/subsystems/session-telemetry.zh.md index d4081664a1..bb08eb3488 100644 --- a/docs/subsystems/session-telemetry.zh.md +++ b/docs/subsystems/session-telemetry.zh.md @@ -2,7 +2,7 @@ [English](session-telemetry.md) | 中文 -对外的会话上报拆分为一项[能力 seam](../capability-seams.zh.md)：Service Definition 与捕获协调器（[dsh-session-telemetry](../../packages/session/session-telemetry)，`ctx.sessionTelemetry`）拥有捕获点、固定分片投影、`session-telemetry/record` 脱敏 waterfall（瀑布式事件）、handoff 游标与最小后端约定；部署方加载的 Service Provider（[dsh-session-telemetry-otel](../../packages/session/session-telemetry-otel)）则是原样配置的 OpenTelemetry JS SDK 日志流水线。它是一项可选能力，不属于 agent loop（智能体循环）主干，这里也没有任何内容会进入模型请求。边界公理（harness 的职责止于 `emit()`；批处理、重试、排队与丢失策略都属于上报 SDK）连同被否决的替代方案，均已在[复活 Agent Note](../../.agents/notes/implemented/feature/2026-07-23-session-telemetry-otel-revival.zh.md)中定案；捕获点、游标与投影的约定见 [Service Definition README](../../packages/session/session-telemetry/README.zh.md)。 +对外的会话上报拆分为一项[能力 seam](../capability-seams.zh.md)：Service Definition 与捕获协调器（[dsh-session-telemetry](../../packages/session/session-telemetry)，`ctx.sessionTelemetry`）拥有完整的权威事件捕获、`session-telemetry/record` 脱敏 waterfall（瀑布式事件）、handoff 游标与最小后端约定；部署方加载的 Service Provider（[dsh-session-telemetry-otel](../../packages/session/session-telemetry-otel)）则是原样配置的 OpenTelemetry JS SDK 日志流水线。它是一项可选能力，不属于 agent loop（智能体循环）主干，这里也没有任何内容会进入模型请求。边界公理（harness 的职责止于 `emit()`；批处理、重试、排队与丢失策略都属于上报 SDK）连同被否决的替代方案，均已在[复活 Agent Note](../../.agents/notes/implemented/feature/2026-07-23-session-telemetry-otel-revival.zh.md)中定案；捕获与游标约定见 [Service Definition README](../../packages/session/session-telemetry/README.zh.md)。 源码：[`packages/session/session-telemetry/src/index.ts`](../../packages/session/session-telemetry/src/index.ts) @@ -37,8 +37,9 @@ interface SessionTelemetryRecord { severity: SessionTelemetrySeverity /** * Identity attributes, deliberately minimal: ledger records carry - * `session.id`, `event.type`, `event.seq`, plus `session.cwd` / - * `session.parent_id` / `session.seed_length` when the header has them; + * `session.id`, `session.format_version`, `event.type`, `event.seq`, plus optional + * `session.cwd` / `session.parent_id`; a seeded Session also carries + * `session.seed_length` from its exact inherited event count; * ops records carry `telemetry.op`, `session.id`, and (for `agent-error`) * `agent.id`, `turn`, `step`, `error.name`. Anything recoverable from the * body is intentionally NOT duplicated here. @@ -54,7 +55,7 @@ interface SessionTelemetryRecord { } ``` -每个 `(turn, step)` 只发出第一条 `assistant/chunk`，即「流已开始」的信号；其余分片在捕获时丢弃，因此传输中的 `seq` 缺口是常态，绝不是数据丢失的信号。其他所有[会话事件](session.zh.md)类型都会完整透传，包括该 seam 从未听说过、由插件合并进来的事件类型。投递是尽力而为的：游标标记的是「已交接」而非「已送达」，记录可能丢失（崩溃、重载窗口）也可能重复（无游标的重新接管、SDK 重试），因此接收端对 ledger 记录基于 `(session.id, event.seq)` 去重；ops 记录刻意省略这类标识——它们是用于告警的信号，而非用于累加的条目，重复被容忍而非被去重。 +每条权威[会话事件](session.zh.md)都会完整透传为一条有序 ledger 记录，包括每条 `assistant/chunk` 以及该 seam 从未听说过、由插件合并进来的类型。新 Session 对象会从 seq 0 回放完整日志，包括构造 seed 历史；重新收养同一对象时会从 handoff 游标之后继续。投递是尽力而为的：游标标记的是「已交接」而非「已送达」，记录可能丢失（崩溃、重载窗口）也可能重复（新对象回放、SDK 重试），因此接收端对 ledger 记录基于 `(session.id, session.format_version, event.seq)` 去重；ops 记录刻意省略这类标识——它们是用于告警的信号，而非用于累加的条目，重复被容忍而非被去重。 ## 共享披露 @@ -122,7 +123,7 @@ interface SessionTelemetrySink { ## 脱敏 waterfall：`session-telemetry/record` -每条记录在投影与 `emit()` 之间都要经过 `session-telemetry/record` [waterfall](../cordis-primer.zh.md#cordis-waterfall-semantics)（[事件条目](#session-telemetryrecord--waterfall)）。seam 自身不带任何规则：未挂载监听器时，记录以捕获时的原样到达后端；导出数据能干净到什么程度，恰恰取决于部署方挂载了什么规则。监听器通过变换 `next()` 的返回值来堆叠；不调用 `next()` 就返回，即替换其下方的全部逻辑；抛出异常的监听器会在协调器的隔离范围内以 fail-closed 方式扣下这一条记录。脱敏只作用于导出副本；权威会话日志永不改写。 +每条记录在权威事件副本与 `emit()` 之间都要经过 `session-telemetry/record` [waterfall](../cordis-primer.zh.md#cordis-waterfall-semantics)（[事件条目](#session-telemetryrecord--waterfall)）。seam 自身不带任何规则：未挂载监听器时，记录以捕获时的原样到达后端；导出数据能干净到什么程度，恰恰取决于部署方挂载了什么规则。监听器通过变换 `next()` 的返回值来堆叠；不调用 `next()` 就返回，即替换其下方的全部逻辑；抛出异常的监听器会在协调器的隔离范围内以 fail-closed 方式扣下这一条记录。脱敏只作用于导出副本；权威会话日志永不改写。 diff --git a/docs/subsystems/session.i18n.yaml b/docs/subsystems/session.i18n.yaml index 51e2da2af5..a18329547e 100644 --- a/docs/subsystems/session.i18n.yaml +++ b/docs/subsystems/session.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write docs/subsystems/session.md -session.md: 40012d1fc7b5be602dd3783f1df24f8f61f8a969 -session.zh.md: 9d4f6c3ce90913e5236b011e025d8299b3192a57 +session.md: 176205799690f9099d9e9231cd5a2d23bc639684 +session.zh.md: 657d500d131f56469078631fc17b6db9ba404125 diff --git a/docs/subsystems/session.md b/docs/subsystems/session.md index 40012d1fc7..1762057996 100644 --- a/docs/subsystems/session.md +++ b/docs/subsystems/session.md @@ -395,9 +395,10 @@ declare class Session { * The first seq appended IN THIS PROCESS: the length of the constructor * seed (0 without one). Events with smaller seq values entered through * construction — replay, fork, or resume — and were never published on the - * `session/event` firehose (constructor seeds do not emit), so consumers - * that replay the log as a publication substitute (telemetry adoption) - * start here. Distinct from {@link inheritedEventCount}, the DURABLE + * `session/event` firehose (constructor seeds do not emit). This offset marks + * the constructor-input boundary for lifecycle ownership and persistence + * adoption; consumers that need complete canonical history still start at + * seq 0. Distinct from {@link inheritedEventCount}, the DURABLE * fork-lineage cut: a resumed session's constructor seed is its full stored * log, while the inherited count keeps the original fork value — this field is the * in-process construction fact. @@ -569,7 +570,7 @@ declare class Session { - `tool/result` → a user message carrying a `tool-result` block. - `user/message` (injected context, i.e. non-`user` source) → a user-role message carrying its `content` verbatim at its chronological position; its typed source names the producer and carries any producer-specific data. -Everything else (`turn/*`, `step/*`, plugin-owned `llm/retry`) is structural and does not project into a message. Token accounting reads per-step `assistant/chunk { type: 'usage' }` records and treats `assistant/message.usage` as the committed-step fallback when no usage chunk exists; failed model-request attempts have no assistant message, so their usage chunk is the durable accounting record. Because this unreleased format intentionally has no compatibility promise, seed/load validation rejects request headers and assistant messages that omit provider/model instead of guessing a route for historical data. +Everything else (`turn/*`, `step/*`, plugin-owned `llm/retry`) is structural and does not project into a message. Token accounting reads per-step `assistant/chunk { type: 'usage' }` records and treats `assistant/message.usage` as the committed-step fallback when no usage chunk exists; failed model-request attempts have no assistant message, so their usage chunk is the durable accounting record. Current logical validation rejects request headers and assistant messages that omit provider/model instead of guessing a route; supported historical representations are normalized and validated by their adjacent format edge before a current Session exists. ## Live-session fork API diff --git a/docs/subsystems/session.zh.md b/docs/subsystems/session.zh.md index 9d4f6c3ce9..657d500d13 100644 --- a/docs/subsystems/session.zh.md +++ b/docs/subsystems/session.zh.md @@ -397,9 +397,10 @@ declare class Session { * The first seq appended IN THIS PROCESS: the length of the constructor * seed (0 without one). Events with smaller seq values entered through * construction — replay, fork, or resume — and were never published on the - * `session/event` firehose (constructor seeds do not emit), so consumers - * that replay the log as a publication substitute (telemetry adoption) - * start here. Distinct from {@link inheritedEventCount}, the DURABLE + * `session/event` firehose (constructor seeds do not emit). This offset marks + * the constructor-input boundary for lifecycle ownership and persistence + * adoption; consumers that need complete canonical history still start at + * seq 0. Distinct from {@link inheritedEventCount}, the DURABLE * fork-lineage cut: a resumed session's constructor seed is its full stored * log, while the inherited count keeps the original fork value — this field is the * in-process construction fact. @@ -571,7 +572,7 @@ declare class Session { - `tool/result` → 一条携带 `tool-result` 块的 user 消息。 - `user/message`（注入上下文，即非 `user` 来源）→ 按时间顺序在相应位置生成一条 user-role 消息，并原样承载其 `content`；其类型化 source 标明生产方，并携带所有生产方专用数据。 -其余所有事件（`turn/*`、`step/*`、插件所属的 `llm/retry`）均为结构信息，不会投影为消息。token 记账读取每个步骤的 `assistant/chunk { type: 'usage' }` 记录；如果没有用量分片，则将 `assistant/message.usage` 作为已提交步骤的后备。失败的模型请求尝试没有 assistant 消息，因此其用量分片是持久化的记账记录。由于这一尚未发布的格式有意不提供兼容性承诺，seed/load 校验会拒绝没有提供方／模型的请求头和 assistant 消息，而不会猜测历史数据应走的提供方路由。 +其余所有事件（`turn/*`、`step/*`、插件所属的 `llm/retry`）均为结构信息，不会投影为消息。token 记账读取每个步骤的 `assistant/chunk { type: 'usage' }` 记录；如果没有用量分片，则将 `assistant/message.usage` 作为已提交步骤的后备。失败的模型请求尝试没有 assistant 消息，因此其用量分片是持久化的记账记录。当前逻辑校验会拒绝没有提供方／模型的 request header 和 assistant 消息，而不会猜测路由；受支持的历史表示会在当前 Session 存在前，由其相邻格式迁移边归一化并校验。 ## 活跃会话 fork API diff --git a/docs/testing.i18n.yaml b/docs/testing.i18n.yaml index 6264efd8b6..50701e8032 100644 --- a/docs/testing.i18n.yaml +++ b/docs/testing.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write docs/testing.md -testing.md: 25514702e7aa8649c10ec213f5a6bf5c6e9e9a09 -testing.zh.md: 5338e4bc392e1e1ff16c32f708970be917e6efa7 +testing.md: 3ed78b7f483d9cc34cfcda2d4ca51d9e523a08c5 +testing.zh.md: 2463bed216f59c7a20d3c09757621fc269e0672c diff --git a/docs/testing.md b/docs/testing.md index 25514702e7..3ed78b7f48 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -10,10 +10,10 @@ How this repo tests, tier by tier, and the rules that keep a green suite meaning - **Coverage gate** (`pnpm run test:coverage`): the gating run, per-file 100% on `packages/*/*/src`. An uncovered line is often dead code the gate flags for deletion, not a missing test to bolt on. Line coverage is necessary, never sufficient — it proves lines ran, not that the feature works as shipped. Per-file 100% on `packages/shell/pwsh-local/src` needs a real `pwsh`: without one its executor suites self-skip and `vitest.config.ts` exempts the file so pwsh-less hosts stay green, while CI runners ship pwsh and enforce the full bar. - **Real-API e2e** (`pnpm run test:e2e`): with-key tests against live provider APIs — the DeepSeek model plus provider-specific smokes that gate on their own keys (`EXA_API_KEY`, `PERPLEXITY_API_KEY`, …); each suite self-skips without its key so keyless CI stays green ([real-API e2e Agent Note](../.agents/notes/implemented/testing/2026-06-19-real-api-e2e-ci.md)). - **Owner-local expected output** (`pnpm run test:expected`): keyless assembled CLI/process expectations without a recorded-session round trip. Drivers use `*.expected.e2e.ts` beside `tests/expected/`; CI runs built exports. Package/script expectations use `test`, while browser expectations use `test:web`. -- **Snapshot** (`pnpm run test:snapshot`): a top-level scenario's recorded `session.jsonl` supplies user input and model replay, then serves as the expected persisted result. Process scenarios start through `dsh`: headless owns one-shot behavior, the SDK owns persistent control, ACP owns automation-protocol behavior, and Web retains browser/ARIA evidence beside the same session. `snapshot.yml` declares the profile, composition/header class, recording policy, exceptional replay or input metadata, and workspace facts. Typed tokens preserve parent/child identity relationships; only header pins own prompt/schema sidecars. A mutating scenario independently compares the complete `workspace.expected/` tree, which record and refresh never rewrite. Use `test:snapshot:record` when a model transcript changes and `test:snapshot:refresh` when replay input remains valid; review every resulting diff. +- **Snapshot** (`pnpm run test:snapshot`): a top-level scenario's highest recorded parent generation supplies user input and model replay, then serves as the expected persisted result. Parent filenames are `session[.vN].jsonl`; child roles are `session.[.vN].jsonl`; v0 omits `.v0`, positive versions require lowercase `.vN`, and each filename must agree with its header. Process scenarios start through `dsh`: headless owns one-shot behavior, the SDK owns persistent control, ACP owns automation-protocol behavior, and Web retains browser/ARIA evidence beside the same Session. `snapshot.yml` declares the profile, composition/header class, recording policy, exceptional replay or input metadata, and workspace facts. Typed tokens preserve parent/child identity relationships; only header pins own prompt/schema sidecars. A mutating scenario independently compares the complete `workspace.expected/` tree, which record and refresh never rewrite. Use `test:snapshot:record` when a model transcript changes and `test:snapshot:refresh` when replay input remains valid; review every resulting diff. - **Web browser snapshot** (`pnpm run test:web`; required Linux PR gate): Chromium compares session-driven output under `snapshots/web/` and UI-only output under `apps/web/tests/expected/`. CI forces read-only `DSH_SNAPSHOT=replay`, never writing expected outputs; record/refresh stay local and every diff is reviewed ([web e2e lane](../.agents/notes/implemented/testing/2026-07-24-web-gui-browser-e2e-lane.md), [CI gate decision](../.agents/notes/implemented/testing/2026-07-30-web-browser-snapshot-ci-gate.md)). `test:web` [builds first](../.agents/notes/implemented/bug-fix/2026-07-28-themed-scrollbars-and-reserved-gutter.md) for plugin CSS. -Session fixtures keep headers and payloads but omit body sequence/time envelopes. Replay synthesizes them. Fixtures use canonical packed rows; [the migrator](../scripts/migrate-packed-session-fixtures.ts) rewrites old layouts. +Session fixtures keep headers and payloads but omit body sequence/time envelopes. Replay synthesizes them. A scenario may retain older generations for migration coverage, but replay, record, and refresh select the numerically highest file for each parent/child role; current v1 writer output uses `.v1`, while committed released-v0 inputs remain suffixless. Fixtures use canonical packed rows; [the migrator](../scripts/migrate-packed-session-fixtures.ts) rewrites old row layouts. ## How specs execute diff --git a/docs/testing.zh.md b/docs/testing.zh.md index 5338e4bc39..2463bed216 100644 --- a/docs/testing.zh.md +++ b/docs/testing.zh.md @@ -10,10 +10,10 @@ - **覆盖率门禁**（`pnpm run test:coverage`）：门禁级运行，对 `packages/*/*/src` 按文件 100% 覆盖。未覆盖的行往往是门禁正确标记出的死代码（应删除），而非需要补写的测试。行覆盖率是必要条件，但永远不是充分条件：它证明行被执行过，不证明功能按交付预期工作。`packages/shell/pwsh-local/src` 的按文件 100% 覆盖需要真实的 `pwsh`：缺少它时其执行器套件会自动跳过，`vitest.config.ts` 会豁免该文件以使无 pwsh 的主机保持绿色，而 CI runner 自带 pwsh，仍按完整标准执行门禁。 - **真实 API e2e**（`pnpm run test:e2e`）：带密钥测试调用真实提供方 API，包括 DeepSeek 模型以及各提供方特有的冒烟测试；这些测试各自由自己的密钥控制（`EXA_API_KEY`、`PERPLEXITY_API_KEY` 等），缺少密钥时套件会自动跳过，使 keyless CI 保持绿色（[真实 API e2e Agent Note](../.agents/notes/implemented/testing/2026-06-19-real-api-e2e-ci.zh.md)）。 - **所属位置的预期输出**（`pnpm run test:expected`）：无录制会话往返的无密钥组装 CLI/进程预期。驱动使用 `*.expected.e2e.ts`，并与 `tests/expected/` 同属一处；CI 针对构建产物运行。包/脚本预期使用 `test`，浏览器预期使用 `test:web`。 -- **快照**（`pnpm run test:snapshot`）：顶层场景的录制 `session.jsonl` 同时提供用户输入和模型回放，并作为持久化结果的预期值。进程级场景都通过 `dsh` 启动：headless 负责一次性行为，SDK 负责持久控制，ACP 负责自动化协议行为，Web 在同一会话旁保留浏览器与 ARIA 证据。`snapshot.yml` 声明 profile、组合与请求头类别、录制策略、例外回放或输入元数据以及工作区事实。带类型的 token 保留父子身份关系；只有请求头 pin 拥有提示词/schema sidecar。变更工作区的场景会独立比较完整的 `workspace.expected/` 目录，record 与 refresh 绝不改写该目录。当模型 transcript（文本记录）变化时使用 `test:snapshot:record`，回放输入仍有效时使用 `test:snapshot:refresh`；请审查所有结果差异。 +- **快照**（`pnpm run test:snapshot`）：顶层场景数值最高的已录制 parent generation 同时提供用户输入和模型回放，并作为持久化结果的预期值。parent 文件名是 `session[.vN].jsonl`；child 角色使用 `session.[.vN].jsonl`；v0 省略 `.v0`，正版本必须使用小写 `.vN`，且每个文件名必须与其 header 一致。进程级场景都通过 `dsh` 启动：headless 负责一次性行为，SDK 负责持久控制，ACP 负责自动化协议行为，Web 在同一 Session 旁保留浏览器与 ARIA 证据。`snapshot.yml` 声明 profile、组合与请求头类别、录制策略、例外回放或输入元数据以及 workspace 事实。带类型的 token 保留父子身份关系；只有请求头 pin 拥有 prompt/schema sidecar。变更 workspace 的场景会独立比较完整的 `workspace.expected/` 目录，record 与 refresh 绝不改写该目录。当模型 transcript（文本记录）变化时使用 `test:snapshot:record`，回放输入仍有效时使用 `test:snapshot:refresh`；请审查所有结果差异。 - **Web 浏览器快照**（`pnpm run test:web`；必需的 Linux PR（Pull Request）门禁）：Chromium 比较 `snapshots/web/` 下由会话驱动的输出，以及 `apps/web/tests/expected/` 下仅含 UI 的输出。CI 强制只读的 `DSH_SNAPSHOT=replay`，绝不写入预期输出；record/refresh 留在本地，每处 diff 都须评审（[web e2e 车道](../.agents/notes/implemented/testing/2026-07-24-web-gui-browser-e2e-lane.zh.md)、[CI 门禁决策](../.agents/notes/implemented/testing/2026-07-30-web-browser-snapshot-ci-gate.zh.md)）。`test:web` 会[先构建](../.agents/notes/implemented/bug-fix/2026-07-28-themed-scrollbars-and-reserved-gutter.zh.md)以交付插件 CSS。 -会话 fixture 保留 header 与 payload，但省略正文序号／时间 envelope。回放会合成这些字段；运行时持久化不变。fixture 使用规范打包行；[迁移器](../scripts/migrate-packed-session-fixtures.ts)会改写旧布局。 +Session fixture 保留 header 与 payload，但省略正文 seq/time envelope。回放会合成这些字段。场景可以为迁移覆盖保留旧 generation，但 replay、record 与 refresh 会为每个 parent/child 角色选择数值最高的文件；当前 v1 writer 输出使用 `.v1`，已提交的 released-v0 输入保持无后缀。fixture 使用规范打包行；[迁移器](../scripts/migrate-packed-session-fixtures.ts)会改写旧行布局。 ## spec 如何被执行 diff --git a/package.json b/package.json index d947add909..eb426110e8 100644 --- a/package.json +++ b/package.json @@ -134,6 +134,8 @@ "verify-doc-graphs": "tsx scripts/gen-doc-graphs.ts --check", "gen-persistence-catalog": "tsx scripts/gen-persistence-catalog.ts", "verify-persistence-catalog": "tsx scripts/gen-persistence-catalog.ts --check", + "gen-session-format-catalog": "tsx scripts/gen-session-format-catalog.ts", + "verify-session-format-catalog": "tsx scripts/gen-session-format-catalog.ts --check", "gen-third-party-notices": "tsx scripts/gen-third-party-notices.ts", "verify-third-party-notices": "tsx scripts/gen-third-party-notices.ts --check", "gen-module-graph": "tsx scripts/gen-module-graph.ts", diff --git a/packages/acp/acp/src/index.ts b/packages/acp/acp/src/index.ts index 1a349c0698..e38788b01a 100644 --- a/packages/acp/acp/src/index.ts +++ b/packages/acp/acp/src/index.ts @@ -47,7 +47,7 @@ import { } from '@agentclientprotocol/sdk' import type { ModelSelection } from '@deepseek-ai/dsh-agent' import type { SessionId } from '@deepseek-ai/dsh-session' -import type {} from '@deepseek-ai/dsh-session-persistence' +import { isReadableSessionPersistenceListing } from '@deepseek-ai/dsh-session-persistence' // Side-effect type import: declaration-merges the approval waterfall answered below. import type {} from '@deepseek-ai/dsh-user-approval' import { supportsAcpImagePrompts } from './content.ts' @@ -244,7 +244,10 @@ export function apply(ctx: Context, config: AcpConfig): void { } activating.add(sessionId) return (async (): Promise => { - const persisted = (await persistence.list(signal)).find(header => header.id === sessionId) + const persisted = (await persistence.list(signal)) + .filter(isReadableSessionPersistenceListing) + .map(listing => listing.header) + .find(header => header.id === sessionId) if (persisted === undefined || persisted.origin === 'subagent' || persisted.parentSession !== undefined) { throw invalidParams(`session is not resumable: ${sessionId}`) } @@ -299,7 +302,9 @@ export function apply(ctx: Context, config: AcpConfig): void { } catch (error: unknown) { throw invalidParams((error as Error).message) } - const listed = await persistence.list(signal) + const listed = (await persistence.list(signal)) + .filter(isReadableSessionPersistenceListing) + .map(listing => listing.header) const filtered = await Promise.all(listed.map(async (header) => { if ( sessions.has(header.id) diff --git a/packages/acp/acp/tests/bridge.spec.ts b/packages/acp/acp/tests/bridge.spec.ts index 5dec9b3786..2de7b64581 100644 --- a/packages/acp/acp/tests/bridge.spec.ts +++ b/packages/acp/acp/tests/bridge.spec.ts @@ -6,7 +6,8 @@ import { join } from 'node:path' import { fileURLToPath } from 'node:url' import { AttachmentError } from '@deepseek-ai/dsh-attachment' import { ToolCallId, type StreamChunk } from '@deepseek-ai/dsh-llm' -import { SessionId } from '@deepseek-ai/dsh-session' +import { SESSION_FORMAT_VERSION, SessionId, type SessionHeader } from '@deepseek-ai/dsh-session' +import type { CurrentSessionPersistenceListing } from '@deepseek-ai/dsh-session-persistence' import { defineContentToolFixture } from '@deepseek-ai/dsh-tools' import { makeBridgeHarness, textResponse, type BridgeHarness } from './harness.ts' import { startHttpMcpFixture } from '../../../mcp/mcp-client/tests/http-fixture.ts' @@ -24,6 +25,15 @@ function oneToolCall(): StreamChunk[] { ] } +function currentListing(header: SessionHeader): CurrentSessionPersistenceListing { + return { + status: 'current', + header, + storedVersion: SESSION_FORMAT_VERSION, + targetVersion: SESSION_FORMAT_VERSION, + } +} + describe('automation-only ACP bridge', () => { let harness: BridgeHarness | undefined @@ -249,13 +259,13 @@ describe('automation-only ACP bridge', () => { await harness.client.initialize({ protocolVersion: PROTOCOL_VERSION, clientCapabilities: {} }) const sessionId = SessionId('other-frontend-live') harness.ctx.sessions.create(sessionId, { meta: { cwd: process.cwd() } }) - vi.spyOn(harness.ctx.sessionPersistence, 'list').mockResolvedValue([{ - version: 0, + vi.spyOn(harness.ctx.sessionPersistence, 'list').mockResolvedValue([currentListing({ + version: SESSION_FORMAT_VERSION, id: sessionId, createdAt: 1, cwd: process.cwd(), isSeeded: false, - }]) + })]) const resume = vi.spyOn(harness.ctx.agents, 'resume') await expect(harness.client.listSessions({})).resolves.toEqual({ sessions: [] }) @@ -364,14 +374,63 @@ describe('automation-only ACP bridge', () => { const active = await harness.client.newSession({ cwd: process.cwd(), mcpServers: [] }) const persistence = harness.ctx.get('sessionPersistence')! vi.spyOn(persistence, 'list').mockResolvedValue([ - { version: 0, id: SessionId(active.sessionId), createdAt: 9, cwd: process.cwd(), isSeeded: false }, - { version: 0, id: SessionId('subagent'), createdAt: 8, cwd: '/missing/filter', isSeeded: false, origin: 'subagent' }, - { version: 0, id: SessionId('fork'), createdAt: 7, cwd: '/missing/filter', isSeeded: true, parentSession: SessionId('parent') }, - { version: 0, id: SessionId('no-cwd'), createdAt: 6, isSeeded: false }, - { version: 0, id: SessionId('relative'), createdAt: 5, cwd: 'relative', isSeeded: false }, - { version: 0, id: SessionId('other'), createdAt: 4, cwd: '/missing/other', isSeeded: false }, - { version: 0, id: SessionId('valid-b'), createdAt: 3, cwd: '/missing/filter', isSeeded: false }, - { version: 0, id: SessionId('valid-a'), createdAt: 3, cwd: '/missing/filter', isSeeded: false }, + currentListing({ + version: SESSION_FORMAT_VERSION, + id: SessionId(active.sessionId), + createdAt: 9, + cwd: process.cwd(), + isSeeded: false, + }), + currentListing({ + version: SESSION_FORMAT_VERSION, + id: SessionId('subagent'), + createdAt: 8, + cwd: '/missing/filter', + isSeeded: false, + origin: 'subagent', + }), + currentListing({ + version: SESSION_FORMAT_VERSION, + id: SessionId('fork'), + createdAt: 7, + cwd: '/missing/filter', + isSeeded: true, + parentSession: SessionId('parent'), + }), + currentListing({ + version: SESSION_FORMAT_VERSION, + id: SessionId('no-cwd'), + createdAt: 6, + isSeeded: false, + }), + currentListing({ + version: SESSION_FORMAT_VERSION, + id: SessionId('relative'), + createdAt: 5, + cwd: 'relative', + isSeeded: false, + }), + currentListing({ + version: SESSION_FORMAT_VERSION, + id: SessionId('other'), + createdAt: 4, + cwd: '/missing/other', + isSeeded: false, + }), + currentListing({ + version: SESSION_FORMAT_VERSION, + id: SessionId('valid-b'), + createdAt: 3, + cwd: '/missing/filter', + isSeeded: false, + }), + currentListing({ + version: SESSION_FORMAT_VERSION, + id: SessionId('valid-a'), + createdAt: 3, + cwd: '/missing/filter', + isSeeded: false, + }), ]) await expect(harness.client.listSessions({ cwd: 'relative' })).rejects.toThrow(/absolute path/) diff --git a/packages/api/session-controller/src/list.ts b/packages/api/session-controller/src/list.ts index 145c77c63a..d8f2b89a14 100644 --- a/packages/api/session-controller/src/list.ts +++ b/packages/api/session-controller/src/list.ts @@ -8,7 +8,11 @@ import { SessionLogOffset } from '@deepseek-ai/dsh-session' import type { Session, SessionEvent, SessionHeader, SessionId } from '@deepseek-ai/dsh-session' import type {} from '@deepseek-ai/dsh-session-projection' import type {} from '@deepseek-ai/dsh-session-projection-cache' -import { SessionQueryError, type SessionSearchCursor } from '@deepseek-ai/dsh-session-query' +import { + SessionQueryError, + type SessionRecord, + type SessionSearchCursor, +} from '@deepseek-ai/dsh-session-query' import { RemoteError } from '@deepseek-ai/dsh-typert-protocol' import { z } from 'zod' import { @@ -139,7 +143,7 @@ export class ApiSessionList { const records = await this.ctx.sessionQuery.listSessions(signal) signal?.throwIfAborted() const items: SessionSummary[] = [] - const cold: SessionHeader[] = [] + const cold: SessionRecord[] = [] for (const record of records) { const live = this.ctx.sessions.get(record.header.id) if (live !== undefined) { @@ -147,11 +151,11 @@ export class ApiSessionList { continue } if (record.header.cwd === undefined) continue - cold.push(record.header) + cold.push(record) } for (let offset = 0; offset < cold.length; offset += COLD_SUMMARY_BATCH_SIZE) { const settled = await Promise.allSettled(cold.slice(offset, offset + COLD_SUMMARY_BATCH_SIZE) - .map(header => this.summarizeCold(header, signal))) + .map(record => this.summarizeCold(record, signal))) for (const result of settled) { if (result.status === 'rejected') throw result.reason items.push(result.value) @@ -162,13 +166,14 @@ export class ApiSessionList { } private async summarizeCold( - header: SessionHeader, + record: SessionRecord, signal: AbortSignal | undefined, ): Promise { + const { header } = record const cached = this.projectionsFor(header, undefined) const projections = cached?.values.sessionListMetadata?.blank === false ? cached - : await this.probeSmallCold(header, signal) ?? cached + : await this.probeSmallCold(record, signal) ?? cached const raced = this.ctx.sessions.get(header.id) if (raced !== undefined) return this.summaryFor(raced) const metadata = projections?.values.sessionListMetadata @@ -184,12 +189,13 @@ export class ApiSessionList { } private async probeSmallCold( - header: SessionHeader, + record: SessionRecord, signal: AbortSignal | undefined, ): Promise { if (this.coldBlankProbeMaxBytes === 0) return undefined + const { header } = record const persistence = this.ctx.get('sessionPersistence') - const location = persistence?.locate(header) + const location = record.location ?? persistence?.locate(header) if (location === undefined) return undefined signal?.throwIfAborted() try { diff --git a/packages/api/session-controller/tests/agent.host.spec.ts b/packages/api/session-controller/tests/agent.host.spec.ts index 3d370bcf28..b3d71c16bc 100644 --- a/packages/api/session-controller/tests/agent.host.spec.ts +++ b/packages/api/session-controller/tests/agent.host.spec.ts @@ -5,7 +5,7 @@ import { Context } from '@deepseek-ai/cordis' import AgentRegistry from '@deepseek-ai/dsh-agent' import type { Agent } from '@deepseek-ai/dsh-agent' import { agentPresetProjectionDefinition } from '@deepseek-ai/dsh-agent-presets' -import SessionStore, { SessionId, SessionLogOffset, SessionSeq } from '@deepseek-ai/dsh-session' +import SessionStore, { SESSION_FORMAT_VERSION, SessionId, SessionLogOffset, SessionSeq } from '@deepseek-ai/dsh-session' import type { SessionEvent, SessionHeader } from '@deepseek-ai/dsh-session' import type { SessionObservation } from '@deepseek-ai/dsh-session-query' import type { SessionInspection } from '@deepseek-ai/dsh-session-persistence' @@ -19,7 +19,7 @@ import { inspectApiSession, } from '../src/agent.ts' import { installModelSelectionProjection } from '../src/model-selection-projection.ts' -import { installSessionReadTestServices, testSessionPersistence } from './test-remote.ts' +import { currentSessionListing, installSessionReadTestServices, testSessionPersistence } from './test-remote.ts' const roots: Context[] = [] @@ -45,7 +45,7 @@ async function harness(): Promise<{ ctx: Context; agents: ApiSessionAgentControl function header(id: string, cwd: string | null = '/workspace'): SessionHeader { return { - version: 0, + version: SESSION_FORMAT_VERSION, id: SessionId(id), createdAt: 1, isSeeded: false, @@ -106,7 +106,7 @@ describe('ApiSession identity failures', () => { const listed = header('cwd-less-catalog', null) const disposeListed = providePersistence(ctx, { - list: () => Promise.resolve([listed]), + list: () => Promise.resolve([currentSessionListing(listed)]), inspect: () => Promise.resolve(unseededInspection(listed)), }) await expect(inspectApiSession(ctx, listed.id)).rejects.toBeInstanceOf(ApiSessionNotFound) @@ -115,7 +115,7 @@ describe('ApiSession identity failures', () => { const catalog = header('cwd-less-inspect') const inspected = header('cwd-less-inspect', null) providePersistence(ctx, { - list: () => Promise.resolve([catalog]), + list: () => Promise.resolve([currentSessionListing(catalog)]), inspect: () => Promise.resolve(unseededInspection(inspected)), }) await expect(inspectApiSession(ctx, catalog.id)).rejects.toBeInstanceOf(ApiSessionNotFound) @@ -187,7 +187,7 @@ describe('ApiSession Agent lookup and recovery', () => { const ordinary = await harness() const ordinaryMeta = header('ordinary-race') providePersistence(ordinary.ctx, { - list: () => Promise.resolve([ordinaryMeta]), + list: () => Promise.resolve([currentSessionListing(ordinaryMeta)]), inspect: () => Promise.resolve(unseededInspection(ordinaryMeta)), }) const winner = agent(ordinary.ctx, ordinaryMeta) @@ -200,7 +200,7 @@ describe('ApiSession Agent lookup and recovery', () => { const child = await harness() const childMeta = header('child-race') providePersistence(child.ctx, { - list: () => Promise.resolve([childMeta]), + list: () => Promise.resolve([currentSessionListing(childMeta)]), inspect: () => Promise.resolve(unseededInspection(childMeta)), }) vi.spyOn(child.ctx.agents, 'resume').mockImplementation(async () => { @@ -227,7 +227,7 @@ describe('ApiSession Agent lookup and recovery', () => { const failed = await harness() const meta = header('failed') providePersistence(failed.ctx, { - list: () => Promise.resolve([meta]), + list: () => Promise.resolve([currentSessionListing(meta)]), inspect: () => Promise.resolve(unseededInspection(meta)), }) vi.spyOn(failed.ctx.agents, 'resume').mockRejectedValue(new Error('factory unavailable')) @@ -380,7 +380,7 @@ describe('ApiSession create or adoption', () => { data: { agentPreset: 'minimal' }, }] as SessionEvent[] providePersistence(ctx, { - list: () => Promise.resolve([meta]), + list: () => Promise.resolve([currentSessionListing(meta)]), inspect: () => Promise.resolve(unseededInspection(meta, events)), }) ctx.provide('agentPresets', { @@ -412,7 +412,7 @@ describe('ApiSession create or adoption', () => { const child = await harness() const childMeta = header('resume-child-race') providePersistence(child.ctx, { - list: () => Promise.resolve([childMeta]), + list: () => Promise.resolve([currentSessionListing(childMeta)]), inspect: () => Promise.resolve(unseededInspection(childMeta)), }) child.ctx.provide('agentPresets', { @@ -431,7 +431,7 @@ describe('ApiSession create or adoption', () => { const conflict = await harness() const stored = header('stored-cwd-conflict', '/stored') providePersistence(conflict.ctx, { - list: () => Promise.resolve([stored]), + list: () => Promise.resolve([currentSessionListing(stored)]), inspect: () => Promise.resolve(unseededInspection(stored)), }) await expect(conflict.agents.ensureSession(stored.id, '/requested', true)) diff --git a/packages/api/session-controller/tests/commands-queue-attachment.host.spec.ts b/packages/api/session-controller/tests/commands-queue-attachment.host.spec.ts index f437ed27a7..07ba0a6887 100644 --- a/packages/api/session-controller/tests/commands-queue-attachment.host.spec.ts +++ b/packages/api/session-controller/tests/commands-queue-attachment.host.spec.ts @@ -4,12 +4,12 @@ import type { Agent, ModelSelectionRef } from '@deepseek-ai/dsh-agent' import { AttachmentError, AttachmentId } from '@deepseek-ai/dsh-attachment' import type { ImageAttachmentRef } from '@deepseek-ai/dsh-attachment' import { createAssistantMessage, createUserMessage, MessageId } from '@deepseek-ai/dsh-llm' -import SessionStore, { SessionId, SessionLogOffset, SessionSeq } from '@deepseek-ai/dsh-session' +import SessionStore, { SESSION_FORMAT_VERSION, SessionId, SessionLogOffset, SessionSeq } from '@deepseek-ai/dsh-session' import type { SessionEvent, SessionHeader } from '@deepseek-ai/dsh-session' import { describe, expect, it, vi } from 'vitest' import { ApiSessionAgentController } from '../src/agent.ts' import { SessionCommandController } from '../src/commands.ts' -import { installSessionReadTestServices, testSessionPersistence } from './test-remote.ts' +import { currentSessionListing, installSessionReadTestServices, testSessionPersistence } from './test-remote.ts' async function commandHarness(): Promise<{ ctx: Context @@ -143,14 +143,14 @@ async function persistedController( await ctx.plugin(SessionStore) const sessionId = SessionId('cold-attachment') const meta: SessionHeader = { - version: 0, + version: SESSION_FORMAT_VERSION, id: sessionId, createdAt: 1, cwd: '/workspace', isSeeded: false, } ctx.provide('sessionPersistence', testSessionPersistence(ctx, { - list: () => Promise.resolve([meta]), + list: () => Promise.resolve([currentSessionListing(meta)]), inspect: () => Promise.resolve({ meta, inheritedEventCount: SessionLogOffset(0), diff --git a/packages/api/session-controller/tests/control-jobs.host.spec.ts b/packages/api/session-controller/tests/control-jobs.host.spec.ts index f827573d03..7b8b803630 100644 --- a/packages/api/session-controller/tests/control-jobs.host.spec.ts +++ b/packages/api/session-controller/tests/control-jobs.host.spec.ts @@ -3,12 +3,13 @@ import AgentRegistry, { Inbox } from '@deepseek-ai/dsh-agent' import type { Agent } from '@deepseek-ai/dsh-agent' import type { JobOutcome } from '@deepseek-ai/dsh-jobs' import LocalJobRegistry from '@deepseek-ai/dsh-jobs-local' -import SessionStore, { SessionId } from '@deepseek-ai/dsh-session' +import SessionStore, { SESSION_FORMAT_VERSION, SessionId } from '@deepseek-ai/dsh-session' import type { Session } from '@deepseek-ai/dsh-session' import SessionProjectionRegistry from '@deepseek-ai/dsh-session-projection' import { describe, expect, it } from 'vitest' import { SessionControlController } from '../src/control.ts' import type { SessionControlFrame } from '../src/types.ts' +import { currentSessionListing } from './test-remote.ts' type BaselineFrame = Extract type JobFrame = Extract @@ -181,7 +182,13 @@ describe('Session control jobs updates', () => { const coldId = SessionId('session-cold-tasks') let loaded = false ctx.provide('sessionPersistence', { - list: async () => [{ version: 0, id: coldId, createdAt: 5, cwd: '/tmp' }], + list: async () => [currentSessionListing({ + version: SESSION_FORMAT_VERSION, + id: coldId, + createdAt: 5, + cwd: '/tmp', + isSeeded: false, + })], locate: () => undefined, load: () => { loaded = true; throw new Error('job projection must not load a cold log') }, } as never) diff --git a/packages/api/session-controller/tests/controller.host.spec.ts b/packages/api/session-controller/tests/controller.host.spec.ts index ef5001cbe2..be4e4e22d5 100644 --- a/packages/api/session-controller/tests/controller.host.spec.ts +++ b/packages/api/session-controller/tests/controller.host.spec.ts @@ -2,13 +2,13 @@ import { Context } from '@deepseek-ai/cordis' import AgentRegistry from '@deepseek-ai/dsh-agent' import type { Agent } from '@deepseek-ai/dsh-agent' import { createUserMessage } from '@deepseek-ai/dsh-llm' -import SessionStore, { SessionId, SessionLogOffset } from '@deepseek-ai/dsh-session' +import SessionStore, { SESSION_FORMAT_VERSION, SessionId, SessionLogOffset } from '@deepseek-ai/dsh-session' import type { SessionEvent, SessionHeader } from '@deepseek-ai/dsh-session' import { RemoteError } from '@deepseek-ai/dsh-typert-protocol' import { describe, expect, it, vi } from 'vitest' import SessionController from '../src/index.ts' import type { ApiSessionAgentController } from '../src/agent.ts' -import { createSessionTestController, testSessionPersistence } from './test-remote.ts' +import { createSessionTestController, currentSessionListing, testSessionPersistence } from './test-remote.ts' const defaults = { defaultModelSelection: () => ({ provider: 'fixture', model: 'fixture-model' }), @@ -26,7 +26,7 @@ describe('SessionController facade', () => { await ctx.plugin(AgentRegistry) const sessionId = SessionId('controller-session') const header: SessionHeader = { - version: 0, + version: SESSION_FORMAT_VERSION, id: sessionId, createdAt: 1, cwd: '/workspace', @@ -39,7 +39,7 @@ describe('SessionController facade', () => { events, })) ctx.provide('sessionPersistence', testSessionPersistence(ctx, { - list: () => Promise.resolve([header]), + list: () => Promise.resolve([currentSessionListing(header)]), inspect, }) as never) const controller = createSessionTestController(ctx, defaults) @@ -122,10 +122,10 @@ describe('SessionController facade', () => { await ctx.plugin(AgentRegistry) const sessionId = SessionId(`background-${outcome}`) const header: SessionHeader = { - version: 0, id: sessionId, createdAt: 1, cwd: '/workspace', isSeeded: false, + version: SESSION_FORMAT_VERSION, id: sessionId, createdAt: 1, cwd: '/workspace', isSeeded: false, } ctx.provide('sessionPersistence', testSessionPersistence(ctx, { - list: () => Promise.resolve([header]), + list: () => Promise.resolve([currentSessionListing(header)]), inspect: () => Promise.resolve({ meta: header, inheritedEventCount: SessionLogOffset(0), @@ -178,10 +178,10 @@ describe('SessionController facade', () => { await ctx.plugin(AgentRegistry) const sessionId = SessionId('background-disposal') const header: SessionHeader = { - version: 0, id: sessionId, createdAt: 1, cwd: '/workspace', isSeeded: false, + version: SESSION_FORMAT_VERSION, id: sessionId, createdAt: 1, cwd: '/workspace', isSeeded: false, } ctx.provide('sessionPersistence', testSessionPersistence(ctx, { - list: () => Promise.resolve([header]), + list: () => Promise.resolve([currentSessionListing(header)]), inspect: () => Promise.resolve({ meta: header, inheritedEventCount: SessionLogOffset(0), diff --git a/packages/api/session-controller/tests/fake-api.client.ts b/packages/api/session-controller/tests/fake-api.client.ts index e3ea48f783..71f50f700e 100644 --- a/packages/api/session-controller/tests/fake-api.client.ts +++ b/packages/api/session-controller/tests/fake-api.client.ts @@ -388,7 +388,7 @@ export class FakeApiClient { yield { type: 'snapshot', header: { - version: 0, + version: 1, id: sessionId, createdAt: 0, ...(request.address.kind === 'subagent' diff --git a/packages/api/session-controller/tests/session-cold.host.spec.ts b/packages/api/session-controller/tests/session-cold.host.spec.ts index a6cc76c8b5..41820244b8 100644 --- a/packages/api/session-controller/tests/session-cold.host.spec.ts +++ b/packages/api/session-controller/tests/session-cold.host.spec.ts @@ -9,7 +9,7 @@ import { mkdtempSync, writeFileSync } from 'node:fs' import { tmpdir } from 'node:os' import { join } from 'node:path' import { Context } from '@deepseek-ai/cordis' -import SessionStore, { SessionLogOffset, SessionSeq } from '@deepseek-ai/dsh-session' +import SessionStore, { SESSION_FORMAT_VERSION, SessionLogOffset, SessionSeq } from '@deepseek-ai/dsh-session' import AgentRegistry from '@deepseek-ai/dsh-agent' import { SessionHistoryController } from '@deepseek-ai/dsh-api-session-controller/src/history.ts' import { subagentIdentityProjectionDefinition } from '@deepseek-ai/dsh-subagent/src/projection.ts' @@ -28,6 +28,7 @@ import { import { ApiSessionList } from '../src/list.ts' import { createSessionTestRemote, + currentSessionListing, installSessionReadTestServices, testSessionPersistence, } from './test-remote.ts' @@ -49,7 +50,7 @@ function promptRequest( } function header(id: string, createdAt: number, extra: Partial = {}): SessionHeader { - return { version: 0, id: sid(id), createdAt, cwd: '/proj', isSeeded: false, ...extra } + return { version: SESSION_FORMAT_VERSION, id: sid(id), createdAt, cwd: '/proj', isSeeded: false, ...extra } } function providePersistence(ctx: Context, persistence: Record): () => void { @@ -62,10 +63,12 @@ describe('sessions.list cold merge', () => { await ctx.plugin(SessionStore) const root = mkdtempSync(join(tmpdir(), 'dsh-cold-')) const smallPath = join(root, 'small.log') + const historicalPath = join(root, 'historical.log') const largePath = join(root, 'large.log') writeFileSync(smallPath, 'x'.repeat(1024)) + writeFileSync(historicalPath, 'historical') writeFileSync(largePath, 'x'.repeat(1025)) - const metas = [ + const metas: SessionHeader[] = [ header('small-blank', 100), header('small-conversation', 200), header('large-unknown', 300), @@ -74,7 +77,8 @@ describe('sessions.list cold merge', () => { header('locationless', 500, { parentSession: sid('session-parent'), origin: 'subagent' }), header('vanished', 600), header('read-failure', 700), - { version: 0, id: sid('missing-cwd'), createdAt: 800, isSeeded: false }, + { version: SESSION_FORMAT_VERSION, id: sid('missing-cwd'), createdAt: 800, isSeeded: false }, + header('migration-small-blank', 900), ] const inspect = vi.fn(async (id: SessionId) => { if (id === sid('small-blank')) { @@ -96,15 +100,33 @@ describe('sessions.list cold merge', () => { ] satisfies SessionEvent[], } } + if (id === sid('migration-small-blank')) { + return { + meta: metas[9]!, + events: [{ type: 'session/end-seed', seq: 0, time: 900, data: {} }] as SessionEvent[], + } + } if (id === sid('read-failure')) throw new Error('simulated read failure') throw new Error(`unexpected cold read: ${id}`) }) providePersistence(ctx, { - list: () => Promise.resolve(metas), + list: () => Promise.resolve([ + ...metas.slice(0, -1).map(currentSessionListing), + { + status: 'migration-required' as const, + storedVersion: 0, + targetVersion: SESSION_FORMAT_VERSION, + header: metas[9]!, + location: { kind: 'jsonl', path: historicalPath }, + }, + ]), locate: (meta: SessionHeader) => { if (meta.id === sid('large-unknown') || meta.id === sid('seeded-cold')) { return { kind: 'jsonl', path: largePath } } + if (meta.id === sid('migration-small-blank')) { + return { kind: 'jsonl', path: join(root, 'missing-current.log') } + } if (meta.id === sid('locationless')) return undefined if (meta.id === sid('vanished')) return { kind: 'jsonl', path: join(root, 'vanished.log') } return { kind: 'jsonl', path: smallPath } @@ -148,11 +170,13 @@ describe('sessions.list cold merge', () => { expect(byId['vanished']).toMatchObject({ blank: false, updatedAt: 600 }) expect(byId['read-failure']).toMatchObject({ blank: false, updatedAt: 700 }) expect(byId['missing-cwd']).toBeUndefined() - expect(inspect).toHaveBeenCalledTimes(3) + expect(byId['migration-small-blank']).toMatchObject({ blank: true, updatedAt: 900 }) + expect(inspect).toHaveBeenCalledTimes(4) expect(inspect.mock.calls.map(([id]) => id)).toEqual(expect.arrayContaining([ sid('small-blank'), sid('small-conversation'), sid('read-failure'), + sid('migration-small-blank'), ])) }) @@ -162,7 +186,7 @@ describe('sessions.list cold merge', () => { const meta = header('probe-disabled', 100) const inspect = vi.fn() providePersistence(ctx, { - list: () => Promise.resolve([meta]), + list: () => Promise.resolve([currentSessionListing(meta)]), locate: () => ({ kind: 'jsonl', path: '/not-read' }), inspect, }) @@ -191,7 +215,7 @@ describe('sessions.list cold merge', () => { list: async () => { started.resolve(undefined) await release.promise - return [meta] + return [currentSessionListing(meta)] }, }) const remote = createSessionTestRemote(ctx, { defaultModelSelection: () => ({ provider: 'p', model: 'm' }), cwd: '/tmp' }) @@ -236,7 +260,7 @@ describe('sessions.list cold merge', () => { writeFileSync(path, 'small') const meta = header('attached-during-probe', 100) providePersistence(ctx, { - list: () => Promise.resolve([meta]), + list: () => Promise.resolve([currentSessionListing(meta)]), locate: () => ({ kind: 'jsonl', path }), inspect: () => { const session = ctx.sessions.create(meta.id, { @@ -268,7 +292,7 @@ describe('sessions.list cold merge', () => { await ctx.plugin(SessionStore) const meta = header('broken-cache', 100) providePersistence(ctx, { - list: () => Promise.resolve([meta]), + list: () => Promise.resolve([currentSessionListing(meta)]), locate: () => { throw new Error('location failed') }, }) const remote = createSessionTestRemote(ctx, { @@ -292,7 +316,7 @@ describe('sessions.list cold merge', () => { writeFileSync(path, 'small') const meta = header('unprojected-small', 100) ctx.provide('sessionPersistence', { - list: () => Promise.resolve([meta]), + list: () => Promise.resolve([currentSessionListing(meta)]), locate: () => ({ kind: 'jsonl', path }), } as never) vi.spyOn(ctx.sessionQuery, 'listSessions').mockResolvedValue([{ @@ -385,7 +409,7 @@ describe('cold history recovery view', () => { ), appendBatch: () => Promise.resolve(), commitRepair: () => Promise.resolve(), - list: () => Promise.resolve([structuredClone(meta)]), + list: () => Promise.resolve([currentSessionListing(structuredClone(meta))]), } const coordinator = new PersistenceCoordinator(ctx, backend) providePersistence(ctx, { @@ -445,7 +469,7 @@ describe('Remote Agent and Session lookup policy', () => { events: [] as SessionEvent[], })) providePersistence(ctx, { - list: () => Promise.resolve([meta]), + list: () => Promise.resolve([currentSessionListing(meta)]), inspect, locate: () => undefined, }) @@ -493,7 +517,7 @@ describe('Remote Agent and Session lookup policy', () => { events: [] as SessionEvent[], })) providePersistence(ctx, { - list: () => Promise.resolve([coldMeta]), + list: () => Promise.resolve([currentSessionListing(coldMeta)]), inspect, locate: () => undefined, }) @@ -573,7 +597,7 @@ describe('subagent ownership fence', () => { events, })) providePersistence(ctx, { - list: () => Promise.resolve([meta]), + list: () => Promise.resolve([currentSessionListing(meta)]), inspect, locate: () => undefined, }) @@ -636,7 +660,7 @@ describe('subagent ownership fence', () => { }, ] satisfies SessionEvent[] providePersistence(ctx, { - list: () => Promise.resolve([meta]), + list: () => Promise.resolve([currentSessionListing(meta)]), inspect: () => Promise.resolve({ meta, inheritedEventCount: SessionLogOffset(0), @@ -900,7 +924,7 @@ describe('sessions.prompt synchronous rejection', () => { const sessionId = sid('race-resume') const meta: SessionHeader = header('race-resume', 1000) providePersistence(ctx, { - list: () => Promise.resolve([meta]), + list: () => Promise.resolve([currentSessionListing(meta)]), inspect: () => Promise.resolve({ meta, inheritedEventCount: SessionLogOffset(0), diff --git a/packages/api/session-controller/tests/session-fork.host.spec.ts b/packages/api/session-controller/tests/session-fork.host.spec.ts index 7942bb92f6..2e5e9f03e6 100644 --- a/packages/api/session-controller/tests/session-fork.host.spec.ts +++ b/packages/api/session-controller/tests/session-fork.host.spec.ts @@ -6,12 +6,12 @@ import AgentRegistry, { agentEvents } from '@deepseek-ai/dsh-agent' import type { Agent, AgentHandle, CreateAgentOptions } from '@deepseek-ai/dsh-agent' import { createUserMessage, ReasoningEffortId } from '@deepseek-ai/dsh-llm' import type { LlmCallConfig } from '@deepseek-ai/dsh-llm' -import SessionStore, { SessionLogOffset, SessionSeq } from '@deepseek-ai/dsh-session' +import SessionStore, { SESSION_FORMAT_VERSION, SessionLogOffset, SessionSeq } from '@deepseek-ai/dsh-session' import type { Session, SessionEvent, SessionHeader, SessionId } from '@deepseek-ai/dsh-session' import SystemPrompt from '@deepseek-ai/dsh-system-prompt' import type { Workspace } from '@deepseek-ai/dsh-workspace' import { - createSessionTestRemote, installSessionReadTestServices, testSessionPersistence, + createSessionTestRemote, currentSessionListing, installSessionReadTestServices, testSessionPersistence, } from './test-remote.ts' const sid = (id: string): SessionId => id as SessionId @@ -151,7 +151,7 @@ describe('sessions.fork', () => { const sourceId = sid('session-cold-subagent') const parentId = sid('session-cold-parent') const header: SessionHeader = { - version: 0, + version: SESSION_FORMAT_VERSION, id: sourceId, createdAt: 1, cwd: '/proj', @@ -179,7 +179,7 @@ describe('sessions.fork', () => { { type: 'turn/end', seq: SessionSeq(2), time: 3, data: { turn: 1, reason: { kind: 'completed' } } }, ] satisfies SessionEvent[] ctx.provide('sessionPersistence', testSessionPersistence(ctx, { - list: () => Promise.resolve([header]), + list: () => Promise.resolve([currentSessionListing(header)]), inspect: () => Promise.resolve({ meta: header, inheritedEventCount: SessionLogOffset(0), diff --git a/packages/api/session-controller/tests/session-projections.host.spec.ts b/packages/api/session-controller/tests/session-projections.host.spec.ts index 3f57e1afe2..6251952a6f 100644 --- a/packages/api/session-controller/tests/session-projections.host.spec.ts +++ b/packages/api/session-controller/tests/session-projections.host.spec.ts @@ -18,7 +18,7 @@ import { AttachmentStore } from '@deepseek-ai/dsh-attachment' import { agentPresetProjectionDefinition } from '@deepseek-ai/dsh-agent-presets' import type { Agent } from '@deepseek-ai/dsh-agent' import { createUserMessage } from '@deepseek-ai/dsh-llm' -import SessionStore, { SessionId, SessionSeq } from '@deepseek-ai/dsh-session' +import SessionStore, { SESSION_FORMAT_VERSION, SessionId, SessionSeq } from '@deepseek-ai/dsh-session' import type { Session } from '@deepseek-ai/dsh-session' import SessionProjectionRegistry from '@deepseek-ai/dsh-session-projection' import type { ProjectionDefinition } from '@deepseek-ai/dsh-session-projection' @@ -27,7 +27,7 @@ import Storage from '@deepseek-ai/dsh-storage' import * as StorageDomain from '@deepseek-ai/dsh-storage-domain' import * as StorageJson from '@deepseek-ai/dsh-storage-json' import type { SessionControlFrame, SessionFollowFrame } from '@deepseek-ai/dsh-api-session-controller/types' -import { createSessionTestRemote, type TestSessionRemote } from './test-remote.ts' +import { createSessionTestRemote, currentSessionListing, type TestSessionRemote } from './test-remote.ts' declare module '@deepseek-ai/dsh-session-projection/types' { interface SessionProjectionStateMap { @@ -132,7 +132,7 @@ function seedMessages(session: Session, count: number): void { const remote = (ctx: Context) => createSessionTestRemote(ctx, { defaultModelSelection: () => ({ provider: 'p', model: 'm' }), cwd: '/tmp' }) describe('session.history projections block', () => { - it('keeps the v0 numeric seed cut on the wire while logical headers expose only lineage', async () => { + it('keeps the numeric seed cut on the wire while logical headers expose only lineage', async () => { const ctx = new Context() await ctx.plugin(SessionStore) await ctx.plugin(AgentRegistry) @@ -154,7 +154,7 @@ describe('session.history projections block', () => { const snapshot = await opening(remote(ctx), child.id) expect(snapshot.header).toEqual({ - version: 0, + version: 1, id: child.id, createdAt: child.header.createdAt, cwd: '/workspace', @@ -435,7 +435,13 @@ describe('session.list projections column', () => { const coldId = SessionId('session-cold-listing') const load = () => { throw new Error('list must not load event logs') } ctx.provide('sessionPersistence', { - list: async () => [{ version: 0, id: coldId, createdAt: 5, cwd: '/tmp' }], + list: async () => [currentSessionListing({ + version: SESSION_FORMAT_VERSION, + id: coldId, + createdAt: 5, + cwd: '/tmp', + isSeeded: false, + })], locate: () => undefined, load, inspect: load, @@ -469,6 +475,35 @@ describe('session.list projections column', () => { }) }) + it('does not use a zero-cut cache row for a seeded cold listing', async () => { + const { ctx } = await harness(true) + const coldId = SessionId('session-seeded-cold-listing') + const load = () => { throw new Error('list must not load event logs') } + ctx.provide('sessionPersistence', { + list: async () => [currentSessionListing({ + version: SESSION_FORMAT_VERSION, + id: coldId, + createdAt: 6, + cwd: '/tmp', + isSeeded: true, + })], + locate: () => undefined, + load, + inspect: load, + readFrom: load, + } as never) + const cachedSnapshot = vi.fn(() => ({ asOfSeq: 7, values: { title: 'Wrong cut' } })) + ctx.provide('sessionProjectionCache', { cachedSnapshot } as never) + + const response = await remote(ctx).list(request({})) + if (!response.ok) throw new Error('unreachable') + const row = response.value.items.find(item => item.sessionId === coldId) + + expect(row).toBeDefined() + expect(row !== undefined && 'projections' in row).toBe(false) + expect(cachedSnapshot).not.toHaveBeenCalled() + }) + it('keeps persisted host-only state out of a cold session.list response', async () => { const root = await mkdtemp(join(tmpdir(), 'dsh-api-projcache-')) const ctx = new Context() @@ -507,7 +542,7 @@ describe('session.list projections column', () => { await owner.dispose() expect(ctx.sessions.get(id)).toBeUndefined() ctx.provide('sessionPersistence', { - list: async () => [header], + list: async () => [currentSessionListing(header)], locate: () => undefined, } as never) @@ -527,7 +562,13 @@ describe('session.list projections column', () => { const { ctx } = await harness(true) const coldId = SessionId('session-cold-uncached') ctx.provide('sessionPersistence', { - list: async () => [{ version: 0, id: coldId, createdAt: 5, cwd: '/tmp' }], + list: async () => [currentSessionListing({ + version: SESSION_FORMAT_VERSION, + id: coldId, + createdAt: 5, + cwd: '/tmp', + isSeeded: false, + })], locate: () => undefined, } as never) const response = await remote(ctx).list(request({})) diff --git a/packages/api/session-controller/tests/session-search.host.spec.ts b/packages/api/session-controller/tests/session-search.host.spec.ts index 89d1a2f424..9500921ff9 100644 --- a/packages/api/session-controller/tests/session-search.host.spec.ts +++ b/packages/api/session-controller/tests/session-search.host.spec.ts @@ -8,7 +8,7 @@ import { describe, expect, it, vi } from 'vitest' import { Context } from '@deepseek-ai/cordis' import AgentRegistry from '@deepseek-ai/dsh-agent' import { createUserMessage } from '@deepseek-ai/dsh-llm' -import SessionStore, { SessionSeq } from '@deepseek-ai/dsh-session' +import SessionStore, { SESSION_FORMAT_VERSION, SessionSeq } from '@deepseek-ai/dsh-session' import type { SessionHeader, SessionId } from '@deepseek-ai/dsh-session' import SessionProjectionRegistry from '@deepseek-ai/dsh-session-projection' import { @@ -17,7 +17,7 @@ import { type SessionSearchHit, type SessionSearchRequest, } from '@deepseek-ai/dsh-session-query' -import { createSessionTestRemote } from './test-remote.ts' +import { createSessionTestRemote, currentSessionListing } from './test-remote.ts' import { ApiSessionList } from '../src/list.ts' const sid = (value: string): SessionId => value as SessionId @@ -29,7 +29,7 @@ function request(query: string): { query: string } { function header(id: string, cwd: string | null = '/project'): SessionHeader { return { - version: 0, + version: SESSION_FORMAT_VERSION, id: sid(id), createdAt: 100, isSeeded: false, @@ -114,7 +114,7 @@ describe('session.search', () => { const cold = header('cold', '/cold') const legacy = header('legacy', null) ctx.provide('sessionPersistence', { - list: () => Promise.resolve([cold, legacy]), + list: () => Promise.resolve([currentSessionListing(cold), currentSessionListing(legacy)]), locate: () => undefined, } as never) @@ -771,7 +771,7 @@ describe('session.search', () => { (_, index) => header(`cold-${index}`, `/cold-${index}`), ) ctx.provide('sessionPersistence', { - list: () => Promise.resolve(cold), + list: () => Promise.resolve(cold.map(currentSessionListing)), locate: () => undefined, } as never) const searchSessions = vi.fn((_request: SessionSearchRequest) => Promise.resolve({ @@ -802,7 +802,7 @@ describe('session.search', () => { const list = vi.fn((signal?: AbortSignal) => { expect(signal).toBe(controller.signal) controller.abort() - return Promise.resolve(cold) + return Promise.resolve(cold.map(currentSessionListing)) }) let locateCalls = 0 ctx.provide('sessionPersistence', { @@ -834,7 +834,7 @@ describe('session.search', () => { const cold = Array.from({ length: 16 }, (_, index) => header(`cold-${index}`, `/cold-${index}`)) const locate = vi.fn((meta: SessionHeader) => ({ kind: 'jsonl', path: `/logs/${meta.id}.jsonl` })) ctx.provide('sessionPersistence', { - list: () => Promise.resolve(cold), + list: () => Promise.resolve(cold.map(currentSessionListing)), locate, } as never) const searchSessions = vi.fn(() => Promise.resolve({ items: [] })) diff --git a/packages/api/session-controller/tests/session-skills.host.spec.ts b/packages/api/session-controller/tests/session-skills.host.spec.ts index 386d37e55c..f48d6dee32 100644 --- a/packages/api/session-controller/tests/session-skills.host.spec.ts +++ b/packages/api/session-controller/tests/session-skills.host.spec.ts @@ -1,7 +1,7 @@ import { Context } from '@deepseek-ai/cordis' import AgentRegistry from '@deepseek-ai/dsh-agent' import type { Agent } from '@deepseek-ai/dsh-agent' -import SessionStore, { SessionId, SessionLogOffset } from '@deepseek-ai/dsh-session' +import SessionStore, { SESSION_FORMAT_VERSION, SessionId, SessionLogOffset } from '@deepseek-ai/dsh-session' import { SessionQueryError, type SessionObservation } from '@deepseek-ai/dsh-session-query' import type {} from '@deepseek-ai/dsh-skill' import { describe, expect, it, vi } from 'vitest' @@ -15,7 +15,7 @@ function observation( const lease = (): SessionObservation => ({ source: 'live', header: { - version: 0, + version: SESSION_FORMAT_VERSION, id: sessionId, createdAt: 1, isSeeded: false, diff --git a/packages/api/session-controller/tests/sessions-service.client.spec.ts b/packages/api/session-controller/tests/sessions-service.client.spec.ts index c98ff399fb..61c2415013 100644 --- a/packages/api/session-controller/tests/sessions-service.client.spec.ts +++ b/packages/api/session-controller/tests/sessions-service.client.spec.ts @@ -268,7 +268,7 @@ describe('Agent scope disposal lifecycle', () => { value: { type: 'snapshot', header: { - version: 0, + version: 1, id: request.address.kind === 'session' ? request.address.sessionId : request.address.childSessionId, @@ -342,7 +342,7 @@ describe('Agent scope disposal lifecycle', () => { done: false, value: { type: 'snapshot', - header: { version: 0, id: sessionId, createdAt: 0 }, + header: { version: 1, id: sessionId, createdAt: 0 }, cursor: -1, records: [], hasMore: false, diff --git a/packages/api/session-controller/tests/test-remote.ts b/packages/api/session-controller/tests/test-remote.ts index 2794e52ffe..5673e1edca 100644 --- a/packages/api/session-controller/tests/test-remote.ts +++ b/packages/api/session-controller/tests/test-remote.ts @@ -2,13 +2,14 @@ import type { Context } from '@deepseek-ai/cordis' import type { ModelSelection as AgentModelSelection } from '@deepseek-ai/dsh-agent' -import { SessionLogOffset } from '@deepseek-ai/dsh-session' -import type { SessionId } from '@deepseek-ai/dsh-session' +import { SESSION_FORMAT_VERSION, SessionLogOffset } from '@deepseek-ai/dsh-session' +import type { SessionHeader, SessionId } from '@deepseek-ai/dsh-session' import { SessionPersistenceCorruptionError, SessionPersistenceNotFoundError, SessionPersistenceRevision, type BorrowedSessionSource, + type CurrentSessionPersistenceListing, type SessionInspection, } from '@deepseek-ai/dsh-session-persistence' import SessionProjectionRegistry from '@deepseek-ai/dsh-session-projection' @@ -98,6 +99,20 @@ type LegacyTestPersistence = Record & { ) => Promise } +/** + * Describe one current logical header through the persistence listing contract. + * @param header - current logical header exposed by the test backend. + * @returns one current-format header-only listing descriptor. + */ +export function currentSessionListing(header: SessionHeader): CurrentSessionPersistenceListing { + return { + status: 'current', + header, + storedVersion: SESSION_FORMAT_VERSION, + targetVersion: SESSION_FORMAT_VERSION, + } +} + /** Add the preparation-backed point-read contract to compact persistence doubles. */ export function testSessionPersistence( ctx: Context, diff --git a/packages/api/session-controller/tests/transport.client.spec.ts b/packages/api/session-controller/tests/transport.client.spec.ts index c5f50ad7de..28a77053d5 100644 --- a/packages/api/session-controller/tests/transport.client.spec.ts +++ b/packages/api/session-controller/tests/transport.client.spec.ts @@ -63,7 +63,7 @@ function snapshot( return { type: 'snapshot', header: { - version: 0, + version: 1, id: ADDRESS.kind === 'session' ? ADDRESS.sessionId : ADDRESS.childSessionId, createdAt: 0, }, diff --git a/packages/api/session-controller/tests/transport.host.spec.ts b/packages/api/session-controller/tests/transport.host.spec.ts index b6eeda6b41..93cc66c1e7 100644 --- a/packages/api/session-controller/tests/transport.host.spec.ts +++ b/packages/api/session-controller/tests/transport.host.spec.ts @@ -1,13 +1,13 @@ import { Context } from '@deepseek-ai/cordis' import { createScope } from '@deepseek-ai/dsh-scope' -import SessionStore, { SessionId, SessionLogOffset, SessionSeq } from '@deepseek-ai/dsh-session' +import SessionStore, { SESSION_FORMAT_VERSION, SessionId, SessionLogOffset, SessionSeq } from '@deepseek-ai/dsh-session' import type { Session, SessionEvent, SessionHeader, SurfaceIntent } from '@deepseek-ai/dsh-session' import type { SessionObservation } from '@deepseek-ai/dsh-session-query' import { snapshotSubagentDescriptor } from '@deepseek-ai/dsh-subagent' import { subagentIdentityProjectionDefinition } from '@deepseek-ai/dsh-subagent/src/projection.ts' import { describe, expect, it, vi } from 'vitest' import { SessionHistoryController } from '../src/history.ts' -import { installSessionReadTestServices, testSessionPersistence } from './test-remote.ts' +import { currentSessionListing, installSessionReadTestServices, testSessionPersistence } from './test-remote.ts' const signal = (): AbortSignal => new AbortController().signal @@ -52,7 +52,7 @@ function cold( ): void { if (header.isSeeded) throw new Error('seeded cold fixtures require an explicit inherited cut') ctx.provide('sessionPersistence', testSessionPersistence(ctx, { - list: () => Promise.resolve([header]), + list: () => Promise.resolve([currentSessionListing(header)]), inspect: () => Promise.resolve({ meta: header, inheritedEventCount: SessionLogOffset(0), @@ -170,8 +170,8 @@ describe('SessionHistoryController', () => { it('subscribes before a cold read and ignores unrelated and replayed buffered events', async () => { const { ctx, transport } = await setup() const sessionId = SessionId('cold-race') - const header = { - version: 0, + const header: SessionHeader = { + version: SESSION_FORMAT_VERSION, id: sessionId, createdAt: 1, cwd: '/workspace', @@ -210,8 +210,8 @@ describe('SessionHistoryController', () => { const ctx = new Context() await ctx.plugin(SessionStore) const sessionId = SessionId('created-during-observation') - const header = { - version: 0, + const header: SessionHeader = { + version: SESSION_FORMAT_VERSION, id: sessionId, createdAt: 1, cwd: '/workspace', @@ -268,8 +268,8 @@ describe('SessionHistoryController', () => { { inject: ['sessions'] }, )) const sessionId = SessionId('cold-attach') - const header = { - version: 0, + const header: SessionHeader = { + version: SESSION_FORMAT_VERSION, id: sessionId, createdAt: 1, cwd: '/workspace', @@ -316,8 +316,8 @@ describe('SessionHistoryController', () => { it('rejects gaps in replayed and live event sequences', async () => { const replay = await setup() const replayId = SessionId('replay-gap') - const replayHeader = { - version: 0, + const replayHeader: SessionHeader = { + version: SESSION_FORMAT_VERSION, id: replayId, createdAt: 1, cwd: '/workspace', @@ -365,8 +365,8 @@ describe('SessionHistoryController', () => { const ctx = new Context() await ctx.plugin(SessionStore) const sessionId = SessionId('projectionless-follow') - const meta = { - version: 0, + const meta: SessionHeader = { + version: SESSION_FORMAT_VERSION, id: sessionId, createdAt: 1, cwd: '/workspace', @@ -399,7 +399,13 @@ describe('SessionHistoryController', () => { const ctx = new Context() await ctx.plugin(SessionStore) const sessionId = SessionId('promotion-failure') - const meta = { version: 0, id: sessionId, createdAt: 1, cwd: '/workspace' } + const meta: SessionHeader = { + version: SESSION_FORMAT_VERSION, + id: sessionId, + createdAt: 1, + cwd: '/workspace', + isSeeded: false, + } const disposePromotion = vi.fn() const promotion = { source: 'prepared', header: meta, events: [], cursor: -1, @@ -468,9 +474,15 @@ describe('SessionHistoryController', () => { const { ctx, transport } = await setup() const sessionId = SessionId('corrupt-cold') const failure = new Error('cold log is corrupt') - const header = { version: 0, id: sessionId, createdAt: 1, cwd: '/workspace' } + const header: SessionHeader = { + version: SESSION_FORMAT_VERSION, + id: sessionId, + createdAt: 1, + cwd: '/workspace', + isSeeded: false, + } ctx.provide('sessionPersistence', testSessionPersistence(ctx, { - list: () => Promise.resolve([header]), + list: () => Promise.resolve([currentSessionListing(header)]), inspect: () => Promise.reject(failure), }) as never) @@ -506,7 +518,7 @@ describe('SessionHistoryController', () => { const corruptId = SessionId('missing-through-seq') cold( corrupt.ctx, - { version: 0, id: corruptId, createdAt: 1, cwd: '/workspace', isSeeded: false }, + { version: SESSION_FORMAT_VERSION, id: corruptId, createdAt: 1, cwd: '/workspace', isSeeded: false }, [event('fixture/start', SessionSeq(0)), event('fixture/gap', SessionSeq(2))], ) await expect(corrupt.transport.page({ @@ -547,9 +559,11 @@ describe('SessionHistoryController', () => { const first = await setup() const sessionId = SessionId('incomplete') const address = { kind: 'session' as const, sessionId } - const firstHeader = { version: 0, id: sessionId, createdAt: 1, isSeeded: false } + const firstHeader: SessionHeader = { + version: SESSION_FORMAT_VERSION, id: sessionId, createdAt: 1, isSeeded: false, + } first.ctx.provide('sessionPersistence', testSessionPersistence(first.ctx, { - list: () => Promise.resolve([firstHeader]), + list: () => Promise.resolve([currentSessionListing(firstHeader)]), inspect: () => Promise.resolve({ meta: firstHeader, inheritedEventCount: SessionLogOffset(0), @@ -560,10 +574,14 @@ describe('SessionHistoryController', () => { .rejects.toMatchObject({ code: 'session/not-found' }) const second = await setup() - const listed = { version: 0, id: sessionId, createdAt: 1, cwd: '/workspace', isSeeded: false } - const inspected = { version: 0, id: sessionId, createdAt: 1, isSeeded: false } + const listed: SessionHeader = { + version: SESSION_FORMAT_VERSION, id: sessionId, createdAt: 1, cwd: '/workspace', isSeeded: false, + } + const inspected: SessionHeader = { + version: SESSION_FORMAT_VERSION, id: sessionId, createdAt: 1, isSeeded: false, + } second.ctx.provide('sessionPersistence', testSessionPersistence(second.ctx, { - list: () => Promise.resolve([listed]), + list: () => Promise.resolve([currentSessionListing(listed)]), inspect: () => Promise.resolve({ meta: inspected, inheritedEventCount: SessionLogOffset(0), @@ -577,8 +595,8 @@ describe('SessionHistoryController', () => { it('serves cold ordinary history and validates every durable subagent descriptor state', async () => { const ordinaryBench = await setup() const ordinaryId = SessionId('cold-ordinary') - const ordinaryHeader = { - version: 0, + const ordinaryHeader: SessionHeader = { + version: SESSION_FORMAT_VERSION, id: ordinaryId, createdAt: 1, cwd: '/workspace', @@ -594,13 +612,13 @@ describe('SessionHistoryController', () => { const parentSessionId = SessionId('cold-parent') const childSessionId = SessionId('cold-child') - const childHeader = { - version: 0, + const childHeader: SessionHeader = { + version: SESSION_FORMAT_VERSION, id: childSessionId, createdAt: 1, cwd: '/workspace', isSeeded: false, - origin: 'subagent' as const, + origin: 'subagent', parentSession: parentSessionId, } const childAddress = { @@ -632,7 +650,7 @@ describe('SessionHistoryController', () => { const parentSessionId = SessionId('missing-projection-parent') const childSessionId = SessionId('missing-projection-child') const meta: SessionHeader = { - version: 0, + version: SESSION_FORMAT_VERSION, id: childSessionId, createdAt: 1, cwd: '/workspace', diff --git a/packages/context/session-reference/README.i18n.yaml b/packages/context/session-reference/README.i18n.yaml index 615ca2a3fa..4b5a7b8128 100644 --- a/packages/context/session-reference/README.i18n.yaml +++ b/packages/context/session-reference/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/context/session-reference/README.md -README.md: 06ca90025aa0cc5e50fd3bea893ecf4bf3191077 -README.zh.md: 041a796a5bc708b5a7ed625a886defda24cdde3d +README.md: b2ade457d29ce5cdaef3b104c1bb70bb27db3c3e +README.zh.md: ed54176d12186f7327acc30c2a107671dcbf4a14 diff --git a/packages/context/session-reference/README.md b/packages/context/session-reference/README.md index 06ca90025a..b2ade457d2 100644 --- a/packages/context/session-reference/README.md +++ b/packages/context/session-reference/README.md @@ -77,7 +77,7 @@ Preparation reads each referenced session's current surface exactly once, when t ### Main flow -The outer `agent/pre-step` listener accepts the step, parses canonical mentions out of direct user messages, then calls `prepare`, which normalizes references (first-mention order, deduplication, self-reference and count rejection), reads every surface in parallel, retains each under `maxReferenceBytes`, and renders the aggregated prompt. Each snapshot is inserted immediately after the message that cited it, and the target log records the readable direct message followed by its sourced context, so source mutation after capture cannot change target replay. +The outer `agent/pre-step` listener accepts the step, parses canonical mentions out of direct user messages, then calls `prepare`, which normalizes references (first-mention order, deduplication, self-reference and count rejection), reads every surface in parallel, retains each under `maxReferenceBytes`, and renders the aggregated prompt. Each durable source record keeps the frozen `capturedThroughSeq` and records a nonzero `capturedFormatVersion`; absence denotes format v0. Each snapshot is inserted immediately after the message that cited it, and the target log records the readable direct message followed by its sourced context, so source mutation after capture cannot change target replay. diff --git a/packages/context/session-reference/README.zh.md b/packages/context/session-reference/README.zh.md index 041a796a5b..ed54176d12 100644 --- a/packages/context/session-reference/README.zh.md +++ b/packages/context/session-reference/README.zh.md @@ -77,7 +77,7 @@ kind: "package-reference" ### 主要流程 -外层 `agent/pre-step` 监听器接受步骤，从直接用户消息中解析规范 mention，再调用 `prepare`：规范化引用（保持首次 mention 顺序、去重、拒绝自引用与超限数量），并行读取每个表层，在 `maxReferenceBytes` 下逐源保留，并渲染聚合提示词。每份快照都插入到引用它的消息紧后，目标日志先记录可读的直接消息、再记录其带来源上下文，因此捕获后的源变更无法改变目标回放。 +外层 `agent/pre-step` 监听器接受步骤，从直接用户消息中解析规范 mention，再调用 `prepare`：规范化引用（保持首次 mention 顺序、去重、拒绝自引用与超限数量），并行读取每个表层，在 `maxReferenceBytes` 下逐源保留，并渲染聚合提示词。每条持久来源记录保留冻结的 `capturedThroughSeq` 并记录非零 `capturedFormatVersion`；字段缺失表示格式 v0。每份快照都插入到引用它的消息紧后，目标日志先记录可读的直接消息、再记录其带来源上下文，因此捕获后的源变更无法改变目标回放。 diff --git a/packages/context/session-reference/src/index.ts b/packages/context/session-reference/src/index.ts index fb7c6ca22a..d9a488e23d 100644 --- a/packages/context/session-reference/src/index.ts +++ b/packages/context/session-reference/src/index.ts @@ -75,6 +75,7 @@ interface PreparedSource { interface RenderedSource { data: ReferencedSessionData stats: ReferenceRetentionStats + capturedFormatVersion: number } /** Exact-read consumer that prepares immutable cross-session message context. */ @@ -306,6 +307,7 @@ export class SessionReferenceResolver extends TypertRemoteService { references: rendered.map((source, index) => ({ sessionId: source.data.sessionId, label: source.data.label, + capturedFormatVersion: source.capturedFormatVersion, capturedThroughSeq: source.data.capturedThroughSeq, ...source.stats, inputIndex: index, @@ -328,7 +330,10 @@ export class SessionReferenceResolver extends TypertRemoteService { 'SESSION_REFERENCE_BUDGET_EXCEEDED', ) } - rendered.push(retained) + rendered.push({ + ...retained, + capturedFormatVersion: source.snapshot.session.version, + }) } return rendered } diff --git a/packages/context/session-reference/src/types.ts b/packages/context/session-reference/src/types.ts index 47965e9528..944ed6898d 100644 --- a/packages/context/session-reference/src/types.ts +++ b/packages/context/session-reference/src/types.ts @@ -18,6 +18,8 @@ export interface SessionReferenceSource { references: { sessionId: string label: string + /** Source Session format generation; absence identifies version 0. */ + capturedFormatVersion?: number capturedThroughSeq: OptionalSessionSeq compacted: boolean originalMessages: number diff --git a/packages/context/session-reference/tests/session-reference.spec.ts b/packages/context/session-reference/tests/session-reference.spec.ts index 07eb12113d..acdaa384f2 100644 --- a/packages/context/session-reference/tests/session-reference.spec.ts +++ b/packages/context/session-reference/tests/session-reference.spec.ts @@ -536,6 +536,31 @@ describe('session reference discovery and preparation', () => { expect(context.content[0].text).not.toContain('later source mutation') }) + it('records the current source format generation without rebasing its frozen sequence', async () => { + const ctx = await harness() + const target = ctx.sessions.create(SessionId('target')) + const source = ctx.sessions.create(SessionId('source')) + appendConversation(source) + const snapshot = await ctx.sessionQuery.readSurface(source.id) + vi.spyOn(ctx.sessionQuery, 'readSurface').mockResolvedValue(snapshot) + + const prepared = await ctx.sessionReferenceResolver.prepare( + fakeAgent(target), + [{ type: 'text', text: 'use @source' }], + [{ sessionId: source.id }], + ) + + const captured = prepared.additionalContext?.source + expect(captured).toMatchObject({ + kind: 'session-reference', + references: [{ + sessionId: source.id, + capturedFormatVersion: snapshot.session.version, + capturedThroughSeq: snapshot.capturedThroughSeq, + }], + }) + }) + it('excludes injected context when projecting a referenced session', async () => { const ctx = await harness() const target = ctx.sessions.create(SessionId('target')) diff --git a/packages/core/agent-loop/src/index.ts b/packages/core/agent-loop/src/index.ts index 572f2bcc7b..b518329845 100644 --- a/packages/core/agent-loop/src/index.ts +++ b/packages/core/agent-loop/src/index.ts @@ -30,6 +30,7 @@ import type {} from '@deepseek-ai/dsh-system-prompt' import type {} from '@deepseek-ai/dsh-tools' import type {} from '@deepseek-ai/dsh-session-projection' import type { ProjectionDefinition } from '@deepseek-ai/dsh-session-projection' +import { SessionPersistenceNotFoundError } from '@deepseek-ai/dsh-session-persistence' import type { SessionPersistence } from '@deepseek-ai/dsh-session-persistence' import { ReactLoopAgent } from './agent.ts' import { DEFAULT_MAX_PARALLEL_TOOL_CALLS } from './constants.ts' @@ -481,11 +482,9 @@ export class AgentLoop extends Service implements AgentFactory { return } catch (error: unknown) { if (!this.ownership.isActive()) return - // A load is the per-id serialization barrier for eager write-behind and - // lifecycle retirement. Only a genuinely absent artifact falls back to - // first creation; corruption and backend failures stay loud. - const exists = (await persistence.list()).some(header => header.id === sessionId) - if (exists) throw error + // Only the persistence service's exact absence result may create this id; + // unsupported formats, corruption, and backend failures stay loud. + if (!(error instanceof SessionPersistenceNotFoundError)) throw error } this.create(sessionId, agentOptions, meta) } diff --git a/packages/core/agent-loop/tests/config-session-id.spec.ts b/packages/core/agent-loop/tests/config-session-id.spec.ts index a19b5da3f5..95e60245f4 100644 --- a/packages/core/agent-loop/tests/config-session-id.spec.ts +++ b/packages/core/agent-loop/tests/config-session-id.spec.ts @@ -1,9 +1,9 @@ import { createUserMessage } from '@deepseek-ai/dsh-llm' import { afterEach, describe, expect, it, vi } from 'vitest' import { Context } from '@deepseek-ai/cordis' -import { mkdtemp, rm } from 'node:fs/promises' +import { mkdtemp, readFile, rm, writeFile } from 'node:fs/promises' import { tmpdir } from 'node:os' -import { join } from 'node:path' +import { dirname, join } from 'node:path' import LlmRuntime from '@deepseek-ai/dsh-llm' import SessionStore, { SessionId, SessionPreparation } from '@deepseek-ai/dsh-session' import SystemPrompt from '@deepseek-ai/dsh-system-prompt' @@ -37,6 +37,22 @@ async function makeCoreContext(): Promise { return ctx } +/** Materialize one ordinary JSONL artifact and return its exact active path. */ +async function persistConfiguredArtifact(sessionId: SessionId): Promise<{ root: string; path: string }> { + const root = await mkdtemp(join(tmpdir(), 'dsh-cfg-artifact-')) + dirs.push(root) + const ctx = await makeCoreContext() + await ctx.plugin(JsonlSessionPersistence, { root, compression: 'none' }) + const session = ctx.sessions.create(sessionId) + session.append('turn/start', { turn: 1 }) + session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) + await ctx.sessions.flush(session) + const path = ctx.sessionPersistence.locate(session.header)?.path + await ctx.fiber.dispose() + if (path === undefined) throw new Error('JSONL test fixture has no artifact path') + return { root, path } +} + describe('config-driven session id', () => { it('applies launcher identities by configured id without changing unmatched entries', async () => { const ctx = await makeCoreContext() @@ -217,7 +233,7 @@ describe('config-driven session id', () => { await ctx.fiber.dispose() }) - it('contains an exact-id persistence lookup failure', async () => { + it('contains an exact-id persistence restore failure', async () => { const root = await mkdtemp(join(tmpdir(), 'dsh-cfg-exact-failure-')) dirs.push(root) const ctx = await makeCoreContext() @@ -231,7 +247,7 @@ describe('config-driven session id', () => { ctx.on('agent-loop/config-start-failed', ({ sessionId, error }) => { failures.push({ sessionId, error }) }) - vi.spyOn(ctx.sessionPersistence, 'list').mockRejectedValue(failure) + vi.spyOn(ctx.sessionPersistence, 'prepare').mockRejectedValue(failure) const warn = vi.spyOn(ctx.logger, 'warn').mockImplementation(() => undefined) await ctx.plugin(AgentLoop, { @@ -253,6 +269,46 @@ describe('config-driven session id', () => { await ctx.fiber.dispose() }) + it.each([ + ['future', (header: Record) => { header['version'] = 99 }, /newer|version 99/i], + ['malformed', (header: Record) => { header['unexpected'] = true }, /invalid current header/i], + ] as const)('keeps a configured exact-id %s artifact failure instead of attempting creation', async ( + _kind, + mutate, + expected, + ) => { + const sessionId = SessionId(`config-exact-${_kind}`) + const { root, path } = await persistConfiguredArtifact(sessionId) + const lines = (await readFile(path, 'utf8')).split('\n') + const header = JSON.parse(lines[0] as string) as Record + mutate(header) + lines[0] = JSON.stringify(header) + const source = lines.join('\n') + const artifactPath = _kind === 'future' + ? join(dirname(path), 'session.v99.jsonl') + : path + await writeFile(artifactPath, source) + if (artifactPath !== path) await rm(path) + + const ctx = await makeCoreContext() + await ctx.plugin(JsonlSessionPersistence, { root, compression: 'none' }) + const failures: unknown[] = [] + ctx.on('agent-loop/config-start-failed', ({ error }) => { failures.push(error) }) + const warn = vi.spyOn(ctx.logger, 'warn').mockImplementation(() => undefined) + await ctx.plugin(AgentLoop, { + agents: [{ id: 'main', sessionId, model: 'mock' }], + }) + + await expect.poll(() => failures.length).toBe(1) + expect(failures[0]).toBeInstanceOf(Error) + expect((failures[0] as Error).message).toMatch(expected) + expect(ctx.agents.get(sessionId)).toBeUndefined() + expect(await readFile(artifactPath, 'utf8')).toBe(source) + expect(warn).not.toHaveBeenCalledWith(expect.stringContaining('already has a persisted log')) + warn.mockRestore() + await ctx.fiber.dispose() + }) + it('contains startup and observer failures whose string coercion throws', async () => { const root = await mkdtemp(join(tmpdir(), 'dsh-cfg-exact-unrenderable-')) dirs.push(root) @@ -269,7 +325,7 @@ describe('config-driven session id', () => { // oxlint-disable-next-line typescript/prefer-promise-reject-errors ctx.on('agent-loop/config-start-failed', () => Promise.reject(unrenderable) as never) ctx.on('agent-loop/config-start-failed', ({ error }) => { failures.push(error) }) - vi.spyOn(ctx.sessionPersistence, 'list').mockRejectedValue(unrenderable) + vi.spyOn(ctx.sessionPersistence, 'prepare').mockRejectedValue(unrenderable) const warn = vi.spyOn(ctx.logger, 'warn').mockImplementation(() => undefined) await ctx.plugin(AgentLoop, { diff --git a/packages/core/agent-loop/tests/resume.spec.ts b/packages/core/agent-loop/tests/resume.spec.ts index 457de1a306..3004fb14b0 100644 --- a/packages/core/agent-loop/tests/resume.spec.ts +++ b/packages/core/agent-loop/tests/resume.spec.ts @@ -1,9 +1,9 @@ import { createUserMessage } from '@deepseek-ai/dsh-llm' import { afterEach, describe, expect, it, vi } from 'vitest' import { Context } from '@deepseek-ai/cordis' -import { mkdtemp, rm } from 'node:fs/promises' +import { mkdir, mkdtemp, readFile, readdir, rm, stat, writeFile } from 'node:fs/promises' import { tmpdir } from 'node:os' -import { join } from 'node:path' +import { dirname, join } from 'node:path' import LlmRuntime from '@deepseek-ai/dsh-llm' import SessionStore, { SESSION_FORMAT_VERSION, Session, SessionId, SessionLogOffset, SessionPreparation, SessionSeq } from '@deepseek-ai/dsh-session' import type { SessionEvent, SessionHeader, SessionLogOffset as SessionLogOffsetType } from '@deepseek-ai/dsh-session' @@ -11,7 +11,8 @@ import SystemPrompt from '@deepseek-ai/dsh-system-prompt' import ToolRuntime from '@deepseek-ai/dsh-tools' import AgentRegistry, { type Agent } from '@deepseek-ai/dsh-agent' -import JsonlSessionPersistence from '@deepseek-ai/dsh-session-persistence-jsonl' +import JsonlSessionPersistence, { type JsonlCompression } from '@deepseek-ai/dsh-session-persistence-jsonl' +import { generationLogFilename } from '@deepseek-ai/dsh-session-persistence-jsonl/src/format.ts' import AgentLoop from '@deepseek-ai/dsh-agent-loop' import SessionProjectionRegistry from '@deepseek-ai/dsh-session-projection' import { MockAdapter, textResponse } from './mock-adapter.ts' @@ -19,13 +20,20 @@ import { MockAdapter, textResponse } from './mock-adapter.ts' const dirs: string[] = [] afterEach(async () => { for (const d of dirs.splice(0)) await rm(d, { recursive: true, force: true }) }) -async function persistentHarness(adapter: MockAdapter): Promise<{ ctx: Context; root: string }> { +async function persistentHarness( + adapter: MockAdapter, + compression?: JsonlCompression, +): Promise<{ ctx: Context; root: string }> { const root = await mkdtemp(join(tmpdir(), 'dsh-resume-')) dirs.push(root) - return { ctx: await mountPersistentHarness(root, adapter), root } + return { ctx: await mountPersistentHarness(root, adapter, compression), root } } -async function mountPersistentHarness(root: string, adapter: MockAdapter): Promise { +async function mountPersistentHarness( + root: string, + adapter: MockAdapter, + compression?: JsonlCompression, +): Promise { const ctx = new Context() await ctx.plugin(LlmRuntime) await ctx.plugin(SessionStore) @@ -34,7 +42,7 @@ async function mountPersistentHarness(root: string, adapter: MockAdapter): Promi await ctx.plugin(ToolRuntime) await ctx.plugin(AgentRegistry) await ctx.plugin(AgentLoop, { agents: [] }) - await ctx.plugin(JsonlSessionPersistence, { root }) + await ctx.plugin(JsonlSessionPersistence, { root, ...(compression === undefined ? {} : { compression }) }) ctx.llm.registerAdapter(['mock'], adapter) return ctx } @@ -98,14 +106,12 @@ function throwUnknown(value: unknown): never { describe('the session-persistence Agent Note: AgentLoop factory create/resume', () => { it('resumes a pre-react-loop session including pre-identity message events', async () => { const sessionId = SessionId('pre-identity-resume') - const first = await persistentHarness(new MockAdapter([])) - await first.ctx.sessionPersistence.create({ - version: SESSION_FORMAT_VERSION, - id: sessionId, - createdAt: 1, - isSeeded: false, - }) - await first.ctx.sessionPersistence.append(sessionId, [ + const first = await persistentHarness(new MockAdapter([]), 'none') + const placeholder = first.ctx.sessions.create(sessionId) + const currentPath = first.ctx.sessionPersistence.locate(placeholder.header)?.path + if (currentPath === undefined) throw new Error('expected a JSONL artifact path') + const v0Path = join(dirname(currentPath), generationLogFilename(0, 'none')) + const legacyRows = [ { type: 'turn/start', seq: 0, time: 1, data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }, @@ -143,10 +149,17 @@ describe('the session-persistence Agent Note: AgentLoop factory create/resume', }, { type: 'step/end', seq: 5, time: 6, data: { turn: 1, step: 1 } }, { type: 'turn/end', seq: 6, time: 7, data: { turn: 1, reason: { kind: 'completed' } } }, - ] as unknown as SessionEvent[]) + ] + const legacySource = [ + { type: 'session', version: 0, id: sessionId, createdAt: 1, delegationDepth: 0 }, + ...legacyRows, + ].map(value => JSON.stringify(value)).join('\n') + '\n' + await mkdir(dirname(v0Path), { recursive: true }) + await writeFile(v0Path, legacySource, { flush: true }) + const v0Before = await stat(v0Path, { bigint: true }) await first.ctx.fiber.dispose() - const ctx = await mountPersistentHarness(first.root, new MockAdapter([textResponse('new answer')])) + const ctx = await mountPersistentHarness(first.root, new MockAdapter([textResponse('new answer')]), 'none') const handle = await ctx.agents.resume({ resumeSessionId: sessionId, agentOptions: { provider: 'mock', model: 'mock' }, @@ -158,6 +171,18 @@ describe('the session-persistence Agent Note: AgentLoop factory create/resume', ]) expect(handle.agent.inbox.nextTurn).toEqual([]) expect(handle.agent.inbox.nextStep).toEqual([]) + const v0AfterMigration = await stat(v0Path, { bigint: true }) + const v1AfterMigration = await stat(currentPath, { bigint: true }) + expect(await readFile(v0Path, 'utf8')).toBe(legacySource) + expect({ dev: v0AfterMigration.dev, ino: v0AfterMigration.ino }) + .toEqual({ dev: v0Before.dev, ino: v0Before.ino }) + expect({ dev: v1AfterMigration.dev, ino: v1AfterMigration.ino }) + .not.toEqual({ dev: v0Before.dev, ino: v0Before.ino }) + expect(JSON.parse((await readFile(currentPath, 'utf8')).split('\n', 1)[0] ?? '')).toMatchObject({ + version: SESSION_FORMAT_VERSION, + id: sessionId, + }) + expect((await readdir(dirname(v0Path))).sort()).toEqual(['session.jsonl', 'session.v1.jsonl']) handle.agent.followup(createUserMessage({ content: [{ type: 'text', text: 'new question' }], @@ -169,6 +194,10 @@ describe('the session-persistence Agent Note: AgentLoop factory create/resume', type: 'turn/end', data: { reason: { kind: 'completed' } }, }) + const v0AfterAppend = await stat(v0Path, { bigint: true }) + expect(await readFile(v0Path, 'utf8')).toBe(legacySource) + expect({ dev: v0AfterAppend.dev, ino: v0AfterAppend.ino }) + .toEqual({ dev: v0Before.dev, ino: v0Before.ino }) await handle.dispose() await ctx.fiber.dispose() }) @@ -882,9 +911,8 @@ describe('configured-start failure edges', () => { const sessionId = SessionId('config-existing-corrupt') const root = await persistSession(sessionId) const ctx = await mountPersistentHarness(root, new MockAdapter([])) - // The artifact exists (list reports it) but its load fails: this is - // corruption, not first creation — the failure must be reported, and no - // fresh same-id session may shadow the broken one. + // The artifact's restore fails: this is corruption, not first creation — + // the failure must be reported, and no fresh same-id session may shadow it. ctx.sessionPersistence.prepare = () => Promise.reject(new Error('artifact corrupt')) const configured = new Context() diff --git a/packages/core/session/README.i18n.yaml b/packages/core/session/README.i18n.yaml index 71c2828f9c..814f517c82 100644 --- a/packages/core/session/README.i18n.yaml +++ b/packages/core/session/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/core/session/README.md -README.md: f5cf910854203021a619cc786dfd13705927ffc1 -README.zh.md: 385d7fd63a6e4dec9c23c9d38a352942d7dbc7f9 +README.md: 79a71c81632dd2034b93f52f5c4efa49a651c912 +README.zh.md: ec72429a9022f719c99671fe41c7351dccd8c9ba diff --git a/packages/core/session/README.md b/packages/core/session/README.md index f5cf910854..79a71c8163 100644 --- a/packages/core/session/README.md +++ b/packages/core/session/README.md @@ -178,7 +178,7 @@ Logging causes no invalidation, and exact reconstruction preserves request-prefi These limits define when the session store needs special care. They are current package constraints, not a task backlog. - **`fork()` cuts only at stable boundaries of live sessions** — the selected prefix must end outside an open turn and the source must be in the store; forking a persisted-but-unloaded session is excluded from the [fork API](../../../.agents/notes/implemented/feature/2026-06-30-session-store-fork-api.md). -- **`SESSION_FORMAT_VERSION` stays pinned at `0`** — pre-release, no broad compatibility implied: `Session` accepts only current seed shapes, a backend refuses any other version, and unknown event types refuse reconstruction unless marked `ignorable` in the envelope ([mechanism](../../../.agents/notes/implemented/architecture/2026-08-10-session-log-version-mechanism.md)). +- **`SESSION_FORMAT_VERSION` names only the current logical representation** — historical headers and events remain in adjacent format packages, while persistence publishes a complete supported chain before constructing `Session`; equal-version unknown events still require the envelope's explicit `ignorable` marker ([mechanism](../../../.agents/notes/implemented/architecture/2026-08-31-released-session-format-migrations.md)). - **`TurnEndReasonMap` omits the ACP-named `refusal` / `max_turn_requests` variants** — producer-gated: they land when an adapter or the loop first emits them. - **No session tree beyond fork** — a pi-style entry tree over branched sessions is deferred unless a consumer needs more than boundary-based forking. diff --git a/packages/core/session/README.zh.md b/packages/core/session/README.zh.md index 385d7fd63a..ec72429a90 100644 --- a/packages/core/session/README.zh.md +++ b/packages/core/session/README.zh.md @@ -178,7 +178,7 @@ session.deriveMessages() // the derived model history 这些限制说明会话存储何时需要特别留意。它们是当前包约束，不是任务积压。 - **`fork()` 仅在实时会话的稳定边界处切分**：所选前缀结束时不得有开放轮次，且源会话必须位于存储中；[fork API](../../../.agents/notes/implemented/feature/2026-06-30-session-store-fork-api.zh.md) 不支持对已持久化但未加载的会话进行 fork。 -- **`SESSION_FORMAT_VERSION` 固定为 `0`**：预发布阶段不承诺广泛兼容性；`Session` 只接受当前 seed 形状，后端拒绝任何其他版本，不认识的事件类型也会拒绝重建，除非信封带 `ignorable` 标记（[机制](../../../.agents/notes/implemented/architecture/2026-08-10-session-log-version-mechanism.zh.md)）。 +- **`SESSION_FORMAT_VERSION` 只命名当前逻辑表示**——历史 header 与事件位于相邻格式包中，持久化会在构造 `Session` 前发布完整受支持链；同版本未知事件仍要求信封显式带有 `ignorable` 标记（[机制](../../../.agents/notes/implemented/architecture/2026-08-31-released-session-format-migrations.zh.md)）。 - **`TurnEndReasonMap` 不含 ACP（Agent Client Protocol）命名的 `refusal`／`max_turn_requests` 变体**：受生产方约束；只有当适配器或循环首次产生这些变体时才加入。 - **fork 之外没有会话树**：基于分支会话的 pi 风格条目树被推迟，除非消费方需要超越基于边界的 forking 的能力。 diff --git a/packages/core/session/src/index.ts b/packages/core/session/src/index.ts index c55cc5fb6a..ee98a0a680 100644 --- a/packages/core/session/src/index.ts +++ b/packages/core/session/src/index.ts @@ -212,9 +212,6 @@ function freezeRestoredObject(value: T): T { /** Validate the fixed event envelope after one-pass JSON materialization. */ function assertSessionEventEnvelope(value: Record, index: number): asserts value is SessionEvent { const event = value - if (event['type'] === 'request/header-delta') { - throw new Error(`seed event at index ${index} uses unsupported legacy request/header-delta format`) - } for (const key in event) { switch (key) { case 'type': @@ -269,6 +266,13 @@ function assertCurrentLlmShape(event: Record, index: number): v throw new Error(`seed request/header at index ${index} has an invalid reasoningEffort`) } assertAdapterDefaults(headerRecord?.['adapterDefaults'], configRecord, index) + const reason = record?.['reason'] + if (reason !== 'initial' && reason !== 'resume' && reason !== 'change' && reason !== 'series') { + throw new Error(`seed request/header at index ${index} has an invalid reason`) + } + if (record?.['startsSeries'] !== undefined && record['startsSeries'] !== true) { + throw new Error(`seed request/header at index ${index} has an invalid startsSeries marker`) + } } const type = event['type'] if (type !== 'user/message' && type !== 'assistant/message' @@ -359,18 +363,6 @@ function hasProviderModel(value: unknown): boolean { && typeof pair['model'] === 'string' && pair['model'].length > 0 } -/** Reject request-header vocabulary removed with the legacy delta codec. */ -function assertSupportedRequestHeader(type: string, data: unknown, location: string): void { - if (type === 'request/header-delta') { - throw new Error(`${location} uses unsupported legacy request/header-delta format`) - } - if (type === 'request/header' - && data !== null && typeof data === 'object' && !Array.isArray(data) - && (data as Record)['reason'] === 'fallback') { - throw new Error(`${location} uses unsupported legacy request/header reason "fallback"`) - } -} - type SessionCallback = (...args: unknown[]) => unknown /** Resolve one listener snapshot, including Cordis's internal dispatch checks. */ @@ -454,9 +446,10 @@ export class Session { * The first seq appended IN THIS PROCESS: the length of the constructor * seed (0 without one). Events with smaller seq values entered through * construction — replay, fork, or resume — and were never published on the - * `session/event` firehose (constructor seeds do not emit), so consumers - * that replay the log as a publication substitute (telemetry adoption) - * start here. Distinct from {@link inheritedEventCount}, the DURABLE + * `session/event` firehose (constructor seeds do not emit). This offset marks + * the constructor-input boundary for lifecycle ownership and persistence + * adoption; consumers that need complete canonical history still start at + * seq 0. Distinct from {@link inheritedEventCount}, the DURABLE * fork-lineage cut: a resumed session's constructor seed is its full stored * log, while the inherited count keeps the original fork value — this field is the * in-process construction fact. @@ -537,7 +530,6 @@ export class Session { throw new Error(`seed event at index ${index} is not losslessly JSON-serializable`) } assertSessionEventEnvelope(snapshot, index) - assertSupportedRequestHeader(snapshot.type, snapshot.data, `seed event at index ${index}`) if (snapshot.seq !== index) { throw new Error(`seed event at index ${index} has seq ${snapshot.seq} (expected ${index}); seed must be contiguous from 0`) } @@ -679,7 +671,6 @@ export class Session { if (dataSnapshot === undefined) { throw new Error(`session event "${type}" carries non-JSON-serializable data`) } - assertSupportedRequestHeader(type, dataSnapshot, `session event "${type}"`) const surfaceMetadataSnapshot = snapshotJsonValue(surfaceMetadata) if (surfaceMetadataSnapshot === undefined) { throw new Error(`session event "${type}" carries non-JSON-serializable surface metadata`) diff --git a/packages/core/session/src/types.ts b/packages/core/session/src/types.ts index 461c31ad36..9ecc3f9ff3 100644 --- a/packages/core/session/src/types.ts +++ b/packages/core/session/src/types.ts @@ -62,11 +62,11 @@ export type SessionSeqCursor = SessionSeq | -1 export type OptionalSessionSeq = SessionSeq | null /** - * The on-disk session format version, stamped into every newly-written {@link SessionHeader} - * and enforced by every persistence backend on load. The single source of truth for the - * version — write sites and the load-time check all read it. - * While the harness is unreleased it is pinned at `0`: no compatibility is - * implied, incompatible logs are rejected, and no migration is provided. + * Current logical Session format version, stamped into every newly written + * {@link SessionHeader}. Current Session and persistence code accept only this + * value; header-only readers classify supported historical formats, while an + * event-body read composes the build-static adjacent chain and publishes only + * this final generation before constructing a Session. * * The version is a single monotonic integer with no major/minor split. Whether * a bump is needed is decided by what the WRITER emits, never by what a newer @@ -79,23 +79,21 @@ export type OptionalSessionSeq = SessionSeq | null * Adding an ordinary event type does not bump — the per-event * {@link SessionEvent.ignorable} guard covers vocabulary growth instead. When * in doubt, bump: a near-identity upgrade step is almost free, a missed bump - * makes older runtimes read new logs wrong silently. The full mechanism - * (upgrade-step chain, in-memory view conversion, migrate-on-continue) is - * recorded in the session-log-version-mechanism Agent Note - * (`.agents/notes/implemented/architecture/2026-08-10-session-log-version-mechanism.md`). + * makes older runtimes read new logs wrong silently. The released migration, + * immutable prior-generation, and current fast-path rules are recorded in + * `.agents/notes/implemented/architecture/2026-08-31-released-session-format-migrations.md`. */ -export const SESSION_FORMAT_VERSION = 0 +export const SESSION_FORMAT_VERSION = 1 /** * Immutable validated storage metadata, kept outside the conversation event log. */ export interface SessionHeader { /** - * On-disk format version, stamped from {@link SESSION_FORMAT_VERSION} when the - * session is created. A persistence backend rejects any other version on load - * (no migration — see the constant). + * Current logical format version, stamped from {@link SESSION_FORMAT_VERSION}. + * Historical physical headers are translated before entering this interface. */ - readonly version: number + readonly version: typeof SESSION_FORMAT_VERSION /** The session's id (mirrors the {@link Session}'s id). */ readonly id: SessionId /** Non-negative safe-integer Unix epoch milliseconds when the session was created. */ diff --git a/packages/core/session/tests/request-header.spec.ts b/packages/core/session/tests/request-header.spec.ts index e3b7bdb92b..f0265d8ce0 100644 --- a/packages/core/session/tests/request-header.spec.ts +++ b/packages/core/session/tests/request-header.spec.ts @@ -1,4 +1,4 @@ -/** Request-header canonicalization, equality, snapshot folding, and format rejection. */ +/** Request-header canonicalization, equality, and snapshot folding. */ import { describe, expect, it } from 'vitest' import { Session, SessionId, SessionSeq, canonicalHeader, foldRequestHeader, headerEquals } from '@deepseek-ai/dsh-session' @@ -86,35 +86,6 @@ describe('foldRequestHeader', () => { }) }) -describe('legacy request-header format', () => { - it('rejects request/header-delta in seeds and untyped appends', () => { - const legacy = [{ - type: 'request/header-delta', seq: 0, time: 1, data: { config: CONFIG }, - }] as unknown as SessionEvent[] - expect(() => Session.create(SessionId('legacy'), legacy)).toThrow(/unsupported legacy request\/header-delta/) - - const session = Session.create(SessionId('legacy-append-delta')) - const appendLegacy = session.append.bind(session) as (type: string, data: unknown) => SessionEvent - expect(() => appendLegacy('request/header-delta', { config: CONFIG })) - .toThrow(/unsupported legacy request\/header-delta/) - expect(session.snapshotEvents()).toHaveLength(0) - }) - - it('rejects the removed fallback reason in seeds and untyped appends', () => { - const legacy = [{ - type: 'request/header', seq: 0, time: 1, data: { header: { config: CONFIG }, reason: 'fallback' }, - }] as unknown as SessionEvent[] - expect(() => Session.create(SessionId('legacy-seed-reason'), legacy)) - .toThrow('unsupported legacy request/header reason "fallback"') - - const session = Session.create(SessionId('legacy-append-reason')) - const appendLegacy = session.append.bind(session) as (type: string, data: unknown) => SessionEvent - expect(() => appendLegacy('request/header', { header: { config: CONFIG }, reason: 'fallback' })) - .toThrow('unsupported legacy request/header reason "fallback"') - expect(session.snapshotEvents()).toHaveLength(0) - }) -}) - describe('Session.requestContext', () => { const CAPACITY = { provider: 'mock', model: 'm', contextWindow: 128_000 } diff --git a/packages/core/session/tests/sequence-types.spec.ts b/packages/core/session/tests/sequence-types.spec.ts index 11ac49b301..debbf5fe97 100644 --- a/packages/core/session/tests/sequence-types.spec.ts +++ b/packages/core/session/tests/sequence-types.spec.ts @@ -1,6 +1,7 @@ import { describe, expect, expectTypeOf, it } from 'vitest' import { Session, + SESSION_FORMAT_VERSION, SessionId, SessionLogOffset, SessionSeq, @@ -50,7 +51,7 @@ describe('Session log positions', () => { expect(() => Session.fromRestore(id, [{ type: 'turn/start', seq: -0, time: 1, data: { turn: 1 }, }] as never, { - version: 0, id, createdAt: 1, isSeeded: false, + version: SESSION_FORMAT_VERSION, id, createdAt: 1, isSeeded: false, }, SessionLogOffset(0))).toThrow(/invalid event envelope/) }) @@ -60,7 +61,7 @@ describe('Session log positions', () => { source.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) const id = SessionId('child') const header: SessionHeader = { - version: 0, + version: SESSION_FORMAT_VERSION, id, createdAt: 1, isSeeded: true, @@ -90,7 +91,7 @@ describe('Session log positions', () => { const id = SessionId('suffix-child') const child = Session.create(id, assembled.snapshotEvents(), { - version: 0, + version: SESSION_FORMAT_VERSION, id, createdAt: 1, isSeeded: true, @@ -109,7 +110,7 @@ describe('Session log positions', () => { it('requires a separately supplied inherited cut for a seeded header', () => { const id = SessionId('missing-cut') expect(() => Session.create(id, [], { - version: 0, + version: SESSION_FORMAT_VERSION, id, createdAt: 1, isSeeded: true, @@ -119,7 +120,7 @@ describe('Session log positions', () => { it('requires an explicit constructor seed for seeded lineage', () => { const id = SessionId('missing-seed') expect(() => Session.create(id, undefined, { - version: 0, + version: SESSION_FORMAT_VERSION, id, createdAt: 1, isSeeded: true, @@ -129,12 +130,12 @@ describe('Session log positions', () => { it('requires the exact cut to agree with lineage and log length', () => { const unseededId = SessionId('unseeded-nonzero-cut') expect(() => Session.create(unseededId, [], { - version: 0, id: unseededId, createdAt: 1, isSeeded: false, + version: SESSION_FORMAT_VERSION, id: unseededId, createdAt: 1, isSeeded: false, }, SessionLogOffset(1))).toThrow(/unseeded session inherited event count must be 0/) const seededId = SessionId('seeded-oversized-cut') expect(() => Session.create(seededId, [], { - version: 0, id: seededId, createdAt: 1, isSeeded: true, + version: SESSION_FORMAT_VERSION, id: seededId, createdAt: 1, isSeeded: true, }, SessionLogOffset(1))).toThrow(/inherited event count exceeds its event log/) }) @@ -143,7 +144,7 @@ describe('Session log positions', () => { (value) => { const id = SessionId(`bad-inherited-count-${value}`) expect(() => Session.create(id, [], { - version: 0, + version: SESSION_FORMAT_VERSION, id, createdAt: 1, isSeeded: true, diff --git a/packages/core/session/tests/session.spec.ts b/packages/core/session/tests/session.spec.ts index 790af32632..8cf700ced6 100644 --- a/packages/core/session/tests/session.spec.ts +++ b/packages/core/session/tests/session.spec.ts @@ -187,6 +187,27 @@ describe('Session', () => { .toEqual([unrelatedPrimitiveData]) }) + it('rejects historical or malformed request-header lifecycle markers on seed/load', () => { + const base = { + type: 'request/header', seq: SessionSeq(0), time: 1, + data: { header: { config: { provider: 'mock', model: 'model' } }, reason: 'initial' }, + } as const + for (const reason of ['fallback', 'unknown', null]) { + const invalid = structuredClone(base) as unknown as SessionEvent + if (invalid.type !== 'request/header') throw new Error('test fixture must be a request header') + invalid.data.reason = reason as never + expect(() => Session.create(SessionId('invalid-header-reason'), [invalid])) + .toThrow('seed request/header at index 0 has an invalid reason') + } + for (const startsSeries of [false, 1, 'true']) { + const invalid = structuredClone(base) as unknown as SessionEvent + if (invalid.type !== 'request/header') throw new Error('test fixture must be a request header') + invalid.data.startsSeries = startsSeries as never + expect(() => Session.create(SessionId('invalid-series-marker'), [invalid])) + .toThrow('seed request/header at index 0 has an invalid startsSeries marker') + } + }) + it('rejects event-specific malformed message shapes on seed/load', () => { const user = { id: 'user', @@ -1002,7 +1023,7 @@ describe('Session', () => { cwd: '/accepted', parentSession: SessionId('parent'), isSeeded: true, - } + } satisfies SessionHeader const session = Session.create(SessionId('header-owned'), [], input, SessionLogOffset(0)) input.cwd = '/caller-mutated' @@ -1067,7 +1088,7 @@ describe('Session', () => { const cases: Array<{ header: unknown; error: RegExp }> = [ { header: 1, error: /not a plain JSON record/ }, { header: null, error: /not a plain JSON record/ }, - { header: { ...base, version: 1 }, error: /header version/ }, + { header: { ...base, version: SESSION_FORMAT_VERSION + 1 }, error: /header version/ }, { header: { ...base, createdAt: '123' }, error: /createdAt must be a non-negative safe integer/ }, { header: { ...base, cwd: 1 }, error: /header cwd must be a string/ }, { header: { ...base, cwd: 'relative' }, error: /header cwd must be an absolute path/ }, diff --git a/packages/experimental/agent-team/tests/projection-events.spec.ts b/packages/experimental/agent-team/tests/projection-events.spec.ts index b2f9911d3a..5a0f4c725a 100644 --- a/packages/experimental/agent-team/tests/projection-events.spec.ts +++ b/packages/experimental/agent-team/tests/projection-events.spec.ts @@ -1,5 +1,5 @@ import { describe, expect, it } from 'vitest' -import { SessionId, SessionSeq } from '@deepseek-ai/dsh-session' +import { SESSION_FORMAT_VERSION, SessionId, SessionSeq } from '@deepseek-ai/dsh-session' import type { SessionEvent, SessionEventMap, SessionEventType } from '@deepseek-ai/dsh-session' import { teamProjectionDefinition } from '../src/projection.ts' import type { TeamProjectionState, TeamState } from '../src/projection.ts' @@ -15,7 +15,12 @@ function event(type: T, data: SessionEventMap[T], se } function project(rootId: SessionId, events: readonly SessionEvent[]): TeamProjectionState { - let state = teamProjectionDefinition.init({ version: 0, id: rootId, createdAt: 0, isSeeded: false }) + let state = teamProjectionDefinition.init({ + version: SESSION_FORMAT_VERSION, + id: rootId, + createdAt: 0, + isSeeded: false, + }) for (const event of events) state = teamProjectionDefinition.apply(state, event) return state } diff --git a/packages/experimental/webworker-packer/tests/image-loadable.spec.ts b/packages/experimental/webworker-packer/tests/image-loadable.spec.ts index e151404717..be15003b20 100644 --- a/packages/experimental/webworker-packer/tests/image-loadable.spec.ts +++ b/packages/experimental/webworker-packer/tests/image-loadable.spec.ts @@ -50,7 +50,7 @@ describe('preview example overlays', () => { .toContain("previewStatus = 'ready'") expect(new TextDecoder().decode(result.files['workspace/.agents/skills/preview-tour/SKILL.md'])) .toContain('name: preview-tour') - expect(Object.keys(result.files).filter(path => path.endsWith('/session.jsonl'))).toHaveLength(3) + expect(Object.keys(result.files).filter(path => path.endsWith('/session.v1.jsonl'))).toHaveLength(3) }) it('fails loud when a declared seed tree is absent', () => { diff --git a/packages/experimental/webworker-runtime/tests/fixtures/vfs-example/home/sessions/--dsh-workspace--/preview-architecture-review/session.jsonl b/packages/experimental/webworker-runtime/tests/fixtures/vfs-example/home/sessions/--dsh-workspace--/preview-architecture-review/session.v1.jsonl similarity index 99% rename from packages/experimental/webworker-runtime/tests/fixtures/vfs-example/home/sessions/--dsh-workspace--/preview-architecture-review/session.jsonl rename to packages/experimental/webworker-runtime/tests/fixtures/vfs-example/home/sessions/--dsh-workspace--/preview-architecture-review/session.v1.jsonl index 7fd433c32a..cd24b7d5eb 100644 --- a/packages/experimental/webworker-runtime/tests/fixtures/vfs-example/home/sessions/--dsh-workspace--/preview-architecture-review/session.jsonl +++ b/packages/experimental/webworker-runtime/tests/fixtures/vfs-example/home/sessions/--dsh-workspace--/preview-architecture-review/session.v1.jsonl @@ -1,4 +1,4 @@ -{"type":"session","version":0,"id":"preview-architecture-review","createdAt":1787472100000,"cwd":"/dsh/workspace","parentSession":"preview-showcase","seedLength":169,"origin":"subagent","delegationDepth":1,"agentPreset":"standard"} +{"type":"session","version":1,"id":"preview-architecture-review","createdAt":1787472100000,"cwd":"/dsh/workspace","parentSession":"preview-showcase","seedLength":169,"origin":"subagent","delegationDepth":1,"agentPreset":"standard"} {"type":"turn/start","data":{"turn":1},"seq":0,"time":1787472000000} {"type":"user/message","data":{"id":"preview-user-01","role":"user","content":[{"type":"text","text":"History checkpoint 01: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":1,"time":1787472000001} {"type":"session/title","data":{"title":"WebWorker Preview Showcase","messageSeqs":[],"source":{"kind":"user"}},"seq":2,"time":1787472000002} diff --git a/packages/experimental/webworker-runtime/tests/fixtures/vfs-example/home/sessions/--dsh-workspace--/preview-follow-up-builder/session.jsonl b/packages/experimental/webworker-runtime/tests/fixtures/vfs-example/home/sessions/--dsh-workspace--/preview-follow-up-builder/session.v1.jsonl similarity index 95% rename from packages/experimental/webworker-runtime/tests/fixtures/vfs-example/home/sessions/--dsh-workspace--/preview-follow-up-builder/session.jsonl rename to packages/experimental/webworker-runtime/tests/fixtures/vfs-example/home/sessions/--dsh-workspace--/preview-follow-up-builder/session.v1.jsonl index 03b884adc6..da647a942f 100644 --- a/packages/experimental/webworker-runtime/tests/fixtures/vfs-example/home/sessions/--dsh-workspace--/preview-follow-up-builder/session.jsonl +++ b/packages/experimental/webworker-runtime/tests/fixtures/vfs-example/home/sessions/--dsh-workspace--/preview-follow-up-builder/session.v1.jsonl @@ -1,4 +1,4 @@ -{"type":"session","version":0,"id":"preview-follow-up-builder","createdAt":1787472200000,"cwd":"/dsh/workspace","parentSession":"preview-showcase","origin":"subagent","delegationDepth":1,"agentPreset":"standard"} +{"type":"session","version":1,"id":"preview-follow-up-builder","createdAt":1787472200000,"cwd":"/dsh/workspace","parentSession":"preview-showcase","origin":"subagent","delegationDepth":1,"agentPreset":"standard"} {"type":"turn/start","data":{"turn":1},"seq":0,"time":1787472200000} {"type":"user/message","data":{"id":"preview-builder-user","role":"user","content":[{"type":"text","text":"Check that the Preview workspace can support follow-up tasks."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":1,"time":1787472200001} {"type":"subagent/descriptor","data":{"version":3,"mode":"continuable","provider":"spawn","label":"Continue preview verification"},"seq":2,"time":1787472200002} diff --git a/packages/experimental/webworker-runtime/tests/fixtures/vfs-example/home/sessions/--dsh-workspace--/preview-showcase/session.jsonl b/packages/experimental/webworker-runtime/tests/fixtures/vfs-example/home/sessions/--dsh-workspace--/preview-showcase/session.v1.jsonl similarity index 99% rename from packages/experimental/webworker-runtime/tests/fixtures/vfs-example/home/sessions/--dsh-workspace--/preview-showcase/session.jsonl rename to packages/experimental/webworker-runtime/tests/fixtures/vfs-example/home/sessions/--dsh-workspace--/preview-showcase/session.v1.jsonl index 7b15d63fb3..3a8d03490e 100644 --- a/packages/experimental/webworker-runtime/tests/fixtures/vfs-example/home/sessions/--dsh-workspace--/preview-showcase/session.jsonl +++ b/packages/experimental/webworker-runtime/tests/fixtures/vfs-example/home/sessions/--dsh-workspace--/preview-showcase/session.v1.jsonl @@ -1,4 +1,4 @@ -{"type":"session","version":0,"id":"preview-showcase","createdAt":1787472000000,"cwd":"/dsh/workspace","delegationDepth":0,"agentPreset":"standard"} +{"type":"session","version":1,"id":"preview-showcase","createdAt":1787472000000,"cwd":"/dsh/workspace","delegationDepth":0,"agentPreset":"standard"} {"type":"turn/start","data":{"turn":1},"seq":0,"time":1787472000000} {"type":"user/message","data":{"id":"preview-user-01","role":"user","content":[{"type":"text","text":"History checkpoint 01: verify deterministic preview state."}],"source":{"kind":"user"}},"surfaceOp":"append","seq":1,"time":1787472000001} {"type":"session/title","data":{"title":"WebWorker Preview Showcase","messageSeqs":[],"source":{"kind":"user"}},"seq":2,"time":1787472000002} diff --git a/packages/experimental/webworker-runtime/tests/fixtures/vfs-example/home/storages/session_projcache.json b/packages/experimental/webworker-runtime/tests/fixtures/vfs-example/home/storages/session_projcache.json index e612124830..8414fdb43f 100644 --- a/packages/experimental/webworker-runtime/tests/fixtures/vfs-example/home/storages/session_projcache.json +++ b/packages/experimental/webworker-runtime/tests/fixtures/vfs-example/home/storages/session_projcache.json @@ -1,13 +1,14 @@ { "unit": { "name": "session_projcache", - "version": 5 + "version": 6 }, "global": null, "tables": { "sessions": { "preview-showcase": { "identity": { + "formatVersion": 1, "createdAt": 1787472000000, "cwd": "/dsh/workspace", "isSeeded": false, diff --git a/packages/experimental/webworker-runtime/tests/vfs-example-fixture.spec.ts b/packages/experimental/webworker-runtime/tests/vfs-example-fixture.spec.ts index bd7c0f6838..57c62329d3 100644 --- a/packages/experimental/webworker-runtime/tests/vfs-example-fixture.spec.ts +++ b/packages/experimental/webworker-runtime/tests/vfs-example-fixture.spec.ts @@ -1,8 +1,11 @@ import { readFileSync, readdirSync } from 'node:fs' import { join, relative } from 'node:path' import { describe, expect, it } from 'vitest' -import { Session, SessionId, type SessionEvent } from '@deepseek-ai/dsh-session' -import { scanLog } from '@deepseek-ai/dsh-session-persistence-jsonl/src/format.ts' +import { SESSION_FORMAT_VERSION, Session, SessionId, type SessionEvent } from '@deepseek-ai/dsh-session' +import { + generationLogFilename, + scanLog, +} from '@deepseek-ai/dsh-session-persistence-jsonl/src/format.ts' import { foldSubagentDescriptor } from '@deepseek-ai/dsh-subagent' import { buildVfsExampleFiles, @@ -28,7 +31,8 @@ function filesUnder(root: string): string[] { function readSession(id: string): ReturnType { return scanLog(readFileSync( - join(VFS_EXAMPLE_ROOT, 'home/sessions/--dsh-workspace--', id, 'session.jsonl'), + join(VFS_EXAMPLE_ROOT, 'home/sessions/--dsh-workspace--', id, + generationLogFilename(SESSION_FORMAT_VERSION, 'none')), )) } @@ -59,14 +63,21 @@ describe('WebWorker preview VFS example', () => { unit: { name: string; version: number } tables: { sessions: Record } } - expect(cache.unit).toEqual({ name: 'session_projcache', version: 5 }) + expect(cache.unit).toEqual({ name: 'session_projcache', version: 6 }) expect(cache.tables.sessions[VFS_EXAMPLE_SESSION_IDS.main]).toMatchObject({ identity: { + formatVersion: SESSION_FORMAT_VERSION, createdAt: 1_787_472_000_000, cwd: '/dsh/workspace', isSeeded: false, diff --git a/packages/experimental/webworker-runtime/tests/vfs-example-fixture.ts b/packages/experimental/webworker-runtime/tests/vfs-example-fixture.ts index 20ac1e3838..c648c9b582 100644 --- a/packages/experimental/webworker-runtime/tests/vfs-example-fixture.ts +++ b/packages/experimental/webworker-runtime/tests/vfs-example-fixture.ts @@ -2,6 +2,7 @@ import { fileURLToPath } from 'node:url' import { + SESSION_FORMAT_VERSION, SessionId, SessionLogOffset, SessionSeq, @@ -11,8 +12,9 @@ import { type SessionSeq as SessionSeqType, } from '@deepseek-ai/dsh-session' import { - eventLines, projectKey, toHeaderLine, + eventLines, generationLogFilename, projectKey, toHeaderLine, } from '@deepseek-ai/dsh-session-persistence-jsonl/src/format.ts' +import { projectionCacheDomainSpec } from '@deepseek-ai/dsh-session-projection-cache' import { snapshotSubagentDescriptor } from '@deepseek-ai/dsh-subagent' /** Root copied by the preview image's repository adapter. */ @@ -383,7 +385,7 @@ function header( const inheritedEventCount = child?.seedLength ?? SessionLogOffset(0) return { meta: { - version: 0, + version: SESSION_FORMAT_VERSION, id, createdAt, cwd: WORKSPACE, @@ -410,14 +412,16 @@ function renderLog( export function buildVfsExampleFiles(): ReadonlyMap { const main = mainLog() const project = projectKey(WORKSPACE) - const sessionPath = (id: string): string => `home/sessions/${project}/${id}/session.jsonl` + const sessionPath = (id: string): string => + `home/sessions/${project}/${id}/${generationLogFilename(SESSION_FORMAT_VERSION, 'none')}` const projectionCache = `${JSON.stringify({ - unit: { name: 'session_projcache', version: 5 }, + unit: { name: 'session_projcache', version: projectionCacheDomainSpec.version }, global: null, tables: { sessions: { [VFS_EXAMPLE_SESSION_IDS.main]: { identity: { + formatVersion: SESSION_FORMAT_VERSION, createdAt: CREATED_AT, cwd: WORKSPACE, isSeeded: false, diff --git a/packages/extensions/tool-cordis/src/api-catalog.ts b/packages/extensions/tool-cordis/src/api-catalog.ts index 55761bf933..5b5985371c 100644 --- a/packages/extensions/tool-cordis/src/api-catalog.ts +++ b/packages/extensions/tool-cordis/src/api-catalog.ts @@ -1465,18 +1465,18 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [ methods: [ { signature: 'abstract locate(meta: SessionHeader): SessionLocation | undefined', - description: 'Resolve this backend\'s independent local artifact for a session without reading, creating, flushing, or otherwise materializing it. A backend that does not own one artifact per Session returns `undefined`.', + description: 'Resolve this backend\'s current-generation target for a session without reading, creating, flushing, or otherwise materializing it. Historical generations may live at other immutable paths; listing descriptors carry the exact selected stored location. A backend without per-Session files returns `undefined`.', parameters: [{ name: 'meta', description: 'the immutable session header whose artifact is requested.' }], returns: 'the backend-specific absolute location, when one exists.', }, { signature: 'abstract readonly supportsRawArtifacts: boolean', - description: 'Whether this backend exposes one verbatim raw artifact per session. A backend that declares `true` must override readRaw.', + description: 'Whether this backend exposes the selected verbatim raw generation per Session. A backend that declares `true` must override readRaw.', parameters: [], }, { signature: 'readRaw(_id: SessionId, signal?: AbortSignal): Promise', - description: 'Read a session\'s backend-owned artifact text verbatim — the exact durable bytes the backend wrote (decoded from its physical encoding, e.g. a decompressed JSONL). The returned `content` is the raw text, not a reconstruction from parsed events, so it preserves backend-specific serialization (chunk packing, key order, line breaks). Callers first test supportsRawArtifacts; `undefined` then means only that the requested session has no materialized artifact.', + description: 'Read a session\'s backend-owned artifact text verbatim — the exact durable bytes the backend wrote (decoded from its physical encoding, e.g. a decompressed JSONL). The returned `content` is the raw text, not a reconstruction from parsed events, so it preserves backend-specific serialization (chunk packing, key order, line breaks). Callers first test supportsRawArtifacts; `undefined` then means only that the requested session has no materialized artifact. Reading a supported historical artifact leaves that generation untouched and exclusively publishes a separate repaired current successor; an already-current artifact is not rewritten.', parameters: [{ name: '_id', description: 'the persisted session to read (unused by the default: no per-session artifact).' }, { name: 'signal', description: 'optional cancellation for backend read work.' }], returns: 'the raw artifact plus its parsed header, or `undefined` when the session is absent.', throws: ['when this backend does not expose per-session raw artifacts.'], @@ -1498,45 +1498,45 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [ }, { signature: 'async prepare(id: SessionId, signal?: AbortSignal): Promise', - description: 'Prepare the exact unpublished Session used by resume. Implementations may reuse object graphs retained by an earlier inspect after confirming their durable revision is still current; disposal releases an unpublished reservation. Revision retries require the durable log to remain unchanged for one read/check round trip; continuous external writers may delay completion.', - parameters: [{ name: 'id', description: 'persisted session to prepare.' }, { name: 'signal', description: 'optional cancellation for preparation work.' }], + description: 'Prepare the exact unpublished Session used by resume. Implementations may reuse object graphs retained by an earlier inspect after confirming their durable revision is still current; disposal releases an unpublished reservation. Revision retries require the durable log to remain unchanged for one read/check round trip; continuous external writers may delay completion. Preparing a supported historical artifact first persists its migration and current-format repair, while current input takes the no-write fast path.', + parameters: [{ name: 'id', description: 'persisted session to prepare.' }, { name: 'signal', description: 'optional cancellation for this caller\'s wait. A shared preparation or historical migration already started for another observer may continue to completion.' }], returns: 'one owned unpublished Session preparation.', }, { signature: 'abstract load(id: SessionId): Promise', - description: 'Load an immutable balanced logical view and commit any required cold recovery. A complete interrupted final turn is preserved and durably closed with missing tool errors plus any open step and turn boundaries; only a torn final record is discarded. Unknown versions and corruption in the committed prefix reject. Implementations MUST NOT crash-repair an identity still bound to a live Session: a balanced live log may return as a durable snapshot, while an open live turn rejects. Returned values may be shared with immutable live or prepared state and must not be mutated. Revision-based implementations may wait for one stable read/check round trip.', + description: 'Load an immutable balanced current logical view and commit any required cold recovery. A supported historical artifact remains immutable while a separate repaired current successor is published before restoration. A complete interrupted final turn is preserved and durably closed with missing tool errors plus any open step and turn boundaries; only a torn final record is discarded. Unknown versions and corruption in the committed prefix reject. Implementations MUST NOT crash-repair an identity still bound to a live Session: a balanced live log may return as a durable snapshot, while an open live turn rejects. Returned values may be shared with immutable live or prepared state and must not be mutated. Revision-based implementations may wait for one stable read/check round trip.', parameters: [{ name: 'id', description: 'the persisted session to reload.' }], returns: 'the header and a log ending on a balanced `turn/end`.', }, { signature: 'abstract inspect(id: SessionId, signal?: AbortSignal): Promise', - description: 'Inspect an immutable logical session without committing recovery or publishing it. A cold complete interrupted turn receives synthetic closers in memory and a torn physical tail remains untouched. An already-live Session instead yields its current immutable snapshot, which may contain an open turn and its `session/end-seed` boundary. Coordinator-backed implementations retain the exact cold unpublished Session for bounded reuse by a later prepare. A stale ready source is reloaded; a source already committing or reserved for resume remains exclusive, and inspection may borrow its immutable view. Callers borrow only the immutable header and log. Continuous external writers may delay revision convergence.', - parameters: [{ name: 'id', description: 'the persisted session to inspect.' }, { name: 'signal', description: 'optional cancellation for queued and backend read work.' }], + description: 'Inspect an immutable current logical session without publishing a live Session. For an already-current cold artifact, a complete interrupted turn receives synthetic closers only in memory and a torn physical tail remains untouched. A supported historical artifact first publishes its separate repaired current successor, so inspection is not storage-read-only in that case. An already-live Session instead yields its current immutable snapshot, which may contain an open turn and its `session/end-seed` boundary. Coordinator-backed implementations retain the exact cold unpublished Session for bounded reuse by a later prepare. A stale ready source is reloaded; a source already committing or reserved for resume remains exclusive, and inspection may borrow its immutable view. Callers borrow only the immutable header and log. Continuous external writers may delay revision convergence.', + parameters: [{ name: 'id', description: 'the persisted session to inspect.' }, { name: 'signal', description: 'optional cancellation for this observer. Shared cold preparation and an already-started historical migration may continue for another inspector or later resume.' }], returns: 'the validated header and current logical event log.', }, { signature: 'abstract borrowSession(id: SessionId, signal?: AbortSignal): Promise', - description: 'Borrow one exact inspection while retaining any reusable prepared source. A cold observation must pin the exact prepared Session that a later prepare reserves. Implementations must not degrade this operation to a detached inspect result.', - parameters: [{ name: 'id', description: 'persisted session to observe.' }, { name: 'signal', description: 'optional cancellation for preparation work.' }], + description: 'Borrow one exact inspection while retaining any reusable prepared source. A cold observation must pin the exact prepared Session that a later prepare reserves. Implementations must not degrade this operation to a detached inspect result. Borrowing a supported historical artifact first persists its migration and current-format repair.', + parameters: [{ name: 'id', description: 'persisted session to observe.' }, { name: 'signal', description: 'optional cancellation for this observer\'s wait; shared preparation or migration work may continue for another owner.' }], returns: 'a disposable immutable observation.', }, { signature: 'abstract readFrom(id: SessionId, fromSeq: SessionLogOffset, signal?: AbortSignal): Promise', - description: 'Read the stored events from `fromSeq` onward — the read-from-seq primitive for read models that resume from a watermark (e.g. a persisted projection cache folding only the tail past its checkpoint). Unlike inspect, it is a detached physical suffix read: no preparation cache, torn-tail truncation, synthetic closers, or coordinator-state publication. Only events from the valid contiguous stored prefix are returned, so a torn fragment never reaches the caller. `fromSeq` at or beyond the stored prefix returns an empty event list (never an error). A backend whose medium can seek by seq may read only the suffix; sequential media such as JSONL still parse the whole artifact and skip forward. The primitive bounds what is returned and refolded, not every backend\'s physical read.', + description: 'Read the stored events from `fromSeq` onward — the read-from-seq primitive for read models that resume from a watermark (e.g. a persisted projection cache folding only the tail past its checkpoint). Unlike inspect, it is a detached physical suffix read: no preparation cache or coordinator-state publication. Current input performs no torn-tail truncation or synthetic repair. A supported historical artifact leaves its exact source unchanged and publishes a separate repaired current successor, so its returned suffix may include those current closers. Only events from the valid contiguous stored prefix are returned, so a torn fragment never reaches the caller. `fromSeq` at or beyond the stored prefix returns an empty event list (never an error). A backend whose medium can seek by seq may read only the suffix; sequential media such as JSONL still parse the whole artifact and skip forward. The primitive bounds what is returned and refolded, not every backend\'s physical read.', parameters: [{ name: 'id', description: 'the persisted session to read.' }, { name: 'fromSeq', description: 'first event offset to include.' }, { name: 'signal', description: 'optional cancellation for queued and backend read work.' }], returns: 'storage metadata, the requested offset, and stored events with `seq >= fromSeq`.', }, { - signature: 'abstract list(signal?: AbortSignal): Promise', + signature: 'abstract list(signal?: AbortSignal): Promise', description: 'Lightweight listing from metadata, without a full-log parse.', parameters: [{ name: 'signal', description: 'optional cancellation for backend listing work.' }], - returns: 'one header per materialized session.', + returns: 'one isolated descriptor per materialized artifact.', }, { signature: 'abstract listSnapshots(signal?: AbortSignal): Promise', - description: 'List materialized sessions with cheap per-log change tokens.\n\nRepeated observations of an unchanged log return the same revision. A successful mutating load repair changes the next listed revision. Revisions also distinguish independently backed stores so backend-local counters cannot compare equal across different persistence sources.', + description: 'List materialized sessions with cheap per-log change tokens.\n\nRepeated observations of an unchanged log return the same revision. A successful mutating load repair changes the next listed revision; so does migration publication from any supported historical body read. Revisions also distinguish independently backed stores so backend-local counters cannot compare equal across different persistence sources.', parameters: [{ name: 'signal', description: 'optional cancellation for backend snapshot-listing work.' }], - returns: 'one header and opaque revision per materialized session without loading full logs.', + returns: 'one isolated current, migration-required, unsupported, or malformed descriptor plus its opaque revision per materialized artifact, without loading full logs.', }, ], }, @@ -3866,6 +3866,10 @@ export const TYPE_API: readonly TypeApiEntry[] = [ name: 'CredentialRef', declaration: 'export type CredentialRef = Branded<\'CredentialRef\'>;', }, + { + name: 'CurrentSessionPersistenceListing', + declaration: 'export interface CurrentSessionPersistenceListing {\n readonly status: \'current\';\n readonly header: SessionHeader;\n readonly storedVersion: number;\n readonly targetVersion: number;\n readonly location?: SessionLocation;\n}', + }, { name: 'DeepSeekLlmApiExtensionMap', declaration: 'export interface DeepSeekLlmApiExtensionMap {\n}', @@ -4374,6 +4378,10 @@ export const TYPE_API: readonly TypeApiEntry[] = [ name: 'LspRange', declaration: 'export interface LspRange {\n readonly start: LspPosition;\n readonly end: LspPosition;\n}', }, + { + name: 'MalformedSessionPersistenceListing', + declaration: 'export interface MalformedSessionPersistenceListing {\n readonly status: \'malformed\';\n readonly targetVersion: number;\n readonly location: SessionLocation;\n readonly reason: string;\n}', + }, { name: 'ManualCompactAgentContext', declaration: 'export interface ManualCompactAgentContext extends CompactionAgentContext {\n runMaintenance(task: (signal: AbortSignal) => Promise): Promise;\n}', @@ -4470,6 +4478,10 @@ export const TYPE_API: readonly TypeApiEntry[] = [ name: 'MessageSourceMap', declaration: 'export interface MessageSourceMap {\n user: {\n kind: \'user\';\n };\n plugin: {\n kind: \'plugin\';\n plugin: string;\n } & ContextFormed;\n model: ModelMessageSource;\n tool: ToolMessageSource;\n}', }, + { + name: 'MigrationRequiredSessionPersistenceListing', + declaration: 'export interface MigrationRequiredSessionPersistenceListing {\n readonly status: \'migration-required\';\n readonly header: SessionHeader;\n readonly storedVersion: number;\n readonly targetVersion: number;\n readonly location?: SessionLocation;\n}', + }, { name: 'ModelCatalog', declaration: 'export interface ModelCatalog {\n readonly default: ModelSelection;\n readonly routableProviders: readonly string[];\n readonly groups: readonly ModelProviderGroup[];\n readonly failures: readonly ModelCatalogFailure[];\n}', @@ -4936,7 +4948,7 @@ export const TYPE_API: readonly TypeApiEntry[] = [ }, { name: 'SessionHeader', - declaration: 'export interface SessionHeader {\n readonly version: number;\n readonly id: SessionId;\n readonly createdAt: number;\n readonly cwd?: string;\n readonly parentSession?: SessionId;\n readonly isSeeded: boolean;\n readonly origin?: \'subagent\';\n readonly delegationDepth?: number;\n readonly agentPreset?: string;\n}', + declaration: 'export interface SessionHeader {\n readonly version: typeof SESSION_FORMAT_VERSION;\n readonly id: SessionId;\n readonly createdAt: number;\n readonly cwd?: string;\n readonly parentSession?: SessionId;\n readonly isSeeded: boolean;\n readonly origin?: \'subagent\';\n readonly delegationDepth?: number;\n readonly agentPreset?: string;\n}', }, { name: 'SessionHistoryRecord', @@ -5006,13 +5018,17 @@ export const TYPE_API: readonly TypeApiEntry[] = [ name: 'SessionPageRequest', declaration: 'export interface SessionPageRequest {\n readonly address: SessionAddress;\n readonly throughSeq: number;\n readonly beforeSeq?: number;\n readonly maxMessages?: number;\n}', }, + { + name: 'SessionPersistenceListing', + declaration: 'export type SessionPersistenceListing = CurrentSessionPersistenceListing | MigrationRequiredSessionPersistenceListing | UnsupportedSessionPersistenceListing | MalformedSessionPersistenceListing;', + }, { name: 'SessionPersistenceRevision', declaration: 'export type SessionPersistenceRevision = Branded<\'SessionPersistenceRevision\'>;', }, { name: 'SessionPersistenceSnapshot', - declaration: 'export interface SessionPersistenceSnapshot {\n header: SessionHeader;\n revision: SessionPersistenceRevision;\n}', + declaration: 'export type SessionPersistenceSnapshot = SessionPersistenceListing & {\n readonly revision: SessionPersistenceRevision;\n};', }, { name: 'SessionPreparation', @@ -5068,7 +5084,7 @@ export const TYPE_API: readonly TypeApiEntry[] = [ }, { name: 'SessionRecord', - declaration: 'export interface SessionRecord {\n header: SessionHeader;\n live: boolean;\n persisted: boolean;\n}', + declaration: 'export interface SessionRecord {\n header: SessionHeader;\n location?: SessionLocation;\n live: boolean;\n persisted: boolean;\n}', }, { name: 'SessionReferenceCandidate', @@ -5946,6 +5962,10 @@ export const TYPE_API: readonly TypeApiEntry[] = [ name: 'TypertTypeModel', declaration: 'export interface TypertTypeModel {\n readonly name: string;\n readonly declaration: string;\n}', }, + { + name: 'UnsupportedSessionPersistenceListing', + declaration: 'export interface UnsupportedSessionPersistenceListing {\n readonly status: \'unsupported\';\n readonly storedVersion?: number;\n readonly targetVersion: number;\n readonly location: SessionLocation;\n readonly reason: string;\n}', + }, { name: 'UpdateTeamTaskRequest', declaration: 'export interface UpdateTeamTaskRequest {\n readonly taskId: TeamTaskId;\n readonly expectedRevision: number;\n readonly action: TeamTaskAction;\n readonly subject?: string;\n readonly description?: string;\n readonly blockedBy?: readonly TeamTaskId[];\n readonly writeScopes?: readonly string[];\n readonly owner?: string;\n}', diff --git a/packages/feedback/message-feedback/src/index.ts b/packages/feedback/message-feedback/src/index.ts index e07a8b809c..95bfdf6425 100644 --- a/packages/feedback/message-feedback/src/index.ts +++ b/packages/feedback/message-feedback/src/index.ts @@ -10,6 +10,7 @@ import s from '@deepseek-ai/schemastery' import { SessionLogOffset } from '@deepseek-ai/dsh-session' import { deriveEventMessage, isAppendSurfaceEvent } from '@deepseek-ai/dsh-session/surface' import type { SessionHeader, SessionId } from '@deepseek-ai/dsh-session/types' +import { isReadableSessionPersistenceListing } from '@deepseek-ai/dsh-session-persistence' import type { SessionInspection } from '@deepseek-ai/dsh-session-persistence' import type { KvTable } from '@deepseek-ai/dsh-storage-domain' import { TypertRemoteService, Remote } from '@deepseek-ai/dsh-typert-protocol' @@ -304,7 +305,8 @@ export class MessageFeedbackService extends TypertRemoteService { private async inspectSession(sessionId: SessionId): Promise { if (this.ctx.sessions.get(sessionId) === undefined) { const snapshots = await this.ctx.sessionPersistence.listSnapshots() - if (!snapshots.some(snapshot => snapshot.header.id === sessionId) + if (!snapshots.some(snapshot => isReadableSessionPersistenceListing(snapshot) + && snapshot.header.id === sessionId) && this.ctx.sessions.get(sessionId) === undefined) { return rejected({ code: 'session-not-found', sessionId }) } diff --git a/packages/feedback/message-feedback/tests/helpers.ts b/packages/feedback/message-feedback/tests/helpers.ts index 3ae1363506..f23ec2f1fb 100644 --- a/packages/feedback/message-feedback/tests/helpers.ts +++ b/packages/feedback/message-feedback/tests/helpers.ts @@ -18,6 +18,7 @@ import SessionPersistence, { type SessionEventSuffix, type SessionInspection, type SessionLocation, + type SessionPersistenceListing, type SessionPersistenceSnapshot, } from '@deepseek-ai/dsh-session-persistence' import Storage from '@deepseek-ai/dsh-storage' @@ -173,14 +174,22 @@ class TestPersistence extends SessionPersistence { } } - list(): Promise { - return Promise.resolve([...this.durable.values()].map(value => value.meta)) + list(): Promise { + return Promise.resolve([...this.durable.values()].map(value => ({ + status: 'current', + header: value.meta, + storedVersion: SESSION_FORMAT_VERSION, + targetVersion: SESSION_FORMAT_VERSION, + }))) } async listSnapshots(): Promise { await this.onListSnapshots?.() return [...this.durable.values()].map((value, index) => ({ + status: 'current', header: value.meta, + storedVersion: SESSION_FORMAT_VERSION, + targetVersion: SESSION_FORMAT_VERSION, revision: SessionPersistenceRevision(`test:${index}:${value.events.length}`), })) } diff --git a/packages/feedback/message-feedback/tests/message-feedback.spec.ts b/packages/feedback/message-feedback/tests/message-feedback.spec.ts index 92b65ce2d4..95a302a5a9 100644 --- a/packages/feedback/message-feedback/tests/message-feedback.spec.ts +++ b/packages/feedback/message-feedback/tests/message-feedback.spec.ts @@ -2,7 +2,9 @@ import { randomUUID } from 'node:crypto' import { afterEach, describe, expect, it, vi } from 'vitest' import { Context } from '@deepseek-ai/cordis' import type { MessageId } from '@deepseek-ai/dsh-llm/brand' -import { Session, SessionId } from '@deepseek-ai/dsh-session' +import { + SESSION_FORMAT_VERSION, Session, SessionId, SessionLogOffset, +} from '@deepseek-ai/dsh-session' import { remoteMethods } from '@deepseek-ai/dsh-typert-protocol' import MessageFeedbackService, { messageFeedbackRowSchema } from '../src/index.ts' import type { @@ -261,8 +263,18 @@ describe('MessageFeedbackService public contract', () => { const rawCtx = new Context() rawCtx.provide('sessions', { get: () => undefined } as never) rawCtx.provide('sessionPersistence', { - listSnapshots: () => Promise.resolve([{ header: fixture.session.header, revision: 'test' }]), - inspect: () => Promise.resolve({ meta: fixture.session.header, events: fixture.session.snapshotEvents() }), + listSnapshots: () => Promise.resolve([{ + status: 'current', + header: fixture.session.header, + storedVersion: SESSION_FORMAT_VERSION, + targetVersion: SESSION_FORMAT_VERSION, + revision: 'test', + }]), + inspect: () => Promise.resolve({ + meta: fixture.session.header, + inheritedEventCount: SessionLogOffset(0), + events: fixture.session.snapshotEvents(), + }), } as never) const raw = new MessageFeedbackService(rawCtx, { maxNoteBytes: 1 }) await expect(raw.list({ sessionId: fixture.session.id })) diff --git a/packages/fs/tool-str-replace-editor/tests/tools.spec.ts b/packages/fs/tool-str-replace-editor/tests/tools.spec.ts index 0965db21d2..7d32b26d72 100644 --- a/packages/fs/tool-str-replace-editor/tests/tools.spec.ts +++ b/packages/fs/tool-str-replace-editor/tests/tools.spec.ts @@ -5,7 +5,7 @@ import { afterEach, describe, expect, it } from 'vitest' import { Context } from '@deepseek-ai/cordis' import { FsVersion } from '@deepseek-ai/dsh-fs' import { ToolCallId } from '@deepseek-ai/dsh-llm' -import { Session, SessionId } from '@deepseek-ai/dsh-session' +import { SESSION_FORMAT_VERSION, Session, SessionId } from '@deepseek-ai/dsh-session' import AgentRegistry, { Inbox } from '@deepseek-ai/dsh-agent' import type { Agent } from '@deepseek-ai/dsh-agent' import LocalFileSystem from '@deepseek-ai/dsh-fs-local' @@ -29,7 +29,9 @@ afterEach(async () => { function agent(ctx: Context, cwd: string): Agent { const id = SessionId(`str-replace-editor-owner-${callNumber}`) const scope = ctx.plugin(() => {}) - const session = Session.create(id, [], { version: 0, id, createdAt: 0, cwd, isSeeded: false }) + const session = Session.create(id, [], { + version: SESSION_FORMAT_VERSION, id, createdAt: 0, cwd, isSeeded: false, + }) const value: Agent = { id, options: {}, diff --git a/packages/preset/agent-presets/tests/session.spec.ts b/packages/preset/agent-presets/tests/session.spec.ts index 255fe0cfa8..31ecfea3bf 100644 --- a/packages/preset/agent-presets/tests/session.spec.ts +++ b/packages/preset/agent-presets/tests/session.spec.ts @@ -1,14 +1,14 @@ /** The Session projection that records which preset a Session runs. */ import { describe, expect, it } from 'vitest' -import { SessionId, SessionSeq } from '@deepseek-ai/dsh-session' +import { SESSION_FORMAT_VERSION, SessionId, SessionSeq } from '@deepseek-ai/dsh-session' import type { SessionEvent, SessionHeader } from '@deepseek-ai/dsh-session' import { agentPresetProjectionDefinition } from '../src/session.ts' /** A header carrying the creation-time preset, if any. */ function header(agentPreset?: string): SessionHeader { return { - version: 0, + version: SESSION_FORMAT_VERSION, id: SessionId('s'), createdAt: 1, isSeeded: false, diff --git a/packages/sandbox/sandbox-policy/tests/policy.spec.ts b/packages/sandbox/sandbox-policy/tests/policy.spec.ts index 7ba166354b..608abeeade 100644 --- a/packages/sandbox/sandbox-policy/tests/policy.spec.ts +++ b/packages/sandbox/sandbox-policy/tests/policy.spec.ts @@ -10,7 +10,7 @@ import { join, resolve, sep } from 'node:path' import { describe, expect, it } from 'vitest' import { Context } from '@deepseek-ai/cordis' import type { Agent } from '@deepseek-ai/dsh-agent' -import { Session, SessionId } from '@deepseek-ai/dsh-session' +import { SESSION_FORMAT_VERSION, Session, SessionId } from '@deepseek-ai/dsh-session' import SandboxPolicyService, { SANDBOX_MODES, setSandboxMode } from '@deepseek-ai/dsh-sandbox-policy' import SessionProjectionRegistry from '@deepseek-ai/dsh-session-projection' import SystemPrompt, { renderContextSnapshot, renderPrompt } from '@deepseek-ai/dsh-system-prompt' @@ -25,7 +25,7 @@ async function mounted(config: { mode?: 'read-only' | 'workspace-write' | 'dange function session(id: string, cwd?: string): Session { const sessionId = SessionId(id) return Session.create(sessionId, undefined, { - version: 0, + version: SESSION_FORMAT_VERSION, id: sessionId, createdAt: 0, isSeeded: false, diff --git a/packages/schedule/schedule/tests/projection.spec.ts b/packages/schedule/schedule/tests/projection.spec.ts index a6221e323b..ed9f8881a9 100644 --- a/packages/schedule/schedule/tests/projection.spec.ts +++ b/packages/schedule/schedule/tests/projection.spec.ts @@ -1,6 +1,6 @@ import { afterEach, describe, expect, it } from 'vitest' import { Context } from '@deepseek-ai/cordis' -import SessionStore, { SessionId, SessionLogOffset, SessionSeq } from '@deepseek-ai/dsh-session' +import SessionStore, { SESSION_FORMAT_VERSION, SessionId, SessionLogOffset, SessionSeq } from '@deepseek-ai/dsh-session' import type { SessionEvent, SessionHeader } from '@deepseek-ai/dsh-session' import SessionProjectionRegistry from '@deepseek-ai/dsh-session-projection' import { apply as applySchedule } from '../src/index.ts' @@ -10,7 +10,7 @@ import type { ScheduleRecord } from '../src/types.ts' const contexts: Context[] = [] const RESTORE_HEADER: SessionHeader = { - version: 0, + version: SESSION_FORMAT_VERSION, id: SessionId('schedule-projection'), createdAt: 0, isSeeded: false, diff --git a/packages/session-query/session-log-export/src/archive.ts b/packages/session-query/session-log-export/src/archive.ts index 2f7d1fbd11..f4c613a427 100644 --- a/packages/session-query/session-log-export/src/archive.ts +++ b/packages/session-query/session-log-export/src/archive.ts @@ -1,7 +1,7 @@ /** * Host-side session-log download: streams one ZIP archive whose files are the * sessions' stored artifact text verbatim plus every referenced media object. - * The root artifact sits under its original base name (`session.jsonl`); each + * The root artifact sits under its backend-reported generation basename; each * subagent descendant under `subagents//`; each image referenced * by any included log under `media/.` (content-addressed, * so one archive never duplicates a shared image). No manifest is written — diff --git a/packages/session-query/session-log-export/tests/archive.host.spec.ts b/packages/session-query/session-log-export/tests/archive.host.spec.ts index 66ebe197c0..ab138f7d17 100644 --- a/packages/session-query/session-log-export/tests/archive.host.spec.ts +++ b/packages/session-query/session-log-export/tests/archive.host.spec.ts @@ -10,7 +10,7 @@ import { describe, expect, it, vi } from 'vitest' import { Context } from '@deepseek-ai/cordis' import { unzipSync, strFromU8 } from 'fflate' import type { ImageAttachmentRef } from '@deepseek-ai/dsh-attachment' -import { SessionLogOffset } from '@deepseek-ai/dsh-session' +import { SESSION_FORMAT_VERSION, SessionLogOffset } from '@deepseek-ai/dsh-session' import type { SessionHeader, SessionId } from '@deepseek-ai/dsh-session' import type { SessionLineageNode } from '@deepseek-ai/dsh-session-query' import type { SessionRawArtifact } from '@deepseek-ai/dsh-session-persistence' @@ -22,7 +22,7 @@ const sid = (id: string): SessionId => id as SessionId function header(id: string, parentSession?: SessionId): SessionHeader { return { - version: 0, + version: SESSION_FORMAT_VERSION, id: sid(id), createdAt: 1000, cwd: '/proj', @@ -165,7 +165,8 @@ describe('session export compression config', () => { describe('session.export download endpoint', () => { it('streams a ZIP with the root artifact verbatim under its original filename', async () => { - const api = await buildApi({ 'session-root': artifact('session-root') }) + const root = { ...artifact('session-root'), filename: 'session.v1.jsonl' } + const api = await buildApi({ 'session-root': root }) const response = await toFetchHandler(api).fetch( new Request('http://host/api/session.export?sessionId=session-root'), ) @@ -173,8 +174,8 @@ describe('session.export download endpoint', () => { expect(response.headers.get('content-type')).toBe('application/zip') expect(response.headers.get('content-disposition')).toContain('dsh-session-session-root.zip') const files = unzipSync(await responseBytes(response)) - expect(Object.keys(files)).toEqual(['session.jsonl']) - expect(strFromU8(files['session.jsonl'] as Uint8Array)).toBe(artifact('session-root').content) + expect(Object.keys(files)).toEqual(['session.v1.jsonl']) + expect(strFromU8(files['session.v1.jsonl'] as Uint8Array)).toBe(root.content) }) it('preflights root preparation through HEAD without streaming a body', async () => { diff --git a/packages/session-query/session-log-export/tests/route.host.spec.ts b/packages/session-query/session-log-export/tests/route.host.spec.ts index 19d07bb9de..9541fc9869 100644 --- a/packages/session-query/session-log-export/tests/route.host.spec.ts +++ b/packages/session-query/session-log-export/tests/route.host.spec.ts @@ -1,7 +1,7 @@ import { Context } from '@deepseek-ai/cordis' import { HostConnectionService } from '@deepseek-ai/dsh-client-connection' import type { BrowserAuth } from '@deepseek-ai/dsh-client-connection/src/browser-auth.ts' -import { SessionLogOffset } from '@deepseek-ai/dsh-session' +import { SESSION_FORMAT_VERSION, SessionLogOffset } from '@deepseek-ai/dsh-session' import type { SessionHeader, SessionId } from '@deepseek-ai/dsh-session' import type { SessionRawArtifact } from '@deepseek-ai/dsh-session-persistence' import { strFromU8, unzipSync } from 'fflate' @@ -17,7 +17,7 @@ const sid = (value: string): SessionId => value as SessionId function artifact(id: string): SessionRawArtifact { const header: SessionHeader = { - version: 0, + version: SESSION_FORMAT_VERSION, id: sid(id), createdAt: 1, cwd: '/workspace', @@ -27,7 +27,7 @@ function artifact(id: string): SessionRawArtifact { return { meta: header, inheritedEventCount: SessionLogOffset(0), - filename: 'session.jsonl', + filename: 'session.v1.jsonl', content: `${JSON.stringify({ type: 'session', version: header.version, @@ -74,7 +74,7 @@ describe('Session log export Fetch route', () => { expect(response.status).toBe(200) expect(response.headers.get('content-type')).toBe('application/zip') const files = unzipSync(new Uint8Array(await response.arrayBuffer())) - const exported = strFromU8(files['session.jsonl'] as Uint8Array) + const exported = strFromU8(files['session.v1.jsonl'] as Uint8Array) expect(exported).toContain('"id":"session-1"') expect(exported).not.toContain('isSeeded') diff --git a/packages/session-query/session-query-sqlite/README.i18n.yaml b/packages/session-query/session-query-sqlite/README.i18n.yaml index 23e7b38fac..e8dd5b73b1 100644 --- a/packages/session-query/session-query-sqlite/README.i18n.yaml +++ b/packages/session-query/session-query-sqlite/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/session-query/session-query-sqlite/README.md -README.md: 1db3999186beebc10a6a3c6874122fa65f2787a3 -README.zh.md: f6ef419293ab0df3a3edfd171cc60bc62ee4bcb0 +README.md: 70c6b6deb1485aee5ddb08fa955f80fc118ec1df +README.zh.md: e32d6d08ecf26f90e660ebc95aa1b654bbf504af diff --git a/packages/session-query/session-query-sqlite/README.md b/packages/session-query/session-query-sqlite/README.md index 1db3999186..70c6b6deb1 100644 --- a/packages/session-query/session-query-sqlite/README.md +++ b/packages/session-query/session-query-sqlite/README.md @@ -9,7 +9,7 @@ English | [中文](README.zh.md) ## Summary -`dsh-session-query-sqlite` searches session history with a SQLite FTS5 index and returns ranked, cursor-paginated results grouped by session or within one session. Mount it together with `dsh-session-query` and you get full-text search plus the whole query surface — exact reads, filters, and traces — at once. Live sessions are indexed from memory and persisted sessions from a dedicated derived-index database, so results always reflect the newest state without touching the session-persistence store. Search is opt-in and off by default in shipped compositions: `openAt` decides whether the index opens at startup, at the first search, or never. Setup and usage come first; the implementation internals live in a collapsible developer section below. +`dsh-session-query-sqlite` searches session history with a SQLite FTS5 index and returns ranked, cursor-paginated results grouped by session or within one session. Mount it together with `dsh-session-query` and you get full-text search plus the whole query surface — exact reads, filters, and traces — at once. Live sessions are indexed from memory and persisted sessions from a dedicated derived-index database, so results reflect the newest stable state. Inspecting a supported historical Session can publish its current generation through the persistence service; the disposable FTS database remains separate from that authoritative store. Search is opt-in and off by default in shipped compositions: `openAt` decides whether the index opens at startup, at the first search, or never. Setup and usage come first; the implementation internals live in a collapsible developer section below. ## Table of Contents @@ -83,7 +83,7 @@ This section explains the design decisions behind the backend and points at the The backend is built on one separation and three commitments: -- **Derived index, never the source store.** The FTS rows live in a dedicated disposable database; the session-persistence database is never opened here. +- **Derived index, not the source store.** The FTS rows live in a dedicated disposable database; this package never opens a persistence artifact directly, but its inherited inspection may ask persistence to migrate a supported historical Session. - **Live-preferred observation.** One serialized state machine compares persistence snapshot revisions, inspects only new or changed logs, and reconciles in one transaction, so a search reflects the newest stable state. - **Generation-bound cursors.** Every corpus change bumps a generation; cursors carry the generation they were created under and fail stale rather than returning a shifted page. - **Literal phrases as data.** Caller query text is quoted into one FTS5 phrase so query syntax stays inert, and reserved highlight markers are stripped from documents before indexing. @@ -101,7 +101,7 @@ The design history lives in the [SQLite FTS5 session search note](../../../.agen ### Index lifecycle -Persisted FTS rows live in a dedicated derived database and survive restarts; live sessions use connection-local TEMP tables that shadow the durable base for the same session and reveal it again when the live owner detaches. Both tables retain the exact inherited cut in numeric `seed_length`; reconstructed headers expose only `isSeeded`, while the cut participates in live fingerprints and persisted source revisions. Each search runs one serialized observation: list persistence snapshots, compare per-session revisions with the indexed rows, inspect only new or changed logs, extract semantic documents, and commit the reconciliation in one transaction before running the query. Repeated queries and unchanged reopens inspect nothing; switching stores or observing new, changed, deleted, or externally repaired sources reconciles on the next stable observation. Source or transaction failure commits nothing and the next search retries. +Persisted FTS rows live in a dedicated derived database and survive restarts; live sessions use connection-local TEMP tables that shadow the durable base for the same session and reveal it again when the live owner detaches. Both tables retain the exact inherited cut in numeric `seed_length`; reconstructed headers expose only `isSeeded`, while the cut participates in live fingerprints and persisted source revisions. Each search runs one serialized observation: list persistence snapshots, compare each source revision and Session format version with the indexed row, inspect only new or changed generations, verify that listing stayed stable across inspection, extract semantic documents, and commit the reconciliation in one transaction before running the query. A format change cannot reuse prior FTS rows or cursors even if an opaque revision collides. Repeated queries and unchanged reopens inspect nothing; switching stores or observing new, changed, deleted, or externally repaired sources reconciles on the next stable observation. Source or transaction failure commits nothing and the next search retries. ### Schema ownership diff --git a/packages/session-query/session-query-sqlite/README.zh.md b/packages/session-query/session-query-sqlite/README.zh.md index f6ef419293..e32d6d08ec 100644 --- a/packages/session-query/session-query-sqlite/README.zh.md +++ b/packages/session-query/session-query-sqlite/README.zh.md @@ -9,7 +9,7 @@ kind: "package-reference" ## 概述 -`dsh-session-query-sqlite` 用 SQLite FTS5 索引搜索会话历史，返回按会话分组或会话内排序、游标分页的结果。与 `dsh-session-query` 一起挂载，即可同时获得全文搜索与完整查询表面——精确读取、过滤与追踪。实时会话从内存索引，持久化会话从专用派生索引数据库索引，因此结果始终反映最新状态，且不触碰会话持久化存储。搜索是可选能力，已发布组合默认关闭：`openAt` 决定索引在启动时、首次搜索时打开，还是永不打开。设置与用法在前；实现内部细节放在下方可折叠的开发者章节中。 +`dsh-session-query-sqlite` 用 SQLite FTS5 索引搜索会话历史，返回按会话分组或会话内排序、游标分页的结果。与 `dsh-session-query` 一起挂载，即可同时获得全文搜索与完整查询表面——精确读取、过滤与追踪。实时会话从内存索引，持久化会话从专用派生索引数据库索引，因此结果反映最新稳定状态。检查受支持的历史 Session 时，持久化服务可能发布其当前 generation；可丢弃的 FTS 数据库仍与该权威存储分离。搜索是可选能力，已发布组合默认关闭：`openAt` 决定索引在启动时、首次搜索时打开，还是永不打开。设置与用法在前；实现内部细节放在下方可折叠的开发者章节中。 ## 目录 @@ -83,7 +83,7 @@ kind: "package-reference" 本后端建立在一个分离与三项承诺之上： -- **派生索引，绝不动源存储。** FTS 行存放在专用可丢弃数据库中；这里的代码从不打开 session-persistence 数据库。 +- **派生索引，不是源存储。** FTS 行存放在专用可丢弃数据库中；本包从不直接打开持久化产物，但其继承的检查可能要求持久化服务迁移受支持的历史 Session。 - **实时优先的观察。** 一个串行化状态机比较持久化快照修订，只检查新增或已更改日志，并在一个事务中对账，因此搜索反映最新的稳定状态。 - **世代绑定的游标。** 每次语料库变化都会递增世代；游标携带其创建时的世代，宁可陈旧失败也不返回偏移后的页面。 - **字面短语即数据。** 调用方查询文本被引成一个 FTS5 短语，查询语法保持惰性；保留高亮标记在索引前从文档中剥离。 @@ -101,7 +101,7 @@ kind: "package-reference" ### 索引生命周期 -持久化 FTS 行存放在专用派生数据库中并跨重启保留；实时会话使用连接本地 TEMP 表，遮蔽同一会话的持久化基库，并在实时所有者脱离后再次显示基库。两类表都在数字 `seed_length` 中保留精确继承切点；重建的 header 只公开 `isSeeded`，而切点参与实时 fingerprint 与持久来源修订。每次搜索执行一次串行化观察：列出持久化快照、把逐会话修订与已索引行比较、只检查新增或已更改日志、提取语义文档，并在运行查询前于一个事务中提交对账。重复查询与不变的重新打开不会检查任何内容；切换存储或观察到新增、已更改、已删除或经外部修复的来源时，会在下次稳定观察时对账。来源或事务失败不提交任何内容，下一次搜索重试。 +持久化 FTS 行存放在专用派生数据库中并跨重启保留；实时会话使用连接本地 TEMP 表，遮蔽同一会话的持久化基库，并在实时所有者脱离后再次显示基库。两类表都在数字 `seed_length` 中保留精确继承切点；重建的 header 只公开 `isSeeded`，而切点参与实时 fingerprint 与持久来源修订。每次搜索执行一次串行化观察：列出持久化快照、把每个来源修订和会话格式版本与已索引行比较、只检查新增或已更改的代际、验证检查期间列表保持稳定、提取语义文档，并在运行查询前于一个事务中提交对账。即使不透明修订发生碰撞，格式变化也不能复用先前 FTS 行或游标。重复查询与不变的重新打开不会检查任何内容；切换存储或观察到新增、已更改、已删除或经外部修复的来源时，会在下次稳定观察时对账。来源或事务失败不提交任何内容，下一次搜索重试。 ### Schema 归属 diff --git a/packages/session-query/session-query-sqlite/src/index.ts b/packages/session-query/session-query-sqlite/src/index.ts index 4722557a65..7a66195062 100644 --- a/packages/session-query/session-query-sqlite/src/index.ts +++ b/packages/session-query/session-query-sqlite/src/index.ts @@ -8,7 +8,7 @@ import { createHash, randomUUID } from 'node:crypto' import type { DatabaseSync } from 'node:sqlite' import { Context, Service, type Fiber } from '@deepseek-ai/cordis' import z from '@deepseek-ai/schemastery' -import { SessionSeq } from '@deepseek-ai/dsh-session' +import { SESSION_FORMAT_VERSION, SessionSeq } from '@deepseek-ai/dsh-session' import type { Session, SessionEvent, @@ -157,6 +157,7 @@ interface Observation { interface IndexedPersistedRow { id: string + version: number revision: string generation: number } @@ -404,7 +405,7 @@ export class SqliteSessionQueryEngine extends SessionQueryEngine { assertNotAborted(signal) const db = this._requireDb() const persistedRows = db.prepare( - 'SELECT id, revision, generation FROM persisted_sessions', + 'SELECT id, version, revision, generation FROM persisted_sessions', ).all() as unknown as IndexedPersistedRow[] const liveRows = db.prepare( 'SELECT id, fingerprint, persisted, generation FROM temp.live_sessions', @@ -506,11 +507,14 @@ export class SqliteSessionQueryEngine extends SessionQueryEngine { assertNotAborted(signal) persisted = materializePersistenceSnapshots(before) for (const entry of persisted.values()) { - if (canReuseIndexed && indexed.get(entry.header.id)?.revision === entry.revision) continue - // Skip work already shadowed by a live owner. `inspect()` is - // non-mutating, so an owner attaching after this check cannot cause - // crash-repair side effects; the live-membership retry below makes - // the returned observation live-preferred. + const current = indexed.get(entry.header.id) + if (canReuseIndexed + && current?.revision === entry.revision + && current.version === entry.header.version) continue + // Skip work already shadowed by a live owner. An owner attaching + // after this check shares persistence's per-id chain, so historical + // publication completes before adoption; the live-membership retry + // below then discards this cold observation in favor of live state. if (initiallyLive.has(entry.header.id) || this.ctx.sessions.get(entry.header.id) !== undefined) continue assertNotAborted(signal) const loaded = await persistence.inspect(entry.header.id, signal) @@ -898,6 +902,7 @@ function materializePersistenceSnapshots( if (typeof snapshot.revision !== 'string') { throw new Error('persistence snapshot revision must be a string') } + if (snapshot.status !== 'current' && snapshot.status !== 'migration-required') continue const header = structuredClone(snapshot.header) if (result.has(header.id)) { throw new Error(`persistence listed duplicate session "${header.id}"`) @@ -935,8 +940,7 @@ function sameSessionIds( } function sameHeader(a: SessionHeader, b: SessionHeader): boolean { - return a.version === b.version - && a.id === b.id + return a.id === b.id && a.createdAt === b.createdAt && a.cwd === b.cwd && a.parentSession === b.parentSession @@ -947,7 +951,7 @@ function sameHeader(a: SessionHeader, b: SessionHeader): boolean { function rowHeader(row: SessionHeaderRow): SessionHeader { return { - version: row.version, + version: SESSION_FORMAT_VERSION, id: row.session_id as SessionId, createdAt: row.created_at, ...row.cwd === null ? {} : { cwd: row.cwd }, diff --git a/packages/session-query/session-query-sqlite/tests/sqlite.spec.ts b/packages/session-query/session-query-sqlite/tests/sqlite.spec.ts index e09d74ba12..adabb8f9ca 100644 --- a/packages/session-query/session-query-sqlite/tests/sqlite.spec.ts +++ b/packages/session-query/session-query-sqlite/tests/sqlite.spec.ts @@ -2,7 +2,7 @@ import { createAssistantMessage, createUserMessage } from '@deepseek-ai/dsh-llm' import { afterEach, describe, expect, it, vi } from 'vitest' import { Context, type Fiber } from '@deepseek-ai/cordis' import { DatabaseSync } from 'node:sqlite' -import { chmod, mkdtemp, rm, stat, writeFile } from 'node:fs/promises' +import { chmod, mkdir, mkdtemp, readFile, readdir, rm, stat, writeFile } from 'node:fs/promises' import { tmpdir } from 'node:os' import { dirname, join } from 'node:path' import SessionStore, { @@ -14,8 +14,14 @@ import SessionStore, { import SessionProjectionRegistry from '@deepseek-ai/dsh-session-projection' import type { SessionEvent, SessionHeader, SessionId as SessionIdType } from '@deepseek-ai/dsh-session' import SessionPersistence, { SessionPersistenceRevision } from '@deepseek-ai/dsh-session-persistence' -import type { SessionEventSuffix, SessionInspection, SessionPersistenceSnapshot } from '@deepseek-ai/dsh-session-persistence' +import type { + SessionEventSuffix, + SessionInspection, + SessionPersistenceListing, + SessionPersistenceSnapshot, +} from '@deepseek-ai/dsh-session-persistence' import JsonlSessionPersistence from '@deepseek-ai/dsh-session-persistence-jsonl' +import { generationLogFilename } from '@deepseek-ai/dsh-session-persistence-jsonl/src/format.ts' import SqliteSessionQueryEngine, { SESSION_QUERY_SQLITE_SCHEMA_VERSION, } from '@deepseek-ai/dsh-session-query-sqlite' @@ -192,11 +198,16 @@ class TestPersistence extends SessionPersistence { return { ...whole, fromSeq, events: whole.events.filter(event => event.seq >= fromSeq) } } - async list(): Promise { + async list(): Promise { TestPersistence.listStarted?.() await TestPersistence.listGate if (TestPersistence.failure !== undefined) throw TestPersistence.failure - return [...TestPersistence.entries.values()].map(entry => structuredClone(entry.meta)) + return [...TestPersistence.entries.values()].map(entry => ({ + status: 'current' as const, + header: structuredClone(entry.meta), + storedVersion: entry.meta.version, + targetVersion: SESSION_FORMAT_VERSION, + })) } @@ -207,7 +218,10 @@ class TestPersistence extends SessionPersistence { if (TestPersistence.failure !== undefined) throw TestPersistence.failure const snapshots = TestPersistence.snapshotOverride?.() ?? [...TestPersistence.entries.values()].map(entry => ({ + status: 'current' as const, header: structuredClone(entry.meta), + storedVersion: entry.meta.version, + targetVersion: SESSION_FORMAT_VERSION, revision: SessionPersistenceRevision(`test:${TestPersistence.revisions.get(entry.meta.id)}`), })) await TestPersistence.snapshotEffect?.(signal) @@ -1047,6 +1061,52 @@ describe('SQLite reconciliation and source lifecycle', () => { await replacement.dispose() }) + it('retries a format-changing revision and expires its prior event cursor', async () => { + const durable = header('format-migration') + const oldEvents = [ + ...messageEvents('old needle one', 1), + { ...messageEvents('old needle two', 2)[0]!, seq: SessionSeq(1) }, + ] + TestPersistence.reset([{ meta: durable, events: oldEvents }]) + const ctx = await liveContext({ path: ':memory:', defaultLimit: 1, maxLimit: 2 }) + await ctx.plugin(TestPersistence) + const first = await ctx.sessionQuery.searchEvents({ + sessionId: durable.id, + query: 'old needle', + limit: 1, + }) + expect(first.nextCursor).toEqual(expect.any(String)) + const cursor = first.nextCursor + if (cursor === undefined) throw new Error('fixture did not return a cursor') + const db = (ctx.sessionQuery as unknown as { _db: DatabaseSync })._db + db.prepare('UPDATE persisted_sessions SET version = ? WHERE id = ?') + .run(SESSION_FORMAT_VERSION - 1, durable.id) + TestPersistence.inspectEffect = (observed) => { + observed.events = [ + ...messageEvents('new needle one', 3), + { ...messageEvents('new needle two', 4)[0]!, seq: SessionSeq(1) }, + ] + TestPersistence.revisions.set(durable.id, ++TestPersistence.nextRevision) + } + + await expect(ctx.sessionQuery.searchEvents({ + sessionId: durable.id, + query: 'old needle', + limit: 1, + cursor, + })).rejects.toThrow(expectCode('SESSION_QUERY_STALE_CURSOR')) + expect(TestPersistence.inspections.get(durable.id)).toBe(3) + expect(TestPersistence.snapshotSignals).toHaveLength(6) + await expect(ctx.sessionQuery.searchEvents({ + sessionId: durable.id, + query: 'new needle', + limit: 2, + })).resolves.toMatchObject({ + session: { version: SESSION_FORMAT_VERSION }, + items: [{}, {}], + }) + }) + it('retries when a successful observation belongs to a source unmounted during listing', async () => { const durable = header('successful-unmount') TestPersistence.reset([{ meta: durable, events: messageEvents('durable needle') }]) @@ -1131,16 +1191,45 @@ describe('SQLite reconciliation and source lifecycle', () => { TestPersistence.snapshotOverride = () => 'not-an-array' as never await expect(ctx.sessionQuery.searchSessions({ query: 'needle' })) .rejects.toThrow(expectCode('SESSION_QUERY_PERSISTENCE_FAILED')) - TestPersistence.snapshotOverride = () => [{ header: durable, revision: 1 as never }] + TestPersistence.snapshotOverride = () => [{ + status: 'current', + header: durable, + storedVersion: durable.version, + targetVersion: SESSION_FORMAT_VERSION, + revision: 1 as never, + }] await expect(ctx.sessionQuery.searchSessions({ query: 'needle' })) .rejects.toThrow(expectCode('SESSION_QUERY_PERSISTENCE_FAILED')) TestPersistence.snapshotOverride = () => [ - { header: durable, revision: SessionPersistenceRevision('duplicate:1') }, - { header: durable, revision: SessionPersistenceRevision('duplicate:2') }, + { + status: 'current', + header: durable, + storedVersion: durable.version, + targetVersion: SESSION_FORMAT_VERSION, + revision: SessionPersistenceRevision('duplicate:1'), + }, + { + status: 'current', + header: durable, + storedVersion: durable.version, + targetVersion: SESSION_FORMAT_VERSION, + revision: SessionPersistenceRevision('duplicate:2'), + }, ] await expect(ctx.sessionQuery.searchSessions({ query: 'needle' })) .rejects.toThrow(expectCode('SESSION_QUERY_PERSISTENCE_FAILED')) + TestPersistence.snapshotOverride = () => [{ + status: 'unsupported', + storedVersion: SESSION_FORMAT_VERSION + 1, + targetVersion: SESSION_FORMAT_VERSION, + location: { kind: 'test', path: '/unsupported/session.jsonl' }, + reason: 'future format', + revision: SessionPersistenceRevision('unsupported:1'), + }] + await expect(ctx.sessionQuery.searchSessions({ query: 'needle' })) + .resolves.toEqual({ items: [] }) + TestPersistence.snapshotOverride = undefined const typed = new SessionQueryError('typed persistence failure', 'SESSION_QUERY_PERSISTENCE_FAILED') TestPersistence.failure = typed @@ -1866,6 +1955,124 @@ describe('SQLite schema, cancellation, and real persistence integration', () => await persistence.dispose() }) + it('migrates a listed v0 JSONL session through search before preparing and forking it', async () => { + const persistenceRoot = await temporaryPath('canonical-v0') + const searchPath = await temporaryPath('derived-v0.db') + const ctx = new Context() + try { + await ctx.plugin(SessionStore) + await ctx.plugin(SessionProjectionRegistry) + await ctx.plugin(JsonlSessionPersistence, { + root: persistenceRoot, + compression: 'none', + }) + const meta = header('real-v0', 10, { cwd: '/work', delegationDepth: 0 }) + const currentLocation = ctx.sessionPersistence.locate(meta) + if (currentLocation === undefined) throw new Error('JSONL backend did not locate the v0 fixture') + const v0Path = join(dirname(currentLocation.path), generationLogFilename(0, 'none')) + const v0Location = { kind: 'jsonl', path: v0Path } + const source = [ + { + type: 'session', version: 0, id: meta.id, createdAt: meta.createdAt, + cwd: meta.cwd, delegationDepth: 0, + }, + { type: 'turn/start', seq: 0, time: 11, data: { turn: 1 } }, + { + type: 'user/message', seq: 1, time: 12, surfaceOp: 'append', + data: createUserMessage({ + content: [{ type: 'text', text: 'migration search needle' }], + source: { kind: 'user' }, + }), + }, + { type: 'turn/end', seq: 2, time: 13, data: { turn: 1, reason: { kind: 'completed' } } }, + ].map(record => JSON.stringify(record)).join('\n') + '\n' + await mkdir(dirname(v0Path), { recursive: true }) + await writeFile(v0Path, source, { flush: true }) + const v0Before = await stat(v0Path, { bigint: true }) + + const snapshots = await ctx.sessionPersistence.listSnapshots() + expect(snapshots).toHaveLength(1) + expect(snapshots[0]).toMatchObject({ + status: 'migration-required', + storedVersion: 0, + targetVersion: SESSION_FORMAT_VERSION, + header: { ...meta, version: SESSION_FORMAT_VERSION }, + location: v0Location, + }) + expect(typeof snapshots[0]?.revision).toBe('string') + expect(await readFile(v0Path, 'utf8')).toBe(source) + expect(await readdir(dirname(v0Path))).toEqual(['session.jsonl']) + + await ctx.plugin(SqliteSessionQueryEngine, { path: searchPath }) + expect(await readFile(v0Path, 'utf8')).toBe(source) + expect(await readdir(dirname(v0Path))).toEqual(['session.jsonl']) + + await expect(ctx.sessionQuery.searchEvents({ + sessionId: meta.id, + query: 'migration search needle', + })).resolves.toMatchObject({ + session: { ...meta, version: SESSION_FORMAT_VERSION }, + items: [{ sessionId: meta.id, seq: 1, type: 'user/message' }], + }) + expect(await readFile(v0Path, 'utf8')).toBe(source) + const v0AfterMigration = await stat(v0Path, { bigint: true }) + const v1AfterMigration = await stat(currentLocation.path, { bigint: true }) + expect({ dev: v0AfterMigration.dev, ino: v0AfterMigration.ino }) + .toEqual({ dev: v0Before.dev, ino: v0Before.ino }) + expect({ dev: v1AfterMigration.dev, ino: v1AfterMigration.ino }) + .not.toEqual({ dev: v0Before.dev, ino: v0Before.ino }) + const current = await readFile(currentLocation.path, 'utf8') + expect((JSON.parse(current.split('\n')[0] as string) as { version: number }).version) + .toBe(SESSION_FORMAT_VERSION) + expect((await readdir(dirname(v0Path))).sort()).toEqual([ + 'session.jsonl', + 'session.v1.jsonl', + ]) + + await expect(ctx.sessionPersistence.readRaw(meta.id)).resolves.toMatchObject({ + meta: { ...meta, version: SESSION_FORMAT_VERSION }, + filename: 'session.v1.jsonl', + content: current, + }) + expect(await readFile(currentLocation.path, 'utf8')).toBe(current) + expect(await readFile(v0Path, 'utf8')).toBe(source) + + const preparation = await ctx.sessionPersistence.prepare(meta.id) + const resumed = preparation.session + const detach = ctx.sessions.enter(resumed) + try { + ctx.sessions.announce(resumed) + const child = ctx.sessions.fork(resumed, SessionSeq(2), SessionId('real-v0-child')) + expect(resumed.header.version).toBe(SESSION_FORMAT_VERSION) + expect(child.header).toMatchObject({ + version: SESSION_FORMAT_VERSION, + parentSession: meta.id, + isSeeded: true, + }) + expect(child.snapshotEvents().map(event => event.type)).toEqual([ + 'turn/start', + 'user/message', + 'turn/end', + 'session/end-seed', + ]) + expect(child.snapshotEvents()[1]).toMatchObject({ + type: 'user/message', + data: { content: [{ type: 'text', text: 'migration search needle' }] }, + }) + } finally { + detach() + preparation[Symbol.dispose]() + } + expect(await readFile(currentLocation.path, 'utf8')).toBe(current) + expect(await readFile(v0Path, 'utf8')).toBe(source) + const v0AfterFork = await stat(v0Path, { bigint: true }) + expect({ dev: v0AfterFork.dev, ino: v0AfterFork.ino }) + .toEqual({ dev: v0Before.dev, ino: v0Before.ino }) + } finally { + await ctx.fiber.dispose() + } + }) + it('reconciles colliding local revisions when a derived index reopens against another JSONL store', async () => { const persistenceRootA = await temporaryPath('canonical-a') const persistenceRootB = await temporaryPath('canonical-b') diff --git a/packages/session-query/session-query/README.i18n.yaml b/packages/session-query/session-query/README.i18n.yaml index c6092ee220..058905fdc3 100644 --- a/packages/session-query/session-query/README.i18n.yaml +++ b/packages/session-query/session-query/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/session-query/session-query/README.md -README.md: 549ec7344d53a09e82568be4d1a21778367c38cc -README.zh.md: d2161b9528306ead31d709c4d4f6fab20feec8fd +README.md: 1e138096815d33dc832c5f469d26580943d4b37e +README.zh.md: 48185acdd6d9b7816dbe3faf4120cd12dc7ebc9d diff --git a/packages/session-query/session-query/README.md b/packages/session-query/session-query/README.md index 549ec7344d..1e13809681 100644 --- a/packages/session-query/session-query/README.md +++ b/packages/session-query/session-query/README.md @@ -101,7 +101,7 @@ The decision history lives in the [unified service decision](../../../.agents/no ### Corpus resolution -`SessionCorpus` binds optional `ctx.sessionPersistence` through a fiber and resolves each read live-first: a known live target is snapshotted without consulting persistence; otherwise the session is listed, inspected non-mutatingly, and re-checked for a live attachment before cloning. Header compatibility is asserted between listed and loaded observations. Batch title reads run one metadata listing and bounded-concurrency inspections, isolating per-session failures while cancellation rejects the whole batch. +`SessionCorpus` binds optional `ctx.sessionPersistence` through a fiber and resolves each read live-first: a known live target is snapshotted without consulting persistence; otherwise the session is listed, inspected, and re-checked for a live attachment before cloning. Already-current inspection keeps recovery in memory; a supported historical body read may first publish migration and repair. Header compatibility is asserted between listed and loaded observations. Batch title reads run one metadata listing and bounded-concurrency inspections, isolating per-session failures while cancellation rejects the whole observer batch; a shared persistence load or migration already admitted for another observer may continue. ### Reads and traces diff --git a/packages/session-query/session-query/README.zh.md b/packages/session-query/session-query/README.zh.md index d2161b9528..48185acdd6 100644 --- a/packages/session-query/session-query/README.zh.md +++ b/packages/session-query/session-query/README.zh.md @@ -101,7 +101,7 @@ kind: "package-reference" ### 语料库解析 -`SessionCorpus` 通过 fiber 绑定可选的 `ctx.sessionPersistence`，并实时优先解析每次读取：已知实时目标直接快照，不查询持久化；否则先列出会话，再以不修改日志的方式检查，并在克隆前重新检查是否出现实时挂载。列表与加载观察之间会断言 header 兼容性。批量标题读取执行一次元数据列表与有界并发检查，把逐会话失败隔离，而取消会拒绝整个批次。 +`SessionCorpus` 通过 fiber 绑定可选的 `ctx.sessionPersistence`，并实时优先解析每次读取：已知实时目标直接快照，不查询持久化；否则先列出会话、执行检查，并在克隆前重新检查是否出现实时挂载。已经是当前格式的检查只在内存中保留恢复；受支持的历史正文读取可能先发布迁移与修复。列表与加载观察之间会断言 header 兼容性。批量标题读取执行一次元数据列表与有界并发检查，把逐会话失败隔离，而取消会拒绝整个观测批次；已经为其他观察方接纳的共享持久化加载或迁移仍可能继续。 ### 读取与追踪 diff --git a/packages/session-query/session-query/src/corpus.ts b/packages/session-query/session-query/src/corpus.ts index a9746b1793..b6514bac7e 100644 --- a/packages/session-query/session-query/src/corpus.ts +++ b/packages/session-query/session-query/src/corpus.ts @@ -9,6 +9,7 @@ import type { SessionLogOffset, } from '@deepseek-ai/dsh-session' import type SessionPersistence from '@deepseek-ai/dsh-session-persistence' +import type { ReadableSessionPersistenceListing } from '@deepseek-ai/dsh-session-persistence' import type { SessionRecord } from './types.ts' import { SessionQueryError } from './config.ts' import { assertSessionHeadersCompatible } from './sources.ts' @@ -71,14 +72,20 @@ export class SessionCorpus { const persisted = persistence === undefined ? [] : await listPersisted(persistence, signal) signal?.throwIfAborted() const records = new Map() - for (const header of persisted) { - records.set(header.id, { header: structuredClone(header), live: false, persisted: true }) + for (const listing of persisted) { + records.set(listing.header.id, { + header: structuredClone(listing.header), + ...listing.location === undefined ? {} : { location: structuredClone(listing.location) }, + live: false, + persisted: true, + }) } for (const session of this._ctx.sessions.list()) { const durable = records.get(session.id) if (durable !== undefined) assertSessionHeadersCompatible(session.header, durable.header) records.set(session.id, { header: structuredClone(session.header), + ...durable?.location === undefined ? {} : { location: structuredClone(durable.location) }, live: true, persisted: durable !== undefined, }) @@ -105,7 +112,8 @@ export class SessionCorpus { } const persistence = this._persistence if (persistence === undefined) throw notFound(sessionId) - const listed = (await listPersisted(persistence, signal)).find(header => header.id === sessionId) + const listed = (await listPersisted(persistence, signal)) + .find(listing => listing.header.id === sessionId) signal?.throwIfAborted() if (listed === undefined) throw notFound(sessionId) const loaded = await inspectPersisted(persistence, sessionId, signal) @@ -116,7 +124,7 @@ export class SessionCorpus { signal?.throwIfAborted() return snapshot } - assertSessionHeadersCompatible(loaded.meta, listed) + assertSessionHeadersCompatible(loaded.meta, listed.header) const snapshot = { header: structuredClone(loaded.meta), inheritedEventCount: loaded.inheritedEventCount, @@ -163,7 +171,7 @@ export class SessionCorpus { return orderedResults(ids, resolved) } - let persisted: SessionHeader[] + let persisted: ReadableSessionPersistenceListing[] try { persisted = await listPersisted(persistence, signal) signal?.throwIfAborted() @@ -174,7 +182,7 @@ export class SessionCorpus { } return orderedResults(ids, resolved) } - const persistedById = new Map(persisted.map(header => [header.id, header])) + const persistedById = new Map(persisted.map(listing => [listing.header.id, listing])) const resolvePersisted = async (sessionId: SessionId): Promise => { const listed = persistedById.get(sessionId) if (listed === undefined) { @@ -193,7 +201,7 @@ export class SessionCorpus { resolved.set(sessionId, projectSource(sessionId, sourceLive(attached), project, signal)) return } - assertSessionHeadersCompatible(loaded.meta, listed) + assertSessionHeadersCompatible(loaded.meta, listed.header) resolved.set(sessionId, projectSource(sessionId, { header: loaded.meta, inheritedEventCount: loaded.inheritedEventCount, @@ -268,9 +276,12 @@ function orderedResults( async function listPersisted( persistence: SessionPersistence, signal?: AbortSignal, -): Promise { +): Promise { try { - return await persistence.list(signal) + return (await persistence.list(signal)) + .flatMap(listing => listing.status === 'current' || listing.status === 'migration-required' + ? [listing] + : []) } catch (error: unknown) { if (signal?.aborted) signal.throwIfAborted() throw new SessionQueryError( diff --git a/packages/session-query/session-query/src/sources.ts b/packages/session-query/session-query/src/sources.ts index efcd83c349..9c73d3d474 100644 --- a/packages/session-query/session-query/src/sources.ts +++ b/packages/session-query/session-query/src/sources.ts @@ -10,8 +10,7 @@ import { SessionQueryError } from './config.ts' */ export function assertSessionHeadersCompatible(a: SessionHeader, b: SessionHeader): void { if ( - a.version !== b.version - || a.id !== b.id + a.id !== b.id || a.createdAt !== b.createdAt || a.cwd !== b.cwd || a.parentSession !== b.parentSession diff --git a/packages/session-query/session-query/src/types.ts b/packages/session-query/session-query/src/types.ts index 01c196aa1f..c8f1563e2a 100644 --- a/packages/session-query/session-query/src/types.ts +++ b/packages/session-query/session-query/src/types.ts @@ -16,6 +16,7 @@ import type { SurfaceEvent, } from '@deepseek-ai/dsh-session' import type { SessionTitleSnapshot } from '@deepseek-ai/dsh-session-title' +import type { SessionLocation } from '@deepseek-ai/dsh-session-persistence' import type { SessionSearchCursor } from './cursor.ts' export type { SessionSearchCursor } from './cursor.ts' @@ -27,6 +28,8 @@ export type SessionEventSurface = 'current' | 'shadowed' | 'log-only' export interface SessionRecord { /** Cloned session header selected from the live-preferred corpus. */ header: SessionHeader + /** Exact listed artifact location, when persistence exposes one. */ + location?: SessionLocation /** Whether the id currently exists in `ctx.sessions`. */ live: boolean /** Whether the active persistence backend currently materializes the id. */ diff --git a/packages/session-query/session-query/tests/observation.spec.ts b/packages/session-query/session-query/tests/observation.spec.ts index ef2460e7a2..ce1647b015 100644 --- a/packages/session-query/session-query/tests/observation.spec.ts +++ b/packages/session-query/session-query/tests/observation.spec.ts @@ -1,5 +1,7 @@ import { Context } from '@deepseek-ai/cordis' -import SessionStore, { Session, SessionId, SessionLogOffset, SessionSeq } from '@deepseek-ai/dsh-session' +import SessionStore, { + SESSION_FORMAT_VERSION, Session, SessionId, SessionLogOffset, SessionSeq, +} from '@deepseek-ai/dsh-session' import type { SessionHeader } from '@deepseek-ai/dsh-session' import { SessionPersistenceRevision } from '@deepseek-ai/dsh-session-persistence' import type { BorrowedSessionSource } from '@deepseek-ai/dsh-session-persistence' @@ -8,7 +10,13 @@ import { describe, expect, it, vi } from 'vitest' import { SessionObservationReader } from '../src/observation.ts' function header(id: string): SessionHeader { - return { version: 0, id: SessionId(id), createdAt: 1, cwd: '/workspace', isSeeded: false } + return { + version: SESSION_FORMAT_VERSION, + id: SessionId(id), + createdAt: 1, + cwd: '/workspace', + isSeeded: false, + } } function preparedSource( diff --git a/packages/session-query/session-query/tests/session-query.spec.ts b/packages/session-query/session-query/tests/session-query.spec.ts index f011df923d..c63afe3089 100644 --- a/packages/session-query/session-query/tests/session-query.spec.ts +++ b/packages/session-query/session-query/tests/session-query.spec.ts @@ -10,7 +10,13 @@ import SessionStore, { import SessionProjectionRegistry from '@deepseek-ai/dsh-session-projection' import type { SessionEvent, SessionHeader, SessionId as SessionIdType } from '@deepseek-ai/dsh-session' import SessionPersistence, { SessionPersistenceCorruptionError, SessionPersistenceRevision } from '@deepseek-ai/dsh-session-persistence' -import type { SessionEventSuffix, SessionInspection } from '@deepseek-ai/dsh-session-persistence' +import type { + CurrentSessionPersistenceListing, + SessionEventSuffix, + SessionInspection, + SessionPersistenceListing, + SessionPersistenceSnapshot, +} from '@deepseek-ai/dsh-session-persistence' import SessionQueryEngine, { SESSION_QUERY_DEFAULT_PERSISTED_INSPECT_CONCURRENCY, type SessionEventSurface, @@ -25,6 +31,15 @@ function header(id: string, createdAt = 1, extra: Partial = {}): return { version: SESSION_FORMAT_VERSION, id: SessionId(id), createdAt, isSeeded: false, ...extra } } +function listing(header: SessionHeader): CurrentSessionPersistenceListing { + return { + status: 'current', + header, + storedVersion: SESSION_FORMAT_VERSION, + targetVersion: SESSION_FORMAT_VERSION, + } +} + function eventLog(text = 'hello'): SessionEvent<'user/message'>[] { return [{ type: 'user/message', @@ -42,7 +57,7 @@ class TestPersistence extends SessionPersistence { static entries = new Map() static listFailure: unknown - static listOverride: ((signal?: AbortSignal) => Promise) | undefined + static listOverride: ((signal?: AbortSignal) => Promise) | undefined static inspectFailure: unknown static inspectEffect: (() => void) | undefined static inspectOverride: (( @@ -123,20 +138,24 @@ class TestPersistence extends SessionPersistence { return { ...whole, fromSeq, events: whole.events.filter(event => event.seq >= fromSeq) } } - list(signal?: AbortSignal): Promise { + list(signal?: AbortSignal): Promise { TestPersistence.listCalls += 1 TestPersistence.listSignals.push(signal) if (TestPersistence.listOverride !== undefined) return TestPersistence.listOverride(signal) if (TestPersistence.listFailure !== undefined) return rejectUnknown(TestPersistence.listFailure) - const headers = [...TestPersistence.entries.values()].map(entry => structuredClone(entry.meta)) + const headers = [...TestPersistence.entries.values()] + .map(entry => listing(structuredClone(entry.meta))) TestPersistence.afterList?.() return Promise.resolve(headers) } - async listSnapshots() { + async listSnapshots(): Promise { return [...TestPersistence.entries.values()].map(entry => ({ + status: 'current', header: structuredClone(entry.meta), + storedVersion: SESSION_FORMAT_VERSION, + targetVersion: SESSION_FORMAT_VERSION, revision: SessionPersistenceRevision(`events:${entry.events.length}`), })) } @@ -263,7 +282,7 @@ describe.each(cancellableSessionListings)('$name cancellation', ({ run }) => { const controller = new AbortController() const reason = new Error('session listing cancelled before persistence returned') const started = Promise.withResolvers() - const listing = Promise.withResolvers() + const listing = Promise.withResolvers() TestPersistence.listOverride = (_signal) => { started.resolve(undefined) return listing.promise @@ -368,7 +387,7 @@ describe.each(cancellableExactReads)('$name cancellation', ({ inspects, run }) = started.resolve(undefined) await release.promise active = false - return [structuredClone(persisted)] + return [listing(structuredClone(persisted))] } } @@ -1133,6 +1152,21 @@ describe('session-query exact reads', () => { ]) }) + it('omits unreadable persistence descriptors from the logical corpus', async () => { + TestPersistence.reset() + const ctx = await liveContext() + await ctx.plugin(TestPersistence) + TestPersistence.listOverride = async () => [{ + status: 'unsupported', + storedVersion: SESSION_FORMAT_VERSION + 1, + targetVersion: SESSION_FORMAT_VERSION, + location: { kind: 'test', path: '/unsupported/session.jsonl' }, + reason: 'future format', + }] + + await expect(ctx.sessionQuery.listSessions()).resolves.toEqual([]) + }) + it('keeps known live reads independent from persistence health', async () => { TestPersistence.reset() const ctx = await liveContext() diff --git a/packages/session-query/session-query/tests/tracing.spec.ts b/packages/session-query/session-query/tests/tracing.spec.ts index e40af94f68..8769b9db39 100644 --- a/packages/session-query/session-query/tests/tracing.spec.ts +++ b/packages/session-query/session-query/tests/tracing.spec.ts @@ -10,7 +10,12 @@ import SessionStore, { import SessionProjectionRegistry from '@deepseek-ai/dsh-session-projection' import type { Session, SessionEvent, SessionHeader, SessionId as SessionIdType } from '@deepseek-ai/dsh-session' import SessionPersistence from '@deepseek-ai/dsh-session-persistence' -import type { SessionEventSuffix, SessionInspection } from '@deepseek-ai/dsh-session-persistence' +import type { + CurrentSessionPersistenceListing, + SessionEventSuffix, + SessionInspection, + SessionPersistenceListing, +} from '@deepseek-ai/dsh-session-persistence' import { type SessionQueryErrorCode } from '@deepseek-ai/dsh-session-query' import { TestSessionQueryEngine } from './test-service.ts' @@ -25,6 +30,15 @@ function header(id: string, createdAt = 1, extra: Partial = {}): return { version: SESSION_FORMAT_VERSION, id: SessionId(id), createdAt, isSeeded: false, ...extra } } +function listing(header: SessionHeader): CurrentSessionPersistenceListing { + return { + status: 'current', + header, + storedVersion: SESSION_FORMAT_VERSION, + targetVersion: SESSION_FORMAT_VERSION, + } +} + function appendEvent(seq: SessionSeq, sources?: number[]): SessionEvent { return { type: 'user/message', @@ -97,10 +111,11 @@ class TracePersistence extends SessionPersistence { return { ...whole, fromSeq, events: whole.events.filter(event => event.seq >= fromSeq) } } - list(): Promise { + list(): Promise { TracePersistence.listCalls += 1 if (TracePersistence.listFailure !== undefined) return Promise.reject(TracePersistence.listFailure) - const result = [...TracePersistence.entries.values()].map(entry => structuredClone(entry.meta)) + const result = [...TracePersistence.entries.values()] + .map(entry => listing(structuredClone(entry.meta))) TracePersistence.afterList?.() return Promise.resolve(result) } diff --git a/packages/session/README.i18n.yaml b/packages/session/README.i18n.yaml index ec8aaf2ef0..6925184d6a 100644 --- a/packages/session/README.i18n.yaml +++ b/packages/session/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/session/README.md -README.md: b47bd7e6384919af4add2746417b232f6873aaf0 -README.zh.md: fa5bc08fc04d32de15f9fe357e7b18b23bb084d3 +README.md: b73c1519bc03271fbcc66f2233dd5e3a91dd1902 +README.zh.md: d74b36f4c29480c9331c7f134b2c839b98bc2e4e diff --git a/packages/session/README.md b/packages/session/README.md index b47bd7e638..b73c1519bc 100644 --- a/packages/session/README.md +++ b/packages/session/README.md @@ -9,7 +9,7 @@ English | [中文](README.zh.md) ## Summary -The session group makes an agent's conversation durable and reusable outside the live loop: the persistence seam stores the event log and restores it on resume, the checkpoint policy keeps requests, tool side effects, and completed steps durable before the next action, projections serve whole log-derived values to client carriers, titles name each session from its content, and telemetry reports session activity outbound. Mount the shipped JSONL persistence provider first, then add the checkpoint policy and any projection, title, or telemetry packages the deployment needs. This page maps the group; every package README owns its contract, and `session-query/` is a sibling group whose read/tool surface consumes persistence independently. +The session group makes an agent's conversation durable and reusable outside the live loop: the static format chain restores released generations, the persistence seam stores the event log and restores it on resume, the checkpoint policy keeps requests, tool side effects, and completed steps durable before the next action, projections serve whole log-derived values to client carriers, titles name each session from its content, and telemetry reports session activity outbound. Mount the shipped JSONL persistence provider first, then add the checkpoint policy and any projection, title, or telemetry packages the deployment needs. This page maps the group; every package README owns its contract, and `session-query/` is a sibling group whose read/tool surface consumes persistence independently. ## Table of Contents @@ -28,8 +28,11 @@ The group splits into four families: durable storage (persistence seam, backends | Package | Role | ctx key | |---|---|---| +| [`session-format/`](session-format/README.md) | Pure adjacent-format chain and artifact validation library | library — no ctx key | +| [`session-format-v0-to-v1/`](session-format-v0-to-v1/README.md) | Frozen released-v0 decoder and identity migration into released v1 | library — no ctx key | +| [`session-format-catalog/`](session-format-catalog/README.md) | Generated static catalog of shipped adjacent migrations | library — no ctx key | | [`session-persistence/`](session-persistence/README.md) | Defines the durable session-storage service and the shared write coordination every backend composes | `ctx.sessionPersistence` | -| [`session-persistence-jsonl/`](session-persistence-jsonl/README.md) | Shipped backend: one append-only JSONL log per session, optionally Zstandard-compressed | registers on `ctx.sessionPersistence` | +| [`session-persistence-jsonl/`](session-persistence-jsonl/README.md) | Shipped backend: immutable canonical generation filenames per Session with exclusive successor publication, optionally Zstandard-compressed | registers on `ctx.sessionPersistence` | | [`session-checkpoint-policy/`](session-checkpoint-policy/README.md) | Makes model requests, top-level tool side effects, and completed steps durable before the next action | wraps `ctx.llm` and `ctx.tools` | | [`session-log-deepseek/`](session-log-deepseek/README.md) | Uploads the incremental canonical log as optional official DeepSeek request metadata | contributes `dsh_session_log` | diff --git a/packages/session/README.zh.md b/packages/session/README.zh.md index fa5bc08fc0..d74b36f4c2 100644 --- a/packages/session/README.zh.md +++ b/packages/session/README.zh.md @@ -9,7 +9,7 @@ kind: "package-group" ## 概述 -session 组让 agent（智能体）的对话在实时 loop 之外持久可复用：持久化 seam 存储事件日志并在恢复时还原，检查点策略让请求、工具副作用与已完成步骤在下一步动作前持久化，投影向客户端载体提供日志派生的完整值，标题根据会话内容为其命名，遥测则向外上报会话活动。先挂载随产品交付的 JSONL 持久化 provider，再按部署需要挂载检查点策略以及投影、标题或遥测包。本页是组的映射；每个包 README 负责各自的约定，`session-query/` 是同级独立组，其读取／工具接口独立消费持久化。 +session 组让 agent（智能体）的对话在实时 loop 之外持久可复用：静态格式链还原已发布 generation，持久化 seam 存储事件日志并在恢复时还原，检查点策略让请求、工具副作用与已完成步骤在下一步动作前持久化，投影向客户端载体提供日志派生的完整值，标题根据会话内容为其命名，遥测则向外上报会话活动。先挂载随产品交付的 JSONL 持久化提供方，再按部署需要挂载检查点策略以及投影、标题或遥测包。本页是组的映射；每个包 README 负责各自的约定，`session-query/` 是同级独立组，其读取／工具接口独立消费持久化。 ## 目录 @@ -28,8 +28,11 @@ session 组让 agent（智能体）的对话在实时 loop 之外持久可复用 | 包 | 职责 | ctx key | |---|---|---| +| [`session-format/`](session-format/README.zh.md) | 纯相邻格式链与产物校验库 | 库，不使用 ctx key | +| [`session-format-v0-to-v1/`](session-format-v0-to-v1/README.zh.md) | 冻结的 released-v0 解码器，以及到 released v1 的恒等迁移 | 库，不使用 ctx key | +| [`session-format-catalog/`](session-format-catalog/README.zh.md) | 已交付相邻迁移的生成式静态目录 | 库，不使用 ctx key | | [`session-persistence/`](session-persistence/README.zh.md) | 定义持久会话存储服务，以及每个后端组合的共享写入协调机制 | `ctx.sessionPersistence` | -| [`session-persistence-jsonl/`](session-persistence-jsonl/README.zh.md) | 随产品交付的后端：每会话一份仅追加 JSONL 日志，可选 Zstandard 压缩 | 注册到 `ctx.sessionPersistence` | +| [`session-persistence-jsonl/`](session-persistence-jsonl/README.zh.md) | 随产品交付的后端：逐 Session 使用不可变规范 generation 文件名并排他发布后继；可选 Zstandard 压缩 | 注册到 `ctx.sessionPersistence` | | [`session-checkpoint-policy/`](session-checkpoint-policy/README.zh.md) | 让模型请求、顶层工具副作用与已完成步骤在下一步动作前持久化 | 包装 `ctx.llm` 与 `ctx.tools` | | [`session-log-deepseek/`](session-log-deepseek/README.zh.md) | 把增量规范日志作为可选的官方 DeepSeek 请求元数据上传 | 贡献 `dsh_session_log` | diff --git a/packages/session/session-checkpoint-policy/tests/session-checkpoint-policy.spec.ts b/packages/session/session-checkpoint-policy/tests/session-checkpoint-policy.spec.ts index 670d410ee3..2695e7fb7d 100644 --- a/packages/session/session-checkpoint-policy/tests/session-checkpoint-policy.spec.ts +++ b/packages/session/session-checkpoint-policy/tests/session-checkpoint-policy.spec.ts @@ -5,7 +5,11 @@ import { agentEvents, type Agent } from '@deepseek-ai/dsh-agent' import LlmRuntime, { ToolCallId, type GenerateOptions, LlmAdapter, type StreamChunk } from '@deepseek-ai/dsh-llm' import SessionStore, { SessionId } from '@deepseek-ai/dsh-session' import type { SessionEvent, SessionHeader, SessionLogOffset } from '@deepseek-ai/dsh-session' -import SessionPersistence, { type SessionEventSuffix, type SessionInspection } from '@deepseek-ai/dsh-session-persistence' +import SessionPersistence, { + type SessionEventSuffix, + type SessionInspection, + type SessionPersistenceListing, +} from '@deepseek-ai/dsh-session-persistence' import SystemPrompt from '@deepseek-ai/dsh-system-prompt' import ToolRuntime, { TOOL_ABORTED_BEFORE_DISPATCH } from '@deepseek-ai/dsh-tools' import * as checkpointPolicy from '../src/index.ts' @@ -30,7 +34,7 @@ class TestPersistence extends SessionPersistence { readFrom(_id: SessionId, _fromSeq: SessionLogOffset): Promise { return Promise.reject(new Error('not used')) } - list(): Promise { return Promise.resolve([]) } + list(): Promise { return Promise.resolve([]) } listSnapshots(): Promise { return Promise.resolve([]) } } diff --git a/packages/session/session-format-catalog/README.i18n.yaml b/packages/session/session-format-catalog/README.i18n.yaml new file mode 100644 index 0000000000..034e237802 --- /dev/null +++ b/packages/session/session-format-catalog/README.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write packages/session/session-format-catalog/README.md +README.md: 5e138e8e1fabe7a933e9c72189d12a2f3c50f479 +README.zh.md: 72b59e0398aba5c55acf9bef252b577e87d8ae39 diff --git a/packages/session/session-format-catalog/README.md b/packages/session/session-format-catalog/README.md new file mode 100644 index 0000000000..5e138e8e1f --- /dev/null +++ b/packages/session/session-format-catalog/README.md @@ -0,0 +1,98 @@ +--- +description: "Build-static first-party Session format codec and adjacent migration assembly for persistence readers." +kind: "package-library" +--- + +# @deepseek-ai/dsh-session-format-catalog + +English | [中文](README.zh.md) + +## Summary + +`dsh-session-format-catalog` gives persistence one deterministic Session format reader without consulting mounted plugins. It assembles the frozen v0 and v1 codecs with the single v0-to-v1 edge, checks the complete gap-free chain at module initialization, and exposes physical dispatch, header-only classification, migration, and current encoding through `sessionFormatCatalog`. + +## Table of Contents + +- [Use this package](#use-this-package) +- [Understand the implementation](#understand-the-implementation) +- [Further Exploration](#further-exploration) +- [Model Experience](#model-experience) +- [Known Limitations and Deferred Work](#known-limitations-and-deferred-work) +- [Dev Note](#dev-note) + +----- + + +## Use this package + +### When to use it + +Import this library from persistence and test-support readers that need the complete first-party released-format inventory before any feature plugin mounts. Feature compositions do not register or reorder its entries. No runtime invariant companion is published because construction rejects an invalid static inventory and each read validates its complete result; the catalog retains no independently mutable runtime relationship. + +### Entry point + +```text +const descriptor = sessionFormatCatalog.readHeader(physicalHeader) +const current = sessionFormatCatalog.migrate(sessionFormatCatalog.decodeArtifact(physicalHeader, rows)) +``` + +Import `sessionFormatCatalog` from the package root. JSONL readers pass parsed header and row JSON values to `decodeArtifact()` or `decodeRecoverableArtifact()`, migrate the logical result with `migrate()`, and serialize only the validated current artifact with `encodeCurrent()`. Listing calls `readHeader()` and never opens event bodies. Header reads validate every adjacent target and then restore the final header through the installed current Session package. + +The catalog contains all supported historical readers directly. A profile cannot add, remove, or reorder an edge by mounting a feature plugin. Its peer dependency on `dsh-session` supplies the installed current event vocabulary and current restoration rules, while historical edge validators remain frozen. + +----- + + +## Understand the implementation + +
+Implementation internals — click to expand + +[`src/generated.ts`](src/generated.ts) is the static owner of codec and edge ordering. [`src/current.ts`](src/current.ts) delegates final header, envelope, message, surface, seed, and current request-header validation to the installed Session semantics. The low-level constructor rejects duplicate codecs, duplicate edges, gaps, and entries beyond the current version before any Session read can begin. + +
+ +----- + + +## Further Exploration + +- [Migration machinery](../session-format/README.md) — catalog construction and dispatch behavior. +- [Released v0 to v1 edge](../session-format-v0-to-v1/README.md) — codec and validator ownership. +- [JSONL persistence](../session-persistence-jsonl/README.md) — immutable generation naming and exclusive publication. + +----- + + +## Model Experience + +### Catalog dispatch + +#### What the model sees + +Nothing directly. The catalog only restores the `SessionEvent` history consumed by request reconstruction. + +#### Token effect + +Zero direct tokens. + +#### KV Cache effect + +No direct effect; restored history determines cache identity in its consumer. + +## Known Limitations and Deferred Work + + + +- **First-party build inventory only** — external migration ownership and distribution are not supported. +- **Generated ordering is closed** — runtime plugin registration cannot supply a missing historical edge. + + +### Dev Note + +
+Working context for maintainers — click to expand + +None. + +
diff --git a/packages/session/session-format-catalog/README.zh.md b/packages/session/session-format-catalog/README.zh.md new file mode 100644 index 0000000000..72b59e0398 --- /dev/null +++ b/packages/session/session-format-catalog/README.zh.md @@ -0,0 +1,98 @@ +--- +description: "供持久化读取方使用的构建期静态第一方 Session 格式编解码器与相邻迁移装配。" +kind: "package-library" +--- + +# @deepseek-ai/dsh-session-format-catalog + +[English](README.md) | 中文 + +## 概述 + +`dsh-session-format-catalog` 为持久化提供一个确定性的 Session 格式读取器，且无需查询已挂载插件。它把冻结的 v0 和 v1 编解码器与唯一的 v0 到 v1 迁移边装配起来，在模块初始化时校验完整且无缺口的迁移链，并通过 `sessionFormatCatalog` 暴露物理分派、仅标头分类、迁移和当前格式编码。 + +## 目录 + +- [使用本包](#use-this-package) +- [理解实现](#understand-the-implementation) +- [进一步探索](#further-exploration) +- [模型体验](#model-experience) +- [已知限制与延期工作](#known-limitations-and-deferred-work) +- [开发备注](#dev-note) + +----- + + +## 使用本包 + +### 何时使用 + +当持久化与测试支持读取方需要在任何功能插件挂载前取得完整第一方已发布格式清单时，导入本库。功能组合不会注册或重排其条目。它不发布运行时不变式伴生入口，因为构造过程会拒绝无效静态清单，每次读取也会校验完整结果；目录不保留可独立分叉的运行时可变关系。 + +### 入口 + +```text +const descriptor = sessionFormatCatalog.readHeader(physicalHeader) +const current = sessionFormatCatalog.migrate(sessionFormatCatalog.decodeArtifact(physicalHeader, rows)) +``` + +从包根导入 `sessionFormatCatalog`。JSONL 读取方把解析后的标头与行 JSON 值传给 `decodeArtifact()` 或 `decodeRecoverableArtifact()`，使用 `migrate()` 迁移逻辑结果，并且只使用 `encodeCurrent()` 序列化经过校验的当前产物。列表读取调用 `readHeader()`，绝不打开事件正文。标头读取会校验每个相邻目标，然后通过已安装的当前 Session 包还原最终标头。 + +该目录直接包含所有受支持的历史读取器。Profile 无法通过挂载功能插件来添加、移除或重新排列迁移边。它通过对 `dsh-session` 的 peer 依赖获得已安装的当前事件词表与当前还原规则，而历史迁移边校验器保持冻结。 + +----- + + +## 理解实现 + +
+实现细节——点击展开 + +[`src/generated.ts`](src/generated.ts) 是编解码器与迁移边顺序的静态所有者。[`src/current.ts`](src/current.ts) 把最终标头、事件信封、消息、表面、种子和当前请求标头校验委托给已安装的 Session 语义。底层构造函数会在开始读取任何 Session 之前拒绝重复编解码器、重复迁移边、缺口，以及超过当前版本的条目。 + +
+ +----- + + +## 进一步探索 + +- [迁移机制](../session-format/README.zh.md)——目录构造与分派行为。 +- [已发布 v0 到 v1 迁移边](../session-format-v0-to-v1/README.zh.md)——编解码器与校验器所有权。 +- [JSONL 持久化](../session-persistence-jsonl/README.zh.md)——不可变 generation 命名与排他发布。 + +----- + + +## 模型体验 + +### 目录分派 + +#### 模型看到什么 + +没有直接内容。该目录只还原由请求重建逻辑消费的 `SessionEvent` 历史。 + +#### Token 影响 + +不直接产生 token。 + +#### KV Cache 影响 + +没有直接影响；还原后的历史在其消费者中决定缓存身份。 + +## 已知限制与延期工作 + + + +- **仅包含第一方构建清单**——尚不支持外部迁移所有权与分发。 +- **生成顺序封闭**——运行时插件注册无法补充缺失的历史迁移边。 + + +### 开发备注 + +
+维护者的工作上下文——点击展开 + +无。 + +
diff --git a/packages/session/session-format-catalog/package.json b/packages/session/session-format-catalog/package.json new file mode 100644 index 0000000000..3385d53936 --- /dev/null +++ b/packages/session/session-format-catalog/package.json @@ -0,0 +1,41 @@ +{ + "name": "@deepseek-ai/dsh-session-format-catalog", + "description": "Build-static first-party Session format codec and migration catalog", + "version": "0.1.2-alpha.3", + "publishConfig": { + "access": "public" + }, + "repository": { + "type": "git", + "url": "git+https://github.com/deepseek-ai/deepseek-harness.git", + "directory": "packages/session/session-format-catalog" + }, + "type": "module", + "main": "lib/index.js", + "types": "lib/types/index.d.ts", + "exports": { + ".": { + "types": "./lib/types/index.d.ts", + "default": "./lib/index.js" + }, + "./src/*": "./src/*", + "./package.json": "./package.json" + }, + "files": [ + "lib/index.js", + "lib/types/**/*.d.ts" + ], + "license": "MIT", + "dependencies": { + "@deepseek-ai/dsh-session-format": "workspace:^", + "@deepseek-ai/dsh-session-format-v0-to-v1": "workspace:^" + }, + "peerDependencies": { + "@deepseek-ai/cordis": "workspace:^", + "@deepseek-ai/dsh-session": "workspace:^" + }, + "devDependencies": { + "@deepseek-ai/cordis": "workspace:^", + "@deepseek-ai/dsh-session": "workspace:^" + } +} diff --git a/packages/session/session-format-catalog/src/current.ts b/packages/session/session-format-catalog/src/current.ts new file mode 100644 index 0000000000..8dbd66b950 --- /dev/null +++ b/packages/session/session-format-catalog/src/current.ts @@ -0,0 +1,48 @@ +/** Current installed Session validation used after vocabulary-aware format restoration. */ + +import { + SESSION_FORMAT_VERSION, + Session, + SessionId, + SessionLogOffset, +} from '@deepseek-ai/dsh-session' +import type { SessionEvent, SessionHeader } from '@deepseek-ai/dsh-session' +import type { SessionFormatArtifact, SessionFormatHeader } from '@deepseek-ai/dsh-session-format' + +/** + * Validate current logical metadata through the installed Session package. + * @param header - detached current logical header. + * @returns nothing after successful validation. + */ +export function validateInstalledCurrentSessionHeader(header: SessionFormatHeader): void { + if (header.version !== SESSION_FORMAT_VERSION) { + throw new Error( + `installed Session format is v${SESSION_FORMAT_VERSION}, got v${header.version}`, + ) + } + Session.fromRestore( + SessionId(header.id), + [], + header as unknown as SessionHeader, + SessionLogOffset(0), + ) +} + +/** + * Validate current header, event envelopes, messages, surface operations, and seed cut through the installed Session package. + * @param artifact - vocabulary-restored current logical artifact. + * @returns nothing after successful validation. + */ +export function validateInstalledCurrentSessionArtifact(artifact: SessionFormatArtifact): void { + if (artifact.header.version !== SESSION_FORMAT_VERSION) { + throw new Error( + `installed Session format is v${SESSION_FORMAT_VERSION}, got v${artifact.header.version}`, + ) + } + Session.fromRestore( + SessionId(artifact.header.id), + artifact.events as SessionEvent[], + artifact.header as unknown as SessionHeader, + SessionLogOffset(artifact.inheritedEventCount), + ) +} diff --git a/packages/session/session-format-catalog/src/generated.ts b/packages/session/session-format-catalog/src/generated.ts new file mode 100644 index 0000000000..e7b390483e --- /dev/null +++ b/packages/session/session-format-catalog/src/generated.ts @@ -0,0 +1,26 @@ +/** + * GENERATED by `scripts/gen-session-format-catalog.ts` — do not edit by hand. + * The direct imports make historical readability independent of mounted plugins. + */ + +import { KNOWN_SESSION_EVENT_TYPES } from '@deepseek-ai/dsh-session' +import { createSessionFormatCatalog } from '@deepseek-ai/dsh-session-format' +import { validateInstalledCurrentSessionArtifact, validateInstalledCurrentSessionHeader } from './current.ts' +import { assertReleasedV1Header, releasedV0SessionFormatCodec, releasedV1SessionFormatCodec, restoreReleasedV1Artifact, sessionFormatV0ToV1 } from '@deepseek-ai/dsh-session-format-v0-to-v1' + +/** Physical codec dispatch and complete adjacent chain, independent of mounted plugins. */ +export const sessionFormatCatalog = createSessionFormatCatalog({ + currentVersion: 1, + codecs: [releasedV0SessionFormatCodec, releasedV1SessionFormatCodec], + migrations: [sessionFormatV0ToV1], + restoreCurrent(artifact) { + const restored = restoreReleasedV1Artifact(artifact, KNOWN_SESSION_EVENT_TYPES) + validateInstalledCurrentSessionArtifact(restored) + return restored + }, + restoreCurrentHeader(header) { + assertReleasedV1Header(header) + validateInstalledCurrentSessionHeader(header) + return header + }, +}) diff --git a/packages/session/session-format-catalog/src/index.ts b/packages/session/session-format-catalog/src/index.ts new file mode 100644 index 0000000000..095ce7bf84 --- /dev/null +++ b/packages/session/session-format-catalog/src/index.ts @@ -0,0 +1,4 @@ +/** Build-static first-party Session format migration catalog. */ + +export { sessionFormatCatalog } from './generated.ts' +export { SessionFormatUnsupportedMigrationError } from '@deepseek-ai/dsh-session-format' diff --git a/packages/session/session-format-catalog/tests/catalog.spec.ts b/packages/session/session-format-catalog/tests/catalog.spec.ts new file mode 100644 index 0000000000..c8a9cf336f --- /dev/null +++ b/packages/session/session-format-catalog/tests/catalog.spec.ts @@ -0,0 +1,57 @@ +import { describe, expect, it } from 'vitest' +import { sessionFormatCatalog } from '../src/index.ts' + +describe('first-party Session format catalog', () => { + it('statically owns the complete v0 to v1 chain', () => { + const header = { + type: 'session', + version: 0, + id: 'catalog', + createdAt: 1, + seedLength: 0, + delegationDepth: 0, + } + + expect(sessionFormatCatalog.currentVersion).toBe(1) + expect(sessionFormatCatalog.readHeader(header)).toEqual({ + status: 'migration-required', + storedVersion: 0, + targetVersion: 1, + header: { + version: 1, + id: 'catalog', + createdAt: 1, + isSeeded: true, + delegationDepth: 0, + }, + }) + + const currentHeader = { ...header, version: 1 } + const current = sessionFormatCatalog.decodeArtifact(currentHeader, [ + { type: 'turn/start', seq: 0, time: 2, data: { turn: 1 } }, + ]) + expect(sessionFormatCatalog.migrate(current)).toMatchObject({ + header: { version: 1, id: 'catalog' }, + }) + }) + + it('restores the installed current vocabulary without freezing ordinary payload additions', () => { + const header = { + type: 'session', version: 1, id: 'current-growth', createdAt: 1, delegationDepth: 0, + } + const extended = sessionFormatCatalog.decodeArtifact(header, [{ + type: 'turn/start', seq: 0, time: 1, data: { turn: 1, postReleaseMember: true }, + }]) + expect(sessionFormatCatalog.migrate(extended).events).toEqual(extended.events) + + const unknownRequired = sessionFormatCatalog.decodeArtifact(header, [{ + type: 'ordinary/not-installed', seq: 0, time: 1, data: 'future', + }]) + expect(() => sessionFormatCatalog.migrate(unknownRequired)).toThrow(/unknown required event/) + + const unknownIgnorable = sessionFormatCatalog.decodeArtifact(header, [{ + type: 'ordinary/external', seq: 0, time: 1, data: null, ignorable: true, + }]) + expect(sessionFormatCatalog.migrate(unknownIgnorable).events).toEqual(unknownIgnorable.events) + }) +}) diff --git a/packages/session/session-format-catalog/tests/current.spec.ts b/packages/session/session-format-catalog/tests/current.spec.ts new file mode 100644 index 0000000000..6b180f192a --- /dev/null +++ b/packages/session/session-format-catalog/tests/current.spec.ts @@ -0,0 +1,52 @@ +import { describe, expect, it } from 'vitest' +import type { SessionFormatArtifact, SessionFormatHeader } from '@deepseek-ai/dsh-session-format' +import { + validateInstalledCurrentSessionArtifact, + validateInstalledCurrentSessionHeader, +} from '../src/current.ts' + +const currentHeader: SessionFormatHeader = { + version: 1, + id: 'installed-current', + createdAt: 1, + isSeeded: false, + delegationDepth: 0, +} + +describe('installed current Session restoration', () => { + it('rejects version skew before entering current Session validation', () => { + expect(() => { validateInstalledCurrentSessionHeader({ ...currentHeader, version: 0 }) }) + .toThrow(/installed Session format is v1, got v0/) + const artifact: SessionFormatArtifact = { + header: { ...currentHeader, version: 0 }, + inheritedEventCount: 0, + events: [], + } + expect(() => { validateInstalledCurrentSessionArtifact(artifact) }) + .toThrow(/installed Session format is v1, got v0/) + }) + + it('accepts only current request-header reasons and the true starts-series marker', () => { + const artifact = (reason: string, startsSeries?: boolean): SessionFormatArtifact => ({ + header: { ...currentHeader }, + inheritedEventCount: 0, + events: [ + { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }, + { + type: 'request/header', seq: 1, time: 2, + data: { + header: { config: { provider: 'mock', model: 'mock' } }, + reason, + ...(startsSeries === undefined ? {} : { startsSeries }), + }, + }, + ], + }) + + expect(() => { validateInstalledCurrentSessionArtifact(artifact('fallback')) }) + .toThrow(/request\/header.*reason/) + expect(() => { validateInstalledCurrentSessionArtifact(artifact('initial', false)) }) + .toThrow(/startsSeries/) + expect(() => { validateInstalledCurrentSessionArtifact(artifact('series', true)) }).not.toThrow() + }) +}) diff --git a/packages/session/session-format-catalog/tsconfig.json b/packages/session/session-format-catalog/tsconfig.json new file mode 100644 index 0000000000..0d32e33846 --- /dev/null +++ b/packages/session/session-format-catalog/tsconfig.json @@ -0,0 +1,27 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": [ + "src" + ], + "references": [ + { + "path": "../../../vendor/cosmokit" + }, + { + "path": "../../../vendor/cordis" + }, + { + "path": "../../core/session" + }, + { + "path": "../session-format" + }, + { + "path": "../session-format-v0-to-v1" + } + ] +} diff --git a/packages/session/session-format-catalog/tsdown.config.ts b/packages/session/session-format-catalog/tsdown.config.ts new file mode 100644 index 0000000000..1207ca66a1 --- /dev/null +++ b/packages/session/session-format-catalog/tsdown.config.ts @@ -0,0 +1,13 @@ +import { defineConfig } from 'tsdown' + +/** Build the static catalog library. */ +export default defineConfig({ + entry: ['lib/types/index.js'], + outDir: 'lib', + format: ['esm'], + platform: 'node', + target: 'es2024', + fixedExtension: false, + dts: false, + clean: false, +}) diff --git a/packages/session/session-format-v0-to-v1/README.i18n.yaml b/packages/session/session-format-v0-to-v1/README.i18n.yaml new file mode 100644 index 0000000000..c6bae8770a --- /dev/null +++ b/packages/session/session-format-v0-to-v1/README.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write packages/session/session-format-v0-to-v1/README.md +README.md: 965f0b2a8dd99cf44ba5ac16207a1b8502839b64 +README.zh.md: 848236e8042a380b9e8091e0ad0609113d3b79cb diff --git a/packages/session/session-format-v0-to-v1/README.md b/packages/session/session-format-v0-to-v1/README.md new file mode 100644 index 0000000000..965f0b2a8d --- /dev/null +++ b/packages/session/session-format-v0-to-v1/README.md @@ -0,0 +1,107 @@ +--- +description: "Frozen released-v0 Session header, event, and packed-row decoder with the identity conversion to v1." +kind: "package-library" +--- + +# @deepseek-ai/dsh-session-format-v0-to-v1 + +English | [中文](README.zh.md) + +## Summary + +`dsh-session-format-v0-to-v1` decodes the complete released-v0 JSONL record language and converts it into the shared-layout v1 format. The edge preserves validated header and event facts except for `version: 0` becoming `version: 1`; it also applies the finite legacy normalizers that v0 persistence accepted. The package freezes the v0 reader, the strict v1 migration target validator, and a vocabulary-neutral v1 physical codec that a later edge can reuse without importing the latest Session representation. + +## Table of Contents + +- [Use this package](#use-this-package) +- [Understand the implementation](#understand-the-implementation) +- [Further Exploration](#further-exploration) +- [Model Experience](#model-experience) +- [Known Limitations and Deferred Work](#known-limitations-and-deferred-work) +- [Dev Note](#dev-note) + +----- + + +## Use this package + +### When to use it + +Persistence obtains this edge through `dsh-session-format-catalog`; feature compositions do not mount it. Import it directly only when assembling or testing the static released-format catalog. No runtime invariant companion is published because every codec and migration call validates its complete source or target artifact and retains no runtime state. + +### Entry point + +```text +const decodedV0 = releasedV0SessionFormatCodec.decodeArtifact(header, rows) +const migratedV1 = sessionFormatV0ToV1.migrate(decodedV0) +``` + +`releasedV0SessionFormatCodec` reads the exact v0 header and physical rows, including packed assistant deltas and range-encoded provenance. `sessionFormatV0ToV1` normalizes and strictly validates a complete detached artifact. `releasedV1SessionFormatCodec` preserves the v1 physical layout without freezing the ordinary event vocabulary; the catalog restores current events against the installed Session package. + +The alpha edge refuses every event type outside its frozen inventory, including an unknown event marked `ignorable: true`. It also refuses unexpected payload members. `tool/result.meta` and nested PTC `arguments` remain explicit opaque JSON fields and are preserved without Session-sequence interpretation. + +The bounded historical normalizers convert `steering/message` to `user/message`, remove `turn/start.trigger`, convert retired `turn/end` reasons, add the current message wrappers and deterministic legacy message ids, and remove the obsolete `request/header.header.messagePrefix` duplicate. Retired `request/header-delta`, `mode/set`, and the `request/header` fallback reason refuse migration. No other event, reference, source, or payload fact may change. + +----- + + +## Understand the implementation + +
+Implementation internals — click to expand + +The physical codec expands each packed row atomically and never mutates parsed input. Recoverable decoding rolls back a complete faulty row and keeps the preceding prefix unless a later decoded `turn/end` proves that the faulty region was committed. The migration validates the frozen payload disposition before changing the header version and validates the exact v1 target again. + +| File | Role | +|---|---| +| [`src/codec.ts`](src/codec.ts) | Frozen v0/v1 physical headers, packed rows, and provenance ranges | +| [`src/dispositions.ts`](src/dispositions.ts) | Released-v0 event and payload-member inventory | +| [`src/migration.ts`](src/migration.ts) | Identity edge and legacy normalization | +| [`src/validation.ts`](src/validation.ts) | Exact source and target validation | + +
+ +----- + + +## Further Exploration + +- [Migration machinery](../session-format/README.md) — pure chain and codec contracts. +- [Static catalog](../session-format-catalog/README.md) — build-owned assembly. +- [Session subsystem](../../../docs/subsystems/session.md) — current logical Session semantics. + +----- + + +## Model Experience + +### Historical restoration + +#### What the model sees + +Nothing directly. After restoration, `deriveMessages()` sees canonical released-v0 events unchanged under v1; bounded historical forms produce the same model-visible content through their defined current wrappers. + +#### Token effect + +Zero direct tokens. + +#### KV Cache effect + +No direct effect for canonical v0 history. Bounded normalizers preserve model-visible content while producing current wrappers and deterministic identities. + +## Known Limitations and Deferred Work + + + +- **Closed first-party inventory** — unknown external-plugin events refuse migration in this alpha policy. +- **One adjacent edge** — this package does not perform publication or select later migrations. + + +### Dev Note + +
+Working context for maintainers — click to expand + +None. + +
diff --git a/packages/session/session-format-v0-to-v1/README.zh.md b/packages/session/session-format-v0-to-v1/README.zh.md new file mode 100644 index 0000000000..848236e804 --- /dev/null +++ b/packages/session/session-format-v0-to-v1/README.zh.md @@ -0,0 +1,107 @@ +--- +description: "冻结的已发布 v0 Session 标头、事件与打包行解码器，以及到 v1 的恒等转换。" +kind: "package-library" +--- + +# @deepseek-ai/dsh-session-format-v0-to-v1 + +[English](README.md) | 中文 + +## 概述 + +`dsh-session-format-v0-to-v1` 解码完整的已发布 v0 JSONL 记录语言，并把它转换为共享布局的 v1 格式。除把 `version: 0` 改为 `version: 1` 外，该迁移边会保留经过校验的标头与事件事实；它也会应用 v0 持久化曾接受的有限旧格式规范化。该包冻结 v0 读取器、严格的 v1 迁移目标校验器，以及不冻结事件词表的 v1 物理编解码器，使后续迁移边无需导入最新 Session 表示即可复用它。 + +## 目录 + +- [使用本包](#use-this-package) +- [理解实现](#understand-the-implementation) +- [进一步探索](#further-exploration) +- [模型体验](#model-experience) +- [已知限制与延期工作](#known-limitations-and-deferred-work) +- [开发备注](#dev-note) + +----- + + +## 使用本包 + +### 何时使用 + +持久化通过 `dsh-session-format-catalog` 获取该迁移边；功能组合不会挂载它。只有在装配或测试静态已发布格式目录时，才直接导入本包。它不发布运行时不变式伴生入口，因为每次 codec 与迁移调用都会校验完整的源或目标 artifact，且不保留运行时状态。 + +### 入口 + +```text +const decodedV0 = releasedV0SessionFormatCodec.decodeArtifact(header, rows) +const migratedV1 = sessionFormatV0ToV1.migrate(decodedV0) +``` + +`releasedV0SessionFormatCodec` 读取精确的 v0 标头与物理行，包括打包的 Assistant 增量和范围编码的来源序号。`sessionFormatV0ToV1` 规范化并严格校验一个完整且分离的产物。`releasedV1SessionFormatCodec` 在不冻结普通事件词表的前提下保留 v1 物理布局；目录会根据已安装的 Session 包还原当前事件。 + +Alpha 迁移边会拒绝冻结清单之外的所有事件类型，包括带有 `ignorable: true` 标记的未知事件。它也会拒绝意外的 payload 成员。`tool/result.meta` 与嵌套 PTC `arguments` 是显式的不透明 JSON 字段；迁移会原样保留它们，不把其中的数字解释为 Session 序号。 + +有限的历史规范化会把 `steering/message` 转换为 `user/message`、移除 `turn/start.trigger`、转换已停用的 `turn/end` reason、添加当前消息包装层与确定性的旧消息 id，并移除已停用且重复的 `request/header.header.messagePrefix`。已停用的 `request/header-delta`、`mode/set` 和 `request/header` fallback reason 会使迁移失败。除此之外，任何事件、引用、来源或 payload 事实都不得改变。 + +----- + + +## 理解实现 + +
+实现细节——点击展开 + +物理编解码器会以行为原子单位展开每个打包行，且绝不修改已解析输入。可恢复解码会回滚完整的故障行并保留此前前缀，除非后续成功解码的 `turn/end` 证明故障区域已经提交。迁移会先校验冻结的 payload 处置，再更改标头版本，并再次校验精确的 v1 目标。 + +| 文件 | 职责 | +|---|---| +| [`src/codec.ts`](src/codec.ts) | 冻结的 v0/v1 物理标头、打包行与来源序号范围 | +| [`src/dispositions.ts`](src/dispositions.ts) | 已发布 v0 事件与 payload 成员清单 | +| [`src/migration.ts`](src/migration.ts) | 恒等迁移边与旧格式规范化 | +| [`src/validation.ts`](src/validation.ts) | 精确的源与目标校验 | + +
+ +----- + + +## 进一步探索 + +- [迁移机制](../session-format/README.zh.md)——纯迁移链与编解码约定。 +- [静态目录](../session-format-catalog/README.zh.md)——构建拥有的装配。 +- [Session 子系统](../../../docs/subsystems/session.zh.md)——当前逻辑 Session 语义。 + +----- + + +## 模型体验 + +### 历史还原 + +#### 模型看到什么 + +没有直接内容。还原后，`deriveMessages()` 会看到在 v1 下保持不变的规范已发布 v0 事件；有限历史结构会通过规定的当前包装层产生相同的模型可见内容。 + +#### Token 影响 + +不直接产生 token。 + +#### KV Cache 影响 + +对规范 v0 历史没有直接影响。有限 normalizer 会在生成当前包装层与确定性标识时保留模型可见内容。 + +## 已知限制与延期工作 + + + +- **封闭的第一方清单**——按照当前 Alpha 策略，未知的外部插件事件会使迁移失败。 +- **单个相邻迁移边**——本包不执行发布，也不选择后续迁移。 + + +### 开发备注 + +
+维护者的工作上下文——点击展开 + +无。 + +
diff --git a/packages/session/session-format-v0-to-v1/package.json b/packages/session/session-format-v0-to-v1/package.json new file mode 100644 index 0000000000..b2fe4135fa --- /dev/null +++ b/packages/session/session-format-v0-to-v1/package.json @@ -0,0 +1,51 @@ +{ + "name": "@deepseek-ai/dsh-session-format-v0-to-v1", + "description": "Frozen released-v0 Session codec and identity migration to v1", + "version": "0.1.2-alpha.3", + "publishConfig": { + "access": "public" + }, + "repository": { + "type": "git", + "url": "git+https://github.com/deepseek-ai/deepseek-harness.git", + "directory": "packages/session/session-format-v0-to-v1" + }, + "type": "module", + "main": "lib/index.js", + "types": "lib/types/index.d.ts", + "exports": { + ".": { + "types": "./lib/types/index.d.ts", + "default": "./lib/index.js" + }, + "./src/*": "./src/*", + "./package.json": "./package.json" + }, + "files": [ + "lib/index.js", + "lib/types/**/*.d.ts" + ], + "license": "MIT", + "dsh": { + "sessionFormatMigration": { + "from": 0, + "to": 1, + "export": ".", + "migration": "sessionFormatV0ToV1", + "sourceCodec": "releasedV0SessionFormatCodec", + "targetCodec": "releasedV1SessionFormatCodec", + "targetHeaderValidator": "assertReleasedV1Header", + "targetRestorer": "restoreReleasedV1Artifact" + } + }, + "dependencies": { + "@deepseek-ai/dsh-session-format": "workspace:^" + }, + "peerDependencies": { + "@deepseek-ai/cordis": "workspace:^" + }, + "devDependencies": { + "@deepseek-ai/cordis": "workspace:^", + "@deepseek-ai/dsh-session": "workspace:^" + } +} diff --git a/packages/session/session-format-v0-to-v1/src/codec.ts b/packages/session/session-format-v0-to-v1/src/codec.ts new file mode 100644 index 0000000000..422ca5eeb0 --- /dev/null +++ b/packages/session/session-format-v0-to-v1/src/codec.ts @@ -0,0 +1,411 @@ +import { + SessionFormatError, + isSessionFormatJsonObject, + sessionFormatCount, + sessionFormatSafeInteger, + snapshotSessionFormatArtifact, + snapshotSessionFormatJson, +} from '@deepseek-ai/dsh-session-format' +import type { + EncodedSessionFormatArtifact, + SessionFormatArtifact, + SessionFormatCodec, + SessionFormatEncodeOptions, + SessionFormatEvent, + SessionFormatHeader, + SessionFormatJsonObject, + SessionFormatJsonValue, +} from '@deepseek-ai/dsh-session-format' +import { + assertReleasedSessionFormatHeader, + assertReleasedV0SourceArtifact, + assertReleasedV1PhysicalArtifact, +} from './validation.ts' +import { assertReleasedV0Keys, releasedV0Record } from './validation-helpers.ts' + +const PHYSICAL_HEADER_REQUIRED = ['type', 'version', 'id', 'createdAt', 'delegationDepth'] as const +const PHYSICAL_HEADER_OPTIONAL = ['cwd', 'parentSession', 'seedLength', 'origin', 'agentPreset'] as const +const PACKED_TAGS = new Set(['text-chunks', 'reasoning-chunks', 'tool-call-chunks']) + +/** Frozen physical JSON codec for the released v0 layout. */ +export const releasedV0SessionFormatCodec: SessionFormatCodec = createReleasedCodec(0) + +/** Frozen physical JSON codec for the shared-layout released v1 format. */ +export const releasedV1SessionFormatCodec: SessionFormatCodec = createReleasedCodec(1) + +function createReleasedCodec(version: 0 | 1): SessionFormatCodec { + return Object.freeze({ + version, + decodeHeader: (value: unknown) => decodeHeader(value, version), + decodeArtifact(headerValue: unknown, rowValues: readonly unknown[]) { + const physical = decodePhysicalHeader(headerValue, version) + const artifact = snapshotSessionFormatArtifact({ + header: physical.header, + inheritedEventCount: physical.inheritedEventCount, + events: scanRows(rowValues, false).events, + }, `released v${version} artifact`) + if (version === 0) assertReleasedV0SourceArtifact(artifact) + else assertReleasedV1PhysicalArtifact(artifact) + return artifact + }, + decodeRecoverableArtifact(headerValue: unknown, rowValues: readonly unknown[]) { + const physical = decodePhysicalHeader(headerValue, version) + const recovered = scanRows(rowValues, true) + const artifact = snapshotSessionFormatArtifact({ + header: physical.header, + inheritedEventCount: physical.inheritedEventCount, + events: recovered.events, + }, `released v${version} recoverable artifact`) + if (version === 0) assertReleasedV0SourceArtifact(artifact) + else assertReleasedV1PhysicalArtifact(artifact) + return artifact + }, + encodeArtifact(artifact: SessionFormatArtifact, options: SessionFormatEncodeOptions) { + if (version === 0) assertReleasedV0SourceArtifact(artifact) + else assertReleasedV1PhysicalArtifact(artifact) + return encodeArtifact(artifact, options, version) + }, + }) +} + +function decodeHeader(value: unknown, version: 0 | 1): SessionFormatHeader { + return decodePhysicalHeader(value, version).header +} + +function decodePhysicalHeader( + value: unknown, + version: 0 | 1, +): { header: SessionFormatHeader; inheritedEventCount: number } { + const source = snapshotSessionFormatJson(value, `released v${version} physical header`) + const record = releasedV0Record(source, `released v${version} physical header`) + assertReleasedV0Keys( + record, + PHYSICAL_HEADER_REQUIRED, + PHYSICAL_HEADER_OPTIONAL, + `released v${version} physical header`, + ) + if (record['type'] !== 'session' || record['version'] !== version) { + throw new SessionFormatError(`expected released v${version} physical Session header`) + } + if (typeof record['id'] !== 'string') throw new SessionFormatError(`released v${version} header id must be a string`) + const createdAt = sessionFormatCount(record['createdAt'], `released v${version} header createdAt`) + const delegationDepth = sessionFormatCount( + record['delegationDepth'], + `released v${version} header delegationDepth`, + ) + const seedLength = record['seedLength'] === undefined + ? 0 + : sessionFormatCount(record['seedLength'], `released v${version} header seedLength`) + for (const key of ['cwd', 'parentSession', 'agentPreset'] as const) { + if (record[key] !== undefined && typeof record[key] !== 'string') { + throw new SessionFormatError(`released v${version} header ${key} must be a string`) + } + } + if (record['origin'] !== undefined && record['origin'] !== 'subagent') { + throw new SessionFormatError(`released v${version} header origin must be "subagent"`) + } + const header = snapshotSessionFormatJson({ + version, + id: record['id'], + createdAt, + ...(record['cwd'] === undefined ? {} : { cwd: record['cwd'] }), + ...(record['parentSession'] === undefined ? {} : { parentSession: record['parentSession'] }), + isSeeded: record['seedLength'] !== undefined, + ...(record['origin'] === undefined ? {} : { origin: record['origin'] }), + delegationDepth, + ...(record['agentPreset'] === undefined ? {} : { agentPreset: record['agentPreset'] }), + }, `released v${version} logical header`) as SessionFormatHeader + assertReleasedSessionFormatHeader(header, version) + return { header, inheritedEventCount: seedLength } +} + +function encodeArtifact( + artifact: SessionFormatArtifact, + options: SessionFormatEncodeOptions, + version: 0 | 1, +): EncodedSessionFormatArtifact { + const header = artifact.header + const physicalHeader = snapshotSessionFormatJson({ + type: 'session', + version, + id: header.id, + createdAt: header.createdAt, + ...(header.cwd === undefined ? {} : { cwd: header.cwd }), + ...(header.parentSession === undefined ? {} : { parentSession: header.parentSession }), + ...(header.isSeeded ? { seedLength: artifact.inheritedEventCount } : {}), + ...(header.origin === undefined ? {} : { origin: header.origin }), + delegationDepth: header.delegationDepth, + ...(header.agentPreset === undefined ? {} : { agentPreset: header.agentPreset }), + }, `released v${version} encoded header`) as SessionFormatJsonObject + const records = options.packChunks ? packChunkRuns(artifact.events) : [...artifact.events] + const rows = Object.freeze(records.map(record => encodeProvenance(record))) + return Object.freeze({ header: physicalHeader, rows }) +} + +function scanRows( + rowValues: readonly unknown[], + recoverable: boolean, +): { readonly events: readonly SessionFormatEvent[] } { + const events: SessionFormatEvent[] = [] + let issue: SessionFormatError | undefined + for (const [rowIndex, value] of rowValues.entries()) { + let decoded: readonly SessionFormatEvent[] + try { + const row = snapshotSessionFormatJson(value, `released Session row ${rowIndex}`) + decoded = decodeRow(row, rowIndex) + } catch (error: unknown) { + const current = error instanceof SessionFormatError + ? error + : new SessionFormatError(`released Session row ${rowIndex} is malformed`, { cause: error }) + if (!recoverable) throw current + issue ??= current + continue + } + if (issue !== undefined) { + if (decoded.some(event => event.type === 'turn/end')) throw issue + continue + } + const rowStart = events.length + for (const event of decoded) { + if (event.seq !== events.length) { + const gap = new SessionFormatError( + `released Session row ${rowIndex} has seq gap (expected ${events.length}, got ${event.seq})`, + ) + events.length = rowStart + if (!recoverable) throw gap + issue = gap + break + } + events.push(event) + } + if (issue !== undefined) { + if (decoded.some(event => event.type === 'turn/end')) throw issue + continue + } + } + return Object.freeze({ events: Object.freeze(events) }) +} + +function decodeRow(value: SessionFormatJsonValue, rowIndex: number): readonly SessionFormatEvent[] { + const record = releasedV0Record(value, `released Session row ${rowIndex}`) + const type = record['type'] + if (typeof type === 'string' && PACKED_TAGS.has(type)) return expandPackedRow(record, type, rowIndex) + if (record['sourceEventSeqs'] !== undefined) { + const seq = sessionFormatCount(record['seq'], `released Session row ${rowIndex} seq`) + return Object.freeze([{ + ...record, + sourceEventSeqs: decodeSeqRanges(record['sourceEventSeqs'], seq), + } as unknown as SessionFormatEvent]) + } + return Object.freeze([record as unknown as SessionFormatEvent]) +} + +function expandPackedRow( + row: Record, + type: string, + rowIndex: number, +): readonly SessionFormatEvent[] { + const label = `released ${type} row ${rowIndex}` + assertReleasedV0Keys(row, ['type', 'seq0', 'time0', 'data'], [], label) + const seq0 = sessionFormatCount(row['seq0'], `${label} seq0`) + let time = sessionFormatSafeInteger(row['time0'], `${label} time0`) + const data = releasedV0Record(row['data'], `${label} data`) + const isTool = type === 'tool-call-chunks' + assertReleasedV0Keys( + data, + isTool ? ['turn', 'step', 'index', 'id', 'dt', 'args'] : ['turn', 'step', 'index', 'dt', 'texts'], + isTool ? ['name'] : [], + `${label} data`, + ) + const payload = data[isTool ? 'args' : 'texts'] + if (!Array.isArray(payload) || payload.length === 0 || payload.some(member => typeof member !== 'string')) { + throw new SessionFormatError(`${label} payload must be a non-empty string array`) + } + const gaps = data['dt'] + if (!Array.isArray(gaps) || gaps.length !== payload.length - 1) { + throw new SessionFormatError(`${label} dt length must match its payload`) + } + for (const gap of gaps) sessionFormatSafeInteger(gap, `${label} dt member`) + if (typeof data['turn'] !== 'number' || typeof data['step'] !== 'number' || typeof data['index'] !== 'number') { + throw new SessionFormatError(`${label} turn, step, and index must be numbers`) + } + if (isTool && (typeof data['id'] !== 'string' + || (data['name'] !== undefined && typeof data['name'] !== 'string'))) { + throw new SessionFormatError(`${label} id and optional name must be strings`) + } + const output: SessionFormatEvent[] = [] + for (let index = 0; index < payload.length; index += 1) { + if (index > 0) time = sessionFormatSafeInteger(time + (gaps[index - 1] as number), `${label} member time`) + const member = payload[index] as string + const chunk = type === 'text-chunks' + ? { type: 'text-delta', index: data['index'], text: member } + : type === 'reasoning-chunks' + ? { type: 'reasoning-delta', index: data['index'], text: member } + : { + type: 'tool-call-delta', + index: data['index'], + id: data['id'], + ...(data['name'] === undefined ? {} : { name: data['name'] }), + argumentsDelta: member, + } + output.push(snapshotSessionFormatJson({ + type: 'assistant/chunk', + seq: sessionFormatCount(seq0 + index, `${label} member seq`), + time, + data: { turn: data['turn'], step: data['step'], chunk }, + }, `${label} member`) as SessionFormatEvent) + } + return Object.freeze(output) +} + +function decodeSeqRanges(value: SessionFormatJsonValue, maxEntries: number): readonly SessionFormatJsonValue[] { + if (!Array.isArray(value)) throw new SessionFormatError('sourceEventSeqs must be an array') + const output: number[] = [] + let hasRange = false + for (const entry of value) { + if (typeof entry === 'number') { + if (output.length >= maxEntries) throw new SessionFormatError('sourceEventSeqs exceeds its event seq') + output.push(sessionFormatCount(entry, 'sourceEventSeqs member')) + continue + } + if (!Array.isArray(entry) || entry.length !== 2) { + throw new SessionFormatError('sourceEventSeqs range must be a [start, end] pair') + } + const start = sessionFormatCount(entry[0], 'sourceEventSeqs range start') + const end = sessionFormatCount(entry[1], 'sourceEventSeqs range end') + if (end < start || end - start + 1 > maxEntries - output.length) { + throw new SessionFormatError('sourceEventSeqs range exceeds its event seq') + } + for (let seq = start; seq <= end; seq += 1) output.push(seq) + hasRange = true + } + if (hasRange && output.some((member, index) => index > 0 && member <= (output[index - 1] as number))) { + throw new SessionFormatError('sourceEventSeqs ranges must be strictly increasing') + } + return Object.freeze(output) +} + +function encodeProvenance(record: SessionFormatEvent | SessionFormatJsonObject): SessionFormatJsonObject { + if (!Object.hasOwn(record, 'sourceEventSeqs')) return record + const sourceEventSeqs = record['sourceEventSeqs'] as readonly SessionFormatJsonValue[] + const values = sourceEventSeqs.map(value => sessionFormatCount(value, 'sourceEventSeqs member')) + return snapshotSessionFormatJson({ ...record, sourceEventSeqs: encodeSeqRanges(values) }) as SessionFormatJsonObject +} + +function encodeSeqRanges(values: readonly number[]): readonly SessionFormatJsonValue[] { + if (values.some((value, index) => index > 0 && value <= (values[index - 1] as number))) return Object.freeze([...values]) + const output: SessionFormatJsonValue[] = [] + for (let start = 0; start < values.length;) { + let end = start + while (end + 1 < values.length && values[end + 1] === (values[end] as number) + 1) end += 1 + if (end - start >= 2) output.push(Object.freeze([values[start] as number, values[end] as number])) + else for (let index = start; index <= end; index += 1) output.push(values[index] as number) + start = end + 1 + } + return Object.freeze(output) +} + +type ChunkKind = 'text-delta' | 'reasoning-delta' | 'tool-call-delta' + +function packChunkRuns(events: readonly SessionFormatEvent[]): readonly (SessionFormatEvent | SessionFormatJsonObject)[] { + const output: Array = [] + let kind: ChunkKind | undefined + let run: SessionFormatEvent[] = [] + const flush = (): void => { + if (kind !== undefined && run.length >= 3) output.push(buildPackedRow(kind, run)) + else output.push(...run) + kind = undefined + run = [] + } + for (const event of events) { + const candidate = classifyChunk(event) + const previous = run.at(-1) + if (candidate !== undefined && candidate === kind && previous !== undefined && continuesChunk(previous, event, candidate)) { + run.push(event) + continue + } + flush() + if (candidate === undefined) output.push(event) + else { + kind = candidate + run = [event] + } + } + flush() + return Object.freeze(output) +} + +function classifyChunk(event: SessionFormatEvent): ChunkKind | undefined { + if (event.type !== 'assistant/chunk') return undefined + const data = event.data + if (!isSessionFormatJsonObject(data) || !hasExactKeys(data, ['turn', 'step', 'chunk'])) return undefined + const chunk = data['chunk'] + if (!isSessionFormatJsonObject(chunk) + || typeof chunk['index'] !== 'number' + || typeof chunk['type'] !== 'string') return undefined + if (chunk['type'] === 'text-delta' || chunk['type'] === 'reasoning-delta') { + return hasExactKeys(chunk, ['type', 'index', 'text']) && typeof chunk['text'] === 'string' + ? chunk['type'] + : undefined + } + if (chunk['type'] !== 'tool-call-delta') return undefined + const exact = hasExactKeys(chunk, ['type', 'index', 'id', 'argumentsDelta']) + || hasExactKeys(chunk, ['type', 'index', 'id', 'name', 'argumentsDelta']) + return exact && typeof chunk['id'] === 'string' + && typeof chunk['argumentsDelta'] === 'string' + && (chunk['name'] === undefined || typeof chunk['name'] === 'string') + ? 'tool-call-delta' + : undefined +} + +function continuesChunk(previous: SessionFormatEvent, next: SessionFormatEvent, kind: ChunkKind): boolean { + const previousData = previous.data as SessionFormatJsonObject + const nextData = next.data as SessionFormatJsonObject + const previousChunk = previousData['chunk'] as SessionFormatJsonObject + const nextChunk = nextData['chunk'] as SessionFormatJsonObject + if (!Number.isSafeInteger(next.time - previous.time)) return false + if (nextData['turn'] !== previousData['turn'] || nextData['step'] !== previousData['step']) return false + if (nextChunk['index'] !== previousChunk['index']) return false + if (kind !== 'tool-call-delta') return true + return nextChunk['id'] === previousChunk['id'] + && Object.hasOwn(nextChunk, 'name') === Object.hasOwn(previousChunk, 'name') + && nextChunk['name'] === previousChunk['name'] +} + +function buildPackedRow(kind: ChunkKind, run: readonly SessionFormatEvent[]): SessionFormatJsonObject { + const first = run[0] as SessionFormatEvent + const firstData = first.data as SessionFormatJsonObject + const firstChunk = firstData['chunk'] as SessionFormatJsonObject + const base = { + turn: firstData['turn'], + step: firstData['step'], + index: firstChunk['index'], + dt: run.slice(1).map((event, index) => event.time - (run[index] as SessionFormatEvent).time), + } + if (kind === 'tool-call-delta') { + return snapshotSessionFormatJson({ + type: 'tool-call-chunks', + seq0: first.seq, + time0: first.time, + data: { + ...base, + id: firstChunk['id'], + ...(firstChunk['name'] === undefined ? {} : { name: firstChunk['name'] }), + args: run.map(event => ((event.data as SessionFormatJsonObject)['chunk'] as SessionFormatJsonObject)['argumentsDelta']), + }, + }) as SessionFormatJsonObject + } + return snapshotSessionFormatJson({ + type: kind === 'text-delta' ? 'text-chunks' : 'reasoning-chunks', + seq0: first.seq, + time0: first.time, + data: { + ...base, + texts: run.map(event => ((event.data as SessionFormatJsonObject)['chunk'] as SessionFormatJsonObject)['text']), + }, + }) as SessionFormatJsonObject +} + +function hasExactKeys(record: Readonly>, keys: readonly string[]): boolean { + return Object.keys(record).length === keys.length && keys.every(key => Object.hasOwn(record, key)) +} diff --git a/packages/session/session-format-v0-to-v1/src/dispositions.ts b/packages/session/session-format-v0-to-v1/src/dispositions.ts new file mode 100644 index 0000000000..257c9c8d3d --- /dev/null +++ b/packages/session/session-format-v0-to-v1/src/dispositions.ts @@ -0,0 +1,110 @@ +/** Exact top-level payload disposition frozen for every released-v0 event type. */ +export interface ReleasedV0PayloadDisposition { + readonly required: readonly string[] + readonly optional: readonly string[] + /** JSON members whose nested representation is intentionally owner-opaque. */ + readonly opaque: readonly string[] +} + +function disposition( + required: readonly string[], + optional: readonly string[] = [], + opaque: readonly string[] = [], +): ReleasedV0PayloadDisposition { + return Object.freeze({ + required: Object.freeze([...required]), + optional: Object.freeze([...optional]), + opaque: Object.freeze([...opaque]), + }) +} + +/** + * Frozen released-v0 event and payload-member inventory. + * Every listed member is preserved by the identity edge; members in `opaque` + * remain lossless JSON but receive no nested Session-sequence interpretation. + */ +export const RELEASED_V0_EVENT_DISPOSITIONS: Readonly> = Object.freeze({ + 'agent-preset/selected': disposition(['agentPreset']), + 'agent/inbox/spliced': disposition(['target', 'start', 'inserted'], ['removedCount', 'outcome']), + 'approval/asked': disposition(['id', 'toolName'], ['callId', 'reason']), + 'approval/decided': disposition(['id', 'outcome']), + 'approval/policy': disposition(['policy'], ['source']), + 'assistant/chunk': disposition(['turn', 'step', 'chunk']), + 'assistant/message': disposition(['turn', 'step', 'message'], ['usage', 'interrupted']), + 'command/done': disposition(['commandId', 'kind'], ['text', 'sourceEventSeq']), + 'command/run': disposition(['commandId', 'name', 'source'], ['args']), + 'compaction/end': disposition(['compactionId', 'turn'], ['sourceCommandId', 'error']), + 'compaction/prune': disposition(['shadowedRange', 'shadowedSeqs', 'shadowedTokenCount']), + 'compaction/start': disposition(['compactionId', 'turn'], ['sourceCommandId']), + 'compaction/summary': disposition( + ['compactionId', 'summary', 'shadowedRange', 'shadowedSeqs', 'shadowedTokenCount', 'provider', 'model'], + ['sourceCommandId', 'maxTokens', 'usage', 'rawOutput', 'llmStreamCall'], + ), + 'feedback/record': disposition(['text']), + 'goal/change': disposition( + ['kind', 'version', 'operation'], + ['goal', 'roundsStarted', 'createdAt', 'updatedAt', 'cleared', 'clearedAt'], + ), + 'hook/invoked': disposition(['turn', 'point', 'dialect', 'handlerId'], ['matcher']), + 'hook/result': disposition( + ['turn', 'point', 'handlerId', 'decision', 'durationMs'], + ['exitCode', 'stderrSummary'], + ), + 'llm/retry': disposition( + ['retryId', 'turn', 'step', 'provider', 'mode', 'policyKey', 'retry', 'delayMs', 'failure'], + ['maxRetries'], + ), + 'llm/retry-started': disposition(['retryId', 'turn', 'step', 'retry']), + 'model/selection': disposition(['provider', 'model'], ['reasoningEffort']), + 'permission/preset': disposition(['preset']), + 'plan/mode': disposition(['active']), + 'request/context': disposition(['provider', 'model'], ['contextWindow']), + 'request/header': disposition(['header', 'reason'], ['startsSeries']), + 'sandbox/mode': disposition(['mode'], ['source']), + 'schedule/change': disposition(['version', 'operation'], ['schedule', 'id', 'acceptedAt']), + 'session-log-deepseek/delivery-accepted': disposition( + ['sessionId', 'throughSeq'], + ), + 'session/end-seed': disposition([]), + 'session/title': disposition(['title', 'messageSeqs', 'source']), + 'session/title-llm-request': disposition( + ['titleProvider', 'messageSeqs', 'route', 'system', 'messages', 'maxTokens'], + ), + 'step/end': disposition(['turn', 'step']), + 'step/start': disposition(['turn', 'step']), + 'subagent/descriptor': disposition( + ['mode', 'version', 'provider'], + ['label', 'agentProvider', 'agentModel', 'agentReasoningEffort', 'persona', 'toolFilter'], + ), + 'subagent/model-selection-policy': disposition(['allowedModels']), + 'team/member': disposition(['version', 'teamId', 'member']), + 'team/message/delivered': disposition(['version', 'teamId', 'messageId', 'targetId']), + 'team/message/queued': disposition(['version', 'teamId', 'message']), + 'team/task': disposition(['version', 'teamId', 'task']), + 'todo/write': disposition(['todos']), + 'tool-workflow/agent-end': disposition(['runId', 'seq', 'outcome']), + 'tool-workflow/agent-start': disposition(['runId', 'seq', 'label', 'childId'], ['phase']), + 'tool-workflow/run-end': disposition(['runId', 'stopReason']), + 'tool-workflow/run-start': disposition(['runId', 'name']), + 'tool/call': disposition(['turn', 'step', 'callId', 'name', 'arguments']), + 'tool/code-dispatch': disposition( + ['rootCallId', 'parentCallId', 'subCallId', 'name', 'arguments', 'isError', 'content'], + [], + ['arguments'], + ), + 'tool/code-dispatch-start': disposition( + ['rootCallId', 'parentCallId', 'subCallId', 'name', 'arguments'], + [], + ['arguments'], + ), + 'tool/result': disposition(['turn', 'step', 'message'], ['error', 'meta'], ['meta']), + 'turn/end': disposition(['turn', 'reason']), + 'turn/start': disposition(['turn']), + 'user/message': disposition(['role', 'id', 'content', 'source']), + 'web/deepseek-search-llm-request': disposition(['endpoint', 'apiVersion', 'body']), +}) + +/** Stable sorted released-v0 event inventory. */ +export const RELEASED_V0_EVENT_TYPES: readonly string[] = Object.freeze( + Object.keys(RELEASED_V0_EVENT_DISPOSITIONS).sort((left, right) => left.localeCompare(right, 'en')), +) diff --git a/packages/session/session-format-v0-to-v1/src/index.ts b/packages/session/session-format-v0-to-v1/src/index.ts new file mode 100644 index 0000000000..c432456806 --- /dev/null +++ b/packages/session/session-format-v0-to-v1/src/index.ts @@ -0,0 +1,10 @@ +/** Frozen released-v0 physical codec and identity migration into shared-layout v1. */ + +export * from './codec.ts' +export * from './dispositions.ts' +export * from './migration.ts' +export { + assertReleasedV1Artifact, + assertReleasedV1Header, + restoreReleasedV1Artifact, +} from './validation.ts' diff --git a/packages/session/session-format-v0-to-v1/src/migration.ts b/packages/session/session-format-v0-to-v1/src/migration.ts new file mode 100644 index 0000000000..21d171f0fc --- /dev/null +++ b/packages/session/session-format-v0-to-v1/src/migration.ts @@ -0,0 +1,308 @@ +import { + SessionFormatError, + SessionFormatUnsupportedMigrationError, + defineSessionFormatMigration, + sessionFormatCount, + snapshotSessionFormatArtifact, +} from '@deepseek-ai/dsh-session-format' +import type { + SessionFormatEvent, + SessionFormatHeader, + SessionFormatJsonObject, + SessionFormatJsonValue, +} from '@deepseek-ai/dsh-session-format' +import { + assertReleasedEventPayload, + assertNormalizedReleasedV0Artifact, + assertReleasedV0SourceArtifact, + assertReleasedV1Artifact, + assertReleasedV1Header, +} from './validation.ts' +import { assertReleasedV0Keys, releasedV0Record } from './validation-helpers.ts' + +/** Identity format edge that promotes released v0 into released v1. */ +export const sessionFormatV0ToV1 = defineSessionFormatMigration({ + name: '@deepseek-ai/dsh-session-format-v0-to-v1', + fromVersion: 0, + toVersion: 1, + migrateHeader(header) { + assertHeaderVersion(header, 0) + return { ...header, version: 1 } + }, + migrate(source) { + assertReleasedV0SourceArtifact(source) + const events = normalizeReleasedV0Events(source.events, source.header.id) + assertNormalizedReleasedV0Artifact({ ...source, events }) + const target = snapshotSessionFormatArtifact({ + header: { ...source.header, version: 1 }, + inheritedEventCount: source.inheritedEventCount, + events, + }, 'released v0-to-v1 target') + assertReleasedV1Artifact(target) + return target + }, + validateTarget: assertReleasedV1Artifact, + validateTargetHeader: assertReleasedV1Header, +}) + +function assertHeaderVersion(header: SessionFormatHeader, version: 0 | 1): void { + if (header.version !== version) throw new SessionFormatError(`expected format v${version} header`) +} + +function normalizeReleasedV0Events( + events: readonly SessionFormatEvent[], + sessionId: string, +): readonly SessionFormatEvent[] { + const messageIds = new Map() + const output: SessionFormatEvent[] = [] + for (const event of events) { + assertSupportedLegacyType(event, sessionId) + const start = normalizeLegacyTurnStart(event, sessionId) + const end = normalizeLegacyTurnEnd(start, sessionId) + const header = normalizeLegacyRequestHeader(end, sessionId) + const steering = normalizeLegacySteering(header, sessionId) + const message = normalizeLegacyMessage(steering, sessionId, messageIds) + assertReleasedEventPayload(message, 0) + output.push(message) + const messageId = eventMessageId(message) + if (messageId !== undefined) messageIds.set(message.seq, messageId) + } + return Object.freeze(output) +} + +function normalizeLegacyRequestHeader(event: SessionFormatEvent, sessionId: string): SessionFormatEvent { + if (event.type !== 'request/header') return event + const data = releasedV0Record(event.data, `request/header ${event.seq} data`) + const header = releasedV0Record(data['header'], `request/header ${event.seq} header`) + if (!Object.hasOwn(header, 'messagePrefix')) return event + if (!Array.isArray(header['messagePrefix'])) { + throw new SessionFormatError( + `session ${JSON.stringify(sessionId)} contains malformed request/header messagePrefix at seq ${event.seq}`, + ) + } + const { messagePrefix: _messagePrefix, ...currentHeader } = header + return { ...event, data: { ...data, header: currentHeader } } +} + +function assertSupportedLegacyType(event: SessionFormatEvent, sessionId: string): void { + if (event.type === 'request/header-delta' || event.type === 'mode/set') { + throw new SessionFormatUnsupportedMigrationError( + `session ${JSON.stringify(sessionId)} contains unsupported legacy ${event.type} event at seq ${event.seq}`, + ) + } + if (event.type === 'request/header') { + const data = releasedV0Record(event.data, `request/header ${event.seq} data`) + if (data['reason'] === 'fallback') { + throw new SessionFormatUnsupportedMigrationError( + `session ${JSON.stringify(sessionId)} contains unsupported request/header reason "fallback" at seq ${event.seq}`, + ) + } + } +} + +function normalizeLegacySteering(event: SessionFormatEvent, sessionId: string): SessionFormatEvent { + if (event.type !== 'steering/message') return event + const data = releasedV0Record(event.data, `steering/message ${event.seq} data`) + const wrapped = data['message'] + if (wrapped !== undefined) { + assertReleasedV0Keys(data, ['turn', 'message'], [], `steering/message ${event.seq} data`) + sessionFormatCount(data['turn'], `steering/message ${event.seq} turn`) + return { ...event, type: 'user/message', data: wrapped } + } + assertReleasedV0Keys(data, ['turn', 'content', 'source'], [], `steering/message ${event.seq} data`) + sessionFormatCount(data['turn'], `steering/message ${event.seq} turn`) + const { turn: _turn, ...message } = data + return { + ...event, + type: 'user/message', + data: { + ...message, + id: legacyMessageId(sessionId, event.seq), + role: 'user', + }, + } +} + +function normalizeLegacyTurnStart(event: SessionFormatEvent, sessionId: string): SessionFormatEvent { + if (event.type !== 'turn/start') return event + const data = releasedV0Record(event.data, `turn/start ${event.seq} data`) + if (!Object.hasOwn(data, 'trigger')) return event + assertReleasedV0Keys(data, ['turn', 'trigger'], [], `turn/start ${event.seq} data`) + const turn = sessionFormatCount(data['turn'], `turn/start ${event.seq} turn`) + const trigger = releasedV0Record(data['trigger'], `turn/start ${event.seq} trigger`) + if (turn < 1 || typeof trigger['kind'] !== 'string' || trigger['kind'].length === 0) { + throw malformedLegacy(sessionId, 'turn/start', event.seq) + } + return { ...event, data: { turn } } +} + +function normalizeLegacyTurnEnd(event: SessionFormatEvent, sessionId: string): SessionFormatEvent { + if (event.type !== 'turn/end') return event + const data = releasedV0Record(event.data, `turn/end ${event.seq} data`) + assertReleasedV0Keys(data, ['turn', 'reason'], [], `turn/end ${event.seq} data`) + const turn = sessionFormatCount(data['turn'], `turn/end ${event.seq} turn`) + if (turn < 1) throw malformedLegacy(sessionId, 'turn/end', event.seq) + const reason = releasedV0Record(data['reason'], `turn/end ${event.seq} reason`) + if (typeof reason['kind'] !== 'string') throw malformedLegacy(sessionId, 'turn/end', event.seq) + + let current: SessionFormatJsonObject + switch (reason['kind']) { + case 'completed': + case 'blocked': + case 'max-tokens': + case 'interrupted': + assertReleasedV0Keys(reason, ['kind'], [], `turn/end ${event.seq} reason`) + return event + case 'aborted': + if (Object.hasOwn(reason, 'reason')) return event + assertReleasedV0Keys(reason, ['kind'], [], `turn/end ${event.seq} reason`) + current = { kind: 'aborted', reason: { kind: 'legacy' } } + break + case 'disposed': + assertReleasedV0Keys(reason, ['kind'], [], `turn/end ${event.seq} reason`) + current = { kind: 'aborted', reason: { kind: 'disposed' } } + break + case 'error': + if (Object.hasOwn(reason, 'error')) return event + current = normalizeLegacyErrorReason(reason, event.seq, sessionId) + break + default: + return event + } + return { ...event, data: { ...data, reason: current } } +} + +function normalizeLegacyErrorReason( + reason: Record, + seq: number, + sessionId: string, +): SessionFormatJsonObject { + sessionFormatCount(reason['step'], `turn/end ${seq} error step`) + const failure = reason['failure'] + if (failure !== undefined) { + assertReleasedV0Keys(reason, ['kind', 'step', 'failure'], [], `turn/end ${seq} reason`) + const record = releasedV0Record(failure, `turn/end ${seq} failure`) + assertReleasedV0Keys( + record, + ['message', 'code'], + ['status', 'providerRetryAfterMs', 'requestId'], + `turn/end ${seq} failure`, + ) + if (typeof record['message'] !== 'string' || typeof record['code'] !== 'string') { + throw malformedLegacy(sessionId, 'turn/end', seq) + } + return { kind: 'error', error: record } + } + assertReleasedV0Keys(reason, ['kind', 'step', 'message'], ['code'], `turn/end ${seq} reason`) + if (typeof reason['message'] !== 'string' + || (reason['code'] !== undefined && typeof reason['code'] !== 'string')) { + throw malformedLegacy(sessionId, 'turn/end', seq) + } + return { + kind: 'error', + error: { + message: reason['message'], + code: typeof reason['code'] === 'string' ? reason['code'] : 'UNKNOWN', + }, + } +} + +function normalizeLegacyMessage( + event: SessionFormatEvent, + sessionId: string, + messageIds: ReadonlyMap, +): SessionFormatEvent { + const data = releasedV0Record(event.data, `${event.type} ${event.seq} data`) + switch (event.type) { + case 'user/message': + if (Object.hasOwn(data, 'id') || Object.hasOwn(data, 'role') + || Object.hasOwn(data, 'message') || !Object.hasOwn(data, 'content') + || !Object.hasOwn(data, 'source')) return event + return { + ...event, + data: { + ...data, + id: legacyMessageId(sessionId, event.seq), + role: 'user', + }, + } + case 'assistant/message': { + if (Object.hasOwn(data, 'message') + || !Object.hasOwn(data, 'content') || !Object.hasOwn(data, 'provenance')) return event + const { content, provenance, ...eventData } = data as typeof data & { + content: SessionFormatJsonValue + provenance: SessionFormatJsonValue + } + const source = releasedV0Record(provenance, `assistant/message ${event.seq} provenance`) + return { + ...event, + data: { + ...eventData, + message: { + id: legacyMessageId(sessionId, event.seq), + role: 'assistant', + content, + source: { ...source, kind: 'model' }, + }, + }, + } + } + case 'tool/result': { + if (Object.hasOwn(data, 'message') + || !Object.hasOwn(data, 'callId') || !Object.hasOwn(data, 'content') + || !Object.hasOwn(data, 'isError')) return event + const { callId, content, isError, ...eventData } = data + if (typeof callId !== 'string' || typeof isError !== 'boolean' || content === undefined) return event + const inheritedId = replacementStart(event) + const messageId = inheritedId === undefined + ? legacyMessageId(sessionId, event.seq) + : messageIds.get(inheritedId) + if (messageId === undefined) { + throw new SessionFormatError(`tool/result ${event.seq} replacement cites a message without identity`) + } + return { + ...event, + data: { + ...eventData, + message: { + id: messageId, + role: 'user', + content: [{ type: 'tool-result', toolCallId: callId, content, isError }], + source: { kind: 'tool', callId }, + }, + }, + } + } + default: + return event + } +} + +function replacementStart(event: SessionFormatEvent): number | undefined { + const operation = event['surfaceOp'] + if (operation === undefined || !releasedIsRecord(operation) || operation['op'] !== 'replace') return undefined + // Source envelope validation admits only non-negative safe replacement endpoints. + return operation['start'] as number +} + +function eventMessageId(event: SessionFormatEvent): string | undefined { + const data = releasedV0Record(event.data, `${event.type} ${event.seq} data`) + const message = event.type === 'user/message' + ? data + : releasedIsRecord(data['message']) ? data['message'] : undefined + return typeof message?.['id'] === 'string' ? message['id'] : undefined +} + +function releasedIsRecord(value: unknown): value is Record { + return typeof value === 'object' && value !== null && !Array.isArray(value) +} + +function legacyMessageId(sessionId: string, seq: number): string { + return `legacy-message:${sessionId}:${seq}` +} + +function malformedLegacy(sessionId: string, type: string, seq: number): SessionFormatError { + return new SessionFormatError( + `session ${JSON.stringify(sessionId)} contains malformed pre-react-loop ${type} at seq ${seq}`, + ) +} diff --git a/packages/session/session-format-v0-to-v1/src/payload-validation.ts b/packages/session/session-format-v0-to-v1/src/payload-validation.ts new file mode 100644 index 0000000000..e66e2f98d9 --- /dev/null +++ b/packages/session/session-format-v0-to-v1/src/payload-validation.ts @@ -0,0 +1,1022 @@ +import { SessionFormatError, sessionFormatCount, sessionFormatSafeInteger } from '@deepseek-ai/dsh-session-format' +import type { + SessionFormatEvent, + SessionFormatJsonValue, +} from '@deepseek-ai/dsh-session-format' +import { assertReleasedV0Keys, releasedV0Record } from './validation-helpers.ts' + +type JsonRecord = Record + +/** + * Validate nested released payload semantics for one known event. + * @param event - known event with exact top-level members. + * @param version - source or current payload generation. + */ +export function assertReleasedPayloadSemantics(event: SessionFormatEvent, version: 0 | 1): void { + const data = releasedV0Record(event.data, `${event.type} ${event.seq} data`) + const label = `${event.type} ${event.seq}` + switch (event.type) { + case 'agent-preset/selected': + stringValue(data['agentPreset'], `${label} agentPreset`) + return + case 'agent/inbox/spliced': + literalValue(data['target'], ['next-turn', 'next-step'], `${label} target`) + countValue(data['start'], `${label} start`) + if (data['removedCount'] !== undefined) countValue(data['removedCount'], `${label} removedCount`) + arrayValue(data['inserted'], `${label} inserted`, (value) => { + messageValue(value, `${label} inserted message`, version, 'user') + }) + if (data['outcome'] !== undefined) literalValue(data['outcome'], ['canceled'], `${label} outcome`) + return + case 'approval/asked': + nonEmptyString(data['id'], `${label} id`) + nonEmptyString(data['toolName'], `${label} toolName`) + if (data['callId'] !== undefined) nonEmptyString(data['callId'], `${label} callId`) + if (data['reason'] !== undefined) stringValue(data['reason'], `${label} reason`) + return + case 'approval/decided': + nonEmptyString(data['id'], `${label} id`) + literalValue(data['outcome'], ['allowed-once', 'rejected', 'cancelled', 'unavailable'], `${label} outcome`) + return + case 'approval/policy': + literalValue(data['policy'], ['ask', 'never'], `${label} policy`) + if (data['source'] !== undefined) literalValue(data['source'], ['delegation'], `${label} source`) + return + case 'assistant/chunk': + coordinatePair(data, label) + streamChunkValue(data['chunk'], `${label} chunk`) + return + case 'assistant/message': + coordinatePair(data, label) + messageValue(data['message'], `${label} message`, version, 'assistant') + if (data['usage'] !== undefined) tokenUsageValue(data['usage'], `${label} usage`) + if (data['interrupted'] !== undefined) literalValue(data['interrupted'], [true], `${label} interrupted`) + return + case 'command/done': + nonEmptyString(data['commandId'], `${label} commandId`) + literalValue(data['kind'], ['success', 'error'], `${label} kind`) + if (data['text'] !== undefined) stringValue(data['text'], `${label} text`) + if (data['sourceEventSeq'] !== undefined) earlierSeq(data['sourceEventSeq'], event.seq, `${label} sourceEventSeq`) + return + case 'command/run': { + nonEmptyString(data['commandId'], `${label} commandId`) + nonEmptyString(data['name'], `${label} name`) + if (data['args'] !== undefined) stringValue(data['args'], `${label} args`) + const source = exactRecord(data['source'], `${label} source`, ['kind']) + literalValue(source['kind'], ['user'], `${label} source kind`) + return + } + case 'compaction/start': + case 'compaction/end': + nonEmptyString(data['compactionId'], `${label} compactionId`) + if (data['sourceCommandId'] !== undefined) nonEmptyString(data['sourceCommandId'], `${label} sourceCommandId`) + nullableValue(data['turn'], `${label} turn`, countValue) + if (data['error'] !== undefined) stringValue(data['error'], `${label} error`) + return + case 'compaction/prune': + shadowedValue(data, event.seq, label) + return + case 'compaction/summary': + if (data['llmStreamCall'] === true && data['rawOutput'] === undefined) { + throw new SessionFormatError(`${label} llmStreamCall requires rawOutput`) + } + nonEmptyString(data['compactionId'], `${label} compactionId`) + if (data['sourceCommandId'] !== undefined) nonEmptyString(data['sourceCommandId'], `${label} sourceCommandId`) + contentBlocksValue(data['summary'], `${label} summary`, version) + shadowedValue(data, event.seq, label) + nonEmptyString(data['provider'], `${label} provider`) + nonEmptyString(data['model'], `${label} model`) + if (data['maxTokens'] !== undefined) countValue(data['maxTokens'], `${label} maxTokens`) + if (data['usage'] !== undefined) tokenUsageValue(data['usage'], `${label} usage`) + if (data['rawOutput'] !== undefined) contentBlocksValue(data['rawOutput'], `${label} rawOutput`, version) + if (data['llmStreamCall'] !== undefined) literalValue(data['llmStreamCall'], [true], `${label} llmStreamCall`) + return + case 'feedback/record': + nonEmptyString(data['text'], `${label} text`) + return + case 'goal/change': + goalChangeValue(data, label) + return + case 'hook/invoked': + countValue(data['turn'], `${label} turn`) + nonEmptyString(data['point'], `${label} point`) + literalValue(data['dialect'], ['claude-code', 'codex'], `${label} dialect`) + if (data['matcher'] !== undefined) stringValue(data['matcher'], `${label} matcher`) + nonEmptyString(data['handlerId'], `${label} handlerId`) + return + case 'hook/result': + countValue(data['turn'], `${label} turn`) + nonEmptyString(data['point'], `${label} point`) + nonEmptyString(data['handlerId'], `${label} handlerId`) + nonEmptyString(data['decision'], `${label} decision`) + if (data['exitCode'] !== undefined) safeIntegerValue(data['exitCode'], `${label} exitCode`) + if (data['stderrSummary'] !== undefined) stringValue(data['stderrSummary'], `${label} stderrSummary`) + if (finiteNumberValue(data['durationMs'], `${label} durationMs`) < 0) { + throw new SessionFormatError(`${label} durationMs must be non-negative`) + } + return + case 'llm/retry': + nonEmptyString(data['retryId'], `${label} retryId`) + coordinatePair(data, label) + nonEmptyString(data['provider'], `${label} provider`) + literalValue(data['mode'], ['normal', 'always'], `${label} mode`) + nonEmptyString(data['policyKey'], `${label} policyKey`) + positiveIntegerValue(data['retry'], `${label} retry`) + if (data['mode'] === 'normal') { + const maxRetries = positiveIntegerValue(data['maxRetries'], `${label} maxRetries`) + if ((data['retry'] as number) > maxRetries) throw new SessionFormatError(`${label} retry exceeds maxRetries`) + } else if (data['maxRetries'] !== undefined) { + throw new SessionFormatError(`${label} always mode must omit maxRetries`) + } + if (countValue(data['delayMs'], `${label} delayMs`) > 2_147_483_647) { + throw new SessionFormatError(`${label} delayMs exceeds the timer range`) + } + llmFailureValue(data['failure'], `${label} failure`) + return + case 'llm/retry-started': + nonEmptyString(data['retryId'], `${label} retryId`) + coordinatePair(data, label) + positiveIntegerValue(data['retry'], `${label} retry`) + return + case 'model/selection': + nonEmptyString(data['provider'], `${label} provider`) + nonEmptyString(data['model'], `${label} model`) + if (data['reasoningEffort'] !== undefined) nonEmptyString(data['reasoningEffort'], `${label} reasoningEffort`) + return + case 'permission/preset': + nonEmptyString(data['preset'], `${label} preset`) + return + case 'plan/mode': + booleanValue(data['active'], `${label} active`) + return + case 'request/context': + nonEmptyString(data['provider'], `${label} provider`) + nonEmptyString(data['model'], `${label} model`) + if (data['contextWindow'] !== undefined) positiveIntegerValue(data['contextWindow'], `${label} contextWindow`) + return + case 'request/header': + requestHeaderValue(data['header'], `${label} header`) + literalValue(data['reason'], ['initial', 'resume', 'change', 'series'], `${label} reason`) + if (data['startsSeries'] !== undefined) literalValue(data['startsSeries'], [true], `${label} startsSeries`) + return + case 'sandbox/mode': + literalValue(data['mode'], ['read-only', 'workspace-write', 'danger-full-access'], `${label} mode`) + if (data['source'] !== undefined) literalValue(data['source'], ['delegation'], `${label} source`) + return + case 'schedule/change': + scheduleChangeValue(data, label) + return + case 'session-log-deepseek/delivery-accepted': + { + const acceptedVersion = data['sessionFormatVersion'] === undefined + ? 0 + : countValue(data['sessionFormatVersion'], `${label} sessionFormatVersion`) + if (acceptedVersion !== version) return + nonEmptyString(data['sessionId'], `${label} sessionId`) + earlierSeq(data['throughSeq'], event.seq, `${label} throughSeq`) + } + return + case 'session/end-seed': + return + case 'session/title': + nonEmptyString(data['title'], `${label} title`) + seqArray(data['messageSeqs'], event.seq, `${label} messageSeqs`, false) + titleSourceValue(data['source'], `${label} source`) + return + case 'session/title-llm-request': + nonEmptyString(data['titleProvider'], `${label} titleProvider`) + seqArray(data['messageSeqs'], event.seq, `${label} messageSeqs`, true) + modelRouteValue(data['route'], `${label} route`) + stringValue(data['system'], `${label} system`) + arrayValue(data['messages'], `${label} messages`, (value) => { + messageValue(value, `${label} message`, version) + }) + positiveIntegerValue(data['maxTokens'], `${label} maxTokens`) + return + case 'step/end': + case 'step/start': + coordinatePair(data, label) + return + case 'subagent/descriptor': + subagentDescriptorValue(data, label) + return + case 'subagent/model-selection-policy': + allowedModelsValue(data['allowedModels'], `${label} allowedModels`) + return + case 'team/member': + teamSelector(data, label) + teamMemberValue(data['member'], `${label} member`) + return + case 'team/message/delivered': + teamSelector(data, label) + nonEmptyString(data['messageId'], `${label} messageId`) + nonEmptyString(data['targetId'], `${label} targetId`) + return + case 'team/message/queued': + teamSelector(data, label) + teamMessageValue(data['message'], `${label} message`, version) + return + case 'team/task': + teamSelector(data, label) + teamTaskValue(data['task'], `${label} task`) + return + case 'todo/write': + arrayValue(data['todos'], `${label} todos`, (value, itemLabel) => { + const item = exactRecord(value, itemLabel, ['content', 'status']) + stringValue(item['content'], `${itemLabel} content`) + literalValue(item['status'], ['pending', 'in_progress', 'completed'], `${itemLabel} status`) + }) + return + case 'tool-workflow/agent-end': + workflowIdentity(data, label) + literalValue(data['outcome'], ['completed', 'failed', 'cancelled'], `${label} outcome`) + return + case 'tool-workflow/agent-start': + workflowIdentity(data, label) + stringValue(data['label'], `${label} label`) + if (data['phase'] !== undefined) stringValue(data['phase'], `${label} phase`) + nonEmptyString(data['childId'], `${label} childId`) + return + case 'tool-workflow/run-end': + nonEmptyString(data['runId'], `${label} runId`) + literalValue(data['stopReason'], ['completed', 'cancelled', 'error'], `${label} stopReason`) + return + case 'tool-workflow/run-start': + nonEmptyString(data['runId'], `${label} runId`) + nonEmptyString(data['name'], `${label} name`) + return + case 'tool/call': + coordinatePair(data, label) + nonEmptyString(data['callId'], `${label} callId`) + nonEmptyString(data['name'], `${label} name`) + stringValue(data['arguments'], `${label} arguments`) + return + case 'tool/code-dispatch': + case 'tool/code-dispatch-start': + nonEmptyString(data['rootCallId'], `${label} rootCallId`) + nonEmptyString(data['parentCallId'], `${label} parentCallId`) + nonEmptyString(data['subCallId'], `${label} subCallId`) + nonEmptyString(data['name'], `${label} name`) + if (event.type === 'tool/code-dispatch') { + booleanValue(data['isError'], `${label} isError`) + contentBlocksValue(data['content'], `${label} content`, version) + } + return + case 'tool/result': + coordinatePair(data, label) + messageValue(data['message'], `${label} message`, version, 'tool') + if (data['error'] !== undefined) { + const error = exactRecord(data['error'], `${label} error`, ['name', 'code']) + nonEmptyString(error['name'], `${label} error name`) + nonEmptyString(error['code'], `${label} error code`) + } + return + case 'turn/end': + countValue(data['turn'], `${label} turn`) + turnEndReasonValue(data['reason'], `${label} reason`) + return + case 'turn/start': + countValue(data['turn'], `${label} turn`) + return + case 'user/message': + messageValue(data, label, version, 'user') + return + case 'web/deepseek-search-llm-request': + nonEmptyString(data['endpoint'], `${label} endpoint`) + nonEmptyString(data['apiVersion'], `${label} apiVersion`) + deepSeekSearchBodyValue(data['body'], `${label} body`) + return + /* v8 ignore next -- the frozen disposition rejects unknown types before semantic dispatch. */ + default: + throw new SessionFormatError(`released payload validator is missing event ${JSON.stringify(event.type)}`) + } +} + +function exactRecord( + value: SessionFormatJsonValue | undefined, + label: string, + required: readonly string[], + optional: readonly string[] = [], +): JsonRecord { + const record = releasedV0Record(value, label) + assertReleasedV0Keys(record, required, optional, label) + return record +} + +function stringValue(value: SessionFormatJsonValue | undefined, label: string): asserts value is string { + if (typeof value !== 'string') throw new SessionFormatError(`${label} must be a string`) +} + +function nonEmptyString(value: SessionFormatJsonValue | undefined, label: string): asserts value is string { + if (typeof value !== 'string' || value.length === 0) throw new SessionFormatError(`${label} must be a non-empty string`) +} + +function booleanValue(value: SessionFormatJsonValue | undefined, label: string): asserts value is boolean { + if (typeof value !== 'boolean') throw new SessionFormatError(`${label} must be a boolean`) +} + +function safeIntegerValue(value: SessionFormatJsonValue | undefined, label: string): number { + return sessionFormatSafeInteger(value, label) +} + +function countValue(value: SessionFormatJsonValue | undefined, label: string): number { + return sessionFormatCount(value, label) +} + +function positiveIntegerValue(value: SessionFormatJsonValue | undefined, label: string): number { + const result = countValue(value, label) + if (result === 0) throw new SessionFormatError(`${label} must be positive`) + return result +} + +function finiteNumberValue(value: SessionFormatJsonValue | undefined, label: string): number { + if (typeof value !== 'number' || !Number.isFinite(value) || Object.is(value, -0)) { + throw new SessionFormatError(`${label} must be a finite number`) + } + return value +} + +function literalValue( + value: SessionFormatJsonValue | undefined, + allowed: readonly SessionFormatJsonValue[], + label: string, +): void { + if (!allowed.some(candidate => candidate === value)) { + throw new SessionFormatError(`${label} must be one of ${allowed.map(String).join(', ')}`) + } +} + +function nullableValue( + value: SessionFormatJsonValue | undefined, + label: string, + validate: (value: SessionFormatJsonValue | undefined, label: string) => unknown, +): void { + if (value !== null) validate(value, label) +} + +function arrayValue( + value: SessionFormatJsonValue | undefined, + label: string, + validate: (value: SessionFormatJsonValue, label: string) => void, +): readonly SessionFormatJsonValue[] { + if (!Array.isArray(value)) throw new SessionFormatError(`${label} must be an array`) + const members = value as readonly SessionFormatJsonValue[] + members.forEach((member, index) => { + validate(member, `${label}[${index}]`) + }) + return members +} + +function coordinatePair(data: JsonRecord, label: string): void { + countValue(data['turn'], `${label} turn`) + countValue(data['step'], `${label} step`) +} + +function earlierSeq(value: SessionFormatJsonValue | undefined, eventSeq: number, label: string): number { + const seq = countValue(value, label) + if (seq >= eventSeq) throw new SessionFormatError(`${label} must identify an earlier event`) + return seq +} + +function seqArray( + value: SessionFormatJsonValue | undefined, + eventSeq: number, + label: string, + requireNonEmpty: boolean, +): readonly SessionFormatJsonValue[] { + const seen = new Set() + const values = arrayValue(value, label, (member, memberLabel) => { + const seq = earlierSeq(member, eventSeq, memberLabel) + if (seen.has(seq)) throw new SessionFormatError(`${label} repeats seq ${seq}`) + seen.add(seq) + }) + if (requireNonEmpty && values.length === 0) throw new SessionFormatError(`${label} must be non-empty`) + return values +} + +function llmFailureValue(value: SessionFormatJsonValue | undefined, label: string): void { + const failure = exactRecord(value, label, ['message', 'code'], ['status', 'providerRetryAfterMs', 'requestId']) + nonEmptyString(failure['message'], `${label} message`) + nonEmptyString(failure['code'], `${label} code`) + if (failure['status'] !== undefined) { + const status = safeIntegerValue(failure['status'], `${label} status`) + if (status < 100 || status > 599) throw new SessionFormatError(`${label} status must be 100 through 599`) + } + if (failure['providerRetryAfterMs'] !== undefined + && finiteNumberValue(failure['providerRetryAfterMs'], `${label} providerRetryAfterMs`) <= 0) { + throw new SessionFormatError(`${label} providerRetryAfterMs must be positive`) + } + if (failure['requestId'] !== undefined) nonEmptyString(failure['requestId'], `${label} requestId`) +} + +function tokenUsageValue(value: SessionFormatJsonValue | undefined, label: string): void { + const usage = exactRecord( + value, + label, + ['inputTokens', 'outputTokens'], + ['totalTokens', 'cacheReadTokens', 'cacheWriteTokens', 'reasoningTokens'], + ) + for (const key of Object.keys(usage)) countValue(usage[key], `${label} ${key}`) +} + +function contentBlocksValue(value: SessionFormatJsonValue | undefined, label: string, version: 0 | 1): void { + arrayValue(value, label, (member, memberLabel) => { + contentBlockValue(member, memberLabel, version) + }) +} + +function contentBlockValue(value: SessionFormatJsonValue, label: string, version: 0 | 1): void { + const block = releasedV0Record(value, label) + switch (block['type']) { + case 'text': + case 'reasoning': + assertReleasedV0Keys(block, ['type', 'text'], [], label) + stringValue(block['text'], `${label} text`) + return + case 'image': + assertReleasedV0Keys(block, ['type', 'attachment'], [], label) + imageAttachmentValue(block['attachment'], `${label} attachment`) + return + case 'tool-call': + assertReleasedV0Keys(block, ['type', 'id', 'name', 'arguments'], [], label) + nonEmptyString(block['id'], `${label} id`) + nonEmptyString(block['name'], `${label} name`) + stringValue(block['arguments'], `${label} arguments`) + return + case 'tool-result': + assertReleasedV0Keys(block, ['type', 'toolCallId', 'content'], ['isError'], label) + nonEmptyString(block['toolCallId'], `${label} toolCallId`) + contentBlocksValue(block['content'], `${label} content`, version) + if (block['isError'] !== undefined) booleanValue(block['isError'], `${label} isError`) + return + default: + nonEmptyString(block['type'], `${label} type`) + return + } +} + +function imageAttachmentValue(value: SessionFormatJsonValue | undefined, label: string): void { + const attachment = exactRecord( + value, + label, + ['attachmentId', 'mediaType', 'bytes', 'width', 'height'], + ['name', 'originalDimensions'], + ) + nonEmptyString(attachment['attachmentId'], `${label} attachmentId`) + literalValue(attachment['mediaType'], ['image/png', 'image/jpeg', 'image/webp', 'image/gif'], `${label} mediaType`) + countValue(attachment['bytes'], `${label} bytes`) + positiveIntegerValue(attachment['width'], `${label} width`) + positiveIntegerValue(attachment['height'], `${label} height`) + if (attachment['name'] !== undefined) stringValue(attachment['name'], `${label} name`) + if (attachment['originalDimensions'] !== undefined) { + const dimensions = exactRecord(attachment['originalDimensions'], `${label} originalDimensions`, ['width', 'height']) + positiveIntegerValue(dimensions['width'], `${label} original width`) + positiveIntegerValue(dimensions['height'], `${label} original height`) + } +} + +function messageValue( + value: SessionFormatJsonValue | undefined, + label: string, + version: 0 | 1, + expected?: 'user' | 'assistant' | 'tool', +): void { + const message = exactRecord(value, label, ['id', 'role', 'content', 'source']) + nonEmptyString(message['id'], `${label} id`) + const role = expected === 'assistant' ? 'assistant' : expected === 'user' || expected === 'tool' ? 'user' : undefined + if (role === undefined) literalValue(message['role'], ['system', 'user', 'assistant'], `${label} role`) + else literalValue(message['role'], [role], `${label} role`) + contentBlocksValue(message['content'], `${label} content`, version) + messageSourceValue(message['source'], `${label} source`, version, expected) + if (expected === 'tool') { + const content = message['content'] + const block = Array.isArray(content) && content.length === 1 + ? releasedV0Record(content[0], `${label} tool result`) + : undefined + const source = releasedV0Record(message['source'], `${label} source`) + if (block?.['type'] !== 'tool-result' || block['toolCallId'] !== source['callId']) { + throw new SessionFormatError(`${label} must contain exactly one tool-result block`) + } + } +} + +function messageSourceValue( + value: SessionFormatJsonValue | undefined, + label: string, + version: 0 | 1, + expected?: 'user' | 'assistant' | 'tool', +): void { + const source = releasedV0Record(value, label) + if (expected === 'assistant' && source['kind'] !== 'model') throw new SessionFormatError(`${label} must be model source`) + if (expected === 'tool' && source['kind'] !== 'tool') throw new SessionFormatError(`${label} must be tool source`) + switch (source['kind']) { + case 'user': + assertReleasedV0Keys(source, ['kind'], ['rpcId', 'clientTimeZone'], label) + if (source['rpcId'] !== undefined) nonEmptyString(source['rpcId'], `${label} rpcId`) + if (source['clientTimeZone'] !== undefined) nonEmptyString(source['clientTimeZone'], `${label} clientTimeZone`) + return + case 'plugin': + pluginSourceValue(source, label) + return + case 'model': + assertReleasedV0Keys(source, ['kind', 'provider', 'model'], ['replayState'], label) + nonEmptyString(source['provider'], `${label} provider`) + nonEmptyString(source['model'], `${label} model`) + return + case 'tool': + assertReleasedV0Keys(source, ['kind', 'callId'], [], label) + nonEmptyString(source['callId'], `${label} callId`) + return + case 'agent-instructions': + assertReleasedV0Keys(source, ['kind', 'form', 'changes'], ['baseline', 'baselineIdentity'], label) + literalValue(source['form'], ['instructions'], `${label} form`) + if (source['baseline'] !== undefined) literalValue(source['baseline'], [true], `${label} baseline`) + if (source['baselineIdentity'] !== undefined) nonEmptyString(source['baselineIdentity'], `${label} baselineIdentity`) + arrayValue(source['changes'], `${label} changes`, (member, memberLabel) => { + const change = exactRecord(member, memberLabel, ['action', 'scope', 'path'], ['digest']) + literalValue(change['action'], ['set', 'replace', 'remove'], `${memberLabel} action`) + stringValue(change['scope'], `${memberLabel} scope`) + stringValue(change['path'], `${memberLabel} path`) + if (change['digest'] !== undefined) stringValue(change['digest'], `${memberLabel} digest`) + }) + return + case 'session-reference': + sessionReferenceSourceValue(source, label, version) + return + case 'team-message': + assertReleasedV0Keys(source, ['kind', 'teamId', 'messageId', 'senderId', 'senderName'], [], label) + for (const key of ['teamId', 'messageId', 'senderId'] as const) nonEmptyString(source[key], `${label} ${key}`) + stringValue(source['senderName'], `${label} senderName`) + return + case 'goal': + assertReleasedV0Keys(source, ['kind', 'goalId', 'revision', 'round'], [], label) + nonEmptyString(source['goalId'], `${label} goalId`) + positiveIntegerValue(source['revision'], `${label} revision`) + positiveIntegerValue(source['round'], `${label} round`) + return + case 'skill-invocation': + assertReleasedV0Keys(source, ['kind', 'name', 'form'], [], label) + nonEmptyString(source['name'], `${label} name`) + literalValue(source['form'], ['instructions'], `${label} form`) + return + case 'skill-catalog': + assertReleasedV0Keys(source, ['kind', 'form', 'entries'], ['update'], label) + literalValue(source['form'], ['catalog'], `${label} form`) + if (source['update'] !== undefined) literalValue(source['update'], [true], `${label} update`) + arrayValue(source['entries'], `${label} entries`, (member, memberLabel) => { + const entry = exactRecord(member, memberLabel, ['name', 'description']) + nonEmptyString(entry['name'], `${memberLabel} name`) + stringValue(entry['description'], `${memberLabel} description`) + }) + return + case 'coordinator': + case 'subagent-report': + assertReleasedV0Keys(source, ['kind', 'form', 'senderSessionId'], [], label) + literalValue(source['form'], ['relay'], `${label} form`) + nonEmptyString(source['senderSessionId'], `${label} senderSessionId`) + return + case 'subagent-settled': + assertReleasedV0Keys(source, ['kind', 'form', 'summary', 'senderSessionId'], [], label) + literalValue(source['form'], ['notice'], `${label} form`) + stringValue(source['summary'], `${label} summary`) + nonEmptyString(source['senderSessionId'], `${label} senderSessionId`) + return + case 'webhook': + assertReleasedV0Keys(source, ['kind', 'provider', 'source', 'deliveryId', 'ruleId', 'form', 'summary'], [], label) + for (const key of ['provider', 'source', 'deliveryId', 'ruleId'] as const) nonEmptyString(source[key], `${label} ${key}`) + literalValue(source['form'], ['notice'], `${label} form`) + stringValue(source['summary'], `${label} summary`) + return + default: + nonEmptyString(source['kind'], `${label} kind`) + return + } +} + +function pluginSourceValue(source: JsonRecord, label: string): void { + const optional = ['form', 'sections', 'summary'] + if (source['plugin'] === 'compact') optional.push('compactionId', 'sourceCommandId') + assertReleasedV0Keys(source, ['kind', 'plugin'], optional, label) + nonEmptyString(source['plugin'], `${label} plugin`) + if (source['plugin'] === 'compact') { + nonEmptyString(source['compactionId'], `${label} compactionId`) + if (source['sourceCommandId'] !== undefined) nonEmptyString(source['sourceCommandId'], `${label} sourceCommandId`) + } + const form = source['form'] + if (form === undefined) return + literalValue(form, ['instructions', 'catalog', 'snapshot', 'notice', 'relay', 'recall'], `${label} form`) + if (form === 'snapshot') { + arrayValue(source['sections'], `${label} sections`, (member, memberLabel) => { + const section = exactRecord(member, memberLabel, ['name', 'text']) + nonEmptyString(section['name'], `${memberLabel} name`) + stringValue(section['text'], `${memberLabel} text`) + }) + } else if (source['sections'] !== undefined) { + throw new SessionFormatError(`${label} sections require snapshot form`) + } + if (form === 'notice') stringValue(source['summary'], `${label} summary`) + else if (source['summary'] !== undefined) throw new SessionFormatError(`${label} summary requires notice form`) +} + +function sessionReferenceSourceValue(source: JsonRecord, label: string, version: 0 | 1): void { + assertReleasedV0Keys(source, ['kind', 'form', 'version', 'references'], [], label) + literalValue(source['form'], ['recall'], `${label} form`) + literalValue(source['version'], [1], `${label} version`) + let expectedInputIndex = 0 + const sessionIds = new Set() + const references = arrayValue(source['references'], `${label} references`, (member, memberLabel) => { + const reference = exactRecord( + member, + memberLabel, + [ + 'sessionId', 'label', 'capturedThroughSeq', 'compacted', 'originalMessages', + 'retainedMessages', 'omittedMessages', 'omittedBytes', 'truncated', 'inputIndex', + ], + version === 1 ? ['capturedFormatVersion'] : [], + ) + nonEmptyString(reference['sessionId'], `${memberLabel} sessionId`) + stringValue(reference['label'], `${memberLabel} label`) + if (reference['capturedThroughSeq'] !== null) countValue(reference['capturedThroughSeq'], `${memberLabel} capturedThroughSeq`) + if (reference['capturedFormatVersion'] !== undefined + && countValue(reference['capturedFormatVersion'], `${memberLabel} capturedFormatVersion`) !== 1) { + throw new SessionFormatError(`${memberLabel} capturedFormatVersion must be 1`) + } + booleanValue(reference['compacted'], `${memberLabel} compacted`) + const original = countValue(reference['originalMessages'], `${memberLabel} originalMessages`) + const retained = countValue(reference['retainedMessages'], `${memberLabel} retainedMessages`) + const omitted = countValue(reference['omittedMessages'], `${memberLabel} omittedMessages`) + const omittedBytes = countValue(reference['omittedBytes'], `${memberLabel} omittedBytes`) + const inputIndex = countValue(reference['inputIndex'], `${memberLabel} inputIndex`) + const truncated = reference['truncated'] + booleanValue(truncated, `${memberLabel} truncated`) + if (retained > original || omitted !== original - retained) { + throw new SessionFormatError(`${memberLabel} message counts are inconsistent`) + } + if (truncated !== (omitted > 0 || omittedBytes > 0)) { + throw new SessionFormatError(`${memberLabel} truncated disagrees with omitted content`) + } + if (inputIndex !== expectedInputIndex) throw new SessionFormatError(`${label} inputIndex must match reference position`) + expectedInputIndex += 1 + const sessionId = reference['sessionId'] + if (sessionIds.has(sessionId)) throw new SessionFormatError(`${label} repeats sessionId ${sessionId}`) + sessionIds.add(sessionId) + }) + if (references.length === 0) throw new SessionFormatError(`${label} references must be non-empty`) +} + +function streamChunkValue(value: SessionFormatJsonValue | undefined, label: string): void { + const chunk = releasedV0Record(value, label) + switch (chunk['type']) { + case 'block-start': + assertReleasedV0Keys(chunk, ['type', 'index', 'blockType'], [], label) + countValue(chunk['index'], `${label} index`) + nonEmptyString(chunk['blockType'], `${label} blockType`) + return + case 'text-delta': + case 'reasoning-delta': + assertReleasedV0Keys(chunk, ['type', 'index', 'text'], [], label) + countValue(chunk['index'], `${label} index`) + stringValue(chunk['text'], `${label} text`) + return + case 'tool-call-delta': + assertReleasedV0Keys(chunk, ['type', 'index', 'id', 'argumentsDelta'], ['name'], label) + countValue(chunk['index'], `${label} index`) + nonEmptyString(chunk['id'], `${label} id`) + if (chunk['name'] !== undefined) stringValue(chunk['name'], `${label} name`) + stringValue(chunk['argumentsDelta'], `${label} argumentsDelta`) + return + case 'block-end': + assertReleasedV0Keys(chunk, ['type', 'index', 'block'], [], label) + countValue(chunk['index'], `${label} index`) + contentBlockValue(chunk['block'] as SessionFormatJsonValue, `${label} block`, 1) + return + case 'usage': + assertReleasedV0Keys(chunk, ['type', 'usage'], [], label) + tokenUsageValue(chunk['usage'], `${label} usage`) + return + case 'finish': + assertReleasedV0Keys(chunk, ['type', 'reason'], ['replayState'], label) + finishReasonValue(chunk['reason'], `${label} reason`) + if (chunk['replayState'] !== undefined) replayEnvelopeValue(chunk['replayState'], `${label} replayState`) + return + default: + throw new SessionFormatError(`${label} has unknown stream chunk type ${JSON.stringify(chunk['type'])}`) + } +} + +function finishReasonValue(value: SessionFormatJsonValue | undefined, label: string): void { + const reason = releasedV0Record(value, label) + if (reason['kind'] === 'aborted' || reason['kind'] === 'error') { + assertReleasedV0Keys(reason, ['kind', 'failure'], [], label) + llmFailureValue(reason['failure'], `${label} failure`) + return + } + if (reason['kind'] === 'stop' || reason['kind'] === 'tool-calls' || reason['kind'] === 'max-tokens') { + assertReleasedV0Keys(reason, ['kind'], [], label) + } + nonEmptyString(reason['kind'], `${label} kind`) +} + +function replayEnvelopeValue(value: SessionFormatJsonValue | undefined, label: string): void { + const replay = exactRecord(value, label, ['response'], ['blocks']) + if (replay['blocks'] !== undefined && !Array.isArray(replay['blocks'])) { + throw new SessionFormatError(`${label} blocks must be an array`) + } +} + +function turnEndReasonValue(value: SessionFormatJsonValue | undefined, label: string): void { + const reason = releasedV0Record(value, label) + switch (reason['kind']) { + case 'completed': + case 'blocked': + case 'max-tokens': + case 'interrupted': + assertReleasedV0Keys(reason, ['kind'], [], label) + return + case 'aborted': { + assertReleasedV0Keys(reason, ['kind', 'reason'], [], label) + const cause = releasedV0Record(reason['reason'], `${label} abort cause`) + if (cause['kind'] === 'hook') { + assertReleasedV0Keys(cause, ['kind', 'reason'], [], `${label} abort cause`) + stringValue(cause['reason'], `${label} abort reason`) + } else { + assertReleasedV0Keys(cause, ['kind'], [], `${label} abort cause`) + literalValue(cause['kind'], ['user', 'parent', 'disposed', 'legacy'], `${label} abort kind`) + } + return + } + case 'error': + assertReleasedV0Keys(reason, ['kind', 'error'], [], label) + llmFailureValue(reason['error'], `${label} error`) + return + default: + nonEmptyString(reason['kind'], `${label} kind`) + return + } +} + +function requestHeaderValue(value: SessionFormatJsonValue | undefined, label: string): void { + const header = exactRecord(value, label, ['config'], ['adapterDefaults', 'system', 'tools']) + const config = exactRecord( + header['config'], + `${label} config`, + ['provider', 'model'], + ['reasoningEffort', 'temperature', 'maxTokens', 'stop'], + ) + nonEmptyString(config['provider'], `${label} provider`) + nonEmptyString(config['model'], `${label} model`) + if (config['reasoningEffort'] !== undefined) nonEmptyString(config['reasoningEffort'], `${label} reasoningEffort`) + if (config['temperature'] !== undefined) finiteNumberValue(config['temperature'], `${label} temperature`) + if (config['maxTokens'] !== undefined) positiveIntegerValue(config['maxTokens'], `${label} maxTokens`) + if (config['stop'] !== undefined) arrayValue(config['stop'], `${label} stop`, stringValue) + if (header['adapterDefaults'] !== undefined) { + const defaults = exactRecord(header['adapterDefaults'], `${label} adapterDefaults`, [], ['reasoningEffort', 'maxTokens']) + for (const [key, marker] of Object.entries(defaults)) { + literalValue(marker, [true], `${label} adapterDefaults ${key}`) + if (!Object.hasOwn(config, key)) throw new SessionFormatError(`${label} adapter default ${key} lacks config value`) + } + } + if (header['system'] !== undefined) stringValue(header['system'], `${label} system`) + if (header['tools'] !== undefined) arrayValue(header['tools'], `${label} tools`, toolSchemaValue) +} + +function toolSchemaValue(value: SessionFormatJsonValue, label: string): void { + const schema = exactRecord(value, label, ['name', 'description', 'parameters']) + nonEmptyString(schema['name'], `${label} name`) + stringValue(schema['description'], `${label} description`) + releasedV0Record(schema['parameters'], `${label} parameters`) +} + +function shadowedValue(data: JsonRecord, eventSeq: number, label: string): void { + const range = exactRecord(data['shadowedRange'], `${label} shadowedRange`, ['start', 'end']) + const start = earlierSeq(range['start'], eventSeq, `${label} shadowedRange start`) + const end = earlierSeq(range['end'], eventSeq, `${label} shadowedRange end`) + if (start > end) throw new SessionFormatError(`${label} shadowedRange is inverted`) + const seqs = seqArray(data['shadowedSeqs'], eventSeq, `${label} shadowedSeqs`, true) + if (seqs[0] !== start || seqs.at(-1) !== end) { + throw new SessionFormatError(`${label} shadowedRange must match shadowedSeqs endpoints`) + } + countValue(data['shadowedTokenCount'], `${label} shadowedTokenCount`) +} + +function goalChangeValue(data: JsonRecord, label: string): void { + literalValue(data['kind'], ['goal/change'], `${label} kind`) + literalValue(data['version'], [1], `${label} version`) + if (data['operation'] === 'clear') { + assertReleasedV0Keys(data, ['kind', 'version', 'operation', 'cleared', 'clearedAt'], [], `${label} data`) + goalRefValue(data['cleared'], `${label} cleared`) + countValue(data['clearedAt'], `${label} clearedAt`) + return + } + assertReleasedV0Keys( + data, + ['kind', 'version', 'operation', 'goal', 'roundsStarted', 'createdAt', 'updatedAt'], + [], + `${label} data`, + ) + literalValue(data['operation'], ['create', 'edit', 'pause', 'resume', 'complete', 'block'], `${label} operation`) + goalSnapshotValue(data['goal'], `${label} goal`) + countValue(data['roundsStarted'], `${label} roundsStarted`) + countValue(data['createdAt'], `${label} createdAt`) + countValue(data['updatedAt'], `${label} updatedAt`) +} + +function goalRefValue(value: SessionFormatJsonValue | undefined, label: string): void { + const ref = exactRecord(value, label, ['id', 'revision']) + nonEmptyString(ref['id'], `${label} id`) + positiveIntegerValue(ref['revision'], `${label} revision`) +} + +function goalSnapshotValue(value: SessionFormatJsonValue | undefined, label: string): void { + const goal = exactRecord(value, label, ['id', 'revision', 'objective', 'phase', 'maxGoalRounds'], ['blockedReason']) + nonEmptyString(goal['id'], `${label} id`) + positiveIntegerValue(goal['revision'], `${label} revision`) + nonEmptyString(goal['objective'], `${label} objective`) + literalValue(goal['phase'], ['active', 'paused', 'blocked', 'complete'], `${label} phase`) + positiveIntegerValue(goal['maxGoalRounds'], `${label} maxGoalRounds`) + if (goal['phase'] === 'blocked') { + const reason = exactRecord(goal['blockedReason'], `${label} blockedReason`, ['code', 'message']) + nonEmptyString(reason['code'], `${label} blocked code`) + nonEmptyString(reason['message'], `${label} blocked message`) + } else if (goal['blockedReason'] !== undefined) { + throw new SessionFormatError(`${label} blockedReason requires blocked phase`) + } +} + +function scheduleChangeValue(data: JsonRecord, label: string): void { + literalValue(data['version'], [1], `${label} version`) + if (data['operation'] === 'create') { + assertReleasedV0Keys(data, ['version', 'operation', 'schedule'], [], `${label} data`) + scheduleRecordValue(data['schedule'], `${label} schedule`) + return + } + assertReleasedV0Keys( + data, + ['version', 'operation', 'id'], + data['operation'] === 'dispatch' ? ['acceptedAt'] : [], + `${label} data`, + ) + literalValue(data['operation'], ['delete', 'dispatch'], `${label} operation`) + scheduleIdValue(data['id'], `${label} id`) + if (data['acceptedAt'] !== undefined) instantValue(data['acceptedAt'], `${label} acceptedAt`) +} + +function scheduleRecordValue(value: SessionFormatJsonValue | undefined, label: string): void { + const record = releasedV0Record(value, label) + if (record['kind'] === 'after') { + assertReleasedV0Keys(record, ['id', 'kind', 'prompt', 'afterSeconds', 'scheduledAt'], [], label) + positiveIntegerValue(record['afterSeconds'], `${label} afterSeconds`) + } else if (record['kind'] === 'at') { + assertReleasedV0Keys(record, ['id', 'kind', 'prompt', 'scheduledAt'], [], label) + } else if (record['kind'] === 'every') { + assertReleasedV0Keys(record, ['id', 'kind', 'prompt', 'everySeconds', 'scheduledAt'], [], label) + const seconds = positiveIntegerValue(record['everySeconds'], `${label} everySeconds`) + if (seconds < 300) throw new SessionFormatError(`${label} everySeconds must be at least 300`) + } else { + throw new SessionFormatError(`${label} has unknown schedule kind`) + } + scheduleIdValue(record['id'], `${label} id`) + nonEmptyString(record['prompt'], `${label} prompt`) + instantValue(record['scheduledAt'], `${label} scheduledAt`) +} + +function scheduleIdValue(value: SessionFormatJsonValue | undefined, label: string): void { + nonEmptyString(value, label) + if (value.trim() !== value) throw new SessionFormatError(`${label} must not have surrounding whitespace`) +} + +function instantValue(value: SessionFormatJsonValue | undefined, label: string): void { + if (typeof value !== 'string' + || !/^(?!0000)\d{4}-(?:0[1-9]|1[0-2])-(?:0[1-9]|[12]\d|3[01])T(?:[01]\d|2[0-3]):[0-5]\d:[0-5]\d\.\d{3}Z$/u.test(value) + || !Number.isFinite(Date.parse(value)) + || new Date(Date.parse(value)).toISOString() !== value) { + throw new SessionFormatError(`${label} must be a canonical UTC instant`) + } +} + +function titleSourceValue(value: SessionFormatJsonValue | undefined, label: string): void { + const source = releasedV0Record(value, label) + if (source['kind'] === 'provider') { + assertReleasedV0Keys(source, ['kind', 'provider'], ['model'], label) + nonEmptyString(source['provider'], `${label} provider`) + if (source['model'] !== undefined) modelRouteValue(source['model'], `${label} model`) + return + } + assertReleasedV0Keys(source, ['kind'], [], label) + literalValue(source['kind'], ['fallback', 'user'], `${label} kind`) +} + +function modelRouteValue(value: SessionFormatJsonValue | undefined, label: string): void { + const route = exactRecord(value, label, ['provider', 'model']) + nonEmptyString(route['provider'], `${label} provider`) + nonEmptyString(route['model'], `${label} model`) +} + +function subagentDescriptorValue(data: JsonRecord, label: string): void { + literalValue(data['version'], [3], `${label} version`) + nonEmptyString(data['provider'], `${label} provider`) + if (data['mode'] === 'one-shot') { + assertReleasedV0Keys(data, ['mode', 'version', 'provider'], ['label'], `${label} data`) + if (data['label'] !== undefined) stringValue(data['label'], `${label} label`) + return + } + literalValue(data['mode'], ['continuable'], `${label} mode`) + nonEmptyString(data['label'], `${label} label`) + for (const key of ['agentProvider', 'agentModel', 'agentReasoningEffort', 'persona'] as const) { + if (data[key] !== undefined) nonEmptyString(data[key], `${label} ${key}`) + } + if ((data['agentProvider'] === undefined) !== (data['agentModel'] === undefined)) { + throw new SessionFormatError(`${label} agentProvider and agentModel must be paired`) + } + if (data['toolFilter'] !== undefined) { + const filter = exactRecord(data['toolFilter'], `${label} toolFilter`, [], ['allow', 'deny']) + if (filter['allow'] === undefined && filter['deny'] === undefined) { + throw new SessionFormatError(`${label} toolFilter requires allow or deny`) + } + if (filter['allow'] !== undefined) arrayValue(filter['allow'], `${label} allow`, nonEmptyString) + if (filter['deny'] !== undefined) arrayValue(filter['deny'], `${label} deny`, nonEmptyString) + } +} + +function allowedModelsValue(value: SessionFormatJsonValue | undefined, label: string): void { + const seen = new Set() + const routes = arrayValue(value, label, (member, memberLabel) => { + const route = exactRecord(member, memberLabel, ['provider', 'model']) + nonEmptyString(route['provider'], `${memberLabel} provider`) + nonEmptyString(route['model'], `${memberLabel} model`) + const key = `${route['provider']}\0${route['model']}` + if (seen.has(key)) throw new SessionFormatError(`${label} repeats route ${key}`) + seen.add(key) + }) + if (routes.length === 0) throw new SessionFormatError(`${label} must be non-empty`) +} + +function teamSelector(data: JsonRecord, label: string): void { + literalValue(data['version'], [1], `${label} version`) + nonEmptyString(data['teamId'], `${label} teamId`) +} + +function teamMemberValue(value: SessionFormatJsonValue | undefined, label: string): void { + const member = exactRecord(value, label, ['id', 'name', 'description', 'provider', 'context', 'phase'], ['error']) + nonEmptyString(member['id'], `${label} id`) + stringValue(member['name'], `${label} name`) + stringValue(member['description'], `${label} description`) + stringValue(member['provider'], `${label} provider`) + literalValue(member['context'], ['fresh', 'fork'], `${label} context`) + literalValue(member['phase'], ['provisioning', 'active', 'failed'], `${label} phase`) + if (member['error'] !== undefined) stringValue(member['error'], `${label} error`) +} + +function teamTaskValue(value: SessionFormatJsonValue | undefined, label: string): void { + const task = exactRecord( + value, + label, + ['id', 'revision', 'subject', 'description', 'status', 'blockedBy', 'writeScopes'], + ['ownerId'], + ) + nonEmptyString(task['id'], `${label} id`) + positiveIntegerValue(task['revision'], `${label} revision`) + stringValue(task['subject'], `${label} subject`) + stringValue(task['description'], `${label} description`) + literalValue(task['status'], ['pending', 'in_progress', 'completed', 'deleted'], `${label} status`) + if (task['ownerId'] !== undefined) nonEmptyString(task['ownerId'], `${label} ownerId`) + arrayValue(task['blockedBy'], `${label} blockedBy`, nonEmptyString) + arrayValue(task['writeScopes'], `${label} writeScopes`, stringValue) +} + +function teamMessageValue(value: SessionFormatJsonValue | undefined, label: string, version: 0 | 1): void { + const message = exactRecord(value, label, ['id', 'senderId', 'senderName', 'targetId', 'delivery', 'content']) + for (const key of ['id', 'senderId', 'targetId'] as const) nonEmptyString(message[key], `${label} ${key}`) + stringValue(message['senderName'], `${label} senderName`) + literalValue(message['delivery'], ['quiet', 'wakeup'], `${label} delivery`) + contentBlocksValue(message['content'], `${label} content`, version) +} + +function workflowIdentity(data: JsonRecord, label: string): void { + nonEmptyString(data['runId'], `${label} runId`) + positiveIntegerValue(data['seq'], `${label} seq`) +} + +function deepSeekSearchBodyValue(value: SessionFormatJsonValue | undefined, label: string): void { + const body = exactRecord(value, label, ['model', 'max_tokens', 'messages', 'tools']) + nonEmptyString(body['model'], `${label} model`) + positiveIntegerValue(body['max_tokens'], `${label} max_tokens`) + const messages = arrayValue(body['messages'], `${label} messages`, (member, memberLabel) => { + const message = exactRecord(member, memberLabel, ['role', 'content']) + literalValue(message['role'], ['user'], `${memberLabel} role`) + const content = arrayValue(message['content'], `${memberLabel} content`, (block, blockLabel) => { + const text = exactRecord(block, blockLabel, ['type', 'text']) + literalValue(text['type'], ['text'], `${blockLabel} type`) + stringValue(text['text'], `${blockLabel} text`) + }) + if (content.length !== 1) throw new SessionFormatError(`${memberLabel} content must contain one text block`) + }) + if (messages.length !== 1) throw new SessionFormatError(`${label} messages must contain one user message`) + const tools = arrayValue(body['tools'], `${label} tools`, (member, memberLabel) => { + const tool = exactRecord(member, memberLabel, ['type', 'name', 'max_uses']) + literalValue(tool['type'], ['web_search_20250305'], `${memberLabel} type`) + literalValue(tool['name'], ['web_search'], `${memberLabel} name`) + positiveIntegerValue(tool['max_uses'], `${memberLabel} max_uses`) + }) + if (tools.length !== 1) throw new SessionFormatError(`${label} tools must contain one web search tool`) +} diff --git a/packages/session/session-format-v0-to-v1/src/relationships.ts b/packages/session/session-format-v0-to-v1/src/relationships.ts new file mode 100644 index 0000000000..e220924840 --- /dev/null +++ b/packages/session/session-format-v0-to-v1/src/relationships.ts @@ -0,0 +1,470 @@ +import { SessionFormatError } from '@deepseek-ai/dsh-session-format' +import type { SessionFormatArtifact, SessionFormatEvent, SessionFormatJsonValue } from '@deepseek-ai/dsh-session-format' +import { releasedV0Record } from './validation-helpers.ts' +import { RELEASED_V0_EVENT_DISPOSITIONS } from './dispositions.ts' + +const SURFACE_TYPES = new Set(['user/message', 'assistant/message', 'tool/result']) + +interface CompactionState { + readonly id: string + readonly sourceCommandId?: string + readonly turn: number | null + readonly startSeq: number + readonly summarized: boolean +} + +interface PtcStart { + readonly root: string + readonly parent: string + readonly name: string + readonly arguments: SessionFormatJsonValue + settled: boolean +} + +interface ToolLifecycle { + readonly name: string + readonly arguments: string + state: 'advertised' | 'started' +} + +/** + * Validate cross-event relationships required to construct one current Session safely. + * @param artifact - complete normalized v0 or exact current v1 artifact. + */ +export function assertReleasedArtifactRelationships(artifact: SessionFormatArtifact): void { + let openTurn: number | null = null + let openStep: number | null = null + let openStepProvider: string | undefined + let nextTurn = 1 + let nextStep = 1 + let surface: number[] = [] + let openCompaction: CompactionState | undefined + const staleCompactionStarts = inheritedOrphanCompactionStarts(artifact.events) + const retries: SessionFormatEvent[] = [] + const retryStarts = new Set() + const ptcRoots = new Map() + const ptcStarts = new Map() + const toolLifecycles = new Map() + const commandRuns = new Set() + + for (const event of artifact.events) { + if (RELEASED_V0_EVENT_DISPOSITIONS[event.type] === undefined) continue + const data = releasedV0Record(event.data, `${event.type} ${event.seq} data`) + if (SURFACE_TYPES.has(event.type)) surface = applySurface(surface, event) + if ((event.type === 'turn/start' || event.type === 'turn/end') + && openCompaction !== undefined && !staleCompactionStarts.has(openCompaction.startSeq)) { + throw new SessionFormatError(`${event.type} crosses an open compaction`) + } + + switch (event.type) { + case 'turn/start': + if (openTurn !== null || data['turn'] !== nextTurn) { + throw new SessionFormatError(`turn/start ${JSON.stringify(data['turn'])} does not open expected turn ${nextTurn}`) + } + openTurn = data['turn'] + openStep = null + openStepProvider = undefined + toolLifecycles.clear() + nextStep = 1 + break + case 'turn/end': + if (openTurn !== data['turn']) { + throw new SessionFormatError(`turn/end ${JSON.stringify(data['turn'])} has no matching open turn`) + } + assertNoUnresolvedTools(toolLifecycles, 'turn/end') + if (openStep !== null) { + throw new SessionFormatError(`turn/end ${JSON.stringify(data['turn'])} crosses an open step`) + } + openTurn = null + nextTurn += 1 + break + case 'step/start': + if (openTurn !== data['turn'] || openStep !== null || data['step'] !== nextStep) { + throw new SessionFormatError(`${event.type} does not match the open turn and next step`) + } + openStep = data['step'] + openStepProvider = undefined + break + case 'step/end': + requireOpenStep(event, data, openTurn, openStep) + assertNoUnresolvedTools(toolLifecycles, 'step/end') + toolLifecycles.clear() + openStep = null + openStepProvider = undefined + nextStep += 1 + break + case 'assistant/chunk': + requireOpenStep(event, data, openTurn, openStep) + break + case 'assistant/message': { + requireOpenStep(event, data, openTurn, openStep) + const message = releasedV0Record(data['message'], `assistant/message ${event.seq} message`) + const content = message['content'] as readonly Record[] + for (const block of content) { + if (block['type'] !== 'tool-call') continue + const callId = block['id'] as string + if (toolLifecycles.has(callId)) { + throw new SessionFormatError(`assistant/message repeats advertised tool call ${callId}`) + } + toolLifecycles.set(callId, { + name: block['name'] as string, + arguments: block['arguments'] as string, + state: 'advertised', + }) + } + break + } + case 'tool/call': { + requireOpenStep(event, data, openTurn, openStep) + const callId = data['callId'] as string + const lifecycle = toolLifecycles.get(callId) + if (lifecycle === undefined || lifecycle.state !== 'advertised' + || lifecycle.name !== data['name'] || lifecycle.arguments !== data['arguments']) { + throw new SessionFormatError(`tool/call ${callId} does not match one advertised tool call`) + } + lifecycle.state = 'started' + break + } + case 'tool/result': + if (event['surfaceOp'] === 'append') { + requireOpenStep(event, data, openTurn, openStep) + const message = releasedV0Record(data['message'], `tool/result ${event.seq} message`) + const source = releasedV0Record(message['source'], `tool/result ${event.seq} source`) + const callId = source['callId'] as string + const content = message['content'] as readonly Record[] + const error = data['error'] === undefined ? undefined : releasedV0Record(data['error'], `tool/result ${event.seq} error`) + const lifecycle = toolLifecycles.get(callId) + if (lifecycle === undefined) { + throw new SessionFormatError(`tool/result ${callId} has no advertised tool lifecycle`) + } + if (lifecycle.state === 'advertised' && !isExactToolNotStartedRepair(event, content, error)) { + throw new SessionFormatError(`tool/result ${callId} is not the exact TOOL_NOT_STARTED repair`) + } + toolLifecycles.delete(callId) + } else if (openTurn === null) { + throw new SessionFormatError('tool/result replacement is outside an open turn') + } + break + case 'request/header': + if (openTurn === null) throw new SessionFormatError(`${event.type} is outside an open turn`) + openStepProvider = (((data['header'] as Record)['config'] as Record)['provider']) as string + break + case 'request/context': + if (openTurn === null) throw new SessionFormatError(`${event.type} is outside an open turn`) + break + case 'tool/code-dispatch-start': + case 'tool/code-dispatch': { + if (openTurn === null) throw new SessionFormatError(`${event.type} is outside an open turn`) + const root = data['rootCallId'] as string + const parent = data['parentCallId'] as string + const child = data['subCallId'] as string + const known = ptcRoots.get(child) + if (known !== undefined && known !== root) throw new SessionFormatError(`${event.type} changes its rootCallId`) + if (parent !== root && ptcRoots.get(parent) !== root) { + throw new SessionFormatError(`${event.type} parentCallId does not belong to rootCallId`) + } + if (event.type === 'tool/code-dispatch-start') { + if (ptcStarts.has(child)) throw new SessionFormatError('tool/code-dispatch-start repeats subCallId') + ptcStarts.set(child, { + root, + parent, + name: data['name'] as string, + arguments: data['arguments'] as SessionFormatJsonValue, + settled: false, + }) + } else { + const start = ptcStarts.get(child) + if (start === undefined || start.settled) throw new SessionFormatError('tool/code-dispatch has no unique start') + if (start.root !== root || start.parent !== parent || start.name !== data['name'] + || !sameJson(start.arguments, data['arguments'] as SessionFormatJsonValue)) { + throw new SessionFormatError('tool/code-dispatch does not match its start') + } + start.settled = true + } + ptcRoots.set(child, root) + break + } + case 'llm/retry': + requireOpenStep(event, data, openTurn, openStep) + if (data['provider'] !== openStepProvider) { + throw new SessionFormatError('llm/retry provider does not match the open request/header') + } + assertRetryChain(retries, data) + retries.push(event) + break + case 'llm/retry-started': { + const scheduled = retries.find((candidate) => { + const prior = candidate.data as Record + return prior['retryId'] === data['retryId'] && prior['retry'] === data['retry'] + }) + if (scheduled === undefined) throw new SessionFormatError('llm/retry-started pairs no prior scheduled attempt') + const prior = scheduled.data as Record + if (prior['turn'] !== data['turn'] || prior['step'] !== data['step']) { + throw new SessionFormatError('llm/retry-started does not match its scheduled turn and step') + } + const key = `${JSON.stringify(data['retryId'])}\0${JSON.stringify(data['retry'])}` + if (retryStarts.has(key)) throw new SessionFormatError('llm/retry-started repeats one scheduled attempt') + retryStarts.add(key) + break + } + case 'session/title': + case 'session/title-llm-request': + assertTitleSources(artifact.events, event, data) + break + case 'command/run': { + const id = data['commandId'] as string + if (commandRuns.has(id)) throw new SessionFormatError(`command/run repeats commandId ${id}`) + commandRuns.add(id) + break + } + case 'command/done': { + const id = data['commandId'] as string + if (!commandRuns.has(id)) throw new SessionFormatError(`command/done ${id} has no prior command/run`) + const sourceSeq = data['sourceEventSeq'] + if (sourceSeq !== undefined) { + const source = artifact.events[sourceSeq as number] + if (data['kind'] !== 'success' || source?.type === 'command/run' || source?.type === 'command/done') { + throw new SessionFormatError(`command/done ${id} has invalid sourceEventSeq`) + } + } + break + } + case 'session-log-deepseek/delivery-accepted': { + const acceptedVersion = data['sessionFormatVersion'] ?? 0 + if (acceptedVersion === artifact.header.version) { + const inherited = artifact.header.parentSession !== undefined && event.seq < artifact.inheritedEventCount + if (!inherited && data['sessionId'] !== artifact.header.id) { + throw new SessionFormatError('current-generation delivery marker names the wrong Session') + } + } + break + } + case 'compaction/start': + if (openCompaction !== undefined) throw new SessionFormatError('compaction/start overlaps an open compaction') + assertCompactionTurn(data['turn'] as number | null, openTurn, 'compaction/start') + openCompaction = { + id: data['compactionId'] as string, + ...(data['sourceCommandId'] === undefined ? {} : { sourceCommandId: data['sourceCommandId'] as string }), + turn: data['turn'] as number | null, + startSeq: event.seq, + summarized: false, + } + break + case 'compaction/summary': + assertCompactionOwner(openCompaction, data, 'compaction/summary') + assertCompactionTurn(openCompaction?.turn as number | null, openTurn, 'compaction/summary') + if (openCompaction?.summarized === true) throw new SessionFormatError('compaction/summary repeats') + assertCurrentSurfaceSpan(surface, data, 'compaction/summary') + openCompaction = { ...(openCompaction as CompactionState), summarized: true } + break + case 'compaction/end': + assertCompactionOwner(openCompaction, data, 'compaction/end') + if (data['turn'] !== openCompaction?.turn) throw new SessionFormatError('compaction/end changes its owner turn') + assertCompactionTurn(openCompaction?.turn as number | null, openTurn, 'compaction/end') + if (data['error'] === undefined && openCompaction?.summarized !== true) { + throw new SessionFormatError('successful compaction/end requires one summary') + } + openCompaction = undefined + break + case 'compaction/prune': + assertCurrentSurfaceSpan(surface, data, 'compaction/prune') + break + case 'user/message': { + const source = releasedV0Record(data['source'], `user/message ${event.seq} source`) + if (event['surfaceOp'] !== 'append' && source['kind'] === 'plugin' && source['plugin'] === 'compact') { + assertCompactionOwner(openCompaction, source, `compaction checkpoint at seq ${event.seq}`) + } + break + } + case 'session/end-seed': + // An unmatched inherited transaction belongs to the ended source lifecycle. + openCompaction = undefined + break + } + } +} + +function inheritedOrphanCompactionStarts(events: readonly SessionFormatEvent[]): ReadonlySet { + const stale = new Set() + let open: number | undefined + for (const event of events) { + if (event.type === 'compaction/start') open = event.seq + else if (event.type === 'compaction/end') open = undefined + else if (event.type === 'session/end-seed') { + if (open !== undefined) stale.add(open) + open = undefined + } + } + return stale +} + +function assertRetryChain( + retries: readonly SessionFormatEvent[], + data: Record, +): void { + const prior = [...retries].reverse().find((candidate) => { + const value = candidate.data as Record + return value['turn'] === data['turn'] && value['step'] === data['step'] + && value['provider'] === data['provider'] && value['policyKey'] === data['policyKey'] + }) + const expected = ((prior?.data as Record | undefined)?.['retry'] as number | undefined ?? 0) + 1 + if (data['retry'] !== expected) throw new SessionFormatError(`llm/retry must use retry ${expected}`) + if (prior !== undefined + && (prior.data as Record)['retryId'] !== data['retryId']) { + throw new SessionFormatError('llm/retry must preserve retryId across one policy chain') + } + if (prior === undefined && retries.some(candidate => + (candidate.data as Record)['retryId'] === data['retryId'])) { + throw new SessionFormatError(`llm/retry reuses retryId ${JSON.stringify(data['retryId'])} across policy chains`) + } +} + +function sameJson(left: SessionFormatJsonValue, right: SessionFormatJsonValue): boolean { + if (left === right) return true + if (Array.isArray(left) || Array.isArray(right)) { + if (!Array.isArray(left) || !Array.isArray(right) || left.length !== right.length) return false + const leftArray = left as readonly SessionFormatJsonValue[] + const rightArray = right as readonly SessionFormatJsonValue[] + return leftArray.every((value, index) => sameJson(value, rightArray[index] as SessionFormatJsonValue)) + } + if (typeof left !== 'object' || left === null || typeof right !== 'object' || right === null) return false + const leftKeys = Object.keys(left) + const leftRecord = left as Record + const rightRecord = right as Record + return leftKeys.length === Object.keys(right).length + && leftKeys.every(key => Object.hasOwn(right, key) + && sameJson(leftRecord[key] as SessionFormatJsonValue, rightRecord[key] as SessionFormatJsonValue)) +} + +function requireOpenStep( + event: SessionFormatEvent, + data: Record, + openTurn: number | null, + openStep: number | null, +): void { + if (data['turn'] !== openTurn || data['step'] !== openStep || openTurn === null || openStep === null) { + throw new SessionFormatError(`${event.type} does not match an open turn and step`) + } +} + +function assertNoUnresolvedTools(lifecycles: ReadonlyMap, boundary: string): void { + const unresolved = lifecycles.keys().next().value + if (unresolved !== undefined) { + throw new SessionFormatError(`${boundary} leaves unresolved tool call ${unresolved}`) + } +} + +function isExactToolNotStartedRepair( + event: SessionFormatEvent, + content: readonly Record[], + error: Record | undefined, +): boolean { + const data = event.data as Record + const message = data['message'] as Record + const source = message['source'] as Record + const callId = source['callId'] as string + const block = content[0] + const repairContent = block?.['content'] as readonly Record[] | undefined + return error?.['name'] === 'ToolNotStartedError' + && error['code'] === 'TOOL_NOT_STARTED' + && event['sourceEventSeqs'] === undefined + && message['id'] === `interrupted-tool-result-${callId}-${event.seq}` + && block?.['isError'] === true + && repairContent?.length === 1 + && repairContent[0]?.['type'] === 'text' + && repairContent[0]['text'] + === 'The tool call was interrupted before the Harness recorded it as started. Retry it if it is still needed.' +} + +function applySurface(surface: readonly number[], event: SessionFormatEvent): number[] { + const operation = event['surfaceOp'] + if (operation === undefined) throw new SessionFormatError(`${event.type} requires a surfaceOp marker`) + if (operation === 'append') return [...surface, event.seq] + const replace = operation as { readonly start: number; readonly end: number } + const start = surface.indexOf(replace.start) + const end = surface.indexOf(replace.end) + if (start < 0 || end < start) throw new SessionFormatError(`${event.type} replacement range is not on the current surface`) + const shadowed = surface.slice(start, end + 1) + const sources = new Set(Array.isArray(event['sourceEventSeqs']) ? event['sourceEventSeqs'] as readonly number[] : []) + if (shadowed.some(seq => !sources.has(seq))) { + throw new SessionFormatError(`${event.type} replacement sourceEventSeqs omit a shadowed surface node`) + } + return [...surface.slice(0, start), event.seq, ...surface.slice(end + 1)] +} + +function assertTitleSources( + events: readonly SessionFormatEvent[], + event: SessionFormatEvent, + data: Record, +): void { + const seqs = data['messageSeqs'] as readonly number[] + if (event.type === 'session/title') { + const titleSource = releasedV0Record(data['source'], `session/title ${event.seq} source`) + if ((seqs.length === 0) !== (titleSource['kind'] === 'user')) { + throw new SessionFormatError(`session/title ${event.seq} messageSeqs must be empty exactly for a user title`) + } + } + const selected: Array<{ readonly seq: number; readonly text: string }> = [] + for (const seq of seqs) { + const source = events[seq] + if (source?.type !== 'user/message') { + throw new SessionFormatError(`${event.type} ${event.seq} messageSeqs must cite earlier human user/message events`) + } + const sourceData = releasedV0Record(source.data, `${source.type} ${seq} data`) + const provenance = releasedV0Record(sourceData['source'], `${source.type} ${seq} source`) + if (provenance['kind'] !== 'user') { + throw new SessionFormatError(`${event.type} ${event.seq} messageSeqs must cite earlier human user/message events`) + } + const content = sourceData['content'] as readonly Record[] + selected.push({ + seq, + text: content.flatMap(block => block['type'] === 'text' && typeof block['text'] === 'string' ? [block['text']] : []).join('\n'), + }) + } + if (event.type === 'session/title-llm-request') { + const messages = data['messages'] as readonly Record[] + const expected = `Generate the session title from this JSON array of human messages:\n${JSON.stringify(selected)}` + const message = messages[0] + const content = message?.['content'] as readonly Record[] | undefined + const source = message === undefined ? undefined : releasedV0Record(message['source'], 'session/title-llm-request message source') + if (messages.length !== 1 || message?.['role'] !== 'user' || content?.length !== 1 + || source?.['kind'] !== 'plugin' || source['plugin'] !== 'dsh-session-title-llm') { + throw new SessionFormatError('session/title-llm-request messages do not represent messageSeqs') + } + const framed = content[0] + if (framed === undefined || framed['type'] !== 'text' || framed['text'] !== expected) { + throw new SessionFormatError('session/title-llm-request messages do not represent messageSeqs') + } + } +} + +function assertCompactionOwner( + open: CompactionState | undefined, + data: Record, + type: string, +): void { + if (open === undefined || data['compactionId'] !== open.id || data['sourceCommandId'] !== open.sourceCommandId) { + throw new SessionFormatError(`${type} has no matching compaction/start`) + } +} + +function assertCompactionTurn(owner: number | null, openTurn: number | null, type: string): void { + if (owner === null ? openTurn !== null : owner !== openTurn) { + throw new SessionFormatError(`${type} does not match the open turn`) + } +} + +function assertCurrentSurfaceSpan( + surface: readonly number[], + data: Record, + type: string, +): void { + const range = data['shadowedRange'] as { readonly start: number; readonly end: number } + const seqs = data['shadowedSeqs'] as readonly number[] + const start = surface.indexOf(range.start) + const end = surface.indexOf(range.end) + const expected = start < 0 || end < start ? [] : surface.slice(start, end + 1) + if (expected.length !== seqs.length || expected.some((seq, index) => seq !== seqs[index])) { + throw new SessionFormatError(`${type} shadowedSeqs do not name an exact current surface span`) + } +} diff --git a/packages/session/session-format-v0-to-v1/src/validation-helpers.ts b/packages/session/session-format-v0-to-v1/src/validation-helpers.ts new file mode 100644 index 0000000000..8122af4b9a --- /dev/null +++ b/packages/session/session-format-v0-to-v1/src/validation-helpers.ts @@ -0,0 +1,33 @@ +import { SessionFormatError, isSessionFormatJsonObject } from '@deepseek-ai/dsh-session-format' +import type { SessionFormatJsonValue } from '@deepseek-ai/dsh-session-format' + +/** + * Require one plain JSON object. + * @param value - candidate JSON value. + * @param label - diagnostic subject. + * @returns validated object record. + */ +export function releasedV0Record(value: unknown, label: string): Record { + if (!isSessionFormatJsonObject(value)) throw new SessionFormatError(`${label} must be a JSON object`) + return value as Record +} + +/** + * Require every named member and no member outside the optional list. + * @param record - candidate object. + * @param required - members that must exist. + * @param optional - additional admitted members. + * @param label - diagnostic subject. + */ +export function assertReleasedV0Keys( + record: Readonly>, + required: readonly string[], + optional: readonly string[] = [], + label: string, +): void { + const allowed = new Set([...required, ...optional]) + const unexpected = Object.keys(record).find(key => !allowed.has(key)) + if (unexpected !== undefined) throw new SessionFormatError(`${label} has unexpected member ${JSON.stringify(unexpected)}`) + const missing = required.find(key => !Object.hasOwn(record, key)) + if (missing !== undefined) throw new SessionFormatError(`${label} lacks required member ${JSON.stringify(missing)}`) +} diff --git a/packages/session/session-format-v0-to-v1/src/validation.ts b/packages/session/session-format-v0-to-v1/src/validation.ts new file mode 100644 index 0000000000..123653e6b1 --- /dev/null +++ b/packages/session/session-format-v0-to-v1/src/validation.ts @@ -0,0 +1,228 @@ +import { isAbsolute } from 'node:path' +import { + SessionFormatError, + SessionFormatUnsupportedMigrationError, + sessionFormatCount, + sessionFormatSafeInteger, + snapshotSessionFormatJson, +} from '@deepseek-ai/dsh-session-format' +import type { + SessionFormatArtifact, + SessionFormatEvent, + SessionFormatHeader, + SessionFormatJsonValue, +} from '@deepseek-ai/dsh-session-format' +import { RELEASED_V0_EVENT_DISPOSITIONS } from './dispositions.ts' +import { assertReleasedPayloadSemantics } from './payload-validation.ts' +import { assertReleasedArtifactRelationships } from './relationships.ts' +import { assertReleasedV0Keys, releasedV0Record } from './validation-helpers.ts' + +const HEADER_REQUIRED = ['version', 'id', 'createdAt', 'isSeeded', 'delegationDepth'] as const +const HEADER_OPTIONAL = ['cwd', 'parentSession', 'origin', 'agentPreset'] as const +const EVENT_REQUIRED = ['type', 'seq', 'time', 'data'] as const +const SURFACE_EVENT_TYPES = new Set(['user/message', 'assistant/message', 'tool/result']) +const SURFACE_OPTIONAL = ['ignorable', 'sourceEventSeqs', 'surfaceOp'] as const +const LOG_OPTIONAL = ['ignorable'] as const +const LEGACY_SOURCE_TYPES = new Set(['steering/message', 'request/header-delta', 'mode/set']) +const RELEASED_V0_EVENT_TYPE_SET: ReadonlySet = new Set(Object.keys(RELEASED_V0_EVENT_DISPOSITIONS)) + +/** + * Validate the logical header shared by released v0 and v1. + * @param header - detached logical header. + * @param version - exact expected generation. + */ +export function assertReleasedSessionFormatHeader(header: SessionFormatHeader, version: 0 | 1): void { + const record = releasedV0Record(header, `format v${version} header`) + assertReleasedV0Keys(record, HEADER_REQUIRED, HEADER_OPTIONAL, `format v${version} header`) + if (record['version'] !== version) throw new SessionFormatError(`expected format v${version} header`) + if (typeof record['id'] !== 'string') throw new SessionFormatError(`format v${version} header id must be a string`) + sessionFormatCount(record['createdAt'], `format v${version} header createdAt`) + if (typeof record['isSeeded'] !== 'boolean') { + throw new SessionFormatError(`format v${version} header isSeeded must be a boolean`) + } + sessionFormatCount(record['delegationDepth'], `format v${version} header delegationDepth`) + for (const key of ['cwd', 'parentSession', 'agentPreset'] as const) { + if (record[key] !== undefined && typeof record[key] !== 'string') { + throw new SessionFormatError(`format v${version} header ${key} must be a string`) + } + } + if (typeof record['cwd'] === 'string' && !isAbsolute(record['cwd'])) { + throw new SessionFormatError(`format v${version} header cwd must be absolute`) + } + if (record['origin'] !== undefined && record['origin'] !== 'subagent') { + throw new SessionFormatError(`format v${version} header origin must be "subagent"`) + } +} + +/** + * Validate one released-v1 logical header. + * @param header - detached logical header. + */ +export function assertReleasedV1Header(header: SessionFormatHeader): void { + assertReleasedSessionFormatHeader(header, 1) +} + +/** + * Validate v0 before historical normalizers run. + * @param artifact - decoded released-v0 source. + */ +export function assertReleasedV0SourceArtifact(artifact: SessionFormatArtifact): void { + assertReleasedSessionFormatHeader(artifact.header, 0) + assertArtifactCoordinates(artifact, true, RELEASED_V0_EVENT_TYPE_SET) +} + +/** + * Validate normalized v0 events before the identity header version changes. + * @param artifact - normalized released-v0 artifact. + */ +export function assertNormalizedReleasedV0Artifact(artifact: SessionFormatArtifact): void { + assertReleasedSessionFormatHeader(artifact.header, 0) + assertArtifactCoordinates(artifact, false, RELEASED_V0_EVENT_TYPE_SET) + for (const event of artifact.events) assertReleasedEventPayload(event, 0) + assertReleasedArtifactRelationships(artifact) +} + +/** + * Validate the exact logical image emitted by the released v1 writer. + * @param artifact - decoded or migration-produced v1 artifact. + */ +export function assertReleasedV1Artifact(artifact: SessionFormatArtifact): void { + assertReleasedV1Header(artifact.header) + assertArtifactCoordinates(artifact, false, RELEASED_V0_EVENT_TYPE_SET) + for (const event of artifact.events) { + if (RELEASED_V0_EVENT_DISPOSITIONS[event.type] !== undefined) assertReleasedEventPayload(event, 1) + } + assertReleasedArtifactRelationships(artifact) +} + +/** + * Restore v1 against the installed build's ordinary event vocabulary without freezing payload additions. + * @param artifact - vocabulary-neutral released-v1 physical decode. + * @param knownEventTypes - event types understood by the installed current Session package. + * @returns the same validated detached artifact. + */ +export function restoreReleasedV1Artifact( + artifact: SessionFormatArtifact, + knownEventTypes: ReadonlySet, +): SessionFormatArtifact { + assertReleasedV1Header(artifact.header) + assertArtifactCoordinates(artifact, false, knownEventTypes) + return artifact +} + +/** + * Validate released-v1 physical layout without interpreting event vocabulary. + * @param artifact - physical-codec output. + */ +export function assertReleasedV1PhysicalArtifact(artifact: SessionFormatArtifact): void { + assertReleasedV1Header(artifact.header) + assertArtifactCoordinates(artifact, false, undefined, true) +} + +function assertArtifactCoordinates( + artifact: SessionFormatArtifact, + allowLegacySteering: boolean, + knownEventTypes?: ReadonlySet, + vocabularyNeutral = false, +): void { + const inheritedEventCount = sessionFormatCount(artifact.inheritedEventCount, 'Session inheritedEventCount') + if (inheritedEventCount > artifact.events.length) { + throw new SessionFormatError('Session inheritedEventCount exceeds its event count') + } + if (!artifact.header.isSeeded && inheritedEventCount !== 0) { + throw new SessionFormatError('unseeded Session inheritedEventCount must be 0') + } + for (let index = 0; index < artifact.events.length; index += 1) { + const event = artifact.events[index] as SessionFormatEvent + const record = releasedV0Record(event, `Session event ${index}`) + const type = record['type'] + if (typeof type !== 'string') throw new SessionFormatError(`Session event ${index} type must be a string`) + const disposition = RELEASED_V0_EVENT_DISPOSITIONS[type] + const legacy = allowLegacySteering && LEGACY_SOURCE_TYPES.has(type) + const currentKnown = knownEventTypes?.has(type) === true + const ignorableCurrent = !allowLegacySteering && !currentKnown && record['ignorable'] === true + if (!currentKnown && !legacy && !ignorableCurrent && !vocabularyNeutral) { + if (allowLegacySteering) { + throw new SessionFormatUnsupportedMigrationError( + `format v0 contains unknown historical event type ${JSON.stringify(type)} at seq ${index}; migration refuses unknown historical events even when ignorable`, + ) + } + throw new SessionFormatUnsupportedMigrationError( + `format v1 contains unknown required event type ${JSON.stringify(type)} at seq ${index}`, + ) + } + const frozenEnvelope = !vocabularyNeutral && knownEventTypes === RELEASED_V0_EVENT_TYPE_SET + const surface = disposition !== undefined ? SURFACE_EVENT_TYPES.has(type) : type === 'steering/message' + const optional = frozenEnvelope + ? surface ? SURFACE_OPTIONAL : LOG_OPTIONAL + : SURFACE_OPTIONAL + assertReleasedV0Keys(record, EVENT_REQUIRED, optional, `Session event ${index}`) + if (record['seq'] !== index) { + throw new SessionFormatError(`Session event ${index} has non-dense seq ${JSON.stringify(record['seq'])}`) + } + sessionFormatSafeInteger(record['time'], `Session event ${index} time`) + if (record['ignorable'] !== undefined && record['ignorable'] !== true) { + throw new SessionFormatError(`Session event ${index} ignorable must be true when present`) + } + if (frozenEnvelope && surface) assertSurfaceMetadata(record, index, type) + } +} + +function assertSurfaceMetadata(record: Record, seq: number, type: string): void { + const sources = record['sourceEventSeqs'] + if (sources !== undefined) { + if (!Array.isArray(sources)) throw new SessionFormatError(`${type} ${seq} sourceEventSeqs must be an array`) + const seen = new Set() + for (const source of sources) { + const current = sessionFormatCount(source, `${type} ${seq} sourceEventSeqs member`) + if (current >= seq || seen.has(current)) { + throw new SessionFormatError(`${type} ${seq} sourceEventSeqs must be unique earlier seqs`) + } + seen.add(current) + } + if (sources.length === 0 && type !== 'assistant/message') { + throw new SessionFormatError(`${type} ${seq} sourceEventSeqs must be non-empty`) + } + } + const operation = record['surfaceOp'] + if (operation === undefined || operation === 'append') return + const replacement = releasedV0Record(operation, `${type} ${seq} surfaceOp`) + assertReleasedV0Keys(replacement, ['op', 'start', 'end'], [], `${type} ${seq} surfaceOp`) + if (replacement['op'] !== 'replace') throw new SessionFormatError(`${type} ${seq} surfaceOp must replace`) + const start = sessionFormatCount(replacement['start'], `${type} ${seq} surface start`) + const end = sessionFormatCount(replacement['end'], `${type} ${seq} surface end`) + if (start > end || end >= seq) throw new SessionFormatError(`${type} ${seq} has an invalid surface replacement`) +} + +/** + * Validate one exact known payload after legacy normalization. + * @param event - known event to validate. + * @param version - payload generation controlling versioned members. + */ +export function assertReleasedEventPayload(event: SessionFormatEvent, version: 0 | 1): void { + const disposition = RELEASED_V0_EVENT_DISPOSITIONS[event.type] + /* v8 ignore next -- artifact coordinate validation admits only the frozen inventory before payload validation. */ + if (disposition === undefined) { + throw new SessionFormatUnsupportedMigrationError( + `format v0 contains unknown event type ${JSON.stringify(event.type)} at seq ${event.seq}`, + ) + } + const data = releasedV0Record(event.data, `${event.type} ${event.seq} data`) + if (event.type === 'subagent/descriptor' && data['version'] !== 3) { + const descriptorVersion = sessionFormatCount(data['version'], `${event.type} ${event.seq} version`) + if (version === 0) { + throw new SessionFormatUnsupportedMigrationError( + `${event.type} ${event.seq} uses unsupported descriptor version ${descriptorVersion}`, + ) + } + return + } + const versionOptional = version === 1 && event.type === 'session-log-deepseek/delivery-accepted' + ? [...disposition.optional, 'sessionFormatVersion'] + : disposition.optional + assertReleasedV0Keys(data, disposition.required, versionOptional, `${event.type} ${event.seq} data`) + for (const key of disposition.opaque) { + if (Object.hasOwn(data, key)) snapshotSessionFormatJson(data[key], `${event.type} ${event.seq} opaque ${key}`) + } + assertReleasedPayloadSemantics(event, version) +} diff --git a/packages/session/session-format-v0-to-v1/tests/codec.spec.ts b/packages/session/session-format-v0-to-v1/tests/codec.spec.ts new file mode 100644 index 0000000000..151788fef2 --- /dev/null +++ b/packages/session/session-format-v0-to-v1/tests/codec.spec.ts @@ -0,0 +1,246 @@ +import { describe, expect, it } from 'vitest' +import type { SessionFormatArtifact, SessionFormatEvent } from '@deepseek-ai/dsh-session-format' +import { + releasedV0SessionFormatCodec, + releasedV1SessionFormatCodec, +} from '../src/index.ts' + +const fullHeader = { + type: 'session', version: 1, id: 'codec', createdAt: 1, cwd: '/work', parentSession: 'parent', + seedLength: 0, origin: 'subagent', delegationDepth: 1, agentPreset: 'default', +} as const +const textBlock = { type: 'text', text: 'text' } as const + +function chunk( + seq: number, + type: 'text-delta' | 'reasoning-delta' | 'tool-call-delta', + value: string, + options: { turn?: number; step?: number; index?: number; time?: number; name?: string } = {}, +): SessionFormatEvent { + const stream = type === 'tool-call-delta' + ? { type, index: options.index ?? 0, id: 'call', ...(options.name === undefined ? {} : { name: options.name }), argumentsDelta: value } + : { type, index: options.index ?? 0, text: value } + return { + type: 'assistant/chunk', seq, time: options.time ?? seq + 1, + data: { turn: options.turn ?? 1, step: options.step ?? 0, chunk: stream }, + } +} + +function artifact(events: readonly SessionFormatEvent[], overrides: Partial = {}): SessionFormatArtifact { + return { + header: { version: 1, id: 'codec', createdAt: 1, isSeeded: false, delegationDepth: 0 }, + inheritedEventCount: 0, + events, + ...overrides, + } +} + +describe('released v0/v1 physical codecs', () => { + it('round-trips every physical header field and seeded zero cut', () => { + const decoded = releasedV1SessionFormatCodec.decodeArtifact(fullHeader, []) + expect(decoded).toEqual({ + header: { + version: 1, id: 'codec', createdAt: 1, cwd: '/work', parentSession: 'parent', + isSeeded: true, origin: 'subagent', delegationDepth: 1, agentPreset: 'default', + }, + inheritedEventCount: 0, + events: [], + }) + expect(releasedV1SessionFormatCodec.encodeArtifact(decoded, { packChunks: false }).header).toEqual(fullHeader) + expect(releasedV0SessionFormatCodec.encodeArtifact({ + ...decoded, + header: { ...decoded.header, version: 0 }, + }, { packChunks: false }).header).toEqual({ ...fullHeader, version: 0 }) + }) + + it.each([ + ['non-object', null], + ['extra member', { ...fullHeader, extra: true }], + ['wrong type', { ...fullHeader, type: 'other' }], + ['wrong version', { ...fullHeader, version: 0 }], + ['non-string id', { ...fullHeader, id: 1 }], + ['negative creation', { ...fullHeader, createdAt: -1 }], + ['negative depth', { ...fullHeader, delegationDepth: -1 }], + ['bad cwd', { ...fullHeader, cwd: 1 }], + ['bad parent', { ...fullHeader, parentSession: 1 }], + ['bad preset', { ...fullHeader, agentPreset: 1 }], + ['bad origin', { ...fullHeader, origin: 'other' }], + ])('refuses malformed physical header: %s', (_name, header) => { + expect(() => releasedV1SessionFormatCodec.decodeHeader(header)).toThrow() + }) + + it('packs and expands text, reasoning, and named tool-call runs exactly', () => { + const events = [ + chunk(0, 'text-delta', 'a'), chunk(1, 'text-delta', 'b'), chunk(2, 'text-delta', 'c'), + chunk(3, 'reasoning-delta', 'd'), chunk(4, 'reasoning-delta', 'e'), chunk(5, 'reasoning-delta', 'f'), + chunk(6, 'tool-call-delta', '{', { name: 'read' }), + chunk(7, 'tool-call-delta', '}', { name: 'read' }), + chunk(8, 'tool-call-delta', '', { name: 'read' }), + ] + const v0 = { ...artifact(events), header: { ...artifact(events).header, version: 0 } } + const encoded = releasedV0SessionFormatCodec.encodeArtifact(v0, { packChunks: true }) + expect(encoded.rows.map(row => row['type'])).toEqual(['text-chunks', 'reasoning-chunks', 'tool-call-chunks']) + expect(releasedV0SessionFormatCodec.decodeArtifact(encoded.header, encoded.rows).events).toEqual(events) + + const unnamed = [ + chunk(0, 'tool-call-delta', 'a'), + chunk(1, 'tool-call-delta', 'b'), + chunk(2, 'tool-call-delta', 'c'), + ] + const unnamedV0 = { ...artifact(unnamed), header: { ...artifact(unnamed).header, version: 0 } } + const unnamedEncoded = releasedV0SessionFormatCodec.encodeArtifact(unnamedV0, { packChunks: true }) + expect(unnamedEncoded.rows[0]?.['data']).not.toHaveProperty('name') + expect(releasedV0SessionFormatCodec.decodeArtifact(unnamedEncoded.header, unnamedEncoded.rows).events).toEqual(unnamed) + }) + + it('keeps short or non-continuing chunk runs unpacked', () => { + const events = [ + chunk(0, 'text-delta', 'a'), + chunk(1, 'text-delta', 'b', { index: 1 }), + chunk(2, 'text-delta', 'c', { turn: 2 }), + chunk(3, 'text-delta', 'd', { step: 1 }), + chunk(4, 'tool-call-delta', 'a'), + chunk(5, 'tool-call-delta', 'b', { name: 'read' }), + chunk(6, 'tool-call-delta', 'c', { name: 'read' }), + ] + const v0 = { ...artifact(events), header: { ...artifact(events).header, version: 0 } } + const encoded = releasedV0SessionFormatCodec.encodeArtifact(v0, { packChunks: true }) + expect(encoded.rows).toEqual(events) + }) + + it.each([ + ['row envelope', { type: 'text-chunks', seq0: 0, time0: 1, data: {}, extra: true }], + ['empty payload', { type: 'text-chunks', seq0: 0, time0: 1, data: { turn: 1, step: 0, index: 0, dt: [], texts: [] } }], + ['non-string payload', { type: 'text-chunks', seq0: 0, time0: 1, data: { turn: 1, step: 0, index: 0, dt: [], texts: [1] } }], + ['gap arity', { type: 'reasoning-chunks', seq0: 0, time0: 1, data: { turn: 1, step: 0, index: 0, dt: [], texts: ['a', 'b'] } }], + ['coordinates', { type: 'text-chunks', seq0: 0, time0: 1, data: { turn: '1', step: 0, index: 0, dt: [], texts: ['a'] } }], + ['tool id', { type: 'tool-call-chunks', seq0: 0, time0: 1, data: { turn: 1, step: 0, index: 0, id: 1, dt: [], args: ['a'] } }], + ['tool name', { type: 'tool-call-chunks', seq0: 0, time0: 1, data: { turn: 1, step: 0, index: 0, id: 'id', name: 1, dt: [], args: ['a'] } }], + ['unsafe time sum', { type: 'text-chunks', seq0: 0, time0: Number.MAX_SAFE_INTEGER, data: { turn: 1, step: 0, index: 0, dt: [1], texts: ['a', 'b'] } }], + ])('refuses malformed packed row: %s', (_name, row) => { + expect(() => releasedV1SessionFormatCodec.decodeArtifact(fullHeader, [row])).toThrow() + }) + + it.each([ + ['not array', 'bad'], + ['too many scalar entries', [0, 0]], + ['malformed range', [[0]]], + ['reversed range', [[2, 1]]], + ['range past event', [[0, 3]]], + ['overlapping ranges', [[0, 1], [1, 2]]], + ])('refuses malformed stored provenance: %s', (_name, sourceEventSeqs) => { + const rows = [ + { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }, + { type: 'user/message', seq: 1, time: 2, data: { + id: 'u', role: 'user', content: [{ type: 'text', text: 'x' }], source: { kind: 'user' }, + }, sourceEventSeqs, surfaceOp: 'append' }, + ] + expect(() => releasedV1SessionFormatCodec.decodeArtifact( + { type: 'session', version: 1, id: 'codec', createdAt: 1, delegationDepth: 0 }, + rows, + )).toThrow() + }) + + it('contains non-SessionFormatError row failures during recoverable scans', () => { + const bad = new Proxy({}, { ownKeys: () => { throw new Error('proxy failure') } }) + const recovered = releasedV1SessionFormatCodec.decodeRecoverableArtifact( + { type: 'session', version: 1, id: 'codec', createdAt: 1, delegationDepth: 0 }, + [bad], + ) + expect(recovered).toMatchObject({ events: [] }) + }) + + it('ignores decodable non-terminal rows after the first recoverable issue', () => { + const header = { type: 'session', version: 1, id: 'codec', createdAt: 1, delegationDepth: 0 } + const recovered = releasedV1SessionFormatCodec.decodeRecoverableArtifact(header, [ + { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }, + { type: 'turn/start', seq: 4, time: 2, data: { turn: 2 } }, + { type: 'step/start', seq: 1, time: 3, data: { turn: 1, step: 0 } }, + ]) + expect(recovered).toMatchObject({ events: [{ seq: 0 }] }) + }) + + it('rejects strict gaps and a recoverable gap row that itself closes a turn', () => { + const currentHeader = { type: 'session', version: 1, id: 'codec', createdAt: 1, delegationDepth: 0 } + expect(() => releasedV1SessionFormatCodec.decodeArtifact(currentHeader, [ + { type: 'turn/start', seq: 1, time: 1, data: { turn: 1 } }, + ])).toThrow(/seq gap/) + expect(() => releasedV1SessionFormatCodec.decodeRecoverableArtifact(currentHeader, [ + { type: 'turn/end', seq: 1, time: 1, data: { turn: 1, reason: { kind: 'completed' } } }, + ])).toThrow(/seq gap/) + }) + + it('refuses overlapping ranges after a valid first range', () => { + const header = { type: 'session', version: 1, id: 'codec', createdAt: 1, delegationDepth: 0 } + const rows = [ + { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }, + { type: 'step/start', seq: 1, time: 2, data: { turn: 1, step: 1 } }, + { type: 'step/end', seq: 2, time: 3, data: { turn: 1, step: 1 } }, + { type: 'turn/end', seq: 3, time: 4, data: { turn: 1, reason: { kind: 'completed' } } }, + { + type: 'user/message', seq: 4, time: 5, surfaceOp: 'append', sourceEventSeqs: [[0, 1], [1, 2]], + data: { id: 'u', role: 'user', content: [textBlock], source: { kind: 'user' } }, + }, + ] + expect(() => releasedV1SessionFormatCodec.decodeArtifact(header, rows)).toThrow(/strictly increasing/) + }) + + it('keeps non-consecutive provenance scalar and leaves invalid v0 chunks unpacked', () => { + const events = [ + { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }, + { type: 'step/start', seq: 1, time: 2, data: { turn: 1, step: 1 } }, + { type: 'step/end', seq: 2, time: 3, data: { turn: 1, step: 1 } }, + { + type: 'user/message', seq: 3, time: 4, surfaceOp: 'append', sourceEventSeqs: [0, 2], + data: { id: 'u', role: 'user', content: [textBlock], source: { kind: 'user' } }, + }, + ] as SessionFormatEvent[] + expect(releasedV1SessionFormatCodec.encodeArtifact(artifact(events), { packChunks: false }).rows[3]) + .toEqual(events[3]) + + const invalidChunk = { type: 'assistant/chunk', seq: 0, time: 1, data: { turn: 1, step: 0, chunk: null } } + const v0 = { + header: { version: 0, id: 'codec', createdAt: 1, isSeeded: false, delegationDepth: 0 }, + inheritedEventCount: 0, + events: [invalidChunk], + } as unknown as SessionFormatArtifact + expect(releasedV0SessionFormatCodec.encodeArtifact(v0, { packChunks: true }).rows).toEqual([invalidChunk]) + + const causal = { + ...v0, + events: [ + { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }, + { type: 'user/message', seq: 1, time: 2, surfaceOp: 'append', data: { + id: 'one', role: 'user', content: [textBlock], source: { kind: 'user' }, + } }, + { type: 'user/message', seq: 2, time: 3, surfaceOp: 'append', data: { + id: 'two', role: 'user', content: [textBlock], source: { kind: 'user' }, + } }, + { type: 'user/message', seq: 3, time: 4, surfaceOp: 'append', sourceEventSeqs: [2, 0], data: { + id: 'three', role: 'user', content: [textBlock], source: { kind: 'user' }, + } }, + ], + } as unknown as SessionFormatArtifact + expect(releasedV0SessionFormatCodec.encodeArtifact(causal, { packChunks: false }).rows[3]?.['sourceEventSeqs']) + .toEqual([2, 0]) + + for (const badData of [ + null, + { turn: 1, step: 1, chunk: null }, + { turn: 1, step: 1, chunk: { type: 'other', index: 0 } }, + { turn: 1, step: 1, chunk: { type: 'text-delta', index: 0, text: 1 } }, + { turn: 1, step: 1, chunk: { type: 'tool-call-delta', index: 0, id: 1, argumentsDelta: 'x' } }, + ]) { + const bad = { ...v0, events: [{ type: 'assistant/chunk', seq: 0, time: 1, data: badData }] } as unknown as SessionFormatArtifact + expect(releasedV0SessionFormatCodec.encodeArtifact(bad, { packChunks: true }).rows).toHaveLength(1) + } + + const farTimes = [ + chunk(0, 'text-delta', 'a', { time: Number.MIN_SAFE_INTEGER }), + chunk(1, 'text-delta', 'b', { time: Number.MAX_SAFE_INTEGER }), + chunk(2, 'text-delta', 'c', { time: Number.MAX_SAFE_INTEGER }), + ] + const far = { ...v0, events: farTimes } as unknown as SessionFormatArtifact + expect(releasedV0SessionFormatCodec.encodeArtifact(far, { packChunks: true }).rows).toHaveLength(3) + }) +}) diff --git a/packages/session/session-format-v0-to-v1/tests/legacy.spec.ts b/packages/session/session-format-v0-to-v1/tests/legacy.spec.ts new file mode 100644 index 0000000000..101b251f24 --- /dev/null +++ b/packages/session/session-format-v0-to-v1/tests/legacy.spec.ts @@ -0,0 +1,295 @@ +import { describe, expect, it } from 'vitest' +import { SessionFormatUnsupportedMigrationError } from '@deepseek-ai/dsh-session-format' +import { + releasedV0SessionFormatCodec, + sessionFormatV0ToV1, +} from '../src/index.ts' + +const header = { + type: 'session', + version: 0, + id: 'legacy', + createdAt: 1, + delegationDepth: 0, +} as const + +function migrate(rows: readonly unknown[]) { + return sessionFormatV0ToV1.migrate(releasedV0SessionFormatCodec.decodeArtifact(header, rows)) +} + +describe('released v0 legacy normalization', () => { + it('restores pre-identity user, assistant, and replacement tool-result identities', () => { + const rows = [ + { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }, + { + type: 'user/message', seq: 1, time: 2, + data: { content: [{ type: 'text', text: 'hi' }], source: { kind: 'user' } }, + surfaceOp: 'append', + }, + { type: 'step/start', seq: 2, time: 3, data: { turn: 1, step: 1 } }, + { + type: 'assistant/message', seq: 3, time: 4, + data: { + turn: 1, + step: 1, + content: [{ type: 'tool-call', id: 'call-1', name: 'read', arguments: '{}' }], + provenance: { provider: 'mock', model: 'mock' }, + }, + surfaceOp: 'append', + }, + { + type: 'tool/call', seq: 4, time: 5, + data: { turn: 1, step: 1, callId: 'call-1', name: 'read', arguments: '{}' }, + }, + { + type: 'tool/result', seq: 5, time: 6, + data: { turn: 1, step: 1, callId: 'call-1', content: [{ type: 'text', text: 'full' }], isError: false }, + sourceEventSeqs: [4], + surfaceOp: 'append', + }, + { + type: 'tool/result', seq: 6, time: 7, + data: { turn: 1, step: 1, callId: 'call-1', content: [{ type: 'text', text: 'pruned' }], isError: false }, + sourceEventSeqs: [5], + surfaceOp: { op: 'replace', start: 5, end: 5 }, + }, + { type: 'step/end', seq: 7, time: 8, data: { turn: 1, step: 1 } }, + { type: 'turn/end', seq: 8, time: 9, data: { turn: 1, reason: { kind: 'completed' } } }, + ] + + const events = migrate(rows).events + + expect(events[1]?.data).toMatchObject({ id: 'legacy-message:legacy:1', role: 'user' }) + expect(events[3]?.data).toMatchObject({ message: { id: 'legacy-message:legacy:3', role: 'assistant' } }) + expect(events[5]?.data).toMatchObject({ message: { id: 'legacy-message:legacy:5', role: 'user' } }) + expect(events[6]?.data).toMatchObject({ message: { id: 'legacy-message:legacy:5', role: 'user' } }) + }) + + it('normalizes wrapped and flat steering plus every accepted old turn ending', () => { + const user = { + id: 'wrapped', + role: 'user', + content: [{ type: 'text', text: 'wrapped' }], + source: { kind: 'user' }, + } + const rows = [ + { type: 'turn/start', seq: 0, time: 1, data: { turn: 1, trigger: { kind: 'message' } } }, + { type: 'steering/message', seq: 1, time: 2, data: { turn: 1, message: user }, surfaceOp: 'append' }, + { + type: 'steering/message', seq: 2, time: 3, + data: { turn: 1, content: [{ type: 'text', text: 'flat' }], source: { kind: 'user' } }, + surfaceOp: 'append', + }, + { type: 'turn/end', seq: 3, time: 4, data: { turn: 1, reason: { kind: 'completed' } } }, + { type: 'turn/start', seq: 4, time: 5, data: { turn: 2, trigger: { kind: 'retry' } } }, + { + type: 'turn/end', seq: 5, time: 6, + data: { turn: 2, reason: { kind: 'error', step: 1, failure: { message: 'provider', code: 'SERVER' } } }, + }, + { type: 'turn/start', seq: 6, time: 7, data: { turn: 3, trigger: { kind: 'message' } } }, + { type: 'turn/end', seq: 7, time: 8, data: { turn: 3, reason: { kind: 'aborted' } } }, + { type: 'turn/start', seq: 8, time: 9, data: { turn: 4, trigger: { kind: 'message' } } }, + { type: 'turn/end', seq: 9, time: 10, data: { turn: 4, reason: { kind: 'disposed' } } }, + { type: 'turn/start', seq: 10, time: 11, data: { turn: 5, trigger: { kind: 'message' } } }, + { + type: 'turn/end', seq: 11, time: 12, + data: { turn: 5, reason: { kind: 'error', step: 1, message: 'thrown' } }, + }, + { type: 'turn/start', seq: 12, time: 13, data: { turn: 6, trigger: { kind: 'message' } } }, + { + type: 'turn/end', seq: 13, time: 14, + data: { + turn: 6, + reason: { + kind: 'error', + step: 0, + failure: { + message: 'detailed', code: 'RATE_LIMIT', status: 429, + providerRetryAfterMs: 1000, requestId: 'request-1', + }, + }, + }, + }, + { type: 'turn/start', seq: 14, time: 15, data: { turn: 7, trigger: { kind: 'message' } } }, + { + type: 'turn/end', seq: 15, time: 16, + data: { turn: 7, reason: { kind: 'error', step: 0, message: 'coded', code: 'CODED' } }, + }, + ] + + const events = migrate(rows).events + + expect(events.filter(event => event.type === 'turn/start').map(event => event.data)) + .toEqual(Array.from({ length: 7 }, (_, index) => ({ turn: index + 1 }))) + expect(events[1]).toMatchObject({ type: 'user/message', data: { id: 'wrapped' } }) + expect(events[2]).toMatchObject({ + type: 'user/message', + data: { id: 'legacy-message:legacy:2', role: 'user' }, + }) + expect(events.filter(event => event.type === 'turn/end').map(event => event.data)).toEqual([ + { turn: 1, reason: { kind: 'completed' } }, + { turn: 2, reason: { kind: 'error', error: { message: 'provider', code: 'SERVER' } } }, + { turn: 3, reason: { kind: 'aborted', reason: { kind: 'legacy' } } }, + { turn: 4, reason: { kind: 'aborted', reason: { kind: 'disposed' } } }, + { turn: 5, reason: { kind: 'error', error: { message: 'thrown', code: 'UNKNOWN' } } }, + { + turn: 6, + reason: { + kind: 'error', + error: { + message: 'detailed', code: 'RATE_LIMIT', status: 429, + providerRetryAfterMs: 1000, requestId: 'request-1', + }, + }, + }, + { turn: 7, reason: { kind: 'error', error: { message: 'coded', code: 'CODED' } } }, + ]) + }) + + it.each([ + ['turn/start trigger', { type: 'turn/start', seq: 0, time: 1, data: { turn: 1, trigger: null } }], + ['flat steering extra', { + type: 'steering/message', seq: 0, time: 1, surfaceOp: 'append', + data: { turn: 1, content: [], source: { kind: 'user' }, extra: true }, + }], + ['steering null', { type: 'steering/message', seq: 0, time: 1, surfaceOp: 'append', data: null }], + ['turn/end extra', { + type: 'turn/end', seq: 0, time: 1, + data: { turn: 1, reason: { kind: 'completed', extra: true } }, + }], + ['turn/end null reason', { type: 'turn/end', seq: 0, time: 1, data: { turn: 1, reason: null } }], + ['turn/end intermediate step', { + type: 'turn/end', seq: 0, time: 1, + data: { turn: 1, step: 1, reason: { kind: 'completed' } }, + }], + ['turn/end aborted extra', { + type: 'turn/end', seq: 0, time: 1, + data: { turn: 1, reason: { kind: 'aborted', extra: true } }, + }], + ['turn/end disposed extra', { + type: 'turn/end', seq: 0, time: 1, + data: { turn: 1, reason: { kind: 'disposed', extra: true } }, + }], + ['turn/end error negative step', { + type: 'turn/end', seq: 0, time: 1, + data: { turn: 1, reason: { kind: 'error', step: -1, message: 'bad' } }, + }], + ['turn/end error numeric code', { + type: 'turn/end', seq: 0, time: 1, + data: { turn: 1, reason: { kind: 'error', step: 0, message: 'bad', code: 1 } }, + }], + ])('refuses malformed legacy %s', (_name, event) => { + expect(() => migrate([event])).toThrow(/malformed|must be a JSON object|unexpected member|non-negative/) + }) + + it.each([ + ['request/header-delta', { config: { model: 'legacy' } }], + ['mode/set', { mode: 'plan' }], + ])('classifies retired %s as unsupported migration', (type, data) => { + expect(() => migrate([{ type, seq: 0, time: 1, data }])).toThrow(SessionFormatUnsupportedMigrationError) + }) + + it('classifies the retired request/header fallback reason as unsupported migration', () => { + expect(() => migrate([{ + type: 'request/header', + seq: 0, + time: 1, + data: { header: { config: { model: 'legacy' } }, reason: 'fallback' }, + }])).toThrow(SessionFormatUnsupportedMigrationError) + }) + + it('leaves current message and turn-end variants unchanged inside the identity edge', () => { + const rows = [ + { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }, + { + type: 'user/message', seq: 1, time: 2, surfaceOp: 'append', + data: { id: 'current', role: 'user', content: [{ type: 'text', text: 'x' }], source: { kind: 'user' } }, + }, + { type: 'turn/end', seq: 2, time: 3, data: { turn: 1, reason: { kind: 'aborted', reason: { kind: 'user' } } } }, + { type: 'turn/start', seq: 3, time: 4, data: { turn: 2 } }, + { type: 'turn/end', seq: 4, time: 5, data: { turn: 2, reason: { kind: 'error', error: { message: 'x', code: 'X' } } } }, + ] + expect(migrate(rows).events).toEqual(rows) + }) + + it('refuses wrong-version headers and additional malformed legacy branches', () => { + expect(() => sessionFormatV0ToV1.migrateHeader({ + version: 1, id: 'x', createdAt: 1, isSeeded: false, delegationDepth: 0, + })).toThrow(/expected format v0/) + expect(() => migrate([{ + type: 'turn/start', seq: 0, time: 1, data: { turn: 0, trigger: { kind: 'message' } }, + }])).toThrow(/malformed/) + expect(() => migrate([{ + type: 'turn/end', seq: 0, time: 1, + data: { turn: 1, reason: { kind: 'error', step: 0, failure: { message: 1, code: 'X' } } }, + }])).toThrow(/malformed/) + expect(() => migrate([{ + type: 'tool/result', seq: 0, time: 1, surfaceOp: 'append', + data: { turn: 1, step: 0, callId: 1, content: [], isError: false }, + }])).toThrow() + expect(() => migrate([{ + type: 'tool/result', seq: 0, time: 1, surfaceOp: 'append', + data: { turn: 1, step: 0, callId: 'call', content: [] }, + }])).toThrow() + expect(migrate([ + { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }, + { type: 'turn/end', seq: 1, time: 2, data: { turn: 1, reason: { kind: 'future-reason' } } }, + ]).events[1]).toMatchObject({ data: { reason: { kind: 'future-reason' } } }) + }) + + it('refuses a legacy replacement whose cited message has no imported identity', () => { + expect(() => migrate([ + { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }, + { + type: 'tool/result', seq: 1, time: 2, + data: { turn: 1, step: 0, callId: 'call', content: [], isError: false }, + sourceEventSeqs: [0], surfaceOp: { op: 'replace', start: 0, end: 0 }, + }, + ])).toThrow(/without identity/) + }) + + it('normalizes the legacy request-header message prefix', () => { + const rows = [ + { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }, + { type: 'request/header', seq: 1, time: 2, data: { + header: { + config: { provider: 'mock', model: 'mock' }, + messagePrefix: [{ role: 'user', content: [{ type: 'text', text: 'obsolete' }] }], + }, + reason: 'initial', + } }, + { type: 'turn/end', seq: 2, time: 3, data: { turn: 1, reason: { kind: 'completed' } } }, + ] + const events = migrate(rows).events + expect(events[1]?.data).not.toHaveProperty('header.messagePrefix') + expect(() => migrate([ + { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }, + { type: 'request/header', seq: 1, time: 2, data: { + header: { config: { provider: 'mock', model: 'mock' }, messagePrefix: 'bad' }, reason: 'initial', + } }, + ])).toThrow(/messagePrefix/) + expect(() => migrate([{ + type: 'turn/end', seq: 0, time: 1, data: { turn: 0, reason: { kind: 'completed' } }, + }])).toThrow(/malformed/) + expect(() => migrate([{ + type: 'turn/end', seq: 0, time: 1, data: { turn: 1, reason: { kind: 1 } }, + }])).toThrow(/malformed/) + }) + + it('refuses invalid legacy relationship facts instead of rewriting them', () => { + const turn = { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } } + const human = { type: 'user/message', seq: 1, time: 2, surfaceOp: 'append', data: { + id: 'human', role: 'user', content: [{ type: 'text', text: 'human' }], source: { kind: 'user' }, + } } + expect(() => migrate([turn, human, { + type: 'user/message', seq: 2, time: 3, sourceEventSeqs: [1], surfaceOp: { op: 'replace', start: 1, end: 1 }, data: { + id: 'compact', role: 'user', content: [{ type: 'text', text: 'summary' }], + source: { kind: 'plugin', plugin: 'compact', compactionId: 'orphan' }, + }, + }])).toThrow(/compaction checkpoint/) + expect(() => migrate([turn, human, { + type: 'session/title', seq: 2, time: 3, + data: { title: 'Pinned', messageSeqs: [1], source: { kind: 'user' } }, + }])).toThrow(/empty exactly/) + }) +}) diff --git a/packages/session/session-format-v0-to-v1/tests/migration.spec.ts b/packages/session/session-format-v0-to-v1/tests/migration.spec.ts new file mode 100644 index 0000000000..33bb1376e6 --- /dev/null +++ b/packages/session/session-format-v0-to-v1/tests/migration.spec.ts @@ -0,0 +1,239 @@ +import { describe, expect, it } from 'vitest' +import { + RELEASED_V0_EVENT_TYPES, + releasedV0SessionFormatCodec, + releasedV1SessionFormatCodec, + restoreReleasedV1Artifact, + sessionFormatV0ToV1, +} from '../src/index.ts' + +describe('released Session format v0 to v1', () => { + it('changes only the version of a canonical decoded artifact', () => { + const header = { + type: 'session', + version: 0, + id: 'identity', + createdAt: 1, + cwd: '/work', + delegationDepth: 0, + } + const rows = [ + { type: 'turn/start', seq: 0, time: 2, data: { turn: 1 } }, + { type: 'step/start', seq: 1, time: 3, data: { turn: 1, step: 1 } }, + { + type: 'text-chunks', + seq0: 2, + time0: 4, + data: { turn: 1, step: 1, index: 0, dt: [1, 1], texts: ['a', 'b', 'c'] }, + }, + ] + const source = releasedV0SessionFormatCodec.decodeArtifact(header, rows) + + const migrated = sessionFormatV0ToV1.migrate(source) + + expect(migrated).toEqual({ + ...source, + header: { ...source.header, version: 1 }, + }) + sessionFormatV0ToV1.validateTarget(migrated) + expect(releasedV1SessionFormatCodec.encodeArtifact(migrated, { packChunks: true })).toEqual({ + header: { ...header, version: 1 }, + rows, + }) + }) + + it('recovers only the complete row prefix and refuses a later committing turn end', () => { + const header = { + type: 'session', + version: 0, + id: 'recoverable', + createdAt: 1, + delegationDepth: 0, + } + const prefix = [ + { type: 'turn/start', seq: 0, time: 2, data: { turn: 1 } }, + { type: 'turn/end', seq: 1, time: 3, data: { turn: 1, reason: { kind: 'completed' } } }, + ] + const badRow = { + type: 'text-chunks', + seq0: 7, + time0: 4, + data: { turn: 2, step: 0, index: 0, dt: [1], texts: ['x', 'y'] }, + } + + expect(releasedV0SessionFormatCodec.decodeRecoverableArtifact(header, [...prefix, badRow])) + .toEqual(releasedV0SessionFormatCodec.decodeArtifact(header, prefix)) + expect(() => releasedV0SessionFormatCodec.decodeRecoverableArtifact(header, [ + ...prefix, + badRow, + { type: 'turn/end', seq: 2, time: 6, data: { turn: 2, reason: { kind: 'interrupted' } } }, + ])).toThrow(/seq gap/) + }) + + it('requires canonical delegation depth and decodes provenance without mutating source rows', () => { + const incompleteHeader = { type: 'session', version: 0, id: 'old', createdAt: 1 } + const header = { ...incompleteHeader, delegationDepth: 0 } + const provenanceRow = { + type: 'assistant/message', + seq: 3, + time: 5, + data: { + turn: 1, + step: 1, + message: { + id: 'message', + role: 'assistant', + content: [], + source: { kind: 'model', provider: 'test', model: 'test' }, + }, + }, + sourceEventSeqs: [[0, 2]], + surfaceOp: 'append', + } + const rows = [ + { type: 'turn/start', seq: 0, time: 2, data: { turn: 1 } }, + { type: 'step/start', seq: 1, time: 3, data: { turn: 1, step: 1 } }, + { + type: 'assistant/chunk', seq: 2, time: 4, + data: { turn: 1, step: 1, chunk: { type: 'text-delta', index: 0, text: 'x' } }, + }, + provenanceRow, + ] + + const decoded = releasedV0SessionFormatCodec.decodeArtifact(header, rows) + + expect(() => releasedV0SessionFormatCodec.decodeArtifact(incompleteHeader, rows)).toThrow(/delegationDepth/) + expect(decoded.header.delegationDepth).toBe(0) + expect(decoded.events[3]?.sourceEventSeqs).toEqual([0, 1, 2]) + expect(provenanceRow.sourceEventSeqs).toEqual([[0, 2]]) + const migrated = sessionFormatV0ToV1.migrate(decoded) + expect(releasedV1SessionFormatCodec.encodeArtifact(migrated, { packChunks: false }).header) + .toEqual({ ...header, version: 1 }) + }) + + it('refuses v1-only generation fields in v0 and accepts them in v1', () => { + const event = { + type: 'session-log-deepseek/delivery-accepted', + seq: 1, + time: 2, + data: { sessionId: 'delivery', throughSeq: 0, sessionFormatVersion: 1 }, + } + const prefix = { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } } + const v0 = { type: 'session', version: 0, id: 'delivery', createdAt: 1, delegationDepth: 0 } + const v1 = { ...v0, version: 1 } + + expect(() => sessionFormatV0ToV1.migrate( + releasedV0SessionFormatCodec.decodeArtifact(v0, [prefix, event]), + )).toThrow(/unexpected member "sessionFormatVersion"/) + expect(releasedV1SessionFormatCodec.decodeArtifact(v1, [prefix, event]).events).toEqual([prefix, event]) + }) + + it('preserves a complete canonical multi-owner log except for header.version', () => { + const physicalHeader = { + type: 'session', version: 0, id: 'full-identity', createdAt: 1, cwd: '/work', delegationDepth: 0, + } + const human = { + id: 'human', role: 'user', content: [{ type: 'text', text: 'hello' }], source: { kind: 'user' }, + } + const rows = [ + { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }, + { type: 'user/message', seq: 1, time: 2, data: human, surfaceOp: 'append' }, + { type: 'step/start', seq: 2, time: 3, data: { turn: 1, step: 1 } }, + { + type: 'assistant/chunk', seq: 3, time: 4, + data: { turn: 1, step: 1, chunk: { type: 'text-delta', index: 0, text: 'hello' } }, + }, + { + type: 'assistant/message', seq: 4, time: 5, sourceEventSeqs: [3], surfaceOp: 'append', + data: { + turn: 1, step: 1, + message: { + id: 'assistant', role: 'assistant', content: [ + { type: 'text', text: 'hello' }, + { type: 'tool-call', id: 'call', name: 'read', arguments: '{}' }, + ], + source: { kind: 'model', provider: 'mock', model: 'mock' }, + }, + }, + }, + { type: 'tool/call', seq: 5, time: 6, data: { turn: 1, step: 1, callId: 'call', name: 'read', arguments: '{}' } }, + { + type: 'tool/result', seq: 6, time: 7, sourceEventSeqs: [5], surfaceOp: 'append', + data: { + turn: 1, step: 1, + message: { + id: 'result', role: 'user', + content: [{ type: 'tool-result', toolCallId: 'call', content: [{ type: 'text', text: 'ok' }], isError: false }], + source: { kind: 'tool', callId: 'call' }, + }, + meta: { opaque: { seq: 999 } }, + }, + }, + { + type: 'tool/code-dispatch-start', seq: 7, time: 8, + data: { rootCallId: 'root', parentCallId: 'root', subCallId: 'sub', name: 'read', arguments: { opaque: [1] } }, + }, + { + type: 'tool/code-dispatch', seq: 8, time: 9, + data: { + rootCallId: 'root', parentCallId: 'root', subCallId: 'sub', name: 'read', + arguments: { opaque: [1] }, isError: false, content: [], + }, + }, + { type: 'step/end', seq: 9, time: 10, data: { turn: 1, step: 1 } }, + { type: 'session/title', seq: 10, time: 11, data: { title: 'Title', messageSeqs: [1], source: { kind: 'fallback' } } }, + { type: 'command/run', seq: 11, time: 12, data: { commandId: 'command', name: 'compact', source: { kind: 'user' } } }, + { type: 'feedback/record', seq: 12, time: 13, data: { text: 'feedback' } }, + { type: 'command/done', seq: 13, time: 14, data: { commandId: 'command', kind: 'success', sourceEventSeq: 12 } }, + { type: 'compaction/start', seq: 14, time: 15, data: { compactionId: 'compact', sourceCommandId: 'command', turn: 1 } }, + { + type: 'compaction/summary', seq: 15, time: 16, + data: { + compactionId: 'compact', sourceCommandId: 'command', summary: [{ type: 'text', text: 'summary' }], + shadowedRange: { start: 1, end: 6 }, shadowedSeqs: [1, 4, 6], shadowedTokenCount: 10, + provider: 'mock', model: 'mock', rawOutput: [{ type: 'text', text: 'summary' }], llmStreamCall: true, + }, + }, + { + type: 'user/message', seq: 16, time: 17, sourceEventSeqs: [1, 4, 6], + surfaceOp: { op: 'replace', start: 1, end: 6 }, + data: { + id: 'checkpoint', role: 'user', content: [{ type: 'text', text: 'summary' }], + source: { kind: 'plugin', plugin: 'compact', compactionId: 'compact', sourceCommandId: 'command' }, + }, + }, + { type: 'compaction/end', seq: 17, time: 18, data: { compactionId: 'compact', sourceCommandId: 'command', turn: 1 } }, + { + type: 'session-log-deepseek/delivery-accepted', seq: 18, time: 19, + data: { sessionId: 'full-identity', throughSeq: 17 }, + }, + { type: 'turn/end', seq: 19, time: 20, data: { turn: 1, reason: { kind: 'completed' } } }, + ] + const source = releasedV0SessionFormatCodec.decodeArtifact(physicalHeader, rows) + const migrated = sessionFormatV0ToV1.migrate(source) + expect(migrated).toEqual({ ...source, header: { ...source.header, version: 1 } }) + }) + + it('keeps the v1 physical codec vocabulary-neutral for current growth and a future source freeze', () => { + const physicalHeader = { + type: 'session', version: 1, id: 'ordinary-growth', createdAt: 1, delegationDepth: 0, + } + const ordinary = { type: 'ordinary/post-v1', seq: 0, time: 1, data: { required: true } } + const decoded = releasedV1SessionFormatCodec.decodeArtifact(physicalHeader, [ordinary]) + + expect(decoded.events).toEqual([ordinary]) + expect(() => { sessionFormatV0ToV1.validateTarget(decoded) }).toThrow(/unknown required event/) + + const generatedCurrentTypes = new Set([...RELEASED_V0_EVENT_TYPES, ordinary.type]) + expect(() => restoreReleasedV1Artifact(decoded, generatedCurrentTypes)).not.toThrow() + + const frozenFutureV1SourceTypes = new Set(generatedCurrentTypes) + expect(() => restoreReleasedV1Artifact(decoded, frozenFutureV1SourceTypes)).not.toThrow() + + const extendedKnownPayload = releasedV1SessionFormatCodec.decodeArtifact(physicalHeader, [{ + type: 'turn/start', seq: 0, time: 1, data: { turn: 1, postReleaseMember: true }, + }]) + expect(() => { sessionFormatV0ToV1.validateTarget(extendedKnownPayload) }).toThrow(/unexpected member/) + expect(() => restoreReleasedV1Artifact(extendedKnownPayload, generatedCurrentTypes)).not.toThrow() + }) +}) diff --git a/packages/session/session-format-v0-to-v1/tests/relationships.spec.ts b/packages/session/session-format-v0-to-v1/tests/relationships.spec.ts new file mode 100644 index 0000000000..1eb74daa41 --- /dev/null +++ b/packages/session/session-format-v0-to-v1/tests/relationships.spec.ts @@ -0,0 +1,598 @@ +import { describe, expect, it } from 'vitest' +import { SessionFormatUnsupportedMigrationError } from '@deepseek-ai/dsh-session-format' +import { + releasedV0SessionFormatCodec, + releasedV1SessionFormatCodec, + assertReleasedV1Artifact, + sessionFormatV0ToV1, +} from '../src/index.ts' + +const header = { + type: 'session', version: 1, id: 'relationships', createdAt: 1, delegationDepth: 0, +} as const +const user = (id: string, source: object = { kind: 'user' }) => ({ + id, role: 'user', content: [{ type: 'text', text: id }], source, +}) + +function decode(rows: readonly unknown[], physicalHeader: unknown = header) { + const artifact = releasedV1SessionFormatCodec.decodeArtifact(physicalHeader, rows) + assertReleasedV1Artifact(artifact) + return artifact +} + +describe('released v1 whole-artifact relationships', () => { + it.each([ + ['lone turn/end', [ + { type: 'turn/end', seq: 0, time: 1, data: { turn: 1, reason: { kind: 'completed' } } }, + ]], + ['orphan retry-started', [ + { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }, + { type: 'step/start', seq: 1, time: 2, data: { turn: 1, step: 1 } }, + { type: 'llm/retry-started', seq: 2, time: 3, data: { retryId: 'retry', turn: 1, step: 1, retry: 1 } }, + ]], + ['title citing a turn marker', [ + { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }, + { type: 'session/title', seq: 1, time: 2, data: { title: 'Bad', messageSeqs: [0], source: { kind: 'fallback' } } }, + ]], + ['compaction/end without start', [ + { type: 'compaction/end', seq: 0, time: 1, data: { compactionId: 'c', turn: null } }, + ]], + ['PTC dispatch outside a turn', [ + { + type: 'tool/code-dispatch-start', seq: 0, time: 1, + data: { rootCallId: 'root', parentCallId: 'root', subCallId: 'sub', name: 'read', arguments: {} }, + }, + ]], + ['surface replace without shadow provenance', [ + { type: 'user/message', seq: 0, time: 1, data: user('one'), surfaceOp: 'append' }, + { + type: 'user/message', seq: 1, time: 2, data: user('two'), + surfaceOp: { op: 'replace', start: 0, end: 0 }, + }, + ]], + ['impossible session-reference statistics', [ + { + type: 'user/message', seq: 0, time: 1, surfaceOp: 'append', + data: user('reference', { + kind: 'session-reference', form: 'recall', version: 1, + references: [{ + sessionId: 'source', label: 'Source', capturedThroughSeq: null, compacted: false, + originalMessages: 1, retainedMessages: 2, omittedMessages: 0, omittedBytes: 0, + truncated: false, inputIndex: 0, + }], + }), + }, + ]], + ])('refuses %s', (_name, rows) => { + expect(() => decode(rows)).toThrow() + }) + + it('refuses a relative restored cwd', () => { + expect(() => decode([], { ...header, cwd: 'relative' })).toThrow(/cwd must be absolute/) + }) + + it('accepts complete core, retry, title, compaction, PTC, and replacement relationships', () => { + const rows = [ + { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }, + { type: 'user/message', seq: 1, time: 2, data: user('one'), surfaceOp: 'append' }, + { type: 'step/start', seq: 2, time: 3, data: { turn: 1, step: 1 } }, + { + type: 'request/header', seq: 3, time: 4, + data: { header: { config: { provider: 'mock', model: 'mock' } }, reason: 'initial' }, + }, + { + type: 'llm/retry', seq: 4, time: 5, + data: { + retryId: 'retry', turn: 1, step: 1, provider: 'mock', mode: 'normal', policyKey: 'default', + retry: 1, maxRetries: 2, delayMs: 1, failure: { message: 'retry', code: 'SERVER' }, + }, + }, + { type: 'llm/retry-started', seq: 5, time: 6, data: { retryId: 'retry', turn: 1, step: 1, retry: 1 } }, + { + type: 'tool/code-dispatch-start', seq: 6, time: 7, + data: { rootCallId: 'root', parentCallId: 'root', subCallId: 'sub', name: 'read', arguments: {} }, + }, + { + type: 'tool/code-dispatch', seq: 7, time: 8, + data: { + rootCallId: 'root', parentCallId: 'root', subCallId: 'sub', name: 'read', arguments: {}, + isError: false, content: [], + }, + }, + { type: 'step/end', seq: 8, time: 9, data: { turn: 1, step: 1 } }, + { type: 'session/title', seq: 9, time: 10, data: { title: 'Title', messageSeqs: [1], source: { kind: 'fallback' } } }, + { type: 'compaction/start', seq: 10, time: 11, data: { compactionId: 'c', turn: 1 } }, + { type: 'compaction/end', seq: 11, time: 12, data: { compactionId: 'c', turn: 1, error: 'skipped' } }, + { + type: 'user/message', seq: 12, time: 13, data: user('replacement'), sourceEventSeqs: [1], + surfaceOp: { op: 'replace', start: 1, end: 1 }, + }, + { type: 'turn/end', seq: 13, time: 14, data: { turn: 1, reason: { kind: 'completed' } } }, + ] + expect(decode(rows).events).toEqual(rows) + }) + + it('preserves merge-extensible nested union variants and ignorable current events', () => { + const rows = [ + { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }, + { + type: 'user/message', seq: 1, time: 2, surfaceOp: 'append', + data: { + id: 'plugin-message', role: 'user', + content: [{ type: 'plugin/block', value: { nested: true } }], + source: { kind: 'plugin/source', detail: 'opaque' }, + }, + }, + { type: 'step/start', seq: 2, time: 3, data: { turn: 1, step: 1 } }, + { + type: 'assistant/chunk', seq: 3, time: 4, + data: { turn: 1, step: 1, chunk: { type: 'finish', reason: { kind: 'plugin-finish', detail: 1 } } }, + }, + { type: 'step/end', seq: 4, time: 5, data: { turn: 1, step: 1 } }, + { type: 'turn/end', seq: 5, time: 6, data: { turn: 1, reason: { kind: 'plugin-turn', detail: 1 } } }, + { type: 'plugin/informational', seq: 6, time: 7, data: { value: 1 }, ignorable: true }, + ] + expect(decode(rows).events).toEqual(rows) + expect(() => decode([{ type: 'plugin/required', seq: 0, time: 1, data: {} }])) + .toThrow(SessionFormatUnsupportedMigrationError) + for (const data of ['value', [1], null]) { + expect(decode([{ type: 'plugin/scalar', seq: 0, time: 1, data, ignorable: true }]).events[0]?.data) + .toEqual(data) + } + }) + + it('binds advertised, started, repaired, and resolved tool lifecycles', () => { + const turnAndStep = [ + { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }, + { type: 'step/start', seq: 1, time: 2, data: { turn: 1, step: 1 } }, + ] + const advertised = { + type: 'assistant/message', seq: 2, time: 3, surfaceOp: 'append', + data: { + turn: 1, step: 1, + message: { + id: 'assistant-tools', role: 'assistant', + content: [{ type: 'tool-call', id: 'a', name: 'read', arguments: '{}' }], + source: { kind: 'model', provider: 'mock', model: 'mock' }, + }, + }, + } + const started = { + type: 'tool/call', seq: 3, time: 4, + data: { turn: 1, step: 1, callId: 'a', name: 'read', arguments: '{}' }, + } + const result = (seq: number, sourceId = 'a', blockId = 'a') => ({ + type: 'tool/result', seq, time: seq + 1, surfaceOp: 'append', + data: { + turn: 1, step: 1, + message: { + id: 'result', role: 'user', + content: [{ type: 'tool-result', toolCallId: blockId, content: [], isError: false }], + source: { kind: 'tool', callId: sourceId }, + }, + }, + }) + + expect(() => decode([...turnAndStep, advertised, { ...started, data: { ...started.data, name: 'write' } }])) + .toThrow(/advertised tool call/) + expect(() => decode([...turnAndStep, { ...started, seq: 2, time: 3 }])).toThrow(/advertised tool call/) + expect(() => decode([...turnAndStep, result(2)])).toThrow(/no advertised tool lifecycle/) + expect(() => decode([...turnAndStep, { + ...advertised, + data: { + ...advertised.data, + message: { + ...advertised.data.message, + content: [ + ...advertised.data.message.content, + { type: 'tool-call', id: 'a', name: 'read', arguments: '{}' }, + ], + }, + }, + }])).toThrow(/repeats advertised tool call/) + expect(() => decode([...turnAndStep, advertised, started, result(4, 'b')])).toThrow(/tool-result block|tool lifecycle/) + expect(() => decode([...turnAndStep, advertised, { + type: 'step/end', seq: 3, time: 4, data: { turn: 1, step: 1 }, + }])).toThrow(/unresolved tool call/) + expect(() => decode([...turnAndStep, advertised, started, { + type: 'turn/end', seq: 4, time: 5, data: { turn: 1, reason: { kind: 'completed' } }, + }])).toThrow(/open step|unresolved tool call/) + expect(decode([...turnAndStep, advertised]).events).toHaveLength(3) + expect(decode([...turnAndStep, advertised, started]).events).toHaveLength(4) + expect(decode([...turnAndStep, advertised, started, result(4)]).events).toHaveLength(5) + + const synthetic = { + ...result(3), + data: { + turn: 1, + step: 1, + message: { + id: 'interrupted-tool-result-a-3', role: 'user', content: [{ + type: 'tool-result', toolCallId: 'a', isError: true, + content: [{ + type: 'text', + text: 'The tool call was interrupted before the Harness recorded it as started. Retry it if it is still needed.', + }], + }], + source: { kind: 'tool', callId: 'a' }, + }, + error: { name: 'ToolNotStartedError', code: 'TOOL_NOT_STARTED' }, + }, + } + expect(decode([...turnAndStep, advertised, synthetic]).events).toHaveLength(4) + expect(() => decode([...turnAndStep, advertised, { + ...synthetic, + data: { ...synthetic.data, error: { name: 'Other', code: 'TOOL_NOT_STARTED' } }, + }])).toThrow(/TOOL_NOT_STARTED repair/) + }) + + it('enforces retry mode, failure, route, and policy-chain relationships', () => { + const prefix = [ + { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }, + { type: 'step/start', seq: 1, time: 2, data: { turn: 1, step: 1 } }, + { type: 'request/header', seq: 2, time: 3, data: { header: { config: { provider: 'p', model: 'm' } }, reason: 'initial' } }, + ] + const retry = (overrides: Record = {}) => ({ + type: 'llm/retry', seq: 3, time: 4, + data: { + retryId: 'r', turn: 1, step: 1, provider: 'p', mode: 'normal', policyKey: 'k', retry: 1, + maxRetries: 2, delayMs: 1, failure: { message: 'x', code: 'X' }, ...overrides, + }, + }) + expect(() => decode([...prefix, retry({ maxRetries: undefined })])).toThrow() + expect(() => decode([...prefix, retry({ retry: 2, maxRetries: 1 })])).toThrow() + expect(() => decode([...prefix, retry({ provider: 'q' })])).toThrow(/provider/) + expect(() => decode([...prefix, retry({ failure: { message: 'x', code: 'X', status: 99 } })])).toThrow(/status/) + expect(() => decode([...prefix, retry({ delayMs: 2_147_483_648 })])).toThrow(/timer/) + expect(() => decode([...prefix, retry({ failure: { message: 'x', code: 'X', providerRetryAfterMs: 0 } })])).toThrow(/positive/) + expect(decode([...prefix, retry({ failure: { message: 'x', code: 'X', providerRetryAfterMs: 1.5 } })]).events) + .toHaveLength(4) + }) + + it('enforces command pairing and authoritative source-event rules', () => { + expect(() => decode([{ + type: 'command/done', seq: 0, time: 1, data: { commandId: 'c', kind: 'success' }, + }])).toThrow(/no prior command\/run/) + const run = { type: 'command/run', seq: 0, time: 1, data: { commandId: 'c', name: 'x', source: { kind: 'user' } } } + expect(() => decode([run, { + type: 'command/done', seq: 1, time: 2, + data: { commandId: 'c', kind: 'error', text: 'x', sourceEventSeq: 0 }, + }])).toThrow(/sourceEventSeq/) + expect(decode([run, { + type: 'command/done', seq: 1, time: 2, data: { commandId: 'c', kind: 'success' }, + }]).events).toHaveLength(2) + expect(() => decode([run, { + type: 'command/done', seq: 1, time: 2, + data: { commandId: 'c', kind: 'success', sourceEventSeq: 0 }, + }])).toThrow(/sourceEventSeq/) + }) + + it('enforces title source cardinality and exact auxiliary framing', () => { + const human = { type: 'user/message', seq: 1, time: 2, data: user('human'), surfaceOp: 'append' } + const prefix = [{ type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }, human] + expect(() => decode([...prefix, { + type: 'session/title', seq: 2, time: 3, + data: { title: 'bad', messageSeqs: [], source: { kind: 'fallback' } }, + }])).toThrow(/empty exactly/) + expect(() => decode([...prefix, { + type: 'session/title', seq: 2, time: 3, + data: { title: 'bad', messageSeqs: [1], source: { kind: 'user' } }, + }])).toThrow(/empty exactly/) + expect(() => decode([...prefix, { + type: 'session/title-llm-request', seq: 2, time: 3, + data: { + titleProvider: 'p', messageSeqs: [1], route: { provider: 'p', model: 'm' }, system: 's', + messages: [user('unrelated', { kind: 'plugin', plugin: 'dsh-session-title-llm' })], maxTokens: 1, + }, + }])).toThrow(/do not represent/) + }) + + it('validates own delivery ids while preserving inherited ancestor markers', () => { + const marker = (sessionId: string, version: number | undefined, seq: number) => ({ + type: 'session-log-deepseek/delivery-accepted', seq, time: seq + 1, + data: { sessionId, ...(version === undefined ? {} : { sessionFormatVersion: version }), throughSeq: 0 }, + }) + const prefix = [{ type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }] + expect(() => decode([...prefix, marker('wrong', 1, 1)])).toThrow(/wrong Session/) + const seeded = { + type: 'session', version: 1, id: 'child', createdAt: 1, parentSession: 'parent', + seedLength: 2, delegationDepth: 1, + } + expect(decode([...prefix, marker('ancestor-of-parent', 1, 1)], seeded).events).toHaveLength(2) + + const inertV0 = { + type: 'session-log-deepseek/delivery-accepted', seq: 1, time: 2, + data: { sessionId: 9, sessionFormatVersion: 0, throughSeq: { futureCoordinate: true } }, + } + const decoded = decode([...prefix, inertV0]) + expect(decoded.events[1]).toEqual(inertV0) + expect(() => { sessionFormatV0ToV1.validateTarget(decoded) }).not.toThrow() + }) + + it('validates versioned subagent descriptors by source/current policy', () => { + const future = { + type: 'subagent/descriptor', seq: 0, time: 1, + data: { version: 4, future: true }, + } + expect(decode([future]).events).toEqual([future]) + const v0Header = { ...header, version: 0 } + expect(() => sessionFormatV0ToV1.migrate(releasedV0SessionFormatCodec.decodeArtifact(v0Header, [future]))) + .toThrow(SessionFormatUnsupportedMigrationError) + }) + + it('enforces compaction ownership, summaries, turn boundaries, and surface spans', () => { + const startTurn = { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } } + const start = (seq: number, turn: number | null = 1) => ({ + type: 'compaction/start', seq, time: seq + 1, data: { compactionId: 'c', turn }, + }) + const end = (seq: number, data: Record = {}) => ({ + type: 'compaction/end', seq, time: seq + 1, + data: { compactionId: 'c', turn: 1, ...data }, + }) + const summary = (seq: number) => ({ + type: 'compaction/summary', seq, time: seq + 1, + data: { + compactionId: 'c', summary: [{ type: 'text', text: 'summary' }], + shadowedRange: { start: 1, end: 1 }, shadowedSeqs: [1], shadowedTokenCount: 1, + provider: 'p', model: 'm', + }, + }) + expect(() => decode([startTurn, start(1, null)])).toThrow(/open turn/) + expect(() => decode([startTurn, start(1), start(2)])).toThrow(/overlaps/) + expect(() => decode([startTurn, start(1), end(2, { turn: 2, error: 'x' })])).toThrow(/owner turn/) + expect(() => decode([startTurn, start(1), end(2)])).toThrow(/requires one summary/) + expect(() => decode([startTurn, start(1), { + type: 'turn/end', seq: 2, time: 3, data: { turn: 1, reason: { kind: 'completed' } }, + }])).toThrow(/crosses/) + expect(() => decode([startTurn, start(1), { + ...summary(2), + data: { ...summary(2).data, shadowedRange: { start: 0, end: 0 }, shadowedSeqs: [0] }, + }])).toThrow(/surface span/) + const userEvent = { type: 'user/message', seq: 1, time: 2, data: user('one'), surfaceOp: 'append' } + expect(() => decode([startTurn, userEvent, start(2), summary(3), summary(4)])).toThrow(/repeats/) + expect(() => decode([startTurn, { + type: 'compaction/prune', seq: 1, time: 2, + data: { shadowedRange: { start: 0, end: 0 }, shadowedSeqs: [0], shadowedTokenCount: 1 }, + }])).toThrow(/surface span/) + expect(() => decode([startTurn, { + type: 'user/message', seq: 1, time: 2, data: user('base'), surfaceOp: 'append', + }, { + type: 'user/message', seq: 2, time: 3, data: { + ...user('checkpoint'), source: { kind: 'plugin', plugin: 'compact', compactionId: 'missing' }, + }, sourceEventSeqs: [1], surfaceOp: { op: 'replace', start: 1, end: 1 }, + }])).toThrow() + }) + + it('requires each PTC settle to match exactly one start', () => { + const turn = { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } } + const start = { + type: 'tool/code-dispatch-start', seq: 1, time: 2, + data: { rootCallId: 'root', parentCallId: 'root', subCallId: 'sub', name: 'a', arguments: { x: 1 } }, + } + const settle = (seq: number, overrides: Record = {}) => ({ + type: 'tool/code-dispatch', seq, time: seq + 1, + data: { + rootCallId: 'root', parentCallId: 'root', subCallId: 'sub', name: 'a', arguments: { x: 1 }, + isError: false, content: [], ...overrides, + }, + }) + expect(() => decode([turn, start, settle(2, { name: 'b' })])).toThrow(/does not match/) + expect(() => decode([turn, start, settle(2, { arguments: { x: 2 } })])).toThrow(/does not match/) + expect(() => decode([turn, start, settle(2), settle(3)])).toThrow(/unique start/) + expect(decode([turn, start]).events).toHaveLength(2) + const arrayStart = { + ...start, + data: { ...start.data, subCallId: 'array', arguments: [1, { x: 2 }] }, + } + const arraySettle = { + ...settle(2), + data: { ...settle(2).data, subCallId: 'array', arguments: [1, { x: 2 }] }, + } + expect(decode([turn, arrayStart, arraySettle]).events).toHaveLength(3) + expect(() => decode([ + turn, + arrayStart, + { ...arraySettle, data: { ...arraySettle.data, arguments: [1, { x: 2 }, 3] } }, + ])).toThrow(/does not match/) + }) + + it('refuses a wrong own-session v0 delivery marker before the header bump', () => { + const v0 = { ...header, version: 0 } + const rows = [ + { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }, + { + type: 'session-log-deepseek/delivery-accepted', seq: 1, time: 2, + data: { sessionId: 'wrong', throughSeq: 0 }, + }, + ] + expect(() => sessionFormatV0ToV1.migrate(releasedV0SessionFormatCodec.decodeArtifact(v0, rows))) + .toThrow(/wrong Session/) + }) + + it('accepts recoverable open tails for later interrupted-turn repair', () => { + const v0 = { ...header, version: 0 } + const rows = [ + { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }, + { type: 'step/start', seq: 1, time: 2, data: { turn: 1, step: 1 } }, + { type: 'request/header', seq: 2, time: 3, data: { header: { config: { provider: 'p', model: 'm' } }, reason: 'initial' } }, + { + type: 'llm/retry', seq: 3, time: 4, + data: { + retryId: 'r', turn: 1, step: 1, provider: 'p', mode: 'normal', policyKey: 'k', retry: 1, + maxRetries: 2, delayMs: 1, failure: { message: 'x', code: 'X' }, + }, + }, + { + type: 'assistant/message', seq: 4, time: 5, surfaceOp: 'append', + data: { + turn: 1, step: 1, + message: { + id: 'tail-assistant', role: 'assistant', + content: [{ type: 'tool-call', id: 'call', name: 'read', arguments: '{}' }], + source: { kind: 'model', provider: 'p', model: 'm' }, + }, + }, + }, + { type: 'tool/call', seq: 5, time: 6, data: { turn: 1, step: 1, callId: 'call', name: 'read', arguments: '{}' } }, + { + type: 'tool/code-dispatch-start', seq: 6, time: 7, + data: { rootCallId: 'root', parentCallId: 'root', subCallId: 'sub', name: 'read', arguments: {} }, + }, + { type: 'compaction/start', seq: 7, time: 8, data: { compactionId: 'c', turn: 1 } }, + ] + const recovered = releasedV0SessionFormatCodec.decodeRecoverableArtifact(v0, rows) + expect(sessionFormatV0ToV1.migrate(recovered).events).toHaveLength(rows.length) + }) + + it('refuses invalid core openings, request placement, and replacement placement', () => { + expect(() => decode([ + { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }, + { type: 'turn/start', seq: 1, time: 2, data: { turn: 2 } }, + ])).toThrow(/expected turn/) + expect(() => decode([ + { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }, + { type: 'step/start', seq: 1, time: 2, data: { turn: 1, step: 2 } }, + ])).toThrow(/next step/) + expect(() => decode([{ + type: 'request/header', seq: 0, time: 1, + data: { header: { config: { provider: 'p', model: 'm' } }, reason: 'initial' }, + }])).toThrow(/outside an open turn/) + expect(() => decode([{ + type: 'request/context', seq: 0, time: 1, data: { provider: 'p', model: 'm' }, + }])).toThrow(/outside an open turn/) + expect(() => decode([ + { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }, + { type: 'step/start', seq: 1, time: 2, data: { turn: 1, step: 1 } }, + { type: 'turn/end', seq: 2, time: 3, data: { turn: 1, reason: { kind: 'completed' } } }, + ])).toThrow(/crosses an open step/) + const base = { type: 'user/message', seq: 0, time: 1, data: user('base'), surfaceOp: 'append' } + const replacement = { + type: 'tool/result', seq: 1, time: 2, sourceEventSeqs: [0], + surfaceOp: { op: 'replace', start: 0, end: 0 }, + data: { + turn: 1, step: 1, + message: { + id: 'r', role: 'user', content: [{ type: 'tool-result', toolCallId: 'c', content: [] }], + source: { kind: 'tool', callId: 'c' }, + }, + }, + } + expect(() => decode([base, replacement])).toThrow(/outside an open turn/) + const { surfaceOp: _surfaceOp, ...withoutSurface } = base + expect(() => decode([withoutSurface])).toThrow(/surfaceOp/) + expect(() => decode([base, { + type: 'feedback/record', seq: 1, time: 2, data: { text: 'log-only' }, + }, { + type: 'user/message', seq: 2, time: 3, data: user('missing'), sourceEventSeqs: [1], + surfaceOp: { op: 'replace', start: 1, end: 1 }, + }])).toThrow(/not on the current surface/) + }) + + it('rejects PTC ancestry/root/start violations', () => { + const turn = { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } } + const start = (seq: number, overrides: Record = {}) => ({ + type: 'tool/code-dispatch-start', seq, time: seq + 1, + data: { rootCallId: 'root', parentCallId: 'root', subCallId: 'sub', name: 'a', arguments: {}, ...overrides }, + }) + expect(() => decode([turn, start(1, { parentCallId: 'missing' })])).toThrow(/parentCallId/) + expect(() => decode([turn, start(1), start(2, { rootCallId: 'other' })])).toThrow(/rootCallId/) + expect(() => decode([turn, start(1), start(2)])).toThrow(/repeats subCallId/) + }) + + it('rejects repeated retry starts and invalid retry chains', () => { + const prefix = [ + { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }, + { type: 'step/start', seq: 1, time: 2, data: { turn: 1, step: 1 } }, + { type: 'request/header', seq: 2, time: 3, data: { header: { config: { provider: 'p', model: 'm' } }, reason: 'initial' } }, + ] + const retry = (seq: number, retry: number, retryId = 'r', policyKey = 'k') => ({ + type: 'llm/retry', seq, time: seq + 1, + data: { + retryId, turn: 1, step: 1, provider: 'p', mode: 'normal', policyKey, retry, + maxRetries: 3, delayMs: 1, failure: { message: 'x', code: 'X' }, + }, + }) + const started = (seq: number, step = 1) => ({ + type: 'llm/retry-started', seq, time: seq + 1, data: { retryId: 'r', turn: 1, step, retry: 1 }, + }) + expect(() => decode([...prefix, retry(3, 1), started(4, 2)])).toThrow(/scheduled turn/) + expect(() => decode([...prefix, retry(3, 1), started(4), started(5)])).toThrow(/repeats/) + expect(() => decode([...prefix, retry(3, 2)])).toThrow(/retry 1/) + expect(() => decode([...prefix, retry(3, 1), retry(4, 2, 'other')])).toThrow(/preserve retryId/) + expect(() => decode([...prefix, retry(3, 1), retry(4, 1, 'r', 'other')])).toThrow(/reuses retryId/) + expect(decode([...prefix, retry(3, 1), retry(4, 2)]).events).toHaveLength(5) + }) + + it('rejects duplicate commands and clears inherited orphan compactions at end-seed', () => { + const run = (seq: number) => ({ + type: 'command/run', seq, time: seq + 1, data: { commandId: 'c', name: 'x', source: { kind: 'user' } }, + }) + expect(() => decode([run(0), run(1)])).toThrow(/repeats commandId/) + const rows = [ + { type: 'compaction/start', seq: 0, time: 1, data: { compactionId: 'c', turn: null } }, + { type: 'session/end-seed', seq: 1, time: 2, data: {} }, + { type: 'turn/start', seq: 2, time: 3, data: { turn: 1 } }, + ] + expect(decode(rows).events).toEqual(rows) + expect(decode([{ type: 'session/end-seed', seq: 0, time: 1, data: {} }]).events).toHaveLength(1) + }) + + it('accepts exact title-LLM framing and rejects non-human citations', () => { + const direct = { type: 'user/message', seq: 1, time: 2, data: user('human'), surfaceOp: 'append' } + const framed = 'Generate the session title from this JSON array of human messages:\n' + + JSON.stringify([{ seq: 1, text: 'human' }]) + const request = { + type: 'session/title-llm-request', seq: 2, time: 3, + data: { + titleProvider: 'p', messageSeqs: [1], route: { provider: 'p', model: 'm' }, system: 's', maxTokens: 1, + messages: [{ + id: 'framed', role: 'user', content: [{ type: 'text', text: framed }], + source: { kind: 'plugin', plugin: 'dsh-session-title-llm' }, + }], + }, + } + const prefix = [{ type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }, direct] + expect(decode([...prefix, request]).events).toHaveLength(3) + const mixedHuman = { + type: 'user/message', seq: 1, time: 2, surfaceOp: 'append', + data: { + id: 'mixed', role: 'user', + content: [{ type: 'reasoning', text: 'hidden' }, { type: 'text', text: 'visible' }], + source: { kind: 'user' }, + }, + } + const mixedFramed = 'Generate the session title from this JSON array of human messages:\n' + + JSON.stringify([{ seq: 1, text: 'visible' }]) + const mixedRequest = { + ...request, + data: { + ...request.data, + messages: [{ + id: 'mixed-frame', role: 'user', content: [{ type: 'text', text: mixedFramed }], + source: { kind: 'plugin', plugin: 'dsh-session-title-llm' }, + }], + }, + } + expect(decode([prefix[0], mixedHuman, mixedRequest]).events).toHaveLength(3) + expect(() => decode([ + { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }, + { type: 'user/message', seq: 1, time: 2, data: user('plugin', { kind: 'plugin', plugin: 'x' }), surfaceOp: 'append' }, + { type: 'session/title', seq: 2, time: 3, data: { title: 'x', messageSeqs: [1], source: { kind: 'fallback' } } }, + ])).toThrow(/human user/) + expect(() => decode([...prefix, { + ...request, + data: { ...request.data, messages: [] }, + }])).toThrow(/do not represent/) + }) + + it('accepts turn-enclosed request context and refuses step work without an open step', () => { + const turn = { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } } + expect(decode([turn, { + type: 'request/context', seq: 1, time: 2, data: { provider: 'p', model: 'm' }, + }]).events).toHaveLength(2) + expect(() => decode([turn, { + type: 'assistant/chunk', seq: 1, time: 2, + data: { turn: 1, step: 1, chunk: { type: 'text-delta', index: 0, text: 'x' } }, + }])).toThrow(/open turn and step/) + }) +}) diff --git a/packages/session/session-format-v0-to-v1/tests/validation.spec.ts b/packages/session/session-format-v0-to-v1/tests/validation.spec.ts new file mode 100644 index 0000000000..be90d4f21e --- /dev/null +++ b/packages/session/session-format-v0-to-v1/tests/validation.spec.ts @@ -0,0 +1,678 @@ +import { describe, expect, it } from 'vitest' +import type { SessionFormatEvent, SessionFormatJsonValue } from '@deepseek-ai/dsh-session-format' +import { KNOWN_SESSION_EVENT_TYPES } from '@deepseek-ai/dsh-session' +import { + RELEASED_V0_EVENT_TYPES, + RELEASED_V0_EVENT_DISPOSITIONS, + assertReleasedV1Artifact, + releasedV0SessionFormatCodec, + releasedV1SessionFormatCodec, + sessionFormatV0ToV1, +} from '../src/index.ts' +import { assertReleasedEventPayload } from '../src/validation.ts' + +const v0Header = { + type: 'session', version: 0, id: 'validation', createdAt: 1, delegationDepth: 0, +} as const +const v1Header = { ...v0Header, version: 1 } as const +const textBlock = { type: 'text', text: 'text' } as const +const userMessage = { + id: 'user-1', role: 'user', content: [textBlock], source: { kind: 'user' }, +} as const +const assistantMessage = { + id: 'assistant-1', role: 'assistant', content: [textBlock], + source: { kind: 'model', provider: 'mock', model: 'mock' }, +} as const +const toolMessage = { + id: 'tool-1', role: 'user', + content: [{ type: 'tool-result', toolCallId: 'call-1', content: [textBlock], isError: false }], + source: { kind: 'tool', callId: 'call-1' }, +} as const + +const validPayloads: Readonly> = { + 'agent-preset/selected': { agentPreset: 'default' }, + 'agent/inbox/spliced': { target: 'next-turn', start: 0, inserted: [userMessage] }, + 'approval/asked': { id: 'approval-1', toolName: 'bash', callId: 'call-1', reason: 'needed' }, + 'approval/decided': { id: 'approval-1', outcome: 'allowed-once' }, + 'approval/policy': { policy: 'ask', source: 'delegation' }, + 'assistant/chunk': { turn: 1, step: 0, chunk: { type: 'text-delta', index: 0, text: 'x' } }, + 'assistant/message': { turn: 1, step: 0, message: assistantMessage, usage: { inputTokens: 1, outputTokens: 1 } }, + 'command/done': { commandId: 'command-1', kind: 'success', sourceEventSeq: 0 }, + 'command/run': { commandId: 'command-1', name: 'compact', args: ' now', source: { kind: 'user' } }, + 'compaction/end': { compactionId: 'compact-1', turn: null }, + 'compaction/prune': { shadowedRange: { start: 0, end: 1 }, shadowedSeqs: [0, 1], shadowedTokenCount: 2 }, + 'compaction/start': { compactionId: 'compact-1', turn: 1 }, + 'compaction/summary': { + compactionId: 'compact-1', summary: [textBlock], + shadowedRange: { start: 0, end: 1 }, shadowedSeqs: [0, 1], shadowedTokenCount: 2, + provider: 'mock', model: 'mock', rawOutput: [textBlock], llmStreamCall: true, + }, + 'feedback/record': { text: 'feedback' }, + 'goal/change': { + kind: 'goal/change', version: 1, operation: 'create', + goal: { id: 'goal-1', revision: 1, objective: 'ship', phase: 'active', maxGoalRounds: 3 }, + roundsStarted: 0, createdAt: 1, updatedAt: 1, + }, + 'hook/invoked': { turn: 1, point: 'PreToolUse', dialect: 'claude-code', handlerId: 'hook-1' }, + 'hook/result': { turn: 1, point: 'PreToolUse', handlerId: 'hook-1', decision: 'pass', durationMs: 1 }, + 'llm/retry': { + retryId: 'retry-1', turn: 1, step: 0, provider: 'mock', mode: 'normal', policyKey: 'default', + retry: 1, maxRetries: 2, delayMs: 10, failure: { message: 'retry', code: 'SERVER' }, + }, + 'llm/retry-started': { retryId: 'retry-1', turn: 1, step: 0, retry: 1 }, + 'model/selection': { provider: 'mock', model: 'mock', reasoningEffort: 'high' }, + 'permission/preset': { preset: 'default' }, + 'plan/mode': { active: true }, + 'request/context': { provider: 'mock', model: 'mock', contextWindow: 8192 }, + 'request/header': { + header: { + config: { provider: 'mock', model: 'mock', reasoningEffort: 'high', maxTokens: 100 }, + adapterDefaults: { reasoningEffort: true, maxTokens: true }, + system: 'system', + tools: [{ name: 'tool', description: 'Tool', parameters: { type: 'object' } }], + }, + reason: 'initial', + }, + 'sandbox/mode': { mode: 'workspace-write', source: 'delegation' }, + 'schedule/change': { + version: 1, operation: 'create', + schedule: { id: 'schedule-1', kind: 'after', prompt: 'remember', afterSeconds: 60, scheduledAt: '2026-08-31T00:00:00.000Z' }, + }, + 'session-log-deepseek/delivery-accepted': { sessionId: 'validation', throughSeq: 0, sessionFormatVersion: 1 }, + 'session/end-seed': {}, + 'session/title': { title: 'Title', messageSeqs: [0], source: { kind: 'fallback' } }, + 'session/title-llm-request': { + titleProvider: 'title-1', messageSeqs: [0], route: { provider: 'mock', model: 'mock' }, + system: 'title', messages: [userMessage], maxTokens: 20, + }, + 'step/end': { turn: 1, step: 0 }, + 'step/start': { turn: 1, step: 0 }, + 'subagent/descriptor': { + mode: 'continuable', version: 3, provider: 'in-process', label: 'child', + agentProvider: 'mock', agentModel: 'mock', toolFilter: { allow: ['read'] }, + }, + 'subagent/model-selection-policy': { allowedModels: [{ provider: 'mock', model: 'mock' }] }, + 'team/member': { + version: 1, teamId: 'team-1', + member: { id: 'member-1', name: 'worker', description: 'work', provider: 'in-process', context: 'fresh', phase: 'active' }, + }, + 'team/message/delivered': { version: 1, teamId: 'team-1', messageId: 'message-1', targetId: 'member-1' }, + 'team/message/queued': { + version: 1, teamId: 'team-1', + message: { + id: 'message-1', senderId: 'lead', senderName: 'lead', targetId: 'member-1', + delivery: 'quiet', content: [textBlock], + }, + }, + 'team/task': { + version: 1, teamId: 'team-1', + task: { + id: 'task-1', revision: 1, subject: 'subject', description: 'description', status: 'pending', + blockedBy: [], writeScopes: ['/work'], + }, + }, + 'todo/write': { todos: [{ content: 'work', status: 'in_progress' }] }, + 'tool-workflow/agent-end': { runId: 'run-1', seq: 1, outcome: 'completed' }, + 'tool-workflow/agent-start': { runId: 'run-1', seq: 1, label: 'worker', phase: 'build', childId: 'child-1' }, + 'tool-workflow/run-end': { runId: 'run-1', stopReason: 'completed' }, + 'tool-workflow/run-start': { runId: 'run-1', name: 'workflow' }, + 'tool/call': { turn: 1, step: 0, callId: 'call-1', name: 'read', arguments: '{}' }, + 'tool/code-dispatch': { + rootCallId: 'root', parentCallId: 'parent', subCallId: 'sub', name: 'read', arguments: { path: '/work' }, + isError: false, content: [textBlock], + }, + 'tool/code-dispatch-start': { + rootCallId: 'root', parentCallId: 'parent', subCallId: 'sub', name: 'read', arguments: { path: '/work' }, + }, + 'tool/result': { turn: 1, step: 0, message: toolMessage, error: { name: 'Error', code: 'FAILED' }, meta: { seq: 999 } }, + 'turn/end': { turn: 1, reason: { kind: 'completed' } }, + 'turn/start': { turn: 1 }, + 'user/message': userMessage, + 'web/deepseek-search-llm-request': { + endpoint: 'https://example.test/messages', apiVersion: '2023-06-01', + body: { + model: 'deepseek-chat', max_tokens: 100, + messages: [{ role: 'user', content: [{ type: 'text', text: 'search' }] }], + tools: [{ type: 'web_search_20250305', name: 'web_search', max_uses: 1 }], + }, + }, +} + +function v1Artifact(type: string, data: SessionFormatJsonValue) { + const surface = type === 'user/message' || type === 'assistant/message' || type === 'tool/result' + const event = { + type, + seq: 3, + time: 4, + data, + ...(surface ? { surfaceOp: 'append' } : {}), + } + return { + header: { + version: 1, id: 'validation', createdAt: 1, isSeeded: false, delegationDepth: 0, + }, + inheritedEventCount: 0, + events: [ + { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }, + { type: 'step/start', seq: 1, time: 2, data: { turn: 1, step: 0 } }, + { type: 'step/end', seq: 2, time: 3, data: { turn: 1, step: 0 } }, + event, + ], + } +} + +function assertPayload(type: string, data: SessionFormatJsonValue): void { + assertReleasedEventPayload({ type, seq: 3, time: 4, data }, 1) +} + +function invalidLeafMutations( + type: string, + value: SessionFormatJsonValue, + path: readonly string[] = [], +): Array<{ readonly path: string; readonly value: SessionFormatJsonValue }> { + const joined = path.join('.') + const opaque = RELEASED_V0_EVENT_DISPOSITIONS[type]?.opaque.includes(path[0] as string) === true + || path.at(-1) === 'replayState' + || path.at(-1) === 'parameters' + if (opaque) return [] + const replacement: SessionFormatJsonValue = value === null + ? false + : typeof value === 'string' ? 1 + : typeof value === 'number' ? 'invalid' + : typeof value === 'boolean' ? 'invalid' + : Array.isArray(value) ? {} + : [] + const mutations: Array<{ readonly path: string; readonly value: SessionFormatJsonValue }> = path.length === 0 + ? [] + : [{ path: joined, value: replacement }] + if (Array.isArray(value)) { + const members = value as readonly SessionFormatJsonValue[] + members.forEach((member, index) => { + mutations.push(...invalidLeafMutations(type, member, [...path, String(index)])) + }) + } else if (typeof value === 'object' && value !== null) { + for (const [key, member] of Object.entries(value)) { + mutations.push(...invalidLeafMutations(type, member, [...path, key])) + } + } + return mutations +} + +function replaceAtPath(value: SessionFormatJsonValue, path: string, replacement: SessionFormatJsonValue): SessionFormatJsonValue { + const copy = structuredClone(value) + const keys = path.split('.') + let current = copy as unknown as Record + for (const key of keys.slice(0, -1)) current = current[key] as unknown as Record + current[keys.at(-1) as string] = replacement + return copy +} + +describe('released event and payload inventory', () => { + it('has an executable valid fixture for every frozen released-v0 event type', () => { + expect(Object.keys(validPayloads).sort()).toEqual([...RELEASED_V0_EVENT_TYPES].sort()) + expect(RELEASED_V0_EVENT_TYPES).toHaveLength(51) + expect(RELEASED_V0_EVENT_TYPES.every(type => KNOWN_SESSION_EVENT_TYPES.has(type))).toBe(true) + for (const [type, data] of Object.entries(validPayloads)) { + expect(() => { assertPayload(type, data) }, type).not.toThrow() + } + }) + + it('refuses an unexpected member on every known payload', () => { + for (const [type, data] of Object.entries(validPayloads)) { + const changed = { ...(data as Record), unexpected: true } + expect(() => { assertPayload(type, changed) }, type).toThrow(/unexpected member/) + } + }) + + it('refuses nested schema drift across core and owner payload families', () => { + const cases: Array<[string, SessionFormatJsonValue]> = [ + ['assistant/message', { + ...(validPayloads['assistant/message'] as Record), + message: { ...assistantMessage, content: [{ ...textBlock, extra: true }] }, + }], + ['llm/retry', { + ...(validPayloads['llm/retry'] as Record), + failure: { message: 'bad', code: 'BAD', extra: true }, + }], + ['request/header', { + ...(validPayloads['request/header'] as Record), + header: { config: { provider: 'mock', model: 'mock', extra: true } }, + }], + ['schedule/change', { + version: 1, operation: 'create', + schedule: { id: 'x', kind: 'at', prompt: 'x', scheduledAt: '2026-08-31T00:00:00.000Z', extra: true }, + }], + ['goal/change', { + ...(validPayloads['goal/change'] as Record), + goal: { id: 'g', revision: 1, objective: 'x', phase: 'active', maxGoalRounds: 1, extra: true }, + }], + ['team/task', { + ...(validPayloads['team/task'] as Record), + task: { + id: 'task-1', revision: 1, subject: 'x', description: 'x', status: 'pending', + blockedBy: [], writeScopes: [], extra: true, + }, + }], + ['todo/write', { todos: [{ content: 'x', status: 'pending', extra: true }] }], + ['web/deepseek-search-llm-request', { + ...(validPayloads['web/deepseek-search-llm-request'] as Record), + body: { model: 'x', max_tokens: 1, messages: [], tools: [], extra: true }, + }], + ] + for (const [index, [type, data]] of cases.entries()) { + expect(() => { assertPayload(type, data) }, `${type}-${index}`).toThrow() + } + }) + + it('refuses type corruption at every non-opaque nested member in the frozen fixture inventory', () => { + let mutations = 0 + for (const [type, data] of Object.entries(validPayloads)) { + for (const mutation of invalidLeafMutations(type, data)) { + const changed = replaceAtPath(data, mutation.path, mutation.value) + expect( + () => { assertPayload(type, changed) }, + `${type}.${mutation.path}`, + ).toThrow() + mutations += 1 + } + } + expect(mutations).toBeGreaterThan(250) + }) + + it('allows explicit opaque tool metadata and PTC arguments losslessly', () => { + for (const type of ['tool/result', 'tool/code-dispatch', 'tool/code-dispatch-start']) { + const data = structuredClone(validPayloads[type] as SessionFormatJsonValue) + expect(() => { assertPayload(type, data) }).not.toThrow() + expect(data).toEqual(validPayloads[type]) + } + }) + + it('refuses unknown v0 events even when the envelope marks them ignorable', () => { + const row = { type: 'plugin/unknown', seq: 0, time: 1, data: {}, ignorable: true } + expect(() => releasedV0SessionFormatCodec.decodeArtifact(v0Header, [row])) + .toThrow(/unknown historical event.*refuses.*ignorable/) + }) + + it('keeps capturedFormatVersion v1-only inside session-reference sources', () => { + const data = { + id: 'reference', role: 'user', content: [textBlock], + source: { + kind: 'session-reference', form: 'recall', version: 1, + references: [{ + sessionId: 'source', label: 'Source', capturedFormatVersion: 1, capturedThroughSeq: 0, + compacted: false, originalMessages: 1, retainedMessages: 1, omittedMessages: 0, + omittedBytes: 0, truncated: false, inputIndex: 0, + }], + }, + } + const event = { type: 'user/message', seq: 0, time: 1, data, surfaceOp: 'append' } + expect(() => sessionFormatV0ToV1.migrate(releasedV0SessionFormatCodec.decodeArtifact(v0Header, [event]))) + .toThrow(/capturedFormatVersion/) + expect(releasedV1SessionFormatCodec.decodeArtifact(v1Header, [event]).events).toEqual([event]) + }) + + it('accepts every released nested union variant and optional member', () => { + const sources: SessionFormatJsonValue[] = [ + { kind: 'user', rpcId: 'rpc-1', clientTimeZone: 'Asia/Shanghai' }, + { kind: 'plugin', plugin: 'plain' }, + { kind: 'plugin', plugin: 'instructions', form: 'instructions' }, + { kind: 'plugin', plugin: 'catalog', form: 'catalog' }, + { kind: 'plugin', plugin: 'snapshot', form: 'snapshot', sections: [{ name: 'one', text: 'value' }] }, + { kind: 'plugin', plugin: 'notice', form: 'notice', summary: 'notice' }, + { kind: 'plugin', plugin: 'relay', form: 'relay' }, + { kind: 'plugin', plugin: 'recall', form: 'recall' }, + { kind: 'plugin', plugin: 'compact', compactionId: 'compact-1', sourceCommandId: 'command-1' }, + { kind: 'model', provider: 'mock', model: 'mock', replayState: { private: true } }, + { kind: 'tool', callId: 'call-1' }, + { + kind: 'agent-instructions', form: 'instructions', baseline: true, baselineIdentity: 'base', + changes: [ + { action: 'set', scope: '.', path: 'AGENTS.md', digest: 'one' }, + { action: 'replace', scope: 'src', path: 'src/AGENTS.md' }, + { action: 'remove', scope: 'old', path: 'old/AGENTS.md' }, + ], + }, + { kind: 'agent-instructions', form: 'instructions', changes: [] }, + { + kind: 'session-reference', form: 'recall', version: 1, + references: [{ + sessionId: 'source', label: 'Source', capturedFormatVersion: 1, capturedThroughSeq: null, + compacted: true, originalMessages: 2, retainedMessages: 1, omittedMessages: 1, + omittedBytes: 20, truncated: true, inputIndex: 0, + }], + }, + { + kind: 'session-reference', form: 'recall', version: 1, + references: [{ + sessionId: 'source-without-version', label: 'Source', capturedThroughSeq: 0, + compacted: false, originalMessages: 1, retainedMessages: 1, omittedMessages: 0, + omittedBytes: 0, truncated: false, inputIndex: 0, + }], + }, + { kind: 'team-message', teamId: 'team', messageId: 'message', senderId: 'sender', senderName: 'Sender' }, + { kind: 'goal', goalId: 'goal', revision: 1, round: 1 }, + { kind: 'skill-invocation', name: 'skill', form: 'instructions' }, + { kind: 'skill-catalog', form: 'catalog', update: true, entries: [{ name: 'skill', description: 'Skill' }] }, + { kind: 'skill-catalog', form: 'catalog', entries: [] }, + { kind: 'coordinator', form: 'relay', senderSessionId: 'parent' }, + { kind: 'subagent-report', form: 'relay', senderSessionId: 'child' }, + { kind: 'subagent-settled', form: 'notice', summary: 'settled', senderSessionId: 'child' }, + { + kind: 'webhook', provider: 'github', source: 'repo', deliveryId: 'delivery', ruleId: 'rule', + form: 'notice', summary: 'push', + }, + ] + for (const [index, source] of sources.entries()) { + const message = { id: `source-${index}`, role: 'user', content: [textBlock], source } + expect(() => { assertPayload('user/message', message) }).not.toThrow() + } + + const blocks: SessionFormatJsonValue[] = [ + { type: 'reasoning', text: 'reasoning' }, + { + type: 'image', attachment: { + attachmentId: 'image', mediaType: 'image/png', bytes: 10, width: 2, height: 2, name: 'x.png', + originalDimensions: { width: 4, height: 4 }, + }, + }, + { + type: 'image', attachment: { + attachmentId: 'minimal-image', mediaType: 'image/jpeg', bytes: 1, width: 1, height: 1, + }, + }, + { type: 'tool-call', id: 'call', name: 'read', arguments: '{}' }, + { type: 'tool-result', toolCallId: 'call', content: [textBlock], isError: true }, + ] + for (const [index, block] of blocks.entries()) { + const message = { ...userMessage, id: `block-${index}`, content: [block] } + expect(() => { assertPayload('user/message', message) }).not.toThrow() + } + + const chunks: SessionFormatJsonValue[] = [ + { type: 'block-start', index: 0, blockType: 'text' }, + { type: 'reasoning-delta', index: 0, text: 'r' }, + { type: 'tool-call-delta', index: 0, id: 'call', name: 'read', argumentsDelta: '{}' }, + { type: 'tool-call-delta', index: 0, id: 'call', argumentsDelta: '{}' }, + { type: 'block-end', index: 0, block: textBlock }, + { type: 'usage', usage: { + inputTokens: 1, outputTokens: 2, totalTokens: 3, cacheReadTokens: 0, + cacheWriteTokens: 0, reasoningTokens: 1, + } }, + { type: 'finish', reason: { kind: 'stop' }, replayState: { response: { id: 'response' }, blocks: [{}] } }, + { type: 'finish', reason: { kind: 'tool-calls' } }, + { type: 'finish', reason: { kind: 'max-tokens' } }, + { type: 'finish', reason: { kind: 'aborted', failure: { message: 'abort', code: 'ABORT' } } }, + { type: 'finish', reason: { kind: 'error', failure: { message: 'error', code: 'ERROR' } } }, + ] + for (const chunk of chunks) { + expect(() => { assertPayload('assistant/chunk', { turn: 1, step: 0, chunk }) }).not.toThrow() + } + + const turnReasons: SessionFormatJsonValue[] = [ + { kind: 'blocked' }, { kind: 'max-tokens' }, { kind: 'interrupted' }, + { kind: 'aborted', reason: { kind: 'user' } }, + { kind: 'aborted', reason: { kind: 'parent' } }, + { kind: 'aborted', reason: { kind: 'disposed' } }, + { kind: 'aborted', reason: { kind: 'legacy' } }, + { kind: 'aborted', reason: { kind: 'hook', reason: 'hook' } }, + { kind: 'error', error: { message: 'error', code: 'ERROR', status: 500, providerRetryAfterMs: 1, requestId: 'id' } }, + ] + for (const reason of turnReasons) { + expect(() => { assertPayload('turn/end', { turn: 1, reason }) }).not.toThrow() + } + + const remaining: Array<[string, SessionFormatJsonValue]> = [ + ['goal/change', { kind: 'goal/change', version: 1, operation: 'clear', cleared: { id: 'g', revision: 2 }, clearedAt: 3 }], + ['goal/change', { + kind: 'goal/change', version: 1, operation: 'block', + goal: { + id: 'g', revision: 2, objective: 'x', phase: 'blocked', maxGoalRounds: 3, + blockedReason: { code: 'waiting', message: 'Wait' }, + }, + roundsStarted: 1, createdAt: 1, updatedAt: 2, + }], + ['schedule/change', { version: 1, operation: 'create', schedule: { id: 'at', kind: 'at', prompt: 'x', scheduledAt: '2026-08-31T00:00:00.000Z' } }], + ['schedule/change', { version: 1, operation: 'create', schedule: { id: 'every', kind: 'every', prompt: 'x', everySeconds: 300, scheduledAt: '2026-08-31T00:00:00.000Z' } }], + ['schedule/change', { version: 1, operation: 'delete', id: 'at' }], + ['schedule/change', { version: 1, operation: 'dispatch', id: 'every', acceptedAt: '2026-08-31T00:00:00.000Z' }], + ['llm/retry', { + retryId: 'r', turn: 1, step: 0, provider: 'p', mode: 'always', policyKey: 'k', retry: 1, + delayMs: 0, failure: { message: 'x', code: 'X' }, + }], + ['session/title', { title: 'User title', messageSeqs: [], source: { kind: 'user' } }], + ['session/title', { title: 'Provider title', messageSeqs: [0], source: { kind: 'provider', provider: 'p', model: { provider: 'p', model: 'm' } } }], + ['session/title', { title: 'Provider title', messageSeqs: [0], source: { kind: 'provider', provider: 'p' } }], + ['subagent/descriptor', { mode: 'one-shot', version: 3, provider: 'p', label: 'child' }], + ['subagent/descriptor', { mode: 'one-shot', version: 3, provider: 'p' }], + ['subagent/descriptor', { + mode: 'continuable', version: 3, provider: 'p', label: 'child', agentProvider: 'p', agentModel: 'm', + agentReasoningEffort: 'high', persona: 'persona', toolFilter: { deny: ['write'] }, + }], + ['subagent/descriptor', { mode: 'continuable', version: 3, provider: 'p', label: 'child' }], + ['approval/asked', { id: 'approval', toolName: 'read' }], + ['team/member', { + version: 1, teamId: 'team', + member: { id: 'm', name: 'm', description: 'd', provider: 'p', context: 'fork', phase: 'failed', error: 'failure' }, + }], + ['team/task', { + version: 1, teamId: 'team', + task: { id: 'task-2', revision: 2, subject: 's', description: 'd', status: 'completed', ownerId: 'm', blockedBy: ['task-1'], writeScopes: [] }, + }], + ['command/done', { commandId: 'c', kind: 'error', text: 'failed' }], + ['compaction/end', { compactionId: 'c', sourceCommandId: 'command', turn: 1, error: 'failure' }], + ] + for (const [type, data] of remaining) { + expect(() => { assertPayload(type, data) }, type).not.toThrow() + } + }) + + it('refuses malformed logical headers, cuts, event envelopes, and surface metadata', () => { + const base = v1Artifact('turn/start', { turn: 1 }) + const invalidHeaders = [ + { ...base.header, version: 0 }, + { ...base.header, id: 1 }, + { ...base.header, createdAt: -1 }, + { ...base.header, isSeeded: 'yes' }, + { ...base.header, delegationDepth: -1 }, + { ...base.header, cwd: 1 }, + { ...base.header, parentSession: 1 }, + { ...base.header, agentPreset: 1 }, + { ...base.header, origin: 'other' }, + ] + for (const header of invalidHeaders) { + expect(() => { assertReleasedV1Artifact({ ...base, header } as never) }).toThrow() + } + expect(() => { assertReleasedV1Artifact({ ...base, inheritedEventCount: base.events.length + 1 }) }).toThrow(/exceeds/) + expect(() => { assertReleasedV1Artifact({ ...base, inheritedEventCount: 1 }) }).toThrow(/unseeded/) + + const rawEvents: unknown[][] = [ + [{ type: 1, seq: 0, time: 1, data: {} }], + [{ type: 'plugin/unknown', seq: 0, time: 1, data: {} }], + [{ type: 'turn/start', seq: 1, time: 1, data: { turn: 1 } }], + [{ type: 'turn/start', seq: 0, time: 1, data: { turn: 1 }, ignorable: false }], + [{ type: 'turn/start', seq: 0, time: 1, data: { turn: 1 }, surfaceOp: 'append' }], + ] + for (const events of rawEvents) { + expect(() => { assertReleasedV1Artifact({ ...base, events } as never) }).toThrow() + } + + const surfaceData = userMessage + const surfaceVariants = [ + { sourceEventSeqs: 'bad', surfaceOp: 'append' }, + { sourceEventSeqs: [3], surfaceOp: 'append' }, + { sourceEventSeqs: [1, 1], surfaceOp: 'append' }, + { sourceEventSeqs: [], surfaceOp: 'append' }, + { surfaceOp: null }, + { surfaceOp: { op: 'append', start: 0, end: 1 } }, + { surfaceOp: { op: 'replace', start: 2, end: 1 } }, + { surfaceOp: { op: 'replace', start: 1, end: 3 } }, + ] + for (const metadata of surfaceVariants) { + const artifact = v1Artifact('user/message', surfaceData) + artifact.events[3] = { ...artifact.events[3], ...metadata } as never + expect(() => { assertReleasedV1Artifact(artifact) }).toThrow() + } + + const assistant = v1Artifact('assistant/message', validPayloads['assistant/message'] as SessionFormatJsonValue) + assistant.events[3] = { ...assistant.events[3], sourceEventSeqs: [], surfaceOp: 'append' } as never + expect(() => { assertReleasedEventPayload(assistant.events[3] as SessionFormatEvent, 1) }).not.toThrow() + }) + + it('accepts remaining optional payload members', () => { + const cases: Array<[string, SessionFormatJsonValue]> = [ + ['agent/inbox/spliced', { target: 'next-step', start: 0, removedCount: 1, inserted: [], outcome: 'canceled' }], + ['assistant/message', { + turn: 1, step: 0, message: assistantMessage, interrupted: true, + usage: { inputTokens: 1, outputTokens: 1, totalTokens: 2, cacheReadTokens: 0, cacheWriteTokens: 0, reasoningTokens: 0 }, + }], + ['compaction/summary', { + compactionId: 'c', sourceCommandId: 'command', summary: [textBlock], + shadowedRange: { start: 0, end: 1 }, shadowedSeqs: [0, 1], shadowedTokenCount: 2, + provider: 'p', model: 'm', maxTokens: 10, usage: { inputTokens: 1, outputTokens: 1 }, + rawOutput: [textBlock], llmStreamCall: true, + }], + ['hook/invoked', { turn: 1, point: 'Stop', dialect: 'codex', matcher: '*', handlerId: 'h' }], + ['hook/result', { + turn: 1, point: 'Stop', handlerId: 'h', decision: 'pass', exitCode: -1, + stderrSummary: 'stderr', durationMs: 1, + }], + ['request/header', { + header: { + config: { provider: 'p', model: 'm', temperature: 0.5, maxTokens: 5, stop: ['stop'] }, + system: '', tools: [], + }, + reason: 'change', startsSeries: true, + }], + ] + for (const [type, data] of cases) { + expect(() => { assertPayload(type, data) }, type).not.toThrow() + } + }) + + it('refuses every relationship-specific invalid payload branch', () => { + const cases: Array<[string, SessionFormatJsonValue]> = [ + ['command/done', { commandId: 'c', kind: 'success', sourceEventSeq: 3 }], + ['session/title-llm-request', { + titleProvider: 'p', messageSeqs: [], route: { provider: 'p', model: 'm' }, + system: 's', messages: [userMessage], maxTokens: 1, + }], + ['session/title', { title: 't', messageSeqs: [0, 0], source: { kind: 'fallback' } }], + ['tool/result', { turn: 1, step: 0, message: { ...toolMessage, content: [] } }], + ['tool/result', { + turn: 1, step: 0, + message: { ...toolMessage, content: [{ type: 'text', text: 'not a result' }] }, + }], + ['user/message', { + ...userMessage, + source: { kind: 'plugin', plugin: 'x', form: 'notice', summary: 'x', sections: [] }, + }], + ['user/message', { + ...userMessage, + source: { + kind: 'session-reference', form: 'recall', version: 1, + references: [{ + sessionId: 's', label: 's', capturedThroughSeq: null, compacted: false, + originalMessages: 1, retainedMessages: 1, omittedMessages: 0, omittedBytes: 0, + truncated: false, inputIndex: 1, + }], + }, + }], + ['user/message', { + ...userMessage, + source: { kind: 'plugin', plugin: 'x', form: 'snapshot', sections: [], summary: 'x' }, + }], + ['user/message', { + ...userMessage, + source: { kind: 'plugin', plugin: 'compact' }, + }], + ['assistant/chunk', { + turn: 1, step: 0, chunk: { type: 'finish', reason: { kind: 'stop' }, replayState: { response: {}, blocks: {} } }, + }], + ['compaction/summary', { + compactionId: 'c', summary: [], shadowedRange: { start: 0, end: 0 }, shadowedSeqs: [0], + shadowedTokenCount: 0, provider: 'p', model: 'm', llmStreamCall: true, + }], + ['hook/result', { turn: 1, point: 'Stop', handlerId: 'h', decision: 'pass', durationMs: -0.5 }], + ['llm/retry', { + retryId: 'r', turn: 1, step: 0, provider: 'p', mode: 'always', policyKey: 'k', retry: 1, + maxRetries: 2, delayMs: 0, failure: { message: 'x', code: 'X' }, + }], + ['session/title-llm-request', { + titleProvider: 'p', messageSeqs: [0], route: { provider: 'p', model: 'm' }, + system: 's', messages: [userMessage], maxTokens: 0, + }], + ['request/header', { + header: { config: { provider: 'p', model: 'm' }, adapterDefaults: { maxTokens: true } }, + reason: 'initial', + }], + ['compaction/prune', { + shadowedRange: { start: 1, end: 0 }, shadowedSeqs: [0, 1], shadowedTokenCount: 1, + }], + ['compaction/prune', { + shadowedRange: { start: 0, end: 2 }, shadowedSeqs: [0, 1], shadowedTokenCount: 1, + }], + ['goal/change', { + kind: 'goal/change', version: 1, operation: 'create', + goal: { id: 'g', revision: 1, objective: 'x', phase: 'active', maxGoalRounds: 1, blockedReason: { code: 'x', message: 'x' } }, + roundsStarted: 0, createdAt: 1, updatedAt: 1, + }], + ['schedule/change', { version: 1, operation: 'create', schedule: { id: 'x', kind: 'unknown', prompt: 'x', scheduledAt: 'x' } }], + ['schedule/change', { version: 1, operation: 'create', schedule: { id: 'x', kind: 'every', prompt: 'x', everySeconds: 299, scheduledAt: '2026-08-31T00:00:00.000Z' } }], + ['schedule/change', { version: 1, operation: 'delete', id: ' x ' }], + ['schedule/change', { version: 1, operation: 'dispatch', id: 'x', acceptedAt: '2026-02-31T00:00:00.000Z' }], + ['subagent/descriptor', { mode: 'continuable', version: 3, provider: 'p', label: 'x', agentProvider: 'p' }], + ['subagent/descriptor', { mode: 'continuable', version: 3, provider: 'p', label: 'x', toolFilter: {} }], + ['subagent/model-selection-policy', { allowedModels: [{ provider: 'p', model: 'm' }, { provider: 'p', model: 'm' }] }], + ['subagent/model-selection-policy', { allowedModels: [] }], + ['user/message', { + ...userMessage, + source: { + kind: 'session-reference', form: 'recall', version: 1, + references: [{ + sessionId: 's', label: 's', capturedFormatVersion: 2, capturedThroughSeq: null, + compacted: false, originalMessages: 1, retainedMessages: 1, omittedMessages: 0, + omittedBytes: 0, truncated: false, inputIndex: 0, + }], + }, + }], + ['user/message', { + ...userMessage, + source: { + kind: 'session-reference', form: 'recall', version: 1, + references: [{ + sessionId: 's', label: 's', capturedThroughSeq: null, compacted: false, + originalMessages: 2, retainedMessages: 1, omittedMessages: 1, omittedBytes: 0, + truncated: false, inputIndex: 0, + }], + }, + }], + ['user/message', { ...userMessage, source: { kind: 'session-reference', form: 'recall', version: 1, references: [] } }], + ['user/message', { + ...userMessage, + source: { + kind: 'session-reference', form: 'recall', version: 1, + references: [0, 1].map(inputIndex => ({ + sessionId: 'same', label: 's', capturedThroughSeq: null, compacted: false, + originalMessages: 1, retainedMessages: 1, omittedMessages: 0, omittedBytes: 0, + truncated: false, inputIndex, + })), + }, + }], + ['web/deepseek-search-llm-request', { + endpoint: 'x', apiVersion: 'x', + body: { model: 'm', max_tokens: 1, messages: [{ role: 'user', content: [] }], tools: [{ type: 'web_search_20250305', name: 'web_search', max_uses: 1 }] }, + }], + ['web/deepseek-search-llm-request', { + endpoint: 'x', apiVersion: 'x', + body: { model: 'm', max_tokens: 1, messages: [], tools: [{ type: 'web_search_20250305', name: 'web_search', max_uses: 1 }] }, + }], + ['web/deepseek-search-llm-request', { + endpoint: 'x', apiVersion: 'x', + body: { model: 'm', max_tokens: 1, messages: [{ role: 'user', content: [{ type: 'text', text: 'x' }] }], tools: [] }, + }], + ] + for (const [index, [type, data]] of cases.entries()) { + expect(() => { assertPayload(type, data) }, `${type}-${index}`).toThrow() + } + }) +}) diff --git a/packages/session/session-format-v0-to-v1/tsconfig.json b/packages/session/session-format-v0-to-v1/tsconfig.json new file mode 100644 index 0000000000..e59c5008e4 --- /dev/null +++ b/packages/session/session-format-v0-to-v1/tsconfig.json @@ -0,0 +1,24 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": [ + "src" + ], + "references": [ + { + "path": "../../../vendor/cosmokit" + }, + { + "path": "../../../vendor/cordis" + }, + { + "path": "../../core/session" + }, + { + "path": "../session-format" + } + ] +} diff --git a/packages/session/session-format-v0-to-v1/tsdown.config.ts b/packages/session/session-format-v0-to-v1/tsdown.config.ts new file mode 100644 index 0000000000..e4ff48895a --- /dev/null +++ b/packages/session/session-format-v0-to-v1/tsdown.config.ts @@ -0,0 +1,13 @@ +import { defineConfig } from 'tsdown' + +/** Build the frozen adjacent-migration library. */ +export default defineConfig({ + entry: ['lib/types/index.js'], + outDir: 'lib', + format: ['esm'], + platform: 'node', + target: 'es2024', + fixedExtension: false, + dts: false, + clean: false, +}) diff --git a/packages/session/session-format/README.i18n.yaml b/packages/session/session-format/README.i18n.yaml new file mode 100644 index 0000000000..c2d0610142 --- /dev/null +++ b/packages/session/session-format/README.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write packages/session/session-format/README.md +README.md: cfc1f7aafba463b454c669aa732cb4c9b049a5ef +README.zh.md: a02f3a6fa2863787207286bb25d92b44cadf6fe5 diff --git a/packages/session/session-format/README.md b/packages/session/session-format/README.md new file mode 100644 index 0000000000..cfc1f7aafb --- /dev/null +++ b/packages/session/session-format/README.md @@ -0,0 +1,104 @@ +--- +description: "Pure adjacent Session format planning, lossless JSON snapshots, header-only migration, and physical codec dispatch." +kind: "package-library" +--- + +# @deepseek-ai/dsh-session-format + +English | [中文](README.zh.md) + +## Summary + +`dsh-session-format` lets persistence code restore a current Session directly or compose a unique sequence of adjacent whole-artifact migrations. It snapshots every durable input and output as detached lossless JSON, validates exact version progress, and keeps header-only listing separate from body reads. Physical framing, compression, immutable generation naming, exclusive publication, and Cordis lifecycle behavior remain outside this pure library. + +## Table of Contents + +- [Use this package](#use-this-package) +- [Understand the implementation](#understand-the-implementation) +- [Further Exploration](#further-exploration) +- [Model Experience](#model-experience) +- [Known Limitations and Deferred Work](#known-limitations-and-deferred-work) +- [Dev Note](#dev-note) + +----- + + +## Use this package + +### When to use it + +Use this library from persistence or format-catalog code that must classify a physical Session header, restore current logical values, or compose released adjacent migrations. It is not a Cordis plugin and has no profile mount row. No runtime invariant companion is published because every operation validates its borrowed artifact before returning and retains no cross-call mutable state. + +### Entry point + +```text +const catalog = createSessionFormatCatalog({ currentVersion, codecs, migrations, restoreCurrent, restoreCurrentHeader }) +const descriptor = catalog.readHeader(physicalHeader) +``` + +`createSessionFormatCatalog()` accepts one frozen codec per supported version, one migration per adjacent version pair, and current artifact and header restorers. `inspectVersion()` reads only the physical version for directional dispatch. `readHeader()` returns a `current`, `migration-required`, `unsupported`, or `malformed` descriptor without reading events. Each edge validates its target header before the final current-header restorer runs. Body readers call `decodeArtifact()` or `decodeRecoverableArtifact()`, then `migrate()`; writers call `encodeCurrent()` only with a validated current artifact. + +The recoverable decoder returns the accepted logical prefix. A codec may drop one malformed or sequence-gapped row and its uncommitted suffix, but a later decoded `turn/end` makes the original issue fatal. + +----- + + +## Understand the implementation + +
+Implementation internals — click to expand + +The chain validates unique gap-free ordering at construction. A current artifact bypasses every migration callback and passes through only the current restorer. An old artifact runs each adjacent whole-document function in memory; only the caller decides whether and how to publish the final result. + +| File | Role | +|---|---| +| [`src/chain.ts`](src/chain.ts) | Adjacent plan construction and current bypass | +| [`src/catalog.ts`](src/catalog.ts) | Physical version dispatch and header classification | +| [`src/json.ts`](src/json.ts) | Detached lossless JSON snapshots and common coordinate checks | + +
+ +----- + + +## Further Exploration + +- [Released v0 to v1 edge](../session-format-v0-to-v1/README.md) — frozen historical decoding and identity conversion. +- [Static catalog](../session-format-catalog/README.md) — first-party codec and migration assembly. +- [JSONL persistence](../session-persistence-jsonl/README.md) — durable framing and generation publication. + +----- + + +## Model Experience + +### Session restoration + +#### What the model sees + +Nothing directly. Consumers reconstruct model history from the validated current artifact through `deriveMessages()`. + +#### Token effect + +Zero direct tokens. + +#### KV Cache effect + +No direct effect. A migration that changes current history can change the cache identity owned by request reconstruction. + +## Known Limitations and Deferred Work + + + +- **Whole-artifact memory use** — supported migrations materialize the complete logical Session; streamed transformation is deferred until measured artifacts require it. +- **Adjacent integer versions only** — the library does not expose spans, stable event identities, or a general reference-rewrite algebra. + + +### Dev Note + +
+Working context for maintainers — click to expand + +None. + +
diff --git a/packages/session/session-format/README.zh.md b/packages/session/session-format/README.zh.md new file mode 100644 index 0000000000..a02f3a6fa2 --- /dev/null +++ b/packages/session/session-format/README.zh.md @@ -0,0 +1,104 @@ +--- +description: "纯函数式相邻 Session 格式规划、无损 JSON 快照、仅标头迁移与物理编解码分派。" +kind: "package-library" +--- + +# @deepseek-ai/dsh-session-format + +[English](README.md) | 中文 + +## 概述 + +`dsh-session-format` 让持久化代码可以直接还原当前 Session，或组合唯一的相邻全产物迁移序列。它会把每个持久化输入和输出快照为分离的无损 JSON，校验精确的版本推进，并把仅标头的列表读取与正文读取分开。物理分帧、压缩、不可变 generation 命名、排他发布和 Cordis 生命周期行为不属于这个纯函数库。 + +## 目录 + +- [使用本包](#use-this-package) +- [理解实现](#understand-the-implementation) +- [进一步探索](#further-exploration) +- [模型体验](#model-experience) +- [已知限制与延期工作](#known-limitations-and-deferred-work) +- [开发备注](#dev-note) + +----- + + +## 使用本包 + +### 何时使用 + +当持久化或格式目录代码需要分类物理 Session header、还原当前逻辑值或组合已发布相邻迁移时，使用本库。它不是 Cordis 插件，也没有 profile 挂载行。它不发布运行时不变式伴生入口，因为每个操作都会在返回前校验借入的完整 artifact，且不保留跨调用的可变状态。 + +### 入口 + +```text +const catalog = createSessionFormatCatalog({ currentVersion, codecs, migrations, restoreCurrent, restoreCurrentHeader }) +const descriptor = catalog.readHeader(physicalHeader) +``` + +`createSessionFormatCatalog()` 接收每个受支持版本的一个冻结编解码器、每组相邻版本的一个迁移，以及当前产物与标头还原器。`inspectVersion()` 只读取物理版本以执行方向分派。`readHeader()` 在不读取事件的情况下返回 `current`、`migration-required`、`unsupported` 或 `malformed` 描述符。每个迁移边会先校验自己的目标标头，然后再运行最终的当前标头还原器。正文读取方调用 `decodeArtifact()` 或 `decodeRecoverableArtifact()`，然后调用 `migrate()`；写入方只使用经过校验的当前产物调用 `encodeCurrent()`。 + +可恢复解码器返回已接受的逻辑前缀。编解码器可以丢弃一个格式错误或序号不连续的行及其未提交后缀，但后续成功解码的 `turn/end` 会使原始问题成为致命错误。 + +----- + + +## 理解实现 + +
+实现细节——点击展开 + +迁移链在构造时校验唯一且无缺口的顺序。当前产物绕过所有迁移回调，只经过当前格式还原器。旧产物在内存中依次运行每个相邻的全产物函数；只有调用方决定是否发布最终结果以及如何发布。 + +| 文件 | 职责 | +|---|---| +| [`src/chain.ts`](src/chain.ts) | 相邻计划构造与当前格式绕过 | +| [`src/catalog.ts`](src/catalog.ts) | 物理版本分派与标头分类 | +| [`src/json.ts`](src/json.ts) | 分离的无损 JSON 快照与通用坐标校验 | + +
+ +----- + + +## 进一步探索 + +- [已发布 v0 到 v1 迁移边](../session-format-v0-to-v1/README.zh.md)——冻结的历史解码与恒等转换。 +- [静态目录](../session-format-catalog/README.zh.md)——第一方编解码器与迁移装配。 +- [JSONL 持久化](../session-persistence-jsonl/README.zh.md)——持久化分帧与代际发布。 + +----- + + +## 模型体验 + +### Session 还原 + +#### 模型看到什么 + +没有直接内容。消费方通过 `deriveMessages()` 从经过校验的当前产物重建模型历史。 + +#### Token 影响 + +不直接产生 token。 + +#### KV Cache 影响 + +没有直接影响。迁移若改变当前历史，可能改变由请求重建逻辑拥有的缓存身份。 + +## 已知限制与延期工作 + + + +- **全产物内存占用**——受支持的迁移会物化完整逻辑 Session；只有实测产物规模提出要求时，才会引入流式转换。 +- **仅支持相邻整数版本**——本库不暴露 span、稳定事件身份或通用引用重写代数。 + + +### 开发备注 + +
+维护者的工作上下文——点击展开 + +无。 + +
diff --git a/packages/session/session-format/package.json b/packages/session/session-format/package.json new file mode 100644 index 0000000000..506375165b --- /dev/null +++ b/packages/session/session-format/package.json @@ -0,0 +1,35 @@ +{ + "name": "@deepseek-ai/dsh-session-format", + "description": "Pure adjacent whole-artifact Session format migration machinery", + "version": "0.1.2-alpha.3", + "publishConfig": { + "access": "public" + }, + "repository": { + "type": "git", + "url": "git+https://github.com/deepseek-ai/deepseek-harness.git", + "directory": "packages/session/session-format" + }, + "type": "module", + "main": "lib/index.js", + "types": "lib/types/index.d.ts", + "exports": { + ".": { + "types": "./lib/types/index.d.ts", + "default": "./lib/index.js" + }, + "./src/*": "./src/*", + "./package.json": "./package.json" + }, + "files": [ + "lib/index.js", + "lib/types/**/*.d.ts" + ], + "license": "MIT", + "peerDependencies": { + "@deepseek-ai/cordis": "workspace:^" + }, + "devDependencies": { + "@deepseek-ai/cordis": "workspace:^" + } +} diff --git a/packages/session/session-format/src/catalog.ts b/packages/session/session-format/src/catalog.ts new file mode 100644 index 0000000000..677b4f979a --- /dev/null +++ b/packages/session/session-format/src/catalog.ts @@ -0,0 +1,163 @@ +import { createSessionFormatChain } from './chain.ts' +import { SessionFormatError, SessionFormatUnsupportedMigrationError } from './error.ts' +import { + inspectSessionFormatVersion, + snapshotSessionFormatArtifact, + snapshotSessionFormatHeader, + snapshotSessionFormatJson, + sessionFormatVersion, +} from './json.ts' +import type { + EncodedSessionFormatArtifact, + SessionFormatCatalog, + SessionFormatCatalogOptions, + SessionFormatCodec, + SessionFormatEncodeOptions, + SessionFormatHeaderReadResult, + SessionFormatJsonObject, +} from './types.ts' + +/** + * Compile a build-static physical codec and adjacent migration catalog. + * @param options - complete codecs, migrations, current version, and restorer. + * @returns immutable physical dispatch and migration operations. + */ +export function createSessionFormatCatalog(options: SessionFormatCatalogOptions): SessionFormatCatalog { + const chain = createSessionFormatChain(options) + const codecs = new Map() + for (const codec of options.codecs) { + const version = sessionFormatVersion(codec.version, 'Session format codec version') + if (codecs.has(version)) throw new SessionFormatError(`Session format codec v${version} is duplicated`) + codecs.set(version, Object.freeze({ ...codec })) + } + for (let version = 0; version <= chain.currentVersion; version += 1) { + if (!codecs.has(version)) throw new SessionFormatError(`Session format codec v${version} is missing`) + } + if (codecs.size !== chain.currentVersion + 1) { + const invalid = [...codecs.keys()].find(version => version > chain.currentVersion) as number + throw new SessionFormatError(`Session format codec v${invalid} is newer than current v${chain.currentVersion}`) + } + + function inspectVersion(headerValue: unknown): number { + return inspectSessionFormatVersion(headerValue) + } + + function readHeader(headerValue: unknown): SessionFormatHeaderReadResult { + let storedVersion: number | undefined + try { + storedVersion = inspectVersion(headerValue) + } catch (error: unknown) { + return malformed(chain.currentVersion, error) + } + if (storedVersion > chain.currentVersion) { + return Object.freeze({ + status: 'unsupported', + storedVersion, + targetVersion: chain.currentVersion, + reason: `stored Session uses newer format v${storedVersion}; this build writes v${chain.currentVersion}`, + }) + } + const codec = codecs.get(storedVersion) + /* v8 ignore next -- construction proves every supported version has exactly one codec. */ + if (codec === undefined) { + return Object.freeze({ + status: 'unsupported', + storedVersion, + targetVersion: chain.currentVersion, + reason: `this build has no Session format codec for v${storedVersion}`, + }) + } + try { + const decoded = snapshotSessionFormatHeader(codec.decodeHeader(headerValue), `format v${storedVersion} header`) + const header = chain.migrateHeader(decoded) + return Object.freeze({ + status: storedVersion === chain.currentVersion ? 'current' : 'migration-required', + storedVersion, + targetVersion: chain.currentVersion, + header, + }) + } catch (error: unknown) { + if (error instanceof SessionFormatUnsupportedMigrationError) { + return Object.freeze({ + status: 'unsupported', + storedVersion, + targetVersion: chain.currentVersion, + reason: error.message, + }) + } + return malformed(chain.currentVersion, error, storedVersion) + } + } + + function artifactCodec(headerValue: unknown): { + readonly storedVersion: number + readonly codec: SessionFormatCodec + } { + const storedVersion = inspectVersion(headerValue) + if (storedVersion > chain.currentVersion) { + throw new SessionFormatUnsupportedMigrationError( + `stored Session uses newer format v${storedVersion}; this build writes v${chain.currentVersion}`, + ) + } + const codec = codecs.get(storedVersion) + /* v8 ignore next -- construction proves every supported version has exactly one codec. */ + if (codec === undefined) { + throw new SessionFormatUnsupportedMigrationError(`this build has no Session format codec for v${storedVersion}`) + } + return { storedVersion, codec } + } + + function decodeArtifact(headerValue: unknown, rowValues: readonly unknown[]) { + const { storedVersion, codec } = artifactCodec(headerValue) + return snapshotSessionFormatArtifact( + codec.decodeArtifact(headerValue, rowValues), + `format v${storedVersion} decoded artifact`, + ) + } + + function decodeRecoverableArtifact(headerValue: unknown, rowValues: readonly unknown[]) { + const { storedVersion, codec } = artifactCodec(headerValue) + return snapshotSessionFormatArtifact( + codec.decodeRecoverableArtifact(headerValue, rowValues), + `format v${storedVersion} recoverable artifact`, + ) + } + + function encodeCurrent( + artifact: Parameters[0], + encodeOptions: SessionFormatEncodeOptions, + ): EncodedSessionFormatArtifact { + if (inspectSessionFormatVersion(artifact.header) !== chain.currentVersion) { + throw new SessionFormatError(`encodeCurrent requires Session format v${chain.currentVersion}`) + } + const current = chain.migrate(artifact) + const codec = codecs.get(chain.currentVersion) as SessionFormatCodec + const encoded = codec.encodeArtifact(current, encodeOptions) + const header = snapshotSessionFormatJson(encoded.header, 'encoded current Session header') as SessionFormatJsonObject + const rows = Object.freeze(encoded.rows.map((row, index) => + snapshotSessionFormatJson(row, `encoded current Session row ${index}`) as SessionFormatJsonObject)) + if (inspectSessionFormatVersion(header) !== chain.currentVersion) { + throw new SessionFormatError('current Session codec returned a non-current header') + } + return Object.freeze({ header, rows }) + } + + return Object.freeze({ + currentVersion: chain.currentVersion, + inspectVersion, + readHeader, + decodeArtifact, + decodeRecoverableArtifact, + migrate: chain.migrate.bind(chain), + encodeCurrent, + }) +} + +function malformed(targetVersion: number, error: unknown, storedVersion?: number): SessionFormatHeaderReadResult { + return Object.freeze({ + status: 'malformed', + ...(storedVersion === undefined ? {} : { storedVersion }), + targetVersion, + reason: error instanceof Error ? error.message : String(error), + }) +} diff --git a/packages/session/session-format/src/chain.ts b/packages/session/session-format/src/chain.ts new file mode 100644 index 0000000000..155e3949d8 --- /dev/null +++ b/packages/session/session-format/src/chain.ts @@ -0,0 +1,166 @@ +import { SessionFormatError, SessionFormatUnsupportedMigrationError } from './error.ts' +import { + inspectSessionFormatVersion, + snapshotSessionFormatArtifact, + snapshotSessionFormatHeader, + sessionFormatVersion, +} from './json.ts' +import type { + SessionFormatArtifact, + SessionFormatChain, + SessionFormatChainOptions, + SessionFormatHeader, + SessionFormatMigration, +} from './types.ts' + +/** + * Validate and freeze one adjacent migration declaration. + * @param migration - named exact adjacent conversion. + * @returns immutable validated declaration. + */ +export function defineSessionFormatMigration(migration: SessionFormatMigration): SessionFormatMigration { + if (typeof migration.name !== 'string' || migration.name.length === 0) { + throw new SessionFormatError('Session migration name must be a non-empty string') + } + const from = sessionFormatVersion(migration.fromVersion, `${migration.name} fromVersion`) + const to = sessionFormatVersion(migration.toVersion, `${migration.name} toVersion`) + if (to !== from + 1) { + throw new SessionFormatError(`${migration.name} must declare adjacent v${from}->v${from + 1}`) + } + return Object.freeze({ ...migration }) +} + +/** + * Compile a unique, complete adjacent migration chain. + * @param options - current version, adjacent declarations, and current restorer. + * @returns immutable planner and whole-artifact runner. + */ +export function createSessionFormatChain(options: SessionFormatChainOptions): SessionFormatChain { + return new CompiledSessionFormatChain(options) +} + +class CompiledSessionFormatChain implements SessionFormatChain { + readonly currentVersion: number + private readonly migrations: readonly SessionFormatMigration[] + private readonly restoreCurrent: SessionFormatChainOptions['restoreCurrent'] + private readonly restoreCurrentHeader: SessionFormatChainOptions['restoreCurrentHeader'] + + constructor(options: SessionFormatChainOptions) { + this.currentVersion = sessionFormatVersion(options.currentVersion, 'current Session format version') + this.restoreCurrent = options.restoreCurrent + this.restoreCurrentHeader = options.restoreCurrentHeader + const byFrom = new Map() + const names = new Set() + for (const candidate of options.migrations) { + const migration = defineSessionFormatMigration(candidate) + if (byFrom.has(migration.fromVersion)) { + throw new SessionFormatError(`Session migration v${migration.fromVersion}->v${migration.toVersion} is duplicated`) + } + if (names.has(migration.name)) throw new SessionFormatError(`Session migration name ${JSON.stringify(migration.name)} is duplicated`) + byFrom.set(migration.fromVersion, migration) + names.add(migration.name) + } + const ordered: SessionFormatMigration[] = [] + for (let version = 0; version < this.currentVersion; version += 1) { + const migration = byFrom.get(version) + if (migration === undefined) { + throw new SessionFormatUnsupportedMigrationError(`Session migration v${version}->v${version + 1} is missing`) + } + ordered.push(migration) + } + if (byFrom.size !== ordered.length) { + const invalid = [...byFrom.keys()].find(version => version >= this.currentVersion) as number + throw new SessionFormatError(`Session migration from v${invalid} does not lead to current v${this.currentVersion}`) + } + this.migrations = Object.freeze(ordered) + } + + plan(fromVersion: number): readonly SessionFormatMigration[] { + const from = sessionFormatVersion(fromVersion, 'stored Session format version') + if (from > this.currentVersion) { + throw new SessionFormatUnsupportedMigrationError( + `stored Session uses newer format v${from}; this build writes v${this.currentVersion}`, + ) + } + return Object.freeze(this.migrations.slice(from)) + } + + migrate(source: SessionFormatArtifact): SessionFormatArtifact { + const storedVersion = inspectSessionFormatVersion(source.header) + let current = snapshotSessionFormatArtifact(source, `format v${storedVersion} source`) + if (storedVersion === this.currentVersion) { + current = snapshotSessionFormatArtifact(this.restoreCurrent(current), 'current Session restoration') + this.assertCurrent(current) + return current + } + for (const migration of this.plan(storedVersion)) { + let migrated: SessionFormatArtifact + try { + migrated = migration.migrate(snapshotSessionFormatArtifact(current, `${migration.name} input`)) + } catch (error: unknown) { + throwUnsupportedRefusal(migration, error) + } + current = snapshotSessionFormatArtifact(migrated, `${migration.name} output`) + if (current.header.version !== migration.toVersion) { + throw new SessionFormatError(`${migration.name} returned v${current.header.version}; expected v${migration.toVersion}`) + } + try { + migration.validateTarget(current) + } catch (error: unknown) { + throwUnsupportedRefusal(migration, error) + } + } + current = snapshotSessionFormatArtifact(this.restoreCurrent(current), 'current Session restoration') + this.assertCurrent(current) + return current + } + + migrateHeader(source: SessionFormatHeader): SessionFormatHeader { + let current = snapshotSessionFormatHeader(source, 'stored Session header') + for (const migration of this.plan(current.version)) { + let migrated: SessionFormatHeader + try { + migrated = migration.migrateHeader(snapshotSessionFormatHeader(current, `${migration.name} header input`)) + } catch (error: unknown) { + throwUnsupportedRefusal(migration, error, 'Session header') + } + current = snapshotSessionFormatHeader(migrated, `${migration.name} header output`) + if (current.version !== migration.toVersion) { + throw new SessionFormatError(`${migration.name} header returned v${current.version}; expected v${migration.toVersion}`) + } + try { + migration.validateTargetHeader(current) + } catch (error: unknown) { + throwUnsupportedRefusal(migration, error, 'Session header') + } + } + current = snapshotSessionFormatHeader(this.restoreCurrentHeader(current), 'current Session header restoration') + if (current.version !== this.currentVersion) { + throw new SessionFormatError( + `current Session header restorer returned v${current.version}; expected v${this.currentVersion}`, + ) + } + return current + } + + private assertCurrent(artifact: SessionFormatArtifact): void { + if (artifact.header.version !== this.currentVersion) { + throw new SessionFormatError( + `current Session restorer returned v${artifact.header.version}; expected v${this.currentVersion}`, + ) + } + } +} + +function throwUnsupportedRefusal( + migration: SessionFormatMigration, + error: unknown, + subject = 'Session', +): never { + if (error instanceof SessionFormatUnsupportedMigrationError) throw error + const detail = error instanceof Error ? error.message : String(error) + throw new SessionFormatUnsupportedMigrationError( + `${migration.name} refuses this format v${migration.fromVersion} ${subject}: ${detail}`, + { cause: error }, + ) +} diff --git a/packages/session/session-format/src/error.ts b/packages/session/session-format/src/error.ts new file mode 100644 index 0000000000..9b7e4c6a00 --- /dev/null +++ b/packages/session/session-format/src/error.ts @@ -0,0 +1,9 @@ +/** Error raised when a durable Session artifact cannot be restored or migrated losslessly. */ +export class SessionFormatError extends Error { + override readonly name: string = 'SessionFormatError' +} + +/** A readable artifact whose released source policy has no supported migration. */ +export class SessionFormatUnsupportedMigrationError extends SessionFormatError { + override readonly name = 'SessionFormatUnsupportedMigrationError' +} diff --git a/packages/session/session-format/src/index.ts b/packages/session/session-format/src/index.ts new file mode 100644 index 0000000000..4fa90c8603 --- /dev/null +++ b/packages/session/session-format/src/index.ts @@ -0,0 +1,7 @@ +/** Pure adjacent whole-artifact Session format migration machinery. */ + +export * from './chain.ts' +export * from './catalog.ts' +export * from './error.ts' +export * from './json.ts' +export * from './types.ts' diff --git a/packages/session/session-format/src/json.ts b/packages/session/session-format/src/json.ts new file mode 100644 index 0000000000..7f150056c3 --- /dev/null +++ b/packages/session/session-format/src/json.ts @@ -0,0 +1,183 @@ +import { SessionFormatError } from './error.ts' +import type { + SessionFormatArtifact, + SessionFormatHeader, + SessionFormatJsonObject, + SessionFormatJsonValue, +} from './types.ts' + +/** + * Test whether a value is a non-null, non-array object. + * @param value - candidate value. + * @returns whether the value is an object record. + */ +export function isSessionFormatJsonObject(value: unknown): value is Record { + return typeof value === 'object' && value !== null && !Array.isArray(value) +} + +/** + * Require a non-negative safe integer without the JSON-unstable negative zero. + * @param value - candidate count. + * @param label - diagnostic subject. + * @returns validated count. + */ +export function sessionFormatCount(value: unknown, label: string): number { + if (!Number.isSafeInteger(value) || (value as number) < 0 || Object.is(value, -0)) { + throw new SessionFormatError(`${label} must be a non-negative safe integer`) + } + return value as number +} + +/** + * Require a safe integer without the JSON-unstable negative zero. + * @param value - candidate integer. + * @param label - diagnostic subject. + * @returns validated integer. + */ +export function sessionFormatSafeInteger(value: unknown, label: string): number { + if (!Number.isSafeInteger(value) || Object.is(value, -0)) { + throw new SessionFormatError(`${label} must be a safe integer`) + } + return value as number +} + +/** + * Require a non-negative integral format version. + * @param value - candidate version. + * @param label - diagnostic subject. + * @returns validated version. + */ +export function sessionFormatVersion(value: unknown, label = 'Session format version'): number { + return sessionFormatCount(value, label) +} + +/** + * Read only the version required for directional dispatch. + * @param headerValue - untrusted physical header value. + * @returns validated stored version. + */ +export function inspectSessionFormatVersion(headerValue: unknown): number { + if (!isSessionFormatJsonObject(headerValue)) { + throw new SessionFormatError('Session header must be a JSON object') + } + return sessionFormatVersion(headerValue['version']) +} + +/** + * Detach and deeply freeze a caller-supplied lossless JSON value. + * @param value - borrowed candidate. + * @param label - diagnostic subject. + * @returns an immutable detached JSON snapshot. + */ +export function snapshotSessionFormatJson(value: unknown, label = 'Session value'): SessionFormatJsonValue { + return snapshotValue(value, label, new Set()) +} + +function snapshotValue(value: unknown, label: string, ancestors: Set): SessionFormatJsonValue { + if (value === null || typeof value === 'string' || typeof value === 'boolean') return value + if (typeof value === 'number') { + if (!Number.isFinite(value) || Object.is(value, -0)) { + throw new SessionFormatError(`${label} contains a number that JSON cannot preserve`) + } + return value + } + if (typeof value !== 'object') throw new SessionFormatError(`${label} contains a non-JSON value`) + if (ancestors.has(value)) throw new SessionFormatError(`${label} contains a cycle`) + ancestors.add(value) + try { + if (Array.isArray(value)) { + if (Reflect.getPrototypeOf(value) !== Array.prototype) { + throw new SessionFormatError(`${label} contains a non-intrinsic JSON array`) + } + const ownKeys = Reflect.ownKeys(value) + const expectedKeys = new Set(['length', ...Array.from({ length: value.length }, (_, index) => String(index))]) + if (ownKeys.some(key => typeof key !== 'string' || !expectedKeys.has(key))) { + throw new SessionFormatError(`${label} contains an array property that JSON cannot preserve`) + } + for (let index = 0; index < value.length; index += 1) { + if (!Object.hasOwn(value, index)) throw new SessionFormatError(`${label} contains a sparse array`) + const descriptor = Reflect.getOwnPropertyDescriptor(value, String(index)) as PropertyDescriptor + if (!descriptor.enumerable || !Object.hasOwn(descriptor, 'value')) { + throw new SessionFormatError(`${label} contains an array member that JSON cannot preserve`) + } + } + return Object.freeze(value.map(member => snapshotValue(member, label, ancestors))) + } + const prototype = Reflect.getPrototypeOf(value) + if (prototype !== Object.prototype && prototype !== null) { + throw new SessionFormatError(`${label} contains a non-plain object`) + } + const output: Record = {} + for (const key of Reflect.ownKeys(value)) { + if (typeof key !== 'string') { + throw new SessionFormatError(`${label} contains a symbol property that JSON cannot preserve`) + } + const descriptor = Reflect.getOwnPropertyDescriptor(value, key) as PropertyDescriptor + if (!descriptor.enumerable || !Object.hasOwn(descriptor, 'value')) { + throw new SessionFormatError(`${label} contains a property that JSON cannot preserve`) + } + Object.defineProperty(output, key, { + configurable: true, + enumerable: true, + value: snapshotValue(descriptor.value, label, ancestors), + writable: true, + }) + } + return Object.freeze(output) + } finally { + ancestors.delete(value) + } +} + +/** + * Snapshot one complete artifact and validate its shared coordinates. + * @param artifact - borrowed logical artifact. + * @param label - diagnostic subject. + * @returns immutable detached artifact. + */ +export function snapshotSessionFormatArtifact( + artifact: SessionFormatArtifact, + label = 'Session artifact', +): SessionFormatArtifact { + const snapshot = snapshotSessionFormatJson(artifact, label) as SessionFormatJsonObject + const header = snapshot['header'] + const inheritedEventCount = snapshot['inheritedEventCount'] + const events = snapshot['events'] + if (!isSessionFormatJsonObject(header)) throw new SessionFormatError(`${label} header must be a JSON object`) + inspectSessionFormatVersion(header) + sessionFormatCount(inheritedEventCount, `${label} inheritedEventCount`) + if (!Array.isArray(events)) throw new SessionFormatError(`${label} events must be an array`) + for (let index = 0; index < events.length; index += 1) { + const event: unknown = events[index] + if (!isSessionFormatJsonObject(event)) throw new SessionFormatError(`${label} event ${index} must be a JSON object`) + if (event['seq'] !== index) { + throw new SessionFormatError(`${label} event ${index} has non-dense seq ${String(event['seq'])}`) + } + if (typeof event['type'] !== 'string' || event['type'].length === 0) { + throw new SessionFormatError(`${label} event ${index} type must be a non-empty string`) + } + sessionFormatSafeInteger(event['time'], `${label} event ${index} time`) + if (!Object.hasOwn(event, 'data')) throw new SessionFormatError(`${label} event ${index} lacks data`) + } + if (inheritedEventCount as number > events.length) { + throw new SessionFormatError(`${label} inheritedEventCount exceeds its event count`) + } + return snapshot as unknown as SessionFormatArtifact +} + +/** + * Snapshot one logical header without inspecting an event body. + * @param header - borrowed logical header. + * @param label - diagnostic subject. + * @returns immutable detached header. + */ +export function snapshotSessionFormatHeader(header: SessionFormatHeader, label = 'Session header'): SessionFormatHeader { + const snapshot = snapshotSessionFormatJson(header, label) + if (!isSessionFormatJsonObject(snapshot)) throw new SessionFormatError(`${label} must be a JSON object`) + inspectSessionFormatVersion(snapshot) + if (typeof snapshot['id'] !== 'string') throw new SessionFormatError(`${label} id must be a string`) + sessionFormatCount(snapshot['createdAt'], `${label} createdAt`) + if (typeof snapshot['isSeeded'] !== 'boolean') throw new SessionFormatError(`${label} isSeeded must be a boolean`) + sessionFormatCount(snapshot['delegationDepth'], `${label} delegationDepth`) + return snapshot as unknown as SessionFormatHeader +} diff --git a/packages/session/session-format/src/types.ts b/packages/session/session-format/src/types.ts new file mode 100644 index 0000000000..bf90c6e5c8 --- /dev/null +++ b/packages/session/session-format/src/types.ts @@ -0,0 +1,152 @@ +/** Scalar value admitted at the durable Session JSON boundary. */ +export type SessionFormatJsonPrimitive = null | boolean | number | string + +/** Lossless JSON value admitted at the durable Session boundary. */ +export type SessionFormatJsonValue = + | SessionFormatJsonPrimitive + | readonly SessionFormatJsonValue[] + | SessionFormatJsonObject + +/** Lossless JSON object admitted at the durable Session boundary. */ +export interface SessionFormatJsonObject { + readonly [key: string]: SessionFormatJsonValue +} + +/** Logical Session metadata shared by supported historical and current formats. */ +export interface SessionFormatHeader extends SessionFormatJsonObject { + readonly version: number + readonly id: string + readonly createdAt: number + readonly cwd?: string + readonly parentSession?: string + readonly isSeeded: boolean + readonly origin?: 'subagent' + readonly delegationDepth: number + readonly agentPreset?: string +} + +/** One decoded logical Session event. */ +export interface SessionFormatEvent extends SessionFormatJsonObject { + readonly type: string + readonly seq: number + readonly time: number + readonly data: SessionFormatJsonValue +} + +/** One detached complete logical Session artifact. */ +export interface SessionFormatArtifact { + readonly header: SessionFormatHeader + /** Exact inherited prefix length, available only after a body read. */ + readonly inheritedEventCount: number + readonly events: readonly SessionFormatEvent[] +} + +/** One independently maintained adjacent whole-artifact migration. */ +export interface SessionFormatMigration { + readonly name: string + readonly fromVersion: number + readonly toVersion: number + /** Convert one header without reading event bodies. */ + migrateHeader(header: SessionFormatHeader): SessionFormatHeader + /** Convert one detached complete artifact to exactly {@link toVersion}. */ + migrate(artifact: SessionFormatArtifact): SessionFormatArtifact + /** Refuse any artifact that the adjacent target writer cannot emit. */ + validateTarget(artifact: SessionFormatArtifact): void + /** Refuse any header that the adjacent target writer cannot emit. */ + validateTargetHeader(header: SessionFormatHeader): void +} + +/** Inputs that compile the unique complete migration chain. */ +export interface SessionFormatChainOptions { + readonly currentVersion: number + readonly migrations: readonly SessionFormatMigration[] + /** Restore and validate a detached current artifact through the current parser. */ + readonly restoreCurrent: (artifact: SessionFormatArtifact) => SessionFormatArtifact + /** Restore and validate a detached current header without reading event bodies. */ + readonly restoreCurrentHeader: (header: SessionFormatHeader) => SessionFormatHeader +} + +/** Pure adjacent planner and whole-artifact migration runner. */ +export interface SessionFormatChain { + readonly currentVersion: number + /** Return the complete ordered plan from one supported stored version. */ + plan(fromVersion: number): readonly SessionFormatMigration[] + /** Restore current input directly or migrate old input entirely in memory. */ + migrate(artifact: SessionFormatArtifact): SessionFormatArtifact + /** Convert only a supported header to the current logical representation. */ + migrateHeader(header: SessionFormatHeader): SessionFormatHeader +} + +/** Physical JSON records emitted by one format-specific codec. */ +export interface EncodedSessionFormatArtifact { + readonly header: SessionFormatJsonObject + readonly rows: readonly SessionFormatJsonObject[] +} + +/** Options that affect only physical row layout, never logical contents. */ +export interface SessionFormatEncodeOptions { + readonly packChunks: boolean +} + +/** Pure physical JSON codec frozen with one released Session format. */ +export interface SessionFormatCodec { + readonly version: number + /** Decode one physical header into body-independent logical metadata. */ + decodeHeader(value: unknown): SessionFormatHeader + /** Decode one complete physical header and row sequence into logical events. */ + decodeArtifact(headerValue: unknown, rowValues: readonly unknown[]): SessionFormatArtifact + /** Decode the row-atomic recoverable prefix used by crash-tail repair. */ + decodeRecoverableArtifact( + headerValue: unknown, + rowValues: readonly unknown[], + ): SessionFormatArtifact + /** Validate and encode one exact-version logical artifact. */ + encodeArtifact(artifact: SessionFormatArtifact, options: SessionFormatEncodeOptions): EncodedSessionFormatArtifact +} + +/** Header-only classification that never inspects event rows. */ +export type SessionFormatHeaderReadResult = + | { + readonly status: 'current' | 'migration-required' + readonly storedVersion: number + readonly targetVersion: number + /** Latest logical header. The exact inherited cut requires a body read. */ + readonly header: SessionFormatHeader + } + | { + readonly status: 'unsupported' + readonly storedVersion: number + readonly targetVersion: number + readonly reason: string + } + | { + readonly status: 'malformed' + readonly storedVersion?: number + readonly targetVersion: number + readonly reason: string + } + +/** Inputs for a build-static physical codec and migration catalog. */ +export interface SessionFormatCatalogOptions extends SessionFormatChainOptions { + readonly codecs: readonly SessionFormatCodec[] +} + +/** Build-static physical dispatch and adjacent migration catalog. */ +export interface SessionFormatCatalog { + readonly currentVersion: number + /** Read only the minimally required physical version. */ + inspectVersion(headerValue: unknown): number + /** Classify and translate one header without reading event rows. */ + readHeader(headerValue: unknown): SessionFormatHeaderReadResult + /** Dispatch a complete physical JSON artifact through its frozen version codec. */ + decodeArtifact(headerValue: unknown, rowValues: readonly unknown[]): SessionFormatArtifact + /** Dispatch a physical artifact through its released row-prefix recovery rules. */ + decodeRecoverableArtifact( + headerValue: unknown, + rowValues: readonly unknown[], + ): SessionFormatArtifact + /** Restore current input directly or run all required adjacent migrations in memory. */ + migrate(artifact: SessionFormatArtifact): SessionFormatArtifact + /** Validate and encode an exact current logical artifact. */ + encodeCurrent(artifact: SessionFormatArtifact, options: SessionFormatEncodeOptions): EncodedSessionFormatArtifact +} diff --git a/packages/session/session-format/tests/catalog.spec.ts b/packages/session/session-format/tests/catalog.spec.ts new file mode 100644 index 0000000000..1077c37cc7 --- /dev/null +++ b/packages/session/session-format/tests/catalog.spec.ts @@ -0,0 +1,221 @@ +import { describe, expect, it, vi } from 'vitest' +import { + createSessionFormatCatalog, + defineSessionFormatMigration, + type SessionFormatArtifact, + type SessionFormatCodec, +} from '../src/index.ts' + +function codec(version: number): SessionFormatCodec { + return { + version, + decodeHeader(value) { + return value as never + }, + decodeArtifact(headerValue, rowValues) { + const header = headerValue as SessionFormatArtifact['header'] + return { + header, + inheritedEventCount: 0, + events: rowValues as SessionFormatArtifact['events'], + } + }, + decodeRecoverableArtifact(headerValue, rowValues) { + return this.decodeArtifact(headerValue, rowValues) + }, + encodeArtifact(artifact) { + return { header: artifact.header, rows: artifact.events } + }, + } +} + +describe('Session format catalog', () => { + it('classifies headers without reading bodies and dispatches physical values by version', () => { + const migrate = vi.fn((artifact: SessionFormatArtifact): SessionFormatArtifact => ({ + ...artifact, + header: { ...artifact.header, version: 1 }, + })) + const catalog = createSessionFormatCatalog({ + currentVersion: 1, + codecs: [codec(0), codec(1)], + migrations: [defineSessionFormatMigration({ + name: '@test/v0-to-v1', + fromVersion: 0, + toVersion: 1, + migrateHeader: header => ({ ...header, version: 1 }), + migrate, + validateTarget: () => {}, + validateTargetHeader: () => {}, + })], + restoreCurrent: artifact => artifact, + restoreCurrentHeader: header => header, + }) + const oldHeader = { + version: 0, + id: 'old', + createdAt: 1, + isSeeded: true, + delegationDepth: 0, + } as const + + expect(catalog.inspectVersion(oldHeader)).toBe(0) + expect(catalog.readHeader(oldHeader)).toEqual({ + status: 'migration-required', + storedVersion: 0, + targetVersion: 1, + header: { ...oldHeader, version: 1 }, + }) + expect(catalog.readHeader({ version: 2 })).toMatchObject({ + status: 'unsupported', + storedVersion: 2, + targetVersion: 1, + }) + expect(catalog.readHeader({ version: 'broken' })).toMatchObject({ status: 'malformed', targetVersion: 1 }) + + const decoded = catalog.decodeArtifact(oldHeader, [ + { type: 'turn/start', seq: 0, time: 2, data: { turn: 1 } }, + ]) + expect(decoded.header.version).toBe(0) + expect(catalog.migrate(decoded).header.version).toBe(1) + expect(migrate).toHaveBeenCalledOnce() + }) + + it('uses the current codec directly for recovery and encoding', () => { + const currentCodec = codec(1) + const catalog = createSessionFormatCatalog({ + currentVersion: 1, + codecs: [codec(0), currentCodec], + migrations: [defineSessionFormatMigration({ + name: '@test/v0-to-v1', fromVersion: 0, toVersion: 1, + migrateHeader: header => ({ ...header, version: 1 }), + migrate: artifact => ({ ...artifact, header: { ...artifact.header, version: 1 } }), + validateTarget: () => {}, + validateTargetHeader: () => {}, + })], + restoreCurrent: artifact => artifact, + restoreCurrentHeader: header => header, + }) + const current = { + header: { + version: 1, id: 'current', createdAt: 1, isSeeded: false, delegationDepth: 0, + }, + inheritedEventCount: 0, + events: [{ type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }], + } satisfies SessionFormatArtifact + + expect(catalog.readHeader(current.header)).toMatchObject({ status: 'current', header: current.header }) + expect(catalog.decodeRecoverableArtifact(current.header, current.events)).toEqual(current) + expect(catalog.encodeCurrent(current, { packChunks: false })).toEqual({ + header: current.header, + rows: current.events, + }) + expect(() => catalog.encodeCurrent({ ...current, header: { ...current.header, version: 0 } }, { packChunks: false })) + .toThrow(/requires Session format v1/) + }) + + it('rejects duplicate, missing, and future codec declarations', () => { + const edge = defineSessionFormatMigration({ + name: '@test/v0-to-v1', fromVersion: 0, toVersion: 1, + migrateHeader: header => ({ ...header, version: 1 }), + migrate: artifact => ({ ...artifact, header: { ...artifact.header, version: 1 } }), + validateTarget: () => {}, + validateTargetHeader: () => {}, + }) + const options = { + currentVersion: 1, + migrations: [edge], + restoreCurrent: (value: SessionFormatArtifact) => value, + restoreCurrentHeader: (value: SessionFormatArtifact['header']) => value, + } + expect(() => createSessionFormatCatalog({ ...options, codecs: [codec(0), codec(0), codec(1)] })) + .toThrow(/codec v0 is duplicated/) + expect(() => createSessionFormatCatalog({ ...options, codecs: [codec(0)] })).toThrow(/codec v1 is missing/) + expect(() => createSessionFormatCatalog({ ...options, codecs: [codec(0), codec(1), codec(2)] })) + .toThrow(/codec v2 is newer/) + }) + + it('returns malformed descriptors for supported headers that their codec refuses', () => { + const refusing: SessionFormatCodec = { + ...codec(0), + decodeHeader: () => { throw 'bad header' }, + } + const catalog = createSessionFormatCatalog({ + currentVersion: 1, + codecs: [refusing, codec(1)], + migrations: [defineSessionFormatMigration({ + name: '@test/v0-to-v1', fromVersion: 0, toVersion: 1, + migrateHeader: header => ({ ...header, version: 1 }), + migrate: artifact => ({ ...artifact, header: { ...artifact.header, version: 1 } }), + validateTarget: () => {}, + validateTargetHeader: () => {}, + })], + restoreCurrent: artifact => artifact, + restoreCurrentHeader: header => header, + }) + expect(catalog.readHeader({ version: 0 })).toEqual({ + status: 'malformed', storedVersion: 0, targetVersion: 1, reason: 'bad header', + }) + expect(() => catalog.decodeArtifact({ version: 2 }, [])).toThrow(/newer/) + expect(() => catalog.decodeRecoverableArtifact({ version: 2 }, [])).toThrow(/newer/) + }) + + it('rejects a current encoder that returns a non-current header', () => { + const bad: SessionFormatCodec = { + ...codec(1), + encodeArtifact: artifact => ({ header: { ...artifact.header, version: 0 }, rows: artifact.events }), + } + const catalog = createSessionFormatCatalog({ + currentVersion: 1, + codecs: [codec(0), bad], + migrations: [defineSessionFormatMigration({ + name: '@test/v0-to-v1', fromVersion: 0, toVersion: 1, + migrateHeader: header => ({ ...header, version: 1 }), + migrate: artifact => ({ ...artifact, header: { ...artifact.header, version: 1 } }), + validateTarget: () => {}, + validateTargetHeader: () => {}, + })], + restoreCurrent: artifact => artifact, + restoreCurrentHeader: header => header, + }) + const current = { + header: { version: 1, id: 'bad', createdAt: 1, isSeeded: false, delegationDepth: 0 }, + inheritedEventCount: 0, + events: [], + } + expect(() => catalog.encodeCurrent(current, { packChunks: false })).toThrow(/non-current header/) + }) + + it('classifies malformed migrated and direct-current logical headers', () => { + const validateTargetHeader = (header: SessionFormatArtifact['header']): void => { + if (header['targetMarker'] !== true) throw new Error('latest header lacks marker') + } + const migration = defineSessionFormatMigration({ + name: '@test/v0-to-v1', fromVersion: 0, toVersion: 1, + migrateHeader: header => ({ ...header, version: 1 }), + migrate: artifact => ({ ...artifact, header: { ...artifact.header, version: 1 } }), + validateTarget: () => {}, + validateTargetHeader, + }) + const catalog = createSessionFormatCatalog({ + currentVersion: 1, + codecs: [codec(0), codec(1)], + migrations: [migration], + restoreCurrent: artifact => artifact, + restoreCurrentHeader: (header: SessionFormatArtifact['header']) => { + if (typeof header.id !== 'string') throw new Error('latest header lacks id') + return header + }, + }) + + const migrated = catalog.readHeader({ + version: 0, id: 'old', createdAt: 1, isSeeded: false, delegationDepth: 0, + }) + expect(migrated.status).toBe('unsupported') + if (migrated.status !== 'unsupported') throw new Error('expected unsupported header') + expect(migrated.reason).toContain('latest header lacks marker') + const current = catalog.readHeader({ version: 1 }) + expect(current.status).toBe('malformed') + if (current.status !== 'malformed') throw new Error('expected malformed current header') + expect(current.reason).toMatch(/id/) + }) +}) diff --git a/packages/session/session-format/tests/chain.spec.ts b/packages/session/session-format/tests/chain.spec.ts new file mode 100644 index 0000000000..6a129cee3f --- /dev/null +++ b/packages/session/session-format/tests/chain.spec.ts @@ -0,0 +1,251 @@ +import { describe, expect, it, vi } from 'vitest' +import { + createSessionFormatChain, + defineSessionFormatMigration, + SessionFormatUnsupportedMigrationError, + type SessionFormatArtifact, +} from '../src/index.ts' + +const currentArtifact: SessionFormatArtifact = { + header: { + version: 1, + id: 'session-1', + createdAt: 1, + isSeeded: false, + delegationDepth: 0, + }, + inheritedEventCount: 0, + events: [{ type: 'turn/start', seq: 0, time: 2, data: { turn: 1 } }], +} + +function captureError(run: () => unknown): Error { + try { + run() + } catch (error: unknown) { + if (error instanceof Error) return error + throw new Error('expected an Error object', { cause: error }) + } + throw new Error('expected callback to throw') +} + +describe('Session format chain', () => { + it('restores current input without invoking an adjacent migration', () => { + const migrate = vi.fn() + const chain = createSessionFormatChain({ + currentVersion: 1, + migrations: [defineSessionFormatMigration({ + name: '@test/v0-to-v1', + fromVersion: 0, + toVersion: 1, + migrateHeader: header => ({ ...header, version: 1 }), + migrate, + validateTarget: () => {}, + validateTargetHeader: () => {}, + })], + restoreCurrent: artifact => artifact, + restoreCurrentHeader: header => header, + }) + + const result = chain.migrate(currentArtifact) + + expect(result).toEqual(currentArtifact) + expect(result).not.toBe(currentArtifact) + expect(Object.isFrozen(result)).toBe(true) + expect(migrate).not.toHaveBeenCalled() + }) + + it('runs one adjacent whole-artifact edge and its header converter', () => { + const validateTarget = vi.fn() + const edge = defineSessionFormatMigration({ + name: '@test/v0-to-v1', + fromVersion: 0, + toVersion: 1, + migrateHeader: header => ({ ...header, version: 1 }), + migrate: artifact => ({ ...artifact, header: { ...artifact.header, version: 1 } }), + validateTarget, + validateTargetHeader: () => {}, + }) + const chain = createSessionFormatChain({ + currentVersion: 1, + migrations: [edge], + restoreCurrent: value => value, + restoreCurrentHeader: value => value, + }) + const source = { ...currentArtifact, header: { ...currentArtifact.header, version: 0 } } + + expect(chain.plan(0)).toEqual([edge]) + expect(chain.migrate(source)).toMatchObject({ header: { version: 1 } }) + expect(validateTarget).toHaveBeenCalledOnce() + expect(chain.migrateHeader(source.header).version).toBe(1) + }) + + it('rejects invalid declarations and incomplete chain construction', () => { + const base = { + name: '@test/v0-to-v1', fromVersion: 0, toVersion: 1, + migrateHeader: (header: SessionFormatArtifact['header']) => ({ ...header, version: 1 }), + migrate: (artifact: SessionFormatArtifact) => ({ ...artifact, header: { ...artifact.header, version: 1 } }), + validateTarget: () => {}, + validateTargetHeader: () => {}, + } + expect(() => defineSessionFormatMigration({ ...base, name: '' })).toThrow(/name/) + expect(() => defineSessionFormatMigration({ ...base, toVersion: 2 })).toThrow(/adjacent/) + expect(() => createSessionFormatChain({ + currentVersion: 1, migrations: [], restoreCurrent: value => value, restoreCurrentHeader: value => value, + })) + .toThrow(/missing/) + expect(() => createSessionFormatChain({ + currentVersion: 1, migrations: [base, base], restoreCurrent: value => value, restoreCurrentHeader: value => value, + })) + .toThrow(/duplicated/) + expect(() => createSessionFormatChain({ + currentVersion: 2, + migrations: [base, { ...base, name: base.name, fromVersion: 1, toVersion: 2 }], + restoreCurrent: value => value, + restoreCurrentHeader: value => value, + })).toThrow(/name .* duplicated/) + expect(() => createSessionFormatChain({ + currentVersion: 1, + migrations: [base, { ...base, name: '@test/v1-to-v2', fromVersion: 1, toVersion: 2 }], + restoreCurrent: value => value, + restoreCurrentHeader: value => value, + })).toThrow(/does not lead/) + }) + + it('rejects newer inputs and callbacks that return the wrong version', () => { + const edge = defineSessionFormatMigration({ + name: '@test/v0-to-v1', fromVersion: 0, toVersion: 1, + migrateHeader: header => header, + migrate: artifact => artifact, + validateTarget: () => {}, + validateTargetHeader: () => {}, + }) + const chain = createSessionFormatChain({ + currentVersion: 1, migrations: [edge], restoreCurrent: value => value, restoreCurrentHeader: value => value, + }) + const source = { ...currentArtifact, header: { ...currentArtifact.header, version: 0 } } + expect(() => chain.plan(2)).toThrow(/newer/) + expect(() => chain.plan(-1)).toThrow(/non-negative/) + expect(() => chain.migrate(source)).toThrow(/returned v0/) + expect(() => chain.migrateHeader(source.header)).toThrow(/header returned v0/) + + const badRestore = createSessionFormatChain({ + currentVersion: 1, + migrations: [defineSessionFormatMigration({ + ...edge, + migrateHeader: header => ({ ...header, version: 1 }), + migrate: artifact => ({ ...artifact, header: { ...artifact.header, version: 1 } }), + })], + restoreCurrent: value => ({ ...value, header: { ...value.header, version: 0 } }), + restoreCurrentHeader: value => value, + }) + expect(() => badRestore.migrate(currentArtifact)).toThrow(/current Session restorer returned v0/) + }) + + it('classifies adjacent target-policy refusal as unsupported but current restoration failure as corruption', () => { + const policyFailure = new Error('target relationship is invalid') + const edge = defineSessionFormatMigration({ + name: '@test/v0-to-v1', fromVersion: 0, toVersion: 1, + migrateHeader: header => ({ ...header, version: 1 }), + migrate: artifact => ({ ...artifact, header: { ...artifact.header, version: 1 } }), + validateTarget: () => { throw policyFailure }, + validateTargetHeader: () => {}, + }) + const chain = createSessionFormatChain({ + currentVersion: 1, + migrations: [edge], + restoreCurrent: value => value, + restoreCurrentHeader: value => value, + }) + const source = { ...currentArtifact, header: { ...currentArtifact.header, version: 0 } } + + const refusal = captureError(() => chain.migrate(source)) + expect(refusal).toBeInstanceOf(SessionFormatUnsupportedMigrationError) + expect(refusal.message).toContain('target relationship is invalid') + expect(refusal.cause).toBe(policyFailure) + + const brokenCurrent = createSessionFormatChain({ + currentVersion: 1, + migrations: [edge], + restoreCurrent: () => { throw new Error('current corruption') }, + restoreCurrentHeader: value => value, + }) + expect(() => brokenCurrent.migrate(currentArtifact)).toThrow('current corruption') + + const migrationFailure = createSessionFormatChain({ + currentVersion: 1, + migrations: [{ + ...edge, + migrate: () => { throw 'source policy token' }, + validateTarget: () => {}, + }], + restoreCurrent: value => value, + restoreCurrentHeader: value => value, + }) + expect(() => migrationFailure.migrate(source)).toThrow(/source policy token/) + + const alreadyUnsupported = new SessionFormatUnsupportedMigrationError('explicit edge refusal') + const preserved = createSessionFormatChain({ + currentVersion: 1, + migrations: [{ ...edge, validateTarget: () => { throw alreadyUnsupported } }], + restoreCurrent: value => value, + restoreCurrentHeader: value => value, + }) + expect(() => preserved.migrate(source)).toThrow(alreadyUnsupported) + }) + + it('validates every adjacent target header and the final current header', () => { + const validateTargetHeader = vi.fn((header: SessionFormatArtifact['header']) => { + if (header['targetMarker'] !== true) throw new Error('target header lacks marker') + }) + const restoreCurrentHeader = vi.fn((header: SessionFormatArtifact['header']) => { + if (typeof header.id !== 'string') throw new Error('current header lacks id') + return header + }) + const edge = defineSessionFormatMigration({ + name: '@test/v0-to-v1', fromVersion: 0, toVersion: 1, + migrateHeader: header => ({ ...header, version: 1 }), + migrate: artifact => ({ ...artifact, header: { ...artifact.header, version: 1 } }), + validateTarget: () => {}, + validateTargetHeader, + }) + const chain = createSessionFormatChain({ + currentVersion: 1, + migrations: [edge], + restoreCurrent: value => value, + restoreCurrentHeader, + }) + const source = { ...currentArtifact.header, version: 0 } + + const refusal = captureError(() => chain.migrateHeader(source)) + expect(refusal).toBeInstanceOf(SessionFormatUnsupportedMigrationError) + expect(refusal.message).toContain('target header lacks marker') + expect(validateTargetHeader).toHaveBeenCalledOnce() + + const rejectingHeader = createSessionFormatChain({ + currentVersion: 1, + migrations: [{ + ...edge, + migrateHeader: () => { throw new Error('historical header policy') }, + }], + restoreCurrent: value => value, + restoreCurrentHeader, + }) + expect(() => rejectingHeader.migrateHeader(source)).toThrow(/historical header policy/) + + const badCurrent = createSessionFormatChain({ + currentVersion: 1, + migrations: [edge], + restoreCurrent: value => value, + restoreCurrentHeader: () => ({ version: 1 } as never), + }) + expect(() => badCurrent.migrateHeader(currentArtifact.header)).toThrow(/current Session header restoration id/) + + const wrongCurrentVersion = createSessionFormatChain({ + currentVersion: 1, + migrations: [edge], + restoreCurrent: value => value, + restoreCurrentHeader: header => ({ ...header, version: 0 }), + }) + expect(() => wrongCurrentVersion.migrateHeader(currentArtifact.header)).toThrow(/header restorer returned v0/) + }) +}) diff --git a/packages/session/session-format/tests/json.spec.ts b/packages/session/session-format/tests/json.spec.ts new file mode 100644 index 0000000000..10f9d71033 --- /dev/null +++ b/packages/session/session-format/tests/json.spec.ts @@ -0,0 +1,73 @@ +import { describe, expect, it } from 'vitest' +import { + inspectSessionFormatVersion, + sessionFormatCount, + sessionFormatSafeInteger, + snapshotSessionFormatArtifact, + snapshotSessionFormatHeader, + snapshotSessionFormatJson, +} from '../src/index.ts' + +describe('lossless Session format JSON snapshots', () => { + it.each([ + ['negative zero', -0], + ['non-finite number', Number.POSITIVE_INFINITY], + ['undefined member', { value: undefined }], + ['sparse array', Array(1)], + ['symbol member', { [Symbol('hidden')]: true }], + ['non-enumerable member', Object.defineProperty({}, 'hidden', { value: true })], + ['accessor member', Object.defineProperty({}, 'value', { enumerable: true, get: () => 1 })], + ['array property', Object.assign([], { extra: true })], + ['array accessor', Object.defineProperty([1], '0', { enumerable: true, get: () => 1 })], + ])('refuses %s that JSON cannot preserve', (_name, value) => { + expect(() => snapshotSessionFormatJson(value)).toThrow(/JSON|sparse|member|property|number/) + }) + + it('detaches, freezes, and retains repeated non-cyclic values and __proto__ keys', () => { + const shared = { value: 1 } + const source = JSON.parse('{"__proto__":{"safe":true}}') as Record + source['values'] = [shared, shared] + + const snapshot = snapshotSessionFormatJson(source) as Record + + expect(snapshot).toEqual(source) + expect(snapshot).not.toBe(source) + expect(Object.isFrozen(snapshot)).toBe(true) + expect(Object.isFrozen(snapshot['values'])).toBe(true) + expect(Object.getPrototypeOf(snapshot)).toBe(Object.prototype) + }) + + it('refuses invalid scalar coordinates, cycles, and custom prototypes', () => { + const cyclic: { self?: unknown } = {} + cyclic.self = cyclic + class RecordValue { value = 1 } + class ArrayValue extends Array {} + + expect(() => sessionFormatCount(-1, 'count')).toThrow(/non-negative/) + expect(() => sessionFormatSafeInteger(1.5, 'integer')).toThrow(/safe integer/) + expect(() => inspectSessionFormatVersion([])).toThrow(/header/) + expect(() => snapshotSessionFormatJson(cyclic)).toThrow(/cycle/) + expect(() => snapshotSessionFormatJson(new RecordValue())).toThrow(/non-plain/) + expect(() => snapshotSessionFormatJson(new ArrayValue(1))).toThrow(/non-intrinsic/) + }) + + it.each([ + ['non-object header', { header: null, inheritedEventCount: 0, events: [] }], + ['non-array events', { header: { version: 1 }, inheritedEventCount: 0, events: null }], + ['non-object event', { header: { version: 1 }, inheritedEventCount: 0, events: [null] }], + ['non-dense seq', { header: { version: 1 }, inheritedEventCount: 0, events: [{ type: 'x', seq: 1, time: 1, data: {} }] }], + ['empty type', { header: { version: 1 }, inheritedEventCount: 0, events: [{ type: '', seq: 0, time: 1, data: {} }] }], + ['invalid time', { header: { version: 1 }, inheritedEventCount: 0, events: [{ type: 'x', seq: 0, time: 1.5, data: {} }] }], + ['missing data', { header: { version: 1 }, inheritedEventCount: 0, events: [{ type: 'x', seq: 0, time: 1 }] }], + ['oversized cut', { header: { version: 1 }, inheritedEventCount: 1, events: [] }], + ])('refuses an artifact with %s', (_name, artifact) => { + expect(() => snapshotSessionFormatArtifact(artifact as never)).toThrow() + }) + + it('refuses a non-object header snapshot', () => { + expect(() => snapshotSessionFormatHeader(null as never)).toThrow(/header|object/) + expect(() => snapshotSessionFormatHeader({ + version: 1, id: 'missing-seeded', createdAt: 1, delegationDepth: 0, + } as never)).toThrow(/isSeeded/) + }) +}) diff --git a/packages/session/session-format/tsconfig.json b/packages/session/session-format/tsconfig.json new file mode 100644 index 0000000000..754725418e --- /dev/null +++ b/packages/session/session-format/tsconfig.json @@ -0,0 +1,18 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "rootDir": "src", + "outDir": "lib/types" + }, + "include": [ + "src" + ], + "references": [ + { + "path": "../../../vendor/cosmokit" + }, + { + "path": "../../../vendor/cordis" + } + ] +} diff --git a/packages/session/session-format/tsdown.config.ts b/packages/session/session-format/tsdown.config.ts new file mode 100644 index 0000000000..7d47937fa5 --- /dev/null +++ b/packages/session/session-format/tsdown.config.ts @@ -0,0 +1,13 @@ +import { defineConfig } from 'tsdown' + +/** Build the pure migration library. */ +export default defineConfig({ + entry: ['lib/types/index.js'], + outDir: 'lib', + format: ['esm'], + platform: 'node', + target: 'es2024', + fixedExtension: false, + dts: false, + clean: false, +}) diff --git a/packages/session/session-log-deepseek/README.i18n.yaml b/packages/session/session-log-deepseek/README.i18n.yaml index 06cd142858..90b021cf0f 100644 --- a/packages/session/session-log-deepseek/README.i18n.yaml +++ b/packages/session/session-log-deepseek/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/session/session-log-deepseek/README.md -README.md: b08b2d3cde78e7a3cf7050e90ff57655ad70d409 -README.zh.md: 0ee216c6c69178695eb629b2a81faef73103b13d +README.md: 047cd8133b88ceacce1fdaafa87a60611532a18c +README.zh.md: be6b97bc889a037847289d4f5757c12823969a63 diff --git a/packages/session/session-log-deepseek/README.md b/packages/session/session-log-deepseek/README.md index b08b2d3cde..047cd8133b 100644 --- a/packages/session/session-log-deepseek/README.md +++ b/packages/session/session-log-deepseek/README.md @@ -34,12 +34,12 @@ Shipped profiles mount the plugin so an overlay can enable it, but the default c ## Request field -For a request carrying a live `sessionId`, the plugin folds the greatest accepted watermark for that exact Session identity, snapshots `Session.events`, and sends the contiguous suffix after the watermark. A process-local fold scans each event once and consumes later appends incrementally; restart and HMR rebuild it from the durable log. The version-1 field contains a v0-compatible raw session header (`seedLength` is present only for a seeded Session), numeric `afterSeq` and `throughSeq`, and every complete canonical event translated to raw-number envelope fields. Forked sessions ignore inherited parent watermarks because each watermark records the Session id sent on the accepted request. +For a request carrying a live `sessionId`, the plugin folds the greatest accepted watermark for that exact Session format generation, snapshots `Session.events`, and sends the contiguous suffix after the watermark. A process-local fold scans each event once and consumes later appends incrementally; restart and HMR rebuild it from the durable log. The version-1 field contains `sessionFormatVersion`, a raw session header (`seedLength` is present only for a seeded Session), numeric `afterSeq` and `throughSeq`, and every complete canonical event translated to raw-number envelope fields. Forked sessions ignore inherited parent watermarks because both the recorded Session id and format generation must match the request source. ## Acceptance and retry -The DeepSeek adapter calls the prepared contribution's `accept()` after HTTP 2xx, before it consumes the SSE body. Acceptance appends `session-log-deepseek/delivery-accepted` with the uploaded `throughSeq`; the next request uploads that event as part of its new suffix. Transport and non-2xx failures append no acceptance record, so later requests resend the uncertain range. Concurrent deliveries may be accepted out of order; folding the maximum matching `throughSeq` prevents cursor regression. +The DeepSeek adapter calls the prepared contribution's `accept()` after HTTP 2xx, before it consumes the SSE body. Acceptance appends `session-log-deepseek/delivery-accepted` with the uploaded `throughSeq` and `sessionFormatVersion`; a record that omits the format field denotes v0. The next request uploads that event as part of its new suffix. Transport and non-2xx failures append no acceptance record, so later requests resend the uncertain range. Concurrent deliveries may be accepted out of order; folding the maximum matching `throughSeq` prevents cursor regression. A crash after server acceptance but before the watermark reaches persistence can replay an accepted range after restart. This is the at-least-once failure direction: uncertainty creates duplicates, never a skipped sequence. The ordinary session checkpoint policy persists the watermark at the next semantic checkpoint; this plugin performs no independent I/O. diff --git a/packages/session/session-log-deepseek/README.zh.md b/packages/session/session-log-deepseek/README.zh.md index 0ee216c6c6..be6b97bc88 100644 --- a/packages/session/session-log-deepseek/README.zh.md +++ b/packages/session/session-log-deepseek/README.zh.md @@ -34,12 +34,12 @@ kind: "package-reference" ## 请求字段 -对于携带存活 `sessionId` 的请求，插件会折叠该确切会话身份的最大已接受水位，对 `Session.events` 取快照，并发送水位之后的连续后缀。进程内 fold 会让每条事件只被扫描一次并增量消费后续追加；重启与 HMR 会从持久日志重建它。版本 1 字段包含兼容 v0 的原始会话 header（仅 seeded Session 携带 `seedLength`）、数值型 `afterSeq` 与 `throughSeq`，以及把完整权威事件翻译为原始数值 envelope 字段后的数组元素。每个水位都会记录已接受请求发送的会话 id，因此 fork 会话会忽略从父会话继承的水位。 +对于携带存活 `sessionId` 的请求，插件会折叠该确切会话格式代的最大已接受水位，对 `Session.events` 取快照，并发送水位之后的连续后缀。进程内 fold 会让每条事件只被扫描一次并增量消费后续追加；重启与 HMR 会从持久日志重建它。版本 1 字段包含 `sessionFormatVersion`、原始会话 header（仅 seeded Session 携带 `seedLength`）、数值型 `afterSeq` 与 `throughSeq`，以及把完整权威事件翻译为原始数值 envelope 字段后的数组元素。只有记录的会话 id 与格式代均匹配请求来源时水位才生效，因此 fork 会话会忽略从父会话继承的水位。 ## 接受与重试 -DeepSeek 适配器会在 HTTP 2xx 后、消费 SSE（Server-Sent Events）正文前调用已准备贡献的 `accept()`。接受操作会追加 `session-log-deepseek/delivery-accepted` 及已上传的 `throughSeq`；下一次请求再把该事件作为新后缀的一部分上传。传输失败与非 2xx 失败不会追加接受记录，因此后续请求会重发不确定范围。并发交付可能乱序得到接受；折叠匹配记录中最大的 `throughSeq` 可以防止游标回退。 +DeepSeek 适配器会在 HTTP 2xx 后、消费 SSE（Server-Sent Events）正文前调用已准备贡献的 `accept()`。接受操作会追加 `session-log-deepseek/delivery-accepted`、已上传的 `throughSeq` 与 `sessionFormatVersion`；省略格式字段的记录表示 v0。下一次请求再把该事件作为新后缀的一部分上传。传输失败与非 2xx 失败不会追加接受记录，因此后续请求会重发不确定范围。并发交付可能乱序得到接受；折叠匹配记录中最大的 `throughSeq` 可以防止游标回退。 服务端接受后、持久化水位前发生崩溃，可能让恢复后的进程重放已经接受的范围。这是至少一次交付的失败方向：不确定性会制造重复，绝不会跳过序列。普通会话检查点策略会在下一个语义检查点持久化水位；本插件不执行独立 I/O。 diff --git a/packages/session/session-log-deepseek/src/index.ts b/packages/session/session-log-deepseek/src/index.ts index 5fde932581..95549478aa 100644 --- a/packages/session/session-log-deepseek/src/index.ts +++ b/packages/session/session-log-deepseek/src/index.ts @@ -50,7 +50,7 @@ interface AcceptanceFold { const acceptanceFolds = new WeakMap() -/** Translate logical Session metadata back to the stable version-0 wire header. */ +/** Translate logical Session metadata to raw external request fields. */ function wireHeader(session: Session): DeepSeekSessionLogWireHeader { const header = session.header return { @@ -88,7 +88,7 @@ function wireEvent(event: SessionEvent): DeepSeekSessionLogWireEvent { } /** - * Highest confirmed sequence for this exact session identity. + * Highest confirmed sequence for this exact Session format generation. * @param session - canonical log whose matching acceptance events are folded. * @returns greatest accepted sequence, or `-1` before any accepted request. */ @@ -103,6 +103,13 @@ export function acceptedThrough(session: Session): SessionSeqCursor { throw new Error(`session-log-deepseek: missing event ${String(index)} below captured length ${String(length)}`) } if (event.type !== 'session-log-deepseek/delivery-accepted') continue + const acceptedFormatVersion = event.data.sessionFormatVersion ?? 0 + if (!Number.isSafeInteger(acceptedFormatVersion) + || acceptedFormatVersion < 0 + || Object.is(acceptedFormatVersion, -0)) { + throw new Error(`session-log-deepseek: malformed acceptance format version at seq ${event.seq}`) + } + if (acceptedFormatVersion !== session.header.version) continue let acceptedSeq: SessionSeqType try { acceptedSeq = SessionSeq(event.data.throughSeq) @@ -141,6 +148,7 @@ export function apply(ctx: Context, config: Config): void { const suffix = session.snapshotEvents(SessionLogOffset(afterSeq + 1)) const value: DeepSeekSessionLogExtension = { version: 1, + sessionFormatVersion: session.header.version, session: wireHeader(session), afterSeq: Number(afterSeq), throughSeq: Number(throughSeq), @@ -149,7 +157,11 @@ export function apply(ctx: Context, config: Config): void { return { value, accept: () => { - session.append('session-log-deepseek/delivery-accepted', { sessionId: session.id, throughSeq }) + session.append('session-log-deepseek/delivery-accepted', { + sessionId: session.id, + sessionFormatVersion: session.header.version, + throughSeq, + }) // TODO: Add an immediate lightweight checkpoint if duplicate replay after a 2xx crash window becomes unacceptable. }, } diff --git a/packages/session/session-log-deepseek/src/invariant.ts b/packages/session/session-log-deepseek/src/invariant.ts index 66a309322b..c855666ded 100644 --- a/packages/session/session-log-deepseek/src/invariant.ts +++ b/packages/session/session-log-deepseek/src/invariant.ts @@ -16,6 +16,16 @@ export const inject = ['invariants'] /** Validate one acceptance watermark against its containing event and session. */ function validateDeliveryAccepted(session: Session, event: SessionEvent<'session-log-deepseek/delivery-accepted'>, fail: InvariantFailure): void { const { sessionId, throughSeq } = event.data + const acceptedFormatVersion = event.data.sessionFormatVersion ?? 0 + if (!Number.isSafeInteger(acceptedFormatVersion) + || acceptedFormatVersion < 0 + || Object.is(acceptedFormatVersion, -0)) { + fail( + 'session-log-deepseek/delivery-accepted sessionFormatVersion must be a non-negative safe integer' + + `, got ${String(acceptedFormatVersion)}`, + ) + } + if (acceptedFormatVersion !== session.header.version) return const inherited = session.header.parentSession !== undefined && !session.isOwnSeq(event.seq) if (sessionId !== session.id && !inherited) { diff --git a/packages/session/session-log-deepseek/src/types.ts b/packages/session/session-log-deepseek/src/types.ts index 3205880a44..03c9cb9d45 100644 --- a/packages/session/session-log-deepseek/src/types.ts +++ b/packages/session/session-log-deepseek/src/types.ts @@ -3,7 +3,7 @@ import type { SessionEvent } from '@deepseek-ai/dsh-session' import type { JsonValue } from '@deepseek-ai/dsh-util-values' -/** Version-0 Session header fields serialized on the external request wire. */ +/** Session header fields serialized as raw JSON primitives on the external request wire. */ export interface DeepSeekSessionLogWireHeader { readonly version: number readonly id: string @@ -36,6 +36,8 @@ export interface DeepSeekSessionLogWireEvent { /** Versioned incremental session-log field carried by an official DeepSeek request. */ export interface DeepSeekSessionLogExtension { readonly version: 1 + /** Session format generation represented by this suffix. */ + readonly sessionFormatVersion: number readonly session: DeepSeekSessionLogWireHeader /** Highest sequence durably recorded as accepted before this request, or `-1`. */ readonly afterSeq: number @@ -57,6 +59,8 @@ declare module '@deepseek-ai/dsh-session/types' { 'session-log-deepseek/delivery-accepted': { /** Session identity the accepted delivery carried; inherited fork markers retain the parent's id. */ sessionId: import('@deepseek-ai/dsh-session/types').SessionId + /** Accepted Session format generation; absence identifies version 0. */ + sessionFormatVersion?: number /** Last canonical event included in the accepted request. */ throughSeq: import('@deepseek-ai/dsh-session/types').SessionSeq } diff --git a/packages/session/session-log-deepseek/tests/invariant.spec.ts b/packages/session/session-log-deepseek/tests/invariant.spec.ts index e9f2f9d3d7..14e8e61499 100644 --- a/packages/session/session-log-deepseek/tests/invariant.spec.ts +++ b/packages/session/session-log-deepseek/tests/invariant.spec.ts @@ -1,7 +1,13 @@ import { afterEach, describe, expect, it } from 'vitest' import { Context } from '@deepseek-ai/cordis' import InvariantRegistry, { InvariantError } from '@deepseek-ai/dsh-invariants' -import SessionStore, { SessionId, SessionLogOffset, SessionSeq } from '@deepseek-ai/dsh-session' +import SessionStore, { + SESSION_FORMAT_VERSION, + SessionId, + SessionLogOffset, + SessionSeq, + type Session, +} from '@deepseek-ai/dsh-session' import * as SessionLogInvariant from '../src/invariant.ts' import type {} from '../src/types.ts' @@ -25,17 +31,69 @@ describe('DeepSeek session-log acceptance invariant', () => { const ctx = await setup() const session = ctx.sessions.create(SessionId('valid')) session.append('turn/start', { turn: 1 }) - expect(() => session.append('session-log-deepseek/delivery-accepted', { sessionId: session.id, throughSeq: SessionSeq(0) })) + expect(() => session.append('session-log-deepseek/delivery-accepted', { + sessionId: session.id, + throughSeq: SessionSeq(0), + sessionFormatVersion: SESSION_FORMAT_VERSION, + })) .not.toThrow() }) - it('rejects a live watermark for another Session or a non-earlier sequence', async () => { + it('treats an omitted format generation as v0 before validating its frozen sequence', async () => { + const ctx = await setup() + const session = ctx.sessions.create(SessionId('other-generation')) + session.append('turn/start', { turn: 1 }) + expect(() => session.append('session-log-deepseek/delivery-accepted', { + sessionId: SessionId('unrelated-old-identity'), + throughSeq: SessionSeq(99), + })).not.toThrow() + }) + + it.each([-1, 0.5])('rejects malformed acceptance format version %s', async (sessionFormatVersion) => { + const ctx = await setup() + const session = ctx.sessions.create(SessionId(`invalid-format-${sessionFormatVersion}`)) + session.append('turn/start', { turn: 1 }) + expect(() => session.append('session-log-deepseek/delivery-accepted', { + sessionId: session.id, + throughSeq: SessionSeq(0), + sessionFormatVersion, + })).toThrow(expect.objectContaining>({ + code: 'INVARIANT', + packageName: '@deepseek-ai/dsh-session-log-deepseek', + })) + }) + + it('rejects negative-zero format versions restored across the owned invariant boundary', async () => { + const ctx = new Context() + contexts.push(ctx) + await ctx.plugin(SessionStore) + await ctx.plugin(InvariantRegistry, { enabled: true }) + const session = { + header: { version: SESSION_FORMAT_VERSION }, + snapshotEvents: () => [{ + type: 'session-log-deepseek/delivery-accepted' as const, + seq: SessionSeq(0), + time: 1, + data: { sessionId: SessionId('negative-zero'), throughSeq: SessionSeq(0), sessionFormatVersion: -0 }, + }], + isOwnSeq: () => true, + } as unknown as Session + ctx.sessions.list = () => [session] + + await expect(ctx.plugin(SessionLogInvariant)).rejects.toMatchObject({ + code: 'INVARIANT', + packageName: '@deepseek-ai/dsh-session-log-deepseek', + }) + }) + + it('rejects current-generation mismatches and leaves v0 watermarks inert', async () => { const ctx = await setup() const wrongId = ctx.sessions.create(SessionId('wrong-id')) wrongId.append('turn/start', { turn: 1 }) expect(() => wrongId.append('session-log-deepseek/delivery-accepted', { sessionId: SessionId('other'), throughSeq: SessionSeq(0), + sessionFormatVersion: SESSION_FORMAT_VERSION, })).toThrow(expect.objectContaining>({ code: 'INVARIANT', packageName: '@deepseek-ai/dsh-session-log-deepseek', @@ -46,6 +104,18 @@ describe('DeepSeek session-log acceptance invariant', () => { expect(() => wrongSeq.append('session-log-deepseek/delivery-accepted', { sessionId: wrongSeq.id, throughSeq: SessionSeq(1), + sessionFormatVersion: SESSION_FORMAT_VERSION, + })).toThrow(expect.objectContaining>({ + code: 'INVARIANT', + packageName: '@deepseek-ai/dsh-session-log-deepseek', + })) + + const invalidSeq = ctx.sessions.create(SessionId('invalid-seq')) + invalidSeq.append('turn/start', { turn: 1 }) + expect(() => invalidSeq.append('session-log-deepseek/delivery-accepted', { + sessionId: invalidSeq.id, + throughSeq: -1 as never, + sessionFormatVersion: SESSION_FORMAT_VERSION, })).toThrow(expect.objectContaining>({ code: 'INVARIANT', packageName: '@deepseek-ai/dsh-session-log-deepseek', @@ -53,10 +123,7 @@ describe('DeepSeek session-log acceptance invariant', () => { expect(() => wrongSeq.append('session-log-deepseek/delivery-accepted', { sessionId: wrongSeq.id, throughSeq: -1 as never, - })).toThrow(expect.objectContaining>({ - code: 'INVARIANT', - packageName: '@deepseek-ai/dsh-session-log-deepseek', - })) + })).not.toThrow() }) it('validates existing history when the invariant loads after the Session', async () => { @@ -71,7 +138,11 @@ describe('DeepSeek session-log acceptance invariant', () => { type: 'session-log-deepseek/delivery-accepted', seq: SessionSeq(1), time: 2, - data: { sessionId: id, throughSeq: SessionSeq(1) }, + data: { + sessionId: id, + throughSeq: SessionSeq(1), + sessionFormatVersion: SESSION_FORMAT_VERSION, + }, }, ] }) @@ -101,7 +172,11 @@ describe('DeepSeek session-log acceptance invariant', () => { type: 'session-log-deepseek/delivery-accepted', seq: SessionSeq(1), time: 2, - data: { sessionId: parentId, throughSeq: SessionSeq(0) }, + data: { + sessionId: parentId, + throughSeq: SessionSeq(0), + sessionFormatVersion: SESSION_FORMAT_VERSION, + }, }, ], inheritedEventCount: SessionLogOffset(2), diff --git a/packages/session/session-log-deepseek/tests/upload.spec.ts b/packages/session/session-log-deepseek/tests/upload.spec.ts index a3663867f1..84d8afbd18 100644 --- a/packages/session/session-log-deepseek/tests/upload.spec.ts +++ b/packages/session/session-log-deepseek/tests/upload.spec.ts @@ -1,6 +1,14 @@ import { afterEach, describe, expect, expectTypeOf, it } from 'vitest' import { Context } from '@deepseek-ai/cordis' -import SessionStore, { Session, SessionId, SessionLogOffset, SessionSeq, type CreateSessionOptions, type SessionEvent } from '@deepseek-ai/dsh-session' +import SessionStore, { + SESSION_FORMAT_VERSION, + Session, + SessionId, + SessionLogOffset, + SessionSeq, + type CreateSessionOptions, + type SessionEvent, +} from '@deepseek-ai/dsh-session' import DeepSeekLlmApiExtensionRegistry from '@deepseek-ai/dsh-deepseek-llm-api-extensions' import { createUserMessage } from '@deepseek-ai/dsh-llm' import type { JsonValue } from '@deepseek-ai/dsh-util-values' @@ -42,6 +50,7 @@ function body(text = 'x'.repeat(300)) { describe('incremental DeepSeek session-log upload', () => { it('publishes raw numeric sequence fields on its external wire DTO', () => { + expectTypeOf().toEqualTypeOf() expectTypeOf().toEqualTypeOf() expectTypeOf().toEqualTypeOf() expectTypeOf().toEqualTypeOf() @@ -72,10 +81,19 @@ describe('incremental DeepSeek session-log upload', () => { const first = await ctx.deepseekLlmApiExtensions.prepare({ body: body(), signal: SIGNAL, sessionId: session.id }) const firstPayload = first.fields.dsh_session_log - expect(firstPayload).toMatchObject({ afterSeq: -1, throughSeq: 1 }) + expect(firstPayload).toMatchObject({ + sessionFormatVersion: SESSION_FORMAT_VERSION, + afterSeq: -1, + throughSeq: 1, + }) expect(firstPayload?.events).toHaveLength(2) await first.accept() expect(SessionLogDeepSeek.acceptedThrough(session)).toBe(1) + expect(session.snapshotEvents().at(-1)?.data).toEqual({ + sessionId: session.id, + throughSeq: 1, + sessionFormatVersion: SESSION_FORMAT_VERSION, + }) session.append('step/end', { turn: 1, step: 1 }) const second = await ctx.deepseekLlmApiExtensions.prepare({ body: body(), signal: SIGNAL, sessionId: session.id }) @@ -130,14 +148,19 @@ describe('incremental DeepSeek session-log upload', () => { type: 'session-log-deepseek/delivery-accepted', seq: SessionSeq(1), time: 2, - data: { sessionId: id, throughSeq: SessionSeq(0) }, + data: { + sessionId: id, + throughSeq: SessionSeq(0), + sessionFormatVersion: SESSION_FORMAT_VERSION, + }, }, ] let reads = 0 const session = { id, - get seq() { return events.length }, - eventAt(seq: number) { + header: { version: SESSION_FORMAT_VERSION }, + get seq() { return SessionLogOffset(events.length) }, + eventAt(seq: ReturnType) { reads++ return events[seq] }, @@ -155,7 +178,11 @@ describe('incremental DeepSeek session-log upload', () => { type: 'session-log-deepseek/delivery-accepted', seq: SessionSeq(3), time: 4, - data: { sessionId: id, throughSeq: SessionSeq(2) }, + data: { + sessionId: id, + throughSeq: SessionSeq(2), + sessionFormatVersion: SESSION_FORMAT_VERSION, + }, }, ) expect(SessionLogDeepSeek.acceptedThrough(session)).toBe(2) @@ -165,7 +192,8 @@ describe('incremental DeepSeek session-log upload', () => { it('rejects a missing event below the captured Session length', () => { const session = { id: SessionId('missing-event'), - seq: 1, + header: { version: SESSION_FORMAT_VERSION }, + seq: SessionLogOffset(1), eventAt: () => undefined, } as unknown as Session @@ -173,6 +201,56 @@ describe('incremental DeepSeek session-log upload', () => { .toThrow('session-log-deepseek: missing event 0 below captured length 1') }) + it('ignores another format generation before interpreting its frozen sequence', () => { + const id = SessionId('migrated-generation') + const events = [ + { type: 'turn/start', seq: SessionSeq(0), time: 1, data: { turn: 1 } }, + { + type: 'session-log-deepseek/delivery-accepted', + seq: SessionSeq(1), + time: 2, + data: { sessionId: id, throughSeq: SessionSeq(99) }, + }, + { type: 'step/start', seq: SessionSeq(2), time: 3, data: { turn: 1, step: 1 } }, + { + type: 'session-log-deepseek/delivery-accepted', + seq: SessionSeq(3), + time: 4, + data: { + sessionId: id, + throughSeq: SessionSeq(2), + sessionFormatVersion: SESSION_FORMAT_VERSION, + }, + }, + ] as SessionEvent[] + const migrated = { + id, + header: { version: SESSION_FORMAT_VERSION }, + get seq() { return SessionLogOffset(events.length) }, + eventAt: (seq: ReturnType) => events[seq], + } as unknown as Session + + expect(SessionLogDeepSeek.acceptedThrough(migrated)).toBe(2) + }) + + it.each([-1, -0, 0.5])('rejects malformed acceptance format version %s', (sessionFormatVersion) => { + const id = SessionId(`malformed-format-${sessionFormatVersion}`) + const events = [{ + type: 'session-log-deepseek/delivery-accepted', + seq: SessionSeq(0), + time: 1, + data: { sessionId: id, throughSeq: SessionSeq(0), sessionFormatVersion }, + }] as unknown as SessionEvent[] + const session = { + id, + header: { version: SESSION_FORMAT_VERSION }, + get seq() { return SessionLogOffset(events.length) }, + eventAt: (seq: ReturnType) => events[seq], + } as unknown as Session + + expect(() => SessionLogDeepSeek.acceptedThrough(session)).toThrow(/malformed acceptance format version/) + }) + it('omits the field for direct or stale requests and uploads the prior acceptance marker next', async () => { const { ctx, session } = await harness('edges') await expect(ctx.deepseekLlmApiExtensions.prepare({ body: body(), signal: SIGNAL })) @@ -199,7 +277,7 @@ describe('incremental DeepSeek session-log upload', () => { expect(prepared.fields.dsh_session_log?.events).toEqual(session.snapshotEvents()) }) - it('translates logical brands and isSeeded into the raw v0 upload DTO', async () => { + it('translates logical brands and isSeeded into the raw upload DTO', async () => { const firstMessage = createUserMessage({ content: [{ type: 'text', text: 'first' }], source: { kind: 'user' }, @@ -227,7 +305,7 @@ describe('incremental DeepSeek session-log upload', () => { }, ] satisfies SessionEvent[] const { ctx, session } = await harness('wire-child', seed, { - inheritedEventCount: SessionLogOffset(0), + inheritedEventCount: SessionLogOffset(seed.length), meta: { cwd: '/wire-workspace', parentSession: SessionId('wire-parent'), @@ -243,11 +321,11 @@ describe('incremental DeepSeek session-log upload', () => { }) const wire = JSON.parse(JSON.stringify(prepared.fields.dsh_session_log)) as Record expect(wire.session).toMatchObject({ - version: 0, + version: SESSION_FORMAT_VERSION, id: 'wire-child', - parentSession: 'wire-parent', cwd: '/wire-workspace', - seedLength: 0, + parentSession: 'wire-parent', + seedLength: seed.length, origin: 'subagent', delegationDepth: 1, agentPreset: 'minimal', @@ -270,17 +348,89 @@ describe('incremental DeepSeek session-log upload', () => { }) }) + it('translates ignorable and surface event envelopes to raw wire values', async () => { + const seed: SessionEvent[] = [ + { type: 'turn/start', seq: SessionSeq(0), time: 1, data: { turn: 1 } }, + { + type: 'user/message', + seq: SessionSeq(1), + time: 2, + data: createUserMessage({ + content: [{ type: 'text', text: 'first' }], + source: { kind: 'user' }, + }), + ignorable: true, + sourceEventSeqs: [SessionSeq(0)], + surfaceOp: 'append', + }, + { + type: 'user/message', + seq: SessionSeq(2), + time: 3, + data: createUserMessage({ + content: [{ type: 'text', text: 'replacement' }], + source: { kind: 'user' }, + }), + sourceEventSeqs: [SessionSeq(1)], + surfaceOp: { op: 'replace', start: SessionSeq(1), end: SessionSeq(1) }, + }, + ] + const { ctx, session } = await harness('wire-events', seed) + + const prepared = await ctx.deepseekLlmApiExtensions.prepare({ + body: body(), signal: SIGNAL, sessionId: session.id, + }) + const events = prepared.fields.dsh_session_log?.events ?? [] + + expect(events[0]).not.toHaveProperty('surfaceOp') + expect(events[1]).toMatchObject({ + type: 'user/message', + ignorable: true, + sourceEventSeqs: [0], + surfaceOp: 'append', + }) + expect(events[2]).toMatchObject({ + type: 'user/message', + sourceEventSeqs: [1], + surfaceOp: { op: 'replace', start: 1, end: 1 }, + }) + }) + it('fails closed on a malformed persisted acceptance watermark', async () => { - for (const [id, throughSeq] of [['current', 0], ['negative', -1]] as const) { - const malformed = [{ - type: 'session-log-deepseek/delivery-accepted', - seq: 0, - time: 1, - data: { sessionId: `malformed-${id}`, throughSeq }, - }] as unknown as SessionEvent[] - const session = Session.create(SessionId(`malformed-${id}`), malformed) - expect(() => SessionLogDeepSeek.acceptedThrough(session)).toThrow(/malformed acceptance watermark/) - } + const malformed = [{ + type: 'session-log-deepseek/delivery-accepted', + seq: 0, + time: 1, + data: { + sessionId: 'malformed', + throughSeq: 0, + sessionFormatVersion: SESSION_FORMAT_VERSION, + }, + }] as unknown as SessionEvent[] + const session = Session.create(SessionId('malformed'), malformed) + expect(() => SessionLogDeepSeek.acceptedThrough(session)).toThrow(/malformed acceptance watermark/) + }) + + it('rejects a negative persisted acceptance watermark before comparing it', () => { + const id = SessionId('negative-watermark') + const events = [{ + type: 'session-log-deepseek/delivery-accepted', + seq: SessionSeq(0), + time: 1, + data: { + sessionId: id, + throughSeq: -1, + sessionFormatVersion: SESSION_FORMAT_VERSION, + }, + }] as unknown as SessionEvent[] + const session = { + id, + header: { version: SESSION_FORMAT_VERSION }, + get seq() { return SessionLogOffset(events.length) }, + eventAt: (seq: ReturnType) => events[seq], + } as unknown as Session + + expect(() => SessionLogDeepSeek.acceptedThrough(session)).toThrow(/malformed acceptance watermark/) }) it('withdraws its request field when the contributing plugin reloads', async () => { diff --git a/packages/session/session-persistence-jsonl/README.i18n.yaml b/packages/session/session-persistence-jsonl/README.i18n.yaml index 0a44d29f0c..0ca707472c 100644 --- a/packages/session/session-persistence-jsonl/README.i18n.yaml +++ b/packages/session/session-persistence-jsonl/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/session/session-persistence-jsonl/README.md -README.md: d3dfaf8affb142979ede69d86bbf0b1587e8c39c -README.zh.md: 15fc815523355674a1f325b604dd2655f56f7f58 +README.md: 546f8052d20624560da17b45cc79179228f6f5b7 +README.zh.md: a8285483201b5e23c21591e703552485f729ed58 diff --git a/packages/session/session-persistence-jsonl/README.md b/packages/session/session-persistence-jsonl/README.md index d3dfaf8aff..546f8052d2 100644 --- a/packages/session/session-persistence-jsonl/README.md +++ b/packages/session/session-persistence-jsonl/README.md @@ -9,7 +9,7 @@ English | [中文](README.zh.md) ## Summary -`dsh-session-persistence-jsonl` stores each session in its own append-only JSONL log — checksummed Zstandard frames by default, raw newline-delimited lines when compression is disabled. It serves the same logical `SessionEvent` stream as any persistence backend, so choosing it changes nothing for the agent loop, the model, or replay; compression, packing, and crash recovery are storage-internal details. Choose it when consumers need a per-session artifact on disk: `locate(meta)` returns the transcript path, and the logs are readable as plain lines when `compression: 'none'` is selected. A root directory is the one required configuration; durability, lazy materialization, and interrupted-turn recovery come with the backend. +`dsh-session-persistence-jsonl` stores each Session in canonical version-named JSONL generations whose ordinary writes append to the current file — checksummed Zstandard frames by default, raw newline-delimited lines when compression is disabled. Released v0 uses `session.jsonl[.zstd]`; v1 and later use lowercase `session.vN.jsonl[.zstd]`. Migration publishes a previously absent successor beside the unchanged source and never renames, replaces, or deletes a committed generation path. The backend serves the same logical `SessionEvent` stream as any persistence backend, so physical naming, compression, packing, migration, and crash recovery remain storage details. Choose it when consumers need per-session artifacts on disk: `locate(meta)` returns the version-qualified target, and raw generations are line-readable with `compression: 'none'`. A root directory is the one required configuration; durability, lazy materialization, and interrupted-turn recovery come with the backend. ## Table of Contents @@ -54,25 +54,27 @@ The generated [configuration catalog](../../../docs/config-catalog.md#deepseek-a ### On-disk layout -Each session gets a session-owned directory under a readable project directory; the first logical line is the private v0 physical header, followed by one storage record per logical event (or one packed chunk row per eligible run). Its optional numeric `seedLength` remains byte-compatible: absence decodes to `SessionHeader.isSeeded: false`, while zero or a positive value decodes to `isSeeded: true` plus the exact `inheritedEventCount`. Storage records use the lossless provenance representation described below: +Each Session gets a session-owned directory under a readable project directory. Every canonical generation starts with a physical header whose version equals its filename, followed by one storage record per logical event or one packed chunk row per eligible run. The format catalog translates every supported historical header and event representation before the persistence coordinator receives current logical values. Storage records use the lossless provenance representation described below: ```text / ----/ # readable project directory (or _no-cwd/) / # session-owned directory - session.jsonl.zstd # default: checksummed header frame + append frames - session.jsonl # only with compression: 'none' + session.jsonl.zstd # released v0, compressed root + session.v1.jsonl.zstd # released v1, compressed root + session.jsonl # released v0, raw root + session.v1.jsonl # released v1, raw root; later versions use vN ``` -Session ids are injectively escaped to one safe path segment before use (no traversal, no collision). The normalized cwd keeps the project directory readable for navigation; cwd strings that normalize alike share a project directory while session ids still select distinct session directories. `locate(meta)` returns `{ kind: 'jsonl', path }` for the fixed transcript inside the resolved directories, performing no filesystem I/O. +Session ids are injectively escaped to one safe path segment before use (no traversal, no collision). The normalized cwd keeps the project directory readable for navigation; cwd strings that normalize alike share a project directory while session ids still select distinct session directories. `locate(meta)` performs no filesystem I/O and returns `{ kind: 'jsonl', path }` for `meta.version`: suffixless for v0 and `.vN` for every positive version. Listing instead reports the exact highest generation it found on disk. ### Durability and crash semantics -A session is materialized lazily: `create(meta)` writes nothing, and the first `append` writes and `fsync`s the encoded header and first batch through a no-overwrite publish — so a created-but-never-appended session leaves nothing on disk unless a lifecycle consumer calls `ensureMaterialized`, which publishes one header frame without an event. Flushed events are never rewritten; each subsequent batch appends lines or one compressed frame, and a caught write or sync failure rolls the file back to its prior length. After a crash, `load` preserves an interrupted final turn: it keeps the complete decoded records of an incomplete last frame, truncates from that frame's start, and re-encodes the records with the synthetic tool, step, and turn closers required by the shared persistence contract. Only a never-fully-written torn tail is discarded; checksum, decompression, or structural failure in the committed prefix rejects as corruption. +A Session is materialized lazily: `create(meta)` writes nothing, and the first `append` writes and `fsync`s the encoded header and first batch through no-overwrite publication at the current version's canonical filename. A created-but-never-appended Session therefore leaves nothing on disk unless a lifecycle consumer calls `ensureMaterialized`, which publishes one header frame without an event. Flushed current-generation events append lines or compressed frames; a caught write or sync failure rolls that file back to its prior length without replacing its path or inode. A body read of a supported historical generation reads one stable exact source, decodes its recoverable prefix, composes all required edges and current interrupted-turn repair in memory, writes and validates only the final target in a same-directory temporary file, rechecks the source fingerprint, then publishes the previously absent target without overwrite and syncs the namespace. POSIX links the temporary inode to the target; Windows moves the temporary file with write-through and no replacement. If another writer already published the target, the backend accepts it only when it is a regular current-format file with exactly the expected bytes. The source path, bytes, and inode remain unchanged, intermediate versions never reach disk, and the committed target is reopened through the current reader before a Session is constructed. ### Reading the logs -`inspect(id)` returns an immutable balanced view with its exact inherited cut without committing recovery. `readFrom(id, fromOffset)` accepts a `SessionLogOffset`, returns stored events at or past that offset, and retains the same cut beside the suffix; sequential media like JSONL parse the whole artifact and skip forward. Header-only listing exposes `isSeeded` without reading event bodies. With `compression: 'none'`, the log is newline-delimited text an external reader can consume directly; the compressed default must be read through the backend. +`inspect(id)` returns an immutable balanced view with its exact inherited cut; it does not commit crash recovery for an already-current generation. `readFrom(id, fromOffset)` accepts a `SessionLogOffset`, returns stored events at or past that offset, and retains the same cut beside the suffix; sequential media like JSONL parse the whole selected generation and skip forward. On its first cold body open, the backend scans the Session directory, selects the numerically highest canonical filename, refuses it when its version is newer than the build, or publishes the final current successor for a supported older version. A validated current selection is cached for later opens in the same backend instance under the one-writer assumption; `list` and `listSnapshots` always rescan and report the directory's highest generation. `readRaw` returns that selected generation and preserves its logical basename in `filename` (`session.jsonl` for v0 or `session.vN.jsonl` for v1+; `.zstd` is omitted because `content` is decoded). Retained lower generations are never automatic fallback, restore, or downgrade inputs. With `compression: 'none'`, every generation is newline-delimited text an external reader can consume directly; compressed generations require Zstandard decoding. ----- @@ -86,11 +88,11 @@ This section explains the physical encoding and write path; the observable contr ### Design concept -The backend is a thin storage layer over the shared [PersistenceCoordinator](../session-persistence/README.md#understand-the-implementation): it loads stored records, appends batches, commits repairs, and delegates lifecycle orchestration to the coordinator. Its physical identity is a file revision: device, inode, size, and nanosecond timestamps identify one log and change after append or repair, which is what `listSnapshots` and retained-preparation validation use. +The backend is a thin storage layer over the shared [PersistenceCoordinator](../session-persistence/README.md#understand-the-implementation): it loads stored records, appends batches, commits repairs, and delegates lifecycle orchestration to the coordinator. Its fused body-read hook carries one stable physical snapshot across generation classification or migration and current decoding, so the selected file is not read twice. After a current generation is validated, the backend caches its path for later same-process body opens; header listing deliberately bypasses that cache. Physical identity remains a file revision: device, inode, size, and nanosecond timestamps identify one generation and change after append or repair, which is what `listSnapshots` and retained-preparation validation use. ### Physical encoding -The default artifact is a standard concatenation of independent [Zstandard frames](../../../.agents/notes/implemented/architecture/2026-07-19-zstandard-jsonl-session-logs.md): one checksummed frame containing only the header line, then one checksummed frame per durable append batch, using Node's built-in Zstandard API at its default compression level (no level knob). `sourceEventSeqs` uses a lossless storage representation: consecutive runs of at least three sequence numbers become `[start, end]` pairs, any other list stays verbatim, and reading expands the exact in-memory array. Listing reads and validates only the header frame. `compression: 'none'` keeps the same storage-form logical lines without frame compression. A root belongs to one encoding: startup discovery and targeted lookup reject the opposite suffix, and there is no format or compression migration, mixed-root fallback, or dual write. When `packChunks` is enabled, an eligible run of ≥3 consecutive same-block `assistant/chunk` delta events becomes one packed row (`text-chunks`/`reasoning-chunks`/`tool-call-chunks`) whose `seq0`/`time0` and per-member `dt` gaps reconstruct every member exactly; the lossless codec lives in `dsh-session` and reading is layout-blind, so packed, unpacked, and mixed files load identically. +The default artifact is a standard concatenation of independent [Zstandard frames](../../../.agents/notes/implemented/architecture/2026-07-19-zstandard-jsonl-session-logs.md): one checksummed frame containing only the header line, then one checksummed frame per durable append batch, using Node's built-in Zstandard API at its default compression level (no level knob). `sourceEventSeqs` uses a lossless storage representation: consecutive runs of at least three sequence numbers become `[start, end]` pairs, any other list stays verbatim, and reading expands the exact in-memory array. Listing reads and validates only the header frame. `compression: 'none'` keeps the same storage-form logical lines without frame compression. The configured suffix selects framing once; generation migration operates on decoded JSON values and uses the same publication algorithm for raw and compressed files. A root belongs to one encoding: startup discovery and targeted lookup reject the opposite suffix, and there is no compression migration, mixed-root fallback, or dual write. When `packChunks` is enabled, eligible chunk runs use the format codec's lossless packed representation. ### Source map @@ -98,8 +100,9 @@ The default artifact is a standard concatenation of independent [Zstandard frame |---|---| | [`src/index.ts`](src/index.ts) | Plugin entry: `Config` schema, backend class, coordinator wiring | | [`src/format.ts`](src/format.ts) | Log path derivation, header encoding, record scanning, packed-row layout | +| [`src/generation.ts`](src/generation.ts) | Exact source reads, final-target staging, source recheck, and exclusive successor publication | | [`src/zstd.ts`](src/zstd.ts) | Zstandard frame compression, decoding, and frame scanning | -| [`src/win32.ts`](src/win32.ts) | Windows write-through publish and directory creation | +| [`src/win32.ts`](src/win32.ts) | Windows write-through no-overwrite file and directory publication | | — | No runtime invariant companion is published; persistence correctness requires backend round-trip and crash-tail tests; this package exposes no continuously observable in-process relation. | @@ -113,6 +116,7 @@ Read these pages when the package-level contract is not enough. They move from t - [Session persistence subsystem](../../../docs/subsystems/persistence.md) — backend-neutral service semantics and provider relationships. - [Session persistence seam](../session-persistence/README.md) — the service contract this backend implements. +- [Released Session migrations](../../../.agents/notes/implemented/architecture/2026-08-31-released-session-format-migrations.md) — adjacent-chain, immutable naming, and exclusive publication guarantees. - [Project-session directory decision](../../../.agents/notes/implemented/architecture/2026-07-24-project-session-directories.md) — the layout tradeoff behind project and session directories. - [Zstandard JSONL session logs](../../../.agents/notes/implemented/architecture/2026-07-19-zstandard-jsonl-session-logs.md) — the checksummed-frame encoding rationale. @@ -142,12 +146,13 @@ JSONL storage does not mutate live request prefixes. A resumed loop can reuse pr These limits define when this backend is a poor fit or needs special operational care. They are current package constraints, not a task backlog. -- **Only the configured encoding and current `SESSION_FORMAT_VERSION` (v0) load** — changing compression requires a separate or fresh root, or selecting raw mode; the pre-release format has no migration. +- **Only the configured encoding loads** — supported older Session formats migrate within that encoding; changing compression still requires a separate or fresh root. - **The flat-file storage layout does not load** — use a separate root or move pre-release artifacts into the project/session directory layout before loading. - **Compressed files are not directly line-readable** — use the backend to load them, or select `compression: 'none'` before writing a fresh root when external line readers are required. -- **Nothing deletes session files** — logs accumulate under `root` until removed externally; the seam has no deletion API. -- **One live writer per session** — append and repair are coordinated only inside the owning backend instance; another instance or process must not write the same session until that owner reaches quiescent disposal. -- **POSIX materialization requires hard-link support** — first append uses `link()` so same-id races fail instead of overwriting a committed log; Windows uses write-through rename without replacement. +- **Nothing deletes session generations** — every canonical generation accumulates under `root` until removed externally; the seam has no deletion API and never falls back from the numerically highest filename to an older one. +- **Retention is not downgrade support** — predecessor files preserve exact evidence and permit explicit operator copying or inspection, but this build does not restore them automatically and makes no promise that an older binary can safely reopen a directory after a successor exists. +- **One live writer per session** — append, repair, and migration are coordinated only inside the owning backend instance; cross-process writer fencing requires a future per-session lock. +- **Publication requires no-overwrite filesystem primitives** — POSIX first materialization and successor publication use `link()` plus parent-directory `fsync`; Windows uses write-through moves that refuse an existing destination. Temporary stages may move or unlink, but committed generation paths do not. ### Dev Note diff --git a/packages/session/session-persistence-jsonl/README.zh.md b/packages/session/session-persistence-jsonl/README.zh.md index 15fc815523..a828548320 100644 --- a/packages/session/session-persistence-jsonl/README.zh.md +++ b/packages/session/session-persistence-jsonl/README.zh.md @@ -9,7 +9,7 @@ kind: "package-reference" ## 概述 -`dsh-session-persistence-jsonl` 把每个会话存为一份仅追加 JSONL 日志——默认以带校验和的 Zstandard 帧存储，禁用压缩时以换行分隔的原始文本行存储。它提供与任何持久化后端相同的逻辑 `SessionEvent` 流，因此选择它不会改变 agent loop、模型或回放的任何行为；压缩、打包与崩溃恢复都是存储内部细节。当消费方需要按会话的磁盘产物时选择它：`locate(meta)` 返回 transcript 路径，选择 `compression: 'none'` 后日志可作为纯文本按行读取。根目录是唯一必填配置；持久性、延迟实体化与中断轮次恢复都随后端提供。 +`dsh-session-persistence-jsonl` 以规范的具名版本 JSONL generation 存储每个 Session，普通写入向当前文件追加：默认使用带 checksum 的 Zstandard frame，禁用压缩时使用换行分隔的原始文本行。已发布 v0 使用 `session.jsonl[.zstd]`；v1 及后续版本使用小写 `session.vN.jsonl[.zstd]`。迁移会在不改变源文件的情况下于其旁边发布此前不存在的后继文件，绝不重命名、替换或删除已提交 generation 路径。后端提供与任何持久化后端相同的逻辑 `SessionEvent` 流，因此物理命名、压缩、打包、迁移与崩溃恢复仍是存储细节。当消费方需要逐会话磁盘产物时选择它：`locate(meta)` 返回版本限定目标，选择 `compression: 'none'` 后原始 generation 可作为纯文本逐行读取。根目录是唯一必填配置；持久性、延迟物化与中断轮次恢复都随后端提供。 ## 目录 @@ -54,25 +54,27 @@ kind: "package-reference" ### 磁盘布局 -每个会话在可读项目目录下获得一个会话自有目录；第一个逻辑行是私有 v0 物理 header，之后每个逻辑事件一条存储记录（或每个符合条件的连续段一条打包分片行）。其可选数字 `seedLength` 保持字节兼容：缺席解码为 `SessionHeader.isSeeded: false`，零或正值解码为 `isSeeded: true` 加精确 `inheritedEventCount`。存储记录使用下文所述的无损来源序列表示： +每个 Session 在可读项目目录下获得一个会话自有目录。每个规范 generation 都以版本与文件名一致的物理 header 开始，之后每个逻辑事件一条存储记录，或每个符合条件的连续段一条打包分片行。格式目录会先翻译所有受支持的历史 header 与事件表示，持久化协调器只接收当前逻辑值。存储记录使用下文所述的无损来源序列表示： ```text / ----/ # readable project directory (or _no-cwd/) / # session-owned directory - session.jsonl.zstd # default: checksummed header frame + append frames - session.jsonl # only with compression: 'none' + session.jsonl.zstd # released v0, compressed root + session.v1.jsonl.zstd # released v1, compressed root + session.jsonl # released v0, raw root + session.v1.jsonl # released v1, raw root; later versions use vN ``` -会话 id 在使用前被单射转义为一个安全路径段（无遍历、无冲突）。规范化 cwd 让项目目录保持可读、便于导航；规范化相同的 cwd 字符串共享项目目录，而会话 id 仍选择不同会话目录。`locate(meta)` 返回已解析目录内固定 transcript 的 `{ kind: 'jsonl', path }`，不执行任何文件系统 I/O。 +会话 id 在使用前被单射转义为一个安全路径段（无遍历、无冲突）。规范化 cwd 让项目目录保持可读、便于导航；规范化相同的 cwd 字符串共享项目目录，而会话 id 仍选择不同会话目录。`locate(meta)` 不执行任何文件系统 I/O，并为 `meta.version` 返回 `{ kind: 'jsonl', path }`：v0 无版本后缀，每个正版本使用 `.vN`。列表则报告它在磁盘上找到的精确最高 generation。 ### 持久性与崩溃语义 -会话延迟实体化：`create(meta)` 不写入任何内容，第一次 `append` 通过无覆盖发布写入并 `fsync` 编码后的 header 与第一批——因此已创建但从未 append 的会话不留下任何磁盘内容，除非生命周期消费方调用 `ensureMaterialized`，以无事件的单个 header 帧发布它。已 flush 事件绝不重写；后续每个批次追加行或一个压缩帧，捕获到写入或同步失败时把文件回滚到之前的字节长度。崩溃后，`load` 保留被中断的最终轮次：保留不完整最后帧中完整解码的记录，从该帧开头截断，并按共享持久化约定的要求，用合成工具、步骤与轮次 closer 重新编码这些记录。只有从未完整写入的撕裂尾部被丢弃；已提交前缀中的校验和、解压或结构失败以损坏拒绝。 +Session 延迟物化：`create(meta)` 不写入任何内容，第一次 `append` 会在当前版本的规范文件名下以不覆盖方式写入并 `fsync` 编码后的 header 与第一批。因此，已创建但从未 append 的 Session 不留下任何磁盘内容，除非生命周期消费方调用 `ensureMaterialized`，发布一个无事件 header frame。已 flush 的当前 generation 事件追加为文本行或压缩 frame；捕获到写入或同步失败时，该文件回滚到之前的字节长度，但其路径或 inode 不会被替换。读取受支持历史 generation 的正文时，后端读取一个稳定精确源，解码其可恢复前缀，在内存中组合全部所需迁移边与当前中断轮次修复，只把最终目标写入同目录临时文件并完成校验，重新检查源 fingerprint，再以不覆盖方式发布此前不存在的目标并同步 namespace。POSIX 把临时 inode 链接到目标；Windows 以 write-through 且不替换的方式移动临时文件。如果另一个 writer 已经发布目标，后端只在该目标是普通当前格式文件且字节与预期完全相同时接受它。源路径、字节与 inode 保持不变，中间版本不进入磁盘；构造 Session 前会通过当前 reader 重新打开已提交目标。 ### 读取日志 -`inspect(id)` 返回带精确继承切点的不可变平衡视图，不提交恢复。`readFrom(id, fromOffset)` 接受 `SessionLogOffset`，返回该偏移及之后的已存储事件，并在后缀旁保留同一切点；JSONL 这类顺序介质解析整个产物并向前跳过。仅 header 的列表读取不读事件正文即可公开 `isSeeded`。选择 `compression: 'none'` 后，日志是外部读取方可直接消费的换行分隔文本；压缩默认值必须经后端读取。 +`inspect(id)` 返回带精确继承 cut 的不可变平衡视图；对于已经是当前 generation 的产物，它不提交崩溃恢复。`readFrom(id, fromOffset)` 接受 `SessionLogOffset`，返回该偏移及之后的已存储事件，并在后缀旁保留同一 cut；JSONL 这类顺序介质解析整个选定 generation 并向前跳过。第一次冷正文打开会扫描 Session 目录，选择数值最高的规范文件名；版本高于当前 build 时拒绝，版本较旧且受支持时发布最终当前后继。后端会在单 writer 假设下缓存已校验的当前选择，供同一实例后续打开使用；`list` 与 `listSnapshots` 始终重新扫描并报告目录的最高 generation。`readRaw` 返回该选定 generation，并在 `filename` 中保留其逻辑 basename（v0 为 `session.jsonl`，v1+ 为 `session.vN.jsonl`；由于 `content` 已解码，因此省略 `.zstd`）。保留的低版本 generation 绝不是自动 fallback、restore 或 downgrade 输入。选择 `compression: 'none'` 后，每个 generation 都是外部读取方可直接消费的换行分隔文本；压缩 generation 需要 Zstandard 解码。 ----- @@ -86,11 +88,11 @@ kind: "package-reference" ### 设计理念 -该后端是共享 [PersistenceCoordinator](../session-persistence/README.zh.md#understand-the-implementation) 之上的一层薄存储：它加载已存储记录、追加批次、提交修复，并把生命周期编排委托给协调器。其物理身份是文件修订值：device、inode、size 与纳秒时间戳标识一份日志，并在追加或修复后改变，这正是 `listSnapshots` 与保留准备结果校验所使用的身份。 +该后端是共享 [PersistenceCoordinator](../session-persistence/README.zh.md#understand-the-implementation) 之上的一层薄存储：它加载已存储记录、追加批次、提交修复，并把生命周期编排委托给协调器。其融合正文读取钩子会让同一个稳定物理快照贯穿 generation 分类或迁移与当前格式解码，因此选定文件不会读取两次。校验当前 generation 后，后端会缓存其路径，供同一进程的后续正文打开使用；仅 header 的列表有意绕过该 cache。物理身份仍是文件修订值：device、inode、size 与纳秒时间戳标识一个 generation，并在 append 或 repair 后改变；`listSnapshots` 与保留准备结果的校验会使用它。 ### 物理编码 -默认产物是独立 [Zstandard 帧](../../../.agents/notes/implemented/architecture/2026-07-19-zstandard-jsonl-session-logs.zh.md) 的标准拼接：一个仅包含 header 行的带校验和帧，后跟每个持久 append 批次一个带校验和帧，使用 Node 内置 Zstandard API 的默认压缩级别（无级别开关）。`sourceEventSeqs` 使用无损存储形式：至少包含三个序列号的连续段会变成 `[start, end]` 区间对，其他列表原样保留；读取时会展开回精确的内存数组。列表只读取并验证 header 帧。`compression: 'none'` 保留相同的存储形式逻辑行，但不使用帧压缩。一个根只属于一种编码：启动发现与定向查找会拒绝相反后缀，且不提供格式或压缩迁移、混合根回退或双写。启用 `packChunks` 时，符合条件的 ≥3 个连续同 block `assistant/chunk` delta 事件连续段会变成一行打包行（`text-chunks`/`reasoning-chunks`/`tool-call-chunks`），其 `seq0`/`time0` 与各成员的 `dt` 间隔精确重建每个成员；无损 codec 位于 `dsh-session`，读取与布局无关，因此打包、非打包与混合文件加载结果一致。 +默认产物是独立 [Zstandard 帧](../../../.agents/notes/implemented/architecture/2026-07-19-zstandard-jsonl-session-logs.zh.md) 的标准拼接：一个仅包含 header 行的带校验和帧，后跟每个持久 append 批次一个带校验和帧，使用 Node 内置 Zstandard API 的默认压缩级别（无级别开关）。`sourceEventSeqs` 使用无损存储形式：至少包含三个序列号的连续段会变成 `[start, end]` 区间对，其他列表原样保留；读取时会展开回精确的内存数组。列表只读取并验证 header 帧。`compression: 'none'` 保留相同的存储形式逻辑行，但不使用帧压缩。配置的后缀只选择一次帧格式；代迁移对已解码 JSON 值工作，原始文本文件与压缩文件使用同一套发布算法。一个根只属于一种编码：启动发现与定向查找会拒绝相反后缀，且不提供压缩迁移、混合根回退或双写。启用 `packChunks` 时，符合条件的分片连续段使用格式 codec 的无损打包表示。 ### 源码地图 @@ -98,8 +100,9 @@ kind: "package-reference" |---|---| | [`src/index.ts`](src/index.ts) | 插件入口：`Config` schema、后端类、协调器接线 | | [`src/format.ts`](src/format.ts) | 日志路径派生、header 编码、记录扫描、打包行布局 | +| [`src/generation.ts`](src/generation.ts) | 精确源读取、最终目标暂存、源重新检查与后继排他发布 | | [`src/zstd.ts`](src/zstd.ts) | Zstandard 帧压缩、解码与帧扫描 | -| [`src/win32.ts`](src/win32.ts) | Windows write-through 发布与目录创建 | +| [`src/win32.ts`](src/win32.ts) | Windows write-through 且不覆盖的文件与目录发布 | | — | 不发布运行时不变式伴生入口；身份在存储层强制。 | @@ -113,6 +116,7 @@ kind: "package-reference" - [会话持久化子系统](../../../docs/subsystems/persistence.zh.md)——后端无关的服务语义与提供方关系。 - [会话持久化 seam](../session-persistence/README.zh.md)——本后端实现的服务约定。 +- [已发布 Session 迁移](../../../.agents/notes/implemented/architecture/2026-08-31-released-session-format-migrations.zh.md)——相邻链、不可变命名与排他发布保证。 - [项目会话目录决策](../../../.agents/notes/implemented/architecture/2026-07-24-project-session-directories.zh.md)——项目与会话目录布局背后的取舍。 - [Zstandard JSONL 会话日志](../../../.agents/notes/implemented/architecture/2026-07-19-zstandard-jsonl-session-logs.zh.md)——带校验和帧编码的理由。 @@ -142,12 +146,13 @@ JSONL 存储不修改实时请求前缀。只有重建历史、当前 envelope 这些限制说明本后端何时不合适，或何时需要特别的运维注意。它们是当前包约束，不是任务积压。 -- **只加载已配置编码和当前 `SESSION_FORMAT_VERSION`（v0）**——更改压缩需要独立或全新根，或选择原始文本模式；预发布格式没有迁移。 +- **只加载已配置编码**——受支持的旧 Session 格式会在该编码内迁移；更改压缩仍需要独立或全新根。 - **平铺文件存储布局不加载**——加载前使用独立根，或将预发布产物移入项目/会话目录布局。 - **压缩文件不能直接按行读取**——使用后端加载；或在写入新根前选择 `compression: 'none'`，供外部行读取方使用。 -- **不删除会话文件**——日志在 `root` 下累积，直到外部移除；seam 无删除接口。 -- **每会话一个活动写入方**——append 与修复只在所属后端实例内协调；在该所有者达到完全停稳的 dispose 前，另一实例或进程不得写入同一会话。 -- **POSIX 实体化需要硬链接支持**——第一次 append 使用 `link()`，使同 id 竞态失败而不覆盖已提交日志；Windows 使用无替换 write-through rename。 +- **不删除 Session generation**——每个规范 generation 都会在 `root` 下累积，直到被外部移除；seam 没有删除 API，也绝不会从数值最高的文件名 fallback 到旧文件。 +- **保留不等于 downgrade 支持**——前任文件保留精确证据，并允许 operator 显式复制或检查，但本 build 不自动恢复它们，也不承诺旧 binary 能在后继文件存在后安全重开该目录。 +- **每会话一个活动写入方**——append、修复与迁移只在所属后端实例内协调；跨进程写入方隔离需要未来的每会话锁。 +- **发布要求不覆盖文件系统原语**——POSIX 的首次物化与后继发布使用 `link()` 加父目录 `fsync`；Windows 使用拒绝既有目标的 write-through move。临时 stage 可以移动或 unlink，但已提交 generation 路径不会。 ### 开发备注 diff --git a/packages/session/session-persistence-jsonl/package.json b/packages/session/session-persistence-jsonl/package.json index ae8967fdf1..c6e01b3431 100644 --- a/packages/session/session-persistence-jsonl/package.json +++ b/packages/session/session-persistence-jsonl/package.json @@ -32,6 +32,7 @@ "@deepseek-ai/cordis": "workspace:^" }, "dependencies": { + "@deepseek-ai/dsh-session-format-catalog": "workspace:^", "koffi": "^3.1.0", "@deepseek-ai/schemastery": "workspace:^" }, diff --git a/packages/session/session-persistence-jsonl/src/format.ts b/packages/session/session-persistence-jsonl/src/format.ts index 29431963ec..565b21406d 100644 --- a/packages/session/session-persistence-jsonl/src/format.ts +++ b/packages/session/session-persistence-jsonl/src/format.ts @@ -8,7 +8,7 @@ * @module dsh-session-persistence-jsonl/format */ -import { join } from 'node:path' +import { isAbsolute, join } from 'node:path' import { decodeSeqRanges, decodeStorageRecord, encodeSeqRanges, packChunkRuns, SESSION_FORMAT_VERSION, SessionLogOffset, @@ -39,7 +39,43 @@ export function logSuffix(compression: JsonlCompression): '.jsonl.zstd' | '.json } /** - * The private version-0 physical header stored as the first JSONL record. + * Return the canonical filename for one immutable Session format generation. + * Version zero retains the original suffix-only name; every later generation + * carries a lowercase numeric `vN` component. + * @param version - non-negative safe Session format version. + * @param compression - configured JSONL artifact encoding. + * @returns the generation filename inside one Session directory. + */ +export function generationLogFilename(version: number, compression: JsonlCompression): string { + if (!Number.isSafeInteger(version) || version < 0 || Object.is(version, -0)) { + throw new TypeError('session log generation version must be a non-negative safe integer') + } + const generation = version === 0 ? '' : `.v${version}` + return `session${generation}${logSuffix(compression)}` +} + +/** + * Parse one canonical generation filename for the selected physical encoding. + * Noncanonical, temporary, uppercase, leading-zero, and version-zero-tagged names do + * not identify committed generations. + * @param filename - one entry from a Session directory. + * @param compression - configured JSONL artifact encoding. + * @returns its format version, or `undefined` when the name is not canonical. + */ +export function parseGenerationLogFilename( + filename: string, + compression: JsonlCompression, +): number | undefined { + const suffix = logSuffix(compression) + if (filename === `session${suffix}`) return 0 + const match = new RegExp(`^session\\.v([1-9][0-9]*)${suffix.replaceAll('.', '\\.')}$`).exec(filename) + if (match === null) return undefined + const version = Number(match[1]) + return Number.isSafeInteger(version) ? version : undefined +} + +/** + * The current shared-layout physical header stored as the first JSONL record. * Its optional numeric `seedLength` translates to logical lineage metadata * plus a separately carried exact inherited cut. */ @@ -56,6 +92,17 @@ interface HeaderLine { agentPreset?: string } +const HEADER_REQUIRED_KEYS = ['type', 'version', 'id', 'createdAt', 'delegationDepth'] as const +const HEADER_OPTIONAL_KEYS = ['cwd', 'parentSession', 'seedLength', 'origin', 'agentPreset'] as const +const HEADER_KEYS = new Set([...HEADER_REQUIRED_KEYS, ...HEADER_OPTIONAL_KEYS]) + +function assertNoRetiredHeaderFields(value: unknown): void { + if (typeof value !== 'object' || value === null) return + if (Object.hasOwn(value, 'sandboxMode') || Object.hasOwn(value, 'approvalPolicy')) { + throw new Error('session header uses retired policy baseline fields') + } +} + /** * Build the header line object from a {@link SessionHeader}. * @param header - the immutable session metadata to serialize. @@ -89,17 +136,14 @@ export function toHeaderLine( } /** - * Translate one version-0 physical header into logical metadata and its cut. + * Translate one current physical header into logical metadata and its cut. * @param line - the shape-checked first line of a log (see the `isHeaderLine` guard). * @returns logical Session metadata paired with the exact inherited prefix length. */ function fromHeaderLine(line: HeaderLine): SessionStorageMetadata { - if (Object.hasOwn(line, 'sandboxMode') || Object.hasOwn(line, 'approvalPolicy')) { - throw new Error('session header uses retired policy baseline fields') - } return { meta: { - version: line.version, + version: SESSION_FORMAT_VERSION, id: line.id, createdAt: line.createdAt, ...line.cwd !== undefined ? { cwd: line.cwd } : {}, @@ -116,7 +160,9 @@ function fromHeaderLine(line: HeaderLine): SessionStorageMetadata { /** Type guard: a parsed first line is a well-formed session header. */ function isHeaderLine(value: unknown): value is HeaderLine { return ( - typeof value === 'object' && value !== null + typeof value === 'object' && value !== null && !Array.isArray(value) + && HEADER_REQUIRED_KEYS.every(key => Object.hasOwn(value, key)) + && Object.keys(value).every(key => HEADER_KEYS.has(key)) && (value as { type?: unknown }).type === 'session' && typeof (value as { version?: unknown }).version === 'number' && typeof (value as { id?: unknown }).id === 'string' @@ -128,6 +174,11 @@ function isHeaderLine(value: unknown): value is HeaderLine { && Number.isSafeInteger((value as { delegationDepth: number }).delegationDepth) && (value as { delegationDepth: number }).delegationDepth >= 0 && !Object.is((value as { delegationDepth: number }).delegationDepth, -0) + && ((value as { cwd?: unknown }).cwd === undefined + || (typeof (value as { cwd?: unknown }).cwd === 'string' + && isAbsolute((value as { cwd: string }).cwd))) + && ((value as { parentSession?: unknown }).parentSession === undefined + || typeof (value as { parentSession?: unknown }).parentSession === 'string') && ((value as { seedLength?: unknown }).seedLength === undefined || (typeof (value as { seedLength?: unknown }).seedLength === 'number' && Number.isSafeInteger((value as { seedLength: number }).seedLength) @@ -224,12 +275,31 @@ export function sessionDir(root: string, cwd: string | undefined, id: SessionId) } /** - * The append-only event-log file path for a session. + * Build one immutable Session format generation path. + * @param root - the backend's session root directory. + * @param cwd - the session's project directory (`undefined` → `_no-cwd`). + * @param id - the session id, path-encoded via {@link encodeSegment} before filesystem use. + * @param version - physical Session format generation. + * @param compression - physical artifact encoding and filename suffix. + * @returns the selected generation's configured JSONL artifact path. + */ +export function generationLogPath( + root: string, + cwd: string | undefined, + id: SessionId, + version: number, + compression: JsonlCompression, +): string { + return join(sessionDir(root, cwd, id), generationLogFilename(version, compression)) +} + +/** + * Build the current generation's append target path for a Session. * @param root - the backend's session root directory. * @param cwd - the session's project directory (`undefined` → `_no-cwd`). * @param id - the session id, path-encoded via {@link encodeSegment} before filesystem use. * @param compression - physical artifact encoding and filename suffix. - * @returns the session's configured JSONL artifact path. + * @returns the current Session format generation path. */ export function logPath( root: string, @@ -237,7 +307,7 @@ export function logPath( id: SessionId, compression: JsonlCompression, ): string { - return join(sessionDir(root, cwd, id), `session${logSuffix(compression)}`) + return generationLogPath(root, cwd, id, SESSION_FORMAT_VERSION, compression) } /** @@ -294,7 +364,6 @@ interface SessionLogScan { committedBytes: number } -/** Parse one complete header record supplied independently from event rows. */ /** * Refuse a header carrying a format version this build does not read BEFORE * validating the current header shape or decoding any event row: a future @@ -311,6 +380,7 @@ function refuseForeignFormatVersion(parsed: unknown): void { ) } +/** Parse one complete header record supplied independently from event rows. */ function parseHeaderRecord(record: Buffer): ReturnType { if (record.length === 0 || record.at(-1) !== 0x0A || record.indexOf(0x0A) !== record.length - 1) { throw new Error('empty or header-less session log') @@ -322,6 +392,7 @@ function parseHeaderRecord(record: Buffer): ReturnType { throw new Error('corrupt session log: header line is not valid JSON') } refuseForeignFormatVersion(parsed) + assertNoRetiredHeaderFields(parsed) if (!isHeaderLine(parsed)) { throw new Error('corrupt session log: first line is not a session header') } @@ -349,9 +420,10 @@ export class SessionLogScanner { /** * Create an event scanner from exactly one newline-terminated header record. * @param headerRecord - the complete first JSONL record, including its newline. + * @param storage - already-decoded current metadata from the same header bytes. */ - constructor(headerRecord: Buffer) { - const parsed = parseHeaderRecord(headerRecord) + constructor(headerRecord: Buffer, storage?: SessionStorageMetadata) { + const parsed = storage ?? parseHeaderRecord(headerRecord) this.meta = parsed.meta this.inheritedEventCount = parsed.inheritedEventCount this.inputBytes = headerRecord.length @@ -483,16 +555,17 @@ export function parseHeader(firstLine: string): SessionStorageMetadata | undefin } catch { return undefined } - refuseForeignFormatVersion(parsed) - if (!isHeaderLine(parsed)) return undefined - return fromHeaderLine(parsed) + return parseHeaderValue(parsed) } /** - * Parse only the logical header fields needed by lightweight listing. - * @param firstLine - first JSONL line without its trailing newline. - * @returns the logical Session header, or `undefined` for a malformed line. + * Translate one already-parsed current physical header without parsing its JSON twice. + * @param value - parsed first JSONL record. + * @returns current storage metadata, or `undefined` for a malformed header. */ -export function parseHeaderMeta(firstLine: string): SessionHeader | undefined { - return parseHeader(firstLine)?.meta +export function parseHeaderValue(value: unknown): SessionStorageMetadata | undefined { + refuseForeignFormatVersion(value) + assertNoRetiredHeaderFields(value) + if (!isHeaderLine(value)) return undefined + return fromHeaderLine(value) } diff --git a/packages/session/session-persistence-jsonl/src/generation.ts b/packages/session/session-persistence-jsonl/src/generation.ts new file mode 100644 index 0000000000..ca8b01f76e --- /dev/null +++ b/packages/session/session-persistence-jsonl/src/generation.ts @@ -0,0 +1,802 @@ +/** + * Durable whole-generation publication for JSONL Session artifacts. + * + * Format packages transform parsed JSON values. This module owns the physical + * encoding, exact source identity, immutable generation files, and exclusive + * current-generation publication for both configured JSONL suffixes. + * @module @deepseek-ai/dsh-session-persistence-jsonl/generation + */ + +import { createHash, randomBytes } from 'node:crypto' +import { + link as fsLink, + lstat as fsLstat, + open as fsOpen, + readFile as fsReadFile, + readdir as fsReaddir, + rm as fsRm, + stat as fsStat, + type FileHandle, +} from 'node:fs/promises' +import { basename, dirname, join } from 'node:path' +import type { JsonlCompression } from './format.ts' +import { logSuffix } from './format.ts' +import { publishNewFileWin32 } from './win32.ts' +import { + compressZstdFrame, + createZstdFrameDecoder, + decompressZstdFrame, + decompressZstdPrefix, + scanZstdFrames, + type ZstdFrameScan, +} from './zstd.ts' + +/** Parsed JSONL values supplied to the format catalog. */ +export interface JsonlDecodedGeneration { + readonly header: Record + readonly rows: readonly unknown[] +} + +/** Current JSONL values returned by the format catalog for physical encoding. */ +export interface JsonlCurrentGeneration extends JsonlDecodedGeneration {} + +/** Pure adapter between backend-owned JSONL framing and the format catalog. */ +export interface JsonlGenerationFormatAdapter { + readonly currentVersion: number + /** Convert one detached historical generation to exact current JSON values. */ + migrate(source: JsonlDecodedGeneration): JsonlCurrentGeneration + /** Validate one decoded current generation, including after committed reopen. */ + validateCurrent(candidate: JsonlCurrentGeneration): void + /** Classify a supported-version artifact that policy refuses to migrate. */ + isUnsupportedMigrationError?(error: unknown): error is Error +} + +/** Inputs for ensuring one already-resolved generation has a current successor. */ +export interface EnsureJsonlGenerationOptions { + /** Immutable generation selected by the backend resolver. */ + readonly sourcePath: string + /** Version selected from the source filename and independently checked against its header. */ + readonly sourceVersion: number + /** Canonical filename for `format.currentVersion` in the same Session directory. */ + readonly currentPath: string + readonly compression: JsonlCompression + readonly format: JsonlGenerationFormatAdapter + /** Validate one selected historical header's identity before any migration write. */ + readonly validateHistoricalHeader?: ( + header: Readonly>, + ) => void | Promise + readonly signal?: AbortSignal +} + +/** + * Result of current classification or exclusive publication. A present + * `snapshot.zstdBody` owns a live decoder: the fused consumer must exhaust it + * or call its disposer on every exit. + */ +export type EnsureJsonlGenerationResult = + | { + readonly status: 'current' + readonly version: number + readonly path: string + readonly snapshot: JsonlPhysicalSnapshot + } + | { + readonly status: 'migrated' + readonly fromVersion: number + readonly toVersion: number + readonly path: string + readonly sourcePath: string + readonly snapshot: JsonlPhysicalSnapshot + } + +/** A future physical header was readable, but this writer cannot interpret it. */ +export class JsonlGenerationNewerVersionError extends Error { + override readonly name = 'JsonlGenerationNewerVersionError' + + /** + * @param storedVersion - version read from the highest stored generation. + * @param currentVersion - version this build writes. + * @param storedId - minimally decoded identity used in the refusal diagnostic. + */ + constructor( + readonly storedVersion: number, + readonly currentVersion: number, + readonly storedId: string, + ) { + super(`session log format v${storedVersion} is newer than current v${currentVersion}`) + } +} + +/** A historical artifact is intact, but the format edge refuses its contents. */ +export class JsonlGenerationUnsupportedMigrationError extends Error { + override readonly name = 'JsonlGenerationUnsupportedMigrationError' + + /** + * @param fromVersion - unchanged source generation version. + * @param reason - format-edge refusal. + */ + constructor( + readonly fromVersion: number, + readonly reason: Error, + ) { + super(reason.message, { cause: reason }) + } +} + +/** A current-generation filename already names different or invalid bytes. */ +export class JsonlGenerationTargetConflictError extends Error { + override readonly name = 'JsonlGenerationTargetConflictError' + + /** + * @param path - immutable target that prevented exclusive publication. + * @param reason - why the existing target cannot be accepted. + */ + constructor( + readonly path: string, + readonly reason: Error, + ) { + super(`current session generation already exists at "${path}": ${reason.message}`, { cause: reason }) + } +} + +/** Stat identity captured together with exact generation bytes. */ +export interface JsonlPhysicalIdentity { + readonly dev: bigint + readonly ino: bigint + readonly size: bigint + readonly mtimeNs: bigint + readonly ctimeNs: bigint +} + +/** One revision-stable physical artifact read reusable by the immediate backend hook. */ +export interface JsonlPhysicalSnapshot { + readonly bytes: Buffer + readonly identity: JsonlPhysicalIdentity + readonly headerValue: Record + readonly headerRecord: Buffer + /** Single-use decoder owner; the fused consumer must dispose it on every exit. */ + readonly zstdBody?: JsonlZstdBodyFrames +} + +/** Live body-frame iterator plus the structural scan and mandatory decoder disposal. */ +export interface JsonlZstdBodyFrames extends Disposable { + readonly frames: Generator + readonly scan: ZstdFrameScan +} + +interface StablePhysicalFile { + readonly bytes: Buffer + readonly identity: JsonlPhysicalIdentity +} + +interface JsonlPhysicalHeader { + readonly value: Record + readonly record: Buffer + readonly zstdBody?: JsonlZstdBodyFrames +} + +class OwnedZstdBodyFrames implements JsonlZstdBodyFrames { + constructor( + readonly frames: Generator, + readonly scan: ZstdFrameScan, + ) {} + + [Symbol.dispose](): void { + this.frames.return() + } +} + +interface DecodedPhysicalJsonl { + readonly bytes: Buffer + readonly torn: boolean +} + +interface GenerationFileSystem { + open(path: string, flags: string, mode?: number): Promise + readFile(path: string, signal?: AbortSignal): Promise + readdir(path: string): Promise + stat(path: string): Promise + lstat(path: string): Promise<{ isFile(): boolean; isSymbolicLink(): boolean }> + link(existingPath: string, newPath: string): Promise + rm(path: string): Promise +} + +type GenerationBarrierPhase = + | 'before-source-check' + | 'after-publication' + +interface JsonlGenerationInternals { + readonly fs: GenerationFileSystem + readonly randomToken: () => string + readonly platform: NodeJS.Platform + readonly publishNewWin32: typeof publishNewFileWin32 + readonly barrier: (phase: GenerationBarrierPhase, attempt: number) => void | Promise +} + +type JsonlGenerationTestOverrides = Partial> & { + readonly fs?: Partial +} + +const defaultFileSystem: GenerationFileSystem = { + open: (path, flags, mode) => fsOpen(path, flags, mode), + readFile: (path, signal) => fsReadFile(path, signal === undefined ? undefined : { signal }), + readdir: path => fsReaddir(path), + stat: path => fsStat(path, { bigint: true }), + lstat: path => fsLstat(path), + link: fsLink, + rm: path => fsRm(path, { force: true }), +} + +const defaultInternals: JsonlGenerationInternals = { + fs: defaultFileSystem, + randomToken: () => randomBytes(8).toString('hex'), + platform: process.platform, + publishNewWin32: publishNewFileWin32, + barrier: () => {}, +} + +function isEEXIST(error: unknown): boolean { + return (error as NodeJS.ErrnoException | null)?.code === 'EEXIST' +} + +function identity(value: JsonlPhysicalIdentity): string { + return [value.dev, value.ino, value.size, value.mtimeNs, value.ctimeNs].join(':') +} + +function fingerprint(value: JsonlPhysicalIdentity, bytes: Buffer): string { + return `${identity(value)}:${createHash('sha256').update(bytes).digest('hex')}` +} + +/** Read exact physical bytes only when the stat identity brackets one stable read. */ +async function readStableSnapshot( + path: string, + signal: AbortSignal | undefined, + fs: GenerationFileSystem, +): Promise { + for (;;) { + signal?.throwIfAborted() + const before = await fs.stat(path) + const bytes = await fs.readFile(path, signal) + signal?.throwIfAborted() + const after = await fs.stat(path) + if (identity(before) === identity(after)) { + signal?.throwIfAborted() + return { bytes, identity: after } + } + } +} + +/** Parse the version discriminator without validating any version-specific field. */ +function storedVersion(header: unknown): number { + if (typeof header !== 'object' || header === null || Array.isArray(header)) { + throw new Error('corrupt session log: first line is not a JSON object') + } + const version = (header as { version?: unknown }).version + if (!Number.isSafeInteger(version) || (version as number) < 0 || Object.is(version, -0)) { + throw new Error('corrupt session log: header version is not a non-negative safe integer') + } + return version as number +} + +function storedId(header: unknown): string { + return String((header as { id?: unknown }).id) +} + +function parseJson(text: string, subject: string): unknown { + try { + return JSON.parse(text) + } catch (error) { + throw new Error(`corrupt session log: ${subject} is not valid JSON`, { cause: error }) + } +} + +function parseGeneration(bytes: Buffer, recoverSuffix = false): JsonlDecodedGeneration { + /* v8 ignore next -- decodePhysicalJsonl supplies a non-empty newline-terminated prefix. */ + if (bytes.length === 0 || bytes.at(-1) !== 0x0A) { + throw new Error('empty or header-less session log') + } + const records = bytes.toString('utf8').slice(0, -1).split('\n') + const parsedHeader = parseJson(records[0] as string, 'header line') + storedVersion(parsedHeader) + const rows: unknown[] = [] + let issue: Error | undefined + for (const [index, record] of records.slice(1).entries()) { + let row: unknown + try { + row = parseJson(record, `row ${index + 1}`) + } catch (error) { + if (!recoverSuffix) throw error + issue ??= error as Error + continue + } + if (issue !== undefined) { + if (typeof row === 'object' && row !== null + && (row as { type?: unknown }).type === 'turn/end') throw issue + continue + } + rows.push(row) + } + return { header: parsedHeader as Record, rows } +} + +function stringifyJson(value: unknown, subject: string): string { + let text: unknown + try { + text = JSON.stringify(value) + } catch (error) { + throw new Error(`${subject} is not lossless JSON`, { cause: error }) + } + if (typeof text !== 'string') throw new Error(`${subject} is not lossless JSON`) + return text +} + +function encodeLogicalJsonl(generation: JsonlCurrentGeneration): Buffer { + const records = [ + stringifyJson(generation.header, 'migrated session header'), + ...generation.rows.map((row, index) => stringifyJson(row, `migrated session row ${index + 1}`)), + ] + return Buffer.from(`${records.join('\n')}\n`) +} + +function assertIndependentHeaderFrame(plaintext: Buffer): void { + if (plaintext.length === 0 || plaintext.indexOf(0x0A) !== plaintext.length - 1) { + throw new Error('corrupt Zstandard session log: first frame is not exactly one header line') + } +} + +async function decodeZstdJsonl(bytes: Buffer, signal?: AbortSignal): Promise { + signal?.throwIfAborted() + const { frames, tornStart } = scanZstdFrames(bytes) + /* v8 ignore next -- the independent header probe already established the first frame. */ + if (frames.length === 0) throw new Error('empty or header-less Zstandard session log') + const complete: Buffer[] = [] + for (const [index, frame] of frames.entries()) { + signal?.throwIfAborted() + const plaintext = await decompressZstdFrame(bytes.subarray(frame.start, frame.end)) + if (index === 0) assertIndependentHeaderFrame(plaintext) + complete.push(plaintext) + } + const completeBytes = Buffer.concat(complete) + if (completeBytes.at(-1) !== 0x0A) { + throw new Error('corrupt Zstandard session log: complete frame contains a torn JSONL record') + } + if (tornStart === undefined) return { bytes: completeBytes, torn: false } + + let recovered = Buffer.alloc(0) + try { + recovered = Buffer.from(await decompressZstdPrefix(bytes.subarray(tornStart))) + } catch { + /* v8 ignore next -- an abort racing decoder failure is timing-dependent */ + if (signal?.aborted) signal.throwIfAborted() + // A structurally torn frame may produce no plaintext; prior frames remain valid. + } + signal?.throwIfAborted() + const newline = recovered.lastIndexOf(0x0A) + return { + bytes: newline === -1 + ? completeBytes + : Buffer.concat([completeBytes, recovered.subarray(0, newline + 1)]), + torn: true, + } +} + +async function decodePhysicalJsonl( + bytes: Buffer, + compression: JsonlCompression, + signal?: AbortSignal, +): Promise { + if (compression === 'zstd') return decodeZstdJsonl(bytes, signal) + signal?.throwIfAborted() + const newline = bytes.lastIndexOf(0x0A) + /* v8 ignore next -- physical header classification already found a newline in the same stable bytes. */ + if (newline === -1) throw new Error('empty or header-less session log') + return { bytes: bytes.subarray(0, newline + 1), torn: newline + 1 !== bytes.length } +} + +async function encodePhysicalJsonl( + logical: Buffer, + generation: JsonlCurrentGeneration, + compression: JsonlCompression, +): Promise { + if (compression === 'none') return logical + const header = Buffer.from(`${stringifyJson(generation.header, 'migrated session header')}\n`) + const headerFrame = await compressZstdFrame(header) + if (generation.rows.length === 0) return headerFrame + const body = logical.subarray(header.length) + return Buffer.concat([headerFrame, await compressZstdFrame(body)]) +} + +function readRawHeader(bytes: Buffer): JsonlPhysicalHeader { + const newline = bytes.indexOf(0x0A) + if (newline === -1) throw new Error('empty or header-less session log') + const record = bytes.subarray(0, newline + 1) + const value = parseJson(record.subarray(0, -1).toString('utf8'), 'header line') + storedVersion(value) + return { value: value as Record, record } +} + +function readZstdHeader(bytes: Buffer, signal?: AbortSignal): JsonlPhysicalHeader { + signal?.throwIfAborted() + const scan = scanZstdFrames(bytes) + const first = scan.frames[0] + if (first === undefined) throw new Error('empty or header-less Zstandard session log') + const decoder = createZstdFrameDecoder() + const decodedFrames = decoder.decode(bytes, scan.frames) + try { + const decoded = decodedFrames.next() + /* v8 ignore next -- one complete frame yields once or the decoder throws. */ + if (decoded.done) throw new Error('empty or header-less Zstandard session log') + signal?.throwIfAborted() + assertIndependentHeaderFrame(decoded.value) + const record = Buffer.from(decoded.value) + const value = parseJson(record.subarray(0, -1).toString('utf8'), 'header line') + storedVersion(value) + return { + value: value as Record, + record, + zstdBody: new OwnedZstdBodyFrames(decodedFrames, scan), + } + } catch (error: unknown) { + decodedFrames.return() + decoder.close() + throw error + } +} + +function readPhysicalHeader( + bytes: Buffer, + compression: JsonlCompression, + signal: AbortSignal | undefined, +): JsonlPhysicalHeader { + if (compression === 'zstd') return readZstdHeader(bytes, signal) + return readRawHeader(bytes) +} + +function generationFilename(version: number, suffix: string): string { + return version === 0 ? `session${suffix}` : `session.v${version}${suffix}` +} + +function assertGenerationPaths( + sourcePath: string, + sourceVersion: number, + currentPath: string, + currentVersion: number, + compression: JsonlCompression, +): string { + const suffix = logSuffix(compression) + const expectedSource = generationFilename(sourceVersion, suffix) + const expectedCurrent = generationFilename(currentVersion, suffix) + if (basename(sourcePath) !== expectedSource) { + throw new Error(`resolved JSONL source path must end with "${expectedSource}": ${sourcePath}`) + } + if (basename(currentPath) !== expectedCurrent) { + throw new Error(`current JSONL generation path must end with "${expectedCurrent}": ${currentPath}`) + } + if (dirname(sourcePath) !== dirname(currentPath)) { + throw new Error('source and current JSONL generations must share one Session directory') + } + return suffix +} + +async function syncDirectory(path: string, internals: JsonlGenerationInternals): Promise { + /* v8 ignore next -- Windows namespace operations request write-through directly. */ + if (internals.platform === 'win32') return + const handle = await internals.fs.open(path, 'r') + try { + await handle.sync() + } finally { + await handle.close() + } +} + +async function writeSyncedTemp( + currentPath: string, + suffix: string, + bytes: Buffer, + internals: JsonlGenerationInternals, +): Promise { + for (;;) { + const path = join(dirname(currentPath), `session.migration.${internals.randomToken()}.tmp${suffix}`) + let handle: FileHandle + try { + handle = await internals.fs.open(path, 'wx', 0o600) + } catch (error) { + if (isEEXIST(error)) continue + throw error + } + let failure: unknown + try { + await handle.writeFile(bytes) + await handle.sync() + } catch (error: unknown) { + failure = error + } + try { + await handle.close() + } catch (error: unknown) { + failure = failure === undefined + ? error + : new AggregateError([failure, error], `failed to write and close migration stage "${path}"`) + } + if (failure !== undefined) { + const writeError = failure instanceof Error + ? failure + : new Error('migration stage write failed with a non-Error rejection', { cause: failure }) + try { + await internals.fs.rm(path) + } catch (cleanupError: unknown) { + throw new AggregateError([writeError, cleanupError], `failed to clean migration stage "${path}"`) + } + throw writeError + } + return path + } +} + +/** Remove one temporary file without hiding the operation failure that made it disposable. */ +async function removeTemporary( + path: string, + primaryFailure: unknown, + internals: JsonlGenerationInternals, +): Promise { + try { + await internals.fs.rm(path) + } catch (cleanupFailure: unknown) { + if (primaryFailure === undefined) throw cleanupFailure + throw new AggregateError( + [primaryFailure, cleanupFailure], + `failed to clean migration temporary "${path}" after an earlier failure`, + ) + } +} + +async function validatePhysicalCurrent( + path: string, + compression: JsonlCompression, + format: JsonlGenerationFormatAdapter, + signal: AbortSignal | undefined, + internals: JsonlGenerationInternals, +): Promise { + const snapshot = await readStableSnapshot(path, signal, internals.fs) + const decoded = await decodePhysicalJsonl(snapshot.bytes, compression, signal) + if (decoded.torn) throw new Error('staged current session generation has a torn physical tail') + const generation = parseGeneration(decoded.bytes) + if (storedVersion(generation.header) !== format.currentVersion) { + throw new Error(`staged session generation is not current v${format.currentVersion}`) + } + format.validateCurrent(generation) + const headerEnd = decoded.bytes.indexOf(0x0A) + /* v8 ignore next -- parseGeneration already required the header newline. */ + if (headerEnd === -1) throw new Error('empty or header-less session log') + return { + ...snapshot, + headerValue: generation.header, + headerRecord: Buffer.from(decoded.bytes.subarray(0, headerEnd + 1)), + } +} + +async function publishCurrentExclusive( + staged: string, + currentPath: string, + internals: JsonlGenerationInternals, +): Promise { + if (internals.platform === 'win32') { + try { + await internals.publishNewWin32(staged, currentPath) + return true + } catch (error) { + /* v8 ignore else -- native helper tests own non-collision Win32 failures. */ + if (isEEXIST(error)) return false + /* v8 ignore next -- the filesystem error is already complete. */ + throw error + } + } + try { + await internals.fs.link(staged, currentPath) + } catch (error) { + /* v8 ignore else -- a non-collision filesystem error propagates unchanged. */ + if (isEEXIST(error)) return false + /* v8 ignore next -- the filesystem error is already complete. */ + throw error + } + try { + await syncDirectory(dirname(currentPath), internals) + } catch (publicationFailure: unknown) { + const failures: unknown[] = [publicationFailure] + try { + await internals.fs.rm(currentPath) + } catch (rollbackFailure: unknown) { + failures.push(rollbackFailure) + } + try { + await syncDirectory(dirname(currentPath), internals) + } catch (rollbackFailure: unknown) { + failures.push(rollbackFailure) + } + if (failures.length === 1) throw publicationFailure + throw new AggregateError( + failures, + `failed to roll back unconfirmed JSONL generation publication "${currentPath}"`, + ) + } + return true +} + +function asError(error: unknown): Error { + return error instanceof Error ? error : new Error('current-generation validation failed with a non-Error rejection', { + cause: error, + }) +} + +async function reopenExpectedCurrent( + currentPath: string, + expectedBytes: Buffer, + compression: JsonlCompression, + format: JsonlGenerationFormatAdapter, + signal: AbortSignal | undefined, + checkCanonicalTargetName: boolean, + internals: JsonlGenerationInternals, +): Promise { + try { + if (checkCanonicalTargetName) { + const expectedName = basename(currentPath) + const names = await internals.fs.readdir(dirname(currentPath)) + if (!names.includes(expectedName)) { + const noncanonical = names.find(name => name.toLowerCase() === expectedName.toLowerCase()) + if (noncanonical !== undefined) { + throw new Error(`target resolves to noncanonical directory entry "${noncanonical}"`) + } + } + } + const info = await internals.fs.lstat(currentPath) + if (info.isSymbolicLink() || !info.isFile()) { + const kind = info.isSymbolicLink() ? 'symbolic link' : 'non-regular file' + throw new Error(`target is a ${kind}`) + } + const snapshot = await validatePhysicalCurrent(currentPath, compression, format, signal, internals) + if (!snapshot.bytes.equals(expectedBytes)) throw new Error('target bytes differ from the migrated generation') + return snapshot + } catch (error: unknown) { + if (signal?.aborted) signal.throwIfAborted() + throw new JsonlGenerationTargetConflictError(currentPath, asError(error)) + } +} + +function withOverrides(overrides: JsonlGenerationTestOverrides): JsonlGenerationInternals { + return { + ...defaultInternals, + ...overrides, + fs: { ...defaultFileSystem, ...overrides.fs }, + } +} + +async function ensureCurrent( + options: EnsureJsonlGenerationOptions, + internals: JsonlGenerationInternals, +): Promise { + const { sourcePath, sourceVersion, currentPath, compression, format, signal } = options + const suffix = assertGenerationPaths( + sourcePath, + sourceVersion, + currentPath, + format.currentVersion, + compression, + ) + let attempt = 0 + for (;;) { + attempt += 1 + signal?.throwIfAborted() + const source = await readStableSnapshot(sourcePath, signal, internals.fs) + const quickHeader = readPhysicalHeader(source.bytes, compression, signal) + const quickVersion = storedVersion(quickHeader.value) + if (quickVersion !== sourceVersion) { + quickHeader.zstdBody?.[Symbol.dispose]() + throw new Error( + `resolved JSONL source filename identifies v${sourceVersion}, but its header identifies v${quickVersion}: ` + + sourcePath, + ) + } + if (sourceVersion > format.currentVersion) { + quickHeader.zstdBody?.[Symbol.dispose]() + throw new JsonlGenerationNewerVersionError(sourceVersion, format.currentVersion, storedId(quickHeader.value)) + } + if (sourceVersion === format.currentVersion) { + return { + status: 'current', + version: quickVersion, + path: sourcePath, + snapshot: { + ...source, + headerValue: quickHeader.value, + headerRecord: quickHeader.record, + ...quickHeader.zstdBody === undefined ? {} : { zstdBody: quickHeader.zstdBody }, + }, + } + } + quickHeader.zstdBody?.[Symbol.dispose]() + const validation = options.validateHistoricalHeader?.(quickHeader.value) + if (validation !== undefined) await validation + + const decodedSource = await decodePhysicalJsonl(source.bytes, compression, signal) + const parsedSource = parseGeneration(decodedSource.bytes, true) + const fromVersion = storedVersion(parsedSource.header) + /* v8 ignore next -- both headers come from the same stable physical snapshot. */ + if (fromVersion !== quickVersion) throw new Error('session format changed within one stable physical snapshot') + const sourceFingerprint = fingerprint(source.identity, source.bytes) + + let migrated: JsonlCurrentGeneration + try { + migrated = format.migrate(parsedSource) + } catch (error: unknown) { + if (format.isUnsupportedMigrationError?.(error) === true) { + throw new JsonlGenerationUnsupportedMigrationError(fromVersion, error) + } + throw error + } + if (storedVersion(migrated.header) !== format.currentVersion) { + throw new Error(`format migration returned v${storedVersion(migrated.header)}, expected v${format.currentVersion}`) + } + const logical = encodeLogicalJsonl(migrated) + const physical = await encodePhysicalJsonl(logical, migrated, compression) + let staged = await writeSyncedTemp(currentPath, suffix, physical, internals) + let failure: unknown + try { + await validatePhysicalCurrent(staged, compression, format, signal, internals) + await internals.barrier('before-source-check', attempt) + const beforePublish = await readStableSnapshot(sourcePath, signal, internals.fs) + if (fingerprint(beforePublish.identity, beforePublish.bytes) !== sourceFingerprint) continue + + const published = await publishCurrentExclusive(staged, currentPath, internals) + if (published && internals.platform === 'win32') staged = '' + await internals.barrier('after-publication', attempt) + signal?.throwIfAborted() + const committed = await reopenExpectedCurrent( + currentPath, + physical, + compression, + format, + signal, + !published, + internals, + ) + return { + status: 'migrated', + fromVersion, + toVersion: format.currentVersion, + path: currentPath, + sourcePath, + snapshot: committed, + } + } catch (error: unknown) { + failure = error + throw error + } finally { + if (staged !== '') await removeTemporary(staged, failure, internals) + } + } +} + +/** + * Ensure one resolved generation has a current-format successor. Current input reads one + * coherent physical snapshot, inspects only its independently readable header, + * invokes no body decoder or migration callback, and returns that snapshot for + * the immediate body-reading backend hook. Historical input remains unchanged; + * only a previously absent current filename can be published. + * @param options - resolved source and target, configured encoding, format adapter, and cancellation. + * @returns whether the source was already current or which immutable successor was published. + */ +export function ensureJsonlGenerationCurrent( + options: EnsureJsonlGenerationOptions, +): Promise { + return ensureCurrent(options, defaultInternals) +} + +/** Private deterministic filesystem, platform, and race seams for package tests. */ +export const __jsonlGenerationTest = { + ensure( + options: EnsureJsonlGenerationOptions, + overrides: JsonlGenerationTestOverrides, + ): Promise { + return ensureCurrent(options, withOverrides(overrides)) + }, +} diff --git a/packages/session/session-persistence-jsonl/src/index.ts b/packages/session/session-persistence-jsonl/src/index.ts index 8ccebf0be0..2452523849 100644 --- a/packages/session/session-persistence-jsonl/src/index.ts +++ b/packages/session/session-persistence-jsonl/src/index.ts @@ -8,17 +8,22 @@ import { Context } from '@deepseek-ai/cordis' import z from '@deepseek-ai/schemastery' -import { readdirSync } from 'node:fs' +import { + SessionFormatUnsupportedMigrationError, + sessionFormatCatalog, +} from '@deepseek-ai/dsh-session-format-catalog' +import { readdirSync, type Dirent } from 'node:fs' import { open, mkdir, readFile, readdir, realpath, link, rm, stat, truncate } from 'node:fs/promises' -import { dirname, join, resolve } from 'node:path' +import { basename, dirname, join, resolve } from 'node:path' import { performance } from 'node:perf_hooks' import { scheduler } from 'node:timers/promises' import { randomBytes } from 'node:crypto' import { DEFAULT_PREPARED_SESSION_CACHE_SIZE, DEFAULT_WRITE_BATCH_MAX_DELAY_MS, MAX_WRITE_BATCH_DELAY_MS, SessionPersistence, SessionPersistenceRevision, PersistenceCoordinator, SessionFormatUnsupportedError, + sessionFormatVersionRefusal, type BorrowedSessionSource, - type PersistenceBackend, type SessionLocation, type SessionPersistenceSnapshot, + type PersistenceBackend, type SessionLocation, type SessionPersistenceListing, type SessionPersistenceSnapshot, type SessionEventSuffix, type SessionInspection, type SessionPersistenceRevision as PersistenceRevision, type SessionRawArtifact, type SessionStorageMetadata, @@ -33,14 +38,28 @@ import type { SessionPreparation, } from '@deepseek-ai/dsh-session' import { - encodeSegment, eventLines, logPath, logSuffix, parseHeader, parseHeaderMeta, projectDir, scanLog, sessionDir, + SESSION_FORMAT_VERSION, + SessionId as makeSessionId, + interruptedTurnClosers, +} from '@deepseek-ai/dsh-session' +import { + encodeSegment, eventLines, generationLogFilename, generationLogPath, logPath, logSuffix, + parseGenerationLogFilename, parseHeader, parseHeaderValue, projectDir, scanLog, sessionDir, SessionLogScanner, toHeaderLine, type JsonlCompression, } from './format.ts' import { compressZstdFrame, createZstdFrameDecoder, decompressZstdFrame, decompressZstdPrefix, scanZstdFrames, + type ZstdFrameDecoder, } from './zstd.ts' import { ensureDurableDirectoryWin32, publishNewFileWin32 } from './win32.ts' +import { + ensureJsonlGenerationCurrent, + JsonlGenerationNewerVersionError, + JsonlGenerationUnsupportedMigrationError, + type JsonlGenerationFormatAdapter, + type JsonlZstdBodyFrames, +} from './generation.ts' export type { JsonlCompression } from './format.ts' @@ -106,6 +125,22 @@ interface FileRevisionIdentity { readonly ctimeNs: bigint } +interface CurrentJsonlFile { + readonly path: string + readonly buffer: Buffer + readonly revision: PersistenceRevision + readonly storage: SessionStorageMetadata + readonly headerRecord: Buffer + readonly zstdBody?: JsonlZstdBodyFrames +} + +/** One authoritative immutable generation selected from a Session directory. */ +interface ResolvedJsonlGeneration { + readonly sourcePath: string + readonly sourceVersion: number + readonly currentPath: string +} + /** Build the source-qualified revision shared by full and lightweight reads. */ function fileRevision(identity: FileRevisionIdentity): PersistenceRevision { return SessionPersistenceRevision([ @@ -154,9 +189,18 @@ export class JsonlSessionPersistence extends SessionPersistence implements Persi private compression: JsonlCompression private coordinator: PersistenceCoordinator private rootEncodingCheck: Promise | undefined + /** Current selections validated by this backend instance under the one-writer assumption. */ + private readonly validatedCurrentGenerations = new Map() + private readonly generationFormat: JsonlGenerationFormatAdapter constructor(ctx: Context, public config: Config) { super(ctx) + if (sessionFormatCatalog.currentVersion !== SESSION_FORMAT_VERSION) { + throw new Error( + `session-persistence-jsonl: format catalog v${sessionFormatCatalog.currentVersion} ` + + `does not match Session v${SESSION_FORMAT_VERSION}`, + ) + } // Resolve once so later process.cwd() changes cannot split one backend across roots. this.root = resolve(config.root) // Programmatic wrappers may construct the backend without Schemastery normalization. @@ -166,6 +210,27 @@ export class JsonlSessionPersistence extends SessionPersistence implements Persi ?? DEFAULT_WRITE_BATCH_MAX_DELAY_MS this.packChunks = config.packChunks ?? DEFAULT_PACK_CHUNKS this.compression = config.compression ?? DEFAULT_COMPRESSION + this.generationFormat = { + currentVersion: sessionFormatCatalog.currentVersion, + migrate: (source) => { + const decoded = sessionFormatCatalog.decodeRecoverableArtifact(source.header, source.rows) + const current = sessionFormatCatalog.migrate(decoded) + const closers = interruptedTurnClosers(current.events as unknown as readonly SessionEvent[]) + const repaired = closers.length === 0 + ? current + : { + ...current, + events: [...current.events, ...closers] as unknown as typeof current.events, + } + return sessionFormatCatalog.encodeCurrent(repaired, { packChunks: this.packChunks }) + }, + validateCurrent: (candidate) => { + const decoded = sessionFormatCatalog.decodeArtifact(candidate.header, candidate.rows) + sessionFormatCatalog.migrate(decoded) + }, + isUnsupportedMigrationError: (error): error is SessionFormatUnsupportedMigrationError => + error instanceof SessionFormatUnsupportedMigrationError, + } this.assertUsableRoot() this.coordinator = new PersistenceCoordinator(this.ctx, this, { preparedSessionCacheSize, @@ -217,20 +282,114 @@ export class JsonlSessionPersistence extends SessionPersistence implements Persi return this.coordinator.readFrom(id, fromSeq, signal) } + override readRaw(id: SessionId, signal?: AbortSignal): Promise { + return this.coordinator.readRaw(id, signal) + } + // One method serves both public `list` and the backend hook; delegating it to // the coordinator would call this hook recursively. /* jscpd:ignore-end */ // --- PersistenceBackend hooks (the file-bytes storage primitives) --- + /** Publish a current successor for one supported historical generation. */ + async ensureCurrent(id: SessionId, signal?: AbortSignal): Promise { + const current = await this.readCurrentFile(id, signal) + if (current === undefined) return + try { + await this.assertStoredIdentity( + current.path, + sessionFormatCatalog.currentVersion, + current.storage.meta, + id, + signal, + ) + this.rememberCurrentGeneration(id, current.path) + } finally { + current.zstdBody?.[Symbol.dispose]() + } + } + + /** Ensure and decode one current stored prefix from a single coherent physical read. */ + async loadCurrentStored( + id: SessionId, + signal?: AbortSignal, + ): Promise | undefined> { + const current = await this.readCurrentFile(id, signal) + if (current === undefined) return undefined + try { + const prefix = await this.decodePrefix(current.path, current.buffer, current.revision, id, signal, current) + this.rememberCurrentGeneration(id, current.path) + return prefix + } finally { + current.zstdBody?.[Symbol.dispose]() + } + } + + /** Resolve, migrate when required, and retain the resulting current physical snapshot. */ + private async readCurrentFile(id: SessionId, signal?: AbortSignal): Promise { + signal?.throwIfAborted() + const selected = await this.findLog(id, signal) + if (selected === undefined) return + try { + const result = await ensureJsonlGenerationCurrent({ + sourcePath: selected.sourcePath, + sourceVersion: selected.sourceVersion, + currentPath: selected.currentPath, + compression: this.compression, + format: this.generationFormat, + validateHistoricalHeader: headerValue => this.validateSourceIdentity( + selected.sourcePath, + selected.sourceVersion, + headerValue, + id, + signal, + ), + ...signal === undefined ? {} : { signal }, + }) + try { + const storage = parseHeaderValue(result.snapshot.headerValue) + if (storage === undefined) { + throw new Error(`corrupt session log: invalid current header in "${result.path}"`) + } + return { + path: result.path, + buffer: result.snapshot.bytes, + revision: fileRevision(result.snapshot.identity), + storage, + headerRecord: result.snapshot.headerRecord, + ...result.snapshot.zstdBody === undefined ? {} : { zstdBody: result.snapshot.zstdBody }, + } + } catch (error: unknown) { + result.snapshot.zstdBody?.[Symbol.dispose]() + throw error + } + } catch (error: unknown) { + if (error instanceof JsonlGenerationNewerVersionError) { + const reason = sessionFormatVersionRefusal(error.storedId, error.storedVersion) + throw new SessionFormatUnsupportedError( + `${reason} (raw log: ${selected.sourcePath})`, + { kind: 'jsonl', path: selected.sourcePath }, + ) + } + if (error instanceof JsonlGenerationUnsupportedMigrationError) { + throw new SessionFormatUnsupportedError( + `${error.message}; source v${error.fromVersion} artifact remains unchanged (raw log: ${selected.sourcePath})`, + { kind: 'jsonl', path: selected.sourcePath }, + ) + } + throw error + } + } + /** Read a stored prefix by id across all project directories when cwd is unknown. */ async loadStored(id: SessionId, signal?: AbortSignal): Promise | undefined> { signal?.throwIfAborted() - await this.ensureRootEncoding() - signal?.throwIfAborted() - const path = await this.findLog(id, signal) - if (path === undefined) return undefined - return this.readPrefix(path, id, signal) + const selected = await this.findLog(id, signal) + if (selected === undefined) return undefined + const prefix = await this.readPrefix(selected.sourcePath, id, signal) + this.rememberDecodedCurrentGeneration(id, selected) + return prefix } /** @@ -239,12 +398,10 @@ export class JsonlSessionPersistence extends SessionPersistence implements Persi */ async readStoredRevision(id: SessionId, signal?: AbortSignal): Promise { signal?.throwIfAborted() - await this.ensureRootEncoding() - signal?.throwIfAborted() - const path = await this.findLog(id, signal) - if (path === undefined) return undefined + const selected = await this.findLog(id, signal) + if (selected === undefined) return undefined try { - const identity = await stat(path, { bigint: true }) + const identity = await stat(selected.sourcePath, { bigint: true }) signal?.throwIfAborted() return fileRevision(identity) } catch (error: unknown) { @@ -267,36 +424,90 @@ export class JsonlSessionPersistence extends SessionPersistence implements Persi * @returns the raw artifact text plus the header parsed from its own first * line, or `undefined` when the session has no stored artifact. */ - override async readRaw(id: SessionId, signal?: AbortSignal): Promise { + async readRawStored(id: SessionId, signal?: AbortSignal): Promise { signal?.throwIfAborted() - await this.ensureRootEncoding() - signal?.throwIfAborted() - const path = await this.findLog(id, signal) - if (path === undefined) return undefined - const { buffer } = await this.readStableFile(path, signal) + const selected = await this.findLog(id, signal) + if (selected === undefined) return undefined + const { buffer } = await this.readStableFile(selected.sourcePath, signal) + const artifact = this.decodeRawStored(selected.sourcePath, buffer, id, signal) + await this.assertStoredIdentity( + selected.sourcePath, + selected.sourceVersion, + artifact.meta, + id, + signal, + ) + this.rememberDecodedCurrentGeneration(id, selected) + return artifact + } + + /** Ensure and decode one current raw artifact from a single coherent physical read. */ + async readCurrentRawStored(id: SessionId, signal?: AbortSignal): Promise { + const current = await this.readCurrentFile(id, signal) + if (current === undefined) return undefined + try { + const artifact = this.decodeRawStored(current.path, current.buffer, id, signal, current) + await this.assertStoredIdentity( + current.path, + sessionFormatCatalog.currentVersion, + artifact.meta, + id, + signal, + ) + this.rememberCurrentGeneration(id, current.path) + return artifact + } finally { + current.zstdBody?.[Symbol.dispose]() + } + } + + private decodeRawStored( + path: string, + buffer: Buffer, + id: SessionId, + signal?: AbortSignal, + current?: CurrentJsonlFile, + ): SessionRawArtifact { let content: string if (this.compression === 'zstd') { - const { frames } = scanZstdFrames(buffer) + const { frames } = current?.zstdBody?.scan ?? scanZstdFrames(buffer) if (frames.length === 0) throw new Error('empty or header-less Zstandard session log') - const decoder = createZstdFrameDecoder() - const plaintexts: Buffer[] = [] + const plaintexts: Buffer[] = current === undefined + ? [] + : [Buffer.from(current.headerRecord)] + let decoder: ZstdFrameDecoder | undefined + let decodedFrames: Generator + if (current?.zstdBody !== undefined) { + decodedFrames = current.zstdBody.frames + } else { + decoder = createZstdFrameDecoder() + decodedFrames = decoder.decode(buffer, current === undefined ? frames : frames.slice(1)) + } // The decoder yields views into a reused buffer; copy each frame's // plaintext immediately so a later concat cannot read overwritten memory. - for (const plaintext of decoder.decode(buffer, frames)) { - signal?.throwIfAborted() - plaintexts.push(Buffer.from(plaintext)) + try { + for (const plaintext of decodedFrames) { + signal?.throwIfAborted() + plaintexts.push(Buffer.from(plaintext)) + } + } finally { + decoder?.close() } content = Buffer.concat(plaintexts).toString('utf8') } else { content = buffer.toString('utf8') } - const storage = parseHeader(content.split('\n', 1)[0] as string) + const storage = current?.storage ?? parseHeader(content.split('\n', 1)[0] as string) if (storage === undefined || storage.meta.id !== id) { throw new Error(`corrupt session log: invalid header line in "${path}"`) } - // The logical artifact name is `session.jsonl` regardless of the physical - // encoding suffix (`.jsonl.zstd` marks compression only). - return { ...storage, filename: 'session.jsonl', content } + // Raw transfer retains the immutable generation name while removing only + // the physical compression suffix from a Zstandard artifact. + const storedFilename = basename(path) + const filename = this.compression === 'zstd' + ? storedFilename.slice(0, -'.zstd'.length) + : storedFilename + return { ...storage, filename, content } } /** @@ -331,13 +542,33 @@ export class JsonlSessionPersistence extends SessionPersistence implements Persi signal?: AbortSignal, ): Promise> { const { buffer, revision } = await this.readStableFile(path, signal) + return this.decodePrefix(path, buffer, revision, expectedId, signal) + } + + /** Decode one already-stable physical snapshot without reopening its file. */ + private async decodePrefix( + path: string, + buffer: Buffer, + revision: PersistenceRevision, + expectedId?: SessionId, + signal?: AbortSignal, + prefetched?: CurrentJsonlFile, + ): Promise> { let prefix: Omit, 'revision'> try { if (this.compression === 'zstd') { - prefix = await this.readZstdPrefix(buffer, signal) + prefix = await this.readZstdPrefix(buffer, signal, prefetched) } else { signal?.throwIfAborted() - const { meta, inheritedEventCount, events, committedBytes } = scanLog(buffer) + let scanned: ReturnType + if (prefetched === undefined) { + scanned = scanLog(buffer) + } else { + const scanner = new SessionLogScanner(prefetched.headerRecord, prefetched.storage) + scanner.write(buffer.subarray(prefetched.headerRecord.byteLength)) + scanned = scanner.finish() + } + const { meta, inheritedEventCount, events, committedBytes } = scanned signal?.throwIfAborted() prefix = { meta, @@ -358,7 +589,10 @@ export class JsonlSessionPersistence extends SessionPersistence implements Persi throw error } signal?.throwIfAborted() - await this.assertStoredIdentity(path, prefix.meta, expectedId, signal) + const storedVersion = parseGenerationLogFilename(basename(path), this.compression) + /* v8 ignore next -- discovery and current-path construction only select canonical names. */ + if (storedVersion === undefined) throw new Error(`invalid JSONL generation path "${path}"`) + await this.assertStoredIdentity(path, storedVersion, prefix.meta, expectedId, signal) signal?.throwIfAborted() return { ...prefix, revision } } @@ -367,25 +601,41 @@ export class JsonlSessionPersistence extends SessionPersistence implements Persi private async readZstdPrefix( buffer: Buffer, signal?: AbortSignal, + prefetched?: CurrentJsonlFile, ): Promise, 'revision'>> { signal?.throwIfAborted() - const { frames, tornStart } = scanZstdFrames(buffer) + const { frames, tornStart } = prefetched?.zstdBody?.scan ?? scanZstdFrames(buffer) signal?.throwIfAborted() if (frames.length === 0) throw new Error('empty or header-less Zstandard session log') - const decoder = createZstdFrameDecoder() + let decoder: ZstdFrameDecoder | undefined let yieldDeadline = performance.now() + ZSTD_DECODE_YIELD_INTERVAL_MS try { - const decodedFrames = decoder.decode(buffer, frames) - signal?.throwIfAborted() - const headerFrame = decodedFrames.next() - signal?.throwIfAborted() - /* v8 ignore next -- a non-empty structural frame list makes the decoder yield its first frame or throw. */ - if (headerFrame.done) throw new Error('empty or header-less Zstandard session log') - assertZstdHeaderFrame(headerFrame.value) - const scanner = new SessionLogScanner(headerFrame.value) - - let remainingFrames = frames.length - 1 + let decodedFrames: Generator + let scanner: SessionLogScanner + let remainingFrames: number + if (prefetched === undefined) { + decoder = createZstdFrameDecoder() + decodedFrames = decoder.decode(buffer, frames) + signal?.throwIfAborted() + const headerFrame = decodedFrames.next() + signal?.throwIfAborted() + /* v8 ignore next -- a non-empty structural frame list makes the decoder yield its first frame or throw. */ + if (headerFrame.done) throw new Error('empty or header-less Zstandard session log') + assertZstdHeaderFrame(headerFrame.value) + scanner = new SessionLogScanner(headerFrame.value) + remainingFrames = frames.length - 1 + } else { + assertZstdHeaderFrame(prefetched.headerRecord) + scanner = new SessionLogScanner(prefetched.headerRecord, prefetched.storage) + if (prefetched.zstdBody === undefined) { + decoder = createZstdFrameDecoder() + decodedFrames = decoder.decode(buffer, frames.slice(1)) + } else { + decodedFrames = prefetched.zstdBody.frames + } + remainingFrames = frames.length - 1 + } for (const plaintext of decodedFrames) { signal?.throwIfAborted() scanner.write(plaintext) @@ -438,7 +688,7 @@ export class JsonlSessionPersistence extends SessionPersistence implements Persi if (signal?.aborted) signal.throwIfAborted() throw error } finally { - decoder.close() + decoder?.close() } } @@ -478,9 +728,9 @@ export class JsonlSessionPersistence extends SessionPersistence implements Persi if (tornMarker !== undefined) this.ctx.logger.warn(`${this.name}: session "${meta.id}" recovered from a torn tail; incomplete tail bytes were discarded`) } - /** List valid unique stored sessions' metadata (header line only — no full-log parse). */ - async list(signal?: AbortSignal): Promise { - return (await this.listArtifacts(signal)).map(artifact => artifact.header) + /** List each Session directory's authoritative generation from its header only. */ + async list(signal?: AbortSignal): Promise { + return (await this.listArtifacts(signal)).map(artifact => artifact.listing) } /** List metadata plus a stat-derived identity for each append-only log. */ @@ -491,10 +741,7 @@ export class JsonlSessionPersistence extends SessionPersistence implements Persi try { const identity = await stat(artifact.path, { bigint: true }) signal?.throwIfAborted() - snapshots.push({ - header: artifact.header, - revision: fileRevision(identity), - }) + snapshots.push({ ...artifact.listing, revision: fileRevision(identity) }) } catch (error: unknown) { signal?.throwIfAborted() if (!isENOENT(error)) throw error @@ -504,45 +751,153 @@ export class JsonlSessionPersistence extends SessionPersistence implements Persi return snapshots } - private async listArtifacts(signal?: AbortSignal): Promise> { + private async listArtifacts(signal?: AbortSignal): Promise> { signal?.throwIfAborted() - await this.ensureRootEncoding() - signal?.throwIfAborted() - const artifacts: Array<{ header: SessionHeader; path: string }> = [] - const ids = new Set() + const artifacts: Array<{ listing: SessionPersistenceListing; path: string }> = [] + const readableIds = new Map() for (const project of await this.listProjectDirs(signal)) { signal?.throwIfAborted() for (const dir of await this.listSessionDirs(project, signal)) { signal?.throwIfAborted() - const opposite = join(dir, `session${logSuffix(this.oppositeCompression())}`) - const oppositeExists = await this.exists(opposite) - signal?.throwIfAborted() - if (oppositeExists) throw this.encodingMismatch(opposite) - const path = join(dir, `session${logSuffix(this.compression)}`) - const pathExists = await this.exists(path) - signal?.throwIfAborted() - if (!pathExists) continue + const selected = await this.resolveGenerationInDirectory(dir, signal) + if (selected === undefined) continue + const path = selected.sourcePath // Read only headers so listing scales with session count, not log size. - const first = this.compression === 'zstd' - ? await this.readFirstZstdLine(path, signal) - : await this.readFirstLine(path, signal) - signal?.throwIfAborted() - if (first === undefined) continue // empty/half-written file - const meta = parseHeaderMeta(first) - if (meta === undefined) continue // not a session header - await this.assertStoredIdentity(path, meta, undefined, signal) - signal?.throwIfAborted() - if (ids.has(meta.id)) { - throw new Error(`duplicate JSONL session id "${meta.id}" appears in multiple project directories`) + const location = { kind: 'jsonl' as const, path } + let listing: SessionPersistenceListing + try { + const first = this.compression === 'zstd' + ? await this.readFirstZstdLine(path, signal) + : await this.readFirstLine(path, signal) + signal?.throwIfAborted() + if (first === undefined) { + listing = { + status: 'malformed', + targetVersion: sessionFormatCatalog.currentVersion, + location, + reason: 'session artifact has no complete independently readable header', + } + } else { + let headerValue: unknown + try { + headerValue = JSON.parse(first) + } catch (error) { + throw new Error('session header is not valid JSON', { cause: error }) + } + const result = sessionFormatCatalog.readHeader(headerValue) + if ('storedVersion' in result && result.storedVersion !== selected.sourceVersion) { + throw new Error( + `session generation filename identifies v${selected.sourceVersion}, ` + + `but its header identifies v${result.storedVersion}`, + ) + } + if (result.status === 'current') { + const header = this.currentHeader(result.header) + await this.assertStoredIdentity(path, selected.sourceVersion, header, undefined, signal) + listing = { + status: 'current', + storedVersion: result.storedVersion, + targetVersion: result.targetVersion, + header, + location, + } + } else if (result.status === 'migration-required') { + const header = this.currentHeader(result.header) + await this.assertStoredIdentity(path, selected.sourceVersion, header, undefined, signal) + listing = { + status: 'migration-required', + storedVersion: result.storedVersion, + targetVersion: result.targetVersion, + header, + location, + } + } else if (result.status === 'unsupported') { + listing = { + status: 'unsupported', + storedVersion: result.storedVersion, + targetVersion: result.targetVersion, + location, + reason: result.reason, + } + } else { + const malformed = result as { readonly targetVersion: number; readonly reason: string } + listing = { + status: 'malformed', + targetVersion: malformed.targetVersion, + location, + reason: malformed.reason, + } + } + } + } catch (error: unknown) { + signal?.throwIfAborted() + let reason: string + /* v8 ignore else -- built-in header readers reject with Error instances. */ + if (error instanceof Error) reason = error.message + else reason = String(error) + listing = { + status: 'malformed', + targetVersion: sessionFormatCatalog.currentVersion, + location, + reason, + } + } + const index = artifacts.push({ listing, path }) - 1 + if (listing.status === 'current' || listing.status === 'migration-required') { + const indices = readableIds.get(listing.header.id) ?? [] + indices.push(index) + readableIds.set(listing.header.id, indices) + } + } + } + for (const [id, indices] of readableIds) { + if (indices.length < 2) continue + for (const index of indices) { + const artifact = artifacts[index] as { listing: SessionPersistenceListing; path: string } + artifact.listing = { + status: 'malformed', + targetVersion: sessionFormatCatalog.currentVersion, + location: { kind: 'jsonl', path: artifact.path }, + reason: `duplicate JSONL session id "${id}" appears in multiple project directories`, } - ids.add(meta.id) - artifacts.push({ header: meta, path }) } } signal?.throwIfAborted() return artifacts } + /** Convert format-catalog string identities to current branded Session metadata. */ + private currentHeader(header: { + readonly version: number + readonly id: string + readonly createdAt: number + readonly cwd?: string + readonly parentSession?: string + readonly isSeeded: boolean + readonly origin?: 'subagent' + readonly delegationDepth: number + readonly agentPreset?: string + }): SessionHeader { + /* v8 ignore next -- readable catalog results always carry the configured current version. */ + if (header.version !== sessionFormatCatalog.currentVersion) { + throw new Error(`format catalog returned non-current logical header v${header.version}`) + } + return { + version: SESSION_FORMAT_VERSION, + id: makeSessionId(header.id), + createdAt: header.createdAt, + ...header.cwd === undefined ? {} : { cwd: header.cwd }, + ...header.parentSession === undefined ? {} : { parentSession: makeSessionId(header.parentSession) }, + isSeeded: header.isSeeded, + ...header.origin === undefined ? {} : { origin: header.origin }, + delegationDepth: header.delegationDepth, + ...header.agentPreset === undefined ? {} : { agentPreset: header.agentPreset }, + } + } + // --- materialization / append / repair (file mechanics) --- /** Atomically write the header line + first batch (temp-write, fsync, publish). */ @@ -559,6 +914,7 @@ export class JsonlSessionPersistence extends SessionPersistence implements Persi } else { await this.materializePosix(project, dir, finalPath, meta.id, content) } + this.rememberCurrentGeneration(meta.id, finalPath) } /* v8 ignore start -- Windows uses the Win32 durable-publish path; POSIX coverage exercises this peer. */ @@ -634,7 +990,7 @@ export class JsonlSessionPersistence extends SessionPersistence implements Persi // already guards the create path, so this is unreachable-in-practice TOCTOU // defense.) /* v8 ignore next 3 -- createCore guards collisions before materialize; this is a TOCTOU backstop */ - if (await this.exists(finalPath)) { + if (await this.resolveGenerationInDirectory(dirname(finalPath)) !== undefined) { throw new Error(`refusing to materialize "${id}": a log already exists on disk (load/resume it instead)`) } } @@ -673,7 +1029,7 @@ export class JsonlSessionPersistence extends SessionPersistence implements Persi return this.compression === 'zstd' ? compressZstdFrame(body) : body } - /** fsync a POSIX directory so a just-created/renamed entry is crash-durable. */ + /** fsync a POSIX directory so a just-created or linked entry is crash-durable. */ /* v8 ignore start -- Windows uses write-through namespace operations; POSIX coverage exercises directory fsync. */ private async syncDirPosix(dir: string): Promise { const handle = await open(dir, 'r') @@ -812,22 +1168,94 @@ export class JsonlSessionPersistence extends SessionPersistence implements Persi } } - /** Find the unique physical log for an id across every project directory. */ - private async findLog(id: SessionId, signal?: AbortSignal): Promise { - const matches: string[] = [] + /** + * Scan one Session directory and select its highest canonical immutable + * generation. Callers that require a current body consult the validated + * per-instance cache before reaching this cold path; header listing always + * scans so it reports the directory's authoritative generation. + */ + private async resolveGenerationInDirectory( + dir: string, + signal?: AbortSignal, + ): Promise { + signal?.throwIfAborted() + const currentPath = join( + dir, + generationLogFilename(sessionFormatCatalog.currentVersion, this.compression), + ) + let entries: Dirent[] + try { + entries = await readdir(dir, { withFileTypes: true }) + } catch (error) { + if (isENOENT(error)) return undefined + throw error + } + signal?.throwIfAborted() + + const generations: Array<{ readonly path: string; readonly version: number }> = [] + const opposite: Array<{ readonly path: string; readonly version: number }> = [] + for (const entry of entries) { + const version = parseGenerationLogFilename(entry.name, this.compression) + if (version !== undefined) { + generations.push({ path: join(dir, entry.name), version }) + continue + } + const oppositeVersion = parseGenerationLogFilename(entry.name, this.oppositeCompression()) + if (oppositeVersion !== undefined) { + opposite.push({ path: join(dir, entry.name), version: oppositeVersion }) + } + } + if (opposite.length > 0) { + const incompatible = opposite[0] as { readonly path: string; readonly version: number } + throw this.encodingMismatch(incompatible.path) + } + if (generations.length === 0) return undefined + const latest = generations.sort((a, b) => b.version - a.version)[0] as { + readonly path: string + readonly version: number + } + return { + sourcePath: latest.path, + sourceVersion: latest.version, + currentPath, + } + } + + /** Retain one already-validated current selection for same-process fast opens. */ + private rememberCurrentGeneration(id: SessionId, path: string): void { + this.validatedCurrentGenerations.set(id, { + sourcePath: path, + sourceVersion: sessionFormatCatalog.currentVersion, + currentPath: path, + }) + } + + /** Cache one direct decoder result only when its canonical filename is current. */ + private rememberDecodedCurrentGeneration(id: SessionId, selected: ResolvedJsonlGeneration): void { + if (selected.sourceVersion !== sessionFormatCatalog.currentVersion) { + throw new Error( + `resolved JSONL source filename identifies v${selected.sourceVersion}, ` + + `but its decoded header identifies v${sessionFormatCatalog.currentVersion}: ${selected.sourcePath}`, + ) + } + this.rememberCurrentGeneration(id, selected.sourcePath) + } + + /** Find the unique authoritative generation for an id across every project directory. */ + private async findLog(id: SessionId, signal?: AbortSignal): Promise { + const cached = this.validatedCurrentGenerations.get(id) + if (cached !== undefined) { + signal?.throwIfAborted() + return cached + } + const matches: ResolvedJsonlGeneration[] = [] for (const project of await this.listProjectDirs(signal)) { signal?.throwIfAborted() await this.rejectLegacyFlatArtifact(project, id, signal) signal?.throwIfAborted() const dir = join(project, encodeSegment(id)) - const path = join(dir, `session${logSuffix(this.compression)}`) - const opposite = join(dir, `session${logSuffix(this.oppositeCompression())}`) - const oppositeExists = await this.exists(opposite) - signal?.throwIfAborted() - if (oppositeExists) throw this.encodingMismatch(opposite) - const pathExists = await this.exists(path) - signal?.throwIfAborted() - if (pathExists) matches.push(path) + const selected = await this.resolveGenerationInDirectory(dir, signal) + if (selected !== undefined) matches.push(selected) } if (matches.length > 1) { throw new Error(`duplicate JSONL session id "${id}" appears in multiple project directories`) @@ -847,28 +1275,53 @@ export class JsonlSessionPersistence extends SessionPersistence implements Persi } /** Reject metadata that does not identify the selected physical log. */ - private async assertStoredIdentity( + private assertStoredIdentity( path: string, + storedVersion: number, meta: SessionHeader, expectedId?: SessionId, signal?: AbortSignal, - ): Promise { + ): void | Promise { signal?.throwIfAborted() if (expectedId !== undefined && meta.id !== expectedId) { throw new Error(`corrupt session log "${path}": requested id "${expectedId}" does not match header id "${meta.id}"`) } let expectedPath: string try { - expectedPath = logPath(this.root, meta.cwd, meta.id, this.compression) + expectedPath = generationLogPath(this.root, meta.cwd, meta.id, storedVersion, this.compression) } catch (error) { throw new Error(`corrupt session log "${path}": header id cannot name a storage path`, { cause: error }) } - if (path !== expectedPath && !await this.sameFile(path, expectedPath, signal)) { - throw new Error(`corrupt session log "${path}": header id "${meta.id}" and cwd identify "${expectedPath}"`) + if (path === expectedPath) return + return this.assertStoredAlias(path, expectedPath, meta.id, signal) + } + + /** Require a non-identical path spelling to resolve to the same physical artifact. */ + private async assertStoredAlias( + path: string, + expectedPath: string, + headerId: SessionId, + signal?: AbortSignal, + ): Promise { + if (!await this.sameFile(path, expectedPath, signal)) { + throw new Error(`corrupt session log "${path}": header id "${headerId}" and cwd identify "${expectedPath}"`) } signal?.throwIfAborted() } + /** Validate a supported source header against the already-selected artifact before migration. */ + private validateSourceIdentity( + path: string, + storedVersion: number, + headerValue: Readonly>, + expectedId: SessionId, + signal?: AbortSignal, + ): void | Promise { + const result = sessionFormatCatalog.readHeader(headerValue) + if (result.status !== 'current' && result.status !== 'migration-required') return + return this.assertStoredIdentity(path, storedVersion, this.currentHeader(result.header), expectedId, signal) + } + /** * Whether two path spellings resolve to the same physical file. This admits * case aliases on case-insensitive filesystems without weakening identity @@ -923,8 +1376,8 @@ export class JsonlSessionPersistence extends SessionPersistence implements Persi private async checkRootEncoding(): Promise { for (const project of await this.listProjectDirs()) { for (const dir of await this.listSessionDirs(project)) { - const incompatible = join(dir, `session${logSuffix(this.oppositeCompression())}`) - if (await this.exists(incompatible)) throw this.encodingMismatch(incompatible) + const incompatible = await this.findOppositeGenerationInDirectory(dir) + if (incompatible !== undefined) throw this.encodingMismatch(incompatible) } } } @@ -945,8 +1398,26 @@ export class JsonlSessionPersistence extends SessionPersistence implements Persi } private async rejectOppositeArtifact(cwd: string | undefined, id: SessionId): Promise { - const path = logPath(this.root, cwd, id, this.oppositeCompression()) - if (await this.exists(path)) throw this.encodingMismatch(path) + const path = await this.findOppositeGenerationInDirectory(sessionDir(this.root, cwd, id)) + if (path !== undefined) throw this.encodingMismatch(path) + } + + /** Return the highest canonical generation encoded with the unconfigured suffix. */ + private async findOppositeGenerationInDirectory(dir: string): Promise { + let entries: Dirent[] + try { + entries = await readdir(dir, { withFileTypes: true }) + } catch (error) { + if (isENOENT(error)) return undefined + throw error + } + const generations: Array<{ readonly name: string; readonly version: number }> = [] + for (const entry of entries) { + const version = parseGenerationLogFilename(entry.name, this.oppositeCompression()) + if (version !== undefined) generations.push({ name: entry.name, version }) + } + const latest = generations.sort((a, b) => b.version - a.version)[0] + return latest === undefined ? undefined : join(dir, latest.name) } private oppositeCompression(): JsonlCompression { diff --git a/packages/session/session-persistence-jsonl/tests/generation.spec.ts b/packages/session/session-persistence-jsonl/tests/generation.spec.ts new file mode 100644 index 0000000000..83683fd23b --- /dev/null +++ b/packages/session/session-persistence-jsonl/tests/generation.spec.ts @@ -0,0 +1,1049 @@ +import { afterEach, describe, expect, it, vi } from 'vitest' +import { + link, + mkdir, + mkdtemp, + open, + readFile, + readdir, + rename, + rm, + stat, + symlink, + writeFile, +} from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { basename, join } from 'node:path' +import { + __jsonlGenerationTest, + ensureJsonlGenerationCurrent, + JsonlGenerationNewerVersionError, + JsonlGenerationTargetConflictError, + JsonlGenerationUnsupportedMigrationError, + type EnsureJsonlGenerationOptions, + type JsonlCurrentGeneration, + type JsonlGenerationFormatAdapter, +} from '../src/generation.ts' +import { compressZstdFrame, decompressZstdFrame, scanZstdFrames } from '../src/zstd.ts' +import type { JsonlCompression } from '../src/format.ts' + +const roots: string[] = [] + +async function tempRoot(): Promise { + const root = await mkdtemp(join(tmpdir(), 'dsh-jsonl-generation-')) + roots.push(root) + return root +} + +afterEach(async () => { + vi.restoreAllMocks() + for (const root of roots.splice(0)) await rm(root, { recursive: true, force: true }) +}) + +function line(value: unknown): string { + return `${JSON.stringify(value)}\n` +} + +function fsError(code: string, message = code): NodeJS.ErrnoException { + const error = new Error(message) as NodeJS.ErrnoException + error.code = code + return error +} + +/** Complete a POSIX-branch simulation on Windows, whose NTFS directory handles reject fsync. */ +async function openWithPosixDirectorySync(path: string, flags: string, mode?: number) { + const handle = await open(path, flags, mode) + if (flags === 'r' && (await stat(path)).isDirectory()) { + vi.spyOn(handle, 'sync').mockResolvedValue(undefined) + } + return handle +} + +function posixSimulationFs>( + overrides: T, +): T & { readonly open: typeof openWithPosixDirectorySync } { + return { open: openWithPosixDirectorySync, ...overrides } +} + +function header(version: number, id = 'generation-test'): Record { + return { type: 'session', version, id, createdAt: 1, delegationDepth: 0 } +} + +const event0 = { type: 'turn/start', seq: 0, time: 2, data: { turn: 1 } } +const event1 = { type: 'turn/end', seq: 1, time: 3, data: { turn: 1, reason: { kind: 'completed' } } } + +function adapter(overrides: Partial = {}): JsonlGenerationFormatAdapter { + return { + currentVersion: 1, + migrate: (source): JsonlCurrentGeneration => ({ + header: { ...source.header, version: 1 }, + rows: source.rows, + }), + validateCurrent: (candidate) => { + if (candidate.header.version !== 1) throw new Error('candidate is not v1') + }, + ...overrides, + } +} + +function generationPath(root: string, version: number, compression: JsonlCompression): string { + const suffix = compression === 'zstd' ? '.jsonl.zstd' : '.jsonl' + return join(root, version === 0 ? `session${suffix}` : `session.v${version}${suffix}`) +} + +function options( + root: string, + compression: JsonlCompression = 'none', + format: JsonlGenerationFormatAdapter = adapter(), + sourceVersion = 0, +): EnsureJsonlGenerationOptions { + return { + sourcePath: generationPath(root, sourceVersion, compression), + sourceVersion, + currentPath: generationPath(root, format.currentVersion, compression), + compression, + format, + } +} + +async function encodeZstd(version: number, rows: readonly unknown[]): Promise { + return Buffer.concat([ + await compressZstdFrame(line(header(version))), + ...rows.length === 0 + ? [] + : [await compressZstdFrame(rows.map(row => line(row)).join(''))], + ]) +} + +async function decodeZstdJsonl(path: string): Promise { + const bytes = await readFile(path) + const { frames, tornStart } = scanZstdFrames(bytes) + expect(tornStart).toBeUndefined() + const plaintext: Buffer[] = [] + for (const frame of frames) plaintext.push(await decompressZstdFrame(bytes.subarray(frame.start, frame.end))) + return Buffer.concat(plaintext).toString('utf8') +} + +describe('JSONL immutable generation publication', () => { + it('publishes v1 beside an immutable suffixless v0 source', async () => { + const root = await tempRoot() + const request = { ...options(root), signal: new AbortController().signal } + const source = Buffer.from(line(header(0)) + line(event0)) + await writeFile(request.sourcePath, source) + + const result = await ensureJsonlGenerationCurrent(request) + + expect(result).toMatchObject({ + status: 'migrated', + fromVersion: 0, + toVersion: 1, + path: request.currentPath, + sourcePath: request.sourcePath, + }) + expect(await readFile(request.sourcePath)).toEqual(source) + expect(await readFile(request.currentPath, 'utf8')).toBe(line(header(1)) + line(event0)) + expect((await readdir(root)).sort()).toEqual(['session.jsonl', 'session.v1.jsonl']) + }) + + it('takes the current fast path with one read and no format callback', async () => { + const root = await tempRoot() + const migrate = vi.fn() + const validateCurrent = vi.fn() + const validateHistoricalHeader = vi.fn() + const request = { + ...options(root, 'none', adapter({ migrate, validateCurrent }), 1), + validateHistoricalHeader, + } + const contents = line(header(1)) + line(event0) + await writeFile(request.sourcePath, contents) + const readStableFile = vi.fn(async (path: string, signal?: AbortSignal) => + readFile(path, signal === undefined ? undefined : { signal })) + + const result = await __jsonlGenerationTest.ensure(request, { fs: { readFile: readStableFile } }) + + expect(result).toMatchObject({ status: 'current', version: 1, path: request.sourcePath }) + expect(readStableFile).toHaveBeenCalledOnce() + expect(migrate).not.toHaveBeenCalled() + expect(validateCurrent).not.toHaveBeenCalled() + expect(validateHistoricalHeader).not.toHaveBeenCalled() + expect(await readFile(request.sourcePath, 'utf8')).toBe(contents) + }) + + it('returns a disposable Zstandard body owner on the current fast path', async () => { + const root = await tempRoot() + const request = options(root, 'zstd', adapter(), 1) + await writeFile(request.sourcePath, await encodeZstd(1, [event0])) + + const result = await ensureJsonlGenerationCurrent(request) + const body = result.snapshot.zstdBody + + expect(body).toBeDefined() + body?.[Symbol.dispose]() + expect(body?.frames.next().done).toBe(true) + }) + + it.each(['none', 'zstd'] as const)( + 'validates the selected %s historical header before invoking migration', + async (compression) => { + const root = await tempRoot() + const request = options(root, compression) + const source = compression === 'zstd' + ? await encodeZstd(0, [event0]) + : Buffer.from(line(header(0)) + line(event0)) + await writeFile(request.sourcePath, source) + const failure = new Error('selected path does not match source header identity') + const migrate = vi.fn() + const validateHistoricalHeader = vi.fn(() => { throw failure }) + + await expect(ensureJsonlGenerationCurrent({ + ...request, + format: adapter({ migrate }), + validateHistoricalHeader, + })).rejects.toBe(failure) + + expect(validateHistoricalHeader).toHaveBeenCalledWith(expect.objectContaining({ id: 'generation-test' })) + expect(migrate).not.toHaveBeenCalled() + expect(await readFile(request.sourcePath)).toEqual(source) + expect(await readdir(root)).toEqual([basename(request.sourcePath)]) + }, + ) + + it('awaits asynchronous historical-header validation before migration', async () => { + const root = await tempRoot() + const request = options(root) + await writeFile(request.sourcePath, line(header(0)) + line(event0)) + const order: string[] = [] + + await ensureJsonlGenerationCurrent({ + ...request, + format: adapter({ + migrate: (source) => { + order.push('migrate') + return { header: { ...source.header, version: 1 }, rows: source.rows } + }, + }), + validateHistoricalHeader: async () => { + await Promise.resolve() + order.push('validate') + }, + }) + + expect(order).toEqual(['validate', 'migrate']) + }) + + it('rejects a resolver/header version disagreement before migration', async () => { + const root = await tempRoot() + const request = options(root) + await writeFile(request.sourcePath, line(header(1)) + line(event0)) + + await expect(ensureJsonlGenerationCurrent(request)).rejects.toThrow( + 'source filename identifies v0, but its header identifies v1', + ) + expect(await readdir(root)).toEqual(['session.jsonl']) + }) + + it('rejects malformed and future version discriminators before migration', async () => { + const root = await tempRoot() + const malformed = options(join(root, 'malformed')) + const future = options(join(root, 'future'), 'none', adapter(), 2) + await mkdir(join(root, 'malformed')) + await mkdir(join(root, 'future')) + await writeFile(malformed.sourcePath, line(header(-1))) + await writeFile(future.sourcePath, line(header(2, 'future-id'))) + + await expect(ensureJsonlGenerationCurrent(malformed)).rejects.toThrow( + 'header version is not a non-negative safe integer', + ) + await expect(ensureJsonlGenerationCurrent(future)).rejects.toMatchObject({ + name: 'JsonlGenerationNewerVersionError', + storedVersion: 2, + currentVersion: 1, + storedId: 'future-id', + } satisfies Partial) + }) + + it.each([ + [null, 'first line is not a JSON object'], + [[], 'first line is not a JSON object'], + [{ ...header(0), version: Number.MAX_SAFE_INTEGER + 1 }, 'header version is not a non-negative safe integer'], + [{ ...header(0), version: '0' }, 'header version is not a non-negative safe integer'], + ] as const)('rejects malformed physical header %#', async (value, message) => { + const root = await tempRoot() + const request = options(root) + await writeFile(request.sourcePath, line(value)) + + await expect(ensureJsonlGenerationCurrent(request)).rejects.toThrow(message) + }) + + it('rejects a negative-zero physical version', async () => { + const root = await tempRoot() + const request = options(root) + await writeFile(request.sourcePath, '{"type":"session","version":-0,"id":"generation-test"}\n') + + await expect(ensureJsonlGenerationCurrent(request)).rejects.toThrow( + 'header version is not a non-negative safe integer', + ) + }) + + it('distinguishes policy refusal from ordinary and invalid-output migration failures', async () => { + const root = await tempRoot() + const blockedRoot = join(root, 'blocked') + const ordinaryRoot = join(root, 'ordinary') + const wrongRoot = join(root, 'wrong') + await mkdir(blockedRoot) + await mkdir(ordinaryRoot) + await mkdir(wrongRoot) + const blocked = new Error('blocked by edge policy') + const ordinary = new Error('malformed source') + for (const dir of [blockedRoot, ordinaryRoot, wrongRoot]) { + await writeFile(generationPath(dir, 0, 'none'), line(header(0)) + line(event0)) + } + + await expect(ensureJsonlGenerationCurrent(options(blockedRoot, 'none', adapter({ + migrate: () => { throw blocked }, + isUnsupportedMigrationError: (error): error is Error => error === blocked, + })))).rejects.toMatchObject({ + name: 'JsonlGenerationUnsupportedMigrationError', + fromVersion: 0, + reason: blocked, + } satisfies Partial) + await expect(ensureJsonlGenerationCurrent(options(ordinaryRoot, 'none', adapter({ + migrate: () => { throw ordinary }, + })))).rejects.toBe(ordinary) + await expect(ensureJsonlGenerationCurrent(options(wrongRoot, 'none', adapter({ + migrate: source => ({ header: { ...source.header, version: 2 }, rows: source.rows }), + })))).rejects.toThrow('format migration returned v2, expected v1') + expect(await readdir(blockedRoot)).toEqual(['session.jsonl']) + expect(await readdir(ordinaryRoot)).toEqual(['session.jsonl']) + expect(await readdir(wrongRoot)).toEqual(['session.jsonl']) + }) + + it('refuses migration output that JSON cannot encode losslessly', async () => { + const circular: Record = {} + circular['self'] = circular + for (const [name, value] of [ + ['bigint', 1n], + ['circular', circular], + ['undefined', undefined], + ] as const) { + const root = await tempRoot() + const request = options(root) + await writeFile(request.sourcePath, line(header(0)) + line(event0)) + + await expect(ensureJsonlGenerationCurrent({ + ...request, + format: adapter({ + migrate: source => ({ header: { ...source.header, version: 1 }, rows: [value] }), + }), + })).rejects.toThrow('migrated session row 1 is not lossless JSON') + expect(await readdir(root), name).toEqual(['session.jsonl']) + } + }) + + it('publishes only the final generation across a multi-edge migration', async () => { + const root = await tempRoot() + const format = adapter({ + currentVersion: 3, + migrate: source => ({ header: { ...source.header, version: 3 }, rows: source.rows }), + validateCurrent: (candidate) => { + if (candidate.header.version !== 3) throw new Error('candidate is not v3') + }, + }) + const request = options(root, 'none', format, 1) + const source = Buffer.from(line(header(1)) + line(event0)) + await writeFile(request.sourcePath, source) + const sourceBefore = await stat(request.sourcePath, { bigint: true }) + + await ensureJsonlGenerationCurrent(request) + + expect(await readFile(request.sourcePath)).toEqual(source) + const sourceAfter = await stat(request.sourcePath, { bigint: true }) + expect([sourceAfter.dev, sourceAfter.ino]).toEqual([sourceBefore.dev, sourceBefore.ino]) + expect(await readFile(request.currentPath, 'utf8')).toBe(line(header(3)) + line(event0)) + expect((await readdir(root)).sort()).toEqual(['session.v1.jsonl', 'session.v3.jsonl']) + }) + + it.each(['none', 'zstd'] as const)( + 'uses one immutable publication algorithm for %s', + async (compression) => { + const root = await tempRoot() + const request = options(root, compression) + const source = compression === 'zstd' + ? await encodeZstd(0, [event0]) + : Buffer.from(line(header(0)) + line(event0)) + await writeFile(request.sourcePath, source) + const sourceBefore = await stat(request.sourcePath, { bigint: true }) + + await ensureJsonlGenerationCurrent(request) + + expect(await readFile(request.sourcePath)).toEqual(source) + const sourceAfter = await stat(request.sourcePath, { bigint: true }) + const current = await stat(request.currentPath, { bigint: true }) + expect([sourceAfter.dev, sourceAfter.ino]).toEqual([sourceBefore.dev, sourceBefore.ino]) + expect([current.dev, current.ino]).not.toEqual([sourceBefore.dev, sourceBefore.ino]) + const currentText = compression === 'zstd' + ? await decodeZstdJsonl(request.currentPath) + : await readFile(request.currentPath, 'utf8') + expect(currentText).toBe(line(header(1)) + line(event0)) + }, + ) + + it('handles header-only and torn-tail historical Zstandard generations', async () => { + const root = await tempRoot() + const headerRoot = join(root, 'header') + const emptyTailRoot = join(root, 'empty-tail') + const tornRoot = join(root, 'torn') + await mkdir(headerRoot) + await mkdir(emptyTailRoot) + await mkdir(tornRoot) + const headerRequest = options(headerRoot, 'zstd') + const emptyTailRequest = options(emptyTailRoot, 'zstd') + const tornRequest = options(tornRoot, 'zstd') + const headerFrame = await compressZstdFrame(line(header(0))) + const eventFrame = await compressZstdFrame(line(event0)) + const recoveredFrame = await compressZstdFrame(line(event1)) + await writeFile(headerRequest.sourcePath, headerFrame) + await writeFile(emptyTailRequest.sourcePath, Buffer.concat([headerFrame, eventFrame.subarray(0, 8)])) + await writeFile(tornRequest.sourcePath, Buffer.concat([headerFrame, eventFrame, recoveredFrame.subarray(0, -3)])) + + await ensureJsonlGenerationCurrent(headerRequest) + await ensureJsonlGenerationCurrent(emptyTailRequest) + await ensureJsonlGenerationCurrent(tornRequest) + + expect(await decodeZstdJsonl(headerRequest.currentPath)).toBe(line(header(1))) + expect(await decodeZstdJsonl(emptyTailRequest.currentPath)).toBe(line(header(1))) + expect(await decodeZstdJsonl(tornRequest.currentPath)).toBe(line(header(1)) + line(event0) + line(event1)) + }) + + it('rejects header-less raw and Zstandard sources and a non-independent Zstandard header frame', async () => { + const root = await tempRoot() + const rawRoot = join(root, 'raw') + const emptyZstdRoot = join(root, 'empty-zstd') + const joinedZstdRoot = join(root, 'joined-zstd') + await mkdir(rawRoot) + await mkdir(emptyZstdRoot) + await mkdir(joinedZstdRoot) + const raw = options(rawRoot) + const emptyZstd = options(emptyZstdRoot, 'zstd') + const joinedZstd = options(joinedZstdRoot, 'zstd') + await writeFile(raw.sourcePath, JSON.stringify(header(0))) + await writeFile(emptyZstd.sourcePath, Buffer.alloc(0)) + await writeFile(joinedZstd.sourcePath, await compressZstdFrame(line(header(0)) + line(event0))) + + await expect(ensureJsonlGenerationCurrent(raw)).rejects.toThrow('empty or header-less session log') + await expect(ensureJsonlGenerationCurrent(emptyZstd)).rejects.toThrow( + 'empty or header-less Zstandard session log', + ) + await expect(ensureJsonlGenerationCurrent(joinedZstd)).rejects.toThrow( + 'first frame is not exactly one header line', + ) + }) + + it('rejects a complete Zstandard frame whose final JSONL record is torn', async () => { + const root = await tempRoot() + const request = options(root, 'zstd') + await writeFile(request.sourcePath, Buffer.concat([ + await compressZstdFrame(line(header(0))), + await compressZstdFrame(JSON.stringify(event0)), + ])) + + await expect(ensureJsonlGenerationCurrent(request)).rejects.toThrow( + 'complete frame contains a torn JSONL record', + ) + expect(await readdir(root)).toEqual(['session.jsonl.zstd']) + }) + + it('drops an uncommitted corrupt raw suffix but refuses corruption before a committed turn end', async () => { + const root = await tempRoot() + const droppedRoot = join(root, 'dropped') + const refusedRoot = join(root, 'refused') + await mkdir(droppedRoot) + await mkdir(refusedRoot) + const dropped = options(droppedRoot) + const refused = options(refusedRoot) + const incomplete = line(header(0)) + line(event0) + '{not-json}\n' + line({ type: 'step/start', seq: 1 }) + const committed = line(header(0)) + line(event0) + '{not-json}\n' + line(event1) + await writeFile(dropped.sourcePath, incomplete) + await writeFile(refused.sourcePath, committed) + + await ensureJsonlGenerationCurrent(dropped) + await expect(ensureJsonlGenerationCurrent(refused)).rejects.toThrow('row 2 is not valid JSON') + + expect(await readFile(dropped.sourcePath, 'utf8')).toBe(incomplete) + expect(await readFile(dropped.currentPath, 'utf8')).toBe(line(header(1)) + line(event0)) + expect(await readFile(refused.sourcePath, 'utf8')).toBe(committed) + expect(await readdir(refusedRoot)).toEqual(['session.jsonl']) + }) + + it('drops a byte-torn raw suffix without altering the source', async () => { + const root = await tempRoot() + const request = options(root) + const source = Buffer.from(line(header(0)) + line(event0) + '{"type":"turn/end"') + await writeFile(request.sourcePath, source) + + await ensureJsonlGenerationCurrent(request) + + expect(await readFile(request.sourcePath)).toEqual(source) + expect(await readFile(request.currentPath, 'utf8')).toBe(line(header(1)) + line(event0)) + }) + + it('validates canonical lowercase generation filenames and one shared directory', async () => { + const root = await tempRoot() + const other = join(root, 'other') + await mkdir(other) + const source = line(header(0)) + const cases = [ + { + request: { ...options(root), sourcePath: join(root, 'session.v0.jsonl') }, + message: 'source path must end with "session.jsonl"', + }, + { + request: { ...options(root), currentPath: join(root, 'session.V1.jsonl') }, + message: 'current JSONL generation path must end with "session.v1.jsonl"', + }, + { + request: { ...options(root), currentPath: generationPath(other, 1, 'none') }, + message: 'must share one Session directory', + }, + ] + await writeFile(generationPath(root, 0, 'none'), source) + + for (const { request, message } of cases) { + await expect(ensureJsonlGenerationCurrent(request)).rejects.toThrow(message) + } + }) + + it('retries a bracketed physical read and a source changed before publication', async () => { + const root = await tempRoot() + const request = options(root) + const first = Buffer.from(line(header(0)) + line(event0)) + const second = Buffer.from(line(header(0)) + line(event0) + line(event1)) + await writeFile(request.sourcePath, first) + let stats = 0 + const statFile = async (path: string) => { + const value = await stat(path, { bigint: true }) + if (path !== request.sourcePath) return value + stats += 1 + return stats === 2 ? { ...value, mtimeNs: value.mtimeNs + 1n } : value + } + const migrate = vi.fn((source: Parameters[0]) => + adapter().migrate(source)) + const barrier = vi.fn(async (phase: string, attempt: number) => { + if (phase === 'before-source-check' && attempt === 1) await writeFile(request.sourcePath, second) + }) + + await __jsonlGenerationTest.ensure( + { ...request, format: adapter({ migrate }) }, + { fs: { stat: statFile }, barrier }, + ) + + expect(stats).toBeGreaterThan(2) + expect(migrate).toHaveBeenCalledTimes(2) + expect(await readFile(request.sourcePath)).toEqual(second) + expect(await readFile(request.currentPath, 'utf8')).toBe(line(header(1)) + line(event0) + line(event1)) + expect((await readdir(root)).every(name => !name.includes('.tmp'))).toBe(true) + }) + + it('never overwrites a colliding exclusive stage name', async () => { + const root = await tempRoot() + const request = options(root) + const collision = join(root, 'session.migration.collision.tmp.jsonl') + await writeFile(request.sourcePath, line(header(0)) + line(event0)) + await writeFile(collision, 'owned-by-another-attempt\n') + const randomToken = vi.fn().mockReturnValueOnce('collision').mockReturnValue('stage') + + await __jsonlGenerationTest.ensure(request, { randomToken }) + + expect(randomToken).toHaveBeenCalledTimes(2) + expect(await readFile(collision, 'utf8')).toBe('owned-by-another-attempt\n') + expect((await readdir(root)).sort()).toEqual([ + 'session.jsonl', + 'session.migration.collision.tmp.jsonl', + 'session.v1.jsonl', + ]) + }) + + it('accepts an identical regular target created by another migration', async () => { + const root = await tempRoot() + const request = options(root) + const source = Buffer.from(line(header(0)) + line(event0)) + const current = Buffer.from(line(header(1)) + line(event0)) + await writeFile(request.sourcePath, source) + await writeFile(request.currentPath, current) + + const result = await ensureJsonlGenerationCurrent(request) + + expect(result).toMatchObject({ status: 'migrated', path: request.currentPath }) + expect(await readFile(request.sourcePath)).toEqual(source) + expect(await readFile(request.currentPath)).toEqual(current) + expect((await readdir(root)).sort()).toEqual(['session.jsonl', 'session.v1.jsonl']) + }) + + it('accepts an identical regular hardlink target', async () => { + const root = await tempRoot() + const request = options(root) + const expected = join(root, 'expected.jsonl') + await writeFile(request.sourcePath, line(header(0)) + line(event0)) + await writeFile(expected, line(header(1)) + line(event0)) + await link(expected, request.currentPath) + + await expect(ensureJsonlGenerationCurrent(request)).resolves.toMatchObject({ path: request.currentPath }) + expect(await readFile(expected, 'utf8')).toBe(line(header(1)) + line(event0)) + }) + + it.each(['different', 'malformed', 'symlink', 'directory'] as const)( + 'fails loud without altering a colliding %s target', + async (kind) => { + const root = await tempRoot() + const request = options(root) + const source = Buffer.from(line(header(0)) + line(event0)) + await writeFile(request.sourcePath, source) + if (kind === 'different') await writeFile(request.currentPath, line(header(1)) + line(event1)) + if (kind === 'malformed') await writeFile(request.currentPath, '{not-json}\n') + if (kind === 'symlink') await symlink(request.sourcePath, request.currentPath) + if (kind === 'directory') await mkdir(request.currentPath) + + await expect(ensureJsonlGenerationCurrent(request)).rejects.toBeInstanceOf( + JsonlGenerationTargetConflictError, + ) + + expect(await readFile(request.sourcePath)).toEqual(source) + expect((await readdir(root)).every(name => !name.includes('.tmp'))).toBe(true) + }, + ) + + it('normalizes a non-Error rejection while reopening an existing target', async () => { + const root = await tempRoot() + let validations = 0 + const format = adapter({ + validateCurrent: () => { + validations += 1 + if (validations === 2) throw 'non-error rejection' + }, + }) + const request = options(root, 'none', format) + await writeFile(request.sourcePath, line(header(0)) + line(event0)) + await writeFile(request.currentPath, line(header(1)) + line(event0)) + + const failure = await ensureJsonlGenerationCurrent(request).then( + () => undefined, + (error: unknown) => error, + ) + if (!(failure instanceof JsonlGenerationTargetConflictError)) throw new Error('expected target conflict') + expect(failure.reason.message).toBe('current-generation validation failed with a non-Error rejection') + }) + + it('leaves source and published target immutable when committed reopen rejects it', async () => { + const root = await tempRoot() + let validations = 0 + const format = adapter({ + validateCurrent: (candidate) => { + adapter().validateCurrent(candidate) + validations += 1 + if (validations === 2) throw new Error('committed reopen rejected') + }, + }) + const request = options(root, 'none', format) + const source = Buffer.from(line(header(0)) + line(event0)) + await writeFile(request.sourcePath, source) + + const failure = await ensureJsonlGenerationCurrent(request).then( + () => undefined, + (error: unknown) => error, + ) + if (!(failure instanceof JsonlGenerationTargetConflictError)) throw new Error('expected target conflict') + expect(failure.reason.message).toBe('committed reopen rejected') + + expect(await readFile(request.sourcePath)).toEqual(source) + expect(await readFile(request.currentPath, 'utf8')).toBe(line(header(1)) + line(event0)) + }) + + it('reopens the target after publication instead of trusting staged validation', async () => { + const root = await tempRoot() + const validateCurrent = vi.fn((candidate: JsonlCurrentGeneration) => { + adapter().validateCurrent(candidate) + }) + const barrier = vi.fn() + const request = options(root, 'none', adapter({ validateCurrent })) + await writeFile(request.sourcePath, line(header(0)) + line(event0)) + + await __jsonlGenerationTest.ensure(request, { barrier }) + + expect(validateCurrent).toHaveBeenCalledTimes(2) + expect(barrier).toHaveBeenCalledWith('after-publication', 1) + }) + + it('removes an unconfirmed POSIX publication after the directory sync fails, then retries', async () => { + const root = await tempRoot() + const request = options(root) + const directorySyncFailure = new Error('published directory sync failed') + await writeFile(request.sourcePath, line(header(0)) + line(event0)) + let directorySyncs = 0 + const openFile = async (path: string, flags: string, mode?: number) => { + const handle = await openWithPosixDirectorySync(path, flags, mode) + if (path === root && flags === 'r') { + directorySyncs += 1 + if (directorySyncs === 1) vi.spyOn(handle, 'sync').mockRejectedValueOnce(directorySyncFailure) + } + return handle + } + + await expect(__jsonlGenerationTest.ensure( + request, + { platform: 'darwin', fs: { open: openFile } }, + )).rejects.toBe(directorySyncFailure) + expect(await readdir(root)).toEqual(['session.jsonl']) + + await expect(ensureJsonlGenerationCurrent(request)).resolves.toMatchObject({ path: request.currentPath }) + expect(await readFile(request.currentPath, 'utf8')).toBe(line(header(1)) + line(event0)) + }) + + it('preserves the primary directory-sync failure when publication rollback also fails', async () => { + const root = await tempRoot() + const request = options(root) + const directorySyncFailure = new Error('published directory sync failed') + const rollbackFailure = new Error('published target rollback failed') + await writeFile(request.sourcePath, line(header(0)) + line(event0)) + const openFile = async (path: string, flags: string, mode?: number) => { + const handle = await open(path, flags, mode) + if (path === root && flags === 'r') vi.spyOn(handle, 'sync').mockRejectedValue(directorySyncFailure) + return handle + } + + const failure = await __jsonlGenerationTest.ensure(request, { + platform: 'darwin', + fs: { + open: openFile, + rm: async (path: string) => { + if (path === request.currentPath) throw rollbackFailure + await rm(path, { force: true }) + }, + }, + }).then(() => undefined, (error: unknown) => error) + + if (!(failure instanceof AggregateError)) throw new Error('expected publication rollback aggregate') + expect(failure.errors).toEqual([directorySyncFailure, rollbackFailure, directorySyncFailure]) + expect(await readFile(request.currentPath, 'utf8')).toBe(line(header(1)) + line(event0)) + }) + + it('rethrows the exact abort reason after publication and leaves the committed target', async () => { + const root = await tempRoot() + const controller = new AbortController() + const reason = new Error('stop after publication') + const request = { ...options(root), signal: controller.signal } + await writeFile(request.sourcePath, line(header(0)) + line(event0)) + + await expect(__jsonlGenerationTest.ensure(request, { + barrier: (phase) => { + if (phase === 'after-publication') controller.abort(reason) + }, + })).rejects.toBe(reason) + expect(await readFile(request.currentPath, 'utf8')).toBe(line(header(1)) + line(event0)) + }) + + it('rejects a noncanonical case-insensitive collision instead of accepting its bytes', async () => { + const root = await tempRoot() + const request = options(root) + await writeFile(request.sourcePath, line(header(0)) + line(event0)) + + const failure = await __jsonlGenerationTest.ensure(request, { + platform: 'darwin', + fs: posixSimulationFs({ + link: async () => { throw fsError('EEXIST') }, + readdir: async () => ['session.V1.jsonl'], + }), + }).then(() => undefined, (error: unknown) => error) + + if (!(failure instanceof JsonlGenerationTargetConflictError)) throw new Error('expected target conflict') + expect(failure.reason.message).toContain('noncanonical directory entry "session.V1.jsonl"') + expect((await readdir(root)).every(name => !name.includes('.tmp'))).toBe(true) + }) + + it('reports an absent exclusive-publication winner as a target conflict', async () => { + const root = await tempRoot() + const request = options(root) + await writeFile(request.sourcePath, line(header(0)) + line(event0)) + + await expect(__jsonlGenerationTest.ensure(request, { + platform: 'darwin', + fs: posixSimulationFs({ link: async () => { throw fsError('EEXIST') } }), + })).rejects.toBeInstanceOf(JsonlGenerationTargetConflictError) + }) + + it('rethrows the exact abort reason during committed reopen and leaves the target', async () => { + const root = await tempRoot() + const controller = new AbortController() + const reason = new Error('stop during committed reopen') + const request = { ...options(root), signal: controller.signal } + await writeFile(request.sourcePath, line(header(0)) + line(event0)) + + await expect(__jsonlGenerationTest.ensure(request, { + fs: { + readFile: async (path, signal) => { + if (path === request.currentPath) controller.abort(reason) + return readFile(path, signal === undefined ? undefined : { signal }) + }, + }, + })).rejects.toBe(reason) + expect(await readFile(request.currentPath, 'utf8')).toBe(line(header(1)) + line(event0)) + }) + + it('leaves a crash-style staging file inert', async () => { + const root = await tempRoot() + const request = options(root) + const crashStage = join(root, 'session.migration.crash.tmp.jsonl') + await writeFile(request.sourcePath, line(header(0)) + line(event0)) + await writeFile(crashStage, line(header(99))) + + await ensureJsonlGenerationCurrent(request) + + expect(await readFile(crashStage, 'utf8')).toBe(line(header(99))) + expect(await readFile(request.currentPath, 'utf8')).toBe(line(header(1)) + line(event0)) + }) + + it('removes an exclusively created stage when writing or syncing it fails', async () => { + const root = await tempRoot() + const request = options(root) + await writeFile(request.sourcePath, line(header(0)) + line(event0)) + let injected = false + const openFile = async (path: string, flags: string, mode?: number) => { + const handle = await open(path, flags, mode) + if (!injected && path.includes('.tmp')) { + injected = true + vi.spyOn(handle, 'sync').mockRejectedValueOnce(new Error('simulated stage fsync failure')) + } + return handle + } + + await expect(__jsonlGenerationTest.ensure(request, { fs: { open: openFile } })).rejects.toThrow( + 'simulated stage fsync failure', + ) + expect(await readdir(root)).toEqual(['session.jsonl']) + }) + + it.each(['open', 'close', 'write-close'] as const)( + 'surfaces %s stage failures without leaving a stage', + async (mode) => { + const root = await tempRoot() + const request = options(root) + await writeFile(request.sourcePath, line(header(0)) + line(event0)) + const openFile = async (path: string, flags: string, fileMode?: number) => { + if (mode === 'open' && flags === 'wx') throw fsError('EACCES', 'stage open denied') + const handle = await open(path, flags, fileMode) + if (mode === 'write-close' && path.includes('.tmp')) { + vi.spyOn(handle, 'sync').mockRejectedValueOnce(new Error('stage write failed')) + } + if (mode !== 'open' && path.includes('.tmp')) { + const close = handle.close.bind(handle) + vi.spyOn(handle, 'close').mockImplementationOnce(async () => { + await close() + throw new Error('stage close failed') + }) + } + return handle + } + + await expect(__jsonlGenerationTest.ensure(request, { fs: { open: openFile } })).rejects.toThrow( + mode === 'open' + ? 'stage open denied' + : mode === 'close' + ? 'stage close failed' + : 'failed to write and close migration stage', + ) + expect(await readdir(root)).toEqual(['session.jsonl']) + }, + ) + + it.each([false, true])('normalizes a non-Error stage failure (cleanup fails: %s)', async (cleanupFails) => { + const root = await tempRoot() + const request = options(root) + await writeFile(request.sourcePath, line(header(0)) + line(event0)) + const openFile = async (path: string, flags: string, mode?: number) => { + const handle = await open(path, flags, mode) + if (path.includes('.tmp')) vi.spyOn(handle, 'sync').mockRejectedValueOnce('non-error failure') + return handle + } + const removeFile = async (path: string) => { + if (cleanupFails && path.includes('.tmp')) throw new Error('stage cleanup failed') + await rm(path, { force: true }) + } + + await expect(__jsonlGenerationTest.ensure( + request, + { fs: { open: openFile, rm: removeFile } }, + )).rejects.toThrow(cleanupFails + ? 'failed to clean migration stage' + : 'migration stage write failed with a non-Error rejection') + }) + + it('preserves a publication failure when temporary cleanup also fails', async () => { + const root = await tempRoot() + const request = options(root) + const publication = new Error('exclusive publication failed') + const cleanup = new Error('stage cleanup failed') + await writeFile(request.sourcePath, line(header(0)) + line(event0)) + const removeFile = async (path: string) => { + if (path.includes('.tmp')) throw cleanup + await rm(path, { force: true }) + } + + const failure = await __jsonlGenerationTest.ensure( + request, + { + platform: 'darwin', + fs: posixSimulationFs({ + link: async () => { throw publication }, + rm: removeFile, + }), + }, + ).then(() => undefined, (error: unknown) => error) + + if (!(failure instanceof AggregateError)) throw new Error('expected an aggregate cleanup failure') + expect(failure.errors).toEqual([publication, cleanup]) + expect(await readFile(request.sourcePath, 'utf8')).toBe(line(header(0)) + line(event0)) + }) + + it('keeps an exclusively published target when successful stage cleanup fails', async () => { + const root = await tempRoot() + const request = options(root) + const cleanup = new Error('published stage cleanup failed') + await writeFile(request.sourcePath, line(header(0)) + line(event0)) + + await expect(__jsonlGenerationTest.ensure(request, { + platform: 'darwin', + fs: posixSimulationFs({ + rm: async (path: string) => { + if (path.includes('.tmp')) throw cleanup + await rm(path, { force: true }) + }, + }), + })).rejects.toBe(cleanup) + + expect(await readFile(request.currentPath, 'utf8')).toBe(line(header(1)) + line(event0)) + }) + + it('surfaces candidate validation errors and cleanup errors without publishing', async () => { + const root = await tempRoot() + const request = options(root, 'none', adapter({ + validateCurrent: () => { throw new Error('candidate validation failed') }, + })) + const cleanup = new Error('cleanup failed') + await writeFile(request.sourcePath, line(header(0)) + line(event0)) + + const failure = await __jsonlGenerationTest.ensure(request, { + fs: { + rm: async () => { throw cleanup }, + }, + }).then(() => undefined, (error: unknown) => error) + + if (!(failure instanceof AggregateError)) throw new Error('expected aggregate validation cleanup failure') + expect(failure.errors[0]).toMatchObject({ message: 'candidate validation failed' }) + expect(failure.errors[1]).toBe(cleanup) + await expect(readFile(request.currentPath)).rejects.toMatchObject({ code: 'ENOENT' }) + }) + + it.each(['torn', 'old', 'invalid-json'] as const)( + 'rejects a %s staged candidate before publication', + async (mode) => { + const root = await tempRoot() + const request = options(root) + await writeFile(request.sourcePath, line(header(0)) + line(event0)) + const readFileForStage = async (path: string, signal?: AbortSignal) => { + const bytes = await readFile(path, signal === undefined ? undefined : { signal }) + if (!path.includes('.tmp')) return bytes + if (mode === 'torn') return bytes.subarray(0, -1) + if (mode === 'old') return Buffer.from(line(header(0)) + line(event0)) + return Buffer.from(line(header(1)) + '{not-json}\n') + } + + await expect(__jsonlGenerationTest.ensure( + request, + { fs: { readFile: readFileForStage } }, + )).rejects.toThrow( + mode === 'torn' + ? 'staged current session generation has a torn physical tail' + : mode === 'old' + ? 'staged session generation is not current v1' + : 'row 1 is not valid JSON', + ) + expect(await readdir(root)).toEqual(['session.jsonl']) + }, + ) + + it('uses Windows write-through exclusive publication without replacing the source', async () => { + const root = await tempRoot() + const request = options(root) + const source = Buffer.from(line(header(0)) + line(event0)) + await writeFile(request.sourcePath, source) + const publishNewWin32 = vi.fn(async (from: string, to: string) => { await rename(from, to) }) + + await __jsonlGenerationTest.ensure(request, { platform: 'win32', publishNewWin32 }) + + expect(publishNewWin32).toHaveBeenCalledOnce() + expect(publishNewWin32.mock.calls[0]?.[1]).toBe(request.currentPath) + expect(await readFile(request.sourcePath)).toEqual(source) + expect(await readFile(request.currentPath, 'utf8')).toBe(line(header(1)) + line(event0)) + }) + + it('accepts an identical target that wins Windows publication', async () => { + const root = await tempRoot() + const request = options(root) + await writeFile(request.sourcePath, line(header(0)) + line(event0)) + const publishNewWin32 = vi.fn(async (_from: string, to: string) => { + await writeFile(to, line(header(1)) + line(event0)) + throw fsError('EEXIST') + }) + + await expect(__jsonlGenerationTest.ensure( + request, + { platform: 'win32', publishNewWin32 }, + )).resolves.toMatchObject({ path: request.currentPath }) + expect((await readdir(root)).every(name => !name.includes('.tmp'))).toBe(true) + }) + + it('propagates non-collision Windows and POSIX publication failures', async () => { + for (const platform of ['win32', 'darwin'] as const) { + const root = await tempRoot() + const request = options(root) + const failure = new Error(`${platform} publication failed`) + await writeFile(request.sourcePath, line(header(0)) + line(event0)) + + await expect(__jsonlGenerationTest.ensure(request, platform === 'win32' + ? { platform, publishNewWin32: async () => { throw failure } } + : { platform, fs: posixSimulationFs({ link: async () => { throw failure } }) })) + .rejects.toBe(failure) + expect(await readdir(root)).toEqual(['session.jsonl']) + } + }) + + it('accepts an identical target that wins POSIX publication', async () => { + const root = await tempRoot() + const request = options(root) + await writeFile(request.sourcePath, line(header(0)) + line(event0)) + let raced = false + const linkFile = async (existingPath: string, newPath: string) => { + await link(existingPath, newPath) + raced = true + throw fsError('EEXIST') + } + + await __jsonlGenerationTest.ensure( + request, + { platform: 'darwin', fs: posixSimulationFs({ link: linkFile }) }, + ) + + expect(raced).toBe(true) + expect(await readFile(request.currentPath, 'utf8')).toBe(line(header(1)) + line(event0)) + }) + + it('honors cancellation before reading a generation', async () => { + const root = await tempRoot() + const controller = new AbortController() + const request = { ...options(root), signal: controller.signal } + await writeFile(request.sourcePath, line(header(0))) + controller.abort(new Error('cancelled migration')) + + await expect(ensureJsonlGenerationCurrent(request)).rejects.toThrow('cancelled migration') + expect(await readdir(root)).toEqual(['session.jsonl']) + }) +}) diff --git a/packages/session/session-persistence-jsonl/tests/jsonl.spec.ts b/packages/session/session-persistence-jsonl/tests/jsonl.spec.ts index 363ec28212..cb32ec9026 100644 --- a/packages/session/session-persistence-jsonl/tests/jsonl.spec.ts +++ b/packages/session/session-persistence-jsonl/tests/jsonl.spec.ts @@ -4,12 +4,14 @@ import { Context } from '@deepseek-ai/cordis' import { appendFile, mkdtemp, mkdir, rm, readFile, writeFile, readdir, stat, symlink } from 'node:fs/promises' import { tmpdir } from 'node:os' import { dirname, isAbsolute, join, relative, resolve } from 'node:path' -import SessionStore, { SessionId, SessionLogOffset, SessionSeq } from '@deepseek-ai/dsh-session' +import SessionStore, { SESSION_FORMAT_VERSION, SessionId, SessionLogOffset, SessionSeq } from '@deepseek-ai/dsh-session' import type { Session, SessionEvent, SessionHeader } from '@deepseek-ai/dsh-session' import JsonlSessionPersistence from '@deepseek-ai/dsh-session-persistence-jsonl' +import type { SessionPersistenceListing } from '@deepseek-ai/dsh-session-persistence' import { - encodeSegment, eventLines, logPath, parseHeader, projectDir, projectKey, scanLog, sessionDir, SessionLogScanner, - toHeaderLine, + encodeSegment, eventLines, generationLogFilename, generationLogPath, logPath, parseGenerationLogFilename, + parseHeader, parseHeaderValue, projectDir, projectKey, scanLog, sessionDir, + SessionLogScanner, toHeaderLine, } from '../src/format.ts' import { runPersistenceContract, meta, oneTurnLog, appendLog } from '../../session-persistence/tests/contract.ts' import { runCoordinatorContract, type CoordinatorFixture } from '../../session-persistence/tests/coordinator-contract.ts' @@ -19,10 +21,43 @@ const statRace = vi.hoisted(() => ({ reads: 0, })) +const physicalReadProbe = vi.hoisted(() => ({ + path: undefined as string | undefined, + opens: 0, + reads: 0, +})) + +const directoryReadFailure = vi.hoisted(() => ({ + path: undefined as string | undefined, + error: undefined as Error | undefined, +})) + +const directoryReadProbe = vi.hoisted(() => ({ + path: undefined as string | undefined, + reads: 0, +})) + vi.mock('node:fs/promises', async (importOriginal) => { const actual = await importOriginal() return { ...actual, + readdir: async (...args: Parameters) => { + if (String(args[0]) === directoryReadProbe.path) directoryReadProbe.reads += 1 + if (String(args[0]) === directoryReadFailure.path && directoryReadFailure.error !== undefined) { + throw directoryReadFailure.error + } + return actual.readdir(...args) + }, + open: async (...args: Parameters) => { + if (typeof args[0] === 'string' && args[0] === physicalReadProbe.path && args[1] === 'r') { + physicalReadProbe.opens += 1 + } + return actual.open(...args) + }, + readFile: (async (...args: Parameters) => { + if (typeof args[0] === 'string' && args[0] === physicalReadProbe.path) physicalReadProbe.reads += 1 + return actual.readFile(...args) + }) as typeof actual.readFile, stat: (async (...args: Parameters) => { const identity = await actual.stat(...args) if (String(args[0]) !== statRace.path || !('mtimeNs' in identity)) return identity @@ -84,9 +119,33 @@ function rawLogPath(root: string, cwd: string | undefined, id: SessionId): strin return logPath(root, cwd, id, 'none') } +function rawGenerationPath(root: string, cwd: string | undefined, id: SessionId, version: number): string { + return generationLogPath(root, cwd, id, version, 'none') +} + +function releasedV0Header(header: SessionHeader): Record { + return { ...toHeaderLine(header), version: 0 } +} + +function listedHeaders(listings: readonly SessionPersistenceListing[]): SessionHeader[] { + return listings.flatMap(listing => + listing.status === 'current' || listing.status === 'migration-required' ? [listing.header] : []) +} + +function listedIds(listings: readonly SessionPersistenceListing[]): SessionId[] { + return listedHeaders(listings).map(header => header.id) +} + afterEach(async () => { statRace.path = undefined statRace.reads = 0 + physicalReadProbe.path = undefined + physicalReadProbe.opens = 0 + physicalReadProbe.reads = 0 + directoryReadFailure.path = undefined + directoryReadFailure.error = undefined + directoryReadProbe.path = undefined + directoryReadProbe.reads = 0 vi.restoreAllMocks() for (const d of dirs.splice(0)) await rm(d, { recursive: true, force: true }) }) @@ -134,11 +193,67 @@ runCoordinatorContract('jsonl-none', async (): Promise => { }) describe('JsonlSessionPersistence: format helpers', () => { + it('enforces seeded header cuts and rejects invalid standalone header JSON', () => { + expect(() => toHeaderLine({ ...meta('missing-seed-cut'), isSeeded: true })) + .toThrow('seeded session header requires an inherited event count') + expect(() => toHeaderLine(meta('unexpected-seed-cut'), SessionLogOffset(1))) + .toThrow('unseeded session header inherited event count must be 0') + expect(parseHeader('{not-json')).toBeUndefined() + }) + + it('classifies already-parsed header values before translating current metadata', () => { + const current = toHeaderLine({ + ...meta('parsed-header', '/work'), + parentSession: SessionId('parsed-parent'), + origin: 'subagent', + delegationDepth: 1, + }) + + expect(parseHeaderValue(null)).toBeUndefined() + expect(parseHeaderValue([])).toBeUndefined() + expect(parseHeaderValue({ ...current, version: '1' })).toBeUndefined() + expect(parseHeaderValue({ ...current, delegationDepth: undefined })).toBeUndefined() + expect(parseHeaderValue(current)?.meta).toMatchObject({ + parentSession: SessionId('parsed-parent'), + origin: 'subagent', + }) + expect(() => parseHeaderValue({ version: 42, id: 123 })) + .toThrow('session "123" uses log format v42') + }) + + it('refuses catalog and Session version skew before probing the configured root', async () => { + const blockedRoot = join(await freshRoot(), 'not-a-directory') + await writeFile(blockedRoot, 'must remain unread') + vi.resetModules() + vi.doMock('@deepseek-ai/dsh-session-format-catalog', async (importOriginal) => { + const actual = await importOriginal() + return { + ...actual, + sessionFormatCatalog: { + ...actual.sessionFormatCatalog, + currentVersion: SESSION_FORMAT_VERSION + 1, + }, + } + }) + const skewCtx = new Context() + try { + const { default: SkewedJsonlSessionPersistence } = await import('../src/index.ts') + expect(() => new SkewedJsonlSessionPersistence(skewCtx, { + root: blockedRoot, + compression: 'none', + })).toThrow(`format catalog v${SESSION_FORMAT_VERSION + 1} does not match Session v${SESSION_FORMAT_VERSION}`) + } finally { + vi.doUnmock('@deepseek-ai/dsh-session-format-catalog') + vi.resetModules() + await skewCtx.fiber.dispose() + } + }) + it.each([ ['absent', undefined, false, 0], ['zero', 0, true, 0], ['nonzero', 3, true, 3], - ] as const)('round-trips the v0 physical seedLength when it is %s', ( + ] as const)('round-trips the current physical seedLength when it is %s', ( _case, seedLength, isSeeded, @@ -146,7 +261,7 @@ describe('JsonlSessionPersistence: format helpers', () => { ) => { const line = { type: 'session', - version: 0, + version: SESSION_FORMAT_VERSION, id: SessionId(`physical-seed-${_case}`), createdAt: 1000, ...seedLength === undefined ? {} : { seedLength }, @@ -212,6 +327,35 @@ describe('JsonlSessionPersistence: format helpers', () => { expect(() => projectKey('')).toThrow(/empty project path/) }) + it('names immutable generations with suffixless v0 and lowercase vN', () => { + expect(generationLogFilename(0, 'none')).toBe('session.jsonl') + expect(generationLogFilename(1, 'none')).toBe('session.v1.jsonl') + expect(generationLogFilename(27, 'zstd')).toBe('session.v27.jsonl.zstd') + expect(rawLogPath('/sessions', '/work', SessionId('current'))) + .toBe(rawGenerationPath('/sessions', '/work', SessionId('current'), SESSION_FORMAT_VERSION)) + for (const invalid of [-1, -0, 1.5, Number.MAX_SAFE_INTEGER + 1, Number.NaN]) { + expect(() => generationLogFilename(invalid, 'none')).toThrow(/non-negative safe integer/) + } + }) + + it('parses only canonical committed generation filenames', () => { + expect(parseGenerationLogFilename('session.jsonl', 'none')).toBe(0) + expect(parseGenerationLogFilename('session.v1.jsonl', 'none')).toBe(1) + expect(parseGenerationLogFilename('session.v42.jsonl.zstd', 'zstd')).toBe(42) + for (const ignored of [ + 'session.v0.jsonl', + 'session.v01.jsonl', + 'session.V1.jsonl', + 'session.v1.20260831T010203000Z.backup.jsonl', + 'session.migration.deadbeef.tmp.jsonl', + 'session.v9007199254740992.jsonl', + 'metadata.json', + ]) { + expect(parseGenerationLogFilename(ignored, 'none')).toBeUndefined() + } + expect(parseGenerationLogFilename('session.v1.jsonl.zstd', 'none')).toBeUndefined() + }) + it('resolves a relative custom root before locating a session', async () => { const absoluteRoot = await freshRoot() const ctx = new Context() @@ -238,7 +382,7 @@ describe('JsonlSessionPersistence: format helpers', () => { // createdAt, unknown fields): the version must be refused before shape // validation, so the user sees the upgrade direction. const id = SessionId('future-shape') - const path = rawLogPath(resolve(absoluteRoot), '/work', id) + const path = rawGenerationPath(resolve(absoluteRoot), '/work', id, 42) await mkdir(dirname(path), { recursive: true }) await writeFile(path, `${JSON.stringify({ type: 'session', version: 42, id, futureOnly: true })}\n{"future":"row"}\n`) const failure = await ctx.sessionPersistence.load(id).then(() => undefined, (error: unknown) => error as Error) @@ -261,7 +405,7 @@ describe('JsonlSessionPersistence: format helpers', () => { await writeFile(path, '42\n') const failure = await ctx.sessionPersistence.load(id).then(() => undefined, (error: unknown) => error as Error) expect(failure?.name).not.toBe('SessionFormatUnsupportedError') - expect(failure?.message).toContain('first line is not a session header') + expect(failure?.message).toContain('first line is not a JSON object') await fiber.dispose() }) @@ -273,7 +417,7 @@ describe('JsonlSessionPersistence: format helpers', () => { // A future header's id field is as untrusted as the rest of its shape: // the refusal must still name the session it read, not crash on the type. const id = SessionId('numeric-id') - const path = rawLogPath(resolve(absoluteRoot), '/work', id) + const path = rawGenerationPath(resolve(absoluteRoot), '/work', id, 42) await mkdir(dirname(path), { recursive: true }) await writeFile(path, `${JSON.stringify({ type: 'session', version: 42, id: 123 })}\n`) const failure = await ctx.sessionPersistence.load(id).then(() => undefined, (error: unknown) => error as Error) @@ -292,15 +436,17 @@ describe('JsonlSessionPersistence: format helpers', () => { const ctx = new Context() await ctx.plugin(SessionStore) const fiber = await ctx.plugin(JsonlSessionPersistence, { root: absoluteRoot, compression: 'none' }) - const m = { ...meta('newer-format', '/work'), version: 7 } - await ctx.sessionPersistence.create(m) - await ctx.sessionPersistence.append(m.id, [ - { type: 'turn/start', seq: SessionSeq(0), time: 1, data: { turn: 1 } }, - { type: 'turn/end', seq: SessionSeq(1), time: 2, data: { turn: 1, reason: { kind: 'completed' } } }, - ]) + const m = meta('newer-format', '/work') + const path = rawGenerationPath(absoluteRoot, m.cwd, m.id, 7) + await mkdir(dirname(path), { recursive: true }) + await writeFile(path, `${JSON.stringify({ ...toHeaderLine(m), version: 7 })}\n`) + const [listing] = await ctx.sessionPersistence.list() + expect(listing).toMatchObject({ status: 'unsupported', storedVersion: 7, targetVersion: 1 }) + const backend = ctx.sessionPersistence as JsonlSessionPersistence + await expect(backend.loadStored(m.id)).rejects.toThrow(`(raw log: ${path})`) const failure = await ctx.sessionPersistence.load(m.id).then(() => undefined, (error: unknown) => error as Error) expect(failure?.name).toBe('SessionFormatUnsupportedError') - expect(failure?.message).toContain(`(raw log: ${rawLogPath(resolve(absoluteRoot), '/work', m.id)})`) + expect(failure?.message).toContain(`(raw log: ${path})`) await fiber.dispose() }) }) @@ -326,17 +472,17 @@ describe('JsonlSessionPersistence: durability and crash semantics', () => { // materializes a file before the first append. const dir = sessionDir(root, '/work', m.id) await expect(stat(rawLogPath(root, '/work', m.id))).rejects.toThrow() - expect((await ctx.sessionPersistence.list()).map(h => h.id)).not.toContain(m.id) + expect(listedIds(await ctx.sessionPersistence.list())).not.toContain(m.id) await ctx.sessionPersistence.append(m.id, oneTurnLog()) expect((await stat(dir)).isDirectory()).toBe(true) expect((await stat(rawLogPath(root, '/work', m.id))).isFile()).toBe(true) - expect((await ctx.sessionPersistence.list()).map(h => h.id)).toContain(m.id) + expect(listedIds(await ctx.sessionPersistence.list())).toContain(m.id) }) it('lists a seeded header without reading an event body', async () => { const id = SessionId('header-only-seeded') - const path = rawLogPath(root, '/work', id) + const path = rawGenerationPath(root, '/work', id, 0) await mkdir(dirname(path), { recursive: true }) await writeFile(path, `${JSON.stringify({ type: 'session', @@ -348,9 +494,56 @@ describe('JsonlSessionPersistence: durability and crash semantics', () => { delegationDepth: 0, })}\n{not-valid-json`) - await expect(ctx.sessionPersistence.list()).resolves.toEqual([ - expect.objectContaining({ id, isSeeded: true }), - ]) + const [listing] = await ctx.sessionPersistence.list() + expect(listing).toMatchObject({ status: 'migration-required', storedVersion: 0 }) + if (listing?.status !== 'migration-required') throw new Error('expected migration-required listing') + expect(listing.header).toMatchObject({ id, isSeeded: true }) + }) + + it('lists current lineage and subagent origin from the header alone', async () => { + const id = SessionId('header-only-lineage') + const parentSession = SessionId('header-only-parent') + const path = rawLogPath(root, '/work', id) + await mkdir(dirname(path), { recursive: true }) + await writeFile(path, `${JSON.stringify(toHeaderLine({ + ...meta(id, '/work'), + parentSession, + origin: 'subagent', + delegationDepth: 1, + }))}\n`) + + const [listing] = await ctx.sessionPersistence.list() + + if (listing?.status !== 'current') throw new Error('expected current listing') + expect(listing.header).toMatchObject({ id, parentSession, origin: 'subagent' }) + await expect(ctx.sessionPersistence.inspect(id)).resolves.toMatchObject({ + meta: { id, parentSession, origin: 'subagent' }, + }) + }) + + it('classifies malformed current headers identically for list, inspect, and raw read', async () => { + const fixtures = [ + ['extra', { unexpected: true }], + ['cwd-type', { cwd: 7 }], + ['cwd-relative', { cwd: 'relative/path' }], + ['parent-type', { parentSession: 7 }], + ] as const + for (const [name, change] of fixtures) { + const id = SessionId(`malformed-current-${name}`) + const path = rawLogPath(root, '/work', id) + await mkdir(dirname(path), { recursive: true }) + await writeFile(path, `${JSON.stringify({ ...toHeaderLine(meta(id, '/work')), ...change })}\n`) + } + + const listings = await ctx.sessionPersistence.list() + + expect(listings).toHaveLength(fixtures.length) + expect(listings.every(listing => listing.status === 'malformed')).toBe(true) + for (const [name] of fixtures) { + const id = SessionId(`malformed-current-${name}`) + await expect(ctx.sessionPersistence.inspect(id)).rejects.toThrow(/invalid current header|cwd must be absolute/) + await expect(ctx.sessionPersistence.readRaw(id)).rejects.toThrow(/invalid current header|cwd must be absolute/) + } }) it('materializes an explicitly durable empty live session without an event row', async () => { @@ -378,13 +571,290 @@ describe('JsonlSessionPersistence: durability and crash semantics', () => { preparation[Symbol.dispose]() }) + it('delegates borrowed current sources through the JSONL provider', async () => { + const m = meta('direct-borrow', '/work') + await ctx.sessionPersistence.create(m) + await ctx.sessionPersistence.append(m.id, oneTurnLog()) + + using borrowed = await ctx.sessionPersistence.borrowSession(m.id) + + expect(borrowed.inspection.meta.id).toBe(m.id) + }) + + it('lists released v0 without mutation, then persists v1 on the first body read', async () => { + expect(SESSION_FORMAT_VERSION).toBe(1) + const m = meta('released-v0-on-read', '/work') + const path = rawGenerationPath(root, m.cwd, m.id, 0) + const currentPath = rawLogPath(root, m.cwd, m.id) + const v0Header = releasedV0Header(m) + const source = Buffer.from(`${JSON.stringify(v0Header)}\n${eventLines(oneTurnLog(), true)}\n`) + await mkdir(dirname(path), { recursive: true }) + await writeFile(path, source) + + const [before] = await ctx.sessionPersistence.list() + expect(before).toMatchObject({ status: 'migration-required', storedVersion: 0, targetVersion: 1 }) + if (before?.status !== 'migration-required') throw new Error('expected migration-required listing') + expect(before.header).toMatchObject({ id: m.id, version: 1 }) + expect(await readFile(path)).toEqual(source) + + directoryReadProbe.path = dirname(path) + await expect(ctx.sessionPersistence.inspect(m.id)).resolves.toEqual({ + meta: { ...m, version: 1, delegationDepth: 0 }, + inheritedEventCount: SessionLogOffset(0), + events: oneTurnLog(), + }) + expect(directoryReadProbe.reads).toBe(1) + await expect(ctx.sessionPersistence.readRaw(m.id)).resolves.toMatchObject({ meta: { id: m.id } }) + expect(directoryReadProbe.reads).toBe(1) + expect(await readFile(path)).toEqual(source) + expect(JSON.parse((await readFile(currentPath, 'utf8')).split('\n')[0] as string)).toMatchObject({ version: 1 }) + await expect(ctx.sessionPersistence.list()).resolves.toEqual([ + expect.objectContaining({ status: 'current', storedVersion: 1 }), + ]) + expect(directoryReadProbe.reads).toBe(2) + }) + + it('cold-opens the numeric highest generation and refuses a retained future successor', async () => { + const m = meta('cold-future-successor', '/work') + const v1Path = rawGenerationPath(root, m.cwd, m.id, 1) + const v2Path = rawGenerationPath(root, m.cwd, m.id, 2) + const v1 = `${JSON.stringify(toHeaderLine(m))}\n${eventLines(oneTurnLog(), true)}\n` + const v2 = `${JSON.stringify({ type: 'session', version: 2, id: m.id })}\n` + await mkdir(dirname(v1Path), { recursive: true }) + await writeFile(v1Path, v1) + await writeFile(v2Path, v2) + + await expect(ctx.sessionPersistence.list()).resolves.toEqual([ + expect.objectContaining({ + status: 'unsupported', + storedVersion: 2, + location: { kind: 'jsonl', path: v2Path }, + }), + ]) + await expect(ctx.sessionPersistence.inspect(m.id)) + .rejects.toThrow(/log format v2.*newer harness/) + expect(await readFile(v1Path, 'utf8')).toBe(v1) + expect(await readFile(v2Path, 'utf8')).toBe(v2) + }) + + it('reuses a validated current selection while header listing always rescans', async () => { + const m = meta('validated-generation-cache', '/work') + const path = rawLogPath(root, m.cwd, m.id) + await mkdir(dirname(path), { recursive: true }) + await writeFile(path, `${JSON.stringify(toHeaderLine(m))}\n${eventLines(oneTurnLog(), true)}\n`) + directoryReadProbe.path = dirname(path) + const backend = ctx.sessionPersistence as JsonlSessionPersistence + + await expect(ctx.sessionPersistence.readRaw(m.id)).resolves.toMatchObject({ meta: { id: m.id } }) + expect(directoryReadProbe.reads).toBe(1) + await expect(ctx.sessionPersistence.readRaw(m.id)).resolves.toMatchObject({ meta: { id: m.id } }) + await expect(backend.readStoredRevision(m.id)).resolves.toBeDefined() + expect(directoryReadProbe.reads).toBe(1) + + await expect(ctx.sessionPersistence.list()).resolves.toEqual([ + expect.objectContaining({ status: 'current', storedVersion: SESSION_FORMAT_VERSION }), + ]) + expect(directoryReadProbe.reads).toBe(2) + + directoryReadProbe.reads = 0 + const reopened = new Context() + await reopened.plugin(SessionStore) + await reopened.plugin(JsonlSessionPersistence, { root, compression: 'none' }) + try { + await expect(reopened.sessionPersistence.readRaw(m.id)).resolves.toMatchObject({ meta: { id: m.id } }) + expect(directoryReadProbe.reads).toBe(1) + } finally { + await reopened.fiber.dispose() + } + }) + + it('caches a newly materialized current generation for later opens', async () => { + const m = meta('materialized-generation-cache', '/work') + await ctx.sessionPersistence.create(m) + await ctx.sessionPersistence.append(m.id, oneTurnLog()) + directoryReadProbe.path = sessionDir(root, m.cwd, m.id) + const backend = ctx.sessionPersistence as JsonlSessionPersistence + + await expect(ctx.sessionPersistence.readRaw(m.id)).resolves.toMatchObject({ meta: { id: m.id } }) + await expect(backend.readStoredRevision(m.id)).resolves.toBeDefined() + expect(directoryReadProbe.reads).toBe(0) + }) + + it('selects the highest canonical generation across gaps and refuses a future version', async () => { + const m = meta('generation-gap-future', '/work') + const v0Path = rawGenerationPath(root, m.cwd, m.id, 0) + const v3Path = rawGenerationPath(root, m.cwd, m.id, 3) + const v0 = `${JSON.stringify(releasedV0Header(m))}\n` + const v3 = `${JSON.stringify({ type: 'session', version: 3, id: m.id })}\n` + await mkdir(dirname(v0Path), { recursive: true }) + await writeFile(v0Path, v0) + await writeFile(v3Path, v3) + + await expect(ctx.sessionPersistence.list()).resolves.toEqual([ + expect.objectContaining({ + status: 'unsupported', + storedVersion: 3, + location: { kind: 'jsonl', path: v3Path }, + }), + ]) + await expect(ctx.sessionPersistence.inspect(m.id)) + .rejects.toThrow(/log format v3.*newer harness/) + expect(await readFile(v0Path, 'utf8')).toBe(v0) + expect(await readFile(v3Path, 'utf8')).toBe(v3) + }) + + it('ignores backup and temporary names while selecting the committed v0 source', async () => { + const m = meta('generation-ignores-archives', '/work') + const v0Path = rawGenerationPath(root, m.cwd, m.id, 0) + await mkdir(dirname(v0Path), { recursive: true }) + await writeFile(v0Path, `${JSON.stringify(releasedV0Header(m))}\n`) + await writeFile(join(dirname(v0Path), 'session.v9.20260831T010203000Z.backup.jsonl'), 'archive') + await writeFile(join(dirname(v0Path), 'session.migration.deadbeef.tmp.jsonl'), 'stage') + + await expect(ctx.sessionPersistence.inspect(m.id)).resolves.toMatchObject({ + meta: { id: m.id, version: SESSION_FORMAT_VERSION }, + }) + expect(await readFile(v0Path, 'utf8')).toBe(`${JSON.stringify(releasedV0Header(m))}\n`) + }) + + it('marks a generation whose filename and physical header disagree as malformed', async () => { + const m = meta('generation-header-mismatch', '/work') + const path = rawGenerationPath(root, m.cwd, m.id, 3) + await mkdir(dirname(path), { recursive: true }) + await writeFile(path, `${JSON.stringify({ type: 'session', version: 2, id: m.id })}\n`) + + const [listing] = await ctx.sessionPersistence.list() + expect(listing?.status).toBe('malformed') + if (listing?.status !== 'malformed') throw new Error('expected malformed listing') + expect(listing.reason).toMatch(/filename identifies v3.*header identifies v2/) + await expect(ctx.sessionPersistence.inspect(m.id)).rejects.toThrow(/filename identifies v3.*header identifies v2/) + }) + + it('rejects a versioned generation encoded with the opposite configured suffix', async () => { + const m = meta('opposite-versioned-generation', '/work') + const path = generationLogPath(root, m.cwd, m.id, 4, 'zstd') + await mkdir(dirname(path), { recursive: true }) + await writeFile(path, 'opposite encoding') + + await expect(ctx.sessionPersistence.list()).rejects.toThrow(/uses \.jsonl\.zstd/) + await expect(ctx.sessionPersistence.inspect(m.id)).rejects.toThrow(/uses \.jsonl\.zstd/) + }) + + it('migrates a disk-only v0 prefix before a direct cold append', async () => { + const m = meta('released-v0-cold-append', '/work') + const path = rawGenerationPath(root, m.cwd, m.id, 0) + const sourceEvents = oneTurnLog() + const source = Buffer.from(`${JSON.stringify(releasedV0Header(m))}\n${eventLines(sourceEvents, true)}\n`) + await mkdir(dirname(path), { recursive: true }) + await writeFile(path, source) + const suffix: SessionEvent[] = [ + { type: 'turn/start', seq: SessionSeq(6), time: 7, data: { turn: 2 } }, + { type: 'turn/end', seq: SessionSeq(7), time: 8, data: { turn: 2, reason: { kind: 'completed' } } }, + ] + + await ctx.sessionPersistence.append(m.id, suffix) + + expect(await readFile(path)).toEqual(source) + const active = await ctx.sessionPersistence.readRaw(m.id) + expect(active?.meta.version).toBe(SESSION_FORMAT_VERSION) + expect(active?.content.trimEnd().split('\n').map(line => JSON.parse(line) as { type: string }).map(row => row.type)) + .toEqual([ + 'session', + 'turn/start', + 'user/message', + 'step/start', + 'assistant/message', + 'step/end', + 'turn/end', + 'turn/start', + 'turn/end', + ]) + expect((await readdir(dirname(path))).filter(name => name.startsWith('session')).sort()) + .toEqual(['session.jsonl', 'session.v1.jsonl']) + }) + + it('leaves the immutable v0 source unchanged when alpha policy refuses an unknown ignorable event', async () => { + const m = meta('released-v0-unknown', '/work') + const path = rawGenerationPath(root, m.cwd, m.id, 0) + const source = Buffer.from([ + JSON.stringify(releasedV0Header(m)), + JSON.stringify({ type: 'external/info', seq: 0, time: 1, data: {}, ignorable: true }), + '', + ].join('\n')) + await mkdir(dirname(path), { recursive: true }) + await writeFile(path, source) + + const failure = await ctx.sessionPersistence.inspect(m.id).then(() => undefined, (error: unknown) => error as Error) + + expect(failure?.name).toBe('SessionFormatUnsupportedError') + expect(failure?.message).toContain('unknown historical event type "external/info" at seq 0') + expect(failure?.message).toContain('source v0 artifact remains unchanged') + expect(failure?.message).toContain(`(raw log: ${path})`) + expect(await readFile(path)).toEqual(source) + await expect(stat(rawLogPath(root, m.cwd, m.id))).rejects.toMatchObject({ code: 'ENOENT' }) + }) + + it('leaves the immutable v0 source unchanged when a historical relationship cannot migrate', async () => { + const m = meta('released-v0-title-refusal', '/work') + const path = rawGenerationPath(root, m.cwd, m.id, 0) + const source = Buffer.from([ + JSON.stringify(releasedV0Header(m)), + JSON.stringify({ + type: 'user/message', seq: 0, time: 1, surfaceOp: 'append', + data: { + id: 'human', role: 'user', content: [{ type: 'text', text: 'hello' }], source: { kind: 'user' }, + }, + }), + JSON.stringify({ + type: 'session/title', seq: 1, time: 2, + data: { title: 'Title', messageSeqs: [0], source: { kind: 'user' } }, + }), + '', + ].join('\n')) + await mkdir(dirname(path), { recursive: true }) + await writeFile(path, source) + + const failure = await ctx.sessionPersistence.inspect(m.id) + .then(() => undefined, (error: unknown) => error as Error) + + expect(failure?.name).toBe('SessionFormatUnsupportedError') + expect(failure?.message).toContain('session/title 1 messageSeqs must be empty exactly for a user title') + expect(failure?.message).toContain('source v0 artifact remains unchanged') + expect(await readFile(path)).toEqual(source) + await expect(stat(rawLogPath(root, m.cwd, m.id))).rejects.toMatchObject({ code: 'ENOENT' }) + }) + + it('retains a torn v0 source exactly and publishes its current repaired prefix', async () => { + const m = meta('released-v0-torn', '/work') + const path = rawGenerationPath(root, m.cwd, m.id, 0) + const open = { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } } + const source = Buffer.from([ + JSON.stringify(releasedV0Header(m)), + JSON.stringify(open), + '{"type":"assistant/chunk","seq":1', + ].join('\n')) + await mkdir(dirname(path), { recursive: true }) + await writeFile(path, source) + + const raw = await ctx.sessionPersistence.readRaw(m.id) + + expect(await readFile(path)).toEqual(source) + const records = raw?.content.trimEnd().split('\n').map(record => JSON.parse(record) as Record) + expect(records?.map(record => record['type'])).toEqual(['session', 'turn/start', 'turn/end']) + expect(records?.at(-1)).toMatchObject({ + seq: 1, + time: 1, + data: { turn: 1, reason: { kind: 'interrupted' } }, + }) + }) + it('readRaw returns the stored artifact text verbatim with its original filename', async () => { const m = meta('raw-read', '/work') await ctx.sessionPersistence.create(m) await ctx.sessionPersistence.append(m.id, oneTurnLog()) const raw = await ctx.sessionPersistence.readRaw(m.id) expect(raw).toBeDefined() - expect(raw!.filename).toBe('session.jsonl') + expect(raw!.filename).toBe(`session.v${SESSION_FORMAT_VERSION}.jsonl`) expect(raw!.meta.id).toBe(m.id) // Byte-identical to the physical file — never a reconstruction. expect(raw!.content).toBe(await readFile(rawLogPath(root, '/work', m.id), 'utf8')) @@ -418,6 +888,145 @@ describe('JsonlSessionPersistence: durability and crash semantics', () => { expect(statRace.reads).toBe(4) }) + it('reads each current body exactly once and consumes its prefetch once', async () => { + const m = meta('current-single-physical-read', '/work') + await ctx.sessionPersistence.create(m) + await ctx.sessionPersistence.append(m.id, oneTurnLog()) + physicalReadProbe.path = rawLogPath(root, m.cwd, m.id) + + await expect(ctx.sessionPersistence.readRaw(m.id)).resolves.toMatchObject({ meta: m }) + expect(physicalReadProbe.opens).toBe(0) + expect(physicalReadProbe.reads).toBe(1) + + await expect(ctx.sessionPersistence.readRaw(m.id)).resolves.toMatchObject({ meta: m }) + expect(physicalReadProbe.opens).toBe(0) + expect(physicalReadProbe.reads).toBe(2) + }) + + it('restores a current inspection from the same one-shot physical read', async () => { + const m = meta('current-single-inspection-read', '/work') + await ctx.sessionPersistence.create(m) + await ctx.sessionPersistence.append(m.id, oneTurnLog()) + physicalReadProbe.path = rawLogPath(root, m.cwd, m.id) + + await expect(ctx.sessionPersistence.inspect(m.id)).resolves.toMatchObject({ meta: m }) + + expect(physicalReadProbe.opens).toBe(0) + expect(physicalReadProbe.reads).toBe(1) + }) + + it('validates current storage identity once in both fused and direct prefix reads', async () => { + const m = meta('current-single-identity-check', '/work') + await ctx.sessionPersistence.create(m) + await ctx.sessionPersistence.append(m.id, oneTurnLog()) + const backend = ctx.sessionPersistence as JsonlSessionPersistence + const internals = backend as unknown as { + assertStoredIdentity(...args: unknown[]): Promise + rememberCurrentGeneration(...args: unknown[]): void + } + const identity = vi.spyOn(internals, 'assertStoredIdentity') + const remember = vi.spyOn(internals, 'rememberCurrentGeneration') + + await backend.loadCurrentStored(m.id) + expect(identity).toHaveBeenCalledTimes(1) + expect(remember).toHaveBeenCalledTimes(1) + + identity.mockClear() + remember.mockClear() + await backend.loadStored(m.id) + expect(identity).toHaveBeenCalledTimes(1) + expect(remember).toHaveBeenCalledTimes(1) + }) + + it('returns one coherent fused snapshot and observes later writes on the next operation', async () => { + const m = meta('current-prefetch-race', '/work') + await ctx.sessionPersistence.create(m) + await ctx.sessionPersistence.append(m.id, oneTurnLog()) + const path = rawLogPath(root, m.cwd, m.id) + physicalReadProbe.path = path + const backend = ctx.sessionPersistence as JsonlSessionPersistence + + const first = await backend.readCurrentRawStored(m.id) + await appendFile(path, `${JSON.stringify({ + type: 'plugin/test', seq: 6, time: 7, data: null, ignorable: true, + })}\n`) + + expect(first?.content).not.toContain('"plugin/test"') + expect(physicalReadProbe.opens).toBe(0) + expect(physicalReadProbe.reads).toBe(1) + const second = await backend.readCurrentRawStored(m.id) + expect(second?.content).toContain('"plugin/test"') + expect(physicalReadProbe.opens).toBe(0) + expect(physicalReadProbe.reads).toBe(2) + }) + + it('rejects a cancelled fused read before touching the artifact', async () => { + const m = meta('current-prefetch-abort', '/work') + await ctx.sessionPersistence.create(m) + await ctx.sessionPersistence.append(m.id, oneTurnLog()) + const path = rawLogPath(root, m.cwd, m.id) + physicalReadProbe.path = path + const backend = ctx.sessionPersistence as JsonlSessionPersistence + const controller = new AbortController() + + const reason = new Error('cancelled before fused read') + controller.abort(reason) + + await expect(backend.readCurrentRawStored(m.id, controller.signal)).rejects.toBe(reason) + expect(physicalReadProbe.opens + physicalReadProbe.reads).toBe(0) + }) + + it('keeps direct legacy raw reads and absent fused reads well-defined', async () => { + const m = meta('legacy-raw-hook', '/work') + await ctx.sessionPersistence.create(m) + await ctx.sessionPersistence.append(m.id, oneTurnLog()) + const path = rawLogPath(root, m.cwd, m.id) + physicalReadProbe.path = path + const backend = ctx.sessionPersistence as JsonlSessionPersistence + + await expect(backend.readRawStored(m.id)).resolves.toMatchObject({ meta: m }) + expect(physicalReadProbe.opens).toBe(0) + expect(physicalReadProbe.reads).toBe(1) + await expect(backend.readCurrentRawStored(SessionId('absent-fused'))).resolves.toBeUndefined() + await expect(backend.readRawStored(SessionId('absent-legacy'))).resolves.toBeUndefined() + await expect(backend.loadStored(SessionId('absent-prefix'))).resolves.toBeUndefined() + await expect(backend.ensureCurrent(SessionId('absent-ensure'))).resolves.toBeUndefined() + }) + + it('keeps direct legacy raw identity validation fail-closed', async () => { + const requested = SessionId('legacy-raw-mismatch') + const path = rawLogPath(root, undefined, requested) + await mkdir(dirname(path), { recursive: true }) + await writeFile(path, `${JSON.stringify(toHeaderLine(meta('different-id')))}\n`) + const backend = ctx.sessionPersistence as JsonlSessionPersistence + + await expect(backend.readRawStored(requested)).rejects.toThrow('invalid header line') + }) + + it('rejects a current header stored under the reserved v0 filename in direct hooks', async () => { + const m = meta('direct-filename-version-mismatch', '/work') + const path = rawGenerationPath(root, m.cwd, m.id, 0) + await mkdir(dirname(path), { recursive: true }) + await writeFile(path, `${JSON.stringify(toHeaderLine(m))}\n${eventLines(oneTurnLog(), true)}\n`) + const backend = ctx.sessionPersistence as JsonlSessionPersistence + + await expect(backend.loadStored(m.id)) + .rejects.toThrow(/filename identifies v0.*decoded header identifies v1/) + await expect(backend.readRawStored(m.id)) + .rejects.toThrow(/filename identifies v0.*decoded header identifies v1/) + }) + + it('rejects malformed current metadata before exposing a fused snapshot', async () => { + const id = SessionId('malformed-current-header') + const path = rawLogPath(root, undefined, id) + await mkdir(dirname(path), { recursive: true }) + await writeFile(path, `${JSON.stringify({ type: 'session', version: 1 })}\n`) + const backend = ctx.sessionPersistence as JsonlSessionPersistence + + await expect(backend.readCurrentRawStored(id)) + .rejects.toThrow('corrupt session log: invalid current header') + }) + it('keeps the same location on resume and gives a fork its own location', async () => { const parent = meta('location-parent', '/work') const parentLocation = ctx.sessionPersistence.locate(parent) @@ -516,16 +1125,21 @@ describe('JsonlSessionPersistence: durability and crash semantics', () => { await ctx.sessionPersistence.append(m.id, oneTurnLog()) const persistence = ctx.sessionPersistence as JsonlSessionPersistence const internals = persistence as unknown as { - findLog(id: SessionId, signal?: AbortSignal): Promise + findLog(id: SessionId, signal?: AbortSignal): Promise<{ + sourcePath: string + sourceVersion: number + currentPath: string + } | undefined> } const path = rawLogPath(root, m.cwd, m.id) - const findLog = vi.spyOn(internals, 'findLog').mockResolvedValue(path) + const resolved = { sourcePath: path, sourceVersion: SESSION_FORMAT_VERSION, currentPath: path } + const findLog = vi.spyOn(internals, 'findLog').mockResolvedValue(resolved) await rm(path) expect(await persistence.readStoredRevision(m.id)).toBeUndefined() const invalidPath = `${path}\0` - findLog.mockResolvedValue(invalidPath) + findLog.mockResolvedValue({ ...resolved, sourcePath: invalidPath }) await expect(persistence.readStoredRevision(m.id)).rejects.toMatchObject({ code: 'ERR_INVALID_ARG_VALUE', }) @@ -534,7 +1148,7 @@ describe('JsonlSessionPersistence: durability and crash semantics', () => { const controller = new AbortController() findLog.mockImplementation(async () => { controller.abort(reason) - return invalidPath + return { ...resolved, sourcePath: invalidPath } }) await expect(persistence.readStoredRevision(m.id, controller.signal)).rejects.toBe(reason) }) @@ -605,11 +1219,18 @@ describe('JsonlSessionPersistence: durability and crash semantics', () => { await ctx.sessionPersistence.create(m) await ctx.sessionPersistence.append(m.id, oneTurnLog()) const persistence = ctx.sessionPersistence as unknown as { - listArtifacts(signal?: AbortSignal): Promise> + listArtifacts(signal?: AbortSignal): Promise> } + const path = rawLogPath(root, m.cwd, m.id) const discovery = vi.spyOn(persistence, 'listArtifacts').mockResolvedValue([{ - header: m, - path: rawLogPath(root, m.cwd, m.id), + listing: { + status: 'current', + header: m, + storedVersion: SESSION_FORMAT_VERSION, + targetVersion: SESSION_FORMAT_VERSION, + location: { kind: 'jsonl', path }, + }, + path, }]) const reason = new Error('JSONL snapshot stat cancelled') const controller = new AbortController() @@ -622,25 +1243,26 @@ describe('JsonlSessionPersistence: durability and crash semantics', () => { it('rejects a stored v0 log containing a legacy request/header-delta event', async () => { const m = meta('legacy-header-delta', '/legacy') - const path = rawLogPath(root, m.cwd, m.id) + const path = rawGenerationPath(root, m.cwd, m.id, 0) await mkdir(sessionDir(root, m.cwd, m.id), { recursive: true }) await writeFile(path, [ - JSON.stringify(toHeaderLine(m)), + JSON.stringify(releasedV0Header(m)), JSON.stringify({ type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }), JSON.stringify({ type: 'request/header-delta', seq: 1, time: 2, data: { config: { model: 'legacy' } } }), JSON.stringify({ type: 'turn/end', seq: 2, time: 3, data: { turn: 1, reason: { kind: 'completed' } } }), '', ].join('\n')) - await expect(ctx.sessionPersistence.load(m.id)).rejects.toThrow(/unsupported legacy request\/header-delta event at seq 1/) + await expect(ctx.sessionPersistence.load(m.id)) + .rejects.toThrow(/unsupported legacy request\/header-delta event at seq 1.*source v0 artifact remains unchanged/) }) it('rejects a stored v0 full header carrying the legacy fallback reason', async () => { const m = meta('legacy-header-fallback', '/legacy') - const path = rawLogPath(root, m.cwd, m.id) + const path = rawGenerationPath(root, m.cwd, m.id, 0) await mkdir(sessionDir(root, m.cwd, m.id), { recursive: true }) await writeFile(path, [ - JSON.stringify(toHeaderLine(m)), + JSON.stringify(releasedV0Header(m)), JSON.stringify({ type: 'request/header', seq: 0, @@ -651,7 +1273,7 @@ describe('JsonlSessionPersistence: durability and crash semantics', () => { ].join('\n')) await expect(ctx.sessionPersistence.load(m.id)) - .rejects.toThrow(/unsupported legacy request\/header reason "fallback" at seq 0/) + .rejects.toThrow(/unsupported request\/header reason "fallback" at seq 0.*source v0 artifact remains unchanged/) }) it('persists a forked child seed through the existing session write path', async () => { @@ -849,6 +1471,24 @@ describe('JsonlSessionPersistence: durability and crash semantics', () => { expect(await readFile(bPath)).toEqual(beforeB) }) + it('rejects a mismatched v0 header before publishing a migrated generation', async () => { + const requested = meta('identity-v0-requested', '/same') + const path = rawGenerationPath(root, requested.cwd, requested.id, 0) + const source = Buffer.from([ + JSON.stringify({ ...releasedV0Header(requested), id: 'identity-v0-other' }), + eventLines(oneTurnLog(), true), + '', + ].join('\n')) + await mkdir(dirname(path), { recursive: true }) + await writeFile(path, source) + + await expect(ctx.sessionPersistence.load(requested.id)) + .rejects.toThrow(/requested id "identity-v0-requested" does not match header id "identity-v0-other"/) + expect(await readFile(path)).toEqual(source) + await expect(readFile(rawLogPath(root, requested.cwd, requested.id))) + .rejects.toMatchObject({ code: 'ENOENT' }) + }) + it('rejects a re-append of an already-stored seq', async () => { const m = meta('reappend') await ctx.sessionPersistence.create(m) @@ -858,7 +1498,7 @@ describe('JsonlSessionPersistence: durability and crash semantics', () => { it('path-traversal session ids are neutralized (no escape from root)', async () => { const evil = SessionId('../../etc/pwn') - const m = { version: 0, id: evil, createdAt: 1, isSeeded: false } + const m: SessionHeader = { version: SESSION_FORMAT_VERSION, id: evil, createdAt: 1, isSeeded: false } await ctx.sessionPersistence.create(m) await ctx.sessionPersistence.append(evil, oneTurnLog()) // The file lives UNDER root, not at ../../etc. @@ -989,7 +1629,7 @@ describe('JsonlSessionPersistence: scanLog unit', () => { ])('rejects a session header with a %s createdAt', (_label, createdAt) => { const log = JSON.stringify({ type: 'session', - version: 0, + version: SESSION_FORMAT_VERSION, id: 'invalid-created-at', createdAt, delegationDepth: 0, @@ -998,7 +1638,7 @@ describe('JsonlSessionPersistence: scanLog unit', () => { }) it('rejects a session header with negative-zero createdAt', () => { - const log = '{"type":"session","version":0,"id":"invalid-created-at","createdAt":-0,"delegationDepth":0}\n' + const log = `{"type":"session","version":${SESSION_FORMAT_VERSION},"id":"invalid-created-at","createdAt":-0,"delegationDepth":0}\n` expect(() => scanLog(Buffer.from(log))).toThrow(/session header/) }) @@ -1010,7 +1650,7 @@ describe('JsonlSessionPersistence: scanLog unit', () => { ])('rejects a session header with %s delegationDepth', (_label, delegationDepth) => { const log = JSON.stringify({ type: 'session', - version: 0, + version: SESSION_FORMAT_VERSION, id: 'invalid-depth', createdAt: 1, ...delegationDepth === undefined ? {} : { delegationDepth }, @@ -1019,13 +1659,13 @@ describe('JsonlSessionPersistence: scanLog unit', () => { }) it('rejects a session header with negative-zero delegationDepth', () => { - const log = '{"type":"session","version":0,"id":"invalid-depth","createdAt":1,"delegationDepth":-0}\n' + const log = `{"type":"session","version":${SESSION_FORMAT_VERSION},"id":"invalid-depth","createdAt":1,"delegationDepth":-0}\n` expect(() => scanLog(Buffer.from(log))).toThrow(/session header/) }) it('round-trips the agent preset a session was composed from', () => { const line = toHeaderLine({ - version: 0, + version: SESSION_FORMAT_VERSION, id: SessionId('composed'), createdAt: 1, isSeeded: false, @@ -1040,14 +1680,14 @@ describe('JsonlSessionPersistence: scanLog unit', () => { }) it('rejects a session header whose agentPreset is not a string', () => { - const log = '{"type":"session","version":0,"id":"bad-preset","createdAt":1,"delegationDepth":0,"agentPreset":7}\n' + const log = `{"type":"session","version":${SESSION_FORMAT_VERSION},"id":"bad-preset","createdAt":1,"delegationDepth":0,"agentPreset":7}\n` expect(() => scanLog(Buffer.from(log))).toThrow(/session header/) }) it('a seq gap after the last turn/end bounds the preserved tail (torn fragment tolerated)', () => { const log = [ - JSON.stringify({ type: 'session', version: 0, id: 'g', createdAt: 1, delegationDepth: 0 }), + JSON.stringify({ type: 'session', version: SESSION_FORMAT_VERSION, id: 'g', createdAt: 1, delegationDepth: 0 }), JSON.stringify({ type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }), JSON.stringify({ type: 'step/start', seq: 2, time: 2, data: { turn: 1, step: 1 } }), // gap: missing seq 1 ].join('\n') + '\n' @@ -1059,7 +1699,7 @@ describe('JsonlSessionPersistence: scanLog unit', () => { it('rejects a seq gap BEFORE a later committed turn/end (committed data damaged)', () => { const log = [ - JSON.stringify({ type: 'session', version: 0, id: 'g2', createdAt: 1, delegationDepth: 0 }), + JSON.stringify({ type: 'session', version: SESSION_FORMAT_VERSION, id: 'g2', createdAt: 1, delegationDepth: 0 }), JSON.stringify({ type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }), JSON.stringify({ type: 'step/start', seq: 2, time: 2, data: { turn: 1, step: 1 } }), // gap: missing seq 1 JSON.stringify({ type: 'turn/end', seq: 3, time: 3, data: { turn: 1, reason: { kind: 'completed' } } }), @@ -1077,7 +1717,7 @@ describe('JsonlSessionPersistence: scanLog unit', () => { ] for (const record of corruptRecords) { const log = [ - JSON.stringify({ type: 'session', version: 0, id: 'c', createdAt: 1, delegationDepth: 0 }), + JSON.stringify({ type: 'session', version: SESSION_FORMAT_VERSION, id: 'c', createdAt: 1, delegationDepth: 0 }), record, JSON.stringify({ type: 'turn/end', seq: 1, time: 2, data: { turn: 1, reason: { kind: 'completed' } } }), ].join('\n') + '\n' @@ -1086,7 +1726,9 @@ describe('JsonlSessionPersistence: scanLog unit', () => { }) it('a header-only log (no event lines at all) preserves nothing — committedBytes is the header', () => { - const log = JSON.stringify({ type: 'session', version: 0, id: 'h0', createdAt: 1, delegationDepth: 0 }) + '\n' + const log = JSON.stringify({ + type: 'session', version: SESSION_FORMAT_VERSION, id: 'h0', createdAt: 1, delegationDepth: 0, + }) + '\n' const scanned = scanLog(Buffer.from(log)) expect(scanned.events).toEqual([]) // committedBytes falls back to the header line's end (no preserved events). @@ -1095,7 +1737,7 @@ describe('JsonlSessionPersistence: scanLog unit', () => { it('a corrupt line after the last turn/end bounds the preserved tail', () => { const log = [ - JSON.stringify({ type: 'session', version: 0, id: 'c2', createdAt: 1, delegationDepth: 0 }), + JSON.stringify({ type: 'session', version: SESSION_FORMAT_VERSION, id: 'c2', createdAt: 1, delegationDepth: 0 }), JSON.stringify({ type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }), '{not json', // corrupt crash fragment, no turn/end committed ].join('\n') + '\n' @@ -1106,7 +1748,7 @@ describe('JsonlSessionPersistence: scanLog unit', () => { it('tolerates a seq gap AFTER a turn/end (uncommitted tail)', () => { const log = [ - JSON.stringify({ type: 'session', version: 0, id: 't', createdAt: 1, delegationDepth: 0 }), + JSON.stringify({ type: 'session', version: SESSION_FORMAT_VERSION, id: 't', createdAt: 1, delegationDepth: 0 }), JSON.stringify({ type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }), JSON.stringify({ type: 'turn/end', seq: 1, time: 2, data: { turn: 1, reason: { kind: 'completed' } } }), JSON.stringify({ type: 'step/start', seq: 9, time: 3, data: { turn: 2, step: 1 } }), // gap in uncommitted tail @@ -1203,7 +1845,9 @@ describe('JsonlSessionPersistence: default packed chunk rows', () => { // file, hand-planted so this packed-config backend adopts it on load). await mkdir(sessionDir(root, '/work', m.id), { recursive: true }) await writeFile(rawLogPath(root, '/work', m.id), [ - JSON.stringify({ type: 'session', version: 0, id: 'mixed', createdAt: 1000, cwd: '/work', delegationDepth: 0 }), + JSON.stringify({ + type: 'session', version: SESSION_FORMAT_VERSION, id: 'mixed', createdAt: 1000, cwd: '/work', delegationDepth: 0, + }), ...log.map(e => JSON.stringify(e)), ].join('\n') + '\n') // Adopt the stored log (cursor = stored length), then append a second turn @@ -1227,7 +1871,7 @@ describe('JsonlSessionPersistence: default packed chunk rows', () => { it('scanLog: a packed row advances the seq cursor by its whole run', () => { const logText = [ - JSON.stringify({ type: 'session', version: 0, id: 'rows', createdAt: 1, delegationDepth: 0 }), + JSON.stringify({ type: 'session', version: SESSION_FORMAT_VERSION, id: 'rows', createdAt: 1, delegationDepth: 0 }), JSON.stringify({ type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }), JSON.stringify({ type: 'text-chunks', seq0: 1, time0: 2, data: { turn: 1, step: 1, index: 0, dt: [1, 1], texts: ['a', 'b', 'c'] } }), JSON.stringify({ type: 'turn/end', seq: 4, time: 5, data: { turn: 1, reason: { kind: 'completed' } } }), @@ -1239,7 +1883,7 @@ describe('JsonlSessionPersistence: default packed chunk rows', () => { it('scanLog: a malformed packed row in the committed region rejects like corrupt JSON', () => { const logText = [ - JSON.stringify({ type: 'session', version: 0, id: 'bad-row', createdAt: 1, delegationDepth: 0 }), + JSON.stringify({ type: 'session', version: SESSION_FORMAT_VERSION, id: 'bad-row', createdAt: 1, delegationDepth: 0 }), // dt arity mismatch — row validation throws, so the line is a committed hole. JSON.stringify({ type: 'text-chunks', seq0: 0, time0: 1, data: { turn: 1, step: 1, index: 0, dt: [], texts: ['a', 'b'] } }), JSON.stringify({ type: 'turn/end', seq: 2, time: 3, data: { turn: 1, reason: { kind: 'completed' } } }), @@ -1249,7 +1893,7 @@ describe('JsonlSessionPersistence: default packed chunk rows', () => { it('scanLog: a packed row with a mid-run seq gap after the last turn/end drops the whole row', () => { const logText = [ - JSON.stringify({ type: 'session', version: 0, id: 'row-gap', createdAt: 1, delegationDepth: 0 }), + JSON.stringify({ type: 'session', version: SESSION_FORMAT_VERSION, id: 'row-gap', createdAt: 1, delegationDepth: 0 }), JSON.stringify({ type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }), // seq0 skips 1 — the run's first member is already a gap; no turn/end follows. JSON.stringify({ type: 'text-chunks', seq0: 2, time0: 2, data: { turn: 1, step: 1, index: 0, dt: [1, 1], texts: ['a', 'b', 'c'] } }), @@ -1316,7 +1960,7 @@ describe('JsonlSessionPersistence: edge cases', () => { await ctx.sessionPersistence.create(meta('p3')) // no cwd → _no-cwd project directory await ctx.sessionPersistence.append(SessionId('p3'), oneTurnLog()) - const ids = (await ctx.sessionPersistence.list()).map(x => x.id).sort() + const ids = listedIds(await ctx.sessionPersistence.list()).sort() expect(ids).toEqual(['p1', 'p2', 'p3']) }) @@ -1333,7 +1977,7 @@ describe('JsonlSessionPersistence: edge cases', () => { encodeSegment(first.id), encodeSegment(second.id), ])) - expect((await ctx.sessionPersistence.list()).map(header => header.id).sort()) + expect(listedIds(await ctx.sessionPersistence.list()).sort()) .toEqual([first.id, second.id].sort()) }) @@ -1350,8 +1994,11 @@ describe('JsonlSessionPersistence: edge cases', () => { await writeFile(join(projectDir(root, m.cwd), 'README'), 'project metadata\n') await mkdir(join(projectDir(root, m.cwd), 'reserved-session'), { recursive: true }) - expect(await readdir(dir)).toEqual(expect.arrayContaining(['metadata.json', 'session.jsonl'])) - expect((await ctx.sessionPersistence.list()).map(header => header.id)).toContain(m.id) + expect(await readdir(dir)).toEqual(expect.arrayContaining([ + 'metadata.json', + generationLogFilename(SESSION_FORMAT_VERSION, 'none'), + ])) + expect(listedIds(await ctx.sessionPersistence.list())).toContain(m.id) expect((await ctx.sessionPersistence.load(m.id)).events).toEqual(oneTurnLog()) }) @@ -1380,7 +2027,7 @@ describe('JsonlSessionPersistence: edge cases', () => { await expect(ctx.sessionPersistence.load(m.id)).rejects.toThrow(/unsupported flat-file layout/) }) - it('list skips empty and non-header session logs (metadata-only read)', async () => { + it('list isolates empty and non-header session logs as malformed descriptors', async () => { // A real session… await ctx.sessionPersistence.create(meta('real', '/p')) await ctx.sessionPersistence.append(SessionId('real'), oneTurnLog()) @@ -1396,18 +2043,21 @@ describe('JsonlSessionPersistence: edge cases', () => { await writeFile(path, content) } - const ids = (await ctx.sessionPersistence.list()).map(x => x.id).sort() - expect(ids).toEqual(['real']) + const listings = await ctx.sessionPersistence.list() + expect(listedIds(listings)).toEqual(['real']) + expect(listings.filter(listing => listing.status === 'malformed')).toHaveLength(3) }) it('list reads a header line longer than the 8KB read chunk', async () => { - // A tolerated extra field makes this valid header exceed the 8192-byte read buffer, proving + // A long valid field makes this header exceed the 8192-byte read buffer, proving // `readFirstLine` accumulates chunks before `list()` parses it. const id = SessionId('big') await mkdir(sessionDir(root, undefined, id), { recursive: true }) - const bigHeader = JSON.stringify({ type: 'session', version: 0, id: 'big', createdAt: 1, delegationDepth: 0, pad: 'x'.repeat(9000) }) - await writeFile(rawLogPath(root, undefined, id), bigHeader + '\n') - const ids = (await ctx.sessionPersistence.list()).map(x => x.id) + const bigHeader = JSON.stringify({ + type: 'session', version: 0, id: 'big', createdAt: 1, delegationDepth: 0, agentPreset: 'x'.repeat(9000), + }) + await writeFile(rawGenerationPath(root, undefined, id, 0), bigHeader + '\n') + const ids = listedIds(await ctx.sessionPersistence.list()) expect(ids).toContain('big') }) @@ -1423,7 +2073,10 @@ describe('JsonlSessionPersistence: edge cases', () => { await ctx.sessionPersistence.append(m.id, oneTurnLog()) await rewriteHeader(rawLogPath(root, m.cwd, m.id), (header) => { header.cwd = '/elsewhere' }) - await expect(ctx.sessionPersistence.list()).rejects.toThrow(/and cwd identify/) + const [listing] = await ctx.sessionPersistence.list() + expect(listing?.status).toBe('malformed') + if (listing?.status !== 'malformed') throw new Error('expected malformed listing') + expect(listing.reason).toMatch(/and cwd identify/) }) it('accepts an alternate project path only when it identifies the same physical log', async () => { @@ -1440,7 +2093,7 @@ describe('JsonlSessionPersistence: edge cases', () => { await rewriteHeader(path, (header) => { header.cwd = aliasCwd }) expect((await ctx.sessionPersistence.load(m.id)).meta.cwd).toBe(aliasCwd) - expect((await ctx.sessionPersistence.list()).map(header => header.id)).toContain(m.id) + expect(listedIds(await ctx.sessionPersistence.list())).toContain(m.id) }) it('list rejects a session header whose id cannot name a storage path', async () => { @@ -1450,7 +2103,29 @@ describe('JsonlSessionPersistence: edge cases', () => { type: 'session', version: 0, id: '', createdAt: 1, delegationDepth: 0, }) + '\n') - await expect(ctx.sessionPersistence.list()).rejects.toThrow(/header id cannot name a storage path/) + const [listing] = await ctx.sessionPersistence.list() + expect(listing?.status).toBe('malformed') + if (listing?.status !== 'malformed') throw new Error('expected malformed listing') + expect(listing.reason).toMatch(/header id cannot name a storage path/) + await expect(ctx.sessionPersistence.load(SessionId('invalid-id'))) + .rejects.toThrow(/requested id "invalid-id" does not match header id ""/) + }) + + it('refuses a malformed historical header before interpreting its body', async () => { + const id = SessionId('malformed-historical-header') + const path = rawGenerationPath(root, undefined, id, 0) + await mkdir(dirname(path), { recursive: true }) + await writeFile(path, `${JSON.stringify({ + type: 'not-a-session', + version: 0, + id, + createdAt: 1, + delegationDepth: 0, + })}\n{not-json}`) + + await expect(ctx.sessionPersistence.load(id)) + .rejects.toThrow('expected released v0 physical Session header') + expect(await readFile(path, 'utf8')).toContain('{not-json}') }) it('load and list reject one id materialized in multiple project directories', async () => { @@ -1463,7 +2138,13 @@ describe('JsonlSessionPersistence: edge cases', () => { } await expect(ctx.sessionPersistence.load(id)).rejects.toThrow(/appears in multiple project directories/) - await expect(ctx.sessionPersistence.list()).rejects.toThrow(/appears in multiple project directories/) + const listings = await ctx.sessionPersistence.list() + expect(listings).toHaveLength(2) + expect(listings.every(listing => listing.status === 'malformed')).toBe(true) + for (const listing of listings) { + if (listing.status !== 'malformed') throw new Error('expected malformed listing') + expect(listing.reason).toMatch(/appears in multiple project directories/) + } }) it('a DIFFERENT live session object reusing a disposed id gets its own init (no stale cache)', async () => { @@ -1588,6 +2269,27 @@ describe('JsonlSessionPersistence: edge cases', () => { await expect(backend.exists(join(blocker, 'child.jsonl'))).rejects.toThrow(/ENOTDIR/) }) + it('per-id cold generation scan surfaces a Session-directory read failure', async () => { + const id = SessionId('generation-directory-failure') + const dir = sessionDir(root, undefined, id) + await mkdir(dir, { recursive: true }) + const reason = new Error('simulated Session-directory read failure') + directoryReadFailure.path = dir + directoryReadFailure.error = reason + + await expect(ctx.sessionPersistence.inspect(id)).rejects.toBe(reason) + }) + + it('append surfaces a Session-directory read failure while checking the opposite encoding', async () => { + const m = meta('opposite-generation-directory-failure', '/x') + await ctx.sessionPersistence.create(m) + const reason = Object.assign(new Error('simulated opposite-generation read failure'), { code: 'EACCES' }) + directoryReadFailure.path = sessionDir(root, m.cwd, m.id) + directoryReadFailure.error = reason + + await expect(ctx.sessionPersistence.append(m.id, oneTurnLog())).rejects.toBe(reason) + }) + it('materialization surfaces a project-directory storage fault', async () => { const cwd = '/x' const ctx2 = new Context() @@ -1708,7 +2410,7 @@ describe('JsonlSessionPersistence: edge cases', () => { circ.self = circ await expect(ctx.sessionPersistence.append(m.id, bad(circ))).rejects.toThrow(/non-JSON-serializable/) // The session was never materialized by any of the rejected appends. - expect((await ctx.sessionPersistence.list()).map(h => h.id)).not.toContain(m.id) + expect(listedIds(await ctx.sessionPersistence.list())).not.toContain(m.id) }) it('accepts well-formed JSON values (null, booleans, nested arrays/objects)', async () => { @@ -1718,7 +2420,7 @@ describe('JsonlSessionPersistence: edge cases', () => { content: [{ type: 'text', text: 'x' }], source: { kind: 'user' }, extra: { a: null, b: true, c: [1, 2, { d: 'nested' }] }, }) }] as unknown as SessionEvent[] await ctx.sessionPersistence.append(m.id, ev) - expect((await ctx.sessionPersistence.list()).map(h => h.id)).toContain(m.id) + expect(listedIds(await ctx.sessionPersistence.list())).toContain(m.id) }) it('Session.append rejects a non-serializable event at the source (never enters the log)', () => { diff --git a/packages/session/session-persistence-jsonl/tests/zstd.spec.ts b/packages/session/session-persistence-jsonl/tests/zstd.spec.ts index 9d311d2b39..f6ecc15477 100644 --- a/packages/session/session-persistence-jsonl/tests/zstd.spec.ts +++ b/packages/session/session-persistence-jsonl/tests/zstd.spec.ts @@ -5,10 +5,11 @@ import type { FileHandle } from 'node:fs/promises' import { tmpdir } from 'node:os' import { join } from 'node:path' import { performance } from 'node:perf_hooks' -import SessionStore, { SessionId, SessionLogOffset, SessionSeq } from '@deepseek-ai/dsh-session' -import type { SessionEvent } from '@deepseek-ai/dsh-session' +import SessionStore, { SESSION_FORMAT_VERSION, SessionId, SessionLogOffset, SessionSeq } from '@deepseek-ai/dsh-session' +import type { SessionEvent, SessionHeader } from '@deepseek-ai/dsh-session' import JsonlSessionPersistence from '@deepseek-ai/dsh-session-persistence-jsonl' -import { logPath, scanLog, sessionDir, toHeaderLine, type JsonlCompression } from '../src/format.ts' +import type { SessionPersistenceListing } from '@deepseek-ai/dsh-session-persistence' +import { generationLogPath, logPath, scanLog, sessionDir, toHeaderLine, type JsonlCompression } from '../src/format.ts' import { compressZstdFrame, createZstdFrameDecoder, decompressZstdFrame, decompressZstdPrefix, scanZstdFrames, type ZstdFrameDecoder, @@ -18,6 +19,29 @@ import { PublicZstdFrameDecoder } from '../src/zstd-public-decoder.ts' import { runPersistenceContract, meta, oneTurnLog } from '../../session-persistence/tests/contract.ts' import { runCoordinatorContract, type CoordinatorFixture } from '../../session-persistence/tests/coordinator-contract.ts' +const physicalReadProbe = vi.hoisted(() => ({ + path: undefined as string | undefined, + opens: 0, + reads: 0, +})) + +vi.mock('node:fs/promises', async (importOriginal) => { + const actual = await importOriginal() + return { + ...actual, + open: async (...args: Parameters) => { + if (typeof args[0] === 'string' && args[0] === physicalReadProbe.path && args[1] === 'r') { + physicalReadProbe.opens += 1 + } + return actual.open(...args) + }, + readFile: (async (...args: Parameters) => { + if (typeof args[0] === 'string' && args[0] === physicalReadProbe.path) physicalReadProbe.reads += 1 + return actual.readFile(...args) + }) as typeof actual.readFile, + } +}) + const MAGIC = Buffer.from([0x28, 0xB5, 0x2F, 0xFD]) const roots: string[] = [] const contexts: Context[] = [] @@ -51,6 +75,11 @@ async function mount(root: string, compression?: JsonlCompression): Promise + listing.status === 'current' || listing.status === 'migration-required' ? [listing.header] : []) +} + async function decodeCompleteFrames(buffer: Buffer): Promise { const { frames, tornStart } = scanZstdFrames(buffer) expect(tornStart).toBeUndefined() @@ -61,6 +90,14 @@ async function decodeCompleteFrames(buffer: Buffer): Promise { return Buffer.concat(plaintext) } +function releasedV0Header(header: ReturnType): Record { + return { ...toHeaderLine(header), version: 0 } +} + +function zstdGenerationPath(root: string, header: ReturnType, version: number): string { + return generationLogPath(root, header.cwd, header.id, version, 'zstd') +} + async function tornFrame( plaintext: string, accepts: (decoded: string) => boolean, @@ -108,6 +145,9 @@ function emptyStructuralFrame(descriptor: number): Buffer { afterEach(async () => { vi.restoreAllMocks() + physicalReadProbe.path = undefined + physicalReadProbe.opens = 0 + physicalReadProbe.reads = 0 for (const ctx of contexts.splice(0).reverse()) await ctx.fiber.dispose() for (const root of roots.splice(0)) await rm(root, { recursive: true, force: true }) }) @@ -343,9 +383,40 @@ describe('JsonlSessionPersistence: default Zstandard encoding', () => { ) await writeFile(path, Buffer.concat([headerFrame, Buffer.from('invalid event frame')])) - await expect(ctx.sessionPersistence.list()).resolves.toEqual([ - expect.objectContaining({ id: header.id, isSeeded: true }), - ]) + const [listing] = await ctx.sessionPersistence.list() + if (listing?.status !== 'current' && listing?.status !== 'migration-required') { + throw new Error('expected readable listing') + } + expect(listing.header).toMatchObject({ id: header.id, isSeeded: true }) + }) + + it('keeps compressed list and fused body reads aligned on malformed current headers', async () => { + const root = await freshRoot() + const ctx = await mount(root) + const fixtures = [ + ['extra', { unexpected: true }], + ['cwd-type', { cwd: 7 }], + ['cwd-relative', { cwd: 'relative/path' }], + ['parent-type', { parentSession: 7 }], + ] as const + for (const [name, change] of fixtures) { + const id = SessionId(`zstd-malformed-current-${name}`) + const path = logPath(root, '/work', id, 'zstd') + await mkdir(sessionDir(root, '/work', id), { recursive: true }) + await writeFile(path, await compressZstdFrame( + `${JSON.stringify({ ...toHeaderLine(meta(id, '/work')), ...change })}\n`, + )) + } + + const listings = await ctx.sessionPersistence.list() + + expect(listings).toHaveLength(fixtures.length) + expect(listings.every(listing => listing.status === 'malformed')).toBe(true) + for (const [name] of fixtures) { + const id = SessionId(`zstd-malformed-current-${name}`) + await expect(ctx.sessionPersistence.inspect(id)).rejects.toThrow(/invalid current header|cwd must be absolute/) + await expect(ctx.sessionPersistence.readRaw(id)).rejects.toThrow(/invalid current header|cwd must be absolute/) + } }) it('materializes an explicitly durable empty session as one header frame', async () => { @@ -399,7 +470,7 @@ describe('JsonlSessionPersistence: default Zstandard encoding', () => { const raw = await ctx.sessionPersistence.readRaw(header.id) expect(raw).toBeDefined() // The logical name drops the physical encoding suffix. - expect(raw!.filename).toBe('session.jsonl') + expect(raw!.filename).toBe(`session.v${SESSION_FORMAT_VERSION}.jsonl`) expect(raw!.meta.id).toBe(header.id) expect(raw!.content).toBe([ JSON.stringify(toHeaderLine(header)), @@ -410,6 +481,146 @@ describe('JsonlSessionPersistence: default Zstandard encoding', () => { expect(scanned.events.map(event => event.type)).toEqual(oneTurnLog().map(event => event.type)) }) + it('reads each current compressed body exactly once', async () => { + const root = await freshRoot() + const ctx = await mount(root) + const header = meta('zstd-current-single-read', '/work') + await ctx.sessionPersistence.create(header) + await ctx.sessionPersistence.append(header.id, oneTurnLog()) + physicalReadProbe.path = logPath(root, header.cwd, header.id, 'zstd') + + await expect(ctx.sessionPersistence.readRaw(header.id)).resolves.toMatchObject({ meta: header }) + expect(physicalReadProbe.reads).toBe(1) + expect(physicalReadProbe.opens).toBe(0) + + await expect(ctx.sessionPersistence.readRaw(header.id)).resolves.toMatchObject({ meta: header }) + expect(physicalReadProbe.reads).toBe(2) + expect(physicalReadProbe.opens).toBe(0) + }) + + it('closes standalone current classification and retains the legacy raw hook', async () => { + const root = await freshRoot() + const ctx = await mount(root) + const header = meta('zstd-standalone-current', '/work') + await ctx.sessionPersistence.create(header) + await ctx.sessionPersistence.append(header.id, oneTurnLog()) + const backend = ctx.sessionPersistence as JsonlSessionPersistence + + await expect(backend.ensureCurrent(header.id, new AbortController().signal)).resolves.toBeUndefined() + await expect(backend.readRawStored(header.id)).resolves.toMatchObject({ meta: header }) + }) + + it('closes current Zstandard classification when latest metadata is malformed', async () => { + const root = await freshRoot() + const ctx = await mount(root) + const id = SessionId('zstd-malformed-current-header') + const path = logPath(root, undefined, id, 'zstd') + await mkdir(sessionDir(root, undefined, id), { recursive: true }) + await writeFile(path, Buffer.concat([ + await compressZstdFrame(`${JSON.stringify({ type: 'session', version: 1 })}\n`), + await compressZstdFrame(`${JSON.stringify({ type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } })}\n`), + ])) + const backend = ctx.sessionPersistence as JsonlSessionPersistence + + await expect(backend.readCurrentRawStored(id)) + .rejects.toThrow('corrupt session log: invalid current header') + }) + + it('closes the current decoder when strict header translation throws', async () => { + const root = await freshRoot() + const id = SessionId('zstd-retired-header-decoder') + const path = logPath(root, undefined, id, 'zstd') + await mkdir(sessionDir(root, undefined, id), { recursive: true }) + await writeFile(path, await compressZstdFrame(`${JSON.stringify({ + type: 'session', + version: 1, + id, + createdAt: 1, + delegationDepth: 0, + sandboxMode: 'read-only', + })}\n`)) + let closed = 0 + vi.resetModules() + vi.doMock('../src/zstd.ts', async (importOriginal) => { + const actual = await importOriginal() + return { + ...actual, + createZstdFrameDecoder(): ZstdFrameDecoder { + const inner = actual.createZstdFrameDecoder() + return { + *decode(source, frames) { + try { + yield* inner.decode(source, frames) + } finally { + closed += 1 + inner.close() + } + }, + close(): void { inner.close() }, + } + }, + } + }) + const ctx = new Context() + contexts.push(ctx) + await ctx.plugin(SessionStore) + try { + const { default: InstrumentedJsonlSessionPersistence } = await import('../src/index.ts') + await ctx.plugin(InstrumentedJsonlSessionPersistence, { root, compression: 'zstd' }) + + await expect(ctx.sessionPersistence.readRaw(id)).rejects.toThrow(/retired policy baseline fields/) + expect(closed).toBe(1) + } finally { + vi.doUnmock('../src/zstd.ts') + vi.resetModules() + } + }) + + it('migrates a released v0 compressed artifact before raw export and keeps its exact frames', async () => { + const root = await freshRoot() + const ctx = await mount(root) + const current = meta('zstd-v0-on-read', '/work') + const path = zstdGenerationPath(root, current, 0) + const currentPath = logPath(root, current.cwd, current.id, 'zstd') + const v0Header = releasedV0Header(current) + const source = Buffer.concat([ + await compressZstdFrame(`${JSON.stringify(v0Header)}\n`), + await compressZstdFrame(`${oneTurnLog().map(event => JSON.stringify(event)).join('\n')}\n`), + ]) + await mkdir(sessionDir(root, current.cwd, current.id), { recursive: true }) + await writeFile(path, source) + + await expect(ctx.sessionPersistence.list()).resolves.toEqual([ + expect.objectContaining({ status: 'migration-required', storedVersion: 0, targetVersion: 1 }), + ]) + expect(await readFile(path)).toEqual(source) + + const raw = await ctx.sessionPersistence.readRaw(current.id) + + expect(raw?.meta.version).toBe(1) + expect(JSON.parse(raw?.content.split('\n')[0] as string)).toMatchObject({ version: 1 }) + expect(await readFile(path)).toEqual(source) + expect(JSON.parse((await decodeCompleteFrames(await readFile(currentPath))).toString().split('\n')[0] as string)) + .toMatchObject({ version: 1 }) + }) + + it('restores a migrated compressed prefix when publication validation consumed its first decoder', async () => { + const root = await freshRoot() + const ctx = await mount(root) + const current = meta('zstd-v0-prefix-on-read', '/work') + const path = zstdGenerationPath(root, current, 0) + await mkdir(sessionDir(root, current.cwd, current.id), { recursive: true }) + await writeFile(path, Buffer.concat([ + await compressZstdFrame(`${JSON.stringify(releasedV0Header(current))}\n`), + await compressZstdFrame(`${oneTurnLog().map(event => JSON.stringify(event)).join('\n')}\n`), + ])) + + await expect(ctx.sessionPersistence.inspect(current.id)).resolves.toMatchObject({ + meta: { id: current.id, version: 1 }, + events: oneTurnLog(), + }) + }) + it('readRaw rejects a present zstd artifact that carries no frame', async () => { const root = await freshRoot() const ctx = await mount(root) @@ -495,7 +706,7 @@ describe('JsonlSessionPersistence: default Zstandard encoding', () => { buffer[eventFrame.end - 1] = buffer[eventFrame.end - 1]! ^ 0xFF await writeFile(path, buffer) - expect((await ctx.sessionPersistence.list()).map(item => item.id)).toEqual([header.id]) + expect(listedHeaders(await ctx.sessionPersistence.list()).map(item => item.id)).toEqual([header.id]) await expect(ctx.sessionPersistence.load(header.id)).rejects.toThrow(/frame at byte .* failed validation/) }) @@ -684,7 +895,7 @@ describe('JsonlSessionPersistence: default Zstandard encoding', () => { expect((await ctx.sessionPersistence.load(header.id)).events).toEqual([...oneTurnLog(), ...secondTurn]) }) - it('skips empty, incomplete, and non-header compressed artifacts while rejecting malformed header frames', async () => { + it('isolates empty, incomplete, and non-header compressed artifacts as malformed descriptors', async () => { const root = await freshRoot() for (const [id, content] of [ ['empty', Buffer.alloc(0)], @@ -696,7 +907,9 @@ describe('JsonlSessionPersistence: default Zstandard encoding', () => { await writeFile(logPath(root, undefined, sessionId, 'zstd'), content) } const ctx = await mount(root) - expect(await ctx.sessionPersistence.list()).toEqual([]) + const initialListings = await ctx.sessionPersistence.list() + expect(initialListings).toHaveLength(3) + expect(initialListings.every(listing => listing.status === 'malformed')).toBe(true) const twoLinesId = SessionId('two-lines') await mkdir(sessionDir(root, undefined, twoLinesId), { recursive: true }) @@ -705,7 +918,14 @@ describe('JsonlSessionPersistence: default Zstandard encoding', () => { JSON.stringify({ type: 'turn/start' }), '', ].join('\n'))) - await expect(ctx.sessionPersistence.list()).rejects.toThrow(/first frame is not exactly one header line/) + const twoLineListing = (await ctx.sessionPersistence.list()) + .find(listing => listing.location?.path.endsWith(join( + 'two-lines', + `session.v${SESSION_FORMAT_VERSION}.jsonl.zstd`, + )) === true) + expect(twoLineListing?.status).toBe('malformed') + if (twoLineListing?.status !== 'malformed') throw new Error('expected malformed listing') + expect(twoLineListing.reason).toMatch(/first frame is not exactly one header line/) await expect(ctx.sessionPersistence.load(SessionId('two-lines'))) .rejects.toThrow(/first frame is not exactly one header line/) }) @@ -721,12 +941,19 @@ describe('JsonlSessionPersistence: default Zstandard encoding', () => { corruptHeader[corruptHeader.length - 1] = corruptHeader[corruptHeader.length - 1]! ^ 0xFF await writeFile(logPath(root, undefined, SessionId('bad-checksum'), 'zstd'), corruptHeader) const ctx = await mount(root) + const backend = ctx.sessionPersistence as JsonlSessionPersistence await expect(ctx.sessionPersistence.load(SessionId('partial-only'))) .rejects.toThrow(/empty or header-less Zstandard session log/) + await expect(backend.readRawStored(SessionId('partial-only'))) + .rejects.toThrow(/empty or header-less Zstandard session log/) + await expect(backend.loadStored(SessionId('partial-only'))) + .rejects.toThrow(/empty or header-less Zstandard session log/) await expect(ctx.sessionPersistence.load(SessionId('empty-header'))) .rejects.toThrow(/first frame is not exactly one header line/) - await expect(ctx.sessionPersistence.list()).rejects.toThrow(/header frame failed validation/) + const listings = await ctx.sessionPersistence.list() + expect(listings.some(listing => + listing.status === 'malformed' && /header frame failed validation/.test(listing.reason))).toBe(true) }) }) @@ -739,6 +966,10 @@ describe('JsonlSessionPersistence: encoding selection', () => { await raw.sessionPersistence.append(rawHeader.id, oneTurnLog()) const defaultBackend = await mount(rawRoot) await expect(defaultBackend.sessionPersistence.list()).rejects.toThrow(/configured for compression "zstd"/) + const blockedZstd = meta('blocked-zstd-write') + await defaultBackend.sessionPersistence.create(blockedZstd) + await expect(defaultBackend.sessionPersistence.append(blockedZstd.id, oneTurnLog())) + .rejects.toThrow(/configured for compression "zstd"/) const zstdRoot = await freshRoot('dsh-jsonl-zstd-mismatch-') const zstd = await mount(zstdRoot) @@ -756,8 +987,8 @@ describe('JsonlSessionPersistence: encoding selection', () => { const loadHeader = meta('late-raw-load', '/late') await mkdir(sessionDir(root, loadHeader.cwd, loadHeader.id), { recursive: true }) - await writeFile(logPath(root, loadHeader.cwd, loadHeader.id, 'none'), [ - JSON.stringify(toHeaderLine(loadHeader)), + await writeFile(generationLogPath(root, loadHeader.cwd, loadHeader.id, 7, 'none'), [ + JSON.stringify({ ...toHeaderLine(loadHeader), version: 7 }), ...oneTurnLog().map(e => JSON.stringify(e)), '', ].join('\n')) @@ -770,15 +1001,18 @@ describe('JsonlSessionPersistence: encoding selection', () => { it('refuses materialization when an opposite artifact appears after create', async () => { const root = await freshRoot() const ctx = await mount(root) - await ctx.sessionPersistence.list() + const priming = meta('prime-encoding-check', '/late') + await ctx.sessionPersistence.create(priming) + await ctx.sessionPersistence.append(priming.id, oneTurnLog()) const header = meta('late-raw-materialize', '/late') await ctx.sessionPersistence.create(header) await mkdir(sessionDir(root, header.cwd, header.id), { recursive: true }) - await writeFile(logPath(root, header.cwd, header.id, 'none'), [ - JSON.stringify(toHeaderLine(header)), + await writeFile(generationLogPath(root, header.cwd, header.id, 7, 'none'), [ + JSON.stringify({ ...toHeaderLine(header), version: 7 }), ...oneTurnLog().map(e => JSON.stringify(e)), '', ].join('\n')) + await writeFile(generationLogPath(root, header.cwd, header.id, 4, 'none'), 'older opposite generation') await expect(ctx.sessionPersistence.append(header.id, oneTurnLog())).rejects.toThrow(/uses \.jsonl/) expect((await readdir(sessionDir(root, header.cwd, header.id))).some(name => name.endsWith('.jsonl.zstd'))).toBe(false) }) diff --git a/packages/session/session-persistence-jsonl/tsconfig.json b/packages/session/session-persistence-jsonl/tsconfig.json index ab6447e29f..8150657604 100644 --- a/packages/session/session-persistence-jsonl/tsconfig.json +++ b/packages/session/session-persistence-jsonl/tsconfig.json @@ -22,6 +22,9 @@ }, { "path": "../session-persistence" + }, + { + "path": "../session-format-catalog" } ] } diff --git a/packages/session/session-persistence/README.i18n.yaml b/packages/session/session-persistence/README.i18n.yaml index 6cb47067fa..deb59b9247 100644 --- a/packages/session/session-persistence/README.i18n.yaml +++ b/packages/session/session-persistence/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/session/session-persistence/README.md -README.md: 6d430173b985d7e32b615e0b42713a925bc8b344 -README.zh.md: a3b6cd90fa4ccba2b28f0e81517bbabab0ce047e +README.md: 8477bda7218d943e9b2a5b10eff54ef593c00876 +README.zh.md: 881a3d14723a6012f0e47125b18d502b4805bb7b diff --git a/packages/session/session-persistence/README.md b/packages/session/session-persistence/README.md index 6d430173b9..8477bda721 100644 --- a/packages/session/session-persistence/README.md +++ b/packages/session/session-persistence/README.md @@ -29,7 +29,7 @@ Mount one persistence backend to make sessions durable. The backend registers it ### Choosing a backend -The seam ships the [JSONL](../session-persistence-jsonl/README.md) backend. It stores one append-only `.jsonl.zstd` artifact per Session and returns its absolute path from `locate(meta)`. A third-party backend may implement the service directly; the [backend contract](#understand-the-implementation) below is what it must honor. +The seam ships the [JSONL](../session-persistence-jsonl/README.md) backend. It stores one append-only current generation per Session and retains older version-named generations beside it; `locate(meta)` returns the absolute target path for the supplied logical header version. A third-party backend may implement the service directly; the [backend contract](#understand-the-implementation) below is what it must honor. ### What the service provides @@ -40,10 +40,12 @@ await ctx.sessionPersistence.create(meta, inheritedEventCount) // cut required w await ctx.sessionPersistence.ensureMaterialized(session) // persist an empty resumable session await ctx.sessionPersistence.append(id, events) // durably persist a batch const { meta, inheritedEventCount, events } = await ctx.sessionPersistence.load(id) -const headers = await ctx.sessionPersistence.list() // every stored session +const listings = await ctx.sessionPersistence.list() // header-only artifact descriptors ``` -`append` resolves only after the batch is durable, so a resolved write survives an OS crash or power loss. Ordinary `create(meta, inheritedEventCount)` remains lazy; `meta.isSeeded: true` requires the sibling exact cut, while unseeded metadata may omit it and rejects a nonzero value. The first materializing batch for a seeded session must reach the complete inherited prefix, so storage never exposes metadata whose cut exceeds its log. A lifecycle frontend calls `ensureMaterialized` only when an empty session must itself appear in durable listing without inventing an event. `load` returns an immutable balanced log and commits any needed crash recovery; `inspect` reads the same complete view without committing recovery. `readFrom` accepts a `SessionLogOffset` and returns a detached `SessionEventSuffix` carrying that `fromSeq`, the unchanged inherited cut, and only stored events at or after the cut. A session's artifact location (`locate`) resolves without filesystem I/O. +`append` resolves only after the batch is durable, so a resolved write survives an OS crash or power loss. Ordinary `create(meta, inheritedEventCount)` remains lazy; `meta.isSeeded: true` requires the sibling exact cut, while unseeded metadata may omit it and rejects a nonzero value. The first materializing batch for a seeded session must reach the complete inherited prefix, so storage never exposes metadata whose cut exceeds its log. A lifecycle frontend calls `ensureMaterialized` only when an empty session must itself appear in durable listing without inventing an event. `list` and `listSnapshots` read only independent headers and return one current, migration-required, unsupported, or malformed descriptor for the highest canonical generation in each Session directory. `load` returns an immutable balanced log and commits any needed crash recovery. For already-current storage, `inspect` keeps synthetic recovery in memory; a supported historical inspection first publishes a repaired current successor beside the unchanged source. `readFrom` accepts a `SessionLogOffset` and returns a detached `SessionEventSuffix` carrying that `fromSeq`, the unchanged inherited cut, and only stored events at or after the cut. A session's version-qualified artifact target (`locate`) resolves without filesystem I/O. + +Cancellation on `prepare`, `inspect`, or `borrowSession` stops only that observer's wait. Shared cold preparation or historical migration that another observer can reuse may continue to completion; cancellation never rolls back a generation already entering durable publication. Detached `readFrom` and `readRaw` instead pass their cancellation signal to the serialized backend read. ### Resuming and crash recovery @@ -51,7 +53,7 @@ Resume is `load` plus session preparation: the stored log comes back with its he ### Failures and recovery -A stored log the current build cannot faithfully interpret is refused with a direction-aware error, never misread. `SESSION_FORMAT_VERSION` remains v0 and this build provides no format-migration path; a newer version instructs the operator to upgrade the harness. The decoder accepts only the bounded same-version record variants named below. An event type unknown to this build refuses unless its envelope marks it `ignorable`, and committed-prefix corruption rejects as `SessionPersistenceCorruptionError`. A `load` on an id still bound to a live session first flushes its snapshot and rejects while its turn is open; a cold load applies recovery. +A stored log the current build cannot faithfully interpret is refused with a direction-aware error, never misread. On a cold body read, the shipped JSONL backend selects the numerically highest canonical generation, refuses a future version even when an older readable file remains, and migrates a supported older generation through the static adjacent catalog. It publishes only the final current filename without overwriting the source or materializing intermediate versions. Header-only listing rescans and does not migrate. Catalog construction refuses a missing edge, committed-prefix corruption rejects as `SessionPersistenceCorruptionError`, equal-version unknown events require `ignorable: true`, and alpha historical migration refuses every unknown v0 event before publication. Retained predecessors are an operator escape hatch only: the service performs no automatic fallback and promises no downgrade compatibility. A `load` on an id still bound to a live session first flushes its snapshot and rejects while its turn is open; a cold load applies recovery. ----- @@ -91,7 +93,7 @@ Each `session/event` copies the event into its session's controller. The first p ### Stored-record compatibility -Backend reads normalize only the explicitly supported v0 record variants before validating current records. The coordinator uses the same normalized view for `load`, `inspect`, `readFrom`, ownerless-state claims, and HMR adoption. Reads do not rewrite stored records, and later appends use current v0. The [pre-identity message](../../../.agents/notes/implemented/bug-fix/2026-07-28-load-pre-identity-session-messages.md) and [pre-react-loop session](../../../.agents/notes/implemented/bug-fix/2026-08-04-load-pre-react-loop-sessions.md) notes own these bounded exceptions; they are not a general format-migration promise. +The coordinator and current Session code receive only the latest logical representation. `@deepseek-ai/dsh-session-format-catalog` supplies the profile-independent complete adjacent chain, and each named edge owns its historical header, physical records, payload inventory, normalizers, and target validator. Every body read completes backend-owned ensure-current work inside the per-Session serialization chain before current restoration. A backend may fuse that work with its current read; the generic fallback calls `ensureCurrent` and then the ordinary current hook. The [released-format lifecycle](../../../.agents/notes/implemented/architecture/2026-08-31-released-session-format-migrations.md) owns immutable generation naming, highest-generation selection, and exclusive successor publication. ----- @@ -102,6 +104,7 @@ Backend reads normalize only the explicitly supported v0 record variants before Read these pages when the package-level contract is not enough. They move from the shared durability model to the shipped backends and the decision evidence. - [Session persistence subsystem](../../../docs/subsystems/persistence.md) — the full service contract, flush checkpoint, crash recovery, and generated Cordis API. +- [Session format chain](../session-format/README.md) — pure historical dispatch and adjacent migration composition. - [JSONL persistence backend](../session-persistence-jsonl/README.md) — the shipped per-session-file backend. - [Session checkpoint policy](../session-checkpoint-policy/README.md) — the plugin that flushes through this service at semantic boundaries. - [Session package map](../README.md) — adjacent persistence, projection, title, and telemetry packages. @@ -133,7 +136,7 @@ Persistence does not mutate live request prefixes. A resumed loop can reuse prov These limits define where the seam's guarantees stop. They are current package constraints, not a task backlog. - **No deletion or retention API** — pruning stored sessions is out-of-band backend maintenance. -- **`list()` is unpaginated and unfiltered** — it returns every stored session's header; fine for local stores, unindexed at scale. +- **`list()` is unpaginated and unfiltered** — it returns one header-only descriptor for every stored Session directory's highest canonical generation; fine for local stores, unindexed at scale. - **Synthetic closers are the only crash story** — a backend must synthesize `tool/result`/`step/end`/`turn/end` closers on load; there is no partial-turn resume that continues an interrupted turn instead of closing it. diff --git a/packages/session/session-persistence/README.zh.md b/packages/session/session-persistence/README.zh.md index a3b6cd90fa..881a3d1472 100644 --- a/packages/session/session-persistence/README.zh.md +++ b/packages/session/session-persistence/README.zh.md @@ -29,7 +29,7 @@ kind: "package-reference" ### 选择后端 -seam 随产品交付 [JSONL](../session-persistence-jsonl/README.zh.md) 后端。它把每个 Session 存为一份仅追加 `.jsonl.zstd` 产物，并由 `locate(meta)` 返回绝对路径。第三方后端可以直接实现该服务；必须遵守的[后端约定](#understand-the-implementation)见下文。 +seam 随产品交付 [JSONL](../session-persistence-jsonl/README.zh.md) 后端。它为每个 Session 存储一个仅追加的当前 generation，并在旁边保留较旧的具名版本 generation；`locate(meta)` 返回所给逻辑 header 版本的绝对目标路径。第三方后端可以直接实现该服务；必须遵守的[后端约定](#understand-the-implementation)见下文。 ### 服务提供什么 @@ -40,10 +40,12 @@ await ctx.sessionPersistence.create(meta, inheritedEventCount) // cut required w await ctx.sessionPersistence.ensureMaterialized(session) // persist an empty resumable session await ctx.sessionPersistence.append(id, events) // durably persist a batch const { meta, inheritedEventCount, events } = await ctx.sessionPersistence.load(id) -const headers = await ctx.sessionPersistence.list() // every stored session +const listings = await ctx.sessionPersistence.list() // header-only artifact descriptors ``` -`append` 只在批次持久后返回，因此成功返回的写入在操作系统崩溃或断电后依然存在。普通 `create(meta, inheritedEventCount)` 保持惰性；`meta.isSeeded: true` 要求单独的精确 cut，unseeded metadata 可以省略它并拒绝非零值。seeded 会话的首个物化批次必须到达完整继承前缀，因此存储绝不公开 cut 超过日志的 metadata。只有当空会话本身必须出现在持久列表中时，生命周期前端才调用 `ensureMaterialized`，且不会虚构事件。`load` 返回不可变的平衡日志并提交任何需要的崩溃恢复；`inspect` 读取同一份完整视图但不提交恢复。`readFrom` 接受 `SessionLogOffset`，并返回分离的 `SessionEventSuffix`，其中携带该 `fromSeq`、不变的继承 cut，以及 cut 位置或之后的存储事件。会话的产物位置（`locate`）不经文件系统 I/O 即可解析。 +`append` 只在批次持久后返回，因此成功返回的写入在操作系统崩溃或断电后依然存在。普通 `create(meta, inheritedEventCount)` 保持惰性；`meta.isSeeded: true` 要求单独的精确 cut，unseeded metadata 可以省略它并拒绝非零值。seeded 会话的首个物化批次必须到达完整继承前缀，因此存储绝不公开 cut 超过日志的 metadata。只有当空会话本身必须出现在持久列表中时，生命周期前端才调用 `ensureMaterialized`，且不会虚构事件。`list` 与 `listSnapshots` 只读取独立 header，并为每个 Session 目录中数值最高的规范 generation 返回一个 current、migration-required、unsupported 或 malformed descriptor。`load` 返回不可变的平衡日志并提交任何需要的崩溃恢复。对于已经是当前格式的存储，`inspect` 只在内存中保留合成恢复；受支持的历史检查会先在不改变源文件的情况下于其旁边发布已修复的当前后继 generation。`readFrom` 接受 `SessionLogOffset`，并返回分离的 `SessionEventSuffix`，其中携带该 `fromSeq`、不变的继承 cut，以及 cut 位置或之后的存储事件。会话的版本限定产物目标（`locate`）不经文件系统 I/O 即可解析。 + +`prepare`、`inspect` 或 `borrowSession` 的取消只停止该观察者等待。可由另一观察者复用的共享冷准备或历史迁移可以继续完成；取消绝不会回滚已经进入持久发布阶段的 generation。分离的 `readFrom` 与 `readRaw` 则会把取消信号传给串行化后端读取。 ### 恢复与崩溃恢复 @@ -51,7 +53,7 @@ const headers = await ctx.sessionPersistence.list() // every stored sessi ### 失败与恢复 -当前构建无法忠实解读的存储日志会以方向感知的错误被拒绝，绝不错读。`SESSION_FORMAT_VERSION` 保持 v0，本构建不提供格式迁移路径；更高版本会要求操作者升级 harness。解码器只接受下文点名的有限同版本记录变体。本构建不认识的事件类型会被拒绝，除非其信封标记为 `ignorable`；已提交前缀中的损坏以 `SessionPersistenceCorruptionError` 拒绝。对仍绑定到活动会话的 id 执行 `load`，会先刷新其快照并在轮次开放时拒绝；冷 load 应用恢复。 +当前构建无法忠实解读的存储日志会以方向感知的错误被拒绝，绝不错读。冷正文读取会让随附 JSONL 后端选择数值最高的规范 generation；即使仍有可读旧文件，未来版本也会被拒绝。受支持的旧 generation 会通过静态相邻 catalog 迁移，并且只以不覆盖源文件、也不物化中间版本的方式发布最终当前文件名。仅 header 的列表会重新扫描且不会迁移。catalog 构造会拒绝缺失迁移边，已提交前缀中的损坏以 `SessionPersistenceCorruptionError` 拒绝，同版本未知事件要求 `ignorable: true`，alpha 历史迁移会在发布前拒绝每个未知 v0 事件。保留的旧 generation 只为 operator 提供逃生通道：服务不会自动 fallback，也不承诺 downgrade compatibility。对仍绑定到活动会话的 id 执行 `load`，会先刷新其快照并在轮次开放时拒绝；冷 load 应用恢复。 ----- @@ -91,7 +93,7 @@ const headers = await ctx.sessionPersistence.list() // every stored sessi ### 存储记录兼容 -后端读取只会在校验当前记录之前，规范化明确支持的 v0 记录变体。协调器对 `load`、`inspect`、`readFrom`、无所有者状态认领与 HMR 接管使用同一份规范化视图。读取不会重写已存记录，后续追加使用当前 v0。[消息标识机制引入前的消息](../../../.agents/notes/implemented/bug-fix/2026-07-28-load-pre-identity-session-messages.zh.md)与 [react-loop 引入前会话](../../../.agents/notes/implemented/bug-fix/2026-08-04-load-pre-react-loop-sessions.zh.md)笔记规定这些有限例外；它们不构成通用格式迁移承诺。 +协调器与当前 Session 代码只接收最新逻辑表示。`@deepseek-ai/dsh-session-format-catalog` 提供与 profile 无关的完整相邻链，每个具名迁移边拥有其历史 header、物理记录、payload 清单、归一化器与目标 validator。每个正文读取都会在当前恢复前，于逐 Session 串行链中完成后端拥有的 ensure-current 工作。后端可以把这项工作与当前读取融合；通用 fallback 会先调用 `ensureCurrent`，再调用普通当前读取钩子。[已发布格式生命周期](../../../.agents/notes/implemented/architecture/2026-08-31-released-session-format-migrations.zh.md)定义不可变 generation 命名、最高 generation 选择与后继排他发布。 ----- @@ -102,6 +104,7 @@ const headers = await ctx.sessionPersistence.list() // every stored sessi 当包级约定不够用时阅读以下页面。它们从共享持久性模型逐步进入随产品交付的后端与决策证据。 - [会话持久化子系统](../../../docs/subsystems/persistence.zh.md)——完整服务约定、flush 检查点、崩溃恢复与生成的 Cordis API。 +- [Session 格式链](../session-format/README.zh.md)——纯历史分派与相邻迁移组合。 - [JSONL 持久化后端](../session-persistence-jsonl/README.zh.md)——随产品交付、按会话存储文件的后端。 - [会话检查点策略](../session-checkpoint-policy/README.zh.md)——在语义边界上经由本服务刷新的插件。 - [会话包映射](../README.zh.md)——相邻的持久化、投影、标题与遥测包。 @@ -133,7 +136,7 @@ seam 不添加提示词或 schema。恢复会将已存储的表层事件还原 这些限制界定 seam 保证的终点。它们是当前包约束，不是任务积压。 - **无删除或保留接口**——剪枝已存储会话属于带外后端维护。 -- **`list()` 无分页且无过滤**——它返回每个已存储会话的 header；适合本地存储，大规模时无索引。 +- **`list()` 无分页且无过滤**——它为每个已存储 Session 目录的最高规范 generation 返回一个仅 header 的 descriptor；适合本地存储，大规模时无索引。 - **合成 closer 是唯一崩溃方案**——后端必须在 load 时合成 `tool/result`/`step/end`/`turn/end` closer；没有继续中断轮次而不先关闭它的部分轮次恢复。 diff --git a/packages/session/session-persistence/src/coordinator.ts b/packages/session/session-persistence/src/coordinator.ts index c16fae6d40..be759299c9 100644 --- a/packages/session/session-persistence/src/coordinator.ts +++ b/packages/session/session-persistence/src/coordinator.ts @@ -13,7 +13,6 @@ import { SESSION_FORMAT_VERSION, SessionLogOffset, SessionPreparation, - SessionSeq, snapshotSessionEvent, } from '@deepseek-ai/dsh-session' import type { @@ -22,7 +21,6 @@ import type { SessionId, SessionHeader, SessionLogOffset as SessionLogOffsetType, - SessionSeq as SessionSeqType, } from '@deepseek-ai/dsh-session' import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout' import { snapshotJsonValue } from '@deepseek-ai/dsh-util-values' @@ -30,7 +28,9 @@ import type { BorrowedSessionSource, SessionEventSuffix, SessionInspection, + SessionPersistenceListing, SessionLocation, + SessionRawArtifact, SessionStorageMetadata, } from './index.ts' import { SessionPersistenceNotFoundError } from './errors.ts' @@ -142,6 +142,25 @@ export interface PersistenceBackend { /** Human-readable backend name, used in the dispose-failure AggregateError. */ readonly name: string + /** + * Optional backend-owned format migration before an event-body or raw + * artifact read. The coordinator invokes it only inside the per-Session + * serialization chain; header-only listing and existence probes bypass it. + * @param id - persisted session id whose highest generation must resolve to current. + * @param signal - optional cancellation for migration work. + */ + ensureCurrent?(id: SessionId, signal?: AbortSignal): Promise + + /** + * Optional fused current-generation prefix read. A backend implements this + * when format detection/migration and physical decoding can share one stable + * source read. The coordinator uses it instead of {@link ensureCurrent} + * followed by {@link loadStored}. + * @param id - persisted session id to ensure and read. + * @param signal - optional cancellation for migration and decode work. + */ + loadCurrentStored?(id: SessionId, signal?: AbortSignal): Promise | undefined> + /** * Read a stored prefix by id, scanning every backend storage scope. Returns * `undefined` if no stored artifact exists. Returned metadata must identify @@ -165,6 +184,23 @@ export interface PersistenceBackend { */ readStoredRevision(id: SessionId, signal?: AbortSignal): Promise + /** + * Optional current-generation raw artifact read. The coordinator invokes + * this hook only after same-Session writes and migration publication have + * crossed the per-id serialization chain. + * @param id - persisted session id to read. + * @param signal - optional cancellation for backend read work. + */ + readRawStored?(id: SessionId, signal?: AbortSignal): Promise + + /** + * Optional fused current-generation raw read. A backend implements this when + * migration and raw decoding can share one stable source read. + * @param id - persisted session id to ensure and read. + * @param signal - optional cancellation for migration and decode work. + */ + readCurrentRawStored?(id: SessionId, signal?: AbortSignal): Promise + /** * Optional seek-capable suffix read behind the service's `readFrom`: return * the header plus the stored events with `seq >= fromSeq` without reading @@ -173,10 +209,8 @@ export interface PersistenceBackend { * omit it and the coordinator falls back to {@link loadStored} plus a * forward skip. Non-mutating (no truncation, no closers). Validation of the * region strictly below `fromSeq` is limited to seq contiguity — the - * service contract scopes this read to the suffix — unless that suffix - * contains a supported legacy shape whose normalization needs earlier - * message-identity facts, in which case the coordinator falls back - * to the complete stored prefix. + * service contract scopes this read to the suffix. Historical migration is + * complete before this current-format hook runs. * Unknown-type refusal follows the same suffix scope: a seek-capable * backend's `readFrom` checks only the returned suffix, while the * sequential fallback parses the whole artifact and refuses on an unknown @@ -223,7 +257,7 @@ export interface PersistenceBackend { * List all stored (materialized) sessions' metadata. * @param signal - optional cancellation for backend listing work. */ - list(signal?: AbortSignal): Promise + list(signal?: AbortSignal): Promise /** * Optional side-effect-free artifact locator, used to point refusal @@ -317,308 +351,14 @@ function sessionStorageMetadata(session: Session): SessionStorageMetadata { return storageMetadata(session.header, session.inheritedEventCount) } -/** Reject events from an obsolete v0 vocabulary that this build cannot replay. */ -function assertSupportedEvents(events: readonly SessionEvent[], id: SessionId): void { - const legacyType: string = 'request/header-delta' - const legacy = events.find(event => event.type === legacyType) - if (legacy !== undefined) { - throw new Error(`session "${id}" contains unsupported legacy request/header-delta event at seq ${legacy.seq}`) - } - const legacyModeType: string = 'mode/set' - const legacyMode = events.find(event => event.type === legacyModeType) - if (legacyMode !== undefined) { - throw new Error(`session "${id}" contains unsupported legacy mode/set event at seq ${legacyMode.seq}`) - } - const fallback = events.find(event => event.type === 'request/header' - && (event.data as { reason?: string }).reason === 'fallback') - if (fallback !== undefined) { - throw new Error(`session "${id}" contains unsupported legacy request/header reason "fallback" at seq ${fallback.seq}`) - } +/** Materialize current stored events as detached immutable snapshots. */ +function snapshotStoredEvents(events: readonly SessionEvent[]): SessionEvent[] { + return events.map(event => snapshotSessionEvent(event)) } -/** Return an object record without widening arrays into message payloads. */ -function asRecord(value: unknown): Record | undefined { - return typeof value === 'object' && value !== null && !Array.isArray(value) - ? value as Record - : undefined -} - -/** Whether a record contains every required key and no key outside the optional extension set. */ -function hasOnlyKeys( - record: Record, - required: readonly string[], - optional: readonly string[] = [], -): boolean { - const allowed = [...required, ...optional] - return Object.keys(record).every(key => allowed.includes(key)) - && required.every(key => Object.hasOwn(record, key)) -} - -type PersistedMessageId = SessionEvent<'user/message'>['data']['id'] - -/** Mint the stable import identity for a message persisted before identities existed. */ -function legacyMessageId(id: SessionId, seq: SessionSeqType): PersistedMessageId { - return `legacy-message:${id}:${seq}` as PersistedMessageId -} - -/** Read a replacement target while leaving malformed surface metadata to the session validator. */ -function replacementStart(event: SessionEvent): SessionSeqType | undefined { - const op = asRecord((event as SessionEvent & { surfaceOp?: unknown }).surfaceOp) - if (op?.['op'] !== 'replace' || typeof op['start'] !== 'number') return undefined - try { - return SessionSeq(op['start']) - } catch { - return undefined - } -} - -/** Whether one suffix event needs facts available only from the preceding stored prefix. */ -function needsLegacyPrefix(event: SessionEvent): boolean { - const data = asRecord(event.data) - const legacySteeringType: string = 'steering/message' - if (event.type === legacySteeringType) return true - if (data === undefined) return false - switch (event.type) { - case 'user/message': - return !Object.hasOwn(data, 'id') && Object.hasOwn(data, 'content') - case 'assistant/message': - return !Object.hasOwn(data, 'message') && Object.hasOwn(data, 'content') - case 'tool/result': - return !Object.hasOwn(data, 'message') && Object.hasOwn(data, 'callId') - default: - return false - } -} - -/** Upgrade the removed steering surface event into its current user-message equivalent. */ -function migrateLegacySteeringEvent(event: SessionEvent, id: SessionId): SessionEvent { - const legacyType: string = 'steering/message' - if (event.type !== legacyType) return event - const data = asRecord(event.data) - if (data === undefined) { - throw new Error(`session "${id}" contains malformed pre-react-loop steering/message at seq ${event.seq}`) - } - const wrapped = asRecord(data['message']) - if (wrapped !== undefined && Number.isSafeInteger(data['turn']) - && hasOnlyKeys(data, ['turn', 'message'])) { - return { ...event, type: 'user/message', data: wrapped } as SessionEvent - } - if (!Number.isSafeInteger(data['turn']) || !hasOnlyKeys(data, ['turn', 'content', 'source'])) { - throw new Error(`session "${id}" contains malformed pre-react-loop steering/message at seq ${event.seq}`) - } - const { turn: _turn, ...message } = data - return { - ...event, - type: 'user/message', - data: { - ...message, - id: legacyMessageId(id, event.seq), - role: 'user', - }, - } as SessionEvent -} - -/** Remove the obsolete trigger after verifying the complete old turn-start envelope. */ -function migrateLegacyTurnStartEvent(event: SessionEvent, id: SessionId): SessionEvent { - if (event.type !== 'turn/start') return event - const data = asRecord(event.data) - if (data === undefined || !Object.hasOwn(data, 'trigger')) return event - const trigger = asRecord(data['trigger']) - if (!Number.isSafeInteger(data['turn']) || (data['turn'] as number) < 1 - || !hasOnlyKeys(data, ['turn', 'trigger']) - || trigger === undefined || typeof trigger['kind'] !== 'string' || trigger['kind'].length === 0) { - throw new Error(`session "${id}" contains malformed pre-react-loop turn/start at seq ${event.seq}`) - } - return { ...event, data: { turn: data['turn'] } } as SessionEvent -} - -/** Upgrade an obsolete turn ending while preserving the latest-master envelope. */ -function migrateLegacyTurnEndEvent(event: SessionEvent, id: SessionId): SessionEvent { - if (event.type !== 'turn/end') return event - const data = asRecord(event.data) - /* v8 ignore next -- a non-record current envelope cannot match a legacy shape. */ - if (data === undefined) return event - const malformed = (): never => { - throw new Error(`session "${id}" contains malformed pre-react-loop turn/end at seq ${event.seq}`) - } - const reason = asRecord(data['reason']) - if (!Number.isSafeInteger(data['turn']) || (data['turn'] as number) < 1 - || !hasOnlyKeys(data, ['turn', 'reason']) - || reason === undefined || typeof reason['kind'] !== 'string') return malformed() - - let currentReason: Record | undefined - switch (reason['kind']) { - case 'completed': - case 'blocked': - case 'max-tokens': - case 'interrupted': - if (!hasOnlyKeys(reason, ['kind'])) return malformed() - return event - case 'aborted': - if (Object.hasOwn(reason, 'reason')) return event - if (!hasOnlyKeys(reason, ['kind'])) return malformed() - currentReason = { kind: 'aborted', reason: { kind: 'legacy' } } - break - case 'disposed': - if (!hasOnlyKeys(reason, ['kind'])) return malformed() - currentReason = { kind: 'aborted', reason: { kind: 'disposed' } } - break - case 'error': { - if (Object.hasOwn(reason, 'error')) return event - if (!Number.isSafeInteger(reason['step']) || (reason['step'] as number) < 0) return malformed() - const failure = asRecord(reason['failure']) - if (failure !== undefined && hasOnlyKeys(reason, ['kind', 'step', 'failure']) - && hasOnlyKeys(failure, ['message', 'code'], ['status', 'providerRetryAfterMs', 'requestId']) - && typeof failure['message'] === 'string' && typeof failure['code'] === 'string' - && (failure['status'] === undefined || typeof failure['status'] === 'number') - && (failure['providerRetryAfterMs'] === undefined || typeof failure['providerRetryAfterMs'] === 'number') - && (failure['requestId'] === undefined || typeof failure['requestId'] === 'string')) { - currentReason = { kind: 'error', error: failure } - break - } - const messageKeys = reason['code'] === undefined - ? ['kind', 'step', 'message'] - : ['kind', 'step', 'message', 'code'] - if (!hasOnlyKeys(reason, messageKeys) - || typeof reason['message'] !== 'string' - || (reason['code'] !== undefined && typeof reason['code'] !== 'string')) return malformed() - currentReason = { - kind: 'error', - error: { - message: reason['message'], - code: typeof reason['code'] === 'string' ? reason['code'] : 'UNKNOWN', - }, - } - break - } - default: - return event - } - - return { - ...event, - data: { - ...data, - reason: currentReason, - }, - } as SessionEvent -} - -/** - * Upgrade one pre-identity message event into the current wrapper shape. - * Current-looking malformed events remain untouched so validation rejects them - * instead of disguising corruption as legacy data. - */ -function migrateLegacyMessageEvent( - event: SessionEvent, - id: SessionId, - messageIds: ReadonlyMap, -): SessionEvent { - const data = asRecord(event.data) - if (data === undefined) return event - switch (event.type) { - case 'user/message': { - if (Object.hasOwn(data, 'id') || Object.hasOwn(data, 'role') - || Object.hasOwn(data, 'message') - || !Object.hasOwn(data, 'content') || !Object.hasOwn(data, 'source')) return event - return { - ...event, - data: { - ...data, - id: legacyMessageId(id, event.seq), - role: 'user', - }, - } as SessionEvent - } - case 'assistant/message': { - if (Object.hasOwn(data, 'message') - || !Object.hasOwn(data, 'content') || !Object.hasOwn(data, 'provenance')) return event - const { content, provenance, ...eventData } = data - return { - ...event, - data: { - ...eventData, - message: { - id: legacyMessageId(id, event.seq), - role: 'assistant', - content, - source: { - ...asRecord(provenance), - kind: 'model', - }, - }, - }, - } as SessionEvent - } - case 'tool/result': { - if (Object.hasOwn(data, 'message') - || !Object.hasOwn(data, 'callId') || !Object.hasOwn(data, 'content') - || !Object.hasOwn(data, 'isError')) return event - const { callId, content, isError, ...eventData } = data - const inheritedId = replacementStart(event) - return { - ...event, - data: { - ...eventData, - message: { - id: inheritedId === undefined - ? legacyMessageId(id, event.seq) - : messageIds.get(inheritedId), - role: 'user', - content: [{ - type: 'tool-result', - toolCallId: callId, - content, - isError, - }], - source: { - kind: 'tool', - callId, - }, - }, - }, - } as SessionEvent - } - default: - return event - } -} - -/** Read the identified message carried by one validated current event. */ -function eventMessageId(event: SessionEvent): PersistedMessageId | undefined { - const data = asRecord(event.data) - const message = event.type === 'user/message' ? data : asRecord(data?.['message']) - return typeof message?.['id'] === 'string' ? message['id'] as PersistedMessageId : undefined -} - -/** Materialize stored events as upgraded, validated snapshots with immutable messages. */ -function snapshotStoredEvents(events: readonly SessionEvent[], id: SessionId): SessionEvent[] { - assertSupportedEvents(events, id) - const messageIds = new Map() - return events.map((event) => { - const migratedStart = migrateLegacyTurnStartEvent(event, id) - const migratedTurn = migrateLegacyTurnEndEvent(migratedStart, id) - const migratedSteering = migrateLegacySteeringEvent(migratedTurn, id) - const snapshot = snapshotSessionEvent(migrateLegacyMessageEvent(migratedSteering, id, messageIds)) - const messageId = eventMessageId(snapshot) - if (messageId !== undefined) messageIds.set(snapshot.seq, messageId) - return snapshot - }) -} - -/** Upgrade and validate an exclusively owned backend result without copying it. */ -function adoptStoredEvents(events: SessionEvent[], id: SessionId): SessionEvent[] { - assertSupportedEvents(events, id) - const messageIds = new Map() - for (const [index, event] of events.entries()) { - const migratedStart = migrateLegacyTurnStartEvent(event, id) - const migratedTurn = migrateLegacyTurnEndEvent(migratedStart, id) - const migratedSteering = migrateLegacySteeringEvent(migratedTurn, id) - const adopted = adoptSessionEvent(migrateLegacyMessageEvent(migratedSteering, id, messageIds)) - events[index] = adopted - const messageId = eventMessageId(adopted) - if (messageId !== undefined) messageIds.set(adopted.seq, messageId) - } +/** Validate and freeze exclusively owned current backend events in place. */ +function adoptStoredEvents(events: SessionEvent[]): SessionEvent[] { + for (const event of events) adoptSessionEvent(event) return events } @@ -695,7 +435,7 @@ export class PersistenceCoordinator { try { storage = storageMetadata(snapshot, inheritedEventCount) } catch (error: unknown) { - /* v8 ignore next -- Session storage validation only throws Error instances. */ + /* v8 ignore next -- storageMetadata throws only built-in Error subclasses. */ return Promise.reject(error instanceof Error ? error : new TypeError('invalid session storage metadata', { cause: error })) @@ -717,6 +457,10 @@ export class PersistenceCoordinator { if (this.backend.materializeHeader === undefined) { throw new Error('session persistence backend cannot materialize an empty session') } + /* v8 ignore next -- successful flush persists every inherited seed event before this check. */ + if (state.cursor < state.storage.inheritedEventCount) { + throw new Error(`session "${session.id}" cannot materialize before its inherited prefix is complete`) + } await this.backend.materializeHeader(state.storage) state.materialized = true this.preparations.invalidate(session.id) @@ -732,7 +476,7 @@ export class PersistenceCoordinator { // A persisted artifact under this id (in ANY scope) blocks creation: load/ // resume identify a session by id alone, so a second artifact would make // resume nondeterministic. - if (await this.backend.loadStored(meta.id) !== undefined) { + if (await this.backend.readStoredRevision(meta.id) !== undefined) { throw new Error(`session "${meta.id}" already has a persisted log on disk; load/resume it instead of creating`) } // Pure lazy: record intent only. No artifact until the first append. @@ -767,14 +511,8 @@ export class PersistenceCoordinator { private async appendCore(id: SessionId, events: readonly SessionEvent[]): Promise { // Every append route converges here: the public service, live write-behind - // drains, and HMR seed/suffix adoption. Legacy-shape rejection stays at - // this shared boundary so a stale JavaScript plugin cannot persist a - // retired shape this backend refuses to load. The unknown-type guard is - // deliberately read-side only: an append-time refusal would stall a live - // session's durability mid-flight, which costs more than a loud refusal at - // the log's next load (trade-off owned by the session-log-version-mechanism - // Agent Note). - assertSupportedEvents(events, id) + // drains, and live-adoption suffix persistence. Unknown-type refusal stays + // read-side so a stale JavaScript plugin cannot stall live durability. if (events.length === 0) return this.preparations.assertWritable(id) let state = this.states.get(id) @@ -866,8 +604,10 @@ export class PersistenceCoordinator { } /** - * Inspect a logical session without publishing it or committing recovery. - * A stale ready source is reloaded. A source already committing or reserved + * Inspect a current logical session without publishing a live Session. + * Current-format recovery stays in memory; a historical body read may first + * publish a separate migrated and repaired current successor. A stale ready source + * is reloaded. A source already committing or reserved * for resume remains exclusive, and inspection may borrow its immutable view. * Revision retries converge once the log is stable for one read/check round * trip; continuous external writers may delay completion. @@ -968,8 +708,65 @@ export class PersistenceCoordinator { } /** - * Read the stored events from `fromSeq` onward, detached and non-mutating - * (the read-from-seq primitive behind the service's `readFrom`). Runs on + * Read one current raw artifact after flushing a live owner and serializing + * against same-Session storage operations. + * @param id - persisted session to read. + * @param signal - optional cancellation for queued and backend read work. + * @returns the backend-owned current artifact, or `undefined` when absent. + */ + async readRaw(id: SessionId, signal?: AbortSignal): Promise { + if (this.backend.readCurrentRawStored === undefined && this.backend.readRawStored === undefined) { + throw new Error('session persistence backend does not expose raw artifacts through its coordinator') + } + signal?.throwIfAborted() + const retirement = this.retirements.get(id) + if (retirement !== undefined) { + const waited = signal === undefined + ? retirement + : observeQueuedAbort(retirement, signal, () => false) + await waited + } + const live = this.ctx.sessions.get(id) + if (live !== undefined) await this.flush(live) + return this.serialize(id, async () => { + signal?.throwIfAborted() + const artifact = await this.readCurrentRawStored(id, signal) + signal?.throwIfAborted() + return artifact + }, signal) + } + + /** Prefer one fused raw read; retain ordered ensure/read fallback for other backends. */ + private async readCurrentRawStored( + id: SessionId, + signal?: AbortSignal, + ): Promise { + if (this.backend.readCurrentRawStored !== undefined) { + return this.backend.readCurrentRawStored(id, signal) + } + await this.backend.ensureCurrent?.(id, signal) + signal?.throwIfAborted() + return this.backend.readRawStored?.(id, signal) + } + + /** Prefer one fused prefix read; retain ordered ensure/read fallback for other backends. */ + private async loadCurrentStored( + id: SessionId, + signal?: AbortSignal, + ): Promise | undefined> { + if (this.backend.loadCurrentStored !== undefined) { + return this.backend.loadCurrentStored(id, signal) + } + await this.backend.ensureCurrent?.(id, signal) + signal?.throwIfAborted() + return this.backend.loadStored(id, signal) + } + + /** + * Read the stored events from `fromSeq` onward as a detached result (the + * read-from-seq primitive behind the service's `readFrom`). Current input is + * non-mutating; historical input may first publish its migrated current + * generation. Runs on * the same per-id chain as writes; a backend with the seek-capable * {@link PersistenceBackend.loadStoredFrom} hook reads only the suffix, * every other backend reads its stored prefix and skips forward here. @@ -986,7 +783,7 @@ export class PersistenceCoordinator { try { SessionLogOffset(fromSeq) } catch (error: unknown) { - /* v8 ignore next -- Session log-offset validation only throws Error instances. */ + /* v8 ignore next -- SessionLogOffset rejects invalid values with a built-in Error subclass. */ return Promise.reject(error instanceof Error ? error : new TypeError('invalid session read offset', { cause: error })) @@ -1003,6 +800,8 @@ export class PersistenceCoordinator { ): Promise { signal?.throwIfAborted() if (this.backend.loadStoredFrom !== undefined) { + await this.backend.ensureCurrent?.(id, signal) + signal?.throwIfAborted() let suffix: StoredSuffix | undefined try { suffix = await this.backend.loadStoredFrom(id, fromSeq, signal) @@ -1014,16 +813,7 @@ export class PersistenceCoordinator { if (suffix === undefined) throw new SessionPersistenceNotFoundError(id) this.assertStoredId(id, suffix.meta) this.assertVersion(suffix.meta) - if (suffix.events.some(needsLegacyPrefix)) { - const whole = await this.readStoredPrefix(id, signal) - return { - meta: whole.meta, - inheritedEventCount: whole.inheritedEventCount, - fromSeq, - events: whole.events.filter(event => event.seq >= fromSeq), - } - } - const events = snapshotStoredEvents(suffix.events, id) + const events = snapshotStoredEvents(suffix.events) this.assertEventsSupported(suffix.meta, events) return { meta: structuredClone(suffix.meta), @@ -1048,12 +838,12 @@ export class PersistenceCoordinator { signal?: AbortSignal, ): Promise { signal?.throwIfAborted() - const stored = await this.backend.loadStored(id, signal) + const stored = await this.loadCurrentStored(id, signal) signal?.throwIfAborted() if (stored === undefined) throw new SessionPersistenceNotFoundError(id) this.assertStoredId(id, stored.meta) this.assertVersion(stored.meta) - const events = snapshotStoredEvents(stored.events, id) + const events = snapshotStoredEvents(stored.events) this.assertEventsSupported(stored.meta, events) return { meta: structuredClone(stored.meta), @@ -1064,13 +854,13 @@ export class PersistenceCoordinator { /** Read, repair in memory, validate, and freeze one cold source once. */ private async prepareCore(id: SessionId): Promise> { - const stored = await this.backend.loadStored(id) + const stored = await this.loadCurrentStored(id) if (stored === undefined) throw new SessionPersistenceNotFoundError(id) try { const { meta, inheritedEventCount, events, revision, tornMarker } = stored this.assertStoredId(id, meta) this.assertVersion(meta) - const storedEvents = adoptStoredEvents(events, id) + const storedEvents = adoptStoredEvents(events) this.assertEventsSupported(meta, storedEvents) if (inheritedEventCount > storedEvents.length) { throw new Error(`session "${id}" inherited event count exceeds its stored event count`) @@ -1231,6 +1021,7 @@ export class PersistenceCoordinator { } private assertVersion(meta: SessionHeader): void { + // oxlint-disable-next-line typescript/no-unnecessary-condition -- durable backends can violate the static current-header type. if (meta.version === SESSION_FORMAT_VERSION) return throw this.unsupported(meta, sessionFormatVersionRefusal(meta.id, meta.version)) } @@ -1240,10 +1031,8 @@ export class PersistenceCoordinator { * writer marked the event ignorable: an unrecognized required event may * change how the rest of the log must be interpreted, so silently skipping * it would reconstruct a wrong session (the envelope contract on - * `SessionEvent.ignorable`). Runs on NORMALIZED events — after - * `snapshotStoredEvents`/`adoptStoredEvents` has upgraded the legacy shapes - * this build still reads and rejected the ones it does not, so those keep - * their specific diagnostics. + * `SessionEvent.ignorable`). Historical migration and current restoration + * complete before this check runs. */ private assertEventsSupported(meta: SessionHeader, events: readonly SessionEvent[]): void { for (const event of events) { @@ -1404,11 +1193,11 @@ export class PersistenceCoordinator { cursor: SessionLogOffsetType, ): Promise { if (cursor === 0) return true - const stored = await this.backend.loadStored(id) + const stored = await this.loadCurrentStored(id) /* v8 ignore next -- a cursor > 0 means the session was materialized, so it exists */ if (stored === undefined) return false this.assertStoredId(id, stored.meta) - return seedCoversPrefix(seed, snapshotStoredEvents(stored.events, id).slice(0, cursor)) + return seedCoversPrefix(seed, snapshotStoredEvents(stored.events).slice(0, cursor)) } /** @@ -1465,7 +1254,7 @@ export class PersistenceCoordinator { // case 2/3: resolve the id once across storage, then let adoption reject a // cwd mismatch before repair or state publication. - const live = await this.backend.loadStored(id) + const live = await this.loadCurrentStored(id) if (live !== undefined) { // Do NOT route through cold preparation: that crash-repairs open turns as // interrupted, which is wrong for HMR while the live Session is still the @@ -1505,7 +1294,7 @@ export class PersistenceCoordinator { throw new Error(`session "${session.header.id}" is already persisted with a different inherited event count (id collision)`) } this.assertVersion(meta) - const storedEvents = snapshotStoredEvents(events, session.header.id) + const storedEvents = snapshotStoredEvents(events) this.assertEventsSupported(meta, storedEvents) if (!seedCoversPrefix(seed, storedEvents)) { throw new Error(`session "${session.header.id}" already has a persisted log on disk that does not match this live session (id collision)`) diff --git a/packages/session/session-persistence/src/index.ts b/packages/session/session-persistence/src/index.ts index d110703d3a..a2386fbf72 100644 --- a/packages/session/session-persistence/src/index.ts +++ b/packages/session/session-persistence/src/index.ts @@ -20,12 +20,83 @@ export type { SessionHeader } from '@deepseek-ai/dsh-session' export { SessionPersistenceRevision } from './revision.ts' export { SessionPersistenceNotFoundError } from './errors.ts' +/** One current-format artifact whose header is already the latest logical type. */ +export interface CurrentSessionPersistenceListing { + readonly status: 'current' + /** Latest logical header decoded without reading event bodies. */ + readonly header: SessionHeader + /** Physical format version read from the artifact header. */ + readonly storedVersion: number + /** Format version this build writes and restores. */ + readonly targetVersion: number + /** Backend artifact location when the backend owns one per Session. */ + readonly location?: SessionLocation +} + +/** One supported historical artifact that will migrate on its first body read. */ +export interface MigrationRequiredSessionPersistenceListing { + readonly status: 'migration-required' + /** Latest logical header translated without reading event bodies. */ + readonly header: SessionHeader + /** Physical historical format version read from the artifact header. */ + readonly storedVersion: number + /** Format version this build will publish on the first body read. */ + readonly targetVersion: number + /** Backend artifact location when the backend owns one per Session. */ + readonly location?: SessionLocation +} + +/** One intact artifact whose format has no complete migration path in this build. */ +export interface UnsupportedSessionPersistenceListing { + readonly status: 'unsupported' + /** Physical version when the minimal header exposes one. */ + readonly storedVersion?: number + /** Format version this build writes and restores. */ + readonly targetVersion: number + /** Stable backend location for operator diagnosis. */ + readonly location: SessionLocation + /** Direction-aware refusal that does not require event-body parsing. */ + readonly reason: string +} + +/** One artifact whose minimal header is not structurally readable. */ +export interface MalformedSessionPersistenceListing { + readonly status: 'malformed' + /** Format version this build writes and restores. */ + readonly targetVersion: number + /** Stable backend location for operator diagnosis. */ + readonly location: SessionLocation + /** Header-only corruption diagnostic. */ + readonly reason: string +} + +/** One complete header-only listing result; event bodies are never read. */ +export type SessionPersistenceListing = + | CurrentSessionPersistenceListing + | MigrationRequiredSessionPersistenceListing + | UnsupportedSessionPersistenceListing + | MalformedSessionPersistenceListing + +/** A listing entry that safely exposes the latest logical Session header. */ +export type ReadableSessionPersistenceListing = + | CurrentSessionPersistenceListing + | MigrationRequiredSessionPersistenceListing + +/** + * Whether a listing entry carries a latest logical Session header. + * @param listing - header-only descriptor to narrow. + * @returns `true` for current and migration-required artifacts. + */ +export function isReadableSessionPersistenceListing( + listing: SessionPersistenceListing, +): listing is ReadableSessionPersistenceListing { + return listing.status === 'current' || listing.status === 'migration-required' +} + /** Lightweight immutable source identity returned without loading a full log. */ -export interface SessionPersistenceSnapshot { - /** Detached metadata for one materialized session. */ - header: SessionHeader - /** Opaque source-qualified token that changes whenever this stored log changes. */ - revision: SessionPersistenceRevision +export type SessionPersistenceSnapshot = SessionPersistenceListing & { + /** Opaque source-qualified token that changes whenever this stored artifact changes. */ + readonly revision: SessionPersistenceRevision } /** Logical Session header paired with its exact inherited cut for body-bearing storage operations. */ @@ -72,7 +143,7 @@ export type BorrowedSessionSource = Disposable & ( /** A backend's own raw artifact text for one session, verbatim. */ export interface SessionRawArtifact extends SessionStorageMetadata { - /** The artifact's base filename on disk, without any physical encoding suffix. */ + /** Selected generation basename; physical `.zstd` is omitted, while `.vN` remains. */ readonly filename: string /** The artifact's full text content, decoded from the backend's physical encoding. */ readonly content: string @@ -125,16 +196,18 @@ export abstract class SessionPersistence extends Service { } /** - * Resolve this backend's independent local artifact for a session without - * reading, creating, flushing, or otherwise materializing it. A backend - * that does not own one artifact per Session returns `undefined`. + * Resolve this backend's current-generation target for a session without + * reading, creating, flushing, or otherwise materializing it. Historical + * generations may live at other immutable paths; listing descriptors carry + * the exact selected stored location. A backend without per-Session files + * returns `undefined`. * @param meta - the immutable session header whose artifact is requested. * @returns the backend-specific absolute location, when one exists. */ abstract locate(meta: SessionHeader): SessionLocation | undefined /** - * Whether this backend exposes one verbatim raw artifact per session. + * Whether this backend exposes the selected verbatim raw generation per Session. * A backend that declares `true` must override {@link readRaw}. */ abstract readonly supportsRawArtifacts: boolean @@ -146,7 +219,9 @@ export abstract class SessionPersistence extends Service { * reconstruction from parsed events, so it preserves backend-specific * serialization (chunk packing, key order, line breaks). Callers first test * {@link supportsRawArtifacts}; `undefined` then means only that the requested - * session has no materialized artifact. + * session has no materialized artifact. Reading a supported historical + * artifact leaves that generation untouched and exclusively publishes a + * separate repaired current successor; an already-current artifact is not rewritten. * @param _id - the persisted session to read (unused by the default: no * per-session artifact). * @param signal - optional cancellation for backend read work. @@ -200,8 +275,12 @@ export abstract class SessionPersistence extends Service { * their durable revision is still current; disposal releases an unpublished * reservation. Revision retries require the durable log to remain unchanged * for one read/check round trip; continuous external writers may delay completion. + * Preparing a supported historical artifact first persists its migration and + * current-format repair, while current input takes the no-write fast path. * @param id - persisted session to prepare. - * @param signal - optional cancellation for preparation work. + * @param signal - optional cancellation for this caller's wait. A shared + * preparation or historical migration already started for another observer + * may continue to completion. * @returns one owned unpublished Session preparation. */ async prepare(id: SessionId, signal?: AbortSignal): Promise { @@ -221,8 +300,10 @@ export abstract class SessionPersistence extends Service { } /** - * Load an immutable balanced logical view and commit any required cold - * recovery. A complete interrupted final turn is preserved and durably + * Load an immutable balanced current logical view and commit any required cold + * recovery. A supported historical artifact remains immutable while a + * separate repaired current successor is published before restoration. A complete + * interrupted final turn is preserved and durably * closed with missing tool errors plus any open step and turn boundaries; * only a torn final record is discarded. Unknown versions and corruption in * the committed prefix reject. Implementations MUST NOT crash-repair an @@ -236,10 +317,12 @@ export abstract class SessionPersistence extends Service { abstract load(id: SessionId): Promise /** - * Inspect an immutable logical session without committing recovery or - * publishing it. A cold complete interrupted turn receives synthetic closers - * in memory and a torn physical tail remains untouched. An already-live - * Session instead yields its current immutable snapshot, which may contain an + * Inspect an immutable current logical session without publishing a live + * Session. For an already-current cold artifact, a complete interrupted turn receives + * synthetic closers only in memory and a torn physical tail remains untouched. + * A supported historical artifact first publishes its separate repaired + * current successor, so inspection is not storage-read-only in that case. An + * already-live Session instead yields its current immutable snapshot, which may contain an * open turn and its `session/end-seed` boundary. Coordinator-backed * implementations retain the exact cold unpublished Session for bounded * reuse by a later {@link prepare}. A stale ready source is reloaded; a source @@ -247,7 +330,9 @@ export abstract class SessionPersistence extends Service { * may borrow its immutable view. Callers borrow only the immutable header and * log. Continuous external writers may delay revision convergence. * @param id - the persisted session to inspect. - * @param signal - optional cancellation for queued and backend read work. + * @param signal - optional cancellation for this observer. Shared cold + * preparation and an already-started historical migration may continue for + * another inspector or later resume. * @returns the validated header and current logical event log. */ abstract inspect(id: SessionId, signal?: AbortSignal): Promise @@ -256,9 +341,11 @@ export abstract class SessionPersistence extends Service { * Borrow one exact inspection while retaining any reusable prepared source. * A cold observation must pin the exact prepared Session that a later * {@link prepare} reserves. Implementations must not degrade this operation - * to a detached {@link inspect} result. + * to a detached {@link inspect} result. Borrowing a supported historical + * artifact first persists its migration and current-format repair. * @param id - persisted session to observe. - * @param signal - optional cancellation for preparation work. + * @param signal - optional cancellation for this observer's wait; shared + * preparation or migration work may continue for another owner. * @returns a disposable immutable observation. */ abstract borrowSession(id: SessionId, signal?: AbortSignal): Promise @@ -268,8 +355,11 @@ export abstract class SessionPersistence extends Service { * primitive for read models that resume from a watermark (e.g. a persisted * projection cache folding only the tail past its checkpoint). Unlike * {@link inspect}, it is a detached physical suffix read: no preparation - * cache, torn-tail truncation, synthetic closers, or coordinator-state - * publication. Only events from the valid contiguous stored prefix are + * cache or coordinator-state publication. Current input performs no + * torn-tail truncation or synthetic repair. A supported historical artifact + * leaves its exact source unchanged and publishes a separate repaired current + * successor, so its returned suffix may include those current closers. + * Only events from the valid contiguous stored prefix are * returned, so a torn fragment never reaches the caller. `fromSeq` at or * beyond the stored prefix returns an empty event list (never an error). * A backend whose medium can seek by seq may read only the suffix; @@ -287,19 +377,22 @@ export abstract class SessionPersistence extends Service { /** * Lightweight listing from metadata, without a full-log parse. * @param signal - optional cancellation for backend listing work. - * @returns one header per materialized session. + * @returns one isolated descriptor per materialized artifact. */ - abstract list(signal?: AbortSignal): Promise + abstract list(signal?: AbortSignal): Promise /** * List materialized sessions with cheap per-log change tokens. * * Repeated observations of an unchanged log return the same revision. A - * successful mutating {@link load} repair changes the next listed revision. + * successful mutating {@link load} repair changes the next listed revision; + * so does migration publication from any supported historical body read. * Revisions also distinguish independently backed stores so backend-local * counters cannot compare equal across different persistence sources. * @param signal - optional cancellation for backend snapshot-listing work. - * @returns one header and opaque revision per materialized session without loading full logs. + * @returns one isolated current, migration-required, unsupported, or malformed + * descriptor plus its opaque revision per materialized artifact, without + * loading full logs. */ abstract listSnapshots(signal?: AbortSignal): Promise } diff --git a/packages/session/session-persistence/tests/contract.ts b/packages/session/session-persistence/tests/contract.ts index b8f611eebb..547392a032 100644 --- a/packages/session/session-persistence/tests/contract.ts +++ b/packages/session/session-persistence/tests/contract.ts @@ -26,6 +26,7 @@ import type { SurfaceIntent, } from '@deepseek-ai/dsh-session' import { ToolCallId, MessageId, createMessage, freezeMessage } from '@deepseek-ai/dsh-llm' +import { isReadableSessionPersistenceListing } from '../src/index.ts' import type { SessionPersistence } from '../src/index.ts' /** A backend under test plus its teardown. */ @@ -157,7 +158,9 @@ export function runPersistenceContract(name: string, make: () => Promise header.id)).not.toContain(seeded.id) + expect((await persistence.list()) + .filter(isReadableSessionPersistenceListing) + .map(listing => listing.header.id)).not.toContain(seeded.id) await persistence.append(seeded.id, oneTurnLog()) await expect(persistence.load(seeded.id)).resolves.toMatchObject({ @@ -181,11 +184,13 @@ export function runPersistenceContract(name: string, make: () => Promise snapshot.header.id === m.id)?.revision + .find(snapshot => isReadableSessionPersistenceListing(snapshot) + && snapshot.header.id === m.id)?.revision const inspected = await persistence.inspect(m.id) const afterInspect = (await persistence.listSnapshots()) - .find(snapshot => snapshot.header.id === m.id)?.revision + .find(snapshot => isReadableSessionPersistenceListing(snapshot) + && snapshot.header.id === m.id)?.revision expect(afterInspect).toBe(beforeRepair) expect(inspected.events.map(e => e.type)).toEqual([ 'turn/start', 'user/message', 'step/start', 'assistant/message', 'step/end', 'turn/end', @@ -197,7 +202,8 @@ export function runPersistenceContract(name: string, make: () => Promise snapshot.header.id === m.id)?.revision + .find(snapshot => isReadableSessionPersistenceListing(snapshot) + && snapshot.header.id === m.id)?.revision expect(afterRepair).not.toBe(beforeRepair) expect(loaded.events.map(e => e.type)).toEqual([ 'turn/start', 'user/message', 'step/start', 'assistant/message', 'step/end', 'turn/end', // turn 1 @@ -322,8 +328,13 @@ export function runPersistenceContract(name: string, make: () => Promise m.id)).not.toContain(SessionId('empty')) - expect((await persistence.listSnapshots()).map(snapshot => snapshot.header.id)) + expect((await persistence.list()) + .filter(isReadableSessionPersistenceListing) + .map(listing => listing.header.id)).not.toContain(SessionId('empty')) + expect((await persistence.listSnapshots()) + .flatMap(snapshot => isReadableSessionPersistenceListing(snapshot) + ? [snapshot.header.id] + : [])) .not.toContain(SessionId('empty')) } finally { await dispose() @@ -401,9 +412,13 @@ export function runPersistenceContract(name: string, make: () => Promise x.id)).toContain(m.id) - const first = (await persistence.listSnapshots()).find(snapshot => snapshot.header.id === m.id) - const repeated = (await persistence.listSnapshots()).find(snapshot => snapshot.header.id === m.id) + expect((await persistence.list()) + .filter(isReadableSessionPersistenceListing) + .map(listing => listing.header.id)).toContain(m.id) + const first = (await persistence.listSnapshots()).find(snapshot => + isReadableSessionPersistenceListing(snapshot) && snapshot.header.id === m.id) + const repeated = (await persistence.listSnapshots()).find(snapshot => + isReadableSessionPersistenceListing(snapshot) && snapshot.header.id === m.id) expect(first).toBeDefined() expect(repeated?.revision).toBe(first?.revision) @@ -413,7 +428,8 @@ export function runPersistenceContract(name: string, make: () => Promise snapshot.header.id === m.id) + const changed = (await persistence.listSnapshots()).find(snapshot => + isReadableSessionPersistenceListing(snapshot) && snapshot.header.id === m.id) expect(changed?.revision).not.toBe(first?.revision) } finally { await dispose() diff --git a/packages/session/session-persistence/tests/coordinator-contract.ts b/packages/session/session-persistence/tests/coordinator-contract.ts index 587d7ff1c3..8dd061a33c 100644 --- a/packages/session/session-persistence/tests/coordinator-contract.ts +++ b/packages/session/session-persistence/tests/coordinator-contract.ts @@ -20,7 +20,8 @@ import SessionStore, { SessionLogOffset, SessionSeq, } from '@deepseek-ai/dsh-session' -import type { SessionEvent } from '@deepseek-ai/dsh-session' +import type { SessionEvent, SessionHeader } from '@deepseek-ai/dsh-session' +import { isReadableSessionPersistenceListing } from '../src/index.ts' import { meta, oneTurnLog, appendLog } from './contract.ts' /** @@ -51,159 +52,6 @@ function send(session: Session, events: readonly SessionEvent[]): void { appendLog(session, events) } -/** A valid persisted log from immediately before messages gained wrappers and identities. */ -export function legacyMessageLog(): SessionEvent[] { - return [ - { type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }, - { - type: 'user/message', - seq: 1, - time: 2, - data: { content: [{ type: 'text', text: 'hi' }], source: { kind: 'user' } }, - surfaceOp: 'append', - }, - { type: 'step/start', seq: 2, time: 3, data: { turn: 1, step: 1 } }, - { - type: 'assistant/message', - seq: 3, - time: 4, - data: { - turn: 1, - step: 1, - content: [{ type: 'tool-call', id: 'call-1', name: 'read', arguments: '{}' }], - provenance: { provider: 'mock', model: 'mock' }, - }, - surfaceOp: 'append', - }, - { - type: 'tool/call', - seq: 4, - time: 5, - data: { turn: 1, step: 1, callId: 'call-1', name: 'read', arguments: '{}' }, - }, - { - type: 'tool/result', - seq: 5, - time: 6, - data: { - turn: 1, - step: 1, - callId: 'call-1', - content: [{ type: 'text', text: 'full result' }], - isError: false, - }, - sourceEventSeqs: [4], - surfaceOp: 'append', - }, - { - type: 'tool/result', - seq: 6, - time: 8, - data: { - turn: 1, - step: 1, - callId: 'call-1', - content: [{ type: 'text', text: 'pruned' }], - isError: false, - }, - sourceEventSeqs: [5], - surfaceOp: { op: 'replace', start: 5, end: 5 }, - }, - { type: 'step/end', seq: 7, time: 9, data: { turn: 1, step: 1 } }, - { type: 'turn/end', seq: 8, time: 10, data: { turn: 1, reason: { kind: 'completed' } } }, - ] as unknown as SessionEvent[] -} - -/** A complete log in the durable event vocabulary of the react-loop refactor base. */ -export function preReactLoopLog(): SessionEvent[] { - const prompt = createUserMessage({ - content: [{ type: 'text', text: 'old prompt' }], - source: { kind: 'user' }, - }) - const steering = createUserMessage({ - content: [{ type: 'text', text: 'old steering' }], - source: { kind: 'user' }, - }) - return [ - { - type: 'turn/start', seq: 0, time: 1, - data: { turn: 1, trigger: { kind: 'message', source: { kind: 'user' } } }, - }, - { type: 'user/message', seq: 1, time: 2, data: prompt, surfaceOp: 'append' }, - { type: 'step/start', seq: 2, time: 3, data: { turn: 1, step: 1 } }, - { - type: 'steering/message', seq: 3, time: 4, - data: { turn: 1, message: steering }, - surfaceOp: 'append', - }, - { type: 'step/end', seq: 4, time: 5, data: { turn: 1, step: 1 } }, - { type: 'turn/end', seq: 5, time: 6, data: { turn: 1, reason: { kind: 'completed' } } }, - { type: 'turn/start', seq: 6, time: 7, data: { turn: 2, trigger: { kind: 'retry' } } }, - { type: 'step/start', seq: 7, time: 8, data: { turn: 2, step: 1 } }, - { type: 'step/end', seq: 8, time: 9, data: { turn: 2, step: 1 } }, - { - type: 'turn/end', seq: 9, time: 10, - data: { - turn: 2, - reason: { - kind: 'error', - step: 1, - failure: { message: 'old provider failure', code: 'SERVER' }, - }, - }, - }, - { - type: 'turn/start', seq: 10, time: 11, - data: { turn: 3, trigger: { kind: 'message', source: { kind: 'user' } } }, - }, - { type: 'turn/end', seq: 11, time: 12, data: { turn: 3, reason: { kind: 'aborted' } } }, - { - type: 'turn/start', seq: 12, time: 13, - data: { turn: 4, trigger: { kind: 'message', source: { kind: 'user' } } }, - }, - { type: 'turn/end', seq: 13, time: 14, data: { turn: 4, reason: { kind: 'disposed' } } }, - { - type: 'turn/start', seq: 14, time: 15, - data: { turn: 5, trigger: { kind: 'message', source: { kind: 'user' } } }, - }, - { type: 'step/start', seq: 15, time: 16, data: { turn: 5, step: 1 } }, - { type: 'step/end', seq: 16, time: 17, data: { turn: 5, step: 1 } }, - { - type: 'turn/end', seq: 17, time: 18, - data: { turn: 5, reason: { kind: 'error', step: 1, message: 'old thrown value' } }, - }, - { - type: 'turn/start', seq: 18, time: 19, - data: { turn: 6, trigger: { kind: 'message', source: { kind: 'user' } } }, - }, - { - type: 'turn/end', seq: 19, time: 20, - data: { - turn: 6, - reason: { - kind: 'error', - step: 0, - failure: { - message: 'old detailed provider failure', - code: 'RATE_LIMIT', - status: 429, - providerRetryAfterMs: 1000, - requestId: 'request-1', - }, - }, - }, - }, - { - type: 'turn/start', seq: 20, time: 21, - data: { turn: 7, trigger: { kind: 'message', source: { kind: 'user' } } }, - }, - { - type: 'turn/end', seq: 21, time: 22, - data: { turn: 7, reason: { kind: 'error', step: 0, message: 'old coded error', code: 'CODED' } }, - }, - ] as unknown as SessionEvent[] -} - /** A live session created inside its OWN fiber, so it survives a backend reload. */ async function liveSessionInFiber( ctx: Context, id: string, cwd: string | undefined, @@ -456,162 +304,7 @@ export function runCoordinatorContract(name: string, makeFixture: () => Promise< } }) - it('loads pre-identity message logs into resumable current sessions', async () => { - const fix = await makeFixture() - const { ctx, fiber } = await freshCtx(fix) - try { - const id = SessionId('legacy-message-load') - await ctx.sessionPersistence.create(meta(id, WORK)) - await ctx.sessionPersistence.append(id, legacyMessageLog()) - - for (const snapshot of [ - await ctx.sessionPersistence.inspect(id), - await ctx.sessionPersistence.load(id), - ]) { - const messages: { id: string }[] = [] - for (const event of snapshot.events) { - if (event.type === 'user/message') messages.push(event.data) - else if (event.type === 'assistant/message' - || event.type === 'tool/result') messages.push(event.data.message) - } - expect(messages.map(message => message.id)).toEqual([ - `legacy-message:${id}:1`, - `legacy-message:${id}:3`, - `legacy-message:${id}:5`, - `legacy-message:${id}:5`, - ]) - expect(messages.every(message => Object.isFrozen(message))).toBe(true) - - const resumed = Session.create(id, snapshot.events, snapshot.meta) - expect(resumed.deriveMessages().map(message => message.id)).toEqual([ - `legacy-message:${id}:1`, - `legacy-message:${id}:3`, - `legacy-message:${id}:5`, - ]) - } - - const replacementSuffix = await ctx.sessionPersistence.readFrom(id, SessionLogOffset(6)) - expect(replacementSuffix.events[0]).toMatchObject({ - type: 'tool/result', - seq: 6, - data: { message: { id: `legacy-message:${id}:5` } }, - }) - } finally { - await fiber.dispose() - await fix.cleanup() - } - }) - - it('loads pre-react-loop session logs into resumable current sessions', async () => { - const fix = await makeFixture() - const { ctx, fiber } = await freshCtx(fix) - try { - const id = SessionId('pre-react-loop-load') - const log = preReactLoopLog() - const legacySteering = log[3] as unknown as { data: { message: { id: string } } } - await ctx.sessionPersistence.create(meta(id, WORK)) - await ctx.sessionPersistence.append(id, log) - - const snapshots = [ - await ctx.sessionPersistence.inspect(id), - await ctx.sessionPersistence.readFrom(id, SessionLogOffset(0)), - await ctx.sessionPersistence.load(id), - ] - for (const snapshot of snapshots) { - expect(snapshot.events.some(event => (event.type as string) === 'steering/message')).toBe(false) - expect(snapshot.events.filter(event => event.type === 'turn/start').map(event => event.data)) - .toEqual([ - { turn: 1 }, { turn: 2 }, { turn: 3 }, { turn: 4 }, { turn: 5 }, { turn: 6 }, { turn: 7 }, - ]) - expect(snapshot.events.filter(event => event.type === 'turn/end').map(event => event.data)).toEqual([ - { turn: 1, reason: { kind: 'completed' } }, - { - turn: 2, - reason: { kind: 'error', error: { message: 'old provider failure', code: 'SERVER' } }, - }, - { turn: 3, reason: { kind: 'aborted', reason: { kind: 'legacy' } } }, - { turn: 4, reason: { kind: 'aborted', reason: { kind: 'disposed' } } }, - { - turn: 5, - reason: { kind: 'error', error: { message: 'old thrown value', code: 'UNKNOWN' } }, - }, - { - turn: 6, - reason: { - kind: 'error', - error: { - message: 'old detailed provider failure', - code: 'RATE_LIMIT', - status: 429, - providerRetryAfterMs: 1000, - requestId: 'request-1', - }, - }, - }, - { - turn: 7, - reason: { kind: 'error', error: { message: 'old coded error', code: 'CODED' } }, - }, - ]) - - const resumed = Session.create(id, snapshot.events, snapshot.meta) - expect(resumed.deriveMessages().map(message => message.content)).toEqual([ - [{ type: 'text', text: 'old prompt' }], - [{ type: 'text', text: 'old steering' }], - ]) - } - - const suffix = await ctx.sessionPersistence.readFrom(id, SessionLogOffset(3)) - expect(suffix.events[0]).toMatchObject({ - type: 'user/message', - seq: 3, - data: { id: legacySteering.data.message.id }, - }) - expect(suffix.events.filter(event => event.type === 'turn/end') - .every(event => !Object.hasOwn(event.data, 'step'))).toBe(true) - - const flatId = SessionId('pre-react-loop-flat-steering') - await ctx.sessionPersistence.create(meta(flatId, WORK)) - await ctx.sessionPersistence.append(flatId, [{ - type: 'steering/message', - seq: 0, - time: 1, - data: { - turn: 1, - content: [{ type: 'text', text: 'flat steering' }], - source: { kind: 'user' }, - }, - surfaceOp: 'append', - } as unknown as SessionEvent]) - expect((await ctx.sessionPersistence.inspect(flatId)).events[0]).toMatchObject({ - type: 'user/message', - data: { - id: `legacy-message:${flatId}:0`, - role: 'user', - content: [{ type: 'text', text: 'flat steering' }], - }, - }) - - const extendedId = SessionId('current-extended-turn-end') - await ctx.sessionPersistence.create(meta(extendedId, WORK)) - await ctx.sessionPersistence.append(extendedId, [ - { type: 'turn/start', seq: SessionSeq(0), time: 1, data: { turn: 1 } }, - { - type: 'turn/end', seq: 1, time: 2, - data: { turn: 1, reason: { kind: 'extension-reason' } }, - } as unknown as SessionEvent, - ]) - expect((await ctx.sessionPersistence.inspect(extendedId)).events[1]).toMatchObject({ - type: 'turn/end', - data: { reason: { kind: 'extension-reason' } }, - }) - } finally { - await fiber.dispose() - await fix.cleanup() - } - }) - - it('rejects malformed persisted message events before returning them', async () => { + it('rejects malformed current message events before returning them', async () => { const fix = await makeFixture() const { ctx, fiber } = await freshCtx(fix) try { @@ -635,114 +328,6 @@ export function runCoordinatorContract(name: string, makeFixture: () => Promise< await expect(ctx.sessionPersistence.load(id)) .rejects.toThrow('message must have role "user"') - const malformedLegacy: { id: string; event: SessionEvent; message: string }[] = [ - { - id: 'invalid-old-turn-start', - event: { - type: 'turn/start', seq: 0, time: 1, - data: { turn: 1, trigger: null }, - } as unknown as SessionEvent, - message: 'malformed pre-react-loop turn/start', - }, - { - id: 'invalid-old-steering', - event: { - type: 'steering/message', seq: 0, time: 1, surfaceOp: 'append', - data: { turn: 1, content: [], source: { kind: 'user' }, extra: true }, - } as unknown as SessionEvent, - message: 'malformed pre-react-loop steering/message', - }, - { - id: 'invalid-old-steering-data', - event: { - type: 'steering/message', seq: 0, time: 1, surfaceOp: 'append', data: null, - } as unknown as SessionEvent, - message: 'malformed pre-react-loop steering/message', - }, - { - id: 'invalid-old-turn-end', - event: { - type: 'turn/end', seq: 0, time: 1, - data: { turn: 1, reason: { kind: 'completed', extra: true } }, - } as unknown as SessionEvent, - message: 'malformed pre-react-loop turn/end', - }, - { - id: 'invalid-old-turn-end-reason', - event: { - type: 'turn/end', seq: 0, time: 1, - data: { turn: 1, reason: null }, - } as unknown as SessionEvent, - message: 'malformed pre-react-loop turn/end', - }, - { - id: 'unsupported-intermediate-turn-end-step', - event: { - type: 'turn/end', seq: 0, time: 1, - data: { turn: 1, step: 1, reason: { kind: 'completed' } }, - } as unknown as SessionEvent, - message: 'malformed pre-react-loop turn/end', - }, - { - id: 'invalid-old-turn-end-aborted', - event: { - type: 'turn/end', seq: 0, time: 1, - data: { turn: 1, reason: { kind: 'aborted', extra: true } }, - } as unknown as SessionEvent, - message: 'malformed pre-react-loop turn/end', - }, - { - id: 'invalid-old-turn-end-disposed', - event: { - type: 'turn/end', seq: 0, time: 1, - data: { turn: 1, reason: { kind: 'disposed', extra: true } }, - } as unknown as SessionEvent, - message: 'malformed pre-react-loop turn/end', - }, - { - id: 'invalid-old-turn-end-error-step', - event: { - type: 'turn/end', seq: 0, time: 1, - data: { turn: 1, reason: { kind: 'error', step: -1, message: 'bad step' } }, - } as unknown as SessionEvent, - message: 'malformed pre-react-loop turn/end', - }, - { - id: 'invalid-old-turn-end-error-code', - event: { - type: 'turn/end', seq: 0, time: 1, - data: { turn: 1, reason: { kind: 'error', step: 0, message: 'bad code', code: 1 } }, - } as unknown as SessionEvent, - message: 'malformed pre-react-loop turn/end', - }, - ] - for (const malformed of malformedLegacy) { - const malformedId = SessionId(malformed.id) - await ctx.sessionPersistence.create(meta(malformedId, WORK)) - await ctx.sessionPersistence.append(malformedId, [malformed.event]) - await expect(ctx.sessionPersistence.inspect(malformedId)).rejects.toThrow(malformed.message) - await expect(ctx.sessionPersistence.readFrom(malformedId, SessionLogOffset(0))) - .rejects.toThrow(malformed.message) - } - - const malformedReplacementId = SessionId('invalid-old-tool-result-replacement') - await ctx.sessionPersistence.create(meta(malformedReplacementId, WORK)) - await ctx.sessionPersistence.append(malformedReplacementId, [{ - type: 'tool/result', - seq: 0, - time: 1, - surfaceOp: { op: 'replace', start: -1, end: -1 }, - data: { - turn: 1, - step: 1, - callId: 'call', - content: [{ type: 'text', text: 'result' }], - isError: false, - }, - } as unknown as SessionEvent]) - await expect(ctx.sessionPersistence.inspect(malformedReplacementId)) - .rejects.toThrow('invalid replace surfaceOp') - for (const type of ['tool/result'] as const) { const malformedId = SessionId(`invalid-${type}`) await ctx.sessionPersistence.create(meta(malformedId, WORK)) @@ -1100,7 +685,9 @@ export function runCoordinatorContract(name: string, makeFixture: () => Promise< // Disposal is an observe-only notification. Poll storage rather than // assuming the owning fiber awaits the coordinator's detached drain. await vi.waitFor(async () => { - expect((await ctx.sessionPersistence.list()).map(meta => meta.id)).toContain(SessionId('buffered')) + expect((await ctx.sessionPersistence.list()) + .filter(isReadableSessionPersistenceListing) + .map(listing => listing.header.id)).toContain(SessionId('buffered')) }) expect((await ctx.sessionPersistence.load(SessionId('buffered'))).events.map(event => event.seq)).toEqual([0, 1]) @@ -1386,7 +973,9 @@ export function runCoordinatorContract(name: string, makeFixture: () => Promise< const m = meta('empty-batch', WORK) await ctx.sessionPersistence.create(m) await ctx.sessionPersistence.append(m.id, []) - expect((await ctx.sessionPersistence.list()).map(h => h.id)).not.toContain(m.id) + expect((await ctx.sessionPersistence.list()) + .filter(isReadableSessionPersistenceListing) + .map(listing => listing.header.id)).not.toContain(m.id) } finally { await fiber.dispose() await fix.cleanup() @@ -1429,38 +1018,6 @@ export function runCoordinatorContract(name: string, makeFixture: () => Promise< } }) - it('rejects a newer format version on load, naming the upgrade direction', async () => { - const fix = await makeFixture() - const { ctx, fiber } = await freshCtx(fix) - try { - const m = { version: 99, id: SessionId('v99'), createdAt: 1, cwd: WORK, isSeeded: false } - await ctx.sessionPersistence.create(m) - await ctx.sessionPersistence.append(m.id, oneTurnLog()) - const failure = await ctx.sessionPersistence.load(m.id).then(() => undefined, (error: unknown) => error as Error) - expect(failure?.name).toBe('SessionFormatUnsupportedError') - expect(failure?.message).toMatch(/written by a newer harness.*upgrade the harness/) - } finally { - await fiber.dispose() - await fix.cleanup() - } - }) - - it('rejects an older format version on load without claiming an upgrade path', async () => { - const fix = await makeFixture() - const { ctx, fiber } = await freshCtx(fix) - try { - const m = { version: -1, id: SessionId('v-older'), createdAt: 1, cwd: WORK, isSeeded: false } - await ctx.sessionPersistence.create(m) - await ctx.sessionPersistence.append(m.id, oneTurnLog()) - const failure = await ctx.sessionPersistence.load(m.id).then(() => undefined, (error: unknown) => error as Error) - expect(failure?.name).toBe('SessionFormatUnsupportedError') - expect(failure?.message).toMatch(/older than the supported v0.*no upgrade path/) - } finally { - await fiber.dispose() - await fix.cleanup() - } - }) - it('rejects an unknown event type on load unless the event is marked ignorable', async () => { const fix = await makeFixture() const { ctx, fiber } = await freshCtx(fix) @@ -1500,7 +1057,7 @@ export function runCoordinatorContract(name: string, makeFixture: () => Promise< cwd: WORK, parentSession: SessionId('the-parent'), isSeeded: false, - } + } satisfies SessionHeader await ctx.sessionPersistence.create(m) await ctx.sessionPersistence.append(m.id, oneTurnLog()) const loaded = await ctx.sessionPersistence.load(m.id) diff --git a/packages/session/session-persistence/tests/listing.spec.ts b/packages/session/session-persistence/tests/listing.spec.ts new file mode 100644 index 0000000000..212a8eb53b --- /dev/null +++ b/packages/session/session-persistence/tests/listing.spec.ts @@ -0,0 +1,40 @@ +import { describe, expect, it } from 'vitest' +import { SESSION_FORMAT_VERSION, SessionId } from '@deepseek-ai/dsh-session' +import { + isReadableSessionPersistenceListing, + type SessionPersistenceListing, +} from '../src/index.ts' + +const header = { + version: SESSION_FORMAT_VERSION, + id: SessionId('listing'), + createdAt: 1, + isSeeded: false, +} as const + +describe('isReadableSessionPersistenceListing', () => { + it.each([ + { status: 'current', header, storedVersion: 1, targetVersion: 1 }, + { status: 'migration-required', header, storedVersion: 0, targetVersion: 1 }, + ])('accepts $status descriptors carrying current logical headers', (listing) => { + expect(isReadableSessionPersistenceListing(listing)).toBe(true) + }) + + it.each([ + { + status: 'unsupported', + storedVersion: 2, + targetVersion: 1, + location: { kind: 'test', path: '/unsupported/session.jsonl' }, + reason: 'future format', + }, + { + status: 'malformed', + targetVersion: 1, + location: { kind: 'test', path: '/malformed/session.jsonl' }, + reason: 'bad header', + }, + ])('rejects $status descriptors without logical headers', (listing) => { + expect(isReadableSessionPersistenceListing(listing)).toBe(false) + }) +}) diff --git a/packages/session/session-persistence/tests/persistence.spec.ts b/packages/session/session-persistence/tests/persistence.spec.ts index dd002882f7..dde89f4818 100644 --- a/packages/session/session-persistence/tests/persistence.spec.ts +++ b/packages/session/session-persistence/tests/persistence.spec.ts @@ -1,6 +1,7 @@ import { describe, expect, it, vi } from 'vitest' import { Context } from '@deepseek-ai/cordis' import SessionStore, { + SESSION_FORMAT_VERSION, Session, SessionId, SessionLogOffset, @@ -15,13 +16,12 @@ import type { import { DEFAULT_PREPARED_SESSION_CACHE_SIZE, DEFAULT_WRITE_BATCH_MAX_DELAY_MS, MAX_WRITE_BATCH_DELAY_MS, SessionPersistence, SessionPersistenceRevision, PersistenceCoordinator, + SessionFormatUnsupportedError, sessionFormatVersionRefusal, type PersistenceBackend, type SessionEventSuffix, type SessionInspection, type SessionPersistenceSnapshot, - type SessionStorageMetadata, type StoredPrefix, type StoredSuffix, + type SessionRawArtifact, type SessionStorageMetadata, type StoredPrefix, type StoredSuffix, } from '../src/index.ts' import { runPersistenceContract, meta, oneTurnLog } from './contract.ts' -import { - legacyMessageLog, preReactLoopLog, runCoordinatorContract, type CoordinatorFixture, -} from './coordinator-contract.ts' +import { runCoordinatorContract, type CoordinatorFixture } from './coordinator-contract.ts' /** The durable store shape: materialized sessions only (no lazy entries). */ type MemoryStore = Map 0) entry.events.push(...structuredClone(closers) as SessionEvent[]) } - async list(signal?: AbortSignal): Promise { + async list(signal?: AbortSignal): Promise { signal?.throwIfAborted() - return [...this.store.values()].map(e => structuredClone(e.meta)) + return [...this.store.values()].map(entry => ({ + status: 'current', + header: structuredClone(entry.meta), + storedVersion: SESSION_FORMAT_VERSION, + targetVersion: SESSION_FORMAT_VERSION, + })) } async listSnapshots(signal?: AbortSignal): Promise { signal?.throwIfAborted() return [...this.store.values()].map(entry => ({ + status: 'current', header: structuredClone(entry.meta), + storedVersion: SESSION_FORMAT_VERSION, + targetVersion: SESSION_FORMAT_VERSION, revision: memoryRevision(entry), })) } @@ -241,8 +219,19 @@ class ControlledBackend implements PersistenceBackend { appendAttempts = 0 loadAttempts = 0 repairAttempts = 0 + rawReadAttempts = 0 + ensureCurrentAttempts = 0 beforeAppend?: (attempt: number) => Promise + beforeEnsureCurrent?: (attempt: number, signal?: AbortSignal) => Promise beforeLoadStored?: (attempt: number, signal?: AbortSignal) => Promise + beforeReadRawStored?: (attempt: number, signal?: AbortSignal) => Promise + + async ensureCurrent(_id: SessionId, signal?: AbortSignal): Promise { + signal?.throwIfAborted() + const attempt = ++this.ensureCurrentAttempts + await this.beforeEnsureCurrent?.(attempt, signal) + signal?.throwIfAborted() + } /** When set, the declared seek hook delegates here so readFrom exercises it; unset throws (tests set it first). */ seekHook?: ( id: SessionId, @@ -278,6 +267,19 @@ class ControlledBackend implements PersistenceBackend { return entry === undefined ? undefined : memoryRevision(entry) } + async readRawStored(id: SessionId, signal?: AbortSignal): Promise { + const attempt = ++this.rawReadAttempts + await this.beforeReadRawStored?.(attempt, signal) + const entry = this.store.get(id) + if (entry === undefined) return undefined + return { + meta: structuredClone(entry.meta), + inheritedEventCount: SessionLogOffset(entry.inheritedEventCount ?? 0), + filename: 'session.jsonl', + content: `${JSON.stringify(entry.meta)}\n`, + } + } + async appendBatch( storage: SessionStorageMetadata, events: readonly SessionEvent[], @@ -309,8 +311,13 @@ class ControlledBackend implements PersistenceBackend { if (entry !== undefined) entry.events.push(...structuredClone(closers) as SessionEvent[]) } - async list(): Promise { - return [...this.store.values()].map(entry => structuredClone(entry.meta)) + async list(): Promise { + return [...this.store.values()].map(entry => ({ + status: 'current', + header: structuredClone(entry.meta), + storedVersion: SESSION_FORMAT_VERSION, + targetVersion: SESSION_FORMAT_VERSION, + })) } async close(): Promise { @@ -499,6 +506,257 @@ describe('PersistenceCoordinator bounded writes', () => { }) describe('PersistenceCoordinator stored identity', () => { + it('checks a create collision without loading the stored event body', async () => { + const ctx = new Context() + await ctx.plugin(SessionStore) + const backend = new ControlledBackend() + const id = SessionId('create-collision-header-only') + backend.store.set(id, { meta: meta(id), events: oneTurnLog() }) + const revision = vi.spyOn(backend, 'readStoredRevision') + let coordinator!: PersistenceCoordinator + const fiber = await ctx.plugin(Object.assign((inner: Context) => { + coordinator = new PersistenceCoordinator(inner, backend) + }, { inject: ['sessions'] })) + + try { + await expect(coordinator.create(meta(id))).rejects.toThrow(/already has a persisted log/) + expect(revision).toHaveBeenCalledExactlyOnceWith(id) + expect(backend.loadAttempts).toBe(0) + } finally { + await fiber.dispose() + await ctx.fiber.dispose() + } + }) + + it('refuses older and newer stored formats with direction-aware located diagnostics', async () => { + const ctx = new Context() + await ctx.plugin(SessionStore) + const backend = new ControlledBackend() + const located: PersistenceBackend = Object.assign(backend, { + locate: (header: SessionHeader) => ({ + kind: 'memory', + path: `/sessions/${header.id}/session.jsonl`, + }), + }) + const oldId = SessionId('unsupported-old-format') + const futureId = SessionId('unsupported-future-format') + const oldVersion = SESSION_FORMAT_VERSION - 1 + const futureVersion = SESSION_FORMAT_VERSION + 1 + backend.store.set(oldId, { + meta: { ...meta(oldId), version: oldVersion } as unknown as SessionHeader, + events: oneTurnLog(), + }) + backend.store.set(futureId, { + meta: { ...meta(futureId), version: futureVersion } as unknown as SessionHeader, + events: oneTurnLog(), + }) + let coordinator!: PersistenceCoordinator + const fiber = await ctx.plugin(Object.assign((inner: Context) => { + coordinator = new PersistenceCoordinator(inner, located) + }, { inject: ['sessions'] })) + + try { + const oldReason = sessionFormatVersionRefusal(oldId, oldVersion) + const futureReason = sessionFormatVersionRefusal(futureId, futureVersion) + expect(oldReason).toContain('older than the supported') + expect(futureReason).toContain('written by a newer harness') + await expect(coordinator.inspect(oldId)).rejects.toMatchObject({ + name: SessionFormatUnsupportedError.name, + message: `${oldReason} (raw log: /sessions/${oldId}/session.jsonl)`, + location: { kind: 'memory', path: `/sessions/${oldId}/session.jsonl` }, + }) + await expect(coordinator.inspect(futureId)).rejects.toMatchObject({ + name: SessionFormatUnsupportedError.name, + message: `${futureReason} (raw log: /sessions/${futureId}/session.jsonl)`, + location: { kind: 'memory', path: `/sessions/${futureId}/session.jsonl` }, + }) + } finally { + await fiber.dispose() + await ctx.fiber.dispose() + } + }) + + it('fails raw reads when the backend exposes neither current nor fallback artifacts', async () => { + const ctx = new Context() + await ctx.plugin(SessionStore) + const backend = new ControlledBackend() + Object.defineProperty(backend, 'readRawStored', { configurable: true, value: undefined }) + let coordinator!: PersistenceCoordinator + const fiber = await ctx.plugin(Object.assign((inner: Context) => { + coordinator = new PersistenceCoordinator(inner, backend) + }, { inject: ['sessions'] })) + + try { + await expect(coordinator.readRaw(SessionId('raw-unsupported-backend'))) + .rejects.toThrow(/does not expose raw artifacts/) + expect(backend.rawReadAttempts).toBe(0) + } finally { + await fiber.dispose() + await ctx.fiber.dispose() + } + }) + + it('flushes a live owner before reading its current raw artifact', async () => { + const ctx = new Context() + await ctx.plugin(SessionStore) + const backend = new ControlledBackend() + let coordinator!: PersistenceCoordinator + const fiber = await ctx.plugin(Object.assign((inner: Context) => { + coordinator = new PersistenceCoordinator(inner, backend) + }, { inject: ['sessions'] })) + + try { + const session = ctx.sessions.create(SessionId('raw-live-owner')) + session.append('turn/start', { turn: 1 }) + session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) + + await expect(coordinator.readRaw(session.id)).resolves.toMatchObject({ + meta: { id: session.id }, + filename: 'session.jsonl', + }) + expect(backend.store.get(session.id)?.events.map(event => event.seq)).toEqual([0, 1]) + expect(backend.rawReadAttempts).toBe(1) + } finally { + await fiber.dispose() + await ctx.fiber.dispose() + } + }) + + it('serializes raw reads after an in-flight append for the same Session', async () => { + const ctx = new Context() + await ctx.plugin(SessionStore) + const backend = new ControlledBackend() + const id = SessionId('serialized-raw-read') + const appendGate = Promise.withResolvers() + backend.beforeAppend = async () => { await appendGate.promise } + let coordinator!: PersistenceCoordinator + const fiber = await ctx.plugin(Object.assign((inner: Context) => { + coordinator = new PersistenceCoordinator(inner, backend) + }, { inject: ['sessions'] })) + + try { + await coordinator.create(meta(id)) + const append = coordinator.append(id, [{ + type: 'turn/start', + seq: SessionSeq(0), + time: 1, + data: { turn: 1 }, + }]) + await vi.waitFor(() => { expect(backend.appendAttempts).toBe(1) }) + + const raw = coordinator.readRaw(id) + await Promise.resolve() + expect(backend.rawReadAttempts).toBe(0) + + appendGate.resolve(undefined) + await expect(append).resolves.toBeUndefined() + await expect(raw).resolves.toMatchObject({ meta: { id }, filename: 'session.jsonl' }) + expect(backend.rawReadAttempts).toBe(1) + } finally { + appendGate.resolve(undefined) + await fiber.dispose() + await ctx.fiber.dispose() + } + }) + + it('ensures the current generation before every public stored-body read', async () => { + const ctx = new Context() + await ctx.plugin(SessionStore) + const backend = new ControlledBackend() + backend.seekHook = async (id, fromSeq, signal) => { + signal?.throwIfAborted() + const entry = backend.store.get(id) + return entry === undefined + ? undefined + : { + meta: structuredClone(entry.meta), + inheritedEventCount: SessionLogOffset(entry.inheritedEventCount ?? 0), + events: structuredClone(entry.events.filter(event => event.seq >= fromSeq)), + } + } + let coordinator!: PersistenceCoordinator + const fiber = await ctx.plugin(Object.assign((inner: Context) => { + coordinator = new PersistenceCoordinator(inner, backend) + }, { inject: ['sessions'] })) + const stored = (name: string): SessionId => { + const id = SessionId(`ensure-current-${name}`) + backend.store.set(id, { meta: meta(id), events: oneTurnLog() }) + return id + } + const expectOneEnsure = async (read: () => Promise): Promise => { + const before = backend.ensureCurrentAttempts + await read() + expect(backend.ensureCurrentAttempts).toBe(before + 1) + } + + try { + await expectOneEnsure(async () => { + const preparation = await coordinator.prepare(stored('prepare')) + preparation[Symbol.dispose]() + }) + await expectOneEnsure(() => coordinator.load(stored('load'))) + await expectOneEnsure(() => coordinator.inspect(stored('inspect'))) + await expectOneEnsure(async () => { + using observation = await coordinator.borrowSession(stored('borrow')) + expect(observation.source).toBe('prepared') + }) + await expectOneEnsure(() => coordinator.readFrom( + stored('read-from'), + SessionLogOffset(0), + )) + await expectOneEnsure(() => coordinator.readRaw(stored('read-raw'))) + } finally { + await fiber.dispose() + await ctx.fiber.dispose() + } + }) + + it('prefers fused current reads while retaining their cancellation and failure semantics', async () => { + const ctx = new Context() + await ctx.plugin(SessionStore) + const backend = new ControlledBackend() + const fused: PersistenceBackend = backend + const fusedLoad = vi.fn((id: SessionId, signal?: AbortSignal) => backend.loadStored(id, signal)) + const legacyRaw = backend.readRawStored.bind(backend) + const fusedRaw = vi.fn((id: SessionId, signal?: AbortSignal) => legacyRaw(id, signal)) + Object.defineProperties(fused, { + loadStoredFrom: { configurable: true, value: undefined }, + readRawStored: { configurable: true, value: undefined }, + loadCurrentStored: { configurable: true, value: fusedLoad }, + readCurrentRawStored: { configurable: true, value: fusedRaw }, + }) + let coordinator!: PersistenceCoordinator + const fiber = await ctx.plugin(Object.assign((inner: Context) => { + coordinator = new PersistenceCoordinator(inner, fused) + }, { inject: ['sessions'] })) + const stored = (name: string): SessionId => { + const id = SessionId(`fused-current-${name}`) + backend.store.set(id, { meta: meta(id), events: oneTurnLog() }) + return id + } + + try { + const preparation = await coordinator.prepare(stored('prepare')) + preparation[Symbol.dispose]() + await coordinator.readFrom(stored('read-from'), SessionLogOffset(0)) + const signal = new AbortController().signal + await coordinator.readRaw(stored('read-raw'), signal) + + expect(fusedLoad).toHaveBeenCalledTimes(2) + expect(fusedRaw).toHaveBeenCalledTimes(1) + expect(fusedRaw.mock.calls[0]?.[1]).toBe(signal) + expect(backend.ensureCurrentAttempts).toBe(0) + + const failure = new Error('fused read failed') + fusedLoad.mockRejectedValueOnce(failure) + await expect(coordinator.inspect(stored('failure'))).rejects.toBe(failure) + expect(backend.ensureCurrentAttempts).toBe(0) + } finally { + await fiber.dispose() + await ctx.fiber.dispose() + } + }) + it('rejects a mismatched backend header before repair or state publication', async () => { const ctx = new Context() await ctx.plugin(SessionStore) @@ -556,6 +814,110 @@ describe('PersistenceCoordinator stored identity', () => { } }) + it('rejects a live owner whose inherited cut differs from tracked ownerless state', async () => { + const ctx = new Context() + await ctx.plugin(SessionStore) + const backend = new ControlledBackend() + let coordinator!: PersistenceCoordinator + const fiber = await ctx.plugin(Object.assign((inner: Context) => { + coordinator = new PersistenceCoordinator(inner, backend) + }, { inject: ['sessions'] })) + const id = SessionId('ownerless-inherited-cut-mismatch') + const header = { ...meta(id, '/workspace'), isSeeded: true } + + try { + await coordinator.create(header, SessionLogOffset(1)) + const live = ctx.sessions.create(id, { + seed: oneTurnLog(), + inheritedEventCount: SessionLogOffset(2), + meta: { cwd: '/workspace', isSeeded: true }, + }) + + await expect(ctx.sessions.flush(live)).rejects.toThrow(/different inherited event count/) + expect(backend.appendAttempts).toBe(0) + } finally { + await fiber.dispose().catch(() => undefined) + await ctx.fiber.dispose().catch(() => undefined) + } + }) + + it('rejects storage-only live adoption across cwd and inherited-cut identities', async () => { + const ctx = new Context() + await ctx.plugin(SessionStore) + const backend = new ControlledBackend() + const cwdId = SessionId('adopt-cwd-mismatch') + const cutId = SessionId('adopt-inherited-cut-mismatch') + backend.store.set(cwdId, { + meta: meta(cwdId, '/stored-workspace'), + events: oneTurnLog(), + }) + backend.store.set(cutId, { + meta: { ...meta(cutId, '/workspace'), isSeeded: true }, + inheritedEventCount: SessionLogOffset(1), + events: oneTurnLog(), + }) + const fiber = await ctx.plugin(Object.assign((inner: Context) => { + new PersistenceCoordinator(inner, backend) + }, { inject: ['sessions'] })) + + try { + const wrongCwd = ctx.sessions.create(cwdId, { + seed: oneTurnLog(), + meta: { cwd: '/live-workspace' }, + }) + await expect(ctx.sessions.flush(wrongCwd)).rejects.toThrow(/different cwd/) + + const wrongCut = ctx.sessions.create(cutId, { + seed: oneTurnLog(), + inheritedEventCount: SessionLogOffset(2), + meta: { cwd: '/workspace', isSeeded: true }, + }) + await expect(ctx.sessions.flush(wrongCut)).rejects.toThrow(/different inherited event count/) + expect(backend.appendAttempts).toBe(0) + } finally { + await fiber.dispose().catch(() => undefined) + await ctx.fiber.dispose().catch(() => undefined) + } + }) + + it('commits a storage-only torn marker before publishing a live owner', async () => { + const ctx = new Context() + await ctx.plugin(SessionStore) + const backend = new ControlledBackend() + const id = SessionId('adopt-torn-marker') + backend.store.set(id, { meta: meta(id), events: oneTurnLog() }) + const loadStored = backend.loadStored.bind(backend) + const commitRepair = backend.commitRepair.bind(backend) + const tornBackend = backend as unknown as PersistenceBackend + tornBackend.loadStored = async (storedId, signal) => { + const stored = await loadStored(storedId, signal) + return stored === undefined ? undefined : { ...stored, tornMarker: 'torn-tail' } + } + const repair = vi.fn(async ( + storage: SessionStorageMetadata, + marker: string | undefined, + closers: readonly SessionEvent[], + ) => { + expect(marker).toBe('torn-tail') + expect(closers).toEqual([]) + await commitRepair(storage, undefined, closers) + }) + tornBackend.commitRepair = repair + const fiber = await ctx.plugin(Object.assign((inner: Context) => { + new PersistenceCoordinator(inner, tornBackend) + }, { inject: ['sessions'] })) + + try { + const live = ctx.sessions.create(id, { seed: oneTurnLog(), meta: meta(id) }) + await expect(ctx.sessions.flush(live)).resolves.toBe(true) + expect(repair).toHaveBeenCalledOnce() + expect(backend.store.get(id)?.events.at(-1)).toMatchObject({ type: 'session/end-seed' }) + } finally { + await fiber.dispose() + await ctx.fiber.dispose() + } + }) + it('reserves a cold id across asynchronous storage repair', async () => { const ctx = new Context() await ctx.plugin(SessionStore) @@ -1298,82 +1660,6 @@ describe('PersistenceCoordinator session preparations', () => { }) }) -describe('PersistenceCoordinator seek reads', () => { - it('loads the whole prefix only when a bounded legacy suffix needs earlier message identities', async () => { - const ctx = new Context() - await ctx.plugin(SessionStore) - const backend = new ControlledBackend() - const id = SessionId('seek-read-from-legacy') - let coordinator!: PersistenceCoordinator - const fiber = await ctx.plugin(Object.assign((inner: Context) => { - coordinator = new PersistenceCoordinator(inner, backend) - }, { inject: ['sessions'] })) - backend.seekHook = async (hookId, fromSeq) => { - const entry = backend.store.get(hookId) - if (entry === undefined) return undefined - return { - meta: structuredClone(entry.meta), - inheritedEventCount: SessionLogOffset(entry.inheritedEventCount ?? 0), - events: entry.events.filter(e => e.seq >= fromSeq), - } - } - - try { - const assertLegacySuffixUsesWholePrefix = async ( - events: SessionEvent[], - fromSeq: SessionLogOffsetType, - firstType: SessionEvent['type'], - ): Promise => { - backend.store.set(id, { meta: meta(id), events }) - const loadsBefore = backend.loadAttempts - const result = await coordinator.readFrom(id, fromSeq) - expect(result.events[0]?.type).toBe(firstType) - expect(backend.loadAttempts).toBe(loadsBefore + 1) - } - const legacyMessages = legacyMessageLog() - await assertLegacySuffixUsesWholePrefix(legacyMessages, SessionLogOffset(1), 'user/message') - await assertLegacySuffixUsesWholePrefix(legacyMessages, SessionLogOffset(3), 'assistant/message') - await assertLegacySuffixUsesWholePrefix(legacyMessages, SessionLogOffset(5), 'tool/result') - await assertLegacySuffixUsesWholePrefix(preReactLoopLog(), SessionLogOffset(3), 'user/message') - - backend.store.set(id, { meta: meta(id), events: legacyMessages }) - const directCurrent = await coordinator.readFrom(id, SessionLogOffset(0)) - backend.store.set(id, { meta: meta(id), events: [...directCurrent.events] }) - const loadsBeforeCurrent = backend.loadAttempts - await coordinator.readFrom(id, SessionLogOffset(1)) - await coordinator.readFrom(id, SessionLogOffset(3)) - await coordinator.readFrom(id, SessionLogOffset(5)) - expect(backend.loadAttempts).toBe(loadsBeforeCurrent) - - for (const [type, data] of [ - ['user/message', {}], - ['assistant/message', {}], - ['tool/result', {}], - ] as const) { - backend.store.set(id, { - meta: meta(id), - events: [{ type, seq: 0, time: 1, data } as unknown as SessionEvent], - }) - const loadsBefore = backend.loadAttempts - await expect(coordinator.readFrom(id, SessionLogOffset(0))).rejects.toThrow('lacks an identified message') - expect(backend.loadAttempts).toBe(loadsBefore) - } - backend.store.set(id, { - meta: meta(id), - events: [{ - type: 'external/null', seq: 0, time: 1, data: null, ignorable: true, - } as unknown as SessionEvent], - }) - const loadsBeforeNullData = backend.loadAttempts - expect((await coordinator.readFrom(id, SessionLogOffset(0))).events[0]?.data).toBeNull() - expect(backend.loadAttempts).toBe(loadsBeforeNullData) - } finally { - await fiber.dispose() - await ctx.fiber.dispose() - } - }) -}) - describe('PersistenceCoordinator observation cancellation', () => { it('borrows live Sessions before, during, and after cold source validation', async () => { const ctx = new Context() @@ -1585,8 +1871,8 @@ describe('PersistenceCoordinator observation cancellation', () => { const backend = new ControlledBackend() const id = SessionId('creating-inspect-cancellation') backend.store.set(id, { meta: meta(id), events: oneTurnLog() }) - const loadGate = Promise.withResolvers() - backend.beforeLoadStored = () => loadGate.promise.then(() => undefined) + const migrationGate = Promise.withResolvers() + backend.beforeEnsureCurrent = () => migrationGate.promise.then(() => undefined) let coordinator!: PersistenceCoordinator const fiber = await ctx.plugin(Object.assign((inner: Context) => { coordinator = new PersistenceCoordinator(inner, backend) @@ -1597,17 +1883,18 @@ describe('PersistenceCoordinator observation cancellation', () => { const controller = new AbortController() const reason = new Error('creating inspect cancelled') const inspection = coordinator.inspect(id, controller.signal) - await vi.waitFor(() => { expect(backend.loadAttempts).toBe(1) }) + await vi.waitFor(() => { expect(backend.ensureCurrentAttempts).toBe(1) }) const reservation = coordinator.prepare(id) controller.abort(reason) await expect(inspection).rejects.toBe(reason) - loadGate.resolve(true) + migrationGate.resolve(true) prepared = await reservation expect(prepared.session.id).toBe(id) + expect(backend.ensureCurrentAttempts).toBe(1) expect(backend.loadAttempts).toBe(1) } finally { - loadGate.resolve(true) + migrationGate.resolve(true) prepared?.[Symbol.dispose]() await fiber.dispose() await ctx.fiber.dispose() @@ -1751,6 +2038,55 @@ describe('PersistenceCoordinator observation cancellation', () => { }) describe('PersistenceCoordinator retirement', () => { + it('queues uncancelled raw and borrowed reads while cancelling one raw observer', async () => { + const ctx = new Context() + await ctx.plugin(SessionStore) + const backend = new ControlledBackend() + let coordinator!: PersistenceCoordinator + const backendFiber = await ctx.plugin(Object.assign((inner: Context) => { + coordinator = new PersistenceCoordinator(inner, backend) + }, { inject: ['sessions'] })) + const internals = coordinator as unknown as CoordinatorInternals + const appendGate = Promise.withResolvers() + backend.beforeAppend = async () => { await appendGate.promise } + let borrowed: Awaited> | undefined + + try { + const id = SessionId('retiring-raw-read') + let session!: Session + const sessionFiber = await ctx.plugin(Object.assign((inner: Context) => { + session = inner.sessions.create(id) + }, { inject: ['sessions'] })) + session.append('turn/start', { turn: 1 }) + session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) + await sessionFiber.dispose() + await vi.waitFor(() => { expect(internals.retirements.has(id)).toBe(true) }) + await vi.waitFor(() => { expect(backend.appendAttempts).toBe(1) }) + + const raw = coordinator.readRaw(id) + const borrow = coordinator.borrowSession(id) + const controller = new AbortController() + const reason = new Error('raw observer cancelled during retirement') + const cancelled = coordinator.readRaw(id, controller.signal) + controller.abort(reason) + + await expect(cancelled).rejects.toBe(reason) + expect(backend.rawReadAttempts).toBe(0) + expect(backend.loadAttempts).toBe(1) + + appendGate.resolve(true) + await expect(raw).resolves.toMatchObject({ meta: { id }, filename: 'session.jsonl' }) + borrowed = await borrow + expect(borrowed).toMatchObject({ source: 'prepared', inspection: { meta: { id } } }) + expect(backend.rawReadAttempts).toBe(1) + } finally { + appendGate.resolve(true) + borrowed?.[Symbol.dispose]() + await backendFiber.dispose() + await ctx.fiber.dispose() + } + }) + it('a retiring unmaterialized owner without buffered events releases its id', async () => { const ctx = new Context() await ctx.plugin(SessionStore) @@ -2134,7 +2470,12 @@ describe('SessionPersistence service registration', () => { await ctx.sessionPersistence.ensureMaterialized(session) await ctx.sessionPersistence.ensureMaterialized(session) - await expect(ctx.sessionPersistence.list()).resolves.toEqual([session.header]) + await expect(ctx.sessionPersistence.list()).resolves.toEqual([{ + status: 'current', + header: session.header, + storedVersion: SESSION_FORMAT_VERSION, + targetVersion: SESSION_FORMAT_VERSION, + }]) await expect(ctx.sessionPersistence.load(session.id)).resolves.toEqual({ meta: session.header, inheritedEventCount: SessionLogOffset(0), @@ -2272,77 +2613,6 @@ describe('SessionPersistence service registration', () => { await fiber.dispose() }) - it('rejects a legacy header delta from a pre-change live producer', async () => { - const ctx = new Context() - await ctx.plugin(SessionStore) - const fiber = await ctx.plugin(MemoryPersistence) - const session = ctx.sessions.create(SessionId('legacy-live'), { meta: { cwd: '/legacy' } }) - // Model the runtime shape available to JavaScript or a hot-loaded plugin - // compiled against the obsolete event vocabulary. - const appendLegacy = session.append.bind(session) as (type: string, data: unknown) => SessionEvent - expect(() => appendLegacy('request/header-delta', { config: { model: 'legacy' } })) - .toThrow(/unsupported legacy request\/header-delta format/) - expect(session.snapshotEvents()).toHaveLength(0) - await fiber.dispose() - }) - - it('rejects a legacy fallback header buffered by a pre-change live producer', async () => { - const ctx = new Context() - await ctx.plugin(SessionStore) - const fiber = await ctx.plugin(MemoryPersistence) - const session = ctx.sessions.create(SessionId('legacy-fallback-live'), { meta: { cwd: '/legacy' } }) - const appendLegacy = session.append.bind(session) as (type: string, data: unknown) => SessionEvent - - expect(() => appendLegacy('request/header', legacyFallbackHeader().data)) - .toThrow('unsupported legacy request/header reason "fallback"') - expect(session.snapshotEvents()).toHaveLength(0) - await fiber.dispose() - }) - - it('rejects a legacy stored prefix during live HMR adoption', async () => { - const id = SessionId('legacy-hmr') - const m = meta(id, '/legacy') - const legacy = legacyHeaderDelta() - const store: MemoryStore = new Map([[id, { meta: m, events: [legacy] }]]) - const ctx = new Context() - await ctx.plugin(SessionStore) - // A current live session cannot carry the obsolete event in its seed, but - // HMR still has to identify the persisted prefix as unsupported rather than - // treating it as an ordinary live-prefix collision. - const session = ctx.sessions.create(id, { meta: { cwd: '/legacy' } }) - const fiber = await ctx.plugin(MemoryPersistence, { store }) - - await expect(ctx.sessions.flush(session)) - .rejects.toThrow(/unsupported legacy request\/header-delta event at seq 0/) - await Promise.allSettled([fiber.dispose()]) - }) - - it('rejects a stored legacy fallback header during load', async () => { - const id = SessionId('legacy-fallback-load') - const m = meta(id, '/legacy') - const store: MemoryStore = new Map([[id, { meta: m, events: [legacyFallbackHeader()] }]]) - const ctx = new Context() - await ctx.plugin(SessionStore) - const fiber = await ctx.plugin(MemoryPersistence, { store }) - - await expect(ctx.sessionPersistence.load(id)) - .rejects.toThrow('unsupported legacy request/header reason "fallback" at seq 0') - await fiber.dispose() - }) - - it('rejects a stored legacy named-mode event during load', async () => { - const id = SessionId('legacy-mode-load') - const m = meta(id, '/legacy') - const store: MemoryStore = new Map([[id, { meta: m, events: [legacyModeSet()] }]]) - const ctx = new Context() - await ctx.plugin(SessionStore) - const fiber = await ctx.plugin(MemoryPersistence, { store }) - - await expect(ctx.sessionPersistence.load(id)) - .rejects.toThrow('unsupported legacy mode/set event at seq 0') - await fiber.dispose() - }) - it('retires all coordinator bookkeeping for disposed sessions', async () => { const ctx = new Context() await ctx.plugin(SessionStore) diff --git a/packages/session/session-projection-cache/README.i18n.yaml b/packages/session/session-projection-cache/README.i18n.yaml index e4fe8490a2..6e704de60a 100644 --- a/packages/session/session-projection-cache/README.i18n.yaml +++ b/packages/session/session-projection-cache/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/session/session-projection-cache/README.md -README.md: 51b9d86724af96304cf09a5c7c1b61b7394d336a -README.zh.md: bb84b67bdde678884fd4f2be1b14b2161da8c2a0 +README.md: 8b519a811834aa245fdafacd4d93a1b951efdcd7 +README.zh.md: ed53309b320b43cf050ae54cd687b7cc7435bd9a diff --git a/packages/session/session-projection-cache/README.md b/packages/session/session-projection-cache/README.md index 51b9d86724..8b519a8118 100644 --- a/packages/session/session-projection-cache/README.md +++ b/packages/session/session-projection-cache/README.md @@ -62,7 +62,7 @@ Three mandatory points always write: session creation persists the seed-derived ### What the cache guarantees -The log leads and the cache follows: a live checkpoint flushes the session's buffered events durably before the cache row lands, so a crash can leave the cache behind the log but never ahead of it. Reads and writes share the storage domain's coherent in-memory state; the per-unit write chain mutates memory only after durability. Each version-stamped record must match the live unit schema and complete lifecycle identity (`createdAt`, `cwd`, `isSeeded`, and `inheritedEventCount`), so a row initialized under one fork cut cannot seed another. The JSON backend stores each record at `/session_projcache/sessions/.json` in an owner-only directory tree. +The log leads and the cache follows: a live checkpoint flushes the session's buffered events durably before the cache row lands, so a crash can leave the cache behind the log but never ahead of it. Reads and writes share the storage domain's coherent in-memory state; the per-unit write chain mutates memory only after durability. Each version-stamped record must match the live unit schema and complete lifecycle identity (`formatVersion`, `createdAt`, `cwd`, `isSeeded`, and `inheritedEventCount`), so a row folded from another Session format generation or fork cut cannot seed the caller. The JSON backend stores each record at `/session_projcache/sessions/.json` in an owner-only directory tree. ----- diff --git a/packages/session/session-projection-cache/README.zh.md b/packages/session/session-projection-cache/README.zh.md index bb84b67bdd..ed53309b32 100644 --- a/packages/session/session-projection-cache/README.zh.md +++ b/packages/session/session-projection-cache/README.zh.md @@ -62,7 +62,7 @@ kind: "package-reference" ### 缓存保证什么 -日志领先，缓存跟随：实时检查点先把会话的缓冲事件持久化，然后才保存缓存记录。因此崩溃可能让缓存落后于日志，但绝不会让缓存领先。读取和写入共享存储域内一致的内存状态；逐单元写入链只在持久化成功后修改内存。每个带版本戳的记录必须匹配实时单元 schema 与完整生命周期身份（`createdAt`、`cwd`、`isSeeded` 和 `inheritedEventCount`），因此在一个 fork 切点下初始化的行不能播种另一个切点。JSON 后端把每条记录存于仅所有者可访问的 `/session_projcache/sessions/.json` 目录树中。 +日志领先，缓存跟随：实时检查点先把会话的缓冲事件持久化，然后才保存缓存记录。因此崩溃可能让缓存落后于日志，但绝不会让缓存领先。读取和写入共享存储域内一致的内存状态；逐单元写入链只在持久化成功后修改内存。每个带版本戳的记录必须匹配实时单元 schema 与完整生命周期身份（`formatVersion`、`createdAt`、`cwd`、`isSeeded` 和 `inheritedEventCount`），因此从另一会话格式代或 fork 切点折叠出的行不能播种调用方。JSON 后端把每条记录存于仅所有者可访问的 `/session_projcache/sessions/.json` 目录树中。 ----- diff --git a/packages/session/session-projection-cache/src/index.ts b/packages/session/session-projection-cache/src/index.ts index 35bf5d9eed..f7205a99b6 100644 --- a/packages/session/session-projection-cache/src/index.ts +++ b/packages/session/session-projection-cache/src/index.ts @@ -25,7 +25,6 @@ import type { SessionEvent, SessionHeader, SessionId, - SessionSeqCursor, } from '@deepseek-ai/dsh-session' import type { ProjectionCheckpoint, @@ -147,15 +146,12 @@ export class SessionProjectionCache extends Service { // The block carries ONE cut: the lowest served watermark is the seq every // value is at least current as of (under-claiming is safe under // higher-seq-wins; over-claiming would let a stale value outrank pushes). - let asOfSeq: SessionSeqCursor | undefined - for (const key of servedKeys) { - const row = record.rows[key] - if (row !== undefined && (asOfSeq === undefined || row.seq < asOfSeq)) { - asOfSeq = row.seq - } + const firstKey = servedKeys[0] as string + let asOfSeq = (record.rows[firstKey] as ProjectionCheckpoint[string]).seq + for (const key of servedKeys.slice(1)) { + const row = record.rows[key] as ProjectionCheckpoint[string] + if (row.seq < asOfSeq) asOfSeq = row.seq } - /* v8 ignore next -- A nonempty checkpoint view contains a stored row for every returned key. */ - if (asOfSeq === undefined) return undefined return { asOfSeq, values } } @@ -360,6 +356,7 @@ function identityOf( throw new Error('unseeded projection-cache identity inherited event count must be 0') } return { + formatVersion: header.version, createdAt: header.createdAt, ...header.cwd === undefined ? {} : { cwd: header.cwd }, isSeeded: header.isSeeded, @@ -369,7 +366,8 @@ function identityOf( /** Whether a stored record's bound identity names the caller's lifecycle. */ function identityMatches(stored: CheckpointIdentity, expected: CheckpointIdentity): boolean { - return stored.createdAt === expected.createdAt + return stored.formatVersion === expected.formatVersion + && stored.createdAt === expected.createdAt && stored.cwd === expected.cwd && stored.isSeeded === expected.isSeeded && stored.inheritedEventCount === expected.inheritedEventCount diff --git a/packages/session/session-projection-cache/src/spec.ts b/packages/session/session-projection-cache/src/spec.ts index 35351917a5..222b4cf6f4 100644 --- a/packages/session/session-projection-cache/src/spec.ts +++ b/packages/session/session-projection-cache/src/spec.ts @@ -40,6 +40,7 @@ export const checkpointRow = z.object({ * the stored header (cold read) before accepting any record. */ export const checkpointIdentity = z.object({ + formatVersion: z.number().int().nonnegative(), createdAt: z.number().int().nonnegative(), cwd: z.string().optional(), isSeeded: z.boolean(), @@ -72,7 +73,7 @@ export type CheckpointRecord = z.infer */ export const projectionCacheDomainSpec = defineDomain({ name: 'session_projcache', - version: 5, + version: 6, layout: 'per-record', tables: { sessions: domainTable(checkpointRecord) }, }) diff --git a/packages/session/session-projection-cache/tests/cache.spec.ts b/packages/session/session-projection-cache/tests/cache.spec.ts index bfdca18567..d9dce619be 100644 --- a/packages/session/session-projection-cache/tests/cache.spec.ts +++ b/packages/session/session-projection-cache/tests/cache.spec.ts @@ -17,12 +17,13 @@ import { dirname, join } from 'node:path' import { Context } from '@deepseek-ai/cordis' import { z } from 'zod' import SessionStore, { + SESSION_FORMAT_VERSION, Session, SessionId, SessionLogOffset, SessionSeq, } from '@deepseek-ai/dsh-session' -import type { SessionEvent } from '@deepseek-ai/dsh-session' +import type { SessionEvent, SessionHeader } from '@deepseek-ai/dsh-session' import SessionProjectionRegistry from '@deepseek-ai/dsh-session-projection' import type { ProjectionDefinition } from '@deepseek-ai/dsh-session-projection' import Storage from '@deepseek-ai/dsh-storage' @@ -39,12 +40,14 @@ import type { CheckpointRecord } from '../src/spec.ts' declare module '@deepseek-ai/dsh-session-projection/types' { interface SessionProjectionStateMap { 'cache-test/marks': MarksState + 'cache-test/secondary-marks': MarksState 'cache-test/marks2': Map 'cache-test/count': number 'cache-test/secret': string } interface SessionProjectionMap { 'cache-test/marks': { marks: string[] } + 'cache-test/secondary-marks': { marks: string[] } } } @@ -79,13 +82,25 @@ const secretUnit = { stateVersion: 1, } satisfies ProjectionDefinition<'cache-test/secret', string> +const secondaryMarksUnit = { + key: 'cache-test/secondary-marks', + stateSchema: z.object({ marks: z.array(z.string()) }).nullable(), + init: () => null, + apply: (state, event) => event.type === 'cache-test/mark' ? event.data : state, + wire: { + viewSchema: z.object({ marks: z.array(z.string()) }), + view: state => state ?? { marks: [] }, + }, + stateVersion: 1, +} satisfies ProjectionDefinition<'cache-test/secondary-marks', MarksState> + /** One session's record document on the per-record medium. */ const recordPath = (root: string, id: Session['id']): string => join(root, projectionCacheDomainSpec.name, 'sessions', `${String(id)}.json`) /** Header shape for cachedSnapshot calls. */ -const headerOf = (id: SessionId, createdAt = 0, cwd?: string) => - ({ version: 0, id, createdAt, isSeeded: false, ...cwd === undefined ? {} : { cwd } }) +const headerOf = (id: SessionId, createdAt = 0, cwd?: string): SessionHeader => + ({ version: SESSION_FORMAT_VERSION, id, createdAt, isSeeded: false, ...cwd === undefined ? {} : { cwd } }) interface HarnessOptions { root?: string @@ -140,6 +155,7 @@ async function seedRecord( id: string, rows: CheckpointRecord['rows'], identity: CheckpointRecord['identity'] = { + formatVersion: SESSION_FORMAT_VERSION, createdAt: 0, isSeeded: false, inheritedEventCount: SessionLogOffset(0), @@ -209,7 +225,8 @@ describe('SessionProjectionCache write policy', () => { mark(session, ['1']) mark(session, ['2']) await vi.waitFor(async () => { - expect((await storedRows(root, session.id))?.['cache-test/marks']?.seq).toBe(-1) // still the creation cut + expect((await storedRows(root, session.id))?.['cache-test/marks']) + .toEqual({ ver: 1, seq: -1, val: null }) // still the creation cut }, { timeout: 5_000 }) mark(session, ['3']) await vi.waitFor(async () => { @@ -303,6 +320,35 @@ describe('SessionProjectionCache write policy', () => { }) describe('SessionProjectionCache listing read', () => { + it('rejects a nonzero inherited cut for an unseeded header', async () => { + const { cache } = await harness() + + expect(() => cache.cachedSnapshot( + headerOf(SessionId('invalid-unseeded-cut')), + SessionLogOffset(1), + )).toThrow('unseeded projection-cache identity inherited event count must be 0') + }) + + it('uses the lowest watermark across every served wire row', async () => { + const root = await mkdtemp(join(tmpdir(), 'dsh-projcache-')) + roots.push(root) + await seedRecord(root, 'watermark-lower', { + 'cache-test/marks': { ver: 1, seq: SessionSeq(4), val: { marks: ['primary'] } }, + 'cache-test/secondary-marks': { ver: 1, seq: SessionSeq(2), val: { marks: ['secondary'] } }, + }) + await seedRecord(root, 'watermark-higher', { + 'cache-test/marks': { ver: 1, seq: SessionSeq(4), val: { marks: ['primary'] } }, + 'cache-test/secondary-marks': { ver: 1, seq: SessionSeq(6), val: { marks: ['secondary'] } }, + }) + const { ctx, cache } = await harness({ root }) + ctx.sessionProjections.register(secondaryMarksUnit) + + expect(cache.cachedSnapshot(headerOf(SessionId('watermark-lower')), SessionLogOffset(0))?.asOfSeq) + .toBe(2) + expect(cache.cachedSnapshot(headerOf(SessionId('watermark-higher')), SessionLogOffset(0))?.asOfSeq) + .toBe(4) + }) + it('refuses a checkpoint created for a different inherited cut', async () => { const root = await mkdtemp(join(tmpdir(), 'dsh-projcache-')) roots.push(root) @@ -312,6 +358,7 @@ describe('SessionProjectionCache listing read', () => { id, { 'cache-test/marks': { ver: 1, seq: SessionSeq(1), val: { marks: ['seed'] } } }, { + formatVersion: SESSION_FORMAT_VERSION, createdAt: 0, isSeeded: true, inheritedEventCount: SessionLogOffset(2), @@ -339,6 +386,26 @@ describe('SessionProjectionCache listing read', () => { .toEqual({ asOfSeq: -1, values: { 'cache-test/marks': { marks: [] } } }) }) + it('refuses a checkpoint folded from another Session format generation', async () => { + const root = await mkdtemp(join(tmpdir(), 'dsh-projcache-')) + roots.push(root) + const id = SessionId('format-identity') + await seedRecord( + root, + id, + { 'cache-test/marks': { ver: 1, seq: SessionSeq(0), val: { marks: ['stale'] } } }, + { + formatVersion: SESSION_FORMAT_VERSION + 1, + createdAt: 0, + isSeeded: false, + inheritedEventCount: SessionLogOffset(0), + }, + ) + const { cache } = await harness({ root }) + + expect(cache.cachedSnapshot(headerOf(id), SessionLogOffset(0))).toBeUndefined() + }) + it('keeps host-only checkpoint state out of cached wire snapshots', async () => { const root = await mkdtemp(join(tmpdir(), 'dsh-projcache-')) roots.push(root) @@ -385,7 +452,12 @@ describe('SessionProjectionCache listing read', () => { await writeFile(path, JSON.stringify({ version: projectionCacheDomainSpec.version + 1, record: { - identity: { createdAt: 0, isSeeded: false, inheritedEventCount: 0 }, + identity: { + formatVersion: SESSION_FORMAT_VERSION, + createdAt: 0, + isSeeded: false, + inheritedEventCount: 0, + }, rows: { 'cache-test/marks': { ver: 1, seq: 4, val: { marks: ['old'] } } }, }, })) @@ -413,6 +485,7 @@ describe('SessionProjectionCache listing read', () => { await seedRecord(root, 'homed', { 'cache-test/marks': { ver: 1, seq: SessionSeq(2), val: { marks: ['w'] } }, }, { + formatVersion: SESSION_FORMAT_VERSION, createdAt: 0, cwd: '/work', isSeeded: false, @@ -508,6 +581,7 @@ describe('SessionProjectionCache cold-read seeding', () => { await seedRecord(root, 'cold-snap', { 'cache-test/count': { ver: 1, seq: SessionSeq(2), val: 3 }, }, { + formatVersion: SESSION_FORMAT_VERSION, createdAt: 9, isSeeded: false, inheritedEventCount: SessionLogOffset(0), diff --git a/packages/session/session-projection/tests/registry.spec.ts b/packages/session/session-projection/tests/registry.spec.ts index 91106860c4..0954b2daf5 100644 --- a/packages/session/session-projection/tests/registry.spec.ts +++ b/packages/session/session-projection/tests/registry.spec.ts @@ -11,6 +11,7 @@ import { describe, expect, it, vi } from 'vitest' import { Context } from '@deepseek-ai/cordis' import { z } from 'zod' import SessionStore, { + SESSION_FORMAT_VERSION, Session, SessionId, SessionLogOffset, @@ -50,7 +51,7 @@ interface StableViewState { } const marksViewSchema: z.ZodType = z.object({ marks: z.array(z.string()) }) const RESTORE_HEADER: SessionHeader = { - version: 0, + version: SESSION_FORMAT_VERSION, id: SessionId('projection-restore'), createdAt: 0, isSeeded: false, diff --git a/packages/session/session-telemetry-otel/README.i18n.yaml b/packages/session/session-telemetry-otel/README.i18n.yaml index 9bdf7f5cfc..cf5550a799 100644 --- a/packages/session/session-telemetry-otel/README.i18n.yaml +++ b/packages/session/session-telemetry-otel/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/session/session-telemetry-otel/README.md -README.md: 2f489cbdae9a64ac95484cf6ed3aeddac3fe2b1e -README.zh.md: 78bc221542d9709acd287f258c54b930ac389106 +README.md: 8ae8d0091ef1a92cd9caf0b57dec5ea56d9f6ba3 +README.zh.md: 3edf91a812d42a14b4a39de53079ab89dc72bd1c diff --git a/packages/session/session-telemetry-otel/README.md b/packages/session/session-telemetry-otel/README.md index 2f489cbdae..8ae8d0091e 100644 --- a/packages/session/session-telemetry-otel/README.md +++ b/packages/session/session-telemetry-otel/README.md @@ -31,8 +31,8 @@ Mount this plugin when a deployment should export session records through OpenTe | `mode` | Behavior | |---|---| -| `FULL` | Every projected record, including lifecycle ops records, is handed to the OTel SDK immediately | -| `FEEDBACK_ONLY` | Each `feedback/record` replays, projects, and redacts the canonical session-log suffix through that event; later records wait for another feedback event and remain local if none arrives | +| `FULL` | Every captured record, including every canonical event and lifecycle ops record, is handed to the OTel SDK immediately | +| `FEEDBACK_ONLY` | Each `feedback/record` replays, copies, and redacts every canonical event after the handoff cursor through that event; later records wait for another feedback event and remain local if none arrives | | `DISABLED` | Default. No coordinator, provider, processor, or exporter is constructed; no telemetry record leaves the process, and a `feedback/record` logs that nothing will be shared | Programmatic TypeScript configuration uses the exported `SessionTelemetryMode` enum; raw string literals are not assignable. The mounted service discloses the resolved mode through the seam's [`SessionTelemetrySharingStatus`](../session-telemetry/README.md#the-sharing-disclosure) `sharing` property (`full` / `feedback-only` / `disabled`), so the `/feedback` acknowledgement reports whether and how the session is shared — even `DISABLED` discloses `disabled`. diff --git a/packages/session/session-telemetry-otel/README.zh.md b/packages/session/session-telemetry-otel/README.zh.md index 78bc221542..3edf91a812 100644 --- a/packages/session/session-telemetry-otel/README.zh.md +++ b/packages/session/session-telemetry-otel/README.zh.md @@ -31,8 +31,8 @@ kind: "package-reference" | `mode` | 行为 | |---|---| -| `FULL` | 每条已投影记录都立即交给 OTel SDK，包括生命周期运维记录 | -| `FEEDBACK_ONLY` | 每个 `feedback/record` 都会回放权威会话日志中截至该事件的后缀，并进行投影与脱敏；后续记录等待下一个反馈事件；如果没有后续反馈，则留在本地 | +| `FULL` | 每条已捕获记录都立即交给 OTel SDK，包括每条权威事件与生命周期运维记录 | +| `FEEDBACK_ONLY` | 每个 `feedback/record` 都会回放、复制并脱敏 handoff 游标之后直至该事件的每条权威事件；后续记录等待下一个反馈事件；如果没有后续反馈，则留在本地 | | `DISABLED` | 默认值。不构造协调器、提供方、处理器或导出器；没有遥测记录会离开进程，`feedback/record` 会记录「不会共享任何内容」 | 程序化 TypeScript 配置使用导出的 `SessionTelemetryMode` 枚举；原始字符串字面量不可赋值。已挂载服务通过 seam 的 [`SessionTelemetrySharingStatus`](../session-telemetry/README.zh.md#the-sharing-disclosure) `sharing` 属性披露解析后的模式（`full` / `feedback-only` / `disabled`），因此 `/feedback` 的确认文本可以报告会话是否以及如何被共享——即使 `DISABLED` 也会披露 `disabled`。 diff --git a/packages/session/session-telemetry-otel/tests/otel.spec.ts b/packages/session/session-telemetry-otel/tests/otel.spec.ts index 0888cf2d05..0851a4c676 100644 --- a/packages/session/session-telemetry-otel/tests/otel.spec.ts +++ b/packages/session/session-telemetry-otel/tests/otel.spec.ts @@ -16,7 +16,7 @@ import { Context } from '@deepseek-ai/cordis' import { getOrCreateAnonymousUserId } from '@deepseek-ai/dsh-anonymous-user-id' import Loader from '@deepseek-ai/cordis-plugin-loader' import { recordFeedback } from '@deepseek-ai/dsh-command-feedback' -import SessionStore, { SessionId, SessionSeq } from '@deepseek-ai/dsh-session' +import SessionStore, { SESSION_FORMAT_VERSION, SessionId, SessionSeq } from '@deepseek-ai/dsh-session' import OpenTelemetrySessionBackend, { Config, DEFAULT_TELEMETRY_MODE, SessionTelemetryMode } from '../src/index.ts' interface Capture { @@ -124,6 +124,16 @@ describe('OpenTelemetrySessionBackend wire', () => { const { ctx, fiber } = await boot(url) const session = ctx.sessions.create(SessionId('wire'), { meta: { cwd: '/tmp/w' } }) session.append('turn/start', { turn: 1 }) + session.append('assistant/chunk', { + turn: 1, + step: 1, + chunk: { type: 'text-delta', index: 0, text: 'first complete chunk' }, + }) + session.append('assistant/chunk', { + turn: 1, + step: 1, + chunk: { type: 'text-delta', index: 0, text: 'second complete chunk' }, + }) session.append('turn/end', { turn: 1, reason: { kind: 'error', error: { message: 'boom', code: 'UNKNOWN' } } }) ctx.sessionTelemetry.emit({ channel: 'ledger', @@ -151,11 +161,59 @@ describe('OpenTelemetrySessionBackend wire', () => { expect(start).toBeDefined() expect(start?.record.severityNumber).toBe(9) expect(BigInt(start!.record.timeUnixNano)).toBe(BigInt(session.snapshotEvents()[0]!.time) * 1_000_000n) + expect(start?.record.attributes).toContainEqual({ + key: 'session.format_version', + value: { intValue: SESSION_FORMAT_VERSION }, + }) expect(start?.record.attributes).toContainEqual({ key: 'session.cwd', value: { stringValue: '/tmp/w' } }) const end = ledger.find(r => r.record.attributes?.some(a => a.key === 'event.type' && a.value.stringValue === 'turn/end')) expect(end?.record.severityNumber).toBe(17) expect(end?.record.severityText).toBe('ERROR') + const chunks = ledger.filter(r => + r.record.attributes?.some(a => a.key === 'event.type' && a.value.stringValue === 'assistant/chunk')) + expect(chunks.map(({ record }) => record.body)).toEqual([ + { + kvlistValue: { + values: [ + { key: 'turn', value: { intValue: 1 } }, + { key: 'step', value: { intValue: 1 } }, + { + key: 'chunk', + value: { + kvlistValue: { + values: [ + { key: 'type', value: { stringValue: 'text-delta' } }, + { key: 'index', value: { intValue: 0 } }, + { key: 'text', value: { stringValue: 'first complete chunk' } }, + ], + }, + }, + }, + ], + }, + }, + { + kvlistValue: { + values: [ + { key: 'turn', value: { intValue: 1 } }, + { key: 'step', value: { intValue: 1 } }, + { + key: 'chunk', + value: { + kvlistValue: { + values: [ + { key: 'type', value: { stringValue: 'text-delta' } }, + { key: 'index', value: { intValue: 0 } }, + { key: 'text', value: { stringValue: 'second complete chunk' } }, + ], + }, + }, + }, + ], + }, + }, + ]) expect(eventTypes(captures)).toContain('manual') expect(ops).toHaveLength(1) diff --git a/packages/session/session-telemetry/README.i18n.yaml b/packages/session/session-telemetry/README.i18n.yaml index 26ab2ac73e..4a70cc6b24 100644 --- a/packages/session/session-telemetry/README.i18n.yaml +++ b/packages/session/session-telemetry/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/session/session-telemetry/README.md -README.md: 69fcaa76bb0d8146d419b4b69f93d70fa854afae -README.zh.md: e9c8266cc7012cfd4e18a3dd9b90fdd69cbca58c +README.md: f9374a96a48f2e0e4789960436b6d9aa8c06e885 +README.zh.md: 75acf1775f8cf8bc040b75e1edf7c59620a485b9 diff --git a/packages/session/session-telemetry/README.md b/packages/session/session-telemetry/README.md index 69fcaa76bb..f9374a96a4 100644 --- a/packages/session/session-telemetry/README.md +++ b/packages/session/session-telemetry/README.md @@ -9,7 +9,7 @@ English | [中文](README.zh.md) ## Summary -`dsh-session-telemetry` captures session activity for outbound reporting: it projects session events into telemetry records, lets a deployment redact them, and hands them to a reporting backend that implements its contract. Deployments do not load this package directly — they load exactly one backend (the shipped OpenTelemetry backend is `dsh-session-telemetry-otel`), which registers `ctx.sessionTelemetry` and composes the capture coordinator. The seam owns capture, redaction, and the sharing disclosure; batching, retry, queueing, and loss policy belong to the backend's SDK and stop at `emit()`. Every mounted backend discloses its deployment-selected sharing policy so acknowledgement surfaces can report whether and how a session is shared. The contract and capture behavior come first; the implementation internals live in a collapsible developer section below. +`dsh-session-telemetry` captures session activity for outbound reporting: it copies each session event into a telemetry record, lets a deployment redact it, and hands it to a reporting backend that implements the contract. Deployments do not load this package directly — they load exactly one backend (the shipped OpenTelemetry backend is `dsh-session-telemetry-otel`), which registers `ctx.sessionTelemetry` and composes the capture coordinator. The seam owns capture, redaction, and the sharing disclosure; batching, retry, queueing, and loss policy belong to the backend's SDK and stop at `emit()`. Every mounted backend discloses its deployment-selected sharing policy so acknowledgement surfaces can report whether and how a session is shared. The contract and capture behavior come first; the implementation internals live in a collapsible developer section below. ## Table of Contents @@ -37,7 +37,7 @@ A backend implements three members: `emit(record)` must be a non-blocking enqueu ### What gets captured -Capture runs in one of two modes. `live` capture follows session events as they are appended, replays already-live sessions at mount time, and records lifecycle markers; `on-demand` capture reads the canonical session log only when the backend requests a prefix through `captureSession(session, throughSeq?)`. Ledger records mirror session events one to one except for one projection: only the first `assistant/chunk` of each `(turn, step)` ships, so `seq` gaps on the wire are routine and never a loss signal. Each record carries the event's complete data, minimal identity attributes, and a pre-mapped severity (`error` for `tool/result.isError`, `turn/end` error reasons, and `agent-error`; `info` otherwise). +Capture runs in one of two modes. `live` capture follows session events as they are appended, replays already-live sessions at mount time, and records lifecycle markers; `on-demand` capture reads the canonical session log only when the backend requests a prefix through `captureSession(session, throughSeq?)`. Every canonical session event maps to one ledger record in order, including every `assistant/chunk` with its complete body. Each ledger record carries the event's complete data, `session.id`, `session.format_version`, the numeric event identity, optional header facts, and a pre-mapped severity (`error` for `tool/result.isError`, `turn/end` error reasons, and `agent-error`; `info` otherwise). ### The sharing disclosure @@ -49,7 +49,7 @@ Every backend discloses its deployment-selected sharing policy through the seam' -Every outbound record passes the `sessionTelemetry/record` waterfall immediately after projection. This package ships no rules: with no listener mounted, records reach the backend exactly as captured, so exported data is as clean as the rules a deployment mounts. Listeners stack by transforming `next()`'s return value; a throwing listener withholds that one record fail-closed. Redaction applies to the outbound copy only — the canonical session log is never rewritten. +Every outbound record passes the `sessionTelemetry/record` waterfall after the coordinator copies its canonical event. This package ships no rules: with no listener mounted, records reach the backend exactly as captured, so exported data is as clean as the rules a deployment mounts. Listeners stack by transforming `next()`'s return value; a throwing listener withholds that one record fail-closed. Redaction applies to the outbound copy only — the canonical session log is never rewritten. ----- @@ -63,22 +63,22 @@ This section explains the capture design; the observable behavior is fully cover ### Design concept -The seam is built on one boundary: the harness's aspect ends at `emit()`. Capture, projection, redaction, and the handoff cursor live here; batching, retry, queueing, and loss policy are the reporting SDK's, deliberately not modelled or wrapped. The design and rejected alternatives are pinned in the [revival Agent Note](../../../.agents/notes/implemented/feature/2026-07-23-session-telemetry-otel-revival.md). +The seam is built on one boundary: the harness's aspect ends at `emit()`. Complete event capture, redaction, and the handoff cursor live here; batching, retry, queueing, and loss policy are the reporting SDK's, deliberately not modelled or wrapped. The design and rejected alternatives are pinned in the [revival Agent Note](../../../.agents/notes/implemented/feature/2026-07-23-session-telemetry-otel-revival.md). ### Source map | File | Role | |---|---| | [`src/index.ts`](src/index.ts) | Service Definition: `SessionTelemetryBackend`/`SessionTelemetrySink` contract, record vocabulary, `session-telemetry/record` waterfall declaration | -| [`src/coordinator.ts`](src/coordinator.ts) | Capture: live listeners, on-demand replay, chunk projection, redaction, handoff cursor, containment | +| [`src/coordinator.ts`](src/coordinator.ts) | Capture: live listeners, complete on-demand replay, redaction, handoff cursor, containment | ### Capture flow -Live capture registers, through the composing fiber's effects: `session/created` adopts the session and replays its log from the handoff cursor; `session/event` projects, deep-copies, redacts, and hands off with zero I/O; `session/flush` forwards the optional hint and returns void so the loop's awaited parallel never waits on telemetry; `session/disposed` captures the session's `shutdown` marker and retires it; `agent/error` is the one live-bus relay, because the session-event vocabulary intentionally has no operational-error record. Disposal captures shutdown markers for still-live sessions, then awaits the backend's `shutdown()`. On-demand capture registers only the disposal effect and reads the canonical log on request. Every synchronous handler runs inside containment so a failing backend or rule can never starve other listeners or reach the agent loop. +Live capture registers, through the composing fiber's effects: `session/created` adopts the session and replays its log from the handoff cursor; `session/event` deep-copies, redacts, and hands off each event with zero I/O; `session/flush` forwards the optional hint and returns void so the loop's awaited parallel never waits on telemetry; `session/disposed` captures the session's `shutdown` marker and retires it; `agent/error` is the one live-bus relay, because the session-event vocabulary intentionally has no operational-error record. Disposal captures shutdown markers for still-live sessions, then awaits the backend's `shutdown()`. On-demand capture registers only the disposal effect and reads the complete requested canonical-log prefix on request. Every synchronous handler runs inside containment so a failing backend or rule can never starve other listeners or reach the agent loop. ### The handoff cursor -A module-scope `WeakMap` records, per session, the highest seq handed off (not delivered). Live capture advances it at append time; on-demand capture advances it only while handing a requested prefix. An uncaptured prefix remains solely in the canonical log, so a coordinator reload adds no telemetry-owned recovery state; a missing cursor safely degrades to re-handing from the session's construction boundary, absorbed by receiver-side dedupe on `(session.id, event.seq)`. This is a narrow, documented exception to the registrations-are-effects discipline: entries die with their sessions, the value is a monotonic watermark, and losing it is never an error. The accepted cost matches at-most-once delivery: a resumed session does not backfill records a previous process failed to deliver. +A module-scope `WeakMap` records, per Session object, the highest seq handed off (not delivered). Live capture advances it at append time; on-demand capture advances it only while handing a requested prefix. Re-adopting the same object resumes after that cursor and does not duplicate its handed-off ledger records. A new Session object has no cursor, so capture starts at seq 0 and includes its complete constructor seed, whether the object represents a fresh session, fork, resume, or migrated stored log. Receivers absorb this deliberate full-log replay and SDK retries by deduplicating on `(session.id, session.format_version, event.seq)`. The object-keyed map is a narrow, documented exception to the registrations-are-effects discipline: entries die with their sessions, and losing one only causes a safe full replay. diff --git a/packages/session/session-telemetry/README.zh.md b/packages/session/session-telemetry/README.zh.md index e9c8266cc7..75acf1775f 100644 --- a/packages/session/session-telemetry/README.zh.md +++ b/packages/session/session-telemetry/README.zh.md @@ -9,7 +9,7 @@ kind: "package-library" ## 概述 -`dsh-session-telemetry` 捕获会话活动用于对外上报：它把会话事件投影为遥测记录，允许部署方脱敏，再交给实现其约定的上报后端。部署方不直接加载本包——它们只加载一个后端（随附的 OpenTelemetry 后端是 `dsh-session-telemetry-otel`），由它注册 `ctx.sessionTelemetry` 并组装捕获协调器。seam 拥有捕获、脱敏与共享披露；批处理、重试、排队与丢失策略属于后端自身的 SDK，止于 `emit()`。每个已挂载后端都披露其部署级共享策略，使确认 surface 能够报告会话是否以及如何被共享。约定与捕获行为在前；实现内部细节放在下方可折叠的开发者章节中。 +`dsh-session-telemetry` 捕获会话活动用于对外上报：它把每个会话事件复制为一条遥测记录，允许部署方脱敏，再交给实现该约定的上报后端。部署方不直接加载本包——它们只加载一个后端（随附的 OpenTelemetry 后端是 `dsh-session-telemetry-otel`），由它注册 `ctx.sessionTelemetry` 并组装捕获协调器。seam 拥有捕获、脱敏与共享披露；批处理、重试、排队与丢失策略属于后端自身的 SDK，止于 `emit()`。每个已挂载后端都披露其部署级共享策略，使确认 surface 能够报告会话是否以及如何被共享。约定与捕获行为在前；实现内部细节放在下方可折叠的开发者章节中。 ## 目录 @@ -37,7 +37,7 @@ kind: "package-library" ### 捕获内容 -捕获以两种模式之一运行。`live` 捕获在追加时跟随会话事件、在挂载时回放已存活会话并记录生命周期标记；`on-demand` 捕获只在后端通过 `captureSession(session, throughSeq?)` 请求前缀时读取权威会话日志。ledger 记录与会话事件一一对应，唯有一个投影例外：每个 `(turn, step)` 只发出第一条 `assistant/chunk`，因此导出流中的 `seq` 缺口是常态，绝不是丢失信号。每条记录携带事件的完整数据、最小身份属性与预先映射的严重级别（`tool/result.isError`、`turn/end` 的错误原因与 `agent-error` 映射为 `error`；其余为 `info`）。 +捕获以两种模式之一运行。`live` 捕获在追加时跟随会话事件、在挂载时回放已存活会话并记录生命周期标记；`on-demand` 捕获只在后端通过 `captureSession(session, throughSeq?)` 请求前缀时读取权威会话日志。每条权威会话事件都按顺序映射为一条 ledger 记录，包括每条 `assistant/chunk` 及其完整 body。每条 ledger 记录携带事件的完整数据、`session.id`、`session.format_version`、数值事件身份、可选 header 事实与预先映射的严重级别（`tool/result.isError`、`turn/end` 的错误原因与 `agent-error` 映射为 `error`；其余为 `info`）。 ### 共享披露 @@ -49,7 +49,7 @@ kind: "package-library" -每条外发记录在投影后立即经过 `sessionTelemetry/record` waterfall（瀑布式事件）。本包不带任何规则：未挂载监听器时，记录以捕获时的原样到达后端，因此导出数据能干净到什么程度，恰恰取决于部署方挂载了什么规则。监听器通过变换 `next()` 的返回值来堆叠；抛出异常的监听器以 fail-closed 方式拦下这一条记录。脱敏只作用于外发副本——权威会话日志永不改写。 +协调器复制权威事件后，每条外发记录都会立即经过 `sessionTelemetry/record` waterfall（瀑布式事件）。本包不带任何规则：未挂载监听器时，记录以捕获时的原样到达后端，因此导出数据能干净到什么程度，恰恰取决于部署方挂载了什么规则。监听器通过变换 `next()` 的返回值来堆叠；抛出异常的监听器以 fail-closed 方式拦下这一条记录。脱敏只作用于外发副本——权威会话日志永不改写。 ----- @@ -63,22 +63,22 @@ kind: "package-library" ### 设计理念 -seam 建立在一个边界之上：harness 的职责止于 `emit()`。捕获、投影、脱敏与 handoff 游标都在这里；批处理、重试、排队与丢失策略属于上报 SDK，本包有意不建模也不包装。设计与被否决的替代方案见[复活 Agent Note](../../../.agents/notes/implemented/feature/2026-07-23-session-telemetry-otel-revival.zh.md)。 +seam 建立在一个边界之上：harness 的职责止于 `emit()`。完整事件捕获、脱敏与 handoff 游标都在这里；批处理、重试、排队与丢失策略属于上报 SDK，本包有意不建模也不包装。设计与被否决的替代方案见[复活 Agent Note](../../../.agents/notes/implemented/feature/2026-07-23-session-telemetry-otel-revival.zh.md)。 ### 源码地图 | 文件 | 职责 | |---|---| | [`src/index.ts`](src/index.ts) | Service Definition：`SessionTelemetryBackend`/`SessionTelemetrySink` 约定、记录词汇、`session-telemetry/record` waterfall 声明 | -| [`src/coordinator.ts`](src/coordinator.ts) | 捕获：live 监听器、on-demand 回放、分片投影、脱敏、handoff 游标、异常隔离 | +| [`src/coordinator.ts`](src/coordinator.ts) | 捕获：live 监听器、完整 on-demand 回放、脱敏、handoff 游标、异常隔离 | ### 捕获流程 -live 捕获通过组合方 fiber 的 effect 注册：`session/created` 收养会话并从 handoff 游标起回放其日志；`session/event` 投影、深拷贝、脱敏并交接，零 I/O；`session/flush` 转发可选的提示并返回 void，使循环所等待的并行任务绝不等待遥测；`session/disposed` 捕获会话的 `shutdown` 标记并退役它；`agent/error` 是唯一的实时总线转发，因为会话事件词汇有意不包含运维错误记录。dispose 会为仍存活的会话捕获 shutdown 标记，然后等待后端的 `shutdown()`。on-demand 捕获只注册 dispose effect，并在请求时读取权威日志。每个同步处理器都运行在异常隔离之内，使失败的后端或规则永远不会饿死其他监听器，也永远不会触及 agent loop。 +live 捕获通过组合方 fiber 的 effect 注册：`session/created` 收养会话并从 handoff 游标起回放其日志；`session/event` 深拷贝、脱敏并交接每个事件，零 I/O；`session/flush` 转发可选的提示并返回 void，使循环所等待的并行任务绝不等待遥测；`session/disposed` 捕获会话的 `shutdown` 标记并退役它；`agent/error` 是唯一的实时总线转发，因为会话事件词汇有意不包含运维错误记录。dispose 会为仍存活的会话捕获 shutdown 标记，然后等待后端的 `shutdown()`。on-demand 捕获只注册 dispose effect，并在请求时读取完整的权威日志请求前缀。每个同步处理器都运行在异常隔离之内，使失败的后端或规则永远不会饿死其他监听器，也永远不会触及 agent loop。 ### handoff 游标 -一个模块作用域的 `WeakMap` 按会话记录已交接（而非已投递）的最高 seq。live 捕获在追加时推进它；on-demand 捕获只在交接所请求的前缀时推进它。未捕获的前缀只留在权威日志中，因此协调器重载不会增加遥测自有的恢复状态；游标缺失时安全退化为从会话构造边界起重新交接，由接收端基于 `(session.id, event.seq)` 的去重吸收。这是对「注册即 effect」纪律的一次有意的、有文档说明的窄例外：条目随其会话消亡，值是单调水位线，丢失它绝不是错误。由此接受的代价与至多一次（at-most-once）投递一致：恢复的会话不会回填上一个进程未能投递的记录。 +一个模块作用域的 `WeakMap` 按 Session 对象记录已交接（而非已投递）的最高 seq。live 捕获在追加时推进它；on-demand 捕获只在交接所请求的前缀时推进它。重新收养同一对象时会从该游标之后继续，且不会重复交接其 ledger 记录。新 Session 对象没有游标，因此捕获从 seq 0 开始，并包含完整的构造 seed；无论该对象表示全新会话、fork、resume 还是已迁移的存储日志，规则都相同。接收端基于 `(session.id, session.format_version, event.seq)` 去重，以吸收这项有意的全日志回放以及 SDK 重试。这个以对象为键的 map 是对「注册即 effect」纪律的一次有意的、有文档说明的窄例外：条目随其 Session 消亡；丢失条目只会触发一次安全的全量回放。 diff --git a/packages/session/session-telemetry/package.json b/packages/session/session-telemetry/package.json index 5905bbd56d..38049ea787 100644 --- a/packages/session/session-telemetry/package.json +++ b/packages/session/session-telemetry/package.json @@ -1,6 +1,6 @@ { "name": "@deepseek-ai/dsh-session-telemetry", - "description": "SessionTelemetryBackend seam for the DeepSeek Harness: session-event capture, projection, redaction, and handoff to a reporting backend", + "description": "SessionTelemetryBackend seam for the DeepSeek Harness: complete session-event capture, projection, redaction, and handoff to a reporting backend", "version": "0.1.2-alpha.3", "publishConfig": { "access": "public" diff --git a/packages/session/session-telemetry/src/coordinator.ts b/packages/session/session-telemetry/src/coordinator.ts index 24150051fb..b417921436 100644 --- a/packages/session/session-telemetry/src/coordinator.ts +++ b/packages/session/session-telemetry/src/coordinator.ts @@ -1,8 +1,8 @@ /** * Capture coordinator for the telemetry capability. Live capture subscribes to * the session firehose plus the one live-bus relay (`agent/error`). Both - * capture paths apply the fixed chunk projection, build logical records, and - * run each through the + * capture paths build one logical record per canonical Session event and run + * each through the * `session-telemetry/record` waterfall (deployment-mounted redaction rules; * pass-through when none), then hands the result to the backend. Live capture * follows the session firehose; on-demand capture replays the canonical log @@ -15,7 +15,6 @@ */ import type { Context } from '@deepseek-ai/cordis' -import { SessionSeq } from '@deepseek-ai/dsh-session' import type { Session, SessionEvent, SessionSeq as SessionSeqType, SessionSeqCursor } from '@deepseek-ai/dsh-session' import type { Agent } from '@deepseek-ai/dsh-agent' import type { SessionTelemetrySink, SessionTelemetryRecord, SessionTelemetrySeverity } from './index.ts' @@ -23,8 +22,8 @@ import type { SessionTelemetrySink, SessionTelemetryRecord, SessionTelemetrySeve /** Whether capture follows live events or reads the canonical log only when requested. */ export type SessionTelemetryCapture = 'live' | 'on-demand' -/** One projected record ready for backend handoff. */ -interface ProjectedRecord { +/** One record ready for backend handoff. */ +interface PendingRecord { readonly record: SessionTelemetryRecord /** Ledger cursor advanced only after the backend accepts this record. */ readonly seq?: SessionSeqType @@ -65,8 +64,6 @@ export class SessionTelemetryCoordinator { * `session/disposed` marks and retires entries. */ private readonly adopted = new Set() - /** Per session, the `turn:step` keys whose first chunk already shipped; rebuilt from the log on re-adoption. */ - private readonly chunkSeen = new WeakMap>() /** * @param ctx - the composing backend's context; listeners bind to its fiber. * @param backend - the backend receiving records; owned elsewhere, never disposed here beyond `shutdown()` forwarding. @@ -127,7 +124,7 @@ export class SessionTelemetryCoordinator { } /** - * Project and hand over the canonical session-log suffix after the handoff + * Copy, redact, and hand over the canonical session-log suffix after the handoff * cursor, optionally stopping at an inclusive sequence boundary. Redaction * runs during this call, so an on-demand caller retains no copied records * before requesting capture and uses the policy mounted at that time. @@ -137,32 +134,23 @@ export class SessionTelemetryCoordinator { * @param throughSeq - optional last sequence included in this capture. */ captureSession(session: Session, throughSeq?: SessionSeqType): void { - const cursor = handoffCursor.get(session) - ?? (session.firstLiveSeq === 0 ? -1 : SessionSeq(session.firstLiveSeq - 1)) + const cursor = handoffCursor.get(session) ?? -1 // Containment is PER EVENT: one rejected record is withheld fail-closed // while the rest of the historical replay proceeds. for (const event of session.snapshotEvents()) { if (throughSeq !== undefined && event.seq > throughSeq) break + if (event.seq <= cursor) continue this.contain(() => { - if (event.seq <= cursor) this.track(session, event) - else this.captureEvent(session, event) + this.captureEvent(session, event) }) } } /** - * Adopt a session: replay its log THROUGH the projection from the handoff - * cursor, then rely on the firehose for everything after. When no cursor - * survived, replay starts at the session's construction boundary - * (`firstLiveSeq`), not seq 0: constructor seeds never publish on the - * firehose, and their content already left the process under another - * identity — the same id in a previous process (resume) or the parent's - * stream (fork, stitched by receivers via `session.seed_length`). Events - * at or below the start still feed the projection state (first-chunk - * tracking) without being re-handed, so a resumed fiber drops mid-step - * chunk continuations exactly like the fiber that saw the step begin. The - * cost, accepted with the capture contract's at-most-once stance: a resume - * does not backfill records a previous process failed to deliver. + * Adopt a session: replay its log after the same-object handoff cursor, then + * rely on the firehose for everything after. A newly constructed Session + * object has no cursor, so replay begins at seq 0 and includes constructor + * seed history. Re-adopting the same object resumes after its cursor. * @param session - the live session to adopt; a second adoption is a no-op. */ private adopt(session: Session): void { @@ -171,25 +159,8 @@ export class SessionTelemetryCoordinator { this.captureSession(session) } - /** Feed the chunk projection without handing off — the ≤cursor half of re-adoption. */ - private track(session: Session, event: SessionEvent): void { - if (event.type === 'assistant/chunk') { - this.seen(session).add(`${event.data.turn}:${event.data.step}`) - } - } - - /** Project, redact, and hand one event to the backend. */ + /** Copy, redact, and hand one canonical event to the backend. */ private captureEvent(session: Session, event: SessionEvent): void { - if (event.type === 'assistant/chunk') { - const key = `${event.data.turn}:${event.data.step}` - const seen = this.seen(session) - // Fixed chunk projection: only the first chunk of each (turn, step) - // ships — the stream-started signal; content is byte-complete in the - // step's assembled assistant/message. Dropped chunks do not advance - // the cursor, so re-adoption re-drops them deterministically. - if (seen.has(key)) return - seen.add(key) - } this.deliver(session, { record: this.redact({ channel: 'ledger', @@ -217,7 +188,7 @@ export class SessionTelemetryCoordinator { } /** Hand one redacted record to the backend, then advance its ledger cursor. */ - private deliver(session: Session, pending: ProjectedRecord): void { + private deliver(session: Session, pending: PendingRecord): void { this.backend.emit(pending.record) if (pending.seq !== undefined) handoffCursor.set(session, pending.seq) } @@ -248,13 +219,6 @@ export class SessionTelemetryCoordinator { }) } - /** Lazily create the per-session first-chunk tracking set. */ - private seen(session: Session): Set { - let set = this.chunkSeen.get(session) - if (!set) this.chunkSeen.set(session, set = new Set()) - return set - } - /** * Run one capture-side step with its exception contained: cordis `emit` * is stop-on-throw, so a throwing listener would starve every subscriber @@ -308,14 +272,15 @@ function errorDetail(error: unknown): { name: string; message: string } { function identityOf(session: Session, event: SessionEvent): Record { const attributes: Record = { 'session.id': String(session.id), + 'session.format_version': session.header.version, 'event.type': event.type, 'event.seq': event.seq, } const { cwd, parentSession, isSeeded } = session.header if (cwd !== undefined) attributes['session.cwd'] = cwd if (parentSession !== undefined) attributes['session.parent_id'] = String(parentSession) - // The durable fork boundary: a forked stream starts here, and its prefix - // lives in the parent's stream — receivers stitch on (parent_id, seed_length). + // The durable fork boundary and lineage: the child ledger is complete; + // parent_id and seed_length identify which leading events were inherited. if (isSeeded) attributes['session.seed_length'] = session.inheritedEventCount return attributes } diff --git a/packages/session/session-telemetry/src/index.ts b/packages/session/session-telemetry/src/index.ts index 0715b073f3..bd839322ec 100644 --- a/packages/session/session-telemetry/src/index.ts +++ b/packages/session/session-telemetry/src/index.ts @@ -1,8 +1,8 @@ /** * SessionTelemetryBackend Service Definition for the DeepSeek Harness. * - * This package owns the CAPTURE side of session-event reporting — which records - * exist (the chunk projection), what they carry (the logical record), when + * This package owns the CAPTURE side of session-event reporting — the complete + * one-record-per-event ledger mirror, what records carry, when * they are captured (adoption, the per-append firehose, lifecycle * forwarding), live versus on-demand canonical-log capture, and the HMR * cursor. Everything downstream of @@ -70,8 +70,9 @@ export interface SessionTelemetryRecord { severity: SessionTelemetrySeverity /** * Identity attributes, deliberately minimal: ledger records carry - * `session.id`, `event.type`, `event.seq`, plus `session.cwd` / - * `session.parent_id` / `session.seed_length` when the header has them; + * `session.id`, `session.format_version`, `event.type`, `event.seq`, plus optional + * `session.cwd` / `session.parent_id`; a seeded Session also carries + * `session.seed_length` from its exact inherited event count; * ops records carry `telemetry.op`, `session.id`, and (for `agent-error`) * `agent.id`, `turn`, `step`, `error.name`. Anything recoverable from the * body is intentionally NOT duplicated here. diff --git a/packages/session/session-telemetry/tests/telemetry.spec.ts b/packages/session/session-telemetry/tests/telemetry.spec.ts index bf097d193f..13c9fa7c7b 100644 --- a/packages/session/session-telemetry/tests/telemetry.spec.ts +++ b/packages/session/session-telemetry/tests/telemetry.spec.ts @@ -2,13 +2,19 @@ import { createToolResultMessage, createUserMessage } from '@deepseek-ai/dsh-llm /** * Coordinator semantics against a bare fake backend — the RFC's named unit * tier for the seam: adoption (fresh, seeded, re-adoption via the handoff - * cursor), the fixed chunk projection, deep-copy isolation, turn-latency and + * cursor), complete-log replay, deep-copy isolation, turn-latency and * dispose-ordering pins, failure containment, and the `agent/error` relay. */ import { describe, expect, it, vi } from 'vitest' import { Context } from '@deepseek-ai/cordis' -import SessionStore, { SessionId, type Session, type SessionEvent } from '@deepseek-ai/dsh-session' +import SessionStore, { + SESSION_FORMAT_VERSION, + Session, + SessionId, + SessionLogOffset, + type SessionEvent, +} from '@deepseek-ai/dsh-session' import type { Agent } from '@deepseek-ai/dsh-agent' import { SessionTelemetryCoordinator, @@ -95,7 +101,12 @@ describe('SessionTelemetryCoordinator capture', () => { const start = backend.ledger()[0]! const message = backend.ledger()[1]! - expect(start.attributes).toMatchObject({ 'session.id': 'cap', 'event.type': 'turn/start', 'event.seq': 0 }) + expect(start.attributes).toMatchObject({ + 'session.id': 'cap', + 'session.format_version': SESSION_FORMAT_VERSION, + 'event.type': 'turn/start', + 'event.seq': 0, + }) expect(start.time).toBe(session.snapshotEvents()[0]!.time) expect(start.severity).toBe('info') expect(message.attributes['event.seq']).toBe(1) @@ -111,6 +122,7 @@ describe('SessionTelemetryCoordinator capture', () => { const session = ctx.sessions.create(SessionId('child'), { meta: { cwd: '/tmp/proj', parentSession: parent } }) appendTurn(session) for (const record of backend.ledger()) { + expect(record.attributes['session.format_version']).toBe(SESSION_FORMAT_VERSION) expect(record.attributes['session.cwd']).toBe('/tmp/proj') expect(record.attributes['session.parent_id']).toBe('parent') } @@ -158,7 +170,7 @@ describe('SessionTelemetryCoordinator capture', () => { expect(record.body).toEqual({ payload: { nested: ['a', 'b'] } }) }) - it('ships only the first chunk of each (turn, step), per session', async () => { + it('ships every assistant chunk in canonical order with its complete body', async () => { const { ctx, backend } = await setup() const a = liveSession(ctx, 'a') const b = liveSession(ctx, 'b') @@ -169,11 +181,17 @@ describe('SessionTelemetryCoordinator capture', () => { chunk(a, 1, 2, 'a12-first') chunk(b, 1, 1, 'b11-first') chunk(b, 1, 1, 'b11-second') - const shipped = backend.ledger().map(r => [r.attributes['session.id'], (r.body as { chunk: { text: string } }).chunk.text]) + const shipped = backend.ledger().map(r => [ + r.attributes['session.id'], + r.attributes['event.seq'], + (r.body as { chunk: { text: string } }).chunk.text, + ]) expect(shipped).toEqual([ - ['a', 'a11-first'], - ['a', 'a12-first'], - ['b', 'b11-first'], + ['a', 0, 'a11-first'], + ['a', 1, 'a11-second'], + ['a', 2, 'a12-first'], + ['b', 0, 'b11-first'], + ['b', 1, 'b11-second'], ]) }) }) @@ -183,7 +201,17 @@ describe('SessionTelemetryCoordinator on-demand capture', () => { const { ctx, backend, coordinator } = await setup(new FakeBackend(), 'on-demand') const session = liveSession(ctx, 'on-demand-prefix') appendTurn(session) - const firstBoundary = session.snapshotEvents()[1]!.seq + session.append('assistant/chunk', { + turn: 1, + step: 1, + chunk: { type: 'text-delta', index: 0, text: 'first' }, + }) + const firstBoundary = session.snapshotEvents()[2]!.seq + session.append('assistant/chunk', { + turn: 1, + step: 1, + chunk: { type: 'text-delta', index: 0, text: 'second' }, + }) session.append('turn/end', { turn: 1, reason: { kind: 'completed' } }) expect(backend.records).toEqual([]) @@ -191,16 +219,23 @@ describe('SessionTelemetryCoordinator on-demand capture', () => { expect(backend.ledger().map(record => record.attributes['event.type'])).toEqual([ 'turn/start', 'user/message', + 'assistant/chunk', ]) + expect(backend.ledger().map(record => record.attributes['event.seq'])).toEqual([0, 1, 2]) + expect(backend.ledger()[2]!.body).toMatchObject({ chunk: { text: 'first' } }) - expect(backend.ledger()).toHaveLength(2) + expect(backend.ledger()).toHaveLength(3) coordinator.captureSession(session) coordinator.captureSession(session) expect(backend.ledger().map(record => record.attributes['event.type'])).toEqual([ 'turn/start', 'user/message', + 'assistant/chunk', + 'assistant/chunk', 'turn/end', ]) + expect(backend.ledger().map(record => record.attributes['event.seq'])).toEqual([0, 1, 2, 3, 4]) + expect(backend.ledger()[3]!.body).toMatchObject({ chunk: { text: 'second' } }) }) it('runs the currently mounted redaction policy during canonical-log capture', async () => { @@ -277,7 +312,7 @@ describe('SessionTelemetryCoordinator on-demand capture', () => { }) describe('SessionTelemetryCoordinator adoption', () => { - it('exports an unpublished suffix without re-exporting constructor history', async () => { + it('replays a new fork object from seq -1, including its inherited prefix', async () => { const backend = new FakeBackend() const ctx = new Context() await ctx.plugin(SessionStore) @@ -295,37 +330,56 @@ describe('SessionTelemetryCoordinator adoption', () => { const seqs = backend.ledger().map(r => [r.attributes['session.id'], r.attributes['event.seq']]) expect(seqs).toEqual(expect.arrayContaining([['seed-parent', 0], ['seed-parent', 1]])) - // 2 end-seed, 3 turn/end: both this lifecycle's own writes, while - // inherited 0-1 stay with the parent stream. - expect(seqs.filter(([id]) => id === 'seeded')).toEqual([['seeded', 2], ['seeded', 3]]) + expect(seqs.filter(([id]) => id === 'seeded')).toEqual([ + ['seeded', 0], + ['seeded', 1], + ['seeded', 2], + ['seeded', 3], + ]) }) - it('resume shape: a full-log seed exports only its own end-seed and rebuilds the chunk projection', async () => { + it('replays a restored post-migration Session from seq -1 with current-version identity', async () => { const backend = new FakeBackend() const ctx = new Context() await ctx.plugin(SessionStore) - const donor = ctx.sessions.create(SessionId('donor'), { meta: {} }) + const donor = Session.create(SessionId('donor')) donor.append('turn/start', { turn: 1 }) donor.append('assistant/chunk', { turn: 1, step: 1, chunk: { type: 'text-delta', index: 0, text: 'first' } }) - const resumed = ctx.sessions.create(SessionId('resumed'), { seed: donor.snapshotEvents(), meta: {} }) await ctx.plugin({ name: 'fake-telemetry', inject: ['sessions'], apply: (inner: Context) => void new SessionTelemetryCoordinator(inner, backend), }) + // Session persistence migrates before it constructs the restored Session; + // telemetry therefore receives a current-format object with the complete + // migrated canonical seed. + const resumed = ctx.sessions.prepare(SessionId('resumed'), { + seed: structuredClone(donor.snapshotEvents()) as SessionEvent[], + meta: { + version: SESSION_FORMAT_VERSION, + id: SessionId('resumed'), + createdAt: 1, + isSeeded: false, + }, + inheritedEventCount: SessionLogOffset(0), + seedSource: 'persistence', + }) + ctx.sessions.enter(resumed) + ctx.sessions.announce(resumed) const ofResumed = () => backend.ledger() .filter(r => r.attributes['session.id'] === 'resumed') - .map(r => r.attributes['event.seq']) - // Nothing inherited is re-exported; seq 2 is this session's own first - // write — the end-seed event its constructor appended after the seed. - expect(ofResumed()).toEqual([2]) - // The seed fed the projection: the (turn 1, step 1) first chunk already - // shipped from the original process, so its continuation is re-dropped… + expect(ofResumed().map(r => r.attributes['event.seq'])).toEqual([0, 1, 2]) + expect(ofResumed().every(r => r.attributes['session.format_version'] === SESSION_FORMAT_VERSION)).toBe(true) resumed.append('assistant/chunk', { turn: 1, step: 1, chunk: { type: 'text-delta', index: 0, text: 'continuation' } }) - expect(ofResumed()).toEqual([2]) - // …while a new step's first chunk exports normally. resumed.append('assistant/chunk', { turn: 1, step: 2, chunk: { type: 'text-delta', index: 0, text: 'next step' } }) - expect(ofResumed()).toEqual([2, 4]) + expect(ofResumed().map(r => r.attributes['event.seq'])).toEqual([0, 1, 2, 3, 4]) + expect(ofResumed().map(r => (r.body as { chunk?: { text: string } }).chunk?.text)).toEqual([ + undefined, + 'first', + undefined, + 'continuation', + 'next step', + ]) }) it('stamps session.seed_length from the exact Session cut so receivers can stitch fork streams', async () => { @@ -371,7 +425,7 @@ describe('SessionTelemetryCoordinator adoption', () => { expect(backend.ledger()).toHaveLength(2) }) - it('resumes from the handoff cursor across a reload, re-dropping mid-step chunks', async () => { + it('resumes from the handoff cursor across same-object re-adoption without duplicates', async () => { const backend = new FakeBackend() const { ctx, fiber } = await setup(backend) const session = liveSession(ctx, 'hmr') @@ -390,9 +444,11 @@ describe('SessionTelemetryCoordinator adoption', () => { inject: ['sessions'], apply: (inner: Context) => void new SessionTelemetryCoordinator(inner, second), }) - // Only the window events past the cursor are re-handed, and the mid-step - // continuation is re-dropped because ≤cursor events rebuilt the projection. - expect(second.ledger().map(r => r.attributes['event.type'])).toEqual(['turn/end']) + // Only window events past the same object's cursor are re-handed. + expect(second.ledger().map(r => [r.attributes['event.seq'], r.attributes['event.type']])).toEqual([ + [2, 'assistant/chunk'], + [3, 'turn/end'], + ]) }) it('replays past a record the backend rejects: one event withheld, the rest adopted', async () => { diff --git a/packages/shell/shell-env/tests/shell-env.spec.ts b/packages/shell/shell-env/tests/shell-env.spec.ts index 4df99a41a4..a8e4f3eb28 100644 --- a/packages/shell/shell-env/tests/shell-env.spec.ts +++ b/packages/shell/shell-env/tests/shell-env.spec.ts @@ -10,6 +10,7 @@ import { afterEach, describe, expect, it, vi } from 'vitest' import { Context } from '@deepseek-ai/cordis' import { ToolCallId } from '@deepseek-ai/dsh-llm' import type { Agent } from '@deepseek-ai/dsh-agent' +import { SESSION_FORMAT_VERSION } from '@deepseek-ai/dsh-session' import type { ToolExecution } from '@deepseek-ai/dsh-tools' import { ShellEnvRegistry } from '@deepseek-ai/dsh-shell-env' import * as BashEnvPlugin from '@deepseek-ai/dsh-shell-env' @@ -28,7 +29,13 @@ function execution(sessionId?: string): ToolExecution { arguments: { command: 'true' }, ...(sessionId === undefined ? {} - : { agent: { session: { header: { version: 0, id: sessionId, createdAt: 0 } } } as Agent }), + : { + agent: { + session: { + header: { version: SESSION_FORMAT_VERSION, id: sessionId, createdAt: 0, isSeeded: false }, + }, + } as unknown as Agent, + }), } } diff --git a/packages/shell/tool-bash-persistent/tests/loader-composition.spec.ts b/packages/shell/tool-bash-persistent/tests/loader-composition.spec.ts index c2aeac1606..a9f4d613ca 100644 --- a/packages/shell/tool-bash-persistent/tests/loader-composition.spec.ts +++ b/packages/shell/tool-bash-persistent/tests/loader-composition.spec.ts @@ -7,7 +7,7 @@ import { Context } from '@deepseek-ai/cordis' import Loader from '@deepseek-ai/cordis-plugin-loader' import Include from '@deepseek-ai/cordis-plugin-include' import { ToolCallId } from '@deepseek-ai/dsh-llm' -import { Session, SessionId } from '@deepseek-ai/dsh-session' +import { SESSION_FORMAT_VERSION, Session, SessionId } from '@deepseek-ai/dsh-session' import AgentRegistry, { Inbox } from '@deepseek-ai/dsh-agent' import type { Agent } from '@deepseek-ai/dsh-agent' import TerminalSessionService from '@deepseek-ai/dsh-terminal' @@ -40,7 +40,9 @@ class PassthroughSandbox extends SandboxProvider { function agent(ctx: Context, cwd: string): Agent { const id = SessionId('persistent-bash-loader-agent') const scope = ctx.plugin(() => {}) - const session = Session.create(id, [], { version: 0, id, createdAt: 0, cwd, isSeeded: false }) + const session = Session.create(id, [], { + version: SESSION_FORMAT_VERSION, id, createdAt: 0, cwd, isSeeded: false, + }) const value: Agent = { id, options: {}, diff --git a/packages/shell/tool-bash-persistent/tests/tools.spec.ts b/packages/shell/tool-bash-persistent/tests/tools.spec.ts index 97d48a4cc5..c7a406663f 100644 --- a/packages/shell/tool-bash-persistent/tests/tools.spec.ts +++ b/packages/shell/tool-bash-persistent/tests/tools.spec.ts @@ -1,7 +1,7 @@ import { afterEach, describe, expect, it } from 'vitest' import { Context } from '@deepseek-ai/cordis' import { ToolCallId } from '@deepseek-ai/dsh-llm' -import { Session, SessionId } from '@deepseek-ai/dsh-session' +import { SESSION_FORMAT_VERSION, Session, SessionId } from '@deepseek-ai/dsh-session' import AgentRegistry, { Inbox } from '@deepseek-ai/dsh-agent' import type { Agent } from '@deepseek-ai/dsh-agent' import TerminalSessionService from '@deepseek-ai/dsh-terminal' @@ -30,7 +30,7 @@ function agent(ctx: Context, cwd: string | undefined): Agent { const id = SessionId(`persistent-bash-owner-${callNumber}`) const scope = ctx.plugin(() => {}) const session = Session.create(id, [], { - version: 0, + version: SESSION_FORMAT_VERSION, id, createdAt: 0, isSeeded: false, diff --git a/packages/shell/tool-bash/tests/tools.spec.ts b/packages/shell/tool-bash/tests/tools.spec.ts index ca8ce99c89..5eeb327ecc 100644 --- a/packages/shell/tool-bash/tests/tools.spec.ts +++ b/packages/shell/tool-bash/tests/tools.spec.ts @@ -11,7 +11,9 @@ import ToolRuntime, { TOOL_ABORTED, TOOL_ABORTED_BEFORE_DISPATCH } from '@deepse import AgentRegistry from '@deepseek-ai/dsh-agent' import type { Agent } from '@deepseek-ai/dsh-agent' import { turnBoundaryProjectionDefinition } from '@deepseek-ai/dsh-agent-loop' -import SessionStore, { SessionId, SessionLogOffset, SessionSeq } from '@deepseek-ai/dsh-session' +import SessionStore, { + SESSION_FORMAT_VERSION, SessionId, SessionLogOffset, SessionSeq, +} from '@deepseek-ai/dsh-session' import JsonlSessionPersistence from '@deepseek-ai/dsh-session-persistence-jsonl' import LocalJobRegistry from '@deepseek-ai/dsh-jobs-local' import * as ToolTasks from '@deepseek-ai/dsh-tool-jobs' @@ -71,7 +73,7 @@ function registerFakeAgent(ctx: Context, sessionId: string, inject: (...args: un id, ctx: scopeFiber.ctx, inject, - session: { id, header: { version: 0, id, createdAt: 0 } }, + session: { id, header: { version: SESSION_FORMAT_VERSION, id, createdAt: 0, isSeeded: false } }, } as unknown as Agent ctx.agents.register(agent) return agent @@ -220,7 +222,7 @@ function sandboxAgent( ...ctx === undefined ? {} : { ctx: ctx.plugin(() => {}).ctx }, session: { id, - header: { version: 0, id, createdAt: 0, isSeeded: false }, + header: { version: SESSION_FORMAT_VERSION, id, createdAt: 0, isSeeded: false }, inheritedEventCount: SessionLogOffset(0), firstLiveSeq: SessionLogOffset(0), get seq() { return SessionLogOffset(events.length) }, @@ -832,7 +834,12 @@ describe('processOutcome', () => { describe('session-cwd routing (per-session workdir)', () => { // An agent whose session header carries a cwd (what session/new records). const agentInCwd = (cwd: string) => - ({ inject: () => undefined, session: { header: { version: 0, id: 'c', createdAt: 0, cwd } } }) as unknown as Agent + ({ + inject: () => undefined, + session: { + header: { version: SESSION_FORMAT_VERSION, id: 'c', createdAt: 0, cwd, isSeeded: false }, + }, + }) as unknown as Agent it('defaults bash to the agent\'s session cwd (not the server launch dir)', async () => { const ctx = await setup() diff --git a/packages/shell/tool-pwsh-persistent/tests/loader-composition.spec.ts b/packages/shell/tool-pwsh-persistent/tests/loader-composition.spec.ts index 58b05082d0..a0b54790b8 100644 --- a/packages/shell/tool-pwsh-persistent/tests/loader-composition.spec.ts +++ b/packages/shell/tool-pwsh-persistent/tests/loader-composition.spec.ts @@ -8,7 +8,7 @@ import { Context } from '@deepseek-ai/cordis' import Loader from '@deepseek-ai/cordis-plugin-loader' import Include from '@deepseek-ai/cordis-plugin-include' import { ToolCallId } from '@deepseek-ai/dsh-llm' -import { Session, SessionId } from '@deepseek-ai/dsh-session' +import { SESSION_FORMAT_VERSION, Session, SessionId } from '@deepseek-ai/dsh-session' import SessionProjectionRegistry from '@deepseek-ai/dsh-session-projection' import AgentRegistry, { Inbox } from '@deepseek-ai/dsh-agent' import type { Agent } from '@deepseek-ai/dsh-agent' @@ -47,7 +47,9 @@ class PassthroughSandbox extends SandboxProvider { function agent(ctx: Context, cwd: string): Agent { const id = SessionId('persistent-pwsh-loader-agent') const scope = ctx.plugin(() => {}) - const session = Session.create(id, [], { version: 0, id, createdAt: 0, cwd, isSeeded: false }) + const session = Session.create(id, [], { + version: SESSION_FORMAT_VERSION, id, createdAt: 0, cwd, isSeeded: false, + }) const value: Agent = { id, options: {}, diff --git a/packages/shell/tool-pwsh-persistent/tests/tools.spec.ts b/packages/shell/tool-pwsh-persistent/tests/tools.spec.ts index d5d9218171..4190bdb604 100644 --- a/packages/shell/tool-pwsh-persistent/tests/tools.spec.ts +++ b/packages/shell/tool-pwsh-persistent/tests/tools.spec.ts @@ -1,7 +1,7 @@ import { afterEach, describe, expect, it } from 'vitest' import { Context } from '@deepseek-ai/cordis' import { ToolCallId } from '@deepseek-ai/dsh-llm' -import { Session, SessionId } from '@deepseek-ai/dsh-session' +import { SESSION_FORMAT_VERSION, Session, SessionId } from '@deepseek-ai/dsh-session' import AgentRegistry, { Inbox } from '@deepseek-ai/dsh-agent' import type { Agent } from '@deepseek-ai/dsh-agent' import TerminalSessionService from '@deepseek-ai/dsh-terminal' @@ -30,7 +30,7 @@ function agent(ctx: Context, cwd: string | undefined): Agent { const id = SessionId(`persistent-pwsh-owner-${callNumber}`) const scope = ctx.plugin(() => {}) const session = Session.create(id, [], { - version: 0, + version: SESSION_FORMAT_VERSION, id, createdAt: 0, isSeeded: false, diff --git a/packages/shell/tool-pwsh/tests/tools.spec.ts b/packages/shell/tool-pwsh/tests/tools.spec.ts index 5e27e3c8e0..2bf101b9e2 100644 --- a/packages/shell/tool-pwsh/tests/tools.spec.ts +++ b/packages/shell/tool-pwsh/tests/tools.spec.ts @@ -22,7 +22,7 @@ import LocalJobRegistry from '@deepseek-ai/dsh-jobs-local' import * as ToolTasks from '@deepseek-ai/dsh-tool-jobs' import AgentRegistry from '@deepseek-ai/dsh-agent' import type { Agent } from '@deepseek-ai/dsh-agent' -import { SessionId, SessionLogOffset, SessionSeq } from '@deepseek-ai/dsh-session' +import { SESSION_FORMAT_VERSION, SessionId, SessionLogOffset, SessionSeq } from '@deepseek-ai/dsh-session' import ApprovalService from '@deepseek-ai/dsh-user-approval' import type { ApprovalOutcome } from '@deepseek-ai/dsh-user-approval' import { ShellExecutor } from '@deepseek-ai/dsh-shell' @@ -253,7 +253,7 @@ function sandboxAgent( ...ctx === undefined ? {} : { ctx: ctx.plugin(() => {}).ctx }, session: { id, - header: { version: 0, id, createdAt: 0, isSeeded: false }, + header: { version: SESSION_FORMAT_VERSION, id, createdAt: 0, isSeeded: false }, inheritedEventCount: SessionLogOffset(0), firstLiveSeq: SessionLogOffset(0), get seq() { return SessionLogOffset(events.length) }, @@ -291,7 +291,7 @@ function registerFakeAgent(ctx: Context, sessionId: string): Agent { ctx: scopeFiber.ctx, session: { id, - header: { version: 0, id, createdAt: 0, isSeeded: false }, + header: { version: SESSION_FORMAT_VERSION, id, createdAt: 0, isSeeded: false }, inheritedEventCount: SessionLogOffset(0), firstLiveSeq: SessionLogOffset(0), seq: SessionLogOffset(0), diff --git a/packages/skill/tool-skill/tests/tool-skill.spec.ts b/packages/skill/tool-skill/tests/tool-skill.spec.ts index 858367134a..d9c9fe89be 100644 --- a/packages/skill/tool-skill/tests/tool-skill.spec.ts +++ b/packages/skill/tool-skill/tests/tool-skill.spec.ts @@ -5,7 +5,9 @@ import { tmpdir } from 'node:os' import { Context } from '@deepseek-ai/cordis' import { createUserMessage, ToolCallId, type Message } from '@deepseek-ai/dsh-llm' import { createScope, type Scope } from '@deepseek-ai/dsh-scope' -import { Session, SessionId, type SessionEvent, type UserMessage } from '@deepseek-ai/dsh-session' +import { + SESSION_FORMAT_VERSION, Session, SessionId, type SessionEvent, type UserMessage, +} from '@deepseek-ai/dsh-session' import SystemPrompt, { renderPrompt } from '@deepseek-ai/dsh-system-prompt' import ToolRuntime, { defineContentToolFixture } from '@deepseek-ai/dsh-tools' import AgentRegistry, { agentEvents, Inbox, type Agent, type PreStepDecision } from '@deepseek-ai/dsh-agent' @@ -38,7 +40,9 @@ async function setup(home: string, config: toolSkill.Config = {}): Promise snapshot.header.id === childId)) { + if (persisted.some(snapshot => ( + snapshot.status === 'current' || snapshot.status === 'migration-required' + ) && snapshot.header.id === childId)) { throw new SubagentError(`subagent "${childId}" already exists`, 'DUPLICATE_CHILD') } } diff --git a/packages/subagent/subagent/tests/continuation.spec.ts b/packages/subagent/subagent/tests/continuation.spec.ts index 851720f228..67ffb76873 100644 --- a/packages/subagent/subagent/tests/continuation.spec.ts +++ b/packages/subagent/subagent/tests/continuation.spec.ts @@ -6,9 +6,10 @@ import { Context } from '@deepseek-ai/cordis' import type { Agent } from '@deepseek-ai/dsh-agent' import AgentLoop from '@deepseek-ai/dsh-agent-loop' import { mountAgentLoopTestDependencies } from '@deepseek-ai/dsh-agent-loop-testkit' -import { SessionId } from '@deepseek-ai/dsh-session' +import { SESSION_FORMAT_VERSION, SessionId } from '@deepseek-ai/dsh-session' import type { SessionEvent } from '@deepseek-ai/dsh-session' import JsonlSessionPersistence from '@deepseek-ai/dsh-session-persistence-jsonl' +import { SessionPersistenceRevision } from '@deepseek-ai/dsh-session-persistence' import SessionProjectionRegistry from '@deepseek-ai/dsh-session-projection' import * as SubagentSpawn from '@deepseek-ai/dsh-subagent-spawn-in-process' import * as SubagentFork from '@deepseek-ai/dsh-subagent-fork-in-process' @@ -255,6 +256,30 @@ describe('SubagentRuntime.startContinuable', () => { expect(ctx.agents.get(reservedId)).toBeUndefined() }) + it('rejects a reserved identity already present as a migration-required Session', async () => { + const { ctx, parent } = await setup([]) + const reservedId = SessionId('00000000-0000-4000-8000-000000000124') + const listSnapshots = vi.spyOn(ctx.sessionPersistence, 'listSnapshots').mockResolvedValue([{ + status: 'migration-required', + header: { + version: SESSION_FORMAT_VERSION, + id: reservedId, + createdAt: 1, + isSeeded: false, + }, + storedVersion: 0, + targetVersion: SESSION_FORMAT_VERSION, + revision: SessionPersistenceRevision('migration-required:1'), + }]) + + await expect(ctx.subagents.startContinuable({ + ...startSpec(parent), + childId: reservedId, + })).rejects.toMatchObject({ code: 'DUPLICATE_CHILD' }) + expect(ctx.agents.get(reservedId)).toBeUndefined() + listSnapshots.mockRestore() + }) + it('rejects without ids when the provider has no prepareContinuable capability', async () => { const { ctx, parent } = await setup([]) const start = vi.fn(async () => { throw new Error('must not dispatch') }) diff --git a/packages/subagent/subagent/tests/list-children.spec.ts b/packages/subagent/subagent/tests/list-children.spec.ts index 54d34f8c95..049b8e5927 100644 --- a/packages/subagent/subagent/tests/list-children.spec.ts +++ b/packages/subagent/subagent/tests/list-children.spec.ts @@ -11,6 +11,7 @@ import SessionStore, { SESSION_FORMAT_VERSION, SessionId, SessionLogOffset, Sess import type { SessionEvent, SessionHeader } from '@deepseek-ai/dsh-session' import type { SessionObservation } from '@deepseek-ai/dsh-session-query' import JsonlSessionPersistence from '@deepseek-ai/dsh-session-persistence-jsonl' +import { isReadableSessionPersistenceListing } from '@deepseek-ai/dsh-session-persistence' import SessionProjectionRegistry from '@deepseek-ai/dsh-session-projection' import type { ProjectionDefinition } from '@deepseek-ai/dsh-session-projection' import SessionProjectionCache from '@deepseek-ai/dsh-session-projection-cache' @@ -570,7 +571,10 @@ describe('SubagentRuntime.listChildren', () => { }) it.each([ - ['version', (meta: SessionHeader): SessionHeader => ({ ...meta, version: meta.version + 1 })], + ['version', (meta: SessionHeader): SessionHeader => ({ + ...meta, + version: SESSION_FORMAT_VERSION + 1, + }) as unknown as SessionHeader], ['id', (meta: SessionHeader): SessionHeader => ({ ...meta, id: SessionId('another-lifecycle') })], ['createdAt', (meta: SessionHeader): SessionHeader => ({ ...meta, createdAt: meta.createdAt + 1 })], ['cwd', (meta: SessionHeader): SessionHeader => ({ ...meta, cwd: '/elsewhere' })], @@ -875,7 +879,9 @@ describe('SubagentRuntime.listChildren', () => { const childId = await startChild(ctx, parent, 'cached child') // The child's turn/end and disposal are the cache's mandatory checkpoint // points; both writes are fail-soft asynchronous, so wait for the row. - const header = (await ctx.sessionPersistence.list()).find(meta => meta.id === childId) + const header = (await ctx.sessionPersistence.list()) + .filter(isReadableSessionPersistenceListing) + .find(listing => listing.header.id === childId)?.header await vi.waitFor(() => { expect(ctx.sessionProjectionCache.cachedSnapshot(header!, SessionLogOffset(0))?.values.subagent).toBeDefined() }, { timeout: 5_000 }) diff --git a/packages/terminal/terminal-bash/tests/index.spec.ts b/packages/terminal/terminal-bash/tests/index.spec.ts index a3edc02058..ff9bb91afb 100644 --- a/packages/terminal/terminal-bash/tests/index.spec.ts +++ b/packages/terminal/terminal-bash/tests/index.spec.ts @@ -3,7 +3,7 @@ import { PassThrough } from 'node:stream' import { resolve } from 'node:path' import { Context } from '@deepseek-ai/cordis' import Loader from '@deepseek-ai/cordis-plugin-loader' -import SessionStore, { Session, SessionId } from '@deepseek-ai/dsh-session' +import SessionStore, { SESSION_FORMAT_VERSION, Session, SessionId } from '@deepseek-ai/dsh-session' import AgentRegistry, { Inbox, type Agent } from '@deepseek-ai/dsh-agent' import SandboxProvider from '@deepseek-ai/dsh-sandbox' import type { ConfinedArgv, SandboxPolicy } from '@deepseek-ai/dsh-sandbox' @@ -51,7 +51,7 @@ function config(): ResolvedConfig { function agent(ctx: Context, cwd?: string): Agent { const id = SessionId('agent') const session = Session.create(id, undefined, { - version: 0, id, createdAt: 0, isSeeded: false, ...cwd === undefined ? {} : { cwd }, + version: SESSION_FORMAT_VERSION, id, createdAt: 0, isSeeded: false, ...cwd === undefined ? {} : { cwd }, }) return { id, options: {}, session, inbox: new Inbox(session, { inserted: () => {}, discarded: () => {}, claimed: () => {} }), diff --git a/packages/test-support/llm-replay/README.i18n.yaml b/packages/test-support/llm-replay/README.i18n.yaml index 410a2bba4a..1786cda15b 100644 --- a/packages/test-support/llm-replay/README.i18n.yaml +++ b/packages/test-support/llm-replay/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/test-support/llm-replay/README.md -README.md: c0643bf6823e96cf3663f8299d0f5645d7974011 -README.zh.md: 1c2fe3393691b4988678497aa5948957f678c80b +README.md: 6b40cebf20976b829150ed00b6c5166bb6473556 +README.zh.md: c4e679c18532bb980a42e5d7fbec66fc614f68f1 diff --git a/packages/test-support/llm-replay/README.md b/packages/test-support/llm-replay/README.md index c0643bf682..6b40cebf20 100644 --- a/packages/test-support/llm-replay/README.md +++ b/packages/test-support/llm-replay/README.md @@ -55,7 +55,7 @@ With `providers` configured, the plugin registers a replay-only adapter whose ca | Field | Default | Meaning | |---|---|---| -| `file` | `$DSH_SNAPSHOT_FILE` | Path to the primary (parent) `session.jsonl` fixture; required (config or env) | +| `file` | `$DSH_SNAPSHOT_FILE` | Path to the selected primary fixture: `session.jsonl` for v0 or `session.vN.jsonl` for a positive generation; required (config or env) | | `overrideFile` | `$DSH_SNAPSHOT_OVERRIDE` | Optional `ReplayOverrideDoc` sidecar for the primary session | | `childFiles` | `$DSH_SNAPSHOT_CHILD_FILES` | Recorded subagent child-session logs for a nested scenario | | `providers` | — | Optional replay-only provider and model catalog; a model may declare `contextWindow`, text/image modalities, and positive `imageRequestTokens` when image-capable; invalid values fail at load and routes never perform provider I/O | @@ -65,11 +65,11 @@ The generated [configuration catalog](../../../docs/config-catalog.md#deepseek-a ### How the fixture works -The fixture is a projection of a persisted session log (`/session.jsonl`) produced by running the real agent once — this plugin does not record. It keeps the header and every event payload but omits body `seq`/`time` envelopes (`seq0`/`time0` for packed rows); replay restores contiguous synthetic envelopes while parsing, and one file cannot mix projected and complete body rows. Runtime persistence continues to write complete logs. Replay derives each model call's chunk sequence from `assistant/chunk` events, so a recorded fixture replays the same logical stream the live model produced. A fixture may carry its `request/header` content tokenized to `{{system}}`/`{{tools}}`; replay is indifferent because derivation reads only the chunk and summary events plus the line-0 session header. +The fixture is a projection of one selected persisted Session generation produced by running the real agent once — this plugin does not record. The snapshot harness supplies the numerically highest canonical parent path (`/session.jsonl` for v0 or `/session.vN.jsonl` for a positive generation), and validates filename/header agreement before replay. The fixture keeps the header and every event payload but omits body `seq`/`time` envelopes (`seq0`/`time0` for packed rows). Replay supplies contiguous sequences and deterministic timestamps, restores typed values replaced by snapshot tokens, rejects partial or mixed envelopes, decodes the complete physical artifact through the build-static Session format catalog, and migrates historical input in memory before it exposes events or the inherited cut; current input takes direct restoration. For a projected v0 header only, an absent `delegationDepth` denotes `0`. The parser never rewrites or renames the fixture. Runtime persistence continues to write complete logs. Replay derives each model call's chunk sequence from current-view `assistant/chunk` events, so a recorded fixture replays the same logical stream the live model produced. A fixture may carry its `request/header` content tokenized to `{{system}}`/`{{tools}}`; replay materializes validation-only values, while derivation reads only the chunk and summary events plus the Session metadata. Two exact repository v0 fixtures use source-qualified legacy extraction only after the catalog returns their manifest-pinned alpha refusal; pathless parsing, copied lookalikes, and changed refusal diagnostics remain strict. This exception affects replay and expected-output comparison only, while the real catalog and persistence continue to refuse those artifacts. ### Nested agents -A scenario where a parent agent delegates to in-process subagents records one log per session: the parent (`session.jsonl`) plus one per child (`session.1.jsonl`, …). Live session ids are freshly random each run, so replay binds each live session to a recorded script by first-call order: the first live session to make a model call claims the first script, the next new session the next, and so on, with each session advancing its own cursor. More distinct live sessions than recorded scripts fails loud. +A scenario where a parent agent delegates to in-process subagents records one role per Session: parent `session[.vN].jsonl`, then contiguous children `session.[.vN].jsonl`. The snapshot harness supplies only the highest generation of each role. Live Session ids are freshly random each run, so replay binds each live Session to a recorded script by first-call order: the first live Session to make a model call claims the parent script, the next new Session the next child script, and so on, with each Session advancing its own cursor. More distinct live Sessions than recorded scripts fails loud. ### Failure modes and overrides @@ -95,13 +95,17 @@ This section explains the design of the replay plugin; the observable behavior i ### Design -Replay is built on one idea: the projected session log is the fixture. `deriveReplayScript` parses the JSONL header (for `id`/`createdAt` ordering facts) and splits `assistant/chunk` events at every `finish` chunk, keyed by `(turn, step)`, so each recorded `stream()` call becomes one `chunks` entry; an assistant group without a `finish` chunk is the fingerprint of a thrown `stream()` and must be expressed through an override sidecar. A `compaction/summary` carrying `llmStreamCall: true` and a complete `rawOutput` replays as one canonical successful stream at that event's position. Scripted strings may embed `{{fromRequest:}}`; at stream time each placeholder resolves against the live request's string leaves, taking the pattern's last match and its first capture group (or the whole match) in place. +Replay treats the selected projected Session generation as the fixture. One parser completes projected envelopes, validates and migrates the whole artifact through `sessionFormatCatalog`, and returns the current header, inherited cut, and event list as one result. `deriveReplayScript` splits the resulting `assistant/chunk` events at every `finish` chunk, keyed by `(turn, step)`, so each recorded `stream()` call becomes one `chunks` entry; an assistant group without a `finish` chunk is the fingerprint of a thrown `stream()` and must be expressed through an override sidecar. A `compaction/summary` carrying `llmStreamCall: true` and a complete `rawOutput` replays as one canonical successful stream at that event's position. Scripted strings may embed `{{fromRequest:}}`; at stream time each placeholder resolves against the live request's string leaves, taking the pattern's last match and its first capture group (or the whole match) in place. + +The committed-corpus test discovers every versioned `session*.jsonl` under `snapshots/`, `packages/`, and `scripts/snapshots/python-sdk-single-exe/`. Every artifact must restore to the current view through the real catalog except two exact manifest refusals: `snapshots/session/agent-instructions/session.jsonl` has an unmatched projected compaction checkpoint, and `snapshots/web/schedule-catalog/session.jsonl` has a title source that contradicts its citations. The shared manifest grants replay-only extraction to those same absolute paths while the corpus continues to assert their real unsupported-migration class and message. A new or changed refusal fails until its underlying data rule is resolved explicitly. ### Source map | File | Role | |---|---| | [`src/index.ts`](src/index.ts) | Types, fixture derivation, override validation, placeholder resolution, session binding, `installLlmReplay`, and the plugin export | +| [`src/alpha-refusal-fixtures.ts`](src/alpha-refusal-fixtures.ts) | Closed source-path and diagnostic manifest for the two replay-only alpha refusals | +| [`tests/session-format-corpus.spec.ts`](tests/session-format-corpus.spec.ts) | Complete committed-generation restoration burn-in and the closed alpha-refusal manifest | | — | No runtime invariant companion is published; this test-only adapter consumes a fixed replay script; its stream grammar is checked by the LLM companion and fixture derivation tests. | ### Binding and stream flow diff --git a/packages/test-support/llm-replay/README.zh.md b/packages/test-support/llm-replay/README.zh.md index 1c2fe33936..c4e679c185 100644 --- a/packages/test-support/llm-replay/README.zh.md +++ b/packages/test-support/llm-replay/README.zh.md @@ -55,7 +55,7 @@ kind: "package-reference" | 字段 | 默认值 | 含义 | |---|---|---| -| `file` | `$DSH_SNAPSHOT_FILE` | 主（父）`session.jsonl` fixture 的路径；必需（配置或 env） | +| `file` | `$DSH_SNAPSHOT_FILE` | 选定 primary fixture 路径：v0 为 `session.jsonl`，正 generation 为 `session.vN.jsonl`；必需（config 或 env） | | `overrideFile` | `$DSH_SNAPSHOT_OVERRIDE` | 主会话的可选 `ReplayOverrideDoc` 伴随文件 | | `childFiles` | `$DSH_SNAPSHOT_CHILD_FILES` | 嵌套场景中已记录的 subagent 子会话日志 | | `providers` | 无 | 可选的仅回放提供方与模型目录；模型可声明 `contextWindow`、文本／图片模态，以及图片模型使用的正整数 `imageRequestTokens`；非法值会在加载时失败，路由绝不执行提供方 I/O | @@ -65,11 +65,11 @@ kind: "package-reference" ### fixture 的工作方式 -fixture 是运行一次真实 agent 所产生的持久化会话日志（`/session.jsonl`）的投影——本插件不录制。它保留 header 与每个事件 payload，但省略正文的 `seq`/`time` envelope（打包行使用 `seq0`/`time0`）；回放在解析时恢复连续的 synthetic envelope，且同一文件不能混用投影正文行与完整正文行。运行时持久化仍写入完整日志。回放从 `assistant/chunk` 事件派生每次模型调用的分片序列，因此已记录 fixture 会回放与在线模型产生的相同逻辑流。fixture 的 `request/header` 内容可能被标记化为 `{{system}}`/`{{tools}}`；回放不受影响，因为派生只读取分片与摘要事件以及第 0 行的会话 header。 +fixture 是运行一次真实 agent 所产生的一份选定持久化 Session generation 投影，本插件不录制。snapshot harness 会提供数值最高的规范 parent 路径（v0 为 `/session.jsonl`，正 generation 为 `/session.vN.jsonl`），并在 replay 前校验文件名与 header 一致。fixture 保留 header 与每个事件 payload，但省略正文的 `seq`/`time` envelope（打包行使用 `seq0`/`time0`）。replay 补充连续序号与确定性 timestamp，恢复被 snapshot token 替换的类型化值，拒绝不完整或混合 envelope，通过构建期静态 Session 格式 catalog 解码完整物理产物，并在公开事件或继承 cut 前于内存中迁移历史输入；当前输入直接 restore。仅对投影 v0 header，缺失的 `delegationDepth` 表示 `0`。parser 从不重写或重命名 fixture。runtime persistence 继续写入完整日志。replay 从当前视图的 `assistant/chunk` 事件派生每次模型调用的 chunk 序列，因此已记录 fixture 会 replay 与在线模型产生的相同逻辑流。fixture 的 `request/header` 内容可能 token 化为 `{{system}}`/`{{tools}}`；replay 会物化仅用于校验的值，而派生只读取 chunk、summary 事件与 Session metadata。仓库中的两个精确 v0 fixture 只在 catalog 返回 manifest 固定的 alpha 拒绝后使用来源限定的旧格式提取；无路径解析、复制的相似 fixture 与变化后的拒绝诊断仍保持严格。此例外只影响 replay 与预期输出比较，真实 catalog 与 persistence 继续拒绝这些产物。 ### 嵌套 agent -父 agent 委托给进程内 subagent 的场景会按会话记录日志：父会话使用 `session.jsonl`，每个子会话各使用一个（`session.1.jsonl` 等）。实时会话 id 每次运行都会重新随机生成，因此回放按首次调用顺序把每个实时会话绑定到已记录脚本：第一个发起模型调用的实时会话取得第一个脚本，下一个新会话取得下一个，依此类推，每个会话分别推进自己的游标。不同实时会话数量超过已记录脚本数时会明确报错。 +parent agent 委托给进程内 subagent 的场景会为每个 Session 记录一个角色：parent 为 `session[.vN].jsonl`，随后是连续 child `session.[.vN].jsonl`。snapshot harness 只提供每个角色的最高 generation。live Session id 每次运行都会重新随机生成，因此 replay 按首次调用顺序把每个 live Session 绑定到已记录脚本：第一个发起模型调用的 live Session 取得 parent 脚本，下一个新 Session 取得下一条 child 脚本，依此类推，每个 Session 分别推进自己的 cursor。不同 live Session 数量超过已记录脚本数时会明确报错。 ### 失败模式与覆盖 @@ -95,13 +95,17 @@ fixture 是运行一次真实 agent 所产生的持久化会话日志（`}}`；流式输出时每个占位符针对实时请求的字符串叶子解析，取该模式的最后一次匹配，用其第一个捕获组（无捕获组时用整个匹配）原位替换。 +replay 把选定的投影 Session generation 视为 fixture。一个 parser 补全投影 envelope，通过 `sessionFormatCatalog` 校验并迁移完整产物，再以一个结果返回当前 header、继承 cut 与事件列表。`deriveReplayScript` 按 `(turn, step)` 键在每次 `finish` chunk 处切分生成的 `assistant/chunk` 事件，使每次已记录的 `stream()` 调用成为一条 `chunks` entry；没有 `finish` chunk 的 assistant group 是 `stream()` 抛出异常的 fingerprint，必须通过 override sidecar 表达。携带 `llmStreamCall: true` 与完整 `rawOutput` 的 `compaction/summary` 会在该事件位置 replay 为一条规范成功 stream。脚本字符串可以内嵌 `{{fromRequest:}}`；stream 输出时每个 placeholder 针对 live request 的 string leaf 解析，取该 pattern 的最后一次 match，用其第一个 capture group（无 capture group 时用整个 match）原位替换。 + +已提交语料测试会发现 `snapshots/`、`packages/` 与 `scripts/snapshots/python-sdk-single-exe/` 下每个带版本的 `session*.jsonl`。除 manifest 中两项精确拒绝外，每个产物都必须通过真实目录还原为当前视图：`snapshots/session/agent-instructions/session.jsonl` 的投影 compaction checkpoint 没有匹配项，`snapshots/web/schedule-catalog/session.jsonl` 的 title 来源与其 citation 矛盾。共享 manifest 只为相同绝对路径授予仅回放提取，而语料仍断言真实的不受支持迁移类型与诊断。新的或发生变化的拒绝会使测试失败，直到其底层数据规则得到显式处理。 ### 源码地图 | 文件 | 职责 | |---|---| | [`src/index.ts`](src/index.ts) | 类型、fixture 派生、override 校验、占位符解析、会话绑定、`installLlmReplay` 与插件导出 | +| [`src/alpha-refusal-fixtures.ts`](src/alpha-refusal-fixtures.ts) | 两项仅回放 alpha 拒绝的封闭来源路径与诊断 manifest | +| [`tests/session-format-corpus.spec.ts`](tests/session-format-corpus.spec.ts) | 完整已提交 generation restore burn-in 与封闭 alpha 拒绝 manifest | | — | 不发布运行时不变式伴生入口；流语法由 LLM 伴生插件与派生测试检验。 | ### 绑定与流式流程 diff --git a/packages/test-support/llm-replay/package.json b/packages/test-support/llm-replay/package.json index 9716020e79..77b2c205bd 100644 --- a/packages/test-support/llm-replay/package.json +++ b/packages/test-support/llm-replay/package.json @@ -46,6 +46,7 @@ "@deepseek-ai/dsh-session": "workspace:^" }, "dependencies": { + "@deepseek-ai/dsh-session-format-catalog": "workspace:^", "@deepseek-ai/dsh-util-values": "workspace:^" } } diff --git a/packages/test-support/llm-replay/src/alpha-refusal-fixtures.ts b/packages/test-support/llm-replay/src/alpha-refusal-fixtures.ts new file mode 100644 index 0000000000..77474ebc0e --- /dev/null +++ b/packages/test-support/llm-replay/src/alpha-refusal-fixtures.ts @@ -0,0 +1,44 @@ +/** Exact repository fixtures allowed to bypass migration only for test replay and comparison. */ + +import { resolve } from 'node:path' + +/** One committed alpha fixture whose real catalog refusal remains required. */ +export interface AlphaSessionFormatRefusalFixture { + /** Repository-relative source identity used by corpus diagnostics. */ + readonly repoRelativePath: string + /** Exact absolute source path accepted by replay-only helpers. */ + readonly path: string + /** Exact source-qualified migration diagnostic the corpus must retain. */ + readonly expectedMessage: string +} + +const REPOSITORY_ROOT = resolve(import.meta.dirname, '../../../..') + +/** Closed replay-only exception inventory; production persistence never imports it. */ +export const ALPHA_SESSION_FORMAT_REFUSAL_FIXTURES: readonly AlphaSessionFormatRefusalFixture[] = Object.freeze([ + Object.freeze({ + repoRelativePath: 'snapshots/session/agent-instructions/session.jsonl', + path: resolve(REPOSITORY_ROOT, 'snapshots/session/agent-instructions/session.jsonl'), + expectedMessage: 'session snapshot line 22: @deepseek-ai/dsh-session-format-v0-to-v1 refuses this format v0 Session: compaction checkpoint at seq 20 has no matching compaction/start', + }), + Object.freeze({ + repoRelativePath: 'snapshots/web/schedule-catalog/session.jsonl', + path: resolve(REPOSITORY_ROOT, 'snapshots/web/schedule-catalog/session.jsonl'), + expectedMessage: 'session snapshot line 4: @deepseek-ai/dsh-session-format-v0-to-v1 refuses this format v0 Session: session/title 2 messageSeqs must be empty exactly for a user title', + }), +]) + +const BY_PATH: ReadonlyMap = new Map( + ALPHA_SESSION_FORMAT_REFUSAL_FIXTURES.map(fixture => [fixture.path, fixture]), +) + +/** + * Resolve one exact replay-only exception without admitting copied lookalikes. + * @param sourcePath - caller-supplied fixture source path. + * @returns the matching closed-manifest entry, or `undefined`. + */ +export function alphaSessionFormatRefusalForPath( + sourcePath: string, +): AlphaSessionFormatRefusalFixture | undefined { + return BY_PATH.get(resolve(sourcePath)) +} diff --git a/packages/test-support/llm-replay/src/index.ts b/packages/test-support/llm-replay/src/index.ts index 7be893cc95..58cf2c0a16 100644 --- a/packages/test-support/llm-replay/src/index.ts +++ b/packages/test-support/llm-replay/src/index.ts @@ -12,8 +12,12 @@ import { delimiter as pathDelimiter } from 'node:path' import type { Context } from '@deepseek-ai/cordis' import type {} from '@deepseek-ai/dsh-compaction' import type {} from '@deepseek-ai/dsh-deepseek-llm-api-extensions' -import { decodeSeqRanges, decodeStorageRecord, SessionLogOffset, type SessionEvent } from '@deepseek-ai/dsh-session' +import { SESSION_FORMAT_VERSION, SessionLogOffset, type SessionEvent } from '@deepseek-ai/dsh-session' import type { SessionLogOffset as SessionLogOffsetType } from '@deepseek-ai/dsh-session' +import { + SessionFormatUnsupportedMigrationError, + sessionFormatCatalog, +} from '@deepseek-ai/dsh-session-format-catalog' import type { ContentBlock, GenerateOptions, @@ -29,9 +33,37 @@ import type { } from '@deepseek-ai/dsh-llm' import { LlmAdapter, LlmError, ReasoningEffortId, requestImageHandleText, resolveRetryPolicy } from '@deepseek-ai/dsh-llm' import { assertNever } from '@deepseek-ai/dsh-util-values' +import { alphaSessionFormatRefusalForPath } from './alpha-refusal-fixtures.ts' + +export { + ALPHA_SESSION_FORMAT_REFUSAL_FIXTURES, + alphaSessionFormatRefusalForPath, + type AlphaSessionFormatRefusalFixture, +} from './alpha-refusal-fixtures.ts' const PACKED_CHUNK_ROW_TYPES = new Set(['text-chunks', 'reasoning-chunks', 'tool-call-chunks']) +if (sessionFormatCatalog.currentVersion !== SESSION_FORMAT_VERSION) { + throw new Error( + `llm-replay: format catalog v${sessionFormatCatalog.currentVersion} ` + + `does not match Session v${SESSION_FORMAT_VERSION}`, + ) +} + +interface ParsedSessionFixture { + readonly id: string + readonly createdAt: number + readonly inheritedEventCount: SessionLogOffsetType + readonly events: SessionEvent[] + readonly artifact: ReturnType + readonly sourceHeader: Readonly> +} + +interface FixtureJsonLine { + readonly lineNumber: number + readonly value: Record +} + /** * One recorded model call. `throw` may replay prefix chunks before failing; * `hang` models cancellation. Derived chunk entries come from ordinary model @@ -97,9 +129,10 @@ export interface ReplayProviderConfig { /** Resolved plugin configuration. */ export interface ReplayConfig { /** - * Path to the PRIMARY (parent) `session.jsonl` fixture. For a single-session - * scenario this is the only log; for a nested-agent scenario it is the parent, - * and the child logs ride in {@link childFiles}. + * Path to the selected PRIMARY fixture (`session.jsonl` for v0 or + * `session.vN.jsonl` for vN). For a single-session scenario this is the only + * log; for a nested-agent scenario it is the parent, and child logs ride in + * {@link childFiles}. */ file: string /** @@ -170,28 +203,33 @@ export interface SessionScript { } /** - * Parse a session `.jsonl` buffer into its event list. Line 0 is the session - * header (a `{type:'session',…}` record), every subsequent non-empty line is a - * {@link SessionEvent} or a packed chunk row. Packed rows expand back into - * events, and JSONL storage-form provenance ranges expand back into - * `number[]`, so physical fixture encodings derive the same script. The - * header is skipped; malformed lines fail loud. + * Parse a projected session `.jsonl` buffer into current events. The first + * non-empty line is the physical header. Body rows either all carry complete + * persistence envelopes or all omit them; projected rows receive deterministic + * dense sequences and zero timestamps. The build-static format catalog then + * decodes and migrates the complete artifact before this function returns. * @param text - the raw `.jsonl` file contents. - * @returns every event after the header, in log order. + * @returns every migrated current event, in log order. */ export function parseSessionLog(text: string): SessionEvent[] { - const events: SessionEvent[] = [] - let nextSeq: SessionLogOffsetType = SessionLogOffset(0) - let headerSkipped = false - // The JSONL backend guarantees line 0 is the session header. Projected - // fixtures omit event envelopes; synthesize them while decoding so callers - // still receive complete SessionEvent values. + return parseSessionFixture(text).events +} + +/** + * Parse one source-qualified fixture for replay, admitting only the closed alpha-refusal manifest. + * @param text - raw persisted or projected Session JSONL. + * @param sourcePath - exact committed fixture path used to select replay-only policy. + * @returns source events for replay-only test adapters. + */ +export function parseSessionLogForReplay(text: string, sourcePath: string): SessionEvent[] { + return parseSessionFixture(text, sourcePath).events +} + +/** Parse, complete, decode, and migrate one projected snapshot artifact without writing its source. */ +function parseSessionFixture(text: string, replaySourcePath?: string): ParsedSessionFixture { + const parsed: FixtureJsonLine[] = [] for (const [index, line] of text.split(/\r?\n/).entries()) { if (line.trim().length === 0) continue - if (!headerSkipped) { - headerSkipped = true - continue - } let value: unknown try { value = JSON.parse(line) as unknown @@ -201,46 +239,255 @@ export function parseSessionLog(text: string): SessionEvent[] { if (value === null || typeof value !== 'object' || Array.isArray(value)) { throw new Error(`session snapshot line ${index + 1} must be a JSON object`) } - const record = value as Record + parsed.push({ lineNumber: index + 1, value: value as Record }) + } + const headerLine = parsed[0] + if (headerLine === undefined) throw new Error('session snapshot must start with a session header') + + const header = normalizeProjectedHeader(headerLine.value) + const rows: Record[] = [] + const rowLines: number[] = [] + const eventLines: number[] = [] + let bodyKind: 'complete' | 'projected' | undefined + let nextSeq = 0 + for (const source of parsed.slice(1)) { + const record = normalizeProjectedRow(source.value) const packed = PACKED_CHUNK_ROW_TYPES.has(record.type as string) const seqKey = packed ? 'seq0' : 'seq' const timeKey = packed ? 'time0' : 'time' - if (!Object.hasOwn(record, seqKey)) record[seqKey] = nextSeq - if (!Object.hasOwn(record, timeKey)) record[timeKey] = 0 - let decoded: SessionEvent[] - try { - if (Object.hasOwn(record, 'sourceEventSeqs')) { - record.sourceEventSeqs = decodeSeqRanges(record.sourceEventSeqs) - } - decoded = decodeStorageRecord(record) - } catch (error) { - /* v8 ignore next -- decodeStorageRecord only throws Error instances; the String arm satisfies unknown narrowing. */ - const detail = error instanceof Error ? error.message : String(error) - throw new Error(`session snapshot line ${index + 1}: ${detail}`, { cause: error }) + const hasSeq = Object.hasOwn(record, seqKey) + const hasTime = Object.hasOwn(record, timeKey) + if (hasSeq !== hasTime) { + throw new Error( + `session snapshot line ${source.lineNumber} must contain both ${seqKey} and ${timeKey}, or neither`, + ) } - events.push(...decoded) - nextSeq = SessionLogOffset(nextSeq + decoded.length) + const currentKind = hasSeq ? 'complete' : 'projected' + if (bodyKind !== undefined && currentKind !== bodyKind) { + throw new Error(`session snapshot line ${source.lineNumber} cannot mix projected and complete body rows`) + } + bodyKind = currentKind + if (currentKind === 'projected') { + record[seqKey] = nextSeq + record[timeKey] = 0 + } + const cardinality = physicalRowCardinality(record) + rows.push(record) + rowLines.push(source.lineNumber) + eventLines.push(...Array.from({ length: cardinality }, () => source.lineNumber)) + nextSeq += cardinality } - return events + + let decoded: ReturnType + try { + decoded = sessionFormatCatalog.decodeArtifact(header, rows) + } catch (error: unknown) { + const physicalRow = locateUnlabelledPhysicalFailure(error, header, rows) + rethrowFixtureFormatError(error, headerLine.lineNumber, rowLines, eventLines, physicalRow) + } + try { + const current = sessionFormatCatalog.migrate(decoded) + return parsedSessionFixture(current, headerLine.value) + } catch (error: unknown) { + const failure = fixtureFormatError(error, headerLine.lineNumber, rowLines, eventLines) + const refusal = replaySourcePath === undefined + ? undefined + : alphaSessionFormatRefusalForPath(replaySourcePath) + if (refusal !== undefined + && failure instanceof SessionFormatUnsupportedMigrationError + && failure.message === refusal.expectedMessage) { + return parsedSessionFixture(decoded, headerLine.value) + } + throw failure + } +} + +/** Materialize the common replay view from a migrated artifact or an exact replay-only v0 refusal. */ +function parsedSessionFixture( + artifact: ReturnType, + sourceHeader: Readonly>, +): ParsedSessionFixture { + return { + id: artifact.header.id, + createdAt: artifact.header.createdAt, + inheritedEventCount: SessionLogOffset(artifact.inheritedEventCount), + events: [...artifact.events] as unknown as SessionEvent[], + artifact, + sourceHeader, + } +} + +/** + * Convert one persisted or projected snapshot fixture to the current physical format in memory. + * Projected cwd tokens remain tokens so the ordinary snapshot normalizer can compare them with a fresh run. + * @param text - one complete Session fixture. + * @returns current-format JSONL with complete event envelopes; the input string and source file remain unchanged. + */ +export function migrateSessionSnapshotFixture(text: string): string { + const parsed = parseSessionFixture(text) + return encodeCurrentSessionSnapshotFixture(text, parsed) +} + +/** + * Prepare one source-qualified fixture for expected-output comparison. + * Exact alpha refusals keep their events and normalize only the physical header + * generation; every other versioned fixture must migrate through the real catalog. + * @param text - one complete Session fixture. + * @param sourcePath - exact committed source path, when known. + * @returns current-generation comparison JSONL without modifying the source file. + */ +export function prepareSessionSnapshotFixtureForComparison( + text: string, + sourcePath?: string, +): string { + const parsed = parseSessionFixture(text, sourcePath) + if (parsed.artifact.header.version !== SESSION_FORMAT_VERSION) { + return rewritePhysicalHeaderVersion(text) + } + return encodeCurrentSessionSnapshotFixture(text, parsed) +} + +/** Encode one migrated fixture while retaining a projected cwd token. */ +function encodeCurrentSessionSnapshotFixture(text: string, parsed: ParsedSessionFixture): string { + const encoded = sessionFormatCatalog.encodeCurrent(parsed.artifact, { packChunks: false }) + const header = { ...encoded.header } + const sourceCwd = parsed.sourceHeader['cwd'] + if (typeof sourceCwd === 'string' && /^\{\{cwd\}\}(?:\/|$)/.test(sourceCwd)) header['cwd'] = sourceCwd + const output = [header, ...encoded.rows].map(record => JSON.stringify(record)).join('\n') + return text.endsWith('\n') ? `${output}\n` : output +} + +/** Rewrite only the first physical header's generation for an exact replay-only refusal. */ +function rewritePhysicalHeaderVersion(text: string): string { + let rewritten = false + return text.split('\n').map((line) => { + if (rewritten || line.trim().length === 0) return line + const header = JSON.parse(line) as Record + header['version'] = SESSION_FORMAT_VERSION + rewritten = true + return JSON.stringify(header) + }).join('\n') +} + +/** Restore typed request-header values replaced by snapshot sidecar tokens. */ +function normalizeProjectedRow(source: Readonly>): Record { + const record = { ...source } + if (record['type'] !== 'request/header') return record + const data = record['data'] + if (data === null || typeof data !== 'object' || Array.isArray(data)) return record + const header = (data as Record)['header'] + if (header === null || typeof header !== 'object' || Array.isArray(header)) return record + const tools = (header as Record)['tools'] + let materializedTools: unknown + if (tools === '{{tools}}') { + materializedTools = [] + } else if (Array.isArray(tools) + && tools.every((tool): tool is string => typeof tool === 'string' && tool.length > 0)) { + materializedTools = tools.map(name => ({ name, description: '', parameters: {} })) + } else { + return record + } + record['data'] = { + ...data, + header: { ...header, tools: materializedTools }, + } + return record +} + +/** Materialize fixture-only header omissions and tokens before physical validation. */ +function normalizeProjectedHeader(header: Readonly>): Record { + const normalized = { ...header } + if (normalized['version'] === 0 && !Object.hasOwn(header, 'delegationDepth')) { + normalized['delegationDepth'] = 0 + } + if (typeof header['cwd'] === 'string' && /^\{\{cwd\}\}(?:\/|$)/.test(header['cwd'])) { + normalized['cwd'] = header['cwd'].replace('{{cwd}}', '/dsh-snapshot-cwd') + } + return normalized +} + +/** Return how many logical events one physical row contributes for deterministic seq completion. */ +function physicalRowCardinality(row: Readonly>): number { + if (!PACKED_CHUNK_ROW_TYPES.has(row['type'] as string)) return 1 + const data = row['data'] + if (data === null || typeof data !== 'object' || Array.isArray(data)) return 1 + const payload = (data as Record)[row['type'] === 'tool-call-chunks' ? 'args' : 'texts'] + return Array.isArray(payload) && payload.length > 0 ? payload.length : 1 +} + +/** Attach the nearest physical source line while preserving unsupported-migration classification. */ +function fixtureFormatError( + error: unknown, + headerLine: number, + rowLines: readonly number[], + eventLines: readonly number[], + physicalRow?: number, +): Error { + const detail = error instanceof Error ? error.message : String(error) + const locationDetail = error instanceof Error && error.cause instanceof Error + ? error.cause.message + : detail + const row = /released (?:Session|(?:text|reasoning|tool-call)-chunks) row (\d+)/.exec(locationDetail) + const event = /Session event (\d+)/.exec(locationDetail) + ?? / at seq (\d+)/.exec(locationDetail) + ?? /^[^ ]+ (\d+) /.exec(locationDetail) + const line = physicalRow === undefined && row === null + ? event === null ? headerLine : eventLines[Number(event[1])] ?? headerLine + : rowLines[physicalRow ?? Number(row?.[1])] ?? headerLine + const message = `session snapshot line ${line}: ${detail}` + if (error instanceof SessionFormatUnsupportedMigrationError) { + return new SessionFormatUnsupportedMigrationError(message, { cause: error }) + } + return new Error(message, { cause: error }) +} + +/** Attach the nearest physical source line and throw the classified fixture error. */ +function rethrowFixtureFormatError( + error: unknown, + headerLine: number, + rowLines: readonly number[], + eventLines: readonly number[], + physicalRow?: number, +): never { + throw fixtureFormatError(error, headerLine, rowLines, eventLines, physicalRow) +} + +/** Locate range-decoder failures whose frozen diagnostic predates physical-row labels. */ +function locateUnlabelledPhysicalFailure( + error: unknown, + header: Readonly>, + rows: readonly Readonly>[], +): number | undefined { + const detail = error instanceof Error ? error.message : String(error) + if (!detail.startsWith('sourceEventSeqs ')) return undefined + const diagnosticHeader = Object.hasOwn(header, 'seedLength') ? { ...header, seedLength: 0 } : header + for (let index = 0; index < rows.length; index += 1) { + try { + sessionFormatCatalog.decodeArtifact(diagnosticHeader, rows.slice(0, index + 1)) + } catch (candidate: unknown) { + const candidateDetail = candidate instanceof Error ? candidate.message : String(candidate) + if (candidateDetail === detail) return index + } + } + return undefined } /** * Read replay identity, ordering, and fork-seed facts from the JSONL header. * - * @param text - the raw `.jsonl` file contents (only the header line is read). - * @returns the header's `id`, `createdAt`, and inherited-event count, defaulted when absent. + * @param text - the raw `.jsonl` file contents; the complete artifact is validated and migrated. + * @returns the migrated header's `id`, `createdAt`, and exact inherited-event count. */ export function parseSessionHeader(text: string): { id: string createdAt: number inheritedEventCount: SessionLogOffsetType } { - const firstLine = text.split('\n').find(line => line.trim().length > 0) ?? '{}' - const parsed = JSON.parse(firstLine) as { id?: unknown; createdAt?: unknown; seedLength?: unknown } + const parsed = parseSessionFixture(text) return { - id: typeof parsed.id === 'string' ? parsed.id : '', - createdAt: typeof parsed.createdAt === 'number' ? parsed.createdAt : 0, - inheritedEventCount: SessionLogOffset(typeof parsed.seedLength === 'number' ? parsed.seedLength : 0), + id: parsed.id, + createdAt: parsed.createdAt, + inheritedEventCount: parsed.inheritedEventCount, } } @@ -577,10 +824,25 @@ function readOverrideDoc(value: unknown, file: string): ReplayOverrideDoc { * @returns the resolved primary-session script. */ export function loadReplayScript(config: ReplayConfig): ReplayEntry[] { + const fixture = readPrimaryFixture(config) + return resolveReplayScript(config, fixture) +} + +/** Read a primary JSONL unless a whole-script sidecar intentionally occupies the same path. */ +function readPrimaryFixture(config: ReplayConfig): ParsedSessionFixture | undefined { + if (!existsSync(config.file) || config.file === config.overrideFile) return undefined + return parseSessionFixture(readFileSync(config.file, 'utf8'), config.file) +} + +/** Resolve an override or derive from one already validated and migrated fixture. */ +function resolveReplayScript( + config: ReplayConfig, + fixture: ParsedSessionFixture | undefined, +): ReplayEntry[] { if (config.overrideFile !== undefined && existsSync(config.overrideFile)) { const doc = readOverrideDoc(JSON.parse(readFileSync(config.overrideFile, 'utf8')) as unknown, config.overrideFile) if (Array.isArray(doc)) return doc - const script = deriveScriptFromFile(config.file) + const script = deriveScriptFromFixture(config.file, fixture) const derivedLength = script.length const seenIndexes = new Set() for (const patch of doc.patches) { @@ -598,15 +860,15 @@ export function loadReplayScript(config: ReplayConfig): ReplayEntry[] { } return script } - return deriveScriptFromFile(config.file) + return deriveScriptFromFixture(config.file, fixture) } -/** Derive the primary script from the session JSONL, failing loud on a missing fixture. */ -function deriveScriptFromFile(file: string): ReplayEntry[] { - if (!existsSync(file)) { +/** Derive a script from an already migrated fixture, failing loud when it is absent. */ +function deriveScriptFromFixture(file: string, fixture: ParsedSessionFixture | undefined): ReplayEntry[] { + if (fixture === undefined) { throw new Error(`llm-replay: fixture not found: ${file} — run \`pnpm run test:snapshot:record\` first`) } - return deriveReplayScript(parseSessionLog(readFileSync(file, 'utf8'))) + return deriveReplayScript(fixture.events) } /** @@ -617,13 +879,12 @@ function deriveScriptFromFile(file: string): ReplayEntry[] { * @returns the primary script first, then the child scripts in bind order. */ export function loadSessionScripts(config: ReplayConfig): SessionScript[] { - const primaryEntries = loadReplayScript(config) + const primaryFixture = readPrimaryFixture(config) + const primaryEntries = resolveReplayScript(config, primaryFixture) // The override path replaces the derived script but carries no header; read // the header off the JSONL when it exists, else use a stable default so an // override-only fixture (header-less) still orders first as the primary. - const primaryHeader = existsSync(config.file) - ? parseSessionHeader(readFileSync(config.file, 'utf8')) - : { id: '', createdAt: 0 } + const primaryHeader = primaryFixture ?? { id: '', createdAt: 0 } const primary: SessionScript = { recordedId: primaryHeader.id, createdAt: primaryHeader.createdAt, entries: primaryEntries, primary: true, } @@ -633,13 +894,13 @@ export function loadSessionScripts(config: ReplayConfig): SessionScript[] { throw new Error(`llm-replay: child fixture not found: ${childFile} — re-record the scenario`) } const text = readFileSync(childFile, 'utf8') - const header = parseSessionHeader(text) + const fixture = parseSessionFixture(text, childFile) // Derive the child's script from its own events only — events AT OR after the seed // boundary. - const ownEvents = parseSessionLog(text).slice(header.inheritedEventCount) + const ownEvents = fixture.events.slice(fixture.inheritedEventCount) children.push({ - recordedId: header.id, - createdAt: header.createdAt, + recordedId: fixture.id, + createdAt: fixture.createdAt, entries: deriveReplayScript(ownEvents), primary: false, }) diff --git a/packages/test-support/llm-replay/tests/llm-replay.spec.ts b/packages/test-support/llm-replay/tests/llm-replay.spec.ts index 8f2a6dd059..deead3b4b6 100644 --- a/packages/test-support/llm-replay/tests/llm-replay.spec.ts +++ b/packages/test-support/llm-replay/tests/llm-replay.spec.ts @@ -1,10 +1,11 @@ -import { existsSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs' +import { existsSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs' import { tmpdir } from 'node:os' -import { join } from 'node:path' +import { join, resolve } from 'node:path' import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' import { Context } from '@deepseek-ai/cordis' -import { SessionSeq } from '@deepseek-ai/dsh-session' +import { SESSION_FORMAT_VERSION, SessionSeq } from '@deepseek-ai/dsh-session' import type { SessionEvent } from '@deepseek-ai/dsh-session' +import { SessionFormatUnsupportedMigrationError } from '@deepseek-ai/dsh-session-format-catalog' import { CompactionId } from '@deepseek-ai/dsh-compaction' import DeepSeekLlmApiExtensionRegistry from '@deepseek-ai/dsh-deepseek-llm-api-extensions' import LlmRuntime, { ToolCallId, createUserMessage, GenerateOptions, LlmAdapter, StreamChunk } from '@deepseek-ai/dsh-llm' @@ -18,11 +19,15 @@ import { installLlmReplay, loadReplayScript, loadSessionScripts, + migrateSessionSnapshotFixture, name, + parseSessionLogForReplay, parseSessionHeader, parseSessionLog, + prepareSessionSnapshotFixtureForComparison, resolveScriptedEntry, } from '../src/index.ts' +import { ALPHA_SESSION_FORMAT_REFUSAL_FIXTURES } from '../src/alpha-refusal-fixtures.ts' declare module '@deepseek-ai/dsh-deepseek-llm-api-extensions/types' { interface DeepSeekLlmApiExtensionMap { @@ -48,15 +53,49 @@ const TEXT_CHUNKS: StreamChunk[] = [ const COMPACTION_ID = CompactionId('replay-compaction') /** Build a minimal session-JSONL string: a header line + the given events. */ -function sessionJsonl(events: SessionEvent[], header?: { id?: string; createdAt?: number; seedLength?: number }): string { +function sessionJsonl( + events: SessionEvent[], + header?: { id?: string; createdAt?: number; seedLength?: number; version?: 0 | 1 }, +): string { const headerLine = JSON.stringify({ type: 'session', - version: 0, + version: header?.version ?? 0, id: header?.id ?? 's1', createdAt: header?.createdAt ?? 0, ...header?.seedLength !== undefined ? { seedLength: header.seedLength } : {}, + delegationDepth: 0, }) - return [headerLine, ...events.map(e => JSON.stringify(e))].join('\n') + '\n' + return [headerLine, ...events.map(event => JSON.stringify(event))].join('\n') + '\n' +} + +/** Build a valid one-turn Session around recorded model calls. */ +function replaySessionJsonl( + calls: readonly StreamChunk[][], + header?: { id?: string; createdAt?: number; seedLength?: number; version?: 0 | 1 }, +): string { + const events: SessionEvent[] = [] + let seq = 0 + const push = (type: string, data: SessionEvent['data']): void => { + events.push({ type, seq: SessionSeq(seq++), time: 0, data } as SessionEvent) + } + push('turn/start', { turn: 1 }) + for (const [index, chunks] of calls.entries()) { + const step = index + 1 + push('step/start', { turn: 1, step }) + for (const chunk of chunks) events.push(chunkEvent(SessionSeq(seq++), 1, step, chunk)) + push('step/end', { turn: 1, step }) + } + push('turn/end', { turn: 1, reason: { kind: 'completed' } }) + return sessionJsonl(events, header) +} + +/** Remove persistence envelopes to produce the committed snapshot projection. */ +function projectSessionJsonl(complete: string): string { + return complete.split('\n').map((line, index) => { + if (index === 0 || line.length === 0) return line + const { seq: _seq, time: _time, ...projected } = JSON.parse(line) as Record + return JSON.stringify(projected) + }).join('\n') } /** A SessionEvent of type assistant/chunk for (turn, step). */ @@ -69,11 +108,8 @@ let file: string /** Write a session log file and return its path. */ function writeSession(filename: string, header: { id: string; createdAt: number }, calls: StreamChunk[][]): string { - let seq = 1 - const events: SessionEvent[] = [] - calls.forEach((chunks, step) => { for (const c of chunks) events.push(chunkEvent(SessionSeq(seq++), 1, step + 1, c)) }) const path = join(dir, filename) - writeFileSync(path, sessionJsonl(events, header), 'utf8') + writeFileSync(path, replaySessionJsonl(calls, header), 'utf8') return path } @@ -92,38 +128,250 @@ async function drain(iter: AsyncIterable): Promise { return out } +describe('Session format package parity', () => { + it('refuses catalog and Session version skew at module load', async () => { + vi.resetModules() + vi.doMock('@deepseek-ai/dsh-session-format-catalog', async (importOriginal) => { + const actual = await importOriginal() + return { + ...actual, + sessionFormatCatalog: { + ...actual.sessionFormatCatalog, + currentVersion: SESSION_FORMAT_VERSION + 1, + }, + } + }) + try { + await expect(import('../src/index.ts')) + .rejects.toThrow(`format catalog v${SESSION_FORMAT_VERSION + 1} does not match Session v${SESSION_FORMAT_VERSION}`) + } finally { + vi.doUnmock('@deepseek-ai/dsh-session-format-catalog') + vi.resetModules() + } + }) +}) + +describe('fixture format diagnostics', () => { + it('attaches the header line to a non-Error catalog failure', async () => { + vi.resetModules() + vi.doMock('@deepseek-ai/dsh-session-format-catalog', async (importOriginal) => { + const actual = await importOriginal() + return { + ...actual, + sessionFormatCatalog: { + ...actual.sessionFormatCatalog, + decodeArtifact(): never { + const failure: unknown = 'decoder exploded' + throw failure + }, + }, + } + }) + try { + const replay = await import('../src/index.ts') + + expect(() => replay.parseSessionLog(sessionJsonl([]))) + .toThrow('session snapshot line 1: decoder exploded') + } finally { + vi.doUnmock('@deepseek-ai/dsh-session-format-catalog') + vi.resetModules() + } + }) + + it('falls back to the header when a source-range diagnostic has no matching physical prefix', async () => { + vi.resetModules() + vi.doMock('@deepseek-ai/dsh-session-format-catalog', async (importOriginal) => { + const actual = await importOriginal() + return { + ...actual, + sessionFormatCatalog: { + ...actual.sessionFormatCatalog, + decodeArtifact(): never { + throw new Error('sourceEventSeqs synthetic unmatched failure') + }, + }, + } + }) + try { + const replay = await import('../src/index.ts') + + expect(() => replay.parseSessionLog(sessionJsonl([]))) + .toThrow('session snapshot line 1: sourceEventSeqs synthetic unmatched failure') + } finally { + vi.doUnmock('@deepseek-ai/dsh-session-format-catalog') + vi.resetModules() + } + }) + + it('maps a matching non-Error source-range prefix failure to its physical row', async () => { + vi.resetModules() + vi.doMock('@deepseek-ai/dsh-session-format-catalog', async (importOriginal) => { + const actual = await importOriginal() + let callCount = 0 + return { + ...actual, + sessionFormatCatalog: { + ...actual.sessionFormatCatalog, + decodeArtifact(): never { + callCount += 1 + if (callCount === 1) throw new Error('sourceEventSeqs synthetic prefix failure') + const failure: unknown = callCount === 2 + ? 'sourceEventSeqs different prefix failure' + : 'sourceEventSeqs synthetic prefix failure' + throw failure + }, + }, + } + }) + try { + const replay = await import('../src/index.ts') + const events: SessionEvent[] = [ + { type: 'turn/start', seq: SessionSeq(0), time: 0, data: { turn: 1 } }, + { type: 'permission/preset', seq: SessionSeq(1), time: 0, data: { preset: 'auto' } }, + ] + + expect(() => replay.parseSessionLog(sessionJsonl(events))) + .toThrow('session snapshot line 3: sourceEventSeqs synthetic prefix failure') + } finally { + vi.doUnmock('@deepseek-ai/dsh-session-format-catalog') + vi.resetModules() + } + }) + + it('falls back to the header for an out-of-range physical-row diagnostic', async () => { + vi.resetModules() + vi.doMock('@deepseek-ai/dsh-session-format-catalog', async (importOriginal) => { + const actual = await importOriginal() + return { + ...actual, + sessionFormatCatalog: { + ...actual.sessionFormatCatalog, + decodeArtifact(): never { + throw new Error('released Session row 99 is malformed') + }, + }, + } + }) + try { + const replay = await import('../src/index.ts') + const event: SessionEvent = { type: 'turn/start', seq: SessionSeq(0), time: 0, data: { turn: 1 } } + + expect(() => replay.parseSessionLog(sessionJsonl([event]))) + .toThrow('session snapshot line 1: released Session row 99 is malformed') + } finally { + vi.doUnmock('@deepseek-ai/dsh-session-format-catalog') + vi.resetModules() + } + }) + + it('falls back to the header for an out-of-range logical-event diagnostic', async () => { + vi.resetModules() + vi.doMock('@deepseek-ai/dsh-session-format-catalog', async (importOriginal) => { + const actual = await importOriginal() + return { + ...actual, + sessionFormatCatalog: { + ...actual.sessionFormatCatalog, + migrate(): never { + throw new Error('Session event 99 is malformed') + }, + }, + } + }) + try { + const replay = await import('../src/index.ts') + const event: SessionEvent = { type: 'turn/start', seq: SessionSeq(0), time: 0, data: { turn: 1 } } + + expect(() => replay.parseSessionLog(sessionJsonl([event]))) + .toThrow('session snapshot line 1: Session event 99 is malformed') + } finally { + vi.doUnmock('@deepseek-ai/dsh-session-format-catalog') + vi.resetModules() + } + }) +}) + describe('parseSessionLog', () => { + it('reports invalid JSON at its physical source line', () => { + const header = JSON.stringify({ type: 'session', version: 0, id: 's1', createdAt: 0 }) + + expect(() => parseSessionLog(`${header}\n{"type":\n`)) + .toThrow('session snapshot line 2 contains invalid JSON') + }) + + it.each(ALPHA_SESSION_FORMAT_REFUSAL_FIXTURES)( + 'keeps pathless parsing strict for $repoRelativePath', + (fixture) => { + const source = readFileSync(fixture.path, 'utf8') + + expect(() => parseSessionLog(source)).toThrow(SessionFormatUnsupportedMigrationError) + }, + ) + + it.each(ALPHA_SESSION_FORMAT_REFUSAL_FIXTURES)( + 'permits replay extraction only for the exact allowlisted path $repoRelativePath', + (fixture) => { + const source = readFileSync(fixture.path, 'utf8') + const lookalike = resolve(dir, fixture.repoRelativePath) + + expect(parseSessionLogForReplay(source, fixture.path).length).toBeGreaterThan(0) + expect(() => parseSessionLogForReplay(source, lookalike)) + .toThrow(SessionFormatUnsupportedMigrationError) + }, + ) + it('skips the header line and parses each event', () => { - const events = [chunkEvent(SessionSeq(1), 1, 1, TEXT_CHUNKS[0] as StreamChunk)] + const events: SessionEvent[] = [{ type: 'turn/start', seq: SessionSeq(0), time: 0, data: { turn: 1 } }] expect(parseSessionLog(sessionJsonl(events))).toEqual(events) }) it('ignores blank lines', () => { const header = JSON.stringify({ type: 'session', version: 0, id: 's1', createdAt: 0 }) - const ev = chunkEvent(SessionSeq(1), 1, 1, TEXT_CHUNKS[0] as StreamChunk) + const ev: SessionEvent = { type: 'turn/start', seq: SessionSeq(0), time: 0, data: { turn: 1 } } expect(parseSessionLog(`${header}\n\n${JSON.stringify(ev)}\n\n`)).toEqual([ev]) }) it('expands range-encoded source provenance', () => { const header = JSON.stringify({ type: 'session', version: 0, id: 's1', createdAt: 0 }) - const event = { - ...chunkEvent(SessionSeq(4), 1, 1, TEXT_CHUNKS[0] as StreamChunk), - sourceEventSeqs: [[1, 3], 5], - } - expect(parseSessionLog(`${header}\n${JSON.stringify(event)}\n`)).toEqual([{ - ...event, - sourceEventSeqs: [1, 2, 3, 5], - }]) + const events = Array.from({ length: 5 }, (_, seq) => ({ + type: 'user/message', + seq, + time: 0, + data: { role: 'user', id: `message-${seq}`, content: [], source: { kind: 'user' } }, + surfaceOp: 'append', + ...(seq === 4 ? { sourceEventSeqs: [[0, 2], 3] } : {}), + })) + const parsed = parseSessionLog(`${header}\n${events.map(event => JSON.stringify(event)).join('\n')}\n`) + expect(parsed[4]).toEqual({ ...events[4], sourceEventSeqs: [0, 1, 2, 3] }) }) it('reports malformed range provenance with its source line', () => { const header = JSON.stringify({ type: 'session', version: 0, id: 's1', createdAt: 0 }) - const event = { - ...chunkEvent(SessionSeq(4), 1, 1, TEXT_CHUNKS[0] as StreamChunk), - sourceEventSeqs: [[3, 1]], - } - expect(() => parseSessionLog(`${header}\n${JSON.stringify(event)}\n`)) - .toThrow('session snapshot line 2: sourceEventSeqs ranges require start <= end') + const events = Array.from({ length: 5 }, (_, seq) => ({ + type: 'user/message', + seq, + time: 0, + data: { role: 'user', id: `message-${seq}`, content: [], source: { kind: 'user' } }, + surfaceOp: 'append', + ...(seq === 4 ? { sourceEventSeqs: [[3, 1]] } : {}), + })) + expect(() => parseSessionLog(`${header}\n${events.map(event => JSON.stringify(event)).join('\n')}\n`)) + .toThrow(/session snapshot line 6: sourceEventSeqs range/) + }) + + it('locates malformed range provenance with a materialized v0 seedLength header', () => { + const header = JSON.stringify({ type: 'session', version: 0, id: 's1', createdAt: 0, seedLength: 0 }) + const events = Array.from({ length: 3 }, (_, seq) => ({ + type: 'user/message', + seq, + time: 0, + data: { role: 'user', id: `message-${seq}`, content: [], source: { kind: 'user' } }, + surfaceOp: 'append', + ...(seq === 2 ? { sourceEventSeqs: [[2, 1]] } : {}), + })) + + expect(() => parseSessionLog(`${header}\n${events.map(event => JSON.stringify(event)).join('\n')}\n`)) + .toThrow(/session snapshot line 4: sourceEventSeqs range/) }) it('rejects non-object body rows with their source line', () => { @@ -132,32 +380,234 @@ describe('parseSessionLog', () => { .toThrow('session snapshot line 2 must be a JSON object') }) + it('rejects partial and mixed projected envelopes at their source lines', () => { + const header = JSON.stringify({ + type: 'session', version: 0, id: 's1', createdAt: 0, delegationDepth: 0, + }) + const partial = JSON.stringify({ type: 'turn/start', seq: 0, data: { turn: 1 } }) + expect(() => parseSessionLog(`${header}\n${partial}\n`)) + .toThrow('session snapshot line 2 must contain both seq and time, or neither') + + const projected = JSON.stringify({ type: 'turn/start', data: { turn: 1 } }) + const complete = JSON.stringify({ type: 'permission/preset', seq: 1, time: 0, data: { preset: 'auto' } }) + expect(() => parseSessionLog(`${header}\n${projected}\n${complete}\n`)) + .toThrow('session snapshot line 3 cannot mix projected and complete body rows') + }) + it('expands a packed chunk row into its events (a fixture recorded with packChunks on)', () => { const header = JSON.stringify({ type: 'session', version: 0, id: 's1', createdAt: 0 }) + const turn = JSON.stringify({ type: 'turn/start', seq: 0, time: 0, data: { turn: 1 } }) + const step = JSON.stringify({ type: 'step/start', seq: 1, time: 0, data: { turn: 1, step: 1 } }) const row = JSON.stringify({ - type: 'text-chunks', seq0: 1, time0: 0, + type: 'text-chunks', seq0: 2, time0: 0, data: { turn: 1, step: 1, index: 0, dt: [0, 0], texts: ['a', 'b', 'c'] }, }) - expect(parseSessionLog(`${header}\n${row}\n`)).toEqual([ - chunkEvent(SessionSeq(1), 1, 1, { type: 'text-delta', index: 0, text: 'a' }), - chunkEvent(SessionSeq(2), 1, 1, { type: 'text-delta', index: 0, text: 'b' }), - chunkEvent(SessionSeq(3), 1, 1, { type: 'text-delta', index: 0, text: 'c' }), + expect(parseSessionLog(`${header}\n${turn}\n${step}\n${row}\n`).slice(2)).toEqual([ + chunkEvent(SessionSeq(2), 1, 1, { type: 'text-delta', index: 0, text: 'a' }), + chunkEvent(SessionSeq(3), 1, 1, { type: 'text-delta', index: 0, text: 'b' }), + chunkEvent(SessionSeq(4), 1, 1, { type: 'text-delta', index: 0, text: 'c' }), ]) }) it('synthesizes omitted ordinary and packed snapshot envelopes', () => { const header = JSON.stringify({ type: 'session', version: 0, id: 's1', createdAt: 7 }) const ordinary = JSON.stringify({ type: 'turn/start', data: { turn: 1 } }) + const step = JSON.stringify({ type: 'step/start', data: { turn: 1, step: 1 } }) const packed = JSON.stringify({ type: 'text-chunks', data: { turn: 1, step: 1, index: 0, dt: [3], texts: ['a', 'b'] }, }) - expect(parseSessionLog(`${header}\n${ordinary}\n${packed}\n`)).toEqual([ + expect(parseSessionLog(`${header}\n${ordinary}\n${step}\n${packed}\n`)).toEqual([ { type: 'turn/start', seq: 0, time: 0, data: { turn: 1 } }, - chunkEvent(SessionSeq(1), 1, 1, { type: 'text-delta', index: 0, text: 'a' }), - { ...chunkEvent(SessionSeq(2), 1, 1, { type: 'text-delta', index: 0, text: 'b' }), time: 3 }, + { type: 'step/start', seq: 1, time: 0, data: { turn: 1, step: 1 } }, + chunkEvent(SessionSeq(2), 1, 1, { type: 'text-delta', index: 0, text: 'a' }), + { ...chunkEvent(SessionSeq(3), 1, 1, { type: 'text-delta', index: 0, text: 'b' }), time: 3 }, ]) }) + + it('materializes tokenized request tools before released-format validation', () => { + const source = [ + JSON.stringify({ type: 'session', version: 0, id: 'tokens', createdAt: 7, delegationDepth: 0 }), + JSON.stringify({ type: 'turn/start', data: { turn: 1 } }), + JSON.stringify({ + type: 'request/header', + data: { + header: { config: { provider: 'mock', model: 'mock' }, system: '{{system}}', tools: '{{tools}}' }, + reason: 'initial', + }, + }), + ].join('\n') + + expect(parseSessionLog(source)[1]).toMatchObject({ + type: 'request/header', + data: { header: { system: '{{system}}', tools: [] } }, + }) + }) + + it('materializes Python snapshot tool-name projections before format validation', () => { + const source = [ + JSON.stringify({ type: 'session', version: 0, id: 'tool-names', createdAt: 7, delegationDepth: 0 }), + JSON.stringify({ type: 'turn/start', data: { turn: 1 } }), + JSON.stringify({ + type: 'request/header', + data: { + header: { config: { provider: 'mock', model: 'mock' }, tools: ['bash', 'workflow'] }, + reason: 'initial', + }, + }), + ].join('\n') + + expect(parseSessionLog(source)[1]).toMatchObject({ + type: 'request/header', + data: { header: { tools: [ + { name: 'bash', description: '', parameters: {} }, + { name: 'workflow', description: '', parameters: {} }, + ] } }, + }) + }) + + it.each([ + ['non-object request data', null], + ['non-object request header', { header: [] }], + ['unsupported tools projection', { header: { tools: [42] } }], + ])('keeps %s unchanged so released-format validation rejects its source line', (_name, data) => { + const source = [ + JSON.stringify({ type: 'session', version: 0, id: 'malformed-request', createdAt: 7, delegationDepth: 0 }), + JSON.stringify({ type: 'request/header', data }), + ].join('\n') + + expect(() => parseSessionLog(source)).toThrow(/session snapshot line 2:/) + }) + + it('synthesizes projected tool-call chunk envelopes from their argument count', () => { + const callId = ToolCallId('call-1') + const source = [ + JSON.stringify({ type: 'session', version: 0, id: 'packed-tool', createdAt: 0 }), + JSON.stringify({ type: 'turn/start', data: { turn: 1 } }), + JSON.stringify({ type: 'step/start', data: { turn: 1, step: 1 } }), + JSON.stringify({ + type: 'tool-call-chunks', + data: { turn: 1, step: 1, index: 0, id: 'call-1', name: 'read', dt: [0], args: ['{', '}'] }, + }), + ].join('\n') + + expect(parseSessionLog(source).slice(2)).toEqual([ + chunkEvent(SessionSeq(2), 1, 1, { + type: 'tool-call-delta', index: 0, id: callId, name: 'read', argumentsDelta: '{', + }), + chunkEvent(SessionSeq(3), 1, 1, { + type: 'tool-call-delta', index: 0, id: callId, name: 'read', argumentsDelta: '}', + }), + ]) + }) + + it.each([ + ['non-object packed data', { type: 'text-chunks', data: null }], + ['empty packed payload', { + type: 'text-chunks', + data: { turn: 1, step: 1, index: 0, dt: [], texts: [] }, + }], + ])('reports %s at its one synthesized event line', (_name, row) => { + const source = [ + JSON.stringify({ type: 'session', version: 0, id: 'malformed-packed', createdAt: 0 }), + JSON.stringify(row), + ].join('\n') + + expect(() => parseSessionLog(source)).toThrow(/session snapshot line 2:/) + }) + + it('migrates projected v0 legacy messages through the released catalog before returning events', () => { + const source = [ + JSON.stringify({ type: 'session', version: 0, id: 'legacy', createdAt: 7, delegationDepth: 0 }), + JSON.stringify({ type: 'turn/start', data: { turn: 1 } }), + JSON.stringify({ type: 'step/start', data: { turn: 1, step: 1 } }), + JSON.stringify({ + type: 'assistant/message', + data: { + turn: 1, + step: 1, + content: [{ type: 'text', text: 'migrated' }], + provenance: { provider: 'mock', model: 'mock' }, + }, + surfaceOp: 'append', + }), + ].join('\n') + + expect(parseSessionLog(source)[2]).toEqual({ + type: 'assistant/message', + seq: 2, + time: 0, + data: { + turn: 1, + step: 1, + message: { + id: 'legacy-message:legacy:2', + role: 'assistant', + content: [{ type: 'text', text: 'migrated' }], + source: { kind: 'model', provider: 'mock', model: 'mock' }, + }, + }, + surfaceOp: 'append', + }) + }) + + it('takes the direct v1 path and rejects v0-only message fields', () => { + const current = projectSessionJsonl(replaySessionJsonl([TEXT_CHUNKS], { version: 1 })) + expect(deriveReplayScript(parseSessionLog(current))).toEqual([{ kind: 'chunks', chunks: TEXT_CHUNKS }]) + + const source = [ + JSON.stringify({ type: 'session', version: 1, id: 'current', createdAt: 7, delegationDepth: 0 }), + JSON.stringify({ + type: 'assistant/message', + data: { + turn: 1, + step: 1, + content: [{ type: 'text', text: 'legacy-only' }], + provenance: { provider: 'mock', model: 'mock' }, + }, + surfaceOp: 'append', + }), + ].join('\n') + + expect(() => parseSessionLog(source)).toThrow(/assistant\/message.*lacks an identified message/) + }) + + it('refuses a retired v0 event through the released migration policy', () => { + const source = [ + JSON.stringify({ type: 'session', version: 0, id: 'legacy', createdAt: 7, delegationDepth: 0 }), + JSON.stringify({ type: 'mode/set', data: { mode: 'plan' } }), + ].join('\n') + + expect(() => parseSessionLog(source)).toThrow(/unsupported legacy mode\/set event at seq 0/) + }) +}) + +describe('prepareSessionSnapshotFixtureForComparison', () => { + it('encodes a migrated fixture without inventing a trailing newline and retains its cwd token', () => { + const projected = projectSessionJsonl(replaySessionJsonl([TEXT_CHUNKS])).trimEnd() + const [headerLine, ...bodyLines] = projected.split('\n') + const header = { ...(JSON.parse(headerLine!) as Record), cwd: '{{cwd}}/workspace' } + const source = [JSON.stringify(header), ...bodyLines].join('\n') + + const prepared = prepareSessionSnapshotFixtureForComparison(source) + const [preparedHeader] = prepared.split('\n') + + expect(JSON.parse(preparedHeader!)).toMatchObject({ version: SESSION_FORMAT_VERSION, cwd: '{{cwd}}/workspace' }) + expect(prepared.endsWith('\n')).toBe(false) + }) + + it('rewrites only the physical header generation for an exact alpha refusal', () => { + const fixture = ALPHA_SESSION_FORMAT_REFUSAL_FIXTURES[0]! + const source = readFileSync(fixture.path, 'utf8') + + const prepared = prepareSessionSnapshotFixtureForComparison(source, fixture.path) + const [sourceHeader, ...sourceBody] = source.split('\n') + const [preparedHeader, ...preparedBody] = prepared.split('\n') + + expect(JSON.parse(sourceHeader!)).toMatchObject({ version: 0 }) + expect(JSON.parse(preparedHeader!)).toMatchObject({ version: SESSION_FORMAT_VERSION }) + expect(preparedBody).toEqual(sourceBody) + }) }) describe('deriveReplayScript', () => { @@ -321,28 +771,50 @@ describe('deriveReplayScript', () => { expect(deriveReplayScript([event])).toEqual([]) }) + it('rejects a durable compact stream marker whose complete output is absent', () => { + const event = { + type: 'compaction/summary', + seq: SessionSeq(1), + time: 0, + data: { llmStreamCall: true }, + } as unknown as SessionEvent + + expect(() => deriveReplayScript([event])) + .toThrow('llm-replay: compaction/summary marks an LLM stream call without rawOutput') + }) + it('rejects a persisted marked compact LLM call without its complete output', () => { - const [event] = parseSessionLog([ - JSON.stringify({ type: 'session', version: 0, id: 'invalid-compact', createdAt: 0 }), + const source = [ + JSON.stringify({ + type: 'session', version: 0, id: 'invalid-compact', createdAt: 0, delegationDepth: 0, + }), + JSON.stringify({ + type: 'user/message', seq: 0, time: 0, + data: { role: 'user', id: 'source', content: [], source: { kind: 'user' } }, + surfaceOp: 'append', + }), + JSON.stringify({ + type: 'compaction/start', seq: 1, time: 0, + data: { compactionId: COMPACTION_ID, turn: 1 }, + }), JSON.stringify({ type: 'compaction/summary', - seq: 1, + seq: 2, time: 0, data: { compactionId: COMPACTION_ID, summary: [{ type: 'text', text: 'missing source events' }], llmStreamCall: true, - shadowedRange: { start: 1, end: 1 }, - shadowedSeqs: [1], + shadowedRange: { start: 0, end: 0 }, + shadowedSeqs: [0], shadowedTokenCount: 20, provider: 'mock', model: 'mock', }, }), - ].join('\n')) + ].join('\n') - expect(() => deriveReplayScript(event === undefined ? [] : [event])) - .toThrow(/LLM stream call without rawOutput/) + expect(() => parseSessionLog(source)).toThrow(/llmStreamCall requires rawOutput/) }) it('derives a compaction/summary stream when usage is unavailable', () => { @@ -424,10 +896,19 @@ describe('deriveReplayScript', () => { describe('loadReplayScript', () => { it('derives from the session JSONL when no override is present', () => { - writeFileSync(file, sessionJsonl(TEXT_CHUNKS.map((c, i) => chunkEvent(SessionSeq(i + 1), 1, 1, c))), 'utf8') + writeFileSync(file, replaySessionJsonl([TEXT_CHUNKS]), 'utf8') expect(loadReplayScript({ file })).toEqual([{ kind: 'chunks', chunks: TEXT_CHUNKS }]) }) + it('never rewrites a projected source fixture while migrating it in memory', () => { + const source = projectSessionJsonl(replaySessionJsonl([TEXT_CHUNKS])) + writeFileSync(file, source, 'utf8') + + expect(loadReplayScript({ file })).toEqual([{ kind: 'chunks', chunks: TEXT_CHUNKS }]) + expect(JSON.parse(migrateSessionSnapshotFixture(source).split('\n')[0] as string)).toMatchObject({ version: 1 }) + expect(readFileSync(file, 'utf8')).toBe(source) + }) + it('uses the sidecar override when present, ignoring the JSONL', () => { writeFileSync(file, sessionJsonl([]), 'utf8') const overrideFile = join(dir, 'replay.override.json') @@ -436,8 +917,16 @@ describe('loadReplayScript', () => { expect(loadReplayScript({ file, overrideFile })).toEqual(override) }) + it('uses a whole-script override reused as the primary fixture path', () => { + const overrideFile = join(dir, 'replay.override.json') + const override: ReplayEntry[] = [{ kind: 'hang' }] + writeFileSync(overrideFile, JSON.stringify(override), 'utf8') + + expect(loadReplayScript({ file: overrideFile, overrideFile })).toEqual(override) + }) + it('falls back to the JSONL when the override path is set but absent', () => { - writeFileSync(file, sessionJsonl(TEXT_CHUNKS.map((c, i) => chunkEvent(SessionSeq(i + 1), 1, 1, c))), 'utf8') + writeFileSync(file, replaySessionJsonl([TEXT_CHUNKS]), 'utf8') expect(loadReplayScript({ file, overrideFile: join(dir, 'nope.json') })) .toEqual([{ kind: 'chunks', chunks: TEXT_CHUNKS }]) }) @@ -459,11 +948,7 @@ describe('loadReplayScript', () => { { type: 'text-delta', index: 0, text: 'two' }, { type: 'finish', reason: { kind: 'stop' } }, ] - let seq = 1 - writeFileSync(file, sessionJsonl([ - ...TEXT_CHUNKS.map(c => chunkEvent(SessionSeq(seq++), 1, 1, c)), - ...callB.map(c => chunkEvent(SessionSeq(seq++), 1, 2, c)), - ]), 'utf8') + writeFileSync(file, replaySessionJsonl([TEXT_CHUNKS, callB]), 'utf8') const overrideFile = join(dir, 'replay.override.json') writeFileSync(overrideFile, JSON.stringify({ patches: [{ at: 0, entry: { kind: 'throw', chunks: [], message: 'transient', code: 'SERVER' } }], @@ -475,7 +960,7 @@ describe('loadReplayScript', () => { }) it('patches form: at == derived length appends (the retry-attempt slot)', () => { - writeFileSync(file, sessionJsonl(TEXT_CHUNKS.map((c, i) => chunkEvent(SessionSeq(i + 1), 1, 1, c))), 'utf8') + writeFileSync(file, replaySessionJsonl([TEXT_CHUNKS]), 'utf8') const overrideFile = join(dir, 'replay.override.json') writeFileSync(overrideFile, JSON.stringify({ patches: [ @@ -490,7 +975,7 @@ describe('loadReplayScript', () => { }) it('patches form: an out-of-range index fails loud with the derived length', () => { - writeFileSync(file, sessionJsonl(TEXT_CHUNKS.map((c, i) => chunkEvent(SessionSeq(i + 1), 1, 1, c))), 'utf8') + writeFileSync(file, replaySessionJsonl([TEXT_CHUNKS]), 'utf8') const overrideFile = join(dir, 'replay.override.json') writeFileSync(overrideFile, JSON.stringify({ patches: [{ at: 2, entry: { kind: 'hang' } }] }), 'utf8') expect(() => loadReplayScript({ file, overrideFile })).toThrow(/patch index 2 out of range.*1 call/s) @@ -522,7 +1007,7 @@ describe('loadReplayScript', () => { }) it('rejects duplicate patch indexes instead of silently taking the last one', () => { - writeFileSync(file, sessionJsonl(TEXT_CHUNKS.map((c, i) => chunkEvent(SessionSeq(i + 1), 1, 1, c))), 'utf8') + writeFileSync(file, replaySessionJsonl([TEXT_CHUNKS]), 'utf8') const overrideFile = join(dir, 'replay.override.json') writeFileSync(overrideFile, JSON.stringify({ patches: [ @@ -536,12 +1021,7 @@ describe('loadReplayScript', () => { describe('installLlmReplay (through the real LlmRuntime)', () => { function writeLog(...calls: StreamChunk[][]): void { - let seq = 1 - const events: SessionEvent[] = [] - calls.forEach((chunks, step) => { - for (const c of chunks) events.push(chunkEvent(SessionSeq(seq++), 1, step + 1, c)) - }) - writeFileSync(file, sessionJsonl(events), 'utf8') + writeFileSync(file, replaySessionJsonl(calls), 'utf8') } it('serves derived chunks back, short-circuiting the adapter', async () => { @@ -1038,10 +1518,7 @@ describe('installLlmReplay (through the real LlmRuntime)', () => { it('assertConsumed reports recorded scripts no live session ever bound', async () => { writeLog(TEXT_CHUNKS) const childFile = join(dir, 'session.1.jsonl') - writeFileSync(childFile, sessionJsonl( - TEXT_CHUNKS.map((chunk, i) => chunkEvent(SessionSeq(i + 1), 1, 1, chunk)), - { id: 'child', createdAt: 10 }, - ), 'utf8') + writeFileSync(childFile, replaySessionJsonl([TEXT_CHUNKS], { id: 'child', createdAt: 10 }), 'utf8') const ctx = new Context() await ctx.plugin(LlmRuntime) const handle = installLlmReplay(ctx, { file, childFiles: [childFile] }) @@ -1058,21 +1535,31 @@ describe('parseSessionHeader', () => { }) it('reads a non-zero v0 seedLength as the inherited event count', () => { - expect(parseSessionHeader('{"type":"session","version":0,"id":"child","createdAt":7,"seedLength":4}\n')) + const events: SessionEvent[] = Array.from({ length: 4 }, (_, seq) => ({ + type: 'permission/preset', seq: SessionSeq(seq), time: 0, data: { preset: 'workspace-write' }, + })) + expect(parseSessionHeader(sessionJsonl(events, { id: 'child', createdAt: 7, seedLength: 4 }))) .toEqual({ id: 'child', createdAt: 7, inheritedEventCount: 4 }) }) - it('falls back to id="" / createdAt=0 / inheritedEventCount=0 when the header lacks them', () => { - expect(parseSessionHeader('{"type":"session","version":0}\n')).toEqual({ id: '', createdAt: 0, inheritedEventCount: 0 }) + it('materializes omitted v0 delegation depth and tokenized cwd for projected validation', () => { + expect(parseSessionHeader( + '{"type":"session","version":0,"id":"projected","createdAt":7,"cwd":"{{cwd}}/workspace"}\n', + )).toEqual({ id: 'projected', createdAt: 7, inheritedEventCount: 0 }) }) - it('falls back on an empty buffer (no header line)', () => { - expect(parseSessionHeader('')).toEqual({ id: '', createdAt: 0, inheritedEventCount: 0 }) + it('rejects a projected header that lacks required identity fields', () => { + expect(() => parseSessionHeader('{"type":"session","version":0}\n')).toThrow(/lacks required member "id"/) + }) + + it('rejects an empty fixture instead of inventing header identity', () => { + expect(() => parseSessionHeader('')).toThrow(/must start with a session header/) }) it.each([-1, 0.5, Number.MAX_SAFE_INTEGER + 1])('rejects invalid v0 seedLength %s', (seedLength) => { - expect(() => parseSessionHeader(JSON.stringify({ type: 'session', version: 0, seedLength }))) - .toThrow(/SessionLogOffset/) + expect(() => parseSessionHeader(JSON.stringify({ + type: 'session', version: 0, id: 'invalid', createdAt: 0, seedLength, delegationDepth: 0, + }))).toThrow(/seedLength/) }) }) @@ -1108,16 +1595,30 @@ describe('loadSessionScripts', () => { const parentChunk: StreamChunk = { type: 'text-delta', index: 0, text: 'PARENT-RESPONSE' } const childChunks: StreamChunk[] = [{ type: 'text-delta', index: 0, text: 'CHILD-RESPONSE' }, { type: 'finish', reason: { kind: 'stop' } }] const f = writeSession('session.jsonl', { id: 'parent', createdAt: 100 }, [TEXT_CHUNKS]) - // The child fixture: 2 seeded parent events (a chunk + its finish) then the - // child's own turn. seedLength = 2 marks where the inherited prefix ends. + // The child fixture contains one complete inherited turn followed by its own turn. + // seedLength = 6 marks the exact cut between those lifecycles. const childEvents: SessionEvent[] = [ - chunkEvent(SessionSeq(0), 1, 1, parentChunk), - chunkEvent(SessionSeq(1), 1, 1, { type: 'finish', reason: { kind: 'stop' } }), - chunkEvent(SessionSeq(2), 2, 1, childChunks[0]!), - chunkEvent(SessionSeq(3), 2, 1, childChunks[1]!), + { type: 'turn/start', seq: SessionSeq(0), time: 0, data: { turn: 1 } }, + { type: 'step/start', seq: SessionSeq(1), time: 0, data: { turn: 1, step: 1 } }, + chunkEvent(SessionSeq(2), 1, 1, parentChunk), + chunkEvent(SessionSeq(3), 1, 1, { type: 'finish', reason: { kind: 'stop' } }), + { type: 'step/end', seq: SessionSeq(4), time: 0, data: { turn: 1, step: 1 } }, + { + type: 'turn/end', seq: SessionSeq(5), time: 0, + data: { turn: 1, reason: { kind: 'completed' } }, + }, + { type: 'turn/start', seq: SessionSeq(6), time: 0, data: { turn: 2 } }, + { type: 'step/start', seq: SessionSeq(7), time: 0, data: { turn: 2, step: 1 } }, + chunkEvent(SessionSeq(8), 2, 1, childChunks[0]!), + chunkEvent(SessionSeq(9), 2, 1, childChunks[1]!), + { type: 'step/end', seq: SessionSeq(10), time: 0, data: { turn: 2, step: 1 } }, + { + type: 'turn/end', seq: SessionSeq(11), time: 0, + data: { turn: 2, reason: { kind: 'completed' } }, + }, ] const childPath = join(dir, 'session.1.jsonl') - writeFileSync(childPath, sessionJsonl(childEvents, { id: 'child', createdAt: 200, seedLength: 2 }), 'utf8') + writeFileSync(childPath, sessionJsonl(childEvents, { id: 'child', createdAt: 200, seedLength: 6 }), 'utf8') const scripts = loadSessionScripts({ file: f, childFiles: [childPath] }) // The child script is ONLY the child's own model call — the parent's seeded @@ -1146,6 +1647,20 @@ describe('loadSessionScripts', () => { expect(scripts[0]).toMatchObject({ recordedId: '', createdAt: 0, primary: true }) }) + it('does not parse a whole-script override reused as the primary fixture path', () => { + const overrideFile = join(dir, 'replay.override.json') + writeFileSync(overrideFile, JSON.stringify([{ kind: 'hang' }]), 'utf8') + + const scripts = loadSessionScripts({ file: overrideFile, overrideFile }) + + expect(scripts).toEqual([{ + recordedId: '', + createdAt: 0, + entries: [{ kind: 'hang' }], + primary: true, + }]) + }) + it('orders two same-createdAt children deterministically after the primary', () => { // Two children sharing a createdAt (both non-primary): exercises the sort // tie-break\'s "both same primary-ness" arm and a non-primary-vs-primary arm. @@ -1341,7 +1856,7 @@ describe('apply (the plugin entry)', () => { }) it('installs replay and its catalog from explicit config', async () => { - writeFileSync(file, sessionJsonl(TEXT_CHUNKS.map((c, i) => chunkEvent(SessionSeq(i + 1), 1, 1, c))), 'utf8') + writeFileSync(file, replaySessionJsonl([TEXT_CHUNKS]), 'utf8') const ctx = new Context() await ctx.plugin(LlmRuntime) apply(ctx, { @@ -1358,7 +1873,7 @@ describe('apply (the plugin entry)', () => { }) it('declares flat image request pricing only for models that configure it', async () => { - writeFileSync(file, sessionJsonl(TEXT_CHUNKS.map((c, i) => chunkEvent(SessionSeq(i + 1), 1, 1, c))), 'utf8') + writeFileSync(file, replaySessionJsonl([TEXT_CHUNKS]), 'utf8') const ctx = new Context() await ctx.plugin(LlmRuntime) installLlmReplay(ctx, { @@ -1432,7 +1947,7 @@ describe('apply (the plugin entry)', () => { }) it('uses only the file when no override path is configured or in the env', async () => { - writeFileSync(file, sessionJsonl(TEXT_CHUNKS.map((c, i) => chunkEvent(SessionSeq(i + 1), 1, 1, c))), 'utf8') + writeFileSync(file, replaySessionJsonl([TEXT_CHUNKS]), 'utf8') process.env.DSH_SNAPSHOT_FILE = file delete process.env.DSH_SNAPSHOT_OVERRIDE const ctx = new Context() @@ -1461,9 +1976,9 @@ describe('apply (the plugin entry)', () => { { type: 'text-delta', index: 0, text: 'kid' }, { type: 'finish', reason: { kind: 'stop' } }, ] - writeFileSync(file, sessionJsonl(TEXT_CHUNKS.map((c, i) => chunkEvent(SessionSeq(i + 1), 1, 1, c)), { id: 'p', createdAt: 1 }), 'utf8') + writeFileSync(file, replaySessionJsonl([TEXT_CHUNKS], { id: 'p', createdAt: 1 }), 'utf8') const childFile = join(dir, 'session.1.jsonl') - writeFileSync(childFile, sessionJsonl(childSecond.map((c, i) => chunkEvent(SessionSeq(i + 1), 1, 1, c)), { id: 'c', createdAt: 2 }), 'utf8') + writeFileSync(childFile, replaySessionJsonl([childSecond], { id: 'c', createdAt: 2 }), 'utf8') const ctx = new Context() await ctx.plugin(LlmRuntime) apply(ctx, { file, childFiles: [childFile] }) @@ -1479,9 +1994,9 @@ describe('apply (the plugin entry)', () => { { type: 'text-delta', index: 0, text: 'env-kid' }, { type: 'finish', reason: { kind: 'stop' } }, ] - writeFileSync(file, sessionJsonl(TEXT_CHUNKS.map((c, i) => chunkEvent(SessionSeq(i + 1), 1, 1, c)), { id: 'p', createdAt: 1 }), 'utf8') + writeFileSync(file, replaySessionJsonl([TEXT_CHUNKS], { id: 'p', createdAt: 1 }), 'utf8') const childFile = join(dir, 'session.1.jsonl') - writeFileSync(childFile, sessionJsonl(childChunks.map((c, i) => chunkEvent(SessionSeq(i + 1), 1, 1, c)), { id: 'c', createdAt: 2 }), 'utf8') + writeFileSync(childFile, replaySessionJsonl([childChunks], { id: 'c', createdAt: 2 }), 'utf8') process.env.DSH_SNAPSHOT_FILE = file process.env.DSH_SNAPSHOT_CHILD_FILES = childFile // single entry, no delimiter needed const ctx = new Context() @@ -1494,7 +2009,7 @@ describe('apply (the plugin entry)', () => { }) it('ignores an empty $DSH_SNAPSHOT_CHILD_FILES (single-session)', async () => { - writeFileSync(file, sessionJsonl(TEXT_CHUNKS.map((c, i) => chunkEvent(SessionSeq(i + 1), 1, 1, c)), { id: 'p', createdAt: 1 }), 'utf8') + writeFileSync(file, replaySessionJsonl([TEXT_CHUNKS], { id: 'p', createdAt: 1 }), 'utf8') process.env.DSH_SNAPSHOT_FILE = file process.env.DSH_SNAPSHOT_CHILD_FILES = '' const ctx = new Context() diff --git a/packages/test-support/llm-replay/tests/session-format-corpus.spec.ts b/packages/test-support/llm-replay/tests/session-format-corpus.spec.ts new file mode 100644 index 0000000000..d2b023f27b --- /dev/null +++ b/packages/test-support/llm-replay/tests/session-format-corpus.spec.ts @@ -0,0 +1,80 @@ +import { readFileSync, readdirSync } from 'node:fs' +import { join, relative, resolve } from 'node:path' +import { describe, expect, it } from 'vitest' +import { SessionFormatUnsupportedMigrationError } from '@deepseek-ai/dsh-session-format-catalog' +import { parseSessionLog } from '../src/index.ts' +import { ALPHA_SESSION_FORMAT_REFUSAL_FIXTURES } from '../src/alpha-refusal-fixtures.ts' + +const repoRoot = resolve(import.meta.dirname, '../../../..') +const excludedDirectories = new Set(['dist', 'lib', 'node_modules']) + +const alphaRefusalManifest: ReadonlyMap = new Map( + ALPHA_SESSION_FORMAT_REFUSAL_FIXTURES.map(fixture => [fixture.repoRelativePath, fixture.expectedMessage]), +) + +function committedSessionFixtures(directory: string): string[] { + const files: string[] = [] + for (const entry of readdirSync(directory, { withFileTypes: true })) { + if (entry.isSymbolicLink()) continue + const path = join(directory, entry.name) + if (entry.isDirectory()) { + if (!excludedDirectories.has(entry.name)) files.push(...committedSessionFixtures(path)) + } else if (entry.name.startsWith('session') && entry.name.endsWith('.jsonl')) { + if (!/^session(?:\.[1-9]\d*)?(?:\.v[1-9]\d*)?\.jsonl$/.test(entry.name)) { + throw new Error(`invalid committed Session filename: ${path}`) + } + files.push(path) + } + } + return files +} + +function declaresFormat(text: string): boolean { + const firstLine = text.split(/\r?\n/).find(line => line.trim().length > 0) + if (firstLine === undefined) return false + const header = JSON.parse(firstLine) as unknown + return header !== null && typeof header === 'object' && !Array.isArray(header) + && Object.hasOwn(header, 'version') +} + +function filenameFormatVersion(path: string): number { + const match = /^session(?:\.[1-9]\d*)?(?:\.v([1-9]\d*))?\.jsonl$/.exec(path.split(/[/\\]/u).at(-1) ?? '') + if (match === null) throw new Error(`invalid committed Session filename: ${path}`) + return match[1] === undefined ? 0 : Number(match[1]) +} + +describe('committed Session format corpus', () => { + it('migrates every versioned fixture or matches one exact alpha refusal', () => { + const seenRefusals = new Set() + const files = ['snapshots', 'packages', 'scripts/snapshots/python-sdk-single-exe'] + .flatMap(root => committedSessionFixtures(join(repoRoot, root))) + .sort() + + for (const file of files) { + const source = readFileSync(file, 'utf8') + if (!declaresFormat(source)) continue + const key = relative(repoRoot, file).split('\\').join('/') + const header = JSON.parse(source.split(/\r?\n/u).find(line => line.trim().length > 0) ?? '{}') as { + version?: unknown + } + expect(header.version, `${key}: filename/header Session generation`).toBe(filenameFormatVersion(file)) + let failure: unknown + try { + parseSessionLog(source) + } catch (error: unknown) { + failure = error + } + const expected = alphaRefusalManifest.get(key) + if (expected === undefined) { + expect(failure, `${key}: unclassified Session format refusal`).toBeUndefined() + } else { + expect(failure, `${key}: refusal no longer occurs`) + .toBeInstanceOf(SessionFormatUnsupportedMigrationError) + expect((failure as Error).message).toBe(expected) + seenRefusals.add(key) + } + } + + expect([...seenRefusals].sort()).toEqual([...alphaRefusalManifest.keys()].sort()) + }) +}) diff --git a/packages/test-support/llm-replay/tsconfig.json b/packages/test-support/llm-replay/tsconfig.json index 76d3c0af5f..5fd6519355 100644 --- a/packages/test-support/llm-replay/tsconfig.json +++ b/packages/test-support/llm-replay/tsconfig.json @@ -25,6 +25,9 @@ }, { "path": "../../core/session" + }, + { + "path": "../../session/session-format-catalog" } ] } diff --git a/packages/test-support/session-snapshot/README.i18n.yaml b/packages/test-support/session-snapshot/README.i18n.yaml index b1bacd3426..04e655f460 100644 --- a/packages/test-support/session-snapshot/README.i18n.yaml +++ b/packages/test-support/session-snapshot/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/test-support/session-snapshot/README.md -README.md: 0e099dba28f45031bdaa3c187f3e3ffce5b3b2b9 -README.zh.md: f81b3038b14b94716f4f045aee2d70daa27930dc +README.md: 589020d29ca3f5d72d7994d64ad3ba2b0312136d +README.zh.md: 453680dc706141acfc348d830f2eaa09a084b126 diff --git a/packages/test-support/session-snapshot/README.md b/packages/test-support/session-snapshot/README.md index 0e099dba28..589020d29c 100644 --- a/packages/test-support/session-snapshot/README.md +++ b/packages/test-support/session-snapshot/README.md @@ -68,17 +68,17 @@ defineAcpSnapshotSuite({ }) ``` -Each recorded-session directory carries a closed `snapshot.yml` manifest plus its `session.jsonl` and contiguous `session..jsonl` child logs. The manifest names the scenario, shipped profile, composition/header class, recording source, and only the replay, platform, permission, environment, workspace, or input facts the completed session cannot reconstruct. The adapter registers expected-output, session-log, and optional `workspace.expected/` comparisons; guards reject orphan directories, missing files, absolute paths, malformed manifests, and platform-specific separators. +Each recorded-session directory carries a closed `snapshot.yml` manifest plus canonical parent and contiguous child roles. Parent filenames are `session[.vN].jsonl`; children are `session.[.vN].jsonl`; v0 omits `.v0`, positive versions use lowercase `.vN`, and every filename agrees with its header. A role may retain older generations, but the harness selects the numerically highest one. The manifest names the scenario, shipped profile, composition/header class, recording source, and only the replay, platform, permission, environment, workspace, or input facts the completed Session cannot reconstruct. The adapter registers expected-output, Session-log, and optional `workspace.expected/` comparisons; guards reject orphan directories, missing roles, noncanonical names, absolute paths, malformed manifests, and platform-specific separators. -`normalizeSessionSnapshot` retains the complete session header and event payloads but omits ordinary `seq`/`time` and packed-row `seq0`/`time0` envelopes from committed fixtures after normalizing paths and scrubbing request headers. Replay synthesizes the envelopes in memory, while runtime persistence continues to write complete logs. Fixtures use canonical packed rows; the [temporary repository migrator](../../../scripts/migrate-packed-session-fixtures.ts) (`pnpm run migrate:packed-session-fixtures`) rewrites older layouts, and its [removal proposal](../../../.agents/notes/proposed/process/2026-07-26-remove-packed-session-fixture-migrator.md) owns its deletion. +`normalizeSessionSnapshot` retains the complete Session header and event payloads but omits ordinary `seq`/`time` and packed-row `seq0`/`time0` envelopes from committed fixtures after normalizing paths and scrubbing request headers. Replay synthesizes the envelopes in memory, while runtime persistence continues to write complete logs. Multi-session comparison restores every selected persisted or projected fixture through the current build-static Session format catalog before identity redaction and normalization, so a retained suffixless v0 replay input and fresh `session.v1.jsonl` writer output compare as one v1 logical Session without rewriting or renaming the v0 file. Expected logs may carry aligned `sourcePaths`; only the two exact alpha-refusal paths receive header-generation normalization after their pinned catalog refusal, while harvested current logs remain pathless and strict. Versionless protocol-adapter unit fixtures remain outside the released Session format corpus. Fixtures use canonical packed rows; the [temporary repository migrator](../../../scripts/migrate-packed-session-fixtures.ts) (`pnpm run migrate:packed-session-fixtures`) rewrites older row layouts, and its [removal proposal](../../../.agents/notes/proposed/process/2026-07-26-remove-packed-session-fixture-migrator.md) owns its deletion. ### Record, replay, and refresh -`pnpm run test:snapshot:record` calls the live LLM and rewrites recorded model fixtures; `pnpm run test:snapshot:refresh` stays keyless, runs the replay overlay, and rewrites stdout, comparable session-log expected outputs, and owned prompt and tool-schema sidecars from committed model scripts. Each composition owner keeps its replay patch beside its live patch; top-level `snapshots/` owns session-driven scenarios, while other expected outputs stay beside their owning package. [`dsh-llm-replay`](../llm-replay/README.md) serves the recorded streams selected through `DSH_SNAPSHOT_*` environment values. +`pnpm run test:snapshot:record` calls the live LLM and writes the harvested current generation under its canonical versioned filename; it never deletes a completed generation, including generations of a child role absent from a later recording. `pnpm run test:snapshot:refresh` stays keyless, runs the selected highest replay input, and writes stdout, the fresh current-generation comparable Session output, and owned prompt and tool-schema sidecars from committed model scripts. Each composition owner keeps its replay patch beside its live patch; top-level `snapshots/` owns Session-driven scenarios, while other expected outputs stay beside their owning package. [`dsh-llm-replay`](../llm-replay/README.md) serves the recorded streams selected through `DSH_SNAPSHOT_*` environment values. ### Pinning request headers -A pin owns its generated `system-prompt.expected.md` or `tool-schemas.expected.json` sidecar by default; `systemPromptSource` and `toolSchemasSource` name another pin when the complete corresponding sequence is identical, so each distinct version is committed once. The pin's `session.jsonl` stores `"system":"{{system}}","tools":"{{tools}}"` while retaining config, reason, and any model-visible prefix. A child session whose own scope composes a different request declares it per fixture index with `pinsChildToolSchemas` and `pinsChildSystemPrompts`. A scenario that changes the request header mid-run declares `expectedHeaderChanges`. +A pin owns its generated `system-prompt.expected.md` or `tool-schemas.expected.json` sidecar by default; `systemPromptSource` and `toolSchemasSource` name another pin when the complete corresponding sequence is identical, so each distinct version is committed once. The selected highest parent fixture stores `"system":"{{system}}","tools":"{{tools}}"` while retaining config, reason, and any model-visible prefix. A child Session whose own scope composes a different request declares it per fixture index with `pinsChildToolSchemas` and `pinsChildSystemPrompts`. A scenario that changes the request header mid-run declares `expectedHeaderChanges`. ### Platform and composition variants @@ -102,7 +102,7 @@ This section explains the design of the kit; the observable behavior is fully co ### Design -The shared core owns manifests, workspace setup/comparison, typed identity mapping, normalizers, and fixture invariants. The ACP adapter adds four composable layers: launcher, scenario harness, normalizers, and suite factory. `launchAcpTestAgent` boots a source profile under tsx or a built `lib` profile under plain Node, connects the SDK client over a raw-byte stdout tee, collects session updates and stderr, fails closed on unhandled permission requests, and owns shutdown. `runScenario` drives ACP JSON-RPC stdio and harvests every persisted raw JSONL session log. The pure normalizers replace cwd paths and typed identities with stable tokens, zero times, expand physical provenance ranges, and scrub request-header bulk. `defineAcpSnapshotSuite` registers comparisons, fixture write-back, and the live uniformity guard. +The shared core owns manifests, generation-qualified role selection, workspace setup/comparison, typed identity mapping, normalizers, and fixture invariants. The ACP adapter adds four composable layers: launcher, scenario harness, normalizers, and suite factory. `launchAcpTestAgent` boots a source profile under tsx or a built `lib` profile under plain Node, connects the SDK client over a raw-byte stdout tee, collects Session updates and stderr, fails closed on unhandled permission requests, and owns shutdown. `runScenario` drives ACP JSON-RPC stdio and harvests the numerically highest persisted raw JSONL generation for every Session directory. The pure normalizers replace cwd paths and typed identities with stable tokens, zero times, expand physical provenance ranges, and scrub request-header bulk. `defineAcpSnapshotSuite` registers comparisons, generation-qualified fixture write-back, and the live uniformity guard. ### Source map @@ -111,6 +111,7 @@ The shared core owns manifests, workspace setup/comparison, typed identity mappi | [`src/launcher.ts`](src/launcher.ts) | Subprocess/client launcher and shutdown ownership | | [`src/harness.ts`](src/harness.ts) | Scripted scenario driver and session-log harvest | | [`src/manifest.ts`](src/manifest.ts) | Closed `snapshot.yml` schema, collection, and ownership rules | +| [`src/session-files.ts`](src/session-files.ts) | Canonical parent/child generation grammar, header agreement, and highest-role selection | | [`src/identity.ts`](src/identity.ts) | Typed first-seen identity tokenization across parent and child logs | | [`src/normalize.ts`](src/normalize.ts) | Pure normalizers and scrubbing helpers | | [`src/workspace.ts`](src/workspace.ts) | Scenario workspace setup and complete expected-state comparison | diff --git a/packages/test-support/session-snapshot/README.zh.md b/packages/test-support/session-snapshot/README.zh.md index f81b3038b1..453680dc70 100644 --- a/packages/test-support/session-snapshot/README.zh.md +++ b/packages/test-support/session-snapshot/README.zh.md @@ -68,17 +68,17 @@ defineAcpSnapshotSuite({ }) ``` -每个已记录会话目录携带封闭的 `snapshot.yml` manifest，以及自身的 `session.jsonl` 与连续的 `session..jsonl` 子会话日志。manifest 指名场景、随附 profile、组合／header 类别、录制来源，以及已完成会话无法重建的 replay、平台、权限、环境、workspace 或输入事实。适配器注册预期输出、会话日志与可选 `workspace.expected/` 比较；保护会拒绝遗留目录、缺失文件、绝对路径、畸形 manifest 与平台专用分隔符。 +每个已记录 Session 目录携带封闭的 `snapshot.yml` manifest，以及规范 parent 与连续 child 角色。parent 文件名是 `session[.vN].jsonl`；child 是 `session.[.vN].jsonl`；v0 省略 `.v0`，正版本使用小写 `.vN`，且每个文件名与其 header 一致。一个角色可以保留旧 generation，但 harness 会选择数值最高的一项。manifest 指名场景、随附 profile、组合／header 类别、录制来源，以及已完成 Session 无法重建的 replay、平台、权限、环境、workspace 或输入事实。适配器注册预期输出、Session 日志与可选 `workspace.expected/` 比较；保护会拒绝遗留目录、缺失角色、非规范名称、绝对路径、malformed manifest 与平台专用分隔符。 -`normalizeSessionSnapshot` 在规范化路径并清理 request header 后，会保留完整会话 header 与事件 payload，但从已提交 fixture 中省略普通行的 `seq`/`time` 和打包行的 `seq0`/`time0` envelope。回放只在内存中合成这些 envelope，而运行时持久化仍写入完整日志。fixture 使用规范打包行；[临时仓库迁移器](../../../scripts/migrate-packed-session-fixtures.ts)（`pnpm run migrate:packed-session-fixtures`）会改写较旧的布局，由其[移除提案](../../../.agents/notes/proposed/process/2026-07-26-remove-packed-session-fixture-migrator.zh.md)负责删除该迁移器。 +`normalizeSessionSnapshot` 在规范化路径并清理 request header 后，会保留完整 Session header 与事件 payload，但从已提交 fixture 中省略普通行的 `seq`/`time` 和打包行的 `seq0`/`time0` envelope。回放只在内存中合成这些 envelope，而运行时持久化仍写入完整日志。多 Session 比较会先通过当前构建期静态 Session 格式目录恢复每个选定的持久化或投影 fixture，再进行身份脱敏与规范化，因此保留的无后缀 v0 replay 输入与新生成的 `session.v1.jsonl` writer 输出会作为同一个 v1 逻辑 Session 比较，且不会重写或重命名 v0 文件。预期日志可以携带对齐的 `sourcePaths`；只有两项精确 alpha 拒绝路径会在 catalog 返回固定拒绝后规范化 header generation，而收集到的当前日志仍无路径且保持严格。无版本的协议适配器单元测试 fixture 不属于已发布 Session 格式语料。fixture 使用规范打包行；[临时仓库迁移器](../../../scripts/migrate-packed-session-fixtures.ts)（`pnpm run migrate:packed-session-fixtures`）会改写较旧行布局，由其[移除提案](../../../.agents/notes/proposed/process/2026-07-26-remove-packed-session-fixture-migrator.zh.md)负责删除该迁移器。 ### 录制、回放与刷新 -`pnpm run test:snapshot:record` 调用在线 LLM（大语言模型），并重写已录制的模型 fixture；`pnpm run test:snapshot:refresh` 保持无密钥，运行回放 overlay，并从已提交模型脚本重写 stdout、可比较会话日志预期输出，以及各 pin 自有的提示词与工具 schema 伴随文件。每个组合 owner 把 replay patch 放在 live patch 旁；顶层 `snapshots/` 拥有会话驱动场景，其他预期输出留在其包 owner 旁。[`dsh-llm-replay`](../llm-replay/README.zh.md) 提供通过 `DSH_SNAPSHOT_*` 环境值选择的已记录流。 +`pnpm run test:snapshot:record` 调用在线 LLM（大语言模型），并在规范具名版本文件下写入收集到的当前 generation；它绝不删除已完成的 generation，即使后续录制不再产生某个 child 角色也一样。`pnpm run test:snapshot:refresh` 保持无密钥，运行选定的最高 replay 输入，并写入 stdout、新鲜当前 generation 的可比较 Session 输出，以及各 pin 自有的 prompt 与工具 schema sidecar。每个组合 owner 把 replay patch 放在 live patch 旁；顶层 `snapshots/` 拥有 Session 驱动场景，其他预期输出留在其 package owner 旁。[`dsh-llm-replay`](../llm-replay/README.zh.md) 提供通过 `DSH_SNAPSHOT_*` 环境值选择的已记录流。 ### 固定请求 header -每个 pin 默认拥有其生成的 `system-prompt.expected.md` 或 `tool-schemas.expected.json` 伴随文件；当完整的对应序列相同时，`systemPromptSource` 与 `toolSchemasSource` 指定另一个 pin 作为来源，因此每个不同版本只提交一次。该 pin 的 `session.jsonl` 存储 `"system":"{{system}}","tools":"{{tools}}"`，同时保留配置、原因与任何模型可见前缀。自身作用域组合出不同请求的子会话按 fixture 索引以 `pinsChildToolSchemas` 与 `pinsChildSystemPrompts` 单独声明。运行中改变请求 header 的场景声明 `expectedHeaderChanges`。 +每个 pin 默认拥有其生成的 `system-prompt.expected.md` 或 `tool-schemas.expected.json` sidecar；当完整的对应序列相同时，`systemPromptSource` 与 `toolSchemasSource` 指定另一个 pin 作为来源，因此每个不同版本只提交一次。选定的最高 parent fixture 存储 `"system":"{{system}}","tools":"{{tools}}"`，同时保留配置、原因与任何模型可见前缀。自身 scope 组合出不同请求的 child Session 按 fixture index 以 `pinsChildToolSchemas` 与 `pinsChildSystemPrompts` 单独声明。运行中改变请求 header 的场景声明 `expectedHeaderChanges`。 ### 平台与组合变体 @@ -102,7 +102,7 @@ defineAcpSnapshotSuite({ ### 设计 -共享核心拥有 manifest、workspace 设置／比较、类型化身份映射、规范化器与 fixture 不变式。ACP 适配器增加四个可组合层：启动器、场景 harness、规范化器与套件工厂。`launchAcpTestAgent` 在 tsx 下启动源码 profile，或在普通 Node 下启动已构建 `lib` profile，通过原始字节 stdout tee 连接 SDK 客户端，收集会话更新与 stderr，默认拒绝未处理的权限请求，并负责关闭。`runScenario` 驱动 ACP JSON-RPC stdio，并收集每个持久化原始 JSONL 会话日志。纯规范化器把 cwd 路径与类型化身份变为稳定 token，将时间归零、展开物理来源区间，并擦除请求 header bulk。`defineAcpSnapshotSuite` 注册比较、fixture 回写与实时一致性保护。 +共享核心拥有 manifest、generation 限定角色选择、workspace 设置／比较、类型化身份映射、normalizer 与 fixture 不变式。ACP 适配器增加四个可组合层：launcher、场景 harness、normalizer 与 suite factory。`launchAcpTestAgent` 在 tsx 下启动源码 profile，或在普通 Node 下启动已构建 `lib` profile，通过原始字节 stdout tee 连接 SDK client，收集 Session update 与 stderr，默认拒绝未处理的权限请求，并负责关闭。`runScenario` 驱动 ACP JSON-RPC stdio，并收集每个 Session 目录中数值最高的持久原始 JSONL generation。纯 normalizer 把 cwd 路径与类型化身份变为稳定 token，将时间归零、展开物理来源区间，并清理 request header bulk。`defineAcpSnapshotSuite` 注册比较、generation 限定 fixture 回写与实时一致性保护。 ### 源码地图 @@ -111,6 +111,7 @@ defineAcpSnapshotSuite({ | [`src/launcher.ts`](src/launcher.ts) | 子进程/客户端启动器与关闭所有权 | | [`src/harness.ts`](src/harness.ts) | 脚本化场景驱动与会话日志收集 | | [`src/manifest.ts`](src/manifest.ts) | 封闭 `snapshot.yml` schema、收集与归属规则 | +| [`src/session-files.ts`](src/session-files.ts) | 规范 parent/child generation grammar、header 一致性与最高角色选择 | | [`src/identity.ts`](src/identity.ts) | 跨父子日志的类型化首次出现身份 token 化 | | [`src/normalize.ts`](src/normalize.ts) | 纯规范化器与擦除辅助 | | [`src/workspace.ts`](src/workspace.ts) | 场景 workspace 设置与完整预期状态比较 | diff --git a/packages/test-support/session-snapshot/package.json b/packages/test-support/session-snapshot/package.json index a526db3ee6..2b3d971faf 100644 --- a/packages/test-support/session-snapshot/package.json +++ b/packages/test-support/session-snapshot/package.json @@ -29,6 +29,7 @@ "dependencies": { "@agentclientprotocol/sdk": "1.4.0", "@deepseek-ai/cordis-plugin-include": "workspace:*", + "@deepseek-ai/dsh-llm-replay": "workspace:^", "@deepseek-ai/dsh-loader-smoke": "workspace:*", "js-yaml": "^4.2.0", "vitest": "^4.1.8" diff --git a/packages/test-support/session-snapshot/src/harness.ts b/packages/test-support/session-snapshot/src/harness.ts index 628d334337..1e07eb0bbf 100644 --- a/packages/test-support/session-snapshot/src/harness.ts +++ b/packages/test-support/session-snapshot/src/harness.ts @@ -35,6 +35,10 @@ import { type AgentUnderTest, type LaunchedAcpTestAgent, } from './launcher.ts' +import { + assertPersistedSessionVersion, + latestPersistedSessionPaths, +} from './session-files.ts' import { captureWorkspaceSnapshot, type WorkspaceSnapshotEntry } from './workspace.ts' export type { AgentUnderTest } from './launcher.ts' @@ -752,13 +756,14 @@ function latestOpenTurn(content: string): number | undefined { } /** - * Harvest EVERY persisted `.jsonl` session log under a sessions root, parse each + * Harvest every latest-generation raw JSONL Session under a sessions root, parse each * header line, and return them ordered primary-first: the top-level session (no * `parentSession`) leads, then each subagent child by ascending `createdAt`. * - * Snapshot configs select the JSONL backend's raw mode, which lays sessions - * out as `///session.jsonl`. Recursive collection - * catches the primary and every child session. Returns `[]` if no log was + * Snapshot configs select the JSONL backend's raw mode, which lays each immutable + * generation beneath `///`. Recursive collection + * chooses the numerically highest generation for the primary and every child. + * Returns `[]` if no log was * produced (a no-session scenario). */ async function harvestSessionLogs(root: string): Promise { @@ -769,9 +774,10 @@ async function harvestSessionLogs(root: string): Promise { return [] } const logs: HarvestedLog[] = [] - for (const file of files) { - if (basename(file) !== 'session.jsonl') continue + for (const file of latestPersistedSessionPaths(files)) { const content = await readFile(join(root, file), 'utf8') + assertPersistedSessionVersion(basename(file), content) + /* v8 ignore next -- the generation validator above rejects header-less content. */ const firstLine = content.split('\n').find(line => line.trim().length > 0) ?? '{}' const header = JSON.parse(firstLine) as { id?: unknown; createdAt?: unknown; parentSession?: unknown } logs.push({ diff --git a/packages/test-support/session-snapshot/src/index.ts b/packages/test-support/session-snapshot/src/index.ts index 053ab53493..4fc0c590d9 100644 --- a/packages/test-support/session-snapshot/src/index.ts +++ b/packages/test-support/session-snapshot/src/index.ts @@ -39,6 +39,7 @@ export { } from './launcher.ts' export { extractSnapshotSpillPaths, + normalizeSessionFormatProvenance, normalizeSessionLog, normalizeSessionSnapshot, normalizeSessionSnapshots, @@ -51,6 +52,7 @@ export { type CwdPathMode, type NormalizeContext, type NormalizeOptions, + type NormalizeSessionSnapshotsOptions, } from './normalize.ts' export { parseSnapshotManifest, @@ -66,6 +68,20 @@ export { type SnapshotSessionReference, type SnapshotWorkspaceManifest, } from './manifest.ts' +export { + assertPersistedSessionVersion, + assertSessionFixtureVersion, + latestPersistedSessionPaths, + parsePersistedSessionFilename, + parseSessionFixtureName, + persistedSessionFilename, + sessionFixtureFiles, + sessionFixtureName, + sessionFixtureNames, + sessionHeaderVersion, + type PersistedSessionFile, + type SessionFixtureFile, +} from './session-files.ts' export { formatSystemPromptSnapshot, formatToolSchemasSnapshot, @@ -78,7 +94,6 @@ export { parseToolSchemasSnapshot, refreshFixtureReplacements, restorePinnedToolSchemas, - sessionFixtureNames, stabilizeFixtureMessageIds, stabilizeRefreshLog, type Scenario, diff --git a/packages/test-support/session-snapshot/src/manifest.ts b/packages/test-support/session-snapshot/src/manifest.ts index 1067343172..ef628cd9b4 100644 --- a/packages/test-support/session-snapshot/src/manifest.ts +++ b/packages/test-support/session-snapshot/src/manifest.ts @@ -69,7 +69,7 @@ export interface SnapshotInputManifest { /** Optional reference to another scenario's canonical session. */ export interface SnapshotSessionReference { - /** Repository-relative POSIX path from this scenario directory to the owning `session.jsonl`. */ + /** Repository-relative POSIX path to the owning scenario's selected parent Session fixture. */ source: string } @@ -99,7 +99,7 @@ export interface SnapshotManifest { workspace?: SnapshotWorkspaceManifest /** Exceptional controller input absent for ordinary log-driven scenarios. */ input?: SnapshotInputManifest - /** Absent when this directory owns `session.jsonl`; present for a read-only borrower. */ + /** Absent when this directory owns its selected parent fixture; present for a read-only borrower. */ session?: SnapshotSessionReference } diff --git a/packages/test-support/session-snapshot/src/normalize.ts b/packages/test-support/session-snapshot/src/normalize.ts index 6e8903bf2a..4eb25276e5 100644 --- a/packages/test-support/session-snapshot/src/normalize.ts +++ b/packages/test-support/session-snapshot/src/normalize.ts @@ -14,6 +14,7 @@ import { SessionSeq, type SessionEvent, } from '@deepseek-ai/dsh-session' +import { prepareSessionSnapshotFixtureForComparison } from '@deepseek-ai/dsh-llm-replay' import { redactSessionSnapshotIds } from './identity.ts' const SESSION_ID = '{{sessionId}}' @@ -108,6 +109,12 @@ export interface NormalizeOptions { identityMode?: 'legacy' | 'preserve' } +/** Multi-Session comparison controls, including optional committed source identities. */ +export interface NormalizeSessionSnapshotsOptions extends Omit { + /** Primary-first source paths; exact alpha-refusal fixtures receive replay-only comparison policy. */ + sourcePaths?: readonly (string | undefined)[] +} + /** Return every known spelling of the generated cwd, most specific first. */ function cwdSpellings(ctx: NormalizeContext): string[] { const spellings = [...new Set([ctx.cwd, ...ctx.cwdAliases ?? []])] @@ -436,17 +443,91 @@ export function normalizeSessionSnapshot( export function normalizeSessionSnapshots( rawLogs: readonly string[], ctx: NormalizeContext, - options: Omit = {}, + options: NormalizeSessionSnapshotsOptions = {}, ): string[] { - return redactSessionSnapshotIds(rawLogs).map(log => repackSessionSnapshot( + const { sourcePaths, ...normalizeOptions } = options + if (sourcePaths !== undefined && sourcePaths.length !== rawLogs.length) { + throw new Error('Session snapshot source path count must match its log count') + } + const currentLogs = rawLogs.map((log, index) => hasSessionFormatVersion(log) + ? prepareSessionSnapshotFixtureForComparison(log, sourcePaths?.[index]) + : log) + const comparableLogs = currentLogs.map(normalizeSessionFormatProvenance) + return redactSessionSnapshotIds(comparableLogs).map(log => repackSessionSnapshot( scrubSessionSnapshot(normalizeSessionLog( log, { ...ctx, sessionIds: [] }, - { ...options, identityMode: 'preserve' }, + { ...normalizeOptions, identityMode: 'preserve' }, )), )) } +/** + * Omit only generation-qualified operational provenance from expected-output comparison. + * @param rawLog - Session records or events as compact JSON lines. + * @returns the same records without delivery or captured-source generation qualifiers. + */ +export function normalizeSessionFormatProvenance(rawLog: string): string { + return rawLog.split('\n').map((line) => { + if (line.trim().length === 0) return line + const record = JSON.parse(line) as Record + let changed = normalizeCapturedFormatProvenance(record) + if (record.type === 'session' && Object.hasOwn(record, 'version')) { + delete record.version + changed = true + } + if (record.type === 'session-log-deepseek/delivery-accepted' + && record.data !== null && typeof record.data === 'object' && !Array.isArray(record.data)) { + const data = { ...record.data as Record } + if (Object.hasOwn(data, 'sessionFormatVersion')) { + delete data.sessionFormatVersion + record.data = data + changed = true + } + } + return changed ? JSON.stringify(record) : line + }).join('\n') +} + +/** Omit captured generations only from an actual current Message source position. */ +function normalizeCapturedFormatProvenance(event: Record): boolean { + if (event.data === null || typeof event.data !== 'object' || Array.isArray(event.data)) return false + const data = event.data as Record + const message = event.type === 'user/message' + ? data + : event.type === 'assistant/message' || event.type === 'tool/result' + ? data.message + : undefined + if (message === null || typeof message !== 'object' || Array.isArray(message)) return false + const source = (message as Record).source + if (source === null || typeof source !== 'object' || Array.isArray(source)) return false + const record = source as Record + if (record.kind !== 'session-reference' || record.form !== 'recall' || record.version !== 1 + || !Array.isArray(record.references)) return false + let changed = false + for (const reference of record.references) { + if (reference === null || typeof reference !== 'object' || Array.isArray(reference)) continue + const captured = reference as Record + if (Object.hasOwn(captured, 'capturedFormatVersion')) { + delete captured.capturedFormatVersion + changed = true + } + } + return changed +} + +/** Whether a fixture declares a released Session format and therefore participates in migration burn-in. */ +function hasSessionFormatVersion(rawLog: string): boolean { + const firstLine = rawLog.split(/\r?\n/).find(line => line.trim().length > 0) + if (firstLine === undefined) throw new Error('session snapshot must start with a session header') + const header = JSON.parse(firstLine) as unknown + if (header === null || typeof header !== 'object' || Array.isArray(header) + || (header as Record)['type'] !== 'session') { + throw new Error('session snapshot must start with a session header') + } + return Object.hasOwn(header, 'version') +} + /** * Replace system-prompt content in request headers with `{{system}}` tokens * while retaining field presence. diff --git a/packages/test-support/session-snapshot/src/session-files.ts b/packages/test-support/session-snapshot/src/session-files.ts new file mode 100644 index 0000000000..0199f8c553 --- /dev/null +++ b/packages/test-support/session-snapshot/src/session-files.ts @@ -0,0 +1,245 @@ +/** Immutable Session-generation filenames used by recorded-session fixtures. */ + +import { basename, dirname } from 'node:path' + +/** One canonical recorded-session fixture filename. */ +export interface SessionFixtureFile { + /** Parent is `0`; positive values are child/ordinal slots. */ + readonly index: number + /** Physical Session format generation; zero is encoded by omission. */ + readonly version: number + /** Canonical filename. */ + readonly name: string +} + +/** One canonical persistence filename in a Session's own storage directory. */ +export interface PersistedSessionFile { + /** Physical Session format generation; zero is encoded by omission. */ + readonly version: number + /** Physical compression selected by the backend. */ + readonly compression: 'raw' | 'zstd' + /** Canonical basename. */ + readonly name: string +} + +const FIXTURE_FILE = /^session(?:\.([1-9]\d*))?(?:\.v([1-9]\d*))?\.jsonl$/u +const PERSISTED_FILE = /^session(?:\.v([1-9]\d*))?\.jsonl(\.zstd)?$/u + +function nonNegativeSafeInteger(value: number, label: string): void { + if (!Number.isSafeInteger(value) || value < 0 || Object.is(value, -0)) { + throw new Error(`${label} must be a non-negative safe integer`) + } +} + +/** + * Return the canonical fixture filename for one parent/ordinal and generation. + * + * @param index - Parent `0` or a positive child/ordinal slot. + * @param version - Physical format generation; `0` is omitted. + * @returns The lowercase canonical JSONL filename. + */ +export function sessionFixtureName(index: number, version: number): string { + nonNegativeSafeInteger(index, 'session fixture index') + nonNegativeSafeInteger(version, 'Session format version') + const ordinal = index === 0 ? '' : `.${index}` + const generation = version === 0 ? '' : `.v${version}` + return `session${ordinal}${generation}.jsonl` +} + +/** + * Parse one canonical recorded-session fixture filename. + * + * @param name - Basename from a scenario directory. + * @returns Parsed role and generation, or `undefined` for an unrelated file. + */ +export function parseSessionFixtureName(name: string): SessionFixtureFile | undefined { + const match = FIXTURE_FILE.exec(name) + if (match === null) { + if (name.startsWith('session') && name.endsWith('.jsonl')) { + throw new Error(`invalid session fixture name: ${name}`) + } + return undefined + } + const index = match[1] === undefined ? 0 : Number(match[1]) + const version = match[2] === undefined ? 0 : Number(match[2]) + if (!Number.isSafeInteger(index) || !Number.isSafeInteger(version)) { + throw new Error(`invalid session fixture name: ${name}`) + } + return { index, version, name } +} + +/** + * Select the highest generation for every parent/ordinal fixture role. + * Older generations remain in the directory but do not count as extra Sessions. + * + * @param names - File basenames in one scenario directory. + * @returns Parent first, followed by contiguous child/ordinal roles. + */ +export function sessionFixtureFiles(names: readonly string[]): SessionFixtureFile[] { + const selected = new Map() + const identities = new Set() + for (const name of names) { + const fixture = parseSessionFixtureName(name) + if (fixture === undefined) continue + const identity = `${fixture.index}/${fixture.version}` + if (identities.has(identity)) throw new Error(`duplicate session fixture generation: ${name}`) + identities.add(identity) + const previous = selected.get(fixture.index) + if (previous === undefined || fixture.version > previous.version) selected.set(fixture.index, fixture) + } + const primary = selected.get(0) + if (primary === undefined) throw new Error('missing parent session fixture') + const ordered = [...selected.values()].sort((left, right) => left.index - right.index) + for (const [offset, fixture] of ordered.entries()) { + if (fixture.index !== offset) { + throw new Error(`session fixture roles must be contiguous: expected index ${offset}, found ${fixture.name}`) + } + } + return ordered +} + +/** + * Validate and order a scenario directory's selected Session fixture filenames. + * + * @param names - File basenames in one scenario directory. + * @returns Highest-generation parent and child/ordinal filenames. + */ +export function sessionFixtureNames(names: readonly string[]): string[] { + return sessionFixtureFiles(names).map(file => file.name) +} + +/** + * Read the declared physical generation from one Session JSONL header. + * + * @param content - Complete UTF-8 JSONL content. + * @param label - Diagnostic filename or path. + * @returns The declared non-negative format generation. + */ +export function sessionHeaderVersion(content: string, label: string): number { + const line = content.split(/\r?\n/u).find(candidate => candidate.trim().length > 0) + if (line === undefined) throw new Error(`${label}: session fixture is empty`) + let value: unknown + try { + value = JSON.parse(line) as unknown + } catch (error) { + throw new Error(`${label}: session header contains invalid JSON`, { cause: error }) + } + if (value === null || typeof value !== 'object' || Array.isArray(value) + || (value as { type?: unknown }).type !== 'session') { + throw new Error(`${label}: first record must be a Session header`) + } + const version = (value as { version?: unknown }).version + if (!Number.isSafeInteger(version) || (version as number) < 0 || Object.is(version, -0)) { + throw new Error(`${label}: Session header version must be a non-negative safe integer`) + } + return version as number +} + +/** + * Require one fixture's canonical filename generation to equal its header. + * + * @param name - Canonical fixture basename. + * @param content - Complete UTF-8 JSONL content. + * @returns The validated format generation. + */ +export function assertSessionFixtureVersion(name: string, content: string): number { + const fixture = parseSessionFixtureName(name) + if (fixture === undefined) throw new Error(`not a session fixture name: ${name}`) + const first = content.split(/\r?\n/u).find(candidate => candidate.trim().length > 0) + if (first !== undefined) { + let projected: unknown + try { + projected = JSON.parse(first) as unknown + } catch { + projected = undefined + } + if (projected !== null && typeof projected === 'object' && !Array.isArray(projected) + && (projected as { type?: unknown }).type === 'session' + && !Object.hasOwn(projected, 'version')) { + return fixture.version + } + } + const headerVersion = sessionHeaderVersion(content, name) + if (headerVersion !== fixture.version) { + throw new Error( + `${name}: filename declares Session format v${fixture.version}, header declares v${headerVersion}`, + ) + } + return headerVersion +} + +/** + * Return the canonical persistence basename for one generation and compression. + * + * @param version - Physical format generation; `0` is omitted. + * @param compression - Backend compression mode. + * @returns The canonical persistence basename. + */ +export function persistedSessionFilename( + version: number, + compression: 'raw' | 'zstd' = 'raw', +): string { + nonNegativeSafeInteger(version, 'Session format version') + return `session${version === 0 ? '' : `.v${version}`}.jsonl${compression === 'zstd' ? '.zstd' : ''}` +} + +/** + * Parse a canonical persistence basename from a Session's own directory. + * + * @param name - Candidate basename. + * @returns Its generation and compression, or `undefined` for noise and noncanonical names. + */ +export function parsePersistedSessionFilename(name: string): PersistedSessionFile | undefined { + const match = PERSISTED_FILE.exec(name) + if (match === null) return undefined + const version = match[1] === undefined ? 0 : Number(match[1]) + if (!Number.isSafeInteger(version)) return undefined + return { + version, + compression: match[2] === undefined ? 'raw' : 'zstd', + name, + } +} + +/** + * Select one highest-generation persistence path per physical Session directory. + * + * @param paths - Relative or absolute paths beneath a sessions root. + * @param compression - Compression selected by the snapshot composition. + * @returns Stable path order with older generations and filesystem noise omitted. + */ +export function latestPersistedSessionPaths( + paths: readonly string[], + compression: 'raw' | 'zstd' = 'raw', +): string[] { + const selected = new Map() + for (const path of paths) { + const parsed = parsePersistedSessionFilename(basename(path)) + if (parsed === undefined || parsed.compression !== compression) continue + const directory = dirname(path) + const previous = selected.get(directory) + if (previous === undefined || parsed.version > previous.version) { + selected.set(directory, { path, version: parsed.version }) + } + } + return [...selected.values()].map(entry => entry.path).sort() +} + +/** + * Require one persistence basename's generation to equal its Session header. + * + * @param name - Canonical persistence basename. + * @param content - Complete uncompressed UTF-8 JSONL content. + * @returns The validated generation. + */ +export function assertPersistedSessionVersion(name: string, content: string): number { + const persisted = parsePersistedSessionFilename(name) + if (persisted === undefined) throw new Error(`not a canonical Session persistence filename: ${name}`) + const headerVersion = sessionHeaderVersion(content, name) + if (headerVersion !== persisted.version) { + throw new Error( + `${name}: filename declares Session format v${persisted.version}, header declares v${headerVersion}`, + ) + } + return headerVersion +} diff --git a/packages/test-support/session-snapshot/src/suite.ts b/packages/test-support/session-snapshot/src/suite.ts index ad9c6e2ac3..200b47d788 100644 --- a/packages/test-support/session-snapshot/src/suite.ts +++ b/packages/test-support/session-snapshot/src/suite.ts @@ -17,7 +17,7 @@ * @module @deepseek-ai/dsh-session-snapshot/suite */ -import { readFile, readdir, rm, writeFile } from 'node:fs/promises' +import { readFile, readdir, writeFile } from 'node:fs/promises' import { existsSync } from 'node:fs' import { join } from 'node:path' import { isSurfaceEligibleType } from '@deepseek-ai/dsh-session/surface' @@ -26,6 +26,12 @@ import { type AgentUnderTest, type HarvestedLog, type InputScript, runScenario } import { parseSnapshotManifest } from './manifest.ts' import { redactSessionSnapshotIds } from './identity.ts' import { captureExpectedWorkspaceSnapshot } from './workspace.ts' +import { + assertSessionFixtureVersion, + sessionFixtureName, + sessionFixtureNames, + sessionHeaderVersion, +} from './session-files.ts' import { type CwdPathMode, type NormalizeContext, @@ -76,14 +82,14 @@ export interface Scenario { hasModelTurn: boolean /** * Whether the run persists a comparable session log to diff against the - * `session.jsonl` fixture. Defaults to {@link hasModelTurn} (a model turn + * selected parent Session fixture. Defaults to {@link hasModelTurn} (a model turn * always produces a log worth comparing). Set it independently for a scenario * that produces a non-trivial durable log without calling the model. */ comparesLog?: boolean /** - * Whether `test:snapshot:record` regenerates this scenario's `session.jsonl` - * from the LIVE API. `recorded` scenarios are model-driven and reproducible; + * Whether `test:snapshot:record` regenerates this scenario's current-version + * Session fixtures from the LIVE API. `recorded` scenarios are model-driven and reproducible; * `authored` scenarios (fixtures hand-written or hand-harvested — e.g. a * provider error or a cancel the live API can't be coaxed into * deterministically, a deterministic hook scenario, or a scripted repetition @@ -93,7 +99,7 @@ export interface Scenario { /** * Whether replay is driven by a hand-written `replay.override.json` sidecar * (a `ReplayOverrideDoc` that replaces or patches the script derived from - * `session.jsonl`) — the throw/hang cases chunks cannot express. The fixture + * selected parent Session fixture) — the throw/hang cases chunks cannot express. The fixture * guard requires the sidecar exactly when this is set: the harness forwards * the file purely on existence, so an unregistered stray sidecar would * silently alter the derived script. The guard fails loud on either @@ -324,41 +330,14 @@ export function assertUniqueSnapshotContents( } } -/** - * Validate and order a scenario directory's session-fixture filenames. - * - * The primary fixture is always `session.jsonl`; child sessions are discovered - * from contiguous `session.1.jsonl` … filenames. The directory is the source of - * truth, so scenario tables do not duplicate a child count that can drift from - * the files. A session-like JSONL with any other suffix fails loud. - * - * @param names File names in one scenario directory. - * @returns The primary and child fixture names in replay/harvest order. - */ -export function sessionFixtureNames(names: readonly string[]): string[] { - if (!names.includes('session.jsonl')) throw new Error('missing session.jsonl') - const children: { name: string; index: number }[] = [] - for (const name of names) { - if (name === 'session.jsonl') continue - if (!name.startsWith('session.') || !name.endsWith('.jsonl')) continue - const match = /^session\.([1-9]\d*)\.jsonl$/.exec(name) - if (match === null) throw new Error(`invalid child session fixture name: ${name}`) - children.push({ name, index: Number(match[1]) }) - } - children.sort((a, b) => a.index - b.index) - for (const [offset, child] of children.entries()) { - const expected = offset + 1 - if (child.index !== expected) { - throw new Error(`child session fixtures must be contiguous: expected session.${expected}.jsonl, found ${child.name}`) - } - } - return ['session.jsonl', ...children.map(child => child.name)] -} - /** Read one scenario directory's validated session-fixture inventory. */ async function sessionFixtures(dir: string): Promise { const entries = await readdir(dir, { withFileTypes: true }) - return sessionFixtureNames(entries.filter(entry => entry.isFile()).map(entry => entry.name)) + const names = sessionFixtureNames(entries.filter(entry => entry.isFile()).map(entry => entry.name)) + await Promise.all(names.map(async (name) => { + assertSessionFixtureVersion(name, await readFile(join(dir, name), 'utf8')) + })) + return names } /** @@ -366,7 +345,7 @@ async function sessionFixtures(dir: string): Promise { * from the live replay run; the non-empty sentinel for missing cwd avoids accidental empty- * string replacement. * - * @param fixture The committed `session.jsonl` content. + * @param fixture The selected committed parent Session fixture content. * @returns The fixture's own volatile values, ready for {@link normalizeSessionLog}. */ export function fixtureContext(fixture: string): NormalizeContext { @@ -1218,14 +1197,15 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void { // Replay/refresh need the committed inventory up front because those // files drive the model scripts. Record mode creates that inventory // from the harvested live logs, so it must also work for a brand-new - // scenario with no session.jsonl yet. + // scenario with no Session fixture yet. let fixtureFiles = RECORDING ? [] : await sessionFixtures(dir) const childFixtureFiles = fixtureFiles.slice(1) + const primaryFixtureFile = fixtureFiles[0] ?? sessionFixtureName(0, 0) const comparesLog = scenario.comparesLog ?? scenario.hasModelTurn const result = await runScenario(input, { agent, mode: childMode, - fixtureFile: join(dir, 'session.jsonl'), + fixtureFile: join(dir, primaryFixtureFile), ...scenario.env !== undefined ? { env: scenario.env } : {}, ...existsSync(overrideFile) ? { overrideFile } : {}, // In REPLAY, forward the recorded child fixtures so each subagent session @@ -1272,13 +1252,14 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void { expect(result.sessionLogs.length, `expected ${fixtureFiles.length} session logs (parent + children)`) .toBe(fixtureFiles.length) } - const outputFixtureFiles = [ - 'session.jsonl', - ...Array.from({ length: result.sessionLogs.length - 1 }, (_, i) => `session.${i + 1}.jsonl`), - ] - const existingFixtures = await Promise.all(outputFixtureFiles.map(async (file) => { - const path = join(dir, file) - return existsSync(path) ? readFile(path, 'utf8') : '' + const outputFixtureFiles = result.sessionLogs.map((log, index) => sessionFixtureName( + index, + sessionHeaderVersion(log.content, `harvested Session ${index}`), + )) + const existingFixtures = await Promise.all(outputFixtureFiles.map(async (_file, index) => { + const file = fixtureFiles[index] + if (file === undefined) return '' + return readFile(join(dir, file), 'utf8') })) const refreshReplacements = REFRESHING ? refreshFixtureReplacements(result.sessionLogs, existingFixtures) @@ -1294,19 +1275,7 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void { const outputFixtures = redactSessionSnapshotIds(stabilizeFixtureMessageIds(freshFixtures, existingFixtures)) await Promise.all(outputFixtures.map((fixture, index) => writeFile(join(dir, outputFixtureFiles[index] as string), fixture))) - if (RECORDING) { - const outputNames = new Set(outputFixtureFiles) - const entries = await readdir(dir, { withFileTypes: true }) - await Promise.all(entries - .filter(entry => entry.isFile() - // Only valid numbered children are record-owned stale output. - // Malformed session-like names stay for the inventory guard to - // reject instead of being silently deleted during mutation. - && /^session\.[1-9]\d*\.jsonl$/.test(entry.name) - && !outputNames.has(entry.name)) - .map(entry => rm(join(dir, entry.name)))) - fixtureFiles = outputFixtureFiles - } + fixtureFiles = outputFixtureFiles if (scenario.pinsHeader === true) { const primary = result.sessionLogs[0] as HarvestedLog const pinningHeaders = pinningHeaderPayloads(primary.content, ctx) @@ -1386,7 +1355,9 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void { cwd: (fixtureContexts[0] as NormalizeContext).cwd, } const actualSnapshots = normalizeSessionSnapshots(harvested, ctx) - const expectedSnapshots = normalizeSessionSnapshots(fixtures, fixtureCtx) + const expectedSnapshots = normalizeSessionSnapshots(fixtures, fixtureCtx, { + sourcePaths: fixtureFiles.map(file => join(dir, file)), + }) for (const [index, actual] of actualSnapshots.entries()) { expect(actual, `${fixtureFiles[index]} mismatch`).toEqual(expectedSnapshots[index]) } @@ -1401,7 +1372,8 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void { /* v8 ignore next -- registration guarantees every scenario class has resolved sources. */ const schemaSource = schemaSourceByClass.get(classOf(scenario)) ?? pinningScenario const pinningDir = join(snapshotsDir, pinningScenario.name) - const pinnedFixture = await readFile(join(pinningDir, 'session.jsonl'), 'utf8') + const [pinningFixtureFile] = await sessionFixtures(pinningDir) + const pinnedFixture = await readFile(join(pinningDir, pinningFixtureFile as string), 'utf8') const pinned = pinningHeaderPayloads(pinnedFixture, fixtureContext(pinnedFixture)) const promptSnapshot = await readFile( join(snapshotsDir, promptSource.name, SYSTEM_PROMPT_SNAPSHOT), @@ -1539,7 +1511,6 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void { existsSync(join(dir, WINDOWS_STDOUT_SNAPSHOT)), `${name}/${WINDOWS_STDOUT_SNAPSHOT} presence must match \`pinsNativeWindowsStdout\``, ).toBe(pinsNativeWindowsStdout === true) - expect(existsSync(join(dir, 'session.jsonl')), `${name}/session.jsonl`).toBe(true) expect(existsSync(join(dir, 'replay.override.json')), `${name}/replay.override.json presence must match \`overridden\``) .toBe(overridden === true) expect(existsSync(join(dir, SYSTEM_PROMPT_SNAPSHOT)), `${name}/${SYSTEM_PROMPT_SNAPSHOT} presence must match snapshot-source ownership`) @@ -1573,7 +1544,9 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void { const promptSource = promptSourceByClass.get(classOf(scenario)) ?? scenario /* v8 ignore next -- registration guarantees every pin has resolved sources. */ const schemaSource = schemaSourceByClass.get(classOf(scenario)) ?? scenario - const fixture = await readFile(join(snapshotsDir, scenario.name, 'session.jsonl'), 'utf8') + const fixtureDir = join(snapshotsDir, scenario.name) + const [fixtureFile] = await sessionFixtures(fixtureDir) + const fixture = await readFile(join(fixtureDir, fixtureFile as string), 'utf8') const headers = pinningHeaderPayloads(fixture, fixtureContext(fixture)) const promptSnapshot = await readFile( join(snapshotsDir, promptSource.name, SYSTEM_PROMPT_SNAPSHOT), diff --git a/packages/test-support/session-snapshot/tests/fixtures/record-suite/rec-child/behavior.json b/packages/test-support/session-snapshot/tests/fixtures/record-suite/rec-child/behavior.json index 971006c139..973fc376e4 100644 --- a/packages/test-support/session-snapshot/tests/fixtures/record-suite/rec-child/behavior.json +++ b/packages/test-support/session-snapshot/tests/fixtures/record-suite/rec-child/behavior.json @@ -1,14 +1,14 @@ { "prompt": "respond", "logs": [ - { "file": "b/parent/session.jsonl", "lines": [ - { "type": "session", "id": "{{SID}}", "createdAt": 700, "cwd": "{{CWD}}", "delegationDepth": 0 }, - { "type": "request/header", "seq": 0, "time": 3, "data": { "header": { "config": { "model": "fake" }, "system": "SYS PROMPT", "tools": [{ "name": "t1", "description": "D1", "parameters": { "type": "object" } }] }, "reason": "initial" } }, + { "file": "b/parent/session.v1.jsonl", "lines": [ + { "type": "session", "version": 1, "id": "{{SID}}", "createdAt": 700, "cwd": "{{CWD}}", "delegationDepth": 0 }, + { "type": "request/header", "seq": 0, "time": 3, "data": { "header": { "config": { "provider": "fake", "model": "fake" }, "system": "SYS PROMPT", "tools": [{ "name": "t1", "description": "D1", "parameters": { "type": "object" } }] }, "reason": "initial" } }, { "type": "user/message", "seq": 1, "time": 5, "data": { "role": "user", "content": [{ "type": "text", "text": "same inherited message" }], "source": { "kind": "user" }, "id": "11111111-1111-4111-8111-111111111111" }, "surfaceOp": "append" } ]}, - { "file": "b/child/session.jsonl", "lines": [ - { "type": "session", "id": "abababab-cdcd-4efe-8ada-badabadabada", "createdAt": 800, "cwd": "{{CWD}}", "parentSession": "{{SID}}", "delegationDepth": 1 }, - { "type": "request/header", "seq": 0, "time": 2, "data": { "header": { "config": { "model": "fake" }, "system": "SYS PROMPT", "tools": [{ "name": "t1", "description": "D1", "parameters": { "type": "object" } }] }, "reason": "initial" } }, + { "file": "b/child/session.v1.jsonl", "lines": [ + { "type": "session", "version": 1, "id": "abababab-cdcd-4efe-8ada-badabadabada", "createdAt": 800, "cwd": "{{CWD}}", "parentSession": "{{SID}}", "delegationDepth": 1 }, + { "type": "request/header", "seq": 0, "time": 2, "data": { "header": { "config": { "provider": "fake", "model": "fake" }, "system": "SYS PROMPT", "tools": [{ "name": "t1", "description": "D1", "parameters": { "type": "object" } }] }, "reason": "initial" } }, { "type": "user/message", "seq": 1, "time": 5, "data": { "role": "user", "content": [{ "type": "text", "text": "same inherited message" }], "source": { "kind": "user" }, "id": "11111111-1111-4111-8111-111111111111" }, "surfaceOp": "append" } ]} ] diff --git a/packages/test-support/session-snapshot/tests/fixtures/record-suite/rec-child/session.1.jsonl b/packages/test-support/session-snapshot/tests/fixtures/record-suite/rec-child/session.1.jsonl index 54a88f6eaf..b79fe3feed 100644 --- a/packages/test-support/session-snapshot/tests/fixtures/record-suite/rec-child/session.1.jsonl +++ b/packages/test-support/session-snapshot/tests/fixtures/record-suite/rec-child/session.1.jsonl @@ -1,3 +1,3 @@ {"type":"session","id":"{{session:2}}","createdAt":800,"cwd":"/var/folders/2g/b32ct0qn1d728l_v6tdkjytr0000gn/T/acp-snap-cwd-KBQJbW","parentSession":"{{session:1}}","delegationDepth":1} -{"type":"request/header","data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}} +{"type":"request/header","data":{"header":{"config":{"provider":"fake","model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}} {"type":"user/message","data":{"role":"user","content":[{"type":"text","text":"same inherited message"}],"source":{"kind":"user"},"id":"{{message:1}}"},"surfaceOp":"append"} diff --git a/packages/test-support/session-snapshot/tests/fixtures/record-suite/rec-child/session.jsonl b/packages/test-support/session-snapshot/tests/fixtures/record-suite/rec-child/session.jsonl index bebae09a98..0686ab430f 100644 --- a/packages/test-support/session-snapshot/tests/fixtures/record-suite/rec-child/session.jsonl +++ b/packages/test-support/session-snapshot/tests/fixtures/record-suite/rec-child/session.jsonl @@ -1,3 +1,3 @@ {"type":"session","id":"{{session:1}}","createdAt":700,"cwd":"/var/folders/2g/b32ct0qn1d728l_v6tdkjytr0000gn/T/acp-snap-cwd-KBQJbW","delegationDepth":0} -{"type":"request/header","data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}} +{"type":"request/header","data":{"header":{"config":{"provider":"fake","model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}} {"type":"user/message","data":{"role":"user","content":[{"type":"text","text":"same inherited message"}],"source":{"kind":"user"},"id":"{{message:1}}"},"surfaceOp":"append"} diff --git a/packages/test-support/session-snapshot/tests/fixtures/record-suite/rec-pin/behavior.json b/packages/test-support/session-snapshot/tests/fixtures/record-suite/rec-pin/behavior.json index 7fffecf747..def950fd0a 100644 --- a/packages/test-support/session-snapshot/tests/fixtures/record-suite/rec-pin/behavior.json +++ b/packages/test-support/session-snapshot/tests/fixtures/record-suite/rec-pin/behavior.json @@ -1,10 +1,10 @@ { "prompt": "respond", "logs": [{ - "file": "b/main/session.jsonl", + "file": "b/main/session.v1.jsonl", "lines": [ - { "type": "session", "id": "{{SID}}", "createdAt": 600, "cwd": "{{CWD}}", "delegationDepth": 0 }, - { "type": "request/header", "seq": 0, "time": 4, "data": { "header": { "config": { "model": "fake" }, "system": "SYS PROMPT", "tools": [{ "name": "t1", "description": "D1", "parameters": { "type": "object" } }] }, "reason": "initial" } } + { "type": "session", "version": 1, "id": "{{SID}}", "createdAt": 600, "cwd": "{{CWD}}", "delegationDepth": 0 }, + { "type": "request/header", "seq": 0, "time": 4, "data": { "header": { "config": { "provider": "fake", "model": "fake" }, "system": "SYS PROMPT", "tools": [{ "name": "t1", "description": "D1", "parameters": { "type": "object" } }] }, "reason": "initial" } } ] }] } diff --git a/packages/test-support/session-snapshot/tests/fixtures/record-suite/rec-pin/session.1.jsonl b/packages/test-support/session-snapshot/tests/fixtures/record-suite/rec-pin/session.1.jsonl index 233b92c601..1af52327d9 100644 --- a/packages/test-support/session-snapshot/tests/fixtures/record-suite/rec-pin/session.1.jsonl +++ b/packages/test-support/session-snapshot/tests/fixtures/record-suite/rec-pin/session.1.jsonl @@ -1,2 +1,2 @@ {"type":"session","id":"{{session:2}}","createdAt":800,"cwd":"/var/folders/2g/b32ct0qn1d728l_v6tdkjytr0000gn/T/acp-snap-cwd-KBQJbW","parentSession":"f6fa7fcf-dd9c-4b39-8815-b25ddcebfd88"} -{"type":"request/header","data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}} +{"type":"request/header","data":{"header":{"config":{"provider":"fake","model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}} diff --git a/packages/test-support/session-snapshot/tests/fixtures/record-suite/rec-pin/session.jsonl b/packages/test-support/session-snapshot/tests/fixtures/record-suite/rec-pin/session.jsonl index c4287520a3..80582f22a3 100644 --- a/packages/test-support/session-snapshot/tests/fixtures/record-suite/rec-pin/session.jsonl +++ b/packages/test-support/session-snapshot/tests/fixtures/record-suite/rec-pin/session.jsonl @@ -1,2 +1,2 @@ {"type":"session","id":"{{session:1}}","createdAt":600,"cwd":"/var/folders/2g/b32ct0qn1d728l_v6tdkjytr0000gn/T/acp-snap-cwd-nOQ4Gy","delegationDepth":0} -{"type":"request/header","data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}} +{"type":"request/header","data":{"header":{"config":{"provider":"fake","model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}} diff --git a/packages/test-support/session-snapshot/tests/fixtures/suite/authored-error/behavior.json b/packages/test-support/session-snapshot/tests/fixtures/suite/authored-error/behavior.json index fd843a3a08..77777a0e5b 100644 --- a/packages/test-support/session-snapshot/tests/fixtures/suite/authored-error/behavior.json +++ b/packages/test-support/session-snapshot/tests/fixtures/suite/authored-error/behavior.json @@ -1,10 +1,10 @@ { "prompt": "error", "logs": [{ - "file": "b/main/session.jsonl", + "file": "b/main/session.v1.jsonl", "lines": [ - { "type": "session", "id": "{{SID}}", "createdAt": 500, "cwd": "{{CWD}}", "delegationDepth": 0 }, - { "type": "turn/end", "seq": 1, "time": 9, "data": { "error": "model exploded" } } + { "type": "session", "version": 1, "id": "{{SID}}", "createdAt": 500, "cwd": "{{CWD}}", "delegationDepth": 0 }, + { "type": "turn/end", "seq": 0, "time": 9, "data": { "error": "model exploded" } } ] }] } diff --git a/packages/test-support/session-snapshot/tests/fixtures/suite/blocked-log/behavior.json b/packages/test-support/session-snapshot/tests/fixtures/suite/blocked-log/behavior.json index 3c8ffc0b86..e5dba35c32 100644 --- a/packages/test-support/session-snapshot/tests/fixtures/suite/blocked-log/behavior.json +++ b/packages/test-support/session-snapshot/tests/fixtures/suite/blocked-log/behavior.json @@ -1,10 +1,10 @@ { "prompt": "error", "logs": [{ - "file": "b/main/session.jsonl", + "file": "b/main/session.v1.jsonl", "lines": [ - { "type": "session", "id": "{{SID}}", "createdAt": 400, "cwd": "{{CWD}}", "delegationDepth": 0 }, - { "type": "hook/result", "seq": 1, "time": 8, "data": { "decision": "block", "durationMs": 37 } } + { "type": "session", "version": 1, "id": "{{SID}}", "createdAt": 400, "cwd": "{{CWD}}", "delegationDepth": 0 }, + { "type": "hook/result", "seq": 0, "time": 8, "data": { "decision": "block", "durationMs": 37 } } ] }] } diff --git a/packages/test-support/session-snapshot/tests/fixtures/suite/pin-turn/behavior.json b/packages/test-support/session-snapshot/tests/fixtures/suite/pin-turn/behavior.json index bd19a95893..1c00508168 100644 --- a/packages/test-support/session-snapshot/tests/fixtures/suite/pin-turn/behavior.json +++ b/packages/test-support/session-snapshot/tests/fixtures/suite/pin-turn/behavior.json @@ -1,12 +1,12 @@ { "prompt": "respond", "logs": [{ - "file": "b/main/session.jsonl", + "file": "b/main/session.v1.jsonl", "lines": [ - { "type": "session", "id": "{{SID}}", "createdAt": 100, "cwd": "{{CWD}}", "delegationDepth": 0 }, - { "type": "request/header", "seq": 0, "time": 100, "data": { "header": { "config": { "model": "fake" }, "system": "SYS PROMPT", "tools": [{ "name": "t1", "description": "D1", "parameters": { "type": "object" } }] }, "reason": "initial" } }, - { "type": "request/header", "seq": 1, "time": 100, "data": { "header": { "config": { "model": "fake" }, "system": "SYS PROMPT\n\nNEW PROMPT LINE", "tools": [{ "name": "t1", "description": "D1", "parameters": { "type": "object" } }] }, "reason": "change" } }, - { "type": "request/header", "seq": 2, "time": 100, "data": { "header": { "config": { "model": "fake" }, "system": "SYS PROMPT\n\nNEW PROMPT LINE", "tools": [{ "name": "t1", "description": "D1", "parameters": { "type": "object" } }] }, "reason": "series" } }, + { "type": "session", "version": 1, "id": "{{SID}}", "createdAt": 100, "cwd": "{{CWD}}", "delegationDepth": 0 }, + { "type": "request/header", "seq": 0, "time": 100, "data": { "header": { "config": { "provider": "fake", "model": "fake" }, "system": "SYS PROMPT", "tools": [{ "name": "t1", "description": "D1", "parameters": { "type": "object" } }] }, "reason": "initial" } }, + { "type": "request/header", "seq": 1, "time": 100, "data": { "header": { "config": { "provider": "fake", "model": "fake" }, "system": "SYS PROMPT\n\nNEW PROMPT LINE", "tools": [{ "name": "t1", "description": "D1", "parameters": { "type": "object" } }] }, "reason": "change" } }, + { "type": "request/header", "seq": 2, "time": 100, "data": { "header": { "config": { "provider": "fake", "model": "fake" }, "system": "SYS PROMPT\n\nNEW PROMPT LINE", "tools": [{ "name": "t1", "description": "D1", "parameters": { "type": "object" } }] }, "reason": "series" } }, { "type": "turn/start", "seq": 3, "time": 100, "data": { "turn": 1 } } ] }] diff --git a/packages/test-support/session-snapshot/tests/fixtures/suite/pin-turn/session.jsonl b/packages/test-support/session-snapshot/tests/fixtures/suite/pin-turn/session.jsonl index 467616c82b..eb487a9784 100644 --- a/packages/test-support/session-snapshot/tests/fixtures/suite/pin-turn/session.jsonl +++ b/packages/test-support/session-snapshot/tests/fixtures/suite/pin-turn/session.jsonl @@ -1,5 +1,5 @@ {"type":"session","id":"{{session:1}}","createdAt":7,"cwd":"/rec/pin-cwd","delegationDepth":0} -{"type":"request/header","data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}} -{"type":"request/header","data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"change"}} -{"type":"request/header","data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"series"}} +{"type":"request/header","data":{"header":{"config":{"provider":"fake","model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}} +{"type":"request/header","data":{"header":{"config":{"provider":"fake","model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"change"}} +{"type":"request/header","data":{"header":{"config":{"provider":"fake","model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"series"}} {"type":"turn/start","data":{"turn":1}} diff --git a/packages/test-support/session-snapshot/tests/fixtures/suite/plain-turn/behavior.json b/packages/test-support/session-snapshot/tests/fixtures/suite/plain-turn/behavior.json index b4e3667519..620857e4b5 100644 --- a/packages/test-support/session-snapshot/tests/fixtures/suite/plain-turn/behavior.json +++ b/packages/test-support/session-snapshot/tests/fixtures/suite/plain-turn/behavior.json @@ -2,14 +2,14 @@ "prompt": "respond", "echoWorkspace": true, "logs": [ - { "file": "b/parent/session.jsonl", "lines": [ - { "type": "session", "id": "{{SID}}", "createdAt": 200, "cwd": "{{CWD}}", "delegationDepth": 0 }, - { "type": "request/header", "seq": 0, "time": 5, "data": { "header": { "config": { "model": "fake" }, "system": "SYS PROMPT", "tools": [{ "name": "t1", "description": "D1", "parameters": { "type": "object" } }] }, "reason": "initial" } }, + { "file": "b/parent/session.v1.jsonl", "lines": [ + { "type": "session", "version": 1, "id": "{{SID}}", "createdAt": 200, "cwd": "{{CWD}}", "delegationDepth": 0 }, + { "type": "request/header", "seq": 0, "time": 5, "data": { "header": { "config": { "provider": "fake", "model": "fake" }, "system": "SYS PROMPT", "tools": [{ "name": "t1", "description": "D1", "parameters": { "type": "object" } }] }, "reason": "initial" } }, { "type": "assistant/chunk", "seq": 1, "time": 5, "data": { "turn": 1, "step": 1, "chunk": { "type": "text-delta", "index": 0, "text": "hi" } } } ]}, - { "file": "b/child/session.jsonl", "lines": [ - { "type": "session", "id": "eeeeeeee-1111-4222-8333-444444444444", "createdAt": 300, "cwd": "{{CWD}}", "parentSession": "{{SID}}", "delegationDepth": 1 }, - { "type": "request/header", "seq": 0, "time": 6, "data": { "header": { "config": { "model": "fake" }, "system": "SYS PROMPT\n\nCHILD GUIDANCE", "tools": [{ "name": "child-only", "description": "Child D", "parameters": { "type": "object" } }] }, "reason": "initial" } } + { "file": "b/child/session.v1.jsonl", "lines": [ + { "type": "session", "version": 1, "id": "eeeeeeee-1111-4222-8333-444444444444", "createdAt": 300, "cwd": "{{CWD}}", "parentSession": "{{SID}}", "delegationDepth": 1 }, + { "type": "request/header", "seq": 0, "time": 6, "data": { "header": { "config": { "provider": "fake", "model": "fake" }, "system": "SYS PROMPT\n\nCHILD GUIDANCE", "tools": [{ "name": "child-only", "description": "Child D", "parameters": { "type": "object" } }] }, "reason": "initial" } } ]} ] } diff --git a/packages/test-support/session-snapshot/tests/fixtures/suite/plain-turn/session.1.jsonl b/packages/test-support/session-snapshot/tests/fixtures/suite/plain-turn/session.1.jsonl index e61b2e1737..79ebf38804 100644 --- a/packages/test-support/session-snapshot/tests/fixtures/suite/plain-turn/session.1.jsonl +++ b/packages/test-support/session-snapshot/tests/fixtures/suite/plain-turn/session.1.jsonl @@ -1,2 +1,2 @@ {"type":"session","id":"{{session:2}}","createdAt":12,"cwd":"/rec/plain-cwd","parentSession":"{{session:1}}","delegationDepth":1} -{"type":"request/header","data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}} +{"type":"request/header","data":{"header":{"config":{"provider":"fake","model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}} diff --git a/packages/test-support/session-snapshot/tests/fixtures/suite/plain-turn/session.jsonl b/packages/test-support/session-snapshot/tests/fixtures/suite/plain-turn/session.jsonl index 0cd166be9e..a4eda6ca1e 100644 --- a/packages/test-support/session-snapshot/tests/fixtures/suite/plain-turn/session.jsonl +++ b/packages/test-support/session-snapshot/tests/fixtures/suite/plain-turn/session.jsonl @@ -1,3 +1,3 @@ {"type":"session","id":"{{session:1}}","createdAt":11,"cwd":"/rec/plain-cwd","delegationDepth":0} -{"type":"request/header","data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}} +{"type":"request/header","data":{"header":{"config":{"provider":"fake","model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}} {"type":"assistant/chunk","data":{"turn":1,"step":1,"chunk":{"type":"text-delta","index":0,"text":"hi"}}} diff --git a/packages/test-support/session-snapshot/tests/fixtures/suite/shared-pin/behavior.json b/packages/test-support/session-snapshot/tests/fixtures/suite/shared-pin/behavior.json index 4de8f25b7e..5e9b93ddee 100644 --- a/packages/test-support/session-snapshot/tests/fixtures/suite/shared-pin/behavior.json +++ b/packages/test-support/session-snapshot/tests/fixtures/suite/shared-pin/behavior.json @@ -1,11 +1,11 @@ { "prompt": "respond", "logs": [{ - "file": "b/main/session.jsonl", + "file": "b/main/session.v1.jsonl", "lines": [ - { "type": "session", "id": "{{SID}}", "createdAt": 100, "cwd": "{{CWD}}", "delegationDepth": 0 }, - { "type": "request/header", "seq": 0, "time": 100, "data": { "header": { "config": { "model": "fake" }, "system": "SYS PROMPT", "tools": [{ "name": "t1", "description": "D1", "parameters": { "type": "object" } }] }, "reason": "initial" } }, - { "type": "request/header", "seq": 1, "time": 100, "data": { "header": { "config": { "model": "fake" }, "system": "SYS PROMPT\n\nNEW PROMPT LINE", "tools": [{ "name": "t1", "description": "D1", "parameters": { "type": "object" } }] }, "reason": "change" } }, + { "type": "session", "version": 1, "id": "{{SID}}", "createdAt": 100, "cwd": "{{CWD}}", "delegationDepth": 0 }, + { "type": "request/header", "seq": 0, "time": 100, "data": { "header": { "config": { "provider": "fake", "model": "fake" }, "system": "SYS PROMPT", "tools": [{ "name": "t1", "description": "D1", "parameters": { "type": "object" } }] }, "reason": "initial" } }, + { "type": "request/header", "seq": 1, "time": 100, "data": { "header": { "config": { "provider": "fake", "model": "fake" }, "system": "SYS PROMPT\n\nNEW PROMPT LINE", "tools": [{ "name": "t1", "description": "D1", "parameters": { "type": "object" } }] }, "reason": "change" } }, { "type": "turn/start", "seq": 2, "time": 100, "data": { "turn": 1 } } ] }] diff --git a/packages/test-support/session-snapshot/tests/fixtures/suite/shared-pin/session.jsonl b/packages/test-support/session-snapshot/tests/fixtures/suite/shared-pin/session.jsonl index 30fe9ff3f3..f281ee72ae 100644 --- a/packages/test-support/session-snapshot/tests/fixtures/suite/shared-pin/session.jsonl +++ b/packages/test-support/session-snapshot/tests/fixtures/suite/shared-pin/session.jsonl @@ -1,4 +1,4 @@ {"type":"session","id":"{{session:1}}","createdAt":7,"cwd":"/rec/shared-pin-cwd","delegationDepth":0} -{"type":"request/header","data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}} -{"type":"request/header","data":{"header":{"config":{"model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"change"}} +{"type":"request/header","data":{"header":{"config":{"provider":"fake","model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}} +{"type":"request/header","data":{"header":{"config":{"provider":"fake","model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"change"}} {"type":"turn/start","data":{"turn":1}} diff --git a/packages/test-support/session-snapshot/tests/harness.spec.ts b/packages/test-support/session-snapshot/tests/harness.spec.ts index 4c88111de5..889ccefb67 100644 --- a/packages/test-support/session-snapshot/tests/harness.spec.ts +++ b/packages/test-support/session-snapshot/tests/harness.spec.ts @@ -495,7 +495,7 @@ describe('runScenario', () => { logs: [{ file: 'project/main/session.jsonl', lines: [ - { type: 'session', id: '{{SID}}', createdAt: 42, cwd: '{{CWD}}' }, + { type: 'session', version: 0, id: '{{SID}}', createdAt: 42, cwd: '{{CWD}}' }, { type: 'turn/start', seq: 1, time: 9, data: { turn: 1 } }, ], }], @@ -1264,11 +1264,12 @@ describe('runScenario', () => { // File names chosen so readdir feeds the sort children-first AND // parent-in-the-middle: the comparator then sees a parent on both // sides of a pair, plus the same-createdAt (localeCompare) tiebreak. - { file: 'b1/aa-child-c/session.jsonl', lines: [{ type: 'session', id: 'cccccccc-0000-4000-8000-000000000000', createdAt: 500, parentSession: '{{SID}}' }] }, - { file: 'b1/bb-parent/session.jsonl', lines: [{ type: 'session', id: '{{SID}}', createdAt: 900 }] }, - { file: 'b1/cc-child-a/session.jsonl', lines: [{ type: 'session', id: 'aaaaaaaa-0000-4000-8000-000000000000', createdAt: 500, parentSession: '{{SID}}' }] }, + { file: 'b1/aa-child-c/session.jsonl', lines: [{ type: 'session', version: 0, id: 'cccccccc-0000-4000-8000-000000000000', createdAt: 500, parentSession: '{{SID}}' }] }, + { file: 'b1/bb-parent/session.jsonl', lines: [{ type: 'session', version: 0, id: 'superseded-parent', createdAt: 1 }] }, + { file: 'b1/bb-parent/session.v1.jsonl', lines: [{ type: 'session', version: 1, id: '{{SID}}', createdAt: 900 }] }, + { file: 'b1/cc-child-a/session.jsonl', lines: [{ type: 'session', version: 0, id: 'aaaaaaaa-0000-4000-8000-000000000000', createdAt: 500, parentSession: '{{SID}}' }] }, // Missing id/createdAt fall back to ''/0; earliest child by createdAt. - { file: 'b2/orphan/session.jsonl', lines: [{ type: 'session', parentSession: '{{SID}}' }] }, + { file: 'b2/orphan/session.jsonl', lines: [{ type: 'session', version: 0, parentSession: '{{SID}}' }] }, ], }) const result = await runScenario( @@ -1284,13 +1285,22 @@ describe('runScenario', () => { expect(result.sessionLogs[1]?.parentSession).toBe(result.sessionId) }) - it('treats an empty log file as a header-less primary with default fields', { timeout: 20_000 }, async () => { + it('rejects an empty persisted generation', { timeout: 20_000 }, async () => { const { fixtureFile } = await scenario({ logs: [{ file: 'b/empty/session.jsonl', lines: [] }] }) - const result = await runScenario( + await expect(runScenario( { steps: boot }, { agent: AGENT, mode: 'replay', fixtureFile }, - ) - expect(result.sessionLogs.map(l => [l.id, l.createdAt, l.parentSession])).toEqual([['', 0, undefined]]) + )).rejects.toThrow('session.jsonl: session fixture is empty') + }) + + it('rejects a persisted filename/header generation mismatch', { timeout: 20_000 }, async () => { + const { fixtureFile } = await scenario({ + logs: [{ file: 'b/mismatch/session.v1.jsonl', lines: [{ type: 'session', version: 0 }] }], + }) + await expect(runScenario( + { steps: boot }, + { agent: AGENT, mode: 'replay', fixtureFile }, + )).rejects.toThrow('filename declares Session format v1, header declares v0') }) it('yields no logs when the sessions root vanished', { timeout: 20_000 }, async () => { diff --git a/packages/test-support/session-snapshot/tests/normalize.spec.ts b/packages/test-support/session-snapshot/tests/normalize.spec.ts index e033f56ee2..3dab384c20 100644 --- a/packages/test-support/session-snapshot/tests/normalize.spec.ts +++ b/packages/test-support/session-snapshot/tests/normalize.spec.ts @@ -1,8 +1,11 @@ +import { readFileSync } from 'node:fs' import { describe, expect, it } from 'vitest' +import { ALPHA_SESSION_FORMAT_REFUSAL_FIXTURES } from '@deepseek-ai/dsh-llm-replay' import { type NormalizeContext, extractSnapshotSpillPaths, normalizeSessionLog, + normalizeSessionFormatProvenance, normalizeSessionSnapshot, normalizeSessionSnapshots, normalizeStdout, @@ -527,7 +530,9 @@ describe('normalizeSessionSnapshot', () => { it('re-packs multi-session fixtures after relationship-preserving id redaction', () => { const raw = [ - JSON.stringify({ type: 'session', version: 0 }), + JSON.stringify({ type: 'session', version: 0, id: '{{session:1}}', createdAt: 0, delegationDepth: 0 }), + JSON.stringify({ type: 'turn/start', data: { turn: 1 } }), + JSON.stringify({ type: 'step/start', data: { turn: 1, step: 1 } }), JSON.stringify({ type: 'reasoning-chunks', data: { turn: 1, step: 1, index: 0, dt: [1, 2], texts: ['a', 'b', 'c'] }, @@ -538,7 +543,9 @@ describe('normalizeSessionSnapshot', () => { }), ].join('\n') + '\n' expect(normalizeSessionSnapshots([raw], ctx)).toEqual([[ - JSON.stringify({ type: 'session', version: 0 }), + JSON.stringify({ type: 'session', id: '{{session:1}}', createdAt: 0, delegationDepth: 0 }), + JSON.stringify({ type: 'turn/start', data: { turn: 1 } }), + JSON.stringify({ type: 'step/start', data: { turn: 1, step: 1 } }), JSON.stringify({ type: 'reasoning-chunks', data: { turn: 1, step: 1, index: 0, dt: [0, 0, 0, 0, 0], texts: ['a', 'b', 'c', 'd', 'e', 'f'] }, @@ -547,6 +554,223 @@ describe('normalizeSessionSnapshot', () => { ].join('\n')]) }) + it('rejects source paths that cannot identify every snapshot', () => { + const raw = `${JSON.stringify({ type: 'session', version: 1 })}\n` + + expect(() => normalizeSessionSnapshots( + [raw], + { sessionIds: [], cwd: '/unused' }, + { sourcePaths: [] }, + )).toThrow('Session snapshot source path count must match its log count') + }) + + it('normalizes an already-projected snapshot without a released-format field', () => { + const raw = `${JSON.stringify({ + type: 'session', + id: '11111111-2222-3333-4444-555555555555', + createdAt: 9, + })}\n` + + expect(normalizeSessionSnapshots([raw], { sessionIds: [], cwd: '/unused' })).toEqual([ + `${JSON.stringify({ type: 'session', id: '{{session:1}}', createdAt: 0 })}\n`, + ]) + }) + + it('rejects an empty snapshot before classifying its released format', () => { + expect(() => normalizeSessionSnapshots(['\n'], { sessionIds: [], cwd: '/unused' })) + .toThrow('session snapshot must start with a session header') + }) + + it('rejects a nonempty snapshot whose first record is not a session header', () => { + const raw = `${JSON.stringify({ type: 'turn/start', data: { turn: 1 } })}\n` + expect(() => normalizeSessionSnapshots([raw], { sessionIds: [], cwd: '/unused' })) + .toThrow('session snapshot must start with a session header') + }) + + it.each(ALPHA_SESSION_FORMAT_REFUSAL_FIXTURES)( + 'generation-normalizes the exact replay-only refusal $repoRelativePath without rewriting it', + (fixture) => { + const source = readFileSync(fixture.path, 'utf8') + + const [normalized] = normalizeSessionSnapshots( + [source], + { sessionIds: [], cwd: '/unused' }, + { sourcePaths: [fixture.path] }, + ) + + expect(JSON.parse(normalized?.split('\n')[0] as string)).not.toHaveProperty('version') + expect(readFileSync(fixture.path, 'utf8')).toBe(source) + }, + ) + + it('compares migrated and fresh delivery watermarks without changing their raw generation identity', () => { + const id = '11111111-2222-3333-4444-555555555555' + const session = (version: 0 | 1, sessionFormatVersion?: number): string => [ + JSON.stringify({ type: 'session', version, id, createdAt: 0, delegationDepth: 0 }), + JSON.stringify({ type: 'turn/start', data: { turn: 1 } }), + JSON.stringify({ + type: 'session-log-deepseek/delivery-accepted', + data: { + sessionId: id, + throughSeq: 0, + ...sessionFormatVersion === undefined ? {} : { sessionFormatVersion }, + }, + }), + JSON.stringify({ type: 'turn/end', data: { turn: 1, reason: { kind: 'completed' } } }), + '', + ].join('\n') + + const migratedV0 = normalizeSessionSnapshots([session(0)], { sessionIds: [], cwd: '/unused' }) + const freshV1 = normalizeSessionSnapshots([session(1, 1)], { sessionIds: [], cwd: '/unused' }) + + expect(migratedV0).toEqual(freshV1) + expect(freshV1[0]).not.toContain('"sessionFormatVersion"') + }) + + it('compares migrated and fresh session-reference captures without hiding other source fields', () => { + const id = '11111111-2222-3333-4444-555555555555' + const sourceId = '22222222-3333-4444-5555-666666666666' + const messageId = '33333333-4444-4555-8666-777777777777' + const session = (version: 0 | 1, capturedFormatVersion?: number): string => [ + JSON.stringify({ type: 'session', version, id, createdAt: 0, delegationDepth: 0 }), + JSON.stringify({ type: 'turn/start', data: { turn: 1 } }), + JSON.stringify({ + type: 'user/message', + data: { + id: messageId, + role: 'user', + content: [{ type: 'text', text: 'remember' }], + source: { + kind: 'session-reference', + form: 'recall', + version: 1, + references: [{ + sessionId: sourceId, + label: 'Source', + capturedThroughSeq: 0, + ...capturedFormatVersion === undefined ? {} : { capturedFormatVersion }, + compacted: false, + originalMessages: 1, + retainedMessages: 1, + omittedMessages: 0, + omittedBytes: 0, + truncated: false, + inputIndex: 0, + }], + }, + }, + surfaceOp: 'append', + }), + JSON.stringify({ type: 'turn/end', data: { turn: 1, reason: { kind: 'completed' } } }), + '', + ].join('\n') + + const migratedV0 = normalizeSessionSnapshots([session(0)], { sessionIds: [], cwd: '/unused' }) + const freshV1 = normalizeSessionSnapshots([session(1, 1)], { sessionIds: [], cwd: '/unused' }) + + expect(migratedV0).toEqual(freshV1) + expect(freshV1[0]).toContain('"label":"Source"') + expect(freshV1[0]).toContain('"version":1') + expect(freshV1[0]).not.toContain('"capturedFormatVersion"') + }) + + it('removes generation qualifiers only from their exact provenance positions', () => { + const raw = [ + JSON.stringify({ + type: 'session', + id: '11111111-2222-3333-4444-555555555555', + createdAt: 0, + }), + JSON.stringify({ + type: 'session-log-deepseek/delivery-accepted', + data: { sessionFormatVersion: 1, otherVersion: 8 }, + }), + JSON.stringify({ + type: 'user/message', + data: { + role: 'user', + content: [], + source: { + kind: 'session-reference', + form: 'recall', + version: 1, + references: [ + null, + 'opaque', + [{ capturedFormatVersion: 6 }], + { capturedFormatVersion: 1, otherVersion: 9 }, + ], + }, + }, + }), + JSON.stringify({ + type: 'assistant/message', + data: { + message: { + role: 'assistant', + content: [], + source: [{ capturedFormatVersion: 7 }], + }, + }, + }), + JSON.stringify({ + type: 'custom/event', + data: { capturedFormatVersion: 5, sessionFormatVersion: 4 }, + ignorable: true, + }), + '', + ].join('\n') + + const [normalized] = normalizeSessionSnapshots([raw], { sessionIds: [], cwd: '/unused' }) + const [, delivery, captured, sourceLookalike, opaqueEvent] = normalized + ?.trimEnd() + .split('\n') + .map(line => JSON.parse(line) as Record) ?? [] + + expect(delivery?.data).toEqual({ otherVersion: 8 }) + expect(captured?.data).toMatchObject({ + source: { + references: [ + null, + 'opaque', + [{ capturedFormatVersion: 6 }], + { otherVersion: 9 }, + ], + }, + }) + expect(sourceLookalike?.data).toEqual({ + message: { + role: 'assistant', + content: [], + source: [{ capturedFormatVersion: 7 }], + }, + }) + expect(opaqueEvent?.data).toEqual({ capturedFormatVersion: 5, sessionFormatVersion: 4 }) + }) + + it('keeps session-reference lookalikes outside Message source positions unchanged', () => { + const lookalike = [ + JSON.stringify({ type: 'session', version: 1, id: 's', createdAt: 0, delegationDepth: 0 }), + JSON.stringify({ + type: 'custom/event', + data: { + meta: { + kind: 'session-reference', + form: 'recall', + version: 1, + references: [{ capturedFormatVersion: 7 }], + }, + }, + ignorable: true, + }), + '', + ].join('\n') + + const normalized = normalizeSessionFormatProvenance(lookalike).split('\n') + expect(JSON.parse(normalized[0] as string)).not.toHaveProperty('version') + expect(normalized[1]).toBe(lookalike.split('\n')[1]) + }) + it('projects persisted provenance ranges back to logical seq arrays', () => { const raw = [ JSON.stringify({ type: 'session', version: 0 }), diff --git a/packages/test-support/session-snapshot/tests/suite.spec.ts b/packages/test-support/session-snapshot/tests/suite.spec.ts index bebf6c39d8..08450ef38e 100644 --- a/packages/test-support/session-snapshot/tests/suite.spec.ts +++ b/packages/test-support/session-snapshot/tests/suite.spec.ts @@ -5,7 +5,16 @@ import { join } from 'node:path' import { fileURLToPath } from 'node:url' import { afterAll, describe, expect, it } from 'vitest' import { + assertPersistedSessionVersion, + assertSessionFixtureVersion, defineAcpSnapshotSuite, + latestPersistedSessionPaths, + parsePersistedSessionFilename, + parseSessionFixtureName, + persistedSessionFilename, + sessionFixtureName, + sessionFixtureNames, + sessionHeaderVersion, stabilizeFixtureMessageIds, tokenizeSessionFixtureCwd, type HarvestedLog, @@ -25,7 +34,6 @@ import { parseToolSchemasSnapshot, refreshFixtureReplacements, scenarioSkipped, - sessionFixtureNames, restorePinnedToolSchemas, type SharedSnapshotClaim, stabilizeRefreshLog, @@ -108,12 +116,13 @@ const RECORD_SCENARIOS: Scenario[] = [ // committed record fixtures and expected outputs in place. const BOOTSTRAP = process.env.ACP_SNAPSHOT_SPEC_BOOTSTRAP === '1' const recordDir = BOOTSTRAP ? RECORD_SRC : mkdtempSync(join(tmpdir(), 'acp-snap-record-suite-')) +const retiredChildFixture = readFileSync(join(RECORD_SRC, 'rec-child', 'session.1.jsonl'), 'utf8') if (!BOOTSTRAP) { cpSync(RECORD_SRC, recordDir, { recursive: true }) // Record mode owns its output inventory: a new scenario has no primary yet, // while a changed child count can leave old numbered fixtures behind. rmSync(join(recordDir, 'rec-pin', 'session.jsonl')) - writeFileSync(join(recordDir, 'rec-child', 'session.2.jsonl'), 'stale child\n') + writeFileSync(join(recordDir, 'rec-child', 'session.2.jsonl'), retiredChildFixture) } const refreshDir = mkdtempSync(join(tmpdir(), 'acp-snap-refresh-suite-')) cpSync(REPLAY_DIR, refreshDir, { recursive: true }) @@ -137,12 +146,12 @@ function staleRefreshFixtures(dir: string): void { writeFileSync(join(dir, 'blocked-log', 'session.jsonl'), [ '{"type":"session","id":"99999999-8888-4777-8666-555555555555","createdAt":13,"cwd":"/rec/blocked-cwd","delegationDepth":0}', - '{"type":"hook/result","seq":1,"time":13,"data":{"decision":"stale","durationMs":99}}', + '{"type":"hook/result","seq":0,"time":13,"data":{"decision":"stale","durationMs":99}}', '', ].join('\n')) writeFileSync(join(dir, 'authored-error', 'session.jsonl'), [ '{"type":"session","id":"77777777-8888-4777-8666-555555555555","createdAt":13,"cwd":"/rec/error-cwd","delegationDepth":0}', - '{"type":"turn/end","seq":1,"time":9,"data":{"error":"stale"}}', + '{"type":"turn/end","seq":0,"time":9,"data":{"error":"stale"}}', '', ].join('\n')) } @@ -170,11 +179,11 @@ describe('defineAcpSnapshotSuite: refresh write-back', () => { // The scenario's own env layer reached the subprocess. expect(stdout).toContain('\\"permissionMode\\":\\"never\\"') - const blocked = readFileSync(join(refreshDir, 'blocked-log', 'session.jsonl'), 'utf8') + const blocked = readFileSync(join(refreshDir, 'blocked-log', 'session.v1.jsonl'), 'utf8') expect(blocked).toContain('"decision":"block"') expect(blocked).not.toContain('"decision":"stale"') - const authored = readFileSync(join(refreshDir, 'authored-error', 'session.jsonl'), 'utf8') + const authored = readFileSync(join(refreshDir, 'authored-error', 'session.v1.jsonl'), 'utf8') expect(authored).toContain('"error":"model exploded"') expect(authored).not.toContain('"error":"stale"') @@ -197,17 +206,21 @@ describe('defineAcpSnapshotSuite: refresh write-back', () => { const childPrompt = readFileSync(join(refreshDir, 'plain-turn', 'system-prompt.1.expected.md'), 'utf8') expect(childPrompt).toBe('SYS PROMPT\n\nCHILD GUIDANCE\n') - const pinSession = readFileSync(join(refreshDir, 'pin-turn', 'session.jsonl'), 'utf8') + const pinSession = readFileSync(join(refreshDir, 'pin-turn', 'session.v1.jsonl'), 'utf8') expect(pinSession).toContain('"cwd":"{{cwd}}"') + expect(readFileSync(join(refreshDir, 'pin-turn', 'session.jsonl'), 'utf8')) + .not.toContain('"version"') }) }) describe('defineAcpSnapshotSuite: record inventory write-back', () => { - it('creates a missing primary fixture and prunes stale child fixtures', () => { - const fixture = readFileSync(join(recordDir, 'rec-pin', 'session.jsonl'), 'utf8') + it('creates a missing primary fixture without deleting a retired child generation', () => { + const fixture = readFileSync(join(recordDir, 'rec-pin', 'session.v1.jsonl'), 'utf8') expect(fixture).toContain('"type":"session"') expect(fixture).toContain('"cwd":"{{cwd}}"') - expect(() => readFileSync(join(recordDir, 'rec-child', 'session.2.jsonl'), 'utf8')).toThrow() + if (!BOOTSTRAP) { + expect(readFileSync(join(recordDir, 'rec-child', 'session.2.jsonl'), 'utf8')).toBe(retiredChildFixture) + } expect(readFileSync(join(recordDir, 'rec-child', 'tool-schemas.1.expected.json'), 'utf8')) .toContain('"name": "t1"') }) @@ -215,7 +228,7 @@ describe('defineAcpSnapshotSuite: record inventory write-back', () => { it('retains an unchanged message relationship across the recorded parent and child fixtures', () => { const existingMessageId = '22222222-2222-4222-8222-222222222222' const freshMessageId = '11111111-1111-4111-8111-111111111111' - const fixtures = ['session.jsonl', 'session.1.jsonl'] + const fixtures = ['session.v1.jsonl', 'session.1.v1.jsonl'] .map(file => readFileSync(join(recordDir, 'rec-child', file), 'utf8')) for (const fixture of fixtures) { @@ -418,14 +431,17 @@ describe('shared snapshot content', () => { }) describe('sessionFixtureNames', () => { - it('orders the primary and contiguous child fixtures while ignoring other files', () => { + it('selects the highest generation for each contiguous role while ignoring other files', () => { expect(sessionFixtureNames([ 'stdout.expected.jsonl', 'session.2.jsonl', 'session.jsonl', 'session.1.jsonl', + 'session.v2.jsonl', + 'session.v1.jsonl', + 'session.1.v3.jsonl', 'input.json', - ])).toEqual(['session.jsonl', 'session.1.jsonl', 'session.2.jsonl']) + ])).toEqual(['session.v2.jsonl', 'session.1.v3.jsonl', 'session.2.jsonl']) }) it('accepts a primary-only scenario', () => { @@ -433,25 +449,109 @@ describe('sessionFixtureNames', () => { }) it('rejects a directory without the primary fixture', () => { - expect(() => sessionFixtureNames(['session.1.jsonl'])).toThrow('missing session.jsonl') + expect(() => sessionFixtureNames(['session.1.jsonl'])).toThrow('missing parent session fixture') }) it('rejects gapped child fixtures', () => { expect(() => sessionFixtureNames(['session.jsonl', 'session.2.jsonl'])) - .toThrow('expected session.1.jsonl, found session.2.jsonl') + .toThrow('expected index 1, found session.2.jsonl') }) - it.each(['session.0.jsonl', 'session.child.jsonl', 'session.01.jsonl'])( - 'rejects invalid child fixture name %s', + it.each([ + 'session.0.jsonl', + 'session.child.jsonl', + 'session.01.jsonl', + 'session.v0.jsonl', + 'session.V1.jsonl', + 'session.v01.jsonl', + 'session.1.v0.jsonl', + ])( + 'rejects invalid fixture name %s', (name) => { expect(() => sessionFixtureNames(['session.jsonl', name])) - .toThrow(`invalid child session fixture name: ${name}`) + .toThrow(`invalid session fixture name: ${name}`) }, ) - it('rejects duplicate child indexes', () => { + it('rejects duplicate role generations', () => { expect(() => sessionFixtureNames(['session.jsonl', 'session.1.jsonl', 'session.1.jsonl'])) - .toThrow('expected session.2.jsonl, found session.1.jsonl') + .toThrow('duplicate session fixture generation: session.1.jsonl') + }) +}) + +describe('Session generation filename helpers', () => { + it('renders canonical parent, child, raw, and compressed names', () => { + expect(sessionFixtureName(0, 0)).toBe('session.jsonl') + expect(sessionFixtureName(0, 2)).toBe('session.v2.jsonl') + expect(sessionFixtureName(3, 0)).toBe('session.3.jsonl') + expect(sessionFixtureName(3, 2)).toBe('session.3.v2.jsonl') + expect(persistedSessionFilename(0)).toBe('session.jsonl') + expect(persistedSessionFilename(2, 'zstd')).toBe('session.v2.jsonl.zstd') + }) + + it.each([ + () => sessionFixtureName(-1, 0), + () => sessionFixtureName(-0, 0), + () => sessionFixtureName(0.5, 0), + () => sessionFixtureName(0, -1), + () => persistedSessionFilename(Number.MAX_SAFE_INTEGER + 1), + ])('rejects invalid numeric filename components', (render) => { + expect(render).toThrow(/non-negative safe integer/) + }) + + it('parses canonical fixture and persistence names without admitting lookalikes', () => { + expect(parseSessionFixtureName('session.2.v3.jsonl')).toEqual({ index: 2, version: 3, name: 'session.2.v3.jsonl' }) + expect(parseSessionFixtureName('notes.jsonl')).toBeUndefined() + expect(() => parseSessionFixtureName(`session.${'9'.repeat(400)}.jsonl`)) + .toThrow('invalid session fixture name') + expect(parsePersistedSessionFilename('session.jsonl')).toEqual({ version: 0, compression: 'raw', name: 'session.jsonl' }) + expect(parsePersistedSessionFilename('session.v4.jsonl.zstd')).toEqual({ version: 4, compression: 'zstd', name: 'session.v4.jsonl.zstd' }) + expect(parsePersistedSessionFilename('session.1.jsonl')).toBeUndefined() + expect(parsePersistedSessionFilename(`session.v${'9'.repeat(400)}.jsonl`)).toBeUndefined() + }) + + it('selects one highest persisted generation per directory and compression', () => { + const paths = [ + 'b/session.v1.jsonl', + 'b/session.jsonl', + 'a/session.jsonl', + 'a/session.v2.jsonl', + 'a/session.v3.jsonl.zstd', + 'a/session.v2.20260101.backup.jsonl', + 'noise.jsonl', + ] + expect(latestPersistedSessionPaths(paths)).toEqual(['a/session.v2.jsonl', 'b/session.v1.jsonl']) + expect(latestPersistedSessionPaths(paths, 'zstd')).toEqual(['a/session.v3.jsonl.zstd']) + }) + + it('validates filename and header generation equality', () => { + const v0 = '{"type":"session","version":0}\n' + const v1 = '{"type":"session","version":1}\n' + expect(sessionHeaderVersion(v1, 'fixture')).toBe(1) + expect(assertSessionFixtureVersion('session.jsonl', v0)).toBe(0) + expect(assertPersistedSessionVersion('session.v1.jsonl', v1)).toBe(1) + expect(() => assertSessionFixtureVersion('notes.jsonl', v0)).toThrow('not a session fixture name') + expect(assertSessionFixtureVersion('session.jsonl', '{"type":"session"}\n')).toBe(0) + expect(() => assertSessionFixtureVersion('session.jsonl', '')).toThrow('session fixture is empty') + expect(() => assertSessionFixtureVersion('session.jsonl', '{')).toThrow('session header contains invalid JSON') + expect(() => assertPersistedSessionVersion('session.1.jsonl', v0)).toThrow('not a canonical Session persistence filename') + expect(() => assertSessionFixtureVersion('session.v1.jsonl', v0)) + .toThrow('filename declares Session format v1, header declares v0') + expect(() => assertPersistedSessionVersion('session.jsonl', v1)) + .toThrow('filename declares Session format v0, header declares v1') + }) + + it.each([ + ['', 'session fixture is empty'], + ['{', 'session header contains invalid JSON'], + ['[]', 'first record must be a Session header'], + ['{"type":"event","version":0}', 'first record must be a Session header'], + ['{"type":"session"}', 'version must be a non-negative safe integer'], + ['{"type":"session","version":-1}', 'version must be a non-negative safe integer'], + ['{"type":"session","version":-0}', 'version must be a non-negative safe integer'], + ['{"type":"session","version":1.5}', 'version must be a non-negative safe integer'], + ])('rejects malformed header %j', (content, message) => { + expect(() => sessionHeaderVersion(content, 'bad.jsonl')).toThrow(message) }) }) diff --git a/packages/test-support/session-snapshot/tsconfig.json b/packages/test-support/session-snapshot/tsconfig.json index 8be01d94b1..eea68b012b 100644 --- a/packages/test-support/session-snapshot/tsconfig.json +++ b/packages/test-support/session-snapshot/tsconfig.json @@ -14,6 +14,9 @@ { "path": "../loader-smoke" }, + { + "path": "../llm-replay" + }, { "path": "../../core/session" } diff --git a/packages/workspace/workspace/src/index.ts b/packages/workspace/workspace/src/index.ts index 87edb28cad..578995a77b 100644 --- a/packages/workspace/workspace/src/index.ts +++ b/packages/workspace/workspace/src/index.ts @@ -10,7 +10,8 @@ import { stat } from 'node:fs/promises' import { basename } from 'node:path' import { Context, Service } from '@deepseek-ai/cordis' import type { SessionHeader, SessionId } from '@deepseek-ai/dsh-session' -import type {} from '@deepseek-ai/dsh-session-persistence' +import { isReadableSessionPersistenceListing } from '@deepseek-ai/dsh-session-persistence' +import type { SessionPersistenceListing } from '@deepseek-ai/dsh-session-persistence' import type { DomainGlobal, KvTable } from '@deepseek-ai/dsh-storage-domain' import { WorkspaceEntity } from './entity.ts' import type { WorkspaceEntityHost } from './entity.ts' @@ -82,6 +83,11 @@ const sameIds = (left: readonly WorkspaceId[], right: readonly WorkspaceId[]): b const compareHeaders = (left: SessionHeader, right: SessionHeader): number => right.createdAt - left.createdAt || String(left.id).localeCompare(String(right.id)) +/** Select the latest logical headers a workspace can index. */ +function listedHeaders(listings: readonly SessionPersistenceListing[]): SessionHeader[] { + return listings.filter(isReadableSessionPersistenceListing).map(listing => listing.header) +} + /** * Durable workspace registry. Startup waits for `sessionPersistence`, builds * one canonical-cwd header index, and completes the one-time history @@ -126,11 +132,11 @@ export class WorkspaceRegistry extends Service { await this.recoverPendingMutation() this.validateStoredState(this.state) if (!this.state.initialized) { - const headers = await this.ctx.sessionPersistence.list() + const headers = listedHeaders(await this.ctx.sessionPersistence.list()) await this.replaceHeaderIndex(headers) await this.bootstrap(headers) } else if (this.table.size > 0) { - await this.replaceHeaderIndex(await this.ctx.sessionPersistence.list()) + await this.replaceHeaderIndex(listedHeaders(await this.ctx.sessionPersistence.list())) } await this.indexLiveSessions() @@ -263,7 +269,7 @@ export class WorkspaceRegistry extends Service { private async sessionKnown(id: SessionId): Promise { if (this.ctx.get('sessions')?.get(id) !== undefined) return true if (this.headers.has(id)) return true - await this.indexHeaders(await this.ctx.sessionPersistence.list()) + await this.indexHeaders(listedHeaders(await this.ctx.sessionPersistence.list())) return this.headers.has(id) } @@ -621,7 +627,7 @@ export class WorkspaceRegistry extends Service { const cached = this.headers.get(id) if (cached !== undefined) return cached - const headers = await this.ctx.sessionPersistence.list() + const headers = listedHeaders(await this.ctx.sessionPersistence.list()) await this.indexHeaders(headers) const header = this.headers.get(id) if (header === undefined) { diff --git a/packages/workspace/workspace/tests/workspace.spec.ts b/packages/workspace/workspace/tests/workspace.spec.ts index 49348cea2a..5401fd8308 100644 --- a/packages/workspace/workspace/tests/workspace.spec.ts +++ b/packages/workspace/workspace/tests/workspace.spec.ts @@ -7,8 +7,9 @@ import Storage from '@deepseek-ai/dsh-storage' import type { StorageBackend } from '@deepseek-ai/dsh-storage' import { DomainFacility } from '@deepseek-ai/dsh-storage-domain' import type { DomainChanged } from '@deepseek-ai/dsh-storage-domain' -import SessionStore, { SessionId } from '@deepseek-ai/dsh-session' +import SessionStore, { SESSION_FORMAT_VERSION, SessionId } from '@deepseek-ai/dsh-session' import type { SessionHeader } from '@deepseek-ai/dsh-session' +import type { CurrentSessionPersistenceListing, SessionPersistenceListing } from '@deepseek-ai/dsh-session-persistence' import { MemoryMediaPool, MemoryStorageBackend } from '../../../storage/storage-domain/tests/helpers/memory-backend.ts' import WorkspaceRegistry, { WorkspaceId, @@ -20,13 +21,20 @@ import type { WorkspaceDomainState, WorkspaceRecord } from '../src/index.ts' const DOMAIN_VERSION = 2 const header = (id: string, cwd?: string, createdAt = 0): SessionHeader => ({ - version: 0, + version: SESSION_FORMAT_VERSION, id: SessionId(id), createdAt, isSeeded: false, ...(cwd === undefined ? {} : { cwd }), }) +const currentListing = (value: SessionHeader): CurrentSessionPersistenceListing => ({ + status: 'current', + header: value, + storedVersion: SESSION_FORMAT_VERSION, + targetVersion: SESSION_FORMAT_VERSION, +}) + interface HarnessOptions { pool?: MemoryMediaPool sessions?: SessionHeader[] @@ -46,7 +54,7 @@ async function harness(options: HarnessOptions = {}) { ctx.provide('storageDomain', facility) let listed = options.sessions ?? [] - const list = vi.fn(async () => listed) + const list = vi.fn(async () => listed.map(currentListing)) const load = vi.fn(() => { throw new Error('event bodies must not be loaded') }) const inspect = vi.fn(() => { throw new Error('event bodies must not be inspected') }) ctx.provide('sessionPersistence', { list, load, inspect } as never) @@ -192,7 +200,7 @@ describe('WorkspaceRegistry lifecycle and bootstrap', () => { expect(ctx.get('workspaceRegistry')).toBeUndefined() expect(pool.media.has('workspace')).toBe(false) - const list = vi.fn(async () => [] as SessionHeader[]) + const list = vi.fn(async () => [] as SessionPersistenceListing[]) ctx.provide('sessionPersistence', { list } as never) await fiber.await() expect(ctx.workspaceRegistry.list()).toEqual([]) diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 4521b1bbf4..a28c1e3906 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -7057,6 +7057,41 @@ importers: specifier: workspace:^ version: link:../../core/tools + packages/session/session-format: + devDependencies: + '@deepseek-ai/cordis': + specifier: workspace:^ + version: link:../../../vendor/cordis + + packages/session/session-format-catalog: + dependencies: + '@deepseek-ai/dsh-session-format': + specifier: workspace:^ + version: link:../session-format + '@deepseek-ai/dsh-session-format-v0-to-v1': + specifier: workspace:^ + version: link:../session-format-v0-to-v1 + devDependencies: + '@deepseek-ai/cordis': + specifier: workspace:^ + version: link:../../../vendor/cordis + '@deepseek-ai/dsh-session': + specifier: workspace:^ + version: link:../../core/session + + packages/session/session-format-v0-to-v1: + dependencies: + '@deepseek-ai/dsh-session-format': + specifier: workspace:^ + version: link:../session-format + devDependencies: + '@deepseek-ai/cordis': + specifier: workspace:^ + version: link:../../../vendor/cordis + '@deepseek-ai/dsh-session': + specifier: workspace:^ + version: link:../../core/session + packages/session/session-log-deepseek: dependencies: '@deepseek-ai/dsh-brand': @@ -7106,6 +7141,9 @@ importers: packages/session/session-persistence-jsonl: dependencies: + '@deepseek-ai/dsh-session-format-catalog': + specifier: workspace:^ + version: link:../session-format-catalog '@deepseek-ai/schemastery': specifier: link:../../../vendor/schemastery version: link:../../../vendor/schemastery @@ -8951,6 +8989,9 @@ importers: packages/test-support/llm-replay: dependencies: + '@deepseek-ai/dsh-session-format-catalog': + specifier: workspace:^ + version: link:../../session/session-format-catalog '@deepseek-ai/dsh-util-values': specifier: workspace:^ version: link:../../util/values @@ -9004,6 +9045,9 @@ importers: '@deepseek-ai/cordis-plugin-include': specifier: workspace:* version: link:../../../vendor/include + '@deepseek-ai/dsh-llm-replay': + specifier: workspace:^ + version: link:../llm-replay '@deepseek-ai/dsh-loader-smoke': specifier: workspace:* version: link:../loader-smoke diff --git a/python/sdk/tests/test_smoke_model.py b/python/sdk/tests/test_smoke_model.py index c0e6b9f70e..344d569288 100644 --- a/python/sdk/tests/test_smoke_model.py +++ b/python/sdk/tests/test_smoke_model.py @@ -71,3 +71,71 @@ def test_mcp_smoke_accepts_the_external_server_result() -> None: for chunk in chunks for choice in chunk.get("choices", []) ) + + +def test_snapshot_comparison_normalizes_only_session_generation_provenance() -> None: + normalize = SMOKE["normalize_session_format_comparison"] + expected = { + "header": {"type": "session", "version": 0, "otherVersion": 7}, + "accepted": { + "type": "session-log-deepseek/delivery-accepted", + "data": {"sessionId": "s", "throughSeq": 4}, + }, + "source": { + "kind": "session-reference", + "references": [{"sessionId": "other", "capturedThroughSeq": 8}], + }, + } + actual = { + "header": {"type": "session", "version": 1, "otherVersion": 7}, + "accepted": { + "type": "session-log-deepseek/delivery-accepted", + "data": {"sessionId": "s", "sessionFormatVersion": 1, "throughSeq": 4}, + }, + "source": { + "kind": "session-reference", + "references": [{ + "sessionId": "other", + "capturedFormatVersion": 1, + "capturedThroughSeq": 8, + }], + }, + } + + assert normalize(expected) == normalize(actual) + assert normalize(expected)["header"]["otherVersion"] == 7 + + +def test_snapshot_generation_names_select_highest_role_without_double_counting( + tmp_path: Path, +) -> None: + render = SMOKE["snapshot_session_filename"] + select = SMOKE["selected_snapshot_session_files"] + assert render(0, 0) == "session.jsonl" + assert render(0, 2) == "session.v2.jsonl" + assert render(3, 0) == "session.3.jsonl" + assert render(3, 2) == "session.3.v2.jsonl" + + (tmp_path / "session.jsonl").write_text( + '{"type":"session","version":0}\n', encoding="utf-8", + ) + (tmp_path / "session.v1.jsonl").write_text( + '{"type":"session","version":1}\n', encoding="utf-8", + ) + (tmp_path / "session.1.jsonl").write_text( + '{"type":"session","version":0}\n', encoding="utf-8", + ) + + assert {index: path.name for index, path in select(tmp_path).items()} == { + 0: "session.v1.jsonl", + 1: "session.1.jsonl", + } + + +def test_snapshot_generation_filename_must_match_header(tmp_path: Path) -> None: + (tmp_path / "session.v1.jsonl").write_text( + '{"type":"session","version":0}\n', encoding="utf-8", + ) + + with pytest.raises(AssertionError, match="filename declares Session format v1"): + SMOKE["selected_snapshot_session_files"](tmp_path) diff --git a/scripts/doc-standard.spec.ts b/scripts/doc-standard.spec.ts index aa2987fec0..91bea8fbe5 100644 --- a/scripts/doc-standard.spec.ts +++ b/scripts/doc-standard.spec.ts @@ -63,6 +63,9 @@ const PACKAGE_LIBRARIES: Readonly> = { 'packages/sandbox/sandbox-windows-acl': 'Windows ACL sandbox library consumed by sandbox-local.', 'packages/sdk/client': 'Client-process library; the spawned runtime owns plugin behavior.', 'packages/sdk/protocol': 'Wire-protocol library with type declarations only.', + 'packages/session/session-format': 'Pure Session format planning, codec dispatch, and lossless JSON library.', + 'packages/session/session-format-catalog': 'Generated build-static Session format inventory with no plugin registration.', + 'packages/session/session-format-v0-to-v1': 'Pure released-v0 codec and adjacent migration library.', 'packages/session/session-telemetry': 'Telemetry Service Definition and capture library; providers mount the backend.', 'packages/session/session-title-llm': 'Shared LLM title-provider registration and request policy.', 'packages/subagent/subagent-in-process-driver': 'Shared one-shot child-agent driver used by provider plugins.', diff --git a/scripts/gen-cordis-catalog.ts b/scripts/gen-cordis-catalog.ts index e0f068e0d7..ba61fb3b61 100644 --- a/scripts/gen-cordis-catalog.ts +++ b/scripts/gen-cordis-catalog.ts @@ -416,6 +416,7 @@ export const LINK_MAP: Readonly> = { BorrowedSessionSource: 'persistence.md', SessionLocation: 'persistence.md', SessionPreparation: 'persistence.md', + SessionPersistenceListing: 'persistence.md', SessionPersistenceSnapshot: 'persistence.md', SessionRawArtifact: 'persistence.md', ConfinedArgv: 'sandbox.md', diff --git a/scripts/gen-persistence-catalog.ts b/scripts/gen-persistence-catalog.ts index c0ed4a5c04..e9d74e6a5d 100644 --- a/scripts/gen-persistence-catalog.ts +++ b/scripts/gen-persistence-catalog.ts @@ -368,7 +368,7 @@ export function render(events: AnnotatedLogEventEntry[], envelopeTypes: EventEnv '', 'This file is GENERATED from source (`scripts/gen-persistence-catalog.ts`) and verified fresh by `pnpm run verify-persistence-catalog` (part of `doc-sync`) — do not edit it by hand. Declaration blocks retain the source declaration and nested property JSDoc, removing only the indentation imposed by a containing interface/module, and use a `ts persistence-catalog` fence (skipped by doc-typecheck because declarations reference types from their owning modules). Type names in a payload link to the page that documents them. See [the persistence-log-catalog Agent Note](../.agents/notes/archived/process/2026-07-04-persistence-log-catalog.md).', '', - 'The envelope declarations below compose each event\'s `type`, monotonic `seq`, epoch-ms `time`, `data`, the optional `ignorable` unknown-type skip marker, and the conditional `surfaceOp`/`sourceEventSeqs` fields. **surface** marks a `SurfaceEventType` member: it produces an LLM message and declares how it joins the surface list. **log-only** marks everything else: a durable, replayable record with no derived-history contribution. Every payload is JSON-serializable (enforced at `Session.append`), and the whole format is pinned at `SESSION_FORMAT_VERSION = 0` — pre-release, no compatibility implied ([the version stance](subsystems/persistence.md)). Scope: the packages in this repo; a downstream plugin can merge further event types, which are outside this catalog by construction.', + 'The envelope declarations below compose each event\'s `type`, monotonic `seq`, epoch-ms `time`, `data`, the optional `ignorable` unknown-type skip marker, and the conditional `surfaceOp`/`sourceEventSeqs` fields. **surface** marks a `SurfaceEventType` member: it produces an LLM message and declares how it joins the surface list. **log-only** marks everything else: a durable, replayable record with no derived-history contribution. Every payload is JSON-serializable (enforced at `Session.append`). Current writers stamp `SESSION_FORMAT_VERSION`; supported historical artifacts reach this current vocabulary through the build-static adjacent migration catalog ([the version lifecycle](subsystems/persistence.md)). Scope: the packages in this repo; a downstream plugin can merge further current-version event types, which are outside this catalog by construction and require an explicit disposition at a later format edge.', '', '## Event envelope', '', diff --git a/scripts/gen-session-format-catalog.spec.ts b/scripts/gen-session-format-catalog.spec.ts new file mode 100644 index 0000000000..30d2c85cdc --- /dev/null +++ b/scripts/gen-session-format-catalog.spec.ts @@ -0,0 +1,165 @@ +import { mkdirSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { afterEach, describe, expect, it, vi } from 'vitest' +import { + collectSessionFormatMigrations, + readCurrentSessionFormatVersion, + renderSessionFormatCatalog, +} from './gen-session-format-catalog.ts' + +const fixtureRoots: string[] = [] + +afterEach(() => { + for (const root of fixtureRoots.splice(0)) rmSync(root, { recursive: true, force: true }) +}) + +function fixture(edges: Array<[number, number]>): string { + const root = mkdtempSync(join(tmpdir(), 'dsh-session-format-catalog-')) + fixtureRoots.push(root) + mkdirSync(join(root, 'packages/core/session/src'), { recursive: true }) + writeFileSync(join(root, 'packages/core/session/src/types.ts'), 'export const SESSION_FORMAT_VERSION = 2\n') + const catalogDependencies: Record = {} + for (const [from, to] of edges) { + const dir = join(root, `packages/session/session-format-v${from}-to-v${to}`) + const name = `@deepseek-ai/dsh-session-format-v${from}-to-v${to}` + mkdirSync(dir, { recursive: true }) + catalogDependencies[name] = 'workspace:^' + writeFileSync(join(dir, 'package.json'), JSON.stringify({ + name, + dsh: { sessionFormatMigration: { + from, to, export: '.', + migration: `sessionFormatV${from}ToV${to}`, + sourceCodec: `releasedV${from}SessionFormatCodec`, + targetCodec: `releasedV${to}SessionFormatCodec`, + targetHeaderValidator: `assertReleasedV${to}Header`, + targetRestorer: `restoreReleasedV${to}Artifact`, + } }, + dependencies: from === 0 + ? { '@deepseek-ai/dsh-session-format': 'workspace:^' } + : { + '@deepseek-ai/dsh-session-format': 'workspace:^', + [`@deepseek-ai/dsh-session-format-v${from - 1}-to-v${from}`]: 'workspace:^', + }, + })) + } + const catalog = join(root, 'packages/session/session-format-catalog') + mkdirSync(catalog, { recursive: true }) + writeFileSync(join(catalog, 'package.json'), JSON.stringify({ + dependencies: catalogDependencies, + peerDependencies: { '@deepseek-ai/dsh-session': 'workspace:^' }, + devDependencies: { '@deepseek-ai/dsh-session': 'workspace:^' }, + })) + return root +} + +function edgeManifest(root: string, from: number, to: number): { + path: string + value: Record & { + dependencies: Record + dsh: { sessionFormatMigration: Record } + } +} { + const path = join(root, `packages/session/session-format-v${from}-to-v${to}/package.json`) + return { + path, + value: JSON.parse(readFileSync(path, 'utf8')) as Record & { + dependencies: Record + dsh: { sessionFormatMigration: Record } + }, + } +} + +describe('session format catalog generator', () => { + it('normalizes Windows separators in discovered manifest paths before validation', async () => { + const root = fixture([[0, 1], [1, 2]]) + vi.resetModules() + vi.doMock('node:fs', async (importOriginal) => { + const actual = await importOriginal() + return { + ...actual, + globSync: () => [ + 'packages\\session\\session-format-v0-to-v1\\package.json', + 'packages\\session\\session-format-v1-to-v2\\package.json', + ], + } + }) + + try { + const { collectSessionFormatMigrations: collect } = await import('./gen-session-format-catalog.ts') + expect(collect(root, 2).map(item => [item.from, item.to])).toEqual([[0, 1], [1, 2]]) + } finally { + vi.doUnmock('node:fs') + vi.resetModules() + } + }) + + it('discovers one complete adjacent chain and renders direct imports', () => { + const root = fixture([[0, 1], [1, 2]]) + const version = readCurrentSessionFormatVersion(root) + const declarations = collectSessionFormatMigrations(root, version) + const output = renderSessionFormatCatalog(declarations, version) + + expect(declarations.map(item => [item.from, item.to])).toEqual([[0, 1], [1, 2]]) + expect(output).toContain("from '@deepseek-ai/dsh-session-format-v0-to-v1'") + expect(output).toContain('currentVersion: 2') + expect(output).toContain('restoreReleasedV2Artifact(artifact, KNOWN_SESSION_EVENT_TYPES)') + expect(output).toContain('assertReleasedV2Header(header)') + expect(output).toContain('validateInstalledCurrentSessionHeader(header)') + expect(output).toContain("from '@deepseek-ai/dsh-session'") + expect(output).toContain("from './current.ts'") + const imports = output.split('\n').filter(line => line.startsWith('import {')) + expect(imports.filter(line => line.includes('releasedV1SessionFormatCodec'))).toHaveLength(1) + }) + + it('refuses a missing adjacent edge', () => { + const root = fixture([[0, 1]]) + expect(() => collectSessionFormatMigrations(root, 2)).toThrow(/exactly one v1->v2/) + }) + + it('refuses a later edge whose declared source codec does not continue the prior target', () => { + const root = fixture([[0, 1], [1, 2]]) + const manifest = edgeManifest(root, 1, 2) + manifest.value.dsh.sessionFormatMigration['sourceCodec'] = 'UnrelatedV1Codec' + writeFileSync(manifest.path, JSON.stringify(manifest.value)) + + expect(() => collectSessionFormatMigrations(root, 2)) + .toThrow(/source codec UnrelatedV1Codec does not continue releasedV1SessionFormatCodec/) + }) + + it('requires every later edge to depend on the package that owns its source codec', () => { + const root = fixture([[0, 1], [1, 2]]) + const manifest = edgeManifest(root, 1, 2) + delete manifest.value.dependencies['@deepseek-ai/dsh-session-format-v0-to-v1'] + writeFileSync(manifest.path, JSON.stringify(manifest.value)) + + expect(() => collectSessionFormatMigrations(root, 2)) + .toThrow(/must depend on @deepseek-ai\/dsh-session-format-v0-to-v1/) + }) + + it('requires the package name to identify its declared adjacent edge', () => { + const root = fixture([[0, 1], [1, 2]]) + const manifest = edgeManifest(root, 1, 2) + manifest.value['name'] = '@deepseek-ai/dsh-session-format-other' + writeFileSync(manifest.path, JSON.stringify(manifest.value)) + + expect(() => collectSessionFormatMigrations(root, 2)) + .toThrow(/name must be @deepseek-ai\/dsh-session-format-v1-to-v2/) + }) + + it('requires the catalog to share the installed Session package as a peer', () => { + const root = fixture([[0, 1], [1, 2]]) + const path = join(root, 'packages/session/session-format-catalog/package.json') + const manifest = JSON.parse(readFileSync(path, 'utf8')) as { + dependencies: Record + peerDependencies: Record + devDependencies: Record + } + manifest.dependencies['@deepseek-ai/dsh-session'] = 'workspace:^' + delete manifest.peerDependencies['@deepseek-ai/dsh-session'] + writeFileSync(path, JSON.stringify(manifest)) + + expect(() => collectSessionFormatMigrations(root, 2)) + .toThrow(/must share @deepseek-ai\/dsh-session through peer \+ dev dependencies/) + }) +}) diff --git a/scripts/gen-session-format-catalog.ts b/scripts/gen-session-format-catalog.ts new file mode 100644 index 0000000000..a05800e4ac --- /dev/null +++ b/scripts/gen-session-format-catalog.ts @@ -0,0 +1,234 @@ +/** Generate the build-static Session format catalog from edge package metadata. */ + +import { globSync, readFileSync, writeFileSync } from 'node:fs' +import { resolve } from 'node:path' +import { pathToFileURL } from 'node:url' + +const root = resolve(import.meta.dirname, '..') +const OUT = 'packages/session/session-format-catalog/src/generated.ts' + +/** One adjacent migration declaration read from a workspace manifest. */ +export interface SessionFormatMigrationManifest { + readonly packageName: string + readonly importPath: string + readonly from: number + readonly to: number + readonly migration: string + readonly sourceCodec: string + readonly targetCodec: string + readonly targetHeaderValidator: string + readonly targetRestorer: string +} + +interface RawManifest { + readonly name?: unknown + readonly dependencies?: Readonly> + readonly peerDependencies?: Readonly> + readonly devDependencies?: Readonly> + readonly dsh?: { + readonly sessionFormatMigration?: Readonly> + } +} + +function readJson(path: string): RawManifest { + return JSON.parse(readFileSync(path, 'utf8')) as RawManifest +} + +function safeVersion(value: unknown, label: string): number { + if (!Number.isSafeInteger(value) || (value as number) < 0 || Object.is(value, -0)) { + throw new Error(`gen-session-format-catalog: ${label} must be a non-negative safe integer`) + } + return value as number +} + +function nonempty(value: unknown, label: string): string { + if (typeof value !== 'string' || value.length === 0) { + throw new Error(`gen-session-format-catalog: ${label} must be a non-empty string`) + } + return value +} + +/** + * Read the current writer version from the core Session source of truth. + * @param scanRoot - repository root whose Session source is authoritative. + * @returns the current non-negative Session format version. + */ +export function readCurrentSessionFormatVersion(scanRoot: string = root): number { + const source = readFileSync(resolve(scanRoot, 'packages/core/session/src/types.ts'), 'utf8') + const match = source.match(/export const SESSION_FORMAT_VERSION = (\d+)\b/) + if (match === null) throw new Error('gen-session-format-catalog: cannot read SESSION_FORMAT_VERSION') + return safeVersion(Number(match[1]), 'SESSION_FORMAT_VERSION') +} + +/** + * Collect and validate the unique complete adjacent migration inventory. + * @param scanRoot - repository root containing migration package manifests. + * @param currentVersion - writer version the inventory must reach exactly. + * @returns ordered adjacent migration declarations from v0 to the current writer. + */ +export function collectSessionFormatMigrations( + scanRoot: string = root, + currentVersion: number = readCurrentSessionFormatVersion(scanRoot), +): SessionFormatMigrationManifest[] { + const declarations: SessionFormatMigrationManifest[] = [] + for (const discovered of globSync('packages/session/session-format-v*-to-v*/package.json', { cwd: scanRoot }).sort()) { + const rel = discovered.replaceAll('\\', '/') + const manifest = readJson(resolve(scanRoot, rel)) + const metadata = manifest.dsh?.sessionFormatMigration + if (metadata === undefined) { + throw new Error(`gen-session-format-catalog: ${rel} lacks dsh.sessionFormatMigration`) + } + const allowed = new Set([ + 'from', 'to', 'export', 'migration', 'sourceCodec', 'targetCodec', + 'targetHeaderValidator', 'targetRestorer', + ]) + const extra = Object.keys(metadata).find(key => !allowed.has(key)) + if (extra !== undefined) throw new Error(`gen-session-format-catalog: ${rel} has unknown metadata member ${extra}`) + const packageName = nonempty(manifest.name, `${rel} name`) + const from = safeVersion(metadata['from'], `${rel} from`) + const to = safeVersion(metadata['to'], `${rel} to`) + if (to !== from + 1) throw new Error(`gen-session-format-catalog: ${rel} must declare adjacent v${from}->v${from + 1}`) + const expectedPackageName = `@deepseek-ai/dsh-session-format-v${from}-to-v${to}` + if (packageName !== expectedPackageName) { + throw new Error(`gen-session-format-catalog: ${rel} name must be ${expectedPackageName}`) + } + const directoryMatch = rel.match(/session-format-v(\d+)-to-v(\d+)\/package\.json$/) + if (directoryMatch === null || Number(directoryMatch[1]) !== from || Number(directoryMatch[2]) !== to) { + throw new Error(`gen-session-format-catalog: ${rel} directory does not match v${from}->v${to}`) + } + const exportPath = nonempty(metadata['export'], `${rel} export`) + declarations.push({ + packageName, + importPath: exportPath === '.' ? packageName : `${packageName}/${exportPath.replace(/^\.\//, '')}`, + from, + to, + migration: nonempty(metadata['migration'], `${rel} migration`), + sourceCodec: nonempty(metadata['sourceCodec'], `${rel} sourceCodec`), + targetCodec: nonempty(metadata['targetCodec'], `${rel} targetCodec`), + targetHeaderValidator: nonempty(metadata['targetHeaderValidator'], `${rel} targetHeaderValidator`), + targetRestorer: nonempty(metadata['targetRestorer'], `${rel} targetRestorer`), + }) + } + declarations.sort((left, right) => left.from - right.from) + for (let version = 0; version < currentVersion; version += 1) { + const matches = declarations.filter(item => item.from === version) + if (matches.length !== 1) { + throw new Error(`gen-session-format-catalog: expected exactly one v${version}->v${version + 1} package, found ${matches.length}`) + } + } + const extra = declarations.find(item => item.from >= currentVersion) + if (extra !== undefined || declarations.length !== currentVersion) { + throw new Error(`gen-session-format-catalog: migration inventory does not end exactly at current v${currentVersion}`) + } + const catalog = readJson(resolve(scanRoot, 'packages/session/session-format-catalog/package.json')) + if (catalog.dependencies?.['@deepseek-ai/dsh-session'] !== undefined + || catalog.peerDependencies?.['@deepseek-ai/dsh-session'] === undefined + || catalog.devDependencies?.['@deepseek-ai/dsh-session'] === undefined) { + throw new Error( + 'gen-session-format-catalog: catalog must share @deepseek-ai/dsh-session through peer + dev dependencies', + ) + } + for (const [index, declaration] of declarations.entries()) { + if (catalog.dependencies?.[declaration.packageName] === undefined) { + throw new Error(`gen-session-format-catalog: catalog package lacks dependency ${declaration.packageName}`) + } + const previous = declarations[index - 1] + if (previous === undefined) continue + if (declaration.sourceCodec !== previous.targetCodec) { + throw new Error( + `gen-session-format-catalog: v${declaration.from} source codec ${declaration.sourceCodec} ` + + `does not continue ${previous.targetCodec}`, + ) + } + const manifest = readJson(resolve( + scanRoot, + `packages/session/session-format-v${declaration.from}-to-v${declaration.to}/package.json`, + )) + if (manifest.dependencies?.[previous.packageName] === undefined) { + throw new Error( + `gen-session-format-catalog: ${declaration.packageName} must depend on ${previous.packageName} ` + + 'to share the adjacent source codec', + ) + } + } + return declarations +} + +/** + * Render the deterministic direct-import catalog source. + * @param declarations - validated adjacent migrations in version order. + * @param currentVersion - writer version reached by the final declaration. + * @returns complete generated TypeScript source. + */ +export function renderSessionFormatCatalog( + declarations: readonly SessionFormatMigrationManifest[], + currentVersion: number, +): string { + const imports = declarations.map((item) => { + const names = [item.targetCodec, item.migration] + if (item.from === 0) names.push(item.sourceCodec) + if (item.to === currentVersion) names.push(item.targetRestorer, item.targetHeaderValidator) + return `import { ${[...new Set(names)].sort().join(', ')} } from '${item.importPath}'` + }) + const first = declarations[0] + const codecs = first === undefined + ? [] + : [first.sourceCodec, ...declarations.map(item => item.targetCodec)] + const restorer = declarations.at(-1)?.targetRestorer + const headerValidator = declarations.at(-1)?.targetHeaderValidator + if (restorer === undefined) throw new Error('gen-session-format-catalog: current format has no target restorer') + if (headerValidator === undefined) { + throw new Error('gen-session-format-catalog: current format has no target header validator') + } + return [ + '/**', + ' * GENERATED by `scripts/gen-session-format-catalog.ts` — do not edit by hand.', + ' * The direct imports make historical readability independent of mounted plugins.', + ' */', + '', + "import { KNOWN_SESSION_EVENT_TYPES } from '@deepseek-ai/dsh-session'", + "import { createSessionFormatCatalog } from '@deepseek-ai/dsh-session-format'", + "import { validateInstalledCurrentSessionArtifact, validateInstalledCurrentSessionHeader } from './current.ts'", + ...imports, + '', + '/** Physical codec dispatch and complete adjacent chain, independent of mounted plugins. */', + 'export const sessionFormatCatalog = createSessionFormatCatalog({', + ` currentVersion: ${currentVersion},`, + ` codecs: [${codecs.join(', ')}],`, + ` migrations: [${declarations.map(item => item.migration).join(', ')}],`, + ' restoreCurrent(artifact) {', + ` const restored = ${restorer}(artifact, KNOWN_SESSION_EVENT_TYPES)`, + ' validateInstalledCurrentSessionArtifact(restored)', + ' return restored', + ' },', + ' restoreCurrentHeader(header) {', + ` ${headerValidator}(header)`, + ' validateInstalledCurrentSessionHeader(header)', + ' return header', + ' },', + '})', + '', + ].join('\n') +} + +function main(): void { + const currentVersion = readCurrentSessionFormatVersion(root) + const declarations = collectSessionFormatMigrations(root, currentVersion) + const output = renderSessionFormatCatalog(declarations, currentVersion) + const target = resolve(root, OUT) + if (process.argv.includes('--check')) { + let current = '' + try { current = readFileSync(target, 'utf8') } catch { /* missing is stale */ } + if (current !== output) { + console.error(`gen-session-format-catalog: ${OUT} is stale; run pnpm run gen-session-format-catalog`) + process.exitCode = 1 + return + } + console.log(`gen-session-format-catalog: ${OUT} is up to date.`) + return + } + writeFileSync(target, output) + console.log(`gen-session-format-catalog: wrote ${OUT}.`) +} + +if (process.argv[1] !== undefined && import.meta.url === pathToFileURL(resolve(process.argv[1])).href) main() diff --git a/scripts/run-gates.ts b/scripts/run-gates.ts index c0b8db432c..46a55057cf 100644 --- a/scripts/run-gates.ts +++ b/scripts/run-gates.ts @@ -732,6 +732,7 @@ function docSyncLeafGates(options: { pnpmScript('tool-catalog', 'verify-tool-catalog', { label: 'tool catalog' }), pnpmScript('config-catalog', 'verify-config-catalog', { label: 'config catalog' }), pnpmScript('persistence-catalog', 'verify-persistence-catalog', { label: 'persistence catalog' }), + pnpmScript('session-format-catalog', 'verify-session-format-catalog', { label: 'Session format catalog' }), pnpmScript('public-repository-links', 'verify-public-repository-links', { label: 'public repository links', quick: true }), pnpmScript('doc-refs', 'verify-doc-refs', { label: 'doc refs', quick: true }), pnpmScript('subsystem-pages', 'verify-subsystem-pages', { label: 'subsystem pages' }), diff --git a/scripts/session-fixture-layout.spec.ts b/scripts/session-fixture-layout.spec.ts index ca1fdfbf4f..1c572be072 100644 --- a/scripts/session-fixture-layout.spec.ts +++ b/scripts/session-fixture-layout.spec.ts @@ -14,7 +14,7 @@ const root = resolve(import.meta.dirname, '..') function chunkRun(): SessionEvent[] { return Array.from({ length: 4 }, (_, index) => ({ type: 'assistant/chunk', - seq: SessionSeq(index), + seq: SessionSeq(index + 2), time: 10 + index, data: { turn: 1, @@ -24,8 +24,16 @@ function chunkRun(): SessionEvent[] { })) } +function fixtureEvents(): SessionEvent[] { + return [ + { type: 'turn/start', seq: SessionSeq(0), time: 1, data: { turn: 1 } }, + { type: 'step/start', seq: SessionSeq(1), time: 2, data: { turn: 1, step: 1 } }, + ...chunkRun(), + ] +} + function unpackedFixture(): string { - return [HEADER, ...chunkRun().map(event => JSON.stringify(event)), ''].join('\n') + return [HEADER, ...fixtureEvents().map(event => JSON.stringify(event)), ''].join('\n') } function decodedBody(content: string): SessionEvent[] { @@ -37,12 +45,14 @@ describe('canonicalSessionFixture', () => { const canonical = canonicalSessionFixture(unpackedFixture(), 'fixture.jsonl') expect(canonical).toBeDefined() expect(canonical?.split('\n')[0]).toBe(HEADER) - const packed = JSON.parse(canonical?.split('\n')[1] ?? '{}') as Record + const packed = canonical?.split('\n') + .map(line => JSON.parse(line || '{}') as Record) + .find(record => record.type === 'text-chunks') expect(packed).toMatchObject({ type: 'text-chunks' }) expect(packed).not.toHaveProperty('seq0') expect(packed).not.toHaveProperty('time0') expect(decodedBody(canonical ?? '').map(({ seq: _seq, time: _time, ...event }) => event)) - .toStrictEqual(chunkRun().map(({ seq: _seq, time: _time, ...event }) => event)) + .toStrictEqual(fixtureEvents().map(({ seq: _seq, time: _time, ...event }) => event)) }) it('ignores JSONL whose first record is not a session header', () => { @@ -71,7 +81,7 @@ describe('canonicalSessionFixture', () => { it('labels malformed packed rows with the fixture path and line', () => { expect(() => canonicalSessionFixture(`${HEADER}\n{"type":"text-chunks"}\n`, 'broken.jsonl')) - .toThrow(/broken\.jsonl: session snapshot line 2: malformed text-chunks storage row/) + .toThrow(/broken\.jsonl: session snapshot line 2: released text-chunks row 0 lacks required member "data"/) }) }) @@ -80,9 +90,15 @@ describe('isPhysicalSessionFixture', () => { expect(isPhysicalSessionFixture( 'packages/experimental/webworker-runtime/tests/fixtures/vfs-example/home/sessions/--dsh-workspace--/main/session.jsonl', )).toBe(true) + expect(isPhysicalSessionFixture( + 'packages/experimental/webworker-runtime/tests/fixtures/vfs-example/home/sessions/--dsh-workspace--/main/session.v1.jsonl', + )).toBe(true) expect(isPhysicalSessionFixture( 'scripts/snapshots/python-sdk-single-exe/advanced/session.1.jsonl', )).toBe(true) + expect(isPhysicalSessionFixture( + 'scripts/snapshots/python-sdk-single-exe/advanced/session.1.v1.jsonl', + )).toBe(true) expect(isPhysicalSessionFixture( 'scripts/snapshots/python-sdk-single-exe/advanced/session.jsonl', )).toBe(true) diff --git a/scripts/session-fixture-layout.ts b/scripts/session-fixture-layout.ts index b26935ccec..d3f763ee86 100644 --- a/scripts/session-fixture-layout.ts +++ b/scripts/session-fixture-layout.ts @@ -4,8 +4,15 @@ import { deepStrictEqual } from 'node:assert' import { execFileSync } from 'node:child_process' import { existsSync, readFileSync } from 'node:fs' import { resolve } from 'node:path' -import { packChunkRuns, type SessionEvent } from '@deepseek-ai/dsh-session' -import { parseSessionLog } from '@deepseek-ai/dsh-llm-replay' +import { + decodeSeqRanges, + decodeStorageRecord, + packChunkRuns, + SessionLogOffset, + type SessionEvent, +} from '@deepseek-ai/dsh-session' +import type { SessionLogOffset as SessionLogOffsetType } from '@deepseek-ai/dsh-session' +import { sessionFormatCatalog } from '@deepseek-ai/dsh-session-format-catalog' /** Physical persistence artifacts validated by the WebWorker runtime fixture spec. */ const WEBWORKER_PHYSICAL_SESSION_FIXTURE_ROOT = @@ -33,21 +40,31 @@ export interface SessionFixtureLayout { */ export function isPhysicalSessionFixture(path: string): boolean { if (path.startsWith(WEBWORKER_PHYSICAL_SESSION_FIXTURE_ROOT)) { - return path.endsWith('/session.jsonl') + return /\/session(?:\.v[1-9]\d*)?\.jsonl$/.test(path) } return path.startsWith(PYTHON_RUNTIME_PHYSICAL_SESSION_FIXTURE_ROOT) - && /\/session(?:\.\d+)?\.jsonl$/.test(path) + && /\/session(?:\.[1-9]\d*)?(?:\.v[1-9]\d*)?\.jsonl$/.test(path) } function isSessionHeader(value: unknown): boolean { return value !== null && typeof value === 'object' && (value as { type?: unknown }).type === 'session' } +function validationHeader(value: unknown): unknown { + if (value === null || typeof value !== 'object' || Array.isArray(value)) return value + const header = { ...value as Record } + if (header.version === 0 && !Object.hasOwn(header, 'delegationDepth')) header.delegationDepth = 0 + if (typeof header.cwd === 'string' && /^\{\{cwd\}\}(?:\/|$)/.test(header.cwd)) { + header.cwd = header.cwd.replace('{{cwd}}', '/dsh-snapshot-cwd') + } + return header +} + function renderFixture(headerLine: string, events: readonly SessionEvent[]): string { return [ headerLine, ...packChunkRuns(events).map((stored) => { - const record = stored as unknown as Record + const record = { ...stored } as unknown as Record delete record.seq delete record.time delete record.seq0 @@ -58,6 +75,79 @@ function renderFixture(headerLine: string, events: readonly SessionEvent[]): str ].join('\n') } +function projectedRowCardinality(record: Readonly>): number { + const data = record.data + if (data === null || typeof data !== 'object' || Array.isArray(data)) return 1 + const key = record.type === 'tool-call-chunks' ? 'args' : 'texts' + const values = (data as Record)[key] + return Array.isArray(values) && values.length > 0 ? values.length : 1 +} + +function parseFixtureObjectLine(line: string, lineNumber: number): Record { + let value: unknown + try { + value = JSON.parse(line) as unknown + } catch (error) { + throw new Error(`session snapshot line ${lineNumber} contains invalid JSON`, { cause: error }) + } + if (value === null || typeof value !== 'object' || Array.isArray(value)) { + throw new Error(`session snapshot line ${lineNumber} must be a JSON object`) + } + return value as Record +} + +function parseFixtureRows(content: string, headerValue: unknown): SessionEvent[] { + const rows: Record[] = [] + const rowLines: number[] = [] + let nextSeq: SessionLogOffsetType = SessionLogOffset(0) + let headerSkipped = false + for (const [index, line] of content.split(/\r?\n/).entries()) { + if (line.trim().length === 0) continue + if (!headerSkipped) { + headerSkipped = true + continue + } + const record = parseFixtureObjectLine(line, index + 1) + const packed = record.type === 'text-chunks' + || record.type === 'reasoning-chunks' + || record.type === 'tool-call-chunks' + const seqKey = packed ? 'seq0' : 'seq' + const timeKey = packed ? 'time0' : 'time' + if (!Object.hasOwn(record, seqKey)) record[seqKey] = nextSeq + if (!Object.hasOwn(record, timeKey)) record[timeKey] = 0 + rows.push(record) + rowLines.push(index + 1) + nextSeq = SessionLogOffset(nextSeq + projectedRowCardinality(record)) + } + // Versionless protocol fixtures are outside the released format catalog; + // they exercise only the current storage-row projection. + if (headerValue === null || typeof headerValue !== 'object' || Array.isArray(headerValue) + || !Object.hasOwn(headerValue, 'version')) { + return rows.flatMap((source, index) => { + const record = { ...source } + try { + if (Object.hasOwn(record, 'sourceEventSeqs')) { + record.sourceEventSeqs = decodeSeqRanges(record.sourceEventSeqs) + } + return decodeStorageRecord(record) + } catch (error) { + const detail = error instanceof Error ? error.message : String(error) + throw new Error(`session snapshot line ${rowLines[index] ?? 1}: ${detail}`, { cause: error }) + } + }) + } + try { + return [ + ...sessionFormatCatalog.decodeArtifact(validationHeader(headerValue), rows).events, + ] as unknown as SessionEvent[] + } catch (error) { + const detail = error instanceof Error ? error.message : String(error) + const storedRow = /\brow (\d+)\b/.exec(detail) + const line = storedRow === null ? 1 : rowLines[Number(storedRow[1])] ?? 1 + throw new Error(`session snapshot line ${line}: ${detail}`, { cause: error }) + } +} + function withoutEnvelope(events: readonly SessionEvent[]): Array> { return events.map((event) => { const { seq: _seq, time: _time, ...projected } = event @@ -89,13 +179,13 @@ export function canonicalSessionFixture(content: string, label = ' parseSessionFixtureName(name) !== undefined) if (manifest.session === undefined) { - expect(existsSync(localSession), `${key}: owner session.jsonl`).toBe(true) + expect(localSessionNames.length, `${key}: owner Session fixture`).toBeGreaterThan(0) } else { - expect(existsSync(localSession), `${key}: borrower must not own session.jsonl`).toBe(false) + expect(localSessionNames, `${key}: borrower must not own a Session fixture`).toEqual([]) const target = resolve(dir, manifest.session.source) expect(existsSync(target), `${key}: session source`).toBe(true) const targetDir = await realpath(dirname(target)) @@ -161,8 +164,11 @@ it('keeps every recorded session owned, pinned, redacted, and header-scrubbed', } if (manifest.session !== undefined) continue - const names = sessionFixtureNames(await readdir(dir)) + const names = sessionFixtureNames(localEntries) const fixtures = await Promise.all(names.map(name => readFile(join(dir, name), 'utf8'))) + for (const name of localSessionNames) { + assertSessionFixtureVersion(name, await readFile(join(dir, name), 'utf8')) + } expect(redactSessionSnapshotIds(fixtures), `${key}: typed identity fixed point`).toEqual(fixtures) for (const [index, fixture] of fixtures.entries()) { expect(scrubSystemPrompts(fixture), `${key}/${names[index]}: system prompt must be a sidecar`).toBe(fixture) diff --git a/scripts/smoke-python-runtime.py b/scripts/smoke-python-runtime.py index 2e02a8527c..5a3bf2ff80 100644 --- a/scripts/smoke-python-runtime.py +++ b/scripts/smoke-python-runtime.py @@ -10,6 +10,7 @@ import importlib.metadata import json import os import queue +import re import subprocess import sys import sysconfig @@ -112,7 +113,9 @@ SNAPSHOT_WORKFLOW_SCRIPT = ( ADVANCED_SNAPSHOT_DIRECTORY = ( Path(__file__).resolve().parent / "snapshots" / "python-sdk-single-exe" / "advanced" ) -ADVANCED_SNAPSHOT_FILENAMES = ("result.json", "session.jsonl", "session.1.jsonl", "session.2.jsonl") +ADVANCED_SNAPSHOT_FILENAMES = ( + "result.json", "session.v1.jsonl", "session.1.v1.jsonl", "session.2.v1.jsonl", +) MINIMAL_SNAPSHOT_DIRECTORY = ( Path(__file__).resolve().parent / "snapshots" / "python-sdk-single-exe" / "minimal" ) @@ -122,7 +125,9 @@ MINIMAL_SNAPSHOT_FILENAMES = ("model-visible.json",) RESTART_SNAPSHOT_DIRECTORY = ( Path(__file__).resolve().parent / "snapshots" / "python-sdk-single-exe" / "restart" ) -RESTART_SNAPSHOT_FILENAMES = ("result.json", "requests.json", "session.1.jsonl", "session.2.jsonl") +RESTART_SNAPSHOT_FILENAMES = ( + "result.json", "requests.json", "session.1.v1.jsonl", "session.2.v1.jsonl", +) MCP_SERVER_SCRIPT = """\ import json import os @@ -1393,11 +1398,108 @@ class RuntimePeer: self.stderr.extend(self.process.stderr) +PERSISTED_SESSION_FILENAME = re.compile(r"^session(?:\.v([1-9]\d*))?\.jsonl(\.zstd)?$") +SNAPSHOT_SESSION_FILENAME = re.compile( + r"^session(?:\.([1-9]\d*))?(?:\.v([1-9]\d*))?\.jsonl$", +) + + +def persisted_session_filename_version(path: Path, compressed: bool = False) -> int | None: + """Return one canonical persistence basename's generation for the selected encoding.""" + match = PERSISTED_SESSION_FILENAME.fullmatch(path.name) + if match is None or (match.group(2) is not None) != compressed: + return None + return int(match.group(1) or 0) + + +def latest_persisted_session_paths(sessions: Path, compressed: bool = False) -> list[Path]: + """Select the numeric-highest immutable generation in each physical Session directory.""" + pattern = "*.jsonl.zstd" if compressed else "*.jsonl" + selected: dict[Path, tuple[int, Path]] = {} + for path in sessions.rglob(pattern): + version = persisted_session_filename_version(path, compressed) + if version is None: + continue + previous = selected.get(path.parent) + if previous is None or version > previous[0]: + selected[path.parent] = (version, path) + return sorted((entry[1] for entry in selected.values()), key=lambda path: str(path)) + + +def session_header_version(content: str, label: str) -> int: + """Read a non-negative physical Session generation from the first JSONL record.""" + first = next((line for line in content.splitlines() if line), None) + if first is None: + raise AssertionError(f"{label}: Session log is empty") + header = json.loads(first) + version = header.get("version") if isinstance(header, dict) and header.get("type") == "session" else None + if not isinstance(version, int) or isinstance(version, bool) or version < 0: + raise AssertionError(f"{label}: Session header has no non-negative integer version") + return version + + +def assert_persisted_session_version(path: Path, content: str) -> int: + """Require a raw persistence basename and header to name the same generation.""" + filename_version = persisted_session_filename_version(path) + if filename_version is None: + raise AssertionError(f"non-canonical Session persistence filename: {path.name}") + header_version = session_header_version(content, path.name) + if filename_version != header_version: + raise AssertionError( + f"{path.name}: filename declares Session format v{filename_version}, " + f"header declares v{header_version}", + ) + return header_version + + +def snapshot_session_filename(index: int, version: int) -> str: + """Render parent/ordinal snapshot role plus an omitted-v0 generation.""" + if index < 0 or version < 0: + raise ValueError("snapshot Session index and version must be non-negative") + ordinal = "" if index == 0 else f".{index}" + generation = "" if version == 0 else f".v{version}" + return f"session{ordinal}{generation}.jsonl" + + +def parse_snapshot_session_filename(name: str) -> tuple[int, int] | None: + """Parse one canonical parent/ordinal snapshot filename.""" + match = SNAPSHOT_SESSION_FILENAME.fullmatch(name) + if match is None: + if name.startswith("session") and name.endswith(".jsonl"): + raise AssertionError(f"invalid snapshot Session filename: {name}") + return None + return int(match.group(1) or 0), int(match.group(2) or 0) + + +def selected_snapshot_session_files(directory: Path) -> dict[int, Path]: + """Select one highest-generation expected file per parent/ordinal role.""" + selected: dict[int, tuple[int, Path]] = {} + for path in directory.iterdir(): + if not path.is_file(): + continue + parsed = parse_snapshot_session_filename(path.name) + if parsed is None: + continue + index, version = parsed + content = path.read_text(encoding="utf-8") + header_version = session_header_version(content, path.name) + if header_version != version: + raise AssertionError( + f"{path.name}: filename declares Session format v{version}, header declares v{header_version}", + ) + previous = selected.get(index) + if previous is None or version > previous[0]: + selected[index] = (version, path) + return {index: value[1] for index, value in selected.items()} + + def assert_session_log(sessions: Path, cwd: Path, *expected_texts: str) -> None: - logs = list(sessions.rglob("*.jsonl")) + logs = latest_persisted_session_paths(sessions) if len(logs) != 1: raise AssertionError(f"expected one JSONL session log under {sessions}, found {logs}") - lines = logs[0].read_text().splitlines() + content = logs[0].read_text() + assert_persisted_session_version(logs[0], content) + lines = content.splitlines() header = json.loads(lines[0]) if header.get("cwd") != str(cwd): raise AssertionError(f"session header cwd is not absolute/canonical: {header}") @@ -1408,7 +1510,7 @@ def assert_session_log(sessions: Path, cwd: Path, *expected_texts: str) -> None: def assert_zstd_session_log(sessions: Path) -> None: - logs = list(sessions.rglob("*.jsonl.zstd")) + logs = latest_persisted_session_paths(sessions, compressed=True) if len(logs) != 1: raise AssertionError(f"expected one Zstandard JSONL session log under {sessions}, found {logs}") if not logs[0].read_bytes().startswith(bytes.fromhex("28b52ffd")): @@ -1418,10 +1520,12 @@ def assert_zstd_session_log(sessions: Path) -> None: def read_session_logs(sessions: Path) -> dict[str, list[dict[str, object]]]: """Parse every persisted JSONL session into a map keyed by header id.""" logs: dict[str, list[dict[str, object]]] = {} - for path in sorted(sessions.rglob("*.jsonl")): + for path in latest_persisted_session_paths(sessions): + content = path.read_text(encoding="utf-8") + assert_persisted_session_version(path, content) records = [ json.loads(line) - for line in path.read_text(encoding="utf-8").splitlines() + for line in content.splitlines() if line ] if not records or records[0].get("type") != "session": @@ -1540,20 +1644,23 @@ def build_snapshot_files( ], } normalized_result = normalize_snapshot_value(result_value, replacements) + parent_records = project_session_snapshot([ + normalize_snapshot_value(record, replacements) for record in logs[SNAPSHOT_SESSION_ID] + ]) files = { "result.json": json.dumps(normalized_result, indent=2, ensure_ascii=False) + "\n", - "session.jsonl": render_jsonl( - project_session_snapshot([ - normalize_snapshot_value(record, replacements) for record in logs[SNAPSHOT_SESSION_ID] - ]) - ), + snapshot_session_filename( + 0, session_header_version(render_jsonl(parent_records), "advanced parent"), + ): render_jsonl(parent_records), } for index, child_id in enumerate(child_ids, start=1): - files[f"session.{index}.jsonl"] = render_jsonl( - project_session_snapshot([ - normalize_snapshot_value(record, replacements) for record in logs[child_id] - ]) - ) + child_records = project_session_snapshot([ + normalize_snapshot_value(record, replacements) for record in logs[child_id] + ]) + child_content = render_jsonl(child_records) + files[snapshot_session_filename( + index, session_header_version(child_content, f"advanced child {index}"), + )] = child_content return files @@ -1590,6 +1697,14 @@ def build_restart_snapshot_files( } for request in requests ] + first_records = project_session_snapshot([ + normalize_snapshot_value(record, replacements) for record in logs[RESTART_FIRST_SESSION_ID] + ]) + second_records = project_session_snapshot([ + normalize_snapshot_value(record, replacements) for record in logs[RESTART_SECOND_SESSION_ID] + ]) + first_content = render_jsonl(first_records) + second_content = render_jsonl(second_records) return { "result.json": json.dumps( normalize_snapshot_value(result_value, replacements), indent=2, ensure_ascii=False, @@ -1597,12 +1712,12 @@ def build_restart_snapshot_files( "requests.json": json.dumps( normalize_snapshot_value(request_value, replacements), indent=2, ensure_ascii=False, ) + "\n", - "session.1.jsonl": render_jsonl(project_session_snapshot([ - normalize_snapshot_value(record, replacements) for record in logs[RESTART_FIRST_SESSION_ID] - ])), - "session.2.jsonl": render_jsonl(project_session_snapshot([ - normalize_snapshot_value(record, replacements) for record in logs[RESTART_SECOND_SESSION_ID] - ])), + snapshot_session_filename( + 1, session_header_version(first_content, "restart Session 1"), + ): first_content, + snapshot_session_filename( + 2, session_header_version(second_content, "restart Session 2"), + ): second_content, } @@ -1715,6 +1830,55 @@ def project_session_snapshot(records: list[dict[str, object]]) -> list[dict[str, return projected +SESSION_FORMAT_PROVENANCE = "{{sessionFormatVersion}}" + + +def normalize_session_format_comparison(value: object) -> object: + """Canonicalize only generation provenance that differs between v0 fixtures and fresh v1 runs.""" + if isinstance(value, list): + return [normalize_session_format_comparison(item) for item in value] + if not isinstance(value, dict): + return value + + normalized = { + key: normalize_session_format_comparison(item) + for key, item in value.items() + } + if normalized.get("type") == "session" and "version" in normalized: + normalized["version"] = SESSION_FORMAT_PROVENANCE + if normalized.get("type") == "session-log-deepseek/delivery-accepted": + data = normalized.get("data") + if isinstance(data, dict): + data.pop("sessionFormatVersion", None) + data["sessionFormatVersion"] = SESSION_FORMAT_PROVENANCE + if normalized.get("kind") == "session-reference": + references = normalized.get("references") + if isinstance(references, list): + for reference in references: + if isinstance(reference, dict): + reference.pop("capturedFormatVersion", None) + reference["capturedFormatVersion"] = SESSION_FORMAT_PROVENANCE + return normalized + + +def normalize_snapshot_comparison_text(name: str, content: str) -> str: + """Normalize Session generation provenance only while comparing committed expected outputs.""" + if name.startswith("session") and name.endswith(".jsonl"): + records = [ + normalize_session_format_comparison(json.loads(line)) + for line in content.splitlines() + if line + ] + return render_jsonl(records) + if name.endswith(".json"): + return json.dumps( + normalize_session_format_comparison(json.loads(content)), + indent=2, + ensure_ascii=False, + ) + "\n" + return content + + def compare_snapshot_files( files: dict[str, str], update: bool, @@ -1731,25 +1895,52 @@ def compare_snapshot_files( (directory / name).write_text(content, encoding="utf-8", newline="\n") print(f"smoke-python-runtime: updated snapshots in {directory}") - existing = { - path.name - for path in directory.iterdir() - if path.is_file() - } if directory.is_dir() else set() - expected = set(filenames) - if existing != expected: + existing = [path for path in directory.iterdir() if path.is_file()] if directory.is_dir() else [] + expected_non_session = { + name for name in filenames if parse_snapshot_session_filename(name) is None + } + existing_non_session = { + path.name for path in existing if parse_snapshot_session_filename(path.name) is None + } + if existing_non_session != expected_non_session: raise AssertionError( f"{scenario} snapshot files differ: " - f"missing={sorted(expected - existing)}, unexpected={sorted(existing - expected)}" + f"missing={sorted(expected_non_session - existing_non_session)}, " + f"unexpected={sorted(existing_non_session - expected_non_session)}" + ) + selected_expected = selected_snapshot_session_files(directory) + actual_sessions: dict[int, tuple[str, str]] = {} + for name, content in files.items(): + parsed = parse_snapshot_session_filename(name) + if parsed is None: + continue + index, filename_version = parsed + header_version = session_header_version(content, name) + if filename_version != header_version: + raise AssertionError( + f"{name}: filename declares Session format v{filename_version}, " + f"header declares v{header_version}", + ) + if index in actual_sessions: + raise AssertionError(f"{scenario} snapshot builder produced duplicate Session role {index}") + actual_sessions[index] = (name, content) + if set(selected_expected) != set(actual_sessions): + raise AssertionError( + f"{scenario} snapshot Session roles differ: " + f"expected={sorted(selected_expected)}, actual={sorted(actual_sessions)}", ) for name, actual in files.items(): - expected_text = (directory / name).read_text(encoding="utf-8") - if actual == expected_text: + parsed = parse_snapshot_session_filename(name) + expected_path = directory / name if parsed is None else selected_expected[parsed[0]] + expected_text = expected_path.read_text(encoding="utf-8") + compared_actual = normalize_snapshot_comparison_text(name, actual) + compared_expected = normalize_snapshot_comparison_text(expected_path.name, expected_text) + if compared_actual == compared_expected: continue diff = "".join(difflib.unified_diff( - expected_text.splitlines(keepends=True), - actual.splitlines(keepends=True), - fromfile=f"expected/{name}", + compared_expected.splitlines(keepends=True), + compared_actual.splitlines(keepends=True), + fromfile=f"expected/{expected_path.name}", tofile=f"actual/{name}", )) raise AssertionError( diff --git a/scripts/type-equiv.manifest.json b/scripts/type-equiv.manifest.json index 7def55bbee..a6550621a3 100644 --- a/scripts/type-equiv.manifest.json +++ b/scripts/type-equiv.manifest.json @@ -751,6 +751,11 @@ "symbol": "SessionReferenceMentionCandidate", "source": "packages/context/session-reference/src/types.ts" }, + { + "doc": "docs/subsystems/session-reference.md", + "symbol": "SessionReferenceSource", + "source": "packages/context/session-reference/src/types.ts" + }, { "doc": "docs/subsystems/session-reference.md", "symbol": "PreparedReferencedMessage", @@ -1521,6 +1526,11 @@ "symbol": "SessionPersistenceRevision", "source": "packages/session/session-persistence/src/revision.ts" }, + { + "doc": "docs/subsystems/persistence.md", + "symbol": "SessionPersistenceListing", + "source": "packages/session/session-persistence/src/index.ts" + }, { "doc": "docs/subsystems/persistence.md", "symbol": "SessionPersistenceSnapshot", diff --git a/snapshots/AGENTS.md b/snapshots/AGENTS.md index cbcc63b547..a6596cd362 100644 --- a/snapshots/AGENTS.md +++ b/snapshots/AGENTS.md @@ -4,7 +4,7 @@ This tree contains only tests whose committed session JSONL is replay input and Every process under test starts through the `dsh` CLI with a shipped profile and optional scenario patches. Test clients may drive a public protocol or browser interface; do not add another application entrypoint, hidden CLI mode, or executable scenario driver. -Each scenario owns or explicitly references one primary `session.jsonl` plus contiguous child files. The owner alone records or refreshes it. For an ordinary one-shot case, derive the user task and replay script from that JSONL; do not duplicate them in an `input.json`. Shared references are read-only, acyclic, and used only when another interface intentionally renders the same recorded behavior. +Each scenario owns or explicitly references one primary Session role plus contiguous child roles. Canonical parent filenames are `session[.vN].jsonl`; children are `session.[.vN].jsonl`; v0 omits `.v0`, positive versions use lowercase `.vN`, and every filename agrees with its header. A directory may retain several generations of one role, but replay, record, and refresh select the numerically highest. The owner alone records or refreshes it. For an ordinary one-shot case, derive the user task and replay script from that selected JSONL; do not duplicate them in an `input.json`. Shared references are read-only, acyclic, and used only when another interface intentionally renders the same recorded behavior. Committed sessions are normalization fixed points. Replace volatile identities with typed relationship-preserving tokens, replace request system prompts and tool schemas with tokens, and keep exactly one readable sidecar owner per header class. Never redact arbitrary user or tool text merely because it resembles an identifier. diff --git a/snapshots/sdk/sdk.snapshot.ts b/snapshots/sdk/sdk.snapshot.ts index 50ececf76a..89ef393f20 100644 --- a/snapshots/sdk/sdk.snapshot.ts +++ b/snapshots/sdk/sdk.snapshot.ts @@ -16,8 +16,11 @@ import { basename, delimiter, join } from 'node:path' import { fileURLToPath } from 'node:url' import { describe, expect, it } from 'vitest' import { + assertPersistedSessionVersion, + assertSessionFixtureVersion, captureExpectedWorkspaceSnapshot, captureWorkspaceSnapshot, + normalizeSessionFormatProvenance, normalizeSessionLog, normalizeSessionSnapshots, normalizeStdout, @@ -32,13 +35,17 @@ import { scrubRequestHeaders, scrubSessionSnapshot, scrubSystemPrompts, + sessionFixtureName, sessionFixtureNames, + sessionHeaderVersion, stabilizeFixtureMessageIds, stabilizeRefreshLog, tokenizeSessionFixtureCwd, materializeProfilePatch, formatSystemPromptSnapshot, formatToolSchemasSnapshot, + latestPersistedSessionPaths, + parseSessionFixtureName, type HarvestedLog, type NormalizeContext, type SnapshotManifest, @@ -218,13 +225,14 @@ interface PersistedLog { async function jsonlFiles(dir: string): Promise { const entries = await readdir(dir, { recursive: true }) - return entries.filter(entry => entry.endsWith('.jsonl')).map(entry => join(dir, entry)).sort() + return latestPersistedSessionPaths(entries).map(entry => join(dir, entry)) } async function persistedLogs(sessionsRoot: string): Promise { const files = await jsonlFiles(sessionsRoot) return Promise.all(files.map(async (path) => { const content = await readFile(path, 'utf8') + assertPersistedSessionVersion(basename(path), content) const header = JSON.parse(content.slice(0, content.indexOf('\n'))) as Record return { path, content, header } })) @@ -300,7 +308,11 @@ function contextOfContents(contents: readonly string[]): NormalizeContext { async function fixtureFiles(scenario: CorpusScenario): Promise { const names = sessionFixtureNames(await readdir(scenario.dir)) - return names.map(name => join(scenario.dir, name)) + return Promise.all(names.map(async (name) => { + const path = join(scenario.dir, name) + assertSessionFixtureVersion(name, await readFile(path, 'utf8')) + return path + })) } async function hydrateReplayFixtures(scenario: CorpusScenario, cwd: string): Promise { @@ -325,7 +337,7 @@ function normalizeNotifications(notifications: readonly HarnessNotification[], c const normalizedEvents = events.length === 0 ? [] : scrubRequestHeaders(normalizeSessionLog( - `${events.map(event => JSON.stringify(event)).join('\n')}\n`, + normalizeSessionFormatProvenance(`${events.map(event => JSON.stringify(event)).join('\n')}\n`), ctx, )).trimEnd().split('\n').map(line => JSON.parse(line) as Record) let eventIndex = 0 @@ -675,7 +687,8 @@ async function verifyHeaders( dshSdkChildConfig?: Readonly>, ): Promise { const pin = headerPin(scenario) - const pinFixture = await readFile(join(pin.dir, 'session.jsonl'), 'utf8') + const [pinFixturePath] = await fixtureFiles(pin) + const pinFixture = await readFile(pinFixturePath as string, 'utf8') const firstLine = pinFixture.split('\n').find(line => line.trim() !== '') ?? '{}' const pinHeader = JSON.parse(firstLine) as JsonObject const pinned = normalizedHeaders(pinFixture, { @@ -731,7 +744,7 @@ describe('TypeScript SDK snapshots over the jsonrpc runtime', () => { const hasWireGoldens = existsSync(notificationsExpectedPath) || existsSync(resultExpectedPath) const assertions = SDK_ASSERTIONS[scenario.name] ?? {} - const files = await fixtureFiles(scenario) + let files = await fixtureFiles(scenario) const { results, notifications, observedMethods, logs, initialWorkspace, finalWorkspace, cwd } = await runScenario(scenario) const ordered = orderLogs( logs, @@ -768,19 +781,20 @@ describe('TypeScript SDK snapshots over the jsonrpc runtime', () => { } if (recording || refreshing) { - const outputFiles = [ - join(scenarioDir, 'session.jsonl'), - ...Array.from({ length: expectedContents.length - 1 }, (_, index) => join(scenarioDir, `session.${index + 1}.jsonl`)), - ] + const outputFiles = ordered.map((log, index) => join(scenarioDir, sessionFixtureName( + index, + sessionHeaderVersion(log.content, `harvested Session ${index}`), + ))) await Promise.all(expectedContents.map((stable, index) => writeFile(outputFiles[index] as string, stable))) if (recording) { const retained = new Set(outputFiles.map(file => basename(file))) for (const entry of await readdir(scenarioDir, { withFileTypes: true })) { - if (entry.isFile() && /^session\.[1-9]\d*\.jsonl$/u.test(entry.name) && !retained.has(entry.name)) { - await rm(join(scenarioDir, entry.name)) - } + if (!entry.isFile() || retained.has(entry.name)) continue + const fixture = parseSessionFixtureName(entry.name) + if (fixture !== undefined && fixture.index >= outputFiles.length) await rm(join(scenarioDir, entry.name)) } } + files = outputFiles await writeHeaderSidecars(scenario, ordered, actualContext) } @@ -794,7 +808,9 @@ describe('TypeScript SDK snapshots over the jsonrpc runtime', () => { // Persisted transcripts match the committed fixtures. const expectedContext = contextOfContents(expectedContents) const actualSnapshots = normalizeSessionSnapshots(ordered.map(log => log.content), actualContext) - const expectedSnapshots = normalizeSessionSnapshots(expectedContents, expectedContext) + const expectedSnapshots = normalizeSessionSnapshots(expectedContents, expectedContext, { + sourcePaths: files, + }) for (const [index, actual] of actualSnapshots.entries()) { expect(actual, `${scenario.name}: session ${index}`).toBe(expectedSnapshots[index]) } diff --git a/snapshots/session/headless.snapshot.ts b/snapshots/session/headless.snapshot.ts index 5da4cb02e9..91f70842c7 100644 --- a/snapshots/session/headless.snapshot.ts +++ b/snapshots/session/headless.snapshot.ts @@ -9,17 +9,21 @@ import { fileURLToPath } from 'node:url' import { describe, expect, it } from 'vitest' import ts from 'typescript' import { + assertPersistedSessionVersion, + assertSessionFixtureVersion, captureExpectedWorkspaceSnapshot, captureWorkspaceSnapshot, fixtureContext, formatSystemPromptSnapshot, formatToolSchemasSnapshot, + latestPersistedSessionPaths, materializeProfilePatch, normalizeSessionSnapshots, normalizedHeaders, normalizedSystemPrompts, normalizedToolSchemas, parseSnapshotManifest, + parseSessionFixtureName, parseToolSchemasSnapshot, redactSessionSnapshotIds, refreshFixtureReplacements, @@ -27,7 +31,9 @@ import { scrubSessionSnapshot, scrubSystemPrompts, scrubToolSchemas, + sessionFixtureName, sessionFixtureNames, + sessionHeaderVersion, snapshotSpillRoot, stabilizeFixtureMessageIds, stabilizeRefreshLog, @@ -157,10 +163,10 @@ function contextOf(logs: readonly string[]): NormalizeContext { async function persistedSessions(cwd: string): Promise { const root = join(cwd, '.dsh', 'sessions') - const files = (await readdir(root, { recursive: true })) - .filter(file => file.endsWith('session.jsonl')) + const files = latestPersistedSessionPaths(await readdir(root, { recursive: true })) const logs = await Promise.all(files.map(async (file): Promise => { const content = await readFile(join(root, file), 'utf8') + assertPersistedSessionVersion(basename(file), content) return { content, header: headerOf(content) } })) return logs.sort((left, right) => { @@ -173,7 +179,19 @@ async function persistedSessions(cwd: string): Promise { async function fixtureSessions(scenario: HeadlessScenario): Promise { const files = sessionFixtureNames(await readdir(scenario.dir)) - return Promise.all(files.map(file => readFile(join(scenario.dir, file), 'utf8'))) + return Promise.all(files.map(async (file) => { + const content = await readFile(join(scenario.dir, file), 'utf8') + assertSessionFixtureVersion(file, content) + return content + })) +} + +async function primaryFixtureFile(dir: string): Promise { + const [primary] = sessionFixtureNames(await readdir(dir)) + if (primary === undefined) throw new Error(`${dir}: missing parent Session fixture`) + const content = await readFile(join(dir, primary), 'utf8') + assertSessionFixtureVersion(primary, content) + return primary } async function writeSessionFixtures( @@ -182,10 +200,10 @@ async function writeSessionFixtures( existing: readonly string[], ctx: NormalizeContext, ): Promise { - const names = [ - 'session.jsonl', - ...Array.from({ length: actualLogs.length - 1 }, (_, index) => `session.${index + 1}.jsonl`), - ] + const names = actualLogs.map((log, index) => sessionFixtureName( + index, + sessionHeaderVersion(log.content, `harvested Session ${index}`), + )) const prior = names.map((_, index) => existing[index] ?? '') const replacements = mode === 'refresh' ? refreshFixtureReplacements(actualLogs.map(harvested), prior) @@ -201,7 +219,8 @@ async function writeSessionFixtures( if (mode === 'record') { const retained = new Set(names) for (const entry of await readdir(scenario.dir, { withFileTypes: true })) { - if (entry.isFile() && /^session\.[1-9]\d*\.jsonl$/.test(entry.name) && !retained.has(entry.name)) { + const fixture = entry.isFile() ? parseSessionFixtureName(entry.name) : undefined + if (fixture !== undefined && fixture.index >= names.length && !retained.has(entry.name)) { await rm(join(scenario.dir, entry.name)) } } @@ -495,7 +514,7 @@ function pinOf(scenario: HeadlessScenario): HeadlessScenario { async function verifyHeaders(scenario: HeadlessScenario, actualLogs: readonly SessionLog[], ctx: NormalizeContext): Promise { const pin = pinOf(scenario) - const fixture = await readFile(join(pin.dir, 'session.jsonl'), 'utf8') + const fixture = await readFile(join(pin.dir, await primaryFixtureFile(pin.dir)), 'utf8') const pinned = normalizedHeaders(fixture, fixtureContext(fixture)) const changes = pin.manifest.header.changes ?? 0 expect(pinned, `${scenario.name}: pin header count`).toHaveLength(1 + changes) @@ -588,8 +607,10 @@ describe('headless recorded-session snapshots', () => { }) it('keeps packed chunk rows logically equal to their unpacked recording', async () => { - const source = await readFile(join(snapshotsRoot, 'hook-cc-pretool-deny', 'session.jsonl'), 'utf8') - const packed = await readFile(join(snapshotsRoot, 'packed-chunks', 'session.jsonl'), 'utf8') + const sourceDir = join(snapshotsRoot, 'hook-cc-pretool-deny') + const packedDir = join(snapshotsRoot, 'packed-chunks') + const source = await readFile(join(sourceDir, await primaryFixtureFile(sourceDir)), 'utf8') + const packed = await readFile(join(packedDir, await primaryFixtureFile(packedDir)), 'utf8') const rowTypes = records(packed).flatMap((record) => { const type = record.type return type === 'text-chunks' || type === 'reasoning-chunks' || type === 'tool-call-chunks' ? [type] : [] @@ -656,12 +677,12 @@ describe('headless recorded-session snapshots', () => { try { model = modelFromSession(primaryFixture) } catch { - model = modelFromSession(await readFile(join(pin.dir, 'session.jsonl'), 'utf8')) + model = modelFromSession(await readFile(join(pin.dir, await primaryFixtureFile(pin.dir)), 'utf8')) } const composition = ownerOf(scenario) const baseComposition = compositionOwners.get('default') if (baseComposition === undefined) throw new Error('headless corpus has no default composition') - const fixtureFiles = sessionFixtureNames(await readdir(scenario.dir)) + let fixtureFiles = sessionFixtureNames(await readdir(scenario.dir)) const replaying = mode !== 'record' const compositionPatch = join(composition.dir, replaying ? 'cordis.snapshot.yml' : 'cordis.yml') const patchSources = [ @@ -677,7 +698,7 @@ describe('headless recorded-session snapshots', () => { let actualLogs: SessionLog[] = [] let initialWorkspace: WorkspaceSnapshotEntry[] | undefined let finalWorkspace: WorkspaceSnapshotEntry[] | undefined - const spillRoot = snapshotSpillRoot(join(scenario.dir, 'session.jsonl')) + const spillRoot = snapshotSpillRoot(join(scenario.dir, fixtureFiles[0] as string)) await rm(spillRoot, { recursive: true, force: true }) let result: Awaited> try { @@ -702,7 +723,7 @@ describe('headless recorded-session snapshots', () => { DSH_SNAPSHOT_PROVIDER: model.provider, DSH_SNAPSHOT_MODEL: model.model, DSH_SNAPSHOT_SPILL_ROOT: spillRoot, - DSH_SNAPSHOT_FILE: join(scenario.dir, 'session.jsonl'), + DSH_SNAPSHOT_FILE: join(scenario.dir, fixtureFiles[0] as string), ...(replaying && fixtureFiles.length > 1 ? { DSH_SNAPSHOT_CHILD_FILES: fixtureFiles.slice(1).map(file => join(scenario.dir, file)).join(delimiter) } : {}), @@ -745,6 +766,10 @@ describe('headless recorded-session snapshots', () => { if (mode !== 'replay') { fixtures = await writeSessionFixtures(scenario, actualLogs, fixtures, contextOf(actualLogs.map(log => log.content))) + fixtureFiles = actualLogs.map((log, index) => sessionFixtureName( + index, + sessionHeaderVersion(log.content, `harvested Session ${index}`), + )) } expect(result.stdout).toBe(`${finalTextFromSession(fixtures[0] as string)}\n`) @@ -753,7 +778,9 @@ describe('headless recorded-session snapshots', () => { const actualContext = contextOf(actualLogs.map(log => log.content)) const fixtureContext = contextOf(fixtures) const actualSnapshots = normalizeSessionSnapshots(actualLogs.map(log => log.content), actualContext) - const expectedSnapshots = normalizeSessionSnapshots(fixtures, fixtureContext) + const expectedSnapshots = normalizeSessionSnapshots(fixtures, fixtureContext, { + sourcePaths: fixtureFiles.map(file => join(scenario.dir, file)), + }) for (const [index, actual] of actualSnapshots.entries()) { expect(actual, `${scenario.name}: session ${index}`).toBe(expectedSnapshots[index]) } diff --git a/tsconfig.base.json b/tsconfig.base.json index b6daa12c06..485620bcff 100644 --- a/tsconfig.base.json +++ b/tsconfig.base.json @@ -324,6 +324,9 @@ "@deepseek-ai/dsh-sdk-minimal": ["./packages/bundle/sdk-minimal/src"], "@deepseek-ai/dsh-session": ["./packages/core/session/src"], "@deepseek-ai/dsh-session-checkpoint-policy": ["./packages/session/session-checkpoint-policy/src"], + "@deepseek-ai/dsh-session-format": ["./packages/session/session-format/src"], + "@deepseek-ai/dsh-session-format-catalog": ["./packages/session/session-format-catalog/src"], + "@deepseek-ai/dsh-session-format-v0-to-v1": ["./packages/session/session-format-v0-to-v1/src"], "@deepseek-ai/dsh-session-log-deepseek": ["./packages/session/session-log-deepseek/src"], "@deepseek-ai/dsh-session-log-deepseek/invariant": ["./packages/session/session-log-deepseek/src/invariant.ts"], "@deepseek-ai/dsh-session-log-export": ["./packages/session-query/session-log-export/src"], diff --git a/tsconfig.host.json b/tsconfig.host.json index 6dfe0a9d31..879a48b72a 100644 --- a/tsconfig.host.json +++ b/tsconfig.host.json @@ -9,6 +9,7 @@ }, "include": [ "apps/web/tests/scaffold.ts", + "apps/web/tests/scaffold-generation.spec.ts", "apps/web/tests/default-model.e2e.ts", "apps/web/tests/github-ready-review.e2e.ts", "apps/web/tests/streaming-fence-highlight.e2e.ts", @@ -155,6 +156,9 @@ { "path": "./packages/api/workspace-controller/tsconfig.host.json" }, { "path": "./packages/typert/loader" }, { "path": "./packages/session/session-persistence" }, + { "path": "./packages/session/session-format" }, + { "path": "./packages/session/session-format-v0-to-v1" }, + { "path": "./packages/session/session-format-catalog" }, { "path": "./packages/session/session-checkpoint-policy" }, { "path": "./packages/session/session-log-deepseek" }, { "path": "./packages/session/session-persistence-jsonl" },