mirror of
https://github.com/deepseek-ai/deepseek-harness.git
synced 2026-08-29 04:26:38 +00:00
Revert "Merge pull request #2698 from deepseek-harness/xtr/session-format-migration"
This reverts commit 4b592eb90df20dc53dd12215921d5a9137214777, reversing changes made to d15d3275d905e4d21229cd70a074388a428189d1.
This commit is contained in:
imccyu
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-06-14-session-persistence.md
|
||||
2026-06-14-session-persistence.md: cef11271f26c304ad484d7851801bc69d0c1dfda
|
||||
2026-06-14-session-persistence.zh.md: b6c2467888d0d348aa492c265542565563b75fab
|
||||
2026-06-14-session-persistence.md: 62228bd2f5b25b13880a563818d08f3a2d52d956
|
||||
2026-06-14-session-persistence.zh.md: ebf004333c383336cd025aa8a4aabc9d1e07f0e5
|
||||
|
||||
@@ -29,7 +29,7 @@ Key durable, contested choices:
|
||||
|
||||
Each key choice above records its rejected alternative where the choice is stated: a **chunk-filtered canonical log** (Codex's `policy.rs` shape) — breaks the contiguous-seq contract; **truncating a crashed turn** — silently destroys a long autonomous run's real work; an **in-log `session/meta` event as line 0** — metadata is not replayable state; **finite fractional `createdAt` values** — have no producer and diverge from integer Unix-millisecond storage and query columns; **adopting a non-pristine unversioned SQLite file** — can overwrite unrelated objects or identity; **hard-injecting `sessionPersistence` into the loop** — would pend non-persistent demos forever.
|
||||
|
||||
Format versioning: the header carries a `version`; cold reads accept the current version or a complete static adjacent-version decoder path and reject future versions or missing steps. The format decoder owns historical header and event conversion, while the Coordinator owns operation-specific recovery after decoding ([Session log versioning](2026-08-10-session-log-version-mechanism.md)). The pre-release session format stays pinned at `SESSION_FORMAT_VERSION = 0` and carries no broad compatibility promise. Append-only + flush is robust to partial trailing writes (tolerated during cold preparation) but not to fsync-less power loss mid-line; a DB/WAL backend is the stronger option there.
|
||||
Format versioning: the header carries a `version`; cold reads reject any non-current version. The pre-release session format stays pinned at `SESSION_FORMAT_VERSION = 0` and carries no broad compatibility promise, while the coordinator may own an explicit narrow import upgrade when persisted user data requires it ([pre-identity message recovery](../bug-fix/2026-07-28-load-pre-identity-session-messages.md)). Append-only + flush is robust to partial trailing writes (tolerated during cold preparation) but not to fsync-less power loss mid-line; a DB/WAL backend is the stronger option there.
|
||||
|
||||
## Consequences
|
||||
|
||||
|
||||
@@ -29,7 +29,7 @@ Status: implemented
|
||||
|
||||
上述每个关键选择都在陈述处记录了被否决的替代方案：**过滤分片的规范日志**（Codex 的 `policy.rs` 形式）破坏连续 seq 约定；**截断崩溃的轮次**会静默销毁长时间自主运行中的真实工作；**日志内 `session/meta` 事件作为第 0 行**——元数据不是可回放状态；**有限的非整数 `createdAt` 值**没有生产方，且与整数 Unix 毫秒存储及查询列不一致；**接受非全新的未版本化 SQLite 文件**可能覆盖无关对象或应用标识；**将 `sessionPersistence` 硬注入循环**会让非持久化的演示永远挂起。
|
||||
|
||||
格式版本控制：header 携带一个 `version`；冷读取接受当前版本或完整的静态相邻版本 decoder 路径，并拒绝未来版本或缺失步骤。Format decoder 负责历史 header 和 event 转换，Coordinator 只在解码后负责各操作自己的 recovery（[Session log 版本机制](2026-08-10-session-log-version-mechanism.zh.md)）。预发布阶段的会话格式仍固定为 `SESSION_FORMAT_VERSION = 0`，不承诺广泛兼容。仅追加 + 刷写对尾部的不完整写入具有健壮性（冷准备时可容忍），但无法抵御未使用 fsync 时在行写入中途断电；数据库/WAL 后端是该场景下更强的选项。
|
||||
格式版本控制：header 携带一个 `version`；冷读取拒绝任何非当前版本。预发布阶段的会话格式仍固定为 `SESSION_FORMAT_VERSION = 0`，不承诺广泛兼容；当持久化用户数据确有需要时，协调器可以负责显式且范围受限的导入升级（[消息标识机制引入前的消息恢复](../bug-fix/2026-07-28-load-pre-identity-session-messages.zh.md)）。仅追加 + 刷写对尾部的不完整写入具有健壮性（冷准备时可容忍），但无法抵御未使用 fsync 时在行写入中途断电；数据库/WAL 后端是该场景下更强的选项。
|
||||
|
||||
## 后果
|
||||
|
||||
|
||||
+2
-2
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-08-10-session-log-version-mechanism.md
|
||||
2026-08-10-session-log-version-mechanism.md: dfbe5c1926cf683a34ec6694f188f57c44b9ca10
|
||||
2026-08-10-session-log-version-mechanism.zh.md: 00d58757d3ea4bf1689a0847613557613d40ebf6
|
||||
2026-08-10-session-log-version-mechanism.md: 81108ceaf23405c8f2def9aaef88505d635808a3
|
||||
2026-08-10-session-log-version-mechanism.zh.md: cbb127420e2695853fdc2ad0bb98a7a0bf230b5b
|
||||
|
||||
@@ -14,21 +14,13 @@ Session logs must be upgradable after release, and the runtime that ships first
|
||||
|
||||
**The writer decides bumps, not the reader.** A bump is required exactly when an old runtime could no longer handle a new log with full semantic correctness. "Parses without error" is not the bar: silently skipping content that shapes reconstruction is a wrong read. Only structural changes qualify — header shape, event envelope, core event semantics, the surface mechanism (`SurfaceEventType` set, `SurfaceOp` variants). When unsure, bump: a near-identity upgrader is almost free, a missed bump silently corrupts old readers.
|
||||
|
||||
**Read rules by direction.** Equal version: decode normally. Newer than the reader: refuse, name the direction ("written by a newer harness — upgrade"), and point at the raw log artifact so the user can still see the text (`SessionFormatUnsupportedError`, distinct from `SessionPersistenceCorruptionError` because nothing is damaged). Older than the reader: require a complete chain of static n→n+1 `SessionFormatMigration` classes; a missing migration refuses the read and names the gap. The registry is part of the build rather than Cordis composition, so one build has the same durable read capability under every plugin set.
|
||||
|
||||
**Format migration is the decoder, not a Coordinator repair branch.** Backends expose parsed durable data as `unknown` through a repeatable `StoredSessionSource`: one raw header, one exact revision, and `readEvents()` factories that create independently consumable `AsyncIterable` streams bound to that revision. Each migration class carries static adjacent `from`/`to` versions. One fresh instance handles one decode attempt: `header()` runs once, `event()` maps each input record to exactly one lossless-JSON output with the same seq, and optional `finish()` validates accumulated state after EOF. Instance fields may retain header and earlier-event facts without sharing state across sessions, concurrent reads, or revision retries. Header-only reads stop after `header()` and never call `finish()`, so that method validates EOF state rather than releasing resources. Any version conversion reads the complete event stream and applies the requested suffix only after all migrations; an equal-version read retains backend suffix seek. The decoder validates each output header version and each migration's seq preservation, then applies current `SessionHeader` and `SessionEvent` validation only after the complete chain.
|
||||
|
||||
**A future format bump adds one format-owned migration.** The change adds `format-migrations/vN-to-vN+1.ts`, exports its class from the static `SESSION_FORMAT_MIGRATIONS` array, and increments `SESSION_FORMAT_VERSION`. The migration owns every old header and event variant it accepts, its instance state, and explicit failure for malformed input. It cannot add, remove, reorder, or renumber events: durable references use seq as event identity. A format change that alters facts consumed by a projection increments that projection's `stateVersion`; unchanged projections retain their cache rows. Backends and the Coordinator do not gain version-specific branches. Historical variants that never changed the version remain isolated in the format-v0 compatibility decoder and are not a template for later version migrations. This decoder maps the historical `compact/start`, `compact/summary`, `compact/end`, and `compact/prune` names to canonical `compaction/*` events while preserving the rest of each record.
|
||||
|
||||
**Recovery and writeback consume current-format data.** `inspect()` and `readFrom()` decode only in memory. Cold `prepare()`/`load()` first decode the whole source, add the current recovery closers, and replace the exact old revision with that complete balanced current-format stream. Live HMR adoption uses the same replacement primitive after seed verification but does not synthesize closers for a turn still owned by the live Session. A successful replacement or revision conflict discards the prepared object and reopens the stored source before continuing.
|
||||
|
||||
**Replacement is an internal backend compare-and-swap.** `replaceStored(expectedRevision, meta, events)` accepts a streaming current-format log and checks storage identity plus the source revision at the commit boundary. JSONL writes and fsyncs a sibling temporary artifact, rechecks the source revision immediately before the atomic replace, atomically replaces the path (using the Windows write-through replacement primitive there), and syncs the parent directory on POSIX; like every other coordinator freshness check, the recheck adds no cross-process writer exclusion — JSONL assumes one live writer per session. SQLite stages the event iterator, then rechecks and replaces the header and event rows in one transaction. A failed commit leaves one complete old or new log; retaining a permanent pre-upgrade copy is a separate recovery policy, not part of the format migration API.
|
||||
**Read rules by direction.** Equal version: read normally. Newer than the reader: refuse, name the direction ("written by a newer harness — upgrade"), and point at the raw log artifact so the user can still see the text (`SessionFormatUnsupportedError`, distinct from `SessionPersistenceCorruptionError` because nothing is damaged). Older than the reader: convert in memory through the chain of n→n+1 upgraders for viewing; persist the converted log only when the session is actually continued (atomic temp-file replace, original kept as backup). A step whose upgrader cannot be written is left empty, which cuts off every version at or below it — those degrade to raw-text viewing.
|
||||
|
||||
**A per-event `ignorable` marker covers vocabulary growth, so ordinary event additions never bump the version.** The event vocabulary is decided by which plugins are mounted, which a single version integer cannot describe. A reader meeting an unrecognized event type refuses to interpret the log unless the event carries `ignorable: true` in its envelope. The default is *required*: forgetting the marker over-refuses a resumable session (an inconvenience), while a default of ignorable would make the same mistake silently resume a gutted one (a safety failure). The architecture makes this sound: model-visible content flows only through the three `surfaceOp`-marked surface event types plus the `request/header`/`request/context` folds, so the dangerous unknowns are exactly the non-surface events that change how the rest of the log is read (`session/end-seed` is the existing example).
|
||||
|
||||
## Consequences
|
||||
|
||||
Format v0 carries direction-aware refusal with the raw-log path; the unknown-event guard against a generated known-vocabulary list (`KNOWN_SESSION_EVENT_TYPES`, emitted by `gen-persistence-catalog` from every `SessionEventMap` merge and kept fresh by `verify-persistence-catalog`); the `ignorable` envelope field accepted by seed validation, both backends (a dedicated SQLite column, `SCHEMA_VERSION` 15), and the BFF wire schema; and the static streaming migration decoder with an empty adjacent-version registry. `SESSION_FORMAT_VERSION` remains 0 until a real v0→v1 step lands. The decoder and backend replacement APIs therefore have direct tests without manufacturing a format bump. Writers do not yet set `ignorable` because no producer needs it. Until a registration surface exists, an out-of-repo plugin's events refuse resume under first-party readers; the refusal is loud rather than silent. The unknown-type guard is read-side only: `appendCore` keeps rejecting retired legacy shapes but does not vocabulary-check new types, because an append-time refusal would stall a live session's durability mid-flight, which costs more than a loud refusal at the log's next load. The JSONL backend additionally refuses a foreign version from the raw header line before validating today's header fields or decoding any event record, so a structurally different future format still reports the upgrade direction instead of "corrupt"; SQLite gates whole-file structure through its own `SCHEMA_VERSION` pragma first.
|
||||
What shipped in v0 (release 0812): direction-aware refusal with the raw-log path; the unknown-event guard against a generated known-vocabulary list (`KNOWN_SESSION_EVENT_TYPES`, emitted by `gen-persistence-catalog` from every `SessionEventMap` merge and kept fresh by `verify-persistence-catalog`); the `ignorable` envelope field accepted by seed validation, both backends (a dedicated SQLite column, `SCHEMA_VERSION` 15), and the BFF wire schema. The upgrader chain itself is deferred until the first real v0→v1 step exists to test it against; writers do not yet set `ignorable` (no producer needs it), so `Session.append` gains that surface with its first user. Until a registration surface exists, an out-of-repo plugin's events refuse resume under first-party readers — the pre-release stance accepts that, and the refusal is loud rather than silent. The unknown-type guard is read-side only: `appendCore` keeps rejecting retired legacy shapes but does not vocabulary-check new types, because an append-time refusal would stall a live session's durability mid-flight, which costs more than a loud refusal at the log's next load. The JSONL backend additionally refuses a foreign version from the raw header line before validating this format version's header shape or decoding any event row, so a structurally different future format still reports the upgrade direction instead of "corrupt"; SQLite gates whole-file structure through its own `SCHEMA_VERSION` pragma first.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
@@ -36,6 +28,3 @@ Format v0 carries direction-aware refusal with the raw-log path; the unknown-eve
|
||||
- **Default-ignorable unknown events** — inverts the failure mode of a forgotten marker from visible over-refusal into silent corruption.
|
||||
- **Auto-migrating on view** — rewriting the artifact on open turns a read into a destructive write: a converter bug corrupts logs at browse time, and a same-directory older runtime loses access because a newer one merely looked.
|
||||
- **Per-plugin runtime registration of known event types** — would make the known set composition-dependent, so a leaner same-version composition would refuse logs a fuller one wrote. The generated repo-wide list keeps same-version reads uniform; out-of-repo plugin events are outside it by construction, and a registration surface for them is deferred until such a consumer exists.
|
||||
- **Materializing migrations as header and event arrays** — makes the framework proportional to complete log size in memory even when each transformation is record-local. Repeatable revision-bound readers plus one-at-a-time event transforms preserve retry semantics without imposing that allocation.
|
||||
- **Version-specific conversion in `PersistenceCoordinator`** — mixes format decoding with operation-specific crash recovery and duplicates behavior across inspect, suffix read, cold continuation, and live adoption. The shared decoder produces only current-format data; each consumer retains its own recovery intent.
|
||||
- **A mandatory permanent backup for every upgrade** — is not needed for atomicity and cannot promise the same physical representation across JSONL and SQLite. Backends may add recovery copies as a separate product policy without changing migrations.
|
||||
|
||||
+2
-13
@@ -14,21 +14,13 @@ Session log 在发布后必须能升级格式，而最先发布的运行时决
|
||||
|
||||
**升不升版本由写入方决定，与读取方能力无关。**当且仅当老运行时无法在语义上完全正确地处理新日志时才必须升版本。"解析不报错"不是标准：静默跳过影响重建的内容就是读错。只有结构性变更够得上这条线：header 形状、事件信封、核心事件语义、surface 机制（`SurfaceEventType` 集合、`SurfaceOp` 变体）。拿不准就升：近似恒等的升级器几乎没有成本，漏升一次会让老读取器静默读坏。
|
||||
|
||||
**读取规则按方向区分。**版本相等：正常解码。比读取器新：拒绝，说明方向（"由更新的 harness 写入，请升级"），并给出原始日志文件的路径，用户仍能看到文本（`SessionFormatUnsupportedError`，与 `SessionPersistenceCorruptionError` 区分，因为数据没有损坏）。比读取器旧：要求静态 n→n+1 `SessionFormatMigration` 类组成完整链路，缺失任何 migration 都会拒绝并指出断点。注册表属于 build 而不是 Cordis composition，因此同一个 build 在任何插件组合下都具有相同的持久化读取能力。
|
||||
|
||||
**格式迁移就是 decoder，不是 Coordinator 的修复分支。**后端通过可重复读取的 `StoredSessionSource` 把解析后的持久化数据作为 `unknown` 暴露：一个原始 header、一个精确 revision，以及每次产生独立 `AsyncIterable` 且绑定该 revision 的 `readEvents()` factory。每个 migration class 用静态且相邻的 `from`／`to` 标识版本。每次 decode 都创建一个新实例：`header()` 调用一次；`event()` 把每条输入记录映射为一条 seq 相同、可无损表示为 JSON 的输出；可选的 `finish()` 在 EOF 后验证累计状态。实例字段可以保留 header 与之前事件的事实，而不会在 Session、并发读取或 revision retry 之间共享状态。只读 header 时在 `header()` 后结束，绝不调用 `finish()`，因此该方法用于验证 EOF 状态而不是释放资源。只要发生版本转换，就读取完整事件流，并在所有 migration 完成后才应用请求的 suffix；版本相等时仍保留 backend suffix seek。Decoder 验证每一步输出的 header version 和每个 migration 是否保持 seq，完整链路结束后才执行当前 `SessionHeader` 和 `SessionEvent` 校验。
|
||||
|
||||
**以后每次 format bump 只增加一个格式 migration。**改动新增 `format-migrations/vN-to-vN+1.ts`，把它的 class 导出到静态 `SESSION_FORMAT_MIGRATIONS` 数组，并递增 `SESSION_FORMAT_VERSION`。Migration 自己负责它接受的所有旧 header 和 event 变体、实例状态，以及对畸形输入的明确失败。它不能增加、删除、重排事件或重编号：持久引用以 seq 作为事件身份。如果格式变化影响了某个 projection 消费的事实，就递增该 projection 的 `stateVersion`；未受影响的 projection 保留 cache 记录。Backend 和 Coordinator 不增加版本特判。没有改变版本号的历史变体继续隔离在 format-v0 compatibility decoder 中，不作为后续版本 migration 的模板。该 decoder 将历史 `compact/start`、`compact/summary`、`compact/end`、`compact/prune` 名称映射为规范的 `compaction/*` 事件，并保留每条记录的其余内容。
|
||||
|
||||
**Recovery 和写回只消费当前格式数据。**`inspect()` 和 `readFrom()` 只在内存中解码。Cold `prepare()`/`load()` 先解码完整 source，补充当前 recovery closers，再用完整、平衡的当前格式 stream 替换精确的旧 revision。Live HMR adoption 在 seed 校验后使用同一个 replacement primitive，但不会为仍由 live Session 掌握的 turn 合成 closer。替换成功或 revision 冲突后都会丢弃 prepared object，重新打开持久化 source 后再继续。
|
||||
|
||||
**Replacement 是 backend 内部的 compare-and-swap。**`replaceStored(expectedRevision, meta, events)` 接受流式当前格式日志，并在提交边界检查存储身份和 source revision。JSONL 写入并 fsync 同目录临时 artifact，在原子替换路径前立即复核 source revision，然后原子替换（Windows 使用 write-through replacement primitive），并在 POSIX 上同步父目录；与协调器的其他新鲜性检查一样，复核不提供跨进程写者排他——JSONL 假定每个 session 同时只有一个 live writer。SQLite 先暂存 event iterator，再在一个事务中复核并替换 header 与 event rows。提交失败后只会留下完整旧日志或完整新日志；永久保留升级前副本是独立的恢复策略，不属于 format migration API。
|
||||
**读取规则按方向区分。**版本相等：正常读。比读取器新：拒绝，说明方向（"由更新的 harness 写入，请升级"），并给出原始日志文件的路径，用户仍能看到文本（`SessionFormatUnsupportedError`，与 `SessionPersistenceCorruptionError` 区分，因为数据没有损坏）。比读取器旧：查看时经 n→n+1 升级器链在内存中逐级转换；只有会话真正被继续时才把转换落盘（临时文件原子替换，原文件留备份）。写不出升级器的那一步留空，这会切断该步及更早所有版本的升级路径，它们降级为只能看原文。
|
||||
|
||||
**逐事件的 `ignorable` 标记吸收词汇表增长，普通的新增事件永远不用升版本。**事件词汇表由挂载了哪些插件决定，单个版本整数描述不了它。读取器遇到不认识的事件类型时拒绝解读日志，除非该事件的信封带 `ignorable: true`。默认为必需：忘写标记的后果是把一个本可恢复的会话拒绝过头（体验问题），而默认可忽略会让同样的疏忽静默恢复出残缺会话（安全事故）。架构保证了这条规则成立：模型可见内容只经三种带 `surfaceOp` 标记的 surface 事件加 `request/header`、`request/context` 折叠进入重建，危险的未知事件恰好是那些不进 surface 但改变日志其余部分解读方式的事件（`session/end-seed` 是现存例子）。
|
||||
|
||||
## 影响
|
||||
|
||||
Format v0 包含：分方向的拒绝并带原始日志路径；基于生成的已知词汇清单（`KNOWN_SESSION_EVENT_TYPES`，由 `gen-persistence-catalog` 从所有 `SessionEventMap` 声明合并生成，`verify-persistence-catalog` 保证新鲜）的未知事件守卫；`ignorable` 信封字段被种子校验、两个后端（SQLite 专用列，`SCHEMA_VERSION` 15）和 BFF 线上 schema 接受；以及使用空相邻版本注册表的静态流式 migration decoder。`SESSION_FORMAT_VERSION` 保持 0，直到真实 v0→v1 步骤合入。Decoder 和 backend replacement API 因此可以直接测试，不需要制造一次 format bump。写入侧目前不写 `ignorable`，因为还没有生产者需要它。在注册表面出现之前，仓库外插件的事件在第一方读取器下无法恢复会话；拒绝是显式的而非静默的。未知类型守卫只在读取侧生效：`appendCore` 继续拒绝已淘汰的 legacy 形状，但不对新类型做词汇检查，因为写入时拒绝会让活跃会话的持久化中途停摆，代价大于下次加载时的显式拒绝。JSONL 后端还会在校验当前 header 字段、解码任何 event record 之前，直接从原始 header 行拒绝外来版本，因此结构完全不同的未来格式仍会报告升级方向而不是"损坏"；SQLite 则先由自己的 `SCHEMA_VERSION` pragma 把关整个文件的结构。
|
||||
v0（0812 发布）交付的内容：分方向的拒绝并带原始日志路径；基于生成的已知词汇清单（`KNOWN_SESSION_EVENT_TYPES`，由 `gen-persistence-catalog` 从所有 `SessionEventMap` 声明合并生成，`verify-persistence-catalog` 保证新鲜）的未知事件守卫；`ignorable` 信封字段被种子校验、两个后端（SQLite 专用列，`SCHEMA_VERSION` 升到 15）和 BFF 线上 schema 接受。升级器链本身推迟到第一个真实的 v0→v1 变更出现、有真实对象可测时再建；写入侧目前不写 `ignorable`（还没有生产者需要它），`Session.append` 的这一表面随第一个使用者一起落地。在注册表面出现之前，仓库外插件的事件在第一方读取器下无法恢复会话，预发布立场接受这一点，而且拒绝是显式的而非静默的。未知类型守卫只在读取侧生效：`appendCore` 继续拒绝已淘汰的 legacy 形状，但不对新类型做词汇检查，因为写入时拒绝会让活跃会话的持久化中途停摆，代价大于下次加载时的显式拒绝。JSONL 后端还会在校验本格式版本的 header 形状、解码任何事件行之前，直接从原始 header 行拒绝外来版本，因此结构完全不同的未来格式仍会报告升级方向而不是"损坏"；SQLite 则先由自己的 `SCHEMA_VERSION` pragma 把关整个文件的结构。
|
||||
|
||||
## 曾考虑的替代方案
|
||||
|
||||
@@ -36,6 +28,3 @@ Format v0 包含：分方向的拒绝并带原始日志路径；基于生成的
|
||||
- **未知事件默认可忽略**：把忘写标记的后果从可见的过度拒绝反转成静默损坏。
|
||||
- **查看时自动迁移落盘**：打开即改写把读操作变成破坏性写操作，转换器的 bug 会在浏览时损坏日志，同目录的旧版本运行时也会因为新版本只是看了一眼就失去访问能力。
|
||||
- **插件运行时注册已知事件类型**：会让已知集依赖插件组合，同版本的精简组合会拒绝完整组合写出的日志。生成的全仓库清单保证同版本读取行为一致；仓库外插件的事件按构造就在清单之外，为它们提供注册表面推迟到真有这样的消费者时再做。
|
||||
- **把 migration 物化为 header 和 event 数组**：即使每步转换只依赖单条 record，也会让框架内存占用与完整日志大小成正比。可重复、绑定 revision 的 reader 加逐事件转换保留重试语义，又不强制这笔分配。
|
||||
- **在 `PersistenceCoordinator` 内写版本转换**：会把格式解码和各操作不同的 crash recovery 混在一起，并在 inspect、suffix read、cold continuation 和 live adoption 间复制行为。共享 decoder 只产出当前格式数据，各 consumer 保留自己的 recovery intent。
|
||||
- **每次升级都强制永久备份**：原子性不依赖永久副本，而且 JSONL 与 SQLite 无法承诺相同的物理表示。Backend 可以把恢复副本作为独立产品策略加入，不需要修改 migration。
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write docs/config-catalog.md
|
||||
config-catalog.md: f1e930c4ae7a5c9f0c045b17e61e224f75ce0116
|
||||
config-catalog.zh.md: e9afb428ccd196ce945d5fed74f771fcec3f46f9
|
||||
config-catalog.md: 20dbf55aff834a77e2049bcbb6485d84cd38589c
|
||||
config-catalog.zh.md: 6cccf68e0c191a0de43bf190380cd0d2329d4391
|
||||
|
||||
@@ -1780,7 +1780,7 @@ export interface Config {
|
||||
export type JsonlCompression = 'zstd' | 'none'
|
||||
```
|
||||
|
||||
Source: [`packages/session/session-persistence-jsonl/src/index.ts:64`](../packages/session/session-persistence-jsonl/src/index.ts)
|
||||
Source: [`packages/session/session-persistence-jsonl/src/index.ts:62`](../packages/session/session-persistence-jsonl/src/index.ts)
|
||||
|
||||
<a id="deepseek-aidsh-session-persistence-sqlite"></a>
|
||||
|
||||
|
||||
@@ -1782,7 +1782,7 @@ export interface Config {
|
||||
export type JsonlCompression = 'zstd' | 'none'
|
||||
```
|
||||
|
||||
来源：[`packages/session/session-persistence-jsonl/src/index.ts:64`](../packages/session/session-persistence-jsonl/src/index.ts)
|
||||
来源：[`packages/session/session-persistence-jsonl/src/index.ts:62`](../packages/session/session-persistence-jsonl/src/index.ts)
|
||||
|
||||
<a id="deepseek-aidsh-session-persistence-sqlite"></a>
|
||||
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write docs/subsystems/persistence.md
|
||||
persistence.md: 5046be0f2ff65faa7fa71f41d8141399d55bfa96
|
||||
persistence.zh.md: 71bbca1121e5b8d1e9441d857a0d0989c9946d51
|
||||
persistence.md: 098f5798e5313ca97e90e67dce1d67177f003ca7
|
||||
persistence.zh.md: d6b3baf7cdb7f1735008e0c1da9740e0b756baff
|
||||
|
||||
@@ -51,8 +51,8 @@ Source: [`packages/core/session/src/types.ts`](../../packages/core/session/src/t
|
||||
interface SessionHeader {
|
||||
/**
|
||||
* On-disk format version, stamped from {@link SESSION_FORMAT_VERSION} when the
|
||||
* session is created. Persistence refuses newer versions and older versions
|
||||
* without a complete registered migration path.
|
||||
* session is created. A persistence backend rejects any other version on load
|
||||
* (no migration — see the constant).
|
||||
*/
|
||||
readonly version: number
|
||||
/** The session's id (mirrors the {@link Session}'s id). */
|
||||
@@ -91,27 +91,7 @@ interface SessionHeader {
|
||||
|
||||
## Format refusal — logs a build cannot faithfully read
|
||||
|
||||
A backend refuses a log it cannot faithfully interpret with `SessionFormatUnsupportedError`, distinct from `SessionPersistenceCorruptionError` because nothing is damaged. A header `version` ahead of `SESSION_FORMAT_VERSION` names the direction ("written by a newer harness — upgrade the harness to open it"); one behind it requires a complete registered adjacent-version migration path or names the missing step. After legacy-shape normalization, an event type outside this build's generated vocabulary (`KNOWN_SESSION_EVENT_TYPES`, emitted by `gen-persistence-catalog`) refuses the same way unless the event's envelope carries `ignorable: true` — silently skipping an unrecognized required event could change how the rest of the log must be read. The message appends the raw log path when the backend keeps one artifact per session, so the refused text stays reachable. The JSONL backend refuses a foreign version straight from the raw header line, before validating today's header shape or decoding any event row — a structurally different future format still reports the upgrade direction, never "corrupt"; SQLite gates whole-file structure through its own `SCHEMA_VERSION` pragma first. Design rationale lives in the [session-log-version-mechanism note](../../.agents/notes/implemented/architecture/2026-08-10-session-log-version-mechanism.md).
|
||||
|
||||
## `SessionFormatMigration` — adjacent static format upgrades
|
||||
|
||||
Each migration class declares one adjacent `from`/`to` pair and creates fresh state for one decode attempt. The decoder snapshots every header and event output as detached lossless JSON before the next migration receives it, preserves event sequence numbers, and calls optional EOF validation only after the complete event stream is consumed. The [package README](../../packages/session/session-persistence/README.md) owns the registration and version-bump procedure.
|
||||
|
||||
```ts type-equiv
|
||||
/** Static identity and constructor for one adjacent-version migration. */
|
||||
interface SessionFormatMigration {
|
||||
/** Input Session format version. */
|
||||
readonly from: number
|
||||
/** Output Session format version; must equal `from + 1`. */
|
||||
readonly to: number
|
||||
/**
|
||||
* Create fresh state for one header decode and its optional complete event
|
||||
* stream. Instances are never shared across sessions or decode attempts.
|
||||
* @returns a single-use migration instance.
|
||||
*/
|
||||
new(): SessionFormatMigrationInstance
|
||||
}
|
||||
```
|
||||
A backend refuses a log it cannot faithfully interpret with `SessionFormatUnsupportedError`, distinct from `SessionPersistenceCorruptionError` because nothing is damaged. A header `version` ahead of `SESSION_FORMAT_VERSION` names the direction ("written by a newer harness — upgrade the harness to open it"); one behind it states that this build ships no upgrade path. After legacy-shape normalization, an event type outside this build's generated vocabulary (`KNOWN_SESSION_EVENT_TYPES`, emitted by `gen-persistence-catalog`) refuses the same way unless the event's envelope carries `ignorable: true` — silently skipping an unrecognized required event could change how the rest of the log must be read. The message appends the raw log path when the backend keeps one artifact per session, so the refused text stays reachable. The JSONL backend refuses a foreign version straight from the raw header line, before validating this format version's header shape or decoding any event row — a structurally different future format still reports the upgrade direction, never "corrupt"; SQLite gates whole-file structure through its own `SCHEMA_VERSION` pragma first. Design rationale and the deferred upgrader chain live in the [session-log-version-mechanism note](../../.agents/notes/implemented/architecture/2026-08-10-session-log-version-mechanism.md).
|
||||
|
||||
## `CreateSessionOptions` — seeding and metadata
|
||||
|
||||
|
||||
@@ -51,8 +51,8 @@ interface SessionLocation {
|
||||
interface SessionHeader {
|
||||
/**
|
||||
* On-disk format version, stamped from {@link SESSION_FORMAT_VERSION} when the
|
||||
* session is created. Persistence refuses newer versions and older versions
|
||||
* without a complete registered migration path.
|
||||
* session is created. A persistence backend rejects any other version on load
|
||||
* (no migration — see the constant).
|
||||
*/
|
||||
readonly version: number
|
||||
/** The session's id (mirrors the {@link Session}'s id). */
|
||||
@@ -91,27 +91,7 @@ interface SessionHeader {
|
||||
|
||||
## 格式拒绝：本构建无法可靠读取的日志
|
||||
|
||||
后端用 `SessionFormatUnsupportedError` 拒绝无法可靠解读的日志，它与 `SessionPersistenceCorruptionError` 区分，因为数据没有损坏。header 的 `version` 比 `SESSION_FORMAT_VERSION` 新时，消息说明方向（"由更新的 harness 写入，请升级 harness 后打开"）；比它旧时则要求一条完整注册的相邻版本迁移路径，否则会指出缺失步骤。经过 legacy 形状归一化后，本构建生成词汇表（`KNOWN_SESSION_EVENT_TYPES`，由 `gen-persistence-catalog` 生成）之外的事件类型同样被拒绝，除非该事件的信封带 `ignorable: true`：静默跳过一个不认识的必需事件可能改变日志其余部分的解读方式。后端为每个会话保留独立文件时，消息附上原始日志路径，被拒绝的文本仍然可读。JSONL 后端直接从原始 header 行拒绝外来版本，先于当前 header 形状校验和任何事件行解码，因此结构完全不同的未来格式仍会报告升级方向，绝不会报"损坏"；SQLite 则先由自己的 `SCHEMA_VERSION` pragma 把关整个文件的结构。设计理由见 [session-log 版本机制 Agent Note](../../.agents/notes/implemented/architecture/2026-08-10-session-log-version-mechanism.zh.md)。
|
||||
|
||||
## `SessionFormatMigration`：相邻静态格式升级
|
||||
|
||||
每个迁移 class 声明一组相邻的 `from`／`to`，并为一次解码创建全新状态。decoder 会将每次 header 和事件输出快照为分离的无损 JSON，再交给下一项迁移，同时保留事件 seq；只有完整消费事件流后，才会调用可选的 EOF 验证。[包 README](../../packages/session/session-persistence/README.zh.md)负责说明注册与版本递增步骤。
|
||||
|
||||
```ts type-equiv
|
||||
/** Static identity and constructor for one adjacent-version migration. */
|
||||
interface SessionFormatMigration {
|
||||
/** Input Session format version. */
|
||||
readonly from: number
|
||||
/** Output Session format version; must equal `from + 1`. */
|
||||
readonly to: number
|
||||
/**
|
||||
* Create fresh state for one header decode and its optional complete event
|
||||
* stream. Instances are never shared across sessions or decode attempts.
|
||||
* @returns a single-use migration instance.
|
||||
*/
|
||||
new(): SessionFormatMigrationInstance
|
||||
}
|
||||
```
|
||||
后端用 `SessionFormatUnsupportedError` 拒绝无法可靠解读的日志，它与 `SessionPersistenceCorruptionError` 区分，因为数据没有损坏。header 的 `version` 比 `SESSION_FORMAT_VERSION` 新时，消息说明方向（"由更新的 harness 写入，请升级 harness 后打开"）；比它旧时说明本构建没有升级路径。经过 legacy 形状归一化后，本构建生成词汇表（`KNOWN_SESSION_EVENT_TYPES`，由 `gen-persistence-catalog` 生成）之外的事件类型同样被拒绝，除非该事件的信封带 `ignorable: true`：静默跳过一个不认识的必需事件可能改变日志其余部分的解读方式。后端为每个会话保留独立文件时，消息附上原始日志路径，被拒绝的文本仍然可读。JSONL 后端直接从原始 header 行拒绝外来版本，先于本格式版本的 header 形状校验和任何事件行解码，因此结构完全不同的未来格式仍会报告升级方向，绝不会报"损坏"；SQLite 则先由自己的 `SCHEMA_VERSION` pragma 把关整个文件的结构。设计理由与推迟建设的升级器链见 [session-log 版本机制 Agent Note](../../.agents/notes/implemented/architecture/2026-08-10-session-log-version-mechanism.zh.md)。
|
||||
|
||||
## `CreateSessionOptions`：seed 与元数据
|
||||
|
||||
|
||||
@@ -24,7 +24,7 @@ import {
|
||||
PersistenceCoordinator,
|
||||
SessionPersistenceRevision,
|
||||
type PersistenceBackend,
|
||||
type StoredSessionSource,
|
||||
type StoredPrefix,
|
||||
} from '@deepseek-ai/dsh-session-persistence'
|
||||
import { ApiSessionList } from '../src/list.ts'
|
||||
import {
|
||||
@@ -359,29 +359,19 @@ describe('cold history recovery view', () => {
|
||||
await ctx.plugin(SessionStore)
|
||||
const sessionId = sid('session-interrupted')
|
||||
const meta = header(sessionId, 1000)
|
||||
const revision = SessionPersistenceRevision('history-recovery-test:1')
|
||||
const stored: StoredSessionSource<never> = {
|
||||
const stored: StoredPrefix<never> = {
|
||||
meta,
|
||||
revision,
|
||||
readEvents: ({ fromSeq = 0 } = {}) => ({
|
||||
events: (async function* () {
|
||||
const events: SessionEvent[] = [
|
||||
{ type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } },
|
||||
]
|
||||
for (const event of events.slice(fromSeq)) yield structuredClone(event)
|
||||
})(),
|
||||
completed: Promise.resolve({}),
|
||||
}),
|
||||
events: [{ type: 'turn/start', seq: 0, time: 1, data: { turn: 1 } }],
|
||||
revision: SessionPersistenceRevision('history-recovery-test:1'),
|
||||
}
|
||||
const backend: PersistenceBackend<never> = {
|
||||
name: 'history-recovery-test',
|
||||
openStored: id => Promise.resolve(id === sessionId ? stored : undefined),
|
||||
loadStored: id => Promise.resolve(id === sessionId ? structuredClone(stored) : undefined),
|
||||
readStoredRevision: id => Promise.resolve(
|
||||
id === sessionId ? revision : undefined,
|
||||
id === sessionId ? SessionPersistenceRevision('history-recovery-test:1') : undefined,
|
||||
),
|
||||
appendBatch: () => Promise.resolve(),
|
||||
commitRepair: () => Promise.resolve(),
|
||||
replaceStored: () => Promise.resolve(),
|
||||
list: () => Promise.resolve([structuredClone(meta)]),
|
||||
}
|
||||
const coordinator = new PersistenceCoordinator(ctx, backend)
|
||||
|
||||
@@ -35,7 +35,7 @@ export function SessionId(id: string): SessionId {
|
||||
* and enforced by every persistence backend on load. The single source of truth for the
|
||||
* version — write sites and the load-time check all read it.
|
||||
* While the harness is unreleased it is pinned at `0`: no compatibility is
|
||||
* implied; older logs load only through a complete adjacent migration path.
|
||||
* implied, incompatible logs are rejected, and no migration is provided.
|
||||
*
|
||||
* The version is a single monotonic integer with no major/minor split. Whether
|
||||
* a bump is needed is decided by what the WRITER emits, never by what a newer
|
||||
@@ -61,8 +61,8 @@ export const SESSION_FORMAT_VERSION = 0
|
||||
export interface SessionHeader {
|
||||
/**
|
||||
* On-disk format version, stamped from {@link SESSION_FORMAT_VERSION} when the
|
||||
* session is created. Persistence refuses newer versions and older versions
|
||||
* without a complete registered migration path.
|
||||
* session is created. A persistence backend rejects any other version on load
|
||||
* (no migration — see the constant).
|
||||
*/
|
||||
readonly version: number
|
||||
/** The session's id (mirrors the {@link Session}'s id). */
|
||||
|
||||
@@ -58,7 +58,6 @@
|
||||
"@deepseek-ai/dsh-sandbox-local": "workspace:^",
|
||||
"@deepseek-ai/dsh-sandbox-policy": "workspace:^",
|
||||
"@deepseek-ai/dsh-session": "workspace:^",
|
||||
"@deepseek-ai/dsh-session-persistence": "workspace:^",
|
||||
"@deepseek-ai/dsh-session-persistence-jsonl": "workspace:^",
|
||||
"@deepseek-ai/dsh-subagent": "workspace:^",
|
||||
"@deepseek-ai/dsh-subprocess-local": "workspace:^",
|
||||
|
||||
@@ -1,9 +1,7 @@
|
||||
import { readFileSync, readdirSync } from 'node:fs'
|
||||
import { join, relative } from 'node:path'
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { Session, SessionId, type SessionEvent, type SessionHeader } from '@deepseek-ai/dsh-session'
|
||||
import { SessionPersistenceRevision } from '@deepseek-ai/dsh-session-persistence'
|
||||
import { decodeStoredSession } from '@deepseek-ai/dsh-session-persistence/src/format-decoder.ts'
|
||||
import { Session, SessionId, type SessionEvent } from '@deepseek-ai/dsh-session'
|
||||
import { scanLog } from '@deepseek-ai/dsh-session-persistence-jsonl/src/format.ts'
|
||||
import { foldSubagentDescriptor } from '@deepseek-ai/dsh-subagent'
|
||||
import {
|
||||
@@ -28,24 +26,10 @@ function filesUnder(root: string): string[] {
|
||||
return files.sort()
|
||||
}
|
||||
|
||||
async function readSession(id: string): Promise<{ meta: SessionHeader; events: SessionEvent[] }> {
|
||||
const stored = scanLog(readFileSync(
|
||||
function readSession(id: string): ReturnType<typeof scanLog> {
|
||||
return scanLog(readFileSync(
|
||||
join(VFS_EXAMPLE_ROOT, 'home/sessions/--dsh-workspace--', id, 'session.jsonl'),
|
||||
))
|
||||
const decoded = decodeStoredSession({
|
||||
meta: stored.meta,
|
||||
revision: SessionPersistenceRevision(`vfs-example:${id}`),
|
||||
readEvents: () => ({
|
||||
events: (async function* (): AsyncIterable<unknown> {
|
||||
yield* stored.events
|
||||
})(),
|
||||
completed: Promise.resolve({}),
|
||||
}),
|
||||
}, SessionId(id))
|
||||
const events: SessionEvent[] = []
|
||||
for await (const event of decoded.events) events.push(event)
|
||||
await decoded.completed
|
||||
return { meta: decoded.meta, events }
|
||||
}
|
||||
|
||||
function textOf(event: SessionEvent): string {
|
||||
@@ -82,8 +66,8 @@ describe('WebWorker preview VFS example', () => {
|
||||
})
|
||||
})
|
||||
|
||||
it('restores the main production log with paging and tool coverage', async () => {
|
||||
const { meta, events } = await readSession(VFS_EXAMPLE_SESSION_IDS.main)
|
||||
it('restores the main production log with paging and tool coverage', () => {
|
||||
const { meta, events } = readSession(VFS_EXAMPLE_SESSION_IDS.main)
|
||||
expect(meta).toMatchObject({
|
||||
id: VFS_EXAMPLE_SESSION_IDS.main,
|
||||
cwd: '/dsh/workspace',
|
||||
@@ -110,13 +94,13 @@ describe('WebWorker preview VFS example', () => {
|
||||
expect(events.some(event => event.type === 'tool/result' && event.data.message.content[0].isError === true)).toBe(true)
|
||||
})
|
||||
|
||||
it('restores one-shot and continuable child Sessions with durable descriptors', async () => {
|
||||
it('restores one-shot and continuable child Sessions with durable descriptors', () => {
|
||||
const expected = [
|
||||
[VFS_EXAMPLE_SESSION_IDS.oneShot, 'one-shot'],
|
||||
[VFS_EXAMPLE_SESSION_IDS.continuable, 'continuable'],
|
||||
] as const
|
||||
for (const [id, mode] of expected) {
|
||||
const { meta, events } = await readSession(id)
|
||||
const { meta, events } = readSession(id)
|
||||
expect(meta).toMatchObject({
|
||||
id,
|
||||
cwd: '/dsh/workspace',
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/session/session-persistence-jsonl/README.md
|
||||
README.md: b7ee18add7716054b24e71d0273a4e04bd544973
|
||||
README.zh.md: e4249ec130c0cf981004de442f38e2e0a7cca471
|
||||
README.md: 0301691acbe42c7973274e717ee9ae6f405ebea1
|
||||
README.zh.md: c05c380166b65a9826b6cb7f31729a51628ab49b
|
||||
|
||||
@@ -35,7 +35,7 @@ The JSONL durable session-persistence backend — a concrete `SessionPersistence
|
||||
|
||||
The default artifact is a standard concatenation of independent [Zstandard frames](../../../.agents/notes/implemented/architecture/2026-07-19-zstandard-jsonl-session-logs.md): one checksummed frame containing only the header line, followed by one checksummed frame per durable append batch. The backend uses Node's built-in Zstandard API with its default compression level and exposes no level knob. Listing reads and validates only the header frame. `compression: 'none'` keeps the same logical lines in the original raw representation.
|
||||
|
||||
A root belongs to one encoding. Startup discovery and targeted lookup reject the opposite suffix with an error naming the incompatible artifact and instructing the caller to select the matching mode or a separate root. Flat `<project>/<id>.jsonl*` artifacts are also rejected instead of ignored. Session format migrations can replace a logical log within its configured encoding; there is no compression migration, mixed-root fallback, or dual write.
|
||||
A root belongs to one encoding. Startup discovery and targeted lookup reject the opposite suffix with an error naming the incompatible artifact and instructing the caller to select the matching mode or a separate root. Flat `<project>/<id>.jsonl*` artifacts are also rejected instead of ignored. There is no migration, mixed-root fallback, or dual write.
|
||||
|
||||
## Durability and crash semantics
|
||||
|
||||
@@ -69,7 +69,7 @@ JSONL storage does not mutate live request prefixes. A resumed loop can reuse pr
|
||||
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
- **Only format versions with a complete registered upgrade path load** — the registry is empty while `SESSION_FORMAT_VERSION` remains v0. Changing compression still requires a separate/fresh root or selecting the legacy raw mode.
|
||||
- **Only the configured encoding and current `SESSION_FORMAT_VERSION` (v0) load** — changing compression requires a separate/fresh root or selecting the legacy raw mode; the pre-release format has no migration.
|
||||
- **The flat-file storage layout does not load** — use a separate root or move pre-release artifacts into the project/session directory layout before loading.
|
||||
- **Compressed files are not directly line-readable** — use the backend to load them, or select `compression: 'none'` before writing a fresh root when external line readers are required.
|
||||
- **Nothing deletes session files** — logs accumulate under `root` until removed externally (the seam has no deletion API).
|
||||
|
||||
@@ -35,7 +35,7 @@ JSONL 持久会话存储后端：`SessionPersistence` 的一个具体实现（`d
|
||||
|
||||
默认产物是独立 [Zstandard frame](../../../.agents/notes/implemented/architecture/2026-07-19-zstandard-jsonl-session-logs.zh.md) 的标准拼接：一个仅包含 header 行的带 checksum frame，后跟每个持久 append 批次一个带 checksum frame。后端使用 Node 内置 Zstandard API 和默认压缩级别，不提供级别开关。列表只读取并验证 header frame。`compression: 'none'` 在原始表示中保留相同逻辑行。
|
||||
|
||||
一个根只属于一种编码。启动发现和定向查找会拒绝相反 suffix，错误会命名不兼容产物，并指示调用方选择匹配 mode 或独立根。平铺 `<project>/<id>.jsonl*` 产物也会被拒绝，而不是忽略。Session 格式迁移可以在已配置编码内替换逻辑日志；不提供压缩迁移、混合根回退或双写。
|
||||
一个根只属于一种编码。启动发现和定向查找会拒绝相反 suffix，错误会命名不兼容产物，并指示调用方选择匹配 mode 或独立根。平铺 `<project>/<id>.jsonl*` 产物也会被拒绝，而不是忽略。不提供迁移、混合根回退或双写。
|
||||
|
||||
## 持久性与崩溃语义
|
||||
|
||||
@@ -69,7 +69,7 @@ JSONL 存储不修改实时请求前缀。只有重建历史、当前 envelope
|
||||
|
||||
## 已知限制与暂缓事项
|
||||
|
||||
- **只加载存在完整注册升级路径的格式版本**：`SESSION_FORMAT_VERSION` 保持 v0 时 registry 为空。更改压缩仍需要独立/全新根，或选择遗留原始 mode。
|
||||
- **只加载已配置编码和当前 `SESSION_FORMAT_VERSION`（v0）**：更改压缩需要独立/全新根，或选择遗留原始 mode；预发布格式没有迁移。
|
||||
- **平铺文件存储布局不加载**：加载前使用独立根，或将预发布产物移入项目/会话目录布局。
|
||||
- **压缩文件不能直接按行读取**：使用后端加载；或在写入新根前选择 `compression: 'none'`，以便外部行 reader 使用。
|
||||
- **不删除会话文件**：日志在 `root` 下累积，直到外部移除（seam 无删除接口）。
|
||||
|
||||
@@ -11,6 +11,7 @@
|
||||
import { join } from 'node:path'
|
||||
import { decodeStorageRecord, packChunkRuns, SESSION_FORMAT_VERSION } from '@deepseek-ai/dsh-session'
|
||||
import type { SessionEvent, SessionHeader, SessionId, StorageRecord } from '@deepseek-ai/dsh-session'
|
||||
import { SessionFormatUnsupportedError, sessionFormatVersionRefusal } from '@deepseek-ai/dsh-session-persistence'
|
||||
|
||||
/** Physical encoding selected for JSONL session artifacts. */
|
||||
export type JsonlCompression = 'zstd' | 'none'
|
||||
@@ -223,26 +224,29 @@ export function eventLines(events: readonly SessionEvent[], packChunks: boolean)
|
||||
}
|
||||
|
||||
interface SessionLogScan {
|
||||
meta: unknown
|
||||
events: unknown[]
|
||||
meta: SessionHeader
|
||||
events: SessionEvent[]
|
||||
committedBytes: number
|
||||
}
|
||||
|
||||
/** Parse the version-independent identity fields from one physical header row. */
|
||||
function parseStoredHeader(value: unknown): Record<string, unknown> | undefined {
|
||||
if (typeof value !== 'object' || value === null || Array.isArray(value)) return undefined
|
||||
const record = value as Record<string, unknown>
|
||||
if (record['type'] !== 'session'
|
||||
|| !Number.isSafeInteger(record['version'])) return undefined
|
||||
if (record['version'] === SESSION_FORMAT_VERSION) {
|
||||
return isHeaderLine(record) ? fromHeaderLine(record) as unknown as Record<string, unknown> : undefined
|
||||
}
|
||||
const { type: _type, ...meta } = record
|
||||
return meta
|
||||
/** Parse one complete header record supplied independently from event rows. */
|
||||
/**
|
||||
* Refuse a header carrying a format version this build does not read BEFORE
|
||||
* validating the current header shape or decoding any event row: a future
|
||||
* format need not satisfy this build's structural checks at all, and its user
|
||||
* must see "upgrade the harness", never "corrupt session log".
|
||||
* @param parsed - the JSON-parsed first line of a session artifact.
|
||||
*/
|
||||
function refuseForeignFormatVersion(parsed: unknown): void {
|
||||
if (typeof parsed !== 'object' || parsed === null) return
|
||||
const { version, id } = parsed as { version?: unknown; id?: unknown }
|
||||
if (typeof version !== 'number' || version === SESSION_FORMAT_VERSION) return
|
||||
throw new SessionFormatUnsupportedError(
|
||||
sessionFormatVersionRefusal(typeof id === 'string' ? id : String(id), version),
|
||||
)
|
||||
}
|
||||
|
||||
/** Parse one complete header record supplied independently from event rows. */
|
||||
function parseHeaderRecord(record: Buffer): unknown {
|
||||
function parseHeaderRecord(record: Buffer): SessionHeader {
|
||||
if (record.length === 0 || record.at(-1) !== 0x0A || record.indexOf(0x0A) !== record.length - 1) {
|
||||
throw new Error('empty or header-less session log')
|
||||
}
|
||||
@@ -252,11 +256,11 @@ function parseHeaderRecord(record: Buffer): unknown {
|
||||
} catch {
|
||||
throw new Error('corrupt session log: header line is not valid JSON')
|
||||
}
|
||||
const meta = parseStoredHeader(parsed)
|
||||
if (meta === undefined) {
|
||||
refuseForeignFormatVersion(parsed)
|
||||
if (!isHeaderLine(parsed)) {
|
||||
throw new Error('corrupt session log: first line is not a session header')
|
||||
}
|
||||
return meta
|
||||
return fromHeaderLine(parsed)
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -266,8 +270,8 @@ function parseHeaderRecord(record: Buffer): unknown {
|
||||
* copied because a decoder may reuse its output buffer after `write()` returns.
|
||||
*/
|
||||
export class SessionLogScanner {
|
||||
private readonly meta: unknown
|
||||
private readonly events: unknown[] = []
|
||||
private readonly meta: SessionHeader
|
||||
private readonly events: SessionEvent[] = []
|
||||
private fragments: Buffer[] = []
|
||||
private fragmentBytes = 0
|
||||
private inputBytes: number
|
||||
@@ -342,7 +346,7 @@ export class SessionLogScanner {
|
||||
/** Decode one complete event row and update the contiguous prefix. */
|
||||
private consumeEventLine(line: Buffer, endByte: number): void {
|
||||
this.eventLine += 1
|
||||
let decoded: unknown[]
|
||||
let decoded: SessionEvent[]
|
||||
try {
|
||||
decoded = decodeStorageRecord(JSON.parse(line.toString('utf8')))
|
||||
} catch {
|
||||
@@ -351,21 +355,20 @@ export class SessionLogScanner {
|
||||
}
|
||||
|
||||
if (this.issue !== undefined) {
|
||||
if (decoded.some(event => (event as { type?: unknown }).type === 'turn/end')) throw this.issue
|
||||
if (decoded.some(event => event.type === 'turn/end')) throw this.issue
|
||||
return
|
||||
}
|
||||
|
||||
const rowStart = this.events.length
|
||||
for (const event of decoded) {
|
||||
const seq = (event as { seq?: unknown }).seq
|
||||
if (seq !== this.events.length) {
|
||||
if (event.seq !== this.events.length) {
|
||||
const expected = this.events.length
|
||||
this.events.length = rowStart
|
||||
this.issue = new Error(
|
||||
`corrupt session log: seq gap in committed region at line ${this.eventLine} `
|
||||
+ `(expected ${expected}, got ${String(seq)})`,
|
||||
+ `(expected ${expected}, got ${event.seq})`,
|
||||
)
|
||||
if (decoded.some(candidate => (candidate as { type?: unknown }).type === 'turn/end')) throw this.issue
|
||||
if (decoded.some(candidate => candidate.type === 'turn/end')) throw this.issue
|
||||
return
|
||||
}
|
||||
this.events.push(event)
|
||||
@@ -391,18 +394,20 @@ export function scanLog(buffer: Buffer): SessionLogScan {
|
||||
}
|
||||
|
||||
/**
|
||||
* Parse only the version-independent identity envelope from one physical
|
||||
* header line. Format migration and current validation run in the persistence
|
||||
* decoder.
|
||||
* @param firstLine - first JSONL record without its newline.
|
||||
* @returns normalized logical header JSON, or `undefined` for invalid framing.
|
||||
* Parse just the header line of a log into a {@link SessionHeader}, or
|
||||
* `undefined` if it is missing/not a header. Used by `list()` to read session
|
||||
* metadata WITHOUT parsing the whole log: a session picker scales with the
|
||||
* number of sessions, not the total size of every conversation.
|
||||
* @param firstLine - the first line of a log file (without its trailing newline).
|
||||
* @returns the parsed header, or `undefined` when the line is not a well-formed session header.
|
||||
*/
|
||||
export function parseStoredHeaderMeta(firstLine: string): Record<string, unknown> | undefined {
|
||||
export function parseHeaderMeta(firstLine: string): SessionHeader | undefined {
|
||||
let parsed: unknown
|
||||
try {
|
||||
parsed = JSON.parse(firstLine)
|
||||
} catch {
|
||||
return undefined
|
||||
}
|
||||
return parseStoredHeader(parsed)
|
||||
if (!isHeaderLine(parsed)) return undefined
|
||||
return fromHeaderLine(parsed)
|
||||
}
|
||||
|
||||
@@ -9,43 +9,41 @@
|
||||
import { Context } from '@deepseek-ai/cordis'
|
||||
import z from '@deepseek-ai/schemastery'
|
||||
import { readdirSync } from 'node:fs'
|
||||
import { open, mkdir, readFile, readdir, realpath, link, rename, rm, stat, truncate } from 'node:fs/promises'
|
||||
import { open, mkdir, readFile, readdir, realpath, link, rm, stat, truncate } from 'node:fs/promises'
|
||||
import { dirname, join, resolve } from 'node:path'
|
||||
import { performance } from 'node:perf_hooks'
|
||||
import { scheduler } from 'node:timers/promises'
|
||||
import { randomBytes } from 'node:crypto'
|
||||
import {
|
||||
DEFAULT_PREPARED_SESSION_CACHE_SIZE, DEFAULT_WRITE_BATCH_MAX_DELAY_MS, MAX_WRITE_BATCH_DELAY_MS,
|
||||
decodeStoredSessionHeader, SessionPersistence, SessionPersistenceRevision,
|
||||
SessionPersistenceRevisionConflictError, PersistenceCoordinator,
|
||||
SessionPersistence, SessionPersistenceRevision, PersistenceCoordinator, SessionFormatUnsupportedError,
|
||||
type BorrowedSessionSource,
|
||||
type PersistenceBackend, type SessionLocation, type SessionPersistenceSnapshot,
|
||||
type SessionInspection, type SessionPersistenceRevision as PersistenceRevision, type SessionRawArtifact,
|
||||
type StoredEventRead, type StoredSessionSource,
|
||||
type SessionInspection,
|
||||
type SessionPersistenceRevision as PersistenceRevision, type SessionRawArtifact,
|
||||
type StoredPrefix,
|
||||
} from '@deepseek-ai/dsh-session-persistence'
|
||||
import { SessionId } from '@deepseek-ai/dsh-session'
|
||||
import type { Session, SessionEvent, SessionHeader, SessionPreparation } from '@deepseek-ai/dsh-session'
|
||||
import type { Session, SessionEvent, SessionId, SessionHeader, SessionPreparation } from '@deepseek-ai/dsh-session'
|
||||
import {
|
||||
encodeSegment, eventLines, logPath, logSuffix, parseStoredHeaderMeta, projectDir, scanLog, sessionDir,
|
||||
encodeSegment, eventLines, logPath, logSuffix, parseHeaderMeta, projectDir, scanLog, sessionDir,
|
||||
SessionLogScanner, toHeaderLine,
|
||||
type JsonlCompression,
|
||||
} from './format.ts'
|
||||
import {
|
||||
compressZstdFrame, createZstdFrameDecoder, decompressZstdFrame, decompressZstdPrefix, scanZstdFrames,
|
||||
} from './zstd.ts'
|
||||
import { ensureDurableDirectoryWin32, publishNewFileWin32, replaceFileWin32 } from './win32.ts'
|
||||
import { ensureDurableDirectoryWin32, publishNewFileWin32 } from './win32.ts'
|
||||
|
||||
export type { JsonlCompression } from './format.ts'
|
||||
|
||||
const DEFAULT_PACK_CHUNKS = true
|
||||
const DEFAULT_COMPRESSION: JsonlCompression = 'zstd'
|
||||
/**
|
||||
* Internal scheduling constants, not deployment configuration: decode yields
|
||||
* balance frame latency against `setImmediate` overhead; replacement batches
|
||||
* bound memory and frame granularity without changing durable behavior.
|
||||
* Internal scheduling constant, not deployment configuration: balance
|
||||
* frame-boundary event-loop yields against `setImmediate` overhead. One frame
|
||||
* remains an indivisible synchronous decode.
|
||||
*/
|
||||
const ZSTD_DECODE_YIELD_INTERVAL_MS = 500
|
||||
const REPLACEMENT_BATCH_SIZE = 128
|
||||
|
||||
/** Assert that the independently decodable first frame contains only the header record. */
|
||||
function assertZstdHeaderFrame(plaintext: Buffer): void {
|
||||
@@ -92,18 +90,6 @@ interface JsonlTornMarker {
|
||||
recoveredEvents: SessionEvent[]
|
||||
}
|
||||
|
||||
interface JsonlStoredPrefix {
|
||||
readonly meta: unknown
|
||||
readonly events: unknown[]
|
||||
readonly revision: PersistenceRevision
|
||||
readonly tornMarker?: JsonlTornMarker
|
||||
}
|
||||
|
||||
interface JsonlStoredHeader {
|
||||
readonly meta: unknown
|
||||
readonly revision: PersistenceRevision
|
||||
}
|
||||
|
||||
interface FileRevisionIdentity {
|
||||
readonly dev: bigint
|
||||
readonly ino: bigint
|
||||
@@ -217,8 +203,8 @@ export class JsonlSessionPersistence extends SessionPersistence implements Persi
|
||||
return this.coordinator.borrowSession(id, signal)
|
||||
}
|
||||
|
||||
// JSONL is sequential media: its source reader parses the stored prefix and
|
||||
// filters only after physical framing and sequence checks.
|
||||
// JSONL is sequential media: no loadStoredFrom hook, so the coordinator
|
||||
// parses the stored prefix (both encodings) and skips forward to fromSeq.
|
||||
readFrom(id: SessionId, fromSeq: number, signal?: AbortSignal): Promise<{ meta: SessionHeader; events: SessionEvent[] }> {
|
||||
return this.coordinator.readFrom(id, fromSeq, signal)
|
||||
}
|
||||
@@ -229,38 +215,14 @@ export class JsonlSessionPersistence extends SessionPersistence implements Persi
|
||||
/* jscpd:ignore-end */
|
||||
// --- PersistenceBackend hooks (the file-bytes storage primitives) ---
|
||||
|
||||
/** Open repeatable reads over one revision resolved across project directories. */
|
||||
async openStored(id: SessionId, signal?: AbortSignal): Promise<StoredSessionSource<JsonlTornMarker> | undefined> {
|
||||
/** Read a stored prefix by id across all project directories when cwd is unknown. */
|
||||
async loadStored(id: SessionId, signal?: AbortSignal): Promise<StoredPrefix<JsonlTornMarker> | undefined> {
|
||||
signal?.throwIfAborted()
|
||||
await this.ensureRootEncoding()
|
||||
signal?.throwIfAborted()
|
||||
const path = await this.findLog(id, signal)
|
||||
if (path === undefined) return undefined
|
||||
const { meta, revision } = await this.readStoredHeader(path, id, signal)
|
||||
return {
|
||||
meta,
|
||||
revision,
|
||||
location: { kind: 'jsonl', path },
|
||||
readEvents: (options = {}): StoredEventRead<JsonlTornMarker> => {
|
||||
const fromSeq = options.fromSeq ?? 0
|
||||
return this.createStoredEventRead(
|
||||
async () => {
|
||||
const prefix = await this.readPrefix(path, id, signal)
|
||||
if (prefix.revision !== revision) {
|
||||
throw new SessionPersistenceRevisionConflictError(
|
||||
`session "${id}" changed while reading revision ${revision}`,
|
||||
)
|
||||
}
|
||||
return prefix
|
||||
},
|
||||
(event) => {
|
||||
const seq = (event as { seq?: unknown }).seq
|
||||
return typeof seq !== 'number' || seq >= fromSeq
|
||||
},
|
||||
signal,
|
||||
)
|
||||
},
|
||||
}
|
||||
return this.readPrefix(path, id, signal)
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -320,11 +282,10 @@ export class JsonlSessionPersistence extends SessionPersistence implements Persi
|
||||
} else {
|
||||
content = buffer.toString('utf8')
|
||||
}
|
||||
const rawMeta = parseStoredHeaderMeta(content.split('\n', 1)[0] as string)
|
||||
if (rawMeta === undefined) {
|
||||
const meta = parseHeaderMeta(content.split('\n', 1)[0] as string)
|
||||
if (meta === undefined || meta.id !== id) {
|
||||
throw new Error(`corrupt session log: invalid header line in "${path}"`)
|
||||
}
|
||||
const meta = decodeStoredSessionHeader(rawMeta, id, { kind: 'jsonl', path })
|
||||
// The logical artifact name is `session.jsonl` regardless of the physical
|
||||
// encoding suffix (`.jsonl.zstd` marks compression only).
|
||||
return { meta, filename: 'session.jsonl', content }
|
||||
@@ -352,31 +313,6 @@ export class JsonlSessionPersistence extends SessionPersistence implements Persi
|
||||
}
|
||||
}
|
||||
|
||||
/** Read one version-independent header at a stable file revision. */
|
||||
private async readStoredHeader(
|
||||
path: string,
|
||||
_expectedId?: SessionId,
|
||||
signal?: AbortSignal,
|
||||
): Promise<JsonlStoredHeader> {
|
||||
for (;;) {
|
||||
signal?.throwIfAborted()
|
||||
const before = fileRevision(await stat(path, { bigint: true }))
|
||||
const firstLine = this.compression === 'zstd'
|
||||
? await this.readFirstZstdLine(path, signal)
|
||||
: await this.readFirstLine(path, signal)
|
||||
const after = fileRevision(await stat(path, { bigint: true }))
|
||||
if (before !== after) continue
|
||||
if (firstLine === undefined) {
|
||||
throw new Error(this.compression === 'zstd'
|
||||
? `empty or header-less Zstandard session log at "${path}"`
|
||||
: `empty or header-less session log at "${path}"`)
|
||||
}
|
||||
const meta = parseStoredHeaderMeta(firstLine)
|
||||
if (meta === undefined) throw new Error(`corrupt session log: first line is not a session header in "${path}"`)
|
||||
return { meta, revision: after }
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Read a stored prefix and convert torn-tail state to the opaque marker the
|
||||
* coordinator can round-trip without knowing the physical encoding.
|
||||
@@ -385,22 +321,32 @@ export class JsonlSessionPersistence extends SessionPersistence implements Persi
|
||||
path: string,
|
||||
expectedId?: SessionId,
|
||||
signal?: AbortSignal,
|
||||
): Promise<JsonlStoredPrefix> {
|
||||
): Promise<StoredPrefix<JsonlTornMarker>> {
|
||||
const { buffer, revision } = await this.readStableFile(path, signal)
|
||||
let prefix: Omit<JsonlStoredPrefix, 'revision'>
|
||||
if (this.compression === 'zstd') {
|
||||
prefix = await this.readZstdPrefix(buffer, signal)
|
||||
} else {
|
||||
signal?.throwIfAborted()
|
||||
const { meta, events, committedBytes } = scanLog(buffer)
|
||||
signal?.throwIfAborted()
|
||||
prefix = {
|
||||
meta,
|
||||
events,
|
||||
...committedBytes < buffer.byteLength
|
||||
? { tornMarker: { truncateTo: committedBytes, recoveredEvents: [] } }
|
||||
: {},
|
||||
let prefix: Omit<StoredPrefix<JsonlTornMarker>, 'revision'>
|
||||
try {
|
||||
if (this.compression === 'zstd') {
|
||||
prefix = await this.readZstdPrefix(buffer, signal)
|
||||
} else {
|
||||
signal?.throwIfAborted()
|
||||
const { meta, events, committedBytes } = scanLog(buffer)
|
||||
signal?.throwIfAborted()
|
||||
prefix = {
|
||||
meta,
|
||||
events,
|
||||
...committedBytes < buffer.byteLength
|
||||
? { tornMarker: { truncateTo: committedBytes, recoveredEvents: [] } }
|
||||
: {},
|
||||
}
|
||||
}
|
||||
} catch (error: unknown) {
|
||||
// A parse-time format refusal predates any SessionHeader, so the
|
||||
// coordinator's locate-based enrichment cannot run; attach the artifact
|
||||
// this read actually refused.
|
||||
if (error instanceof SessionFormatUnsupportedError && error.location === undefined) {
|
||||
throw new SessionFormatUnsupportedError(`${error.message} (raw log: ${path})`, { kind: 'jsonl', path })
|
||||
}
|
||||
throw error
|
||||
}
|
||||
signal?.throwIfAborted()
|
||||
await this.assertStoredIdentity(path, prefix.meta, expectedId, signal)
|
||||
@@ -412,7 +358,7 @@ export class JsonlSessionPersistence extends SessionPersistence implements Persi
|
||||
private async readZstdPrefix(
|
||||
buffer: Buffer,
|
||||
signal?: AbortSignal,
|
||||
): Promise<Omit<JsonlStoredPrefix, 'revision'>> {
|
||||
): Promise<Omit<StoredPrefix<JsonlTornMarker>, 'revision'>> {
|
||||
signal?.throwIfAborted()
|
||||
const { frames, tornStart } = scanZstdFrames(buffer)
|
||||
signal?.throwIfAborted()
|
||||
@@ -470,7 +416,7 @@ export class JsonlSessionPersistence extends SessionPersistence implements Persi
|
||||
events: recoveredPrefix.events,
|
||||
tornMarker: {
|
||||
truncateTo: tornStart,
|
||||
recoveredEvents: recoveredPrefix.events.slice(complete.eventCount) as SessionEvent[],
|
||||
recoveredEvents: recoveredPrefix.events.slice(complete.eventCount),
|
||||
},
|
||||
}
|
||||
} catch (error) {
|
||||
@@ -513,61 +459,6 @@ export class JsonlSessionPersistence extends SessionPersistence implements Persi
|
||||
if (tornMarker !== undefined) this.ctx.logger.warn(`${this.name}: session "${meta.id}" recovered from a torn tail; incomplete tail bytes were discarded`)
|
||||
}
|
||||
|
||||
/** Replace one exact source revision through a synced sibling and atomic namespace update. */
|
||||
async replaceStored(
|
||||
expectedRevision: PersistenceRevision,
|
||||
meta: SessionHeader,
|
||||
events: AsyncIterable<SessionEvent>,
|
||||
): Promise<void> {
|
||||
await this.ensureRootEncoding()
|
||||
const path = await this.findLog(meta.id)
|
||||
if (path === undefined) {
|
||||
throw new SessionPersistenceRevisionConflictError(
|
||||
`session "${meta.id}" no longer has revision ${expectedRevision}`,
|
||||
)
|
||||
}
|
||||
let current: JsonlStoredHeader
|
||||
try {
|
||||
current = await this.readStoredHeader(path, meta.id)
|
||||
} catch (error: unknown) {
|
||||
if (isENOENT(error)) {
|
||||
throw new SessionPersistenceRevisionConflictError(
|
||||
`session "${meta.id}" no longer has revision ${expectedRevision}`,
|
||||
)
|
||||
}
|
||||
throw error
|
||||
}
|
||||
if (current.revision !== expectedRevision) {
|
||||
throw new SessionPersistenceRevisionConflictError(
|
||||
`session "${meta.id}" changed before replacement of revision ${expectedRevision}`,
|
||||
)
|
||||
}
|
||||
const currentIdentity = this.storedIdentity(current.meta, path)
|
||||
if (meta.cwd !== currentIdentity.cwd) {
|
||||
throw new Error(`replacement for session "${meta.id}" changes its stored identity`)
|
||||
}
|
||||
|
||||
const tmp = `${path}.${randomBytes(6).toString('hex')}.upgrade.tmp`
|
||||
try {
|
||||
await this.writeReplacement(tmp, meta, events)
|
||||
const beforeCommit = fileRevision(await stat(path, { bigint: true }))
|
||||
if (beforeCommit !== expectedRevision) {
|
||||
throw new SessionPersistenceRevisionConflictError(
|
||||
`session "${meta.id}" changed before replacement of revision ${expectedRevision}`,
|
||||
)
|
||||
}
|
||||
/* v8 ignore next -- Windows uses its write-through replacement primitive. */
|
||||
if (process.platform === 'win32') {
|
||||
await replaceFileWin32(tmp, path)
|
||||
} else {
|
||||
await rename(tmp, path)
|
||||
await this.syncDirPosix(dirname(path))
|
||||
}
|
||||
} finally {
|
||||
await rm(tmp, { force: true })
|
||||
}
|
||||
}
|
||||
|
||||
/** List valid unique stored sessions' metadata (header line only — no full-log parse). */
|
||||
async list(signal?: AbortSignal): Promise<SessionHeader[]> {
|
||||
return (await this.listArtifacts(signal)).map(artifact => artifact.header)
|
||||
@@ -618,15 +509,9 @@ export class JsonlSessionPersistence extends SessionPersistence implements Persi
|
||||
: await this.readFirstLine(path, signal)
|
||||
signal?.throwIfAborted()
|
||||
if (first === undefined) continue // empty/half-written file
|
||||
const rawMeta = parseStoredHeaderMeta(first)
|
||||
if (rawMeta === undefined) continue // not a session header
|
||||
const rawId = rawMeta['id']
|
||||
const expectedId = typeof rawId === 'string'
|
||||
? SessionId(rawId)
|
||||
: SessionId('')
|
||||
const meta = decodeStoredSessionHeader(rawMeta, expectedId, { kind: 'jsonl', path })
|
||||
this.storedIdentity(rawMeta, path)
|
||||
await this.assertStoredIdentity(path, rawMeta, undefined, signal)
|
||||
const meta = parseHeaderMeta(first)
|
||||
if (meta === undefined) continue // not a session header
|
||||
await this.assertStoredIdentity(path, meta, undefined, signal)
|
||||
signal?.throwIfAborted()
|
||||
if (ids.has(meta.id)) {
|
||||
throw new Error(`duplicate JSONL session id "${meta.id}" appears in multiple project directories`)
|
||||
@@ -746,34 +631,6 @@ export class JsonlSessionPersistence extends SessionPersistence implements Persi
|
||||
return tmp
|
||||
}
|
||||
|
||||
/** Stream one complete current-format replacement into a synced temp file. */
|
||||
private async writeReplacement(
|
||||
path: string,
|
||||
meta: SessionHeader,
|
||||
events: AsyncIterable<SessionEvent>,
|
||||
): Promise<void> {
|
||||
const handle = await open(path, 'wx', 0o600)
|
||||
try {
|
||||
const header = JSON.stringify(toHeaderLine(meta)) + '\n'
|
||||
await handle.writeFile(this.compression === 'zstd' ? await compressZstdFrame(header) : header)
|
||||
let batch: SessionEvent[] = []
|
||||
const writeBatch = async (): Promise<void> => {
|
||||
if (batch.length === 0) return
|
||||
const body = eventLines(batch, this.packChunks) + '\n'
|
||||
await handle.writeFile(this.compression === 'zstd' ? await compressZstdFrame(body) : body)
|
||||
batch = []
|
||||
}
|
||||
for await (const event of events) {
|
||||
batch.push(event)
|
||||
if (batch.length === REPLACEMENT_BATCH_SIZE) await writeBatch()
|
||||
}
|
||||
await writeBatch()
|
||||
await handle.sync()
|
||||
} finally {
|
||||
await handle.close()
|
||||
}
|
||||
}
|
||||
|
||||
/** Encode the header and first batch without combining their frame boundaries. */
|
||||
private async encodeMaterialization(meta: SessionHeader, events: readonly SessionEvent[]): Promise<Buffer | string> {
|
||||
const header = JSON.stringify(toHeaderLine(meta)) + '\n'
|
||||
@@ -969,45 +826,26 @@ export class JsonlSessionPersistence extends SessionPersistence implements Persi
|
||||
/** Reject metadata that does not identify the selected physical log. */
|
||||
private async assertStoredIdentity(
|
||||
path: string,
|
||||
meta: unknown,
|
||||
meta: SessionHeader,
|
||||
expectedId?: SessionId,
|
||||
signal?: AbortSignal,
|
||||
): Promise<void> {
|
||||
signal?.throwIfAborted()
|
||||
const identity = this.storedIdentity(meta, path)
|
||||
if (expectedId !== undefined && identity.id !== expectedId) {
|
||||
throw new Error(`corrupt session log "${path}": requested id "${expectedId}" does not match header id "${identity.id}"`)
|
||||
if (expectedId !== undefined && meta.id !== expectedId) {
|
||||
throw new Error(`corrupt session log "${path}": requested id "${expectedId}" does not match header id "${meta.id}"`)
|
||||
}
|
||||
let expectedPath: string
|
||||
try {
|
||||
expectedPath = logPath(this.root, identity.cwd, identity.id, this.compression)
|
||||
expectedPath = logPath(this.root, meta.cwd, meta.id, this.compression)
|
||||
} catch (error) {
|
||||
throw new Error(`corrupt session log "${path}": header id cannot name a storage path`, { cause: error })
|
||||
}
|
||||
if (path !== expectedPath && !await this.sameFile(path, expectedPath, signal)) {
|
||||
throw new Error(`corrupt session log "${path}": header id "${identity.id}" and cwd identify "${expectedPath}"`)
|
||||
throw new Error(`corrupt session log "${path}": header id "${meta.id}" and cwd identify "${expectedPath}"`)
|
||||
}
|
||||
signal?.throwIfAborted()
|
||||
}
|
||||
|
||||
/** Read storage identity fields shared by every Session format version. */
|
||||
private storedIdentity(meta: unknown, path: string): { id: SessionId; cwd?: string } {
|
||||
if (typeof meta !== 'object' || meta === null || Array.isArray(meta)) {
|
||||
throw new Error(`corrupt session log "${path}": header is not a record`)
|
||||
}
|
||||
const record = meta as Record<string, unknown>
|
||||
if (typeof record['id'] !== 'string') {
|
||||
throw new Error(`corrupt session log "${path}": header id is not a string`)
|
||||
}
|
||||
if (record['cwd'] !== undefined && typeof record['cwd'] !== 'string') {
|
||||
throw new Error(`corrupt session log "${path}": header cwd is not a string`)
|
||||
}
|
||||
return {
|
||||
id: SessionId(record['id']),
|
||||
...typeof record['cwd'] === 'string' ? { cwd: record['cwd'] } : {},
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether two path spellings resolve to the same physical file. This admits
|
||||
* case aliases on case-insensitive filesystems without weakening identity
|
||||
|
||||
@@ -28,7 +28,6 @@ interface Win32ErrnoException extends NodeJS.ErrnoException {
|
||||
}
|
||||
|
||||
const MOVEFILE_WRITE_THROUGH = 0x00000008
|
||||
const MOVEFILE_REPLACE_EXISTING = 0x00000001
|
||||
const ERROR_FILE_NOT_FOUND = 2
|
||||
const ERROR_PATH_NOT_FOUND = 3
|
||||
const ERROR_ACCESS_DENIED = 5
|
||||
@@ -120,19 +119,6 @@ export async function publishNewFileWin32(existing: string, replacement: string)
|
||||
if (ok === 0) throw win32Error('MoveFileExW', api.getLastError(), existing, replacement)
|
||||
}
|
||||
|
||||
/**
|
||||
* Atomically replace an existing file with a synced staging file and request
|
||||
* write-through namespace durability. The move stays within one volume.
|
||||
* @param existing - synced staging path to move.
|
||||
* @param replacement - existing final path to replace.
|
||||
*/
|
||||
export async function replaceFileWin32(existing: string, replacement: string): Promise<void> {
|
||||
const api = await win32()
|
||||
const flags = MOVEFILE_REPLACE_EXISTING | MOVEFILE_WRITE_THROUGH
|
||||
const ok = api.moveFileExW(toNamespacedPath(existing), toNamespacedPath(replacement), flags)
|
||||
if (ok === 0) throw win32Error('MoveFileExW', api.getLastError(), existing, replacement)
|
||||
}
|
||||
|
||||
/**
|
||||
* Create `target` and its missing ancestors with durable Windows namespace
|
||||
* publication. Each missing directory is first created as a random staging
|
||||
|
||||
@@ -8,12 +8,7 @@ import SessionStore, { SessionId } from '@deepseek-ai/dsh-session'
|
||||
import type { Session, SessionEvent, SessionHeader } from '@deepseek-ai/dsh-session'
|
||||
import JsonlSessionPersistence from '@deepseek-ai/dsh-session-persistence-jsonl'
|
||||
import {
|
||||
SessionPersistenceRevisionConflictError,
|
||||
type StoredEventRead,
|
||||
} from '@deepseek-ai/dsh-session-persistence'
|
||||
import {
|
||||
encodeSegment, eventLines, fromHeaderLine, logPath, parseStoredHeaderMeta, projectDir, projectKey, scanLog, sessionDir,
|
||||
SessionLogScanner, toHeaderLine,
|
||||
encodeSegment, eventLines, logPath, projectDir, projectKey, scanLog, sessionDir, SessionLogScanner, toHeaderLine,
|
||||
} from '../src/format.ts'
|
||||
import { runPersistenceContract, meta, oneTurnLog, appendLog } from '../../session-persistence/tests/contract.ts'
|
||||
import { runCoordinatorContract, type CoordinatorFixture } from '../../session-persistence/tests/coordinator-contract.ts'
|
||||
@@ -21,8 +16,6 @@ import { runCoordinatorContract, type CoordinatorFixture } from '../../session-p
|
||||
const statRace = vi.hoisted(() => ({
|
||||
path: undefined as string | undefined,
|
||||
reads: 0,
|
||||
renamePath: undefined as string | undefined,
|
||||
renameError: undefined as Error | undefined,
|
||||
}))
|
||||
|
||||
vi.mock('node:fs/promises', async (importOriginal) => {
|
||||
@@ -36,25 +29,6 @@ vi.mock('node:fs/promises', async (importOriginal) => {
|
||||
if (statRace.reads !== 2) return identity
|
||||
return { ...identity, mtimeNs: identity.mtimeNs + 1n }
|
||||
}) as typeof actual.stat,
|
||||
rename: async (...args: Parameters<typeof actual.rename>) => {
|
||||
if (String(args[1]) === statRace.renamePath && statRace.renameError !== undefined) {
|
||||
throw statRace.renameError
|
||||
}
|
||||
return actual.rename(...args)
|
||||
},
|
||||
}
|
||||
})
|
||||
|
||||
vi.mock('../src/win32.ts', async (importOriginal) => {
|
||||
const actual = await importOriginal<typeof import('../src/win32.ts')>()
|
||||
return {
|
||||
...actual,
|
||||
replaceFileWin32: async (existing: string, replacement: string) => {
|
||||
if (replacement === statRace.renamePath && statRace.renameError !== undefined) {
|
||||
throw statRace.renameError
|
||||
}
|
||||
return actual.replaceFileWin32(existing, replacement)
|
||||
},
|
||||
}
|
||||
})
|
||||
|
||||
@@ -68,17 +42,6 @@ function mutableHeader(header: SessionHeader): MutableSessionHeader {
|
||||
return header
|
||||
}
|
||||
|
||||
async function collectStoredRead(read: StoredEventRead<unknown>): Promise<unknown[]> {
|
||||
const events: unknown[] = []
|
||||
for await (const event of read.events) events.push(event)
|
||||
await read.completed
|
||||
return events
|
||||
}
|
||||
|
||||
async function* replacementEvents(events: readonly SessionEvent[]): AsyncIterable<SessionEvent> {
|
||||
for (const event of events) yield structuredClone(event)
|
||||
}
|
||||
|
||||
/** Rewrite only a stored header while preserving every event byte below it. */
|
||||
async function rewriteHeader(path: string, update: (header: Record<string, unknown>) => void): Promise<void> {
|
||||
const lines = (await readFile(path, 'utf8')).split('\n')
|
||||
@@ -123,8 +86,6 @@ function rawLogPath(root: string, cwd: string | undefined, id: SessionId): strin
|
||||
afterEach(async () => {
|
||||
statRace.path = undefined
|
||||
statRace.reads = 0
|
||||
statRace.renamePath = undefined
|
||||
statRace.renameError = undefined
|
||||
vi.restoreAllMocks()
|
||||
for (const d of dirs.splice(0)) await rm(d, { recursive: true, force: true })
|
||||
})
|
||||
@@ -172,18 +133,6 @@ runCoordinatorContract('jsonl-none', async (): Promise<CoordinatorFixture> => {
|
||||
})
|
||||
|
||||
describe('JsonlSessionPersistence: format helpers', () => {
|
||||
it('parses only the version-independent stored header envelope', () => {
|
||||
expect(parseStoredHeaderMeta('{')).toBeUndefined()
|
||||
expect(parseStoredHeaderMeta('42')).toBeUndefined()
|
||||
expect(parseStoredHeaderMeta(JSON.stringify({ type: 'event', version: 9, id: 'wrong-type' })))
|
||||
.toBeUndefined()
|
||||
expect(parseStoredHeaderMeta(JSON.stringify({ type: 'session', version: 9, id: 'future', futureOnly: true })))
|
||||
.toEqual({ version: 9, id: 'future', futureOnly: true })
|
||||
expect(parseStoredHeaderMeta(JSON.stringify({
|
||||
type: 'session', version: 0, id: 'current', createdAt: 1, delegationDepth: 0,
|
||||
}))).toEqual({ version: 0, id: 'current', createdAt: 1, delegationDepth: 0 })
|
||||
})
|
||||
|
||||
it('encodeSegment neutralizes traversal, separators, and absolute paths', () => {
|
||||
expect(encodeSegment('..')).toBe('~002E~002E')
|
||||
expect(encodeSegment('.')).toBe('~002E')
|
||||
@@ -370,8 +319,7 @@ describe('JsonlSessionPersistence: durability and crash semantics', () => {
|
||||
expect(raw!.content).toBe(await readFile(rawLogPath(root, '/work', m.id), 'utf8'))
|
||||
expect(raw!.content.split('\n')[0]).toBe(JSON.stringify(toHeaderLine(m)))
|
||||
const scanned = scanLog(Buffer.from(raw!.content))
|
||||
expect(scanned.events.map(event => (event as SessionEvent).type))
|
||||
.toEqual(oneTurnLog().map(event => event.type))
|
||||
expect(scanned.events.map(event => event.type)).toEqual(oneTurnLog().map(event => event.type))
|
||||
})
|
||||
|
||||
it('readRaw is undefined for an absent session', async () => {
|
||||
@@ -469,261 +417,28 @@ describe('JsonlSessionPersistence: durability and crash semantics', () => {
|
||||
await otherCtx.fiber.dispose()
|
||||
})
|
||||
|
||||
it('binds a stored source to the same revision as a lightweight read', async () => {
|
||||
it('binds a full stored prefix to the same revision as a lightweight read', async () => {
|
||||
const m = meta('stored-prefix-revision')
|
||||
await ctx.sessionPersistence.create(m)
|
||||
await ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
const persistence = ctx.sessionPersistence as JsonlSessionPersistence
|
||||
|
||||
const stored = await persistence.openStored(m.id)
|
||||
const stored = await persistence.loadStored(m.id)
|
||||
expect(stored?.revision).toBe(await persistence.readStoredRevision(m.id))
|
||||
expect(await persistence.readStoredRevision(SessionId('missing-revision'))).toBeUndefined()
|
||||
})
|
||||
|
||||
it('retries a revision-bound source read when the file changes during the read', async () => {
|
||||
it('retries a full-prefix read when the file revision changes during the read', async () => {
|
||||
const m = meta('stored-prefix-revision-race')
|
||||
await ctx.sessionPersistence.create(m)
|
||||
await ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
const persistence = ctx.sessionPersistence as JsonlSessionPersistence
|
||||
const stored = await persistence.openStored(m.id)
|
||||
if (stored === undefined) throw new Error('test session must be materialized')
|
||||
statRace.path = rawLogPath(root, m.cwd, m.id)
|
||||
|
||||
await expect(collectStoredRead(stored.readEvents())).resolves.toEqual(oneTurnLog())
|
||||
await expect(persistence.loadStored(m.id)).resolves.toMatchObject({ events: oneTurnLog() })
|
||||
expect(statRace.reads).toBe(4)
|
||||
})
|
||||
|
||||
it('rejects a revision-bound source after a complete append changes its revision', async () => {
|
||||
const m = meta('stored-source-stale')
|
||||
await ctx.sessionPersistence.create(m)
|
||||
await ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
const persistence = ctx.sessionPersistence as JsonlSessionPersistence
|
||||
const stored = await persistence.openStored(m.id)
|
||||
if (stored === undefined) throw new Error('test session must be materialized')
|
||||
|
||||
await ctx.sessionPersistence.append(m.id, [
|
||||
{ type: 'turn/start', seq: oneTurnLog().length, time: 7, data: { turn: 2 } },
|
||||
])
|
||||
|
||||
const read = stored.readEvents()
|
||||
const completion = read.completed.catch((error: unknown) => error)
|
||||
await expect(collectStoredRead(read)).rejects.toBeInstanceOf(SessionPersistenceRevisionConflictError)
|
||||
await expect(completion).resolves.toBeInstanceOf(SessionPersistenceRevisionConflictError)
|
||||
})
|
||||
|
||||
it('retries a header read whose revision changes around the first-line read', async () => {
|
||||
const m = meta('stored-header-revision-race')
|
||||
await ctx.sessionPersistence.create(m)
|
||||
await ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
const persistence = ctx.sessionPersistence as JsonlSessionPersistence
|
||||
const path = rawLogPath(root, m.cwd, m.id)
|
||||
const internals = persistence as unknown as {
|
||||
findLog(id: SessionId): Promise<string | undefined>
|
||||
}
|
||||
vi.spyOn(internals, 'findLog').mockResolvedValue(path)
|
||||
statRace.path = path
|
||||
|
||||
await expect(persistence.openStored(m.id)).resolves.toMatchObject({ meta: { id: m.id } })
|
||||
expect(statRace.reads).toBe(4)
|
||||
})
|
||||
|
||||
it('reports a present empty plaintext artifact as header-less', async () => {
|
||||
const m = meta('empty-plaintext-log')
|
||||
await ctx.sessionPersistence.create(m)
|
||||
await ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
await writeFile(rawLogPath(root, m.cwd, m.id), '')
|
||||
const persistence = ctx.sessionPersistence as JsonlSessionPersistence
|
||||
|
||||
await expect(persistence.openStored(m.id))
|
||||
.rejects.toThrow('empty or header-less session log')
|
||||
})
|
||||
|
||||
it('forwards prepare through the concrete backend API', async () => {
|
||||
const m = meta('jsonl-prepare-forward')
|
||||
await ctx.sessionPersistence.create(m)
|
||||
await ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
const persistence = ctx.sessionPersistence as JsonlSessionPersistence
|
||||
|
||||
const preparation = await persistence.prepare(m.id)
|
||||
expect(preparation.session.id).toBe(m.id)
|
||||
preparation[Symbol.dispose]()
|
||||
})
|
||||
|
||||
it('atomically replaces one exact revision and rejects a stale replacement', async () => {
|
||||
const m = meta('format-replace', '/work')
|
||||
const original = [
|
||||
...oneTurnLog(),
|
||||
{ type: 'turn/start', seq: 6, time: 7, data: { turn: 2 } },
|
||||
{ type: 'turn/end', seq: 7, time: 8, data: { turn: 2, reason: { kind: 'completed' } } },
|
||||
] as SessionEvent[]
|
||||
await ctx.sessionPersistence.create(m)
|
||||
await ctx.sessionPersistence.append(m.id, original)
|
||||
const persistence = ctx.sessionPersistence as JsonlSessionPersistence
|
||||
const source = await persistence.openStored(m.id)
|
||||
if (source === undefined) throw new Error('test session must be materialized')
|
||||
|
||||
await persistence.replaceStored(source.revision, m, replacementEvents(oneTurnLog()))
|
||||
const replaced = await persistence.openStored(m.id)
|
||||
if (replaced === undefined) throw new Error('replacement must preserve the session')
|
||||
expect(replaced.revision).not.toBe(source.revision)
|
||||
expect(await collectStoredRead(replaced.readEvents())).toEqual(oneTurnLog())
|
||||
|
||||
await expect(
|
||||
persistence.replaceStored(source.revision, m, replacementEvents(original)),
|
||||
).rejects.toBeInstanceOf(SessionPersistenceRevisionConflictError)
|
||||
const afterConflict = await persistence.openStored(m.id)
|
||||
if (afterConflict === undefined) throw new Error('conflict must preserve the session')
|
||||
expect(await collectStoredRead(afterConflict.readEvents())).toEqual(oneTurnLog())
|
||||
})
|
||||
|
||||
it('preserves the old complete log when atomic replacement rename fails', async () => {
|
||||
const m = meta('format-replace-rename-failure', '/work')
|
||||
await ctx.sessionPersistence.create(m)
|
||||
await ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
const persistence = ctx.sessionPersistence as JsonlSessionPersistence
|
||||
const source = await persistence.openStored(m.id)
|
||||
if (source === undefined) throw new Error('test session must be materialized')
|
||||
const path = rawLogPath(root, m.cwd, m.id)
|
||||
const failure = new Error('simulated format replacement rename failure')
|
||||
statRace.renamePath = path
|
||||
statRace.renameError = failure
|
||||
|
||||
await expect(
|
||||
persistence.replaceStored(source.revision, m, replacementEvents([])),
|
||||
).rejects.toBe(failure)
|
||||
|
||||
statRace.renameError = undefined
|
||||
const preserved = await persistence.openStored(m.id)
|
||||
if (preserved === undefined) throw new Error('failed replacement must preserve the session')
|
||||
expect(await collectStoredRead(preserved.readEvents())).toEqual(oneTurnLog())
|
||||
expect((await readdir(dirname(path))).some(name => name.endsWith('.upgrade.tmp'))).toBe(false)
|
||||
})
|
||||
|
||||
it('rejects replacement when the artifact disappears before or after discovery', async () => {
|
||||
const m = meta('format-replace-disappeared', '/work')
|
||||
await ctx.sessionPersistence.create(m)
|
||||
await ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
const persistence = ctx.sessionPersistence as JsonlSessionPersistence
|
||||
const source = await persistence.openStored(m.id)
|
||||
if (source === undefined) throw new Error('test session must be materialized')
|
||||
const path = rawLogPath(root, m.cwd, m.id)
|
||||
|
||||
await rm(path)
|
||||
await expect(persistence.replaceStored(source.revision, m, replacementEvents([])))
|
||||
.rejects.toBeInstanceOf(SessionPersistenceRevisionConflictError)
|
||||
|
||||
const internals = persistence as unknown as {
|
||||
findLog(id: SessionId): Promise<string | undefined>
|
||||
}
|
||||
vi.spyOn(internals, 'findLog').mockResolvedValue(path)
|
||||
await expect(persistence.replaceStored(source.revision, m, replacementEvents([])))
|
||||
.rejects.toBeInstanceOf(SessionPersistenceRevisionConflictError)
|
||||
})
|
||||
|
||||
it('propagates a non-absence error while rechecking a replacement source', async () => {
|
||||
const m = meta('format-replace-header-error', '/work')
|
||||
await ctx.sessionPersistence.create(m)
|
||||
await ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
const persistence = ctx.sessionPersistence as JsonlSessionPersistence
|
||||
const source = await persistence.openStored(m.id)
|
||||
if (source === undefined) throw new Error('test session must be materialized')
|
||||
const failure = new Error('simulated header read failure')
|
||||
const internals = persistence as unknown as {
|
||||
readStoredHeader(path: string, id: SessionId): Promise<unknown>
|
||||
}
|
||||
vi.spyOn(internals, 'readStoredHeader').mockRejectedValue(failure)
|
||||
|
||||
await expect(persistence.replaceStored(source.revision, m, replacementEvents([])))
|
||||
.rejects.toBe(failure)
|
||||
})
|
||||
|
||||
it('rejects a replacement that changes cwd storage identity', async () => {
|
||||
const m = meta('format-replace-identity', '/work')
|
||||
await ctx.sessionPersistence.create(m)
|
||||
await ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
const persistence = ctx.sessionPersistence as JsonlSessionPersistence
|
||||
const source = await persistence.openStored(m.id)
|
||||
if (source === undefined) throw new Error('test session must be materialized')
|
||||
|
||||
await expect(persistence.replaceStored(
|
||||
source.revision,
|
||||
{ ...m, cwd: '/other' },
|
||||
replacementEvents(oneTurnLog()),
|
||||
)).rejects.toThrow(/changes its stored identity/)
|
||||
})
|
||||
|
||||
it('rejects a replacement when the source changes after the temp file is synced', async () => {
|
||||
const m = meta('format-replace-final-cas', '/work')
|
||||
await ctx.sessionPersistence.create(m)
|
||||
await ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
const persistence = ctx.sessionPersistence as JsonlSessionPersistence
|
||||
const source = await persistence.openStored(m.id)
|
||||
if (source === undefined) throw new Error('test session must be materialized')
|
||||
const path = rawLogPath(root, m.cwd, m.id)
|
||||
const internals = persistence as unknown as {
|
||||
writeReplacement(path: string, meta: SessionHeader, events: AsyncIterable<SessionEvent>): Promise<void>
|
||||
}
|
||||
const writeReplacement = internals.writeReplacement.bind(internals)
|
||||
vi.spyOn(internals, 'writeReplacement').mockImplementation(async (...args) => {
|
||||
await writeReplacement(...args)
|
||||
await appendFile(path, '\n')
|
||||
})
|
||||
|
||||
await expect(persistence.replaceStored(source.revision, m, replacementEvents(oneTurnLog())))
|
||||
.rejects.toBeInstanceOf(SessionPersistenceRevisionConflictError)
|
||||
expect((await readdir(dirname(path))).some(name => name.endsWith('.upgrade.tmp'))).toBe(false)
|
||||
})
|
||||
|
||||
it('streams replacement events in bounded batches', async () => {
|
||||
const m = meta('format-replace-batches', '/work')
|
||||
const events = Array.from({ length: 128 }, (_, seq): SessionEvent => ({
|
||||
type: 'turn/start', seq, time: seq + 1, data: { turn: seq + 1 },
|
||||
}))
|
||||
await ctx.sessionPersistence.create(m)
|
||||
await ctx.sessionPersistence.append(m.id, oneTurnLog())
|
||||
const persistence = ctx.sessionPersistence as JsonlSessionPersistence
|
||||
const source = await persistence.openStored(m.id)
|
||||
if (source === undefined) throw new Error('test session must be materialized')
|
||||
|
||||
await persistence.replaceStored(source.revision, m, replacementEvents(events))
|
||||
|
||||
const replaced = await persistence.openStored(m.id)
|
||||
if (replaced === undefined) throw new Error('replacement must preserve the session')
|
||||
expect(await collectStoredRead(replaced.readEvents())).toHaveLength(128)
|
||||
})
|
||||
|
||||
it('rejects malformed version-independent storage identity fields', async () => {
|
||||
const persistence = ctx.sessionPersistence as JsonlSessionPersistence
|
||||
const path = rawLogPath(root, '/work', SessionId('identity-fields'))
|
||||
const internals = persistence as unknown as {
|
||||
storedIdentity(meta: unknown, path: string): { id: SessionId; cwd?: string }
|
||||
}
|
||||
|
||||
expect(() => internals.storedIdentity(null, path)).toThrow(/header is not a record/)
|
||||
expect(() => internals.storedIdentity({ id: 1 }, path)).toThrow(/header id is not a string/)
|
||||
expect(() => internals.storedIdentity({ id: 'identity-fields', cwd: 1 }, path))
|
||||
.toThrow(/header cwd is not a string/)
|
||||
})
|
||||
|
||||
it('rejects a physical log whose requested id differs from its header id', async () => {
|
||||
const persistence = ctx.sessionPersistence as JsonlSessionPersistence
|
||||
const requested = SessionId('requested-identity')
|
||||
const path = rawLogPath(root, '/work', requested)
|
||||
const internals = persistence as unknown as {
|
||||
assertStoredIdentity(
|
||||
path: string,
|
||||
meta: unknown,
|
||||
expectedId?: SessionId,
|
||||
): Promise<void>
|
||||
}
|
||||
|
||||
await expect(internals.assertStoredIdentity(
|
||||
path,
|
||||
{ id: 'different-identity', cwd: '/work' },
|
||||
requested,
|
||||
)).rejects.toThrow(/requested id .* does not match header id/)
|
||||
})
|
||||
|
||||
it('handles revision-stat races and errors after log discovery', async () => {
|
||||
const m = meta('stored-revision-race')
|
||||
await ctx.sessionPersistence.create(m)
|
||||
@@ -1053,7 +768,7 @@ describe('JsonlSessionPersistence: durability and crash semantics', () => {
|
||||
const beforeB = await readFile(bPath)
|
||||
|
||||
await expect(ctx.sessionPersistence.load(a.id))
|
||||
.rejects.toThrow(/identity mismatch: requested "identity-a", header contains "identity-b"/)
|
||||
.rejects.toThrow(/requested id "identity-a" does not match header id "identity-b"/)
|
||||
expect(await readFile(aPath)).toEqual(beforeA)
|
||||
expect(await readFile(bPath)).toEqual(beforeB)
|
||||
})
|
||||
@@ -1244,20 +959,7 @@ describe('JsonlSessionPersistence: scanLog unit', () => {
|
||||
|
||||
// The preset decides the resumed session's tools and prompt; dropping it
|
||||
// on disk would restore a composition the logged history contradicts.
|
||||
expect((scanLog(Buffer.from(log)).meta as SessionHeader).agentPreset).toBe('minimal')
|
||||
})
|
||||
|
||||
it('round-trips and validates a subagent origin', () => {
|
||||
const header: SessionHeader = {
|
||||
...meta('subagent-origin'),
|
||||
delegationDepth: 1,
|
||||
origin: 'subagent',
|
||||
}
|
||||
const line = toHeaderLine(header)
|
||||
|
||||
expect(fromHeaderLine(line)).toEqual(header)
|
||||
expect(() => scanLog(Buffer.from(`${JSON.stringify({ ...line, origin: 'parent' })}\n`)))
|
||||
.toThrow(/session header/)
|
||||
expect(scanLog(Buffer.from(log)).meta.agentPreset).toBe('minimal')
|
||||
})
|
||||
|
||||
it('rejects a session header whose agentPreset is not a string', () => {
|
||||
@@ -1275,7 +977,7 @@ describe('JsonlSessionPersistence: scanLog unit', () => {
|
||||
// No committed turn/end, so the gap is a tolerated crash boundary: scanLog PRESERVES the
|
||||
// contiguous prefix (turn/start seq 0) — real interrupted-turn work, not discarded — and
|
||||
// stops at the gap. `loadCore`, not this scanner, later closes the orphaned turn.
|
||||
expect(scanLog(Buffer.from(log)).events.map(e => (e as SessionEvent).seq)).toEqual([0])
|
||||
expect(scanLog(Buffer.from(log)).events.map(e => e.seq)).toEqual([0])
|
||||
})
|
||||
|
||||
it('rejects a seq gap BEFORE a later committed turn/end (committed data damaged)', () => {
|
||||
@@ -1315,7 +1017,7 @@ describe('JsonlSessionPersistence: scanLog unit', () => {
|
||||
].join('\n') + '\n'
|
||||
// The contiguous prefix (turn/start seq 0) is preserved; the corrupt
|
||||
// fragment after it is the tolerated crash boundary.
|
||||
expect(scanLog(Buffer.from(log)).events.map(e => (e as SessionEvent).seq)).toEqual([0])
|
||||
expect(scanLog(Buffer.from(log)).events.map(e => e.seq)).toEqual([0])
|
||||
})
|
||||
|
||||
it('tolerates a seq gap AFTER a turn/end (uncommitted tail)', () => {
|
||||
@@ -1326,7 +1028,7 @@ describe('JsonlSessionPersistence: scanLog unit', () => {
|
||||
JSON.stringify({ type: 'step/start', seq: 9, time: 3, data: { turn: 2, step: 1 } }), // gap in uncommitted tail
|
||||
].join('\n') + '\n'
|
||||
const { events } = scanLog(Buffer.from(log))
|
||||
expect(events.map(e => (e as SessionEvent).seq)).toEqual([0, 1]) // tail dropped
|
||||
expect(events.map(e => e.seq)).toEqual([0, 1]) // tail dropped
|
||||
})
|
||||
})
|
||||
|
||||
@@ -1447,7 +1149,7 @@ describe('JsonlSessionPersistence: default packed chunk rows', () => {
|
||||
JSON.stringify({ type: 'turn/end', seq: 4, time: 5, data: { turn: 1, reason: { kind: 'completed' } } }),
|
||||
].join('\n') + '\n'
|
||||
const { events } = scanLog(Buffer.from(logText))
|
||||
expect(events.map(e => (e as SessionEvent).seq)).toEqual([0, 1, 2, 3, 4])
|
||||
expect(events.map(e => e.seq)).toEqual([0, 1, 2, 3, 4])
|
||||
expect(events[2]).toEqual({ type: 'assistant/chunk', seq: 2, time: 3, data: { turn: 1, step: 1, chunk: { type: 'text-delta', index: 0, text: 'b' } } })
|
||||
})
|
||||
|
||||
@@ -1469,7 +1171,7 @@ describe('JsonlSessionPersistence: default packed chunk rows', () => {
|
||||
JSON.stringify({ type: 'text-chunks', seq0: 2, time0: 2, data: { turn: 1, step: 1, index: 0, dt: [1, 1], texts: ['a', 'b', 'c'] } }),
|
||||
].join('\n') + '\n'
|
||||
const scanned = scanLog(Buffer.from(logText))
|
||||
expect(scanned.events.map(e => (e as SessionEvent).seq)).toEqual([0])
|
||||
expect(scanned.events.map(e => e.seq)).toEqual([0])
|
||||
// committedBytes stays on the line boundary BEFORE the dropped row.
|
||||
const headerAndTurn = logText.split('\n').slice(0, 2).join('\n') + '\n'
|
||||
expect(scanned.committedBytes).toBe(Buffer.byteLength(headerAndTurn, 'utf8'))
|
||||
@@ -1544,23 +1246,6 @@ describe('JsonlSessionPersistence: edge cases', () => {
|
||||
expect(await ctx.sessionPersistence.list()).toEqual([])
|
||||
})
|
||||
|
||||
it('listing refuses a future format before validating current identity fields', async () => {
|
||||
const id = SessionId('future-list')
|
||||
const path = rawLogPath(root, '/work', id)
|
||||
await mkdir(dirname(path), { recursive: true })
|
||||
await writeFile(path, `${JSON.stringify({ type: 'session', version: 42, id: 123 })}\n`)
|
||||
|
||||
for (const list of [
|
||||
() => ctx.sessionPersistence.list(),
|
||||
() => ctx.sessionPersistence.listSnapshots(),
|
||||
]) {
|
||||
const failure = await list().then(() => undefined, (error: unknown) => error as Error)
|
||||
expect(failure?.name).toBe('SessionFormatUnsupportedError')
|
||||
expect(failure?.message).toContain('session "123" uses log format v42')
|
||||
expect(failure?.message).toMatch(/written by a newer harness.*upgrade the harness/)
|
||||
}
|
||||
})
|
||||
|
||||
it('keeps the transcript in an extensible session-owned directory', async () => {
|
||||
const m = meta('owned-directory', '/project')
|
||||
await ctx.sessionPersistence.create(m)
|
||||
@@ -1728,7 +1413,7 @@ describe('JsonlSessionPersistence: edge cases', () => {
|
||||
// The "/w" log is untouched — no no-cwd events were grafted onto it, and no
|
||||
// `_no-cwd` log for "x" was created.
|
||||
const inW = scanLog(await readFile(rawLogPath(root, '/w', SessionId('x'))))
|
||||
expect((inW.meta as SessionHeader).cwd).toBe('/w')
|
||||
expect(inW.meta.cwd).toBe('/w')
|
||||
expect(inW.events).toHaveLength(6)
|
||||
await expect(stat(rawLogPath(root, undefined, SessionId('x')))).rejects.toThrow()
|
||||
await ctx2.fiber.dispose()
|
||||
|
||||
@@ -11,7 +11,6 @@ import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
|
||||
const MOVEFILE_WRITE_THROUGH = 0x00000008
|
||||
const MOVEFILE_REPLACE_EXISTING = 0x00000001
|
||||
const ERROR_FILE_NOT_FOUND = 2
|
||||
const ERROR_PATH_NOT_FOUND = 3
|
||||
const ERROR_ACCESS_DENIED = 5
|
||||
@@ -91,17 +90,6 @@ async function importWithFilesystemMove(): Promise<typeof import('../src/win32.t
|
||||
})
|
||||
}
|
||||
|
||||
async function importWithFilesystemReplace(): Promise<typeof import('../src/win32.ts')> {
|
||||
return importWithMove((existing, replacement, flags, setLastError) => {
|
||||
expect(flags).toBe(MOVEFILE_REPLACE_EXISTING | MOVEFILE_WRITE_THROUGH)
|
||||
const from = stripNamespace(existing)
|
||||
const to = stripNamespace(replacement)
|
||||
if (!existsSync(from)) { setLastError(ERROR_FILE_NOT_FOUND); return 0 }
|
||||
renameSync(from, to)
|
||||
return 1
|
||||
})
|
||||
}
|
||||
|
||||
afterEach(async () => {
|
||||
vi.doUnmock('koffi')
|
||||
vi.doUnmock('node:fs/promises')
|
||||
@@ -153,30 +141,6 @@ describe('Windows durable namespace helpers', () => {
|
||||
expect(readFileSync(final, 'utf8')).toBe('content')
|
||||
})
|
||||
|
||||
it('replaces an existing file with write-through MoveFileExW semantics', async () => {
|
||||
const { replaceFileWin32 } = await importWithFilesystemReplace()
|
||||
const root = await tempRoot()
|
||||
const tmp = join(root, 'log.tmp')
|
||||
const final = join(root, 'log.jsonl')
|
||||
await writeFile(tmp, 'replacement')
|
||||
await writeFile(final, 'original')
|
||||
|
||||
await replaceFileWin32(tmp, final)
|
||||
expect(existsSync(tmp)).toBe(false)
|
||||
expect(readFileSync(final, 'utf8')).toBe('replacement')
|
||||
})
|
||||
|
||||
it('maps a Win32 replacement failure to a Node-style error', async () => {
|
||||
const { replaceFileWin32 } = await importWithError(ERROR_ACCESS_DENIED)
|
||||
|
||||
await expect(replaceFileWin32('from', 'to')).rejects.toMatchObject({
|
||||
code: 'EACCES',
|
||||
win32Code: ERROR_ACCESS_DENIED,
|
||||
path: 'from',
|
||||
dest: 'to',
|
||||
})
|
||||
})
|
||||
|
||||
it('maps Win32 publish failures to Node-style errno codes', async () => {
|
||||
const cases = [
|
||||
[ERROR_FILE_NOT_FOUND, 'ENOENT'],
|
||||
|
||||
@@ -332,27 +332,6 @@ describe('Zstandard frame structure', () => {
|
||||
})
|
||||
|
||||
describe('JsonlSessionPersistence: default Zstandard encoding', () => {
|
||||
it('atomically replaces a stored revision with compressed header and event frames', async () => {
|
||||
const root = await freshRoot()
|
||||
const ctx = await mount(root)
|
||||
const header = meta('replace-zstd', '/work')
|
||||
await ctx.sessionPersistence.create(header)
|
||||
await ctx.sessionPersistence.append(header.id, oneTurnLog())
|
||||
const persistence = ctx.sessionPersistence as JsonlSessionPersistence
|
||||
const source = await persistence.openStored(header.id)
|
||||
if (source === undefined) throw new Error('test session must be materialized')
|
||||
const replacement = oneTurnLog().slice(0, 2)
|
||||
|
||||
await persistence.replaceStored(source.revision, header, (async function* () {
|
||||
yield* replacement
|
||||
})())
|
||||
|
||||
const buffer = await readFile(logPath(root, header.cwd, header.id, 'zstd'))
|
||||
expect(scanZstdFrames(buffer).frames).toHaveLength(2)
|
||||
const plaintext = (await decodeCompleteFrames(buffer)).toString()
|
||||
expect(scanLog(Buffer.from(plaintext)).events).toEqual(replacement)
|
||||
})
|
||||
|
||||
it('materializes an explicitly durable empty session as one header frame', async () => {
|
||||
const root = await freshRoot()
|
||||
const ctx = await mount(root)
|
||||
@@ -408,8 +387,7 @@ describe('JsonlSessionPersistence: default Zstandard encoding', () => {
|
||||
'',
|
||||
].join('\n'))
|
||||
const scanned = scanLog(Buffer.from(raw!.content))
|
||||
expect(scanned.events.map(event => (event as SessionEvent).type))
|
||||
.toEqual(oneTurnLog().map(event => event.type))
|
||||
expect(scanned.events.map(event => event.type)).toEqual(oneTurnLog().map(event => event.type))
|
||||
})
|
||||
|
||||
it('readRaw rejects a present zstd artifact that carries no frame', async () => {
|
||||
@@ -422,32 +400,6 @@ describe('JsonlSessionPersistence: default Zstandard encoding', () => {
|
||||
await writeFile(logPath(root, '/work', header.id, 'zstd'), Buffer.alloc(0))
|
||||
await expect(ctx.sessionPersistence.readRaw(header.id))
|
||||
.rejects.toThrow('empty or header-less Zstandard session log')
|
||||
await expect(ctx.sessionPersistence.load(header.id))
|
||||
.rejects.toThrow('empty or header-less Zstandard session log')
|
||||
})
|
||||
|
||||
it('rejects a zero-frame artifact through an already-open stored reader', async () => {
|
||||
const root = await freshRoot()
|
||||
const ctx = await mount(root)
|
||||
const header = meta('stored-zero-frame', '/work')
|
||||
await ctx.sessionPersistence.create(header)
|
||||
await ctx.sessionPersistence.append(header.id, oneTurnLog())
|
||||
const persistence = ctx.sessionPersistence as JsonlSessionPersistence
|
||||
const source = await persistence.openStored(header.id)
|
||||
if (source === undefined) throw new Error('test session must be materialized')
|
||||
await writeFile(logPath(root, '/work', header.id, 'zstd'), Buffer.alloc(0))
|
||||
|
||||
const read = source.readEvents()
|
||||
const completion = read.completed.catch((error: unknown) => error)
|
||||
const consumption = (async (): Promise<void> => {
|
||||
for await (const _event of read.events) {
|
||||
// A zero-frame artifact cannot yield a logical event.
|
||||
}
|
||||
})().catch((error: unknown) => error)
|
||||
const [streamFailure, completionFailure] = await Promise.all([consumption, completion])
|
||||
|
||||
expect(streamFailure).toBe(completionFailure)
|
||||
expect(streamFailure).toMatchObject({ message: 'empty or header-less Zstandard session log' })
|
||||
})
|
||||
|
||||
it('resolves the default when a programmatic wrapper bypasses Loader schema normalization', async () => {
|
||||
@@ -790,7 +742,7 @@ describe('JsonlSessionPersistence: encoding selection', () => {
|
||||
'',
|
||||
].join('\n'))
|
||||
await expect(ctx.sessionPersistence.load(loadHeader.id)).rejects.toThrow(/uses \.jsonl/)
|
||||
await expect((ctx.sessionPersistence as JsonlSessionPersistence).openStored(loadHeader.id))
|
||||
await expect((ctx.sessionPersistence as JsonlSessionPersistence).loadStored(loadHeader.id))
|
||||
.rejects.toThrow(/uses \.jsonl/)
|
||||
await expect(ctx.sessionPersistence.list()).rejects.toThrow(/uses \.jsonl/)
|
||||
})
|
||||
|
||||
@@ -10,21 +10,17 @@ import { lstat, mkdir, open } from 'node:fs/promises'
|
||||
import { dirname, resolve } from 'node:path'
|
||||
import type { DatabaseSync, StatementSync } from 'node:sqlite'
|
||||
import {
|
||||
SessionId,
|
||||
type SessionEvent,
|
||||
type SessionHeader,
|
||||
type SessionId,
|
||||
} from '@deepseek-ai/dsh-session'
|
||||
import {
|
||||
createStoredEventRead,
|
||||
decodeStoredSessionHeader,
|
||||
SessionPersistenceRevision,
|
||||
SessionPersistenceRevisionConflictError,
|
||||
type PersistenceBackend,
|
||||
type SessionPersistenceRevision as PersistenceRevision,
|
||||
type SessionPersistenceSnapshot,
|
||||
type StoredEventRead,
|
||||
type StoredEventReadOptions,
|
||||
type StoredSessionSource,
|
||||
type StoredPrefix,
|
||||
type StoredSuffix,
|
||||
} from '@deepseek-ai/dsh-session-persistence'
|
||||
import {
|
||||
MAX_PACKED_ROW_MEMBERS,
|
||||
@@ -56,21 +52,6 @@ export interface SqliteStoreOptions {
|
||||
readonly busyTimeoutMs: number
|
||||
}
|
||||
|
||||
/** A stored session's header, valid event prefix, and revision at one snapshot. */
|
||||
interface SqliteStoredPrefix {
|
||||
readonly meta: SessionHeader
|
||||
readonly events: SessionEvent[]
|
||||
readonly revision: PersistenceRevision
|
||||
readonly tornMarker?: number
|
||||
}
|
||||
|
||||
/** A stored session's suffix (events at or past a seq) and its snapshot revision. */
|
||||
interface SqliteStoredSuffix {
|
||||
readonly meta: SessionHeader
|
||||
readonly events: SessionEvent[]
|
||||
readonly revision: PersistenceRevision
|
||||
}
|
||||
|
||||
/** SQLite implementation of the coordinator's physical backend hooks. */
|
||||
export class SqliteStore implements PersistenceBackend<number> {
|
||||
readonly name = 'session-persistence-sqlite'
|
||||
@@ -150,13 +131,7 @@ export class SqliteStore implements PersistenceBackend<number> {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Load one row's complete validated prefix at a single snapshot.
|
||||
* @param id - persisted session id to resolve.
|
||||
* @param signal - optional cancellation for backend read work.
|
||||
* @returns the stored prefix, or `undefined` when the session has no stored row.
|
||||
*/
|
||||
async loadStored(id: SessionId, signal?: AbortSignal): Promise<SqliteStoredPrefix | undefined> {
|
||||
async loadStored(id: SessionId, signal?: AbortSignal): Promise<StoredPrefix<number> | undefined> {
|
||||
await this.observe(signal)
|
||||
const snapshot = this.readTransaction(() => {
|
||||
const row = this.rowFor(id)
|
||||
@@ -182,14 +157,7 @@ export class SqliteStore implements PersistenceBackend<number> {
|
||||
return row === undefined ? undefined : sqliteRevision(this.storeIdentity, row)
|
||||
}
|
||||
|
||||
/**
|
||||
* Load one row's physical suffix at or past a sequence at a single snapshot.
|
||||
* @param id - persisted session id to resolve.
|
||||
* @param fromSeq - first physical event sequence to include.
|
||||
* @param signal - optional cancellation for backend read work.
|
||||
* @returns the stored suffix, or `undefined` when the session has no stored row.
|
||||
*/
|
||||
async loadStoredFrom(id: SessionId, fromSeq: number, signal?: AbortSignal): Promise<SqliteStoredSuffix | undefined> {
|
||||
async loadStoredFrom(id: SessionId, fromSeq: number, signal?: AbortSignal): Promise<StoredSuffix | undefined> {
|
||||
await this.observe(signal)
|
||||
const snapshot = this.readTransaction(() => {
|
||||
const row = this.rowFor(id)
|
||||
@@ -199,48 +167,7 @@ export class SqliteStore implements PersistenceBackend<number> {
|
||||
signal?.throwIfAborted()
|
||||
if (snapshot === undefined) return undefined
|
||||
const { preserved } = scanRows(snapshot.eventRows, snapshot.base)
|
||||
return {
|
||||
meta: rowToMeta(snapshot.row),
|
||||
events: preserved.filter(event => event.seq >= fromSeq),
|
||||
revision: sqliteRevision(this.storeIdentity, snapshot.row),
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Open repeatable reads over one row revision. Each event reader reproduces
|
||||
* this revision or rejects when a concurrent writer changed the row.
|
||||
* @param id - persisted session id to resolve.
|
||||
* @param signal - optional cancellation for backend read work.
|
||||
* @returns the source, or `undefined` when the session has no stored row.
|
||||
*/
|
||||
async openStored(id: SessionId, signal?: AbortSignal): Promise<StoredSessionSource<number> | undefined> {
|
||||
await this.observe(signal)
|
||||
const row = this.rowFor(id)
|
||||
signal?.throwIfAborted()
|
||||
if (row === undefined) return undefined
|
||||
const revision = sqliteRevision(this.storeIdentity, row)
|
||||
return {
|
||||
meta: rowToMeta(row),
|
||||
revision,
|
||||
readEvents: (options: StoredEventReadOptions = {}): StoredEventRead<number> => {
|
||||
const fromSeq = options.fromSeq ?? 0
|
||||
return createStoredEventRead(
|
||||
async () => {
|
||||
const stored = fromSeq === 0
|
||||
? await this.loadStored(id, signal)
|
||||
: await this.loadStoredFrom(id, fromSeq, signal)
|
||||
if (stored === undefined || stored.revision !== revision) {
|
||||
throw new SessionPersistenceRevisionConflictError(
|
||||
`session "${id}" changed while reading revision ${revision}`,
|
||||
)
|
||||
}
|
||||
return stored
|
||||
},
|
||||
() => true,
|
||||
signal,
|
||||
)
|
||||
},
|
||||
}
|
||||
return { meta: rowToMeta(snapshot.row), events: preserved.filter(event => event.seq >= fromSeq) }
|
||||
}
|
||||
|
||||
async appendBatch(
|
||||
@@ -324,64 +251,11 @@ export class SqliteStore implements PersistenceBackend<number> {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Atomically replace one exact stored revision with a complete current log.
|
||||
* The streamed events are staged in memory, then the swap commits in one
|
||||
* transaction that rechecks the revision and storage identity.
|
||||
* @param expectedRevision - exact source revision decoded by the caller.
|
||||
* @param meta - complete current-format header.
|
||||
* @param events - complete current-format event stream.
|
||||
*/
|
||||
async replaceStored(
|
||||
expectedRevision: PersistenceRevision,
|
||||
meta: SessionHeader,
|
||||
events: AsyncIterable<SessionEvent>,
|
||||
): Promise<void> {
|
||||
await this.open()
|
||||
const observed = this.rowFor(meta.id)
|
||||
if (observed === undefined
|
||||
|| sqliteRevision(this.storeIdentity, observed) !== expectedRevision) {
|
||||
throw new SessionPersistenceRevisionConflictError(
|
||||
`session "${meta.id}" changed before replacement of revision ${expectedRevision}`,
|
||||
)
|
||||
}
|
||||
if (meta.cwd !== (observed.cwd ?? undefined)) {
|
||||
throw new Error(`replacement for session "${meta.id}" changes its stored identity`)
|
||||
}
|
||||
// Stage the complete replacement before the swap transaction so a failed
|
||||
// or cancelled stream leaves the stored log untouched.
|
||||
const staged: SessionEvent[] = []
|
||||
for await (const event of events) staged.push(event)
|
||||
const records = packChunkRuns(staged)
|
||||
this.db.exec(sql('begin-immediate'))
|
||||
try {
|
||||
validateSchemaForMutation(this.databaseConstructor, this.db, this.databasePath)
|
||||
const row = this.rowFor(meta.id)
|
||||
if (row === undefined
|
||||
|| sqliteRevision(this.storeIdentity, row) !== expectedRevision) {
|
||||
throw new SessionPersistenceRevisionConflictError(
|
||||
`session "${meta.id}" changed before replacement of revision ${expectedRevision}`,
|
||||
)
|
||||
}
|
||||
if (meta.cwd !== (row.cwd ?? undefined)) {
|
||||
throw new Error(`replacement for session "${meta.id}" changes its stored identity`)
|
||||
}
|
||||
this.db.prepare(sql('delete-events-from')).run(meta.id, 0)
|
||||
const insert = this.insertStatement()
|
||||
for (const record of records) this.insertRecord(insert, meta.id, bindRecord(record))
|
||||
this.writeRow(meta)
|
||||
this.incrementRevision(meta.id)
|
||||
this.db.exec(sql('commit'))
|
||||
} catch (error: unknown) {
|
||||
this.rollback(error, 'replacement')
|
||||
}
|
||||
}
|
||||
|
||||
async list(signal?: AbortSignal): Promise<SessionHeader[]> {
|
||||
await this.observe(signal)
|
||||
const rows = this.sessionRows()
|
||||
signal?.throwIfAborted()
|
||||
return rows.map(row => decodeStoredSessionHeader(rowToMeta(row), SessionId(row.id)))
|
||||
return rows.map(rowToMeta)
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -394,7 +268,7 @@ export class SqliteStore implements PersistenceBackend<number> {
|
||||
const rows = this.sessionRows()
|
||||
signal?.throwIfAborted()
|
||||
return rows.map(row => ({
|
||||
header: decodeStoredSessionHeader(rowToMeta(row), SessionId(row.id)),
|
||||
header: rowToMeta(row),
|
||||
revision: sqliteRevision(this.storeIdentity, row),
|
||||
}))
|
||||
}
|
||||
|
||||
-3
@@ -1,3 +0,0 @@
|
||||
SELECT COUNT(*) AS n
|
||||
FROM events
|
||||
WHERE session_id = ?;
|
||||
-5
@@ -1,5 +0,0 @@
|
||||
CREATE TEMP TRIGGER fail_format_replace
|
||||
BEFORE UPDATE ON sessions
|
||||
BEGIN
|
||||
SELECT RAISE(ABORT, 'simulated format replacement failure');
|
||||
END
|
||||
-2
@@ -1,2 +0,0 @@
|
||||
DELETE FROM sessions
|
||||
WHERE id = ?;
|
||||
-1
@@ -1 +0,0 @@
|
||||
DROP TRIGGER fail_format_replace;
|
||||
@@ -1,3 +0,0 @@
|
||||
UPDATE sessions
|
||||
SET cwd = ?
|
||||
WHERE id = ?;
|
||||
-3
@@ -1,3 +0,0 @@
|
||||
UPDATE sessions
|
||||
SET revision = revision + 1
|
||||
WHERE id = ?;
|
||||
@@ -15,14 +15,12 @@ import SessionPersistenceSqlite, {
|
||||
DEFAULT_BUSY_TIMEOUT_MS,
|
||||
SCHEMA_VERSION,
|
||||
} from '@deepseek-ai/dsh-session-persistence-sqlite'
|
||||
import { SessionPersistenceRevisionConflictError } from '@deepseek-ai/dsh-session-persistence'
|
||||
import {
|
||||
runCoordinatorContract,
|
||||
type CoordinatorFixture,
|
||||
} from '../../session-persistence/tests/coordinator-contract.ts'
|
||||
import {
|
||||
meta,
|
||||
oneTurnLog,
|
||||
runPersistenceContract,
|
||||
} from '../../session-persistence/tests/contract.ts'
|
||||
import { MAX_PACKED_DATA_BYTES } from '../src/codec.ts'
|
||||
@@ -201,11 +199,6 @@ async function measureWriteTraffic(
|
||||
}
|
||||
}
|
||||
|
||||
/** Yield immutable event copies as one replacement stream. */
|
||||
async function* replacementEvents(events: readonly SessionEvent[]): AsyncIterable<SessionEvent> {
|
||||
for (const event of events) yield structuredClone(event)
|
||||
}
|
||||
|
||||
runPersistenceContract('sqlite', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
@@ -861,157 +854,3 @@ describe('SessionPersistenceSqlite edge behavior', () => {
|
||||
await store.close()
|
||||
})
|
||||
})
|
||||
|
||||
describe('SessionPersistenceSqlite stored-source and replacement primitives', () => {
|
||||
it('binds a stored source to the same revision as a lightweight read', async () => {
|
||||
const path = await freshDbPath()
|
||||
const store = new SqliteStore({ path, journalMode: 'wal', busyTimeoutMs: DEFAULT_BUSY_TIMEOUT_MS })
|
||||
const m = meta('stored-prefix-revision')
|
||||
await store.appendBatch(m, oneTurnLog(), false)
|
||||
|
||||
const stored = await store.openStored(m.id)
|
||||
expect(stored?.revision).toBe(await store.readStoredRevision(m.id))
|
||||
expect(await store.readStoredRevision(SessionId('missing-revision'))).toBeUndefined()
|
||||
await store.close()
|
||||
})
|
||||
|
||||
it('rejects revision-bound full and suffix readers after the row changes or disappears', async () => {
|
||||
const path = await freshDbPath()
|
||||
const store = new SqliteStore({ path, journalMode: 'wal', busyTimeoutMs: DEFAULT_BUSY_TIMEOUT_MS })
|
||||
const m = meta('stored-reader-conflict')
|
||||
await store.appendBatch(m, oneTurnLog(), false)
|
||||
const changed = await store.openStored(m.id)
|
||||
if (changed === undefined) throw new Error('test session must be materialized')
|
||||
await store.appendBatch(m, [
|
||||
{ type: 'turn/start', seq: oneTurnLog().length, time: 7, data: { turn: 2 } },
|
||||
], true)
|
||||
const changedRead = changed.readEvents()
|
||||
const changedCompletion = changedRead.completed.catch((error: unknown) => error)
|
||||
await expect((async () => { for await (const _event of changedRead.events) { /* consume */ } })())
|
||||
.rejects.toBeInstanceOf(SessionPersistenceRevisionConflictError)
|
||||
await expect(changedCompletion).resolves.toBeInstanceOf(SessionPersistenceRevisionConflictError)
|
||||
|
||||
const removed = await store.openStored(m.id)
|
||||
if (removed === undefined) throw new Error('test session must remain materialized')
|
||||
const db = (store as unknown as { db: DatabaseSync }).db
|
||||
db.prepare(testSql('delete-session-by-id')).run(m.id)
|
||||
const removedRead = removed.readEvents({ fromSeq: 1 })
|
||||
const removedCompletion = removedRead.completed.catch((error: unknown) => error)
|
||||
await expect((async () => { for await (const _event of removedRead.events) { /* consume */ } })())
|
||||
.rejects.toBeInstanceOf(SessionPersistenceRevisionConflictError)
|
||||
await expect(removedCompletion).resolves.toBeInstanceOf(SessionPersistenceRevisionConflictError)
|
||||
await store.close()
|
||||
})
|
||||
|
||||
it('rolls back a suffix snapshot when its SQL read fails and reports absent direct snapshots', async () => {
|
||||
const path = await freshDbPath()
|
||||
const store = new SqliteStore({ path, journalMode: 'wal', busyTimeoutMs: DEFAULT_BUSY_TIMEOUT_MS })
|
||||
expect(await store.loadStored(SessionId('missing-prefix'))).toBeUndefined()
|
||||
expect(await store.loadStoredFrom(SessionId('missing-suffix'), 1)).toBeUndefined()
|
||||
|
||||
const m = meta('suffix-rollback')
|
||||
await store.appendBatch(m, oneTurnLog(), false)
|
||||
const db = (store as unknown as { db: DatabaseSync }).db
|
||||
const prepare = db.prepare.bind(db)
|
||||
const spy = vi.spyOn(db, 'prepare').mockImplementation((source) => {
|
||||
if (source.includes('seq >= ?')) throw new Error('simulated suffix SELECT failure')
|
||||
return prepare(source)
|
||||
})
|
||||
await expect(store.loadStoredFrom(m.id, 1)).rejects.toThrow('simulated suffix SELECT failure')
|
||||
spy.mockRestore()
|
||||
expect((db.prepare(testSql('count-session-events')).get(m.id) as { n: number }).n)
|
||||
.toBe(oneTurnLog().length)
|
||||
await store.close()
|
||||
})
|
||||
|
||||
it('atomically replaces one exact revision and rejects a stale replacement', async () => {
|
||||
const path = await freshDbPath()
|
||||
const store = new SqliteStore({ path, journalMode: 'wal', busyTimeoutMs: DEFAULT_BUSY_TIMEOUT_MS })
|
||||
const m = meta('format-replace')
|
||||
const original = [
|
||||
...oneTurnLog(),
|
||||
{ type: 'turn/start', seq: oneTurnLog().length, time: 7, data: { turn: 2 } },
|
||||
{ type: 'turn/end', seq: oneTurnLog().length + 1, time: 8, data: { turn: 2, reason: { kind: 'completed' } } },
|
||||
] as SessionEvent[]
|
||||
await store.appendBatch(m, original, false)
|
||||
const source = await store.openStored(m.id)
|
||||
if (source === undefined) throw new Error('test session must be materialized')
|
||||
|
||||
await store.replaceStored(source.revision, m, replacementEvents(oneTurnLog()))
|
||||
const replaced = await store.openStored(m.id)
|
||||
if (replaced === undefined) throw new Error('replacement must preserve the session')
|
||||
expect(replaced.revision).not.toBe(source.revision)
|
||||
expect((await store.loadStored(m.id))?.events).toEqual(oneTurnLog())
|
||||
|
||||
await expect(
|
||||
store.replaceStored(source.revision, m, replacementEvents(original)),
|
||||
).rejects.toBeInstanceOf(SessionPersistenceRevisionConflictError)
|
||||
expect((await store.loadStored(m.id))?.events).toEqual(oneTurnLog())
|
||||
await store.close()
|
||||
})
|
||||
|
||||
it('rejects replacement identity changes before and during the transaction', async () => {
|
||||
const path = await freshDbPath()
|
||||
const store = new SqliteStore({ path, journalMode: 'wal', busyTimeoutMs: DEFAULT_BUSY_TIMEOUT_MS })
|
||||
const m = meta('format-replace-identity', '/work')
|
||||
await store.appendBatch(m, oneTurnLog(), false)
|
||||
const source = await store.openStored(m.id)
|
||||
if (source === undefined) throw new Error('test session must be materialized')
|
||||
|
||||
await expect(store.replaceStored(
|
||||
source.revision,
|
||||
{ ...m, cwd: '/other' },
|
||||
replacementEvents(oneTurnLog()),
|
||||
)).rejects.toThrow(/changes its stored identity/)
|
||||
|
||||
const db = (store as unknown as { db: DatabaseSync }).db
|
||||
const changesDuringStaging = (async function* (): AsyncIterable<SessionEvent> {
|
||||
yield* oneTurnLog()
|
||||
db.prepare(testSql('update-session-cwd')).run('/raced', m.id)
|
||||
})()
|
||||
await expect(store.replaceStored(source.revision, m, changesDuringStaging))
|
||||
.rejects.toThrow(/changes its stored identity/)
|
||||
expect((db.prepare(testSql('count-session-events')).get(m.id) as { n: number }).n)
|
||||
.toBe(oneTurnLog().length)
|
||||
await store.close()
|
||||
})
|
||||
|
||||
it('rejects a revision change that occurs while replacement events are staged', async () => {
|
||||
const path = await freshDbPath()
|
||||
const store = new SqliteStore({ path, journalMode: 'wal', busyTimeoutMs: DEFAULT_BUSY_TIMEOUT_MS })
|
||||
const m = meta('format-replace-staging-race', '/work')
|
||||
await store.appendBatch(m, oneTurnLog(), false)
|
||||
const source = await store.openStored(m.id)
|
||||
if (source === undefined) throw new Error('test session must be materialized')
|
||||
const db = (store as unknown as { db: DatabaseSync }).db
|
||||
const changesDuringStaging = (async function* (): AsyncIterable<SessionEvent> {
|
||||
yield* oneTurnLog()
|
||||
db.prepare(testSql('update-session-revision')).run(m.id)
|
||||
})()
|
||||
|
||||
await expect(store.replaceStored(source.revision, m, changesDuringStaging))
|
||||
.rejects.toBeInstanceOf(SessionPersistenceRevisionConflictError)
|
||||
expect((db.prepare(testSql('count-session-events')).get(m.id) as { n: number }).n)
|
||||
.toBe(oneTurnLog().length)
|
||||
await store.close()
|
||||
})
|
||||
|
||||
it('rolls back the complete replacement when the transaction fails after it begins', async () => {
|
||||
const path = await freshDbPath()
|
||||
const store = new SqliteStore({ path, journalMode: 'wal', busyTimeoutMs: DEFAULT_BUSY_TIMEOUT_MS })
|
||||
const m = meta('format-replace-rollback')
|
||||
await store.appendBatch(m, oneTurnLog(), false)
|
||||
const source = await store.openStored(m.id)
|
||||
if (source === undefined) throw new Error('test session must be materialized')
|
||||
const db = (store as unknown as { db: DatabaseSync }).db
|
||||
db.exec(testSql('create-temp-replace-trigger'))
|
||||
|
||||
await expect(
|
||||
store.replaceStored(source.revision, m, replacementEvents([])),
|
||||
).rejects.toThrow(/simulated format replacement failure/)
|
||||
db.exec(testSql('drop-temp-replace-trigger'))
|
||||
|
||||
expect((await store.loadStored(m.id))?.events).toEqual(oneTurnLog())
|
||||
await store.close()
|
||||
})
|
||||
})
|
||||
|
||||
@@ -8,14 +8,10 @@ export type TestSqlName =
|
||||
| 'count-ignorable-events'
|
||||
| 'count-packed-events'
|
||||
| 'count-physical-types'
|
||||
| 'count-session-events'
|
||||
| 'create-loose-schema'
|
||||
| 'create-temp-replace-trigger'
|
||||
| 'create-unrelated-table'
|
||||
| 'delete-persistence-state'
|
||||
| 'delete-session-by-id'
|
||||
| 'delete-session-events'
|
||||
| 'drop-temp-replace-trigger'
|
||||
| 'empty-store-id'
|
||||
| 'insert-corrupt-event'
|
||||
| 'measure-write-traffic'
|
||||
@@ -29,8 +25,6 @@ export type TestSqlName =
|
||||
| 'set-user-version-16'
|
||||
| 'set-user-version-17'
|
||||
| 'update-invalid-session-metadata'
|
||||
| 'update-session-cwd'
|
||||
| 'update-session-revision'
|
||||
|
||||
/** Load one fixed test SQL resource. */
|
||||
export function testSql(name: TestSqlName): string {
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/session/session-persistence/README.md
|
||||
README.md: 323d7b23cff6438264ae4aa4a3fecbd06a832037
|
||||
README.zh.md: bb667f6989f1a0df9d258d223f0f6721a233433a
|
||||
README.md: 76df109936070e0dd7afb18e98c6c94855be4f21
|
||||
README.zh.md: 6c366bf8287e4c96052935e46410fd27d28714f5
|
||||
|
||||
@@ -17,9 +17,9 @@ The persisted unit IS the existing `SessionEvent` (event-sourced model — the l
|
||||
| `ensureMaterialized(session): Promise<void>` | Explicitly make an exact live session durable even with zero events, without inventing an event. Lifecycle frontends use this only when the empty session itself is a resumable resource; ordinary creation remains lazy. |
|
||||
| `append(id, events): Promise<void>` | Durably persist a batch. Append-only; first event `seq` == stored next-seq after any repair; rejects non-JSON-serializable data naming the offending type. |
|
||||
| `prepare(id, signal?): Promise<SessionPreparation>` | Reserve the exact unpublished Session used by resume. A coordinator reuses an earlier inspection when available, commits pending recovery, and releases an unpublished reservation back to its bounded cache on disposal. |
|
||||
| `load(id): Promise<{ meta; events }>` | Return an immutable balanced logical log after decoding a supported format path and committing any format replacement plus cold recovery. A live load first flushes its snapshot and rejects while its turn is open; a cold load preserves an interrupted final turn and durably closes it with synthetic `tool/result`/`step/end?`/`turn/end {interrupted}` events. Only a torn tail fragment is dropped; committed corruption and malformed records reject as `SessionPersistenceCorruptionError`, while an unsupported format `version` or an event type unknown to this build (without the envelope's `ignorable` marker) refuses as `SessionFormatUnsupportedError`, naming the refusal direction and the raw log path when the backend keeps one artifact per session. |
|
||||
| `load(id): Promise<{ meta; events }>` | Return an immutable balanced logical log after converting supported older records from the same format version and committing cold recovery. A live load first flushes its snapshot and rejects while its turn is open; a cold load preserves an interrupted final turn and durably closes it with synthetic `tool/result`/`step/end?`/`turn/end {interrupted}` events. Only a torn tail fragment is dropped; committed corruption and malformed records reject as `SessionPersistenceCorruptionError`, while an unsupported format `version` or an event type unknown to this build (without the envelope's `ignorable` marker) refuses as `SessionFormatUnsupportedError`, naming the refusal direction and the raw log path when the backend keeps one artifact per session. |
|
||||
| `inspect(id, signal?): Promise<{ meta; events }>` | Return an upgraded, validated, deeply frozen logical view without committing recovery or publishing a Session. A cold view receives in-memory synthetic recovery closers while its physical torn tail remains untouched; an already-live view is its current immutable snapshot and may contain an open turn. Coordinator-backed implementations retain the exact cold unpublished Session in a bounded LRU for later `prepare`, but discard and reload it when the stored revision changes. Same-id inspections share an in-flight read. |
|
||||
| `readFrom(id, fromSeq, signal?): Promise<{ meta; events }>` | Return valid stored events with `seq >= fromSeq` without preparation caching, truncation, closers, or coordinator state. A `fromSeq` at or past the stored end returns an empty event list; a negative or non-safe-integer `fromSeq` rejects. Current-format reads request a suffix from the backend; a format migration requires the complete source and applies `fromSeq` only after migration. Sequential media may still scan framing before filtering, while seek-capable media can avoid reading earlier rows. Intended for checkpoint consumers that apply only events after a stored sequence number. |
|
||||
| `readFrom(id, fromSeq, signal?): Promise<{ meta; events }>` | Return valid stored events with `seq >= fromSeq` without preparation caching, truncation, closers, or coordinator state. A `fromSeq` at or past the stored end returns an empty event list; a negative or non-safe-integer `fromSeq` rejects. Seek-capable backends (SQLite) read only the suffix unless converting a supported older record requires earlier records; sequential backends (JSONL) parse the whole artifact and skip forward. Unknown-type refusal follows that access pattern: a seek read checks only the returned suffix, while the sequential fallback also refuses on an unknown required event below the window. Intended for checkpoint consumers that apply only events after a stored sequence number. |
|
||||
| `list(signal?): Promise<SessionHeader[]>` | Lightweight listing from metadata, no full-log parse. The optional signal cancels backend listing work. A zero-event session is absent until a consumer explicitly materializes it. |
|
||||
| `listSnapshots(signal?): Promise<SessionPersistenceSnapshot[]>` | Lightweight metadata plus an opaque branded per-log revision, without loading event logs. A revision stays equal while that log and its backing store are unchanged, changes after append or mutating load repair, and cannot collide solely because two stores use the same local counter. The optional signal requests cancellation of backend discovery work; first-party backends settle any started listing work before rejecting so an awaited call is quiescent. |
|
||||
|
||||
@@ -36,15 +36,9 @@ The persisted unit IS the existing `SessionEvent` (event-sourced model — the l
|
||||
|
||||
Each `session/event` copies its event into the session controller. The first pending event starts a fixed batching window; later events join without resetting its deadline. The configured `writeBatchMaxDelayMs` bounds this intentional wait, not event-loop, initialization, serialized-operation, or backend latency. Events admitted during a write form a new bounded batch. `session/flush` cancels the wait and is a shared quiescence barrier that drains events admitted while it runs. A background failure is logged once, retains the ordered batch, and pauses automatic retry; a new event starts a fresh window, while explicit flush or backend teardown retries immediately and surfaces a repeated failure.
|
||||
|
||||
Crash repair is cold-only. For a live id, `load(id)` snapshots the authoritative in-memory log, waits for that snapshot to become durable, and returns it only when balanced; an open live turn rejects instead of receiving synthetic interruption closers. For a cold id, inspection reads, validates, freezes, and constructs one unpublished Session; repeated inspection reuses that object graph only while its source revision remains current. `prepare(id)` performs the same check before repair, reserves the exact Session, commits any pending torn-tail/interrupted-turn repair, and returns it for publication. HMR adoption opens the same revision-bound source, applies the coordinator's cwd check, and never closes the active turn.
|
||||
Crash repair is cold-only. For a live id, `load(id)` snapshots the authoritative in-memory log, waits for that snapshot to become durable, and returns it only when balanced; an open live turn rejects instead of receiving synthetic interruption closers. For a cold id, inspection reads, validates, freezes, and constructs one unpublished Session; repeated inspection reuses that object graph only while its source revision remains current. `prepare(id)` performs the same check before repair, reserves the exact Session, commits any pending torn-tail/interrupted-turn repair, and returns it for publication. HMR adoption reads through `loadStored`, applies the coordinator's cwd check, and never closes the active turn.
|
||||
|
||||
## Format decoding and upgrades
|
||||
|
||||
Every logical read opens a repeatable `StoredSessionSource` containing an untrusted header, an exact revision, and a `readEvents()` factory. The static decoder chooses a complete adjacent-version path, creates one migration instance per version, calls `header()` once, calls `event()` once per input record, and calls optional `finish()` after EOF. It then validates the final header and events as the current format. `inspect()` and `readFrom()` do not write. Cold continuation and live adoption replace a converted source through the backend's revision compare-and-swap, then reopen it; a concurrent change discards the decoded result and restarts from the new source. The [session-log versioning Agent Note](../../../.agents/notes/implemented/architecture/2026-08-10-session-log-version-mechanism.md) owns the rationale and refusal rules.
|
||||
|
||||
A future vN→vN+1 change adds `src/format-migrations/vN-to-vN+1.ts`, exports its class from the static `SESSION_FORMAT_MIGRATIONS` array, and increments `SESSION_FORMAT_VERSION`. Static `from`/`to` identify adjacent versions; instance fields retain header and cross-event state. `header()` validates and converts the old header, `event()` returns exactly one lossless-JSON event with the input event's seq, and optional `finish()` validates state that can be settled only at EOF. Header-only reads do not call `finish()`. A migration that changes facts consumed by a projection also increments that projection's `stateVersion`; persistence does not invalidate every projection cache entry. Backends and the coordinator remain version-independent.
|
||||
|
||||
The v0 decoder also recognizes the bounded pre-versioning variants recorded by the [pre-identity message](../../../.agents/notes/implemented/bug-fix/2026-07-28-load-pre-identity-session-messages.md) and [pre-react-loop session](../../../.agents/notes/implemented/bug-fix/2026-08-04-load-pre-react-loop-sessions.md) decisions, and normalizes the historical `compact/start`, `compact/summary`, `compact/end`, and `compact/prune` names to their canonical `compaction/*` names. These compatibility transforms are not format migrations.
|
||||
Backend reads convert the exact supported older records from the same format version before validating current records. Pre-identity messages receive the deterministic id `legacy-message:<session-id>:<event-seq>`; a tool-result content replacement inherits its target's imported id. A pre-react-loop `turn/start` loses its obsolete trigger, a removed `steering/message` becomes the same identified `user/message`, and an older `turn/end` maps its terminal reason without inventing a caller that the old record did not name. The coordinator uses the same converted view for `load`, `inspect`, `readFrom`, ownerless-state claims, and HMR prefix adoption. Storage remains append-only: reads do not rewrite old records, and later appends use the current format. These are narrow import exceptions from the [pre-identity message](../../../.agents/notes/implemented/bug-fix/2026-07-28-load-pre-identity-session-messages.md) and [pre-react-loop session](../../../.agents/notes/implemented/bug-fix/2026-08-04-load-pre-react-loop-sessions.md) decisions, not a general v0 migration promise.
|
||||
|
||||
When a live session emits `session/disposed`, the coordinator waits for its controller, serializes a final drain, then releases state owned by that exact `Session` object. Failed retirement leaves the controller in the live-session map, so backend teardown can retry it. Backend teardown stops event admission first, flushes every remaining controller, awaits per-id operations, and only then closes the storage handle.
|
||||
|
||||
@@ -55,12 +49,12 @@ The `PersistenceBackend<TornMarker>` hooks (the only contract between the coordi
|
||||
| Hook | Role |
|
||||
|---|---|
|
||||
| `name` | Backend label for the dispose-failure `AggregateError`. |
|
||||
| `openStored(id, signal?)` | Open an untrusted header plus repeatable event readers bound to one exact source revision. Each `readEvents({ fromSeq? })` reproduces that revision and exposes backend-owned torn-tail metadata only after EOF, or rejects with `SessionPersistenceRevisionConflictError` when the source changed. |
|
||||
| `readStoredRevision(id, signal?)` | Read the current source-qualified revision for one id without loading its event log. It uses the same revision representation as `openStored` and returns `undefined` when the id is absent. |
|
||||
| `loadStored(id, signal?)` | Read a stored prefix by id across every storage scope. Used by resume/load, non-mutating inspect, live adoption, and the create-collision probe. The optional signal belongs to observation-only reads. Returned metadata identifies `id`; `revision` identifies exactly the returned header and events; an opaque `tornMarker` is present iff a torn tail must be truncated. |
|
||||
| `readStoredRevision(id, signal?)` | Read the current source-qualified revision for one id without loading its event log. It uses the same revision representation as `loadStored` and returns `undefined` when the id is absent. |
|
||||
| `loadStoredFrom?(id, fromSeq, signal?)` | Optional seek-capable suffix read behind the service's `readFrom`: the header plus stored events with `seq >= fromSeq`, non-mutating, no torn marker. SQLite implements it (`WHERE seq >= ?`); a backend that omits it gets the coordinator's fallback — `loadStored` plus a forward skip. |
|
||||
| `appendBatch(meta, events, isMaterialized)` | Durably append a contiguous batch, lazily materializing ATOMICALLY when not yet materialized. |
|
||||
| `materializeHeader?(meta)` | Durably create a header-only artifact for `ensureMaterialized`; required by providers that support durable empty sessions. |
|
||||
| `commitRepair(meta, tornMarker, closers)` | Make a crash repair durable: truncate the torn tail (iff `tornMarker !== undefined` — a marker may be falsy, e.g. seq/offset `0`) and append `closers`. NOT required to be atomic. Used by load (truncate + closers) and live-adoption (truncate only). |
|
||||
| `replaceStored(expectedRevision, meta, events)` | Atomically replace one exact revision with a complete current-format header and event stream. Revision and stored identity checks occur at the commit boundary — immediately before the atomic rename on JSONL, inside the replacing transaction on SQLite; the checks add no cross-process writer exclusion. A mismatch rejects with `SessionPersistenceRevisionConflictError`. |
|
||||
| `list(signal?)` | List all stored metadata, observing optional cancellation. |
|
||||
| `close?()` | Optional lifecycle teardown (e.g. close a db handle), awaited after the dispose drain. |
|
||||
|
||||
|
||||
@@ -17,9 +17,9 @@
|
||||
| `ensureMaterialized(session): Promise<void>` | 在不虚构事件的情况下，显式使一个确切 live session 即使零事件也保持持久。只有当空会话本身是可恢复资源时，生命周期前端才使用它；普通创建仍保持延迟实体化。 |
|
||||
| `append(id, events): Promise<void>` | 持久保存一个批次。仅追加；任何修复后，第一个事件 `seq` == 已存储 next-seq；非 JSON 可序列化数据会被拒绝，并命名违规类型。 |
|
||||
| `prepare(id, signal?): Promise<SessionPreparation>` | 预留恢复所使用的那个未发布 Session。协调器会尽可能复用之前的检查结果、提交待处理恢复，并在 dispose（资源释放）时将未发布 reservation 释放回有界缓存。 |
|
||||
| `load(id): Promise<{ meta; events }>` | 沿受支持的格式路径解码，并提交格式替换与冷恢复后，返回不可变、平衡的逻辑日志。实时 load 先 flush 其快照，并在轮次开放时拒绝；冷 load 保留中断的最终轮次，并用合成 `tool/result`/`step/end?`/`turn/end {interrupted}` 事件持久关闭它。只丢弃撕裂尾部碎片；已提交损坏和格式错误的记录以 `SessionPersistenceCorruptionError` 拒绝，不支持的格式 `version` 或本构建不认识且信封未带 `ignorable` 标记的事件类型以 `SessionFormatUnsupportedError` 拒绝，消息说明拒绝方向，并在后端为每个会话保留独立文件时给出原始日志路径。 |
|
||||
| `load(id): Promise<{ meta; events }>` | 转换同一格式版本中受支持的旧记录后，返回不可变、平衡的逻辑日志，并提交冷恢复。实时 load 先 flush 其快照，并在轮次开放时拒绝；冷 load 保留中断的最终轮次，并用合成 `tool/result`/`step/end?`/`turn/end {interrupted}` 事件持久关闭它。只丢弃撕裂尾部碎片；已提交损坏和格式错误的记录以 `SessionPersistenceCorruptionError` 拒绝，不支持的格式 `version` 或本构建不认识且信封未带 `ignorable` 标记的事件类型以 `SessionFormatUnsupportedError` 拒绝，消息说明拒绝方向，并在后端为每个会话保留独立文件时给出原始日志路径。 |
|
||||
| `inspect(id, signal?): Promise<{ meta; events }>` | 返回已经升级、验证和深度冻结的逻辑视图，但不提交恢复或发布 Session。冷视图会获得仅存在于内存的合成恢复 closer，物理撕裂尾部保持不变；实时状态下的视图则是当前不可变快照，可能包含开放的轮次。基于协调器的实现会在有界 LRU 中保留该冷状态下未发布的 Session 本身，供后续 `prepare` 使用，但已存储修订值变化后会丢弃并重新读取。同 id 检查共享进行中的读取。 |
|
||||
| `readFrom(id, fromSeq, signal?): Promise<{ meta; events }>` | 返回 `seq >= fromSeq` 的有效已存储事件，不进入 preparation 缓存、不截断、不合成 closer，也不发布协调器状态。`fromSeq` 达到或超过已存储末尾时返回空事件列表；负数或非安全整数 `fromSeq` 会被拒绝。当前格式读取向后端请求 suffix；存在格式迁移时则读取完整 source，迁移后才应用 `fromSeq`。顺序介质可能仍需扫描物理 framing 后再过滤，可寻址介质则可不读取更早的记录。供 checkpoint 消费方只应用已存序号之后的事件。 |
|
||||
| `readFrom(id, fromSeq, signal?): Promise<{ meta; events }>` | 返回 `seq >= fromSeq` 的有效已存储事件，不进入 preparation 缓存、不截断、不合成 closer，也不发布协调器状态。`fromSeq` 达到或超过已存储末尾时返回空事件列表；负数或非安全整数 `fromSeq` 会被拒绝。可寻址后端（SQLite）只读后缀，除非转换受支持的旧记录需要读取更早的记录；顺序后端（JSONL）解析整个产物并向前跳过。未知类型拒绝遵循同一读取方式：寻址读取只检查返回的后缀，顺序回退路径还会拒绝窗口以下的未知必需事件。供 checkpoint 消费方只应用已存序号之后的事件。 |
|
||||
| `list(signal?): Promise<SessionHeader[]>` | 从元数据轻量列出，不解析完整日志。可选信号取消后端列表工作。零事件会话在 consumer 显式实体化前不在 `list` 中。 |
|
||||
| `listSnapshots(signal?): Promise<SessionPersistenceSnapshot[]>` | 返回轻量元数据和每份日志一个不透明、带品牌类型的修订值，不加载事件日志。日志及其后端存储不变时，修订保持相等；append 或变更性 load 修复后会改变；不会仅因两个存储使用相同本地计数器而冲突。可选信号请求取消后端发现工作；第一方后端会先等待所有已启动的列出工作结束，再予以拒绝，因此调用返回拒绝时，相关工作已完全停稳。 |
|
||||
|
||||
@@ -36,15 +36,9 @@
|
||||
|
||||
每个 `session/event` 将事件复制到会话 controller。第一个待处理事件会开启固定批处理窗口；后续事件会加入该批次，但不会重置截止时间。配置的 `writeBatchMaxDelayMs` 只限制这段有意等待，而不限制事件循环、初始化、串行化操作或后端延迟。写入期间接纳的事件会形成一个新的有界批次。`session/flush` 会取消等待，并作为共享的完全停稳屏障，排空屏障运行期间接纳的事件。后台写入失败只记录一次日志，保留顺序不变的批次，并暂停自动重试；新事件会开启新的固定窗口，而显式 flush 或后端拆卸会立即重试，并在失败再次发生时向调用方暴露失败。
|
||||
|
||||
崩溃修复只适用于冷状态。对于已有活动会话的 id，`load(id)` 为权威内存日志制作快照，等待该快照持久，并只在平衡时返回；活动会话中开放的轮次会被拒绝，而不会收到合成中断 closer。对于冷 id，检查只读取、验证、冻结并构造一次未发布 Session；只有来源修订值仍然是当前值时，重复检查才会复用该对象图。`prepare(id)` 在修复前执行相同校验，预留该 Session 本身，提交任何待处理的撕裂尾部或中断轮次修复，并将其返回用于发布。HMR（热模块替换）接管打开同一份绑定 revision 的 source，应用协调器 cwd 检查，并绝不关闭活动轮次。
|
||||
崩溃修复只适用于冷状态。对于已有活动会话的 id，`load(id)` 为权威内存日志制作快照，等待该快照持久，并只在平衡时返回；活动会话中开放的轮次会被拒绝，而不会收到合成中断 closer。对于冷 id，检查只读取、验证、冻结并构造一次未发布 Session；只有来源修订值仍然是当前值时，重复检查才会复用该对象图。`prepare(id)` 在修复前执行相同校验，预留该 Session 本身，提交任何待处理的撕裂尾部或中断轮次修复，并将其返回用于发布。HMR（热模块替换）接管通过 `loadStored` 读取，应用协调器 cwd 检查，并绝不关闭活动轮次。
|
||||
|
||||
## 格式解码与升级
|
||||
|
||||
每次逻辑读取都会打开可重复使用的 `StoredSessionSource`，其中包含不可信 header、精确 revision 和 `readEvents()` factory。静态 decoder 选择完整的相邻版本路径，为每个版本创建一个 migration 实例，调用一次 `header()`，为每条输入记录调用一次 `event()`，并在 EOF 后调用可选的 `finish()`，最后按当前格式验证 header 与事件。`inspect()` 和 `readFrom()` 不写存储；冷 continuation 与实时接管通过后端的 revision compare-and-swap 替换已转换 source，然后重新打开。并发变更会丢弃解码结果，并从新 source 重新开始。[Session log 版本机制 Agent Note](../../../.agents/notes/implemented/architecture/2026-08-10-session-log-version-mechanism.zh.md)规定其原因和拒绝规则。
|
||||
|
||||
以后新增 vN→vN+1 时，在 `src/format-migrations/vN-to-vN+1.ts` 添加 class，从静态 `SESSION_FORMAT_MIGRATIONS` 数组导出，并递增 `SESSION_FORMAT_VERSION`。静态 `from`／`to` 标识相邻版本，实例字段保留 header 和跨事件状态。`header()` 验证并转换旧 header；`event()` 只返回一条可无损表示为 JSON 且 seq 与输入相同的事件；可选的 `finish()` 验证只能在 EOF 时结算的状态。只读 header 时不调用 `finish()`。如果 migration 改变了某个 projection 消费的事实，还要递增该 projection 的 `stateVersion`；persistence 不统一作废所有 projection cache 记录。后端和协调器不增加版本特判。
|
||||
|
||||
v0 decoder 还识别[消息标识机制引入前的消息](../../../.agents/notes/implemented/bug-fix/2026-07-28-load-pre-identity-session-messages.zh.md)与 [react-loop 引入前会话](../../../.agents/notes/implemented/bug-fix/2026-08-04-load-pre-react-loop-sessions.zh.md)决策所限定的版本机制建立前变体，并将历史 `compact/start`、`compact/summary`、`compact/end`、`compact/prune` 名称归一化为规范的 `compaction/*` 名称。这些兼容转换不是格式迁移。
|
||||
后端读取会在验证当前记录前，转换同一格式版本中明确受支持的旧记录。消息标识机制引入前的消息会获得确定性的 id `legacy-message:<session-id>:<event-seq>`；工具结果的内容替换会继承其目标导入后的 id。react-loop 引入前的 `turn/start` 会移除过时的 trigger，已移除的 steering（中途引导）事件 `steering/message` 会转换为同一条带标识的 `user/message`；旧版 `turn/end` 会映射终止原因，但不会虚构旧记录中没有记载的调用方。协调器对 `load`、`inspect`、`readFrom`、无所有者状态的认领和 HMR 前缀接管使用同一份转换后视图。存储仍然仅追加：读取不会重写旧记录，此后追加的事件使用当前格式。这些是[消息标识机制引入前的消息](../../../.agents/notes/implemented/bug-fix/2026-07-28-load-pre-identity-session-messages.zh.md)与 [react-loop 引入前会话](../../../.agents/notes/implemented/bug-fix/2026-08-04-load-pre-react-loop-sessions.zh.md)决策所规定的范围受限的导入例外，并不构成通用的 v0 迁移承诺。
|
||||
|
||||
活动会话发出 `session/disposed` 时，协调器等待其 controller，以串行方式执行最终 drain，然后释放该精确 `Session` 对象拥有的状态。失败退役会将 controller 保留在活动会话 map 中，使后端拆卸可重试。后端拆卸先停止事件接纳，flush 每个剩余 controller，等待每 id 操作，最后才关闭存储句柄。
|
||||
|
||||
@@ -55,12 +49,12 @@ v0 decoder 还识别[消息标识机制引入前的消息](../../../.agents/note
|
||||
| 钩子 | 职责 |
|
||||
|---|---|
|
||||
| `name` | dispose 失败 `AggregateError` 的后端标签。 |
|
||||
| `openStored(id, signal?)` | 打开不可信 header 和绑定同一精确 source revision 的可重复事件 reader。每次 `readEvents({ fromSeq? })` 都重现该 revision，并只在 EOF 后暴露 backend 自有 torn-tail metadata；source 已变化时以 `SessionPersistenceRevisionConflictError` 拒绝。 |
|
||||
| `readStoredRevision(id, signal?)` | 在不加载事件日志的情况下读取一个 id 当前的来源限定修订值。它使用与 `openStored` 相同的修订值表示；id 不存在时返回 `undefined`。 |
|
||||
| `loadStored(id, signal?)` | 在全部存储范围中按 id 读取已存储前缀。用于恢复／加载、非修改式 inspect、活动会话接管和 create 冲突探测。可选信号属于仅观察读取。返回元数据标识 `id`；`revision` 精确标识返回的 header 和事件；当且仅当必须截断撕裂尾部时才存在不透明 `tornMarker`。 |
|
||||
| `readStoredRevision(id, signal?)` | 在不加载事件日志的情况下读取一个 id 当前的来源限定修订值。它使用与 `loadStored` 相同的修订值表示；id 不存在时返回 `undefined`。 |
|
||||
| `loadStoredFrom?(id, fromSeq, signal?)` | 服务 `readFrom` 背后的可选可寻址后缀读取：返回 header 和 `seq >= fromSeq` 的已存储事件，非修改式、无撕裂标记。SQLite 实现它（`WHERE seq >= ?`）；不实现的后端使用协调器回退——`loadStored` 加向前跳过。 |
|
||||
| `appendBatch(meta, events, isMaterialized)` | 持久追加连续批次；尚未实体化时以原子方式延迟实体化。 |
|
||||
| `materializeHeader?(meta)` | 为 `ensureMaterialized` 持久创建仅含 header 的 artifact；支持持久空会话的 provider 必须实现。 |
|
||||
| `commitRepair(meta, tornMarker, closers)` | 使崩溃修复持久：截断撕裂尾部（当且仅当 `tornMarker !== undefined`；标记可为 falsy，例如 seq/offset `0`），并追加 `closers`。不要求原子性。由 load（截断 + closer）和活动会话接管（仅截断）使用。 |
|
||||
| `replaceStored(expectedRevision, meta, events)` | 用完整的当前格式 header 与事件流原子替换一个精确 revision。Revision 与存储身份检查发生在提交边界——JSONL 在原子替换前立即检查，SQLite 在替换事务内检查；该检查不提供跨进程写者排他。不匹配时以 `SessionPersistenceRevisionConflictError` 拒绝。 |
|
||||
| `list(signal?)` | 列出全部已存储元数据，并遵循可选的取消信号。 |
|
||||
| `close?()` | 可选生命周期拆卸（例如关闭 db 句柄），在 dispose drain 后等待其完成。 |
|
||||
|
||||
|
||||
@@ -9,24 +9,16 @@ import { Context } from '@deepseek-ai/cordis'
|
||||
import {
|
||||
adoptSessionEvent,
|
||||
interruptedTurnClosers,
|
||||
KNOWN_SESSION_EVENT_TYPES,
|
||||
SESSION_FORMAT_VERSION,
|
||||
SessionPreparation,
|
||||
snapshotJsonValue,
|
||||
snapshotSessionEvent,
|
||||
} from '@deepseek-ai/dsh-session'
|
||||
import type { Session, SessionEvent, SessionId, SessionHeader } from '@deepseek-ai/dsh-session'
|
||||
import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout'
|
||||
import type { BorrowedSessionSource, SessionInspection } from './index.ts'
|
||||
import {
|
||||
decodeStoredSession,
|
||||
SessionFormatUnsupportedError,
|
||||
} from './format-decoder.ts'
|
||||
import { assertNoRetiredSessionEvent } from './format-json.ts'
|
||||
import type {
|
||||
DecodedSession,
|
||||
StoredSessionSource,
|
||||
} from './format-decoder.ts'
|
||||
import type { BorrowedSessionSource, SessionInspection, SessionLocation } from './index.ts'
|
||||
import { SessionPersistenceNotFoundError } from './errors.ts'
|
||||
import { SessionPersistenceRevisionConflictError } from './revision.ts'
|
||||
import type { SessionPersistenceRevision } from './revision.ts'
|
||||
import { observeQueuedAbort, SessionPreparations } from './preparations.ts'
|
||||
import type { SessionPreparationReservation } from './preparations.ts'
|
||||
@@ -53,6 +45,42 @@ export class SessionPersistenceCorruptionError extends Error {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The stored log is intact but this runtime cannot faithfully interpret it:
|
||||
* the header carries an unsupported format version, or an event's type is
|
||||
* unknown to this build and the event is not marked ignorable. Distinct from
|
||||
* {@link SessionPersistenceCorruptionError} — nothing is damaged; the raw log
|
||||
* remains readable at {@link location} when the backend keeps one artifact
|
||||
* per session.
|
||||
*/
|
||||
export class SessionFormatUnsupportedError extends Error {
|
||||
/**
|
||||
* @param message - stable reason the log cannot be interpreted, already
|
||||
* including the raw-log path when one exists.
|
||||
* @param location - the backend's artifact location, when one exists.
|
||||
*/
|
||||
constructor(message: string, readonly location?: SessionLocation) {
|
||||
super(message)
|
||||
this.name = 'SessionFormatUnsupportedError'
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Direction-aware refusal text for a stored session whose format version this
|
||||
* build does not read. Shared by the coordinator's load-time check and by
|
||||
* backends that must refuse BEFORE decoding version-dependent structure (a
|
||||
* future format may not satisfy this build's structural checks at all, and the
|
||||
* user must see "upgrade the harness", never "corrupt").
|
||||
* @param id - the stored session id, for message context.
|
||||
* @param version - the stored format version.
|
||||
* @returns the stable refusal text, without a raw-log path suffix.
|
||||
*/
|
||||
export function sessionFormatVersionRefusal(id: string, version: number): string {
|
||||
return version > SESSION_FORMAT_VERSION
|
||||
? `session "${id}" uses log format v${version}, but this harness reads only v${SESSION_FORMAT_VERSION}: the log was written by a newer harness — upgrade the harness to open it`
|
||||
: `session "${id}" uses log format v${version}, older than the supported v${SESSION_FORMAT_VERSION}, and this build ships no upgrade path for it`
|
||||
}
|
||||
|
||||
/** Coordinator policy supplied by a concrete persistence backend. */
|
||||
export interface PersistenceCoordinatorOptions {
|
||||
/** Maximum completed unpublished preparations retained for reuse. */
|
||||
@@ -61,6 +89,32 @@ export interface PersistenceCoordinatorOptions {
|
||||
readonly writeBatchMaxDelayMs: number
|
||||
}
|
||||
|
||||
/**
|
||||
* A stored session's header, valid contiguous event prefix, source-qualified
|
||||
* revision, and optional opaque torn-tail marker. The revision identifies the
|
||||
* exact detached prefix. The coordinator only checks marker presence and
|
||||
* returns its value to {@link PersistenceBackend.commitRepair}; each backend
|
||||
* owns the marker type.
|
||||
*/
|
||||
export interface StoredPrefix<TornMarker = unknown> {
|
||||
meta: SessionHeader
|
||||
events: SessionEvent[]
|
||||
/** Revision observed for exactly this detached prefix. */
|
||||
revision: SessionPersistenceRevision
|
||||
tornMarker?: TornMarker
|
||||
}
|
||||
|
||||
/**
|
||||
* A stored session's header plus the events at or past a requested seq — the
|
||||
* return shape of the optional seek-capable
|
||||
* {@link PersistenceBackend.loadStoredFrom} hook. Non-mutating reads carry no
|
||||
* torn marker: there is nothing to repair.
|
||||
*/
|
||||
export interface StoredSuffix {
|
||||
meta: SessionHeader
|
||||
events: SessionEvent[]
|
||||
}
|
||||
|
||||
/**
|
||||
* The storage contract between {@link PersistenceCoordinator} and a concrete
|
||||
* backend: the minimal set of durable primitives the orchestration calls. A
|
||||
@@ -68,21 +122,27 @@ export interface PersistenceCoordinatorOptions {
|
||||
* coordinator supplies everything else (buffering, serialization, cursors,
|
||||
* adoption, crash repair sequencing, dispose quiescence).
|
||||
*
|
||||
* @typeParam TornMarker - the backend's opaque torn-tail repair token returned
|
||||
* after a complete event read. The coordinator treats it as fully opaque.
|
||||
* @typeParam TornMarker - the backend's opaque torn-tail repair token (see
|
||||
* {@link StoredPrefix}). The coordinator treats it as fully opaque.
|
||||
*/
|
||||
export interface PersistenceBackend<TornMarker = unknown> {
|
||||
/** Human-readable backend name, used in the dispose-failure AggregateError. */
|
||||
readonly name: string
|
||||
|
||||
/**
|
||||
* Open repeatable access to one stored revision by id, scanning every backend
|
||||
* storage scope. Returns `undefined` if no artifact exists. Each event reader
|
||||
* reproduces this revision or rejects when a concurrent writer changed it.
|
||||
* Read a stored prefix by id, scanning every backend storage scope. Returns
|
||||
* `undefined` if no stored artifact exists. Returned metadata must identify
|
||||
* `id` before repair or state publication. Used by resume/load, live adoption,
|
||||
* and — via `!== undefined` — the create-collision probe. The returned
|
||||
* `tornMarker` is present iff there is a torn tail to truncate. Every header
|
||||
* and event graph must be fresh, mutually unaliased, and unretained by the
|
||||
* backend because preparation freezes and publishes them in place. The
|
||||
* returned revision must identify exactly those values and use the same
|
||||
* representation as {@link readStoredRevision}.
|
||||
* @param id - persisted session id to resolve.
|
||||
* @param signal - optional cancellation for backend read work.
|
||||
*/
|
||||
openStored(id: SessionId, signal?: AbortSignal): Promise<StoredSessionSource<TornMarker> | undefined>
|
||||
loadStored(id: SessionId, signal?: AbortSignal): Promise<StoredPrefix<TornMarker> | undefined>
|
||||
|
||||
/**
|
||||
* Read the current source-qualified revision for one stored session without
|
||||
@@ -92,6 +152,30 @@ export interface PersistenceBackend<TornMarker = unknown> {
|
||||
*/
|
||||
readStoredRevision(id: SessionId, signal?: AbortSignal): Promise<SessionPersistenceRevision | undefined>
|
||||
|
||||
/**
|
||||
* Optional seek-capable suffix read behind the service's `readFrom`: return
|
||||
* the header plus the stored events with `seq >= fromSeq` without reading
|
||||
* the whole log. A backend whose medium can address events by seq (SQLite)
|
||||
* implements this so `readFrom` scales with the suffix; sequential backends
|
||||
* omit it and the coordinator falls back to {@link loadStored} plus a
|
||||
* forward skip. Non-mutating (no truncation, no closers). Validation of the
|
||||
* region strictly below `fromSeq` is limited to seq contiguity — the
|
||||
* service contract scopes this read to the suffix — unless that suffix
|
||||
* contains a supported legacy shape whose normalization needs earlier
|
||||
* message-identity facts, in which case the coordinator falls back
|
||||
* to the complete stored prefix.
|
||||
* Unknown-type refusal follows the same suffix scope: a seek-capable
|
||||
* backend's `readFrom` checks only the returned suffix, while the
|
||||
* sequential fallback parses the whole artifact and refuses on an unknown
|
||||
* required event anywhere in it — over-refusal on the sequential side is
|
||||
* accepted rather than widening the seek read.
|
||||
* @param id - persisted session id to resolve.
|
||||
* @param fromSeq - first event seq to include (non-negative safe integer,
|
||||
* validated by the coordinator before this hook runs).
|
||||
* @param signal - optional cancellation for backend read work.
|
||||
*/
|
||||
loadStoredFrom?(id: SessionId, fromSeq: number, signal?: AbortSignal): Promise<StoredSuffix | undefined>
|
||||
|
||||
/** Durably create an empty header-only session artifact. */
|
||||
materializeHeader?(meta: SessionHeader): Promise<void>
|
||||
|
||||
@@ -112,27 +196,20 @@ export interface PersistenceBackend<TornMarker = unknown> {
|
||||
*/
|
||||
commitRepair(meta: SessionHeader, tornMarker: TornMarker | undefined, closers: readonly SessionEvent[]): Promise<void>
|
||||
|
||||
/**
|
||||
* Atomically replace one exact stored revision with a complete current log.
|
||||
* The backend checks revision and storage identity at the commit boundary:
|
||||
* immediately before the atomic rename on JSONL, inside the replacing
|
||||
* transaction on SQLite. The check adds no cross-process writer exclusion.
|
||||
* @param expectedRevision - exact source revision decoded by the caller.
|
||||
* @param meta - complete current-format header.
|
||||
* @param events - complete current-format event stream.
|
||||
*/
|
||||
replaceStored(
|
||||
expectedRevision: SessionPersistenceRevision,
|
||||
meta: SessionHeader,
|
||||
events: AsyncIterable<SessionEvent>,
|
||||
): Promise<void>
|
||||
|
||||
/**
|
||||
* List all stored (materialized) sessions' metadata.
|
||||
* @param signal - optional cancellation for backend listing work.
|
||||
*/
|
||||
list(signal?: AbortSignal): Promise<SessionHeader[]>
|
||||
|
||||
/**
|
||||
* Optional side-effect-free artifact locator, used to point refusal
|
||||
* diagnostics ({@link SessionFormatUnsupportedError}) at the raw log.
|
||||
* Backends without one artifact per session omit it or return `undefined`.
|
||||
* @param meta - the header whose artifact is requested.
|
||||
*/
|
||||
locate?(meta: SessionHeader): SessionLocation | undefined
|
||||
|
||||
/**
|
||||
* Optional lifecycle teardown (e.g. close a database handle). Awaited by the
|
||||
* coordinator's dispose effect AFTER the quiescence drain. A stateless file
|
||||
@@ -172,7 +249,6 @@ interface PreparedSessionSource<TornMarker> {
|
||||
readonly inspection: SessionInspection
|
||||
readonly session: Session
|
||||
readonly revision: SessionPersistenceRevision
|
||||
readonly sourceVersion: number
|
||||
/** Session length after constructor-owned seed markers were appended. */
|
||||
readonly sessionLength: number
|
||||
readonly tornMarker: TornMarker | undefined
|
||||
@@ -198,34 +274,306 @@ function seedCoversPrefix(seed: readonly SessionEvent[], prefix: readonly Sessio
|
||||
})
|
||||
}
|
||||
|
||||
/** Reject obsolete v0 event records before a live writer persists them. */
|
||||
/** Reject events from an obsolete v0 vocabulary that this build cannot replay. */
|
||||
function assertSupportedEvents(events: readonly SessionEvent[], id: SessionId): void {
|
||||
for (const event of events) assertNoRetiredSessionEvent(event, id)
|
||||
}
|
||||
|
||||
/** Materialize one decoded event read and observe its physical EOF metadata. */
|
||||
async function collectDecodedEvents<TornMarker>(
|
||||
read: DecodedSession<TornMarker>,
|
||||
): Promise<{ events: SessionEvent[]; tornMarker: TornMarker | undefined }> {
|
||||
const events: SessionEvent[] = []
|
||||
try {
|
||||
for await (const event of read.events) events.push(event)
|
||||
} catch (error: unknown) {
|
||||
await read.completed.catch(() => undefined)
|
||||
throw error
|
||||
const legacyType: string = 'request/header-delta'
|
||||
const legacy = events.find(event => event.type === legacyType)
|
||||
if (legacy !== undefined) {
|
||||
throw new Error(`session "${id}" contains unsupported legacy request/header-delta event at seq ${legacy.seq}`)
|
||||
}
|
||||
const legacyModeType: string = 'mode/set'
|
||||
const legacyMode = events.find(event => event.type === legacyModeType)
|
||||
if (legacyMode !== undefined) {
|
||||
throw new Error(`session "${id}" contains unsupported legacy mode/set event at seq ${legacyMode.seq}`)
|
||||
}
|
||||
const fallback = events.find(event => event.type === 'request/header'
|
||||
&& (event.data as { reason?: string }).reason === 'fallback')
|
||||
if (fallback !== undefined) {
|
||||
throw new Error(`session "${id}" contains unsupported legacy request/header reason "fallback" at seq ${fallback.seq}`)
|
||||
}
|
||||
const { tornMarker } = await read.completed
|
||||
return { events, tornMarker }
|
||||
}
|
||||
|
||||
/** Yield an immutable event array as one replacement stream. */
|
||||
function eventStream(events: readonly SessionEvent[]): AsyncIterable<SessionEvent> {
|
||||
/** Return an object record without widening arrays into message payloads. */
|
||||
function asRecord(value: unknown): Record<string, unknown> | undefined {
|
||||
return typeof value === 'object' && value !== null && !Array.isArray(value)
|
||||
? value as Record<string, unknown>
|
||||
: undefined
|
||||
}
|
||||
|
||||
/** Whether a record contains every required key and no key outside the optional extension set. */
|
||||
function hasOnlyKeys(
|
||||
record: Record<string, unknown>,
|
||||
required: readonly string[],
|
||||
optional: readonly string[] = [],
|
||||
): boolean {
|
||||
const allowed = [...required, ...optional]
|
||||
return Object.keys(record).every(key => allowed.includes(key))
|
||||
&& required.every(key => Object.hasOwn(record, key))
|
||||
}
|
||||
|
||||
type PersistedMessageId = SessionEvent<'user/message'>['data']['id']
|
||||
|
||||
/** Mint the stable import identity for a message persisted before identities existed. */
|
||||
function legacyMessageId(id: SessionId, seq: number): PersistedMessageId {
|
||||
return `legacy-message:${id}:${seq}` as PersistedMessageId
|
||||
}
|
||||
|
||||
/** Read a replacement target while leaving malformed surface metadata to the session validator. */
|
||||
function replacementStart(event: SessionEvent): number | undefined {
|
||||
const op = asRecord((event as SessionEvent & { surfaceOp?: unknown }).surfaceOp)
|
||||
return op?.['op'] === 'replace' && typeof op['start'] === 'number'
|
||||
? op['start']
|
||||
: undefined
|
||||
}
|
||||
|
||||
/** Whether one suffix event needs facts available only from the preceding stored prefix. */
|
||||
function needsLegacyPrefix(event: SessionEvent): boolean {
|
||||
const data = asRecord(event.data)
|
||||
const legacySteeringType: string = 'steering/message'
|
||||
if (event.type === legacySteeringType) return true
|
||||
if (data === undefined) return false
|
||||
switch (event.type) {
|
||||
case 'user/message':
|
||||
return !Object.hasOwn(data, 'id') && Object.hasOwn(data, 'content')
|
||||
case 'assistant/message':
|
||||
return !Object.hasOwn(data, 'message') && Object.hasOwn(data, 'content')
|
||||
case 'tool/result':
|
||||
return !Object.hasOwn(data, 'message') && Object.hasOwn(data, 'callId')
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
/** Upgrade the removed steering surface event into its current user-message equivalent. */
|
||||
function migrateLegacySteeringEvent(event: SessionEvent, id: SessionId): SessionEvent {
|
||||
const legacyType: string = 'steering/message'
|
||||
if (event.type !== legacyType) return event
|
||||
const data = asRecord(event.data)
|
||||
if (data === undefined) {
|
||||
throw new Error(`session "${id}" contains malformed pre-react-loop steering/message at seq ${event.seq}`)
|
||||
}
|
||||
const wrapped = asRecord(data['message'])
|
||||
if (wrapped !== undefined && Number.isSafeInteger(data['turn'])
|
||||
&& hasOnlyKeys(data, ['turn', 'message'])) {
|
||||
return { ...event, type: 'user/message', data: wrapped } as SessionEvent
|
||||
}
|
||||
if (!Number.isSafeInteger(data['turn']) || !hasOnlyKeys(data, ['turn', 'content', 'source'])) {
|
||||
throw new Error(`session "${id}" contains malformed pre-react-loop steering/message at seq ${event.seq}`)
|
||||
}
|
||||
const { turn: _turn, ...message } = data
|
||||
return {
|
||||
[Symbol.asyncIterator]() {
|
||||
const iterator = events[Symbol.iterator]()
|
||||
return { next: () => Promise.resolve(iterator.next()) }
|
||||
...event,
|
||||
type: 'user/message',
|
||||
data: {
|
||||
...message,
|
||||
id: legacyMessageId(id, event.seq),
|
||||
role: 'user',
|
||||
},
|
||||
} as SessionEvent
|
||||
}
|
||||
|
||||
/** Remove the obsolete trigger after verifying the complete old turn-start envelope. */
|
||||
function migrateLegacyTurnStartEvent(event: SessionEvent, id: SessionId): SessionEvent {
|
||||
if (event.type !== 'turn/start') return event
|
||||
const data = asRecord(event.data)
|
||||
if (data === undefined || !Object.hasOwn(data, 'trigger')) return event
|
||||
const trigger = asRecord(data['trigger'])
|
||||
if (!Number.isSafeInteger(data['turn']) || (data['turn'] as number) < 1
|
||||
|| !hasOnlyKeys(data, ['turn', 'trigger'])
|
||||
|| trigger === undefined || typeof trigger['kind'] !== 'string' || trigger['kind'].length === 0) {
|
||||
throw new Error(`session "${id}" contains malformed pre-react-loop turn/start at seq ${event.seq}`)
|
||||
}
|
||||
return { ...event, data: { turn: data['turn'] } } as SessionEvent
|
||||
}
|
||||
|
||||
/** Upgrade an obsolete turn ending while preserving the latest-master envelope. */
|
||||
function migrateLegacyTurnEndEvent(event: SessionEvent, id: SessionId): SessionEvent {
|
||||
if (event.type !== 'turn/end') return event
|
||||
const data = asRecord(event.data)
|
||||
/* v8 ignore next -- a non-record current envelope cannot match a legacy shape. */
|
||||
if (data === undefined) return event
|
||||
const malformed = (): never => {
|
||||
throw new Error(`session "${id}" contains malformed pre-react-loop turn/end at seq ${event.seq}`)
|
||||
}
|
||||
const reason = asRecord(data['reason'])
|
||||
if (!Number.isSafeInteger(data['turn']) || (data['turn'] as number) < 1
|
||||
|| !hasOnlyKeys(data, ['turn', 'reason'])
|
||||
|| reason === undefined || typeof reason['kind'] !== 'string') return malformed()
|
||||
|
||||
let currentReason: Record<string, unknown> | undefined
|
||||
switch (reason['kind']) {
|
||||
case 'completed':
|
||||
case 'blocked':
|
||||
case 'max-tokens':
|
||||
case 'interrupted':
|
||||
if (!hasOnlyKeys(reason, ['kind'])) return malformed()
|
||||
return event
|
||||
case 'aborted':
|
||||
if (Object.hasOwn(reason, 'reason')) return event
|
||||
if (!hasOnlyKeys(reason, ['kind'])) return malformed()
|
||||
currentReason = { kind: 'aborted', reason: { kind: 'legacy' } }
|
||||
break
|
||||
case 'disposed':
|
||||
if (!hasOnlyKeys(reason, ['kind'])) return malformed()
|
||||
currentReason = { kind: 'aborted', reason: { kind: 'disposed' } }
|
||||
break
|
||||
case 'error': {
|
||||
if (Object.hasOwn(reason, 'error')) return event
|
||||
if (!Number.isSafeInteger(reason['step']) || (reason['step'] as number) < 0) return malformed()
|
||||
const failure = asRecord(reason['failure'])
|
||||
if (failure !== undefined && hasOnlyKeys(reason, ['kind', 'step', 'failure'])
|
||||
&& hasOnlyKeys(failure, ['message', 'code'], ['status', 'providerRetryAfterMs', 'requestId'])
|
||||
&& typeof failure['message'] === 'string' && typeof failure['code'] === 'string'
|
||||
&& (failure['status'] === undefined || typeof failure['status'] === 'number')
|
||||
&& (failure['providerRetryAfterMs'] === undefined || typeof failure['providerRetryAfterMs'] === 'number')
|
||||
&& (failure['requestId'] === undefined || typeof failure['requestId'] === 'string')) {
|
||||
currentReason = { kind: 'error', error: failure }
|
||||
break
|
||||
}
|
||||
const messageKeys = reason['code'] === undefined
|
||||
? ['kind', 'step', 'message']
|
||||
: ['kind', 'step', 'message', 'code']
|
||||
if (!hasOnlyKeys(reason, messageKeys)
|
||||
|| typeof reason['message'] !== 'string'
|
||||
|| (reason['code'] !== undefined && typeof reason['code'] !== 'string')) return malformed()
|
||||
currentReason = {
|
||||
kind: 'error',
|
||||
error: {
|
||||
message: reason['message'],
|
||||
code: typeof reason['code'] === 'string' ? reason['code'] : 'UNKNOWN',
|
||||
},
|
||||
}
|
||||
break
|
||||
}
|
||||
default:
|
||||
return event
|
||||
}
|
||||
|
||||
return {
|
||||
...event,
|
||||
data: {
|
||||
...data,
|
||||
reason: currentReason,
|
||||
},
|
||||
} as SessionEvent
|
||||
}
|
||||
|
||||
/**
|
||||
* Upgrade one pre-identity message event into the current wrapper shape.
|
||||
* Current-looking malformed events remain untouched so validation rejects them
|
||||
* instead of disguising corruption as legacy data.
|
||||
*/
|
||||
function migrateLegacyMessageEvent(
|
||||
event: SessionEvent,
|
||||
id: SessionId,
|
||||
messageIds: ReadonlyMap<number, PersistedMessageId>,
|
||||
): SessionEvent {
|
||||
const data = asRecord(event.data)
|
||||
if (data === undefined) return event
|
||||
switch (event.type) {
|
||||
case 'user/message': {
|
||||
if (Object.hasOwn(data, 'id') || Object.hasOwn(data, 'role')
|
||||
|| Object.hasOwn(data, 'message')
|
||||
|| !Object.hasOwn(data, 'content') || !Object.hasOwn(data, 'source')) return event
|
||||
return {
|
||||
...event,
|
||||
data: {
|
||||
...data,
|
||||
id: legacyMessageId(id, event.seq),
|
||||
role: 'user',
|
||||
},
|
||||
} as SessionEvent
|
||||
}
|
||||
case 'assistant/message': {
|
||||
if (Object.hasOwn(data, 'message')
|
||||
|| !Object.hasOwn(data, 'content') || !Object.hasOwn(data, 'provenance')) return event
|
||||
const { content, provenance, ...eventData } = data
|
||||
return {
|
||||
...event,
|
||||
data: {
|
||||
...eventData,
|
||||
message: {
|
||||
id: legacyMessageId(id, event.seq),
|
||||
role: 'assistant',
|
||||
content,
|
||||
source: {
|
||||
...asRecord(provenance),
|
||||
kind: 'model',
|
||||
},
|
||||
},
|
||||
},
|
||||
} as SessionEvent
|
||||
}
|
||||
case 'tool/result': {
|
||||
if (Object.hasOwn(data, 'message')
|
||||
|| !Object.hasOwn(data, 'callId') || !Object.hasOwn(data, 'content')
|
||||
|| !Object.hasOwn(data, 'isError')) return event
|
||||
const { callId, content, isError, ...eventData } = data
|
||||
const inheritedId = replacementStart(event)
|
||||
return {
|
||||
...event,
|
||||
data: {
|
||||
...eventData,
|
||||
message: {
|
||||
id: inheritedId === undefined
|
||||
? legacyMessageId(id, event.seq)
|
||||
: messageIds.get(inheritedId),
|
||||
role: 'user',
|
||||
content: [{
|
||||
type: 'tool-result',
|
||||
toolCallId: callId,
|
||||
content,
|
||||
isError,
|
||||
}],
|
||||
source: {
|
||||
kind: 'tool',
|
||||
callId,
|
||||
},
|
||||
},
|
||||
},
|
||||
} as SessionEvent
|
||||
}
|
||||
default:
|
||||
return event
|
||||
}
|
||||
}
|
||||
|
||||
/** Read the identified message carried by one validated current event. */
|
||||
function eventMessageId(event: SessionEvent): PersistedMessageId | undefined {
|
||||
const data = asRecord(event.data)
|
||||
const message = event.type === 'user/message' ? data : asRecord(data?.['message'])
|
||||
return typeof message?.['id'] === 'string' ? message['id'] as PersistedMessageId : undefined
|
||||
}
|
||||
|
||||
/** Materialize stored events as upgraded, validated snapshots with immutable messages. */
|
||||
function snapshotStoredEvents(events: readonly SessionEvent[], id: SessionId): SessionEvent[] {
|
||||
assertSupportedEvents(events, id)
|
||||
const messageIds = new Map<number, PersistedMessageId>()
|
||||
return events.map((event) => {
|
||||
const migratedStart = migrateLegacyTurnStartEvent(event, id)
|
||||
const migratedTurn = migrateLegacyTurnEndEvent(migratedStart, id)
|
||||
const migratedSteering = migrateLegacySteeringEvent(migratedTurn, id)
|
||||
const snapshot = snapshotSessionEvent(migrateLegacyMessageEvent(migratedSteering, id, messageIds))
|
||||
const messageId = eventMessageId(snapshot)
|
||||
if (messageId !== undefined) messageIds.set(snapshot.seq, messageId)
|
||||
return snapshot
|
||||
})
|
||||
}
|
||||
|
||||
/** Upgrade and validate an exclusively owned backend result without copying it. */
|
||||
function adoptStoredEvents(events: SessionEvent[], id: SessionId): SessionEvent[] {
|
||||
assertSupportedEvents(events, id)
|
||||
const messageIds = new Map<number, PersistedMessageId>()
|
||||
for (const [index, event] of events.entries()) {
|
||||
const migratedStart = migrateLegacyTurnStartEvent(event, id)
|
||||
const migratedTurn = migrateLegacyTurnEndEvent(migratedStart, id)
|
||||
const migratedSteering = migrateLegacySteeringEvent(migratedTurn, id)
|
||||
const adopted = adoptSessionEvent(migrateLegacyMessageEvent(migratedSteering, id, messageIds))
|
||||
events[index] = adopted
|
||||
const messageId = eventMessageId(adopted)
|
||||
if (messageId !== undefined) messageIds.set(adopted.seq, messageId)
|
||||
}
|
||||
return events
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -326,7 +674,7 @@ export class PersistenceCoordinator<TornMarker = unknown> {
|
||||
// A persisted artifact under this id (in ANY scope) blocks creation: load/
|
||||
// resume identify a session by id alone, so a second artifact would make
|
||||
// resume nondeterministic.
|
||||
if (await this.backend.openStored(meta.id) !== undefined) {
|
||||
if (await this.backend.loadStored(meta.id) !== undefined) {
|
||||
throw new Error(`session "${meta.id}" already has a persisted log on disk; load/resume it instead of creating`)
|
||||
}
|
||||
// Pure lazy: record intent only. No artifact until the first append.
|
||||
@@ -555,8 +903,9 @@ export class PersistenceCoordinator<TornMarker = unknown> {
|
||||
/**
|
||||
* Read the stored events from `fromSeq` onward, detached and non-mutating
|
||||
* (the read-from-seq primitive behind the service's `readFrom`). Runs on
|
||||
* the same per-id chain as writes. The format decoder requests a backend
|
||||
* suffix only when every selected transform can start at `fromSeq`.
|
||||
* the same per-id chain as writes; a backend with the seek-capable
|
||||
* {@link PersistenceBackend.loadStoredFrom} hook reads only the suffix,
|
||||
* every other backend reads its stored prefix and skips forward here.
|
||||
* @param id - persisted session to read.
|
||||
* @param fromSeq - first event seq to include; a non-negative safe integer.
|
||||
* @param signal - optional cancellation for queued and backend read work.
|
||||
@@ -576,64 +925,90 @@ export class PersistenceCoordinator<TornMarker = unknown> {
|
||||
fromSeq: number,
|
||||
signal?: AbortSignal,
|
||||
): Promise<{ meta: SessionHeader; events: SessionEvent[] }> {
|
||||
for (;;) {
|
||||
signal?.throwIfAborted()
|
||||
const stored = await this.backend.openStored(id, signal)
|
||||
signal?.throwIfAborted()
|
||||
if (stored === undefined) throw new SessionPersistenceNotFoundError(id)
|
||||
signal?.throwIfAborted()
|
||||
if (this.backend.loadStoredFrom !== undefined) {
|
||||
let suffix: StoredSuffix | undefined
|
||||
try {
|
||||
const current = decodeStoredSession(stored, id, fromSeq)
|
||||
const { events } = await collectDecodedEvents(current)
|
||||
signal?.throwIfAborted()
|
||||
return { meta: structuredClone(current.meta), events }
|
||||
suffix = await this.backend.loadStoredFrom(id, fromSeq, signal)
|
||||
} catch (error: unknown) {
|
||||
signal?.throwIfAborted()
|
||||
if (error instanceof SessionPersistenceRevisionConflictError) continue
|
||||
if (signal?.aborted) signal.throwIfAborted()
|
||||
throw error
|
||||
}
|
||||
signal?.throwIfAborted()
|
||||
if (suffix === undefined) throw new SessionPersistenceNotFoundError(id)
|
||||
this.assertStoredId(id, suffix.meta)
|
||||
this.assertVersion(suffix.meta)
|
||||
if (suffix.events.some(needsLegacyPrefix)) {
|
||||
const whole = await this.readStoredPrefix(id, signal)
|
||||
return { meta: whole.meta, events: whole.events.filter(event => event.seq >= fromSeq) }
|
||||
}
|
||||
const events = snapshotStoredEvents(suffix.events, id)
|
||||
this.assertEventsSupported(suffix.meta, events)
|
||||
return { meta: structuredClone(suffix.meta), events }
|
||||
}
|
||||
const whole = await this.readStoredPrefix(id, signal)
|
||||
// Sequential fallback: contiguous seqs from 0 make the suffix an index slice.
|
||||
return { meta: whole.meta, events: whole.events.slice(fromSeq) }
|
||||
}
|
||||
|
||||
/** Read one detached physical prefix without logical recovery or caching. */
|
||||
private async readStoredPrefix(
|
||||
id: SessionId,
|
||||
signal?: AbortSignal,
|
||||
): Promise<{ meta: SessionHeader; events: SessionEvent[] }> {
|
||||
signal?.throwIfAborted()
|
||||
const stored = await this.backend.loadStored(id, signal)
|
||||
signal?.throwIfAborted()
|
||||
if (stored === undefined) throw new SessionPersistenceNotFoundError(id)
|
||||
this.assertStoredId(id, stored.meta)
|
||||
this.assertVersion(stored.meta)
|
||||
const events = snapshotStoredEvents(stored.events, id)
|
||||
this.assertEventsSupported(stored.meta, events)
|
||||
return {
|
||||
meta: structuredClone(stored.meta),
|
||||
events,
|
||||
}
|
||||
}
|
||||
|
||||
/** Read, repair in memory, validate, and freeze one cold source once. */
|
||||
private async prepareCore(id: SessionId): Promise<PreparedSessionSource<TornMarker>> {
|
||||
for (;;) {
|
||||
const stored = await this.backend.openStored(id)
|
||||
if (stored === undefined) throw new SessionPersistenceNotFoundError(id)
|
||||
try {
|
||||
const current = decodeStoredSession(stored, id)
|
||||
const { events: storedEvents, tornMarker } = await collectDecodedEvents(current)
|
||||
const stored = await this.backend.loadStored(id)
|
||||
if (stored === undefined) throw new SessionPersistenceNotFoundError(id)
|
||||
try {
|
||||
const { meta, events, revision, tornMarker } = stored
|
||||
this.assertStoredId(id, meta)
|
||||
this.assertVersion(meta)
|
||||
const storedEvents = adoptStoredEvents(events, id)
|
||||
this.assertEventsSupported(meta, storedEvents)
|
||||
|
||||
// Preserve complete interrupted events and synthesize only missing closers.
|
||||
const closers = interruptedTurnClosers(storedEvents).map(adoptSessionEvent)
|
||||
const balanced = [...storedEvents, ...closers]
|
||||
const session = this.ctx.sessions.prepare(id, {
|
||||
seed: balanced,
|
||||
meta: current.meta,
|
||||
seedSource: 'persistence',
|
||||
})
|
||||
const inspection: SessionInspection = Object.freeze({
|
||||
meta: session.header,
|
||||
events: Object.freeze(balanced),
|
||||
})
|
||||
return {
|
||||
inspection,
|
||||
session,
|
||||
revision: current.revision,
|
||||
sourceVersion: current.sourceVersion,
|
||||
sessionLength: session.events.length,
|
||||
tornMarker,
|
||||
closers,
|
||||
}
|
||||
} catch (error: unknown) {
|
||||
if (error instanceof SessionPersistenceRevisionConflictError) continue
|
||||
// An unsupported format is a refusal over an intact log, not damage —
|
||||
// surface it unwrapped so callers can point at the raw artifact.
|
||||
if (error instanceof SessionFormatUnsupportedError) throw error
|
||||
throw new SessionPersistenceCorruptionError(
|
||||
`stored session "${id}" failed validation: ${String(error)}`,
|
||||
{ cause: error },
|
||||
)
|
||||
// Preserve complete interrupted events and synthesize only missing closers.
|
||||
const closers = interruptedTurnClosers(storedEvents).map(adoptSessionEvent)
|
||||
const balanced = [...storedEvents, ...closers]
|
||||
const session = this.ctx.sessions.prepare(id, {
|
||||
seed: balanced,
|
||||
meta,
|
||||
seedSource: 'persistence',
|
||||
})
|
||||
const inspection: SessionInspection = Object.freeze({
|
||||
meta: session.header,
|
||||
events: Object.freeze(balanced),
|
||||
})
|
||||
return {
|
||||
inspection,
|
||||
session,
|
||||
revision,
|
||||
sessionLength: session.events.length,
|
||||
tornMarker,
|
||||
closers,
|
||||
}
|
||||
} catch (error: unknown) {
|
||||
// An unsupported format is a refusal over an intact log, not damage —
|
||||
// surface it unwrapped so callers can point at the raw artifact.
|
||||
if (error instanceof SessionFormatUnsupportedError) throw error
|
||||
throw new SessionPersistenceCorruptionError(
|
||||
`stored session "${id}" failed validation: ${String(error)}`,
|
||||
{ cause: error },
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -648,19 +1023,6 @@ export class PersistenceCoordinator<TornMarker = unknown> {
|
||||
throw new Error(`session "${id}" already has a live persistence owner`)
|
||||
}
|
||||
if (!await this.isPreparedSourceCurrent(source)) return undefined
|
||||
if (source.sourceVersion !== SESSION_FORMAT_VERSION) {
|
||||
try {
|
||||
await this.backend.replaceStored(
|
||||
source.revision,
|
||||
source.inspection.meta,
|
||||
eventStream(source.inspection.events),
|
||||
)
|
||||
} catch (error: unknown) {
|
||||
if (!(error instanceof SessionPersistenceRevisionConflictError)) throw error
|
||||
}
|
||||
// A commit has a new revision; a conflict names a different source.
|
||||
return undefined
|
||||
}
|
||||
if (source.tornMarker !== undefined || source.closers.length > 0) {
|
||||
await this.backend.commitRepair(source.inspection.meta, source.tornMarker, source.closers)
|
||||
// The repair changed the durable revision. Reload the exact committed
|
||||
@@ -763,6 +1125,44 @@ export class PersistenceCoordinator<TornMarker = unknown> {
|
||||
}
|
||||
}
|
||||
|
||||
private assertVersion(meta: SessionHeader): void {
|
||||
if (meta.version === SESSION_FORMAT_VERSION) return
|
||||
throw this.unsupported(meta, sessionFormatVersionRefusal(meta.id, meta.version))
|
||||
}
|
||||
|
||||
/**
|
||||
* Refuse a log containing an event type this build does not know, unless the
|
||||
* writer marked the event ignorable: an unrecognized required event may
|
||||
* change how the rest of the log must be interpreted, so silently skipping
|
||||
* it would reconstruct a wrong session (the envelope contract on
|
||||
* `SessionEvent.ignorable`). Runs on NORMALIZED events — after
|
||||
* `snapshotStoredEvents`/`adoptStoredEvents` has upgraded the legacy shapes
|
||||
* this build still reads and rejected the ones it does not, so those keep
|
||||
* their specific diagnostics.
|
||||
*/
|
||||
private assertEventsSupported(meta: SessionHeader, events: readonly SessionEvent[]): void {
|
||||
for (const event of events) {
|
||||
if (KNOWN_SESSION_EVENT_TYPES.has(event.type) || event.ignorable === true) continue
|
||||
throw this.unsupported(meta, `session "${meta.id}" contains event type "${event.type}" (seq ${event.seq}) unknown to this harness and not marked ignorable; refusing to interpret the log — it was likely written by a newer harness`)
|
||||
}
|
||||
}
|
||||
|
||||
/** Build a format refusal that points at the raw artifact when the backend has one. */
|
||||
private unsupported(meta: SessionHeader, reason: string): SessionFormatUnsupportedError {
|
||||
const location = this.backend.locate?.(meta)
|
||||
return new SessionFormatUnsupportedError(
|
||||
location === undefined ? reason : `${reason} (raw log: ${location.path})`,
|
||||
location,
|
||||
)
|
||||
}
|
||||
|
||||
/** Reject backend metadata that is not bound to the requested session id. */
|
||||
private assertStoredId(id: SessionId, meta: SessionHeader): void {
|
||||
if (meta.id !== id) {
|
||||
throw new Error(`stored session identity mismatch: requested "${id}", header contains "${meta.id}"`)
|
||||
}
|
||||
}
|
||||
|
||||
// --- write path (session/event → flush drain) ---
|
||||
|
||||
private installWritePath(): void {
|
||||
@@ -895,19 +1295,11 @@ export class PersistenceCoordinator<TornMarker = unknown> {
|
||||
*/
|
||||
private async seedMatchesPersisted(id: SessionId, seed: readonly SessionEvent[], cursor: number): Promise<boolean> {
|
||||
if (cursor === 0) return true
|
||||
for (;;) {
|
||||
const stored = await this.backend.openStored(id)
|
||||
/* v8 ignore next -- a cursor > 0 means the session was materialized, so it exists */
|
||||
if (stored === undefined) return false
|
||||
try {
|
||||
const current = decodeStoredSession(stored, id)
|
||||
const { events } = await collectDecodedEvents(current)
|
||||
return seedCoversPrefix(seed, events.slice(0, cursor))
|
||||
} catch (error: unknown) {
|
||||
if (error instanceof SessionPersistenceRevisionConflictError) continue
|
||||
throw error
|
||||
}
|
||||
}
|
||||
const stored = await this.backend.loadStored(id)
|
||||
/* v8 ignore next -- a cursor > 0 means the session was materialized, so it exists */
|
||||
if (stored === undefined) return false
|
||||
this.assertStoredId(id, stored.meta)
|
||||
return seedCoversPrefix(seed, snapshotStoredEvents(stored.events, id).slice(0, cursor))
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -961,18 +1353,13 @@ export class PersistenceCoordinator<TornMarker = unknown> {
|
||||
|
||||
// case 2/3: resolve the id once across storage, then let adoption reject a
|
||||
// cwd mismatch before repair or state publication.
|
||||
for (;;) {
|
||||
const live = await this.backend.openStored(id)
|
||||
if (live === undefined) break
|
||||
const live = await this.backend.loadStored(id)
|
||||
if (live !== undefined) {
|
||||
// Do NOT route through cold preparation: that crash-repairs open turns as
|
||||
// interrupted, which is wrong for HMR while the live Session is still the
|
||||
// authority and may append the real step/turn end later.
|
||||
try {
|
||||
if (await this.adoptLivePrefix(session, seed, live)) return
|
||||
} catch (error: unknown) {
|
||||
if (error instanceof SessionPersistenceRevisionConflictError) continue
|
||||
throw error
|
||||
}
|
||||
await this.adoptLivePrefix(session, seed, live)
|
||||
return
|
||||
}
|
||||
|
||||
// case 4: a genuinely new session. Register its meta (lazy), then persist its
|
||||
@@ -993,39 +1380,28 @@ export class PersistenceCoordinator<TornMarker = unknown> {
|
||||
* the live Session is still the authority), bind ownership, and persist the
|
||||
* live suffix that was ahead of the stored prefix.
|
||||
*/
|
||||
private async adoptLivePrefix(
|
||||
session: Session,
|
||||
seed: readonly SessionEvent[],
|
||||
stored: StoredSessionSource<TornMarker>,
|
||||
): Promise<boolean> {
|
||||
const current = decodeStoredSession(stored, session.header.id)
|
||||
if (current.meta.cwd !== session.header.cwd) {
|
||||
throw new Error(`session "${session.header.id}" is already persisted at a different cwd (persisted: ${String(current.meta.cwd)}, live: ${String(session.header.cwd)}) (id collision)`)
|
||||
private async adoptLivePrefix(session: Session, seed: readonly SessionEvent[], stored: StoredPrefix<TornMarker>): Promise<void> {
|
||||
const { meta, events, tornMarker } = stored
|
||||
this.assertStoredId(session.header.id, meta)
|
||||
if (meta.cwd !== session.header.cwd) {
|
||||
throw new Error(`session "${session.header.id}" is already persisted at a different cwd (persisted: ${String(meta.cwd)}, live: ${String(session.header.cwd)}) (id collision)`)
|
||||
}
|
||||
const { events: storedEvents, tornMarker } = await collectDecodedEvents(current)
|
||||
this.assertVersion(meta)
|
||||
const storedEvents = snapshotStoredEvents(events, session.header.id)
|
||||
this.assertEventsSupported(meta, storedEvents)
|
||||
if (!seedCoversPrefix(seed, storedEvents)) {
|
||||
throw new Error(`session "${session.header.id}" already has a persisted log on disk that does not match this live session (id collision)`)
|
||||
}
|
||||
if (current.sourceVersion !== SESSION_FORMAT_VERSION) {
|
||||
await this.backend.replaceStored(
|
||||
current.revision,
|
||||
current.meta,
|
||||
eventStream(storedEvents),
|
||||
)
|
||||
// Reopen after the commit because it produced a new source revision.
|
||||
return false
|
||||
}
|
||||
// Truncate-only repair (no closers): the open turn is NOT closed here.
|
||||
if (tornMarker !== undefined) await this.backend.commitRepair(current.meta, tornMarker, [])
|
||||
if (tornMarker !== undefined) await this.backend.commitRepair(meta, tornMarker, [])
|
||||
this.states.set(session.header.id, {
|
||||
meta: { ...current.meta },
|
||||
meta: { ...meta },
|
||||
cursor: storedEvents.length,
|
||||
materialized: true,
|
||||
owner: session,
|
||||
})
|
||||
const suffix = seed.slice(storedEvents.length)
|
||||
if (suffix.length > 0) await this.appendCore(session.header.id, suffix)
|
||||
return true
|
||||
}
|
||||
|
||||
private async flush(session: Session): Promise<void> {
|
||||
|
||||
@@ -1,500 +0,0 @@
|
||||
/**
|
||||
* Static Session format decoding from backend-owned JSON records to the
|
||||
* current durable header and event types.
|
||||
* @module @deepseek-ai/dsh-session-persistence/format-decoder
|
||||
*/
|
||||
|
||||
import {
|
||||
adoptSessionEvent,
|
||||
KNOWN_SESSION_EVENT_TYPES,
|
||||
SESSION_FORMAT_VERSION,
|
||||
Session,
|
||||
SessionId,
|
||||
snapshotJsonValue,
|
||||
} from '@deepseek-ai/dsh-session'
|
||||
import type { SessionEvent, SessionHeader } from '@deepseek-ai/dsh-session'
|
||||
import {
|
||||
unversionedFormatCompatibility,
|
||||
} from './format-v0-compat.ts'
|
||||
import type { UnversionedFormatCompatibility } from './format-v0-compat.ts'
|
||||
import { asStoredRecord, assertNoRetiredSessionEvent, readStoredEventEnvelope } from './format-json.ts'
|
||||
import type { SessionLocation } from './index.ts'
|
||||
import { SESSION_FORMAT_MIGRATIONS } from './format-migrations/index.ts'
|
||||
import type { SessionPersistenceRevision } from './revision.ts'
|
||||
|
||||
/** One single-use adjacent-version migration instance. */
|
||||
interface SessionFormatMigrationInstance {
|
||||
/**
|
||||
* Transform and validate the header fields understood by this migration.
|
||||
* The detached result must carry the constructor's `to` version and preserve
|
||||
* the source id and cwd.
|
||||
* @param meta - detached input header for the constructor's `from` version.
|
||||
* @returns detached header JSON carrying the constructor's `to` version.
|
||||
*/
|
||||
header(meta: unknown): unknown
|
||||
/**
|
||||
* Transform exactly one event into detached lossless JSON while retaining
|
||||
* its sequence number. Instance fields may accumulate facts from the header
|
||||
* and earlier events.
|
||||
* @param event - detached input event in durable sequence order.
|
||||
* @returns exactly one detached event for the same sequence number.
|
||||
*/
|
||||
event(event: unknown): unknown
|
||||
/**
|
||||
* Validate accumulated state after the complete input stream reaches EOF.
|
||||
* Header-only reads do not call this method; it cannot emit another event.
|
||||
*/
|
||||
finish?(): void
|
||||
}
|
||||
|
||||
/** Static identity and constructor for one adjacent-version migration. */
|
||||
export interface SessionFormatMigration {
|
||||
/** Input Session format version. */
|
||||
readonly from: number
|
||||
/** Output Session format version; must equal `from + 1`. */
|
||||
readonly to: number
|
||||
/**
|
||||
* Create fresh state for one header decode and its optional complete event
|
||||
* stream. Instances are never shared across sessions or decode attempts.
|
||||
* @returns a single-use migration instance.
|
||||
*/
|
||||
new(): SessionFormatMigrationInstance
|
||||
}
|
||||
|
||||
/** Options for one physical event read. */
|
||||
export interface StoredEventReadOptions {
|
||||
/** First physical event sequence to request. */
|
||||
readonly fromSeq?: number
|
||||
}
|
||||
|
||||
/** Completion metadata produced after a physical event stream reaches EOF. */
|
||||
export interface StoredEventReadCompletion<TornMarker> {
|
||||
/** Backend-owned token for a recoverable physical tail. */
|
||||
readonly tornMarker?: TornMarker
|
||||
}
|
||||
|
||||
/** One revision-bound physical event stream. */
|
||||
export interface StoredEventRead<TornMarker> {
|
||||
/** Parsed JSON records from the exact source revision. */
|
||||
readonly events: AsyncIterable<unknown>
|
||||
/** Resolves only after the stream reaches EOF at the same revision. */
|
||||
readonly completed: Promise<StoredEventReadCompletion<TornMarker>>
|
||||
}
|
||||
|
||||
/** Repeatable access to one stored header and exact durable revision. */
|
||||
export interface StoredSessionSource<TornMarker> {
|
||||
/** Parsed header JSON; format validation belongs to the decoder. */
|
||||
readonly meta: unknown
|
||||
/** Exact backend revision every event read must reproduce or reject. */
|
||||
readonly revision: SessionPersistenceRevision
|
||||
/** Raw artifact location used to enrich unsupported-format diagnostics. */
|
||||
readonly location?: SessionLocation
|
||||
/**
|
||||
* Open a new event read bound to {@link revision}. A concurrent replacement
|
||||
* rejects the read instead of returning events from another revision.
|
||||
* @param options - optional suffix request.
|
||||
* @returns one independently consumable physical event read.
|
||||
*/
|
||||
readEvents(options?: StoredEventReadOptions): StoredEventRead<TornMarker>
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the standard lazy event stream and EOF metadata around one backend
|
||||
* read, shared by every first-party backend.
|
||||
* @param load - revision-checked batch loader owned by the backend.
|
||||
* @param include - whether one loaded event belongs in this physical read.
|
||||
* @param signal - optional cancellation checked between yielded events.
|
||||
* @returns an independently consumable event read.
|
||||
*/
|
||||
export function createStoredEventRead<TornMarker>(
|
||||
load: () => Promise<{ readonly events: readonly unknown[]; readonly tornMarker?: TornMarker }>,
|
||||
include: (event: unknown) => boolean,
|
||||
signal?: AbortSignal,
|
||||
): StoredEventRead<TornMarker> {
|
||||
const completed = Promise.withResolvers<StoredEventReadCompletion<TornMarker>>()
|
||||
const events = (async function* (): AsyncIterable<unknown> {
|
||||
try {
|
||||
const batch = await load()
|
||||
for (const event of batch.events) {
|
||||
signal?.throwIfAborted()
|
||||
if (include(event)) yield event
|
||||
}
|
||||
completed.resolve(batch.tornMarker === undefined ? {} : { tornMarker: batch.tornMarker })
|
||||
} catch (error: unknown) {
|
||||
completed.reject(error)
|
||||
throw error
|
||||
}
|
||||
})()
|
||||
return { events, completed: completed.promise }
|
||||
}
|
||||
|
||||
/** One decoded current-format read bound to an exact stored revision. */
|
||||
export interface DecodedSession<TornMarker> {
|
||||
/** Validated current-format header. */
|
||||
readonly meta: SessionHeader
|
||||
/** Version observed before any format migration ran. */
|
||||
readonly sourceVersion: number
|
||||
/** Exact backend revision represented by this source. */
|
||||
readonly revision: SessionPersistenceRevision
|
||||
/** Validated current-format events at or past the requested sequence. */
|
||||
readonly events: AsyncIterable<SessionEvent>
|
||||
/**
|
||||
* Completion metadata from the physical read supplying the events. Settles
|
||||
* only after the events iterable is fully consumed or fails.
|
||||
*/
|
||||
readonly completed: Promise<StoredEventReadCompletion<TornMarker>>
|
||||
}
|
||||
|
||||
/**
|
||||
* The stored log is intact but this runtime cannot faithfully interpret its
|
||||
* format version or required event vocabulary.
|
||||
*/
|
||||
export class SessionFormatUnsupportedError extends Error {
|
||||
/**
|
||||
* @param message - stable refusal reason, including the raw location when available.
|
||||
* @param location - backend artifact location when one exists.
|
||||
*/
|
||||
constructor(message: string, readonly location?: SessionLocation) {
|
||||
super(message)
|
||||
this.name = 'SessionFormatUnsupportedError'
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Direction-aware refusal text for a stored format version this build cannot
|
||||
* decode.
|
||||
* @param id - stored session identity.
|
||||
* @param version - stored format version.
|
||||
* @returns stable refusal text without a raw-location suffix.
|
||||
*/
|
||||
export function sessionFormatVersionRefusal(id: string, version: number): string {
|
||||
return version > SESSION_FORMAT_VERSION
|
||||
? `session "${id}" uses log format v${version}, but this harness reads only v${SESSION_FORMAT_VERSION}: the log was written by a newer harness — upgrade the harness to open it`
|
||||
: `session "${id}" uses log format v${version}, older than the supported v${SESSION_FORMAT_VERSION}, and this build ships no upgrade path for it`
|
||||
}
|
||||
|
||||
function buildMigrationIndex(
|
||||
migrations: readonly SessionFormatMigration[],
|
||||
): ReadonlyMap<number, SessionFormatMigration> {
|
||||
const byFrom = new Map<number, SessionFormatMigration>()
|
||||
for (const Migration of migrations) {
|
||||
if (!Number.isSafeInteger(Migration.from) || Migration.from < 0 || Migration.to !== Migration.from + 1) {
|
||||
throw new TypeError(`Session format migration must be an adjacent non-negative version, got v${Migration.from} -> v${Migration.to}`)
|
||||
}
|
||||
if (byFrom.has(Migration.from)) {
|
||||
throw new TypeError(`duplicate Session format migration from v${Migration.from}`)
|
||||
}
|
||||
if (Migration.to > SESSION_FORMAT_VERSION) {
|
||||
throw new TypeError(`Session format migration v${Migration.from} -> v${Migration.to} targets a version newer than this build's v${SESSION_FORMAT_VERSION}`)
|
||||
}
|
||||
byFrom.set(Migration.from, Migration)
|
||||
}
|
||||
// A missing migration is a per-session concern, decided by planMigrations() at decode
|
||||
// time: it refuses sessions at or below the gap, while later versions whose
|
||||
// path to the current version is complete still upgrade. Initialization
|
||||
// therefore checks only migration legality and duplicates here.
|
||||
return byFrom
|
||||
}
|
||||
|
||||
const MIGRATION_BY_FROM = buildMigrationIndex(SESSION_FORMAT_MIGRATIONS)
|
||||
|
||||
type PlannedMigration = readonly [SessionFormatMigration, SessionFormatMigrationInstance]
|
||||
|
||||
interface DecodedHeader {
|
||||
readonly meta: SessionHeader
|
||||
readonly sourceVersion: number
|
||||
readonly migrations: readonly PlannedMigration[]
|
||||
readonly unversionedCompatibility?: UnversionedFormatCompatibility
|
||||
}
|
||||
|
||||
interface StoredHeaderSource {
|
||||
readonly meta: unknown
|
||||
readonly location?: SessionLocation
|
||||
}
|
||||
|
||||
function unsupported(
|
||||
source: StoredHeaderSource,
|
||||
reason: string,
|
||||
): SessionFormatUnsupportedError {
|
||||
const location = source.location
|
||||
return new SessionFormatUnsupportedError(
|
||||
location === undefined ? reason : `${reason} (raw log: ${location.path})`,
|
||||
location,
|
||||
)
|
||||
}
|
||||
|
||||
function readSourceHeader(
|
||||
source: StoredHeaderSource,
|
||||
expectedId: SessionId,
|
||||
): { meta: Record<string, unknown>; version: number; id: SessionId } {
|
||||
const snapshot = snapshotJsonValue(source.meta)
|
||||
const meta = asStoredRecord(snapshot)
|
||||
if (meta === undefined) throw new Error('stored session header is not a lossless JSON record')
|
||||
if (!Number.isSafeInteger(meta['version'])) {
|
||||
throw new Error(`stored session header has invalid format version ${String(meta['version'])}`)
|
||||
}
|
||||
const version = meta['version'] as number
|
||||
if (version > SESSION_FORMAT_VERSION) {
|
||||
throw unsupported(source, sessionFormatVersionRefusal(String(meta['id']), version))
|
||||
}
|
||||
if (typeof meta['id'] !== 'string') throw new Error('stored session header has no string id')
|
||||
const id = SessionId(meta['id'])
|
||||
if (id !== expectedId) {
|
||||
throw new Error(`stored session identity mismatch: requested "${expectedId}", header contains "${id}"`)
|
||||
}
|
||||
return { meta, version, id }
|
||||
}
|
||||
|
||||
function planMigrations(
|
||||
source: StoredHeaderSource,
|
||||
id: SessionId,
|
||||
fromVersion: number,
|
||||
): readonly SessionFormatMigration[] {
|
||||
const migrations: SessionFormatMigration[] = []
|
||||
for (let version = fromVersion; version < SESSION_FORMAT_VERSION; version++) {
|
||||
const Migration = MIGRATION_BY_FROM.get(version)
|
||||
if (Migration === undefined) {
|
||||
throw unsupported(
|
||||
source,
|
||||
`session "${id}" uses log format v${fromVersion}, older than the supported v${SESSION_FORMAT_VERSION}, and this build has no upgrade path to it: missing v${version} -> v${version + 1}`,
|
||||
)
|
||||
}
|
||||
migrations.push(Migration)
|
||||
}
|
||||
return migrations
|
||||
}
|
||||
|
||||
function decodeHeader(
|
||||
source: StoredHeaderSource,
|
||||
expectedId: SessionId,
|
||||
): DecodedHeader {
|
||||
const stored = readSourceHeader(source, expectedId)
|
||||
const migrations: PlannedMigration[] = []
|
||||
let meta: unknown = stored.meta
|
||||
for (const Migration of planMigrations(source, stored.id, stored.version)) {
|
||||
let instance: SessionFormatMigrationInstance
|
||||
try {
|
||||
instance = new Migration()
|
||||
meta = snapshotJsonValue(instance.header(meta))
|
||||
} catch (error: unknown) {
|
||||
throw new Error(
|
||||
`session "${stored.id}" header migration v${Migration.from} -> v${Migration.to} failed`,
|
||||
{ cause: error },
|
||||
)
|
||||
}
|
||||
const record = asStoredRecord(meta)
|
||||
const actual = record?.['version']
|
||||
if (actual !== Migration.to) {
|
||||
throw new Error(`Session format migration v${Migration.from} -> v${Migration.to} returned header version ${String(actual)}`)
|
||||
}
|
||||
if (record === undefined
|
||||
|| record['id'] !== stored.id
|
||||
|| record['cwd'] !== stored.meta['cwd']) {
|
||||
throw new Error(`Session format migration v${Migration.from} -> v${Migration.to} changed session storage identity`)
|
||||
}
|
||||
migrations.push([Migration, instance])
|
||||
}
|
||||
const current = Session.create(stored.id, undefined, meta as SessionHeader).header
|
||||
const compatibility = unversionedFormatCompatibility(stored.version)
|
||||
return {
|
||||
meta: current,
|
||||
sourceVersion: stored.version,
|
||||
migrations,
|
||||
...(compatibility === undefined ? {} : { unversionedCompatibility: compatibility }),
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Decode one stored header without opening its event log. Listing uses the
|
||||
* same static format path as full Session reads.
|
||||
* @param meta - parsed backend header JSON.
|
||||
* @param expectedId - identity selected by the backend or caller.
|
||||
* @param location - optional raw artifact location for refusal diagnostics.
|
||||
* @returns the validated current-format header.
|
||||
*/
|
||||
export function decodeStoredSessionHeader(
|
||||
meta: unknown,
|
||||
expectedId: SessionId,
|
||||
location?: SessionLocation,
|
||||
): SessionHeader {
|
||||
return decodeHeader({ meta, ...location === undefined ? {} : { location } }, expectedId).meta
|
||||
}
|
||||
|
||||
function assertCurrentEnvelope(value: unknown, id: SessionId): SessionEvent {
|
||||
const snapshot = snapshotJsonValue(value)
|
||||
return readStoredEventEnvelope(snapshot, id)
|
||||
}
|
||||
|
||||
function assertCurrentEventSupported<TornMarker>(
|
||||
source: StoredSessionSource<TornMarker>,
|
||||
meta: SessionHeader,
|
||||
event: SessionEvent,
|
||||
): void {
|
||||
assertNoRetiredSessionEvent(event, meta.id)
|
||||
if (KNOWN_SESSION_EVENT_TYPES.has(event.type) || event.ignorable === true) return
|
||||
throw unsupported(
|
||||
source,
|
||||
`session "${meta.id}" contains event type "${event.type}" (seq ${event.seq}) unknown to this harness and not marked ignorable; refusing to interpret the log — it was likely written by a newer harness`,
|
||||
)
|
||||
}
|
||||
|
||||
async function* decodeCurrentEvents<TornMarker>(
|
||||
source: StoredSessionSource<TornMarker>,
|
||||
meta: SessionHeader,
|
||||
events: AsyncIterable<unknown>,
|
||||
expectedSeq: number,
|
||||
): AsyncIterable<SessionEvent> {
|
||||
let nextSeq = expectedSeq
|
||||
for await (const raw of events) {
|
||||
const event = assertCurrentEnvelope(raw, meta.id)
|
||||
if (event.seq !== nextSeq) {
|
||||
throw new Error(`session "${meta.id}" event seq mismatch: expected ${nextSeq}, got ${event.seq}`)
|
||||
}
|
||||
const current = adoptSessionEvent(event)
|
||||
assertCurrentEventSupported(source, meta, current)
|
||||
nextSeq += 1
|
||||
yield current
|
||||
}
|
||||
}
|
||||
|
||||
async function* transformEvents(
|
||||
events: AsyncIterable<unknown>,
|
||||
migrations: readonly PlannedMigration[],
|
||||
id: SessionId,
|
||||
): AsyncIterable<unknown> {
|
||||
for await (let value of events) {
|
||||
for (const [Migration, instance] of migrations) {
|
||||
const sourceSeq = asStoredRecord(value)?.['seq']
|
||||
let output: unknown
|
||||
try {
|
||||
output = snapshotJsonValue(instance.event(value))
|
||||
if (output === undefined) {
|
||||
throw new Error('migration returned an event that is not losslessly JSON-serializable')
|
||||
}
|
||||
} catch (error: unknown) {
|
||||
throw new Error(
|
||||
`session "${id}" event migration v${Migration.from} -> v${Migration.to} failed at seq ${String(sourceSeq)}`,
|
||||
{ cause: error },
|
||||
)
|
||||
}
|
||||
const targetSeq = asStoredRecord(output)?.['seq']
|
||||
if (targetSeq !== sourceSeq) {
|
||||
throw new Error(`session "${id}" event migration v${Migration.from} -> v${Migration.to} changed event seq ${String(sourceSeq)} to ${String(targetSeq)}`)
|
||||
}
|
||||
value = output
|
||||
}
|
||||
yield value
|
||||
}
|
||||
for (const [Migration, instance] of migrations) {
|
||||
try {
|
||||
instance.finish?.()
|
||||
} catch (error: unknown) {
|
||||
throw new Error(
|
||||
`session "${id}" event migration v${Migration.from} -> v${Migration.to} failed at EOF`,
|
||||
{ cause: error },
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async function* snapshotStoredEvents(
|
||||
events: AsyncIterable<unknown>,
|
||||
id: SessionId,
|
||||
): AsyncIterable<unknown> {
|
||||
for await (const event of events) {
|
||||
const snapshot = snapshotJsonValue(event)
|
||||
if (snapshot === undefined) {
|
||||
throw new Error(`session "${id}" contains an event that is not losslessly JSON-serializable`)
|
||||
}
|
||||
yield snapshot
|
||||
}
|
||||
}
|
||||
|
||||
function decodedRead<TornMarker>(
|
||||
source: StoredSessionSource<TornMarker>,
|
||||
header: DecodedHeader,
|
||||
requestedFromSeq: number,
|
||||
): {
|
||||
readonly events: AsyncIterable<SessionEvent>
|
||||
readonly completed: Promise<StoredEventReadCompletion<TornMarker>>
|
||||
} {
|
||||
const completion = Promise.withResolvers<StoredEventReadCompletion<TornMarker>>()
|
||||
const migrating = header.migrations.length > 0
|
||||
const compatibility = header.unversionedCompatibility
|
||||
let physical: StoredEventRead<TornMarker> | undefined
|
||||
|
||||
const events = (async function* (): AsyncIterable<SessionEvent> {
|
||||
try {
|
||||
let physicalFromSeq = migrating ? 0 : requestedFromSeq
|
||||
physical = source.readEvents({ fromSeq: physicalFromSeq })
|
||||
void physical.completed.catch(() => undefined)
|
||||
let raw: AsyncIterable<unknown> = physical.events
|
||||
let physicalCompletion: StoredEventReadCompletion<TornMarker> | undefined
|
||||
|
||||
if (!migrating && requestedFromSeq > 0 && compatibility !== undefined) {
|
||||
const suffix: unknown[] = []
|
||||
for await (const value of raw) suffix.push(value)
|
||||
physicalCompletion = await physical.completed
|
||||
if (suffix.some(value => compatibility.requiresPrefix(value))) {
|
||||
physicalFromSeq = 0
|
||||
physical = source.readEvents({ fromSeq: 0 })
|
||||
void physical.completed.catch(() => undefined)
|
||||
raw = physical.events
|
||||
physicalCompletion = undefined
|
||||
} else {
|
||||
raw = (async function* () {
|
||||
for (const value of suffix) yield await Promise.resolve(value)
|
||||
})()
|
||||
}
|
||||
}
|
||||
|
||||
const storedEvents = snapshotStoredEvents(raw, header.meta.id)
|
||||
const canonicalEvents = compatibility === undefined
|
||||
? storedEvents
|
||||
: compatibility.canonicalizeEvents(storedEvents, header.meta.id)
|
||||
const transformed = transformEvents(
|
||||
canonicalEvents,
|
||||
header.migrations,
|
||||
header.meta.id,
|
||||
)
|
||||
const current = decodeCurrentEvents(source, header.meta, transformed, physicalFromSeq)
|
||||
for await (const event of current) {
|
||||
if (event.seq >= requestedFromSeq) yield event
|
||||
}
|
||||
completion.resolve(physicalCompletion ?? await physical.completed)
|
||||
} catch (error: unknown) {
|
||||
completion.reject(error)
|
||||
throw error
|
||||
}
|
||||
})()
|
||||
|
||||
return { events, completed: completion.promise }
|
||||
}
|
||||
|
||||
/**
|
||||
* Decode one backend source through the static adjacent-version migrations and
|
||||
* the current header/event validators. Format selection is complete before any
|
||||
* consumer-specific recovery runs.
|
||||
* @param source - backend-owned header, revision, and event reader factory.
|
||||
* @param expectedId - session identity selected by the caller.
|
||||
* @param fromSeq - first current-format event sequence to return.
|
||||
* @returns one decoded current-format stream bound to the stored revision.
|
||||
*/
|
||||
export function decodeStoredSession<TornMarker>(
|
||||
source: StoredSessionSource<TornMarker>,
|
||||
expectedId: SessionId,
|
||||
fromSeq = 0,
|
||||
): DecodedSession<TornMarker> {
|
||||
if (!Number.isSafeInteger(fromSeq) || fromSeq < 0) {
|
||||
throw new TypeError(`stored event fromSeq must be a non-negative safe integer, got ${String(fromSeq)}`)
|
||||
}
|
||||
const header = decodeHeader(source, expectedId)
|
||||
const read = decodedRead(source, header, fromSeq)
|
||||
return {
|
||||
meta: header.meta,
|
||||
sourceVersion: header.sourceVersion,
|
||||
revision: source.revision,
|
||||
events: read.events,
|
||||
completed: read.completed,
|
||||
}
|
||||
}
|
||||
@@ -1,56 +0,0 @@
|
||||
/** Shared JSON validation for stored Session format records. */
|
||||
|
||||
import type { SessionEvent, SessionId } from '@deepseek-ai/dsh-session'
|
||||
|
||||
/**
|
||||
* Narrow an unknown JSON value to a non-array object.
|
||||
* @param value - parsed JSON value.
|
||||
* @returns the object, or `undefined` for every other JSON value.
|
||||
*/
|
||||
export function asStoredRecord(value: unknown): Record<string, unknown> | undefined {
|
||||
return typeof value === 'object' && value !== null && !Array.isArray(value)
|
||||
? value as Record<string, unknown>
|
||||
: undefined
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate fields common to every stored Session event envelope.
|
||||
* @param value - detached parsed event JSON.
|
||||
* @param id - Session identity used in diagnostics.
|
||||
* @returns the structurally valid event envelope.
|
||||
*/
|
||||
export function readStoredEventEnvelope(value: unknown, id: SessionId): SessionEvent {
|
||||
const event = asStoredRecord(value)
|
||||
if (event === undefined) throw new Error(`session "${id}" contains a non-record event`)
|
||||
if (typeof event['type'] !== 'string') throw new Error(`session "${id}" contains an event without a string type`)
|
||||
if (!Number.isSafeInteger(event['seq']) || (event['seq'] as number) < 0) {
|
||||
throw new Error(`session "${id}" contains event type "${event['type']}" with invalid seq ${String(event['seq'])}`)
|
||||
}
|
||||
if (typeof event['time'] !== 'number' || !Number.isFinite(event['time'])) {
|
||||
throw new Error(`session "${id}" contains event type "${event['type']}" at seq ${String(event['seq'])} with invalid time`)
|
||||
}
|
||||
if (!Object.hasOwn(event, 'data')) {
|
||||
throw new Error(`session "${id}" contains event type "${event['type']}" at seq ${String(event['seq'])} without data`)
|
||||
}
|
||||
return event as unknown as SessionEvent
|
||||
}
|
||||
|
||||
/**
|
||||
* Reject event records retired before the current durable event vocabulary.
|
||||
* @param event - current-envelope event presented for reading or writing.
|
||||
* @param id - Session identity used in diagnostics.
|
||||
*/
|
||||
export function assertNoRetiredSessionEvent(event: SessionEvent, id: SessionId): void {
|
||||
const retiredType: string = 'request/header-delta'
|
||||
if (event.type === retiredType) {
|
||||
throw new Error(`session "${id}" contains unsupported legacy request/header-delta event at seq ${event.seq}`)
|
||||
}
|
||||
const retiredModeType: string = 'mode/set'
|
||||
if (event.type === retiredModeType) {
|
||||
throw new Error(`session "${id}" contains unsupported legacy mode/set event at seq ${event.seq}`)
|
||||
}
|
||||
if (event.type === 'request/header'
|
||||
&& (event.data as { reason?: string }).reason === 'fallback') {
|
||||
throw new Error(`session "${id}" contains unsupported legacy request/header reason "fallback" at seq ${event.seq}`)
|
||||
}
|
||||
}
|
||||
@@ -1,6 +0,0 @@
|
||||
/** Static adjacent-version Session format migrations shipped by this build. */
|
||||
|
||||
import type { SessionFormatMigration } from '../format-decoder.ts'
|
||||
|
||||
/** Ordered durable format migrations; format v0 is current, so the chain is empty. */
|
||||
export const SESSION_FORMAT_MIGRATIONS: readonly SessionFormatMigration[] = Object.freeze([])
|
||||
@@ -1,297 +0,0 @@
|
||||
/**
|
||||
* Same-version normalization for durable format-v0 Session records.
|
||||
* @module @deepseek-ai/dsh-session-persistence/format-v0-compat
|
||||
*/
|
||||
|
||||
import type { SessionEvent, SessionId } from '@deepseek-ai/dsh-session'
|
||||
import { asStoredRecord, readStoredEventEnvelope } from './format-json.ts'
|
||||
|
||||
/** One format-specific normalizer selected before adjacent-version migrations. */
|
||||
export interface UnversionedFormatCompatibility {
|
||||
/** Header version whose historical records require this normalizer. */
|
||||
readonly version: number
|
||||
/**
|
||||
* Whether converting one suffix record requires facts from earlier events.
|
||||
* @param value - parsed event JSON from a suffix read.
|
||||
* @returns whether the decoder must reopen the complete event stream.
|
||||
*/
|
||||
requiresPrefix(value: unknown): boolean
|
||||
/**
|
||||
* Convert recognized historical records into the canonical representation
|
||||
* carrying the same version number.
|
||||
* @param events - parsed event JSON in durable sequence order.
|
||||
* @param sessionId - identity read from the stored header.
|
||||
* @returns a lazy stream in the canonical representation for {@link version}.
|
||||
*/
|
||||
canonicalizeEvents(events: AsyncIterable<unknown>, sessionId: SessionId): AsyncIterable<unknown>
|
||||
}
|
||||
|
||||
function hasOnlyKeys(
|
||||
record: Record<string, unknown>,
|
||||
required: readonly string[],
|
||||
optional: readonly string[] = [],
|
||||
): boolean {
|
||||
const allowed = [...required, ...optional]
|
||||
return Object.keys(record).every(key => allowed.includes(key))
|
||||
&& required.every(key => Object.hasOwn(record, key))
|
||||
}
|
||||
|
||||
type PersistedMessageId = SessionEvent<'user/message'>['data']['id']
|
||||
|
||||
function legacyMessageId(id: SessionId, seq: number): PersistedMessageId {
|
||||
return `legacy-message:${id}:${seq}` as PersistedMessageId
|
||||
}
|
||||
|
||||
function replacementStart(event: SessionEvent): number | undefined {
|
||||
const op = asStoredRecord((event as SessionEvent & { surfaceOp?: unknown }).surfaceOp)
|
||||
return op?.['op'] === 'replace' && typeof op['start'] === 'number'
|
||||
? op['start']
|
||||
: undefined
|
||||
}
|
||||
|
||||
function requiresV0Prefix(value: unknown): boolean {
|
||||
const event = asStoredRecord(value)
|
||||
if (event === undefined) return false
|
||||
const data = asStoredRecord(event['data'])
|
||||
if (event['type'] === 'steering/message') return true
|
||||
if (data === undefined) return false
|
||||
switch (event['type']) {
|
||||
case 'user/message':
|
||||
return !Object.hasOwn(data, 'id') && Object.hasOwn(data, 'content')
|
||||
case 'assistant/message':
|
||||
return !Object.hasOwn(data, 'message') && Object.hasOwn(data, 'content')
|
||||
case 'tool/result':
|
||||
return !Object.hasOwn(data, 'message') && Object.hasOwn(data, 'callId')
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
function readV0Event(value: unknown, id: SessionId): SessionEvent {
|
||||
return readStoredEventEnvelope(value, id)
|
||||
}
|
||||
|
||||
/**
|
||||
* PR #2302 changed these durable v0 discriminants without a format-version bump.
|
||||
* @see https://github.com/deepseek-harness/deepseek-harness/pull/2302
|
||||
*/
|
||||
function canonicalizeLegacyCompactionEvent(event: SessionEvent): SessionEvent {
|
||||
const type: string = event.type
|
||||
switch (type) {
|
||||
case 'compact/start':
|
||||
return { ...event, type: 'compaction/start' } as SessionEvent
|
||||
case 'compact/summary':
|
||||
return { ...event, type: 'compaction/summary' } as SessionEvent
|
||||
case 'compact/end':
|
||||
return { ...event, type: 'compaction/end' } as SessionEvent
|
||||
case 'compact/prune':
|
||||
return { ...event, type: 'compaction/prune' } as SessionEvent
|
||||
default:
|
||||
return event
|
||||
}
|
||||
}
|
||||
|
||||
function canonicalizeLegacySteeringEvent(event: SessionEvent, id: SessionId): SessionEvent {
|
||||
const legacyType: string = 'steering/message'
|
||||
if (event.type !== legacyType) return event
|
||||
const data = asStoredRecord(event.data)
|
||||
if (data === undefined) {
|
||||
throw new Error(`session "${id}" contains malformed pre-react-loop steering/message at seq ${event.seq}`)
|
||||
}
|
||||
const wrapped = asStoredRecord(data['message'])
|
||||
if (wrapped !== undefined && Number.isSafeInteger(data['turn'])
|
||||
&& hasOnlyKeys(data, ['turn', 'message'])) {
|
||||
return { ...event, type: 'user/message', data: wrapped } as SessionEvent
|
||||
}
|
||||
if (!Number.isSafeInteger(data['turn']) || !hasOnlyKeys(data, ['turn', 'content', 'source'])) {
|
||||
throw new Error(`session "${id}" contains malformed pre-react-loop steering/message at seq ${event.seq}`)
|
||||
}
|
||||
const { turn: _turn, ...message } = data
|
||||
return {
|
||||
...event,
|
||||
type: 'user/message',
|
||||
data: { ...message, id: legacyMessageId(id, event.seq), role: 'user' },
|
||||
} as SessionEvent
|
||||
}
|
||||
|
||||
function canonicalizeLegacyTurnStartEvent(event: SessionEvent, id: SessionId): SessionEvent {
|
||||
if (event.type !== 'turn/start') return event
|
||||
const data = asStoredRecord(event.data)
|
||||
if (data === undefined || !Object.hasOwn(data, 'trigger')) return event
|
||||
const trigger = asStoredRecord(data['trigger'])
|
||||
if (!Number.isSafeInteger(data['turn']) || (data['turn'] as number) < 1
|
||||
|| !hasOnlyKeys(data, ['turn', 'trigger'])
|
||||
|| trigger === undefined || typeof trigger['kind'] !== 'string' || trigger['kind'].length === 0) {
|
||||
throw new Error(`session "${id}" contains malformed pre-react-loop turn/start at seq ${event.seq}`)
|
||||
}
|
||||
return { ...event, data: { turn: data['turn'] } } as SessionEvent
|
||||
}
|
||||
|
||||
function canonicalizeLegacyTurnEndEvent(event: SessionEvent, id: SessionId): SessionEvent {
|
||||
if (event.type !== 'turn/end') return event
|
||||
const data = asStoredRecord(event.data)
|
||||
if (data === undefined) return event
|
||||
const malformed = (): never => {
|
||||
throw new Error(`session "${id}" contains malformed pre-react-loop turn/end at seq ${event.seq}`)
|
||||
}
|
||||
const reason = asStoredRecord(data['reason'])
|
||||
if (!Number.isSafeInteger(data['turn']) || (data['turn'] as number) < 1
|
||||
|| !hasOnlyKeys(data, ['turn', 'reason'])
|
||||
|| reason === undefined || typeof reason['kind'] !== 'string') return malformed()
|
||||
|
||||
let currentReason: Record<string, unknown> | undefined
|
||||
switch (reason['kind']) {
|
||||
case 'completed':
|
||||
case 'blocked':
|
||||
case 'max-tokens':
|
||||
case 'interrupted':
|
||||
if (!hasOnlyKeys(reason, ['kind'])) return malformed()
|
||||
return event
|
||||
case 'aborted':
|
||||
if (Object.hasOwn(reason, 'reason')) return event
|
||||
if (!hasOnlyKeys(reason, ['kind'])) return malformed()
|
||||
currentReason = { kind: 'aborted', reason: { kind: 'legacy' } }
|
||||
break
|
||||
case 'disposed':
|
||||
if (!hasOnlyKeys(reason, ['kind'])) return malformed()
|
||||
currentReason = { kind: 'aborted', reason: { kind: 'disposed' } }
|
||||
break
|
||||
case 'error': {
|
||||
if (Object.hasOwn(reason, 'error')) return event
|
||||
if (!Number.isSafeInteger(reason['step']) || (reason['step'] as number) < 0) return malformed()
|
||||
const failure = asStoredRecord(reason['failure'])
|
||||
if (failure !== undefined && hasOnlyKeys(reason, ['kind', 'step', 'failure'])
|
||||
&& hasOnlyKeys(failure, ['message', 'code'], ['status', 'providerRetryAfterMs', 'requestId'])
|
||||
&& typeof failure['message'] === 'string' && typeof failure['code'] === 'string'
|
||||
&& (failure['status'] === undefined || typeof failure['status'] === 'number')
|
||||
&& (failure['providerRetryAfterMs'] === undefined || typeof failure['providerRetryAfterMs'] === 'number')
|
||||
&& (failure['requestId'] === undefined || typeof failure['requestId'] === 'string')) {
|
||||
currentReason = { kind: 'error', error: failure }
|
||||
break
|
||||
}
|
||||
const messageKeys = reason['code'] === undefined
|
||||
? ['kind', 'step', 'message']
|
||||
: ['kind', 'step', 'message', 'code']
|
||||
if (!hasOnlyKeys(reason, messageKeys)
|
||||
|| typeof reason['message'] !== 'string'
|
||||
|| (reason['code'] !== undefined && typeof reason['code'] !== 'string')) return malformed()
|
||||
currentReason = {
|
||||
kind: 'error',
|
||||
error: {
|
||||
message: reason['message'],
|
||||
code: typeof reason['code'] === 'string' ? reason['code'] : 'UNKNOWN',
|
||||
},
|
||||
}
|
||||
break
|
||||
}
|
||||
default:
|
||||
return event
|
||||
}
|
||||
return { ...event, data: { ...data, reason: currentReason } } as SessionEvent
|
||||
}
|
||||
|
||||
function canonicalizeLegacyMessageEvent(
|
||||
event: SessionEvent,
|
||||
id: SessionId,
|
||||
messageIds: ReadonlyMap<number, PersistedMessageId>,
|
||||
): SessionEvent {
|
||||
const data = asStoredRecord(event.data)
|
||||
if (data === undefined) return event
|
||||
switch (event.type) {
|
||||
case 'user/message':
|
||||
if (Object.hasOwn(data, 'id') || Object.hasOwn(data, 'role')
|
||||
|| Object.hasOwn(data, 'message')
|
||||
|| !Object.hasOwn(data, 'content') || !Object.hasOwn(data, 'source')) return event
|
||||
return { ...event, data: { ...data, id: legacyMessageId(id, event.seq), role: 'user' } } as SessionEvent
|
||||
case 'assistant/message': {
|
||||
if (Object.hasOwn(data, 'message')
|
||||
|| !Object.hasOwn(data, 'content') || !Object.hasOwn(data, 'provenance')) return event
|
||||
const { content, provenance, ...eventData } = data
|
||||
return {
|
||||
...event,
|
||||
data: {
|
||||
...eventData,
|
||||
message: {
|
||||
id: legacyMessageId(id, event.seq),
|
||||
role: 'assistant',
|
||||
content,
|
||||
source: { ...asStoredRecord(provenance), kind: 'model' },
|
||||
},
|
||||
},
|
||||
} as SessionEvent
|
||||
}
|
||||
case 'tool/result': {
|
||||
if (Object.hasOwn(data, 'message')
|
||||
|| !Object.hasOwn(data, 'callId') || !Object.hasOwn(data, 'content')
|
||||
|| !Object.hasOwn(data, 'isError')) return event
|
||||
const { callId, content, isError, ...eventData } = data
|
||||
const inheritedId = replacementStart(event)
|
||||
return {
|
||||
...event,
|
||||
data: {
|
||||
...eventData,
|
||||
message: {
|
||||
id: inheritedId === undefined ? legacyMessageId(id, event.seq) : messageIds.get(inheritedId),
|
||||
role: 'user',
|
||||
content: [{ type: 'tool-result', toolCallId: callId, content, isError }],
|
||||
source: { kind: 'tool', callId },
|
||||
},
|
||||
},
|
||||
} as SessionEvent
|
||||
}
|
||||
default:
|
||||
return event
|
||||
}
|
||||
}
|
||||
|
||||
function eventMessageId(event: SessionEvent): PersistedMessageId | undefined {
|
||||
const data = asStoredRecord(event.data)
|
||||
const message = event.type === 'user/message' ? data : asStoredRecord(data?.['message'])
|
||||
return typeof message?.['id'] === 'string' ? message['id'] as PersistedMessageId : undefined
|
||||
}
|
||||
|
||||
async function* canonicalizeV0Events(
|
||||
events: AsyncIterable<unknown>,
|
||||
id: SessionId,
|
||||
): AsyncIterable<unknown> {
|
||||
const messageIds = new Map<number, PersistedMessageId>()
|
||||
for await (const value of events) {
|
||||
const event = readV0Event(value, id)
|
||||
const compaction = canonicalizeLegacyCompactionEvent(event)
|
||||
const turnStart = canonicalizeLegacyTurnStartEvent(compaction, id)
|
||||
const turnEnd = canonicalizeLegacyTurnEndEvent(turnStart, id)
|
||||
const steering = canonicalizeLegacySteeringEvent(turnEnd, id)
|
||||
const canonical = canonicalizeLegacyMessageEvent(steering, id, messageIds)
|
||||
const messageId = eventMessageId(canonical)
|
||||
if (messageId !== undefined) messageIds.set(canonical.seq, messageId)
|
||||
yield canonical
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Durable v0 includes first-party records whose structural changes were not
|
||||
* accompanied by a format-version change. Their headers cannot select an
|
||||
* adjacent-version migration, so this exact legacy recognition runs before
|
||||
* any v0-to-v1 step and produces canonical v0 without changing the version.
|
||||
* It remains necessary while v0 is current and whenever v0 is an upgrade
|
||||
* source. Normalization alone is read-only; a selected versioned migration
|
||||
* causes the canonicalized events to participate in atomic replacement.
|
||||
*/
|
||||
const V0_UNVERSIONED_FORMAT_COMPATIBILITY: UnversionedFormatCompatibility = Object.freeze({
|
||||
version: 0,
|
||||
requiresPrefix: requiresV0Prefix,
|
||||
canonicalizeEvents: canonicalizeV0Events,
|
||||
})
|
||||
|
||||
/**
|
||||
* Select same-version compatibility for one stored header version.
|
||||
* @param version - format version read from the stored header.
|
||||
* @returns the static normalizer for that version, if one is required.
|
||||
*/
|
||||
export function unversionedFormatCompatibility(
|
||||
version: number,
|
||||
): UnversionedFormatCompatibility | undefined {
|
||||
return version === V0_UNVERSIONED_FORMAT_COMPATIBILITY.version
|
||||
? V0_UNVERSIONED_FORMAT_COMPATIBILITY
|
||||
: undefined
|
||||
}
|
||||
@@ -9,11 +9,10 @@ import { Context, Service } from '@deepseek-ai/cordis'
|
||||
import { SessionPreparation } from '@deepseek-ai/dsh-session'
|
||||
import type { Session, SessionEvent, SessionId, SessionHeader } from '@deepseek-ai/dsh-session'
|
||||
import type { SessionPersistenceRevision } from './revision.ts'
|
||||
import { createStoredEventRead, type StoredEventRead } from './format-decoder.ts'
|
||||
|
||||
// Re-export the metadata vocabulary so Consumers import it from the Service Definition.
|
||||
export type { SessionHeader } from '@deepseek-ai/dsh-session'
|
||||
export { SessionPersistenceRevision, SessionPersistenceRevisionConflictError } from './revision.ts'
|
||||
export { SessionPersistenceRevision } from './revision.ts'
|
||||
export { SessionPersistenceNotFoundError } from './errors.ts'
|
||||
|
||||
/** Lightweight immutable source identity returned without loading a full log. */
|
||||
@@ -68,18 +67,17 @@ export {
|
||||
DEFAULT_WRITE_BATCH_MAX_DELAY_MS,
|
||||
MAX_WRITE_BATCH_DELAY_MS,
|
||||
PersistenceCoordinator,
|
||||
SessionFormatUnsupportedError,
|
||||
SessionPersistenceCorruptionError,
|
||||
sessionFormatVersionRefusal,
|
||||
} from './coordinator.ts'
|
||||
export type {
|
||||
PersistenceBackend,
|
||||
PersistenceCoordinatorOptions,
|
||||
StoredPrefix,
|
||||
StoredSuffix,
|
||||
} from './coordinator.ts'
|
||||
export {
|
||||
createStoredEventRead,
|
||||
decodeStoredSessionHeader,
|
||||
SessionFormatUnsupportedError,
|
||||
sessionFormatVersionRefusal,
|
||||
} from './format-decoder.ts'
|
||||
|
||||
declare module '@deepseek-ai/cordis' {
|
||||
interface Context {
|
||||
sessionPersistence: SessionPersistence
|
||||
@@ -109,21 +107,6 @@ export abstract class SessionPersistence extends Service {
|
||||
super(ctx, 'sessionPersistence')
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the standard lazy event stream and EOF metadata around one backend read.
|
||||
* @param load - revision-checked batch loader owned by the backend.
|
||||
* @param include - whether one loaded event belongs in this physical read.
|
||||
* @param signal - optional cancellation checked between yielded events.
|
||||
* @returns an independently consumable event read.
|
||||
*/
|
||||
protected createStoredEventRead<TornMarker>(
|
||||
load: () => Promise<{ readonly events: readonly unknown[]; readonly tornMarker?: TornMarker }>,
|
||||
include: (event: unknown) => boolean,
|
||||
signal?: AbortSignal,
|
||||
): StoredEventRead<TornMarker> {
|
||||
return createStoredEventRead(load, include, signal)
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve this backend's independent local artifact for a session without
|
||||
* reading, creating, flushing, or otherwise materializing it. Backends such
|
||||
@@ -300,11 +283,3 @@ export abstract class SessionPersistence extends Service {
|
||||
}
|
||||
|
||||
export default SessionPersistence
|
||||
|
||||
export type {
|
||||
SessionFormatMigration,
|
||||
StoredEventRead,
|
||||
StoredEventReadCompletion,
|
||||
StoredEventReadOptions,
|
||||
StoredSessionSource,
|
||||
} from './format-decoder.ts'
|
||||
|
||||
@@ -16,12 +16,3 @@ export type SessionPersistenceRevision = Branded<'SessionPersistenceRevision'>
|
||||
export function SessionPersistenceRevision(value: string): SessionPersistenceRevision {
|
||||
return value as SessionPersistenceRevision
|
||||
}
|
||||
|
||||
/** A repeatable source can no longer reproduce the revision it represents. */
|
||||
export class SessionPersistenceRevisionConflictError extends Error {
|
||||
/** @param message - source identity and expected revision context. */
|
||||
constructor(message: string) {
|
||||
super(message)
|
||||
this.name = 'SessionPersistenceRevisionConflictError'
|
||||
}
|
||||
}
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -5,13 +5,10 @@ import type { SessionEvent, SessionHeader } from '@deepseek-ai/dsh-session'
|
||||
import {
|
||||
DEFAULT_PREPARED_SESSION_CACHE_SIZE, DEFAULT_WRITE_BATCH_MAX_DELAY_MS, MAX_WRITE_BATCH_DELAY_MS,
|
||||
SessionPersistence, SessionPersistenceRevision, PersistenceCoordinator,
|
||||
SessionPersistenceRevisionConflictError,
|
||||
type PersistenceBackend, type SessionPersistenceSnapshot, type StoredEventRead,
|
||||
type StoredEventReadCompletion, type StoredSessionSource,
|
||||
type PersistenceBackend, type SessionPersistenceSnapshot, type StoredPrefix, type StoredSuffix,
|
||||
} from '../src/index.ts'
|
||||
import { runPersistenceContract, meta, oneTurnLog } from './contract.ts'
|
||||
import { runCoordinatorContract, type CoordinatorFixture } from './coordinator-contract.ts'
|
||||
import * as formatDecoder from '../src/format-decoder.ts'
|
||||
|
||||
/** The durable store shape: materialized sessions only (no lazy entries). */
|
||||
type MemoryStore = Map<string, { meta: SessionHeader; events: SessionEvent[] }>
|
||||
@@ -21,45 +18,6 @@ function memoryRevision(entry: { meta: SessionHeader; events: SessionEvent[] }):
|
||||
return SessionPersistenceRevision(JSON.stringify(entry))
|
||||
}
|
||||
|
||||
/** Build one lazy physical read whose completion follows iterator exhaustion. */
|
||||
function storedRead<TornMarker>(
|
||||
load: () => Promise<{ events: readonly unknown[]; tornMarker?: TornMarker }>,
|
||||
): StoredEventRead<TornMarker> {
|
||||
const completed = Promise.withResolvers<StoredEventReadCompletion<TornMarker>>()
|
||||
const events = (async function* (): AsyncIterable<unknown> {
|
||||
try {
|
||||
const loaded = await load()
|
||||
yield* loaded.events
|
||||
completed.resolve(loaded.tornMarker === undefined ? {} : { tornMarker: loaded.tornMarker })
|
||||
} catch (error: unknown) {
|
||||
completed.reject(error)
|
||||
throw error
|
||||
}
|
||||
})()
|
||||
return { events, completed: completed.promise }
|
||||
}
|
||||
|
||||
/** Materialize an async replacement stream for the map-backed test stores. */
|
||||
async function collectReplacement(events: AsyncIterable<SessionEvent>): Promise<SessionEvent[]> {
|
||||
const collected: SessionEvent[] = []
|
||||
for await (const event of events) collected.push(structuredClone(event))
|
||||
return collected
|
||||
}
|
||||
|
||||
async function replaceMemoryStored(
|
||||
store: MemoryStore,
|
||||
expectedRevision: SessionPersistenceRevision,
|
||||
m: SessionHeader,
|
||||
events: AsyncIterable<SessionEvent>,
|
||||
): Promise<void> {
|
||||
const entry = store.get(m.id)
|
||||
if (entry === undefined || memoryRevision(entry) !== expectedRevision) {
|
||||
throw new SessionPersistenceRevisionConflictError(`session "${m.id}" changed before replacement`)
|
||||
}
|
||||
if (entry.meta.cwd !== m.cwd) throw new Error(`replacement for session "${m.id}" changes its stored identity`)
|
||||
store.set(m.id, { meta: structuredClone(m), events: await collectReplacement(events) })
|
||||
}
|
||||
|
||||
/** An obsolete event fixture that emulates an untyped pre-change producer. */
|
||||
function legacyHeaderDelta(seq = 0): SessionEvent {
|
||||
return {
|
||||
@@ -124,7 +82,7 @@ class MemoryPersistence extends SessionPersistence implements PersistenceBackend
|
||||
super(ctx)
|
||||
// Assign the store BEFORE constructing the coordinator: the coordinator's
|
||||
// constructor installs the write path and synchronously seeds existing live
|
||||
// sessions through openStored(), so store must exist first.
|
||||
// sessions through loadStored(), so store must exist first.
|
||||
this.store = config?.store ?? new Map<string, { meta: SessionHeader; events: SessionEvent[] }>()
|
||||
this.coordinator = new PersistenceCoordinator<never>(this.ctx, this)
|
||||
}
|
||||
@@ -171,20 +129,13 @@ class MemoryPersistence extends SessionPersistence implements PersistenceBackend
|
||||
// --- PersistenceBackend hooks (the Map storage primitives) ---
|
||||
|
||||
// A Map-backed store has no torn tails, so `tornMarker` is never set.
|
||||
async openStored(id: SessionId): Promise<StoredSessionSource<never> | undefined> {
|
||||
async loadStored(id: SessionId): Promise<StoredPrefix<never> | undefined> {
|
||||
const entry = this.store.get(id)
|
||||
if (!entry) return undefined
|
||||
const revision = memoryRevision(entry)
|
||||
return {
|
||||
meta: structuredClone(entry.meta),
|
||||
revision,
|
||||
readEvents: ({ fromSeq = 0 } = {}) => storedRead(async () => {
|
||||
const current = this.store.get(id)
|
||||
if (current === undefined || memoryRevision(current) !== revision) {
|
||||
throw new SessionPersistenceRevisionConflictError(`session "${id}" changed during read`)
|
||||
}
|
||||
return { events: structuredClone(current.events.filter(event => event.seq >= fromSeq)) }
|
||||
}),
|
||||
events: structuredClone(entry.events),
|
||||
revision: memoryRevision(entry),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -223,14 +174,6 @@ class MemoryPersistence extends SessionPersistence implements PersistenceBackend
|
||||
if (closers.length > 0) entry.events.push(...structuredClone(closers) as SessionEvent[])
|
||||
}
|
||||
|
||||
async replaceStored(
|
||||
expectedRevision: SessionPersistenceRevision,
|
||||
m: SessionHeader,
|
||||
events: AsyncIterable<SessionEvent>,
|
||||
): Promise<void> {
|
||||
await replaceMemoryStored(this.store, expectedRevision, m, events)
|
||||
}
|
||||
|
||||
async list(signal?: AbortSignal): Promise<SessionHeader[]> {
|
||||
signal?.throwIfAborted()
|
||||
return [...this.store.values()].map(e => structuredClone(e.meta))
|
||||
@@ -256,36 +199,23 @@ class ControlledBackend implements PersistenceBackend<never> {
|
||||
repairAttempts = 0
|
||||
beforeAppend?: (attempt: number) => Promise<void>
|
||||
beforeLoadStored?: (attempt: number, signal?: AbortSignal) => Promise<void>
|
||||
/** Optional physical suffix hook used by readFrom-specific tests. */
|
||||
seekHook?: (
|
||||
id: SessionId,
|
||||
fromSeq: number,
|
||||
signal?: AbortSignal,
|
||||
) => Promise<{ meta: SessionHeader; events: SessionEvent[] } | undefined>
|
||||
/** When set, the declared seek hook delegates here so readFrom exercises it; unset throws (tests set it first). */
|
||||
seekHook?: (id: SessionId, fromSeq: number, signal?: AbortSignal) => Promise<StoredSuffix | undefined>
|
||||
|
||||
async openStored(id: SessionId, signal?: AbortSignal): Promise<StoredSessionSource<never> | undefined> {
|
||||
loadStoredFrom(id: SessionId, fromSeq: number, signal?: AbortSignal): Promise<StoredSuffix | undefined> {
|
||||
if (this.seekHook === undefined) throw new Error('seekHook not configured for this test')
|
||||
return this.seekHook(id, fromSeq, signal)
|
||||
}
|
||||
|
||||
async loadStored(id: SessionId, signal?: AbortSignal): Promise<StoredPrefix<never> | undefined> {
|
||||
const attempt = ++this.loadAttempts
|
||||
await this.beforeLoadStored?.(attempt, signal)
|
||||
const entry = this.store.get(id)
|
||||
if (entry === undefined) return undefined
|
||||
const revision = memoryRevision(entry)
|
||||
return {
|
||||
meta: structuredClone(entry.meta),
|
||||
revision,
|
||||
readEvents: ({ fromSeq = 0 } = {}) => storedRead(async () => {
|
||||
signal?.throwIfAborted()
|
||||
const loaded = this.seekHook === undefined
|
||||
? { meta: entry.meta, events: entry.events.filter(event => event.seq >= fromSeq) }
|
||||
: await this.seekHook(id, fromSeq, signal)
|
||||
if (loaded === undefined) {
|
||||
throw new SessionPersistenceRevisionConflictError(`session "${id}" disappeared during read`)
|
||||
}
|
||||
const current = this.store.get(id)
|
||||
if (current === undefined || memoryRevision(current) !== revision) {
|
||||
throw new SessionPersistenceRevisionConflictError(`session "${id}" changed during read`)
|
||||
}
|
||||
return { events: structuredClone(loaded.events) }
|
||||
}),
|
||||
events: structuredClone(entry.events),
|
||||
revision: memoryRevision(entry),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -313,14 +243,6 @@ class ControlledBackend implements PersistenceBackend<never> {
|
||||
if (entry !== undefined) entry.events.push(...structuredClone(closers) as SessionEvent[])
|
||||
}
|
||||
|
||||
async replaceStored(
|
||||
expectedRevision: SessionPersistenceRevision,
|
||||
m: SessionHeader,
|
||||
events: AsyncIterable<SessionEvent>,
|
||||
): Promise<void> {
|
||||
await replaceMemoryStored(this.store, expectedRevision, m, events)
|
||||
}
|
||||
|
||||
async list(): Promise<SessionHeader[]> {
|
||||
return [...this.store.values()].map(entry => structuredClone(entry.meta))
|
||||
}
|
||||
@@ -651,11 +573,6 @@ describe('PersistenceCoordinator session preparations', () => {
|
||||
}, { inject: ['sessions'] }))
|
||||
|
||||
try {
|
||||
const immediatelyLive = Session.create(prepareId, oneTurnLog(), meta(prepareId))
|
||||
const immediateGet = vi.spyOn(ctx.sessions, 'get').mockReturnValue(immediatelyLive)
|
||||
await expect(coordinator.prepare(prepareId)).rejects.toThrow(/while it is live/)
|
||||
immediateGet.mockRestore()
|
||||
|
||||
const prepareLive = Session.create(prepareId, oneTurnLog(), meta(prepareId))
|
||||
const prepareGet = vi.spyOn(ctx.sessions, 'get')
|
||||
.mockReturnValueOnce(undefined)
|
||||
@@ -1556,7 +1473,7 @@ describe('PersistenceCoordinator observation cancellation', () => {
|
||||
}
|
||||
})
|
||||
|
||||
it('readFrom via the source reader: serves the suffix, reports absence, and relays reader failures by abort state', async () => {
|
||||
it('readFrom via the seek hook: serves the suffix, maps undefined to not-found, and relays hook failures by abort state', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const backend = new ControlledBackend()
|
||||
@@ -1577,7 +1494,7 @@ describe('PersistenceCoordinator observation cancellation', () => {
|
||||
}
|
||||
const suffix = await coordinator.readFrom(id, 3)
|
||||
expect(suffix.events).toEqual(log.slice(3))
|
||||
// Absence is established while opening the source, before an event read.
|
||||
// The hook's `undefined` is the backend contract's not-found result.
|
||||
await expect(coordinator.readFrom(SessionId('missing-seek'), 0)).rejects.toThrow('not found')
|
||||
|
||||
// A hook failure with no cancellation in play propagates as-is.
|
||||
@@ -1585,20 +1502,6 @@ describe('PersistenceCoordinator observation cancellation', () => {
|
||||
backend.seekHook = () => Promise.reject(hookFailure)
|
||||
await expect(coordinator.readFrom(id, 0)).rejects.toBe(hookFailure)
|
||||
|
||||
// A revision conflict is retryable because it names no stable source.
|
||||
let conflictAttempts = 0
|
||||
backend.seekHook = async (hookId, fromSeq) => {
|
||||
conflictAttempts += 1
|
||||
if (conflictAttempts === 1) {
|
||||
throw new SessionPersistenceRevisionConflictError('source changed during readFrom')
|
||||
}
|
||||
const entry = backend.store.get(hookId)
|
||||
if (entry === undefined) return undefined
|
||||
return { meta: entry.meta, events: entry.events.filter(event => event.seq >= fromSeq) }
|
||||
}
|
||||
await expect(coordinator.readFrom(id, 2)).resolves.toMatchObject({ events: log.slice(2) })
|
||||
expect(conflictAttempts).toBe(2)
|
||||
|
||||
// A hook failure after cancellation surfaces the caller's abort reason,
|
||||
// not the backend's internal teardown error. The abort fires only once
|
||||
// the hook is provably entered, so the failure exercises the catch (not
|
||||
@@ -1728,13 +1631,14 @@ describe('PersistenceCoordinator retirement', () => {
|
||||
}, { inject: ['sessions'] }))
|
||||
await ctx.sessions.flush(first)
|
||||
|
||||
// Occupy the per-id serialize chain with a gated source open:
|
||||
// Occupy the per-id serialize chain with a gated physical read:
|
||||
// inspect() correctly borrows the still-live Session without entering
|
||||
// the backend chain, while both retirements must queue behind readFrom().
|
||||
const readEntered = Promise.withResolvers<undefined>()
|
||||
backend.beforeLoadStored = async () => {
|
||||
backend.seekHook = async () => {
|
||||
readEntered.resolve(undefined)
|
||||
await readGate.promise
|
||||
return undefined
|
||||
}
|
||||
const parked = coordinator.readFrom(id, 0).catch((error: unknown) => error)
|
||||
await readEntered.promise
|
||||
@@ -1758,7 +1662,7 @@ describe('PersistenceCoordinator retirement', () => {
|
||||
// delete the successor's entry (exact-entry guard); the successor's own
|
||||
// forget() then clears the map.
|
||||
readGate.resolve(true)
|
||||
expect(await parked).toBeInstanceOf(Error) // the parked read (not found) is observed
|
||||
expect(await parked).toBeInstanceOf(Error) // the parked inspect (not found) is observed
|
||||
await firstRetirement
|
||||
await vi.waitFor(() => { expect(internals.retirements.has(id)).toBe(false) })
|
||||
} finally {
|
||||
@@ -2233,235 +2137,6 @@ describe('SessionPersistence service registration', () => {
|
||||
await Promise.allSettled([fiber.dispose()])
|
||||
})
|
||||
|
||||
it('rejects obsolete event variants passed directly to the persistence writer', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const backend = new ControlledBackend()
|
||||
let coordinator!: PersistenceCoordinator<never>
|
||||
const fiber = await ctx.plugin(Object.assign((inner: Context) => {
|
||||
coordinator = new PersistenceCoordinator(inner, backend)
|
||||
}, { inject: ['sessions'] }))
|
||||
const id = SessionId('legacy-direct-append')
|
||||
await coordinator.create(meta(id))
|
||||
|
||||
try {
|
||||
await expect(coordinator.append(id, [legacyHeaderDelta()]))
|
||||
.rejects.toThrow(/unsupported legacy request\/header-delta event/)
|
||||
await expect(coordinator.append(id, [legacyModeSet()]))
|
||||
.rejects.toThrow(/unsupported legacy mode\/set event/)
|
||||
await expect(coordinator.append(id, [legacyFallbackHeader()]))
|
||||
.rejects.toThrow(/unsupported legacy request\/header reason "fallback"/)
|
||||
expect(backend.store.has(id)).toBe(false)
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await ctx.fiber.dispose()
|
||||
}
|
||||
})
|
||||
|
||||
it('retries cold preparation when its physical source revision changes', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const backend = new ControlledBackend()
|
||||
const id = SessionId('prepare-source-conflict')
|
||||
backend.store.set(id, { meta: meta(id), events: oneTurnLog() })
|
||||
let attempts = 0
|
||||
backend.seekHook = async (hookId, fromSeq) => {
|
||||
attempts += 1
|
||||
if (attempts === 1) throw new SessionPersistenceRevisionConflictError('prepare source changed')
|
||||
const entry = backend.store.get(hookId)
|
||||
if (entry === undefined) return undefined
|
||||
return { meta: entry.meta, events: entry.events.filter(event => event.seq >= fromSeq) }
|
||||
}
|
||||
let coordinator!: PersistenceCoordinator<never>
|
||||
const fiber = await ctx.plugin(Object.assign((inner: Context) => {
|
||||
coordinator = new PersistenceCoordinator(inner, backend)
|
||||
}, { inject: ['sessions'] }))
|
||||
|
||||
try {
|
||||
await expect(coordinator.inspect(id)).resolves.toMatchObject({ events: oneTurnLog() })
|
||||
expect(attempts).toBe(2)
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await ctx.fiber.dispose()
|
||||
}
|
||||
})
|
||||
|
||||
it('retries live-prefix adoption when the physical source revision changes', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const backend = new ControlledBackend()
|
||||
const id = SessionId('hmr-source-conflict')
|
||||
const m = meta(id, '/work')
|
||||
backend.store.set(id, { meta: m, events: oneTurnLog() })
|
||||
const session = ctx.sessions.create(id, { seed: oneTurnLog(), meta: { cwd: '/work' } })
|
||||
let attempts = 0
|
||||
backend.seekHook = async (hookId, fromSeq) => {
|
||||
attempts += 1
|
||||
if (attempts === 1) throw new SessionPersistenceRevisionConflictError('live source changed')
|
||||
const entry = backend.store.get(hookId)
|
||||
if (entry === undefined) return undefined
|
||||
return { meta: entry.meta, events: entry.events.filter(event => event.seq >= fromSeq) }
|
||||
}
|
||||
const fiber = await ctx.plugin(Object.assign((inner: Context) => {
|
||||
new PersistenceCoordinator(inner, backend)
|
||||
}, { inject: ['sessions'] }))
|
||||
|
||||
try {
|
||||
await expect(ctx.sessions.flush(session)).resolves.toBe(true)
|
||||
expect(attempts).toBe(2)
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await ctx.fiber.dispose()
|
||||
}
|
||||
})
|
||||
|
||||
it('retries ownerless seed verification when the physical source revision changes', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const backend = new ControlledBackend()
|
||||
const id = SessionId('seed-source-conflict')
|
||||
const m = meta(id, '/work')
|
||||
backend.store.set(id, { meta: m, events: oneTurnLog() })
|
||||
let coordinator!: PersistenceCoordinator<never>
|
||||
const fiber = await ctx.plugin(Object.assign((inner: Context) => {
|
||||
coordinator = new PersistenceCoordinator(inner, backend)
|
||||
}, { inject: ['sessions'] }))
|
||||
|
||||
try {
|
||||
await coordinator.load(id)
|
||||
let attempts = 0
|
||||
backend.seekHook = async (hookId, fromSeq) => {
|
||||
attempts += 1
|
||||
if (attempts === 1) throw new SessionPersistenceRevisionConflictError('seed source changed')
|
||||
const entry = backend.store.get(hookId)
|
||||
if (entry === undefined) return undefined
|
||||
return { meta: entry.meta, events: entry.events.filter(event => event.seq >= fromSeq) }
|
||||
}
|
||||
const session = ctx.sessions.create(id, { seed: oneTurnLog(), meta: { cwd: '/work' } })
|
||||
|
||||
await expect(ctx.sessions.flush(session)).resolves.toBe(true)
|
||||
expect(attempts).toBe(2)
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await ctx.fiber.dispose()
|
||||
}
|
||||
})
|
||||
|
||||
it('streams an old-format prepared source into replacement and propagates non-conflict failures', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const backend = new ControlledBackend()
|
||||
const id = SessionId('prepared-format-replacement')
|
||||
const m = meta(id)
|
||||
backend.store.set(id, { meta: m, events: oneTurnLog() })
|
||||
let coordinator!: PersistenceCoordinator<never>
|
||||
const fiber = await ctx.plugin(Object.assign((inner: Context) => {
|
||||
coordinator = new PersistenceCoordinator(inner, backend)
|
||||
}, { inject: ['sessions'] }))
|
||||
const source = {
|
||||
inspection: Object.freeze({ meta: m, events: Object.freeze(oneTurnLog()) }),
|
||||
session: Session.create(id, oneTurnLog(), m),
|
||||
revision: memoryRevision(backend.store.get(id)!),
|
||||
sourceVersion: -1,
|
||||
sessionLength: oneTurnLog().length,
|
||||
tornMarker: undefined,
|
||||
closers: [],
|
||||
}
|
||||
const internals = coordinator as unknown as {
|
||||
commitPrepared(value: typeof source): Promise<unknown>
|
||||
}
|
||||
const replace = vi.spyOn(backend, 'replaceStored')
|
||||
|
||||
try {
|
||||
await expect(internals.commitPrepared(source)).resolves.toBeUndefined()
|
||||
expect(replace).toHaveBeenCalledOnce()
|
||||
expect(backend.store.get(id)?.events).toEqual(oneTurnLog())
|
||||
|
||||
const failure = new Error('replacement backend failed')
|
||||
replace.mockRejectedValueOnce(failure)
|
||||
source.revision = memoryRevision(backend.store.get(id)!)
|
||||
await expect(internals.commitPrepared(source)).rejects.toBe(failure)
|
||||
|
||||
replace.mockRejectedValueOnce(new SessionPersistenceRevisionConflictError('replacement raced'))
|
||||
await expect(internals.commitPrepared(source)).resolves.toBeUndefined()
|
||||
} finally {
|
||||
await fiber.dispose()
|
||||
await ctx.fiber.dispose()
|
||||
}
|
||||
})
|
||||
|
||||
it('routes live adoption of a decoded old format through the same replacement primitive', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const backend = new ControlledBackend()
|
||||
const id = SessionId('live-format-replacement')
|
||||
const m = meta(id, '/work')
|
||||
const log = oneTurnLog()
|
||||
backend.store.set(id, { meta: m, events: log })
|
||||
let coordinator!: PersistenceCoordinator<never>
|
||||
const fiber = await ctx.plugin(Object.assign((inner: Context) => {
|
||||
coordinator = new PersistenceCoordinator(inner, backend)
|
||||
}, { inject: ['sessions'] }))
|
||||
const revision = memoryRevision(backend.store.get(id)!)
|
||||
const stored: StoredSessionSource<never> = {
|
||||
meta: m,
|
||||
revision,
|
||||
readEvents: () => storedRead(async () => ({ events: log })),
|
||||
}
|
||||
const decoded = {
|
||||
meta: m,
|
||||
sourceVersion: -1,
|
||||
revision,
|
||||
events: (async function* (): AsyncIterable<SessionEvent> { yield* log })(),
|
||||
completed: Promise.resolve({}),
|
||||
}
|
||||
const decode = vi.spyOn(formatDecoder, 'decodeStoredSession').mockReturnValue(decoded)
|
||||
const replace = vi.spyOn(backend, 'replaceStored')
|
||||
const internals = coordinator as unknown as {
|
||||
adoptLivePrefix(
|
||||
session: Session,
|
||||
seed: readonly SessionEvent[],
|
||||
source: StoredSessionSource<never>,
|
||||
): Promise<boolean>
|
||||
}
|
||||
|
||||
try {
|
||||
const session = Session.create(id, log, m)
|
||||
await expect(internals.adoptLivePrefix(session, log, stored)).resolves.toBe(false)
|
||||
expect(replace).toHaveBeenCalledOnce()
|
||||
expect(backend.store.get(id)?.events).toEqual(log)
|
||||
} finally {
|
||||
decode.mockRestore()
|
||||
await fiber.dispose()
|
||||
await ctx.fiber.dispose()
|
||||
}
|
||||
})
|
||||
|
||||
it('propagates a non-conflict failure during ownerless seed verification', async () => {
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(SessionStore)
|
||||
const backend = new ControlledBackend()
|
||||
const id = SessionId('seed-source-failure')
|
||||
const m = meta(id, '/work')
|
||||
backend.store.set(id, { meta: m, events: oneTurnLog() })
|
||||
let coordinator!: PersistenceCoordinator<never>
|
||||
const fiber = await ctx.plugin(Object.assign((inner: Context) => {
|
||||
coordinator = new PersistenceCoordinator(inner, backend)
|
||||
}, { inject: ['sessions'] }))
|
||||
|
||||
try {
|
||||
await coordinator.load(id)
|
||||
const failure = new Error('seed reader failed')
|
||||
backend.seekHook = () => Promise.reject(failure)
|
||||
const session = ctx.sessions.create(id, { seed: oneTurnLog(), meta: { cwd: '/work' } })
|
||||
|
||||
await expect(ctx.sessions.flush(session)).rejects.toBe(failure)
|
||||
} finally {
|
||||
await Promise.allSettled([fiber.dispose()])
|
||||
await ctx.fiber.dispose()
|
||||
}
|
||||
})
|
||||
|
||||
it('rejects a stored legacy fallback header during load', async () => {
|
||||
const id = SessionId('legacy-fallback-load')
|
||||
const m = meta(id, '/legacy')
|
||||
|
||||
Generated
-3
@@ -4723,9 +4723,6 @@ importers:
|
||||
'@deepseek-ai/dsh-session':
|
||||
specifier: workspace:^
|
||||
version: link:../../core/session
|
||||
'@deepseek-ai/dsh-session-persistence':
|
||||
specifier: workspace:^
|
||||
version: link:../../session/session-persistence
|
||||
'@deepseek-ai/dsh-session-persistence-jsonl':
|
||||
specifier: workspace:^
|
||||
version: link:../../session/session-persistence-jsonl
|
||||
|
||||
@@ -556,11 +556,6 @@
|
||||
"symbol": "SessionLocation",
|
||||
"source": "packages/session/session-persistence/src/index.ts"
|
||||
},
|
||||
{
|
||||
"doc": "docs/subsystems/persistence.md",
|
||||
"symbol": "SessionFormatMigration",
|
||||
"source": "packages/session/session-persistence/src/format-decoder.ts"
|
||||
},
|
||||
{
|
||||
"doc": "docs/subsystems/persistence.md",
|
||||
"symbol": "SessionRawArtifact",
|
||||
|
||||
Reference in New Issue
Block a user