fix(snapshot): enforce fixture storage for every selected role

sessionFixtures already selects the highest generation independently for each parent or child role. Filtering its output to files[0] silently exempted every child from prompt and schema fixed points and prompt-before-request ordering. Merge those assertions into the per-role loop and expose the actual checker for focused negative controls without mocking Vitest registration.

The three selected-child controls reject missing system/message, raw prompt text, and raw tool schemas; each resolved incorrectly with the parent-only filter. A positive mixed-generation case proves retained predecessors remain unselected. Correct the two versionless record-suite child fixtures that the restored enforcement exposes, including the retired-child copy used by recording tests. No released historical generation or broad recording is rewritten.

Evidence: pnpm exec vitest run packages/test-support/session-snapshot/tests/storage-policy.spec.ts packages/test-support/session-snapshot/tests/suite.spec.ts --coverage --coverage.include=packages/test-support/session-snapshot/src/suite.ts: 143 pass, one intentional record-mode skip, suite.ts 100% statements/branches/functions/lines. Update and re-record the session-snapshot README EN/ZH pair.
This commit is contained in:
Tianyi Cui
2026-09-06 20:42:43 +08:00
parent 1adcd8203a
commit 7b182e95e1
7 changed files with 80 additions and 31 deletions
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/test-support/session-snapshot/README.md
README.md: 8deaa4c9854094b8a7a6517c41de52309c2e9611
README.zh.md: 1494be346c272fcafc07bc3df5277253a3b00362
README.md: 80f46431f87d6e8e41c48b7f1fa8cc01b6ce9ee1
README.zh.md: 8774c1963b0846e49db040121d788707bce51969
@@ -68,7 +68,7 @@ defineAcpSnapshotSuite({
})
```
Each recorded-session directory carries a closed `snapshot.yml` manifest plus canonical parent and contiguous child roles. Parent filenames are `session[.vN].jsonl`; children are `session.<ordinal>[.vN].jsonl`; v0 omits `.v0`, positive versions use lowercase `.vN`, and every filename agrees with its header. A role may retain older generations, but the harness selects the numerically highest one. An owning manifest may declare `sessionFormat.version` plus one or more closed `coverage` names to keep that historical generation as an explicit migration fixture; absence tracks the current writer. The manifest also names the scenario, shipped profile, composition/header class, recording source, and only the replay, platform, permission, environment, workspace, or input facts the completed Session cannot reconstruct. The adapter registers expected-output, Session-log, and optional `workspace.expected/` comparisons; guards reject orphan directories, missing roles, noncanonical names, absolute paths, malformed manifests, and platform-specific separators.
Each recorded-session directory carries a closed `snapshot.yml` manifest plus canonical parent and contiguous child roles. Parent filenames are `session[.vN].jsonl`; children are `session.<ordinal>[.vN].jsonl`; v0 omits `.v0`, positive versions use lowercase `.vN`, and every filename agrees with its header. A role may retain older generations, but the harness selects the numerically highest one. An owning manifest may declare `sessionFormat.version` plus one or more closed `coverage` names to keep that historical generation as an explicit migration fixture; absence tracks the current writer. The manifest also names the scenario, shipped profile, composition/header class, recording source, and only the replay, platform, permission, environment, workspace, or input facts the completed Session cannot reconstruct. Storage guards check the selected highest generation of every parent and child role, including prompt/schema scrubbing and prompt-before-request ordering. The adapter registers expected-output, Session-log, and optional `workspace.expected/` comparisons; guards reject orphan directories, missing roles, noncanonical names, absolute paths, malformed manifests, and platform-specific separators.
`normalizeSessionSnapshot` retains the complete Session header and event payloads but omits top-level `seq`/`time` envelopes from committed fixtures after normalizing paths and scrubbing system-prompt text and tool schemas; it also normalizes embedded stream clocks and historical packed-row `seq0`/`time0` envelopes. Replay synthesizes the top-level envelopes in memory, while runtime persistence continues to write complete logs. Multi-session comparison restores every selected persisted or projected fixture through the current build-static Session format catalog before identity redaction and normalization, so retained v0/v1/v2 replay input and fresh `session.v3.jsonl` writer output compare as one v3 logical Session without rewriting or renaming historical files. Expected and harvested logs use the same strict restoration path; source filenames cannot alter format validation. Normalization preserves unexpected request-header fields, including `system`, so regressions remain visible. Versionless protocol-adapter unit fixtures remain outside the released Session format corpus. Current v3 fixtures use one row per event; retained v0/v1 fixtures may use canonical packed rows. The [temporary repository migrator](../../../scripts/migrate-packed-session-fixtures.ts) (`pnpm run migrate:packed-session-fixtures`) rewrites older historical layouts, and its [removal proposal](../../../.agents/notes/proposed/process/2026-07-26-remove-packed-session-fixture-migrator.md) owns its deletion.
@@ -68,7 +68,7 @@ defineAcpSnapshotSuite({
})
```
每个已记录 Session 目录携带封闭的 `snapshot.yml` manifest,以及规范 parent 与连续 child 角色。parent 文件名是 `session[.vN].jsonl`child 是 `session.<ordinal>[.vN].jsonl`v0 省略 `.v0`,正版本使用小写 `.vN`,且每个文件名与其 header 一致。一个角色可以保留旧 generation,但 harness 会选择数值最高的一项。拥有 fixture 的 manifest 可以声明 `sessionFormat.version` 与一个或多个封闭 `coverage` 名称,把该历史 generation 保留为显式迁移 fixture;省略此字段时跟随当前 writer。manifest 还会指名场景、随附 profile、组合/header 类别、录制来源,以及已完成 Session 无法重建的 replay、平台、权限、环境、workspace 或输入事实。适配器注册预期输出、Session 日志与可选 `workspace.expected/` 比较;保护会拒绝遗留目录、缺失角色、非规范名称、绝对路径、malformed manifest 与平台专用分隔符。
每个已记录 Session 目录携带封闭的 `snapshot.yml` manifest,以及规范 parent 与连续 child 角色。parent 文件名是 `session[.vN].jsonl`child 是 `session.<ordinal>[.vN].jsonl`v0 省略 `.v0`,正版本使用小写 `.vN`,且每个文件名与其 header 一致。一个角色可以保留旧 generation,但 harness 会选择数值最高的一项。拥有 fixture 的 manifest 可以声明 `sessionFormat.version` 与一个或多个封闭 `coverage` 名称,把该历史 generation 保留为显式迁移 fixture;省略此字段时跟随当前 writer。manifest 还会指名场景、随附 profile、组合/header 类别、录制来源,以及已完成 Session 无法重建的 replay、平台、权限、环境、workspace 或输入事实。存储保护检查每个 parent 与 child 角色选定的最高 generation,包括提示词/schema 擦除以及提示词先于请求的顺序。适配器注册预期输出、Session 日志与可选 `workspace.expected/` 比较;保护会拒绝遗留目录、缺失角色、非规范名称、绝对路径、malformed manifest 与平台专用分隔符。
`normalizeSessionSnapshot` 在规范化路径并擦除系统提示文本与工具 schema 后,会保留完整 Session header 与事件 payload,但从已提交 fixture 中省略顶层 `seq`/`time` envelope;它还会规范化嵌入式 stream clock 与历史 packed-row 的 `seq0`/`time0` envelope。Replay 只在内存中合成顶层 envelope,而运行时持久化仍写入完整日志。多 Session 比较会先通过当前构建期静态 Session 格式目录恢复每个选定的持久化或投影 fixture,再进行身份脱敏与规范化,因此保留的 v0/v1/v2 replay 输入与新生成的 `session.v3.jsonl` writer 输出会作为同一个 v3 logical Session 比较,且不会重写或重命名历史文件。预期日志与收集日志使用同一条严格恢复路径;来源文件名不能改变格式校验。归一化保留意外的 request-header 字段(包括 `system`),使回归保持可见。无版本的协议适配器单元测试 fixture 不属于已发布 Session 格式语料。当前 v3 fixture 每个事件占一行;保留的 v0/v1 fixture 可以使用规范 packed row。[临时仓库迁移器](../../../scripts/migrate-packed-session-fixtures.ts)`pnpm run migrate:packed-session-fixtures`)会改写更旧的历史布局,由其[移除提案](../../../.agents/notes/proposed/process/2026-07-26-remove-packed-session-fixture-migrator.zh.md)负责删除该迁移器。
@@ -1145,6 +1145,31 @@ export function stabilizeRefreshLog(
return records.map(record => JSON.stringify(record)).join('\n') + '\n'
}
/**
* Check the selected generation of every parent and child fixture against storage policy.
* @param dir Scenario directory containing canonical Session fixtures.
* @param scenarioName Scenario name used in failure diagnostics.
* @returns Resolves when all selected fixtures satisfy the storage checks.
*/
export async function assertSessionFixtureStorage(dir: string, scenarioName: string): Promise<void> {
const files = await sessionFixtures(dir)
for (const file of files) {
const fixture = await readFile(join(dir, file), 'utf8')
expect(unknownToolCallIds(fixture), `${scenarioName}/${file} contains UNKNOWN_TOOL`)
.toEqual([])
expect(fixture, `${scenarioName}/${file} carries a non-canonical macOS cwd token`)
.not.toContain('/private{{cwd}}')
expect(scrubSystemPrompts(fixture), `${scenarioName}/${file} carries an unscrubbed system prompt`)
.toEqual(fixture)
expect(scrubToolSchemas(fixture), `${scenarioName}/${file} carries unscrubbed tool schemas`)
.toEqual(fixture)
expect(systemPromptPrecedesRequests(fixture), `${scenarioName}/${file} has a request/header with no preceding system/message`)
.toBe(true)
}
const fixtures = await Promise.all(files.map(file => readFile(join(dir, file), 'utf8')))
expect(redactSessionSnapshotIds(fixtures), `${scenarioName}: identity redaction fixed point`).toEqual(fixtures)
}
/**
* Register the suite: one test per scenario (the expected-output and log comparisons and
* the header and prompt uniformity guard) plus the fixture guard block (no orphan
@@ -1684,31 +1709,7 @@ export function defineAcpSnapshotSuite(options: SnapshotSuiteOptions): void {
// platform realpath prefix. Fixed-point checks make these storage rules
// fail loud.
for (const scenario of scenarios) {
const dir = join(snapshotsDir, scenario.name)
const files = await sessionFixtures(dir)
// A retained historical generation predates the `system/message` node
// vocabulary and cannot satisfy current-writer storage rules; only
// the selected current generation (highest fixture version) is a
// fixed point of the current prompt/schema scrub shape.
const currentFiles = files.filter(file => file === files[0])
for (const file of files) {
const fixture = await readFile(join(dir, file), 'utf8')
expect(unknownToolCallIds(fixture), `${scenario.name}/${file} contains UNKNOWN_TOOL`)
.toEqual([])
expect(fixture, `${scenario.name}/${file} carries a non-canonical macOS cwd token`)
.not.toContain('/private{{cwd}}')
}
for (const file of currentFiles) {
const fixture = await readFile(join(dir, file), 'utf8')
expect(scrubSystemPrompts(fixture), `${scenario.name}/${file} carries an unscrubbed system prompt`)
.toEqual(fixture)
expect(scrubToolSchemas(fixture), `${scenario.name}/${file} carries unscrubbed tool schemas`)
.toEqual(fixture)
expect(systemPromptPrecedesRequests(fixture), `${scenario.name}/${file} has a request/header with no preceding system/message`)
.toBe(true)
}
const fixtures = await Promise.all(files.map(file => readFile(join(dir, file), 'utf8')))
expect(redactSessionSnapshotIds(fixtures), `${scenario.name}: identity redaction fixed point`).toEqual(fixtures)
await assertSessionFixtureStorage(join(snapshotsDir, scenario.name), scenario.name)
}
})
})
@@ -1,3 +1,4 @@
{"type":"session","id":"{{session:2}}","createdAt":800,"cwd":"/var/folders/2g/b32ct0qn1d728l_v6tdkjytr0000gn/T/acp-snap-cwd-KBQJbW","parentSession":"{{session:1}}","delegationDepth":1}
{"type":"request/header","data":{"header":{"config":{"provider":"fake","model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}}
{"type":"system/message","data":{"turn":1,"step":1,"message":{"role":"system","content":[{"type":"text","text":"{{system}}"}],"source":{"kind":"plugin","plugin":"@deepseek-ai/dsh-system-prompt"}}},"surfaceOp":"append"}
{"type":"request/header","data":{"header":{"config":{"provider":"fake","model":"fake"},"tools":"{{tools}}"},"reason":"initial"}}
{"type":"user/message","data":{"role":"user","content":[{"type":"text","text":"same inherited message"}],"source":{"kind":"user"},"id":"{{message:1}}"},"surfaceOp":"append"}
@@ -1,2 +1,3 @@
{"type":"session","id":"{{session:2}}","createdAt":800,"cwd":"/var/folders/2g/b32ct0qn1d728l_v6tdkjytr0000gn/T/acp-snap-cwd-KBQJbW","parentSession":"f6fa7fcf-dd9c-4b39-8815-b25ddcebfd88"}
{"type":"request/header","data":{"header":{"config":{"provider":"fake","model":"fake"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}}
{"type":"system/message","data":{"turn":1,"step":1,"message":{"role":"system","content":[{"type":"text","text":"{{system}}"}],"source":{"kind":"plugin","plugin":"@deepseek-ai/dsh-system-prompt"}}},"surfaceOp":"append"}
{"type":"request/header","data":{"header":{"config":{"provider":"fake","model":"fake"},"tools":"{{tools}}"},"reason":"initial"}}
@@ -0,0 +1,46 @@
import { mkdtempSync, mkdirSync, rmSync, writeFileSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { afterEach, expect, test } from 'vitest'
import { assertSessionFixtureStorage } from '../src/suite.ts'
const roots: string[] = []
afterEach(() => {
for (const root of roots.splice(0)) rmSync(root, { recursive: true, force: true })
})
const system = { type: 'system/message', data: { message: {
role: 'system', content: [{ type: 'text', text: '{{system}}' }],
} } }
const request = { type: 'request/header', data: { header: { tools: '{{tools}}' } } }
function fixture(id: number, version: number, events: unknown[]): string {
return [{ type: 'session', version, id: `{{session:${id}}}`, createdAt: 0, delegationDepth: id - 1 }, ...events]
.map(record => JSON.stringify(record)).join('\n') + '\n'
}
function registerGuard(childEvents: unknown[]): () => Promise<void> {
const root = mkdtempSync(join(tmpdir(), 'snapshot-storage-policy-'))
roots.push(root)
const dir = join(root, 'pin')
mkdirSync(dir)
writeFileSync(join(dir, 'session.v2.jsonl'), fixture(1, 2, [system, request]))
// Each role selects its highest generation; the older child deliberately violates the policy.
writeFileSync(join(dir, 'session.1.v1.jsonl'), fixture(2, 1, [request]))
writeFileSync(join(dir, 'session.1.v2.jsonl'), fixture(2, 2, childEvents))
return () => assertSessionFixtureStorage(dir, 'pin')
}
test('checks only the highest generation of every parent and child role', async () => {
await expect(registerGuard([system, request])()).resolves.toBeUndefined()
})
test.each([
['missing prompt', [request], 'has a request/header with no preceding system/message'],
['unscrubbed prompt', [{ ...system, data: { message: {
role: 'system', content: [{ type: 'text', text: 'leaked child prompt' }],
} } }, request], 'carries an unscrubbed system prompt'],
['unscrubbed tools', [system, { type: 'request/header', data: { header: { tools: [] } } }], 'carries unscrubbed tool schemas'],
] as const)('rejects a selected child with %s', async (_name, events, message) => {
await expect(registerGuard([...events])()).rejects.toThrow(`pin/session.1.v2.jsonl ${message}`)
})