mirror of
https://github.com/deepseek-ai/deepseek-harness.git
synced 2026-08-29 04:26:38 +00:00
Merge remote-tracking branch 'origin/master' into dshw/pr-deepseek-harness-deepseek-harness-2781
This commit is contained in:
_Kerman
+6
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-08-24-route-priced-image-request-pressure.md
|
||||
2026-08-24-route-priced-image-request-pressure.md: 45a29211730474369607ed5fb933f380d640bf27
|
||||
2026-08-24-route-priced-image-request-pressure.zh.md: cf005a3ee343edf5774d554a4ec78cb876703774
|
||||
@@ -0,0 +1,39 @@
|
||||
# Agent Note: Route-priced image request pressure
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-08-24-route-priced-image-request-pressure.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
The token meter priced an `ImageBlock` as the structural JSON of its durable reference — roughly forty tokens — while a DeepSeek request image costs up to 384 visual tokens, so an image-dense session could carry hundreds of thousands of unbilled estimated tokens. Provider usage anchors only completed requests: the first multimodal request, images added after the anchor, and offload-set changes all fed automatic compaction a pressure figure that was wrong by orders of magnitude, triggering it far too late (context overflow) or, after a route change, too early. The [version-one simplification](../simplification/2026-07-29-simplify-web-image-input-v1.md) had deliberately rejected a provider-neutral tile formula and deferred visual pricing until a provider-aware estimator had a concrete consumer.
|
||||
|
||||
## Decision
|
||||
|
||||
Compaction pressure is now priced by the routed model's own request projection. `LlmAdapter.imageRequestPricing(provider, model)` is an optional synchronous hook returning an `LlmImageRequestPricing` for one exact route, resolved through `ctx.llm.imageRequestPricing()`; the base adapter declares none and unknown providers degrade to `undefined`, never throw. Each ordered image occurrence resolves to an `LlmImageRequestPrice`: the provider's visual tokens for a retained image plus the model-visible text the wire actually carries (request-preview handle, offload placeholder, or text-only substitution), with the text left to the caller's own estimator so no provider fixes a text tokenization.
|
||||
|
||||
The DeepSeek adapter implements the hook from its connection snapshot (`request-pricing.ts`): uncatalogued and text-only models price every occurrence as its `textOnlyImageText` substitution; image-capable models reproduce the serializer's first-stage oldest-first offload through the shared `offloadedImagePrefixCount()`, build handle and placeholder text through the same execution-world access resolution the serializer uses, and price retained images at their `requestImageDimensions` projection with `deepSeekImageTokens()` — a verbatim port of the provider's published v4 vision calculator (14px patches, 3:1 downsampling, 384-token cap, minimum-pixel scale-up, 8:1 width clamp), priced at the worst-case pad-to-4 alignment. The pure geometry moved from `attachment-local` to `dsh-attachment` so provider and pricing share it.
|
||||
|
||||
The token meter's surface fold stores route-neutral facts per node — the fixed-heuristic price, the image-free price, and the durable image occurrences — and `measure()` prices the surface under the effective envelope's route on every call. The anchor holds its raw materials (surface snapshot, provider-output price, usage) instead of a precomputed baseline, so a matching header reprices both the anchor and the current surface under one route and the signed delta compares like with like; the usage-versus-estimated choice happens per measurement against the route-priced anchor. Public `TokenSurfaceNode` carries both `tokens` (route-priced; read by trigger, retention, range selection, and the summary-shrink comparison) and `heuristicTokens` (fixed; the shadow-price protocol's unit, so `compaction/summary` and `compaction/prune` stay consistent with the O(1) projection fold's own appends). The `contextPressure` and `contextBreakdown` projections deliberately stay on the fixed heuristic.
|
||||
|
||||
The test-support replay adapter declares a flat per-model `imageRequestTokens` so keyless assembled scenarios exercise the seam; the `image-compaction` ACP snapshot proves six inline images push the second turn's pre-step measurement over an automatic threshold that the text-only heuristic stays under, and that the triggered compaction shadows the image message at its heuristic price.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**Price images inside the provider-neutral estimator.** Rejected by the [version-one note](../simplification/2026-07-29-simplify-web-image-input-v1.md) and still wrong: visual pricing varies by provider, model, detail mode, and preprocessing, and a hard-coded figure would look authoritative on routes it does not describe. The hook keeps every constant in the adapter that owns the route.
|
||||
|
||||
**Correct pressure only from provider usage.** Usage cannot price the first multimodal request, an image added after the anchor, or a changed offload set — exactly the cases that made compaction fire too late. Usage stays the anchor for completed requests; the route projection prices the increment.
|
||||
|
||||
**Reproduce the full serialization pipeline, including prepared-version bytes and the base64 fallback budgets.** The second-stage offload depends on encoded request bytes that only exist after asynchronous image preparation. The pricing reproduces the deterministic first stage from durable byte lengths; a fallback request can only offload more and cost less, so the estimate stays conservative without I/O in a synchronous hook.
|
||||
|
||||
**Route-price the shadow-price protocol too.** Logged `shadowedTokenCount` feeds the O(1) projection fold, whose appends are priced by the fixed heuristic; pricing replacements by route would make the persisted running total drift. Keeping the protocol on `heuristicTokens` preserves the fold's by-construction agreement.
|
||||
|
||||
**Fold route pricing into the meter's replay state.** A fold keyed to one route would have to replay on every route change and could not answer a `requestHeader` override for a different model. Storing route-neutral node facts and pricing at `measure()` keeps replay single-pass and measurement O(surface), which the contract already promises.
|
||||
|
||||
## Consequences
|
||||
|
||||
Automatic compaction now triggers on the pressure the routed model's next request will actually carry: image-dense DeepSeek sessions compact before overflow instead of after it, text-only routes charge substitution text instead of phantom visual tokens, and offloaded images cost their placeholder. The worst-case alignment pad overprices an image by at most three tokens, and the unreproduced base64-fallback budgets can only overprice — both errors are conservative; an execution-world access path that changes between pricing and the request shifts a descriptor's text price by its own length, and provider usage remains the authoritative anchor once a request completes. The published v4 calculator constants live in `llm-deepseek` alone; if the provider revises its vision projection, that one module and its pinned vectors are the change site. Measurement cost gains one pricing resolution and one image-occurrence walk per call, still O(surface).
|
||||
|
||||
## Testing
|
||||
|
||||
Formula vectors in `image-tokens.spec.ts` pin the published calculator's outputs, including the aspect-clamp, scale-up floor, one-column solver, odd-grid trim, and second-pass convergence cases, cross-checked against the reference implementation over a dimension grid and 50,000-point fuzz during development. `request-pricing.spec.ts` covers text-only substitution, the low-detail preset, and count- and byte-driven offload boundaries. Token-meter specs cover the first multimodal estimate, post-anchor image deltas over usage, text-only repricing under a header override, pricer-less neutrality, occurrence-count mismatch, and nested tool-result images. Compaction specs prove trigger, retention, range selection, and the summary-shrink comparison read the route price while the logged shadow price stays heuristic, including a summary that only route-priced shrink accepts. Access-resolution threading is covered at the pricing function and the adapter override. The keyless `image-compaction` ACP snapshot exercises the assembled application end to end.
|
||||
+39
@@ -0,0 +1,39 @@
|
||||
# Agent Note: 按路由定价的图片请求压力
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-08-24-route-priced-image-request-pressure.md) | 中文
|
||||
|
||||
## Problem
|
||||
|
||||
token 计量服务把 `ImageBlock` 按其持久引用的 JSON 结构计价，约四十个 token，而一张 DeepSeek 请求图片最多消耗 384 个视觉 token，因此图片密集的会话可能携带数十万个未计入估算的 token。provider usage 只锚定已完成的请求：首次多模态请求、锚点之后新增的图片、offload 集合的变化，都会让自动 compaction 拿到数量级错误的压力值，触发得过晚（上下文溢出）或在路由切换后过早。[版本一简化](../simplification/2026-07-29-simplify-web-image-input-v1.zh.md)曾有意否决 provider-neutral 的 tile 公式，把视觉定价推迟到 provider-aware 估算器出现具体消费方之时。
|
||||
|
||||
## Decision
|
||||
|
||||
compaction 压力现在按路由模型自身的请求投影定价。`LlmAdapter.imageRequestPricing(provider, model)` 是可选的同步钩子，为一条确切路由返回 `LlmImageRequestPricing`，经 `ctx.llm.imageRequestPricing()` 解析；基类不声明定价，未注册的 provider 降级为 `undefined` 而绝不抛出。每个按序的图片出现处解析为一个 `LlmImageRequestPrice`：保留图片的提供方视觉 token，加上线上实际携带的模型可见文本（请求预览句柄、offload 占位文本或纯文本替换），文本交由调用方自己的估算器计价，避免任何提供方固定一种文本 token 化。
|
||||
|
||||
DeepSeek 适配器基于连接快照实现该钩子（`request-pricing.ts`）：未编目和纯文本模型把每个出现处按其 `textOnlyImageText` 替换计价；支持图片的模型通过共享的 `offloadedImagePrefixCount()` 复现序列化器第一阶段的最旧优先 offload，经序列化器同一套执行环境访问解析构建句柄与占位文本，并按 `requestImageDimensions` 投影尺寸用 `deepSeekImageTokens()` 为保留图片计价，后者是提供方公布的 v4 视觉计算器的逐句移植（14px patch、3:1 降采样、384 token 上限、最小像素放大、8:1 宽度钳制），按最坏的 pad-to-4 对齐计价。纯几何函数从 `attachment-local` 上移到 `dsh-attachment`，供提供方与定价共享。
|
||||
|
||||
token 计量服务的表层 fold 为每个节点存储与路由无关的事实：固定启发式价格、去图价格与持久图片出现处；`measure()` 在每次调用时按生效 envelope 的路由为表层定价。锚点保存原始材料（表层快照、提供方输出价格、usage）而非预先计算的基线，因此匹配的标头会把锚点与当前表层放在同一路由下重新定价，带符号 delta 的比较口径一致；usage 与估算的选择在每次计量时针对路由定价锚点做出。公开的 `TokenSurfaceNode` 同时携带 `tokens`（路由定价；触发、保留、选段与摘要收缩比较读取它）和 `heuristicTokens`（固定值；影子价协议的计量单位，使 `compaction/summary` 与 `compaction/prune` 与 O(1) 投影 fold 自身的追加保持一致）。`contextPressure` 与 `contextBreakdown` 投影有意保持固定启发式规则。
|
||||
|
||||
test-support 的回放适配器按模型声明固定的 `imageRequestTokens`，让 keyless 装配场景走通这条 seam；`image-compaction` ACP 快照证明六张内联图片把第二轮 pre-step 计量推过自动阈值，而纯文本启发式保持在阈值之下，且被触发的 compaction 按启发式价格遮蔽了图片消息。
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**在 provider-neutral 估算器里为图片定价。** 已被[版本一 note](../simplification/2026-07-29-simplify-web-image-input-v1.zh.md)否决且依然错误：视觉定价随提供方、模型、细节档位与预处理而不同，写死的数字在它不描述的路由上会显得权威却错误。钩子把每个常量留在拥有该路由的适配器里。
|
||||
|
||||
**只用 provider usage 校正压力。** usage 无法为首次多模态请求、锚点后新增图片或变化的 offload 集合定价，而这些正是让 compaction 触发过晚的情形。usage 仍是已完成请求的锚点；增量由路由投影定价。
|
||||
|
||||
**复现完整序列化管线，包括请求版本字节与 base64 回退预算。** 第二阶段 offload 依赖异步图片准备之后才存在的编码字节。定价复现由持久字节长度决定的确定性第一阶段；回退请求只会 offload 更多、花费更少，因此估算在同步无 I/O 的钩子里保持保守。
|
||||
|
||||
**让影子价协议也按路由定价。** 记录的 `shadowedTokenCount` 供 O(1) 投影 fold 消费，而该 fold 的追加按固定启发式计价；替换若按路由定价会让持久化的累计值漂移。协议保持在 `heuristicTokens` 上，维持 fold 的构造性一致。
|
||||
|
||||
**把路由定价并入计量服务的回放状态。** 绑定单一路由的 fold 在路由每次变化时都得重放，也无法回答指向另一模型的 `requestHeader` 覆盖。存储与路由无关的节点事实并在 `measure()` 时定价，保持单遍回放与契约已承诺的 O(surface) 计量。
|
||||
|
||||
## Consequences
|
||||
|
||||
自动 compaction 现在按路由模型下一次请求实际携带的压力触发：图片密集的 DeepSeek 会话在溢出之前而非之后压缩，纯文本路由收取替换文本而非幻影视觉 token，被 offload 的图片按占位文本计费。最坏对齐 pad 对单图最多多计三个 token，未复现的 base64 回退预算只会多计——两种误差都偏保守；执行环境访问路径若在定价与请求之间变化，只会按其自身长度改变描述文本的价格，请求完成后 provider usage 仍是权威锚点。公布的 v4 计算器常量只存在于 `llm-deepseek`；提供方若修订其视觉投影，改动点就是这一个模块与其钉死的向量。每次计量多一次定价解析与一次图片出现处遍历，仍为 O(surface)。
|
||||
|
||||
## Testing
|
||||
|
||||
`image-tokens.spec.ts` 的公式向量钉死公布计算器的输出，覆盖宽高比钳制、放大下限、单列求解、奇数网格裁剪与第二遍收敛的用例，开发期间与参考实现在尺寸网格及五万点模糊测试上对拍。`request-pricing.spec.ts` 覆盖纯文本替换、低细节预设以及数量与字节驱动的 offload 边界。token-meter 测试覆盖首次多模态估算、usage 之上的锚后图片 delta、标头覆盖下的纯文本重定价、无定价器时的中性行为、出现处数量不匹配与嵌套工具结果图片。compaction 测试证明触发、保留、选段与摘要收缩比较读取路由价格而记录的影子价保持启发式，包括一个只有路由定价收缩才接受的摘要。访问解析的传递在定价函数与适配器覆写两处都有覆盖。keyless 的 `image-compaction` ACP 快照端到端验证装配后的应用。
|
||||
+2
-2
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/simplification/2026-07-29-simplify-web-image-input-v1.md
|
||||
2026-07-29-simplify-web-image-input-v1.md: e7847dc2ae18f48146cb2b686dcda64beebf2350
|
||||
2026-07-29-simplify-web-image-input-v1.zh.md: 47f6bb5face5b896b5184f7198eef7d99ee25afe
|
||||
2026-07-29-simplify-web-image-input-v1.md: f13abfda3be890f80a8ff852acbc68118b91941c
|
||||
2026-07-29-simplify-web-image-input-v1.zh.md: c92974efe5f84da5b91c09ea4acf820e7853ca7d
|
||||
|
||||
@@ -32,6 +32,6 @@ The attachment seam exposes its limits plus storage-free `validateImage`, `saveI
|
||||
|
||||
## Consequences
|
||||
|
||||
The feature retains the two batch limits and one storage-free validation method required by multi-image prompts, while removing unrelated public fields, lifecycle operations, policy snapshots, and route-assembly branches. Provider/model selection remains composition or profile configuration. Pre-request token pressure may undercount visual input until a provider-aware estimator is designed, while reported usage remains exact.
|
||||
The feature retains the two batch limits and one storage-free validation method required by multi-image prompts, while removing unrelated public fields, lifecycle operations, policy snapshots, and route-assembly branches. Provider/model selection remains composition or profile configuration. Pre-request token pressure keeps the structural heuristic only on routes without declared image pricing; the [route-priced estimator](../feature/2026-08-24-route-priced-image-request-pressure.md) supplies the provider-aware figure, and reported usage remains exact.
|
||||
|
||||
Reintroducing any removed surface requires a concrete consumer and its failure, lifecycle, replay, and testing contract rather than compatibility with this pre-release shape.
|
||||
|
||||
+1
-1
@@ -32,6 +32,6 @@ Status: implemented
|
||||
|
||||
## 后果
|
||||
|
||||
该功能保留了多图片提示词所需的两个批次上限和一个不触碰存储的校验方法，同时移除了无关的公开字段、生命周期操作、策略快照和路由组装分支。提供方／模型选择仍属于组合或 profile 配置。在设计出提供方感知型估算器之前，请求前的 token 压力计算可能少计视觉输入，而上报的用量仍保持精确。
|
||||
该功能保留了多图片提示词所需的两个批次上限和一个不触碰存储的校验方法，同时移除了无关的公开字段、生命周期操作、策略快照和路由组装分支。提供方／模型选择仍属于组合或 profile 配置。请求前的 token 压力只在未声明图片定价的路由上保留结构启发式；[按路由定价的估算器](../feature/2026-08-24-route-priced-image-request-pressure.zh.md)提供提供方感知的数值，上报的用量仍保持精确。
|
||||
|
||||
重新引入任何已移除表面时，都必须有具体消费方，并为其定义失败、生命周期、回放和测试契约，而不是为了兼容这一预发布形态。
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write docs/config-catalog.md
|
||||
config-catalog.md: c492cad74b53796d8e816ad410a0f8ede4b281e3
|
||||
config-catalog.zh.md: 3096d562624d99757be2dd3f4cf7e6155b2607c6
|
||||
config-catalog.md: a2cf48a332698d79ba20f975e682b10a68340d34
|
||||
config-catalog.zh.md: c2563cf978fe13d3ca2911c52ca5f980936db7ac
|
||||
|
||||
+11
-2
@@ -964,7 +964,7 @@ export interface DeepSeekCatalogModel {
|
||||
|
||||
Depends on: [`ModelModality`](../packages/llm/llm/src/index.ts) · [`RetryPolicyConfig`](../packages/llm/llm/src/index.ts)
|
||||
|
||||
Source: [`packages/llm/llm-deepseek/src/index.ts:117`](../packages/llm/llm-deepseek/src/index.ts)
|
||||
Source: [`packages/llm/llm-deepseek/src/index.ts:124`](../packages/llm/llm-deepseek/src/index.ts)
|
||||
|
||||
<a id="deepseek-aidsh-llm-pi-ai"></a>
|
||||
|
||||
@@ -1286,6 +1286,15 @@ export interface ReplayModelConfig {
|
||||
* omit one, so replay reconstructs the request header a live catalog produced.
|
||||
*/
|
||||
defaultMaxTokens?: number
|
||||
/**
|
||||
* Optional flat visual-token price the replay route declares for every
|
||||
* retained request image, so keyless scenarios exercise route-priced
|
||||
* request pressure; each occurrence is priced at this value plus its
|
||||
* request-preview handle text. Requires {@link inputModalities} to include
|
||||
* `image` — a text-only route never sends visual tokens. Absent declares
|
||||
* no image pricing.
|
||||
*/
|
||||
imageRequestTokens?: number
|
||||
/** Optional reasoning-effort ids the replay route accepts, in display order. */
|
||||
reasoningEfforts?: string[]
|
||||
/**
|
||||
@@ -1298,7 +1307,7 @@ export interface ReplayModelConfig {
|
||||
|
||||
Depends on: [`ModelModality`](../packages/llm/llm/src/index.ts) · [`RetryPolicyConfig`](../packages/llm/llm/src/index.ts)
|
||||
|
||||
Source: [`packages/test-support/llm-replay/src/index.ts:892`](../packages/test-support/llm-replay/src/index.ts)
|
||||
Source: [`packages/test-support/llm-replay/src/index.ts:914`](../packages/test-support/llm-replay/src/index.ts)
|
||||
|
||||
<a id="deepseek-aidsh-llm-retry"></a>
|
||||
|
||||
|
||||
@@ -1288,6 +1288,15 @@ export interface ReplayModelConfig {
|
||||
* omit one, so replay reconstructs the request header a live catalog produced.
|
||||
*/
|
||||
defaultMaxTokens?: number
|
||||
/**
|
||||
* Optional flat visual-token price the replay route declares for every
|
||||
* retained request image, so keyless scenarios exercise route-priced
|
||||
* request pressure; each occurrence is priced at this value plus its
|
||||
* request-preview handle text. Requires {@link inputModalities} to include
|
||||
* `image` — a text-only route never sends visual tokens. Absent declares
|
||||
* no image pricing.
|
||||
*/
|
||||
imageRequestTokens?: number
|
||||
/** Optional reasoning-effort ids the replay route accepts, in display order. */
|
||||
reasoningEfforts?: string[]
|
||||
/**
|
||||
@@ -1300,7 +1309,7 @@ export interface ReplayModelConfig {
|
||||
|
||||
依赖：[`ModelModality`](../packages/llm/llm/src/index.ts) · [`RetryPolicyConfig`](../packages/llm/llm/src/index.ts)
|
||||
|
||||
来源：[`packages/test-support/llm-replay/src/index.ts:892`](../packages/test-support/llm-replay/src/index.ts)
|
||||
来源：[`packages/test-support/llm-replay/src/index.ts:914`](../packages/test-support/llm-replay/src/index.ts)
|
||||
|
||||
<a id="deepseek-aidsh-llm-retry"></a>
|
||||
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write docs/event-producer-consumer.md
|
||||
event-producer-consumer.md: e744054d21eb9f07a7d7f6ba2a1f17a758a9689e
|
||||
event-producer-consumer.zh.md: 9f91e52ab1515fc66d02e15e339b7681c2a8a767
|
||||
event-producer-consumer.md: 7182dd4bfba60922ecf419eaf39fa6518fdea194
|
||||
event-producer-consumer.zh.md: 2ada50a065014802a10a9ea2cc153f3a076f52bf
|
||||
|
||||
@@ -43,7 +43,7 @@ This matrix shows which packages dispatch each harness-owned event and which pac
|
||||
| `fs/write-intent` | `waterfall` | [`packages/fs/fs/src/index.ts:58`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`waterfall`), [`tool-str-replace-editor`](../packages/fs/tool-str-replace-editor) (`waterfall`) | [`fs-observation-policy`](../packages/fs/fs-observation-policy) |
|
||||
| `goal/changed` | `emit` | [`packages/goal/goal/src/domain.ts:114`](../packages/goal/goal/src/domain.ts) | [`goal`](../packages/goal/goal) (`emit`) | [`goal-round-driver`](../packages/goal/goal-round-driver) |
|
||||
| `llm/adapters-updated` | `emit` | [`packages/llm/llm/src/types.ts:23`](../packages/llm/llm/src/types.ts) | [`llm`](../packages/llm/llm) (`events.dispatch`) | [`acp`](../packages/acp/acp), [`llm`](../packages/llm/llm), `remotes` |
|
||||
| `llm/stream` | `waterfall` | [`packages/llm/llm/src/index.ts:65`](../packages/llm/llm/src/index.ts) | [`llm`](../packages/llm/llm) (`waterfall`) | [`agent-loop`](../packages/core/agent-loop), [`llm`](../packages/llm/llm), [`llm-replay`](../packages/test-support/llm-replay), [`session-checkpoint-policy`](../packages/session/session-checkpoint-policy), [`session-title`](../packages/session/session-title) |
|
||||
| `llm/stream` | `waterfall` | [`packages/llm/llm/src/index.ts:66`](../packages/llm/llm/src/index.ts) | [`llm`](../packages/llm/llm) (`waterfall`) | [`agent-loop`](../packages/core/agent-loop), [`llm`](../packages/llm/llm), [`llm-replay`](../packages/test-support/llm-replay), [`session-checkpoint-policy`](../packages/session/session-checkpoint-policy), [`session-title`](../packages/session/session-title) |
|
||||
| `session-telemetry/record` | `waterfall` | [`packages/session/session-telemetry/src/index.ts:43`](../packages/session/session-telemetry/src/index.ts) | [`session-telemetry`](../packages/session/session-telemetry) (`waterfall`) | - |
|
||||
| `session/created` | `emit` | [`packages/core/session/src/index.ts:54`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`events.dispatch`) | [`compaction`](../packages/compaction/compaction), [`goal`](../packages/goal/goal), [`hook-protocol`](../packages/hooks/hook-protocol), [`llm-retry`](../packages/llm/llm-retry), [`permission-presets`](../packages/interaction/permission-presets), [`plan-mode`](../packages/plan/plan-mode), [`schedule`](../packages/schedule/schedule), `server`, [`session`](../packages/core/session), `session-controller`, [`session-log-deepseek`](../packages/session/session-log-deepseek), [`session-persistence`](../packages/session/session-persistence), [`session-projection`](../packages/session/session-projection), [`session-projection-cache`](../packages/session/session-projection-cache), [`session-telemetry`](../packages/session/session-telemetry), [`time-context`](../packages/context/time-context), [`tool-todo`](../packages/todo/tool-todo), [`tool-workflow`](../packages/workflow/tool-workflow), [`tools`](../packages/core/tools), [`user-approval`](../packages/interaction/user-approval) |
|
||||
| `session/disposed` | `emit` | [`packages/core/session/src/index.ts:64`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`events.dispatch`) | [`agent-loop`](../packages/core/agent-loop), `agent-team`, `session-controller`, [`session-persistence`](../packages/session/session-persistence), [`session-projection-cache`](../packages/session/session-projection-cache), [`session-telemetry`](../packages/session/session-telemetry), [`session-title`](../packages/session/session-title) |
|
||||
|
||||
@@ -45,7 +45,7 @@
|
||||
| `fs/write-intent` | `waterfall` | [`packages/fs/fs/src/index.ts:58`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`waterfall`), [`tool-str-replace-editor`](../packages/fs/tool-str-replace-editor) (`waterfall`) | [`fs-observation-policy`](../packages/fs/fs-observation-policy) |
|
||||
| `goal/changed` | `emit` | [`packages/goal/goal/src/domain.ts:114`](../packages/goal/goal/src/domain.ts) | [`goal`](../packages/goal/goal) (`emit`) | [`goal-round-driver`](../packages/goal/goal-round-driver) |
|
||||
| `llm/adapters-updated` | `emit` | [`packages/llm/llm/src/types.ts:23`](../packages/llm/llm/src/types.ts) | [`llm`](../packages/llm/llm) (`events.dispatch`) | [`acp`](../packages/acp/acp), [`llm`](../packages/llm/llm), `remotes` |
|
||||
| `llm/stream` | `waterfall` | [`packages/llm/llm/src/index.ts:65`](../packages/llm/llm/src/index.ts) | [`llm`](../packages/llm/llm) (`waterfall`) | [`agent-loop`](../packages/core/agent-loop), [`llm`](../packages/llm/llm), [`llm-replay`](../packages/test-support/llm-replay), [`session-checkpoint-policy`](../packages/session/session-checkpoint-policy), [`session-title`](../packages/session/session-title) |
|
||||
| `llm/stream` | `waterfall` | [`packages/llm/llm/src/index.ts:66`](../packages/llm/llm/src/index.ts) | [`llm`](../packages/llm/llm) (`waterfall`) | [`agent-loop`](../packages/core/agent-loop), [`llm`](../packages/llm/llm), [`llm-replay`](../packages/test-support/llm-replay), [`session-checkpoint-policy`](../packages/session/session-checkpoint-policy), [`session-title`](../packages/session/session-title) |
|
||||
| `session-telemetry/record` | `waterfall` | [`packages/session/session-telemetry/src/index.ts:43`](../packages/session/session-telemetry/src/index.ts) | [`session-telemetry`](../packages/session/session-telemetry) (`waterfall`) | - |
|
||||
| `session/created` | `emit` | [`packages/core/session/src/index.ts:54`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`events.dispatch`) | [`compaction`](../packages/compaction/compaction), [`goal`](../packages/goal/goal), [`hook-protocol`](../packages/hooks/hook-protocol), [`llm-retry`](../packages/llm/llm-retry), [`permission-presets`](../packages/interaction/permission-presets), [`plan-mode`](../packages/plan/plan-mode), [`schedule`](../packages/schedule/schedule), `server`, [`session`](../packages/core/session), `session-controller`, [`session-log-deepseek`](../packages/session/session-log-deepseek), [`session-persistence`](../packages/session/session-persistence), [`session-projection`](../packages/session/session-projection), [`session-projection-cache`](../packages/session/session-projection-cache), [`session-telemetry`](../packages/session/session-telemetry), [`time-context`](../packages/context/time-context), [`tool-todo`](../packages/todo/tool-todo), [`tool-workflow`](../packages/workflow/tool-workflow), [`tools`](../packages/core/tools), [`user-approval`](../packages/interaction/user-approval) |
|
||||
| `session/disposed` | `emit` | [`packages/core/session/src/index.ts:64`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`events.dispatch`) | [`agent-loop`](../packages/core/agent-loop), `agent-team`, `session-controller`, [`session-persistence`](../packages/session/session-persistence), [`session-projection-cache`](../packages/session/session-projection-cache), [`session-telemetry`](../packages/session/session-telemetry), [`session-title`](../packages/session/session-title) |
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write docs/subsystems/llm-streaming.md
|
||||
llm-streaming.md: 29efabd2b01659bdf2cc798ceadb4bb495e1731e
|
||||
llm-streaming.zh.md: 21ad56e526b9a507644b436b41ad063c5310b2ce
|
||||
llm-streaming.md: 73e8dc4a6a5b5408a5c85dcbeac4cfa2108a9ddd
|
||||
llm-streaming.zh.md: 7f98029f35100ec9c72f55c509f20b6709315f47
|
||||
|
||||
@@ -236,6 +236,44 @@ interface LlmFailure {
|
||||
}
|
||||
```
|
||||
|
||||
## Request-image pricing
|
||||
|
||||
An adapter whose provider charges visual tokens for request images declares per-route pricing by overriding `LlmAdapter.imageRequestPricing`, and `ctx.llm.imageRequestPricing(provider, model)` resolves it synchronously for consumers. The token meter resolves the routed model's pricing on every measurement so compaction pressure, retention, and range selection price image history as the routed request actually sends it; the DeepSeek adapter reproduces its own request projection (per-model pixel budget, oldest-first offload) and prices retained images with the published v4 vision accounting, while provider usage remains the authoritative anchor for completed requests.
|
||||
|
||||
```ts type-equiv
|
||||
/**
|
||||
* Request price of one ordered image occurrence under one exact model route's
|
||||
* request projection. Every occurrence resolves to the pair the wire actually
|
||||
* carries: provider visual tokens for a retained image, plus the model-visible
|
||||
* text sent with or instead of it (request-preview handle, offload placeholder,
|
||||
* or text-only substitution). The caller prices `text` with its own text
|
||||
* estimator so provider pricing never fixes a text tokenization.
|
||||
*/
|
||||
interface LlmImageRequestPrice {
|
||||
/** Provider visual tokens for the retained request image; 0 when only text represents this occurrence. */
|
||||
visualTokens: number
|
||||
/** Model-visible text sent for this occurrence, to be priced by the caller's text estimator. */
|
||||
text: string
|
||||
}
|
||||
```
|
||||
|
||||
```ts type-equiv
|
||||
/**
|
||||
* Provider-side request-image pricing for one exact model route. Implemented
|
||||
* by adapters whose provider charges visual tokens; consumers (the token
|
||||
* meter) resolve it synchronously per measurement, so implementations must not
|
||||
* perform I/O.
|
||||
*/
|
||||
interface LlmImageRequestPricing {
|
||||
/**
|
||||
* Price every image occurrence of one request projection.
|
||||
* @param images - durable image references in request order, one entry per occurrence.
|
||||
* @returns one price per occurrence, aligned by index with `images`.
|
||||
*/
|
||||
priceImages(images: readonly ImageAttachmentRef[]): readonly LlmImageRequestPrice[]
|
||||
}
|
||||
```
|
||||
|
||||
## The adapter contract
|
||||
|
||||
Every adapter MUST obey these, and every consumer may rely on them:
|
||||
@@ -736,6 +774,16 @@ declare abstract class LlmAdapter {
|
||||
* @returns a resolved policy, or `undefined` to use the normal defaults.
|
||||
*/
|
||||
providerRetryPolicy(_provider: string): ResolvedRetryPolicy | undefined;
|
||||
/**
|
||||
* Resolve provider-side request-image pricing for one exact model route.
|
||||
* The default declares none, so consumers fall back to their own neutral
|
||||
* estimate. Implementations must answer synchronously without I/O; the
|
||||
* token meter resolves this per measurement.
|
||||
* @param _provider - a route passed to `registerAdapter()` for this instance.
|
||||
* @param _model - exact model id passed to {@link GenerateOptions.model}.
|
||||
* @returns route-owned image pricing, or `undefined` when the route declares none.
|
||||
*/
|
||||
imageRequestPricing(_provider: string, _model: string): LlmImageRequestPricing | undefined;
|
||||
/**
|
||||
* List models this adapter can currently advertise for one owned provider.
|
||||
* The result is advisory: an adapter may accept unlisted model ids, and
|
||||
@@ -883,6 +931,17 @@ async discoverModels( settingsNs: string, request: LlmModelDiscoveryRequest, ):
|
||||
*/
|
||||
providerRetryPolicy(provider: string): ResolvedRetryPolicy
|
||||
|
||||
/**
|
||||
* Resolve provider-side request-image pricing for one exact route, or
|
||||
* `undefined` when the provider is unregistered or declares none. Unknown
|
||||
* providers degrade to `undefined` rather than throwing because callers
|
||||
* price durable history whose route may no longer be mounted.
|
||||
* @param provider - provider route named by a request header.
|
||||
* @param model - exact model id named by the same header.
|
||||
* @returns the owning adapter's image pricing for the route, when declared.
|
||||
*/
|
||||
imageRequestPricing(provider: string, model: string): LlmImageRequestPricing | undefined
|
||||
|
||||
/**
|
||||
* Discover models advertised by one registered provider. Catalog membership
|
||||
* is advisory and never changes routing or request validation.
|
||||
|
||||
@@ -238,6 +238,44 @@ interface LlmFailure {
|
||||
}
|
||||
```
|
||||
|
||||
## 请求图片定价
|
||||
|
||||
提供方对请求图片收取视觉 token 的适配器通过覆写 `LlmAdapter.imageRequestPricing` 声明按路由的定价，消费方经 `ctx.llm.imageRequestPricing(provider, model)` 同步解析。token 计量服务在每次计量时解析路由模型的定价，使 compaction 的压力、保留与选段都按路由请求实际发送的形式为图片历史计价；DeepSeek 适配器复现自身的请求投影（按模型的像素预算、最旧优先 offload），并用官方公布的 v4 视觉计量为保留图片定价，已完成请求仍以 provider usage 为权威锚点。
|
||||
|
||||
```ts type-equiv
|
||||
/**
|
||||
* Request price of one ordered image occurrence under one exact model route's
|
||||
* request projection. Every occurrence resolves to the pair the wire actually
|
||||
* carries: provider visual tokens for a retained image, plus the model-visible
|
||||
* text sent with or instead of it (request-preview handle, offload placeholder,
|
||||
* or text-only substitution). The caller prices `text` with its own text
|
||||
* estimator so provider pricing never fixes a text tokenization.
|
||||
*/
|
||||
interface LlmImageRequestPrice {
|
||||
/** Provider visual tokens for the retained request image; 0 when only text represents this occurrence. */
|
||||
visualTokens: number
|
||||
/** Model-visible text sent for this occurrence, to be priced by the caller's text estimator. */
|
||||
text: string
|
||||
}
|
||||
```
|
||||
|
||||
```ts type-equiv
|
||||
/**
|
||||
* Provider-side request-image pricing for one exact model route. Implemented
|
||||
* by adapters whose provider charges visual tokens; consumers (the token
|
||||
* meter) resolve it synchronously per measurement, so implementations must not
|
||||
* perform I/O.
|
||||
*/
|
||||
interface LlmImageRequestPricing {
|
||||
/**
|
||||
* Price every image occurrence of one request projection.
|
||||
* @param images - durable image references in request order, one entry per occurrence.
|
||||
* @returns one price per occurrence, aligned by index with `images`.
|
||||
*/
|
||||
priceImages(images: readonly ImageAttachmentRef[]): readonly LlmImageRequestPrice[]
|
||||
}
|
||||
```
|
||||
|
||||
## 适配器约定
|
||||
|
||||
每个适配器必须遵守以下规则，每个消费方可以依赖它们：
|
||||
@@ -742,6 +780,16 @@ declare abstract class LlmAdapter {
|
||||
* @returns a resolved policy, or `undefined` to use the normal defaults.
|
||||
*/
|
||||
providerRetryPolicy(_provider: string): ResolvedRetryPolicy | undefined;
|
||||
/**
|
||||
* Resolve provider-side request-image pricing for one exact model route.
|
||||
* The default declares none, so consumers fall back to their own neutral
|
||||
* estimate. Implementations must answer synchronously without I/O; the
|
||||
* token meter resolves this per measurement.
|
||||
* @param _provider - a route passed to `registerAdapter()` for this instance.
|
||||
* @param _model - exact model id passed to {@link GenerateOptions.model}.
|
||||
* @returns route-owned image pricing, or `undefined` when the route declares none.
|
||||
*/
|
||||
imageRequestPricing(_provider: string, _model: string): LlmImageRequestPricing | undefined;
|
||||
/**
|
||||
* List models this adapter can currently advertise for one owned provider.
|
||||
* The result is advisory: an adapter may accept unlisted model ids, and
|
||||
@@ -889,6 +937,17 @@ async discoverModels( settingsNs: string, request: LlmModelDiscoveryRequest, ):
|
||||
*/
|
||||
providerRetryPolicy(provider: string): ResolvedRetryPolicy
|
||||
|
||||
/**
|
||||
* Resolve provider-side request-image pricing for one exact route, or
|
||||
* `undefined` when the provider is unregistered or declares none. Unknown
|
||||
* providers degrade to `undefined` rather than throwing because callers
|
||||
* price durable history whose route may no longer be mounted.
|
||||
* @param provider - provider route named by a request header.
|
||||
* @param model - exact model id named by the same header.
|
||||
* @returns the owning adapter's image pricing for the route, when declared.
|
||||
*/
|
||||
imageRequestPricing(provider: string, model: string): LlmImageRequestPricing | undefined
|
||||
|
||||
/**
|
||||
* Discover models advertised by one registered provider. Catalog membership
|
||||
* is advisory and never changes routing or request validation.
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write docs/subsystems/token-meter.md
|
||||
token-meter.md: b8b2add194cbafafc250c6fc15b23e246d87d9c1
|
||||
token-meter.zh.md: a1366d0d1d113c0a7df77b5b3bc53c9121fe9ae3
|
||||
token-meter.md: 9c4a1e4b95ffd84f65f7a73e208be245378a3301
|
||||
token-meter.zh.md: d9e2e7f773041ccb6d1e4c3cc4d81a342db0cc01
|
||||
|
||||
@@ -19,14 +19,14 @@ interface TokenMeasurement {
|
||||
readonly surfaceDeltaTokens: number
|
||||
/** Non-negative current request-and-response pressure. */
|
||||
readonly totalTokens: number
|
||||
/** Total heuristic tokens across the current surface. */
|
||||
/** Total route-priced request tokens across the current surface; equals the sum of the node prices. */
|
||||
readonly surfaceTokens: number
|
||||
/** Current surface nodes in positional head-to-tail order. */
|
||||
readonly nodes: readonly TokenSurfaceNode[]
|
||||
}
|
||||
```
|
||||
|
||||
`baseline.kind === 'usage'` means the latest successful provider call has the same canonical request envelope and its total is no lower than that call's full heuristic anchor. `estimated` means no reusable conservative usage anchor exists, so the service priced the complete envelope and surface with its fixed heuristic. A later successful request replaces the earlier anchor; signed `surfaceDeltaTokens` preserves growth and shrinkage relative to a matching anchor. `totalTokens` remains request-and-response pressure, while `surfaceTokens` is the surface-only heuristic total and equals the sum of the node prices.
|
||||
Every measurement resolves the effective envelope's routed provider/model to that route's declared request-image pricing through `ctx.llm`, so image occurrences are priced as the visual tokens plus model-visible text the request actually sends; routes and compositions without declared pricing keep the fixed heuristic. `baseline.kind === 'usage'` means the latest successful provider call has the same canonical request envelope and its total is no lower than that call's full route-priced anchor. `estimated` means no reusable conservative usage anchor exists, so the service priced the complete envelope and surface itself. A later successful request replaces the earlier anchor; signed `surfaceDeltaTokens` preserves growth and shrinkage relative to a matching anchor, repricing both sides under the same route. `totalTokens` remains request-and-response pressure, while `surfaceTokens` is the surface-only route-priced total and equals the sum of the node prices.
|
||||
|
||||
## `TokenSurfaceNode`
|
||||
|
||||
@@ -35,8 +35,19 @@ interface TokenMeasurement {
|
||||
interface TokenSurfaceNode {
|
||||
/** Durable sequence number of the surface event. */
|
||||
readonly seq: number
|
||||
/** Heuristic tokens for the exact message projected by this node. */
|
||||
/**
|
||||
* Request-pressure tokens for the exact message projected by this node under
|
||||
* the measured route: image occurrences carry the route's declared visual
|
||||
* price when the routed adapter declares one, and the fixed heuristic
|
||||
* otherwise. Trigger, retention, and range selection all read this price.
|
||||
*/
|
||||
readonly tokens: number
|
||||
/**
|
||||
* Fixed-heuristic tokens for the same message, independent of any route.
|
||||
* The shadow-price protocol prices replacements with this value so the O(1)
|
||||
* projection fold stays in agreement with its own appends.
|
||||
*/
|
||||
readonly heuristicTokens: number
|
||||
}
|
||||
```
|
||||
|
||||
@@ -60,14 +71,18 @@ Replay owner for one service-wide estimator and isolated per-session folds.
|
||||
/**
|
||||
* Measure current request pressure and surface through the durable tail.
|
||||
*
|
||||
* Provider usage is reused only when the latest successful call's canonical
|
||||
* request envelope matches `requestHeader` and its total is no lower than
|
||||
* that call's full heuristic anchor; otherwise the complete envelope and
|
||||
* surface are heuristically repriced.
|
||||
* The effective envelope's routed provider/model selects the request-image
|
||||
* pricing every node is priced under: a route whose adapter declares image
|
||||
* pricing charges each retained image its visual tokens plus its
|
||||
* model-visible text, while other routes keep the fixed heuristic. Provider
|
||||
* usage is reused only when the latest successful call's canonical request
|
||||
* envelope matches `requestHeader` and its total is no lower than that
|
||||
* call's full route-priced anchor; otherwise the complete envelope and
|
||||
* surface are repriced.
|
||||
*
|
||||
* `requestHeader` affects request pressure only; surface fields always
|
||||
* describe the current session surface. Every call clones those positional
|
||||
* nodes, so measurement is O(surface).
|
||||
* `requestHeader` replaces the latest logged envelope for pressure and node
|
||||
* pricing; the node set always describes the current session surface. Every
|
||||
* call clones those positional nodes, so measurement is O(surface).
|
||||
*
|
||||
* @param session - session to replay through its current durable tail.
|
||||
* @param requestHeader - optional effective request envelope replacing the latest logged header.
|
||||
|
||||
@@ -19,14 +19,14 @@ interface TokenMeasurement {
|
||||
readonly surfaceDeltaTokens: number
|
||||
/** Non-negative current request-and-response pressure. */
|
||||
readonly totalTokens: number
|
||||
/** Total heuristic tokens across the current surface. */
|
||||
/** Total route-priced request tokens across the current surface; equals the sum of the node prices. */
|
||||
readonly surfaceTokens: number
|
||||
/** Current surface nodes in positional head-to-tail order. */
|
||||
readonly nodes: readonly TokenSurfaceNode[]
|
||||
}
|
||||
```
|
||||
|
||||
`baseline.kind === 'usage'` 表示最近一次成功的提供方调用具有相同的规范请求 envelope，且该调用的总量不低于其完整启发式锚点。`estimated` 表示不存在可复用的保守 usage 锚点，因此服务使用固定启发式规则对完整信封和表层定价。后续成功请求会替换早先的锚点；有符号的 `surfaceDeltaTokens` 会保留相对于匹配锚点的增长与缩减。`totalTokens` 仍表示请求与响应压力，`surfaceTokens` 则是仅针对表层的启发式总量，等于所有节点价格之和。
|
||||
每次计量都会通过 `ctx.llm` 把生效信封的路由 provider/model 解析为该路由声明的请求图片定价，因此图片出现处按请求实际发送的视觉 token 加模型可见文本计价；未声明定价的路由与组合保持固定启发式规则。`baseline.kind === 'usage'` 表示最近一次成功的提供方调用具有相同的规范请求 envelope，且该调用的总量不低于其完整路由定价锚点。`estimated` 表示不存在可复用的保守 usage 锚点，因此服务自行对完整信封和表层定价。后续成功请求会替换早先的锚点；有符号的 `surfaceDeltaTokens` 会保留相对于匹配锚点的增长与缩减，且两侧按同一路由重新定价。`totalTokens` 仍表示请求与响应压力，`surfaceTokens` 则是表层的路由定价总量，等于所有节点价格之和。
|
||||
|
||||
## `TokenSurfaceNode`
|
||||
|
||||
@@ -35,8 +35,19 @@ interface TokenMeasurement {
|
||||
interface TokenSurfaceNode {
|
||||
/** Durable sequence number of the surface event. */
|
||||
readonly seq: number
|
||||
/** Heuristic tokens for the exact message projected by this node. */
|
||||
/**
|
||||
* Request-pressure tokens for the exact message projected by this node under
|
||||
* the measured route: image occurrences carry the route's declared visual
|
||||
* price when the routed adapter declares one, and the fixed heuristic
|
||||
* otherwise. Trigger, retention, and range selection all read this price.
|
||||
*/
|
||||
readonly tokens: number
|
||||
/**
|
||||
* Fixed-heuristic tokens for the same message, independent of any route.
|
||||
* The shadow-price protocol prices replacements with this value so the O(1)
|
||||
* projection fold stays in agreement with its own appends.
|
||||
*/
|
||||
readonly heuristicTokens: number
|
||||
}
|
||||
```
|
||||
|
||||
@@ -60,14 +71,18 @@ Replay owner for one service-wide estimator and isolated per-session folds.
|
||||
/**
|
||||
* Measure current request pressure and surface through the durable tail.
|
||||
*
|
||||
* Provider usage is reused only when the latest successful call's canonical
|
||||
* request envelope matches `requestHeader` and its total is no lower than
|
||||
* that call's full heuristic anchor; otherwise the complete envelope and
|
||||
* surface are heuristically repriced.
|
||||
* The effective envelope's routed provider/model selects the request-image
|
||||
* pricing every node is priced under: a route whose adapter declares image
|
||||
* pricing charges each retained image its visual tokens plus its
|
||||
* model-visible text, while other routes keep the fixed heuristic. Provider
|
||||
* usage is reused only when the latest successful call's canonical request
|
||||
* envelope matches `requestHeader` and its total is no lower than that
|
||||
* call's full route-priced anchor; otherwise the complete envelope and
|
||||
* surface are repriced.
|
||||
*
|
||||
* `requestHeader` affects request pressure only; surface fields always
|
||||
* describe the current session surface. Every call clones those positional
|
||||
* nodes, so measurement is O(surface).
|
||||
* `requestHeader` replaces the latest logged envelope for pressure and node
|
||||
* pricing; the node set always describes the current session surface. Every
|
||||
* call clones those positional nodes, so measurement is O(surface).
|
||||
*
|
||||
* @param session - session to replay through its current durable tail.
|
||||
* @param requestHeader - optional effective request envelope replacing the latest logged header.
|
||||
|
||||
@@ -22,7 +22,7 @@ export { canPassThroughNormalization, normalizeImage } from './normalization.ts'
|
||||
export type { NormalizedImage, NormalizationPolicy } from './normalization.ts'
|
||||
export { commitPreparedImageFile, prepareImageFile, readImageFile, saveImageFile, validateImageFile } from './store.ts'
|
||||
export type { PreparedImageFile } from './store.ts'
|
||||
export { readRequestImageFile, requestImageDimensions, requestImageVariantId } from './request-image.ts'
|
||||
export { readRequestImageFile, requestImageVariantId } from './request-image.ts'
|
||||
|
||||
/** Default maximum encoded bytes for one submitted image; oversized sources are refused, not shrunk. */
|
||||
export const DEFAULT_MAX_IMAGE_BYTES = 20 * 1024 * 1024
|
||||
|
||||
@@ -1,10 +1,9 @@
|
||||
/** Deterministic provider-independent image normalization. */
|
||||
|
||||
import sharp, { type Sharp } from 'sharp'
|
||||
import { AttachmentError } from '@deepseek-ai/dsh-attachment'
|
||||
import { AttachmentError, requestImageDimensions } from '@deepseek-ai/dsh-attachment'
|
||||
import type { ImageMediaType } from '@deepseek-ai/dsh-attachment'
|
||||
import { encodeFirstWithinLimit, encodingLadder, isExhaustedEncoding } from './encoding.ts'
|
||||
import { requestImageDimensions } from './request-image.ts'
|
||||
import { detectImage, encodedAlphaIsCompatible } from './image.ts'
|
||||
import type { DetectedImage } from './image.ts'
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@ import { createHash, randomUUID } from 'node:crypto'
|
||||
import { mkdir, readFile, rename, rm, writeFile } from 'node:fs/promises'
|
||||
import { dirname, join } from 'node:path'
|
||||
import sharp, { type Sharp } from 'sharp'
|
||||
import { AttachmentError, ImageVariantId } from '@deepseek-ai/dsh-attachment'
|
||||
import { AttachmentError, ImageVariantId, requestImageDimensions } from '@deepseek-ai/dsh-attachment'
|
||||
import type {
|
||||
ImageMediaType,
|
||||
ImageAttachmentRef,
|
||||
@@ -39,38 +39,6 @@ function digest(value: string | Uint8Array): string {
|
||||
return createHash('sha256').update(value).digest('hex')
|
||||
}
|
||||
|
||||
/**
|
||||
* Compute aspect-preserving integer dimensions within a hard total-pixel budget.
|
||||
* @param width - positive source width.
|
||||
* @param height - positive source height.
|
||||
* @param maxPixels - positive width-times-height cap.
|
||||
* @returns inward-rounded dimensions; small images are not enlarged.
|
||||
*/
|
||||
export function requestImageDimensions(
|
||||
width: number,
|
||||
height: number,
|
||||
maxPixels: number,
|
||||
): { width: number; height: number } {
|
||||
const scale = Math.min(1, Math.sqrt(maxPixels / (width * height)))
|
||||
if (scale === 1) return { width, height }
|
||||
if (width >= height) {
|
||||
let projectedWidth = Math.max(1, Math.floor(width * scale))
|
||||
let projectedHeight = Math.max(1, Math.round(projectedWidth * height / width))
|
||||
while (projectedWidth * projectedHeight > maxPixels && projectedWidth > 1) {
|
||||
projectedWidth -= 1
|
||||
projectedHeight = Math.max(1, Math.round(projectedWidth * height / width))
|
||||
}
|
||||
return { width: projectedWidth, height: projectedHeight }
|
||||
}
|
||||
let projectedHeight = Math.max(1, Math.floor(height * scale))
|
||||
let projectedWidth = Math.max(1, Math.round(projectedHeight * width / height))
|
||||
while (projectedWidth * projectedHeight > maxPixels && projectedHeight > 1) {
|
||||
projectedHeight -= 1
|
||||
projectedWidth = Math.max(1, Math.round(projectedHeight * width / height))
|
||||
}
|
||||
return { width: projectedWidth, height: projectedHeight }
|
||||
}
|
||||
|
||||
function checkedInteger(value: number, name: string): number {
|
||||
if (!Number.isSafeInteger(value) || value <= 0) {
|
||||
throw new AttachmentError(`${name} must be a positive integer.`, 'INVALID_ATTACHMENT_REF')
|
||||
|
||||
@@ -5,7 +5,7 @@ import { Context } from '@deepseek-ai/cordis'
|
||||
import sharp from 'sharp'
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest'
|
||||
import { CompressionLimiter } from '../src/compression-limiter.ts'
|
||||
import LocalAttachmentStore, { requestImageDimensions } from '../src/index.ts'
|
||||
import LocalAttachmentStore from '../src/index.ts'
|
||||
|
||||
const homes: string[] = []
|
||||
|
||||
@@ -42,34 +42,6 @@ afterEach(async () => {
|
||||
await Promise.all(homes.splice(0).map(home => rm(home, { recursive: true, force: true })))
|
||||
})
|
||||
|
||||
describe('request image dimensions', () => {
|
||||
it.each([
|
||||
[4096, 4096, 800, 800],
|
||||
[4096, 2048, 1130, 565],
|
||||
[3840, 2160, 1066, 600],
|
||||
[320, 240, 320, 240],
|
||||
])('projects %sx%s under 640,000 pixels as %sx%s', (width, height, expectedWidth, expectedHeight) => {
|
||||
const projected = requestImageDimensions(width, height, 640_000)
|
||||
expect(projected).toEqual({
|
||||
width: expectedWidth,
|
||||
height: expectedHeight,
|
||||
})
|
||||
expect(projected.width * projected.height).toBeLessThanOrEqual(640_000)
|
||||
})
|
||||
|
||||
it('projects a portrait within the same total-pixel budget', () => {
|
||||
const projected = requestImageDimensions(2160, 3840, 640_000)
|
||||
|
||||
expect(projected).toEqual({ width: 600, height: 1066 })
|
||||
expect(projected.width * projected.height).toBeLessThanOrEqual(640_000)
|
||||
})
|
||||
|
||||
it('rounds a portrait inward when integer aspect rounding crosses the pixel cap', () => {
|
||||
expect(requestImageDimensions(2, 4, 5)).toEqual({ width: 1, height: 2 })
|
||||
})
|
||||
|
||||
})
|
||||
|
||||
describe('local request-image cache', () => {
|
||||
it('passes through an in-budget attachment and composes ordered request reads', async () => {
|
||||
const attachments = await store()
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/attachment/attachment/README.md
|
||||
README.md: 21030a492464aae528d4c06b4b72d4a94c0a2603
|
||||
README.zh.md: 0540996f99b3250331e567e174264cf7da8aa474
|
||||
README.md: 976bfc82a4cf8a626259ffcddabcbead8ba03154
|
||||
README.zh.md: bc33293b34295250c332d6111fb0d52e506c47db
|
||||
|
||||
@@ -4,7 +4,7 @@ English | [中文](README.zh.md)
|
||||
|
||||
The durable attachment seam. `ctx.attachments` validates and durably commits a provider-independent normalized image, then returns a serializable `ImageAttachmentRef`; consumers never persist browser paths, object URLs, provider URLs, local storage paths, or base64 in session events.
|
||||
|
||||
Unsent composer images remain browser-owned temporary drafts. `validateImage` runs the complete admission policy without persisting. `saveImages` owns batch count and aggregate-byte limits, prepares every normalized attachment before publishing any member, then commits in order and returns references only after the complete batch succeeds. A later storage failure returns no partial references, although an earlier immutable content-addressed object may remain unreachable until reference-aware garbage collection exists. `AttachmentError.code` uses the closed `AttachmentErrorCode` string union. Its `ImageAdmissionErrorCode` subset marks caller-correctable image-input failures; `isImageAdmissionError` recognizes that subset at runtime so each protocol adapter can map its own error vocabulary. `saveImage` commits one accepted image before any model-visible session event is published and returns its `ImageAttachmentRef`. When normalization reduces the raster, the reference records the orientation-applied input size in `originalDimensions`. `readImage` verifies the normalized attachment against its logged metadata. `readImageRequest` deterministically derives a route-sized request version whose identity covers the attachment id, transform version, pixel and byte budgets, and encoder settings. `imageHostPath` optionally exposes the provider-owned object's absolute host path; it makes no claim that the current model tools can read that path. An LLM consumer combines this location with the mounted filesystem's execution-world mapping when it serializes a request. That current access path remains separate from the request version and its `variantId`. Callers compose ordered batches with `Promise.all(refs.map(...))`; the local implementation still bounds compression through its instance limiter, cache, and singleflight. Callers may cancel reads and projections; implementations preserve cancellation instead of translating it into a storage failure.
|
||||
Unsent composer images remain browser-owned temporary drafts. `validateImage` runs the complete admission policy without persisting. `saveImages` owns batch count and aggregate-byte limits, prepares every normalized attachment before publishing any member, then commits in order and returns references only after the complete batch succeeds. A later storage failure returns no partial references, although an earlier immutable content-addressed object may remain unreachable until reference-aware garbage collection exists. `AttachmentError.code` uses the closed `AttachmentErrorCode` string union. Its `ImageAdmissionErrorCode` subset marks caller-correctable image-input failures; `isImageAdmissionError` recognizes that subset at runtime so each protocol adapter can map its own error vocabulary. `saveImage` commits one accepted image before any model-visible session event is published and returns its `ImageAttachmentRef`. When normalization reduces the raster, the reference records the orientation-applied input size in `originalDimensions`. `readImage` verifies the normalized attachment against its logged metadata. `readImageRequest` deterministically derives a route-sized request version whose identity covers the attachment id, transform version, pixel and byte budgets, and encoder settings. The pure `requestImageDimensions` export computes that projection's aspect-preserving dimensions from a total-pixel budget, so providers and request pricing share one geometry. `imageHostPath` optionally exposes the provider-owned object's absolute host path; it makes no claim that the current model tools can read that path. An LLM consumer combines this location with the mounted filesystem's execution-world mapping when it serializes a request. That current access path remains separate from the request version and its `variantId`. Callers compose ordered batches with `Promise.all(refs.map(...))`; the local implementation still bounds compression through its instance limiter, cache, and singleflight. Callers may cancel reads and projections; implementations preserve cancellation instead of translating it into a storage failure.
|
||||
|
||||
`admitEncodedImages(attachments, images)` is the shared wire entry used by every RPC endpoint that accepts browser uploads (the session prompt endpoint and the command executor): it enforces canonical base64 on every member, then delegates batch admission — limits, validation, ordered commit — to `saveImages`. The base64 upload form is `EncodedImageAttachment`, exported from `@deepseek-ai/dsh-attachment/types` so wire contracts can reference it.
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
持久附件服务边界。`ctx.attachments` 校验并持久提交提供方无关的规范化图片，随后返回可序列化的 `ImageAttachmentRef`；消费方绝不会在会话事件中持久保存浏览器路径、对象 URL、提供方 URL、本地存储路径或 base64。
|
||||
|
||||
未发送的输入区图片仍是由浏览器持有的临时草稿。`validateImage` 运行完整准入策略但不执行持久化。`saveImages` 负责批次图片数量和总字节限制，在发布任何成员前准备全部规范化附件，然后按顺序提交，并且只在完整批次成功后返回引用。后续存储失败不会返回部分引用，但较早写入的不可变内容寻址对象可能保持不可达，直至具备按引用感知的垃圾回收。`AttachmentError.code` 使用封闭的 `AttachmentErrorCode` 字符串联合类型。其 `ImageAdmissionErrorCode` 子集标记可由调用方修正的图片输入失败；`isImageAdmissionError` 在运行时识别该子集，使每个协议适配器可以映射自己的错误词汇。`saveImage` 会在发布任何模型可见的会话事件前提交一张已接受的图片，并直接返回 `ImageAttachmentRef`。规范化过程缩小图片时，引用会通过 `originalDimensions` 记录应用方向后的输入尺寸。`readImage` 根据已记录的元数据校验规范化附件。`readImageRequest` 确定性派生路由所需的请求版本，其身份覆盖附件 ID、变换策略版本、像素和字节预算及编码参数。`imageHostPath` 可以给出提供方所持对象的绝对宿主路径，但不保证当前模型工具能够读取它。LLM 消费方在序列化请求时将这个位置与当前文件系统提供的执行环境映射组合起来。解析出的访问路径独立于请求版本及其 `variantId`。调用方通过 `Promise.all(refs.map(...))` 组合有序批次，本地实现仍通过实例级限流器、缓存和 singleflight 限制压缩并发。调用方可以取消读取和投影；实现保留取消结果，不把它转换为存储失败。
|
||||
未发送的输入区图片仍是由浏览器持有的临时草稿。`validateImage` 运行完整准入策略但不执行持久化。`saveImages` 负责批次图片数量和总字节限制，在发布任何成员前准备全部规范化附件，然后按顺序提交，并且只在完整批次成功后返回引用。后续存储失败不会返回部分引用，但较早写入的不可变内容寻址对象可能保持不可达，直至具备按引用感知的垃圾回收。`AttachmentError.code` 使用封闭的 `AttachmentErrorCode` 字符串联合类型。其 `ImageAdmissionErrorCode` 子集标记可由调用方修正的图片输入失败；`isImageAdmissionError` 在运行时识别该子集，使每个协议适配器可以映射自己的错误词汇。`saveImage` 会在发布任何模型可见的会话事件前提交一张已接受的图片，并直接返回 `ImageAttachmentRef`。规范化过程缩小图片时，引用会通过 `originalDimensions` 记录应用方向后的输入尺寸。`readImage` 根据已记录的元数据校验规范化附件。`readImageRequest` 确定性派生路由所需的请求版本，其身份覆盖附件 ID、变换策略版本、像素和字节预算及编码参数。纯函数导出 `requestImageDimensions` 按总像素预算计算该投影的保持宽高比尺寸，使提供方与请求定价共享同一套几何计算。`imageHostPath` 可以给出提供方所持对象的绝对宿主路径，但不保证当前模型工具能够读取它。LLM 消费方在序列化请求时将这个位置与当前文件系统提供的执行环境映射组合起来。解析出的访问路径独立于请求版本及其 `variantId`。调用方通过 `Promise.all(refs.map(...))` 组合有序批次，本地实现仍通过实例级限流器、缓存和 singleflight 限制压缩并发。调用方可以取消读取和投影；实现保留取消结果，不把它转换为存储失败。
|
||||
|
||||
`admitEncodedImages(attachments, images)` 是每个接受浏览器上传的 RPC 端点（会话 prompt 端点与命令执行器）共用的 wire 入口：它对每个成员强制执行规范 base64，随后把批量准入——限额、校验、有序提交——委托给 `saveImages`。base64 上传形式为 `EncodedImageAttachment`，从 `@deepseek-ai/dsh-attachment/types` 导出，供 wire 契约引用。
|
||||
|
||||
|
||||
@@ -15,6 +15,7 @@ export { AttachmentId, ImageVariantId } from './brand.ts'
|
||||
export { AttachmentError, isImageAdmissionError } from './error.ts'
|
||||
export type { AttachmentErrorCode, ImageAdmissionErrorCode } from './error.ts'
|
||||
export { admitEncodedImages } from './admission.ts'
|
||||
export { requestImageDimensions } from './request-projection.ts'
|
||||
export type {
|
||||
AttachmentId as AttachmentIdType,
|
||||
EncodedImageAttachment,
|
||||
|
||||
@@ -0,0 +1,36 @@
|
||||
/**
|
||||
* Pure request-projection geometry shared by attachment providers and
|
||||
* provider-side request pricing. @module @deepseek-ai/dsh-attachment/request-projection
|
||||
*/
|
||||
|
||||
/**
|
||||
* Compute aspect-preserving integer dimensions within a hard total-pixel budget.
|
||||
* @param width - positive source width.
|
||||
* @param height - positive source height.
|
||||
* @param maxPixels - positive width-times-height cap.
|
||||
* @returns inward-rounded dimensions; small images are not enlarged.
|
||||
*/
|
||||
export function requestImageDimensions(
|
||||
width: number,
|
||||
height: number,
|
||||
maxPixels: number,
|
||||
): { width: number; height: number } {
|
||||
const scale = Math.min(1, Math.sqrt(maxPixels / (width * height)))
|
||||
if (scale === 1) return { width, height }
|
||||
if (width >= height) {
|
||||
let projectedWidth = Math.max(1, Math.floor(width * scale))
|
||||
let projectedHeight = Math.max(1, Math.round(projectedWidth * height / width))
|
||||
while (projectedWidth * projectedHeight > maxPixels && projectedWidth > 1) {
|
||||
projectedWidth -= 1
|
||||
projectedHeight = Math.max(1, Math.round(projectedWidth * height / width))
|
||||
}
|
||||
return { width: projectedWidth, height: projectedHeight }
|
||||
}
|
||||
let projectedHeight = Math.max(1, Math.floor(height * scale))
|
||||
let projectedWidth = Math.max(1, Math.round(projectedHeight * width / height))
|
||||
while (projectedWidth * projectedHeight > maxPixels && projectedHeight > 1) {
|
||||
projectedHeight -= 1
|
||||
projectedWidth = Math.max(1, Math.round(projectedHeight * width / height))
|
||||
}
|
||||
return { width: projectedWidth, height: projectedHeight }
|
||||
}
|
||||
@@ -0,0 +1,29 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { requestImageDimensions } from '../src/index.ts'
|
||||
|
||||
describe('request image dimensions', () => {
|
||||
it.each([
|
||||
[4096, 4096, 800, 800],
|
||||
[4096, 2048, 1130, 565],
|
||||
[3840, 2160, 1066, 600],
|
||||
[320, 240, 320, 240],
|
||||
])('projects %sx%s under 640,000 pixels as %sx%s', (width, height, expectedWidth, expectedHeight) => {
|
||||
const projected = requestImageDimensions(width, height, 640_000)
|
||||
expect(projected).toEqual({
|
||||
width: expectedWidth,
|
||||
height: expectedHeight,
|
||||
})
|
||||
expect(projected.width * projected.height).toBeLessThanOrEqual(640_000)
|
||||
})
|
||||
|
||||
it('projects a portrait within the same total-pixel budget', () => {
|
||||
const projected = requestImageDimensions(2160, 3840, 640_000)
|
||||
|
||||
expect(projected).toEqual({ width: 600, height: 1066 })
|
||||
expect(projected.width * projected.height).toBeLessThanOrEqual(640_000)
|
||||
})
|
||||
|
||||
it('rounds a portrait inward when integer aspect rounding crosses the pixel cap', () => {
|
||||
expect(requestImageDimensions(2, 4, 5)).toEqual({ width: 1, height: 2 })
|
||||
})
|
||||
})
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/compaction/compaction-basic/README.md
|
||||
README.md: 82df7b7e399cef80d92611819e9e58f13babf175
|
||||
README.zh.md: d79ea39ad8db7e2ab7f0a5b2ef52b61ab054d01b
|
||||
README.md: e45228080db414c503420d22f5faca3daf1d3966
|
||||
README.zh.md: 1ff7bede73f36ec81d1dba0f2b414736ade09457
|
||||
|
||||
@@ -10,7 +10,7 @@ This package owns the Service Provider role of the compaction capability — see
|
||||
|
||||
This backend owns the compaction policy:
|
||||
|
||||
- **Measurement** — the singleton `ctx.tokenMeter` prices the latest canonical logged envelope and current surface at one consumed-log revision. Step-boundary pressure therefore includes the actual system prompt, tools, routing, assistant completion, tool results, buffered context, and steering.
|
||||
- **Measurement** — the singleton `ctx.tokenMeter` prices the latest canonical logged envelope and current surface at one consumed-log revision, under the routed model's declared request-image pricing when its adapter declares one. Step-boundary pressure therefore includes the actual system prompt, tools, routing, assistant completion, tool results, buffered context, steering, and route-priced image history; trigger, recent-tail retention, range selection, and the summary-shrink comparison all read the same route-priced per-node figures, while the logged shadow price of a replaced range stays on the route-independent fixed heuristic so pure projection folds remain consistent.
|
||||
- **Routed policy** — proactive pressure resolves capacity from the adapter that owns the latest durable provider/model route, then scales the default policy plus an optional exact-target override into concrete token budgets. Model discovery remains advisory and is not consulted.
|
||||
- **Model-free pruning** — after pressure or canonical overflow qualifies, the optional [`ctx.toolResultPruner`](../compaction-tool-result-pruner/README.md) service rewrites oversized tool results before range selection. Compact-basic remeasures through `ctx.tokenMeter`, skips summarization when pressure becomes safe, and otherwise summarizes the pruned surface. Below-pressure step checks never prune.
|
||||
- **Retention** — compact the oldest whole surface units while preserving a recent tail and balanced tool-call/result cuts through the [`dsh-compaction` boundary helpers](../compaction/README.md#tool-pairing-boundaries). Turn boundaries do not protect old steps inside a runaway turn. An open indivisible tail declines until it closes. The optional pruner can repair an oversized closed tool unit when its text-bearing result is the removable bulk; indivisible non-tool units and non-prunable tool remainders remain out of scope.
|
||||
@@ -157,7 +157,7 @@ The replayed system prompt, tools, and shadowed-region messages match the conver
|
||||
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
- **Meter accuracy follows the fixed heuristic** — missing reusable provider usage falls back to character count plus structural overhead rather than exact tokenization.
|
||||
- **Meter accuracy follows the fixed heuristic** — missing reusable provider usage falls back to character count plus structural overhead rather than exact tokenization; image occurrences carry provider-exact visual tokens only on routes whose adapter declares request-image pricing.
|
||||
- **Overflow classification is adapter-maintained** — provider wording can change; both DeepSeek adapters normalize currently recognized context-limit failures to `CONTEXT_WINDOW_EXCEEDED`.
|
||||
- **Some indivisible-unit and envelope-only overflow remains outside surface compaction** — recovery cannot shrink system/tools/prefix, split an indivisible non-tool node, or repair a tool unit whose non-prunable remainder still exceeds the window. The optional pruner can shrink text-bearing tool-result bulk inside an otherwise indivisible pair.
|
||||
- **`compactRegion` requires an open turn** — a manual call on a fully-closed session throws ("no open turn") rather than compacting.
|
||||
|
||||
@@ -10,7 +10,7 @@
|
||||
|
||||
该后端拥有压缩策略：
|
||||
|
||||
- **测量**：单例 `ctx.tokenMeter` 会在同一个已消费日志 revision 上，计量最新一份规范化已记录 envelope 与当前表层的 token 用量。因此，步骤边界的压力计量会包含实际系统提示词、工具、路由、assistant 完成、工具结果、缓冲上下文与 steering（中途引导）。
|
||||
- **测量**：单例 `ctx.tokenMeter` 会在同一个已消费日志 revision 上，计量最新一份规范化已记录 envelope 与当前表层的 token 用量；当路由模型的适配器声明了请求图片定价时，按该定价计量。因此，步骤边界的压力计量会包含实际系统提示词、工具、路由、assistant 完成、工具结果、缓冲上下文、steering（中途引导）与按路由定价的图片历史；触发、近期尾部保留、范围选择与摘要收缩比较读取同一套路由定价的逐节点数字，而被替换范围记录的影子价保持在与路由无关的固定启发式规则上，使纯投影 fold 保持一致。
|
||||
- **路由策略**：主动压力从拥有最新持久提供方／模型路由的适配器解析容量，再将默认策略与可选的精确目标覆盖缩放为具体 token 预算。模型发现仍仅供参考，不参与此处的策略解析。
|
||||
- **不依赖模型的剪枝**：在压力或规范溢出符合条件后，可选的 [`ctx.toolResultPruner`](../compaction-tool-result-pruner/README.zh.md) 服务会在选择范围之前改写超大工具结果。Compact-basic 通过 `ctx.tokenMeter` 重新测量；如果压力已回到安全范围，就跳过摘要，否则对已剪枝的表层进行摘要。低于压力的步骤检查绝不剪枝。
|
||||
- **保留**：压缩最旧的完整表层单元，同时保留近期尾部，并通过 [`dsh-compaction` 边界 helper](../compaction/README.zh.md#tool-pairing-boundaries) 将切分点调整到工具调用／结果配对平衡的位置。轮次边界不会保护失控轮次内的旧步骤。尚未闭合且不可分的尾部会在闭合前拒绝压缩。当闭合的超大工具单元以文本型结果为可移除主体时，可选 pruner 可以修复它；不可分的非工具单元与不可剪枝的工具剩余部分不在范围内。
|
||||
@@ -157,7 +157,7 @@ Rules:
|
||||
|
||||
## 已知限制与暂缓事项
|
||||
|
||||
- **计量准确度取决于固定启发式规则**：可复用提供方用量缺失时，会回退到字符数加结构开销，而非精确的 token 化。
|
||||
- **计量准确度取决于固定启发式规则**：可复用提供方用量缺失时，会回退到字符数加结构开销，而非精确的 token 化；只有在适配器声明了请求图片定价的路由上，图片出现处才携带提供方精确的视觉 token。
|
||||
- **溢出分类由适配器维护**：提供方措辞可能改变；两个 DeepSeek 适配器将当前可识别的上下文限制失败规范化为 `CONTEXT_WINDOW_EXCEEDED`。
|
||||
- **部分不可分单元与仅 envelope 溢出仍不在表层压缩范围内**：恢复无法缩减系统／工具／前缀、拆分不可分的非工具节点，或修复不可剪枝剩余部分仍超出窗口的工具单元。可选 pruner 可以缩减原本不可分工具对内的文本型工具结果主体。
|
||||
- **`compactRegion` 要求存在未结束的轮次**：在完全关闭的会话上手动调用会抛出异常（「no open turn」），而不是执行压缩。
|
||||
|
||||
@@ -43,6 +43,8 @@ interface PreparedCompaction extends SurfaceSelection {
|
||||
readonly measurement: TokenMeasurement
|
||||
readonly selectedNodes: TokenMeasurement['nodes']
|
||||
readonly shadowedTokenCount: number
|
||||
/** Route-priced total of the selected span; the shrink comparison's unit. */
|
||||
readonly shadowedRouteTokenCount: number
|
||||
readonly input: SummarizationInput
|
||||
}
|
||||
|
||||
@@ -351,7 +353,12 @@ function prepareCompaction(
|
||||
...selection,
|
||||
measurement,
|
||||
selectedNodes,
|
||||
shadowedTokenCount: selectedNodes.reduce((total, node) => total + node.tokens, 0),
|
||||
// The shadow-price protocol prices replacements with the fixed heuristic
|
||||
// so the O(1) projection fold stays in agreement with its own appends;
|
||||
// retention, range selection, and the shrink comparison read the
|
||||
// route-priced `tokens` instead.
|
||||
shadowedTokenCount: selectedNodes.reduce((total, node) => total + node.heuristicTokens, 0),
|
||||
shadowedRouteTokenCount: selectedNodes.reduce((total, node) => total + node.tokens, 0),
|
||||
input: buildSummarizationInput(session, selection.shadowedSeqs),
|
||||
}
|
||||
}
|
||||
@@ -370,10 +377,13 @@ async function summarizeCompaction(
|
||||
content: frameSummary(summaryResult.summary),
|
||||
source: compactCheckpointSource(compactionId, sourceCommandId),
|
||||
})
|
||||
// The checkpoint is text-only, so its fixed-heuristic price IS its route
|
||||
// price; comparing it against the span's route price asks the real
|
||||
// question — does the replacement lower the next request's pressure.
|
||||
const framedSummaryTokenCount = dependencies.meter.estimateMessage(checkpointMessage)
|
||||
if (framedSummaryTokenCount >= prepared.shadowedTokenCount) {
|
||||
if (framedSummaryTokenCount >= prepared.shadowedRouteTokenCount) {
|
||||
throw new Error(
|
||||
`summary is not smaller than the shadowed content (${framedSummaryTokenCount} estimated framed tokens >= ${prepared.shadowedTokenCount})`,
|
||||
`summary is not smaller than the shadowed content (${framedSummaryTokenCount} estimated framed tokens >= ${prepared.shadowedRouteTokenCount})`,
|
||||
)
|
||||
}
|
||||
return {
|
||||
|
||||
@@ -4,6 +4,7 @@ import { AttachmentId } from '@deepseek-ai/dsh-attachment'
|
||||
import BasicCompactionEngine from '@deepseek-ai/dsh-compaction-basic'
|
||||
import type { BasicCompactionConfig } from '@deepseek-ai/dsh-compaction-basic'
|
||||
import { selectCompactableRange } from '@deepseek-ai/dsh-compaction-basic/src/region.ts'
|
||||
import { frameSummary } from '@deepseek-ai/dsh-compaction-basic/src/summarizer.ts'
|
||||
import type { SummarizationInput, SummaryResult } from '@deepseek-ai/dsh-compaction-basic/src/summarizer.ts'
|
||||
import { CompactionId, toolPairingBalancedAfter, toolPairingBalancedBefore } from '@deepseek-ai/dsh-compaction'
|
||||
import {
|
||||
@@ -1878,3 +1879,146 @@ describe('automatic listener and loader composition', () => {
|
||||
expect(await recover(ctx, agent(session, MODEL), overflow())).toBe(false)
|
||||
})
|
||||
})
|
||||
|
||||
describe('route-priced image pressure', () => {
|
||||
const IMAGE_VISUAL_TOKENS = 300
|
||||
const IMAGE_HANDLE_TEXT = 'request preview'
|
||||
|
||||
class PricedContextAdapter extends ContextAdapter {
|
||||
override imageRequestPricing(): { priceImages: (images: readonly unknown[]) => Array<{ visualTokens: number; text: string }> } {
|
||||
return {
|
||||
priceImages: images => images.map(() => ({
|
||||
visualTokens: IMAGE_VISUAL_TOKENS,
|
||||
text: IMAGE_HANDLE_TEXT,
|
||||
})),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function pricedContext(contextWindow = 1_000): Context {
|
||||
const ctx = new Context()
|
||||
void new LlmRuntime(ctx)
|
||||
void new TokenMeter(ctx)
|
||||
ctx.llm.registerAdapter([MODEL], new PricedContextAdapter(contextWindow))
|
||||
return ctx
|
||||
}
|
||||
|
||||
/** Closed short-text turns whose user messages each carry one image. */
|
||||
function imageConversation(turns = 4): Session {
|
||||
const session = Session.create(SessionId(`image-dense-${turns}`))
|
||||
for (let turn = 1; turn <= turns; turn += 1) {
|
||||
session.append('turn/start', { turn })
|
||||
session.append('user/message', createUserMessage({
|
||||
content: [
|
||||
{ type: 'text', text: `image turn ${turn}` },
|
||||
{
|
||||
type: 'image',
|
||||
attachment: {
|
||||
attachmentId: AttachmentId(`sha256:${String(turn).repeat(8)}`),
|
||||
mediaType: 'image/png',
|
||||
bytes: 2048,
|
||||
width: 800,
|
||||
height: 800,
|
||||
name: `shot-${turn}`,
|
||||
},
|
||||
},
|
||||
],
|
||||
source: { kind: 'user' },
|
||||
}), { surfaceOp: 'append' })
|
||||
session.append('step/start', { turn, step: 1 })
|
||||
if (turn === 1) {
|
||||
session.append('request/header', {
|
||||
header: { config: { provider: MODEL, model: MODEL } },
|
||||
reason: 'initial',
|
||||
})
|
||||
}
|
||||
session.append('assistant/message', {
|
||||
turn,
|
||||
step: 1,
|
||||
message: createMessage({
|
||||
role: 'assistant',
|
||||
content: [{ type: 'text', text: `ok ${turn}` }],
|
||||
source: {
|
||||
kind: 'model',
|
||||
...{ provider: MODEL, model: MODEL },
|
||||
},
|
||||
}),
|
||||
}, { surfaceOp: 'append' })
|
||||
session.append('step/end', { turn, step: 1 })
|
||||
session.append('turn/end', { turn, reason: { kind: 'completed' } })
|
||||
}
|
||||
session.append('turn/start', { turn: turns + 1 })
|
||||
return session
|
||||
}
|
||||
|
||||
it('selects an image-dense range only when the routed price counts visual tokens', () => {
|
||||
const session = imageConversation()
|
||||
const routed = pricedContext().tokenMeter.measure(session)
|
||||
const neutral = createContext().tokenMeter.measure(session)
|
||||
|
||||
expect(routed.surfaceTokens).toBeGreaterThan(neutral.surfaceTokens + 4 * IMAGE_VISUAL_TOKENS - 200)
|
||||
expect(routed.nodes.map(node => node.seq)).toEqual(neutral.nodes.map(node => node.seq))
|
||||
expect(routed.nodes.map(node => node.heuristicTokens)).toEqual(neutral.nodes.map(node => node.tokens))
|
||||
|
||||
// The same verbatim tail budget retains almost everything under the
|
||||
// neutral heuristic but forces a cut once visual tokens are counted.
|
||||
expect(selectCompactableRange(session, neutral, 350)).toBeNull()
|
||||
const range = selectCompactableRange(session, routed, 350)
|
||||
expect(range).not.toBeNull()
|
||||
})
|
||||
|
||||
it('accepts a summary larger than the span heuristic when the route price shrinks', async () => {
|
||||
// A single short image message prices below a framed summary under the
|
||||
// fixed heuristic but far above it under the route: the shrink comparison
|
||||
// must ask whether the replacement lowers route pressure.
|
||||
const ctx = pricedContext(1_000)
|
||||
const session = imageConversation(1)
|
||||
const before = ctx.tokenMeter.measure(session)
|
||||
const imageNode = before.nodes[0]!
|
||||
const compact = new TestCompactionEngine(ctx, { auto: false })
|
||||
compact.summary = [{
|
||||
type: 'text',
|
||||
text: 'summary text sized between the heuristic and route prices of the shadowed image message, '
|
||||
+ 'long enough that the fixed heuristic alone would reject it as not smaller '
|
||||
+ 'while the route-priced comparison accepts the pressure reduction.',
|
||||
}]
|
||||
const framed = ctx.tokenMeter.estimateMessage(createUserMessage({
|
||||
content: frameSummary(compact.summary),
|
||||
source: { kind: 'plugin', plugin: 'test' },
|
||||
}))
|
||||
expect(framed).toBeGreaterThan(imageNode.heuristicTokens)
|
||||
expect(framed).toBeLessThan(imageNode.tokens)
|
||||
|
||||
const result = await compact.compactRegion(imageNode.seq, imageNode.seq, agent(session), SIGNAL)
|
||||
expect(result.shadowedSeqs).toEqual([imageNode.seq])
|
||||
expect(result.shadowedTokenCount).toBe(imageNode.heuristicTokens)
|
||||
})
|
||||
|
||||
it('triggers pressure compaction from routed visual tokens and logs heuristic shadow prices', async () => {
|
||||
const ctx = pricedContext(1_000)
|
||||
const session = imageConversation()
|
||||
const before = ctx.tokenMeter.measure(session)
|
||||
const compact = new TestCompactionEngine(ctx, {
|
||||
auto: false,
|
||||
thresholdRatio: 0.8,
|
||||
retainTokens: 350,
|
||||
})
|
||||
|
||||
// The same history stays below the 800-token threshold without pricing.
|
||||
const neutralResult = await compactIfNeeded(service({
|
||||
auto: false,
|
||||
thresholdRatio: 0.8,
|
||||
retainTokens: 350,
|
||||
}), session)
|
||||
expect(neutralResult).toBeNull()
|
||||
|
||||
const result = await compact.compactIfNeeded(agent(session), 'pressure', SIGNAL)
|
||||
expect(result).not.toBeNull()
|
||||
const summaryEvent = session.events.find(event => event.type === 'compaction/summary')
|
||||
expect(summaryEvent).toBeDefined()
|
||||
const shadowedHeuristic = before.nodes
|
||||
.filter(node => result?.shadowedSeqs.includes(node.seq))
|
||||
.reduce((total, node) => total + node.heuristicTokens, 0)
|
||||
expect(summaryEvent?.data.shadowedTokenCount).toBe(shadowedHeuristic)
|
||||
})
|
||||
})
|
||||
|
||||
@@ -1033,6 +1033,12 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [
|
||||
parameters: [{ name: 'provider', description: 'registered provider route to inspect.' }],
|
||||
returns: 'the provider-owned policy, with normal defaults already resolved.',
|
||||
},
|
||||
{
|
||||
signature: 'imageRequestPricing(provider: string, model: string): LlmImageRequestPricing | undefined',
|
||||
description: 'Resolve provider-side request-image pricing for one exact route, or `undefined` when the provider is unregistered or declares none. Unknown providers degrade to `undefined` rather than throwing because callers price durable history whose route may no longer be mounted.',
|
||||
parameters: [{ name: 'provider', description: 'provider route named by a request header.' }, { name: 'model', description: 'exact model id named by the same header.' }],
|
||||
returns: 'the owning adapter\'s image pricing for the route, when declared.',
|
||||
},
|
||||
{
|
||||
signature: 'async listModels(provider: string): Promise<LlmModelInfo[]>',
|
||||
description: 'Discover models advertised by one registered provider. Catalog membership is advisory and never changes routing or request validation.',
|
||||
@@ -2214,7 +2220,7 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [
|
||||
methods: [
|
||||
{
|
||||
signature: 'measure(session: Session, requestHeader?: EpochHeader): TokenMeasurement',
|
||||
description: 'Measure current request pressure and surface through the durable tail.\n\nProvider usage is reused only when the latest successful call\'s canonical request envelope matches `requestHeader` and its total is no lower than that call\'s full heuristic anchor; otherwise the complete envelope and surface are heuristically repriced.\n\n`requestHeader` affects request pressure only; surface fields always describe the current session surface. Every call clones those positional nodes, so measurement is O(surface).',
|
||||
description: 'Measure current request pressure and surface through the durable tail.\n\nThe effective envelope\'s routed provider/model selects the request-image pricing every node is priced under: a route whose adapter declares image pricing charges each retained image its visual tokens plus its model-visible text, while other routes keep the fixed heuristic. Provider usage is reused only when the latest successful call\'s canonical request envelope matches `requestHeader` and its total is no lower than that call\'s full route-priced anchor; otherwise the complete envelope and surface are repriced.\n\n`requestHeader` replaces the latest logged envelope for pressure and node pricing; the node set always describes the current session surface. Every call clones those positional nodes, so measurement is O(surface).',
|
||||
parameters: [{ name: 'session', description: 'session to replay through its current durable tail.' }, { name: 'requestHeader', description: 'optional effective request envelope replacing the latest logged header.' }],
|
||||
returns: 'a detached deeply immutable pressure and surface measurement.',
|
||||
},
|
||||
@@ -3915,7 +3921,7 @@ export const TYPE_API: readonly TypeApiEntry[] = [
|
||||
},
|
||||
{
|
||||
name: 'LlmAdapter',
|
||||
declaration: 'export abstract class LlmAdapter {\n providerInfo(provider: string): LlmProviderInfo;\n providerRetryPolicy(_provider: string): ResolvedRetryPolicy | undefined;\n listModels(_provider: string): Promise<readonly LlmModelInfo[]>;\n resolveModel(provider: string, model: string, _signal?: AbortSignal): Promise<LlmResolvedModelInfo>;\n async prepareCall(provider: string, model: string, signal?: AbortSignal): Promise<PreparedAdapterCall>;\n abstract stream(options: GenerateOptions): AsyncIterable<StreamChunk>;\n}',
|
||||
declaration: 'export abstract class LlmAdapter {\n providerInfo(provider: string): LlmProviderInfo;\n providerRetryPolicy(_provider: string): ResolvedRetryPolicy | undefined;\n imageRequestPricing(_provider: string, _model: string): LlmImageRequestPricing | undefined;\n listModels(_provider: string): Promise<readonly LlmModelInfo[]>;\n resolveModel(provider: string, model: string, _signal?: AbortSignal): Promise<LlmResolvedModelInfo>;\n async prepareCall(provider: string, model: string, signal?: AbortSignal): Promise<PreparedAdapterCall>;\n abstract stream(options: GenerateOptions): AsyncIterable<StreamChunk>;\n}',
|
||||
},
|
||||
{
|
||||
name: 'LlmCallConfig',
|
||||
@@ -3937,6 +3943,14 @@ export const TYPE_API: readonly TypeApiEntry[] = [
|
||||
name: 'LlmFailure',
|
||||
declaration: 'export interface LlmFailure {\n readonly message: string;\n readonly code: string;\n readonly status?: number;\n readonly providerRetryAfterMs?: number;\n readonly requestId?: ProviderRequestId;\n}',
|
||||
},
|
||||
{
|
||||
name: 'LlmImageRequestPrice',
|
||||
declaration: 'export interface LlmImageRequestPrice {\n visualTokens: number;\n text: string;\n}',
|
||||
},
|
||||
{
|
||||
name: 'LlmImageRequestPricing',
|
||||
declaration: 'export interface LlmImageRequestPricing {\n priceImages(images: readonly ImageAttachmentRef[]): readonly LlmImageRequestPrice[];\n}',
|
||||
},
|
||||
{
|
||||
name: 'LlmModelContext',
|
||||
declaration: 'export interface LlmModelContext {\n contextWindow: number;\n}',
|
||||
@@ -3967,7 +3981,7 @@ export const TYPE_API: readonly TypeApiEntry[] = [
|
||||
},
|
||||
{
|
||||
name: 'LlmRuntime',
|
||||
declaration: 'export class LlmRuntime extends Service {\n constructor(ctx: Context);\n registerAdapter(providers: string[], adapter: LlmAdapter): AdapterRegistrationHandle;\n listProviders(): LlmProviderInfo[];\n registerConfigurableProviders(entries: readonly LlmConfigurableProvider[]): DirectoryRegistrationHandle;\n listConfigurableProviders(): LlmConfigurableProvider[];\n registerModelDiscovery(settingsNs: string, discover: (request: LlmModelDiscoveryRequest) => Promise<readonly LlmDiscoveredModel[]>): () => void;\n async discoverModels(settingsNs: string, request: LlmModelDiscoveryRequest): Promise<LlmDiscoveredModel[]>;\n providerRetryPolicy(provider: string): ResolvedRetryPolicy;\n async listModels(provider: string): Promise<LlmModelInfo[]>;\n async resolveModelInfo(provider: string, model: string, signal?: AbortSignal): Promise<LlmResolvedModelInfo>;\n async resolveCallConfig(config: LlmCallConfig, signal?: AbortSignal): Promise<LlmCallConfig>;\n async prepareCall(config: LlmCallConfig, signal?: AbortSignal): Promise<PreparedLlmCall>;\n stream(options: GenerateOptions): AsyncIterable<StreamChunk>;\n}',
|
||||
declaration: 'export class LlmRuntime extends Service {\n constructor(ctx: Context);\n registerAdapter(providers: string[], adapter: LlmAdapter): AdapterRegistrationHandle;\n listProviders(): LlmProviderInfo[];\n registerConfigurableProviders(entries: readonly LlmConfigurableProvider[]): DirectoryRegistrationHandle;\n listConfigurableProviders(): LlmConfigurableProvider[];\n registerModelDiscovery(settingsNs: string, discover: (request: LlmModelDiscoveryRequest) => Promise<readonly LlmDiscoveredModel[]>): () => void;\n async discoverModels(settingsNs: string, request: LlmModelDiscoveryRequest): Promise<LlmDiscoveredModel[]>;\n providerRetryPolicy(provider: string): ResolvedRetryPolicy;\n imageRequestPricing(provider: string, model: string): LlmImageRequestPricing | undefined;\n async listModels(provider: string): Promise<LlmModelInfo[]>;\n async resolveModelInfo(provider: string, model: string, signal?: AbortSignal): Promise<LlmResolvedModelInfo>;\n async resolveCallConfig(config: LlmCallConfig, signal?: AbortSignal): Promise<LlmCallConfig>;\n async prepareCall(config: LlmCallConfig, signal?: AbortSignal): Promise<PreparedLlmCall>;\n stream(options: GenerateOptions): AsyncIterable<StreamChunk>;\n}',
|
||||
},
|
||||
{
|
||||
name: 'LspHover',
|
||||
@@ -5239,7 +5253,7 @@ export const TYPE_API: readonly TypeApiEntry[] = [
|
||||
},
|
||||
{
|
||||
name: 'TokenSurfaceNode',
|
||||
declaration: 'export interface TokenSurfaceNode {\n readonly seq: number;\n readonly tokens: number;\n}',
|
||||
declaration: 'export interface TokenSurfaceNode {\n readonly seq: number;\n readonly tokens: number;\n readonly heuristicTokens: number;\n}',
|
||||
},
|
||||
{
|
||||
name: 'TokenUsage',
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/llm/llm-deepseek/README.md
|
||||
README.md: 11ee4c775c6565e0842707928683587a1e2f1eb8
|
||||
README.zh.md: 86da6c75891d7e458b870b630db877c799c33127
|
||||
README.md: 5fe9302ce851f35d0a90a9af4cf6e121dd680ab9
|
||||
README.zh.md: 1761afdd62b85b0c20e24199a6573a8a0b2afe5e
|
||||
|
||||
@@ -119,7 +119,7 @@ The selected DeepSeek model receives the harness system prompt, message history,
|
||||
|
||||
#### Token effect
|
||||
|
||||
Provider tokenization governs exact text and image-token input. Reasoning passback carries every reasoned turn's chain of thought into later requests, while dropping over-budget images avoids paying those tokens again; cache-read usage is reported when available.
|
||||
Provider tokenization governs exact text and image-token input. The adapter additionally declares per-route request-image pricing (`imageRequestPricing`): it reproduces the request projection's oldest-first offload from durable byte lengths and prices each retained image with the published v4 vision accounting (14px patch grid, 3:1 downsampling, 384-token cap, worst-case alignment pad) at its projected request dimensions, so the token meter can price image pressure before a request is sent; reported usage remains authoritative. Reasoning passback carries every reasoned turn's chain of thought into later requests, while dropping over-budget images avoids paying those tokens again; cache-read usage is reported when available.
|
||||
|
||||
#### KV Cache effect
|
||||
|
||||
|
||||
@@ -119,7 +119,7 @@ DeepSeek 请求身份独立于应用归因。凭据解析成功后，每个提
|
||||
|
||||
#### Token 影响
|
||||
|
||||
精确文本与图片 token 输入取决于提供方 tokenization。推理回传会把每个含推理轮次的思维链带入后续请求，丢弃超出上限的图片则避免再次支付这些 token；可用时会报告 cache-read 用量。
|
||||
精确文本与图片 token 输入取决于提供方 tokenization。适配器另外声明按路由的请求图片定价（`imageRequestPricing`）：它根据持久字节长度复现请求投影的最旧优先 offload，并按投影后的请求尺寸用官方公布的 v4 视觉计量（14px patch 网格、3:1 降采样、单图 384 token 上限、最坏对齐 pad）为每张保留图片计价，使 token 计量服务能在请求发出前为图片压力定价；上报的 usage 仍是权威值。推理回传会把每个含推理轮次的思维链带入后续请求，丢弃超出上限的图片则避免再次支付这些 token；可用时会报告 cache-read 用量。
|
||||
|
||||
#### KV Cache 影响
|
||||
|
||||
|
||||
@@ -25,7 +25,6 @@ import type {
|
||||
AttachmentId,
|
||||
AttachmentStore,
|
||||
ImageAttachmentRef,
|
||||
ImageRequestPolicy,
|
||||
RequestImageAttachment,
|
||||
} from '@deepseek-ai/dsh-attachment'
|
||||
import type { CredentialRef } from '@deepseek-ai/dsh-credentials'
|
||||
@@ -38,6 +37,7 @@ import type {
|
||||
} from '@deepseek-ai/dsh-deepseek-llm-api-extensions'
|
||||
import { serializeRequest, serializeRequestWithImages } from './serialize.ts'
|
||||
import type { ImageWireLocation, RequestDefaults } from './serialize.ts'
|
||||
import { deepSeekImageRequestPricing, resolveRequestImagePolicy } from './request-pricing.ts'
|
||||
import { DeepSeekFileStore } from './file-store.ts'
|
||||
import type { DeepSeekFilePolicy } from './file-store.ts'
|
||||
import type { DeepSeekFileId } from './file-id.ts'
|
||||
@@ -140,18 +140,8 @@ export const DEFAULT_STREAM_IDLE_TIMEOUT_MS = 300_000
|
||||
export const DEFAULT_CONTEXT_WINDOW = 1_000_000
|
||||
/** Default per-request output-token cap. */
|
||||
export const DEFAULT_MAX_TOKENS = 256_000
|
||||
/** Default bound on accumulated file-referenced image bytes per request. */
|
||||
export const DEFAULT_MAX_REQUEST_FILES_BYTES = 128 * 1024 * 1024
|
||||
/** Default bound on accumulated base64 image payload after Files API fallback. */
|
||||
export const DEFAULT_MAX_INLINE_REQUEST_IMAGE_BYTES = 20 * 1024 * 1024
|
||||
/** Provider request image-count limit. */
|
||||
export const DEFAULT_MAX_IMAGES_PER_REQUEST = 600
|
||||
/** Total-pixel budget matching DeepSeek's normal vision projection. */
|
||||
export const DEFAULT_REQUEST_IMAGE_PIXEL_BUDGET = 640_000
|
||||
/** Total-pixel budget matching provider low-detail image input. */
|
||||
export const DEFAULT_LOW_DETAIL_IMAGE_PIXEL_BUDGET = 512 * 512
|
||||
/** Encoded-byte target for one deterministic model-request image; the smallest quality-ladder output is used when no quality fits. */
|
||||
export const DEFAULT_REQUEST_IMAGE_MAX_BYTES = 1024 * 1024
|
||||
/** Deterministic raw-byte removal step. */
|
||||
export const DEFAULT_IMAGE_OFFLOAD_BYTE_QUANTUM = 64 * 1024 * 1024
|
||||
/** Deterministic base64-byte removal step after Files API fallback. */
|
||||
@@ -220,24 +210,6 @@ function collectImageRefs(
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve the request-image budgets owned by one DeepSeek model route.
|
||||
* @param model - Advertised model route and its optional image overrides.
|
||||
* @returns Complete pixel and encoded-byte budgets.
|
||||
* @internal
|
||||
*/
|
||||
export function resolveRequestImagePolicy(model: DeepSeekCatalogModel): ImageRequestPolicy {
|
||||
const maxPixels = model.imagePixelBudget === 'low'
|
||||
? DEFAULT_LOW_DETAIL_IMAGE_PIXEL_BUDGET
|
||||
: model.imagePixelBudget ?? DEFAULT_REQUEST_IMAGE_PIXEL_BUDGET
|
||||
return {
|
||||
maxPixels,
|
||||
maxBytes: model.imageMaxBytes === undefined
|
||||
? DEFAULT_REQUEST_IMAGE_MAX_BYTES
|
||||
: model.imageMaxBytes,
|
||||
}
|
||||
}
|
||||
|
||||
async function prepareRequestImages(
|
||||
options: GenerateOptions,
|
||||
attachments: AttachmentStore,
|
||||
@@ -394,6 +366,18 @@ export class DeepSeekAdapter extends LlmAdapter {
|
||||
return this.config.options().retryPolicy
|
||||
}
|
||||
|
||||
override imageRequestPricing(_provider: string, model: string): ReturnType<LlmAdapter['imageRequestPricing']> {
|
||||
// The same access resolution the serializer uses, so priced handle and
|
||||
// placeholder text matches what the request actually sends.
|
||||
const attachments = this.config.resolveAttachments?.()
|
||||
const resolveAccess = attachments === undefined
|
||||
? undefined
|
||||
: (ref: ImageAttachmentRef): ImageAttachmentAccess | undefined => (
|
||||
this.config.resolveImageAccess?.(attachments, ref)
|
||||
)
|
||||
return deepSeekImageRequestPricing(this.config.options(), model, resolveAccess)
|
||||
}
|
||||
|
||||
override listModels(provider: string): Promise<readonly LlmModelInfo[]> {
|
||||
return Promise.resolve(this.config.options().models.map(model => modelInfo(provider, model)))
|
||||
}
|
||||
|
||||
@@ -0,0 +1,154 @@
|
||||
/**
|
||||
* DeepSeek v4 vision-token accounting: the provider's published image-token
|
||||
* calculator (api-docs.deepseek.com, Token & Token Usage) ported verbatim.
|
||||
* The provider resizes every request image onto a 14px-patch grid, downsamples
|
||||
* 3:1 per axis, and caps one image at 384 tokens; the port prices the
|
||||
* pad-to-4 alignment at its 3-token upper bound because request pricing has
|
||||
* no preceding-token position. Actual usage remains authoritative.
|
||||
*
|
||||
* @module dsh-llm-deepseek/image-tokens
|
||||
*/
|
||||
|
||||
/** Vision patch edge in pixels. */
|
||||
const PATCH_SIZE = 14
|
||||
/** Per-axis patch-to-token downsampling ratio. */
|
||||
const DOWNSAMPLE_RATIO = 3
|
||||
/** Provider cap on tokens for one request image. */
|
||||
const MAX_IMAGE_TOKENS = 384
|
||||
/** Token-alignment quantum; pricing charges its worst-case `QUANTUM - 1` pad. */
|
||||
const COMPRESS_PAD_TO = 4
|
||||
/** Width is clamped to this multiple of height before grid projection. */
|
||||
const MAX_WIDTH_HEIGHT_RATIO = 8
|
||||
/** Total-pixel floor; smaller images are scaled up before grid projection. */
|
||||
const MIN_PIXELS = 384 * 384
|
||||
|
||||
const intDiv = (value: number, divisor: number): number => Math.floor(value / divisor)
|
||||
const ceilDiv = (value: number, divisor: number): number => Math.floor((value + divisor - 1) / divisor)
|
||||
|
||||
interface GridResize {
|
||||
readonly gridHeight: number
|
||||
readonly gridWidth: number
|
||||
readonly bestHeight: number
|
||||
readonly bestWidth: number
|
||||
readonly numTokens: number
|
||||
}
|
||||
|
||||
/** Token count of one grid, including row separators and framing. */
|
||||
function gridTokens(gridHeight: number, gridWidth: number): number {
|
||||
let tokens = gridHeight * (gridWidth + 1) + 2
|
||||
if (gridHeight % 2 === 1) tokens += gridWidth + 1
|
||||
tokens += (ceilDiv(gridHeight, 2) * (gridWidth + 1) % 2) * 2
|
||||
return tokens
|
||||
}
|
||||
|
||||
/** Solve the largest grid within `budget` tokens preserving the aspect ratio. */
|
||||
function solveResizeRatio(height: number, width: number, budget: number): GridResize {
|
||||
const aspect = height / width
|
||||
const idealGridWidth = Math.sqrt((budget - 2) / aspect + 0.25) - 0.5
|
||||
const idealGridHeight = idealGridWidth * aspect
|
||||
let bestHeight: number
|
||||
let bestWidth: number
|
||||
if (idealGridWidth < 1) {
|
||||
const solvedGridWidth = 1
|
||||
let solvedGridHeight = intDiv(budget - 2, solvedGridWidth + 1)
|
||||
// v8 ignore: at the provider budget the one-column solve always lands on
|
||||
// the odd 189-row grid, so the even path is unreachable; kept for parity
|
||||
// with the published solver.
|
||||
/* v8 ignore next */
|
||||
if (solvedGridHeight % 2 === 1) solvedGridHeight -= 1
|
||||
bestWidth = solvedGridWidth * PATCH_SIZE * DOWNSAMPLE_RATIO
|
||||
bestHeight = solvedGridHeight * PATCH_SIZE * DOWNSAMPLE_RATIO
|
||||
/* v8 ignore start -- unreachable at the provider budget: idealGridWidth >= 1
|
||||
bounds the aspect at (budget - 2) / 2, making idealGridHeight >= 2 for
|
||||
every budget this module solves; kept for parity with the published
|
||||
solver. */
|
||||
} else if (idealGridHeight < 2) {
|
||||
const solvedGridHeight = 2
|
||||
const solvedGridWidth = intDiv(budget - 2, solvedGridHeight) - 1
|
||||
if (!(solvedGridWidth > 1)) throw new Error('deepseek image tokens: no grid fits the token budget')
|
||||
bestWidth = solvedGridWidth * PATCH_SIZE * DOWNSAMPLE_RATIO
|
||||
bestHeight = solvedGridHeight * PATCH_SIZE * DOWNSAMPLE_RATIO
|
||||
/* v8 ignore stop */
|
||||
} else {
|
||||
const solvedGridWidth = Math.trunc(idealGridWidth)
|
||||
let solvedGridHeight = Math.trunc(idealGridHeight)
|
||||
if (solvedGridHeight % 2 === 1) solvedGridHeight -= 1
|
||||
const widthScale = solvedGridWidth * PATCH_SIZE * DOWNSAMPLE_RATIO / width
|
||||
const heightScale = solvedGridHeight * PATCH_SIZE * DOWNSAMPLE_RATIO / height
|
||||
const scale = Math.min(widthScale, heightScale)
|
||||
bestWidth = Math.trunc(width * scale / PATCH_SIZE) * PATCH_SIZE
|
||||
bestHeight = Math.trunc(height * scale / PATCH_SIZE) * PATCH_SIZE
|
||||
}
|
||||
const gridHeight = ceilDiv(intDiv(bestHeight, PATCH_SIZE), DOWNSAMPLE_RATIO)
|
||||
const gridWidth = ceilDiv(intDiv(bestWidth, PATCH_SIZE), DOWNSAMPLE_RATIO)
|
||||
return { gridHeight, gridWidth, bestHeight, bestWidth, numTokens: gridTokens(gridHeight, gridWidth) }
|
||||
}
|
||||
|
||||
/** Project padded pixel dimensions onto the largest in-budget token grid. */
|
||||
function safeResize(height: number, width: number, paddedHeight: number, paddedWidth: number): GridResize {
|
||||
const gridHeight = ceilDiv(intDiv(paddedHeight, PATCH_SIZE), DOWNSAMPLE_RATIO)
|
||||
const gridWidth = ceilDiv(intDiv(paddedWidth, PATCH_SIZE), DOWNSAMPLE_RATIO)
|
||||
const pad = COMPRESS_PAD_TO - 1
|
||||
const budget = MAX_IMAGE_TOKENS - pad
|
||||
let result: GridResize = {
|
||||
gridHeight,
|
||||
gridWidth,
|
||||
bestHeight: paddedHeight,
|
||||
bestWidth: paddedWidth,
|
||||
numTokens: gridTokens(gridHeight, gridWidth),
|
||||
}
|
||||
if (result.numTokens > budget) {
|
||||
result = solveResizeRatio(height, width, budget)
|
||||
/* v8 ignore next 4 -- the published solver's safety net; the closed-form
|
||||
solve stays within budget for every geometry the clamps admit. */
|
||||
for (let reduced = budget; result.numTokens > budget; reduced -= 1) {
|
||||
result = solveResizeRatio(height, width, reduced)
|
||||
}
|
||||
}
|
||||
return { ...result, numTokens: result.numTokens + pad }
|
||||
}
|
||||
|
||||
/** One clamp-scale-pad-project pass; the caller iterates it to a fixpoint. */
|
||||
function resizeOnce(width: number, height: number): GridResize {
|
||||
let clampedWidth = width
|
||||
let clampedHeight = height
|
||||
if (clampedWidth > clampedHeight * MAX_WIDTH_HEIGHT_RATIO) {
|
||||
clampedWidth = clampedHeight * MAX_WIDTH_HEIGHT_RATIO
|
||||
}
|
||||
const pixels = clampedWidth * clampedHeight
|
||||
if (pixels < MIN_PIXELS && pixels > 0) {
|
||||
const scale = Math.sqrt(MIN_PIXELS / pixels)
|
||||
clampedWidth = Math.trunc(clampedWidth * scale)
|
||||
clampedHeight = Math.trunc(clampedHeight * scale)
|
||||
}
|
||||
const paddedWidth = ceilDiv(clampedWidth, PATCH_SIZE) * PATCH_SIZE
|
||||
const paddedHeight = ceilDiv(clampedHeight, PATCH_SIZE) * PATCH_SIZE
|
||||
return safeResize(clampedHeight, clampedWidth, paddedHeight, paddedWidth)
|
||||
}
|
||||
|
||||
function sameResize(a: GridResize, b: GridResize): boolean {
|
||||
return a.gridHeight === b.gridHeight
|
||||
&& a.gridWidth === b.gridWidth
|
||||
&& a.bestHeight === b.bestHeight
|
||||
&& a.bestWidth === b.bestWidth
|
||||
&& a.numTokens === b.numTokens
|
||||
}
|
||||
|
||||
/**
|
||||
* Vision tokens DeepSeek v4 charges for one request image of the given
|
||||
* dimensions, at the worst-case alignment pad.
|
||||
* @param width - positive integer request-image width in pixels.
|
||||
* @param height - positive integer request-image height in pixels.
|
||||
* @returns the provider vision-token price, at most 384.
|
||||
*/
|
||||
export function deepSeekImageTokens(width: number, height: number): number {
|
||||
let result = resizeOnce(width, height)
|
||||
for (let iteration = 1; iteration < 10; iteration += 1) {
|
||||
const next = resizeOnce(result.bestWidth, result.bestHeight)
|
||||
if (sameResize(next, result)) return result.numTokens
|
||||
result = next
|
||||
}
|
||||
/* v8 ignore next 2 -- the published solver's non-convergence guard; every
|
||||
pass is a projection, so a second identical pass is a fixpoint. */
|
||||
throw new Error(`deepseek image tokens: resize did not converge for ${width}x${height}`)
|
||||
}
|
||||
@@ -30,17 +30,19 @@ import {
|
||||
DEFAULT_IMAGE_OFFLOAD_BYTE_QUANTUM,
|
||||
DEFAULT_IMAGE_OFFLOAD_COUNT_QUANTUM,
|
||||
DEFAULT_INLINE_IMAGE_OFFLOAD_BYTE_QUANTUM,
|
||||
DEFAULT_LOW_DETAIL_IMAGE_PIXEL_BUDGET,
|
||||
DEFAULT_MAX_INLINE_REQUEST_IMAGE_BYTES,
|
||||
DEFAULT_MAX_IMAGES_PER_REQUEST,
|
||||
DEFAULT_MAX_REQUEST_FILES_BYTES,
|
||||
DEFAULT_MAX_TOKENS,
|
||||
DEFAULT_REQUEST_IMAGE_MAX_BYTES,
|
||||
DEFAULT_REQUEST_IMAGE_PIXEL_BUDGET,
|
||||
DEFAULT_STREAM_IDLE_TIMEOUT_MS,
|
||||
DeepSeekAdapter,
|
||||
} from './adapter.ts'
|
||||
import type { DeepSeekCatalogModel, DeepSeekConnectionOptions } from './adapter.ts'
|
||||
import {
|
||||
DEFAULT_LOW_DETAIL_IMAGE_PIXEL_BUDGET,
|
||||
DEFAULT_MAX_IMAGES_PER_REQUEST,
|
||||
DEFAULT_MAX_REQUEST_FILES_BYTES,
|
||||
DEFAULT_REQUEST_IMAGE_MAX_BYTES,
|
||||
DEFAULT_REQUEST_IMAGE_PIXEL_BUDGET,
|
||||
} from './request-pricing.ts'
|
||||
|
||||
export {
|
||||
DEFAULT_CONTEXT_WINDOW,
|
||||
@@ -51,17 +53,22 @@ export {
|
||||
DEFAULT_IMAGE_OFFLOAD_BYTE_QUANTUM,
|
||||
DEFAULT_IMAGE_OFFLOAD_COUNT_QUANTUM,
|
||||
DEFAULT_INLINE_IMAGE_OFFLOAD_BYTE_QUANTUM,
|
||||
DEFAULT_LOW_DETAIL_IMAGE_PIXEL_BUDGET,
|
||||
DEFAULT_MAX_INLINE_REQUEST_IMAGE_BYTES,
|
||||
DEFAULT_MAX_IMAGES_PER_REQUEST,
|
||||
DEFAULT_MAX_REQUEST_FILES_BYTES,
|
||||
DEFAULT_MAX_TOKENS,
|
||||
DEFAULT_REQUEST_IMAGE_MAX_BYTES,
|
||||
DEFAULT_REQUEST_IMAGE_PIXEL_BUDGET,
|
||||
DEFAULT_STREAM_IDLE_TIMEOUT_MS,
|
||||
DeepSeekAdapter,
|
||||
} from './adapter.ts'
|
||||
export type { DeepSeekAdapterOptions, DeepSeekCatalogModel, DeepSeekConnectionOptions } from './adapter.ts'
|
||||
export {
|
||||
DEFAULT_LOW_DETAIL_IMAGE_PIXEL_BUDGET,
|
||||
DEFAULT_MAX_IMAGES_PER_REQUEST,
|
||||
DEFAULT_MAX_REQUEST_FILES_BYTES,
|
||||
DEFAULT_REQUEST_IMAGE_MAX_BYTES,
|
||||
DEFAULT_REQUEST_IMAGE_PIXEL_BUDGET,
|
||||
deepSeekImageRequestPricing,
|
||||
resolveRequestImagePolicy,
|
||||
} from './request-pricing.ts'
|
||||
export { deepSeekImageTokens } from './image-tokens.ts'
|
||||
export { DeepSeekFileStore, MAX_CHAT_IMAGE_BYTES } from './file-store.ts'
|
||||
export type { DeepSeekFileConnection, DeepSeekFilePolicy, DeepSeekFileReference } from './file-store.ts'
|
||||
export { DeepSeekFilesClient, MAX_FILE_EXPIRY_SECONDS, MAX_FILE_UPLOAD_BYTES, MAX_STORED_FILE_BYTES, MAX_STORED_FILE_COUNT, MIN_FILE_EXPIRY_SECONDS } from './files-api.ts'
|
||||
|
||||
@@ -0,0 +1,106 @@
|
||||
/**
|
||||
* Provider-side request-image pricing for DeepSeek routes: reproduces the
|
||||
* adapter's deterministic request projection (per-model pixel budget,
|
||||
* oldest-first offload under the raw-byte and count budgets) and prices every
|
||||
* retained image with the published v4 vision-token accounting. Consumed
|
||||
* synchronously by the token meter through `LlmAdapter.imageRequestPricing`;
|
||||
* provider usage remains the authoritative anchor for completed requests.
|
||||
*
|
||||
* @module dsh-llm-deepseek/request-pricing
|
||||
*/
|
||||
|
||||
import { offloadedImageText, offloadedImagePrefixCount, requestImageHandleText, textOnlyImageText } from '@deepseek-ai/dsh-llm'
|
||||
import type { ImageAttachmentAccessResolver, LlmImageRequestPrice, LlmImageRequestPricing } from '@deepseek-ai/dsh-llm'
|
||||
import { requestImageDimensions } from '@deepseek-ai/dsh-attachment'
|
||||
import type { ImageAttachmentRef, ImageRequestPolicy } from '@deepseek-ai/dsh-attachment'
|
||||
import { deepSeekImageTokens } from './image-tokens.ts'
|
||||
import type { DeepSeekCatalogModel, DeepSeekConnectionOptions } from './adapter.ts'
|
||||
|
||||
/** Default bound on accumulated file-referenced image bytes per request. */
|
||||
export const DEFAULT_MAX_REQUEST_FILES_BYTES = 128 * 1024 * 1024
|
||||
/** Provider request image-count limit. */
|
||||
export const DEFAULT_MAX_IMAGES_PER_REQUEST = 600
|
||||
/** Total-pixel budget matching DeepSeek's normal vision projection. */
|
||||
export const DEFAULT_REQUEST_IMAGE_PIXEL_BUDGET = 640_000
|
||||
/** Total-pixel budget matching provider low-detail image input. */
|
||||
export const DEFAULT_LOW_DETAIL_IMAGE_PIXEL_BUDGET = 512 * 512
|
||||
/** Encoded-byte target for one deterministic model-request image; the smallest quality-ladder output is used when no quality fits. */
|
||||
export const DEFAULT_REQUEST_IMAGE_MAX_BYTES = 1024 * 1024
|
||||
|
||||
/**
|
||||
* Resolve the request-image budgets owned by one DeepSeek model route.
|
||||
* @param model - Advertised model route and its optional image overrides.
|
||||
* @returns Complete pixel and encoded-byte budgets.
|
||||
* @internal
|
||||
*/
|
||||
export function resolveRequestImagePolicy(model: DeepSeekCatalogModel): ImageRequestPolicy {
|
||||
const maxPixels = model.imagePixelBudget === 'low'
|
||||
? DEFAULT_LOW_DETAIL_IMAGE_PIXEL_BUDGET
|
||||
: model.imagePixelBudget ?? DEFAULT_REQUEST_IMAGE_PIXEL_BUDGET
|
||||
return {
|
||||
maxPixels,
|
||||
maxBytes: model.imageMaxBytes === undefined
|
||||
? DEFAULT_REQUEST_IMAGE_MAX_BYTES
|
||||
: model.imageMaxBytes,
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Price one occurrence a text-only route substitutes with deterministic text,
|
||||
* reproducing the `projectImagesForTextModel` substitution `LlmRuntime`
|
||||
* applies before dispatching to a route without the `image` modality.
|
||||
*/
|
||||
function textOnlyPrice(ref: ImageAttachmentRef): LlmImageRequestPrice {
|
||||
return { visualTokens: 0, text: textOnlyImageText(ref) }
|
||||
}
|
||||
|
||||
/**
|
||||
* Build the request-image pricing for one DeepSeek route from a validated
|
||||
* connection snapshot. Uncatalogued and text-only models price every
|
||||
* occurrence as its deterministic text substitution; image-capable models
|
||||
* reproduce the adapter's first-stage oldest-first offload from durable byte
|
||||
* lengths and price retained images by their projected request dimensions,
|
||||
* with each occurrence's handle or placeholder text built through the same
|
||||
* access resolution the serializer uses. The base64 fallback's tighter inline
|
||||
* budget is not reproduced, so a fallback request can only cost less than
|
||||
* this estimate; access paths resolve at pricing time, so a path that changes
|
||||
* before the request only shifts the text price by its own length.
|
||||
* @param connection - validated connection facts of the pricing resolution.
|
||||
* @param model - exact model id named by the request header.
|
||||
* @param resolveAccess - current execution-world access resolution shared with request serialization.
|
||||
* @returns synchronous per-occurrence pricing for the route.
|
||||
*/
|
||||
export function deepSeekImageRequestPricing(
|
||||
connection: DeepSeekConnectionOptions,
|
||||
model: string,
|
||||
resolveAccess?: ImageAttachmentAccessResolver,
|
||||
): LlmImageRequestPricing {
|
||||
const catalogModel = connection.models.find(entry => entry.id === model)
|
||||
if (catalogModel?.inputModalities?.includes('image') !== true) {
|
||||
return { priceImages: images => images.map(textOnlyPrice) }
|
||||
}
|
||||
const policy = resolveRequestImagePolicy(catalogModel)
|
||||
return {
|
||||
priceImages: (images) => {
|
||||
const offloaded = offloadedImagePrefixCount(
|
||||
images.map(ref => Math.min(ref.bytes, policy.maxBytes)),
|
||||
{
|
||||
maxBytes: connection.maxRequestFilesBytes,
|
||||
maxImages: connection.maxImagesPerRequest,
|
||||
byteQuantum: connection.imageOffloadByteQuantum,
|
||||
countQuantum: connection.imageOffloadCountQuantum,
|
||||
},
|
||||
)
|
||||
return images.map((ref, index) => {
|
||||
if (index < offloaded) {
|
||||
return { visualTokens: 0, text: offloadedImageText(ref, resolveAccess?.(ref)) }
|
||||
}
|
||||
const dimensions = requestImageDimensions(ref.width, ref.height, policy.maxPixels)
|
||||
return {
|
||||
visualTokens: deepSeekImageTokens(dimensions.width, dimensions.height),
|
||||
text: requestImageHandleText(ref, dimensions, resolveAccess?.(ref)),
|
||||
}
|
||||
})
|
||||
},
|
||||
}
|
||||
}
|
||||
@@ -21,7 +21,8 @@ import DeepSeekLlmApiExtensionRegistry from '@deepseek-ai/dsh-deepseek-llm-api-e
|
||||
import type { PreparedDeepSeekLlmApiExtensions } from '@deepseek-ai/dsh-deepseek-llm-api-extensions'
|
||||
import * as LlmDeepSeek from '@deepseek-ai/dsh-llm-deepseek'
|
||||
import { DeepSeekAdapter, resolveAdapterOptions } from '@deepseek-ai/dsh-llm-deepseek'
|
||||
import { httpErrorCode, resolveRequestImagePolicy } from '../src/adapter.ts'
|
||||
import { httpErrorCode } from '../src/adapter.ts'
|
||||
import { resolveRequestImagePolicy } from '../src/request-pricing.ts'
|
||||
import { assemble } from './assemble.ts'
|
||||
import { closeMockServers, mockServer, textEvents } from './mock-server.ts'
|
||||
import type { Behavior } from './mock-server.ts'
|
||||
@@ -157,6 +158,33 @@ describe('request image policy', () => {
|
||||
])('resolves route-owned defaults and overrides for %s', (model, expected) => {
|
||||
expect(resolveRequestImagePolicy(model)).toEqual(expected)
|
||||
})
|
||||
|
||||
it('answers image request pricing from the current connection snapshot', () => {
|
||||
const adapter = adapterOf({
|
||||
models: [{ id: 'vision', inputModalities: ['text', 'image'] }],
|
||||
})
|
||||
const priced = adapter.imageRequestPricing('deepseek-official', 'vision')?.priceImages([imageRef])
|
||||
expect(priced).toHaveLength(1)
|
||||
expect(priced?.[0]!.visualTokens).toBeGreaterThan(0)
|
||||
const textOnly = adapter.imageRequestPricing('deepseek-official', 'unlisted')?.priceImages([imageRef])
|
||||
expect(textOnly?.[0]!.visualTokens).toBe(0)
|
||||
})
|
||||
|
||||
it('prices descriptor text through the serializer\'s access resolution', () => {
|
||||
const attachments = {} as AttachmentStore
|
||||
const adapter = new DeepSeekAdapter({
|
||||
options: () => resolveAdapterOptions({ models: [{ id: 'vision', inputModalities: ['text', 'image'] }] }),
|
||||
resolveApiKey: () => Promise.resolve('k'),
|
||||
resolveUserId: () => TEST_USER_ID,
|
||||
resolveAttachments: () => attachments,
|
||||
resolveImageAccess: (store, ref) => (store === attachments && ref === imageRef
|
||||
? { readonlyPath: '/world/img.png' }
|
||||
: undefined),
|
||||
prepareExtensions: noExtensions,
|
||||
})
|
||||
const priced = adapter.imageRequestPricing('deepseek-official', 'vision')?.priceImages([imageRef])
|
||||
expect(priced?.[0]?.text).toContain('/world/img.png')
|
||||
})
|
||||
})
|
||||
|
||||
describe('DeepSeekAdapter against a mock server', () => {
|
||||
|
||||
@@ -0,0 +1,53 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { deepSeekImageTokens } from '../src/image-tokens.ts'
|
||||
|
||||
describe('DeepSeek v4 image tokens', () => {
|
||||
// Reference values from the provider's published image token calculator
|
||||
// (api-docs.deepseek.com, Token & Token Usage), at the worst-case pad.
|
||||
it.each([
|
||||
[100, 100, 117],
|
||||
[384, 384, 117],
|
||||
[640, 480, 209],
|
||||
[800, 800, 349],
|
||||
[1024, 768, 357],
|
||||
[1920, 1080, 369],
|
||||
[2000, 2000, 349],
|
||||
[5000, 5000, 349],
|
||||
[300, 50, 101],
|
||||
])('prices %sx%s as %s tokens', (width, height, expected) => {
|
||||
expect(deepSeekImageTokens(width, height)).toBe(expected)
|
||||
})
|
||||
|
||||
it('caps every image at 384 tokens regardless of source size', () => {
|
||||
for (const [width, height] of [[2000, 2000], [5000, 5000], [8192, 8192], [16, 8192]]) {
|
||||
expect(deepSeekImageTokens(width!, height!)).toBeLessThanOrEqual(384)
|
||||
}
|
||||
})
|
||||
|
||||
it('prices small images at the documented scale-up floor', () => {
|
||||
// Below roughly 384x384 total pixels the provider scales up, so a tiny
|
||||
// square costs the same as a 384x384 one.
|
||||
expect(deepSeekImageTokens(100, 100)).toBe(deepSeekImageTokens(384, 384))
|
||||
})
|
||||
|
||||
it('clamps extreme width by the aspect-ratio bound', () => {
|
||||
// Width beyond 8x height projects onto the same clamped grid.
|
||||
expect(deepSeekImageTokens(9000, 1)).toBe(113)
|
||||
expect(deepSeekImageTokens(8192, 100)).toBe(113)
|
||||
})
|
||||
|
||||
it('solves a one-column grid for an extremely tall image', () => {
|
||||
// Height-dominant aspect drives the solver's single-column branch.
|
||||
expect(deepSeekImageTokens(16, 8192)).toBe(381)
|
||||
expect(deepSeekImageTokens(1, 9000)).toBe(381)
|
||||
})
|
||||
|
||||
it('trims an odd solved grid height to the even row count', () => {
|
||||
expect(deepSeekImageTokens(100, 4036)).toBe(253)
|
||||
})
|
||||
|
||||
it('converges through a second projection pass when the first is not a fixpoint', () => {
|
||||
expect(deepSeekImageTokens(4921, 353)).toBe(289)
|
||||
expect(deepSeekImageTokens(97, 7289)).toBe(245)
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,106 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { offloadedImageText, requestImageHandleText, textOnlyImageText } from '@deepseek-ai/dsh-llm'
|
||||
import { AttachmentId } from '@deepseek-ai/dsh-attachment'
|
||||
import type { ImageAttachmentRef } from '@deepseek-ai/dsh-attachment'
|
||||
import { deepSeekImageRequestPricing } from '../src/request-pricing.ts'
|
||||
import { resolveAdapterOptions } from '../src/index.ts'
|
||||
import type { Config } from '../src/index.ts'
|
||||
|
||||
const VISION_MODEL = {
|
||||
id: 'vision',
|
||||
inputModalities: ['text', 'image'] as Array<'text' | 'image'>,
|
||||
}
|
||||
|
||||
function ref(name: string, width: number, height: number, bytes = 1024): ImageAttachmentRef {
|
||||
return {
|
||||
attachmentId: AttachmentId(`sha256:${name.padEnd(8, '0')}`),
|
||||
mediaType: 'image/png',
|
||||
bytes,
|
||||
width,
|
||||
height,
|
||||
name,
|
||||
}
|
||||
}
|
||||
|
||||
function connection(config: Omit<Config, 'models'> = {}): ReturnType<typeof resolveAdapterOptions> {
|
||||
return resolveAdapterOptions(Object.assign({ models: [VISION_MODEL] }, config))
|
||||
}
|
||||
|
||||
describe('DeepSeek request-image pricing', () => {
|
||||
it('prices an uncatalogued model as its text-only substitution', () => {
|
||||
const image = ref('photo', 1920, 1080)
|
||||
const prices = deepSeekImageRequestPricing(connection(), 'unlisted').priceImages([image])
|
||||
expect(prices).toEqual([{ visualTokens: 0, text: textOnlyImageText(image) }])
|
||||
})
|
||||
|
||||
it('prices a catalogued text-only model as its text-only substitution', () => {
|
||||
const image = ref('photo', 1920, 1080)
|
||||
const options = resolveAdapterOptions({ models: [{ id: 'text-only' }] })
|
||||
const prices = deepSeekImageRequestPricing(options, 'text-only').priceImages([image])
|
||||
expect(prices).toEqual([{ visualTokens: 0, text: textOnlyImageText(image) }])
|
||||
})
|
||||
|
||||
it('prices a retained image by its projected request dimensions plus its handle text', () => {
|
||||
const image = ref('photo', 1920, 1080)
|
||||
const prices = deepSeekImageRequestPricing(connection(), 'vision').priceImages([image])
|
||||
expect(prices).toEqual([{
|
||||
visualTokens: 369,
|
||||
text: requestImageHandleText(image, { width: 1066, height: 600 }),
|
||||
}])
|
||||
})
|
||||
|
||||
it('honors the low-detail pixel budget preset', () => {
|
||||
const image = ref('photo', 4096, 4096)
|
||||
const options = resolveAdapterOptions({
|
||||
models: [{ ...VISION_MODEL, imagePixelBudget: 'low' as const }],
|
||||
})
|
||||
const prices = deepSeekImageRequestPricing(options, 'vision').priceImages([image])
|
||||
expect(prices[0]!.visualTokens).toBe(201)
|
||||
})
|
||||
|
||||
it('builds handle and placeholder text through the supplied access resolution', () => {
|
||||
const access = { readonlyPath: '/world/attachments/photo.png' }
|
||||
const images = [ref('first', 800, 800), ref('second', 800, 800)]
|
||||
const prices = deepSeekImageRequestPricing(
|
||||
connection({ maxImagesPerRequest: 1, imageOffloadCountQuantum: 1 }),
|
||||
'vision',
|
||||
() => access,
|
||||
).priceImages(images)
|
||||
expect(prices[0]).toEqual({ visualTokens: 0, text: offloadedImageText(images[0]!, access) })
|
||||
expect(prices[1]).toEqual({
|
||||
visualTokens: 349,
|
||||
text: requestImageHandleText(images[1]!, { width: 800, height: 800 }, access),
|
||||
})
|
||||
expect(prices[1]?.text).toContain('/world/attachments/photo.png')
|
||||
})
|
||||
|
||||
it('prices count-offloaded oldest occurrences as their placeholder text', () => {
|
||||
const images = [ref('first', 800, 800), ref('second', 800, 800), ref('third', 800, 800)]
|
||||
const prices = deepSeekImageRequestPricing(
|
||||
connection({ maxImagesPerRequest: 2, imageOffloadCountQuantum: 1 }),
|
||||
'vision',
|
||||
).priceImages(images)
|
||||
expect(prices).toEqual([
|
||||
{ visualTokens: 0, text: offloadedImageText(images[0]!) },
|
||||
{ visualTokens: 349, text: requestImageHandleText(images[1]!, { width: 800, height: 800 }) },
|
||||
{ visualTokens: 349, text: requestImageHandleText(images[2]!, { width: 800, height: 800 }) },
|
||||
])
|
||||
})
|
||||
|
||||
it('caps each occurrence at the per-image byte target before the byte budget', () => {
|
||||
// Each 5 MiB source counts as the 1 MiB request target, so a 2 MiB budget
|
||||
// with a one-byte quantum removes exactly the oldest occurrence.
|
||||
const oversized = 5 * 1024 * 1024
|
||||
const images = [
|
||||
ref('first', 800, 800, oversized),
|
||||
ref('second', 800, 800, oversized),
|
||||
ref('third', 800, 800, oversized),
|
||||
]
|
||||
const prices = deepSeekImageRequestPricing(
|
||||
connection({ maxRequestFilesBytes: 2 * 1024 * 1024, imageOffloadByteQuantum: 1 }),
|
||||
'vision',
|
||||
).priceImages(images)
|
||||
expect(prices.map(price => price.visualTokens)).toEqual([0, 349, 349])
|
||||
expect(prices[0]!.text).toBe(offloadedImageText(images[0]!))
|
||||
})
|
||||
})
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/llm/llm/README.md
|
||||
README.md: ef58516790a2723bce34aa1bbae2e1629050f6a5
|
||||
README.zh.md: 8af1240cdb4f3b65f1c0f841ade620c85443129b
|
||||
README.md: 13c42a8ce0e158d721e78fa99f3f2a7b334de764
|
||||
README.zh.md: 803863d7474b034ed326b47ed94b94d89490ad77
|
||||
|
||||
@@ -57,7 +57,7 @@ Exact-model metadata is a separate correctness query, not a catalog decoration o
|
||||
|
||||
Message content is an array of typed blocks: `text`, `reasoning`, `image`, `tool-call`, `tool-result`. An `ImageBlock` carries only a durable `ImageAttachmentRef`; provider bytes and request dimensions are resolved later. The union remains merge-extensible through `ContentBlockMap`, so plugins can add further block types via declaration merging. Assistant messages use a model source carrying the provider and model that produced them plus optional adapter-private replay state. Before dispatch, `LlmRuntime` retains that state only when the historical provider route and target provider route are currently owned by the exact same adapter instance; the adapter then decides whether it can restore or convert the state across models/providers.
|
||||
|
||||
Every dispatch uses the exact model modalities captured with its adapter generation. An image-capable adapter projects durable image references into route-specific request versions. `resolveImageAttachmentAccess()` separately combines an attachment provider's optional host object with a consumer-supplied mapping from that host path into the current tool execution world. The result never enters `RequestImageAttachment` or its `variantId`. A text-only route instead receives deterministic attachment placeholders, including nested tool-result images, without changing append-only session history. `offloadRequestImagesWithPolicy()` provides deterministic oldest-first image removal with raw or base64 accounting and count or byte quanta; adapters supply the exact derived-version byte length and the required per-image placeholder text.
|
||||
Every dispatch uses the exact model modalities captured with its adapter generation. An image-capable adapter projects durable image references into route-specific request versions. `resolveImageAttachmentAccess()` separately combines an attachment provider's optional host object with a consumer-supplied mapping from that host path into the current tool execution world. The result never enters `RequestImageAttachment` or its `variantId`. A text-only route instead receives deterministic attachment placeholders, including nested tool-result images, without changing append-only session history. `offloadRequestImagesWithPolicy()` provides deterministic oldest-first image removal with raw or base64 accounting and count or byte quanta; adapters supply the exact derived-version byte length and the required per-image placeholder text, and the pure `offloadedImagePrefixCount()` exposes the same removal decision so route-owned request pricing reproduces it without building the projection. Adapters whose provider charges visual tokens declare per-route `imageRequestPricing`; `ctx.llm.imageRequestPricing(provider, model)` resolves it synchronously for the token meter.
|
||||
|
||||
Streaming is a raw chunk protocol (`block-start`, `text-delta`, `reasoning-delta`, `tool-call-delta`, `block-end`, `usage`, `finish`). Every adapter outcome reaches consumers as one terminal `finish`; operational failure uses its `error` or `aborted` reason rather than throwing across the stream API. `BlockAssembler` is the single shared implementation that assembles chunks into blocks/messages. A successful `finish` may carry a `ReplayEnvelope` — opaque response-level replay metadata plus optional per-block entries aligned with the emitted block sequence. Assembly makes one keep/drop decision for content and metadata together: a `max-tokens` finish drops tool calls that may have been truncated, and the envelope loses the entry at each dropped position, so stored metadata always describes stored content.
|
||||
|
||||
|
||||
@@ -57,7 +57,7 @@
|
||||
|
||||
消息内容是类型化内容块数组：`text`、`reasoning`、`image`、`tool-call`、`tool-result`。`ImageBlock` 只携带持久 `ImageAttachmentRef`；提供方字节和请求尺寸之后再解析。联合仍从可合并扩展的 `ContentBlockMap` 派生，因此插件可以通过 declaration merging 添加其他块类型。assistant 消息使用模型来源，其中携带生成该消息的提供方和模型，以及可选的适配器私有回放状态。dispatch 前，`LlmRuntime` 只在历史提供方路由与目标提供方路由当前由完全相同的适配器实例拥有时才保留该状态；随后由适配器判定能否在模型或提供方间恢复或转换该状态。
|
||||
|
||||
每次分发都使用随适配器世代捕获的确切模型模态。支持图片的适配器把持久图片引用投影为路由专用请求版本。`resolveImageAttachmentAccess()` 单独组合附件提供方可选的宿主对象，以及消费方给出的宿主路径到当前工具执行环境的映射。解析结果不进入 `RequestImageAttachment` 或其 `variantId`。纯文本路由则收到确定性的附件占位文本，其中也包括嵌套工具结果图片，追加式会话历史不会改变。`offloadRequestImagesWithPolicy()` 提供确定性的从旧到新图片移除，支持按原始字节或 base64 计数，也支持图片数量或字节量步长；适配器提供确切派生版本的字节长度和必填的逐图占位文本。
|
||||
每次分发都使用随适配器世代捕获的确切模型模态。支持图片的适配器把持久图片引用投影为路由专用请求版本。`resolveImageAttachmentAccess()` 单独组合附件提供方可选的宿主对象，以及消费方给出的宿主路径到当前工具执行环境的映射。解析结果不进入 `RequestImageAttachment` 或其 `variantId`。纯文本路由则收到确定性的附件占位文本，其中也包括嵌套工具结果图片，追加式会话历史不会改变。`offloadRequestImagesWithPolicy()` 提供确定性的从旧到新图片移除，支持按原始字节或 base64 计数，也支持图片数量或字节量步长；适配器提供确切派生版本的字节长度和必填的逐图占位文本，纯函数 `offloadedImagePrefixCount()` 公开同一移除决策，使路由所属的请求定价无需构建投影即可复现它。提供方对图片收取视觉 token 的适配器声明按路由的 `imageRequestPricing`；`ctx.llm.imageRequestPricing(provider, model)` 为 token 计量服务同步解析它。
|
||||
|
||||
流式输出是原始分片协议（`block-start`、`text-delta`、`reasoning-delta`、`tool-call-delta`、`block-end`、`usage`、`finish`）。每个适配器结果都以一个终止 `finish` 到达消费方；运行故障使用 `error` 或 `aborted` 作为结束原因，而不会跨流 API 抛出。`BlockAssembler` 是将分片组装为块／消息的唯一共享实现。成功的 `finish` 可以携带 `ReplayEnvelope`——不透明的响应级回放元数据，加上与发射块序列对齐的可选逐块条目。组装对内容与元数据只做一次保留/丢弃决定：`max-tokens` 结束会丢弃可能被截断的工具调用，数据在每个被丢弃的位置同步失去对应条目，因此存储的元数据始终描述存储的内容。
|
||||
|
||||
|
||||
@@ -81,13 +81,13 @@ export function textOnlyImageText(ref: ImageAttachmentRef): string {
|
||||
* attachment id, so one shared version may serve occurrences whose display
|
||||
* names differ.
|
||||
* @param ref - the occurrence's durable normalized attachment.
|
||||
* @param version - exact request image shown beside the text.
|
||||
* @param version - exact request-image dimensions shown beside the text.
|
||||
* @param access - optional path resolved for the current tool execution world.
|
||||
* @returns attachment handle and request-image dimensions.
|
||||
*/
|
||||
export function requestImageHandleText(
|
||||
ref: ImageAttachmentRef,
|
||||
version: RequestImageAttachment,
|
||||
version: Pick<RequestImageAttachment, 'width' | 'height'>,
|
||||
access?: ImageAttachmentAccess,
|
||||
): string {
|
||||
const preview = `Image ${imageIdentity(ref)}; request preview ${version.width}x${version.height}px.`
|
||||
@@ -229,6 +229,39 @@ export function projectImagesForTextModel(messages: readonly Message[]): readonl
|
||||
})
|
||||
}
|
||||
|
||||
/**
|
||||
* Number of oldest image occurrences one request projection removes, in whole
|
||||
* count and byte quanta, once a route budget is exceeded. The result depends
|
||||
* only on the represented lengths, so provider request pricing reproduces the
|
||||
* exact serialization decision without building the projected messages.
|
||||
* @param lengths - represented byte length of every occurrence, in request order.
|
||||
* @param policy - count/byte budgets and removal quanta; unbounded when absent.
|
||||
* @returns how many leading occurrences the projection replaces with placeholders.
|
||||
*/
|
||||
export function offloadedImagePrefixCount(
|
||||
lengths: readonly number[],
|
||||
policy: Pick<RequestImageOffloadPolicy, 'maxImages' | 'maxBytes' | 'countQuantum' | 'byteQuantum'>,
|
||||
): number {
|
||||
const total = lengths.reduce((sum, bytes) => sum + bytes, 0)
|
||||
const excessCount = policy.maxImages === undefined ? 0 : Math.max(0, lengths.length - policy.maxImages)
|
||||
const excessBytes = policy.maxBytes === undefined ? 0 : Math.max(0, total - policy.maxBytes)
|
||||
if (excessCount === 0 && excessBytes === 0) return 0
|
||||
const countQuantum = policy.countQuantum ?? 1
|
||||
const byteQuantum = policy.byteQuantum ?? 1
|
||||
const removeCount = excessCount === 0 ? 0 : Math.ceil(excessCount / countQuantum) * countQuantum
|
||||
const removeBytes = excessBytes === 0 ? 0 : Math.ceil(excessBytes / byteQuantum) * byteQuantum
|
||||
let count = 0
|
||||
let removedBytes = 0
|
||||
for (const imageBytes of lengths) {
|
||||
const byteTargetMet = removeBytes === 0
|
||||
|| (byteQuantum === 1 ? removedBytes >= removeBytes : removedBytes > removeBytes)
|
||||
if (count >= removeCount && byteTargetMet) break
|
||||
removedBytes += imageBytes
|
||||
count += 1
|
||||
}
|
||||
return count
|
||||
}
|
||||
|
||||
/**
|
||||
* Return a deterministic transient projection whose oldest images are replaced
|
||||
* in whole count and byte quanta after a route budget is exceeded. The target
|
||||
@@ -246,23 +279,8 @@ export function offloadRequestImagesWithPolicy(
|
||||
): readonly Message[] {
|
||||
const lengths: number[] = []
|
||||
for (const message of messages) collectImageLengths(message.content, lengths, policy)
|
||||
const total = lengths.reduce((sum, bytes) => sum + bytes, 0)
|
||||
const excessCount = policy.maxImages === undefined ? 0 : Math.max(0, lengths.length - policy.maxImages)
|
||||
const excessBytes = policy.maxBytes === undefined ? 0 : Math.max(0, total - policy.maxBytes)
|
||||
if (excessCount === 0 && excessBytes === 0) return messages
|
||||
const countQuantum = policy.countQuantum ?? 1
|
||||
const byteQuantum = policy.byteQuantum ?? 1
|
||||
const removeCount = excessCount === 0 ? 0 : Math.ceil(excessCount / countQuantum) * countQuantum
|
||||
const removeBytes = excessBytes === 0 ? 0 : Math.ceil(excessBytes / byteQuantum) * byteQuantum
|
||||
let count = 0
|
||||
let removedBytes = 0
|
||||
for (const imageBytes of lengths) {
|
||||
const byteTargetMet = removeBytes === 0
|
||||
|| (byteQuantum === 1 ? removedBytes >= removeBytes : removedBytes > removeBytes)
|
||||
if (count >= removeCount && byteTargetMet) break
|
||||
removedBytes += imageBytes
|
||||
count += 1
|
||||
}
|
||||
const count = offloadedImagePrefixCount(lengths, policy)
|
||||
if (count === 0) return messages
|
||||
const remaining = { count }
|
||||
return messages.map((message) => {
|
||||
const content = replaceOldestImages(message.content, remaining, policy.placeholder)
|
||||
|
||||
@@ -12,6 +12,7 @@ import type {
|
||||
LlmConfigurableProvider,
|
||||
LlmDiscoveredModel,
|
||||
LlmFailure,
|
||||
LlmImageRequestPricing,
|
||||
LlmModelContext,
|
||||
LlmModelDiscoveryRequest,
|
||||
LlmModelInfo,
|
||||
@@ -207,6 +208,19 @@ export abstract class LlmAdapter {
|
||||
return undefined
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve provider-side request-image pricing for one exact model route.
|
||||
* The default declares none, so consumers fall back to their own neutral
|
||||
* estimate. Implementations must answer synchronously without I/O; the
|
||||
* token meter resolves this per measurement.
|
||||
* @param _provider - a route passed to `registerAdapter()` for this instance.
|
||||
* @param _model - exact model id passed to {@link GenerateOptions.model}.
|
||||
* @returns route-owned image pricing, or `undefined` when the route declares none.
|
||||
*/
|
||||
imageRequestPricing(_provider: string, _model: string): LlmImageRequestPricing | undefined {
|
||||
return undefined
|
||||
}
|
||||
|
||||
/**
|
||||
* List models this adapter can currently advertise for one owned provider.
|
||||
* The result is advisory: an adapter may accept unlisted model ids, and
|
||||
@@ -594,6 +608,19 @@ export class LlmRuntime extends Service {
|
||||
return this.registration(provider).retryPolicy
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve provider-side request-image pricing for one exact route, or
|
||||
* `undefined` when the provider is unregistered or declares none. Unknown
|
||||
* providers degrade to `undefined` rather than throwing because callers
|
||||
* price durable history whose route may no longer be mounted.
|
||||
* @param provider - provider route named by a request header.
|
||||
* @param model - exact model id named by the same header.
|
||||
* @returns the owning adapter's image pricing for the route, when declared.
|
||||
*/
|
||||
imageRequestPricing(provider: string, model: string): LlmImageRequestPricing | undefined {
|
||||
return this.adapters.get(provider)?.adapter.imageRequestPricing(provider, model)
|
||||
}
|
||||
|
||||
/** Detach typed adapter-owned modality metadata. */
|
||||
private detachedModalities(modalities: readonly ModelModality[] | undefined): ModelModality[] | undefined {
|
||||
return modalities === undefined ? undefined : [...modalities]
|
||||
|
||||
@@ -148,6 +148,36 @@ export interface TokenUsage {
|
||||
reasoningTokens?: number
|
||||
}
|
||||
|
||||
/**
|
||||
* Request price of one ordered image occurrence under one exact model route's
|
||||
* request projection. Every occurrence resolves to the pair the wire actually
|
||||
* carries: provider visual tokens for a retained image, plus the model-visible
|
||||
* text sent with or instead of it (request-preview handle, offload placeholder,
|
||||
* or text-only substitution). The caller prices `text` with its own text
|
||||
* estimator so provider pricing never fixes a text tokenization.
|
||||
*/
|
||||
export interface LlmImageRequestPrice {
|
||||
/** Provider visual tokens for the retained request image; 0 when only text represents this occurrence. */
|
||||
visualTokens: number
|
||||
/** Model-visible text sent for this occurrence, to be priced by the caller's text estimator. */
|
||||
text: string
|
||||
}
|
||||
|
||||
/**
|
||||
* Provider-side request-image pricing for one exact model route. Implemented
|
||||
* by adapters whose provider charges visual tokens; consumers (the token
|
||||
* meter) resolve it synchronously per measurement, so implementations must not
|
||||
* perform I/O.
|
||||
*/
|
||||
export interface LlmImageRequestPricing {
|
||||
/**
|
||||
* Price every image occurrence of one request projection.
|
||||
* @param images - durable image references in request order, one entry per occurrence.
|
||||
* @returns one price per occurrence, aligned by index with `images`.
|
||||
*/
|
||||
priceImages(images: readonly ImageAttachmentRef[]): readonly LlmImageRequestPrice[]
|
||||
}
|
||||
|
||||
/** Display metadata for one registered provider route. */
|
||||
export interface LlmProviderInfo {
|
||||
/** Provider route key used by {@link GenerateOptions.provider}. */
|
||||
|
||||
@@ -5,6 +5,7 @@ import {
|
||||
CallId,
|
||||
createUserMessage,
|
||||
offloadedImageText,
|
||||
offloadedImagePrefixCount,
|
||||
offloadRequestImagesWithPolicy,
|
||||
projectImagesForTextModel,
|
||||
resolveImageAttachmentAccess,
|
||||
@@ -113,6 +114,19 @@ describe('base64 request-image offload', () => {
|
||||
})
|
||||
})
|
||||
|
||||
describe('offloadedImagePrefixCount', () => {
|
||||
it('removes nothing under unbounded budgets and whole quanta past them', () => {
|
||||
const lengths = [4, 4, 4, 4]
|
||||
expect(offloadedImagePrefixCount(lengths, {})).toBe(0)
|
||||
expect(offloadedImagePrefixCount(lengths, { maxBytes: 16 })).toBe(0)
|
||||
expect(offloadedImagePrefixCount(lengths, { maxImages: 4 })).toBe(0)
|
||||
// One excess image rounds up to the whole count quantum.
|
||||
expect(offloadedImagePrefixCount([...lengths, 4], { maxImages: 4, countQuantum: 2 })).toBe(2)
|
||||
// One excess byte removes a whole byte quantum, crossing the second image.
|
||||
expect(offloadedImagePrefixCount([...lengths, 1], { maxBytes: 16, byteQuantum: 5 })).toBe(2)
|
||||
})
|
||||
})
|
||||
|
||||
describe('offloadRequestImagesWithPolicy', () => {
|
||||
it('drops 129 MiB to 64 MiB and keeps the removed prefix stable through 192 MiB', () => {
|
||||
const mib = 1024 * 1024
|
||||
|
||||
@@ -266,3 +266,27 @@ describe('model discovery registry', () => {
|
||||
await expect(ctx.llm.discoverModels('llm-example', { provider: 'known-route' })).resolves.toEqual([])
|
||||
})
|
||||
})
|
||||
|
||||
describe('imageRequestPricing resolution', () => {
|
||||
it('resolves the owning adapter declaration and degrades everywhere else to undefined', async () => {
|
||||
const ctx = await setup()
|
||||
const pricing = { priceImages: () => [] }
|
||||
class PricingAdapter extends NoopAdapter {
|
||||
override imageRequestPricing(provider: string, model: string): typeof pricing | undefined {
|
||||
return provider === 'a' && model === 'vision' ? pricing : undefined
|
||||
}
|
||||
}
|
||||
const dispose = ctx.llm.registerAdapter(['a'], new PricingAdapter())
|
||||
ctx.llm.registerAdapter(['plain'], new NoopAdapter())
|
||||
|
||||
expect(ctx.llm.imageRequestPricing('a', 'vision')).toBe(pricing)
|
||||
expect(ctx.llm.imageRequestPricing('a', 'other')).toBeUndefined()
|
||||
// The base adapter declares none.
|
||||
expect(ctx.llm.imageRequestPricing('plain', 'vision')).toBeUndefined()
|
||||
// Unregistered providers degrade instead of throwing: callers price
|
||||
// durable history whose route may no longer be mounted.
|
||||
expect(ctx.llm.imageRequestPricing('missing', 'vision')).toBeUndefined()
|
||||
dispose()
|
||||
expect(ctx.llm.imageRequestPricing('a', 'vision')).toBeUndefined()
|
||||
})
|
||||
})
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/llm/token-meter/README.md
|
||||
README.md: ee80412476c4730e409e6a854d3a78922912bba7
|
||||
README.zh.md: 332cc4df33e3d4da5c786fbaf88af210b02cbc85
|
||||
README.md: 399fad26527e046b163fe23abd8fd83312df29a1
|
||||
README.zh.md: ea5e755bb2b13601180ac40a921ad0ba429c8822
|
||||
|
||||
@@ -15,9 +15,9 @@ The estimator has no settings. It intentionally uses one fixed heuristic: four c
|
||||
- `measure(session, requestHeader?)` returns request pressure and the current priced surface at one consumed-log revision.
|
||||
- `estimateMessage(message)` prices one message with the fixed heuristic.
|
||||
|
||||
`measure()` synchronizes once and returns one detached, deeply immutable snapshot. `totalTokens` is request-and-response pressure, while `surfaceTokens` is the surface-only heuristic total and equals the sum of `nodes[].tokens`. A `requestHeader` override affects pressure fields only; the surface fields still describe the current session. Every call clones the positional nodes, so measurement is O(surface).
|
||||
`measure()` synchronizes once and returns one detached, deeply immutable snapshot. `totalTokens` is request-and-response pressure, while `surfaceTokens` is the surface-only route-priced total and equals the sum of `nodes[].tokens`. A `requestHeader` override selects the priced route and the pressure fields; the node set still describes the current session. Every call clones the positional nodes, so measurement is O(surface).
|
||||
|
||||
The fold tracks full request-header snapshots, step boundaries, surface appends and replacements, successful assistant messages, provider usage, and the chunk seqs cited by each assistant message. Provider usage is reused only when the latest successful call's canonical request envelope matches the measured envelope and its total is no lower than that call's full heuristic anchor; a later success replaces the earlier anchor. Otherwise the complete current envelope and surface are estimated. Surface changes remain signed relative to a matching anchor, including negative deltas after shrinking replacements.
|
||||
The fold tracks full request-header snapshots, step boundaries, surface appends and replacements, successful assistant messages, provider usage, and the chunk seqs cited by each assistant message. Each measurement resolves the effective envelope's provider/model to that route's declared request-image pricing through the optional `llm` service: image occurrences are then priced as the visual tokens plus model-visible text the routed request actually sends, while routes and compositions without declared pricing keep the fixed heuristic. Every node also carries `heuristicTokens`, the route-independent fixed price the shadow-price protocol uses for replacements. Provider usage is reused only when the latest successful call's canonical request envelope matches the measured envelope and its total is no lower than that call's full route-priced anchor; a later success replaces the earlier anchor. Otherwise the complete current envelope and surface are estimated. Surface changes remain signed relative to a matching anchor repriced under the same route, including negative deltas after shrinking replacements.
|
||||
|
||||
Usage accounting sums disjoint input, cache-read, cache-write, and output buckets; reasoning is not added again. Every successful call records an assistant anchor, including content-less calls. An explicit empty `sourceEventSeqs` list means a known empty provider stream, while an absent legacy list conservatively treats the durable assistant output as provider output.
|
||||
|
||||
@@ -33,7 +33,7 @@ Token-meter also owns the browser-safe pure fold from one complete Turn's durabl
|
||||
|
||||
`projectedTokens` is what the NEXT request's prompt would cost: the sample plus the heuristic repricing of everything the surface gained or lost since it was taken, clamped at zero. Its O(1) fold in `surface-projection.ts` tracks appends and consumes the logged shadow price immediately before a replacement; on fully metered logs it agrees with the measurement service's positional plan/commit fold without retaining per-node prices. Only the delta is estimated, so the figure stays anchored to the provider while reacting the moment content lands — or a compaction shadows a span. That last case is why the field exists: compaction summarizes through a direct `ctx.llm.stream()` call and appends no usage of its own, so `pressureTokens` alone reports the pre-compaction prompt until an entire further turn completes. Occupancy displays read `projectedTokens`.
|
||||
|
||||
`contextBreakdown` carries heuristic `systemTokens`, `toolsTokens`, and `messageTokens` — the context's composition rather than its provider-billed size. The envelope figures reprice last-wins on every `request/header`; the message figure replays the same O(1) shadow-price fold as `contextPressure`, so on fully metered logs it equals `measure().surfaceTokens` at every event boundary and compaction shrinks it the way it shrinks the next request. A replacement without an adjacent shadow-price claim leaves this bounded projection unchanged because it cannot reconstruct the replaced range. All three figures use the measurement service's fixed heuristic and are estimates: they will not sum to `projectedTokens`, whose provider anchor carries exactly the error — CJK text and JSON schemas underprice badly at four characters per token — that the composition rows still contain. Present them as an approximate composition, never as a total.
|
||||
`contextBreakdown` carries heuristic `systemTokens`, `toolsTokens`, and `messageTokens` — the context's composition rather than its provider-billed size. The envelope figures reprice last-wins on every `request/header`; the message figure replays the same O(1) shadow-price fold as `contextPressure`, so on fully metered logs it equals the sum of `measure().nodes[].heuristicTokens` at every event boundary and compaction shrinks it by its logged shadow price. The route-priced `measure().surfaceTokens` diverges when the routed model reprices images. A replacement without an adjacent shadow-price claim leaves this bounded projection unchanged because it cannot reconstruct the replaced range. All three figures use the measurement service's fixed heuristic and are estimates: they will not sum to `projectedTokens`, whose provider anchor carries exactly the error — CJK text and JSON schemas underprice badly at four characters per token — that the composition rows still contain. Present them as an approximate composition, never as a total.
|
||||
|
||||
All three units use the standard projection baseline, live frame, higher-seq-wins store, and JSON checkpoint paths. Unloading token-meter removes all three keys. A composition without the projection seam keeps the measurement service's existing behavior.
|
||||
|
||||
@@ -52,7 +52,7 @@ The [Agent Note](../../../.agents/notes/implemented/architecture/2026-07-29-proj
|
||||
- name: '@deepseek-ai/dsh-compaction-basic'
|
||||
```
|
||||
|
||||
Both plugins have usable defaults. The meter remains independent of model routing and optional compaction. A deployment configures capacity on its LLM adapter and compaction policy on `dsh-compaction-basic`.
|
||||
Both plugins have usable defaults. The meter consumes only the optional `llm` service, and only to resolve route-declared request-image pricing; compaction remains optional. A deployment configures capacity and image pricing on its LLM adapter and compaction policy on `dsh-compaction-basic`.
|
||||
|
||||
## Model Experience
|
||||
|
||||
@@ -64,7 +64,7 @@ No direct invalidation; the named consumer owns any request-prefix changes.
|
||||
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
- **The fixed heuristic is approximate** — content without reusable provider usage is priced by character count plus structural overhead, not an exact provider tokenizer or request serializer.
|
||||
- **The fixed heuristic is approximate** — text without reusable provider usage is priced by character count plus structural overhead, not an exact provider tokenizer or request serializer; only image occurrences on routes with declared pricing carry provider-exact visual tokens.
|
||||
- **Every measurement clones the current surface** — coherent immutable snapshots make reads O(surface), including below-threshold pressure checks.
|
||||
- **Provider usage is only reusable for an identical canonical envelope** — prompt, prefix, tools, provider, model, or call-config changes deliberately fall back to full heuristic estimation.
|
||||
- **Missing legacy source seqs are handled conservatively** — assistant messages without `sourceEventSeqs` cannot distinguish provider output from listener rewrites, so the fold avoids claiming a known empty or exact chunk stream.
|
||||
|
||||
@@ -15,9 +15,9 @@
|
||||
- `measure(session, requestHeader?)` 在同一个已消费日志 revision 上返回请求压力与当前已计价表层。
|
||||
- `estimateMessage(message)` 使用固定启发式规则为一条消息计价。
|
||||
|
||||
`measure()` 会同步一次，并返回一个独立且深度不可变的快照。`totalTokens` 是请求与响应压力，`surfaceTokens` 是仅表层启发式总量，等于 `nodes[].tokens` 之和。`requestHeader` 覆盖只影响压力字段；表层字段仍描述当前会话。每次调用都会克隆带位置的节点，因此测量是 O(surface)。
|
||||
`measure()` 会同步一次，并返回一个独立且深度不可变的快照。`totalTokens` 是请求与响应压力，`surfaceTokens` 是表层的路由定价总量，等于 `nodes[].tokens` 之和。`requestHeader` 覆盖会选择计价路由并影响压力字段；节点集合仍描述当前会话。每次调用都会克隆带位置的节点，因此测量是 O(surface)。
|
||||
|
||||
fold 跟踪完整请求标头快照、步骤边界、表层追加与替换、成功 assistant 消息、提供方用量，以及每条 assistant 消息引用的分片 seq。只有当最新成功调用的规范请求 envelope 与已测量 envelope 匹配，且其总量不低于该调用的完整启发式锚点时，才会复用提供方用量；后续成功会替换较早锚点。否则会对当前 envelope 与表层进行完整估算。表层变更保持相对于匹配锚点的带符号值，包括缩减替换后的负 delta。
|
||||
fold 跟踪完整请求标头快照、步骤边界、表层追加与替换、成功 assistant 消息、提供方用量，以及每条 assistant 消息引用的分片 seq。每次计量都会通过可选的 `llm` 服务把生效 envelope 的 provider/model 解析为该路由声明的请求图片定价：图片出现处按路由请求实际发送的视觉 token 加模型可见文本计价，未声明定价的路由与组合保持固定启发式规则。每个节点还携带与路由无关的固定价格 `heuristicTokens`，供影子价协议为替换计价。只有当最新成功调用的规范请求 envelope 与已测量 envelope 匹配，且其总量不低于该调用的完整路由定价锚点时，才会复用提供方用量；后续成功会替换较早锚点。否则会对当前 envelope 与表层进行完整估算。表层变更保持相对于匹配锚点（按同一路由重新定价）的带符号值，包括缩减替换后的负 delta。
|
||||
|
||||
用量计量会求和不重叠的输入、cache-read、cache-write 与输出 bucket；不会再次添加推理（reasoning）。每次成功调用都会记录一个 assistant 锚点，包括无内容调用。显式的空 `sourceEventSeqs` 列表表示已知空提供方流；遗留记录缺少该列表时，fold 会保守地将持久 assistant 输出视为提供方输出。
|
||||
|
||||
@@ -33,7 +33,7 @@ token-meter 还拥有一份可安全用于浏览器的纯 fold，将一个完整
|
||||
|
||||
`projectedTokens` 是「下一个请求的提示词要花多少」：在该样本之上，加上自取样以来表层增减部分的启发式重新计价，并将下界钳制为零。它在 `surface-projection.ts` 中的 O(1) 折叠会跟踪追加，并消费紧邻替换之前记录的影子价；在完整计量的日志上，它无需保留逐节点价格也能与测量服务的带位置 plan/commit 折叠一致。只有增量部分是估算的，因此这个数字既锚定在提供方读数上，又能在内容落地——或压缩遮蔽一段区间——的瞬间做出反应。最后这种情况正是该字段存在的理由：压缩通过直连的 `ctx.llm.stream()` 调用生成摘要，自身不追加任何用量，所以仅凭 `pressureTokens` 会一直报告压缩前的提示词规模，直到再完成一整个轮次为止。占用率展示读取 `projectedTokens`。
|
||||
|
||||
`contextBreakdown` 携带启发式的 `systemTokens`、`toolsTokens` 与 `messageTokens`，描述上下文的组成而非提供方计费规模。envelope 数字在每条 `request/header` 上按后者胜重新计价；消息数字重放与 `contextPressure` 相同的 O(1) 影子价折叠，因此在完整计量的日志上，它在每个事件边界都等于 `measure().surfaceTokens`，压缩会像缩小下一个请求那样缩小它。若替换前没有紧邻的影子价声明，这个有界投影会保持不变，因为它无法重建被替换区间。三个数字都使用测量服务的固定启发式规则，属于估算值：它们加起来不等于 `projectedTokens`——后者的提供方锚点所体现的恰好是这些明细行仍然带着的误差（按「4 字符 ≈ 1 token」计价，CJK 文本与 JSON schema 会被严重低估）。请把它们当作近似的**组成**呈现，而不是总量。
|
||||
`contextBreakdown` 携带启发式的 `systemTokens`、`toolsTokens` 与 `messageTokens`，描述上下文的组成而非提供方计费规模。envelope 数字在每条 `request/header` 上按后者胜重新计价；消息数字重放与 `contextPressure` 相同的 O(1) 影子价折叠，因此在完整计量的日志上，它在每个事件边界都等于 `measure().nodes[].heuristicTokens` 之和，压缩会按记录的影子价缩小该值。路由定价的 `measure().surfaceTokens` 在路由模型重新为图片计价时会与该值不同。若替换前没有紧邻的影子价声明，这个有界投影会保持不变，因为它无法重建被替换区间。三个数字都使用测量服务的固定启发式规则，属于估算值。它们加起来不等于 `projectedTokens`，后者的提供方锚点体现了这些明细行仍然带有的误差（按「4 字符 ≈ 1 token」计价时，CJK 文本与 JSON schema 会被严重低估）。请把它们当作近似的**组成**呈现，而不是总量。
|
||||
|
||||
三个单元都使用标准的投影基线、实时帧、seq 高者胜值仓和 JSON 检查点路径。卸载 token-meter 会移除这三个键。不带投影 seam 的组合会保留测量服务的既有行为。
|
||||
|
||||
@@ -52,7 +52,7 @@ token-meter 还拥有一份可安全用于浏览器的纯 fold，将一个完整
|
||||
- name: '@deepseek-ai/dsh-compaction-basic'
|
||||
```
|
||||
|
||||
两个插件都有可用默认值。meter 保持与模型路由和可选压缩无关。部署会在 LLM（大语言模型）适配器上配置容量，并在 `dsh-compaction-basic` 上配置压缩策略。
|
||||
两个插件都有可用默认值。meter 只消费可选的 `llm` 服务，且仅用于解析路由声明的请求图片定价；压缩保持可选。部署会在 LLM（大语言模型）适配器上配置容量与图片定价，并在 `dsh-compaction-basic` 上配置压缩策略。
|
||||
|
||||
## 模型体验
|
||||
|
||||
@@ -64,7 +64,7 @@ token-meter 还拥有一份可安全用于浏览器的纯 fold，将一个完整
|
||||
|
||||
## 已知限制与暂缓事项
|
||||
|
||||
- **固定启发式规则是近似值**：没有可复用提供方用量的内容按字符数加结构开销计价，而不是使用精确提供方 tokenizer 或请求 serializer。
|
||||
- **固定启发式规则是近似值**：没有可复用提供方用量的文本按字符数加结构开销计价，而不是使用精确提供方 tokenizer 或请求 serializer；只有声明了定价的路由上的图片出现处携带提供方精确的视觉 token。
|
||||
- **每次测量都会克隆当前表层**：一致且不可变的快照使读取成为 O(surface)，包括低于阈值的压力检查。
|
||||
- **提供方用量只能为完全相同的规范 envelope 复用**：提示词、前缀、工具、提供方、模型或调用配置变更都会有意回退到完整启发式估算。
|
||||
- **保守处理缺少源事件 seq 的遗留记录**：没有 `sourceEventSeqs` 的 assistant 消息无法区分提供方输出与 listener 改写，因此 fold 不会声称已知空流或精确分片流。
|
||||
|
||||
@@ -46,9 +46,11 @@ const breakdownSchema = z.object({
|
||||
*
|
||||
* Envelope figures are last-wins per `request/header`; the message figure
|
||||
* rides {@link foldSurfaceProjection} — the same O(1) fold the occupancy
|
||||
* projection uses — so fully metered logs equal `measure().surfaceTokens` at
|
||||
* every event boundary and compaction shrinks the figure by its logged shadow
|
||||
* price. A replacement without a claim preserves the previous total. The
|
||||
* projection uses — so fully metered logs equal the sum of
|
||||
* `measure().nodes[].heuristicTokens` at every event boundary and compaction
|
||||
* shrinks the figure by its logged shadow price; the route-priced
|
||||
* `measure().surfaceTokens` deliberately diverges by the routed model's image
|
||||
* repricing. A replacement without a claim preserves the previous total. The
|
||||
* state is a fixed handful of numbers, so the persisted checkpoint stays
|
||||
* O(1) over the session's life.
|
||||
*/
|
||||
|
||||
@@ -18,6 +18,17 @@ const BLOCK_OVERHEAD = 4
|
||||
/** Role-field framing overhead added to every priced message. */
|
||||
export const ROLE_OVERHEAD = 4
|
||||
|
||||
/**
|
||||
* Structural JSON price of one block outside the typed pricing arms: the
|
||||
* fixed heuristic for merge-extended blocks and for image references, whose
|
||||
* request price is route-owned rather than fixed.
|
||||
* @param block - block to price without mutation.
|
||||
* @returns heuristic tokens for the block's JSON structure.
|
||||
*/
|
||||
export function estimateStructuralBlock(block: ContentBlock): number {
|
||||
return BLOCK_OVERHEAD + Math.ceil(JSON.stringify(block).length / CHARS_PER_TOKEN)
|
||||
}
|
||||
|
||||
/**
|
||||
* Price content blocks recursively under the fixed density heuristic.
|
||||
* @param blocks - content blocks to price without mutation.
|
||||
@@ -40,9 +51,10 @@ export function estimateContent(blocks: readonly ContentBlock[]): number {
|
||||
tokens += estimateContent(block.content) + BLOCK_OVERHEAD
|
||||
break
|
||||
default:
|
||||
// ContentBlockMap is merge-extensible; unknown blocks retain a
|
||||
// ContentBlockMap is merge-extensible; unknown blocks (and image
|
||||
// references, whose request price is route-owned) retain a
|
||||
// conservative structural JSON price under the fixed heuristic.
|
||||
tokens += BLOCK_OVERHEAD + Math.ceil(JSON.stringify(block).length / CHARS_PER_TOKEN)
|
||||
tokens += estimateStructuralBlock(block)
|
||||
}
|
||||
}
|
||||
return tokens
|
||||
|
||||
@@ -7,7 +7,7 @@
|
||||
import { Context, Service } from '@deepseek-ai/cordis'
|
||||
import z from '@deepseek-ai/schemastery'
|
||||
import { BlockAssembler, deepFreeze } from '@deepseek-ai/dsh-llm'
|
||||
import type { Message, TokenUsage } from '@deepseek-ai/dsh-llm'
|
||||
import type { LlmImageRequestPricing, Message, TokenUsage } from '@deepseek-ai/dsh-llm'
|
||||
import type { EpochHeader, Session, SessionEvent } from '@deepseek-ai/dsh-session'
|
||||
import { canonicalHeader, headerEquals, isSurfaceEvent } from '@deepseek-ai/dsh-session'
|
||||
// Type-only: resolves the optional projection registry Context declaration.
|
||||
@@ -16,27 +16,36 @@ import type {
|
||||
TokenMeasurement,
|
||||
TokenMeasurementBaseline,
|
||||
TokenMeterConfig,
|
||||
TokenSurfaceNode,
|
||||
} from './types.ts'
|
||||
import { contextBreakdownProjectionDefinition } from './breakdown-projection.ts'
|
||||
import { contextPressureProjectionDefinition, tokenUsageProjectionDefinition } from './usage-projection.ts'
|
||||
import { estimateContent, estimateHeader, estimateMessage, ROLE_OVERHEAD } from './estimate.ts'
|
||||
import { commitSurfaceTokens, planSurfaceTokens } from './surface-fold.ts'
|
||||
import type { MeterSurfaceNode } from './surface-fold.ts'
|
||||
import { priceSurface } from './route-pricing.ts'
|
||||
|
||||
export type * from './types.ts'
|
||||
|
||||
/**
|
||||
* Raw anchor facts captured at the latest successful call; the baseline is
|
||||
* derived per measurement so the anchored surface reprices under the same
|
||||
* route pricing as the current surface it is compared with.
|
||||
*/
|
||||
interface MeasurementAnchor {
|
||||
readonly header: EpochHeader | undefined
|
||||
readonly surfaceTokens: number
|
||||
readonly baseline: Exclude<TokenMeasurementBaseline, { kind: 'none' }>
|
||||
/** Surface snapshot the anchored request was derived from. */
|
||||
readonly nodes: readonly MeterSurfaceNode[]
|
||||
/** Fixed-heuristic price of the call's provider output. */
|
||||
readonly assistantTokens: number
|
||||
/** Provider usage of the call, when it reported one under a known header. */
|
||||
readonly usage: TokenUsage | undefined
|
||||
}
|
||||
|
||||
interface ReplayState {
|
||||
consumedEvents: number
|
||||
header: EpochHeader | undefined
|
||||
surface: TokenSurfaceNode[]
|
||||
surfaceTokens: number
|
||||
stepStart: { turn: number; step: number; surfaceTokens: number } | undefined
|
||||
surface: MeterSurfaceNode[]
|
||||
stepStart: { turn: number; step: number; nodes: readonly MeterSurfaceNode[] } | undefined
|
||||
anchor: MeasurementAnchor | undefined
|
||||
}
|
||||
|
||||
@@ -100,14 +109,18 @@ export class TokenMeter extends Service {
|
||||
/**
|
||||
* Measure current request pressure and surface through the durable tail.
|
||||
*
|
||||
* Provider usage is reused only when the latest successful call's canonical
|
||||
* request envelope matches `requestHeader` and its total is no lower than
|
||||
* that call's full heuristic anchor; otherwise the complete envelope and
|
||||
* surface are heuristically repriced.
|
||||
* The effective envelope's routed provider/model selects the request-image
|
||||
* pricing every node is priced under: a route whose adapter declares image
|
||||
* pricing charges each retained image its visual tokens plus its
|
||||
* model-visible text, while other routes keep the fixed heuristic. Provider
|
||||
* usage is reused only when the latest successful call's canonical request
|
||||
* envelope matches `requestHeader` and its total is no lower than that
|
||||
* call's full route-priced anchor; otherwise the complete envelope and
|
||||
* surface are repriced.
|
||||
*
|
||||
* `requestHeader` affects request pressure only; surface fields always
|
||||
* describe the current session surface. Every call clones those positional
|
||||
* nodes, so measurement is O(surface).
|
||||
* `requestHeader` replaces the latest logged envelope for pressure and node
|
||||
* pricing; the node set always describes the current session surface. Every
|
||||
* call clones those positional nodes, so measurement is O(surface).
|
||||
*
|
||||
* @param session - session to replay through its current durable tail.
|
||||
* @param requestHeader - optional effective request envelope replacing the latest logged header.
|
||||
@@ -118,20 +131,33 @@ export class TokenMeter extends Service {
|
||||
const header = requestHeader === undefined
|
||||
? state.header
|
||||
: canonicalHeader(requestHeader)
|
||||
const pricing = this._routeImagePricing(header)
|
||||
const surface = priceSurface(state.surface, pricing)
|
||||
const anchor = state.anchor
|
||||
|
||||
let baseline: TokenMeasurementBaseline
|
||||
let surfaceDeltaTokens: number
|
||||
if (anchor !== undefined && optionalHeaderEquals(anchor.header, header)) {
|
||||
baseline = anchor.baseline
|
||||
surfaceDeltaTokens = state.surfaceTokens - anchor.surfaceTokens
|
||||
} else if (header === undefined && state.surfaceTokens === 0) {
|
||||
// Matching headers share one route, so the anchored snapshot reprices
|
||||
// under the same pricing as the current surface and the signed delta
|
||||
// compares like with like.
|
||||
const anchorSurfaceTokens = priceSurface(anchor.nodes, pricing).surfaceTokens
|
||||
+ anchor.assistantTokens
|
||||
const estimatedAnchorTokens = estimateHeader(header) + anchorSurfaceTokens
|
||||
const usage = anchor.usage
|
||||
// Signed heuristic deltas remain conservative only from an anchor
|
||||
// that is at least as large as the matching full heuristic price.
|
||||
baseline = usage !== undefined && usageTokens(usage) >= estimatedAnchorTokens
|
||||
? { kind: 'usage', tokens: usageTokens(usage), usage }
|
||||
: { kind: 'estimated', tokens: estimatedAnchorTokens }
|
||||
surfaceDeltaTokens = surface.surfaceTokens - anchorSurfaceTokens
|
||||
} else if (header === undefined && surface.surfaceTokens === 0) {
|
||||
baseline = { kind: 'none', tokens: 0 }
|
||||
surfaceDeltaTokens = 0
|
||||
} else {
|
||||
baseline = {
|
||||
kind: 'estimated',
|
||||
tokens: estimateHeader(header) + state.surfaceTokens,
|
||||
tokens: estimateHeader(header) + surface.surfaceTokens,
|
||||
}
|
||||
surfaceDeltaTokens = 0
|
||||
}
|
||||
@@ -141,11 +167,18 @@ export class TokenMeter extends Service {
|
||||
baseline,
|
||||
surfaceDeltaTokens,
|
||||
totalTokens: Math.max(0, baseline.tokens + surfaceDeltaTokens),
|
||||
surfaceTokens: state.surfaceTokens,
|
||||
nodes: state.surface,
|
||||
surfaceTokens: surface.surfaceTokens,
|
||||
nodes: surface.nodes,
|
||||
}))
|
||||
}
|
||||
|
||||
/** Resolve the routed model's image pricing, when the llm service and route declare one. */
|
||||
private _routeImagePricing(header: EpochHeader | undefined): LlmImageRequestPricing | undefined {
|
||||
const config = header?.config
|
||||
if (config === undefined) return undefined
|
||||
return this.ctx.get('llm')?.imageRequestPricing(config.provider, config.model)
|
||||
}
|
||||
|
||||
/**
|
||||
* Heuristically price one model-visible message (instance face of the pure
|
||||
* `estimateMessage` export from `estimate.ts`).
|
||||
@@ -164,7 +197,6 @@ export class TokenMeter extends Service {
|
||||
consumedEvents: 0,
|
||||
header: undefined,
|
||||
surface: [],
|
||||
surfaceTokens: 0,
|
||||
stepStart: undefined,
|
||||
anchor: undefined,
|
||||
}
|
||||
@@ -200,7 +232,7 @@ export class TokenMeter extends Service {
|
||||
`token meter: step/start at seq ${event.seq} arrived before turn ${state.stepStart.turn}/step ${state.stepStart.step} ended`,
|
||||
)
|
||||
}
|
||||
nextStepStart = { ...event.data, surfaceTokens: state.surfaceTokens }
|
||||
nextStepStart = { ...event.data, nodes: [...state.surface] }
|
||||
break
|
||||
case 'step/end':
|
||||
if (state.stepStart === undefined
|
||||
@@ -230,32 +262,18 @@ export class TokenMeter extends Service {
|
||||
// oxlint-disable-next-line typescript/no-non-null-assertion
|
||||
const eventTokens = plan!.tokens
|
||||
if (event.data.usage !== undefined && nextHeader !== undefined) {
|
||||
const providerAssistantTokens = this._estimateProviderAssistant(
|
||||
session,
|
||||
event,
|
||||
eventTokens,
|
||||
)
|
||||
const anchorSurfaceTokens = stepStart.surfaceTokens + providerAssistantTokens
|
||||
const providerTokens = usageTokens(event.data.usage)
|
||||
const estimatedAnchorTokens = estimateHeader(nextHeader) + anchorSurfaceTokens
|
||||
nextAnchor = {
|
||||
header: nextHeader,
|
||||
surfaceTokens: anchorSurfaceTokens,
|
||||
// Signed heuristic deltas remain conservative only from an anchor
|
||||
// that is at least as large as the matching full heuristic price.
|
||||
baseline: providerTokens >= estimatedAnchorTokens
|
||||
? { kind: 'usage', tokens: providerTokens, usage: event.data.usage }
|
||||
: { kind: 'estimated', tokens: estimatedAnchorTokens },
|
||||
nodes: stepStart.nodes,
|
||||
assistantTokens: this._estimateProviderAssistant(session, event, eventTokens),
|
||||
usage: event.data.usage,
|
||||
}
|
||||
} else {
|
||||
const anchorSurfaceTokens = stepStart.surfaceTokens + eventTokens
|
||||
nextAnchor = {
|
||||
header: nextHeader,
|
||||
surfaceTokens: anchorSurfaceTokens,
|
||||
baseline: {
|
||||
kind: 'estimated',
|
||||
tokens: estimateHeader(nextHeader) + anchorSurfaceTokens,
|
||||
},
|
||||
nodes: stepStart.nodes,
|
||||
assistantTokens: eventTokens,
|
||||
usage: undefined,
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -264,7 +282,6 @@ export class TokenMeter extends Service {
|
||||
state.stepStart = nextStepStart
|
||||
if (plan !== undefined) {
|
||||
commitSurfaceTokens(state.surface, plan)
|
||||
state.surfaceTokens += plan.deltaTokens
|
||||
}
|
||||
state.anchor = nextAnchor
|
||||
}
|
||||
|
||||
@@ -22,9 +22,11 @@ export const inject = ['invariants']
|
||||
* not be monotone when a final sample corrects an earlier chunk, and the
|
||||
* composition fold prices through the same `estimate.ts` heuristic as the
|
||||
* measurement service and subtracts producer-logged shadow prices derived
|
||||
* from that service's own nodes, which makes its message figure equal
|
||||
* `measure().surfaceTokens` by construction rather than by a relation worth
|
||||
* observing at runtime.
|
||||
* from that service's own fixed-heuristic node prices, which makes its
|
||||
* message figure equal the sum of `measure().nodes[].heuristicTokens` by
|
||||
* construction rather than by a relation worth observing at runtime; the
|
||||
* route-priced `surfaceTokens` deliberately diverges by the routed model's
|
||||
* image repricing.
|
||||
*/
|
||||
const install: InvariantInstaller = () => {}
|
||||
|
||||
|
||||
@@ -0,0 +1,68 @@
|
||||
/**
|
||||
* Route-aware surface pricing: projects the fold's fixed-heuristic nodes onto
|
||||
* the routed model's request, replacing every image occurrence's structural
|
||||
* price with the route's declared visual tokens plus the model-visible text it
|
||||
* actually sends. Without declared pricing every node keeps its fixed
|
||||
* heuristic price, so provider-neutral behavior is unchanged.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-token-meter/route-pricing
|
||||
*/
|
||||
|
||||
import type { LlmImageRequestPricing } from '@deepseek-ai/dsh-llm'
|
||||
import { estimateContent } from './estimate.ts'
|
||||
import type { MeterSurfaceNode } from './surface-fold.ts'
|
||||
import type { TokenSurfaceNode } from './types.ts'
|
||||
|
||||
/** One surface priced for a request route: public nodes plus their total. */
|
||||
export interface PricedSurface {
|
||||
/** Positional nodes carrying both the route price and the fixed-heuristic price. */
|
||||
readonly nodes: TokenSurfaceNode[]
|
||||
/** Sum of the route prices across the surface. */
|
||||
readonly surfaceTokens: number
|
||||
}
|
||||
|
||||
/**
|
||||
* Price one ordered surface under a route's request-image pricing.
|
||||
* @param nodes - the fold's current or snapshotted surface, in model-visible order.
|
||||
* @param pricing - the routed model's image pricing, or undefined to keep the fixed heuristic.
|
||||
* @returns detached public nodes and their route-priced total.
|
||||
* @throws when the pricing answers a different occurrence count than it was
|
||||
* asked — misalignment would silently misprice nodes, so it must fail loud.
|
||||
*/
|
||||
export function priceSurface(
|
||||
nodes: readonly MeterSurfaceNode[],
|
||||
pricing: LlmImageRequestPricing | undefined,
|
||||
): PricedSurface {
|
||||
const images = pricing === undefined ? [] : nodes.flatMap(node => node.images)
|
||||
if (pricing === undefined || images.length === 0) {
|
||||
let surfaceTokens = 0
|
||||
const publicNodes = nodes.map((node) => {
|
||||
surfaceTokens += node.heuristicTokens
|
||||
return { seq: node.seq, tokens: node.heuristicTokens, heuristicTokens: node.heuristicTokens }
|
||||
})
|
||||
return { nodes: publicNodes, surfaceTokens }
|
||||
}
|
||||
const prices = pricing.priceImages(images)
|
||||
if (prices.length !== images.length) {
|
||||
throw new Error(
|
||||
`token meter: route image pricing answered ${prices.length} prices for ${images.length} occurrences`,
|
||||
)
|
||||
}
|
||||
let cursor = 0
|
||||
let surfaceTokens = 0
|
||||
const publicNodes = nodes.map((node) => {
|
||||
let tokens = node.heuristicTokens
|
||||
if (node.images.length > 0) {
|
||||
tokens = node.imageFreeTokens
|
||||
for (let occurrence = 0; occurrence < node.images.length; occurrence += 1) {
|
||||
// oxlint-disable-next-line typescript/no-non-null-assertion -- length equality is asserted above
|
||||
const price = prices[cursor]!
|
||||
cursor += 1
|
||||
tokens += price.visualTokens + estimateContent([{ type: 'text', text: price.text }])
|
||||
}
|
||||
}
|
||||
surfaceTokens += tokens
|
||||
return { seq: node.seq, tokens, heuristicTokens: node.heuristicTokens }
|
||||
})
|
||||
return { nodes: publicNodes, surfaceTokens }
|
||||
}
|
||||
@@ -4,20 +4,35 @@
|
||||
* units do NOT share this fold — their state must stay O(1) for the
|
||||
* persisted checkpoint, so they ride `surface-projection.ts`'s shadow-price
|
||||
* protocol; the two agree because both price through `estimate.ts` and every
|
||||
* logged shadow price derives from this fold's nodes.
|
||||
* logged shadow price derives from this fold's fixed-heuristic node prices.
|
||||
*
|
||||
* The fold is a plan/commit pair: {@link planSurfaceTokens} runs every
|
||||
* fallible step read-only and {@link commitSurfaceTokens} mutates in place,
|
||||
* so a throw leaves the caller's state untouched and the same malformed
|
||||
* event fails identically on every retry.
|
||||
* Nodes also carry their durable image occurrences and image-free heuristic
|
||||
* price, so `measure()` can reprice image content for the routed model.
|
||||
*
|
||||
* @module @deepseek-ai/dsh-token-meter/surface-fold
|
||||
*/
|
||||
|
||||
import { deriveEventMessage } from '@deepseek-ai/dsh-session'
|
||||
import type { SurfaceEvent } from '@deepseek-ai/dsh-session'
|
||||
import type { TokenSurfaceNode } from './types.ts'
|
||||
import { estimateMessage } from './estimate.ts'
|
||||
import type { ContentBlock, Message } from '@deepseek-ai/dsh-llm'
|
||||
import type { ImageAttachmentRef } from '@deepseek-ai/dsh-attachment'
|
||||
import { estimateMessage, estimateStructuralBlock } from './estimate.ts'
|
||||
|
||||
/** One priced surface node with the image occurrences route pricing replaces. */
|
||||
export interface MeterSurfaceNode {
|
||||
/** Durable sequence number of the surface event. */
|
||||
readonly seq: number
|
||||
/** Fixed-heuristic price of the node's exact message. */
|
||||
readonly heuristicTokens: number
|
||||
/** Fixed-heuristic price with every image occurrence's structural price removed. */
|
||||
readonly imageFreeTokens: number
|
||||
/** Durable image occurrences in message order; empty for image-free nodes. */
|
||||
readonly images: readonly ImageAttachmentRef[]
|
||||
}
|
||||
|
||||
/** One validated surface transition that has not mutated the priced surface yet. */
|
||||
export interface SurfaceTokenPlan {
|
||||
@@ -26,11 +41,39 @@ export interface SurfaceTokenPlan {
|
||||
/** Signed change in the surface total: `tokens` minus anything shadowed. */
|
||||
readonly deltaTokens: number
|
||||
/** The priced node the commit inserts for this event. */
|
||||
readonly node: TokenSurfaceNode
|
||||
readonly node: MeterSurfaceNode
|
||||
/** Commit position: `append`, or the inclusive replaced index range. */
|
||||
readonly target: 'append' | { readonly startIdx: number; readonly endIdx: number }
|
||||
}
|
||||
|
||||
/** Collect image occurrences recursively and total their structural prices. */
|
||||
function collectImages(blocks: readonly ContentBlock[], images: ImageAttachmentRef[]): number {
|
||||
let structuralTokens = 0
|
||||
for (const block of blocks) {
|
||||
if (block.type === 'image') {
|
||||
images.push(block.attachment)
|
||||
structuralTokens += estimateStructuralBlock(block)
|
||||
} else if (block.type === 'tool-result') {
|
||||
structuralTokens += collectImages(block.content, images)
|
||||
}
|
||||
}
|
||||
return structuralTokens
|
||||
}
|
||||
|
||||
/** Build one priced node from a surface event's derived message. */
|
||||
function analyzeNode(seq: number, message: Message | null): MeterSurfaceNode {
|
||||
if (message === null) return { seq, heuristicTokens: 0, imageFreeTokens: 0, images: [] }
|
||||
const heuristicTokens = estimateMessage(message)
|
||||
const images: ImageAttachmentRef[] = []
|
||||
const imageStructuralTokens = collectImages(message.content, images)
|
||||
return {
|
||||
seq,
|
||||
heuristicTokens,
|
||||
imageFreeTokens: heuristicTokens - imageStructuralTokens,
|
||||
images,
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate and price one surface event without mutating the surface.
|
||||
* @param nodes - the priced surface preceding this event, in model-visible order.
|
||||
@@ -41,12 +84,11 @@ export interface SurfaceTokenPlan {
|
||||
* corruption and must fail loud rather than skip the event.
|
||||
*/
|
||||
export function planSurfaceTokens(
|
||||
nodes: readonly TokenSurfaceNode[],
|
||||
nodes: readonly MeterSurfaceNode[],
|
||||
event: SurfaceEvent,
|
||||
): SurfaceTokenPlan {
|
||||
const message = deriveEventMessage(event)
|
||||
const tokens = message === null ? 0 : estimateMessage(message)
|
||||
const node = { seq: event.seq, tokens }
|
||||
const node = analyzeNode(event.seq, deriveEventMessage(event))
|
||||
const tokens = node.heuristicTokens
|
||||
const op = event.surfaceOp
|
||||
if (op === 'append') {
|
||||
return { tokens, deltaTokens: tokens, node, target: 'append' }
|
||||
@@ -58,9 +100,9 @@ export function planSurfaceTokens(
|
||||
`token surface: replace at seq ${event.seq} has invalid current range ${op.start}-${op.end}`,
|
||||
)
|
||||
}
|
||||
let removed = 0
|
||||
// oxlint-disable-next-line typescript/no-non-null-assertion -- startIdx..endIdx are validated indices
|
||||
for (let index = startIdx; index <= endIdx; index += 1) removed += nodes[index]!.tokens
|
||||
const removed = nodes
|
||||
.slice(startIdx, endIdx + 1)
|
||||
.reduce((total, candidate) => total + candidate.heuristicTokens, 0)
|
||||
return { tokens, deltaTokens: tokens - removed, node, target: { startIdx, endIdx } }
|
||||
}
|
||||
|
||||
@@ -70,7 +112,7 @@ export function planSurfaceTokens(
|
||||
* @param nodes - the exact priced surface the plan was built against.
|
||||
* @param plan - the transition returned by {@link planSurfaceTokens}.
|
||||
*/
|
||||
export function commitSurfaceTokens(nodes: TokenSurfaceNode[], plan: SurfaceTokenPlan): void {
|
||||
export function commitSurfaceTokens(nodes: MeterSurfaceNode[], plan: SurfaceTokenPlan): void {
|
||||
if (plan.target === 'append') {
|
||||
nodes.push(plan.node)
|
||||
return
|
||||
|
||||
@@ -27,7 +27,7 @@ export interface TokenMeasurement {
|
||||
readonly surfaceDeltaTokens: number
|
||||
/** Non-negative current request-and-response pressure. */
|
||||
readonly totalTokens: number
|
||||
/** Total heuristic tokens across the current surface. */
|
||||
/** Total route-priced request tokens across the current surface; equals the sum of the node prices. */
|
||||
readonly surfaceTokens: number
|
||||
/** Current surface nodes in positional head-to-tail order. */
|
||||
readonly nodes: readonly TokenSurfaceNode[]
|
||||
@@ -37,6 +37,17 @@ export interface TokenMeasurement {
|
||||
export interface TokenSurfaceNode {
|
||||
/** Durable sequence number of the surface event. */
|
||||
readonly seq: number
|
||||
/** Heuristic tokens for the exact message projected by this node. */
|
||||
/**
|
||||
* Request-pressure tokens for the exact message projected by this node under
|
||||
* the measured route: image occurrences carry the route's declared visual
|
||||
* price when the routed adapter declares one, and the fixed heuristic
|
||||
* otherwise. Trigger, retention, and range selection all read this price.
|
||||
*/
|
||||
readonly tokens: number
|
||||
/**
|
||||
* Fixed-heuristic tokens for the same message, independent of any route.
|
||||
* The shadow-price protocol prices replacements with this value so the O(1)
|
||||
* projection fold stays in agreement with its own appends.
|
||||
*/
|
||||
readonly heuristicTokens: number
|
||||
}
|
||||
|
||||
@@ -0,0 +1,203 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { Context } from '@deepseek-ai/cordis'
|
||||
import { LlmRuntime, LlmAdapter, createMessage, createUserMessage } from '@deepseek-ai/dsh-llm'
|
||||
import type { GenerateOptions, LlmImageRequestPricing, Message, StreamChunk, TokenUsage, UserMessage } from '@deepseek-ai/dsh-llm'
|
||||
import { AttachmentId } from '@deepseek-ai/dsh-attachment'
|
||||
import type { ImageAttachmentRef } from '@deepseek-ai/dsh-attachment'
|
||||
import { Session, SessionId, canonicalHeader } from '@deepseek-ai/dsh-session'
|
||||
import type { EpochHeader } from '@deepseek-ai/dsh-session'
|
||||
import SessionProjectionRegistry from '@deepseek-ai/dsh-session-projection'
|
||||
import TokenMeter from '@deepseek-ai/dsh-token-meter'
|
||||
import { estimateContent, estimateMessage } from '../src/estimate.ts'
|
||||
|
||||
/** Adapter double declaring fixed per-occurrence image prices for one route. */
|
||||
class PricingAdapter extends LlmAdapter {
|
||||
constructor(private readonly pricing: (model: string) => LlmImageRequestPricing | undefined) {
|
||||
super()
|
||||
}
|
||||
|
||||
override imageRequestPricing(_provider: string, model: string): LlmImageRequestPricing | undefined {
|
||||
return this.pricing(model)
|
||||
}
|
||||
|
||||
async * stream(_options: GenerateOptions): AsyncIterable<StreamChunk> {
|
||||
throw new Error('the pricing adapter double does not stream')
|
||||
}
|
||||
}
|
||||
|
||||
const VISUAL_TOKENS = 100
|
||||
const HANDLE_TEXT = 'Image handle text'
|
||||
|
||||
const fixedPricing: LlmImageRequestPricing = {
|
||||
priceImages: images => images.map(() => ({ visualTokens: VISUAL_TOKENS, text: HANDLE_TEXT })),
|
||||
}
|
||||
|
||||
function imageRef(name: string): ImageAttachmentRef {
|
||||
return {
|
||||
attachmentId: AttachmentId(`sha256:${name.padEnd(8, '0')}`),
|
||||
mediaType: 'image/png',
|
||||
bytes: 2048,
|
||||
width: 800,
|
||||
height: 800,
|
||||
name,
|
||||
}
|
||||
}
|
||||
|
||||
function imageMessage(name: string, text = 'look at this'): UserMessage {
|
||||
return createUserMessage({
|
||||
content: [
|
||||
{ type: 'text', text },
|
||||
{ type: 'image', attachment: imageRef(name) },
|
||||
],
|
||||
source: { kind: 'user' },
|
||||
})
|
||||
}
|
||||
|
||||
function header(model: string): EpochHeader {
|
||||
return canonicalHeader({ config: { provider: 'mock', model } })
|
||||
}
|
||||
|
||||
interface Harness {
|
||||
meter: TokenMeter
|
||||
session: Session
|
||||
}
|
||||
|
||||
async function harness(pricing: (model: string) => LlmImageRequestPricing | undefined): Promise<Harness> {
|
||||
const ctx = new Context()
|
||||
new SessionProjectionRegistry(ctx)
|
||||
const llm = new LlmRuntime(ctx)
|
||||
llm.registerAdapter(['mock'], new PricingAdapter(pricing))
|
||||
const meter = new TokenMeter(ctx)
|
||||
return { meter, session: Session.create(SessionId('route-priced')) }
|
||||
}
|
||||
|
||||
/** Route price of one image-bearing message under the fixed pricing double. */
|
||||
function routedMessageTokens(message: Message): number {
|
||||
const imageFree = estimateMessage({
|
||||
...message,
|
||||
content: message.content.filter(block => block.type !== 'image'),
|
||||
})
|
||||
return imageFree + VISUAL_TOKENS + estimateContent([{ type: 'text', text: HANDLE_TEXT }])
|
||||
}
|
||||
|
||||
function appendSuccessfulCall(session: Session, value: EpochHeader, usage?: TokenUsage): void {
|
||||
session.append('step/start', { turn: 1, step: 1 })
|
||||
session.append('request/header', { header: value, reason: 'initial' })
|
||||
session.append('assistant/message', {
|
||||
turn: 1,
|
||||
step: 1,
|
||||
message: createMessage({
|
||||
role: 'assistant',
|
||||
content: [{ type: 'text', text: 'answer' }],
|
||||
source: { kind: 'model', provider: value.config.provider, model: value.config.model },
|
||||
}),
|
||||
...usage === undefined ? {} : { usage },
|
||||
}, { surfaceOp: 'append' })
|
||||
session.append('step/end', { turn: 1, step: 1 })
|
||||
}
|
||||
|
||||
describe('route-aware image pricing', () => {
|
||||
it('prices a first multimodal request estimate with the routed visual tokens', async () => {
|
||||
const { meter, session } = await harness(() => fixedPricing)
|
||||
const message = imageMessage('photo')
|
||||
session.append('user/message', message, { surfaceOp: 'append' })
|
||||
session.append('request/header', { header: header('vision'), reason: 'initial' })
|
||||
|
||||
const measurement = meter.measure(session)
|
||||
const expectedNode = routedMessageTokens(message)
|
||||
expect(measurement.nodes).toHaveLength(1)
|
||||
const node = measurement.nodes[0]!
|
||||
expect(node.tokens).toBe(expectedNode)
|
||||
expect(node.heuristicTokens).toBe(estimateMessage(message))
|
||||
expect(node.tokens).toBeGreaterThan(node.heuristicTokens)
|
||||
expect(measurement.baseline.kind).toBe('estimated')
|
||||
expect(measurement.surfaceTokens).toBe(expectedNode)
|
||||
expect(measurement.totalTokens).toBe(expectedNode)
|
||||
})
|
||||
|
||||
it('adds a post-anchor image at its routed price on top of provider usage', async () => {
|
||||
const { meter, session } = await harness(() => fixedPricing)
|
||||
const usage: TokenUsage = { inputTokens: 5000, outputTokens: 50 }
|
||||
appendSuccessfulCall(session, header('vision'), usage)
|
||||
const before = meter.measure(session)
|
||||
expect(before.baseline).toMatchObject({ kind: 'usage', tokens: 5050 })
|
||||
|
||||
const message = imageMessage('fresh')
|
||||
session.append('user/message', message, { surfaceOp: 'append' })
|
||||
const after = meter.measure(session)
|
||||
expect(after.baseline).toMatchObject({ kind: 'usage', tokens: 5050 })
|
||||
expect(after.surfaceDeltaTokens - before.surfaceDeltaTokens).toBe(routedMessageTokens(message))
|
||||
expect(after.totalTokens).toBe(5050 + after.surfaceDeltaTokens)
|
||||
})
|
||||
|
||||
it('reprices the surface under the substitution pricing of a text-only route', async () => {
|
||||
const placeholder = '[image omitted for the text-only route]'
|
||||
const substitution: LlmImageRequestPricing = {
|
||||
priceImages: images => images.map(() => ({ visualTokens: 0, text: placeholder })),
|
||||
}
|
||||
const { meter, session } = await harness(model => (model === 'vision' ? fixedPricing : substitution))
|
||||
const message = imageMessage('photo')
|
||||
session.append('user/message', message, { surfaceOp: 'append' })
|
||||
session.append('request/header', { header: header('vision'), reason: 'initial' })
|
||||
|
||||
const textOnly = meter.measure(session, header('text-only'))
|
||||
const imageFree = estimateMessage({
|
||||
...message,
|
||||
content: message.content.filter(block => block.type !== 'image'),
|
||||
})
|
||||
expect(textOnly.nodes[0]!.tokens)
|
||||
.toBe(imageFree + estimateContent([{ type: 'text', text: placeholder }]))
|
||||
expect(textOnly.totalTokens).toBeLessThan(meter.measure(session).totalTokens)
|
||||
})
|
||||
|
||||
it('keeps the fixed heuristic for routes and services that declare no pricing', async () => {
|
||||
const { meter, session } = await harness(() => undefined)
|
||||
const message = imageMessage('photo')
|
||||
session.append('user/message', message, { surfaceOp: 'append' })
|
||||
session.append('request/header', { header: header('vision'), reason: 'initial' })
|
||||
const declared = meter.measure(session)
|
||||
expect(declared.nodes[0]!.tokens).toBe(estimateMessage(message))
|
||||
|
||||
const unknownRoute = meter.measure(
|
||||
session,
|
||||
canonicalHeader({ config: { provider: 'unregistered', model: 'any' } }),
|
||||
)
|
||||
expect(unknownRoute.nodes[0]!.tokens).toBe(estimateMessage(message))
|
||||
})
|
||||
|
||||
it('fails loud when a route answers a mismatched occurrence count', async () => {
|
||||
const broken: LlmImageRequestPricing = { priceImages: () => [] }
|
||||
const { meter, session } = await harness(() => broken)
|
||||
session.append('user/message', imageMessage('photo'), { surfaceOp: 'append' })
|
||||
session.append('request/header', { header: header('vision'), reason: 'initial' })
|
||||
expect(() => meter.measure(session))
|
||||
.toThrow('route image pricing answered 0 prices for 1 occurrences')
|
||||
})
|
||||
|
||||
it('prices nested tool-result images through the same route pricing', async () => {
|
||||
const { meter, session } = await harness(() => fixedPricing)
|
||||
const nested = createUserMessage({
|
||||
content: [{
|
||||
type: 'tool-result',
|
||||
toolCallId: 'call-1' as never,
|
||||
content: [
|
||||
{ type: 'text', text: 'screenshot below' },
|
||||
{ type: 'image', attachment: imageRef('nested') },
|
||||
],
|
||||
}],
|
||||
source: { kind: 'user' },
|
||||
})
|
||||
session.append('user/message', nested, { surfaceOp: 'append' })
|
||||
session.append('request/header', { header: header('vision'), reason: 'initial' })
|
||||
const measurement = meter.measure(session)
|
||||
const imageFree = estimateMessage({
|
||||
...nested,
|
||||
content: [{
|
||||
...nested.content[0] as Extract<Message['content'][number], { type: 'tool-result' }>,
|
||||
content: [{ type: 'text', text: 'screenshot below' }],
|
||||
}],
|
||||
})
|
||||
expect(measurement.nodes[0]!.tokens)
|
||||
.toBe(imageFree + VISUAL_TOKENS + estimateContent([{ type: 'text', text: HANDLE_TEXT }]))
|
||||
})
|
||||
})
|
||||
@@ -185,7 +185,8 @@ describe('TokenMeter pricing', () => {
|
||||
expect(Object.isFrozen(snapshot.nodes[0])).toBe(true)
|
||||
expectSurfaceTotal(snapshot)
|
||||
expect(() => {
|
||||
;(snapshot.nodes as Array<{ seq: number; tokens: number }>).push({ seq: 99, tokens: 1 })
|
||||
;(snapshot.nodes as Array<{ seq: number; tokens: number; heuristicTokens: number }>)
|
||||
.push({ seq: 99, tokens: 1, heuristicTokens: 1 })
|
||||
}).toThrow(TypeError)
|
||||
expect(() => {
|
||||
;(snapshot.nodes[0] as { seq: number; tokens: number }).tokens = 1
|
||||
@@ -437,7 +438,7 @@ describe('replay anchors and surface folds', () => {
|
||||
})
|
||||
const measurement = meter().measure(session)
|
||||
const assistant = session.events.find(event => event.type === 'assistant/message')!
|
||||
expect(measurement.nodes).toEqual([{ seq: assistant.seq, tokens: 0 }])
|
||||
expect(measurement.nodes).toEqual([{ seq: assistant.seq, tokens: 0, heuristicTokens: 0 }])
|
||||
expect(measurement.surfaceTokens).toBe(0)
|
||||
expectSurfaceTotal(measurement)
|
||||
})
|
||||
@@ -486,12 +487,11 @@ describe('malformed replay and listener lifecycle', () => {
|
||||
}, { surfaceOp: 'append', sourceEventSeqs: [] })
|
||||
const service = meter()
|
||||
const states = (service as unknown as {
|
||||
states: WeakMap<Session, { surface: unknown[]; surfaceTokens: number }>
|
||||
states: WeakMap<Session, { surface: unknown[] }>
|
||||
}).states
|
||||
expectRepeatedFailure(service, session, /no matching step\/start/)
|
||||
const state = states.get(session)
|
||||
expect(state?.surface).toEqual([])
|
||||
expect(state?.surfaceTokens).toBe(0)
|
||||
})
|
||||
|
||||
it('clears completed step boundaries and rejects overlapping or late step events', () => {
|
||||
|
||||
@@ -23,10 +23,6 @@
|
||||
* provider's fixed permission fact.
|
||||
* - `MOCK_CRASH_ON_INITIALIZE` — exit while the unpublished initialize
|
||||
* operation is active.
|
||||
* - `MOCK_CLOSE_PROTOCOL_ON_INITIALIZE` — close stdout while keeping the
|
||||
* process alive, producing initialize-stage transport.
|
||||
* - `MOCK_CLOSE_PROTOCOL_ON_PROMPT` — close stdout while keeping the process
|
||||
* alive, producing a prompt-stage transport failure.
|
||||
* - `MOCK_CRASH_AFTER_CHUNK` — exit after streaming the assistant chunk, so
|
||||
* the parent preserves partial output with process facts.
|
||||
* - `MOCK_ECHO_CWD` — if `1`, ignore MOCK_TEXT and stream two lines instead:
|
||||
@@ -98,10 +94,8 @@ const IGNORE_PERMISSION_DECISION = process.env.MOCK_PERMISSION_IGNORE_DECISION =
|
||||
const NO_ALLOW = process.env.MOCK_NO_ALLOW === '1'
|
||||
const THOUGHT = process.env.MOCK_THOUGHT === '1'
|
||||
const CRASH_ON_INITIALIZE = process.env.MOCK_CRASH_ON_INITIALIZE === '1'
|
||||
const CLOSE_PROTOCOL_ON_INITIALIZE = process.env.MOCK_CLOSE_PROTOCOL_ON_INITIALIZE === '1'
|
||||
const CRASH_ON_CANCEL = process.env.MOCK_CRASH_ON_CANCEL === '1'
|
||||
const CRASH_ON_PROMPT = process.env.MOCK_CRASH_ON_PROMPT === '1'
|
||||
const CLOSE_PROTOCOL_ON_PROMPT = process.env.MOCK_CLOSE_PROTOCOL_ON_PROMPT === '1'
|
||||
const CRASH_AFTER_CHUNK = process.env.MOCK_CRASH_AFTER_CHUNK === '1'
|
||||
const IGNORE_CANCEL = process.env.MOCK_IGNORE_CANCEL === '1'
|
||||
const TOOL_KIND = process.env.MOCK_TOOL_KIND as ToolKind | undefined
|
||||
@@ -123,11 +117,6 @@ function makeAgent() {
|
||||
return {
|
||||
initialize(_params: InitializeRequest): Promise<InitializeResponse> {
|
||||
if (CRASH_ON_INITIALIZE) process.exit(11)
|
||||
if (CLOSE_PROTOCOL_ON_INITIALIZE) {
|
||||
process.stdout.end()
|
||||
setInterval(() => { /* keep the process alive after protocol EOF */ }, 1000)
|
||||
return new Promise<InitializeResponse>(() => {})
|
||||
}
|
||||
return Promise.resolve({
|
||||
protocolVersion: PROTOCOL_VERSION,
|
||||
agentCapabilities: { promptCapabilities: { image: false, audio: false, embeddedContext: false } },
|
||||
@@ -152,11 +141,6 @@ function makeAgent() {
|
||||
},
|
||||
async prompt(params: PromptRequest, conn: AgentContext): Promise<PromptResponse> {
|
||||
if (CRASH_ON_PROMPT) process.exit(1)
|
||||
if (CLOSE_PROTOCOL_ON_PROMPT) {
|
||||
process.stdout.end()
|
||||
setInterval(() => { /* keep the process alive after protocol EOF */ }, 1000)
|
||||
return new Promise<PromptResponse>(() => {})
|
||||
}
|
||||
if (WANT_PERMISSION) {
|
||||
// Ask the client to approve before answering; honor its decision. Under
|
||||
// MOCK_NO_ALLOW the only options are reject-shaped, so an `allow`-policy
|
||||
|
||||
@@ -4,6 +4,7 @@ import Loader from '@deepseek-ai/cordis-plugin-loader'
|
||||
import { chmodSync, existsSync, mkdtempSync, realpathSync, rmSync, writeFileSync } from 'node:fs'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join, resolve } from 'node:path'
|
||||
import { PassThrough, type Readable } from 'node:stream'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
import SubagentRuntime from '@deepseek-ai/dsh-subagent'
|
||||
import type { Agent } from '@deepseek-ai/dsh-agent'
|
||||
@@ -121,6 +122,63 @@ function tapBoundedExitWait(child: SubprocessHandle, onWait: () => void): Subpro
|
||||
}
|
||||
}
|
||||
|
||||
function replaceProtocolStreams(
|
||||
child: SubprocessHandle,
|
||||
stdin: PassThrough,
|
||||
stdout: Readable,
|
||||
): SubprocessHandle {
|
||||
if (child.stdin === undefined) throw new Error('expected piped child stdin')
|
||||
stdin.pipe(child.stdin)
|
||||
return {
|
||||
pid: child.pid,
|
||||
stdin,
|
||||
stdout,
|
||||
stderr: child.stderr,
|
||||
collected: child.collected,
|
||||
done: child.done,
|
||||
terminate: () => { child.terminate() },
|
||||
waitForExit: (signal?: AbortSignal) => child.waitForExit(signal),
|
||||
}
|
||||
}
|
||||
|
||||
function closeProtocolImmediately(child: SubprocessHandle): SubprocessHandle {
|
||||
const stdout = new PassThrough()
|
||||
stdout.end()
|
||||
return replaceProtocolStreams(child, new PassThrough(), stdout)
|
||||
}
|
||||
|
||||
function closeProtocolOnPrompt(child: SubprocessHandle, onClose: () => void = () => {}): SubprocessHandle {
|
||||
if (child.stdout === undefined) throw new Error('expected piped child stdout')
|
||||
const stdin = new PassThrough()
|
||||
const stdout = new PassThrough()
|
||||
child.stdout.pipe(stdout)
|
||||
let requestText = ''
|
||||
let closed = false
|
||||
stdin.on('data', (chunk: Buffer) => {
|
||||
if (closed) return
|
||||
requestText += chunk.toString('utf8')
|
||||
if (!requestText.includes('"session/prompt"')) return
|
||||
closed = true
|
||||
child.stdout?.unpipe(stdout)
|
||||
stdout.end()
|
||||
onClose()
|
||||
})
|
||||
return replaceProtocolStreams(child, stdin, stdout)
|
||||
}
|
||||
|
||||
function replaceProcessOutcome(child: SubprocessHandle, outcome: SubprocessOutcome): SubprocessHandle {
|
||||
return {
|
||||
pid: child.pid,
|
||||
stdin: child.stdin,
|
||||
stdout: child.stdout,
|
||||
stderr: child.stderr,
|
||||
collected: child.collected,
|
||||
done: child.done.then(() => outcome),
|
||||
terminate: () => { child.terminate() },
|
||||
waitForExit: (signal?: AbortSignal) => child.waitForExit(signal),
|
||||
}
|
||||
}
|
||||
|
||||
describe('acpStopReason', () => {
|
||||
it('maps each ACP stop reason to the harness vocabulary', () => {
|
||||
expect(acpStopReason('end_turn')).toBe('completed')
|
||||
@@ -589,18 +647,16 @@ describe('dsh-subagent-acp', () => {
|
||||
)
|
||||
})
|
||||
|
||||
// Windows anonymous pipes do not surface a child stdout half-close while
|
||||
// the child stays alive.
|
||||
it.skipIf(process.platform === 'win32')('reports initialize-stage transport when the child closes the protocol but stays alive', async () => {
|
||||
it('reports initialize-stage transport when the child closes the protocol but stays alive', async () => {
|
||||
const error = await startAcpRun(request(), {
|
||||
command: process.execPath,
|
||||
args: [mockServer],
|
||||
cwd: process.cwd(),
|
||||
permission: 'reject',
|
||||
env: { MOCK_CLOSE_PROTOCOL_ON_INITIALIZE: '1' },
|
||||
env: {},
|
||||
disposeEofGraceMs: 50,
|
||||
disposeGraceMs: 50,
|
||||
spawn: spawnSubprocess,
|
||||
spawn: spec => closeProtocolImmediately(spawnSubprocess(spec)),
|
||||
}).catch((cause: unknown) => cause)
|
||||
expect(error).toBeInstanceOf(Error)
|
||||
expect((error as Error).message).toBe(
|
||||
@@ -938,18 +994,16 @@ describe('dsh-subagent-acp', () => {
|
||||
await run.dispose()
|
||||
})
|
||||
|
||||
// Windows anonymous pipes do not surface a child stdout half-close while
|
||||
// the child stays alive.
|
||||
it.skipIf(process.platform === 'win32')('classifies a prompt transport failure without copying SDK text', async () => {
|
||||
it('classifies a prompt transport failure without copying SDK text', async () => {
|
||||
const run = await startAcpRun(request('private prompt text'), {
|
||||
command: process.execPath,
|
||||
args: [mockServer],
|
||||
cwd: process.cwd(),
|
||||
permission: 'reject',
|
||||
env: { MOCK_CLOSE_PROTOCOL_ON_PROMPT: '1' },
|
||||
env: { MOCK_HANG: '1' },
|
||||
disposeEofGraceMs: 100,
|
||||
disposeGraceMs: 100,
|
||||
spawn: spawnSubprocess,
|
||||
spawn: spec => closeProtocolOnPrompt(spawnSubprocess(spec)),
|
||||
})
|
||||
const result = await run.result
|
||||
expect(result).toEqual({
|
||||
@@ -961,9 +1015,7 @@ describe('dsh-subagent-acp', () => {
|
||||
await run.dispose()
|
||||
})
|
||||
|
||||
// Windows anonymous pipes do not surface a child stdout half-close while
|
||||
// the child stays alive.
|
||||
it.skipIf(process.platform === 'win32')('lets local cancellation interrupt prompt-failure process observation', async () => {
|
||||
it('lets local cancellation interrupt prompt-failure process observation', async () => {
|
||||
const controller = new AbortController()
|
||||
const protocolEnded = Promise.withResolvers<undefined>()
|
||||
let boundedExitWaits = 0
|
||||
@@ -972,13 +1024,15 @@ describe('dsh-subagent-acp', () => {
|
||||
args: [mockServer],
|
||||
cwd: process.cwd(),
|
||||
permission: 'reject',
|
||||
env: { MOCK_CLOSE_PROTOCOL_ON_PROMPT: '1' },
|
||||
env: { MOCK_HANG: '1' },
|
||||
disposeEofGraceMs: 100,
|
||||
disposeGraceMs: 5000,
|
||||
spawn: (spec) => {
|
||||
const child = spawnSubprocess(spec)
|
||||
child.stdout?.once('end', () => { protocolEnded.resolve(undefined) })
|
||||
return tapBoundedExitWait(child, () => { boundedExitWaits += 1 })
|
||||
return closeProtocolOnPrompt(
|
||||
tapBoundedExitWait(child, () => { boundedExitWaits += 1 }),
|
||||
() => { protocolEnded.resolve(undefined) },
|
||||
)
|
||||
},
|
||||
})
|
||||
await protocolEnded.promise
|
||||
@@ -1006,6 +1060,29 @@ describe('dsh-subagent-acp', () => {
|
||||
await run.dispose()
|
||||
})
|
||||
|
||||
it('reports a signal-only process outcome', async () => {
|
||||
const run = await startAcpRun(request(), {
|
||||
command: process.execPath,
|
||||
args: [mockServer],
|
||||
cwd: process.cwd(),
|
||||
permission: 'reject',
|
||||
env: { MOCK_CRASH_AFTER_CHUNK: '1' },
|
||||
disposeEofGraceMs: DEFAULT_DISPOSE_EOF_GRACE_MS,
|
||||
disposeGraceMs: DEFAULT_DISPOSE_GRACE_MS,
|
||||
spawn: spec => replaceProcessOutcome(
|
||||
spawnSubprocess(spec),
|
||||
{ exitCode: null, signal: 'SIGTERM' },
|
||||
),
|
||||
})
|
||||
const result = await run.result
|
||||
expect(result).toEqual({
|
||||
output: [{ type: 'text', text: 'mock child answer' }],
|
||||
diagnostic: expectedFailure('stage: process; category: process-exit; signal: SIGTERM'),
|
||||
stopReason: 'error',
|
||||
})
|
||||
await run.dispose()
|
||||
})
|
||||
|
||||
it('rejects a spawn failure after provider-owned cleanup', async () => {
|
||||
const privateCommand = '/nonexistent/private/SECRET_TOKEN/acp-agent'
|
||||
const error = await startAcpRun(
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/test-support/llm-replay/README.md
|
||||
README.md: e12d359950e1caf6e31b9d25035c50bb83c73747
|
||||
README.zh.md: 83964fed15c6a985df8a92327748918c1616a83e
|
||||
README.md: 5e2354cb3ae7ad0ffca6a85c461c7d4b24d8ed31
|
||||
README.zh.md: fb2e927fce17e13ed97c49110f9ae6558f117e9d
|
||||
|
||||
@@ -31,7 +31,7 @@ Replay keys every call by its calling session id (`GenerateOptions.sessionId`, s
|
||||
| `file` | string | `$DSH_SNAPSHOT_FILE` | Path to the primary (parent) `session.jsonl` fixture. Required (config or env). |
|
||||
| `overrideFile` | string | `$DSH_SNAPSHOT_OVERRIDE` | Optional `ReplayOverrideDoc` sidecar for the primary session: a bare `ReplayEntry[]` replaces its derived script, while `{ patches }` augments it by call index. |
|
||||
| `childFiles` | string[] | `$DSH_SNAPSHOT_CHILD_FILES` (path-delimited) | Recorded subagent child-session logs for a nested scenario; empty for a single-session scenario. |
|
||||
| `providers` | `ReplayProviderConfig[]` | — | Optional replay-only provider and model catalog. Each provider may set `retryPolicy`, and each model may publish `contextWindow` and an `inputModalities` array containing only `text` and `image`; invalid modalities fail during plugin loading. Configured routes dispatch through the replay adapter and never perform provider I/O. |
|
||||
| `providers` | `ReplayProviderConfig[]` | — | Optional replay-only provider and model catalog. Each provider may set `retryPolicy`, and each model may publish `contextWindow`, an `inputModalities` array containing only `text` and `image`, and a positive-integer `imageRequestTokens` flat visual-token price the route declares for every retained request image (its model must also declare the `image` modality); invalid modalities, a non-positive price, or visual pricing on a text-only model fail during plugin loading. Configured routes dispatch through the replay adapter and never perform provider I/O. |
|
||||
| `paceMs` | number | — (burst) | Optional per-chunk delay in ms so downstream transports (e.g. the web SSE mux observed by a real browser) see genuinely incremental delivery. A realism knob only — tests must not depend on it for correctness. Non-negative integer; abort during a pace wait cancels the stream promptly. |
|
||||
|
||||
```yaml
|
||||
|
||||
@@ -31,7 +31,7 @@ fixture 是持久化会话日志（`<scenario>/session.jsonl`）的投影：它
|
||||
| `file` | string | `$DSH_SNAPSHOT_FILE` | 主（父）`session.jsonl` fixture 的路径。必需（配置或 env）。 |
|
||||
| `overrideFile` | string | `$DSH_SNAPSHOT_OVERRIDE` | 主会话的可选 `ReplayOverrideDoc` 伴随文件：裸 `ReplayEntry[]` 替换其派生脚本，`{ patches }` 则按调用索引增补该脚本。 |
|
||||
| `childFiles` | string[] | `$DSH_SNAPSHOT_CHILD_FILES`（以路径分隔符分隔） | 嵌套场景中已记录的 subagent 子会话日志；单会话场景为空。 |
|
||||
| `providers` | `ReplayProviderConfig[]` | 无 | 可选的仅回放提供方和模型目录。每个提供方可以设置 `retryPolicy`，每个模型可以发布 `contextWindow` 和仅包含 `text`、`image` 的 `inputModalities` 数组；模态配置无效时，插件加载会失败。已配置路由通过回放适配器分派，绝不执行提供方 I/O。 |
|
||||
| `providers` | `ReplayProviderConfig[]` | 无 | 可选的仅回放提供方和模型目录。每个提供方可以设置 `retryPolicy`，每个模型可以发布 `contextWindow`、仅包含 `text`、`image` 的 `inputModalities` 数组，以及正整数 `imageRequestTokens`（该路由为每张保留请求图片声明的固定视觉 token 价格，其模型必须同时声明 `image` 模态）；模态配置无效、价格非正或在纯文本模型上声明视觉定价时，插件加载会失败。已配置路由通过回放适配器分派，绝不执行提供方 I/O。 |
|
||||
| `paceMs` | number | 无（突发） | 可选的每分片延迟（单位为毫秒），使下游传输（例如真实浏览器观察到的 Web SSE（Server-Sent Events）多路复用器）看到真正的增量传递。它只是用于提高真实性的调节项，测试不得依赖它保证正确性。值必须是非负整数；pace 等待期间中止会迅速取消流。 |
|
||||
|
||||
```yaml
|
||||
|
||||
@@ -16,6 +16,7 @@ import { decodeStorageRecord, type SessionEvent } from '@deepseek-ai/dsh-session
|
||||
import type {
|
||||
ContentBlock,
|
||||
GenerateOptions,
|
||||
LlmImageRequestPricing,
|
||||
LlmModelInfo,
|
||||
LlmProviderInfo,
|
||||
LlmResolvedModelInfo,
|
||||
@@ -25,7 +26,7 @@ import type {
|
||||
StreamChunk,
|
||||
TokenUsage,
|
||||
} from '@deepseek-ai/dsh-llm'
|
||||
import { LlmAdapter, LlmError, ReasoningEffortId, assertNever, resolveRetryPolicy } from '@deepseek-ai/dsh-llm'
|
||||
import { LlmAdapter, LlmError, ReasoningEffortId, assertNever, requestImageHandleText, resolveRetryPolicy } from '@deepseek-ai/dsh-llm'
|
||||
|
||||
const PACKED_CHUNK_ROW_TYPES = new Set(['text-chunks', 'reasoning-chunks', 'tool-call-chunks'])
|
||||
|
||||
@@ -61,6 +62,15 @@ export interface ReplayModelConfig {
|
||||
* omit one, so replay reconstructs the request header a live catalog produced.
|
||||
*/
|
||||
defaultMaxTokens?: number
|
||||
/**
|
||||
* Optional flat visual-token price the replay route declares for every
|
||||
* retained request image, so keyless scenarios exercise route-priced
|
||||
* request pressure; each occurrence is priced at this value plus its
|
||||
* request-preview handle text. Requires {@link inputModalities} to include
|
||||
* `image` — a text-only route never sends visual tokens. Absent declares
|
||||
* no image pricing.
|
||||
*/
|
||||
imageRequestTokens?: number
|
||||
/** Optional reasoning-effort ids the replay route accepts, in display order. */
|
||||
reasoningEfforts?: string[]
|
||||
/**
|
||||
@@ -658,6 +668,18 @@ class ReplayAdapter extends LlmAdapter {
|
||||
: resolveRetryPolicy(configured.retryPolicy, `llm-replay: provider "${provider}" retryPolicy`)
|
||||
}
|
||||
|
||||
override imageRequestPricing(provider: string, model: string): LlmImageRequestPricing | undefined {
|
||||
const configured = this.providers.get(provider)
|
||||
const visualTokens = configured?.models?.find(candidate => candidate.id === model)?.imageRequestTokens
|
||||
if (visualTokens === undefined) return undefined
|
||||
return {
|
||||
priceImages: images => images.map(ref => ({
|
||||
visualTokens,
|
||||
text: requestImageHandleText(ref, { width: ref.width, height: ref.height }),
|
||||
})),
|
||||
}
|
||||
}
|
||||
|
||||
override listModels(provider: string): Promise<readonly LlmModelInfo[]> {
|
||||
const configured = this.providers.get(provider)
|
||||
/* v8 ignore next -- LlmRuntime only asks about routes registered from this same map. */
|
||||
@@ -906,18 +928,34 @@ export interface Config {
|
||||
paceMs?: number
|
||||
}
|
||||
|
||||
function validateConfiguredModalities(providers: ReplayProviderConfig[] | undefined): void {
|
||||
function validateConfiguredModels(providers: ReplayProviderConfig[] | undefined): void {
|
||||
for (const provider of providers ?? []) {
|
||||
for (const model of provider.models ?? []) {
|
||||
const modalities: unknown = model.inputModalities
|
||||
if (modalities === undefined) continue
|
||||
if (!Array.isArray(modalities)
|
||||
|| !modalities.every((modality: unknown) => modality === 'text' || modality === 'image')) {
|
||||
if (modalities !== undefined && (!Array.isArray(modalities)
|
||||
|| !modalities.every((modality: unknown) => modality === 'text' || modality === 'image'))) {
|
||||
throw new Error(
|
||||
`llm-replay: provider "${provider.id}" model "${model.id}" inputModalities `
|
||||
+ 'must be an array containing only "text" and "image"',
|
||||
)
|
||||
}
|
||||
const imageRequestTokens: unknown = model.imageRequestTokens
|
||||
if (imageRequestTokens !== undefined
|
||||
&& (!Number.isSafeInteger(imageRequestTokens) || (imageRequestTokens as number) <= 0)) {
|
||||
throw new Error(
|
||||
`llm-replay: provider "${provider.id}" model "${model.id}" imageRequestTokens `
|
||||
+ 'must be a positive safe integer',
|
||||
)
|
||||
}
|
||||
// A text-only route never sends visual tokens: LlmRuntime substitutes
|
||||
// its images with deterministic text before dispatch, so declared
|
||||
// visual pricing would contradict the actual request projection.
|
||||
if (imageRequestTokens !== undefined && model.inputModalities?.includes('image') !== true) {
|
||||
throw new Error(
|
||||
`llm-replay: provider "${provider.id}" model "${model.id}" imageRequestTokens `
|
||||
+ 'requires inputModalities to include "image"',
|
||||
)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -927,7 +965,7 @@ export function apply(ctx: Context, config: Config = {}): void {
|
||||
if (file === undefined || file.length === 0) {
|
||||
throw new Error('llm-replay: a fixture path is required (Config.file or $DSH_SNAPSHOT_FILE)')
|
||||
}
|
||||
validateConfiguredModalities(config.providers)
|
||||
validateConfiguredModels(config.providers)
|
||||
const overrideFile = config.overrideFile ?? process.env.DSH_SNAPSHOT_OVERRIDE
|
||||
const childEnv = process.env.DSH_SNAPSHOT_CHILD_FILES
|
||||
const childFiles = config.childFiles
|
||||
|
||||
@@ -1329,6 +1329,56 @@ describe('apply (the plugin entry)', () => {
|
||||
expect(await drain(ctx.llm.stream({ provider: 'm', model: 'm', messages: [] }))).toEqual(TEXT_CHUNKS)
|
||||
})
|
||||
|
||||
it('declares flat image request pricing only for models that configure it', async () => {
|
||||
writeFileSync(file, sessionJsonl(TEXT_CHUNKS.map((c, i) => chunkEvent(i + 1, 1, 1, c))), 'utf8')
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmRuntime)
|
||||
installLlmReplay(ctx, {
|
||||
file,
|
||||
providers: [{
|
||||
id: 'deepseek',
|
||||
models: [
|
||||
{ id: 'vision', inputModalities: ['text', 'image'], imageRequestTokens: 384 },
|
||||
{ id: 'plain' },
|
||||
],
|
||||
}],
|
||||
})
|
||||
const pricing = ctx.llm.imageRequestPricing('deepseek', 'vision')
|
||||
expect(pricing).toBeDefined()
|
||||
const ref = {
|
||||
attachmentId: 'sha256:aaaaaaaa',
|
||||
mediaType: 'image/png',
|
||||
bytes: 10,
|
||||
width: 640,
|
||||
height: 480,
|
||||
} as never
|
||||
const priced = pricing?.priceImages([ref, ref])
|
||||
expect(priced?.map(price => price.visualTokens)).toEqual([384, 384])
|
||||
expect(priced?.every(price => price.text.includes('640x480px'))).toBe(true)
|
||||
expect(ctx.llm.imageRequestPricing('deepseek', 'plain')).toBeUndefined()
|
||||
})
|
||||
|
||||
it('rejects imageRequestTokens on a model without the image modality during load', () => {
|
||||
const ctx = new Context()
|
||||
const providers = [{ id: 'm', models: [{ id: 'm', imageRequestTokens: 384 }] }] as unknown as
|
||||
NonNullable<Config['providers']>
|
||||
expect(() => { apply(ctx, { file, providers }) }).toThrow(
|
||||
'llm-replay: provider "m" model "m" imageRequestTokens requires inputModalities to include "image"',
|
||||
)
|
||||
})
|
||||
|
||||
it.each([
|
||||
['zero', 0],
|
||||
['a float', 1.5],
|
||||
])('rejects imageRequestTokens configured as %s during load', (_case, imageRequestTokens) => {
|
||||
const ctx = new Context()
|
||||
const providers = [{ id: 'm', models: [{ id: 'm', imageRequestTokens }] }] as unknown as
|
||||
NonNullable<Config['providers']>
|
||||
expect(() => { apply(ctx, { file, providers }) }).toThrow(
|
||||
'llm-replay: provider "m" model "m" imageRequestTokens must be a positive safe integer',
|
||||
)
|
||||
})
|
||||
|
||||
it.each([
|
||||
['a string', 'image'],
|
||||
['an unknown modality', ['audio']],
|
||||
|
||||
@@ -257,6 +257,7 @@ export const LINK_MAP: Readonly<Record<string, string>> = {
|
||||
LlmModelReasoningInfo: 'llm-streaming.md',
|
||||
LlmResolvedModelInfo: 'llm-streaming.md',
|
||||
LlmFailure: 'llm-streaming.md',
|
||||
LlmImageRequestPricing: 'llm-streaming.md',
|
||||
LlmModelInfo: 'llm-streaming.md',
|
||||
LlmProviderInfo: 'llm-streaming.md',
|
||||
LlmConfigurableProvider: 'llm-streaming.md',
|
||||
|
||||
@@ -402,6 +402,16 @@
|
||||
"symbol": "TokenUsage",
|
||||
"source": "packages/llm/llm/src/types.ts"
|
||||
},
|
||||
{
|
||||
"doc": "docs/subsystems/llm-streaming.md",
|
||||
"symbol": "LlmImageRequestPrice",
|
||||
"source": "packages/llm/llm/src/types.ts"
|
||||
},
|
||||
{
|
||||
"doc": "docs/subsystems/llm-streaming.md",
|
||||
"symbol": "LlmImageRequestPricing",
|
||||
"source": "packages/llm/llm/src/types.ts"
|
||||
},
|
||||
{
|
||||
"doc": "docs/subsystems/llm-streaming.md",
|
||||
"symbol": "ContentBlockMap",
|
||||
|
||||
@@ -23,7 +23,11 @@ function snapshotMode(value: string | undefined): SnapshotSuiteOptions['mode'] {
|
||||
}
|
||||
}
|
||||
|
||||
const controllerCases = [
|
||||
const controllerCases: readonly {
|
||||
readonly name: string
|
||||
readonly hasModelTurn: boolean
|
||||
readonly configPath?: string
|
||||
}[] = [
|
||||
{ name: 'handshake', hasModelTurn: false },
|
||||
{ name: 'reject-extra-dirs', hasModelTurn: false },
|
||||
{ name: 'cancel', hasModelTurn: true },
|
||||
@@ -31,14 +35,25 @@ const controllerCases = [
|
||||
{ name: 'escalation-approved', hasModelTurn: true },
|
||||
{ name: 'escalation-rejected', hasModelTurn: true },
|
||||
{ name: 'fs-escalation-approved', hasModelTurn: true },
|
||||
{
|
||||
name: 'image-compaction',
|
||||
hasModelTurn: true,
|
||||
configPath: join(corpusDir, 'image-compaction', 'cordis.yml'),
|
||||
},
|
||||
] as const
|
||||
|
||||
function localScenarioSource(source: string | undefined): string | undefined {
|
||||
return source?.includes('/') === false ? source : undefined
|
||||
}
|
||||
|
||||
const scenarios: Scenario[] = controllerCases.map((controller) => {
|
||||
const manifestPath = join(corpusDir, controller.name, 'snapshot.yml')
|
||||
const manifest = parseSnapshotManifest(readFileSync(manifestPath, 'utf8'), manifestPath)
|
||||
if (manifest.recording === undefined || manifest.header === undefined) {
|
||||
throw new Error(`${controller.name}: ACP snapshot manifest lacks recording or header metadata`)
|
||||
}
|
||||
const systemPromptSource = localScenarioSource(manifest.header.systemPromptSource)
|
||||
const toolSchemasSource = localScenarioSource(manifest.header.toolSchemasSource)
|
||||
return {
|
||||
...controller,
|
||||
recorded: manifest.recording === 'live',
|
||||
@@ -46,8 +61,11 @@ const scenarios: Scenario[] = controllerCases.map((controller) => {
|
||||
...(manifest.header.pin === true ? { pinsHeader: true } : {}),
|
||||
...(manifest.header.changes === undefined ? {} : { expectedHeaderChanges: manifest.header.changes }),
|
||||
headerClass: manifest.header.class,
|
||||
...(systemPromptSource === undefined ? {} : { systemPromptSource }),
|
||||
...(toolSchemasSource === undefined ? {} : { toolSchemasSource }),
|
||||
...(manifest.platform === 'posix' ? { posixOnly: true } : {}),
|
||||
...(manifest.platform === 'pwsh' ? { pwshOnly: true } : {}),
|
||||
...(controller.configPath === undefined ? {} : { configPath: controller.configPath }),
|
||||
...manifest.permission === undefined && manifest.environment === undefined
|
||||
? {}
|
||||
: {
|
||||
|
||||
@@ -0,0 +1,64 @@
|
||||
# Keyless replay for the image-compaction scenario. This profile patch swaps
|
||||
# the adapter and re-pins the recorded vision model. The replay catalog declares image input,
|
||||
# so the strict read_image gate accepts the route and the tool result carries
|
||||
# the durable image block.
|
||||
- id: llm-deepseek
|
||||
name: '@deepseek-ai/dsh-llm-deepseek'
|
||||
disabled: true
|
||||
|
||||
- id: acp
|
||||
name: '@deepseek-ai/dsh-acp'
|
||||
config:
|
||||
provider: deepseek-official
|
||||
model: deepseek-v4-flash-vision-exp
|
||||
|
||||
- id: session-persistence-jsonl
|
||||
name: '@deepseek-ai/dsh-session-persistence-jsonl'
|
||||
config:
|
||||
root: !!js process.env.DSH_SNAPSHOT_SESSIONS_ROOT ?? './.sessions'
|
||||
compression: none
|
||||
|
||||
- id: agent-instructions
|
||||
name: '@deepseek-ai/dsh-agent-instructions'
|
||||
config:
|
||||
maxBytes: 65536
|
||||
|
||||
- id: system-prompt
|
||||
name: '@deepseek-ai/dsh-system-prompt'
|
||||
config:
|
||||
persona: |
|
||||
You are a coding assistant powered by the {{model}} model. Your working directory is {{cwd}}. Your bash tool runs under a file sandbox — a `[sandbox: file access denied …]` result is policy, not a command bug.
|
||||
|
||||
Verify your work by running the code or tests. Keep answers brief and factual.
|
||||
|
||||
- insert:
|
||||
- id: llm-replay
|
||||
name: '@deepseek-ai/dsh-llm-replay'
|
||||
config:
|
||||
providers:
|
||||
- id: deepseek-official
|
||||
name: DeepSeek
|
||||
models:
|
||||
- id: deepseek-v4-flash
|
||||
inputModalities: [text]
|
||||
- id: deepseek-v4-pro
|
||||
inputModalities: [text]
|
||||
- id: deepseek-v4-flash-vision-exp
|
||||
inputModalities: [text, image]
|
||||
# Route-priced request images at the provider cap; the small
|
||||
# context window turns that visual pressure into an automatic
|
||||
# compaction the text-only heuristic would not trigger.
|
||||
contextWindow: 11600
|
||||
imageRequestTokens: 384
|
||||
|
||||
- id: attachment-local
|
||||
name: '@deepseek-ai/dsh-attachment-local'
|
||||
|
||||
# Tight automatic compaction budget sized to the route-priced visual tokens of
|
||||
# the scenario's inline images: the text-only history stays under the
|
||||
# threshold, so a triggered compaction proves the routed model's image pricing
|
||||
# drove the pressure decision.
|
||||
- id: compaction-basic
|
||||
name: '@deepseek-ai/dsh-compaction-basic'
|
||||
config:
|
||||
retainTokens: 100
|
||||
@@ -0,0 +1,42 @@
|
||||
# Image-compaction overlay: the image scenario plus a compaction budget the
|
||||
# scenario's inline images exceed only under route-priced visual tokens.
|
||||
# Adds the durable attachment store the read_image tool
|
||||
# commits through. The store resolves its root from $DSH_HOME, which the
|
||||
# snapshot harness scopes per run, so the patch itself carries no attachment
|
||||
# path. The ACP row selects the shipped vision model.
|
||||
- id: acp
|
||||
name: '@deepseek-ai/dsh-acp'
|
||||
config:
|
||||
provider: deepseek-official
|
||||
model: deepseek-v4-flash-vision-exp
|
||||
|
||||
- id: session-persistence-jsonl
|
||||
name: '@deepseek-ai/dsh-session-persistence-jsonl'
|
||||
config:
|
||||
root: !!js process.env.DSH_SNAPSHOT_SESSIONS_ROOT ?? './.sessions'
|
||||
compression: none
|
||||
|
||||
- id: agent-instructions
|
||||
name: '@deepseek-ai/dsh-agent-instructions'
|
||||
config:
|
||||
maxBytes: 65536
|
||||
|
||||
- id: system-prompt
|
||||
name: '@deepseek-ai/dsh-system-prompt'
|
||||
config:
|
||||
persona: |
|
||||
You are a coding assistant powered by the {{model}} model. Your working directory is {{cwd}}. Your bash tool runs under a file sandbox — a `[sandbox: file access denied …]` result is policy, not a command bug.
|
||||
|
||||
Verify your work by running the code or tests. Keep answers brief and factual.
|
||||
|
||||
- id: attachment-local
|
||||
name: '@deepseek-ai/dsh-attachment-local'
|
||||
|
||||
# Tight automatic compaction budget sized to the route-priced visual tokens of
|
||||
# the scenario's inline images: the text-only history stays under the
|
||||
# threshold, so a triggered compaction proves the routed model's image pricing
|
||||
# drove the pressure decision.
|
||||
- id: compaction-basic
|
||||
name: '@deepseek-ai/dsh-compaction-basic'
|
||||
config:
|
||||
retainTokens: 100
|
||||
@@ -0,0 +1,86 @@
|
||||
{
|
||||
"steps": [
|
||||
{
|
||||
"op": "initialize"
|
||||
},
|
||||
{
|
||||
"op": "newSession"
|
||||
},
|
||||
{
|
||||
"op": "promptContent",
|
||||
"content": [
|
||||
{
|
||||
"type": "text",
|
||||
"text": "Here are six reference screenshots of the dashboard: "
|
||||
},
|
||||
{
|
||||
"type": "image",
|
||||
"data": "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAADElEQVR4nGP4z8AAAAMBAQDJ/pLvAAAAAElFTkSuQmCC",
|
||||
"mimeType": "image/png"
|
||||
},
|
||||
{
|
||||
"type": "text",
|
||||
"text": " (frame 1) "
|
||||
},
|
||||
{
|
||||
"type": "image",
|
||||
"data": "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAADElEQVR4nGP4z8AAAAMBAQDJ/pLvAAAAAElFTkSuQmCC",
|
||||
"mimeType": "image/png"
|
||||
},
|
||||
{
|
||||
"type": "text",
|
||||
"text": " (frame 2) "
|
||||
},
|
||||
{
|
||||
"type": "image",
|
||||
"data": "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAADElEQVR4nGP4z8AAAAMBAQDJ/pLvAAAAAElFTkSuQmCC",
|
||||
"mimeType": "image/png"
|
||||
},
|
||||
{
|
||||
"type": "text",
|
||||
"text": " (frame 3) "
|
||||
},
|
||||
{
|
||||
"type": "image",
|
||||
"data": "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAADElEQVR4nGP4z8AAAAMBAQDJ/pLvAAAAAElFTkSuQmCC",
|
||||
"mimeType": "image/png"
|
||||
},
|
||||
{
|
||||
"type": "text",
|
||||
"text": " (frame 4) "
|
||||
},
|
||||
{
|
||||
"type": "image",
|
||||
"data": "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAADElEQVR4nGP4z8AAAAMBAQDJ/pLvAAAAAElFTkSuQmCC",
|
||||
"mimeType": "image/png"
|
||||
},
|
||||
{
|
||||
"type": "text",
|
||||
"text": " (frame 5) "
|
||||
},
|
||||
{
|
||||
"type": "image",
|
||||
"data": "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAADElEQVR4nGP4z8AAAAMBAQDJ/pLvAAAAAElFTkSuQmCC",
|
||||
"mimeType": "image/png"
|
||||
},
|
||||
{
|
||||
"type": "text",
|
||||
"text": " (frame 6) "
|
||||
},
|
||||
{
|
||||
"type": "text",
|
||||
"text": "Acknowledge receipt briefly; we will discuss them next."
|
||||
}
|
||||
]
|
||||
},
|
||||
{
|
||||
"op": "promptContent",
|
||||
"content": [
|
||||
{
|
||||
"type": "text",
|
||||
"text": "Now reply with exactly the single word DONE."
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,37 @@
|
||||
{"type":"session","version":0,"id":"{{session:1}}","createdAt":1783952000000,"cwd":"{{cwd}}","delegationDepth":0}
|
||||
{"type":"permission/preset","data":{"preset":"danger-full-access"}}
|
||||
{"type":"sandbox/mode","data":{"mode":"danger-full-access"}}
|
||||
{"type":"approval/policy","data":{"policy":"never"}}
|
||||
{"type":"agent/inbox/spliced","data":{"target":"next-turn","start":0,"inserted":[{"content":[{"type":"text","text":"Here are six reference screenshots of the dashboard: "},{"type":"image","attachment":{"attachmentId":"sha256:b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640","mediaType":"image/png","width":1,"height":1,"bytes":69}},{"type":"text","text":" (frame 1) "},{"type":"image","attachment":{"attachmentId":"sha256:b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640","mediaType":"image/png","width":1,"height":1,"bytes":69}},{"type":"text","text":" (frame 2) "},{"type":"image","attachment":{"attachmentId":"sha256:b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640","mediaType":"image/png","width":1,"height":1,"bytes":69}},{"type":"text","text":" (frame 3) "},{"type":"image","attachment":{"attachmentId":"sha256:b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640","mediaType":"image/png","width":1,"height":1,"bytes":69}},{"type":"text","text":" (frame 4) "},{"type":"image","attachment":{"attachmentId":"sha256:b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640","mediaType":"image/png","width":1,"height":1,"bytes":69}},{"type":"text","text":" (frame 5) "},{"type":"image","attachment":{"attachmentId":"sha256:b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640","mediaType":"image/png","width":1,"height":1,"bytes":69}},{"type":"text","text":" (frame 6) Acknowledge receipt briefly; we will discuss them next."}],"source":{"kind":"user"},"role":"user","id":"{{message:1}}"}]}}
|
||||
{"type":"turn/start","data":{"turn":1}}
|
||||
{"type":"agent/inbox/spliced","data":{"target":"next-turn","start":0,"removedCount":1,"inserted":[]}}
|
||||
{"type":"step/start","data":{"turn":1,"step":1}}
|
||||
{"type":"user/message","data":{"content":[{"type":"text","text":"Here are six reference screenshots of the dashboard: "},{"type":"image","attachment":{"attachmentId":"sha256:b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640","mediaType":"image/png","width":1,"height":1,"bytes":69}},{"type":"text","text":" (frame 1) "},{"type":"image","attachment":{"attachmentId":"sha256:b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640","mediaType":"image/png","width":1,"height":1,"bytes":69}},{"type":"text","text":" (frame 2) "},{"type":"image","attachment":{"attachmentId":"sha256:b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640","mediaType":"image/png","width":1,"height":1,"bytes":69}},{"type":"text","text":" (frame 3) "},{"type":"image","attachment":{"attachmentId":"sha256:b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640","mediaType":"image/png","width":1,"height":1,"bytes":69}},{"type":"text","text":" (frame 4) "},{"type":"image","attachment":{"attachmentId":"sha256:b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640","mediaType":"image/png","width":1,"height":1,"bytes":69}},{"type":"text","text":" (frame 5) "},{"type":"image","attachment":{"attachmentId":"sha256:b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640","mediaType":"image/png","width":1,"height":1,"bytes":69}},{"type":"text","text":" (frame 6) Acknowledge receipt briefly; we will discuss them next."}],"source":{"kind":"user"},"role":"user","id":"{{message:1}}"},"surfaceOp":"append"}
|
||||
{"type":"user/message","data":{"content":[{"type":"text","text":"Current runtime context. This snapshot supersedes earlier runtime-context snapshots.\n\nCurrent DSH file policy: danger-full-access. The DSH file sandbox does not restrict file modifications by available operations.\n\nApproval prompts are disabled in this session: actions that require approval are rejected automatically — do not request sandbox escalation (do not set `sandbox_permissions`)."}],"source":{"kind":"plugin","plugin":"@deepseek-ai/dsh-system-prompt","form":"snapshot","sections":[{"name":"sandbox:policy","text":"Current DSH file policy: danger-full-access. The DSH file sandbox does not restrict file modifications by available operations."},{"name":"approval:policy","text":"Approval prompts are disabled in this session: actions that require approval are rejected automatically — do not request sandbox escalation (do not set `sandbox_permissions`)."}]},"role":"user","id":"{{message:2}}"},"surfaceOp":"append"}
|
||||
{"type":"session/title","data":{"title":"Here are six reference screenshots","messageSeqs":[7],"source":{"kind":"fallback"}}}
|
||||
{"type":"request/header","data":{"header":{"config":{"provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}}
|
||||
{"type":"request/context","data":{"provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp","contextWindow":11600}}
|
||||
{"type":"assistant/chunk","data":{"turn":1,"step":1,"chunk":{"type":"block-start","index":0,"blockType":"text"}}}
|
||||
{"type":"assistant/chunk","data":{"turn":1,"step":1,"chunk":{"type":"block-end","index":0,"block":{"type":"text","text":"Received six dashboard frames; ready to discuss."}}}}
|
||||
{"type":"assistant/chunk","data":{"turn":1,"step":1,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}}}
|
||||
{"type":"assistant/chunk","data":{"turn":1,"step":1,"chunk":{"type":"finish","reason":{"kind":"stop"}}}}
|
||||
{"type":"assistant/message","data":{"turn":1,"step":1,"message":{"role":"assistant","content":[{"type":"text","text":"Received six dashboard frames; ready to discuss."}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:3}}"},"usage":{"inputTokens":3,"outputTokens":3}},"sourceEventSeqs":[12,13,14,15],"surfaceOp":"append"}
|
||||
{"type":"step/end","data":{"turn":1,"step":1}}
|
||||
{"type":"turn/end","data":{"turn":1,"reason":{"kind":"completed"}}}
|
||||
{"type":"agent/inbox/spliced","data":{"target":"next-turn","start":0,"inserted":[{"content":[{"type":"text","text":"Now reply with exactly the single word DONE."}],"source":{"kind":"user"},"role":"user","id":"{{message:4}}"}]}}
|
||||
{"type":"turn/start","data":{"turn":2}}
|
||||
{"type":"agent/inbox/spliced","data":{"target":"next-turn","start":0,"removedCount":1,"inserted":[]}}
|
||||
{"type":"compaction/start","data":{"compactionId":"{{id:1}}","turn":2}}
|
||||
{"type":"compaction/summary","data":{"compactionId":"{{id:1}}","summary":[{"type":"text","text":"Six identical 1x1 dashboard reference screenshots were shared and acknowledged."}],"rawOutput":[{"type":"text","text":"Six identical 1x1 dashboard reference screenshots were shared and acknowledged."}],"llmStreamCall":true,"shadowedRange":{"start":7,"end":7},"shadowedSeqs":[7],"shadowedTokenCount":366,"provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp","maxTokens":8192,"usage":{"inputTokens":20,"outputTokens":16}}}
|
||||
{"type":"user/message","data":{"content":[{"type":"text","text":"This is an automatically generated checkpoint condensing an earlier span of the conversation to free up context. Treat the captured context as established background and build on it without restating it. Continue the task directly from the messages that follow, without acknowledging this checkpoint.\n\n<compacted-summary>"},{"type":"text","text":"Six identical 1x1 dashboard reference screenshots were shared and acknowledged."},{"type":"text","text":"</compacted-summary>"}],"source":{"kind":"plugin","plugin":"compact","compactionId":"{{id:1}}"},"role":"user","id":"{{message:5}}"},"sourceEventSeqs":[22,23,7],"surfaceOp":{"op":"replace","start":7,"end":7}}
|
||||
{"type":"compaction/end","data":{"compactionId":"{{id:1}}","turn":2}}
|
||||
{"type":"step/start","data":{"turn":2,"step":1}}
|
||||
{"type":"user/message","data":{"content":[{"type":"text","text":"Now reply with exactly the single word DONE."}],"source":{"kind":"user"},"role":"user","id":"{{message:4}}"},"surfaceOp":"append"}
|
||||
{"type":"request/header","data":{"header":{"config":{"provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"system":"{{system}}","tools":"{{tools}}"},"reason":"series"}}
|
||||
{"type":"assistant/chunk","data":{"turn":2,"step":1,"chunk":{"type":"block-start","index":0,"blockType":"text"}}}
|
||||
{"type":"assistant/chunk","data":{"turn":2,"step":1,"chunk":{"type":"block-end","index":0,"block":{"type":"text","text":"DONE"}}}}
|
||||
{"type":"assistant/chunk","data":{"turn":2,"step":1,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}}}
|
||||
{"type":"assistant/chunk","data":{"turn":2,"step":1,"chunk":{"type":"finish","reason":{"kind":"stop"}}}}
|
||||
{"type":"assistant/message","data":{"turn":2,"step":1,"message":{"role":"assistant","content":[{"type":"text","text":"DONE"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"{{message:6}}"},"usage":{"inputTokens":3,"outputTokens":3}},"sourceEventSeqs":[29,30,31,32],"surfaceOp":"append"}
|
||||
{"type":"step/end","data":{"turn":2,"step":1}}
|
||||
{"type":"turn/end","data":{"turn":2,"reason":{"kind":"completed"}}}
|
||||
@@ -0,0 +1,16 @@
|
||||
version: 1
|
||||
scenario: image-compaction
|
||||
profile: acp
|
||||
composition: image-compaction
|
||||
recording: authored
|
||||
header:
|
||||
class: image-compaction
|
||||
pin: true
|
||||
systemPromptSource: session/read-image
|
||||
toolSchemasSource: escalation-approved
|
||||
permission: danger-full-access
|
||||
input:
|
||||
attachments:
|
||||
- id: sha256:b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640
|
||||
mediaType: image/png
|
||||
data: iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAADElFTkSuQmCC
|
||||
@@ -0,0 +1,8 @@
|
||||
{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":1,"agentInfo":{"name":"deepseek-harness-acp","version":"0.0.1"},"agentCapabilities":{"mcpCapabilities":{"http":true},"promptCapabilities":{"image":true,"audio":false,"embeddedContext":false},"sessionCapabilities":{"close":{},"list":{},"resume":{}}},"authMethods":[]}}
|
||||
{"jsonrpc":"2.0","id":2,"result":{"sessionId":"{{sessionId}}","configOptions":[{"id":"model","name":"Model","category":"model","type":"select","currentValue":"[\"deepseek-official\",\"deepseek-v4-flash-vision-exp\"]","options":[{"group":"deepseek-official","name":"DeepSeek","options":[{"value":"[\"deepseek-official\",\"deepseek-v4-flash\"]","name":"deepseek-v4-flash"},{"value":"[\"deepseek-official\",\"deepseek-v4-pro\"]","name":"deepseek-v4-pro"},{"value":"[\"deepseek-official\",\"deepseek-v4-flash-vision-exp\"]","name":"deepseek-v4-flash-vision-exp"}]}]}]}}
|
||||
{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_message_chunk","messageId":"{{messageId}}","content":{"type":"text","text":"Received six dashboard frames; ready to discuss."}}}}
|
||||
{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"usage_update","used":"{{usedTokens}}","size":11600}}}
|
||||
{"jsonrpc":"2.0","id":3,"result":{"stopReason":"end_turn"}}
|
||||
{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_message_chunk","messageId":"{{messageId}}","content":{"type":"text","text":"DONE"}}}}
|
||||
{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"usage_update","used":"{{usedTokens}}","size":11600}}}
|
||||
{"jsonrpc":"2.0","id":4,"result":{"stopReason":"end_turn"}}
|
||||
@@ -0,0 +1 @@
|
||||
../../session/read-image/system-prompt.expected.md
|
||||
Reference in New Issue
Block a user