Merge pull request #2676 from deepseek-harness/worktree/image-management-strategy

feat(attachment): add normalized image and Files API pipeline
This commit is contained in:
CreatixChu
2026-08-21 19:40:51 +08:00
committed by GitHub
145 changed files with 7660 additions and 1065 deletions
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-07-05-reconstructable-requests.md
2026-07-05-reconstructable-requests.md: 63146fa2d392a45543daa32ce2b00158782fddb2
2026-07-05-reconstructable-requests.zh.md: 94c1d323be0107eb8b6072a05d1e8832ebd1fffc
2026-07-05-reconstructable-requests.md: 3f49ba71a6b98a84b05530c900e902b0cf9f6449
2026-07-05-reconstructable-requests.zh.md: 7b8a9df65b60f975bc3ae60b2c1b0c3a8cc22e95
@@ -51,5 +51,6 @@ Like MiniCode, the conversation advances append-only and resets only when model-
- What still costs full price at the provider is inherent and logged: compaction (its `compaction/*` events and replacement entry), a real prompt, tool, or config change (`request/header` with reason `change`), or a process boundary with drift (a differing `resume` snapshot). The provider's own reasoning-content exclusion is managed server-side.
- `agent/pre-step` is the current-request message channel; direct inbox mutation is the eventual later-request channel.
- Tool-result trimming needs no new mechanism: a logged single-entry surface replace (`start === end`) carrying a trimmed `tool/result` under the same `callId` — compaction-family, replay-correct, cache-bust batched by the same pressure logic.
- Unreadable referenced attachment objects still fail model requests; [automatic attachment quarantine](../../proposed/bug-fix/2026-08-20-attachment-read-quarantine.md) records the proposed recovery without weakening byte-exact reconstruction.
- Session logs grow one `request/header` snapshot per loop instance plus snapshots on real changes. This is larger than a delta codec but small beside chunk-heavy logs and retains one replay representation. `SESSION_FORMAT_VERSION` stays `0`; legacy delta events are rejected rather than migrated.
- Snapshot expected outputs changed once (every transcript gains its header events); the fs-writing fixtures are stored in the normalized authored form with cwd-relative tool arguments, because replay only round-trips cwd-independent argument paths.
@@ -51,5 +51,6 @@ Status: implemented
- 在提供方处仍需全价计算的内容是固有的且已记录的:压缩(其 `compaction/*` 事件和替换条目)、真正的提示词、工具或配置变更(reason 为 `change``request/header`),或带漂移的进程边界(不同的 `resume` 快照)。提供方自身的 reasoning-content 排除由服务端管理。
- `agent/pre-step` 是当前请求的消息通道;直接修改 inbox 则是最终进入后续请求的通道。
- 工具结果裁剪无需新机制:一个已记录的单条目 surface replace`start === end`),携带同一 `callId` 下裁剪后的 `tool/result`——属压缩家族,回放正确,缓存失效由相同的压力逻辑批量处理。
- 无法读取的被引用附件对象仍会让模型请求失败;[附件自动隔离](../../proposed/bug-fix/2026-08-20-attachment-read-quarantine.zh.md)记录了不削弱字节精确重建的拟议恢复方案。
- 会话日志每个循环实例增长一个 `request/header` 快照,并在真正变更时增加快照。它比 delta 编解码器更大,但相对分片密集型日志仍然很小,并只保留一种回放表示。`SESSION_FORMAT_VERSION` 保持 `0`;旧的 delta 事件被拒绝而非迁移。
- 快照预期输出变更一次(每个 transcript(文本记录)增加其 header 事件);写入文件系统的 fixture(测试前置数据)以规范化的撰写形式存储,工具参数使用 cwd 相对路径,因为回放只对 cwd 无关的参数路径做往返。
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-08-12-pi-ai-route-default-input-modalities.md
2026-08-12-pi-ai-route-default-input-modalities.md: efd20b2cd73979208bb777fa42536bc5b918e29e
2026-08-12-pi-ai-route-default-input-modalities.zh.md: 069a7916c8d4ffe738ff910a851e0a9cd1f66d0a
2026-08-12-pi-ai-route-default-input-modalities.md: eb03d5330a1283d439e16262325965cf2e7e8087
2026-08-12-pi-ai-route-default-input-modalities.zh.md: dfbbd2ae6db7a78e82955db66fe506d3506209fd
@@ -18,17 +18,17 @@ The assumption was justified in the source as the adapter's real capability rath
**The route value is a fallback, not an override — the catalog outranks it.** This is the `default*` ordering rather than `compat`'s, and the two are not interchangeable: `compat` shadows the catalog because a route-level protocol repoint invalidates the catalog's reasoning-dispatch facts wholesale, while a modality is a per-model property the catalog states accurately for the models it ships. Making the route value win would mean `defaultInput: [text]` silently strips images from every catalog vision model on the route — a footgun with no matching benefit, since narrowing one such model is what that model's own `input` is for.
**Undeclared means `[text]`, and that is the absence of a declaration rather than a guess at the endpoint.** Nothing can interrogate a gateway for its modalities no OpenAI-compatible listing endpoint reports them — so the only honest floor is the modality every supported protocol certainly carries. This is where the modality fallback parts company with the capacity ones: 262,144 tokens is merely plausible and wrong in both directions (a gateway serving 8k overflows, one serving 1M is wasted), while text is safe in one direction. The two wrong answers do not cost the same either. Under-claiming refuses the image before it is attached, naming the model, and the remedy is one documented line. Over-claiming admits an image the provider then rejects mid-turn, *after* prompt admission has committed the message durably, so the session keeps re-sending a request that cannot succeed and model selection refuses a switch to any text-only model. A cheap refusal at the earliest resolvable point beats an expensive one at the latest.
**Undeclared means `[text]`, and that is the absence of a declaration rather than a guess at the endpoint.** Nothing can interrogate a gateway for its modalities because no OpenAI-compatible listing endpoint reports them. The only safe floor is the modality every supported protocol certainly carries. Under-claiming refuses the image before it is attached, names the model, and has a documented configuration remedy. Over-claiming admits and persists an image before the provider can reject it. Later requests to that same incorrectly declared route will encounter the image again, although the user can select a text-only model because request assembly projects durable images to placeholders.
**An entry's empty list means the same as an absent one; the route's is refused.** `[]` describes a model that accepts nothing and could serve no request, so it states no answer and resolution continues past it. That reading is not cosmetic: the config schema materializes `[]` for an absent array, so treating it as "accepts nothing" would silently strip images from every catalog vision model a `models` list happens to name. The route value has nothing below it to answer instead, so its empty list is refused where it is written. The route's `models` list already resolves absent-and-empty the same way for the same reason.
**No configuration surface edits `input`.** It joins `compat`, `reasoningEfforts`, `thinkingBudgets`, and `headers` as a settings-document field, and the model-list editor stays a hand-written form over id, name, and the two capacities. This costs nothing durable because that card was already built to carry fields it does not edit: its row patch spreads the stored row before applying changes, and adoption keeps an existing row over a rediscovered candidate, so a hand-written `input` survives both.
The DeepSeek chat-completions adapter is untouched. Its `['text']` is a fact about its serializer, not a missing declaration, and it keeps refusing before the send.
The direct DeepSeek adapter owns a separate exact-model catalog. Its supported vision entry declares image input, while its text models and unlisted pass-through ids remain text-only.
## Alternatives considered
- **An optimistic `[text, image]` default** — makes the motivating case work with zero configuration, and the web form writes no modality at all, so a conservative default leaves the remedy in a file a web-only user has no reason to open. Rejected on the severity of being wrong: a refused attachment is a speed bump with a documented fix, while a provider rejection poisons the session, presents as an unexplained repeating failure, and is escapable only by switching models or starting over. Documenting the remedy on the model-configuration page closes the discoverability gap; nothing closes the poisoned session.
- **An optimistic `[text, image]` default** — makes the motivating case work with zero configuration, and the web form writes no modality at all, so a conservative default leaves the remedy in the settings document. Rejected because a false positive persists an image before the provider refuses it and causes repeated failure on that route. Text-only request projection provides recovery but does not make the declaration true.
- **A route value that overrides the catalog** (`compat`'s ordering: entry → route → catalog) — lets a deployment that repoints a catalog route at its own gateway declare "no vision here" once. Rejected because the same sentence then silently disables every catalog vision model on a route where someone wrote it by analogy with the capacity fields, and the legitimate case is served by that model's own `input`. An override would also have to be named `input` at the route, since calling it `default*` beside two genuine fallbacks would misdescribe it.
- **No route field at all, only the entry one** — closest to upstream, which has no route-level concept. Rejected on the bulk case the product's own flow produces: "fetch available models" adopts thirty ids with no modality, and an all-vision gateway would need `input` hand-written on each.
- **A route-level `defaultInput` with no entry field** — cannot mix modalities on one route or correct a single catalog model, leaving "split the provider across two route keys" as the only workaround, at the cost of a second permanent provider id and a duplicate entry in every model selector.
@@ -42,7 +42,7 @@ A vision model on a custom provider costs one line, `input: [text, image]`, writ
The image-admission gate keeps its meaning everywhere, because every modality it reads is now either recorded by the installed catalog or written by a person. Nothing claims a capability on a deployment's behalf.
A model that declares images its endpoint does not serve is not caught locally the claim is not verified — and the resulting failure is expensive. Prompt admission commits the user message durably (`agent/inbox/spliced`) before the request is built, so the rejected image stays in the session log: that model keeps re-sending it, and model selection refuses a switch to any text-only model. Recovery is to select a model that does serve images, fork before the image, or start a session. Making that failure non-destructive — rolling an unconsumed image message back out of the log when the send fails — is the change that would make an optimistic default reconsiderable, and is not attempted here.
A model that declares image input its endpoint does not serve is not caught locally because the claim is not verified. Prompt admission commits the user message durably before request construction, so the rejected image stays in the session log and later requests to that route can fail again. Recovery is to correct the declaration, select an image-capable route, or select a text-only route whose request projection replaces durable images with placeholders.
## Testing
@@ -18,17 +18,17 @@ Harness 把缺失的模态当作否定能力,并有三个准入点在构造任
**路由值是回退值而非覆盖值——catalog 的优先级更高。** 这采用的是 `default*` 的顺序而非 `compat` 的,两者不可互换:`compat` 之所以盖住 catalog,是因为路由级的协议改指会整体作废 catalog 关于推理分派的事实;而模态是按模型的属性,对 catalog 自己出货的那些模型,它记录得准确无误。让路由值获胜就意味着 `defaultInput: [text]` 会悄悄剥掉该路由上每一个 catalog 视觉模型的图片能力——一个没有对应收益的坑,因为收窄其中某个模型正是该模型自己的 `input` 要做的事。
**未声明即 `[text]`,而这是「尚未声明」,不是对端点的猜测。** 没有任何环节能询问网关的模态——没有任何 OpenAI 兼容列表端点会报告它们——因此唯一诚实的底线是每个受支持协议都确定携带的那个模态。这也正是模态回退值与容量回退值分道扬镳之处:262,144 只是个说得过去的数字,且两个方向都会错(网关只给 8k 会溢出,给 1M 则被浪费),而 text 在一个方向上是安全的。两种猜错的代价同样并不对等。少声明会在图片附加之前拒绝点名模型,补救办法是一行有文档可依的配置。多声明会接纳一张图片再由提供方在轮次中途拒绝——而此时 prompt 准入**早已**把消息持久化提交,于是会话会不断重发一个不可能成功的请求,且模型选择拒绝切换到任何纯文本模型。在最早可解析点付出一次廉价的拒绝,胜过在最晚点付出一次昂贵的
**未声明即 `[text]`,而这是「尚未声明」,不是对端点的猜测。** 没有任何环节能询问网关的模态,因为 OpenAI 兼容列表端点会报告它们。安全的底线是每个受支持协议都确定携带的模态。少声明会在图片附加之前拒绝点名模型,并给出有文档的配置补救方法。多声明会接纳并持久化图片再由提供方拒绝。之后对同一错误声明路由的请求还会再次遇到图片,但用户可以选择纯文本模型,因为请求组装会把持久图片投影为占位符
**条目的空列表与缺省同义;路由的空列表则被拒绝。** `[]` 描述的是一个什么都不接受、无法服务任何请求的模型,因此不作答,解析继续往下走。这个读法不是修辞:配置 schema 会为缺省数组物化出 `[]`,把它当作“什么都不接受”,会悄悄剥掉 `models` 列表恰好点到的每一个 catalog 视觉模型的图片能力。而路由值下面没有可以代为作答的层级,因此它的空列表在写入处即被拒绝。路由的 `models` 列表出于同样的理由,早已用同一种方式解析缺省与空。
**没有任何配置界面编辑 `input`。** 它和 `compat``reasoningEfforts``thinkingBudgets``headers` 一样是 settings 文档字段,而模型列表编辑器仍是一张只覆盖 id、名称和两个容量的手写表单。这不会带来持久代价,因为那张卡片本来就是按“承载自己并不编辑的字段”建造的:它的行 patch 会先展开已存储的行再应用改动,而采纳候选时已有行优先于重新发现的候选,因此手写的 `input` 在两条路径上都能存活。
DeepSeek chat-completions 适配器保持不动。它的 `['text']` 是关于其序列化器的事实,而不是一处缺失的声明,它继续在发送前拒绝
DeepSeek 直接适配器拥有独立的精确模型目录。支持视觉的条目声明图片输入,纯文本模型和未列出的透传 ID 保持纯文本
## 备选方案
- **乐观的 `[text, image]` 默认值** —— 让触发本次变更的场景零配置即可工作;而且网页表单不会写入任何模态,因此保守默认值会把补救法留在一个纯 Web 用户没有理由打开的文件里。否决的理由是猜错时的严重程度:被拒绝的附件是一个有文档可依的减速带,而提供方拒绝会毒化整个会话、表现为一次无从解释的反复失败,且只能靠换模型或重开会话脱身。把补救办法写进配置模型页即可补上可发现性的缺口;而毒化的会话没有任何东西能补
- **乐观的 `[text, image]` 默认值** —— 让触发场景无需配置即可工作,而网页表单不会写入模态,因此保守默认值会把补救法留在 settings 文档里。否决原因是错误的肯定声明会在提供方拒绝之前持久化图片,并让该路由重复失败。纯文本请求投影提供了恢复方法,但不能让错误声明变成事实
- **让路由值盖住 catalog**`compat` 的顺序:条目 → 路由 → catalog)—— 可以让把 catalog 路由改指到自家网关的部署,一句话声明「这里没有视觉能力」。被否决是因为同一句话也会在有人照着容量字段类比写下它的路由上,悄悄禁用每一个 catalog 视觉模型;而那个正当场景由该模型自己的 `input` 承担。覆盖值还必须在路由级改名叫 `input`,因为在两个货真价实的回退值旁边把它叫作 `default*` 是名不副实。
- **完全不要路由字段,只要条目字段** —— 最贴近上游(上游没有路由级概念)。被否决的理由是产品自身流程会产生的批量场景:「获取可用模型」一次采纳三十个不带模态的 id,全是视觉模型的网关就得逐个手写 `input`
- **只要路由级 `defaultInput`,不要条目字段** —— 无法在一条路由上混合模态,也无法修正单个 catalog 模型,唯一的变通办法只剩「把该提供方拆成两个路由键」,代价是多一个永久的 provider id 和每个模型选择器里的一项重复。
@@ -42,7 +42,7 @@ DeepSeek chat-completions 适配器保持不动。它的 `['text']` 是关于其
图片准入门禁在各处都保住了自己的意义,因为它读到的每一个模态,如今要么由已安装 catalog 记录,要么由人写下。没有任何环节会替部署宣称一项能力。
声明了端点并不提供的图片能力的模型不会在本地被拦下——该断言不经验证——而由此产生的失败代价高昂。prompt 准入在构造请求之前就把用户消息持久化提交(`agent/inbox/spliced`,因此被拒绝的图片留在会话日志里:该模型会不断重发它,而模型选择拒绝切换到任何纯文本模型。恢复途径是选择一个确实提供图片能力的模型、fork 到图片之前,或者开启新会话。让这次失败不具破坏性——发送失败时把尚未消费的图片消息从日志中回滚出去——才是能让乐观默认值重新可考虑的那项改动,本次未做尝试
声明了端点并不提供的图片能力时,本地无法发现该错误,因为声明不会被远端验证。prompt 准入会在请求构造前持久化用户消息,因此被拒绝的图片留在会话日志中,之后对该路由的请求可能再次失败。恢复方法是修正声明、选择支持图片的路由,或选择由请求投影把持久图片替换为占位符的纯文本路由
## 测试
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/bug-fix/2026-07-29-atomic-web-image-admission.md
2026-07-29-atomic-web-image-admission.md: c09d376f101a41994df3a10c22c06da4e59f06f6
2026-07-29-atomic-web-image-admission.zh.md: 8785f7489b0c433cba43a1747533b1d38aada3d3
2026-07-29-atomic-web-image-admission.md: dd2faf1e14c6147c80bcba571d5310899c2e8e22
2026-07-29-atomic-web-image-admission.zh.md: 8f15f8848dcb38fe6be178b86082a72b4ff0c8eb
@@ -6,24 +6,24 @@ English | [中文](2026-07-29-atomic-web-image-admission.zh.md)
## Problem
Image prompt admission and `session.selectModel` each read session modality state across asynchronous model and attachment lookups. Without one ordering boundary, an image prompt could validate an image-capable target while a concurrent selection installed a text-only target, or selection could miss a prompt after inbox dequeue but before its durable message event. Scanning the immutable event log avoided the second race but permanently blocked a text-only selection even after compaction removed the image from current model history.
Image prompt admission and `session.selectModel` each cross asynchronous model and attachment lookups. Without one ordering point, an image prompt could validate an image-capable target while a concurrent selection installed a text-only target. Selection could also change the route after admission had begun but before the durable message event was published.
## Decision
Each live Web agent has one private promise chain shared by image-bearing prompt admission and model selection. A failed operation settles its caller normally and leaves the chain usable. Text-only prompts bypass the chain because they cannot change the modality constraint.
Each live Web agent has one private promise chain shared by image-bearing prompt admission and model selection. A failed operation settles its caller normally and leaves the chain usable. Text-only prompts bypass the chain because they cannot create this ordering conflict.
The pending-publication set records a queued occurrence at dequeue and a steering occurrence already at enqueue (steering items never enter the queued UI mirror), and retains each until its matching `user/message` or `steering/message` event publishes. If admission ends without publishing, the transition to idle retires the entries; inbox discard retires the listed work, and session disposal retires every remaining entry. Model selection checks that set, the queued UI mirror, and `Session.deriveMessages()`, which is the current model-visible history after compaction.
The chain gives the two operations a deterministic order. When selection runs first, later image admission observes the selected model and refuses an unsupported image before persistence. When image admission runs first, its attachment and event publication complete before selection changes the route. The shared LLM runtime can then project durable image blocks to deterministic text placeholders for a text-only request without rewriting the session log. Steering uses the same admission chain even though it does not enter the queued UI mirror.
Provider adapters remain the final enforcement boundary. The host ordering only prevents its mutable route and pending image state from contradicting each other before request assembly.
## Alternatives considered
**Scan every immutable session event.** This catches published images but treats compacted-away content as permanently model-visible, preventing a valid later switch to a text-only route.
**Scan durable or derived history before selection.** This prevented a text-only route from being selected whenever history contained an image. Request-local projection now supports that route directly, so history is no longer a selection constraint.
**Retire the pending mirror at inbox dequeue.** Dequeue precedes the durable message append and leaves the exact interval in which model selection can miss both pending and published state.
**Track pending publication separately.** A queued occurrence could be retained from dequeue through its matching event. The promise chain already keeps selection behind the complete admission operation, so a second lifecycle mirror is unnecessary.
**Serialize every prompt and session mutation.** Text-only prompts and unrelated session operations cannot introduce an image requirement. A broader lock would add latency and ownership without closing another modality race.
## Consequences
An image prompt and a concurrent model selection have deterministic order, and a text-only target cannot strand an image that has been admitted but not yet published. Selection may wait for an in-flight image admission, while unrelated prompts retain their existing concurrency. Compaction can make a text-only target valid once no pending or derived image remains.
An image prompt and a concurrent model selection have deterministic order. Selection may wait for in-flight image admission, while unrelated text prompts retain their existing concurrency. Text-only model selection remains available after images enter durable history because request assembly projects those images to placeholders.
@@ -6,24 +6,24 @@ Status: implemented
## 问题
包含图片的提示词准入与 `session.selectModel` 都会跨越异步模型查询附件查询的过程中读取会话模态状态。如果没有统一的顺序边界,包含图片的提示词可能在支持图片的目标上通过校验,并发选择操作却设置了纯文本目标选择操作也可能在提示词已从 inbox 出队、但其持久消息事件尚未发布时漏掉该提示词。扫描不可变事件日志可以避免第二种竞态,但即使压缩(compaction)已经从当前模型历史中移除图片,仍会永久阻止选择纯文本目标
包含图片的提示词准入与 `session.selectModel` 都会跨越异步模型查询附件查询没有统一的排序点时,包含图片的提示词可能在支持图片的目标上通过校验,并发选择却设置了纯文本目标选择也可能在准入已经开始、持久消息事件尚未发布时改变路由
## 决策
每个活跃 Web agent(智能体)都有一条私有 promise 链,由包含图片的提示词准入与模型选择共享。操作失败会照常传递给调用方,且不会使该链失效。纯文本提示词绕过该链,因为它们不会改变模态约束
每个活跃 Web agent(智能体)都有一条私有 promise 链,由包含图片的提示词准入与模型选择共享。操作失败会照常传递给调用方,且不会使该链失效。纯文本提示词绕过该链,因为它们不会产生这类排序冲突
待发布集合会在排队条目出队时记录它,而 steering 条目在入队时即被记录(steering 条目从不进入排队 UI 镜像),并各自保留到匹配的 `user/message``steering/message` 事件发布。若准入结束时未发布事件,转为空闲状态会移除这些条目;inbox 丢弃会移除列出的工作项,会话 dispose(资源释放)则会移除所有剩余条目。模型选择会检查该集合、排队 UI 镜像以及 `Session.deriveMessages()`;后者表示压缩后模型当前可见的历史
该链为两个操作提供确定顺序。模型选择先执行时,后续图片准入会看到已选模型,并在持久化之前拒绝不支持的图片。图片准入先执行时,附件和事件会在模型选择改变路由之前完成发布。之后,共享 LLM 运行时可以在纯文本请求中把持久图片块投影为确定的文本占位符,无需改写会话日志。steering 不进入排队 UI 镜像,但仍使用同一条准入链
提供方适配器仍是最终的强制检查边界。宿主的顺序控制仅用于避免其可变路由与待发布图片状态在请求组装前彼此矛盾。
## 曾考虑的替代方案
**扫描每个不可变会话事件。** 这能捕获已发布的图片,但会把经压缩移除的内容视为永久对模型可见,从而阻止之后合法切换到纯文本路由
**选择前扫描持久历史或派生历史。** 这会在历史包含图片时阻止选择纯文本路由。请求期投影已经可以直接支持该路由,因此历史不再是选择约束
**在 inbox 出队时退役待处理镜像。** 出队早于持久消息追加,因此恰好会留下一个时间区间,让模型选择既看不到待处理状态,也看不到已发布状态
**单独跟踪待发布状态。** 排队条目可以从出队一直保留到匹配事件发布。promise 链已经让模型选择等待完整的准入操作,因此不需要第二套生命周期镜像
**序列化每个提示词和会话变更。** 纯文本提示词和无关的会话操作无法引入图片要求。更宽的锁会增加延迟与所有权复杂度,却不会再消除任何模态竞态。
## 后果
包含图片的提示词准入与并发模型选择之间具有确定的先后顺序,纯文本目标无法使已获准入但尚未发布的图片搁浅。模型选择可能等待正在进行的图片准入完成,无关提示词仍按现有方式并发处理。当没有图片等待发布,且派生历史经过压缩后也不再含图片时,纯文本目标可以变得有效
包含图片的提示词准入与并发模型选择之间具有确定顺序。模型选择可能等待正在进行的图片准入完成,无关的纯文本提示词仍按现有方式并发处理。图片进入持久历史后仍可选择纯文本模型,因为请求组装会把图片投影为占位符
@@ -1,6 +0,0 @@
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/bug-fix/2026-08-17-image-dimension-admission-limit.md
2026-08-17-image-dimension-admission-limit.md: 027259c0949d142ce8d8af27e7daa2abd54769ab
2026-08-17-image-dimension-admission-limit.zh.md: 38422615aa93b7f1639877d7d3751322c77de1eb
@@ -1,30 +0,0 @@
# Agent Note: Per-side image dimension admission limit
Status: implemented
English | [中文](2026-08-17-image-dimension-admission-limit.zh.md)
## Problem
`read_image` durably committed an image and appended its block to session history before any dimension check beyond byte count and total pixels. Deployed model routes reject a request with HTTP 400 when it carries many images and any of them has a side above 2000px. An admitted image rides every later request of its session, so one oversized read poisoned the durable history: the next model request failed, and so did every retry, permanently killing the session. The same gap applied to every other image producer (host uploads, MCP tool images) because admission had no per-side bound at all.
## Decision
`ImageAttachmentLimits` carries `maxImageDimension`, enforced during the admission full decode (`detectImage`) as `IMAGE_DIMENSION_TOO_LARGE`, so every producer that commits through the attachment service refuses an oversized image before anything reaches durable history. `LocalAttachmentStore` exposes it as the `maxImageDimension` config field with default `DEFAULT_MAX_IMAGE_DIMENSION = 2000`, the strictest per-side bound deployed routes enforce; deployments with laxer routes raise it from cordis.yml. `read_image` maps `IMAGE_DIMENSION_TOO_LARGE` and `IMAGE_TOO_MANY_PIXELS` to model-facing errors that name the resolved path and the limit and tell the model to downscale and retry — the turn continues as a recoverable tool error. The Web composer surfaces `IMAGE_DIMENSION_TOO_LARGE` with dedicated copy naming the limit. The `read-image-dimension` snapshot scenario replays the refusal keylessly through the assembled app: a 2001x1 workspace fixture, a recoverable tool error, and a completed turn.
## Alternatives considered
- **Downscale at admission instead of refusing.** Resampling changes the stored bytes away from what the caller supplied, adds a resampling-quality policy, and hides the limit from the model. Refusal keeps admission a pure gate; the model or user can downscale with full knowledge. Worth revisiting only if refusals prove frequent in practice.
- **Enforce at the provider adapter per route.** Too late: by the time a request is assembled the image is already durable history, so every route and every retry re-fails. Admission is the last point where a provider-rejected image can be kept out.
- **Repair already-poisoned sessions** (drop or replace the oversized block on later requests). Out of scope for this fix; admission prevents new poisonings, and history rewriting needs its own design against the model-visible ⟺ logged invariant.
## Related
- [Minimal read_image tool](../feature/2026-08-10-minimal-read-image-tool.md) — the tool whose admission gap this closes.
- [Web image intake and limits alignment](../feature/2026-08-12-web-image-intake-and-limits-alignment.md) — the composer-side surfacing of the same `ImageAttachmentLimits`.
## Consequences
- One oversized `read_image` can no longer break a session; the model sees an actionable error and the turn completes.
- Images with a side above 2000px are refused even in compositions whose routes would accept them on small requests; such deployments must raise `maxImageDimension` explicitly.
- Sessions that already carry an oversized image remain broken; this change does not repair existing history.
@@ -1,30 +0,0 @@
# Agent Note: 图片单边尺寸准入上限
Status: implemented
[English](2026-08-17-image-dimension-admission-limit.md) | 中文
## Problem
`read_image` 在字节数与总像素之外没有任何尺寸检查,就把图片持久提交并追加进会话历史。已部署的模型路由在请求携带多张图片且其中任何一张单边超过 2000px 时会以 HTTP 400 拒绝整个请求。已接纳的图片会随该会话之后的每次请求发送,因此一次超限读取就毒化了持久历史:下一次模型请求失败,之后的每次重试同样失败,会话被永久杀死。其他图片来源(宿主上传、MCP 工具图片)存在同样的缺口,因为准入完全没有单边上限。
## Decision
`ImageAttachmentLimits` 增加 `maxImageDimension`,在准入完整解码(`detectImage`)中以 `IMAGE_DIMENSION_TOO_LARGE` 强制执行,因此所有经附件服务提交的来源都会在任何内容进入持久历史之前拒绝超限图片。`LocalAttachmentStore` 将其暴露为 `maxImageDimension` 配置项,默认值 `DEFAULT_MAX_IMAGE_DIMENSION = 2000`,即已部署路由强制执行的最严格单边上限;路由更宽松的部署可在 cordis.yml 中调高。`read_image``IMAGE_DIMENSION_TOO_LARGE``IMAGE_TOO_MANY_PIXELS` 映射为面向模型的错误,指明解析后的路径与上限并提示缩图重试,本轮以可恢复的工具错误继续。Web 输入框对 `IMAGE_DIMENSION_TOO_LARGE` 给出指明上限的专用文案。`read-image-dimension` 快照场景通过组装后的应用无 key 回放这次拒绝:2001x1 的工作区 fixture、一条可恢复的工具错误、一个正常完成的轮次。
## Alternatives considered
- **准入时缩图而非拒绝。** 重采样会让存储字节偏离调用方提供的内容,引入重采样质量策略,还会对模型隐藏上限。拒绝让准入保持为纯粹的门禁;模型或用户可以在知情的前提下自行缩图。只有当拒绝在实践中频繁出现时才值得重新考虑。
- **在 provider 适配器按路由强制执行。** 为时已晚:组装请求时图片已是持久历史,每条路由、每次重试都会再次失败。准入是把必然被上游拒绝的图片挡在外面的最后一道关口。
- **修复已被毒化的会话**(在之后的请求中丢弃或替换超限图片块)。不在本次修复范围内;准入阻止新的毒化,而重写历史需要针对「模型可见 ⟺ 已记录」不变量单独设计。
## Related
- [最小 read_image 工具](../feature/2026-08-10-minimal-read-image-tool.zh.md),本次修复补上的正是该工具的准入缺口。
- [Web 图片摄入与限制对齐](../feature/2026-08-12-web-image-intake-and-limits-alignment.zh.md),同一组 `ImageAttachmentLimits` 在输入框侧的呈现。
## Consequences
- 一次超限的 `read_image` 不再能弄坏会话;模型看到可操作的错误,轮次正常完成。
- 单边超过 2000px 的图片即使在其路由本可接受(小请求)的组合中也会被拒绝;这类部署必须显式调高 `maxImageDimension`
- 已经携带超限图片的会话仍然是坏的;本次改动不修复既有历史。
@@ -1,36 +0,0 @@
# Agent Note: Request-level image payload bound
Status: implemented
English | [中文](2026-08-18-request-image-payload-bound.zh.md)
## Problem
Every image in session history is base64-inlined into every model request by the pi-ai adapter, so a long session's request body grows monotonically with each admitted image. Gateways cap request-body size; once the accumulated payload crossed such a cap the request was rejected with 413 (`Failed to buffer the request body: length limit exceeded`), and because nothing bounds or trims the assembled request, every retry resent the same oversized body. The session was permanently unusable, and the failure text matched no `classifyPiAiError` rule, so it surfaced as the generic `PI_AI_ERROR`. Admission bounds (per image, per message) cannot prevent this: each image is individually admissible, and the sum still grows without bound. Two screenshots were enough to trigger it in production.
## Decision
The pi-ai provider profile and direct DeepSeek adapter carry `maxRequestImageBytes` (default `DEFAULT_MAX_REQUEST_IMAGE_BYTES = 20MiB`, a positive integer, changeable from cordis.yml and settings). The provider-neutral `offloadRequestImages` conversion sums the base64 length of every image in history from `ImageAttachmentRef.bytes` without reading data and, while the sum exceeds the bound, replaces the oldest image occurrences with a fixed model-facing placeholder. The placeholder tells the model to read the file again when a path is available or ask the user to attach the image again. The most recent images are omitted last; an image larger than the bound is itself omitted. Occurrence-order replacement does not depend on object identity, so replaying the same JSON log produces the same request. Offloaded images are never read from the attachment store. Both adapters classify 413 as `INVALID_REQUEST`; pi-ai also recognizes specific request-body-cap wording. Four images admitted at the attachment store's 3.5MiB raw-image default occupy at most 18.67MiB after base64 expansion. The 20MiB default therefore retains four such images and leaves headroom under the direct API's 30MiB request limit, while deployments behind stricter gateways lower the value per route.
## Offload is conversion, not history
The placeholder is model-visible but not logged as a session event. It stays within the model-visible ⟺ logged invariant the same way the adapter's other serialization does (`(no output)` fallbacks, text-only folding): the offload locations are a pure function of the logged history and the route configuration, so the exact request remains reconstructable from the session log plus the composition. A logged elision event becomes necessary only when offload decisions gain non-deterministic inputs (for example live gateway feedback), which belongs to the deferred capability-metadata design.
## Alternatives considered
- **Fail the request with a clear error instead of offloading.** Keeps the model informed but leaves the session wedged: the user cannot remove images from durable history, so a hard failure at the bound is permanent. Offload keeps the session serviceable, which is the point of the fix.
- **Upload images once and reference them by URL / file id.** Removes the linear body growth entirely and is the right medium-term shape (providers and the internal gateway both document a Files path), but it introduces upload lifecycle management across providers and is far beyond a P0 hotfix.
- **Count the full request body, not only images.** Text and tools contribute little and their sizes are only known after full serialization per protocol; bounding the dominant term with explicit headroom is accurate enough for the failure being fixed and much simpler. Revisit inside the route-capability design.
- **Trim at admission instead.** Admission cannot see future accumulation; only the assembled request knows its total. Admission-side bounds (per-side dimension, bytes) remain as the first layer and are owned by [the dimension-limit note](2026-08-17-image-dimension-admission-limit.md).
## Related
- [Per-side image dimension admission limit](2026-08-17-image-dimension-admission-limit.md) — the admission-layer companion fix; together they close the two observed session-poisoning failures (400 dimension, 413 body size).
- [Direct DeepSeek vision input](../feature/2026-08-19-direct-deepseek-vision-input.md) — applies this provider-neutral conversion to the official multimodal route.
## Consequences
- An image-heavy long session keeps completing requests. The oldest images are omitted first; the most recent image is omitted only when it cannot fit within the bound.
- Crossing the bound rewrites an early message, so the provider prompt-cache prefix ends at the newly offloaded image until the offloaded prefix stabilizes.
- The bound counts base64 image payload only; deployments must keep it below their gateway's request-body cap with headroom, and the shipped default cannot know a private gateway's cap.
- Route capability metadata driving admission and assembly together (image count, per-image size, request size, provider token formulas) remains deferred design work tracked outside this fix.
@@ -1,36 +0,0 @@
# Agent Note: 请求级图片载荷上限
Status: implemented
[English](2026-08-18-request-image-payload-bound.md) | 中文
## Problem
pi-ai 适配器把会话历史中的每张图片 base64 内联进每一个模型请求,长会话的请求体随每张入库图片单调增长。网关对请求体大小设有上限;累积载荷一旦越线,请求被以 413 拒绝(`Failed to buffer the request body: length limit exceeded`),而组装层没有任何约束或裁剪,每次重试都会原样重发同一个超限请求体,会话永久不可用。该报错文本不匹配 `classifyPiAiError` 的任何规则,只能落进笼统的 `PI_AI_ERROR`。准入上限(单图、单消息)无法阻止这一点:每张图片单独看都合规,总和仍然无界增长。线上两张截图即可触发。
## Decision
pi-ai provider profile 与直接 DeepSeek 适配器都提供 `maxRequestImageBytes`(默认 `DEFAULT_MAX_REQUEST_IMAGE_BYTES = 20MiB`,正整数,可从 cordis.yml 与 settings 修改)。提供方无关的 `offloadRequestImages` 转换由 `ImageAttachmentRef.bytes` 推算每张历史图片的 base64 长度(无需读取数据)求和,总和超过上限时从最老的图片出现位置开始替换为一段固定的模型可见占位文本。占位文本要求模型在有路径时重新读取文件,否则请用户重新附上图片。越新的图片越晚被省略;单张图片本身超过上限时也会被省略。按出现顺序替换不依赖对象身份,因此重放同一份 JSON 日志会产生相同请求。被 offload 的图片不会从附件存储读取。两个适配器都把 413 归类为 `INVALID_REQUEST`;pi-ai 还会识别明确的请求体上限措辞。四张按附件存储默认上限准入的 3.5MiB 原始图片,经 base64 膨胀后最多占 18.67MiB。20MiB 默认上限因此可保留四张这样的图片,并在直接 API 的 30MiB 请求上限下留出余量;网关更严格的部署则按路由调低该值。
## offload 是转换而非历史
占位文本模型可见,但不记录为会话事件。它与适配器的其他序列化(`(no output)` 回退、纯文本折叠)以同样的方式满足「模型可见 ⟺ 已记录」不变量:offload 位置是已记录历史与路由配置的纯函数,确切请求仍可由会话日志加组合配置重建。只有当 offload 决策引入非确定性输入(例如网关的实时反馈)时才需要记录省略事件,那属于暂缓的能力元数据设计。
## Alternatives considered
- **在上限处直接报错而不 offload。** 模型知情,但会话仍然卡死:用户无法从持久历史中删除图片,越线即永久失败。offload 让会话保持可用,这正是本修复的目标。
- **图片上传一次、按 URL / file id 引用。** 从结构上消除请求体线性增长,是正确的中期形态(各提供方与内部网关都有 Files 路径),但要跨提供方管理上传生命周期,远超 P0 热修复范围。
- **统计完整请求体而非只统计图片。** 文本与工具占比很小,且其大小要到按协议完整序列化后才可知;对主导项设上限并留出显式余量,对所修故障足够精确且简单得多。留到路由能力设计中再议。
- **改在准入侧裁剪。** 准入看不到未来的累积,只有组装后的请求知道自己的总量。准入侧上限(单边尺寸、字节)作为第一层保留,归[尺寸上限笔记](2026-08-17-image-dimension-admission-limit.zh.md)所有。
## Related
- [图片单边尺寸准入上限](2026-08-17-image-dimension-admission-limit.zh.md),准入层的配套修复;两者合起来封住已观测到的两类会话毒化故障(400 尺寸、413 请求体)。
- [直接 DeepSeek 视觉输入](../feature/2026-08-19-direct-deepseek-vision-input.zh.md)把这项提供方无关转换应用于官方多模态路由。
## Consequences
- 图片较多的长会话持续可用。最老的图片优先省略;仅当最新图片本身无法装进上限时才会省略它。
- 越过上限会改写较早的一条消息,提供方 prompt cache 前缀在新被 offload 的图片处截止,直到被 offload 的前缀稳定。
- 上限只统计 base64 图片载荷;部署必须让它低于自家网关的请求体上限并留出余量,发行默认值无法预知私有网关的上限。
- 由路由能力元数据同时驱动准入与组装(图片数量、单图大小、请求大小、提供方 token 公式)的设计仍为暂缓工作,在本修复之外跟踪。
@@ -1,6 +1,6 @@
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-08-19-direct-deepseek-vision-input.md
2026-08-19-direct-deepseek-vision-input.md: 76d3244e67a73c1cdf4419a6537ada38e0a75bd5
2026-08-19-direct-deepseek-vision-input.zh.md: a77231104156371fe698f8a8ad386cfa03251990
# pnpm run verify-translation-pairing --write .agents/notes/implemented/bug-fix/2026-08-21-deepseek-files-inline-fallback.md
2026-08-21-deepseek-files-inline-fallback.md: 7442089038e2cf47f37661c0f098054d03f67aef
2026-08-21-deepseek-files-inline-fallback.zh.md: 0534514f9bc52f7871f43c288466643cebc7d69c
@@ -0,0 +1,37 @@
# Agent Note: Recover DeepSeek image requests from Files resolution failures
Status: implemented
English | [中文](2026-08-21-deepseek-files-inline-fallback.zh.md)
## Problem
The direct DeepSeek vision route uses provider file ids so repeated requests do not resend image bytes. An unavailable, unsupported, or stalled Files endpoint can prevent chat before the model request begins even though the same endpoint still accepts inline image data. A fallback that retains the 128MiB Files budget would exceed the inline request-body limit, while a fallback that independently transforms images could send different pixels from the failed file-id attempt.
## Decision
Files remains the preferred transport. Each request-image file resolution has the configurable `filesApiTimeoutMs` deadline, one minute by default. The stream idle deadline defaults to five minutes, so the Files deadline normally leaves time for inline fallback. A deployment may configure the stream idle deadline to expire first. Successful resolutions refresh the outer idle watchdog. Caller cancellation and the outer stream deadline remain terminal outcomes.
A file resolution failure discards the transient file parts assembled for that chat attempt and rebuilds the complete image request with base64 data URLs. Every retained image uses the already prepared deterministic `RequestImageAttachment`; the fallback performs no additional decode, resize, or encode, and a chat request never mixes file ids with inline images. Upload mappings committed before a later image fails remain available to later requests. The next request tries Files again, so recovery requires no process-wide outage state.
Inline fallback has a separate base64-expanded high watermark, `maxInlineRequestImageBytes`, of 20MiB by default. `inlineImageOffloadByteQuantum` defaults to 10MiB, so crossing the high watermark advances the deterministic oldest-image prefix to the next 10MiB removal boundary. The existing 600-image bound and count quantum still apply. File mode retains its 128MiB high watermark and 64MiB removal quantum.
Provider chat errors keep their existing classifications. A stale file id is invalidated, re-uploaded, and retried once. If that replacement resolution fails, the permitted retry uses the inline representation. A generic chat failure does not switch transports because it does not establish that Files resolution failed.
## Alternatives considered
**Send inline images first.** Rejected because successful Files uploads allow deterministic request bytes to be reused across turns without repeating base64 in every request.
**Mix resolved file ids with inline images after one upload fails.** Rejected because the request would still depend on the failing Files service and would have two independent image budgets.
**Apply the 128MiB Files bound to inline fallback.** Rejected because base64 expands the payload and can exceed the chat request-body limit. The 20MiB budget leaves space for JSON, text history, and tools.
**Remember an outage and bypass Files on later requests.** Rejected because a process-local circuit state introduces recovery timing and shared failure state. Retrying Files on the next request detects service recovery without another timer.
## Verification
Serializer tests cover file and data-URL representations over the same request versions, all supported media types, tool-result placement, and 20-to-10 base64 offload. Adapter tests cover immediate resolution failure, failure after a partial set of file ids, deadline-triggered fallback, stale-id replacement failure, all-inline request bodies, caller cancellation without fallback, and generic chat failure without a transport switch. Configuration tests cover both inline bounds and independent Files and stream idle deadlines.
## Consequences
A Files outage no longer prevents an image chat that fits the inline budget. Fallback repeats image bytes and may omit more history than file mode because its limit is lower. A request can leave successful uploads behind when a later image fails, but their indexed mappings are reusable and do not change the chat body sent by the fallback. Explicit file-management operations continue to expose their own failures.
@@ -0,0 +1,37 @@
# Agent Note: DeepSeek Files 解析失败时恢复图片请求
Status: implemented
[English](2026-08-21-deepseek-files-inline-fallback.md) | 中文
## Problem
DeepSeek 官方视觉路由使用提供方文件 ID,使重复请求不必再次发送图片字节。如果 Files 端点不可用、不受支持或一直不返回,chat 会在模型请求开始前失败,即使同一端点仍接受内联图片数据。沿用 128MiB Files 预算的回退会超过内联请求体上限,独立转换图片的回退则可能发送与失败 file ID 尝试不同的像素。
## Decision
Files 仍是首选传输方式。每张请求图片的文件解析都有可配置的 `filesApiTimeoutMs` 时限,默认一分钟。stream idle 时限默认为五分钟,因此 Files 时限通常会为内联回退留出时间。部署也可以把 stream idle 时限设得更短,让它先终止请求。每次成功解析都会刷新外层 idle watchdog。调用方取消和外层流时限仍直接终止请求。
文件解析失败后,适配器会丢弃为该次 chat 尝试组装的临时文件块,并用 base64 data URL 重新组装完整图片请求。每张保留图片都复用已经准备好的确定性 `RequestImageAttachment`;回退不会再次解码、缩放或编码,同一个 chat 请求也不会混用 file ID 和内联图片。较早图片在后续图片失败前已经提交的上传映射会保留,供之后请求使用。下一次请求会重新尝试 Files,因此不需要保存进程级故障状态。
内联回退使用独立的 base64 膨胀后高水位,`maxInlineRequestImageBytes` 默认为 20MiB。`inlineImageOffloadByteQuantum` 默认为 10MiB,因此越过高水位时,确定性的最旧图片前缀会推进到下一个 10MiB 移除边界。现有 600 张图片上限和数量步长继续生效。文件模式继续使用 128MiB 高水位和 64MiB 移除步长。
提供方 chat 错误继续使用现有分类。失效 file ID 会被清除、重新上传并重试一次。如果替换解析失败,这次允许的重试会使用内联表示。普通 chat 错误不能证明 Files 解析失败,因此不会切换传输方式。
## Alternatives considered
**优先发送内联图片。** 不采用,因为 Files 上传成功后可以跨轮次复用确定性的请求字节,不必在每次请求中重复 base64。
**某次上传失败后混用已解析 file ID 和内联图片。** 不采用,因为请求仍依赖发生故障的 Files 服务,而且需要同时处理两套图片预算。
**把 128MiB Files 上限用于内联回退。** 不采用,因为 base64 会扩大负载,并可能超过 chat 请求体上限。20MiB 预算会为 JSON、文本历史和工具留下空间。
**记住故障,并在后续请求中跳过 Files。** 不采用,因为进程级状态会引入恢复时间和共享故障状态。下一次请求重新尝试 Files,可以在无需新增计时器的情况下发现服务恢复。
## Verification
序列化测试覆盖相同请求版本的文件和 data URL 表示、全部支持的媒体类型、工具结果位置,以及 20MiB 到 10MiB 的 base64 offload。适配器测试覆盖立即解析失败、部分 file ID 成功后的失败、时限触发的回退、失效 ID 替换失败、全内联请求体、调用方取消时不回退,以及普通 chat 错误不切换传输方式。配置测试覆盖两项内联预算,以及相互独立的 Files 和 stream idle 时限。
## Consequences
符合内联预算的图片 chat 不会再因 Files 故障而失败。回退会重复发送图片字节,而且由于上限更低,可能比文件模式省略更多历史。后续图片失败时,请求可能留下较早图片的成功上传,但这些索引映射可以复用,也不会改变回退发送的 chat 请求体。显式文件管理操作继续暴露自身错误。
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-22-web-multimodal-image-input-and-durable-attachments.md
2026-07-22-web-multimodal-image-input-and-durable-attachments.md: 3f77ab8d55f8eca821cd12a4591c6239c2ea10f5
2026-07-22-web-multimodal-image-input-and-durable-attachments.zh.md: c95c5abe664635f3cde3a1fc2d569c9474c69665
2026-07-22-web-multimodal-image-input-and-durable-attachments.md: 30ac1dcff9e6400a3bcf58f7b8e5237e20bd5c04
2026-07-22-web-multimodal-image-input-and-durable-attachments.zh.md: 359bb9048632222518d87aadd348bca217c8f7c4
@@ -69,7 +69,7 @@ interface ComposerAttachment {
This split uses the session provide channel's input hook and actions as the single subscription path for live composer state while keeping non-serializable browser objects out of persisted JSON. Only the plain-text draft mirror uses `localStorage`; attachment identifiers, browser `File` objects, and object URLs remain scoped to the live session input shell. Unsent images therefore do not survive reload or session-scope disposal. A Workspace switch moves a mixed text-and-image draft only when the destination shell accepts the complete image batch; refusal leaves both parts with the source. A native client may stage input in an OS temporary directory, but it must treat that path exactly like the browser object URL: delete it when no longer needed and copy the bytes into the durable store before message acceptance.
The local attachment backend resolves an explicit `dshHome`, then `$DSH_HOME`, then `~/.dsh`. It stores content-addressed objects below `$DSH_HOME/attachments/v1/objects/<prefix>/<sha256>` with owner-only directory and file permissions. On each process's first save for one home, it creates that home and synchronizes every ancestor entry to the filesystem root; existence is not treated as durability because another process may still be between `mkdir` and parent `fsync`. A temporary file is then written, synchronized, atomically published, and made durable with directory syncs on the publication path (POSIX; Windows relies on filesystem metadata journaling) before the service returns a reference. The content digest is encoded in the opaque `sha256:<digest>` identifier. Admission and reads fully decode supported rasters before accepting their format and dimensions, and every read also verifies the digest, byte length, and logged metadata.
The local attachment backend resolves an explicit `dshHome`, then `$DSH_HOME`, then `~/.dsh`. It stores content-addressed objects below `$DSH_HOME/attachments/v1/objects/<prefix>/<sha256>` with owner-only directory and file permissions. On each process's first save for one home, it creates that home and synchronizes every ancestor entry to the filesystem root; existence is not treated as durability because another process may still be between `mkdir` and parent `fsync`. A temporary file is then written, synchronized, atomically published, and made durable with directory syncs on the publication path (POSIX; Windows relies on filesystem metadata journaling) before the service returns a reference. The content digest is encoded in the opaque `sha256:<digest>` identifier. Admission prepares a provider-independent master by applying orientation, removing metadata, converting to 8-bit sRGB/sRGBA, and preserving aspect ratio under independent dimension and byte limits. Reads verify the digest, byte length, and logged metadata. Route-specific deterministic request versions are cached separately; the full policy is recorded in [Unified image masters, request versions, and provider files](2026-08-20-unified-image-request-pipeline.md).
The store performs no automatic deletion in version one. Sent user images and model-generated images remain reachable for history, resume, and fork. Reference-aware garbage collection needs a separate design because an age-only rule can delete data still referenced by a durable session. Deployment byte and pixel limits are admission policy on writes; reads verify the digest and recorded metadata without reapplying current admission limits, so lowering policy does not invalidate older history.
@@ -114,7 +114,7 @@ type PromptInputPart =
}
```
Base64 crosses a wire boundary once and is discarded after persistence. Each front door validates canonical base64 and declared MIME shape, then calls `AttachmentStore.saveImages()` with the whole decoded batch. The service owns image count, aggregate bytes, individual bytes, fully decoded raster/MIME agreement, intrinsic dimensions, and decoded-pixel count; it validates every batch member before saving any member, so one malformed image cannot strand the batch's valid members as unreferenced objects. Storage commits then run in submission order to bound full-raster decoder memory. If a later storage I/O operation fails, the caller appends no model-visible event and receives no partial references, but an earlier immutable content-addressed object may remain unreferenced; version one leaves cleanup to future reference-aware garbage collection instead of adding destructive rollback to the deduplicated store. Only after every image succeeds does the front door call the agent with normalized text and durable image blocks in wire order. A failure exposes no attachment path or raw bytes.
Base64 crosses a wire boundary once and is discarded after persistence. Each front door validates canonical base64 and declared MIME fields, then calls `AttachmentStore.saveImages()` with the whole decoded batch. The service owns image count, aggregate bytes, individual bytes, fully decoded raster/MIME agreement, intrinsic dimensions, decoded-pixel count, and master preparation. It prepares and verifies every batch member once before publishing any member, so one malformed image cannot create partial references and large images are not decoded and encoded again at commit. Storage commits then run in submission order. If a later storage I/O operation fails, the caller appends no model-visible event and receives no partial references, but an earlier immutable content-addressed object may remain unreferenced under the existing storage rule. Only after every image succeeds does the front door call the agent with normalized text and durable image blocks in wire order. A failure exposes no attachment path or raw bytes.
`session.attachment` is a read-only, session-scoped endpoint. The host serves bytes only when a durable event in that session references the requested attachment identifier. The client deduplicates loads by session and attachment identifier while that session is rendered, revokes resolved URLs on rendered-session disposal, and rejects invalidated late loads before allocating an object URL so an unmounted session or disposed service cannot repopulate the cache.
@@ -122,15 +122,15 @@ Base64 crosses a wire boundary once and is discarded after persistence. Each fro
Model catalog entries gain optional merge-extensible input modality declarations. A missing declaration means unknown; a present list without `image` is an explicit negative capability.
The host is the authoritative preflight boundary. It resolves the session's latest routed provider/model, falling back through agent options to host defaults; if that model explicitly excludes image input, it rejects the prompt before writing any attachment or event, and the client restores the draft. Image-bearing prompt admission and model selection share one per-agent serial boundary, and a dequeued prompt remains pending until its durable message event publishes ([ordering decision](../bug-fix/2026-07-29-atomic-web-image-admission.md)); a steering carrier gates from its enqueue until its `steering/message` event publishes, closing the outbox hop that never enters the queued mirror. Selection rejects a text-only target while an image is pending publication or remains in the session's current derived history. Compaction can remove old images and make a later text-only selection valid; idle without publication releases a claimed queued carrier, while steering retained in the outbox stays gated until publication or discard. `session.updateQueue` edits accept text content only, so a queue edit cannot inject an image past this admission boundary. Unknown capability proceeds to the adapter guard so uncatalogued model identifiers remain usable. The browser rejects unsupported declared image media types before allocating preview URLs, but it does not snapshot deployment limits or model capability: a handshake snapshot cannot represent a session's current target after `session.selectModel`, and deployment policy may change independently. The host validates the complete batch against current byte, count, aggregate, media, dimension, pixel, and routed-model policy before writing any attachment or event; its rejection announces through the composer's transient toast.
The host is the authoritative preflight point. It resolves the session's latest routed provider and model, falling back through agent options to host defaults; if that model explicitly excludes image input, it rejects a new image prompt before writing an attachment or event, and the client restores the draft. Image-bearing prompt admission and model selection share one per-agent serial chain ([ordering decision](../bug-fix/2026-07-29-atomic-web-image-admission.md)), including steering that does not enter the queued UI mirror. This gives a prompt and concurrent selection a deterministic order. Selection itself may target a text-only model after images enter durable history; the shared LLM runtime replaces retained image blocks with deterministic text placeholders for that request. `session.updateQueue` edits accept text content only, so a queue edit cannot inject an image past admission. Unknown capability proceeds to the adapter guard so uncatalogued model identifiers remain usable. The browser rejects unsupported declared image media types before allocating preview URLs, but it does not snapshot deployment limits or model capability. The host validates the complete batch against current byte, count, aggregate, media, dimension, pixel, and routed-model policy before writing an attachment or event; its rejection appears through the composer's transient toast.
Pi-AI and the direct DeepSeek adapter resolve `ctx.attachments` at request time, recursively convert each durable image reference including references nested inside tool results, and emit native image content only for models that declare image input. The direct route advertises `deepseek-v4-flash-vision-exp` as image-capable and accepts configured image-capable catalog entries; its Flash, Pro, custom models without an image declaration, and unlisted pass-through ids remain text-only. Request-time service resolution keeps Cordis load order from freezing optional attachment availability. No adapter may flatten or skip a retained image; unsupported roles and models fail with typed `UNSUPPORTED_CONTENT`.
Pi-AI and the direct DeepSeek adapter resolve `ctx.attachments` at request time, recursively convert each retained image reference including references nested inside tool results, and emit native image content only for models that declare image input. Both adapters request the same deterministic route-specific version from the durable normalized attachment. Pi-AI carries it inline under a base64-aware request budget. The built-in DeepSeek route advertises `deepseek-v4-flash-vision-exp`, uploads every retained version through Files API, and sends `file_id` blocks with indexed reuse, expiry, bounded stale-id retry, quota cleanup, and explicit deletion. DeepSeek text models, custom models without an image declaration, and unlisted pass-through ids remain text-only. Request-time service resolution keeps Cordis load order from freezing optional attachment availability. No adapter may flatten or silently skip a retained image; unsupported roles and models fail with typed `UNSUPPORTED_CONTENT`.
Core supports structured assistant image blocks, but no current production provider route is certified for image output. Any future output-capable adapter must retrieve provider bytes under bounded size and time policy, validate them through the same attachment service, persist them, and only then publish the atomic `ImageBlock`. A URL in assistant Markdown remains text and is never downloaded automatically.
Provider-neutral token estimation does not guess visual pricing from image dimensions; provider-reported usage remains authoritative. ACP advertises image prompts only when its configured exact route and attachment deployment can accept them, persists inline input before publishing the user event, and re-reads committed assistant image references for native ACP image updates. MCP keeps canonical raw blocks for programmatic callers while projecting admitted images to durable core blocks; Code Mode carries any settled image-bearing sub-result through the outer result as logged source-attributed context.
Compaction replays the selected conversation prefix, including image references, into the configured summarization route. A visual-capable route resolves those references through its adapter; a text-only route fails explicitly instead of silently dropping the visual context. The synthesized checkpoint remains text-only, and `compaction-basic` rejects image summary output with `UNSUPPORTED_CONTENT`.
Compaction replays the selected conversation prefix, including image references, into the configured summarization route. A visual-capable route uses the same deterministic request versions as ordinary turns. A text-only route receives the same deterministic attachment placeholders as any other LLM request. The synthesized checkpoint remains text-only, and `compaction-basic` rejects image summary output with `UNSUPPORTED_CONTENT`.
### History rendering and original preview
@@ -140,7 +140,7 @@ Composer thumbnails and each `MessageImage` own ephemeral original-preview state
### Limits and trust boundaries
Version one accepts PNG, JPEG, WebP, and GIF only. SVG and remote URLs are excluded. Default limits are 3.5 MiB per image, 20 images and 100 MiB aggregate image bytes per message, 40 million intrinsic pixels per image, and 2000 pixels on either side. These deployment-varying limits are validated backend configuration and enforced by the host before persistence. The client connection carrier has an independent configurable `maxRequestBodyBytes` cap (160 MiB by default) for every API request and fails load if it cannot hold the attachment service's aggregate image limit after base64 and envelope expansion; lowering image policy therefore never silently lowers the carrier limit for valid text or other RPCs. A body without a declared length is rejected the moment it crosses the cap rather than drained to its end.
Version one accepts PNG, JPEG, WebP, and GIF only. SVG and remote URLs are excluded. Source intake defaults are 32 MiB per image, 20 images and 100 MiB aggregate image bytes per message, 100 million decoded pixels per image, and 16384px on either side. The provider-independent master defaults to a 2048px long edge and 4 MiB safety cap. Provider request pixel and encoded-byte limits are separate route policies. These deployment-varying limits are validated backend configuration and enforced before persistence or request transmission. The client connection carrier has an independent configurable `maxRequestBodyBytes` cap, 160 MiB by default, and fails load if it cannot hold the aggregate source limit after base64 and envelope expansion. A body without a declared length is rejected when it crosses the cap rather than drained to its end.
Malformed base64, unsupported or mismatched media, truncated image payloads, excess bytes, excess image count, excess pixels, excess per-side dimensions, missing objects, and integrity mismatches return stable structured failures. Original filenames are reduced to a display basename, control characters are removed, and no local path is logged or returned to the browser.
@@ -148,11 +148,11 @@ Malformed base64, unsupported or mismatched media, truncated image payloads, exc
| Surface | Responsibility |
| --- | --- |
| `packages/attachment/attachment` | Opaque attachment identifier, image reference, limits, failures, and single/batch admission through `ctx.attachments`. |
| `packages/attachment/attachment-local` | Private content-addressed storage, complete raster decoding, integrity verification, and configuration. |
| `packages/llm/llm` | Role-neutral `ImageBlock` and input-modality metadata. |
| `packages/llm/llm-pi-ai` | Resolve durable supported image input into native provider content. |
| `packages/llm/llm-deepseek` | Resolve declared official vision input and reject images for text-only models. |
| `packages/attachment/attachment` | Opaque attachment and request-version identifiers, image references, policies, failures, batch admission, derived reads, and crops through `ctx.attachments`. |
| `packages/attachment/attachment-local` | Private content-addressed masters, deterministic request cache, complete raster decoding, integrity verification, and configuration. |
| `packages/llm/llm` | Role-neutral `ImageBlock`, input-modality metadata, exact adapter generations, and text-only request projection. |
| `packages/llm/llm-pi-ai` | Resolve durable images to deterministic inline request versions. |
| `packages/llm/llm-deepseek` | Resolve official vision input to deterministic request versions and Files API ids. |
| `packages/compaction/compaction-basic` | Preserve images in summary input and reject non-text checkpoint output explicitly. |
| `packages/host/apiproxy` and `packages/bundle/base` | Narrow upload wire, shared batch admission, limits and routed-model preflight, persist-before-event ordering, session-authorized reads, and default profile composition. |
| `packages/client/connection` and `packages/client/runtime` | Bounded request buffering, wire types, fixture images, prompt uploads, attachment reads, and durable-reference folding. |
@@ -165,7 +165,7 @@ The attachment packages form the interface/implementation side of one capability
### Implementation
The implemented slice includes the attachment seam and shared batch admission, role-neutral image block, Pi-AI and direct DeepSeek input conversion, durable Web/ACP/MCP ordering, Web upload/read protocol, conditional ACP image wire support, lossless MCP canonical results with durable image projection, generic Code Mode rich-result forwarding, current image-limit enforcement, bounded Web request bodies, in-memory draft images, paste/drop rail, user and assistant history rendering, single-click preview, compaction handling, and keyless assembled Web and ACP coverage.
The implemented capability includes shared prepare-once batch admission, provider-independent masters, deterministic request versions, DeepSeek Files reuse, stable crop handles, role-neutral image blocks, Pi-AI and DeepSeek input conversion, durable Web/ACP/MCP ordering, Web upload/read protocol, conditional ACP image support, lossless MCP results with durable image projection, Code Mode rich-result forwarding, bounded Web requests, draft and historical image UI, compaction handling, and keyless assembled coverage.
No compatibility shim is required for the pre-release prompt wire; all call sites and fixtures change with the introducing slice.
@@ -210,11 +210,11 @@ Rejected because tool renderers are pure, synchronous, and replayable. MCP prepa
## Testing
- Storage tests cover content-addressed deduplication, private permissions, admission failures, corruption/missing-object failures, and reading history after deployment limits are lowered.
- Host and protocol tests cover persist-before-event ordering, absence of base64 in logs, session-scoped authorization, capability rejection, upload limits, bounded HTTP request bodies, image-admission/model-selection races (queued and steering placements), pending publication, idle release without publication, text-only queue edits, and selection against current derived history after compaction.
- Host and protocol tests cover persist-before-event ordering, absence of base64 in logs, session-scoped authorization, capability rejection, upload limits, bounded HTTP request bodies, image-admission/model-selection ordering, text-only queue edits, and text-only request projection.
- Client unit tests cover paste and drop, mixed clipboard text, image-only send, draft restoration, ordering, draft/session-scope/application object-URL cleanup, and a deferred historical read that completes after disposal; the keyless assembled built-client lane (`apps/web/tests/image-display.snapshot.ts`, `DSH_EXAMPLE_MODE=lib pnpm run test:snapshot`) covers the historical user and assistant galleries over the authorized attachment route, the original-size lightbox, and the composer paste rail.
- Adapter and compaction tests cover native Pi-AI image conversion, late attachment-service composition, text-only rejection, recursively nested tool-result images, preserved summary input, and explicit image-output rejection.
- Adapter and compaction tests cover deterministic Pi-AI request versions, DeepSeek Files upload and reuse, stale-id recovery, text-only projection, recursively nested tool-result images, shared summary request versions, and explicit image-output rejection.
- Attachment, MCP, ACP, and Code Mode tests cover all-member validation before writes, mixed text/image ordering, no inline base64 in durable events, exact route-capability gates, explicit unsupported-content diagnostics, post-execute replacement/block precedence, cancellation during admission, verified assistant-image delivery, and generic nested-image forwarding. A keyless assembled ACP snapshot sends a real inline PNG and pins only its durable reference in the session log.
- A credentialed real-API test sends a PNG through the Anthropic `claude-opus-4-8` route and requires the model to identify its QR code.
- Credentialed real-API tests cover the configured Anthropic route and the built-in `deepseek-official` Files path. The DeepSeek test does not use a custom provider entry.
- The current production adapter set has no certified image-output route; output-provider certification remains outside version one.
## Consequences
@@ -69,7 +69,7 @@ interface ComposerAttachment {
这一拆分把会话 provide 通道的输入 hook 与 actions 用作实时输入区状态的唯一订阅路径,同时避免把不可序列化的浏览器对象写进持久 JSON。只有纯文本草稿镜像使用 `localStorage`;附件标识符、浏览器 `File` 对象和对象 URL 都限定在实时会话输入外壳的 scope 内。未发送图片因此无法跨重载或会话 scope 释放保留。切换 Workspace 时,只有目标外壳接受完整图片批次,图文混合草稿才会移动;拒绝时,文本和图片都留在来源外壳。原生客户端可以在操作系统临时目录中暂存输入,但必须像对待浏览器对象 URL 一样对待该路径:不再需要时删除,并在消息被接受前把字节复制进持久存储。
本地附件后端依次解析显式 `dshHome``$DSH_HOME``~/.dsh`。它把内容寻址对象存储在 `$DSH_HOME/attachments/v1/objects/<prefix>/<sha256>` 下,并为目录和文件设置仅所有者可访问的权限。每个进程首次为某个 home 保存对象时,都会创建该 home,并逐级同步每个祖先目录项直至文件系统根目录;不能把存在视为持久性,因为另一个进程可能仍处于 `mkdir` 与父目录 `fsync` 之间。随后,服务写入并同步临时文件,再以原子方式发布,并对发布路径执行目录同步使其持久(POSIX;Windows 依赖文件系统元数据日志),之后才返回引用。内容摘要编码在不透明的 `sha256:<digest>` 标识符中。写入准入与读取都会完整解码受支持的光栅图片,之后才接受其格式和尺寸;每次读取会校验摘要、字节长度和已记录元数据。
本地附件后端依次解析显式 `dshHome``$DSH_HOME``~/.dsh`。它把内容寻址对象存储在 `$DSH_HOME/attachments/v1/objects/<prefix>/<sha256>` 下,并为目录和文件设置仅所有者可访问的权限。每个进程首次为某个 home 保存对象时,都会创建该 home,并逐级同步每个祖先目录项直至文件系统根目录;不能把存在视为持久性,因为另一个进程可能仍处于 `mkdir` 与父目录 `fsync` 之间。随后,服务写入并同步临时文件,再以原子方式发布,并对发布路径执行目录同步使其持久(POSIX;Windows 依赖文件系统元数据日志),之后才返回引用。内容摘要编码在不透明的 `sha256:<digest>` 标识符中。准入会应用方向、删除元数据、转换为 8-bit sRGB/sRGBA,并在独立尺寸和字节上限内保持宽高比,生成与提供方无关的主版本。读取会校验摘要、字节长度和已记录元数据。路由专用的确定性请求版本单独缓存,完整策略见[统一图片主版本、请求版本和提供方文件](2026-08-20-unified-image-request-pipeline.zh.md)。
第一版不对存储执行自动删除。已发送的用户图片和模型生成图片会一直保留,以供历史记录、恢复和 fork 使用。按引用感知的垃圾回收需要单独设计,因为仅按时间清理可能删除仍被持久会话引用的数据。部署的字节和像素限制是写入时的准入策略;读取时会校验摘要和已记录的元数据,但不重新应用当前准入限制,因此收紧策略不会导致旧历史记录失效。
@@ -114,7 +114,7 @@ type PromptInputPart =
}
```
Base64 只跨越一次协议边界,并在持久化后丢弃。每个入口都会校验规范 base64 与声明的 MIME 形状,再用完整解码批次调用 `AttachmentStore.saveImages()`。服务负责图片数量、总字节数、单张图片字节数、声明 MIME 与完整解码后的光栅图片是否一致、固有尺寸解码像素数它会在保存任何成员之前校验每个批次成员,因此一张畸形图片不会把批次中的有效成员留成无引用对象。随后按提交顺序执行存储提交,以限制完整光栅解码器的内存占用。如果后续存储 I/O 操作失败,调用方不会追加模型可见事件,也不会收到部分引用,但先前的不可变内容寻址对象可能保持无引用状态;第一版将清理留给未来按引用感知的垃圾回收,而不向去重存储添加破坏性回滚。只有每张图片都成功后,入口才会用规范化文本和按协议顺序排列的持久图片块调用 agent。失败时不公开任何附件路径或原始字节。
Base64 只跨越一次协议边界,并在持久化后丢弃。每个入口都会校验规范 base64 与声明的 MIME 字段,再用完整解码批次调用 `AttachmentStore.saveImages()`。服务负责图片数量、总字节数、单张图片字节数、声明 MIME 与完整解码后的光栅图片是否一致、固有尺寸解码像素数和主版本准备。它会在发布任何成员之前只准备并验证每个批次成员一次,因此一张畸形图片不会产生部分引用,大图也不会在提交时重复解码和编码。随后按顺序提交存储。如果后续存储 I/O 操作失败,调用方不会追加模型可见事件,也不会收到部分引用,但先前的不可变内容寻址对象可能按现有存储规则保持无引用状态。只有每张图片都成功后,入口才会用规范化文本和按协议顺序排列的持久图片块调用 agent。失败时不公开任何附件路径或原始字节。
`session.attachment` 是只读且限定于会话作用域的端点。只有该会话中的持久事件引用了所请求的附件标识符,宿主才提供字节。会话处于渲染状态时,客户端会按会话和附件标识符对加载操作去重;已渲染会话释放时会撤销已解析的 URL,并在分配对象 URL 前拒绝已失效的延迟加载,以免已卸载的会话或已释放的服务重新写入缓存。
@@ -122,15 +122,15 @@ Base64 只跨越一次协议边界,并在持久化后丢弃。每个入口都
模型目录项增加可选且可合并扩展的输入模态声明。缺少声明表示未知;声明存在但不含 `image`,则明确表示不支持图片。
宿主是权威的前置检查边界。它会解析会话最新路由到的提供方和模型,并在缺失时依次回退到 agent 选项和宿主默认值;如果模型明确排除图片输入,宿主会在写入任何附件或事件前拒绝提示词,客户端则恢复草稿。包含图片的提示词准入与模型选择共用一逐 agent 串行边界,而且已经出队的提示词在其持久消息事件发布前仍保持待发布状态[顺序决策](../bug-fix/2026-07-29-atomic-web-image-admission.zh.md);steering 载体则从入队起就参与门槛,直到其 `steering/message` 事件发布为止,堵住了从不进入排队镜像的 outbox 窗口。当图片正等待发布或仍存在于会话当前的派生历史中时,模型选择会拒绝纯文本目标。压缩(compaction)可以移除旧图片,使之后选择纯文本目标变得有效;未发布任何事件即转入空闲时,已认领的 queued 载体会被释放,而保留在 outbox 中的 steering 在发布或丢弃前始终受门槛约束`session.updateQueue` 的编辑只接受文本内容,因此队列编辑无法绕过准入边界注入图片。能力未知时继续进入适配器强制检查,使未收录的模型标识符仍然可用。浏览器会在分配预览 URL 前拒绝声明不支持的图片媒体类型,但不会为部署限制或模型能力保留快照:握手快照无法表达 `session.selectModel` 之后会话的当前目标,部署策略也可能独立变化。宿主会根据当前的单张字节数、图片数量、总字节数、媒体类型、尺寸、像素数和路由模型策略校验整个批次,再写入任何附件或事件;拒绝通过 composer 的短时 toast 播报
宿主是权威的前置检查。它会解析会话最新路由到的提供方和模型,并在缺失时依次回退到 agent 选项和宿主默认值;如果模型明确排除图片输入,宿主会在写入附件或事件前拒绝新的图片提示词,客户端则恢复草稿。包含图片的提示词准入与模型选择共用一逐 agent 串行[顺序决策](../bug-fix/2026-07-29-atomic-web-image-admission.zh.md),也包括不进入排队 UI 镜像的 steering。这会为提示词和并发选择提供确定顺序。图片进入持久历史后仍可选择纯文本模型;共享 LLM 运行时会在该请求中把保留的图片块替换为确定的文本占位符`session.updateQueue` 只接受文本内容,因此队列编辑无法绕过准入注入图片。能力未知时继续进入适配器强制检查,使未收录的模型标识符仍然可用。浏览器会在分配预览 URL 前拒绝声明不支持的图片媒体类型,但不会为部署限制或模型能力保留快照。宿主会根据当前的单张字节数、图片数量、总字节数、媒体类型、尺寸、像素数和路由模型策略校验整个批次,再写入附件或事件;拒绝通过 composer 的短时 toast 显示
Pi-AI 与直接 DeepSeek 适配器都会在请求时解析 `ctx.attachments`,递归转换每个持久图片引用,包括嵌套在工具结果中的引用,并且仅为声明支持图片输入的模型生成提供方原生图片内容。直接路由会将 `deepseek-v4-flash-vision-exp` 公布为支持图片,并接受已配置且支持图片的 catalog 配置项;其 Flash、Pro、未声明图片能力的自定义模型和未列出原样传递 id 仍仅支持文本。在请求时解析服务,可避免 Cordis 加载顺序将可选附件服务的可用性固化。任何适配器不得将保留的图片展平或跳过;不支持的角色与模型会以类型化的 `UNSUPPORTED_CONTENT` 失败。
Pi-AI 与直接 DeepSeek 适配器都会在请求时解析 `ctx.attachments`,递归转换每个保留的图片引用,包括嵌套在工具结果中的引用,并且仅为声明支持图片输入的模型生成提供方原生图片内容。两个适配器都从持久主版本请求同一个确定性路由版本。Pi-AI 在考虑 base64 扩张的请求预算内内联携带它。内置 DeepSeek 路由公布 `deepseek-v4-flash-vision-exp`,把每个保留的版本上传到 Files API,并通过索引复用、过期处理、有界陈旧 ID 重试、配额清理和显式删除发送 `file_id` 块。DeepSeek 纯文本模型、未声明图片能力的自定义模型和未列出的透传 ID 保持纯文本。在请求时解析服务,可避免 Cordis 加载顺序将可选附件服务的可用性固化。适配器不得展平或静默跳过保留图片;不支持的角色与模型会以类型化的 `UNSUPPORTED_CONTENT` 失败。
核心层支持结构化助手图片块,但当前没有任何生产提供方路径通过图片输出认证。未来任何支持输出的适配器都必须在有界的大小和时间策略下获取提供方字节,通过同一个附件服务校验并持久化字节,之后才能以原子方式发布 `ImageBlock`。助手 Markdown 中的 URL 仍是文本,绝不自动下载。
提供方无关的 token 估算不会根据图片尺寸猜测视觉定价;提供方返回的用量仍是权威值。只有配置的确切路由与附件部署可以接受图片时,ACPAgent Client Protocol)才公布图片提示词能力;它会在发布用户事件前持久化内联输入,并重新读取已提交的助手图片引用来发送原生 ACP 图片更新。MCP 为程序化调用方保留规范原始块,同时把已准入图片投影为持久核心块;Code Mode 会把任何已经结算且含图片的子结果经外层结果转运为带来源归属且写入日志的上下文。
压缩会把选定的会话前缀(包含图片引用回放到已配置的摘要生成路径。支持视觉的路径会通过适配器解析这些引用;仅文本路径会明确失败,而不是静默丢弃视觉上下文。合成的检查点仍仅包含文本,`compaction-basic` 会以 `UNSUPPORTED_CONTENT` 拒绝包含图片的摘要输出。
压缩会把选定的会话前缀和其中的图片引用回放到已配置的摘要生成路径。支持视觉的路径使用与普通轮次相同的确定性请求版本。纯文本路径接收与其他 LLM 请求相同的确定性附件占位符。合成的检查点仍仅包含文本,`compaction-basic` 会以 `UNSUPPORTED_CONTENT` 拒绝包含图片的摘要输出。
### 历史渲染与原图预览
@@ -140,7 +140,7 @@ Pi-AI 与直接 DeepSeek 适配器都会在请求时解析 `ctx.attachments`
### 限制与信任边界
第一版仅接受 PNG、JPEG、WebP 和 GIF。不接受 SVG 和远程 URL。默认限制为每张图片 3.5 MiB、每条消息 20 张图片和 100 MiB 图片总字节数、每张图片 4,000 万个固有像素,以及任一边 2,000 像素。这些随部署变化的限制属于经过校验的后端配置,并由宿主在持久化前强制执行。客户端连接载体为每个 API 请求设置独立且可配置的 `maxRequestBodyBytes` 上限默认 160 MiB;如果该上限无法容纳附件服务的图片总量限制经 base64 和请求封装膨胀后的大小,加载就会失败。因此,降低图片策略绝不会静默降低有效文本或其他 RPC 的载体上限。未声明长度的请求体在越过上限的瞬间即被拒绝,而不是先读完再拒。
第一版仅接受 PNG、JPEG、WebP 和 GIF。不接受 SVG 和远程 URL。源文件输入默认限制为每张图片 32 MiB、每条消息 20 张图片和 100 MiB 图片总字节数、每张图片一亿解码像素,以及任一边 16384px。与提供方无关的主版本默认长边 2048px,独立安全上限 4 MiB。提供方请求的像素和编码字节上限是单独的路由策略。这些随部署变化的限制属于经过校验的后端配置,并在持久化或请求发送前强制执行。客户端连接载体为每个 API 请求设置独立且可配置的 `maxRequestBodyBytes` 上限默认 160 MiB;如果该上限无法容纳源文件总量限制经 base64 和请求封装膨胀后的大小,加载就会失败。未声明长度的请求体在越过上限即被拒绝,而不是先读完再拒。
格式错误的 base64、不支持或不匹配的媒体、截断的图片数据、超出字节限制、超出图片数量、超出像素限制、超出单边尺寸限制、对象缺失和完整性不匹配都会返回稳定的结构化错误。原始文件名只保留用于显示的末段,控制字符会被移除,并且任何本地路径都不会写入日志或返回浏览器。
@@ -148,11 +148,11 @@ Pi-AI 与直接 DeepSeek 适配器都会在请求时解析 `ctx.attachments`
| 接口 | 职责 |
| --- | --- |
| `packages/attachment/attachment` | 不透明附件标识符、图片引用、限制、错误,以及通过 `ctx.attachments` 提供的单张/批量准入。 |
| `packages/attachment/attachment-local` | 私有内容寻址存、完整光栅解码、完整性校验和配置。 |
| `packages/llm/llm` | 角色无关的 `ImageBlock`输入模态元数据。 |
| `packages/llm/llm-pi-ai` | 持久且受支持的图片输入解析为提供方原生内容。 |
| `packages/llm/llm-deepseek` | 解析已声明的官方视觉输入,并拒绝纯文本模型的图片。 |
| `packages/attachment/attachment` | 不透明附件和请求版本标识符、图片引用、策略、错误,以及通过 `ctx.attachments` 提供的批量准入、派生读取和裁剪。 |
| `packages/attachment/attachment-local` | 私有内容寻址主版本、确定性请求缓存、完整光栅解码、完整性校验和配置。 |
| `packages/llm/llm` | 角色无关的 `ImageBlock`输入模态元数据、精确适配器代次和纯文本请求投影。 |
| `packages/llm/llm-pi-ai` | 持久图片解析为确定性内联请求版本。 |
| `packages/llm/llm-deepseek` | 把官方视觉输入解析为确定性请求版本和 Files API ID。 |
| `packages/compaction/compaction-basic` | 在摘要输入中保留图片,并明确拒绝非文本检查点输出。 |
| `packages/host/apiproxy``packages/bundle/base` | 范围狭窄的上传协议、共享批量准入、限制和路由模型前置检查、先持久化再追加事件的顺序、会话授权读取,以及默认 profile 组合。 |
| `packages/client/connection``packages/client/runtime` | 有界请求缓冲、协议类型、fixture(测试前置数据)图片、提示词上传、附件读取和持久引用折叠。 |
@@ -165,7 +165,7 @@ Pi-AI 与直接 DeepSeek 适配器都会在请求时解析 `ctx.attachments`
### 实现
已实现的范围包括附件服务边界与共享批量准入、角色无关图片块、Pi-AI 与直接 DeepSeek 输入转换、WebACPMCP 持久化顺序、Web 上传与读取协议、条件式 ACP 图片协议支持、无损 MCP 规范结果与持久图片投影、通用 Code Mode 丰富结果转发、当前图片限制执行、大小受限的 Web 请求体、内存草稿图片、粘贴与拖放附件栏、用户与助手历史图片渲染、单击预览、压缩处理,以及组装后无需密钥的 Web 与 ACP 覆盖。
已实现能力包括只准备一次的共享批量准入、与提供方无关的主版本、确定性请求版本、DeepSeek Files 复用、稳定裁剪句柄、角色无关图片块、Pi-AI DeepSeek 输入转换、WebACPMCP 持久化顺序、Web 上传与读取协议、条件式 ACP 图片支持、带持久图片投影的无损 MCP 结果、Code Mode 丰富结果转发、有界 Web 请求、草稿与历史图片 UI、压缩处理,以及组装后的无密钥覆盖。
预发布提示词协议不需要兼容包装层;引入相应切片时会同时修改所有调用点和 fixture。
@@ -210,11 +210,11 @@ UI 状态可能陈旧,也无法保护直接 SDK、ACP、回放或未收录模
## 测试
- 存储测试覆盖内容寻址去重、私有权限、准入失败、对象损坏或缺失时的失败,以及收紧部署限制后读取历史数据。
- 宿主与协议测试覆盖先持久化再追加事件的顺序、日志中不含 base64、会话作用域授权、能力拒绝、上传限制、大小受限的 HTTP 请求体、图片准入与模型选择的竞态(排队与 steering 两种放置)、待发布状态、未发布即空闲时的门槛释放、仅文本的队列编辑,以及压缩后依据当前派生历史进行的选择
- 宿主与协议测试覆盖先持久化再追加事件的顺序、日志中不含 base64、会话作用域授权、能力拒绝、上传限制、大小受限的 HTTP 请求体、图片准入与模型选择的排序、仅文本的队列编辑,以及纯文本请求投影
- 客户端单元测试覆盖粘贴与拖放、混合剪贴板文本、仅图片发送、草稿恢复、顺序、草稿、会话作用域和应用层级的对象 URL 清理,以及一项在释放后才完成的延迟历史读取;keyless 的组装后构建产物通道(`apps/web/tests/image-display.snapshot.ts``DSH_EXAMPLE_MODE=lib pnpm run test:snapshot`)覆盖经授权附件路由渲染的历史用户与助手图片画廊、原图 lightbox,以及 composer 粘贴缩略图条。
- 适配器与压缩测试覆盖 Pi-AI 原生图片转换、后置附件服务组合、仅文本拒绝、递归嵌套在工具结果中的图片、保留摘要输入,以及明确拒绝图片输出。
- 适配器与压缩测试覆盖确定性 Pi-AI 请求版本、DeepSeek Files 上传与复用、陈旧 ID 恢复、纯文本投影、递归嵌套在工具结果中的图片、共享摘要请求版本,以及明确拒绝图片输出。
- 附件、MCP、ACP 与 Code Mode 测试覆盖写入前校验全部成员、图文混合顺序、持久事件不含内联 base64、确切路由能力门禁、明确的不支持内容诊断、post-execute 替换/阻止优先级、准入期间取消、经过校验的助手图片交付,以及通用嵌套图片转发。组装后的无密钥 ACP 快照发送真实内联 PNG,并在会话日志中只固定其持久引用。
- 需要凭据的实际 API 测试会通过 Anthropic `claude-opus-4-8` 路径发送一张 PNG,并要求模型识别其中的二维码
- 需要凭据的实际 API 测试会覆盖配置的 Anthropic 路由和内置 `deepseek-official` Files 路径。DeepSeek 测试不使用自定义提供方条目
- 当前生产适配器集合没有经过认证的图片输出路由;输出提供方认证仍不在第一版范围内。
## 后果
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-08-10-minimal-read-image-tool.md
2026-08-10-minimal-read-image-tool.md: a43e53d70e98bac7a50aa6bbabbb1e177237df01
2026-08-10-minimal-read-image-tool.zh.md: a94e4b296425ad50876b0b45a689442c896a85a1
2026-08-10-minimal-read-image-tool.md: 19306a35fe709a04d94090a62056575b4d51f7bc
2026-08-10-minimal-read-image-tool.zh.md: c7562c433e909d1f81361c0ced56318795e6469e
@@ -6,28 +6,27 @@ English | [中文](2026-08-10-minimal-read-image-tool.zh.md)
## Problem
The multimodal attachment work gave user uploads a complete durable path — bytes committed to the content-addressed attachment store before the owning `user/message`, an `ImageBlock` carrying only the `sha256:` reference, and the pi-ai route re-reading verified bytes per request — but the model itself had no way to look at an image on disk. `read` rejects binary content by contract, so an agent asked about a screenshot or a rendered chart either failed or shelled out to lossy workarounds. A first standalone attempt (PR #598) solved this together with loop-level route scoping: an `agent/request-ready` extension point publishing exact-model modalities before assembly, per-route schema/guidance visibility, and a reversible `image-placeholder-v1` history projection so text routes could continue over placeholder text. That design worked but coupled a tool to new agent-loop machinery, three new session-log concepts, and per-step registration churn — far more surface than the capability needs.
The multimodal attachment work gave user uploads a complete durable path, but the model itself had no way to inspect an image on disk. `read` rejects binary content by contract, so an agent asked about a screenshot or rendered chart either failed or used a lossy workaround. A standalone attempt in PR #598 combined the tool with loop-level route scoping, per-route schema visibility, and new session-log concepts. Those features were not required to publish a logged image tool result.
## Decision
Ship the smallest tool that loads an image into the next request's context, entirely over existing seams; the withdrawn PR #598 design is the explicit counter-example this note records.
Both image-reading operations live in `dsh-tool-fs` and publish ordinary logged tool results over existing extension points.
- **`read_image` lives in `dsh-tool-fs`** beside `read`/`write`/`edit`. Extension selects the declared PNG/JPEG/WebP/GIF media type; the attachment store's magic-byte and pixel validation stays authoritative. Bytes travel `ctx.fs.stat` → bounded `ctx.fs.readBytes``ctx.attachments.saveImage``fs/observed`, and the tool result is the metadata envelope plus a real `ImageBlock``ToolResultBlock.content` already admits image blocks, the pi-ai adapter already renders them, and the Web host's model-switch guard already scans tool results, so nothing downstream changes.
- **`read_image` reads a filesystem path.** Extension selects the declared PNG/JPEG/WebP/GIF media type; the attachment store's magic-byte and pixel validation stays authoritative. Bytes travel `ctx.fs.stat` → bounded `ctx.fs.readBytes``ctx.attachments.saveImage``fs/observed`. The tool result contains metadata and an `ImageBlock`.
- **`FileSystem.readBytes(target, signal, maxBytes)`** is a new required provider primitive: the byte bound lives at the seam so no backend can buffer an unbounded file, with the stat-size short-circuit and a one-byte-past-cap stream guard against post-stat growth (`FS_TOO_LARGE`).
- **Registration is composition-conditional, execution is route-gated.** The tool registers only under `ctx.inject(['attachments'], …)` — no store, no tool. At execution, before any I/O, the strict gate resolves the calling route (latest `request/header` config, falling back to agent options) through `ctx.llm.resolveModelInfo` and requires `image` in `inputModalities`; unknown capability refuses. A refusal is a plain `isError` result, so a text route's durable history never acquires an image block and the session cannot brick its own route.
- **Registration is composition-conditional, execution is route-gated.** The tools register only under `ctx.inject(['attachments'], …)`. Before I/O, the strict gate resolves the calling route through `ctx.llm.resolveModelInfo` and requires `image` in `inputModalities`; unknown capability refuses. A text-only route can still consume prior durable images because the shared LLM runtime projects them to placeholders at request assembly.
- **Code Mode forwards the image out-of-band**: a nested dispatch returns the canonical value (execution-local, no image block) and defers a `user`-role context message carrying the envelope and image, so the picture still reaches the next request.
- **llm-replay models may declare `inputModalities`**, which is what lets the two keyless ACP snapshots pin both sides of the gate — the sha256-referenced success on an image-capable replay route and the verbatim refusal on a text-only one.
- **llm-replay models may declare `inputModalities`**, which lets keyless ACP snapshots cover the image-capable result and the text-only refusal.
## Alternatives considered
- **PR #598's route-scoped design** (request-ready seam, per-route schema/guidance visibility, reversible history projection) — withdrawn in favor of this note's shape. What it bought: text routes could keep running after images entered history, and the tool disappeared from prompts where it cannot succeed. What it cost: agent-loop changes, three new durable concepts (`agent/request-ready`, `messageProjection`, availability notices), and registration that churned per step. The capability itself — see an image on the next request — never needed any of it. If per-route projection becomes a real requirement, that PR's history is the reference implementation.
- **PR #598's route-scoped design** used a request-ready extension point, per-route schema visibility, reversible projection, and three durable concepts. Shared LLM request projection now handles text-only routes without putting tool registration or session formats into agent-loop.
- **`agent.inject()` instead of the image-bearing tool result** — routes the image around the tool result as a separate injected user message. Rejected: the image *is* the tool's result; splitting them adds a second logged message with no gain, and the tool-result path already works end to end.
- **Magic-byte sniffing instead of extension declaration** — sniffing duplicates detection the attachment store already owns (sharp-backed, authoritative). The extension is only a *declaration*; a mismatch fails closed with a rename remedy rather than being silently accepted, which also keeps the model's mental map (file name ↔ content) honest.
- **Registering unconditionally and failing on a missing store** — rejected; a deployment without an attachment store cannot ever satisfy the tool, so its schema would be a standing lie. The route gate, by contrast, is per-call state and correctly lives at the execution boundary.
## Consequences
- A text-only route refuses instead of degrading: no placeholder projection means no delegated-viewing story here — that is deliberately the next PR (subagent image readback rebuilt on the current subagent seams).
- The route gate races a concurrent model switch; the Web host's image-aware switch guard covers its surface, and other front doors own their equivalent. Recorded as a tool-fs Known Limitation.
- Repeated image results accumulate request-token cost until compaction; content addressing deduplicates bytes only.
- The tools refuse execution on a text-only route, while existing images in session history are represented by request-local placeholders.
- Repeated image results accumulate request cost until request projection or compaction removes them; content addressing deduplicates durable bytes.
- The tool-result card renders the durable reference, not pixels; inline preview is deferred to the UI packages.
@@ -6,28 +6,27 @@ Status: implemented
## 问题
多模态附件工作为用户上传建立了完整的持久路径:字节在所属 `user/message` 之前提交到内容寻址的附件存储,`ImageBlock` 只携带 `sha256:` 引用,pi-ai 路由在每次请求时重新读取并校验字节。但模型自己没有查看磁盘图像的手段`read` 按约定拒绝二进制内容,因此被问到截图或渲染图表的 agent 要么失败,要么退到有损的变通法。第一次独立尝试(PR #598)把这个问题与循环级路由作用域一起解决:新增在组装前发布确切模型模态的 `agent/request-ready` 扩展点、按路由控制 schema/指导可见性,以及可逆的 `image-placeholder-v1` 历史投影让文本路由能在占位符上继续。该设计可行,但让一个工具耦合了新的 agent-loop 机制、三个新的会话日志概念和每步的注册变动,远超这项能力本身的需要
多模态附件工作为用户上传建立了完整的持久路径,但模型无法查看磁盘图`read` 按约定拒绝二进制内容,因此被问到截图或渲染图表的 agent 要么失败,要么使用有损的变通法。PR #598 的独立尝试把工具与循环级路由作用域、按路由控制 schema 可见性和新的会话日志概念放在一起。这些能力不是发布一条带图片且已记录的工具结果所必需的
## 决定
只交付能把图像载入下一次请求上下文的最小工具,完全建立在既有 seam 之上;撤回的 PR #598 设计是本记录明确保留的反例
两个图片读取操作都放在 `dsh-tool-fs`,通过现有扩展点发布普通的持久工具结果
- **`read_image` 放在 `dsh-tool-fs`**,与 `read`/`write`/`edit` 并列。扩展名选择声明的 PNG/JPEG/WebP/GIF 媒体类型附件存储的魔数与像素校验保持权威。字节沿 `ctx.fs.stat` → 有界 `ctx.fs.readBytes``ctx.attachments.saveImage``fs/observed` 流动工具结果元数据信封加真正的 `ImageBlock`——`ToolResultBlock.content` 本就允许图像块,pi-ai 适配器本就会渲染它们,Web 宿主的模型切换防护本就会扫描工具结果,下游无需任何改动
- **`read_image` 读取文件系统路径。** 扩展名选择声明的 PNG/JPEG/WebP/GIF 媒体类型附件存储的魔数与像素校验保持权威。字节沿 `ctx.fs.stat` → 有界 `ctx.fs.readBytes``ctx.attachments.saveImage``fs/observed` 流动工具结果包含元数据和一个 `ImageBlock`
- **`FileSystem.readBytes(target, signal, maxBytes)`** 是新的必备提供方原语:字节上限放在 seam 上,任何后端都无法无界缓冲文件;stat 大小先短路,随后的流最多多读一个字节以防 stat 之后的增长(`FS_TOO_LARGE`)。
- **注册随组合条件挂载,执行按路由门禁。** 工具只在 `ctx.inject(['attachments'], …)` 作用域内注册——没有存储就没有工具。执行时在任何 I/O 之前,严格门禁通过 `ctx.llm.resolveModelInfo` 解析调用路由(最新 `request/header` 配置,缺失时回退到 agent 选项),要求 `inputModalities` 包含 `image`;能力未知即拒绝。拒绝是普通的 `isError` 结果,因此文本路由的持久历史绝不会出现图像块,会话不会毁掉自己的路由
- **注册随组合条件挂载,执行按路由门禁。** 工具只在 `ctx.inject(['attachments'], …)` 作用域内注册。执行时在 I/O 之前通过 `ctx.llm.resolveModelInfo` 解析调用路由,要求 `inputModalities` 包含 `image`;能力未知即拒绝。纯文本路由仍可使用此前的持久图片,因为共享 LLM 运行时会在请求组装时把图片投影为占位符
- **Code Mode 以带外方式转发图像**:嵌套分派返回规范值(仅限本次执行,不含图像块),并延迟提交一条携带信封和图像的 `user` 角色上下文消息,图片仍会到达下一次请求。
- **llm-replay 模型可以声明 `inputModalities`**这正是两个 keyless ACP 快照能钉住门禁两侧的原因:图像路由上以 sha256 引用的成功结果和纯文本路由上逐字的拒绝。
- **llm-replay 模型可以声明 `inputModalities`**因此 keyless ACP 快照可以覆盖支持图片的结果和纯文本拒绝。
## 考虑过的替代方案
- **PR #598 的路由作用域设计**request-ready 扩展点、按路由 schema/指导可见性、可逆历史投影)——被本记录的形态取代后撤回。它换来的是:图像进入历史后文本路由仍能运行,工具在注定失败的提示词里消失。它付出的是:改动 agent-loop、三个新的持久概念(`agent/request-ready``messageProjection`、可用性通知)和每步变动的注册。而这项能力本身——下一次请求看到图像——从不需要这些。如果按路由投影将来成为真实需求,该 PR 的历史就是参考实现
- **PR #598 的路由作用域设计**使用 request-ready 扩展点、按路由控制 schema 可见性、可逆投影和三个持久概念。共享 LLM 请求投影现在可以处理纯文本路由,无需把工具注册或会话格式放进 agent-loop
- **用 `agent.inject()` 代替带图像的工具结果**——把图像绕过工具结果,作为单独注入的用户消息。拒绝:图像就是工具的结果;拆开只会多一条无收益的日志消息,而工具结果路径本就端到端可用。
- **用魔数嗅探代替扩展名声明**——嗅探重复了附件存储已拥有的检测(基于 sharp,权威)。扩展名只是声明;不匹配时按改名修复提示失败关闭,而不是被静默接受,这也让模型对文件名与内容的对应保持诚实。
- **无条件注册、缺存储时执行报错**——拒绝;没有附件存储的部署永远无法满足该工具,其 schema 会是常态谎言。相反,路由门禁是逐调用状态,正确的位置就是执行边界。
## 后果
- 纯文本路由得到拒绝而不是降级:没有占位符投影意味着这里没有委托查看的方案——那有意留给下一个 PR(基于当前 subagent seam 重建的 subagent image readback
- 路由门禁与并发模型切换存在竞态;Web 宿主的图像感知切换防护覆盖其表面,其他前端拥有各自的等价防护。已记入 tool-fs 的已知限制
- 重复的图像结果在压缩之前持续累积请求 token 成本;内容寻址只去重字节。
- 工具在纯文本路由上拒绝执行,而会话历史中已经存在的图片会由请求期占位符表示
- 重复的图片结果会累积请求成本,直到请求投影或压缩将其移除;内容寻址只去重持久字节
- 工具结果卡片渲染持久引用而非像素;内嵌预览延后到 UI 包处理。
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-08-12-web-image-intake-and-limits-alignment.md
2026-08-12-web-image-intake-and-limits-alignment.md: 00cf7ea99d63e848c4b5839da1d97d94c9fb8464
2026-08-12-web-image-intake-and-limits-alignment.zh.md: 7bf7f3621d6d305baf8e7c1c060bbc5810f28b77
2026-08-12-web-image-intake-and-limits-alignment.md: 0bb8cadc8db4b4c28cf311bc9420c32744e173fb
2026-08-12-web-image-intake-and-limits-alignment.zh.md: fafa7652756554a54a0a0952e843bf5f4b81b00d
@@ -16,7 +16,7 @@ The second alignment step for issue #2248, after the [attachment display note](2
**History thumbnails (DeepSeek Chat rules).** A message's lone image renders at 240px on its long edge with the displayed ratio clamped to [0.25, 4], cropped by `cover` with the anchor at the top of very tall images and the left of very wide ones, never upscaled; several images render as fixed 64px square tiles in one wrapping row (10px gap, user messages right-aligned). Consecutive assistant `image` blocks merge into one gallery so they tile instead of each opening a one-image row.
**Limits aligned and projected.** Defaults are 20 images / 3.5 MiB per image / 100 MiB aggregate (`attachment-local`), with the HTTP carrier cap raised to one shared `DEFAULT_MAX_REQUEST_BODY_BYTES = 160 MiB` (http-bridge, previously two independent 32 MiB literals) to satisfy the load-time capacity assertion (aggregate × 4/3 + headroom ≈ 134.3 MiB). Consumer products cluster at 1020 attachments (ChatGPT 10, Gemini 10, Claude 20; DeepSeek Chat's 50 is the outlier), and a vision-model image costs roughly 13004800 tokens, so 50 images can fill a 200k context in one message. Including base64 padding, a 3.5 MiB encoded file occupies at most 4.67 MiB and leaves 0.33 MiB below a 5 MiB route check. Deployments using only routes with larger limits can override it. A 512 MiB aggregate cannot pass this transport because base64-in-JSON would need a single JSON string past V8's ~512 MiB string ceiling. The limits reach clients as the `imageLimits` session projection a constant-per-boot unit (`apply` returns the same state reference, so baselines alone carry it and no change frames exist) registered by **apiproxy**, not the attachment Service Definition: `dsh-llm` depends on `dsh-attachment` (`ImageBlock``ImageAttachmentRef`), so the seam package referencing `dsh-session-projection` (whose graph reaches `dsh-llm` through `dsh-session`) closes a project-reference cycle, and the per-message count/aggregate rules the value describes are the proxy's own admission checks anyway. The `SessionProjectionMap` merge rides the proxy's sessions wire-contract file, which every client program already includes through the carrier's type re-exports.
**Limits aligned and projected.** Intake defaults are 20 images, 32 MiB per source, 100 MiB aggregate source bytes, 100 million decoded pixels, and 16384px per source side. The attachment backend prepares a separate durable master with a 2048px long edge and 4 MiB safety cap. Model requests have their own route-specific pixel and encoded-byte budgets, so source admission does not use provider request limits. The HTTP carrier uses one shared `DEFAULT_MAX_REQUEST_BODY_BYTES = 160 MiB` to satisfy the load-time capacity assertion for the 100 MiB aggregate after base64 and envelope expansion. A 512 MiB aggregate cannot pass this transport because base64-in-JSON would require a JSON string near V8's string-size limit. The intake limits reach clients as the `imageLimits` session projection, a constant-per-boot unit registered by **apiproxy** rather than the attachment Service Definition. `dsh-llm` depends on `dsh-attachment`, while `dsh-session-projection` reaches `dsh-llm` through `dsh-session`; registering the projection in the seam package would create a project-reference cycle. The per-message count and aggregate rules are also enforced by the proxy. The `SessionProjectionMap` merge remains in the proxy sessions wire file, which clients already consume through carrier type re-exports.
**Intake pre-check and error copy.** Both intake gestures converge on one `intakeImages` wrapper in InputBar that checks count, per-image bytes, and aggregate bytes against the projection before `addImages`: a violating batch is refused whole (DeepSeek Chat semantics) with an immediate banner naming the limit — no submit-time rollback theater. The host checks stay as the backstop for callers that bypass the composer. Banner copy follows one principle the user set: reasons a user can act on (model without vision, count, size, resolution, format — now a positive list of supported formats instead of echoing the rejected MIME type) get product sentences naming the way out; reasons they cannot act on (corrupt base64, lost references, read failures) fold into one send-failed sentence that keeps the reason code, because the product currently faces developers and a reportable code beats a dead end. Non-attachment error codes keep the raw message + code presentation.
@@ -16,7 +16,7 @@ issue #2248 的第二步对齐,接在[附件展示 note](2026-08-11-web-attach
**历史缩略图(DeepSeek Chat 规则)。** 一条消息仅有的一张图长边 240px、展示比例钳制在 [0.25, 4]`cover` 裁切,特别高的图锚定顶部、特别宽的锚定左侧,从不放大;多张图渲染为固定 64px 方块,单个可换行的横排(10px 间距,用户消息右对齐)。assistant 连续的 `image` 块合并进同一个画廊,平铺而不是各占一行。
**上限对齐并投影。** 默认值每条消息 20 张、单图 3.5 MiB、总量 100 MiB`attachment-local`),HTTP 载体上限提为唯一共享的 `DEFAULT_MAX_REQUEST_BODY_BYTES = 160 MiB`http-bridge,原先是两个独立的 32 MiB 字面量),以满足加载时的容量断言(总量 × 4/3 加余量 ≈ 134.3 MiB)。消费级产品集中在 10 到 20 个附件(ChatGPT 10、Gemini 10、Claude 20DeepSeek Chat 的 50 是例外),且视觉模型一张图约 1300 到 4800 token,因此 50 张图可在一条消息中填满 200k 上下文。3.5 MiB 编码文件包括 base64 填充在内最多占 4.67 MiB,在 5 MiB 路由检查下保留 0.33 MiB 余量。仅使用较大上限路由的部署可以覆盖该值。512 MiB 总量无法通过当前传输,因为 base64 进 JSON 需要一个超过 V8 约 512 MiB 字符串上限的单个 JSON 字符串。限额以 `imageLimits` 会话投影到达客户端。它是每次启动恒定的单元`apply` 返回同一状态引用,因此只靠基线携带、不存在变更帧),由 **apiproxy** 而非 attachment Service Definition 注册`dsh-llm` 依赖 `dsh-attachment``ImageBlock``ImageAttachmentRef`),seam 包引用 `dsh-session-projection`(其图谱`dsh-session` 到达 `dsh-llm`)会闭合 project-reference 环,而该值描述的每消息数量总量规则本来就是 proxy 自己的准入检查`SessionProjectionMap` 合并放在 proxy 的 sessions 协议文件里,每个客户端程序都经载体类型再导出包含它。
**上限对齐并投影。** 输入默认值每条消息 20 张、每个源文件 32 MiB、源文件总量 100 MiB、每张图片一亿解码像素,以及源文件任一边 16384px。附件后端另行生成长边 2048px、独立安全上限 4 MiB 的持久主版本。模型请求使用各路由自己的像素和编码字节预算,因此源文件准入不采用提供方请求限制。HTTP 载体统一使用 `DEFAULT_MAX_REQUEST_BODY_BYTES = 160 MiB`,满足 100 MiB 总量经过 base64 和请求封装扩张后的加载时容量断言。512 MiB 总量无法通过当前传输,因为 base64 进 JSON 后会需要一个接近 V8 字符串大小上限的 JSON 字符串。输入上限通过 `imageLimits` 会话投影到达客户端。它是每次启动恒定的单元,由 **apiproxy** 而非 attachment Service Definition 注册`dsh-llm` 依赖 `dsh-attachment`,而 `dsh-session-projection` `dsh-session` 到达 `dsh-llm`;在 seam 包注册投影会形成 project-reference 环。每条消息数量总量规则也由 proxy 强制执行`SessionProjectionMap` 合并继续放在 proxy 的 sessions 协议文件中,客户端已经通过载体类型再导出使用它。
**加入预检与错误文案。** 两种加入手势汇合到 InputBar 的一个 `intakeImages` 包装:在 `addImages` 之前按投影检查数量、单图字节与总字节,违规的一批整体拒收(DeepSeek Chat 语义)并立刻弹出点名上限的横幅——不再有提交时的回滚戏码。宿主检查保留,兜底绕过 composer 的调用方。横幅文案遵循用户定下的一条原则:用户能解决的原因(模型不支持视觉、数量、大小、分辨率、格式——格式改为正面列出支持列表而不是回显被拒的 MIME 类型)用点明出路的产品句子;用户无法解决的原因(base64 损坏、引用丢失、读取失败)折叠为一条保留原因码的发送失败句子,因为产品当前面向开发者,可上报的码好过死胡同。非附件错误码保留原文加错误码的展示。
@@ -1,34 +0,0 @@
# Agent Note: Direct DeepSeek vision input
Status: implemented
English | [中文](2026-08-19-direct-deepseek-vision-input.zh.md)
## Problem
DeepSeek vision deployments use the chat-completions image protocol, but the direct `deepseek-official` adapter declares every catalog and pass-through model text-only and rejects every `ImageBlock`. The durable attachment path therefore works only through configurable pi-ai routes, and a deployment cannot pass user uploads or image-bearing tool results through the direct provider.
## Decision
The shipped catalog declares `deepseek-v4-flash-vision-exp` with `inputModalities: [text, image]`; configured catalogs use the same declaration to opt another exact model into image input, and validation rejects empty, unknown, or duplicate modalities. Flash, Pro, unlisted ids, and configured models that omit `inputModalities` remain explicitly text-only.
The adapter resolves `ctx.attachments` per image request, reads each retained durable reference with the request signal, and serializes verified bytes as ordered OpenAI-compatible `image_url` data URLs. Text-only user messages retain string content. Tool results retain string-only `tool` messages; image-only results use `(see attached image)`, and consecutive retained tool-result images follow in one `user` message beginning `Attached image(s) from tool result:`. System and assistant history images fail with `UNSUPPORTED_CONTENT` before attachment or network I/O.
The direct adapter and pi-ai conversion share the deterministic [request-level image payload bound](../bug-fix/2026-08-18-request-image-payload-bound.md). Both default to 20 MiB of accumulated base64 payload, replace oldest image occurrences with the same fixed placeholder, and never read omitted attachments. Direct HTTP 413 responses are `INVALID_REQUEST`; attachment failures retain their stable attachment code rather than becoming `TRANSPORT`.
Canonical messages continue to store only `ImageAttachmentRef`. Data URLs exist only while preparing one provider request, so no session event, persistence format, API schema, or SDK projection changes. The route accepts PNG, JPEG, WebP, and GIF already admitted by the attachment service. External image URLs, the Files API, and image output remain unsupported.
## Alternatives considered
- **Use only the pi-ai DeepSeek provider.** Its generic multimodal path proves the content conversion, but it does not make the direct official route truthful or usable with the official model id.
- **Declare the whole provider image-capable.** This would let Flash, Pro, and unknown pass-through ids accept durable images that their exact wire model cannot promise to consume. Capability remains exact-model metadata.
- **Send images inside `tool` message content.** The documented compatible form keeps tool content a string. A following user message avoids relying on an undocumented multimodal tool-role form while preserving call-result order.
- **Add external URLs or Files uploads.** Both require new canonical input, authorization, lifetime, cleanup, and replay decisions. Transient base64 uses the existing durable attachment contract without expanding those concerns.
## Verification
Package tests pin model discovery and fallback capabilities, configuration validation and live settings updates, user and tool-result wire messages, all admitted MIME types, cancellation, attachment failures, 413 classification, exact image-bound behavior, and pi-ai equivalence. A keyless assembled ACP request records the native adapter's tool-result data URL and oldest-image placeholder. A real-API smoke test with an explicit image-capable catalog entry sends a deterministic image only when `DEEPSEEK_VISION_E2E=1` is set in addition to the provider key.
## Consequences
The official DeepSeek vision route and configured vision routes can consume durable user and tool-result images without changing session durability or response streaming. Repeated history still expands request bodies, but deterministic oldest-first offload bounds the dominant payload and leaves headroom below the official 30 MiB request-body limit. Image token pricing remains provider-owned because the official image token formula is not available.
@@ -1,34 +0,0 @@
# Agent Note: 直接 DeepSeek 视觉输入
Status: implemented
[English](2026-08-19-direct-deepseek-vision-input.md) | 中文
## Problem
DeepSeek 视觉部署使用 chat-completions 图片协议,但直接 `deepseek-official` 适配器把所有 catalog 与原样传递模型都声明为仅文本,并拒绝每一个 `ImageBlock`。因此,持久附件路径只能经可配置 pi-ai 路由工作,部署方无法通过直接提供方传递用户上传或包含图片的工具结果。
## Decision
随附目录为 `deepseek-v4-flash-vision-exp` 声明 `inputModalities: [text, image]`;已配置目录可以用同一声明让另一个确切模型支持图片输入,校验会拒绝空列表、未知模态或重复模态。Flash、Pro、未列出 id,以及省略 `inputModalities` 的已配置模型仍明确仅支持文本。
适配器会对每个图片请求解析 `ctx.attachments`,用请求 signal 读取每个保留的持久引用,并将校验后的字节按顺序序列化为 OpenAI 兼容的 `image_url` data URL。纯文本 user 消息保留字符串内容。工具结果保留仅字符串的 `tool` 消息;仅含图片的结果使用 `(see attached image)`,连续工具结果中保留的图片随后合并进一条以 `Attached image(s) from tool result:` 开头的 `user` 消息。System 与 assistant 历史图片会在附件或网络 I/O 前以 `UNSUPPORTED_CONTENT` 失败。
直接适配器与 pi-ai 转换共享确定性的[请求级图片载荷上限](../bug-fix/2026-08-18-request-image-payload-bound.zh.md)。两者都以 20 MiB 累计 base64 payload 为默认值,用相同固定占位文本替换最旧的图片出现位置,并且绝不读取被省略的附件。直接 HTTP 413 响应归类为 `INVALID_REQUEST`;附件失败会保留其稳定附件 code,不会变成 `TRANSPORT`
规范消息继续只存储 `ImageAttachmentRef`。Data URL 只在准备单次提供方请求时存在,因此无需修改会话事件、持久化格式、API schema 或 SDK 投影。路由接受已经由附件服务准入的 PNG、JPEG、WebP 和 GIF。不支持外部图片 URL、Files API 和图片输出。
## Alternatives considered
- **只使用 pi-ai DeepSeek 提供方。** 其通用多模态路径验证了内容转换,但无法让直接官方路由如实公布能力,也无法让它配合官方模型 id 使用。
- **把整个提供方声明为支持图片。** 这样会让 Flash、Pro 和未知的原样传递 id 接受持久图片,但其确切协议模型无法承诺消费这些图片。能力仍属于确切模型元数据。
- **在 `tool` 消息内容中发送图片。** 已记录的兼容形式要求工具内容保持字符串。随后发送 user 消息可避免依赖未记录的多模态 tool role 形式,同时保留调用结果顺序。
- **增加外部 URL 或 Files 上传。** 两者都需要新的规范输入、授权、生命周期、清理和重放决策。瞬态 base64 可以复用现有持久附件约定,不扩展这些问题。
## Verification
包测试固定模型发现与回退能力、配置校验与存活 settings 更新、user 和工具结果协议消息、所有已准入 MIME 类型、取消、附件失败、413 分类、确切图片上限行为和 pi-ai 等价性。无需密钥的组装 ACP 请求会记录原生适配器的工具结果 data URL 与最旧图片占位文本。真实 API 冒烟测试会配置明确支持图片的目录项,并且仅在提供方密钥之外还设置 `DEEPSEEK_VISION_E2E=1` 时发送确定性图片。
## Consequences
官方 DeepSeek 视觉路由与已配置视觉路由可以消费持久 user 与工具结果图片,而无需改变会话持久性或响应流。重复历史仍会扩张请求正文,但确定性的最旧优先 offload 会限制主导 payload,并在官方 30 MiB 请求正文上限下保留余量。由于官方图片 token 公式尚不可用,图片 token 定价仍由提供方掌握。
@@ -1,6 +1,6 @@
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/bug-fix/2026-08-18-request-image-payload-bound.md
2026-08-18-request-image-payload-bound.md: 0ec4594888db6157fb8cfd3e7bdb231b842d53c1
2026-08-18-request-image-payload-bound.zh.md: 7cdf6bb768251cb792b6d590fafa094646e77ada
# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-08-20-unified-image-request-pipeline.md
2026-08-20-unified-image-request-pipeline.md: ada15d540539977c631e359ffdc7baa4fa84c78e
2026-08-20-unified-image-request-pipeline.zh.md: 85c9a1f837d82cba2bc62b30402433f50c873cbe
@@ -0,0 +1,69 @@
# Agent Note: Unified normalized attachments, request versions, and provider files
Status: implemented
English | [中文](2026-08-20-unified-image-request-pipeline.zh.md)
## Problem
Durable image history, provider resolution, inline request size, and remote file reuse have different limits. Treating an admitted image as the bytes sent on every later request forced one byte cap and one raster to serve all four concerns. Large but ordinary input was refused, clean 16-bit PNG could pass into history and fail at DeepSeek, repeated base64 expanded long requests, and a provider rejection repeated because the same durable image stayed in every future request.
## Decision
The image path has two explicit versions. The attachment backend owns a provider-independent durable normalized attachment. Each image-capable model route owns a deterministic request policy, and the attachment backend derives and caches the exact request version from that attachment. Session history contains only the normalized attachment reference; inline bytes and provider file ids remain transient request projections.
### Provider-independent normalized attachment
Admission accepts at most 20 images and 200MiB of encoded source bytes per message. Each source is fully decoded under configurable 20MiB, 64,000,000-pixel, and 8192px-per-side limits. Normalization applies EXIF orientation, removes metadata and color profiles, converts to 8-bit sRGB/sRGBA, and preserves aspect ratio while limiting the long edge to `normalizedImageMaxDimension`, 2048px by default. When scaling reduces the raster, `originalDimensions` records its orientation-applied width and height before normalization.
The normalized attachment has an independent `normalizedImageMaxBytes` safety cap, 4MiB by default. Alpha is never flattened. A nearest-neighbour bounded sample classifies color complexity without averaging high-frequency pixels. Confirmed low-color input tries PNG, with palette encoding only when no alpha channel is present, followed by WebP qualities 85, 80, and 75. Other alpha input tries WebP at those qualities; other opaque input tries JPEG. Candidates execute in order and stop at the first result within the cap. Dimensions shrink only after every candidate at one size exceeds the cap. The source extension does not classify a PNG as low color. A clean, single-frame 8-bit sRGB/sRGBA PNG, JPEG, or WebP within both normalization limits passes through byte-identically and retains content-addressed deduplication. GIF, animation, metadata, orientation, 16-bit PNG, and incompatible color spaces force conversion. The source and a converted output are each fully decoded once; the output must match its format, dimensions, depth, color space, and alpha facts before its digest enters the reference.
Batch admission prepares and verifies every normalized attachment once before publishing any member. Validation failure starts no writes. Publication uses those prepared bytes directly, so a large batch does not repeat full decoding and encoding during commit. A later storage failure returns no partial references; already published immutable objects may remain unreachable under the existing storage rule.
### Deterministic request versions
`AttachmentStore.readImageRequest` derives a request version under route-owned total-pixel and encoded-byte budgets. Scaling is `min(1, sqrt(maxPixels / (width * height)))`, with no enlargement, followed by inward integer rounding so the encoded raster never exceeds the total-pixel cap. DeepSeek V4 Flash Vision Exp uses 640,000 total pixels and 1MiB raw encoded bytes by default; low detail uses 512 by 512 total pixels. A 2048 by 1024 normalized attachment projects to 1130 by 565 under the hard cap. Request encoding uses the same color branches, with PNG (palette only without alpha) then WebP 85 and 80 for low-color input, WebP 85 then 80 for other alpha input, and JPEG 85 then 80 for other opaque input. Each fallback runs only after the previous result exceeds 1MiB, and dimensions shrink only after both quality attempts exceed it. The same derivation is used by normal agent turns, direct `ctx.llm.stream` calls, compaction, and other auxiliary streams.
The `variantId` and cache path cover the normalized attachment id, transform version, route pixel and byte budgets, and fixed encoder parameters. A new cache entry is fully decoded before publication. Cache hits use a header probe to check format, 8-bit sRGB/sRGBA facts, dimensions, alpha, and byte limits without decoding the complete raster again; a mismatch regenerates the entry. DeepSeek Files and pi-ai inline base64 therefore use the same deterministic bytes for the same policy. Inline accounting uses the derived byte length after base64 expansion, not the normalized attachment byte count. Equal in-process `variantId` calls share one transform and cache write. Each caller can cancel its own wait; the shared transform is aborted only after every waiter has cancelled. Callers preserve order by applying `Promise.all` to singular `readImageRequest` calls. The local implementation runs normalization and request transforms through one FIFO limiter; `imageCompressionConcurrency` is configurable from 1 through 8 and defaults to 2. Batch publication remains sequential after every normalized attachment has been prepared.
Request-size offload is a deterministic oldest-first projection. Before reading attachments, each route uses `min(attachmentBytes, requestVersionMaxBytes)` as a conservative upper bound and removes the oldest over-budget prefix. Only retained attachments are read and transformed, so an omitted missing or corrupt object cannot block the request. A second projection uses exact derived lengths without bringing omitted images back. DeepSeek defaults to 128MiB and 600 referenced images. Its removed prefix advances past successive 64MiB byte boundaries and in 20-image count quanta, so 129 one-megabyte images remove the oldest 65, retain 64MiB, and keep that prefix stable until total history passes 192MiB. Pi-ai retains a configurable base64 request bound. A text-only route receives deterministic attachment placeholders, including nested tool-result images, while append-only session history keeps the original references.
### Stable handles
Every retained request image is preceded by its complete attachment id and actual request dimensions. User messages, tool results, agent-loop requests, compaction, and direct `ctx.llm.stream` calls share this projection.
### DeepSeek Files lifecycle
The direct `deepseek-official` adapter normally uploads every retained request version through the OpenAI-compatible Files API and sends `file_id` content blocks. A [bounded inline fallback](../bug-fix/2026-08-21-deepseek-files-inline-fallback.md) sends the same deterministic request versions when file resolution fails. The default catalog advertises `deepseek-v4-flash-vision-exp` as image-capable. Uploaded ids are indexed by endpoint and API-key scope plus `variantId`. Uploads request seven days by default and record the returned `expires_at`; a mapping with no more than one hour remaining is replaced without a preceding retrieve call. The index never stores the API key.
An upload is indexed only after the response returns a complete file object, matching byte count, and `expires_at`. A missing or inconsistent response leaves no local mapping, so a later request uploads again. Concurrent upload resolution for one scoped `variantId` shares one provider operation; one waiter cannot cancel another, and the upload stops when every waiter has cancelled. A malformed upload index is an empty cache and is replaced on the next successful upload; filesystem I/O failures remain errors. If chat reports expired, deleted, missing, or invalid ids and names one or more ids used by the request, only those mappings are removed. A stale-file response without a specific id removes every mapping used by that chat attempt. The affected request bytes are uploaded again and chat is retried once. A second stale rejection clears the mappings identified by its response and returns the error without a third chat attempt. One upload quota error first lists the configured number of oldest harness-owned `dsh-` files, then deletes that collected set and retries once; deleting after pagination keeps provider cursors valid. Public file operations expose list, retrieve, delete, one-variant release, and namespace-wide release. Every Files request carries the shared Harness `User-Agent`. The client enforces the documented 128MiB upload limit, 32MiB chat-image limit, 10,000-file and 25GiB quotas, and one-hour to 30-day expiry range.
### Diagnostics
A 16-bit RGB or RGBA PNG is normal admitted input and converts to 8-bit sRGB/sRGBA. If local conversion fails, `read_image` names the path, detected 16-bit PNG, required normalized form, and manual conversion remedy. If DeepSeek rejects a normalized request version, the primary error names the attachment or display name, durable message and image position, normalized media type, 8-bit sRGB/sRGBA depth, dimensions, and provider message. An ambiguous multi-image rejection lists every candidate. The raw provider body remains the error cause rather than the only visible message.
Historical attachment objects that later disappear or fail integrity verification remain fail-loud. Durable quarantine and verified recovery require session events and are tracked by [Quarantine unreadable historical attachments](../../proposed/bug-fix/2026-08-20-attachment-read-quarantine.md).
## Alternatives considered
**Use one 1MiB normalized attachment for storage and requests.** This makes model resolution determine durable image detail and combines local storage, inline expansion, Files quota, and model pixels into one setting. Independent normalization and request policies keep those responsibilities explicit.
**Reject images above provider dimensions or at the encoding quality floor.** A provider limit is route-specific and future requests may use another model. Proportional normalization and request projection accept ordinary large images while bounding each later representation.
**Treat PNG as a screenshot and reject 16-bit PNG.** File format does not reveal pixel complexity, and 16-bit RGB/RGBA is a convertible sample depth rather than an unsupported image type. Pixel sampling and post-conversion probes give the required facts.
**Keep DeepSeek data URLs as the primary transport.** Inline base64 repeats bytes on every request and caps usable image history by request-body size. Files API references reuse uploaded deterministic request bytes and provide explicit expiry and deletion; the bounded fallback uses data URLs only when file resolution fails.
**Trust a locally indexed file id indefinitely.** Remote expiry, deletion, and lost upload responses make local and provider state diverge. Response-directed invalidation and one re-upload recover without an unbounded retry loop; an ambiguous stale-file response must invalidate every file used by that attempt because it provides no safe exact target.
**Refuse text-only model selection after any image.** Durable history can outlive the model that first consumed it. Request-local placeholders keep the session usable without rewriting history.
**Remove one image whenever a request crosses its limit.** That changes an early request message after nearly every new upload. Quantized removed prefixes keep cache invalidation occasional while honoring the configured high bound.
## Verification
Package tests generate 16-bit RGB and RGBA PNG fixtures, prove 8-bit conversion and clean 8-bit passthrough, retain alpha under byte pressure, distinguish high-frequency and ordinary photos from low-color graphics, stop lazy encoding after the first fitting candidate, cover square and wide 640,000-pixel projections, enforce 1MiB request bytes, singleflight equal variants and uploads without shared-cancellation leaks, bound transform concurrency, preserve cache and upload identity, skip attachment reads for conservatively offloaded history, prepare batches once, reject inconsistent Files responses, refresh near-expiry ids without retrieve, recover once from single-id, multiple-id, and ambiguous stale responses, fall back to bounded all-inline requests after file resolution failure, paginate before quota deletion, normalize provider diagnostics, project text-only history, and share normal/compaction request bytes. Keyless assembled snapshots cover the real tool schemas and image request path. A credentialed test uses the built-in `deepseek-official` route and its configured endpoint, never a custom provider entry.
## Consequences
Normalized attachments consume up to the independent local safety cap, while request caches and remote Files consume additional derived storage. Deterministic identities and singleflight make that work reusable across turns and sessions sharing the same DSH home. Two simultaneous transforms reduce batch latency while increasing peak RSS relative to serial execution; deployments with tighter memory can set the limit to one. Encoder or transform-version changes create new future identities without rewriting existing history. DeepSeek image requests prefer Files reuse; bounded stale-id recovery handles inconsistent remote state, while file-resolution failures use the smaller inline budget. Missing or corrupt durable attachments still require the separate quarantine design.
@@ -0,0 +1,69 @@
# Agent Note: 统一规范化附件、请求版本与提供方文件
Status: implemented
[English](2026-08-20-unified-image-request-pipeline.md) | 中文
## Problem
持久图片历史、提供方分辨率、内联请求大小和远端文件复用有不同限制。过去把已接纳图片直接作为之后每次请求发送的字节,导致一个字节上限和一份光栅同时承担四种职责。普通大图会被拒绝;干净的 16-bit PNG 可以进入历史,之后才被 DeepSeek 拒绝;重复 base64 使长会话请求持续增长;提供方拒绝后,同一持久图片还会进入每次后续请求。
## Decision
图片路径有两个显式版本。附件后端拥有提供方无关的持久规范化附件。每条支持图片的模型路由拥有确定性请求策略,附件后端从该附件派生并缓存确切请求版本。会话历史只包含规范化附件引用;内联字节和提供方文件 ID 都是瞬时请求投影。
### 提供方无关的规范化附件
每条消息最多准入 20 张图片,源图编码字节总量不超过 200MiB。每张源图会在可配置的 20MiB、64,000,000 像素和单边 8192px 限制内完整解码。规范化过程会应用 EXIF 方向,删除元数据和色彩配置文件,转换为 8-bit sRGB/sRGBA,并保持宽高比把长边限制到 `normalizedImageMaxDimension`,默认 2048px。缩放减小光栅时,`originalDimensions` 记录规范化之前、应用方向之后的输入宽高。
规范化附件有独立的 `normalizedImageMaxBytes` 安全上限,默认 4MiB。透明通道绝不铺平。系统通过 nearest-neighbour 对有界样本判断色彩复杂度,不会通过像素平均把高频图片误判为低色数。确认的低色数输入先尝试 PNG,只有不带 alpha 通道时才使用 palette,随后依次尝试质量 85、80、75 的 WebP;其他透明输入依次尝试这些质量的 WebP;其他非透明输入依次尝试这些质量的 JPEG。候选按顺序执行,首个不超过上限的结果会立即返回。同一尺寸的候选全部超限后才会缩小尺寸。源扩展名不会把 PNG 归类为低色数图片。处于两个规范化上限内的干净、单帧、8-bit sRGB/sRGBA PNG、JPEG 或 WebP 按字节原样直通,并保留内容寻址去重。GIF、动图、元数据、方向、16-bit PNG 和不兼容色彩空间都会触发转换。源图和转换输出各完整解码一次;输出的格式、尺寸、位深、色彩空间和透明通道事实通过校验后,其摘要才会进入引用。
批量准入在发布任何成员前,为每张图片各准备并验证一次规范化附件。校验失败不会开始写入。发布直接使用这些已准备字节,因此大批次不会在提交时重复完整解码和编码。之后发生的存储失败不会返回部分引用;按现有存储规则,已经发布的不可变对象可能保持不可达。
### 确定性请求版本
`AttachmentStore.readImageRequest` 按路由拥有的总像素和编码字节预算派生请求版本。缩放公式为 `min(1, sqrt(maxPixels / (width * height)))`,不会放大小图,随后向预算内取整,确保编码光栅不超过总像素上限。DeepSeek V4 Flash Vision Exp 默认使用总像素 640,000 和原始编码字节 1MiBlow detail 使用总像素 512×512。2048×1024 规范化附件在这个硬上限下会投影为 1130×565。请求编码使用相同的分类分支:低色数输入先尝试 PNG,只有不带 alpha 通道时才使用 palette,随后依次尝试质量 85、80 的 WebP;其他透明输入依次尝试质量 85、80 的 WebP;其他非透明输入依次尝试质量 85、80 的 JPEG。只有前一结果超过 1MiB 时才执行下一个候选;两个质量档都超限后才缩小尺寸。普通 agent 轮次、直接 `ctx.llm.stream` 调用、压缩和其他辅助流都使用同一派生过程。
`variantId` 和缓存路径覆盖规范化附件 ID、变换策略版本、路由像素和字节预算及固定编码参数。新缓存条目在发布前会完整解码。缓存命中只探测文件头,校验格式、8-bit sRGB/sRGBA、尺寸、透明通道和字节上限,不会再次完整解码光栅;不匹配时会重新生成。因此,同一策略下的 DeepSeek Files 和 pi-ai 内联 base64 使用相同的确定性字节。内联计量使用派生字节经过 base64 膨胀后的长度,不使用规范化附件字节数。同一进程内相同 `variantId` 的调用共享一次变换和缓存写入。每个调用方可以取消自己的等待;只有全部等待方都取消时,共享变换才会中止。调用方对单数 `readImageRequest` 使用 `Promise.all` 保持结果顺序。本地实现通过一个 FIFO 限流器运行规范化和请求变换,`imageCompressionConcurrency` 的可配置范围为 1 至 8,默认值为 2。全部规范化附件准备完成后,批次仍按顺序发布。
请求大小 offload 是确定性的从旧到新投影。读取附件前,每条路由先以 `min(附件字节数, 请求版本字节上限)` 作为保守上界,移除超出预算的最旧前缀。系统只读取并转换保留的附件,因此已省略的缺失或损坏对象不会阻塞请求。第二次投影使用确切派生长度,但不会重新加入已省略图片。DeepSeek 默认上限为 128MiB 和 600 张引用图片。被移除前缀会越过连续的 64MiB 字节边界,并按 20 张图片数量步长递增,因此 129 张 1MiB 图片会移除最旧的 65 张并保留 64MiB;持久历史超过 192MiB 前,该前缀保持不变。Pi-ai 保留可配置的 base64 请求上限。纯文本路由会收到确定性的附件占位文本,其中包括嵌套工具结果图片;追加式会话历史继续保留原始引用。
### 稳定句柄
每张保留请求图片前都有完整附件 ID 和实际请求尺寸。用户消息、工具结果、agent loop 请求、压缩和直接 `ctx.llm.stream` 调用共享这套投影。
### DeepSeek Files 生命周期
直接 `deepseek-official` 适配器通常通过 OpenAI 兼容 Files API 上传每张保留的请求版本,并发送 `file_id` 内容块。文件解析失败时,[有界内联回退](../bug-fix/2026-08-21-deepseek-files-inline-fallback.zh.md)会发送相同的确定性请求版本。默认 catalog 把 `deepseek-v4-flash-vision-exp` 公布为支持图片。上传 ID 按端点和 API key 作用域以及 `variantId` 写入索引。上传默认请求 7 天有效期,并记录返回的 `expires_at`;本地映射剩余时间不超过一小时时会直接替换,不会先查询远端文件。索引绝不存储 API key。
只有上传响应返回完整文件对象、匹配的字节数和 `expires_at` 时,上传结果才会写入索引。缺失或不一致的响应不会留下本地映射,后续请求会重新上传。同一作用域和 `variantId` 的并发解析共享一次提供方上传;单个等待方无法取消其他等待方,全部等待方取消时才会停止上传。格式损坏的上传索引按空缓存处理,并在下一次成功上传时替换;文件系统 I/O 失败仍是错误。如果 chat 报告 ID 已过期、删除、缺失或无效,并指出本次请求使用的一个或多个 ID,适配器只删除这些映射。如果响应只说明文件状态失效而没有指出具体 ID,适配器会删除该次 chat 使用的全部映射。受影响的请求字节会重新上传,chat 只重试一次。第二次仍报告文件失效时,适配器会按响应清理映射并返回错误,不会发起第三次 chat。一次上传配额错误会先列出配置数量的最旧 `dsh-` 文件,再删除收集到的文件并重试一次;分页完成后才删除,避免游标失效。公开文件操作提供列表、查询、删除、单个变体释放和整个作用域释放。每个 Files 请求都携带 Harness 的共享 `User-Agent`。客户端执行文档规定的 Files 单次上传 128MiB、chat 单图 32MiB、10,000 个文件、25GiB,以及一小时到 30 天有效期限制。
### 诊断
16-bit RGB 或 RGBA PNG 属于普通可接纳输入,会转换为 8-bit sRGB/sRGBA。本地转换失败时,`read_image` 会写明路径、检测到的 16-bit PNG、所需规范形式和手工转换方法。如果 DeepSeek 拒绝已规范化请求版本,主错误会写明附件 ID 或显示名称、持久消息和图片位置、规范化媒体类型、8-bit sRGB/sRGBA 位深、尺寸和提供方消息。多图片错误无法确定对象时会列出全部候选图片。原始提供方正文保留为错误 cause,不会成为唯一可见消息。
持久附件对象之后缺失或无法通过完整性校验时,系统仍会明确失败。持久隔离和经校验恢复需要新增会话事件,由[隔离不可读历史附件](../../proposed/bug-fix/2026-08-20-attachment-read-quarantine.zh.md)继续跟踪。
## Alternatives considered
**使用一份 1MiB 规范化附件同时负责存储和请求。** 这种做法让模型分辨率决定持久图片细节,并把本地存储、内联膨胀、Files 配额和模型像素合并成一个设置。独立的规范化和请求策略会明确区分这些职责。
**拒绝超过提供方尺寸或达到编码质量下限的图片。** 提供方限制属于具体路由,未来请求可能改用另一个模型。按比例规范化和投影请求版本可以接纳普通大图,同时约束每种后续表示。
**把 PNG 当作截图,并拒绝 16-bit PNG。** 文件格式不能说明像素复杂度,16-bit RGB/RGBA 是可转换位深,不是不支持的图片类型。像素采样和转换后探测能提供所需事实。
**把 DeepSeek data URL 作为首选传输方式。** 内联 base64 会在每次请求中重复字节,并按请求正文大小限制可用图片历史。Files API 引用会复用上传后的确定性请求字节,并提供显式有效期和删除操作;有界回退只在文件解析失败时使用 data URL。
**永久信任本地索引中的文件 ID。** 远端过期、删除和上传响应丢失会使本地与提供方状态不一致。按响应失效和一次重新上传可以恢复,同时避免无界重试;响应没有给出可安全使用的精确目标时,必须使该次请求使用的全部文件失效。
**历史中出现图片后拒绝选择纯文本模型。** 持久历史可能比最初读取它的模型存活更久。按请求生成的占位文本可以保持会话可用,无需改写历史。
**请求每次越过上限就移除一张图片。** 这种做法会在几乎每次新增图片后改写较早的请求消息。按固定步长递增的移除前缀会降低缓存失效频率,同时遵守配置的上限。
## Verification
包测试会生成 16-bit RGB 和 RGBA PNG fixture,验证 8-bit 转换与干净 8-bit 字节直通、字节压力下保留透明通道、区分高频和普通照片与低色数图形、首个候选合规后停止编码、正方形和宽屏 640,000 像素投影、请求字节不超过 1MiB、相同变体与上传 singleflight 且不会共享取消、变换并发上限、缓存与上传身份、跳过已保守 offload 的历史附件读取、批量只准备一次、Files 响应不一致、进入刷新余量时不查询远端并更新 ID、单个 ID、多个 ID 和模糊失效响应只恢复一次、文件解析失败后回退到有界全内联请求、删除配额文件前完成分页、规范化提供方诊断、纯文本投影,以及普通请求与压缩共享请求字节。无需密钥的组装快照覆盖真实工具 schema 和图片请求路径。使用凭据的测试只使用内置 `deepseek-official` 路由及其已配置端点,不使用自定义提供方条目。
## Consequences
持久规范化附件最多占用独立的本地安全上限,请求缓存和远端 Files 还会占用额外派生存储。确定性身份和 singleflight 使这些成本可以被共享同一 DSH home 的轮次和会话复用。同时执行两个变换会降低批次延迟,但峰值 RSS 高于串行执行;内存更紧张的部署可以把上限设为 1。编码器或变换策略版本变化会为未来内容产生新身份,不会改写已有历史。DeepSeek 图片请求优先复用 Files;有界的陈旧 ID 恢复会处理远端状态不一致,文件解析失败则使用较小的内联预算。缺失或损坏的持久附件仍需要单独的隔离设计。
@@ -0,0 +1,6 @@
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/proposed/bug-fix/2026-08-20-attachment-read-quarantine.md
2026-08-20-attachment-read-quarantine.md: 28e0f26cee2ec1e257fd4d43b4edc4300e2c6f23
2026-08-20-attachment-read-quarantine.zh.md: 7f4ceae4e1fe9e656ed762de9828e14145976dc3
@@ -0,0 +1,38 @@
# Agent Note: Quarantine unreadable historical attachments
Status: proposed
English | [中文](2026-08-20-attachment-read-quarantine.zh.md)
## Problem
An admitted `ImageAttachmentRef` remains in durable history and therefore participates in every later request until compaction replaces it. `AttachmentStore.readImage()` fails with `ATTACHMENT_NOT_FOUND`, `ATTACHMENT_CORRUPT`, or `ATTACHMENT_READ_FAILED` when the referenced object disappears, fails integrity verification, or cannot be read. The unchanged history then makes every later model request fail on the same object, leaving the session unable to continue even though the remaining messages are usable. This is the unavailable-object case left fail-loud by [reconstructable requests](../../implemented/architecture/2026-07-05-reconstructable-requests.md).
## Proposal
A session-backed image-request projection records unreadable references before provider dispatch. `ATTACHMENT_NOT_FOUND` and `ATTACHMENT_CORRUPT` immediately append `attachment/quarantine`; `ATTACHMENT_READ_FAILED` receives one cancellation-aware read retry and appends the same event with a retryable reason if the retry fails. Cancellation and unclassified failures do not quarantine data.
The quarantine event identifies the attachment and failure class. Projection replaces each quarantined image with deterministic text containing its display name when present, attachment-id prefix, and failure class. Later requests derive the same replacement from the log and skip `readImage()` for that reference, while the original image block remains in append-only history. A request that discovers and records a quarantine reprojects before calling the provider, so the failed read does not become a terminal model-request attempt.
Explicit recovery calls `readImage()` and appends `attachment/recovered` only after digest and metadata verification succeeds. Projection then restores the original image reference. Missing or corrupt bytes are never overwritten automatically, and clearing quarantine without verification is invalid.
The shared request-projection consumer owns this policy. Attachment storage continues to report exact read failures, and provider adapters do not invent independent placeholders or recovery state.
## Alternatives considered
- **Keep failing every request.** This preserves strict error reporting but makes an otherwise usable durable session permanently unavailable after one storage fault.
- **Delete or rewrite the historical image block.** That loses evidence, violates append-only history, and prevents a repaired content-addressed object from restoring the original request.
- **Catch the error independently in each adapter.** An unlogged placeholder would make replay depend on which adapter and storage state happened to be present, while duplicated policies would drift.
- **Replace missing or corrupt bytes automatically.** The reference names verified immutable content; substituting different bytes under that identity would defeat integrity checking.
## Acceptance criteria
- A missing or corrupt historical image produces one durable quarantine transition and a stable placeholder; later model requests do not read that object or fail because of it.
- A general read failure is retried once without ignoring cancellation, then follows the retryable quarantine path.
- Restart and fork reconstruct the same quarantined request from the session log.
- Recovery restores image projection only after the original reference passes complete read verification.
- Package tests cover error classification, idempotent quarantine, cancellation, retry, recovery, and nested tool-result images; a keyless runnable snapshot pins the model-visible placeholder and durable events.
## Risks
Quarantine and recovery each change the provider prefix once. The implementation must identify the exact failing reference before recording state and must coordinate concurrent requests so duplicate failures produce one effective transition. Auxiliary calls without a live session cannot record recovery state; their failure policy remains explicit implementation scope rather than an adapter fallback.
@@ -0,0 +1,38 @@
# Agent Note: 隔离无法读取的历史附件
Status: proposed
[English](2026-08-20-attachment-read-quarantine.md) | 中文
## 问题
已接纳的 `ImageAttachmentRef` 会留在持久历史中,因此在被压缩替换前都会参与之后的每次请求。引用对象丢失、完整性校验失败或无法读取时,`AttachmentStore.readImage()` 会返回 `ATTACHMENT_NOT_FOUND``ATTACHMENT_CORRUPT``ATTACHMENT_READ_FAILED`。未变化的历史随后会让之后每次模型请求在同一对象上失败,使会话无法继续,即使其余消息仍可使用。这是[可重建请求](../../implemented/architecture/2026-07-05-reconstructable-requests.zh.md)保留为明确失败的对象不可用情况。
## 提案
由会话支撑的图片请求投影在分派给提供方之前记录无法读取的引用。`ATTACHMENT_NOT_FOUND``ATTACHMENT_CORRUPT` 立即追加 `attachment/quarantine``ATTACHMENT_READ_FAILED` 先执行一次服从取消信号的读取重试,重试仍失败时追加同一事件并标记为可重试原因。取消和未分类失败不会隔离数据。
隔离事件标识附件和失败类别。投影把每张已隔离图片替换为确定性文本,包含可用时的显示名称、附件 ID 前缀和失败类别。之后的请求从日志派生相同替换结果,并跳过该引用的 `readImage()`,原始图片块仍留在仅追加历史中。请求发现并记录隔离后,会在调用提供方前重新投影,因此读取失败不会成为终止性的模型请求尝试。
显式恢复会调用 `readImage()`,且仅在内容摘要和元数据校验成功后追加 `attachment/recovered`。投影随后恢复原始图片引用。系统绝不会自动覆盖丢失或损坏的字节,也不允许未经验证就清除隔离。
共享请求投影消费方拥有这项策略。附件存储继续报告准确的读取失败,提供方适配器不会各自生成占位或恢复状态。
## 考虑过的替代方案
- **让每次请求继续失败。** 这保留了严格错误报告,但一次存储故障会让其他部分仍可使用的持久会话永久不可用。
- **删除或重写历史图片块。** 这会丢失证据、违反仅追加历史,并使修复后的内容寻址对象无法恢复原始请求。
- **由每个适配器分别捕获错误。** 未记录的占位会让回放取决于当时存在的适配器和存储状态,重复策略也会发生偏差。
- **自动替换丢失或损坏的字节。** 引用标识经过验证的不可变内容;在该身份下替换成其他字节会破坏完整性校验。
## 接受标准
- 缺失或损坏的历史图片产生一次持久隔离转换和稳定占位;之后的模型请求不再读取该对象,也不会因它失败。
- 一般读取失败会在服从取消信号的前提下重试一次,随后进入可重试隔离路径。
- 重启和 fork 后会从会话日志重建相同的隔离请求。
- 仅在原始引用通过完整读取校验后,恢复操作才恢复图片投影。
- 包测试覆盖错误分类、幂等隔离、取消、重试、恢复和嵌套工具结果图片;一个无需密钥的可运行快照钉住模型可见占位和持久事件。
## 风险
隔离和恢复各会改变一次提供方前缀。实现必须在记录状态前识别准确的失败引用,并协调并发请求,使重复失败只产生一次有效转换。没有活跃会话的辅助调用无法记录恢复状态;它们的失败策略属于明确的实现范围,不能退回到适配器自行处理。
+2 -2
View File
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write docs/config-catalog.md
config-catalog.md: da7eae6642bf9516755cf4f68268b40b26ac2ee1
config-catalog.zh.md: bd1903f878aad3071e74d10df8c0782dbd8f3d57
config-catalog.md: de340b7ffade528301b4538b0553bc11ec969985
config-catalog.zh.md: eb17ee89fd7860cc0774073bea542aca315ce652
+45 -11
View File
@@ -327,20 +327,26 @@ Source: [`packages/core/agent-tool-presentation/src/index.ts:38`](../packages/co
export interface Config {
/** Explicit harness home; omitted follows `DSH_HOME`, then `~/.dsh`. */
dshHome?: string
/** Maximum encoded bytes accepted for one image. */
/** Maximum encoded bytes accepted for one submitted image. Default: 20 MiB. */
maxImageBytes?: number
/** Maximum image count accepted in one submitted message. */
/** Maximum image count accepted in one submitted message. Default: 20. */
maxImagesPerMessage?: number
/** Maximum aggregate encoded image bytes accepted in one submitted message. */
/** Maximum aggregate encoded image bytes accepted in one submitted message. Default: 200 MiB. */
maxMessageImageBytes?: number
/** Maximum intrinsic width multiplied by height accepted for one image. */
/** Maximum intrinsic width multiplied by height accepted for one submitted image. Default: 64,000,000. */
maxImagePixels?: number
/** Maximum intrinsic width and maximum intrinsic height accepted for one image. */
/** Maximum intrinsic width and maximum intrinsic height accepted for one submitted image. Default: 8192px. */
maxImageDimension?: number
/** Long-edge pixel cap of the stored provider-independent normalized image. */
normalizedImageMaxDimension?: number
/** Encoded-byte safety cap of the stored provider-independent normalized image. */
normalizedImageMaxBytes?: number
/** Maximum simultaneous normalization or request-image transformations in this service instance. */
imageCompressionConcurrency?: number
}
```
Source: [`packages/attachment/attachment-local/src/index.ts:31`](../packages/attachment/attachment-local/src/index.ts)
Source: [`packages/attachment/attachment-local/src/index.ts:51`](../packages/attachment/attachment-local/src/index.ts)
<a id="deepseek-aidsh-bash-local"></a>
@@ -407,7 +413,7 @@ export interface ConnectionConfig {
* that is not a bare, canonical authority fails the plugin load.
*/
trustedHosts?: string[]
/** Maximum buffered JSON body for every `/api` request. */
/** Maximum buffered JSON body for every `/api` request. Default: 300 MiB. */
maxRequestBodyBytes?: number
}
```
@@ -932,8 +938,26 @@ export interface Config {
models?: DeepSeekCatalogModel[]
/** Maximum provider idle time while one stream read is outstanding (default five minutes). */
streamIdleTimeoutMs?: number
/** Maximum accumulated base64 image payload per request (default 20 MiB). */
maxRequestImageBytes?: number
/** Maximum accumulated file-referenced image bytes per chat request (default 128 MiB). */
maxRequestFilesBytes?: number
/** Maximum accumulated base64 image payload after Files API fallback (default 20 MiB). */
maxInlineRequestImageBytes?: number
/** Maximum number of represented images per chat request (default 600). */
maxImagesPerRequest?: number
/** Raw-byte removal step after the request exceeds its file bound (default 64 MiB). */
imageOffloadByteQuantum?: number
/** Base64-byte removal step after inline fallback exceeds its bound (default 10 MiB). */
inlineImageOffloadByteQuantum?: number
/** Image-count removal step after the request exceeds its count bound (default 20). */
imageOffloadCountQuantum?: number
/** Maximum duration of one request-image Files API resolution (default one minute). */
filesApiTimeoutMs?: number
/** Explicit lifetime assigned to each uploaded image (default seven days). */
fileExpiresAfterSeconds?: number
/** Remaining lifetime below which an indexed file is replaced (default one hour). */
fileRefreshMarginSeconds?: number
/** Oldest harness-owned files deleted before one quota-recovery upload retry (default 100). */
fileQuotaCleanupBatch?: number
/** Provider-owned model-request retry policy; omission uses normal mode with five retries. */
retryPolicy?: RetryPolicyConfig
}
@@ -952,12 +976,18 @@ export interface DeepSeekCatalogModel {
maxTokens?: number
/** Accepted request modalities; omission is text-only. */
inputModalities?: ModelModality[]
/** Total-pixel budget for one deterministic request preview. */
imagePixelBudget?: number
/** Encoded-byte cap for one deterministic request preview. */
imageMaxBytes?: number
/** Provider detail tier; `low` uses the 512-by-512 total-pixel default. */
imageDetail?: 'auto' | 'low'
}
```
Depends on: [`ModelModality`](../packages/llm/llm/src/index.ts) · [`RetryPolicyConfig`](../packages/llm/llm/src/index.ts)
Source: [`packages/llm/llm-deepseek/src/index.ts:72`](../packages/llm/llm-deepseek/src/index.ts)
Source: [`packages/llm/llm-deepseek/src/index.ts:106`](../packages/llm/llm-deepseek/src/index.ts)
<a id="deepseek-aidsh-llm-pi-ai"></a>
@@ -1059,6 +1089,10 @@ export interface PiAiProviderProfile {
* requests instead of being rejected by a request-size cap.
*/
maxRequestImageBytes?: number
/** Total-pixel budget for each deterministic inline request version. */
requestImagePixelBudget?: number
/** Raw encoded-byte cap for each deterministic inline request version. */
requestImageMaxBytes?: number
/** Provider-owned model-request retry policy; omission uses normal mode with five retries. */
retryPolicy?: RetryPolicyConfig
}
@@ -1207,7 +1241,7 @@ export type PiAiThinkingFormat = NonNullable<OpenAICompletionsCompat['thinkingFo
Depends on: `Api` (`@earendil-works/pi-ai`) · `CacheRetention` (`@earendil-works/pi-ai`) · `Model` (`@earendil-works/pi-ai`) · `ModelThinkingLevel` (`@earendil-works/pi-ai`) · `OpenAICompletionsCompat` (`@earendil-works/pi-ai`) · [`RetryPolicyConfig`](../packages/llm/llm/src/index.ts) · `ThinkingBudgets` (`@earendil-works/pi-ai`) · `Transport` (`@earendil-works/pi-ai`)
Source: [`packages/llm/llm-pi-ai/src/config.ts:201`](../packages/llm/llm-pi-ai/src/config.ts)
Source: [`packages/llm/llm-pi-ai/src/config.ts:213`](../packages/llm/llm-pi-ai/src/config.ts)
<a id="deepseek-aidsh-llm-replay"></a>
+45 -11
View File
@@ -329,20 +329,26 @@ export interface Config {
export interface Config {
/** Explicit harness home; omitted follows `DSH_HOME`, then `~/.dsh`. */
dshHome?: string
/** Maximum encoded bytes accepted for one image. */
/** Maximum encoded bytes accepted for one submitted image. Default: 20 MiB. */
maxImageBytes?: number
/** Maximum image count accepted in one submitted message. */
/** Maximum image count accepted in one submitted message. Default: 20. */
maxImagesPerMessage?: number
/** Maximum aggregate encoded image bytes accepted in one submitted message. */
/** Maximum aggregate encoded image bytes accepted in one submitted message. Default: 200 MiB. */
maxMessageImageBytes?: number
/** Maximum intrinsic width multiplied by height accepted for one image. */
/** Maximum intrinsic width multiplied by height accepted for one submitted image. Default: 64,000,000. */
maxImagePixels?: number
/** Maximum intrinsic width and maximum intrinsic height accepted for one image. */
/** Maximum intrinsic width and maximum intrinsic height accepted for one submitted image. Default: 8192px. */
maxImageDimension?: number
/** Long-edge pixel cap of the stored provider-independent normalized image. */
normalizedImageMaxDimension?: number
/** Encoded-byte safety cap of the stored provider-independent normalized image. */
normalizedImageMaxBytes?: number
/** Maximum simultaneous normalization or request-image transformations in this service instance. */
imageCompressionConcurrency?: number
}
```
来源:[`packages/attachment/attachment-local/src/index.ts:31`](../packages/attachment/attachment-local/src/index.ts)
来源:[`packages/attachment/attachment-local/src/index.ts:51`](../packages/attachment/attachment-local/src/index.ts)
<a id="deepseek-aidsh-bash-local"></a>
@@ -409,7 +415,7 @@ export interface ConnectionConfig {
* that is not a bare, canonical authority fails the plugin load.
*/
trustedHosts?: string[]
/** Maximum buffered JSON body for every `/api` request. */
/** Maximum buffered JSON body for every `/api` request. Default: 300 MiB. */
maxRequestBodyBytes?: number
}
```
@@ -934,8 +940,26 @@ export interface Config {
models?: DeepSeekCatalogModel[]
/** Maximum provider idle time while one stream read is outstanding (default five minutes). */
streamIdleTimeoutMs?: number
/** Maximum accumulated base64 image payload per request (default 20 MiB). */
maxRequestImageBytes?: number
/** Maximum accumulated file-referenced image bytes per chat request (default 128 MiB). */
maxRequestFilesBytes?: number
/** Maximum accumulated base64 image payload after Files API fallback (default 20 MiB). */
maxInlineRequestImageBytes?: number
/** Maximum number of represented images per chat request (default 600). */
maxImagesPerRequest?: number
/** Raw-byte removal step after the request exceeds its file bound (default 64 MiB). */
imageOffloadByteQuantum?: number
/** Base64-byte removal step after inline fallback exceeds its bound (default 10 MiB). */
inlineImageOffloadByteQuantum?: number
/** Image-count removal step after the request exceeds its count bound (default 20). */
imageOffloadCountQuantum?: number
/** Maximum duration of one request-image Files API resolution (default one minute). */
filesApiTimeoutMs?: number
/** Explicit lifetime assigned to each uploaded image (default seven days). */
fileExpiresAfterSeconds?: number
/** Remaining lifetime below which an indexed file is replaced (default one hour). */
fileRefreshMarginSeconds?: number
/** Oldest harness-owned files deleted before one quota-recovery upload retry (default 100). */
fileQuotaCleanupBatch?: number
/** Provider-owned model-request retry policy; omission uses normal mode with five retries. */
retryPolicy?: RetryPolicyConfig
}
@@ -954,12 +978,18 @@ export interface DeepSeekCatalogModel {
maxTokens?: number
/** Accepted request modalities; omission is text-only. */
inputModalities?: ModelModality[]
/** Total-pixel budget for one deterministic request preview. */
imagePixelBudget?: number
/** Encoded-byte cap for one deterministic request preview. */
imageMaxBytes?: number
/** Provider detail tier; `low` uses the 512-by-512 total-pixel default. */
imageDetail?: 'auto' | 'low'
}
```
依赖:[`ModelModality`](../packages/llm/llm/src/index.ts) · [`RetryPolicyConfig`](../packages/llm/llm/src/index.ts)
来源:[`packages/llm/llm-deepseek/src/index.ts:72`](../packages/llm/llm-deepseek/src/index.ts)
来源:[`packages/llm/llm-deepseek/src/index.ts:106`](../packages/llm/llm-deepseek/src/index.ts)
<a id="deepseek-aidsh-llm-pi-ai"></a>
@@ -1061,6 +1091,10 @@ export interface PiAiProviderProfile {
* requests instead of being rejected by a request-size cap.
*/
maxRequestImageBytes?: number
/** Total-pixel budget for each deterministic inline request version. */
requestImagePixelBudget?: number
/** Raw encoded-byte cap for each deterministic inline request version. */
requestImageMaxBytes?: number
/** Provider-owned model-request retry policy; omission uses normal mode with five retries. */
retryPolicy?: RetryPolicyConfig
}
@@ -1209,7 +1243,7 @@ export type PiAiThinkingFormat = NonNullable<OpenAICompletionsCompat['thinkingFo
依赖:`Api``@earendil-works/pi-ai`)· `CacheRetention``@earendil-works/pi-ai`)· `Model``@earendil-works/pi-ai`)· `ModelThinkingLevel``@earendil-works/pi-ai`)· `OpenAICompletionsCompat``@earendil-works/pi-ai`)· [`RetryPolicyConfig`](../packages/llm/llm/src/index.ts) · `ThinkingBudgets``@earendil-works/pi-ai`)· `Transport``@earendil-works/pi-ai`)
来源:[`packages/llm/llm-pi-ai/src/config.ts:201`](../packages/llm/llm-pi-ai/src/config.ts)
来源:[`packages/llm/llm-pi-ai/src/config.ts:213`](../packages/llm/llm-pi-ai/src/config.ts)
<a id="deepseek-aidsh-llm-replay"></a>
+2 -2
View File
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write docs/event-producer-consumer.md
event-producer-consumer.md: d68f92d317dec2fd05813c1ce487bb88c369bd6e
event-producer-consumer.zh.md: 5b3454e6000f4e1e017cb494423736f2c0f75f31
event-producer-consumer.md: 1fb65d55f5a0d8121f4f171c956196fde746103f
event-producer-consumer.zh.md: d8db21e5266f83a5fc403a9b825bc05530e61b8d
+1 -1
View File
@@ -38,7 +38,7 @@ This matrix shows which packages dispatch each harness-owned event and which pac
| `fs/write-intent` | `waterfall` | [`packages/fs/fs/src/index.ts:58`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`waterfall`), [`tool-str-replace-editor`](../packages/fs/tool-str-replace-editor) (`waterfall`) | [`fs-observation-policy`](../packages/fs/fs-observation-policy) |
| `goal/changed` | `emit` | [`packages/goal/goal/src/domain.ts:114`](../packages/goal/goal/src/domain.ts) | [`goal`](../packages/goal/goal) (`emit`) | [`goal-round-driver`](../packages/goal/goal-round-driver) |
| `llm/adapters-updated` | `emit` | [`packages/llm/llm/src/types.ts:23`](../packages/llm/llm/src/types.ts) | [`llm`](../packages/llm/llm) (`events.dispatch`) | `apiproxy`, [`llm`](../packages/llm/llm) |
| `llm/stream` | `waterfall` | [`packages/llm/llm/src/index.ts:64`](../packages/llm/llm/src/index.ts) | [`llm`](../packages/llm/llm) (`waterfall`) | [`agent-loop`](../packages/core/agent-loop), [`llm`](../packages/llm/llm), [`llm-replay`](../packages/test-support/llm-replay), [`session-checkpoint-policy`](../packages/session/session-checkpoint-policy), [`session-title`](../packages/session/session-title) |
| `llm/stream` | `waterfall` | [`packages/llm/llm/src/index.ts:65`](../packages/llm/llm/src/index.ts) | [`llm`](../packages/llm/llm) (`waterfall`) | [`agent-loop`](../packages/core/agent-loop), [`llm`](../packages/llm/llm), [`llm-replay`](../packages/test-support/llm-replay), [`session-checkpoint-policy`](../packages/session/session-checkpoint-policy), [`session-title`](../packages/session/session-title) |
| `session-telemetry/record` | `waterfall` | [`packages/session/session-telemetry/src/index.ts:43`](../packages/session/session-telemetry/src/index.ts) | [`session-telemetry`](../packages/session/session-telemetry) (`waterfall`) | - |
| `session/created` | `emit` | [`packages/core/session/src/index.ts:54`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`events.dispatch`) | `apiproxy`, [`compaction`](../packages/compaction/compaction), [`goal`](../packages/goal/goal), [`hook-protocol`](../packages/hooks/hook-protocol), [`llm-retry`](../packages/llm/llm-retry), [`permission-presets`](../packages/interaction/permission-presets), [`plan-mode`](../packages/plan/plan-mode), [`schedule`](../packages/schedule/schedule), `server`, [`session`](../packages/core/session), [`session-persistence`](../packages/session/session-persistence), [`session-telemetry`](../packages/session/session-telemetry), [`time-context`](../packages/context/time-context), [`tool-workflow`](../packages/workflow/tool-workflow), [`tools`](../packages/core/tools), [`user-approval`](../packages/interaction/user-approval) |
| `session/disposed` | `emit` | [`packages/core/session/src/index.ts:64`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`events.dispatch`) | [`agent-loop`](../packages/core/agent-loop), `agent-team`, `apiproxy`, [`session-persistence`](../packages/session/session-persistence), [`session-projection-cache`](../packages/session/session-projection-cache), [`session-telemetry`](../packages/session/session-telemetry), [`session-title`](../packages/session/session-title) |
+1 -1
View File
@@ -40,7 +40,7 @@
| `fs/write-intent` | `waterfall` | [`packages/fs/fs/src/index.ts:58`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`waterfall`), [`tool-str-replace-editor`](../packages/fs/tool-str-replace-editor) (`waterfall`) | [`fs-observation-policy`](../packages/fs/fs-observation-policy) |
| `goal/changed` | `emit` | [`packages/goal/goal/src/domain.ts:114`](../packages/goal/goal/src/domain.ts) | [`goal`](../packages/goal/goal) (`emit`) | [`goal-round-driver`](../packages/goal/goal-round-driver) |
| `llm/adapters-updated` | `emit` | [`packages/llm/llm/src/types.ts:23`](../packages/llm/llm/src/types.ts) | [`llm`](../packages/llm/llm) (`events.dispatch`) | `apiproxy`, [`llm`](../packages/llm/llm) |
| `llm/stream` | `waterfall` | [`packages/llm/llm/src/index.ts:64`](../packages/llm/llm/src/index.ts) | [`llm`](../packages/llm/llm) (`waterfall`) | [`agent-loop`](../packages/core/agent-loop), [`llm`](../packages/llm/llm), [`llm-replay`](../packages/test-support/llm-replay), [`session-checkpoint-policy`](../packages/session/session-checkpoint-policy), [`session-title`](../packages/session/session-title) |
| `llm/stream` | `waterfall` | [`packages/llm/llm/src/index.ts:65`](../packages/llm/llm/src/index.ts) | [`llm`](../packages/llm/llm) (`waterfall`) | [`agent-loop`](../packages/core/agent-loop), [`llm`](../packages/llm/llm), [`llm-replay`](../packages/test-support/llm-replay), [`session-checkpoint-policy`](../packages/session/session-checkpoint-policy), [`session-title`](../packages/session/session-title) |
| `session-telemetry/record` | `waterfall` | [`packages/session/session-telemetry/src/index.ts:43`](../packages/session/session-telemetry/src/index.ts) | [`session-telemetry`](../packages/session/session-telemetry) (`waterfall`) | - |
| `session/created` | `emit` | [`packages/core/session/src/index.ts:54`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`events.dispatch`) | `apiproxy`, [`compaction`](../packages/compaction/compaction), [`goal`](../packages/goal/goal), [`hook-protocol`](../packages/hooks/hook-protocol), [`llm-retry`](../packages/llm/llm-retry), [`permission-presets`](../packages/interaction/permission-presets), [`plan-mode`](../packages/plan/plan-mode), [`schedule`](../packages/schedule/schedule), `server`, [`session`](../packages/core/session), [`session-persistence`](../packages/session/session-persistence), [`session-telemetry`](../packages/session/session-telemetry), [`time-context`](../packages/context/time-context), [`tool-workflow`](../packages/workflow/tool-workflow), [`tools`](../packages/core/tools), [`user-approval`](../packages/interaction/user-approval) |
| `session/disposed` | `emit` | [`packages/core/session/src/index.ts:64`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`events.dispatch`) | [`agent-loop`](../packages/core/agent-loop), `agent-team`, `apiproxy`, [`session-persistence`](../packages/session/session-persistence), [`session-projection-cache`](../packages/session/session-projection-cache), [`session-telemetry`](../packages/session/session-telemetry), [`session-title`](../packages/session/session-title) |
+2 -2
View File
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write docs/module-graph.md
module-graph.md: b3d061b0b07ec830f88c9fa56fd6921b343d35cc
module-graph.zh.md: b4a748dbe0084afdd554e8295e5ba504e0cf5cea
module-graph.md: 531def00118795d3d8c6812baa215e71ca499bc0
module-graph.zh.md: 6d6794da2f1545c7d420f4c4f161c2711143b16c
+4 -1
View File
@@ -413,8 +413,11 @@ flowchart TD
pkg_settings_file --> pkg_invariants
pkg_settings_file --> pkg_settings
pkg_llm_deepseek --> pkg_anonymous_user_id
pkg_llm_deepseek --> pkg_atomic_write
pkg_llm_deepseek --> pkg_attachment
pkg_llm_deepseek --> pkg_brand
pkg_llm_deepseek --> pkg_credentials
pkg_llm_deepseek --> pkg_home_paths
pkg_llm_deepseek --> pkg_invariants
pkg_llm_deepseek --> pkg_launch_environment
pkg_llm_deepseek --> pkg_llm
@@ -1506,7 +1509,7 @@ flowchart TD
| [`client-hmr`](../packages/client/hmr) | `client` | [`client-modules`](../packages/client/modules), [`host-webserver`](../packages/host/webserver), [`invariants`](../packages/runtime-diagnostics/invariants) |
| [`credentials-local`](../packages/credentials/credentials-local) | `credentials` | [`atomic-write`](../packages/util/atomic-write), [`credentials`](../packages/credentials/credentials), [`home-paths`](../packages/util/home-paths), [`invariants`](../packages/runtime-diagnostics/invariants), [`launch-environment`](../packages/util/launch-environment) |
| [`settings-file`](../packages/settings/settings-file) | `settings` | [`atomic-write`](../packages/util/atomic-write), [`home-paths`](../packages/util/home-paths), [`invariants`](../packages/runtime-diagnostics/invariants), [`settings`](../packages/settings/settings) |
| [`llm-deepseek`](../packages/llm/llm-deepseek) | `llm` | [`anonymous-user-id`](../packages/identity/anonymous-user-id), [`attachment`](../packages/attachment/attachment), [`credentials`](../packages/credentials/credentials), [`invariants`](../packages/runtime-diagnostics/invariants), [`launch-environment`](../packages/util/launch-environment), [`llm`](../packages/llm/llm), [`settings`](../packages/settings/settings), [`timeout`](../packages/util/timeout) |
| [`llm-deepseek`](../packages/llm/llm-deepseek) | `llm` | [`anonymous-user-id`](../packages/identity/anonymous-user-id), [`atomic-write`](../packages/util/atomic-write), [`attachment`](../packages/attachment/attachment), [`brand`](../packages/util/brand), [`credentials`](../packages/credentials/credentials), [`home-paths`](../packages/util/home-paths), [`invariants`](../packages/runtime-diagnostics/invariants), [`launch-environment`](../packages/util/launch-environment), [`llm`](../packages/llm/llm), [`settings`](../packages/settings/settings), [`timeout`](../packages/util/timeout) |
| [`session`](../packages/core/session) | `core` | [`brand`](../packages/util/brand), [`invariants`](../packages/runtime-diagnostics/invariants), [`llm`](../packages/llm/llm), [`scope`](../packages/core/scope), [`typert-protocol`](../packages/typert/protocol) |
| [`system-prompt`](../packages/core/system-prompt) | `core` | [`invariants`](../packages/runtime-diagnostics/invariants), [`llm`](../packages/llm/llm), [`scope`](../packages/core/scope) |
| [`skill`](../packages/skill/skill) | `skill` | [`invariants`](../packages/runtime-diagnostics/invariants), [`llm`](../packages/llm/llm), [`scope`](../packages/core/scope) |
+4 -1
View File
@@ -415,8 +415,11 @@ flowchart TD
pkg_settings_file --> pkg_invariants
pkg_settings_file --> pkg_settings
pkg_llm_deepseek --> pkg_anonymous_user_id
pkg_llm_deepseek --> pkg_atomic_write
pkg_llm_deepseek --> pkg_attachment
pkg_llm_deepseek --> pkg_brand
pkg_llm_deepseek --> pkg_credentials
pkg_llm_deepseek --> pkg_home_paths
pkg_llm_deepseek --> pkg_invariants
pkg_llm_deepseek --> pkg_launch_environment
pkg_llm_deepseek --> pkg_llm
@@ -1508,7 +1511,7 @@ flowchart TD
| [`client-hmr`](../packages/client/hmr) | `client` | [`client-modules`](../packages/client/modules), [`host-webserver`](../packages/host/webserver), [`invariants`](../packages/runtime-diagnostics/invariants) |
| [`credentials-local`](../packages/credentials/credentials-local) | `credentials` | [`atomic-write`](../packages/util/atomic-write), [`credentials`](../packages/credentials/credentials), [`home-paths`](../packages/util/home-paths), [`invariants`](../packages/runtime-diagnostics/invariants), [`launch-environment`](../packages/util/launch-environment) |
| [`settings-file`](../packages/settings/settings-file) | `settings` | [`atomic-write`](../packages/util/atomic-write), [`home-paths`](../packages/util/home-paths), [`invariants`](../packages/runtime-diagnostics/invariants), [`settings`](../packages/settings/settings) |
| [`llm-deepseek`](../packages/llm/llm-deepseek) | `llm` | [`anonymous-user-id`](../packages/identity/anonymous-user-id), [`attachment`](../packages/attachment/attachment), [`credentials`](../packages/credentials/credentials), [`invariants`](../packages/runtime-diagnostics/invariants), [`launch-environment`](../packages/util/launch-environment), [`llm`](../packages/llm/llm), [`settings`](../packages/settings/settings), [`timeout`](../packages/util/timeout) |
| [`llm-deepseek`](../packages/llm/llm-deepseek) | `llm` | [`anonymous-user-id`](../packages/identity/anonymous-user-id), [`atomic-write`](../packages/util/atomic-write), [`attachment`](../packages/attachment/attachment), [`brand`](../packages/util/brand), [`credentials`](../packages/credentials/credentials), [`home-paths`](../packages/util/home-paths), [`invariants`](../packages/runtime-diagnostics/invariants), [`launch-environment`](../packages/util/launch-environment), [`llm`](../packages/llm/llm), [`settings`](../packages/settings/settings), [`timeout`](../packages/util/timeout) |
| [`session`](../packages/core/session) | `core` | [`brand`](../packages/util/brand), [`invariants`](../packages/runtime-diagnostics/invariants), [`llm`](../packages/llm/llm), [`scope`](../packages/core/scope), [`typert-protocol`](../packages/typert/protocol) |
| [`system-prompt`](../packages/core/system-prompt) | `core` | [`invariants`](../packages/runtime-diagnostics/invariants), [`llm`](../packages/llm/llm), [`scope`](../packages/core/scope) |
| [`skill`](../packages/skill/skill) | `skill` | [`invariants`](../packages/runtime-diagnostics/invariants), [`llm`](../packages/llm/llm), [`scope`](../packages/core/scope) |
+2 -2
View File
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write docs/subsystems/attachment.md
attachment.md: 180b7e06f0461dd4136779917e0732921704803b
attachment.zh.md: 35aa24ec5957b41e12a543fd76ea20604894cd18
attachment.md: e6d0a53db2827a38a1535380319b6220aa37f0a4
attachment.zh.md: 8328ec610d4d68624f75f00d6a397b13fdf31c4e
+61 -10
View File
@@ -18,7 +18,7 @@ type ImageMediaType = 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif'
```
```ts type-equiv
/** Durable, serializable metadata for one immutable image object. */
/** Durable, serializable reference to one immutable normalized image. */
interface ImageAttachmentRef {
/** Opaque storage identifier; never a filesystem path or bearer URL. */
attachmentId: AttachmentId
@@ -32,6 +32,14 @@ interface ImageAttachmentRef {
height: number
/** Optional display name stripped of local path information. */
name?: string
/**
* Input dimensions after applying EXIF orientation and before normalization
* scaling. Present only when normalization reduced the image.
*/
originalDimensions?: {
width: number
height: number
}
}
```
@@ -48,6 +56,8 @@ interface ImageAttachmentLimits {
}
```
The local backend admits at most 20 images and 200 MiB of encoded source data per message. One source may use up to 20 MiB, 64,000,000 pixels, and 8192 pixels on either side. These source limits precede the independent normalization stage, which limits the long edge to 2048 pixels and encoded data to 4 MiB by default.
The reference records intrinsic dimensions and encoded length so clients can lay out history without decoding first, while every authoritative read still re-checks digest, media signature, dimensions, and metadata against the object.
## Commit and verified-read payloads
@@ -83,7 +93,39 @@ interface StoredImageAttachment {
}
```
`saveImage()` validates bytes and atomically commits one object before returning its reference. `validateImage()` runs the same admission checks without persisting anything; batch callers validate every member through it before saving any member, so validation rejection leaves no partial objects behind. `admitEncodedImages()` is the wire entry for base64 uploads: it enforces canonical base64, then delegates batch admission to `saveImages()`, which owns the count and aggregate-byte limits and the validate-all-before-save order. `readImage()` accepts a reference from an authorized session path and returns bytes only after integrity verification. The service is deliberately retention-neutral: resumed and forked sessions may share objects, so reference-aware garbage collection is deferred rather than tied to any one session's deletion.
```ts type-equiv
/** Deterministic request-image policy selected by one exact model route. */
interface ImageRequestPolicy {
/** Maximum width multiplied by height after aspect-preserving projection. */
maxPixels: number
/** Encoded-byte cap before base64 expansion or Files API upload. */
maxBytes: number
}
```
```ts type-equiv
/** Cached request version derived from one provider-independent normalized attachment. */
interface RequestImageAttachment {
/** Cache and upload-index key over the attachment id, policy, and fixed encoder parameters. */
variantId: ImageVariantId
/** Durable normalized attachment from which this request version was derived. */
attachment: ImageAttachmentRef
/** Encoded request bytes. */
data: Uint8Array
mediaType: ImageMediaType
bytes: number
width: number
height: number
/** Provider-compatible sample depth proven after request encoding. */
depth: 'uchar'
/** Provider-compatible color space proven after request encoding. */
space: 'srgb'
/** Whether the encoded request version retains an alpha channel. */
hasAlpha: boolean
}
```
`saveImage()` prepares and atomically commits a provider-independent normalized attachment before returning its `ImageAttachmentRef`. `saveImages()` prepares every validated attachment once before publishing the batch, so validation rejection leaves no partial objects and publication does not repeat decoding or quality selection. `admitEncodedImages()` is the wire entry for base64 uploads and delegates count, aggregate-byte, and ordered batch admission to `saveImages()`. `readImage()` verifies a normalized attachment from an authorized session path. `readImageRequest()` derives and caches one request version under an exact route pixel and byte budget; new entries are fully decoded before publication, while cache hits use a bounded metadata probe. Callers use `Promise.all` over the singular method when they need an ordered batch. The local implementation lazily encodes preferred candidates, singleflights equal request identities, lets each waiter cancel independently, stops shared work when no waiter remains, and bounds all transforms with its instance-level limiter, which defaults to two simultaneous transformations. The service is retention-neutral: resumed and forked sessions may share objects, so reference-aware garbage collection is deferred rather than tied to one session's deletion.
<!-- BEGIN GENERATED cordis-surface (gen-cordis-catalog.ts) — do not edit between markers -->
@@ -109,19 +151,19 @@ Immutable binary attachment service. Implementations validate bytes before publi
abstract validateImage(input: SaveImageAttachment): Promise<void>
/**
* Validate one ordered image batch before committing any member.
* Validation failures start no writes; storage failures return no partial
* references, although already published content-addressed objects may stay
* unreachable until a future retention policy collects them.
* @param inputs - encoded images in their owning message order.
* @returns durable references in the exact input order.
* Validate and durably commit one ordered image batch.
* @param inputs - encoded images in owning-message order.
* @returns durable normalized attachment references in the same order after every member succeeds.
*/
async saveImages(inputs: readonly SaveImageAttachment[]): Promise<readonly ImageAttachmentRef[]>
/**
* Validate and durably commit one image before its owning session event is appended.
* The returned reference describes the persisted normalized image. When
* normalization reduces the raster, its `originalDimensions` records the
* orientation-applied input dimensions.
* @param input - encoded bytes, declared media type, and optional display name.
* @returns a durable content-addressed reference.
* @returns the durable content-addressed normalized image reference.
*/
abstract saveImage(input: SaveImageAttachment): Promise<ImageAttachmentRef>
@@ -129,10 +171,19 @@ abstract saveImage(input: SaveImageAttachment): Promise<ImageAttachmentRef>
* Read one image and verify that bytes still match the recorded reference.
* @param ref - durable reference from the session log.
* @param signal - optional cancellation for backend read and verification work.
* @returns the verified bytes and canonical reference.
* @returns the verified bytes and normalized attachment reference.
* @throws the signal reason when aborted, or a storage error when verification fails.
*/
abstract readImage(ref: ImageAttachmentRef, signal?: AbortSignal): Promise<StoredImageAttachment>
/**
* Generate or read one deterministic model-request version from the stored normalized image.
* @param ref - durable provider-independent normalized attachment reference.
* @param policy - exact route pixel and encoded-byte budget.
* @param signal - optional cancellation.
* @returns request bytes and the cache/upload identity covering every transform input.
*/
readImageRequest( ref: ImageAttachmentRef, policy: ImageRequestPolicy, signal?: AbortSignal, ): Promise<RequestImageAttachment>
```
Source: [`packages/attachment/attachment/src/index.ts`](../../packages/attachment/attachment/src/index.ts)
+61 -10
View File
@@ -18,7 +18,7 @@ type ImageMediaType = 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif'
```
```ts type-equiv
/** Durable, serializable metadata for one immutable image object. */
/** Durable, serializable reference to one immutable normalized image. */
interface ImageAttachmentRef {
/** Opaque storage identifier; never a filesystem path or bearer URL. */
attachmentId: AttachmentId
@@ -32,6 +32,14 @@ interface ImageAttachmentRef {
height: number
/** Optional display name stripped of local path information. */
name?: string
/**
* Input dimensions after applying EXIF orientation and before normalization
* scaling. Present only when normalization reduced the image.
*/
originalDimensions?: {
width: number
height: number
}
}
```
@@ -48,6 +56,8 @@ interface ImageAttachmentLimits {
}
```
本地后端每条消息最多准入 20 张图片,源图编码数据总量不超过 200 MiB。单张源图不得超过 20 MiB、64,000,000 像素和单边 8192 像素。这些源文件限制先于独立的规范化阶段执行;该阶段默认把长边限制为 2048 像素,把编码数据限制为 4 MiB。
引用记录固有尺寸和编码长度,使客户端无需先解码即可排布历史记录;每次权威读取仍会根据对象重新校验摘要、媒体签名、尺寸和元数据。
## 提交与经校验读取的数据
@@ -83,7 +93,39 @@ interface StoredImageAttachment {
}
```
`saveImage()` 校验字节并以原子方式提交一个对象,之后才返回其引用。`validateImage()` 执行相同的准入检查,但不持久化任何内容;批量调用方会在保存任何成员前通过它校验所有成员,因此校验拒绝不会留下部分对象。`admitEncodedImages()` 是面向 base64 上传的 wire 入口:强制执行规范 base64,随后把批量准入委托给 `saveImages()`,由后者负责张数与聚合字节上限以及先全量校验再保存的顺序。`readImage()` 接受来自已授权会话路径的引用,只在完整性校验通过后返回字节。该服务刻意不规定保留策略:恢复和 fork 后的会话可能共享对象,因此基于引用的垃圾回收会延期实现,而不是与任何一个会话的删除绑定。
```ts type-equiv
/** Deterministic request-image policy selected by one exact model route. */
interface ImageRequestPolicy {
/** Maximum width multiplied by height after aspect-preserving projection. */
maxPixels: number
/** Encoded-byte cap before base64 expansion or Files API upload. */
maxBytes: number
}
```
```ts type-equiv
/** Cached request version derived from one provider-independent normalized attachment. */
interface RequestImageAttachment {
/** Cache and upload-index key over the attachment id, policy, and fixed encoder parameters. */
variantId: ImageVariantId
/** Durable normalized attachment from which this request version was derived. */
attachment: ImageAttachmentRef
/** Encoded request bytes. */
data: Uint8Array
mediaType: ImageMediaType
bytes: number
width: number
height: number
/** Provider-compatible sample depth proven after request encoding. */
depth: 'uchar'
/** Provider-compatible color space proven after request encoding. */
space: 'srgb'
/** Whether the encoded request version retains an alpha channel. */
hasAlpha: boolean
}
```
`saveImage()` 准备并原子提交提供方无关的规范化附件,然后直接返回 `ImageAttachmentRef``saveImages()` 在发布批次前为每个成员各准备一次经过验证的附件,因此校验拒绝不会留下部分对象,发布也不会重复解码或选择质量。`admitEncodedImages()` 是面向 base64 上传的 wire 入口,把张数、聚合字节和有序批量准入交给 `saveImages()``readImage()` 校验来自已授权会话路径的规范化附件。`readImageRequest()` 按确切路由的像素和字节预算派生并缓存请求版本;新条目在发布前完整解码,缓存命中只做有界元数据探测。调用方需要有序批次时,对单数方法使用 `Promise.all`。本地实现按需编码首选候选、合并相同请求身份的并发任务、允许每个等待方单独取消、没有等待方时停止共享任务,并通过实例级限流器限制全部变换,默认同时执行两项。该服务不规定保留策略:恢复和 fork 后的会话可能共享对象,因此基于引用的垃圾回收会延期实现,不与单个会话的删除绑定。
<!-- BEGIN GENERATED cordis-surface (gen-cordis-catalog.ts) — do not edit between markers -->
@@ -109,19 +151,19 @@ Immutable binary attachment service. Implementations validate bytes before publi
abstract validateImage(input: SaveImageAttachment): Promise<void>
/**
* Validate one ordered image batch before committing any member.
* Validation failures start no writes; storage failures return no partial
* references, although already published content-addressed objects may stay
* unreachable until a future retention policy collects them.
* @param inputs - encoded images in their owning message order.
* @returns durable references in the exact input order.
* Validate and durably commit one ordered image batch.
* @param inputs - encoded images in owning-message order.
* @returns durable normalized attachment references in the same order after every member succeeds.
*/
async saveImages(inputs: readonly SaveImageAttachment[]): Promise<readonly ImageAttachmentRef[]>
/**
* Validate and durably commit one image before its owning session event is appended.
* The returned reference describes the persisted normalized image. When
* normalization reduces the raster, its `originalDimensions` records the
* orientation-applied input dimensions.
* @param input - encoded bytes, declared media type, and optional display name.
* @returns a durable content-addressed reference.
* @returns the durable content-addressed normalized image reference.
*/
abstract saveImage(input: SaveImageAttachment): Promise<ImageAttachmentRef>
@@ -129,10 +171,19 @@ abstract saveImage(input: SaveImageAttachment): Promise<ImageAttachmentRef>
* Read one image and verify that bytes still match the recorded reference.
* @param ref - durable reference from the session log.
* @param signal - optional cancellation for backend read and verification work.
* @returns the verified bytes and canonical reference.
* @returns the verified bytes and normalized attachment reference.
* @throws the signal reason when aborted, or a storage error when verification fails.
*/
abstract readImage(ref: ImageAttachmentRef, signal?: AbortSignal): Promise<StoredImageAttachment>
/**
* Generate or read one deterministic model-request version from the stored normalized image.
* @param ref - durable provider-independent normalized attachment reference.
* @param policy - exact route pixel and encoded-byte budget.
* @param signal - optional cancellation.
* @returns request bytes and the cache/upload identity covering every transform input.
*/
readImageRequest( ref: ImageAttachmentRef, policy: ImageRequestPolicy, signal?: AbortSignal, ): Promise<RequestImageAttachment>
```
Source: [`packages/attachment/attachment/src/index.ts`](../../packages/attachment/attachment/src/index.ts)
+2 -2
View File
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write docs/subsystems/llm-streaming.md
llm-streaming.md: 4f322ca1024b9d74a4906e34f93fc6f8e4082cbf
llm-streaming.zh.md: c74cabe27c1f5fdd44711ac0aae7cd6b0a7ba7dd
llm-streaming.md: 1b2356983be4045666f7a9d40d8d191bdb4910a2
llm-streaming.zh.md: 0c2b64830dda74595deac7797c759be1970ccb32
+12
View File
@@ -675,6 +675,8 @@ interface PreparedLlmCall {
readonly retryPolicy: ResolvedRetryPolicy
/** Detached context metadata resolved with the registration-bound call. */
readonly context?: LlmModelContext
/** Exact model modalities captured with the adapter dispatch generation. */
readonly inputModalities?: readonly ModelModality[]
/** Config fields materialized by the captured adapter rather than proposed by the caller. */
readonly adapterDefaults: LlmCallConfigAdapterDefaults
/**
@@ -730,6 +732,16 @@ declare abstract class LlmAdapter {
model: string,
_signal?: AbortSignal,
): Promise<LlmResolvedModelInfo>;
/**
* Bind exact model metadata and the eventual request dispatch to one adapter generation.
* Dynamic adapters override this so settings changes between preparation and
* dispatch cannot combine one generation's capabilities with another's endpoint.
* @param provider - registered provider route.
* @param model - exact model id.
* @param signal - cancellation for model resolution.
* @returns model metadata and a one-generation stream entry point.
*/
async prepareCall(provider: string, model: string, signal?: AbortSignal): Promise<PreparedAdapterCall>;
/**
* Stream one model call as raw chunks. The only required method.
* @param options - the fully-assembled request; implementations must honor `options.signal`.
+12
View File
@@ -681,6 +681,8 @@ interface PreparedLlmCall {
readonly retryPolicy: ResolvedRetryPolicy
/** Detached context metadata resolved with the registration-bound call. */
readonly context?: LlmModelContext
/** Exact model modalities captured with the adapter dispatch generation. */
readonly inputModalities?: readonly ModelModality[]
/** Config fields materialized by the captured adapter rather than proposed by the caller. */
readonly adapterDefaults: LlmCallConfigAdapterDefaults
/**
@@ -736,6 +738,16 @@ declare abstract class LlmAdapter {
model: string,
_signal?: AbortSignal,
): Promise<LlmResolvedModelInfo>;
/**
* Bind exact model metadata and the eventual request dispatch to one adapter generation.
* Dynamic adapters override this so settings changes between preparation and
* dispatch cannot combine one generation's capabilities with another's endpoint.
* @param provider - registered provider route.
* @param model - exact model id.
* @param signal - cancellation for model resolution.
* @returns model metadata and a one-generation stream entry point.
*/
async prepareCall(provider: string, model: string, signal?: AbortSignal): Promise<PreparedAdapterCall>;
/**
* Stream one model call as raw chunks. The only required method.
* @param options - the fully-assembled request; implementations must honor `options.signal`.
+2 -2
View File
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write docs/tool-catalog.md
tool-catalog.md: 92b6d8b92050d2dc822f016c18d31a81b43ef447
tool-catalog.zh.md: e57eaf0a74c4a3cb5858d991a73decc694c7abc0
tool-catalog.md: 1fa650f1e4e025274d069f27a6522abff46af2e2
tool-catalog.zh.md: c3209e7007e9cf05770ccee0698f9e98a32e8363
+3 -3
View File
@@ -24,7 +24,7 @@ This table connects model-visible tool names to the plugin package and service s
| `@deepseek-ai/dsh-tool-bash-persistent` | `bash` | `ctx.tools`, `ctx.terminals`, `an owning Agent at execution time` | `tool/call`, `PTY shell state`, `tool/result` | - | One owner-isolated persistent bash tool; deployment composition supplies the PTY backend and may override the model-facing environment description. |
| `@deepseek-ai/dsh-tool-pwsh-persistent` | `pwsh` | `ctx.tools`, `ctx.terminals`, `an owning Agent at execution time` | `tool/call`, `PTY shell state`, `tool/result` | - | One owner-isolated persistent pwsh tool, the Windows counterpart of the persistent bash tool; deployment composition supplies a pwsh-dialect PTY backend and may override the model-facing environment description. |
| `@deepseek-ai/dsh-tool-str-replace-editor` | `str_replace_editor` | `ctx.tools`, `ctx.fs` | `tool/call`, `fs/observed after view presence/absence, edit absence, or successful mutation`, `tool/result` | - | Standalone view/create/unique literal replace/line insert tool over the filesystem seam; it composes with any shell or terminal API. |
| `@deepseek-ai/dsh-tool-fs` | `edit`, `read`, `read_image`, `write` | `ctx.tools`, `ctx.fs`, `ctx.systemPrompt`, `ctx.attachments (read_image registration)`, `ctx.llm + an image-capable route (read_image execution)` | `tool/call`, `fs/write-intent or fs/edit-intent for mutations`, `fs/observed after read presence/absence or successful file operation`, `durable attachment (read_image)`, `tool/result` | - | The read-before-write/edit policy is added by `@deepseek-ai/dsh-fs-observation-policy` (an `fs/*` event-gate plugin, no schema change); a deployment that loads these tools is expected to also load it. `read_image` is not registered without `ctx.attachments`; its schema is route-independent, and execution refuses unless the exact routed model declares image input. |
| `@deepseek-ai/dsh-tool-fs` | `edit`, `read`, `read_image`, `write` | `ctx.tools`, `ctx.fs`, `ctx.systemPrompt`, `ctx.attachments (image-tool registration)`, `ctx.llm + an image-capable route (image-tool execution)` | `tool/call`, `fs/write-intent or fs/edit-intent for mutations`, `fs/observed after read presence/absence or successful file operation`, `durable attachment (read_image)`, `tool/result` | - | The read-before-write/edit policy is added by `@deepseek-ai/dsh-fs-observation-policy` (an `fs/*` event-gate plugin, no schema change); a deployment that loads these tools is expected to also load it. The image tool is not registered without `ctx.attachments`; its schema is route-independent, and execution refuses unless the exact routed model declares image input. |
| `@deepseek-ai/dsh-tool-fs-search` | `glob`, `grep` | `ctx.tools`, `ctx.subprocess`, `ctx.systemPrompt` | `tool/call`, `tool/result` | - | glob and grep are unconditional discovery tools that spawn the packaged ripgrep binary (`@vscode/ripgrep`) through ctx.subprocess as ordinary foreground calls (never background jobs) — no host `rg` install and no shell layer. The catalog uses `sampleOverCapGlobResults: true`; deployments must choose that behavior explicitly. Capped results save the complete formatted list through the optional ctx.spillStore backend; returned locators are follow-up-readable/searchable when the backend exposes local paths in co-located deployments. |
| `@deepseek-ai/dsh-tool-terminal` | `terminal_close`, `terminal_list`, `terminal_open`, `terminal_read`, `terminal_send`, `terminal_signal` | `ctx.tools`, `ctx.terminals`, `ctx.systemPrompt`, `ctx.jobs at call time for run_in_background` | `tool/call`, `tool/result` | - | The six terminal tools are opt-in and complement one-shot shell/filesystem tools. `terminal_send(run_in_background: true)` registers with `ctx.jobs`; TUI, named key sequences, BEL, resize, auto-start, and cross-agent sharing are absent from the schema. |
| `@deepseek-ai/dsh-tool-goal` | `create_goal`, `get_goal`, `update_goal` | `ctx.tools`, `ctx.agents`, `ctx.goals`, `ctx.systemPrompt`, `a calling Agent in an authorized open turn` | `tool/call`, `goal/change for mutations`, `tool/result` | - | create, edit, pause, and resume require direct-human root authority; complete and blocked also accept the exact current goal round. The default blocked lower bound is three admitted rounds. |
@@ -695,7 +695,7 @@ Source: [`packages/fs/tool-fs/src/index.ts`](../packages/fs/tool-fs/src/index.ts
### `read_image`
Read a PNG/JPEG/WebP/GIF file and return the image itself. Requires the current model to accept image input.
Read a PNG/JPEG/WebP/GIF file and return the image itself. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.
```json
{
@@ -740,7 +740,7 @@ Create or fully replace a UTF-8 text file.
Source: [`packages/fs/tool-fs/src/index.ts`](../packages/fs/tool-fs/src/index.ts)
The read-before-write/edit policy is added by `@deepseek-ai/dsh-fs-observation-policy` (an `fs/*` event-gate plugin, no schema change); a deployment that loads these tools is expected to also load it. `read_image` is not registered without `ctx.attachments`; its schema is route-independent, and execution refuses unless the exact routed model declares image input.
The read-before-write/edit policy is added by `@deepseek-ai/dsh-fs-observation-policy` (an `fs/*` event-gate plugin, no schema change); a deployment that loads these tools is expected to also load it. The image tool is not registered without `ctx.attachments`; its schema is route-independent, and execution refuses unless the exact routed model declares image input.
<a id="deepseek-aidsh-tool-fs-search"></a>
+3 -3
View File
@@ -28,7 +28,7 @@
| `@deepseek-ai/dsh-tool-bash-persistent` | `bash` | `ctx.tools``ctx.terminals``an owning Agent at execution time` | `tool/call``PTY shell state``tool/result` | - | 一个按所有者隔离的持久 bash 工具;部署组合提供 PTY 后端,并可覆盖面向模型的环境描述。 |
| `@deepseek-ai/dsh-tool-pwsh-persistent` | `pwsh` | `ctx.tools``ctx.terminals``an owning Agent at execution time` | `tool/call``PTY shell state``tool/result` | - | 一个按所有者隔离的持久 pwsh 工具,持久 bash 工具的 Windows 对应物;部署组合提供 pwsh 方言的 PTY 后端,并可覆盖面向模型的环境描述。 |
| `@deepseek-ai/dsh-tool-str-replace-editor` | `str_replace_editor` | `ctx.tools``ctx.fs` | `tool/call``fs/observed after view presence/absence, edit absence, or successful mutation``tool/result` | - | 基于文件系统 seam 的独立查看/创建/唯一字面量替换/按行插入工具;可与任何 shell 或终端接口组合。 |
| `@deepseek-ai/dsh-tool-fs` | `edit``read``read_image``write` | `ctx.tools``ctx.fs``ctx.systemPrompt``ctx.attachments (read_image registration)``ctx.llm + an image-capable route (read_image execution)` | `tool/call``fs/write-intent or fs/edit-intent for mutations``fs/observed after read presence/absence or successful file operation``durable attachment (read_image)``tool/result` | - | 先读后写/编辑策略由 `@deepseek-ai/dsh-fs-observation-policy` 添加;它是一个 `fs/*` 事件门禁插件,不会改变 schema。加载这些工具的部署按预期也应加载该插件。没有 `ctx.attachments` `read_image` 不会注册;其 schema 与路由无关,执行时除非确切路由的模型声明图输入,否则拒绝。 |
| `@deepseek-ai/dsh-tool-fs` | `edit``read``read_image``write` | `ctx.tools``ctx.fs``ctx.systemPrompt``ctx.attachments (image-tool registration)``ctx.llm + an image-capable route (image-tool execution)` | `tool/call``fs/write-intent or fs/edit-intent for mutations``fs/observed after read presence/absence or successful file operation``durable attachment (read_image)``tool/result` | - | 先读后写/编辑策略由 `@deepseek-ai/dsh-fs-observation-policy` 添加;它是一个 `fs/*` 事件门禁插件,不会改变 schema。加载这些工具的部署按预期也应加载该插件。没有 `ctx.attachments`图片工具不会注册;其 schema 与路由无关,执行时除非确切路由的模型声明图输入,否则拒绝。 |
| `@deepseek-ai/dsh-tool-fs-search` | `glob``grep` | `ctx.tools``ctx.subprocess``ctx.systemPrompt` | `tool/call``tool/result` | - | glob 和 grep 是无条件可用的发现工具,通过 ctx.subprocess spawn 随包提供的 ripgrep 二进制文件(`@vscode/ripgrep`),并作为普通前台调用运行,绝不作为后台任务;无需在宿主机安装 `rg`,也不经过 shell 层。本目录使用 `sampleOverCapGlobResults: true`;部署必须显式选择该行为。结果超过上限时,会通过可选的 ctx.spillStore 后端保存完整的格式化列表;在共置部署中,如果后端公开本地路径,返回的定位信息可供后续读取/搜索。 |
| `@deepseek-ai/dsh-tool-terminal` | `terminal_close``terminal_list``terminal_open``terminal_read``terminal_send``terminal_signal` | `ctx.tools``ctx.terminals``ctx.systemPrompt``ctx.jobs at call time for run_in_background` | `tool/call``tool/result` | - | 这 6 个终端工具需要选择启用,用于补充一次性 bash/文件系统工具。`terminal_send(run_in_background: true)` 会注册到 `ctx.jobs`;schema 不包含 TUI、具名按键序列、BEL、调整尺寸、自动启动和跨 agent 共享。 |
| `@deepseek-ai/dsh-tool-goal` | `create_goal``get_goal``update_goal` | `ctx.tools``ctx.agents``ctx.goals``ctx.systemPrompt``a calling Agent in an authorized open turn` | `tool/call``goal/change for mutations``tool/result` | - | create、edit、pause 和 resume 要求直接来自人类的根权限;complete 和 blocked 也接受确切的当前 Goal Round。blocked 的默认下限是 3 个获准的 Round。 |
@@ -701,7 +701,7 @@ pwsh 工具是 Windows 组合中 bash 执行器 seam 的 PowerShell 方言消费
### `read_image`
读取 PNG/JPEG/WebP/GIF 文件并返回图像本身。要求当前模型接受图像输入。
读取 PNG/JPEG/WebP/GIF 文件并返回图像本身。Harness 会在下一次模型请求前校验并缩小受支持的大图,因此仅为查看图片时应直接使用此工具,无需安装图片库或创建缩略图。可以用小批次并发读取彼此独立的文件。要求当前模型接受图像输入。
```json
{
@@ -746,7 +746,7 @@ pwsh 工具是 Windows 组合中 bash 执行器 seam 的 PowerShell 方言消费
来源:[`packages/fs/tool-fs/src/index.ts`](../packages/fs/tool-fs/src/index.ts)
先读后写/编辑策略由 `@deepseek-ai/dsh-fs-observation-policy` 添加;它是一个 `fs/*` 事件门禁插件,不会改变 schema。加载这些工具的部署按预期也应加载该插件。没有 `ctx.attachments` `read_image` 不会注册;其 schema 与路由无关,执行时除非确切路由的模型声明图输入,否则拒绝。
先读后写/编辑策略由 `@deepseek-ai/dsh-fs-observation-policy` 添加;它是一个 `fs/*` 事件门禁插件,不会改变 schema。加载这些工具的部署按预期也应加载该插件。没有 `ctx.attachments`图片工具不会注册;其 schema 与路由无关,执行时除非确切路由的模型声明图输入,否则拒绝。
<a id="deepseek-aidsh-tool-fs-search"></a>
+114 -56
View File
@@ -250,10 +250,10 @@ const SCENARIOS: Scenario[] = [
toolSchemasSource: 'read-image',
configPath: IMAGE_TEXT_ROUTE_CONFIG,
},
// Authored keyless replay of the oversized-image refusal: admission rejects
// the 2001x1 fixture at the default 2000px per-side limit, the model sees a
// recoverable tool error, and the turn still completes — the image never
// enters durable history.
// Authored keyless replay of wide-image admission: the 2001x1 fixture sits
// inside the wide source envelope and the canonical budget, so read_image
// succeeds and the attachment keeps the source bytes byte-identically —
// the same read the pre-canonicalization 2000px admission cap refused.
{
name: 'read-image-dimension',
hasModelTurn: true,
@@ -702,30 +702,70 @@ defineAcpSnapshotSuite({
hasPwsh,
})
it('pins native DeepSeek image offload in the request sent by the assembled app', async () => {
it('pins native DeepSeek Files offload and inline fallback in assembled requests', async () => {
const requests: Record<string, unknown>[] = []
const fileRequests: Array<{ method: string; path: string; bytes: number }> = []
let rejectFiles = false
const server = createServer((request: IncomingMessage, response: ServerResponse) => {
let body = ''
request.setEncoding('utf8')
request.on('data', (chunk: string) => { body += chunk })
const chunks: Buffer[] = []
request.on('data', (chunk: Buffer) => { chunks.push(chunk) })
request.on('end', () => {
requests.push(JSON.parse(body) as Record<string, unknown>)
response.writeHead(200, { 'content-type': 'text/event-stream' })
const events = requests.length === 1
? [
'data: {"choices":[{"delta":{"tool_calls":[{"index":0,"id":"native-read-image","type":"function","function":{"name":"read_image","arguments":"{\\"file_path\\":\\"red.png\\"}"}}]},"index":0,"finish_reason":null}]}',
'data: {"choices":[{"delta":{},"index":0,"finish_reason":"tool_calls"}],"usage":{"prompt_tokens":3,"completion_tokens":1}}',
'data: [DONE]',
'',
]
: [
'data: {"choices":[{"delta":{"role":"assistant","content":""},"index":0,"finish_reason":null}]}',
'data: {"choices":[{"delta":{"content":"DONE"},"index":0,"finish_reason":null}]}',
'data: {"choices":[{"delta":{},"index":0,"finish_reason":"stop"}],"usage":{"prompt_tokens":3,"completion_tokens":1}}',
'data: [DONE]',
'',
]
response.end(events.join('\n\n'))
void (async () => {
const url = new URL(request.url ?? '/', 'http://localhost')
const body = Buffer.concat(chunks)
if (url.pathname === '/files' && request.method === 'POST') {
const headers = new Headers()
for (const [name, value] of Object.entries(request.headers)) {
if (value !== undefined) headers.set(name, Array.isArray(value) ? value.join(', ') : value)
}
const form = await new Request('http://localhost/files', {
method: 'POST', headers, body,
}).formData()
const file = form.get('file')
if (!(file instanceof Blob)) throw new Error('snapshot Files upload omitted file')
fileRequests.push({ method: 'POST', path: url.pathname, bytes: file.size })
if (rejectFiles) {
response.writeHead(503, { 'content-type': 'application/json' }).end(JSON.stringify({
error: { message: 'Files temporarily unavailable' },
}))
return
}
const createdAt = Math.floor(Date.now() / 1_000)
response.writeHead(200, { 'content-type': 'application/json' }).end(JSON.stringify({
id: 'file-api-snapshot-1',
object: 'file',
bytes: file.size,
created_at: createdAt,
filename: 'dsh-snapshot.png',
purpose: 'user_data',
expires_at: createdAt + Number(form.get('expires_after[seconds]')),
}))
return
}
if (url.pathname !== '/chat/completions') {
response.writeHead(404).end()
return
}
requests.push(JSON.parse(body.toString('utf8')) as Record<string, unknown>)
response.writeHead(200, { 'content-type': 'text/event-stream' })
const events = requests.length === 1
? [
'data: {"choices":[{"delta":{"tool_calls":[{"index":0,"id":"native-read-image","type":"function","function":{"name":"read_image","arguments":"{\\"file_path\\":\\"red.png\\"}"}}]},"index":0,"finish_reason":null}]}',
'data: {"choices":[{"delta":{},"index":0,"finish_reason":"tool_calls"}],"usage":{"prompt_tokens":3,"completion_tokens":1}}',
'data: [DONE]',
'',
]
: [
'data: {"choices":[{"delta":{"role":"assistant","content":""},"index":0,"finish_reason":null}]}',
'data: {"choices":[{"delta":{"content":"DONE"},"index":0,"finish_reason":null}]}',
'data: {"choices":[{"delta":{},"index":0,"finish_reason":"stop"}],"usage":{"prompt_tokens":3,"completion_tokens":1}}',
'data: [DONE]',
'',
]
response.end(events.join('\n\n'))
})().catch((error: unknown) => {
response.writeHead(500, { 'content-type': 'text/plain' }).end(String(error))
})
})
})
await new Promise<void>(resolve => server.listen(0, '127.0.0.1', resolve))
@@ -764,34 +804,21 @@ it('pins native DeepSeek image offload in the request sent by the assembled app'
})
expect(result.stderr).toBe('')
expect(requests).toHaveLength(2)
expect(fileRequests).toEqual([{ method: 'POST', path: '/files', bytes: 69 }])
const messages = requests[0]?.messages as { content?: unknown }[] | undefined
const offloaded = messages?.find(message => JSON.stringify(message.content).includes('[image omitted'))
expect(offloaded?.content).toMatchInlineSnapshot(`
[
{
"text": "Compare the older image ",
"type": "text",
},
{
"text": "[image omitted to keep the request within its image limit; older images are omitted first. If this image is still needed, read its file again when a path is available; otherwise ask the user to attach it again.]",
"type": "text",
},
{
"text": " with the newer image ",
"type": "text",
},
{
"image_url": {
"url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAADElEQVR4nGP4z8AAAAMBAQDJ/pLvAAAAAElFTkSuQmCC",
},
"type": "image_url",
},
{
"text": ", then use read_image on red.png and reply with DONE.",
"type": "text",
},
]
`)
expect(offloaded?.content).toEqual([
{ type: 'text', text: 'Compare the older image ' },
{ type: 'text', text: OFFLOADED_IMAGE_TEXT },
{ type: 'text', text: ' with the newer image ' },
{
type: 'text',
text: '\nImage sha256:b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640; '
+ 'request image 1x1px.',
},
{ type: 'file', file_id: 'file-api-snapshot-1' },
{ type: 'text', text: ', then use read_image on red.png and reply with DONE.' },
])
const followup = structuredClone((requests[1]?.messages as unknown[]).slice(1)) as Array<{
role?: unknown
@@ -830,19 +857,50 @@ it('pins native DeepSeek image offload in the request sent by the assembled app'
{
role: 'tool',
tool_call_id: 'native-read-image',
content: '<path>{{cwd}}/red.png</path>\n<type>image</type>\n<content>\nimage/png image, 1x1 px, 69 bytes\n</content>',
content: '<path>{{cwd}}/red.png</path>\n<type>image</type>\n<content>\nimage/png image, 1x1 px, 69 bytes\n'
+ '</content>\nImage sha256:b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640; request image 1x1px.',
},
{
role: 'user',
content: [
{ type: 'text', text: 'Attached image(s) from tool result:' },
{
type: 'image_url',
image_url: { url: `data:image/png;base64,${image}` },
},
{ type: 'file', file_id: 'file-api-snapshot-1' },
],
},
])
rejectFiles = true
const fallback = await runScenario(input, {
agent: AGENT,
mode: 'record',
configPath: IMAGE_OFFLOAD_CONFIG,
fixtureFile: join(SNAPSHOTS_DIR, 'image-offload-request', 'session.jsonl'),
workspaceDir: join(SNAPSHOTS_DIR, 'read-image', 'workspace'),
env: {
DSH_SNAPSHOT_API_KEY: 'snapshot-fallback-key',
DSH_SNAPSHOT_BASE_URL: `http://127.0.0.1:${address.port}`,
},
})
expect(fallback.stderr).toBe('')
expect(fileRequests).toEqual([
{ method: 'POST', path: '/files', bytes: 69 },
{ method: 'POST', path: '/files', bytes: 69 },
])
expect(requests).toHaveLength(3)
const fallbackMessages = requests[2]?.messages as { content?: unknown }[] | undefined
const fallbackInput = fallbackMessages?.find(message => JSON.stringify(message.content).includes('[image omitted'))
expect(fallbackInput?.content).toEqual([
{ type: 'text', text: 'Compare the older image ' },
{ type: 'text', text: OFFLOADED_IMAGE_TEXT },
{ type: 'text', text: ' with the newer image ' },
{
type: 'text',
text: '\nImage sha256:b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640; '
+ 'request image 1x1px.',
},
{ type: 'image_url', image_url: { url: `data:image/png;base64,${image}` } },
{ type: 'text', text: ', then use read_image on red.png and reply with DONE.' },
])
} finally {
await new Promise<void>(resolve => server.close(() => { resolve() }))
}
+2 -1
View File
@@ -12,7 +12,8 @@
apiKeyEnv: DSH_SNAPSHOT_API_KEY
baseURL: !!js process.env.DSH_SNAPSHOT_BASE_URL
thinking: disabled
maxRequestImageBytes: 92
maxRequestFilesBytes: 92
imageOffloadByteQuantum: 1
models:
- id: deepseek-v4-flash-vision-exp
contextWindow: 32768
@@ -125,7 +125,7 @@ interface ToolArgsMap {
/** Maximum number of lines to return. Defaults to 2000. */
limit?: number;
} & Record<string, JsonValue>;
/** Read a PNG/JPEG/WebP/GIF file and return the image itself. Requires the current model to accept image input. */
/** Read a PNG/JPEG/WebP/GIF file and return the image itself. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input. */
read_image: {
/** Path to the image file, resolved by the filesystem backend. */
file_path: string;
@@ -363,6 +363,10 @@ interface ToolOutputMap {
width: number;
height: number;
name?: string;
originalDimensions?: {
width: number;
height: number;
};
};
};
send_message: {
@@ -8,7 +8,7 @@
},
{
"op": "prompt",
"text": "Use read_image on wide.png in the current directory. If the tool refuses because the image is too large, reply with exactly the single word TOOLARGE."
"text": "Use read_image on wide.png in the current directory, then reply with exactly the single word WIDE."
}
]
}
@@ -1,9 +1,9 @@
{"type":"session","version":0,"id":"33333333-3333-4333-8333-333333333333","createdAt":1783951000000,"cwd":"{{cwd}}","delegationDepth":0}
{"type":"agent/inbox/spliced","data":{"target":"next-turn","start":0,"inserted":[{"content":[{"type":"text","text":"Use read_image on wide.png in the current directory. If the tool refuses because the image is too large, reply with exactly the single word TOOLARGE."}],"source":{"kind":"user"},"role":"user","id":"0a0a0a0a-0000-4000-8000-000000000001"}]}}
{"type":"agent/inbox/spliced","data":{"target":"next-turn","start":0,"inserted":[{"content":[{"type":"text","text":"Use read_image on wide.png in the current directory, then reply with exactly the single word WIDE."}],"source":{"kind":"user"},"role":"user","id":"0a0a0a0a-0000-4000-8000-000000000001"}]}}
{"type":"turn/start","data":{"turn":1}}
{"type":"agent/inbox/spliced","data":{"target":"next-turn","start":0,"removedCount":1,"inserted":[]}}
{"type":"step/start","data":{"turn":1,"step":1}}
{"type":"user/message","data":{"content":[{"type":"text","text":"Use read_image on wide.png in the current directory. If the tool refuses because the image is too large, reply with exactly the single word TOOLARGE."}],"source":{"kind":"user"},"role":"user","id":"0a0a0a0a-0000-4000-8000-000000000001"},"surfaceOp":"append"}
{"type":"user/message","data":{"content":[{"type":"text","text":"Use read_image on wide.png in the current directory, then reply with exactly the single word WIDE."}],"source":{"kind":"user"},"role":"user","id":"0a0a0a0a-0000-4000-8000-000000000001"},"surfaceOp":"append"}
{"type":"user/message","data":{"content":[{"type":"text","text":"Current runtime context. This snapshot supersedes earlier runtime-context snapshots.\n\nCurrent DSH file policy: danger-full-access. The DSH file sandbox does not restrict file modifications by available operations.\n\nApproval prompts are disabled in this session: actions that require approval are rejected automatically — do not request sandbox escalation (do not set `sandbox_permissions`)."}],"source":{"kind":"plugin","plugin":"@deepseek-ai/dsh-system-prompt","form":"snapshot","sections":[{"name":"sandbox:policy","text":"Current DSH file policy: danger-full-access. The DSH file sandbox does not restrict file modifications by available operations."},{"name":"approval:policy","text":"Approval prompts are disabled in this session: actions that require approval are rejected automatically — do not request sandbox escalation (do not set `sandbox_permissions`)."}]},"role":"user","id":"11a08f07-014a-408b-bfc5-634770ce7179"},"surfaceOp":"append"}
{"type":"session/title","data":{"title":"Use read_image on wide.png in","messageSeqs":[4],"source":{"kind":"fallback"}}}
{"type":"request/header","data":{"header":{"config":{"provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"system":"{{system}}","tools":"{{tools}}"},"reason":"initial"}}
@@ -14,13 +14,13 @@
{"type":"assistant/chunk","data":{"turn":1,"step":1,"chunk":{"type":"finish","reason":{"kind":"tool-calls"}}}}
{"type":"assistant/message","data":{"turn":1,"step":1,"message":{"role":"assistant","content":[{"type":"tool-call","id":"read-image-dimension","name":"read_image","arguments":"{\"file_path\":\"wide.png\"}"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"a25d70ac-2bd6-4e44-9121-ed74975ee229"},"usage":{"inputTokens":3,"outputTokens":3}},"sourceEventSeqs":[9,10,11,12],"surfaceOp":"append"}
{"type":"tool/call","data":{"turn":1,"step":1,"callId":"read-image-dimension","name":"read_image","arguments":"{\"file_path\":\"wide.png\"}"}}
{"type":"tool/result","data":{"turn":1,"step":1,"message":{"source":{"kind":"tool","callId":"read-image-dimension"},"content":[{"type":"tool-result","toolCallId":"read-image-dimension","content":[{"type":"text","text":"Error: cannot read \"{{cwd}}/wide.png\": at least one image side exceeds the 2000px limit; downscale the image and read the smaller copy"}],"isError":true}],"role":"user","id":"ee31751e-df5a-458e-8497-8113cf6107ef"}},"sourceEventSeqs":[14],"surfaceOp":"append"}
{"type":"tool/result","data":{"turn":1,"step":1,"message":{"source":{"kind":"tool","callId":"read-image-dimension"},"content":[{"type":"tool-result","toolCallId":"read-image-dimension","content":[{"type":"text","text":"<path>{{cwd}}/wide.png</path>\n<type>image</type>\n<content>\nimage/png image, 2001x1 px, 133 bytes\n</content>"},{"type":"image","attachment":{"attachmentId":"sha256:0333f95051f5c038cab720d90112f1775e9ff1f8f7dddc86653e80ff241c5720","mediaType":"image/png","bytes":133,"width":2001,"height":1,"name":"wide.png"}}],"isError":false}],"role":"user","id":"ee31751e-df5a-458e-8497-8113cf6107ef"}},"sourceEventSeqs":[14],"surfaceOp":"append"}
{"type":"step/end","data":{"turn":1,"step":1}}
{"type":"step/start","data":{"turn":1,"step":2}}
{"type":"assistant/chunk","data":{"turn":1,"step":2,"chunk":{"type":"block-start","index":0,"blockType":"text"}}}
{"type":"assistant/chunk","data":{"turn":1,"step":2,"chunk":{"type":"block-end","index":0,"block":{"type":"text","text":"TOOLARGE"}}}}
{"type":"assistant/chunk","data":{"turn":1,"step":2,"chunk":{"type":"block-end","index":0,"block":{"type":"text","text":"WIDE"}}}}
{"type":"assistant/chunk","data":{"turn":1,"step":2,"chunk":{"type":"usage","usage":{"inputTokens":3,"outputTokens":3}}}}
{"type":"assistant/chunk","data":{"turn":1,"step":2,"chunk":{"type":"finish","reason":{"kind":"stop"}}}}
{"type":"assistant/message","data":{"turn":1,"step":2,"message":{"role":"assistant","content":[{"type":"text","text":"TOOLARGE"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"3a95dd83-34f7-4bc0-afb6-7ba3c9b483be"},"usage":{"inputTokens":3,"outputTokens":3}},"sourceEventSeqs":[18,19,20,21],"surfaceOp":"append"}
{"type":"assistant/message","data":{"turn":1,"step":2,"message":{"role":"assistant","content":[{"type":"text","text":"WIDE"}],"source":{"kind":"model","provider":"deepseek-official","model":"deepseek-v4-flash-vision-exp"},"id":"3a95dd83-34f7-4bc0-afb6-7ba3c9b483be"},"usage":{"inputTokens":3,"outputTokens":3}},"sourceEventSeqs":[18,19,20,21],"surfaceOp":"append"}
{"type":"step/end","data":{"turn":1,"step":2}}
{"type":"turn/end","data":{"turn":1,"reason":{"kind":"completed"}}}
@@ -1,4 +1,4 @@
{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":1,"agentInfo":{"name":"deepseek-harness-acp","version":"0.0.1"},"agentCapabilities":{"promptCapabilities":{"image":true,"audio":false,"embeddedContext":false}},"authMethods":[]}}
{"jsonrpc":"2.0","id":2,"result":{"sessionId":"{{sessionId}}"}}
{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_message_chunk","content":{"type":"text","text":"TOOLARGE"}}}}
{"jsonrpc":"2.0","method":"session/update","params":{"sessionId":"{{sessionId}}","update":{"sessionUpdate":"agent_message_chunk","content":{"type":"text","text":"WIDE"}}}}
{"jsonrpc":"2.0","id":3,"result":{"stopReason":"end_turn"}}
@@ -246,7 +246,7 @@
},
{
"name": "read_image",
"description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. Requires the current model to accept image input.",
"description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.",
"parameters": {
"type": "object",
"properties": {
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/attachment/attachment-local/README.md
README.md: e4f2d5748768a1dc2a6b79c3ed9e364c56a67248
README.zh.md: 6b548fb993faef996f1508ba9f9efc31b20fea64
README.md: 3ed4ab3251b0a609807c76930226bec63f0164cd
README.zh.md: 85abd10389acc46c2d89dd85628f5d201b089710
@@ -2,7 +2,11 @@
English | [中文](README.zh.md)
The private local implementation of [`@deepseek-ai/dsh-attachment`](../attachment). Objects land at `<DSH_HOME>/attachments/v1/objects/<sha256-prefix>/<sha256>` and are addressed by an opaque `sha256:` id. Each process proves a home durable once by syncing every ancestor entry to the filesystem root, so a directory another process created but has not yet synced is never mistaken for a safe boundary. Writes then use a private staging directory, owner-only files, a synced temporary file, an atomic exclusive hard-link publish, and directory syncs on the publication path (POSIX; Windows relies on filesystem metadata journaling) so the reported reference survives a crash. Write admission and reads fully decode the raster before accepting its format and dimensions; reads also re-check the digest and logged metadata. Byte, total-pixel, and per-side dimension limits are write-time admission policy, so a later policy reduction does not make already-admitted history unreadable. The per-side default (2000px) stays below the strictest dimension bound deployed model routes enforce on requests carrying many images: an admitted image rides every later request of its session, so admission is the last point where a provider-rejected image can be kept out of durable history.
The private local implementation of [`@deepseek-ai/dsh-attachment`](../attachment). Objects land at `<DSH_HOME>/attachments/v1/objects/<sha256-prefix>/<sha256>` and are addressed by an opaque `sha256:` id. Each process proves a home durable once by syncing every ancestor entry to the filesystem root. Writes use a private staging directory, owner-only files, a synced temporary file, an atomic exclusive hard-link publish, and directory syncs on the publication path (POSIX; Windows relies on filesystem metadata journaling) so the reported reference survives a crash.
Admission accepts at most 20 images and 200MiB of encoded source bytes per message. Each source may use up to 20MiB, 64,000,000 pixels, and 8192px per side. It then prepares a provider-independent normalized attachment. EXIF orientation is applied, metadata and color profiles are removed, pixels become 8-bit sRGB/sRGBA, and the long edge is reduced proportionally to `normalizedImageMaxDimension` (2048px by default). The normalized attachment has its own `normalizedImageMaxBytes` safety cap (4MiB by default). Transparent pixels are retained; Sharp/libvips may omit an alpha plane whose samples are all opaque. A nearest-neighbour bounded sample classifies color complexity without averaging high-frequency pixels. Confirmed low-color images try PNG, using a palette only when the input has no alpha channel, then WebP at qualities 85, 80, and 75. Other alpha images try WebP at those qualities; other opaque images try JPEG. Each candidate runs only after the preceding candidate exceeds the cap. Dimensions shrink only after every candidate at one size exceeds the cap. A clean, single-frame 8-bit sRGB/sRGBA PNG, JPEG, or WebP already within both normalization limits passes through byte-identically; 16-bit PNG, GIF, animated input, metadata, orientation, and incompatible color spaces force conversion. The source and converted attachment are each fully decoded once. `saveImages` prepares and verifies every normalized attachment once before publishing the batch, so validation failure leaves no partial references and commit does not repeat full image encoding.
Request versions live below `<DSH_HOME>/attachments/v1/request-images/`. `readImageRequest` scales the stored normalized attachment under a total-pixel budget without enlargement, then enforces a separate encoded-byte cap. The request encoder uses the same color branches, with PNG (palette only without alpha) before WebP 85 and 80 for low-color images, WebP 85 then 80 for other alpha images, and JPEG 85 then 80 for other opaque images. It executes candidates lazily and reduces dimensions only after both quality attempts exceed the request cap. Its cache identity includes the attachment id, transform version, pixel and byte budgets, and fixed encoder settings. Cached bytes are fully decoded and checked as 8-bit sRGB/sRGBA before use. Concurrent calls for one identity share one transform and cache write; cancelling one waiter does not cancel the shared work. Callers compose ordered batches from singular reads, while the service's FIFO limiter applies `imageCompressionConcurrency` to simultaneous normalization and request transforms. The setting ranges from 1 through 8 and defaults to 2; file publication remains ordered after preparation.
`DSH_HOME` resolves through the shared path policy: explicit config, `$DSH_HOME`, then `~/.dsh`. Session logs contain only the reference and verified metadata, never this host path. `readImage` forwards optional cancellation into the filesystem read, observes it around verification, and preserves it instead of wrapping it as `ATTACHMENT_READ_FAILED`.
@@ -12,10 +16,11 @@ Indirectly, through durable replay of historical user images and structured mode
#### KV Cache effect
None beyond the image block owned by the requesting adapter.
Normalization and request projection are deterministic. An unchanged attachment and route policy reuse identical cached request bytes on later turns.
## Known Limitations and Deferred Work
- Objects are retained indefinitely; reference-aware garbage collection is deferred.
- The local backend assumes the host and provider adapter share this filesystem service.
- Animated GIF metadata is validated from the logical screen; frame-level decoding policy is provider-owned.
- Animated GIF sources keep only their first frame; animation is outside the version-one image contract.
- The normalization and request encoders are pinned by the installed sharp/libvips build; an encoder or transform-version upgrade re-addresses future normalized attachments or request variants while existing objects stay valid.
@@ -2,7 +2,11 @@
[English](README.md) | 中文
这是 [`@deepseek-ai/dsh-attachment`](../attachment) 的私有本地实现。对象存放在 `<DSH_HOME>/attachments/v1/objects/<sha256-prefix>/<sha256>`,并通过不透明的 `sha256:` 标识符寻址。每个进程都会通过将每个祖先目录项逐级同步到文件系统根目录,为某个 home 一次性证明其持久性,因此绝不会把另一个进程已经创建但尚未同步的目录误认为安全边界。随后,写入过程使用私有暂存目录、仅所有者可访问的文件、经过同步的临时文件、原子且排他的硬链接发布,并对发布路径执行目录同步(适用于 POSIX;Windows 依赖文件系统元数据日志),确保已报告的引用能够在崩溃后继续存在。写入准入与读取都会完整解码光栅图片,之后才接受其格式和尺寸;读取还会重新校验摘要和已记录的元数据。字节、总像素和单边尺寸限制属于写入时的准入策略,因此后续收紧限制不会导致已经接纳的历史记录变得不可读。单边默认值(2000px)低于已部署模型路由对携带多张图片的请求所强制执行的最严格尺寸上限:一张已接纳的图片会随会话之后的每次请求发送,准入是把必然被上游拒绝的图片挡在持久历史之外的最后一道关口。
这是 [`@deepseek-ai/dsh-attachment`](../attachment) 的私有本地实现。对象存放在 `<DSH_HOME>/attachments/v1/objects/<sha256-prefix>/<sha256>`,并通过不透明的 `sha256:` 标识符寻址。每个进程都会把每级祖先目录项同步到文件系统根目录,以此一次性证明 home 已持久化。写入使用私有暂存目录、仅所有者可访问的文件、经过同步的临时文件、原子且排他的硬链接发布,并对发布路径执行目录同步(适用于 POSIX;Windows 依赖文件系统元数据日志),确保已报告的引用能够在崩溃后继续存在。
每条消息最多准入 20 张图片,源图编码字节总量不超过 200MiB。每张源图不得超过 20MiB、64,000,000 像素和单边 8192px。随后生成提供方无关的规范化附件:应用 EXIF 方向,删除元数据和色彩配置文件,转换为 8-bit sRGB/sRGBA,并保持宽高比把长边限制到 `normalizedImageMaxDimension`(默认 2048px)。规范化附件有独立的 `normalizedImageMaxBytes` 安全上限(默认 4MiB)。透明像素会保留;当所有 alpha 样本均为不透明时,Sharp/libvips 可能省略没有实际作用的 alpha 平面。系统用 nearest-neighbour 对有界样本分类,不会通过像素平均把高频图片误判为低色数。确认的低色数图片先尝试 PNG,只有不带 alpha 通道时才使用 palette,随后依次尝试质量 85、80、75 的 WebP;其他透明图片依次尝试这些质量的 WebP;其他非透明图片依次尝试这些质量的 JPEG。只有前一个候选超限时才会执行下一个候选;同一尺寸的候选全部超限后才缩小尺寸。已经处于两个规范化上限内的干净、单帧、8-bit sRGB/sRGBA PNG、JPEG 或 WebP 按字节原样直通;16-bit PNG、GIF、动图、元数据、方向和不兼容色彩空间都会触发转换。源图和转换后的附件各完整解码一次。`saveImages` 在发布任何批次成员前为每张图片各准备并验证一次规范化附件,因此校验失败不会留下部分引用,提交阶段也不会重复执行完整图片编码。
请求版本保存在 `<DSH_HOME>/attachments/v1/request-images/``readImageRequest` 在不放大小图的前提下,把存储的规范化附件缩放到总像素预算内,再执行独立的编码字节上限。请求编码器使用同一分类分支:低色数图片先尝试 PNG,只有不带 alpha 通道时才使用 palette,再尝试质量 85 和 80 的 WebP;其他透明图片依次尝试质量 85 和 80 的 WebP;其他非透明图片依次尝试质量 85 和 80 的 JPEG。候选按需执行,两个质量档均超限后才缩小尺寸。缓存身份包含附件 ID、变换策略版本、像素和字节预算及固定编码参数。缓存字节在使用前会完整解码并校验为 8-bit sRGB/sRGBA。同一身份的并发调用共享一次变换和缓存写入;取消一个等待方不会取消共享任务。调用方组合单数读取得到有序批次,服务的 FIFO 限流器通过 `imageCompressionConcurrency` 限制同时执行的规范化和请求变换。该配置范围为 1 至 8,默认值为 2;文件发布仍在准备结束后按顺序执行。
`DSH_HOME` 按共享路径策略解析:显式配置、`$DSH_HOME`,最后是 `~/.dsh`。会话日志只包含引用和经过校验的元数据,绝不包含这个宿主路径。`readImage` 会把可选取消信号传入文件系统读取、在校验前后观察该信号,并保留取消语义,而不会将其包装成 `ATTACHMENT_READ_FAILED`
@@ -12,10 +16,11 @@
#### KV 缓存影响
除发起请求的适配器所持有的图片块外,不产生其他影响
规范化和请求投影都是确定性的。附件和路由策略不变时,之后各轮会复用相同的缓存请求字节
## 已知限制与待完成工作
- 对象会无限期保留;基于引用的垃圾回收尚未实现。
- 本地后端假定宿主与提供方适配器共享同一个文件系统服务。
- 动态 GIF 的元数据根据逻辑屏幕进行校验;逐帧解码策略由提供方持有
- 动态 GIF 源图只保留首帧;动画在版本一图片契约之外
- 规范化和请求版本编码器由安装的 sharp/libvips 构建钉定;编码器或变换策略版本升级会让未来的规范化附件或请求变体产生新地址,已有对象保持有效。
@@ -0,0 +1,43 @@
/** Instance-owned concurrency bound for native image transformations. */
/** FIFO limiter for asynchronous compression work. */
export class CompressionLimiter {
private active = 0
private readonly waiting: Array<() => void> = []
/**
* @param concurrency - positive maximum number of active tasks.
*/
constructor(readonly concurrency: number) {}
/**
* Run one task after an instance slot becomes available.
* @param task - compression operation occupying one slot until settlement.
* @returns the task result.
*/
run<T>(task: () => Promise<T>): Promise<T> {
return new Promise<T>((resolve, reject) => {
const start = (): void => {
this.active += 1
const release = (): void => {
this.active -= 1
this.waiting.shift()?.()
}
void Promise.resolve().then(task).then(
(value) => {
release()
resolve(value)
},
(error: unknown) => {
release()
reject(error instanceof Error
? error
: new Error('Image compression task rejected with a non-Error value.', { cause: error }))
},
)
}
if (this.active < this.concurrency) start()
else this.waiting.push(start)
})
}
}
@@ -0,0 +1,46 @@
/** Shared lazy candidate execution for normalization and request-image encoders. */
/** One encoded candidate carrying its complete bytes. */
export interface EncodedCandidate {
data: Uint8Array
}
/** Result of exhausting candidates at one raster size without a fitting output. */
export interface ExhaustedEncoding<T extends EncodedCandidate> {
smallest: T
}
/**
* Execute encoding candidates in preference order and stop after the first fitting output.
* @param attempts - lazy encoders ordered from preferred to fallback representation.
* @param maxBytes - positive encoded-byte cap.
* @returns the first fitting candidate, otherwise the smallest completed fallback.
*/
export async function encodeFirstWithinLimit<T extends EncodedCandidate>(
attempts: readonly (() => Promise<T>)[],
maxBytes: number,
): Promise<T | ExhaustedEncoding<T>> {
const [first, ...remaining] = attempts
if (first === undefined) throw new Error('image encoding requires at least one candidate')
let smallest = await first()
if (smallest.data.byteLength <= maxBytes) return smallest
for (const attempt of remaining) {
const candidate = await attempt()
if (candidate.data.byteLength <= maxBytes) return candidate
if (candidate.data.byteLength < smallest.data.byteLength) {
smallest = candidate
}
}
return { smallest }
}
/**
* Whether a lazy encoding result exhausted every candidate at one size.
* @param result - first fitting candidate or exhausted result.
* @returns whether every candidate exceeded the byte cap.
*/
export function isExhaustedEncoding<T extends EncodedCandidate>(
result: T | ExhaustedEncoding<T>,
): result is ExhaustedEncoding<T> {
return 'smallest' in result
}
@@ -7,8 +7,38 @@ import type { ImageMediaType } from '@deepseek-ai/dsh-attachment'
/** Decoded metadata from a supported image. */
export interface DetectedImage {
mediaType: ImageMediaType
/** Intrinsic width with EXIF orientation applied — the width a viewer perceives. */
width: number
/** Intrinsic height with EXIF orientation applied — the height a viewer perceives. */
height: number
/** Whether the container carries more than one frame. */
animated: boolean
/** Whether the bytes carry descriptive metadata, a color profile, or orientation. */
carriesMetadata: boolean
/** Sharp sample depth reported for the decoded channels. */
depth: string
/** Sharp colour space reported for the decoded pixels. */
space: string
/** Whether decoded pixels carry an alpha channel. */
hasAlpha: boolean
}
/**
* Check alpha metadata for bytes produced by this package's encoders.
* Sharp/libvips may omit an all-opaque alpha plane from WebP output; every
* other addition or removal indicates that the encoded result is incompatible
* with its source facts.
* @param sourceHasAlpha - whether the source bytes declare an alpha plane, or undefined when the source frame is unspecified.
* @param output - decoded media type and alpha metadata from the encoded result.
* @returns whether the output alpha metadata is compatible with the source.
*/
export function encodedAlphaIsCompatible(
sourceHasAlpha: boolean | undefined,
output: Pick<DetectedImage, 'mediaType' | 'hasAlpha'>,
): boolean {
return sourceHasAlpha === undefined
|| output.hasAlpha === sourceHasAlpha
|| (sourceHasAlpha && !output.hasAlpha && output.mediaType === 'image/webp')
}
const MEDIA_TYPES: Readonly<Record<string, ImageMediaType>> = {
@@ -18,13 +48,36 @@ const MEDIA_TYPES: Readonly<Record<string, ImageMediaType>> = {
gif: 'image/gif',
}
function carriesRetainedMetadata(metadata: Awaited<ReturnType<Sharp['metadata']>>): boolean {
return metadata.exif !== undefined
|| metadata.xmp !== undefined
|| metadata.iptc !== undefined
|| metadata.icc !== undefined
|| metadata.hasProfile
|| metadata.tifftagPhotoshop !== undefined
|| metadata.comments !== undefined
|| metadata.orientation !== undefined
}
async function imageMetadata(image: Sharp): Promise<DetectedImage> {
const metadata = await image.metadata()
const mediaType = MEDIA_TYPES[metadata.format as string]
if (mediaType === undefined) {
throw new AttachmentError('Unsupported or malformed image data.', 'INVALID_IMAGE')
}
return { mediaType, width: metadata.width, height: metadata.height }
// EXIF orientations 5-8 transpose the stored raster; report the perceived
// axes so limits, source facts, and coordinate advice all share them.
const transposed = metadata.orientation !== undefined && metadata.orientation >= 5
return {
mediaType,
width: transposed ? metadata.height : metadata.width,
height: transposed ? metadata.width : metadata.height,
animated: (metadata.pages ?? 1) > 1,
carriesMetadata: carriesRetainedMetadata(metadata),
depth: metadata.depth,
space: metadata.space,
hasAlpha: metadata.hasAlpha,
}
}
/**
+184 -21
View File
@@ -4,43 +4,130 @@ import { join, resolve } from 'node:path'
import { Context } from '@deepseek-ai/cordis'
import z from '@deepseek-ai/schemastery'
import { AttachmentStore } from '@deepseek-ai/dsh-attachment'
import type { ImageAttachmentLimits, ImageAttachmentRef, SaveImageAttachment, StoredImageAttachment } from '@deepseek-ai/dsh-attachment'
import type {
ImageAttachmentLimits,
ImageAttachmentRef,
ImageRequestPolicy,
RequestImageAttachment,
SaveImageAttachment,
StoredImageAttachment,
} from '@deepseek-ai/dsh-attachment'
import { resolveDshHome } from '@deepseek-ai/dsh-home-paths'
import { readImageFile, saveImageFile, validateImageFile } from './store.ts'
import type { NormalizationPolicy } from './normalization.ts'
import { CompressionLimiter } from './compression-limiter.ts'
import { commitPreparedImageFile, prepareImageFile, readImageFile, validateImageFile } from './store.ts'
import { readRequestImageFile, requestImageVariantId } from './request-image.ts'
export { readImageFile, saveImageFile, validateImageFile } from './store.ts'
export { canPassThroughNormalization, normalizeImage } from './normalization.ts'
export type { NormalizedImage, NormalizationPolicy } from './normalization.ts'
export { commitPreparedImageFile, prepareImageFile, readImageFile, saveImageFile, validateImageFile } from './store.ts'
export type { PreparedImageFile } from './store.ts'
export { readRequestImageFile, requestImageDimensions, requestImageVariantId } from './request-image.ts'
/** Default maximum encoded bytes for one image. */
export const DEFAULT_MAX_IMAGE_BYTES = 3.5 * 1024 * 1024
/** Default maximum encoded bytes for one submitted image; oversized sources are refused, not shrunk. */
export const DEFAULT_MAX_IMAGE_BYTES = 20 * 1024 * 1024
/** Default maximum images in one prompt. */
export const DEFAULT_MAX_IMAGES_PER_MESSAGE = 20
/** Default maximum aggregate image bytes in one prompt. */
export const DEFAULT_MAX_MESSAGE_IMAGE_BYTES = 100 * 1024 * 1024
/** Default maximum intrinsic pixels for one image. */
export const DEFAULT_MAX_IMAGE_PIXELS = 40_000_000
export const DEFAULT_MAX_MESSAGE_IMAGE_BYTES = 200 * 1024 * 1024
/** Default maximum intrinsic pixels for one submitted image. */
export const DEFAULT_MAX_IMAGE_PIXELS = 64_000_000
/** Default per-side pixel cap for one submitted image. */
export const DEFAULT_MAX_IMAGE_DIMENSION = 8192
/**
* Default maximum intrinsic width and height for one image. Deployed model
* routes reject any request whose history carries an image with a side above
* 2000px once the request holds many images, and an admitted image rides
* every later request of its session, so admission refuses at the same line
* to keep the durable history streamable.
* Default long-edge target of the stored normalized image. A larger source
* is admitted and downscaled to this edge, so admission bounds what rides
* every later model request without refusing ordinary large sources.
*/
export const DEFAULT_MAX_IMAGE_DIMENSION = 2000
export const DEFAULT_NORMALIZED_IMAGE_MAX_DIMENSION = 2048
/** Default independent safety cap for one stored normalized image. */
export const DEFAULT_NORMALIZED_IMAGE_MAX_BYTES = 4 * 1024 * 1024
/** Conservative default number of simultaneous native image transformations per store. */
export const DEFAULT_IMAGE_COMPRESSION_CONCURRENCY = 2
/** Maximum configurable native image transformations per store. */
export const MAX_IMAGE_COMPRESSION_CONCURRENCY = 8
/** Local attachment backend configuration. */
export interface Config {
/** Explicit harness home; omitted follows `DSH_HOME`, then `~/.dsh`. */
dshHome?: string
/** Maximum encoded bytes accepted for one image. */
/** Maximum encoded bytes accepted for one submitted image. Default: 20 MiB. */
maxImageBytes?: number
/** Maximum image count accepted in one submitted message. */
/** Maximum image count accepted in one submitted message. Default: 20. */
maxImagesPerMessage?: number
/** Maximum aggregate encoded image bytes accepted in one submitted message. */
/** Maximum aggregate encoded image bytes accepted in one submitted message. Default: 200 MiB. */
maxMessageImageBytes?: number
/** Maximum intrinsic width multiplied by height accepted for one image. */
/** Maximum intrinsic width multiplied by height accepted for one submitted image. Default: 64,000,000. */
maxImagePixels?: number
/** Maximum intrinsic width and maximum intrinsic height accepted for one image. */
/** Maximum intrinsic width and maximum intrinsic height accepted for one submitted image. Default: 8192px. */
maxImageDimension?: number
/** Long-edge pixel cap of the stored provider-independent normalized image. */
normalizedImageMaxDimension?: number
/** Encoded-byte safety cap of the stored provider-independent normalized image. */
normalizedImageMaxBytes?: number
/** Maximum simultaneous normalization or request-image transformations in this service instance. */
imageCompressionConcurrency?: number
}
function abortReason(signal: AbortSignal): Error {
const reason: unknown = signal.reason
return reason instanceof Error
? reason
: new Error('Attachment request cancelled with a non-Error reason.', { cause: reason })
}
class SharedRequest<T> {
readonly controller = new AbortController()
readonly promise: Promise<T>
private settled = false
private waiters = 0
constructor(start: (signal: AbortSignal) => Promise<T>) {
this.promise = start(this.controller.signal).finally(() => {
this.settled = true
})
}
wait(signal?: AbortSignal): Promise<T> {
signal?.throwIfAborted()
this.waiters += 1
if (signal === undefined) {
return this.promise.finally(() => {
this.release(false)
})
}
let released = false
const release = (cancelled: boolean): void => {
if (released) return
released = true
this.release(cancelled, signal)
}
return new Promise<T>((resolve, reject) => {
const abort = (): void => {
release(true)
reject(abortReason(signal))
}
signal.addEventListener('abort', abort, { once: true })
void this.promise.then((value) => {
signal.removeEventListener('abort', abort)
release(false)
resolve(value)
}, (error: unknown) => {
signal.removeEventListener('abort', abort)
release(false)
// CompressionLimiter normalizes task rejections before this handler.
// oxlint-disable-next-line typescript/prefer-promise-reject-errors
reject(error)
})
})
}
private release(cancelled: boolean, signal?: AbortSignal): void {
this.waiters -= 1
if (cancelled && this.waiters === 0 && !this.settled && signal !== undefined) {
this.controller.abort(abortReason(signal))
}
}
}
/** Persistent content-addressed local attachment store. */
@@ -52,11 +139,21 @@ export class LocalAttachmentStore extends AttachmentStore {
maxMessageImageBytes: z.number().step(1).min(1).default(DEFAULT_MAX_MESSAGE_IMAGE_BYTES),
maxImagePixels: z.number().step(1).min(1).default(DEFAULT_MAX_IMAGE_PIXELS),
maxImageDimension: z.number().step(1).min(1).default(DEFAULT_MAX_IMAGE_DIMENSION),
normalizedImageMaxDimension: z.number().step(1).min(1).default(DEFAULT_NORMALIZED_IMAGE_MAX_DIMENSION),
normalizedImageMaxBytes: z.number().step(1).min(1).default(DEFAULT_NORMALIZED_IMAGE_MAX_BYTES),
imageCompressionConcurrency: z.number().step(1).min(1).max(MAX_IMAGE_COMPRESSION_CONCURRENCY)
.default(DEFAULT_IMAGE_COMPRESSION_CONCURRENCY),
})
/** Absolute versioned storage root. */
readonly root: string
readonly imageLimits: ImageAttachmentLimits
/** Resolved provider-independent normalization policy. */
readonly normalizationPolicy: Readonly<NormalizationPolicy>
/** Resolved instance-level compression limit. */
readonly imageCompressionConcurrency: number
private readonly compression: CompressionLimiter
private readonly requestInflight = new Map<string, SharedRequest<RequestImageAttachment>>()
constructor(ctx: Context, config: Config) {
super(ctx)
@@ -69,19 +166,85 @@ export class LocalAttachmentStore extends AttachmentStore {
maxImageDimension: config.maxImageDimension ?? DEFAULT_MAX_IMAGE_DIMENSION,
mediaTypes: Object.freeze(['image/png', 'image/jpeg', 'image/webp', 'image/gif'] as const),
})
this.normalizationPolicy = Object.freeze({
maxDimension: config.normalizedImageMaxDimension ?? DEFAULT_NORMALIZED_IMAGE_MAX_DIMENSION,
maxBytes: config.normalizedImageMaxBytes ?? DEFAULT_NORMALIZED_IMAGE_MAX_BYTES,
})
const compressionConcurrency = config.imageCompressionConcurrency ?? DEFAULT_IMAGE_COMPRESSION_CONCURRENCY
if (!Number.isSafeInteger(compressionConcurrency)
|| compressionConcurrency < 1
|| compressionConcurrency > MAX_IMAGE_COMPRESSION_CONCURRENCY) {
throw new Error(
`attachment-local: imageCompressionConcurrency must be an integer from 1 through ${MAX_IMAGE_COMPRESSION_CONCURRENCY}`,
)
}
this.imageCompressionConcurrency = compressionConcurrency
this.compression = new CompressionLimiter(compressionConcurrency)
}
async validateImage(input: SaveImageAttachment): Promise<void> {
await validateImageFile(input, this.imageLimits)
await this.compression.run(() => validateImageFile(input, this.imageLimits, this.normalizationPolicy))
}
override async saveImages(inputs: readonly SaveImageAttachment[]): Promise<readonly ImageAttachmentRef[]> {
this.validateImageBatch(inputs)
const prepared = await Promise.all(inputs.map(input => this.compression.run(
() => prepareImageFile(input, this.imageLimits, this.normalizationPolicy),
)))
const refs: ImageAttachmentRef[] = []
for (const image of prepared) refs.push(await commitPreparedImageFile(this.root, image))
return refs
}
async saveImage(input: SaveImageAttachment): Promise<ImageAttachmentRef> {
return saveImageFile(this.root, input, this.imageLimits)
const prepared = await this.compression.run(
() => prepareImageFile(input, this.imageLimits, this.normalizationPolicy),
)
return commitPreparedImageFile(this.root, prepared)
}
async readImage(ref: ImageAttachmentRef, signal?: AbortSignal): Promise<StoredImageAttachment> {
return readImageFile(this.root, ref, signal)
}
override async readImageRequest(
ref: ImageAttachmentRef,
policy: ImageRequestPolicy,
signal?: AbortSignal,
): Promise<RequestImageAttachment> {
return this.requestVersion(ref, policy, undefined, signal)
}
private requestVersion(
ref: ImageAttachmentRef,
policy: ImageRequestPolicy,
stored: StoredImageAttachment | undefined,
signal: AbortSignal | undefined,
): Promise<RequestImageAttachment> {
signal?.throwIfAborted()
const variantId = requestImageVariantId(ref, policy)
const key = String(variantId)
let operation = this.requestInflight.get(key)
if (operation?.controller.signal.aborted) {
this.requestInflight.delete(key)
operation = undefined
}
if (operation === undefined) {
const shared = new SharedRequest<RequestImageAttachment>(sharedSignal => this.compression.run(async () => readRequestImageFile(
this.root,
stored ?? await this.readImage(ref, sharedSignal),
policy,
sharedSignal,
)))
operation = shared
this.requestInflight.set(key, shared)
void shared.promise.finally(() => {
if (this.requestInflight.get(key) === shared) this.requestInflight.delete(key)
}).catch(() => {})
}
return operation.wait(signal)
}
}
export default LocalAttachmentStore
@@ -0,0 +1,206 @@
/** Deterministic provider-independent image normalization. */
import sharp, { type Sharp } from 'sharp'
import { AttachmentError } from '@deepseek-ai/dsh-attachment'
import type { ImageMediaType } from '@deepseek-ai/dsh-attachment'
import { encodeFirstWithinLimit, isExhaustedEncoding } from './encoding.ts'
import { detectImage, encodedAlphaIsCompatible } from './image.ts'
import type { DetectedImage } from './image.ts'
/** Deployment-resolved policy for the persisted normalized attachment. */
export interface NormalizationPolicy {
/** Long-edge cap in pixels; larger sources are downscaled proportionally. */
maxDimension: number
/** Independent safety cap for encoded normalized image bytes. */
maxBytes: number
}
/** Normalized bytes beside the facts recorded by a durable reference. */
export interface NormalizedImage {
data: Uint8Array
mediaType: ImageMediaType
width: number
height: number
}
const NORMALIZATION_QUALITIES = [85, 80, 75] as const
const LOW_COLOUR_SAMPLE_EDGE = 128
const LOW_COLOUR_LIMIT = 256
const MIN_SCALE_STEP = 0.9
/** Encode one prepared pipeline and report exact output facts. */
async function encode(
pipeline: Sharp,
mediaType: 'image/png' | 'image/jpeg' | 'image/webp',
quality?: number,
palette = true,
): Promise<NormalizedImage> {
const encoded = mediaType === 'image/png'
? pipeline.png({ compressionLevel: 9, palette })
: mediaType === 'image/webp'
? pipeline.webp({ quality })
: pipeline.jpeg({ quality })
const { data, info } = await encoded.toBuffer({ resolveWithObject: true })
return { data: new Uint8Array(data), mediaType, width: info.width, height: info.height }
}
/**
* Whether bytes already satisfy the normalization requirements.
* @param detected - fully decoded source facts.
* @param bytes - encoded source length.
* @param policy - resolved normalization limits.
* @returns whether the source can pass through byte-identically.
*/
export function canPassThroughNormalization(
detected: DetectedImage,
bytes: number,
policy: NormalizationPolicy,
): boolean {
return detected.mediaType !== 'image/gif'
&& !detected.animated
&& !detected.carriesMetadata
&& detected.depth === 'uchar'
&& detected.space === 'srgb'
&& bytes <= policy.maxBytes
&& Math.max(detected.width, detected.height) <= policy.maxDimension
}
/**
* Classify a bounded pixel sample without assuming that a PNG source is a screenshot.
* @param pipeline - oriented sRGB source pipeline before output resizing.
* @returns whether the nearest-neighbour sample stays within the low-color threshold.
*/
export async function hasLowColourCount(pipeline: Sharp): Promise<boolean> {
const { data, info } = await pipeline.clone().resize({
width: LOW_COLOUR_SAMPLE_EDGE,
height: LOW_COLOUR_SAMPLE_EDGE,
fit: 'inside',
withoutEnlargement: true,
kernel: sharp.kernel.nearest,
fastShrinkOnLoad: false,
}).raw().toBuffer({ resolveWithObject: true })
const colours = new Set<number>()
for (let offset = 0; offset < data.length; offset += info.channels) {
const red = data.readUInt8(offset)
const green = data.readUInt8(offset + 1)
const blue = data.readUInt8(offset + 2)
const alpha = info.channels === 4 ? data.readUInt8(offset + 3) : 255
colours.add(((red >> 3) << 15) | ((green >> 3) << 10) | ((blue >> 3) << 5) | (alpha >> 3))
if (colours.size > LOW_COLOUR_LIMIT) return false
}
return true
}
/** Assert that a normalized output is an 8-bit sRGB/sRGBA single-frame image with matching facts. */
async function verifyNormalizedImage(
image: NormalizedImage,
expectedAlpha: boolean | undefined,
): Promise<NormalizedImage> {
const detected = await detectImage(image.data)
if (detected.mediaType !== image.mediaType
|| detected.width !== image.width
|| detected.height !== image.height
|| detected.animated
|| detected.carriesMetadata
|| detected.depth !== 'uchar'
|| detected.space !== 'srgb'
|| !encodedAlphaIsCompatible(expectedAlpha, detected)) {
throw new AttachmentError(
'Image normalization did not produce a single-frame 8-bit sRGB image with matching metadata.',
'ATTACHMENT_WRITE_FAILED',
)
}
return image
}
/** Build one fixed-size, oriented, metadata-free sRGB pipeline from submitted bytes. */
function preparedPipeline(data: Uint8Array, width: number, height: number): Sharp {
return sharp(data, { failOn: 'error', limitInputPixels: false })
.rotate()
.toColourspace('srgb')
.resize({ width, height, fit: 'inside', withoutEnlargement: true })
}
/** Dimensions after the long edge is capped without changing aspect ratio. */
function initialDimensions(detected: DetectedImage, maxDimension: number): { width: number; height: number } {
const scale = Math.min(1, maxDimension / Math.max(detected.width, detected.height))
return {
width: Math.max(1, Math.round(detected.width * scale)),
height: Math.max(1, Math.round(detected.height * scale)),
}
}
/** Lazy encoding order for one size, separated by sampled colour complexity and alpha. */
function encodingAttemptsAtSize(
data: Uint8Array,
width: number,
height: number,
hasAlpha: boolean,
lowColour: boolean,
): Array<() => Promise<NormalizedImage>> {
const prepared = preparedPipeline(data, width, height)
const webp = NORMALIZATION_QUALITIES.map(quality => (
() => encode(prepared.clone(), 'image/webp', quality)
))
if (lowColour) {
return [() => encode(prepared.clone(), 'image/png', undefined, !hasAlpha), ...webp]
}
if (hasAlpha) return webp
return NORMALIZATION_QUALITIES.map(quality => (
() => encode(prepared.clone(), 'image/jpeg', quality)
))
}
/**
* Produce the persisted provider-independent normalized version of one fully decoded source.
* The source is passed through only when it is already clean, single-frame, 8-bit sRGB/sRGBA,
* and inside both normalization limits. Re-encoding never removes transparency. After the fixed
* quality floor is reached, dimensions continue shrinking until the independent byte cap holds.
* @param data - complete admitted source bytes.
* @param detected - fully decoded source facts.
* @param policy - resolved independent normalization limits.
* @returns verified provider-independent normalized bytes and metadata.
*/
export async function normalizeImage(
data: Uint8Array,
detected: DetectedImage,
policy: NormalizationPolicy,
): Promise<NormalizedImage> {
if (canPassThroughNormalization(detected, data.byteLength, policy)) {
return { data, mediaType: detected.mediaType, width: detected.width, height: detected.height }
}
try {
let { width, height } = initialDimensions(detected, policy.maxDimension)
const classificationPipeline = sharp(data, { failOn: 'error', limitInputPixels: false })
.rotate()
.toColourspace('srgb')
const lowColour = await hasLowColourCount(classificationPipeline)
for (;;) {
const encoded = await encodeFirstWithinLimit(
encodingAttemptsAtSize(data, width, height, detected.hasAlpha, lowColour),
policy.maxBytes,
)
if (!isExhaustedEncoding(encoded)) {
return await verifyNormalizedImage(encoded, detected.mediaType === 'image/gif' ? undefined : detected.hasAlpha)
}
if (width === 1 && height === 1) break
const sizeScale = Math.sqrt(policy.maxBytes / encoded.smallest.data.byteLength) * 0.95
const scale = Math.min(MIN_SCALE_STEP, sizeScale)
const nextWidth = Math.max(1, Math.floor(width * scale))
const nextHeight = Math.max(1, Math.floor(height * scale))
width = nextWidth
height = nextHeight
}
} catch (error) {
if (error instanceof AttachmentError) throw error
const source = detected.mediaType === 'image/png' && detected.depth !== 'uchar'
? `${detected.depth === 'ushort' ? '16-bit' : detected.depth} PNG`
: `${detected.depth} ${detected.mediaType.slice('image/'.length).toUpperCase()}`
throw new AttachmentError(
`The ${source} could not be converted to the normalized 8-bit sRGB form.`,
'ATTACHMENT_WRITE_FAILED',
{ cause: error },
)
}
throw new AttachmentError('Image cannot be encoded within the configured normalized-image byte cap.', 'IMAGE_TOO_LARGE')
}
@@ -0,0 +1,279 @@
/** Deterministic cached image versions for model requests. */
import { createHash, randomUUID } from 'node:crypto'
import { mkdir, readFile, rename, rm, writeFile } from 'node:fs/promises'
import { dirname, join } from 'node:path'
import sharp, { type Sharp } from 'sharp'
import { AttachmentError, ImageVariantId } from '@deepseek-ai/dsh-attachment'
import type {
ImageMediaType,
ImageAttachmentRef,
ImageRequestPolicy,
RequestImageAttachment,
StoredImageAttachment,
} from '@deepseek-ai/dsh-attachment'
import { hasLowColourCount } from './normalization.ts'
import { encodeFirstWithinLimit, isExhaustedEncoding } from './encoding.ts'
import { detectImage, encodedAlphaIsCompatible, probeImage } from './image.ts'
/** Transform version included in every cache and upload-index identity. */
export const REQUEST_IMAGE_TRANSFORM_VERSION = 'request-image-v4'
/** DeepSeek request versions normally fit at these two preferred qualities. */
export const REQUEST_IMAGE_QUALITIES = [85, 80] as const
interface EncodedRequestImage {
data: Uint8Array
mediaType: ImageMediaType
width: number
height: number
}
interface VerifiedRequestImage extends EncodedRequestImage {
hasAlpha: boolean
}
function digest(value: string | Uint8Array): string {
return createHash('sha256').update(value).digest('hex')
}
/**
* Compute aspect-preserving integer dimensions within a hard total-pixel budget.
* @param width - positive source width.
* @param height - positive source height.
* @param maxPixels - positive width-times-height cap.
* @returns inward-rounded dimensions; small images are not enlarged.
*/
export function requestImageDimensions(
width: number,
height: number,
maxPixels: number,
): { width: number; height: number } {
const scale = Math.min(1, Math.sqrt(maxPixels / (width * height)))
if (scale === 1) return { width, height }
if (width >= height) {
let projectedWidth = Math.max(1, Math.floor(width * scale))
let projectedHeight = Math.max(1, Math.round(projectedWidth * height / width))
while (projectedWidth * projectedHeight > maxPixels && projectedWidth > 1) {
projectedWidth -= 1
projectedHeight = Math.max(1, Math.round(projectedWidth * height / width))
}
return { width: projectedWidth, height: projectedHeight }
}
let projectedHeight = Math.max(1, Math.floor(height * scale))
let projectedWidth = Math.max(1, Math.round(projectedHeight * width / height))
while (projectedWidth * projectedHeight > maxPixels && projectedHeight > 1) {
projectedHeight -= 1
projectedWidth = Math.max(1, Math.round(projectedHeight * width / height))
}
return { width: projectedWidth, height: projectedHeight }
}
function checkedInteger(value: number, name: string): number {
if (!Number.isSafeInteger(value) || value <= 0) {
throw new AttachmentError(`${name} must be a positive integer.`, 'INVALID_ATTACHMENT_REF')
}
return value
}
function validatePolicy(policy: ImageRequestPolicy): void {
checkedInteger(policy.maxPixels, 'Image request maxPixels')
checkedInteger(policy.maxBytes, 'Image request maxBytes')
}
function descriptor(attachment: ImageAttachmentRef, policy: ImageRequestPolicy): string {
return JSON.stringify({
transformVersion: REQUEST_IMAGE_TRANSFORM_VERSION,
attachmentId: attachment.attachmentId,
routePixelBudget: policy.maxPixels,
encodedByteBudget: policy.maxBytes,
encoding: {
png: { compressionLevel: 9, palette: 'opaque-only' },
webpQualities: REQUEST_IMAGE_QUALITIES,
jpegQualities: REQUEST_IMAGE_QUALITIES,
order: ['low-colour:png-webp', 'alpha:webp', 'opaque:jpeg'],
colourspace: 'srgb',
},
})
}
/**
* Complete deterministic identity for one attachment and route-owned request policy.
* @param attachment - provider-independent durable normalized attachment reference.
* @param policy - route-owned pixel and byte policy.
* @returns branded digest over every request transform input.
*/
export function requestImageVariantId(
attachment: ImageAttachmentRef,
policy: ImageRequestPolicy,
): ReturnType<typeof ImageVariantId> {
return ImageVariantId(`sha256:${digest(descriptor(attachment, policy))}`)
}
function pipeline(attachment: StoredImageAttachment, width: number, height: number): Sharp {
return sourcePipeline(attachment)
.resize({ width, height, fit: 'inside', withoutEnlargement: true })
}
function sourcePipeline(attachment: StoredImageAttachment): Sharp {
return sharp(attachment.data, { failOn: 'error', limitInputPixels: false }).toColourspace('srgb')
}
async function encoded(
image: Sharp,
mediaType: 'image/png' | 'image/jpeg' | 'image/webp',
quality?: number,
palette = true,
): Promise<EncodedRequestImage> {
const output = mediaType === 'image/png'
? image.png({ compressionLevel: 9, palette })
: mediaType === 'image/webp'
? image.webp({ quality })
: image.jpeg({ quality })
const { data, info } = await output.toBuffer({ resolveWithObject: true })
return { data: new Uint8Array(data), mediaType, width: info.width, height: info.height }
}
function encodingAttempts(
attachment: StoredImageAttachment,
width: number,
height: number,
hasAlpha: boolean,
lowColour: boolean,
): Array<() => Promise<EncodedRequestImage>> {
const prepared = pipeline(attachment, width, height)
const webp = REQUEST_IMAGE_QUALITIES.map(quality => (
() => encoded(prepared.clone(), 'image/webp', quality)
))
if (lowColour) return [() => encoded(prepared.clone(), 'image/png', undefined, !hasAlpha), ...webp]
if (hasAlpha) return webp
return REQUEST_IMAGE_QUALITIES.map(quality => (
() => encoded(prepared.clone(), 'image/jpeg', quality)
))
}
async function createRequestImage(
attachment: StoredImageAttachment,
policy: ImageRequestPolicy,
hasAlpha: boolean,
): Promise<EncodedRequestImage> {
let dimensions = requestImageDimensions(attachment.ref.width, attachment.ref.height, policy.maxPixels)
if (dimensions.width === attachment.ref.width
&& dimensions.height === attachment.ref.height
&& attachment.data.byteLength <= policy.maxBytes) {
return {
data: attachment.data,
mediaType: attachment.ref.mediaType,
width: attachment.ref.width,
height: attachment.ref.height,
}
}
const lowColour = await hasLowColourCount(sourcePipeline(attachment))
for (;;) {
const encodedVersion = await encodeFirstWithinLimit(
encodingAttempts(attachment, dimensions.width, dimensions.height, hasAlpha, lowColour),
policy.maxBytes,
)
if (!isExhaustedEncoding(encodedVersion)) return encodedVersion
if (dimensions.width === 1 && dimensions.height === 1) break
const scale = Math.min(0.9, Math.sqrt(policy.maxBytes / encodedVersion.smallest.data.byteLength) * 0.95)
dimensions = {
width: Math.max(1, Math.floor(dimensions.width * scale)),
height: Math.max(1, Math.floor(dimensions.height * scale)),
}
}
throw new AttachmentError('Image cannot be encoded within the model-request byte budget.', 'IMAGE_TOO_LARGE')
}
function cachePath(root: string, hash: string): string {
return join(root, 'request-images', hash.slice(0, 2), hash)
}
async function readCached(
path: string,
attachment: StoredImageAttachment,
policy: ImageRequestPolicy,
expectedAlpha: boolean,
signal?: AbortSignal,
): Promise<VerifiedRequestImage | undefined> {
try {
const data = new Uint8Array(await readFile(path, { signal }))
const detected = await probeImage(data)
const maximum = requestImageDimensions(attachment.ref.width, attachment.ref.height, policy.maxPixels)
if (data.byteLength > policy.maxBytes || detected.depth !== 'uchar' || detected.space !== 'srgb'
|| detected.width > maximum.width || detected.height > maximum.height
|| !encodedAlphaIsCompatible(expectedAlpha, detected)) return undefined
return { data, mediaType: detected.mediaType, width: detected.width, height: detected.height, hasAlpha: detected.hasAlpha }
} catch (error: unknown) {
if ((error as NodeJS.ErrnoException | null)?.code === 'ENOENT') return undefined
signal?.throwIfAborted()
return undefined
}
}
async function verifyRequestImage(
image: EncodedRequestImage,
expectedAlpha: boolean,
): Promise<VerifiedRequestImage> {
const detected = await detectImage(image.data)
if (detected.depth !== 'uchar' || detected.space !== 'srgb'
|| detected.width !== image.width || detected.height !== image.height
|| detected.mediaType !== image.mediaType || !encodedAlphaIsCompatible(expectedAlpha, detected)) {
throw new AttachmentError(
'Encoded model-request image does not match its verified 8-bit sRGB metadata.',
'ATTACHMENT_WRITE_FAILED',
)
}
return { ...image, hasAlpha: detected.hasAlpha }
}
async function writeCached(path: string, data: Uint8Array): Promise<void> {
await mkdir(dirname(path), { recursive: true, mode: 0o700 })
const temporary = `${path}.${randomUUID()}.tmp`
try {
await writeFile(temporary, data, { mode: 0o600, flag: 'wx' })
await rename(temporary, path)
} finally {
await rm(temporary, { force: true })
}
}
/**
* Generate or reuse one request image below the local attachment root.
* @param root - absolute versioned attachment storage root.
* @param attachment - verified normalized attachment bytes and reference.
* @param policy - exact route request-image policy.
* @param signal - optional cancellation for cache I/O and image transformation.
* @returns verified request bytes and deterministic variant identity.
*/
export async function readRequestImageFile(
root: string,
attachment: StoredImageAttachment,
policy: ImageRequestPolicy,
signal?: AbortSignal,
): Promise<RequestImageAttachment> {
signal?.throwIfAborted()
validatePolicy(policy)
const source = await probeImage(attachment.data)
const variantId = requestImageVariantId(attachment.ref, policy)
const hash = String(variantId).slice('sha256:'.length)
const path = cachePath(root, hash)
const cached = await readCached(path, attachment, policy, source.hasAlpha, signal)
const created = cached ?? await createRequestImage(attachment, policy, source.hasAlpha)
const version = cached ?? (created.data === attachment.data
? { ...created, hasAlpha: source.hasAlpha }
: await verifyRequestImage(created, source.hasAlpha))
signal?.throwIfAborted()
if (cached === undefined && version.data !== attachment.data) await writeCached(path, version.data)
return {
variantId,
attachment: attachment.ref,
data: version.data,
mediaType: version.mediaType,
bytes: version.data.byteLength,
width: version.width,
height: version.height,
depth: 'uchar',
space: 'srgb',
hasAlpha: version.hasAlpha,
}
}
@@ -14,7 +14,10 @@ import type {
SaveImageAttachment,
StoredImageAttachment,
} from '@deepseek-ai/dsh-attachment'
import { normalizeImage } from './normalization.ts'
import type { NormalizationPolicy } from './normalization.ts'
import { detectImage, probeImage } from './image.ts'
import type { DetectedImage } from './image.ts'
const ID_PATTERN = /^sha256:([a-f0-9]{64})$/
const durableHomes = new Set<string>()
@@ -47,24 +50,70 @@ async function inspectMetadata(
data: Uint8Array,
declaredMediaType: ImageAttachmentRef['mediaType'],
limits: ImageAttachmentLimits,
): Promise<Omit<ImageAttachmentRef, 'attachmentId' | 'name'>> {
): Promise<DetectedImage> {
if (data.byteLength === 0) throw new AttachmentError('Image is empty.', 'INVALID_IMAGE')
const detected = await detectImage(data, { maxPixels: limits.maxImagePixels, maxDimension: limits.maxImageDimension })
if (detected.mediaType !== declaredMediaType) throw new AttachmentError('Declared image type does not match its bytes.', 'IMAGE_TYPE_MISMATCH')
return { ...detected, bytes: data.byteLength }
return detected
}
/**
* Run the full admission policy for one image without touching storage.
* Run the full admission policy for one image without touching storage,
* including normalization: a batch whose members all validate cannot later
* be refused by the normalized image byte cap during publication.
* @param input - encoded bytes and declared metadata.
* @param limits - resolved storage policy.
* @returns completion after the encoded raster has been fully decoded.
* @param limits - resolved source admission policy.
* @param policy - resolved normalization policy.
* @returns completion after the raster has been decoded and its normalized version proven to fit.
*/
export async function validateImageFile(input: SaveImageAttachment, limits: ImageAttachmentLimits): Promise<void> {
export async function validateImageFile(
input: SaveImageAttachment,
limits: ImageAttachmentLimits,
policy: NormalizationPolicy,
): Promise<void> {
await prepareImageFile(input, limits, policy)
}
/** Fully prepared normalized object, verified before any batch member is persisted. */
export interface PreparedImageFile {
/** Deterministic normalized bytes whose digest is {@link ref.attachmentId}. */
data: Uint8Array
/** Durable reference describing {@link data}. */
ref: ImageAttachmentRef
}
/**
* Decode, normalize, and verify one submitted image without touching storage.
* @param input - submitted encoded bytes and declared media type.
* @param limits - source admission policy.
* @param policy - independent normalization policy.
* @returns immutable reference facts beside bytes ready for atomic publication.
*/
export async function prepareImageFile(
input: SaveImageAttachment,
limits: ImageAttachmentLimits,
policy: NormalizationPolicy,
): Promise<PreparedImageFile> {
if (input.data.byteLength > limits.maxImageBytes) {
throw new AttachmentError('Image exceeds the configured byte limit.', 'IMAGE_TOO_LARGE')
}
await inspectMetadata(input.data, input.mediaType, limits)
const detected = await inspectMetadata(input.data, input.mediaType, limits)
const normalized = await normalizeImage(input.data, detected, policy)
const sha256 = digest(normalized.data)
const name = displayName(input.name)
const downscaled = detected.width !== normalized.width || detected.height !== normalized.height
return {
data: normalized.data,
ref: {
attachmentId: AttachmentId(`sha256:${sha256}`),
mediaType: normalized.mediaType,
width: normalized.width,
height: normalized.height,
bytes: normalized.data.byteLength,
...(name !== undefined ? { name } : {}),
...downscaled ? { originalDimensions: { width: detected.width, height: detected.height } } : {},
},
}
}
/**
@@ -127,16 +176,20 @@ async function ensureDurableHome(path: string): Promise<string> {
}
/**
* Save and verify immutable image bytes below a versioned attachment root.
* Publish one already verified normalized image below a versioned attachment root.
* @param root - absolute `DSH_HOME/attachments/v1` root.
* @param input - encoded bytes and declared metadata.
* @param limits - resolved storage policy.
* @returns durable content-addressed reference.
* @param prepared - deterministic normalized bytes and reference.
* @returns durable content-addressed normalized image reference.
*/
export async function saveImageFile(root: string, input: SaveImageAttachment, limits: ImageAttachmentLimits): Promise<ImageAttachmentRef> {
if (input.data.byteLength > limits.maxImageBytes) throw new AttachmentError('Image exceeds the configured byte limit.', 'IMAGE_TOO_LARGE')
const metadata = await inspectMetadata(input.data, input.mediaType, limits)
const sha256 = digest(input.data)
export async function commitPreparedImageFile(
root: string,
prepared: PreparedImageFile,
): Promise<ImageAttachmentRef> {
const normalized = prepared.data
const sha256 = ensureReference(prepared.ref)
if (digest(normalized) !== sha256 || normalized.byteLength !== prepared.ref.bytes) {
throw new AttachmentError('Prepared attachment bytes do not match their reference.', 'ATTACHMENT_CORRUPT')
}
const bucket = join(root, 'objects', sha256.slice(0, 2))
const staging = join(root, 'tmp')
// Establish DSH_HOME itself against the filesystem root once per process.
@@ -150,7 +203,7 @@ export async function saveImageFile(root: string, input: SaveImageAttachment, li
let handle
try {
handle = await open(temporary, constants.O_CREAT | constants.O_EXCL | constants.O_WRONLY, 0o600)
await handle.writeFile(input.data)
await handle.writeFile(normalized)
await handle.sync()
await handle.close()
handle = undefined
@@ -185,12 +238,24 @@ export async function saveImageFile(root: string, input: SaveImageAttachment, li
if (error instanceof AttachmentError) throw error
throw new AttachmentError('Unable to persist image attachment.', 'ATTACHMENT_WRITE_FAILED', { cause: error })
}
const name = displayName(input.name)
return {
attachmentId: AttachmentId(`sha256:${sha256}`),
...metadata,
...(name !== undefined ? { name } : {}),
}
return prepared.ref
}
/**
* Decode and normalize one image once, then publish the prepared object.
* @param root - absolute `DSH_HOME/attachments/v1` root.
* @param input - submitted encoded bytes and declared media type.
* @param limits - resolved source admission policy.
* @param policy - resolved normalization policy.
* @returns durable content-addressed normalized image reference.
*/
export async function saveImageFile(
root: string,
input: SaveImageAttachment,
limits: ImageAttachmentLimits,
policy: NormalizationPolicy,
): Promise<ImageAttachmentRef> {
return commitPreparedImageFile(root, await prepareImageFile(input, limits, policy))
}
/**
@@ -0,0 +1,96 @@
import { describe, expect, it, vi } from 'vitest'
import { CompressionLimiter } from '../src/compression-limiter.ts'
import { encodeFirstWithinLimit, isExhaustedEncoding } from '../src/encoding.ts'
describe('lazy image encoding', () => {
it('does not execute fallback qualities after the first fitting candidate', async () => {
const first = vi.fn(() => Promise.resolve({ data: new Uint8Array(8), quality: 85 }))
const fallback = vi.fn(() => Promise.resolve({ data: new Uint8Array(4), quality: 80 }))
await expect(encodeFirstWithinLimit([first, fallback], 8)).resolves.toMatchObject({ quality: 85 })
expect(first).toHaveBeenCalledTimes(1)
expect(fallback).not.toHaveBeenCalled()
})
it('executes later candidates only after earlier candidates exceed the cap', async () => {
const first = vi.fn(() => Promise.resolve({ data: new Uint8Array(12), quality: 85 }))
const second = vi.fn(() => Promise.resolve({ data: new Uint8Array(7), quality: 80 }))
const third = vi.fn(() => Promise.resolve({ data: new Uint8Array(5), quality: 75 }))
await expect(encodeFirstWithinLimit([first, second, third], 8)).resolves.toMatchObject({ quality: 80 })
expect(first).toHaveBeenCalledTimes(1)
expect(second).toHaveBeenCalledTimes(1)
expect(third).not.toHaveBeenCalled()
})
it('rejects an empty candidate list and reports the smallest exhausted candidate', async () => {
await expect(encodeFirstWithinLimit([], 8)).rejects.toThrow('requires at least one candidate')
const result = await encodeFirstWithinLimit([
() => Promise.resolve({ data: new Uint8Array(12), quality: 85 }),
() => Promise.resolve({ data: new Uint8Array(9), quality: 80 }),
() => Promise.resolve({ data: new Uint8Array(10), quality: 75 }),
], 8)
expect(isExhaustedEncoding(result)).toBe(true)
expect(result).toMatchObject({ smallest: { quality: 80 } })
expect(isExhaustedEncoding({ data: new Uint8Array(1) })).toBe(false)
})
})
describe('CompressionLimiter', () => {
it('starts at most the configured number of tasks and preserves queued progress', async () => {
const limiter = new CompressionLimiter(2)
const gates = Array.from({ length: 4 }, () => Promise.withResolvers<undefined>())
let active = 0
let maximum = 0
const started: number[] = []
const tasks = gates.map((gate, index) => limiter.run(async () => {
active += 1
maximum = Math.max(maximum, active)
started.push(index)
await gate.promise
active -= 1
return index
}))
await Promise.resolve()
expect(started).toEqual([0, 1])
gates[0]!.resolve(undefined)
await tasks[0]
await Promise.resolve()
expect(started).toEqual([0, 1, 2])
gates[1]!.resolve(undefined)
gates[2]!.resolve(undefined)
await Promise.all([tasks[1], tasks[2]])
await Promise.resolve()
expect(started).toEqual([0, 1, 2, 3])
gates[3]!.resolve(undefined)
await expect(Promise.all(tasks)).resolves.toEqual([0, 1, 2, 3])
expect(maximum).toBe(2)
})
it('releases a slot when a task throws before returning a promise', async () => {
const limiter = new CompressionLimiter(1)
const failed = limiter.run(() => {
throw new Error('synchronous setup failure')
})
const next = limiter.run(() => Promise.resolve('next'))
await expect(failed).rejects.toThrow('synchronous setup failure')
await expect(next).resolves.toBe('next')
})
it('normalizes a non-Error rejection and releases its slot', async () => {
const limiter = new CompressionLimiter(1)
// oxlint-disable-next-line typescript/prefer-promise-reject-errors -- Native bindings can reject non-Error values.
const failed = limiter.run(() => Promise.reject('native failure'))
const next = limiter.run(() => Promise.resolve('next'))
await expect(failed).rejects.toMatchObject({
message: 'Image compression task rejected with a non-Error value.',
cause: 'native failure',
})
await expect(next).resolves.toBe('next')
})
})
@@ -18,7 +18,7 @@ describe('raster decoding', () => {
['gif', 'image/gif'],
] as const) {
await expect(detectImage(await raster(format)))
.resolves.toEqual({ mediaType, width: 3, height: 2 })
.resolves.toMatchObject({ mediaType, width: 3, height: 2, animated: false, carriesMetadata: false, depth: 'uchar', space: 'srgb' })
}
})
@@ -31,7 +31,7 @@ describe('raster decoding', () => {
await expect(detectImage(await raster('png'), { maxDimension: 2 }))
.rejects.toMatchObject({ code: 'IMAGE_DIMENSION_TOO_LARGE' })
await expect(detectImage(await raster('png'), { maxDimension: 3 }))
.resolves.toEqual({ mediaType: 'image/png', width: 3, height: 2 })
.resolves.toMatchObject({ mediaType: 'image/png', width: 3, height: 2, animated: false, carriesMetadata: false, depth: 'uchar', space: 'srgb' })
})
it('rejects malformed bytes and truncated payloads with readable headers', async () => {
@@ -47,6 +47,39 @@ describe('raster decoding', () => {
await expect(detectImage(truncated)).rejects.toMatchObject({ code: 'INVALID_IMAGE' })
})
it('reports animation from a multi-frame container and perceived axes from EXIF orientation', async () => {
const header = Buffer.from('47494638396101000100800000000000ffffff', 'hex')
const frame = Buffer.from('21f90401000000002c0000000001000100000202440100', 'hex')
const twoFrameGif = Uint8Array.from(Buffer.concat([header, frame, frame, Buffer.from('3b', 'hex')]))
await expect(detectImage(twoFrameGif)).resolves.toMatchObject({ mediaType: 'image/gif', animated: true })
const oriented = new Uint8Array(await sharp({
create: { width: 4, height: 2, channels: 3, background: { r: 1, g: 2, b: 3 } },
}).jpeg().withMetadata({ orientation: 6 }).toBuffer())
await expect(detectImage(oriented)).resolves.toMatchObject({
mediaType: 'image/jpeg', width: 2, height: 4, animated: false, carriesMetadata: true,
})
const flipped = new Uint8Array(await sharp({
create: { width: 4, height: 2, channels: 3, background: { r: 1, g: 2, b: 3 } },
}).jpeg().withMetadata({ orientation: 3 }).toBuffer())
await expect(detectImage(flipped)).resolves.toMatchObject({
mediaType: 'image/jpeg', width: 4, height: 2, animated: false, carriesMetadata: true,
})
})
it('reports color profiles and encoder metadata as metadata', async () => {
const profiled = new Uint8Array(await sharp({
create: { width: 4, height: 2, channels: 3, background: { r: 1, g: 2, b: 3 } },
}).png().withIccProfile('p3').toBuffer())
await expect(detectImage(profiled)).resolves.toMatchObject({ carriesMetadata: true })
const commented = new Uint8Array(await sharp({
create: { width: 4, height: 2, channels: 3, background: { r: 1, g: 2, b: 3 } },
}).png().withMetadata().toBuffer())
await expect(detectImage(commented)).resolves.toMatchObject({ carriesMetadata: true })
})
it('probes malformed bytes and unsupported formats into the same stable error', async () => {
await expect(probeImage(Uint8Array.of(1, 2, 3)))
.rejects.toMatchObject({ code: 'INVALID_IMAGE' })
@@ -4,7 +4,11 @@ import { mkdtemp, rm } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { describe, expect, it } from 'vitest'
import sharp from 'sharp'
import LocalAttachmentStore, {
DEFAULT_NORMALIZED_IMAGE_MAX_BYTES,
DEFAULT_NORMALIZED_IMAGE_MAX_DIMENSION,
DEFAULT_IMAGE_COMPRESSION_CONCURRENCY,
DEFAULT_MAX_IMAGE_BYTES,
DEFAULT_MAX_IMAGE_DIMENSION,
DEFAULT_MAX_IMAGE_PIXELS,
@@ -15,7 +19,11 @@ import LocalAttachmentStore, {
describe('local attachment service', () => {
it('resolves every omitted admission limit explicitly', () => {
const service = new LocalAttachmentStore(new Context(), {})
expect(DEFAULT_MAX_IMAGE_BYTES).toBe(3.5 * 1024 * 1024)
expect(DEFAULT_MAX_IMAGE_BYTES).toBe(20 * 1024 * 1024)
expect(DEFAULT_MAX_IMAGES_PER_MESSAGE).toBe(20)
expect(DEFAULT_MAX_MESSAGE_IMAGE_BYTES).toBe(200 * 1024 * 1024)
expect(DEFAULT_MAX_IMAGE_PIXELS).toBe(64_000_000)
expect(DEFAULT_MAX_IMAGE_DIMENSION).toBe(8192)
expect(service.imageLimits).toEqual({
maxImageBytes: DEFAULT_MAX_IMAGE_BYTES,
maxImagesPerMessage: DEFAULT_MAX_IMAGES_PER_MESSAGE,
@@ -24,6 +32,19 @@ describe('local attachment service', () => {
maxImageDimension: DEFAULT_MAX_IMAGE_DIMENSION,
mediaTypes: ['image/png', 'image/jpeg', 'image/webp', 'image/gif'],
})
expect(service.normalizationPolicy).toEqual({
maxDimension: DEFAULT_NORMALIZED_IMAGE_MAX_DIMENSION,
maxBytes: DEFAULT_NORMALIZED_IMAGE_MAX_BYTES,
})
expect(service.imageCompressionConcurrency).toBe(DEFAULT_IMAGE_COMPRESSION_CONCURRENCY)
})
it('resolves and validates the instance image-compression concurrency', () => {
expect(new LocalAttachmentStore(new Context(), { imageCompressionConcurrency: 1 }).imageCompressionConcurrency).toBe(1)
for (const imageCompressionConcurrency of [0, 1.5, 9]) {
expect(() => new LocalAttachmentStore(new Context(), { imageCompressionConcurrency }))
.toThrow(/imageCompressionConcurrency must be an integer from 1 through 8/)
}
})
it('saves and reads through the service boundary', async () => {
@@ -31,7 +52,7 @@ describe('local attachment service', () => {
try {
const service = new LocalAttachmentStore(new Context(), { dshHome })
const data = Uint8Array.from(Buffer.from(
'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNk+A8AAQUBAScY42YAAAAASUVORK5CYII=',
'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAACXBIWXMAAAPoAAAD6AG1e1JrAAAADElEQVQImWNgZGIGAAAOAAeCcsnOAAAAAElFTkSuQmCC',
'base64',
))
const ref = await service.saveImage({ data, mediaType: 'image/png' })
@@ -41,6 +62,67 @@ describe('local attachment service', () => {
}
})
it('commits a fully prepared image batch in input order', async () => {
const dshHome = await mkdtemp(join(tmpdir(), 'dsh-attachment-batch-success-'))
try {
const service = new LocalAttachmentStore(new Context(), { dshHome })
const first = new Uint8Array(await sharp({
create: { width: 2, height: 1, channels: 3, background: { r: 1, g: 2, b: 3 } },
}).png().toBuffer())
const second = new Uint8Array(await sharp({
create: { width: 1, height: 2, channels: 3, background: { r: 4, g: 5, b: 6 } },
}).png().toBuffer())
const refs = await service.saveImages([
{ data: first, mediaType: 'image/png', name: 'first.png' },
{ data: second, mediaType: 'image/png', name: 'second.png' },
])
expect(refs.map(ref => ref.name)).toEqual(['first.png', 'second.png'])
await expect(Promise.all(refs.map(ref => service.readImage(ref))))
.resolves.toHaveLength(2)
} finally {
await rm(dshHome, { recursive: true, force: true })
}
})
it.each([3, 4] as const)('admits a 16-bit %s-channel PNG as an 8-bit normalized object', async (channels) => {
const dshHome = await mkdtemp(join(tmpdir(), 'dsh-attachment-16-bit-'))
try {
const service = new LocalAttachmentStore(new Context(), { dshHome })
const source = new Uint8Array(await sharp({
create: { width: 7, height: 5, channels, background: { r: 12, g: 34, b: 56, alpha: 0.5 } },
}).toColourspace('rgb16').png().toBuffer())
const saved = await service.saveImage({ data: source, mediaType: 'image/png' })
const stored = await service.readImage(saved)
const metadata = await sharp(stored.data).metadata()
expect(stored.data).not.toEqual(source)
expect(metadata).toMatchObject({ depth: 'uchar', space: 'srgb', hasAlpha: channels === 4 })
} finally {
await rm(dshHome, { recursive: true, force: true })
}
})
it('prepares every batch member before any write', async () => {
const dshHome = await mkdtemp(join(tmpdir(), 'dsh-attachment-batch-'))
try {
const service = new LocalAttachmentStore(new Context(), { dshHome, normalizedImageMaxBytes: 1 })
const valid = Uint8Array.from(Buffer.from(
'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAACXBIWXMAAAPoAAAD6AG1e1JrAAAADElEQVQImWNgZGIGAAAOAAeCcsnOAAAAAElFTkSuQmCC',
'base64',
))
await expect(service.saveImages([
{ data: valid, mediaType: 'image/png' },
{ data: valid, mediaType: 'image/png' },
])).rejects.toMatchObject({ code: 'IMAGE_TOO_LARGE' })
expect(existsSync(service.root)).toBe(false)
} finally {
await rm(dshHome, { recursive: true, force: true })
}
})
it('validates without persisting: a rejected image leaves no storage root behind', async () => {
const dshHome = await mkdtemp(join(tmpdir(), 'dsh-attachment-validate-'))
try {
@@ -48,7 +130,7 @@ describe('local attachment service', () => {
await expect(service.validateImage({ data: Uint8Array.of(1, 2, 3), mediaType: 'image/png' }))
.rejects.toThrow(/Unsupported or malformed image data/)
const valid = Uint8Array.from(Buffer.from(
'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNk+A8AAQUBAScY42YAAAAASUVORK5CYII=',
'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAACXBIWXMAAAPoAAAD6AG1e1JrAAAADElEQVQImWNgZGIGAAAOAAeCcsnOAAAAAElFTkSuQmCC',
'base64',
))
const limited = new LocalAttachmentStore(new Context(), { dshHome, maxImageBytes: 1 })
@@ -0,0 +1,363 @@
import { describe, expect, it } from 'vitest'
import sharp from 'sharp'
import { hasLowColourCount, canPassThroughNormalization, normalizeImage } from '../src/normalization.ts'
import type { NormalizationPolicy } from '../src/normalization.ts'
import { detectImage } from '../src/image.ts'
const POLICY: NormalizationPolicy = { maxDimension: 2048, maxBytes: 4 * 1024 * 1024 }
/** Deterministic pseudo-random RGB noise; PNG cannot compress it below raw size. */
function noisePixels(width: number, height: number): Uint8Array {
const pixels = new Uint8Array(width * height * 3)
let state = 0x2545f491
for (let index = 0; index < pixels.length; index += 1) {
state ^= state << 13
state ^= state >>> 17
state ^= state << 5
pixels[index] = state & 0xff
}
return pixels
}
async function noiseImage(width: number, height: number, format: 'png' | 'jpeg' | 'webp' | 'gif'): Promise<Uint8Array> {
const image = sharp(noisePixels(width, height), { raw: { width, height, channels: 3 } })
return new Uint8Array(await image.toFormat(format).toBuffer())
}
async function flatImage(width: number, height: number, format: 'png' | 'jpeg' | 'webp' | 'gif', alpha = false): Promise<Uint8Array> {
const image = sharp({
create: { width, height, channels: alpha ? 4 : 3, background: { r: 12, g: 200, b: 64, alpha: alpha ? 0.5 : 1 } },
})
return new Uint8Array(await image.toFormat(format, format === 'webp' && alpha ? { lossless: true } : {}).toBuffer())
}
describe('canPassThroughNormalization', () => {
it('accepts an in-budget clean PNG/JPEG/WebP and refuses GIF, animation, metadata, oversized edges, and oversized bytes', () => {
const clean = { animated: false, carriesMetadata: false, depth: 'uchar', space: 'srgb', hasAlpha: false }
expect(canPassThroughNormalization({ mediaType: 'image/png', width: 2048, height: 4, ...clean }, 100, POLICY)).toBe(true)
expect(canPassThroughNormalization({ mediaType: 'image/gif', width: 4, height: 4, ...clean }, 100, POLICY)).toBe(false)
expect(canPassThroughNormalization({ mediaType: 'image/webp', width: 4, height: 4, animated: true, carriesMetadata: false, depth: 'uchar', space: 'srgb', hasAlpha: false }, 100, POLICY)).toBe(false)
expect(canPassThroughNormalization({ mediaType: 'image/jpeg', width: 4, height: 4, animated: false, carriesMetadata: true, depth: 'uchar', space: 'srgb', hasAlpha: false }, 100, POLICY)).toBe(false)
expect(canPassThroughNormalization({ mediaType: 'image/png', width: 4, height: 4, ...clean, depth: 'ushort' }, 100, POLICY)).toBe(false)
expect(canPassThroughNormalization({ mediaType: 'image/png', width: 4, height: 4, ...clean, space: 'rgb16' }, 100, POLICY)).toBe(false)
expect(canPassThroughNormalization({ mediaType: 'image/jpeg', width: 2049, height: 4, ...clean }, 100, POLICY)).toBe(false)
expect(canPassThroughNormalization({ mediaType: 'image/webp', width: 4, height: 4, ...clean }, POLICY.maxBytes + 1, POLICY)).toBe(false)
})
})
describe('normalizeImage', () => {
it('passes an already-normalized source through byte-identically', async () => {
const data = await flatImage(6, 4, 'webp')
const detected = await detectImage(data)
const normalized = await normalizeImage(data, detected, POLICY)
expect(normalized.data).toBe(data)
expect(normalized).toMatchObject({ mediaType: 'image/webp', width: 6, height: 4 })
})
it.each([3, 4] as const)('converts a 16-bit %s-channel PNG to 8-bit sRGB without passthrough', async (channels) => {
const data = new Uint8Array(await sharp({
create: { width: 7, height: 5, channels, background: { r: 12, g: 34, b: 56, alpha: 0.5 } },
}).toColourspace('rgb16').png().toBuffer())
const detected = await detectImage(data)
expect(detected).toMatchObject({ depth: 'ushort', space: 'rgb16', hasAlpha: channels === 4 })
const normalized = await normalizeImage(data, detected, POLICY)
expect(normalized.data).not.toBe(data)
expect(normalized.data).not.toEqual(data)
await expect(detectImage(normalized.data)).resolves.toMatchObject({
depth: 'uchar', space: 'srgb', hasAlpha: channels === 4, width: 7, height: 5,
})
})
it('downscales an oversized PNG to the long-edge target and stays PNG', async () => {
const data = await flatImage(10, 6, 'png')
const detected = await detectImage(data)
const normalized = await normalizeImage(data, detected, { maxDimension: 5, maxBytes: POLICY.maxBytes })
expect(normalized).toMatchObject({ mediaType: 'image/png', width: 5, height: 3 })
await expect(detectImage(normalized.data)).resolves.toMatchObject({ mediaType: 'image/png', width: 5, height: 3, animated: false, carriesMetadata: false, depth: 'uchar', space: 'srgb' })
const again = await normalizeImage(data, detected, { maxDimension: 5, maxBytes: POLICY.maxBytes })
expect(again.data).toEqual(normalized.data)
})
it('re-encodes the normalized output of a resize into itself (idempotence)', async () => {
const data = await flatImage(10, 6, 'png')
const first = await normalizeImage(data, await detectImage(data), { maxDimension: 5, maxBytes: POLICY.maxBytes })
const second = await normalizeImage(first.data, await detectImage(first.data), { maxDimension: 5, maxBytes: POLICY.maxBytes })
expect(second.data).toBe(first.data)
})
it('always re-encodes GIF to the PNG of its first frame', async () => {
const data = await flatImage(6, 4, 'gif')
const detected = await detectImage(data)
const normalized = await normalizeImage(data, detected, POLICY)
expect(normalized.mediaType).toBe('image/png')
await expect(detectImage(normalized.data)).resolves.toMatchObject({ mediaType: 'image/png', width: 6, height: 4, animated: false, carriesMetadata: false, depth: 'uchar', space: 'srgb' })
})
it('keeps a low-colour alpha source on PNG when the budget holds', async () => {
const data = await flatImage(9, 5, 'webp', true)
const detected = await detectImage(data)
const normalized = await normalizeImage(data, detected, { maxDimension: 4, maxBytes: POLICY.maxBytes })
expect(normalized).toMatchObject({ mediaType: 'image/png', width: 4, height: 2 })
})
it('accepts WebP output that omits an all-opaque source alpha plane', async () => {
const width = 64
const height = 32
const rgb = noisePixels(width, height)
const rgba = new Uint8Array(width * height * 4)
for (let pixel = 0; pixel < width * height; pixel += 1) {
rgba[pixel * 4] = rgb[pixel * 3] ?? 0
rgba[pixel * 4 + 1] = rgb[pixel * 3 + 1] ?? 0
rgba[pixel * 4 + 2] = rgb[pixel * 3 + 2] ?? 0
rgba[pixel * 4 + 3] = 255
}
const data = new Uint8Array(await sharp(rgba, {
raw: { width, height, channels: 4 },
}).png().toBuffer())
await expect(detectImage(data)).resolves.toMatchObject({ hasAlpha: true })
const normalized = await normalizeImage(data, await detectImage(data), {
maxDimension: 32,
maxBytes: POLICY.maxBytes,
})
expect(normalized).toMatchObject({ mediaType: 'image/webp', width: 32, height: 16 })
await expect(detectImage(normalized.data)).resolves.toMatchObject({ hasAlpha: false })
})
it('keeps transparency when the byte cap requires another encoding and smaller dimensions', async () => {
const side = 128
const pixels = new Uint8Array(side * side * 4)
const noise = noisePixels(side, side)
for (let pixel = 0; pixel < side * side; pixel += 1) {
const target = pixel * 4
const source = pixel * 3
pixels[target] = noise[source] ?? 0
pixels[target + 1] = noise[source + 1] ?? 0
pixels[target + 2] = noise[source + 2] ?? 0
pixels[target + 3] = pixel & 0xff
}
const data = new Uint8Array(await sharp(pixels, { raw: { width: side, height: side, channels: 4 } }).png().toBuffer())
const normalized = await normalizeImage(data, await detectImage(data), { maxDimension: side, maxBytes: 1_024 })
expect(normalized.data.byteLength).toBeLessThanOrEqual(1_024)
expect(normalized.width).toBeLessThan(side)
await expect(detectImage(normalized.data)).resolves.toMatchObject({ hasAlpha: true, depth: 'uchar', space: 'srgb' })
})
it('re-encodes an oversized photographic JPEG as JPEG', async () => {
const data = await noiseImage(64, 32, 'jpeg')
const detected = await detectImage(data)
const normalized = await normalizeImage(data, detected, { maxDimension: 32, maxBytes: POLICY.maxBytes })
expect(normalized).toMatchObject({ mediaType: 'image/jpeg', width: 32, height: 16 })
})
it('classifies a photographic PNG by pixels and uses an opaque photographic encoding', async () => {
// A smooth gradient: palette quantization dithers it into a sizable PNG
// while JPEG at quality 85 stays far smaller, so the budget between the
// two forces exactly one ladder hop.
const side = 256
const pixels = new Uint8Array(side * side * 3)
for (let y = 0; y < side; y += 1) {
for (let x = 0; x < side; x += 1) {
const index = (y * side + x) * 3
pixels[index] = x & 0xff
pixels[index + 1] = y & 0xff
pixels[index + 2] = (x + y) >> 1 & 0xff
}
}
const data = new Uint8Array(await sharp(pixels, { raw: { width: side, height: side, channels: 3 } }).png().toBuffer())
const detected = await detectImage(data)
const budget = { maxDimension: 128, maxBytes: POLICY.maxBytes }
const normalized = await normalizeImage(data, detected, budget)
expect(normalized.mediaType).toBe('image/jpeg')
expect(normalized).toMatchObject({ width: 128, height: 128 })
expect(normalized.data.byteLength).toBeLessThanOrEqual(budget.maxBytes)
})
it('shrinks dimensions after the quality floor instead of refusing an oversized encoding', async () => {
const data = await noiseImage(64, 64, 'png')
const normalized = await normalizeImage(data, await detectImage(data), { maxDimension: 2048, maxBytes: 512 })
expect(normalized.data.byteLength).toBeLessThanOrEqual(512)
expect(normalized.width).toBeLessThan(64)
expect(normalized.height).toBeLessThan(64)
})
it('re-encodes an in-budget oriented JPEG, baking rotation and stripping metadata', async () => {
const data = new Uint8Array(await sharp({
create: { width: 4, height: 2, channels: 3, background: { r: 1, g: 2, b: 3 } },
}).jpeg().withMetadata({ orientation: 6 }).toBuffer())
const detected = await detectImage(data)
// Orientation 6 rotates 90°: the perceived source is 2x4.
expect(detected).toMatchObject({ width: 2, height: 4, carriesMetadata: true })
const normalized = await normalizeImage(data, detected, POLICY)
expect(normalized.data).not.toBe(data)
expect(normalized).toMatchObject({ width: 2, height: 4 })
await expect(detectImage(normalized.data)).resolves.toMatchObject({ width: 2, height: 4, carriesMetadata: false })
})
it('re-encodes an in-budget image with an ICC profile and strips the profile', async () => {
const data = new Uint8Array(await sharp({
create: { width: 4, height: 2, channels: 3, background: { r: 1, g: 2, b: 3 } },
}).png().withIccProfile('p3').toBuffer())
const detected = await detectImage(data)
expect(detected.carriesMetadata).toBe(true)
const normalized = await normalizeImage(data, detected, POLICY)
expect(normalized.data).not.toBe(data)
await expect(detectImage(normalized.data)).resolves.toMatchObject({ carriesMetadata: false })
})
it('maps an encoder fault on undecodable bytes to a storage failure', async () => {
const detected = {
mediaType: 'image/png', width: 5000, height: 5000, animated: false, carriesMetadata: false,
depth: 'ushort', space: 'rgb16', hasAlpha: true,
} as const
await expect(normalizeImage(Uint8Array.of(1, 2, 3), detected, POLICY))
.rejects.toMatchObject({
code: 'ATTACHMENT_WRITE_FAILED',
message: 'The 16-bit PNG could not be converted to the normalized 8-bit sRGB form.',
})
})
it.each([
['float PNG', { mediaType: 'image/png', depth: 'float' }],
['uchar JPEG', { mediaType: 'image/jpeg', depth: 'uchar' }],
] as const)('describes a failed %s conversion without exposing the encoder error', async (source, fields) => {
const detected = {
...fields,
width: 5000,
height: 5000,
animated: false,
carriesMetadata: false,
space: 'srgb',
hasAlpha: false,
} as const
await expect(normalizeImage(Uint8Array.of(1, 2, 3), detected, POLICY))
.rejects.toMatchObject({
code: 'ATTACHMENT_WRITE_FAILED',
message: `The ${source} could not be converted to the normalized 8-bit sRGB form.`,
})
})
it('rejects a converted normalized image whose verified alpha metadata disagrees with the source facts', async () => {
const data = await flatImage(8, 8, 'png', true)
const detected = await detectImage(data)
await expect(normalizeImage(data, { ...detected, hasAlpha: false }, {
maxDimension: 4,
maxBytes: POLICY.maxBytes,
})).rejects.toMatchObject({
code: 'ATTACHMENT_WRITE_FAILED',
message: 'Image normalization did not produce a single-frame 8-bit sRGB image with matching metadata.',
})
})
})
describe('hasLowColourCount', () => {
it('distinguishes photographic rasters from low-colour graphics without averaged sampling', async () => {
const side = 512
const highFrequency = sharp(noisePixels(side, side), { raw: { width: side, height: side, channels: 3 } })
const gradientPixels = new Uint8Array(side * side * 3)
for (let y = 0; y < side; y += 1) {
for (let x = 0; x < side; x += 1) {
const offset = (y * side + x) * 3
gradientPixels[offset] = x & 0xff
gradientPixels[offset + 1] = y & 0xff
gradientPixels[offset + 2] = (x * 3 + y * 5) & 0xff
}
}
const ordinaryPhoto = sharp(gradientPixels, { raw: { width: side, height: side, channels: 3 } })
const solid = sharp({
create: { width: side, height: side, channels: 3, background: { r: 12, g: 34, b: 56 } },
})
const text = sharp(Buffer.from(`
<svg width="512" height="256" xmlns="http://www.w3.org/2000/svg">
<rect width="512" height="256" fill="white"/>
<text x="24" y="145" font-size="96" fill="#16324f">DeepSeek 16-bit</text>
</svg>
`))
const transparentData = await sharp({
create: { width: side, height: side, channels: 4, background: { r: 0, g: 0, b: 0, alpha: 0 } },
}).composite([{ input: Buffer.from(`
<svg width="512" height="512" xmlns="http://www.w3.org/2000/svg">
<circle cx="256" cy="256" r="180" fill="#0f8" fill-opacity="0.7"/>
</svg>
`) }]).png().toBuffer()
const transparent = sharp(transparentData)
await expect(hasLowColourCount(highFrequency)).resolves.toBe(false)
await expect(hasLowColourCount(ordinaryPhoto)).resolves.toBe(false)
await expect(hasLowColourCount(solid)).resolves.toBe(true)
await expect(hasLowColourCount(text)).resolves.toBe(true)
await expect(hasLowColourCount(transparent)).resolves.toBe(true)
})
it('reads grayscale-alpha samples without treating alpha or the next pixel as RGB', async () => {
const symbols: number[] = []
for (let first = 0; first < 32; first += 1) {
for (let second = 0; second < 32; second += 1) symbols.push(first, second)
}
const pixels = new Uint8Array(symbols.length * 2)
for (const [index, symbol] of symbols.entries()) {
pixels[index * 2] = symbol * 8
pixels[index * 2 + 1] = symbol * 8
}
const grayscaleAlpha = sharp(pixels, {
raw: { width: 128, height: 16, channels: 2 },
})
await expect(hasLowColourCount(grayscaleAlpha)).resolves.toBe(true)
})
it('reads one-channel grayscale samples as equal RGB values', async () => {
const pixels = new Uint8Array(128 * 16)
for (let index = 0; index < pixels.length; index += 1) pixels[index] = index & 0xff
await expect(hasLowColourCount(sharp(pixels, {
raw: { width: 128, height: 16, channels: 1 },
}))).resolves.toBe(true)
})
it('keeps an antialiased text screenshot readable on the low-colour PNG path', async () => {
const source = new Uint8Array(await sharp(Buffer.from(`
<svg width="1024" height="512" xmlns="http://www.w3.org/2000/svg">
<rect width="1024" height="512" fill="white"/>
<text x="48" y="290" font-size="170" fill="#112f4d">Readable text</text>
</svg>
`)).removeAlpha().png().toBuffer())
const normalized = await normalizeImage(source, await detectImage(source), {
maxDimension: 512,
maxBytes: POLICY.maxBytes,
})
const stats = await sharp(normalized.data).greyscale().stats()
expect(normalized).toMatchObject({ mediaType: 'image/png', width: 512, height: 256 })
expect(stats.channels[0]?.min).toBeLessThan(80)
expect(stats.channels[0]?.max).toBeGreaterThan(240)
})
})
@@ -0,0 +1,47 @@
import { mkdtemp, rm } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { Context } from '@deepseek-ai/cordis'
import sharp from 'sharp'
import { afterEach, describe, expect, it, vi } from 'vitest'
const control = vi.hoisted(() => ({ mismatch: false }))
vi.mock('../src/image.ts', async (importOriginal) => {
const actual = await importOriginal<typeof import('../src/image.ts')>()
return {
...actual,
async detectImage(data: Uint8Array): Promise<Awaited<ReturnType<typeof actual.detectImage>>> {
const detected = await actual.detectImage(data)
return control.mismatch ? { ...detected, width: detected.width + 1 } : detected
},
}
})
import LocalAttachmentStore from '../src/index.ts'
const homes: string[] = []
afterEach(async () => {
control.mismatch = false
await Promise.all(homes.splice(0).map(home => rm(home, { recursive: true, force: true })))
})
describe('request image verification', () => {
it('rejects an encoded request whose decoded facts disagree with the encoder result', async () => {
const dshHome = await mkdtemp(join(tmpdir(), 'dsh-request-verification-'))
homes.push(dshHome)
const attachments = new LocalAttachmentStore(new Context(), { dshHome })
const source = new Uint8Array(await sharp({
create: { width: 64, height: 32, channels: 3, background: { r: 12, g: 34, b: 56 } },
}).png().toBuffer())
const attachment = await attachments.saveImage({ data: source, mediaType: 'image/png' })
control.mismatch = true
await expect(attachments.readImageRequest(attachment, { maxPixels: 16 * 16, maxBytes: 1024 * 1024 }))
.rejects.toMatchObject({
code: 'ATTACHMENT_WRITE_FAILED',
message: 'Encoded model-request image does not match its verified 8-bit sRGB metadata.',
})
})
})
@@ -0,0 +1,348 @@
import { mkdtemp, rm, writeFile } from 'node:fs/promises'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import { Context } from '@deepseek-ai/cordis'
import sharp from 'sharp'
import { afterEach, describe, expect, it, vi } from 'vitest'
import { CompressionLimiter } from '../src/compression-limiter.ts'
import LocalAttachmentStore, { requestImageDimensions } from '../src/index.ts'
const homes: string[] = []
async function store(): Promise<LocalAttachmentStore> {
const dshHome = await mkdtemp(join(tmpdir(), 'dsh-request-image-'))
homes.push(dshHome)
return new LocalAttachmentStore(new Context(), { dshHome })
}
async function image(width: number, height: number): Promise<Uint8Array> {
return new Uint8Array(await sharp({
create: { width, height, channels: 3, background: { r: 12, g: 34, b: 56 } },
}).png().toBuffer())
}
async function complexOpaqueAlphaImage(width: number, height: number): Promise<Uint8Array> {
const pixels = new Uint8Array(width * height * 4)
let state = 0x2545f491
for (let offset = 0; offset < pixels.length; offset += 4) {
for (let channel = 0; channel < 3; channel += 1) {
state ^= state << 13
state ^= state >>> 17
state ^= state << 5
pixels[offset + channel] = state & 0xff
}
pixels[offset + 3] = 255
}
return new Uint8Array(await sharp(pixels, {
raw: { width, height, channels: 4 },
}).png().toBuffer())
}
afterEach(async () => {
await Promise.all(homes.splice(0).map(home => rm(home, { recursive: true, force: true })))
})
describe('request image dimensions', () => {
it.each([
[4096, 4096, 800, 800],
[4096, 2048, 1130, 565],
[3840, 2160, 1066, 600],
[320, 240, 320, 240],
])('projects %sx%s under 640,000 pixels as %sx%s', (width, height, expectedWidth, expectedHeight) => {
const projected = requestImageDimensions(width, height, 640_000)
expect(projected).toEqual({
width: expectedWidth,
height: expectedHeight,
})
expect(projected.width * projected.height).toBeLessThanOrEqual(640_000)
})
it('projects a portrait within the same total-pixel budget', () => {
const projected = requestImageDimensions(2160, 3840, 640_000)
expect(projected).toEqual({ width: 600, height: 1066 })
expect(projected.width * projected.height).toBeLessThanOrEqual(640_000)
})
it('rounds a portrait inward when integer aspect rounding crosses the pixel cap', () => {
expect(requestImageDimensions(2, 4, 5)).toEqual({ width: 1, height: 2 })
})
})
describe('local request-image cache', () => {
it('passes through an in-budget attachment and composes ordered request reads', async () => {
const attachments = await store()
const first = await attachments.saveImage({ data: await image(8, 4), mediaType: 'image/png' })
const second = await attachments.saveImage({ data: await image(4, 8), mediaType: 'image/png' })
const firstStored = await attachments.readImage(first)
const policy = { maxPixels: 1_000, maxBytes: 1024 * 1024 }
const request = await attachments.readImageRequest(first, policy)
const batch = await Promise.all([first, second].map(
attachment => attachments.readImageRequest(attachment, policy),
))
expect(request.data).toEqual(firstStored.data)
expect(batch.map(value => value.attachment.attachmentId)).toEqual([first.attachmentId, second.attachmentId])
})
it('rejects invalid request policies', async () => {
const attachments = await store()
const attachment = await attachments.saveImage({ data: await image(8, 4), mediaType: 'image/png' })
await expect(attachments.readImageRequest(attachment, { maxPixels: 0, maxBytes: 100 }))
.rejects.toThrow('Image request maxPixels must be a positive integer')
await expect(attachments.readImageRequest(attachment, { maxPixels: 100, maxBytes: 0 }))
.rejects.toThrow('Image request maxBytes must be a positive integer')
})
it('refuses a one-pixel request that cannot meet the encoded-byte budget', async () => {
const attachments = await store()
const attachment = await attachments.saveImage({ data: await image(1, 1), mediaType: 'image/png' })
await expect(attachments.readImageRequest(attachment, { maxPixels: 1, maxBytes: 1 }))
.rejects.toMatchObject({ code: 'IMAGE_TOO_LARGE' })
})
it('regenerates invalid, oversized, incompatible, or mismatched cached variants', async () => {
const attachments = await store()
const attachment = await attachments.saveImage({ data: await image(64, 32), mediaType: 'image/png' })
const policy = { maxPixels: 16 * 16, maxBytes: 4_096 }
const initial = await attachments.readImageRequest(attachment, policy)
const hash = String(initial.variantId).slice('sha256:'.length)
const path = join(attachments.root, 'request-images', hash.slice(0, 2), hash)
const noisyPixels = new Uint8Array(64 * 64 * 3)
let state = 0x2545f491
for (let index = 0; index < noisyPixels.length; index += 1) {
state ^= state << 13
state ^= state >>> 17
state ^= state << 5
noisyPixels[index] = state & 0xff
}
const oversized = new Uint8Array(await sharp(noisyPixels, {
raw: { width: 64, height: 64, channels: 3 },
}).png().toBuffer())
const depth16 = new Uint8Array(await sharp({
create: { width: 16, height: 8, channels: 3, background: { r: 1, g: 2, b: 3 } },
}).toColourspace('rgb16').png().toBuffer())
const cmyk = new Uint8Array(await sharp({
create: { width: 16, height: 8, channels: 3, background: { r: 1, g: 2, b: 3 } },
}).toColourspace('cmyk').jpeg().toBuffer())
const tooWide = await image(23, 11)
const unexpectedAlpha = new Uint8Array(await sharp({
create: { width: 16, height: 8, channels: 4, background: { r: 1, g: 2, b: 3, alpha: 0.5 } },
}).png().toBuffer())
for (const invalid of [
oversized,
depth16,
cmyk,
tooWide,
unexpectedAlpha,
Uint8Array.of(1, 2, 3),
]) {
await writeFile(path, invalid)
const regenerated = await attachments.readImageRequest(attachment, policy)
expect(regenerated.data).toEqual(initial.data)
}
})
it('derives stable square and wide previews and separates route budgets in the cache key', async () => {
const attachments = await store()
const square = await attachments.saveImage({
data: await image(2048, 2048), mediaType: 'image/png', name: 'square.png',
})
const wide = await attachments.saveImage({
data: await image(2048, 1024), mediaType: 'image/png', name: 'wide.png',
})
const squareRequest = await attachments.readImageRequest(square, { maxPixels: 640_000, maxBytes: 1024 * 1024 })
const wideRequest = await attachments.readImageRequest(wide, { maxPixels: 640_000, maxBytes: 1024 * 1024 })
const repeated = await attachments.readImageRequest(wide, { maxPixels: 640_000, maxBytes: 1024 * 1024 })
const low = await attachments.readImageRequest(wide, { maxPixels: 512 * 512, maxBytes: 1024 * 1024 })
expect(squareRequest).toMatchObject({ width: 800, height: 800 })
expect(wideRequest).toMatchObject({ width: 1130, height: 565 })
expect(repeated.variantId).toBe(wideRequest.variantId)
expect(repeated.data).toEqual(wideRequest.data)
expect(Buffer.from(repeated.data).toString('base64')).toBe(Buffer.from(wideRequest.data).toString('base64'))
expect(low.variantId).not.toBe(wideRequest.variantId)
expect(low.width * low.height).toBeLessThanOrEqual(512 * 512 + low.width)
})
it('classifies opaque PNG pixels and preserves alpha while enforcing the request budget', async () => {
const attachments = await store()
const side = 256
const photoPixels = new Uint8Array(side * side * 3)
const alphaPixels = new Uint8Array(side * side * 4)
let state = 0x2545f491
for (let pixel = 0; pixel < side * side; pixel += 1) {
state ^= state << 13
state ^= state >>> 17
state ^= state << 5
const photo = pixel * 3
const alpha = pixel * 4
photoPixels[photo] = state & 0xff
photoPixels[photo + 1] = state >> 8 & 0xff
photoPixels[photo + 2] = state >> 16 & 0xff
alphaPixels[alpha] = photoPixels[photo] ?? 0
alphaPixels[alpha + 1] = photoPixels[photo + 1] ?? 0
alphaPixels[alpha + 2] = photoPixels[photo + 2] ?? 0
alphaPixels[alpha + 3] = pixel & 0xff
}
const photoSource = new Uint8Array(await sharp(photoPixels, {
raw: { width: side, height: side, channels: 3 },
}).png().toBuffer())
const alphaSource = new Uint8Array(await sharp(alphaPixels, {
raw: { width: side, height: side, channels: 4 },
}).png().toBuffer())
const photo = await attachments.saveImage({ data: photoSource, mediaType: 'image/png' })
const alpha = await attachments.saveImage({ data: alphaSource, mediaType: 'image/png' })
const photoRequest = await attachments.readImageRequest(photo, { maxPixels: 128 * 128, maxBytes: 1024 * 1024 })
const alphaRequest = await attachments.readImageRequest(alpha, { maxPixels: 128 * 128, maxBytes: 4_096 })
expect(photoRequest.mediaType).toBe('image/jpeg')
expect(alphaRequest.bytes).toBeLessThanOrEqual(4_096)
expect(alphaRequest.width).toBeLessThan(128)
await expect(sharp(alphaRequest.data).metadata()).resolves.toMatchObject({ hasAlpha: true, depth: 'uchar', space: 'srgb' })
})
it.each([3, 4] as const)('projects a 16-bit %s-channel PNG as a bounded 8-bit request image', async (channels) => {
const attachments = await store()
const source = new Uint8Array(await sharp({
create: { width: 64, height: 32, channels, background: { r: 12, g: 34, b: 56, alpha: 0.5 } },
}).toColourspace('rgb16').png().toBuffer())
const attachment = await attachments.saveImage({ data: source, mediaType: 'image/png' })
const request = await attachments.readImageRequest(attachment, { maxPixels: 16 * 16, maxBytes: 1024 * 1024 })
expect(request.bytes).toBeLessThanOrEqual(1024 * 1024)
expect(request.width * request.height).toBeLessThanOrEqual(16 * 16)
await expect(sharp(request.data).metadata()).resolves.toMatchObject({
depth: 'uchar', space: 'srgb', hasAlpha: channels === 4,
})
})
it('accepts a resized WebP request version that omits an all-opaque alpha plane', async () => {
const attachments = await store()
const source = await complexOpaqueAlphaImage(64, 32)
const attachment = await attachments.saveImage({ data: source, mediaType: 'image/png' })
const request = await attachments.readImageRequest(attachment, { maxPixels: 16 * 16, maxBytes: 1024 * 1024 })
expect(request.mediaType).toBe('image/webp')
await expect(sharp(request.data).metadata()).resolves.toMatchObject({ hasAlpha: false })
})
it('keeps a complex 640,000-pixel request version below 1 MiB', async () => {
const attachments = await store()
const side = 1024
const pixels = new Uint8Array(side * side * 3)
let state = 0x6d2b79f5
for (let index = 0; index < pixels.length; index += 1) {
state ^= state << 13
state ^= state >>> 17
state ^= state << 5
pixels[index] = state & 0xff
}
const source = new Uint8Array(await sharp(pixels, {
raw: { width: side, height: side, channels: 3 },
}).png().toBuffer())
const attachment = await attachments.saveImage({ data: source, mediaType: 'image/png' })
const request = await attachments.readImageRequest(attachment, { maxPixels: 640_000, maxBytes: 1024 * 1024 })
expect(request).toMatchObject({ width: 800, height: 800 })
expect(request.bytes).toBeLessThanOrEqual(1024 * 1024)
})
it('shares one request transform between concurrent callers without sharing cancellation', async () => {
const attachments = await store()
const attachment = await attachments.saveImage({
data: await image(2048, 1024), mediaType: 'image/png', name: 'shared.png',
})
const run = vi.spyOn(CompressionLimiter.prototype, 'run')
const controller = new AbortController()
const policy = { maxPixels: 640_000, maxBytes: 1024 * 1024 }
const cancelled = attachments.readImageRequest(attachment, policy, controller.signal)
const completed = attachments.readImageRequest(attachment, policy)
const reason = new Error('cancel one waiter')
controller.abort(reason)
await expect(cancelled).rejects.toBe(reason)
await expect(completed).resolves.toMatchObject({ width: 1130, height: 565 })
expect(run).toHaveBeenCalledTimes(1)
run.mockRestore()
})
it('aborts the underlying request transform after its only waiter cancels', async () => {
const attachments = await store()
const attachment = await attachments.saveImage({
data: await image(2048, 1024), mediaType: 'image/png', name: 'cancelled.png',
})
let readSignal: AbortSignal | undefined
const read = vi.spyOn(attachments, 'readImage').mockImplementation((_ref, signal) => {
readSignal = signal
return new Promise((_resolve, reject) => {
signal?.addEventListener('abort', () => {
reject(new Error('request transform aborted', { cause: signal.reason }))
}, { once: true })
})
})
const controller = new AbortController()
const request = attachments.readImageRequest(
attachment,
{ maxPixels: 640_000, maxBytes: 1024 * 1024 },
controller.signal,
)
await vi.waitFor(() => {
expect(read).toHaveBeenCalledTimes(1)
})
const reason = new Error('cancel only transform waiter')
controller.abort(reason)
await expect(request).rejects.toBe(reason)
expect(readSignal?.reason).toBe(reason)
})
it('normalizes a non-Error cancellation and replaces an aborted shared transform', async () => {
const attachments = await store()
const attachment = await attachments.saveImage({
data: await image(2048, 1024), mediaType: 'image/png', name: 'replace.png',
})
const actualRead = attachments.readImage.bind(attachments)
let calls = 0
vi.spyOn(attachments, 'readImage').mockImplementation((ref, signal) => {
calls += 1
if (calls === 1) {
return new Promise((_resolve, reject) => {
signal?.addEventListener('abort', () => {
reject(new Error('request transform aborted', { cause: signal.reason }))
}, { once: true })
})
}
return actualRead(ref, signal)
})
const controller = new AbortController()
const policy = { maxPixels: 640_000, maxBytes: 1024 * 1024 }
const cancelled = attachments.readImageRequest(attachment, policy, controller.signal)
await vi.waitFor(() => {
expect(calls).toBe(1)
})
controller.abort('cancelled')
const replacement = attachments.readImageRequest(attachment, policy)
await expect(cancelled).rejects.toMatchObject({
message: 'Attachment request cancelled with a non-Error reason.',
cause: 'cancelled',
})
await expect(replacement).resolves.toMatchObject({ width: 1130, height: 565 })
expect(calls).toBe(2)
})
})
@@ -7,7 +7,8 @@ import { mkdtemp, rm } from 'node:fs/promises'
import { afterEach, describe, expect, it, vi } from 'vitest'
import sharp from 'sharp'
import type { ImageAttachmentLimits } from '@deepseek-ai/dsh-attachment'
import { readImageFile, saveImageFile } from '../src/store.ts'
import type { NormalizationPolicy } from '../src/normalization.ts'
import { commitPreparedImageFile, prepareImageFile, readImageFile, saveImageFile } from '../src/store.ts'
const fsControl = vi.hoisted(() => ({
readSignals: [] as AbortSignal[],
@@ -34,10 +35,12 @@ vi.mock('node:fs/promises', async (importOriginal) => {
})
const PNG = Uint8Array.from(Buffer.from(
'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNk+A8AAQUBAScY42YAAAAASUVORK5CYII=',
'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAACXBIWXMAAAPoAAAD6AG1e1JrAAAADElEQVQImWNgZGIGAAAOAAeCcsnOAAAAAElFTkSuQmCC',
'base64',
))
const POLICY: NormalizationPolicy = { maxDimension: 2048, maxBytes: 1024 * 1024 }
const LIMITS: ImageAttachmentLimits = {
maxImageBytes: 1024,
maxImagesPerMessage: 2,
@@ -79,7 +82,7 @@ describe('local attachment store', () => {
const bucket = join(objects, sha256.slice(0, 2))
fsControl.syncedDirectories.length = 0
await saveImageFile(storageRoot, { data: PNG, mediaType: 'image/png' }, LIMITS)
await saveImageFile(storageRoot, { data: PNG, mediaType: 'image/png' }, LIMITS, POLICY)
// Each process first proves DSH_HOME durable all the way to the filesystem
// root; existence alone cannot vouch for a concurrent creator's fsync.
@@ -104,7 +107,7 @@ describe('local attachment store', () => {
it('creates and persists a missing nested home directory against the filesystem root', async () => {
const storageRoot = join(await root(), 'home', 'attachments', 'v1')
const ref = await saveImageFile(storageRoot, { data: PNG, mediaType: 'image/png' }, LIMITS)
const ref = await saveImageFile(storageRoot, { data: PNG, mediaType: 'image/png' }, LIMITS, POLICY)
await expect(readImageFile(storageRoot, ref)).resolves.toEqual({ ref, data: PNG })
})
@@ -113,8 +116,8 @@ describe('local attachment store', () => {
const storageRoot = await root()
const first = await saveImageFile(storageRoot, {
data: PNG, mediaType: 'image/png', name: '/private/tmp/pixel.png',
}, LIMITS)
const second = await saveImageFile(storageRoot, { data: PNG, mediaType: 'image/png' }, LIMITS)
}, LIMITS, POLICY)
const second = await saveImageFile(storageRoot, { data: PNG, mediaType: 'image/png' }, LIMITS, POLICY)
const sha256 = createHash('sha256').update(PNG).digest('hex')
const object = join(storageRoot, 'objects', sha256.slice(0, 2), sha256)
@@ -135,16 +138,39 @@ describe('local attachment store', () => {
await expect(readImageFile(storageRoot, first)).resolves.toEqual({ ref: first, data: PNG })
})
it('stores the normalized image of an oversized source and reads it back verified', async () => {
const storageRoot = await root()
const oversized = new Uint8Array(await sharp({
create: { width: 4, height: 4, channels: 3, background: { r: 9, g: 9, b: 9 } },
}).png().toBuffer())
const saved = await saveImageFile(storageRoot, {
data: oversized, mediaType: 'image/png', name: 'big.png',
}, { ...LIMITS, maxImagePixels: 64 }, { maxDimension: 2, maxBytes: 1024 * 1024 })
expect(saved).toMatchObject({
mediaType: 'image/png',
width: 2,
height: 2,
name: 'big.png',
originalDimensions: { width: 4, height: 4 },
})
expect(saved.bytes).not.toBe(oversized.byteLength)
const read = await readImageFile(storageRoot, saved)
expect(read.data.byteLength).toBe(saved.bytes)
expect(String(saved.attachmentId)).toBe(`sha256:${createHash('sha256').update(read.data).digest('hex')}`)
})
it('keeps admitted history readable after deployment limits become stricter', async () => {
const storageRoot = await root()
const ref = await saveImageFile(storageRoot, { data: PNG, mediaType: 'image/png' }, LIMITS)
const ref = await saveImageFile(storageRoot, { data: PNG, mediaType: 'image/png' }, LIMITS, POLICY)
await expect(readImageFile(storageRoot, ref)).resolves.toEqual({ ref, data: PNG })
})
it('forwards read cancellation to the filesystem and preserves its reason', async () => {
const storageRoot = await root()
const ref = await saveImageFile(storageRoot, { data: PNG, mediaType: 'image/png' }, LIMITS)
const ref = await saveImageFile(storageRoot, { data: PNG, mediaType: 'image/png' }, LIMITS, POLICY)
const controller = new AbortController()
fsControl.readSignals.length = 0
@@ -160,35 +186,35 @@ describe('local attachment store', () => {
const storageRoot = await root()
await expect(saveImageFile(storageRoot, {
data: new Uint8Array(0), mediaType: 'image/png',
}, LIMITS)).rejects.toMatchObject({ code: 'INVALID_IMAGE' })
}, LIMITS, POLICY)).rejects.toMatchObject({ code: 'INVALID_IMAGE' })
await expect(saveImageFile(storageRoot, {
data: Uint8Array.of(1, 2, 3), mediaType: 'image/png',
}, LIMITS)).rejects.toMatchObject({ code: 'INVALID_IMAGE' })
}, LIMITS, POLICY)).rejects.toMatchObject({ code: 'INVALID_IMAGE' })
await expect(saveImageFile(storageRoot, {
data: PNG, mediaType: 'image/jpeg',
}, LIMITS)).rejects.toMatchObject({ code: 'IMAGE_TYPE_MISMATCH' })
}, LIMITS, POLICY)).rejects.toMatchObject({ code: 'IMAGE_TYPE_MISMATCH' })
await expect(saveImageFile(storageRoot, {
data: PNG, mediaType: 'image/png',
}, { ...LIMITS, maxImageBytes: 1 })).rejects.toMatchObject({ code: 'IMAGE_TOO_LARGE' })
}, { ...LIMITS, maxImageBytes: 1 }, POLICY)).rejects.toMatchObject({ code: 'IMAGE_TOO_LARGE' })
const wide = new Uint8Array(await sharp({
create: { width: 5, height: 5, channels: 4, background: { r: 0, g: 0, b: 0, alpha: 1 } },
}).png().toBuffer())
await expect(saveImageFile(storageRoot, {
data: wide, mediaType: 'image/png',
}, LIMITS)).rejects.toMatchObject({ code: 'IMAGE_TOO_MANY_PIXELS' })
}, LIMITS, POLICY)).rejects.toMatchObject({ code: 'IMAGE_TOO_MANY_PIXELS' })
await expect(saveImageFile(storageRoot, {
data: wide, mediaType: 'image/png',
}, { ...LIMITS, maxImagePixels: 25, maxImageDimension: 4 })).rejects.toMatchObject({ code: 'IMAGE_DIMENSION_TOO_LARGE' })
}, { ...LIMITS, maxImagePixels: 25, maxImageDimension: 4 }, POLICY)).rejects.toMatchObject({ code: 'IMAGE_DIMENSION_TOO_LARGE' })
const unnamed = await saveImageFile(storageRoot, {
data: PNG, mediaType: 'image/png', name: '\u0000',
}, LIMITS)
}, LIMITS, POLICY)
expect(unnamed).not.toHaveProperty('name')
})
it('fails closed when an object is missing, corrupted, or addressed by an invalid reference', async () => {
const storageRoot = await root()
const ref = await saveImageFile(storageRoot, { data: PNG, mediaType: 'image/png' }, LIMITS)
const ref = await saveImageFile(storageRoot, { data: PNG, mediaType: 'image/png' }, LIMITS, POLICY)
const sha256 = String(ref.attachmentId).slice('sha256:'.length)
const object = join(storageRoot, 'objects', sha256.slice(0, 2), sha256)
await chmod(object, 0o600)
@@ -216,11 +242,11 @@ describe('local attachment store', () => {
const target = join(storageRoot, 'objects', sha256.slice(0, 2), sha256)
await mkdir(join(storageRoot, 'objects', sha256.slice(0, 2)), { recursive: true })
await writeFile(target, Uint8Array.of(1, 2, 3))
await expect(saveImageFile(storageRoot, { data: PNG, mediaType: 'image/png' }, LIMITS))
await expect(saveImageFile(storageRoot, { data: PNG, mediaType: 'image/png' }, LIMITS, POLICY))
.rejects.toMatchObject({ code: 'ATTACHMENT_CORRUPT' })
await writeFile(target, PNG)
const ref = await saveImageFile(storageRoot, { data: PNG, mediaType: 'image/png' }, LIMITS)
const ref = await saveImageFile(storageRoot, { data: PNG, mediaType: 'image/png' }, LIMITS, POLICY)
await expect(readImageFile(storageRoot, { ...ref, width: ref.width + 1 }))
.rejects.toMatchObject({ code: 'ATTACHMENT_CORRUPT' })
})
@@ -231,7 +257,17 @@ describe('local attachment store', () => {
const target = join(storageRoot, 'objects', sha256.slice(0, 2), sha256)
await mkdir(target, { recursive: true })
await expect(saveImageFile(storageRoot, { data: PNG, mediaType: 'image/png' }, LIMITS))
await expect(saveImageFile(storageRoot, { data: PNG, mediaType: 'image/png' }, LIMITS, POLICY))
.rejects.toMatchObject({ code: 'ATTACHMENT_WRITE_FAILED' })
})
it('rejects prepared bytes that no longer match their content-addressed reference', async () => {
const storageRoot = await root()
const prepared = await prepareImageFile({ data: PNG, mediaType: 'image/png' }, LIMITS, POLICY)
await expect(commitPreparedImageFile(storageRoot, {
...prepared,
data: Uint8Array.of(...prepared.data, 0),
})).rejects.toMatchObject({ code: 'ATTACHMENT_CORRUPT' })
})
})
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/attachment/attachment/README.md
README.md: 19232bd4bb86ed33e56fcdca93999967822422ab
README.zh.md: e5e7aab7c1af30b2b101bdcd218044cd1095ae0d
README.md: 3ad568c7308f1ab85cb4af3fcc2afd3cba9a611a
README.zh.md: fadbb1c5bbf097c599da651055d63a1ed64cd579
+3 -3
View File
@@ -2,15 +2,15 @@
English | [中文](README.zh.md)
The durable attachment seam. `ctx.attachments` validates and durably commits immutable image bytes, then returns a serializable `ImageAttachmentRef`; consumers never persist browser paths, object URLs, provider URLs, or base64 in session events.
The durable attachment seam. `ctx.attachments` validates and durably commits a provider-independent normalized image, then returns a serializable `ImageAttachmentRef`; consumers never persist browser paths, object URLs, provider URLs, or base64 in session events.
Unsent composer images remain browser-owned temporary drafts. `validateImage` runs the same admission policy without persisting. `saveImages` owns batch count and aggregate-byte limits, validates every member before writing any member, then commits in order and returns references only after the complete batch succeeds. A later storage failure returns no partial references, although an earlier immutable content-addressed object may remain unreachable until reference-aware garbage collection exists. `AttachmentError.code` uses the closed `AttachmentErrorCode` string union. Its `ImageAdmissionErrorCode` subset marks caller-correctable image-input failures; `isImageAdmissionError` recognizes that subset at runtime so each protocol adapter can map its own error vocabulary. `saveImage` commits one accepted image before any model-visible session event is published, and `readImage` verifies the content-addressed object against its logged metadata. Callers may cancel `readImage`; implementations observe cancellation around backend and verification work and preserve it instead of translating it into a storage failure.
Unsent composer images remain browser-owned temporary drafts. `validateImage` runs the complete admission policy without persisting. `saveImages` owns batch count and aggregate-byte limits, prepares every normalized attachment before publishing any member, then commits in order and returns references only after the complete batch succeeds. A later storage failure returns no partial references, although an earlier immutable content-addressed object may remain unreachable until reference-aware garbage collection exists. `AttachmentError.code` uses the closed `AttachmentErrorCode` string union. Its `ImageAdmissionErrorCode` subset marks caller-correctable image-input failures; `isImageAdmissionError` recognizes that subset at runtime so each protocol adapter can map its own error vocabulary. `saveImage` commits one accepted image before any model-visible session event is published and returns its `ImageAttachmentRef`. When normalization reduces the raster, the reference records the orientation-applied input size in `originalDimensions`. `readImage` verifies the normalized attachment against its logged metadata. `readImageRequest` deterministically derives a route-sized request version whose identity covers the attachment id, transform version, pixel and byte budgets, and encoder settings. Callers compose ordered batches with `Promise.all(refs.map(...))`; the local implementation still bounds compression through its instance limiter, cache, and singleflight. Callers may cancel reads and projections; implementations preserve cancellation instead of translating it into a storage failure.
`admitEncodedImages(attachments, images)` is the shared wire entry used by every RPC endpoint that accepts browser uploads (the session prompt endpoint and the command executor): it enforces canonical base64 on every member, then delegates batch admission — limits, validation, ordered commit — to `saveImages`. The base64 upload form is `EncodedImageAttachment`, exported from `@deepseek-ai/dsh-attachment/types` so wire contracts can reference it.
## Model Experience
Indirectly, through the role-neutral core `ImageBlock` and provider adapters that resolve its durable reference.
Indirectly, through the role-neutral core `ImageBlock` and provider adapters that resolve its durable reference into an exact request version. Request descriptors expose the complete attachment id and actual request dimensions.
#### KV Cache effect
+3 -3
View File
@@ -2,15 +2,15 @@
[English](README.md) | 中文
持久附件服务边界。`ctx.attachments` 校验并持久提交不可变图片字节,随后返回可序列化的 `ImageAttachmentRef`;消费方绝不会在会话事件中持久保存浏览器路径、对象 URL、提供方 URL 或 base64。
持久附件服务边界。`ctx.attachments` 校验并持久提交提供方无关的规范化图片,随后返回可序列化的 `ImageAttachmentRef`;消费方绝不会在会话事件中持久保存浏览器路径、对象 URL、提供方 URL 或 base64。
未发送的输入区图片仍是由浏览器持有的临时草稿。`validateImage` 运行相同的准入策略但不执行持久化。`saveImages` 负责批次图片数量和总字节限制,先校验全部成员,再按顺序提交,并且只在完整批次成功后返回引用。后续存储失败不会返回部分引用,但较早写入的不可变内容寻址对象可能保持不可达,直至具备按引用感知的垃圾回收。`AttachmentError.code` 使用封闭的 `AttachmentErrorCode` 字符串联合类型。其 `ImageAdmissionErrorCode` 子集标记可由调用方修正的图片输入失败;`isImageAdmissionError` 在运行时识别该子集,使每个协议适配器可以映射自己的错误词汇。`saveImage` 会在发布任何模型可见的会话事件前提交一张已接受的图片,`readImage` 根据已记录的元数据校验内容寻址对象。调用方可以取消 `readImage`;实现会在后端读取与校验工作的边界观察取消,并保留取消语义,而不会将其转换为存储失败。
未发送的输入区图片仍是由浏览器持有的临时草稿。`validateImage` 运行完整准入策略但不执行持久化。`saveImages` 负责批次图片数量和总字节限制,在发布任何成员前准备全部规范化附件,然后按顺序提交,并且只在完整批次成功后返回引用。后续存储失败不会返回部分引用,但较早写入的不可变内容寻址对象可能保持不可达,直至具备按引用感知的垃圾回收。`AttachmentError.code` 使用封闭的 `AttachmentErrorCode` 字符串联合类型。其 `ImageAdmissionErrorCode` 子集标记可由调用方修正的图片输入失败;`isImageAdmissionError` 在运行时识别该子集,使每个协议适配器可以映射自己的错误词汇。`saveImage` 会在发布任何模型可见的会话事件前提交一张已接受的图片,并直接返回 `ImageAttachmentRef`。规范化过程缩小图片时,引用会通过 `originalDimensions` 记录应用方向后的输入尺寸。`readImage` 根据已记录的元数据校验规范化附件。`readImageRequest` 确定性派生路由所需的请求版本,其身份覆盖附件 ID、变换策略版本、像素和字节预算及编码参数。调用方通过 `Promise.all(refs.map(...))` 组合有序批次,本地实现仍通过实例级限流器、缓存和 singleflight 限制压缩并发。调用方可以取消读取和投影;实现保留取消结果,不把它转换为存储失败。
`admitEncodedImages(attachments, images)` 是每个接受浏览器上传的 RPC 端点(会话 prompt 端点与命令执行器)共用的 wire 入口:它对每个成员强制执行规范 base64,随后把批量准入——限额、校验、有序提交——委托给 `saveImages`。base64 上传形式为 `EncodedImageAttachment`,从 `@deepseek-ai/dsh-attachment/types` 导出,供 wire 契约引用。
## 模型体验
该包通过角色无关的核心 `ImageBlock`,以及解析其持久引用的提供方适配器,间接影响模型。
该包通过角色无关的核心 `ImageBlock`,以及持久引用解析为确定请求版本的提供方适配器,间接影响模型。请求描述会公开完整附件 ID 和实际请求尺寸。
#### KV 缓存影响
@@ -13,3 +13,15 @@ export type AttachmentId = Branded<'AttachmentId'>
export function AttachmentId(value: string): AttachmentId {
return value as AttachmentId
}
/** Opaque deterministic identity for one request-image transformation. */
export type ImageVariantId = Branded<'ImageVariantId'>
/**
* Brand a validated request-image transformation identifier.
* @param value - attachment-provider-produced opaque identifier.
* @returns the branded identifier.
*/
export function ImageVariantId(value: string): ImageVariantId {
return value as ImageVariantId
}
@@ -23,6 +23,7 @@ export type AttachmentErrorCode =
| 'ATTACHMENT_WRITE_FAILED'
| 'ATTACHMENT_NOT_FOUND'
| 'ATTACHMENT_READ_FAILED'
| 'ATTACHMENT_PROJECTION_UNSUPPORTED'
/** Runtime membership for structurally compatible errors crossing package boundaries. */
const IMAGE_ADMISSION_ERROR_CODE_SET: ReadonlySet<string> = new Set(IMAGE_ADMISSION_ERROR_CODES)
+42 -4
View File
@@ -5,11 +5,13 @@ import { AttachmentError } from './error.ts'
import type {
ImageAttachmentLimits,
ImageAttachmentRef,
ImageRequestPolicy,
RequestImageAttachment,
SaveImageAttachment,
StoredImageAttachment,
} from './types.ts'
export { AttachmentId } from './brand.ts'
export { AttachmentId, ImageVariantId } from './brand.ts'
export { AttachmentError, isImageAdmissionError } from './error.ts'
export type { AttachmentErrorCode, ImageAdmissionErrorCode } from './error.ts'
export { admitEncodedImages } from './admission.ts'
@@ -18,7 +20,9 @@ export type {
EncodedImageAttachment,
ImageAttachmentLimits,
ImageAttachmentRef,
ImageRequestPolicy,
ImageMediaType,
RequestImageAttachment,
SaveImageAttachment,
StoredImageAttachment,
} from './types.ts'
@@ -54,7 +58,7 @@ export abstract class AttachmentStore extends Service {
* @param inputs - encoded images in their owning message order.
* @returns durable references in the exact input order.
*/
async saveImages(inputs: readonly SaveImageAttachment[]): Promise<readonly ImageAttachmentRef[]> {
protected validateImageBatch(inputs: readonly SaveImageAttachment[]): void {
const { maxImagesPerMessage, maxMessageImageBytes, mediaTypes } = this.imageLimits
if (inputs.length > maxImagesPerMessage) {
throw new AttachmentError('Image batch exceeds the configured image-count limit.', 'TOO_MANY_IMAGES')
@@ -68,6 +72,15 @@ export abstract class AttachmentStore extends Service {
throw new AttachmentError(`Image type ${input.mediaType} is not accepted by this deployment.`, 'UNSUPPORTED_IMAGE_TYPE')
}
}
}
/**
* Validate and durably commit one ordered image batch.
* @param inputs - encoded images in owning-message order.
* @returns durable normalized attachment references in the same order after every member succeeds.
*/
async saveImages(inputs: readonly SaveImageAttachment[]): Promise<readonly ImageAttachmentRef[]> {
this.validateImageBatch(inputs)
for (const input of inputs) await this.validateImage(input)
const refs: ImageAttachmentRef[] = []
@@ -77,8 +90,11 @@ export abstract class AttachmentStore extends Service {
/**
* Validate and durably commit one image before its owning session event is appended.
* The returned reference describes the persisted normalized image. When
* normalization reduces the raster, its `originalDimensions` records the
* orientation-applied input dimensions.
* @param input - encoded bytes, declared media type, and optional display name.
* @returns a durable content-addressed reference.
* @returns the durable content-addressed normalized image reference.
*/
abstract saveImage(input: SaveImageAttachment): Promise<ImageAttachmentRef>
@@ -86,10 +102,32 @@ export abstract class AttachmentStore extends Service {
* Read one image and verify that bytes still match the recorded reference.
* @param ref - durable reference from the session log.
* @param signal - optional cancellation for backend read and verification work.
* @returns the verified bytes and canonical reference.
* @returns the verified bytes and normalized attachment reference.
* @throws the signal reason when aborted, or a storage error when verification fails.
*/
abstract readImage(ref: ImageAttachmentRef, signal?: AbortSignal): Promise<StoredImageAttachment>
/**
* Generate or read one deterministic model-request version from the stored normalized image.
* @param ref - durable provider-independent normalized attachment reference.
* @param policy - exact route pixel and encoded-byte budget.
* @param signal - optional cancellation.
* @returns request bytes and the cache/upload identity covering every transform input.
*/
readImageRequest(
ref: ImageAttachmentRef,
policy: ImageRequestPolicy,
signal?: AbortSignal,
): Promise<RequestImageAttachment> {
signal?.throwIfAborted()
void ref
void policy
return Promise.reject(new AttachmentError(
'The mounted attachment provider cannot derive model-request images.',
'ATTACHMENT_PROJECTION_UNSUPPORTED',
))
}
}
export default AttachmentStore
+38 -2
View File
@@ -1,13 +1,13 @@
/** Durable attachment vocabulary. @module @deepseek-ai/dsh-attachment/types */
import type { AttachmentId } from './brand.ts'
import type { AttachmentId, ImageVariantId } from './brand.ts'
export type { AttachmentId } from './brand.ts'
/** Raster image formats accepted by the version-one attachment path. */
export type ImageMediaType = 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif'
/** Durable, serializable metadata for one immutable image object. */
/** Durable, serializable reference to one immutable normalized image. */
export interface ImageAttachmentRef {
/** Opaque storage identifier; never a filesystem path or bearer URL. */
attachmentId: AttachmentId
@@ -21,6 +21,14 @@ export interface ImageAttachmentRef {
height: number
/** Optional display name stripped of local path information. */
name?: string
/**
* Input dimensions after applying EXIF orientation and before normalization
* scaling. Present only when normalization reduced the image.
*/
originalDimensions?: {
width: number
height: number
}
}
/** Deployment-resolved limits used by upload admission and request buffering. */
@@ -58,3 +66,31 @@ export interface StoredImageAttachment {
ref: ImageAttachmentRef
data: Uint8Array
}
/** Deterministic request-image policy selected by one exact model route. */
export interface ImageRequestPolicy {
/** Maximum width multiplied by height after aspect-preserving projection. */
maxPixels: number
/** Encoded-byte cap before base64 expansion or Files API upload. */
maxBytes: number
}
/** Cached request version derived from one provider-independent normalized attachment. */
export interface RequestImageAttachment {
/** Cache and upload-index key over the attachment id, policy, and fixed encoder parameters. */
variantId: ImageVariantId
/** Durable normalized attachment from which this request version was derived. */
attachment: ImageAttachmentRef
/** Encoded request bytes. */
data: Uint8Array
mediaType: ImageMediaType
bytes: number
width: number
height: number
/** Provider-compatible sample depth proven after request encoding. */
depth: 'uchar'
/** Provider-compatible color space proven after request encoding. */
space: 'srgb'
/** Whether the encoded request version retains an alpha channel. */
hasAlpha: boolean
}
@@ -3,9 +3,12 @@ import { describe, expect, it } from 'vitest'
import AttachmentStore, {
AttachmentError,
AttachmentId,
ImageVariantId,
isImageAdmissionError,
type ImageAttachmentRef,
type ImageMediaType,
type ImageRequestPolicy,
type RequestImageAttachment,
type SaveImageAttachment,
type StoredImageAttachment,
} from '../src/index.ts'
@@ -48,6 +51,41 @@ class RecordingStore extends AttachmentStore {
readImage(_ref: ImageAttachmentRef): Promise<StoredImageAttachment> {
throw new Error('not used')
}
override readImageRequest(
ref: ImageAttachmentRef,
_policy: ImageRequestPolicy,
): Promise<RequestImageAttachment> {
this.calls.push(`request:${ref.name}`)
return Promise.resolve({
variantId: ImageVariantId(`sha256:${String(ref.bytes).padStart(64, '0')}`),
attachment: ref,
data: Uint8Array.of(ref.bytes),
mediaType: ref.mediaType,
bytes: 1,
width: ref.width,
height: ref.height,
depth: 'uchar',
space: 'srgb',
hasAlpha: false,
})
}
}
class UnsupportedProjectionStore extends AttachmentStore {
readonly imageLimits = LIMITS
validateImage(): Promise<void> {
return Promise.resolve()
}
saveImage(): Promise<ImageAttachmentRef> {
throw new Error('not used')
}
readImage(): Promise<StoredImageAttachment> {
throw new Error('not used')
}
}
function image(value: number, mediaType: ImageMediaType = 'image/png'): SaveImageAttachment {
@@ -97,6 +135,19 @@ describe('AttachmentStore.saveImages', () => {
})
})
describe('AttachmentStore.readImageRequest', () => {
it('reports unsupported request projection while preserving cancellation', async () => {
const store = new UnsupportedProjectionStore(new Context())
const ref = await new RecordingStore(new Context()).saveImage(image(1))
await expect(store.readImageRequest(ref, { maxPixels: 1, maxBytes: 1 }))
.rejects.toMatchObject({ code: 'ATTACHMENT_PROJECTION_UNSUPPORTED' })
const controller = new AbortController()
const reason = new Error('cancel unsupported projection')
controller.abort(reason)
expect(() => store.readImageRequest(ref, { maxPixels: 1, maxBytes: 1 }, controller.signal)).toThrow(reason)
})
})
describe('isImageAdmissionError', () => {
it('separates caller-correctable image admission failures from storage faults', () => {
expect(isImageAdmissionError(new AttachmentError('bad bytes', 'INVALID_IMAGE'))).toBe(true)
+2 -2
View File
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/client/connection/README.md
README.md: a7562b9dac57930b1abc0b76b9079a6865a38b35
README.zh.md: 24c56e598ebd4b5ca39e433c5782399909f528b8
README.md: 71ef204a589bb67c15ccab58d3cac5a13782ce27
README.zh.md: 6d33ac3c13cdfceeba6e7472b618084267d09bbc
+1 -1
View File
@@ -23,4 +23,4 @@ None; this package neither assembles nor sends a provider request.
## Known Limitations and Deferred Work
- **History resumes an unattached session** — opening history may create the host-side agent and add latency to the first open; there is no persistence-only read path.
- **The `/api` bridge buffers each request body in memory**`maxRequestBodyBytes` (default 160 MiB, sized for the default 100 MiB aggregate image limit after base64 expansion plus envelope headroom) is therefore also the per-request resident bound; a streaming body path would be needed to lower it without shrinking the image limits.
- **The `/api` bridge buffers each request body in memory**`maxRequestBodyBytes` (default 300 MiB, sized for the default 200 MiB aggregate image limit after base64 expansion plus envelope headroom) is therefore also the per-request resident bound; a streaming body path would be needed to lower it without shrinking the image limits.
+1 -1
View File
@@ -23,4 +23,4 @@ node 半侧在桥接或 upgrade 前守卫 `/api` 下的每个入口(`src/api-r
## 已知限制与暂缓事项
- **History 会恢复未附加的会话**:打开 history 可能创建宿主侧 agent,并增加首次打开的延迟;没有仅从持久化读取的路径。
- **`/api` 桥把每个请求体整体缓冲在内存里**`maxRequestBodyBytes`(默认 160 MiB,按默认 100 MiB 图片总量上限经 base64 膨胀加信封余量得出)因此同时是单请求的驻留内存上界;要降低它而不缩小图片限额,需要流式请求体路径。
- **`/api` 桥把每个请求体整体缓冲在内存里**`maxRequestBodyBytes`(默认 300 MiB,按默认 200 MiB 图片总量上限经 base64 膨胀加信封余量得出)因此同时是单请求的驻留内存上界;要降低它而不缩小图片限额,需要流式请求体路径。
@@ -6,10 +6,10 @@
import type { IncomingMessage, ServerResponse } from 'node:http'
/** Default carrier cap for all HTTP RPC bodies: sized for the default
* aggregate image limit (100 MiB) after base64 expansion plus envelope
* headroom (~134.3 MiB required), rounded up for slack. The bridge buffers
* aggregate image limit (200 MiB) after base64 expansion plus envelope
* headroom (~267.7 MiB required), rounded up for slack. The bridge buffers
* each body in memory, so this cap is also the per-request resident bound. */
export const DEFAULT_MAX_REQUEST_BODY_BYTES = 160 * 1024 * 1024
export const DEFAULT_MAX_REQUEST_BODY_BYTES = 300 * 1024 * 1024
/** Transport-independent request handler consumed by the Host HTTP bridge. */
export interface FetchHandler {
+1 -1
View File
@@ -57,7 +57,7 @@ export interface ConnectionConfig {
* that is not a bare, canonical authority fails the plugin load.
*/
trustedHosts?: string[]
/** Maximum buffered JSON body for every `/api` request. */
/** Maximum buffered JSON body for every `/api` request. Default: 300 MiB. */
maxRequestBodyBytes?: number
}
@@ -11,6 +11,7 @@ import type { AttachmentStore } from '@deepseek-ai/dsh-attachment'
import { RpcId, type ClientRequest } from '@deepseek-ai/dsh-host-apiproxy/api'
import type { WebServer, WebRoute, WebUpgradeRoute } from '@deepseek-ai/dsh-host-webserver'
import { API_PATH, apply, HOST_EVENTS_PATH, inject, MUX_EVENTS_PATH, type HostConnectionHandle } from '../src/index.ts'
import { DEFAULT_MAX_REQUEST_BODY_BYTES } from '../src/http-bridge.ts'
/** Structural webServer fake recording both route registries. */
function fakeHttpServer(
@@ -90,6 +91,11 @@ async function mounted(config?: { trustedHosts?: string[] }): Promise<{
}
describe('connection node half', () => {
it('reserves enough default carrier capacity for the 200 MiB image batch', () => {
expect(DEFAULT_MAX_REQUEST_BODY_BYTES).toBe(300 * 1024 * 1024)
expect(DEFAULT_MAX_REQUEST_BODY_BYTES).toBeGreaterThan(Math.ceil(200 * 1024 * 1024 * 4 / 3) + 1024 * 1024)
})
it('fails loud when the carrier cap cannot hold the configured image batch', () => {
const ctx = new Context()
const routes: WebRoute[] = []
@@ -438,23 +438,29 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [
},
{
signature: 'async saveImages(inputs: readonly SaveImageAttachment[]): Promise<readonly ImageAttachmentRef[]>',
description: 'Validate one ordered image batch before committing any member. Validation failures start no writes; storage failures return no partial references, although already published content-addressed objects may stay unreachable until a future retention policy collects them.',
parameters: [{ name: 'inputs', description: 'encoded images in their owning message order.' }],
returns: 'durable references in the exact input order.',
description: 'Validate and durably commit one ordered image batch.',
parameters: [{ name: 'inputs', description: 'encoded images in owning-message order.' }],
returns: 'durable normalized attachment references in the same order after every member succeeds.',
},
{
signature: 'abstract saveImage(input: SaveImageAttachment): Promise<ImageAttachmentRef>',
description: 'Validate and durably commit one image before its owning session event is appended.',
description: 'Validate and durably commit one image before its owning session event is appended. The returned reference describes the persisted normalized image. When normalization reduces the raster, its `originalDimensions` records the orientation-applied input dimensions.',
parameters: [{ name: 'input', description: 'encoded bytes, declared media type, and optional display name.' }],
returns: 'a durable content-addressed reference.',
returns: 'the durable content-addressed normalized image reference.',
},
{
signature: 'abstract readImage(ref: ImageAttachmentRef, signal?: AbortSignal): Promise<StoredImageAttachment>',
description: 'Read one image and verify that bytes still match the recorded reference.',
parameters: [{ name: 'ref', description: 'durable reference from the session log.' }, { name: 'signal', description: 'optional cancellation for backend read and verification work.' }],
returns: 'the verified bytes and canonical reference.',
returns: 'the verified bytes and normalized attachment reference.',
throws: ['the signal reason when aborted, or a storage error when verification fails.'],
},
{
signature: 'readImageRequest( ref: ImageAttachmentRef, policy: ImageRequestPolicy, signal?: AbortSignal, ): Promise<RequestImageAttachment>',
description: 'Generate or read one deterministic model-request version from the stored normalized image.',
parameters: [{ name: 'ref', description: 'durable provider-independent normalized attachment reference.' }, { name: 'policy', description: 'exact route pixel and encoded-byte budget.' }, { name: 'signal', description: 'optional cancellation.' }],
returns: 'request bytes and the cache/upload identity covering every transform input.',
},
],
},
{
@@ -3445,7 +3451,7 @@ export const TYPE_API: readonly TypeApiEntry[] = [
},
{
name: 'ImageAttachmentRef',
declaration: 'export interface ImageAttachmentRef {\n attachmentId: AttachmentId;\n mediaType: ImageMediaType;\n bytes: number;\n width: number;\n height: number;\n name?: string;\n}',
declaration: 'export interface ImageAttachmentRef {\n attachmentId: AttachmentId;\n mediaType: ImageMediaType;\n bytes: number;\n width: number;\n height: number;\n name?: string;\n originalDimensions?: {\n width: number;\n height: number;\n };\n}',
},
{
name: 'ImageBlock',
@@ -3455,6 +3461,14 @@ export const TYPE_API: readonly TypeApiEntry[] = [
name: 'ImageMediaType',
declaration: 'export type ImageMediaType = \'image/png\' | \'image/jpeg\' | \'image/webp\' | \'image/gif\';',
},
{
name: 'ImageRequestPolicy',
declaration: 'export interface ImageRequestPolicy {\n maxPixels: number;\n maxBytes: number;\n}',
},
{
name: 'ImageVariantId',
declaration: 'export type ImageVariantId = Branded<\'ImageVariantId\'>;',
},
{
name: 'Inbox',
declaration: 'export class Inbox {\n constructor(private readonly session: Session, private readonly notifications: InboxNotifications);\n get nextTurn(): readonly UserMessage[];\n get nextStep(): readonly UserMessage[];\n get hasPending(): boolean;\n clear(): void;\n claim(target: InboxTarget, turn: number): UserMessage[];\n append(target: InboxTarget, message: UserMessage): void;\n prepend(target: InboxTarget, message: UserMessage): void;\n replace(messageId: MessageId, newMessage: UserMessage): boolean;\n remove(messageId: MessageId): boolean;\n splice(target: InboxTarget, start: number, deleteCount: number, inserted: UserMessage[]): UserMessage[];\n}',
@@ -3581,7 +3595,7 @@ export const TYPE_API: readonly TypeApiEntry[] = [
},
{
name: 'LlmAdapter',
declaration: 'export abstract class LlmAdapter {\n providerInfo(provider: string): LlmProviderInfo;\n providerRetryPolicy(_provider: string): ResolvedRetryPolicy | undefined;\n listModels(_provider: string): Promise<readonly LlmModelInfo[]>;\n resolveModel(provider: string, model: string, _signal?: AbortSignal): Promise<LlmResolvedModelInfo>;\n abstract stream(options: GenerateOptions): AsyncIterable<StreamChunk>;\n}',
declaration: 'export abstract class LlmAdapter {\n providerInfo(provider: string): LlmProviderInfo;\n providerRetryPolicy(_provider: string): ResolvedRetryPolicy | undefined;\n listModels(_provider: string): Promise<readonly LlmModelInfo[]>;\n resolveModel(provider: string, model: string, _signal?: AbortSignal): Promise<LlmResolvedModelInfo>;\n async prepareCall(provider: string, model: string, signal?: AbortSignal): Promise<PreparedAdapterCall>;\n abstract stream(options: GenerateOptions): AsyncIterable<StreamChunk>;\n}',
},
{
name: 'LlmCallConfig',
@@ -3799,9 +3813,13 @@ export const TYPE_API: readonly TypeApiEntry[] = [
name: 'PostToolDecision',
declaration: 'export type PostToolDecision = {\n kind: \'accept\';\n content?: ContentBlock[];\n value?: never;\n additionalContexts?: UserMessage[];\n} | {\n kind: \'accept\';\n value: JsonValue;\n content?: never;\n additionalContexts?: UserMessage[];\n} | {\n kind: \'block\';\n feedback: ContentBlock[];\n additionalContexts?: UserMessage[];\n};',
},
{
name: 'PreparedAdapterCall',
declaration: 'export interface PreparedAdapterCall {\n readonly model: LlmResolvedModelInfo;\n stream(options: GenerateOptions): AsyncIterable<StreamChunk>;\n}',
},
{
name: 'PreparedLlmCall',
declaration: 'export interface PreparedLlmCall {\n readonly config: LlmCallConfig;\n readonly retryPolicy: ResolvedRetryPolicy;\n readonly context?: LlmModelContext;\n readonly adapterDefaults: LlmCallConfigAdapterDefaults;\n stream(options: GenerateOptions): AsyncIterable<StreamChunk>;\n}',
declaration: 'export interface PreparedLlmCall {\n readonly config: LlmCallConfig;\n readonly retryPolicy: ResolvedRetryPolicy;\n readonly context?: LlmModelContext;\n readonly inputModalities?: readonly ModelModality[];\n readonly adapterDefaults: LlmCallConfigAdapterDefaults;\n stream(options: GenerateOptions): AsyncIterable<StreamChunk>;\n}',
},
{
name: 'PreparedReferencedMessage',
@@ -3911,6 +3929,10 @@ export const TYPE_API: readonly TypeApiEntry[] = [
name: 'RequestHeaderReason',
declaration: 'export type RequestHeaderReason = \'initial\' | \'resume\' | \'change\';',
},
{
name: 'RequestImageAttachment',
declaration: 'export interface RequestImageAttachment {\n variantId: ImageVariantId;\n attachment: ImageAttachmentRef;\n data: Uint8Array;\n mediaType: ImageMediaType;\n bytes: number;\n width: number;\n height: number;\n depth: \'uchar\';\n space: \'srgb\';\n hasAlpha: boolean;\n}',
},
{
name: 'RequestRunOutcome',
declaration: 'export type RequestRunOutcome = \'approved\' | \'completed\' | \'rejected\' | \'cancelled\' | \'failed\';',
+2 -2
View File
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/fs/tool-fs/README.md
README.md: ce7c0ea9070e30c1e6b538933ff5c4605b8d59cc
README.zh.md: 88d27289a7ef087aa8ad2791fc9b9e0d3e1fba2e
README.md: 763cb831233da5b1f14c73e353920e9d6a87ced9
README.zh.md: aa55de59452e7779ef278748abac17837800ba02
+7 -7
View File
@@ -14,7 +14,7 @@ await ctx.plugin(ToolFs) // this package — re
`@deepseek-ai/dsh-fs-observation-policy` is **optional**: omit it and the tools run against the bare provider (unconditional write/overwrite/edit, no observed-state). A deployment that loads these tools is expected to also load it, so the behavior is read-before-write/edit.
`read_image` registers only while a durable `ctx.attachments` service is mounted — without one the deployment cannot commit image bytes, so the tool never appears. Execution additionally requires the exact routed model to declare `image` input (resolved through `ctx.llm.resolveModelInfo` from the session's latest request header, falling back to agent options); an unknown or text-only route gets a refusal result before any filesystem I/O, so a text route's durable history stays free of image blocks.
`read_image` registers only while a durable `ctx.attachments` service is mounted. Execution additionally requires the exact routed model to declare `image` input, resolved through `ctx.llm.resolveModelInfo` from the session's latest request header and then from agent options.
## Config
@@ -32,13 +32,13 @@ All keys are optional; the defaults are the shipped read caps.
| Tool | Arguments | Behavior |
|---|---|---|
| `read` | `file_path`, `offset?`, `limit?` | Line-numbered UTF-8 content with a pagination footer. `offset` is 1-based; `limit` defaults to and caps at the configured `readLimit` (2000). |
| `read_image` | `file_path` | Reads a PNG/JPEG/WebP/GIF file through the bounded byte seam, persists it through `ctx.attachments.saveImage`, and returns an image block beside a small metadata envelope. It succeeds only when the exact routed model declares image input. |
| `read_image` | `file_path` | Reads a PNG/JPEG/WebP/GIF file through the bounded byte seam, persists it through `ctx.attachments.saveImage`, and returns an image block beside a small metadata envelope. Harness validates and downscales large supported images before the next model request, so the model can read the source directly without first creating a thumbnail. It succeeds only when the exact routed model declares image input. |
| `write` | `file_path`, `content` | Create or fully replace a file. With the policy plugin: overwriting an existing file requires a prior `read` at the unchanged version; creating a new file does not. Without it: unconditional. |
| `edit` | `file_path`, non-empty `old_string`, `new_string`, `replace_all?` | Literal replacement; unique match required unless `replace_all` is true. With the policy plugin: requires a prior `read` (any window) and the file unchanged since. Without it: unconditional. |
Field names are snake_case to match Claude Code and existing harness tool schemas.
Canonical successes are `read``{ path, offset, lines: [{ number, text }], totalLines }`, `read_image``{ path, image: { attachmentId, mediaType, bytes, width, height, name? } }`, `write``{ path, operation: 'create' | 'update', before: string | null, after }`, and `edit``{ path, before, after }`. Native renderers preserve the line-numbered read and mutation acknowledgements below. `write`/`edit` derive replayable diff-card metadata, and `read` derives a replayable read-card window `{ path, offset, lines, totalLines, lang? }`, from these canonical values; the canonical values themselves are execution-local and are not added to `tool/result`, only the derived presentation metadata is persisted.
Structured successes are `read``{ path, offset, lines: [{ number, text }], totalLines }`, `read_image``{ path, image: { attachmentId, mediaType, bytes, width, height, name?, originalDimensions?: { width, height } } }`, `write``{ path, operation: 'create' | 'update', before: string | null, after }`, and `edit``{ path, before, after }`. `originalDimensions` appears only when normalization downscaled the submitted raster and records its orientation-applied input size. Native renderers preserve the line-numbered read and mutation acknowledgements below. `write`/`edit` derive replayable diff-card metadata, and `read` derives a replayable read-card window `{ path, offset, lines, totalLines, lang? }`; execution-local structured values are not added to `tool/result`, while image renderers emit the durable image blocks that the result logs.
## The tool is the executor; policy is an event gate
@@ -99,7 +99,7 @@ Prefix-stable while the plugin scope and guidance text are unchanged. Tool restr
#### What the model sees
The model sees the generated [`read`, `read_image`, `write`, and `edit` schemas](../../../docs/tool-catalog.md#deepseek-aidsh-tool-fs), with snake_case arguments. `read_image` appears only while a durable attachment store is mounted; the schema itself is route-independent, and the strict gate refuses at execution. Scoped tool restrictions can remove any definition for one agent.
The model sees the generated [`read`, `read_image`, `write`, and `edit` schemas](../../../docs/tool-catalog.md#deepseek-aidsh-tool-fs), with snake_case arguments. The image tool appears only while a durable attachment store is mounted; its schema is route-independent, and the strict gate refuses at execution. Scoped tool restrictions can remove any definition for one agent.
#### Token effect
@@ -127,7 +127,7 @@ Append-only; newly visible content follows the reusable request prefix and does
#### What the model sees
A successful `read_image` returns `<path><displayPath></path>`, `<type>image</type>`, and a `<content>` envelope naming the media type, dimensions, and byte size, followed by the image itself as a native image block. The session log stores only the durable `sha256:` attachment reference; the routed provider re-reads and digest-verifies the bytes on each request.
A successful `read_image` returns `<path><displayPath></path>`, `<type>image</type>`, and a `<content>` envelope naming the media type, normalized dimensions, and byte size, followed by the image itself as a native image block. The result is logged with its durable reference before the next model request.
#### Token effect
@@ -155,7 +155,7 @@ Append-only; newly visible content follows the reusable request prefix and does
#### What the model sees
Failures are normalized as `Error: <message>`. This package's stable validation and read messages are `file_path must be a non-empty string`, `limit must be less than or equal to <max>`, `old_string must be a non-empty string`, `old_string and new_string must differ`, `cannot read "<path>": not found`, `cannot read "<path>": not a regular file`, `offset <offset> is out of range for "<path>" (<total> lines)`, `cannot read "<path>": read_image only accepts PNG/JPEG/WebP/GIF paths`, `cannot read "<path>" as an image: model "<model>" does not declare image input; switch to an image-capable model to read images`, and the mismatch repair `cannot read "<path>": the <ext> extension declares <type>, but the bytes use a different image format; rename the file to match its actual format if it is PNG/JPEG/WebP/GIF, or convert it to one of those formats`; provider and policy templates are quoted in their package READMEs. Guarded-mutation failures additionally carry their recovery instruction in the message, appended by this package's model-facing error wrapper: `FS_STALE_VERSION` gets `— re-read the file, then retry`, and `FS_NOT_OBSERVED` gets `— read the file, then retry`; the structured code is preserved. After that reread confirms absence, edit reports `FS_NOT_FOUND` instead of repeating a stale remedy, while write uses guarded creation.
Failures are normalized as `Error: <message>`. This package's stable validation and read messages are `file_path must be a non-empty string`, `limit must be less than or equal to <max>`, `old_string must be a non-empty string`, `old_string and new_string must differ`, `cannot read "<path>": not found`, `cannot read "<path>": not a regular file`, `offset <offset> is out of range for "<path>" (<total> lines)`, `cannot read "<path>": read_image only accepts PNG/JPEG/WebP/GIF paths`, `cannot read "<path>" as an image: model "<model>" does not declare image input; switch to an image-capable model to read images`, and the mismatch repair `cannot read "<path>": the <ext> extension declares <type>, but the bytes use a different image format; rename the file to match its actual format if it is PNG/JPEG/WebP/GIF, or convert it to one of those formats`. A failed 16-bit conversion reports `cannot read "<path>": the 16-bit PNG could not be converted to the normalized 8-bit sRGB form; convert it to an 8-bit PNG/JPEG/WebP and retry`. Provider and policy templates are quoted in their package READMEs. Guarded-mutation failures additionally carry their recovery instruction in the message, appended by this package's model-facing error wrapper: `FS_STALE_VERSION` gets `— re-read the file, then retry`, and `FS_NOT_OBSERVED` gets `— read the file, then retry`; the structured code is preserved. After that reread confirms absence, edit reports `FS_NOT_FOUND` instead of repeating a stale remedy, while write uses guarded creation.
#### Token effect
@@ -169,7 +169,7 @@ Append-only; newly visible content follows the reusable request prefix and does
- **No model-facing directory listing ships**`ctx.fs.listDir` serves provider code such as skill discovery, while the sibling [`dsh-tool-fs-search`](../tool-fs-search/) package supplies ripgrep-backed `glob` and `grep` rather than extending the filesystem seam.
- **`read` handles UTF-8 text files only** — images use the separate extension-routed `read_image` tool; PDF, audio, and video remain deferred. A directory target is `FS_NOT_REGULAR_FILE`.
- **The route gate races a concurrent model switch**`read_image` checks the latest routed model at execution; a switch committed between that check and the next request can leave an image block on a route that rejects image content. The Web host already refuses switching an image-bearing session to a text-only model; other front doors own their equivalent guard.
- **Extension-declared media type** — the extension selects the declared type and the attachment store's magic-byte validation stays authoritative; a correctly formatted image under a wrong extension is refused with the rename remedy rather than sniffed.
- **No inline image preview on the tool-result card** — UI surfaces render the image result generically (the durable reference, not pixels); inline rendering is deferred to the UI packages.
- **No attachment-region tool** — an agent may crop an image through other available tools when it has a filesystem path. A pasted or dragged image without a path cannot be re-read at a higher resolution.
- **No timeout surface**`read`/`write`/`edit` take no timeout argument and declare no `timeout-policy` budget; cancellation rides `exec.signal` only ([provider rationale](../README.md#no-timeouts-on-file-io)).
+8 -8
View File
@@ -2,7 +2,7 @@
[English](README.md) | 中文
**面向模型的文件系统工具**`read``read_image``write``edit`)及其**执行器**。这是文件系统栈的消费方层:拥有工具名称、JSON Schema、参数校验、提示词段、**读取窗口逻辑**和结果格式化。它**直接**通过 `ctx.fs` 提供方约定([`@deepseek-ai/dsh-fs`](../fs))读取写入编辑。新鲜度观察策略由独立插件([`@deepseek-ai/dsh-fs-observation-policy`](../fs-observation-policy))通过 `fs/*` 事件门禁贡献;工具不与其方法耦合。使用施加沙箱限制的提供方时,逐会话执行需要共享沙箱策略服务,工具还会为文件系统变更提供升权路径。
**面向模型的文件系统工具**`read``read_image``write``edit`)及其**执行器**。这是文件系统栈的消费方层:拥有工具名称、JSON Schema、参数校验、提示词段、**读取窗口逻辑**和结果格式化。它**直接**通过 `ctx.fs` 提供方约定([`@deepseek-ai/dsh-fs`](../fs))读取写入编辑。新鲜度观察策略由独立插件([`@deepseek-ai/dsh-fs-observation-policy`](../fs-observation-policy))通过 `fs/*` 事件门禁贡献;工具不与其方法耦合。使用施加沙箱限制的提供方时,逐会话执行需要共享沙箱策略服务,工具还会为文件系统变更提供升权路径。
```ts ignore-check
// Default deployment: a ctx.fs provider, the policy plugin, then the tools.
@@ -14,7 +14,7 @@ await ctx.plugin(ToolFs) // this package — re
`@deepseek-ai/dsh-fs-observation-policy` 是**可选的**:省略时,工具直接使用裸提供方(无条件写入/覆盖/编辑,无已观察状态)。加载这些工具的部署也应加载该插件,从而提供写入/编辑前读取行为。
`read_image` 只在持久 `ctx.attachments` 服务已挂载时注册:没有它,部署无法持久提交图像字节,工具就不会出现。执行时还要求确切路由的模型声明 `image` 输入通过 `ctx.llm.resolveModelInfo` 从会话最新请求 header 解析,缺失时回退到 agent 选项);未知或纯文本路由在任何文件系统 I/O 之前就得到拒绝结果,因此文本路由的持久历史不会出现图像块
`read_image` 只在持久 `ctx.attachments` 服务已挂载时注册。执行时还要求确切路由的模型声明 `image` 输入通过 `ctx.llm.resolveModelInfo` 依次从会话最新请求 header 和 agent 选项解析。
## 配置
@@ -32,13 +32,13 @@ await ctx.plugin(ToolFs) // this package — re
| 工具 | 参数 | 行为 |
|---|---|---|
| `read` | `file_path``offset?``limit?` | 带行号的 UTF-8 内容和分页 footer。`offset` 从 1 开始;`limit` 默认为配置的 `readLimit`2000),上限也为该值。 |
| `read_image` | `file_path` | 通过有界字节 seam 读取 PNG/JPEG/WebP/GIF 文件,经 `ctx.attachments.saveImage` 持久保存,并在小型元数据信封旁返回图像块。只有确切路由的模型声明图像输入时才会成功。 |
| `read_image` | `file_path` | 通过有界字节 seam 读取 PNG/JPEG/WebP/GIF 文件,经 `ctx.attachments.saveImage` 持久保存,并在小型元数据信封旁返回图像块。Harness 会在下一次模型请求前校验并缩小受支持的大图,因此模型可以直接读取源文件,无需先创建缩略图。只有确切路由的模型声明图像输入时才会成功。 |
| `write` | `file_path``content` | 创建文件或完整替换文件。有策略插件时:覆盖现有文件要求先在未变版本上执行 `read`;创建新文件不需要。没有插件时:无条件执行。 |
| `edit` | `file_path`、非空 `old_string``new_string``replace_all?` | 字面量替换;除非 `replace_all` 为 true,否则要求唯一匹配。有策略插件时:要求先执行 `read`(任何窗口),且文件此后未变。没有插件时:无条件执行。 |
字段名使用 snake_case,与 Claude Code 和现有 harness 工具 schema 一致。
规范成功值分别为:`read``{ path, offset, lines: [{ number, text }], totalLines }``read_image``{ path, image: { attachmentId, mediaType, bytes, width, height, name? } }``write``{ path, operation: 'create' | 'update', before: string | null, after }``edit``{ path, before, after }`。原生渲染器会保留下方带行号的读取结果和变更确认。`write`/`edit` 从这些规范值派生可回放的 diff 卡片元数据,`read` 派生可回放的读取卡片窗口 `{ path, offset, lines, totalLines, lang? }`规范值本身仅限于本次执行,不会添加到 `tool/result`只有派生出的呈现元数据会被持久化
结构化成功值分别为:`read``{ path, offset, lines: [{ number, text }], totalLines }``read_image``{ path, image: { attachmentId, mediaType, bytes, width, height, name?, originalDimensions?: { width, height } } }``write``{ path, operation: 'create' | 'update', before: string | null, after }``edit``{ path, before, after }``originalDimensions` 只在规范化过程缩小提交光栅时出现,并记录应用方向后的输入尺寸。原生渲染器会保留下方带行号的读取结果和变更确认。`write``edit` 从这些值派生可回放的 diff 卡片元数据,`read` 派生可回放的读取卡片窗口 `{ path, offset, lines, totalLines, lang? }`仅用于执行的结构化值不会添加到 `tool/result`图片渲染器则会发出由结果记录的持久图片块
## 工具就是执行器;策略是事件门禁
@@ -99,7 +99,7 @@ Use the edit tool for targeted changes to existing UTF-8 text files. It replaces
#### 模型看到的内容
模型会看到已生成的 [`read`、`read_image`、`write` 和 `edit` schema](../../../docs/tool-catalog.zh.md#deepseek-aidsh-tool-fs),参数使用 snake_case。`read_image` 只在持久附件存储已挂载时出现;schema 本身与路由无关,严格门禁在执行时拒绝。作用域工具限制可以为某个 agent 移除任一定义。
模型会看到已生成的 [`read`、`read_image`、`write` 和 `edit` schema](../../../docs/tool-catalog.zh.md#deepseek-aidsh-tool-fs),参数使用 snake_case。图片工具只在持久附件存储已挂载时出现;schema 本身与路由无关,严格门禁在执行时拒绝。作用域工具限制可以为某个 agent 移除任一定义。
#### Token 影响
@@ -127,7 +127,7 @@ Use the edit tool for targeted changes to existing UTF-8 text files. It replaces
#### 模型看到的内容
成功的 `read_image` 返回 `<path><displayPath></path>``<type>image</type>` 和写明媒体类型、尺寸与字节数的 `<content>` 信封,随后是作为原生图像块的图像本身。会话日志只存储持久的 `sha256:` 附件引用;路由到的提供方在每次请求时重新读取并校验字节摘要
成功的 `read_image` 返回 `<path><displayPath></path>``<type>image</type>` 和写明媒体类型、规范化尺寸与字节数的 `<content>` 信封,随后是作为原生图像块的图像本身。结果会随持久引用写入会话日志,然后才进入下一次模型请求
#### Token 影响
@@ -155,7 +155,7 @@ Use the edit tool for targeted changes to existing UTF-8 text files. It replaces
#### 模型看到的内容
失败会规范化为 `Error: <message>`。本包稳定的校验和读取消息是 `file_path must be a non-empty string``limit must be less than or equal to <max>``old_string must be a non-empty string``old_string and new_string must differ``cannot read "<path>": not found``cannot read "<path>": not a regular file``offset <offset> is out of range for "<path>" (<total> lines)``cannot read "<path>": read_image only accepts PNG/JPEG/WebP/GIF paths``cannot read "<path>" as an image: model "<model>" does not declare image input; switch to an image-capable model to read images`,以及类型不匹配的修复消息 `cannot read "<path>": the <ext> extension declares <type>, but the bytes use a different image format; rename the file to match its actual format if it is PNG/JPEG/WebP/GIF, or convert it to one of those formats`提供方和策略模板在各自包的 README 中逐字列出。防护变更失败还会在消息中携带恢复指令,由本包面向模型的错误包装追加:`FS_STALE_VERSION` 追加 `re-read the file, then retry``FS_NOT_OBSERVED` 追加 `read the file, then retry`;结构化错误码保持不变。该次重新读取确认缺失后,edit 会报告 `FS_NOT_FOUND`不会重复陈旧恢复指令;write 则使用带防护的创建。
失败会规范化为 `Error: <message>`。本包稳定的校验和读取消息是 `file_path must be a non-empty string``limit must be less than or equal to <max>``old_string must be a non-empty string``old_string and new_string must differ``cannot read "<path>": not found``cannot read "<path>": not a regular file``offset <offset> is out of range for "<path>" (<total> lines)``cannot read "<path>": read_image only accepts PNG/JPEG/WebP/GIF paths``cannot read "<path>" as an image: model "<model>" does not declare image input; switch to an image-capable model to read images`,以及类型不匹配的修复消息 `cannot read "<path>": the <ext> extension declares <type>, but the bytes use a different image format; rename the file to match its actual format if it is PNG/JPEG/WebP/GIF, or convert it to one of those formats`。16-bit 转换失败会报告 `cannot read "<path>": the 16-bit PNG could not be converted to the normalized 8-bit sRGB form; convert it to an 8-bit PNG/JPEG/WebP and retry`提供方和策略模板在各自包的 README 中逐字列出。防护变更失败还会在消息中携带恢复指令,由本包面向模型的错误包装追加:`FS_STALE_VERSION` 追加 `re-read the file, then retry``FS_NOT_OBSERVED` 追加 `read the file, then retry`;结构化错误码保持不变。该次重新读取确认缺失后,edit 会报告 `FS_NOT_FOUND`,不会重复陈旧恢复指令;write 则使用带防护的创建。
#### Token 影响
@@ -169,7 +169,7 @@ Use the edit tool for targeted changes to existing UTF-8 text files. It replaces
- **未交付面向模型的目录列表工具**`ctx.fs.listDir` 服务于 skill(技能)发现等提供方代码,同级 [`dsh-tool-fs-search`](../tool-fs-search/) 包则提供基于 ripgrep 的 `glob``grep`,而不是扩展文件系统 seam。
- **`read` 只处理 UTF-8 文本文件**:图像使用独立的、按扩展名路由的 `read_image` 工具;PDF、音频和视频仍延期处理。目录目标为 `FS_NOT_REGULAR_FILE`
- **路由门禁与并发模型切换存在竞态**`read_image` 在执行时检查最新路由的模型;在该检查与下一次请求之间提交的切换,可能让图像块落在拒绝图像内容的路由上。Web 宿主已拒绝把含图像的会话切到纯文本模型;其他前端拥有各自的等价防护。
- **媒体类型按扩展名声明**:扩展名选择声明类型,附件存储的魔数校验保持权威;扩展名错误但格式正确的图像会得到改名修复提示,而不是被嗅探接受。
- **工具结果卡片没有内嵌图像预览**:UI 表面以通用形式渲染图像结果(持久引用而非像素);内嵌渲染延后到 UI 包处理。
- **没有附件局部读取工具**:图片具有文件路径时,agent 可以用其他可用工具裁剪。粘贴或拖入但没有路径的图片无法按更高分辨率重新读取。
- **没有超时接口**`read`/`write`/`edit` 不接受超时参数,也不声明 `timeout-policy` 预算;取消只通过 `exec.signal` 传递(见[提供方理由](../README.zh.md#no-timeouts-on-file-io))。
+74 -29
View File
@@ -1,13 +1,10 @@
/**
* The model-facing `read_image` tool: reads a PNG/JPEG/WebP/GIF file, durably
* commits its bytes through the attachment service (the same lifecycle as a
* user-uploaded image), and returns an image block so the image enters model
* context from the next request onward.
* The model-facing `read_image` tool commits a PNG/JPEG/WebP/GIF file.
*
* The route gate is deliberately stricter than the host upload preflight: a
* tool result enters durable session history, so emitting an image on a route
* that cannot carry it would break that route's continuation. Unknown
* capability therefore refuses instead of relying on the adapter guard.
* The route gate is deliberately stricter than the host upload preflight. An
* image-reading tool is useful only when the exact calling route can inspect
* its result, so unknown capability refuses instead of relying on an adapter
* failure after filesystem and attachment work.
* @module @deepseek-ai/dsh-tool-fs/src/read-image
*/
@@ -30,7 +27,29 @@ const IMAGE_EXTENSIONS: Readonly<Record<string, ImageMediaType>> = {
'.gif': 'image/gif',
}
/** The canonical outcome declared by the `read_image` output schema. */
const IMAGE_VALUE_SCHEMA = {
type: 'object',
additionalProperties: false,
required: true,
properties: {
attachmentId: { type: 'string', required: true },
mediaType: { type: 'string', enum: ['image/png', 'image/jpeg', 'image/webp', 'image/gif'], required: true },
bytes: { type: 'integer', required: true },
width: { type: 'integer', required: true },
height: { type: 'integer', required: true },
name: { type: 'string' },
originalDimensions: {
type: 'object',
additionalProperties: false,
properties: {
width: { type: 'integer', required: true },
height: { type: 'integer', required: true },
},
},
},
} as const
/** The structured outcome declared by the `read_image` output schema. */
export interface ImageReadValue {
path: string
image: {
@@ -40,6 +59,11 @@ export interface ImageReadValue {
width: number
height: number
name?: string
/** Orientation-applied file dimensions before normalization; present only when storage reduced it. */
originalDimensions?: {
width: number
height: number
}
}
}
@@ -75,9 +99,9 @@ export async function assertImageCapableRoute(ctx: Context, exec: ToolExecution,
}
/**
* Re-brand a canonical image outcome into the durable attachment reference an
* Re-brand a structured image outcome into the durable attachment reference an
* `ImageBlock` carries.
* @param image - the canonical image metadata from the output schema.
* @param image - the image metadata from the output schema.
* @returns the branded attachment reference.
*/
export function imageRefFromValue(image: ImageReadValue['image']): ImageAttachmentRef {
@@ -88,26 +112,42 @@ export function imageRefFromValue(image: ImageReadValue['image']): ImageAttachme
width: image.width,
height: image.height,
...image.name === undefined ? {} : { name: image.name },
...image.originalDimensions === undefined ? {} : {
originalDimensions: { ...image.originalDimensions },
},
}
}
/**
* Format an image read as the model-facing envelope beside its image block.
* A downscaled read names the on-disk dimensions and the multiplier that maps
* coordinates measured on the attached image back onto the original file.
* @param displayPath - the backend-resolved path rendered in the envelope's `<path>` element.
* @param image - the canonical image metadata to summarize.
* @param image - the image metadata to summarize.
* @returns the model-facing envelope; the image itself rides the adjacent image block.
*/
export function formatImageReadOutput(displayPath: string, image: ImageReadValue['image']): string {
let scaled = ''
if (image.originalDimensions !== undefined) {
// Integer rounding can give the two axes slightly different ratios, so the
// advice names one multiplier only when both round to the same value.
const x = (image.originalDimensions.width / image.width).toFixed(2)
const y = (image.originalDimensions.height / image.height).toFixed(2)
const advice = x === y
? `multiply coordinates by ${x}`
: `multiply x coordinates by ${x} and y coordinates by ${y}`
scaled = ` (downscaled from ${image.originalDimensions.width}x${image.originalDimensions.height} px; ${advice} to locate features in the original file)`
}
return `<path>${displayPath}</path>
<type>image</type>
<content>
${image.mediaType} image, ${image.width}x${image.height} px, ${image.bytes} bytes
${image.mediaType} image, ${image.width}x${image.height} px, ${image.bytes} bytes${scaled}
</content>`
}
/**
* Project one canonical image read into its model-facing envelope and image.
* @param value - the canonical image-read outcome.
* Project one structured image read into its model-facing envelope and image.
* @param value - the image-read outcome.
* @returns the two content blocks used by native and nested dispatches.
*/
function imageReadContent(value: ImageReadValue): ContentBlock[] {
@@ -129,7 +169,9 @@ function imageReadContent(value: ImageReadValue): ContentBlock[] {
export function applyReadImageTool(ctx: Context): void {
ctx.tools.register(defineTool({
name: 'read_image',
description: 'Read a PNG/JPEG/WebP/GIF file and return the image itself. Requires the current model to accept image input.',
description: 'Read a PNG/JPEG/WebP/GIF file and return the image itself. '
+ 'Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. '
+ 'Independent files may be read concurrently in small batches. Requires the current model to accept image input.',
parameters: {
file_path: { type: 'string', required: true, description: 'Path to the image file, resolved by the filesystem backend.' },
},
@@ -139,19 +181,7 @@ export function applyReadImageTool(ctx: Context): void {
additionalProperties: false,
properties: {
path: { type: 'string', required: true },
image: {
type: 'object',
additionalProperties: false,
required: true,
properties: {
attachmentId: { type: 'string', required: true },
mediaType: { type: 'string', enum: ['image/png', 'image/jpeg', 'image/webp', 'image/gif'], required: true },
bytes: { type: 'integer', required: true },
width: { type: 'integer', required: true },
height: { type: 'integer', required: true },
name: { type: 'string' },
},
},
image: IMAGE_VALUE_SCHEMA,
},
},
render: (_args, value) => imageReadContent(value),
@@ -205,6 +235,18 @@ export function applyReadImageTool(ctx: Context): void {
{ cause: error },
)
}
if (error.code === 'IMAGE_TOO_LARGE') {
throw new Error(
`cannot read "${target.displayPath}": the image cannot be stored within the deployment's byte limits; downscale the image and read the smaller copy`,
{ cause: error },
)
}
if (error.code === 'ATTACHMENT_WRITE_FAILED' && /16-bit PNG/iu.test(error.message)) {
throw new Error(
`cannot read "${target.displayPath}": the 16-bit PNG could not be converted to the normalized 8-bit sRGB form; convert it to an 8-bit PNG/JPEG/WebP and retry`,
{ cause: error },
)
}
if (error.code !== 'IMAGE_TYPE_MISMATCH') throw error
const extension = extname(target.displayPath).toLowerCase()
throw new Error(
@@ -222,6 +264,9 @@ export function applyReadImageTool(ctx: Context): void {
width: ref.width,
height: ref.height,
...ref.name === undefined ? {} : { name: ref.name },
...ref.originalDimensions === undefined ? {} : {
originalDimensions: { ...ref.originalDimensions },
},
},
}
return value
+66 -2
View File
@@ -13,7 +13,7 @@ import { Context } from '@deepseek-ai/cordis'
import { CodeRuntime } from '@deepseek-ai/dsh-code-runtime'
import type { CodeRunRequest, CodeRunResult } from '@deepseek-ai/dsh-code-runtime'
import { CallId, LlmAdapter, LlmRuntime } from '@deepseek-ai/dsh-llm'
import type { GenerateOptions, LlmModelInfo, LlmResolvedModelInfo, StreamChunk } from '@deepseek-ai/dsh-llm'
import type { GenerateOptions, LlmModelInfo, LlmResolvedModelInfo, Message, StreamChunk } from '@deepseek-ai/dsh-llm'
import SystemPrompt from '@deepseek-ai/dsh-system-prompt'
import ToolRuntime, { RUN_CODE_NAME } from '@deepseek-ai/dsh-tools'
import type { Config as ToolConfig } from '@deepseek-ai/dsh-tools'
@@ -122,12 +122,13 @@ async function setup(options: SetupOptions = {}) {
}
/** A fake calling agent pinned to one routed provider/model. */
function agentOn(model: string | undefined, provider = 'visual'): object {
function agentOn(model: string | undefined, provider = 'visual', messages: readonly Message[] = []): object {
return {
options: {},
session: {
header: { cwd: dir },
requestHeader: () => (model === undefined ? undefined : { config: { provider, model } }),
deriveMessages: () => [...messages],
append: () => undefined,
},
}
@@ -169,6 +170,8 @@ describe('imageRefFromValue', () => {
const base = { attachmentId: 'sha256:00', mediaType: 'image/png' as const, bytes: 1, width: 1, height: 1 }
expect(imageRefFromValue(base)).toEqual(base)
expect(imageRefFromValue({ ...base, name: 'a.png' })).toEqual({ ...base, name: 'a.png' })
expect(imageRefFromValue({ ...base, originalDimensions: { width: 4, height: 2 } }))
.toEqual({ ...base, originalDimensions: { width: 4, height: 2 } })
})
})
@@ -438,6 +441,20 @@ describe('image admission failures', () => {
expect(storageFault.isError).toBe(true)
expect(text(storageFault)).toContain('Unable to persist image attachment.')
FailingStore.failure = new AttachmentError(
'The 16-bit PNG could not be converted to the normalized 8-bit sRGB form.',
'ATTACHMENT_WRITE_FAILED',
)
const sixteenBit = await readImage(ctx, { file_path: 'red.png' }, agentOn('vision-model'))
expect(text(sixteenBit)).toContain(
`cannot read "${join(dir, 'red.png')}": the 16-bit PNG could not be converted to the normalized 8-bit sRGB form; convert it to an 8-bit PNG/JPEG/WebP and retry`,
)
FailingStore.failure = new AttachmentError('Image cannot be encoded within the configured normalized-image byte cap.', 'IMAGE_TOO_LARGE')
const overBudget = await readImage(ctx, { file_path: 'red.png' }, agentOn('vision-model'))
expect(overBudget.isError).toBe(true)
expect(text(overBudget)).toContain('cannot be stored within the deployment\'s byte limits; downscale the image and read the smaller copy')
FailingStore.failure = new Error('unrelated infrastructure failure')
const unrelated = await readImage(ctx, { file_path: 'red.png' }, agentOn('vision-model'))
expect(unrelated.isError).toBe(true)
@@ -491,6 +508,53 @@ describe('image admission failures', () => {
const image = result.content[1] as { attachment: ImageAttachmentRef }
expect(image.attachment.name).toBeUndefined()
})
it('names the on-disk dimensions and coordinate multiplier when storage downscales', async () => {
/** Store whose normalized image halves the input on both sides. */
class DownscalingStore extends AttachmentStore {
readonly imageLimits: ImageAttachmentLimits = Object.freeze({
maxImageBytes: 1024,
maxImagesPerMessage: 1,
maxMessageImageBytes: 1024,
maxImagePixels: 100,
maxImageDimension: 2000,
mediaTypes: Object.freeze(['image/png'] as const),
})
validateImage(_input: SaveImageAttachment): Promise<void> {
return Promise.resolve()
}
async saveImage(input: SaveImageAttachment): Promise<ImageAttachmentRef> {
return {
attachmentId: AttachmentId('sha256:feed'),
mediaType: input.mediaType,
bytes: 7,
width: 2,
height: 1,
originalDimensions: { width: 4, height: 2 },
}
}
readImage(_ref: ImageAttachmentRef): Promise<StoredImageAttachment> {
throw new Error('unreachable in this test')
}
}
await writeFile(join(dir, 'red.png'), PNG_1X1)
const ctx = await setup({ attachments: false })
await ctx.plugin(DownscalingStore)
const result = await readImage(ctx, { file_path: 'red.png' }, agentOn('vision-model'))
expect(result.isError).toBe(false)
expect(text(result)).toContain('image/png image, 2x1 px, 7 bytes (downscaled from 4x2 px; multiply coordinates by 2.00 to locate features in the original file)')
})
it('names per-axis multipliers when integer rounding makes the ratios differ', () => {
const envelope = formatImageReadOutput('/img/photo.jpg', {
attachmentId: 'sha256:feed', mediaType: 'image/jpeg', bytes: 9, width: 2, height: 1,
originalDimensions: { width: 5, height: 2 },
})
expect(envelope).toContain('downscaled from 5x2 px; multiply x coordinates by 2.50 and y coordinates by 2.00 to locate features in the original file')
})
})
describe('registration surface', () => {
+1 -18
View File
@@ -14,7 +14,7 @@ import type { Agent, ModelSelection, ModelSelectionRef, AgentOptions, AgentStatu
import type {} from '@deepseek-ai/dsh-agent-presets/types'
import { AttachmentError, admitEncodedImages } from '@deepseek-ai/dsh-attachment'
import type { ImageAttachmentRef } from '@deepseek-ai/dsh-attachment'
import { contentHasImage, createUserMessage, freezeMessage, ReasoningEffortId } from '@deepseek-ai/dsh-llm'
import { createUserMessage, freezeMessage, ReasoningEffortId } from '@deepseek-ai/dsh-llm'
import { errorChain } from '@deepseek-ai/dsh-llm'
import type { ContentBlock, MessageSource } from '@deepseek-ai/dsh-llm'
import { isAppendSurfaceEvent, isJsonValue } from '@deepseek-ai/dsh-session'
@@ -182,11 +182,6 @@ function imageInEvent(event: SessionEvent, match: (ref: ImageAttachmentRef) => b
return undefined
}
/** True when the current model-visible surface contains an image. */
function messagesHaveImage(messages: readonly { content: readonly ContentBlock[] }[]): boolean {
return messages.some(message => contentHasImage(message.content))
}
/** Resolve the first reference matching one opaque id. */
function referencedImage(events: readonly SessionEvent[], attachmentId: string): ImageAttachmentRef | undefined {
for (const event of events) {
@@ -2209,18 +2204,6 @@ export function createApiProxy(ctx: Context, defaults: ApiProxyDefaults): ApiPro
? {}
: { reasoningEffort: ReasoningEffortId(reasoningEffort) },
})
const pendingImage = [...found.agent.inbox.nextTurn, ...found.agent.inbox.nextStep]
.some(message => contentHasImage(message.content))
if (pendingImage || messagesHaveImage(found.agent.session.deriveMessages())) {
const info = await ctx.llm.resolveModelInfo(resolved.provider, resolved.model)
if (info.inputModalities !== undefined && !info.inputModalities.includes('image')) {
return err(request, {
code: 'model-unavailable',
message: `Model "${resolved.model}" does not accept image input, but this session already contains images; select an image-capable model.`,
details: { provider, model },
})
}
}
const selected: ModelSelection = {
provider: resolved.provider,
model: resolved.model,
@@ -153,12 +153,7 @@ describe('Web session model selection', () => {
validateImage,
saveImage,
}
ctx.provide('attachments', {
...attachments,
saveImages(inputs: readonly Parameters<typeof saveImage>[0][]) {
return AttachmentStore.prototype.saveImages.call(attachments, inputs)
},
} as never)
ctx.provide('attachments', Object.setPrototypeOf(attachments, AttachmentStore.prototype) as never)
const followup = vi.fn()
Object.assign(agent, { followup })
const api = createApiProxy(ctx, {
@@ -204,7 +199,7 @@ describe('Web session model selection', () => {
await ctx.fiber.dispose()
})
it('refuses a text-only selection while durable or pending image content remains visible', async () => {
it('allows a text-only selection while durable or pending images remain available for later models', async () => {
const { ctx, agent, sessionId } = await harness()
registerTextOnly(ctx)
const api = createApiProxy(ctx, {
@@ -218,9 +213,9 @@ describe('Web session model selection', () => {
agent.session.append('user/message', {
id: 'image-message', role: 'user', source: { kind: 'user' }, content: [image],
} as never, { surfaceOp: 'append' })
expect((await api.sessions.selectModel(request({
expect(expectValue(await api.sessions.selectModel(request({
sessionId, provider: 'text-only', model: 'plain',
}))).result).toMatchObject({ ok: false, error: { code: 'model-unavailable' } })
}))).selected).toEqual({ provider: 'text-only', model: 'plain' })
agent.session.append('user/message', {
id: 'summary', role: 'user', source: { kind: 'plugin', plugin: 'compact' },
@@ -232,10 +227,6 @@ describe('Web session model selection', () => {
;(agent.inbox.nextTurn as UserMessage[]).push({
id: 'pending-image', role: 'user', source: { kind: 'user' }, content: [image],
} as never)
expect((await api.sessions.selectModel(request({
sessionId, provider: 'text-only', model: 'plain',
}))).result.ok).toBe(false)
;(agent.inbox.nextTurn as UserMessage[]).length = 0
expect(expectValue(await api.sessions.selectModel(request({
sessionId, provider: 'text-only', model: 'plain',
}))).selected).toEqual({ provider: 'text-only', model: 'plain' })
@@ -483,6 +483,12 @@ describe('image attachments', () => {
...input.name === undefined ? {} : { name: input.name },
})
}),
validateImageBatch(inputs: readonly unknown[]) {
const validate = AttachmentStore.prototype as unknown as {
validateImageBatch(this: unknown, batch: readonly unknown[]): void
}
validate.validateImageBatch.call(this, inputs)
},
// The real base-class batch method over this double's limits and members.
saveImages(inputs: readonly unknown[]) {
return (AttachmentStore.prototype.saveImages as (this: unknown, batch: readonly unknown[]) => Promise<unknown[]>).call(this, inputs)
@@ -585,7 +591,9 @@ describe('image attachments', () => {
const store = storeOf()
store.saveImage.mockImplementationOnce((input: { mediaType: string }) => {
controller.abort('operator cancelled during admission')
return Promise.resolve({ attachmentId: 'att-late', mediaType: input.mediaType, bytes: 3, width: 1, height: 1 })
return Promise.resolve({
attachmentId: 'att-late', mediaType: input.mediaType, bytes: 3, width: 1, height: 1,
})
})
ctx.provide('attachments', store)
const { agent } = await mintAgentScope(ctx, 'a')
+2 -2
View File
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/llm/llm-deepseek/README.md
README.md: bae9011135a9cbc14467086e4b6ebc6f052ed230
README.zh.md: 0a5f0224dbebd62766775822260585825579f4a7
README.md: 8a62b7b587323de152ea3322ce310d48a41247cc
README.zh.md: 009732f4256c49d7ae8d702c41df532236f77c5f

Some files were not shown because too many files have changed in this diff Show More