refactor(image): remove region reads

This commit is contained in:
creatixchu
2026-08-21 15:06:24 +08:00
parent 0c9a664223
commit 724783b024
54 changed files with 132 additions and 1040 deletions
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-08-10-minimal-read-image-tool.md
2026-08-10-minimal-read-image-tool.md: 0c0c6a95fa3d8be1dbe895ecd83ff44e1e1eac17
2026-08-10-minimal-read-image-tool.zh.md: c3c2fe1095637a19c3ebaa21cf23a501fe83c480
2026-08-10-minimal-read-image-tool.md: 19306a35fe709a04d94090a62056575b4d51f7bc
2026-08-10-minimal-read-image-tool.zh.md: c7562c433e909d1f81361c0ced56318795e6469e
@@ -6,14 +6,13 @@ English | [中文](2026-08-10-minimal-read-image-tool.zh.md)
## Problem
The multimodal attachment work gave user uploads a complete durable path, but the model itself had no way to inspect an image on disk or crop a durable user upload that had no path. `read` rejects binary content by contract, so an agent asked about a screenshot or rendered chart either failed or used a lossy workaround. A standalone attempt in PR #598 combined the tool with loop-level route scoping, per-route schema visibility, and new session-log concepts. Those features were not required to publish a logged image tool result.
The multimodal attachment work gave user uploads a complete durable path, but the model itself had no way to inspect an image on disk. `read` rejects binary content by contract, so an agent asked about a screenshot or rendered chart either failed or used a lossy workaround. A standalone attempt in PR #598 combined the tool with loop-level route scoping, per-route schema visibility, and new session-log concepts. Those features were not required to publish a logged image tool result.
## Decision
Both image-reading operations live in `dsh-tool-fs` and publish ordinary logged tool results over existing extension points.
- **`read_image` reads a filesystem path.** Extension selects the declared PNG/JPEG/WebP/GIF media type; the attachment store's magic-byte and pixel validation stays authoritative. Bytes travel `ctx.fs.stat` → bounded `ctx.fs.readBytes``ctx.attachments.saveImage``fs/observed`. The tool result contains metadata and an `ImageBlock`.
- **`read_image_region` crops a durable session attachment.** The request names the complete attachment id, current preview dimensions, and a preview-coordinate rectangle. The tool authorizes the id against images already referenced by the calling session, maps the rectangle to the durable master, crops that master, and persists the result as a new attachment. Its result contains the cropped `ImageBlock`, so the model-visible crop is reconstructable from the log. This is the path for pasted or dragged images that have no filesystem location.
- **`FileSystem.readBytes(target, signal, maxBytes)`** is a new required provider primitive: the byte bound lives at the seam so no backend can buffer an unbounded file, with the stat-size short-circuit and a one-byte-past-cap stream guard against post-stat growth (`FS_TOO_LARGE`).
- **Registration is composition-conditional, execution is route-gated.** The tools register only under `ctx.inject(['attachments'], …)`. Before I/O, the strict gate resolves the calling route through `ctx.llm.resolveModelInfo` and requires `image` in `inputModalities`; unknown capability refuses. A text-only route can still consume prior durable images because the shared LLM runtime projects them to placeholders at request assembly.
- **Code Mode forwards the image out-of-band**: a nested dispatch returns the canonical value (execution-local, no image block) and defers a `user`-role context message carrying the envelope and image, so the picture still reaches the next request.
@@ -29,6 +28,5 @@ Both image-reading operations live in `dsh-tool-fs` and publish ordinary logged
## Consequences
- The tools refuse execution on a text-only route, while existing images in session history are represented by request-local placeholders.
- Pasted and dragged images can be cropped without exposing local paths. Session reference authorization prevents access to attachments outside the current session.
- Repeated image results accumulate request cost until request projection or compaction removes them; content addressing deduplicates durable bytes.
- The tool-result card renders the durable reference, not pixels; inline preview is deferred to the UI packages.
@@ -6,14 +6,13 @@ Status: implemented
## 问题
多模态附件工作为用户上传建立了完整的持久路径,但模型无法查看磁盘图片,也无法裁剪没有文件路径的持久用户上传`read` 按约定拒绝二进制内容,因此被问到截图或渲染图表的 agent 要么失败,要么使用有损的变通方法。PR #598 的独立尝试把工具与循环级路由作用域、按路由控制 schema 可见性和新的会话日志概念放在一起。这些能力不是发布一条带图片且已记录的工具结果所必需的。
多模态附件工作为用户上传建立了完整的持久路径,但模型无法查看磁盘图片。`read` 按约定拒绝二进制内容,因此被问到截图或渲染图表的 agent 要么失败,要么使用有损的变通方法。PR #598 的独立尝试把工具与循环级路由作用域、按路由控制 schema 可见性和新的会话日志概念放在一起。这些能力不是发布一条带图片且已记录的工具结果所必需的。
## 决定
两个图片读取操作都放在 `dsh-tool-fs`,通过现有扩展点发布普通的持久工具结果。
- **`read_image` 读取文件系统路径。** 扩展名选择声明的 PNG/JPEG/WebP/GIF 媒体类型,附件存储的魔数与像素校验保持权威。字节沿 `ctx.fs.stat` → 有界 `ctx.fs.readBytes``ctx.attachments.saveImage``fs/observed` 流动。工具结果包含元数据和一个 `ImageBlock`
- **`read_image_region` 裁剪会话中的持久附件。** 请求给出完整附件 ID、当前预览尺寸和预览坐标矩形。工具根据当前会话已引用的图片授权该 ID,把矩形映射到持久主版本,从主版本裁剪,并把结果保存为新附件。结果包含裁剪后的 `ImageBlock`,因此模型可见裁剪可以从日志重建。这也是粘贴或拖入且没有文件路径的图片所使用的入口。
- **`FileSystem.readBytes(target, signal, maxBytes)`** 是新的必备提供方原语:字节上限放在 seam 上,任何后端都无法无界缓冲文件;stat 大小先短路,随后的流最多多读一个字节以防 stat 之后的增长(`FS_TOO_LARGE`)。
- **注册随组合条件挂载,执行按路由门禁。** 工具只在 `ctx.inject(['attachments'], …)` 作用域内注册。执行时在 I/O 之前通过 `ctx.llm.resolveModelInfo` 解析调用路由,并要求 `inputModalities` 包含 `image`;能力未知即拒绝。纯文本路由仍可使用此前的持久图片,因为共享 LLM 运行时会在请求组装时把图片投影为占位符。
- **Code Mode 以带外方式转发图像**:嵌套分派返回规范值(仅限本次执行,不含图像块),并延迟提交一条携带信封和图像的 `user` 角色上下文消息,图片仍会到达下一次请求。
@@ -29,6 +28,5 @@ Status: implemented
## 后果
- 工具在纯文本路由上拒绝执行,而会话历史中已经存在的图片会由请求期占位符表示。
- 粘贴和拖入的图片无需暴露本地路径即可裁剪。会话引用授权会阻止访问当前会话范围外的附件。
- 重复的图片结果会累积请求成本,直到请求投影或压缩将其移除;内容寻址只去重持久字节。
- 工具结果卡片渲染持久引用而非像素;内嵌预览延后到 UI 包处理。
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-08-20-unified-image-request-pipeline.md
2026-08-20-unified-image-request-pipeline.md: c4af375d94ebf2b52fbdd0e8d3d4ee715f87f50e
2026-08-20-unified-image-request-pipeline.zh.md: a1e10c63804b42da127bd115c35587191f0a60f0
2026-08-20-unified-image-request-pipeline.md: f0ef01de3b22c7132e7f698d0948a0da945726ba
2026-08-20-unified-image-request-pipeline.zh.md: b1a14ac418987ab8bfee9b731ad38cb48e21753e
@@ -6,7 +6,7 @@ English | [中文](2026-08-20-unified-image-request-pipeline.zh.md)
## Problem
Durable image history, provider resolution, inline request size, and remote file reuse have different limits. Treating an admitted image as the bytes sent on every later request forced one byte cap and one raster to serve all four concerns. Large but ordinary input was refused, clean 16-bit PNG could pass into history and fail at DeepSeek, repeated base64 expanded long requests, and a provider rejection repeated because the same durable image stayed in every future request. A model also had no stable way to crop a user upload that had no filesystem path.
Durable image history, provider resolution, inline request size, and remote file reuse have different limits. Treating an admitted image as the bytes sent on every later request forced one byte cap and one raster to serve all four concerns. Large but ordinary input was refused, clean 16-bit PNG could pass into history and fail at DeepSeek, repeated base64 expanded long requests, and a provider rejection repeated because the same durable image stayed in every future request.
## Decision
@@ -24,13 +24,13 @@ Batch admission prepares and verifies every master once before publishing any me
`AttachmentStore.readImageRequest` derives a request version under route-owned total-pixel and encoded-byte budgets. Scaling is `min(1, sqrt(maxPixels / (width * height)))`, with no enlargement, followed by inward integer rounding so the encoded raster never exceeds the total-pixel cap. DeepSeek V4 Flash Vision Exp uses 640,000 total pixels and 1MiB raw encoded bytes by default; low detail uses 512 by 512 total pixels. A 2048 by 1024 master projects to 1130 by 565 under the hard cap. Request encoding uses the same color branches, with PNG (palette only without alpha) then WebP 85 and 80 for low-color input, WebP 85 then 80 for other alpha input, and JPEG 85 then 80 for other opaque input. Each fallback runs only after the previous result exceeds 1MiB, and dimensions shrink only after both quality attempts exceed it. The same derivation is used by normal agent turns, direct `ctx.llm.stream` calls, compaction, and other auxiliary streams.
The `variantId` and cache path cover the master attachment id, transform version, route pixel and byte budgets, optional master-coordinate crop, and fixed encoder parameters. A new cache entry is fully decoded before publication. Cache hits use a header probe to check format, 8-bit sRGB/sRGBA facts, dimensions, alpha, and byte limits without decoding the complete raster again; a mismatch regenerates the entry. DeepSeek Files and pi-ai inline base64 therefore use the same deterministic bytes for the same policy. Inline accounting uses the derived byte length after base64 expansion, not the master byte count. Equal in-process `variantId` calls share one transform and cache write. Each caller can cancel its own wait; the shared transform is aborted only after every waiter has cancelled. `AttachmentStore.readImageRequests` preserves input order while the local implementation runs master and request transforms through one FIFO limiter. `imageCompressionConcurrency` is configurable from 1 through 8 and defaults to 2. Batch publication remains sequential after every master has been prepared.
The `variantId` and cache path cover the master attachment id, transform version, route pixel and byte budgets, and fixed encoder parameters. A new cache entry is fully decoded before publication. Cache hits use a header probe to check format, 8-bit sRGB/sRGBA facts, dimensions, alpha, and byte limits without decoding the complete raster again; a mismatch regenerates the entry. DeepSeek Files and pi-ai inline base64 therefore use the same deterministic bytes for the same policy. Inline accounting uses the derived byte length after base64 expansion, not the master byte count. Equal in-process `variantId` calls share one transform and cache write. Each caller can cancel its own wait; the shared transform is aborted only after every waiter has cancelled. `AttachmentStore.readImageRequests` preserves input order while the local implementation runs master and request transforms through one FIFO limiter. `imageCompressionConcurrency` is configurable from 1 through 8 and defaults to 2. Batch publication remains sequential after every master has been prepared.
Request-size offload is a deterministic oldest-first projection. Before reading attachments, each route uses `min(masterBytes, requestVersionMaxBytes)` as a conservative upper bound and removes the oldest over-budget prefix. Only retained masters are read and transformed, so an omitted missing or corrupt object cannot block the request. A second projection uses exact derived lengths without bringing omitted images back. DeepSeek defaults to 128MiB and 600 referenced images. Its removed prefix advances past successive 64MiB byte boundaries and in 20-image count quanta, so 129 one-megabyte images remove the oldest 65, retain 64MiB, and keep that prefix stable until total history passes 192MiB. Pi-ai retains a configurable base64 request bound. A text-only route receives deterministic attachment placeholders, including nested tool-result images, while append-only session history keeps the original references.
### Stable handles and master-coordinate crops
### Stable handles
Every retained request image is preceded by its complete attachment id and actual request dimensions. When the active request exposes `read_image_region`, the text also supplies its preview-coordinate arguments. The tool accepts only an attachment already referenced by the calling session. It maps the supplied preview rectangle to the 2048px master with floor-at-origin and ceil-at-far-edge rounding, crops the master rather than the preview, and persists the result as a new attachment. The tool result contains the new `ImageBlock`, so model-visible output and the durable log remain equivalent.
Every retained request image is preceded by its complete attachment id and actual request dimensions. User messages, tool results, agent-loop requests, compaction, and direct `ctx.llm.stream` calls share this projection.
### DeepSeek Files lifecycle
@@ -46,7 +46,7 @@ Historical attachment objects that later disappear or fail integrity verificatio
## Alternatives considered
**Use one 1MiB canonical image for storage and requests.** This makes model resolution determine durable quality, reduces the source for later crops, and combines local storage, inline expansion, Files quota, and model pixels into one setting. Independent master and request policies keep those responsibilities explicit.
**Use one 1MiB canonical image for storage and requests.** This makes model resolution determine durable image detail and combines local storage, inline expansion, Files quota, and model pixels into one setting. Independent master and request policies keep those responsibilities explicit.
**Reject images above provider dimensions or at the encoding quality floor.** A provider limit is route-specific and future requests may use another model. Proportional master preparation and request projection accept ordinary large images while bounding each later representation.
@@ -56,15 +56,13 @@ Historical attachment objects that later disappear or fail integrity verificatio
**Trust a locally indexed file id indefinitely.** Remote expiry, deletion, and lost upload responses make local and provider state diverge. Response-directed invalidation and one re-upload recover without an unbounded retry loop; an ambiguous stale-file response must invalidate every file used by that attempt because it provides no safe exact target.
**Crop the request preview.** Repeated crops would compound the 640,000-pixel reduction and make coordinates depend on previous encodes. Mapping back to the master preserves the available local detail.
**Refuse text-only model selection after any image.** Durable history can outlive the model that first consumed it. Request-local placeholders keep the session usable without rewriting history.
**Remove one image whenever a request crosses its limit.** That changes an early request message after nearly every new upload. Quantized removed prefixes keep cache invalidation occasional while honoring the configured high bound.
## Verification
Package tests generate 16-bit RGB and RGBA PNG fixtures, prove 8-bit conversion and clean 8-bit passthrough, retain alpha under byte pressure, distinguish high-frequency and ordinary photos from low-color graphics, stop lazy encoding after the first fitting candidate, cover square and wide 640,000-pixel projections, enforce 1MiB request bytes, singleflight equal variants and uploads without shared-cancellation leaks, bound transform concurrency, preserve cache and upload identity, skip attachment reads for conservatively offloaded history, map preview crops to the master, prepare batches once, reject inconsistent Files responses, refresh near-expiry ids without retrieve, recover once from single-id, multiple-id, and ambiguous stale responses, paginate before quota deletion, normalize provider diagnostics, project text-only history, and share normal/compaction request bytes. Keyless assembled snapshots cover the real tool schemas and image request path. A credentialed test uses the built-in `deepseek-official` route and its configured endpoint, never a custom provider entry.
Package tests generate 16-bit RGB and RGBA PNG fixtures, prove 8-bit conversion and clean 8-bit passthrough, retain alpha under byte pressure, distinguish high-frequency and ordinary photos from low-color graphics, stop lazy encoding after the first fitting candidate, cover square and wide 640,000-pixel projections, enforce 1MiB request bytes, singleflight equal variants and uploads without shared-cancellation leaks, bound transform concurrency, preserve cache and upload identity, skip attachment reads for conservatively offloaded history, prepare batches once, reject inconsistent Files responses, refresh near-expiry ids without retrieve, recover once from single-id, multiple-id, and ambiguous stale responses, paginate before quota deletion, normalize provider diagnostics, project text-only history, and share normal/compaction request bytes. Keyless assembled snapshots cover the real tool schemas and image request path. A credentialed test uses the built-in `deepseek-official` route and its configured endpoint, never a custom provider entry.
## Consequences
@@ -6,7 +6,7 @@ Status: implemented
## Problem
持久图片历史、提供方分辨率、内联请求大小和远端文件复用有不同限制。过去把已接纳图片直接作为之后每次请求发送的字节,导致一个字节上限和一份光栅同时承担四种职责。普通大图会被拒绝;干净的 16-bit PNG 可以进入历史,之后才被 DeepSeek 拒绝;重复 base64 使长会话请求持续增长;提供方拒绝后,同一持久图片还会进入每次后续请求。模型也无法稳定裁剪没有文件系统路径的用户上传图片。
持久图片历史、提供方分辨率、内联请求大小和远端文件复用有不同限制。过去把已接纳图片直接作为之后每次请求发送的字节,导致一个字节上限和一份光栅同时承担四种职责。普通大图会被拒绝;干净的 16-bit PNG 可以进入历史,之后才被 DeepSeek 拒绝;重复 base64 使长会话请求持续增长;提供方拒绝后,同一持久图片还会进入每次后续请求。
## Decision
@@ -24,13 +24,13 @@ Status: implemented
`AttachmentStore.readImageRequest` 按路由拥有的总像素和编码字节预算派生请求版本。缩放公式为 `min(1, sqrt(maxPixels / (width * height)))`,不会放大小图,随后向预算内取整,确保编码光栅不超过总像素上限。DeepSeek V4 Flash Vision Exp 默认使用总像素 640,000 和原始编码字节 1MiBlow detail 使用总像素 512×512。2048×1024 主版本在这个硬上限下会投影为 1130×565。请求编码使用相同的分类分支:低色数输入先尝试 PNG,只有不带 alpha 通道时才使用 palette,随后依次尝试质量 85、80 的 WebP;其他透明输入依次尝试质量 85、80 的 WebP;其他非透明输入依次尝试质量 85、80 的 JPEG。只有前一结果超过 1MiB 时才执行下一个候选;两个质量档都超限后才缩小尺寸。普通 agent 轮次、直接 `ctx.llm.stream` 调用、压缩和其他辅助流都使用同一派生过程。
`variantId` 和缓存路径覆盖主附件 ID、变换策略版本、路由像素和字节预算、可选的主版本坐标裁剪区域及固定编码参数。新缓存条目在发布前会完整解码。缓存命中只探测文件头,校验格式、8-bit sRGB/sRGBA、尺寸、透明通道和字节上限,不会再次完整解码光栅;不匹配时会重新生成。因此,同一策略下的 DeepSeek Files 和 pi-ai 内联 base64 使用相同的确定性字节。内联计量使用派生字节经过 base64 膨胀后的长度,不使用主版本字节数。同一进程内相同 `variantId` 的调用共享一次变换和缓存写入。每个调用方可以取消自己的等待;只有全部等待方都取消时,共享变换才会中止。`AttachmentStore.readImageRequests` 保持输入顺序,本地实现则通过一个 FIFO 限流器运行主版本和请求版本变换。`imageCompressionConcurrency` 的可配置范围为 1 至 8,默认值为 2。全部主版本准备完成后,批次仍按顺序发布。
`variantId` 和缓存路径覆盖主附件 ID、变换策略版本、路由像素和字节预算及固定编码参数。新缓存条目在发布前会完整解码。缓存命中只探测文件头,校验格式、8-bit sRGB/sRGBA、尺寸、透明通道和字节上限,不会再次完整解码光栅;不匹配时会重新生成。因此,同一策略下的 DeepSeek Files 和 pi-ai 内联 base64 使用相同的确定性字节。内联计量使用派生字节经过 base64 膨胀后的长度,不使用主版本字节数。同一进程内相同 `variantId` 的调用共享一次变换和缓存写入。每个调用方可以取消自己的等待;只有全部等待方都取消时,共享变换才会中止。`AttachmentStore.readImageRequests` 保持输入顺序,本地实现则通过一个 FIFO 限流器运行主版本和请求版本变换。`imageCompressionConcurrency` 的可配置范围为 1 至 8,默认值为 2。全部主版本准备完成后,批次仍按顺序发布。
请求大小 offload 是确定性的从旧到新投影。读取附件前,每条路由先以 `min(主版本字节数, 请求版本字节上限)` 作为保守上界,移除超出预算的最旧前缀。系统只读取并转换保留的主版本,因此已省略的缺失或损坏对象不会阻塞请求。第二次投影使用确切派生长度,但不会重新加入已省略图片。DeepSeek 默认上限为 128MiB 和 600 张引用图片。被移除前缀会越过连续的 64MiB 字节边界,并按 20 张图片数量步长递增,因此 129 张 1MiB 图片会移除最旧的 65 张并保留 64MiB;持久历史超过 192MiB 前,该前缀保持不变。Pi-ai 保留可配置的 base64 请求上限。纯文本路由会收到确定性的附件占位文本,其中包括嵌套工具结果图片;追加式会话历史继续保留原始引用。
### 稳定句柄与主版本坐标裁剪
### 稳定句柄
每张保留请求图片前都有完整附件 ID 和实际请求尺寸。当前请求公开 `read_image_region` 时,这段文本还会提供预览坐标参数。该工具只接受调用会话已经引用的附件。它按起点向下取整、远端边界向上取整,把提交的预览矩形映射到 2048px 主版本,从主版本而非预览图裁剪,并把结果保存为新附件。工具结果包含新的 `ImageBlock`,因此模型可见输出与持久日志保持一致
每张保留请求图片前都有完整附件 ID 和实际请求尺寸。用户消息、工具结果、agent loop 请求、压缩和直接 `ctx.llm.stream` 调用共享这套投影
### DeepSeek Files 生命周期
@@ -46,7 +46,7 @@ Status: implemented
## Alternatives considered
**使用一份 1MiB 规范图片同时负责存储和请求。** 这种做法让模型分辨率决定持久质量,降低之后裁剪可用的源信息,并把本地存储、内联膨胀、Files 配额和模型像素合并成一个设置。独立的主版本和请求策略会明确区分这些职责。
**使用一份 1MiB 规范图片同时负责存储和请求。** 这种做法让模型分辨率决定持久图片细节,并把本地存储、内联膨胀、Files 配额和模型像素合并成一个设置。独立的主版本和请求策略会明确区分这些职责。
**拒绝超过提供方尺寸或达到编码质量下限的图片。** 提供方限制属于具体路由,未来请求可能改用另一个模型。按比例准备主版本和投影请求版本可以接纳普通大图,同时约束每种后续表示。
@@ -56,15 +56,13 @@ Status: implemented
**永久信任本地索引中的文件 ID。** 远端过期、删除和上传响应丢失会使本地与提供方状态不一致。按响应失效和一次重新上传可以恢复,同时避免无界重试;响应没有给出可安全使用的精确目标时,必须使该次请求使用的全部文件失效。
**从请求预览图裁剪。** 重复裁剪会叠加 640,000 像素缩小,坐标也会依赖之前的编码。映射回主版本能保留本地可用细节。
**历史中出现图片后拒绝选择纯文本模型。** 持久历史可能比最初读取它的模型存活更久。按请求生成的占位文本可以保持会话可用,无需改写历史。
**请求每次越过上限就移除一张图片。** 这种做法会在几乎每次新增图片后改写较早的请求消息。按固定步长递增的移除前缀会降低缓存失效频率,同时遵守配置的上限。
## Verification
包测试会生成 16-bit RGB 和 RGBA PNG fixture,验证 8-bit 转换与干净 8-bit 字节直通、字节压力下保留透明通道、区分高频和普通照片与低色数图形、首个候选合规后停止编码、正方形和宽屏 640,000 像素投影、请求字节不超过 1MiB、相同变体与上传 singleflight 且不会共享取消、变换并发上限、缓存与上传身份、跳过已保守 offload 的历史附件读取、预览到主版本坐标映射、批量只准备一次、Files 响应不一致、进入刷新余量时不查询远端并更新 ID、单个 ID、多个 ID 和模糊失效响应只恢复一次、删除配额文件前完成分页、规范化提供方诊断、纯文本投影,以及普通请求与压缩共享请求字节。无需密钥的组装快照覆盖真实工具 schema 和图片请求路径。使用凭据的测试只使用内置 `deepseek-official` 路由及其已配置端点,不使用自定义提供方条目。
包测试会生成 16-bit RGB 和 RGBA PNG fixture,验证 8-bit 转换与干净 8-bit 字节直通、字节压力下保留透明通道、区分高频和普通照片与低色数图形、首个候选合规后停止编码、正方形和宽屏 640,000 像素投影、请求字节不超过 1MiB、相同变体与上传 singleflight 且不会共享取消、变换并发上限、缓存与上传身份、跳过已保守 offload 的历史附件读取、批量只准备一次、Files 响应不一致、进入刷新余量时不查询远端并更新 ID、单个 ID、多个 ID 和模糊失效响应只恢复一次、删除配额文件前完成分页、规范化提供方诊断、纯文本投影,以及普通请求与压缩共享请求字节。无需密钥的组装快照覆盖真实工具 schema 和图片请求路径。使用凭据的测试只使用内置 `deepseek-official` 路由及其已配置端点,不使用自定义提供方条目。
## Consequences
+2 -2
View File
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write docs/config-catalog.md
config-catalog.md: 661e9a50200fd5c650c389d9bb631c04de61d228
config-catalog.zh.md: 4299bccc1f59899bd78fd64f915c784e26eea49d
config-catalog.md: d288fe3b85f1599da6ecef3dcf59c04c4e8c85d5
config-catalog.zh.md: 266465fd09312c5dde9df4453c34f3aa774db7e2
+1 -1
View File
@@ -346,7 +346,7 @@ export interface Config {
}
```
Source: [`packages/attachment/attachment-local/src/index.ts:53`](../packages/attachment/attachment-local/src/index.ts)
Source: [`packages/attachment/attachment-local/src/index.ts:52`](../packages/attachment/attachment-local/src/index.ts)
<a id="deepseek-aidsh-bash-local"></a>
+1 -1
View File
@@ -348,7 +348,7 @@ export interface Config {
}
```
来源:[`packages/attachment/attachment-local/src/index.ts:53`](../packages/attachment/attachment-local/src/index.ts)
来源:[`packages/attachment/attachment-local/src/index.ts:52`](../packages/attachment/attachment-local/src/index.ts)
<a id="deepseek-aidsh-bash-local"></a>
+2 -2
View File
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write docs/subsystems/attachment.md
attachment.md: ea15172e3e1fafec2e09c3bedc2590fc7551eb2e
attachment.zh.md: c04114c9691fa1ba03446f903c4baf5ae021da4c
attachment.md: 7c55bc192088f67ae7d117bc150aa0ea6fdf8b09
attachment.zh.md: d5a140e283c1b7aa6ee5c991c2932ff65de0b88e
+2 -37
View File
@@ -89,16 +89,6 @@ interface StoredImageAttachment {
}
```
```ts type-equiv
/** Pixel rectangle in the oriented 2048px master-version coordinate system. */
interface MasterImageCrop {
x: number
y: number
width: number
height: number
}
```
```ts type-equiv
/** Deterministic request-image policy selected by one exact model route. */
interface ImageRequestPolicy {
@@ -106,27 +96,13 @@ interface ImageRequestPolicy {
maxPixels: number
/** Encoded-byte cap before base64 expansion or Files API upload. */
maxBytes: number
/** Optional master-coordinate crop applied before pixel-budget scaling. */
crop?: MasterImageCrop
}
```
```ts type-equiv
/** Crop coordinates measured by a model on the request preview it received. */
interface PreviewImageCrop {
previewWidth: number
previewHeight: number
x: number
y: number
width: number
height: number
}
```
```ts type-equiv
/** Cached request version derived from one provider-independent master attachment. */
interface RequestImageAttachment {
/** Cache and upload-index key over the master id, policy, crop, and fixed encoder parameters. */
/** Cache and upload-index key over the master id, policy, and fixed encoder parameters. */
variantId: ImageVariantId
/** Durable master reference from which this request version was derived. */
master: ImageAttachmentRef
@@ -142,12 +118,10 @@ interface RequestImageAttachment {
space: 'srgb'
/** Whether the encoded request version retains an alpha channel. */
hasAlpha: boolean
/** Applied master-coordinate crop, when present. */
crop?: MasterImageCrop
}
```
`saveImage()` prepares a provider-independent 2048px, 4MiB master and atomically commits it before returning its reference. `saveImages()` prepares every validated master once before publishing the batch, so validation rejection leaves no partial objects and publication does not repeat decoding or quality selection. `admitEncodedImages()` is the wire entry for base64 uploads and delegates count, aggregate-byte, and ordered batch admission to `saveImages()`. `readImage()` verifies a master from an authorized session path. `readImageRequest()` derives and caches one request version under an exact route pixel and byte budget; new entries are fully decoded before publication, while cache hits use a bounded metadata probe. `readImageRequests()` lets an implementation apply its configured transform concurrency to an ordered batch. The local implementation lazily encodes preferred candidates, singleflights equal request identities, lets each waiter cancel independently, stops shared work when no waiter remains, and defaults to two simultaneous transformations. `cropImage()` maps model preview coordinates back to the master and returns another durable attachment. The service is retention-neutral: resumed and forked sessions may share objects, so reference-aware garbage collection is deferred rather than tied to one session's deletion.
`saveImage()` prepares a provider-independent 2048px, 4MiB master and atomically commits it before returning its reference. `saveImages()` prepares every validated master once before publishing the batch, so validation rejection leaves no partial objects and publication does not repeat decoding or quality selection. `admitEncodedImages()` is the wire entry for base64 uploads and delegates count, aggregate-byte, and ordered batch admission to `saveImages()`. `readImage()` verifies a master from an authorized session path. `readImageRequest()` derives and caches one request version under an exact route pixel and byte budget; new entries are fully decoded before publication, while cache hits use a bounded metadata probe. `readImageRequests()` lets an implementation apply its configured transform concurrency to an ordered batch. The local implementation lazily encodes preferred candidates, singleflights equal request identities, lets each waiter cancel independently, stops shared work when no waiter remains, and defaults to two simultaneous transformations. The service is retention-neutral: resumed and forked sessions may share objects, so reference-aware garbage collection is deferred rather than tied to one session's deletion.
<!-- BEGIN GENERATED cordis-surface (gen-cordis-catalog.ts) — do not edit between markers -->
@@ -217,15 +191,6 @@ readImageRequest( ref: ImageAttachmentRef, policy: ImageRequestPolicy, signal?:
* @returns request versions in the same order as `refs`.
*/
async readImageRequests( refs: readonly ImageAttachmentRef[], policy: ImageRequestPolicy, signal?: AbortSignal, ): Promise<readonly RequestImageAttachment[]>
/**
* Crop the stored master by coordinates measured on a model request preview and persist the result.
* @param ref - session-authorized master attachment.
* @param crop - preview dimensions and preview-coordinate rectangle.
* @param signal - optional cancellation.
* @returns a new durable attachment reference suitable for a logged tool result.
*/
cropImage( ref: ImageAttachmentRef, crop: PreviewImageCrop, signal?: AbortSignal, ): Promise<SavedImageAttachment>
```
Source: [`packages/attachment/attachment/src/index.ts`](../../packages/attachment/attachment/src/index.ts)
+2 -37
View File
@@ -89,16 +89,6 @@ interface StoredImageAttachment {
}
```
```ts type-equiv
/** Pixel rectangle in the oriented 2048px master-version coordinate system. */
interface MasterImageCrop {
x: number
y: number
width: number
height: number
}
```
```ts type-equiv
/** Deterministic request-image policy selected by one exact model route. */
interface ImageRequestPolicy {
@@ -106,27 +96,13 @@ interface ImageRequestPolicy {
maxPixels: number
/** Encoded-byte cap before base64 expansion or Files API upload. */
maxBytes: number
/** Optional master-coordinate crop applied before pixel-budget scaling. */
crop?: MasterImageCrop
}
```
```ts type-equiv
/** Crop coordinates measured by a model on the request preview it received. */
interface PreviewImageCrop {
previewWidth: number
previewHeight: number
x: number
y: number
width: number
height: number
}
```
```ts type-equiv
/** Cached request version derived from one provider-independent master attachment. */
interface RequestImageAttachment {
/** Cache and upload-index key over the master id, policy, crop, and fixed encoder parameters. */
/** Cache and upload-index key over the master id, policy, and fixed encoder parameters. */
variantId: ImageVariantId
/** Durable master reference from which this request version was derived. */
master: ImageAttachmentRef
@@ -142,12 +118,10 @@ interface RequestImageAttachment {
space: 'srgb'
/** Whether the encoded request version retains an alpha channel. */
hasAlpha: boolean
/** Applied master-coordinate crop, when present. */
crop?: MasterImageCrop
}
```
`saveImage()` 准备提供方无关的 2048px、4MiB 主版本,并在返回引用前以原子方式提交。`saveImages()` 在发布批次前为每个成员各准备一次经过验证的主版本,因此校验拒绝不会留下部分对象,发布也不会重复解码或选择质量。`admitEncodedImages()` 是面向 base64 上传的 wire 入口,把张数、聚合字节和有序批量准入交给 `saveImages()``readImage()` 校验来自已授权会话路径的主版本。`readImageRequest()` 按确切路由的像素和字节预算派生并缓存请求版本;新条目在发布前完整解码,缓存命中只做有界元数据探测。`readImageRequests()` 允许实现按自身配置的变换并发处理有序批次。本地实现按需编码首选候选、合并相同请求身份的并发任务、允许每个等待方单独取消、没有等待方时停止共享任务,默认同时执行两项变换。`cropImage()` 把模型预览坐标映射回主版本,并返回另一个持久附件。该服务不规定保留策略:恢复和 fork 后的会话可能共享对象,因此基于引用的垃圾回收会延期实现,不与单个会话的删除绑定。
`saveImage()` 准备提供方无关的 2048px、4MiB 主版本,并在返回引用前以原子方式提交。`saveImages()` 在发布批次前为每个成员各准备一次经过验证的主版本,因此校验拒绝不会留下部分对象,发布也不会重复解码或选择质量。`admitEncodedImages()` 是面向 base64 上传的 wire 入口,把张数、聚合字节和有序批量准入交给 `saveImages()``readImage()` 校验来自已授权会话路径的主版本。`readImageRequest()` 按确切路由的像素和字节预算派生并缓存请求版本;新条目在发布前完整解码,缓存命中只做有界元数据探测。`readImageRequests()` 允许实现按自身配置的变换并发处理有序批次。本地实现按需编码首选候选、合并相同请求身份的并发任务、允许每个等待方单独取消、没有等待方时停止共享任务,默认同时执行两项变换。该服务不规定保留策略:恢复和 fork 后的会话可能共享对象,因此基于引用的垃圾回收会延期实现,不与单个会话的删除绑定。
<!-- BEGIN GENERATED cordis-surface (gen-cordis-catalog.ts) — do not edit between markers -->
@@ -217,15 +191,6 @@ readImageRequest( ref: ImageAttachmentRef, policy: ImageRequestPolicy, signal?:
* @returns request versions in the same order as `refs`.
*/
async readImageRequests( refs: readonly ImageAttachmentRef[], policy: ImageRequestPolicy, signal?: AbortSignal, ): Promise<readonly RequestImageAttachment[]>
/**
* Crop the stored master by coordinates measured on a model request preview and persist the result.
* @param ref - session-authorized master attachment.
* @param crop - preview dimensions and preview-coordinate rectangle.
* @param signal - optional cancellation.
* @returns a new durable attachment reference suitable for a logged tool result.
*/
cropImage( ref: ImageAttachmentRef, crop: PreviewImageCrop, signal?: AbortSignal, ): Promise<SavedImageAttachment>
```
Source: [`packages/attachment/attachment/src/index.ts`](../../packages/attachment/attachment/src/index.ts)
+2 -2
View File
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write docs/tool-catalog.md
tool-catalog.md: 11a7aead7938fca40d20096e3689890258fbe31c
tool-catalog.zh.md: f29d489441b36318523e0afa2eeab9104e639fd0
tool-catalog.md: 1fa650f1e4e025274d069f27a6522abff46af2e2
tool-catalog.zh.md: c3209e7007e9cf05770ccee0698f9e98a32e8363
+3 -54
View File
@@ -24,7 +24,7 @@ This table connects model-visible tool names to the plugin package and service s
| `@deepseek-ai/dsh-tool-bash-persistent` | `bash` | `ctx.tools`, `ctx.terminals`, `an owning Agent at execution time` | `tool/call`, `PTY shell state`, `tool/result` | - | One owner-isolated persistent bash tool; deployment composition supplies the PTY backend and may override the model-facing environment description. |
| `@deepseek-ai/dsh-tool-pwsh-persistent` | `pwsh` | `ctx.tools`, `ctx.terminals`, `an owning Agent at execution time` | `tool/call`, `PTY shell state`, `tool/result` | - | One owner-isolated persistent pwsh tool, the Windows counterpart of the persistent bash tool; deployment composition supplies a pwsh-dialect PTY backend and may override the model-facing environment description. |
| `@deepseek-ai/dsh-tool-str-replace-editor` | `str_replace_editor` | `ctx.tools`, `ctx.fs` | `tool/call`, `fs/observed after view presence/absence, edit absence, or successful mutation`, `tool/result` | - | Standalone view/create/unique literal replace/line insert tool over the filesystem seam; it composes with any shell or terminal API. |
| `@deepseek-ai/dsh-tool-fs` | `edit`, `read`, `read_image`, `read_image_region`, `write` | `ctx.tools`, `ctx.fs`, `ctx.systemPrompt`, `ctx.attachments (image-tool registration)`, `ctx.llm + an image-capable route (image-tool execution)` | `tool/call`, `fs/write-intent or fs/edit-intent for mutations`, `fs/observed after read presence/absence or successful file operation`, `durable attachment (read_image and read_image_region)`, `tool/result` | - | The read-before-write/edit policy is added by `@deepseek-ai/dsh-fs-observation-policy` (an `fs/*` event-gate plugin, no schema change); a deployment that loads these tools is expected to also load it. The image tools are not registered without `ctx.attachments`; their schemas are route-independent, and execution refuses unless the exact routed model declares image input. |
| `@deepseek-ai/dsh-tool-fs` | `edit`, `read`, `read_image`, `write` | `ctx.tools`, `ctx.fs`, `ctx.systemPrompt`, `ctx.attachments (image-tool registration)`, `ctx.llm + an image-capable route (image-tool execution)` | `tool/call`, `fs/write-intent or fs/edit-intent for mutations`, `fs/observed after read presence/absence or successful file operation`, `durable attachment (read_image)`, `tool/result` | - | The read-before-write/edit policy is added by `@deepseek-ai/dsh-fs-observation-policy` (an `fs/*` event-gate plugin, no schema change); a deployment that loads these tools is expected to also load it. The image tool is not registered without `ctx.attachments`; its schema is route-independent, and execution refuses unless the exact routed model declares image input. |
| `@deepseek-ai/dsh-tool-fs-search` | `glob`, `grep` | `ctx.tools`, `ctx.subprocess`, `ctx.systemPrompt` | `tool/call`, `tool/result` | - | glob and grep are unconditional discovery tools that spawn the packaged ripgrep binary (`@vscode/ripgrep`) through ctx.subprocess as ordinary foreground calls (never background jobs) — no host `rg` install and no shell layer. The catalog uses `sampleOverCapGlobResults: true`; deployments must choose that behavior explicitly. Capped results save the complete formatted list through the optional ctx.spillStore backend; returned locators are follow-up-readable/searchable when the backend exposes local paths in co-located deployments. |
| `@deepseek-ai/dsh-tool-terminal` | `terminal_close`, `terminal_list`, `terminal_open`, `terminal_read`, `terminal_send`, `terminal_signal` | `ctx.tools`, `ctx.terminals`, `ctx.systemPrompt`, `ctx.jobs at call time for run_in_background` | `tool/call`, `tool/result` | - | The six terminal tools are opt-in and complement one-shot shell/filesystem tools. `terminal_send(run_in_background: true)` registers with `ctx.jobs`; TUI, named key sequences, BEL, resize, auto-start, and cross-agent sharing are absent from the schema. |
| `@deepseek-ai/dsh-tool-goal` | `create_goal`, `get_goal`, `update_goal` | `ctx.tools`, `ctx.agents`, `ctx.goals`, `ctx.systemPrompt`, `a calling Agent in an authorized open turn` | `tool/call`, `goal/change for mutations`, `tool/result` | - | create, edit, pause, and resume require direct-human root authority; complete and blocked also accept the exact current goal round. The default blocked lower bound is three admitted rounds. |
@@ -695,7 +695,7 @@ Source: [`packages/fs/tool-fs/src/index.ts`](../packages/fs/tool-fs/src/index.ts
### `read_image`
Read a PNG/JPEG/WebP/GIF file and return the image itself. Requires the current model to accept image input.
Read a PNG/JPEG/WebP/GIF file and return the image itself. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.
```json
{
@@ -714,57 +714,6 @@ Read a PNG/JPEG/WebP/GIF file and return the image itself. Requires the current
Source: [`packages/fs/tool-fs/src/index.ts`](../packages/fs/tool-fs/src/index.ts)
### `read_image_region`
Crop a region from an image attachment already visible in this session. Coordinates use the preview dimensions supplied beside that image.
```json
{
"type": "object",
"properties": {
"attachment_id": {
"type": "string",
"description": "Complete attachment id shown beside the image."
},
"preview_width": {
"type": "integer",
"description": "Width of the preview shown to the model."
},
"preview_height": {
"type": "integer",
"description": "Height of the preview shown to the model."
},
"x": {
"type": "integer",
"description": "Left edge in preview pixels."
},
"y": {
"type": "integer",
"description": "Top edge in preview pixels."
},
"width": {
"type": "integer",
"description": "Crop width in preview pixels."
},
"height": {
"type": "integer",
"description": "Crop height in preview pixels."
}
},
"required": [
"attachment_id",
"preview_width",
"preview_height",
"x",
"y",
"width",
"height"
]
}
```
Source: [`packages/fs/tool-fs/src/index.ts`](../packages/fs/tool-fs/src/index.ts)
### `write`
Create or fully replace a UTF-8 text file.
@@ -791,7 +740,7 @@ Create or fully replace a UTF-8 text file.
Source: [`packages/fs/tool-fs/src/index.ts`](../packages/fs/tool-fs/src/index.ts)
The read-before-write/edit policy is added by `@deepseek-ai/dsh-fs-observation-policy` (an `fs/*` event-gate plugin, no schema change); a deployment that loads these tools is expected to also load it. The image tools are not registered without `ctx.attachments`; their schemas are route-independent, and execution refuses unless the exact routed model declares image input.
The read-before-write/edit policy is added by `@deepseek-ai/dsh-fs-observation-policy` (an `fs/*` event-gate plugin, no schema change); a deployment that loads these tools is expected to also load it. The image tool is not registered without `ctx.attachments`; its schema is route-independent, and execution refuses unless the exact routed model declares image input.
<a id="deepseek-aidsh-tool-fs-search"></a>
+2 -53
View File
@@ -28,7 +28,7 @@
| `@deepseek-ai/dsh-tool-bash-persistent` | `bash` | `ctx.tools``ctx.terminals``an owning Agent at execution time` | `tool/call``PTY shell state``tool/result` | - | 一个按所有者隔离的持久 bash 工具;部署组合提供 PTY 后端,并可覆盖面向模型的环境描述。 |
| `@deepseek-ai/dsh-tool-pwsh-persistent` | `pwsh` | `ctx.tools``ctx.terminals``an owning Agent at execution time` | `tool/call``PTY shell state``tool/result` | - | 一个按所有者隔离的持久 pwsh 工具,持久 bash 工具的 Windows 对应物;部署组合提供 pwsh 方言的 PTY 后端,并可覆盖面向模型的环境描述。 |
| `@deepseek-ai/dsh-tool-str-replace-editor` | `str_replace_editor` | `ctx.tools``ctx.fs` | `tool/call``fs/observed after view presence/absence, edit absence, or successful mutation``tool/result` | - | 基于文件系统 seam 的独立查看/创建/唯一字面量替换/按行插入工具;可与任何 shell 或终端接口组合。 |
| `@deepseek-ai/dsh-tool-fs` | `edit``read``read_image``read_image_region``write` | `ctx.tools``ctx.fs``ctx.systemPrompt``ctx.attachments (image-tool registration)``ctx.llm + an image-capable route (image-tool execution)` | `tool/call``fs/write-intent or fs/edit-intent for mutations``fs/observed after read presence/absence or successful file operation``durable attachment (read_image and read_image_region)``tool/result` | - | 先读后写/编辑策略由 `@deepseek-ai/dsh-fs-observation-policy` 添加;它是一个 `fs/*` 事件门禁插件,不会改变 schema。加载这些工具的部署按预期也应加载该插件。没有 `ctx.attachments` 时图片工具不会注册;其 schema 与路由无关,执行时除非确切路由的模型声明图片输入,否则拒绝。 |
| `@deepseek-ai/dsh-tool-fs` | `edit``read``read_image``write` | `ctx.tools``ctx.fs``ctx.systemPrompt``ctx.attachments (image-tool registration)``ctx.llm + an image-capable route (image-tool execution)` | `tool/call``fs/write-intent or fs/edit-intent for mutations``fs/observed after read presence/absence or successful file operation``durable attachment (read_image)``tool/result` | - | 先读后写/编辑策略由 `@deepseek-ai/dsh-fs-observation-policy` 添加;它是一个 `fs/*` 事件门禁插件,不会改变 schema。加载这些工具的部署按预期也应加载该插件。没有 `ctx.attachments` 时图片工具不会注册;其 schema 与路由无关,执行时除非确切路由的模型声明图片输入,否则拒绝。 |
| `@deepseek-ai/dsh-tool-fs-search` | `glob``grep` | `ctx.tools``ctx.subprocess``ctx.systemPrompt` | `tool/call``tool/result` | - | glob 和 grep 是无条件可用的发现工具,通过 ctx.subprocess spawn 随包提供的 ripgrep 二进制文件(`@vscode/ripgrep`),并作为普通前台调用运行,绝不作为后台任务;无需在宿主机安装 `rg`,也不经过 shell 层。本目录使用 `sampleOverCapGlobResults: true`;部署必须显式选择该行为。结果超过上限时,会通过可选的 ctx.spillStore 后端保存完整的格式化列表;在共置部署中,如果后端公开本地路径,返回的定位信息可供后续读取/搜索。 |
| `@deepseek-ai/dsh-tool-terminal` | `terminal_close``terminal_list``terminal_open``terminal_read``terminal_send``terminal_signal` | `ctx.tools``ctx.terminals``ctx.systemPrompt``ctx.jobs at call time for run_in_background` | `tool/call``tool/result` | - | 这 6 个终端工具需要选择启用,用于补充一次性 bash/文件系统工具。`terminal_send(run_in_background: true)` 会注册到 `ctx.jobs`;schema 不包含 TUI、具名按键序列、BEL、调整尺寸、自动启动和跨 agent 共享。 |
| `@deepseek-ai/dsh-tool-goal` | `create_goal``get_goal``update_goal` | `ctx.tools``ctx.agents``ctx.goals``ctx.systemPrompt``a calling Agent in an authorized open turn` | `tool/call``goal/change for mutations``tool/result` | - | create、edit、pause 和 resume 要求直接来自人类的根权限;complete 和 blocked 也接受确切的当前 Goal Round。blocked 的默认下限是 3 个获准的 Round。 |
@@ -701,7 +701,7 @@ pwsh 工具是 Windows 组合中 bash 执行器 seam 的 PowerShell 方言消费
### `read_image`
读取 PNG/JPEG/WebP/GIF 文件并返回图像本身。要求当前模型接受图像输入。
读取 PNG/JPEG/WebP/GIF 文件并返回图像本身。Harness 会在下一次模型请求前校验并缩小受支持的大图,因此仅为查看图片时应直接使用此工具,无需安装图片库或创建缩略图。可以用小批次并发读取彼此独立的文件。要求当前模型接受图像输入。
```json
{
@@ -720,57 +720,6 @@ pwsh 工具是 Windows 组合中 bash 执行器 seam 的 PowerShell 方言消费
来源:[`packages/fs/tool-fs/src/index.ts`](../packages/fs/tool-fs/src/index.ts)
### `read_image_region`
裁剪当前会话中模型已经可见的图片附件。坐标采用该图片旁给出的预览尺寸。
```json
{
"type": "object",
"properties": {
"attachment_id": {
"type": "string",
"description": "Complete attachment id shown beside the image."
},
"preview_width": {
"type": "integer",
"description": "Width of the preview shown to the model."
},
"preview_height": {
"type": "integer",
"description": "Height of the preview shown to the model."
},
"x": {
"type": "integer",
"description": "Left edge in preview pixels."
},
"y": {
"type": "integer",
"description": "Top edge in preview pixels."
},
"width": {
"type": "integer",
"description": "Crop width in preview pixels."
},
"height": {
"type": "integer",
"description": "Crop height in preview pixels."
}
},
"required": [
"attachment_id",
"preview_width",
"preview_height",
"x",
"y",
"width",
"height"
]
}
```
来源:[`packages/fs/tool-fs/src/index.ts`](../packages/fs/tool-fs/src/index.ts)
### `write`
创建或完全替换 UTF-8 文本文件。
+2 -5
View File
@@ -807,8 +807,7 @@ it('pins native DeepSeek Files image offload in the request sent by the assemble
{
type: 'text',
text: '\nImage sha256:b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640; '
+ 'preview 1x1px. Crop coordinates use this preview. Call read_image_region with this attachment_id, '
+ 'preview_width=1, preview_height=1, x, y, width, and height.',
+ 'request image 1x1px.',
},
{ type: 'file', file_id: 'file-api-snapshot-1' },
{ type: 'text', text: ', then use read_image on red.png and reply with DONE.' },
@@ -852,9 +851,7 @@ it('pins native DeepSeek Files image offload in the request sent by the assemble
role: 'tool',
tool_call_id: 'native-read-image',
content: '<path>{{cwd}}/red.png</path>\n<type>image</type>\n<content>\nimage/png image, 1x1 px, 69 bytes\n'
+ '</content>\nImage sha256:b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640; preview 1x1px. '
+ 'Crop coordinates use this preview. Call read_image_region with this attachment_id, preview_width=1, '
+ 'preview_height=1, x, y, width, and height.',
+ '</content>\nImage sha256:b1ff9c8ea3a780bad09b346c423d2d0e46815926879b18e841d928376a946640; request image 1x1px.',
},
{
role: 'user',
@@ -125,28 +125,11 @@ interface ToolArgsMap {
/** Maximum number of lines to return. Defaults to 2000. */
limit?: number;
} & Record<string, JsonValue>;
/** Read a PNG/JPEG/WebP/GIF file and return the image itself. Requires the current model to accept image input. */
/** Read a PNG/JPEG/WebP/GIF file and return the image itself. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input. */
read_image: {
/** Path to the image file, resolved by the filesystem backend. */
file_path: string;
} & Record<string, JsonValue>;
/** Crop a region from an image attachment already visible in this session. Coordinates use the preview dimensions supplied beside that image. */
read_image_region: {
/** Complete attachment id shown beside the image. */
attachment_id: string;
/** Width of the preview shown to the model. */
preview_width: number;
/** Height of the preview shown to the model. */
preview_height: number;
/** Left edge in preview pixels. */
x: number;
/** Top edge in preview pixels. */
y: number;
/** Crop width in preview pixels. */
width: number;
/** Crop height in preview pixels. */
height: number;
} & Record<string, JsonValue>;
/** Send a message to a background subagent by its subagent id, continuing the same conversation. It becomes the subagent's next turn: if it is still working, the message waits until its current turn finishes, so it cannot redirect work already underway. This call returns no answer from the subagent — only confirmation that the message was delivered — so use it to give it more work. A failure means the message was NOT delivered. */
send_message: {
/** The subagent id returned when the background subagent was started. */
@@ -384,29 +367,6 @@ interface ToolOutputMap {
sourceHeight?: number;
};
};
read_image_region: {
sourceAttachmentId: string;
preview: {
width: number;
height: number;
};
crop: {
x: number;
y: number;
width: number;
height: number;
};
image: {
attachmentId: string;
mediaType: "image/png" | "image/jpeg" | "image/webp" | "image/gif";
bytes: number;
width: number;
height: number;
name?: string;
sourceWidth?: number;
sourceHeight?: number;
};
};
send_message: {
messageId: string;
};
@@ -246,7 +246,7 @@
},
{
"name": "read_image",
"description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. Requires the current model to accept image input.",
"description": "Read a PNG/JPEG/WebP/GIF file and return the image itself. Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. Independent files may be read concurrently in small batches. Requires the current model to accept image input.",
"parameters": {
"type": "object",
"properties": {
@@ -260,52 +260,6 @@
]
}
},
{
"name": "read_image_region",
"description": "Crop a region from an image attachment already visible in this session. Coordinates use the preview dimensions supplied beside that image.",
"parameters": {
"type": "object",
"properties": {
"attachment_id": {
"type": "string",
"description": "Complete attachment id shown beside the image."
},
"preview_width": {
"type": "integer",
"description": "Width of the preview shown to the model."
},
"preview_height": {
"type": "integer",
"description": "Height of the preview shown to the model."
},
"x": {
"type": "integer",
"description": "Left edge in preview pixels."
},
"y": {
"type": "integer",
"description": "Top edge in preview pixels."
},
"width": {
"type": "integer",
"description": "Crop width in preview pixels."
},
"height": {
"type": "integer",
"description": "Crop height in preview pixels."
}
},
"required": [
"attachment_id",
"preview_width",
"preview_height",
"x",
"y",
"width",
"height"
]
}
},
{
"name": "send_message",
"description": "Send a message to a background subagent by its subagent id, continuing the same conversation. It becomes the subagent's next turn: if it is still working, the message waits until its current turn finishes, so it cannot redirect work already underway. This call returns no answer from the subagent — only confirmation that the message was delivered — so use it to give it more work. A failure means the message was NOT delivered.",
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/attachment/attachment-local/README.md
README.md: 6141b7559492aa4c50831c8124a917bfdb704f4b
README.zh.md: 2a8ed6e1aef8022aba5053bf1ef0f9728340d086
README.md: d4831f864dbb061319008242395e2c8ff6d9f642
README.zh.md: 45bddf47ea5f68c15778040de5b29817e8f62956
@@ -6,7 +6,7 @@ The private local implementation of [`@deepseek-ai/dsh-attachment`](../attachmen
Admission accepts at most 20 images and 200MiB of encoded source bytes per message. Each source may use up to 20MiB, 64,000,000 pixels, and 8192px per side. It then prepares a provider-independent master. EXIF orientation is applied, metadata and color profiles are removed, pixels become 8-bit sRGB/sRGBA, and the long edge is reduced proportionally to `masterMaxDimension` (2048px by default). The master has its own `masterMaxBytes` safety cap (4MiB by default). Alpha is retained. A nearest-neighbour bounded sample classifies color complexity without averaging high-frequency pixels. Confirmed low-color images try PNG, using a palette only when the input has no alpha channel, then WebP at qualities 85, 80, and 75. Other alpha images try WebP at those qualities; other opaque images try JPEG. Each candidate runs only after the preceding candidate exceeds the cap. Dimensions shrink only after every candidate at one size exceeds the cap. A clean, single-frame 8-bit sRGB/sRGBA PNG, JPEG, or WebP already within both master limits passes through byte-identically; 16-bit PNG, GIF, animated input, metadata, orientation, and incompatible color spaces force conversion. The source and a converted master are each fully decoded once. `saveImages` prepares and verifies every master once before publishing the batch, so validation failure leaves no partial references and commit does not repeat full image encoding.
Request versions live below `<DSH_HOME>/attachments/v1/request-images/`. `readImageRequest` scales the stored master under a total-pixel budget without enlargement, then enforces a separate encoded-byte cap. The request encoder uses the same color branches, with PNG (palette only without alpha) before WebP 85 and 80 for low-color images, WebP 85 then 80 for other alpha images, and JPEG 85 then 80 for other opaque images. It also executes candidates lazily and reduces dimensions only after both quality attempts exceed the request cap. Its cache identity includes the master id, transform version, pixel and byte budgets, optional master-coordinate crop, and fixed encoder settings. Cached bytes are fully decoded and checked as 8-bit sRGB/sRGBA before use. Concurrent calls for one identity share one transform and cache write; cancelling one waiter does not cancel the shared work. `readImageRequests` schedules batches through the service's FIFO limiter. `imageCompressionConcurrency` controls simultaneous master and request transforms from 1 through 8 and defaults to 2; file publication remains ordered after preparation. `cropImage` maps coordinates measured on a model preview back to the master, crops the master rather than the preview, and commits the crop as another durable attachment.
Request versions live below `<DSH_HOME>/attachments/v1/request-images/`. `readImageRequest` scales the stored master under a total-pixel budget without enlargement, then enforces a separate encoded-byte cap. The request encoder uses the same color branches, with PNG (palette only without alpha) before WebP 85 and 80 for low-color images, WebP 85 then 80 for other alpha images, and JPEG 85 then 80 for other opaque images. It also executes candidates lazily and reduces dimensions only after both quality attempts exceed the request cap. Its cache identity includes the master id, transform version, pixel and byte budgets, and fixed encoder settings. Cached bytes are fully decoded and checked as 8-bit sRGB/sRGBA before use. Concurrent calls for one identity share one transform and cache write; cancelling one waiter does not cancel the shared work. `readImageRequests` schedules batches through the service's FIFO limiter. `imageCompressionConcurrency` controls simultaneous master and request transforms from 1 through 8 and defaults to 2; file publication remains ordered after preparation.
`DSH_HOME` resolves through the shared path policy: explicit config, `$DSH_HOME`, then `~/.dsh`. Session logs contain only the reference and verified metadata, never this host path. `readImage` forwards optional cancellation into the filesystem read, observes it around verification, and preserves it instead of wrapping it as `ATTACHMENT_READ_FAILED`.
@@ -6,7 +6,7 @@
每条消息最多准入 20 张图片,源图编码字节总量不超过 200MiB。每张源图不得超过 20MiB、64,000,000 像素和单边 8192px。随后生成提供方无关的主版本:应用 EXIF 方向,删除元数据和色彩配置文件,转换为 8-bit sRGB/sRGBA,并保持宽高比把长边限制到 `masterMaxDimension`(默认 2048px)。主版本有独立的 `masterMaxBytes` 安全上限(默认 4MiB)。透明通道会保留。系统用 nearest-neighbour 对有界样本分类,不会通过像素平均把高频图片误判为低色数。确认的低色数图片先尝试 PNG,只有不带 alpha 通道时才使用 palette,随后依次尝试质量 85、80、75 的 WebP;其他透明图片依次尝试这些质量的 WebP;其他非透明图片依次尝试这些质量的 JPEG。只有前一个候选超限时才会执行下一个候选;同一尺寸的候选全部超限后才缩小尺寸。已经处于两个主版本上限内的干净、单帧、8-bit sRGB/sRGBA PNG、JPEG 或 WebP 按字节原样直通;16-bit PNG、GIF、动图、元数据、方向和不兼容色彩空间都会触发转换。源图和转换后的主版本各完整解码一次。`saveImages` 在发布任何批次成员前为每张图片各准备并验证一次主版本,因此校验失败不会留下部分引用,提交阶段也不会重复执行完整图片编码。
请求版本保存在 `<DSH_HOME>/attachments/v1/request-images/``readImageRequest` 在不放大小图的前提下,把存储的主版本缩放到总像素预算内,再执行独立的编码字节上限。请求编码器使用同一分类分支:低色数图片先尝试 PNG,只有不带 alpha 通道时才使用 palette,再尝试质量 85 和 80 的 WebP;其他透明图片依次尝试质量 85 和 80 的 WebP;其他非透明图片依次尝试质量 85 和 80 的 JPEG。候选仍按需执行,两个质量档均超限后才缩小尺寸。缓存身份包含主版本 ID、变换策略版本、像素和字节预算、可选的主版本坐标裁剪区域以及固定编码参数。缓存字节在使用前会完整解码并校验为 8-bit sRGB/sRGBA。同一身份的并发调用共享一次变换和缓存写入;取消一个等待方不会取消共享任务。`readImageRequests` 通过服务的 FIFO 限流器调度批次。`imageCompressionConcurrency` 控制同时执行的主版本和请求版本变换,范围为 1 至 8,默认值为 2;文件发布仍在准备结束后按顺序执行。`cropImage` 把模型在预览图上测得的坐标映射回主版本,从主版本而非预览图裁剪,并把裁剪结果提交为另一个持久附件。
请求版本保存在 `<DSH_HOME>/attachments/v1/request-images/``readImageRequest` 在不放大小图的前提下,把存储的主版本缩放到总像素预算内,再执行独立的编码字节上限。请求编码器使用同一分类分支:低色数图片先尝试 PNG,只有不带 alpha 通道时才使用 palette,再尝试质量 85 和 80 的 WebP;其他透明图片依次尝试质量 85 和 80 的 WebP;其他非透明图片依次尝试质量 85 和 80 的 JPEG。候选仍按需执行,两个质量档均超限后才缩小尺寸。缓存身份包含主版本 ID、变换策略版本、像素和字节预算及固定编码参数。缓存字节在使用前会完整解码并校验为 8-bit sRGB/sRGBA。同一身份的并发调用共享一次变换和缓存写入;取消一个等待方不会取消共享任务。`readImageRequests` 通过服务的 FIFO 限流器调度批次。`imageCompressionConcurrency` 控制同时执行的主版本和请求版本变换,范围为 1 至 8,默认值为 2;文件发布仍在准备结束后按顺序执行。
`DSH_HOME` 按共享路径策略解析:显式配置、`$DSH_HOME`,最后是 `~/.dsh`。会话日志只包含引用和经过校验的元数据,绝不包含这个宿主路径。`readImage` 会把可选取消信号传入文件系统读取、在校验前后观察该信号,并保留取消语义,而不会将其包装成 `ATTACHMENT_READ_FAILED`
@@ -8,7 +8,6 @@ import type {
ImageAttachmentLimits,
ImageAttachmentRef,
ImageRequestPolicy,
PreviewImageCrop,
RequestImageAttachment,
SaveImageAttachment,
SavedImageAttachment,
@@ -18,13 +17,13 @@ import { resolveDshHome } from '@deepseek-ai/dsh-home-paths'
import type { MasterImagePolicy } from './canonical.ts'
import { CompressionLimiter } from './compression-limiter.ts'
import { commitPreparedImageFile, prepareImageFile, readImageFile, validateImageFile } from './store.ts'
import { previewCropToMaster, readRequestImageFile, requestImageVariantId } from './request-image.ts'
import { readRequestImageFile, requestImageVariantId } from './request-image.ts'
export { isMasterImage, prepareMasterImage } from './canonical.ts'
export type { MasterImage, MasterImagePolicy } from './canonical.ts'
export { commitPreparedImageFile, prepareImageFile, readImageFile, saveImageFile, validateImageFile } from './store.ts'
export type { PreparedImageFile } from './store.ts'
export { previewCropToMaster, readRequestImageFile, requestImageDimensions, requestImageVariantId } from './request-image.ts'
export { readRequestImageFile, requestImageDimensions, requestImageVariantId } from './request-image.ts'
/** Default maximum encoded bytes for one submitted image; oversized sources are refused, not shrunk. */
export const DEFAULT_MAX_IMAGE_BYTES = 20 * 1024 * 1024
@@ -255,26 +254,6 @@ export class LocalAttachmentStore extends AttachmentStore {
return operation.wait(signal)
}
override async cropImage(
ref: ImageAttachmentRef,
crop: PreviewImageCrop,
signal?: AbortSignal,
): Promise<SavedImageAttachment> {
const master = await this.readImage(ref, signal)
const region = previewCropToMaster(ref.width, ref.height, crop)
const version = await this.requestVersion(ref, {
maxPixels: region.width * region.height,
maxBytes: this.masterPolicy.maxBytes,
crop: region,
}, master, signal)
signal?.throwIfAborted()
const stem = ref.name?.replace(/\.[^.]+$/u, '') ?? String(ref.attachmentId).slice(0, 15)
return this.saveImage({
data: version.data,
mediaType: version.mediaType,
name: `${stem}-crop.${version.mediaType.slice('image/'.length).replace('jpeg', 'jpg')}`,
})
}
}
export default LocalAttachmentStore
@@ -1,4 +1,4 @@
/** Deterministic cached image versions for model requests and region reads. */
/** Deterministic cached image versions for model requests. */
import { createHash, randomUUID } from 'node:crypto'
import { mkdir, readFile, rename, rm, writeFile } from 'node:fs/promises'
@@ -9,8 +9,6 @@ import type {
ImageMediaType,
ImageAttachmentRef,
ImageRequestPolicy,
MasterImageCrop,
PreviewImageCrop,
RequestImageAttachment,
StoredImageAttachment,
} from '@deepseek-ai/dsh-attachment'
@@ -80,22 +78,6 @@ function checkedInteger(value: number, name: string): number {
function validatePolicy(policy: ImageRequestPolicy): void {
checkedInteger(policy.maxPixels, 'Image request maxPixels')
checkedInteger(policy.maxBytes, 'Image request maxBytes')
if (policy.crop !== undefined) {
if (!Number.isSafeInteger(policy.crop.x) || policy.crop.x < 0
|| !Number.isSafeInteger(policy.crop.y) || policy.crop.y < 0) {
throw new AttachmentError('Image crop origin must use non-negative integer pixels.', 'INVALID_ATTACHMENT_REF')
}
checkedInteger(policy.crop.width, 'Image crop width')
checkedInteger(policy.crop.height, 'Image crop height')
}
}
function checkedCrop(master: StoredImageAttachment, crop: MasterImageCrop | undefined): MasterImageCrop | undefined {
if (crop === undefined) return undefined
if (crop.x + crop.width > master.ref.width || crop.y + crop.height > master.ref.height) {
throw new AttachmentError('Image crop extends beyond the stored master image.', 'INVALID_ATTACHMENT_REF')
}
return crop
}
function descriptor(master: ImageAttachmentRef, policy: ImageRequestPolicy): string {
@@ -104,7 +86,6 @@ function descriptor(master: ImageAttachmentRef, policy: ImageRequestPolicy): str
masterAttachmentId: master.attachmentId,
routePixelBudget: policy.maxPixels,
encodedByteBudget: policy.maxBytes,
crop: policy.crop ?? null,
encoding: {
png: { compressionLevel: 9, palette: 'opaque-only' },
webpQualities: REQUEST_IMAGE_QUALITIES,
@@ -118,7 +99,7 @@ function descriptor(master: ImageAttachmentRef, policy: ImageRequestPolicy): str
/**
* Complete deterministic identity for one master and route-owned request policy.
* @param master - provider-independent durable master reference.
* @param policy - route-owned pixel, byte, and crop policy.
* @param policy - route-owned pixel and byte policy.
* @returns branded digest over every request transform input.
*/
export function requestImageVariantId(
@@ -128,20 +109,13 @@ export function requestImageVariantId(
return ImageVariantId(`sha256:${digest(descriptor(master, policy))}`)
}
function pipeline(master: StoredImageAttachment, crop: MasterImageCrop | undefined, width: number, height: number): Sharp {
return sourcePipeline(master, crop)
function pipeline(master: StoredImageAttachment, width: number, height: number): Sharp {
return sourcePipeline(master)
.resize({ width, height, fit: 'inside', withoutEnlargement: true })
}
function sourcePipeline(master: StoredImageAttachment, crop: MasterImageCrop | undefined): Sharp {
let image = sharp(master.data, { failOn: 'error', limitInputPixels: false }).toColourspace('srgb')
if (crop !== undefined) image = image.extract({
left: crop.x,
top: crop.y,
width: crop.width,
height: crop.height,
})
return image
function sourcePipeline(master: StoredImageAttachment): Sharp {
return sharp(master.data, { failOn: 'error', limitInputPixels: false }).toColourspace('srgb')
}
async function encoded(
@@ -161,13 +135,12 @@ async function encoded(
function encodingAttempts(
master: StoredImageAttachment,
crop: MasterImageCrop | undefined,
width: number,
height: number,
hasAlpha: boolean,
lowColour: boolean,
): Array<() => Promise<EncodedRequestImage>> {
const prepared = pipeline(master, crop, width, height)
const prepared = pipeline(master, width, height)
const webp = REQUEST_IMAGE_QUALITIES.map(quality => (
() => encoded(prepared.clone(), 'image/webp', quality)
))
@@ -183,12 +156,8 @@ async function createRequestImage(
policy: ImageRequestPolicy,
hasAlpha: boolean,
): Promise<EncodedRequestImage> {
const crop = checkedCrop(master, policy.crop)
const sourceWidth = crop?.width ?? master.ref.width
const sourceHeight = crop?.height ?? master.ref.height
let dimensions = requestImageDimensions(sourceWidth, sourceHeight, policy.maxPixels)
if (crop === undefined
&& dimensions.width === master.ref.width
let dimensions = requestImageDimensions(master.ref.width, master.ref.height, policy.maxPixels)
if (dimensions.width === master.ref.width
&& dimensions.height === master.ref.height
&& master.data.byteLength <= policy.maxBytes) {
return {
@@ -198,10 +167,10 @@ async function createRequestImage(
height: master.ref.height,
}
}
const lowColour = await hasLowColourCount(sourcePipeline(master, crop))
const lowColour = await hasLowColourCount(sourcePipeline(master))
for (;;) {
const encodedVersion = await encodeFirstWithinLimit(
encodingAttempts(master, crop, dimensions.width, dimensions.height, hasAlpha, lowColour),
encodingAttempts(master, dimensions.width, dimensions.height, hasAlpha, lowColour),
policy.maxBytes,
)
if (!isExhaustedEncoding(encodedVersion)) return encodedVersion
@@ -229,8 +198,7 @@ async function readCached(
try {
const data = new Uint8Array(await readFile(path, { signal }))
const detected = await probeImage(data)
const crop = policy.crop
const maximum = requestImageDimensions(crop?.width ?? master.ref.width, crop?.height ?? master.ref.height, policy.maxPixels)
const maximum = requestImageDimensions(master.ref.width, master.ref.height, policy.maxPixels)
if (data.byteLength > policy.maxBytes || detected.depth !== 'uchar' || detected.space !== 'srgb'
|| detected.width > maximum.width || detected.height > maximum.height
|| detected.hasAlpha !== expectedAlpha) return undefined
@@ -285,7 +253,6 @@ export async function readRequestImageFile(
): Promise<RequestImageAttachment> {
signal?.throwIfAborted()
validatePolicy(policy)
checkedCrop(master, policy.crop)
const source = await probeImage(master.data)
const variantId = requestImageVariantId(master.ref, policy)
const hash = String(variantId).slice('sha256:'.length)
@@ -308,42 +275,5 @@ export async function readRequestImageFile(
depth: 'uchar',
space: 'srgb',
hasAlpha: version.hasAlpha,
...policy.crop === undefined ? {} : { crop: policy.crop },
}
}
/**
* Map a preview-coordinate rectangle to the oriented stored master.
* @param masterWidth - stored master width.
* @param masterHeight - stored master height.
* @param crop - rectangle measured on the model-visible preview.
* @returns covering integer rectangle in master coordinates.
*/
export function previewCropToMaster(
masterWidth: number,
masterHeight: number,
crop: PreviewImageCrop,
): MasterImageCrop {
checkedInteger(masterWidth, 'Master image width')
checkedInteger(masterHeight, 'Master image height')
checkedInteger(crop.previewWidth, 'Preview width')
checkedInteger(crop.previewHeight, 'Preview height')
if (!Number.isSafeInteger(crop.x) || crop.x < 0 || !Number.isSafeInteger(crop.y) || crop.y < 0) {
throw new AttachmentError('Preview crop origin must use non-negative integer pixels.', 'INVALID_ATTACHMENT_REF')
}
checkedInteger(crop.width, 'Preview crop width')
checkedInteger(crop.height, 'Preview crop height')
if (crop.x + crop.width > crop.previewWidth || crop.y + crop.height > crop.previewHeight) {
throw new AttachmentError('Preview crop extends beyond the image shown to the model.', 'INVALID_ATTACHMENT_REF')
}
const x = Math.floor(crop.x * masterWidth / crop.previewWidth)
const y = Math.floor(crop.y * masterHeight / crop.previewHeight)
const right = Math.ceil((crop.x + crop.width) * masterWidth / crop.previewWidth)
const bottom = Math.ceil((crop.y + crop.height) * masterHeight / crop.previewHeight)
return {
x,
y,
width: Math.max(1, Math.min(masterWidth, right) - x),
height: Math.max(1, Math.min(masterHeight, bottom) - y),
}
}
@@ -5,7 +5,7 @@ import { Context } from '@deepseek-ai/cordis'
import sharp from 'sharp'
import { afterEach, describe, expect, it, vi } from 'vitest'
import { CompressionLimiter } from '../src/compression-limiter.ts'
import LocalAttachmentStore, { previewCropToMaster, requestImageDimensions } from '../src/index.ts'
import LocalAttachmentStore, { requestImageDimensions } from '../src/index.ts'
const homes: string[] = []
@@ -51,23 +51,6 @@ describe('request image dimensions', () => {
expect(requestImageDimensions(2, 4, 5)).toEqual({ width: 1, height: 2 })
})
it('rejects invalid preview dimensions, origins, sizes, and bounds', () => {
expect(() => previewCropToMaster(0, 10, {
previewWidth: 10, previewHeight: 10, x: 0, y: 0, width: 1, height: 1,
})).toThrow('Master image width must be a positive integer')
expect(() => previewCropToMaster(10, 10, {
previewWidth: 0, previewHeight: 10, x: 0, y: 0, width: 1, height: 1,
})).toThrow('Preview width must be a positive integer')
expect(() => previewCropToMaster(10, 10, {
previewWidth: 10, previewHeight: 10, x: -1, y: 0, width: 1, height: 1,
})).toThrow('Preview crop origin must use non-negative integer pixels')
expect(() => previewCropToMaster(10, 10, {
previewWidth: 10, previewHeight: 10, x: 0, y: 0, width: 0, height: 1,
})).toThrow('Preview crop width must be a positive integer')
expect(() => previewCropToMaster(10, 10, {
previewWidth: 10, previewHeight: 10, x: 9, y: 0, width: 2, height: 1,
})).toThrow('Preview crop extends beyond the image shown to the model')
})
})
describe('local request-image cache', () => {
@@ -85,7 +68,7 @@ describe('local request-image cache', () => {
expect(batch.map(value => value.master.attachmentId)).toEqual([first.attachmentId, second.attachmentId])
})
it('rejects invalid request policies and master crop bounds', async () => {
it('rejects invalid request policies', async () => {
const attachments = await store()
const master = (await attachments.saveImage({ data: await image(8, 4), mediaType: 'image/png' })).ref
@@ -93,15 +76,6 @@ describe('local request-image cache', () => {
.rejects.toThrow('Image request maxPixels must be a positive integer')
await expect(attachments.readImageRequest(master, { maxPixels: 100, maxBytes: 0 }))
.rejects.toThrow('Image request maxBytes must be a positive integer')
await expect(attachments.readImageRequest(master, {
maxPixels: 100, maxBytes: 100, crop: { x: -1, y: 0, width: 1, height: 1 },
})).rejects.toThrow('Image crop origin must use non-negative integer pixels')
await expect(attachments.readImageRequest(master, {
maxPixels: 100, maxBytes: 100, crop: { x: 0, y: 0, width: 0, height: 1 },
})).rejects.toThrow('Image crop width must be a positive integer')
await expect(attachments.readImageRequest(master, {
maxPixels: 100, maxBytes: 100, crop: { x: 7, y: 0, width: 2, height: 1 },
})).rejects.toThrow('Image crop extends beyond the stored master image')
})
it('refuses a one-pixel request that cannot meet the encoded-byte budget', async () => {
@@ -178,51 +152,6 @@ describe('local request-image cache', () => {
expect(low.width * low.height).toBeLessThanOrEqual(512 * 512 + low.width)
})
it('maps preview coordinates to the 2048px master and crops the master instead of the preview', async () => {
const attachments = await store()
const pixels = Buffer.alloc(2048 * 1024 * 3)
for (let y = 0; y < 1024; y += 1) {
for (let x = 0; x < 2048; x += 1) {
const offset = (y * 2048 + x) * 3
pixels[offset] = x < 1024 ? 255 : 0
pixels[offset + 1] = x < 1024 ? 0 : 255
pixels[offset + 2] = 0
}
}
const source = new Uint8Array(await sharp(pixels, { raw: { width: 2048, height: 1024, channels: 3 } }).png().toBuffer())
const master = (await attachments.saveImage({ data: source, mediaType: 'image/png', name: 'halves.png' })).ref
const preview = await attachments.readImageRequest(master, { maxPixels: 640_000, maxBytes: 1024 * 1024 })
const previewCrop = {
previewWidth: preview.width,
previewHeight: preview.height,
x: Math.floor(preview.width / 2),
y: 0,
width: preview.width - Math.floor(preview.width / 2),
height: preview.height,
}
const mapped = previewCropToMaster(master.width, master.height, previewCrop)
const cropped = await attachments.cropImage(master, previewCrop)
const stored = await attachments.readImage(cropped.ref)
const pixel = await sharp(stored.data).resize(1, 1).removeAlpha().raw().toBuffer()
expect(mapped).toEqual({ x: 1024, y: 0, width: 1024, height: 1024 })
expect(cropped.ref.width).toBe(mapped.width)
expect(cropped.ref.height).toBe(mapped.height)
expect(pixel[1]).toBeGreaterThan(pixel[0] ?? 0)
})
it('names a crop from an unnamed attachment id', async () => {
const attachments = await store()
const master = (await attachments.saveImage({ data: await image(8, 4), mediaType: 'image/png' })).ref
const cropped = await attachments.cropImage(master, {
previewWidth: 8, previewHeight: 4, x: 0, y: 0, width: 4, height: 4,
})
expect(cropped.ref.name).toMatch(/^sha256:[0-9a-f]{8}-crop\.(?:png|webp|jpg)$/u)
})
it('classifies opaque PNG pixels and preserves alpha while enforcing the request budget', async () => {
const attachments = await store()
const side = 256
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/attachment/attachment/README.md
README.md: c4925addf079cdd65defb733e6bc40f91ed6384f
README.zh.md: 5623e0944c6f67e2cdaa90076d794cd617c46d5f
README.md: 66ce5f308cfa1ce6a028dbd248ceef1fdcc31a7c
README.zh.md: 4470956987330a451e3717d419a111def98dd6cb
+2 -2
View File
@@ -4,13 +4,13 @@ English | [中文](README.zh.md)
The durable attachment seam. `ctx.attachments` validates and durably commits a provider-independent master image, then returns a serializable `ImageAttachmentRef`; consumers never persist browser paths, object URLs, provider URLs, or base64 in session events.
Unsent composer images remain browser-owned temporary drafts. `validateImage` runs the complete admission policy without persisting. `saveImages` owns batch count and aggregate-byte limits, prepares every validated master once before publishing any member, then commits in order and returns references only after the complete batch succeeds. A later storage failure returns no partial references, although an earlier immutable content-addressed object may remain unreachable until reference-aware garbage collection exists. `AttachmentError.code` uses the closed `AttachmentErrorCode` string union. Its `ImageAdmissionErrorCode` subset marks caller-correctable image-input failures; `isImageAdmissionError` recognizes that subset at runtime so each protocol adapter can map its own error vocabulary. `saveImage` commits one accepted image before any model-visible session event is published and resolves `SavedImageAttachment`: the returned `ref` describes the stored master while `source` (`SourceImageInfo`) preserves the submitted raster's media type, byte length, and orientation-applied dimensions. `readImage` verifies that master against its logged metadata. `readImageRequest` deterministically derives a route-sized request version whose identity covers the master id, transform version, pixel and byte budgets, crop, and encoder settings; `readImageRequests` preserves ordered results while implementations apply their own bounded concurrency. `cropImage` maps preview coordinates to the stored master and persists the result as a new attachment. Callers may cancel reads and projections; implementations preserve cancellation instead of translating it into a storage failure.
Unsent composer images remain browser-owned temporary drafts. `validateImage` runs the complete admission policy without persisting. `saveImages` owns batch count and aggregate-byte limits, prepares every validated master once before publishing any member, then commits in order and returns references only after the complete batch succeeds. A later storage failure returns no partial references, although an earlier immutable content-addressed object may remain unreachable until reference-aware garbage collection exists. `AttachmentError.code` uses the closed `AttachmentErrorCode` string union. Its `ImageAdmissionErrorCode` subset marks caller-correctable image-input failures; `isImageAdmissionError` recognizes that subset at runtime so each protocol adapter can map its own error vocabulary. `saveImage` commits one accepted image before any model-visible session event is published and resolves `SavedImageAttachment`: the returned `ref` describes the stored master while `source` (`SourceImageInfo`) preserves the submitted raster's media type, byte length, and orientation-applied dimensions. `readImage` verifies that master against its logged metadata. `readImageRequest` deterministically derives a route-sized request version whose identity covers the master id, transform version, pixel and byte budgets, and encoder settings; `readImageRequests` preserves ordered results while implementations apply their own bounded concurrency. Callers may cancel reads and projections; implementations preserve cancellation instead of translating it into a storage failure.
`admitEncodedImages(attachments, images)` is the shared wire entry used by every RPC endpoint that accepts browser uploads (the session prompt endpoint and the command executor): it enforces canonical base64 on every member, then delegates batch admission — limits, validation, ordered commit — to `saveImages`. The base64 upload form is `EncodedImageAttachment`, exported from `@deepseek-ai/dsh-attachment/types` so wire contracts can reference it.
## Model Experience
Indirectly, through the role-neutral core `ImageBlock` and provider adapters that resolve its durable reference into an exact request version. Request descriptors expose the complete attachment id, actual preview dimensions, and the `read_image_region` coordinate system.
Indirectly, through the role-neutral core `ImageBlock` and provider adapters that resolve its durable reference into an exact request version. Request descriptors expose the complete attachment id and actual request dimensions.
#### KV Cache effect
+2 -2
View File
@@ -4,13 +4,13 @@
持久附件服务边界。`ctx.attachments` 校验并持久提交提供方无关的图片主版本,随后返回可序列化的 `ImageAttachmentRef`;消费方绝不会在会话事件中持久保存浏览器路径、对象 URL、提供方 URL 或 base64。
未发送的输入区图片仍是由浏览器持有的临时草稿。`validateImage` 运行完整准入策略但不执行持久化。`saveImages` 负责批次图片数量和总字节限制,在发布任何成员前为全部成员各准备一次经过验证的主版本,然后按顺序提交,并且只在完整批次成功后返回引用。后续存储失败不会返回部分引用,但较早写入的不可变内容寻址对象可能保持不可达,直至具备按引用感知的垃圾回收。`AttachmentError.code` 使用封闭的 `AttachmentErrorCode` 字符串联合类型。其 `ImageAdmissionErrorCode` 子集标记可由调用方修正的图片输入失败;`isImageAdmissionError` 在运行时识别该子集,使每个协议适配器可以映射自己的错误词汇。`saveImage` 会在发布任何模型可见的会话事件前提交一张已接受的图片,并解析为 `SavedImageAttachment`:返回的 `ref` 描述实际存储的主版本,而 `source``SourceImageInfo`)保留所提交光栅的媒体类型、字节长度和应用方向后的尺寸。`readImage` 根据已记录的元数据校验该主版本。`readImageRequest` 确定性派生路由所需的请求版本,其身份覆盖主版本 ID、变换策略版本、像素和字节预算、裁剪区域及编码参数;`readImageRequests` 保持结果顺序,并由实现施加自己的有界并发。`cropImage` 把预览坐标映射到存储的主版本,并把结果保存为新附件。调用方可以取消读取和投影;实现保留取消结果,不把它转换为存储失败。
未发送的输入区图片仍是由浏览器持有的临时草稿。`validateImage` 运行完整准入策略但不执行持久化。`saveImages` 负责批次图片数量和总字节限制,在发布任何成员前为全部成员各准备一次经过验证的主版本,然后按顺序提交,并且只在完整批次成功后返回引用。后续存储失败不会返回部分引用,但较早写入的不可变内容寻址对象可能保持不可达,直至具备按引用感知的垃圾回收。`AttachmentError.code` 使用封闭的 `AttachmentErrorCode` 字符串联合类型。其 `ImageAdmissionErrorCode` 子集标记可由调用方修正的图片输入失败;`isImageAdmissionError` 在运行时识别该子集,使每个协议适配器可以映射自己的错误词汇。`saveImage` 会在发布任何模型可见的会话事件前提交一张已接受的图片,并解析为 `SavedImageAttachment`:返回的 `ref` 描述实际存储的主版本,而 `source``SourceImageInfo`)保留所提交光栅的媒体类型、字节长度和应用方向后的尺寸。`readImage` 根据已记录的元数据校验该主版本。`readImageRequest` 确定性派生路由所需的请求版本,其身份覆盖主版本 ID、变换策略版本、像素和字节预算及编码参数;`readImageRequests` 保持结果顺序,并由实现施加自己的有界并发。调用方可以取消读取和投影;实现保留取消结果,不把它转换为存储失败。
`admitEncodedImages(attachments, images)` 是每个接受浏览器上传的 RPC 端点(会话 prompt 端点与命令执行器)共用的 wire 入口:它对每个成员强制执行规范 base64,随后把批量准入——限额、校验、有序提交——委托给 `saveImages`。base64 上传形式为 `EncodedImageAttachment`,从 `@deepseek-ai/dsh-attachment/types` 导出,供 wire 契约引用。
## 模型体验
该包通过角色无关的核心 `ImageBlock`,以及把持久引用解析为确定请求版本的提供方适配器,间接影响模型。请求描述会公开完整附件 ID、实际预览尺寸和 `read_image_region` 使用的坐标系
该包通过角色无关的核心 `ImageBlock`,以及把持久引用解析为确定请求版本的提供方适配器,间接影响模型。请求描述会公开完整附件 ID 和实际请求尺寸
#### KV 缓存影响
@@ -6,7 +6,6 @@ import type {
ImageAttachmentLimits,
ImageAttachmentRef,
ImageRequestPolicy,
PreviewImageCrop,
RequestImageAttachment,
SaveImageAttachment,
SavedImageAttachment,
@@ -24,8 +23,6 @@ export type {
ImageAttachmentRef,
ImageRequestPolicy,
ImageMediaType,
MasterImageCrop,
PreviewImageCrop,
RequestImageAttachment,
SaveImageAttachment,
SavedImageAttachment,
@@ -153,26 +150,6 @@ export abstract class AttachmentStore extends Service {
return versions
}
/**
* Crop the stored master by coordinates measured on a model request preview and persist the result.
* @param ref - session-authorized master attachment.
* @param crop - preview dimensions and preview-coordinate rectangle.
* @param signal - optional cancellation.
* @returns a new durable attachment reference suitable for a logged tool result.
*/
cropImage(
ref: ImageAttachmentRef,
crop: PreviewImageCrop,
signal?: AbortSignal,
): Promise<SavedImageAttachment> {
signal?.throwIfAborted()
void ref
void crop
return Promise.reject(new AttachmentError(
'The mounted attachment provider cannot crop stored images.',
'ATTACHMENT_PROJECTION_UNSUPPORTED',
))
}
}
export default AttachmentStore
+1 -23
View File
@@ -63,27 +63,17 @@ export interface StoredImageAttachment {
data: Uint8Array
}
/** Pixel rectangle in the oriented 2048px master-version coordinate system. */
export interface MasterImageCrop {
x: number
y: number
width: number
height: number
}
/** Deterministic request-image policy selected by one exact model route. */
export interface ImageRequestPolicy {
/** Maximum width multiplied by height after aspect-preserving projection. */
maxPixels: number
/** Encoded-byte cap before base64 expansion or Files API upload. */
maxBytes: number
/** Optional master-coordinate crop applied before pixel-budget scaling. */
crop?: MasterImageCrop
}
/** Cached request version derived from one provider-independent master attachment. */
export interface RequestImageAttachment {
/** Cache and upload-index key over the master id, policy, crop, and fixed encoder parameters. */
/** Cache and upload-index key over the master id, policy, and fixed encoder parameters. */
variantId: ImageVariantId
/** Durable master reference from which this request version was derived. */
master: ImageAttachmentRef
@@ -99,18 +89,6 @@ export interface RequestImageAttachment {
space: 'srgb'
/** Whether the encoded request version retains an alpha channel. */
hasAlpha: boolean
/** Applied master-coordinate crop, when present. */
crop?: MasterImageCrop
}
/** Crop coordinates measured by a model on the request preview it received. */
export interface PreviewImageCrop {
previewWidth: number
previewHeight: number
x: number
y: number
width: number
height: number
}
/** Intrinsic facts of the submitted source raster, before master-version preparation. */
@@ -151,22 +151,15 @@ describe('AttachmentStore.readImageRequests', () => {
expect(versions.map(version => version.master.name)).toEqual(['1.png', '2.png'])
})
it('reports unsupported request projection and crop operations, preserving cancellation', async () => {
it('reports unsupported request projection while preserving cancellation', async () => {
const store = new UnsupportedProjectionStore(new Context())
const ref = (await new RecordingStore(new Context()).saveImage(image(1))).ref
await expect(store.readImageRequest(ref, { maxPixels: 1, maxBytes: 1 }))
.rejects.toMatchObject({ code: 'ATTACHMENT_PROJECTION_UNSUPPORTED' })
await expect(store.cropImage(ref, {
previewWidth: 1, previewHeight: 1, x: 0, y: 0, width: 1, height: 1,
})).rejects.toMatchObject({ code: 'ATTACHMENT_PROJECTION_UNSUPPORTED' })
const controller = new AbortController()
const reason = new Error('cancel unsupported projection')
controller.abort(reason)
expect(() => store.readImageRequest(ref, { maxPixels: 1, maxBytes: 1 }, controller.signal)).toThrow(reason)
expect(() => store.cropImage(ref, {
previewWidth: 1, previewHeight: 1, x: 0, y: 0, width: 1, height: 1,
}, controller.signal)).toThrow(reason)
})
})
@@ -31,7 +31,7 @@ describe('gen-tool-catalog collectToolCatalog', () => {
'cordis_undefine', 'create_goal', 'edit', 'exit_plan_mode', 'followup_task', 'get_goal', 'glob', 'grep',
'interrupt_agent', 'interrupt_agent', 'job_kill', 'job_list', 'job_output',
'list_agents', 'list_agents', 'lsp', 'pwsh', 'pwsh', 'ralph',
'read', 'read_image', 'read_image_region', 'report', 'run_code', 'schedule_create', 'schedule_delete',
'read', 'read_image', 'report', 'run_code', 'schedule_create', 'schedule_delete',
'schedule_list', 'send_message', 'send_message', 'session_event_read', 'session_event_search',
'session_event_trace', 'session_search', 'session_trace', 'skill', 'spawn_teammate',
'str_replace_editor', 'subagent', 'team_task_create',
@@ -467,12 +467,6 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [
parameters: [{ name: 'refs', description: 'durable provider-independent master references in request order.' }, { name: 'policy', description: 'exact route pixel and encoded-byte budget shared by the batch.' }, { name: 'signal', description: 'optional cancellation.' }],
returns: 'request versions in the same order as `refs`.',
},
{
signature: 'cropImage( ref: ImageAttachmentRef, crop: PreviewImageCrop, signal?: AbortSignal, ): Promise<SavedImageAttachment>',
description: 'Crop the stored master by coordinates measured on a model request preview and persist the result.',
parameters: [{ name: 'ref', description: 'session-authorized master attachment.' }, { name: 'crop', description: 'preview dimensions and preview-coordinate rectangle.' }, { name: 'signal', description: 'optional cancellation.' }],
returns: 'a new durable attachment reference suitable for a logged tool result.',
},
],
},
{
@@ -3480,7 +3474,7 @@ export const TYPE_API: readonly TypeApiEntry[] = [
},
{
name: 'ImageRequestPolicy',
declaration: 'export interface ImageRequestPolicy {\n maxPixels: number;\n maxBytes: number;\n crop?: MasterImageCrop;\n}',
declaration: 'export interface ImageRequestPolicy {\n maxPixels: number;\n maxBytes: number;\n}',
},
{
name: 'ImageVariantId',
@@ -3710,10 +3704,6 @@ export const TYPE_API: readonly TypeApiEntry[] = [
name: 'ManualCompactAgentContext',
declaration: 'export interface ManualCompactAgentContext extends CompactionAgentContext {\n runMaintenance<T>(task: (signal: AbortSignal) => Promise<T>): Promise<T>;\n}',
},
{
name: 'MasterImageCrop',
declaration: 'export interface MasterImageCrop {\n x: number;\n y: number;\n width: number;\n height: number;\n}',
},
{
name: 'Message',
declaration: 'export interface Message {\n readonly id: MessageId;\n readonly role: \'system\' | \'user\' | \'assistant\';\n readonly content: ContentBlock[];\n readonly source: MessageSource;\n}',
@@ -3870,10 +3860,6 @@ export const TYPE_API: readonly TypeApiEntry[] = [
name: 'PreToolDecision',
declaration: 'export type PreToolDecision = {\n kind: \'allow\';\n} | {\n kind: \'deny\';\n reason: string;\n} | {\n kind: \'ask\';\n reason?: string;\n};',
},
{
name: 'PreviewImageCrop',
declaration: 'export interface PreviewImageCrop {\n previewWidth: number;\n previewHeight: number;\n x: number;\n y: number;\n width: number;\n height: number;\n}',
},
{
name: 'ProjectionChangeListener',
declaration: 'export type ProjectionChangeListener = (session: Session, key: Extract<keyof SessionProjectionMap, string>, value: unknown, seq: number) => void;',
@@ -3956,7 +3942,7 @@ export const TYPE_API: readonly TypeApiEntry[] = [
},
{
name: 'RequestImageAttachment',
declaration: 'export interface RequestImageAttachment {\n variantId: ImageVariantId;\n master: ImageAttachmentRef;\n data: Uint8Array;\n mediaType: ImageMediaType;\n bytes: number;\n width: number;\n height: number;\n depth: \'uchar\';\n space: \'srgb\';\n hasAlpha: boolean;\n crop?: MasterImageCrop;\n}',
declaration: 'export interface RequestImageAttachment {\n variantId: ImageVariantId;\n master: ImageAttachmentRef;\n data: Uint8Array;\n mediaType: ImageMediaType;\n bytes: number;\n width: number;\n height: number;\n depth: \'uchar\';\n space: \'srgb\';\n hasAlpha: boolean;\n}',
},
{
name: 'RequestRunOutcome',
+2 -2
View File
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/fs/tool-fs/README.md
README.md: 94af10c501bcb86465d685f1f20c7d42f3b9d117
README.zh.md: 4b8e826db3ae15b825d2f888e7d37fc3cafd1b23
README.md: ab01840f122d6e0df2782b86840432914b27ebd0
README.zh.md: ef738a3715b6db45d386d56ba2a776960dd341c1
+8 -9
View File
@@ -2,7 +2,7 @@
English | [中文](README.zh.md)
The **model-facing filesystem tools**`read`, `read_image`, `read_image_region`, `write`, `edit` — and their **executor**. This is the consumer layer of the filesystem stack: it owns tool names, JSON schemas, argument validation, prompt sections, **read windowing**, and result formatting. It reads/writes/edits through the `ctx.fs` provider contract ([`@deepseek-ai/dsh-fs`](../fs)) **directly**. The freshness/observation policy is contributed by a separate plugin ([`@deepseek-ai/dsh-fs-observation-policy`](../fs-observation-policy)) through the `fs/*` event gate; the tool is not method-coupled to it. Under a confining provider, the shared sandbox-policy service is required for per-session execution and the tool exposes escalation for filesystem mutations.
The **model-facing filesystem tools**`read`, `read_image`, `write`, `edit` — and their **executor**. This is the consumer layer of the filesystem stack: it owns tool names, JSON schemas, argument validation, prompt sections, **read windowing**, and result formatting. It reads/writes/edits through the `ctx.fs` provider contract ([`@deepseek-ai/dsh-fs`](../fs)) **directly**. The freshness/observation policy is contributed by a separate plugin ([`@deepseek-ai/dsh-fs-observation-policy`](../fs-observation-policy)) through the `fs/*` event gate; the tool is not method-coupled to it. Under a confining provider, the shared sandbox-policy service is required for per-session execution and the tool exposes escalation for filesystem mutations.
```ts ignore-check
// Default deployment: a ctx.fs provider, the policy plugin, then the tools.
@@ -14,7 +14,7 @@ await ctx.plugin(ToolFs) // this package — re
`@deepseek-ai/dsh-fs-observation-policy` is **optional**: omit it and the tools run against the bare provider (unconditional write/overwrite/edit, no observed-state). A deployment that loads these tools is expected to also load it, so the behavior is read-before-write/edit.
`read_image` and `read_image_region` register only while a durable `ctx.attachments` service is mounted. Execution additionally requires the exact routed model to declare `image` input (resolved through `ctx.llm.resolveModelInfo` from the session's latest request header, falling back to agent options). `read_image_region` accepts only a complete attachment id already referenced by the calling session, so it can crop a user upload without a filesystem path but cannot cross session scope.
`read_image` registers only while a durable `ctx.attachments` service is mounted. Execution additionally requires the exact routed model to declare `image` input, resolved through `ctx.llm.resolveModelInfo` from the session's latest request header and then from agent options.
## Config
@@ -32,14 +32,13 @@ All keys are optional; the defaults are the shipped read caps.
| Tool | Arguments | Behavior |
|---|---|---|
| `read` | `file_path`, `offset?`, `limit?` | Line-numbered UTF-8 content with a pagination footer. `offset` is 1-based; `limit` defaults to and caps at the configured `readLimit` (2000). |
| `read_image` | `file_path` | Reads a PNG/JPEG/WebP/GIF file through the bounded byte seam, persists it through `ctx.attachments.saveImage`, and returns an image block beside a small metadata envelope. It succeeds only when the exact routed model declares image input. |
| `read_image_region` | `attachment_id`, `preview_width`, `preview_height`, `x`, `y`, `width`, `height` | Resolves a session-authorized image, maps the preview-coordinate rectangle to its stored master, persists the crop, and returns the new image block. |
| `read_image` | `file_path` | Reads a PNG/JPEG/WebP/GIF file through the bounded byte seam, persists it through `ctx.attachments.saveImage`, and returns an image block beside a small metadata envelope. Harness validates and downscales large supported images before the next model request, so the model can read the source directly without first creating a thumbnail. It succeeds only when the exact routed model declares image input. |
| `write` | `file_path`, `content` | Create or fully replace a file. With the policy plugin: overwriting an existing file requires a prior `read` at the unchanged version; creating a new file does not. Without it: unconditional. |
| `edit` | `file_path`, non-empty `old_string`, `new_string`, `replace_all?` | Literal replacement; unique match required unless `replace_all` is true. With the policy plugin: requires a prior `read` (any window) and the file unchanged since. Without it: unconditional. |
Field names are snake_case to match Claude Code and existing harness tool schemas.
Structured successes are `read``{ path, offset, lines: [{ number, text }], totalLines }`, `read_image``{ path, image: { attachmentId, mediaType, bytes, width, height, name?, sourceWidth?, sourceHeight? } }`, `read_image_region``{ sourceAttachmentId, preview, crop, image }`, `write``{ path, operation: 'create' | 'update', before: string | null, after }`, and `edit``{ path, before, after }`. The image source fields appear only when master preparation downscaled the submitted raster. Native renderers preserve the line-numbered read and mutation acknowledgements below. `write`/`edit` derive replayable diff-card metadata, and `read` derives a replayable read-card window `{ path, offset, lines, totalLines, lang? }`; execution-local structured values are not added to `tool/result`, while image renderers emit the durable image blocks that the result logs.
Structured successes are `read``{ path, offset, lines: [{ number, text }], totalLines }`, `read_image``{ path, image: { attachmentId, mediaType, bytes, width, height, name?, sourceWidth?, sourceHeight? } }`, `write``{ path, operation: 'create' | 'update', before: string | null, after }`, and `edit``{ path, before, after }`. The image source fields appear only when master preparation downscaled the submitted raster. Native renderers preserve the line-numbered read and mutation acknowledgements below. `write`/`edit` derive replayable diff-card metadata, and `read` derives a replayable read-card window `{ path, offset, lines, totalLines, lang? }`; execution-local structured values are not added to `tool/result`, while image renderers emit the durable image blocks that the result logs.
## The tool is the executor; policy is an event gate
@@ -47,7 +46,6 @@ The tools do **not** inject a policy service or inspect any cache. Each tool res
- **read** — one `ctx.fs.stat` (type + size routing + version), then `readText`/`streamText`, then builds the line window, then emits `fs/observed` with a plain `ctx.emit`. (1 stat.)
- **read_image** — validates the argument, extension, attachment availability, deployment media types, and the image-capable route before any I/O; then one `ctx.fs.stat` (recording an `absent` observation for a missing target, like `read`), a bounded `ctx.fs.readBytes` capped at the smaller of `imageLimits.maxImageBytes` and `imageLimits.maxMessageImageBytes` (the result is one message carrying one image), `attachments.saveImage` (content-addressed, so the image block references a durably committed object by the time `tool/result` is appended), and finally `fs/observed`. (1 stat.)
- **read_image_region** — resolves the full attachment id only from current session messages, validates integer preview coordinates, maps the rectangle to the stored master through `attachments.cropImage`, and returns the persisted crop as an image block. It performs no filesystem-path operation and emits no `fs/observed` event.
- **write**`ctx.waterfall('fs/write-intent', target, exec, () => undefined)` for the optional guard, then `ctx.fs.writeText(target, content, intent)`, then `fs/observed`. (0 stat.)
- **edit**`ctx.waterfall('fs/edit-intent', target, exec, () => undefined)` for the optional guard, then `ctx.fs.editText(target, edit, intent)`, then `fs/observed`. (0 stat.)
@@ -101,7 +99,7 @@ Prefix-stable while the plugin scope and guidance text are unchanged. Tool restr
#### What the model sees
The model sees the generated [`read`, `read_image`, `read_image_region`, `write`, and `edit` schemas](../../../docs/tool-catalog.md#deepseek-aidsh-tool-fs), with snake_case arguments. The image tools appear only while a durable attachment store is mounted; their schemas are route-independent, and the strict gate refuses at execution. Scoped tool restrictions can remove any definition for one agent.
The model sees the generated [`read`, `read_image`, `write`, and `edit` schemas](../../../docs/tool-catalog.md#deepseek-aidsh-tool-fs), with snake_case arguments. The image tool appears only while a durable attachment store is mounted; its schema is route-independent, and the strict gate refuses at execution. Scoped tool restrictions can remove any definition for one agent.
#### Token effect
@@ -129,7 +127,7 @@ Append-only; newly visible content follows the reusable request prefix and does
#### What the model sees
A successful `read_image` returns `<path><displayPath></path>`, `<type>image</type>`, and a `<content>` envelope naming the media type, master dimensions, and byte size, followed by the image itself as a native image block. A successful `read_image_region` returns an `image-region` envelope naming the source attachment, supplied preview dimensions and rectangle, and result dimensions, followed by the crop as a native image block. The result is logged with its new durable reference before the next model request. Request adapters derive previews from the master, so later region reads never crop an already reduced preview.
A successful `read_image` returns `<path><displayPath></path>`, `<type>image</type>`, and a `<content>` envelope naming the media type, master dimensions, and byte size, followed by the image itself as a native image block. The result is logged with its durable reference before the next model request.
#### Token effect
@@ -157,7 +155,7 @@ Append-only; newly visible content follows the reusable request prefix and does
#### What the model sees
Failures are normalized as `Error: <message>`. This package's stable validation and read messages are `file_path must be a non-empty string`, `limit must be less than or equal to <max>`, `old_string must be a non-empty string`, `old_string and new_string must differ`, `cannot read "<path>": not found`, `cannot read "<path>": not a regular file`, `offset <offset> is out of range for "<path>" (<total> lines)`, `cannot read "<path>": read_image only accepts PNG/JPEG/WebP/GIF paths`, `cannot read "<path>" as an image: model "<model>" does not declare image input; switch to an image-capable model to read images`, and the mismatch repair `cannot read "<path>": the <ext> extension declares <type>, but the bytes use a different image format; rename the file to match its actual format if it is PNG/JPEG/WebP/GIF, or convert it to one of those formats`. A failed 16-bit conversion reports `cannot read "<path>": the 16-bit PNG could not be converted to the canonical 8-bit sRGB form; convert it to an 8-bit PNG/JPEG/WebP and retry`. Region reads reject empty or out-of-scope attachment ids and invalid preview rectangles before storage mutation. Provider and policy templates are quoted in their package READMEs. Guarded-mutation failures additionally carry their recovery instruction in the message, appended by this package's model-facing error wrapper: `FS_STALE_VERSION` gets `— re-read the file, then retry`, and `FS_NOT_OBSERVED` gets `— read the file, then retry`; the structured code is preserved. After that reread confirms absence, edit reports `FS_NOT_FOUND` instead of repeating a stale remedy, while write uses guarded creation.
Failures are normalized as `Error: <message>`. This package's stable validation and read messages are `file_path must be a non-empty string`, `limit must be less than or equal to <max>`, `old_string must be a non-empty string`, `old_string and new_string must differ`, `cannot read "<path>": not found`, `cannot read "<path>": not a regular file`, `offset <offset> is out of range for "<path>" (<total> lines)`, `cannot read "<path>": read_image only accepts PNG/JPEG/WebP/GIF paths`, `cannot read "<path>" as an image: model "<model>" does not declare image input; switch to an image-capable model to read images`, and the mismatch repair `cannot read "<path>": the <ext> extension declares <type>, but the bytes use a different image format; rename the file to match its actual format if it is PNG/JPEG/WebP/GIF, or convert it to one of those formats`. A failed 16-bit conversion reports `cannot read "<path>": the 16-bit PNG could not be converted to the canonical 8-bit sRGB form; convert it to an 8-bit PNG/JPEG/WebP and retry`. Provider and policy templates are quoted in their package READMEs. Guarded-mutation failures additionally carry their recovery instruction in the message, appended by this package's model-facing error wrapper: `FS_STALE_VERSION` gets `— re-read the file, then retry`, and `FS_NOT_OBSERVED` gets `— read the file, then retry`; the structured code is preserved. After that reread confirms absence, edit reports `FS_NOT_FOUND` instead of repeating a stale remedy, while write uses guarded creation.
#### Token effect
@@ -173,4 +171,5 @@ Append-only; newly visible content follows the reusable request prefix and does
- **`read` handles UTF-8 text files only** — images use the separate extension-routed `read_image` tool; PDF, audio, and video remain deferred. A directory target is `FS_NOT_REGULAR_FILE`.
- **Extension-declared media type** — the extension selects the declared type and the attachment store's magic-byte validation stays authoritative; a correctly formatted image under a wrong extension is refused with the rename remedy rather than sniffed.
- **No inline image preview on the tool-result card** — UI surfaces render the image result generically (the durable reference, not pixels); inline rendering is deferred to the UI packages.
- **No attachment-region tool** — an agent may crop an image through other available tools when it has a filesystem path. A pasted or dragged image without a path cannot be re-read at a higher resolution.
- **No timeout surface**`read`/`write`/`edit` take no timeout argument and declare no `timeout-policy` budget; cancellation rides `exec.signal` only ([provider rationale](../README.md#no-timeouts-on-file-io)).
+8 -9
View File
@@ -2,7 +2,7 @@
[English](README.md) | 中文
**面向模型的文件系统工具**`read``read_image``read_image_region``write``edit`)及其**执行器**。这是文件系统栈的消费方层:拥有工具名称、JSON Schema、参数校验、提示词段、**读取窗口逻辑**和结果格式化。它**直接**通过 `ctx.fs` 提供方约定([`@deepseek-ai/dsh-fs`](../fs))读取、写入和编辑。新鲜度与观察策略由独立插件([`@deepseek-ai/dsh-fs-observation-policy`](../fs-observation-policy))通过 `fs/*` 事件门禁贡献;工具不与其方法耦合。使用施加沙箱限制的提供方时,逐会话执行需要共享沙箱策略服务,工具还会为文件系统变更提供升权路径。
**面向模型的文件系统工具**`read``read_image``write``edit`)及其**执行器**。这是文件系统栈的消费方层:拥有工具名称、JSON Schema、参数校验、提示词段、**读取窗口逻辑**和结果格式化。它**直接**通过 `ctx.fs` 提供方约定([`@deepseek-ai/dsh-fs`](../fs))读取、写入和编辑。新鲜度与观察策略由独立插件([`@deepseek-ai/dsh-fs-observation-policy`](../fs-observation-policy))通过 `fs/*` 事件门禁贡献;工具不与其方法耦合。使用施加沙箱限制的提供方时,逐会话执行需要共享沙箱策略服务,工具还会为文件系统变更提供升权路径。
```ts ignore-check
// Default deployment: a ctx.fs provider, the policy plugin, then the tools.
@@ -14,7 +14,7 @@ await ctx.plugin(ToolFs) // this package — re
`@deepseek-ai/dsh-fs-observation-policy` 是**可选的**:省略时,工具直接使用裸提供方(无条件写入/覆盖/编辑,无已观察状态)。加载这些工具的部署也应加载该插件,从而提供写入/编辑前读取行为。
`read_image``read_image_region` 只在持久 `ctx.attachments` 服务已挂载时注册。执行时还要求确切路由的模型声明 `image` 输入,通过 `ctx.llm.resolveModelInfo` 从会话最新请求 header 解析,缺失时回退到 agent 选项。`read_image_region` 只接受调用会话已经引用的完整附件 ID,因此可以裁剪没有文件路径的用户上传图片,但不能越过会话范围
`read_image` 只在持久 `ctx.attachments` 服务已挂载时注册。执行时还要求确切路由的模型声明 `image` 输入,通过 `ctx.llm.resolveModelInfo` 依次从会话最新请求 header 和 agent 选项解析。
## 配置
@@ -32,14 +32,13 @@ await ctx.plugin(ToolFs) // this package — re
| 工具 | 参数 | 行为 |
|---|---|---|
| `read` | `file_path``offset?``limit?` | 带行号的 UTF-8 内容和分页 footer。`offset` 从 1 开始;`limit` 默认为配置的 `readLimit`2000),上限也为该值。 |
| `read_image` | `file_path` | 通过有界字节 seam 读取 PNG/JPEG/WebP/GIF 文件,经 `ctx.attachments.saveImage` 持久保存,并在小型元数据信封旁返回图像块。只有确切路由的模型声明图像输入时才会成功。 |
| `read_image_region` | `attachment_id``preview_width``preview_height``x``y``width``height` | 解析会话有权访问的图片,把预览坐标矩形映射到存储主版本,持久保存裁剪结果并返回新图片块。 |
| `read_image` | `file_path` | 通过有界字节 seam 读取 PNG/JPEG/WebP/GIF 文件,经 `ctx.attachments.saveImage` 持久保存,并在小型元数据信封旁返回图像块。Harness 会在下一次模型请求前校验并缩小受支持的大图,因此模型可以直接读取源文件,无需先创建缩略图。只有确切路由的模型声明图像输入时才会成功。 |
| `write` | `file_path``content` | 创建文件或完整替换文件。有策略插件时:覆盖现有文件要求先在未变版本上执行 `read`;创建新文件不需要。没有插件时:无条件执行。 |
| `edit` | `file_path`、非空 `old_string``new_string``replace_all?` | 字面量替换;除非 `replace_all` 为 true,否则要求唯一匹配。有策略插件时:要求先执行 `read`(任何窗口),且文件此后未变。没有插件时:无条件执行。 |
字段名使用 snake_case,与 Claude Code 和现有 harness 工具 schema 一致。
结构化成功值分别为:`read``{ path, offset, lines: [{ number, text }], totalLines }``read_image``{ path, image: { attachmentId, mediaType, bytes, width, height, name?, sourceWidth?, sourceHeight? } }``read_image_region``{ sourceAttachmentId, preview, crop, image }``write``{ path, operation: 'create' | 'update', before: string | null, after }``edit``{ path, before, after }`。图片 source 字段只在主版本准备缩小了提交光栅时出现。原生渲染器会保留下方带行号的读取结果和变更确认。`write``edit` 从这些值派生可回放的 diff 卡片元数据,`read` 派生可回放的读取卡片窗口 `{ path, offset, lines, totalLines, lang? }`;仅用于执行的结构化值不会添加到 `tool/result`,图片渲染器则会发出由结果记录的持久图片块。
结构化成功值分别为:`read``{ path, offset, lines: [{ number, text }], totalLines }``read_image``{ path, image: { attachmentId, mediaType, bytes, width, height, name?, sourceWidth?, sourceHeight? } }``write``{ path, operation: 'create' | 'update', before: string | null, after }``edit``{ path, before, after }`。图片 source 字段只在主版本准备缩小了提交光栅时出现。原生渲染器会保留下方带行号的读取结果和变更确认。`write``edit` 从这些值派生可回放的 diff 卡片元数据,`read` 派生可回放的读取卡片窗口 `{ path, offset, lines, totalLines, lang? }`;仅用于执行的结构化值不会添加到 `tool/result`,图片渲染器则会发出由结果记录的持久图片块。
## 工具就是执行器;策略是事件门禁
@@ -47,7 +46,6 @@ await ctx.plugin(ToolFs) // this package — re
- **read**:一次 `ctx.fs.stat`(用于类型、大小路由和版本),随后调用 `readText`/`streamText`,构建行窗口,再发出 `fs/observed`,使用普通 `ctx.emit`。(1 次 stat。)
- **read_image**:在任何 I/O 之前校验参数、扩展名、附件可用性、部署接受的媒体类型和图像路由;随后一次 `ctx.fs.stat`(目标缺失时与 `read` 一样记录 `absent` 观察)、以 `imageLimits.maxImageBytes``imageLimits.maxMessageImageBytes` 中较小者为上限的有界 `ctx.fs.readBytes`(结果是携带一张图像的一条消息)、`attachments.saveImage`(内容寻址,因此在 `tool/result` 事件追加时图像块引用的对象已持久提交),最后发出 `fs/observed`。(1 次 stat。)
- **read_image_region**:只从当前会话消息解析完整附件 ID,校验整数预览坐标,通过 `attachments.cropImage` 把矩形映射到存储主版本,并把持久裁剪结果作为图片块返回。它不执行文件系统路径操作,也不发出 `fs/observed` 事件。
- **write**:调用 `ctx.waterfall('fs/write-intent', target, exec, () => undefined)` 取得可选防护,然后调用 `ctx.fs.writeText(target, content, intent)`,再发出 `fs/observed`。(0 次 stat。)
- **edit**:调用 `ctx.waterfall('fs/edit-intent', target, exec, () => undefined)` 取得可选防护,然后调用 `ctx.fs.editText(target, edit, intent)`,再发出 `fs/observed`。(0 次 stat。)
@@ -101,7 +99,7 @@ Use the edit tool for targeted changes to existing UTF-8 text files. It replaces
#### 模型看到的内容
模型会看到已生成的 [`read`、`read_image`、`read_image_region`、`write` 和 `edit` schema](../../../docs/tool-catalog.zh.md#deepseek-aidsh-tool-fs),参数使用 snake_case。图片工具只在持久附件存储已挂载时出现;schema 本身与路由无关,严格门禁在执行时拒绝。作用域工具限制可以为某个 agent 移除任一定义。
模型会看到已生成的 [`read`、`read_image`、`write` 和 `edit` schema](../../../docs/tool-catalog.zh.md#deepseek-aidsh-tool-fs),参数使用 snake_case。图片工具只在持久附件存储已挂载时出现;schema 本身与路由无关,严格门禁在执行时拒绝。作用域工具限制可以为某个 agent 移除任一定义。
#### Token 影响
@@ -129,7 +127,7 @@ Use the edit tool for targeted changes to existing UTF-8 text files. It replaces
#### 模型看到的内容
成功的 `read_image` 返回 `<path><displayPath></path>``<type>image</type>` 和写明媒体类型、主版本尺寸与字节数的 `<content>` 信封,随后是作为原生图像块的图像本身。成功的 `read_image_region` 返回 `image-region` 信封,写明源附件、提交的预览尺寸和矩形及结果尺寸,随后是作为原生图像块的裁剪结果。新持久引用会随结果写入会话日志,然后才进入下一次模型请求。请求适配器从主版本派生预览,因此之后的局部读取不会从已经缩小的预览继续裁剪
成功的 `read_image` 返回 `<path><displayPath></path>``<type>image</type>` 和写明媒体类型、主版本尺寸与字节数的 `<content>` 信封,随后是作为原生图像块的图像本身。结果会随持久引用写入会话日志,然后才进入下一次模型请求
#### Token 影响
@@ -157,7 +155,7 @@ Use the edit tool for targeted changes to existing UTF-8 text files. It replaces
#### 模型看到的内容
失败会规范化为 `Error: <message>`。本包稳定的校验和读取消息是 `file_path must be a non-empty string``limit must be less than or equal to <max>``old_string must be a non-empty string``old_string and new_string must differ``cannot read "<path>": not found``cannot read "<path>": not a regular file``offset <offset> is out of range for "<path>" (<total> lines)``cannot read "<path>": read_image only accepts PNG/JPEG/WebP/GIF paths``cannot read "<path>" as an image: model "<model>" does not declare image input; switch to an image-capable model to read images`,以及类型不匹配的修复消息 `cannot read "<path>": the <ext> extension declares <type>, but the bytes use a different image format; rename the file to match its actual format if it is PNG/JPEG/WebP/GIF, or convert it to one of those formats`。16-bit 转换失败会报告 `cannot read "<path>": the 16-bit PNG could not be converted to the canonical 8-bit sRGB form; convert it to an 8-bit PNG/JPEG/WebP and retry`局部读取会在改变存储前拒绝空白或超出会话范围的附件 ID 以及无效预览矩形。提供方和策略模板在各自包的 README 中逐字列出。防护变更失败还会在消息中携带恢复指令,由本包面向模型的错误包装追加:`FS_STALE_VERSION` 追加 `re-read the file, then retry``FS_NOT_OBSERVED` 追加 `read the file, then retry`;结构化错误码保持不变。该次重新读取确认缺失后,edit 会报告 `FS_NOT_FOUND`,不会重复陈旧恢复指令;write 则使用带防护的创建。
失败会规范化为 `Error: <message>`。本包稳定的校验和读取消息是 `file_path must be a non-empty string``limit must be less than or equal to <max>``old_string must be a non-empty string``old_string and new_string must differ``cannot read "<path>": not found``cannot read "<path>": not a regular file``offset <offset> is out of range for "<path>" (<total> lines)``cannot read "<path>": read_image only accepts PNG/JPEG/WebP/GIF paths``cannot read "<path>" as an image: model "<model>" does not declare image input; switch to an image-capable model to read images`,以及类型不匹配的修复消息 `cannot read "<path>": the <ext> extension declares <type>, but the bytes use a different image format; rename the file to match its actual format if it is PNG/JPEG/WebP/GIF, or convert it to one of those formats`。16-bit 转换失败会报告 `cannot read "<path>": the 16-bit PNG could not be converted to the canonical 8-bit sRGB form; convert it to an 8-bit PNG/JPEG/WebP and retry`。提供方和策略模板在各自包的 README 中逐字列出。防护变更失败还会在消息中携带恢复指令,由本包面向模型的错误包装追加:`FS_STALE_VERSION` 追加 `re-read the file, then retry``FS_NOT_OBSERVED` 追加 `read the file, then retry`;结构化错误码保持不变。该次重新读取确认缺失后,edit 会报告 `FS_NOT_FOUND`,不会重复陈旧恢复指令;write 则使用带防护的创建。
#### Token 影响
@@ -173,4 +171,5 @@ Use the edit tool for targeted changes to existing UTF-8 text files. It replaces
- **`read` 只处理 UTF-8 文本文件**:图像使用独立的、按扩展名路由的 `read_image` 工具;PDF、音频和视频仍延期处理。目录目标为 `FS_NOT_REGULAR_FILE`
- **媒体类型按扩展名声明**:扩展名选择声明类型,附件存储的魔数校验保持权威;扩展名错误但格式正确的图像会得到改名修复提示,而不是被嗅探接受。
- **工具结果卡片没有内嵌图像预览**:UI 表面以通用形式渲染图像结果(持久引用而非像素);内嵌渲染延后到 UI 包处理。
- **没有附件局部读取工具**:图片具有文件路径时,agent 可以用其他可用工具裁剪。粘贴或拖入但没有路径的图片无法按更高分辨率重新读取。
- **没有超时接口**`read`/`write`/`edit` 不接受超时参数,也不声明 `timeout-policy` 预算;取消只通过 `exec.signal` 传递(见[提供方理由](../README.zh.md#no-timeouts-on-file-io))。
+5 -145
View File
@@ -1,7 +1,5 @@
/**
* The model-facing image tools: `read_image` commits a PNG/JPEG/WebP/GIF file,
* while `read_image_region` crops a session-authorized durable attachment by
* coordinates measured on the exact preview shown to the model.
* The model-facing `read_image` tool commits a PNG/JPEG/WebP/GIF file.
*
* The route gate is deliberately stricter than the host upload preflight. An
* image-reading tool is useful only when the exact calling route can inspect
@@ -13,7 +11,7 @@
import { basename, extname } from 'node:path'
import type { Context } from '@deepseek-ai/cordis'
import { AttachmentError, AttachmentId } from '@deepseek-ai/dsh-attachment'
import type { ImageAttachmentRef, ImageMediaType, PreviewImageCrop } from '@deepseek-ai/dsh-attachment'
import type { ImageAttachmentRef, ImageMediaType } from '@deepseek-ai/dsh-attachment'
import type { ContentBlock } from '@deepseek-ai/dsh-llm'
import { defineTool } from '@deepseek-ai/dsh-tools'
import type { GenericCallView, ToolExecution } from '@deepseek-ai/dsh-tools'
@@ -62,14 +60,6 @@ export interface ImageReadValue {
}
}
/** Structured result of cropping a session-authorized image attachment. */
export interface ImageRegionReadValue {
sourceAttachmentId: string
preview: { width: number; height: number }
crop: { x: number; y: number; width: number; height: number }
image: ImageReadValue['image']
}
/**
* Map a model-supplied path to its declared image media type by extension.
* @param filePath - the raw `file_path` argument (not yet resolved).
@@ -120,55 +110,6 @@ export function imageRefFromValue(image: ImageReadValue['image']): ImageAttachme
}
}
function findImageRef(
content: readonly ContentBlock[],
attachmentId: string,
): ImageAttachmentRef | undefined {
for (const block of content) {
if (block.type === 'image' && block.attachment.attachmentId === attachmentId) return block.attachment
if (block.type === 'tool-result') {
const nested = findImageRef(block.content, attachmentId)
if (nested !== undefined) return nested
}
}
return undefined
}
function sessionImageRef(exec: ToolExecution, attachmentId: string): ImageAttachmentRef {
const session = exec.agent?.session
if (session === undefined) {
throw new Error('read_image_region requires an active agent session')
}
for (const message of session.deriveMessages()) {
const ref = findImageRef(message.content, attachmentId)
if (ref !== undefined) return ref
}
throw new Error(`attachment "${attachmentId}" is not referenced by the current session`)
}
function positiveInteger(value: number, name: string): number {
if (!Number.isSafeInteger(value) || value <= 0) throw new Error(`${name} must be a positive integer`)
return value
}
function nonNegativeInteger(value: number, name: string): number {
if (!Number.isSafeInteger(value) || value < 0) throw new Error(`${name} must be a non-negative integer`)
return value
}
function regionReadContent(value: ImageRegionReadValue): ContentBlock[] {
return [
{
type: 'text',
text: `<attachment>${value.sourceAttachmentId}</attachment>\n<type>image-region</type>\n<content>\n`
+ `preview ${value.preview.width}x${value.preview.height} px; crop `
+ `x=${value.crop.x}, y=${value.crop.y}, width=${value.crop.width}, height=${value.crop.height}; `
+ `result ${value.image.width}x${value.image.height} px\n</content>`,
},
{ type: 'image', attachment: imageRefFromValue(value.image) },
]
}
/**
* Format an image read as the model-facing envelope beside its image block.
* A downscaled read names the on-disk dimensions and the multiplier that maps
@@ -220,7 +161,9 @@ function imageReadContent(value: ImageReadValue): ContentBlock[] {
export function applyReadImageTool(ctx: Context): void {
ctx.tools.register(defineTool({
name: 'read_image',
description: 'Read a PNG/JPEG/WebP/GIF file and return the image itself. Requires the current model to accept image input.',
description: 'Read a PNG/JPEG/WebP/GIF file and return the image itself. '
+ 'Harness validates and downscales large supported images before the next model request, so use this tool directly instead of installing image libraries or creating thumbnails merely to inspect an image. '
+ 'Independent files may be read concurrently in small batches. Requires the current model to accept image input.',
parameters: {
file_path: { type: 'string', required: true, description: 'Path to the image file, resolved by the filesystem backend.' },
},
@@ -333,87 +276,4 @@ export function applyReadImageTool(ctx: Context): void {
}
},
}))
ctx.tools.register(defineTool({
name: 'read_image_region',
description: 'Crop a region from an image attachment already visible in this session. Coordinates use the preview dimensions supplied beside that image.',
parameters: {
attachment_id: { type: 'string', required: true, description: 'Complete attachment id shown beside the image.' },
preview_width: { type: 'integer', required: true, description: 'Width of the preview shown to the model.' },
preview_height: { type: 'integer', required: true, description: 'Height of the preview shown to the model.' },
x: { type: 'integer', required: true, description: 'Left edge in preview pixels.' },
y: { type: 'integer', required: true, description: 'Top edge in preview pixels.' },
width: { type: 'integer', required: true, description: 'Crop width in preview pixels.' },
height: { type: 'integer', required: true, description: 'Crop height in preview pixels.' },
},
output: {
schema: {
type: 'object',
additionalProperties: false,
properties: {
sourceAttachmentId: { type: 'string', required: true },
preview: {
type: 'object',
additionalProperties: false,
required: true,
properties: {
width: { type: 'integer', required: true },
height: { type: 'integer', required: true },
},
},
crop: {
type: 'object',
additionalProperties: false,
required: true,
properties: {
x: { type: 'integer', required: true },
y: { type: 'integer', required: true },
width: { type: 'integer', required: true },
height: { type: 'integer', required: true },
},
},
image: IMAGE_VALUE_SCHEMA,
},
},
render: (_args, value) => regionReadContent(value),
},
isConcurrencySafe: () => true,
async execute(args, exec) {
const attachmentId = args.attachment_id.trim()
if (attachmentId.length === 0) throw new Error('attachment_id must be a non-empty string')
const ref = sessionImageRef(exec, attachmentId)
await assertImageCapableRoute(ctx, exec, attachmentId)
const crop: PreviewImageCrop = {
previewWidth: positiveInteger(args.preview_width, 'preview_width'),
previewHeight: positiveInteger(args.preview_height, 'preview_height'),
x: nonNegativeInteger(args.x, 'x'),
y: nonNegativeInteger(args.y, 'y'),
width: positiveInteger(args.width, 'width'),
height: positiveInteger(args.height, 'height'),
}
const saved = await ctx.attachments.cropImage(ref, crop, exec.signal)
return {
sourceAttachmentId: ref.attachmentId,
preview: { width: crop.previewWidth, height: crop.previewHeight },
crop: { x: crop.x, y: crop.y, width: crop.width, height: crop.height },
image: {
attachmentId: saved.ref.attachmentId,
mediaType: saved.ref.mediaType,
bytes: saved.ref.bytes,
width: saved.ref.width,
height: saved.ref.height,
...saved.ref.name === undefined ? {} : { name: saved.ref.name },
...saved.ref.sourceWidth === undefined ? {} : { sourceWidth: saved.ref.sourceWidth },
...saved.ref.sourceHeight === undefined ? {} : { sourceHeight: saved.ref.sourceHeight },
},
}
},
presentCall(args): GenericCallView {
return {
card: 'generic',
title: `Read image region ${args.attachment_id}`,
kind: 'read',
}
},
}))
}
+3 -217
View File
@@ -12,7 +12,7 @@ import { join } from 'node:path'
import { Context } from '@deepseek-ai/cordis'
import { CodeRuntime } from '@deepseek-ai/dsh-code-runtime'
import type { CodeRunRequest, CodeRunResult } from '@deepseek-ai/dsh-code-runtime'
import { CallId, createUserMessage, LlmAdapter, LlmRuntime } from '@deepseek-ai/dsh-llm'
import { CallId, LlmAdapter, LlmRuntime } from '@deepseek-ai/dsh-llm'
import type { GenerateOptions, LlmModelInfo, LlmResolvedModelInfo, Message, StreamChunk } from '@deepseek-ai/dsh-llm'
import SystemPrompt from '@deepseek-ai/dsh-system-prompt'
import ToolRuntime, { RUN_CODE_NAME } from '@deepseek-ai/dsh-tools'
@@ -175,214 +175,6 @@ describe('imageRefFromValue', () => {
})
})
describe('read_image_region', () => {
it('crops a session-visible attachment and returns a new logged image reference', async () => {
const ctx = await setup()
const attachments = ctx.attachments
const source = await attachments.saveImage({ data: PNG_3X3, mediaType: 'image/png', name: 'grid.png' })
const history = [createUserMessage({
content: [{ type: 'image', attachment: source.ref }],
source: { kind: 'plugin', plugin: 'test' },
})]
const result = await call(ctx, 'read_image_region', {
attachment_id: source.ref.attachmentId,
preview_width: 3,
preview_height: 3,
x: 1,
y: 0,
width: 2,
height: 2,
}, agentOn('vision-model', 'visual', history))
expect(result.isError).toBe(false)
expect(result.content[0]).toMatchObject({
type: 'text',
text: expect.stringContaining('crop x=1, y=0, width=2, height=2') as string,
})
expect(result.content[1]).toMatchObject({
type: 'image',
attachment: { width: 2, height: 2, name: 'grid-crop.png' },
})
const cropped = result.content[1]
if (cropped?.type !== 'image') throw new Error('expected cropped image block')
await expect(attachments.readImage(cropped.attachment)).resolves.toMatchObject({
ref: { attachmentId: cropped.attachment.attachmentId },
})
})
it('refuses an attachment that is absent from the current session', async () => {
const ctx = await setup()
const result = await call(ctx, 'read_image_region', {
attachment_id: `sha256:${'f'.repeat(64)}`,
preview_width: 800,
preview_height: 800,
x: 0,
y: 0,
width: 100,
height: 100,
}, agentOn('vision-model'))
expect(result.isError).toBe(true)
expect(text(result)).toContain('not referenced by the current session')
})
it('finds images nested in tool results after skipping a non-matching nested result', async () => {
const ctx = await setup()
const source = await ctx.attachments.saveImage({ data: PNG_3X3, mediaType: 'image/png' })
const history = [createUserMessage({
content: [
{ type: 'tool-result', toolCallId: CallId('unrelated'), content: [{ type: 'text', text: 'none' }] },
{ type: 'tool-result', toolCallId: CallId('nested'), content: [{ type: 'image', attachment: source.ref }] },
],
source: { kind: 'plugin', plugin: 'test' },
})]
const result = await call(ctx, 'read_image_region', {
attachment_id: source.ref.attachmentId,
preview_width: 3,
preview_height: 3,
x: 0,
y: 0,
width: 1,
height: 1,
}, agentOn('vision-model', 'visual', history))
expect(result.isError).toBe(false)
})
it('continues across an earlier session message without the requested image', async () => {
const ctx = await setup()
const source = await ctx.attachments.saveImage({ data: PNG_3X3, mediaType: 'image/png' })
const history = [
createUserMessage({
content: [{ type: 'text', text: 'before image' }],
source: { kind: 'plugin', plugin: 'test' },
}),
createUserMessage({
content: [{ type: 'image', attachment: source.ref }],
source: { kind: 'plugin', plugin: 'test' },
}),
]
const result = await call(ctx, 'read_image_region', {
attachment_id: source.ref.attachmentId,
preview_width: 3,
preview_height: 3,
x: 0,
y: 0,
width: 1,
height: 1,
}, agentOn('vision-model', 'visual', history))
expect(result.isError).toBe(false)
})
it('rejects a missing session, empty id, and invalid coordinate arguments', async () => {
const ctx = await setup()
const base = {
attachment_id: `sha256:${'f'.repeat(64)}`,
preview_width: 1,
preview_height: 1,
x: 0,
y: 0,
width: 1,
height: 1,
}
const noSession = await call(ctx, 'read_image_region', base)
expect(text(noSession)).toContain('requires an active agent session')
const empty = await call(ctx, 'read_image_region', { ...base, attachment_id: ' ' }, agentOn('vision-model'))
expect(text(empty)).toContain('attachment_id must be a non-empty string')
const source = await ctx.attachments.saveImage({ data: PNG_1X1, mediaType: 'image/png' })
const history = [createUserMessage({
content: [{ type: 'image', attachment: source.ref }],
source: { kind: 'plugin', plugin: 'test' },
})]
const agent = agentOn('vision-model', 'visual', history)
for (const [field, value, expected] of [
['preview_width', 0, 'preview_width must be a positive integer'],
['preview_height', 0, 'preview_height must be a positive integer'],
['x', -1, 'x must be a non-negative integer'],
['y', -1, 'y must be a non-negative integer'],
['width', 0, 'width must be a positive integer'],
['height', 0, 'height must be a positive integer'],
] as const) {
const result = await call(ctx, 'read_image_region', {
...base,
attachment_id: source.ref.attachmentId,
[field]: value,
}, agent)
expect(text(result)).toContain(expected)
}
})
it('projects optional crop metadata from a provider result', async () => {
class CropMetadataStore extends AttachmentStore {
readonly imageLimits: ImageAttachmentLimits = {
maxImageBytes: 1024,
maxImagesPerMessage: 1,
maxMessageImageBytes: 1024,
maxImagePixels: 100,
maxImageDimension: 100,
mediaTypes: ['image/png'],
}
validateImage(): Promise<void> { return Promise.resolve() }
saveImage(): Promise<SavedImageAttachment> { throw new Error('not used') }
readImage(): Promise<StoredImageAttachment> { throw new Error('not used') }
override cropImage(ref: ImageAttachmentRef): Promise<SavedImageAttachment> {
return Promise.resolve({
ref: { ...ref, sourceWidth: 2, sourceHeight: 2 },
source: { mediaType: ref.mediaType, bytes: ref.bytes, width: 2, height: 2 },
})
}
}
const ctx = await setup({ attachments: false })
await ctx.plugin(CropMetadataStore)
const ref: ImageAttachmentRef = {
attachmentId: AttachmentId(`sha256:${'a'.repeat(64)}`),
mediaType: 'image/png', bytes: 1, width: 1, height: 1,
}
const history = [createUserMessage({
content: [{ type: 'image', attachment: ref }],
source: { kind: 'plugin', plugin: 'test' },
})]
const result = await call(ctx, 'read_image_region', {
attachment_id: ref.attachmentId,
preview_width: 1,
preview_height: 1,
x: 0,
y: 0,
width: 1,
height: 1,
}, agentOn('vision-model', 'visual', history))
expect(result.content[1]).toMatchObject({
type: 'image',
attachment: { sourceWidth: 2, sourceHeight: 2 },
})
expect(result.content[1]).not.toHaveProperty('attachment.name')
})
it('declares a generic read presentation for image-region calls', async () => {
const ctx = await setup()
expect(ctx.tools.get('read_image_region')?.presentCall?.({
attachment_id: 'sha256:abc',
preview_width: 1,
preview_height: 1,
x: 0,
y: 0,
width: 1,
height: 1,
}))
.toEqual({ card: 'generic', title: 'Read image region sha256:abc', kind: 'read' })
})
})
describe('read_image happy path', () => {
it('commits the bytes durably and renders the envelope beside an image block', async () => {
await writeFile(join(dir, 'red.png'), PNG_1X1)
@@ -773,7 +565,7 @@ describe('registration surface', () => {
const attachmentsFiber = await ctx.plugin(LocalAttachmentStore, { dshHome: home })
const toolFsFiber = await ctx.plugin(ToolFs)
const names = () => ctx.tools.schemas().map(schema => schema.name).sort()
expect(names()).toEqual(['edit', 'read', 'read_image', 'read_image_region', 'write'])
expect(names()).toEqual(['edit', 'read', 'read_image', 'write'])
// Disposing only the attachment store tears down the scoped inject fiber:
// read_image withdraws while the unconditional tools stay registered.
@@ -782,7 +574,7 @@ describe('registration surface', () => {
// Remounting the store restores the conditional registration.
const remounted = await ctx.plugin(LocalAttachmentStore, { dshHome: home })
expect(names()).toEqual(['edit', 'read', 'read_image', 'read_image_region', 'write'])
expect(names()).toEqual(['edit', 'read', 'read_image', 'write'])
// Disposing the whole plugin withdraws every tool, read_image included.
await toolFsFiber.dispose()
@@ -801,12 +593,6 @@ describe('registration surface', () => {
kind: 'read',
locations: [{ path: 'shot.png' }],
})
expect(ctx.tools.executionMode({
signal: testToolSignal,
callId: CallId('region-parallel'),
name: 'read_image_region',
arguments: { attachment_id: 'sha256:a', preview_width: 1, preview_height: 1, x: 0, y: 0, width: 1, height: 1 },
})).toEqual({ kind: 'parallel' })
})
})
+2 -2
View File
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/llm/llm-deepseek/README.md
README.md: b20d93394055e3e10dfb5a932660b6a510428492
README.zh.md: 6e8166227d740c0431c17c091d68b5d56aea0dc5
README.md: bb7f6a520701134cd43ff6223ef4efbf82d02eb4
README.zh.md: 934c189232711655aa785a7497f5bb6dff1cbb46
+3 -3
View File
@@ -49,11 +49,11 @@ The package root exposes the Cordis plugin contract and `DeepSeekAdapter`; wire
The plugin registers the single provider route `deepseek-official` together with its resolved `retryPolicy`; omission resolves to normal mode with five retries. A request selects it with `provider: deepseek-official`; its `model` is passed through as the wire `model` string, so changing DeepSeek models does not require lifecycle-time registration. Omitting `models` advertises `deepseek-v4-flash`, `deepseek-v4-pro`, and the image-capable `deepseek-v4-flash-vision-exp`, each with a 1,000,000-token context window; an explicit list replaces those defaults, while `models: []` advertises none. Catalog entries are exposed through `ctx.llm.listModels('deepseek-official')` for clients such as ACP editors and the Web selector, but remain advisory: unlisted model ids still pass through unchanged as text-only routes. An omitted entry name defaults to its id, and omitted `inputModalities` means `text` only.
An image-capable catalog entry declares `inputModalities: [text, image]` and may set `imagePixelBudget`, `imageMaxBytes`, or `imageDetail: low`. The ordinary default is 640,000 total pixels and 1MiB encoded bytes; low detail defaults to 512 by 512 total pixels. The attachment store scales by `min(1, sqrt(pixelBudget / (width * height)))` and rounds inward to keep the pixel count at or below the hard cap, so a 2048 by 1024 master becomes about 1130 by 565 instead of a forced square. Request encoders run lazily: low-color images try PNG (palette only without alpha) then WebP 85 and 80, other alpha images try WebP 85 then 80, and other opaque images try JPEG 85 then 80; dimensions shrink only when both quality attempts exceed 1MiB. Concurrent generation of one `variantId` shares one transform. A caller can cancel its own wait without interrupting other waiters; the transform stops when no waiter remains. The adapter uploads the exact derived request bytes through `POST /files` and sends `{type: "file", file_id}` blocks. It never falls back to an inline data URL. Every retained image is preceded by stable text naming the complete attachment id and actual request dimensions. Preview-coordinate arguments are included only when the request exposes `read_image_region`. User, tool-result, agent-loop, compaction, and direct `ctx.llm.stream` requests all use this projection. Text-only routes receive stable attachment placeholders while durable history keeps its image references.
An image-capable catalog entry declares `inputModalities: [text, image]` and may set `imagePixelBudget`, `imageMaxBytes`, or `imageDetail: low`. The ordinary default is 640,000 total pixels and 1MiB encoded bytes; low detail defaults to 512 by 512 total pixels. The attachment store scales by `min(1, sqrt(pixelBudget / (width * height)))` and rounds inward to keep the pixel count at or below the hard cap, so a 2048 by 1024 master becomes about 1130 by 565 instead of a forced square. Request encoders run lazily: low-color images try PNG (palette only without alpha) then WebP 85 and 80, other alpha images try WebP 85 then 80, and other opaque images try JPEG 85 then 80; dimensions shrink only when both quality attempts exceed 1MiB. Concurrent generation of one `variantId` shares one transform. A caller can cancel its own wait without interrupting other waiters; the transform stops when no waiter remains. The adapter uploads the exact derived request bytes through `POST /files` and sends `{type: "file", file_id}` blocks. It never falls back to an inline data URL. Every retained image is preceded by stable text naming the complete attachment id and actual request dimensions. User, tool-result, agent-loop, compaction, and direct `ctx.llm.stream` requests all use this projection. Text-only routes receive stable attachment placeholders while durable history keeps its image references.
`maxRequestFilesBytes` and `maxImagesPerRequest` bound the retained request versions at 128MiB and 600 images by default. The byte and count quanta must not exceed their corresponding bounds. Before attachment reads, the adapter uses each route's request-version byte cap as a conservative upper bound and removes the oldest over-budget prefix; only retained masters are read and transformed. Exact derived lengths are checked again without restoring omitted images. When the byte bound is crossed, the oldest prefix advances past the next 64MiB boundary; 129 one-megabyte images remove the oldest 65 and retain 64MiB, and that prefix stays unchanged until durable history exceeds 192MiB. Count overflow advances independently in `imageOffloadCountQuantum` steps. Removed images become the fixed model-visible placeholder `[image omitted to keep the request within its image limit; older images are omitted first. If this image is still needed, read its file again when a path is available; otherwise ask the user to attach it again.]`. This high-watermark projection avoids changing an old request prefix after every new image.
Uploaded ids are indexed below `DSH_HOME` by endpoint/API-key scope and request `variantId`. The variant covers the master attachment id, transform version, route pixel and byte budgets, crop, and encoder parameters, so Files API and inline-capable adapters refer to the same deterministic bytes. Uploads request a seven-day lifetime by default and store the server's `expires_at`. A local mapping with no more than one hour remaining is replaced before use; the adapter does not retrieve every remote file before chat. If chat reports expired, deleted, missing, or invalid file ids and names one or more ids used by the request, the adapter removes exactly those mappings. If the provider identifies stale file state without naming an id, it removes every file mapping used by that chat attempt. It then uploads the affected request versions again and retries chat once. A second stale-file rejection clears the mappings identified by that response and is returned without a third chat attempt. An upload response without a complete file object, matching byte count, and `expires_at` is never indexed; a later request therefore uploads again instead of trusting inconsistent local state. A malformed local upload index is treated as an empty cache and replaced by the next successful upload; permission and filesystem I/O failures still fail the request.
Uploaded ids are indexed below `DSH_HOME` by endpoint/API-key scope and request `variantId`. The variant covers the master attachment id, transform version, route pixel and byte budgets, and encoder parameters, so Files API and inline-capable adapters refer to the same deterministic bytes. Uploads request a seven-day lifetime by default and store the server's `expires_at`. A local mapping with no more than one hour remaining is replaced before use; the adapter does not retrieve every remote file before chat. If chat reports expired, deleted, missing, or invalid file ids and names one or more ids used by the request, the adapter removes exactly those mappings. If the provider identifies stale file state without naming an id, it removes every file mapping used by that chat attempt. It then uploads the affected request versions again and retries chat once. A second stale-file rejection clears the mappings identified by that response and is returned without a third chat attempt. An upload response without a complete file object, matching byte count, and `expires_at` is never indexed; a later request therefore uploads again instead of trusting inconsistent local state. A malformed local upload index is treated as an empty cache and replaced by the next successful upload; permission and filesystem I/O failures still fail the request.
Concurrent resolution of one scoped `variantId` shares one Files upload with waiter-local cancellation. One quota upload failure first paginates and collects the configured number of oldest `dsh-` files, then deletes that set before one upload retry. `DeepSeekFilesClient.delete`, `DeepSeekFileStore.release`, and `releaseAll` expose explicit remote-space reclamation. The current provider limits represented by this package are 128MiB per Files upload, 32MiB per chat-referenced image, 10,000 stored files, and 25GiB per API key; the default 1MiB request version remains below the two per-file limits.
@@ -104,7 +104,7 @@ Non-2xx responses throw `LlmError` with stable codes: `AUTH` (401/403), `QUOTA`
#### What the model sees
The selected DeepSeek model receives the harness system prompt, message history, tool schemas, stop sequences, and call config. The vision model receives retained user and tool-result images as Files API references beside stable attachment handles and preview dimensions; an over-budget older image is represented by the documented placeholder. Reasoning content from a prior assistant turn is passed back verbatim, whether or not that turn called a tool.
The selected DeepSeek model receives the harness system prompt, message history, tool schemas, stop sequences, and call config. The vision model receives retained user and tool-result images as Files API references beside stable attachment handles and request-image dimensions; an over-budget older image is represented by the documented placeholder. Reasoning content from a prior assistant turn is passed back verbatim, whether or not that turn called a tool.
#### Token effect
+3 -3
View File
@@ -49,11 +49,11 @@ harness LLM(大语言模型)seam 的 DeepSeek chat-completions 适配器:
该插件注册唯一提供方路由 `deepseek-official`,并一同注册解析后的 `retryPolicy`;省略时会解析为 normal 模式并重试五次。请求使用 `provider: deepseek-official` 选择该路由;其 `model` 会作为协议 `model` 字符串原样传递,因此更改 DeepSeek 模型不需要生命周期时注册。省略 `models` 会公布 `deepseek-v4-flash``deepseek-v4-pro` 与支持图片输入的 `deepseek-v4-flash-vision-exp`,三者的上下文窗口均为 1,000,000 token;显式列表会替换这些默认值,`models: []` 则不公布任何模型。Catalog 配置项通过 `ctx.llm.listModels('deepseek-official')` 公开给 ACPAgent Client Protocol)编辑器和 Web 选择器等客户端,但仍只提供建议:未列出模型 id 仍原样传递,并按纯文本路由处理。省略配置项 name 默认为其 id,省略 `inputModalities` 则表示仅支持 `text`
支持图片的 catalog 配置项声明 `inputModalities: [text, image]`,并可设置 `imagePixelBudget``imageMaxBytes``imageDetail: low`。普通默认值为总像素 640,000、编码字节 1MiBlow detail 的默认总像素为 512×512。附件存储按 `min(1, sqrt(pixelBudget / (width * height)))` 缩放,并向预算内取整,确保总像素不超过硬上限。因此 2048×1024 主版本会得到约 1130×565 的请求版本,而不会被强制变成正方形。请求编码按需执行:低色数图片先尝试 PNG,只有不带 alpha 通道时才使用 palette,再尝试质量 85 和 80 的 WebP;其他透明图片依次尝试质量 85 和 80 的 WebP;其他非透明图片依次尝试质量 85 和 80 的 JPEG。两个质量档均超过 1MiB 时才缩小尺寸。同一 `variantId` 的并发生成共享一次变换。调用方可以单独取消等待,不会中断其他等待方;没有等待方时才会停止变换。适配器通过 `POST /files` 上传确切的派生请求字节,再发送 `{type: "file", file_id}` 块,不会回退到内联 data URL。每张保留图片前都有稳定文本,写明完整附件 ID 和实际请求尺寸。只有当前请求公开 `read_image_region` 时才会提供预览坐标参数。User、工具结果、agent loop、压缩和直接 `ctx.llm.stream` 请求都使用该投影。纯文本路由会收到稳定的附件占位文本,持久历史继续保留图片引用。
支持图片的 catalog 配置项声明 `inputModalities: [text, image]`,并可设置 `imagePixelBudget``imageMaxBytes``imageDetail: low`。普通默认值为总像素 640,000、编码字节 1MiBlow detail 的默认总像素为 512×512。附件存储按 `min(1, sqrt(pixelBudget / (width * height)))` 缩放,并向预算内取整,确保总像素不超过硬上限。因此 2048×1024 主版本会得到约 1130×565 的请求版本,而不会被强制变成正方形。请求编码按需执行:低色数图片先尝试 PNG,只有不带 alpha 通道时才使用 palette,再尝试质量 85 和 80 的 WebP;其他透明图片依次尝试质量 85 和 80 的 WebP;其他非透明图片依次尝试质量 85 和 80 的 JPEG。两个质量档均超过 1MiB 时才缩小尺寸。同一 `variantId` 的并发生成共享一次变换。调用方可以单独取消等待,不会中断其他等待方;没有等待方时才会停止变换。适配器通过 `POST /files` 上传确切的派生请求字节,再发送 `{type: "file", file_id}` 块,不会回退到内联 data URL。每张保留图片前都有稳定文本,写明完整附件 ID 和实际请求尺寸。User、工具结果、agent loop、压缩和直接 `ctx.llm.stream` 请求都使用该投影。纯文本路由会收到稳定的附件占位文本,持久历史继续保留图片引用。
`maxRequestFilesBytes``maxImagesPerRequest` 限制请求中保留的请求版本,默认值分别为 128MiB 和 600 张。字节和数量步长不得超过对应上限。读取附件前,适配器以路由的请求版本字节上限作为保守上界,移除超预算的最旧前缀,只读取并转换保留的主版本。系统随后用确切派生长度再次检查,但不会重新加入已省略图片。字节数越过上限时,被移除的最旧前缀会越过下一个 64MiB 边界。由 1MiB 图片组成的历史达到 129MiB 时会移除最旧的 65 张并保留 64MiB;直到持久历史超过 192MiB,这个前缀才再次变化。图片数量超限时则按 `imageOffloadCountQuantum` 独立递增。移除的图片会变成固定模型可见占位文本 `[image omitted to keep the request within its image limit; older images are omitted first. If this image is still needed, read its file again when a path is available; otherwise ask the user to attach it again.]`。这种定量投影不会因每新增一张图片就改写较早的请求前缀。
上传 ID 按端点和 API key 作用域以及请求 `variantId` 记录在 `DSH_HOME` 下。变体身份覆盖主附件 ID、变换策略版本、路由像素和字节预算、裁剪区域及编码参数,因此 Files API 和支持内联的适配器引用同一份确定性字节。上传默认请求 7 天有效期,并保存服务端返回的 `expires_at`。本地映射剩余时间不超过一小时时会在使用前替换;适配器不会在每次 chat 前查询远端文件。如果 chat 报告文件 ID 已过期、删除、缺失或无效,并指出本次请求使用的一个或多个 ID,适配器只删除这些映射。如果响应只说明文件状态失效而没有指出 ID,适配器会删除该次 chat 使用的全部文件映射。随后重新上传受影响的请求版本,并重试一次 chat。第二次 chat 仍报告文件失效时,适配器会按该响应清理映射并返回错误,不会发起第三次 chat。上传响应若没有完整文件对象、匹配的字节数和 `expires_at`,就不会写入索引;后续请求会再次上传,而不是信任不一致的本地状态。本地上传索引格式损坏时按空缓存处理,并由下一次成功上传替换;权限和文件系统 I/O 错误仍使请求失败。
上传 ID 按端点和 API key 作用域以及请求 `variantId` 记录在 `DSH_HOME` 下。变体身份覆盖主附件 ID、变换策略版本、路由像素和字节预算及编码参数,因此 Files API 和支持内联的适配器引用同一份确定性字节。上传默认请求 7 天有效期,并保存服务端返回的 `expires_at`。本地映射剩余时间不超过一小时时会在使用前替换;适配器不会在每次 chat 前查询远端文件。如果 chat 报告文件 ID 已过期、删除、缺失或无效,并指出本次请求使用的一个或多个 ID,适配器只删除这些映射。如果响应只说明文件状态失效而没有指出 ID,适配器会删除该次 chat 使用的全部文件映射。随后重新上传受影响的请求版本,并重试一次 chat。第二次 chat 仍报告文件失效时,适配器会按该响应清理映射并返回错误,不会发起第三次 chat。上传响应若没有完整文件对象、匹配的字节数和 `expires_at`,就不会写入索引;后续请求会再次上传,而不是信任不一致的本地状态。本地上传索引格式损坏时按空缓存处理,并由下一次成功上传替换;权限和文件系统 I/O 错误仍使请求失败。
同一作用域和 `variantId` 的并发解析共享一次 Files 上传,每个等待方可以单独取消。一次上传配额错误会先分页收集配置数量的最旧 `dsh-` 文件,再删除这些文件并重试一次上传。`DeepSeekFilesClient.delete``DeepSeekFileStore.release``releaseAll` 提供主动远端空间回收。本包记录的当前提供方限制为 Files 单次上传 128MiB、chat 单图引用 32MiB、每个 API key 最多 10,000 个文件和 25GiB;默认 1MiB 请求版本低于两个单文件上限。
@@ -104,7 +104,7 @@ DeepSeek 请求身份独立于应用归因。凭据解析成功后,每个提
#### 模型看到的内容
所选 DeepSeek 模型会收到 harness 系统提示词、消息历史、工具 schema、stop sequence 和调用配置。视觉模型会通过 Files API 引用收到保留的 user 与工具结果图片,旁边带有稳定附件句柄和预览尺寸;超出上限的较旧图片由已记录的占位文本表示。之前 assistant 轮次的推理内容会原文回传,无论该轮次是否调用了工具。
所选 DeepSeek 模型会收到 harness 系统提示词、消息历史、工具 schema、stop sequence 和调用配置。视觉模型会通过 Files API 引用收到保留的 user 与工具结果图片,旁边带有稳定附件句柄和请求图片尺寸;超出上限的较旧图片由已记录的占位文本表示。之前 assistant 轮次的推理内容会原文回传,无论该轮次是否调用了工具。
#### Token 影响
-1
View File
@@ -543,7 +543,6 @@ export class DeepSeekAdapter extends LlmAdapter {
maxImagesPerRequest: connection.maxImagesPerRequest,
byteQuantum: connection.imageOffloadByteQuantum,
countQuantum: connection.imageOffloadCountQuantum,
cropAvailable: options.tools?.some(tool => tool.name === 'read_image_region') ?? false,
}, connection.defaults)
const payload = JSON.stringify(body)
+3 -6
View File
@@ -6,7 +6,7 @@
* @module dsh-llm-deepseek/serialize
*/
import { contentHasImage, LlmError, offloadRequestImagesWithPolicy, requestImagePreviewText } from '@deepseek-ai/dsh-llm'
import { contentHasImage, LlmError, offloadRequestImagesWithPolicy, requestImageHandleText } from '@deepseek-ai/dsh-llm'
import type { ContentBlock, GenerateOptions, Message } from '@deepseek-ai/dsh-llm'
import type { ImageAttachmentRef, RequestImageAttachment } from '@deepseek-ai/dsh-attachment'
import type {
@@ -47,8 +47,6 @@ export interface ImageSerializationOptions {
byteQuantum?: number
/** Image-count removal step applied after the request exceeds its count bound. */
countQuantum?: number
/** Whether the active request exposes the region-read tool. */
cropAvailable?: boolean
}
/** Durable message and image ordinal used in provider diagnostics. */
@@ -120,11 +118,10 @@ function assertSupportedImageRoles(messages: readonly Message[]): void {
function imageHandle(
version: RequestImageAttachment,
precededByContent: boolean,
cropAvailable: boolean,
): WireTextContentPart {
return {
type: 'text',
text: `${precededByContent ? '\n' : ''}${requestImagePreviewText(version, cropAvailable)}`,
text: `${precededByContent ? '\n' : ''}${requestImageHandleText(version)}`,
}
}
@@ -143,7 +140,7 @@ async function imageParts(
)
}
return [
imageHandle(version, precededByContent, images.cropAvailable === true),
imageHandle(version, precededByContent),
{ type: 'file', file_id: await images.resolveFileId(version, block, location) },
]
}
@@ -15,7 +15,7 @@ export interface DeepSeekUploadRecord {
scope: DeepSeekFileScopeType
/** Provider-independent master attachment from which the uploaded request version was derived. */
masterAttachmentId: AttachmentId
/** Complete request transformation identity, including crop and encoder parameters. */
/** Complete request transformation identity, including route budgets and encoder parameters. */
variantId: ImageVariantIdType
fileId: DeepSeekFileIdType
bytes: number
@@ -176,7 +176,6 @@ describe('DeepSeekAdapter against a mock server', () => {
await drain(adapter.stream({
provider: 'deepseek-official',
model: 'deepseek-v4-flash-vision-exp',
tools: [{ name: 'read_image_region', description: 'crop', parameters: { type: 'object' } }],
messages: [createUserMessage({
content: [
{ type: 'text', text: 'describe ' },
@@ -192,7 +191,7 @@ describe('DeepSeekAdapter against a mock server', () => {
role: 'user',
content: [
{ type: 'text', text: 'describe ' },
{ type: 'text', text: expect.stringContaining('Call read_image_region') as string },
{ type: 'text', text: expect.stringContaining(`Image ${imageRef.attachmentId}; request image 1x1px.`) as string },
{ type: 'file', file_id: 'file-api-1' },
],
}],
@@ -60,7 +60,6 @@ function imageOptions(
resolveFileId,
requestImages: new Map(refs.map(ref => [ref.attachmentId, requestVersion(ref)])),
maxRequestFilesBytes,
cropAvailable: true,
}
}
@@ -353,14 +352,14 @@ describe('image serialization', () => {
role: 'user',
content: [
{ type: 'text', text: 'before' },
{ type: 'text', text: expect.stringContaining(`Image ${ref.attachmentId}; preview 1x1px`) as string },
{ type: 'text', text: expect.stringContaining(`Image ${ref.attachmentId}; request image 1x1px`) as string },
{ type: 'file', file_id: 'file-api-image' },
{ type: 'text', text: 'after' },
],
}])
})
it('gives image-only input a stable handle and preview coordinate system', async () => {
it('gives image-only input a stable handle and request dimensions', async () => {
const ref = imageRef()
const wire = await serializeRequestWithImages(request({
model: 'deepseek-v4-flash-vision-exp',
@@ -373,32 +372,12 @@ describe('image serialization', () => {
expect(wire.messages).toEqual([{
role: 'user',
content: [
{ type: 'text', text: expect.stringContaining('Call read_image_region') as string },
{ type: 'text', text: `Image ${ref.attachmentId}; request image 1x1px.` },
{ type: 'file', file_id: 'file-api-image' },
],
}])
})
it('does not advertise region reads when the request omits that tool', async () => {
const ref = imageRef()
const images = { ...imageOptions([ref]), cropAvailable: false }
const wire = await serializeRequestWithImages(request({
model: 'deepseek-v4-flash-vision-exp',
messages: [createUserMessage({
content: [{ type: 'image', attachment: ref }],
source: { kind: 'plugin', plugin: 'test' },
})],
}), images)
expect(wire.messages[0]).toMatchObject({
role: 'user',
content: [
{ type: 'text', text: `Image ${ref.attachmentId}; preview 1x1px.` },
{ type: 'file', file_id: 'file-api-image' },
],
})
})
it('rejects an image whose prepared request version is absent', async () => {
const ref = imageRef()
await expect(serializeMessagesWithImages([createUserMessage({
@@ -528,14 +507,14 @@ describe('image serialization', () => {
{
role: 'tool',
tool_call_id: 'before-system',
content: expect.stringContaining('Call read_image_region') as string,
content: expect.stringContaining('request image 1x1px') as string,
},
expect.objectContaining({ role: 'user' }),
{ role: 'system', content: 'system history' },
{
role: 'tool',
tool_call_id: 'before-assistant',
content: expect.stringContaining('Call read_image_region') as string,
content: expect.stringContaining('request image 1x1px') as string,
},
expect.objectContaining({ role: 'user' }),
{ role: 'assistant', content: 'assistant history' },
+2 -2
View File
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/llm/llm-pi-ai/README.md
README.md: 044038aa69535ad90c9dc59ad63f05ab68560d28
README.zh.md: d4b5dff10ea0f3668038cc4d3a6876f52ae273cb
README.md: 8f4d1537d8ccec3e89c0553f877541d11b285f66
README.zh.md: 354851018de0ea79b82215c3d970266cd2be5763
+2 -2
View File
@@ -123,7 +123,7 @@ A model that carries reasoning metadata — from the installed catalog or from i
A model **without** that metadata — a hand-declared one whose entry declares no `reasoningEfforts`, and a catalog model pi-ai marks as non-reasoning — exposes no `reasoning` at all. pi-ai reports such a model as supporting the single level `off`, but `off` is translated to *omitting* the reasoning option, which is byte-for-byte the request that naming no effort already produces: selecting it could not disable anything, so a provider whose own default is to think would keep thinking with `off` shown as selected. Reporting the capability as unavailable leaves a surface offering the provider's default and nothing that misrepresents it. The profile `reasoning` value, including `off`, is the deployment default when configured; omitting it preserves the provider default. Per-request `GenerateOptions.reasoningEffort` takes precedence, and a level absent from the exact model capability fails the REQUEST with `UNSUPPORTED_REASONING_EFFORT` before network I/O instead of being clamped. Describing a model never fails that way: the models under one provider disagree about which levels they accept, so `resolveModel` reports a profile level the exact model cannot take as no default at all rather than throwing. A throw there would take the whole provider out of every model catalog built over it — one mis-set profile field hiding even the models that do support the level — so a bad configuration surfaces where it is acted on, not where it is described. pi-ai's common stream options represent `off` by omitting `reasoning`.
Supported profile fields are `apiKeyEnv`, `displayName`, `api`, `baseURL`, `models`, `modelOverrides`, `compat`, `defaultContextWindow`, `defaultMaxTokens`, `defaultInput`, `headers`, `reasoning`, `thinkingBudgets`, `cacheRetention`, `transport`, `timeoutMs`, `websocketConnectTimeoutMs`, `streamIdleTimeoutMs`, `maxRequestImageBytes`, `requestImagePixelBudget`, `requestImageMaxBytes`, and `retryPolicy`. Each resolved profile retry policy is captured with that provider route; omission uses the shared bounded normal default of five retries. The stream-idle interval is a positive finite Node timer delay, defaults to five minutes, and covers only an outstanding provider read, not consumer think time. Every image route derives a deterministic request version from the provider-independent master under `requestImagePixelBudget` (default 2048 by 2048 total pixels) and `requestImageMaxBytes` (default 1MiB raw bytes). Before reading masters, `maxRequestImageBytes` applies to conservative request-version upper bounds and replaces the oldest over-budget images with fixed text; exact base64 lengths are checked again after retained versions are generated. The 20MiB default can retain fifteen maximum-size 1MiB versions after base64 expansion while leaving request-body headroom. The same version feeds inline base64, and its stable descriptor exposes the attachment id and actual preview dimensions. Harness app attribution wins a conflicting configured header name.
Supported profile fields are `apiKeyEnv`, `displayName`, `api`, `baseURL`, `models`, `modelOverrides`, `compat`, `defaultContextWindow`, `defaultMaxTokens`, `defaultInput`, `headers`, `reasoning`, `thinkingBudgets`, `cacheRetention`, `transport`, `timeoutMs`, `websocketConnectTimeoutMs`, `streamIdleTimeoutMs`, `maxRequestImageBytes`, `requestImagePixelBudget`, `requestImageMaxBytes`, and `retryPolicy`. Each resolved profile retry policy is captured with that provider route; omission uses the shared bounded normal default of five retries. The stream-idle interval is a positive finite Node timer delay, defaults to five minutes, and covers only an outstanding provider read, not consumer think time. Every image route derives a deterministic request version from the provider-independent master under `requestImagePixelBudget` (default 2048 by 2048 total pixels) and `requestImageMaxBytes` (default 1MiB raw bytes). Before reading masters, `maxRequestImageBytes` applies to conservative request-version upper bounds and replaces the oldest over-budget images with fixed text; exact base64 lengths are checked again after retained versions are generated. The 20MiB default can retain fifteen maximum-size 1MiB versions after base64 expansion while leaving request-body headroom. The same version feeds inline base64, and its stable descriptor exposes the attachment id and actual request-image dimensions. Harness app attribution wins a conflicting configured header name.
The adapter forces pi-ai's SDK `maxRetries` to zero so one `stream()` call makes one provider request. The removed profile fields `maxRetries` and `maxRetryDelayMs` fail load instead of silently multiplying or hiding the separately composed agent-level retry budget. Idle expiry aborts the SDK's stable request signal and surfaces `TIMEOUT`; an earlier caller abort remains `ABORTED`.
@@ -173,7 +173,7 @@ pi-ai installs several provider SDKs and lazy-loads the one selected by the cata
#### What the model sees
The selected catalog model receives `GenerateOptions.system`, history, tools, and sampling fields supported by pi-ai's common streaming API. Each retained image is preceded by stable text naming its complete attachment id and actual request dimensions. The text includes `read_image_region` preview coordinates only when that tool is present in the request. When accumulated base64 image payload exceeds the route's `maxRequestImageBytes`, each offloaded image (oldest first) is replaced by fixed text that tells the model to read the file again when a path is available or ask the user to attach it again. Offloaded masters are not read or transformed. Provider-native replay metadata is restored only when the adapter validates it for the historical content.
The selected catalog model receives `GenerateOptions.system`, history, tools, and sampling fields supported by pi-ai's common streaming API. Each retained image is preceded by stable text naming its complete attachment id and actual request dimensions. When accumulated base64 image payload exceeds the route's `maxRequestImageBytes`, each offloaded image (oldest first) is replaced by fixed text that tells the model to read the file again when a path is available or ask the user to attach it again. Offloaded masters are not read or transformed. Provider-native replay metadata is restored only when the adapter validates it for the historical content.
#### Token effect
+2 -2
View File
@@ -124,7 +124,7 @@ pi-ai 依据提供方 id 与 baseURL 决定每个请求的形状:系统提示
**没有**这份元数据的模型——条目未声明 `reasoningEfforts` 的手工声明模型,以及 pi-ai 标记为不具备推理能力的 catalog 模型——完全不公开 `reasoning`。pi-ai 会把这类模型报告为只支持 `off` 一档,但 `off` 会被翻译成*省略* reasoning 选项,而那与「不点名任何档位」产出的请求逐字节相同:选它关不掉任何东西,于是自身默认就在思考的提供方,会在界面显示 `off` 被选中的同时继续思考。把该能力报告为不可用,界面就只剩提供方默认这一项,不会再出现自相矛盾的控件。配置 profile 的 `reasoning` 值(包括 `off`)在存在时是部署默认值;省略它会保留提供方默认值。每次请求的 `GenerateOptions.reasoningEffort` 优先;未出现在确切模型能力中的档位会让**请求**在网络 I/O 前以 `UNSUPPORTED_REASONING_EFFORT` 失败,而不会被自动调整。**描述**一个模型则从不这样失败:同一提供方下各模型接受的档位并不一致,因此 `resolveModel` 对该模型拿不下的 profile 档位报告为「没有默认值」,而不是抛错。在那里抛错会让整个提供方从任何基于它构建的模型目录中消失——一个配错的 profile 字段连支持该档位的模型也一并藏起来——所以坏配置暴露在被执行处,而不是被描述处。pi-ai 的通用流选项通过省略 `reasoning` 表示 `off`
受支持的 profile 字段是 `apiKeyEnv``displayName``api``baseURL``models``modelOverrides``compat``defaultContextWindow``defaultMaxTokens``defaultInput``headers``reasoning``thinkingBudgets``cacheRetention``transport``timeoutMs``websocketConnectTimeoutMs``streamIdleTimeoutMs``maxRequestImageBytes``requestImagePixelBudget``requestImageMaxBytes``retryPolicy`。每条 profile 解析后的重试策略会随该提供方路由一同捕获;省略时使用共享的有界 normal 默认值并重试五次。流空闲间隔必须是正的有限 Node 定时器延迟,默认为五分钟,且只覆盖未完成提供方读取,不包括消费方思考时间。每条图片路由从提供方无关的主版本派生确定性请求版本,受 `requestImagePixelBudget`(默认总像素 2048×2048)和 `requestImageMaxBytes`(默认原始字节 1MiB)约束。读取主版本前,`maxRequestImageBytes` 先按请求版本的保守上界替换超预算的最旧图片;保留版本生成后再用确切 base64 长度检查。20MiB 默认值可保留十五个按 1MiB 上限生成的请求版本,并为请求正文留下余量。同一版本用于内联 base64,其稳定描述会公开附件 ID 和实际预览尺寸。若已配置标头中有同名项,则以 Harness 应用归因为准。
受支持的 profile 字段是 `apiKeyEnv``displayName``api``baseURL``models``modelOverrides``compat``defaultContextWindow``defaultMaxTokens``defaultInput``headers``reasoning``thinkingBudgets``cacheRetention``transport``timeoutMs``websocketConnectTimeoutMs``streamIdleTimeoutMs``maxRequestImageBytes``requestImagePixelBudget``requestImageMaxBytes``retryPolicy`。每条 profile 解析后的重试策略会随该提供方路由一同捕获;省略时使用共享的有界 normal 默认值并重试五次。流空闲间隔必须是正的有限 Node 定时器延迟,默认为五分钟,且只覆盖未完成提供方读取,不包括消费方思考时间。每条图片路由从提供方无关的主版本派生确定性请求版本,受 `requestImagePixelBudget`(默认总像素 2048×2048)和 `requestImageMaxBytes`(默认原始字节 1MiB)约束。读取主版本前,`maxRequestImageBytes` 先按请求版本的保守上界替换超预算的最旧图片;保留版本生成后再用确切 base64 长度检查。20MiB 默认值可保留十五个按 1MiB 上限生成的请求版本,并为请求正文留下余量。同一版本用于内联 base64,其稳定描述会公开附件 ID 和实际请求图片尺寸。若已配置标头中有同名项,则以 Harness 应用归因为准。
适配器强制 pi-ai SDK `maxRetries` 为零,因此一次 `stream()` 调用只会发起一次提供方请求。已移除 profile 字段 `maxRetries``maxRetryDelayMs` 会使加载失败,而不是静默倍增或隐藏单独组合的 agent(智能体)级重试预算。空闲超时会 abort SDK 的稳定请求信号,并以 `TIMEOUT` 呈现;较早的调用方 abort 仍为 `ABORTED`
@@ -174,7 +174,7 @@ pi-ai 会安装多个提供方 SDK,并延迟加载 catalog 模型所选的 SDK
#### 模型看到的内容
所选 catalog 模型会收到 `GenerateOptions.system`、历史、工具,以及 pi-ai 通用流式 API 支持的采样字段。每张保留图片前都有稳定文本,写明完整附件 ID 和实际请求尺寸。只有请求包含 `read_image_region` 时,文本才会提供该工具使用的预览坐标。请求累积的 base64 图片载荷超过路由的 `maxRequestImageBytes` 时,被 offload 的图片会从最老开始替换为固定文本,要求模型在有路径时重新读取文件,否则请用户重新附上图片。系统不会读取或转换被 offload 的主版本。只有当适配器验证提供方原生回放元数据与历史内容匹配时,才会恢复这些元数据。
所选 catalog 模型会收到 `GenerateOptions.system`、历史、工具,以及 pi-ai 通用流式 API 支持的采样字段。每张保留图片前都有稳定文本,写明完整附件 ID 和实际请求尺寸。请求累积的 base64 图片载荷超过路由的 `maxRequestImageBytes` 时,被 offload 的图片会从最老开始替换为固定文本,要求模型在有路径时重新读取文件,否则请用户重新附上图片。系统不会读取或转换被 offload 的主版本。只有当适配器验证提供方原生回放元数据与历史内容匹配时,才会恢复这些元数据。
#### Token 影响
+5 -7
View File
@@ -4,7 +4,7 @@
* @module dsh-llm-pi-ai/context
*/
import { CallId, contentHasImage, LlmError, offloadRequestImagesWithPolicy, requestImagePreviewText } from '@deepseek-ai/dsh-llm'
import { CallId, contentHasImage, LlmError, offloadRequestImagesWithPolicy, requestImageHandleText } from '@deepseek-ai/dsh-llm'
import type { ContentBlock, GenerateOptions, Message } from '@deepseek-ai/dsh-llm'
import type {
AttachmentId,
@@ -48,7 +48,6 @@ function assertSupportedImageRoles(messages: readonly Message[]): void {
async function userContent(
blocks: readonly ContentBlock[],
requestImages: ReadonlyMap<AttachmentId, RequestImageAttachment>,
cropAvailable: boolean,
): Promise<string | (TextContent | ImageContent)[]> {
const content: (TextContent | ImageContent)[] = []
for (const block of blocks) {
@@ -61,7 +60,7 @@ async function userContent(
if (version === undefined) {
throw new LlmError(`pi-ai request image ${block.attachment.attachmentId} was not prepared`, 'INVALID_REQUEST')
}
content.push({ type: 'text', text: requestImagePreviewText(version, cropAvailable) })
content.push({ type: 'text', text: requestImageHandleText(version) })
content.push({
type: 'image',
data: Buffer.from(version.data).toString('base64'),
@@ -71,7 +70,7 @@ async function userContent(
}
case 'tool-result':
{
const nested = await userContent(block.content, requestImages, cropAvailable)
const nested = await userContent(block.content, requestImages)
if (typeof nested === 'string') {
if (nested.length > 0) content.push({ type: 'text', text: nested })
} else {
@@ -241,7 +240,6 @@ async function toPiContextWithImages(
byteQuantum: 1,
byteLength: ref => requestImages.get(ref.attachmentId)?.bytes ?? ref.bytes,
})
const cropAvailable = options.tools?.some(tool => tool.name === 'read_image_region') ?? false
const toolNames = new Map<CallId, string>()
const messages: PiMessage[] = []
@@ -263,7 +261,7 @@ async function toPiContextWithImages(
}
// user role: text + tool results (each result becomes its own message).
const regular = message.content.filter(block => block.type !== 'tool-result')
const content = await userContent(regular, requestImages, cropAvailable)
const content = await userContent(regular, requestImages)
const results = message.content.filter((block): block is Extract<ContentBlock, { type: 'tool-result' }> => (
block.type === 'tool-result'
))
@@ -271,7 +269,7 @@ async function toPiContextWithImages(
messages.push({ role: 'user', content, timestamp: 0 })
}
for (const result of results) {
const resultContent = await userContent(result.content, requestImages, cropAvailable)
const resultContent = await userContent(result.content, requestImages)
messages.push({
role: 'toolResult',
toolCallId: result.toolCallId,
@@ -309,17 +309,6 @@ describe('pi-ai request context conversion', () => {
expect(readImageRequest.mock.calls[0]?.[0]).toEqual(recent)
})
it('advertises region reads only when the request exposes the tool', async () => {
const withoutCrop = await toPiContext(request([user([{ type: 'image', attachment: ref }])]), attachments)
const withCrop = await toPiContext({
...request([user([{ type: 'image', attachment: ref }])]),
tools: [{ name: 'read_image_region', description: 'crop', parameters: { type: 'object' } }],
}, attachments)
expect(JSON.stringify(withoutCrop.messages)).not.toContain('Call read_image_region')
expect(JSON.stringify(withCrop.messages)).toContain('Call read_image_region')
})
it('keeps every image at exactly the payload bound and drops all of them when even the newest cannot fit', async () => {
const sized: ImageAttachmentRef = { ...ref, bytes: 3 }
const exact = await toPiContext(request([
+4 -9
View File
@@ -19,17 +19,12 @@ export function textOnlyImageText(ref: ImageAttachmentRef): string {
}
/**
* Stable model-facing handle and coordinate description for one exact request preview.
* Stable model-facing handle for one exact request image.
* @param version - exact request image shown beside the text.
* @param cropAvailable - whether the active request exposes `read_image_region`.
* @returns attachment handle, preview dimensions, and crop-coordinate guidance.
* @returns attachment handle and request-image dimensions.
*/
export function requestImagePreviewText(version: RequestImageAttachment, cropAvailable: boolean): string {
const identity = `Image ${version.master.attachmentId}; preview ${version.width}x${version.height}px.`
return cropAvailable
? `${identity} Crop coordinates use this preview. Call read_image_region with this attachment_id, `
+ `preview_width=${version.width}, preview_height=${version.height}, x, y, width, and height.`
: identity
export function requestImageHandleText(version: RequestImageAttachment): string {
return `Image ${version.master.attachmentId}; request image ${version.width}x${version.height}px.`
}
/**
-1
View File
@@ -294,7 +294,6 @@ export const LINK_MAP: Readonly<Record<string, string>> = {
EncodedImageAttachment: 'attachment.md',
ImageAttachmentRef: 'attachment.md',
ImageRequestPolicy: 'attachment.md',
PreviewImageCrop: 'attachment.md',
RequestImageAttachment: 'attachment.md',
SaveImageAttachment: 'attachment.md',
SavedImageAttachment: 'attachment.md',
+3 -3
View File
@@ -315,17 +315,17 @@ const TOOL_PACKAGES: ToolPackage[] = [
dir: 'tool-fs',
source: 'packages/fs/tool-fs/src/index.ts',
requires: ['ctx.tools', 'ctx.fs', 'ctx.systemPrompt', 'ctx.attachments (image-tool registration)', 'ctx.llm + an image-capable route (image-tool execution)'],
writes: ['tool/call', 'fs/write-intent or fs/edit-intent for mutations', 'fs/observed after read presence/absence or successful file operation', 'durable attachment (read_image and read_image_region)', 'tool/result'],
writes: ['tool/call', 'fs/write-intent or fs/edit-intent for mutations', 'fs/observed after read presence/absence or successful file operation', 'durable attachment (read_image)', 'tool/result'],
async mount(ctx) {
// The tool needs `fs`; the bare provider is sufficient because policy
// changes behavior, not schema shape. The catalog seam marker opts into
// both attachments-conditional image schemas without attachment I/O.
// the attachments-conditional image schema without attachment I/O.
await ctx.plugin(LocalFileSystem)
await ctx.plugin(CatalogAttachmentStore)
await ctx.plugin(ToolFs)
},
note:
'The read-before-write/edit policy is added by `@deepseek-ai/dsh-fs-observation-policy` (an `fs/*` event-gate plugin, no schema change); a deployment that loads these tools is expected to also load it. The image tools are not registered without `ctx.attachments`; their schemas are route-independent, and execution refuses unless the exact routed model declares image input.',
'The read-before-write/edit policy is added by `@deepseek-ai/dsh-fs-observation-policy` (an `fs/*` event-gate plugin, no schema change); a deployment that loads these tools is expected to also load it. The image tool is not registered without `ctx.attachments`; its schema is route-independent, and execution refuses unless the exact routed model declares image input.',
},
{
pkg: '@deepseek-ai/dsh-tool-fs-search',
-10
View File
@@ -930,21 +930,11 @@
"symbol": "StoredImageAttachment",
"source": "packages/attachment/attachment/src/types.ts"
},
{
"doc": "docs/subsystems/attachment.md",
"symbol": "MasterImageCrop",
"source": "packages/attachment/attachment/src/types.ts"
},
{
"doc": "docs/subsystems/attachment.md",
"symbol": "ImageRequestPolicy",
"source": "packages/attachment/attachment/src/types.ts"
},
{
"doc": "docs/subsystems/attachment.md",
"symbol": "PreviewImageCrop",
"source": "packages/attachment/attachment/src/types.ts"
},
{
"doc": "docs/subsystems/attachment.md",
"symbol": "RequestImageAttachment",