mirror of
https://github.com/deepseek-ai/deepseek-harness.git
synced 2026-08-29 04:26:38 +00:00
fix(images): address unified pipeline review
This commit is contained in:
+2
-2
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-08-20-unified-image-request-pipeline.md
|
||||
2026-08-20-unified-image-request-pipeline.md: 72382b6130086ba5c36d386ffe7ebe413cd2243d
|
||||
2026-08-20-unified-image-request-pipeline.zh.md: 15560ad475af669cc4a2d9c46354a4da08528e0b
|
||||
2026-08-20-unified-image-request-pipeline.md: 07632e9e0c3aac33d89acd8aebc0f0114550ddb6
|
||||
2026-08-20-unified-image-request-pipeline.zh.md: 9d95346dab2a7747c4bcef9f213ec0fa8e5ba067
|
||||
|
||||
@@ -24,19 +24,19 @@ Batch admission prepares and verifies every master once before publishing any me
|
||||
|
||||
`AttachmentStore.readImageRequest` derives a request version under route-owned total-pixel and encoded-byte budgets. Scaling is `min(1, sqrt(maxPixels / (width * height)))`, with no enlargement, followed by inward integer rounding so the encoded raster never exceeds the total-pixel cap. DeepSeek V4 Flash Vision Exp uses 640,000 total pixels and 1MiB raw encoded bytes by default; low detail uses 512 by 512 total pixels. A 2048 by 1024 master projects to 1130 by 565 under the hard cap. Request encoding uses the same color branches, with PNG (palette only without alpha) then WebP 85 and 80 for low-color input, WebP 85 then 80 for other alpha input, and JPEG 85 then 80 for other opaque input. Each fallback runs only after the previous result exceeds 1MiB, and dimensions shrink only after both quality attempts exceed it. The same derivation is used by normal agent turns, direct `ctx.llm.stream` calls, compaction, and other auxiliary streams.
|
||||
|
||||
The `variantId` and cache path cover the master attachment id, transform version, route pixel and byte budgets, optional master-coordinate crop, and fixed encoder parameters. Cached output is fully decoded before reuse. DeepSeek Files and pi-ai inline base64 therefore use the same deterministic bytes for the same policy. Inline accounting uses the derived byte length after base64 expansion, not the master byte count. Equal in-process `variantId` calls share one transform and cache write; cancellation rejects only that waiter. `AttachmentStore.readImageRequests` preserves input order while the local implementation runs master and request transforms through one FIFO limiter. `imageCompressionConcurrency` is configurable from 1 through 8 and defaults to 2. Batch publication remains sequential after every master has been prepared.
|
||||
The `variantId` and cache path cover the master attachment id, transform version, route pixel and byte budgets, optional master-coordinate crop, and fixed encoder parameters. A new cache entry is fully decoded before publication. Cache hits use a header probe to check format, 8-bit sRGB/sRGBA facts, dimensions, alpha, and byte limits without decoding the complete raster again; a mismatch regenerates the entry. DeepSeek Files and pi-ai inline base64 therefore use the same deterministic bytes for the same policy. Inline accounting uses the derived byte length after base64 expansion, not the master byte count. Equal in-process `variantId` calls share one transform and cache write. Each caller can cancel its own wait; the shared transform is aborted only after every waiter has cancelled. `AttachmentStore.readImageRequests` preserves input order while the local implementation runs master and request transforms through one FIFO limiter. `imageCompressionConcurrency` is configurable from 1 through 8 and defaults to 2. Batch publication remains sequential after every master has been prepared.
|
||||
|
||||
Request-size offload is a deterministic oldest-first projection. DeepSeek defaults to 128MiB and 600 referenced images. Its removed prefix advances past successive 64MiB byte boundaries and in 20-image count quanta, so 129 one-megabyte images remove the oldest 65, retain 64MiB, and keep that prefix stable until total history passes 192MiB. Pi-ai retains a configurable base64 request bound. A text-only route receives deterministic attachment placeholders, including nested tool-result images, while append-only session history keeps the original references.
|
||||
Request-size offload is a deterministic oldest-first projection. Before reading attachments, each route uses `min(masterBytes, requestVersionMaxBytes)` as a conservative upper bound and removes the oldest over-budget prefix. Only retained masters are read and transformed, so an omitted missing or corrupt object cannot block the request. A second projection uses exact derived lengths without bringing omitted images back. DeepSeek defaults to 128MiB and 600 referenced images. Its removed prefix advances past successive 64MiB byte boundaries and in 20-image count quanta, so 129 one-megabyte images remove the oldest 65, retain 64MiB, and keep that prefix stable until total history passes 192MiB. Pi-ai retains a configurable base64 request bound. A text-only route receives deterministic attachment placeholders, including nested tool-result images, while append-only session history keeps the original references.
|
||||
|
||||
### Stable handles and master-coordinate crops
|
||||
|
||||
Every retained request image is preceded by its complete attachment id, actual request dimensions, and the preview-coordinate arguments for `read_image_region`. The tool accepts only an attachment already referenced by the calling session. It maps the supplied preview rectangle to the 2048px master with floor-at-origin and ceil-at-far-edge rounding, crops the master rather than the preview, and persists the result as a new attachment. The tool result contains the new `ImageBlock`, so model-visible output and the durable log remain equivalent.
|
||||
Every retained request image is preceded by its complete attachment id and actual request dimensions. When the active request exposes `read_image_region`, the text also supplies its preview-coordinate arguments. The tool accepts only an attachment already referenced by the calling session. It maps the supplied preview rectangle to the 2048px master with floor-at-origin and ceil-at-far-edge rounding, crops the master rather than the preview, and persists the result as a new attachment. The tool result contains the new `ImageBlock`, so model-visible output and the durable log remain equivalent.
|
||||
|
||||
### DeepSeek Files lifecycle
|
||||
|
||||
The direct `deepseek-official` adapter uploads every retained request version through the OpenAI-compatible Files API and sends only `file_id` content blocks. There is no inline fallback. The default catalog advertises `deepseek-v4-flash-vision-exp` as image-capable. Uploaded ids are indexed by endpoint and API-key scope plus `variantId`. Uploads request seven days by default and record the returned `expires_at`; a mapping with no more than one hour remaining is replaced without a preceding retrieve call. The index never stores the API key.
|
||||
|
||||
An upload is indexed only after the response returns a complete file object, matching byte count, and `expires_at`. A missing or inconsistent response leaves no local mapping, so a later request uploads again. A malformed upload index is an empty cache and is replaced on the next successful upload; filesystem I/O failures remain errors. If chat reports expired, deleted, missing, or invalid ids and names one or more ids used by the request, only those mappings are removed. A stale-file response without a specific id removes every mapping used by that chat attempt. The affected request bytes are uploaded again and chat is retried once. A second stale rejection clears the mappings identified by its response and returns the error without a third chat attempt. One upload quota error deletes the configured number of oldest harness-owned `dsh-` files and retries once. Public file operations expose list, retrieve, delete, one-variant release, and namespace-wide release. The client enforces the documented 128MiB upload limit, 32MiB chat-image limit, 10,000-file and 25GiB quotas, and one-hour to 30-day expiry range.
|
||||
An upload is indexed only after the response returns a complete file object, matching byte count, and `expires_at`. A missing or inconsistent response leaves no local mapping, so a later request uploads again. Concurrent upload resolution for one scoped `variantId` shares one provider operation; one waiter cannot cancel another, and the upload stops when every waiter has cancelled. A malformed upload index is an empty cache and is replaced on the next successful upload; filesystem I/O failures remain errors. If chat reports expired, deleted, missing, or invalid ids and names one or more ids used by the request, only those mappings are removed. A stale-file response without a specific id removes every mapping used by that chat attempt. The affected request bytes are uploaded again and chat is retried once. A second stale rejection clears the mappings identified by its response and returns the error without a third chat attempt. One upload quota error first lists the configured number of oldest harness-owned `dsh-` files, then deletes that collected set and retries once; deleting after pagination keeps provider cursors valid. Public file operations expose list, retrieve, delete, one-variant release, and namespace-wide release. Every Files request carries the shared Harness `User-Agent`. The client enforces the documented 128MiB upload limit, 32MiB chat-image limit, 10,000-file and 25GiB quotas, and one-hour to 30-day expiry range.
|
||||
|
||||
### Diagnostics
|
||||
|
||||
@@ -64,7 +64,7 @@ Historical attachment objects that later disappear or fail integrity verificatio
|
||||
|
||||
## Verification
|
||||
|
||||
Package tests generate 16-bit RGB and RGBA PNG fixtures, prove 8-bit conversion and clean 8-bit passthrough, retain alpha under byte pressure, distinguish high-frequency and ordinary photos from low-color graphics, stop lazy encoding after the first fitting candidate, cover square and wide 640,000-pixel projections, enforce 1MiB request bytes, singleflight equal variants, bound transform concurrency, preserve cache and upload identity, map preview crops to the master, prepare batches once, reject inconsistent Files responses, refresh near-expiry ids without retrieve, recover once from single-id, multiple-id, and ambiguous stale responses, delete quota files, normalize provider diagnostics, project text-only history, and share normal/compaction request bytes. Keyless assembled snapshots cover the real tool schemas and image request path. A credentialed test uses the built-in `deepseek-official` route and its configured endpoint, never a custom provider entry.
|
||||
Package tests generate 16-bit RGB and RGBA PNG fixtures, prove 8-bit conversion and clean 8-bit passthrough, retain alpha under byte pressure, distinguish high-frequency and ordinary photos from low-color graphics, stop lazy encoding after the first fitting candidate, cover square and wide 640,000-pixel projections, enforce 1MiB request bytes, singleflight equal variants and uploads without shared-cancellation leaks, bound transform concurrency, preserve cache and upload identity, skip attachment reads for conservatively offloaded history, map preview crops to the master, prepare batches once, reject inconsistent Files responses, refresh near-expiry ids without retrieve, recover once from single-id, multiple-id, and ambiguous stale responses, paginate before quota deletion, normalize provider diagnostics, project text-only history, and share normal/compaction request bytes. Keyless assembled snapshots cover the real tool schemas and image request path. A credentialed test uses the built-in `deepseek-official` route and its configured endpoint, never a custom provider entry.
|
||||
|
||||
## Consequences
|
||||
|
||||
|
||||
@@ -24,19 +24,19 @@ Status: implemented
|
||||
|
||||
`AttachmentStore.readImageRequest` 按路由拥有的总像素和编码字节预算派生请求版本。缩放公式为 `min(1, sqrt(maxPixels / (width * height)))`,不会放大小图,随后向预算内取整,确保编码光栅不超过总像素上限。DeepSeek V4 Flash Vision Exp 默认使用总像素 640,000 和原始编码字节 1MiB;low detail 使用总像素 512×512。2048×1024 主版本在这个硬上限下会投影为 1130×565。请求编码使用相同的分类分支:低色数输入先尝试 PNG,只有不带 alpha 通道时才使用 palette,随后依次尝试质量 85、80 的 WebP;其他透明输入依次尝试质量 85、80 的 WebP;其他非透明输入依次尝试质量 85、80 的 JPEG。只有前一结果超过 1MiB 时才执行下一个候选;两个质量档都超限后才缩小尺寸。普通 agent 轮次、直接 `ctx.llm.stream` 调用、压缩和其他辅助流都使用同一派生过程。
|
||||
|
||||
`variantId` 和缓存路径覆盖主附件 ID、变换策略版本、路由像素和字节预算、可选的主版本坐标裁剪区域及固定编码参数。缓存输出会在复用前完整解码。因此,同一策略下的 DeepSeek Files 和 pi-ai 内联 base64 使用相同的确定性字节。内联计量使用派生字节经过 base64 膨胀后的长度,不使用主版本字节数。同一进程内相同 `variantId` 的调用共享一次变换和缓存写入;取消只拒绝对应等待方。`AttachmentStore.readImageRequests` 保持输入顺序,本地实现则通过一个 FIFO 限流器运行主版本和请求版本变换。`imageCompressionConcurrency` 的可配置范围为 1 至 8,默认值为 2。全部主版本准备完成后,批次仍按顺序发布。
|
||||
`variantId` 和缓存路径覆盖主附件 ID、变换策略版本、路由像素和字节预算、可选的主版本坐标裁剪区域及固定编码参数。新缓存条目在发布前会完整解码。缓存命中只探测文件头,校验格式、8-bit sRGB/sRGBA、尺寸、透明通道和字节上限,不会再次完整解码光栅;不匹配时会重新生成。因此,同一策略下的 DeepSeek Files 和 pi-ai 内联 base64 使用相同的确定性字节。内联计量使用派生字节经过 base64 膨胀后的长度,不使用主版本字节数。同一进程内相同 `variantId` 的调用共享一次变换和缓存写入。每个调用方可以取消自己的等待;只有全部等待方都取消时,共享变换才会中止。`AttachmentStore.readImageRequests` 保持输入顺序,本地实现则通过一个 FIFO 限流器运行主版本和请求版本变换。`imageCompressionConcurrency` 的可配置范围为 1 至 8,默认值为 2。全部主版本准备完成后,批次仍按顺序发布。
|
||||
|
||||
请求大小 offload 是确定性的从旧到新投影。DeepSeek 默认上限为 128MiB 和 600 张引用图片。被移除前缀会越过连续的 64MiB 字节边界,并按 20 张图片数量步长递增,因此 129 张 1MiB 图片会移除最旧的 65 张并保留 64MiB;持久历史超过 192MiB 前,该前缀保持不变。Pi-ai 保留可配置的 base64 请求上限。纯文本路由会收到确定性的附件占位文本,其中包括嵌套工具结果图片;追加式会话历史继续保留原始引用。
|
||||
请求大小 offload 是确定性的从旧到新投影。读取附件前,每条路由先以 `min(主版本字节数, 请求版本字节上限)` 作为保守上界,移除超出预算的最旧前缀。系统只读取并转换保留的主版本,因此已省略的缺失或损坏对象不会阻塞请求。第二次投影使用确切派生长度,但不会重新加入已省略图片。DeepSeek 默认上限为 128MiB 和 600 张引用图片。被移除前缀会越过连续的 64MiB 字节边界,并按 20 张图片数量步长递增,因此 129 张 1MiB 图片会移除最旧的 65 张并保留 64MiB;持久历史超过 192MiB 前,该前缀保持不变。Pi-ai 保留可配置的 base64 请求上限。纯文本路由会收到确定性的附件占位文本,其中包括嵌套工具结果图片;追加式会话历史继续保留原始引用。
|
||||
|
||||
### 稳定句柄与主版本坐标裁剪
|
||||
|
||||
每张保留请求图片前都有完整附件 ID、实际请求尺寸和 `read_image_region` 所需的预览坐标参数。该工具只接受调用会话已经引用的附件。它按起点向下取整、远端边界向上取整,把提交的预览矩形映射到 2048px 主版本,从主版本而非预览图裁剪,并把结果保存为新附件。工具结果包含新的 `ImageBlock`,因此模型可见输出与持久日志保持一致。
|
||||
每张保留请求图片前都有完整附件 ID 和实际请求尺寸。当前请求公开 `read_image_region` 时,这段文本还会提供预览坐标参数。该工具只接受调用会话已经引用的附件。它按起点向下取整、远端边界向上取整,把提交的预览矩形映射到 2048px 主版本,从主版本而非预览图裁剪,并把结果保存为新附件。工具结果包含新的 `ImageBlock`,因此模型可见输出与持久日志保持一致。
|
||||
|
||||
### DeepSeek Files 生命周期
|
||||
|
||||
直接 `deepseek-official` 适配器通过 OpenAI 兼容 Files API 上传每张保留的请求版本,只发送 `file_id` 内容块,不提供内联回退。默认 catalog 把 `deepseek-v4-flash-vision-exp` 公布为支持图片。上传 ID 按端点和 API key 作用域以及 `variantId` 写入索引。上传默认请求 7 天有效期,并记录返回的 `expires_at`;本地映射剩余时间不超过一小时时会直接替换,不会先查询远端文件。索引绝不存储 API key。
|
||||
|
||||
只有上传响应返回完整文件对象、匹配的字节数和 `expires_at` 时,上传结果才会写入索引。缺失或不一致的响应不会留下本地映射,后续请求会重新上传。格式损坏的上传索引按空缓存处理,并在下一次成功上传时替换;文件系统 I/O 失败仍是错误。如果 chat 报告 ID 已过期、删除、缺失或无效,并指出本次请求使用的一个或多个 ID,适配器只删除这些映射。如果响应只说明文件状态失效而没有指出具体 ID,适配器会删除该次 chat 使用的全部映射。受影响的请求字节会重新上传,chat 只重试一次。第二次仍报告文件失效时,适配器会按响应清理映射并返回错误,不会发起第三次 chat。一次上传配额错误会删除配置数量的最旧 `dsh-` 文件,然后重试一次。公开文件操作提供列表、查询、删除、单个变体释放和整个作用域释放。客户端执行文档规定的 Files 单次上传 128MiB、chat 单图 32MiB、10,000 个文件、25GiB,以及一小时到 30 天有效期限制。
|
||||
只有上传响应返回完整文件对象、匹配的字节数和 `expires_at` 时,上传结果才会写入索引。缺失或不一致的响应不会留下本地映射,后续请求会重新上传。同一作用域和 `variantId` 的并发解析共享一次提供方上传;单个等待方无法取消其他等待方,全部等待方取消时才会停止上传。格式损坏的上传索引按空缓存处理,并在下一次成功上传时替换;文件系统 I/O 失败仍是错误。如果 chat 报告 ID 已过期、删除、缺失或无效,并指出本次请求使用的一个或多个 ID,适配器只删除这些映射。如果响应只说明文件状态失效而没有指出具体 ID,适配器会删除该次 chat 使用的全部映射。受影响的请求字节会重新上传,chat 只重试一次。第二次仍报告文件失效时,适配器会按响应清理映射并返回错误,不会发起第三次 chat。一次上传配额错误会先列出配置数量的最旧 `dsh-` 文件,再删除收集到的文件并重试一次;分页完成后才删除,避免游标失效。公开文件操作提供列表、查询、删除、单个变体释放和整个作用域释放。每个 Files 请求都携带 Harness 的共享 `User-Agent`。客户端执行文档规定的 Files 单次上传 128MiB、chat 单图 32MiB、10,000 个文件、25GiB,以及一小时到 30 天有效期限制。
|
||||
|
||||
### 诊断
|
||||
|
||||
@@ -64,7 +64,7 @@ Status: implemented
|
||||
|
||||
## Verification
|
||||
|
||||
包测试会生成 16-bit RGB 和 RGBA PNG fixture,验证 8-bit 转换与干净 8-bit 字节直通、字节压力下保留透明通道、区分高频和普通照片与低色数图形、首个候选合规后停止编码、正方形和宽屏 640,000 像素投影、请求字节不超过 1MiB、相同变体 singleflight、变换并发上限、缓存与上传身份、预览到主版本坐标映射、批量只准备一次、Files 响应不一致、进入刷新余量时不查询远端并更新 ID、单个 ID、多个 ID 和模糊失效响应只恢复一次、配额删除、规范化提供方诊断、纯文本投影,以及普通请求与压缩共享请求字节。无需密钥的组装快照覆盖真实工具 schema 和图片请求路径。使用凭据的测试只使用内置 `deepseek-official` 路由及其已配置端点,不使用自定义提供方条目。
|
||||
包测试会生成 16-bit RGB 和 RGBA PNG fixture,验证 8-bit 转换与干净 8-bit 字节直通、字节压力下保留透明通道、区分高频和普通照片与低色数图形、首个候选合规后停止编码、正方形和宽屏 640,000 像素投影、请求字节不超过 1MiB、相同变体与上传 singleflight 且不会共享取消、变换并发上限、缓存与上传身份、跳过已保守 offload 的历史附件读取、预览到主版本坐标映射、批量只准备一次、Files 响应不一致、进入刷新余量时不查询远端并更新 ID、单个 ID、多个 ID 和模糊失效响应只恢复一次、删除配额文件前完成分页、规范化提供方诊断、纯文本投影,以及普通请求与压缩共享请求字节。无需密钥的组装快照覆盖真实工具 schema 和图片请求路径。使用凭据的测试只使用内置 `deepseek-official` 路由及其已配置端点,不使用自定义提供方条目。
|
||||
|
||||
## Consequences
|
||||
|
||||
|
||||
@@ -237,7 +237,7 @@ describe('the shipped Web composition', () => {
|
||||
// depend on ripgrep being present on the machine.
|
||||
expect(toolNames(ctx, handle.agent).filter(name => name !== 'glob' && name !== 'grep')).toEqual([
|
||||
'ask_user_question', 'bash', 'create_goal', 'edit', 'exit_plan_mode',
|
||||
'get_goal', 'interrupt_agent', 'job_kill', 'job_list', 'job_output', 'list_agents', 'ralph', 'read', 'read_image', 'send_message', 'skill',
|
||||
'get_goal', 'interrupt_agent', 'job_kill', 'job_list', 'job_output', 'list_agents', 'ralph', 'read', 'read_image', 'read_image_region', 'send_message', 'skill',
|
||||
'subagent', 'subagent_fork', 'todo_write', 'update_goal', 'web_search',
|
||||
'workflow', 'write',
|
||||
])
|
||||
|
||||
@@ -48,6 +48,7 @@ const EXPECTED_TOOLS = [
|
||||
'ralph',
|
||||
'read',
|
||||
'read_image',
|
||||
'read_image_region',
|
||||
'send_message',
|
||||
'skill',
|
||||
'subagent',
|
||||
|
||||
@@ -145,7 +145,7 @@ interface RequestImageAttachment {
|
||||
}
|
||||
```
|
||||
|
||||
`saveImage()` prepares a provider-independent 2048px, 4MiB master and atomically commits it before returning its reference. `saveImages()` prepares every validated master once before publishing the batch, so validation rejection leaves no partial objects and publication does not repeat decoding or quality selection. `admitEncodedImages()` is the wire entry for base64 uploads and delegates count, aggregate-byte, and ordered batch admission to `saveImages()`. `readImage()` verifies a master from an authorized session path. `readImageRequest()` derives and caches one request version under an exact route pixel and byte budget; `readImageRequests()` lets an implementation apply its configured bounded transform concurrency to an ordered batch. The local implementation lazily encodes preferred candidates, singleflights equal request identities, and defaults to two simultaneous transformations. `cropImage()` maps model preview coordinates back to the master and returns another durable attachment. The service is retention-neutral: resumed and forked sessions may share objects, so reference-aware garbage collection is deferred rather than tied to one session's deletion.
|
||||
`saveImage()` prepares a provider-independent 2048px, 4MiB master and atomically commits it before returning its reference. `saveImages()` prepares every validated master once before publishing the batch, so validation rejection leaves no partial objects and publication does not repeat decoding or quality selection. `admitEncodedImages()` is the wire entry for base64 uploads and delegates count, aggregate-byte, and ordered batch admission to `saveImages()`. `readImage()` verifies a master from an authorized session path. `readImageRequest()` derives and caches one request version under an exact route pixel and byte budget; new entries are fully decoded before publication, while cache hits use a bounded metadata probe. `readImageRequests()` lets an implementation apply its configured transform concurrency to an ordered batch. The local implementation lazily encodes preferred candidates, singleflights equal request identities, lets each waiter cancel independently, stops shared work when no waiter remains, and defaults to two simultaneous transformations. `cropImage()` maps model preview coordinates back to the master and returns another durable attachment. The service is retention-neutral: resumed and forked sessions may share objects, so reference-aware garbage collection is deferred rather than tied to one session's deletion.
|
||||
|
||||
<!-- BEGIN GENERATED cordis-surface (gen-cordis-catalog.ts) — do not edit between markers -->
|
||||
|
||||
|
||||
@@ -145,7 +145,7 @@ interface RequestImageAttachment {
|
||||
}
|
||||
```
|
||||
|
||||
`saveImage()` 准备提供方无关的 2048px、4MiB 主版本,并在返回引用前以原子方式提交。`saveImages()` 在发布批次前为每个成员各准备一次经过验证的主版本,因此校验拒绝不会留下部分对象,发布也不会重复解码或选择质量。`admitEncodedImages()` 是面向 base64 上传的 wire 入口,把张数、聚合字节和有序批量准入交给 `saveImages()`。`readImage()` 校验来自已授权会话路径的主版本。`readImageRequest()` 按确切路由的像素和字节预算派生并缓存请求版本;`readImageRequests()` 允许实现按自身配置的有界变换并发处理有序批次。本地实现按需编码首选候选、合并相同请求身份的并发任务,默认同时执行两项变换。`cropImage()` 把模型预览坐标映射回主版本,并返回另一个持久附件。该服务不规定保留策略:恢复和 fork 后的会话可能共享对象,因此基于引用的垃圾回收会延期实现,不与单个会话的删除绑定。
|
||||
`saveImage()` 准备提供方无关的 2048px、4MiB 主版本,并在返回引用前以原子方式提交。`saveImages()` 在发布批次前为每个成员各准备一次经过验证的主版本,因此校验拒绝不会留下部分对象,发布也不会重复解码或选择质量。`admitEncodedImages()` 是面向 base64 上传的 wire 入口,把张数、聚合字节和有序批量准入交给 `saveImages()`。`readImage()` 校验来自已授权会话路径的主版本。`readImageRequest()` 按确切路由的像素和字节预算派生并缓存请求版本;新条目在发布前完整解码,缓存命中只做有界元数据探测。`readImageRequests()` 允许实现按自身配置的变换并发处理有序批次。本地实现按需编码首选候选、合并相同请求身份的并发任务、允许每个等待方单独取消、没有等待方时停止共享任务,默认同时执行两项变换。`cropImage()` 把模型预览坐标映射回主版本,并返回另一个持久附件。该服务不规定保留策略:恢复和 fork 后的会话可能共享对象,因此基于引用的垃圾回收会延期实现,不与单个会话的删除绑定。
|
||||
|
||||
<!-- BEGIN GENERATED cordis-surface (gen-cordis-catalog.ts) — do not edit between markers -->
|
||||
|
||||
|
||||
@@ -78,8 +78,8 @@ export async function hasLowColourCount(pipeline: Sharp): Promise<boolean> {
|
||||
const colours = new Set<number>()
|
||||
for (let offset = 0; offset < data.length; offset += info.channels) {
|
||||
const red = data[offset] ?? 0
|
||||
const green = data[offset + 1] ?? red
|
||||
const blue = data[offset + 2] ?? red
|
||||
const green = info.channels < 3 ? red : data[offset + 1] ?? red
|
||||
const blue = info.channels < 3 ? red : data[offset + 2] ?? red
|
||||
const alpha = info.channels === 2
|
||||
? data[offset + 1] ?? 255
|
||||
: info.channels === 4 ? data[offset + 3] ?? 255 : 255
|
||||
|
||||
@@ -71,21 +71,59 @@ export interface Config {
|
||||
imageCompressionConcurrency?: number
|
||||
}
|
||||
|
||||
function waitForShared<T>(operation: Promise<T>, signal: AbortSignal | undefined): Promise<T> {
|
||||
if (signal === undefined) return operation
|
||||
signal.throwIfAborted()
|
||||
return new Promise<T>((resolve, reject) => {
|
||||
const abort = (): void => {
|
||||
const reason: unknown = signal.reason
|
||||
reject(reason instanceof Error
|
||||
? reason
|
||||
: new Error('Attachment request cancelled with a non-Error reason.', { cause: reason }))
|
||||
}
|
||||
signal.addEventListener('abort', abort, { once: true })
|
||||
void operation.then(resolve, reject).finally(() => {
|
||||
signal.removeEventListener('abort', abort)
|
||||
function abortReason(signal: AbortSignal): Error {
|
||||
const reason: unknown = signal.reason
|
||||
return reason instanceof Error
|
||||
? reason
|
||||
: new Error('Attachment request cancelled with a non-Error reason.', { cause: reason })
|
||||
}
|
||||
|
||||
class SharedRequest<T> {
|
||||
readonly controller = new AbortController()
|
||||
readonly promise: Promise<T>
|
||||
private settled = false
|
||||
private waiters = 0
|
||||
|
||||
constructor(start: (signal: AbortSignal) => Promise<T>) {
|
||||
this.promise = start(this.controller.signal).finally(() => {
|
||||
this.settled = true
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
wait(signal?: AbortSignal): Promise<T> {
|
||||
signal?.throwIfAborted()
|
||||
this.waiters += 1
|
||||
if (signal === undefined) return this.promise.finally(() => this.release(false))
|
||||
let released = false
|
||||
const release = (cancelled: boolean): void => {
|
||||
if (released) return
|
||||
released = true
|
||||
this.release(cancelled, signal)
|
||||
}
|
||||
return new Promise<T>((resolve, reject) => {
|
||||
const abort = (): void => {
|
||||
release(true)
|
||||
reject(abortReason(signal))
|
||||
}
|
||||
signal.addEventListener('abort', abort, { once: true })
|
||||
void this.promise.then((value) => {
|
||||
signal.removeEventListener('abort', abort)
|
||||
release(false)
|
||||
resolve(value)
|
||||
}, (error: unknown) => {
|
||||
signal.removeEventListener('abort', abort)
|
||||
release(false)
|
||||
reject(error)
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
private release(cancelled: boolean, signal?: AbortSignal): void {
|
||||
this.waiters -= 1
|
||||
if (cancelled && this.waiters === 0 && !this.settled && signal !== undefined) {
|
||||
this.controller.abort(abortReason(signal))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** Persistent content-addressed local attachment store. */
|
||||
@@ -111,7 +149,7 @@ export class LocalAttachmentStore extends AttachmentStore {
|
||||
/** Resolved instance-level compression limit. */
|
||||
readonly imageCompressionConcurrency: number
|
||||
private readonly compression: CompressionLimiter
|
||||
private readonly requestInflight = new Map<string, Promise<RequestImageAttachment>>()
|
||||
private readonly requestInflight = new Map<string, SharedRequest<RequestImageAttachment>>()
|
||||
|
||||
constructor(ctx: Context, config: Config) {
|
||||
super(ctx)
|
||||
@@ -191,18 +229,24 @@ export class LocalAttachmentStore extends AttachmentStore {
|
||||
const variantId = requestImageVariantId(ref, policy)
|
||||
const key = String(variantId)
|
||||
let operation = this.requestInflight.get(key)
|
||||
if (operation?.controller.signal.aborted) {
|
||||
this.requestInflight.delete(key)
|
||||
operation = undefined
|
||||
}
|
||||
if (operation === undefined) {
|
||||
operation = this.compression.run(async () => readRequestImageFile(
|
||||
const shared = new SharedRequest<RequestImageAttachment>(sharedSignal => this.compression.run(async () => readRequestImageFile(
|
||||
this.root,
|
||||
master ?? await this.readImage(ref),
|
||||
master ?? await this.readImage(ref, sharedSignal),
|
||||
policy,
|
||||
))
|
||||
this.requestInflight.set(key, operation)
|
||||
void operation.finally(() => {
|
||||
if (this.requestInflight.get(key) === operation) this.requestInflight.delete(key)
|
||||
sharedSignal,
|
||||
)))
|
||||
operation = shared
|
||||
this.requestInflight.set(key, shared)
|
||||
void shared.promise.finally(() => {
|
||||
if (this.requestInflight.get(key) === shared) this.requestInflight.delete(key)
|
||||
}).catch(() => {})
|
||||
}
|
||||
return waitForShared(operation, signal)
|
||||
return operation.wait(signal)
|
||||
}
|
||||
|
||||
override async cropImage(
|
||||
|
||||
@@ -19,7 +19,7 @@ import { encodeFirstWithinLimit, isExhaustedEncoding } from './encoding.ts'
|
||||
import { detectImage, probeImage } from './image.ts'
|
||||
|
||||
/** Transform version included in every cache and upload-index identity. */
|
||||
export const REQUEST_IMAGE_TRANSFORM_VERSION = 'request-image-v2'
|
||||
export const REQUEST_IMAGE_TRANSFORM_VERSION = 'request-image-v3'
|
||||
/** DeepSeek request versions normally fit at these two preferred qualities. */
|
||||
export const REQUEST_IMAGE_QUALITIES = [85, 80] as const
|
||||
|
||||
@@ -228,7 +228,7 @@ async function readCached(
|
||||
): Promise<VerifiedRequestImage | undefined> {
|
||||
try {
|
||||
const data = new Uint8Array(await readFile(path, { signal }))
|
||||
const detected = await detectImage(data)
|
||||
const detected = await probeImage(data)
|
||||
const crop = policy.crop
|
||||
const maximum = requestImageDimensions(crop?.width ?? master.ref.width, crop?.height ?? master.ref.height, policy.maxPixels)
|
||||
if (data.byteLength > policy.maxBytes || detected.depth !== 'uchar' || detected.space !== 'srgb'
|
||||
@@ -278,7 +278,7 @@ async function writeCached(path: string, data: Uint8Array): Promise<void> {
|
||||
* @param root - absolute versioned attachment storage root.
|
||||
* @param master - verified stored master bytes and reference.
|
||||
* @param policy - exact route request-image policy.
|
||||
* @param signal - optional cancellation for cache I/O.
|
||||
* @param signal - optional cancellation for cache I/O and image transformation.
|
||||
* @returns verified request bytes and deterministic variant identity.
|
||||
*/
|
||||
export async function readRequestImageFile(
|
||||
|
||||
@@ -271,6 +271,23 @@ describe('hasLowColourCount', () => {
|
||||
await expect(hasLowColourCount(transparent)).resolves.toBe(true)
|
||||
})
|
||||
|
||||
it('reads grayscale-alpha samples without treating alpha or the next pixel as RGB', async () => {
|
||||
const symbols: number[] = []
|
||||
for (let first = 0; first < 32; first += 1) {
|
||||
for (let second = 0; second < 32; second += 1) symbols.push(first, second)
|
||||
}
|
||||
const pixels = new Uint8Array(symbols.length * 2)
|
||||
for (const [index, symbol] of symbols.entries()) {
|
||||
pixels[index * 2] = symbol * 8
|
||||
pixels[index * 2 + 1] = symbol * 8
|
||||
}
|
||||
const grayscaleAlpha = sharp(pixels, {
|
||||
raw: { width: 128, height: 16, channels: 2 },
|
||||
})
|
||||
|
||||
await expect(hasLowColourCount(grayscaleAlpha)).resolves.toBe(true)
|
||||
})
|
||||
|
||||
it('keeps an antialiased text screenshot readable on the low-colour PNG path', async () => {
|
||||
const source = new Uint8Array(await sharp(Buffer.from(`
|
||||
<svg width="1024" height="512" xmlns="http://www.w3.org/2000/svg">
|
||||
|
||||
@@ -206,4 +206,31 @@ describe('local request-image cache', () => {
|
||||
expect(run).toHaveBeenCalledTimes(1)
|
||||
run.mockRestore()
|
||||
})
|
||||
|
||||
it('aborts the underlying request transform after its only waiter cancels', async () => {
|
||||
const attachments = await store()
|
||||
const master = (await attachments.saveImage({
|
||||
data: await image(2048, 1024), mediaType: 'image/png', name: 'cancelled.png',
|
||||
})).ref
|
||||
let readSignal: AbortSignal | undefined
|
||||
const read = vi.spyOn(attachments, 'readImage').mockImplementation((_ref, signal) => {
|
||||
readSignal = signal
|
||||
return new Promise((_resolve, reject) => {
|
||||
signal?.addEventListener('abort', () => reject(signal.reason), { once: true })
|
||||
})
|
||||
})
|
||||
const controller = new AbortController()
|
||||
const request = attachments.readImageRequest(
|
||||
master,
|
||||
{ maxPixels: 640_000, maxBytes: 1024 * 1024 },
|
||||
controller.signal,
|
||||
)
|
||||
await vi.waitFor(() => expect(read).toHaveBeenCalledTimes(1))
|
||||
|
||||
const reason = new Error('cancel only transform waiter')
|
||||
controller.abort(reason)
|
||||
|
||||
await expect(request).rejects.toBe(reason)
|
||||
expect(readSignal?.reason).toBe(reason)
|
||||
})
|
||||
})
|
||||
|
||||
@@ -31,7 +31,7 @@ describe('gen-tool-catalog collectToolCatalog', () => {
|
||||
'cordis_undefine', 'create_goal', 'edit', 'exit_plan_mode', 'followup_task', 'get_goal', 'glob', 'grep',
|
||||
'interrupt_agent', 'interrupt_agent', 'job_kill', 'job_list', 'job_output',
|
||||
'list_agents', 'list_agents', 'lsp', 'pwsh', 'pwsh', 'ralph',
|
||||
'read', 'read_image', 'report', 'run_code', 'schedule_create', 'schedule_delete',
|
||||
'read', 'read_image', 'read_image_region', 'report', 'run_code', 'schedule_create', 'schedule_delete',
|
||||
'schedule_list', 'send_message', 'send_message', 'session_event_read', 'session_event_search',
|
||||
'session_event_trace', 'session_search', 'session_trace', 'skill', 'spawn_teammate',
|
||||
'str_replace_editor', 'subagent', 'team_task_create',
|
||||
|
||||
@@ -29,6 +29,22 @@ const IMAGE_EXTENSIONS: Readonly<Record<string, ImageMediaType>> = {
|
||||
'.gif': 'image/gif',
|
||||
}
|
||||
|
||||
const IMAGE_VALUE_SCHEMA = {
|
||||
type: 'object',
|
||||
additionalProperties: false,
|
||||
required: true,
|
||||
properties: {
|
||||
attachmentId: { type: 'string', required: true },
|
||||
mediaType: { type: 'string', enum: ['image/png', 'image/jpeg', 'image/webp', 'image/gif'], required: true },
|
||||
bytes: { type: 'integer', required: true },
|
||||
width: { type: 'integer', required: true },
|
||||
height: { type: 'integer', required: true },
|
||||
name: { type: 'string' },
|
||||
sourceWidth: { type: 'integer' },
|
||||
sourceHeight: { type: 'integer' },
|
||||
},
|
||||
} as const
|
||||
|
||||
/** The structured outcome declared by the `read_image` output schema. */
|
||||
export interface ImageReadValue {
|
||||
path: string
|
||||
@@ -214,21 +230,7 @@ export function applyReadImageTool(ctx: Context): void {
|
||||
additionalProperties: false,
|
||||
properties: {
|
||||
path: { type: 'string', required: true },
|
||||
image: {
|
||||
type: 'object',
|
||||
additionalProperties: false,
|
||||
required: true,
|
||||
properties: {
|
||||
attachmentId: { type: 'string', required: true },
|
||||
mediaType: { type: 'string', enum: ['image/png', 'image/jpeg', 'image/webp', 'image/gif'], required: true },
|
||||
bytes: { type: 'integer', required: true },
|
||||
width: { type: 'integer', required: true },
|
||||
height: { type: 'integer', required: true },
|
||||
name: { type: 'string' },
|
||||
sourceWidth: { type: 'integer' },
|
||||
sourceHeight: { type: 'integer' },
|
||||
},
|
||||
},
|
||||
image: IMAGE_VALUE_SCHEMA,
|
||||
},
|
||||
},
|
||||
render: (_args, value) => imageReadContent(value),
|
||||
@@ -370,21 +372,7 @@ export function applyReadImageTool(ctx: Context): void {
|
||||
height: { type: 'integer', required: true },
|
||||
},
|
||||
},
|
||||
image: {
|
||||
type: 'object',
|
||||
additionalProperties: false,
|
||||
required: true,
|
||||
properties: {
|
||||
attachmentId: { type: 'string', required: true },
|
||||
mediaType: { type: 'string', enum: ['image/png', 'image/jpeg', 'image/webp', 'image/gif'], required: true },
|
||||
bytes: { type: 'integer', required: true },
|
||||
width: { type: 'integer', required: true },
|
||||
height: { type: 'integer', required: true },
|
||||
name: { type: 'string' },
|
||||
sourceWidth: { type: 'integer' },
|
||||
sourceHeight: { type: 'integer' },
|
||||
},
|
||||
},
|
||||
image: IMAGE_VALUE_SCHEMA,
|
||||
},
|
||||
},
|
||||
render: (_args, value) => regionReadContent(value),
|
||||
|
||||
@@ -185,7 +185,6 @@ function imageInEvent(event: SessionEvent, match: (ref: ImageAttachmentRef) => b
|
||||
return undefined
|
||||
}
|
||||
|
||||
/** True when the current model-visible surface contains an image. */
|
||||
/** Resolve the first reference matching one opaque id. */
|
||||
function referencedImage(events: readonly SessionEvent[], attachmentId: string): ImageAttachmentRef | undefined {
|
||||
for (const event of events) {
|
||||
|
||||
@@ -486,6 +486,11 @@ describe('image attachments', () => {
|
||||
source: { mediaType: input.mediaType, bytes: 3, width: 1, height: 1 },
|
||||
})
|
||||
}),
|
||||
validateImageBatch(inputs: readonly unknown[]) {
|
||||
return (AttachmentStore.prototype as unknown as {
|
||||
validateImageBatch(this: unknown, batch: readonly unknown[]): void
|
||||
}).validateImageBatch.call(this, inputs)
|
||||
},
|
||||
// The real base-class batch method over this double's limits and members.
|
||||
saveImages(inputs: readonly unknown[]) {
|
||||
return (AttachmentStore.prototype.saveImages as (this: unknown, batch: readonly unknown[]) => Promise<unknown[]>).call(this, inputs)
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/llm/llm-deepseek/README.md
|
||||
README.md: ea82956d77f3157638aa078c56630994ccc61d75
|
||||
README.zh.md: ff8780f59a3caac7e08e5ab6e08c4b2b15d1b57d
|
||||
README.md: b20d93394055e3e10dfb5a932660b6a510428492
|
||||
README.zh.md: 6e8166227d740c0431c17c091d68b5d56aea0dc5
|
||||
|
||||
@@ -23,6 +23,7 @@ The package root exposes the Cordis plugin contract and `DeepSeekAdapter`; wire
|
||||
maxRequestFilesBytes: 134217728 # optional positive integer; 128 MiB raw request-image default
|
||||
maxImagesPerRequest: 600 # provider request image-count limit
|
||||
imageOffloadByteQuantum: 67108864 # oldest-image removal advances in 64 MiB steps
|
||||
imageOffloadCountQuantum: 20 # count overflow advances in 20-image steps
|
||||
fileExpiresAfterSeconds: 604800 # uploaded image lifetime; 1 hour to 30 days
|
||||
fileRefreshMarginSeconds: 3600 # replace ids with less lifetime remaining
|
||||
fileQuotaCleanupBatch: 100 # oldest harness-owned files deleted before one quota retry
|
||||
@@ -48,13 +49,13 @@ The package root exposes the Cordis plugin contract and `DeepSeekAdapter`; wire
|
||||
|
||||
The plugin registers the single provider route `deepseek-official` together with its resolved `retryPolicy`; omission resolves to normal mode with five retries. A request selects it with `provider: deepseek-official`; its `model` is passed through as the wire `model` string, so changing DeepSeek models does not require lifecycle-time registration. Omitting `models` advertises `deepseek-v4-flash`, `deepseek-v4-pro`, and the image-capable `deepseek-v4-flash-vision-exp`, each with a 1,000,000-token context window; an explicit list replaces those defaults, while `models: []` advertises none. Catalog entries are exposed through `ctx.llm.listModels('deepseek-official')` for clients such as ACP editors and the Web selector, but remain advisory: unlisted model ids still pass through unchanged as text-only routes. An omitted entry name defaults to its id, and omitted `inputModalities` means `text` only.
|
||||
|
||||
An image-capable catalog entry declares `inputModalities: [text, image]` and may set `imagePixelBudget`, `imageMaxBytes`, or `imageDetail: low`. The ordinary default is 640,000 total pixels and 1MiB encoded bytes; low detail defaults to 512 by 512 total pixels. The attachment store scales by `min(1, sqrt(pixelBudget / (width * height)))` and rounds inward to keep the pixel count at or below the hard cap, so a 2048 by 1024 master becomes about 1130 by 565 instead of a forced square. Request encoders run lazily: low-color images try PNG (palette only without alpha) then WebP 85 and 80, other alpha images try WebP 85 then 80, and other opaque images try JPEG 85 then 80; dimensions shrink only when both quality attempts exceed 1MiB. Concurrent generation of one `variantId` shares one transform. The adapter uploads the exact derived request bytes through `POST /files` and sends `{type: "file", file_id}` blocks. It never falls back to an inline data URL. Every retained image is preceded by stable text naming the complete attachment id, actual request dimensions, and the preview-coordinate arguments for `read_image_region`. User, tool-result, agent-loop, compaction, and direct `ctx.llm.stream` requests all use this projection. Text-only routes receive stable attachment placeholders while durable history keeps its image references.
|
||||
An image-capable catalog entry declares `inputModalities: [text, image]` and may set `imagePixelBudget`, `imageMaxBytes`, or `imageDetail: low`. The ordinary default is 640,000 total pixels and 1MiB encoded bytes; low detail defaults to 512 by 512 total pixels. The attachment store scales by `min(1, sqrt(pixelBudget / (width * height)))` and rounds inward to keep the pixel count at or below the hard cap, so a 2048 by 1024 master becomes about 1130 by 565 instead of a forced square. Request encoders run lazily: low-color images try PNG (palette only without alpha) then WebP 85 and 80, other alpha images try WebP 85 then 80, and other opaque images try JPEG 85 then 80; dimensions shrink only when both quality attempts exceed 1MiB. Concurrent generation of one `variantId` shares one transform. A caller can cancel its own wait without interrupting other waiters; the transform stops when no waiter remains. The adapter uploads the exact derived request bytes through `POST /files` and sends `{type: "file", file_id}` blocks. It never falls back to an inline data URL. Every retained image is preceded by stable text naming the complete attachment id and actual request dimensions. Preview-coordinate arguments are included only when the request exposes `read_image_region`. User, tool-result, agent-loop, compaction, and direct `ctx.llm.stream` requests all use this projection. Text-only routes receive stable attachment placeholders while durable history keeps its image references.
|
||||
|
||||
`maxRequestFilesBytes` and `maxImagesPerRequest` bound the retained request versions at 128MiB and 600 images by default. When the byte bound is crossed, the oldest prefix advances past the next 64MiB boundary; 129 one-megabyte images remove the oldest 65 and retain 64MiB, and that prefix stays unchanged until durable history exceeds 192MiB. Count overflow advances independently in `imageOffloadCountQuantum` steps. Removed images become the fixed model-visible placeholder `[image omitted to keep the request within its image limit; older images are omitted first. If this image is still needed, read its file again when a path is available; otherwise ask the user to attach it again.]`. This high-watermark projection avoids changing an old request prefix after every new image.
|
||||
`maxRequestFilesBytes` and `maxImagesPerRequest` bound the retained request versions at 128MiB and 600 images by default. The byte and count quanta must not exceed their corresponding bounds. Before attachment reads, the adapter uses each route's request-version byte cap as a conservative upper bound and removes the oldest over-budget prefix; only retained masters are read and transformed. Exact derived lengths are checked again without restoring omitted images. When the byte bound is crossed, the oldest prefix advances past the next 64MiB boundary; 129 one-megabyte images remove the oldest 65 and retain 64MiB, and that prefix stays unchanged until durable history exceeds 192MiB. Count overflow advances independently in `imageOffloadCountQuantum` steps. Removed images become the fixed model-visible placeholder `[image omitted to keep the request within its image limit; older images are omitted first. If this image is still needed, read its file again when a path is available; otherwise ask the user to attach it again.]`. This high-watermark projection avoids changing an old request prefix after every new image.
|
||||
|
||||
Uploaded ids are indexed below `DSH_HOME` by endpoint/API-key scope and request `variantId`. The variant covers the master attachment id, transform version, route pixel and byte budgets, crop, and encoder parameters, so Files API and inline-capable adapters refer to the same deterministic bytes. Uploads request a seven-day lifetime by default and store the server's `expires_at`. A local mapping with no more than one hour remaining is replaced before use; the adapter does not retrieve every remote file before chat. If chat reports expired, deleted, missing, or invalid file ids and names one or more ids used by the request, the adapter removes exactly those mappings. If the provider identifies stale file state without naming an id, it removes every file mapping used by that chat attempt. It then uploads the affected request versions again and retries chat once. A second stale-file rejection clears the mappings identified by that response and is returned without a third chat attempt. An upload response without a complete file object, matching byte count, and `expires_at` is never indexed; a later request therefore uploads again instead of trusting inconsistent local state. A malformed local upload index is treated as an empty cache and replaced by the next successful upload; permission and filesystem I/O failures still fail the request.
|
||||
|
||||
One quota upload failure triggers deletion of the configured number of oldest `dsh-` files and one upload retry. `DeepSeekFilesClient.delete`, `DeepSeekFileStore.release`, and `releaseAll` expose explicit remote-space reclamation. The current provider limits represented by this package are 128MiB per Files upload, 32MiB per chat-referenced image, 10,000 stored files, and 25GiB per API key; the default 1MiB request version remains below the two per-file limits.
|
||||
Concurrent resolution of one scoped `variantId` shares one Files upload with waiter-local cancellation. One quota upload failure first paginates and collects the configured number of oldest `dsh-` files, then deletes that set before one upload retry. `DeepSeekFilesClient.delete`, `DeepSeekFileStore.release`, and `releaseAll` expose explicit remote-space reclamation. The current provider limits represented by this package are 128MiB per Files upload, 32MiB per chat-referenced image, 10,000 stored files, and 25GiB per API key; the default 1MiB request version remains below the two per-file limits.
|
||||
|
||||
`contextWindow` is optional per configured model and is not exposed through the advisory catalog. `ctx.llm.resolveModelInfo('deepseek-official', model).context` returns an exact model value first, then `defaultContextWindow` for an entry without capacity or an unlisted pass-through id. The adapter default is 1,000,000; pressure-sensitive plugins therefore get deployment-owned capacity without treating the model selector as authoritative. Registering another adapter for `deepseek-official` throws `LlmError('DUPLICATE_ADAPTER')`.
|
||||
|
||||
@@ -80,7 +81,7 @@ The plugin also declares its route in the configurable-provider directory (`ctx.
|
||||
|
||||
## App attribution
|
||||
|
||||
Every request carries the shared attribution header from dsh-llm's `attributionHeaders()` - the mandatory `User-Agent` baseline identifying the harness (see [dsh-llm § App attribution](../llm/README.md#app-attribution-attributionts)). Direct DeepSeek requests and OpenAI-compatible gateway requests get no provider-specific app-attribution headers under this adapter contract; OpenRouter app attribution is deferred to a future explicit OpenRouter adapter or mode. A request whose `GenerateOptions.purpose` is `compaction` (dsh-compaction-basic's auxiliary summarization call) additionally carries `x-deepseek-harness-compact: 1`, so the host can separate compaction traffic from conversation requests.
|
||||
Every chat and Files API request carries the shared attribution header from dsh-llm's `attributionHeaders()`, the mandatory `User-Agent` baseline identifying the harness (see [dsh-llm § App attribution](../llm/README.md#app-attribution-attributionts)). Direct DeepSeek requests and OpenAI-compatible gateway requests get no provider-specific app-attribution headers under this adapter contract; OpenRouter app attribution is deferred to a future explicit OpenRouter adapter or mode. A request whose `GenerateOptions.purpose` is `compaction` (dsh-compaction-basic's auxiliary summarization call) additionally carries `x-deepseek-harness-compact: 1`, so the host can separate compaction traffic from conversation requests.
|
||||
|
||||
DeepSeek request identity is separate from app attribution. After credential resolution, every provider request carries `x-deepseek-harness-user-id` with the stable anonymous id from [`@deepseek-ai/dsh-anonymous-user-id`](../../identity/anonymous-user-id/README.md); a request carrying `GenerateOptions.sessionId` also sends that exact value as `x-deepseek-harness-session-id`, while a direct call without a session omits the session header. Both headers go to the resolved `baseURL`, including a configured gateway, and remain outside the request body and model-visible content.
|
||||
|
||||
|
||||
@@ -23,6 +23,7 @@ harness LLM(大语言模型)seam 的 DeepSeek chat-completions 适配器:
|
||||
maxRequestFilesBytes: 134217728 # optional positive integer; 128 MiB raw request-image default
|
||||
maxImagesPerRequest: 600 # provider request image-count limit
|
||||
imageOffloadByteQuantum: 67108864 # oldest-image removal advances in 64 MiB steps
|
||||
imageOffloadCountQuantum: 20 # count overflow advances in 20-image steps
|
||||
fileExpiresAfterSeconds: 604800 # uploaded image lifetime; 1 hour to 30 days
|
||||
fileRefreshMarginSeconds: 3600 # replace ids with less lifetime remaining
|
||||
fileQuotaCleanupBatch: 100 # oldest harness-owned files deleted before one quota retry
|
||||
@@ -48,13 +49,13 @@ harness LLM(大语言模型)seam 的 DeepSeek chat-completions 适配器:
|
||||
|
||||
该插件注册唯一提供方路由 `deepseek-official`,并一同注册解析后的 `retryPolicy`;省略时会解析为 normal 模式并重试五次。请求使用 `provider: deepseek-official` 选择该路由;其 `model` 会作为协议 `model` 字符串原样传递,因此更改 DeepSeek 模型不需要生命周期时注册。省略 `models` 会公布 `deepseek-v4-flash`、`deepseek-v4-pro` 与支持图片输入的 `deepseek-v4-flash-vision-exp`,三者的上下文窗口均为 1,000,000 token;显式列表会替换这些默认值,`models: []` 则不公布任何模型。Catalog 配置项通过 `ctx.llm.listModels('deepseek-official')` 公开给 ACP(Agent Client Protocol)编辑器和 Web 选择器等客户端,但仍只提供建议:未列出模型 id 仍原样传递,并按纯文本路由处理。省略配置项 name 默认为其 id,省略 `inputModalities` 则表示仅支持 `text`。
|
||||
|
||||
支持图片的 catalog 配置项声明 `inputModalities: [text, image]`,并可设置 `imagePixelBudget`、`imageMaxBytes` 或 `imageDetail: low`。普通默认值为总像素 640,000、编码字节 1MiB;low detail 的默认总像素为 512×512。附件存储按 `min(1, sqrt(pixelBudget / (width * height)))` 缩放,并向预算内取整,确保总像素不超过硬上限。因此 2048×1024 主版本会得到约 1130×565 的请求版本,而不会被强制变成正方形。请求编码按需执行:低色数图片先尝试 PNG,只有不带 alpha 通道时才使用 palette,再尝试质量 85 和 80 的 WebP;其他透明图片依次尝试质量 85 和 80 的 WebP;其他非透明图片依次尝试质量 85 和 80 的 JPEG。两个质量档均超过 1MiB 时才缩小尺寸。同一 `variantId` 的并发生成共享一次变换。适配器通过 `POST /files` 上传确切的派生请求字节,再发送 `{type: "file", file_id}` 块,不会回退到内联 data URL。每张保留图片前都有稳定文本,写明完整附件 ID、实际请求尺寸,以及 `read_image_region` 所需的预览坐标参数。User、工具结果、agent loop、压缩和直接 `ctx.llm.stream` 请求都使用该投影。纯文本路由会收到稳定的附件占位文本,持久历史继续保留图片引用。
|
||||
支持图片的 catalog 配置项声明 `inputModalities: [text, image]`,并可设置 `imagePixelBudget`、`imageMaxBytes` 或 `imageDetail: low`。普通默认值为总像素 640,000、编码字节 1MiB;low detail 的默认总像素为 512×512。附件存储按 `min(1, sqrt(pixelBudget / (width * height)))` 缩放,并向预算内取整,确保总像素不超过硬上限。因此 2048×1024 主版本会得到约 1130×565 的请求版本,而不会被强制变成正方形。请求编码按需执行:低色数图片先尝试 PNG,只有不带 alpha 通道时才使用 palette,再尝试质量 85 和 80 的 WebP;其他透明图片依次尝试质量 85 和 80 的 WebP;其他非透明图片依次尝试质量 85 和 80 的 JPEG。两个质量档均超过 1MiB 时才缩小尺寸。同一 `variantId` 的并发生成共享一次变换。调用方可以单独取消等待,不会中断其他等待方;没有等待方时才会停止变换。适配器通过 `POST /files` 上传确切的派生请求字节,再发送 `{type: "file", file_id}` 块,不会回退到内联 data URL。每张保留图片前都有稳定文本,写明完整附件 ID 和实际请求尺寸。只有当前请求公开 `read_image_region` 时才会提供预览坐标参数。User、工具结果、agent loop、压缩和直接 `ctx.llm.stream` 请求都使用该投影。纯文本路由会收到稳定的附件占位文本,持久历史继续保留图片引用。
|
||||
|
||||
`maxRequestFilesBytes` 和 `maxImagesPerRequest` 限制请求中保留的请求版本,默认值分别为 128MiB 和 600 张。字节数越过上限时,被移除的最旧前缀会越过下一个 64MiB 边界。由 1MiB 图片组成的历史达到 129MiB 时会移除最旧的 65 张并保留 64MiB;直到持久历史超过 192MiB,这个前缀才再次变化。图片数量超限时则按 `imageOffloadCountQuantum` 独立递增。移除的图片会变成固定模型可见占位文本 `[image omitted to keep the request within its image limit; older images are omitted first. If this image is still needed, read its file again when a path is available; otherwise ask the user to attach it again.]`。这种定量投影不会因每新增一张图片就改写较早的请求前缀。
|
||||
`maxRequestFilesBytes` 和 `maxImagesPerRequest` 限制请求中保留的请求版本,默认值分别为 128MiB 和 600 张。字节和数量步长不得超过对应上限。读取附件前,适配器以路由的请求版本字节上限作为保守上界,移除超预算的最旧前缀,只读取并转换保留的主版本。系统随后用确切派生长度再次检查,但不会重新加入已省略图片。字节数越过上限时,被移除的最旧前缀会越过下一个 64MiB 边界。由 1MiB 图片组成的历史达到 129MiB 时会移除最旧的 65 张并保留 64MiB;直到持久历史超过 192MiB,这个前缀才再次变化。图片数量超限时则按 `imageOffloadCountQuantum` 独立递增。移除的图片会变成固定模型可见占位文本 `[image omitted to keep the request within its image limit; older images are omitted first. If this image is still needed, read its file again when a path is available; otherwise ask the user to attach it again.]`。这种定量投影不会因每新增一张图片就改写较早的请求前缀。
|
||||
|
||||
上传 ID 按端点和 API key 作用域以及请求 `variantId` 记录在 `DSH_HOME` 下。变体身份覆盖主附件 ID、变换策略版本、路由像素和字节预算、裁剪区域及编码参数,因此 Files API 和支持内联的适配器引用同一份确定性字节。上传默认请求 7 天有效期,并保存服务端返回的 `expires_at`。本地映射剩余时间不超过一小时时会在使用前替换;适配器不会在每次 chat 前查询远端文件。如果 chat 报告文件 ID 已过期、删除、缺失或无效,并指出本次请求使用的一个或多个 ID,适配器只删除这些映射。如果响应只说明文件状态失效而没有指出 ID,适配器会删除该次 chat 使用的全部文件映射。随后重新上传受影响的请求版本,并重试一次 chat。第二次 chat 仍报告文件失效时,适配器会按该响应清理映射并返回错误,不会发起第三次 chat。上传响应若没有完整文件对象、匹配的字节数和 `expires_at`,就不会写入索引;后续请求会再次上传,而不是信任不一致的本地状态。本地上传索引格式损坏时按空缓存处理,并由下一次成功上传替换;权限和文件系统 I/O 错误仍使请求失败。
|
||||
|
||||
一次上传配额错误会触发删除配置数量的最旧 `dsh-` 文件,然后重试一次上传。`DeepSeekFilesClient.delete`、`DeepSeekFileStore.release` 和 `releaseAll` 提供主动远端空间回收。本包记录的当前提供方限制为 Files 单次上传 128MiB、chat 单图引用 32MiB、每个 API key 最多 10,000 个文件和 25GiB;默认 1MiB 请求版本低于两个单文件上限。
|
||||
同一作用域和 `variantId` 的并发解析共享一次 Files 上传,每个等待方可以单独取消。一次上传配额错误会先分页收集配置数量的最旧 `dsh-` 文件,再删除这些文件并重试一次上传。`DeepSeekFilesClient.delete`、`DeepSeekFileStore.release` 和 `releaseAll` 提供主动远端空间回收。本包记录的当前提供方限制为 Files 单次上传 128MiB、chat 单图引用 32MiB、每个 API key 最多 10,000 个文件和 25GiB;默认 1MiB 请求版本低于两个单文件上限。
|
||||
|
||||
`contextWindow` 对每个已配置模型都可选,不会通过建议 catalog 公开。`ctx.llm.resolveModelInfo('deepseek-official', model).context` 先返回精确模型值,再对不含容量的配置项或未列出原样传递 id 返回 `defaultContextWindow`。适配器默认值为 1,000,000;因此,压力敏感插件可以获得由部署决定的容量,不会将模型 selector 视为权威。为 `deepseek-official` 注册另一个适配器会抛出 `LlmError('DUPLICATE_ADAPTER')`。
|
||||
|
||||
@@ -80,7 +81,7 @@ harness LLM(大语言模型)seam 的 DeepSeek chat-completions 适配器:
|
||||
|
||||
## 应用归因
|
||||
|
||||
每个请求都携带 dsh-llm `attributionHeaders()` 的共享归因标头,即用于识别 harness 的必需 `User-Agent` 基线(见 [dsh-llm § 应用归因](../llm/README.zh.md#app-attribution-attributionts))。在该适配器约定(adapter contract)下,直接 DeepSeek 请求与 OpenAI 兼容 gateway 请求都不会获得提供方特定应用归因标头;OpenRouter 应用归因暂缓到未来的显式 OpenRouter 适配器或模式。`GenerateOptions.purpose` 为 `compaction` 的请求(dsh-compaction-basic 的辅助摘要调用)还会携带 `x-deepseek-harness-compact: 1`,让宿主可以将压缩流量与会话请求分开。
|
||||
每个 chat 和 Files API 请求都携带 dsh-llm `attributionHeaders()` 的共享归因标头,即用于识别 harness 的必需 `User-Agent` 基线(见 [dsh-llm § 应用归因](../llm/README.zh.md#app-attribution-attributionts))。在该适配器约定(adapter contract)下,直接 DeepSeek 请求与 OpenAI 兼容 gateway 请求都不会获得提供方特定应用归因标头;OpenRouter 应用归因暂缓到未来的显式 OpenRouter 适配器或模式。`GenerateOptions.purpose` 为 `compaction` 的请求(dsh-compaction-basic 的辅助摘要调用)还会携带 `x-deepseek-harness-compact: 1`,让宿主可以将压缩流量与会话请求分开。
|
||||
|
||||
DeepSeek 请求身份独立于应用归因。凭据解析成功后,每个提供方请求都会通过 `x-deepseek-harness-user-id` 携带来自 [`@deepseek-ai/dsh-anonymous-user-id`](../../identity/anonymous-user-id/README.zh.md) 的稳定匿名 id;携带 `GenerateOptions.sessionId` 的请求还会通过 `x-deepseek-harness-session-id` 发送该确切值,缺少会话的直接调用则省略会话标头。两个标头都会发送至解析后的 `baseURL`(包括已配置的 gateway),且不会进入请求正文或模型可见内容。
|
||||
|
||||
|
||||
@@ -8,7 +8,7 @@
|
||||
* @module dsh-llm-deepseek/adapter
|
||||
*/
|
||||
|
||||
import { attributionHeaders, contentHasImage, CONTEXT_WINDOW_EXCEEDED_CODE, isContextWindowExceededError, isQuotaExceededError, LlmAdapter, LlmError, ProviderRequestId, QUOTA_EXCEEDED_CODE, ReasoningEffortId } from '@deepseek-ai/dsh-llm'
|
||||
import { attributionHeaders, contentHasImage, CONTEXT_WINDOW_EXCEEDED_CODE, isContextWindowExceededError, isQuotaExceededError, LlmAdapter, LlmError, offloadRequestImagesWithPolicy, ProviderRequestId, QUOTA_EXCEEDED_CODE, ReasoningEffortId } from '@deepseek-ai/dsh-llm'
|
||||
import type {
|
||||
ContentBlock,
|
||||
GenerateOptions,
|
||||
@@ -510,14 +510,24 @@ export class DeepSeekAdapter extends LlmAdapter {
|
||||
|
||||
const fileConnection = { baseURL: connection.baseURL, apiKey }
|
||||
const model = connection.models.find(entry => entry.id === options.model)
|
||||
const policy = model === undefined ? undefined : resolveRequestImagePolicy(model)
|
||||
const requestMessages = policy === undefined ? options.messages : offloadRequestImagesWithPolicy(options.messages, {
|
||||
representation: 'raw',
|
||||
maxBytes: connection.maxRequestFilesBytes,
|
||||
maxImages: connection.maxImagesPerRequest,
|
||||
byteQuantum: connection.imageOffloadByteQuantum,
|
||||
countQuantum: connection.imageOffloadCountQuantum,
|
||||
byteLength: ref => Math.min(ref.bytes, policy.maxBytes),
|
||||
})
|
||||
const requestOptions = requestMessages === options.messages ? options : { ...options, messages: [...requestMessages] }
|
||||
const requestImages = attachments === undefined || model === undefined
|
||||
? new Map<AttachmentId, RequestImageAttachment>()
|
||||
: await prepareRequestImages(options, attachments, model, signal)
|
||||
: await prepareRequestImages(requestOptions, attachments, model, signal)
|
||||
for (let fileAttempt = 0; fileAttempt < 2; fileAttempt += 1) {
|
||||
const usedFiles: UsedRequestFile[] = []
|
||||
const body = attachments === undefined
|
||||
? serializeRequest(options, connection.defaults)
|
||||
: await serializeRequestWithImages(options, {
|
||||
? serializeRequest(requestOptions, connection.defaults)
|
||||
: await serializeRequestWithImages(requestOptions, {
|
||||
requestImages,
|
||||
resolveFileId: async (version, _block, location) => {
|
||||
const resolved = await this.files.ensureUploaded(
|
||||
@@ -533,6 +543,7 @@ export class DeepSeekAdapter extends LlmAdapter {
|
||||
maxImagesPerRequest: connection.maxImagesPerRequest,
|
||||
byteQuantum: connection.imageOffloadByteQuantum,
|
||||
countQuantum: connection.imageOffloadCountQuantum,
|
||||
cropAvailable: options.tools?.some(tool => tool.name === 'read_image_region') ?? false,
|
||||
}, connection.defaults)
|
||||
const payload = JSON.stringify(body)
|
||||
|
||||
|
||||
@@ -36,6 +36,51 @@ interface FileStoreOptions {
|
||||
fetch?: typeof fetch
|
||||
}
|
||||
|
||||
interface SharedUpload {
|
||||
controller: AbortController
|
||||
promise: Promise<DeepSeekFileReference>
|
||||
settled: boolean
|
||||
waiters: number
|
||||
}
|
||||
|
||||
function abortReason(signal: AbortSignal): Error {
|
||||
const reason: unknown = signal.reason
|
||||
return reason instanceof Error
|
||||
? reason
|
||||
: new Error('DeepSeek file upload cancelled with a non-Error reason.', { cause: reason })
|
||||
}
|
||||
|
||||
function waitForUpload(operation: SharedUpload, signal: AbortSignal | undefined): Promise<DeepSeekFileReference> {
|
||||
signal?.throwIfAborted()
|
||||
operation.waiters += 1
|
||||
let released = false
|
||||
const release = (cancelled: boolean): void => {
|
||||
if (released) return
|
||||
released = true
|
||||
operation.waiters -= 1
|
||||
if (cancelled && operation.waiters === 0 && !operation.settled) {
|
||||
operation.controller.abort(signal === undefined ? undefined : abortReason(signal))
|
||||
}
|
||||
}
|
||||
if (signal === undefined) return operation.promise.finally(() => release(false))
|
||||
return new Promise<DeepSeekFileReference>((resolve, reject) => {
|
||||
const abort = (): void => {
|
||||
release(true)
|
||||
reject(abortReason(signal))
|
||||
}
|
||||
signal.addEventListener('abort', abort, { once: true })
|
||||
void operation.promise.then((value) => {
|
||||
signal.removeEventListener('abort', abort)
|
||||
release(false)
|
||||
resolve(value)
|
||||
}, (error: unknown) => {
|
||||
signal.removeEventListener('abort', abort)
|
||||
release(false)
|
||||
reject(error)
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
function extension(mediaType: RequestImageAttachment['mediaType']): 'png' | 'jpeg' | 'webp' | 'gif' {
|
||||
switch (mediaType) {
|
||||
case 'image/png': return 'png'
|
||||
@@ -56,7 +101,7 @@ export class DeepSeekFileStore {
|
||||
private readonly index: DeepSeekUploadIndex
|
||||
private readonly now: () => number
|
||||
private readonly fetchImpl: typeof fetch | undefined
|
||||
private readonly inflight = new Map<string, Promise<DeepSeekFileReference>>()
|
||||
private readonly inflight = new Map<string, SharedUpload>()
|
||||
|
||||
/**
|
||||
* @param options - testable index, clock, and transport boundaries.
|
||||
@@ -76,11 +121,11 @@ export class DeepSeekFileStore {
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve or upload one deterministic request image. Concurrent calls in this process share one promise.
|
||||
* Resolve or upload one deterministic request image. Concurrent calls share one upload while retaining independent waits.
|
||||
* @param version - deterministic model-request bytes and complete transformation identity.
|
||||
* @param connection - endpoint and API-key snapshot.
|
||||
* @param policy - expiry and quota-recovery policy.
|
||||
* @param signal - request cancellation.
|
||||
* @param signal - cancellation of this wait; shared transport stops when no waiter remains.
|
||||
* @returns a reusable file id and whether this call published a new upload.
|
||||
*/
|
||||
ensureUploaded(
|
||||
@@ -89,16 +134,34 @@ export class DeepSeekFileStore {
|
||||
policy: DeepSeekFilePolicy,
|
||||
signal?: AbortSignal,
|
||||
): Promise<DeepSeekFileReference> {
|
||||
signal?.throwIfAborted()
|
||||
const scope = deepSeekFileScope(connection.baseURL, connection.apiKey)
|
||||
const key = `${scope}\0${version.variantId}`
|
||||
const active = this.inflight.get(key)
|
||||
if (active !== undefined) return active
|
||||
const operation = this.ensureUploadedOnce(version, connection, policy, signal)
|
||||
this.inflight.set(key, operation)
|
||||
void operation.finally(() => {
|
||||
if (this.inflight.get(key) === operation) this.inflight.delete(key)
|
||||
let active = this.inflight.get(key)
|
||||
if (active?.controller.signal.aborted) {
|
||||
this.inflight.delete(key)
|
||||
active = undefined
|
||||
}
|
||||
if (active !== undefined) return waitForUpload(active, signal)
|
||||
const controller = new AbortController()
|
||||
const shared: SharedUpload = {
|
||||
controller,
|
||||
settled: false,
|
||||
waiters: 0,
|
||||
promise: undefined as never,
|
||||
}
|
||||
shared.promise = this.ensureUploadedOnce(version, connection, policy, controller.signal).then((value) => {
|
||||
shared.settled = true
|
||||
return value
|
||||
}, (error: unknown) => {
|
||||
shared.settled = true
|
||||
throw error
|
||||
})
|
||||
this.inflight.set(key, shared)
|
||||
void shared.promise.finally(() => {
|
||||
if (this.inflight.get(key) === shared) this.inflight.delete(key)
|
||||
}).catch(() => {})
|
||||
return operation
|
||||
return waitForUpload(shared, signal)
|
||||
}
|
||||
|
||||
private async ensureUploadedOnce(
|
||||
@@ -218,8 +281,8 @@ export class DeepSeekFileStore {
|
||||
): Promise<number> {
|
||||
const client = this.client(connection)
|
||||
let after: DeepSeekFileId | undefined
|
||||
let deleted = 0
|
||||
while (deleted < count) {
|
||||
const owned: DeepSeekFileId[] = []
|
||||
while (owned.length < count) {
|
||||
const page = await client.list({
|
||||
...after === undefined ? {} : { after },
|
||||
limit: 1_000,
|
||||
@@ -228,14 +291,14 @@ export class DeepSeekFileStore {
|
||||
})
|
||||
for (const file of page.data) {
|
||||
if (!file.filename.startsWith(OWNED_FILE_PREFIX)) continue
|
||||
await client.delete(file.id, signal)
|
||||
deleted += 1
|
||||
if (deleted === count) break
|
||||
owned.push(file.id)
|
||||
if (owned.length === count) break
|
||||
}
|
||||
if (!page.hasMore || page.lastId === undefined || page.lastId === after) break
|
||||
after = page.lastId
|
||||
}
|
||||
return deleted
|
||||
for (const fileId of owned) await client.delete(fileId, signal)
|
||||
return owned.length
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
/** OpenAI-compatible DeepSeek Files API transport. @module dsh-llm-deepseek/files-api */
|
||||
|
||||
import { LlmError } from '@deepseek-ai/dsh-llm'
|
||||
import { attributionHeaders, LlmError } from '@deepseek-ai/dsh-llm'
|
||||
import type { ImageMediaType } from '@deepseek-ai/dsh-attachment'
|
||||
import { DeepSeekFileId } from './file-id.ts'
|
||||
import type { DeepSeekFileId as DeepSeekFileIdType } from './file-id.ts'
|
||||
@@ -142,7 +142,8 @@ export class DeepSeekFilesClient {
|
||||
private async request(path: string, init: RequestInit, signal?: AbortSignal): Promise<Response> {
|
||||
let response: Response
|
||||
try {
|
||||
const headers = new Headers(init.headers)
|
||||
const headers = new Headers(attributionHeaders())
|
||||
for (const [name, value] of new Headers(init.headers)) headers.set(name, value)
|
||||
headers.set('authorization', `Bearer ${this.apiKey}`)
|
||||
response = await this.fetchImpl(`${this.baseURL}${path}`, {
|
||||
...init,
|
||||
|
||||
@@ -291,10 +291,16 @@ export function resolveAdapterOptions(config: Config, environment?: LaunchEnviro
|
||||
if (!Number.isSafeInteger(imageOffloadByteQuantum) || imageOffloadByteQuantum <= 0) {
|
||||
throw new Error('llm-deepseek: imageOffloadByteQuantum must be a positive safe integer')
|
||||
}
|
||||
if (imageOffloadByteQuantum > maxRequestFilesBytes) {
|
||||
throw new Error('llm-deepseek: imageOffloadByteQuantum must not exceed maxRequestFilesBytes')
|
||||
}
|
||||
const imageOffloadCountQuantum = config.imageOffloadCountQuantum ?? DEFAULT_IMAGE_OFFLOAD_COUNT_QUANTUM
|
||||
if (!Number.isSafeInteger(imageOffloadCountQuantum) || imageOffloadCountQuantum <= 0) {
|
||||
throw new Error('llm-deepseek: imageOffloadCountQuantum must be a positive safe integer')
|
||||
}
|
||||
if (imageOffloadCountQuantum > maxImagesPerRequest) {
|
||||
throw new Error('llm-deepseek: imageOffloadCountQuantum must not exceed maxImagesPerRequest')
|
||||
}
|
||||
const fileExpiresAfterSeconds = config.fileExpiresAfterSeconds ?? DEFAULT_FILE_EXPIRY_SECONDS
|
||||
if (!Number.isSafeInteger(fileExpiresAfterSeconds)
|
||||
|| fileExpiresAfterSeconds < 3_600
|
||||
|
||||
@@ -37,7 +37,7 @@ export interface ImageSerializationOptions {
|
||||
block: Extract<ContentBlock, { type: 'image' }>,
|
||||
location: ImageWireLocation,
|
||||
) => Promise<string>
|
||||
/** Request versions prepared before offload selection, keyed by master attachment id. */
|
||||
/** Request versions prepared for the conservatively retained masters, keyed by attachment id. */
|
||||
requestImages: ReadonlyMap<ImageAttachmentRef['attachmentId'], RequestImageAttachment>
|
||||
/** Positive bound on accumulated referenced image bytes. */
|
||||
maxRequestFilesBytes: number
|
||||
@@ -47,6 +47,8 @@ export interface ImageSerializationOptions {
|
||||
byteQuantum?: number
|
||||
/** Image-count removal step applied after the request exceeds its count bound. */
|
||||
countQuantum?: number
|
||||
/** Whether the active request exposes the region-read tool. */
|
||||
cropAvailable?: boolean
|
||||
}
|
||||
|
||||
/** Durable message and image ordinal used in provider diagnostics. */
|
||||
@@ -115,10 +117,14 @@ function assertSupportedImageRoles(messages: readonly Message[]): void {
|
||||
}
|
||||
|
||||
/** Describe the exact request preview and its model-callable coordinate system. */
|
||||
function imageHandle(version: RequestImageAttachment, precededByContent: boolean): WireTextContentPart {
|
||||
function imageHandle(
|
||||
version: RequestImageAttachment,
|
||||
precededByContent: boolean,
|
||||
cropAvailable: boolean,
|
||||
): WireTextContentPart {
|
||||
return {
|
||||
type: 'text',
|
||||
text: `${precededByContent ? '\n' : ''}${requestImagePreviewText(version)}`,
|
||||
text: `${precededByContent ? '\n' : ''}${requestImagePreviewText(version, cropAvailable)}`,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -137,7 +143,7 @@ async function imageParts(
|
||||
)
|
||||
}
|
||||
return [
|
||||
imageHandle(version, precededByContent),
|
||||
imageHandle(version, precededByContent, images.cropAvailable === true),
|
||||
{ type: 'file', file_id: await images.resolveFileId(version, block, location) },
|
||||
]
|
||||
}
|
||||
|
||||
@@ -206,6 +206,49 @@ describe('DeepSeekAdapter against a mock server', () => {
|
||||
expect(policies).toEqual([{ maxPixels: 640_000, maxBytes: 1024 * 1024 }])
|
||||
})
|
||||
|
||||
it('does not prepare an old image removed by request offload', async () => {
|
||||
const server = await mockServer([{ kind: 'sse', events: textEvents }])
|
||||
const old = { ...imageRef, attachmentId: AttachmentId(`sha256:${'c'.repeat(64)}`), bytes: 3 }
|
||||
const recent = { ...imageRef, attachmentId: AttachmentId(`sha256:${'d'.repeat(64)}`), bytes: 3 }
|
||||
const attachmentMocks = attachmentStoreOf((ref) => {
|
||||
if (ref.attachmentId === old.attachmentId) throw new Error('old image must not be read')
|
||||
return Promise.resolve(requestImage(ref))
|
||||
})
|
||||
const adapter = adapterOf({
|
||||
baseURL: server.url,
|
||||
models: [{ id: 'deepseek-v4-flash-vision-exp', inputModalities: ['text', 'image'] }],
|
||||
maxRequestFilesBytes: 4,
|
||||
imageOffloadByteQuantum: 2,
|
||||
}, attachmentMocks.store)
|
||||
|
||||
await drain(adapter.stream({
|
||||
provider: 'deepseek-official',
|
||||
model: 'deepseek-v4-flash-vision-exp',
|
||||
messages: [createUserMessage({
|
||||
content: [
|
||||
{ type: 'image', attachment: old },
|
||||
{ type: 'image', attachment: recent },
|
||||
],
|
||||
source: { kind: 'plugin', plugin: 'test' },
|
||||
})],
|
||||
}))
|
||||
|
||||
expect(attachmentMocks.readImageRequests).toHaveBeenCalledWith(
|
||||
[recent],
|
||||
{ maxPixels: 640_000, maxBytes: 1024 * 1024 },
|
||||
expect.any(AbortSignal),
|
||||
)
|
||||
const body = server.requests[0] as { messages: unknown[] }
|
||||
expect(body.messages[0]).toMatchObject({
|
||||
role: 'user',
|
||||
content: [
|
||||
{ type: 'text', text: expect.stringContaining('older images are omitted first') as string },
|
||||
{ type: 'text', text: expect.stringContaining(String(recent.attachmentId)) as string },
|
||||
{ type: 'file', file_id: 'file-api-1' },
|
||||
],
|
||||
})
|
||||
})
|
||||
|
||||
it('projects nested tool-result images with route-owned request budgets', async () => {
|
||||
const server = await mockServer([
|
||||
{ kind: 'sse', events: textEvents },
|
||||
@@ -1515,6 +1558,17 @@ describe('plugin registration and config', () => {
|
||||
},
|
||||
)
|
||||
|
||||
it('rejects offload quanta larger than their request bounds', () => {
|
||||
expect(() => resolveAdapterOptions({
|
||||
maxRequestFilesBytes: 10,
|
||||
imageOffloadByteQuantum: 11,
|
||||
})).toThrow(/imageOffloadByteQuantum must not exceed maxRequestFilesBytes/)
|
||||
expect(() => resolveAdapterOptions({
|
||||
maxImagesPerRequest: 10,
|
||||
imageOffloadCountQuantum: 11,
|
||||
})).toThrow(/imageOffloadCountQuantum must not exceed maxImagesPerRequest/)
|
||||
})
|
||||
|
||||
it.each([0, 1.5, Number.MAX_SAFE_INTEGER + 1])(
|
||||
'rejects invalid request file bound %s',
|
||||
async (maxRequestFilesBytes) => {
|
||||
|
||||
@@ -81,6 +81,63 @@ describe('DeepSeekFileStore', () => {
|
||||
expect(remote.uploads()).toBe(1)
|
||||
})
|
||||
|
||||
it('keeps a shared upload alive while another waiter remains', async () => {
|
||||
const dir = await mkdtemp(join(tmpdir(), 'dsh-file-store-'))
|
||||
const index = new DeepSeekUploadIndex(join(dir, 'index.json'))
|
||||
let complete: ((response: Response) => void) | undefined
|
||||
let uploadSignal: AbortSignal | undefined
|
||||
const fetchImpl = vi.fn((_url: string | URL | Request, init?: RequestInit) => {
|
||||
uploadSignal = init?.signal ?? undefined
|
||||
return new Promise<Response>((resolve, reject) => {
|
||||
complete = resolve
|
||||
uploadSignal?.addEventListener('abort', () => reject(uploadSignal?.reason), { once: true })
|
||||
})
|
||||
}) as typeof fetch
|
||||
const store = new DeepSeekFileStore({ index, now: () => NOW, fetch: fetchImpl })
|
||||
const controller = new AbortController()
|
||||
|
||||
const cancelled = store.ensureUploaded(VERSION, CONNECTION, POLICY, controller.signal)
|
||||
const completed = store.ensureUploaded(VERSION, CONNECTION, POLICY)
|
||||
await vi.waitFor(() => expect(fetchImpl).toHaveBeenCalledTimes(1))
|
||||
const reason = new Error('cancel one upload waiter')
|
||||
controller.abort(reason)
|
||||
|
||||
await expect(cancelled).rejects.toBe(reason)
|
||||
expect(uploadSignal?.aborted).toBe(false)
|
||||
complete?.(new Response(JSON.stringify({
|
||||
id: 'file-api-shared',
|
||||
object: 'file',
|
||||
bytes: 3,
|
||||
created_at: NOW / 1_000,
|
||||
filename: `dsh-${'a'.repeat(16)}-${'b'.repeat(8)}.png`,
|
||||
purpose: 'user_data',
|
||||
expires_at: NOW / 1_000 + POLICY.expiresAfterSeconds,
|
||||
}), { status: 200 }))
|
||||
await expect(completed).resolves.toMatchObject({ record: { fileId: 'file-api-shared' } })
|
||||
})
|
||||
|
||||
it('aborts the shared upload after its only waiter cancels', async () => {
|
||||
const dir = await mkdtemp(join(tmpdir(), 'dsh-file-store-'))
|
||||
const index = new DeepSeekUploadIndex(join(dir, 'index.json'))
|
||||
let uploadSignal: AbortSignal | undefined
|
||||
const fetchImpl = vi.fn((_url: string | URL | Request, init?: RequestInit) => {
|
||||
uploadSignal = init?.signal ?? undefined
|
||||
return new Promise<Response>((_resolve, reject) => {
|
||||
uploadSignal?.addEventListener('abort', () => reject(uploadSignal?.reason), { once: true })
|
||||
})
|
||||
}) as typeof fetch
|
||||
const store = new DeepSeekFileStore({ index, now: () => NOW, fetch: fetchImpl })
|
||||
const controller = new AbortController()
|
||||
const upload = store.ensureUploaded(VERSION, CONNECTION, POLICY, controller.signal)
|
||||
await vi.waitFor(() => expect(fetchImpl).toHaveBeenCalledTimes(1))
|
||||
|
||||
const reason = new Error('cancel only upload waiter')
|
||||
controller.abort(reason)
|
||||
|
||||
await expect(upload).rejects.toBe(reason)
|
||||
expect(uploadSignal?.reason).toBe(reason)
|
||||
})
|
||||
|
||||
it('does not persist an upload whose response is missing and retries on the next request', async () => {
|
||||
const dir = await mkdtemp(join(tmpdir(), 'dsh-file-store-'))
|
||||
const index = new DeepSeekUploadIndex(join(dir, 'index.json'))
|
||||
@@ -132,4 +189,42 @@ describe('DeepSeekFileStore', () => {
|
||||
await expect(store.release(VERSION, CONNECTION, POLICY)).resolves.toBe(false)
|
||||
expect(remote.fetchImpl).toHaveBeenCalledTimes(2)
|
||||
})
|
||||
|
||||
it('finishes pagination before deleting cursor files during quota recovery', async () => {
|
||||
const dir = await mkdtemp(join(tmpdir(), 'dsh-file-store-'))
|
||||
const deleted = new Set<string>()
|
||||
const fetchImpl = vi.fn(async (input: string | URL | Request, init?: RequestInit) => {
|
||||
const target = new URL(requestUrl(input))
|
||||
if (init?.method === 'DELETE') {
|
||||
const id = target.pathname.split('/').at(-1) ?? ''
|
||||
deleted.add(id)
|
||||
return new Response(JSON.stringify({ id, object: 'file', deleted: true }), { status: 200 })
|
||||
}
|
||||
const after = target.searchParams.get('after')
|
||||
if (after !== null && deleted.has(after)) throw new Error('deleted cursor cannot be reused')
|
||||
const id = after === null ? 'file-api-oldest' : 'file-api-next'
|
||||
return new Response(JSON.stringify({
|
||||
object: 'list',
|
||||
data: [{
|
||||
id,
|
||||
object: 'file',
|
||||
bytes: 3,
|
||||
created_at: NOW / 1_000,
|
||||
filename: `dsh-${id}.png`,
|
||||
purpose: 'user_data',
|
||||
}],
|
||||
first_id: id,
|
||||
last_id: id,
|
||||
has_more: after === null,
|
||||
}), { status: 200 })
|
||||
}) as typeof fetch
|
||||
const store = new DeepSeekFileStore({
|
||||
index: new DeepSeekUploadIndex(join(dir, 'index.json')),
|
||||
now: () => NOW,
|
||||
fetch: fetchImpl,
|
||||
})
|
||||
|
||||
await expect(store.reclaimOldestOwned(CONNECTION, 2)).resolves.toBe(2)
|
||||
expect([...deleted]).toEqual(['file-api-oldest', 'file-api-next'])
|
||||
})
|
||||
})
|
||||
|
||||
@@ -1,6 +1,12 @@
|
||||
import { describe, expect, it, vi } from 'vitest'
|
||||
import { userAgent } from '@deepseek-ai/dsh-llm'
|
||||
import { DeepSeekFileId } from '../src/file-id.ts'
|
||||
import { DeepSeekFilesClient, isFilesQuotaError } from '../src/files-api.ts'
|
||||
import {
|
||||
DeepSeekFilesClient,
|
||||
DeepSeekFilesError,
|
||||
isFilesQuotaError,
|
||||
MAX_FILE_UPLOAD_BYTES,
|
||||
} from '../src/files-api.ts'
|
||||
|
||||
function requestUrl(input: string | URL | Request): string {
|
||||
if (typeof input === 'string') return input
|
||||
@@ -25,7 +31,9 @@ describe('DeepSeekFilesClient', () => {
|
||||
const fetchImpl = vi.fn(async (url: string | URL | Request, init?: RequestInit) => {
|
||||
expect(requestUrl(url)).toBe('https://api.deepseek.com/files')
|
||||
expect(init?.method).toBe('POST')
|
||||
expect(new Headers(init?.headers).get('authorization')).toBe('Bearer key')
|
||||
const headers = new Headers(init?.headers)
|
||||
expect(headers.get('authorization')).toBe('Bearer key')
|
||||
expect(headers.get('user-agent')).toBe(userAgent())
|
||||
const form = init?.body
|
||||
expect(form).toBeInstanceOf(FormData)
|
||||
if (!(form instanceof FormData)) throw new Error('expected multipart body')
|
||||
@@ -69,7 +77,7 @@ describe('DeepSeekFilesClient', () => {
|
||||
}) as typeof fetch
|
||||
const client = new DeepSeekFilesClient({ baseURL: 'https://api.deepseek.com', apiKey: 'key', fetch: fetchImpl })
|
||||
|
||||
await expect(client.list({ limit: 20, order: 'desc' })).resolves.toMatchObject({
|
||||
await expect(client.list({ after: DeepSeekFileId('file-api-before'), limit: 20, order: 'desc' })).resolves.toMatchObject({
|
||||
data: [{ id: 'file-api-one' }], firstId: 'file-api-one', lastId: 'file-api-one', hasMore: false,
|
||||
})
|
||||
await expect(client.retrieve(DeepSeekFileId('file-api-one'))).resolves.toMatchObject({ id: 'file-api-one' })
|
||||
@@ -98,5 +106,155 @@ describe('DeepSeekFilesClient', () => {
|
||||
data: Uint8Array.of(1), mediaType: 'image/png', filename: 'image.png', expiresAfterSeconds: 3_600,
|
||||
}).catch((caught: unknown) => caught)
|
||||
expect(isFilesQuotaError(error)).toBe(true)
|
||||
expect(isFilesQuotaError(new Error('storage quota'))).toBe(false)
|
||||
})
|
||||
|
||||
it.each([
|
||||
[401, 'AUTH'],
|
||||
[403, 'AUTH'],
|
||||
[429, 'RATE_LIMIT'],
|
||||
[500, 'SERVER'],
|
||||
[400, 'FILES_API'],
|
||||
] as const)('classifies HTTP %i Files failures as %s', async (status, code) => {
|
||||
const client = new DeepSeekFilesClient({
|
||||
baseURL: 'https://api.deepseek.com',
|
||||
apiKey: 'key',
|
||||
fetch: vi.fn(() => Promise.resolve(new Response('not-json', { status }))) as typeof fetch,
|
||||
})
|
||||
await expect(client.retrieve(DeepSeekFileId('missing'))).rejects.toMatchObject({
|
||||
name: 'DeepSeekFilesError',
|
||||
code,
|
||||
detail: '',
|
||||
})
|
||||
})
|
||||
|
||||
it.each([
|
||||
null,
|
||||
[],
|
||||
{},
|
||||
{ error: null },
|
||||
{ error: [] },
|
||||
{ error: { message: 1, type: 2, code: 3 } },
|
||||
])('falls back to the HTTP status for an unstructured provider error %#', async (body) => {
|
||||
const client = new DeepSeekFilesClient({
|
||||
baseURL: 'https://api.deepseek.com',
|
||||
apiKey: 'key',
|
||||
fetch: vi.fn(() => Promise.resolve(new Response(JSON.stringify(body), { status: 400 }))) as typeof fetch,
|
||||
})
|
||||
const error = await client.retrieve(DeepSeekFileId('missing')).catch((caught: unknown) => caught)
|
||||
expect(error).toBeInstanceOf(DeepSeekFilesError)
|
||||
expect(error).toMatchObject({ message: 'DeepSeek Files API error (HTTP 400)', detail: '' })
|
||||
})
|
||||
|
||||
it('wraps transport failures but preserves an aborted request reason', async () => {
|
||||
const transport = new Error('socket closed')
|
||||
const client = new DeepSeekFilesClient({
|
||||
baseURL: 'https://api.deepseek.com',
|
||||
apiKey: 'key',
|
||||
fetch: vi.fn(() => Promise.reject(transport)) as typeof fetch,
|
||||
})
|
||||
await expect(client.retrieve(DeepSeekFileId('one'))).rejects.toMatchObject({
|
||||
code: 'TRANSPORT',
|
||||
cause: transport,
|
||||
})
|
||||
|
||||
const controller = new AbortController()
|
||||
const reason = new Error('cancelled')
|
||||
controller.abort(reason)
|
||||
await expect(client.retrieve(DeepSeekFileId('one'), controller.signal)).rejects.toBe(transport)
|
||||
})
|
||||
|
||||
it.each([
|
||||
null,
|
||||
[],
|
||||
file({ id: 1 }),
|
||||
file({ id: '' }),
|
||||
file({ object: 'wrong' }),
|
||||
file({ bytes: 1.5 }),
|
||||
file({ bytes: -1 }),
|
||||
file({ created_at: 1.5 }),
|
||||
file({ created_at: -1 }),
|
||||
file({ filename: 1 }),
|
||||
file({ filename: '' }),
|
||||
file({ purpose: 'assistants' }),
|
||||
file({ expires_at: 1.5 }),
|
||||
file({ expires_at: -1 }),
|
||||
])('rejects an invalid file object %#', async (body) => {
|
||||
const client = new DeepSeekFilesClient({
|
||||
baseURL: 'https://api.deepseek.com',
|
||||
apiKey: 'key',
|
||||
fetch: vi.fn(() => Promise.resolve(new Response(JSON.stringify(body), { status: 200 }))) as typeof fetch,
|
||||
})
|
||||
await expect(client.retrieve(DeepSeekFileId('one'))).rejects.toMatchObject({ code: 'INVALID_RESPONSE' })
|
||||
})
|
||||
|
||||
it.each([
|
||||
3_599,
|
||||
2_592_001,
|
||||
3_600.5,
|
||||
])('refuses invalid file expiry %s before transport', async (expiresAfterSeconds) => {
|
||||
const fetchImpl = vi.fn() as typeof fetch
|
||||
const client = new DeepSeekFilesClient({ baseURL: 'https://api.deepseek.com', apiKey: 'key', fetch: fetchImpl })
|
||||
await expect(client.upload({
|
||||
data: Uint8Array.of(1), mediaType: 'image/png', filename: 'image.png', expiresAfterSeconds,
|
||||
})).rejects.toMatchObject({ code: 'INVALID_REQUEST' })
|
||||
expect(fetchImpl).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('refuses a file larger than the upload limit before transport', async () => {
|
||||
const fetchImpl = vi.fn() as typeof fetch
|
||||
const client = new DeepSeekFilesClient({ baseURL: 'https://api.deepseek.com', apiKey: 'key', fetch: fetchImpl })
|
||||
const data = { byteLength: MAX_FILE_UPLOAD_BYTES + 1 } as Uint8Array
|
||||
await expect(client.upload({
|
||||
data, mediaType: 'image/png', filename: 'image.png', expiresAfterSeconds: 3_600,
|
||||
})).rejects.toMatchObject({ code: 'INVALID_REQUEST' })
|
||||
expect(fetchImpl).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it.each([
|
||||
null,
|
||||
[],
|
||||
{},
|
||||
{ object: 'wrong', data: [], has_more: false },
|
||||
{ object: 'list', data: null, has_more: false },
|
||||
{ object: 'list', data: [], has_more: 0 },
|
||||
{ object: 'list', data: [], has_more: false, first_id: 1 },
|
||||
{ object: 'list', data: [], has_more: false, last_id: 1 },
|
||||
])('rejects an invalid list response %#', async (body) => {
|
||||
const client = new DeepSeekFilesClient({
|
||||
baseURL: 'https://api.deepseek.com',
|
||||
apiKey: 'key',
|
||||
fetch: vi.fn(() => Promise.resolve(new Response(JSON.stringify(body), { status: 200 }))) as typeof fetch,
|
||||
})
|
||||
await expect(client.list()).rejects.toMatchObject({ code: 'INVALID_RESPONSE' })
|
||||
})
|
||||
|
||||
it('accepts a list without cursors and uses the global fetch default', async () => {
|
||||
const fetchImpl = vi.fn(() => Promise.resolve(new Response(JSON.stringify({
|
||||
object: 'list', data: [], has_more: false,
|
||||
}), { status: 200 })))
|
||||
vi.stubGlobal('fetch', fetchImpl)
|
||||
try {
|
||||
const client = new DeepSeekFilesClient({ baseURL: 'https://api.deepseek.com///', apiKey: 'key' })
|
||||
await expect(client.list()).resolves.toEqual({ data: [], hasMore: false })
|
||||
} finally {
|
||||
vi.unstubAllGlobals()
|
||||
}
|
||||
})
|
||||
|
||||
it.each([
|
||||
null,
|
||||
[],
|
||||
{},
|
||||
{ id: 'wrong', object: 'file', deleted: true },
|
||||
{ id: 'file-api-one', object: 'wrong', deleted: true },
|
||||
{ id: 'file-api-one', object: 'file', deleted: false },
|
||||
])('rejects an invalid delete response %#', async (body) => {
|
||||
const client = new DeepSeekFilesClient({
|
||||
baseURL: 'https://api.deepseek.com',
|
||||
apiKey: 'key',
|
||||
fetch: vi.fn(() => Promise.resolve(new Response(JSON.stringify(body), { status: 200 }))) as typeof fetch,
|
||||
})
|
||||
await expect(client.delete(DeepSeekFileId('file-api-one'))).rejects.toMatchObject({ code: 'INVALID_RESPONSE' })
|
||||
})
|
||||
})
|
||||
|
||||
@@ -60,6 +60,7 @@ function imageOptions(
|
||||
resolveFileId,
|
||||
requestImages: new Map(refs.map(ref => [ref.attachmentId, requestVersion(ref)])),
|
||||
maxRequestFilesBytes,
|
||||
cropAvailable: true,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -378,6 +379,26 @@ describe('image serialization', () => {
|
||||
}])
|
||||
})
|
||||
|
||||
it('does not advertise region reads when the request omits that tool', async () => {
|
||||
const ref = imageRef()
|
||||
const images = { ...imageOptions([ref]), cropAvailable: false }
|
||||
const wire = await serializeRequestWithImages(request({
|
||||
model: 'deepseek-v4-flash-vision-exp',
|
||||
messages: [createUserMessage({
|
||||
content: [{ type: 'image', attachment: ref }],
|
||||
source: { kind: 'plugin', plugin: 'test' },
|
||||
})],
|
||||
}), images)
|
||||
|
||||
expect(wire.messages[0]).toMatchObject({
|
||||
role: 'user',
|
||||
content: [
|
||||
{ type: 'text', text: `Image ${ref.attachmentId}; preview 1x1px.` },
|
||||
{ type: 'file', file_id: 'file-api-image' },
|
||||
],
|
||||
})
|
||||
})
|
||||
|
||||
it('keeps tool content textual and groups consecutive tool-result images afterward', async () => {
|
||||
const messages = [
|
||||
createUserMessage({
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { mkdtemp, readFile, writeFile } from 'node:fs/promises'
|
||||
import { mkdir, mkdtemp, readFile, writeFile } from 'node:fs/promises'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import { describe, expect, it } from 'vitest'
|
||||
@@ -10,6 +10,11 @@ const ATTACHMENT = AttachmentId(`sha256:${'a'.repeat(64)}`)
|
||||
const VARIANT = ImageVariantId(`sha256:${'b'.repeat(64)}`)
|
||||
|
||||
describe('DeepSeekUploadIndex', () => {
|
||||
it('normalizes trailing endpoint slashes in the credential scope', () => {
|
||||
expect(deepSeekFileScope('https://api.deepseek.com///', 'key'))
|
||||
.toBe(deepSeekFileScope('https://api.deepseek.com', 'key'))
|
||||
})
|
||||
|
||||
it('isolates API-key namespaces and reuses only records above the refresh margin', async () => {
|
||||
const dir = await mkdtemp(join(tmpdir(), 'dsh-upload-index-'))
|
||||
const index = new DeepSeekUploadIndex(join(dir, 'index.json'))
|
||||
@@ -70,4 +75,106 @@ describe('DeepSeekUploadIndex', () => {
|
||||
await expect(index.get(scope, VARIANT, 1, 1)).resolves.toEqual(record)
|
||||
expect(JSON.parse(await readFile(path, 'utf8'))).toMatchObject({ formatVersion: 2 })
|
||||
})
|
||||
|
||||
it.each([
|
||||
'null',
|
||||
'[]',
|
||||
'{}',
|
||||
'{"formatVersion":1,"records":[]}',
|
||||
'{"formatVersion":2,"records":null}',
|
||||
'{"formatVersion":2,"records":[null]}',
|
||||
'{"formatVersion":2,"records":[[]]}',
|
||||
'{"formatVersion":2,"records":[{}]}',
|
||||
`{"formatVersion":2,"records":[${JSON.stringify({
|
||||
scope: 'x'.repeat(64), masterAttachmentId: ATTACHMENT, variantId: VARIANT,
|
||||
fileId: 'file-api-one', bytes: 3, createdAt: 1, expiresAt: 10_000,
|
||||
})}]}`,
|
||||
`{"formatVersion":2,"records":[${JSON.stringify({
|
||||
scope: 'a'.repeat(64), masterAttachmentId: 'wrong', variantId: VARIANT,
|
||||
fileId: 'file-api-one', bytes: 3, createdAt: 1, expiresAt: 10_000,
|
||||
})}]}`,
|
||||
`{"formatVersion":2,"records":[${JSON.stringify({
|
||||
scope: 'a'.repeat(64), masterAttachmentId: ATTACHMENT, variantId: 'wrong',
|
||||
fileId: 'file-api-one', bytes: 3, createdAt: 1, expiresAt: 10_000,
|
||||
})}]}`,
|
||||
`{"formatVersion":2,"records":[${JSON.stringify({
|
||||
scope: 'a'.repeat(64), masterAttachmentId: ATTACHMENT, variantId: VARIANT,
|
||||
fileId: '', bytes: 3, createdAt: 1, expiresAt: 10_000,
|
||||
})}]}`,
|
||||
`{"formatVersion":2,"records":[${JSON.stringify({
|
||||
scope: 'a'.repeat(64), masterAttachmentId: ATTACHMENT, variantId: VARIANT,
|
||||
fileId: 'file-api-one', bytes: -1, createdAt: 1, expiresAt: 10_000,
|
||||
})}]}`,
|
||||
`{"formatVersion":2,"records":[${JSON.stringify({
|
||||
scope: 'a'.repeat(64), masterAttachmentId: ATTACHMENT, variantId: VARIANT,
|
||||
fileId: 'file-api-one', bytes: 1.5, createdAt: 1, expiresAt: 10_000,
|
||||
})}]}`,
|
||||
`{"formatVersion":2,"records":[${JSON.stringify({
|
||||
scope: 'a'.repeat(64), masterAttachmentId: ATTACHMENT, variantId: VARIANT,
|
||||
fileId: 'file-api-one', bytes: 3, createdAt: -1, expiresAt: 10_000,
|
||||
})}]}`,
|
||||
`{"formatVersion":2,"records":[${JSON.stringify({
|
||||
scope: 'a'.repeat(64), masterAttachmentId: ATTACHMENT, variantId: VARIANT,
|
||||
fileId: 'file-api-one', bytes: 3, createdAt: 1.5, expiresAt: 10_000,
|
||||
})}]}`,
|
||||
`{"formatVersion":2,"records":[${JSON.stringify({
|
||||
scope: 'a'.repeat(64), masterAttachmentId: ATTACHMENT, variantId: VARIANT,
|
||||
fileId: 'file-api-one', bytes: 3, createdAt: 1, expiresAt: -1,
|
||||
})}]}`,
|
||||
`{"formatVersion":2,"records":[${JSON.stringify({
|
||||
scope: 'a'.repeat(64), masterAttachmentId: ATTACHMENT, variantId: VARIANT,
|
||||
fileId: 'file-api-one', bytes: 3, createdAt: 1, expiresAt: 1.5,
|
||||
})}]}`,
|
||||
])('treats an invalid persisted index as empty %#', async (text) => {
|
||||
const dir = await mkdtemp(join(tmpdir(), 'dsh-upload-index-'))
|
||||
const path = join(dir, 'index.json')
|
||||
await writeFile(path, text, 'utf8')
|
||||
const index = new DeepSeekUploadIndex(path)
|
||||
await expect(index.get(
|
||||
deepSeekFileScope('https://api.deepseek.com', 'key'), VARIANT, 1, 1,
|
||||
)).resolves.toBeUndefined()
|
||||
})
|
||||
|
||||
it('rejects duplicate persisted mappings as a corrupt cache', async () => {
|
||||
const dir = await mkdtemp(join(tmpdir(), 'dsh-upload-index-'))
|
||||
const path = join(dir, 'index.json')
|
||||
const scope = deepSeekFileScope('https://api.deepseek.com', 'key')
|
||||
const record = {
|
||||
scope, masterAttachmentId: ATTACHMENT, variantId: VARIANT,
|
||||
fileId: DeepSeekFileId('file-api-one'), bytes: 3, createdAt: 1, expiresAt: 10_000,
|
||||
}
|
||||
await writeFile(path, JSON.stringify({ formatVersion: 2, records: [record, record] }), 'utf8')
|
||||
const index = new DeepSeekUploadIndex(path)
|
||||
await expect(index.get(scope, VARIANT, 1, 1)).resolves.toBeUndefined()
|
||||
})
|
||||
|
||||
it('drops expired records on commit and clears only the selected namespace', async () => {
|
||||
const dir = await mkdtemp(join(tmpdir(), 'dsh-upload-index-'))
|
||||
const index = new DeepSeekUploadIndex(join(dir, 'index.json'))
|
||||
const first = deepSeekFileScope('https://api.deepseek.com', 'first')
|
||||
const second = deepSeekFileScope('https://api.deepseek.com', 'second')
|
||||
const expired = {
|
||||
scope: first, masterAttachmentId: ATTACHMENT, variantId: VARIANT,
|
||||
fileId: DeepSeekFileId('file-api-expired'), bytes: 3, createdAt: 1, expiresAt: 2,
|
||||
}
|
||||
const live = {
|
||||
...expired, scope: second, fileId: DeepSeekFileId('file-api-live'), expiresAt: 10_000,
|
||||
}
|
||||
await index.commit(expired, 0, 0)
|
||||
await index.commit(live, 3, 1)
|
||||
await index.clear(first)
|
||||
await index.clear(second)
|
||||
await expect(index.get(second, VARIANT, 3, 1)).resolves.toBeUndefined()
|
||||
await index.clear(second)
|
||||
})
|
||||
|
||||
it('propagates non-cache filesystem read failures', async () => {
|
||||
const dir = await mkdtemp(join(tmpdir(), 'dsh-upload-index-'))
|
||||
const path = join(dir, 'directory')
|
||||
await mkdir(path)
|
||||
const index = new DeepSeekUploadIndex(path)
|
||||
await expect(index.get(
|
||||
deepSeekFileScope('https://api.deepseek.com', 'key'), VARIANT, 1, 1,
|
||||
)).rejects.toBeInstanceOf(Error)
|
||||
})
|
||||
})
|
||||
|
||||
@@ -123,7 +123,7 @@ A model that carries reasoning metadata — from the installed catalog or from i
|
||||
|
||||
A model **without** that metadata — a hand-declared one whose entry declares no `reasoningEfforts`, and a catalog model pi-ai marks as non-reasoning — exposes no `reasoning` at all. pi-ai reports such a model as supporting the single level `off`, but `off` is translated to *omitting* the reasoning option, which is byte-for-byte the request that naming no effort already produces: selecting it could not disable anything, so a provider whose own default is to think would keep thinking with `off` shown as selected. Reporting the capability as unavailable leaves a surface offering the provider's default and nothing that misrepresents it. The profile `reasoning` value, including `off`, is the deployment default when configured; omitting it preserves the provider default. Per-request `GenerateOptions.reasoningEffort` takes precedence, and a level absent from the exact model capability fails the REQUEST with `UNSUPPORTED_REASONING_EFFORT` before network I/O instead of being clamped. Describing a model never fails that way: the models under one provider disagree about which levels they accept, so `resolveModel` reports a profile level the exact model cannot take as no default at all rather than throwing. A throw there would take the whole provider out of every model catalog built over it — one mis-set profile field hiding even the models that do support the level — so a bad configuration surfaces where it is acted on, not where it is described. pi-ai's common stream options represent `off` by omitting `reasoning`.
|
||||
|
||||
Supported profile fields are `apiKeyEnv`, `displayName`, `api`, `baseURL`, `models`, `modelOverrides`, `compat`, `defaultContextWindow`, `defaultMaxTokens`, `defaultInput`, `headers`, `reasoning`, `thinkingBudgets`, `cacheRetention`, `transport`, `timeoutMs`, `websocketConnectTimeoutMs`, `streamIdleTimeoutMs`, `maxRequestImageBytes`, `requestImagePixelBudget`, `requestImageMaxBytes`, and `retryPolicy`. Each resolved profile retry policy is captured with that provider route; omission uses the shared bounded normal default of five retries. The stream-idle interval is a positive finite Node timer delay, defaults to five minutes, and covers only an outstanding provider read, not consumer think time. Every image route first derives a deterministic request version from the provider-independent master under `requestImagePixelBudget` (default 2048 by 2048 total pixels) and `requestImageMaxBytes` (default 1MiB raw bytes). The same version feeds inline base64, and its stable descriptor exposes the attachment id and actual preview dimensions. `maxRequestImageBytes` then bounds the accumulated base64 length (default 20MiB): the oldest request versions are replaced by a fixed text placeholder until the request fits. Harness app attribution wins a conflicting configured header name.
|
||||
Supported profile fields are `apiKeyEnv`, `displayName`, `api`, `baseURL`, `models`, `modelOverrides`, `compat`, `defaultContextWindow`, `defaultMaxTokens`, `defaultInput`, `headers`, `reasoning`, `thinkingBudgets`, `cacheRetention`, `transport`, `timeoutMs`, `websocketConnectTimeoutMs`, `streamIdleTimeoutMs`, `maxRequestImageBytes`, `requestImagePixelBudget`, `requestImageMaxBytes`, and `retryPolicy`. Each resolved profile retry policy is captured with that provider route; omission uses the shared bounded normal default of five retries. The stream-idle interval is a positive finite Node timer delay, defaults to five minutes, and covers only an outstanding provider read, not consumer think time. Every image route derives a deterministic request version from the provider-independent master under `requestImagePixelBudget` (default 2048 by 2048 total pixels) and `requestImageMaxBytes` (default 1MiB raw bytes). Before reading masters, `maxRequestImageBytes` applies to conservative request-version upper bounds and replaces the oldest over-budget images with fixed text; exact base64 lengths are checked again after retained versions are generated. The 20MiB default can retain fifteen maximum-size 1MiB versions after base64 expansion while leaving request-body headroom. The same version feeds inline base64, and its stable descriptor exposes the attachment id and actual preview dimensions. Harness app attribution wins a conflicting configured header name.
|
||||
|
||||
The adapter forces pi-ai's SDK `maxRetries` to zero so one `stream()` call makes one provider request. The removed profile fields `maxRetries` and `maxRetryDelayMs` fail load instead of silently multiplying or hiding the separately composed agent-level retry budget. Idle expiry aborts the SDK's stable request signal and surfaces `TIMEOUT`; an earlier caller abort remains `ABORTED`.
|
||||
|
||||
@@ -173,7 +173,7 @@ pi-ai installs several provider SDKs and lazy-loads the one selected by the cata
|
||||
|
||||
#### What the model sees
|
||||
|
||||
The selected catalog model receives `GenerateOptions.system`, history, tools, and sampling fields supported by pi-ai's common streaming API. Each retained image is preceded by stable text naming its complete attachment id, actual request dimensions, and `read_image_region` preview coordinates. When accumulated base64 image payload exceeds the route's `maxRequestImageBytes`, each offloaded image (oldest first) is replaced by fixed text that tells the model to read the file again when a path is available or ask the user to attach it again. Provider-native replay metadata is restored only when the adapter validates it for the historical content.
|
||||
The selected catalog model receives `GenerateOptions.system`, history, tools, and sampling fields supported by pi-ai's common streaming API. Each retained image is preceded by stable text naming its complete attachment id and actual request dimensions. The text includes `read_image_region` preview coordinates only when that tool is present in the request. When accumulated base64 image payload exceeds the route's `maxRequestImageBytes`, each offloaded image (oldest first) is replaced by fixed text that tells the model to read the file again when a path is available or ask the user to attach it again. Offloaded masters are not read or transformed. Provider-native replay metadata is restored only when the adapter validates it for the historical content.
|
||||
|
||||
#### Token effect
|
||||
|
||||
|
||||
@@ -124,7 +124,7 @@ pi-ai 依据提供方 id 与 baseURL 决定每个请求的形状:系统提示
|
||||
|
||||
**没有**这份元数据的模型——条目未声明 `reasoningEfforts` 的手工声明模型,以及 pi-ai 标记为不具备推理能力的 catalog 模型——完全不公开 `reasoning`。pi-ai 会把这类模型报告为只支持 `off` 一档,但 `off` 会被翻译成*省略* reasoning 选项,而那与「不点名任何档位」产出的请求逐字节相同:选它关不掉任何东西,于是自身默认就在思考的提供方,会在界面显示 `off` 被选中的同时继续思考。把该能力报告为不可用,界面就只剩提供方默认这一项,不会再出现自相矛盾的控件。配置 profile 的 `reasoning` 值(包括 `off`)在存在时是部署默认值;省略它会保留提供方默认值。每次请求的 `GenerateOptions.reasoningEffort` 优先;未出现在确切模型能力中的档位会让**请求**在网络 I/O 前以 `UNSUPPORTED_REASONING_EFFORT` 失败,而不会被自动调整。**描述**一个模型则从不这样失败:同一提供方下各模型接受的档位并不一致,因此 `resolveModel` 对该模型拿不下的 profile 档位报告为「没有默认值」,而不是抛错。在那里抛错会让整个提供方从任何基于它构建的模型目录中消失——一个配错的 profile 字段连支持该档位的模型也一并藏起来——所以坏配置暴露在被执行处,而不是被描述处。pi-ai 的通用流选项通过省略 `reasoning` 表示 `off`。
|
||||
|
||||
受支持的 profile 字段是 `apiKeyEnv`、`displayName`、`api`、`baseURL`、`models`、`modelOverrides`、`compat`、`defaultContextWindow`、`defaultMaxTokens`、`defaultInput`、`headers`、`reasoning`、`thinkingBudgets`、`cacheRetention`、`transport`、`timeoutMs`、`websocketConnectTimeoutMs`、`streamIdleTimeoutMs`、`maxRequestImageBytes`、`requestImagePixelBudget`、`requestImageMaxBytes` 和 `retryPolicy`。每条 profile 解析后的重试策略会随该提供方路由一同捕获;省略时使用共享的有界 normal 默认值并重试五次。流空闲间隔必须是正的有限 Node 定时器延迟,默认为五分钟,且只覆盖未完成提供方读取,不包括消费方思考时间。每条图片路由先从提供方无关的主版本派生确定性请求版本,受 `requestImagePixelBudget`(默认总像素 2048×2048)和 `requestImageMaxBytes`(默认原始字节 1MiB)约束。同一版本用于内联 base64,其稳定描述会公开附件 ID 和实际预览尺寸。`maxRequestImageBytes` 再限制累计 base64 长度(默认 20MiB);超出时从最旧请求版本开始替换为固定文本占位,直到请求可容纳。若已配置标头中有同名项,则以 Harness 应用归因为准。
|
||||
受支持的 profile 字段是 `apiKeyEnv`、`displayName`、`api`、`baseURL`、`models`、`modelOverrides`、`compat`、`defaultContextWindow`、`defaultMaxTokens`、`defaultInput`、`headers`、`reasoning`、`thinkingBudgets`、`cacheRetention`、`transport`、`timeoutMs`、`websocketConnectTimeoutMs`、`streamIdleTimeoutMs`、`maxRequestImageBytes`、`requestImagePixelBudget`、`requestImageMaxBytes` 和 `retryPolicy`。每条 profile 解析后的重试策略会随该提供方路由一同捕获;省略时使用共享的有界 normal 默认值并重试五次。流空闲间隔必须是正的有限 Node 定时器延迟,默认为五分钟,且只覆盖未完成提供方读取,不包括消费方思考时间。每条图片路由从提供方无关的主版本派生确定性请求版本,受 `requestImagePixelBudget`(默认总像素 2048×2048)和 `requestImageMaxBytes`(默认原始字节 1MiB)约束。读取主版本前,`maxRequestImageBytes` 先按请求版本的保守上界替换超预算的最旧图片;保留版本生成后再用确切 base64 长度检查。20MiB 默认值可保留十五个按 1MiB 上限生成的请求版本,并为请求正文留下余量。同一版本用于内联 base64,其稳定描述会公开附件 ID 和实际预览尺寸。若已配置标头中有同名项,则以 Harness 应用归因为准。
|
||||
|
||||
适配器强制 pi-ai SDK `maxRetries` 为零,因此一次 `stream()` 调用只会发起一次提供方请求。已移除 profile 字段 `maxRetries` 和 `maxRetryDelayMs` 会使加载失败,而不是静默倍增或隐藏单独组合的 agent(智能体)级重试预算。空闲超时会 abort SDK 的稳定请求信号,并以 `TIMEOUT` 呈现;较早的调用方 abort 仍为 `ABORTED`。
|
||||
|
||||
@@ -174,7 +174,7 @@ pi-ai 会安装多个提供方 SDK,并延迟加载 catalog 模型所选的 SDK
|
||||
|
||||
#### 模型看到的内容
|
||||
|
||||
所选 catalog 模型会收到 `GenerateOptions.system`、历史、工具,以及 pi-ai 通用流式 API 支持的采样字段。每张保留图片前都有稳定文本,写明完整附件 ID、实际请求尺寸和 `read_image_region` 使用的预览坐标。请求累积的 base64 图片载荷超过路由的 `maxRequestImageBytes` 时,被 offload 的图片会从最老开始替换为固定文本,要求模型在有路径时重新读取文件,否则请用户重新附上图片。只有当适配器验证提供方原生回放元数据与历史内容匹配时,才会恢复这些元数据。
|
||||
所选 catalog 模型会收到 `GenerateOptions.system`、历史、工具,以及 pi-ai 通用流式 API 支持的采样字段。每张保留图片前都有稳定文本,写明完整附件 ID 和实际请求尺寸。只有请求包含 `read_image_region` 时,文本才会提供该工具使用的预览坐标。请求累积的 base64 图片载荷超过路由的 `maxRequestImageBytes` 时,被 offload 的图片会从最老开始替换为固定文本,要求模型在有路径时重新读取文件,否则请用户重新附上图片。系统不会读取或转换被 offload 的主版本。只有当适配器验证提供方原生回放元数据与历史内容匹配时,才会恢复这些元数据。
|
||||
|
||||
#### Token 影响
|
||||
|
||||
|
||||
@@ -360,7 +360,7 @@ export class PiAiAdapter extends LlmAdapter {
|
||||
}
|
||||
const context = attachments === undefined
|
||||
? toPiContext(options, undefined, onReplayDegrade)
|
||||
: await toPiContext(options, attachments, onReplayDegrade, profile.maxRequestImageBytes, {
|
||||
: await toPiContext({ ...options, signal: watchdog.signal }, attachments, onReplayDegrade, profile.maxRequestImageBytes, {
|
||||
maxPixels: profile.requestImagePixelBudget,
|
||||
maxBytes: profile.requestImageMaxBytes,
|
||||
})
|
||||
|
||||
@@ -46,9 +46,9 @@ export const DEFAULT_STREAM_IDLE_TIMEOUT_MS = 300_000
|
||||
* Default request-level bound on base64-encoded image payload. Every image in
|
||||
* history is re-encoded into every request body, so an unbounded conversation
|
||||
* eventually exceeds a provider or gateway request-size cap and the session
|
||||
* can never complete another request. The 20MiB default admits four images at
|
||||
* the attachment store's 3.5MiB raw-image default after base64 expansion and
|
||||
* reserves request capacity for system prompts, history, tools, and JSON.
|
||||
* can never complete another request. The 20MiB default admits fifteen 1MiB
|
||||
* request versions after base64 expansion and reserves request capacity for
|
||||
* system prompts, history, tools, and JSON.
|
||||
* Deployments behind stricter gateways lower it per route.
|
||||
*/
|
||||
export const DEFAULT_MAX_REQUEST_IMAGE_BYTES = 20 * 1024 * 1024
|
||||
|
||||
@@ -48,6 +48,7 @@ function assertSupportedImageRoles(messages: readonly Message[]): void {
|
||||
async function userContent(
|
||||
blocks: readonly ContentBlock[],
|
||||
requestImages: ReadonlyMap<AttachmentId, RequestImageAttachment>,
|
||||
cropAvailable: boolean,
|
||||
): Promise<string | (TextContent | ImageContent)[]> {
|
||||
const content: (TextContent | ImageContent)[] = []
|
||||
for (const block of blocks) {
|
||||
@@ -60,7 +61,7 @@ async function userContent(
|
||||
if (version === undefined) {
|
||||
throw new LlmError(`pi-ai request image ${block.attachment.attachmentId} was not prepared`, 'INVALID_REQUEST')
|
||||
}
|
||||
content.push({ type: 'text', text: requestImagePreviewText(version) })
|
||||
content.push({ type: 'text', text: requestImagePreviewText(version, cropAvailable) })
|
||||
content.push({
|
||||
type: 'image',
|
||||
data: Buffer.from(version.data).toString('base64'),
|
||||
@@ -70,7 +71,7 @@ async function userContent(
|
||||
}
|
||||
case 'tool-result':
|
||||
{
|
||||
const nested = await userContent(block.content, requestImages)
|
||||
const nested = await userContent(block.content, requestImages, cropAvailable)
|
||||
if (typeof nested === 'string') {
|
||||
if (nested.length > 0) content.push({ type: 'text', text: nested })
|
||||
} else {
|
||||
@@ -101,11 +102,16 @@ async function prepareRequestImages(
|
||||
messages: readonly Message[],
|
||||
attachments: AttachmentStore,
|
||||
policy: ImageRequestPolicy,
|
||||
signal?: AbortSignal,
|
||||
): Promise<Map<AttachmentId, RequestImageAttachment>> {
|
||||
const refs = new Map<AttachmentId, ImageAttachmentRef>()
|
||||
for (const message of messages) collectImageRefs(message.content, refs)
|
||||
const orderedRefs = [...refs.values()]
|
||||
const prepared = await attachments.readImageRequests(orderedRefs, policy, signal)
|
||||
const versions = new Map<AttachmentId, RequestImageAttachment>()
|
||||
for (const [id, ref] of refs) versions.set(id, await attachments.readImageRequest(ref, policy))
|
||||
for (const [index, ref] of orderedRefs.entries()) {
|
||||
versions.set(ref.attachmentId, prepared[index] as RequestImageAttachment)
|
||||
}
|
||||
return versions
|
||||
}
|
||||
|
||||
@@ -222,17 +228,24 @@ async function toPiContextWithImages(
|
||||
},
|
||||
): Promise<PiContext> {
|
||||
assertSupportedImageRoles(options.messages)
|
||||
const requestImages = await prepareRequestImages(options.messages, attachments, requestImagePolicy)
|
||||
const requestMessages = offloadRequestImagesWithPolicy(options.messages, {
|
||||
representation: 'base64',
|
||||
...maxRequestImageBytes === undefined ? {} : { maxBytes: maxRequestImageBytes },
|
||||
byteQuantum: 1,
|
||||
byteLength: ref => Math.min(ref.bytes, requestImagePolicy.maxBytes),
|
||||
})
|
||||
const requestImages = await prepareRequestImages(requestMessages, attachments, requestImagePolicy, options.signal)
|
||||
const exactMessages = offloadRequestImagesWithPolicy(requestMessages, {
|
||||
representation: 'base64',
|
||||
...maxRequestImageBytes === undefined ? {} : { maxBytes: maxRequestImageBytes },
|
||||
byteQuantum: 1,
|
||||
byteLength: ref => requestImages.get(ref.attachmentId)?.bytes ?? ref.bytes,
|
||||
})
|
||||
const cropAvailable = options.tools?.some(tool => tool.name === 'read_image_region') ?? false
|
||||
const toolNames = new Map<CallId, string>()
|
||||
const messages: PiMessage[] = []
|
||||
|
||||
for (const message of requestMessages) {
|
||||
for (const message of exactMessages) {
|
||||
if (message.role === 'system') {
|
||||
// pi-ai has a single systemPrompt slot; in-history system messages are
|
||||
// folded into user messages to preserve order (rare in practice — the
|
||||
@@ -250,7 +263,7 @@ async function toPiContextWithImages(
|
||||
}
|
||||
// user role: text + tool results (each result becomes its own message).
|
||||
const regular = message.content.filter(block => block.type !== 'tool-result')
|
||||
const content = await userContent(regular, requestImages)
|
||||
const content = await userContent(regular, requestImages, cropAvailable)
|
||||
const results = message.content.filter((block): block is Extract<ContentBlock, { type: 'tool-result' }> => (
|
||||
block.type === 'tool-result'
|
||||
))
|
||||
@@ -258,7 +271,7 @@ async function toPiContextWithImages(
|
||||
messages.push({ role: 'user', content, timestamp: 0 })
|
||||
}
|
||||
for (const result of results) {
|
||||
const resultContent = await userContent(result.content, requestImages)
|
||||
const resultContent = await userContent(result.content, requestImages, cropAvailable)
|
||||
messages.push({
|
||||
role: 'toolResult',
|
||||
toolCallId: result.toolCallId,
|
||||
|
||||
@@ -239,7 +239,11 @@ describe('PiAiAdapter provider routing', () => {
|
||||
}
|
||||
const readImage = vi.fn((_ref: ImageAttachmentRef): Promise<StoredImageAttachment> =>
|
||||
Promise.resolve({ ref, data: Uint8Array.of(1) }))
|
||||
const readImageRequest = vi.fn((value: ImageAttachmentRef, _policy: ImageRequestPolicy): Promise<RequestImageAttachment> => (
|
||||
const readImageRequest = vi.fn((
|
||||
value: ImageAttachmentRef,
|
||||
_policy: ImageRequestPolicy,
|
||||
_signal?: AbortSignal,
|
||||
): Promise<RequestImageAttachment> => (
|
||||
Promise.resolve({
|
||||
variantId: ImageVariantId(`sha256:${'b'.repeat(64)}`),
|
||||
master: value,
|
||||
@@ -276,8 +280,12 @@ describe('PiAiAdapter provider routing', () => {
|
||||
return readImage(value)
|
||||
}
|
||||
|
||||
override readImageRequest(value: ImageAttachmentRef, policy: ImageRequestPolicy): Promise<RequestImageAttachment> {
|
||||
return readImageRequest(value, policy)
|
||||
override readImageRequest(
|
||||
value: ImageAttachmentRef,
|
||||
policy: ImageRequestPolicy,
|
||||
signal?: AbortSignal,
|
||||
): Promise<RequestImageAttachment> {
|
||||
return readImageRequest(value, policy, signal)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -301,7 +309,7 @@ describe('PiAiAdapter provider routing', () => {
|
||||
expect(readImageRequest).toHaveBeenCalledWith(ref, {
|
||||
maxPixels: 2048 * 2048,
|
||||
maxBytes: 1024 * 1024,
|
||||
})
|
||||
}, expect.any(AbortSignal))
|
||||
expect(server.paths).toEqual(['/v1/responses'])
|
||||
})
|
||||
|
||||
|
||||
@@ -1,6 +1,11 @@
|
||||
import { describe, expect, it, vi } from 'vitest'
|
||||
import { AttachmentId, ImageVariantId } from '@deepseek-ai/dsh-attachment'
|
||||
import type { AttachmentStore, ImageAttachmentRef, RequestImageAttachment } from '@deepseek-ai/dsh-attachment'
|
||||
import type {
|
||||
AttachmentStore,
|
||||
ImageAttachmentRef,
|
||||
ImageRequestPolicy,
|
||||
RequestImageAttachment,
|
||||
} from '@deepseek-ai/dsh-attachment'
|
||||
import { CallId, createMessage, createUserMessage, OFFLOADED_IMAGE_TEXT } from '@deepseek-ai/dsh-llm'
|
||||
import type { ContentBlock, GenerateOptions, Message } from '@deepseek-ai/dsh-llm'
|
||||
import { toPiContext } from '../src/context.ts'
|
||||
@@ -30,11 +35,22 @@ function requestImage(value: ImageAttachmentRef, data: Uint8Array): RequestImage
|
||||
}
|
||||
|
||||
function projectionStore(
|
||||
readImageRequest = vi.fn((value: ImageAttachmentRef) => (
|
||||
readImageRequest: (
|
||||
value: ImageAttachmentRef,
|
||||
policy: ImageRequestPolicy,
|
||||
signal?: AbortSignal,
|
||||
) => Promise<RequestImageAttachment> = vi.fn((value: ImageAttachmentRef) => (
|
||||
Promise.resolve(requestImage(value, Uint8Array.of(1)))
|
||||
)),
|
||||
): AttachmentStore {
|
||||
return { readImageRequest } as unknown as AttachmentStore
|
||||
return {
|
||||
readImageRequest,
|
||||
readImageRequests: (
|
||||
refs: readonly ImageAttachmentRef[],
|
||||
policy: Parameters<AttachmentStore['readImageRequest']>[1],
|
||||
signal?: AbortSignal,
|
||||
) => Promise.all(refs.map(value => readImageRequest(value, policy, signal))),
|
||||
} as unknown as AttachmentStore
|
||||
}
|
||||
|
||||
const attachments = projectionStore()
|
||||
@@ -268,6 +284,42 @@ describe('pi-ai request context conversion', () => {
|
||||
expect(readImageRequest).toHaveBeenCalledTimes(1)
|
||||
})
|
||||
|
||||
it('does not prepare an old image removed by the conservative request projection', async () => {
|
||||
const old = { ...ref, attachmentId: AttachmentId(`sha256:${'c'.repeat(64)}`), bytes: 3 }
|
||||
const recent = { ...ref, attachmentId: AttachmentId(`sha256:${'d'.repeat(64)}`), bytes: 3 }
|
||||
const readImageRequest = vi.fn((value: ImageAttachmentRef) => {
|
||||
if (value.attachmentId === old.attachmentId) throw new Error('old image must not be read')
|
||||
return Promise.resolve(requestImage(value, Uint8Array.of(1, 2, 3)))
|
||||
})
|
||||
|
||||
const context = await toPiContext(request([user([
|
||||
{ type: 'image', attachment: old },
|
||||
{ type: 'image', attachment: recent },
|
||||
])]), projectionStore(readImageRequest), undefined, 4)
|
||||
|
||||
expect(context.messages[0]).toMatchObject({
|
||||
role: 'user',
|
||||
content: [
|
||||
{ type: 'text', text: OFFLOADED_IMAGE_TEXT },
|
||||
{ type: 'text', text: expect.stringContaining(String(recent.attachmentId)) as string },
|
||||
{ type: 'image' },
|
||||
],
|
||||
})
|
||||
expect(readImageRequest).toHaveBeenCalledTimes(1)
|
||||
expect(readImageRequest.mock.calls[0]?.[0]).toEqual(recent)
|
||||
})
|
||||
|
||||
it('advertises region reads only when the request exposes the tool', async () => {
|
||||
const withoutCrop = await toPiContext(request([user([{ type: 'image', attachment: ref }])]), attachments)
|
||||
const withCrop = await toPiContext({
|
||||
...request([user([{ type: 'image', attachment: ref }])]),
|
||||
tools: [{ name: 'read_image_region', description: 'crop', parameters: { type: 'object' } }],
|
||||
}, attachments)
|
||||
|
||||
expect(JSON.stringify(withoutCrop.messages)).not.toContain('Call read_image_region')
|
||||
expect(JSON.stringify(withCrop.messages)).toContain('Call read_image_region')
|
||||
})
|
||||
|
||||
it('keeps every image at exactly the payload bound and drops all of them when even the newest cannot fit', async () => {
|
||||
const sized: ImageAttachmentRef = { ...ref, bytes: 3 }
|
||||
const exact = await toPiContext(request([
|
||||
@@ -298,7 +350,7 @@ describe('pi-ai request context conversion', () => {
|
||||
expect(oversized.messages).toEqual([
|
||||
{ role: 'user', content: OFFLOADED_IMAGE_TEXT, timestamp: 0 },
|
||||
])
|
||||
expect(readImageRequest).toHaveBeenCalledTimes(1)
|
||||
expect(readImageRequest).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('offloads repeated image-block occurrences by position rather than shared object identity', async () => {
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { describe, expect, it, vi } from 'vitest'
|
||||
import { AttachmentId, ImageVariantId } from '@deepseek-ai/dsh-attachment'
|
||||
import type { AttachmentStore, ImageAttachmentRef, RequestImageAttachment } from '@deepseek-ai/dsh-attachment'
|
||||
import type { AttachmentStore, ImageAttachmentRef, ImageRequestPolicy, RequestImageAttachment } from '@deepseek-ai/dsh-attachment'
|
||||
import { createUserMessage, CallId, CONTEXT_WINDOW_EXCEEDED_CODE, EMPTY_RESPONSE_CODE, createMessage } from '@deepseek-ai/dsh-llm'
|
||||
import type { ContentBlock, StreamChunk } from '@deepseek-ai/dsh-llm'
|
||||
import type { AssistantMessage, AssistantMessageEvent, Usage } from '@earendil-works/pi-ai'
|
||||
@@ -58,6 +58,23 @@ function requestVersion(ref: ImageAttachmentRef): RequestImageAttachment {
|
||||
}
|
||||
}
|
||||
|
||||
function attachmentStore(readImageRequest: (
|
||||
ref: ImageAttachmentRef,
|
||||
policy: ImageRequestPolicy,
|
||||
signal?: AbortSignal,
|
||||
) => Promise<RequestImageAttachment>): AttachmentStore {
|
||||
return {
|
||||
readImageRequest,
|
||||
readImageRequests: (
|
||||
refs: readonly ImageAttachmentRef[],
|
||||
policy: ImageRequestPolicy,
|
||||
signal?: AbortSignal,
|
||||
) => Promise.all(
|
||||
refs.map(ref => readImageRequest(ref, policy, signal)),
|
||||
),
|
||||
} as unknown as AttachmentStore
|
||||
}
|
||||
|
||||
describe('toPiContext', () => {
|
||||
it('maps system prompt, user text, and tools', () => {
|
||||
const context = toPiContext({
|
||||
@@ -91,7 +108,9 @@ describe('toPiContext', () => {
|
||||
width: 1,
|
||||
height: 1,
|
||||
}
|
||||
const readImageRequest = vi.fn((value: ImageAttachmentRef) => Promise.resolve(requestVersion(value)))
|
||||
const readImageRequest = vi.fn((value: ImageAttachmentRef, _policy: ImageRequestPolicy) => (
|
||||
Promise.resolve(requestVersion(value))
|
||||
))
|
||||
const context = await toPiContext({
|
||||
provider: 'openai',
|
||||
model: 'gpt-4.1',
|
||||
@@ -99,11 +118,12 @@ describe('toPiContext', () => {
|
||||
content: [{ type: 'text', text: 'describe' }, { type: 'image', attachment }],
|
||||
source: { kind: 'plugin', plugin: 'test' },
|
||||
})],
|
||||
}, { readImageRequest } as unknown as AttachmentStore)
|
||||
}, attachmentStore(readImageRequest))
|
||||
|
||||
expect(readImageRequest).toHaveBeenCalledWith(
|
||||
attachment,
|
||||
{ maxPixels: 2048 * 2048, maxBytes: 1024 * 1024 },
|
||||
undefined,
|
||||
)
|
||||
expect(context.messages[0]).toEqual({
|
||||
role: 'user',
|
||||
@@ -124,7 +144,9 @@ describe('toPiContext', () => {
|
||||
width: 1,
|
||||
height: 1,
|
||||
}
|
||||
const readImageRequest = vi.fn((value: ImageAttachmentRef) => Promise.resolve(requestVersion(value)))
|
||||
const readImageRequest = vi.fn((value: ImageAttachmentRef, _policy: ImageRequestPolicy) => (
|
||||
Promise.resolve(requestVersion(value))
|
||||
))
|
||||
const context = await toPiContext({
|
||||
provider: 'openai',
|
||||
model: 'gpt-4.1',
|
||||
@@ -148,7 +170,7 @@ describe('toPiContext', () => {
|
||||
}],
|
||||
source: { kind: 'plugin', plugin: 'test' },
|
||||
})],
|
||||
}, { readImageRequest } as unknown as AttachmentStore)
|
||||
}, attachmentStore(readImageRequest))
|
||||
|
||||
expect(context.messages).toEqual([{
|
||||
role: 'toolResult',
|
||||
|
||||
@@ -21,12 +21,15 @@ export function textOnlyImageText(ref: ImageAttachmentRef): string {
|
||||
/**
|
||||
* Stable model-facing handle and coordinate description for one exact request preview.
|
||||
* @param version - exact request image shown beside the text.
|
||||
* @param cropAvailable - whether the active request exposes `read_image_region`.
|
||||
* @returns attachment handle, preview dimensions, and crop-coordinate guidance.
|
||||
*/
|
||||
export function requestImagePreviewText(version: RequestImageAttachment): string {
|
||||
return `Image ${version.master.attachmentId}; preview ${version.width}x${version.height}px. `
|
||||
+ 'Crop coordinates use this preview. Call read_image_region with this attachment_id, '
|
||||
+ `preview_width=${version.width}, preview_height=${version.height}, x, y, width, and height.`
|
||||
export function requestImagePreviewText(version: RequestImageAttachment, cropAvailable: boolean): string {
|
||||
const identity = `Image ${version.master.attachmentId}; preview ${version.width}x${version.height}px.`
|
||||
return cropAvailable
|
||||
? `${identity} Crop coordinates use this preview. Call read_image_region with this attachment_id, `
|
||||
+ `preview_width=${version.width}, preview_height=${version.height}, x, y, width, and height.`
|
||||
: identity
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
Reference in New Issue
Block a user