refactor(attachment): 分离宿主位置与模型访问路径

This commit is contained in:
creatixchu
2026-08-24 14:51:03 +08:00
parent 7f4cf99eeb
commit 558f08780c
57 changed files with 366 additions and 236 deletions
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-08-20-unified-image-request-pipeline.md
2026-08-20-unified-image-request-pipeline.md: 7af0bea0acb117844b258a0333e97e97facf493c
2026-08-20-unified-image-request-pipeline.zh.md: ecdb71548190ba22573a5e05c3b3fde76c28d511
2026-08-20-unified-image-request-pipeline.md: 85296f1d8bd7d6ee458f8bc4230e52f1d3128bff
2026-08-20-unified-image-request-pipeline.zh.md: bcbc0110001e9f974e57a456140e9ecdf4eb888c
@@ -26,11 +26,11 @@ Batch admission prepares and verifies every normalized attachment once before pu
The `variantId` and cache path cover the normalized attachment id, transform version, route pixel and byte budgets, and fixed encoder parameters. A new cache entry is fully decoded before publication. Cache hits use a header probe to check format, 8-bit sRGB/sRGBA facts, dimensions, alpha, and byte limits without decoding the complete raster again; a mismatch regenerates the entry. DeepSeek Files and pi-ai inline base64 therefore use the same deterministic bytes for the same policy. Inline accounting uses the derived byte length after base64 expansion, not the normalized attachment byte count. Equal in-process `variantId` calls share one transform and cache write. Each caller can cancel its own wait; the shared transform is aborted only after every waiter has cancelled. Callers preserve order by applying `Promise.all` to singular `readImageRequest` calls. The local implementation runs normalization and request transforms through one FIFO limiter; `imageCompressionConcurrency` is configurable from 1 through 8 and defaults to 2. Batch publication remains sequential after every normalized attachment has been prepared.
Request-size offload is a deterministic oldest-first projection. Before reading attachments, each route uses `min(attachmentBytes, requestVersionMaxBytes)` as a conservative upper bound and removes the oldest over-budget prefix. Only retained attachments are read and transformed, so an omitted missing or corrupt object cannot block the request. A second projection uses exact derived lengths without bringing omitted images back. DeepSeek defaults to 128MiB and 600 referenced images. Its removed prefix advances past successive 64MiB byte boundaries and in 20-image count quanta, so 129 one-megabyte images remove the oldest 65, retain 64MiB, and keep that prefix stable until total history passes 192MiB. Pi-ai retains a configurable base64 request bound. Each omitted image becomes a deterministic per-image placeholder that retains its identity and current provider access facts, including nested tool-result images, while append-only session history keeps the original references.
Request-size offload is a deterministic oldest-first projection. Before reading attachments, each route uses `min(attachmentBytes, requestVersionMaxBytes)` as a conservative upper bound and removes the oldest over-budget prefix. Only retained attachments are read and transformed, so an omitted missing or corrupt object cannot block the request. A second projection uses exact derived lengths without bringing omitted images back. DeepSeek defaults to 128MiB and 600 referenced images. Its removed prefix advances past successive 64MiB byte boundaries and in 20-image count quanta, so 129 one-megabyte images remove the oldest 65, retain 64MiB, and keep that prefix stable until total history passes 192MiB. Pi-ai retains a configurable base64 request bound. Each omitted image becomes a per-image placeholder that retains its identity and access resolved for the current tool execution world, including nested tool-result images, while append-only session history keeps the original references.
### Stable handles
Every retained request image is preceded by its display name or complete attachment id, actual request dimensions, and current provider access facts. The local provider supplies the absolute read-only normalized-object path, normalized dimensions, and media type. The descriptor states that normalization or request projection may have resized or re-encoded the upload, so the model cannot infer original upload properties from either representation. User messages, tool results, agent-loop requests, compaction, and direct `ctx.llm.stream` calls share this projection. The path is derived from the logged reference and current provider root at request time; it does not enter the durable reference or session log.
Every retained request image is preceded by its display name or complete attachment id and actual request dimensions. The attachment provider can supply its host object location; the LLM consumer combines it with the current filesystem mapping before adding an absolute read-only path, normalized dimensions, and media type. The descriptor states that normalization or request projection may have resized or re-encoded the upload, so the model cannot infer original upload properties from either representation. User messages, tool results, agent-loop requests, compaction, and direct `ctx.llm.stream` calls share this projection. The path is resolved independently from the deterministic request version and does not enter its `variantId`, the durable reference, or the session log.
### DeepSeek Files lifecycle
@@ -26,11 +26,11 @@ Status: implemented
`variantId` 和缓存路径覆盖规范化附件 ID、变换策略版本、路由像素和字节预算及固定编码参数。新缓存条目在发布前会完整解码。缓存命中只探测文件头,校验格式、8-bit sRGB/sRGBA、尺寸、透明通道和字节上限,不会再次完整解码光栅;不匹配时会重新生成。因此,同一策略下的 DeepSeek Files 和 pi-ai 内联 base64 使用相同的确定性字节。内联计量使用派生字节经过 base64 膨胀后的长度,不使用规范化附件字节数。同一进程内相同 `variantId` 的调用共享一次变换和缓存写入。每个调用方可以取消自己的等待;只有全部等待方都取消时,共享变换才会中止。调用方对单数 `readImageRequest` 使用 `Promise.all` 保持结果顺序。本地实现通过一个 FIFO 限流器运行规范化和请求变换,`imageCompressionConcurrency` 的可配置范围为 1 至 8,默认值为 2。全部规范化附件准备完成后,批次仍按顺序发布。
请求大小 offload 是确定性的从旧到新投影。读取附件前,每条路由先以 `min(附件字节数, 请求版本字节上限)` 作为保守上界,移除超出预算的最旧前缀。系统只读取并转换保留的附件,因此已省略的缺失或损坏对象不会阻塞请求。第二次投影使用确切派生长度,但不会重新加入已省略图片。DeepSeek 默认上限为 128MiB 和 600 张引用图片。被移除前缀会越过连续的 64MiB 字节边界,并按 20 张图片数量步长递增,因此 129 张 1MiB 图片会移除最旧的 65 张并保留 64MiB;持久历史超过 192MiB 前,该前缀保持不变。Pi-ai 保留可配置的 base64 请求上限。每张省略图片都会变成确定性的逐图占位文本,保留自己的身份和当前提供方访问事实,嵌套工具结果图片也使用相同规则;追加式会话历史继续保留原始引用。
请求大小 offload 是确定性的从旧到新投影。读取附件前,每条路由先以 `min(附件字节数, 请求版本字节上限)` 作为保守上界,移除超出预算的最旧前缀。系统只读取并转换保留的附件,因此已省略的缺失或损坏对象不会阻塞请求。第二次投影使用确切派生长度,但不会重新加入已省略图片。DeepSeek 默认上限为 128MiB 和 600 张引用图片。被移除前缀会越过连续的 64MiB 字节边界,并按 20 张图片数量步长递增,因此 129 张 1MiB 图片会移除最旧的 65 张并保留 64MiB;持久历史超过 192MiB 前,该前缀保持不变。Pi-ai 保留可配置的 base64 请求上限。每张省略图片都会变成逐图占位文本,保留自己的身份和本次工具执行环境解析出的访问方式,嵌套工具结果图片也使用相同规则;追加式会话历史继续保留原始引用。
### 稳定句柄
每张保留请求图片前都有显示名称或完整附件 ID实际请求尺寸,以及当前提供方访问事实。本地提供方会给出规范化对象的绝对只读路径、规范化尺寸和媒体类型。描述会说明规范化或请求投影可能缩小或重新编码上传图片,因此模型不能从任一版本推断上传图片原本的属性。用户消息、工具结果、agent loop 请求、压缩和直接 `ctx.llm.stream` 调用共享这套投影。路径在请求时根据已记录引用和当前提供方根目录派生,不进入持久引用或会话日志。
每张保留请求图片前都有显示名称或完整附件 ID,以及实际请求尺寸。附件提供方可以给出宿主对象位置;LLM 消费方将其与当前文件系统映射组合后,再加入绝对只读路径、规范化尺寸和媒体类型。描述会说明规范化或请求投影可能缩小或重新编码上传图片,因此模型不能从任一版本推断上传图片原本的属性。用户消息、工具结果、agent loop 请求、压缩和直接 `ctx.llm.stream` 调用共享这套投影。路径独立于确定性请求版本解析,不进入其 `variantId`持久引用或会话日志。
### DeepSeek Files 生命周期
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-08-21-model-readable-image-paths.md
2026-08-21-model-readable-image-paths.md: 729e6347e8623694e1cc4f4779bf71447006cb08
2026-08-21-model-readable-image-paths.zh.md: d67817cf699afb6b094524388dd6d5a48a653e90
2026-08-21-model-readable-image-paths.md: 728d25ee84836a07805f9d8728e47e9319e0863d
2026-08-21-model-readable-image-paths.zh.md: 2596dec5a21a8908d5875ab031bed6655b2305ca
@@ -1,4 +1,4 @@
# Agent Note: Provider-resolved image paths in model requests
# Agent Note: Execution-world image paths in model requests
Status: implemented
@@ -10,21 +10,29 @@ An uploaded image becomes an opaque durable `ImageAttachmentRef`. Image-capable
## Decision
`ImageAttachmentRef` remains portable session data and contains no host path. `AttachmentStore.imageAccess(ref)` resolves optional access facts from the current provider. The local provider derives an absolute object path from the resolved `DSH_HOME`, attachment storage version, and validated digest, then asks the mounted `ctx.fs` to map that host file into its execution world. `FileSystem.processPathFromHostPath(hostPath)` returns no mapping by default; the host-backed provider returns the absolute process path, while E2B and other remote providers return no access facts. A request version carries available facts transiently for serialization.
`ImageAttachmentRef` remains portable session data and contains no host path. `AttachmentStore.imageHostPath(ref)` exposes only the attachment provider's optional host object location. The local provider derives that absolute path from the resolved `DSH_HOME`, attachment storage version, and validated digest; it does not inspect the model-tool execution world.
The shared LLM image descriptor names the display name or full attachment id, the exact request-preview dimensions, and the provider-resolved normalized path when available. Local access text includes normalized dimensions and media type, identifies the object as read-only, directs the model to copy it to a writable path with the matching extension before editing, and states that normalization or request projection may have resized or re-encoded the upload. DeepSeek Files and pi-ai inline requests use the same descriptor.
Each LLM provider plugin supplies the bridge at request assembly. It combines the attachment store with the current `ctx.fs.processPathFromHostPath(hostPath)` mapping and passes the resulting `ImageAttachmentAccess` independently into retained-image serialization and offload placeholders. The filesystem service is authoritative for this question because the mounted filesystem and subprocess providers share one execution world. A separate subprocess-presence check would not prove path reachability. Host-backed filesystems return a process path; E2B and other remote backends without a shared mount return no mapping.
Request-size offload requires a per-image placeholder function; the previous shared placeholder constant and its byte-bound wrapper had no remaining production caller and are removed. DeepSeek and pi-ai replace each omitted occurrence with its own attachment identity and current access facts without reading or transforming the omitted object. Offload selection, byte accounting, and quantized prefix behavior remain unchanged.
`RequestImageAttachment` remains a deterministic version selected by the attachment and route policy. It contains `variantId`, encoded bytes, dimensions, and encoding metadata, but no access path. Execution-world access may change with the host or mounted providers and does not participate in `variantId`.
Descriptor identity comes from each occurrence's own durable reference, not from the prepared request version: versions are deduplicated per attachment id, so two uploads of the same content under different names share one version while each occurrence keeps its own display name. Access resolution validates the logged attachment id; a malformed reference in durable history fails the request at assembly, the earliest point that resolves it.
The shared LLM image descriptor names the display name or full attachment id, the exact request-preview dimensions, and the execution-world path when the bridge resolves one. Local access text includes normalized dimensions and media type, identifies the object as read-only, directs the model to copy it to a writable path with the matching extension before editing, and states that normalization or request projection may have resized or re-encoded the upload. DeepSeek Files and pi-ai inline requests use the same descriptor.
Absolute paths stay out of session events. Model-visible path text is reconstructed from the logged attachment reference and the providers mounted for the current process. Restoring the same session with a different `DSH_HOME` produces the path valid on that host; restoring it with a remote execution world that has no shared mount produces no path. Published attachment objects use owner-read-only mode, including deduplicated objects, and model instructions require a writable copy for modifications.
Request-size offload requires a per-image placeholder function; the previous shared placeholder constant and its byte-bound wrapper had no remaining production caller and are removed. DeepSeek and pi-ai replace each omitted occurrence with its own attachment identity and access resolved for that request without reading or transforming the omitted object. Offload selection, byte accounting, and quantized prefix behavior remain unchanged.
Descriptor identity comes from each occurrence's own durable reference, not from the prepared request version: versions are deduplicated per attachment id, so two uploads of the same content under different names share one version while each occurrence keeps its own display name. Host-location resolution validates the logged attachment id; a malformed reference in durable history fails the request at assembly, the earliest point that resolves it.
Absolute paths stay out of session events and request-image versions. Model-visible path text is reconstructed from the logged attachment reference, the current attachment provider, and the filesystem mounted for the current execution world. Restoring the same session with a different `DSH_HOME` produces the path valid on that host; restoring it with a remote execution world that has no shared mount produces no path. Published attachment objects use owner-read-only mode, including deduplicated objects, and model instructions require a writable copy for modifications.
## Alternatives considered
**Persist the absolute path in `ImageAttachmentRef`.** A durable host path becomes stale after moving a session, changing `DSH_HOME`, or mounting another provider. Resolving it at request time preserves portable history.
**Teach each LLM adapter the `~/.dsh` layout.** Explicit `dshHome` and `$DSH_HOME` can select another root, and non-local providers may expose no path. The attachment provider owns this fact.
**Teach each LLM adapter the `~/.dsh` layout.** Explicit `dshHome` and `$DSH_HOME` can select another root, and non-local providers may expose no path. The attachment provider owns the object location; the shared LLM helper owns the composition with an execution-world mapper.
**Let the attachment provider inspect `ctx.fs`.** The attachment provider knows where its object lives but does not own the model tools' execution environment. Combining both services in the LLM consumer also avoids a package reference cycle between attachment, filesystem, and LLM definitions.
**Store `ImageAttachmentAccess` in `RequestImageAttachment`.** Request versions are deterministic cache and upload values. A path that changes with the host or execution environment must not affect their identity or appear to be part of their reproducible data.
**Infer path sharing from the filesystem provider's package or class name.** Provider identity does not establish that a host file is mounted into its execution world. The filesystem provider instead answers the exact mapping question and can support an explicit shared mount without changing the attachment provider.
@@ -32,8 +40,8 @@ Absolute paths stay out of session events. Model-visible path text is reconstruc
## Verification
Package tests cover provider access defaults, host-backed path mapping, absence without a mapped filesystem, digest-to-path resolution, owner-read-only publication and deduplication, request-version access propagation, retained-image descriptions, per-image nested offload placeholders, source-property warnings, and matching extensions. A keyless assembled ACP snapshot checks the exact local object path in both a retained DeepSeek Files image handle and an offloaded image placeholder.
Package tests cover provider host-location defaults, host-backed path mapping, absence without a mapped filesystem, digest-to-path resolution, owner-read-only publication and deduplication, access passed independently from request versions, retained-image descriptions, per-image nested offload placeholders, source-property warnings, and matching extensions. A keyless assembled ACP snapshot checks the exact local object path in both a retained DeepSeek Files image handle and an offloaded image placeholder.
## Consequences
When the execution world maps the attachment object, the selected model provider receives its path. This disclosure lets the model operate on the stored image and is limited to normalized attachment objects already in that request's authorized history. Descriptor text adds tokens for every retained or offloaded image. Paths change when the provider root changes, while deterministic image bytes and session references remain unchanged. Remote execution worlds without a shared mount receive the existing no-path recovery text. A missing local object still fails when a model tool attempts to read it.
When the execution world maps the attachment object, the selected model provider receives its path. This disclosure lets the model operate on the stored image and is limited to normalized attachment objects already in that request's authorized history. Descriptor text adds tokens for every retained or offloaded image. Paths can change while deterministic image bytes, `variantId`, and session references remain unchanged; the changed historical text can prevent KV-cache reuse from the first affected image even when no image is offloaded. Remote execution worlds without a shared mount receive the existing no-path recovery text. A missing local object still fails when a model tool attempts to read it.
@@ -1,4 +1,4 @@
# Agent Note:在模型请求中提供由附件提供方解析的图片路径
# Agent Note:在模型请求中提供执行环境图片路径
状态:已实现
@@ -10,21 +10,29 @@
## 决策
`ImageAttachmentRef` 继续作为可移植的会话数据,不包含宿主路径。`AttachmentStore.imageAccess(ref)` 从当前提供方解析可选访问事实。本地提供方根据已解析的 `DSH_HOME`、附件存储版本和经过校验的摘要派生对象的绝对路径,再要求当前 `ctx.fs` 把这个宿主文件映射到其执行世界。`FileSystem.processPathFromHostPath(hostPath)` 默认不返回映射。宿主后端返回绝对进程路径,E2B 和其他远程后端不返回访问事实。请求版本只在序列化期间携带当前可用的事实
`ImageAttachmentRef` 继续作为可移植的会话数据,不包含宿主路径。`AttachmentStore.imageHostPath(ref)` 只公开附件提供方可选的宿主对象位置。本地提供方根据已解析的 `DSH_HOME`、附件存储版本和经过校验的摘要派生绝对路径,不检查模型工具的执行环境
共用 LLM 图片描述会写明显示名称或完整附件 ID、确切请求预览尺寸,以及当前可用的提供方规范化路径。本地访问文本还包含规范化尺寸和媒体类型,说明对象只供读取,要求模型在编辑前按匹配扩展名复制到可写路径,并指出规范化或请求投影可能缩小或重新编码上传图片。DeepSeek Files 与 pi-ai 内联请求使用同一描述
每个 LLM 提供方插件在组装请求时提供桥接逻辑。它把附件存储与当前 `ctx.fs.processPathFromHostPath(hostPath)` 映射组合起来,再将得到的 `ImageAttachmentAccess` 单独传给保留图片序列化和 offload 占位内容。当前文件系统服务可以回答这个问题,因为挂载的文件系统和子进程提供方共享同一执行环境。另行检查子进程服务是否存在并不能证明路径可达。宿主文件系统返回进程路径;E2B 和其他没有共享挂载的远程后端不返回映射
请求大小 offload 要求提供逐图占位文本函数;原先共享的占位常量及其字节上限包装函数已没有生产调用方,因此删除。DeepSeek 和 pi-ai 会把每个省略位置替换为该图片自己的附件身份和当前访问事实,无需读取或转换省略对象。Offload 的选择、字节计量和按固定步长变化的前缀行为保持不变
`RequestImageAttachment` 仍是由附件和路由策略确定的请求版本。它包含 `variantId`、编码字节、尺寸和编码元数据,不包含访问路径。执行环境访问方式可能随宿主或挂载的提供方变化,不参与 `variantId`
描述文本的身份来自每个出现位置自己的持久引用,而不是准备好的请求版本:请求版本按附件 ID 去重,同一内容以不同文件名上传两次会共享一个版本,但每个出现位置保留自己的显示名称。访问解析会校验已记录的附件 ID;持久历史中的畸形引用会在请求组装时失败,这是能解析它的最早时点
共用 LLM 图片描述会写明显示名称或完整附件 ID、确切请求预览尺寸,以及桥接逻辑能够解析出的执行环境路径。本地访问文本还包含规范化尺寸和媒体类型,说明对象只供读取,要求模型在编辑前按匹配扩展名复制到可写路径,并指出规范化或请求投影可能缩小或重新编码上传图片。DeepSeek Files 与 pi-ai 内联请求使用同一描述
绝对路径不会进入会话事件。模型可见路径根据已记录附件引用和当前进程挂载的提供方重建。同一会话在不同 `DSH_HOME` 下恢复时会得到该宿主上的有效路径;在没有共享挂载的远程执行世界中恢复时不会得到路径。发布的附件对象采用仅所有者可读的权限,去重对象也执行该权限。模型指令要求先复制到可写位置再修改
请求大小 offload 要求提供逐图占位文本函数;原先共享的占位常量及其字节上限包装函数已没有生产调用方,因此删除。DeepSeek 和 pi-ai 会把每个省略位置替换为该图片自己的附件身份和本次请求解析出的访问方式,无需读取或转换省略对象。Offload 的选择、字节计量和按固定步长变化的前缀行为保持不变
描述文本的身份来自每个出现位置自己的持久引用,而不是准备好的请求版本:请求版本按附件 ID 去重,同一内容以不同文件名上传两次会共享一个版本,但每个出现位置保留自己的显示名称。宿主位置解析会校验已记录的附件 ID;持久历史中的畸形引用会在请求组装时失败,这是能解析它的最早时点。
绝对路径不会进入会话事件或请求图片版本。模型可见路径根据已记录附件引用、当前附件提供方和当前执行环境挂载的文件系统重建。同一会话在不同 `DSH_HOME` 下恢复时会得到该宿主上的有效路径;在没有共享挂载的远程执行环境中恢复时不会得到路径。发布的附件对象采用仅所有者可读的权限,去重对象也执行该权限。模型指令要求先复制到可写位置再修改。
## 考虑过的替代方案
**把绝对路径持久保存到 `ImageAttachmentRef`。** 移动会话、更改 `DSH_HOME` 或挂载其他提供方后,持久宿主路径会失效。请求时解析可以保持历史可移植。
**让每个 LLM 适配器了解 `~/.dsh` 布局。** 显式 `dshHome``$DSH_HOME` 可以选择其他根目录,非本地提供方也可能没有路径。该事实属于附件提供方。
**让每个 LLM 适配器了解 `~/.dsh` 布局。** 显式 `dshHome``$DSH_HOME` 可以选择其他根目录,非本地提供方也可能没有路径。附件提供方拥有对象位置,共用 LLM 帮助函数负责与执行环境映射组合
**让附件提供方检查 `ctx.fs`。** 附件提供方知道对象在哪里,但不拥有模型工具的执行环境。在 LLM 消费方组合两个服务,也避免了附件、文件系统和 LLM 定义包之间的项目引用循环。
**把 `ImageAttachmentAccess` 放进 `RequestImageAttachment`。** 请求版本是确定性的缓存和上传值。随宿主或执行环境变化的路径不能影响其身份,也不能表现为可重复生成的数据。
**根据文件系统提供方的包名或类名判断是否共享路径。** 提供方身份不能证明宿主文件已挂载到其执行世界。文件系统提供方直接回答具体路径能否映射,也允许未来的显式共享挂载在不修改附件提供方的情况下返回映射。
@@ -32,8 +40,8 @@
## 验证
包测试覆盖提供方访问默认值、宿主后端路径映射、没有文件系统映射时省略路径、本地摘要到路径的解析、仅所有者可读的发布与去重、请求版本访问事实传播、保留图片描述、逐图嵌套 offload 占位文本、源属性提醒和匹配扩展名。Keyless ACP 组装快照会检查保留的 DeepSeek Files 图片句柄和被 offload 图片占位文本中的确切本地对象路径。
包测试覆盖提供方宿主位置默认值、宿主后端路径映射、没有文件系统映射时省略路径、本地摘要到路径的解析、仅所有者可读的发布与去重、独立于请求版本传入访问方式、保留图片描述、逐图嵌套 offload 占位文本、源属性提醒和匹配扩展名。Keyless ACP 组装快照会检查保留的 DeepSeek Files 图片句柄和被 offload 图片占位文本中的确切本地对象路径。
## 后果
执行世界能够映射附件对象时,所选模型提供方会收到该路径。模型可借此操作存储图片,路径只指向该请求已授权历史中的规范化附件对象。每张保留或被 offload 的图片都会增加描述文本 token。提供方根目录变化时路径变化,确定性图片字节和会话引用保持不变。没有共享挂载的远程执行世界会收到原有的无路径恢复文本。本地对象缺失时,模型工具读取它仍会失败。
执行环境能够映射附件对象时,所选模型提供方会收到该路径。模型可借此操作存储图片,路径只指向该请求已授权历史中的规范化附件对象。每张保留或被 offload 的图片都会增加描述文本 token。路径变化,确定性图片字节`variantId` 和会话引用可以保持不变;即使没有图片被 offload,变化后的历史文本也可能使 KV 缓存从首张受影响图片起无法复用。没有共享挂载的远程执行环境会收到原有的无路径恢复文本。本地对象缺失时,模型工具读取它仍会失败。
+2 -2
View File
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write docs/config-catalog.md
config-catalog.md: 7af4d4e47866736ca5f100d07dbd04f8341ae4a0
config-catalog.zh.md: 9e1d3739d25b309172f2a9261cb10e8c5775af12
config-catalog.md: 3b8b44211794a8334fd4cc32171c3fe33994eade
config-catalog.zh.md: e1554020282a454047716a11632e602916929b0b
+2 -2
View File
@@ -317,7 +317,7 @@ export interface Config {
}
```
Source: [`packages/attachment/attachment-local/src/index.ts:53`](../packages/attachment/attachment-local/src/index.ts)
Source: [`packages/attachment/attachment-local/src/index.ts:51`](../packages/attachment/attachment-local/src/index.ts)
<a id="deepseek-aidsh-bash-local"></a>
@@ -950,7 +950,7 @@ export interface DeepSeekCatalogModel {
Depends on: [`ModelModality`](../packages/llm/llm/src/index.ts) · [`RetryPolicyConfig`](../packages/llm/llm/src/index.ts)
Source: [`packages/llm/llm-deepseek/src/index.ts:106`](../packages/llm/llm-deepseek/src/index.ts)
Source: [`packages/llm/llm-deepseek/src/index.ts:107`](../packages/llm/llm-deepseek/src/index.ts)
<a id="deepseek-aidsh-llm-pi-ai"></a>
+2 -2
View File
@@ -319,7 +319,7 @@ export interface Config {
}
```
来源:[`packages/attachment/attachment-local/src/index.ts:53`](../packages/attachment/attachment-local/src/index.ts)
来源:[`packages/attachment/attachment-local/src/index.ts:51`](../packages/attachment/attachment-local/src/index.ts)
<a id="deepseek-aidsh-bash-local"></a>
@@ -952,7 +952,7 @@ export interface DeepSeekCatalogModel {
依赖:[`ModelModality`](../packages/llm/llm/src/index.ts) · [`RetryPolicyConfig`](../packages/llm/llm/src/index.ts)
来源:[`packages/llm/llm-deepseek/src/index.ts:106`](../packages/llm/llm-deepseek/src/index.ts)
来源:[`packages/llm/llm-deepseek/src/index.ts:107`](../packages/llm/llm-deepseek/src/index.ts)
<a id="deepseek-aidsh-llm-pi-ai"></a>
+2 -2
View File
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write docs/subsystems/attachment.md
attachment.md: d855a31c4b463098e30652a30e807d648298c952
attachment.zh.md: 314a6486266fe27d5e5fda17ae1b0c09ee65c5f3
attachment.md: 7fafae424902632d8190e5d156b72795c31dcbbc
attachment.zh.md: 2eed9515a82fa8bba290a5d07f41e521dd7565d8
+5 -15
View File
@@ -10,7 +10,7 @@ Source: [`packages/attachment/attachment/src/types.ts`](../../packages/attachmen
## Identity and verified metadata
`AttachmentId` is a branded opaque string. The local backend currently emits `sha256:<digest>`, but consumers must neither parse that representation nor derive a filesystem path from it. Consumers call `imageAccess()` when they need a path resolved by the mounted provider.
`AttachmentId` is a branded opaque string. The local backend currently emits `sha256:<digest>`, but consumers must neither parse that representation nor derive a filesystem path from it. A consumer may ask the attachment provider for its object location through `imageHostPath()`, then must use the current execution filesystem to decide whether model tools can read that host path.
```ts type-equiv
/** Raster image formats accepted by the version-one attachment path. */
@@ -93,14 +93,6 @@ interface StoredImageAttachment {
}
```
```ts type-equiv
/** Provider-resolved ways for model tools to access one normalized attachment. */
interface ImageAttachmentAccess {
/** Absolute path to immutable normalized bytes; callers must treat it as read-only. */
readonlyPath: string
}
```
```ts type-equiv
/** Deterministic request-image policy selected by one exact model route. */
interface ImageRequestPolicy {
@@ -118,8 +110,6 @@ interface RequestImageAttachment {
variantId: ImageVariantId
/** Durable normalized attachment from which this request version was derived. */
attachment: ImageAttachmentRef
/** Transient provider access facts; never persisted in the durable reference. */
access?: ImageAttachmentAccess
/** Encoded request bytes. */
data: Uint8Array
mediaType: ImageMediaType
@@ -135,7 +125,7 @@ interface RequestImageAttachment {
}
```
`saveImage()` prepares and atomically commits a provider-independent normalized attachment before returning its `ImageAttachmentRef`. `saveImages()` prepares every validated attachment once before publishing the batch, so validation rejection leaves no partial objects and publication does not repeat decoding or quality selection. `admitEncodedImages()` is the wire entry for base64 uploads and delegates count, aggregate-byte, and ordered batch admission to `saveImages()`. `readImage()` verifies a normalized attachment from an authorized session path. `imageAccess()` resolves current-provider access facts without storing host paths in session data. `readImageRequest()` derives and caches one request version under an exact route pixel and byte budget; new entries are fully decoded before publication, while cache hits use a bounded metadata probe. Callers use `Promise.all` over the singular method when they need an ordered batch. The local implementation lazily encodes preferred candidates, singleflights equal request identities, lets each waiter cancel independently, stops shared work when no waiter remains, and bounds all transforms with its instance-level limiter, which defaults to two simultaneous transformations. The service is retention-neutral: resumed and forked sessions may share objects, so reference-aware garbage collection is deferred rather than tied to one session's deletion.
`saveImage()` prepares and atomically commits a provider-independent normalized attachment before returning its `ImageAttachmentRef`. `saveImages()` prepares every validated attachment once before publishing the batch, so validation rejection leaves no partial objects and publication does not repeat decoding or quality selection. `admitEncodedImages()` is the wire entry for base64 uploads and delegates count, aggregate-byte, and ordered batch admission to `saveImages()`. `readImage()` verifies a normalized attachment from an authorized session path. `imageHostPath()` exposes only the provider-owned host object location; it does not decide whether the current tool execution world can read it. `readImageRequest()` derives and caches one deterministic request version under an exact route pixel and byte budget. That version contains encoded bytes and metadata but no execution-world path. New entries are fully decoded before publication, while cache hits use a bounded metadata probe. Callers use `Promise.all` over the singular method when they need an ordered batch. The local implementation lazily encodes preferred candidates, singleflights equal request identities, lets each waiter cancel independently, stops shared work when no waiter remains, and bounds all transforms with its instance-level limiter, which defaults to two simultaneous transformations. The service is retention-neutral: resumed and forked sessions may share objects, so reference-aware garbage collection is deferred rather than tied to one session's deletion.
<!-- BEGIN GENERATED cordis-surface (gen-cordis-catalog.ts) — do not edit between markers -->
@@ -187,12 +177,12 @@ abstract saveImage(input: SaveImageAttachment): Promise<ImageAttachmentRef>
abstract readImage(ref: ImageAttachmentRef, signal?: AbortSignal): Promise<StoredImageAttachment>
/**
* Resolve provider-specific model-tool access without adding host facts to session history.
* Locate the provider-owned normalized object in the harness host filesystem.
* @param ref - durable normalized attachment reference.
* @returns current-provider access facts, or undefined when this backend exposes no local path.
* @returns an absolute host path, or undefined when this backend is not host-file-backed.
* @throws an AttachmentError when the durable reference is invalid.
*/
imageAccess(ref: ImageAttachmentRef): ImageAttachmentAccess | undefined
imageHostPath(ref: ImageAttachmentRef): string | undefined
/**
* Generate or read one deterministic model-request version from the stored normalized image.
+5 -15
View File
@@ -10,7 +10,7 @@
## 标识与经过校验的元数据
`AttachmentId` 是带类型标记的不透明字符串。本地后端目前生成 `sha256:<digest>`,但消费方既不能解析这种表示,也不能据此派生文件系统路径。消费方需要路径时调用 `imageAccess()`,由当前挂载的提供方负责解析
`AttachmentId` 是带类型标记的不透明字符串。本地后端目前生成 `sha256:<digest>`,但消费方既不能解析这种表示,也不能据此派生文件系统路径。消费方可以通过 `imageHostPath()` 询问附件提供方所持对象的位置,然后必须由当前执行文件系统判断模型工具能否读取该宿主路径
```ts type-equiv
/** Raster image formats accepted by the version-one attachment path. */
@@ -93,14 +93,6 @@ interface StoredImageAttachment {
}
```
```ts type-equiv
/** Provider-resolved ways for model tools to access one normalized attachment. */
interface ImageAttachmentAccess {
/** Absolute path to immutable normalized bytes; callers must treat it as read-only. */
readonlyPath: string
}
```
```ts type-equiv
/** Deterministic request-image policy selected by one exact model route. */
interface ImageRequestPolicy {
@@ -118,8 +110,6 @@ interface RequestImageAttachment {
variantId: ImageVariantId
/** Durable normalized attachment from which this request version was derived. */
attachment: ImageAttachmentRef
/** Transient provider access facts; never persisted in the durable reference. */
access?: ImageAttachmentAccess
/** Encoded request bytes. */
data: Uint8Array
mediaType: ImageMediaType
@@ -135,7 +125,7 @@ interface RequestImageAttachment {
}
```
`saveImage()` 准备并原子提交提供方无关的规范化附件,然后直接返回 `ImageAttachmentRef`。`saveImages()` 在发布批次前为每个成员各准备一次经过验证的附件,因此校验拒绝不会留下部分对象,发布也不会重复解码或选择质量。`admitEncodedImages()` 是面向 base64 上传的 wire 入口,把张数、聚合字节和有序批量准入交给 `saveImages()`。`readImage()` 校验来自已授权会话路径的规范化附件。`imageAccess()` 解析当前提供方的访问信息,无需把宿主路径写入会话数据。`readImageRequest()` 按确切路由的像素和字节预算派生并缓存请求版本新条目在发布前完整解码,缓存命中只做有界元数据探测。调用方需要有序批次时,对单数方法使用 `Promise.all`。本地实现按需编码首选候选、合并相同请求身份的并发任务、允许每个等待方单独取消、没有等待方时停止共享任务,并通过实例级限流器限制全部变换,默认同时执行两项。该服务不规定保留策略:恢复和 fork 后的会话可能共享对象,因此基于引用的垃圾回收会延期实现,不与单个会话的删除绑定。
`saveImage()` 准备并原子提交提供方无关的规范化附件,然后直接返回 `ImageAttachmentRef`。`saveImages()` 在发布批次前为每个成员各准备一次经过验证的附件,因此校验拒绝不会留下部分对象,发布也不会重复解码或选择质量。`admitEncodedImages()` 是面向 base64 上传的 wire 入口,把张数、聚合字节和有序批量准入交给 `saveImages()`。`readImage()` 校验来自已授权会话路径的规范化附件。`imageHostPath()` 只公开提供方所持对象的宿主位置,不判断当前工具执行环境能否读取它。`readImageRequest()` 按确切路由的像素和字节预算派生并缓存确定性请求版本。该版本包含编码字节和元数据,不包含执行环境路径。新条目在发布前完整解码,缓存命中只做有界元数据探测。调用方需要有序批次时,对单数方法使用 `Promise.all`。本地实现按需编码首选候选、合并相同请求身份的并发任务、允许每个等待方单独取消、没有等待方时停止共享任务,并通过实例级限流器限制全部变换,默认同时执行两项。该服务不规定保留策略:恢复和 fork 后的会话可能共享对象,因此基于引用的垃圾回收会延期实现,不与单个会话的删除绑定。
<!-- BEGIN GENERATED cordis-surface (gen-cordis-catalog.ts) — do not edit between markers -->
@@ -187,12 +177,12 @@ abstract saveImage(input: SaveImageAttachment): Promise<ImageAttachmentRef>
abstract readImage(ref: ImageAttachmentRef, signal?: AbortSignal): Promise<StoredImageAttachment>
/**
* Resolve provider-specific model-tool access without adding host facts to session history.
* Locate the provider-owned normalized object in the harness host filesystem.
* @param ref - durable normalized attachment reference.
* @returns current-provider access facts, or undefined when this backend exposes no local path.
* @returns an absolute host path, or undefined when this backend is not host-file-backed.
* @throws an AttachmentError when the durable reference is invalid.
*/
imageAccess(ref: ImageAttachmentRef): ImageAttachmentAccess | undefined
imageHostPath(ref: ImageAttachmentRef): string | undefined
/**
* Generate or read one deterministic model-request version from the stored normalized image.
+2 -2
View File
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write docs/subsystems/llm-streaming.md
llm-streaming.md: 4ad7af5673f3894d86af72b04db365fcc0f36608
llm-streaming.zh.md: ff62bccdac018ea57b58d5edec9b7dae448be76a
llm-streaming.md: bdc830a5d387cde6967575551ec9b0a9b2626f46
llm-streaming.zh.md: b602336bc06cd88a2634f5259eff117da3dcd986
+12
View File
@@ -30,6 +30,18 @@ interface ContentBlockMap {
The block interfaces (full fields in source): `TextBlock` (`text`), `ReasoningBlock` (thinking, distinct from visible text), `ImageBlock` (a durable [image attachment](attachment.md)), `ToolCallBlock` (`id: CallId`, `name`, raw-JSON `arguments`), and `ToolResultBlock` (`toolCallId`, nested `content: ContentBlock[]`, `isError?`). `ContentBlock = ContentBlockMap[ContentBlockType]`. A new modality belongs in the merge-extensible map only when its adapter, UI, compaction, and durable replay paths honor it.
Image access belongs to request serialization rather than the durable attachment or deterministic request-image version. `resolveImageAttachmentAccess()` combines the attachment provider's optional host object path with a mapping supplied by the consumer for the current tool execution filesystem. The result is available only for that request and does not participate in `variantId`.
Source: [`packages/llm/llm/src/content.ts`](../../packages/llm/llm/src/content.ts)
```ts type-equiv
/** Execution-world path that model tools can use to read one normalized attachment. */
interface ImageAttachmentAccess {
/** Absolute path to immutable normalized bytes; callers must treat it as read-only. */
readonlyPath: string
}
```
Source: [`packages/llm/llm/src/message.ts`](../../packages/llm/llm/src/message.ts)
A `Message` is one identified, immutable role/source/content value. Model-produced assistant messages name the provider and model that produced them and carry optional adapter-private replay data in their source:
+12
View File
@@ -30,6 +30,18 @@ interface ContentBlockMap {
各块接口(完整字段见源码):`TextBlock``text`)、`ReasoningBlock`(thinking,区别于可见文本)、`ImageBlock`(一个持久的[图片附件](attachment.zh.md))、`ToolCallBlock``id: CallId`、`name`、原始 JSON `arguments`),以及 `ToolResultBlock``toolCallId`、嵌套 `content: ContentBlock[]`、`isError?`)。`ContentBlock = ContentBlockMap[ContentBlockType]`。仅当适配器、UI、压缩(compaction)和持久回放路径均支持某种新模态时,才将其纳入可合并扩展的 map。
图片访问方式属于请求序列化,不属于持久附件或确定性请求图片版本。`resolveImageAttachmentAccess()` 把附件提供方可选的宿主对象路径,与消费方为当前工具执行文件系统提供的映射组合起来。结果只适用于本次请求,不参与 `variantId`。
源码:[`packages/llm/llm/src/content.ts`](../../packages/llm/llm/src/content.ts)
```ts type-equiv
/** Execution-world path that model tools can use to read one normalized attachment. */
interface ImageAttachmentAccess {
/** Absolute path to immutable normalized bytes; callers must treat it as read-only. */
readonlyPath: string
}
```
源码:[`packages/llm/llm/src/message.ts`](../../packages/llm/llm/src/message.ts)
`Message` 是一个带标识且不可变的角色/来源/内容值。模型生成的 assistant 消息会在来源中记录生成它的提供方和模型,以及可选的适配器私有回放数据:
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/attachment/attachment-local/README.md
README.md: c93f32ede4599d8e07536062dd7d44134bea16b0
README.zh.md: f06c3aeec937e0eb09ea0aa2ef25d41119e54aa8
README.md: d9b6839c9b02fdfa4a0661ed44ae77e32e9a255e
README.zh.md: 11b4a6239e8b14ecfcc905e40a4cdc5042de5fcf
@@ -8,15 +8,15 @@ Admission accepts at most 20 images and 200MiB of encoded source bytes per messa
Request versions live below `<DSH_HOME>/attachments/v1/request-images/`. `readImageRequest` scales the stored normalized attachment under a total-pixel budget without enlargement, then enforces a separate encoded-byte cap. The request encoder uses the same color branches, with PNG (palette only without alpha) before WebP 85 and 80 for low-color images, WebP 85 then 80 for other alpha images, and JPEG 85 then 80 for other opaque images. It executes candidates lazily and reduces dimensions only after both quality attempts exceed the request cap. Its cache identity includes the attachment id, transform version, pixel and byte budgets, and fixed encoder settings. Cached bytes are fully decoded and checked as 8-bit sRGB/sRGBA before use. Concurrent calls for one identity share one transform and cache write; cancelling one waiter does not cancel the shared work. Callers compose ordered batches from singular reads, while the service's FIFO limiter applies `imageCompressionConcurrency` to simultaneous normalization and request transforms. The setting ranges from 1 through 8 and defaults to 2; file publication remains ordered after preparation.
`DSH_HOME` resolves through the shared path policy: explicit config, `$DSH_HOME`, then `~/.dsh`. Session logs contain only the reference and verified metadata. At request assembly, `imageAccess` derives the normalized object's absolute host path and asks the mounted `ctx.fs` to map that file into its execution world. Host-backed filesystems return a read-only process path; remote filesystems without a shared mount return no path. The path is absent from durable history. `readImage` forwards optional cancellation into the filesystem read, observes it around verification, and preserves it instead of wrapping it as `ATTACHMENT_READ_FAILED`.
`DSH_HOME` resolves through the shared path policy: explicit config, `$DSH_HOME`, then `~/.dsh`. Session logs contain only the reference and verified metadata. `imageHostPath` derives the normalized object's absolute host path and does not inspect the tool execution world. At request assembly, an LLM consumer asks the mounted filesystem to map that host object into its execution world. A host-backed filesystem returns a process path; a remote filesystem without a shared mount returns no path. The mapped path is absent from durable history and from `RequestImageAttachment`. `readImage` forwards optional cancellation into the filesystem read, observes it around verification, and preserves it instead of wrapping it as `ATTACHMENT_READ_FAILED`.
## Model Experience
Indirectly, through request descriptors that give the model each retained or offloaded image's identity, dimensions, media type, available read-only normalized-object path, matching extension for a writable copy, and a warning that normalization may have resized or re-encoded the upload.
Indirectly, through request descriptors. When the current execution filesystem maps this backend's host object, the model receives each retained or offloaded image's identity, dimensions, media type, read-only mapped path, matching extension for a writable copy, and a warning that normalization may have resized or re-encoded the upload.
#### KV Cache effect
Normalization and request projection are deterministic. An unchanged attachment and route policy reuse identical cached request bytes on later turns.
Normalization and request projection are deterministic. An unchanged attachment and route policy reuse identical cached request bytes on later turns. Execution-world path mapping is resolved separately and can change historical descriptor text without changing those bytes or their `variantId`.
## Known Limitations and Deferred Work
@@ -8,15 +8,15 @@
请求版本保存在 `<DSH_HOME>/attachments/v1/request-images/``readImageRequest` 在不放大小图的前提下,把存储的规范化附件缩放到总像素预算内,再执行独立的编码字节上限。请求编码器使用同一分类分支:低色数图片先尝试 PNG,只有不带 alpha 通道时才使用 palette,再尝试质量 85 和 80 的 WebP;其他透明图片依次尝试质量 85 和 80 的 WebP;其他非透明图片依次尝试质量 85 和 80 的 JPEG。候选按需执行,两个质量档均超限后才缩小尺寸。缓存身份包含附件 ID、变换策略版本、像素和字节预算及固定编码参数。缓存字节在使用前会完整解码并校验为 8-bit sRGB/sRGBA。同一身份的并发调用共享一次变换和缓存写入;取消一个等待方不会取消共享任务。调用方组合单数读取得到有序批次,服务的 FIFO 限流器通过 `imageCompressionConcurrency` 限制同时执行的规范化和请求变换。该配置范围为 1 至 8,默认值为 2;文件发布仍在准备结束后按顺序执行。
`DSH_HOME` 按共享路径策略解析:显式配置、`$DSH_HOME`,最后是 `~/.dsh`。会话日志只包含引用和经过校验的元数据。组装请求时,`imageAccess` 派生规范化对象的绝对宿主路径,再要求当前 `ctx.fs` 把该文件映射到其执行世界。宿主文件系统返回只读进程路径;没有共享挂载的远程文件系统不返回路径。路径不进入持久历史。`readImage` 会把可选取消信号传入文件系统读取、在校验前后观察该信号,并保留取消语义,而不会将其包装成 `ATTACHMENT_READ_FAILED`
`DSH_HOME` 按共享路径策略解析:显式配置、`$DSH_HOME`,最后是 `~/.dsh`。会话日志只包含引用和经过校验的元数据。`imageHostPath` 派生规范化对象的绝对宿主路径,不检查工具执行环境。组装请求时,LLM 消费方要求当前文件系统把该宿主对象映射到其执行环境。宿主文件系统返回进程路径;没有共享挂载的远程文件系统不返回路径。映射后的路径不进入持久历史,也不进入 `RequestImageAttachment``readImage` 会把可选取消信号传入文件系统读取、在校验前后观察该信号,并保留取消语义,而不会将其包装成 `ATTACHMENT_READ_FAILED`
## 模型体验
该包通过请求描述间接影响模型,描述会给出每张保留或被 offload 图片的身份、尺寸、媒体类型、当前可用的只读规范化对象路径、复制到可写位置时使用的匹配扩展名,以及规范化过程可能缩小或重新编码上传图片的提醒。
该包通过请求描述间接影响模型。当前执行文件系统能够映射本后端的宿主对象时,描述会给出每张保留或被 offload 图片的身份、尺寸、媒体类型、映射后的只读路径、复制到可写位置时使用的匹配扩展名,以及规范化过程可能缩小或重新编码上传图片的提醒。
#### KV 缓存影响
规范化和请求投影都是确定性的。附件和路由策略不变时,之后各轮会复用相同的缓存请求字节。
规范化和请求投影都是确定性的。附件和路由策略不变时,之后各轮会复用相同的缓存请求字节。执行环境路径单独解析;它的映射变化会改变历史描述文本,但不会改变请求字节或 `variantId`
## 已知限制与待完成工作
@@ -23,7 +23,6 @@
"license": "MIT",
"peerDependencies": {
"@deepseek-ai/dsh-attachment": "workspace:^",
"@deepseek-ai/dsh-fs": "workspace:^",
"@deepseek-ai/dsh-invariants": "workspace:^",
"@deepseek-ai/dsh-home-paths": "workspace:^",
"@deepseek-ai/cordis": "workspace:^"
@@ -34,8 +33,6 @@
},
"devDependencies": {
"@deepseek-ai/dsh-attachment": "workspace:^",
"@deepseek-ai/dsh-fs": "workspace:^",
"@deepseek-ai/dsh-fs-local": "workspace:^",
"@deepseek-ai/dsh-invariants": "workspace:^",
"@deepseek-ai/dsh-home-paths": "workspace:^",
"@deepseek-ai/cordis": "workspace:^"
@@ -5,7 +5,6 @@ import { Context } from '@deepseek-ai/cordis'
import z from '@deepseek-ai/schemastery'
import { AttachmentStore } from '@deepseek-ai/dsh-attachment'
import type {
ImageAttachmentAccess,
ImageAttachmentLimits,
ImageAttachmentRef,
ImageRequestPolicy,
@@ -13,7 +12,6 @@ import type {
SaveImageAttachment,
StoredImageAttachment,
} from '@deepseek-ai/dsh-attachment'
import type {} from '@deepseek-ai/dsh-fs'
import { resolveDshHome } from '@deepseek-ai/dsh-home-paths'
import type { NormalizationPolicy } from './normalization.ts'
import { CompressionLimiter } from './compression-limiter.ts'
@@ -209,10 +207,8 @@ export class LocalAttachmentStore extends AttachmentStore {
return readImageFile(this.root, ref, signal)
}
override imageAccess(ref: ImageAttachmentRef): ImageAttachmentAccess | undefined {
const hostPath = normalizedImagePath(this.root, ref)
const readonlyPath = this.ctx.get('fs')?.processPathFromHostPath(hostPath)
return readonlyPath === undefined ? undefined : { readonlyPath }
override imageHostPath(ref: ImageAttachmentRef): string {
return normalizedImagePath(this.root, ref)
}
override async readImageRequest(
@@ -245,8 +241,7 @@ export class LocalAttachmentStore extends AttachmentStore {
policy,
sharedSignal,
)
const access = this.imageAccess(ref)
return { ...request, ...(access === undefined ? {} : { access }) }
return request
}))
operation = shared
this.requestInflight.set(key, shared)
@@ -1,6 +1,5 @@
import { Context } from '@deepseek-ai/cordis'
import { AttachmentId } from '@deepseek-ai/dsh-attachment'
import { LocalFileSystem } from '@deepseek-ai/dsh-fs-local'
import { existsSync } from 'node:fs'
import { mkdtemp, readFile, rm } from 'node:fs/promises'
import { tmpdir } from 'node:os'
@@ -46,8 +45,13 @@ describe('local attachment service', () => {
width: 1,
height: 1,
}
expect(service.imageAccess(ref)).toBeUndefined()
expect(() => service.imageAccess({ ...ref, attachmentId: AttachmentId('invalid') }))
expect(service.imageHostPath(ref)).toBe(join(
service.root,
'objects',
'aa',
'a'.repeat(64),
))
expect(() => service.imageHostPath({ ...ref, attachmentId: AttachmentId('invalid') }))
.toThrow(expect.objectContaining({ code: 'INVALID_ATTACHMENT_REF' }))
})
@@ -61,20 +65,16 @@ describe('local attachment service', () => {
it('saves and reads through the service boundary', async () => {
const dshHome = await mkdtemp(join(tmpdir(), 'dsh-attachment-service-'))
const ctx = new Context()
new LocalFileSystem(ctx, { cwd: dshHome, diffBasisMaxBytes: 10 * 1024 * 1024 })
try {
const service = new LocalAttachmentStore(ctx, { dshHome })
const service = new LocalAttachmentStore(new Context(), { dshHome })
const data = Uint8Array.from(Buffer.from(
'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAACXBIWXMAAAPoAAAD6AG1e1JrAAAADElEQVQImWNgZGIGAAAOAAeCcsnOAAAAAElFTkSuQmCC',
'base64',
))
const ref = await service.saveImage({ data, mediaType: 'image/png' })
await expect(service.readImage(ref)).resolves.toEqual({ ref, data })
const access = service.imageAccess(ref)
expect(access).toBeDefined()
if (access === undefined) throw new Error('expected a host-path mapping from fs-local')
expect(access.readonlyPath).toBe(join(
const hostPath = service.imageHostPath(ref)
expect(hostPath).toBe(join(
dshHome,
'attachments',
'v1',
@@ -82,9 +82,9 @@ describe('local attachment service', () => {
String(ref.attachmentId).slice('sha256:'.length, 'sha256:'.length + 2),
String(ref.attachmentId).slice('sha256:'.length),
))
await expect(readFile(access.readonlyPath)).resolves.toEqual(Buffer.from(data))
await expect(readFile(hostPath)).resolves.toEqual(Buffer.from(data))
const request = await service.readImageRequest(ref, { maxPixels: 1, maxBytes: 1024 })
expect(request.access).toEqual(access)
expect(request).not.toHaveProperty('access')
} finally {
await rm(dshHome, { recursive: true, force: true })
}
@@ -6,7 +6,6 @@
{ "path": "../../../vendor/cosmokit" },
{ "path": "../../../vendor/cordis" },
{ "path": "../attachment" },
{ "path": "../../fs/fs" },
{ "path": "../../util/home-paths" },
{ "path": "../../runtime-diagnostics/invariants" }
]
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/attachment/attachment/README.md
README.md: 63c6a41fca3d0d7ea87d9f30e682e7e4e7aa5906
README.zh.md: 4eb08dd169e1d0f2b20ada050513caaf88a84891
README.md: 21030a492464aae528d4c06b4b72d4a94c0a2603
README.zh.md: 0540996f99b3250331e567e174264cf7da8aa474
+3 -3
View File
@@ -4,17 +4,17 @@ English | [中文](README.zh.md)
The durable attachment seam. `ctx.attachments` validates and durably commits a provider-independent normalized image, then returns a serializable `ImageAttachmentRef`; consumers never persist browser paths, object URLs, provider URLs, local storage paths, or base64 in session events.
Unsent composer images remain browser-owned temporary drafts. `validateImage` runs the complete admission policy without persisting. `saveImages` owns batch count and aggregate-byte limits, prepares every normalized attachment before publishing any member, then commits in order and returns references only after the complete batch succeeds. A later storage failure returns no partial references, although an earlier immutable content-addressed object may remain unreachable until reference-aware garbage collection exists. `AttachmentError.code` uses the closed `AttachmentErrorCode` string union. Its `ImageAdmissionErrorCode` subset marks caller-correctable image-input failures; `isImageAdmissionError` recognizes that subset at runtime so each protocol adapter can map its own error vocabulary. `saveImage` commits one accepted image before any model-visible session event is published and returns its `ImageAttachmentRef`. When normalization reduces the raster, the reference records the orientation-applied input size in `originalDimensions`. `readImage` verifies the normalized attachment against its logged metadata. `readImageRequest` deterministically derives a route-sized request version whose identity covers the attachment id, transform version, pixel and byte budgets, and encoder settings. `imageAccess` resolves optional current-provider access facts, such as an absolute read-only path, without adding host-specific data to the durable reference. Callers compose ordered batches with `Promise.all(refs.map(...))`; the local implementation still bounds compression through its instance limiter, cache, and singleflight. Callers may cancel reads and projections; implementations preserve cancellation instead of translating it into a storage failure.
Unsent composer images remain browser-owned temporary drafts. `validateImage` runs the complete admission policy without persisting. `saveImages` owns batch count and aggregate-byte limits, prepares every normalized attachment before publishing any member, then commits in order and returns references only after the complete batch succeeds. A later storage failure returns no partial references, although an earlier immutable content-addressed object may remain unreachable until reference-aware garbage collection exists. `AttachmentError.code` uses the closed `AttachmentErrorCode` string union. Its `ImageAdmissionErrorCode` subset marks caller-correctable image-input failures; `isImageAdmissionError` recognizes that subset at runtime so each protocol adapter can map its own error vocabulary. `saveImage` commits one accepted image before any model-visible session event is published and returns its `ImageAttachmentRef`. When normalization reduces the raster, the reference records the orientation-applied input size in `originalDimensions`. `readImage` verifies the normalized attachment against its logged metadata. `readImageRequest` deterministically derives a route-sized request version whose identity covers the attachment id, transform version, pixel and byte budgets, and encoder settings. `imageHostPath` optionally exposes the provider-owned object's absolute host path; it makes no claim that the current model tools can read that path. An LLM consumer combines this location with the mounted filesystem's execution-world mapping when it serializes a request. That current access path remains separate from the request version and its `variantId`. Callers compose ordered batches with `Promise.all(refs.map(...))`; the local implementation still bounds compression through its instance limiter, cache, and singleflight. Callers may cancel reads and projections; implementations preserve cancellation instead of translating it into a storage failure.
`admitEncodedImages(attachments, images)` is the shared wire entry used by every RPC endpoint that accepts browser uploads (the session prompt endpoint and the command executor): it enforces canonical base64 on every member, then delegates batch admission — limits, validation, ordered commit — to `saveImages`. The base64 upload form is `EncodedImageAttachment`, exported from `@deepseek-ai/dsh-attachment/types` so wire contracts can reference it.
## Model Experience
Indirectly, through the role-neutral core `ImageBlock` and provider adapters that resolve its durable reference into an exact request version. Request descriptors expose the complete attachment id and actual request dimensions. A backend that offers model-tool access can also expose its current read-only normalized path; the descriptor states that normalization may have resized or re-encoded the upload.
Indirectly, through the role-neutral core `ImageBlock` and provider adapters that resolve its durable reference into an exact request version. Request descriptors expose the complete attachment id and actual request dimensions. When the attachment backend exposes a host object and the current execution filesystem maps it, the descriptor also exposes the resulting read-only path; it states that normalization may have resized or re-encoded the upload.
#### KV Cache effect
Adding an image changes the provider request and therefore invalidates the affected request suffix.
Adding an image changes the provider request and therefore invalidates the affected request suffix. A changed execution-world path can also change historical descriptor text without changing the deterministic request version.
## Known Limitations and Deferred Work
+3 -3
View File
@@ -4,17 +4,17 @@
持久附件服务边界。`ctx.attachments` 校验并持久提交提供方无关的规范化图片,随后返回可序列化的 `ImageAttachmentRef`;消费方绝不会在会话事件中持久保存浏览器路径、对象 URL、提供方 URL、本地存储路径或 base64。
未发送的输入区图片仍是由浏览器持有的临时草稿。`validateImage` 运行完整准入策略但不执行持久化。`saveImages` 负责批次图片数量和总字节限制,在发布任何成员前准备全部规范化附件,然后按顺序提交,并且只在完整批次成功后返回引用。后续存储失败不会返回部分引用,但较早写入的不可变内容寻址对象可能保持不可达,直至具备按引用感知的垃圾回收。`AttachmentError.code` 使用封闭的 `AttachmentErrorCode` 字符串联合类型。其 `ImageAdmissionErrorCode` 子集标记可由调用方修正的图片输入失败;`isImageAdmissionError` 在运行时识别该子集,使每个协议适配器可以映射自己的错误词汇。`saveImage` 会在发布任何模型可见的会话事件前提交一张已接受的图片,并直接返回 `ImageAttachmentRef`。规范化过程缩小图片时,引用会通过 `originalDimensions` 记录应用方向后的输入尺寸。`readImage` 根据已记录的元数据校验规范化附件。`readImageRequest` 确定性派生路由所需的请求版本,其身份覆盖附件 ID、变换策略版本、像素和字节预算及编码参数。`imageAccess` 解析当前提供方可选的访问事实,例如绝对只读路径,同时避免把宿主信息加入持久引用。调用方通过 `Promise.all(refs.map(...))` 组合有序批次,本地实现仍通过实例级限流器、缓存和 singleflight 限制压缩并发。调用方可以取消读取和投影;实现保留取消结果,不把它转换为存储失败。
未发送的输入区图片仍是由浏览器持有的临时草稿。`validateImage` 运行完整准入策略但不执行持久化。`saveImages` 负责批次图片数量和总字节限制,在发布任何成员前准备全部规范化附件,然后按顺序提交,并且只在完整批次成功后返回引用。后续存储失败不会返回部分引用,但较早写入的不可变内容寻址对象可能保持不可达,直至具备按引用感知的垃圾回收。`AttachmentError.code` 使用封闭的 `AttachmentErrorCode` 字符串联合类型。其 `ImageAdmissionErrorCode` 子集标记可由调用方修正的图片输入失败;`isImageAdmissionError` 在运行时识别该子集,使每个协议适配器可以映射自己的错误词汇。`saveImage` 会在发布任何模型可见的会话事件前提交一张已接受的图片,并直接返回 `ImageAttachmentRef`。规范化过程缩小图片时,引用会通过 `originalDimensions` 记录应用方向后的输入尺寸。`readImage` 根据已记录的元数据校验规范化附件。`readImageRequest` 确定性派生路由所需的请求版本,其身份覆盖附件 ID、变换策略版本、像素和字节预算及编码参数。`imageHostPath` 可以给出提供方所持对象的绝对宿主路径,但不保证当前模型工具能够读取它。LLM 消费方在序列化请求时将这个位置与当前文件系统提供的执行环境映射组合起来。解析出的访问路径独立于请求版本及其 `variantId`。调用方通过 `Promise.all(refs.map(...))` 组合有序批次,本地实现仍通过实例级限流器、缓存和 singleflight 限制压缩并发。调用方可以取消读取和投影;实现保留取消结果,不把它转换为存储失败。
`admitEncodedImages(attachments, images)` 是每个接受浏览器上传的 RPC 端点(会话 prompt 端点与命令执行器)共用的 wire 入口:它对每个成员强制执行规范 base64,随后把批量准入——限额、校验、有序提交——委托给 `saveImages`。base64 上传形式为 `EncodedImageAttachment`,从 `@deepseek-ai/dsh-attachment/types` 导出,供 wire 契约引用。
## 模型体验
该包通过角色无关的核心 `ImageBlock`,以及把持久引用解析为确定请求版本的提供方适配器,间接影响模型。请求描述会公开完整附件 ID 和实际请求尺寸。支持模型工具访问的后端还可以公开当前规范化附件的只读路径;描述会说明规范化过程可能缩小或重新编码上传图片。
该包通过角色无关的核心 `ImageBlock`,以及把持久引用解析为确定请求版本的提供方适配器,间接影响模型。请求描述会公开完整附件 ID 和实际请求尺寸。附件后端给出宿主对象且当前执行文件系统能够映射该对象时,描述还会公开映射后的只读路径,并说明规范化过程可能缩小或重新编码上传图片。
#### KV 缓存影响
添加图片会改变提供方请求,因此会使受影响的请求后缀失效。
添加图片会改变提供方请求,因此会使受影响的请求后缀失效。即使确定性的请求版本不变,执行环境路径变化也会改变历史描述文本。
## 已知限制与待完成工作
+3 -5
View File
@@ -4,7 +4,6 @@ import { Context, Service } from '@deepseek-ai/cordis'
import { AttachmentError } from './error.ts'
import type {
ImageAttachmentLimits,
ImageAttachmentAccess,
ImageAttachmentRef,
ImageRequestPolicy,
RequestImageAttachment,
@@ -19,7 +18,6 @@ export { admitEncodedImages } from './admission.ts'
export type {
AttachmentId as AttachmentIdType,
EncodedImageAttachment,
ImageAttachmentAccess,
ImageAttachmentLimits,
ImageAttachmentRef,
ImageRequestPolicy,
@@ -110,12 +108,12 @@ export abstract class AttachmentStore extends Service {
abstract readImage(ref: ImageAttachmentRef, signal?: AbortSignal): Promise<StoredImageAttachment>
/**
* Resolve provider-specific model-tool access without adding host facts to session history.
* Locate the provider-owned normalized object in the harness host filesystem.
* @param ref - durable normalized attachment reference.
* @returns current-provider access facts, or undefined when this backend exposes no local path.
* @returns an absolute host path, or undefined when this backend is not host-file-backed.
* @throws an AttachmentError when the durable reference is invalid.
*/
imageAccess(ref: ImageAttachmentRef): ImageAttachmentAccess | undefined {
imageHostPath(ref: ImageAttachmentRef): string | undefined {
void ref
return undefined
}
@@ -67,12 +67,6 @@ export interface StoredImageAttachment {
data: Uint8Array
}
/** Provider-resolved ways for model tools to access one normalized attachment. */
export interface ImageAttachmentAccess {
/** Absolute path to immutable normalized bytes; callers must treat it as read-only. */
readonlyPath: string
}
/** Deterministic request-image policy selected by one exact model route. */
export interface ImageRequestPolicy {
/** Maximum width multiplied by height after aspect-preserving projection. */
@@ -87,8 +81,6 @@ export interface RequestImageAttachment {
variantId: ImageVariantId
/** Durable normalized attachment from which this request version was derived. */
attachment: ImageAttachmentRef
/** Transient provider access facts; never persisted in the durable reference. */
access?: ImageAttachmentAccess
/** Encoded request bytes. */
data: Uint8Array
mediaType: ImageMediaType
@@ -147,10 +147,10 @@ describe('AttachmentStore.readImageRequest', () => {
expect(() => store.readImageRequest(ref, { maxPixels: 1, maxBytes: 1 }, controller.signal)).toThrow(reason)
})
it('exposes no provider-specific path by default', async () => {
it('exposes no provider-owned host path by default', async () => {
const store = new RecordingStore(new Context())
const ref = await store.saveImage(image(1))
expect(store.imageAccess(ref)).toBeUndefined()
expect(store.imageHostPath(ref)).toBeUndefined()
})
})
@@ -450,10 +450,10 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [
throws: ['the signal reason when aborted, or a storage error when verification fails.'],
},
{
signature: 'imageAccess(ref: ImageAttachmentRef): ImageAttachmentAccess | undefined',
description: 'Resolve provider-specific model-tool access without adding host facts to session history.',
signature: 'imageHostPath(ref: ImageAttachmentRef): string | undefined',
description: 'Locate the provider-owned normalized object in the harness host filesystem.',
parameters: [{ name: 'ref', description: 'durable normalized attachment reference.' }],
returns: 'current-provider access facts, or undefined when this backend exposes no local path.',
returns: 'an absolute host path, or undefined when this backend is not host-file-backed.',
throws: ['an AttachmentError when the durable reference is invalid.'],
},
{
@@ -3726,10 +3726,6 @@ export const TYPE_API: readonly TypeApiEntry[] = [
name: 'GrantRecord',
declaration: 'export interface GrantRecord {\n readonly kind: \'grant\';\n readonly payload: unknown;\n}',
},
{
name: 'ImageAttachmentAccess',
declaration: 'export interface ImageAttachmentAccess {\n readonlyPath: string;\n}',
},
{
name: 'ImageAttachmentLimits',
declaration: 'export interface ImageAttachmentLimits {\n maxImageBytes: number;\n maxImagesPerMessage: number;\n maxMessageImageBytes: number;\n maxImagePixels: number;\n maxImageDimension: number;\n mediaTypes: readonly ImageMediaType[];\n}',
@@ -4236,7 +4232,7 @@ export const TYPE_API: readonly TypeApiEntry[] = [
},
{
name: 'RequestImageAttachment',
declaration: 'export interface RequestImageAttachment {\n variantId: ImageVariantId;\n attachment: ImageAttachmentRef;\n access?: ImageAttachmentAccess;\n data: Uint8Array;\n mediaType: ImageMediaType;\n bytes: number;\n width: number;\n height: number;\n depth: \'uchar\';\n space: \'srgb\';\n hasAlpha: boolean;\n}',
declaration: 'export interface RequestImageAttachment {\n variantId: ImageVariantId;\n attachment: ImageAttachmentRef;\n data: Uint8Array;\n mediaType: ImageMediaType;\n bytes: number;\n width: number;\n height: number;\n depth: \'uchar\';\n space: \'srgb\';\n hasAlpha: boolean;\n}',
},
{
name: 'RequestRunOutcome',
+2 -2
View File
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/llm/llm-deepseek/README.md
README.md: 31aa82dcc0dde68b7ed23edcd0dc477320d55505
README.zh.md: dad1dbcea639130ff55430684d6478a86717154c
README.md: 629a1ebe00e5b14ac575a7b088a2e737e89d0c07
README.zh.md: d5f11f8f2f8b9053685ba553a69fe71e40458ca3
+3 -3
View File
@@ -52,7 +52,7 @@ The package root exposes the Cordis plugin contract and `DeepSeekAdapter`; wire
The plugin registers the single provider route `deepseek-official` together with its resolved `retryPolicy`; omission resolves to normal mode with five retries. A request selects it with `provider: deepseek-official`; its `model` is passed through as the wire `model` string, so changing DeepSeek models does not require lifecycle-time registration. Omitting `models` advertises `deepseek-v4-flash`, `deepseek-v4-pro`, and the image-capable `deepseek-v4-flash-vision-exp`, each with a 1,000,000-token context window; an explicit list replaces those defaults, while `models: []` advertises none. Catalog entries are exposed through `ctx.llm.listModels('deepseek-official')` for clients such as ACP editors and the Web selector, but remain advisory: unlisted model ids still pass through unchanged as text-only routes. An omitted entry name defaults to its id, and omitted `inputModalities` means `text` only.
An image-capable catalog entry declares `inputModalities: [text, image]` and may set `imagePixelBudget` to an exact positive integer or `low`; omission uses 640,000 total pixels, while `low` selects 512 by 512 total pixels. `imageMaxBytes` defaults to 1MiB. The attachment store scales by `min(1, sqrt(pixelBudget / (width * height)))` and rounds inward to keep the pixel count at or below the hard cap, so a 2048 by 1024 normalized attachment becomes about 1130 by 565 instead of a forced square. Request encoders run lazily: low-color images try PNG (palette only without alpha) then WebP 85 and 80, other alpha images try WebP 85 then 80, and other opaque images try JPEG 85 then 80; dimensions shrink only when both quality attempts exceed 1MiB. Concurrent generation of one `variantId` shares one transform. A caller can cancel its own wait without interrupting other waiters; the transform stops when no waiter remains. The adapter normally uploads the exact derived request bytes through `POST /files` and sends `{type: "file", file_id}` blocks. A failed or timed-out file-id resolution rebuilds the whole chat request with those same request versions as base64 data URLs; one request never mixes file ids and inline images. Every retained image is preceded by stable text naming the complete attachment id, actual request dimensions, and the local normalized-object path when its attachment provider exposes one. The text marks that path read-only, gives the matching extension for a writable copy, and states that the preview and normalized image may differ from the upload. User, tool-result, agent-loop, compaction, and direct `ctx.llm.stream` requests all use this projection. Text-only routes receive stable attachment placeholders while durable history keeps its image references.
An image-capable catalog entry declares `inputModalities: [text, image]` and may set `imagePixelBudget` to an exact positive integer or `low`; omission uses 640,000 total pixels, while `low` selects 512 by 512 total pixels. `imageMaxBytes` defaults to 1MiB. The attachment store scales by `min(1, sqrt(pixelBudget / (width * height)))` and rounds inward to keep the pixel count at or below the hard cap, so a 2048 by 1024 normalized attachment becomes about 1130 by 565 instead of a forced square. Request encoders run lazily: low-color images try PNG (palette only without alpha) then WebP 85 and 80, other alpha images try WebP 85 then 80, and other opaque images try JPEG 85 then 80; dimensions shrink only when both quality attempts exceed 1MiB. Concurrent generation of one `variantId` shares one transform. A caller can cancel its own wait without interrupting other waiters; the transform stops when no waiter remains. The adapter normally uploads the exact derived request bytes through `POST /files` and sends `{type: "file", file_id}` blocks. A failed or timed-out file-id resolution rebuilds the whole chat request with those same request versions as base64 data URLs; one request never mixes file ids and inline images. Every retained image is preceded by text naming the complete attachment id and actual request dimensions. When the attachment provider exposes a host object and the current filesystem maps it into the tool execution world, the text also includes that read-only path and the matching extension for a writable copy. This access is resolved independently from the deterministic request version and its `variantId`. The descriptor states that the preview and normalized image may differ from the upload. User, tool-result, agent-loop, compaction, and direct `ctx.llm.stream` requests all use this projection. Text-only routes receive stable attachment placeholders while durable history keeps its image references.
`maxRequestFilesBytes` and `maxImagesPerRequest` bound the retained request versions at 128MiB and 600 images by default. The byte and count quanta must not exceed their corresponding bounds. Before attachment reads, the adapter uses each route's request-version byte cap as a conservative upper bound and removes the oldest over-budget prefix; only retained normalized attachments are read and transformed. Exact derived lengths are checked again without restoring omitted images. When the byte bound is crossed, the oldest prefix advances past the next 64MiB boundary; 129 one-megabyte images remove the oldest 65 and retain 64MiB, and that prefix stays unchanged until durable history exceeds 192MiB. Count overflow advances independently in `imageOffloadCountQuantum` steps. Each removed image becomes its own model-visible placeholder with its display name or attachment id and, when available, normalized dimensions, media type, and current read-only local path. This high-watermark projection avoids changing an old request prefix after every new image.
@@ -115,7 +115,7 @@ Non-2xx responses throw `LlmError` with stable codes: `AUTH` (401/403), `QUOTA`
#### What the model sees
The selected DeepSeek model receives the harness system prompt, message history, tool schemas, stop sequences, and call config without adapter-authored prompt prose. Provider-specific request extension fields remain outside that model input. The vision model normally receives retained user and tool-result images as Files API references beside stable attachment handles, request-preview dimensions, and the current normalized-object path when available; a Files resolution failure sends all retained images as inline data URLs instead. The descriptor tells the model that this read-only local copy may be resized or re-encoded and must not be used to infer upload properties. An over-budget older image keeps the same access facts in its per-image placeholder. Reasoning content from a prior assistant turn is passed back verbatim, whether or not that turn called a tool.
The selected DeepSeek model receives the harness system prompt, message history, tool schemas, stop sequences, and call config without adapter-authored prompt prose. Provider-specific request extension fields remain outside that model input. The vision model normally receives retained user and tool-result images as Files API references beside attachment handles and request-preview dimensions. It also receives a normalized-object path when the current execution filesystem maps the attachment provider's host object; a Files resolution failure sends all retained images as inline data URLs instead. The descriptor tells the model that this read-only local copy may be resized or re-encoded and must not be used to infer upload properties. An over-budget older image keeps the access currently resolved for that request in its per-image placeholder. Reasoning content from a prior assistant turn is passed back verbatim, whether or not that turn called a tool.
#### Token effect
@@ -123,7 +123,7 @@ Provider tokenization governs exact text and image-token input. Reasoning passba
#### KV Cache effect
An unchanged assembled prefix, including deterministically encoded retained images and placeholders, is eligible for DeepSeek cache reuse, which this adapter reports in usage. A model-route change or any upstream prompt, schema, prefix, history, or image-budget change may prevent reuse from the first changed token; reasoning passback appends on every reasoned turn.
An unchanged assembled prefix is eligible for DeepSeek cache reuse, which this adapter reports in usage. Deterministic request-image bytes do not make the complete prefix immutable: a changed execution-world path rewrites historical descriptor text even without offload, a refreshed upload can replace a `file_id`, and Files-to-base64 fallback changes the image representation. Any of these may prevent reuse from the first affected image. Model-route, prompt, schema, history, and image-budget changes have the same suffix effect; reasoning passback appends on every reasoned turn.
### DeepSeek response
+3 -3
View File
@@ -52,7 +52,7 @@ harness LLM(大语言模型)seam 的 DeepSeek chat-completions 适配器:
该插件注册唯一提供方路由 `deepseek-official`,并一同注册解析后的 `retryPolicy`;省略时会解析为 normal 模式并重试五次。请求使用 `provider: deepseek-official` 选择该路由;其 `model` 会作为协议 `model` 字符串原样传递,因此更改 DeepSeek 模型不需要生命周期时注册。省略 `models` 会公布 `deepseek-v4-flash``deepseek-v4-pro` 与支持图片输入的 `deepseek-v4-flash-vision-exp`,三者的上下文窗口均为 1,000,000 token;显式列表会替换这些默认值,`models: []` 则不公布任何模型。Catalog 配置项通过 `ctx.llm.listModels('deepseek-official')` 公开给 ACPAgent Client Protocol)编辑器和 Web 选择器等客户端,但仍只提供建议:未列出模型 id 仍原样传递,并按纯文本路由处理。省略配置项 name 默认为其 id,省略 `inputModalities` 则表示仅支持 `text`
支持图片的 catalog 配置项声明 `inputModalities: [text, image]`,并可把 `imagePixelBudget` 设为确切正整数或 `low`;省略时使用总像素 640,000`low` 选择总像素 512×512。`imageMaxBytes` 默认值为 1MiB。附件存储按 `min(1, sqrt(pixelBudget / (width * height)))` 缩放,并向预算内取整,确保总像素不超过硬上限。因此 2048×1024 规范化附件会得到约 1130×565 的请求版本,而不会被强制变成正方形。请求编码按需执行:低色数图片先尝试 PNG,只有不带 alpha 通道时才使用 palette,再尝试质量 85 和 80 的 WebP;其他透明图片依次尝试质量 85 和 80 的 WebP;其他非透明图片依次尝试质量 85 和 80 的 JPEG。两个质量档均超过 1MiB 时才缩小尺寸。同一 `variantId` 的并发生成共享一次变换。调用方可以单独取消等待,不会中断其他等待方;没有等待方时才会停止变换。适配器通常通过 `POST /files` 上传确切的派生请求字节,再发送 `{type: "file", file_id}` 块。File ID 解析失败或超时后,适配器会用相同请求版本的 base64 data URL 重新组装整个 chat 请求;同一请求不会混用 file ID 和内联图片。每张保留图片前都有稳定文本,写明完整附件 ID实际请求尺寸,以及附件提供方支持时的本地规范化对象路径。文本会把该路径标记为只读,指出复制到可写路径时应使用的匹配扩展名,并说明预览和规范化图片可能与上传图片不同。User、工具结果、agent loop、压缩和直接 `ctx.llm.stream` 请求都使用该投影。纯文本路由会收到稳定的附件占位文本,持久历史继续保留图片引用。
支持图片的 catalog 配置项声明 `inputModalities: [text, image]`,并可把 `imagePixelBudget` 设为确切正整数或 `low`;省略时使用总像素 640,000`low` 选择总像素 512×512。`imageMaxBytes` 默认值为 1MiB。附件存储按 `min(1, sqrt(pixelBudget / (width * height)))` 缩放,并向预算内取整,确保总像素不超过硬上限。因此 2048×1024 规范化附件会得到约 1130×565 的请求版本,而不会被强制变成正方形。请求编码按需执行:低色数图片先尝试 PNG,只有不带 alpha 通道时才使用 palette,再尝试质量 85 和 80 的 WebP;其他透明图片依次尝试质量 85 和 80 的 WebP;其他非透明图片依次尝试质量 85 和 80 的 JPEG。两个质量档均超过 1MiB 时才缩小尺寸。同一 `variantId` 的并发生成共享一次变换。调用方可以单独取消等待,不会中断其他等待方;没有等待方时才会停止变换。适配器通常通过 `POST /files` 上传确切的派生请求字节,再发送 `{type: "file", file_id}` 块。File ID 解析失败或超时后,适配器会用相同请求版本的 base64 data URL 重新组装整个 chat 请求;同一请求不会混用 file ID 和内联图片。每张保留图片前都有文本,写明完整附件 ID实际请求尺寸附件提供方给出宿主对象且当前文件系统能够将其映射到工具执行环境时,文本还会给出该只读路径,并指出复制到可写路径时应使用的匹配扩展名。该访问方式独立于确定性的请求版本及其 `variantId`。描述也会说明预览和规范化图片可能与上传图片不同。User、工具结果、agent loop、压缩和直接 `ctx.llm.stream` 请求都使用该投影。纯文本路由会收到稳定的附件占位文本,持久历史继续保留图片引用。
`maxRequestFilesBytes``maxImagesPerRequest` 限制请求中保留的请求版本,默认值分别为 128MiB 和 600 张。字节和数量步长不得超过对应上限。读取附件前,适配器以路由的请求版本字节上限作为保守上界,移除超预算的最旧前缀,只读取并转换保留的规范化附件。系统随后用确切派生长度再次检查,但不会重新加入已省略图片。字节数越过上限时,被移除的最旧前缀会越过下一个 64MiB 边界。由 1MiB 图片组成的历史达到 129MiB 时会移除最旧的 65 张并保留 64MiB;直到持久历史超过 192MiB,这个前缀才再次变化。图片数量超限时则按 `imageOffloadCountQuantum` 独立递增。每张被移除的图片都有自己的模型可见占位文本,其中包含显示名称或附件 ID;如果当前提供方支持,还会包含规范化尺寸、媒体类型和当前只读本地路径。这种定量投影不会因每新增一张图片就改写较早的请求前缀。
@@ -115,7 +115,7 @@ DeepSeek 请求身份独立于应用归因。凭据解析成功后,每个提
#### 模型看到的内容
所选 DeepSeek 模型会收到 harness 系统提示词、消息历史、工具 schema、stop sequence 和调用配置,不含适配器撰写的提示词文本。提供方特定请求扩展字段仍位于该模型输入之外。视觉模型通常通过 Files API 引用收到保留的 user 与工具结果图片,旁边带有稳定附件句柄请求预览尺寸,以及当前可用的规范化对象路径;Files 解析失败时,所有保留图片改用内联 data URL。描述会告诉模型,该本地副本只供读取,可能经过缩小或重新编码,不能据此推断上传图片的属性。超出上限的较旧图片会在自己的占位文本中保留相同的访问事实。之前 assistant 轮次的推理内容会原文回传,无论该轮次是否调用了工具。
所选 DeepSeek 模型会收到 harness 系统提示词、消息历史、工具 schema、stop sequence 和调用配置,不含适配器撰写的提示词文本。提供方特定请求扩展字段仍位于该模型输入之外。视觉模型通常通过 Files API 引用收到保留的 user 与工具结果图片,旁边带有附件句柄请求预览尺寸。当前执行文件系统能够映射附件提供方的宿主对象时,模型还会收到规范化对象路径;Files 解析失败时,所有保留图片改用内联 data URL。描述会告诉模型,该本地副本只供读取,可能经过缩小或重新编码,不能据此推断上传图片的属性。超出上限的较旧图片会在自己的占位文本中保留本次请求解析出的访问方式。之前 assistant 轮次的推理内容会原文回传,无论该轮次是否调用了工具。
#### Token 影响
@@ -123,7 +123,7 @@ DeepSeek 请求身份独立于应用归因。凭据解析成功后,每个提
#### KV Cache 影响
未更改的已组装前缀,包括确定性编码的保留图片与占位文本,可使用 DeepSeek cache 复用,适配器会在 usage 中报告它。模型路由变更,或任何上游提示词、schema、前缀、历史图片上限变更,都可能使从首个发生变化的 token 起的复用失效;推理回传会在每个含推理的轮次上追加。
未更改的已组装前缀可使用 DeepSeek cache 复用,适配器会在 usage 中报告它。确定性的请求图片字节不能保证完整前缀不变:执行环境路径变化会在没有 offload 时改写历史描述,重新上传可能替换 `file_id`Files 转为 base64 回退也会改变图片表示。这些变化都可能使复用从首张受影响图片起失效。模型路由、提示词、schema、历史图片上限变化会产生同样的后缀影响;推理回传会在每个含推理的轮次上追加。
### DeepSeek 响应
+2
View File
@@ -37,6 +37,7 @@
"@deepseek-ai/dsh-brand": "workspace:^",
"@deepseek-ai/dsh-credentials": "workspace:^",
"@deepseek-ai/dsh-deepseek-llm-api-extensions": "workspace:^",
"@deepseek-ai/dsh-fs": "workspace:^",
"@deepseek-ai/dsh-launch-environment": "workspace:^",
"@deepseek-ai/dsh-invariants": "workspace:^",
"@deepseek-ai/dsh-llm": "workspace:^",
@@ -57,6 +58,7 @@
"@deepseek-ai/dsh-brand": "workspace:^",
"@deepseek-ai/dsh-credentials": "workspace:^",
"@deepseek-ai/dsh-deepseek-llm-api-extensions": "workspace:^",
"@deepseek-ai/dsh-fs": "workspace:^",
"@deepseek-ai/dsh-plugin-package-inventory-deepseek": "workspace:^",
"@deepseek-ai/dsh-launch-environment": "workspace:^",
"@deepseek-ai/dsh-invariants": "workspace:^",
+10 -1
View File
@@ -12,6 +12,7 @@ import { attributionHeaders, contentHasImage, CONTEXT_WINDOW_EXCEEDED_CODE, isCo
import type {
ContentBlock,
GenerateOptions,
ImageAttachmentAccess,
LlmModelInfo,
LlmProviderInfo,
PreparedAdapterCall,
@@ -125,6 +126,8 @@ export interface DeepSeekAdapterOptions {
resolveUserId: () => AnonymousUserId
/** Resolve the current durable attachment service; absence rejects image input. */
resolveAttachments?: () => AttachmentStore | undefined
/** Bridge one attachment reference into the current model-tool execution world. */
resolveImageAccess?: (attachments: AttachmentStore, ref: ImageAttachmentRef) => ImageAttachmentAccess | undefined
/** Resolve the process-wide upload reuse store. */
resolveFiles?: () => DeepSeekFileStore
/** Prepare the official API's plugin-contributed top-level fields for one exact wire request. */
@@ -538,6 +541,10 @@ export class DeepSeekAdapter extends LlmAdapter {
const fileConnection = { baseURL: connection.baseURL, apiKey }
const model = connection.models.find(entry => entry.id === options.model)
const policy = model === undefined ? undefined : resolveRequestImagePolicy(model)
const resolveImageAccess = attachments === undefined
? undefined
: (ref: ImageAttachmentRef): ImageAttachmentAccess | undefined => this.config.resolveImageAccess?.(attachments, ref)
const imageAccessOptions = resolveImageAccess === undefined ? {} : { resolveImageAccess }
const requestMessages = policy === undefined ? options.messages : offloadRequestImagesWithPolicy(options.messages, {
representation: 'raw',
maxBytes: connection.maxRequestFilesBytes,
@@ -545,7 +552,7 @@ export class DeepSeekAdapter extends LlmAdapter {
byteQuantum: connection.imageOffloadByteQuantum,
countQuantum: connection.imageOffloadCountQuantum,
byteLength: ref => Math.min(ref.bytes, policy.maxBytes),
placeholder: ref => offloadedImageText(ref, attachments?.imageAccess(ref)),
placeholder: ref => offloadedImageText(ref, resolveImageAccess?.(ref)),
})
const requestOptions = requestMessages === options.messages ? options : { ...options, messages: [...requestMessages] }
const requestImages = attachments === undefined || model === undefined
@@ -562,6 +569,7 @@ export class DeepSeekAdapter extends LlmAdapter {
body = await serializeRequestWithImages(requestOptions, {
representation: { kind: 'base64' },
requestImages,
...imageAccessOptions,
maxRequestImageBytes: connection.maxInlineRequestImageBytes,
maxImagesPerRequest: connection.maxImagesPerRequest,
byteQuantum: connection.inlineImageOffloadByteQuantum,
@@ -592,6 +600,7 @@ export class DeepSeekAdapter extends LlmAdapter {
},
},
requestImages,
...imageAccessOptions,
maxRequestImageBytes: connection.maxRequestFilesBytes,
maxImagesPerRequest: connection.maxImagesPerRequest,
byteQuantum: connection.imageOffloadByteQuantum,
+7 -1
View File
@@ -13,8 +13,9 @@
import type { Context } from '@deepseek-ai/cordis'
import z from '@deepseek-ai/schemastery'
import { assertUsableApiKey, LlmError, resolveRetryPolicy, RetryPolicySchema } from '@deepseek-ai/dsh-llm'
import { assertUsableApiKey, LlmError, resolveImageAttachmentAccess, resolveRetryPolicy, RetryPolicySchema } from '@deepseek-ai/dsh-llm'
import type { ModelModality, RetryPolicyConfig } from '@deepseek-ai/dsh-llm'
import type {} from '@deepseek-ai/dsh-fs'
import { credentialRef } from '@deepseek-ai/dsh-credentials'
import { launchEnvironmentOf, type LaunchEnvironmentSnapshot } from '@deepseek-ai/dsh-launch-environment'
import { deepEqualJson, installSettingsSection, settingsNamespace } from '@deepseek-ai/dsh-settings'
@@ -438,6 +439,11 @@ export function apply(ctx: Context, config: Config): void {
resolveApiKey,
resolveUserId,
resolveAttachments: () => ctx.get('attachments'),
resolveImageAccess: (attachments, ref) => resolveImageAttachmentAccess(
attachments,
hostPath => ctx.get('fs')?.processPathFromHostPath(hostPath),
ref,
),
prepareExtensions: (request) => {
const extensions = ctx.get('deepseekLlmApiExtensions')
return extensions?.prepare(request)
+9 -9
View File
@@ -7,7 +7,7 @@
*/
import { contentHasImage, LlmError, offloadedImageText, offloadRequestImagesWithPolicy, requestImageHandleText } from '@deepseek-ai/dsh-llm'
import type { ContentBlock, GenerateOptions, Message } from '@deepseek-ai/dsh-llm'
import type { ContentBlock, GenerateOptions, ImageAttachmentAccessResolver, Message } from '@deepseek-ai/dsh-llm'
import type { ImageAttachmentRef, RequestImageAttachment } from '@deepseek-ai/dsh-attachment'
import type {
WireImageContentPart,
@@ -48,6 +48,8 @@ export interface ImageSerializationOptions {
representation: ImageRequestRepresentation
/** Request versions prepared for the conservatively retained normalized attachments, keyed by attachment id. */
requestImages: ReadonlyMap<ImageAttachmentRef['attachmentId'], RequestImageAttachment>
/** Resolve current tool access independently from deterministic request-image versions. */
resolveImageAccess?: ImageAttachmentAccessResolver
/** Positive bound on accumulated represented image bytes. */
maxRequestImageBytes: number
/** Maximum represented images in one request. */
@@ -127,11 +129,12 @@ function assertSupportedImageRoles(messages: readonly Message[]): void {
function imageHandle(
ref: ImageAttachmentRef,
version: RequestImageAttachment,
resolveAccess: ImageAttachmentAccessResolver | undefined,
precededByContent: boolean,
): WireTextContentPart {
return {
type: 'text',
text: `${precededByContent ? '\n' : ''}${requestImageHandleText(ref, version)}`,
text: `${precededByContent ? '\n' : ''}${requestImageHandleText(ref, version, resolveAccess?.(ref))}`,
}
}
@@ -155,7 +158,7 @@ async function imageParts(
type: 'image_url',
image_url: { url: `data:${version.mediaType};base64,${Buffer.from(version.data).toString('base64')}` },
}
return [imageHandle(block.attachment, version, precededByContent), image]
return [imageHandle(block.attachment, version, images.resolveImageAccess, precededByContent), image]
}
/** Convert user or nested tool-result blocks into ordered wire parts. */
@@ -390,10 +393,10 @@ export function serializeRequest(
/**
* Build one image-capable request while keeping durable bytes out of session
* messages. Oversized oldest images become deterministic text after their
* messages. Oversized oldest images become per-image text after their
* exact request-version byte lengths are known and before provider serialization.
* @param options - harness request containing image-capable user content.
* @param images - attachment resolver, request bound, and cancellation.
* @param images - request versions, optional current access resolver, and request bounds.
* @param defaults - adapter-level thinking defaults.
* @returns the fully materialized DeepSeek request body.
*/
@@ -416,10 +419,7 @@ export async function serializeRequestWithImages(
...images.maxImagesPerRequest === undefined ? {} : { maxImages: images.maxImagesPerRequest },
...images.byteQuantum === undefined ? {} : { byteQuantum: images.byteQuantum },
...images.countQuantum === undefined ? {} : { countQuantum: images.countQuantum },
placeholder: (ref) => {
const version = images.requestImages.get(ref.attachmentId)
return offloadedImageText(ref, version?.access)
},
placeholder: ref => offloadedImageText(ref, images.resolveImageAccess?.(ref)),
})
const messages: WireMessage[] = []
if (options.system !== undefined) {
@@ -109,7 +109,7 @@ function attachmentStoreOf(
} {
const readImageRequest = vi.fn(project)
return {
store: { readImageRequest, imageAccess: () => undefined } as unknown as AttachmentStore,
store: { readImageRequest, imageHostPath: () => undefined } as unknown as AttachmentStore,
readImageRequest,
}
}
@@ -57,7 +57,7 @@ function imageOptions(
refs: readonly ImageAttachmentRef[],
resolveFileId: FileResolver = fileResolver(),
maxRequestImageBytes = 20 * 1024 * 1024,
) {
): ImageSerializationOptions {
return {
representation: { kind: 'file' as const, resolveFileId },
requestImages: new Map(refs.map(ref => [ref.attachmentId, requestVersion(ref)])),
@@ -426,7 +426,7 @@ describe('image serialization', () => {
const version = images.requestImages.get(ref.attachmentId) as RequestImageAttachment
version.width = 1130
version.height = 565
version.access = { readonlyPath: '/tmp/dsh/objects/aa/object' }
images.resolveImageAccess = () => ({ readonlyPath: '/tmp/dsh/objects/aa/object' })
const wire = await serializeRequestWithImages(request({
model: 'deepseek-v4-flash-vision-exp',
messages: [createUserMessage({
@@ -594,8 +594,9 @@ describe('image serialization', () => {
const png = imageRef('image/png', 3)
const jpeg = imageRef('image/jpeg', 3)
const images = imageOptions([png, jpeg], resolveFileId, 4)
const pngVersion = images.requestImages.get(png.attachmentId) as RequestImageAttachment
pngVersion.access = { readonlyPath: '/tmp/dsh/objects/png' }
images.resolveImageAccess = ref => ref.mediaType === 'image/png'
? { readonlyPath: '/tmp/dsh/objects/png' }
: undefined
const wire = await serializeRequestWithImages(request({
model: 'deepseek-v4-flash-vision-exp',
messages: [createUserMessage({
+3
View File
@@ -23,6 +23,9 @@
{
"path": "../../attachment/attachment"
},
{
"path": "../../fs/fs"
},
{
"path": "../../util/atomic-write"
},
+2 -2
View File
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/llm/llm-pi-ai/README.md
README.md: 7f0c2a32315200e3244cb83d396bbf2ee30d728f
README.zh.md: f11d4ba1d991cc70c1967eef8f8b880727df36ce
README.md: 1ae0e641f7ffaebc1c2c1060e143c72a40631396
README.zh.md: 17bf059ea9bb3fbbb80cadc0c9eefed76af26abd
+3 -3
View File
@@ -123,7 +123,7 @@ A model that carries reasoning metadata — from the installed catalog or from i
A model **without** that metadata — a hand-declared one whose entry declares no `reasoningEfforts`, and a catalog model pi-ai marks as non-reasoning — exposes no `reasoning` at all. pi-ai reports such a model as supporting the single level `off`, but `off` is translated to *omitting* the reasoning option, which is byte-for-byte the request that naming no effort already produces: selecting it could not disable anything, so a provider whose own default is to think would keep thinking with `off` shown as selected. Reporting the capability as unavailable leaves a surface offering the provider's default and nothing that misrepresents it. The profile `reasoning` value, including `off`, is the deployment default when configured; omitting it preserves the provider default. Per-request `GenerateOptions.reasoningEffort` takes precedence, and a level absent from the exact model capability fails the REQUEST with `UNSUPPORTED_REASONING_EFFORT` before network I/O instead of being clamped. Describing a model never fails that way: the models under one provider disagree about which levels they accept, so `resolveModel` reports a profile level the exact model cannot take as no default at all rather than throwing. A throw there would take the whole provider out of every model catalog built over it — one mis-set profile field hiding even the models that do support the level — so a bad configuration surfaces where it is acted on, not where it is described. pi-ai's common stream options represent `off` by omitting `reasoning`.
Supported profile fields are `apiKeyEnv`, `displayName`, `api`, `baseURL`, `models`, `modelOverrides`, `compat`, `defaultContextWindow`, `defaultMaxTokens`, `defaultInput`, `headers`, `reasoning`, `thinkingBudgets`, `cacheRetention`, `transport`, `timeoutMs`, `websocketConnectTimeoutMs`, `streamIdleTimeoutMs`, `maxRequestImageBytes`, `requestImagePixelBudget`, `requestImageMaxBytes`, and `retryPolicy`. Each resolved profile retry policy is captured with that provider route; omission uses the shared bounded normal default of five retries. The stream-idle interval is a positive finite Node timer delay, defaults to five minutes, and covers only an outstanding provider read, not consumer think time. Every image route derives a deterministic request version from the provider-independent normalized attachment under `requestImagePixelBudget` (default 2048 by 2048 total pixels) and `requestImageMaxBytes` (default 1MiB raw bytes). Before reading attachments, `maxRequestImageBytes` applies to conservative request-version upper bounds and replaces the oldest over-budget images with per-image text; exact base64 lengths are checked again after retained versions are generated. The 20MiB default can retain fifteen maximum-size 1MiB versions after base64 expansion while leaving request-body headroom. The same version feeds inline base64, and its stable descriptor exposes the attachment id, actual request-image dimensions, and provider-resolved normalized-object path when available. Harness app attribution wins a conflicting configured header name.
Supported profile fields are `apiKeyEnv`, `displayName`, `api`, `baseURL`, `models`, `modelOverrides`, `compat`, `defaultContextWindow`, `defaultMaxTokens`, `defaultInput`, `headers`, `reasoning`, `thinkingBudgets`, `cacheRetention`, `transport`, `timeoutMs`, `websocketConnectTimeoutMs`, `streamIdleTimeoutMs`, `maxRequestImageBytes`, `requestImagePixelBudget`, `requestImageMaxBytes`, and `retryPolicy`. Each resolved profile retry policy is captured with that provider route; omission uses the shared bounded normal default of five retries. The stream-idle interval is a positive finite Node timer delay, defaults to five minutes, and covers only an outstanding provider read, not consumer think time. Every image route derives a deterministic request version from the provider-independent normalized attachment under `requestImagePixelBudget` (default 2048 by 2048 total pixels) and `requestImageMaxBytes` (default 1MiB raw bytes). Before reading attachments, `maxRequestImageBytes` applies to conservative request-version upper bounds and replaces the oldest over-budget images with per-image text; exact base64 lengths are checked again after retained versions are generated. The 20MiB default can retain fifteen maximum-size 1MiB versions after base64 expansion while leaving request-body headroom. The same version feeds inline base64. Its descriptor exposes the attachment id and actual request-image dimensions, plus a normalized-object path only when the attachment provider exposes a host object and the current filesystem maps it into the tool execution world. The path is resolved separately from the request version and its `variantId`. Harness app attribution wins a conflicting configured header name.
The adapter forces pi-ai's SDK `maxRetries` to zero so one `stream()` call makes one provider request. The removed profile fields `maxRetries` and `maxRetryDelayMs` fail load instead of silently multiplying or hiding the separately composed agent-level retry budget. Idle expiry aborts the SDK's stable request signal and surfaces `TIMEOUT`; an earlier caller abort remains `ABORTED`.
@@ -173,7 +173,7 @@ pi-ai installs several provider SDKs and lazy-loads the one selected by the cata
#### What the model sees
The selected catalog model receives `GenerateOptions.system`, history, tools, and sampling fields supported by pi-ai's common streaming API. Each retained image is preceded by stable text naming its complete attachment id, actual request dimensions, and current normalized-object path when available. The descriptor marks that path read-only and warns that normalization or request projection may have resized or re-encoded the upload. When accumulated base64 image payload exceeds the route's `maxRequestImageBytes`, each offloaded image keeps its own identity and available local path in replacement text. Offloaded normalized attachments are not read or transformed. Provider-native replay metadata is restored only when the adapter validates it for the historical content.
The selected catalog model receives `GenerateOptions.system`, history, tools, and sampling fields supported by pi-ai's common streaming API. Each retained image is preceded by text naming its complete attachment id and actual request dimensions. The text includes a normalized-object path when the current execution filesystem maps the attachment provider's host object, marks that path read-only, and warns that normalization or request projection may have resized or re-encoded the upload. When accumulated base64 image payload exceeds the route's `maxRequestImageBytes`, each offloaded image keeps its own identity and access currently resolved for that request in replacement text. Offloaded normalized attachments are not read or transformed. Provider-native replay metadata is restored only when the adapter validates it for the historical content.
#### Token effect
@@ -181,7 +181,7 @@ Provider tokenization governs exact input. Retained images add the stable attach
#### KV Cache effect
Conversion preserves logical request order without adding text, while the selected provider's serialization and replay state determine reuse. Changing adapter instance, provider, model, or any upstream request token may prevent reuse from the first difference. Crossing the image bound rewrites an early message (the newly offloaded image becomes placeholder text), so reuse ends at that message until the offloaded prefix stabilizes.
Conversion preserves logical request order, while image handles and offload placeholders add model-visible text. Stable attachment identity and request bytes do not make that text immutable: a changed execution-world path rewrites a historical handle even without offload and may prevent reuse from that image. Changing adapter instance, provider, model, or any other upstream request token has the same suffix effect. Crossing the image bound replaces an earlier image with placeholder text, so reuse ends at that message until the offloaded prefix stabilizes.
### Provider response
+3 -3
View File
@@ -124,7 +124,7 @@ pi-ai 依据提供方 id 与 baseURL 决定每个请求的形状:系统提示
**没有**这份元数据的模型——条目未声明 `reasoningEfforts` 的手工声明模型,以及 pi-ai 标记为不具备推理能力的 catalog 模型——完全不公开 `reasoning`。pi-ai 会把这类模型报告为只支持 `off` 一档,但 `off` 会被翻译成*省略* reasoning 选项,而那与「不点名任何档位」产出的请求逐字节相同:选它关不掉任何东西,于是自身默认就在思考的提供方,会在界面显示 `off` 被选中的同时继续思考。把该能力报告为不可用,界面就只剩提供方默认这一项,不会再出现自相矛盾的控件。配置 profile 的 `reasoning` 值(包括 `off`)在存在时是部署默认值;省略它会保留提供方默认值。每次请求的 `GenerateOptions.reasoningEffort` 优先;未出现在确切模型能力中的档位会让**请求**在网络 I/O 前以 `UNSUPPORTED_REASONING_EFFORT` 失败,而不会被自动调整。**描述**一个模型则从不这样失败:同一提供方下各模型接受的档位并不一致,因此 `resolveModel` 对该模型拿不下的 profile 档位报告为「没有默认值」,而不是抛错。在那里抛错会让整个提供方从任何基于它构建的模型目录中消失——一个配错的 profile 字段连支持该档位的模型也一并藏起来——所以坏配置暴露在被执行处,而不是被描述处。pi-ai 的通用流选项通过省略 `reasoning` 表示 `off`
受支持的 profile 字段是 `apiKeyEnv``displayName``api``baseURL``models``modelOverrides``compat``defaultContextWindow``defaultMaxTokens``defaultInput``headers``reasoning``thinkingBudgets``cacheRetention``transport``timeoutMs``websocketConnectTimeoutMs``streamIdleTimeoutMs``maxRequestImageBytes``requestImagePixelBudget``requestImageMaxBytes``retryPolicy`。每条 profile 解析后的重试策略会随该提供方路由一同捕获;省略时使用共享的有界 normal 默认值并重试五次。流空闲间隔必须是正的有限 Node 定时器延迟,默认为五分钟,且只覆盖未完成提供方读取,不包括消费方思考时间。每条图片路由从提供方无关的规范化附件派生确定性请求版本,受 `requestImagePixelBudget`(默认总像素 2048×2048)和 `requestImageMaxBytes`(默认原始字节 1MiB)约束。读取附件前,`maxRequestImageBytes` 先按请求版本的保守上界把超预算的最旧图片替换为逐图文本;保留版本生成后再用确切 base64 长度检查。20MiB 默认值可保留十五个按 1MiB 上限生成的请求版本,并为请求正文留下余量。同一版本用于内联 base64,其稳定描述会公开附件 ID实际请求图片尺寸,以及当前提供方支持时的规范化对象路径。若已配置标头中有同名项,则以 Harness 应用归因为准。
受支持的 profile 字段是 `apiKeyEnv``displayName``api``baseURL``models``modelOverrides``compat``defaultContextWindow``defaultMaxTokens``defaultInput``headers``reasoning``thinkingBudgets``cacheRetention``transport``timeoutMs``websocketConnectTimeoutMs``streamIdleTimeoutMs``maxRequestImageBytes``requestImagePixelBudget``requestImageMaxBytes``retryPolicy`。每条 profile 解析后的重试策略会随该提供方路由一同捕获;省略时使用共享的有界 normal 默认值并重试五次。流空闲间隔必须是正的有限 Node 定时器延迟,默认为五分钟,且只覆盖未完成提供方读取,不包括消费方思考时间。每条图片路由从提供方无关的规范化附件派生确定性请求版本,受 `requestImagePixelBudget`(默认总像素 2048×2048)和 `requestImageMaxBytes`(默认原始字节 1MiB)约束。读取附件前,`maxRequestImageBytes` 先按请求版本的保守上界把超预算的最旧图片替换为逐图文本;保留版本生成后再用确切 base64 长度检查。20MiB 默认值可保留十五个按 1MiB 上限生成的请求版本,并为请求正文留下余量。同一版本用于内联 base64。对应描述会公开附件 ID实际请求图片尺寸;只有附件提供方给出宿主对象且当前文件系统能够将其映射到工具执行环境时,描述才会加入规范化对象路径。该路径独立于请求版本及其 `variantId`。若已配置标头中有同名项,则以 Harness 应用归因为准。
适配器强制 pi-ai SDK `maxRetries` 为零,因此一次 `stream()` 调用只会发起一次提供方请求。已移除 profile 字段 `maxRetries``maxRetryDelayMs` 会使加载失败,而不是静默倍增或隐藏单独组合的 agent(智能体)级重试预算。空闲超时会 abort SDK 的稳定请求信号,并以 `TIMEOUT` 呈现;较早的调用方 abort 仍为 `ABORTED`
@@ -174,7 +174,7 @@ pi-ai 会安装多个提供方 SDK,并延迟加载 catalog 模型所选的 SDK
#### 模型看到的内容
所选 catalog 模型会收到 `GenerateOptions.system`、历史、工具,以及 pi-ai 通用流式 API 支持的采样字段。每张保留图片前都有稳定文本,写明完整附件 ID实际请求尺寸,以及当前可用的规范化对象路径。描述会把该路径标记为只读,并说明规范化或请求投影可能缩小或重新编码上传图片。请求累积的 base64 图片载荷超过路由的 `maxRequestImageBytes` 时,每张被 offload 的图片会在替代文本中保留自己的身份和可用本地路径。系统不会读取或转换被 offload 的规范化附件。只有当适配器验证提供方原生回放元数据与历史内容匹配时,才会恢复这些元数据。
所选 catalog 模型会收到 `GenerateOptions.system`、历史、工具,以及 pi-ai 通用流式 API 支持的采样字段。每张保留图片前都有文本,写明完整附件 ID实际请求尺寸。当前执行文件系统能够映射附件提供方的宿主对象时,文本还会给出规范化对象路径,将其标记为只读,并说明规范化或请求投影可能缩小或重新编码上传图片。请求累积的 base64 图片载荷超过路由的 `maxRequestImageBytes` 时,每张被 offload 的图片会在替代文本中保留自己的身份和本次请求解析出的访问方式。系统不会读取或转换被 offload 的规范化附件。只有当适配器验证提供方原生回放元数据与历史内容匹配时,才会恢复这些元数据。
#### Token 影响
@@ -182,7 +182,7 @@ pi-ai 会安装多个提供方 SDK,并延迟加载 catalog 模型所选的 SDK
#### KV Cache 影响
转换保留逻辑请求顺序,不添加文本;复用取决于所选提供方的序列化与回放状态。更改适配器实例、提供方、模型或任何上游请求 token,都可能使复用从首个出现差异的 token 起失效。跨过图片上限会改写较早的一条消息(新被 offload 的图片变为占位文本,复用在该消息处截止,直到被 offload 的前缀稳定。
转换保留逻辑请求顺序,图片句柄和 offload 占位内容会加入模型可见文本。稳定的附件身份和请求字节不能保证这些文本不变:执行环境路径变化会在没有 offload 时改写历史句柄,并可能使复用从该图片起失效。更改适配器实例、提供方、模型或其他上游请求 token 会产生同样的后缀影响。跨过图片上限会把较早图片替换为占位文本,复用在该消息处截止,直到被 offload 的前缀稳定。
### 提供方响应
+2
View File
@@ -35,6 +35,7 @@
"@deepseek-ai/dsh-attachment": "workspace:^",
"@deepseek-ai/dsh-authorization": "workspace:^",
"@deepseek-ai/dsh-credentials": "workspace:^",
"@deepseek-ai/dsh-fs": "workspace:^",
"@deepseek-ai/dsh-launch-environment": "workspace:^",
"@deepseek-ai/dsh-invariants": "workspace:^",
"@deepseek-ai/dsh-llm": "workspace:^",
@@ -50,6 +51,7 @@
"@deepseek-ai/dsh-attachment": "workspace:^",
"@deepseek-ai/dsh-authorization": "workspace:^",
"@deepseek-ai/dsh-credentials": "workspace:^",
"@deepseek-ai/dsh-fs": "workspace:^",
"@deepseek-ai/dsh-launch-environment": "workspace:^",
"@deepseek-ai/dsh-invariants": "workspace:^",
"@deepseek-ai/dsh-llm": "workspace:^",
+13 -5
View File
@@ -47,6 +47,7 @@ import {
} from '@deepseek-ai/dsh-llm'
import type {
GenerateOptions,
ImageAttachmentAccess,
LlmModelInfo,
LlmProviderInfo,
LlmResolvedModelInfo,
@@ -55,7 +56,7 @@ import type {
ResolvedRetryPolicy,
StreamChunk,
} from '@deepseek-ai/dsh-llm'
import type { AttachmentStore } from '@deepseek-ai/dsh-attachment'
import type { AttachmentStore, ImageAttachmentRef } from '@deepseek-ai/dsh-attachment'
import { idleWatchdog, timeoutOf } from '@deepseek-ai/dsh-timeout'
import type { ResolvedPiAiProviderProfile } from './config.ts'
import { toPiContext } from './context.ts'
@@ -93,6 +94,8 @@ export interface PiAiAdapterOptions {
auth: PiAiAuthInjection
/** Resolve the optional durable attachment service at request time. */
resolveAttachments?: () => AttachmentStore | undefined
/** Bridge one attachment reference into the current model-tool execution world. */
resolveImageAccess?: (attachments: AttachmentStore, ref: ImageAttachmentRef) => ImageAttachmentAccess | undefined
/**
* Observe one assistant history message degrading to provider-neutral
* conversion because its stored replay state is unusable by this build.
@@ -360,10 +363,15 @@ export class PiAiAdapter extends LlmAdapter {
}
const context = attachments === undefined
? toPiContext(options, undefined, onReplayDegrade)
: await toPiContext({ ...options, signal: watchdog.signal }, attachments, onReplayDegrade, profile.maxRequestImageBytes, {
maxPixels: profile.requestImagePixelBudget,
maxBytes: profile.requestImageMaxBytes,
})
: await toPiContext({ ...options, signal: watchdog.signal }, {
attachments,
resolveImageAccess: ref => this.config.resolveImageAccess?.(attachments, ref),
maxRequestImageBytes: profile.maxRequestImageBytes,
requestImagePolicy: {
maxPixels: profile.requestImagePixelBudget,
maxBytes: profile.requestImageMaxBytes,
},
}, onReplayDegrade)
const events = snapshot.models.streamSimple(model, context, {
...profileOptions(profile, reasoning, apiKey),
...options.temperature === undefined ? {} : { temperature: options.temperature },
+35 -25
View File
@@ -5,7 +5,7 @@
*/
import { CallId, contentHasImage, LlmError, offloadedImageText, offloadRequestImagesWithPolicy, requestImageHandleText } from '@deepseek-ai/dsh-llm'
import type { ContentBlock, GenerateOptions, Message } from '@deepseek-ai/dsh-llm'
import type { ContentBlock, GenerateOptions, ImageAttachmentAccessResolver, Message } from '@deepseek-ai/dsh-llm'
import type {
AttachmentId,
AttachmentStore,
@@ -48,6 +48,7 @@ function assertSupportedImageRoles(messages: readonly Message[]): void {
async function userContent(
blocks: readonly ContentBlock[],
requestImages: ReadonlyMap<AttachmentId, RequestImageAttachment>,
resolveImageAccess: ImageAttachmentAccessResolver,
): Promise<string | (TextContent | ImageContent)[]> {
const content: (TextContent | ImageContent)[] = []
for (const block of blocks) {
@@ -57,7 +58,10 @@ async function userContent(
break
case 'image': {
const version = requestImages.get(block.attachment.attachmentId) as RequestImageAttachment
content.push({ type: 'text', text: requestImageHandleText(block.attachment, version) })
content.push({
type: 'text',
text: requestImageHandleText(block.attachment, version, resolveImageAccess(block.attachment)),
})
content.push({
type: 'image',
data: Buffer.from(version.data).toString('base64'),
@@ -67,7 +71,7 @@ async function userContent(
}
case 'tool-result':
{
const nested = await userContent(block.content, requestImages)
const nested = await userContent(block.content, requestImages, resolveImageAccess)
if (typeof nested === 'string') {
if (nested.length > 0) content.push({ type: 'text', text: nested })
} else {
@@ -170,17 +174,29 @@ function textOnlyContext(options: GenerateOptions, onReplayDegrade?: (reason: st
return piContext(options, messages)
}
/** Inputs that bind deterministic request images to one current tool execution world. */
export interface PiImageRequestContext {
/** Durable provider that resolves request-image bytes and provider-owned host objects. */
attachments: AttachmentStore
/** Resolve current tool access separately from deterministic request-image versions. */
resolveImageAccess: ImageAttachmentAccessResolver
/** Request-level bound on base64-encoded image payload; omission leaves every image in place. */
maxRequestImageBytes?: number
/** Route pixel and raw encoded-byte budgets. */
requestImagePolicy?: ImageRequestPolicy
}
/**
* Convert text-only harness history to a synchronous pi-ai Context. Tool
* result names are recovered from preceding assistant tool calls.
* @param options - the harness request; `options.system` maps to pi-ai's single `systemPrompt` slot.
* @param attachments - absent; selects the synchronous conversion.
* @param images - absent; selects the synchronous conversion.
* @param onReplayDegrade - forwarded to {@link toPiAssistant} for each assistant message.
* @returns the pi-ai context; `tools` is omitted when the request declares none.
*/
export function toPiContext(
options: GenerateOptions,
attachments?: undefined,
images?: undefined,
onReplayDegrade?: (reason: string) => void,
): PiContext
/**
@@ -190,48 +206,42 @@ export function toPiContext(
* oldest images are replaced by text placeholders until the request fits, so
* an image-heavy session keeps clearing gateway request-size caps.
* @param options - the harness request; `options.system` maps to pi-ai's single `systemPrompt` slot.
* @param attachments - durable byte resolver for image references.
* @param images - attachment provider, current path resolver, and request limits.
* @param onReplayDegrade - forwarded to {@link toPiAssistant} for each assistant message.
* @param maxRequestImageBytes - request-level bound on base64-encoded image payload; omission leaves every image in place.
* @param requestImagePolicy - route pixel and raw encoded-byte budgets.
* @returns the asynchronously resolved pi-ai context.
*/
export function toPiContext(
options: GenerateOptions,
attachments: AttachmentStore,
images: PiImageRequestContext,
onReplayDegrade?: (reason: string) => void,
maxRequestImageBytes?: number,
requestImagePolicy?: ImageRequestPolicy,
): Promise<PiContext>
export function toPiContext(
options: GenerateOptions,
attachments?: AttachmentStore,
images?: PiImageRequestContext,
onReplayDegrade?: (reason: string) => void,
maxRequestImageBytes?: number,
requestImagePolicy?: ImageRequestPolicy,
): PiContext | Promise<PiContext> {
return attachments === undefined
return images === undefined
? textOnlyContext(options, onReplayDegrade)
: toPiContextWithImages(options, attachments, onReplayDegrade, maxRequestImageBytes, requestImagePolicy)
: toPiContextWithImages(options, images, onReplayDegrade)
}
async function toPiContextWithImages(
options: GenerateOptions,
attachments: AttachmentStore,
images: PiImageRequestContext,
onReplayDegrade?: (reason: string) => void,
maxRequestImageBytes?: number,
requestImagePolicy: ImageRequestPolicy = {
): Promise<PiContext> {
const { attachments, resolveImageAccess, maxRequestImageBytes } = images
const requestImagePolicy = images.requestImagePolicy ?? {
maxPixels: DEFAULT_REQUEST_IMAGE_PIXEL_BUDGET,
maxBytes: DEFAULT_REQUEST_IMAGE_MAX_BYTES,
},
): Promise<PiContext> {
}
assertSupportedImageRoles(options.messages)
const requestMessages = offloadRequestImagesWithPolicy(options.messages, {
representation: 'base64',
...maxRequestImageBytes === undefined ? {} : { maxBytes: maxRequestImageBytes },
byteQuantum: 1,
byteLength: ref => Math.min(ref.bytes, requestImagePolicy.maxBytes),
placeholder: ref => offloadedImageText(ref, attachments.imageAccess(ref)),
placeholder: ref => offloadedImageText(ref, resolveImageAccess(ref)),
})
const requestImages = await prepareRequestImages(requestMessages, attachments, requestImagePolicy, options.signal)
const exactMessages = offloadRequestImagesWithPolicy(requestMessages, {
@@ -239,7 +249,7 @@ async function toPiContextWithImages(
...maxRequestImageBytes === undefined ? {} : { maxBytes: maxRequestImageBytes },
byteQuantum: 1,
byteLength: ref => (requestImages.get(ref.attachmentId) as RequestImageAttachment).bytes,
placeholder: ref => offloadedImageText(ref, requestImages.get(ref.attachmentId)?.access),
placeholder: ref => offloadedImageText(ref, resolveImageAccess(ref)),
})
const toolNames = new Map<CallId, string>()
const messages: PiMessage[] = []
@@ -262,7 +272,7 @@ async function toPiContextWithImages(
}
// user role: text + tool results (each result becomes its own message).
const regular = message.content.filter(block => block.type !== 'tool-result')
const content = await userContent(regular, requestImages)
const content = await userContent(regular, requestImages, resolveImageAccess)
const results = message.content.filter((block): block is Extract<ContentBlock, { type: 'tool-result' }> => (
block.type === 'tool-result'
))
@@ -270,7 +280,7 @@ async function toPiContextWithImages(
messages.push({ role: 'user', content, timestamp: 0 })
}
for (const result of results) {
const resultContent = await userContent(result.content, requestImages)
const resultContent = await userContent(result.content, requestImages, resolveImageAccess)
messages.push({
role: 'toolResult',
toolCallId: result.toolCallId,
+7 -1
View File
@@ -57,8 +57,9 @@
import type { Context } from '@deepseek-ai/cordis'
import { launchEnvironmentOf } from '@deepseek-ai/dsh-launch-environment'
import { assertUsableApiKey, LlmError } from '@deepseek-ai/dsh-llm'
import { assertUsableApiKey, LlmError, resolveImageAttachmentAccess } from '@deepseek-ai/dsh-llm'
import type { AdapterRegistrationHandle, DirectoryRegistrationHandle, LlmConfigurableProvider } from '@deepseek-ai/dsh-llm'
import type {} from '@deepseek-ai/dsh-fs'
import { deepEqualJson, installSettingsSection, settingsNamespace } from '@deepseek-ai/dsh-settings'
import { PiAiAdapter } from './adapter.ts'
import { authContextFrom, credentialStoreFrom } from './auth.ts'
@@ -197,6 +198,11 @@ export function apply(ctx: Context, config: Config): void {
resolveApiKey,
auth,
resolveAttachments: () => ctx.get('attachments'),
resolveImageAccess: (attachments, ref) => resolveImageAttachmentAccess(
attachments,
hostPath => ctx.get('fs')?.processPathFromHostPath(hostPath),
ref,
),
onReplayDegrade: ({ provider, model, reason }) => {
ctx.logger.warn(
`llm-pi-ai: unusable replay state on assistant history for route "${provider}/${model}";`
+30 -16
View File
@@ -9,6 +9,7 @@ import type {
import { CallId, createMessage, createUserMessage, offloadedImageText } from '@deepseek-ai/dsh-llm'
import type { ContentBlock, GenerateOptions, Message } from '@deepseek-ai/dsh-llm'
import { toPiContext } from '../src/context.ts'
import type { PiImageRequestContext } from '../src/context.ts'
import { toPiAssistant } from '../src/replay.ts'
const ref: ImageAttachmentRef = {
@@ -43,11 +44,18 @@ function projectionStore(
Promise.resolve(requestImage(value, Uint8Array.of(1)))
)),
): AttachmentStore {
return { readImageRequest, imageAccess: () => undefined } as unknown as AttachmentStore
return { readImageRequest, imageHostPath: () => undefined } as unknown as AttachmentStore
}
const attachments = projectionStore()
function imageContext(
store: AttachmentStore,
overrides: Partial<Omit<PiImageRequestContext, 'attachments'>> = {},
): PiImageRequestContext {
return { attachments: store, resolveImageAccess: () => undefined, ...overrides }
}
function request(messages: GenerateOptions['messages']): GenerateOptions {
return {
provider: 'openai',
@@ -138,7 +146,7 @@ describe('pi-ai request context conversion', () => {
{ type: 'image', attachment: ref },
],
}]),
]), attachments)
]), imageContext(attachments))
expect(context.messages).toEqual([
{ role: 'user', content: '', timestamp: 0 },
@@ -180,9 +188,10 @@ describe('pi-ai request context conversion', () => {
...requestImage(value, Uint8Array.of(1)),
width: 1130,
height: 565,
access: { readonlyPath: '/tmp/dsh/objects/aa/object' },
}))
const context = await toPiContext(request([user([{ type: 'image', attachment: named }])]), store)
const context = await toPiContext(request([user([{ type: 'image', attachment: named }])]), imageContext(store, {
resolveImageAccess: () => ({ readonlyPath: '/tmp/dsh/objects/aa/object' }),
}))
expect(context.messages[0]).toMatchObject({
role: 'user',
content: [
@@ -211,7 +220,7 @@ describe('pi-ai request context conversion', () => {
content: [{ type: 'image', attachment: ref }],
},
],
}])]), attachments)
}])]), imageContext(attachments))
expect(context.messages).toEqual([{
role: 'toolResult',
@@ -265,7 +274,7 @@ describe('pi-ai request context conversion', () => {
}]),
user([{ type: 'image', attachment: sized }, { type: 'text', text: 'newer' }]),
user([{ type: 'image', attachment: sized }]),
]), store, undefined, 8)
]), imageContext(store, { maxRequestImageBytes: 8 }))
expect(context.messages).toEqual([
{
@@ -308,7 +317,7 @@ describe('pi-ai request context conversion', () => {
const context = await toPiContext(request([user([
{ type: 'image', attachment: old },
{ type: 'image', attachment: recent },
])]), projectionStore(readImageRequest), undefined, 4)
])]), imageContext(projectionStore(readImageRequest), { maxRequestImageBytes: 4 }))
expect(context.messages[0]).toMatchObject({
role: 'user',
@@ -322,17 +331,19 @@ describe('pi-ai request context conversion', () => {
expect(readImageRequest.mock.calls[0]?.[0]).toEqual(recent)
})
it('uses the generated request access when exact encoded bytes require offload', async () => {
it('uses independently resolved access when exact encoded bytes require offload', async () => {
const sized: ImageAttachmentRef = { ...ref, bytes: 3 }
const access = { readonlyPath: '/tmp/dsh-normalized-image' }
const readImageRequest = vi.fn((value: ImageAttachmentRef) => Promise.resolve({
...requestImage(value, Uint8Array.of(1, 2, 3, 4)),
access,
}))
const context = await toPiContext(request([
user([{ type: 'image', attachment: sized }]),
]), projectionStore(readImageRequest), undefined, 4)
]), imageContext(projectionStore(readImageRequest), {
maxRequestImageBytes: 4,
resolveImageAccess: () => access,
}))
expect(context.messages).toEqual([{
role: 'user',
@@ -347,7 +358,7 @@ describe('pi-ai request context conversion', () => {
const exact = await toPiContext(request([
user([{ type: 'image', attachment: sized }]),
user([{ type: 'image', attachment: sized }]),
]), attachments, undefined, 8)
]), imageContext(attachments, { maxRequestImageBytes: 8 }))
expect(exact.messages).toEqual([
{
role: 'user',
@@ -367,7 +378,7 @@ describe('pi-ai request context conversion', () => {
const store = projectionStore(readImageRequest)
const oversized = await toPiContext(request([
user([{ type: 'image', attachment: { ...ref, bytes: 300 } }]),
]), store, undefined, 8)
]), imageContext(store, { maxRequestImageBytes: 8 }))
// All-text content collapses to the string form; the placeholder still reaches the model.
expect(oversized.messages).toEqual([
{ role: 'user', content: offloadedImageText({ ...ref, bytes: 300 }), timestamp: 0 },
@@ -382,11 +393,14 @@ describe('pi-ai request context conversion', () => {
Promise.resolve(requestImage(value, Uint8Array.of(1, 2, 3)))
))
const store = projectionStore(readImageRequest)
const aliased = await toPiContext(request([user([shared, shared])]), store, undefined, 4)
const aliased = await toPiContext(
request([user([shared, shared])]),
imageContext(store, { maxRequestImageBytes: 4 }),
)
const replayed = await toPiContext(request([user([
{ type: 'image', attachment: { ...sized } },
{ type: 'image', attachment: { ...sized } },
])]), store, undefined, 4)
])]), imageContext(store, { maxRequestImageBytes: 4 }))
const expected = [{
role: 'user',
@@ -430,7 +444,7 @@ describe('pi-ai request context conversion', () => {
const store = projectionStore(readImageRequest)
await expect(toPiContext(request([
history(role, [{ type: 'image', attachment: ref }]),
]), store, undefined, 1)).rejects.toMatchObject({ code: 'UNSUPPORTED_CONTENT' })
]), imageContext(store, { maxRequestImageBytes: 1 }))).rejects.toMatchObject({ code: 'UNSUPPORTED_CONTENT' })
expect(readImageRequest).not.toHaveBeenCalled()
}
@@ -438,7 +452,7 @@ describe('pi-ai request context conversion', () => {
history('system', [{ type: 'text', text: 'history system' }]),
history('assistant', [{ type: 'text', text: 'answer' }]),
user([{ type: 'text', text: 'plain' }]),
]), attachments)).resolves.toMatchObject({
]), imageContext(attachments))).resolves.toMatchObject({
messages: [
{ role: 'user', content: 'history system' },
{ role: 'assistant' },
+7 -3
View File
@@ -63,7 +63,11 @@ function attachmentStore(readImageRequest: (
policy: ImageRequestPolicy,
signal?: AbortSignal,
) => Promise<RequestImageAttachment>): AttachmentStore {
return { readImageRequest, imageAccess: () => undefined } as unknown as AttachmentStore
return { readImageRequest, imageHostPath: () => undefined } as unknown as AttachmentStore
}
function imageContext(attachments: AttachmentStore) {
return { attachments, resolveImageAccess: () => undefined }
}
describe('toPiContext', () => {
@@ -109,7 +113,7 @@ describe('toPiContext', () => {
content: [{ type: 'text', text: 'describe' }, { type: 'image', attachment }],
source: { kind: 'plugin', plugin: 'test' },
})],
}, attachmentStore(readImageRequest))
}, imageContext(attachmentStore(readImageRequest)))
expect(readImageRequest).toHaveBeenCalledWith(
attachment,
@@ -161,7 +165,7 @@ describe('toPiContext', () => {
}],
source: { kind: 'plugin', plugin: 'test' },
})],
}, attachmentStore(readImageRequest))
}, imageContext(attachmentStore(readImageRequest)))
expect(context.messages).toEqual([{
role: 'toolResult',
+3
View File
@@ -26,6 +26,9 @@
{
"path": "../../attachment/attachment"
},
{
"path": "../../fs/fs"
},
{
"path": "../../credentials/credentials"
},
+2 -2
View File
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write packages/llm/llm/README.md
README.md: d08d28497c4821c578fac43515b3bebfa4ed8131
README.zh.md: 3d7a183aa2319acd869d8e4bc68ec9493c3ea322
README.md: ef58516790a2723bce34aa1bbae2e1629050f6a5
README.zh.md: 8af1240cdb4f3b65f1c0f841ade620c85443129b
+2 -2
View File
@@ -57,7 +57,7 @@ Exact-model metadata is a separate correctness query, not a catalog decoration o
Message content is an array of typed blocks: `text`, `reasoning`, `image`, `tool-call`, `tool-result`. An `ImageBlock` carries only a durable `ImageAttachmentRef`; provider bytes and request dimensions are resolved later. The union remains merge-extensible through `ContentBlockMap`, so plugins can add further block types via declaration merging. Assistant messages use a model source carrying the provider and model that produced them plus optional adapter-private replay state. Before dispatch, `LlmRuntime` retains that state only when the historical provider route and target provider route are currently owned by the exact same adapter instance; the adapter then decides whether it can restore or convert the state across models/providers.
Every dispatch uses the exact model modalities captured with its adapter generation. An image-capable adapter projects durable image references into route-specific request versions. A text-only route instead receives deterministic attachment placeholders, including nested tool-result images, without changing append-only session history. `offloadRequestImagesWithPolicy()` provides deterministic oldest-first image removal with raw or base64 accounting and count or byte quanta; adapters supply the exact derived-version byte length and the required per-image placeholder text.
Every dispatch uses the exact model modalities captured with its adapter generation. An image-capable adapter projects durable image references into route-specific request versions. `resolveImageAttachmentAccess()` separately combines an attachment provider's optional host object with a consumer-supplied mapping from that host path into the current tool execution world. The result never enters `RequestImageAttachment` or its `variantId`. A text-only route instead receives deterministic attachment placeholders, including nested tool-result images, without changing append-only session history. `offloadRequestImagesWithPolicy()` provides deterministic oldest-first image removal with raw or base64 accounting and count or byte quanta; adapters supply the exact derived-version byte length and the required per-image placeholder text.
Streaming is a raw chunk protocol (`block-start`, `text-delta`, `reasoning-delta`, `tool-call-delta`, `block-end`, `usage`, `finish`). Every adapter outcome reaches consumers as one terminal `finish`; operational failure uses its `error` or `aborted` reason rather than throwing across the stream API. `BlockAssembler` is the single shared implementation that assembles chunks into blocks/messages. A successful `finish` may carry a `ReplayEnvelope` — opaque response-level replay metadata plus optional per-block entries aligned with the emitted block sequence. Assembly makes one keep/drop decision for content and metadata together: a `max-tokens` finish drops tool calls that may have been truncated, and the envelope loses the entry at each dropped position, so stored metadata always describes stored content.
@@ -95,7 +95,7 @@ None, as adapters choose when to add the shared image descriptors and per-image
#### KV Cache effect
Reasoning-effort materialization preserves the assembled request prefix. Image descriptors add deterministic text beside each image, and a request-limit transition replaces the deterministic oldest prefix with per-image text.
Reasoning-effort materialization preserves the assembled request prefix. Image identity and request-preview text are deterministic, while the optional execution-world path is resolved for each request. A changed path can alter a historical descriptor and prevent reuse from that image even without offload. Crossing a request limit also replaces an older image with per-image text.
## Known Limitations and Deferred Work
+2 -2
View File
@@ -57,7 +57,7 @@
消息内容是类型化内容块数组:`text``reasoning``image``tool-call``tool-result``ImageBlock` 只携带持久 `ImageAttachmentRef`;提供方字节和请求尺寸之后再解析。联合仍从可合并扩展的 `ContentBlockMap` 派生,因此插件可以通过 declaration merging 添加其他块类型。assistant 消息使用模型来源,其中携带生成该消息的提供方和模型,以及可选的适配器私有回放状态。dispatch 前,`LlmRuntime` 只在历史提供方路由与目标提供方路由当前由完全相同的适配器实例拥有时才保留该状态;随后由适配器判定能否在模型或提供方间恢复或转换该状态。
每次分发都使用随适配器世代捕获的确切模型模态。支持图片的适配器把持久图片引用投影为路由专用请求版本。纯文本路由则收到确定性的附件占位文本,其中也包括嵌套工具结果图片,追加式会话历史不会改变。`offloadRequestImagesWithPolicy()` 提供确定性的从旧到新图片移除,支持按原始字节或 base64 计数,也支持图片数量或字节量步长;适配器提供确切派生版本的字节长度和必填的逐图占位文本。
每次分发都使用随适配器世代捕获的确切模型模态。支持图片的适配器把持久图片引用投影为路由专用请求版本。`resolveImageAttachmentAccess()` 单独组合附件提供方可选的宿主对象,以及消费方给出的宿主路径到当前工具执行环境的映射。解析结果不进入 `RequestImageAttachment` 或其 `variantId`纯文本路由则收到确定性的附件占位文本,其中也包括嵌套工具结果图片,追加式会话历史不会改变。`offloadRequestImagesWithPolicy()` 提供确定性的从旧到新图片移除,支持按原始字节或 base64 计数,也支持图片数量或字节量步长;适配器提供确切派生版本的字节长度和必填的逐图占位文本。
流式输出是原始分片协议(`block-start``text-delta``reasoning-delta``tool-call-delta``block-end``usage``finish`)。每个适配器结果都以一个终止 `finish` 到达消费方;运行故障使用 `error``aborted` 作为结束原因,而不会跨流 API 抛出。`BlockAssembler` 是将分片组装为块/消息的唯一共享实现。成功的 `finish` 可以携带 `ReplayEnvelope`——不透明的响应级回放元数据,加上与发射块序列对齐的可选逐块条目。组装对内容与元数据只做一次保留/丢弃决定:`max-tokens` 结束会丢弃可能被截断的工具调用,数据在每个被丢弃的位置同步失去对应条目,因此存储的元数据始终描述存储的内容。
@@ -97,7 +97,7 @@
#### KV Cache 影响
推理强度填入不会改变已组装的请求前缀。图片描述会在每张图片旁加入确定性文本;请求越过上限时,确定性的最旧前缀会替换为逐图文本。
推理强度填入不会改变已组装的请求前缀。图片身份和请求预览文本具有确定性,可选的执行环境路径则按请求解析。路径变化会改写历史图片描述,即使没有 offload,也可能使缓存从该图片起无法复用。请求越过上限时,较旧图片也会替换为逐图文本。
<a id="known-limitations-and-deferred-work"></a>
+43 -4
View File
@@ -2,9 +2,43 @@
import type { ContentBlock } from './types.ts'
import type { Message } from './message.ts'
import type { ImageAttachmentAccess, ImageAttachmentRef, ImageMediaType, RequestImageAttachment } from '@deepseek-ai/dsh-attachment'
import type { AttachmentStore, ImageAttachmentRef, ImageMediaType, RequestImageAttachment } from '@deepseek-ai/dsh-attachment'
import { assertNever } from './never.ts'
/** Execution-world path that model tools can use to read one normalized attachment. */
export interface ImageAttachmentAccess {
/** Absolute path to immutable normalized bytes; callers must treat it as read-only. */
readonlyPath: string
}
/**
* Resolve current execution-world access for one durable image reference.
* @param ref - durable normalized attachment reference.
* @returns a read-only execution-world path, or undefined when unavailable.
*/
export type ImageAttachmentAccessResolver = (ref: ImageAttachmentRef) => ImageAttachmentAccess | undefined
/**
* Bridge one attachment provider's host object location into the mounted
* tool execution world. The consumer supplies the current filesystem
* provider's mapping without making attachment or LLM definitions depend on it.
* @param attachments - provider that owns the normalized attachment object.
* @param mapHostPath - map one absolute host path into the current tool execution world.
* @param ref - durable normalized attachment reference.
* @returns a read-only execution-world path, or undefined when either provider exposes no mapping.
* @throws an attachment error when the durable reference is invalid.
*/
export function resolveImageAttachmentAccess(
attachments: AttachmentStore,
mapHostPath: (hostPath: string) => string | undefined,
ref: ImageAttachmentRef,
): ImageAttachmentAccess | undefined {
const hostPath = attachments.imageHostPath(ref)
if (hostPath === undefined) return undefined
const readonlyPath = mapHostPath(hostPath)
return readonlyPath === undefined ? undefined : { readonlyPath }
}
function quoted(value: string): string {
return JSON.stringify(value)
}
@@ -48,13 +82,18 @@ export function textOnlyImageText(ref: ImageAttachmentRef): string {
* names differ.
* @param ref - the occurrence's durable normalized attachment.
* @param version - exact request image shown beside the text.
* @param access - optional path resolved for the current tool execution world.
* @returns attachment handle and request-image dimensions.
*/
export function requestImageHandleText(ref: ImageAttachmentRef, version: RequestImageAttachment): string {
export function requestImageHandleText(
ref: ImageAttachmentRef,
version: RequestImageAttachment,
access?: ImageAttachmentAccess,
): string {
const preview = `Image ${imageIdentity(ref)}; request preview ${version.width}x${version.height}px.`
return version.access === undefined
return access === undefined
? `${preview} It may be resized or re-encoded; source dimensions, format, and byte size may differ.`
: preview + normalizedAccessText(ref, version.access)
: preview + normalizedAccessText(ref, access)
}
/**
+29 -3
View File
@@ -1,12 +1,13 @@
import { describe, expect, it } from 'vitest'
import { AttachmentId, ImageVariantId } from '@deepseek-ai/dsh-attachment'
import type { ImageMediaType } from '@deepseek-ai/dsh-attachment'
import type { AttachmentStore, ImageMediaType } from '@deepseek-ai/dsh-attachment'
import {
CallId,
createUserMessage,
offloadedImageText,
offloadRequestImagesWithPolicy,
projectImagesForTextModel,
resolveImageAttachmentAccess,
requestImageHandleText,
} from '../src/index.ts'
import type { ContentBlock, Message } from '../src/index.ts'
@@ -186,10 +187,10 @@ describe('model-facing image access', () => {
height: 1536,
name: 'source "map".png',
}
const access = { readonlyPath: '/tmp/.dsh/attachments/v1/objects/bb/object' }
const version = {
variantId: ImageVariantId(`sha256:${'c'.repeat(64)}`),
attachment,
access: { readonlyPath: '/tmp/.dsh/attachments/v1/objects/bb/object' },
data: Uint8Array.of(1),
mediaType: 'image/png' as const,
bytes: 1,
@@ -199,7 +200,7 @@ describe('model-facing image access', () => {
space: 'srgb' as const,
hasAlpha: true,
}
expect(requestImageHandleText(attachment, version)).toBe(
expect(requestImageHandleText(attachment, version, access)).toBe(
`Image "source \\"map\\".png" (${attachment.attachmentId}); request preview 923x692px.`
+ ' Normalized copy (read-only; may be resized or re-encoded): "/tmp/.dsh/attachments/v1/objects/bb/object" (2048x1536px, image/png).'
+ ' Source dimensions, format, and byte size may differ.'
@@ -207,6 +208,31 @@ describe('model-facing image access', () => {
)
})
it('bridges a provider host object only through the mounted filesystem mapping', () => {
const attachment = image(1).attachment
const attachments = {
imageHostPath: () => '/host/.dsh/attachments/object',
} as unknown as AttachmentStore
const mapped = (hostPath: string): string | undefined => hostPath === '/host/.dsh/attachments/object'
? '/workspace/.attachments/object'
: undefined
expect(resolveImageAttachmentAccess(
attachments,
mapped,
attachment,
)).toEqual({ readonlyPath: '/workspace/.attachments/object' })
expect(resolveImageAttachmentAccess(
attachments,
() => undefined,
attachment,
)).toBeUndefined()
expect(resolveImageAttachmentAccess(
{ imageHostPath: () => undefined } as unknown as AttachmentStore,
mapped,
attachment,
)).toBeUndefined()
})
it('names each occurrence from its own reference when one prepared version is shared', () => {
const attachment = {
attachmentId: AttachmentId(`sha256:${'b'.repeat(64)}`),
+6 -6
View File
@@ -1136,12 +1136,6 @@ importers:
'@deepseek-ai/dsh-attachment':
specifier: workspace:^
version: link:../attachment
'@deepseek-ai/dsh-fs':
specifier: workspace:^
version: link:../../fs/fs
'@deepseek-ai/dsh-fs-local':
specifier: workspace:^
version: link:../../fs/fs-local
'@deepseek-ai/dsh-home-paths':
specifier: workspace:^
version: link:../../util/home-paths
@@ -6189,6 +6183,9 @@ importers:
'@deepseek-ai/dsh-deepseek-llm-api-extensions':
specifier: workspace:^
version: link:../deepseek-llm-api-extensions
'@deepseek-ai/dsh-fs':
specifier: workspace:^
version: link:../../fs/fs
'@deepseek-ai/dsh-home-paths':
specifier: workspace:^
version: link:../../util/home-paths
@@ -6238,6 +6235,9 @@ importers:
'@deepseek-ai/dsh-credentials':
specifier: workspace:^
version: link:../../credentials/credentials
'@deepseek-ai/dsh-fs':
specifier: workspace:^
version: link:../../fs/fs
'@deepseek-ai/dsh-invariants':
specifier: workspace:^
version: link:../../runtime-diagnostics/invariants
+1 -1
View File
@@ -335,7 +335,7 @@ export const LINK_MAP: Readonly<Record<string, string>> = {
ApprovalService: 'approval.md',
AskUserQuestionRequestEvent: 'user-questions.md',
EncodedImageAttachment: 'attachment.md',
ImageAttachmentAccess: 'attachment.md',
ImageAttachmentAccess: 'llm-streaming.md',
ImageAttachmentRef: 'attachment.md',
ImageRequestPolicy: 'attachment.md',
RequestImageAttachment: 'attachment.md',
+5 -5
View File
@@ -36,6 +36,11 @@
"symbol": "ContextFormed",
"source": "packages/llm/llm/src/message.ts"
},
{
"doc": "docs/subsystems/llm-streaming.md",
"symbol": "ImageAttachmentAccess",
"source": "packages/llm/llm/src/content.ts"
},
{
"doc": "docs/subsystems/llm-streaming.md",
"symbol": "FinishReasonMap",
@@ -931,11 +936,6 @@
"symbol": "StoredImageAttachment",
"source": "packages/attachment/attachment/src/types.ts"
},
{
"doc": "docs/subsystems/attachment.md",
"symbol": "ImageAttachmentAccess",
"source": "packages/attachment/attachment/src/types.ts"
},
{
"doc": "docs/subsystems/attachment.md",
"symbol": "ImageRequestPolicy",