mirror of
https://github.com/deepseek-ai/deepseek-harness.git
synced 2026-08-29 04:26:38 +00:00
Merge Claude permission modes into Codex permission modes
This commit is contained in:
pku-xht
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/bug-fix/2026-08-18-request-image-payload-bound.md
|
||||
2026-08-18-request-image-payload-bound.md: df10df39d18c7da4660b566e8f0b6a5a60ff8dc1
|
||||
2026-08-18-request-image-payload-bound.zh.md: 070f2d194f1459f3f2728fdf9d5d2db2c3a24385
|
||||
@@ -0,0 +1,35 @@
|
||||
# Agent Note: Request-level image payload bound
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-08-18-request-image-payload-bound.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
Every image in session history is base64-inlined into every model request by the pi-ai adapter, so a long session's request body grows monotonically with each admitted image. Gateways cap request-body size; once the accumulated payload crossed such a cap the request was rejected with 413 (`Failed to buffer the request body: length limit exceeded`), and because nothing bounds or trims the assembled request, every retry resent the same oversized body. The session was permanently unusable, and the failure text matched no `classifyPiAiError` rule, so it surfaced as the generic `PI_AI_ERROR`. Admission bounds (per image, per message) cannot prevent this: each image is individually admissible, and the sum still grows without bound. Two screenshots were enough to trigger it in production.
|
||||
|
||||
## Decision
|
||||
|
||||
The pi-ai provider profile carries `maxRequestImageBytes` (default `DEFAULT_MAX_REQUEST_IMAGE_BYTES = 20MiB`, a positive integer, per route, changeable from cordis.yml and the `llm-pi-ai` settings section). At request conversion, `toPiContext` sums the base64 length of every image in history (derived from `ImageAttachmentRef.bytes` without reading data) and, while the sum exceeds the bound, replaces the oldest images with a fixed model-facing placeholder. The placeholder tells the model to read the file again when a path is available or ask the user to attach the image again. The most recent images are omitted last; an image larger than the bound is itself omitted. Offload locations use message and nested block indexes rather than object identity, so replaying the same JSON log produces the same request. Offloaded images are never read from the attachment store. `classifyPiAiError` classifies 413 and specific request-body-cap wording as `INVALID_REQUEST` (resending the same body cannot succeed). Four images admitted at the attachment store's 3.5MiB raw-image default occupy at most 18.67MiB after base64 expansion. The 20MiB request-image default therefore retains four such images and reserves the rest of a 32MiB request for system prompts, history, tools, and JSON. Deployments behind stricter gateways lower the value per route.
|
||||
|
||||
## Offload is conversion, not history
|
||||
|
||||
The placeholder is model-visible but not logged as a session event. It stays within the model-visible ⟺ logged invariant the same way the adapter's other serialization does (`(no output)` fallbacks, text-only folding): the offload locations are a pure function of the logged history and the route configuration, so the exact request remains reconstructable from the session log plus the composition. A logged elision event becomes necessary only when offload decisions gain non-deterministic inputs (for example live gateway feedback), which belongs to the deferred capability-metadata design.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **Fail the request with a clear error instead of offloading.** Keeps the model informed but leaves the session wedged: the user cannot remove images from durable history, so a hard failure at the bound is permanent. Offload keeps the session serviceable, which is the point of the fix.
|
||||
- **Upload images once and reference them by URL / file id.** Removes the linear body growth entirely and is the right medium-term shape (providers and the internal gateway both document a Files path), but it introduces upload lifecycle management across providers and is far beyond a P0 hotfix.
|
||||
- **Count the full request body, not only images.** Text and tools contribute little and their sizes are only known after full serialization per protocol; bounding the dominant term with explicit headroom is accurate enough for the failure being fixed and much simpler. Revisit inside the route-capability design.
|
||||
- **Trim at admission instead.** Admission cannot see future accumulation; only the assembled request knows its total. Admission-side bounds (per-side dimension, bytes) remain as the first layer and are owned by [the dimension-limit note](2026-08-17-image-dimension-admission-limit.md).
|
||||
|
||||
## Related
|
||||
|
||||
- [Per-side image dimension admission limit](2026-08-17-image-dimension-admission-limit.md) — the admission-layer companion fix; together they close the two observed session-poisoning failures (400 dimension, 413 body size).
|
||||
|
||||
## Consequences
|
||||
|
||||
- An image-heavy long session keeps completing requests. The oldest images are omitted first; the most recent image is omitted only when it cannot fit within the bound.
|
||||
- Crossing the bound rewrites an early message, so the provider prompt-cache prefix ends at the newly offloaded image until the offloaded prefix stabilizes.
|
||||
- The bound counts base64 image payload only; deployments must keep it below their gateway's request-body cap with headroom, and the shipped default cannot know a private gateway's cap.
|
||||
- Route capability metadata driving admission and assembly together (image count, per-image size, request size, provider token formulas) remains deferred design work tracked outside this fix.
|
||||
@@ -0,0 +1,35 @@
|
||||
# Agent Note: 请求级图片载荷上限
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-08-18-request-image-payload-bound.md) | 中文
|
||||
|
||||
## Problem
|
||||
|
||||
pi-ai 适配器把会话历史中的每张图片 base64 内联进每一个模型请求，长会话的请求体随每张入库图片单调增长。网关对请求体大小设有上限；累积载荷一旦越线，请求被以 413 拒绝（`Failed to buffer the request body: length limit exceeded`），而组装层没有任何约束或裁剪，每次重试都会原样重发同一个超限请求体，会话永久不可用。该报错文本不匹配 `classifyPiAiError` 的任何规则，只能落进笼统的 `PI_AI_ERROR`。准入上限（单图、单消息）无法阻止这一点：每张图片单独看都合规，总和仍然无界增长。线上两张截图即可触发。
|
||||
|
||||
## Decision
|
||||
|
||||
pi-ai provider profile 增加 `maxRequestImageBytes`（默认 `DEFAULT_MAX_REQUEST_IMAGE_BYTES = 20MiB`，正整数，按路由生效，可从 cordis.yml 与 `llm-pi-ai` settings 段修改）。请求转换时，`toPiContext` 由 `ImageAttachmentRef.bytes` 推算每张历史图片的 base64 长度（无需读取数据）求和，总和超过上限时从最老的图片开始替换为一段固定的模型可见占位文本。占位文本要求模型在有路径时重新读取文件，否则请用户重新附上图片。越新的图片越晚被省略；单张图片本身超过上限时也会被省略。offload 位置用消息与嵌套块的索引表示，不依赖对象身份，因此重放同一份 JSON 日志会产生相同请求。被 offload 的图片不会从附件存储读取。`classifyPiAiError` 把 413 与明确的请求体上限措辞归类为 `INVALID_REQUEST`（原样重发不可能成功）。四张按附件存储默认上限准入的 3.5MiB 原始图片，经 base64 膨胀后最多占 18.67MiB。20MiB 请求图片默认上限因此可保留四张这样的图片，并在 32MiB 请求内为系统提示词、历史、工具与 JSON 保留其余容量。网关更严格的部署按路由调低该值。
|
||||
|
||||
## offload 是转换而非历史
|
||||
|
||||
占位文本模型可见，但不记录为会话事件。它与适配器的其他序列化（`(no output)` 回退、纯文本折叠）以同样的方式满足「模型可见 ⟺ 已记录」不变量：offload 位置是已记录历史与路由配置的纯函数，确切请求仍可由会话日志加组合配置重建。只有当 offload 决策引入非确定性输入（例如网关的实时反馈）时才需要记录省略事件，那属于暂缓的能力元数据设计。
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- **在上限处直接报错而不 offload。** 模型知情，但会话仍然卡死：用户无法从持久历史中删除图片，越线即永久失败。offload 让会话保持可用，这正是本修复的目标。
|
||||
- **图片上传一次、按 URL / file id 引用。** 从结构上消除请求体线性增长，是正确的中期形态（各提供方与内部网关都有 Files 路径），但要跨提供方管理上传生命周期，远超 P0 热修复范围。
|
||||
- **统计完整请求体而非只统计图片。** 文本与工具占比很小，且其大小要到按协议完整序列化后才可知；对主导项设上限并留出显式余量，对所修故障足够精确且简单得多。留到路由能力设计中再议。
|
||||
- **改在准入侧裁剪。** 准入看不到未来的累积，只有组装后的请求知道自己的总量。准入侧上限（单边尺寸、字节）作为第一层保留，归[尺寸上限笔记](2026-08-17-image-dimension-admission-limit.md)所有。
|
||||
|
||||
## Related
|
||||
|
||||
- [图片单边尺寸准入上限](2026-08-17-image-dimension-admission-limit.md)，准入层的配套修复；两者合起来封住已观测到的两类会话毒化故障（400 尺寸、413 请求体）。
|
||||
|
||||
## Consequences
|
||||
|
||||
- 图片较多的长会话持续可用。最老的图片优先省略；仅当最新图片本身无法装进上限时才会省略它。
|
||||
- 越过上限会改写较早的一条消息，提供方 prompt cache 前缀在新被 offload 的图片处截止，直到被 offload 的前缀稳定。
|
||||
- 上限只统计 base64 图片载荷；部署必须让它低于自家网关的请求体上限并留出余量，发行默认值无法预知私有网关的上限。
|
||||
- 由路由能力元数据同时驱动准入与组装（图片数量、单图大小、请求大小、提供方 token 公式）的设计仍为暂缓工作，在本修复之外跟踪。
|
||||
+2
-2
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-22-web-multimodal-image-input-and-durable-attachments.md
|
||||
2026-07-22-web-multimodal-image-input-and-durable-attachments.md: 8c07b8b786aeeb87a4c2db7c0e6e49928b0ddf2c
|
||||
2026-07-22-web-multimodal-image-input-and-durable-attachments.zh.md: 5d2d2963d6bee729362852701ea58f191f9721b2
|
||||
2026-07-22-web-multimodal-image-input-and-durable-attachments.md: 6a12380992c3205d6bb3f5701f093b3335fae2f9
|
||||
2026-07-22-web-multimodal-image-input-and-durable-attachments.zh.md: a8e94154e435a5359e7928d3b11eba7d1aee92f7
|
||||
|
||||
+1
-1
@@ -140,7 +140,7 @@ Composer thumbnails and each `MessageImage` own ephemeral original-preview state
|
||||
|
||||
### Limits and trust boundaries
|
||||
|
||||
Version one accepts PNG, JPEG, WebP, and GIF only. SVG and remote URLs are excluded. Default limits are 5 MiB per image, 20 images and 100 MiB aggregate image bytes per message, 40 million intrinsic pixels per image, and 2000 pixels on either side. These deployment-varying limits are validated backend configuration and enforced by the host before persistence. The client connection carrier has an independent configurable `maxRequestBodyBytes` cap (160 MiB by default) for every API request and fails load if it cannot hold the attachment service's aggregate image limit after base64 and envelope expansion; lowering image policy therefore never silently lowers the carrier limit for valid text or other RPCs. A body without a declared length is rejected the moment it crosses the cap rather than drained to its end.
|
||||
Version one accepts PNG, JPEG, WebP, and GIF only. SVG and remote URLs are excluded. Default limits are 3.5 MiB per image, 20 images and 100 MiB aggregate image bytes per message, 40 million intrinsic pixels per image, and 2000 pixels on either side. These deployment-varying limits are validated backend configuration and enforced by the host before persistence. The client connection carrier has an independent configurable `maxRequestBodyBytes` cap (160 MiB by default) for every API request and fails load if it cannot hold the attachment service's aggregate image limit after base64 and envelope expansion; lowering image policy therefore never silently lowers the carrier limit for valid text or other RPCs. A body without a declared length is rejected the moment it crosses the cap rather than drained to its end.
|
||||
|
||||
Malformed base64, unsupported or mismatched media, truncated image payloads, excess bytes, excess image count, excess pixels, excess per-side dimensions, missing objects, and integrity mismatches return stable structured failures. Original filenames are reduced to a display basename, control characters are removed, and no local path is logged or returned to the browser.
|
||||
|
||||
|
||||
+1
-1
@@ -140,7 +140,7 @@ Pi-AI 适配器是首条视觉输入路径：它在请求时解析 `ctx.attachme
|
||||
|
||||
### 限制与信任边界
|
||||
|
||||
第一版仅接受 PNG、JPEG、WebP 和 GIF。不接受 SVG 和远程 URL。默认限制为每张图片 5 MiB、每条消息 20 张图片和 100 MiB 图片总字节数、每张图片 4,000 万个固有像素，以及任一边 2,000 像素。这些随部署变化的限制属于经过校验的后端配置，并由宿主在持久化前强制执行。客户端连接载体为每个 API 请求设置独立且可配置的 `maxRequestBodyBytes` 上限（默认 160 MiB）；如果该上限无法容纳附件服务的图片总量限制经 base64 和请求封装膨胀后的大小，加载就会失败。因此，降低图片策略绝不会静默降低有效文本或其他 RPC 的载体上限。未声明长度的请求体在越过上限的瞬间即被拒绝，而不是先读完再拒。
|
||||
第一版仅接受 PNG、JPEG、WebP 和 GIF。不接受 SVG 和远程 URL。默认限制为每张图片 3.5 MiB、每条消息 20 张图片和 100 MiB 图片总字节数、每张图片 4,000 万个固有像素，以及任一边 2,000 像素。这些随部署变化的限制属于经过校验的后端配置，并由宿主在持久化前强制执行。客户端连接载体为每个 API 请求设置独立且可配置的 `maxRequestBodyBytes` 上限（默认 160 MiB）；如果该上限无法容纳附件服务的图片总量限制经 base64 和请求封装膨胀后的大小，加载就会失败。因此，降低图片策略绝不会静默降低有效文本或其他 RPC 的载体上限。未声明长度的请求体在越过上限的瞬间即被拒绝，而不是先读完再拒。
|
||||
|
||||
格式错误的 base64、不支持或不匹配的媒体、截断的图片数据、超出字节限制、超出图片数量、超出像素限制、超出单边尺寸限制、对象缺失和完整性不匹配都会返回稳定的结构化错误。原始文件名只保留用于显示的末段，控制字符会被移除，并且任何本地路径都不会写入日志或返回浏览器。
|
||||
|
||||
|
||||
+2
-2
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-08-12-web-image-intake-and-limits-alignment.md
|
||||
2026-08-12-web-image-intake-and-limits-alignment.md: 2f8b99bb4850d9875dcba0a03ae8ad9f340d1506
|
||||
2026-08-12-web-image-intake-and-limits-alignment.zh.md: 62d5ebd54275ae8de0e0b9ba701ba34042dfcde7
|
||||
2026-08-12-web-image-intake-and-limits-alignment.md: 00cf7ea99d63e848c4b5839da1d97d94c9fb8464
|
||||
2026-08-12-web-image-intake-and-limits-alignment.zh.md: d88ba25a3f3e3be4a3655080a45123ada6cdebcd
|
||||
|
||||
+1
-1
@@ -16,7 +16,7 @@ The second alignment step for issue #2248, after the [attachment display note](2
|
||||
|
||||
**History thumbnails (DeepSeek Chat rules).** A message's lone image renders at 240px on its long edge with the displayed ratio clamped to [0.25, 4], cropped by `cover` with the anchor at the top of very tall images and the left of very wide ones, never upscaled; several images render as fixed 64px square tiles in one wrapping row (10px gap, user messages right-aligned). Consecutive assistant `image` blocks merge into one gallery so they tile instead of each opening a one-image row.
|
||||
|
||||
**Limits aligned and projected.** Defaults are 20 images / 5 MiB per image / 100 MiB aggregate (`attachment-local`), with the HTTP carrier cap raised to one shared `DEFAULT_MAX_REQUEST_BODY_BYTES = 160 MiB` (http-bridge, previously two independent 32 MiB literals) to satisfy the load-time capacity assertion (aggregate × 4/3 + headroom ≈ 134.3 MiB). Consumer products cluster at 10–20 attachments (ChatGPT 10, Gemini 10, Claude 20; DeepSeek Chat's 50 is the outlier), and a vision-model image costs roughly 1300–4800 tokens, so 50 images can fill a 200k context in one message. The 5 MiB per-image default admits images across Anthropic routes that impose either a 5 MiB or 10 MiB maximum; deployments using only routes with the larger limit can override it. A 512 MiB aggregate cannot pass this transport because base64-in-JSON would need a single JSON string past V8's ~512 MiB string ceiling. The limits reach clients as the `imageLimits` session projection — a constant-per-boot unit (`apply` returns the same state reference, so baselines alone carry it and no change frames exist) registered by **apiproxy**, not the attachment Service Definition: `dsh-llm` depends on `dsh-attachment` (`ImageBlock` → `ImageAttachmentRef`), so the seam package referencing `dsh-session-projection` (whose graph reaches `dsh-llm` through `dsh-session`) closes a project-reference cycle, and the per-message count/aggregate rules the value describes are the proxy's own admission checks anyway. The `SessionProjectionMap` merge rides the proxy's sessions wire-contract file, which every client program already includes through the carrier's type re-exports.
|
||||
**Limits aligned and projected.** Defaults are 20 images / 3.5 MiB per image / 100 MiB aggregate (`attachment-local`), with the HTTP carrier cap raised to one shared `DEFAULT_MAX_REQUEST_BODY_BYTES = 160 MiB` (http-bridge, previously two independent 32 MiB literals) to satisfy the load-time capacity assertion (aggregate × 4/3 + headroom ≈ 134.3 MiB). Consumer products cluster at 10–20 attachments (ChatGPT 10, Gemini 10, Claude 20; DeepSeek Chat's 50 is the outlier), and a vision-model image costs roughly 1300–4800 tokens, so 50 images can fill a 200k context in one message. Including base64 padding, a 3.5 MiB encoded file occupies at most 4.67 MiB and leaves 0.33 MiB below a 5 MiB route check. Deployments using only routes with larger limits can override it. A 512 MiB aggregate cannot pass this transport because base64-in-JSON would need a single JSON string past V8's ~512 MiB string ceiling. The limits reach clients as the `imageLimits` session projection — a constant-per-boot unit (`apply` returns the same state reference, so baselines alone carry it and no change frames exist) registered by **apiproxy**, not the attachment Service Definition: `dsh-llm` depends on `dsh-attachment` (`ImageBlock` → `ImageAttachmentRef`), so the seam package referencing `dsh-session-projection` (whose graph reaches `dsh-llm` through `dsh-session`) closes a project-reference cycle, and the per-message count/aggregate rules the value describes are the proxy's own admission checks anyway. The `SessionProjectionMap` merge rides the proxy's sessions wire-contract file, which every client program already includes through the carrier's type re-exports.
|
||||
|
||||
**Intake pre-check and error copy.** Both intake gestures converge on one `intakeImages` wrapper in InputBar that checks count, per-image bytes, and aggregate bytes against the projection before `addImages`: a violating batch is refused whole (DeepSeek Chat semantics) with an immediate banner naming the limit — no submit-time rollback theater. The host checks stay as the backstop for callers that bypass the composer. Banner copy follows one principle the user set: reasons a user can act on (model without vision, count, size, resolution, format — now a positive list of supported formats instead of echoing the rejected MIME type) get product sentences naming the way out; reasons they cannot act on (corrupt base64, lost references, read failures) fold into one send-failed sentence that keeps the reason code, because the product currently faces developers and a reportable code beats a dead end. Non-attachment error codes keep the raw message + code presentation.
|
||||
|
||||
|
||||
+1
-1
@@ -16,7 +16,7 @@ issue #2248 的第二步对齐，接在[附件展示 note](2026-08-11-web-attach
|
||||
|
||||
**历史缩略图（DeepSeek Chat 规则）。** 一条消息仅有的一张图长边 240px、展示比例钳制在 [0.25, 4]，`cover` 裁切，特别高的图锚定顶部、特别宽的锚定左侧，从不放大；多张图渲染为固定 64px 方块，单个可换行的横排（10px 间距，用户消息右对齐）。assistant 连续的 `image` 块合并进同一个画廊，平铺而不是各占一行。
|
||||
|
||||
**上限对齐并投影。** 默认值为每条消息 20 张、单图 5 MiB、总量 100 MiB（`attachment-local`），HTTP 载体上限提为唯一共享的 `DEFAULT_MAX_REQUEST_BODY_BYTES = 160 MiB`（http-bridge，原先是两个独立的 32 MiB 字面量），以满足加载时的容量断言（总量 × 4/3 加余量 ≈ 134.3 MiB）。消费级产品集中在 10 到 20 个附件（ChatGPT 10、Gemini 10、Claude 20；DeepSeek Chat 的 50 是例外），且视觉模型一张图约 1300 到 4800 token，因此 50 张图可在一条消息中填满 200k 上下文。默认单图上限采用 5 MiB，可适用于分别采用 5 MiB 或 10 MiB 上限的 Anthropic 路由；仅使用较大上限路由的部署可以覆盖该值。512 MiB 总量无法通过当前传输，因为 base64 进 JSON 需要一个超过 V8 约 512 MiB 字符串上限的单个 JSON 字符串。限额以 `imageLimits` 会话投影到达客户端。它是每次启动恒定的单元（`apply` 返回同一状态引用，因此只靠基线携带、不存在变更帧），由 **apiproxy** 而非 attachment Service Definition 注册：`dsh-llm` 依赖 `dsh-attachment`（`ImageBlock` → `ImageAttachmentRef`），seam 包引用 `dsh-session-projection`（其图谱经 `dsh-session` 到达 `dsh-llm`）会闭合 project-reference 环，而该值描述的每消息数量与总量规则本来就是 proxy 自己的准入检查。`SessionProjectionMap` 合并放在 proxy 的 sessions 协议文件里，每个客户端程序都经载体的类型再导出包含它。
|
||||
**上限对齐并投影。** 默认值为每条消息 20 张、单图 3.5 MiB、总量 100 MiB（`attachment-local`），HTTP 载体上限提为唯一共享的 `DEFAULT_MAX_REQUEST_BODY_BYTES = 160 MiB`（http-bridge，原先是两个独立的 32 MiB 字面量），以满足加载时的容量断言（总量 × 4/3 加余量 ≈ 134.3 MiB）。消费级产品集中在 10 到 20 个附件（ChatGPT 10、Gemini 10、Claude 20；DeepSeek Chat 的 50 是例外），且视觉模型一张图约 1300 到 4800 token，因此 50 张图可在一条消息中填满 200k 上下文。3.5 MiB 编码文件包括 base64 填充在内最多占 4.67 MiB，在 5 MiB 路由检查下保留 0.33 MiB 余量。仅使用较大上限路由的部署可以覆盖该值。512 MiB 总量无法通过当前传输，因为 base64 进 JSON 需要一个超过 V8 约 512 MiB 字符串上限的单个 JSON 字符串。限额以 `imageLimits` 会话投影到达客户端。它是每次启动恒定的单元（`apply` 返回同一状态引用，因此只靠基线携带、不存在变更帧），由 **apiproxy** 而非 attachment Service Definition 注册：`dsh-llm` 依赖 `dsh-attachment`（`ImageBlock` → `ImageAttachmentRef`），seam 包引用 `dsh-session-projection`（其图谱经 `dsh-session` 到达 `dsh-llm`）会闭合 project-reference 环，而该值描述的每消息数量与总量规则本来就是 proxy 自己的准入检查。`SessionProjectionMap` 合并放在 proxy 的 sessions 协议文件里，每个客户端程序都经载体的类型再导出包含它。
|
||||
|
||||
**加入预检与错误文案。** 两种加入手势汇合到 InputBar 的一个 `intakeImages` 包装：在 `addImages` 之前按投影检查数量、单图字节与总字节，违规的一批整体拒收（DeepSeek Chat 语义）并立刻弹出点名上限的横幅——不再有提交时的回滚戏码。宿主检查保留，兜底绕过 composer 的调用方。横幅文案遵循用户定下的一条原则：用户能解决的原因（模型不支持视觉、数量、大小、分辨率、格式——格式改为正面列出支持列表而不是回显被拒的 MIME 类型）用点明出路的产品句子；用户无法解决的原因（base64 损坏、引用丢失、读取失败）折叠为一条保留原因码的发送失败句子，因为产品当前面向开发者，可上报的码好过死胡同。非附件错误码保留原文加错误码的展示。
|
||||
|
||||
|
||||
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/process/2026-08-17-readme-assets-on-cdn.md
|
||||
2026-08-17-readme-assets-on-cdn.md: 8918e2118123382553e1ea295351482030e0e640
|
||||
2026-08-17-readme-assets-on-cdn.zh.md: a1314068ff0bbebfef5bb0053f04aebe650f6101
|
||||
@@ -0,0 +1,35 @@
|
||||
# Agent Note: README assets publish from a dedicated repository
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-08-17-readme-assets-on-cdn.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
The public Chinese README embeds three community QR codes. Repository-relative images make each replacement depend on a source change and the separate public-repository publication flow, even though the image bytes do not change product code or documentation text.
|
||||
|
||||
The images need stable public URLs while their source bytes, publication credentials, cache behavior, and update history remain explicit and reviewable.
|
||||
|
||||
## Decision
|
||||
|
||||
The README references fixed URLs under `https://cdn.deepseek.com/harness/readme/`. The private [`deepseek-harness/readme-cdn-assets`](https://github.com/deepseek-harness/readme-cdn-assets) repository owns the three allowlisted PNG files, their tests, and their publication code. A push to its `master` branch runs `publish.yml`, which installs the pinned Huawei OBS SDK, tests `scripts/upload.mjs`, and publishes the images.
|
||||
|
||||
The uploader accepts only the three README filenames, verifies each source is a PNG file, and uploads it to `dp-cdn-deepseek/harness/readme/` with `Content-Type: image/png` and `Cache-Control: no-store`. It checks the OBS response status, reports the resulting public URL, and closes the client on both success and failure. Repository Actions Secrets supply `OBS_DSH_README_ACCESS_KEY_ID` and `OBS_DSH_README_SECRET_ACCESS_KEY`; the OBS identity needs write access only to that object prefix.
|
||||
|
||||
The assets repository provides the update history and rollback source. The public README keeps the same URLs across image replacements, so ordinary image updates do not require a product-repository change or a public-repository synchronization.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**Keep repository-relative images on `master`.** This preserves GitHub as the only image host, but every operational QR-code replacement remains coupled to the code review and public-repository publication path.
|
||||
|
||||
**Keep a long-lived assets branch in the product repository.** A branch avoids product `master` changes, but it leaves image ownership, OBS credentials, and publication workflow attached to the product repository and its repository-wide automation. A dedicated repository gives that operational source one default branch and one narrow responsibility.
|
||||
|
||||
**Use content-addressed CDN object names.** Immutable objects avoid stale caches, but each image replacement must also change the README URL, which removes the independent update path this workflow exists to provide.
|
||||
|
||||
**Allow the uploader to publish arbitrary paths.** A generic uploader could serve future assets without code changes, but the same credentials could then overwrite unrelated CDN objects. The fixed allowlist keeps this publication job limited to the README images it owns.
|
||||
|
||||
## Consequences
|
||||
|
||||
Community QR codes can change through one assets-repository push while the public README remains unchanged. The product repository carries no OBS dependency or credential, uploads retain an auditable git source, and CDN responses carry `Cache-Control: no-store`.
|
||||
|
||||
The README depends on the public CDN and GitHub's image proxy, while publication depends on a second private repository and its two Actions Secrets. `no-store` gives up edge and browser caching for these small files.
|
||||
@@ -0,0 +1,35 @@
|
||||
# Agent Note: README 资产通过专用仓库发布
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-08-17-readme-assets-on-cdn.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
公开中文 README 嵌入了 3 张社区二维码。使用仓库相对路径时，每次替换都依赖源码变更以及独立的公开仓库发布流程，即使图片字节并未改变产品代码或文档文字。
|
||||
|
||||
这些图片需要稳定的公开 URL，同时必须明确并可评审地保存源文件字节、发布凭证、缓存行为和更新历史。
|
||||
|
||||
## 决策
|
||||
|
||||
README 引用 `https://cdn.deepseek.com/harness/readme/` 下的固定 URL。私有仓库 [`deepseek-harness/readme-cdn-assets`](https://github.com/deepseek-harness/readme-cdn-assets) 负责管理 3 张允许发布的 PNG 文件、相应测试和发布代码。向该仓库的 `master` 分支 push 会运行 `publish.yml`，安装固定版本的华为云 OBS SDK、测试 `scripts/upload.mjs` 并发布图片。
|
||||
|
||||
上传脚本只接受 3 个 README 图片文件名，验证每个源文件均为 PNG，并以 `Content-Type: image/png` 和 `Cache-Control: no-store` 上传到 `dp-cdn-deepseek/harness/readme/`。脚本检查 OBS 响应状态、报告对应公开 URL，并在成功或失败后关闭客户端。仓库级 GitHub Actions Secret 提供 `OBS_DSH_README_ACCESS_KEY_ID` 和 `OBS_DSH_README_SECRET_ACCESS_KEY`；OBS 身份只需拥有该对象前缀的写权限。
|
||||
|
||||
资产仓库提供更新记录和回滚真源。图片替换后，公开 README 继续使用相同 URL，因此常规图片更新无需修改产品仓库或同步公开仓库。
|
||||
|
||||
## 曾考虑的替代方案
|
||||
|
||||
**继续在 `master` 上使用仓库相对图片。**这种做法只使用 GitHub 托管图片，但每次运营二维码替换仍与代码评审和公开仓库发布流程耦合。
|
||||
|
||||
**在产品仓库中保留长期资产分支。**资产分支可以避免修改产品 `master`，但图片所有权、OBS 凭证和发布工作流仍依附于产品仓库及其全仓自动化。专用仓库为这项运营资源提供单一默认分支和单一职责。
|
||||
|
||||
**使用内容寻址的 CDN 对象名。**不可变对象不会产生陈旧缓存，但每次替换图片还必须修改 README URL，无法提供此工作流所需的独立更新路径。
|
||||
|
||||
**允许上传脚本发布任意路径。**通用上传脚本可以在不改代码的情况下支持未来资产，但同一组凭证也能覆盖无关 CDN 对象。固定允许列表将发布任务限制在它负责的 README 图片内。
|
||||
|
||||
## 后果
|
||||
|
||||
社区二维码可以通过一次资产仓库 push 更新，公开 README 无需改变。产品仓库不携带 OBS 依赖或凭证；上传内容保留可审计的 git 真源；CDN 响应携带 `Cache-Control: no-store`。
|
||||
|
||||
README 依赖公开 CDN 和 GitHub 图片代理，发布流程则依赖另一个私有仓库及其 2 个 GitHub Actions Secret。`no-store` 为这些小文件放弃边缘节点和浏览器缓存。
|
||||
+1
-1
@@ -3,4 +3,4 @@
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write README.md
|
||||
README.md: 8a4bd01332a23ce4144c661784bc549e0ba72d21
|
||||
README.zh.md: b7bc214bfb1fd8a76a47de3f0aa242122aeb7603
|
||||
README.zh.md: c507bf884bd426feead6a96adbdb5c136456e3b5
|
||||
|
||||
+3
-3
@@ -50,9 +50,9 @@ pnpm dsh web
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td align="center"><img src="assets/community-wecom-assistant.png" alt="DeepSeek Harness 企微小助手二维码" width="180" height="180"></td>
|
||||
<td align="center"><a href="https://trtgsjkv6r.feishu.cn/share/base/form/shrcnIt5twSVdLGD52KJBckGCgg"><img src="assets/community-wecom-survey.png" alt="DeepSeek Harness 入群问卷二维码" width="180" height="180"></a></td>
|
||||
<td align="center"><img src="assets/community-wechat-official-account.png" alt="DeepSeek Harness 团队微信公众号二维码" width="180" height="180"></td>
|
||||
<td align="center"><img src="https://cdn.deepseek.com/harness/readme/community-wecom-assistant.png" alt="DeepSeek Harness 企微小助手二维码" width="180" height="180"></td>
|
||||
<td align="center"><a href="https://trtgsjkv6r.feishu.cn/share/base/form/shrcnIt5twSVdLGD52KJBckGCgg"><img src="https://cdn.deepseek.com/harness/readme/community-wecom-survey.png" alt="DeepSeek Harness 入群问卷二维码" width="180" height="180"></a></td>
|
||||
<td align="center"><img src="https://cdn.deepseek.com/harness/readme/community-wechat-official-account.png" alt="DeepSeek Harness 团队微信公众号二维码" width="180" height="180"></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 35 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 59 KiB |
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write docs/config-catalog.md
|
||||
config-catalog.md: 53c7dd3b46ba4221a05655ce7694d6c47499b6b7
|
||||
config-catalog.zh.md: ee7c8a3a34ef784fea74b7282ce9c01b72ccbdf9
|
||||
config-catalog.md: 3a471ea06e911d3d29ebef4ce80353b15a21cb43
|
||||
config-catalog.zh.md: f1b53d4a2b3c15abb3176c7af5de9c577f74825f
|
||||
|
||||
@@ -985,6 +985,13 @@ export interface PiAiProviderProfile {
|
||||
websocketConnectTimeoutMs?: number
|
||||
/** Maximum provider idle time while one stream read is outstanding. */
|
||||
streamIdleTimeoutMs?: number
|
||||
/**
|
||||
* Maximum base64-encoded image payload per request. When a request's
|
||||
* accumulated images exceed it, the oldest images are replaced by text
|
||||
* placeholders until the request fits, so a long session keeps completing
|
||||
* requests instead of being rejected by a request-size cap.
|
||||
*/
|
||||
maxRequestImageBytes?: number
|
||||
/** Provider-owned model-request retry policy; omission uses normal defaults. */
|
||||
retryPolicy?: RetryPolicyConfig
|
||||
}
|
||||
@@ -1081,7 +1088,7 @@ type WithheldThinkingFormat = 'chat-template' | 'qwen-chat-template'
|
||||
|
||||
Depends on: `Api` (`@earendil-works/pi-ai`) · `CacheRetention` (`@earendil-works/pi-ai`) · `Model` (`@earendil-works/pi-ai`) · `ModelThinkingLevel` (`@earendil-works/pi-ai`) · `OpenAICompletionsCompat` (`@earendil-works/pi-ai`) · [`RetryPolicyConfig`](../packages/llm/llm/src/index.ts) · `ThinkingBudgets` (`@earendil-works/pi-ai`) · `Transport` (`@earendil-works/pi-ai`)
|
||||
|
||||
Source: [`packages/llm/llm-pi-ai/src/config.ts:172`](../packages/llm/llm-pi-ai/src/config.ts)
|
||||
Source: [`packages/llm/llm-pi-ai/src/config.ts:192`](../packages/llm/llm-pi-ai/src/config.ts)
|
||||
|
||||
<a id="deepseek-aidsh-llm-replay"></a>
|
||||
|
||||
|
||||
@@ -987,6 +987,13 @@ export interface PiAiProviderProfile {
|
||||
websocketConnectTimeoutMs?: number
|
||||
/** Maximum provider idle time while one stream read is outstanding. */
|
||||
streamIdleTimeoutMs?: number
|
||||
/**
|
||||
* Maximum base64-encoded image payload per request. When a request's
|
||||
* accumulated images exceed it, the oldest images are replaced by text
|
||||
* placeholders until the request fits, so a long session keeps completing
|
||||
* requests instead of being rejected by a request-size cap.
|
||||
*/
|
||||
maxRequestImageBytes?: number
|
||||
/** Provider-owned model-request retry policy; omission uses normal defaults. */
|
||||
retryPolicy?: RetryPolicyConfig
|
||||
}
|
||||
@@ -1083,7 +1090,7 @@ type WithheldThinkingFormat = 'chat-template' | 'qwen-chat-template'
|
||||
|
||||
依赖：`Api`（`@earendil-works/pi-ai`）· `CacheRetention`（`@earendil-works/pi-ai`）· `Model`（`@earendil-works/pi-ai`）· `ModelThinkingLevel`（`@earendil-works/pi-ai`）· `OpenAICompletionsCompat`（`@earendil-works/pi-ai`）· [`RetryPolicyConfig`](../packages/llm/llm/src/index.ts) · `ThinkingBudgets`（`@earendil-works/pi-ai`）· `Transport`（`@earendil-works/pi-ai`）
|
||||
|
||||
来源：[`packages/llm/llm-pi-ai/src/config.ts:172`](../packages/llm/llm-pi-ai/src/config.ts)
|
||||
来源：[`packages/llm/llm-pi-ai/src/config.ts:192`](../packages/llm/llm-pi-ai/src/config.ts)
|
||||
|
||||
<a id="deepseek-aidsh-llm-replay"></a>
|
||||
|
||||
|
||||
@@ -1,11 +1,19 @@
|
||||
import { fileURLToPath } from 'node:url'
|
||||
import { readFileSync } from 'node:fs'
|
||||
import { spawnSync } from 'node:child_process'
|
||||
import { createServer } from 'node:http'
|
||||
import type { IncomingMessage, ServerResponse } from 'node:http'
|
||||
import { mkdir, utimes, writeFile } from 'node:fs/promises'
|
||||
import { dirname, join } from 'node:path'
|
||||
import { homedir } from 'node:os'
|
||||
import { expect, it } from 'vitest'
|
||||
import { defineAcpSnapshotSuite, type Scenario, type SnapshotSuiteOptions } from '@deepseek-ai/dsh-acp-snapshot'
|
||||
import {
|
||||
defineAcpSnapshotSuite,
|
||||
runScenario,
|
||||
type InputScript,
|
||||
type Scenario,
|
||||
type SnapshotSuiteOptions,
|
||||
} from '@deepseek-ai/dsh-acp-snapshot'
|
||||
import { resolvePwshPath } from '@deepseek-ai/dsh-pwsh-local'
|
||||
import { decodeStorageRecord } from '@deepseek-ai/dsh-session'
|
||||
|
||||
@@ -40,6 +48,7 @@ const ADVANCED_CONFIG = fileURLToPath(new URL('../advanced.cordis.yml', import.m
|
||||
const FS_CONFIG = fileURLToPath(new URL('../fs.cordis.yml', import.meta.url))
|
||||
const SESSION_QUERY_CONFIG = fileURLToPath(new URL('../session-query.cordis.yml', import.meta.url))
|
||||
const IMAGE_CONFIG = fileURLToPath(new URL('../image.cordis.yml', import.meta.url))
|
||||
const IMAGE_OFFLOAD_CONFIG = fileURLToPath(new URL('./fixtures/image-offload.cordis.yml', import.meta.url))
|
||||
const IMAGE_TEXT_ROUTE_CONFIG = fileURLToPath(new URL('../image-text-route.cordis.yml', import.meta.url))
|
||||
const PTY_CONFIG = fileURLToPath(new URL('../pty.cordis.yml', import.meta.url))
|
||||
const DEPTH_TWO_CONFIG = fileURLToPath(new URL('../depth-two.cordis.yml', import.meta.url))
|
||||
@@ -666,6 +675,92 @@ defineAcpSnapshotSuite({
|
||||
hasPwsh,
|
||||
})
|
||||
|
||||
it('pins pi-ai image offload in the request sent by the assembled app', async () => {
|
||||
const requests: Record<string, unknown>[] = []
|
||||
const server = createServer((request: IncomingMessage, response: ServerResponse) => {
|
||||
let body = ''
|
||||
request.setEncoding('utf8')
|
||||
request.on('data', (chunk: string) => { body += chunk })
|
||||
request.on('end', () => {
|
||||
requests.push(JSON.parse(body) as Record<string, unknown>)
|
||||
response.writeHead(200, { 'content-type': 'text/event-stream' })
|
||||
response.end([
|
||||
'data: {"choices":[{"delta":{"role":"assistant","content":""},"index":0,"finish_reason":null}]}',
|
||||
'data: {"choices":[{"delta":{"content":"DONE"},"index":0,"finish_reason":null}]}',
|
||||
'data: {"choices":[{"delta":{},"index":0,"finish_reason":"stop"}],"usage":{"prompt_tokens":3,"completion_tokens":1}}',
|
||||
'data: [DONE]',
|
||||
'',
|
||||
].join('\n\n'))
|
||||
})
|
||||
})
|
||||
await new Promise<void>(resolve => server.listen(0, '127.0.0.1', resolve))
|
||||
const address = server.address()
|
||||
if (address === null || typeof address === 'string') throw new Error('image-offload snapshot server has no port')
|
||||
|
||||
const image = 'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAADElEQVR4nGP4z8AAAAMBAQDJ/pLvAAAAAElFTkSuQmCC'
|
||||
const input: InputScript = {
|
||||
steps: [
|
||||
{ op: 'initialize' },
|
||||
{ op: 'newSession' },
|
||||
{
|
||||
op: 'promptContent',
|
||||
content: [
|
||||
{ type: 'text', text: 'Compare the older image ' },
|
||||
{ type: 'image', data: image, mimeType: 'image/png' },
|
||||
{ type: 'text', text: ' with the newer image ' },
|
||||
{ type: 'image', data: image, mimeType: 'image/png' },
|
||||
{ type: 'text', text: ', then reply with DONE.' },
|
||||
],
|
||||
},
|
||||
],
|
||||
}
|
||||
|
||||
try {
|
||||
const result = await runScenario(input, {
|
||||
agent: AGENT,
|
||||
mode: 'record',
|
||||
configPath: IMAGE_OFFLOAD_CONFIG,
|
||||
fixtureFile: join(SNAPSHOTS_DIR, 'image-offload-request', 'session.jsonl'),
|
||||
env: {
|
||||
DSH_SNAPSHOT_API_KEY: 'snapshot-key',
|
||||
DSH_SNAPSHOT_BASE_URL: `http://127.0.0.1:${address.port}/v1`,
|
||||
},
|
||||
})
|
||||
expect(result.stderr).toBe('')
|
||||
expect(requests).toHaveLength(1)
|
||||
const messages = requests[0]?.messages as { content?: unknown }[] | undefined
|
||||
const offloaded = messages?.find(message => JSON.stringify(message.content).includes('[image omitted'))
|
||||
expect(offloaded?.content).toMatchInlineSnapshot(`
|
||||
[
|
||||
{
|
||||
"text": "Compare the older image ",
|
||||
"type": "text",
|
||||
},
|
||||
{
|
||||
"text": "[image omitted to keep the request within its image limit; older images are omitted first. If this image is still needed, read its file again when a path is available; otherwise ask the user to attach it again.]",
|
||||
"type": "text",
|
||||
},
|
||||
{
|
||||
"text": " with the newer image ",
|
||||
"type": "text",
|
||||
},
|
||||
{
|
||||
"image_url": {
|
||||
"url": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAIAAACQd1PeAAAADElEQVR4nGP4z8AAAAMBAQDJ/pLvAAAAAElFTkSuQmCC",
|
||||
},
|
||||
"type": "image_url",
|
||||
},
|
||||
{
|
||||
"text": ", then reply with DONE.",
|
||||
"type": "text",
|
||||
},
|
||||
]
|
||||
`)
|
||||
} finally {
|
||||
await new Promise<void>(resolve => server.close(() => { resolve() }))
|
||||
}
|
||||
}, 45_000)
|
||||
|
||||
it('packed ACP fixture retains every chunk row kind without changing the logical session', () => {
|
||||
const source = fixtureRecords(PACKED_CHUNKS_SOURCE)
|
||||
const packed = fixtureRecords('packed-chunks')
|
||||
|
||||
@@ -0,0 +1,41 @@
|
||||
# Keyless assembled-request snapshot for pi-ai image offload. The local
|
||||
# provider endpoint is supplied by the snapshot test; the real attachment
|
||||
# store and ACP bridge carry two uploaded images into one model request.
|
||||
- id: base
|
||||
name: '@deepseek-ai/cordis-plugin-include'
|
||||
config:
|
||||
path: ../../cordis.yml
|
||||
patches:
|
||||
- id: llm-deepseek
|
||||
name: '@deepseek-ai/dsh-llm-deepseek'
|
||||
disabled: true
|
||||
- id: acp-agent
|
||||
name: '@deepseek-ai/dsh-acp-demo'
|
||||
config:
|
||||
provider: snapshot
|
||||
model: vision
|
||||
persistenceRoot: !!js process.env.DSH_SNAPSHOT_SESSIONS_ROOT ?? './.sessions'
|
||||
persistenceCompression: none
|
||||
workspaceContext:
|
||||
maxBytes: 65536
|
||||
persona: |
|
||||
You are a coding assistant powered by the {{model}} model. Your working directory is {{cwd}}.
|
||||
|
||||
Keep answers brief and factual.
|
||||
- insert:
|
||||
- id: attachment-local
|
||||
name: '@deepseek-ai/dsh-attachment-local'
|
||||
- id: llm-pi-ai
|
||||
name: '@deepseek-ai/dsh-llm-pi-ai'
|
||||
config:
|
||||
providers:
|
||||
snapshot:
|
||||
apiKeyEnv: DSH_SNAPSHOT_API_KEY
|
||||
api: openai-completions
|
||||
baseURL: !!js process.env.DSH_SNAPSHOT_BASE_URL
|
||||
maxRequestImageBytes: 92
|
||||
models:
|
||||
- id: vision
|
||||
contextWindow: 32768
|
||||
maxTokens: 1024
|
||||
input: [text, image]
|
||||
@@ -11,7 +11,7 @@ import { readImageFile, saveImageFile, validateImageFile } from './store.ts'
|
||||
export { readImageFile, saveImageFile, validateImageFile } from './store.ts'
|
||||
|
||||
/** Default maximum encoded bytes for one image. */
|
||||
export const DEFAULT_MAX_IMAGE_BYTES = 5 * 1024 * 1024
|
||||
export const DEFAULT_MAX_IMAGE_BYTES = 3.5 * 1024 * 1024
|
||||
/** Default maximum images in one prompt. */
|
||||
export const DEFAULT_MAX_IMAGES_PER_MESSAGE = 20
|
||||
/** Default maximum aggregate image bytes in one prompt. */
|
||||
|
||||
@@ -15,7 +15,7 @@ import LocalAttachmentStore, {
|
||||
describe('local attachment service', () => {
|
||||
it('resolves every omitted admission limit explicitly', () => {
|
||||
const service = new LocalAttachmentStore(new Context(), {})
|
||||
expect(DEFAULT_MAX_IMAGE_BYTES).toBe(5 * 1024 * 1024)
|
||||
expect(DEFAULT_MAX_IMAGE_BYTES).toBe(3.5 * 1024 * 1024)
|
||||
expect(service.imageLimits).toEqual({
|
||||
maxImageBytes: DEFAULT_MAX_IMAGE_BYTES,
|
||||
maxImagesPerMessage: DEFAULT_MAX_IMAGES_PER_MESSAGE,
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/llm/llm-pi-ai/README.md
|
||||
README.md: d775e72616822ce0deee063ac0f3fc453af1a126
|
||||
README.zh.md: 621d67d1c181c6d4c78ea0078f521acccce92653
|
||||
README.md: 5dbcb905451f72a700dd09b4052dcb2f88e858c9
|
||||
README.zh.md: 217244c7b4d7ecd5aa88427990feb58c96e5acaf
|
||||
|
||||
@@ -113,7 +113,7 @@ A model that carries reasoning metadata — from the installed catalog or from i
|
||||
|
||||
A model **without** that metadata — a hand-declared one whose entry declares no `reasoningEfforts`, and a catalog model pi-ai marks as non-reasoning — exposes no `reasoning` at all. pi-ai reports such a model as supporting the single level `off`, but `off` is translated to *omitting* the reasoning option, which is byte-for-byte the request that naming no effort already produces: selecting it could not disable anything, so a provider whose own default is to think would keep thinking with `off` shown as selected. Reporting the capability as unavailable leaves a surface offering the provider's default and nothing that misrepresents it. The profile `reasoning` value, including `off`, is the deployment default when configured; omitting it preserves the provider default. Per-request `GenerateOptions.reasoningEffort` takes precedence, and a level absent from the exact model capability fails the REQUEST with `UNSUPPORTED_REASONING_EFFORT` before network I/O instead of being clamped. Describing a model never fails that way: the models under one provider disagree about which levels they accept, so `resolveModel` reports a profile level the exact model cannot take as no default at all rather than throwing. A throw there would take the whole provider out of every model catalog built over it — one mis-set profile field hiding even the models that do support the level — so a bad configuration surfaces where it is acted on, not where it is described. pi-ai's common stream options represent `off` by omitting `reasoning`.
|
||||
|
||||
Supported profile fields are `apiKeyEnv`, `displayName`, `api`, `baseURL`, `models`, `modelOverrides`, `compat`, `defaultContextWindow`, `defaultMaxTokens`, `defaultInput`, `headers`, `reasoning`, `thinkingBudgets`, `cacheRetention`, `transport`, `timeoutMs`, `websocketConnectTimeoutMs`, `streamIdleTimeoutMs`, and `retryPolicy`. Each profile's optional retry policy is captured with that provider route; omission uses bounded normal defaults. The stream-idle interval is a positive finite Node timer delay, defaults to five minutes, and covers only an outstanding provider read, not consumer think time. Harness app attribution wins a conflicting configured header name.
|
||||
Supported profile fields are `apiKeyEnv`, `displayName`, `api`, `baseURL`, `models`, `modelOverrides`, `compat`, `defaultContextWindow`, `defaultMaxTokens`, `defaultInput`, `headers`, `reasoning`, `thinkingBudgets`, `cacheRetention`, `transport`, `timeoutMs`, `websocketConnectTimeoutMs`, `streamIdleTimeoutMs`, `maxRequestImageBytes`, and `retryPolicy`. Each profile's optional retry policy is captured with that provider route; omission uses bounded normal defaults. The stream-idle interval is a positive finite Node timer delay, defaults to five minutes, and covers only an outstanding provider read, not consumer think time. `maxRequestImageBytes` bounds one request's base64-encoded image payload (default 20MiB, a positive integer): every image in history is re-encoded into every request, so when the accumulated payload exceeds the bound, the oldest images are replaced by a fixed text placeholder until the request fits, keeping an image-heavy session serviceable instead of permanently rejected by a gateway request-size cap. The default leaves capacity for system prompts, history, tools, and JSON; deployments behind stricter gateways lower it per route. Harness app attribution wins a conflicting configured header name.
|
||||
|
||||
The adapter forces pi-ai's SDK `maxRetries` to zero so one `stream()` call makes one provider request. The removed profile fields `maxRetries` and `maxRetryDelayMs` fail load instead of silently multiplying or hiding the separately composed agent-level retry budget. Idle expiry aborts the SDK's stable request signal and surfaces `TIMEOUT`; an earlier caller abort remains `ABORTED`.
|
||||
|
||||
@@ -163,15 +163,15 @@ pi-ai installs several provider SDKs and lazy-loads the one selected by the cata
|
||||
|
||||
#### What the model sees
|
||||
|
||||
The selected catalog model receives `GenerateOptions.system`, history, tools, and sampling fields supported by pi-ai's common streaming API. This package adds no prompt prose. Provider-native replay metadata is restored only when the adapter validates it for the historical content.
|
||||
The selected catalog model receives `GenerateOptions.system`, history, tools, and sampling fields supported by pi-ai's common streaming API. This package adds no prompt prose, with one exception: when a request's accumulated base64 image payload exceeds the route's `maxRequestImageBytes`, each offloaded image (oldest first) is replaced by fixed text. The text tells the model to read the file again when a path is available or ask the user to attach the image again. Provider-native replay metadata is restored only when the adapter validates it for the historical content.
|
||||
|
||||
#### Token effect
|
||||
|
||||
Provider tokenization governs exact input. Conversion adds no model-visible text; replay metadata may let a native API reuse provider-side state.
|
||||
Provider tokenization governs exact input. Conversion adds no model-visible text beyond the image-offload placeholder, which replaces the offloaded image's visual tokens with a short fixed sentence; replay metadata may let a native API reuse provider-side state.
|
||||
|
||||
#### KV Cache effect
|
||||
|
||||
Conversion preserves logical request order without adding text, while the selected provider's serialization and replay state determine reuse. Changing adapter instance, provider, model, or any upstream request token may prevent reuse from the first difference.
|
||||
Conversion preserves logical request order without adding text, while the selected provider's serialization and replay state determine reuse. Changing adapter instance, provider, model, or any upstream request token may prevent reuse from the first difference. Crossing the image bound rewrites an early message (the newly offloaded image becomes placeholder text), so reuse ends at that message until the offloaded prefix stabilizes.
|
||||
|
||||
### Provider response
|
||||
|
||||
@@ -189,6 +189,7 @@ Recorded response content appends to the next request and does not invalidate it
|
||||
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
- **`maxRequestImageBytes` counts base64 image payload only** — text, tools, and JSON structure ride outside the bound, so it must sit below the gateway's request-body cap with headroom. Offload is decided at request conversion as a pure function of history and configuration and is not recorded as a session event; per-route capability metadata (image count, per-image size, total request size) driving admission and assembly together is deferred design work.
|
||||
- **A provider that authenticates through OAuth alone is not offered** — pi-ai resolves OAuth from a *stored* OAuth credential, and this adapter builds its `Models` collection with no credential store and runs no login flow, so every request on such a route fails `Provider is not configured` before it goes out. The configurable-provider directory withholds them; `openai-codex` is the only one the installed catalog ships. A route a settings document already names keeps its entry so a configuration surface can edit or delete it, and `apiKeyEnv` still authenticates it with that key — which for Codex is a token that expires with nothing here to refresh it.
|
||||
- **Provider-native discovery reads the process environment only** — a route naming no credential defers to the catalog provider's own resolution, which interrogates environment variables (`AZURE_OPENAI_API_KEY`, `AWS_PROFILE`, `AWS_ACCESS_KEY_ID`, and each provider's own set). It reads no local credential directory, so `~/.aws/credentials` without an exported `AWS_PROFILE` resolves as unconfigured, and a value held by the harness credential seam is invisible to it unless the process environment carries it too.
|
||||
- **Settings can add or override routes, not remove composition routes** — the user layer merges over the composition `base`, so deleting a `cordis.yml`-provided provider is a composition change; `replace` on the namespace only resets the user layer.
|
||||
|
||||
@@ -114,7 +114,7 @@ profile 的 `models` 列表是*替换*该路由已安装 catalog，而不是扩
|
||||
|
||||
**没有**这份元数据的模型——条目未声明 `reasoningEfforts` 的手工声明模型，以及 pi-ai 标记为不具备推理能力的 catalog 模型——完全不公开 `reasoning`。pi-ai 会把这类模型报告为只支持 `off` 一档，但 `off` 会被翻译成*省略* reasoning 选项，而那与「不点名任何档位」产出的请求逐字节相同：选它关不掉任何东西，于是自身默认就在思考的提供方，会在界面显示 `off` 被选中的同时继续思考。把该能力报告为不可用，界面就只剩提供方默认这一项，不会再出现自相矛盾的控件。配置 profile 的 `reasoning` 值（包括 `off`）在存在时是部署默认值；省略它会保留提供方默认值。每次请求的 `GenerateOptions.reasoningEffort` 优先；未出现在确切模型能力中的档位会让**请求**在网络 I/O 前以 `UNSUPPORTED_REASONING_EFFORT` 失败，而不会被自动调整。**描述**一个模型则从不这样失败：同一提供方下各模型接受的档位并不一致，因此 `resolveModel` 对该模型拿不下的 profile 档位报告为「没有默认值」，而不是抛错。在那里抛错会让整个提供方从任何基于它构建的模型目录中消失——一个配错的 profile 字段连支持该档位的模型也一并藏起来——所以坏配置暴露在被执行处，而不是被描述处。pi-ai 的通用流选项通过省略 `reasoning` 表示 `off`。
|
||||
|
||||
受支持的 profile 字段是 `apiKeyEnv`、`displayName`、`api`、`baseURL`、`models`、`modelOverrides`、`compat`、`defaultContextWindow`、`defaultMaxTokens`、`defaultInput`、`headers`、`reasoning`、`thinkingBudgets`、`cacheRetention`、`transport`、`timeoutMs`、`websocketConnectTimeoutMs`、`streamIdleTimeoutMs` 和 `retryPolicy`。每个 profile 的可选重试策略都会与该提供方路由一同捕获；省略时使用有界的常规默认值。流空闲间隔必须是正的有限 Node 定时器延迟，默认为五分钟，且只覆盖未完成提供方读取，不包括消费方思考时间。若已配置标头中有同名项，则以 Harness 应用归因为准。
|
||||
受支持的 profile 字段是 `apiKeyEnv`、`displayName`、`api`、`baseURL`、`models`、`modelOverrides`、`compat`、`defaultContextWindow`、`defaultMaxTokens`、`defaultInput`、`headers`、`reasoning`、`thinkingBudgets`、`cacheRetention`、`transport`、`timeoutMs`、`websocketConnectTimeoutMs`、`streamIdleTimeoutMs`、`maxRequestImageBytes` 和 `retryPolicy`。每个 profile 的可选重试策略都会与该提供方路由一同捕获；省略时使用有界的常规默认值。流空闲间隔必须是正的有限 Node 定时器延迟，默认为五分钟，且只覆盖未完成提供方读取，不包括消费方思考时间。`maxRequestImageBytes` 约束单个请求的 base64 编码图片载荷（默认 20MiB，正整数）：历史中的每张图片都会重新编码进每个请求，累积载荷超过上限时，从最老的图片开始替换为固定文本占位，直到请求装得下，使图片较多的会话保持可用，而不是被网关请求体上限永久拒绝。默认值为系统提示词、历史、工具与 JSON 保留请求容量；网关更严格的部署按路由调低该值。若已配置标头中有同名项，则以 Harness 应用归因为准。
|
||||
|
||||
适配器强制 pi-ai SDK `maxRetries` 为零，因此一次 `stream()` 调用只会发起一次提供方请求。已移除 profile 字段 `maxRetries` 和 `maxRetryDelayMs` 会使加载失败，而不是静默倍增或隐藏单独组合的 agent（智能体）级重试预算。空闲超时会 abort SDK 的稳定请求信号，并以 `TIMEOUT` 呈现；较早的调用方 abort 仍为 `ABORTED`。
|
||||
|
||||
@@ -164,15 +164,15 @@ pi-ai 会安装多个提供方 SDK，并延迟加载 catalog 模型所选的 SDK
|
||||
|
||||
#### 模型看到的内容
|
||||
|
||||
所选 catalog 模型会收到 `GenerateOptions.system`、历史、工具，以及 pi-ai 通用流式 API 支持的采样字段。本包不添加提示词文本。只有当适配器验证提供方原生回放元数据与历史内容匹配时，才会恢复这些元数据。
|
||||
所选 catalog 模型会收到 `GenerateOptions.system`、历史、工具，以及 pi-ai 通用流式 API 支持的采样字段。本包不添加提示词文本，仅有一个例外：请求累积的 base64 图片载荷超过路由的 `maxRequestImageBytes` 时，被 offload 的图片（从最老开始）会被替换为一段固定文本。该文本要求模型在有路径时重新读取文件，否则请用户重新附上图片。只有当适配器验证提供方原生回放元数据与历史内容匹配时，才会恢复这些元数据。
|
||||
|
||||
#### Token 影响
|
||||
|
||||
精确输入取决于提供方 tokenization。转换不添加模型可见文本；回放元数据可能让原生 API 复用提供方侧状态。
|
||||
精确输入取决于提供方 tokenization。除图片 offload 占位文本外，转换不添加模型可见文本；占位文本用一句固定短句替代被省略图片的视觉 token。回放元数据可能让原生 API 复用提供方侧状态。
|
||||
|
||||
#### KV Cache 影响
|
||||
|
||||
转换保留逻辑请求顺序，不添加文本；复用取决于所选提供方的序列化与回放状态。更改适配器实例、提供方、模型或任何上游请求 token，都可能使复用从首个出现差异的 token 起失效。
|
||||
转换保留逻辑请求顺序，不添加文本；复用取决于所选提供方的序列化与回放状态。更改适配器实例、提供方、模型或任何上游请求 token，都可能使复用从首个出现差异的 token 起失效。跨过图片上限会改写较早的一条消息（新被 offload 的图片变为占位文本），复用在该消息处截止，直到被 offload 的前缀稳定。
|
||||
|
||||
### 提供方响应
|
||||
|
||||
@@ -190,6 +190,7 @@ pi-ai 事件会变为 harness 推理、文本、工具调用、usage 与 finish
|
||||
|
||||
## 已知限制与暂缓事项
|
||||
|
||||
- **`maxRequestImageBytes` 只统计 base64 图片载荷**：文本、工具与 JSON 结构不计入上限，因此该值必须低于网关请求体上限并留出余量。offload 在请求转换时决定，是历史与配置的纯函数，不记录为会话事件；由按路由能力元数据（图片数量、单图大小、请求总大小）同时驱动准入与组装的完整设计属于暂缓工作。
|
||||
- **仅以 OAuth 认证的提供方不予提供**：pi-ai 的 OAuth 只从*已存储*的 OAuth 凭据解析，而本适配器构造 `Models` 集合时不注入凭据存储、也不运行登录流程，因此这类路由的每个请求都会在发出之前以 `Provider is not configured` 失败。可配置提供方目录因此不列出它们；已安装 catalog 中只有 `openai-codex` 属于此类。settings 文档已经写过的路由仍保留目录条目，配置界面据此可以编辑或删除；`apiKeyEnv` 也仍能用该密钥完成认证——对 Codex 而言那是一个会过期、且这里没有任何环节会去刷新的 token。
|
||||
- **提供方自带的凭据发现只读进程环境**：不指定凭据的路由交由 catalog 提供方自行解析，而它探测的是环境变量（`AZURE_OPENAI_API_KEY`、`AWS_PROFILE`、`AWS_ACCESS_KEY_ID` 以及各提供方自己的那一组）。它不读任何本地凭据目录，因此只有 `~/.aws/credentials` 而未导出 `AWS_PROFILE` 会被解析为未配置；由 harness 凭据 seam 保管的值，除非进程环境里也有，否则对它不可见。
|
||||
- **settings 能新增或覆盖路由，但不能移除组合路由**：用户层合并在组合 `base` 之上，因此删除 `cordis.yml` 提供的提供方属于组合变更；对该 namespace 执行 `replace` 只会重置用户层。
|
||||
|
||||
@@ -317,7 +317,7 @@ export class PiAiAdapter extends LlmAdapter {
|
||||
}
|
||||
const context = attachments === undefined
|
||||
? toPiContext(options, undefined, onReplayDegrade)
|
||||
: await toPiContext(options, attachments, onReplayDegrade)
|
||||
: await toPiContext(options, attachments, onReplayDegrade, profile.maxRequestImageBytes)
|
||||
const events = snapshot.models.streamSimple(model, context, {
|
||||
...profileOptions(profile, reasoning, apiKey),
|
||||
...options.temperature === undefined ? {} : { temperature: options.temperature },
|
||||
|
||||
@@ -34,6 +34,17 @@ import { buildProvider, supportedProtocols } from './provider.ts'
|
||||
/** Default maximum idle interval while an adapter stream read is outstanding. */
|
||||
export const DEFAULT_STREAM_IDLE_TIMEOUT_MS = 300_000
|
||||
|
||||
/**
|
||||
* Default request-level bound on base64-encoded image payload. Every image in
|
||||
* history is re-encoded into every request body, so an unbounded conversation
|
||||
* eventually exceeds a provider or gateway request-size cap and the session
|
||||
* can never complete another request. The 20MiB default admits four images at
|
||||
* the attachment store's 3.5MiB raw-image default after base64 expansion and
|
||||
* reserves request capacity for system prompts, history, tools, and JSON.
|
||||
* Deployments behind stricter gateways lower it per route.
|
||||
*/
|
||||
export const DEFAULT_MAX_REQUEST_IMAGE_BYTES = 20 * 1024 * 1024
|
||||
|
||||
/** Context capacity assumed for a model neither configuration nor the catalog sizes. */
|
||||
export const DEFAULT_CONTEXT_WINDOW = 262_144
|
||||
|
||||
@@ -136,6 +147,13 @@ export interface PiAiProviderProfile {
|
||||
websocketConnectTimeoutMs?: number
|
||||
/** Maximum provider idle time while one stream read is outstanding. */
|
||||
streamIdleTimeoutMs?: number
|
||||
/**
|
||||
* Maximum base64-encoded image payload per request. When a request's
|
||||
* accumulated images exceed it, the oldest images are replaced by text
|
||||
* placeholders until the request fits, so a long session keeps completing
|
||||
* requests instead of being rejected by a request-size cap.
|
||||
*/
|
||||
maxRequestImageBytes?: number
|
||||
/** Provider-owned model-request retry policy; omission uses normal defaults. */
|
||||
retryPolicy?: RetryPolicyConfig
|
||||
}
|
||||
@@ -151,6 +169,8 @@ export interface ResolvedPiAiProviderProfile
|
||||
apiKeyEnv?: CredentialRef
|
||||
/** Positive finite provider-idle interval after defaulting. */
|
||||
streamIdleTimeoutMs: number
|
||||
/** Positive request-level base64 image payload bound after defaulting. */
|
||||
maxRequestImageBytes: number
|
||||
/** Immutable retry policy captured with this provider route. */
|
||||
retryPolicy: ResolvedRetryPolicy
|
||||
/**
|
||||
@@ -248,6 +268,7 @@ const profile = z.object({
|
||||
timeoutMs: z.natural(),
|
||||
websocketConnectTimeoutMs: z.natural(),
|
||||
streamIdleTimeoutMs: z.number().min(Number.MIN_VALUE).max(MAX_TIMER_DELAY_MS).default(DEFAULT_STREAM_IDLE_TIMEOUT_MS),
|
||||
maxRequestImageBytes: z.number().step(1).min(1).default(DEFAULT_MAX_REQUEST_IMAGE_BYTES),
|
||||
retryPolicy: RetryPolicySchema,
|
||||
})
|
||||
|
||||
@@ -323,6 +344,10 @@ export function resolveProfiles(
|
||||
`llm-pi-ai: provider "${provider}" streamIdleTimeoutMs must be a positive finite number no greater than ${MAX_TIMER_DELAY_MS}`,
|
||||
)
|
||||
}
|
||||
const maxRequestImageBytes = source.maxRequestImageBytes ?? DEFAULT_MAX_REQUEST_IMAGE_BYTES
|
||||
if (!Number.isInteger(maxRequestImageBytes) || maxRequestImageBytes <= 0) {
|
||||
throw new Error(`llm-pi-ai: provider "${provider}" maxRequestImageBytes must be a positive integer`)
|
||||
}
|
||||
// Detached from the configuration object because pi-ai types `Model.input`
|
||||
// mutable. The schema's explicit default covers an absent key, so an empty
|
||||
// list here is always one someone typed — and unlike an entry's, nothing
|
||||
@@ -354,6 +379,7 @@ export function resolveProfiles(
|
||||
displayName,
|
||||
...apiKeyEnv === undefined ? {} : { apiKeyEnv: credentialRef(apiKeyEnv) },
|
||||
streamIdleTimeoutMs,
|
||||
maxRequestImageBytes,
|
||||
retryPolicy: resolveRetryPolicy(retryPolicy, `llm-pi-ai: provider "${provider}" retryPolicy`),
|
||||
...rest.headers === undefined ? {} : { headers: { ...rest.headers } },
|
||||
...rest.thinkingBudgets === undefined ? {} : { thinkingBudgets: { ...rest.thinkingBudgets } },
|
||||
|
||||
@@ -26,17 +26,82 @@ function toolResultText(blocks: readonly ContentBlock[]): string {
|
||||
: block.type === 'tool-result' ? toolResultText(block.content) : '').join('')
|
||||
}
|
||||
|
||||
/** Model-facing stand-in for an image dropped to fit the request bound. */
|
||||
export const OFFLOADED_IMAGE_TEXT
|
||||
= '[image omitted to keep the request within its image limit; older images are omitted first. If this image is still needed, read its file again when a path is available; otherwise ask the user to attach it again.]'
|
||||
|
||||
/** Base64 length of `bytes` raw bytes (4 output characters per 3 input bytes, padded). */
|
||||
function base64Length(bytes: number): number {
|
||||
return Math.ceil(bytes / 3) * 4
|
||||
}
|
||||
|
||||
/**
|
||||
* Select the images a request must drop to fit the per-request payload bound.
|
||||
* History order is oldest-first, so the most recent images are omitted last.
|
||||
* A single image larger than the bound is itself omitted. Locations use
|
||||
* message and nested block indexes so JSON replay cannot change the result by
|
||||
* splitting or preserving shared object identities.
|
||||
* @param messages - complete request history, oldest first.
|
||||
* @param maxRequestImageBytes - bound on total base64-encoded image payload; undefined leaves every image in place.
|
||||
* @returns the image locations the conversion replaces with {@link OFFLOADED_IMAGE_TEXT}.
|
||||
*/
|
||||
function offloadedImages(
|
||||
messages: readonly Message[],
|
||||
maxRequestImageBytes: number | undefined,
|
||||
): ReadonlySet<string> {
|
||||
const offloaded = new Set<string>()
|
||||
if (maxRequestImageBytes === undefined) return offloaded
|
||||
const images: { location: string; base64Bytes: number }[] = []
|
||||
const collect = (messageIndex: number, blocks: readonly ContentBlock[], prefix: readonly number[] = []): void => {
|
||||
for (const [blockIndex, block] of blocks.entries()) {
|
||||
const path = [...prefix, blockIndex]
|
||||
if (block.type === 'image') {
|
||||
images.push({
|
||||
location: `${messageIndex}:${path.join('.')}`,
|
||||
base64Bytes: base64Length(block.attachment.bytes),
|
||||
})
|
||||
} else if (block.type === 'tool-result') {
|
||||
collect(messageIndex, block.content, path)
|
||||
}
|
||||
}
|
||||
}
|
||||
for (const [messageIndex, message] of messages.entries()) collect(messageIndex, message.content)
|
||||
let total = images.reduce((sum, image) => sum + image.base64Bytes, 0)
|
||||
for (const image of images) {
|
||||
if (total <= maxRequestImageBytes) break
|
||||
offloaded.add(image.location)
|
||||
total -= image.base64Bytes
|
||||
}
|
||||
return offloaded
|
||||
}
|
||||
|
||||
interface LocatedContentBlock {
|
||||
readonly block: ContentBlock
|
||||
readonly path: readonly number[]
|
||||
}
|
||||
|
||||
/** Attach stable nested indexes to blocks from one message. */
|
||||
function locatedBlocks(blocks: readonly ContentBlock[], prefix: readonly number[] = []): LocatedContentBlock[] {
|
||||
return blocks.map((block, index) => ({ block, path: [...prefix, index] }))
|
||||
}
|
||||
|
||||
async function userContent(
|
||||
blocks: readonly ContentBlock[],
|
||||
blocks: readonly LocatedContentBlock[],
|
||||
attachments: AttachmentStore,
|
||||
offloaded: ReadonlySet<string>,
|
||||
messageIndex: number,
|
||||
): Promise<string | (TextContent | ImageContent)[]> {
|
||||
const content: (TextContent | ImageContent)[] = []
|
||||
for (const block of blocks) {
|
||||
for (const { block, path } of blocks) {
|
||||
switch (block.type) {
|
||||
case 'text':
|
||||
if (block.text.length > 0) content.push({ type: 'text', text: block.text })
|
||||
break
|
||||
case 'image': {
|
||||
if (offloaded.has(`${messageIndex}:${path.join('.')}`)) {
|
||||
content.push({ type: 'text', text: OFFLOADED_IMAGE_TEXT })
|
||||
break
|
||||
}
|
||||
const stored = await attachments.readImage(block.attachment)
|
||||
content.push({
|
||||
type: 'image',
|
||||
@@ -47,7 +112,7 @@ async function userContent(
|
||||
}
|
||||
case 'tool-result':
|
||||
{
|
||||
const nested = await userContent(block.content, attachments)
|
||||
const nested = await userContent(locatedBlocks(block.content, path), attachments, offloaded, messageIndex)
|
||||
if (typeof nested === 'string') {
|
||||
if (nested.length > 0) content.push({ type: 'text', text: nested })
|
||||
} else {
|
||||
@@ -136,36 +201,44 @@ export function toPiContext(
|
||||
): PiContext
|
||||
/**
|
||||
* Convert harness history to a pi-ai Context while resolving durable images.
|
||||
* Tool result names are recovered from preceding assistant tool calls.
|
||||
* Tool result names are recovered from preceding assistant tool calls. When
|
||||
* the accumulated base64 image payload exceeds `maxRequestImageBytes`, the
|
||||
* oldest images are replaced by text placeholders until the request fits, so
|
||||
* an image-heavy session keeps clearing gateway request-size caps.
|
||||
* @param options - the harness request; `options.system` maps to pi-ai's single `systemPrompt` slot.
|
||||
* @param attachments - durable byte resolver for image references.
|
||||
* @param onReplayDegrade - forwarded to {@link toPiAssistant} for each assistant message.
|
||||
* @param maxRequestImageBytes - request-level bound on base64-encoded image payload; omission leaves every image in place.
|
||||
* @returns the asynchronously resolved pi-ai context.
|
||||
*/
|
||||
export function toPiContext(
|
||||
options: GenerateOptions,
|
||||
attachments: AttachmentStore,
|
||||
onReplayDegrade?: (reason: string) => void,
|
||||
maxRequestImageBytes?: number,
|
||||
): Promise<PiContext>
|
||||
export function toPiContext(
|
||||
options: GenerateOptions,
|
||||
attachments?: AttachmentStore,
|
||||
onReplayDegrade?: (reason: string) => void,
|
||||
maxRequestImageBytes?: number,
|
||||
): PiContext | Promise<PiContext> {
|
||||
return attachments === undefined
|
||||
? textOnlyContext(options, onReplayDegrade)
|
||||
: toPiContextWithImages(options, attachments, onReplayDegrade)
|
||||
: toPiContextWithImages(options, attachments, onReplayDegrade, maxRequestImageBytes)
|
||||
}
|
||||
|
||||
async function toPiContextWithImages(
|
||||
options: GenerateOptions,
|
||||
attachments: AttachmentStore,
|
||||
onReplayDegrade?: (reason: string) => void,
|
||||
maxRequestImageBytes?: number,
|
||||
): Promise<PiContext> {
|
||||
const offloaded = offloadedImages(options.messages, maxRequestImageBytes)
|
||||
const toolNames = new Map<CallId, string>()
|
||||
const messages: PiMessage[] = []
|
||||
|
||||
for (const message of options.messages) {
|
||||
for (const [messageIndex, message] of options.messages.entries()) {
|
||||
if (message.role === 'system') {
|
||||
if (contentHasImage(message.content)) {
|
||||
throw new LlmError('pi-ai cannot represent an image in an in-history system message', 'UNSUPPORTED_CONTENT')
|
||||
@@ -185,14 +258,17 @@ async function toPiContextWithImages(
|
||||
continue
|
||||
}
|
||||
// user role: text + tool results (each result becomes its own message).
|
||||
const regular = message.content.filter(block => block.type !== 'tool-result')
|
||||
const content = await userContent(regular, attachments)
|
||||
const results = message.content.filter(block => block.type === 'tool-result')
|
||||
const located = locatedBlocks(message.content)
|
||||
const regular = located.filter(({ block }) => block.type !== 'tool-result')
|
||||
const content = await userContent(regular, attachments, offloaded, messageIndex)
|
||||
const results = located.filter((entry): entry is LocatedContentBlock & { block: Extract<ContentBlock, { type: 'tool-result' }> } => (
|
||||
entry.block.type === 'tool-result'
|
||||
))
|
||||
if (content.length > 0 || results.length === 0) {
|
||||
messages.push({ role: 'user', content, timestamp: 0 })
|
||||
}
|
||||
for (const result of results) {
|
||||
const resultContent = await userContent(result.content, attachments)
|
||||
for (const { block: result, path } of results) {
|
||||
const resultContent = await userContent(locatedBlocks(result.content, path), attachments, offloaded, messageIndex)
|
||||
messages.push({
|
||||
role: 'toolResult',
|
||||
toolCallId: result.toolCallId,
|
||||
|
||||
@@ -40,6 +40,9 @@ function classifyPiAiError(message: string): string {
|
||||
if (/\b(?:401|403)\b/.test(message)) return 'AUTH'
|
||||
if (isQuotaExceededError(message)) return QUOTA_EXCEEDED_CODE
|
||||
if (/\b429\b|rate.?limit/i.test(message)) return 'RATE_LIMIT'
|
||||
// A rejected request body (gateway or provider size cap): resending the
|
||||
// same request cannot succeed, so it is invalid, not transient.
|
||||
if (/\b413\b|failed to buffer the request body:\s*length limit exceeded|payload too large|request body too large/i.test(message)) return 'INVALID_REQUEST'
|
||||
if (/\b400\b|invalid.?request/i.test(message)) return 'INVALID_REQUEST'
|
||||
if (/\b5\d\d\b/.test(message)) return 'SERVER'
|
||||
if (/\btime(?:d)?\s*out\b|timeout/i.test(message)) return 'TIMEOUT'
|
||||
|
||||
@@ -12,7 +12,7 @@ import * as LlmPiAi from '@deepseek-ai/dsh-llm-pi-ai'
|
||||
import { PiAiAdapter } from '@deepseek-ai/dsh-llm-pi-ai'
|
||||
import { MAX_TIMER_DELAY_MS } from '@deepseek-ai/dsh-timeout'
|
||||
import { getBuiltinModels } from '@earendil-works/pi-ai/providers/all'
|
||||
import { resolveProfiles } from '../src/config.ts'
|
||||
import { DEFAULT_MAX_REQUEST_IMAGE_BYTES, resolveProfiles } from '../src/config.ts'
|
||||
import { assemble } from './assemble.ts'
|
||||
import { closeMockServers, mockServer, textEvents } from './mock-server.ts'
|
||||
|
||||
@@ -697,6 +697,7 @@ describe('provider profile lifecycle', () => {
|
||||
})
|
||||
|
||||
it('validates empty, underspecified, legacy-shaped, and explicitly blank profiles', () => {
|
||||
expect(DEFAULT_MAX_REQUEST_IMAGE_BYTES).toBe(20 * 1024 * 1024)
|
||||
// Empty and omitted dicts are the dormant zero-route posture, not errors.
|
||||
expect(resolveProfiles({}).size).toBe(0)
|
||||
expect(resolveProfiles(undefined).size).toBe(0)
|
||||
@@ -710,6 +711,11 @@ describe('provider profile lifecycle', () => {
|
||||
expect(() => resolveProfiles({ openai: { provider: 'openai' } as never })).toThrow(/moved to the providers dict key/)
|
||||
expect(() => resolveProfiles({ openai: { baseURL: '' } })).toThrow(/empty baseURL/)
|
||||
expect(() => resolveProfiles({ openai: { apiKeyEnv: 'not-a-var!' } })).toThrow(/must match/)
|
||||
expect(() => resolveProfiles({ openai: { maxRequestImageBytes: 0 } })).toThrow(/maxRequestImageBytes/)
|
||||
expect(resolveProfiles({ openai: {} }).get('openai')?.maxRequestImageBytes)
|
||||
.toBe(DEFAULT_MAX_REQUEST_IMAGE_BYTES)
|
||||
expect(resolveProfiles({ openai: { maxRequestImageBytes: 1024 } }).get('openai')?.maxRequestImageBytes)
|
||||
.toBe(1024)
|
||||
})
|
||||
|
||||
it.each(['maxRetries', 'maxRetryDelayMs'] as const)(
|
||||
@@ -731,6 +737,9 @@ describe('provider profile lifecycle', () => {
|
||||
{ streamIdleTimeoutMs: 0 },
|
||||
{ streamIdleTimeoutMs: Number.NaN },
|
||||
{ streamIdleTimeoutMs: MAX_TIMER_DELAY_MS + 1 },
|
||||
{ maxRequestImageBytes: 0 },
|
||||
{ maxRequestImageBytes: 1.5 },
|
||||
{ maxRequestImageBytes: Number.NaN },
|
||||
]
|
||||
for (const entry of invalid) {
|
||||
const ctx = new Context()
|
||||
|
||||
@@ -3,7 +3,7 @@ import { AttachmentId } from '@deepseek-ai/dsh-attachment'
|
||||
import type { AttachmentStore, ImageAttachmentRef } from '@deepseek-ai/dsh-attachment'
|
||||
import { CallId, createMessage, createUserMessage } from '@deepseek-ai/dsh-llm'
|
||||
import type { ContentBlock, GenerateOptions, Message } from '@deepseek-ai/dsh-llm'
|
||||
import { toPiContext } from '../src/context.ts'
|
||||
import { OFFLOADED_IMAGE_TEXT, toPiContext } from '../src/context.ts'
|
||||
import { toPiAssistant } from '../src/replay.ts'
|
||||
|
||||
const ref: ImageAttachmentRef = {
|
||||
@@ -140,6 +140,92 @@ describe('pi-ai request context conversion', () => {
|
||||
])
|
||||
})
|
||||
|
||||
it('replaces the oldest images with placeholders once the request payload bound is exceeded', async () => {
|
||||
const readImage = vi.fn(() => Promise.resolve({ ref: { ...ref, bytes: 3 }, data: Uint8Array.of(1, 2, 3) }))
|
||||
const store = { readImage } as unknown as AttachmentStore
|
||||
const sized: ImageAttachmentRef = { ...ref, bytes: 3 }
|
||||
const callId = CallId('shot-call')
|
||||
// Three 3-byte images cost 4 base64 characters each (12 total); a bound of
|
||||
// 8 forces exactly the oldest one out, including one nested in a tool result.
|
||||
const context = await toPiContext(request([
|
||||
user([{
|
||||
type: 'tool-result',
|
||||
toolCallId: callId,
|
||||
content: [{ type: 'image', attachment: sized }],
|
||||
}]),
|
||||
user([{ type: 'image', attachment: sized }, { type: 'text', text: 'newer' }]),
|
||||
user([{ type: 'image', attachment: sized }]),
|
||||
]), store, undefined, 8)
|
||||
|
||||
expect(context.messages).toEqual([
|
||||
{
|
||||
role: 'toolResult',
|
||||
toolCallId: 'shot-call',
|
||||
toolName: 'unknown',
|
||||
content: [{ type: 'text', text: OFFLOADED_IMAGE_TEXT }],
|
||||
isError: false,
|
||||
timestamp: 0,
|
||||
},
|
||||
{
|
||||
role: 'user',
|
||||
content: [
|
||||
{ type: 'image', data: 'AQID', mimeType: 'image/png' },
|
||||
{ type: 'text', text: 'newer' },
|
||||
],
|
||||
timestamp: 0,
|
||||
},
|
||||
{ role: 'user', content: [{ type: 'image', data: 'AQID', mimeType: 'image/png' }], timestamp: 0 },
|
||||
])
|
||||
expect(readImage).toHaveBeenCalledTimes(2)
|
||||
})
|
||||
|
||||
it('keeps every image at exactly the payload bound and drops all of them when even the newest cannot fit', async () => {
|
||||
const sized: ImageAttachmentRef = { ...ref, bytes: 3 }
|
||||
const exact = await toPiContext(request([
|
||||
user([{ type: 'image', attachment: sized }]),
|
||||
user([{ type: 'image', attachment: sized }]),
|
||||
]), attachments, undefined, 8)
|
||||
expect(exact.messages).toEqual([
|
||||
{ role: 'user', content: [expect.objectContaining({ type: 'image' })], timestamp: 0 },
|
||||
{ role: 'user', content: [expect.objectContaining({ type: 'image' })], timestamp: 0 },
|
||||
])
|
||||
|
||||
const readImage = vi.fn()
|
||||
const store = { readImage } as unknown as AttachmentStore
|
||||
const oversized = await toPiContext(request([
|
||||
user([{ type: 'image', attachment: { ...ref, bytes: 300 } }]),
|
||||
]), store, undefined, 8)
|
||||
// All-text content collapses to the string form; the placeholder still reaches the model.
|
||||
expect(oversized.messages).toEqual([
|
||||
{ role: 'user', content: OFFLOADED_IMAGE_TEXT, timestamp: 0 },
|
||||
])
|
||||
expect(readImage).not.toHaveBeenCalled()
|
||||
})
|
||||
|
||||
it('offloads repeated image-block occurrences by position rather than shared object identity', async () => {
|
||||
const sized: ImageAttachmentRef = { ...ref, bytes: 3 }
|
||||
const shared: ContentBlock = { type: 'image', attachment: sized }
|
||||
const readImage = vi.fn(() => Promise.resolve({ ref: sized, data: Uint8Array.of(1, 2, 3) }))
|
||||
const store = { readImage } as unknown as AttachmentStore
|
||||
const aliased = await toPiContext(request([user([shared, shared])]), store, undefined, 4)
|
||||
const replayed = await toPiContext(request([user([
|
||||
{ type: 'image', attachment: { ...sized } },
|
||||
{ type: 'image', attachment: { ...sized } },
|
||||
])]), store, undefined, 4)
|
||||
|
||||
const expected = [{
|
||||
role: 'user',
|
||||
content: [
|
||||
{ type: 'text', text: OFFLOADED_IMAGE_TEXT },
|
||||
{ type: 'image', data: 'AQID', mimeType: 'image/png' },
|
||||
],
|
||||
timestamp: 0,
|
||||
}]
|
||||
expect(aliased.messages).toEqual(expected)
|
||||
expect(replayed.messages).toEqual(expected)
|
||||
expect(readImage).toHaveBeenCalledTimes(2)
|
||||
})
|
||||
|
||||
it('keeps empty text-only users while separating result-only messages', () => {
|
||||
const callId = CallId('unknown-call')
|
||||
expect(toPiContext(request([
|
||||
|
||||
@@ -779,6 +779,16 @@ describe('mapStopReason / mapUsage', () => {
|
||||
stopReason: 'error',
|
||||
errorMessage: 'HTTP 400: invalid input: temperature exceeds maximum allowed value',
|
||||
}))).toMatchObject({ kind: 'error', failure: { code: 'INVALID_REQUEST' } })
|
||||
expect(mapStopReason(assistant({ stopReason: 'error', errorMessage: 'HTTP 413: Payload Too Large' })))
|
||||
.toMatchObject({ kind: 'error', failure: { code: 'INVALID_REQUEST' } })
|
||||
expect(mapStopReason(assistant({
|
||||
stopReason: 'error',
|
||||
errorMessage: 'Failed to buffer the request body: length limit exceeded',
|
||||
}))).toMatchObject({ kind: 'error', failure: { code: 'INVALID_REQUEST' } })
|
||||
expect(mapStopReason(assistant({
|
||||
stopReason: 'error',
|
||||
errorMessage: 'vector length limit exceeded',
|
||||
}))).toMatchObject({ kind: 'error', failure: { code: 'PI_AI_ERROR' } })
|
||||
})
|
||||
|
||||
it.each([
|
||||
|
||||
|
Before Width: | Height: | Size: 30 KiB After Width: | Height: | Size: 30 KiB |
@@ -210,7 +210,7 @@ for (const profile of providerCases) {
|
||||
if (profile.provider === 'anthropic') {
|
||||
it('sends a real image through the authenticated Anthropic visual path', async () => {
|
||||
const data = new Uint8Array(await readFile(
|
||||
new URL('../../../../assets/community-wecom-survey.png', import.meta.url),
|
||||
new URL('./fixtures/qr-code.png', import.meta.url),
|
||||
))
|
||||
const ref: ImageAttachmentRef = {
|
||||
attachmentId: AttachmentId(`sha256:${'a'.repeat(64)}`),
|
||||
|
||||
@@ -12,7 +12,7 @@
|
||||
},
|
||||
{
|
||||
"role": "assistant",
|
||||
"content": "# DeepSeek Harness\n\n[English](README.md) | 中文\n\nDeepSeek Harness（`dsh`）是由 [DeepSeek AI](https://deepseek.com) 开发的开源 agent harness（智能体框架）。\n\n它采用**一切皆插件**的架构，并由 [Cordis](https://github.com/cordiverse/cordis) 驱动，其设计参见论文 [_A Programming Paradigm for Spatiotemporal Composability_](https://github.com/cordiverse/paper)。\n\n## 开发者预览\n\nDeepSeek Harness 目前处于 _开发者预览_ 阶段，正在快速迭代。**未来将出现破坏兼容性的变更。**\n\n## 运行\n\n### 通过 `npm` 运行\n\n安装 `Node.js`，然后运行：\n\n```sh\nnpx @deepseek-ai/dsh web\n```\n\n该命令会启动 Web UI，默认地址为 `http://127.0.0.1:3080`。详见 [Web UI 指南](docs/user/guide/index.md)。\n\n### 从源码运行\n\n如需从仓库源码运行：\n\n```sh\ngit clone https://github.com/deepseek-ai/deepseek-harness.git\ncd deepseek-harness\npnpm install\npnpm run build\npnpm dsh web\n```\n\n## 社区与支持\n\n- 欢迎通过 [GitHub Discussions](https://github.com/deepseek-ai/deepseek-harness/discussions) 提交反馈或 bug 报告。\n- 为你的插件仓库添加 [`dsh-plugin`](https://github.com/topics/dsh-plugin) 话题，便于被发现。\n- 欢迎加入 DeepSeek Harness 企微群：扫码添加企微小助手并填写入群问卷，完成后小助手会邀请你入群。\n\n<table>\n <thead>\n <tr>\n <th align=\"center\">企微小助手</th>\n <th align=\"center\">入群问卷</th>\n <th align=\"center\">微信公众号</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <td align=\"center\"><img src=\"assets/community-wecom-assistant.png\" alt=\"DeepSeek Harness 企微小助手二维码\" width=\"180\" height=\"180\"></td>\n <td align=\"center\"><a href=\"https://trtgsjkv6r.feishu.cn/share/base/form/shrcnIt5twSVdLGD52KJBckGCgg\"><img src=\"assets/community-wecom-survey.png\" alt=\"DeepSeek Harness 入群问卷二维码\" width=\"180\" height=\"180\"></a></td>\n <td align=\"center\"><img src=\"assets/community-wechat-official-account.png\" alt=\"DeepSeek Harness 团队微信公众号二维码\" width=\"180\" height=\"180\"></td>\n </tr>\n </tbody>\n</table>\n\n## 参与贡献\n\n参见 [CONTRIBUTING.md](CONTRIBUTING.md)。\n\n## 开发\n\n请先阅读[开发指南](docs/development.md)与[架构文档](docs/architecture.md)。\n\n面向 agent：请遵循 [AGENTS.md](AGENTS.md)。\n\n## 许可证\n\n[MIT](LICENSE)\n\n第三方依赖及其许可证见 [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md)。\n"
|
||||
"content": "# DeepSeek Harness\n\n[English](README.md) | 中文\n\nDeepSeek Harness（`dsh`）是由 [DeepSeek AI](https://deepseek.com) 开发的开源 agent harness（智能体框架）。\n\n它采用**一切皆插件**的架构，并由 [Cordis](https://github.com/cordiverse/cordis) 驱动，其设计参见论文 [_A Programming Paradigm for Spatiotemporal Composability_](https://github.com/cordiverse/paper)。\n\n## 开发者预览\n\nDeepSeek Harness 目前处于 _开发者预览_ 阶段，正在快速迭代。**未来将出现破坏兼容性的变更。**\n\n## 运行\n\n### 通过 `npm` 运行\n\n安装 `Node.js`，然后运行：\n\n```sh\nnpx @deepseek-ai/dsh web\n```\n\n该命令会启动 Web UI，默认地址为 `http://127.0.0.1:3080`。详见 [Web UI 指南](docs/user/guide/index.md)。\n\n### 从源码运行\n\n如需从仓库源码运行：\n\n```sh\ngit clone https://github.com/deepseek-ai/deepseek-harness.git\ncd deepseek-harness\npnpm install\npnpm run build\npnpm dsh web\n```\n\n## 社区与支持\n\n- 欢迎通过 [GitHub Discussions](https://github.com/deepseek-ai/deepseek-harness/discussions) 提交反馈或 bug 报告。\n- 为你的插件仓库添加 [`dsh-plugin`](https://github.com/topics/dsh-plugin) 话题，便于被发现。\n- 欢迎加入 DeepSeek Harness 企微群：扫码添加企微小助手并填写入群问卷，完成后小助手会邀请你入群。\n\n<table>\n <thead>\n <tr>\n <th align=\"center\">企微小助手</th>\n <th align=\"center\">入群问卷</th>\n <th align=\"center\">微信公众号</th>\n </tr>\n </thead>\n <tbody>\n <tr>\n <td align=\"center\"><img src=\"https://cdn.deepseek.com/harness/readme/community-wecom-assistant.png\" alt=\"DeepSeek Harness 企微小助手二维码\" width=\"180\" height=\"180\"></td>\n <td align=\"center\"><a href=\"https://trtgsjkv6r.feishu.cn/share/base/form/shrcnIt5twSVdLGD52KJBckGCgg\"><img src=\"https://cdn.deepseek.com/harness/readme/community-wecom-survey.png\" alt=\"DeepSeek Harness 入群问卷二维码\" width=\"180\" height=\"180\"></a></td>\n <td align=\"center\"><img src=\"https://cdn.deepseek.com/harness/readme/community-wechat-official-account.png\" alt=\"DeepSeek Harness 团队微信公众号二维码\" width=\"180\" height=\"180\"></td>\n </tr>\n </tbody>\n</table>\n\n## 参与贡献\n\n参见 [CONTRIBUTING.md](CONTRIBUTING.md)。\n\n## 开发\n\n请先阅读[开发指南](docs/development.md)与[架构文档](docs/architecture.md)。\n\n面向 agent：请遵循 [AGENTS.md](AGENTS.md)。\n\n## 许可证\n\n[MIT](LICENSE)\n\n第三方依赖及其许可证见 [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md)。\n"
|
||||
},
|
||||
{
|
||||
"role": "user",
|
||||
|
||||
Reference in New Issue
Block a user