diff --git a/.agents/notes/implemented/feature/2026-07-24-provider-retry-policies.i18n.yaml b/.agents/notes/implemented/feature/2026-07-24-provider-retry-policies.i18n.yaml index c0694ef835..03486a909a 100644 --- a/.agents/notes/implemented/feature/2026-07-24-provider-retry-policies.i18n.yaml +++ b/.agents/notes/implemented/feature/2026-07-24-provider-retry-policies.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-07-24-provider-retry-policies.md -2026-07-24-provider-retry-policies.md: d85a6eb285e3c4dd47c5d0d3fa27fbbb51490449 -2026-07-24-provider-retry-policies.zh.md: e353624801252d4fc5e083852800bd6c58a7f01d +2026-07-24-provider-retry-policies.md: aa72e05f8ca74621d2ab37f9c2f3bea86cbcf7d9 +2026-07-24-provider-retry-policies.zh.md: a3b998196133ce5880bfacb6985d6c95e64bb650 diff --git a/.agents/notes/implemented/feature/2026-07-24-provider-retry-policies.md b/.agents/notes/implemented/feature/2026-07-24-provider-retry-policies.md index d85a6eb285..aa72e05f8c 100644 --- a/.agents/notes/implemented/feature/2026-07-24-provider-retry-policies.md +++ b/.agents/notes/implemented/feature/2026-07-24-provider-retry-policies.md @@ -12,7 +12,7 @@ Provider policy must follow the request that actually failed, including a route ## Decision -Each concrete adapter accepts an optional `retryPolicy` inside its provider configuration. The adapter validates and resolves the policy, and `ctx.llm` captures it when that exact provider route registers. A multi-provider adapter may also expose a composition default that it resolves into every route without an explicit policy; `llm-pi-ai` names this field `defaultRetryPolicy`, and a profile policy wins over it. This remains route-owned registration state rather than a global retry-executor setting. When a call enters its final adapter boundary, `ctx.llm` binds the serving registration's immutable policy to that call; the agent loop passes it to closed-step recovery even if the route is disposed or replaced while the request is in flight. `@deepseek-ai/dsh-llm-retry` combines that call-local policy with the failed step's durable provider identity. A call that never reaches a final adapter has no serving policy and delegates. A provider without either policy uses the normal defaults of two retries. The shipped Web bundle configures its `deepseek-official` route and the pi-ai adapter default for five normal-mode retries; this deployment choice tolerates longer transient gateway outages while remaining bounded. The `llm-deepseek` user-settings layer or a pi-ai profile policy can override its corresponding default. Other profiles and provider routes retain their own policies. +Each concrete adapter accepts an optional `retryPolicy` inside its provider configuration, validates an explicit value, and exposes that resolved route override through `providerRetryPolicy()`. `LlmRuntime` accepts one deployment-wide `defaultRetryPolicy`, resolves it at service construction, and applies it only when the adapter supplies no route override; omitting both preserves the normal defaults of two retries. The resulting effective policy remains route-owned registration state rather than a retry-executor setting. When a call enters its final adapter boundary, `ctx.llm` binds the serving registration's immutable policy to that call; the agent loop passes it to closed-step recovery even if the route is disposed or replaced while the request is in flight. `@deepseek-ai/dsh-llm-retry` combines that call-local policy with the failed step's durable provider identity. A call that never reaches a final adapter has no serving policy and delegates. The shipped Web bundle configures `LlmRuntime.defaultRetryPolicy` for five normal-mode retries; every current or future Web adapter route inherits this larger bounded budget unless its provider configuration overrides it. Other profiles retain the core default unless their composition chooses another value. ```yaml providers: @@ -44,12 +44,14 @@ Each scheduled retry appends a non-surface `llm/retry` event with the failed pro ## Alternatives considered -**One global `always` switch** — rejected because it cannot isolate the unbounded cost and latency risk to the provider that needs it and can silently apply after runtime rerouting. +**One retry-executor-level `always` switch** — rejected because it cannot isolate the unbounded cost and latency risk to the provider that needs it and can silently apply after runtime rerouting. The LLM deployment default is different: provider route overrides remain authoritative, and the effective policy is captured only after routing selects a registration. **A separate exact-provider list on `dsh-llm-retry`** — rejected because it duplicates provider route names outside their owning adapter configuration and lets provider registration drift from recovery policy. **A very large finite retry count** — rejected because it eventually violates the requested keep-retrying contract and serializes an arbitrary operational limit as if it were meaningful. +**Adapter-level shared defaults** — rejected because a deployment-wide budget would have to be repeated by every adapter family and every future adapter. Putting a discriminated policy object into an adapter's settings base also lets recursive settings layering retain normal-only keys when a user switches the policy to always mode. + **Raise the provider omission default to five** — rejected because it would also increase request cost and latency for headless and custom profiles. The Web bundle's provider configuration expresses the interactive deployment's larger bounded budget without changing other profiles. **Stamp five retries into profiles when the Web UI writes them** — rejected because existing profiles and settings written outside that UI would retain two retries. The deployment default belongs in the Web composition and is inherited uniformly. @@ -60,7 +62,7 @@ Each scheduled retry appends a non-surface `llm/retry` event with the failed pro ## Verification -Adapter tests validate nested policies at provider load, prove registration captures configured and default policies, and retain the serving policy across in-flight route replacement. The pi-ai tests prove its adapter default reaches omitted profiles, explicit profile policies win, invalid defaults fail even while dormant, and settings changes atomically re-register the route set while changing only inherited policy values. Unit tests select policies from the failed request's serving registration, separate provider and changed-policy histories, exercise always mode beyond the normal budget, pin jitter and delay caps, prove downstream recovery ordering, prove cancellation and disposal drain delegated recovery before reaching quiescence, and prove both abort active backoff waits. Request-level coverage compares the complete messages of failed and retried attempts and rejects both provider error text and discarded partial output. A keyless headless `stream-json` snapshot runs failure, retry, and success through the assembled app, pins the complete `llm/retry` record, and rejects any model-message change between attempts. The shipped Web composition snapshot pins both the resolved `deepseek-official` policy and a settings-added pi-ai route at five retries. JSONL and SQLite tests round-trip an always event without `Infinity`; invariant tests bind provider identity to the request header, validate failure and mode-specific timer bounds, and bind retry numbers to provider-policy keys; TUI tests render finite and infinite limits. +Adapter tests validate nested policies at provider load, prove explicit profile policies reach registration, and retain the serving policy across in-flight route replacement. LLM service tests prove adapter overrides win over the deployment default, omission retains the core two-retry behavior, and an invalid deployment policy fails before any route registers. Unit tests select policies from the failed request's serving registration, separate provider and changed-policy histories, exercise always mode beyond the normal budget, pin jitter and delay caps, prove downstream recovery ordering, prove cancellation and disposal drain delegated recovery before reaching quiescence, and prove both abort active backoff waits. Request-level coverage compares the complete messages of failed and retried attempts and rejects both provider error text and discarded partial output. A keyless headless `stream-json` snapshot runs failure, retry, and success through the assembled app, pins the complete `llm/retry` record, and rejects any model-message change between attempts. The shipped Web composition snapshot pins omitted DeepSeek and pi-ai policies at five retries, then proves settings can replace either with always mode without inheriting normal-only fields. JSONL and SQLite tests round-trip an always event without `Infinity`; invariant tests bind provider identity to the request header, validate failure and mode-specific timer bounds, and bind retry numbers to provider-policy keys; TUI tests render finite and infinite limits. ## Consequences diff --git a/.agents/notes/implemented/feature/2026-07-24-provider-retry-policies.zh.md b/.agents/notes/implemented/feature/2026-07-24-provider-retry-policies.zh.md index e353624801..a3b9981961 100644 --- a/.agents/notes/implemented/feature/2026-07-24-provider-retry-policies.zh.md +++ b/.agents/notes/implemented/feature/2026-07-24-provider-retry-policies.zh.md @@ -12,7 +12,7 @@ Status: implemented ## 决策 -每个具体适配器都在其提供方配置中接受可选的 `retryPolicy`。适配器负责校验并解析策略,`ctx.llm` 则在该特定提供方路由注册时捕获策略。多提供方适配器还可以公开一个组合默认值,并将它解析进每条未显式配置策略的路由;`llm-pi-ai` 将该字段命名为 `defaultRetryPolicy`,profile 策略优先于它。这仍然是路由拥有的注册状态,而不是全局重试执行器设置。当调用进入最终适配器边界时,`ctx.llm` 会把实际提供服务的注册项所持不可变策略绑定到该调用;即使路由在请求进行期间被 dispose(资源释放)或替换,agent loop(智能体循环)仍会把该策略传给已关闭步骤恢复。`@deepseek-ai/dsh-llm-retry` 会把绑定到该调用的策略与失败步骤的持久化提供方标识结合起来。未到达最终适配器的调用没有实际提供服务的策略,因而会委托后续处理。两层策略都未配置的提供方使用两次重试的 normal 默认值。随附的 Web bundle 为自己的 `deepseek-official` 路由和 pi-ai 适配器默认值配置五次 normal 模式重试;这项部署选择能容忍更长的短暂 gateway 故障,同时仍然有界。`llm-deepseek` 用户设置层或 pi-ai profile 策略可以覆盖各自对应的默认值。其他 profile 与提供方路由保留各自的策略。 +每个具体适配器都在其提供方配置中接受可选的 `retryPolicy`,校验显式值,并通过 `providerRetryPolicy()` 公开解析后的路由覆盖。`LlmRuntime` 接受一个部署级 `defaultRetryPolicy`,在服务构造时解析它,并且只在适配器未提供路由覆盖时应用;两者都省略时保留两次重试的 normal 默认值。最终得到的有效策略仍然是路由拥有的注册状态,而不是重试执行器设置。当调用进入最终适配器边界时,`ctx.llm` 会把实际提供服务的注册项所持不可变策略绑定到该调用;即使路由在请求进行期间被 dispose(资源释放)或替换,agent loop(智能体循环)仍会把该策略传给已关闭步骤恢复。`@deepseek-ai/dsh-llm-retry` 会把绑定到该调用的策略与失败步骤的持久化提供方标识结合起来。未到达最终适配器的调用没有实际提供服务的策略,因而会委托后续处理。随附的 Web bundle 为 `LlmRuntime.defaultRetryPolicy` 配置五次 normal 模式重试;当前及未来的每条 Web 适配器路由都会继承这项更大的有界预算,除非其提供方配置覆盖它。其他 profile 保留核心默认值,除非自身组合选择其他值。 ```yaml providers: @@ -44,12 +44,14 @@ always 模式先请求下游恢复,使上下文溢出压缩(compaction)之 ## 曾考虑的替代方案 -**单一全局 `always` 开关**:不予采纳,因为它无法把无界成本与延迟风险限制在确有需要的提供方,还可能在运行时重新路由后悄然生效。 +**重试执行器级的单一 `always` 开关**:不予采纳,因为它无法把无界成本与延迟风险限制在确有需要的提供方,还可能在运行时重新路由后悄然生效。LLM 部署默认值不同:提供方路由覆盖仍然权威,而且只有在路由选定注册后才捕获有效策略。 **在 `dsh-llm-retry` 上维护单独的指定提供方列表**:不予采纳,因为它会在所属适配器配置之外重复提供方路由名称,并让提供方注册与恢复策略发生偏差。 **设置很大的有限重试次数**:不予采纳,因为它最终仍会违反持续重试的约定,并把任意选取的运维上限序列化成看似有意义的数值。 +**适配器级共享默认值**:不予采纳,因为部署级预算必须在每种适配器族以及未来的每个适配器中重复配置。把可辨识策略对象放入适配器的 settings base,还会让 settings 递归分层在用户把策略切换为 always 模式时保留仅属于 normal 的字段。 + **将提供方省略配置时的默认值提高到五次**:不予采纳,因为这也会增加 headless 与自定义 profile 的请求成本和延迟。Web bundle 的提供方配置能表达互动式部署所需的更大有界预算,而不改变其他 profile。 **在 Web UI 写入 profile 时把五次重试写死进去**:不予采纳,因为现有 profile 与从该 UI 之外写入的 settings 仍会保留两次重试。部署默认值应位于 Web 组合中,并被统一继承。 @@ -60,7 +62,7 @@ always 模式先请求下游恢复,使上下文溢出压缩(compaction)之 ## 验证 -适配器测试会在提供方加载时校验嵌套策略,证明注册流程会捕获已配置策略和默认策略,并证明请求进行期间替换路由后仍会保留实际提供服务的策略。pi-ai 测试会证明适配器默认值抵达省略策略的 profile、显式 profile 策略优先、无效默认值即使在休眠状态也会失败,以及 settings 变更会原子地重新注册路由集合,同时只改变继承的策略值。单元测试根据失败请求实际使用的注册项选择策略、分离不同提供方和策略变更后的重试历史、验证 always 模式可越过 normal 预算、固定抖动和延迟上限、证明下游恢复顺序、证明取消与 dispose 会先排空已委托的恢复再达到完全停稳,并证明二者都会停止正在进行的退避等待。请求级覆盖会比较失败尝试与重试尝试的完整消息,并排除提供方错误文本和丢弃的部分输出。一个无密钥 headless `stream-json` 快照会通过组装后的应用执行失败、重试与成功流程,固定完整的 `llm/retry` 记录,并拒绝各次尝试之间出现任何模型消息变化。随附的 Web 组合快照将解析后的 `deepseek-official` 策略和一条由 settings 新增的 pi-ai 路由都固定为五次重试。JSONL 与 SQLite 测试会往返读写不含 `Infinity` 的 always 事件;不变式测试会将提供方标识绑定到请求头、校验失败事实和各模式的计时器边界,并将重试编号绑定到提供方策略键;TUI 测试会渲染有限和无限上限。 +适配器测试会在提供方加载时校验嵌套策略,证明显式 profile 策略抵达注册流程,并证明请求进行期间替换路由后仍会保留实际提供服务的策略。LLM 服务测试会证明适配器覆盖优先于部署默认值、省略配置时保留核心的两次重试行为,以及无效部署策略会在任何路由注册前失败。单元测试根据失败请求实际使用的注册项选择策略、分离不同提供方和策略变更后的重试历史、验证 always 模式可越过 normal 预算、固定抖动和延迟上限、证明下游恢复顺序、证明取消与 dispose 会先排空已委托的恢复再达到完全停稳,并证明二者都会停止正在进行的退避等待。请求级覆盖会比较失败尝试与重试尝试的完整消息,并排除提供方错误文本和丢弃的部分输出。一个无密钥 headless `stream-json` 快照会通过组装后的应用执行失败、重试与成功流程,固定完整的 `llm/retry` 记录,并拒绝各次尝试之间出现任何模型消息变化。随附的 Web 组合快照会把省略配置的 DeepSeek 与 pi-ai 策略固定为五次重试,再证明 settings 可以把任一策略替换为 always 模式,且不会继承仅属于 normal 的字段。JSONL 与 SQLite 测试会往返读写不含 `Infinity` 的 always 事件;不变式测试会将提供方标识绑定到请求头、校验失败事实和各模式的计时器边界,并将重试编号绑定到提供方策略键;TUI 测试会渲染有限和无限上限。 ## 后果 diff --git a/apps/web/tests/shipped-composition.e2e.ts b/apps/web/tests/shipped-composition.e2e.ts index 0e873f4be4..92618c75d8 100644 --- a/apps/web/tests/shipped-composition.e2e.ts +++ b/apps/web/tests/shipped-composition.e2e.ts @@ -93,8 +93,22 @@ it('assembles the shipped Web catalog, file-reference guidance, retry policy, an ], } `) + await ctx.settings.update(settingsNamespace('llm-deepseek'), { + retryPolicy: { mode: 'always' }, + }) + expect(ctx.llm.providerRetryPolicy('deepseek-official')).toMatchInlineSnapshot(` + { + "initialDelayMs": 500, + "jitterRatio": 0.1, + "maxDelayMs": 10000, + "mode": "always", + } + `) await ctx.settings.update(settingsNamespace('llm-pi-ai'), { - providers: { openai: {} }, + providers: { + openai: {}, + anthropic: { retryPolicy: { mode: 'always' } }, + }, }) expect(ctx.llm.providerRetryPolicy('openai')).toMatchInlineSnapshot(` { @@ -112,6 +126,14 @@ it('assembles the shipped Web catalog, file-reference guidance, retry policy, an ], } `) + expect(ctx.llm.providerRetryPolicy('anthropic')).toMatchInlineSnapshot(` + { + "initialDelayMs": 500, + "jitterRatio": 0.1, + "maxDelayMs": 10000, + "mode": "always", + } + `) // The catalog belongs to an AGENT, not to the process: every model-facing row // now lives in a preset mounted under one session's scope, so the global // layer holds nothing and a caller must name the agent to see anything. This diff --git a/docs/config-catalog.i18n.yaml b/docs/config-catalog.i18n.yaml index 0f79950a23..6578853f9a 100644 --- a/docs/config-catalog.i18n.yaml +++ b/docs/config-catalog.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write docs/config-catalog.md -config-catalog.md: b3dc82000c5b11be0dccf6e0132911a283547513 -config-catalog.zh.md: 4165d92956fdbfe6b6de6ec98289ceee5362d2ca +config-catalog.md: 9a330b26cc57302be1b782df7f1d1f29879681b5 +config-catalog.zh.md: 453d9dcc6347c1b984b57f92c5bbeaef7b002b83 diff --git a/docs/config-catalog.md b/docs/config-catalog.md index b3dc82000c..9a330b26cc 100644 --- a/docs/config-catalog.md +++ b/docs/config-catalog.md @@ -836,6 +836,53 @@ export interface Config { Source: [`packages/jobs/jobs-local/src/index.ts:31`](../packages/jobs/jobs-local/src/index.ts) + + +## `@deepseek-ai/dsh-llm` + +```ts config-catalog +/** Deployment-wide defaults applied to provider routes without adapter-owned overrides. */ +export interface Config { + /** Model-request retry policy inherited by routes whose adapter omits one; omission uses normal defaults. */ + defaultRetryPolicy?: RetryPolicyConfig +} + +/** Model-request retry policy configuration for a provider route or deployment default. */ +export type RetryPolicyConfig = NormalRetryPolicyConfig | AlwaysRetryPolicyConfig + +/** Current bounded transient retry behavior for one provider route. */ +export interface NormalRetryPolicyConfig { + /** Retry only configured transient failure codes. */ + mode: 'normal' + /** Maximum eligible retries after the first request (default 2). */ + maxRetries?: number + /** Stable failure codes eligible for this policy. */ + retryableCodes?: string[] + /** Local exponential-backoff and jitter configuration. */ + backoff?: BackoffConfig +} + +/** Unbounded retry behavior for every model-request failure on one provider route. */ +export interface AlwaysRetryPolicyConfig { + /** Retry every model-request failure until success, cancellation, or disposal. */ + mode: 'always' + /** Local exponential-backoff and jitter configuration. */ + backoff?: BackoffConfig +} + +/** Bounded exponential backoff with symmetric jitter around each local delay. */ +export interface BackoffConfig { + /** Initial local exponential-backoff delay in milliseconds (default 500). */ + initialDelayMs?: number + /** Maximum locally scheduled or accepted provider delay in milliseconds (default 10000). */ + maxDelayMs?: number + /** Symmetric random multiplier range around one (default 0.1). */ + jitterRatio?: number +} +``` + +Source: [`packages/llm/llm/src/index.ts:176`](../packages/llm/llm/src/index.ts) + ## `@deepseek-ai/dsh-llm-deepseek` @@ -868,7 +915,7 @@ export interface Config { models?: DeepSeekCatalogModel[] /** Maximum provider idle time while one stream read is outstanding (default five minutes). */ streamIdleTimeoutMs?: number - /** Provider-owned model-request retry policy; omission uses normal defaults. */ + /** Provider-owned model-request retry policy; omission inherits the LLM deployment default. */ retryPolicy?: RetryPolicyConfig } @@ -898,13 +945,8 @@ Source: [`packages/llm/llm-deepseek/src/index.ts:62`](../packages/llm/llm-deepse Requires: `llm` ```ts config-catalog -/** Plugin configuration: the provider routes this instance owns and their shared defaults. */ +/** Plugin configuration: the provider routes this instance owns. */ export interface Config { - /** - * Retry policy inherited by every provider profile that omits its own - * `retryPolicy`; omission here uses the bounded normal defaults. - */ - defaultRetryPolicy?: RetryPolicyConfig /** * pi-ai provider routes, keyed by provider. An empty (or omitted) dict is * the dormant settings-driven posture: the adapter mounts with no routes @@ -988,7 +1030,7 @@ export interface PiAiProviderProfile { websocketConnectTimeoutMs?: number /** Maximum provider idle time while one stream read is outstanding. */ streamIdleTimeoutMs?: number - /** Provider-owned model-request retry policy; omission inherits the adapter default, then normal defaults. */ + /** Provider-owned model-request retry policy; omission inherits the LLM deployment default. */ retryPolicy?: RetryPolicyConfig } @@ -1084,7 +1126,7 @@ type WithheldThinkingFormat = 'chat-template' | 'qwen-chat-template' Depends on: `Api` (`@earendil-works/pi-ai`) · `CacheRetention` (`@earendil-works/pi-ai`) · `Model` (`@earendil-works/pi-ai`) · `ModelThinkingLevel` (`@earendil-works/pi-ai`) · `OpenAICompletionsCompat` (`@earendil-works/pi-ai`) · [`RetryPolicyConfig`](../packages/llm/llm/src/index.ts) · `ThinkingBudgets` (`@earendil-works/pi-ai`) · `Transport` (`@earendil-works/pi-ai`) -Source: [`packages/llm/llm-pi-ai/src/config.ts:174`](../packages/llm/llm-pi-ai/src/config.ts) +Source: [`packages/llm/llm-pi-ai/src/config.ts:172`](../packages/llm/llm-pi-ai/src/config.ts) @@ -3076,7 +3118,6 @@ These load from a `cordis.yml` entry with no `config:` block; they declare no co - `@deepseek-ai/dsh-host-directory-picker-auto` — requires `webServer` · `loader` ([`packages/host/directory-picker-auto/src/index.ts`](../packages/host/directory-picker-auto/src/index.ts)) - `@deepseek-ai/dsh-host-directory-picker-native` ([`packages/host/directory-picker-native/src/index.ts`](../packages/host/directory-picker-native/src/index.ts)) - `@deepseek-ai/dsh-host-plugin-inventory` — requires `loader` ([`packages/host/plugin-inventory/src/index.ts`](../packages/host/plugin-inventory/src/index.ts)) -- `@deepseek-ai/dsh-llm` ([`packages/llm/llm/src/index.ts`](../packages/llm/llm/src/index.ts)) - `@deepseek-ai/dsh-lsp` ([`packages/lsp/lsp/src/index.ts`](../packages/lsp/lsp/src/index.ts)) - `@deepseek-ai/dsh-schedule` — requires `agents` · `sessions` · `tools` · `sessionPersistence` ([`packages/schedule/schedule/src/index.ts`](../packages/schedule/schedule/src/index.ts)) - `@deepseek-ai/dsh-session` ([`packages/core/session/src/index.ts`](../packages/core/session/src/index.ts)) diff --git a/docs/config-catalog.zh.md b/docs/config-catalog.zh.md index 4165d92956..453d9dcc63 100644 --- a/docs/config-catalog.zh.md +++ b/docs/config-catalog.zh.md @@ -838,6 +838,53 @@ export interface Config { 来源:[`packages/jobs/jobs-local/src/index.ts:31`](../packages/jobs/jobs-local/src/index.ts) + + +## `@deepseek-ai/dsh-llm` + +```ts config-catalog +/** Deployment-wide defaults applied to provider routes without adapter-owned overrides. */ +export interface Config { + /** Model-request retry policy inherited by routes whose adapter omits one; omission uses normal defaults. */ + defaultRetryPolicy?: RetryPolicyConfig +} + +/** Model-request retry policy configuration for a provider route or deployment default. */ +export type RetryPolicyConfig = NormalRetryPolicyConfig | AlwaysRetryPolicyConfig + +/** Current bounded transient retry behavior for one provider route. */ +export interface NormalRetryPolicyConfig { + /** Retry only configured transient failure codes. */ + mode: 'normal' + /** Maximum eligible retries after the first request (default 2). */ + maxRetries?: number + /** Stable failure codes eligible for this policy. */ + retryableCodes?: string[] + /** Local exponential-backoff and jitter configuration. */ + backoff?: BackoffConfig +} + +/** Unbounded retry behavior for every model-request failure on one provider route. */ +export interface AlwaysRetryPolicyConfig { + /** Retry every model-request failure until success, cancellation, or disposal. */ + mode: 'always' + /** Local exponential-backoff and jitter configuration. */ + backoff?: BackoffConfig +} + +/** Bounded exponential backoff with symmetric jitter around each local delay. */ +export interface BackoffConfig { + /** Initial local exponential-backoff delay in milliseconds (default 500). */ + initialDelayMs?: number + /** Maximum locally scheduled or accepted provider delay in milliseconds (default 10000). */ + maxDelayMs?: number + /** Symmetric random multiplier range around one (default 0.1). */ + jitterRatio?: number +} +``` + +来源:[`packages/llm/llm/src/index.ts:176`](../packages/llm/llm/src/index.ts) + ## `@deepseek-ai/dsh-llm-deepseek` @@ -870,7 +917,7 @@ export interface Config { models?: DeepSeekCatalogModel[] /** Maximum provider idle time while one stream read is outstanding (default five minutes). */ streamIdleTimeoutMs?: number - /** Provider-owned model-request retry policy; omission uses normal defaults. */ + /** Provider-owned model-request retry policy; omission inherits the LLM deployment default. */ retryPolicy?: RetryPolicyConfig } @@ -900,13 +947,8 @@ export interface DeepSeekCatalogModel { 需要:`llm` ```ts config-catalog -/** Plugin configuration: the provider routes this instance owns and their shared defaults. */ +/** Plugin configuration: the provider routes this instance owns. */ export interface Config { - /** - * Retry policy inherited by every provider profile that omits its own - * `retryPolicy`; omission here uses the bounded normal defaults. - */ - defaultRetryPolicy?: RetryPolicyConfig /** * pi-ai provider routes, keyed by provider. An empty (or omitted) dict is * the dormant settings-driven posture: the adapter mounts with no routes @@ -990,7 +1032,7 @@ export interface PiAiProviderProfile { websocketConnectTimeoutMs?: number /** Maximum provider idle time while one stream read is outstanding. */ streamIdleTimeoutMs?: number - /** Provider-owned model-request retry policy; omission inherits the adapter default, then normal defaults. */ + /** Provider-owned model-request retry policy; omission inherits the LLM deployment default. */ retryPolicy?: RetryPolicyConfig } @@ -1086,7 +1128,7 @@ type WithheldThinkingFormat = 'chat-template' | 'qwen-chat-template' 依赖:`Api`(`@earendil-works/pi-ai`)· `CacheRetention`(`@earendil-works/pi-ai`)· `Model`(`@earendil-works/pi-ai`)· `ModelThinkingLevel`(`@earendil-works/pi-ai`)· `OpenAICompletionsCompat`(`@earendil-works/pi-ai`)· [`RetryPolicyConfig`](../packages/llm/llm/src/index.ts) · `ThinkingBudgets`(`@earendil-works/pi-ai`)· `Transport`(`@earendil-works/pi-ai`) -来源:[`packages/llm/llm-pi-ai/src/config.ts:174`](../packages/llm/llm-pi-ai/src/config.ts) +来源:[`packages/llm/llm-pi-ai/src/config.ts:172`](../packages/llm/llm-pi-ai/src/config.ts) @@ -3078,7 +3120,6 @@ export interface Config { - `@deepseek-ai/dsh-host-directory-picker-auto` — 需要 `webServer` · `loader`([`packages/host/directory-picker-auto/src/index.ts`](../packages/host/directory-picker-auto/src/index.ts)) - `@deepseek-ai/dsh-host-directory-picker-native`([`packages/host/directory-picker-native/src/index.ts`](../packages/host/directory-picker-native/src/index.ts)) - `@deepseek-ai/dsh-host-plugin-inventory` — 需要 `loader`([`packages/host/plugin-inventory/src/index.ts`](../packages/host/plugin-inventory/src/index.ts)) -- `@deepseek-ai/dsh-llm`([`packages/llm/llm/src/index.ts`](../packages/llm/llm/src/index.ts)) - `@deepseek-ai/dsh-lsp`([`packages/lsp/lsp/src/index.ts`](../packages/lsp/lsp/src/index.ts)) - `@deepseek-ai/dsh-schedule` — 需要 `agents` · `sessions` · `tools` · `sessionPersistence`([`packages/schedule/schedule/src/index.ts`](../packages/schedule/schedule/src/index.ts)) - `@deepseek-ai/dsh-session`([`packages/core/session/src/index.ts`](../packages/core/session/src/index.ts)) diff --git a/docs/event-producer-consumer.i18n.yaml b/docs/event-producer-consumer.i18n.yaml index 7e67006a40..78670a641b 100644 --- a/docs/event-producer-consumer.i18n.yaml +++ b/docs/event-producer-consumer.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write docs/event-producer-consumer.md -event-producer-consumer.md: 6a79e6f7ce5addc64b10efa8da7a886dcfb36dc2 -event-producer-consumer.zh.md: f7576a8e28e4f1db2c65c324595c05c98b8fe488 +event-producer-consumer.md: c17a6163ef829f0176163024a4632b712a7195aa +event-producer-consumer.zh.md: 5e6073ad3dbe1e240b5bc6f5fbc55b499c9373c2 diff --git a/docs/event-producer-consumer.md b/docs/event-producer-consumer.md index 6a79e6f7ce..c17a6163ef 100644 --- a/docs/event-producer-consumer.md +++ b/docs/event-producer-consumer.md @@ -36,7 +36,7 @@ This matrix shows which packages dispatch each harness-owned event and which pac | `fs/write-intent` | `waterfall` | [`packages/fs/fs/src/index.ts:58`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`waterfall`), [`tool-str-replace-editor`](../packages/fs/tool-str-replace-editor) (`waterfall`) | [`fs-observation-policy`](../packages/fs/fs-observation-policy) | | `goal/changed` | `emit` | [`packages/goal/goal/src/domain.ts:114`](../packages/goal/goal/src/domain.ts) | [`goal`](../packages/goal/goal) (`emit`) | [`goal-round-driver`](../packages/goal/goal-round-driver) | | `llm/adapters-updated` | `emit` | [`packages/llm/llm/src/types.ts:23`](../packages/llm/llm/src/types.ts) | [`llm`](../packages/llm/llm) (`events.dispatch`) | `apiproxy`, [`llm`](../packages/llm/llm) | -| `llm/stream` | `waterfall` | [`packages/llm/llm/src/index.ts:64`](../packages/llm/llm/src/index.ts) | [`llm`](../packages/llm/llm) (`waterfall`) | [`agent-loop`](../packages/core/agent-loop), [`llm`](../packages/llm/llm), [`llm-replay`](../packages/test-support/llm-replay), [`session-checkpoint-policy`](../packages/session/session-checkpoint-policy), [`session-title`](../packages/session/session-title) | +| `llm/stream` | `waterfall` | [`packages/llm/llm/src/index.ts:65`](../packages/llm/llm/src/index.ts) | [`llm`](../packages/llm/llm) (`waterfall`) | [`agent-loop`](../packages/core/agent-loop), [`llm`](../packages/llm/llm), [`llm-replay`](../packages/test-support/llm-replay), [`session-checkpoint-policy`](../packages/session/session-checkpoint-policy), [`session-title`](../packages/session/session-title) | | `session-telemetry/record` | `waterfall` | [`packages/session/session-telemetry/src/index.ts:43`](../packages/session/session-telemetry/src/index.ts) | [`session-telemetry`](../packages/session/session-telemetry) (`waterfall`) | - | | `session/created` | `emit` | [`packages/core/session/src/index.ts:54`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`events.dispatch`) | `apiproxy`, [`compaction`](../packages/compaction/compaction), [`goal`](../packages/goal/goal), [`hook-protocol`](../packages/hooks/hook-protocol), [`llm-retry`](../packages/llm/llm-retry), [`permission-presets`](../packages/interaction/permission-presets), [`plan-mode`](../packages/plan/plan-mode), [`schedule`](../packages/schedule/schedule), `server`, [`session`](../packages/core/session), [`session-persistence`](../packages/session/session-persistence), [`session-telemetry`](../packages/session/session-telemetry), [`time-context`](../packages/context/time-context), [`tool-workflow`](../packages/workflow/tool-workflow), [`tools`](../packages/core/tools), [`user-approval`](../packages/interaction/user-approval) | | `session/disposed` | `emit` | [`packages/core/session/src/index.ts:64`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`events.dispatch`) | [`agent-loop`](../packages/core/agent-loop), `apiproxy`, [`session-persistence`](../packages/session/session-persistence), [`session-projection-cache`](../packages/session/session-projection-cache), [`session-telemetry`](../packages/session/session-telemetry), [`session-title`](../packages/session/session-title) | diff --git a/docs/event-producer-consumer.zh.md b/docs/event-producer-consumer.zh.md index f7576a8e28..5e6073ad3d 100644 --- a/docs/event-producer-consumer.zh.md +++ b/docs/event-producer-consumer.zh.md @@ -38,7 +38,7 @@ | `fs/write-intent` | `waterfall` | [`packages/fs/fs/src/index.ts:58`](../packages/fs/fs/src/index.ts) | [`tool-fs`](../packages/fs/tool-fs) (`waterfall`), [`tool-str-replace-editor`](../packages/fs/tool-str-replace-editor) (`waterfall`) | [`fs-observation-policy`](../packages/fs/fs-observation-policy) | | `goal/changed` | `emit` | [`packages/goal/goal/src/domain.ts:114`](../packages/goal/goal/src/domain.ts) | [`goal`](../packages/goal/goal) (`emit`) | [`goal-round-driver`](../packages/goal/goal-round-driver) | | `llm/adapters-updated` | `emit` | [`packages/llm/llm/src/types.ts:23`](../packages/llm/llm/src/types.ts) | [`llm`](../packages/llm/llm) (`events.dispatch`) | `apiproxy`, [`llm`](../packages/llm/llm) | -| `llm/stream` | `waterfall` | [`packages/llm/llm/src/index.ts:64`](../packages/llm/llm/src/index.ts) | [`llm`](../packages/llm/llm) (`waterfall`) | [`agent-loop`](../packages/core/agent-loop), [`llm`](../packages/llm/llm), [`llm-replay`](../packages/test-support/llm-replay), [`session-checkpoint-policy`](../packages/session/session-checkpoint-policy), [`session-title`](../packages/session/session-title) | +| `llm/stream` | `waterfall` | [`packages/llm/llm/src/index.ts:65`](../packages/llm/llm/src/index.ts) | [`llm`](../packages/llm/llm) (`waterfall`) | [`agent-loop`](../packages/core/agent-loop), [`llm`](../packages/llm/llm), [`llm-replay`](../packages/test-support/llm-replay), [`session-checkpoint-policy`](../packages/session/session-checkpoint-policy), [`session-title`](../packages/session/session-title) | | `session-telemetry/record` | `waterfall` | [`packages/session/session-telemetry/src/index.ts:43`](../packages/session/session-telemetry/src/index.ts) | [`session-telemetry`](../packages/session/session-telemetry) (`waterfall`) | - | | `session/created` | `emit` | [`packages/core/session/src/index.ts:54`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`events.dispatch`) | `apiproxy`, [`compaction`](../packages/compaction/compaction), [`goal`](../packages/goal/goal), [`hook-protocol`](../packages/hooks/hook-protocol), [`llm-retry`](../packages/llm/llm-retry), [`permission-presets`](../packages/interaction/permission-presets), [`plan-mode`](../packages/plan/plan-mode), [`schedule`](../packages/schedule/schedule), `server`, [`session`](../packages/core/session), [`session-persistence`](../packages/session/session-persistence), [`session-telemetry`](../packages/session/session-telemetry), [`time-context`](../packages/context/time-context), [`tool-workflow`](../packages/workflow/tool-workflow), [`tools`](../packages/core/tools), [`user-approval`](../packages/interaction/user-approval) | | `session/disposed` | `emit` | [`packages/core/session/src/index.ts:64`](../packages/core/session/src/index.ts) | [`session`](../packages/core/session) (`events.dispatch`) | [`agent-loop`](../packages/core/agent-loop), `apiproxy`, [`session-persistence`](../packages/session/session-persistence), [`session-projection-cache`](../packages/session/session-projection-cache), [`session-telemetry`](../packages/session/session-telemetry), [`session-title`](../packages/session/session-title) | diff --git a/docs/subsystems/llm-streaming.i18n.yaml b/docs/subsystems/llm-streaming.i18n.yaml index 8f287e2a0f..35e513dcd7 100644 --- a/docs/subsystems/llm-streaming.i18n.yaml +++ b/docs/subsystems/llm-streaming.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write docs/subsystems/llm-streaming.md -llm-streaming.md: 7c0e0865f8dcc0e7722bb2205d0129d9e0ca3086 -llm-streaming.zh.md: 5c31909ee79137c6c5eef101235b43a2419b1339 +llm-streaming.md: 4627f3cd9e224ef2cced3253b82d2b8461d73986 +llm-streaming.zh.md: a161267162aa5773279ca4a417caec57cd721115 diff --git a/docs/subsystems/llm-streaming.md b/docs/subsystems/llm-streaming.md index 7c0e0865f8..4627f3cd9e 100644 --- a/docs/subsystems/llm-streaming.md +++ b/docs/subsystems/llm-streaming.md @@ -240,7 +240,7 @@ Every adapter MUST obey these, and every consumer may rely on them: ## `ResolvedRetryPolicy` -Provider configuration resolves before route registration into an immutable discriminated union. Normal mode carries `mode: 'normal'`, finite `maxRetries`, `retryableCodes`, and required `initialDelayMs`, `maxDelayMs`, and `jitterRatio`; always mode carries `mode: 'always'` and the same required backoff fields without a finite maximum. `LlmRuntime.providerRetryPolicy(provider)` returns the currently registered value and supplies normal defaults when the adapter omits one; `llmRetryPolicyOf(stream)` returns the value captured from the serving registration after the call selects that registration, so later route disposal or replacement cannot change an in-flight failure's recovery policy. The [generated config catalog](../config-catalog.md) lists the optional input fields. +Retry configuration resolves before route registration into an immutable discriminated union. Normal mode carries `mode: 'normal'`, finite `maxRetries`, `retryableCodes`, and required `initialDelayMs`, `maxDelayMs`, and `jitterRatio`; always mode carries `mode: 'always'` and the same required backoff fields without a finite maximum. An explicit adapter route policy overrides `LlmRuntime.defaultRetryPolicy`; omitting both uses the normal default of two retries. `LlmRuntime.providerRetryPolicy(provider)` returns that effective registered value, and `llmRetryPolicyOf(stream)` returns the value captured from the serving registration after the call selects it, so later route disposal or replacement cannot change an in-flight failure's recovery policy. The [generated config catalog](../config-catalog.md) lists the optional input fields. ## `AppIdentity` — app attribution @@ -655,7 +655,7 @@ interface LlmCallConfigAdapterDefaults { ## Service and provider contracts -`LlmAdapter` is the provider contract: subclass, implement `stream()`, and register one adapter instance with `ctx.llm.registerAdapter(providers, adapter)`. `GenerateOptions.provider` selects the registered adapter; `GenerateOptions.model` is passed to that adapter and need not be registered at lifecycle start. Duplicate provider routes fail atomically. Optional `providerRetryPolicy()` is captured per route with normal defaults, while `providerInfo()` and asynchronous `listModels()` feed `LlmRuntime.listProviders()` / `listModels()` with detached selector metadata. That catalog is advisory rather than a request whitelist: the adapter remains authoritative and may accept unlisted model ids. One asynchronous `resolveModel()` query returns exact model identity plus optional correctness-sensitive context capacity, an adapter-configured `defaultMaxTokens`, and ordered model-owned reasoning ids with an optional deployment default; absent fields mean unavailable metadata or provider-owned behavior, not invalid catalog membership. The resolver receives optional cancellation and must settle promptly after abort. `LlmRuntime.resolveModelInfo()` validates and detaches the aggregate. At the final adapter boundary, `resolveCallConfig()` materializes the output default only when `maxTokens` is absent and validates and materializes reasoning, so direct calls cannot bypass either configured behavior; direct dispatch captures one registration before awaiting that resolution. The agent loop instead uses `prepareCall()` to keep the same registration across model resolution, durable header logging, and dispatch, retain detached context metadata from that exact lookup, and report which config fields the adapter defaulted. Adapter lookup happens at the terminal continuation of the `llm/stream` waterfall, so a listener may short-circuit the call or route a mutable one-shot request before lookup. AgentLoop observes a request attempt once the outer waterfall returns a stream handle; that limited boundary does not prove a lazy terminal adapter was constructed or began provider I/O. The `block-start` / `block-end` `index` correlation and the assembler together mean an adapter only has to emit well-formed chunks — block reassembly is not each adapter's problem. [architecture.md](../architecture.md#turn-flow) shows where `ctx.llm.stream()` and the `llm/stream` waterfall sit in one turn. +`LlmAdapter` is the provider contract: subclass, implement `stream()`, and register one adapter instance with `ctx.llm.registerAdapter(providers, adapter)`. `GenerateOptions.provider` selects the registered adapter; `GenerateOptions.model` is passed to that adapter and need not be registered at lifecycle start. Duplicate provider routes fail atomically. Optional `providerRetryPolicy()` supplies an explicit route override; omission inherits the LLM deployment default. `providerInfo()` and asynchronous `listModels()` feed `LlmRuntime.listProviders()` / `listModels()` with detached selector metadata. That catalog is advisory rather than a request whitelist: the adapter remains authoritative and may accept unlisted model ids. One asynchronous `resolveModel()` query returns exact model identity plus optional correctness-sensitive context capacity, an adapter-configured `defaultMaxTokens`, and ordered model-owned reasoning ids with an optional deployment default; absent fields mean unavailable metadata or provider-owned behavior, not invalid catalog membership. The resolver receives optional cancellation and must settle promptly after abort. `LlmRuntime.resolveModelInfo()` validates and detaches the aggregate. At the final adapter boundary, `resolveCallConfig()` materializes the output default only when `maxTokens` is absent and validates and materializes reasoning, so direct calls cannot bypass either configured behavior; direct dispatch captures one registration before awaiting that resolution. The agent loop instead uses `prepareCall()` to keep the same registration across model resolution, durable header logging, and dispatch, retain detached context metadata from that exact lookup, and report which config fields the adapter defaulted. Adapter lookup happens at the terminal continuation of the `llm/stream` waterfall, so a listener may short-circuit the call or route a mutable one-shot request before lookup. AgentLoop observes a request attempt once the outer waterfall returns a stream handle; that limited boundary does not prove a lazy terminal adapter was constructed or began provider I/O. The `block-start` / `block-end` `index` correlation and the assembler together mean an adapter only has to emit well-formed chunks — block reassembly is not each adapter's problem. [architecture.md](../architecture.md#turn-flow) shows where `ctx.llm.stream()` and the `llm/stream` waterfall sit in one turn. ```ts type-equiv /** One model call whose config and adapter registration were resolved together. */ @@ -694,9 +694,9 @@ declare abstract class LlmAdapter { */ providerInfo(provider: string): LlmProviderInfo; /** - * Return the provider-owned retry policy captured with this route. + * Return an explicit provider-owned retry policy override for this route. * @param _provider - a route passed to `registerAdapter()` for this instance. - * @returns a resolved policy, or `undefined` to use the normal defaults. + * @returns a resolved override, or `undefined` to inherit the LLM deployment default. */ providerRetryPolicy(_provider: string): ResolvedRetryPolicy | undefined; /** @@ -803,9 +803,9 @@ registerModelDiscovery( settingsNs: string, discover: (request: LlmModelDiscover async discoverModels( settingsNs: string, request: LlmModelDiscoveryRequest, ): Promise /** - * Resolve the retry policy captured when one provider route was registered. + * Read the effective retry policy captured when one provider route was registered. * @param provider - registered provider route to inspect. - * @returns the provider-owned policy, with normal defaults already resolved. + * @returns the adapter override or deployment default, fully resolved. */ providerRetryPolicy(provider: string): ResolvedRetryPolicy @@ -864,7 +864,7 @@ async prepareCall(config: LlmCallConfig, signal?: AbortSignal): Promise ``` -Source: [`packages/llm/llm/src/index.ts:284`](../../packages/llm/llm/src/index.ts) +Source: [`packages/llm/llm/src/index.ts:291`](../../packages/llm/llm/src/index.ts) @@ -913,5 +913,5 @@ Waterfall around every streaming model call (retry, replay, routing). Bound to t 'llm/stream'(this: LlmRuntime, options: GenerateOptions, next: () => AsyncIterable): AsyncIterable ``` -Source: [`packages/llm/llm/src/index.ts:64`](../../packages/llm/llm/src/index.ts) +Source: [`packages/llm/llm/src/index.ts:65`](../../packages/llm/llm/src/index.ts) diff --git a/docs/subsystems/llm-streaming.zh.md b/docs/subsystems/llm-streaming.zh.md index 5c31909ee7..a161267162 100644 --- a/docs/subsystems/llm-streaming.zh.md +++ b/docs/subsystems/llm-streaming.zh.md @@ -242,7 +242,7 @@ interface LlmFailure { ## `ResolvedRetryPolicy` -提供方配置会在路由注册前解析为不可变的可辨识联合。normal mode 携带 `mode: 'normal'`、有限的 `maxRetries`、`retryableCodes`,以及必填的 `initialDelayMs`、`maxDelayMs` 与 `jitterRatio`;always mode 携带 `mode: 'always'` 和相同的必填退避字段,但没有有限上限。`LlmRuntime.providerRetryPolicy(provider)` 返回当前注册的值,并在适配器省略策略时提供 normal 默认值;调用选定该注册后,`llmRetryPolicyOf(stream)` 返回为该调用服务的注册所捕获的值,因此之后释放或替换路由都无法改变进行中失败的恢复策略。可选配置输入字段由[生成的配置目录](../config-catalog.md)列出。 +重试配置会在路由注册前解析为不可变的可辨识联合。normal mode 携带 `mode: 'normal'`、有限的 `maxRetries`、`retryableCodes`,以及必填的 `initialDelayMs`、`maxDelayMs` 与 `jitterRatio`;always mode 携带 `mode: 'always'` 和相同的必填退避字段,但没有有限上限。显式适配器路由策略会覆盖 `LlmRuntime.defaultRetryPolicy`;两者都省略时使用两次重试的 normal 默认值。`LlmRuntime.providerRetryPolicy(provider)` 返回注册的该有效值;调用选定实际提供服务的注册后,`llmRetryPolicyOf(stream)` 返回从中捕获的值,因此之后释放或替换路由都无法改变进行中失败的恢复策略。可选配置输入字段由[生成的配置目录](../config-catalog.md)列出。 ## `AppIdentity`:应用归属 @@ -661,7 +661,7 @@ interface LlmCallConfigAdapterDefaults { ## 服务与提供方约定 -`LlmAdapter` 是提供方约定:创建子类、实现 `stream()`,再用 `ctx.llm.registerAdapter(providers, adapter)` 注册一个适配器实例。`GenerateOptions.provider` 选择已注册适配器;`GenerateOptions.model` 会传给该适配器,无需在生命周期启动时注册。重复提供方路由会原子失败。可选的 `providerRetryPolicy()` 会按路由捕获并填入 normal 默认值,`providerInfo()` 与异步 `listModels()` 方法则为 `LlmRuntime.listProviders()` / `listModels()` 提供分离的 selector 元数据。该目录仅供参考,不是请求白名单:适配器仍是权威,并可接受未列出的模型 id。单次异步 `resolveModel()` 查询返回确切模型身份,以及可选的对正确性敏感的上下文容量、适配器配置的 `defaultMaxTokens`、由模型持有的有序推理强度 ID 和可选的部署默认值;字段缺失表示元数据不可用或保留提供方持有的行为,而不表示目录成员关系无效。解析器会接收可选的取消信号,并且必须在信号中止后迅速完成结算。`LlmRuntime.resolveModelInfo()` 会校验聚合结果并返回分离值。在最终适配器边界,`resolveCallConfig()` 仅在 `maxTokens` 缺失时填入输出默认值,并校验和填入推理强度,因此直接调用也无法绕过任何一项已配置行为;直接分派会在等待解析前捕获一项适配器注册。agent loop 则使用 `prepareCall()`,使模型解析、请求头持久记录和分派全程使用同一项注册,保留来自同一次查询的分离上下文元数据,并报告适配器填入的配置字段。适配器查找发生在 `llm/stream` waterfall 的终端 continuation,因此 listener 可以在查找前短路调用,或路由一个可变的一次性请求。AgentLoop 在外层 waterfall 返回流句柄时观察到一次请求尝试;这个有限边界不能证明惰性终端适配器已构造完成或开始提供方 I/O。`block-start` / `block-end` 的 `index` 关联与 assembler 共同意味着适配器只需 emit 格式正确的分片——块重组不是每个适配器各自的问题。`ctx.llm.stream()` 与 `llm/stream` waterfall 在一个轮次中的位置见 [architecture.md](../architecture.md#turn-flow)。 +`LlmAdapter` 是提供方约定:创建子类、实现 `stream()`,再用 `ctx.llm.registerAdapter(providers, adapter)` 注册一个适配器实例。`GenerateOptions.provider` 选择已注册适配器;`GenerateOptions.model` 会传给该适配器,无需在生命周期启动时注册。重复提供方路由会原子失败。可选的 `providerRetryPolicy()` 提供显式路由覆盖;省略时继承 LLM 部署默认值。`providerInfo()` 与异步 `listModels()` 方法则为 `LlmRuntime.listProviders()` / `listModels()` 提供分离的 selector 元数据。该目录仅供参考,不是请求白名单:适配器仍是权威,并可接受未列出的模型 id。单次异步 `resolveModel()` 查询返回确切模型身份,以及可选的对正确性敏感的上下文容量、适配器配置的 `defaultMaxTokens`、由模型持有的有序推理强度 ID 和可选的部署默认值;字段缺失表示元数据不可用或保留提供方持有的行为,而不表示目录成员关系无效。解析器会接收可选的取消信号,并且必须在信号中止后迅速完成结算。`LlmRuntime.resolveModelInfo()` 会校验聚合结果并返回分离值。在最终适配器边界,`resolveCallConfig()` 仅在 `maxTokens` 缺失时填入输出默认值,并校验和填入推理强度,因此直接调用也无法绕过任何一项已配置行为;直接分派会在等待解析前捕获一项适配器注册。agent loop 则使用 `prepareCall()`,使模型解析、请求头持久记录和分派全程使用同一项注册,保留来自同一次查询的分离上下文元数据,并报告适配器填入的配置字段。适配器查找发生在 `llm/stream` waterfall 的终端 continuation,因此 listener 可以在查找前短路调用,或路由一个可变的一次性请求。AgentLoop 在外层 waterfall 返回流句柄时观察到一次请求尝试;这个有限边界不能证明惰性终端适配器已构造完成或开始提供方 I/O。`block-start` / `block-end` 的 `index` 关联与 assembler 共同意味着适配器只需 emit 格式正确的分片——块重组不是每个适配器各自的问题。`ctx.llm.stream()` 与 `llm/stream` waterfall 在一个轮次中的位置见 [architecture.md](../architecture.md#turn-flow)。 ```ts type-equiv /** One model call whose config and adapter registration were resolved together. */ @@ -700,9 +700,9 @@ declare abstract class LlmAdapter { */ providerInfo(provider: string): LlmProviderInfo; /** - * Return the provider-owned retry policy captured with this route. + * Return an explicit provider-owned retry policy override for this route. * @param _provider - a route passed to `registerAdapter()` for this instance. - * @returns a resolved policy, or `undefined` to use the normal defaults. + * @returns a resolved override, or `undefined` to inherit the LLM deployment default. */ providerRetryPolicy(_provider: string): ResolvedRetryPolicy | undefined; /** @@ -809,9 +809,9 @@ registerModelDiscovery( settingsNs: string, discover: (request: LlmModelDiscover async discoverModels( settingsNs: string, request: LlmModelDiscoveryRequest, ): Promise /** - * Resolve the retry policy captured when one provider route was registered. + * Read the effective retry policy captured when one provider route was registered. * @param provider - registered provider route to inspect. - * @returns the provider-owned policy, with normal defaults already resolved. + * @returns the adapter override or deployment default, fully resolved. */ providerRetryPolicy(provider: string): ResolvedRetryPolicy @@ -870,7 +870,7 @@ async prepareCall(config: LlmCallConfig, signal?: AbortSignal): Promise ``` -Source: [`packages/llm/llm/src/index.ts:284`](../../packages/llm/llm/src/index.ts) +Source: [`packages/llm/llm/src/index.ts:291`](../../packages/llm/llm/src/index.ts) @@ -919,5 +919,5 @@ Waterfall around every streaming model call (retry, replay, routing). Bound to t 'llm/stream'(this: LlmRuntime, options: GenerateOptions, next: () => AsyncIterable): AsyncIterable ``` -Source: [`packages/llm/llm/src/index.ts:64`](../../packages/llm/llm/src/index.ts) +Source: [`packages/llm/llm/src/index.ts:65`](../../packages/llm/llm/src/index.ts) diff --git a/packages/bundle/web-app/README.i18n.yaml b/packages/bundle/web-app/README.i18n.yaml index 3db5227434..db9ae65f97 100644 --- a/packages/bundle/web-app/README.i18n.yaml +++ b/packages/bundle/web-app/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/bundle/web-app/README.md -README.md: 1593b801154bcd43625f8aadbf7ca2a0f6df69a2 -README.zh.md: 605d207f3c7870fad9cf1174f8873d1904684c39 +README.md: 30ad018ef60ea2c1c8926f482dd839dec2ee61d8 +README.zh.md: ccc7acff3027ce7cd973f3d9e9a48d831ade00e5 diff --git a/packages/bundle/web-app/README.md b/packages/bundle/web-app/README.md index 1593b80115..30ad018ef6 100644 --- a/packages/bundle/web-app/README.md +++ b/packages/bundle/web-app/README.md @@ -6,7 +6,7 @@ The dsh browser-surface bundle. [`cordis.patch.yml`](cordis.patch.yml) rides ove ## Model retry defaults -The Web layer configures normal-mode retries with `maxRetries: 5`, allowing at most five eligible retries after the initial request. This applies to the shipped `deepseek-official` route and to every pi-ai provider profile that omits its own `retryPolicy`. The `llm-deepseek` user-settings section, a pi-ai profile-level policy, or a later profile patch may replace the corresponding deployment default; non-Web profiles retain their own policies. +The Web layer configures `LlmRuntime.defaultRetryPolicy` with normal mode and `maxRetries: 5`, allowing at most five eligible retries after the initial request. Every adapter route without an explicit provider policy inherits it, including `deepseek-official`, settings-added pi-ai routes, and future adapters. A `llm-deepseek` or pi-ai provider `retryPolicy` overrides the deployment default; non-Web profiles retain the core two-retry default unless their own composition chooses another value. ## Model Experience diff --git a/packages/bundle/web-app/README.zh.md b/packages/bundle/web-app/README.zh.md index 605d207f3c..ccc7acff30 100644 --- a/packages/bundle/web-app/README.zh.md +++ b/packages/bundle/web-app/README.zh.md @@ -6,7 +6,7 @@ dsh 浏览器表层组合包。[`cordis.patch.yml`](cordis.patch.yml) 叠加在 ## 模型重试默认值 -Web 层会配置 normal 模式重试与 `maxRetries: 5`,使其在初次请求后最多再重试 5 次符合条件的失败。该默认值适用于随附的 `deepseek-official` 路由,以及每个未自行配置 `retryPolicy` 的 pi-ai 提供方 profile。`llm-deepseek` 用户设置分节、pi-ai profile 级策略或更后面的 profile patch 可以替换对应的部署默认值;非 Web profile 仍使用各自的策略。 +Web 层会为 `LlmRuntime.defaultRetryPolicy` 配置 normal 模式与 `maxRetries: 5`,使其在初次请求后最多再重试 5 次符合条件的失败。每条没有显式提供方策略的适配器路由都会继承它,包括 `deepseek-official`、由 settings 新增的 pi-ai 路由及未来适配器。`llm-deepseek` 或 pi-ai 提供方的 `retryPolicy` 会覆盖部署默认值;非 Web profile 保留核心的两次重试默认值,除非自身组合选择其他值。 ## 模型体验 diff --git a/packages/bundle/web-app/cordis.patch.yml b/packages/bundle/web-app/cordis.patch.yml index c3b84bf2f7..445095284b 100644 --- a/packages/bundle/web-app/cordis.patch.yml +++ b/packages/bundle/web-app/cordis.patch.yml @@ -40,16 +40,9 @@ # once the web UI owns the choice per session. mode: !!js process.env.DSH_TOOLS_MODE -# The Web deployment gives its model routes a larger bounded transient-failure -# budget than the provider omission default. DeepSeek settings can override -# its policy, and each pi-ai provider profile can override the inherited one. -- id: llm-deepseek - config: - retryPolicy: - mode: normal - maxRetries: 5 - -- id: llm-pi-ai +# The Web deployment gives every model route without an explicit provider +# policy a larger bounded transient-failure budget than the core default. +- id: llm config: defaultRetryPolicy: mode: normal diff --git a/packages/extensions/tool-cordis/src/api-catalog.ts b/packages/extensions/tool-cordis/src/api-catalog.ts index 7fb624d21f..b0db0ed002 100644 --- a/packages/extensions/tool-cordis/src/api-catalog.ts +++ b/packages/extensions/tool-cordis/src/api-catalog.ts @@ -826,9 +826,9 @@ export const SERVICE_API: readonly ServiceApiEntry[] = [ }, { signature: 'providerRetryPolicy(provider: string): ResolvedRetryPolicy', - description: 'Resolve the retry policy captured when one provider route was registered.', + description: 'Read the effective retry policy captured when one provider route was registered.', parameters: [{ name: 'provider', description: 'registered provider route to inspect.' }], - returns: 'the provider-owned policy, with normal defaults already resolved.', + returns: 'the adapter override or deployment default, fully resolved.', }, { signature: 'async listModels(provider: string): Promise', @@ -3315,7 +3315,7 @@ export const TYPE_API: readonly TypeApiEntry[] = [ }, { name: 'LlmRuntime', - declaration: 'export class LlmRuntime extends Service {\n constructor(ctx: Context);\n registerAdapter(providers: string[], adapter: LlmAdapter): AdapterRegistrationHandle;\n listProviders(): LlmProviderInfo[];\n registerConfigurableProviders(entries: readonly LlmConfigurableProvider[]): DirectoryRegistrationHandle;\n listConfigurableProviders(): LlmConfigurableProvider[];\n registerModelDiscovery(settingsNs: string, discover: (request: LlmModelDiscoveryRequest) => Promise): () => void;\n async discoverModels(settingsNs: string, request: LlmModelDiscoveryRequest): Promise;\n providerRetryPolicy(provider: string): ResolvedRetryPolicy;\n async listModels(provider: string): Promise;\n async resolveModelInfo(provider: string, model: string, signal?: AbortSignal): Promise;\n async resolveCallConfig(config: LlmCallConfig, signal?: AbortSignal): Promise;\n async prepareCall(config: LlmCallConfig, signal?: AbortSignal): Promise;\n stream(options: GenerateOptions): AsyncIterable;\n}', + declaration: 'export class LlmRuntime extends Service {\n static Config: z;\n constructor(ctx: Context, config: Config = {});\n registerAdapter(providers: string[], adapter: LlmAdapter): AdapterRegistrationHandle;\n listProviders(): LlmProviderInfo[];\n registerConfigurableProviders(entries: readonly LlmConfigurableProvider[]): DirectoryRegistrationHandle;\n listConfigurableProviders(): LlmConfigurableProvider[];\n registerModelDiscovery(settingsNs: string, discover: (request: LlmModelDiscoveryRequest) => Promise): () => void;\n async discoverModels(settingsNs: string, request: LlmModelDiscoveryRequest): Promise;\n providerRetryPolicy(provider: string): ResolvedRetryPolicy;\n async listModels(provider: string): Promise;\n async resolveModelInfo(provider: string, model: string, signal?: AbortSignal): Promise;\n async resolveCallConfig(config: LlmCallConfig, signal?: AbortSignal): Promise;\n async prepareCall(config: LlmCallConfig, signal?: AbortSignal): Promise;\n stream(options: GenerateOptions): AsyncIterable;\n}', }, { name: 'LspHover', diff --git a/packages/llm/llm-deepseek/README.i18n.yaml b/packages/llm/llm-deepseek/README.i18n.yaml index 4ac4cf4ed0..d0caf1d4ff 100644 --- a/packages/llm/llm-deepseek/README.i18n.yaml +++ b/packages/llm/llm-deepseek/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/llm/llm-deepseek/README.md -README.md: af89e7ee3bab6ec209349d047f81308eb6e87cef -README.zh.md: e9ce3206027ffeee9bc49eb7a2ed76ddcbe7bfc8 +README.md: 9087c57844a3502f8e1ba650979722638eb231f0 +README.zh.md: aea543563d1294d953b36210327cd1bb1d92b961 diff --git a/packages/llm/llm-deepseek/README.md b/packages/llm/llm-deepseek/README.md index af89e7ee3b..9087c57844 100644 --- a/packages/llm/llm-deepseek/README.md +++ b/packages/llm/llm-deepseek/README.md @@ -20,7 +20,7 @@ The package root exposes the Cordis plugin contract and `DeepSeekAdapter`; wire reasoningEffort: high # optional; off | low | high | max — omitted ⇒ high maxTokens: 256000 # optional positive per-request output cap; this is the default streamIdleTimeoutMs: 300000 # optional; positive finite Node timer delay; five-minute default - retryPolicy: # optional; omission uses bounded normal defaults + retryPolicy: # optional; omission inherits the LLM deployment default mode: always # normal | always backoff: initialDelayMs: 500 @@ -35,7 +35,7 @@ The package root exposes the Cordis plugin contract and `DeepSeekAdapter`; wire contextWindow: 512000 ``` -The plugin registers the single provider route `deepseek-official` together with its resolved `retryPolicy`. A request selects it with `provider: deepseek-official`; its `model` is passed through as the wire `model` string, so changing DeepSeek models does not require lifecycle-time registration. Omitting `models` advertises `deepseek-v4-flash` as `DeepSeek-V4-Flash` and `deepseek-v4-pro` as `DeepSeek-V4-Pro`, each with a 1,000,000-token context window; an explicit list replaces those defaults, while `models: []` advertises none. Catalog entries are exposed through `ctx.llm.listModels('deepseek-official')` for clients such as ACP editors and the Web selector, but remain advisory: unlisted model ids still pass through unchanged. An omitted entry name defaults to its id. +The plugin registers the single provider route `deepseek-official` together with its explicit `retryPolicy` override when configured; `LlmRuntime` supplies the deployment default otherwise. A request selects it with `provider: deepseek-official`; its `model` is passed through as the wire `model` string, so changing DeepSeek models does not require lifecycle-time registration. Omitting `models` advertises `deepseek-v4-flash` as `DeepSeek-V4-Flash` and `deepseek-v4-pro` as `DeepSeek-V4-Pro`, each with a 1,000,000-token context window; an explicit list replaces those defaults, while `models: []` advertises none. Catalog entries are exposed through `ctx.llm.listModels('deepseek-official')` for clients such as ACP editors and the Web selector, but remain advisory: unlisted model ids still pass through unchanged. An omitted entry name defaults to its id. `contextWindow` is optional per configured model and is not exposed through the advisory catalog. `ctx.llm.resolveModelInfo('deepseek-official', model).context` returns an exact model value first, then `defaultContextWindow` for an entry without capacity or an unlisted pass-through id. The adapter default is 1,000,000; pressure-sensitive plugins therefore get deployment-owned capacity without treating the model selector as authoritative. Registering another adapter for `deepseek-official` throws `LlmError('DUPLICATE_ADAPTER')`. @@ -45,7 +45,7 @@ The same exact-model result exposes ordered `off`, `low`, `high`, and `max` effo `thinking: disabled` is a deployment lock that publishes only `off` with `off` as its default. Omitting `reasoningEffort` or configuring it as `off` is valid; configuring `low`, `high`, or `max` fails plugin loading, and a direct per-request attempt to enable thinking fails before network I/O. A request with `GenerateOptions.purpose: 'session-title'` also forces thinking disabled and omits the already-resolved effort, reserving its bounded output for visible title text without changing conversation or compaction defaults. -`streamIdleTimeoutMs` bounds each outstanding provider read, including the initial `fetch`, without counting time the consumer spends between chunks. DeepSeek SSE comments rearm an outstanding read as transport activity but never become `StreamChunk` values or session-log events. One stable abort signal reaches the request and body reader for the whole call; expiry stops the transport and throws `LlmError('TIMEOUT')`, while an earlier caller abort throws `LlmError('ABORTED')`. The adapter makes exactly one provider request per `stream()` call; it registers the configured policy as provider metadata, and `dsh-llm-retry` separately executes it at durable agent-step boundaries. +`streamIdleTimeoutMs` bounds each outstanding provider read, including the initial `fetch`, without counting time the consumer spends between chunks. DeepSeek SSE comments rearm an outstanding read as transport activity but never become `StreamChunk` values or session-log events. One stable abort signal reaches the request and body reader for the whole call; expiry stops the transport and throws `LlmError('TIMEOUT')`, while an earlier caller abort throws `LlmError('ABORTED')`. The adapter makes exactly one provider request per `stream()` call; it registers only an explicit provider policy override, `LlmRuntime` captures the effective route policy, and `dsh-llm-retry` separately executes it at durable agent-step boundaries. ## Dynamic configuration (settings + credentials) @@ -54,7 +54,7 @@ Connection facts are not frozen at load. `resolveAdapterOptions` is the one expl - **`ctx.settings`** — the plugin registers the `llm-deepseek` namespace with this same `Config` schema and its `cordis.yml` entry as the composition `base`, so a `llm-deepseek:` section in the user settings document overrides any field without a restart. Without a mounted settings service the entry config alone drives the adapter, unchanged. A live settings snapshot that passes the schema but fails a beyond-schema bound (a duplicate catalog id, a broken thinking/effort pair) keeps the last good facts and logs the failure; the entry config itself still fails plugin load. - **`ctx.credentials`** — the API key resolves per stream call, from the *same* resolved snapshot that supplies the endpoint. Configuration carries only `apiKeyEnv`, never a literal key: the reference resolves through the credential seam, and without a mounted seam through the trusted environment layers. Because credential facts travel with the connection facts, a settings snapshot the resolver rejects contributes neither its endpoint nor its key: the whole previous generation keeps serving. Every resolved key is format-checked before use, so a value no HTTP header can carry is refused with `LlmError('INVALID_CREDENTIAL')` naming the failing entry point — never any part of the key — instead of surfacing as an opaque `fetch` `TypeError`. A request with no key anywhere fails with `MISSING_CREDENTIAL` naming every configuration entry point, while the route stays registered and the catalog stays browsable — first-run onboarding is "browse models, store the key, prompt again", with no restart between. -The one registration-captured fact is the retry policy: when its resolved value changes, the plugin re-registers the route in place (same adapter instance, one synchronous section), so `ctx.llm.providerRetryPolicy('deepseek-official')` always reports the current policy. +The one adapter-owned registration fact is the explicit retry policy override: when its resolved value changes or is removed, the plugin re-registers the route in place (same adapter instance, one synchronous section). `ctx.llm.providerRetryPolicy('deepseek-official')` therefore reports either the current override or `LlmRuntime`'s deployment default. The plugin also declares its route in the configurable-provider directory (`ctx.llm.listConfigurableProviders()`): provider `deepseek-official`, settings namespace `llm-deepseek`, empty settings path — the whole section is the profile. Configuration surfaces use that entry to offer this adapter alongside dormant pi-ai providers. diff --git a/packages/llm/llm-deepseek/README.zh.md b/packages/llm/llm-deepseek/README.zh.md index e9ce320602..aea543563d 100644 --- a/packages/llm/llm-deepseek/README.zh.md +++ b/packages/llm/llm-deepseek/README.zh.md @@ -20,7 +20,7 @@ harness LLM(大语言模型)seam 的 DeepSeek chat-completions 适配器: reasoningEffort: high # optional; off | low | high | max — omitted ⇒ high maxTokens: 256000 # optional positive per-request output cap; this is the default streamIdleTimeoutMs: 300000 # optional; positive finite Node timer delay; five-minute default - retryPolicy: # optional; omission uses bounded normal defaults + retryPolicy: # optional; omission inherits the LLM deployment default mode: always # normal | always backoff: initialDelayMs: 500 @@ -35,7 +35,7 @@ harness LLM(大语言模型)seam 的 DeepSeek chat-completions 适配器: contextWindow: 512000 ``` -该插件注册唯一提供方路由 `deepseek-official`,同时注册解析后的 `retryPolicy`。请求使用 `provider: deepseek-official` 选择该路由;其 `model` 会作为协议 `model` 字符串原样传递,因此更改 DeepSeek 模型不需要生命周期时注册。省略 `models` 会公布 `deepseek-v4-flash`(名称为 `DeepSeek-V4-Flash`)和 `deepseek-v4-pro`(名称为 `DeepSeek-V4-Pro`),两者的上下文窗口均为 1,000,000 token;显式列表会替换这些默认值,`models: []` 则不公布任何模型。Catalog 配置项通过 `ctx.llm.listModels('deepseek-official')` 公开给 ACP(Agent Client Protocol)编辑器和 Web 选择器等客户端,但仍只提供建议:未列出模型 id 仍原样传递。省略配置项 name 默认为其 id。 +该插件注册唯一提供方路由 `deepseek-official`,并在配置存在时一同注册显式 `retryPolicy` 覆盖;否则由 `LlmRuntime` 提供部署默认值。请求使用 `provider: deepseek-official` 选择该路由;其 `model` 会作为协议 `model` 字符串原样传递,因此更改 DeepSeek 模型不需要生命周期时注册。省略 `models` 会公布 `deepseek-v4-flash`(名称为 `DeepSeek-V4-Flash`)和 `deepseek-v4-pro`(名称为 `DeepSeek-V4-Pro`),两者的上下文窗口均为 1,000,000 token;显式列表会替换这些默认值,`models: []` 则不公布任何模型。Catalog 配置项通过 `ctx.llm.listModels('deepseek-official')` 公开给 ACP(Agent Client Protocol)编辑器和 Web 选择器等客户端,但仍只提供建议:未列出模型 id 仍原样传递。省略配置项 name 默认为其 id。 `contextWindow` 对每个已配置模型都可选,不会通过建议 catalog 公开。`ctx.llm.resolveModelInfo('deepseek-official', model).context` 先返回精确模型值,再对不含容量的配置项或未列出原样传递 id 返回 `defaultContextWindow`。适配器默认值为 1,000,000;因此,压力敏感插件可以获得由部署决定的容量,不会将模型 selector 视为权威。为 `deepseek-official` 注册另一个适配器会抛出 `LlmError('DUPLICATE_ADAPTER')`。 @@ -45,7 +45,7 @@ harness LLM(大语言模型)seam 的 DeepSeek chat-completions 适配器: `thinking: disabled` 是部署锁定:它只公布 `off`,并以 `off` 为默认值。省略 `reasoningEffort` 或将其配置为 `off` 均有效;配置 `low`、`high` 或 `max` 会使插件加载失败,直接按请求启用思考也会在网络 I/O 前失败。携带 `GenerateOptions.purpose: 'session-title'` 的请求也会强制禁用思考并省略已解析的推理强度,将有界输出保留给可见标题文本,不改变会话或压缩(compaction)默认值。 -`streamIdleTimeoutMs` 会限制每次未完成提供方读取,包括初始 `fetch`,但不计入消费方在分片间花费的时间。DeepSeek SSE 注释会作为传输活动使尚未完成的读取重新布防,但绝不会成为 `StreamChunk` 值或会话日志事件。同一个稳定的 abort 信号会在整个调用期间传递给请求与 body reader;过期会停止传输并抛出 `LlmError('TIMEOUT')`,较早的调用方 abort 则抛出 `LlmError('ABORTED')`。适配器每次 `stream()` 调用恰好发起一次提供方请求;它把已配置策略注册为提供方元数据,再由 `dsh-llm-retry` 在持久化的 agent(智能体)步骤边界单独执行该策略。 +`streamIdleTimeoutMs` 会限制每次未完成提供方读取,包括初始 `fetch`,但不计入消费方在分片间花费的时间。DeepSeek SSE 注释会作为传输活动使尚未完成的读取重新布防,但绝不会成为 `StreamChunk` 值或会话日志事件。同一个稳定的 abort 信号会在整个调用期间传递给请求与 body reader;过期会停止传输并抛出 `LlmError('TIMEOUT')`,较早的调用方 abort 则抛出 `LlmError('ABORTED')`。适配器每次 `stream()` 调用恰好发起一次提供方请求;它只注册显式提供方策略覆盖,`LlmRuntime` 捕获有效路由策略,再由 `dsh-llm-retry` 在持久化的 agent(智能体)步骤边界单独执行该策略。 ## 动态配置(settings + credentials) @@ -54,7 +54,7 @@ harness LLM(大语言模型)seam 的 DeepSeek chat-completions 适配器: - **`ctx.settings`**——插件用同一份 `Config` schema 注册 `llm-deepseek` namespace,并以其 `cordis.yml` 条目为组合 `base`,因此用户设置文档中的 `llm-deepseek:` 分节可以免重启覆盖任何字段。未挂载 settings 服务时,仅由 entry 配置驱动适配器,行为不变。存活 settings 快照若通过 schema 却违反 schema 之外的约束(重复的 catalog id、无法成立的 thinking/推理强度组合),则保留最后可用事实并记录失败;entry 配置本身仍会使插件加载失败。 - **`ctx.credentials`**——API 密钥按每次 stream 调用解析,取自与端点*同一*份解析后的快照。配置只携带 `apiKeyEnv`,从不携带字面密钥:该引用经凭据 seam 解析,未挂载 seam 时则经受信环境层解析。由于凭据事实与连接事实同行,被 resolver 拒绝的 settings 快照既不贡献自己的端点,也不贡献自己的密钥:整个先前世代继续服务。每个解析出的密钥在使用前都会被校验格式,因此 HTTP 标头无法承载的值会以 `LlmError('INVALID_CREDENTIAL')` 被拒绝,点名失败的入口,但绝不透露密钥的任何部分,而不是以语义不明的 `fetch` `TypeError` 形式浮现。任何地方都没有密钥的请求以 `MISSING_CREDENTIAL` 失败,并点名每个配置入口,同时路由保持注册、catalog 保持可浏览——首次运行的上手流程就是「浏览模型、存入密钥、再次发起提示」,中间无需任何重启。 -唯一在注册期捕获的事实是重试策略:其解析值变化时,插件原地重新注册该路由(同一适配器实例、一个同步区段),因此 `ctx.llm.providerRetryPolicy('deepseek-official')` 始终报告当前策略。 +适配器拥有的唯一注册事实是显式重试策略覆盖:其解析值变化或被移除时,插件原地重新注册该路由(同一适配器实例、一个同步区段)。因此,`ctx.llm.providerRetryPolicy('deepseek-official')` 会报告当前覆盖或 `LlmRuntime` 的部署默认值。 该插件还会在可配置提供方目录(`ctx.llm.listConfigurableProviders()`)中声明自己的路由:提供方为 `deepseek-official`,settings namespace 为 `llm-deepseek`,settings path 为空——整个分节就是 profile。配置界面借助该条目,把本适配器与休眠的 pi-ai 提供方一并呈现。 diff --git a/packages/llm/llm-deepseek/src/adapter.ts b/packages/llm/llm-deepseek/src/adapter.ts index bc30d46c3b..ca76130d5a 100644 --- a/packages/llm/llm-deepseek/src/adapter.ts +++ b/packages/llm/llm-deepseek/src/adapter.ts @@ -66,8 +66,8 @@ export interface DeepSeekConnectionOptions { models: readonly DeepSeekCatalogModel[] /** Maximum provider idle time while one stream read is outstanding. */ streamIdleTimeoutMs: number - /** Provider-owned model-request retry policy, already resolved. */ - retryPolicy: ResolvedRetryPolicy + /** Explicit provider-owned model-request retry policy, already resolved. */ + retryPolicy?: ResolvedRetryPolicy } /** Constructor options for {@link DeepSeekAdapter}: the operation-local resolution hooks the plugin owns. */ @@ -166,7 +166,7 @@ export class DeepSeekAdapter extends LlmAdapter { return { id: provider, name: 'DeepSeek' } } - override providerRetryPolicy(_provider: string): ResolvedRetryPolicy { + override providerRetryPolicy(_provider: string): ResolvedRetryPolicy | undefined { return this.config.options().retryPolicy } diff --git a/packages/llm/llm-deepseek/src/index.ts b/packages/llm/llm-deepseek/src/index.ts index a7b22ddd88..88a45ce165 100644 --- a/packages/llm/llm-deepseek/src/index.ts +++ b/packages/llm/llm-deepseek/src/index.ts @@ -76,7 +76,7 @@ export interface Config { models?: DeepSeekCatalogModel[] /** Maximum provider idle time while one stream read is outstanding (default five minutes). */ streamIdleTimeoutMs?: number - /** Provider-owned model-request retry policy; omission uses normal defaults. */ + /** Provider-owned model-request retry policy; omission inherits the LLM deployment default. */ retryPolicy?: RetryPolicyConfig } @@ -193,7 +193,9 @@ export function resolveAdapterOptions(config: Config, environment?: LaunchEnviro defaultContextWindow: config.defaultContextWindow ?? DEFAULT_CONTEXT_WINDOW, models: resolveModels(config.models), streamIdleTimeoutMs, - retryPolicy: resolveRetryPolicy(config.retryPolicy, 'llm-deepseek: retryPolicy'), + ...config.retryPolicy === undefined ? {} : { + retryPolicy: resolveRetryPolicy(config.retryPolicy, 'llm-deepseek: retryPolicy'), + }, } } diff --git a/packages/llm/llm-pi-ai/README.i18n.yaml b/packages/llm/llm-pi-ai/README.i18n.yaml index 7f81e5c853..7657430d3c 100644 --- a/packages/llm/llm-pi-ai/README.i18n.yaml +++ b/packages/llm/llm-pi-ai/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/llm/llm-pi-ai/README.md -README.md: d67db4014fabf0139df938c1a68132cadb7ad118 -README.zh.md: 12808defab58b651b5329d8bb19a15f67ea1963e +README.md: f7e9421d6c782e0419b0a1ba4db659b8b2adeaf2 +README.zh.md: 4a502a504fef2c02f4647ec1b9526e2054d2d53f diff --git a/packages/llm/llm-pi-ai/README.md b/packages/llm/llm-pi-ai/README.md index d67db4014f..f7e9421d6c 100644 --- a/packages/llm/llm-pi-ai/README.md +++ b/packages/llm/llm-pi-ai/README.md @@ -8,16 +8,12 @@ The package root exposes the Cordis plugin contract, `PiAiAdapter`, and `support ## Config -Configure credentials, the model catalog, and deployment-specific transport settings per provider, keyed by the provider route itself. An optional top-level `defaultRetryPolicy` applies to every profile that does not configure `retryPolicy`; when both are omitted, the core bounded normal defaults apply. `apiKeyEnv` is a credential *reference* resolved per request, so no secret enters this file. Omitting it leaves the route unauthenticated, which for an installed catalog route means pi-ai's provider-native ambient discovery; a configured reference that resolves to nothing fails the request with `MISSING_CREDENTIAL` instead, because falling through would authenticate with whatever unrelated key the environment happens to hold. One credential serves every model on its route. +Configure credentials, the model catalog, and deployment-specific transport settings per provider, keyed by the provider route itself. Each profile may set an explicit `retryPolicy`; omission inherits `LlmRuntime`'s deployment default. `apiKeyEnv` is a credential *reference* resolved per request, so no secret enters this file. Omitting it leaves the route unauthenticated, which for an installed catalog route means pi-ai's provider-native ambient discovery; a configured reference that resolves to nothing fails the request with `MISSING_CREDENTIAL` instead, because falling through would authenticate with whatever unrelated key the environment happens to hold. One credential serves every model on its route. ```yaml - id: llm name: '@deepseek-ai/dsh-llm-pi-ai' config: - # Deployment default inherited by profiles without retryPolicy. - defaultRetryPolicy: - mode: normal - maxRetries: 5 providers: # Catalog route: endpoint, protocol, and models all come from pi-ai. openai: @@ -107,7 +103,7 @@ Resolution still fails loud, naming the offending route and model, when a route ## Dynamic configuration (settings + credentials) -The adapter reads its profiles through a thunk **once per operation** instead of freezing them at construction. The plugin registers the `llm-pi-ai` namespace on the optional `ctx.settings` seam with this same `Config` schema and its `cordis.yml` entry as the composition `base`, and because `providers` is a dict, the base and the user's `llm-pi-ai:` settings section merge **per provider**: a user can add a route, override one field of a composition route, or point a route at another proxy, all effective on the next request with no restart. The top-level `defaultRetryPolicy` participates in the same layering; changing it changes every inheriting route's registration fact and atomically replaces the adapter registration, while a profile-level policy's resolved value remains unchanged. Without a mounted settings service the entry config alone drives the adapter, unchanged. +The adapter reads its profiles through a thunk **once per operation** instead of freezing them at construction. The plugin registers the `llm-pi-ai` namespace on the optional `ctx.settings` seam with this same `Config` schema and its `cordis.yml` entry as the composition `base`, and because `providers` is a dict, the base and the user's `llm-pi-ai:` settings section merge **per provider**: a user can add a route, override one field of a composition route, or point a route at another proxy, all effective on the next request with no restart. Without a mounted settings service the entry config alone drives the adapter, unchanged. Credentials resolve per stream call through `apiKeyEnv` and the optional `ctx.credentials` seam; without that seam, the adapter reads exactly the referenced environment variable. A profile naming no credential at all — and only that case — defers to pi-ai's ambient discovery. Every resolved key is trimmed and format-checked before use, so a value no HTTP header can carry is refused instead of surfacing as an opaque `fetch` `TypeError`; the refusal throws `LlmError('INVALID_CREDENTIAL')` naming the failing route and credential reference but never any part of the key. The route set and each route's captured retry policy are the registration-level facts: when either changes, the plugin replaces its registration atomically (same adapter instance, candidate set validated first), so a route another adapter already owns leaves the previous routes serving and reverting to a working configuration re-applies. Provider key order never counts as a change. A section this adapter could not serve is refused where it is written — the registered `validate` resolves the whole profile set, so `ctx.settings.mutate` rejects with the resolver's own error (the wire surface reports it as `settings-rejected`) and nothing is stored. A stored section that becomes unserviceable some other way — an external edit of `settings.yaml` — keeps the namespace's last good value at the settings seam and warns. The entry config itself still fails plugin load, and a route the llm registry refuses (one another adapter family already owns) is logged while the previously registered routes keep serving. @@ -117,7 +113,7 @@ A model that carries reasoning metadata — from the installed catalog or from i A model **without** that metadata — a hand-declared one whose entry declares no `reasoningEfforts`, and a catalog model pi-ai marks as non-reasoning — exposes no `reasoning` at all. pi-ai reports such a model as supporting the single level `off`, but `off` is translated to *omitting* the reasoning option, which is byte-for-byte the request that naming no effort already produces: selecting it could not disable anything, so a provider whose own default is to think would keep thinking with `off` shown as selected. Reporting the capability as unavailable leaves a surface offering the provider's default and nothing that misrepresents it. The profile `reasoning` value, including `off`, is the deployment default when configured; omitting it preserves the provider default. Per-request `GenerateOptions.reasoningEffort` takes precedence, and a level absent from the exact model capability fails the REQUEST with `UNSUPPORTED_REASONING_EFFORT` before network I/O instead of being clamped. Describing a model never fails that way: the models under one provider disagree about which levels they accept, so `resolveModel` reports a profile level the exact model cannot take as no default at all rather than throwing. A throw there would take the whole provider out of every model catalog built over it — one mis-set profile field hiding even the models that do support the level — so a bad configuration surfaces where it is acted on, not where it is described. pi-ai's common stream options represent `off` by omitting `reasoning`. -The top-level configuration supports `defaultRetryPolicy`. Supported profile fields are `apiKeyEnv`, `displayName`, `api`, `baseURL`, `models`, `modelOverrides`, `compat`, `defaultContextWindow`, `defaultMaxTokens`, `defaultInput`, `headers`, `reasoning`, `thinkingBudgets`, `cacheRetention`, `transport`, `timeoutMs`, `websocketConnectTimeoutMs`, `streamIdleTimeoutMs`, and `retryPolicy`. Each profile's resolved retry policy is captured with that provider route: its own policy wins, then the adapter default, then the bounded normal omission defaults. The stream-idle interval is a positive finite Node timer delay, defaults to five minutes, and covers only an outstanding provider read, not consumer think time. Harness app attribution wins a conflicting configured header name. +Supported profile fields are `apiKeyEnv`, `displayName`, `api`, `baseURL`, `models`, `modelOverrides`, `compat`, `defaultContextWindow`, `defaultMaxTokens`, `defaultInput`, `headers`, `reasoning`, `thinkingBudgets`, `cacheRetention`, `transport`, `timeoutMs`, `websocketConnectTimeoutMs`, `streamIdleTimeoutMs`, and `retryPolicy`. An explicit profile retry policy is captured with that provider route; omission inherits the LLM deployment default, which itself falls back to bounded normal behavior. The stream-idle interval is a positive finite Node timer delay, defaults to five minutes, and covers only an outstanding provider read, not consumer think time. Harness app attribution wins a conflicting configured header name. The adapter forces pi-ai's SDK `maxRetries` to zero so one `stream()` call makes one provider request. The removed profile fields `maxRetries` and `maxRetryDelayMs` fail load instead of silently multiplying or hiding the separately composed agent-level retry budget. Idle expiry aborts the SDK's stable request signal and surfaces `TIMEOUT`; an earlier caller abort remains `ABORTED`. @@ -205,4 +201,4 @@ Recorded response content appends to the next request and does not invalidate it - **`GenerateOptions.stop` is unsupported** — pi-ai's common stream options cannot guarantee stop-sequence behavior across providers, so the adapter rejects the field. - **In-history `system` messages use pi-ai's common context conversion** — provider-specific placement follows pi-ai rather than a harness-owned wire override. - **Provider HTTP status is unavailable** — pi-ai error events do not expose a stable HTTP status across providers; failures expose only stable harness error codes. -- **Retry policy is provider-owned, not an SDK retry** — the adapter may supply `defaultRetryPolicy`, and each provider profile may override it with nested `retryPolicy`; the resolved route policy is what `dsh-llm-retry` executes at the agent failed-step extension point. pi-ai SDK retries stay disabled so durable agent steps and `llm/retry` events own every visible attempt, and direct `ctx.llm.stream()` calls remain single-attempt. +- **Retry policy is provider-owned, not an SDK retry** — each provider profile may supply nested `retryPolicy`; omission inherits `LlmRuntime`'s deployment default, and the effective route policy is what `dsh-llm-retry` executes at the agent failed-step extension point. pi-ai SDK retries stay disabled so durable agent steps and `llm/retry` events own every visible attempt, and direct `ctx.llm.stream()` calls remain single-attempt. diff --git a/packages/llm/llm-pi-ai/README.zh.md b/packages/llm/llm-pi-ai/README.zh.md index 12808defab..4a502a504f 100644 --- a/packages/llm/llm-pi-ai/README.zh.md +++ b/packages/llm/llm-pi-ai/README.zh.md @@ -8,16 +8,12 @@ ## 配置 -按提供方配置凭据、模型 catalog 与部署特定传输设置,并以提供方路由本身为键。可选的顶层 `defaultRetryPolicy` 会应用于每个未配置 `retryPolicy` 的 profile;两者都省略时使用核心层有界的 normal 默认值。`apiKeyEnv` 是按请求解析的凭据*引用*,因此机密不进入该文件。省略它会让该路由处于未认证状态;对已安装 catalog 路由而言,这意味着交给 pi-ai 的提供方原生环境发现。已配置却解析不出任何值的引用则相反,会让请求以 `MISSING_CREDENTIAL` 失败,因为放行下去就会用环境里恰好持有的某个无关密钥完成认证。一条凭据服务该路由下的全部模型。 +按提供方配置凭据、模型 catalog 与部署特定传输设置,并以提供方路由本身为键。每个 profile 都可以设置显式 `retryPolicy`;省略时继承 `LlmRuntime` 的部署默认值。`apiKeyEnv` 是按请求解析的凭据*引用*,因此机密不进入该文件。省略它会让该路由处于未认证状态;对已安装 catalog 路由而言,这意味着交给 pi-ai 的提供方原生环境发现。已配置却解析不出任何值的引用则相反,会让请求以 `MISSING_CREDENTIAL` 失败,因为放行下去就会用环境里恰好持有的某个无关密钥完成认证。一条凭据服务该路由下的全部模型。 ```yaml - id: llm name: '@deepseek-ai/dsh-llm-pi-ai' config: - # Deployment default inherited by profiles without retryPolicy. - defaultRetryPolicy: - mode: normal - maxRetries: 5 providers: # Catalog route: endpoint, protocol, and models all come from pi-ai. openai: @@ -108,7 +104,7 @@ profile 的 `models` 列表是*替换*该路由已安装 catalog,而不是扩 ## 动态配置(settings + credentials) -适配器经由一个 thunk **每操作读取一次** profile,而非在构造期冻结。插件在可选的 `ctx.settings` seam 上用同一份 `Config` schema 注册 `llm-pi-ai` namespace,并以其 `cordis.yml` 条目为组合 `base`;由于 `providers` 是字典,base 与用户的 `llm-pi-ai:` settings 分节**按提供方**合并:用户可以新增路由、覆盖组合路由的单个字段,或把路由指向另一个 proxy,全部在下一次请求生效,无需重启。顶层 `defaultRetryPolicy` 参与同一套分层;修改它会改变每条继承路由的注册事实,并原子地替换适配器注册,而 profile 级策略的解析值保持不变。未挂载 settings 服务时,仅由 entry 配置驱动适配器,行为不变。 +适配器经由一个 thunk **每操作读取一次** profile,而非在构造期冻结。插件在可选的 `ctx.settings` seam 上用同一份 `Config` schema 注册 `llm-pi-ai` namespace,并以其 `cordis.yml` 条目为组合 `base`;由于 `providers` 是字典,base 与用户的 `llm-pi-ai:` settings 分节**按提供方**合并:用户可以新增路由、覆盖组合路由的单个字段,或把路由指向另一个 proxy,全部在下一次请求生效,无需重启。未挂载 settings 服务时,仅由 entry 配置驱动适配器,行为不变。 凭据在每次流调用时通过 `apiKeyEnv` 与可选的 `ctx.credentials` seam 解析;未挂载该 seam 时,适配器只读取该引用指向的环境变量。只有完全没有点名任何凭据的 profile——仅限这一种情况——才交给 pi-ai 的环境发现。每个解析出的密钥都会在使用前去除首尾空白并校验格式,因此 HTTP 标头无法承载的值会被拒绝,而不是以语义不明的 `fetch` `TypeError` 形式浮现;这种拒绝会抛出 `LlmError('INVALID_CREDENTIAL')`,点名失败的路由与凭据引用,但绝不透露密钥的任何部分。路由集合与每条路由捕获的重试策略是注册级事实:两者任一变化时,插件都会原子地替换自己的注册(同一适配器实例,候选集合先经校验),因此某条路由若已被另一适配器占有,先前的路由会继续服务,而改回可用配置时注册会重新生效。提供方键的顺序绝不算作变化。本适配器无法服务的分节会在写入处被拒——注册的 `validate` 会解析整份 profile 集合,因此 `ctx.settings.mutate` 以 resolver 自身的错误拒绝(该协议将其报为 `settings-rejected`),什么都不会存储。已存储分节若因其他途径变得不可服务——比如外部编辑了 `settings.yaml`——则由 settings seam 保留该 namespace 最后可用的值并告警。entry 配置本身仍会使插件加载失败;而 llm 注册表拒绝的路由(已被另一适配器族占有的那种)会被记录下来,先前注册的路由继续服务。 @@ -118,7 +114,7 @@ profile 的 `models` 列表是*替换*该路由已安装 catalog,而不是扩 **没有**这份元数据的模型——条目未声明 `reasoningEfforts` 的手工声明模型,以及 pi-ai 标记为不具备推理能力的 catalog 模型——完全不公开 `reasoning`。pi-ai 会把这类模型报告为只支持 `off` 一档,但 `off` 会被翻译成*省略* reasoning 选项,而那与「不点名任何档位」产出的请求逐字节相同:选它关不掉任何东西,于是自身默认就在思考的提供方,会在界面显示 `off` 被选中的同时继续思考。把该能力报告为不可用,界面就只剩提供方默认这一项,不会再出现自相矛盾的控件。配置 profile 的 `reasoning` 值(包括 `off`)在存在时是部署默认值;省略它会保留提供方默认值。每次请求的 `GenerateOptions.reasoningEffort` 优先;未出现在确切模型能力中的档位会让**请求**在网络 I/O 前以 `UNSUPPORTED_REASONING_EFFORT` 失败,而不会被自动调整。**描述**一个模型则从不这样失败:同一提供方下各模型接受的档位并不一致,因此 `resolveModel` 对该模型拿不下的 profile 档位报告为「没有默认值」,而不是抛错。在那里抛错会让整个提供方从任何基于它构建的模型目录中消失——一个配错的 profile 字段连支持该档位的模型也一并藏起来——所以坏配置暴露在被执行处,而不是被描述处。pi-ai 的通用流选项通过省略 `reasoning` 表示 `off`。 -顶层配置支持 `defaultRetryPolicy`。受支持的 profile 字段是 `apiKeyEnv`、`displayName`、`api`、`baseURL`、`models`、`modelOverrides`、`compat`、`defaultContextWindow`、`defaultMaxTokens`、`defaultInput`、`headers`、`reasoning`、`thinkingBudgets`、`cacheRetention`、`transport`、`timeoutMs`、`websocketConnectTimeoutMs`、`streamIdleTimeoutMs` 和 `retryPolicy`。每个 profile 解析后的重试策略都会与该提供方路由一同捕获:自身策略优先,其次是适配器默认值,最后是有界的 normal 省略默认值。流空闲间隔必须是正的有限 Node 定时器延迟,默认为五分钟,且只覆盖未完成提供方读取,不包括消费方思考时间。若已配置标头中有同名项,则以 Harness 应用归因为准。 +受支持的 profile 字段是 `apiKeyEnv`、`displayName`、`api`、`baseURL`、`models`、`modelOverrides`、`compat`、`defaultContextWindow`、`defaultMaxTokens`、`defaultInput`、`headers`、`reasoning`、`thinkingBudgets`、`cacheRetention`、`transport`、`timeoutMs`、`websocketConnectTimeoutMs`、`streamIdleTimeoutMs` 和 `retryPolicy`。显式 profile 重试策略会随该提供方路由一同捕获;省略时继承 LLM 部署默认值,而该默认值自身会回退到有界 normal 行为。流空闲间隔必须是正的有限 Node 定时器延迟,默认为五分钟,且只覆盖未完成提供方读取,不包括消费方思考时间。若已配置标头中有同名项,则以 Harness 应用归因为准。 适配器强制 pi-ai SDK `maxRetries` 为零,因此一次 `stream()` 调用只会发起一次提供方请求。已移除 profile 字段 `maxRetries` 和 `maxRetryDelayMs` 会使加载失败,而不是静默倍增或隐藏单独组合的 agent(智能体)级重试预算。空闲超时会 abort SDK 的稳定请求信号,并以 `TIMEOUT` 呈现;较早的调用方 abort 仍为 `ABORTED`。 @@ -206,4 +202,4 @@ pi-ai 事件会变为 harness 推理、文本、工具调用、usage 与 finish - **不支持 `GenerateOptions.stop`**:pi-ai 的通用流选项无法保证所有提供方都支持 stop sequence,因此适配器会拒绝该字段。 - **历史中的 `system` 消息使用 pi-ai 通用上下文转换**:提供方特定位置由 pi-ai 决定,而非由 harness 拥有的协议覆盖决定。 - **无法获取提供方 HTTP 状态**:pi-ai 错误事件不会在所有提供方上公开稳定 HTTP 状态;失败只公开稳定 harness 错误 code。 -- **重试策略由提供方持有,而不是 SDK 重试**:适配器可以提供 `defaultRetryPolicy`,每个提供方 profile 也可以用嵌套的 `retryPolicy` 覆盖它;`dsh-llm-retry` 在 agent 的失败步骤扩展点上执行解析后的路由策略。pi-ai SDK 重试仍保持禁用,因此持久化的 agent 步骤与 `llm/retry` 事件记录每次可见尝试,直接 `ctx.llm.stream()` 调用仍只尝试一次。 +- **重试策略由提供方持有,而不是 SDK 重试**:每个提供方 profile 都可以提供嵌套的 `retryPolicy`;省略时继承 `LlmRuntime` 的部署默认值,`dsh-llm-retry` 会在 agent 的失败步骤扩展点上执行有效路由策略。pi-ai SDK 重试仍保持禁用,因此持久化的 agent 步骤与 `llm/retry` 事件记录每次可见尝试,直接 `ctx.llm.stream()` 调用仍只尝试一次。 diff --git a/packages/llm/llm-pi-ai/src/config.ts b/packages/llm/llm-pi-ai/src/config.ts index 75f32c8f4a..0641649c08 100644 --- a/packages/llm/llm-pi-ai/src/config.ts +++ b/packages/llm/llm-pi-ai/src/config.ts @@ -1,9 +1,7 @@ /** * Configuration schema and provider-profile validation for the pi-ai adapter. * Profiles are a dict keyed by provider route, so the composition base and a - * user-settings layer merge per provider and the route set is structural. An - * adapter-level retry default lets a deployment choose one policy for every - * route that does not override it without changing the core omission default. + * user-settings layer merge per provider and the route set is structural. * * A route key is not required to name an installed pi-ai provider. When it does, * that provider's endpoint, protocol, display name, and model catalog are the @@ -138,7 +136,7 @@ export interface PiAiProviderProfile { websocketConnectTimeoutMs?: number /** Maximum provider idle time while one stream read is outstanding. */ streamIdleTimeoutMs?: number - /** Provider-owned model-request retry policy; omission inherits the adapter default, then normal defaults. */ + /** Provider-owned model-request retry policy; omission inherits the LLM deployment default. */ retryPolicy?: RetryPolicyConfig } @@ -153,8 +151,8 @@ export interface ResolvedPiAiProviderProfile apiKeyEnv?: CredentialRef /** Positive finite provider-idle interval after defaulting. */ streamIdleTimeoutMs: number - /** Immutable retry policy captured with this provider route. */ - retryPolicy: ResolvedRetryPolicy + /** Explicit immutable retry policy captured with this provider route. */ + retryPolicy?: ResolvedRetryPolicy /** * The pi-ai provider this route registers, built from the resolved models. * Construction happens here so an unserviceable protocol or an underspecified @@ -170,13 +168,8 @@ export interface ResolvedPiAiProviderProfile configuredMaxTokens: ReadonlyMap } -/** Plugin configuration: the provider routes this instance owns and their shared defaults. */ +/** Plugin configuration: the provider routes this instance owns. */ export interface Config { - /** - * Retry policy inherited by every provider profile that omits its own - * `retryPolicy`; omission here uses the bounded normal defaults. - */ - defaultRetryPolicy?: RetryPolicyConfig /** * pi-ai provider routes, keyed by provider. An empty (or omitted) dict is * the dormant settings-driven posture: the adapter mounts with no routes @@ -260,7 +253,6 @@ const profile = z.object({ /** Runtime schema for {@link Config}. */ export const Config: z = z.object({ - defaultRetryPolicy: RetryPolicySchema, providers: z.dict(profile).default({}), }) @@ -277,7 +269,7 @@ export const Config: z = z.object({ * @throws Error naming the route and model that cannot be served. */ export function assertServiceable(config: Config): void { - resolveProfiles(config.providers, config.defaultRetryPolicy) + resolveProfiles(config.providers) } /** Reject removed pre-release profile fields and name their replacements. */ @@ -302,23 +294,17 @@ function rejectRemovedFields(provider: string, source: PiAiProviderProfile): voi * Validate profiles and return a detached route-keyed map suitable for * per-request reads. This is the one explicit resolve step, so an omitted dict * resolves to the empty (dormant) route set here rather than through a hidden - * fallback, and each route's models, retry policy, and pi-ai provider are - * materialized once. + * fallback, and each route's models, explicit retry policy, and pi-ai provider + * are materialized once. * @param providers - configured provider profiles keyed by route. - * @param defaultRetryPolicy - adapter policy inherited by profiles that omit one. * @returns validated profiles in configuration order. */ export function resolveProfiles( providers: Readonly> | undefined, - defaultRetryPolicy?: RetryPolicyConfig, ): Map { if (Array.isArray(providers)) { throw new Error('llm-pi-ai: providers is now a dict keyed by provider route, not an array of profiles') } - const resolvedDefaultRetryPolicy = resolveRetryPolicy( - defaultRetryPolicy, - 'llm-pi-ai: defaultRetryPolicy', - ) const entries = Object.entries(providers ?? {}) const resolved = new Map() for (const [provider, source] of entries) { @@ -369,9 +355,9 @@ export function resolveProfiles( displayName, ...apiKeyEnv === undefined ? {} : { apiKeyEnv: credentialRef(apiKeyEnv) }, streamIdleTimeoutMs, - retryPolicy: retryPolicy === undefined - ? resolvedDefaultRetryPolicy - : resolveRetryPolicy(retryPolicy, `llm-pi-ai: provider "${provider}" retryPolicy`), + ...retryPolicy === undefined ? {} : { + retryPolicy: resolveRetryPolicy(retryPolicy, `llm-pi-ai: provider "${provider}" retryPolicy`), + }, ...rest.headers === undefined ? {} : { headers: { ...rest.headers } }, ...rest.thinkingBudgets === undefined ? {} : { thinkingBudgets: { ...rest.thinkingBudgets } }, configuredMaxTokens: catalog.configuredMaxTokens, diff --git a/packages/llm/llm-pi-ai/src/index.ts b/packages/llm/llm-pi-ai/src/index.ts index 956a03fddd..23133ee44b 100644 --- a/packages/llm/llm-pi-ai/src/index.ts +++ b/packages/llm/llm-pi-ai/src/index.ts @@ -5,7 +5,7 @@ * pi-ai does not ship is declared outright. Profile facts resolve per request * over the optional `llm-pi-ai` user-settings section and the optional * credential seam, so a changed key, endpoint, model, or knob reaches the next - * request without a restart; a changed *route set* (or a route's resolved, + * request without a restart; a changed *route set* (or a route's explicit, * registration-captured retry policy) re-registers the same adapter instance * in place. * @@ -13,10 +13,6 @@ * - id: llm * name: '@deepseek-ai/dsh-llm-pi-ai' * config: - * # Optional deployment default; a provider profile may override it. - * defaultRetryPolicy: - * mode: normal - * maxRetries: 5 * providers: * # Catalog route: everything but the credential comes from pi-ai. * openai: @@ -103,7 +99,7 @@ function registrationFacts(profiles: ReadonlyMap ({ provider, displayName: profile.displayName, - retryPolicy: profile.retryPolicy, + ...profile.retryPolicy === undefined ? {} : { retryPolicy: profile.retryPolicy }, })) .sort((left, right) => left.provider.localeCompare(right.provider)) } @@ -169,7 +165,7 @@ export function apply(ctx: Context, config: Config): void { const profiles = (): ReadonlyMap => { const raw = current() if (raw === lastRaw && memoized !== undefined) return memoized - const next = resolveProfiles(raw.providers, raw.defaultRetryPolicy) + const next = resolveProfiles(raw.providers) lastRaw = raw memoized = next return next diff --git a/packages/llm/llm-pi-ai/tests/adapter.spec.ts b/packages/llm/llm-pi-ai/tests/adapter.spec.ts index bfc186f10a..51ce347e9a 100644 --- a/packages/llm/llm-pi-ai/tests/adapter.spec.ts +++ b/packages/llm/llm-pi-ai/tests/adapter.spec.ts @@ -752,27 +752,6 @@ describe('provider profile lifecycle', () => { expect(ctx.llm.listProviders()).toEqual([]) }) - it('inherits the adapter retry default unless a provider profile overrides it', () => { - const profiles = resolveProfiles({ - openai: {}, - anthropic: { retryPolicy: { mode: 'normal', maxRetries: 1 } }, - }, { mode: 'normal', maxRetries: 5 }) - - expect(profiles.get('openai')?.retryPolicy).toMatchObject({ mode: 'normal', maxRetries: 5 }) - expect(profiles.get('anthropic')?.retryPolicy).toMatchObject({ mode: 'normal', maxRetries: 1 }) - }) - - it('rejects an invalid adapter retry default even while dormant', async () => { - const invalid = { mode: 'normal', maxRetries: -1 } as const - expect(() => resolveProfiles(undefined, invalid)).toThrow(/defaultRetryPolicy/) - - const ctx = new Context() - await ctx.plugin(LlmRuntime) - await expect(ctx.plugin(LlmPiAi, { defaultRetryPolicy: invalid })) - .rejects.toThrow(/defaultRetryPolicy|retryPolicy/) - expect(ctx.llm.listProviders()).toEqual([]) - }) - it('constructs the adapter directly and rejects routes it does not own', async () => { const adapter = adapterOf({ openai: {} }) await expect(adapter.listModels('anthropic')).rejects.toMatchObject({ code: 'NO_ADAPTER' }) diff --git a/packages/llm/llm-pi-ai/tests/dynamic-config.spec.ts b/packages/llm/llm-pi-ai/tests/dynamic-config.spec.ts index 0f90ad8802..1019354156 100644 --- a/packages/llm/llm-pi-ai/tests/dynamic-config.spec.ts +++ b/packages/llm/llm-pi-ai/tests/dynamic-config.spec.ts @@ -156,26 +156,6 @@ describe('request-level dynamic profiles', () => { expect(ctx.llm.listProviders().map(provider => provider.id)).toEqual(['openai']) }) - it('re-registers inherited policies when the adapter retry default changes', async () => { - const dir = await home() - const ctx = await boot(dir, { - defaultRetryPolicy: { mode: 'normal', maxRetries: 5 }, - providers: { - openai: {}, - anthropic: { retryPolicy: { mode: 'normal', maxRetries: 1 } }, - }, - }) - - expect(ctx.llm.providerRetryPolicy('openai')).toMatchObject({ mode: 'normal', maxRetries: 5 }) - expect(ctx.llm.providerRetryPolicy('anthropic')).toMatchObject({ mode: 'normal', maxRetries: 1 }) - - await ctx.settings.update(NS, { - defaultRetryPolicy: { mode: 'normal', maxRetries: 4 }, - }) - expect(ctx.llm.providerRetryPolicy('openai')).toMatchObject({ mode: 'normal', maxRetries: 4 }) - expect(ctx.llm.providerRetryPolicy('anthropic')).toMatchObject({ mode: 'normal', maxRetries: 1 }) - }) - it('refuses a settings write this adapter could not serve, leaving its routes alone', async () => { const dir = await home() const ctx = await boot(dir, { providers: { openai: {} } }) diff --git a/packages/llm/llm/README.i18n.yaml b/packages/llm/llm/README.i18n.yaml index fce8fa059b..1df7d050e4 100644 --- a/packages/llm/llm/README.i18n.yaml +++ b/packages/llm/llm/README.i18n.yaml @@ -2,5 +2,5 @@ # side as of the last confirmed-consistent state. Both languages carry equal authority; # after editing either side, bring the other along and re-record with: # pnpm run verify-translation-pairing --write packages/llm/llm/README.md -README.md: fb6bd84240b41dd730d45b3eb34c35827dc4c991 -README.zh.md: 5c22767a7c654972cbf614505382fa4755d7d318 +README.md: 91d4b745c2e179c1d943628ea7c4337cb48291f1 +README.zh.md: f9492e6cd3edb5b6b969cfde0fb8a759f95ae77c diff --git a/packages/llm/llm/README.md b/packages/llm/llm/README.md index fb6bd84240..91d4b745c2 100644 --- a/packages/llm/llm/README.md +++ b/packages/llm/llm/README.md @@ -8,6 +8,10 @@ Provider-neutral LLM vocabulary and abstract service. This package defines the c An adapter registry plus a single streaming call API, interceptable via a waterfall event. +### Configuration + +`defaultRetryPolicy` is the deployment policy inherited by every provider route whose adapter supplies no explicit override. Omitting it preserves the bounded normal default of two retries. An adapter override wins, and the effective resolved policy is captured with the route registration; this service stores that policy but does not execute retries. + ### Public API - `ctx.llm.registerAdapter(providers: string[], adapter: LlmAdapter): AdapterRegistrationHandle` Register one adapter instance for the given provider routes. Registration is all-or-nothing, and is disposed with the calling fiber. The returned disposer also carries `replace(providers)`: the candidate route set is validated in full before anything moves, so a conflict with another adapter leaves the current routes registered and serving, and the swap itself is one synchronous section with no observable gap. `replace([])` is legal — a registration holding zero routes — unlike an empty initial registration. @@ -17,7 +21,7 @@ An adapter registry plus a single streaming call API, interceptable via a waterf - `ctx.llm.registerModelDiscovery(settingsNs: string, discover): () => void` Offer to interrogate provider endpoints for the settings namespace this plugin owns. One offer per namespace (`INVALID_DISCOVERY`/`DUPLICATE_DISCOVERY`), disposed with the calling fiber. - `ctx.llm.listModelDiscoveryNamespaces(): string[]` List the namespaces that can interrogate an endpoint, so a surface offers the action only where it works. - `ctx.llm.discoverModels(settingsNs: string, request: LlmModelDiscoveryRequest): Promise` Ask one endpoint which models it advertises. -- `ctx.llm.providerRetryPolicy(provider: string): ResolvedRetryPolicy` Return the provider-owned retry policy captured during registration, with normal defaults resolved. +- `ctx.llm.providerRetryPolicy(provider: string): ResolvedRetryPolicy` Return the effective retry policy captured during registration: the adapter override when present, otherwise the deployment default. - `ctx.llm.listModels(provider: string): Promise` Discover the models one registered provider currently advertises. - `ctx.llm.resolveModelInfo(provider: string, model: string, signal?: AbortSignal): Promise` Resolve validated exact-model identity plus available context, output-default, and reasoning metadata from the owning adapter, with optional cancellation for asynchronous adapters. - `ctx.llm.resolveCallConfig(config: LlmCallConfig, signal?: AbortSignal): Promise` Validate an explicit effort and materialize adapter-configured call defaults without clamping. @@ -28,7 +32,7 @@ An adapter registry plus a single streaming call API, interceptable via a waterf Interrogating an endpoint is configuration-time work over a *draft*, keyed by settings namespace rather than by provider route — the provider a surface is adding does not exist yet, so there is no route to name. The request may still *name* a route it is editing, and an adapter that already describes that route answers from its own knowledge without a network call; `baseURL` is optional and one of the two is required. The request otherwise carries the endpoint, the protocol, and a credential the harness uses for that one interrogation and never stores — nothing here reads or writes settings or credentials, and the reply is candidate metadata a surface may offer for adoption, never a registered catalog. `LlmDiscoveredModel` makes every field but `id` optional because most provider listings disclose an id and nothing else; a surface adopting one still owes the capacities its adapter requires. Duplicate and unusable ids are dropped, an unserved namespace fails with `NO_DISCOVERY`, and a request naming neither a route nor an endpoint fails with `INVALID_DISCOVERY`. -Provider and model metadata is a discovery surface, not a routing whitelist. `registerAdapter()` still owns provider exclusivity and captures the adapter's retry policy for each route, while an adapter may accept model ids absent from `listModels()`; consumers must not reject a request because its model is unlisted. Returned selector metadata is detached and invalid or duplicate adapter entries fail with `INVALID_ADAPTER` or `INVALID_CATALOG`. +Provider and model metadata is a discovery surface, not a routing whitelist. `registerAdapter()` still owns provider exclusivity and captures each route's effective retry policy, while an adapter may accept model ids absent from `listModels()`; consumers must not reject a request because its model is unlisted. Returned selector metadata is detached and invalid or duplicate adapter entries fail with `INVALID_ADAPTER` or `INVALID_CATALOG`. Every topology commit point — adapter routes registering or disposing, directory entries appearing or withdrawing — emits the payload-free `llm/adapters-updated` event after the mutation, so consumers re-read `listProviders()`/`listModels()`/`listConfigurableProviders()` instead of polling. Observer failures are contained (logged, non-vetoing); only `INVARIANT`-coded failures rethrow after the fan-out. @@ -44,7 +48,7 @@ Exact-model metadata is a separate correctness query, not a catalog decoration o ### Extension points -- Subclass `LlmAdapter` and call `ctx.llm.registerAdapter(providers, adapter)` to add one or more provider routes. `GenerateOptions.provider` selects the adapter; `GenerateOptions.model` is adapter-owned and may be resolved dynamically. Override `providerRetryPolicy()` to supply provider-owned recovery configuration, `providerInfo()` and asynchronous `listModels()` to expose selector metadata, then implement `resolveModel()` when exact identity, capacity, an output default, or selectable reasoning efforts are available; an asynchronous resolver must honor its optional cancellation signal. The defaults use bounded normal retry policy, use the route and model ids as names, advertise no models, and return no capacity, output default, or reasoning metadata. +- Subclass `LlmAdapter` and call `ctx.llm.registerAdapter(providers, adapter)` to add one or more provider routes. `GenerateOptions.provider` selects the adapter; `GenerateOptions.model` is adapter-owned and may be resolved dynamically. Override `providerRetryPolicy()` only for an explicit provider-owned recovery policy; omission inherits `LlmRuntime`'s deployment default. Override `providerInfo()` and asynchronous `listModels()` to expose selector metadata, then implement `resolveModel()` when exact identity, capacity, an output default, or selectable reasoning efforts are available; an asynchronous resolver must honor its optional cancellation signal. The defaults use the route and model ids as names, advertise no models, and return no capacity, output default, or reasoning metadata. - Wrap `llm/stream` via `ctx.on()` waterfall listeners for caching, logging, or routing. A wrapper that retries after emitting a chunk has no durable attempt boundary; shipped agent retry policy therefore uses `agent/request-error` instead. ### Messages (`message.ts`) and content blocks (`types.ts`) diff --git a/packages/llm/llm/README.zh.md b/packages/llm/llm/README.zh.md index 5c22767a7c..f9492e6cd3 100644 --- a/packages/llm/llm/README.zh.md +++ b/packages/llm/llm/README.zh.md @@ -8,6 +8,10 @@ 一个适配器注册表加单一流式调用接口,可通过 waterfall(瀑布式事件)拦截。 +### 配置 + +`defaultRetryPolicy` 是每条未由适配器提供显式覆盖的提供方路由所继承的部署策略。省略它会保留两次重试的有界 normal 默认值。适配器覆盖优先,解析后的有效策略会随路由注册一同捕获;本服务存储该策略,但不执行重试。 + ### 公开 API - `ctx.llm.registerAdapter(providers: string[], adapter: LlmAdapter): AdapterRegistrationHandle` 为给定提供方路由注册一个适配器实例。注册要么全部成功,要么全部不生效,并且会随调用 fiber 一起 dispose(资源释放)。返回的句柄还提供 `replace(providers)`:候选路由集合会在注册状态发生任何变化前完成整体验证,因此与另一适配器发生冲突时,当前路由仍保持注册并继续提供服务。替换会在一次同步操作中完成,不会出现可观察的空档。`replace([])` 合法,表示保留注册但不持有任何路由;初始注册则不得为空。 @@ -17,7 +21,7 @@ - `ctx.llm.registerModelDiscovery(settingsNs: string, discover): () => void` 为本插件拥有的 settings namespace 提供查询提供方端点的能力。每个 namespace 只能有一个(`INVALID_DISCOVERY`/`DUPLICATE_DISCOVERY`),并随调用 fiber dispose。 - `ctx.llm.listModelDiscoveryNamespaces(): string[]` 列出可以询问端点的 namespace,让界面只在可用之处提供该动作。 - `ctx.llm.discoverModels(settingsNs: string, request: LlmModelDiscoveryRequest): Promise` 询问某个端点它公布了哪些模型。 -- `ctx.llm.providerRetryPolicy(provider: string): ResolvedRetryPolicy` 返回注册时捕获的提供方自身的重试策略,并解析 normal 默认值。 +- `ctx.llm.providerRetryPolicy(provider: string): ResolvedRetryPolicy` 返回注册时捕获的有效重试策略:有适配器覆盖时使用该覆盖,否则使用部署默认值。 - `ctx.llm.listModels(provider: string): Promise` 发现某个已注册提供方当前公布的模型。 - `ctx.llm.resolveModelInfo(provider: string, model: string, signal?: AbortSignal): Promise` 从拥有该精确路由的适配器中,解析并校验确切模型身份,以及可用上下文、输出默认值和推理(reasoning)元数据;异步适配器可选地支持取消。 - `ctx.llm.resolveCallConfig(config: LlmCallConfig, signal?: AbortSignal): Promise` 校验显式推理强度,并填入适配器配置的调用默认值,但不自动调整。 @@ -28,7 +32,7 @@ 询问端点属于配置期针对**草稿**的操作,以 settings namespace 而非提供方路由为键——界面正在新增的提供方还不存在,也就没有路由可点名。但请求仍可**点名**它正在编辑的路由,而已经描述该路由的适配器会用自己的知识作答,无需联网;路由名称和 `baseURL` 至少需要提供一项。除此之外,请求携带端点、协议,以及一条 harness 只用于这一次询问、绝不存储的凭据。这里既不读取也不写入 settings 或 credentials;返回内容是界面可以提供给用户采纳的候选元数据,而不是已注册的 catalog。`LlmDiscoveredModel` 除 `id` 外每个字段都是可选的,因为大多数提供方列表只公布 id;采纳其中一条的界面仍要补上其适配器所需的容量。重复与不可用的 id 会被丢弃,无人服务的 namespace 以 `NO_DISCOVERY` 失败,既不点名路由也不给端点的请求以 `INVALID_DISCOVERY` 失败。 -提供方和模型元数据用于发现,不构成路由白名单。`registerAdapter()` 仍拥有提供方路由的排他性,并为每条路由捕获适配器的重试策略;适配器可以接受未出现在 `listModels()` 中的模型 id,消费方不得仅因模型未列出而拒绝请求。返回的 selector 元数据已分离;无效或重复的适配器条目会以 `INVALID_ADAPTER` 或 `INVALID_CATALOG` 失败。 +提供方和模型元数据用于发现,不构成路由白名单。`registerAdapter()` 仍拥有提供方路由的排他性,并为每条路由捕获有效重试策略;适配器可以接受未出现在 `listModels()` 中的模型 id,消费方不得仅因模型未列出而拒绝请求。返回的 selector 元数据已分离;无效或重复的适配器条目会以 `INVALID_ADAPTER` 或 `INVALID_CATALOG` 失败。 每个拓扑提交点——适配器路由注册或 dispose、目录条目出现或撤回——都会在变更之后发出无载荷的 `llm/adapters-updated` 事件,消费方因此会重新读取 `listProviders()`/`listModels()`/`listConfigurableProviders()`,而不是轮询。观察者故障会被记录并隔离,不能否决变更;只有带 `INVARIANT` 码的故障会在通知完所有观察者后重新抛出。 @@ -44,7 +48,7 @@ ### 扩展点 -- 继承 `LlmAdapter` 并调用 `ctx.llm.registerAdapter(providers, adapter)`,添加一条或多条提供方路由。`GenerateOptions.provider` 选择适配器;`GenerateOptions.model` 属于适配器,可以动态解析。覆盖 `providerRetryPolicy()` 以提供由提供方定义的恢复配置,覆盖 `providerInfo()` 和异步 `listModels()` 以公开 selector 元数据;精确身份、容量、输出默认值或可选推理强度可用时,实现 `resolveModel()`;异步解析器必须响应其可选的取消 signal。默认实现使用有界的 normal 重试策略,将路由和模型 id 用作名称,不公布模型,也不返回容量、输出默认值或推理元数据。 +- 继承 `LlmAdapter` 并调用 `ctx.llm.registerAdapter(providers, adapter)`,添加一条或多条提供方路由。`GenerateOptions.provider` 选择适配器;`GenerateOptions.model` 属于适配器,可以动态解析。仅在提供方拥有显式恢复策略时覆盖 `providerRetryPolicy()`;省略时继承 `LlmRuntime` 的部署默认值。覆盖 `providerInfo()` 和异步 `listModels()` 以公开 selector 元数据;精确身份、容量、输出默认值或可选推理强度可用时,实现 `resolveModel()`;异步解析器必须响应其可选的取消 signal。默认实现将路由和模型 id 用作名称,不公布模型,也不返回容量、输出默认值或推理元数据。 - 包装 `llm/stream` 时,通过 `ctx.on()` waterfall listener 实现缓存、日志或路由。包装层如果在已经发出分片后重试,就没有可持久记录的尝试边界;因此,随产品交付的 agent 重试策略改用 `agent/request-error`。 ### 消息(`message.ts`)与内容块(`types.ts`) diff --git a/packages/llm/llm/src/index.ts b/packages/llm/llm/src/index.ts index e87c428d06..9e4ce473d0 100644 --- a/packages/llm/llm/src/index.ts +++ b/packages/llm/llm/src/index.ts @@ -7,6 +7,7 @@ */ import { Context, Service } from '@deepseek-ai/cordis' +import z from '@deepseek-ai/schemastery' import type { GenerateOptions, LlmConfigurableProvider, @@ -21,8 +22,8 @@ import type { StreamChunk, } from './types.ts' import { freezeMessage, type Message } from './message.ts' -import { resolveRetryPolicy } from './retry-policy.ts' -import type { ResolvedRetryPolicy } from './retry-policy.ts' +import { resolveRetryPolicy, RetryPolicySchema } from './retry-policy.ts' +import type { ResolvedRetryPolicy, RetryPolicyConfig } from './retry-policy.ts' import type { ProviderRequestId } from './brand.ts' import { callConfigEquals, deepFreeze } from './call-config.ts' import type { LlmCallConfig, LlmCallConfigAdapterDefaults } from './call-config.ts' @@ -171,6 +172,12 @@ export interface PreparedLlmCall { stream(options: GenerateOptions): AsyncIterable } +/** Deployment-wide defaults applied to provider routes without adapter-owned overrides. */ +export interface Config { + /** Model-request retry policy inherited by routes whose adapter omits one; omission uses normal defaults. */ + defaultRetryPolicy?: RetryPolicyConfig +} + /** * Provider-wire adapter for the harness message and stream vocabulary. Register implementations * with `ctx.llm.registerAdapter(providers, adapter)`. Every provider HTTP request must include @@ -188,9 +195,9 @@ export abstract class LlmAdapter { } /** - * Return the provider-owned retry policy captured with this route. + * Return an explicit provider-owned retry policy override for this route. * @param _provider - a route passed to `registerAdapter()` for this instance. - * @returns a resolved policy, or `undefined` to use the normal defaults. + * @returns a resolved override, or `undefined` to inherit the LLM deployment default. */ providerRetryPolicy(_provider: string): ResolvedRetryPolicy | undefined { return undefined @@ -282,15 +289,22 @@ export interface DirectoryRegistrationHandle { * API, interceptable via the `llm/stream` waterfall. */ export class LlmRuntime extends Service { + static Config: z = z.object({ + defaultRetryPolicy: RetryPolicySchema, + }) + private adapters = new Map() private directory = new Map() private discoveries = new Map< string, (request: LlmModelDiscoveryRequest) => Promise >() + /** Resolved once because service configuration is fixed for this runtime instance. */ + private readonly defaultRetryPolicy: ResolvedRetryPolicy - constructor(ctx: Context) { + constructor(ctx: Context, config: Config = {}) { super(ctx, 'llm') + this.defaultRetryPolicy = resolveRetryPolicy(config.defaultRetryPolicy, 'llm: defaultRetryPolicy') } /** Notify topology observers without letting one broken listener veto the commit. */ @@ -385,7 +399,7 @@ export class LlmRuntime extends Service { } unique.add(provider) const retryPolicy = adapter.providerRetryPolicy(provider) - ?? resolveRetryPolicy(undefined, `llm: provider "${provider}" retryPolicy`) + ?? this.defaultRetryPolicy registrations.push({ adapter, provider: { id: info.id, name: info.name }, @@ -559,9 +573,9 @@ export class LlmRuntime extends Service { } /** - * Resolve the retry policy captured when one provider route was registered. + * Read the effective retry policy captured when one provider route was registered. * @param provider - registered provider route to inspect. - * @returns the provider-owned policy, with normal defaults already resolved. + * @returns the adapter override or deployment default, fully resolved. */ providerRetryPolicy(provider: string): ResolvedRetryPolicy { return this.registration(provider).retryPolicy diff --git a/packages/llm/llm/src/retry-policy.ts b/packages/llm/llm/src/retry-policy.ts index ad9c7af65c..99f38e84b3 100644 --- a/packages/llm/llm/src/retry-policy.ts +++ b/packages/llm/llm/src/retry-policy.ts @@ -1,5 +1,5 @@ /** - * Provider-owned request-retry policy configuration and resolution. + * Provider-route request-retry policy configuration and resolution. * * Adapters expose one resolved policy per registered provider route; the * optional dsh-llm-retry plugin executes it on the agent's failed-step extension point. @@ -53,7 +53,7 @@ export interface AlwaysRetryPolicyConfig { backoff?: BackoffConfig } -/** Provider-owned model-request retry policy configuration. */ +/** Model-request retry policy configuration for a provider route or deployment default. */ export type RetryPolicyConfig = NormalRetryPolicyConfig | AlwaysRetryPolicyConfig /** Fully resolved backoff shared by both retry modes. */ @@ -137,9 +137,9 @@ function resolveBackoff(config: BackoffConfig | undefined, path: string): Resolv } /** - * Validate, default, and detach one provider-owned retry policy. - * @param config - optional provider configuration; omission selects normal defaults. - * @param path - diagnostic path naming the provider config that owns the value. + * Validate, default, and detach one retry policy. + * @param config - optional route or deployment configuration; omission selects normal defaults. + * @param path - diagnostic path naming the configuration that owns the value. * @returns an immutable policy safe to capture in provider registration state. */ export function resolveRetryPolicy( diff --git a/packages/llm/llm/tests/service.spec.ts b/packages/llm/llm/tests/service.spec.ts index ed7d8d9bf8..2b8827aff5 100644 --- a/packages/llm/llm/tests/service.spec.ts +++ b/packages/llm/llm/tests/service.spec.ts @@ -203,7 +203,7 @@ describe('LlmRuntime', () => { expect(adapter.lastOptions?.messages[0]).toBe(message) }) - it('captures provider-owned retry policy at registration and defaults omission', async () => { + it('captures an adapter retry override ahead of the deployment default', async () => { const configured = resolveRetryPolicy({ mode: 'always' }, 'test retryPolicy') const adapter = new class extends ScriptedAdapter { override providerRetryPolicy(provider: string) { @@ -211,19 +211,42 @@ describe('LlmRuntime', () => { } }(SCRIPT) const ctx = new Context() - await ctx.plugin(LlmRuntime) + await ctx.plugin(LlmRuntime, { + defaultRetryPolicy: { mode: 'normal', maxRetries: 5 }, + }) ctx.llm.registerAdapter(['configured', 'defaulted'], adapter) expect(ctx.llm.providerRetryPolicy('configured')).toBe(configured) expect(ctx.llm.providerRetryPolicy('defaulted')).toMatchObject({ mode: 'normal', - maxRetries: 2, + maxRetries: 5, }) expect(() => ctx.llm.providerRetryPolicy('missing')).toThrow( expect.objectContaining({ code: 'NO_ADAPTER' }), ) }) + it('uses bounded normal defaults when the deployment omits a retry policy', async () => { + const ctx = new Context() + await ctx.plugin(LlmRuntime) + ctx.llm.registerAdapter(['defaulted'], new ScriptedAdapter(SCRIPT)) + + expect(ctx.llm.providerRetryPolicy('defaulted')).toMatchObject({ + mode: 'normal', + maxRetries: 2, + }) + }) + + it('rejects an invalid deployment retry policy before any route registers', async () => { + const ctx = new Context() + await expect(ctx.plugin(LlmRuntime, { + defaultRetryPolicy: { + mode: 'normal', + backoff: { initialDelayMs: 10, maxDelayMs: 5 }, + }, + })).rejects.toThrow(/llm: defaultRetryPolicy/) + }) + it('keeps a prepared registration and retry policy after route replacement', async () => { const oldPolicy = resolveRetryPolicy({ mode: 'always' }, 'old retryPolicy') const newPolicy = resolveRetryPolicy({ mode: 'normal', maxRetries: 0 }, 'new retryPolicy')