mirror of
https://github.com/deepseek-ai/deepseek-harness.git
synced 2026-08-29 04:26:38 +00:00
fix(llm-pi-ai): expose the pi-ai wire-compat surface
pi-ai infers a request's shape from the provider id and baseURL, and for an endpoint it does not recognize it answers as though it were OpenAI itself. A hand-declared route is by construction such an endpoint, so a model declaring reasoningEfforts sent its system prompt as the developer role with no configuration able to say otherwise — a gateway rejecting that role could not be connected at all. Writing the switch anyway validated, persisted, and was then dropped, so the misconfiguration looked applied. Three drift gates classify all thirty upstream compat fields as offered or withheld, keyed by `keyof` so a pi-ai upgrade fails the build until the new field is classified. Twenty are offered: what a private URL cannot imply. The rest stay withheld because pi-ai's installed catalog sets them for a named vendor. Protocol applicability is now per field rather than per block, so supportsDeveloperRole reaches an openai-responses route and the anthropic-messages switches reach theirs. A compat key no protocol declares, or one a gate withholds, is refused where it is written. Fixes #2646 Refs #1976
This commit is contained in:
Yichen Jiang
+2
-2
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-08-08-pi-ai-per-model-reasoning-declarations.md
|
||||
2026-08-08-pi-ai-per-model-reasoning-declarations.md: b6264feeb724e3693078fa3fc3e3fc16ed01aacb
|
||||
2026-08-08-pi-ai-per-model-reasoning-declarations.zh.md: 1b30f7e0c42974c777a535e133a47caa217e2e5e
|
||||
2026-08-08-pi-ai-per-model-reasoning-declarations.md: 0e8d5c3ca4017e89332f6b22e4eb0a06062918e6
|
||||
2026-08-08-pi-ai-per-model-reasoning-declarations.zh.md: c4c060b21c8b7ee5b11b5e98c8d7da0ba6c032fc
|
||||
|
||||
+1
-1
@@ -14,7 +14,7 @@ Two adjacent gaps compounded this. pi-ai decides the reasoning *wire dialect* (`
|
||||
|
||||
`PiAiModelProfile` gains `reasoningEfforts`: **each key is a level selectors offer, its value the spelling dispatch sends on the wire**. The declaration translates to pi-ai's `Model.reasoning` + `thinkingLevelMap` with all seven levels decided explicitly — declared levels carry their wire value, undeclared levels are pinned `null` — so the profile author never needs pi-ai's asymmetric defaulting rule (absent means "supported" for the five base levels but "unsupported" for `xhigh`/`max`). `off` is the one three-state key: left out, no Off is offered and an explicit Off request is refused (an effortless request still goes out bare, leaving the provider its default); declared valueless, Off is offered and dispatch sends nothing (the `deepseek` dialect sends `thinking: {type: "disabled"}`); declared with a value, that value goes on the wire. `false` declares a non-reasoning model; an empty declaration is refused rather than guessed at. The spelling for "disable" is `false` rather than `{}` because schemastery materializes an absent dict as `{}` — only a `z.union([z.const(false), dict])` keeps absent, disabled, and declared distinguishable, and a bare `reasoningEfforts:` (YAML null) slips through that union unvalidated, so resolution refuses it explicitly.
|
||||
|
||||
`compat.thinkingFormat` and `compat.supportsReasoningEffort` become configurable at two levels — route (its models' default) and model (winning per field) — resolving model → route → installed catalog entry → pi-ai's URL guess. They exist only on `openai-completions` (pi-ai types them nowhere else): a model-level switch on another protocol fails resolution, a route-level default skips such models, and a route with no completions model at all is refused. The two `chat-template` formats stay withheld for want of `chatTemplateKwargs`. Both enums are pinned to pi-ai's types through `Record<UpstreamUnion, true>` drift gates, so a pi-ai upgrade that adds a format fails compilation until the new member is classified (verified against the published 0.84.1 tarball, whose `thinkingFormat` union adds `baseten` over the pinned 0.82.1).
|
||||
`compat.thinkingFormat` and `compat.supportsReasoningEffort` become configurable at two levels — route (its models' default) and model (winning per field) — resolving model → route → installed catalog entry → pi-ai's URL guess. `thinkingFormat` is pinned to pi-ai's union through a `Record<UpstreamUnion, true>` drift gate, so a pi-ai upgrade that adds a format fails compilation until the new member is classified (verified against the published 0.84.1 tarball, whose `thinkingFormat` union adds `baseten` over the pinned 0.82.1). Which fields `compat` carries, which protocols take each of them, and how an unreadable key is refused are owned by [[2026-08-18-pi-ai-wire-compat-surface]]; the two-level resolution order above is what that surface generalizes.
|
||||
|
||||
`modelOverrides` reshapes individual catalog models without replacing the served set: key = catalog model id, value = a `models` entry minus `id`, materialized by handing the override to the existing entry path so capacities, efforts, compat, and request-default semantics stay identical. Unlike Pi's own config layer, which ignores unknown ids, every override that lands nowhere is refused — beside a `models` list, on a hand-declared route, naming an unknown model, or smuggling an `id` in the value (the schema passes unknown keys through, and a smuggled id would quietly rename the model).
|
||||
|
||||
|
||||
+1
-1
@@ -14,7 +14,7 @@ Status: implemented
|
||||
|
||||
`PiAiModelProfile` 新增 `reasoningEfforts`：**每个键是选择器提供的一个档位，其值是分派在协议中发送的拼写**。该声明会转换为 pi-ai 的 `Model.reasoning` + `thinkingLevelMap`，七个档位全部显式决定——已声明的档位携带自己的协议值，未声明的档位一律固定为 `null`——因此 profile 作者永远不需要了解 pi-ai 那条不对称的默认规则（键缺席对五个基础档位意味着「支持」，对 `xhigh`/`max` 却意味着「不支持」）。`off` 是唯一的三态键：不写，选择器不提供 Off，显式请求 Off 会被拒绝（不点名档位的请求仍会不带参数地发出，提供方保留自己的默认行为）；声明而不给值，则提供 Off，分派什么也不发送（`deepseek` 方言发送 `thinking: {type: "disabled"}`）；声明并给值，该值就在协议中发送。`false` 声明一个不具备推理能力的模型；空声明会被拒绝，而不是去猜。「禁用」的拼写取 `false` 而非 `{}`，因为 schemastery 会把缺席的字典物化成 `{}`——只有 `z.union([z.const(false), dict])` 才能让缺席、禁用与已声明三态保持可区分；而裸写的 `reasoningEfforts:`（YAML null）会不经校验地从该 union 溜过去，因此解析对它显式拒绝。
|
||||
|
||||
`compat.thinkingFormat` 与 `compat.supportsReasoningEffort` 变为两级可配置——路由级（作为其模型的默认值）与模型级（逐字段胜出）——解析顺序为模型 → 路由 → 已安装 catalog 条目 → pi-ai 按 URL 得出的猜测。两者只存在于 `openai-completions` 上（pi-ai 也只在这一协议上为它们建了类型）：在其他协议的模型上设模型级开关会使解析失败，路由级默认值会跳过这类模型，而完全没有 completions 模型的路由则被拒绝。两个 `chat-template` 格式因缺 `chatTemplateKwargs` 而继续保持不开放。两个枚举都经 `Record<UpstreamUnion, true>` 漂移门禁钉在 pi-ai 的类型上，因此新增格式的 pi-ai 升级会编译失败，直到新成员被归类（对照已发布的 0.84.1 tarball 验证过：其 `thinkingFormat` 联合类型相对钉住的 0.82.1 新增了 `baseten`）。
|
||||
`compat.thinkingFormat` 与 `compat.supportsReasoningEffort` 变为两级可配置——路由级（作为其模型的默认值）与模型级（逐字段胜出）——解析顺序为模型 → 路由 → 已安装 catalog 条目 → pi-ai 按 URL 得出的猜测。`thinkingFormat` 经 `Record<UpstreamUnion, true>` 漂移门禁钉在 pi-ai 的联合类型上，因此新增格式的 pi-ai 升级会编译失败，直到新成员被归类（对照已发布的 0.84.1 tarball 验证过：其 `thinkingFormat` 联合类型相对钉住的 0.82.1 新增了 `baseten`）。`compat` 承载哪些字段、每个字段由哪些协议接受、以及无法读取的键如何被拒绝，归 [[2026-08-18-pi-ai-wire-compat-surface]] 所有；上面这条两级解析顺序正是该面所推广的东西。
|
||||
|
||||
`modelOverrides` 就地重塑单个 catalog 模型而不替换所服务的集合：键 = catalog 模型 id，值 = 去掉 `id` 的 `models` 条目，物化时把覆盖交给既有的条目路径，因此容量、档位、compat 与请求默认值语义完全一致。与忽略未知 id 的 Pi 自有配置层不同，凡是落不到任何地方的覆盖都会被拒绝——与 `models` 列表并存、写在手工声明的路由上、点名未知模型，或在值里夹带 `id`（schema 会放行未知键，被夹带的 id 会悄悄把模型改名）。
|
||||
|
||||
|
||||
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-08-18-pi-ai-wire-compat-surface.md
|
||||
2026-08-18-pi-ai-wire-compat-surface.md: c7e0bc75806e8ba022db9c9f17cfe2b621c21611
|
||||
2026-08-18-pi-ai-wire-compat-surface.zh.md: c592044e239421110813feb364bacc142a9d2d32
|
||||
@@ -0,0 +1,51 @@
|
||||
# Agent Note: pi-ai Wire-Compatibility Surface in llm-pi-ai
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-08-18-pi-ai-wire-compat-surface.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
pi-ai shapes every request from the provider id and the baseURL — which role carries the system prompt, which field caps output, whether `store` and `stream_options` go out, whether tool definitions carry `strict`. For an endpoint its detection does not recognize, the answer is "this is OpenAI itself": `detectCompat` returns `supportsDeveloperRole: true`, `maxTokensField: "max_completion_tokens"`, `supportsStore: true`. A hand-declared route is by construction an endpoint pi-ai does not ship, so every such route received OpenAI's own request shape.
|
||||
|
||||
The adapter offered two of pi-ai's thirty compat fields ([[2026-08-08-pi-ai-per-model-reasoning-declarations]] scoped them to "the switches pi-ai's reasoning dispatch reads"), and `supportsDeveloperRole` fell inside that scope while being absent from it: its send site is `model.reasoning && compat.supportsDeveloperRole`. A hand-declared model declaring `reasoningEfforts` therefore sent its system prompt as `role: "developer"`, which most OpenAI-compatible gateways reject, and no configuration could say otherwise — the gateway could not be connected at all.
|
||||
|
||||
Writing the field anyway was worse than unsupported. schemastery passes unknown keys through, and resolution read only two names, so `compat: {supportsDeveloperRole: false}` validated, persisted, and was then dropped: the operator saw an accepted write and an unchanged failure. `maxTokensField` carried the same defect over a wider blast radius, since it shapes every request rather than only a reasoning model's.
|
||||
|
||||
## Decision
|
||||
|
||||
Three drift gates — one per pi-ai compat type, keyed `Record<keyof OpenAICompletionsCompat | …, CompatDisposition>` — classify all thirty upstream fields as `offer` or `withhold`. Twenty are offered. The line is what a private URL can imply: a deployment must be able to state what nothing can infer from an unrecognized endpoint, while a field pi-ai's installed catalog sets for a named vendor stays withheld, because a route reaching for `openRouterRouting` or `deferredToolsMode` is a catalog route that should be named as such and inherit the value.
|
||||
|
||||
`PiAiCompatProfile` stays an explicit interface with per-field JSDoc — it is what a configuration surface renders and what `docs/config-catalog.md` pastes — and a type-level `AssertNever` over the symmetric difference proves it names exactly the offered set. The schemastery schema is declared `z<PiAiCompatProfile>`, so the four faces lock together: an upstream field added, a gate entry missing, an interface field forgotten, or a schema key omitted each fails compilation naming the field.
|
||||
|
||||
Protocol applicability is per field rather than per block. `supportsDeveloperRole` is settable wherever pi-ai declares it (`openai-completions` and `openai-responses`), `thinkingFormat` only on the former, `supportsTemperature` only on `anthropic-messages`. A model-level switch its protocol does not take fails resolution naming what that protocol does offer; a route-level one lands on the models that read it and skips the rest, and is refused only when no model on the route could read it. `chatTemplateKwargs` is offered, which is what makes the two `chat-template` thinking formats nameable.
|
||||
|
||||
A `compat` key no protocol declares, and one a gate withholds, are both refused where they are written rather than dropped. The check runs over every key before any protocol resolves, so a misspelling fails even on a route whose models never reach the protocol that would have taken it. It reads raw keys deliberately: a withheld or undeclared name is absent from the schema, so schemastery cannot have materialized it and a person wrote it. Fields carrying a value are then filtered separately, because schemastery materializes an absent dict as `{}` and `chatTemplateKwargs` is present on every parsed profile whether or not anyone wrote one.
|
||||
|
||||
## Where a refusal lands
|
||||
|
||||
Every check runs in `resolveProfiles`, which no request path re-enters: the adapter memoizes by raw-snapshot identity and `apply` resolves once eagerly. A refusal therefore reaches `settings.mutate` as `settings-rejected` before persistence, a `cordis.yml` `config:` block as a failed plugin mount, and a stored section as a failed `settings.register` at startup.
|
||||
|
||||
An external edit to the settings file is the one path that cannot report: the provider watcher calls `publish()`, which catches a failing section, logs `settings: keeping last good "%s"`, and leaves the namespace serving its previous value. That is the settings seam's behavior for every schema and validator failure, not something this surface introduces, and closing it belongs to that seam rather than here. What changes for compat is the failure model, not the reporting: a key that formerly stayed inert forever now stops the next start.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**Add `supportsDeveloperRole` alone.** It fixes the reported gateway and leaves `maxTokensField` — which shapes every request, not only a reasoning model's — breaking a whole class of endpoints, with the next upstream addition free to lag silently again.
|
||||
|
||||
**Offer every upstream field.** pi-ai's own custom-provider documentation converges on a far smaller set, its flagship example naming six, and the remainder are vendor-bound switches its catalog already sets. Exposing `zaiToolStream` or `vercelGatewayRouting` on a hand-declared route offers a knob whose correct use is to not be a hand-declared route.
|
||||
|
||||
**Key `compat` by protocol** (`compat: {openai-completions: {…}}`). A hand-declared route has exactly one `api`, so the nesting states what the route already said, and it breaks every profile written against the flat shape for nothing.
|
||||
|
||||
**Accept an opaque passthrough dict.** The schema is also the shape a configuration surface renders and the declaration `verify-config-catalog` cross-checks, both of which an unstructured dict defeats; it would also let a responses-only field land on a completions model, which per-field applicability exists to refuse.
|
||||
|
||||
**Warn instead of refusing an unknown key.** That is the posture that hid this defect for the life of the surface: an accepted write and an unchanged failure teaches the operator that the switch does not work, not that the name is wrong.
|
||||
|
||||
**Suggest a near spelling on an unknown key.** No repository utility computes edit distance, and adding a dependency or hand-rolling one under the per-file coverage gate is disproportionate for a diagnostic. Naming the offered fields answers the same question deterministically: the vocabulary check runs before any protocol resolves, so it names the whole offered set, while the per-protocol refusal narrows to what that protocol takes.
|
||||
|
||||
## Consequences
|
||||
|
||||
- An OpenAI-compatible gateway that rejects the `developer` role, `max_completion_tokens`, `store`, `stream_options`, or `strict` is now configuration rather than an unreachable provider, and the same holds for an Anthropic-compatible gateway rejecting `temperature` or tool `cache_control`.
|
||||
- A pi-ai upgrade that adds a compat field fails the build until someone classifies it, which is how `chatTemplateKwargs` and the `chat-template` formats stopped being a standing exception.
|
||||
- Unknown compat keys join every other configuration error's failure model. The improvement over the previous silent drop is bounded by the settings seam: an external file edit still keeps its last good value and warns, so the operator's signal is a restart rather than the write.
|
||||
- **Deferred, not closed:** `publish()` reports a rejected stored section only through `ctx.logger.warn`, with no user-visible channel. It affects every settings namespace and is owned by `dsh-settings`.
|
||||
- [[2026-08-08-pi-ai-per-model-reasoning-declarations]] is partially superseded: its compat-scope statements are restated here, while its `reasoningEfforts` shape, the alternatives that shape beat, and `modelOverrides` remain the current authority.
|
||||
@@ -0,0 +1,51 @@
|
||||
# Agent Note: pi-ai Wire-Compatibility Surface in llm-pi-ai
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-08-18-pi-ai-wire-compat-surface.md) | 中文
|
||||
|
||||
## Problem
|
||||
|
||||
pi-ai 依据提供方 id 与 baseURL 决定每个请求的形状——系统提示词由哪个角色承载、输出上限写在哪个字段、是否发出 `store` 与 `stream_options`、工具定义是否携带 `strict`。对于其检测无法识别的端点，答案是「这就是 OpenAI 本身」：`detectCompat` 返回 `supportsDeveloperRole: true`、`maxTokensField: "max_completion_tokens"`、`supportsStore: true`。而手工声明的路由按其构造就是 pi-ai 未随附的端点，于是每一条这样的路由都收到了 OpenAI 自己的请求形状。
|
||||
|
||||
适配器只开放了 pi-ai 三十个 compat 字段中的两个（[[2026-08-08-pi-ai-per-model-reasoning-declarations]] 把它们限定为「pi-ai 推理分派读取的那些开关」），而 `supportsDeveloperRole` 恰恰落在该作用域之内却不在其中：它的发送点是 `model.reasoning && compat.supportsDeveloperRole`。因此一个声明了 `reasoningEfforts` 的手工声明模型会把系统提示词以 `role: "developer"` 发出——多数 OpenAI 兼容网关会拒绝该角色——而没有任何配置能够更正，该网关根本接不进来。
|
||||
|
||||
硬写这个字段比不支持更糟。schemastery 会放行未知键，而解析只读取两个名字，于是 `compat: {supportsDeveloperRole: false}` 通过校验、落盘，随后被丢弃：运维看到的是一次被接受的写入和一个毫无变化的故障。`maxTokensField` 带着同一缺陷、却有更大的波及面，因为它塑造每一个请求，而不只是推理模型的请求。
|
||||
|
||||
## Decision
|
||||
|
||||
三张漂移门禁——每个 pi-ai compat 类型一张，以 `Record<keyof OpenAICompletionsCompat | …, CompatDisposition>` 为键——把全部三十个上游字段分类为 `offer` 或 `withhold`，其中二十个开放。分界线在于私有 URL 能推出什么：凡是无法从未识别端点推断的，部署方必须能够说出口；而 pi-ai 已安装 catalog 为具名厂商设定的字段保持扣留，因为伸手去够 `openRouterRouting` 或 `deferredToolsMode` 的路由，本就是一条应当以该厂商命名、并继承其值的 catalog 路由。
|
||||
|
||||
`PiAiCompatProfile` 保持为带逐字段 JSDoc 的显式 interface——它是配置界面所渲染、也是 `docs/config-catalog.md` 所粘贴的东西——并由一个作用在对称差上的类型级 `AssertNever` 证明它恰好命名了开放集。schemastery schema 声明为 `z<PiAiCompatProfile>`，于是四个面互锁：上游新增字段、门禁漏一条、interface 忘记一个字段、schema 少一个键，都会在编译期以点名该字段的方式失败。
|
||||
|
||||
协议适用性逐字段判断，而非整块判断。`supportsDeveloperRole` 在 pi-ai 声明它的任何地方均可设置（`openai-completions` 与 `openai-responses`），`thinkingFormat` 只在前者，`supportsTemperature` 只在 `anthropic-messages`。模型级开关若其协议并不接受，解析失败并点名该协议实际提供哪些开关；路由级开关则落在读取它的模型上、跳过其余模型，只有当路由上没有任何模型能读取它时才被拒绝。`chatTemplateKwargs` 予以开放，这正是两个 `chat-template` 思考格式得以命名的前提。
|
||||
|
||||
没有任何协议声明的 `compat` 键，以及被门禁扣留的键，都在其被写下之处遭到拒绝而非丢弃。该检查在任何协议解析之前遍历全部键，因此即便路由上的模型永远不会走到那个本会接受它的协议，笔误同样失败。它刻意读取原始键：被扣留或未声明的名字不在 schema 中，所以 schemastery 不可能物化它，写下它的必然是人。随后再单独过滤携带值的字段，因为 schemastery 会把缺省的 dict 物化成 `{}`，于是无论有没有人写过，`chatTemplateKwargs` 都出现在每一个解析过的 profile 上。
|
||||
|
||||
## Where a refusal lands
|
||||
|
||||
所有检查都在 `resolveProfiles` 中运行，而请求路径不会重新进入它：适配器按原始快照的标识 memoize，且 `apply` 会主动预先解析一次。因此一次拒绝会以 `settings-rejected` 的形式在落盘之前抵达 `settings.mutate`，以插件挂载失败的形式抵达 `cordis.yml` 的 `config:` 块，以 `settings.register` 启动失败的形式抵达已存的 section。
|
||||
|
||||
对 settings 文件的外部编辑是唯一无法报告的路径：提供方监听器调用 `publish()`，它捕获失败的 section、记录 `settings: keeping last good "%s"`，并让该 namespace 继续服务其先前的值。这是 settings seam 对每一种 schema 与校验器失败的既有行为，并非本次开放引入，弥合它属于那个 seam 而不属于此处。对 compat 而言改变的是失败模型而非报告方式：一个从前永远静默无效的键，如今会拦下下一次启动。
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**只补 `supportsDeveloperRole`。** 它修好了报告中的那个网关，却放任 `maxTokensField`——它塑造每一个请求，而不只是推理模型的请求——继续拖垮一整类端点，而且下一个上游新增字段依然可以静默落后。
|
||||
|
||||
**开放全部上游字段。** pi-ai 自己的 custom-provider 文档收敛到一个小得多的集合，其旗舰示例只点名六个，其余都是其 catalog 已经设定好的厂商绑定开关。在手工声明路由上暴露 `zaiToolStream` 或 `vercelGatewayRouting`，等于提供一个「正确用法是别做手工声明路由」的旋钮。
|
||||
|
||||
**把 `compat` 按协议分层**（`compat: {openai-completions: {…}}`）。手工声明路由恰好只有一个 `api`，因此这层嵌套只是复述路由已经说过的事，还白白破坏了所有按扁平形状写下的 profile。
|
||||
|
||||
**接受一个不透明的透传 dict。** 该 schema 同时是配置界面渲染的形状、也是 `verify-config-catalog` 交叉校验的声明，无结构的 dict 会同时击溃两者；它还会让 responses 独有的字段落到 completions 模型上，而逐字段适用性正是为拒绝这种情况而存在。
|
||||
|
||||
**未知键只告警不拒绝。** 这恰恰是让本缺陷伴随该面存活至今的姿态：一次被接受的写入加一个毫无变化的故障，教给运维的是「这个开关没用」，而不是「这个名字写错了」。
|
||||
|
||||
**为未知键给出近似拼写建议。** 仓库中没有计算编辑距离的工具，在逐文件覆盖率门禁之下为一条诊断引入依赖或手搓一个都不成比例。点名开放字段能确定地回答同一个问题：词汇检查跑在任何协议解析之前，因此它列出整个开放集，而按协议的拒绝则收窄到该协议实际接受的字段。
|
||||
|
||||
## Consequences
|
||||
|
||||
- 拒绝 `developer` 角色、`max_completion_tokens`、`store`、`stream_options` 或 `strict` 的 OpenAI 兼容网关，如今属于配置问题而非无法接入的提供方；拒绝 `temperature` 或工具 `cache_control` 的 Anthropic 兼容网关同理。
|
||||
- pi-ai 升级新增 compat 字段会使构建失败，直到有人为它做出分类——`chatTemplateKwargs` 与那两个 `chat-template` 格式正是因此不再是一项长期例外。
|
||||
- 未知 compat 键并入了其余所有配置错误的失败模型。相对此前静默丢弃的改善程度受 settings seam 限制：外部文件编辑仍会保留其上一个有效值并告警，因此运维拿到的信号是一次重启，而不是那次写入。
|
||||
- **搁置而非解决：** `publish()` 对被拒绝的已存 section 只通过 `ctx.logger.warn` 报告，没有面向用户的通道。它影响每一个 settings namespace，归属 `dsh-settings`。
|
||||
- [[2026-08-08-pi-ai-per-model-reasoning-declarations]] 被部分取代：其 compat 作用域的陈述在此重述，而其 `reasoningEfforts` 形状、该形状所击败的备选方案以及 `modelOverrides` 仍是当前权威。
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write docs/config-catalog.md
|
||||
config-catalog.md: 9e3dc2b14a04877c191227703752d0b1dbb2b92b
|
||||
config-catalog.zh.md: 56c1fd075b680c79c7730b9bdcb1275e83bb3f64
|
||||
config-catalog.md: ad9bb6284354b934bd6e0bf3235dcd0b74f33cc5
|
||||
config-catalog.zh.md: 7038b951a14115873a032fab0066be2a990e3a3f
|
||||
|
||||
+65
-27
@@ -938,10 +938,11 @@ export interface PiAiProviderProfile {
|
||||
*/
|
||||
modelOverrides?: Record<string, PiAiModelOverride>
|
||||
/**
|
||||
* Reasoning-dispatch switches for every `openai-completions` model on this
|
||||
* route; each model's own `compat` overrides per field. What neither sets
|
||||
* keeps the installed catalog entry's value, then pi-ai's baseURL-derived
|
||||
* detection.
|
||||
* pi-ai wire-compatibility switches defaulting every model on this route
|
||||
* whose protocol declares them; each model's own `compat` overrides per
|
||||
* field. What neither sets keeps the installed catalog entry's value, then
|
||||
* pi-ai's own detection. A switch no model on the route could read is
|
||||
* refused rather than left looking applied.
|
||||
*/
|
||||
compat?: PiAiCompatProfile
|
||||
/**
|
||||
@@ -1022,7 +1023,7 @@ export interface PiAiModelProfile {
|
||||
* declares the offered levels and their wire spellings.
|
||||
*/
|
||||
reasoningEfforts?: false | PiAiReasoningEfforts
|
||||
/** Reasoning-dispatch switches for this model, winning over the route's. */
|
||||
/** pi-ai wire-compatibility switches for this model, winning over the route's per field; one its protocol does not declare is refused. */
|
||||
compat?: PiAiCompatProfile
|
||||
}
|
||||
|
||||
@@ -1036,19 +1037,65 @@ export interface PiAiModelProfile {
|
||||
export type PiAiModelOverride = Omit<PiAiModelProfile, 'id'>
|
||||
|
||||
/**
|
||||
* Reasoning-dispatch compatibility switches, set on the route (its models'
|
||||
* default) or per model (winning over the route). Only the switches pi-ai's
|
||||
* reasoning dispatch reads are offered; the rest of pi-ai's compat surface
|
||||
* keeps its baseURL-derived auto-detection. pi-ai types both fields only on
|
||||
* `OpenAICompletionsCompat` — the other wire protocols define their reasoning
|
||||
* fields in the protocol itself — so resolution rejects a model-level switch
|
||||
* anywhere else, while a route-level default skips past models it cannot fit.
|
||||
* pi-ai wire-compatibility switches, set on the route (its models' default) or
|
||||
* per model (winning over the route, field by field).
|
||||
*
|
||||
* pi-ai decides each of these from the provider id and baseURL when no layer
|
||||
* sets it, and a private gateway's URL says nothing: for an endpoint it does
|
||||
* not recognize the detection answers as though it were OpenAI itself, which
|
||||
* is wrong for most OpenAI-compatible gateways. So every field here is one a
|
||||
* deployment must be able to state because nothing can infer it, while the
|
||||
* fields pi-ai's catalog sets for a named vendor stay withheld.
|
||||
*
|
||||
* A field belongs to the protocols whose upstream compat type declares it: a
|
||||
* model-level switch its protocol does not take fails resolution, and a
|
||||
* route-level one skips past models it cannot fit.
|
||||
*/
|
||||
export interface PiAiCompatProfile {
|
||||
/** Reasoning parameter format the endpoint expects; absent keeps the catalog entry's, then pi-ai's baseURL-derived guess. */
|
||||
thinkingFormat?: PiAiThinkingFormat
|
||||
/** Whether the endpoint accepts `reasoning_effort`; absent keeps the catalog entry's, then pi-ai's baseURL-derived guess. */
|
||||
/** Whether the endpoint accepts `store`; `openai-completions`. */
|
||||
supportsStore?: boolean
|
||||
/**
|
||||
* Whether the endpoint accepts the `developer` role for the system prompt,
|
||||
* which pi-ai sends only to a reasoning model; `false` keeps `system`.
|
||||
* `openai-completions`, `openai-responses`.
|
||||
*/
|
||||
supportsDeveloperRole?: boolean
|
||||
/** Whether the endpoint accepts `reasoning_effort`; `openai-completions`. */
|
||||
supportsReasoningEffort?: boolean
|
||||
/** Whether the endpoint accepts `stream_options: {include_usage: true}`; `openai-completions`. */
|
||||
supportsUsageInStreaming?: boolean
|
||||
/** Which output-cap field the endpoint reads; `openai-completions`. */
|
||||
maxTokensField?: 'max_completion_tokens' | 'max_tokens'
|
||||
/** Whether tool results must carry `name`; `openai-completions`. */
|
||||
requiresToolResultName?: boolean
|
||||
/** Whether a user message after tool results needs an assistant message between; `openai-completions`. */
|
||||
requiresAssistantAfterToolResult?: boolean
|
||||
/** Whether thinking blocks must travel as text in `<thinking>` delimiters; `openai-completions`. */
|
||||
requiresThinkingAsText?: boolean
|
||||
/** Whether replayed assistant messages need an empty `reasoning_content` while reasoning is on; `openai-completions`. */
|
||||
requiresReasoningContentOnAssistantMessages?: boolean
|
||||
/** Reasoning parameter format the endpoint expects; `openai-completions`. */
|
||||
thinkingFormat?: PiAiThinkingFormat
|
||||
/** Kwargs sent as `chat_template_kwargs`, for the two `chat-template` thinking formats; `openai-completions`. */
|
||||
chatTemplateKwargs?: Record<string, ChatTemplateKwargValue>
|
||||
/** Whether the endpoint accepts `strict` in tool definitions; `openai-completions`, `openai-responses`. */
|
||||
supportsStrictMode?: boolean
|
||||
/** Prompt-cache marker convention; `openai-completions`. */
|
||||
cacheControlFormat?: 'anthropic'
|
||||
/** Whether the endpoint accepts long prompt-cache retention; all three protocols. */
|
||||
supportsLongCacheRetention?: boolean
|
||||
/** Whether the endpoint accepts per-tool `eager_input_streaming`; `anthropic-messages`. */
|
||||
supportsEagerToolInputStreaming?: boolean
|
||||
/** Whether the endpoint accepts `cache_control` on tool definitions; `anthropic-messages`. */
|
||||
supportsCacheControlOnTools?: boolean
|
||||
/** Whether the endpoint accepts the `temperature` request field; `anthropic-messages`. */
|
||||
supportsTemperature?: boolean
|
||||
/** Whether to force adaptive thinking regardless of model id; `anthropic-messages`. */
|
||||
forceAdaptiveThinking?: boolean
|
||||
/** Whether to replay an empty thinking signature instead of converting thinking to text; `anthropic-messages`. */
|
||||
allowEmptySignature?: boolean
|
||||
/** Whether the endpoint accepts Anthropic strict tool schemas; `anthropic-messages`. */
|
||||
supportsStrictTools?: boolean
|
||||
}
|
||||
|
||||
/** One request modality a pi-ai model may accept. */
|
||||
@@ -1065,21 +1112,12 @@ export type PiAiModality = Model<Api>['input'][number]
|
||||
export type PiAiReasoningEfforts = Partial<Record<ModelThinkingLevel, string | null>>
|
||||
|
||||
/** One reasoning-dispatch wire format a profile may name. */
|
||||
export type PiAiThinkingFormat = Exclude<PiThinkingFormat, WithheldThinkingFormat>
|
||||
|
||||
/** The `compat.thinkingFormat` spellings pi-ai accepts on an `openai-completions` model. */
|
||||
type PiThinkingFormat = NonNullable<OpenAICompletionsCompat['thinkingFormat']>
|
||||
|
||||
/**
|
||||
* pi-ai thinking formats a profile cannot name: both drive the request through
|
||||
* `chatTemplateKwargs`, which this configuration does not expose.
|
||||
*/
|
||||
type WithheldThinkingFormat = 'chat-template' | 'qwen-chat-template'
|
||||
export type PiAiThinkingFormat = NonNullable<OpenAICompletionsCompat['thinkingFormat']>
|
||||
```
|
||||
|
||||
Depends on: `Api` (`@earendil-works/pi-ai`) · `CacheRetention` (`@earendil-works/pi-ai`) · `Model` (`@earendil-works/pi-ai`) · `ModelThinkingLevel` (`@earendil-works/pi-ai`) · `OpenAICompletionsCompat` (`@earendil-works/pi-ai`) · [`RetryPolicyConfig`](../packages/llm/llm/src/index.ts) · `ThinkingBudgets` (`@earendil-works/pi-ai`) · `Transport` (`@earendil-works/pi-ai`)
|
||||
Depends on: `Api` (`@earendil-works/pi-ai`) · `CacheRetention` (`@earendil-works/pi-ai`) · `ChatTemplateKwargValue` (`@earendil-works/pi-ai`) · `Model` (`@earendil-works/pi-ai`) · `ModelThinkingLevel` (`@earendil-works/pi-ai`) · `OpenAICompletionsCompat` (`@earendil-works/pi-ai`) · [`RetryPolicyConfig`](../packages/llm/llm/src/index.ts) · `ThinkingBudgets` (`@earendil-works/pi-ai`) · `Transport` (`@earendil-works/pi-ai`)
|
||||
|
||||
Source: [`packages/llm/llm-pi-ai/src/config.ts:172`](../packages/llm/llm-pi-ai/src/config.ts)
|
||||
Source: [`packages/llm/llm-pi-ai/src/config.ts:173`](../packages/llm/llm-pi-ai/src/config.ts)
|
||||
|
||||
<a id="deepseek-aidsh-llm-replay"></a>
|
||||
|
||||
|
||||
+65
-27
@@ -940,10 +940,11 @@ export interface PiAiProviderProfile {
|
||||
*/
|
||||
modelOverrides?: Record<string, PiAiModelOverride>
|
||||
/**
|
||||
* Reasoning-dispatch switches for every `openai-completions` model on this
|
||||
* route; each model's own `compat` overrides per field. What neither sets
|
||||
* keeps the installed catalog entry's value, then pi-ai's baseURL-derived
|
||||
* detection.
|
||||
* pi-ai wire-compatibility switches defaulting every model on this route
|
||||
* whose protocol declares them; each model's own `compat` overrides per
|
||||
* field. What neither sets keeps the installed catalog entry's value, then
|
||||
* pi-ai's own detection. A switch no model on the route could read is
|
||||
* refused rather than left looking applied.
|
||||
*/
|
||||
compat?: PiAiCompatProfile
|
||||
/**
|
||||
@@ -1024,7 +1025,7 @@ export interface PiAiModelProfile {
|
||||
* declares the offered levels and their wire spellings.
|
||||
*/
|
||||
reasoningEfforts?: false | PiAiReasoningEfforts
|
||||
/** Reasoning-dispatch switches for this model, winning over the route's. */
|
||||
/** pi-ai wire-compatibility switches for this model, winning over the route's per field; one its protocol does not declare is refused. */
|
||||
compat?: PiAiCompatProfile
|
||||
}
|
||||
|
||||
@@ -1038,19 +1039,65 @@ export interface PiAiModelProfile {
|
||||
export type PiAiModelOverride = Omit<PiAiModelProfile, 'id'>
|
||||
|
||||
/**
|
||||
* Reasoning-dispatch compatibility switches, set on the route (its models'
|
||||
* default) or per model (winning over the route). Only the switches pi-ai's
|
||||
* reasoning dispatch reads are offered; the rest of pi-ai's compat surface
|
||||
* keeps its baseURL-derived auto-detection. pi-ai types both fields only on
|
||||
* `OpenAICompletionsCompat` — the other wire protocols define their reasoning
|
||||
* fields in the protocol itself — so resolution rejects a model-level switch
|
||||
* anywhere else, while a route-level default skips past models it cannot fit.
|
||||
* pi-ai wire-compatibility switches, set on the route (its models' default) or
|
||||
* per model (winning over the route, field by field).
|
||||
*
|
||||
* pi-ai decides each of these from the provider id and baseURL when no layer
|
||||
* sets it, and a private gateway's URL says nothing: for an endpoint it does
|
||||
* not recognize the detection answers as though it were OpenAI itself, which
|
||||
* is wrong for most OpenAI-compatible gateways. So every field here is one a
|
||||
* deployment must be able to state because nothing can infer it, while the
|
||||
* fields pi-ai's catalog sets for a named vendor stay withheld.
|
||||
*
|
||||
* A field belongs to the protocols whose upstream compat type declares it: a
|
||||
* model-level switch its protocol does not take fails resolution, and a
|
||||
* route-level one skips past models it cannot fit.
|
||||
*/
|
||||
export interface PiAiCompatProfile {
|
||||
/** Reasoning parameter format the endpoint expects; absent keeps the catalog entry's, then pi-ai's baseURL-derived guess. */
|
||||
thinkingFormat?: PiAiThinkingFormat
|
||||
/** Whether the endpoint accepts `reasoning_effort`; absent keeps the catalog entry's, then pi-ai's baseURL-derived guess. */
|
||||
/** Whether the endpoint accepts `store`; `openai-completions`. */
|
||||
supportsStore?: boolean
|
||||
/**
|
||||
* Whether the endpoint accepts the `developer` role for the system prompt,
|
||||
* which pi-ai sends only to a reasoning model; `false` keeps `system`.
|
||||
* `openai-completions`, `openai-responses`.
|
||||
*/
|
||||
supportsDeveloperRole?: boolean
|
||||
/** Whether the endpoint accepts `reasoning_effort`; `openai-completions`. */
|
||||
supportsReasoningEffort?: boolean
|
||||
/** Whether the endpoint accepts `stream_options: {include_usage: true}`; `openai-completions`. */
|
||||
supportsUsageInStreaming?: boolean
|
||||
/** Which output-cap field the endpoint reads; `openai-completions`. */
|
||||
maxTokensField?: 'max_completion_tokens' | 'max_tokens'
|
||||
/** Whether tool results must carry `name`; `openai-completions`. */
|
||||
requiresToolResultName?: boolean
|
||||
/** Whether a user message after tool results needs an assistant message between; `openai-completions`. */
|
||||
requiresAssistantAfterToolResult?: boolean
|
||||
/** Whether thinking blocks must travel as text in `<thinking>` delimiters; `openai-completions`. */
|
||||
requiresThinkingAsText?: boolean
|
||||
/** Whether replayed assistant messages need an empty `reasoning_content` while reasoning is on; `openai-completions`. */
|
||||
requiresReasoningContentOnAssistantMessages?: boolean
|
||||
/** Reasoning parameter format the endpoint expects; `openai-completions`. */
|
||||
thinkingFormat?: PiAiThinkingFormat
|
||||
/** Kwargs sent as `chat_template_kwargs`, for the two `chat-template` thinking formats; `openai-completions`. */
|
||||
chatTemplateKwargs?: Record<string, ChatTemplateKwargValue>
|
||||
/** Whether the endpoint accepts `strict` in tool definitions; `openai-completions`, `openai-responses`. */
|
||||
supportsStrictMode?: boolean
|
||||
/** Prompt-cache marker convention; `openai-completions`. */
|
||||
cacheControlFormat?: 'anthropic'
|
||||
/** Whether the endpoint accepts long prompt-cache retention; all three protocols. */
|
||||
supportsLongCacheRetention?: boolean
|
||||
/** Whether the endpoint accepts per-tool `eager_input_streaming`; `anthropic-messages`. */
|
||||
supportsEagerToolInputStreaming?: boolean
|
||||
/** Whether the endpoint accepts `cache_control` on tool definitions; `anthropic-messages`. */
|
||||
supportsCacheControlOnTools?: boolean
|
||||
/** Whether the endpoint accepts the `temperature` request field; `anthropic-messages`. */
|
||||
supportsTemperature?: boolean
|
||||
/** Whether to force adaptive thinking regardless of model id; `anthropic-messages`. */
|
||||
forceAdaptiveThinking?: boolean
|
||||
/** Whether to replay an empty thinking signature instead of converting thinking to text; `anthropic-messages`. */
|
||||
allowEmptySignature?: boolean
|
||||
/** Whether the endpoint accepts Anthropic strict tool schemas; `anthropic-messages`. */
|
||||
supportsStrictTools?: boolean
|
||||
}
|
||||
|
||||
/** One request modality a pi-ai model may accept. */
|
||||
@@ -1067,21 +1114,12 @@ export type PiAiModality = Model<Api>['input'][number]
|
||||
export type PiAiReasoningEfforts = Partial<Record<ModelThinkingLevel, string | null>>
|
||||
|
||||
/** One reasoning-dispatch wire format a profile may name. */
|
||||
export type PiAiThinkingFormat = Exclude<PiThinkingFormat, WithheldThinkingFormat>
|
||||
|
||||
/** The `compat.thinkingFormat` spellings pi-ai accepts on an `openai-completions` model. */
|
||||
type PiThinkingFormat = NonNullable<OpenAICompletionsCompat['thinkingFormat']>
|
||||
|
||||
/**
|
||||
* pi-ai thinking formats a profile cannot name: both drive the request through
|
||||
* `chatTemplateKwargs`, which this configuration does not expose.
|
||||
*/
|
||||
type WithheldThinkingFormat = 'chat-template' | 'qwen-chat-template'
|
||||
export type PiAiThinkingFormat = NonNullable<OpenAICompletionsCompat['thinkingFormat']>
|
||||
```
|
||||
|
||||
依赖：`Api`（`@earendil-works/pi-ai`）· `CacheRetention`（`@earendil-works/pi-ai`）· `Model`（`@earendil-works/pi-ai`）· `ModelThinkingLevel`（`@earendil-works/pi-ai`）· `OpenAICompletionsCompat`（`@earendil-works/pi-ai`）· [`RetryPolicyConfig`](../packages/llm/llm/src/index.ts) · `ThinkingBudgets`（`@earendil-works/pi-ai`）· `Transport`（`@earendil-works/pi-ai`）
|
||||
依赖：`Api`（`@earendil-works/pi-ai`）· `CacheRetention`（`@earendil-works/pi-ai`）· `ChatTemplateKwargValue`（`@earendil-works/pi-ai`）· `Model`（`@earendil-works/pi-ai`）· `ModelThinkingLevel`（`@earendil-works/pi-ai`）· `OpenAICompletionsCompat`（`@earendil-works/pi-ai`）· [`RetryPolicyConfig`](../packages/llm/llm/src/index.ts) · `ThinkingBudgets`（`@earendil-works/pi-ai`）· `Transport`（`@earendil-works/pi-ai`）
|
||||
|
||||
来源：[`packages/llm/llm-pi-ai/src/config.ts:172`](../packages/llm/llm-pi-ai/src/config.ts)
|
||||
来源：[`packages/llm/llm-pi-ai/src/config.ts:173`](../packages/llm/llm-pi-ai/src/config.ts)
|
||||
|
||||
<a id="deepseek-aidsh-llm-replay"></a>
|
||||
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/llm/llm-pi-ai/README.md
|
||||
README.md: d775e72616822ce0deee063ac0f3fc453af1a126
|
||||
README.zh.md: 621d67d1c181c6d4c78ea0078f521acccce92653
|
||||
README.md: 3251cc7e71e343fa82ab76f95fe341949a0836c6
|
||||
README.zh.md: b62a4069e9f5c57533054d439fd63b9cbd6f44cd
|
||||
|
||||
@@ -51,9 +51,12 @@ Configure credentials, the model catalog, and deployment-specific transport sett
|
||||
apiKeyEnv: ACME_GATEWAY_API_KEY
|
||||
api: openai-completions
|
||||
baseURL: https://gateway.acme.example/v1
|
||||
# Reasoning dialect for an endpoint whose URL pi-ai cannot recognize.
|
||||
# Request shape for an endpoint whose URL pi-ai cannot recognize; it
|
||||
# would otherwise be addressed as though it were OpenAI itself.
|
||||
compat:
|
||||
thinkingFormat: deepseek
|
||||
supportsDeveloperRole: false
|
||||
maxTokensField: max_tokens
|
||||
models:
|
||||
- id: acme-large
|
||||
name: Acme Large
|
||||
@@ -85,9 +88,11 @@ A profile's `models` list *replaces* the route's installed catalog rather than e
|
||||
|
||||
The declaration translates to pi-ai's `Model.reasoning` + `thinkingLevelMap` with every level decided explicitly — undeclared levels are pinned unsupported rather than left to pi-ai's own defaulting, which is asymmetric (an absent key means "supported" for the five base levels but "unsupported" for `xhigh`/`max`) and which a profile author should not need to know. `off` is the one three-state key: left out, selectors offer no Off and an explicit Off request is refused — a request naming no effort still goes out without the parameter, so what the provider then does is its own default; declared with no value (`off:`), Off is offered and selecting it sends nothing — for the `deepseek` dialect an explicit `thinking: {type: "disabled"}` — which also covers a request naming no effort at all; declared with a value (`off: none`), that value goes on the wire as the effort parameter. There is no spelling for restoring a catalog map key to "unset": the declaration is the whole offer, so restate the catalog levels you keep.
|
||||
|
||||
### Reasoning-dispatch compat switches
|
||||
### Wire-compatibility switches
|
||||
|
||||
How a thinking level travels — `reasoning_effort` alone, DeepSeek's `thinking: {type}` plus effort, z.ai's `thinking` object, and so on — is pi-ai's `compat.thinkingFormat`, which pi-ai guesses from the endpoint URL; a private gateway's URL says nothing, so a DeepSeek-dialect gateway would be spoken to in the OpenAI dialect with no way to correct it. `compat.thinkingFormat` and `compat.supportsReasoningEffort` are therefore configurable on the route (its models' default) and per model (winning per field), resolving model → route → installed catalog entry → pi-ai's URL-derived guess; setting a route-level switch shadows the catalog entry's value for every model on the route, and there is no spelling for handing a field back to the catalog short of restating its value. `thinkingFormat` accepts pi-ai's dispatchable formats except the two `chat-template` variants, which need `chatTemplateKwargs` this configuration does not expose. Both switches exist only on `openai-completions` — the other protocols carry their reasoning shape in the protocol itself — so a model-level switch elsewhere fails resolution, a route-level one skips models of other protocols, and a route with no `openai-completions` model at all is refused. The rest of pi-ai's compat surface (`supportsStore`, `maxTokensField`, …) stays auto-detected and is deliberately not configurable here.
|
||||
pi-ai shapes each request from the provider id and baseURL: which role carries the system prompt, which field caps output, how a thinking level travels. A private gateway's URL says nothing, and for an endpoint pi-ai does not recognize the detection answers as though it were OpenAI itself — a reasoning model's system prompt goes out as `developer`, the output cap as `max_completion_tokens`, the thinking level as a bare `reasoning_effort` — and most OpenAI-compatible gateways reject at least one of those. `compat` is therefore configurable on the route (its models' default) and per model (winning per field), resolving model → route → installed catalog entry → pi-ai's own detection; a route-level switch shadows the catalog entry's value for every model that reads it, and there is no spelling for handing a field back to the catalog short of restating its value.
|
||||
|
||||
Each switch belongs to the protocols whose pi-ai compat type declares it: `supportsDeveloperRole` is settable on an `openai-completions` or `openai-responses` route, `thinkingFormat` only on the former, `supportsTemperature` only on `anthropic-messages`. A model-level switch its protocol does not take fails resolution naming what that protocol does offer; a route-level one lands on the models that read it and skips the rest, and is refused only when no model on the route could read it at all. Two kinds of key are refused rather than dropped: one no protocol declares (a misspelling), and one pi-ai's installed catalog owns for a named vendor (`openRouterRouting`, `zaiToolStream`, `deferredToolsMode`, `sessionAffinityFormat`, `supportsOpenAIGrammarTools`, `supportsToolSearch`, `supportsExplicitPromptCacheMode`, `supportsToolReferences`, `vercelGatewayRouting`, `sendSessionAffinityHeaders`) — a route needing a vendor's own switch is a catalog route that should be named as such. The offered set is pinned to pi-ai's three compat types by drift gates, so an upgrade adding a field fails the build until someone classifies it.
|
||||
|
||||
A model neither the entry nor the installed catalog sizes takes the route's `defaultContextWindow` (262,144) and `defaultMaxTokens` (32,768), so a listing that discloses nothing but ids still yields a serviceable route. Both fallbacks are guesses by construction, which is why they are route fields a deployment whose gateway serves smaller models corrects once rather than constants buried in the adapter; the fallback sizes the model and never becomes a per-request cap.
|
||||
|
||||
|
||||
@@ -51,9 +51,12 @@
|
||||
apiKeyEnv: ACME_GATEWAY_API_KEY
|
||||
api: openai-completions
|
||||
baseURL: https://gateway.acme.example/v1
|
||||
# Reasoning dialect for an endpoint whose URL pi-ai cannot recognize.
|
||||
# Request shape for an endpoint whose URL pi-ai cannot recognize; it
|
||||
# would otherwise be addressed as though it were OpenAI itself.
|
||||
compat:
|
||||
thinkingFormat: deepseek
|
||||
supportsDeveloperRole: false
|
||||
maxTokensField: max_tokens
|
||||
models:
|
||||
- id: acme-large
|
||||
name: Acme Large
|
||||
@@ -85,9 +88,11 @@ profile 的 `models` 列表是*替换*该路由已安装 catalog，而不是扩
|
||||
|
||||
该声明会转换为 pi-ai 的 `Model.reasoning` + `thinkingLevelMap`，其中每个档位都被显式决定——未声明的档位一律固定为不支持，而不是留给 pi-ai 自己的默认规则：那套规则并不对称（键缺席对五个基础档位意味着「支持」，对 `xhigh`/`max` 却意味着「不支持」），也本不该要求 profile 作者了解。`off` 是唯一的三态键：不写它，选择器不提供 Off，显式请求 Off 会被拒绝——不点名任何档位的请求仍会在不带该参数的情况下发出，提供方随后做什么是它自己的默认行为；声明而不给值（`off:`），则会提供 Off，选中它时什么也不发送——对 `deepseek` 方言则是一个显式的 `thinking: {type: "disabled"}`——这同时覆盖完全不点名任何档位的请求；声明并给值（`off: none`），该值就会作为档位参数在协议中发送。没有任何写法能把 catalog 映射中的键恢复为「未设置」：这份声明就是对外提供的全部，因此把你要保留的 catalog 档位重述出来。
|
||||
|
||||
### 推理分派的 compat 开关
|
||||
### 协议兼容开关
|
||||
|
||||
思考级别如何在协议中传输——单独一个 `reasoning_effort`、DeepSeek 的 `thinking: {type}` 加上档位、z.ai 的 `thinking` 对象，诸如此类——就是 pi-ai 的 `compat.thinkingFormat`，pi-ai 会从端点 URL 猜测它；私有网关的 URL 什么也说明不了，于是说 DeepSeek 方言的网关只会收到 OpenAI 方言的请求，且无从更正。因此 `compat.thinkingFormat` 与 `compat.supportsReasoningEffort` 既可配置在路由上（作为其模型的默认值），也可按模型配置（逐字段胜出），解析顺序为模型 → 路由 → 已安装 catalog 条目 → pi-ai 按 URL 得出的猜测；设置路由级开关会为路由上的每个模型遮蔽 catalog 条目的值，而且除了重述其值，没有任何写法能把某个字段交还给 catalog。`thinkingFormat` 接受 pi-ai 可分派的各种格式，但不含两个 `chat-template` 变体：它们需要的 `chatTemplateKwargs` 本配置并不暴露。两个开关都只存在于 `openai-completions` 上——其余协议的推理形状由协议本身承载——因此在其他协议的模型上设置模型级开关会使解析失败，路由级开关会跳过其他协议的模型，而完全没有 `openai-completions` 模型的路由则会被拒绝。pi-ai compat 面的其余部分（`supportsStore`、`maxTokensField`……）保持自动检测，特意不在此处开放配置。
|
||||
pi-ai 依据提供方 id 与 baseURL 决定每个请求的形状：系统提示词由哪个角色承载、输出上限写在哪个字段、思考级别如何传输。私有网关的 URL 什么也说明不了，而对于 pi-ai 无法识别的端点，其检测会当作 OpenAI 本身来回答——推理模型的系统提示词以 `developer` 发出、输出上限写作 `max_completion_tokens`、思考级别只发一个裸的 `reasoning_effort`——而多数 OpenAI 兼容网关至少会拒绝其中之一。因此 `compat` 既可配置在路由上（作为其模型的默认值），也可按模型配置（逐字段胜出），解析顺序为模型 → 路由 → 已安装 catalog 条目 → pi-ai 自身的检测；路由级开关会为每个读取它的模型遮蔽 catalog 条目的值，而且除了重述其值，没有任何写法能把某个字段交还给 catalog。
|
||||
|
||||
每个开关归属于其 pi-ai compat 类型声明了它的那些协议：`supportsDeveloperRole` 可设在 `openai-completions` 或 `openai-responses` 路由上，`thinkingFormat` 只能设在前者，`supportsTemperature` 只能设在 `anthropic-messages` 上。模型级开关若其协议并不接受，解析失败并点名该协议实际提供哪些开关；路由级开关则落在读取它的模型上、跳过其余模型，只有当路由上没有任何模型能读取它时才被拒绝。两类键会被拒绝而非丢弃：没有任何协议声明的键（笔误），以及 pi-ai 已安装 catalog 为具名厂商掌管的键（`openRouterRouting`、`zaiToolStream`、`deferredToolsMode`、`sessionAffinityFormat`、`supportsOpenAIGrammarTools`、`supportsToolSearch`、`supportsExplicitPromptCacheMode`、`supportsToolReferences`、`vercelGatewayRouting`、`sendSessionAffinityHeaders`）——需要某厂商专属开关的路由，本就是一条应当以该厂商命名的 catalog 路由。开放集由漂移门禁钉在 pi-ai 的三个 compat 类型上，因此上游新增字段会使构建失败，直到有人为它做出分类。
|
||||
|
||||
条目与已安装 catalog 都没有给出尺寸的模型，会采用该路由的 `defaultContextWindow`（262,144）与 `defaultMaxTokens`（32,768），因此一份只公布 id 的列表同样能产出可服务的路由。两个回退值本质上都是猜测，这正是它们作为路由字段、供网关服务更小模型的部署一次性更正的原因，而不是埋在适配器里的常量；回退值只用于给模型定尺寸，绝不会变成单次请求上限。
|
||||
|
||||
|
||||
@@ -15,11 +15,14 @@
|
||||
import { builtinProviders, getBuiltinModels, getBuiltinProviders } from '@earendil-works/pi-ai/providers/all'
|
||||
import type { BuiltinProvider } from '@earendil-works/pi-ai/providers/all'
|
||||
import type {
|
||||
AnthropicMessagesCompat,
|
||||
Api,
|
||||
ChatTemplateKwargValue,
|
||||
Model,
|
||||
ModelCost,
|
||||
ModelThinkingLevel,
|
||||
OpenAICompletionsCompat,
|
||||
OpenAIResponsesCompat,
|
||||
Provider,
|
||||
ThinkingLevelMap,
|
||||
} from '@earendil-works/pi-ai'
|
||||
@@ -79,23 +82,16 @@ const THINKING_LEVEL_GATE: Record<ModelThinkingLevel, true> = {
|
||||
/** Every pi-ai thinking level a profile may declare, in escalation order. */
|
||||
export const THINKING_LEVELS = Object.keys(THINKING_LEVEL_GATE) as readonly ModelThinkingLevel[]
|
||||
|
||||
/** The `compat.thinkingFormat` spellings pi-ai accepts on an `openai-completions` model. */
|
||||
type PiThinkingFormat = NonNullable<OpenAICompletionsCompat['thinkingFormat']>
|
||||
|
||||
/**
|
||||
* pi-ai thinking formats a profile cannot name: both drive the request through
|
||||
* `chatTemplateKwargs`, which this configuration does not expose.
|
||||
*/
|
||||
type WithheldThinkingFormat = 'chat-template' | 'qwen-chat-template'
|
||||
|
||||
/** One reasoning-dispatch wire format a profile may name. */
|
||||
export type PiAiThinkingFormat = Exclude<PiThinkingFormat, WithheldThinkingFormat>
|
||||
export type PiAiThinkingFormat = NonNullable<OpenAICompletionsCompat['thinkingFormat']>
|
||||
|
||||
/**
|
||||
* The nameable reasoning-dispatch formats, most-reached first. The `Record`
|
||||
* key type is a drift gate: a pi-ai upgrade that adds a format (0.84 added
|
||||
* `baseten`) fails compilation here until the format is classified as offered
|
||||
* here or withheld above, so the offer never silently lags the upstream set.
|
||||
* `baseten`) fails compilation here until the new format is named, so the
|
||||
* offer never silently lags the upstream set. The two `chat-template` variants
|
||||
* are nameable because {@link PiAiCompatProfile.chatTemplateKwargs} carries
|
||||
* the kwargs they dispatch through.
|
||||
*/
|
||||
const THINKING_FORMAT_GATE: Record<PiAiThinkingFormat, true> = {
|
||||
'openai': true,
|
||||
@@ -104,6 +100,8 @@ const THINKING_FORMAT_GATE: Record<PiAiThinkingFormat, true> = {
|
||||
'together': true,
|
||||
'zai': true,
|
||||
'qwen': true,
|
||||
'chat-template': true,
|
||||
'qwen-chat-template': true,
|
||||
'string-thinking': true,
|
||||
'ant-ling': true,
|
||||
}
|
||||
@@ -183,19 +181,251 @@ export function catalogModels(provider: string): Map<string, Model<Api>> {
|
||||
export type PiAiReasoningEfforts = Partial<Record<ModelThinkingLevel, string | null>>
|
||||
|
||||
/**
|
||||
* Reasoning-dispatch compatibility switches, set on the route (its models'
|
||||
* default) or per model (winning over the route). Only the switches pi-ai's
|
||||
* reasoning dispatch reads are offered; the rest of pi-ai's compat surface
|
||||
* keeps its baseURL-derived auto-detection. pi-ai types both fields only on
|
||||
* `OpenAICompletionsCompat` — the other wire protocols define their reasoning
|
||||
* fields in the protocol itself — so resolution rejects a model-level switch
|
||||
* anywhere else, while a route-level default skips past models it cannot fit.
|
||||
* Whether one pi-ai compat field is configurable on a profile.
|
||||
*
|
||||
* `withhold` is the disposition for a field pi-ai's installed catalog already
|
||||
* sets for a named vendor. Reaching for one of those on a hand-declared route
|
||||
* means configuring a provider that should have been named as a catalog route
|
||||
* instead, where the installed entry carries the right value already.
|
||||
*/
|
||||
type CompatDisposition = 'offer' | 'withhold'
|
||||
|
||||
/**
|
||||
* Disposition of every `OpenAICompletionsCompat` field. The `Record` key type
|
||||
* is a drift gate: a pi-ai upgrade that adds a field fails compilation here
|
||||
* until it is classified, so the offer never silently lags the upstream set.
|
||||
*/
|
||||
const COMPLETIONS_COMPAT_GATE = {
|
||||
supportsStore: 'offer',
|
||||
supportsDeveloperRole: 'offer',
|
||||
supportsReasoningEffort: 'offer',
|
||||
supportsUsageInStreaming: 'offer',
|
||||
maxTokensField: 'offer',
|
||||
requiresToolResultName: 'offer',
|
||||
requiresAssistantAfterToolResult: 'offer',
|
||||
requiresThinkingAsText: 'offer',
|
||||
requiresReasoningContentOnAssistantMessages: 'offer',
|
||||
thinkingFormat: 'offer',
|
||||
chatTemplateKwargs: 'offer',
|
||||
supportsStrictMode: 'offer',
|
||||
cacheControlFormat: 'offer',
|
||||
supportsLongCacheRetention: 'offer',
|
||||
openRouterRouting: 'withhold',
|
||||
vercelGatewayRouting: 'withhold',
|
||||
zaiToolStream: 'withhold',
|
||||
supportsOpenAIGrammarTools: 'withhold',
|
||||
sendSessionAffinityHeaders: 'withhold',
|
||||
deferredToolsMode: 'withhold',
|
||||
sessionAffinityFormat: 'withhold',
|
||||
} as const satisfies Record<keyof OpenAICompletionsCompat, CompatDisposition>
|
||||
|
||||
/** Disposition of every `OpenAIResponsesCompat` field; a drift gate like the one above. */
|
||||
const RESPONSES_COMPAT_GATE = {
|
||||
supportsDeveloperRole: 'offer',
|
||||
supportsStrictMode: 'offer',
|
||||
supportsLongCacheRetention: 'offer',
|
||||
sessionAffinityFormat: 'withhold',
|
||||
supportsOpenAIGrammarTools: 'withhold',
|
||||
supportsToolSearch: 'withhold',
|
||||
supportsExplicitPromptCacheMode: 'withhold',
|
||||
} as const satisfies Record<keyof OpenAIResponsesCompat, CompatDisposition>
|
||||
|
||||
/** Disposition of every `AnthropicMessagesCompat` field; a drift gate like the one above. */
|
||||
const ANTHROPIC_COMPAT_GATE = {
|
||||
supportsEagerToolInputStreaming: 'offer',
|
||||
supportsLongCacheRetention: 'offer',
|
||||
supportsCacheControlOnTools: 'offer',
|
||||
supportsTemperature: 'offer',
|
||||
forceAdaptiveThinking: 'offer',
|
||||
allowEmptySignature: 'offer',
|
||||
supportsStrictTools: 'offer',
|
||||
sendSessionAffinityHeaders: 'withhold',
|
||||
supportsToolReferences: 'withhold',
|
||||
} as const satisfies Record<keyof AnthropicMessagesCompat, CompatDisposition>
|
||||
|
||||
/**
|
||||
* The compat gate of every wire protocol a profile may configure, in the
|
||||
* protocol table's order. A protocol absent here takes no configured compat,
|
||||
* which is why an unrecognized `api` refuses every switch rather than
|
||||
* silently dropping it.
|
||||
*/
|
||||
const COMPAT_GATES: Readonly<Record<string, Readonly<Record<string, CompatDisposition>>>> = {
|
||||
'openai-completions': COMPLETIONS_COMPAT_GATE,
|
||||
'openai-responses': RESPONSES_COMPAT_GATE,
|
||||
'anthropic-messages': ANTHROPIC_COMPAT_GATE,
|
||||
}
|
||||
|
||||
/** The field names one gate offers. */
|
||||
type OfferedIn<G> = { [K in keyof G]: G[K] extends 'offer' ? K : never }[keyof G]
|
||||
|
||||
/** Every compat field name a profile may set, on whichever protocol takes it. */
|
||||
type OfferedCompatField =
|
||||
| OfferedIn<typeof COMPLETIONS_COMPAT_GATE>
|
||||
| OfferedIn<typeof RESPONSES_COMPAT_GATE>
|
||||
| OfferedIn<typeof ANTHROPIC_COMPAT_GATE>
|
||||
|
||||
/**
|
||||
* pi-ai wire-compatibility switches, set on the route (its models' default) or
|
||||
* per model (winning over the route, field by field).
|
||||
*
|
||||
* pi-ai decides each of these from the provider id and baseURL when no layer
|
||||
* sets it, and a private gateway's URL says nothing: for an endpoint it does
|
||||
* not recognize the detection answers as though it were OpenAI itself, which
|
||||
* is wrong for most OpenAI-compatible gateways. So every field here is one a
|
||||
* deployment must be able to state because nothing can infer it, while the
|
||||
* fields pi-ai's catalog sets for a named vendor stay withheld.
|
||||
*
|
||||
* A field belongs to the protocols whose upstream compat type declares it: a
|
||||
* model-level switch its protocol does not take fails resolution, and a
|
||||
* route-level one skips past models it cannot fit.
|
||||
*/
|
||||
export interface PiAiCompatProfile {
|
||||
/** Reasoning parameter format the endpoint expects; absent keeps the catalog entry's, then pi-ai's baseURL-derived guess. */
|
||||
thinkingFormat?: PiAiThinkingFormat
|
||||
/** Whether the endpoint accepts `reasoning_effort`; absent keeps the catalog entry's, then pi-ai's baseURL-derived guess. */
|
||||
/** Whether the endpoint accepts `store`; `openai-completions`. */
|
||||
supportsStore?: boolean
|
||||
/**
|
||||
* Whether the endpoint accepts the `developer` role for the system prompt,
|
||||
* which pi-ai sends only to a reasoning model; `false` keeps `system`.
|
||||
* `openai-completions`, `openai-responses`.
|
||||
*/
|
||||
supportsDeveloperRole?: boolean
|
||||
/** Whether the endpoint accepts `reasoning_effort`; `openai-completions`. */
|
||||
supportsReasoningEffort?: boolean
|
||||
/** Whether the endpoint accepts `stream_options: {include_usage: true}`; `openai-completions`. */
|
||||
supportsUsageInStreaming?: boolean
|
||||
/** Which output-cap field the endpoint reads; `openai-completions`. */
|
||||
maxTokensField?: 'max_completion_tokens' | 'max_tokens'
|
||||
/** Whether tool results must carry `name`; `openai-completions`. */
|
||||
requiresToolResultName?: boolean
|
||||
/** Whether a user message after tool results needs an assistant message between; `openai-completions`. */
|
||||
requiresAssistantAfterToolResult?: boolean
|
||||
/** Whether thinking blocks must travel as text in `<thinking>` delimiters; `openai-completions`. */
|
||||
requiresThinkingAsText?: boolean
|
||||
/** Whether replayed assistant messages need an empty `reasoning_content` while reasoning is on; `openai-completions`. */
|
||||
requiresReasoningContentOnAssistantMessages?: boolean
|
||||
/** Reasoning parameter format the endpoint expects; `openai-completions`. */
|
||||
thinkingFormat?: PiAiThinkingFormat
|
||||
/** Kwargs sent as `chat_template_kwargs`, for the two `chat-template` thinking formats; `openai-completions`. */
|
||||
chatTemplateKwargs?: Record<string, ChatTemplateKwargValue>
|
||||
/** Whether the endpoint accepts `strict` in tool definitions; `openai-completions`, `openai-responses`. */
|
||||
supportsStrictMode?: boolean
|
||||
/** Prompt-cache marker convention; `openai-completions`. */
|
||||
cacheControlFormat?: 'anthropic'
|
||||
/** Whether the endpoint accepts long prompt-cache retention; all three protocols. */
|
||||
supportsLongCacheRetention?: boolean
|
||||
/** Whether the endpoint accepts per-tool `eager_input_streaming`; `anthropic-messages`. */
|
||||
supportsEagerToolInputStreaming?: boolean
|
||||
/** Whether the endpoint accepts `cache_control` on tool definitions; `anthropic-messages`. */
|
||||
supportsCacheControlOnTools?: boolean
|
||||
/** Whether the endpoint accepts the `temperature` request field; `anthropic-messages`. */
|
||||
supportsTemperature?: boolean
|
||||
/** Whether to force adaptive thinking regardless of model id; `anthropic-messages`. */
|
||||
forceAdaptiveThinking?: boolean
|
||||
/** Whether to replay an empty thinking signature instead of converting thinking to text; `anthropic-messages`. */
|
||||
allowEmptySignature?: boolean
|
||||
/** Whether the endpoint accepts Anthropic strict tool schemas; `anthropic-messages`. */
|
||||
supportsStrictTools?: boolean
|
||||
}
|
||||
|
||||
/** Compile-time constraint that `T` is `never`. */
|
||||
type AssertNever<T extends never> = T
|
||||
|
||||
/**
|
||||
* Proof that every documented field is one a gate offers. A field the profile
|
||||
* declares past the gates fails compilation with its own name in the error.
|
||||
*/
|
||||
export type EveryProfileFieldIsOffered = AssertNever<Exclude<keyof PiAiCompatProfile, OfferedCompatField>>
|
||||
|
||||
/**
|
||||
* Proof that every offered field is documented. A gate entry flipped to
|
||||
* `offer` without a profile field fails compilation with its own name in the
|
||||
* error, which is the half a schema alone cannot catch.
|
||||
*/
|
||||
export type EveryOfferedFieldIsDocumented = AssertNever<Exclude<OfferedCompatField, keyof PiAiCompatProfile>>
|
||||
|
||||
/**
|
||||
* The compat entries a profile actually set.
|
||||
*
|
||||
* schemastery materializes an absent dict as `{}` — the behavior
|
||||
* `reasoningEfforts` works around with a union — so every parsed profile
|
||||
* carries a `chatTemplateKwargs` key whether or not anyone wrote one. An empty
|
||||
* one states nothing here: it would send no kwargs, which is exactly what
|
||||
* leaving the field out does, so absent and empty are the same request and
|
||||
* neither may make a route look like it configured a switch.
|
||||
* @param compat - the configured switches, when any.
|
||||
* @returns the entries carrying a value, in declaration order.
|
||||
*/
|
||||
function configuredCompatEntries(compat: PiAiCompatProfile | undefined): readonly (readonly [string, unknown])[] {
|
||||
return Object.entries(compat ?? {}).flatMap(([field, value]) => {
|
||||
if (value === undefined) return []
|
||||
const empty = typeof value === 'object' && value !== null && !Array.isArray(value)
|
||||
&& Object.keys(value as object).length === 0
|
||||
return empty ? [] : [[field, value] as const]
|
||||
})
|
||||
}
|
||||
|
||||
/**
|
||||
* The protocols offering one compat field, in {@link COMPAT_GATES} order.
|
||||
* @param field - configured compat field name.
|
||||
* @returns the protocols whose compat takes it; empty when none does, which
|
||||
* is either a withheld field or a name no upstream compat type declares.
|
||||
*/
|
||||
function compatProtocols(field: string): readonly string[] {
|
||||
return Object.entries(COMPAT_GATES).flatMap(([api, gate]) => gate[field] === 'offer' ? [api] : [])
|
||||
}
|
||||
|
||||
/**
|
||||
* The compat fields one protocol offers, for a diagnostic that has to show
|
||||
* what was available instead of the name that missed.
|
||||
* @param api - wire protocol.
|
||||
* @returns the offered field names, or an empty list for a protocol taking no compat.
|
||||
*/
|
||||
function offeredCompatFields(api: string): readonly string[] {
|
||||
return Object.entries(COMPAT_GATES[api] ?? {}).flatMap(([field, disposition]) => disposition === 'offer' ? [field] : [])
|
||||
}
|
||||
|
||||
/**
|
||||
* Every offered field name, deduplicated, for the one diagnostic that cannot
|
||||
* narrow by protocol: the vocabulary check runs before any protocol resolves,
|
||||
* which is what lets it refuse a misspelling on a route whose models would
|
||||
* never have reached the protocol that declares the intended field.
|
||||
* @returns the offered field names across every protocol, in gate order.
|
||||
*/
|
||||
function allOfferedCompatFields(): readonly string[] {
|
||||
const fields = new Set<string>()
|
||||
for (const api of Object.keys(COMPAT_GATES)) {
|
||||
for (const field of offeredCompatFields(api)) fields.add(field)
|
||||
}
|
||||
return [...fields]
|
||||
}
|
||||
|
||||
/**
|
||||
* Reject a compat key no protocol offers. Runs before any protocol is
|
||||
* resolved, so a withheld field or a misspelling fails even on a route whose
|
||||
* models never reach the protocol that would have taken it — the alternative
|
||||
* being the silent drop that let an unreadable switch look applied.
|
||||
* @param provider - provider route key, for diagnostics.
|
||||
* @param site - the configuration site, for diagnostics.
|
||||
* @param compat - the configured switches, when any.
|
||||
* @throws Error naming the offending key.
|
||||
*/
|
||||
function assertOfferedCompatFields(
|
||||
provider: string,
|
||||
site: string,
|
||||
compat: PiAiCompatProfile | undefined,
|
||||
): void {
|
||||
// Every key, not only the ones carrying a value: a withheld or undeclared
|
||||
// name is never in the schema, so schemastery cannot have materialized it —
|
||||
// whatever its value, a person wrote it and expects it to do something.
|
||||
for (const field of Object.keys(compat ?? {})) {
|
||||
if (compatProtocols(field).length > 0) continue
|
||||
const declared = Object.values(COMPAT_GATES).some(gate => gate[field] !== undefined)
|
||||
if (declared) {
|
||||
invalid(provider, `${site} sets compat "${field}", which is not configurable here: pi-ai's installed`
|
||||
+ ' catalog sets it for the vendors that need it, so name that provider as the route instead')
|
||||
}
|
||||
invalid(provider, `${site} sets compat "${field}", which no wire protocol declares; the configurable`
|
||||
+ ` switches are ${allOfferedCompatFields().join(', ')}`)
|
||||
}
|
||||
}
|
||||
|
||||
/** One configured model entry: an id plus the catalog fields it overrides. */
|
||||
@@ -233,7 +463,7 @@ export interface PiAiModelProfile {
|
||||
* declares the offered levels and their wire spellings.
|
||||
*/
|
||||
reasoningEfforts?: false | PiAiReasoningEfforts
|
||||
/** Reasoning-dispatch switches for this model, winning over the route's. */
|
||||
/** pi-ai wire-compatibility switches for this model, winning over the route's per field; one its protocol does not declare is refused. */
|
||||
compat?: PiAiCompatProfile
|
||||
}
|
||||
|
||||
@@ -258,7 +488,7 @@ export interface RouteCatalogRequest {
|
||||
models?: readonly PiAiModelProfile[]
|
||||
/** Installed-catalog customizations by model id; only meaningful while `models` is absent. */
|
||||
modelOverrides?: Readonly<Record<string, PiAiModelOverride>>
|
||||
/** Reasoning-dispatch switches for every `openai-completions` model on the route; entries override per field. */
|
||||
/** Route-level wire-compatibility switches, landing on each model whose protocol declares them; entries override per field. */
|
||||
compat?: PiAiCompatProfile
|
||||
/** Context capacity for a model neither the entry nor the catalog sizes. */
|
||||
defaultContextWindow: number
|
||||
@@ -368,16 +598,20 @@ function resolveModelReasoning(
|
||||
return { reasoning: true, thinkingLevelMap: map }
|
||||
}
|
||||
|
||||
/** The compat block a materialized model carries, whichever protocol it speaks. */
|
||||
type ModelCompat = OpenAICompletionsCompat | OpenAIResponsesCompat | AnthropicMessagesCompat
|
||||
|
||||
/**
|
||||
* Resolve one model's compat block from the profile's reasoning switches.
|
||||
* Resolve one model's compat block from the profile's switches.
|
||||
*
|
||||
* A model switch wins over the route switch; whatever neither sets keeps the
|
||||
* installed entry's value, and a field no layer decides falls through to
|
||||
* pi-ai's baseURL-derived detection. Only an `openai-completions` model takes
|
||||
* the switches at all: a model-level switch on any other protocol fails
|
||||
* resolution, while a route-level default skips past such models — the same
|
||||
* posture as the route-level `reasoning` default, which also must not fail
|
||||
* models it does not fit.
|
||||
* A model switch wins over the route switch field by field; whatever neither
|
||||
* sets keeps the installed entry's value, and a field no layer decides falls
|
||||
* through to pi-ai's own detection. A model-level switch its protocol does not
|
||||
* take fails resolution — about one named model it can only be a mistake —
|
||||
* while a route-level one skips past such models, since a route default must
|
||||
* stay settable on a route whose models do not all speak one protocol. Every
|
||||
* field reaching here is offered by some protocol; {@link
|
||||
* assertOfferedCompatFields} has already refused the rest.
|
||||
* @param provider - provider route key, for diagnostics.
|
||||
* @param entry - the configured model entry.
|
||||
* @param route - the route-level switches, when any.
|
||||
@@ -391,31 +625,31 @@ function resolveModelCompat(
|
||||
route: PiAiCompatProfile | undefined,
|
||||
base: Model<Api> | undefined,
|
||||
api: string,
|
||||
): { compat: OpenAICompletionsCompat } | Record<string, never> {
|
||||
const thinkingFormat = entry.compat?.thinkingFormat ?? route?.thinkingFormat
|
||||
const supportsReasoningEffort = entry.compat?.supportsReasoningEffort ?? route?.supportsReasoningEffort
|
||||
if (thinkingFormat === undefined && supportsReasoningEffort === undefined) return {}
|
||||
if (api !== 'openai-completions') {
|
||||
if (entry.compat?.thinkingFormat !== undefined || entry.compat?.supportsReasoningEffort !== undefined) {
|
||||
invalid(provider, `model "${entry.id}" sets compat reasoning switches, but its api is "${api}";`
|
||||
+ ' thinkingFormat and supportsReasoningEffort exist only on openai-completions')
|
||||
}
|
||||
return {}
|
||||
): { compat: ModelCompat } | Record<string, never> {
|
||||
const gate = COMPAT_GATES[api]
|
||||
const configured: Record<string, unknown> = {}
|
||||
for (const [field, value] of configuredCompatEntries(route)) {
|
||||
if (gate?.[field] !== 'offer') continue
|
||||
configured[field] = value
|
||||
}
|
||||
for (const [field, value] of configuredCompatEntries(entry.compat)) {
|
||||
if (gate?.[field] !== 'offer') {
|
||||
const offered = offeredCompatFields(api)
|
||||
invalid(provider, `model "${entry.id}" sets compat "${field}", but its api is "${api}", which does not`
|
||||
+ ` take it; that switch exists on ${compatProtocols(field).join(', ')}, and "${api}" offers`
|
||||
+ ` ${offered.length === 0 ? 'no configurable compat' : offered.join(', ')}`)
|
||||
}
|
||||
configured[field] = value
|
||||
}
|
||||
if (Object.keys(configured).length === 0) return {}
|
||||
// The installed entry's compat matches the entry's OWN api — a route-level
|
||||
// `api` repoint (an anthropic catalog served through an OpenAI-compatible
|
||||
// gateway) leaves `base.compat` in the other protocol's shape, so it is
|
||||
// inherited only while the resolved api still is the entry's. A repointed
|
||||
// model starts from pi-ai's baseURL-derived detection instead, which is
|
||||
// what a protocol change means for every other compat field too.
|
||||
const inherited: OpenAICompletionsCompat | undefined = base?.api === api ? base.compat : undefined
|
||||
return {
|
||||
compat: {
|
||||
...inherited,
|
||||
...thinkingFormat === undefined ? {} : { thinkingFormat },
|
||||
...supportsReasoningEffort === undefined ? {} : { supportsReasoningEffort },
|
||||
},
|
||||
}
|
||||
const inherited = base?.api === api ? base.compat : undefined
|
||||
return { compat: { ...inherited, ...configured } as ModelCompat }
|
||||
}
|
||||
|
||||
/** One route's materialized catalog, plus the request caps its profile chose. */
|
||||
@@ -485,8 +719,13 @@ export function resolveRouteModels(request: RouteCatalogRequest): RouteCatalog {
|
||||
+ ' must be listed in configuration')
|
||||
}
|
||||
const routeApi = sharedCatalogApi(defaults)
|
||||
const routeCompatDefined = request.compat?.thinkingFormat !== undefined
|
||||
|| request.compat?.supportsReasoningEffort !== undefined
|
||||
// Vocabulary before protocols: a withheld or undeclared switch is refused
|
||||
// wherever it is written, so it cannot look applied on a route whose models
|
||||
// never reach the protocol that would have taken it.
|
||||
assertOfferedCompatFields(provider, 'route', request.compat)
|
||||
for (const entry of entries) {
|
||||
assertOfferedCompatFields(provider, `model "${entry.id}"`, entry.compat)
|
||||
}
|
||||
const seen = new Set<string>()
|
||||
const configuredMaxTokens = new Map<string, number>()
|
||||
const models = entries.map((entry) => {
|
||||
@@ -538,9 +777,15 @@ export function resolveRouteModels(request: RouteCatalogRequest): RouteCatalog {
|
||||
...resolveModelCompat(provider, entry, request.compat, base, api),
|
||||
}
|
||||
})
|
||||
if (routeCompatDefined && !models.some(model => model.api === 'openai-completions')) {
|
||||
invalid(provider, 'sets compat reasoning switches, but no model on the route speaks openai-completions;'
|
||||
+ ' thinkingFormat and supportsReasoningEffort exist only on that protocol')
|
||||
// Per field, not per block: a route may default a switch its completions
|
||||
// models take beside one only its anthropic models do, and neither should
|
||||
// fail for the other's sake. What is refused is a route default no model on
|
||||
// the route could ever read, which is a route that will not behave as written.
|
||||
for (const [field] of configuredCompatEntries(request.compat)) {
|
||||
const takers = compatProtocols(field)
|
||||
if (models.some(model => takers.includes(model.api))) continue
|
||||
invalid(provider, `sets compat "${field}", but no model on the route speaks a protocol that takes it;`
|
||||
+ ` it exists on ${takers.join(', ')}`)
|
||||
}
|
||||
return { models, configuredMaxTokens }
|
||||
}
|
||||
|
||||
@@ -14,7 +14,7 @@
|
||||
* @module dsh-llm-pi-ai/config
|
||||
*/
|
||||
|
||||
import type { CacheRetention, ModelThinkingLevel, Provider, ThinkingBudgets, Transport } from '@earendil-works/pi-ai'
|
||||
import type { CacheRetention, ChatTemplateKwargValue, ModelThinkingLevel, Provider, ThinkingBudgets, Transport } from '@earendil-works/pi-ai'
|
||||
import z from '@deepseek-ai/schemastery'
|
||||
import { credentialRef } from '@deepseek-ai/dsh-credentials'
|
||||
import type { CredentialRef } from '@deepseek-ai/dsh-credentials'
|
||||
@@ -91,10 +91,11 @@ export interface PiAiProviderProfile {
|
||||
*/
|
||||
modelOverrides?: Record<string, PiAiModelOverride>
|
||||
/**
|
||||
* Reasoning-dispatch switches for every `openai-completions` model on this
|
||||
* route; each model's own `compat` overrides per field. What neither sets
|
||||
* keeps the installed catalog entry's value, then pi-ai's baseURL-derived
|
||||
* detection.
|
||||
* pi-ai wire-compatibility switches defaulting every model on this route
|
||||
* whose protocol declares them; each model's own `compat` overrides per
|
||||
* field. What neither sets keeps the installed catalog entry's value, then
|
||||
* pi-ai's own detection. A switch no model on the route could read is
|
||||
* refused rather than left looking applied.
|
||||
*/
|
||||
compat?: PiAiCompatProfile
|
||||
/**
|
||||
@@ -185,9 +186,43 @@ const thinkingBudgets = z.object({
|
||||
high: z.number(),
|
||||
})
|
||||
|
||||
/**
|
||||
* One `chat_template_kwargs` value. The `$var` member is pi-ai's placeholder
|
||||
* for a value dispatch fills from the request's thinking state, which is what
|
||||
* makes a chat-template gateway configurable without restating its template.
|
||||
*/
|
||||
const chatTemplateKwarg: z<ChatTemplateKwargValue> = z.union([
|
||||
z.string(),
|
||||
z.number(),
|
||||
z.boolean(),
|
||||
z.const(null),
|
||||
z.object({
|
||||
$var: z.union(['thinking.enabled', 'thinking.effort'] as const).required(),
|
||||
omitWhenOff: z.boolean(),
|
||||
}),
|
||||
])
|
||||
|
||||
const compatProfile: z<PiAiCompatProfile> = z.object({
|
||||
thinkingFormat: z.union(SUPPORTED_THINKING_FORMATS),
|
||||
supportsStore: z.boolean(),
|
||||
supportsDeveloperRole: z.boolean(),
|
||||
supportsReasoningEffort: z.boolean(),
|
||||
supportsUsageInStreaming: z.boolean(),
|
||||
maxTokensField: z.union(['max_completion_tokens', 'max_tokens'] as const),
|
||||
requiresToolResultName: z.boolean(),
|
||||
requiresAssistantAfterToolResult: z.boolean(),
|
||||
requiresThinkingAsText: z.boolean(),
|
||||
requiresReasoningContentOnAssistantMessages: z.boolean(),
|
||||
thinkingFormat: z.union(SUPPORTED_THINKING_FORMATS),
|
||||
chatTemplateKwargs: z.dict(chatTemplateKwarg),
|
||||
supportsStrictMode: z.boolean(),
|
||||
cacheControlFormat: z.union(['anthropic'] as const),
|
||||
supportsLongCacheRetention: z.boolean(),
|
||||
supportsEagerToolInputStreaming: z.boolean(),
|
||||
supportsCacheControlOnTools: z.boolean(),
|
||||
supportsTemperature: z.boolean(),
|
||||
forceAdaptiveThinking: z.boolean(),
|
||||
allowEmptySignature: z.boolean(),
|
||||
supportsStrictTools: z.boolean(),
|
||||
})
|
||||
|
||||
/**
|
||||
|
||||
@@ -596,6 +596,46 @@ describe('provider profile lifecycle', () => {
|
||||
expect(server.requests[1]).not.toHaveProperty('reasoning_effort')
|
||||
})
|
||||
|
||||
it('keeps the system role on a declared route whose gateway rejects the developer one', async () => {
|
||||
vi.stubEnv('PI_TEST_KEY', 'test-key')
|
||||
const server = await mockServer([{ events: textEvents }, { events: textEvents }])
|
||||
const ctx = new Context()
|
||||
await ctx.plugin(LlmRuntime)
|
||||
await ctx.plugin(LlmPiAi, {
|
||||
providers: {
|
||||
'acme-gateway': {
|
||||
apiKeyEnv: 'PI_TEST_KEY',
|
||||
api: 'openai-completions',
|
||||
baseURL: `${server.url}/v1`,
|
||||
models: [
|
||||
// pi-ai sends the system prompt as `developer` to a reasoning
|
||||
// model whenever its URL detection says the endpoint is OpenAI —
|
||||
// which is what an unrecognized private URL resolves to. Most
|
||||
// OpenAI-compatible gateways reject that role.
|
||||
{ id: 'acme-think', reasoningEfforts: { off: null, high: 'high' }, compat: { supportsDeveloperRole: false } },
|
||||
{ id: 'acme-guess', reasoningEfforts: { off: null, high: 'high' } },
|
||||
],
|
||||
},
|
||||
},
|
||||
})
|
||||
const roles = async (model: string): Promise<string[]> => {
|
||||
await assemble(ctx, {
|
||||
provider: 'acme-gateway',
|
||||
model,
|
||||
reasoningEffort: ReasoningEffortId('high'),
|
||||
system: 'you are a harness',
|
||||
messages: [],
|
||||
})
|
||||
const request = server.requests.at(-1) as { messages: { role: string }[] }
|
||||
return request.messages.map(message => message.role)
|
||||
}
|
||||
|
||||
expect(await roles('acme-think')).toEqual(['system'])
|
||||
// The switch is the only thing that changes it: the same route, same
|
||||
// endpoint, same reasoning declaration still gets pi-ai's guess.
|
||||
expect(await roles('acme-guess')).toEqual(['developer'])
|
||||
})
|
||||
|
||||
it('sends a declared off value as the effort parameter instead of omitting it', async () => {
|
||||
vi.stubEnv('PI_TEST_KEY', 'test-key')
|
||||
const server = await mockServer([{ events: textEvents }])
|
||||
|
||||
@@ -755,7 +755,7 @@ describe('modelOverrides', () => {
|
||||
})
|
||||
})
|
||||
|
||||
describe('reasoning-dispatch compat switches', () => {
|
||||
describe('compat switches', () => {
|
||||
/** The materialized models of one route, keyed by id. */
|
||||
function modelsOf(providers: Record<string, LlmPiAi.PiAiProviderProfile>, route: string): Map<string, Model<Api>> {
|
||||
const models = resolveProfiles(providers).get(route)?.piProvider.getModels() ?? []
|
||||
@@ -813,18 +813,133 @@ describe('reasoning-dispatch compat switches', () => {
|
||||
expect(models.get(responses.id)?.compat).toEqual(responses.compat)
|
||||
})
|
||||
|
||||
it('rejects a model-level switch on a protocol that has no such field', () => {
|
||||
it('rejects a model-level switch on a protocol that has no such field, naming what it offers', () => {
|
||||
expect(() => resolveProfiles({
|
||||
anthropic: {
|
||||
models: [{ id: 'claude-sonnet-4-5', compat: { thinkingFormat: 'openai' } }],
|
||||
},
|
||||
})).toThrow(/exist only on openai-completions/)
|
||||
})).toThrow(/its api is "anthropic-messages", which does not take it.*exists on openai-completions/s)
|
||||
})
|
||||
|
||||
it('rejects route switches no model on the route can take', () => {
|
||||
expect(() => resolveProfiles({
|
||||
anthropic: { compat: { thinkingFormat: 'openai' } },
|
||||
})).toThrow(/no model on the route speaks openai-completions/)
|
||||
})).toThrow(/no model on the route speaks a protocol that takes it/)
|
||||
})
|
||||
|
||||
it('carries the developer-role switch onto a hand-declared reasoning model', () => {
|
||||
// pi-ai reads this switch only for a reasoning model, and detects it from
|
||||
// the endpoint URL — which for a private gateway answers as though it were
|
||||
// OpenAI itself, so the route must be able to say otherwise.
|
||||
const models = modelsOf({
|
||||
'acme-gateway': {
|
||||
api: 'openai-completions',
|
||||
baseURL: 'https://acme.test',
|
||||
compat: { supportsDeveloperRole: false, maxTokensField: 'max_tokens' },
|
||||
models: [{ id: 'acme-think', reasoningEfforts: { off: null, high: 'high' } }],
|
||||
},
|
||||
}, 'acme-gateway')
|
||||
|
||||
expect(models.get('acme-think')?.compat).toEqual({
|
||||
supportsDeveloperRole: false,
|
||||
maxTokensField: 'max_tokens',
|
||||
})
|
||||
})
|
||||
|
||||
it('carries a switch both OpenAI protocols declare onto an openai-responses route', () => {
|
||||
const models = modelsOf({
|
||||
'acme-responses': {
|
||||
api: 'openai-responses',
|
||||
baseURL: 'https://acme.test',
|
||||
compat: { supportsDeveloperRole: false },
|
||||
models: [{ id: 'acme-r', reasoningEfforts: { off: null, high: 'high' } }],
|
||||
},
|
||||
}, 'acme-responses')
|
||||
|
||||
expect(models.get('acme-r')?.compat).toEqual({ supportsDeveloperRole: false })
|
||||
})
|
||||
|
||||
it('carries an anthropic-only switch onto an anthropic-messages route', () => {
|
||||
const models = modelsOf({
|
||||
'acme-claude': {
|
||||
api: 'anthropic-messages',
|
||||
baseURL: 'https://acme.test',
|
||||
compat: { supportsTemperature: false, supportsCacheControlOnTools: false },
|
||||
models: [{ id: 'acme-opus' }],
|
||||
},
|
||||
}, 'acme-claude')
|
||||
|
||||
expect(models.get('acme-opus')?.compat).toEqual({
|
||||
supportsTemperature: false,
|
||||
supportsCacheControlOnTools: false,
|
||||
})
|
||||
})
|
||||
|
||||
it('lands each route switch only on the models whose protocol declares it', () => {
|
||||
const catalog = getBuiltinModels('xai') as readonly Model<Api>[]
|
||||
const completions = catalog.find(model => model.api === 'openai-completions')
|
||||
const responses = catalog.find(model => model.api === 'openai-responses')
|
||||
if (completions === undefined || responses === undefined) throw new Error('xai no longer ships a mixed catalog')
|
||||
|
||||
const models = modelsOf({
|
||||
xai: {
|
||||
// Both protocols take the first switch; only completions takes the second.
|
||||
compat: { supportsDeveloperRole: false, thinkingFormat: 'openai' },
|
||||
models: [{ id: completions.id }, { id: responses.id }],
|
||||
},
|
||||
}, 'xai')
|
||||
|
||||
const onCompletions = models.get(completions.id)?.compat as OpenAICompletionsCompat
|
||||
expect(onCompletions.supportsDeveloperRole).toBe(false)
|
||||
expect(onCompletions.thinkingFormat).toBe('openai')
|
||||
const onResponses = models.get(responses.id)?.compat as { supportsDeveloperRole?: boolean; thinkingFormat?: string }
|
||||
expect(onResponses.supportsDeveloperRole).toBe(false)
|
||||
expect(onResponses.thinkingFormat).toBeUndefined()
|
||||
})
|
||||
|
||||
it('carries chat-template kwargs beside the thinking format that dispatches through them', () => {
|
||||
const models = modelsOf({
|
||||
'acme-qwen': {
|
||||
api: 'openai-completions',
|
||||
baseURL: 'https://acme.test',
|
||||
models: [{
|
||||
id: 'qwen-local',
|
||||
reasoningEfforts: { off: null, medium: 'medium' },
|
||||
compat: {
|
||||
thinkingFormat: 'qwen-chat-template',
|
||||
chatTemplateKwargs: { enable_thinking: { $var: 'thinking.enabled' } },
|
||||
},
|
||||
}],
|
||||
},
|
||||
}, 'acme-qwen')
|
||||
|
||||
expect(models.get('qwen-local')?.compat).toEqual({
|
||||
thinkingFormat: 'qwen-chat-template',
|
||||
chatTemplateKwargs: { enable_thinking: { $var: 'thinking.enabled' } },
|
||||
})
|
||||
})
|
||||
|
||||
it('refuses a compat key no wire protocol declares instead of dropping it', () => {
|
||||
// The silent drop is what let an unreadable switch look applied: schemastery
|
||||
// passes unknown keys through, and resolution used to read only two fields.
|
||||
expect(() => resolveProfiles({
|
||||
'acme-gateway': {
|
||||
api: 'openai-completions',
|
||||
baseURL: 'https://acme.test',
|
||||
compat: { supportsDevelperRole: false } as never,
|
||||
models: [{ id: 'acme-a' }],
|
||||
},
|
||||
})).toThrow(/compat "supportsDevelperRole", which no wire protocol declares; the configurable switches are .*\bsupportsDeveloperRole\b/)
|
||||
})
|
||||
|
||||
it('refuses a compat key pi-ai’s catalog owns, pointing at the catalog route', () => {
|
||||
expect(() => resolveProfiles({
|
||||
'acme-gateway': {
|
||||
api: 'openai-completions',
|
||||
baseURL: 'https://acme.test',
|
||||
models: [{ id: 'acme-a', compat: { openRouterRouting: {} } as never }],
|
||||
},
|
||||
})).toThrow(/compat "openRouterRouting", which is not configurable here/)
|
||||
})
|
||||
})
|
||||
|
||||
|
||||
Reference in New Issue
Block a user