mirror of
https://github.com/deepseek-ai/deepseek-harness.git
synced 2026-08-29 04:26:38 +00:00
docs(user): guide gateway request-compatibility switches
The Models page has no field for `compat`, and the symptom it addresses — a gateway holding a working key at a reachable address while refusing every request — reads as a credential or connectivity problem. Give it the same treatment `input` already has: name the symptom, show the two switches that account for most of it, and state the resolution order. Refs #2646
This commit is contained in:
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write docs/user/guide/providers.md
|
||||
providers.md: 099f434ec4602aa402239e83c708d81fcadd7732
|
||||
providers.zh.md: 367c90b525ad628b3cd86b2d22045c25064e88a1
|
||||
providers.md: 969ca13c0b288c8c0c313318a7a3162d7c4b361e
|
||||
providers.zh.md: 51079896ca6f38e30a65d7e73170a4a7e7b15569
|
||||
|
||||
@@ -79,6 +79,40 @@ Every list must name at least one modality except a model's own, where an empty
|
||||
|
||||
Both fields state a claim about your endpoint rather than checking it. A model that declares images its endpoint does not serve is not caught here; the provider rejects the request instead.
|
||||
|
||||
### Request compatibility
|
||||
|
||||
A gateway can hold a working key at a reachable address and still refuse every request. pi-ai decides the shape of a request — which role carries the system prompt, which field caps the output, how a thinking level travels — from the endpoint's URL, and an address it does not recognize is addressed as though it were OpenAI itself. Most OpenAI-compatible gateways refuse at least one thing OpenAI accepts.
|
||||
|
||||
Two account for most of it. A model that declares reasoning has its system prompt sent as `role: "developer"`, which many gateways reject outright, and the output cap is sent as `max_completion_tokens`, which a server that only knows `max_tokens` refuses. The form has no field for either; correct them on the route in `$DSH_HOME/settings.yaml`:
|
||||
|
||||
```yaml
|
||||
llm-pi-ai:
|
||||
providers:
|
||||
my-gateway:
|
||||
apiKeyEnv: GATEWAY_API_KEY
|
||||
api: openai-completions
|
||||
baseURL: https://gateway.example/v1
|
||||
compat:
|
||||
supportsDeveloperRole: false
|
||||
maxTokensField: max_tokens
|
||||
models:
|
||||
- id: my-model
|
||||
```
|
||||
|
||||
A route's `compat` is the default for its models, and a model's own wins field by field, so one model can be corrected without restating the route:
|
||||
|
||||
```yaml
|
||||
models:
|
||||
- id: my-model
|
||||
- id: my-reasoner
|
||||
compat:
|
||||
thinkingFormat: deepseek
|
||||
```
|
||||
|
||||
What neither sets keeps the installed catalog's value for that model, and what the catalog does not describe falls to pi-ai's detection. Give every switch you name a value: a key left empty (`supportsDeveloperRole:`) is refused rather than ignored, because an empty value would erase what the catalog knows while saying nothing in its place. A name no protocol accepts is refused too, and the message lists the ones that are available.
|
||||
|
||||
Each switch belongs to the protocols that declare it, so a switch valid on one `api` may be refused on another — the message names what that protocol does offer. Like `input` above, a switch states a claim about your endpoint rather than checking it: setting one your gateway does not actually need simply sends a different request.
|
||||
|
||||
## Select a model
|
||||
|
||||
Configured providers appear in the model picker. Selecting a model also makes it the default for new sessions. A session that has already sent a request retains the model recorded in its own log.
|
||||
@@ -90,6 +124,9 @@ If a saved default names a provider that was deleted, the composer displays **Se
|
||||
- **`MISSING_CREDENTIAL`** — Store the provider key through the Models page or supply the referenced environment variable.
|
||||
- **`UNKNOWN_MODEL`** — Select a configured model or add the missing model to the custom provider.
|
||||
- **Fetching available models returns 401** — Check the key. Model discovery calls the OpenAI-compatible `GET /models` endpoint; enter models manually for endpoints that do not provide it.
|
||||
- **The gateway refuses every request although the key and URL are right** — Its request shape differs from OpenAI's. Start with `compat.supportsDeveloperRole: false` and `compat.maxTokensField: max_tokens` on the route.
|
||||
- **Only reasoning models fail** — pi-ai sends their system prompt as the `developer` role, which the gateway rejects. Set `compat.supportsDeveloperRole: false`.
|
||||
- **A compat switch is refused as having no value** — A key written with nothing after the colon. Give it a value, or remove the key to keep the installed catalog's.
|
||||
- **An image is refused before sending** — The model declares no image modality. Give a custom provider's model `input: [text, image]`; DeepSeek's own chat-completions route is text-only and cannot be configured otherwise.
|
||||
- **The provider rejects a request carrying an image** — The model declares images its endpoint does not actually serve. Remove `image` from whichever list granted it — the model's `input`, or the route's `defaultInput` — then start a new session: the attached image stays in the session log, so the same request repeats until the session moves off it.
|
||||
|
||||
|
||||
@@ -79,6 +79,40 @@ llm-pi-ai:
|
||||
|
||||
这两个字段都是对你端点的断言,而不是对它的检查。声明了端点并不提供的图片能力的模型不会在这里被拦下,改由提供方拒绝该请求。
|
||||
|
||||
### 请求兼容性
|
||||
|
||||
网关可能持有可用的密钥、地址也通得到,却仍然拒绝每一个请求。pi-ai 依据端点的 URL 决定请求的形状——系统提示词由哪个角色承载、输出上限写在哪个字段、思考级别如何传输——而对于它无法识别的地址,会当作 OpenAI 本身来对待。多数 OpenAI 兼容网关至少会拒绝 OpenAI 所接受的某一样东西。
|
||||
|
||||
其中两样占了绝大多数。声明了推理能力的模型,其系统提示词会以 `role: "developer"` 发出,很多网关直接拒绝;输出上限则写作 `max_completion_tokens`,只认 `max_tokens` 的服务端会拒绝。表单里没有这两个字段;请在 `$DSH_HOME/settings.yaml` 的路由上更正:
|
||||
|
||||
```yaml
|
||||
llm-pi-ai:
|
||||
providers:
|
||||
my-gateway:
|
||||
apiKeyEnv: GATEWAY_API_KEY
|
||||
api: openai-completions
|
||||
baseURL: https://gateway.example/v1
|
||||
compat:
|
||||
supportsDeveloperRole: false
|
||||
maxTokensField: max_tokens
|
||||
models:
|
||||
- id: my-model
|
||||
```
|
||||
|
||||
路由的 `compat` 是其模型的默认值,模型自身的则逐字段胜出,因此更正某一个模型无需重述整条路由:
|
||||
|
||||
```yaml
|
||||
models:
|
||||
- id: my-model
|
||||
- id: my-reasoner
|
||||
compat:
|
||||
thinkingFormat: deepseek
|
||||
```
|
||||
|
||||
两者都未设置的字段,沿用已安装 catalog 为该模型记录的值;catalog 也未描述的,落到 pi-ai 的检测。凡是写下的开关都要给值:冒号后留空的键(`supportsDeveloperRole:`)会被拒绝而不是被忽略,因为空值会抹掉 catalog 已知的信息,却又没有给出任何替代。任何协议都不接受的名字同样会被拒绝,报错会列出可用的那些。
|
||||
|
||||
每个开关归属于声明了它的那些协议,因此在某个 `api` 上合法的开关,在另一个上可能被拒绝——报错会点名该协议实际提供哪些。与上面的 `input` 一样,开关陈述的是关于你的端点的一个断言,而不是对它的检查:设置一个网关其实并不需要的开关,只是发出一个不同的请求而已。
|
||||
|
||||
## 选择模型
|
||||
|
||||
已配置的提供方会出现在模型选择器中。选择模型也会将其设为新会话的默认值。已发送过请求的会话会保留自身日志中记录的模型。
|
||||
@@ -90,6 +124,9 @@ llm-pi-ai:
|
||||
- **`MISSING_CREDENTIAL`**:通过模型页存储提供方密钥,或提供被引用的环境变量。
|
||||
- **`UNKNOWN_MODEL`**:选择已配置的模型,或向自定义提供方添加缺失的模型。
|
||||
- **获取可用模型返回 401**:检查密钥。模型发现会调用 OpenAI 兼容的 `GET /models` 端点;对于不提供该端点的服务,请手动输入模型。
|
||||
- **密钥与地址都正确,网关却拒绝每一个请求**:它的请求形状与 OpenAI 不同。先在路由上设 `compat.supportsDeveloperRole: false` 与 `compat.maxTokensField: max_tokens`。
|
||||
- **只有推理模型失败**:pi-ai 把它们的系统提示词以 `developer` 角色发出,而网关拒绝该角色。设 `compat.supportsDeveloperRole: false`。
|
||||
- **某个 compat 开关因没有值而被拒绝**:冒号后什么都没写。给它一个值,或删掉该键以沿用已安装 catalog 的值。
|
||||
- **图片在发送前被拒绝**:该模型未声明图片模态。请给自定义提供方的模型加上 `input: [text, image]`;DeepSeek 自身的 chat-completions 路由是纯文本的,且无法通过配置改变。
|
||||
- **提供方拒绝了带图片的请求**:该模型声明了其端点实际并不提供的图片能力。请从授予它图片能力的那个列表中移除 `image`——可能是模型的 `input`,也可能是路由的 `defaultInput`——然后开启新会话:附加的图片会留在会话日志里,因此在会话离开它之前,同一个请求会不断重复。
|
||||
|
||||
|
||||
Reference in New Issue
Block a user