deepseek-harness/packages/llm/llm-pi-ai

description, kind

description	kind
The pi-ai-backed multi-provider adapter for users and maintainers routing the harness LLM service through pi-ai catalogs and hand-declared gateways.	package-reference

@deepseek-ai/dsh-llm-pi-ai

English | 中文

Summary

@deepseek-ai/dsh-llm-pi-ai is the pi-ai-backed multi-provider adapter for the harness LLM service: one plugin instance owns a dictionary of provider routes, each served through @earendil-works/pi-ai. A route naming an installed pi-ai provider inherits its endpoint, wire protocol, and model catalog as defaults; a route pi-ai does not ship is declared outright, so an OpenAI-compatible gateway or self-hosted server is configuration, not a code change. Profiles and credentials resolve per request over the optional settings and credential seams, so editing the user settings document changes the next request without a restart. A provider that ships a login can be signed into through the harness authorization seam, and the stored sign-in — an OAuth grant, or a key typed into pi-ai's own login prompt — authenticates its route and refreshes itself under the store's cross-process lock. The plugin can mount dormant with zero routes and activate them the moment a settings section supplies profiles.

Table of Contents

• Use this package
• Understand the implementation
• Further Exploration
• Model Experience
• Known Limitations and Deferred Work
• Dev Note

---

Use this package

Mount this plugin when a composition routes model requests through pi-ai's provider catalogs or through gateways that pi-ai's installed catalog does not describe. The providers dictionary is the whole configuration surface: each key is the provider route name a request selects with GenerateOptions.provider.

When to choose it

Choose this adapter when the same composition serves several providers, when a route needs pi-ai's catalog defaults with a few fields corrected, or when a hand-declared gateway must be reached through its own endpoint and protocol. Choose dsh-llm-deepseek for the direct DeepSeek route when the deployment needs no other provider. Both adapters can be mounted together because their route names do not collide; registering a route another adapter already owns fails plugin loading.

Configure provider routes

Each profile may set a retryPolicy; omission uses normal mode with five retries. apiKeyEnv is a credential reference resolved per request through the harness credential seam, so no secret enters the configuration file; a reference that resolves to nothing fails the request with MISSING_CREDENTIAL. Omitting it leaves the route configured-but-keyless, which for an installed catalog route defers to pi-ai's provider-native ambient discovery.

- name: '@deepseek-ai/dsh-llm-pi-ai'
  config:
    providers:
      openai:
        apiKeyEnv: OPENAI_API_KEY
        baseURL: https://proxy.example.com:8443
        reasoning: high
        requestImagePixelBudget: 4194304 # total pixels; 2048 by 2048 default
        requestImageMaxBytes: 1048576    # raw bytes before base64 expansion
        maxRequestImageBytes: 20971520   # accumulated base64 payload
        retryPolicy:
          mode: normal
          maxRetries: 3
      anthropic:
        apiKeyEnv: ANTHROPIC_API_KEY
        models:
          - id: claude-sonnet-4-5
            contextWindow: 200000
      acme-gateway:
        displayName: Acme Gateway
        apiKeyEnv: ACME_GATEWAY_API_KEY
        api: openai-completions
        baseURL: https://gateway.acme.example/v1
        compat:
          thinkingFormat: deepseek
        models:
          - id: acme-think
            name: Acme Think
            contextWindow: 262144
            reasoningEfforts:
              off:
              high: high

Field	Default	Meaning
apiKeyEnv	absent	Credential reference resolved per request; omission defers to pi-ai ambient discovery
displayName	provider name	Label shown by selector surfaces
api	catalog protocol	Wire protocol; only needed for routes the catalog does not supply
baseURL	catalog endpoint	Endpoint of every model on the route
models	installed catalog	Replaces the route's catalog wholesale; each entry defaults from the installed model
modelOverrides	none	Reshapes individual installed-catalog models without replacing the rest
compat	catalog detection	Wire-compatibility switches for unrecognized endpoints
defaultContextWindow	262,144	Capacity fallback for undescribed models
defaultMaxTokens	32,768	Output-cap fallback for undescribed models
requestImagePixelBudget	4,194,304	Total-pixel budget for each deterministic request image
requestImageMaxBytes	1 MiB	Encoded-byte target for each request image before base64 expansion
maxRequestImageBytes	20 MiB	Aggregate base64 image-payload bound with oldest-first offload
retryPolicy	normal, 5 retries	Provider-owned retry policy executed by dsh-llm-retry

The generated configuration catalog is the exhaustive source for every accepted field and its JSDoc.

Sign in to a provider

A provider pi-ai ships a login for can be signed into through the harness authorization seam: the flow offers OAuth or an interactive key prompt (a key is typed into pi-ai's own login prompt, not into the settings form), and the resulting credential is stored in the harness credential store at llm-pi-ai/<provider id>. The stored sign-in authenticates its route beneath any apiKeyEnv override and refreshes itself under the store's cross-process lock; signing out deletes the stored record. A hand-declared route key outside the record grammar — a lowercase hyphenated identifier — cannot be signed into, because a record write for it refuses with LlmError('UNSTORABLE_PROVIDER_ID'); such a route authenticates through apiKeyEnv or ambient provider settings instead.

Resolve the model catalog

A profile's models list replaces the route's installed catalog rather than extending it; each entry defaults its unset fields from the installed model of the same id, so narrowing a route to two models, correcting one capacity, or adding a model newer than the installed catalog are one-line edits. modelOverrides reshapes individual installed-catalog models without that cost — correct one model, keep the other thirty-seven — and is refused when set beside a models list, on a hand-declared route, or naming a model the catalog does not describe, because a silently unchanged model would be a typo someone hunts for later.

Run with reasoning and wire compatibility

reasoningEfforts declares a model's selectable thinking levels: each key is a level selectors offer, its value the spelling dispatch sends on the wire, so max: ultra renames a level for a gateway with its own vocabulary. Omitting the field keeps the installed catalog entry's capability; false declares a non-reasoning model. compat switches reshape the request for endpoints pi-ai cannot recognize — which role carries the system prompt, which field caps output, how a thinking level travels — configurable per route and per model. A model neither the entry nor the installed catalog sizes takes the route's defaultContextWindow and defaultMaxTokens fallbacks.

Change configuration at runtime

Profiles are re-read once per operation through the optional settings seam: the base and the user's llm-pi-ai: settings section merge per provider, so a user can add a route, override one field of a composition route, or point a route at another proxy, all effective on the next request with no restart. A section the adapter could not serve is refused where it is written — settings.mutate answers settings-rejected — and a stored section that later fails keeps the namespace's last good value. When the route set or a route's retry policy changes, the plugin re-registers atomically: a conflicting route leaves the previous routes serving.

Discover models from endpoints

The plugin answers "which models can this provider serve?" for a route a configuration surface is editing or drafting. A route the installed catalog ships is answered from that catalog with no network call; only a route the catalog does not describe is interrogated over the wire (openai-completions and openai-responses shapes). The reply is candidate metadata a surface may offer for adoption — nothing is stored, and settings.yaml remains the only thing that decides what a route serves.

Failures and recovery

A route pi-ai does not ship needs api, baseURL, and a non-empty models list; an unserviceable profile is refused where it is written, naming the route and model. Failures carry stable codes: a credential that cannot be used fails with INVALID_CREDENTIAL naming the route and reference, a route whose apiKeyEnv reference resolves to nothing fails with MISSING_CREDENTIAL, an unconfigured model fails with UNKNOWN_MODEL, and terminal provider failures distinguish QUOTA from transient RATE_LIMIT. GenerateOptions.stop is rejected with UNSUPPORTED_OPTION because pi-ai's common streaming UI cannot guarantee it across providers.

---

Understand the implementation

Implementation internals — click to expand

This section explains the design behind the adapter; the observable behavior is fully covered in Use this package.

Design philosophy

The adapter is built on immutable snapshots and per-operation resolution. Each operation captures a whole snapshot — the profiles plus a createModels() collection holding the Provider each route built — before its first await, and a configuration change builds a new collection rather than mutating the one in use, so a request that started under one configuration never finishes under another. A route's own credential reference resolves through the harness seam and rides as the request's apiKey option, which pi-ai treats as the highest-priority auth override — that is what keeps the fail-loud reference semantics. Everything that override does not cover reaches pi-ai through the collection's own auth: the credential store holds the records a login wrote and a refresh rotates (addressed as llm-pi-ai/<provider id>), and the auth context answers the ambient questions a provider asks while resolving. Both are stable across snapshots, so a configuration change rebuilds the collection without forgetting who is signed in.

Source map

File	Role
src/index.ts	Plugin entry: profile resolution, settings wiring, directory and route registration
src/auth.ts	The credential store and ambient auth context over the harness credential plane
src/login.ts	Authorization flows for the installed providers that ship a login
src/config.ts	Profile schema, resolution, and serviceability checks
src/catalog.ts	Installed-catalog integration and drift gates
src/provider.ts	The supported-protocol table and provider construction
src/context.ts	Harness-to-pi-ai context conversion, image handling, replay restore
src/stream.ts	pi-ai event conversion into harness StreamChunk values
src/replay.ts	Versioned ReplayEnvelope storage and validation
src/discovery.ts	Endpoint interrogation for configuration surfaces

Registration and directory

The plugin declares every installed catalog provider it can authenticate in the configurable-provider directory, joined with every route the current profiles declare, so configuration surfaces can offer the full catalog before any route exists. Each entry carries declared — whether pi-ai ships nothing under that key — because only the adapter can distinguish a hand-declared route from a narrowed catalog route. Route registration is atomic: a candidate set that collides with another adapter leaves the previous routes serving. A bare mount with zero routes is the dormant posture: nothing registers until a settings section supplies profiles, and routes drop when it empties.

Replay and vocabulary

Successful assistant responses store a versioned, lossless-JSON replay state beside the provider and model that produced them — response-level facts plus one per-block entry per streamed block. At request time, LlmRuntime passes replay state only when the same adapter instance owns both routes; the adapter validates it and restores native response ids and provider signatures, degrading an unusable state to provider-neutral content instead of failing the request. pi-ai tool-call arguments are parsed objects, so the adapter parses input and re-stringifies output to the harness raw-JSON convention; pi-ai in-stream error events map to terminal finish chunks.

---

Further Exploration

Read these pages when the package-level contract is not enough. They move from the service contract to the twin adapter and the shared types.

• dsh-llm service — the provider-neutral service this adapter registers on.
• llm-deepseek adapter — the direct DeepSeek twin for the deepseek-official route.
• LLM streaming subsystem — the StreamChunk protocol and adapter contract.
• llm-retry — the retry executor that applies each profile's retryPolicy.
• Twin LLM adapters — why the DeepSeek route ships two structurally different adapters.
• Generated configuration catalog — every accepted config field and its source declaration.

---

Model Experience

Provider request through pi-ai

What the model sees

The selected catalog model receives GenerateOptions.system, history, tools, and sampling fields supported by pi-ai's common streaming API. Each retained image is preceded by text naming its complete attachment id and actual request dimensions. When the current execution filesystem maps the attachment provider's host object, the text also carries a read-only normalized-object path and warns that normalization or request projection may have resized or re-encoded the upload. When accumulated base64 image payload exceeds the route's maxRequestImageBytes, each offloaded image keeps its own identity and currently resolved access in replacement text. Offloaded normalized attachments are not read or transformed. Provider-native replay metadata is restored only when the adapter validates it for the historical content.

Token effect

Provider tokenization governs exact input. Retained images add the stable attachment and coordinate descriptor; the offload placeholder replaces an omitted image's visual tokens. Replay metadata may let a native API reuse provider-side state.

KV Cache effect

Conversion preserves logical request order, while image handles and offload placeholders add model-visible text. A changed execution-world path rewrites a historical handle and can prevent reuse from that image even when attachment identity and request bytes stay stable. Changing adapter instance, provider, model, or another upstream token has the same suffix effect. Crossing the image bound replaces an earlier image with placeholder text, so reuse ends at that message until the offloaded prefix stabilizes.

Provider response

What the model sees

pi-ai events become harness reasoning, text, tool-call, usage, and finish chunks. The adapter passes parsed tool arguments to the harness as raw JSON strings.

Token effect

Generated content affects later inputs only after the loop records it. pi-ai folds reasoning tokens into output usage when the provider does not report them separately, and preserves its exact totalTokens value unchanged.

KV Cache effect

Recorded response content appends to the next request and does not invalidate its earlier reusable prefix. Unrecorded transport metadata and usage accounting do not affect cache identity.

Known Limitations and Deferred Work

These limits define where the adapter stops and future work begins. They are current package constraints, not a general pi-ai comparison or a task backlog.

• maxRequestImageBytes counts base64 image payload only — text, tools, descriptors, and JSON structure ride outside the bound, so it must sit below the gateway's request-body cap with headroom. Offload is a deterministic request projection and is not recorded as a session event.
• A sign-in lives only in the process that started it — an authorization attempt is not durable, so reloading the page mid-login abandons it and the human starts over. Signing out is deleteRecord on the stored record, which forgets it locally without telling the issuer.
• Provider-native discovery answers through this plugin's ambient context — a route naming no credential defers to the catalog provider's own resolution, which asks for environment values (AZURE_OPENAI_API_KEY, AWS_PROFILE, and each provider's own set) and for local credential files. Both questions are answered here: the credential seam is consulted before the process environment, and file existence is checked against the host process's filesystem with ~ expanded. What it cannot do is read a credential file's contents — a provider that parses ~/.aws/credentials itself does so directly, outside the seam.
• Settings can add or override routes, not remove composition routes — the user layer merges over the composition base, so deleting a cordis.yml-provided provider is a composition change.
• The layered merge has no delete for dict keys — a reasoningEfforts level, modelOverrides entry, or compat field the base declares can be overridden but not removed by the user layer.
• headers can carry a credential the redactor never sees — the profile's headers dict is plain strings; store credentials as apiKeyEnv references.
• A route's catalog never refreshes itself — the catalog is whatever settings.yaml says; nothing here queries a provider for the models it serves.
• One wire protocol per route — a mixed-protocol catalog route cannot host a model of the other protocol; splitting the provider across two route keys is the workaround.
• A modality declaration is not verified — a model declaring image its gateway does not serve is refused by the provider after prompt admission. The durable image remains in history and the same misdeclared model can fail again; switching to a text-only model remains possible because the shared LLM runtime projects image references into stable text for that request.
• An unauthenticated route depends on its protocol — a route naming no credential resolves as configured-but-keyless, but pi-ai's OpenAI-compatible implementation still requires an API key or an Authorization header, so a keyless local server needs a placeholder credential referenced by apiKeyEnv or an Authorization entry in headers.
• GenerateOptions.stop is unsupported — pi-ai's common stream options cannot guarantee stop-sequence behavior across providers.
• In-history system messages use pi-ai's common context conversion — provider-specific placement follows pi-ai rather than a harness-owned wire override.
• Provider HTTP status is unavailable — pi-ai error events do not expose a stable HTTP status across providers.
• Retry policy is provider-owned, not an SDK retry — pi-ai SDK retries stay disabled so durable agent steps and llm/retry events own every visible attempt, and direct ctx.llm.stream() calls remain single-attempt.

Dev Note

Working context for maintainers — click to expand

This Dev Note is non-authoritative working context: undecided directions and notes for maintainers. Shipped behavior and accepted rationale live in the sections above, the package code, and the linked Agent Notes.

• The offered protocol set is deliberately narrower than pi-ai's full API set: Bedrock, Vertex, Azure, and Codex authenticate through flows a profile cannot completely describe with a key, an endpoint, and headers; catalog routes still reach them through their own provider, and only an explicit override is refused. Codex is sign-in-able through the authorization flow's OAuth grant.
• The compat switch set is pinned to pi-ai's compat types by drift gates; an upstream upgrade that adds a field, gives a further protocol a compat type, or widens a value union fails the build until someone classifies it.