mirror of
https://github.com/deepseek-ai/deepseek-harness.git
synced 2026-08-29 04:26:38 +00:00
fix(tools): scope bash SDK example to its schema
This commit is contained in:
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/bug-fix/2026-08-07-code-mode-executor-collapse.md
|
||||
2026-08-07-code-mode-executor-collapse.md: 0644342de1583d6c2a6d036027195edb6e33de73
|
||||
2026-08-07-code-mode-executor-collapse.zh.md: 013b2c19399da8ceed33e3455e30862c8c7f0363
|
||||
2026-08-07-code-mode-executor-collapse.md: 76265d5dd5f37f03c8e56366bf2791ddd2f7cb18
|
||||
2026-08-07-code-mode-executor-collapse.zh.md: f2d33993eb815d3f6dcd952557527c9aab5faefb
|
||||
|
||||
@@ -42,5 +42,5 @@ No provider guarantees interception of unadvertised names; the reported session
|
||||
- `both` and `native` behavior is unchanged; SDK sub-dispatches are unchanged (the `parent` token is the discriminator).
|
||||
- A collapsed call is rejected at `prepare`, BEFORE the extensible policy pipeline: pre-execute listeners, approval `ask`, and guards never observe it. `executionMode` also fails closed (`exclusive`), so scheduling has no observable difference.
|
||||
- Native-tool guidance sections (`tool:read`, `tool:write`, `tool:bash`, etc.) remain in the system prompt because they describe capabilities available through the generated SDK as well as native function calls, and several carry cross-tool routing policy (`read` over `bash cat`, `read` before `write` for the default fs-observation-policy, `subagent` over `workflow`) that no single tool description can hold. The executor collapse, not prompt filtering, prevents model-direct native calls.
|
||||
- The prompt STATES the collapse, in the `tools:code-only` section ordered ahead of first-party per-tool guidance. Those sections name their tool without qualifying how it is reached, so a model that read only them emitted a native call, received `UNKNOWN_TOOL` for a tool the same prompt declared, and concluded the deployment was inconsistent rather than correcting itself. The denial carries the route for the same reason. The TypeScript SDK section repeats the distinction next to the generated declarations, labels them as program-only bindings, states that only separately supplied tool schemas grant direct-call availability, and shows a complete `run_code` call around `tools.bash(...)` because the declaration list can otherwise be read as native tool availability. `both` renders the rule empty: its native calls do execute, so stating it there would be false — which is why `both-mode-turn` no longer shares `code-mode-turn`'s expected prompt.
|
||||
- The prompt STATES the collapse, in the `tools:code-only` section ordered ahead of first-party per-tool guidance. Those sections name their tool without qualifying how it is reached, so a model that read only them emitted a native call, received `UNKNOWN_TOOL` for a tool the same prompt declared, and concluded the deployment was inconsistent rather than correcting itself. The denial carries the route for the same reason. The TypeScript SDK section repeats the distinction next to the generated declarations, labels them as program-only bindings, and states that only separately supplied tool schemas grant direct-call availability. Because the declaration list can otherwise be read as native tool availability, the section emits a complete `run_code` call around `tools.bash(...)` when the current `bash` parameter schema accepts the example arguments. `both` renders the rule empty: its native calls do execute, so stating it there would be false — which is why `both-mode-turn` no longer shares `code-mode-turn`'s expected prompt.
|
||||
- Any future composite transport that sets a `parent` token opts its sub-dispatches into the full table, matching the nested-call semantics the token already documents.
|
||||
|
||||
@@ -42,5 +42,5 @@ guard 是可选的插件扩展;安全不变量不能依赖部署恰好组装
|
||||
- `both` 与 `native` 行为不变;SDK 子调用不变(判别信号是 `parent` token)。
|
||||
- 被塌缩的调用在 `prepare` 阶段即被拒绝——在可扩展策略流水线之前:pre-execute 监听器、approval `ask` 与 guard 永远不会观察到它。`executionMode` 同样 fail-closed(`exclusive`),调度无可观察差异。
|
||||
- 原生工具指引段(`tool:read`、`tool:write`、`tool:bash` 等)保留在系统提示词中,因为它们同时描述了通过生成 SDK 及原生函数调用可用的能力,其中若干段还承载着任何单个工具描述都装不下的跨工具路由策略(`read` 优先于 `bash cat`、默认 fs-observation-policy 要求先 `read` 再 `write`、一两个委派用 `subagent` 而非 `workflow`)。防止模型直呼原生工具的是执行器塌缩,而非提示词过滤。
|
||||
- 提示词会**声明**这条塌缩,位于 first-party 逐工具指导之前的 `tools:code-only` 段。那些段只写出工具名而不限定其可达方式,因此只读到它们的模型会发出原生调用,为一个同一份提示词刚刚声明过的工具收到 `UNKNOWN_TOOL`,进而判定部署不一致,而不是自行纠正。拒绝信息给出正确路径也是同一原因。TypeScript SDK 段在生成声明旁再次区分两者,将其标为只能在程序内使用的绑定,说明只有单独提供的工具 schema 才赋予直呼权限,并给出以 `run_code` 包住 `tools.bash(...)` 的完整调用,因为声明列表可能被误读为原生工具可用性。`both` 下该规则渲染为空:它的原生调用确实会执行,在那里声明就是假话——这也是 `both-mode-turn` 不再与 `code-mode-turn` 共用期望提示词的原因。
|
||||
- 提示词会**声明**这条塌缩,位于 first-party 逐工具指导之前的 `tools:code-only` 段。那些段只写出工具名而不限定其可达方式,因此只读到它们的模型会发出原生调用,为一个同一份提示词刚刚声明过的工具收到 `UNKNOWN_TOOL`,进而判定部署不一致,而不是自行纠正。拒绝信息给出正确路径也是同一原因。TypeScript SDK 段在生成声明旁再次区分两者,将其标为只能在程序内使用的绑定,并说明只有单独提供的工具 schema 才赋予直呼权限。声明列表可能被误读为原生工具可用性,因此当当前 `bash` 参数 schema 接受示例参数时,该段会给出以 `run_code` 包住 `tools.bash(...)` 的完整调用。`both` 下该规则渲染为空:它的原生调用确实会执行,在那里声明就是假话——这也是 `both-mode-turn` 不再与 `code-mode-turn` 共用期望提示词的原因。
|
||||
- 未来任何设置 `parent` token 的组合传输,其子调用自动走全表,与该 token 已有的嵌套调用语义一致。
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/core/tools/README.md
|
||||
README.md: 86c2f3549248d7c93f4bd4bc9ca10c8ab64b2bb0
|
||||
README.zh.md: caffa098090b142e4fd584410ac12a4c2574a6b0
|
||||
README.md: ed62ac8314e3bbd3720921738b9cbe3b0ee7315a
|
||||
README.zh.md: 1c42cbffe27bf2affdf7818085896c6d70a5655c
|
||||
|
||||
@@ -168,9 +168,9 @@ Prefix-stable while visible definitions and their order are unchanged. Registrat
|
||||
|
||||
#### What the model sees
|
||||
|
||||
Code Mode exposes the generated [`run_code` schema](../../../docs/tool-catalog.md#deepseek-aidsh-tools), the SDK instructions below, and the generated exact SDK block for the loaded runtime's language. The instructions identify generated declarations as program-only bindings and show a complete `run_code` call around `tools.bash(...)`. The `tools:sdk` section uses first-party order 5000. `both` exposes normal schemas and this Code Mode API; under `code` the prompt also carries the `tools:code-only` rule earlier in the first-party order, so the model reads which tools it may call before it reads what each one is for.
|
||||
Code Mode exposes the generated [`run_code` schema](../../../docs/tool-catalog.md#deepseek-aidsh-tools), the SDK instructions below, and the generated exact SDK block for the loaded runtime's language. The TypeScript instructions identify generated declarations as program-only bindings. When the current `bash` parameter schema accepts the example arguments, they also show a complete `run_code` call around `tools.bash(...)`. The `tools:sdk` section uses first-party order 5000. `both` exposes normal schemas and this Code Mode API; under `code` the prompt also carries the `tools:code-only` rule earlier in the first-party order, so the model reads which tools it may call before it reads what each one is for.
|
||||
|
||||
##### Code Mode SDK instructions
|
||||
##### TypeScript Code Mode SDK instructions with bash
|
||||
|
||||
```markdown
|
||||
## Writing code for run_code
|
||||
|
||||
@@ -168,9 +168,9 @@ ctx.tools.register(defineTool({
|
||||
|
||||
#### 模型看到什么
|
||||
|
||||
Code Mode 会公开生成的 [`run_code` schema](../../../docs/tool-catalog.zh.md#deepseek-aidsh-tools)、下方 SDK 说明,以及按所加载运行时语言生成的精确 SDK 块。说明会把生成声明明确标为只能在程序内使用的绑定,并给出以 `run_code` 包住 `tools.bash(...)` 的完整调用。`tools:sdk` 段使用 first-party 顺序 5000。`both` 会同时公开普通 schema 与此 Code Mode API;在 `code` 下,提示词还会带上处于更早 first-party 顺序的 `tools:code-only` 规则,让模型先读到「可以调用哪些工具」再读「每个工具做什么」。
|
||||
Code Mode 会公开生成的 [`run_code` schema](../../../docs/tool-catalog.zh.md#deepseek-aidsh-tools)、下方 SDK 说明,以及按所加载运行时语言生成的精确 SDK 块。TypeScript 说明会把生成声明明确标为只能在程序内使用的绑定。当当前 `bash` 参数 schema 接受示例参数时,说明还会给出以 `run_code` 包住 `tools.bash(...)` 的完整调用。`tools:sdk` 段使用 first-party 顺序 5000。`both` 会同时公开普通 schema 与此 Code Mode API;在 `code` 下,提示词还会带上处于更早 first-party 顺序的 `tools:code-only` 规则,让模型先读到「可以调用哪些工具」再读「每个工具做什么」。
|
||||
|
||||
##### Code Mode SDK 说明
|
||||
##### 带 bash 的 TypeScript Code Mode SDK 说明
|
||||
|
||||
```markdown
|
||||
## Writing code for run_code
|
||||
|
||||
@@ -249,11 +249,9 @@ export function jsonSchemaToTs(schema: unknown, indent = 0): string {
|
||||
/** The fixed model-facing usage contract rendered above the declarations (see the Code Mode Agent Note's "What the model sees"). */
|
||||
const SDK_INSTRUCTIONS = `## Writing code for run_code
|
||||
|
||||
\`run_code\` takes two required arguments: \`code\` — the body of an async TypeScript function (erasable syntax only — no \`enum\` or namespaces; type annotations are advisory, the code runs type-stripped) — and \`description\`, a short summary of what the program does. The declarations below are SDK bindings for this program. A declaration does not make its name a directly callable tool; only names supplied as separate tool schemas may be called directly. When no separate \`bash\` schema is supplied, invoke a declared \`bash\` binding inside \`run_code\`:
|
||||
\`run_code\` takes two required arguments: \`code\` — the body of an async TypeScript function (erasable syntax only — no \`enum\` or namespaces; type annotations are advisory, the code runs type-stripped) — and \`description\`, a short summary of what the program does. The declarations below are SDK bindings for this program. A declaration does not make its name a directly callable tool; only names supplied as separate tool schemas may be called directly.`
|
||||
|
||||
\`run_code({ code: "return await tools.bash({ command: 'pwd', description: 'Show current directory' })", description: "Show current directory" })\`
|
||||
|
||||
Inside the program:
|
||||
const SDK_PROGRAM_INSTRUCTIONS = `Inside the program:
|
||||
|
||||
- Call tools as \`await tools.name(args)\` — quoted access for exotic names: \`tools["my-tool"](args)\`. Every call resolves to the tool's typed canonical JSON value. Tool arguments must be lossless JSON.
|
||||
- A FAILED tool call rejects with \`ToolCallError\`, whose \`toolName\` identifies the failed tool and whose \`message\` is human-readable — \`try/catch\` it to handle and continue.
|
||||
@@ -262,6 +260,28 @@ Inside the program:
|
||||
|
||||
Program-only SDK bindings:`
|
||||
|
||||
/** Whether one string schema accepts the literal used by the bash example. */
|
||||
function acceptsExampleString(schema: JsonSchemaNode | undefined, value: string): boolean {
|
||||
return schema?.type === 'string'
|
||||
&& (schema.const === undefined || schema.const === value)
|
||||
&& (schema.enum === undefined || schema.enum.includes(value))
|
||||
}
|
||||
|
||||
/** Render the bash example only when its literal arguments satisfy the current parameter schema. */
|
||||
function renderBashExample(schemas: ToolSdkSchema[]): string {
|
||||
const bash = schemas.find(schema => schema.name === 'bash')
|
||||
if (bash === undefined) return ''
|
||||
const parameters = bash.parameters as JsonSchemaNode
|
||||
if (parameters.type !== 'object') return ''
|
||||
const required = parameters.required ?? []
|
||||
if (required.some(name => name !== 'command' && name !== 'description')) return ''
|
||||
if (!acceptsExampleString(parameters.properties?.command, 'pwd')) return ''
|
||||
const needsDescription = required.includes('description')
|
||||
if (needsDescription && !acceptsExampleString(parameters.properties?.description, 'Show current directory')) return ''
|
||||
const description = needsDescription ? ", description: 'Show current directory'" : ''
|
||||
return ` When no separate \`bash\` schema is supplied, invoke a declared \`bash\` binding inside \`run_code\`:\n\n\`run_code({ code: "return await tools.bash({ command: 'pwd'${description} })", description: "Show current directory" })\``
|
||||
}
|
||||
|
||||
/**
|
||||
* Render the full `tools:sdk` prompt section: the fixed usage instructions
|
||||
* plus one `declare const tools` interface covering every given tool.
|
||||
@@ -293,5 +313,5 @@ export function renderToolsSdk(schemas: ToolSdkSchema[]): string {
|
||||
['declare const tools: {', ' [K in ToolName]: (args: ToolArgsMap[K]) => Promise<ToolOutputMap[K]>;', '}'].join('\n'),
|
||||
].join('\n\n')
|
||||
const jsonValue = 'type JsonValue = null | boolean | number | string | JsonValue[] | { [key: string]: JsonValue }'
|
||||
return `${SDK_INSTRUCTIONS}\n\n\`\`\`ts\n${jsonValue}\n\n${declaration}\n\`\`\``
|
||||
return `${SDK_INSTRUCTIONS}${renderBashExample(sorted)}\n\n${SDK_PROGRAM_INSTRUCTIONS}\n\n\`\`\`ts\n${jsonValue}\n\n${declaration}\n\`\`\``
|
||||
}
|
||||
|
||||
@@ -133,6 +133,7 @@ describe('mode-aware wire contribution', () => {
|
||||
expect(sdk?.text).toContain('declare const tools: {')
|
||||
expect(sdk?.text).toContain('echo: {')
|
||||
expect(sdk?.text).not.toContain('run_code:')
|
||||
expect(sdk?.text).not.toContain('tools.bash(')
|
||||
})
|
||||
|
||||
it("mode 'code' states the run_code-only rule BEFORE the per-tool guidance that names each tool", async () => {
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { jsonSchemaToTs, renderToolsSdk } from '@deepseek-ai/dsh-tools/src/ts-types.ts'
|
||||
import type { ToolSdkSchema } from '@deepseek-ai/dsh-tools/src/ts-types.ts'
|
||||
import type { JsonSchemaNode } from '@deepseek-ai/dsh-tools/src/json-schema.ts'
|
||||
import { parameterSchemaSpecToJsonSchema } from '@deepseek-ai/dsh-tools'
|
||||
|
||||
describe('jsonSchemaToTs', () => {
|
||||
@@ -164,11 +165,57 @@ describe('renderToolsSdk', () => {
|
||||
const text = renderToolsSdk([bash])
|
||||
expect(text).toContain('A declaration does not make its name a directly callable tool')
|
||||
expect(text).toContain('only names supplied as separate tool schemas may be called directly')
|
||||
expect(text).toContain('`run_code({ code: "return await tools.bash(')
|
||||
expect(text).toContain('`run_code({ code: "return await tools.bash({ command: \'pwd\', description: \'Show current directory\' })"')
|
||||
expect(text).toContain('Program-only SDK bindings:')
|
||||
expect(text).not.toContain('The available tools:')
|
||||
})
|
||||
|
||||
it('only shows a bash example accepted by the declared binding', () => {
|
||||
expect(renderToolsSdk([exotic])).not.toContain('tools.bash(')
|
||||
|
||||
const commandOnly = {
|
||||
...bash,
|
||||
parameters: parameterSchemaSpecToJsonSchema({
|
||||
command: { type: 'string', required: true },
|
||||
}) as unknown as Record<string, unknown>,
|
||||
}
|
||||
expect(renderToolsSdk([commandOnly]))
|
||||
.toContain('tools.bash({ command: \'pwd\' })')
|
||||
|
||||
const incompatible = {
|
||||
...bash,
|
||||
parameters: parameterSchemaSpecToJsonSchema({
|
||||
command: { type: 'string', required: true },
|
||||
cwd: { type: 'string', required: true },
|
||||
}) as unknown as Record<string, unknown>,
|
||||
}
|
||||
expect(renderToolsSdk([incompatible])).not.toContain('tools.bash(')
|
||||
|
||||
const parameters = (value: JsonSchemaNode): ToolSdkSchema => ({
|
||||
...bash,
|
||||
parameters: value as Record<string, unknown>,
|
||||
})
|
||||
const rejected: JsonSchemaNode[] = [
|
||||
{ type: 'string' },
|
||||
{ type: 'object', properties: {} },
|
||||
{ type: 'object', properties: { command: { type: 'number' } } },
|
||||
{ type: 'object', properties: { command: { type: 'string', const: 'date' } } },
|
||||
{ type: 'object', properties: { command: { type: 'string', enum: ['date'] } } },
|
||||
{
|
||||
type: 'object',
|
||||
properties: { command: { type: 'string' }, description: { type: 'number' } },
|
||||
required: ['command', 'description'],
|
||||
},
|
||||
]
|
||||
for (const schema of rejected) expect(renderToolsSdk([parameters(schema)])).not.toContain('tools.bash(')
|
||||
|
||||
const constrained = parameters({
|
||||
type: 'object',
|
||||
properties: { command: { type: 'string', const: 'pwd', enum: ['pwd'] } },
|
||||
})
|
||||
expect(renderToolsSdk([constrained])).toContain('tools.bash({ command: \'pwd\' })')
|
||||
})
|
||||
|
||||
it('is deterministic: same tool set, byte-identical text regardless of input order', () => {
|
||||
expect(renderToolsSdk([bash, exotic])).toBe(renderToolsSdk([exotic, bash]))
|
||||
// Equal names sort stably (the comparator's equal arm).
|
||||
|
||||
Reference in New Issue
Block a user