mirror of
https://github.com/deepseek-ai/deepseek-harness.git
synced 2026-08-29 04:26:38 +00:00
refactor(examples): fold the sandbox composition into acp-agent as its variant
sandbox-acp-agent's whole surface moves into examples/acp-agent following the example's existing multi-variant shape (the code-mode/both-mode overlays): sandbox.cordis.yml + its replay overlay, the four snapshot scenarios (config-options / mode-switching / escalation-approved / escalation-rejected) as a 'sandbox' header class over per-scenario configPath — goldens byte-identical, zero re-recording — and escalation.e2e.ts unchanged apart from the config path. One ACP example remains; demo:sandbox-acp keeps its name and boots the variant. References (both RFCs, group/package READMEs, the examples table and test map, the e2e workflow comment) now point at the variant.
This commit is contained in:
kingwl
@@ -82,7 +82,7 @@ jobs:
|
||||
- name: Install (immutable)
|
||||
run: pnpm install --frozen-lockfile
|
||||
|
||||
# The with-key escalation e2e (examples/sandbox-acp-agent/tests/
|
||||
# The with-key escalation e2e (examples/acp-agent/tests/
|
||||
# escalation.e2e.ts) self-skips without a usable runner — without this
|
||||
# step it would never actually execute anywhere (CI had no bwrap, dev
|
||||
# macs run Seatbelt instead), which is exactly how a broken harness
|
||||
|
||||
@@ -23,7 +23,7 @@ One `cordis.yml` entry mounts the seam. Not loading it is the fail-closed opt-ou
|
||||
# policy: never # deployment default for sessions without an override; 'ask' when omitted
|
||||
```
|
||||
|
||||
The entry alone provides mechanism, not a channel: with no answerer composed, every ask resolves `unavailable` and the asking tool call denies — fail-closed needs no configuration. Composing the ACP app (`@deepseek-ai/dsh-acp-agent`, as in [the sandbox example](../../../../examples/sandbox-acp-agent/README.md)) completes the loop: its bridge registers an answerer that prompts the owning editor session via `session/request_permission`, so a hook's `ask` or an escalation request surfaces as a one-shot Allow/Reject prompt attached to the already-streamed tool call. `policy: never` is the unattended stance — every ask auto-rejects deterministically, stated in the system prompt, no human in the loop. `policy` is validated against the closed list at plugin load; anything else throws.
|
||||
The entry alone provides mechanism, not a channel: with no answerer composed, every ask resolves `unavailable` and the asking tool call denies — fail-closed needs no configuration. Composing the ACP app (`@deepseek-ai/dsh-acp-agent`, as in [the acp-agent example's sandbox variant](../../../../examples/acp-agent/README.md)) completes the loop: its bridge registers an answerer that prompts the owning editor session via `session/request_permission`, so a hook's `ask` or an escalation request surfaces as a one-shot Allow/Reject prompt attached to the already-streamed tool call. `policy: never` is the unattended stance — every ask auto-rejects deterministically, stated in the system prompt, no human in the loop. `policy` is validated against the closed list at plugin load; anything else throws.
|
||||
|
||||
What a composed deployment observes: `allowed-once` lets exactly that call proceed; rejection, dismissal, and channel absence deny with three distinct reasons the model can tell apart; every ask lands a durable `approval/asked`/`approval/decided` pair on the asking agent's session log; nothing about a grant persists past the call that asked.
|
||||
|
||||
|
||||
@@ -16,7 +16,7 @@ One seam, one per-platform chain of local backends, one consumer, and two levers
|
||||
|
||||
### How a deployment uses it
|
||||
|
||||
Three `cordis.yml` entries turn an unconfined coding agent into the sandboxed product path; [`examples/sandbox-acp-agent`](../../../../examples/sandbox-acp-agent/README.md) is this composition, live:
|
||||
Three `cordis.yml` entries turn an unconfined coding agent into the sandboxed product path; [`examples/acp-agent`'s sandbox variant](../../../../examples/acp-agent/README.md) (`sandbox.cordis.yml`) is this composition, live:
|
||||
|
||||
```yaml
|
||||
- id: sandbox
|
||||
@@ -133,9 +133,9 @@ FIXME: Revisit this tool-local boundary. The follow-up design needs to determine
|
||||
### Testing
|
||||
|
||||
- Unit tier (no real runner anywhere): profile dialects, per-platform chain selection (sole candidate unprobed, no chain fails closed, multi-candidate probe order), verdict caching, the fail-closed end, probe-report parsing, and the launcher/`sandbox-exec` CLI contracts via fake runner scripts in `dsh-sandbox-local`; wrapping, policy hand-off, fact stamping, and runner-failure-outranks-denial classification (foreground throw, background `runnerFailed` fact) against a fake provider in `dsh-bash-sandbox`; the error's structured identity in `dsh-sandbox`. The escalation matrix spans the three bash packages: verbatim carry-through in `dsh-bash-local`, stamp/branch/per-task-facts in `dsh-bash-sandbox`, and the capability gate, `justification` pairing, fail-closed texts (pinned verbatim), and grant stamping in `dsh-tool-bash`. The switching surface pins the folds, the stamping precedence, the `'never'` gate, per-session section rendering, the full narrator matrix (cold start, coalescing, net-zero, resume drift with operator wording, positional attribution, persona-shadow hardening), and the bridge's advertisement gating, validation rejections, idle-vs-mid-turn anchoring (dev invariants mounted), and `session/load` reporting over a real two-process JSONL round trip.
|
||||
- Keyless real-runner e2e, split along the seam and per rung: CI's `sandbox-e2e` matrix runs bwrap and Landlock on Linux (the Landlock leg once per architecture, each confining through the registry-installed launcher) and Seatbelt on macOS against real kernels, failing on a silent all-skip. World-proofs live in `dsh-sandbox-local` (denied writes absent on disk, workspace writes landing, temp-area grants pinned, kernel denial text matching the advertised dialect) and `dsh-bash-sandbox` (the through-`ctx.bash` consumer proofs, including denied-then-overridden-write-lands). This package's own publish path is rehearsed without publishing (`packed-install.e2e.ts`): `pnpm pack`, tarballs installed into a throwaway consumer with the launcher family resolving from the registry, plain `node` confining through the INSTALLED launcher — asserted executable apart, so a mode-stripped binary can never masquerade as a non-enforcing kernel. The switching surface has its own keyless e2e (`examples/sandbox-acp-agent`): the real `cordis.yml` tree advertises both options, honors switches end to end, and rejects out-of-vocabulary values.
|
||||
- With-key e2e (`examples/sandbox-acp-agent/tests/escalation.e2e.ts`): real model + real runner + the REAL bridge answerer, world-verified — denied under `read-only`, escalates with justification, the scripted editor grants and the retried write lands on disk, while a rejected escalation leaves no write. Self-skips without `DEEPSEEK_API_KEY` or a usable runner (e2e.yml installs bubblewrap so it actually executes in CI).
|
||||
- Snapshot tier (`examples/sandbox-acp-agent/tests/acp.snapshot.ts`): the keyless config-option wire; the recorded mode-switching arc as the suite's pinned header — necessarily, since mid-session switches emit the `request/header-delta`s the uniformity guard licenses only in the pin — committing both switches, the prompt-section delta and one "changed by the user" notice per knob, and a confined write landing under the switched mode; and both recorded escalation branches over scripted `permissionAnswers` (grant runs confined under `workspace-write`; rejection executes nothing and pins the fail-closed text). Replay re-executes every fixture's bash calls under the host's real runner (ci.yml's snapshot lane installs bubblewrap). Deliberately absent: a fixture carrying a real DENIAL — denial stderr is the backend's dialect and would pin a fixture to its recording platform; the escalation prompts assert the prior denial instead, and the denial→marker path stays on the tiers above.
|
||||
- Keyless real-runner e2e, split along the seam and per rung: CI's `sandbox-e2e` matrix runs bwrap and Landlock on Linux (the Landlock leg once per architecture, each confining through the registry-installed launcher) and Seatbelt on macOS against real kernels, failing on a silent all-skip. World-proofs live in `dsh-sandbox-local` (denied writes absent on disk, workspace writes landing, temp-area grants pinned, kernel denial text matching the advertised dialect) and `dsh-bash-sandbox` (the through-`ctx.bash` consumer proofs, including denied-then-overridden-write-lands). This package's own publish path is rehearsed without publishing (`packed-install.e2e.ts`): `pnpm pack`, tarballs installed into a throwaway consumer with the launcher family resolving from the registry, plain `node` confining through the INSTALLED launcher — asserted executable apart, so a mode-stripped binary can never masquerade as a non-enforcing kernel. The switching surface has its own keyless e2e (the acp-agent example's `escalation.e2e.ts`): the real `sandbox.cordis.yml` tree advertises both options, honors switches end to end, and rejects out-of-vocabulary values.
|
||||
- With-key e2e (`examples/acp-agent/tests/escalation.e2e.ts`): real model + real runner + the REAL bridge answerer, world-verified — denied under `read-only`, escalates with justification, the scripted editor grants and the retried write lands on disk, while a rejected escalation leaves no write. Self-skips without `DEEPSEEK_API_KEY` or a usable runner (e2e.yml installs bubblewrap so it actually executes in CI).
|
||||
- Snapshot tier (`examples/acp-agent/tests/acp.snapshot.ts`): the keyless config-option wire; the recorded mode-switching arc as the suite's pinned header — necessarily, since mid-session switches emit the `request/header-delta`s the uniformity guard licenses only in the pin — committing both switches, the prompt-section delta and one "changed by the user" notice per knob, and a confined write landing under the switched mode; and both recorded escalation branches over scripted `permissionAnswers` (grant runs confined under `workspace-write`; rejection executes nothing and pins the fail-closed text). Replay re-executes every fixture's bash calls under the host's real runner (ci.yml's snapshot lane installs bubblewrap). Deliberately absent: a fixture carrying a real DENIAL — denial stderr is the backend's dialect and would pin a fixture to its recording platform; the escalation prompts assert the prior denial instead, and the denial→marker path stays on the tiers above.
|
||||
|
||||
## Deferred phases
|
||||
|
||||
|
||||
+1
-2
@@ -22,7 +22,6 @@ A keyless smoke that spawns the example from a temp cwd must set `TSX_TSCONFIG_P
|
||||
| `echo-agent` | `tests/echo.e2e.ts` — boots the real `cordis.yml`, drives the echo tool round-trip and the direct canned reply | **N/A — keyless by nature** (the `mock-echo` model has no real provider) |
|
||||
| `coding-agent` | `tests/keyless-smoke.e2e.ts` — boots the full real tree (dummy key, no prompt → no model call), asserts banner + clean exit; `tests/code-mode-keyless-smoke.e2e.ts` — the same boot guard for the Code Mode overlay | `tests/{full-loop,coding-task,resume,compaction,todo-write}.e2e.ts` — real model + real bash + real todo_write, world-verified; `tests/code-mode.e2e.ts` — a real model composes two bash calls in one `run_code` program; collapsed header, dispatch events, written file all verified |
|
||||
| `cordis-agent` | `tests/keyless-smoke.e2e.ts` — boots the real tree incl. `@deepseek-ai/dsh-tool-cordis` by package name; the tool logic is unit-tested in `packages/cordis/tool-cordis` | `tests/cordis-tools.e2e.ts` — real model mounts a listener (tagged line fires), builds+calls its own tool, composes two mounts via provide/inject |
|
||||
| `sandbox-acp-agent` | `escalation.e2e.ts` — boots the real tree (sandbox + approval + bridge) keyless: initialize + `session/new` | same file — denied → escalates → a scripted client grants (the write must land) or rejects (it must not); skips without key/runner |
|
||||
| `acp-agent` | `pnpm run test:snapshot` — boots the real ACP subprocess and replays a recorded session keyless (incl. the hook matrix: a scenario per hook point × outcome for BOTH the Claude and Codex bridges — block, deny, ask, context-fold, force-continue); `tests/acp.e2e.ts` also asserts stdout purity without a key | `tests/acp.e2e.ts` — real ACP prompt, verifies a file the agent wrote; `tests/hooks.e2e.ts` — a real `PreToolUse` hook blocks bash, verifies the file is NOT written |
|
||||
| `acp-agent` | `pnpm run test:snapshot` — boots the real ACP subprocess and replays a recorded session keyless (incl. the hook matrix: a scenario per hook point × outcome for BOTH the Claude and Codex bridges — block, deny, ask, context-fold, force-continue); `tests/acp.e2e.ts` also asserts stdout purity without a key; `tests/escalation.e2e.ts` boots the sandbox variant's real tree (sandbox + approval + bridge) keyless: initialize + `session/new` | `tests/acp.e2e.ts` — real ACP prompt, verifies a file the agent wrote; `tests/hooks.e2e.ts` — a real `PreToolUse` hook blocks bash, verifies the file is NOT written; `tests/escalation.e2e.ts` — denied → escalates → a scripted client grants (the write must land) or rejects (it must not); skips without key/runner |
|
||||
|
||||
See [the root AGENTS.md](../AGENTS.md) for repo-wide conventions and [docs/architecture.md](../docs/architecture.md) for the design.
|
||||
|
||||
+1
-5
@@ -33,8 +33,4 @@ An agent demo exposed as an **Agent Client Protocol (ACP)** server over JSON-RPC
|
||||
|
||||
Run with: `pnpm run demo:acp` (needs `DEEPSEEK_API_KEY`); `pnpm run demo:code-mode acp` boots the same server in Code Mode via the `code-mode.cordis.yml` overlay. See [acp-agent/README.md](acp-agent/README.md) for the Zed setup and the snapshot-test design.
|
||||
|
||||
## sandbox-acp-agent
|
||||
|
||||
The coding agent with its bash executor swapped for the sandbox stack ([`@deepseek-ai/dsh-sandbox-local`](../packages/sandbox/sandbox-local) + [`@deepseek-ai/dsh-bash-sandbox`](../packages/bash/bash-sandbox) — the one-entry executor swap the `ctx.bash` capability seam exists for), served over ACP with [`@deepseek-ai/dsh-user-approval`](../packages/ui/user-approval) mounted — the first composition where the approval loop is LIVE: a sandbox denial escalated by the model becomes a `session/request_permission` prompt in the editor, and "Allow once" runs exactly that command under the wider mode.
|
||||
|
||||
Run with: `pnpm run demo:sandbox-acp` (needs `DEEPSEEK_API_KEY`; bwrap, a Landlock-enforcing kernel, or macOS for confined runs). See [sandbox-acp-agent/README.md](sandbox-acp-agent/README.md).
|
||||
The **sandbox variant** (`sandbox.cordis.yml`) swaps the bash executor for the sandbox stack ([`@deepseek-ai/dsh-sandbox-local`](../packages/sandbox/sandbox-local) + [`@deepseek-ai/dsh-bash-sandbox`](../packages/bash/bash-sandbox) — the one-entry executor swap the `ctx.bash` capability seam exists for) and mounts [`@deepseek-ai/dsh-user-approval`](../packages/ui/user-approval) — the composition where the approval loop is LIVE: a sandbox denial escalated by the model becomes a `session/request_permission` prompt in the editor, and "Allow once" runs exactly that command under the wider mode. Run with: `pnpm run demo:sandbox-acp` (needs `DEEPSEEK_API_KEY`; bwrap, a Landlock-enforcing kernel, or macOS for confined runs).
|
||||
|
||||
@@ -35,6 +35,17 @@ The editor sets each session's `cwd` to the project it opens; both the agent's b
|
||||
|
||||
This example is the home of the harness's **snapshot tests** — they boot this server as a real subprocess, drive it with a deterministic input script, and diff its normalized output against committed golden files. The model is made deterministic by `@deepseek-ai/dsh-llm-replay`, a function/namespace plugin that installs an `llm/stream` waterfall listener and short-circuits it, serving model streams reconstructed from a recorded **session JSONL** fixture (`<scenario>/session.jsonl`) — so replay needs no API key. The fixture IS the persisted session log: its `assistant/chunk` events carry every `StreamChunk`, so grouping them by `(turn, step)` reconstructs each `stream()` call (one model call per loop step). Recording is therefore "run the real agent once and harvest the `.jsonl`"; use `pnpm run test:snapshot:record` when the model transcript itself should change, and `pnpm run test:snapshot:refresh` when the committed model transcript is still the right mock input and only the current replay output/goldens need to be rewritten. The two failure modes not expressible as logged chunks — a pure throw before any chunk, and cancel/hang — use an optional `<scenario>/replay.override.json` sidecar (a `ReplayEntry[]` that replaces the derived script). A scenario that needs the agent to operate on existing files ships an optional `<scenario>/workspace/` directory — the harness copies its contents into the temp cwd before the run (see `workspace-edit`). See [the ACP snapshot tests RFC](../../docs/rfc/implemented/testing/2026-06-19-acp-snapshot-tests.md) for the full design.
|
||||
|
||||
## The sandbox variant (`sandbox.cordis.yml`)
|
||||
|
||||
The same server with the bash executor swapped for the sandbox stack ([`@deepseek-ai/dsh-sandbox-local`](../../packages/sandbox/sandbox-local/) + [`@deepseek-ai/dsh-bash-sandbox`](../../packages/bash/bash-sandbox/) — the one-entry executor swap the `ctx.bash` capability seam exists for) plus [`@deepseek-ai/dsh-user-approval`](../../packages/ui/user-approval/) — the composition where the approval loop is LIVE end to end: bash runs under `read-only`, a denial comes back as the structured marker, the model retries once with `sandbox_permissions` + `justification`, the ACP bridge's answerer turns that ask into a `session/request_permission` prompt in your editor, and "Allow once" runs exactly that command under the wider mode ([sandbox RFC § Escalation](../../docs/rfc/implemented/feature/2026-07-06-sandbox.md)). Run it with `pnpm run demo:sandbox-acp`; Zed setup is the same as above with this command.
|
||||
|
||||
- **Every approval is one-shot** (`Allow once` / `Reject` — no `allow_always`: the harness has no grant storage yet), and a dismissed prompt or a rejected ask fails closed with its own error text; so does every ask when no editor is attached to answer.
|
||||
- **Two session config options are live** ([sandbox RFC § Per-session mode switching](../../docs/rfc/implemented/feature/2026-07-06-sandbox.md)): a capable client shows `Sandbox` (`read-only`/`workspace-write`/`danger-full-access`) and `Approvals` (`ask`/`never`) selectors per session — a switch is one log-only event on that session's log and execution follows it; the sandbox mode is deliberately NOT stated in the prompt or narrated (the model learns the boundary from the denial marker — behavior, not belief), while an approval switch to `never` is stated and narrated; a resumed session reports its overrides back on `session/load`.
|
||||
- **The write boundary is config-fixed**: an escalated `workspace-write` run may write under the launch directory (`workspaceRoot: process.cwd()`) plus the platform temp area — a per-session root is config-phase future work in the [sandbox RFC](../../docs/rfc/implemented/feature/2026-07-06-sandbox.md).
|
||||
- **No usable runner fails closed per command** (structured `SANDBOX_UNAVAILABLE`), and the variant loads no filesystem tools: they would bypass the bash sandbox.
|
||||
|
||||
Variant tests, in this example's suites: `tests/escalation.e2e.ts` — keyless, it boots the real `sandbox.cordis.yml` through the Loader as an ACP subprocess, proves the whole tree (sandbox executor + approval service + bridge) initializes and opens a session, and drives the config options end to end; with a key and a usable runner, a scripted ACP client plays the human — the real model gets denied, escalates, the client answers `allow-once`, and the retried write must land on disk. Four scenarios in `tests/acp.snapshot.ts` run against the variant's `sandbox.cordis.snapshot.yml` replay overlay under the `sandbox` header class: the keyless `config-options` exchange, the recorded `mode-switching` arc (that class's pinned header — the approval prompt-section delta, its "changed by the user" notice, and a confined write landing under the switched mode), and both recorded escalation branches (`session/request_permission` answered allow-once / reject-once). Replay re-executes every recorded bash call under the host's real runner — Seatbelt works out of the box on macOS; on Linux install bubblewrap first, exactly what ci.yml's snapshot lane does. No fixture carries a real denial: denial stderr is backend dialect and would pin a fixture to its recording platform (the rationale comment atop the suite file).
|
||||
|
||||
## MVP limitations
|
||||
|
||||
The bridge supports N concurrent sessions per connection, each in its own workspace `cwd` (RFC 011). Remaining limits: prompts support ACP's baseline `text` and `resource_link` blocks only, and `additionalDirectories` and `mcpServers` are rejected. Permission prompts (`session/request_permission`) are wired through the approval seam, but this example composes no ask-producing policy, so tools run with the executor's full authority. See `packages/ui/acp/README.md` for the full contract.
|
||||
The bridge supports N concurrent sessions per connection, each in its own workspace `cwd` (RFC 011). Remaining limits: prompts support ACP's baseline `text` and `resource_link` blocks only, and `additionalDirectories` and `mcpServers` are rejected. Permission prompts (`session/request_permission`) are wired through the approval seam; the MAIN tree composes no ask-producing policy, so its tools run with the executor's full authority — the sandbox variant above is the composition that exercises the live prompt. See `packages/ui/acp/README.md` for the full contract.
|
||||
|
||||
+1
-1
@@ -17,7 +17,7 @@
|
||||
- id: base
|
||||
name: '@cordisjs/plugin-include'
|
||||
config:
|
||||
path: ./cordis.yml
|
||||
path: ./sandbox.cordis.yml
|
||||
patches:
|
||||
# The name is an assertion, not an override: the include skips the patch
|
||||
# (warning) when the id points at a different plugin, so this can never
|
||||
@@ -1,13 +1,13 @@
|
||||
# The sandbox-acp-agent plugin tree: the sandboxed coding agent served over the
|
||||
# Agent Client Protocol, with the approval seam composed — the first LIVE
|
||||
# approval composition. A sandbox denial escalated by the model
|
||||
# The SANDBOX VARIANT of the acp-agent example: the sandboxed coding agent
|
||||
# served over the Agent Client Protocol, with the approval seam composed —
|
||||
# the live approval composition (`pnpm run demo:sandbox-acp`). A sandbox denial escalated by the model
|
||||
# (sandbox_permissions + justification) reaches the EDITOR as a
|
||||
# session/request_permission prompt through the ACP bridge's answerer, and an
|
||||
# "Allow once" runs exactly that command under the wider mode.
|
||||
#
|
||||
# CRITICAL: this tree loads NO stdout logger and NO hmr — stdout is reserved
|
||||
# for the ACP JSON-RPC protocol (a property of @deepseek-ai/dsh-acp-agent,
|
||||
# same as examples/acp-agent).
|
||||
# same as the sibling cordis.yml).
|
||||
#
|
||||
# Requires DEEPSEEK_API_KEY (and optionally DEEPSEEK_BASE_URL) — the
|
||||
# dsh-acp-agent bin loads the gitignored repo-root .env first (on STDERR only).
|
||||
@@ -27,6 +27,10 @@ const AGENT = {
|
||||
const CODE_MODE_CONFIG = fileURLToPath(new URL('../code-mode.cordis.yml', import.meta.url))
|
||||
const BOTH_MODE_CONFIG = fileURLToPath(new URL('../both-mode.cordis.yml', import.meta.url))
|
||||
|
||||
// The sandbox variant (its own composition, not an include patch: sandboxed
|
||||
// bash executor + the approval seam over the same app spine).
|
||||
const SANDBOX_CONFIG = fileURLToPath(new URL('../sandbox.cordis.yml', import.meta.url))
|
||||
|
||||
function snapshotModeFromEnv(value: string | undefined): SnapshotSuiteOptions['mode'] {
|
||||
switch (value) {
|
||||
case undefined:
|
||||
@@ -121,6 +125,37 @@ const SCENARIOS: Scenario[] = [
|
||||
// therefore pins its own class.
|
||||
{ name: 'code-mode-turn', hasModelTurn: true, recorded: true, pinsHeader: true, headerClass: 'code', configPath: CODE_MODE_CONFIG },
|
||||
{ name: 'both-mode-turn', hasModelTurn: true, recorded: true, pinsHeader: true, headerClass: 'both', configPath: BOTH_MODE_CONFIG },
|
||||
// The SANDBOX variant (sandbox.cordis.yml: sandboxed bash + approval seam).
|
||||
// Replay swaps only the MODEL for the recorded transcript — every bash call
|
||||
// re-executes for real under the host's actual runner (Seatbelt on macOS,
|
||||
// bwrap on Linux CI), so these recordings double as cross-backend
|
||||
// confinement regression; their commands are limited to `cat`/`printf`
|
||||
// shapes whose bytes are identical across those backends and across
|
||||
// GNU/BSD userlands. Deliberately ABSENT: a scenario whose transcript
|
||||
// carries a real sandbox DENIAL — the denied command's own stderr is the
|
||||
// backend's dialect (EROFS/EACCES/EPERM phrasing), so such a fixture
|
||||
// replays only on the platform that recorded it; the denial→marker path
|
||||
// stays on dsh-tool-bash's unit tests and the real-kernel sandbox e2e legs,
|
||||
// and the escalation scenarios sidestep it by having the USER assert the
|
||||
// prior denial. config-options: the session config-option surface this
|
||||
// composition adds (both advertised selects, the refreshed state every
|
||||
// set_config_option answers with, both rejection shapes) — protocol-only,
|
||||
// replays on runner-less hosts. mode-switching: the runtime switching arc
|
||||
// and NECESSARILY this class's pinned-header scenario (an approval-policy
|
||||
// switch rewrites its prompt section; the resulting request/header-delta is
|
||||
// legal only in the pinning scenario) — the pin commits the full sandbox
|
||||
// header (persona, tool schemas WITH the escalation fields) plus the
|
||||
// approval delta and its "changed by the user" notice; the SANDBOX switch
|
||||
// stays deliberately silent (the visibility asymmetry), proven by BEHAVIOR.
|
||||
// escalation-approved/rejected: the approval wire end-to-end under the
|
||||
// default read-only/ask — the escalating call streams,
|
||||
// session/request_permission attaches to it, and the scripted answer drives
|
||||
// each branch (approved runs CONFINED under the granted workspace-write;
|
||||
// rejected executes nothing, failing with the deterministic text).
|
||||
{ name: 'config-options', hasModelTurn: false, recorded: false, headerClass: 'sandbox', configPath: SANDBOX_CONFIG },
|
||||
{ name: 'mode-switching', hasModelTurn: true, recorded: true, pinsHeader: true, expectedHeaderDeltas: 1, headerClass: 'sandbox', configPath: SANDBOX_CONFIG },
|
||||
{ name: 'escalation-approved', hasModelTurn: true, recorded: true, headerClass: 'sandbox', configPath: SANDBOX_CONFIG },
|
||||
{ name: 'escalation-rejected', hasModelTurn: true, recorded: true, headerClass: 'sandbox', configPath: SANDBOX_CONFIG },
|
||||
]
|
||||
|
||||
defineAcpSnapshotSuite({
|
||||
|
||||
+5
-5
@@ -17,9 +17,9 @@ import {
|
||||
} from '@agentclientprotocol/sdk'
|
||||
|
||||
/**
|
||||
* examples/sandbox-acp-agent end to end.
|
||||
* The sandbox variant (`sandbox.cordis.yml`) end to end.
|
||||
*
|
||||
* Keyless smoke: boot the REAL `cordis.yml` through the `dsh-acp-agent` bin as
|
||||
* Keyless smoke: boot the REAL `sandbox.cordis.yml` through the `dsh-acp-agent` bin as
|
||||
* an ACP subprocess and drive initialize + session/new — the real-Loader-path
|
||||
* guard (postmortem 0001) for THIS tree's export shapes, which now include the
|
||||
* sandbox executor AND the approval service. No prompt is sent, so neither the
|
||||
@@ -36,7 +36,7 @@ import {
|
||||
*/
|
||||
|
||||
const binScript = fileURLToPath(new URL('../../../packages/ui/acp-agent/src/bin.ts', import.meta.url))
|
||||
const configPath = fileURLToPath(new URL('../cordis.yml', import.meta.url))
|
||||
const configPath = fileURLToPath(new URL('../sandbox.cordis.yml', import.meta.url))
|
||||
const tsxLoader = fileURLToPath(import.meta.resolve('tsx'))
|
||||
// The subprocess runs from a temp cwd OUTSIDE the repo; point tsx at the repo
|
||||
// tsconfig so the unbuilt `paths` map resolves (see examples/AGENTS.md).
|
||||
@@ -116,7 +116,7 @@ afterEach(async () => {
|
||||
workdir = undefined
|
||||
})
|
||||
|
||||
describe('sandbox-acp-agent keyless smoke (real cordis.yml via the Loader)', () => {
|
||||
describe('sandbox variant keyless smoke (real sandbox.cordis.yml via the Loader)', () => {
|
||||
it('boots the tree — sandbox executor + approval service + bridge — and opens a session', async () => {
|
||||
workdir = await mkdtemp(join(tmpdir(), 'sandbox-acp-smoke-'))
|
||||
spawned = spawnSandboxAcpAgent(workdir, 'reject-once')
|
||||
@@ -161,7 +161,7 @@ describe('sandbox-acp-agent keyless smoke (real cordis.yml via the Loader)', ()
|
||||
}, 30_000)
|
||||
})
|
||||
|
||||
describe.skipIf(!process.env.DEEPSEEK_API_KEY || !hasRunner)('sandbox-acp-agent e2e: the live approval loop', () => {
|
||||
describe.skipIf(!process.env.DEEPSEEK_API_KEY || !hasRunner)('sandbox variant e2e: the live approval loop', () => {
|
||||
it('denial → model escalation → editor prompt → allow-once → the retried write lands on disk', async () => {
|
||||
workdir = await mkdtemp(join(tmpdir(), 'sandbox-acp-e2e-'))
|
||||
spawned = spawnSandboxAcpAgent(workdir, 'allow-once')
|
||||
@@ -1,16 +0,0 @@
|
||||
# sandbox-acp-agent
|
||||
|
||||
The coding agent with its bash executor swapped for the sandbox stack ([`@deepseek-ai/dsh-sandbox-local`](../../packages/sandbox/sandbox-local/) + [`@deepseek-ai/dsh-bash-sandbox`](../../packages/bash/bash-sandbox/) — the one-entry executor swap the `ctx.bash` capability seam exists for), served over the **Agent Client Protocol**, plus [`@deepseek-ai/dsh-user-approval`](../../packages/ui/user-approval/) — which makes this the first composition where the approval loop is LIVE end to end: bash runs under `read-only`, a denial comes back as the structured marker, the model retries once with `sandbox_permissions` + `justification`, the ACP bridge's answerer turns that ask into a `session/request_permission` prompt in your editor, and "Allow once" runs exactly that command under the wider mode ([sandbox RFC § Escalation](../../docs/rfc/implemented/feature/2026-07-06-sandbox.md)).
|
||||
|
||||
```sh
|
||||
pnpm run demo:sandbox-acp # needs DEEPSEEK_API_KEY; drive it from Zed or any ACP client
|
||||
```
|
||||
|
||||
Zed setup is the same as [acp-agent](../acp-agent/README.md) with this example's command; only the leaf `cordis.yml` differs (the sandbox stack + the approval entry in place of the local bash executor and the extra tool stacks).
|
||||
|
||||
- **Every approval is one-shot** (`Allow once` / `Reject` — no `allow_always`: the harness has no grant storage yet), and a dismissed prompt or a rejected ask fails closed with its own error text; so does every ask when no editor is attached to answer.
|
||||
- **Two session config options are live** ([sandbox RFC § Per-session mode switching](../../docs/rfc/implemented/feature/2026-07-06-sandbox.md)): a capable client shows `Sandbox` (`read-only`/`workspace-write`/`danger-full-access`) and `Approvals` (`ask`/`never`) selectors per session — a switch is one log-only event on that session's log and execution follows it; the sandbox mode is deliberately NOT stated in the prompt or narrated (the model learns the boundary from the denial marker — behavior, not belief), while an approval switch to `never` is stated and narrated; a resumed session reports its overrides back on `session/load`.
|
||||
- **The write boundary is config-fixed**: an escalated `workspace-write` run may write under the launch directory (`workspaceRoot: process.cwd()`) plus the platform temp area — a per-session root is config-phase future work in the [sandbox RFC](../../docs/rfc/implemented/feature/2026-07-06-sandbox.md).
|
||||
- **No usable runner fails closed per command** (structured `SANDBOX_UNAVAILABLE`), and the filesystem tools stay unloaded for the same reason as `sandbox-agent`: they would bypass the bash sandbox.
|
||||
|
||||
Tests: `tests/escalation.e2e.ts` — keyless, it boots the real `cordis.yml` through the Loader as an ACP subprocess, proves the whole tree (sandbox executor + approval service + bridge) initializes and opens a session, and drives the config options end to end (both advertised with composition currents, switches honored and echoed as complete state, out-of-vocabulary values rejected); with a key and a usable runner, a scripted ACP client plays the human — the real model gets denied, escalates, the client answers `allow-once`, and the retried write must land on disk. `tests/acp.snapshot.ts` (the [shared snapshot kit](../../packages/support/acp-snapshot/) over this composition's `cordis.snapshot.yml` replay overlay) pins four scenarios as committed wire bytes: the keyless config-option exchange, the recorded `mode-switching` arc (the suite's pinned header — both switches, their prompt-section deltas, one "changed by the user" notice per knob, and a confined write landing under the switched mode), and both recorded escalation branches (`session/request_permission` answered allow-once / reject-once). Replay re-executes every recorded bash call under the host's real runner — Seatbelt works out of the box on macOS; on Linux install bubblewrap (or build the Landlock launcher) first, exactly what ci.yml's snapshot lane does. No fixture carries a real denial: denial stderr is backend dialect and would pin a fixture to its recording platform (the rationale comment atop the suite file).
|
||||
@@ -1,7 +0,0 @@
|
||||
{
|
||||
"name": "sandbox-acp-agent-example",
|
||||
"description": "Runnable demo: the sandboxed coding agent as an ACP server, with sandbox-escalation approval prompts answered by the editor",
|
||||
"private": true,
|
||||
"version": "0.0.1",
|
||||
"type": "module"
|
||||
}
|
||||
@@ -1,82 +0,0 @@
|
||||
import { dirname, join } from 'node:path'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
import { defineAcpSnapshotSuite, type Scenario, type SnapshotSuiteOptions } from '@deepseek-ai/dsh-acp-snapshot'
|
||||
|
||||
/**
|
||||
* Snapshot suite for the SANDBOXED composition (`../cordis.yml`, swapped to
|
||||
* the sibling `cordis.snapshot.yml` replay overlay by the bin under
|
||||
* `DSH_SNAPSHOT=replay`). Replay swaps only the MODEL for the recorded
|
||||
* transcript — every bash call re-executes for real under the host's actual
|
||||
* runner (Seatbelt on macOS, bwrap on Linux CI: ci.yml's snapshot lane
|
||||
* installs bubblewrap for exactly this), so the recorded scenarios double as
|
||||
* cross-backend confinement regression: an allowed command a runner change
|
||||
* starts denying fails replay outright. Their commands are limited to
|
||||
* `cat`/`printf` shapes whose bytes are identical across those backends and
|
||||
* across GNU/BSD userlands.
|
||||
*
|
||||
* Deliberately ABSENT: a scenario whose transcript carries a real sandbox
|
||||
* DENIAL. The harness-authored `[sandbox: file access denied …]` marker is
|
||||
* byte-stable, but the denied command's own stderr is the backend's dialect
|
||||
* (bwrap EROFS "Read-only file system", Landlock EACCES "Permission
|
||||
* denied", Seatbelt EPERM "Operation not permitted", GNU vs BSD phrasing on
|
||||
* top), and stderr reaches both compared surfaces — such a fixture replays
|
||||
* only on the platform that recorded it. The denial→marker path stays on
|
||||
* dsh-tool-bash's unit tests and the real-kernel sandbox e2e legs
|
||||
* (.github/workflows/sandbox.yml); the escalation scenarios below sidestep
|
||||
* it by having the USER assert the prior denial, so the recorded model
|
||||
* escalates without a platform-variant denial in the log.
|
||||
*/
|
||||
const SCENARIOS: Scenario[] = [
|
||||
// Protocol-only (keyless, authored): the session config-option surface
|
||||
// this composition adds — both advertised selects on session/new, the
|
||||
// complete refreshed state every session/set_config_option answers with,
|
||||
// and both rejection shapes — as committed wire bytes. No bash runs, so
|
||||
// this one still replays on runner-less hosts.
|
||||
{ name: 'config-options', hasModelTurn: false, recorded: false },
|
||||
// The runtime mode-switching arc, and NECESSARILY the pinned-header
|
||||
// scenario: an approval-policy switch rewrites its prompt section, and the
|
||||
// resulting request/header-delta is legal only in the pinning scenario
|
||||
// (the factory's uniformity guard). The pin commits this composition's
|
||||
// full header — persona, tool schemas WITH the escalation fields — plus
|
||||
// the approval delta and its "changed by the user" notice verbatim. The
|
||||
// SANDBOX switch is deliberately silent (no section, no notice — the
|
||||
// sandbox RFC's visibility asymmetry): the recorded arc proves it by
|
||||
// BEHAVIOR, a confined write landing under the switched mode with no
|
||||
// header change.
|
||||
{ name: 'mode-switching', hasModelTurn: true, recorded: true, pinsHeader: true, expectedHeaderDeltas: 1 },
|
||||
// The approval wire end-to-end, under the DEFAULT read-only/ask (a switch
|
||||
// would emit a header-delta the uniformity guard forbids here): the
|
||||
// escalating bash call streams, session/request_permission attaches to it
|
||||
// (allow-once / reject-once), and the scripted answer drives each branch —
|
||||
// an approved run executes CONFINED under the granted workspace-write; a
|
||||
// rejected one executes nothing and fails with the deterministic
|
||||
// rejection text.
|
||||
{ name: 'escalation-approved', hasModelTurn: true, recorded: true },
|
||||
{ name: 'escalation-rejected', hasModelTurn: true, recorded: true },
|
||||
]
|
||||
|
||||
function snapshotModeFromEnv(value: string | undefined): SnapshotSuiteOptions['mode'] {
|
||||
switch (value) {
|
||||
case undefined:
|
||||
case '':
|
||||
case 'replay':
|
||||
return 'replay'
|
||||
case 'record':
|
||||
return 'record'
|
||||
case 'refresh':
|
||||
return 'refresh'
|
||||
default:
|
||||
throw new Error(`unknown DSH_SNAPSHOT mode: ${value}`)
|
||||
}
|
||||
}
|
||||
|
||||
defineAcpSnapshotSuite({
|
||||
agent: {
|
||||
binScript: fileURLToPath(new URL('../../../packages/ui/acp-agent/src/bin.ts', import.meta.url)),
|
||||
configPath: fileURLToPath(new URL('../cordis.yml', import.meta.url)),
|
||||
tsconfigPath: fileURLToPath(new URL('../../../tsconfig.json', import.meta.url)),
|
||||
},
|
||||
snapshotsDir: join(dirname(fileURLToPath(import.meta.url)), 'snapshots'),
|
||||
scenarios: SCENARIOS,
|
||||
mode: snapshotModeFromEnv(process.env.DSH_SNAPSHOT),
|
||||
})
|
||||
+1
-1
@@ -70,7 +70,7 @@
|
||||
"demo:code-mode": "node scripts/demo-code-mode.mjs",
|
||||
"demo:cordis": "node --expose-internals --import tsx packages/ui/stdio-agent/src/bin.ts examples/cordis-agent/cordis.yml",
|
||||
"demo:acp": "node --import tsx packages/ui/acp-agent/src/bin.ts examples/acp-agent/cordis.yml",
|
||||
"demo:sandbox-acp": "node --import tsx packages/ui/acp-agent/src/bin.ts examples/sandbox-acp-agent/cordis.yml",
|
||||
"demo:sandbox-acp": "node --import tsx packages/ui/acp-agent/src/bin.ts examples/acp-agent/sandbox.cordis.yml",
|
||||
"postinstall": "node scripts/install-lefthook.mjs"
|
||||
},
|
||||
"devDependencies": {
|
||||
|
||||
@@ -9,4 +9,4 @@ The canonical three-package capability seam (see [capability seams](../../docs/r
|
||||
| `bash-sandbox/` | Sandbox-consuming `BashExecutor` (wraps every command argv via `ctx.sandbox`, stamps denial/enforcement facts; extends `bash-local`'s mechanics) | (registers `ctx.bash`) |
|
||||
| `tool-bash/` | Model-facing `bash`/`bash_output`/`bash_kill` tool schemas | (registers on `ctx.tools`) |
|
||||
|
||||
The interface lives at `bash/bash/`. `bash-sandbox` replacing `bash-local` without touching the interface or the tool is the split doing exactly what it exists for — a leaf `cordis.yml` picks one executor entry, plus a `ctx.sandbox` provider entry for the confined one (see [examples/sandbox-acp-agent](../../examples/sandbox-acp-agent/)).
|
||||
The interface lives at `bash/bash/`. `bash-sandbox` replacing `bash-local` without touching the interface or the tool is the split doing exactly what it exists for — a leaf `cordis.yml` picks one executor entry, plus a `ctx.sandbox` provider entry for the confined one (see [the acp-agent example's `sandbox.cordis.yml`](../../examples/acp-agent/)).
|
||||
|
||||
@@ -30,4 +30,4 @@ Deny-only at the seam: a denial is a reported fact, and this executor never nego
|
||||
workspaceRoot: !!js process.cwd()
|
||||
```
|
||||
|
||||
The keyless consumer-integration proofs are `tests/bwrap.e2e.ts`, `tests/landlock.e2e.ts`, and `tests/seatbelt.e2e.ts` (the real provider + real runner driven through `ctx.bash`, world-verified, each self-skipping where its runner is absent); see [`examples/sandbox-acp-agent`](../../../examples/sandbox-acp-agent/) for the runnable demo.
|
||||
The keyless consumer-integration proofs are `tests/bwrap.e2e.ts`, `tests/landlock.e2e.ts`, and `tests/seatbelt.e2e.ts` (the real provider + real runner driven through `ctx.bash`, world-verified, each self-skipping where its runner is absent); see [the acp-agent example's sandbox variant](../../../examples/acp-agent/) for the runnable demo.
|
||||
|
||||
@@ -9,4 +9,4 @@ The confinement half of the [capability-seam split](../../docs/rfc/implemented/a
|
||||
|
||||
The seam confines SAME-WORLD subprocesses only (shared filesystem and kernel). Containers, microVMs, and remote executors are NOT backends here — they replace whole capability implementations (`ctx.bash`, `ctx.fs`) as environment-coherent groups; the boundary is recorded in [the sandbox RFC](../../docs/rfc/implemented/feature/2026-07-06-sandbox.md).
|
||||
|
||||
Consumers today: [`bash/bash-sandbox`](../bash/bash-sandbox/) (wraps `['bash', '-c', command]`; see [examples/sandbox-acp-agent](../../examples/sandbox-acp-agent/) for the composed leaf). In-process tools (fs/web) cannot be confined by an OS wrapper — their sandbox semantics are policy at their own seams (the sandbox RFC's cross-family phase).
|
||||
Consumers today: [`bash/bash-sandbox`](../bash/bash-sandbox/) (wraps `['bash', '-c', command]`; see [the acp-agent example's sandbox variant](../../examples/acp-agent/) for the composed leaf). In-process tools (fs/web) cannot be confined by an OS wrapper — their sandbox semantics are policy at their own seams (the sandbox RFC's cross-family phase).
|
||||
|
||||
@@ -15,4 +15,4 @@ Every rung has its keyless world-proof (`tests/bwrap.e2e.ts`, `tests/landlock.e2
|
||||
name: '@deepseek-ai/dsh-sandbox-local'
|
||||
```
|
||||
|
||||
Consumers: [`@deepseek-ai/dsh-bash-sandbox`](../../bash/bash-sandbox/); see [`examples/sandbox-acp-agent`](../../../examples/sandbox-acp-agent/) for the runnable composition.
|
||||
Consumers: [`@deepseek-ai/dsh-bash-sandbox`](../../bash/bash-sandbox/); see [the acp-agent example's sandbox variant (`sandbox.cordis.yml`)](../../../examples/acp-agent/) for the runnable composition.
|
||||
|
||||
Reference in New Issue
Block a user