mirror of
https://github.com/deepseek-ai/deepseek-harness.git
synced 2026-08-29 04:26:38 +00:00
feat(webworker): support fs watches and confinement
This commit is contained in:
imccyu
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-08-20-webworker-node-face.md
|
||||
2026-08-20-webworker-node-face.md: 08119cce96eff244f8e9ada3462ce5d35c1b538d
|
||||
2026-08-20-webworker-node-face.zh.md: 573c0be055d066d2d6d0db11a2476ba528517727
|
||||
2026-08-20-webworker-node-face.md: 6e69af83354f1139a03d047a700e84e7e918a013
|
||||
2026-08-20-webworker-node-face.zh.md: 96a60e459372828273b3a4d1330a7b7eb8f2994f
|
||||
|
||||
@@ -6,21 +6,21 @@ English | [中文](2026-08-20-webworker-node-face.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
The worker runs the web profile's Cordis configuration byte for byte — no worker-specific rows — so a browser's missing platform must be replaced at the module layer, where a proxied module keeps its identity and changes its implementation. That covers three fronts: the Node builtins the tree imports, the filesystem those builtins answer from, and a process layer for the bash tool, which mounted, advertised itself to the model, and then failed on every call while `node:child_process` was a structural stub.
|
||||
The worker runs the web profile's Cordis configuration byte for byte — no worker-specific rows — so a browser's missing platform must be replaced at the module layer, where a proxied module keeps its identity and changes its implementation. That covers three fronts: the Node builtins the tree imports, the filesystem those builtins answer from, and a process layer for the bash tool. A structural `node:child_process` stub would let that tool mount and advertise itself to the model while every call fails.
|
||||
|
||||
## Decision
|
||||
|
||||
**Builtins.** The proxy table replaces Node builtins and external npm packages, never workspace or vendored modules. `./implemented/<module>.ts` carries real semantics over a worker data source; `./mock/<module>.ts` mounts silently and reports the missing capability when a call reaches it. The loader's table holds one memoized thunk per specifier — evaluation happens at first `require`, not at assembly — and each shim's exported face typechecks against Node's own module type, with the narrow, documented exceptions where structural identity (a real class) cannot be satisfied. The worker installs the `process` global itself and fills it into the table at assembly.
|
||||
|
||||
**VFS.** Memory is the truth. `statSync(path, { bigint: true })` returns Node's BigInt shape, and two fields carry real information because `dsh-fs-local`'s stale-write guard depends on them: `ino` is per-path identity from a monotonic counter (a recreated path reports a new identity), and `mtimeMs` is strictly increasing per entry (`max(now, previous + 1)`), because in-memory writes routinely land in one millisecond and an equal timestamp would let a stale overwrite pass. The hunt that produced this also fixed the silence around it: cordis's logger verbosity counts UP, so an exporter that declares no level drops every warning — `startWorkerHost` installs a console exporter with `levels: { default: 2 }` before any entry mounts.
|
||||
**VFS.** Memory is the truth. `statSync(path, { bigint: true })` returns Node's BigInt shape, and two fields carry real information because `dsh-fs-local`'s stale-write guard depends on them: `ino` is per-path identity from a monotonic counter (a recreated path reports a new identity), and `mtimeMs` is strictly increasing per entry (`max(now, previous + 1)`), because in-memory writes routinely land in one millisecond and an equal timestamp would let a stale overwrite pass. Committed mutations also drive the [Node-compatible watcher and confinement implementation](2026-08-23-webworker-vfs-watch-and-landlock.md). Boot diagnostics remain visible because cordis logger verbosity counts UP: `startWorkerHost` installs a console exporter with `levels: { default: 2 }` before any entry mounts, while an exporter with no declared level drops every warning.
|
||||
|
||||
**Shell.** `node:child_process` is a real implementation over the VFS. The grammar is bought — `@yarnpkg/parsers`' `parseShell` — and the evaluator and command table are owned, because every candidate interpreter brings its own filesystem: pipelines are strings handed along, and each program is a function over the VFS. The table is the machine's whole `/bin`; an absent name reports `command not found` (127). Each `spawn` starts a child Web Worker from this same bundle, its first frame declaring the shell-process role, so the termination ladder is real: `SIGTERM` asks at the next command boundary, `SIGKILL` terminates the worker mid-loop — the preemption an in-thread interpreter can never have. The filesystem face is asynchronous end to end (child frames to the host VFS); `execSync`, `execFileSync`, and `fork` refuse, and `node-pty` stays a stub.
|
||||
**Shell.** `node:child_process` is a real implementation over the VFS. The grammar is bought — `@yarnpkg/parsers`' `parseShell` — and the evaluator and command table are owned, because every candidate interpreter brings its own filesystem: pipelines are strings handed along, and each program is a function over the VFS. Ordinary commands resolve from that table; native-package protocols may contribute Worker-owned virtual executable wrappers through the [watcher and confinement decision](2026-08-23-webworker-vfs-watch-and-landlock.md). A name in neither set reports `ENOENT` at direct spawn or `command not found` (127) inside shell source. Each `spawn` starts a child Web Worker from this same bundle, its first frame declaring the shell-process role, so the termination ladder is real: `SIGTERM` asks at the next command boundary, `SIGKILL` terminates the worker mid-loop — the preemption an in-thread interpreter can never have. The filesystem face is asynchronous end to end (child frames to the host VFS); `execSync`, `execFileSync`, and `fork` refuse, and `node-pty` stays a stub.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**Replacing `dsh-subprocess-local` or the bash executor.** The first would let the proxy table replace a workspace package against its own classification and invert the layering; the second trips `dsh-permission-presets`' boot-time `sandboxMode` validation and drops tested timeout/output behavior.
|
||||
|
||||
**`@yarnpkg/shell`, WASM shells, WebContainer.** The matching interpreter is built on real Node streams (~1.5 MB closure to own); WASM was removed from this deployment by decision and WASI has no `fork`; all of them arrive with their own filesystem, the one part that cannot be reused.
|
||||
**`@yarnpkg/shell`, WASM shells, WebContainer.** The matching interpreter is built on real Node streams (~1.5 MB closure to own); this deployment excludes WASM and WASI has no `fork`; all of them arrive with their own filesystem, the one part that cannot be reused.
|
||||
|
||||
**`SharedArrayBuffer` + `Atomics.wait` for a synchronous child filesystem.** Measured on the deployment target: without COOP/COEP headers `SharedArrayBuffer` is not defined, and GitHub Pages cannot set response headers. The asynchronous face is a superset; a SAB backend can slot under it later without touching a program.
|
||||
|
||||
@@ -28,7 +28,7 @@ The worker runs the web profile's Cordis configuration byte for byte — no work
|
||||
|
||||
## Consequences
|
||||
|
||||
- Sandbox modes other than `danger-full-access` fail loud: `SandboxEnforcement` has no "nothing was enforced" value and a browser has no kernel, so `ctx.sandbox.confine` fails closed and the command never starts. Real enforcement at the VFS frame gate is a designed follow-up, not this note.
|
||||
- `read-only` and `workspace-write` interpret the native Landlock launcher protocol and enforce per-process grants at the VFS frame gate; `danger-full-access` keeps the direct process path. The [watcher and confinement decision](2026-08-23-webworker-vfs-watch-and-landlock.md) owns the narrower meaning of `full` in this execution world.
|
||||
- The Node-host ladder test (`tests/node/child-process.spec.ts`) is registered windows-unsupported: the ladder's win32 kill rung is taskkill-by-real-pid, undeliverable to a process-table pid, while the worker itself always reports `linux`.
|
||||
- Output is incremental but not streamed: programs write into sinks forwarded as `data` events, and a pipeline stage completes before the next starts.
|
||||
- The runtime's tests mirror `src/` (`tests/node/`, `tests/shell/`, `tests/storage/`, …), so each shim family owns its behavior cases beside the oracle-diff suites.
|
||||
|
||||
@@ -6,21 +6,21 @@
|
||||
|
||||
## 问题
|
||||
|
||||
worker 逐字节运行 web profile 的 Cordis 配置——没有 worker 专属行——因此浏览器缺失的平台必须在模块层被替换：被代理的模块保持身份、更换实现。这覆盖三条战线：树所 import 的 Node builtin、这些 builtin 背后应答的文件系统，以及 bash 工具的进程层——在 `node:child_process` 还是结构桩的时期，工具照常挂载、向模型自我宣告，然后每次调用都失败。
|
||||
worker 逐字节运行 web profile 的 Cordis 配置——没有 worker 专属行——因此浏览器缺失的平台必须在模块层被替换：被代理的模块保持身份、更换实现。这覆盖三条战线：树所 import 的 Node builtin、这些 builtin 背后应答的文件系统，以及 bash 工具的进程层。如果 `node:child_process` 只是结构桩，工具仍会照常挂载并向模型自我宣告，但每次调用都会失败。
|
||||
|
||||
## 决定
|
||||
|
||||
**Builtin。** 代理表只替换 Node builtin 与外部 npm 包，绝不替换 workspace 或 vendored 模块。`./implemented/<module>.ts` 在 worker 数据源之上承载真语义；`./mock/<module>.ts` 静默挂载、在调用真正抵达时报告缺失的能力。装载器的表按 specifier 各持一个 memoized thunk——求值发生在首次 `require` 而非装配期——且每个垫片的导出面对 Node 自身的模块类型作类型检查，仅在结构身份（真实类）确不可满足处留最窄的、有说明的例外。`process` 全局由 worker 自装，装配期填入表中。
|
||||
|
||||
**VFS。** 内存为真相。`statSync(path, { bigint: true })` 返回 Node 的 BigInt 形状，其中两个字段承载真实信息，因为 `dsh-fs-local` 的 stale-write guard 依赖它们：`ino` 是按路径的身份（单调计数器分配，路径重建即新身份），`mtimeMs` 按条目严格递增（`max(now, previous + 1)`）——内存写例行落在同一毫秒内，相等的时间戳会放过陈旧覆写。这场排查同时修掉了它周围的静默：cordis 日志器的详细度数值向上计数，未声明等级的 exporter 会丢掉所有 warning——`startWorkerHost` 在任何 entry 挂载前安装 `levels: { default: 2 }` 的 console exporter。
|
||||
**VFS。** 内存为真相。`statSync(path, { bigint: true })` 返回 Node 的 BigInt 形状，其中两个字段承载真实信息，因为 `dsh-fs-local` 的 stale-write guard 依赖它们：`ino` 是按路径的身份（单调计数器分配，路径重建即新身份），`mtimeMs` 按条目严格递增（`max(now, previous + 1)`）——内存写例行落在同一毫秒内，相等的时间戳会放过陈旧覆写。已提交的 mutation 还会驱动 [Node 兼容 watcher 与 confinement 实现](2026-08-23-webworker-vfs-watch-and-landlock.zh.md)。Cordis 日志器的详细度数值向上计数，因此 `startWorkerHost` 会在任何 entry 挂载前安装 `levels: { default: 2 }` 的 console exporter，避免未声明等级的 exporter 丢掉所有 warning。
|
||||
|
||||
**Shell。** `node:child_process` 是 VFS 之上的真实现。语法是买来的——`@yarnpkg/parsers` 的 `parseShell`——求值器与命令表是自有的，因为每个候选解释器都自带文件系统：管道是逐段传递的字符串，每个程序是 VFS 上的一个函数。命令表就是这台机器的全部 `/bin`；不存在的名字报告 `command not found`（127）。每次 `spawn` 从同一个 bundle 起一个子 Web Worker，首帧声明 shell 进程角色，因此终止梯是真的：`SIGTERM` 在下一命令边界处请求停止，`SIGKILL` 在任意时刻终止 worker——这是线程内解释器永远没有的抢占。文件系统面端到端异步（子进程经帧到宿主 VFS）；`execSync`、`execFileSync`、`fork` 拒绝，`node-pty` 保持桩。
|
||||
**Shell。** `node:child_process` 是 VFS 之上的真实现。语法是买来的——`@yarnpkg/parsers` 的 `parseShell`——求值器与命令表是自有的，因为每个候选解释器都自带文件系统：管道是逐段传递的字符串，每个程序是 VFS 上的一个函数。普通命令从该表解析；native 包协议可以通过 [watcher 与 confinement 决策](2026-08-23-webworker-vfs-watch-and-landlock.zh.md)提供 Worker 自有的虚拟 executable wrapper。两处都没有的名字在直接 spawn 时报告 `ENOENT`，在 shell source 中则报告 `command not found`（127）。每次 `spawn` 从同一个 bundle 起一个子 Web Worker，首帧声明 shell 进程角色，因此终止梯是真的：`SIGTERM` 在下一命令边界处请求停止，`SIGKILL` 在任意时刻终止 worker——这是线程内解释器永远没有的抢占。文件系统面端到端异步（子进程经帧到宿主 VFS）；`execSync`、`execFileSync`、`fork` 拒绝，`node-pty` 保持桩。
|
||||
|
||||
## 曾考虑的替代方案
|
||||
|
||||
**整包替换 `dsh-subprocess-local` 或替换 bash 执行器。** 前者让代理表首次替换 workspace 包、违背其自身分类并倒置分层；后者撞上 `dsh-permission-presets` 对 `sandboxMode` 的 boot 期硬校验，并丢掉执行器已被测试钉住的超时/输出行为。
|
||||
|
||||
**`@yarnpkg/shell`、WASM shell、WebContainer。** 配套解释器建立在真实 Node streams 之上（约 1.5 MB 闭包要自养）；WASM 已被本部署的决定排除，WASI 没有 `fork`；且它们全都自带文件系统——恰是无法复用的那部分。
|
||||
**`@yarnpkg/shell`、WASM shell、WebContainer。** 配套解释器建立在真实 Node streams 之上（约 1.5 MB 闭包要自养）；本部署排除 WASM，WASI 没有 `fork`；且它们全都自带文件系统——恰是无法复用的那部分。
|
||||
|
||||
**`SharedArrayBuffer` + `Atomics.wait` 给子进程同步文件系统。** 在部署目标实测：无 COOP/COEP 头时 `SharedArrayBuffer` 未定义，而 GitHub Pages 无法设置响应头。异步面是超集；SAB 后端将来可垫入其下而不动任何程序。
|
||||
|
||||
@@ -28,7 +28,7 @@ worker 逐字节运行 web profile 的 Cordis 配置——没有 worker 专属
|
||||
|
||||
## 后果
|
||||
|
||||
- `danger-full-access` 之外的沙箱档 fail loud：`SandboxEnforcement` 没有「未执法」值、浏览器没有内核，`ctx.sandbox.confine` 落闭、命令零启动。在 VFS 帧闸口做真执法是设计中的后续，不属本条。
|
||||
- `read-only` 与 `workspace-write` 解释 native Landlock launcher 协议，并在 VFS 帧闸口执行逐进程授权；`danger-full-access` 保持直接进程路径。[Watcher 与 confinement 决策](2026-08-23-webworker-vfs-watch-and-landlock.zh.md)拥有该执行世界中 `full` 的更窄含义。
|
||||
- Node 宿主的阶梯测试（`tests/node/child-process.spec.ts`）登记为 windows 不支持：阶梯的 win32 kill 梯级是按真 pid 的 taskkill，对进程表 pid 不可投递，而 worker 自身恒报 `linux`。
|
||||
- 输出增量但不流式：程序写入的 sink 以 `data` 事件转发，一个管道阶段完成后下一阶段才开始。
|
||||
- 运行时的测试镜像 `src/`（`tests/node/`、`tests/shell/`、`tests/storage/`……），每个垫片族在 oracle-diff 套件旁拥有自己的行为用例。
|
||||
|
||||
+6
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-08-23-webworker-vfs-watch-and-landlock.md
|
||||
2026-08-23-webworker-vfs-watch-and-landlock.md: 61254092e0f32e8e4291fd7c21489684110a1d15
|
||||
2026-08-23-webworker-vfs-watch-and-landlock.zh.md: 32e1b36e0ef17e4574252693e246f9d7cd4d4712
|
||||
@@ -0,0 +1,80 @@
|
||||
# Agent Note: Web Worker VFS watching and CLI-compatible confinement
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-08-23-webworker-vfs-watch-and-landlock.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
The Web Worker preview boots the same Web profile and Agent presets as the Node host. Without a VFS change source, refusing `node:fs.watchFile` makes `skill-filesystem` return an incomplete observation and re-scan on every lookup, while an inert success leaves an existing root waiting forever for Chokidar's `ready`. Settings and credentials likewise need real external-edit events rather than a package-specific fake.
|
||||
|
||||
The same composition mounts `sandbox-local`, whose Linux chain probes bwrap and then `@deepseek-ai/node-addon-landlock-run`. A Worker cannot execute either binary. Ending the selection there makes `workspace-write` and `read-only` unusable even though every shell filesystem operation already crosses a Host-side VFS call point.
|
||||
|
||||
The filesystem compatibility boundary follows the [Worker Node face decision](2026-08-20-webworker-node-face.md): pure JavaScript watcher packages run unchanged over Node-compatible modules. Native or binary packages may keep their public JavaScript API and executable protocol while replacing the backend. An API that cannot preserve its caller-visible Node behavior remains explicitly unavailable; `node:vm` is outside this decision.
|
||||
|
||||
## Decision
|
||||
|
||||
### VFS mutation source and filesystem watchers
|
||||
|
||||
`MemoryVfs` publishes committed `write`, `mkdir`, `remove`, and `chmod` mutations to any number of subscribers. Publication happens after state changes, failed operations publish nothing, image seeding stays silent, and one throwing subscriber cannot fail the filesystem operation or starve another subscriber. Rename is a source removal plus complete destination mkdir/write records; destination writes mark the directory entry as changed, so watchers report `rename` while a future durable sink receives the bytes needed to materialize the destination. Directory mtimes advance when their immediate entry set changes, so polling detects child creation and removal as Node does.
|
||||
|
||||
The mutation record is shared with WebFS persistence rather than defining a second notification path. Writes carry their complete post-commit bytes and virtual permission bits, plus an append offset when only a tail changed. `MemoryVfs` accepts an optional asynchronous `VfsMutationSink`, sends the same records to that sink and live watcher subscribers, and exposes `flush()` through file-handle `sync()` and `datasync()`. Hydration supplies `{ mode, mtimeMs }` explicitly, so image permissions and durable timestamps cannot occupy the same positional argument. This change mounts no durable sink; it keeps the synchronous in-memory tree authoritative so an OPFS or user-directory mirror can hydrate before publication and write behind without changing `node:fs`.
|
||||
|
||||
The `node:fs` implementation provides callback `stat` and `lstat`, `watch`, `watchFile`, `unwatchFile`, `FSWatcher`, and `StatWatcher`; `node:fs/promises.watch` provides the abortable async iterator. One path shares one `StatWatcher` across listeners, listener-specific unwatching leaves peers active, and missing paths report zero-valued Stats before later creation, deletion, and recreation transitions. Callback dispatch captures the registration-time async context and checks closure before every queued delivery.
|
||||
|
||||
`fs.watch` maps entry creation, removal, and rename destinations to `rename`, and maps content or mode changes to `change`. Non-recursive directory watches report immediate child names; recursive watches report paths relative to the watched directory. The VFS has no symlinks, so this implementation does not invent symlink events.
|
||||
|
||||
### Streams and unchanged npm packages
|
||||
|
||||
`node:stream` uses the maintained `readable-stream` browser implementation for `Readable`, `Writable`, `Duplex`, `Transform`, `PassThrough`, pipeline helpers, async iteration, backpressure, aborts, and teardown ordering. The compatibility module sets the byte high-water default to the 64 KiB value used by the repository's Node 22+ engines. VFS-backed `ReadStream` and `WriteStream` supply file descriptors, inclusive ranges, encoding, append or replace behavior, byte accounting, AbortSignal handling, and `open`/`ready`/`finish`/`end`/`close` ordering.
|
||||
|
||||
Chokidar and readdirp are ordinary image dependencies, not module replacements. Their package code runs unchanged and imports the Worker implementations of `node:fs`, `node:fs/promises`, `node:stream`, `node:events`, `node:path`, and `node:os`. Chokidar therefore retains its own initial scan, `ready`, polling, atomic-write normalization, write-settle delay, shared watcher, and close behavior.
|
||||
|
||||
### Landlock CLI over per-process VFS grants
|
||||
|
||||
`@deepseek-ai/node-addon-landlock-run` is an ordinary image dependency, not a module replacement. Its unchanged JavaScript entry runs through the Worker implementations of `node:child_process`, `node:module`, `node:path`, and `node:url`, so the package remains the sole owner of `LAUNCHER_BIN`, `LAUNCHER_FAILURE_EXIT`, `launcherPath()`, `grantArgs()`, and `probe()`. The image may include the matching Linux optional package, but package resolution does not decide whether the Worker platform supplies Landlock: the entry package's deterministic fallback path reaches the same platform executable implementation when that optional package is absent.
|
||||
|
||||
The process layer has a table of Worker platform executables identified by logical executable name rather than one package-manager path. Its `landlock-run` provider accepts a bare command or an absolute launcher path, parses the native package's unchanged CLI, validates every grant root, and delegates the inner argv to the existing shell process runner. `node:child_process` performs only generic executable lookup, output delivery, and settlement. The unchanged package's synchronous `probe()` therefore observes the provider through `spawnSync` and reports `full`. A usage error, missing grant root, or unknown inner executable prints one `landlock-run: ...` line, exits `125`, and never runs the inner command. The bwrap probe remains unavailable, so the unmodified `sandbox-local` Linux chain selects this Landlock backend.
|
||||
|
||||
Each launched process receives its own `ShellFileSystem` guard. `stat`, `list`, and `readText` require a read-only or read-write grant; `writeText`, `mkdir`, and `remove` require a read-write grant; `rename` requires both source and destination to be writable. Denials carry `EACCES` and `permission denied`, preserving `bash-sandbox` denial classification. `/tmp` maps to the VFS `/dsh/tmp`, while `/dev/null` is a virtual empty-read and discarded-write file that stores no bytes.
|
||||
|
||||
The Worker's `full` verdict covers every file operation expressible through its shell command table and Host-served VFS protocol. It does not claim Linux kernel Landlock, arbitrary native executable support, or protection against a future shell program that bypasses `ShellFileSystem`.
|
||||
|
||||
### Explicitly deferred behavior
|
||||
|
||||
`node:vm`, `node:worker_threads`, `node:net`, `node:sqlite`, native PTY, Sharp, and ripgrep remain outside this change. The VFS remains POSIX-only, in-memory, and symlink-free. Browser Workers have no libuv-style ref-counted event loop, so watcher `persistent`, `ref()`, and `unref()` preserve the API and observable state but cannot decide Worker lifetime.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**Disable watcher and sandbox rows in the Worker profile.** A smaller composition would stop testing the same Host tree and would hide package integration failures specific to preview deployment.
|
||||
|
||||
**Make `watchFile` an inert success.** Missing roots would never advance, and an existing root would wait forever for Chokidar `ready`.
|
||||
|
||||
**Notify watchers only from `node:fs`.** Shell process requests and any direct VFS writer would bypass the notification point. The commit owner, `MemoryVfs`, is the only complete source.
|
||||
|
||||
**Keep a VFS-specific Chokidar replacement.** This duplicates directory scans, ready accounting, write settling, atomic replacement, shared watcher ownership, and teardown already maintained upstream.
|
||||
|
||||
**Replace the Landlock entry package with a Worker module.** Reimplementing its exported constants, grant builder, launcher resolution, and probe would create a second copy of a package contract that already runs over the Worker Node compatibility layer. Only the platform executable implementation differs.
|
||||
|
||||
**Recognize one exact launcher path.** Optional-dependency installation and the entry package's documented fallback produce different absolute paths for the same executable. Package-manager layout is not the identity of a platform capability, so executable dispatch uses the logical `landlock-run` name.
|
||||
|
||||
**Add a Worker branch to `sandbox-local`.** This would copy policy-to-grant mapping into a business package. Interpreting the existing launcher protocol preserves the provider, consumer, configuration, diagnostics, and native package API.
|
||||
|
||||
**Store one active policy on the global VFS.** Concurrent foreground, background, and escalated commands would overwrite one another's authority. Grants belong to one process handle and its filesystem adapter.
|
||||
|
||||
## Verification
|
||||
|
||||
- `fs-watch-stream.spec.ts` compares missing/create/change/remove `watchFile` transitions and file-stream lifecycle, chunking, range, backpressure, byte count, defaults, and abort identity with the running Node version.
|
||||
- `chokidar.spec.ts` loads both lockfile-selected Chokidar and readdirp dependency pairs through the Worker transformer and module loader, then proves `ready`, callback watching, polling, missing-file creation, removal, and quiescent close over `MemoryVfs`.
|
||||
- `image-loadable.spec.ts` packs and loads the real `@deepseek-ai/node-addon-landlock-run` JavaScript, proves it is absent from the replacement table, and runs its fallback `launcherPath()` and `probe()` through the Worker platform executable. `child-process.spec.ts` and `sandbox-stack.spec.ts` then prove the launcher failure code, malformed argv and grant failures, `/tmp` and `/dev/null`, rename denial, all three permission modes, and concurrent process-local grants through the production sandbox and subprocess packages.
|
||||
- `preview-boot.e2e.ts` builds and boots the packed browser deployment, creates a Workspace and Session, advances missing skill roots into a live Chokidar watch, lists the catalog, and completes settings and credential writes without watcher warnings.
|
||||
|
||||
## Consequences
|
||||
|
||||
The preview now runs npm watcher consumers without source forks, and filesystem mutations observed from Host code or shell process Workers share one ordered commit source. A WebFS/OPFS integration remains an asynchronous mirror around this synchronous authority and consumes that same source; it does not add another Chokidar implementation or a competing mutation protocol.
|
||||
|
||||
Worker `read-only` and `workspace-write` preserve the product's permission vocabulary and denial reporting without forking the Landlock npm package. Their security claim is narrower than native Landlock but complete inside the Worker execution world; any new filesystem message or shell program must continue through the guarded `ShellFileSystem`. Native-backed packages follow the same ownership rule: their JavaScript remains upstream, while the Worker platform replaces only the native artifact behind it.
|
||||
|
||||
The worker bundle gains `readable-stream` and its small browser dependency closure. In return, stream state and backpressure remain maintained upstream instead of becoming local compatibility code.
|
||||
|
||||
Watcher event timing is deterministic from VFS commits rather than inherited from an operating-system backend. This stays within Node's watcher contract, which does not guarantee native event coalescing, while tests pin every event distinction the current consumers require.
|
||||
+80
@@ -0,0 +1,80 @@
|
||||
# Agent Note: Web Worker VFS 监听与 CLI 兼容 confinement
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-08-23-webworker-vfs-watch-and-landlock.md) | 中文
|
||||
|
||||
## Problem
|
||||
|
||||
Web Worker preview 启动与 Node host 相同的 Web profile 和 Agent preset。缺少 VFS 变更源时，拒绝 `node:fs.watchFile` 会让 `skill-filesystem` 返回不完整观测并在每次查询时重新扫描，而无事件的成功调用会让已有根永远等待 Chokidar 的 `ready`。Settings 和 credentials 同样需要真实的外部编辑事件，而不是包专用 fake。
|
||||
|
||||
同一组合挂载 `sandbox-local`，其 Linux 选择链依次探测 bwrap 和 `@deepseek-ai/node-addon-landlock-run`。Worker 无法执行这两个二进制文件。如果选择链到此结束，`workspace-write` 与 `read-only` 将不可用，尽管 shell 的每项文件系统操作已经经过 Host 侧 VFS 调用点。
|
||||
|
||||
文件系统兼容边界遵循 [Worker Node face 决策](2026-08-20-webworker-node-face.zh.md)：纯 JavaScript watcher 包在 Node 兼容模块之上保持原样运行。Native 或 binary 包可以保持公开 JavaScript API 与可执行文件协议，同时替换执行后端。无法维持调用方可见 Node 行为的 API 继续明确标记为不可用；`node:vm` 不属于本决策范围。
|
||||
|
||||
## Decision
|
||||
|
||||
### VFS mutation source 与文件 watcher
|
||||
|
||||
`MemoryVfs` 向任意数量的订阅方发布已提交的 `write`、`mkdir`、`remove` 和 `chmod` mutation。状态改变后才发布，失败操作不发布，镜像 seed 保持无事件，一个抛错的订阅方也不能让文件系统操作失败或阻止其他订阅方。Rename 被表达为源路径删除与包含完整状态的目标 mkdir/write 记录；目标 write 会标记目录项已改变，因此 watcher 报告 `rename`，未来的 durable sink 同时拿到物化目标所需的字节。目录的直接条目集合改变时，其 mtime 会推进，因此 polling 能像 Node 一样发现子项创建和删除。
|
||||
|
||||
Mutation record 与 WebFS 持久化共用，而不建立第二条通知路径。Write 记录携带提交后的完整字节与虚拟权限位，并在只有尾部变化时携带 append offset。`MemoryVfs` 接受可选的异步 `VfsMutationSink`，把同一批记录交给 sink 与实时 watcher 订阅方，并通过文件句柄的 `sync()` 和 `datasync()` 暴露 `flush()`。水合通过显式的 `{ mode, mtimeMs }` 传入元数据，因此镜像权限与持久化时间戳不会占用同一个位置参数。本次变更不挂载 durable sink；同步内存树继续作为权威，因此 OPFS 或用户目录 mirror 可以先水合、再异步写回，而无需改变 `node:fs`。
|
||||
|
||||
`node:fs` 实现 callback `stat` 和 `lstat`、`watch`、`watchFile`、`unwatchFile`、`FSWatcher` 与 `StatWatcher`；`node:fs/promises.watch` 提供可由 abort 取消的异步迭代器。同一路径的 listener 共享一个 `StatWatcher`，按 listener 取消监听不会影响其他 listener；缺失路径先报告零值 Stats，随后再报告创建、删除和重建状态。Callback 分发捕获注册时的异步上下文，并在每次排队交付前检查 watcher 是否已经关闭。
|
||||
|
||||
`fs.watch` 把条目创建、删除和 rename 目标映射为 `rename`，把内容或 mode 变化映射为 `change`。非递归目录 watcher 报告直接子项名，递归 watcher 报告相对被监听目录的路径。VFS 没有符号链接，因此该实现不会制造符号链接事件。
|
||||
|
||||
### Stream 与未修改的 NPM 包
|
||||
|
||||
`node:stream` 使用维护中的 `readable-stream` 浏览器实现来提供 `Readable`、`Writable`、`Duplex`、`Transform`、`PassThrough`、pipeline helper、异步迭代、backpressure、abort 和 teardown 顺序。兼容模块把字节流 high-water mark 默认值设为仓库 Node 22+ 引擎使用的 64 KiB。VFS 支持的 `ReadStream` 与 `WriteStream` 提供文件描述符、闭区间范围、encoding、追加或替换行为、字节计数、AbortSignal 处理，以及 `open`、`ready`、`finish`、`end`、`close` 顺序。
|
||||
|
||||
Chokidar 和 readdirp 作为普通镜像依赖运行，不属于模块 replacement。它们的包代码保持原样，并导入 Worker 实现的 `node:fs`、`node:fs/promises`、`node:stream`、`node:events`、`node:path` 与 `node:os`。因此，初次扫描、`ready`、polling、原子写归一化、写入稳定等待、共享 watcher 与关闭行为仍由 Chokidar 自己负责。
|
||||
|
||||
### 基于逐进程 VFS 授权的 Landlock CLI
|
||||
|
||||
`@deepseek-ai/node-addon-landlock-run` 是普通镜像依赖，不是模块 replacement。其未经修改的 JavaScript 入口通过 Worker 实现的 `node:child_process`、`node:module`、`node:path` 与 `node:url` 运行，因此该包仍是 `LAUNCHER_BIN`、`LAUNCHER_FAILURE_EXIT`、`launcherPath()`、`grantArgs()` 和 `probe()` 的唯一所有者。镜像可以包含匹配的 Linux optional package，但包解析不决定 Worker 平台是否提供 Landlock；缺少该 optional package 时，入口包产生的确定性 fallback 路径仍到达同一个平台可执行文件实现。
|
||||
|
||||
进程层持有按逻辑可执行文件名识别的 Worker 平台可执行文件表，而不依赖某一个包管理器路径。其 `landlock-run` provider 接受裸命令或绝对 launcher 路径，解析 native 包未经修改的 CLI、校验每个授权根，并把内部 argv 交给既有 shell 进程 runner。`node:child_process` 只负责通用的可执行文件查找、输出投递与结束处理。因此，原包的同步 `probe()` 会通过 `spawnSync` 观察到该 provider 并报告 `full`。用法错误、缺失的授权根或未知内部可执行文件只输出一行 `landlock-run: ...`，以 `125` 退出，并且绝不运行内部命令。bwrap 仍探测为不可用，因此未修改的 `sandbox-local` Linux 选择链会选中该 Landlock 后端。
|
||||
|
||||
每个已启动进程分别获得一个 `ShellFileSystem` guard。`stat`、`list` 和 `readText` 需要只读或读写授权；`writeText`、`mkdir` 和 `remove` 需要读写授权；`rename` 要求源和目标都可写。拒绝错误包含 `EACCES` 与 `permission denied`，从而保持 `bash-sandbox` 的拒绝分类。`/tmp` 映射到 VFS 的 `/dsh/tmp`，`/dev/null` 则是空读、丢弃写入且不保存任何字节的虚拟文件。
|
||||
|
||||
Worker 的 `full` 结论覆盖 shell 命令表和 Host 服务 VFS 协议能够表达的全部文件操作。它不表示 Linux 内核 Landlock、不支持任意 native 可执行文件，也无法约束未来绕过 `ShellFileSystem` 的 shell 程序。
|
||||
|
||||
### 明确延后的行为
|
||||
|
||||
`node:vm`、`node:worker_threads`、`node:net`、`node:sqlite`、native PTY、Sharp 和 ripgrep 不属于本次变更。VFS 仍然只支持 POSIX、内存存储且没有符号链接。Browser Worker 没有 libuv 风格的引用计数事件循环，因此 watcher 的 `persistent`、`ref()` 和 `unref()` 保留 API 与可观察状态，但不能决定 Worker 生存期。
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**在 Worker profile 中禁用 watcher 与 sandbox 配置项。** 缩减组合后将不再测试相同的 Host tree，还会隐藏 preview 部署特有的包集成故障。
|
||||
|
||||
**让 `watchFile` 成为无事件的成功调用。** 缺失根永远无法推进，已有根则会永久等待 Chokidar `ready`。
|
||||
|
||||
**只从 `node:fs` 通知 watcher。** Shell 进程请求以及直接写 VFS 的实现可以绕过通知点。只有提交状态的 `MemoryVfs` 才是完整真源。
|
||||
|
||||
**保留 VFS 专用的 Chokidar replacement。** 这会重复实现上游已经维护的目录扫描、ready 计数、写入稳定等待、原子替换、共享 watcher 所有权和 teardown。
|
||||
|
||||
**用 Worker 模块替换 Landlock 入口包。** 重新实现其导出常量、授权参数构造、launcher 解析和 probe，会为一个已经能在 Worker Node 兼容层上运行的包约定建立第二份副本。只有平台可执行文件实现需要不同。
|
||||
|
||||
**只识别一个精确 launcher 路径。** Optional dependency 的安装状态与入口包已有的 fallback 会为同一个可执行文件产生不同的绝对路径。包管理器布局不是平台能力的身份，因此可执行文件分发使用逻辑名称 `landlock-run`。
|
||||
|
||||
**在 `sandbox-local` 中增加 Worker 分支。** 这会把策略到授权的映射复制到业务包中。解释现有 launcher 协议可以保持 provider、consumer、配置、诊断和 native 包 API 不变。
|
||||
|
||||
**在全局 VFS 上保存一个当前策略。** 并发前台、后台和升权命令会覆盖彼此的权限。授权必须归属于单个进程句柄及其文件系统适配器。
|
||||
|
||||
## Verification
|
||||
|
||||
- `fs-watch-stream.spec.ts` 对照当前 Node 版本验证缺失、创建、修改、删除的 `watchFile` 状态转换，以及文件流生命周期、分片、范围、backpressure、字节计数、默认值和 abort 身份。
|
||||
- `chokidar.spec.ts` 通过 Worker transformer 与模块 loader 加载 lockfile 选定的两组 Chokidar 和 readdirp 依赖，并在 `MemoryVfs` 上验证 `ready`、callback watcher、polling、缺失文件创建、删除和完全停稳的关闭。
|
||||
- `image-loadable.spec.ts` 打包并加载真实的 `@deepseek-ai/node-addon-landlock-run` JavaScript，验证它不在 replacement 表中，并让其 fallback `launcherPath()` 与 `probe()` 经过 Worker 平台可执行文件。`child-process.spec.ts` 与 `sandbox-stack.spec.ts` 随后通过生产 sandbox 和 subprocess 包验证 launcher 失败码、错误 argv 与授权失败、`/tmp` 与 `/dev/null`、rename 拒绝、三种权限模式和逐进程并发授权。
|
||||
- `preview-boot.e2e.ts` 构建并启动打包后的浏览器部署，创建 Workspace 与 Session，把缺失的 skill 根逐级推进到可用的 Chokidar watch，读取 catalog，并在没有 watcher 警告的情况下完成 settings 与 credential 写入。
|
||||
|
||||
## Consequences
|
||||
|
||||
Preview 现在可以在不 fork 源码的情况下运行 NPM watcher 消费方；Host 代码与 shell 进程 Worker 产生的文件系统 mutation 共享同一个有序提交源。WebFS/OPFS 集成仍是围绕该同步权威的异步 mirror，并消费同一个变更源；它不会增加另一份 Chokidar 实现或互相竞争的 mutation 协议。
|
||||
|
||||
Worker `read-only` 与 `workspace-write` 在不 fork Landlock NPM 包的情况下保留产品权限词汇和拒绝报告。其安全结论比 native Landlock 更窄，但完整覆盖 Worker 执行世界；任何新的文件系统消息或 shell 程序都必须继续经过受 guard 保护的 `ShellFileSystem`。Native-backed 包遵循同一所有权规则：其 JavaScript 保持上游实现，Worker 平台只替换背后的 native artifact。
|
||||
|
||||
Worker bundle 增加 `readable-stream` 及其少量浏览器依赖。相应地，stream 状态和 backpressure 继续由上游维护，不成为本地兼容代码。
|
||||
|
||||
Watcher 事件时序由 VFS 提交确定，而不是继承操作系统后端。Node watcher 约定本身不保证 native 事件合并方式，因此该实现仍符合约定；测试固定当前消费方依赖的每一种事件区别。
|
||||
@@ -87,6 +87,7 @@ External packages that a workspace package resolves at runtime. The tier covers
|
||||
| [`picomatch`](https://github.com/micromatch/picomatch) | MIT |
|
||||
| [`react`](https://github.com/facebook/react) | MIT |
|
||||
| [`react-dom`](https://github.com/facebook/react) | MIT |
|
||||
| [`readable-stream`](https://github.com/nodejs/readable-stream) | MIT |
|
||||
| [`sharp`](https://github.com/lovell/sharp) | Apache-2.0 |
|
||||
| [`shiki`](https://github.com/shikijs/shiki) | MIT |
|
||||
| [`supports-color`](https://github.com/chalk/supports-color) | MIT |
|
||||
@@ -140,6 +141,7 @@ External packages **directly declared** only by repository tooling, test infrast
|
||||
| [`@types/picomatch`](https://github.com/DefinitelyTyped/DefinitelyTyped) | MIT |
|
||||
| [`@types/react`](https://github.com/DefinitelyTyped/DefinitelyTyped) | MIT |
|
||||
| [`@types/react-dom`](https://github.com/DefinitelyTyped/DefinitelyTyped) | MIT |
|
||||
| [`@types/readable-stream`](https://github.com/DefinitelyTyped/DefinitelyTyped) | MIT |
|
||||
| [`@types/spdx-expression-parse`](https://github.com/DefinitelyTyped/DefinitelyTyped) | MIT |
|
||||
| [`@types/turndown`](https://github.com/DefinitelyTyped/DefinitelyTyped) | MIT |
|
||||
| [`@types/use-sync-external-store`](https://github.com/DefinitelyTyped/DefinitelyTyped) | MIT |
|
||||
|
||||
@@ -8,7 +8,9 @@
|
||||
* Two milestones prove that happened — the host's `tree active` boot line,
|
||||
* whose lowering contract must be the one this checkout's packer emits, and the
|
||||
* workspace hero, which paints only after the client tree comes up over the
|
||||
* tunnel.
|
||||
* tunnel. The same page then creates a Workspace and Session, lists skills,
|
||||
* and writes through the settings and credentials providers, exercising the
|
||||
* upstream Chokidar instances over the Worker filesystem implementation.
|
||||
*
|
||||
* The site is served the way a static host serves it: bytes from `dist/` with
|
||||
* no rewrite rules, so a missing file is a 404 rather than the index page.
|
||||
@@ -212,6 +214,7 @@ it('boots the packed worker deployment to an interactive page', async () => {
|
||||
async function bootPreview(origin: string, browser: Browser): Promise<void> {
|
||||
const page = await newEnglishPage(browser)
|
||||
const pageErrors: Error[] = []
|
||||
const consoleErrors: string[] = []
|
||||
page.on('pageerror', (error) => { pageErrors.push(error) })
|
||||
// Registered before navigation: the worker reports its tree long before the
|
||||
// tunnel serves the client, so a listener added later would miss the line.
|
||||
@@ -219,6 +222,7 @@ async function bootPreview(origin: string, browser: Browser): Promise<void> {
|
||||
page.on('console', (message) => {
|
||||
const text = message.text()
|
||||
if (text.includes(TREE_ACTIVE)) reported(text)
|
||||
if (message.type() === 'error' || message.type() === 'warning') consoleErrors.push(text)
|
||||
})
|
||||
})
|
||||
try {
|
||||
@@ -232,7 +236,92 @@ async function bootPreview(origin: string, browser: Browser): Promise<void> {
|
||||
// surface, so it appears only once the startup chain completed over the
|
||||
// tunnel.
|
||||
await page.getByRole('textbox', { name: 'Choose workspace' }).waitFor({ timeout: HERO_TIMEOUT_MS })
|
||||
const continueButton = page.getByRole('button', { name: 'Continue' })
|
||||
if (await continueButton.isVisible()) await continueButton.click()
|
||||
await page.getByRole('button', { name: 'Configure later' }).click()
|
||||
await page.getByRole('textbox', { name: 'Choose workspace' }).click()
|
||||
const dialog = page.getByRole('dialog', { name: 'Select Workspace Directory' })
|
||||
await dialog.waitFor({ timeout: 10_000 })
|
||||
await dialog.getByRole('button', { name: 'Edit path' }).click()
|
||||
const pathInput = dialog.getByRole('textbox', { name: 'Edit path' })
|
||||
await pathInput.fill('/dsh/workspace')
|
||||
await pathInput.press('Enter')
|
||||
await dialog.getByRole('button', { name: 'Open', exact: true }).click()
|
||||
await page.locator('textarea:enabled[placeholder="Describe what you want to build"]')
|
||||
.waitFor({ timeout: 30_000 })
|
||||
|
||||
const exercised = await page.evaluate(async () => {
|
||||
type Result<T> = { result: { ok: true; value: T } | { ok: false; error: { code: string; message: string } } }
|
||||
interface PreviewApi {
|
||||
host: { createDirectory(payload: { path: string; name: string }): Promise<Result<{ path: string }>> }
|
||||
skills: { list(payload: { sessionId: string }): Promise<Result<{ skills: unknown[] }>> }
|
||||
settings: {
|
||||
describe(payload: object): Promise<Result<{ namespaces: Array<{ ns: string; revision: number }> }>>
|
||||
update(payload: { ns: string; patch: object; expectedRevision: number }): Promise<Result<unknown>>
|
||||
}
|
||||
credentials: {
|
||||
set(payload: { ref: string; value: string }): Promise<Result<unknown>>
|
||||
unset(payload: { ref: string }): Promise<Result<unknown>>
|
||||
describe(payload: { refs: string[] }): Promise<Result<{
|
||||
credentials: Record<string, { configured: boolean }>
|
||||
}>>
|
||||
}
|
||||
}
|
||||
interface PreviewTransport {
|
||||
fetch(input: string, init: RequestInit): Promise<Response>
|
||||
createApiClient(): PreviewApi
|
||||
}
|
||||
const transport = (globalThis as typeof globalThis & { __DSH_TRANSPORT__?: PreviewTransport }).__DSH_TRANSPORT__
|
||||
if (transport === undefined) throw new Error('preview transport is absent after boot')
|
||||
const response = await transport.fetch('/api/session/list', {
|
||||
method: 'POST',
|
||||
headers: { 'content-type': 'application/json' },
|
||||
body: JSON.stringify({
|
||||
type: 'client-request', rpcId: 'preview-session-list', method: 'session/list',
|
||||
payload: { args: { _request: {} } },
|
||||
}),
|
||||
})
|
||||
const sessions = await response.json() as Result<{ items: Array<{ sessionId: string }> }>
|
||||
if (!sessions.result.ok) throw new Error(`session/list failed: ${sessions.result.error.message}`)
|
||||
const sessionId = sessions.result.value.items[0]?.sessionId
|
||||
if (sessionId === undefined) throw new Error('workspace adoption created no Session')
|
||||
|
||||
const api = transport.createApiClient()
|
||||
const skills = await api.skills.list({ sessionId })
|
||||
if (!skills.result.ok) throw new Error(`skill.list failed: ${skills.result.error.message}`)
|
||||
const createDirectory = async (path: string, name: string): Promise<void> => {
|
||||
const created = await api.host.createDirectory({ path, name })
|
||||
if (!created.result.ok) throw new Error(`host.createDirectory failed: ${created.result.error.message}`)
|
||||
await new Promise((resolve) => { setTimeout(resolve, 250) })
|
||||
const refreshed = await api.skills.list({ sessionId })
|
||||
if (!refreshed.result.ok) throw new Error(`skill.list refresh failed: ${refreshed.result.error.message}`)
|
||||
}
|
||||
await createDirectory('/dsh/workspace', '.agents')
|
||||
await createDirectory('/dsh/workspace/.agents', 'skills')
|
||||
await createDirectory('/dsh/workspace/.agents/skills', 'placeholder')
|
||||
const settings = await api.settings.describe({})
|
||||
if (!settings.result.ok) throw new Error(`settings.describe failed: ${settings.result.error.message}`)
|
||||
const shell = settings.result.value.namespaces.find(namespace => namespace.ns === 'shell')
|
||||
if (shell === undefined) throw new Error('settings.describe omitted the shell namespace')
|
||||
const updated = await api.settings.update({ ns: 'shell', patch: { timeoutMs: 61_000 }, expectedRevision: shell.revision })
|
||||
if (!updated.result.ok) throw new Error(`settings.update failed: ${updated.result.error.message}`)
|
||||
const stored = await api.credentials.set({ ref: 'PREVIEW_TEST_SECRET', value: 'worker-only' })
|
||||
if (!stored.result.ok) throw new Error(`credentials.set failed: ${stored.result.error.message}`)
|
||||
const credentials = await api.credentials.describe({ refs: ['PREVIEW_TEST_SECRET'] })
|
||||
if (!credentials.result.ok) throw new Error(`credentials.describe failed: ${credentials.result.error.message}`)
|
||||
const removed = await api.credentials.unset({ ref: 'PREVIEW_TEST_SECRET' })
|
||||
if (!removed.result.ok) throw new Error(`credentials.unset failed: ${removed.result.error.message}`)
|
||||
await new Promise((resolve) => { setTimeout(resolve, 250) })
|
||||
return {
|
||||
skillCount: skills.result.value.skills.length,
|
||||
credentialConfigured: credentials.result.value.credentials.PREVIEW_TEST_SECRET?.configured,
|
||||
}
|
||||
})
|
||||
expect(exercised.skillCount).toBeGreaterThanOrEqual(0)
|
||||
expect(exercised.credentialConfigured).toBe(true)
|
||||
expect(pageErrors.map(error => error.message)).toEqual([])
|
||||
expect(consoleErrors.filter(line =>
|
||||
/watchFile|failed to watch|node-addon-landlock-run\.probe|sandbox backend is usable|SANDBOX_UNAVAILABLE/i.test(line))).toEqual([])
|
||||
} catch (error) {
|
||||
await saveFailureShot(page, 'preview-boot')
|
||||
throw pageErrors.length === 0
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/experimental/webworker-packer/README.md
|
||||
README.md: eb6c4c60106ebb7f6bb123116a749f152dea79a0
|
||||
README.zh.md: defa06c048125d627002c445aca491b4a33c4026
|
||||
README.md: a04f7579c8b3ddc7e94d2f8ed21251ed3efae302
|
||||
README.zh.md: 7876212ffb6ded5c45659502306758d8dd316f67
|
||||
|
||||
@@ -10,7 +10,7 @@ The pack is a three-layer standard stack:
|
||||
2. **Publish view** — each workspace package contributes the slice npm would publish (`files` through picomatch) minus the rule tables in `src/rules.ts` (no sources, no workspace `dist/`; external packages keep their trees minus the same exclude globs).
|
||||
3. **Reachability sweep** — the runtime loader's own resolution walks from every workspace export face plus the worker assembly's seeds (`IMAGE_ENTRY_SEEDS`), lowering each reached module to the wrapper contract at pack time. Page assets (`lib/client.js` behind `./client` exports) ship verbatim; an unresolvable request from our own code fails the pack, third-party ones are tolerated to fail loud at require time.
|
||||
|
||||
`repository.ts` owns the repo-shaped inputs (workspace scan of `vendor/`, `packages/`, `apps/`; profile composition through the real CLI dump path); `pack.ts` owns none of them, so the same library packs a different tree by being called differently. The CLI is `dsh-pack-vfs-image --out <file> [--profile web]`; `apps/web`'s `build:preview` runs it after the preview shell build.
|
||||
`repository.ts` owns the repo-shaped inputs (workspace scan of `vendor/`, `packages/`, `native/landlock-run/packages/`, and `apps/`; profile composition through the real CLI dump path); `pack.ts` owns none of them, so the same library packs a different tree by being called differently. The native scan makes the Landlock entry package an ordinary published-view dependency while its executable remains a Worker platform implementation. The CLI is `dsh-pack-vfs-image --out <file> [--profile web]`; `apps/web`'s `build:preview` runs it after the preview shell build.
|
||||
|
||||
## Model Experience
|
||||
|
||||
|
||||
@@ -10,7 +10,7 @@ VFS 镜像打包器：把一份合成 profile 变成浏览器 worker 解压后
|
||||
2. **发布视图**——每个 workspace 包贡献 npm 会发布的切片（`files` 走 picomatch），再减去 `src/rules.ts` 的规则表（无源码、无 workspace `dist/`；外部包保留整棵减同一套 exclude glob）。
|
||||
3. **可达性 sweep**——用运行时加载器自己的解析，从全部 workspace 导出面加 worker 装配种子（`IMAGE_ENTRY_SEEDS`）出发，pack 时把每个可达模块降低到包装契约。页面资产（`./client` 导出背后的 `lib/client.js`）原样直发；自家代码的不可解析请求打包即失败，第三方的容忍到 require 时 fail loud。
|
||||
|
||||
`repository.ts` 拥有仓库形态输入（`vendor/`、`packages/`、`apps/` 的 workspace 扫描；经真 CLI dump 路径合成 profile）；`pack.ts` 一概不拥有，同一库换参即可打另一棵树。CLI 为 `dsh-pack-vfs-image --out <file> [--profile web]`；`apps/web` 的 `build:preview` 在预览壳构建后运行它。
|
||||
`repository.ts` 拥有仓库形态输入（`vendor/`、`packages/`、`native/landlock-run/packages/` 与 `apps/` 的 workspace 扫描；经真 CLI dump 路径合成 profile）；`pack.ts` 一概不拥有，同一库换参即可打另一棵树。Native 扫描使 Landlock 入口包成为普通发布视图依赖，其可执行文件仍由 Worker 平台实现。CLI 为 `dsh-pack-vfs-image --out <file> [--profile web]`；`apps/web` 的 `build:preview` 在预览壳构建后运行它。
|
||||
|
||||
## 模型体验
|
||||
|
||||
|
||||
@@ -16,11 +16,11 @@ import type { ConfigTree, PackResult } from './pack.ts'
|
||||
|
||||
/**
|
||||
* Repository directories scanned for workspace and vendored packages. The
|
||||
* image only ever materializes runtime packages, which all live here;
|
||||
* examples, python, and native are never on a roster's dependency chain (the
|
||||
* native addon is a replaced external).
|
||||
* image only ever materializes runtime packages, which live here. The Landlock
|
||||
* package family contributes its unchanged JavaScript entry from `native/`;
|
||||
* examples and python never occur on a roster's dependency chain.
|
||||
*/
|
||||
const WORKSPACE_SCAN_ROOTS = ['vendor', 'packages', 'apps']
|
||||
const WORKSPACE_SCAN_ROOTS = ['vendor', 'packages', 'native/landlock-run/packages', 'apps']
|
||||
|
||||
/** Composition entry point package: the `dsh` CLI, run from source. */
|
||||
const CLI_PACKAGE = 'apps/cli'
|
||||
|
||||
@@ -21,7 +21,9 @@ import { join } from 'node:path'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { createNodeBuiltins, REPLACED_PREFIXES } from '@deepseek-ai/dsh-experimental-webworker-runtime/src/node/builtins.ts'
|
||||
import { WorkerModuleLoader } from '@deepseek-ai/dsh-experimental-webworker-runtime/src/module-system/module-loader.ts'
|
||||
import {
|
||||
setActiveModuleLoader, WorkerModuleLoader,
|
||||
} from '@deepseek-ai/dsh-experimental-webworker-runtime/src/module-system/module-loader.ts'
|
||||
import { inflateImage } from '@deepseek-ai/dsh-experimental-webworker-runtime/src/storage/image-gzip.ts'
|
||||
import { loadVfsImage } from '@deepseek-ai/dsh-experimental-webworker-runtime/src/storage/memory.ts'
|
||||
import { indexWorkspacePackages } from '../src/repository.ts'
|
||||
@@ -31,6 +33,7 @@ const repoRoot = fileURLToPath(new URL('../../../../', import.meta.url))
|
||||
|
||||
/** A leaf workspace package: real build output, no dependencies to drag in. */
|
||||
const SUBJECT = '@deepseek-ai/dsh-timeout'
|
||||
const LANDLOCK = '@deepseek-ai/node-addon-landlock-run'
|
||||
|
||||
const workspaces = indexWorkspacePackages(repoRoot)
|
||||
|
||||
@@ -53,6 +56,15 @@ const packed = (): ReturnType<typeof packVfsImage> => memo ??= packVfsImage({
|
||||
entries: [],
|
||||
})
|
||||
|
||||
let landlockMemo: ReturnType<typeof packVfsImage> | undefined
|
||||
const packedLandlock = (): ReturnType<typeof packVfsImage> => landlockMemo ??= packVfsImage({
|
||||
config: `- id: subject\n name: '${LANDLOCK}'\n`,
|
||||
profile: 'landlock-package-check',
|
||||
workspaces,
|
||||
resolveFrom: repoRoot,
|
||||
entries: [],
|
||||
})
|
||||
|
||||
/** The image's archive, inflated once: mounting reads the tar, not the gzip member. */
|
||||
let archiveMemo: Uint8Array | undefined
|
||||
const archive = async (): Promise<Uint8Array> =>
|
||||
@@ -138,6 +150,41 @@ const archive = async (): Promise<Uint8Array> =>
|
||||
expect(loader.usage().modules).toBeGreaterThan(0)
|
||||
})
|
||||
|
||||
it('runs the unchanged Landlock entry package over the Worker platform executable', async () => {
|
||||
const result = packedLandlock()
|
||||
expect(workspaces.has(LANDLOCK)).toBe(true)
|
||||
expect(result.packages.has(LANDLOCK)).toBe(true)
|
||||
expect(result.missing).toEqual([])
|
||||
expect(Object.hasOwn(result.files, `node_modules/${LANDLOCK}/lib/index.js`)).toBe(true)
|
||||
expect(createNodeBuiltins()[LANDLOCK]).toBeUndefined()
|
||||
|
||||
const vfs = loadVfsImage(await inflateImage(result.image, 'the packed Landlock package'), DEFAULT_ROOT)
|
||||
const loader = new WorkerModuleLoader({
|
||||
vfs,
|
||||
root: DEFAULT_ROOT,
|
||||
staticModules: createNodeBuiltins(),
|
||||
staticModulePrefixes: REPLACED_PREFIXES,
|
||||
})
|
||||
setActiveModuleLoader(loader)
|
||||
const landlock = loader.requireFrom(`${DEFAULT_ROOT}/workspace`)(LANDLOCK) as {
|
||||
LAUNCHER_BIN: string
|
||||
LAUNCHER_FAILURE_EXIT: number
|
||||
launcherPath(): string
|
||||
grantArgs(grants: { readOnly?: readonly string[]; readWrite?: readonly string[] }): string[]
|
||||
probe(): string
|
||||
}
|
||||
|
||||
expect(landlock.LAUNCHER_BIN).toBe('landlock-run')
|
||||
expect(landlock.LAUNCHER_FAILURE_EXIT).toBe(125)
|
||||
expect(landlock.grantArgs({ readOnly: ['/'], readWrite: ['/tmp'] })).toEqual([
|
||||
'--ro', '/', '--rw', '/tmp',
|
||||
])
|
||||
expect(landlock.launcherPath()).toBe(
|
||||
`${DEFAULT_ROOT}/node_modules/${LANDLOCK}/node_modules/${LANDLOCK}-${process.platform}-${process.arch}/bin/landlock-run`,
|
||||
)
|
||||
expect(landlock.probe()).toBe('full')
|
||||
})
|
||||
|
||||
it('refuses a body the packer did not lower, naming the image', async () => {
|
||||
// The case above only proves the packed bytes are wrappable. This is the
|
||||
// other half: the loader has no transform to fall back on, so an entry the
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write packages/experimental/webworker-runtime/README.md
|
||||
README.md: b82c65b981be6a9405ae72e3a24a42c68b52696e
|
||||
README.zh.md: 97160641a38095026d5103f2e423846bfb41d5a6
|
||||
README.md: 88d4213b5eb2f82cc41ad5059d9473d4a8abf53b
|
||||
README.zh.md: 6f4fd2612d5daa28831890ff64171ad780274958
|
||||
|
||||
@@ -7,8 +7,8 @@ The browser worker host: the whole harness plugin tree runs inside one dedicated
|
||||
Three artifacts from one tsdown pipeline:
|
||||
|
||||
- **`lib/index.js` (assembly library)** — `createWorkerHost`/`startWorkerHost` mount the image (`storage/`), install the module loader (`module-system/`) and the `process` shim, boot the tree through the image's own `dsh-app-boot`, and hand the tunnel its serving seams. The image layout contract (`image-layout.ts`: virtual root, config/manifest paths, empty directories, the `lowered` wrapper-contract gate) is shared with the packer. Boot patches force the deployment-shaped rows: frontend serving off, JSONL session logs on the plaintext path, preset roots onto the image's `config/agent-presets`.
|
||||
- **`lib/worker.js` (worker bundle)** — the assembly plus this package's Node-compatibility layer as one self-contained ES module. The module proxy table (`module-proxies.ts`) is the only platform fork: `node:*` builtins over VFS/tunnel/browser primitives, structural stubs that fail loud on the console for what a browser cannot do, and replaced externals. AsyncLocalStorage carries sync-stack causality across `await` through the snapshot/restore faces the pack-time lowering injects. The worker holds no compiler: an image the packer did not lower is refused at mount ([note](../../../.agents/notes/implemented/architecture/2026-08-20-webworker-pack-lowering-and-preview.md)).
|
||||
- **`src/shell/` (the worker's own process layer)** — a browser worker cannot fork, so `node:child_process` is not a stub but an implementation: `spawn` starts the command in its own Web Worker — this same bundle, told by its first frame to be a shell process — and reports it through the `ChildProcess` surface the subprocess service consumes. The command runs off the host's thread, `SIGKILL` terminates it whatever it is doing, and it reaches the VFS only by message (the host serves those frames). The grammar is `@yarnpkg/parsers`' `parseShell`; this package owns the evaluator (pipelines, `&&`/`||`, subshells, redirections, expansion, globs) and the command table, which is the only `/bin` that exists — a name it does not hold reports `command not found`, and `execSync`/`fork` still refuse, because they need a real process.
|
||||
- **`lib/worker.js` (worker bundle)** — the assembly plus this package's Node-compatibility layer as one self-contained ES module. The module proxy table (`module-proxies.ts`) is the only platform fork: `node:*` builtins over VFS/tunnel/browser primitives, structural stubs that fail loud on the console for what a browser cannot do, and native/binary package replacements. VFS mutations drive `node:fs` callback, polling, and promise watchers; `readable-stream` supplies the stream state machine used by file streams and unchanged image packages such as Chokidar and readdirp. AsyncLocalStorage carries sync-stack causality across `await` through the snapshot/restore faces the pack-time lowering injects. The worker holds no compiler: an image the packer did not lower is refused at mount ([note](../../../.agents/notes/implemented/architecture/2026-08-20-webworker-pack-lowering-and-preview.md)).
|
||||
- **`src/shell/` (the worker's own process layer)** — a browser worker cannot fork, so `node:child_process` is not a stub but an implementation: `spawn` starts the command in its own Web Worker — this same bundle, told by its first frame to be a shell process — and reports it through the `ChildProcess` surface the subprocess service consumes. The command runs off the host's thread, `SIGKILL` terminates it whatever it is doing, and it reaches the VFS only by message (the host serves those frames). Worker platform executables preserve native-package protocols such as Landlock without replacing their JavaScript packages or coupling their implementations to `node:child_process`; ordinary commands use the package's evaluator and coreutils command table. The grammar is `@yarnpkg/parsers`' `parseShell`, while `execSync`/`fork` still refuse because they need a real process.
|
||||
- **`lib/client.js` (page half)** — `connectWorkerHost(worker, { image? })` completes the pre-Cordis handshake: the opening `init` frame carries the image URL (the one deployment-shaped input), the boot payload delivers the structured index-injection table, and `applyIndexInjections` executes it before the shell entry runs. The tunnel exposes fetch-shaped transport, the API client, and `loadBundle` for the shell's boot seam.
|
||||
|
||||
Acceptance lives in `apps/web/tests/preview-boot.e2e.ts`, which serves the real built pages and drives the worker boot in headless Chromium.
|
||||
@@ -24,9 +24,9 @@ None; this package neither assembles nor sends a provider request.
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
- **The worker composition writes plaintext session logs** (`compression: 'none'` boot patch): it carries no Zstandard codec, so exported logs are `.jsonl`, never `.jsonl.zstd`.
|
||||
- **The skill catalog is never cached in the worker** — `skill-filesystem` watches its roots through `node:fs.watchFile`, which this package refuses, so every discovery pass returns an incomplete observation and re-scans. Discovery itself stays correct; the cost is a re-scan on every pass.
|
||||
- **`node:vm`, `node:net`, `node:sqlite`, `node:worker_threads` are structural stubs**: every call reports its refusal on the console and throws. Rows needing a real process or realm isolation cannot run here.
|
||||
- **The bash tool runs only under `danger-full-access`**: a browser has no kernel to confine a command with, so `ctx.sandbox.confine` fails loud in every other permission preset and the command never starts. The mode is the deployment's own user-facing switch, not a worker-specific composition.
|
||||
- **Filesystem watchers observe only the mounted VFS**: image seeding is silent and the VFS has no symlinks or external writers. `persistent`, `ref()`, and `unref()` preserve the Node API but cannot control a dedicated Worker's lifetime because browsers expose no ref-counted event loop.
|
||||
- **Worker confinement is a VFS boundary, not kernel Landlock**: `read-only` and `workspace-write` run the unchanged `@deepseek-ai/node-addon-landlock-run` JavaScript and launcher argv, but the process layer implements the logical `landlock-run` executable and enforces its grants on every shell filesystem request. `full` therefore covers the Worker command table and mounted VFS only; it does not claim arbitrary native-process execution or Linux kernel isolation.
|
||||
- **The worker bundle pins a path inside `@yarnpkg/parsers`** — the build resolves the package's own `lib/shell.js` instead of its root, whose barrel also re-exports the Syml parser and so drags js-yaml into a bundle that never parses that format (around 175 kB, plus its module body at worker start). The path is derived from the package manifest, so a layout change fails the build rather than reinstating the barrel; upgrading the dependency means re-checking that the shell parser still lives there.
|
||||
- **The shell is not bash**: no loops, functions, `case`, job control, or process substitution — the grammar stops at pipelines, `&&`/`||`, subshells, groups, redirections, and expansion. `&` runs its command to completion in place, `sed` accepts only substitution scripts, patterns are JavaScript regular expressions, and the command table holds coreutils only (no `git`, no network tools).
|
||||
- **A shell process has no synchronous filesystem**: it reads and writes the host's VFS by message, because blocking on a reply would need `SharedArrayBuffer`, which requires a cross-origin isolation GitHub Pages cannot grant. Directory-walking commands therefore cost one round trip per entry, and two concurrent commands can interleave their writes.
|
||||
|
||||
@@ -7,8 +7,8 @@
|
||||
一条 tsdown 管线出三个产物：
|
||||
|
||||
- **`lib/index.js`（装配库）**——`createWorkerHost`/`startWorkerHost` 挂载镜像（`storage/`）、安装模块加载器（`module-system/`）与 `process` shim、经镜像自带的 `dsh-app-boot` 启动插件树，并把服务缝隙交给隧道。镜像布局契约（`image-layout.ts`：虚拟根、config/manifest 路径、空目录、`lowered` 包装契约门）与 packer 共享。boot patch 强制部署形态行：关前端静态服务、JSONL 会话日志走明文、preset 根指向镜像内 `config/agent-presets`。
|
||||
- **`lib/worker.js`（worker 束）**——装配库加本包的 Node 兼容层，合成一个自含 ES module。模块代理表（`module-proxies.ts`）是唯一平台叉口：`node:*` 内建走 VFS/隧道/浏览器原语，浏览器做不到的走结构化 stub（调用即 console 报错并抛出），外部包整体替换。AsyncLocalStorage 经 pack 时降低注入的 snapshot/restore 面在 `await` 间携带同步栈因果。worker 不带编译器：packer 未降低的镜像在挂载时被拒（[note](../../../.agents/notes/implemented/architecture/2026-08-20-webworker-pack-lowering-and-preview.zh.md)）。
|
||||
- **`src/shell/`（worker 自己的进程层）**——浏览器 worker 无法 fork，所以 `node:child_process` 不是 stub 而是实现：`spawn` 把命令放进它自己的 Web Worker——就是这同一个束，由首帧告诉它「你是 shell 进程」——并以 subprocess 服务消费的 `ChildProcess` 面报告结果。命令不占宿主线程，`SIGKILL` 不管它在干什么都能终止它，而它只能靠消息触达 VFS（由宿主应答这些帧）。语法来自 `@yarnpkg/parsers` 的 `parseShell`；求值器（管道、`&&`/`||`、子 shell、重定向、展开、glob）与命令表由本包自持，而命令表就是这里唯一存在的 `/bin`——表里没有的名字报 `command not found`，`execSync`/`fork` 依然拒绝，因为它们需要真进程。
|
||||
- **`lib/worker.js`（worker 束）**——装配库加本包的 Node 兼容层，合成一个自含 ES module。模块代理表（`module-proxies.ts`）是唯一平台叉口：`node:*` 内建走 VFS、隧道和浏览器原语，浏览器做不到的走结构化 stub（调用即在 console 报错并抛出），native/binary 包则替换执行后端。VFS mutation 驱动 `node:fs` 的 callback、polling 和 promise watcher；`readable-stream` 提供文件流以及 Chokidar、readdirp 等未修改镜像包所用的流状态机。AsyncLocalStorage 经 pack 时降低注入的 snapshot/restore 面在 `await` 间携带同步栈因果。worker 不带编译器：packer 未降低的镜像在挂载时被拒（[note](../../../.agents/notes/implemented/architecture/2026-08-20-webworker-pack-lowering-and-preview.zh.md)）。
|
||||
- **`src/shell/`（worker 自己的进程层）**——浏览器 worker 无法 fork，所以 `node:child_process` 不是 stub 而是实现：`spawn` 把命令放进它自己的 Web Worker——就是这同一个束，由首帧告诉它「你是 shell 进程」——并以 subprocess 服务消费的 `ChildProcess` 面报告结果。命令不占宿主线程，`SIGKILL` 不管它在干什么都能终止它，而它只能靠消息触达 VFS（由宿主应答这些帧）。Worker 平台 executable 在不替换 JavaScript 包、也不把具体实现耦合进 `node:child_process` 的情况下保持 Landlock 等 native 包协议；普通命令使用本包的求值器与 coreutils 命令表。语法来自 `@yarnpkg/parsers` 的 `parseShell`，而 `execSync`/`fork` 依然拒绝，因为它们需要真进程。
|
||||
- **`lib/client.js`（页面半）**——`connectWorkerHost(worker, { image? })` 完成 pre-Cordis 握手：开局 `init` 帧携带镜像 URL（唯一部署形态输入），boot 载荷送达结构化 index 注入表，`applyIndexInjections` 在壳入口运行前逐行执行。隧道暴露 fetch 形传输、API 客户端与壳启动缝隙用的 `loadBundle`。
|
||||
|
||||
验收在 `apps/web/tests/preview-boot.e2e.ts`：静态服务真实构建页面，在 headless Chromium 里驱动 worker 启动。
|
||||
@@ -24,9 +24,9 @@
|
||||
## Known Limitations and Deferred Work
|
||||
|
||||
- **worker 组合写明文会话日志**（`compression: 'none'` boot patch）：不带 Zstandard 编解码器，导出日志是 `.jsonl`，不会是 `.jsonl.zstd`。
|
||||
- **worker 里的技能目录从不缓存**——`skill-filesystem` 用 `node:fs.watchFile` 监听各个根，而本包拒绝该调用，于是每轮发现都返回不完整观测并重新扫描。发现本身仍然正确，代价是每轮都要重扫。
|
||||
- **`node:vm`、`node:net`、`node:sqlite`、`node:worker_threads` 是结构化 stub**：每次调用在 console 报告拒绝并抛出。需要真进程或真 realm 隔离的行在此无法运行。
|
||||
- **bash 工具只在 `danger-full-access` 下可用**：浏览器没有内核可以约束命令，因此在其余权限档位下 `ctx.sandbox.confine` 会响亮失败、命令根本不会启动。该档位是部署本身的用户面开关，不是 worker 特有的组合差异。
|
||||
- **文件 watcher 只能观察已挂载的 VFS**：镜像 seed 不产生事件，VFS 也没有符号链接或外部写入方。`persistent`、`ref()` 和 `unref()` 保留 Node API，但浏览器没有引用计数事件循环，因此这些接口不能控制 dedicated Worker 的生存期。
|
||||
- **Worker confinement 是 VFS 边界，不是内核 Landlock**：`read-only` 和 `workspace-write` 运行未经修改的 `@deepseek-ai/node-addon-landlock-run` JavaScript 与 launcher argv，进程层则实现逻辑 `landlock-run` 可执行文件，并在 shell 的每次文件系统请求上执行其授权。`full` 仅覆盖 Worker 命令表和已挂载 VFS，不表示能够执行任意 native 进程，也不表示 Linux 内核隔离。
|
||||
- **worker 束钉住了 `@yarnpkg/parsers` 的包内路径**——构建解析到该包自己的 `lib/shell.js` 而非包根，因为包根 barrel 还 re-export 了 Syml 解析器，会把 js-yaml 拖进一个从不解析该格式的束（约 175 kB，外加 worker 启动时的模块体求值）。该路径由包 manifest 派生，包内布局一变即构建期失败、不会静默退回 barrel；升级这个依赖时须复核 shell 解析器是否仍在那里。
|
||||
- **这个 shell 不是 bash**：没有循环、函数、`case`、作业控制或进程替换——语法止步于管道、`&&`/`||`、子 shell、group、重定向与展开。`&` 会就地把命令跑完，`sed` 只接受替换脚本，模式是 JavaScript 正则，命令表只有 coreutils（没有 `git`，没有网络工具）。
|
||||
- **shell 进程没有同步文件面**：它靠消息读写宿主的 VFS，因为阻塞等待回帧需要 `SharedArrayBuffer`，而那要求 GitHub Pages 给不了的跨源隔离。因此目录遍历类命令每个条目一次往返，并发的两条命令写入可以交错。
|
||||
|
||||
@@ -35,7 +35,8 @@
|
||||
"@yarnpkg/parsers": "^3.1.0",
|
||||
"acorn": "^8.17.0",
|
||||
"buffer": "^6.0.3",
|
||||
"picomatch": "^4.0.4"
|
||||
"picomatch": "^4.0.4",
|
||||
"readable-stream": "^4.7.0"
|
||||
},
|
||||
"peerDependencies": {
|
||||
"@deepseek-ai/cordis": "workspace:^",
|
||||
@@ -49,12 +50,18 @@
|
||||
"@deepseek-ai/cordis": "workspace:^",
|
||||
"@deepseek-ai/cordis-plugin-loader": "workspace:^",
|
||||
"@deepseek-ai/dsh-api-gateway": "workspace:^",
|
||||
"@deepseek-ai/dsh-bash-sandbox": "workspace:^",
|
||||
"@deepseek-ai/dsh-client-modules": "workspace:^",
|
||||
"@deepseek-ai/dsh-host-apiproxy": "workspace:^",
|
||||
"@deepseek-ai/dsh-host-webserver": "workspace:^",
|
||||
"@deepseek-ai/dsh-invariants": "workspace:^",
|
||||
"@deepseek-ai/dsh-sandbox-local": "workspace:^",
|
||||
"@deepseek-ai/dsh-sandbox-policy": "workspace:^",
|
||||
"@deepseek-ai/dsh-subprocess-local": "workspace:^",
|
||||
"@types/picomatch": "^3.0.2"
|
||||
"@deepseek-ai/node-addon-landlock-run": "workspace:^",
|
||||
"@types/picomatch": "^3.0.2",
|
||||
"@types/readable-stream": "^4.0.24",
|
||||
"chokidar": "^5.0.0"
|
||||
},
|
||||
"files": [
|
||||
"lib/index.js",
|
||||
|
||||
@@ -56,7 +56,7 @@ export const MODULE_PROXIES: Record<string, string> = {
|
||||
'node:child_process': './node/builtin_modules/implemented/child_process.ts',
|
||||
// Structural mocks: every symbol exists, every call throws.
|
||||
'node:net': './node/builtin_modules/mock/net.ts',
|
||||
'node:stream': './node/builtin_modules/mock/stream.ts',
|
||||
'node:stream': './node/builtin_modules/implemented/stream.ts',
|
||||
'node:vm': './node/builtin_modules/mock/vm.ts',
|
||||
'node:worker_threads': './node/builtin_modules/mock/worker_threads.ts',
|
||||
'node:sqlite': './node/builtin_modules/mock/sqlite.ts',
|
||||
@@ -66,10 +66,8 @@ export const MODULE_PROXIES: Record<string, string> = {
|
||||
'node-pty': './node/external_packages/node-pty.ts',
|
||||
'@vscode/ripgrep': './node/external_packages/ripgrep.ts',
|
||||
'@earendil-works/pi-ai': './node/external_packages/pi-ai.ts',
|
||||
'@deepseek-ai/node-addon-landlock-run': './node/external_packages/node-addon-landlock-run.ts',
|
||||
// Constructible fakes whose methods are never reached.
|
||||
'ws': './node/external_packages/ws.ts',
|
||||
'chokidar': './node/external_packages/chokidar.ts',
|
||||
}
|
||||
|
||||
|
||||
|
||||
+80
-26
@@ -5,9 +5,10 @@
|
||||
* `spawn` starts the argv as a shell process (`src/shell/process/`) — its own
|
||||
* Web Worker, off this thread — and reports it through the `ChildProcess`
|
||||
* surface the subprocess service consumes: pipes, `exit`/`close`, pid, and
|
||||
* signals, with `SIGKILL` terminating the worker for real. The command table
|
||||
* is the only `/bin` that exists, so a name it does not hold fails with
|
||||
* `ENOENT`, exactly as a missing binary does on a real host.
|
||||
* signals, with `SIGKILL` terminating the worker for real. Worker-owned
|
||||
* executable wrappers resolve before the shell's command table; anything in
|
||||
* neither set fails with `ENOENT`, exactly as a missing binary does on a real
|
||||
* host.
|
||||
*
|
||||
* What stays impossible is what needs a real process: synchronous execution
|
||||
* (`execSync`, and `spawnSync` for a known program) and `fork`.
|
||||
@@ -19,7 +20,11 @@ import { EventEmitter } from './events.ts'
|
||||
import { notImplementedFail } from '../../notImplementedFail.ts'
|
||||
import { registerProcess, releaseProcess, signalProcess } from '../../process-table.ts'
|
||||
import { startProcess } from '../../../shell/process/host.ts'
|
||||
import { hostFileSystem } from '../../../shell/fs-access.ts'
|
||||
import { virtualExecutable } from '../../../shell/process/virtual-executables.ts'
|
||||
import type { VirtualExecutableExit } from '../../../shell/process/virtual-executables.ts'
|
||||
import { standardPrograms } from '../../../shell/programs/index.ts'
|
||||
import type { ShellFileSystem } from '../../../shell/types.ts'
|
||||
import { DSH_ROOT } from '../../../storage/paths.ts'
|
||||
|
||||
const MODULE = 'node:child_process'
|
||||
@@ -219,9 +224,6 @@ export function spawn(
|
||||
const entry = registerProcess()
|
||||
const child = new WorkerChildProcess(entry.pid, stdio)
|
||||
|
||||
const script = shellScriptOf(argv)
|
||||
const known = script !== undefined || standardPrograms().has(program)
|
||||
|
||||
const emit = (stream: 'stdout' | 'stderr', text: string): void => {
|
||||
if (text === '') return
|
||||
const pipe = stream === 'stdout' ? child.stdout : child.stderr
|
||||
@@ -236,7 +238,10 @@ export function spawn(
|
||||
}
|
||||
}
|
||||
|
||||
let settled = false
|
||||
const settle = (exitCode: number): void => {
|
||||
if (settled) return
|
||||
settled = true
|
||||
releaseProcess(entry.pid)
|
||||
// A signalled command reports no exit code, which is what makes the
|
||||
// subprocess service classify it as killed rather than finished.
|
||||
@@ -248,31 +253,68 @@ export function spawn(
|
||||
child.emit('exit', child.exitCode, signal)
|
||||
child.emit('close', child.exitCode, signal)
|
||||
}
|
||||
const failSpawn = (error: Error): void => {
|
||||
if (settled) return
|
||||
settled = true
|
||||
releaseProcess(entry.pid)
|
||||
child.emit('error', error)
|
||||
}
|
||||
|
||||
// The command starts on a microtask, so a caller that attaches listeners and
|
||||
// writes standard input right after `spawn()` — the subprocess service does
|
||||
// exactly that — is never racing the first output.
|
||||
queueMicrotask(() => {
|
||||
if (!known) {
|
||||
releaseProcess(entry.pid)
|
||||
child.emit('error', spawnEnoent(program))
|
||||
return
|
||||
}
|
||||
entry.process = startProcess({
|
||||
script,
|
||||
argv,
|
||||
cwd: options.cwd ?? DSH_ROOT,
|
||||
env: environmentOf(options.env),
|
||||
stdin: child.stdin?.contents() ?? '',
|
||||
onOutput: emit,
|
||||
onExit: settle,
|
||||
void (async () => {
|
||||
const cwd = options.cwd ?? DSH_ROOT
|
||||
let commandArgv: readonly string[] = argv
|
||||
let filesystem: ShellFileSystem | undefined
|
||||
let missingExecutable: VirtualExecutableExit | undefined
|
||||
const executable = virtualExecutable(program)
|
||||
if (executable !== undefined) {
|
||||
const prepared = await executable.prepare(args, { cwd, filesystem: hostFileSystem() })
|
||||
if (prepared.kind === 'exit') {
|
||||
emit('stdout', prepared.stdout)
|
||||
emit('stderr', prepared.stderr)
|
||||
settle(prepared.exitCode)
|
||||
return
|
||||
}
|
||||
commandArgv = prepared.argv
|
||||
filesystem = prepared.filesystem
|
||||
missingExecutable = prepared.missingExecutable
|
||||
}
|
||||
|
||||
const command = commandArgv[0] as string
|
||||
const script = shellScriptOf(commandArgv)
|
||||
const known = script !== undefined || standardPrograms().has(command)
|
||||
if (!known) {
|
||||
if (missingExecutable !== undefined) {
|
||||
emit('stdout', missingExecutable.stdout)
|
||||
emit('stderr', missingExecutable.stderr)
|
||||
settle(missingExecutable.exitCode)
|
||||
} else {
|
||||
failSpawn(spawnEnoent(program))
|
||||
}
|
||||
return
|
||||
}
|
||||
entry.process = startProcess({
|
||||
script,
|
||||
argv: commandArgv,
|
||||
cwd,
|
||||
env: environmentOf(options.env),
|
||||
stdin: child.stdin?.contents() ?? '',
|
||||
onOutput: emit,
|
||||
onExit: settle,
|
||||
...filesystem === undefined ? {} : { fs: filesystem },
|
||||
})
|
||||
// A signal that arrived while the process was still starting has to reach
|
||||
// it now; the table recorded it but had nothing to deliver it to.
|
||||
if (entry.signal !== undefined) {
|
||||
if (entry.signal === 'SIGKILL') entry.process.destroy()
|
||||
else entry.process.interrupt()
|
||||
}
|
||||
})().catch((error: unknown) => {
|
||||
failSpawn(error instanceof Error ? error : new Error(String(error)))
|
||||
})
|
||||
// A signal that arrived while the process was still starting has to reach
|
||||
// it now; the table recorded it but had nothing to deliver it to.
|
||||
if (entry.signal !== undefined) {
|
||||
if (entry.signal === 'SIGKILL') entry.process.destroy()
|
||||
else entry.process.interrupt()
|
||||
}
|
||||
})
|
||||
|
||||
return child
|
||||
@@ -298,10 +340,22 @@ export interface WorkerSpawnSyncResult {
|
||||
* answers in the same shape: absent programs report `ENOENT`, and a program
|
||||
* this shell *does* have reports that only the asynchronous path can run it.
|
||||
* @param program - the program name.
|
||||
* @param args - arguments passed to the virtual launcher probe.
|
||||
* @returns the Node-shaped synchronous result carrying the failure.
|
||||
*/
|
||||
export function spawnSync(program: string): WorkerSpawnSyncResult {
|
||||
export function spawnSync(program: string, args: readonly string[] = []): WorkerSpawnSyncResult {
|
||||
const empty = Buffer.alloc(0)
|
||||
const executable = virtualExecutable(program)
|
||||
if (executable !== undefined) {
|
||||
const result = executable.runSync(args)
|
||||
if (result.kind === 'asynchronous') {
|
||||
const error = new Error(`${MODULE}.spawnSync cannot run ${program} in the worker host: commands run asynchronously`)
|
||||
return { pid: -1, status: null, signal: null, stdout: empty, stderr: empty, output: [null, empty, empty], error }
|
||||
}
|
||||
const stdout = Buffer.from(result.stdout)
|
||||
const stderr = Buffer.from(result.stderr)
|
||||
return { pid: -1, status: result.exitCode, signal: null, stdout, stderr, output: [null, stdout, stderr] }
|
||||
}
|
||||
const error = standardPrograms().has(program)
|
||||
? new Error(`${MODULE}.spawnSync cannot run ${program} in the worker host: commands run asynchronously`)
|
||||
: spawnEnoent(program)
|
||||
|
||||
+419
@@ -0,0 +1,419 @@
|
||||
/** Node filesystem watching over the active in-memory VFS. */
|
||||
import { Buffer } from 'buffer'
|
||||
import { EventEmitter } from './events.ts'
|
||||
import { captureAsyncContext, runWithAsyncContext } from './async_hooks.ts'
|
||||
import { basename, relative, resolve, sep } from './path.ts'
|
||||
import { requireActiveVfs } from '../../../storage/active.ts'
|
||||
import type { VfsBigIntStats, VfsMutation, VfsStats } from '../../../storage/types.ts'
|
||||
|
||||
type PathArg = string | URL | Uint8Array
|
||||
type WatchListener = (eventType: 'rename' | 'change', filename: string | Buffer | null) => void
|
||||
type WatchStats = VfsStats | VfsBigIntStats
|
||||
type StatListener = (current: WatchStats, previous: WatchStats) => void
|
||||
|
||||
/** Options shared by the callback and promise watch faces. */
|
||||
export interface WatchOptions {
|
||||
persistent?: boolean
|
||||
recursive?: boolean
|
||||
encoding?: BufferEncoding | 'buffer'
|
||||
signal?: AbortSignal
|
||||
}
|
||||
/** Poll-style watch options. */
|
||||
export interface WatchFileOptions {
|
||||
persistent?: boolean
|
||||
interval?: number
|
||||
bigint?: boolean
|
||||
}
|
||||
|
||||
const asPath = (path: PathArg): string => {
|
||||
if (typeof path === 'string') return resolve(path)
|
||||
if (path instanceof URL) return resolve(decodeURIComponent(path.pathname))
|
||||
return resolve(new TextDecoder().decode(path))
|
||||
}
|
||||
|
||||
const missingStats = (bigint: boolean): WatchStats => ({
|
||||
size: bigint ? 0n : 0,
|
||||
ino: bigint ? 0n : 0,
|
||||
mtimeMs: bigint ? 0n : 0,
|
||||
ctimeMs: bigint ? 0n : 0,
|
||||
atimeMs: bigint ? 0n : 0,
|
||||
birthtimeMs: bigint ? 0n : 0,
|
||||
mtime: new Date(0),
|
||||
mode: bigint ? 0n : 0,
|
||||
...bigint ? {
|
||||
dev: 0n,
|
||||
nlink: 0n,
|
||||
mtimeNs: 0n,
|
||||
ctimeNs: 0n,
|
||||
atimeNs: 0n,
|
||||
birthtimeNs: 0n,
|
||||
ctime: new Date(0),
|
||||
atime: new Date(0),
|
||||
birthtime: new Date(0),
|
||||
} : {},
|
||||
isFile: () => false,
|
||||
isDirectory: () => false,
|
||||
isSymbolicLink: () => false,
|
||||
isFIFO: () => false,
|
||||
isSocket: () => false,
|
||||
isBlockDevice: () => false,
|
||||
isCharacterDevice: () => false,
|
||||
}) as WatchStats
|
||||
|
||||
const statOrMissing = (path: string, bigint: boolean): WatchStats => {
|
||||
try {
|
||||
return requireActiveVfs().statSync(path, { bigint })
|
||||
} catch (error) {
|
||||
if ((error as NodeJS.ErrnoException).code === 'ENOENT') return missingStats(bigint)
|
||||
throw error
|
||||
}
|
||||
}
|
||||
|
||||
const statsChanged = (left: WatchStats, right: WatchStats): boolean =>
|
||||
left.size !== right.size
|
||||
|| left.mtimeMs !== right.mtimeMs
|
||||
|| left.mode !== right.mode
|
||||
|| left.ino !== right.ino
|
||||
|| left.isFile() !== right.isFile()
|
||||
|| left.isDirectory() !== right.isDirectory()
|
||||
|
||||
const contains = (parent: string, child: string): boolean =>
|
||||
parent === '/' || child === parent || child.startsWith(`${parent}${sep}`)
|
||||
|
||||
const overlaps = (left: string, right: string): boolean => contains(left, right) || contains(right, left)
|
||||
|
||||
const abortError = (reason?: unknown): Error & { code: string; cause?: unknown } => {
|
||||
const error = new Error('The operation was aborted') as Error & { code: string }
|
||||
error.name = 'AbortError'
|
||||
error.code = 'ABORT_ERR'
|
||||
if (reason !== undefined) error.cause = reason
|
||||
return error
|
||||
}
|
||||
|
||||
/** `fs.FSWatcher` over VFS mutations. */
|
||||
export class FSWatcher extends EventEmitter {
|
||||
private readonly disposeMutation: () => void
|
||||
private readonly signal: AbortSignal | undefined
|
||||
private readonly onAbort: (() => void) | undefined
|
||||
private closed = false
|
||||
private referenced: boolean
|
||||
|
||||
constructor(
|
||||
private readonly target: string,
|
||||
private readonly directory: boolean,
|
||||
private readonly options: WatchOptions,
|
||||
listener?: WatchListener,
|
||||
) {
|
||||
super()
|
||||
this.referenced = options.persistent ?? true
|
||||
const context = captureAsyncContext()
|
||||
if (listener !== undefined) this.on('change', listener as (...args: unknown[]) => void)
|
||||
this.disposeMutation = requireActiveVfs().subscribe((mutation) => {
|
||||
if (!this.matches(mutation)) return
|
||||
const eventType = mutation.kind === 'write' && !mutation.entryChanged || mutation.kind === 'chmod'
|
||||
? 'change'
|
||||
: 'rename'
|
||||
const filename = this.filename(mutation.path)
|
||||
queueMicrotask(() => {
|
||||
if (this.closed) return
|
||||
runWithAsyncContext(context, () => { this.emit('change', eventType, filename) })
|
||||
})
|
||||
})
|
||||
this.signal = options.signal
|
||||
this.onAbort = options.signal === undefined ? undefined : () => { this.close() }
|
||||
if (options.signal?.aborted === true) {
|
||||
this.disposeMutation()
|
||||
this.closed = true
|
||||
throw abortError(options.signal.reason)
|
||||
}
|
||||
options.signal?.addEventListener('abort', this.onAbort as () => void, { once: true })
|
||||
}
|
||||
|
||||
private matches(mutation: VfsMutation): boolean {
|
||||
if (mutation.path === this.target) return true
|
||||
if (mutation.kind === 'remove' && contains(mutation.path, this.target)) return true
|
||||
if (!this.directory || !contains(this.target, mutation.path)) return false
|
||||
if (this.options.recursive === true) return true
|
||||
const child = relative(this.target, mutation.path)
|
||||
return child !== '' && !child.startsWith('..') && !child.includes(sep)
|
||||
}
|
||||
|
||||
private filename(path: string): string | Buffer {
|
||||
const relativePath = relative(this.target, path)
|
||||
const value = this.directory && contains(this.target, path)
|
||||
? this.options.recursive === true ? relativePath : relativePath.split(sep)[0] ?? ''
|
||||
: basename(this.target)
|
||||
return this.options.encoding === 'buffer' ? Buffer.from(value) : value
|
||||
}
|
||||
|
||||
/** Stop observing and publish `close` once. */
|
||||
close(): void {
|
||||
if (this.closed) return
|
||||
this.closed = true
|
||||
this.disposeMutation()
|
||||
if (this.onAbort !== undefined) this.signal?.removeEventListener('abort', this.onAbort)
|
||||
queueMicrotask(() => { this.emit('close') })
|
||||
}
|
||||
|
||||
/**
|
||||
* Mark this watcher as process-liveness-bearing.
|
||||
* @returns This watcher.
|
||||
*/
|
||||
ref(): this {
|
||||
this.referenced = true
|
||||
return this
|
||||
}
|
||||
|
||||
/**
|
||||
* Clear the process-liveness flag; dedicated Workers have no ref-counted event loop.
|
||||
* @returns This watcher.
|
||||
*/
|
||||
unref(): this {
|
||||
this.referenced = false
|
||||
return this
|
||||
}
|
||||
|
||||
/**
|
||||
* Read the retained process-liveness flag.
|
||||
* @returns Whether this watcher is marked as keeping its owner alive.
|
||||
*/
|
||||
hasRef(): boolean {
|
||||
return this.referenced
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Watch one path through the active VFS.
|
||||
* @param path - File or directory path.
|
||||
* @param optionsOrListener - Watch options, encoding, or the change listener.
|
||||
* @param maybeListener - Change listener when the second argument carries options.
|
||||
* @returns The closeable watcher.
|
||||
*/
|
||||
export function watch(
|
||||
path: PathArg,
|
||||
optionsOrListener?: WatchOptions | BufferEncoding | 'buffer' | WatchListener,
|
||||
maybeListener?: WatchListener,
|
||||
): FSWatcher {
|
||||
const options: WatchOptions = typeof optionsOrListener === 'object'
|
||||
? optionsOrListener
|
||||
: typeof optionsOrListener === 'string' ? { encoding: optionsOrListener } : {}
|
||||
const listener = typeof optionsOrListener === 'function' ? optionsOrListener : maybeListener
|
||||
const target = asPath(path)
|
||||
const stats = requireActiveVfs().statSync(target)
|
||||
return new FSWatcher(target, stats.isDirectory(), options, listener)
|
||||
}
|
||||
|
||||
/** `fs.StatWatcher` returned from `watchFile`. */
|
||||
export class StatWatcher extends EventEmitter {
|
||||
private readonly disposeMutation: () => void
|
||||
private timer: ReturnType<typeof setTimeout> | undefined
|
||||
private previous: WatchStats
|
||||
private stopped = false
|
||||
private referenced: boolean
|
||||
private readonly context: ReturnType<typeof captureAsyncContext>
|
||||
private readonly interval: number
|
||||
private readonly bigint: boolean
|
||||
|
||||
constructor(readonly path: string, options: WatchFileOptions) {
|
||||
super()
|
||||
this.referenced = options.persistent ?? true
|
||||
this.interval = options.interval ?? 5007
|
||||
this.bigint = options.bigint ?? false
|
||||
this.previous = statOrMissing(path, this.bigint)
|
||||
this.context = captureAsyncContext()
|
||||
this.disposeMutation = requireActiveVfs().subscribe((mutation) => {
|
||||
if (overlaps(path, mutation.path)) this.schedule()
|
||||
})
|
||||
if (!this.previous.isFile() && !this.previous.isDirectory()) this.schedule(true)
|
||||
}
|
||||
|
||||
private schedule(initialMissing = false): void {
|
||||
if (this.stopped || this.timer !== undefined) return
|
||||
this.timer = setTimeout(() => {
|
||||
this.timer = undefined
|
||||
if (this.stopped) return
|
||||
const current = statOrMissing(this.path, this.bigint)
|
||||
const previous = this.previous
|
||||
this.previous = current
|
||||
if (initialMissing || statsChanged(current, previous)) {
|
||||
runWithAsyncContext(this.context, () => { this.emit('change', current, previous) })
|
||||
}
|
||||
}, this.interval)
|
||||
if (!this.referenced) timerUnref(this.timer)
|
||||
}
|
||||
|
||||
/** Stop polling and release the VFS subscription. */
|
||||
stop(): void {
|
||||
if (this.stopped) return
|
||||
this.stopped = true
|
||||
this.disposeMutation()
|
||||
if (this.timer !== undefined) clearTimeout(this.timer)
|
||||
this.timer = undefined
|
||||
this.emit('stop')
|
||||
}
|
||||
|
||||
/** Alias used by callers treating the watcher as a closeable handle. */
|
||||
close(): void {
|
||||
this.stop()
|
||||
}
|
||||
|
||||
/**
|
||||
* Mark this watcher as process-liveness-bearing.
|
||||
* @returns This watcher.
|
||||
*/
|
||||
ref(): this {
|
||||
this.referenced = true
|
||||
if (this.timer !== undefined) timerRef(this.timer)
|
||||
return this
|
||||
}
|
||||
|
||||
/**
|
||||
* Mark this watcher as not keeping its owner alive.
|
||||
* @returns This watcher.
|
||||
*/
|
||||
unref(): this {
|
||||
this.referenced = false
|
||||
if (this.timer !== undefined) timerUnref(this.timer)
|
||||
return this
|
||||
}
|
||||
|
||||
/**
|
||||
* Read the retained process-liveness flag.
|
||||
* @returns Whether this watcher is marked as keeping its owner alive.
|
||||
*/
|
||||
hasRef(): boolean {
|
||||
return this.referenced
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
type RefTimer = { ref?: () => unknown; unref?: () => unknown }
|
||||
|
||||
/** Browser timers are numeric; Node timers expose optional liveness methods. */
|
||||
const timerRef = (timer: ReturnType<typeof setTimeout>): void => {
|
||||
;(timer as unknown as RefTimer).ref?.()
|
||||
}
|
||||
|
||||
/** Browser timers are numeric; Node timers expose optional liveness methods. */
|
||||
const timerUnref = (timer: ReturnType<typeof setTimeout>): void => {
|
||||
;(timer as unknown as RefTimer).unref?.()
|
||||
}
|
||||
|
||||
const statWatchers = new Map<string, StatWatcher>()
|
||||
|
||||
/**
|
||||
* Register a stat-poll watcher for one path.
|
||||
* @param path - File or directory path, including a currently missing path.
|
||||
* @param optionsOrListener - Polling options or the change listener.
|
||||
* @param maybeListener - Change listener when the second argument carries options.
|
||||
* @returns The path's shared stat watcher.
|
||||
*/
|
||||
export function watchFile(
|
||||
path: PathArg,
|
||||
optionsOrListener: WatchFileOptions | StatListener,
|
||||
maybeListener?: StatListener,
|
||||
): StatWatcher {
|
||||
const options = typeof optionsOrListener === 'function' ? {} : optionsOrListener
|
||||
const listener = typeof optionsOrListener === 'function' ? optionsOrListener : maybeListener
|
||||
if (listener === undefined) throw new TypeError('The "listener" argument must be of type function')
|
||||
const target = asPath(path)
|
||||
let watcher = statWatchers.get(target)
|
||||
if (watcher === undefined) {
|
||||
watcher = new StatWatcher(target, options)
|
||||
statWatchers.set(target, watcher)
|
||||
watcher.once('stop', () => { statWatchers.delete(target) })
|
||||
}
|
||||
watcher.on('change', listener as (...args: unknown[]) => void)
|
||||
return watcher
|
||||
}
|
||||
|
||||
/**
|
||||
* Remove one listener or every listener for a path.
|
||||
* @param path - Watched path.
|
||||
* @param listener - Specific registration to remove; omission removes all registrations.
|
||||
*/
|
||||
export function unwatchFile(path: PathArg, listener?: StatListener): void {
|
||||
const target = asPath(path)
|
||||
const watcher = statWatchers.get(target)
|
||||
if (watcher === undefined) return
|
||||
if (listener === undefined) watcher.removeAllListeners('change')
|
||||
else watcher.removeListener('change', listener as (...args: unknown[]) => void)
|
||||
if (watcher.listenerCount('change') === 0) watcher.stop()
|
||||
}
|
||||
|
||||
/**
|
||||
* Create the promise-based watch iterator over the callback watcher.
|
||||
* @param path - File or directory path.
|
||||
* @param options - Watch options and cancellation signal.
|
||||
* @returns An iterator of change records that closes its watcher on return or failure.
|
||||
*/
|
||||
export function watchAsync(
|
||||
path: PathArg,
|
||||
options: WatchOptions = {},
|
||||
): AsyncIterableIterator<{ eventType: 'rename' | 'change'; filename: string | Buffer | null }> {
|
||||
type WatchEvent = { eventType: 'rename' | 'change'; filename: string | Buffer | null }
|
||||
type Waiting = {
|
||||
resolve(result: IteratorResult<WatchEvent>): void
|
||||
reject(reason: unknown): void
|
||||
}
|
||||
const queued: WatchEvent[] = []
|
||||
const waiting: Waiting[] = []
|
||||
let watcher: FSWatcher | undefined
|
||||
let failure: Error | undefined
|
||||
let closed = false
|
||||
|
||||
const settleFailure = (reason: unknown): void => {
|
||||
if (failure !== undefined || closed) return
|
||||
const error = reason instanceof Error ? reason : new Error(String(reason))
|
||||
failure = error
|
||||
watcher?.close()
|
||||
for (const pending of waiting.splice(0)) pending.reject(error)
|
||||
}
|
||||
const onAbort = (): void => { settleFailure(abortError(options.signal?.reason)) }
|
||||
const start = (): void => {
|
||||
if (watcher !== undefined || closed || failure !== undefined) return
|
||||
try {
|
||||
watcher = watch(path, options, (eventType, filename) => {
|
||||
const event = { eventType, filename }
|
||||
const pending = waiting.shift()
|
||||
if (pending === undefined) queued.push(event)
|
||||
else pending.resolve({ done: false, value: event })
|
||||
})
|
||||
watcher.on('error', settleFailure)
|
||||
options.signal?.addEventListener('abort', onAbort, { once: true })
|
||||
} catch (error) {
|
||||
settleFailure(error)
|
||||
}
|
||||
}
|
||||
const close = (): void => {
|
||||
if (closed) return
|
||||
closed = true
|
||||
queued.length = 0
|
||||
options.signal?.removeEventListener('abort', onAbort)
|
||||
watcher?.close()
|
||||
for (const pending of waiting.splice(0)) pending.resolve({ done: true, value: undefined })
|
||||
}
|
||||
|
||||
return {
|
||||
[Symbol.asyncIterator]() {
|
||||
return this
|
||||
},
|
||||
next(): Promise<IteratorResult<WatchEvent>> {
|
||||
start()
|
||||
if (failure !== undefined) return Promise.reject(failure)
|
||||
const event = queued.shift()
|
||||
if (event !== undefined) return Promise.resolve({ done: false, value: event })
|
||||
if (closed) return Promise.resolve({ done: true, value: undefined })
|
||||
return new Promise<IteratorResult<WatchEvent>>((resolve, reject) => { waiting.push({ resolve, reject }) })
|
||||
},
|
||||
return(): Promise<IteratorResult<WatchEvent>> {
|
||||
close()
|
||||
return Promise.resolve({ done: true, value: undefined })
|
||||
},
|
||||
throw(reason?: unknown): Promise<IteratorResult<WatchEvent>> {
|
||||
close()
|
||||
// AsyncIterator.throw forwards the caller's exact reason, including non-Error values.
|
||||
return Promise.reject(reason)
|
||||
},
|
||||
}
|
||||
}
|
||||
+292
-53
@@ -2,19 +2,20 @@
|
||||
* `node:fs` bridge over the worker's in-memory VFS. `MemoryVfs` owns paths,
|
||||
* bytes, the directory tree, and Node's error codes; this module adds only what
|
||||
* is Node-API-shaped and not VFS business: Buffer results, `Dirent` objects,
|
||||
* file descriptors, `mkdtemp`, access checks, inert watches, and the promise face.
|
||||
* file descriptors, `mkdtemp`, access checks, watchers, streams, and the promise face.
|
||||
*/
|
||||
import { requireActiveVfs } from '../../../storage/active.ts'
|
||||
import type { MemoryVfs } from '../../../storage/memory.ts'
|
||||
import type { VfsBigIntStats, VfsStatOptions, VfsStats, VfsWriteOptions } from '../../../storage/types.ts'
|
||||
import type { Vfs, VfsBigIntStats, VfsStatOptions, VfsStats, VfsWriteOptions } from '../../../storage/types.ts'
|
||||
import { Buffer } from 'buffer'
|
||||
import { Readable, Writable } from './stream.ts'
|
||||
import { dirname } from './path.ts'
|
||||
import {
|
||||
FSWatcher, StatWatcher, unwatchFile, watch, watchAsync, watchFile,
|
||||
} from './fs-watch.ts'
|
||||
|
||||
const vfs = (): MemoryVfs => requireActiveVfs()
|
||||
const vfs = (): Vfs => requireActiveVfs()
|
||||
|
||||
const notImplemented = (method: string, subject: string): never => {
|
||||
throw new Error(`web-preview: node:fs.${method} is not implemented in the worker host (${subject})`)
|
||||
}
|
||||
export { FSWatcher, StatWatcher, unwatchFile, watch, watchFile }
|
||||
|
||||
type PathArg = string | URL | Uint8Array
|
||||
|
||||
@@ -150,6 +151,29 @@ export function statSync(path: PathArg, options?: VfsStatOptions): VfsStats | Vf
|
||||
return vfs().statSync(asPath(path), options)
|
||||
}
|
||||
|
||||
/**
|
||||
* Read stats through Node's callback form.
|
||||
* @param path - Path to stat.
|
||||
* @param optionsOrCallback - Stat options or the completion callback.
|
||||
* @param maybeCallback - Completion callback when options are present.
|
||||
*/
|
||||
export function stat(
|
||||
path: PathArg,
|
||||
optionsOrCallback: VfsStatOptions | ((error: NodeJS.ErrnoException | null, stats?: VfsStats | VfsBigIntStats) => void),
|
||||
maybeCallback?: (error: NodeJS.ErrnoException | null, stats?: VfsStats | VfsBigIntStats) => void,
|
||||
): void {
|
||||
const options = typeof optionsOrCallback === 'function' ? undefined : optionsOrCallback
|
||||
const callback = typeof optionsOrCallback === 'function' ? optionsOrCallback : maybeCallback
|
||||
if (callback === undefined) throw new TypeError('The "callback" argument must be of type function')
|
||||
queueMicrotask(() => {
|
||||
try {
|
||||
callback(null, statSync(path, options))
|
||||
} catch (error) {
|
||||
callback(error as NodeJS.ErrnoException)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
/**
|
||||
* Change an entry's permission bits; stat reads back exactly what was set.
|
||||
* @param path - the path.
|
||||
@@ -169,6 +193,20 @@ export function lstatSync(path: PathArg, options?: VfsStatOptions): VfsStats | V
|
||||
return statSync(path, options)
|
||||
}
|
||||
|
||||
/**
|
||||
* Read link stats through Node's callback form; this symlink-free VFS delegates to stat.
|
||||
* @param path - Path to stat.
|
||||
* @param optionsOrCallback - Stat options or the completion callback.
|
||||
* @param maybeCallback - Completion callback when options are present.
|
||||
*/
|
||||
export function lstat(
|
||||
path: PathArg,
|
||||
optionsOrCallback: VfsStatOptions | ((error: NodeJS.ErrnoException | null, stats?: VfsStats | VfsBigIntStats) => void),
|
||||
maybeCallback?: (error: NodeJS.ErrnoException | null, stats?: VfsStats | VfsBigIntStats) => void,
|
||||
): void {
|
||||
stat(path, optionsOrCallback, maybeCallback)
|
||||
}
|
||||
|
||||
/**
|
||||
* Canonical path (normalization only: the image is symlink-free).
|
||||
* @param path - the path.
|
||||
@@ -265,9 +303,10 @@ let nextFd = 3
|
||||
* @param path - file path.
|
||||
* @param flags - Node flag string: 'r', 'w', 'a', with optional '+' and the
|
||||
* exclusive 'x' (create-only) modifier.
|
||||
* @param mode - creation permission bits.
|
||||
* @returns the descriptor.
|
||||
*/
|
||||
export function openSync(path: PathArg, flags = 'r'): number {
|
||||
export function openSync(path: PathArg, flags = 'r', mode?: number): number {
|
||||
const target = asPath(path)
|
||||
const exists = vfs().existsSync(target)
|
||||
if (flags.includes('x') && exists) {
|
||||
@@ -277,7 +316,9 @@ export function openSync(path: PathArg, flags = 'r'): number {
|
||||
throw error
|
||||
}
|
||||
if (flags.startsWith('r')) vfs().realpathSync(target)
|
||||
else if (flags.startsWith('w') || !exists) vfs().writeFileSync(target, new Uint8Array(0))
|
||||
else if (flags.startsWith('w') || !exists) {
|
||||
vfs().writeFileSync(target, new Uint8Array(0), mode === undefined ? undefined : { mode })
|
||||
}
|
||||
const fd = nextFd++
|
||||
openFiles.set(fd, { path: target, position: 0, append: flags.startsWith('a') })
|
||||
return fd
|
||||
@@ -356,8 +397,8 @@ export function linkSync(from: PathArg, to: PathArg): void {
|
||||
|
||||
/**
|
||||
* Open file handle (`fs.FileHandle` subset): the atomic-write and durability
|
||||
* pair the storage backends use. `sync`/`datasync` are no-ops — an in-memory
|
||||
* filesystem has nothing to flush, and a worker reload loses it either way.
|
||||
* pair the storage backends use. `sync`/`datasync` settle the active VFS's
|
||||
* optional write-behind sink.
|
||||
*/
|
||||
export interface FileHandle {
|
||||
readonly fd: number
|
||||
@@ -377,13 +418,14 @@ export interface FileHandle {
|
||||
* helpers do before an fsync.
|
||||
* @param path - file or directory path.
|
||||
* @param flags - Node flag string.
|
||||
* @param mode - creation permission bits.
|
||||
* @returns the handle.
|
||||
*/
|
||||
export function openHandleSync(path: PathArg, flags = 'r'): FileHandle {
|
||||
export function openHandleSync(path: PathArg, flags = 'r', mode?: number): FileHandle {
|
||||
const target = asPath(path)
|
||||
const directory = vfs().existsSync(target) && vfs().statSync(target).isDirectory()
|
||||
const append = flags.startsWith('a')
|
||||
const fd = directory ? -1 : openSync(target, flags)
|
||||
const fd = directory ? -1 : openSync(target, flags, mode)
|
||||
return {
|
||||
fd,
|
||||
readFile: async (options?: EncodingOption) => readFileSync(target, options),
|
||||
@@ -403,56 +445,251 @@ export function openHandleSync(path: PathArg, flags = 'r'): FileHandle {
|
||||
truncate: async (length = 0) => {
|
||||
writeFileSync(target, bytesOf(target).subarray(0, length))
|
||||
},
|
||||
sync: async () => { /* memory-backed: nothing to flush */ },
|
||||
datasync: async () => { /* memory-backed: nothing to flush */ },
|
||||
sync: async () => { await vfs().flush() },
|
||||
datasync: async () => { await vfs().flush() },
|
||||
close: async () => {
|
||||
if (fd !== -1) closeSync(fd)
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Watch registration refuses loudly, and NOT because watching is hard.
|
||||
*
|
||||
* An inert watcher would not serve this caller. `skill-filesystem` does not
|
||||
* merely register a listener — `openStableWatcher` opens a watcher and then
|
||||
* loops until two consecutive mode probes agree, so a watcher that reports
|
||||
* success and never fires leaves `observeRoots()` awaiting forever: the skill
|
||||
* catalog RPC never answers and the worker's single thread stops serving `/api`
|
||||
* for the rest of the session. A refusal instead fails that path fast, which the
|
||||
* provider already handles by returning an incomplete observation.
|
||||
*
|
||||
* So the family split is about what the CALLER does with the capability, not
|
||||
* about the capability: a listener registration tolerates absence, a watcher
|
||||
* whose progress is awaited does not.
|
||||
* @param path - the path a caller wanted watched, named in the refusal.
|
||||
* @returns Never — it throws naming the unavailable member.
|
||||
*/
|
||||
export function watchFile(path: PathArg): never {
|
||||
return notImplemented('watchFile', asPath(path))
|
||||
/** Options supported by the VFS-backed read stream. */
|
||||
export interface ReadStreamOptions {
|
||||
flags?: string
|
||||
encoding?: BufferEncoding | null
|
||||
autoClose?: boolean
|
||||
emitClose?: boolean
|
||||
start?: number
|
||||
end?: number
|
||||
highWaterMark?: number
|
||||
signal?: AbortSignal
|
||||
}
|
||||
|
||||
/** Watch removal; teardown paths call it unconditionally, and nothing was watched. */
|
||||
export function unwatchFile(): void {
|
||||
// No watch was ever established.
|
||||
/** Options supported by the VFS-backed write stream. */
|
||||
export interface WriteStreamOptions {
|
||||
flags?: string
|
||||
encoding?: BufferEncoding | null
|
||||
mode?: number
|
||||
autoClose?: boolean
|
||||
emitClose?: boolean
|
||||
start?: number
|
||||
highWaterMark?: number
|
||||
signal?: AbortSignal
|
||||
}
|
||||
|
||||
const aborted = (reason?: unknown): Error => {
|
||||
const error = new Error('The operation was aborted', { cause: reason }) as Error & { code: string }
|
||||
error.name = 'AbortError'
|
||||
error.code = 'ABORT_ERR'
|
||||
return error
|
||||
}
|
||||
|
||||
/** Read stream over one VFS file. */
|
||||
export class ReadStream extends Readable {
|
||||
/** Resolved path opened by this stream. */
|
||||
readonly path: string
|
||||
/** Open descriptor, or null before open and after close. */
|
||||
fd: number | null = null
|
||||
/** Whether the descriptor is still waiting to open. */
|
||||
pending = true
|
||||
/** Bytes delivered by this stream. */
|
||||
bytesRead = 0
|
||||
private readonly start: number
|
||||
private readonly end: number
|
||||
private readonly flags: string
|
||||
private readonly signal: AbortSignal | undefined
|
||||
private readonly onAbort: (() => void) | undefined
|
||||
private position: number
|
||||
|
||||
constructor(path: PathArg, options: ReadStreamOptions = {}) {
|
||||
super({
|
||||
autoDestroy: options.autoClose ?? true,
|
||||
emitClose: options.emitClose ?? true,
|
||||
highWaterMark: options.highWaterMark ?? 64 * 1024,
|
||||
})
|
||||
this.path = asPath(path)
|
||||
this.start = options.start ?? 0
|
||||
this.end = options.end ?? Number.POSITIVE_INFINITY
|
||||
this.flags = options.flags ?? 'r'
|
||||
this.position = this.start
|
||||
this.signal = options.signal
|
||||
this.onAbort = options.signal === undefined ? undefined : () => { this.destroy(aborted(options.signal?.reason)) }
|
||||
if (options.encoding !== undefined && options.encoding !== null) this.setEncoding(options.encoding)
|
||||
options.signal?.addEventListener('abort', this.onAbort as () => void, { once: true })
|
||||
}
|
||||
|
||||
override _construct(callback: (error?: Error | null) => void): void {
|
||||
if (this.start < 0 || this.end < this.start) {
|
||||
callback(new RangeError('The value of "start" is out of range'))
|
||||
return
|
||||
}
|
||||
if (this.signal?.aborted === true) {
|
||||
callback(aborted(this.signal.reason))
|
||||
return
|
||||
}
|
||||
try {
|
||||
this.fd = openSync(this.path, this.flags)
|
||||
this.pending = false
|
||||
this.emit('open', this.fd)
|
||||
this.emit('ready')
|
||||
callback()
|
||||
} catch (error) {
|
||||
callback(error as Error)
|
||||
}
|
||||
}
|
||||
|
||||
override _read(size: number): void {
|
||||
if (this.fd === null) return
|
||||
const remaining = this.end === Number.POSITIVE_INFINITY ? size : Math.min(size, this.end - this.position + 1)
|
||||
if (remaining <= 0) {
|
||||
this.push(null)
|
||||
return
|
||||
}
|
||||
const buffer = Buffer.allocUnsafe(remaining)
|
||||
let count: number
|
||||
try {
|
||||
count = readSync(this.fd, buffer, 0, remaining, this.position)
|
||||
} catch (error) {
|
||||
this.destroy(error as Error)
|
||||
return
|
||||
}
|
||||
if (count === 0) {
|
||||
this.push(null)
|
||||
return
|
||||
}
|
||||
this.position += count
|
||||
this.bytesRead += count
|
||||
this.push(buffer.subarray(0, count))
|
||||
}
|
||||
|
||||
override _destroy(error: Error | null, callback: (error?: Error | null) => void): void {
|
||||
this.signal?.removeEventListener('abort', this.onAbort as () => void)
|
||||
if (this.fd !== null) closeSync(this.fd)
|
||||
this.fd = null
|
||||
this.pending = false
|
||||
callback(error)
|
||||
}
|
||||
|
||||
/**
|
||||
* Close the stream and release its descriptor.
|
||||
* @param callback - Optional completion callback after `close`.
|
||||
*/
|
||||
close(callback?: (error?: NodeJS.ErrnoException | null) => void): void {
|
||||
if (callback !== undefined) this.once('close', () => { callback(null) })
|
||||
this.destroy()
|
||||
}
|
||||
}
|
||||
|
||||
/** Writable stream committing chunks through the VFS file-descriptor face. */
|
||||
export class WriteStream extends Writable {
|
||||
/** Resolved path opened by this stream. */
|
||||
readonly path: string
|
||||
/** Open descriptor, or null before open and after close. */
|
||||
fd: number | null = null
|
||||
/** Whether the descriptor is still waiting to open. */
|
||||
pending = true
|
||||
/** Bytes committed by this stream. */
|
||||
bytesWritten = 0
|
||||
private readonly flags: string
|
||||
private readonly mode: number | undefined
|
||||
private readonly start: number | undefined
|
||||
private readonly signal: AbortSignal | undefined
|
||||
private readonly onAbort: (() => void) | undefined
|
||||
|
||||
constructor(path: PathArg, options: WriteStreamOptions = {}) {
|
||||
super({
|
||||
autoDestroy: options.autoClose ?? true,
|
||||
decodeStrings: true,
|
||||
defaultEncoding: options.encoding ?? 'utf8',
|
||||
emitClose: options.emitClose ?? true,
|
||||
highWaterMark: options.highWaterMark ?? 64 * 1024,
|
||||
})
|
||||
this.path = asPath(path)
|
||||
this.flags = options.flags ?? 'w'
|
||||
this.mode = options.mode
|
||||
this.start = options.start
|
||||
this.signal = options.signal
|
||||
this.onAbort = options.signal === undefined ? undefined : () => { this.destroy(aborted(options.signal?.reason)) }
|
||||
options.signal?.addEventListener('abort', this.onAbort as () => void, { once: true })
|
||||
}
|
||||
|
||||
override _construct(callback: (error?: Error | null) => void): void {
|
||||
if (this.start !== undefined && this.start < 0) {
|
||||
callback(new RangeError('The value of "start" is out of range'))
|
||||
return
|
||||
}
|
||||
if (this.signal?.aborted === true) {
|
||||
callback(aborted(this.signal.reason))
|
||||
return
|
||||
}
|
||||
try {
|
||||
this.fd = openSync(this.path, this.flags, this.mode)
|
||||
if (this.start !== undefined) fileOf(this.fd, 'write').position = this.start
|
||||
this.pending = false
|
||||
this.emit('open', this.fd)
|
||||
this.emit('ready')
|
||||
callback()
|
||||
} catch (error) {
|
||||
callback(error as Error)
|
||||
}
|
||||
}
|
||||
|
||||
override _write(
|
||||
chunk: string | Uint8Array,
|
||||
encoding: BufferEncoding,
|
||||
callback: (error?: Error | null) => void,
|
||||
): void {
|
||||
try {
|
||||
if (this.fd === null) throw new Error('EBADF: bad file descriptor, write')
|
||||
const data = typeof chunk === 'string' ? Buffer.from(chunk, encoding) : chunk
|
||||
this.bytesWritten += writeSync(this.fd, data)
|
||||
callback()
|
||||
} catch (error) {
|
||||
callback(error as Error)
|
||||
}
|
||||
}
|
||||
|
||||
override _destroy(error: Error | null, callback: (error: Error | null) => void): void {
|
||||
this.signal?.removeEventListener('abort', this.onAbort as () => void)
|
||||
closeDescriptor(this.fd)
|
||||
this.fd = null
|
||||
this.pending = false
|
||||
callback(error)
|
||||
}
|
||||
|
||||
/**
|
||||
* Close the stream and release its descriptor.
|
||||
* @param callback - Optional completion callback after `close`.
|
||||
*/
|
||||
close(callback?: (error?: NodeJS.ErrnoException | null) => void): void {
|
||||
if (callback !== undefined) this.once('close', () => { callback(null) })
|
||||
this.destroy()
|
||||
}
|
||||
}
|
||||
|
||||
/** Close a stream-owned descriptor when it has opened successfully. */
|
||||
function closeDescriptor(fd: number | null): void {
|
||||
if (fd !== null) closeSync(fd)
|
||||
}
|
||||
|
||||
/**
|
||||
* Streaming read is unavailable: node:stream has no implementation here.
|
||||
* @param path - the path a caller wanted streamed, named in the refusal.
|
||||
* @returns Never — it throws naming the unavailable member.
|
||||
* Create a Node-compatible readable file stream over the VFS.
|
||||
* @param path - File path.
|
||||
* @param options - Encoding, range, open, buffer, and abort options.
|
||||
* @returns The readable file stream.
|
||||
*/
|
||||
export function createReadStream(path: PathArg): never {
|
||||
return notImplemented('createReadStream', asPath(path))
|
||||
export function createReadStream(path: PathArg, options?: ReadStreamOptions | BufferEncoding): ReadStream {
|
||||
return new ReadStream(path, typeof options === 'string' ? { encoding: options } : options)
|
||||
}
|
||||
|
||||
/**
|
||||
* Streaming write counterpart of {@link createReadStream}.
|
||||
* @param path - the path a caller wanted streamed, named in the refusal.
|
||||
* @returns Never — it throws naming the unavailable member.
|
||||
* Create a Node-compatible writable file stream over the VFS.
|
||||
* @param path - File path.
|
||||
* @param options - Encoding, open, buffer, and abort options.
|
||||
* @returns The writable file stream.
|
||||
*/
|
||||
export function createWriteStream(path: PathArg): never {
|
||||
return notImplemented('createWriteStream', asPath(path))
|
||||
export function createWriteStream(path: PathArg, options?: WriteStreamOptions | BufferEncoding): WriteStream {
|
||||
return new WriteStream(path, typeof options === 'string' ? { encoding: options } : options)
|
||||
}
|
||||
|
||||
/** Open directory handle (`fs.Dir` subset): iteration plus the close pair. */
|
||||
@@ -538,11 +775,12 @@ export const promises = {
|
||||
// The VFS has no inodes, so a hard link is a byte copy: the caller's contract
|
||||
// is only that both names read the same content until one is removed.
|
||||
link: async (from: PathArg, to: PathArg): Promise<void> => { linkSync(from, to) },
|
||||
open: async (path: PathArg, flags?: string): Promise<FileHandle> => openHandleSync(path, flags),
|
||||
open: async (path: PathArg, flags?: string, mode?: number): Promise<FileHandle> => openHandleSync(path, flags, mode),
|
||||
opendir: async (path: PathArg): Promise<Dir> => opendirSync(path),
|
||||
truncate: async (path: PathArg, length = 0): Promise<void> => {
|
||||
writeFileSync(path, bytesOf(asPath(path)).subarray(0, length))
|
||||
},
|
||||
watch: watchAsync,
|
||||
constants,
|
||||
} satisfies Partial<Record<keyof typeof import('node:fs/promises'), unknown>>
|
||||
|
||||
@@ -559,10 +797,11 @@ export const __esModule = true
|
||||
* the subsets the host tree reads.
|
||||
*/
|
||||
type OwnSignature =
|
||||
| 'constants' | 'promises' | 'Dirent'
|
||||
| 'constants' | 'promises' | 'Dirent' | 'FSWatcher' | 'StatWatcher' | 'ReadStream' | 'WriteStream'
|
||||
| 'readFileSync' | 'writeFileSync' | 'appendFileSync' | 'statSync' | 'lstatSync' | 'realpathSync'
|
||||
| 'readdirSync' | 'mkdirSync' | 'mkdtempSync' | 'rmSync' | 'opendirSync'
|
||||
| 'openSync' | 'readSync' | 'writeSync'
|
||||
| 'openSync' | 'readSync' | 'writeSync' | 'stat' | 'lstat' | 'watch' | 'watchFile' | 'unwatchFile'
|
||||
| 'createReadStream' | 'createWriteStream'
|
||||
|
||||
/**
|
||||
* The `node:fs` declarations this module stands in for. Every other member is
|
||||
@@ -574,10 +813,10 @@ type NodeFace = Partial<Omit<typeof import('node:fs'), OwnSignature>>
|
||||
|
||||
/** CommonJS default export: the members `require()` hands a caller of this module. */
|
||||
export default {
|
||||
constants, promises, Dirent,
|
||||
readFileSync, writeFileSync, appendFileSync, existsSync, statSync, lstatSync, realpathSync, chmodSync,
|
||||
constants, promises, Dirent, FSWatcher, StatWatcher, ReadStream, WriteStream,
|
||||
readFileSync, writeFileSync, appendFileSync, existsSync, statSync, stat, lstatSync, lstat, realpathSync, chmodSync,
|
||||
readdirSync, mkdirSync, mkdtempSync, rmSync, unlinkSync, renameSync, accessSync, opendirSync,
|
||||
openHandleSync, linkSync,
|
||||
openSync, readSync, writeSync, closeSync, watchFile, unwatchFile,
|
||||
openSync, readSync, writeSync, closeSync, watch, watchFile, unwatchFile,
|
||||
createReadStream, createWriteStream,
|
||||
} satisfies NodeFace
|
||||
|
||||
+1
-1
@@ -9,7 +9,7 @@ import { Dirent, promises } from '../fs.ts'
|
||||
/** The promise members of the VFS bridge, as `node:fs/promises` names them. */
|
||||
export const {
|
||||
readFile, writeFile, appendFile, mkdir, mkdtemp, readdir, stat, lstat, realpath, rm, unlink,
|
||||
rename, access, chmod, cp, link, open, opendir, truncate, constants,
|
||||
rename, access, chmod, cp, link, open, opendir, truncate, watch, constants,
|
||||
} = promises
|
||||
|
||||
export { Dirent }
|
||||
|
||||
+82
@@ -0,0 +1,82 @@
|
||||
/**
|
||||
* `node:stream` compatibility backed by readable-stream's browser build.
|
||||
*
|
||||
* readable-stream is the userland copy of Node's stream implementation. The
|
||||
* worker owns only platform adapters such as VFS file streams; stream state,
|
||||
* backpressure, async iteration, abort handling, and event ordering stay in
|
||||
* that maintained implementation.
|
||||
*/
|
||||
import Stream from 'readable-stream'
|
||||
|
||||
type StreamRuntime = typeof import('node:stream') & {
|
||||
compose(...streams: unknown[]): unknown
|
||||
destroy(stream: unknown, error?: Error): void
|
||||
isDisturbed(stream: unknown): boolean
|
||||
}
|
||||
|
||||
type StreamStatics = typeof import('node:stream').Stream & {
|
||||
getDefaultHighWaterMark(objectMode: boolean): number
|
||||
isDestroyed(stream: unknown): boolean | null
|
||||
isWritable(stream: unknown): boolean | null
|
||||
setDefaultHighWaterMark(objectMode: boolean, value: number): void
|
||||
}
|
||||
|
||||
const nodeStream = Stream as unknown as StreamRuntime
|
||||
|
||||
const {
|
||||
Duplex, PassThrough, Readable, Stream: StreamBase, Transform, Writable,
|
||||
addAbortSignal, compose, destroy, finished, isDisturbed, isErrored, isReadable, pipeline, promises,
|
||||
} = nodeStream
|
||||
const streamStatics = StreamBase as unknown as StreamStatics
|
||||
const {
|
||||
getDefaultHighWaterMark, isDestroyed, isWritable, setDefaultHighWaterMark,
|
||||
} = streamStatics
|
||||
|
||||
// readable-stream tracks Node 18's 16 KiB byte default; this repository runs
|
||||
// Node 22+, whose generic and file streams use 64 KiB.
|
||||
if (getDefaultHighWaterMark(false) !== 64 * 1024) setDefaultHighWaterMark(false, 64 * 1024)
|
||||
|
||||
/**
|
||||
* Test whether a value is an ArrayBuffer view.
|
||||
* @param value - Candidate value.
|
||||
* @returns Whether the value is a typed-array or DataView instance.
|
||||
*/
|
||||
const _isArrayBufferView = (value: unknown): value is ArrayBufferView => ArrayBuffer.isView(value)
|
||||
|
||||
/** Default-import namespace carrying Node's stream class and static helpers. */
|
||||
const streamDefault = Object.assign(Stream, {
|
||||
_isArrayBufferView,
|
||||
getDefaultHighWaterMark,
|
||||
isDestroyed,
|
||||
isWritable,
|
||||
setDefaultHighWaterMark,
|
||||
})
|
||||
|
||||
export {
|
||||
Duplex,
|
||||
PassThrough,
|
||||
Readable,
|
||||
StreamBase as Stream,
|
||||
Transform,
|
||||
Writable,
|
||||
addAbortSignal,
|
||||
compose,
|
||||
destroy,
|
||||
finished,
|
||||
getDefaultHighWaterMark,
|
||||
_isArrayBufferView,
|
||||
isDestroyed,
|
||||
isDisturbed,
|
||||
isErrored,
|
||||
isReadable,
|
||||
isWritable,
|
||||
pipeline,
|
||||
promises,
|
||||
setDefaultHighWaterMark,
|
||||
}
|
||||
|
||||
/** CommonJS interop marker consumed by the worker module loader. */
|
||||
export const __esModule = true
|
||||
|
||||
/** CommonJS-compatible namespace for default imports. */
|
||||
export default streamDefault
|
||||
@@ -1,38 +0,0 @@
|
||||
/**
|
||||
* `node:stream` stub. Every harness import of this module in the reachable tree
|
||||
* is type-only (`Duplex`/`Readable`/`Writable` annotations), so nothing here runs
|
||||
* unless a value import appears; then it says so.
|
||||
*/
|
||||
import { notImplementedFail } from '../../notImplementedFail.ts'
|
||||
|
||||
const MODULE = 'node:stream'
|
||||
|
||||
/** Readable stream (unavailable; use WHATWG ReadableStream). */
|
||||
export const Readable: typeof import('node:stream').Readable = notImplementedFail(MODULE, 'Readable')
|
||||
|
||||
/** Writable stream (unavailable). */
|
||||
export const Writable: typeof import('node:stream').Writable = notImplementedFail(MODULE, 'Writable')
|
||||
|
||||
/** Duplex stream (unavailable). */
|
||||
export const Duplex: typeof import('node:stream').Duplex = notImplementedFail(MODULE, 'Duplex')
|
||||
|
||||
/** Transform stream (unavailable). */
|
||||
export const Transform: typeof import('node:stream').Transform = notImplementedFail(MODULE, 'Transform')
|
||||
|
||||
/** PassThrough stream (unavailable). */
|
||||
export const PassThrough: typeof import('node:stream').PassThrough = notImplementedFail(MODULE, 'PassThrough')
|
||||
|
||||
/** Pipeline helper (unavailable). */
|
||||
export const pipeline: typeof import('node:stream').pipeline = notImplementedFail(MODULE, 'pipeline')
|
||||
|
||||
/** Finished helper (unavailable). */
|
||||
export const finished: typeof import('node:stream').finished = notImplementedFail(MODULE, 'finished')
|
||||
|
||||
/** CommonJS interop marker: the worker loader hands `default` to default imports (see ./builtins.ts). */
|
||||
export const __esModule = true
|
||||
|
||||
/** The `node:stream` declarations this module stands in for. */
|
||||
type NodeFace = Partial<typeof import('node:stream')>
|
||||
|
||||
/** CommonJS default export: the members `require()` hands a caller of this module. */
|
||||
export default { Readable, Writable, Duplex, Transform, PassThrough, pipeline, finished } satisfies NodeFace
|
||||
@@ -32,6 +32,7 @@ import * as nodeModule from './builtin_modules/implemented/module.ts'
|
||||
import * as nodeOs from './builtin_modules/implemented/os.ts'
|
||||
import * as nodePath from './builtin_modules/implemented/path.ts'
|
||||
import * as nodePerfHooks from './builtin_modules/implemented/perf_hooks.ts'
|
||||
import * as nodeStream from './builtin_modules/implemented/stream.ts'
|
||||
import * as nodeTimersPromises from './builtin_modules/implemented/timers/promises.ts'
|
||||
import * as nodeUrl from './builtin_modules/implemented/url.ts'
|
||||
import * as nodeUtil from './builtin_modules/implemented/util.ts'
|
||||
@@ -40,12 +41,9 @@ import * as nodeZlib from './builtin_modules/implemented/zlib.ts'
|
||||
import * as nodeChildProcess from './builtin_modules/implemented/child_process.ts'
|
||||
import * as nodeNet from './builtin_modules/mock/net.ts'
|
||||
import * as nodeSqlite from './builtin_modules/mock/sqlite.ts'
|
||||
import * as nodeStream from './builtin_modules/mock/stream.ts'
|
||||
import * as nodeVm from './builtin_modules/mock/vm.ts'
|
||||
import * as nodeWorkerThreads from './builtin_modules/mock/worker_threads.ts'
|
||||
import * as chokidar from './external_packages/chokidar.ts'
|
||||
import * as koffi from './external_packages/koffi.ts'
|
||||
import * as landlockRun from './external_packages/node-addon-landlock-run.ts'
|
||||
import * as nodePty from './external_packages/node-pty.ts'
|
||||
import * as piAi from './external_packages/pi-ai.ts'
|
||||
import * as ripgrep from './external_packages/ripgrep.ts'
|
||||
@@ -83,14 +81,12 @@ const BUILTINS: Record<string, StaticModuleFactory> = {
|
||||
|
||||
/** External npm packages replaced wholesale (structural not-implemented stubs and fakes). */
|
||||
const EXTERNALS: Record<string, StaticModuleFactory> = {
|
||||
'chokidar': () => chokidar,
|
||||
'koffi': () => koffi,
|
||||
'sharp': () => sharp,
|
||||
'node-pty': () => nodePty,
|
||||
'ws': () => ws,
|
||||
'@vscode/ripgrep': () => ripgrep,
|
||||
'@earendil-works/pi-ai': () => piAi,
|
||||
'@deepseek-ai/node-addon-landlock-run': () => landlockRun,
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -1,68 +0,0 @@
|
||||
/**
|
||||
* `chokidar` stub: a constructible watcher that never fires. Settings and
|
||||
* credentials call `watch()` unconditionally in `[Service.init]`, and the
|
||||
* in-memory VFS has no external writer, so "no events" is the truth here rather
|
||||
* than a degradation.
|
||||
*/
|
||||
|
||||
/** No-op watcher with chokidar's chainable face. */
|
||||
export class FSWatcher {
|
||||
/**
|
||||
* Register a listener; no event is ever emitted.
|
||||
* @returns this watcher.
|
||||
*/
|
||||
on(): this {
|
||||
return this
|
||||
}
|
||||
|
||||
/**
|
||||
* Register a one-shot listener; no event is ever emitted.
|
||||
* @returns this watcher.
|
||||
*/
|
||||
once(): this {
|
||||
return this
|
||||
}
|
||||
|
||||
/**
|
||||
* Add paths to the (inert) watch set.
|
||||
* @returns this watcher.
|
||||
*/
|
||||
add(): this {
|
||||
return this
|
||||
}
|
||||
|
||||
/**
|
||||
* Remove paths from the (inert) watch set.
|
||||
* @returns this watcher.
|
||||
*/
|
||||
unwatch(): this {
|
||||
return this
|
||||
}
|
||||
|
||||
/**
|
||||
* Watched paths, as chokidar reports them.
|
||||
* @returns An empty record; nothing is ever watched.
|
||||
*/
|
||||
getWatched(): Record<string, string[]> {
|
||||
return {}
|
||||
}
|
||||
|
||||
/** Close the watcher. */
|
||||
async close(): Promise<void> {
|
||||
// Nothing was ever watched.
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Create an inert watcher.
|
||||
* @returns the watcher.
|
||||
*/
|
||||
export function watch(): FSWatcher {
|
||||
return new FSWatcher()
|
||||
}
|
||||
|
||||
/** CommonJS interop marker: the worker loader hands `default` to default imports (see ./builtins.ts). */
|
||||
export const __esModule = true
|
||||
|
||||
/** CommonJS default export: the members `require()` hands a caller of this module. */
|
||||
export default { watch, FSWatcher }
|
||||
-31
@@ -1,31 +0,0 @@
|
||||
/**
|
||||
* `@deepseek-ai/node-addon-landlock-run` stub: the Landlock launcher. Sandboxing
|
||||
* is part of the declared excluded surface, so `sandbox-local` mounts with the
|
||||
* launcher path and probe present and fails when it tries to confine a process.
|
||||
*/
|
||||
import { notImplementedFail } from '../notImplementedFail.ts'
|
||||
|
||||
const MODULE = '@deepseek-ai/node-addon-landlock-run'
|
||||
|
||||
/** Launcher executable name, read at module scope by sandbox-local. */
|
||||
export const LAUNCHER_BIN = 'landlock-run'
|
||||
|
||||
/** Exit code the launcher reports when confinement itself fails. */
|
||||
export const LAUNCHER_FAILURE_EXIT = 126
|
||||
|
||||
/**
|
||||
* Path of the launcher binary; nothing in a browser can execute it.
|
||||
* @returns The image path consumers read before failing on their own terms.
|
||||
*/
|
||||
export function launcherPath(): string {
|
||||
return `/dsh/bin/${LAUNCHER_BIN}`
|
||||
}
|
||||
|
||||
/** Landlock availability probe (unavailable). */
|
||||
export const probe = notImplementedFail(MODULE, 'probe')
|
||||
|
||||
/** CommonJS interop marker: the worker loader hands `default` to default imports (see ./builtins.ts). */
|
||||
export const __esModule = true
|
||||
|
||||
/** CommonJS default export: the members `require()` hands a caller of this module. */
|
||||
export default { LAUNCHER_BIN, LAUNCHER_FAILURE_EXIT, launcherPath, probe }
|
||||
-2
@@ -8,10 +8,8 @@
|
||||
|
||||
/** External packages served from the worker bundle instead of the VFS. */
|
||||
export const REPLACED_EXTERNAL_PACKAGES: readonly string[] = [
|
||||
'@deepseek-ai/node-addon-landlock-run',
|
||||
'@earendil-works/pi-ai',
|
||||
'@vscode/ripgrep',
|
||||
'chokidar',
|
||||
'koffi',
|
||||
'node-pty',
|
||||
'sharp',
|
||||
|
||||
@@ -57,7 +57,8 @@ export function describeFailure(program: string, path: string, error: unknown):
|
||||
* @returns the error to throw.
|
||||
*/
|
||||
export function filesystemError(code: string, syscall: string, path: string): VfsError {
|
||||
const error = new Error(`${code}: ${syscall} failed, ${syscall} '${path}'`) as VfsError
|
||||
const reason = code === 'EACCES' ? 'permission denied' : `${syscall} failed`
|
||||
const error = new Error(`${code}: ${reason}, ${syscall} '${path}'`) as VfsError
|
||||
error.code = code
|
||||
error.path = path
|
||||
error.syscall = syscall
|
||||
@@ -75,7 +76,6 @@ function statsOf(stats: VfsStats): ShellStats {
|
||||
*/
|
||||
export function hostFileSystem(): ShellFileSystem {
|
||||
const vfs = (): ReturnType<typeof requireActiveVfs> => requireActiveVfs()
|
||||
// oxlint-disable-next-line typescript/require-await -- async face, in-memory backend; see the note below.
|
||||
const stat = async (path: string): Promise<ShellStats | undefined> => {
|
||||
try {
|
||||
return statsOf(vfs().statSync(path) as VfsStats)
|
||||
@@ -87,7 +87,6 @@ export function hostFileSystem(): ShellFileSystem {
|
||||
}
|
||||
// Several members take no await: the face is asynchronous because a process
|
||||
// worker's filesystem is, while this backend answers from memory.
|
||||
/* oxlint-disable typescript/require-await -- see the note above. */
|
||||
return {
|
||||
stat,
|
||||
list: async (path: string): Promise<ShellDirent[]> => {
|
||||
@@ -116,5 +115,4 @@ export function hostFileSystem(): ShellFileSystem {
|
||||
vfs().renameSync(from, to)
|
||||
},
|
||||
}
|
||||
/* oxlint-enable typescript/require-await */
|
||||
}
|
||||
|
||||
@@ -0,0 +1,188 @@
|
||||
/** Landlock launcher parsing and per-process VFS enforcement for the worker shell. */
|
||||
import { resolve } from '../../module-system/posix-path.ts'
|
||||
import { DSH_TMP } from '../../storage/paths.ts'
|
||||
import { filesystemError } from '../fs-access.ts'
|
||||
import type { ShellDirent, ShellFileSystem, ShellStats } from '../types.ts'
|
||||
import type { VirtualExecutable, VirtualExecutableExit } from './virtual-executables.ts'
|
||||
|
||||
/** Parsed invocation of the native launcher's unchanged argv grammar. */
|
||||
export type LandlockInvocation =
|
||||
| { readonly kind: 'probe' }
|
||||
| {
|
||||
readonly kind: 'run'
|
||||
readonly readOnly: readonly string[]
|
||||
readonly readWrite: readonly string[]
|
||||
readonly argv: readonly string[]
|
||||
}
|
||||
|
||||
/** Launcher-owned failure; callers print its message with the `landlock-run:` prefix. */
|
||||
export class LandlockLauncherError extends Error {}
|
||||
|
||||
/**
|
||||
* Parse the native launcher's argv grammar.
|
||||
* @param args - Arguments after the launcher executable.
|
||||
* @returns A probe or confined-run request.
|
||||
*/
|
||||
export function parseLandlockArguments(args: readonly string[]): LandlockInvocation {
|
||||
const readOnly: string[] = []
|
||||
const readWrite: string[] = []
|
||||
for (let index = 0; index < args.length;) {
|
||||
const argument = args[index] as string
|
||||
if (argument === '--probe') {
|
||||
if (args.length !== 1) throw new LandlockLauncherError('usage error: --probe takes no other arguments')
|
||||
return { kind: 'probe' }
|
||||
}
|
||||
if (argument === '--ro' || argument === '--rw') {
|
||||
const path = args[index + 1]
|
||||
if (path === undefined) throw new LandlockLauncherError(`usage error: ${argument} requires a path`)
|
||||
;(argument === '--ro' ? readOnly : readWrite).push(path)
|
||||
index += 2
|
||||
continue
|
||||
}
|
||||
if (argument === '--') {
|
||||
const argv = args.slice(index + 1)
|
||||
if (argv.length === 0) throw new LandlockLauncherError('usage error: missing `-- <argv>...` command')
|
||||
return { kind: 'run', readOnly, readWrite, argv }
|
||||
}
|
||||
throw new LandlockLauncherError(`usage error: unknown argument: ${argument}`)
|
||||
}
|
||||
throw new LandlockLauncherError('usage error: missing `-- <argv>...` command')
|
||||
}
|
||||
|
||||
/** Map the host launcher's temp path into the Worker VFS. */
|
||||
function vfsPath(path: string, cwd: string): string {
|
||||
const absolute = resolve(cwd, path)
|
||||
if (absolute === '/tmp') return DSH_TMP
|
||||
if (absolute.startsWith('/tmp/')) return `${DSH_TMP}${absolute.slice('/tmp'.length)}`
|
||||
return absolute
|
||||
}
|
||||
|
||||
/** Whether a normalized path is the root itself or one of its descendants. */
|
||||
function contains(root: string, path: string): boolean {
|
||||
return root === '/' || path === root || path.startsWith(`${root}/`)
|
||||
}
|
||||
|
||||
/** Throw the denial dialect consumed by `dsh-bash-sandbox`. */
|
||||
function deny(syscall: string, path: string): never {
|
||||
throw filesystemError('EACCES', syscall, path)
|
||||
}
|
||||
|
||||
/** Stats for the virtual `/dev/null` file. */
|
||||
const NULL_STATS: ShellStats = { directory: false, size: 0, mtimeMs: 0 }
|
||||
const DEV_ROOT = '/dev'
|
||||
const NULL_PATH = '/dev/null'
|
||||
|
||||
/** Build one launcher-owned terminal result. */
|
||||
function launcherExit(exitCode: number, stdout = '', stderr = ''): VirtualExecutableExit {
|
||||
return { kind: 'exit', exitCode, stdout, stderr }
|
||||
}
|
||||
|
||||
/** Convert a parser or grant failure into the native launcher's fatal dialect. */
|
||||
function launcherFailure(error: unknown): VirtualExecutableExit {
|
||||
const detail = error instanceof LandlockLauncherError ? error.message : String(error)
|
||||
return launcherExit(125, '', `landlock-run: ${detail}\n`)
|
||||
}
|
||||
|
||||
/**
|
||||
* Validate grant roots and create one process-local filesystem guard.
|
||||
* @param base - Host-side VFS adapter all permitted calls delegate to.
|
||||
* @param invocation - Parsed confined-run request.
|
||||
* @param cwd - Launcher's working directory for relative grant paths.
|
||||
* @returns A filesystem enforcing only this invocation's grants.
|
||||
*/
|
||||
export async function landlockFileSystem(
|
||||
base: ShellFileSystem,
|
||||
invocation: Extract<LandlockInvocation, { kind: 'run' }>,
|
||||
cwd: string,
|
||||
): Promise<ShellFileSystem> {
|
||||
const normalizeGrant = async (path: string): Promise<string> => {
|
||||
if (path === '') throw new LandlockLauncherError('cannot open rule path: : No such file or directory')
|
||||
const target = vfsPath(path, cwd)
|
||||
if (target !== DEV_ROOT && target !== NULL_PATH && await base.stat(target) === undefined) {
|
||||
throw new LandlockLauncherError(`cannot open rule path: ${path}: No such file or directory`)
|
||||
}
|
||||
return target
|
||||
}
|
||||
const readOnly = await Promise.all(invocation.readOnly.map(normalizeGrant))
|
||||
const readWrite = await Promise.all(invocation.readWrite.map(normalizeGrant))
|
||||
const readable = [...readOnly, ...readWrite]
|
||||
|
||||
const readPath = (path: string, syscall: string): string => {
|
||||
const target = vfsPath(path, cwd)
|
||||
if (!readable.some(root => contains(root, target))) deny(syscall, path)
|
||||
return target
|
||||
}
|
||||
const writePath = (path: string, syscall: string): string => {
|
||||
const target = vfsPath(path, cwd)
|
||||
if (!readWrite.some(root => contains(root, target))) deny(syscall, path)
|
||||
return target
|
||||
}
|
||||
|
||||
return {
|
||||
stat: async (path: string): Promise<ShellStats | undefined> => {
|
||||
const target = readPath(path, 'stat')
|
||||
if (target === NULL_PATH) return NULL_STATS
|
||||
if (target === DEV_ROOT && !await base.stat(target)) return { directory: true, size: 0, mtimeMs: 0 }
|
||||
return await base.stat(target)
|
||||
},
|
||||
list: async (path: string): Promise<ShellDirent[]> => {
|
||||
const target = readPath(path, 'scandir')
|
||||
if (target === DEV_ROOT) return [{ name: 'null', directory: false }]
|
||||
if (target === NULL_PATH) throw filesystemError('ENOTDIR', 'scandir', path)
|
||||
return await base.list(target)
|
||||
},
|
||||
readText: async (path: string): Promise<string> => {
|
||||
const target = readPath(path, 'open')
|
||||
return target === NULL_PATH ? '' : await base.readText(target)
|
||||
},
|
||||
writeText: async (path: string, text: string, append = false): Promise<void> => {
|
||||
const target = writePath(path, 'open')
|
||||
if (target !== NULL_PATH) await base.writeText(target, text, append)
|
||||
},
|
||||
mkdir: async (path: string, recursive: boolean): Promise<void> => {
|
||||
const target = writePath(path, 'mkdir')
|
||||
if (target === NULL_PATH) throw filesystemError('EEXIST', 'mkdir', path)
|
||||
await base.mkdir(target, recursive)
|
||||
},
|
||||
remove: async (path: string, options: { recursive: boolean; force: boolean }): Promise<void> => {
|
||||
const target = writePath(path, 'rm')
|
||||
if (target === NULL_PATH) deny('rm', path)
|
||||
await base.remove(target, options)
|
||||
},
|
||||
rename: async (from: string, to: string): Promise<void> => {
|
||||
const source = writePath(from, 'rename')
|
||||
const destination = writePath(to, 'rename')
|
||||
if (source === NULL_PATH || destination === NULL_PATH) deny('rename', source === NULL_PATH ? from : to)
|
||||
await base.rename(source, destination)
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
/** Virtual executable implementing the native launcher's CLI over VFS grants. */
|
||||
export const LANDLOCK_EXECUTABLE: VirtualExecutable = {
|
||||
name: 'landlock-run',
|
||||
async prepare(args, context) {
|
||||
try {
|
||||
const invocation = parseLandlockArguments(args)
|
||||
if (invocation.kind === 'probe') return launcherExit(0, 'landlock: fully enforced\n')
|
||||
return {
|
||||
kind: 'delegate',
|
||||
argv: invocation.argv,
|
||||
filesystem: await landlockFileSystem(context.filesystem, invocation, context.cwd),
|
||||
missingExecutable: launcherExit(125, '', 'landlock-run: exec failed: No such file or directory\n'),
|
||||
}
|
||||
} catch (error) {
|
||||
return launcherFailure(error)
|
||||
}
|
||||
},
|
||||
runSync(args) {
|
||||
try {
|
||||
const invocation = parseLandlockArguments(args)
|
||||
return invocation.kind === 'probe'
|
||||
? launcherExit(0, 'landlock: fully enforced\n')
|
||||
: { kind: 'asynchronous' }
|
||||
} catch (error) {
|
||||
return launcherFailure(error)
|
||||
}
|
||||
},
|
||||
}
|
||||
@@ -0,0 +1,61 @@
|
||||
/** Virtual executable registry used by the Worker process launcher. */
|
||||
import { basename } from '../../module-system/posix-path.ts'
|
||||
import type { ShellFileSystem } from '../types.ts'
|
||||
import { LANDLOCK_EXECUTABLE } from './landlock.ts'
|
||||
|
||||
/** Completed virtual executable invocation. */
|
||||
export interface VirtualExecutableExit {
|
||||
readonly kind: 'exit'
|
||||
readonly exitCode: number
|
||||
readonly stdout: string
|
||||
readonly stderr: string
|
||||
}
|
||||
|
||||
/** Invocation delegated to the normal Worker command runner after preparation. */
|
||||
export interface VirtualExecutableDelegate {
|
||||
readonly kind: 'delegate'
|
||||
readonly argv: readonly string[]
|
||||
readonly filesystem: ShellFileSystem
|
||||
readonly missingExecutable: VirtualExecutableExit
|
||||
}
|
||||
|
||||
/** Result of preparing an asynchronous virtual executable invocation. */
|
||||
export type VirtualExecutablePreparation = VirtualExecutableExit | VirtualExecutableDelegate
|
||||
|
||||
/** Result available to the synchronous child-process face. */
|
||||
export type VirtualExecutableSyncResult = VirtualExecutableExit | { readonly kind: 'asynchronous' }
|
||||
|
||||
/** One executable implemented by the Worker instead of an operating-system binary. */
|
||||
export interface VirtualExecutable {
|
||||
/** Platform executable name, independent of package-manager installation path. */
|
||||
readonly name: string
|
||||
/**
|
||||
* Prepare an invocation or complete it without entering the command runner.
|
||||
* @param args - Arguments after the executable path.
|
||||
* @param context - Working directory and ambient Worker filesystem.
|
||||
* @returns The completed result or delegated command and filesystem.
|
||||
*/
|
||||
prepare(
|
||||
args: readonly string[],
|
||||
context: { readonly cwd: string; readonly filesystem: ShellFileSystem },
|
||||
): Promise<VirtualExecutablePreparation>
|
||||
/**
|
||||
* Handle the subset that can complete synchronously.
|
||||
* @param args - Arguments after the executable path.
|
||||
* @returns A completed result or the asynchronous marker.
|
||||
*/
|
||||
runSync(args: readonly string[]): VirtualExecutableSyncResult
|
||||
}
|
||||
|
||||
const EXECUTABLES: ReadonlyMap<string, VirtualExecutable> = new Map([
|
||||
[LANDLOCK_EXECUTABLE.name, LANDLOCK_EXECUTABLE],
|
||||
])
|
||||
|
||||
/**
|
||||
* Resolve a Worker platform executable by logical name.
|
||||
* @param path - Bare name or executable path passed to `spawn`.
|
||||
* @returns Its implementation, or undefined for the normal command table.
|
||||
*/
|
||||
export function virtualExecutable(path: string): VirtualExecutable | undefined {
|
||||
return EXECUTABLES.get(basename(path))
|
||||
}
|
||||
@@ -4,15 +4,15 @@
|
||||
* which backend the worker entry mounted.
|
||||
* @module @deepseek-ai/dsh-experimental-webworker-runtime/src/storage/active
|
||||
*/
|
||||
import type { MemoryVfs } from './memory.ts'
|
||||
import type { Vfs } from './types.ts'
|
||||
|
||||
let active: MemoryVfs | undefined
|
||||
let active: Vfs | undefined
|
||||
|
||||
/**
|
||||
* Publish the filesystem the `node:fs` proxy reads.
|
||||
* @param vfs - Filesystem mounted by the worker entry.
|
||||
*/
|
||||
export function setActiveVfs(vfs: MemoryVfs): void {
|
||||
export function setActiveVfs(vfs: Vfs): void {
|
||||
active = vfs
|
||||
}
|
||||
|
||||
@@ -20,7 +20,7 @@ export function setActiveVfs(vfs: MemoryVfs): void {
|
||||
* Read the mounted filesystem.
|
||||
* @returns The active filesystem.
|
||||
*/
|
||||
export function requireActiveVfs(): MemoryVfs {
|
||||
export function requireActiveVfs(): Vfs {
|
||||
if (active === undefined) {
|
||||
throw new Error('webworker vfs: no filesystem is mounted; the worker entry must call setActiveVfs before any node:fs access')
|
||||
}
|
||||
|
||||
@@ -1,14 +1,14 @@
|
||||
/**
|
||||
* In-memory filesystem behind the worker's `node:fs` proxy. Contents come from
|
||||
* the build-time image (see {@link loadVfsImage}); writes stay in memory and
|
||||
* vanish with the worker.
|
||||
* the build-time image (see {@link loadVfsImage}); this remains the synchronous
|
||||
* authority when an asynchronous durable sink mirrors selected subtrees.
|
||||
* @module @deepseek-ai/dsh-experimental-webworker-runtime/src/storage/memory
|
||||
*/
|
||||
import { dirname, join, normalize, resolve, SEP } from '../module-system/posix-path.ts'
|
||||
import { parseTar } from './tar.ts'
|
||||
import type {
|
||||
VfsBigIntStats, VfsDir, VfsDirent, VfsEncoding, VfsError, VfsFileHandle, VfsReadOptions, VfsStatOptions,
|
||||
VfsStats, VfsWriteOptions,
|
||||
Vfs, VfsBigIntStats, VfsDir, VfsDirent, VfsEncoding, VfsError, VfsFileHandle, VfsMutation,
|
||||
VfsMutationListener, VfsMutationSink, VfsReadOptions, VfsSeedOptions, VfsStatOptions, VfsStats, VfsWriteOptions,
|
||||
} from './types.ts'
|
||||
|
||||
const decoder = new TextDecoder()
|
||||
@@ -46,10 +46,14 @@ function encodingOf(options: VfsReadOptions): VfsEncoding | undefined {
|
||||
// stored value — the round-trip consumers like dsh-credentials-local's
|
||||
// owner-only check rely on. The bits are never enforced: a single-owner
|
||||
// filesystem reads and writes as its owner regardless, like root.
|
||||
function statsOf(size: number, mtimeMs: number, directory: boolean, mode: number): VfsStats {
|
||||
function statsOf(size: number, mtimeMs: number, directory: boolean, ino: bigint, mode: number): VfsStats {
|
||||
return {
|
||||
size,
|
||||
ino: Number(ino),
|
||||
mtimeMs,
|
||||
ctimeMs: mtimeMs,
|
||||
atimeMs: mtimeMs,
|
||||
birthtimeMs: mtimeMs,
|
||||
mtime: new Date(mtimeMs),
|
||||
mode: (directory ? 0o040000 : 0o100000) | (mode & 0o777),
|
||||
isFile: () => !directory,
|
||||
@@ -107,16 +111,26 @@ function bigIntStatsOf(size: number, mtimeMs: number, directory: boolean, ino: b
|
||||
}
|
||||
}
|
||||
|
||||
/** Construction inputs for {@link MemoryVfs}. */
|
||||
export interface MemoryVfsOptions {
|
||||
/** Durable write-behind observer; absent leaves the filesystem ephemeral. */
|
||||
readonly sink?: VfsMutationSink
|
||||
}
|
||||
|
||||
/**
|
||||
* Filesystem held in two maps: one for file bytes, one for directories.
|
||||
* Every path is normalized to an absolute POSIX path without a trailing
|
||||
* separator, so callers may pass either form.
|
||||
*/
|
||||
export class MemoryVfs {
|
||||
export class MemoryVfs implements Vfs {
|
||||
private readonly files = new Map<string, FileNode>()
|
||||
private readonly directories = new Set<string>([SEP])
|
||||
/** Directory permission bits; absence means {@link DEFAULT_DIRECTORY_MODE}. */
|
||||
private readonly directoryModes = new Map<string, number>()
|
||||
/** Directory mtimes advance when their immediate entry set changes. */
|
||||
private readonly directoryMtimes = new Map<string, number>()
|
||||
private readonly mutationListeners = new Set<VfsMutationListener>()
|
||||
private readonly sink: VfsMutationSink | undefined
|
||||
private temporaries = 0
|
||||
// Identity per path, assigned on first stat and dropped when the path goes:
|
||||
// the filesystem service builds its version token from `ino` plus the
|
||||
@@ -124,6 +138,47 @@ export class MemoryVfs {
|
||||
private readonly identities = new Map<string, bigint>()
|
||||
private lastIdentity = 0n
|
||||
|
||||
/**
|
||||
* Build the synchronous filesystem authority.
|
||||
* @param options - Optional durable write-behind sink.
|
||||
*/
|
||||
constructor(options: MemoryVfsOptions = {}) {
|
||||
this.sink = options.sink
|
||||
}
|
||||
|
||||
/**
|
||||
* Settle the durable sink without changing in-memory success.
|
||||
* @returns A promise that resolves when all recorded mutations are stored.
|
||||
*/
|
||||
async flush(): Promise<void> {
|
||||
await this.sink?.flush()
|
||||
}
|
||||
|
||||
/**
|
||||
* Observe committed runtime mutations. Image seeding is deliberately silent.
|
||||
* @param listener - Consumer called after each successful mutation.
|
||||
* @returns A disposer that prevents future calls.
|
||||
*/
|
||||
subscribe(listener: VfsMutationListener): () => void {
|
||||
this.mutationListeners.add(listener)
|
||||
return () => { this.mutationListeners.delete(listener) }
|
||||
}
|
||||
|
||||
/** Publish after state changes; one faulty observer cannot roll back a write. */
|
||||
private publish(mutation: VfsMutation): void {
|
||||
const observers: VfsMutationListener[] = [
|
||||
...(this.sink === undefined ? [] : [(change: VfsMutation): void => { this.sink?.record(change) }]),
|
||||
...this.mutationListeners,
|
||||
]
|
||||
for (const listener of observers) {
|
||||
try {
|
||||
listener(mutation)
|
||||
} catch (error) {
|
||||
console.error('webworker vfs: mutation observer failed', error)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/** Promise face mirroring `node:fs/promises` for the methods the roster uses. */
|
||||
readonly promises = {
|
||||
readFile: async (path: string, options?: VfsReadOptions): Promise<string | Uint8Array> => this.readFileSync(path, options),
|
||||
@@ -198,11 +253,12 @@ export class MemoryVfs {
|
||||
const [size, mtimeMs, directory, mode] = node !== undefined
|
||||
? [node.bytes.length, node.mtimeMs, false, node.mode] as const
|
||||
: this.directories.has(target)
|
||||
? [0, 0, true, this.directoryModes.get(target) ?? DEFAULT_DIRECTORY_MODE] as const
|
||||
? [0, this.directoryMtimes.get(target) ?? 0, true, this.directoryModes.get(target) ?? DEFAULT_DIRECTORY_MODE] as const
|
||||
: fail('ENOENT', 'stat', target)
|
||||
const identity = this.identityOf(target)
|
||||
return options?.bigint === true
|
||||
? bigIntStatsOf(size, mtimeMs, directory, this.identityOf(target), mode)
|
||||
: statsOf(size, mtimeMs, directory, mode)
|
||||
? bigIntStatsOf(size, mtimeMs, directory, identity, mode)
|
||||
: statsOf(size, mtimeMs, directory, identity, mode)
|
||||
}
|
||||
|
||||
/** @returns Stats in the plain shape, for internal callers that read `size`/`mtimeMs`. */
|
||||
@@ -244,6 +300,13 @@ export class MemoryVfs {
|
||||
return previous === undefined ? now : Math.max(now, previous + 1)
|
||||
}
|
||||
|
||||
/** Advance a directory's mtime after its immediate children change. */
|
||||
private touchDirectory(target: string): void {
|
||||
const previous = this.directoryMtimes.get(target)
|
||||
const now = Date.now()
|
||||
this.directoryMtimes.set(target, previous === undefined ? now : Math.max(now, previous + 1))
|
||||
}
|
||||
|
||||
/**
|
||||
* List a directory.
|
||||
* @param path - Directory path.
|
||||
@@ -312,7 +375,11 @@ export class MemoryVfs {
|
||||
this.mkdirSync(parent, options)
|
||||
}
|
||||
this.directories.add(target)
|
||||
if (options?.mode !== undefined) this.directoryModes.set(target, options.mode & 0o777)
|
||||
this.touchDirectory(target)
|
||||
this.touchDirectory(parent)
|
||||
const mode = (options?.mode ?? DEFAULT_DIRECTORY_MODE) & 0o777
|
||||
if (mode !== DEFAULT_DIRECTORY_MODE) this.directoryModes.set(target, mode)
|
||||
this.publish({ kind: 'mkdir', path: target, mode })
|
||||
return target
|
||||
}
|
||||
|
||||
@@ -331,8 +398,14 @@ export class MemoryVfs {
|
||||
if (flag.startsWith('a')) { this.appendFileSync(target, data); return }
|
||||
// POSIX open(O_CREAT): the mode applies at creation only; a rewrite keeps
|
||||
// the entry's bits.
|
||||
const mode = this.files.get(target)?.mode ?? (options?.mode !== undefined ? options.mode & 0o777 : DEFAULT_FILE_MODE)
|
||||
this.files.set(target, { bytes: typeof data === 'string' ? encoder.encode(data) : data, mtimeMs: this.touch(target), mode })
|
||||
const previous = this.files.get(target)
|
||||
const mode = previous?.mode ?? (options?.mode !== undefined ? options.mode & 0o777 : DEFAULT_FILE_MODE)
|
||||
const bytes = typeof data === 'string' ? encoder.encode(data) : data
|
||||
this.files.set(target, { bytes, mtimeMs: this.touch(target), mode })
|
||||
if (previous === undefined) this.touchDirectory(dirname(target))
|
||||
this.publish({
|
||||
kind: 'write', path: target, bytes, mode, entryChanged: previous === undefined,
|
||||
})
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -405,7 +478,9 @@ export class MemoryVfs {
|
||||
truncate: async (length = 0): Promise<void> => {
|
||||
const node = this.files.get(target)
|
||||
if (node === undefined) fail('ENOENT', 'ftruncate', target)
|
||||
this.files.set(target, { bytes: node.bytes.slice(0, length), mtimeMs: this.touch(target), mode: node.mode })
|
||||
const bytes = node.bytes.slice(0, length)
|
||||
this.files.set(target, { bytes, mtimeMs: this.touch(target), mode: node.mode })
|
||||
this.publish({ kind: 'write', path: target, bytes, mode: node.mode, entryChanged: false })
|
||||
},
|
||||
...this.handleTail(target),
|
||||
}
|
||||
@@ -414,17 +489,17 @@ export class MemoryVfs {
|
||||
/**
|
||||
* The handle members that do not depend on how the file was opened.
|
||||
*
|
||||
* `sync`/`datasync` have nothing to flush — the bytes are already the stored
|
||||
* ones — and `close` releases nothing, so both directory and file handles
|
||||
* share this tail.
|
||||
* `sync`/`datasync` settle an attached durable sink; an ephemeral filesystem
|
||||
* resolves immediately. `close` releases nothing, so both directory and file
|
||||
* handles share this tail.
|
||||
* @param target - Normalized path the handle was opened on.
|
||||
* @returns Metadata plus the no-op durability and release calls.
|
||||
*/
|
||||
private handleTail(target: string): Pick<VfsFileHandle, 'stat' | 'sync' | 'datasync' | 'close'> {
|
||||
return {
|
||||
stat: async (): Promise<VfsStats> => this.plainStats(target),
|
||||
sync: async (): Promise<void> => {},
|
||||
datasync: async (): Promise<void> => {},
|
||||
sync: async (): Promise<void> => { await this.flush() },
|
||||
datasync: async (): Promise<void> => { await this.flush() },
|
||||
close: async (): Promise<void> => {},
|
||||
}
|
||||
}
|
||||
@@ -443,6 +518,10 @@ export class MemoryVfs {
|
||||
merged.set(existing.bytes)
|
||||
merged.set(addition, existing.bytes.length)
|
||||
this.files.set(target, { bytes: merged, mtimeMs: this.touch(target), mode: existing.mode })
|
||||
this.publish({
|
||||
kind: 'write', path: target, bytes: merged, mode: existing.mode,
|
||||
entryChanged: false, appendedFrom: existing.bytes.length,
|
||||
})
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -460,15 +539,23 @@ export class MemoryVfs {
|
||||
this.files.set(destination, node)
|
||||
this.forgetIdentity(source)
|
||||
this.forgetIdentity(destination)
|
||||
this.touchDirectory(dirname(source))
|
||||
this.touchDirectory(dirname(destination))
|
||||
this.publish({ kind: 'remove', path: source })
|
||||
this.publish({ kind: 'write', path: destination, bytes: node.bytes, mode: node.mode, entryChanged: true })
|
||||
return
|
||||
}
|
||||
if (!this.directories.has(source)) fail('ENOENT', 'rename', source)
|
||||
const prefix = `${source}${SEP}`
|
||||
const movedFiles: Array<{ path: string; bytes: Uint8Array; mode: number }> = []
|
||||
for (const [candidate, value] of [...this.files]) {
|
||||
if (!candidate.startsWith(prefix)) continue
|
||||
this.files.delete(candidate)
|
||||
this.files.set(join(destination, candidate.slice(prefix.length)), value)
|
||||
const target = join(destination, candidate.slice(prefix.length))
|
||||
this.files.set(target, value)
|
||||
movedFiles.push({ path: target, bytes: value.bytes, mode: value.mode })
|
||||
}
|
||||
const movedDirectories: Array<{ path: string; mode: number }> = []
|
||||
for (const candidate of [...this.directories]) {
|
||||
if (!candidate.startsWith(prefix) && candidate !== source) continue
|
||||
const moved = candidate === source ? destination : join(destination, candidate.slice(prefix.length))
|
||||
@@ -477,9 +564,24 @@ export class MemoryVfs {
|
||||
const bits = this.directoryModes.get(candidate)
|
||||
this.directoryModes.delete(candidate)
|
||||
if (bits !== undefined) this.directoryModes.set(moved, bits)
|
||||
movedDirectories.push({ path: moved, mode: bits ?? DEFAULT_DIRECTORY_MODE })
|
||||
const mtime = this.directoryMtimes.get(candidate)
|
||||
this.directoryMtimes.delete(candidate)
|
||||
if (mtime !== undefined) this.directoryMtimes.set(moved, mtime)
|
||||
}
|
||||
this.forgetIdentity(source)
|
||||
this.forgetIdentity(destination)
|
||||
this.touchDirectory(dirname(source))
|
||||
this.touchDirectory(dirname(destination))
|
||||
this.publish({ kind: 'remove', path: source })
|
||||
for (const directory of movedDirectories) {
|
||||
this.publish({ kind: 'mkdir', path: directory.path, mode: directory.mode })
|
||||
}
|
||||
for (const entry of movedFiles) {
|
||||
this.publish({
|
||||
kind: 'write', path: entry.path, bytes: entry.bytes, mode: entry.mode, entryChanged: true,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -499,6 +601,8 @@ export class MemoryVfs {
|
||||
if (this.files.has(target) || this.directories.has(target)) fail('EEXIST', 'link', target)
|
||||
if (!this.directories.has(dirname(target))) fail('ENOENT', 'link', target)
|
||||
this.files.set(target, node)
|
||||
this.touchDirectory(dirname(target))
|
||||
this.publish({ kind: 'write', path: target, bytes: node.bytes, mode: node.mode, entryChanged: true })
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -510,7 +614,9 @@ export class MemoryVfs {
|
||||
const target = this.key(path)
|
||||
const node = this.files.get(target)
|
||||
if (node === undefined) fail('ENOENT', 'truncate', target)
|
||||
this.files.set(target, { bytes: node.bytes.slice(0, length), mtimeMs: this.touch(target), mode: node.mode })
|
||||
const bytes = node.bytes.slice(0, length)
|
||||
this.files.set(target, { bytes, mtimeMs: this.touch(target), mode: node.mode })
|
||||
this.publish({ kind: 'write', path: target, bytes, mode: node.mode, entryChanged: false })
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -523,10 +629,13 @@ export class MemoryVfs {
|
||||
const node = this.files.get(target)
|
||||
if (node !== undefined) {
|
||||
node.mode = mode & 0o777
|
||||
this.publish({ kind: 'chmod', path: target, mode: node.mode })
|
||||
return
|
||||
}
|
||||
if (this.directories.has(target)) {
|
||||
this.directoryModes.set(target, mode & 0o777)
|
||||
const bits = mode & 0o777
|
||||
this.directoryModes.set(target, bits)
|
||||
this.publish({ kind: 'chmod', path: target, mode: bits })
|
||||
return
|
||||
}
|
||||
fail('ENOENT', 'chmod', target)
|
||||
@@ -540,6 +649,8 @@ export class MemoryVfs {
|
||||
const target = this.key(path)
|
||||
if (!this.files.delete(target)) fail('ENOENT', 'unlink', target)
|
||||
this.forgetIdentity(target)
|
||||
this.touchDirectory(dirname(target))
|
||||
this.publish({ kind: 'remove', path: target })
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -551,6 +662,8 @@ export class MemoryVfs {
|
||||
const target = this.key(path)
|
||||
if (this.files.delete(target)) {
|
||||
this.forgetIdentity(target)
|
||||
this.touchDirectory(dirname(target))
|
||||
this.publish({ kind: 'remove', path: target })
|
||||
return
|
||||
}
|
||||
if (this.directories.has(target)) {
|
||||
@@ -561,10 +674,14 @@ export class MemoryVfs {
|
||||
if (!candidate.startsWith(prefix)) continue
|
||||
this.directories.delete(candidate)
|
||||
this.directoryModes.delete(candidate)
|
||||
this.directoryMtimes.delete(candidate)
|
||||
}
|
||||
this.directories.delete(target)
|
||||
this.directoryModes.delete(target)
|
||||
this.directoryMtimes.delete(target)
|
||||
this.forgetIdentity(target)
|
||||
this.touchDirectory(dirname(target))
|
||||
this.publish({ kind: 'remove', path: target })
|
||||
return
|
||||
}
|
||||
if (options?.force !== true) fail('ENOENT', 'rm', target)
|
||||
@@ -586,23 +703,36 @@ export class MemoryVfs {
|
||||
* Seed a file and its parent directories, for image loading and tests.
|
||||
* @param path - File path.
|
||||
* @param data - Text or bytes.
|
||||
* @param mode - Permission bits recorded for the entry.
|
||||
* @param options - Permission bits and modification time supplied by the image or durable store.
|
||||
*/
|
||||
seed(path: string, data: string | Uint8Array, mode = DEFAULT_FILE_MODE): void {
|
||||
seed(path: string, data: string | Uint8Array, options: VfsSeedOptions = {}): void {
|
||||
const target = this.key(path)
|
||||
this.mkdirSync(dirname(target), { recursive: true })
|
||||
this.files.set(target, { bytes: typeof data === 'string' ? encoder.encode(data) : data, mtimeMs: this.touch(target), mode: mode & 0o777 })
|
||||
this.seedDirectory(dirname(target))
|
||||
this.files.set(target, {
|
||||
bytes: typeof data === 'string' ? encoder.encode(data) : data,
|
||||
mtimeMs: options.mtimeMs ?? this.touch(target),
|
||||
mode: (options.mode ?? DEFAULT_FILE_MODE) & 0o777,
|
||||
})
|
||||
this.touchDirectory(dirname(target))
|
||||
}
|
||||
|
||||
/**
|
||||
* Create a directory and its parents.
|
||||
* @param path - Directory path.
|
||||
* @param mode - Permission bits recorded for the directory itself.
|
||||
* @param options - Permission bits and modification time supplied by the image or durable store.
|
||||
*/
|
||||
seedDirectory(path: string, mode = DEFAULT_DIRECTORY_MODE): void {
|
||||
seedDirectory(path: string, options: VfsSeedOptions = {}): void {
|
||||
const target = this.key(path)
|
||||
this.mkdirSync(target, { recursive: true })
|
||||
if (mode !== DEFAULT_DIRECTORY_MODE) this.directoryModes.set(target, mode & 0o777)
|
||||
if (!this.directories.has(target)) {
|
||||
const parent = dirname(target)
|
||||
if (parent !== target) this.seedDirectory(parent)
|
||||
if (this.files.has(target)) fail('EEXIST', 'mkdir', target)
|
||||
this.directories.add(target)
|
||||
this.directoryMtimes.set(target, options.mtimeMs ?? Date.now())
|
||||
this.touchDirectory(parent)
|
||||
}
|
||||
if (options.mode !== undefined) this.directoryModes.set(target, options.mode & 0o777)
|
||||
if (options.mtimeMs !== undefined) this.directoryMtimes.set(target, options.mtimeMs)
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -636,10 +766,10 @@ export function loadVfsImage(image: Uint8Array, root = '/dsh', vfs = new MemoryV
|
||||
}
|
||||
const target = join(root, relativeName)
|
||||
if (entry.directory) {
|
||||
vfs.seedDirectory(target, entry.mode)
|
||||
vfs.seedDirectory(target, { mode: entry.mode })
|
||||
continue
|
||||
}
|
||||
vfs.seed(target, entry.bytes, entry.mode)
|
||||
vfs.seed(target, entry.bytes, { mode: entry.mode })
|
||||
}
|
||||
return vfs
|
||||
}
|
||||
|
||||
@@ -22,7 +22,12 @@ export interface VfsError extends Error {
|
||||
/** Subset of `fs.Stats` the roster reads. */
|
||||
export interface VfsStats {
|
||||
readonly size: number
|
||||
/** Stable identity while an entry exists; recreation receives another value. */
|
||||
readonly ino: number
|
||||
readonly mtimeMs: number
|
||||
readonly ctimeMs: number
|
||||
readonly atimeMs: number
|
||||
readonly birthtimeMs: number
|
||||
readonly mtime: Date
|
||||
readonly mode: number
|
||||
isFile(): boolean
|
||||
@@ -85,6 +90,12 @@ export interface VfsWriteOptions {
|
||||
readonly flag?: string
|
||||
}
|
||||
|
||||
/** Explicit metadata for image or durable-store hydration. */
|
||||
export interface VfsSeedOptions {
|
||||
readonly mode?: number
|
||||
readonly mtimeMs?: number
|
||||
}
|
||||
|
||||
/** Directory entry as `readdir` with `withFileTypes` reports it. */
|
||||
export interface VfsDirent {
|
||||
readonly name: string
|
||||
@@ -113,3 +124,90 @@ export interface VfsFileHandle {
|
||||
datasync(): Promise<void>
|
||||
close(): Promise<void>
|
||||
}
|
||||
|
||||
/**
|
||||
* One completed change to the authoritative in-memory filesystem.
|
||||
*
|
||||
* A durable mirror receives the post-write bytes, virtual permission bits, and optional append offset;
|
||||
* live watchers use `entryChanged` to distinguish directory-entry replacement
|
||||
* from content writes. Rename is represented as source removal plus complete
|
||||
* destination mkdir/write records, so a sink never receives a path without the
|
||||
* state needed to materialize it.
|
||||
*/
|
||||
export type VfsMutation =
|
||||
| {
|
||||
readonly kind: 'write'
|
||||
readonly path: string
|
||||
readonly bytes: Uint8Array
|
||||
readonly mode: number
|
||||
readonly entryChanged: boolean
|
||||
readonly appendedFrom?: number
|
||||
}
|
||||
| { readonly kind: 'mkdir'; readonly path: string; readonly mode: number }
|
||||
| { readonly kind: 'remove'; readonly path: string }
|
||||
| { readonly kind: 'chmod'; readonly path: string; readonly mode: number }
|
||||
|
||||
/** Receives one committed VFS mutation. */
|
||||
export type VfsMutationListener = (mutation: VfsMutation) => void
|
||||
|
||||
/** Durable observer attached to the synchronous VFS. */
|
||||
export interface VfsMutationSink {
|
||||
/**
|
||||
* Record one completed mutation without delaying its caller.
|
||||
* @param mutation - Post-commit state to mirror.
|
||||
*/
|
||||
record(mutation: VfsMutation): void
|
||||
/**
|
||||
* Settle all previously recorded mutations.
|
||||
* Implementations report persistence failures and stop mirroring rather than
|
||||
* rejecting, because the in-memory mutation has already committed.
|
||||
* @returns A promise that resolves when the sink has no pending work.
|
||||
*/
|
||||
flush(): Promise<void>
|
||||
}
|
||||
|
||||
/** Synchronous filesystem used by the worker's Node compatibility modules. */
|
||||
export interface Vfs {
|
||||
readonly promises: {
|
||||
readFile(path: string, options?: VfsReadOptions): Promise<string | Uint8Array>
|
||||
writeFile(path: string, data: string | Uint8Array, options?: VfsWriteOptions): Promise<void>
|
||||
appendFile(path: string, data: string | Uint8Array): Promise<void>
|
||||
mkdir(path: string, options?: { recursive?: boolean; mode?: number }): Promise<string | undefined>
|
||||
readdir(path: string, options?: { withFileTypes?: boolean }): Promise<string[] & VfsDirent[]>
|
||||
stat(path: string, options?: VfsStatOptions): Promise<VfsStats | VfsBigIntStats>
|
||||
lstat(path: string, options?: VfsStatOptions): Promise<VfsStats | VfsBigIntStats>
|
||||
realpath(path: string): Promise<string>
|
||||
rename(from: string, to: string): Promise<void>
|
||||
unlink(path: string): Promise<void>
|
||||
rm(path: string, options?: { recursive?: boolean; force?: boolean }): Promise<void>
|
||||
mkdtemp(prefix: string): Promise<string>
|
||||
link(existing: string, next: string): Promise<void>
|
||||
truncate(path: string, length?: number): Promise<void>
|
||||
chmod(path: string, mode: number): Promise<void>
|
||||
opendir(path: string): Promise<VfsDir>
|
||||
open(path: string, flags?: string, mode?: number): Promise<VfsFileHandle>
|
||||
access(path: string): Promise<void>
|
||||
}
|
||||
readFileSync(path: string, options?: VfsReadOptions): string | Uint8Array
|
||||
existsSync(path: string): boolean
|
||||
statSync(path: string, options?: VfsStatOptions): VfsStats | VfsBigIntStats
|
||||
readdirSync(path: string, options?: { withFileTypes?: boolean }): string[] & VfsDirent[]
|
||||
realpathSync(path: string): string
|
||||
mkdirSync(path: string, options?: { recursive?: boolean; mode?: number }): string | undefined
|
||||
writeFileSync(path: string, data: string | Uint8Array, options?: VfsWriteOptions): void
|
||||
appendFileSync(path: string, data: string | Uint8Array): void
|
||||
renameSync(from: string, to: string): void
|
||||
linkSync(existing: string, next: string): void
|
||||
truncateSync(path: string, length?: number): void
|
||||
chmodSync(path: string, mode: number): void
|
||||
unlinkSync(path: string): void
|
||||
rmSync(path: string, options?: { recursive?: boolean; force?: boolean }): void
|
||||
mkdtempSync(prefix: string): string
|
||||
seed(path: string, data: string | Uint8Array, options?: VfsSeedOptions): void
|
||||
seedDirectory(path: string, options?: VfsSeedOptions): void
|
||||
usage(): { files: number; directories: number; bytes: number }
|
||||
/** Register one observer and return its synchronous disposer. */
|
||||
subscribe(listener: VfsMutationListener): () => void
|
||||
/** Settle the attached durable mutation sink, if any. */
|
||||
flush(): Promise<void>
|
||||
}
|
||||
|
||||
@@ -16,13 +16,22 @@ import { afterEach, beforeEach, expect, it, vi } from 'vitest'
|
||||
import { MemoryVfs } from '@deepseek-ai/dsh-experimental-webworker-runtime/src/storage/memory.ts'
|
||||
import { setActiveVfs } from '@deepseek-ai/dsh-experimental-webworker-runtime/src/storage/active.ts'
|
||||
import { spawn, spawnSync } from '@deepseek-ai/dsh-experimental-webworker-runtime/src/node/builtin_modules/implemented/child_process.ts'
|
||||
import {
|
||||
LAUNCHER_FAILURE_EXIT, grantArgs, launcherPath, probe,
|
||||
} from '@deepseek-ai/node-addon-landlock-run'
|
||||
import { processAlive, signalProcess } from '@deepseek-ai/dsh-experimental-webworker-runtime/src/node/process-table.ts'
|
||||
import { hostFileSystem } from '@deepseek-ai/dsh-experimental-webworker-runtime/src/shell/fs-access.ts'
|
||||
import {
|
||||
LANDLOCK_EXECUTABLE, landlockFileSystem, parseLandlockArguments,
|
||||
} from '@deepseek-ai/dsh-experimental-webworker-runtime/src/shell/process/landlock.ts'
|
||||
import { spawnSubprocess } from '@deepseek-ai/dsh-subprocess-local/src/spawn.ts'
|
||||
|
||||
vi.mock('node:child_process', async () =>
|
||||
await import('@deepseek-ai/dsh-experimental-webworker-runtime/src/node/builtin_modules/implemented/child_process.ts'))
|
||||
|
||||
const WORKSPACE = '/dsh/workspace'
|
||||
const HOME = '/dsh/home'
|
||||
const TMP = '/dsh/tmp'
|
||||
|
||||
let vfs: MemoryVfs
|
||||
|
||||
@@ -30,6 +39,8 @@ beforeEach(() => {
|
||||
vfs = new MemoryVfs()
|
||||
setActiveVfs(vfs)
|
||||
vfs.mkdirSync(WORKSPACE, { recursive: true })
|
||||
vfs.mkdirSync(HOME, { recursive: true })
|
||||
vfs.mkdirSync(TMP, { recursive: true })
|
||||
vi.spyOn(process, 'kill').mockImplementation((pid: number, signal?: string | number): true => {
|
||||
if (signal === 0) {
|
||||
if (processAlive(pid)) return true
|
||||
@@ -91,6 +102,180 @@ it('refuses a command name that is not a string, as Node does', () => {
|
||||
it('reports that a synchronous run cannot happen, without throwing at the probe', () => {
|
||||
expect(spawnSync('bwrap').error?.code).toBe('ENOENT')
|
||||
expect(spawnSync('echo').error?.message).toContain('commands run asynchronously')
|
||||
expect(spawnSync(launcherPath(), ['--probe'])).toMatchObject({
|
||||
status: 0,
|
||||
stdout: Buffer.from('landlock: fully enforced\n'),
|
||||
})
|
||||
expect(spawnSync(launcherPath(), ['--ro', '/', '--', 'echo', 'x']).error?.message)
|
||||
.toContain('commands run asynchronously')
|
||||
expect(spawnSync(launcherPath(), ['--probe', '--'])).toMatchObject({
|
||||
status: LAUNCHER_FAILURE_EXIT,
|
||||
stderr: expect.any(Buffer),
|
||||
})
|
||||
})
|
||||
|
||||
it('keeps the native Landlock package API and CLI failure contract', async () => {
|
||||
expect(probe()).toBe('full')
|
||||
expect(probe('/not-the-worker-launcher')).toBe('unusable')
|
||||
expect(probe('/another-package-layout/bin/landlock-run')).toBe('full')
|
||||
expect(launcherPath(() => '/ignored/package.json')).toBe('/ignored/bin/landlock-run')
|
||||
expect(LAUNCHER_FAILURE_EXIT).toBe(125)
|
||||
expect(await collect(spawn(launcherPath(), ['--probe']))).toEqual({
|
||||
stdout: 'landlock: fully enforced\n', stderr: '', code: 0,
|
||||
})
|
||||
const malformed = spawn(launcherPath(), ['--rw'], { cwd: WORKSPACE })
|
||||
expect(await collect(malformed)).toEqual({
|
||||
stdout: '',
|
||||
stderr: 'landlock-run: usage error: --rw requires a path\n',
|
||||
code: 125,
|
||||
})
|
||||
const missingGrant = spawn(launcherPath(), ['--rw', '/dsh/missing', '--', 'touch', `${WORKSPACE}/never`], { cwd: WORKSPACE })
|
||||
expect(await collect(missingGrant)).toEqual({
|
||||
stdout: '',
|
||||
stderr: 'landlock-run: cannot open rule path: /dsh/missing: No such file or directory\n',
|
||||
code: 125,
|
||||
})
|
||||
expect(vfs.existsSync(`${WORKSPACE}/never`)).toBe(false)
|
||||
const missingCommand = spawn(launcherPath(), ['--ro', '/', '--', 'not-a-program'], { cwd: WORKSPACE })
|
||||
expect(await collect(missingCommand)).toEqual({
|
||||
stdout: '',
|
||||
stderr: 'landlock-run: exec failed: No such file or directory\n',
|
||||
code: 125,
|
||||
})
|
||||
})
|
||||
|
||||
it('enforces every ShellFileSystem operation and virtual device edge', async () => {
|
||||
vfs.writeFileSync(`${HOME}/private.txt`, 'private\n')
|
||||
const invocation = parseLandlockArguments([
|
||||
...grantArgs({ readOnly: ['/dev'], readWrite: [WORKSPACE, '/dev/null'] }), '--', 'true',
|
||||
])
|
||||
if (invocation.kind !== 'run') throw new Error('expected a confined run invocation')
|
||||
const guarded = await landlockFileSystem(hostFileSystem(), invocation, WORKSPACE)
|
||||
|
||||
expect(await guarded.stat('/dev/null')).toEqual({ directory: false, size: 0, mtimeMs: 0 })
|
||||
expect(await guarded.stat('/dev')).toEqual({ directory: true, size: 0, mtimeMs: 0 })
|
||||
expect(await guarded.list('/dev')).toEqual([{ name: 'null', directory: false }])
|
||||
await expect(guarded.list('/dev/null')).rejects.toMatchObject({ code: 'ENOTDIR' })
|
||||
expect(await guarded.readText('/dev/null')).toBe('')
|
||||
await guarded.writeText('/dev/null', 'discarded')
|
||||
await expect(guarded.mkdir('/dev/null', false)).rejects.toMatchObject({ code: 'EEXIST' })
|
||||
await expect(guarded.remove('/dev/null', { recursive: false, force: false })).rejects.toMatchObject({ code: 'EACCES' })
|
||||
await expect(guarded.rename('/dev/null', `${WORKSPACE}/null`)).rejects.toMatchObject({ code: 'EACCES' })
|
||||
await expect(guarded.readText(`${HOME}/private.txt`)).rejects.toMatchObject({ code: 'EACCES' })
|
||||
|
||||
await guarded.mkdir('created', false)
|
||||
await guarded.writeText('created/file', 'one')
|
||||
await guarded.writeText('created/file', ' two', true)
|
||||
expect(await guarded.readText(`${WORKSPACE}/created/file`)).toBe('one two')
|
||||
expect(await guarded.list(`${WORKSPACE}/created`)).toEqual([{ name: 'file', directory: false }])
|
||||
await guarded.rename('created/file', 'created/moved')
|
||||
await expect(guarded.rename('created/moved', '/dev/null')).rejects.toMatchObject({ code: 'EACCES' })
|
||||
await guarded.remove('created', { recursive: true, force: false })
|
||||
expect(vfs.existsSync(`${WORKSPACE}/created`)).toBe(false)
|
||||
})
|
||||
|
||||
it('turns an unexpected virtual-launcher preparation failure into exit 125', async () => {
|
||||
const base = hostFileSystem()
|
||||
const result = await LANDLOCK_EXECUTABLE.prepare(
|
||||
['--ro', '/', '--', 'true'],
|
||||
{
|
||||
cwd: WORKSPACE,
|
||||
filesystem: { ...base, stat: () => Promise.reject(new Error('storage unavailable')) },
|
||||
},
|
||||
)
|
||||
expect(result).toEqual({
|
||||
kind: 'exit', exitCode: 125, stdout: '', stderr: 'landlock-run: Error: storage unavailable\n',
|
||||
})
|
||||
})
|
||||
|
||||
it.each([
|
||||
{ args: [], message: 'missing `-- <argv>...` command' },
|
||||
{ args: ['--unknown', '--', 'true'], message: 'unknown argument: --unknown' },
|
||||
{ args: ['--probe', '--'], message: '--probe takes no other arguments' },
|
||||
{ args: ['--'], message: 'missing `-- <argv>...` command' },
|
||||
{ args: ['--rw', '', '--', 'true'], message: 'cannot open rule path' },
|
||||
])('rejects malformed Landlock argv before execution: $message', async ({ args, message }) => {
|
||||
const child = spawn(launcherPath(), args, { cwd: WORKSPACE })
|
||||
const result = await collect(child)
|
||||
expect(result.code).toBe(LAUNCHER_FAILURE_EXIT)
|
||||
expect(result.stderr).toContain(message)
|
||||
})
|
||||
|
||||
it('enforces read-only and workspace-write grants over the VFS', async () => {
|
||||
vfs.writeFileSync(`${HOME}/readable.txt`, 'visible\n')
|
||||
const readOnly = spawn(launcherPath(), [
|
||||
...grantArgs({ readOnly: ['/'], readWrite: ['/dev/null'] }),
|
||||
'--', 'bash', '-c', `cat ${HOME}/readable.txt; echo discarded > /dev/null; echo denied > ${WORKSPACE}/denied.txt`,
|
||||
], { cwd: WORKSPACE })
|
||||
const strict = await collect(readOnly)
|
||||
expect(strict.code).toBe(1)
|
||||
expect(strict.stdout).toBe('visible\n')
|
||||
expect(strict.stderr.toLowerCase()).toContain('permission denied')
|
||||
expect(vfs.existsSync(`${WORKSPACE}/denied.txt`)).toBe(false)
|
||||
|
||||
const workspaceWrite = spawn(launcherPath(), [
|
||||
...grantArgs({ readOnly: ['/'], readWrite: ['/dev/null', '/tmp', WORKSPACE] }),
|
||||
'--', 'bash', '-c', `echo workspace > ${WORKSPACE}/allowed.txt; echo temporary > /tmp/temp.txt; cat /tmp/temp.txt`,
|
||||
], { cwd: WORKSPACE })
|
||||
expect(await collect(workspaceWrite)).toEqual({ stdout: 'temporary\n', stderr: '', code: 0 })
|
||||
expect(vfs.readFileSync(`${WORKSPACE}/allowed.txt`, 'utf8')).toBe('workspace\n')
|
||||
expect(vfs.readFileSync(`${TMP}/temp.txt`, 'utf8')).toBe('temporary\n')
|
||||
expect(vfs.existsSync('/dev/null')).toBe(false)
|
||||
})
|
||||
|
||||
it('normalizes relative grants and denies sibling-prefix escapes and unreadable paths', async () => {
|
||||
vfs.mkdirSync(`${WORKSPACE}/nested`)
|
||||
vfs.mkdirSync(`${WORKSPACE}-other`)
|
||||
vfs.writeFileSync(`${HOME}/private.txt`, 'private\n')
|
||||
const child = spawn(launcherPath(), [
|
||||
...grantArgs({ readOnly: [WORKSPACE], readWrite: ['.'] }),
|
||||
'--', 'bash', '-c', `echo kept > nested/relative.txt; echo escaped > ${WORKSPACE}-other/escape.txt; cat ${HOME}/private.txt`,
|
||||
], { cwd: WORKSPACE })
|
||||
const result = await collect(child)
|
||||
expect(result.code).toBe(1)
|
||||
expect(result.stderr.toLowerCase()).toContain('permission denied')
|
||||
expect(vfs.readFileSync(`${WORKSPACE}/nested/relative.txt`, 'utf8')).toBe('kept\n')
|
||||
expect(vfs.existsSync(`${WORKSPACE}-other/escape.txt`)).toBe(false)
|
||||
expect(result.stdout).not.toContain('private')
|
||||
})
|
||||
|
||||
it('presents the virtual device directory without storing it in the VFS', async () => {
|
||||
const child = spawn(launcherPath(), [
|
||||
...grantArgs({ readOnly: ['/'], readWrite: ['/dev/null'] }),
|
||||
'--', 'bash', '-c', 'ls /dev; cat /dev/null',
|
||||
], { cwd: WORKSPACE })
|
||||
expect(await collect(child)).toEqual({ stdout: 'null\n', stderr: '', code: 0 })
|
||||
expect(vfs.existsSync('/dev')).toBe(false)
|
||||
})
|
||||
|
||||
it('requires both rename paths to be writable', async () => {
|
||||
vfs.writeFileSync(`${WORKSPACE}/source.txt`, 'kept\n')
|
||||
const child = spawn(launcherPath(), [
|
||||
...grantArgs({ readOnly: ['/'], readWrite: [WORKSPACE] }),
|
||||
'--', 'mv', `${WORKSPACE}/source.txt`, `${HOME}/moved.txt`,
|
||||
], { cwd: WORKSPACE })
|
||||
const result = await collect(child)
|
||||
expect(result.code).toBe(1)
|
||||
expect(result.stderr.toLowerCase()).toContain('permission denied')
|
||||
expect(vfs.readFileSync(`${WORKSPACE}/source.txt`, 'utf8')).toBe('kept\n')
|
||||
expect(vfs.existsSync(`${HOME}/moved.txt`)).toBe(false)
|
||||
})
|
||||
|
||||
it('keeps concurrent Landlock grants process-local', async () => {
|
||||
const strict = spawn(launcherPath(), [
|
||||
...grantArgs({ readOnly: ['/'], readWrite: ['/dev/null'] }),
|
||||
'--', 'bash', '-c', `sleep 0.02; echo denied > ${WORKSPACE}/strict.txt`,
|
||||
], { cwd: WORKSPACE })
|
||||
const writable = spawn(launcherPath(), [
|
||||
...grantArgs({ readOnly: ['/'], readWrite: ['/dev/null', WORKSPACE] }),
|
||||
'--', 'bash', '-c', `echo allowed > ${WORKSPACE}/writable.txt`,
|
||||
], { cwd: WORKSPACE })
|
||||
const [strictResult, writableResult] = await Promise.all([collect(strict), collect(writable)])
|
||||
expect(strictResult.code).toBe(1)
|
||||
expect(strictResult.stderr.toLowerCase()).toContain('permission denied')
|
||||
expect(writableResult).toEqual({ stdout: '', stderr: '', code: 0 })
|
||||
expect(vfs.existsSync(`${WORKSPACE}/strict.txt`)).toBe(false)
|
||||
expect(vfs.readFileSync(`${WORKSPACE}/writable.txt`, 'utf8')).toBe('allowed\n')
|
||||
})
|
||||
|
||||
it('carries a command through the real local subprocess service', async () => {
|
||||
|
||||
@@ -0,0 +1,210 @@
|
||||
/** Upstream Chokidar running unchanged through the shipped Worker module loader. */
|
||||
import { existsSync, readFileSync } from 'node:fs'
|
||||
import { createRequire } from 'node:module'
|
||||
import { dirname, join } from 'node:path'
|
||||
import { afterEach, beforeEach, describe, expect, it } from 'vitest'
|
||||
import { lowerModuleSource } from '../../src/compile/transform.ts'
|
||||
import { WorkerModuleLoader } from '../../src/module-system/module-loader.ts'
|
||||
import { createNodeBuiltins } from '../../src/node/builtins.ts'
|
||||
import { MemoryVfs } from '../../src/storage/memory.ts'
|
||||
import { setActiveVfs } from '../../src/storage/active.ts'
|
||||
|
||||
const ROOT = '/dsh/workspace/skills'
|
||||
let vfs: MemoryVfs
|
||||
let chokidar: typeof import('chokidar')
|
||||
const openWatchers: import('chokidar').FSWatcher[] = []
|
||||
|
||||
interface ChokidarFixture {
|
||||
readonly label: string
|
||||
readonly consumerManifest: string
|
||||
readonly chokidarFiles: readonly string[]
|
||||
readonly readdirpFiles: readonly string[]
|
||||
}
|
||||
|
||||
const CHOKIDAR_FIXTURES: readonly ChokidarFixture[] = [
|
||||
{
|
||||
label: 'Chokidar 4 from settings and credentials',
|
||||
consumerManifest: 'packages/settings/settings-file/package.json',
|
||||
chokidarFiles: ['package.json', 'esm/package.json', 'esm/index.js', 'esm/handler.js'],
|
||||
readdirpFiles: ['package.json', 'esm/package.json', 'esm/index.js'],
|
||||
},
|
||||
{
|
||||
label: 'Chokidar 5 from skill-filesystem',
|
||||
consumerManifest: 'packages/skill/skill-filesystem/package.json',
|
||||
chokidarFiles: ['package.json', 'index.js', 'handler.js'],
|
||||
readdirpFiles: ['package.json', 'index.js'],
|
||||
},
|
||||
]
|
||||
|
||||
/** Copy one installed JavaScript package into the VFS exactly as the packer does. */
|
||||
function packageRoot(name: string, entry: string): string {
|
||||
for (let directory = dirname(entry);;) {
|
||||
const manifest = join(directory, 'package.json')
|
||||
if (existsSync(manifest)) {
|
||||
const parsed = JSON.parse(readFileSync(manifest, 'utf8')) as { name?: unknown }
|
||||
if (parsed.name === name) return directory
|
||||
}
|
||||
const parent = dirname(directory)
|
||||
if (parent === directory) throw new Error(`cannot locate package root for ${name}`)
|
||||
directory = parent
|
||||
}
|
||||
}
|
||||
|
||||
/** Copy the package files selected by the packer's import condition. */
|
||||
function mountPackage(name: string, directory: string, files: readonly string[]): void {
|
||||
for (const file of files) {
|
||||
const source = readFileSync(join(directory, file), 'utf8')
|
||||
const path = `/dsh/node_modules/${name}/${file}`
|
||||
vfs.seed(path, file.endsWith('.js') ? lowerModuleSource({ filename: path, source }).code : source)
|
||||
}
|
||||
}
|
||||
|
||||
/** Load one consumer's exact Chokidar and readdirp versions through the Worker loader. */
|
||||
function loadChokidar(fixture: ChokidarFixture): typeof import('chokidar') {
|
||||
const consumerManifest = join(process.cwd(), fixture.consumerManifest)
|
||||
const chokidarEntry = createRequire(consumerManifest).resolve('chokidar')
|
||||
const readdirpEntry = createRequire(chokidarEntry).resolve('readdirp')
|
||||
mountPackage('chokidar', packageRoot('chokidar', chokidarEntry), fixture.chokidarFiles)
|
||||
mountPackage('readdirp', packageRoot('readdirp', readdirpEntry), fixture.readdirpFiles)
|
||||
const loader = new WorkerModuleLoader({ vfs, staticModules: createNodeBuiltins() })
|
||||
return loader.createRequire('/dsh/')('chokidar') as typeof import('chokidar')
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
vfs = new MemoryVfs()
|
||||
setActiveVfs(vfs)
|
||||
vfs.mkdirSync(ROOT, { recursive: true })
|
||||
})
|
||||
|
||||
afterEach(async () => {
|
||||
await Promise.all(openWatchers.splice(0).map(async (watcher) => { await watcher.close() }))
|
||||
})
|
||||
|
||||
/** Await one emitter event while rejecting hangs deterministically. */
|
||||
function onceEvent<T>(watcher: import('chokidar').FSWatcher, event: string): Promise<T> {
|
||||
return new Promise<T>((resolve, reject) => {
|
||||
const timeout = setTimeout(() => { reject(new Error(`timed out waiting for chokidar ${event}`)) }, 2_000)
|
||||
const emitter = watcher as unknown as {
|
||||
once(name: string, listener: (...args: unknown[]) => void): void
|
||||
}
|
||||
emitter.once(event, (...args: unknown[]) => {
|
||||
clearTimeout(timeout)
|
||||
resolve(args[0] as T)
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
/** Let watcher timers and promise-based stats reach a stable point. */
|
||||
async function delay(ms: number): Promise<void> {
|
||||
await new Promise<void>((resolve) => { setTimeout(resolve, ms) })
|
||||
}
|
||||
|
||||
/** Construct one tracked watcher with deterministic event normalization. */
|
||||
function watchPath(path: string, options: import('chokidar').ChokidarOptions = {}): import('chokidar').FSWatcher {
|
||||
const watcher = chokidar.watch(path, {
|
||||
ignoreInitial: true,
|
||||
atomic: false,
|
||||
awaitWriteFinish: false,
|
||||
...options,
|
||||
})
|
||||
openWatchers.push(watcher)
|
||||
return watcher
|
||||
}
|
||||
|
||||
describe.each(CHOKIDAR_FIXTURES)('$label running unchanged', (fixture) => {
|
||||
beforeEach(() => {
|
||||
chokidar = loadChokidar(fixture)
|
||||
})
|
||||
|
||||
it('reaches ready and reports a file lifecycle through fs.watch', async () => {
|
||||
const watcher = watchPath(ROOT, { depth: 1 })
|
||||
await onceEvent(watcher, 'ready')
|
||||
|
||||
const directory = `${ROOT}/sample`
|
||||
const file = `${directory}/SKILL.md`
|
||||
const addDirectory = onceEvent<string>(watcher, 'addDir')
|
||||
const addFile = onceEvent<string>(watcher, 'add')
|
||||
vfs.mkdirSync(directory)
|
||||
vfs.writeFileSync(file, '# sample\n')
|
||||
await expect(addDirectory).resolves.toBe(directory)
|
||||
await expect(addFile).resolves.toBe(file)
|
||||
|
||||
const changed = onceEvent<string>(watcher, 'change')
|
||||
vfs.writeFileSync(file, '# changed\n')
|
||||
await expect(changed).resolves.toBe(file)
|
||||
|
||||
await new Promise((resolve) => { setTimeout(resolve, 10) })
|
||||
const removed = onceEvent<string>(watcher, 'unlink')
|
||||
vfs.rmSync(file)
|
||||
await expect(removed).resolves.toBe(file)
|
||||
})
|
||||
|
||||
it('watches a missing file through its existing parent', async () => {
|
||||
const path = '/dsh/home/settings.yaml'
|
||||
vfs.mkdirSync('/dsh/home', { recursive: true })
|
||||
const watcher = watchPath(path)
|
||||
await onceEvent(watcher, 'ready')
|
||||
|
||||
const added = onceEvent<string>(watcher, 'add')
|
||||
vfs.writeFileSync(path, 'theme: dark\n')
|
||||
await expect(added).resolves.toBe(path)
|
||||
|
||||
const removed = onceEvent<string>(watcher, 'unlink')
|
||||
vfs.rmSync(path)
|
||||
await expect(removed).resolves.toBe(path)
|
||||
})
|
||||
|
||||
it('discovers directory children through watchFile polling mode', async () => {
|
||||
const watcher = watchPath(ROOT, { usePolling: true, interval: 5 })
|
||||
await onceEvent(watcher, 'ready')
|
||||
const path = `${ROOT}/standalone.md`
|
||||
const added = onceEvent<string>(watcher, 'add')
|
||||
vfs.writeFileSync(path, '# standalone\n')
|
||||
await expect(added).resolves.toBe(path)
|
||||
})
|
||||
|
||||
it('normalizes a short unlink/add replacement into one atomic change', async () => {
|
||||
const path = `${ROOT}/atomic.md`
|
||||
vfs.writeFileSync(path, 'before')
|
||||
const watcher = watchPath(path, { atomic: 40 })
|
||||
await onceEvent(watcher, 'ready')
|
||||
const events: string[] = []
|
||||
watcher.on('all', (event) => { events.push(event) })
|
||||
const changed = onceEvent<string>(watcher, 'change')
|
||||
vfs.rmSync(path)
|
||||
await delay(5)
|
||||
vfs.writeFileSync(path, 'after')
|
||||
await expect(changed).resolves.toBe(path)
|
||||
await delay(60)
|
||||
expect(events).toEqual(['change'])
|
||||
})
|
||||
|
||||
it('waits for a write burst to stabilize before publishing one add', async () => {
|
||||
const path = `${ROOT}/settling.md`
|
||||
const watcher = watchPath(ROOT, {
|
||||
awaitWriteFinish: { stabilityThreshold: 30, pollInterval: 5 },
|
||||
})
|
||||
await onceEvent(watcher, 'ready')
|
||||
const events: string[] = []
|
||||
watcher.on('all', (event) => { events.push(event) })
|
||||
const added = onceEvent<string>(watcher, 'add')
|
||||
vfs.writeFileSync(path, 'a')
|
||||
await delay(10)
|
||||
vfs.appendFileSync(path, 'b')
|
||||
await delay(10)
|
||||
vfs.appendFileSync(path, 'c')
|
||||
await expect(added).resolves.toBe(path)
|
||||
expect(events).toEqual(['add'])
|
||||
})
|
||||
|
||||
it('emits nothing after close has reached quiescence', async () => {
|
||||
const watcher = watchPath(ROOT)
|
||||
const events: string[] = []
|
||||
watcher.on('all', (event) => { events.push(event) })
|
||||
await onceEvent(watcher, 'ready')
|
||||
await watcher.close()
|
||||
vfs.writeFileSync(`${ROOT}/after.md`, '# after\n')
|
||||
await Promise.resolve()
|
||||
expect(events).toEqual([])
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,507 @@
|
||||
/** Node differential checks for the Worker filesystem watcher and stream faces. */
|
||||
import {
|
||||
createReadStream as createNodeReadStream,
|
||||
createWriteStream as createNodeWriteStream,
|
||||
mkdtempSync,
|
||||
readFileSync,
|
||||
rmSync,
|
||||
unwatchFile as unwatchNodeFile,
|
||||
watchFile as watchNodeFile,
|
||||
writeFileSync,
|
||||
} from 'node:fs'
|
||||
import { tmpdir } from 'node:os'
|
||||
import { join } from 'node:path'
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
import { MemoryVfs } from '../../src/storage/memory.ts'
|
||||
import { setActiveVfs } from '../../src/storage/active.ts'
|
||||
import * as workerFs from '../../src/node/builtin_modules/implemented/fs.ts'
|
||||
import * as workerFsp from '../../src/node/builtin_modules/implemented/fs/promises.ts'
|
||||
import * as workerStream from '../../src/node/builtin_modules/implemented/stream.ts'
|
||||
|
||||
const VFS_ROOT = '/dsh/watch-stream'
|
||||
const nativeRoots: string[] = []
|
||||
let vfs: MemoryVfs
|
||||
|
||||
beforeEach(() => {
|
||||
vfs = new MemoryVfs()
|
||||
setActiveVfs(vfs)
|
||||
vfs.mkdirSync(VFS_ROOT, { recursive: true })
|
||||
})
|
||||
|
||||
afterEach(() => {
|
||||
for (const root of nativeRoots.splice(0)) rmSync(root, { recursive: true, force: true })
|
||||
vi.restoreAllMocks()
|
||||
})
|
||||
|
||||
/** Await the next callback value with a bounded failure instead of an open watcher. */
|
||||
function nextValue<T>(install: (resolve: (value: T) => void) => void): Promise<T> {
|
||||
return new Promise<T>((resolve, reject) => {
|
||||
const timeout = setTimeout(() => { reject(new Error('timed out waiting for filesystem event')) }, 2_000)
|
||||
install((value) => {
|
||||
clearTimeout(timeout)
|
||||
resolve(value)
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
interface ReadableFileStream {
|
||||
readonly bytesRead: number
|
||||
on(event: string, listener: (...args: unknown[]) => void): ReadableFileStream
|
||||
}
|
||||
|
||||
/** Collect byte chunks and lifecycle events from one read stream implementation. */
|
||||
async function readScenario(create: () => ReadableFileStream): Promise<{
|
||||
chunks: string[]
|
||||
events: string[]
|
||||
bytesRead: number
|
||||
}> {
|
||||
const stream = create()
|
||||
const chunks: string[] = []
|
||||
const events: string[] = []
|
||||
stream.on('open', () => { events.push('open') })
|
||||
stream.on('ready', () => { events.push('ready') })
|
||||
stream.on('data', (chunk: unknown) => {
|
||||
events.push('data')
|
||||
chunks.push(Buffer.from(chunk as Uint8Array).toString('utf8'))
|
||||
})
|
||||
stream.on('end', () => { events.push('end') })
|
||||
await new Promise<void>((resolve, reject) => {
|
||||
stream.on('error', reject)
|
||||
stream.on('close', () => {
|
||||
events.push('close')
|
||||
resolve()
|
||||
})
|
||||
})
|
||||
return { chunks, events, bytesRead: stream.bytesRead }
|
||||
}
|
||||
|
||||
interface WritableFileStream {
|
||||
readonly bytesWritten: number
|
||||
on(event: string, listener: (...args: unknown[]) => void): WritableFileStream
|
||||
write(chunk: string): boolean
|
||||
end(chunk?: string): void
|
||||
}
|
||||
|
||||
/** Write the same chunks and record backpressure plus lifecycle ordering. */
|
||||
async function writeScenario(create: () => WritableFileStream): Promise<{
|
||||
writes: boolean[]
|
||||
events: string[]
|
||||
bytesWritten: number
|
||||
}> {
|
||||
const stream = create()
|
||||
const events: string[] = []
|
||||
for (const event of ['open', 'ready', 'drain', 'finish'] as const) {
|
||||
stream.on(event, () => { events.push(event) })
|
||||
}
|
||||
const writes = [stream.write('ab'), stream.write('cd')]
|
||||
stream.end('ef')
|
||||
await new Promise<void>((resolve, reject) => {
|
||||
stream.on('error', reject)
|
||||
stream.on('close', () => {
|
||||
events.push('close')
|
||||
resolve()
|
||||
})
|
||||
})
|
||||
return { writes, events, bytesWritten: stream.bytesWritten }
|
||||
}
|
||||
|
||||
describe('file streams', () => {
|
||||
it('matches Node chunking, inclusive ranges, and read lifecycle ordering', async () => {
|
||||
const nativeRoot = mkdtempSync(join(tmpdir(), 'dsh-stream-diff-'))
|
||||
nativeRoots.push(nativeRoot)
|
||||
const nativePath = join(nativeRoot, 'input.txt')
|
||||
const workerPath = `${VFS_ROOT}/input.txt`
|
||||
writeFileSync(nativePath, '0123456789')
|
||||
vfs.writeFileSync(workerPath, '0123456789')
|
||||
|
||||
const native = await readScenario(() => createNodeReadStream(nativePath, { start: 2, end: 7, highWaterMark: 2 }))
|
||||
const worker = await readScenario(() => workerFs.createReadStream(workerPath, { start: 2, end: 7, highWaterMark: 2 }))
|
||||
expect(worker).toEqual(native)
|
||||
})
|
||||
|
||||
it('matches Node write backpressure, lifecycle ordering, and byte accounting', async () => {
|
||||
const nativeRoot = mkdtempSync(join(tmpdir(), 'dsh-stream-diff-'))
|
||||
nativeRoots.push(nativeRoot)
|
||||
const nativePath = join(nativeRoot, 'output.txt')
|
||||
const workerPath = `${VFS_ROOT}/output.txt`
|
||||
|
||||
const native = await writeScenario(() => createNodeWriteStream(nativePath, { highWaterMark: 2 }))
|
||||
const worker = await writeScenario(() => workerFs.createWriteStream(workerPath, { highWaterMark: 2 }))
|
||||
expect(worker).toEqual(native)
|
||||
expect(workerFs.readFileSync(workerPath, 'utf8')).toBe('abcdef')
|
||||
})
|
||||
|
||||
it('uses the maintained stream implementation for backpressure and async iteration', async () => {
|
||||
const values: string[] = []
|
||||
for await (const value of workerStream.Readable.from(['one', 'two'])) values.push(String(value))
|
||||
expect(values).toEqual(['one', 'two'])
|
||||
expect(typeof workerStream.pipeline).toBe('function')
|
||||
expect(typeof workerStream.finished).toBe('function')
|
||||
expect(workerStream.getDefaultHighWaterMark(false)).toBe(64 * 1024)
|
||||
expect(workerStream.default._isArrayBufferView(new Uint8Array())).toBe(true)
|
||||
})
|
||||
|
||||
it('matches Node file-stream defaults and abort error identity', async () => {
|
||||
const nativeRoot = mkdtempSync(join(tmpdir(), 'dsh-stream-diff-'))
|
||||
nativeRoots.push(nativeRoot)
|
||||
const nativePath = join(nativeRoot, 'input.txt')
|
||||
const workerPath = `${VFS_ROOT}/input.txt`
|
||||
writeFileSync(nativePath, 'content')
|
||||
vfs.writeFileSync(workerPath, 'content')
|
||||
const nativeRead = createNodeReadStream(nativePath)
|
||||
const nativeWrite = createNodeWriteStream(join(nativeRoot, 'output.txt'))
|
||||
const workerRead = workerFs.createReadStream(workerPath)
|
||||
const workerWrite = workerFs.createWriteStream(`${VFS_ROOT}/output.txt`)
|
||||
expect([workerRead.readableHighWaterMark, workerWrite.writableHighWaterMark]).toEqual([
|
||||
nativeRead.readableHighWaterMark,
|
||||
nativeWrite.writableHighWaterMark,
|
||||
])
|
||||
interface CloseableStream {
|
||||
once(event: string, listener: (...args: unknown[]) => void): unknown
|
||||
destroy(): unknown
|
||||
}
|
||||
const streams = [nativeRead, nativeWrite, workerRead, workerWrite] as unknown as CloseableStream[]
|
||||
const closed = streams.map(stream => new Promise<void>((resolve) => {
|
||||
stream.once('error', () => {})
|
||||
stream.once('close', () => { resolve() })
|
||||
}))
|
||||
for (const stream of streams) stream.destroy()
|
||||
await Promise.all(closed)
|
||||
|
||||
const controller = new AbortController()
|
||||
controller.abort(new Error('stop'))
|
||||
const aborted = workerFs.createReadStream(workerPath, { signal: controller.signal })
|
||||
const error = await nextValue<Error & { code?: string }>((resolve) => { aborted.once('error', resolve) })
|
||||
expect(error).toMatchObject({ name: 'AbortError', code: 'ABORT_ERR' })
|
||||
})
|
||||
|
||||
it('matches Node positional overwrite and missing-file failure', async () => {
|
||||
const nativeRoot = mkdtempSync(join(tmpdir(), 'dsh-stream-diff-'))
|
||||
nativeRoots.push(nativeRoot)
|
||||
const nativePath = join(nativeRoot, 'position.txt')
|
||||
const workerPath = `${VFS_ROOT}/position.txt`
|
||||
writeFileSync(nativePath, 'abcdef')
|
||||
vfs.writeFileSync(workerPath, 'abcdef')
|
||||
|
||||
const writeAt = async (stream: WritableFileStream): Promise<void> => {
|
||||
stream.end('XY')
|
||||
await new Promise<void>((resolve) => { stream.on('close', () => { resolve() }) })
|
||||
}
|
||||
await writeAt(createNodeWriteStream(nativePath, { flags: 'r+', start: 2 }))
|
||||
await writeAt(workerFs.createWriteStream(workerPath, { flags: 'r+', start: 2 }))
|
||||
expect(workerFs.readFileSync(workerPath, 'utf8')).toBe(readFileSync(nativePath, 'utf8'))
|
||||
|
||||
const missing = workerFs.createReadStream(`${VFS_ROOT}/missing.txt`)
|
||||
const events: string[] = []
|
||||
missing.on('error', () => { events.push('error') })
|
||||
await new Promise<void>((resolve) => {
|
||||
missing.on('close', () => {
|
||||
events.push('close')
|
||||
resolve()
|
||||
})
|
||||
})
|
||||
expect(events).toEqual(['error', 'close'])
|
||||
})
|
||||
})
|
||||
|
||||
interface StatTransition {
|
||||
currentExists: boolean
|
||||
previousExists: boolean
|
||||
currentSize: number
|
||||
previousSize: number
|
||||
currentOtherKinds: boolean[]
|
||||
}
|
||||
|
||||
/** Observe missing, creation, rewrite, and deletion through one watchFile implementation. */
|
||||
async function watchFileScenario(
|
||||
path: string,
|
||||
watchFile: typeof watchNodeFile,
|
||||
unwatchFile: typeof unwatchNodeFile,
|
||||
write: (text: string) => void,
|
||||
remove: () => void,
|
||||
): Promise<StatTransition[]> {
|
||||
const waiting: Array<(value: StatTransition) => void> = []
|
||||
const queued: StatTransition[] = []
|
||||
const listener = (current: import('node:fs').Stats, previous: import('node:fs').Stats): void => {
|
||||
const transition = {
|
||||
currentExists: current.isFile(),
|
||||
previousExists: previous.isFile(),
|
||||
currentSize: current.size,
|
||||
previousSize: previous.size,
|
||||
currentOtherKinds: [
|
||||
current.isDirectory(), current.isSymbolicLink(), current.isFIFO(),
|
||||
current.isSocket(), current.isBlockDevice(), current.isCharacterDevice(),
|
||||
],
|
||||
}
|
||||
const resolve = waiting.shift()
|
||||
if (resolve === undefined) queued.push(transition)
|
||||
else resolve(transition)
|
||||
}
|
||||
const next = async (): Promise<StatTransition> => {
|
||||
const queuedValue = queued.shift()
|
||||
if (queuedValue !== undefined) return queuedValue
|
||||
return await nextValue((resolve) => { waiting.push(resolve) })
|
||||
}
|
||||
watchFile(path, { interval: 10, persistent: false }, listener)
|
||||
try {
|
||||
const missing = await next()
|
||||
write('a')
|
||||
const created = await next()
|
||||
write('longer')
|
||||
const changed = await next()
|
||||
remove()
|
||||
const removed = await next()
|
||||
return [missing, created, changed, removed]
|
||||
} finally {
|
||||
unwatchFile(path, listener)
|
||||
}
|
||||
}
|
||||
|
||||
describe('watchers', () => {
|
||||
it('matches Node watchFile state transitions for a missing and recreated file', async () => {
|
||||
const nativeRoot = mkdtempSync(join(tmpdir(), 'dsh-watch-diff-'))
|
||||
nativeRoots.push(nativeRoot)
|
||||
const nativePath = join(nativeRoot, 'watched.txt')
|
||||
const workerPath = `${VFS_ROOT}/watched.txt`
|
||||
const native = await watchFileScenario(
|
||||
nativePath,
|
||||
watchNodeFile,
|
||||
unwatchNodeFile,
|
||||
(text) => { writeFileSync(nativePath, text) },
|
||||
() => { rmSync(nativePath) },
|
||||
)
|
||||
const worker = await watchFileScenario(
|
||||
workerPath,
|
||||
workerFs.watchFile as unknown as typeof watchNodeFile,
|
||||
workerFs.unwatchFile as unknown as typeof unwatchNodeFile,
|
||||
(text) => { vfs.writeFileSync(workerPath, text) },
|
||||
() => { vfs.rmSync(workerPath) },
|
||||
)
|
||||
expect(worker).toEqual(native)
|
||||
})
|
||||
|
||||
it('shares one StatWatcher and removes only the named listener', async () => {
|
||||
const path = `${VFS_ROOT}/shared.txt`
|
||||
vfs.writeFileSync(path, 'a')
|
||||
const firstEvents: number[] = []
|
||||
const secondEvents: number[] = []
|
||||
const first = (): void => { firstEvents.push(1) }
|
||||
const second = (): void => { secondEvents.push(1) }
|
||||
const firstWatcher = workerFs.watchFile(path, { interval: 1, persistent: false }, first)
|
||||
const secondWatcher = workerFs.watchFile(path, { interval: 1, persistent: false }, second)
|
||||
expect(secondWatcher).toBe(firstWatcher)
|
||||
workerFs.unwatchFile(path, first)
|
||||
vfs.writeFileSync(path, 'bb')
|
||||
await nextValue<undefined>((resolve) => {
|
||||
const poll = setInterval(() => {
|
||||
if (secondEvents.length === 0) return
|
||||
clearInterval(poll)
|
||||
resolve(undefined)
|
||||
}, 1)
|
||||
})
|
||||
expect(firstEvents).toEqual([])
|
||||
expect(secondEvents).toEqual([1])
|
||||
workerFs.unwatchFile(path)
|
||||
})
|
||||
|
||||
it('reports direct and recursive names, then reaches quiescence on close', async () => {
|
||||
const root = `${VFS_ROOT}/tree`
|
||||
vfs.mkdirSync(`${root}/nested`, { recursive: true })
|
||||
const directEvents: Array<[string, string]> = []
|
||||
const recursiveEvents: Array<[string, string]> = []
|
||||
const direct = workerFs.watch(root, (_event, _filename) => {})
|
||||
direct.on('change', (event, filename) => { directEvents.push([String(event), String(filename)]) })
|
||||
const recursive = workerFs.watch(root, { recursive: true }, (event, filename) => {
|
||||
recursiveEvents.push([event, String(filename)])
|
||||
})
|
||||
vfs.writeFileSync(`${root}/top.txt`, 'top')
|
||||
vfs.writeFileSync(`${root}/nested/deep.txt`, 'deep')
|
||||
await Promise.resolve()
|
||||
expect(directEvents).toEqual([['rename', 'top.txt']])
|
||||
expect(recursiveEvents).toEqual([
|
||||
['rename', 'top.txt'],
|
||||
['rename', 'nested/deep.txt'],
|
||||
])
|
||||
direct.close()
|
||||
recursive.close()
|
||||
vfs.writeFileSync(`${root}/after.txt`, 'after')
|
||||
await Promise.resolve()
|
||||
expect(directEvents).toHaveLength(1)
|
||||
expect(recursiveEvents).toHaveLength(2)
|
||||
})
|
||||
|
||||
it('supports Buffer filenames, file targets, abort closure, and ref state', async () => {
|
||||
const path = `${VFS_ROOT}/encoded.txt`
|
||||
vfs.writeFileSync(path, 'before')
|
||||
const controller = new AbortController()
|
||||
const event = nextValue<[string, Buffer]>((resolve) => {
|
||||
const watcher = workerFs.watch(
|
||||
new TextEncoder().encode(path),
|
||||
{ encoding: 'buffer', persistent: false, signal: controller.signal },
|
||||
(eventType, filename) => { resolve([eventType, filename as Buffer]) },
|
||||
)
|
||||
expect(watcher.hasRef()).toBe(false)
|
||||
expect(watcher.ref().hasRef()).toBe(true)
|
||||
expect(watcher.unref().hasRef()).toBe(false)
|
||||
})
|
||||
vfs.writeFileSync(path, 'after')
|
||||
const [eventType, filename] = await event
|
||||
expect(eventType).toBe('change')
|
||||
expect(Buffer.isBuffer(filename)).toBe(true)
|
||||
expect(filename.toString()).toBe('encoded.txt')
|
||||
|
||||
const watcher = workerFs.watch(path, { signal: controller.signal })
|
||||
let closes = 0
|
||||
const closed = nextValue<undefined>((resolve) => {
|
||||
watcher.on('close', () => {
|
||||
closes += 1
|
||||
resolve(undefined)
|
||||
})
|
||||
})
|
||||
controller.abort(new Error('stop'))
|
||||
await closed
|
||||
watcher.close()
|
||||
await Promise.resolve()
|
||||
expect(closes).toBe(1)
|
||||
})
|
||||
|
||||
it('supports the string encoding overload and suppresses queued delivery after close', async () => {
|
||||
const encoded = nextValue<Buffer>((resolve) => {
|
||||
const watcher = workerFs.watch(VFS_ROOT, 'buffer', (_eventType, filename) => {
|
||||
watcher.close()
|
||||
resolve(filename as Buffer)
|
||||
})
|
||||
})
|
||||
vfs.writeFileSync(`${VFS_ROOT}/buffer-name.txt`, 'x')
|
||||
await expect(encoded).resolves.toEqual(Buffer.from('buffer-name.txt'))
|
||||
|
||||
let calls = 0
|
||||
const closed = workerFs.watch(VFS_ROOT, () => { calls += 1 })
|
||||
vfs.writeFileSync(`${VFS_ROOT}/queued.txt`, 'x')
|
||||
closed.close()
|
||||
await Promise.resolve()
|
||||
expect(calls).toBe(0)
|
||||
})
|
||||
|
||||
it('reports removal of an ancestor to a watched file', async () => {
|
||||
const directory = `${VFS_ROOT}/removed-parent`
|
||||
const path = `${directory}/file.txt`
|
||||
vfs.mkdirSync(directory)
|
||||
vfs.writeFileSync(path, 'x')
|
||||
const event = nextValue<[string, string]>((resolve) => {
|
||||
const watcher = workerFs.watch(path, (eventType, filename) => {
|
||||
watcher.close()
|
||||
resolve([eventType, String(filename)])
|
||||
})
|
||||
})
|
||||
vfs.rmSync(directory, { recursive: true })
|
||||
await expect(event).resolves.toEqual(['rename', 'file.txt'])
|
||||
})
|
||||
|
||||
it('rejects an already-aborted callback watcher without retaining a subscription', () => {
|
||||
const controller = new AbortController()
|
||||
const reason = new Error('already stopped')
|
||||
controller.abort(reason)
|
||||
try {
|
||||
workerFs.watch(VFS_ROOT, { signal: controller.signal })
|
||||
throw new Error('watch unexpectedly opened')
|
||||
} catch (error) {
|
||||
expect(error).toMatchObject({ name: 'AbortError', code: 'ABORT_ERR', cause: reason })
|
||||
}
|
||||
expect(() => { vfs.writeFileSync(`${VFS_ROOT}/after-abort.txt`, 'x') }).not.toThrow()
|
||||
})
|
||||
|
||||
it('reports an atomic replacement destination as rename even when it existed', async () => {
|
||||
const target = `${VFS_ROOT}/target.txt`
|
||||
const replacement = `${VFS_ROOT}/replacement.txt`
|
||||
vfs.writeFileSync(target, 'old')
|
||||
vfs.writeFileSync(replacement, 'new')
|
||||
const event = nextValue<[string, string]>((resolve) => {
|
||||
const watcher = workerFs.watch(VFS_ROOT, (eventType, filename) => {
|
||||
if (String(filename) !== 'target.txt') return
|
||||
watcher.close()
|
||||
resolve([eventType, String(filename)])
|
||||
})
|
||||
})
|
||||
vfs.renameSync(replacement, target)
|
||||
await expect(event).resolves.toEqual(['rename', 'target.txt'])
|
||||
})
|
||||
|
||||
it('supports BigInt watchFile state, default options, and idempotent stop', async () => {
|
||||
const path = `${VFS_ROOT}/bigint.txt`
|
||||
const states = nextValue<[bigint, bigint]>((resolve) => {
|
||||
const watcher = workerFs.watchFile(new URL(`file://${path}`), { bigint: true, interval: 1 }, (current, previous) => {
|
||||
resolve([current.size as bigint, previous.size as bigint])
|
||||
})
|
||||
expect(watcher.hasRef()).toBe(true)
|
||||
expect(watcher.unref().hasRef()).toBe(false)
|
||||
expect(watcher.ref().hasRef()).toBe(true)
|
||||
})
|
||||
vfs.writeFileSync(path, 'big')
|
||||
await expect(states).resolves.toEqual([3n, 0n])
|
||||
workerFs.unwatchFile(path)
|
||||
workerFs.unwatchFile(path)
|
||||
|
||||
vfs.writeFileSync(`${VFS_ROOT}/default.txt`, 'x')
|
||||
const listener = (): void => {}
|
||||
const defaultWatcher = workerFs.watchFile(`${VFS_ROOT}/default.txt`, listener)
|
||||
expect(defaultWatcher.hasRef()).toBe(true)
|
||||
defaultWatcher.close()
|
||||
defaultWatcher.close()
|
||||
expect(() => workerFs.watchFile(`${VFS_ROOT}/default.txt`, {})).toThrow(/listener/)
|
||||
|
||||
let cancelledCalls = 0
|
||||
const cancelled = workerFs.watchFile(`${VFS_ROOT}/never-created`, { interval: 1 }, () => { cancelledCalls += 1 })
|
||||
cancelled.close()
|
||||
cancelled.close()
|
||||
await new Promise<void>((resolve) => { setTimeout(resolve, 5) })
|
||||
expect(cancelledCalls).toBe(0)
|
||||
})
|
||||
|
||||
it('propagates non-absence stat failures from watchFile', () => {
|
||||
const failure = Object.assign(new Error('denied'), { code: 'EACCES' })
|
||||
vi.spyOn(vfs, 'statSync').mockImplementationOnce(() => { throw failure })
|
||||
expect(() => workerFs.watchFile(`${VFS_ROOT}/denied`, () => {})).toThrow(failure)
|
||||
})
|
||||
|
||||
it('exposes promise watch as an abortable async iterator', async () => {
|
||||
const controller = new AbortController()
|
||||
const iterator = workerFsp.watch(VFS_ROOT, { signal: controller.signal })[Symbol.asyncIterator]()
|
||||
const event = iterator.next()
|
||||
vfs.writeFileSync(`${VFS_ROOT}/async.txt`, 'x')
|
||||
await expect(event).resolves.toEqual({ done: false, value: { eventType: 'rename', filename: 'async.txt' } })
|
||||
controller.abort()
|
||||
await expect(iterator.next()).rejects.toMatchObject({ name: 'AbortError', code: 'ABORT_ERR' })
|
||||
})
|
||||
|
||||
it('lets promise-watch return interrupt a pending next call', async () => {
|
||||
const iterator = workerFsp.watch(VFS_ROOT)[Symbol.asyncIterator]()
|
||||
const pending = iterator.next()
|
||||
await expect(iterator.return?.()).resolves.toEqual({ done: true, value: undefined })
|
||||
await expect(pending).resolves.toEqual({ done: true, value: undefined })
|
||||
vfs.writeFileSync(`${VFS_ROOT}/after-return.txt`, 'x')
|
||||
await expect(iterator.next()).resolves.toEqual({ done: true, value: undefined })
|
||||
})
|
||||
|
||||
it('propagates promise-watch startup and throw failures', async () => {
|
||||
const missing = workerFsp.watch(`${VFS_ROOT}/missing`)[Symbol.asyncIterator]()
|
||||
await expect(missing.next()).rejects.toMatchObject({ code: 'ENOENT' })
|
||||
|
||||
const iterator = workerFsp.watch(VFS_ROOT)[Symbol.asyncIterator]()
|
||||
const reason = { reason: 'caller stopped iteration' }
|
||||
if (iterator.throw === undefined) throw new Error('watch iterator has no throw method')
|
||||
await expect(iterator.throw(reason)).rejects.toBe(reason)
|
||||
await expect(iterator.next()).resolves.toEqual({ done: true, value: undefined })
|
||||
})
|
||||
|
||||
it('queues promise-watch events when no next call is waiting', async () => {
|
||||
const iterator = workerFsp.watch(VFS_ROOT)[Symbol.asyncIterator]()
|
||||
const first = iterator.next()
|
||||
vfs.writeFileSync(`${VFS_ROOT}/one.txt`, 'one')
|
||||
vfs.writeFileSync(`${VFS_ROOT}/two.txt`, 'two')
|
||||
await expect(first).resolves.toEqual({ done: false, value: { eventType: 'rename', filename: 'one.txt' } })
|
||||
await expect(iterator.next()).resolves.toEqual({ done: false, value: { eventType: 'rename', filename: 'two.txt' } })
|
||||
await expect(iterator.return?.()).resolves.toEqual({ done: true, value: undefined })
|
||||
await expect(iterator.return?.()).resolves.toEqual({ done: true, value: undefined })
|
||||
})
|
||||
})
|
||||
@@ -12,9 +12,17 @@ import { MemoryVfs } from '@deepseek-ai/dsh-experimental-webworker-runtime/src/s
|
||||
import { setActiveVfs } from '@deepseek-ai/dsh-experimental-webworker-runtime/src/storage/active.ts'
|
||||
import * as fs from '@deepseek-ai/dsh-experimental-webworker-runtime/src/node/builtin_modules/implemented/fs.ts'
|
||||
import * as fsp from '@deepseek-ai/dsh-experimental-webworker-runtime/src/node/builtin_modules/implemented/fs/promises.ts'
|
||||
import type { VfsBigIntStats, VfsStats } from '@deepseek-ai/dsh-experimental-webworker-runtime/src/storage/types.ts'
|
||||
import type { VfsBigIntStats, VfsMutationSink, VfsStats } from '@deepseek-ai/dsh-experimental-webworker-runtime/src/storage/types.ts'
|
||||
|
||||
const vfs = new MemoryVfs()
|
||||
let flushes = 0
|
||||
const sink: VfsMutationSink = {
|
||||
record: () => {},
|
||||
flush: () => {
|
||||
flushes += 1
|
||||
return Promise.resolve()
|
||||
},
|
||||
}
|
||||
const vfs = new MemoryVfs({ sink })
|
||||
setActiveVfs(vfs)
|
||||
|
||||
// Identity precondition: the bridge must read this exact mounted VFS; successful
|
||||
@@ -76,9 +84,6 @@ throws('readFileSync missing', () => fs.readFileSync('/dsh/missing'), 'ENOENT')
|
||||
throws('statSync missing', () => fs.statSync('/dsh/missing'), 'ENOENT')
|
||||
throws('accessSync missing', () =>{ fs.accessSync('/dsh/missing') }, 'ENOENT')
|
||||
throws('readdirSync missing', () => fs.readdirSync('/dsh/missing'), 'ENOENT')
|
||||
throws('watchFile is loud', () => fs.watchFile('/dsh/config/cordis.yml'), 'not implemented')
|
||||
throws('createReadStream is loud', () => fs.createReadStream('/dsh/config/cordis.yml'), 'not implemented')
|
||||
|
||||
const appendFd = fs.openSync('/dsh/log.jsonl', 'a')
|
||||
fs.writeSync(appendFd, '{"a":1}\n')
|
||||
fs.writeSync(appendFd, '{"a":2}\n')
|
||||
@@ -112,6 +117,7 @@ const appendHandle = await fsp.open('/dsh/log-handle.jsonl', 'a')
|
||||
check('append handle sees the existing size', (await appendHandle.stat()).size, 7)
|
||||
await appendHandle.writeFile('batch-1\n')
|
||||
await appendHandle.sync()
|
||||
check('handle.sync flushes the active VFS', flushes, 1)
|
||||
await appendHandle.close()
|
||||
const secondHandle = await fsp.open('/dsh/log-handle.jsonl', 'a')
|
||||
await secondHandle.writeFile('batch-2\n')
|
||||
|
||||
@@ -16,17 +16,14 @@ import { notAvailableError, notImplementedFail } from '../../src/node/notImpleme
|
||||
import * as childProcess from '../../src/node/builtin_modules/implemented/child_process.ts'
|
||||
import * as net from '../../src/node/builtin_modules/mock/net.ts'
|
||||
import * as sqlite from '../../src/node/builtin_modules/mock/sqlite.ts'
|
||||
import * as stream from '../../src/node/builtin_modules/mock/stream.ts'
|
||||
import * as stream from '../../src/node/builtin_modules/implemented/stream.ts'
|
||||
import * as vm from '../../src/node/builtin_modules/mock/vm.ts'
|
||||
import * as workerThreads from '../../src/node/builtin_modules/mock/worker_threads.ts'
|
||||
import * as chokidar from '../../src/node/external_packages/chokidar.ts'
|
||||
import * as landlock from '../../src/node/external_packages/node-addon-landlock-run.ts'
|
||||
import * as nodePty from '../../src/node/external_packages/node-pty.ts'
|
||||
import * as piAi from '../../src/node/external_packages/pi-ai.ts'
|
||||
import * as ripgrep from '../../src/node/external_packages/ripgrep.ts'
|
||||
import * as ws from '../../src/node/external_packages/ws.ts'
|
||||
import { REPLACED_EXTERNAL_PACKAGES } from '../../src/node/external_packages/replaced-externals.ts'
|
||||
import * as fs from '../../src/node/builtin_modules/implemented/fs.ts'
|
||||
import * as os from '../../src/node/builtin_modules/implemented/os.ts'
|
||||
import * as perfHooks from '../../src/node/builtin_modules/implemented/perf_hooks.ts'
|
||||
import { DSH_HOME, DSH_TMP } from '../../src/storage/paths.ts'
|
||||
@@ -43,9 +40,7 @@ const CALLED: [string, Record<string, unknown>, readonly string[]][] = [
|
||||
// The rest of `node:child_process` runs commands (see child-process.spec.ts);
|
||||
// these three need a real process, so they stay refusals.
|
||||
['node:child_process', childProcess, ['execFileSync', 'execSync', 'fork']],
|
||||
['node:stream', stream, ['Readable', 'Writable', 'Duplex', 'Transform', 'PassThrough', 'pipeline', 'finished']],
|
||||
['node-pty', nodePty, ['spawn', 'open']],
|
||||
['@deepseek-ai/node-addon-landlock-run', landlock, ['probe']],
|
||||
['@deepseek-ai/pi-ai', piAi, [
|
||||
'createProvider', 'createModels', 'openAICompletionsApi', 'openAIResponsesApi', 'anthropicMessagesApi',
|
||||
'isContextOverflow', 'getSupportedThinkingLevels',
|
||||
@@ -95,7 +90,7 @@ describe('not-implemented stubs', () => {
|
||||
}
|
||||
|
||||
it('keeps the CommonJS interop marker and a default export on every replaced module', () => {
|
||||
for (const namespace of [net, sqlite, vm, workerThreads, childProcess, stream, chokidar, ws, nodePty, piAi, os, perfHooks]) {
|
||||
for (const namespace of [net, sqlite, vm, workerThreads, childProcess, stream, ws, nodePty, piAi, os, perfHooks]) {
|
||||
const holder = namespace as { __esModule?: unknown; default?: unknown }
|
||||
expect(holder.__esModule).toBe(true)
|
||||
expect(holder.default).toBeDefined()
|
||||
@@ -104,19 +99,6 @@ describe('not-implemented stubs', () => {
|
||||
})
|
||||
|
||||
describe('constructible-but-inert fakes', () => {
|
||||
// These two are constructed in `[Service.init]` bodies and field initializers,
|
||||
// so construction must succeed; only the members that would move bytes refuse.
|
||||
it('chokidar watches nothing and says so by never emitting', async () => {
|
||||
const watcher = chokidar.watch()
|
||||
expect(watcher).toBeInstanceOf(chokidar.FSWatcher)
|
||||
expect(watcher.on()).toBe(watcher)
|
||||
expect(watcher.once()).toBe(watcher)
|
||||
expect(watcher.add()).toBe(watcher)
|
||||
expect(watcher.unwatch()).toBe(watcher)
|
||||
expect(watcher.getWatched()).toEqual({})
|
||||
await expect(watcher.close()).resolves.toBeUndefined()
|
||||
})
|
||||
|
||||
it('a ws server constructs, accepts listeners, and refuses to carry an upgrade', () => {
|
||||
quiet()
|
||||
expect(ws.Server).toBe(ws.WebSocketServer)
|
||||
@@ -133,16 +115,14 @@ describe('constructible-but-inert fakes', () => {
|
||||
|
||||
describe('replaced external packages', () => {
|
||||
it('lists the packages the loader serves from the bundle', () => {
|
||||
expect(REPLACED_EXTERNAL_PACKAGES).toContain('chokidar')
|
||||
expect(REPLACED_EXTERNAL_PACKAGES).not.toContain('chokidar')
|
||||
expect(REPLACED_EXTERNAL_PACKAGES).not.toContain('@deepseek-ai/node-addon-landlock-run')
|
||||
expect(REPLACED_EXTERNAL_PACKAGES).toContain('ws')
|
||||
})
|
||||
|
||||
it('answers the values callers read without invoking anything', () => {
|
||||
// The ripgrep binary path and the landlock launcher are read as data by
|
||||
// consumers that then fail on their own terms.
|
||||
// The ripgrep binary path is read as data by its consumer.
|
||||
expect(typeof ripgrep.rgPath).toBe('string')
|
||||
expect(typeof landlock.LAUNCHER_BIN).toBe('string')
|
||||
expect(typeof landlock.LAUNCHER_FAILURE_EXIT).toBe('number')
|
||||
})
|
||||
})
|
||||
|
||||
@@ -190,17 +170,3 @@ describe('node:perf_hooks', () => {
|
||||
expect(perfHooks.performance.now()).toBeGreaterThan(0)
|
||||
})
|
||||
})
|
||||
|
||||
describe('watching', () => {
|
||||
// Watching stays a loud refusal because `skill-filesystem` AWAITS watcher
|
||||
// progress rather than merely registering a listener; an inert watcher left
|
||||
// its discovery hanging. `fs.ts` records the experiment and the mechanism.
|
||||
it('refuses, naming the member, so an awaiting caller fails fast', () => {
|
||||
quiet()
|
||||
expect(() => fs.watchFile('/dsh/config/cordis.yml')).toThrow(/watchFile is not implemented in the worker host/)
|
||||
})
|
||||
|
||||
it('accepts the unconditional teardown call, since nothing was watched', () => {
|
||||
expect(() => { fs.unwatchFile() }).not.toThrow()
|
||||
})
|
||||
})
|
||||
|
||||
@@ -0,0 +1,98 @@
|
||||
/** The unchanged sandbox-local → bash-sandbox → subprocess stack over the Worker Node layer. */
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
|
||||
import { Context } from '@deepseek-ai/cordis'
|
||||
import { SandboxBashExecutor } from '@deepseek-ai/dsh-bash-sandbox'
|
||||
import LocalSandboxProvider from '@deepseek-ai/dsh-sandbox-local'
|
||||
import { SandboxPolicyService } from '@deepseek-ai/dsh-sandbox-policy'
|
||||
import LocalSubprocessRuntime from '@deepseek-ai/dsh-subprocess-local'
|
||||
import { MemoryVfs } from '../../src/storage/memory.ts'
|
||||
import { setActiveVfs } from '../../src/storage/active.ts'
|
||||
import { processAlive, signalProcess } from '../../src/node/process-table.ts'
|
||||
|
||||
vi.mock('node:child_process', async () => await import('../../src/node/builtin_modules/implemented/child_process.ts'))
|
||||
|
||||
const WORKSPACE = '/dsh/workspace'
|
||||
const OUTSIDE = '/dsh/home'
|
||||
let vfs: MemoryVfs
|
||||
const contexts: Context[] = []
|
||||
|
||||
beforeEach(() => {
|
||||
vfs = new MemoryVfs()
|
||||
setActiveVfs(vfs)
|
||||
vfs.mkdirSync(WORKSPACE, { recursive: true })
|
||||
vfs.mkdirSync(OUTSIDE, { recursive: true })
|
||||
vfs.mkdirSync('/dsh/tmp', { recursive: true })
|
||||
vi.spyOn(process, 'kill').mockImplementation((pid: number, signal?: string | number): true => {
|
||||
if (signal === 0) {
|
||||
if (processAlive(pid)) return true
|
||||
const error = new Error('kill ESRCH') as NodeJS.ErrnoException
|
||||
error.code = 'ESRCH'
|
||||
throw error
|
||||
}
|
||||
signalProcess(pid, (signal ?? 'SIGTERM') as NodeJS.Signals)
|
||||
return true
|
||||
})
|
||||
})
|
||||
|
||||
afterEach(async () => {
|
||||
await Promise.all(contexts.splice(0).map(async (ctx) => { await ctx.fiber.dispose() }))
|
||||
vi.restoreAllMocks()
|
||||
})
|
||||
|
||||
/** Boot the production providers while only their platform primitives are replaced. */
|
||||
async function setup(mode: 'read-only' | 'workspace-write' | 'danger-full-access'): Promise<SandboxBashExecutor> {
|
||||
const ctx = new Context()
|
||||
contexts.push(ctx)
|
||||
await ctx.plugin(LocalSandboxProvider)
|
||||
await ctx.plugin(SandboxPolicyService, { mode, workspaceRoot: WORKSPACE })
|
||||
await ctx.plugin(LocalSubprocessRuntime)
|
||||
await ctx.plugin(SandboxBashExecutor, { cwd: WORKSPACE })
|
||||
return ctx.shell as SandboxBashExecutor
|
||||
}
|
||||
|
||||
describe('Worker Landlock through the production sandbox stack', () => {
|
||||
it('allows workspace and temp writes while classifying an outside write as denied', async () => {
|
||||
const bash = await setup('workspace-write')
|
||||
const allowed = await bash.run(bash.resolve({
|
||||
command: `echo workspace > ${WORKSPACE}/allowed.txt; echo temp > /tmp/allowed.txt`,
|
||||
}))
|
||||
expect(allowed.sandbox).toEqual({ mode: 'workspace-write', denied: false, enforcement: 'full' })
|
||||
expect(vfs.readFileSync(`${WORKSPACE}/allowed.txt`, 'utf8')).toBe('workspace\n')
|
||||
expect(vfs.readFileSync('/dsh/tmp/allowed.txt', 'utf8')).toBe('temp\n')
|
||||
|
||||
const denied = await bash.run(bash.resolve({ command: `echo denied > ${OUTSIDE}/denied.txt` }))
|
||||
expect(denied.exitCode).toBe(1)
|
||||
expect(denied.sandbox).toEqual({ mode: 'workspace-write', denied: true, enforcement: 'full' })
|
||||
expect(vfs.existsSync(`${OUTSIDE}/denied.txt`)).toBe(false)
|
||||
})
|
||||
|
||||
it('keeps read-only confined and danger-full-access unwrapped', async () => {
|
||||
const readOnly = await setup('read-only')
|
||||
const strict = await readOnly.run(readOnly.resolve({
|
||||
command: `echo discarded > /dev/null; echo denied > ${WORKSPACE}/strict.txt`,
|
||||
}))
|
||||
expect(strict.sandbox).toEqual({ mode: 'read-only', denied: true, enforcement: 'full' })
|
||||
expect(vfs.existsSync(`${WORKSPACE}/strict.txt`)).toBe(false)
|
||||
|
||||
const unrestricted = await setup('danger-full-access')
|
||||
const result = await unrestricted.run(unrestricted.resolve({ command: `echo allowed > ${OUTSIDE}/full.txt` }))
|
||||
expect(result.sandbox).toEqual({ mode: 'danger-full-access', denied: false })
|
||||
expect(vfs.readFileSync(`${OUTSIDE}/full.txt`, 'utf8')).toBe('allowed\n')
|
||||
})
|
||||
|
||||
it('does not leak a concurrent command policy into another process', async () => {
|
||||
const bash = await setup('read-only')
|
||||
const strict = bash.run(bash.resolve({
|
||||
command: `sleep 0.02; echo denied > ${WORKSPACE}/strict.txt`,
|
||||
}))
|
||||
const writable = bash.run(bash.resolve({
|
||||
command: `echo allowed > ${WORKSPACE}/writable.txt`,
|
||||
sandboxPolicy: { mode: 'workspace-write', workspaceRoot: WORKSPACE },
|
||||
}))
|
||||
const [strictResult, writableResult] = await Promise.all([strict, writable])
|
||||
expect(strictResult.sandbox).toEqual({ mode: 'read-only', denied: true, enforcement: 'full' })
|
||||
expect(writableResult.sandbox).toEqual({ mode: 'workspace-write', denied: false, enforcement: 'full' })
|
||||
expect(vfs.existsSync(`${WORKSPACE}/strict.txt`)).toBe(false)
|
||||
expect(vfs.readFileSync(`${WORKSPACE}/writable.txt`, 'utf8')).toBe('allowed\n')
|
||||
})
|
||||
})
|
||||
@@ -1,6 +1,7 @@
|
||||
/**
|
||||
* The identity, timestamp, and link guarantees MemoryVfs owes its consumers,
|
||||
* asserted on the filesystem directly rather than through the `node:fs` bridge.
|
||||
* The identity, timestamp, link, mutation, and durability-sink guarantees
|
||||
* MemoryVfs owes its consumers, asserted directly rather than through the
|
||||
* `node:fs` bridge.
|
||||
*
|
||||
* `dsh-fs-local` builds a version token from `dev:ino:size:mtimeNs:ctimeNs` and
|
||||
* refuses a write whose token moved since it read. Two properties carry that:
|
||||
@@ -11,7 +12,7 @@
|
||||
*/
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest'
|
||||
import { MemoryVfs } from '../../src/storage/memory.ts'
|
||||
import type { VfsBigIntStats, VfsStats } from '../../src/storage/types.ts'
|
||||
import type { VfsBigIntStats, VfsMutation, VfsMutationSink, VfsStats } from '../../src/storage/types.ts'
|
||||
|
||||
const identity = (vfs: MemoryVfs, path: string): bigint =>
|
||||
(vfs.statSync(path, { bigint: true }) as VfsBigIntStats).ino
|
||||
@@ -55,6 +56,16 @@ describe('entry identity', () => {
|
||||
})
|
||||
|
||||
describe('modification time', () => {
|
||||
it('hydrates explicit metadata without confusing timestamps with permission bits', () => {
|
||||
const vfs = new MemoryVfs()
|
||||
vfs.seed('/dsh/restored', 'value', { mode: 0o600, mtimeMs: 1_600_000_000_000 })
|
||||
vfs.seedDirectory('/dsh/restored-directory', { mode: 0o700, mtimeMs: 1_600_000_000_001 })
|
||||
const stats = vfs.statSync('/dsh/restored') as VfsStats
|
||||
const directory = vfs.statSync('/dsh/restored-directory') as VfsStats
|
||||
expect([stats.mode & 0o777, stats.mtimeMs]).toEqual([0o600, 1_600_000_000_000])
|
||||
expect([directory.mode & 0o777, directory.mtimeMs]).toEqual([0o700, 1_600_000_000_001])
|
||||
})
|
||||
|
||||
it('advances on every write even while the clock stands still', () => {
|
||||
vi.spyOn(Date, 'now').mockReturnValue(1_700_000_000_000)
|
||||
const vfs = new MemoryVfs()
|
||||
@@ -80,6 +91,108 @@ describe('modification time', () => {
|
||||
vfs.writeFileSync('/dsh/log.jsonl', 'second\n')
|
||||
expect(modified(vfs, '/dsh/log.jsonl')).toBe(1_700_000_005_000)
|
||||
})
|
||||
|
||||
it('advances a directory only when its immediate entry set changes', () => {
|
||||
vi.spyOn(Date, 'now').mockReturnValue(1_700_000_000_000)
|
||||
const vfs = new MemoryVfs()
|
||||
vfs.seedDirectory('/dsh/workspace')
|
||||
const empty = modified(vfs, '/dsh/workspace')
|
||||
vfs.writeFileSync('/dsh/workspace/file.txt', 'one')
|
||||
const created = modified(vfs, '/dsh/workspace')
|
||||
vfs.writeFileSync('/dsh/workspace/file.txt', 'two')
|
||||
const rewritten = modified(vfs, '/dsh/workspace')
|
||||
vfs.rmSync('/dsh/workspace/file.txt')
|
||||
const removed = modified(vfs, '/dsh/workspace')
|
||||
expect([created > empty, rewritten === created, removed > rewritten]).toEqual([true, true, true])
|
||||
})
|
||||
})
|
||||
|
||||
describe('mutation publication', () => {
|
||||
it('publishes only committed runtime changes and keeps image seeding silent', () => {
|
||||
const vfs = new MemoryVfs()
|
||||
const mutations: VfsMutation[] = []
|
||||
vfs.subscribe((mutation) => { mutations.push(mutation) })
|
||||
vfs.seed('/dsh/seeded.txt', 'seeded')
|
||||
expect(mutations).toEqual([])
|
||||
vfs.writeFileSync('/dsh/seeded.txt', 'changed')
|
||||
vfs.mkdirSync('/dsh/created')
|
||||
vfs.chmodSync('/dsh/created', 0o700)
|
||||
vfs.renameSync('/dsh/seeded.txt', '/dsh/renamed.txt')
|
||||
vfs.rmSync('/dsh/created', { recursive: true })
|
||||
expect(mutations.map(mutation => ({
|
||||
kind: mutation.kind,
|
||||
path: mutation.path,
|
||||
...mutation.kind === 'write' ? { entryChanged: mutation.entryChanged } : {},
|
||||
...mutation.kind === 'chmod' ? { mode: mutation.mode } : {},
|
||||
}))).toEqual([
|
||||
{ kind: 'write', path: '/dsh/seeded.txt', entryChanged: false },
|
||||
{ kind: 'mkdir', path: '/dsh/created' },
|
||||
{ kind: 'chmod', path: '/dsh/created', mode: 0o700 },
|
||||
{ kind: 'remove', path: '/dsh/seeded.txt' },
|
||||
{ kind: 'write', path: '/dsh/renamed.txt', entryChanged: true },
|
||||
{ kind: 'remove', path: '/dsh/created' },
|
||||
])
|
||||
const renamed = mutations[4]
|
||||
expect(renamed?.kind === 'write' && new TextDecoder().decode(renamed.bytes)).toBe('changed')
|
||||
expect(() => { vfs.writeFileSync('/missing/file', 'no') }).toThrow(/ENOENT/)
|
||||
expect(mutations).toHaveLength(6)
|
||||
})
|
||||
|
||||
it('contains a faulty observer and lets disposal stop later notifications', () => {
|
||||
const vfs = new MemoryVfs()
|
||||
vfs.seedDirectory('/dsh')
|
||||
const reported = vi.spyOn(console, 'error').mockImplementation(() => {})
|
||||
const first = vfs.subscribe(() => { throw new Error('observer failed') })
|
||||
const seen: string[] = []
|
||||
const second = vfs.subscribe((mutation) => { seen.push(mutation.path) })
|
||||
vfs.writeFileSync('/dsh/one', '1')
|
||||
first()
|
||||
second()
|
||||
vfs.writeFileSync('/dsh/two', '2')
|
||||
expect(seen).toEqual(['/dsh/one'])
|
||||
expect(reported).toHaveBeenCalledOnce()
|
||||
})
|
||||
|
||||
it('feeds the same complete mutations to a durable sink and live subscribers', async () => {
|
||||
const recorded: VfsMutation[] = []
|
||||
let flushes = 0
|
||||
const sink: VfsMutationSink = {
|
||||
record: (mutation) => { recorded.push(mutation) },
|
||||
flush: async () => { flushes += 1 },
|
||||
}
|
||||
const vfs = new MemoryVfs({ sink })
|
||||
vfs.seedDirectory('/dsh')
|
||||
const observed: VfsMutation[] = []
|
||||
vfs.subscribe((mutation) => { observed.push(mutation) })
|
||||
vfs.writeFileSync('/dsh/log', 'a')
|
||||
vfs.appendFileSync('/dsh/log', 'bc')
|
||||
await vfs.flush()
|
||||
expect(observed).toEqual(recorded)
|
||||
expect(observed[0]).toBe(recorded[0])
|
||||
expect(recorded[0]).toMatchObject({ kind: 'write', path: '/dsh/log', mode: 0o644, entryChanged: true })
|
||||
expect(recorded[1]).toMatchObject({ kind: 'write', path: '/dsh/log', mode: 0o644, entryChanged: false, appendedFrom: 1 })
|
||||
expect(recorded[1]?.kind === 'write' && new TextDecoder().decode(recorded[1].bytes)).toBe('abc')
|
||||
expect(flushes).toBe(1)
|
||||
})
|
||||
|
||||
it('decomposes a directory rename into replayable destination state', () => {
|
||||
const recorded: VfsMutation[] = []
|
||||
const vfs = new MemoryVfs({
|
||||
sink: { record: (mutation) => { recorded.push(mutation) }, flush: () => Promise.resolve() },
|
||||
})
|
||||
vfs.seedDirectory('/dsh/staging/nested', { mode: 0o700 })
|
||||
vfs.seed('/dsh/staging/nested/file', 'value', { mode: 0o600 })
|
||||
vfs.renameSync('/dsh/staging', '/dsh/published')
|
||||
|
||||
expect(recorded.map(mutation => [mutation.kind, mutation.path])).toEqual([
|
||||
['remove', '/dsh/staging'],
|
||||
['mkdir', '/dsh/published'],
|
||||
['mkdir', '/dsh/published/nested'],
|
||||
['write', '/dsh/published/nested/file'],
|
||||
])
|
||||
expect(recorded[3]).toMatchObject({ kind: 'write', mode: 0o600, entryChanged: true })
|
||||
expect(recorded[3]?.kind === 'write' && new TextDecoder().decode(recorded[3].bytes)).toBe('value')
|
||||
})
|
||||
})
|
||||
|
||||
describe('hard links', () => {
|
||||
|
||||
Generated
+73
@@ -4814,6 +4814,9 @@ importers:
|
||||
picomatch:
|
||||
specifier: ^4.0.4
|
||||
version: 4.0.4
|
||||
readable-stream:
|
||||
specifier: ^4.7.0
|
||||
version: 4.7.0
|
||||
devDependencies:
|
||||
'@deepseek-ai/cordis':
|
||||
specifier: workspace:^
|
||||
@@ -4824,6 +4827,9 @@ importers:
|
||||
'@deepseek-ai/dsh-api-gateway':
|
||||
specifier: workspace:^
|
||||
version: link:../../api/gateway
|
||||
'@deepseek-ai/dsh-bash-sandbox':
|
||||
specifier: workspace:^
|
||||
version: link:../../shell/bash-sandbox
|
||||
'@deepseek-ai/dsh-client-modules':
|
||||
specifier: workspace:^
|
||||
version: link:../../client/modules
|
||||
@@ -4836,12 +4842,27 @@ importers:
|
||||
'@deepseek-ai/dsh-invariants':
|
||||
specifier: workspace:^
|
||||
version: link:../../runtime-diagnostics/invariants
|
||||
'@deepseek-ai/dsh-sandbox-local':
|
||||
specifier: workspace:^
|
||||
version: link:../../sandbox/sandbox-local
|
||||
'@deepseek-ai/dsh-sandbox-policy':
|
||||
specifier: workspace:^
|
||||
version: link:../../sandbox/sandbox-policy
|
||||
'@deepseek-ai/dsh-subprocess-local':
|
||||
specifier: workspace:^
|
||||
version: link:../../subprocess/subprocess-local
|
||||
'@deepseek-ai/node-addon-landlock-run':
|
||||
specifier: workspace:^
|
||||
version: link:../../../native/landlock-run/packages/entry
|
||||
'@types/picomatch':
|
||||
specifier: ^3.0.2
|
||||
version: 3.0.2
|
||||
'@types/readable-stream':
|
||||
specifier: ^4.0.24
|
||||
version: 4.0.24
|
||||
chokidar:
|
||||
specifier: ^5.0.0
|
||||
version: 5.0.0
|
||||
|
||||
packages/extensions/cordis-client-runner:
|
||||
devDependencies:
|
||||
@@ -12641,6 +12662,9 @@ packages:
|
||||
'@types/react@18.3.31':
|
||||
resolution: {integrity: sha512-vfEqpXTvwT91yhmwdfouStN2hSKwTvyRs8qpLfADyrq/kxDw0hZM7Wk9Ug1FELj8hIby+S/+kQCSRFF32nv2Qw==}
|
||||
|
||||
'@types/readable-stream@4.0.24':
|
||||
resolution: {integrity: sha512-NRvUNC/JFGPJvqdAfEve8oginbM6V08u5NzLWpG8MwA2kTPOLnqk+wpwuPT+mp3aUsxyuT6m2gnrPuHYCruzEg==}
|
||||
|
||||
'@types/retry@0.12.0':
|
||||
resolution: {integrity: sha512-wWKOClTTiizcZhXnPY4wikVAwmdYHp8q6DmC+EJUzAMsycb7HB32Kh9RN4+0gExjmPmZSAQjgURXIGATPegAvA==}
|
||||
|
||||
@@ -12892,6 +12916,10 @@ packages:
|
||||
resolution: {integrity: sha512-WoxUM/Be4hfsX06FxsvpGgfYqwgivMV7/Ol7aFuSfSmY6rRaiju4QxOEe9RUS0iYcSHWl5i9AhB1cMoE0p+XiA==}
|
||||
engines: {node: '>=18.12.0'}
|
||||
|
||||
abort-controller@3.0.0:
|
||||
resolution: {integrity: sha512-h8lQ8tacZYnR3vNQTgibj+tODHI5/+l06Au2Pcriv/Gmet0eaj4TwWH41sO9wnHDiQsEj19q0drzdWdeAHtweg==}
|
||||
engines: {node: '>=6.5'}
|
||||
|
||||
accepts@2.0.0:
|
||||
resolution: {integrity: sha512-5cvg6CtKwfgdmVqY1WIiXKc3Q1bkRqGLi+2W/6ao+6Y7gu/RCwRuAhGEzh5B4KlszSuTLgZYuqFqo5bImjNKng==}
|
||||
engines: {node: '>= 0.6'}
|
||||
@@ -13583,9 +13611,17 @@ packages:
|
||||
resolution: {integrity: sha512-aIL5Fx7mawVa300al2BnEE4iNvo1qETxLrPI/o05L7z6go7fCw1J6EQmbK4FmJ2AS7kgVF/KEZWufBfdClMcPg==}
|
||||
engines: {node: '>= 0.6'}
|
||||
|
||||
event-target-shim@5.0.1:
|
||||
resolution: {integrity: sha512-i/2XbnSz/uxRCU6+NdVJgKWDTM427+MqYbkQzD321DuCQJUqOuJKIA0IM2+W2xtYHdKOmZ4dR6fExsd4SXL+WQ==}
|
||||
engines: {node: '>=6'}
|
||||
|
||||
eventemitter3@4.0.7:
|
||||
resolution: {integrity: sha512-8guHBZCwKnFhYdHr2ysuRWErTwhoN2X8XELRlrRwpmfeY2jjuUN4taQMsULKUVo1K4DvZl+0pgfyoysHxvmvEw==}
|
||||
|
||||
events@3.3.0:
|
||||
resolution: {integrity: sha512-mQw+2fkQbALzQ7V0MY0IqdnXNOeTtP4r0lN9z7AAawCXgqea7bDii20AYrIBrFd/Hx0M2Ocz6S111CaFkUcb0Q==}
|
||||
engines: {node: '>=0.8.x'}
|
||||
|
||||
eventsource-parser@3.1.0:
|
||||
resolution: {integrity: sha512-kJezFj9YFAMLeORyi7aCLxLbD5/qWMQnoMVlVPyHIll7lgRJCc3JVln9Vgl9nwQi0YkMnhdGTMNn7CkRRAptMg==}
|
||||
engines: {node: '>=18.0.0'}
|
||||
@@ -14818,6 +14854,10 @@ packages:
|
||||
process-nextick-args@2.0.1:
|
||||
resolution: {integrity: sha512-3ouUOpQhtgrbOa17J7+uxOTpITYWaGP7/AhoR3+A+/1e9skrzelGi/dXzEYyvbxubEF6Wn2ypscTKiKJFFn1ag==}
|
||||
|
||||
process@0.11.10:
|
||||
resolution: {integrity: sha512-cdGef/drWFoydD1JsMzuFf8100nZl+GT+yacc2bEced5f9Rjk4z+WtFUTBu9PhOi9j/jfmBPu0mMEY4wIdAF8A==}
|
||||
engines: {node: '>= 0.6.0'}
|
||||
|
||||
property-information@7.2.0:
|
||||
resolution: {integrity: sha512-IAtzIB6sUiWaJYrX9smp3V46pBGbBeLFRGdh25kg1334VcBlD8HzhPeNIWQH9zhGmo2itIe25EHt9dQP7G5hmg==}
|
||||
|
||||
@@ -14875,6 +14915,10 @@ packages:
|
||||
readable-stream@2.3.8:
|
||||
resolution: {integrity: sha512-8p0AUk4XODgIewSi0l8Epjs+EVnWiK7NoDIEGU0HhE7+ZyY8D1IMY7odu5lRrFXGg71L15KG8QrPmum45RTtdA==}
|
||||
|
||||
readable-stream@4.7.0:
|
||||
resolution: {integrity: sha512-oIGGmcpTLwPga8Bn6/Z75SVaH1z5dUut2ibSyAMVhmUggWpmDn2dapB0n7f8nwaSiRtepAsfJyfXIO5DCVAODg==}
|
||||
engines: {node: ^12.22.0 || ^14.17.0 || >=16.0.0}
|
||||
|
||||
readdirp@4.1.2:
|
||||
resolution: {integrity: sha512-GDhwkLfywWL2s6vEjyhri+eXmfH6j1L7JE27WhqLeYzoh/A3DBaYGEj2H/HFZCn/kMfim73FXxEJTw06WtxQwg==}
|
||||
engines: {node: '>= 14.18.0'}
|
||||
@@ -15122,6 +15166,9 @@ packages:
|
||||
string_decoder@1.1.1:
|
||||
resolution: {integrity: sha512-n/ShnvDi6FHbbVfviro+WojiFzv+s8MPMHBczVePfUpDJLwoLT0ht1l4YwBCbi8pJAveEEdnkHyPyTP/mzRfwg==}
|
||||
|
||||
string_decoder@1.3.0:
|
||||
resolution: {integrity: sha512-hkRX8U1WjJFd8LsDJ2yQ/wWWxaopEsABU1XfkM8A+j0+85JAGppt16cr1Whg6KIbb4okU6Mql6BOj+uup/wKeA==}
|
||||
|
||||
stringify-entities@4.0.4:
|
||||
resolution: {integrity: sha512-IwfBptatlO+QCJUo19AqvrPNqlVMpW9YEL2LIVY+Rpv2qsjCGxaDLNRgeGsQWJhfItebuJhsGSLjaBbNSQ+ieg==}
|
||||
|
||||
@@ -17914,6 +17961,10 @@ snapshots:
|
||||
'@types/prop-types': 15.7.15
|
||||
csstype: 3.2.3
|
||||
|
||||
'@types/readable-stream@4.0.24':
|
||||
dependencies:
|
||||
'@types/node': 22.20.0
|
||||
|
||||
'@types/retry@0.12.0': {}
|
||||
|
||||
'@types/spdx-expression-parse@4.0.0': {}
|
||||
@@ -18191,6 +18242,10 @@ snapshots:
|
||||
js-yaml: 4.3.1
|
||||
tslib: 2.8.1
|
||||
|
||||
abort-controller@3.0.0:
|
||||
dependencies:
|
||||
event-target-shim: 5.0.1
|
||||
|
||||
accepts@2.0.0:
|
||||
dependencies:
|
||||
mime-types: 3.0.2
|
||||
@@ -18951,8 +19006,12 @@ snapshots:
|
||||
|
||||
etag@1.8.1: {}
|
||||
|
||||
event-target-shim@5.0.1: {}
|
||||
|
||||
eventemitter3@4.0.7: {}
|
||||
|
||||
events@3.3.0: {}
|
||||
|
||||
eventsource-parser@3.1.0: {}
|
||||
|
||||
eventsource@3.0.7:
|
||||
@@ -20426,6 +20485,8 @@ snapshots:
|
||||
|
||||
process-nextick-args@2.0.1: {}
|
||||
|
||||
process@0.11.10: {}
|
||||
|
||||
property-information@7.2.0: {}
|
||||
|
||||
protobufjs@7.6.4:
|
||||
@@ -20498,6 +20559,14 @@ snapshots:
|
||||
string_decoder: 1.1.1
|
||||
util-deprecate: 1.0.2
|
||||
|
||||
readable-stream@4.7.0:
|
||||
dependencies:
|
||||
abort-controller: 3.0.0
|
||||
buffer: 6.0.3
|
||||
events: 3.3.0
|
||||
process: 0.11.10
|
||||
string_decoder: 1.3.0
|
||||
|
||||
readdirp@4.1.2: {}
|
||||
|
||||
readdirp@5.0.0: {}
|
||||
@@ -20840,6 +20909,10 @@ snapshots:
|
||||
dependencies:
|
||||
safe-buffer: 5.1.2
|
||||
|
||||
string_decoder@1.3.0:
|
||||
dependencies:
|
||||
safe-buffer: 5.2.1
|
||||
|
||||
stringify-entities@4.0.4:
|
||||
dependencies:
|
||||
character-entities-html4: 2.1.0
|
||||
|
||||
Reference in New Issue
Block a user