mirror of
https://github.com/deepseek-ai/deepseek-harness.git
synced 2026-08-29 04:26:38 +00:00
feat(webworker): model-executed shell over nested worker processes
Buy the grammar, own the execution: @yarnpkg/parsers parses the command line - aliased at bundle time to its shell entry so the root barrel's syml/js-yaml closure stays out of the worker - and a VFS-backed evaluator with a coreutils command table runs it inside the worker host. Each shell process is a real child WebWorker spawned from the same bundle (the first frame decides the role), so the TERM-then-KILL ladder is real - TERM requests, KILL terminates the worker - and the file face stays asynchronous end to end, since the deployment target serves no COOP/COEP headers and SharedArrayBuffer never exists there. node:child_process reports through the ChildProcess surface the subprocess service consumes; execSync, execFileSync and fork refuse, and node-pty stays stubbed.
This commit is contained in:
imccyu
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/architecture/2026-08-20-webworker-node-face.md
|
||||
2026-08-20-webworker-node-face.md: 08119cce96eff244f8e9ada3462ce5d35c1b538d
|
||||
2026-08-20-webworker-node-face.zh.md: 573c0be055d066d2d6d0db11a2476ba528517727
|
||||
@@ -0,0 +1,34 @@
|
||||
# Agent Note: the worker's Node face — builtins, VFS, and the shell process layer
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-08-20-webworker-node-face.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
The worker runs the web profile's Cordis configuration byte for byte — no worker-specific rows — so a browser's missing platform must be replaced at the module layer, where a proxied module keeps its identity and changes its implementation. That covers three fronts: the Node builtins the tree imports, the filesystem those builtins answer from, and a process layer for the bash tool, which mounted, advertised itself to the model, and then failed on every call while `node:child_process` was a structural stub.
|
||||
|
||||
## Decision
|
||||
|
||||
**Builtins.** The proxy table replaces Node builtins and external npm packages, never workspace or vendored modules. `./implemented/<module>.ts` carries real semantics over a worker data source; `./mock/<module>.ts` mounts silently and reports the missing capability when a call reaches it. The loader's table holds one memoized thunk per specifier — evaluation happens at first `require`, not at assembly — and each shim's exported face typechecks against Node's own module type, with the narrow, documented exceptions where structural identity (a real class) cannot be satisfied. The worker installs the `process` global itself and fills it into the table at assembly.
|
||||
|
||||
**VFS.** Memory is the truth. `statSync(path, { bigint: true })` returns Node's BigInt shape, and two fields carry real information because `dsh-fs-local`'s stale-write guard depends on them: `ino` is per-path identity from a monotonic counter (a recreated path reports a new identity), and `mtimeMs` is strictly increasing per entry (`max(now, previous + 1)`), because in-memory writes routinely land in one millisecond and an equal timestamp would let a stale overwrite pass. The hunt that produced this also fixed the silence around it: cordis's logger verbosity counts UP, so an exporter that declares no level drops every warning — `startWorkerHost` installs a console exporter with `levels: { default: 2 }` before any entry mounts.
|
||||
|
||||
**Shell.** `node:child_process` is a real implementation over the VFS. The grammar is bought — `@yarnpkg/parsers`' `parseShell` — and the evaluator and command table are owned, because every candidate interpreter brings its own filesystem: pipelines are strings handed along, and each program is a function over the VFS. The table is the machine's whole `/bin`; an absent name reports `command not found` (127). Each `spawn` starts a child Web Worker from this same bundle, its first frame declaring the shell-process role, so the termination ladder is real: `SIGTERM` asks at the next command boundary, `SIGKILL` terminates the worker mid-loop — the preemption an in-thread interpreter can never have. The filesystem face is asynchronous end to end (child frames to the host VFS); `execSync`, `execFileSync`, and `fork` refuse, and `node-pty` stays a stub.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**Replacing `dsh-subprocess-local` or the bash executor.** The first would let the proxy table replace a workspace package against its own classification and invert the layering; the second trips `dsh-permission-presets`' boot-time `sandboxMode` validation and drops tested timeout/output behavior.
|
||||
|
||||
**`@yarnpkg/shell`, WASM shells, WebContainer.** The matching interpreter is built on real Node streams (~1.5 MB closure to own); WASM was removed from this deployment by decision and WASI has no `fork`; all of them arrive with their own filesystem, the one part that cannot be reused.
|
||||
|
||||
**`SharedArrayBuffer` + `Atomics.wait` for a synchronous child filesystem.** Measured on the deployment target: without COOP/COEP headers `SharedArrayBuffer` is not defined, and GitHub Pages cannot set response headers. The asynchronous face is a superset; a SAB backend can slot under it later without touching a program.
|
||||
|
||||
**Fabricating stats or widening error predicates instead of honoring `bigint`.** Constant `ino`/wall-clock `mtimeNs` silently disable the stale-write guard; swallowing `FS_IO_ERROR` in skill discovery would have made the same bug a permanently empty catalog with no failure anywhere.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Sandbox modes other than `danger-full-access` fail loud: `SandboxEnforcement` has no "nothing was enforced" value and a browser has no kernel, so `ctx.sandbox.confine` fails closed and the command never starts. Real enforcement at the VFS frame gate is a designed follow-up, not this note.
|
||||
- The Node-host ladder test (`tests/node/child-process.spec.ts`) is registered windows-unsupported: the ladder's win32 kill rung is taskkill-by-real-pid, undeliverable to a process-table pid, while the worker itself always reports `linux`.
|
||||
- Output is incremental but not streamed: programs write into sinks forwarded as `data` events, and a pipeline stage completes before the next starts.
|
||||
- The runtime's tests mirror `src/` (`tests/node/`, `tests/shell/`, `tests/storage/`, …), so each shim family owns its behavior cases beside the oracle-diff suites.
|
||||
@@ -0,0 +1,34 @@
|
||||
# Agent Note：worker 的 Node 面——builtin、VFS 与 shell 进程层
|
||||
|
||||
状态：已实施
|
||||
|
||||
[English](2026-08-20-webworker-node-face.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
worker 逐字节运行 web profile 的 Cordis 配置——没有 worker 专属行——因此浏览器缺失的平台必须在模块层被替换：被代理的模块保持身份、更换实现。这覆盖三条战线：树所 import 的 Node builtin、这些 builtin 背后应答的文件系统，以及 bash 工具的进程层——在 `node:child_process` 还是结构桩的时期，工具照常挂载、向模型自我宣告，然后每次调用都失败。
|
||||
|
||||
## 决定
|
||||
|
||||
**Builtin。** 代理表只替换 Node builtin 与外部 npm 包，绝不替换 workspace 或 vendored 模块。`./implemented/<module>.ts` 在 worker 数据源之上承载真语义；`./mock/<module>.ts` 静默挂载、在调用真正抵达时报告缺失的能力。装载器的表按 specifier 各持一个 memoized thunk——求值发生在首次 `require` 而非装配期——且每个垫片的导出面对 Node 自身的模块类型作类型检查，仅在结构身份（真实类）确不可满足处留最窄的、有说明的例外。`process` 全局由 worker 自装，装配期填入表中。
|
||||
|
||||
**VFS。** 内存为真相。`statSync(path, { bigint: true })` 返回 Node 的 BigInt 形状，其中两个字段承载真实信息，因为 `dsh-fs-local` 的 stale-write guard 依赖它们：`ino` 是按路径的身份（单调计数器分配，路径重建即新身份），`mtimeMs` 按条目严格递增（`max(now, previous + 1)`）——内存写例行落在同一毫秒内，相等的时间戳会放过陈旧覆写。这场排查同时修掉了它周围的静默：cordis 日志器的详细度数值向上计数，未声明等级的 exporter 会丢掉所有 warning——`startWorkerHost` 在任何 entry 挂载前安装 `levels: { default: 2 }` 的 console exporter。
|
||||
|
||||
**Shell。** `node:child_process` 是 VFS 之上的真实现。语法是买来的——`@yarnpkg/parsers` 的 `parseShell`——求值器与命令表是自有的，因为每个候选解释器都自带文件系统：管道是逐段传递的字符串，每个程序是 VFS 上的一个函数。命令表就是这台机器的全部 `/bin`；不存在的名字报告 `command not found`（127）。每次 `spawn` 从同一个 bundle 起一个子 Web Worker，首帧声明 shell 进程角色，因此终止梯是真的：`SIGTERM` 在下一命令边界处请求停止，`SIGKILL` 在任意时刻终止 worker——这是线程内解释器永远没有的抢占。文件系统面端到端异步（子进程经帧到宿主 VFS）；`execSync`、`execFileSync`、`fork` 拒绝，`node-pty` 保持桩。
|
||||
|
||||
## 曾考虑的替代方案
|
||||
|
||||
**整包替换 `dsh-subprocess-local` 或替换 bash 执行器。** 前者让代理表首次替换 workspace 包、违背其自身分类并倒置分层；后者撞上 `dsh-permission-presets` 对 `sandboxMode` 的 boot 期硬校验，并丢掉执行器已被测试钉住的超时/输出行为。
|
||||
|
||||
**`@yarnpkg/shell`、WASM shell、WebContainer。** 配套解释器建立在真实 Node streams 之上（约 1.5 MB 闭包要自养）；WASM 已被本部署的决定排除，WASI 没有 `fork`；且它们全都自带文件系统——恰是无法复用的那部分。
|
||||
|
||||
**`SharedArrayBuffer` + `Atomics.wait` 给子进程同步文件系统。** 在部署目标实测：无 COOP/COEP 头时 `SharedArrayBuffer` 未定义，而 GitHub Pages 无法设置响应头。异步面是超集；SAB 后端将来可垫入其下而不动任何程序。
|
||||
|
||||
**伪造 stats 或放宽错误谓词，而非如实实现 `bigint`。** 常量 `ino`/纯挂钟 `mtimeNs` 会静默废掉 stale-write guard；让技能发现吞下 `FS_IO_ERROR` 则会把同一个 bug 变成处处无失败的永久空目录。
|
||||
|
||||
## 后果
|
||||
|
||||
- `danger-full-access` 之外的沙箱档 fail loud：`SandboxEnforcement` 没有「未执法」值、浏览器没有内核，`ctx.sandbox.confine` 落闭、命令零启动。在 VFS 帧闸口做真执法是设计中的后续，不属本条。
|
||||
- Node 宿主的阶梯测试（`tests/node/child-process.spec.ts`）登记为 windows 不支持：阶梯的 win32 kill 梯级是按真 pid 的 taskkill，对进程表 pid 不可投递，而 worker 自身恒报 `linux`。
|
||||
- 输出增量但不流式：程序写入的 sink 以 `data` 事件转发，一个管道阶段完成后下一阶段才开始。
|
||||
- 运行时的测试镜像 `src/`（`tests/node/`、`tests/shell/`、`tests/storage/`……），每个垫片族在 oracle-diff 套件旁拥有自己的行为用例。
|
||||
@@ -51,6 +51,9 @@ export const MODULE_PROXIES: Record<string, string> = {
|
||||
'node:perf_hooks': './node/builtin_modules/implemented/perf_hooks.ts',
|
||||
// Real zstd codec: session-log appends compress on every write.
|
||||
'node:zlib': './node/builtin_modules/implemented/zlib.ts',
|
||||
// The worker's own process layer: `bash -c` and the command table run against
|
||||
// the VFS, because a browser worker has no processes to fork.
|
||||
'node:child_process': './node/builtin_modules/implemented/child_process.ts',
|
||||
// Structural mocks: every symbol exists, every call throws.
|
||||
'node:net': './node/builtin_modules/mock/net.ts',
|
||||
'node:stream': './node/builtin_modules/mock/stream.ts',
|
||||
|
||||
+398
@@ -0,0 +1,398 @@
|
||||
/**
|
||||
* `node:child_process` over the worker's own shell.
|
||||
*
|
||||
* A browser worker cannot fork, so this module IS the machine's process layer:
|
||||
* `spawn` starts the argv as a shell process (`src/shell/process/`) — its own
|
||||
* Web Worker, off this thread — and reports it through the `ChildProcess`
|
||||
* surface the subprocess service consumes: pipes, `exit`/`close`, pid, and
|
||||
* signals, with `SIGKILL` terminating the worker for real. The command table
|
||||
* is the only `/bin` that exists, so a name it does not hold fails with
|
||||
* `ENOENT`, exactly as a missing binary does on a real host.
|
||||
*
|
||||
* What stays impossible is what needs a real process: synchronous execution
|
||||
* (`execSync`, and `spawnSync` for a known program) and `fork`.
|
||||
* @module @deepseek-ai/dsh-experimental-webworker-runtime/src/node/builtin_modules/implemented/child_process
|
||||
*/
|
||||
|
||||
import { Buffer } from 'buffer'
|
||||
import { EventEmitter } from './events.ts'
|
||||
import { notImplementedFail } from '../../notImplementedFail.ts'
|
||||
import { registerProcess, releaseProcess, signalProcess } from '../../process-table.ts'
|
||||
import { startProcess } from '../../../shell/process/host.ts'
|
||||
import { standardPrograms } from '../../../shell/programs/index.ts'
|
||||
import { DSH_ROOT } from '../../../storage/paths.ts'
|
||||
|
||||
const MODULE = 'node:child_process'
|
||||
|
||||
/** Per-stream disposition, as Node's `stdio` array spells it. */
|
||||
type StdioSetting = 'pipe' | 'ignore' | 'inherit'
|
||||
|
||||
/** The spawn options this shim reads; Node accepts more, none of which apply here. */
|
||||
export interface WorkerSpawnOptions {
|
||||
cwd?: string | undefined
|
||||
env?: Record<string, string | undefined> | undefined
|
||||
stdio?: StdioSetting | readonly StdioSetting[] | undefined
|
||||
/** Accepted and ignored: process groups do not exist, so there is no group to detach into. */
|
||||
detached?: boolean | undefined
|
||||
}
|
||||
|
||||
/**
|
||||
* The readable half of a pipe: `data` events carrying Buffers, `end`, and a
|
||||
* `destroy` that stops delivery.
|
||||
*
|
||||
* The stream-shaping members below are no-ops rather than omissions. A caller
|
||||
* that configures the pipe before reading it (the browser launcher calls
|
||||
* `setEncoding`) would otherwise die of a TypeError on the configuration line,
|
||||
* hiding the real outcome — which for an unknown program is the `ENOENT` this
|
||||
* shim is about to emit.
|
||||
*/
|
||||
class WorkerReadable extends EventEmitter {
|
||||
private destroyed = false
|
||||
|
||||
/**
|
||||
* Accept an encoding (chunks are always UTF-8 text carried as Buffers).
|
||||
* @returns this stream.
|
||||
*/
|
||||
setEncoding(): this {
|
||||
return this
|
||||
}
|
||||
|
||||
/**
|
||||
* Accept a flow-control request; delivery is driven by the command, which
|
||||
* has already produced whatever it produced.
|
||||
* @returns this stream.
|
||||
*/
|
||||
pause(): this {
|
||||
return this
|
||||
}
|
||||
|
||||
/** @returns this stream; see {@link pause}. */
|
||||
resume(): this {
|
||||
return this
|
||||
}
|
||||
|
||||
/**
|
||||
* Deliver one chunk to the `data` listeners.
|
||||
* @param text - the text written by the command.
|
||||
*/
|
||||
push(text: string): void {
|
||||
if (this.destroyed || text === '') return
|
||||
this.emit('data', Buffer.from(text, 'utf8'))
|
||||
}
|
||||
|
||||
/** Signal end of stream. */
|
||||
finish(): void {
|
||||
if (this.destroyed) return
|
||||
this.emit('end')
|
||||
}
|
||||
|
||||
/** Stop delivering; the collector calls this once the process settles. */
|
||||
destroy(): void {
|
||||
this.destroyed = true
|
||||
this.emit('close')
|
||||
}
|
||||
}
|
||||
|
||||
/** The writable half of stdin: the batch write the subprocess service performs. */
|
||||
class WorkerWritable extends EventEmitter {
|
||||
private text = ''
|
||||
|
||||
/**
|
||||
* Buffer one write.
|
||||
* @param chunk - text or bytes to add to standard input.
|
||||
* @returns true, since nothing here applies backpressure.
|
||||
*/
|
||||
write(chunk: string | Uint8Array): boolean {
|
||||
this.text += typeof chunk === 'string' ? chunk : Buffer.from(chunk).toString('utf8')
|
||||
return true
|
||||
}
|
||||
|
||||
/**
|
||||
* Finish standard input.
|
||||
* @param chunk - optional final write.
|
||||
*/
|
||||
end(chunk?: string | Uint8Array): void {
|
||||
if (chunk !== undefined) this.write(chunk)
|
||||
this.emit('finish')
|
||||
}
|
||||
|
||||
/** @returns everything written so far. */
|
||||
contents(): string {
|
||||
return this.text
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* One running command, wearing the parts of `ChildProcess` its consumers read.
|
||||
*/
|
||||
export class WorkerChildProcess extends EventEmitter {
|
||||
/** The worker's own process id for this command, from the process table. */
|
||||
readonly pid: number
|
||||
/** Standard input, when the caller asked for a pipe; null otherwise. */
|
||||
readonly stdin: WorkerWritable | null
|
||||
/** Standard output, when the caller asked for a pipe; null otherwise. */
|
||||
readonly stdout: WorkerReadable | null
|
||||
/** Standard error, when the caller asked for a pipe; null otherwise. */
|
||||
readonly stderr: WorkerReadable | null
|
||||
/** Exit status once settled; null while running and after a signal. */
|
||||
exitCode: number | null = null
|
||||
/** The signal that ended the command, or null when it exited on its own. */
|
||||
signalCode: NodeJS.Signals | null = null
|
||||
|
||||
constructor(pid: number, stdio: readonly StdioSetting[]) {
|
||||
super()
|
||||
this.pid = pid
|
||||
this.stdin = stdio[0] === 'pipe' ? new WorkerWritable() : null
|
||||
this.stdout = stdio[1] === 'pipe' ? new WorkerReadable() : null
|
||||
this.stderr = stdio[2] === 'pipe' ? new WorkerReadable() : null
|
||||
}
|
||||
|
||||
/**
|
||||
* Deliver a signal to this command.
|
||||
* @param signal - signal name; every one of them terminates.
|
||||
* @returns true when the command was still running.
|
||||
*/
|
||||
kill(signal: NodeJS.Signals = 'SIGTERM'): boolean {
|
||||
return signalProcess(this.pid, signal)
|
||||
}
|
||||
}
|
||||
|
||||
/** Normalize the `stdio` option into the three-entry form the shim reads. */
|
||||
function stdioOf(option: WorkerSpawnOptions['stdio']): StdioSetting[] {
|
||||
if (typeof option === 'string') return [option, option, option]
|
||||
if (option === undefined) return ['pipe', 'pipe', 'pipe']
|
||||
return [option[0] ?? 'pipe', option[1] ?? 'pipe', option[2] ?? 'pipe']
|
||||
}
|
||||
|
||||
/** The environment a command runs with: the caller's map, minus the removals Node allows. */
|
||||
function environmentOf(option: WorkerSpawnOptions['env']): Record<string, string> {
|
||||
const inherited = (globalThis as { process?: { env?: Record<string, string> } }).process?.env ?? {}
|
||||
const source = option ?? inherited
|
||||
return Object.fromEntries(Object.entries(source).filter(([, value]) => value !== undefined) as [string, string][])
|
||||
}
|
||||
|
||||
/**
|
||||
* A missing program fails the way Node fails a missing binary, so consumers
|
||||
* that classify spawn errors by `code`, `path`, and `syscall` keep working.
|
||||
*/
|
||||
function spawnEnoent(program: string): NodeJS.ErrnoException {
|
||||
const error = new Error(`spawn ${program} ENOENT`) as NodeJS.ErrnoException
|
||||
error.code = 'ENOENT'
|
||||
error.errno = -2
|
||||
error.path = program
|
||||
error.syscall = `spawn ${program}`
|
||||
return error
|
||||
}
|
||||
|
||||
/** Whether this argv is a shell invocation whose script the interpreter should parse. */
|
||||
function shellScriptOf(argv: readonly string[]): string | undefined {
|
||||
const [program, flag, script] = argv
|
||||
if ((program !== 'bash' && program !== 'sh') || flag !== '-c') return undefined
|
||||
return script ?? ''
|
||||
}
|
||||
|
||||
/**
|
||||
* Run one command in the worker.
|
||||
*
|
||||
* The call returns immediately with a handle; the command runs in its own
|
||||
* worker (or inline where no `Worker` exists) and reports back through the
|
||||
* handle's pipes and events.
|
||||
* @param program - the program name, as argv[0].
|
||||
* @param args - its arguments.
|
||||
* @param options - working directory, environment, and stdio dispositions.
|
||||
* @returns the running command's handle.
|
||||
*/
|
||||
export function spawn(
|
||||
program: string,
|
||||
args: readonly string[] = [],
|
||||
options: WorkerSpawnOptions = {},
|
||||
): WorkerChildProcess {
|
||||
if (typeof program !== 'string' || program === '') {
|
||||
// Node refuses a non-string command with this error rather than starting
|
||||
// anything; a caller whose own lookup produced nothing reads why.
|
||||
const invalid = new TypeError(`The "file" argument must be a non-empty string. Received ${program as unknown as string}`) as NodeJS.ErrnoException
|
||||
invalid.code = 'ERR_INVALID_ARG_TYPE'
|
||||
throw invalid
|
||||
}
|
||||
const argv = [program, ...args]
|
||||
const stdio = stdioOf(options.stdio)
|
||||
const entry = registerProcess()
|
||||
const child = new WorkerChildProcess(entry.pid, stdio)
|
||||
|
||||
const script = shellScriptOf(argv)
|
||||
const known = script !== undefined || standardPrograms().has(program)
|
||||
|
||||
const emit = (stream: 'stdout' | 'stderr', text: string): void => {
|
||||
if (text === '') return
|
||||
const pipe = stream === 'stdout' ? child.stdout : child.stderr
|
||||
if (pipe !== null) {
|
||||
pipe.push(text)
|
||||
return
|
||||
}
|
||||
// An inherited stream belongs to the host: the worker's console is the
|
||||
// only place it can go, and an ignored one goes nowhere.
|
||||
if (stdio[stream === 'stdout' ? 1 : 2] === 'inherit') {
|
||||
(stream === 'stdout' ? console.log : console.error)(text.replace(/\n$/, ''))
|
||||
}
|
||||
}
|
||||
|
||||
const settle = (exitCode: number): void => {
|
||||
releaseProcess(entry.pid)
|
||||
// A signalled command reports no exit code, which is what makes the
|
||||
// subprocess service classify it as killed rather than finished.
|
||||
const signal = entry.signal ?? null
|
||||
child.exitCode = signal === null ? exitCode : null
|
||||
child.signalCode = signal
|
||||
child.stdout?.finish()
|
||||
child.stderr?.finish()
|
||||
child.emit('exit', child.exitCode, signal)
|
||||
child.emit('close', child.exitCode, signal)
|
||||
}
|
||||
|
||||
// The command starts on a microtask, so a caller that attaches listeners and
|
||||
// writes standard input right after `spawn()` — the subprocess service does
|
||||
// exactly that — is never racing the first output.
|
||||
queueMicrotask(() => {
|
||||
if (!known) {
|
||||
releaseProcess(entry.pid)
|
||||
child.emit('error', spawnEnoent(program))
|
||||
return
|
||||
}
|
||||
entry.process = startProcess({
|
||||
script,
|
||||
argv,
|
||||
cwd: options.cwd ?? DSH_ROOT,
|
||||
env: environmentOf(options.env),
|
||||
stdin: child.stdin?.contents() ?? '',
|
||||
onOutput: emit,
|
||||
onExit: settle,
|
||||
})
|
||||
// A signal that arrived while the process was still starting has to reach
|
||||
// it now; the table recorded it but had nothing to deliver it to.
|
||||
if (entry.signal !== undefined) {
|
||||
if (entry.signal === 'SIGKILL') entry.process.destroy()
|
||||
else entry.process.interrupt()
|
||||
}
|
||||
})
|
||||
|
||||
return child
|
||||
}
|
||||
|
||||
/** The result shape `spawnSync` returns, holding only the members consumers read. */
|
||||
export interface WorkerSpawnSyncResult {
|
||||
pid: number
|
||||
status: number | null
|
||||
signal: NodeJS.Signals | null
|
||||
stdout: Buffer
|
||||
stderr: Buffer
|
||||
output: (Buffer | null)[]
|
||||
/** Why the run did not happen; carries `code` for the callers that classify by it. */
|
||||
error?: NodeJS.ErrnoException
|
||||
}
|
||||
|
||||
/**
|
||||
* Report that a command cannot run synchronously.
|
||||
*
|
||||
* Callers use `spawnSync` to probe for a binary (the sandbox runner probes do)
|
||||
* and Node answers a missing one with an `error` rather than a throw, so this
|
||||
* answers in the same shape: absent programs report `ENOENT`, and a program
|
||||
* this shell *does* have reports that only the asynchronous path can run it.
|
||||
* @param program - the program name.
|
||||
* @returns the Node-shaped synchronous result carrying the failure.
|
||||
*/
|
||||
export function spawnSync(program: string): WorkerSpawnSyncResult {
|
||||
const empty = Buffer.alloc(0)
|
||||
const error = standardPrograms().has(program)
|
||||
? new Error(`${MODULE}.spawnSync cannot run ${program} in the worker host: commands run asynchronously`)
|
||||
: spawnEnoent(program)
|
||||
return { pid: -1, status: null, signal: null, stdout: empty, stderr: empty, output: [null, empty, empty], error }
|
||||
}
|
||||
|
||||
/** Callback `exec` and `execFile` report through. */
|
||||
type ExecCallback = (error: Error | null, stdout: string, stderr: string) => void
|
||||
|
||||
/** Split the optional options argument from the callback Node allows in either position. */
|
||||
function execArguments(
|
||||
options: WorkerSpawnOptions | ExecCallback | undefined,
|
||||
callback: ExecCallback | undefined,
|
||||
): { options: WorkerSpawnOptions; callback: ExecCallback | undefined } {
|
||||
if (typeof options === 'function') return { options: {}, callback: options }
|
||||
return { options: options ?? {}, callback }
|
||||
}
|
||||
|
||||
/**
|
||||
* Run a command line and report its output through a callback.
|
||||
* @param command - the shell source to run.
|
||||
* @param options - working directory and environment, or the callback.
|
||||
* @param callback - receives the failure (nonzero status included), stdout, and stderr.
|
||||
* @returns the running command's handle.
|
||||
*/
|
||||
export function exec(
|
||||
command: string,
|
||||
options?: WorkerSpawnOptions | ExecCallback,
|
||||
callback?: ExecCallback,
|
||||
): WorkerChildProcess {
|
||||
const settled = execArguments(options, callback)
|
||||
return execute(['bash', '-c', command], settled.options, settled.callback)
|
||||
}
|
||||
|
||||
/**
|
||||
* Run one program with an explicit argv and report its output through a callback.
|
||||
* @param program - the program name.
|
||||
* @param args - its arguments, or the options, or the callback.
|
||||
* @param options - working directory and environment, or the callback.
|
||||
* @param callback - receives the failure (nonzero status included), stdout, and stderr.
|
||||
* @returns the running command's handle.
|
||||
*/
|
||||
export function execFile(
|
||||
program: string,
|
||||
args?: readonly string[] | WorkerSpawnOptions | ExecCallback,
|
||||
options?: WorkerSpawnOptions | ExecCallback,
|
||||
callback?: ExecCallback,
|
||||
): WorkerChildProcess {
|
||||
const argv = Array.isArray(args) ? [program, ...args as string[]] : [program]
|
||||
const shifted = Array.isArray(args) ? options : args as WorkerSpawnOptions | ExecCallback | undefined
|
||||
const settled = execArguments(shifted, typeof options === 'function' ? options : callback)
|
||||
return execute(argv, settled.options, settled.callback)
|
||||
}
|
||||
|
||||
/** Shared body of `exec` and `execFile`: spawn, collect both streams, then report. */
|
||||
function execute(argv: readonly string[], options: WorkerSpawnOptions, callback: ExecCallback | undefined): WorkerChildProcess {
|
||||
const child = spawn(argv[0] as string, argv.slice(1), { ...options, stdio: 'pipe' })
|
||||
let stdout = ''
|
||||
let stderr = ''
|
||||
child.stdout?.on('data', (chunk: unknown) => { stdout += String(chunk) })
|
||||
child.stderr?.on('data', (chunk: unknown) => { stderr += String(chunk) })
|
||||
child.on('error', (error: unknown) => { callback?.(error instanceof Error ? error : new Error(String(error)), stdout, stderr) })
|
||||
child.on('close', (code: unknown) => {
|
||||
const status = typeof code === 'number' ? code : 1
|
||||
callback?.(status === 0 ? null : new Error(`Command failed: ${argv.join(' ')}`), stdout, stderr)
|
||||
})
|
||||
return child
|
||||
}
|
||||
|
||||
/** Run a command line synchronously (unavailable: the interpreter is asynchronous). */
|
||||
export const execSync: typeof import('node:child_process').execSync = notImplementedFail(MODULE, 'execSync')
|
||||
|
||||
/** Run one program synchronously (unavailable: the interpreter is asynchronous). */
|
||||
export const execFileSync: typeof import('node:child_process').execFileSync = notImplementedFail(MODULE, 'execFileSync')
|
||||
|
||||
/** Start a Node child (unavailable: the worker cannot create another Node runtime). */
|
||||
export const fork: typeof import('node:child_process').fork = notImplementedFail(MODULE, 'fork')
|
||||
|
||||
/** CommonJS interop marker: the worker loader hands `default` to default imports (see ../../builtins.ts). */
|
||||
export const __esModule = true
|
||||
|
||||
/**
|
||||
* The `node:child_process` declarations this module stands in for. The four
|
||||
* process starters keep this module's own types: they answer
|
||||
* {@link WorkerChildProcess} and {@link WorkerSpawnSyncResult}, the pipes and exit
|
||||
* facts a shell worker can carry, where Node declares a `ChildProcess` holding OS
|
||||
* stream objects and, for `exec`/`execFile`, an overload ladder over encodings
|
||||
* this shell reports as UTF-8 text.
|
||||
*/
|
||||
type NodeFace = Partial<Omit<typeof import('node:child_process'), 'spawn' | 'spawnSync' | 'exec' | 'execFile'>>
|
||||
& Record<'spawn' | 'spawnSync' | 'exec' | 'execFile', unknown>
|
||||
|
||||
/** CommonJS default export: the members `require()` hands a caller of this module. */
|
||||
export default { spawn, spawnSync, exec, execFile, execFileSync, execSync, fork } satisfies NodeFace
|
||||
@@ -37,6 +37,7 @@ import * as nodeUrl from './builtin_modules/implemented/url.ts'
|
||||
import * as nodeUtil from './builtin_modules/implemented/util.ts'
|
||||
import * as nodeUtilTypes from './builtin_modules/implemented/util/types.ts'
|
||||
import * as nodeZlib from './builtin_modules/implemented/zlib.ts'
|
||||
import * as nodeChildProcess from './builtin_modules/implemented/child_process.ts'
|
||||
import * as nodeNet from './builtin_modules/mock/net.ts'
|
||||
import * as nodeSqlite from './builtin_modules/mock/sqlite.ts'
|
||||
import * as nodeStream from './builtin_modules/mock/stream.ts'
|
||||
@@ -57,6 +58,7 @@ import type { StaticModuleFactory } from '../module-system/module-loader.ts'
|
||||
const BUILTINS: Record<string, StaticModuleFactory> = {
|
||||
async_hooks: () => nodeAsyncHooks,
|
||||
buffer: () => nodeBuffer,
|
||||
child_process: () => nodeChildProcess,
|
||||
crypto: () => nodeCrypto,
|
||||
events: () => nodeEvents,
|
||||
fs: () => nodeFs,
|
||||
|
||||
@@ -0,0 +1,76 @@
|
||||
/**
|
||||
* The worker's process table. A browser worker cannot fork, so the
|
||||
* `node:child_process` shim keeps its own table: one entry per running
|
||||
* command, with the pid `process.kill` and the subprocess service's tree
|
||||
* bookkeeping address it by.
|
||||
*
|
||||
* Kept apart from both consumers because they need it from opposite sides —
|
||||
* the shim registers entries, the `process` global signals them.
|
||||
* @module @deepseek-ai/dsh-experimental-webworker-runtime/src/node/process-table
|
||||
*/
|
||||
|
||||
import type { RunningProcess } from '../shell/process/host.ts'
|
||||
|
||||
/** One running command, as the table tracks it. */
|
||||
export interface WorkerProcessEntry {
|
||||
/** Identifier handed out to the host tree; unique for the worker's lifetime. */
|
||||
readonly pid: number
|
||||
/** The first signal delivered, which decides how the process reports its death. */
|
||||
signal: NodeJS.Signals | undefined
|
||||
/** The started command, attached once it exists; signals reach it through this. */
|
||||
process: RunningProcess | undefined
|
||||
}
|
||||
|
||||
const entries = new Map<number, WorkerProcessEntry>()
|
||||
|
||||
// Pid 1 is the worker host itself (`process.pid`), so commands start above it.
|
||||
let lastPid = 1
|
||||
|
||||
/**
|
||||
* Reserve one pid before its command starts, so a handle can report it
|
||||
* synchronously.
|
||||
* @returns the new table entry, still without its process.
|
||||
*/
|
||||
export function registerProcess(): WorkerProcessEntry {
|
||||
lastPid += 1
|
||||
const entry: WorkerProcessEntry = { pid: lastPid, signal: undefined, process: undefined }
|
||||
entries.set(entry.pid, entry)
|
||||
return entry
|
||||
}
|
||||
|
||||
/**
|
||||
* Drop one entry once its command has settled.
|
||||
* @param pid - the entry's pid.
|
||||
*/
|
||||
export function releaseProcess(pid: number): void {
|
||||
entries.delete(pid)
|
||||
}
|
||||
|
||||
/**
|
||||
* Whether a command with this pid is still running.
|
||||
* @param pid - pid to look up; a negative value addresses the group, which here
|
||||
* holds exactly the one process that leads it.
|
||||
* @returns true while the entry is in the table.
|
||||
*/
|
||||
export function processAlive(pid: number): boolean {
|
||||
return entries.has(Math.abs(pid))
|
||||
}
|
||||
|
||||
/**
|
||||
* Deliver a signal to one running command.
|
||||
*
|
||||
* `SIGKILL` stops the command whatever it is doing; every other signal asks it
|
||||
* to stop at its next command boundary. That distinction is real only for a
|
||||
* worker-backed process — see {@link RunningProcess.destroy}.
|
||||
* @param pid - pid or negative process-group id.
|
||||
* @param signal - the signal name to record and deliver.
|
||||
* @returns true when an entry received it, false when no such process exists.
|
||||
*/
|
||||
export function signalProcess(pid: number, signal: NodeJS.Signals): boolean {
|
||||
const entry = entries.get(Math.abs(pid))
|
||||
if (entry === undefined) return false
|
||||
entry.signal ??= signal
|
||||
if (signal === 'SIGKILL') entry.process?.destroy()
|
||||
else entry.process?.interrupt()
|
||||
return true
|
||||
}
|
||||
@@ -0,0 +1,23 @@
|
||||
/**
|
||||
* The parsed command line, as this shell names it.
|
||||
*
|
||||
* `@yarnpkg/parsers` re-exports only part of its grammar's type map from the
|
||||
* package root, and its `exports` field forbids reaching the grammar module
|
||||
* directly, so the three missing members are derived from the ones it does
|
||||
* publish. `CommandChain` is `Command` plus an optional pipeline link, which
|
||||
* makes it usable wherever a command node is expected.
|
||||
* @module @deepseek-ai/dsh-experimental-webworker-runtime/src/shell/ast
|
||||
*/
|
||||
|
||||
import type { Argument, CommandChain } from '@yarnpkg/parsers'
|
||||
|
||||
export type { ArgumentSegment, ArithmeticExpression, CommandChain, CommandLine, ShellLine } from '@yarnpkg/parsers'
|
||||
|
||||
/** One command node: a program call, a subshell, a group, or bare assignments. */
|
||||
export type Command = CommandChain
|
||||
|
||||
/** An argument that becomes argv fields. */
|
||||
export type ValueArgument = Extract<Argument, { type: 'argument' }>
|
||||
|
||||
/** An argument that rewires a descriptor instead of becoming argv. */
|
||||
export type RedirectArgument = Extract<Argument, { type: 'redirection' }>
|
||||
@@ -0,0 +1,228 @@
|
||||
/**
|
||||
* Word expansion: one parsed argument becomes the zero or more fields a
|
||||
* program receives in its argv. Covers the segment kinds the grammar produces
|
||||
* — literal text, variables (with `:-` / `:+` forms), command substitution,
|
||||
* arithmetic, and globs matched against the VFS.
|
||||
* @module @deepseek-ai/dsh-experimental-webworker-runtime/src/shell/expand
|
||||
*/
|
||||
|
||||
import picomatch from 'picomatch'
|
||||
import type { ArgumentSegment, ArithmeticExpression, ShellLine, ValueArgument } from './ast.ts'
|
||||
import { resolve } from '../module-system/posix-path.ts'
|
||||
import type { ShellFileSystem, ShellState } from './types.ts'
|
||||
|
||||
/** Characters that make the grammar treat a whole word as a glob pattern. */
|
||||
const GLOB_PATTERN = /[*?]|\[[^\]]*\]/
|
||||
|
||||
/**
|
||||
* Whether one word is a glob the shell should match against the filesystem.
|
||||
* Handed to `parseShell`, which decides between a `text` and a `glob` segment.
|
||||
* @param word - the word exactly as it was written.
|
||||
* @returns true when the word contains a wildcard.
|
||||
*/
|
||||
export function isGlobPattern(word: string): boolean {
|
||||
return GLOB_PATTERN.test(word)
|
||||
}
|
||||
|
||||
/**
|
||||
* Read one variable the way `$name` does.
|
||||
*
|
||||
* Shell variables shadow the environment (an assignment without `export` is
|
||||
* only visible to this shell), and the specials report what a shell without
|
||||
* job control or positional parameters can honestly report.
|
||||
* @param state - the shell state to read.
|
||||
* @param name - variable name, or one of `?`, `$`, `#`, `@`, `*`, `0`.
|
||||
* @returns the value, or undefined when the variable is unset.
|
||||
*/
|
||||
export function readVariable(state: ShellState, name: string): string | undefined {
|
||||
switch (name) {
|
||||
case '?': return String(state.lastStatus)
|
||||
// The worker host runs the whole tree as pid 1; `$$` reports it verbatim.
|
||||
case '$': return '1'
|
||||
case '0': return 'bash'
|
||||
// No positional parameters reach a `bash -c` command line here.
|
||||
case '#': return '0'
|
||||
case '@': case '*': return ''
|
||||
default: return state.variables[name] ?? state.environment[name]
|
||||
}
|
||||
}
|
||||
|
||||
/** Evaluate `$(( … ))`. */
|
||||
function arithmetic(expression: ArithmeticExpression, state: ShellState): number {
|
||||
switch (expression.type) {
|
||||
case 'number': return expression.value
|
||||
case 'variable': return Number.parseInt(readVariable(state, expression.name) ?? '0', 10) || 0
|
||||
case 'addition': return arithmetic(expression.left, state) + arithmetic(expression.right, state)
|
||||
case 'subtraction': return arithmetic(expression.left, state) - arithmetic(expression.right, state)
|
||||
case 'multiplication': return arithmetic(expression.left, state) * arithmetic(expression.right, state)
|
||||
case 'division': return Math.trunc(arithmetic(expression.left, state) / arithmetic(expression.right, state))
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Expand one glob against the filesystem, one path segment at a time.
|
||||
*
|
||||
* Matches keep the pattern's own spelling: a relative pattern yields relative
|
||||
* paths, so `ls *.ts` prints what the model typed.
|
||||
* @param pattern - the glob as written.
|
||||
* @param cwd - directory a relative pattern starts from.
|
||||
* @param fs - the filesystem to match against.
|
||||
* @returns sorted matches, or an empty array when nothing matches.
|
||||
*/
|
||||
export async function expandGlob(pattern: string, cwd: string, fs: ShellFileSystem): Promise<string[]> {
|
||||
const absolute = pattern.startsWith('/')
|
||||
const segments = pattern.split('/').filter(segment => segment !== '')
|
||||
// A glob walks paths that may not exist or may not be directories; both
|
||||
// simply contribute no matches, so listing failures are absorbed here.
|
||||
const safeList = async (path: string): Promise<{ name: string; directory: boolean }[]> => {
|
||||
try {
|
||||
return await fs.list(path)
|
||||
} catch {
|
||||
return []
|
||||
}
|
||||
}
|
||||
// Each frontier entry pairs the directory to search with the prefix that
|
||||
// reproduces the caller's spelling for anything found under it.
|
||||
let frontier: { path: string; display: string }[] = [{ path: absolute ? '/' : cwd, display: absolute ? '/' : '' }]
|
||||
for (const [index, segment] of segments.entries()) {
|
||||
const last = index === segments.length - 1
|
||||
const next: { path: string; display: string }[] = []
|
||||
for (const entry of frontier) {
|
||||
if (segment === '**') {
|
||||
// `**` stands for this directory and every directory below it.
|
||||
const stack = [entry]
|
||||
while (stack.length > 0) {
|
||||
const current = stack.pop() as { path: string; display: string }
|
||||
next.push(current)
|
||||
for (const child of await safeList(current.path)) {
|
||||
if (child.directory) {
|
||||
stack.push({ path: resolve(current.path, child.name), display: `${current.display}${child.name}/` })
|
||||
}
|
||||
}
|
||||
}
|
||||
continue
|
||||
}
|
||||
if (!isGlobPattern(segment)) {
|
||||
const path = resolve(entry.path, segment)
|
||||
if (await fs.stat(path) === undefined) continue
|
||||
next.push({ path, display: `${entry.display}${segment}${last ? '' : '/'}` })
|
||||
continue
|
||||
}
|
||||
const matches = picomatch(segment, { dot: segment.startsWith('.') })
|
||||
for (const child of await safeList(entry.path)) {
|
||||
if (!matches(child.name)) continue
|
||||
if (!last && !child.directory) continue
|
||||
next.push({ path: resolve(entry.path, child.name), display: `${entry.display}${child.name}${last ? '' : '/'}` })
|
||||
}
|
||||
}
|
||||
frontier = next
|
||||
}
|
||||
// A `**` frontier carries trailing separators from its own expansion; the
|
||||
// shell reports directory matches without one.
|
||||
return [...new Set(frontier.map(entry => entry.display.replace(/\/$/, '')))].filter(match => match !== '').sort()
|
||||
}
|
||||
|
||||
/**
|
||||
* Everything expansion needs that the argument itself cannot supply: how to
|
||||
* run a command substitution, and the state variables resolve against.
|
||||
*/
|
||||
export interface ExpansionContext {
|
||||
state: ShellState
|
||||
/** The filesystem globs match against. */
|
||||
fs: ShellFileSystem
|
||||
/**
|
||||
* Run one nested command line and return its standard output.
|
||||
* @param shell - the parsed line inside `$( … )`.
|
||||
* @returns the captured output, with trailing newlines already stripped.
|
||||
*/
|
||||
substitute(shell: ShellLine): Promise<string>
|
||||
}
|
||||
|
||||
/**
|
||||
* Expand one argument into fields.
|
||||
*
|
||||
* Unquoted expansions split on whitespace the way a shell does, so
|
||||
* `cat $FILES` with two names runs `cat` with two arguments while
|
||||
* `cat "$FILES"` runs it with one.
|
||||
* @param argument - the parsed argument.
|
||||
* @param context - substitution hook and shell state.
|
||||
* @returns the fields this argument contributes to argv.
|
||||
*/
|
||||
export async function expandArgument(argument: ValueArgument, context: ExpansionContext): Promise<string[]> {
|
||||
const fields: string[] = []
|
||||
// `undefined` means "no field started yet": an unset unquoted variable must
|
||||
// contribute nothing rather than an empty argument.
|
||||
let current: string | undefined
|
||||
|
||||
const append = (text: string): void => { current = (current ?? '') + text }
|
||||
const appendSplit = (text: string): void => {
|
||||
const parts = text.split(/\s+/)
|
||||
for (const [index, part] of parts.entries()) {
|
||||
if (index > 0) {
|
||||
if (current !== undefined) fields.push(current)
|
||||
current = undefined
|
||||
}
|
||||
if (part !== '') append(part)
|
||||
}
|
||||
}
|
||||
|
||||
for (const segment of argument.segments) {
|
||||
switch (segment.type) {
|
||||
case 'text':
|
||||
append(segment.text)
|
||||
break
|
||||
case 'arithmetic':
|
||||
append(String(arithmetic(segment.arithmetic, context.state)))
|
||||
break
|
||||
case 'variable': {
|
||||
const value = await expandVariable(segment, context)
|
||||
if (segment.quoted) append(value)
|
||||
else appendSplit(value)
|
||||
break
|
||||
}
|
||||
case 'shell': {
|
||||
const output = await context.substitute(segment.shell)
|
||||
if (segment.quoted) append(output)
|
||||
else appendSplit(output)
|
||||
break
|
||||
}
|
||||
case 'glob': {
|
||||
const matches = await expandGlob(segment.pattern, context.state.cwd, context.fs)
|
||||
if (matches.length === 0) {
|
||||
// No match: a POSIX shell passes the pattern through unchanged.
|
||||
append(segment.pattern)
|
||||
break
|
||||
}
|
||||
for (const [index, match] of matches.entries()) {
|
||||
if (index > 0) {
|
||||
fields.push(current as string)
|
||||
current = undefined
|
||||
}
|
||||
append(match)
|
||||
}
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
if (current !== undefined) fields.push(current)
|
||||
return fields
|
||||
}
|
||||
|
||||
/** Resolve one `${name}` segment, including its `:-` and `:+` alternatives. */
|
||||
async function expandVariable(
|
||||
segment: Extract<ArgumentSegment, { type: 'variable' }>,
|
||||
context: ExpansionContext,
|
||||
): Promise<string> {
|
||||
const value = readVariable(context.state, segment.name)
|
||||
const set = value !== undefined && value !== ''
|
||||
if (!set && segment.defaultValue !== undefined) return await joinArguments(segment.defaultValue, context)
|
||||
if (set && segment.alternativeValue !== undefined) return await joinArguments(segment.alternativeValue, context)
|
||||
return value ?? ''
|
||||
}
|
||||
|
||||
/** Expand a `:-` / `:+` operand, which is itself a list of arguments. */
|
||||
async function joinArguments(operand: ValueArgument[], context: ExpansionContext): Promise<string> {
|
||||
const parts: string[] = []
|
||||
for (const argument of operand) parts.push(...await expandArgument(argument, context))
|
||||
return parts.join(' ')
|
||||
}
|
||||
@@ -0,0 +1,120 @@
|
||||
/**
|
||||
* The in-host filesystem for shell runs: {@link ShellFileSystem} straight over
|
||||
* the mounted VFS, plus the path and diagnostic helpers every program shares.
|
||||
*
|
||||
* This implementation answers from memory. A command running in its own
|
||||
* worker uses the message-backed one (`./process/child.ts`), which this one
|
||||
* serves from the host side.
|
||||
* @module @deepseek-ai/dsh-experimental-webworker-runtime/src/shell/fs-access
|
||||
*/
|
||||
|
||||
import { resolve } from '../module-system/posix-path.ts'
|
||||
import { requireActiveVfs } from '../storage/active.ts'
|
||||
import type { VfsError, VfsStats } from '../storage/types.ts'
|
||||
import type { ShellDirent, ShellFileSystem, ShellStats } from './types.ts'
|
||||
|
||||
/**
|
||||
* Resolve one shell word into an absolute VFS path.
|
||||
* @param cwd - the shell's working directory.
|
||||
* @param path - absolute or relative path as the command line spelled it.
|
||||
* @returns the absolute normalized path.
|
||||
*/
|
||||
export function resolveIn(cwd: string, path: string): string {
|
||||
return resolve(cwd, path)
|
||||
}
|
||||
|
||||
/**
|
||||
* Restate a filesystem failure the way a shell utility reports it, so the model
|
||||
* reads `cat: /dsh/none: No such file or directory` instead of a Node error
|
||||
* string.
|
||||
* @param program - the utility's name, used as the message prefix.
|
||||
* @param path - the path the utility was working on.
|
||||
* @param error - the failure the filesystem raised.
|
||||
* @returns the single-line diagnostic, without a trailing newline.
|
||||
*/
|
||||
export function describeFailure(program: string, path: string, error: unknown): string {
|
||||
const code = (error as Partial<VfsError>).code
|
||||
const reason = code === 'ENOENT'
|
||||
? 'No such file or directory'
|
||||
: code === 'ENOTDIR'
|
||||
? 'Not a directory'
|
||||
: code === 'EISDIR'
|
||||
? 'Is a directory'
|
||||
: code === 'ENOTEMPTY'
|
||||
? 'Directory not empty'
|
||||
: code === 'EEXIST'
|
||||
? 'File exists'
|
||||
: error instanceof Error ? error.message : String(error)
|
||||
return `${program}: ${path}: ${reason}`
|
||||
}
|
||||
|
||||
/**
|
||||
* Build a Node-shaped filesystem error, for the conditions this layer detects
|
||||
* itself and for the worker transport, which can carry a code but not a class.
|
||||
* @param code - the Node error code (`ENOENT`, `EISDIR`, …).
|
||||
* @param syscall - the operation that failed.
|
||||
* @param path - the path it failed on.
|
||||
* @returns the error to throw.
|
||||
*/
|
||||
export function filesystemError(code: string, syscall: string, path: string): VfsError {
|
||||
const error = new Error(`${code}: ${syscall} failed, ${syscall} '${path}'`) as VfsError
|
||||
error.code = code
|
||||
error.path = path
|
||||
error.syscall = syscall
|
||||
return error
|
||||
}
|
||||
|
||||
/** Project VFS stats onto the facts a program reads. */
|
||||
function statsOf(stats: VfsStats): ShellStats {
|
||||
return { directory: stats.isDirectory(), size: stats.size, mtimeMs: stats.mtimeMs }
|
||||
}
|
||||
|
||||
/**
|
||||
* The filesystem backed by the VFS mounted in this thread.
|
||||
* @returns the in-host {@link ShellFileSystem}.
|
||||
*/
|
||||
export function hostFileSystem(): ShellFileSystem {
|
||||
const vfs = (): ReturnType<typeof requireActiveVfs> => requireActiveVfs()
|
||||
// oxlint-disable-next-line typescript/require-await -- async face, in-memory backend; see the note below.
|
||||
const stat = async (path: string): Promise<ShellStats | undefined> => {
|
||||
try {
|
||||
return statsOf(vfs().statSync(path) as VfsStats)
|
||||
} catch {
|
||||
// Absence is the answer callers branch on; every other failure mode of
|
||||
// the in-memory backend is also "this path holds nothing readable".
|
||||
return undefined
|
||||
}
|
||||
}
|
||||
// Several members take no await: the face is asynchronous because a process
|
||||
// worker's filesystem is, while this backend answers from memory.
|
||||
/* oxlint-disable typescript/require-await -- see the note above. */
|
||||
return {
|
||||
stat,
|
||||
list: async (path: string): Promise<ShellDirent[]> => {
|
||||
const names = [...vfs().readdirSync(path) as string[]].sort()
|
||||
const entries: ShellDirent[] = []
|
||||
for (const name of names) {
|
||||
entries.push({ name, directory: (await stat(resolve(path, name)))?.directory ?? false })
|
||||
}
|
||||
return entries
|
||||
},
|
||||
readText: async (path: string): Promise<string> => {
|
||||
if ((await stat(path))?.directory === true) throw filesystemError('EISDIR', 'read', path)
|
||||
return vfs().readFileSync(path, 'utf8') as string
|
||||
},
|
||||
writeText: async (path: string, text: string, append = false): Promise<void> => {
|
||||
if (append) vfs().appendFileSync(path, text)
|
||||
else vfs().writeFileSync(path, text)
|
||||
},
|
||||
mkdir: async (path: string, recursive: boolean): Promise<void> => {
|
||||
vfs().mkdirSync(path, { recursive })
|
||||
},
|
||||
remove: async (path: string, options: { recursive: boolean; force: boolean }): Promise<void> => {
|
||||
vfs().rmSync(path, options)
|
||||
},
|
||||
rename: async (from: string, to: string): Promise<void> => {
|
||||
vfs().renameSync(from, to)
|
||||
},
|
||||
}
|
||||
/* oxlint-enable typescript/require-await */
|
||||
}
|
||||
@@ -0,0 +1,389 @@
|
||||
/**
|
||||
* The interpreter: it walks the parsed command line and runs the command table
|
||||
* against the VFS. Structure (`;` `&` `|` `|&` `&&` `||`, subshells, groups,
|
||||
* redirections, prefix assignments) is honored here; what a command *does*
|
||||
* belongs to its program in `./programs/`.
|
||||
*
|
||||
* Output is text, not streams: every program is a JavaScript function that
|
||||
* returns before the next one runs, so a pipeline hands a string along instead
|
||||
* of plumbing byte streams a browser worker has no way to schedule between.
|
||||
* @module @deepseek-ai/dsh-experimental-webworker-runtime/src/shell/interpret
|
||||
*/
|
||||
|
||||
import { parseShell } from '@yarnpkg/parsers'
|
||||
import type { Command, CommandChain, CommandLine, RedirectArgument, ShellLine, ValueArgument } from './ast.ts'
|
||||
import { expandArgument, isGlobPattern } from './expand.ts'
|
||||
import type { ExpansionContext } from './expand.ts'
|
||||
import { describeFailure, hostFileSystem, resolveIn } from './fs-access.ts'
|
||||
import { standardPrograms } from './programs/index.ts'
|
||||
import type { ShellFileSystem, ShellIo, ShellProgram, ShellRunOutcome, ShellState } from './types.ts'
|
||||
|
||||
/** Status a command line reports once the caller's abort signal has fired. */
|
||||
const ABORTED_STATUS = 130
|
||||
|
||||
/** Status of a command name the table does not hold, as POSIX shells report it. */
|
||||
const NOT_FOUND_STATUS = 127
|
||||
|
||||
/** Nesting limit for `$( … )`; a deeper line is a runaway, not a command. */
|
||||
const MAX_SUBSTITUTION_DEPTH = 16
|
||||
|
||||
/** Everything one `bash -c` invocation needs. */
|
||||
export interface ShellRunOptions {
|
||||
/** Working directory the line starts in. */
|
||||
cwd: string
|
||||
/** Environment the line starts with. */
|
||||
env: Record<string, string>
|
||||
/** Standard input contents; absent means empty. */
|
||||
stdin?: string | undefined
|
||||
/** Cancellation: an aborted line stops before its next command. */
|
||||
signal?: AbortSignal | undefined
|
||||
/**
|
||||
* The filesystem this run acts on; defaults to the VFS mounted in this
|
||||
* thread. A run inside a process worker passes the message-backed one.
|
||||
*/
|
||||
fs?: ShellFileSystem | undefined
|
||||
/**
|
||||
* Called with each write as it happens, before the run settles. The returned
|
||||
* outcome still carries the complete text; this only lets a caller that
|
||||
* reports progress (a background job's incremental reads) see output while
|
||||
* the line is still running.
|
||||
*/
|
||||
onOutput?: ((stream: 'stdout' | 'stderr', text: string) => void) | undefined
|
||||
}
|
||||
|
||||
/** Accumulates one output stream. */
|
||||
interface Sink {
|
||||
write: (text: string) => void
|
||||
}
|
||||
|
||||
/** A sink over a string buffer, for pipelines and command substitution. */
|
||||
function buffer(): Sink & { text(): string } {
|
||||
const chunks: string[] = []
|
||||
return {
|
||||
write: (text: string) => { chunks.push(text) },
|
||||
text: () => chunks.join(''),
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Run one shell command line to completion.
|
||||
* @param source - the command source, exactly as `bash -c` would receive it.
|
||||
* @param options - starting directory, environment, standard input, cancellation, filesystem, output callback.
|
||||
* @returns the exit status and the complete standard output and standard error.
|
||||
*/
|
||||
export async function runShellCommand(source: string, options: ShellRunOptions): Promise<ShellRunOutcome> {
|
||||
const run = startRun(options)
|
||||
let line: ShellLine
|
||||
try {
|
||||
line = parseShell(source, { isGlobPattern })
|
||||
} catch (error) {
|
||||
run.io.err(`bash: syntax error: ${error instanceof Error ? error.message.split('\n')[0] : String(error)}\n`)
|
||||
return run.settle(2)
|
||||
}
|
||||
const interpreter = new Interpreter(standardPrograms(), options.fs ?? hostFileSystem(), options.signal)
|
||||
return run.settle(await interpreter.line(line, run.state, run.io))
|
||||
}
|
||||
|
||||
/**
|
||||
* Run one program directly, without a command line to parse.
|
||||
*
|
||||
* This is the path for an argv the caller already has in pieces — a spawn that
|
||||
* names a program instead of handing `bash` a script — so nothing re-quotes
|
||||
* words that were never quoted in the first place.
|
||||
* @param argv - the program name at index 0, then its arguments.
|
||||
* @param options - starting directory, environment, standard input, cancellation, filesystem, output callback.
|
||||
* @returns the exit status and the complete standard output and standard error.
|
||||
*/
|
||||
export async function runShellProgram(argv: readonly string[], options: ShellRunOptions): Promise<ShellRunOutcome> {
|
||||
const run = startRun(options)
|
||||
const name = argv[0]
|
||||
const program = name === undefined ? undefined : standardPrograms().get(name)
|
||||
if (name === undefined || program === undefined) {
|
||||
run.io.err(`bash: ${name ?? ''}: command not found\n`)
|
||||
return run.settle(NOT_FOUND_STATUS)
|
||||
}
|
||||
if (options.signal?.aborted === true) return run.settle(ABORTED_STATUS)
|
||||
try {
|
||||
return run.settle(await program(argv, run.io, run.state, options.fs ?? hostFileSystem()))
|
||||
} catch (error) {
|
||||
run.io.err(`bash: ${name}: ${error instanceof Error ? error.message : String(error)}\n`)
|
||||
return run.settle(1)
|
||||
}
|
||||
}
|
||||
|
||||
/** Build the state, the sinks, and the settlement one run reports through. */
|
||||
function startRun(options: ShellRunOptions): {
|
||||
state: ShellState
|
||||
io: ShellIo
|
||||
settle: (exitCode: number) => ShellRunOutcome
|
||||
} {
|
||||
const stdout = buffer()
|
||||
const stderr = buffer()
|
||||
const report = options.onOutput
|
||||
return {
|
||||
state: {
|
||||
cwd: options.cwd,
|
||||
environment: { ...options.env },
|
||||
variables: {},
|
||||
lastStatus: 0,
|
||||
exitRequested: undefined,
|
||||
signal: options.signal,
|
||||
},
|
||||
io: {
|
||||
stdin: options.stdin ?? '',
|
||||
out: (text: string) => {
|
||||
stdout.write(text)
|
||||
report?.('stdout', text)
|
||||
},
|
||||
err: (text: string) => {
|
||||
stderr.write(text)
|
||||
report?.('stderr', text)
|
||||
},
|
||||
},
|
||||
settle: (exitCode: number) => ({ exitCode, stdout: stdout.text(), stderr: stderr.text() }),
|
||||
}
|
||||
}
|
||||
|
||||
/** One interpretation pass; holds what every nested command shares. */
|
||||
class Interpreter {
|
||||
constructor(
|
||||
private readonly programs: ReadonlyMap<string, ShellProgram>,
|
||||
private readonly fs: ShellFileSystem,
|
||||
private readonly signal: AbortSignal | undefined,
|
||||
private readonly depth = 0,
|
||||
) {}
|
||||
|
||||
/**
|
||||
* Run every command of one line, left to right.
|
||||
* @param line - the parsed line.
|
||||
* @param state - shell state the line reads and mutates.
|
||||
* @param io - standard input and the output sinks.
|
||||
* @returns the status of the last command that ran.
|
||||
*/
|
||||
async line(line: ShellLine, state: ShellState, io: ShellIo): Promise<number> {
|
||||
let status = state.lastStatus
|
||||
for (const entry of line) {
|
||||
if (this.signal?.aborted === true) return ABORTED_STATUS
|
||||
// `&` starts no background job here: the worker has no scheduler that
|
||||
// could run one, so a backgrounded command runs to completion in place.
|
||||
status = await this.commandLine(entry.command, state, io)
|
||||
state.lastStatus = status
|
||||
if (state.exitRequested !== undefined) return state.exitRequested
|
||||
}
|
||||
return status
|
||||
}
|
||||
|
||||
/**
|
||||
* Run one `&&` / `||` chain.
|
||||
*
|
||||
* The grammar nests these to the right, while a shell evaluates them left to
|
||||
* right: `false && a || b` runs `b`. Flattening first is what makes the
|
||||
* skipped `&&` hand its status to the following `||` instead of taking the
|
||||
* whole remainder of the line with it.
|
||||
*/
|
||||
private async commandLine(commandLine: CommandLine, state: ShellState, io: ShellIo): Promise<number> {
|
||||
const links: { type: '&&' | '||'; chain: CommandChain }[] = []
|
||||
for (let current = commandLine.then; current !== undefined; current = current.line.then) {
|
||||
links.push({ type: current.type, chain: current.line.chain })
|
||||
}
|
||||
let status = await this.pipeline(commandLine.chain, state, io)
|
||||
state.lastStatus = status
|
||||
for (const link of links) {
|
||||
if (state.exitRequested !== undefined) return status
|
||||
if (link.type === '&&' ? status !== 0 : status === 0) continue
|
||||
status = await this.pipeline(link.chain, state, io)
|
||||
state.lastStatus = status
|
||||
}
|
||||
return status
|
||||
}
|
||||
|
||||
/** Run one `|` / `|&` pipeline; its status is the last stage's. */
|
||||
private async pipeline(chain: CommandChain, state: ShellState, io: ShellIo): Promise<number> {
|
||||
const stages: { command: CommandChain; mergesStderr: boolean }[] = []
|
||||
for (let current: CommandChain | undefined = chain; current !== undefined;) {
|
||||
const link: CommandChain['then'] = current.then
|
||||
stages.push({ command: current, mergesStderr: link?.type === '|&' })
|
||||
current = link?.chain
|
||||
}
|
||||
let input = io.stdin
|
||||
let status = 0
|
||||
for (const [index, stage] of stages.entries()) {
|
||||
if (this.signal?.aborted === true) return ABORTED_STATUS
|
||||
const last = index === stages.length - 1
|
||||
const piped = buffer()
|
||||
const stageIo: ShellIo = last
|
||||
? { stdin: input, out: io.out, err: io.err }
|
||||
: { stdin: input, out: piped.write, err: stage.mergesStderr ? piped.write : io.err }
|
||||
status = await this.command(stage.command, state, stageIo)
|
||||
if (!last) input = piped.text()
|
||||
if (state.exitRequested !== undefined) return status
|
||||
}
|
||||
return status
|
||||
}
|
||||
|
||||
/** Run one command node: a program call, a subshell, a group, or bare assignments. */
|
||||
private async command(command: Command, state: ShellState, io: ShellIo): Promise<number> {
|
||||
switch (command.type) {
|
||||
case 'envs':
|
||||
for (const env of command.envs) assign(state, env.name, await this.assignedValue(env.args[0], state))
|
||||
return 0
|
||||
case 'subshell': {
|
||||
// A subshell sees a copy: its `cd` and its assignments die with it.
|
||||
const nested = { ...state, environment: { ...state.environment }, variables: { ...state.variables } }
|
||||
return await this.redirected(command.args, state, io, async inner => await this.line(command.subshell, nested, inner))
|
||||
}
|
||||
case 'group':
|
||||
return await this.redirected(command.args, state, io, async inner => await this.line(command.group, state, inner))
|
||||
case 'command':
|
||||
return await this.program(command, state, io)
|
||||
}
|
||||
}
|
||||
|
||||
/** Expand a command's words and run the program they name. */
|
||||
private async program(command: Extract<Command, { type: 'command' }>, state: ShellState, io: ShellIo): Promise<number> {
|
||||
const argv: string[] = []
|
||||
const redirections: RedirectArgument[] = []
|
||||
for (const argument of command.args) {
|
||||
if (argument.type === 'redirection') {
|
||||
redirections.push(argument)
|
||||
continue
|
||||
}
|
||||
argv.push(...await expandArgument(argument, this.context(state)))
|
||||
}
|
||||
const prefix: Record<string, string> = {}
|
||||
for (const env of command.envs) prefix[env.name] = await this.assignedValue(env.args[0], state)
|
||||
|
||||
if (argv.length === 0) {
|
||||
for (const [name, value] of Object.entries(prefix)) assign(state, name, value)
|
||||
return 0
|
||||
}
|
||||
// A prefixed command sees the assignments as environment for its run only,
|
||||
// which also means it cannot change the caller's directory.
|
||||
const scope = Object.keys(prefix).length === 0
|
||||
? state
|
||||
: { ...state, environment: { ...state.environment, ...prefix } }
|
||||
|
||||
const name = argv[0] as string
|
||||
const program = this.programs.get(name)
|
||||
if (program === undefined) {
|
||||
io.err(`bash: ${name}: command not found\n`)
|
||||
return NOT_FOUND_STATUS
|
||||
}
|
||||
return await this.redirected(redirections, state, io, async (inner) => {
|
||||
try {
|
||||
return await program(argv, inner, scope, this.fs)
|
||||
} catch (error) {
|
||||
// A program's own defect must not take the whole worker down with it.
|
||||
inner.err(`bash: ${name}: ${error instanceof Error ? error.message : String(error)}\n`)
|
||||
return 1
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
/**
|
||||
* Apply redirections around one body, then restore nothing: every sink is a
|
||||
* value, so the caller's own `io` is untouched by construction.
|
||||
*/
|
||||
private async redirected(
|
||||
redirections: readonly RedirectArgument[],
|
||||
state: ShellState,
|
||||
io: ShellIo,
|
||||
body: (io: ShellIo) => Promise<number>,
|
||||
): Promise<number> {
|
||||
let stdin = io.stdin
|
||||
let out = io.out
|
||||
let err = io.err
|
||||
// Every file write this redirection set started, awaited before the
|
||||
// command's status is reported: a `> file` must be complete on return.
|
||||
const writes: Promise<void>[] = []
|
||||
for (const redirection of redirections) {
|
||||
const targets: string[] = []
|
||||
for (const argument of redirection.args) targets.push(...await expandArgument(argument, this.context(state)))
|
||||
const target = targets[0]
|
||||
if (target === undefined || targets.length > 1) {
|
||||
io.err('bash: ambiguous redirect\n')
|
||||
return 1
|
||||
}
|
||||
try {
|
||||
switch (redirection.subtype) {
|
||||
case '<':
|
||||
stdin = await this.fs.readText(resolveIn(state.cwd, target))
|
||||
break
|
||||
case '<<<':
|
||||
stdin = `${target}\n`
|
||||
break
|
||||
case '>':
|
||||
case '>>': {
|
||||
const path = resolveIn(state.cwd, target)
|
||||
// Truncation happens at redirect time, so `> file` empties it even
|
||||
// when the command writes nothing.
|
||||
if (redirection.subtype === '>') await this.fs.writeText(path, '')
|
||||
// Appends are ordered by the queue below: a sink is synchronous to
|
||||
// its caller, so writes are chained rather than raced.
|
||||
let pending: Promise<void> = Promise.resolve()
|
||||
const sink = (text: string): void => {
|
||||
pending = pending.then(async () => { await this.fs.writeText(path, text, true) })
|
||||
writes.push(pending)
|
||||
}
|
||||
if (redirection.fd === 2) err = sink
|
||||
else out = sink
|
||||
break
|
||||
}
|
||||
case '>&': {
|
||||
// Only descriptor duplication between stdout and stderr is
|
||||
// meaningful here: those are the only two the shell owns.
|
||||
if (redirection.fd === 2 && target === '1') err = out
|
||||
else if ((redirection.fd === null || redirection.fd === 1) && target === '2') out = err
|
||||
else {
|
||||
io.err(`bash: ${String(redirection.fd ?? 1)}>&${target}: unsupported descriptor redirection\n`)
|
||||
return 1
|
||||
}
|
||||
break
|
||||
}
|
||||
case '<&':
|
||||
io.err(`bash: <&${target}: unsupported descriptor redirection\n`)
|
||||
return 1
|
||||
}
|
||||
} catch (error) {
|
||||
io.err(`${describeFailure('bash', resolveIn(state.cwd, target), error)}\n`)
|
||||
return 1
|
||||
}
|
||||
}
|
||||
const status = await body({ stdin, out, err })
|
||||
await Promise.all(writes)
|
||||
return status
|
||||
}
|
||||
|
||||
/** The expansion hook: `$( … )` runs on a nested interpreter of the same table. */
|
||||
private context(state: ShellState): ExpansionContext {
|
||||
return {
|
||||
state,
|
||||
fs: this.fs,
|
||||
substitute: async (shell: ShellLine): Promise<string> => {
|
||||
if (this.depth >= MAX_SUBSTITUTION_DEPTH) {
|
||||
throw new Error(`command substitution nested deeper than ${String(MAX_SUBSTITUTION_DEPTH)} levels`)
|
||||
}
|
||||
const captured = buffer()
|
||||
const nested = { ...state, environment: { ...state.environment }, variables: { ...state.variables } }
|
||||
const inner = new Interpreter(this.programs, this.fs, this.signal, this.depth + 1)
|
||||
await inner.line(shell, nested, { stdin: '', out: captured.write, err: () => {} })
|
||||
return captured.text().replace(/\n+$/, '')
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
/** Expand the right-hand side of one `NAME=value` assignment. */
|
||||
private async assignedValue(argument: ValueArgument | undefined, state: ShellState): Promise<string> {
|
||||
if (argument === undefined) return ''
|
||||
return (await expandArgument(argument, this.context(state))).join(' ')
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Record one assignment. An exported name keeps its export (the environment
|
||||
* copy is what programs read); anything else stays a shell variable.
|
||||
*/
|
||||
function assign(state: ShellState, name: string, value: string): void {
|
||||
if (name in state.environment) state.environment[name] = value
|
||||
else state.variables[name] = value
|
||||
}
|
||||
@@ -0,0 +1,94 @@
|
||||
/**
|
||||
* The process worker's own half: a fresh worker that received a
|
||||
* {@link ShellStartFrame} runs one command here and then closes.
|
||||
*
|
||||
* It mounts no VFS image, boots no Cordis tree, and loads no plugins — the
|
||||
* only thing it shares with the host worker is the bundle it was started from.
|
||||
* Its filesystem is the host's, reached by message.
|
||||
* @module @deepseek-ai/dsh-experimental-webworker-runtime/src/shell/process/child
|
||||
*/
|
||||
|
||||
import { runShellCommand, runShellProgram } from '../interpret.ts'
|
||||
import { filesystemError } from '../fs-access.ts'
|
||||
import type { ShellDirent, ShellFileSystem, ShellStats } from '../types.ts'
|
||||
import type { FilesystemOperation, FromProcessFrame, ShellStartFrame, ToProcessFrame } from './protocol.ts'
|
||||
|
||||
/** The messaging face this module needs from a worker scope. */
|
||||
export interface ProcessScope {
|
||||
postMessage(frame: FromProcessFrame): void
|
||||
addEventListener(type: 'message', listener: (event: MessageEvent) => void): void
|
||||
close(): void
|
||||
}
|
||||
|
||||
/**
|
||||
* Run one command as this worker's whole purpose, then close.
|
||||
*
|
||||
* Output is forwarded as it is written, so a caller reading a background job
|
||||
* sees progress before the command settles.
|
||||
* @param start - the frame that named the command, its directory, and its input.
|
||||
* @param scope - the worker scope to message through (`self`).
|
||||
*/
|
||||
export function runShellProcess(start: ShellStartFrame, scope: ProcessScope): void {
|
||||
const pending = new Map<number, { settle: (value: unknown) => void; fail: (error: unknown) => void }>()
|
||||
const stopping = new AbortController()
|
||||
let nextCall = 0
|
||||
|
||||
scope.addEventListener('message', (event: MessageEvent) => {
|
||||
const frame = event.data as ToProcessFrame
|
||||
if (frame.t === 'shell-signal') {
|
||||
// The host's first termination rung: the command stops at its next
|
||||
// command boundary. A command that ignores it gets terminated instead.
|
||||
stopping.abort(new Error('killed by signal'))
|
||||
return
|
||||
}
|
||||
if (frame.t !== 'fs-reply') return
|
||||
const waiting = pending.get(frame.id)
|
||||
if (waiting === undefined) return
|
||||
pending.delete(frame.id)
|
||||
if (frame.failure === undefined) waiting.settle(frame.value)
|
||||
else waiting.fail(filesystemError(frame.failure.code ?? 'EIO', 'fs', frame.failure.message))
|
||||
})
|
||||
|
||||
const call = async (op: FilesystemOperation, args: readonly unknown[]): Promise<unknown> => {
|
||||
nextCall += 1
|
||||
const id = nextCall
|
||||
const reply = new Promise<unknown>((settle, fail) => { pending.set(id, { settle, fail }) })
|
||||
scope.postMessage({ t: 'fs-call', id, op, args })
|
||||
return await reply
|
||||
}
|
||||
|
||||
const fs: ShellFileSystem = {
|
||||
stat: async (path: string) => await call('stat', [path]) as ShellStats | undefined,
|
||||
list: async (path: string) => await call('list', [path]) as ShellDirent[],
|
||||
readText: async (path: string) => await call('readText', [path]) as string,
|
||||
writeText: async (path: string, text: string, append = false) => { await call('writeText', [path, text, append]) },
|
||||
mkdir: async (path: string, recursive: boolean) => { await call('mkdir', [path, recursive]) },
|
||||
remove: async (path: string, options: { recursive: boolean; force: boolean }) => { await call('remove', [path, options]) },
|
||||
rename: async (from: string, to: string) => { await call('rename', [from, to]) },
|
||||
}
|
||||
|
||||
const options = {
|
||||
cwd: start.cwd,
|
||||
env: start.env,
|
||||
stdin: start.stdin,
|
||||
signal: stopping.signal,
|
||||
fs,
|
||||
onOutput: (stream: 'stdout' | 'stderr', text: string) => { scope.postMessage({ t: 'shell-out', stream, text }) },
|
||||
}
|
||||
const run = start.script === undefined
|
||||
? runShellProgram(start.argv, options)
|
||||
: runShellCommand(start.script, options)
|
||||
void run.then(
|
||||
(outcome) => {
|
||||
scope.postMessage({ t: 'shell-exit', code: outcome.exitCode })
|
||||
scope.close()
|
||||
},
|
||||
(error: unknown) => {
|
||||
// The interpreter contains its own failures; reaching here means the
|
||||
// shell machinery itself broke, which the host reports as a failed spawn.
|
||||
scope.postMessage({ t: 'shell-out', stream: 'stderr', text: `bash: ${String(error)}\n` })
|
||||
scope.postMessage({ t: 'shell-exit', code: 1 })
|
||||
scope.close()
|
||||
},
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,181 @@
|
||||
/**
|
||||
* Starting and supervising shell processes from the host worker.
|
||||
*
|
||||
* A process is a Web Worker started from this same bundle, told by its first
|
||||
* frame to be a shell process rather than a host. That is what buys real
|
||||
* process semantics in a browser: the command runs off the host's thread, and
|
||||
* `terminate()` stops it even mid-loop — the one thing a cooperative in-thread
|
||||
* interpreter can never do.
|
||||
*
|
||||
* Where no `Worker` constructor exists (a Node test host), the same command
|
||||
* runs inline on this thread. Everything except preemption behaves the same,
|
||||
* and the difference is named rather than hidden: {@link RunningProcess.destroy}
|
||||
* can only ask an inline command to stop.
|
||||
* @module @deepseek-ai/dsh-experimental-webworker-runtime/src/shell/process/host
|
||||
*/
|
||||
|
||||
import { runShellCommand, runShellProgram } from '../interpret.ts'
|
||||
import { hostFileSystem } from '../fs-access.ts'
|
||||
import type { ShellFileSystem } from '../types.ts'
|
||||
import type { FilesystemOperation, FromProcessFrame, ShellStartFrame } from './protocol.ts'
|
||||
import { runShellProcess } from './child.ts'
|
||||
import type { ProcessScope } from './child.ts'
|
||||
|
||||
/** What the caller must supply to start one process. */
|
||||
export interface ProcessStartOptions {
|
||||
/** Command source for `bash -c`, or undefined when `argv` names a program. */
|
||||
script?: string | undefined
|
||||
/** The program and its arguments. */
|
||||
argv: readonly string[]
|
||||
/** Working directory the command starts in. */
|
||||
cwd: string
|
||||
/** Environment the command starts with. */
|
||||
env: Record<string, string>
|
||||
/** Everything on standard input. */
|
||||
stdin: string
|
||||
/** Receives output as it is produced. */
|
||||
onOutput: (stream: 'stdout' | 'stderr', text: string) => void
|
||||
/** Receives the settled status exactly once. */
|
||||
onExit: (code: number) => void
|
||||
/** The filesystem the command acts on; defaults to the mounted VFS. */
|
||||
fs?: ShellFileSystem | undefined
|
||||
}
|
||||
|
||||
/** A started command, from the host's side. */
|
||||
export interface RunningProcess {
|
||||
/** Ask the command to stop at its next command boundary (the `SIGTERM` rung). */
|
||||
interrupt(): void
|
||||
/**
|
||||
* Stop the command now (the `SIGKILL` rung). A worker-backed process dies
|
||||
* whatever it was doing; an inline one can only be asked, because nothing
|
||||
* can preempt a synchronous loop on its own thread.
|
||||
*/
|
||||
destroy(): void
|
||||
}
|
||||
|
||||
/** Whether this thread can start a real process worker. */
|
||||
function canSpawnWorker(): boolean {
|
||||
return typeof Worker === 'function' && typeof self !== 'undefined' && typeof self.location.href === 'string'
|
||||
}
|
||||
|
||||
/**
|
||||
* Start one command.
|
||||
* @param options - the command, its environment, and the sinks for its output and status.
|
||||
* @returns the handle the process table signals through.
|
||||
*/
|
||||
export function startProcess(options: ProcessStartOptions): RunningProcess {
|
||||
return canSpawnWorker() ? startWorkerProcess(options) : startInlineProcess(options)
|
||||
}
|
||||
|
||||
/** Serve one filesystem call for a process worker. */
|
||||
async function serveFilesystemCall(fs: ShellFileSystem, op: FilesystemOperation, args: readonly unknown[]): Promise<unknown> {
|
||||
switch (op) {
|
||||
case 'stat': return await fs.stat(args[0] as string)
|
||||
case 'list': return await fs.list(args[0] as string)
|
||||
case 'readText': return await fs.readText(args[0] as string)
|
||||
case 'writeText':
|
||||
await fs.writeText(args[0] as string, args[1] as string, args[2] as boolean)
|
||||
return undefined
|
||||
case 'mkdir':
|
||||
await fs.mkdir(args[0] as string, args[1] as boolean)
|
||||
return undefined
|
||||
case 'remove':
|
||||
await fs.remove(args[0] as string, args[1] as { recursive: boolean; force: boolean })
|
||||
return undefined
|
||||
case 'rename':
|
||||
await fs.rename(args[0] as string, args[1] as string)
|
||||
return undefined
|
||||
default:
|
||||
// The op crossed a worker postMessage: a name the union does not carry
|
||||
// must fail the call rather than answer `{ value: undefined }`.
|
||||
throw new Error(`webworker shell: unknown filesystem op ${String(op)}`)
|
||||
}
|
||||
}
|
||||
|
||||
/** The worker-backed process: a second copy of this bundle, running one command. */
|
||||
function startWorkerProcess(options: ProcessStartOptions): RunningProcess {
|
||||
const fs = options.fs ?? hostFileSystem()
|
||||
// Same bundle, different role: the first frame decides. Starting from this
|
||||
// worker's own URL keeps the deployment free of a second static asset and of
|
||||
// the build-order trap a sibling artifact would bring.
|
||||
const worker = new Worker(self.location.href, { type: 'module' })
|
||||
let settled = false
|
||||
const settle = (code: number): void => {
|
||||
if (settled) return
|
||||
settled = true
|
||||
worker.terminate()
|
||||
options.onExit(code)
|
||||
}
|
||||
|
||||
worker.addEventListener('message', (event: MessageEvent) => {
|
||||
const frame = event.data as FromProcessFrame
|
||||
if (frame.t === 'shell-out') {
|
||||
options.onOutput(frame.stream, frame.text)
|
||||
return
|
||||
}
|
||||
if (frame.t === 'shell-exit') {
|
||||
settle(frame.code)
|
||||
return
|
||||
}
|
||||
void serveFilesystemCall(fs, frame.op, frame.args).then(
|
||||
(value) => { worker.postMessage({ t: 'fs-reply', id: frame.id, value }) },
|
||||
(error: unknown) => {
|
||||
const failure = {
|
||||
code: (error as { code?: string }).code,
|
||||
message: error instanceof Error ? error.message : String(error),
|
||||
}
|
||||
worker.postMessage({ t: 'fs-reply', id: frame.id, failure })
|
||||
},
|
||||
)
|
||||
})
|
||||
worker.addEventListener('error', (event: ErrorEvent) => {
|
||||
options.onOutput('stderr', `bash: process worker failed: ${event.message}\n`)
|
||||
settle(1)
|
||||
})
|
||||
|
||||
const start: ShellStartFrame = {
|
||||
t: 'shell-start',
|
||||
script: options.script,
|
||||
argv: options.argv,
|
||||
cwd: options.cwd,
|
||||
env: options.env,
|
||||
stdin: options.stdin,
|
||||
}
|
||||
worker.postMessage(start)
|
||||
|
||||
return {
|
||||
interrupt: () => { if (!settled) worker.postMessage({ t: 'shell-signal' }) },
|
||||
// The reason this whole module exists: a worker dies on command, even
|
||||
// mid-loop, so a timeout is enforceable rather than advisory.
|
||||
destroy: () => { settle(130) },
|
||||
}
|
||||
}
|
||||
|
||||
/** The inline process: the same command on this thread, stoppable only by asking. */
|
||||
function startInlineProcess(options: ProcessStartOptions): RunningProcess {
|
||||
const stopping = new AbortController()
|
||||
const runOptions = {
|
||||
cwd: options.cwd,
|
||||
env: options.env,
|
||||
stdin: options.stdin,
|
||||
signal: stopping.signal,
|
||||
fs: options.fs ?? hostFileSystem(),
|
||||
onOutput: options.onOutput,
|
||||
}
|
||||
const run = options.script === undefined
|
||||
? runShellProgram(options.argv, runOptions)
|
||||
: runShellCommand(options.script, runOptions)
|
||||
void run.then(
|
||||
(outcome) => { options.onExit(outcome.exitCode) },
|
||||
(error: unknown) => {
|
||||
options.onOutput('stderr', `bash: ${String(error)}\n`)
|
||||
options.onExit(1)
|
||||
},
|
||||
)
|
||||
const stop = (): void => { stopping.abort(new Error('killed by signal')) }
|
||||
return { interrupt: stop, destroy: stop }
|
||||
}
|
||||
|
||||
/** Re-exported for the worker entry, which decides its role from the first frame. */
|
||||
export { runShellProcess }
|
||||
export type { ProcessScope }
|
||||
@@ -0,0 +1,82 @@
|
||||
/**
|
||||
* The frames a shell process and its host exchange.
|
||||
*
|
||||
* A command runs in its own Web Worker, which owns no filesystem: the VFS
|
||||
* stays in the host worker and every read or write is a request on this
|
||||
* channel. Blocking the child on a reply is impossible here (that would need
|
||||
* `SharedArrayBuffer`, which requires a cross-origin isolation this deployment
|
||||
* cannot have), so the filesystem face is asynchronous end to end.
|
||||
* @module @deepseek-ai/dsh-experimental-webworker-runtime/src/shell/process/protocol
|
||||
*/
|
||||
|
||||
/** The first frame a process worker receives; it also selects its role. */
|
||||
export interface ShellStartFrame {
|
||||
t: 'shell-start'
|
||||
/** Command source for `bash -c`, or undefined when `argv` names a program directly. */
|
||||
script?: string | undefined
|
||||
/** The program and arguments, used when `script` is absent. */
|
||||
argv: readonly string[]
|
||||
/** Working directory the command starts in. */
|
||||
cwd: string
|
||||
/** Environment the command starts with. */
|
||||
env: Record<string, string>
|
||||
/** Everything on standard input. */
|
||||
stdin: string
|
||||
}
|
||||
|
||||
/** Output produced by the command, forwarded as it is written. */
|
||||
export interface ShellOutputFrame {
|
||||
t: 'shell-out'
|
||||
stream: 'stdout' | 'stderr'
|
||||
text: string
|
||||
}
|
||||
|
||||
/** The command settled; the process worker closes itself right after. */
|
||||
export interface ShellExitFrame {
|
||||
t: 'shell-exit'
|
||||
code: number
|
||||
}
|
||||
|
||||
/** A signal the command is asked to honor (the terminate ladder's first rung). */
|
||||
export interface ShellSignalFrame {
|
||||
t: 'shell-signal'
|
||||
}
|
||||
|
||||
/** Filesystem operations a process worker can ask its host to perform. */
|
||||
export type FilesystemOperation = 'stat' | 'list' | 'readText' | 'writeText' | 'mkdir' | 'remove' | 'rename'
|
||||
|
||||
/** One filesystem call, awaiting its reply by `id`. */
|
||||
export interface FilesystemCallFrame {
|
||||
t: 'fs-call'
|
||||
id: number
|
||||
op: FilesystemOperation
|
||||
args: readonly unknown[]
|
||||
}
|
||||
|
||||
/**
|
||||
* One filesystem reply. A failure carries the Node error `code` because the
|
||||
* utilities branch on it (`ENOENT` prints "No such file or directory"), and an
|
||||
* Error instance does not survive structured cloning with its class.
|
||||
*/
|
||||
export interface FilesystemReplyFrame {
|
||||
t: 'fs-reply'
|
||||
id: number
|
||||
value?: unknown
|
||||
failure?: { code?: string | undefined; message: string }
|
||||
}
|
||||
|
||||
/** Everything the host sends to a process worker. */
|
||||
export type ToProcessFrame = ShellStartFrame | ShellSignalFrame | FilesystemReplyFrame
|
||||
|
||||
/** Everything a process worker sends to its host. */
|
||||
export type FromProcessFrame = ShellOutputFrame | ShellExitFrame | FilesystemCallFrame
|
||||
|
||||
/**
|
||||
* Whether a message is the frame that turns a fresh worker into a shell
|
||||
* process. The host worker's entry reads this to pick its role.
|
||||
* @param data - the raw message payload.
|
||||
* @returns true when the payload starts a shell process.
|
||||
*/
|
||||
export function isShellStartFrame(data: unknown): data is ShellStartFrame {
|
||||
return typeof data === 'object' && data !== null && (data as { t?: unknown }).t === 'shell-start'
|
||||
}
|
||||
@@ -0,0 +1,192 @@
|
||||
/**
|
||||
* Shell builtins: the programs that read or change the shell's own state
|
||||
* (directory, environment, exit status) rather than the filesystem.
|
||||
* @module @deepseek-ai/dsh-experimental-webworker-runtime/src/shell/programs/builtins
|
||||
*/
|
||||
|
||||
import { readVariable } from '../expand.ts'
|
||||
import { resolveIn } from '../fs-access.ts'
|
||||
import type { ShellProgram, ShellStats } from '../types.ts'
|
||||
import { parseOptions } from './options.ts'
|
||||
|
||||
/** Status a command reports when a signal ended it, as a shell renders `128 + SIGINT`. */
|
||||
const SIGNAL_EXIT_STATUS = 130
|
||||
|
||||
const cd: ShellProgram = async (argv, io, state, fs) => {
|
||||
const target = argv[1] ?? state.environment['HOME'] ?? '/'
|
||||
const path = target === '-' ? state.variables['OLDPWD'] ?? state.cwd : resolveIn(state.cwd, target)
|
||||
const stats = await fs.stat(path)
|
||||
if (stats === undefined) {
|
||||
io.err(`cd: ${target}: No such file or directory\n`)
|
||||
return 1
|
||||
}
|
||||
if (!stats.directory) {
|
||||
io.err(`cd: ${target}: Not a directory\n`)
|
||||
return 1
|
||||
}
|
||||
state.variables['OLDPWD'] = state.cwd
|
||||
state.cwd = path
|
||||
// `$PWD` is what scripts read back, so it has to follow the real directory.
|
||||
if ('PWD' in state.environment) state.environment['PWD'] = path
|
||||
return 0
|
||||
}
|
||||
|
||||
const pwd: ShellProgram = (_argv, io, state) => {
|
||||
io.out(`${state.cwd}\n`)
|
||||
return 0
|
||||
}
|
||||
|
||||
const exportProgram: ShellProgram = (argv, io, state) => {
|
||||
const options = parseOptions(argv)
|
||||
if (options.operands.length === 0) {
|
||||
for (const [name, value] of Object.entries(state.environment).sort()) io.out(`declare -x ${name}="${value}"\n`)
|
||||
return 0
|
||||
}
|
||||
for (const operand of options.operands) {
|
||||
const separator = operand.indexOf('=')
|
||||
if (separator < 0) {
|
||||
// Exporting an existing shell variable moves it into the environment.
|
||||
state.environment[operand] = state.variables[operand] ?? state.environment[operand] ?? ''
|
||||
continue
|
||||
}
|
||||
state.environment[operand.slice(0, separator)] = operand.slice(separator + 1)
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
const unset: ShellProgram = (argv, _io, state) => {
|
||||
const removed = new Set(argv.slice(1))
|
||||
const without = (source: Record<string, string>): Record<string, string> =>
|
||||
Object.fromEntries(Object.entries(source).filter(([name]) => !removed.has(name)))
|
||||
state.environment = without(state.environment)
|
||||
state.variables = without(state.variables)
|
||||
return 0
|
||||
}
|
||||
|
||||
const env: ShellProgram = (_argv, io, state) => {
|
||||
for (const [name, value] of Object.entries(state.environment).sort()) io.out(`${name}=${value}\n`)
|
||||
return 0
|
||||
}
|
||||
|
||||
const exitProgram: ShellProgram = (argv, _io, state) => {
|
||||
const status = argv[1] === undefined ? state.lastStatus : Number.parseInt(argv[1], 10) || 0
|
||||
state.exitRequested = status
|
||||
return status
|
||||
}
|
||||
|
||||
/** `test` / `[`: the file and string predicates a generated command line uses. */
|
||||
const test: ShellProgram = async (argv, io, state, fs) => {
|
||||
const words = argv[0] === '[' ? argv.slice(1, argv[argv.length - 1] === ']' ? -1 : undefined) : argv.slice(1)
|
||||
const status = (value: boolean): number => value ? 0 : 1
|
||||
const statOf = async (operand: string): Promise<ShellStats | undefined> => await fs.stat(resolveIn(state.cwd, operand))
|
||||
if (words.length === 1) return status((words[0] as string) !== '')
|
||||
if (words.length === 2) {
|
||||
const operator = words[0] as string
|
||||
const operand = words[1] ?? ''
|
||||
switch (operator) {
|
||||
case '-e': return status(await statOf(operand) !== undefined)
|
||||
case '-f': return status((await statOf(operand))?.directory === false)
|
||||
case '-d': return status((await statOf(operand))?.directory === true)
|
||||
case '-s': return status(((await statOf(operand))?.size ?? 0) > 0)
|
||||
case '-r': case '-w': return status(await statOf(operand) !== undefined)
|
||||
case '-z': return status(operand === '')
|
||||
case '-n': return status(operand !== '')
|
||||
case '!': return status(operand === '')
|
||||
default:
|
||||
io.err(`test: ${operator}: unsupported unary operator\n`)
|
||||
return 2
|
||||
}
|
||||
}
|
||||
if (words.length === 3) {
|
||||
const [left, operator, right] = words as [string, string, string]
|
||||
switch (operator) {
|
||||
case '=': case '==': return status(left === right)
|
||||
case '!=': return status(left !== right)
|
||||
case '-eq': return status(Number(left) === Number(right))
|
||||
case '-ne': return status(Number(left) !== Number(right))
|
||||
case '-lt': return status(Number(left) < Number(right))
|
||||
case '-le': return status(Number(left) <= Number(right))
|
||||
case '-gt': return status(Number(left) > Number(right))
|
||||
case '-ge': return status(Number(left) >= Number(right))
|
||||
default:
|
||||
io.err(`test: ${operator}: unsupported binary operator\n`)
|
||||
return 2
|
||||
}
|
||||
}
|
||||
io.err('test: unsupported expression\n')
|
||||
return 2
|
||||
}
|
||||
|
||||
const sleep: ShellProgram = async (argv, io, state) => {
|
||||
const seconds = Number.parseFloat(argv[1] ?? '')
|
||||
if (!Number.isFinite(seconds) || seconds < 0) {
|
||||
io.err(`sleep: invalid time interval '${argv[1] ?? ''}'\n`)
|
||||
return 2
|
||||
}
|
||||
// A killed command must settle at once: waiting out the full interval would
|
||||
// keep the caller's process handle open long after its signal arrived.
|
||||
const killed = await new Promise<boolean>((settle) => {
|
||||
const timer = setTimeout(() => {
|
||||
state.signal?.removeEventListener('abort', onAbort)
|
||||
settle(false)
|
||||
}, seconds * 1000)
|
||||
function onAbort(): void {
|
||||
clearTimeout(timer)
|
||||
settle(true)
|
||||
}
|
||||
if (state.signal?.aborted === true) onAbort()
|
||||
else state.signal?.addEventListener('abort', onAbort, { once: true })
|
||||
})
|
||||
return killed ? SIGNAL_EXIT_STATUS : 0
|
||||
}
|
||||
|
||||
const date: ShellProgram = (_argv, io) => {
|
||||
io.out(`${new Date().toISOString()}\n`)
|
||||
return 0
|
||||
}
|
||||
|
||||
const seq: ShellProgram = (argv, io) => {
|
||||
const numbers = argv.slice(1).map(value => Number.parseInt(value, 10))
|
||||
const [first, second, third] = numbers
|
||||
const from = numbers.length > 1 ? first as number : 1
|
||||
const step = numbers.length > 2 ? second as number : 1
|
||||
const to = numbers.length > 2 ? third as number : numbers.length > 1 ? second as number : first
|
||||
if (to === undefined || !Number.isFinite(to) || step === 0) {
|
||||
io.err('seq: expected numeric bounds\n')
|
||||
return 2
|
||||
}
|
||||
for (let value = from; step > 0 ? value <= to : value >= to; value += step) io.out(`${String(value)}\n`)
|
||||
return 0
|
||||
}
|
||||
|
||||
/** `printenv NAME`, which scripts prefer over `echo $NAME` when the name is computed. */
|
||||
const printenv: ShellProgram = (argv, io, state) => {
|
||||
const name = argv[1]
|
||||
if (name === undefined) {
|
||||
for (const [key, value] of Object.entries(state.environment).sort()) io.out(`${key}=${value}\n`)
|
||||
return 0
|
||||
}
|
||||
const value = readVariable(state, name)
|
||||
if (value === undefined) return 1
|
||||
io.out(`${value}\n`)
|
||||
return 0
|
||||
}
|
||||
|
||||
/** The state builtins, keyed by the name a command line uses. */
|
||||
export const BUILTIN_PROGRAMS: Readonly<Record<string, ShellProgram>> = {
|
||||
cd,
|
||||
pwd,
|
||||
export: exportProgram,
|
||||
unset,
|
||||
env,
|
||||
printenv,
|
||||
exit: exitProgram,
|
||||
test,
|
||||
'[': test,
|
||||
sleep,
|
||||
date,
|
||||
seq,
|
||||
'true': () => 0,
|
||||
'false': () => 1,
|
||||
':': () => 0,
|
||||
}
|
||||
@@ -0,0 +1,293 @@
|
||||
/**
|
||||
* File and directory utilities of the command table, all of them over the
|
||||
* shell's filesystem. Listings print one entry per line: nothing here is ever
|
||||
* a terminal, so the column layout a real `ls` picks for a tty would only be
|
||||
* noise in a tool result.
|
||||
* @module @deepseek-ai/dsh-experimental-webworker-runtime/src/shell/programs/files
|
||||
*/
|
||||
|
||||
import picomatch from 'picomatch'
|
||||
import { basename, dirname, resolve } from '../../module-system/posix-path.ts'
|
||||
import { describeFailure, resolveIn } from '../fs-access.ts'
|
||||
import type { ShellFileSystem, ShellProgram, ShellStats } from '../types.ts'
|
||||
import { parseOptions } from './options.ts'
|
||||
|
||||
/** Format one entry the way `ls -l` does, with the facts the VFS actually holds. */
|
||||
function longEntry(stats: ShellStats | undefined, name: string): string {
|
||||
const size = String(stats?.size ?? 0).padStart(8)
|
||||
const modified = new Date(stats?.mtimeMs ?? 0).toISOString().replace('T', ' ').slice(0, 16)
|
||||
return `${stats?.directory === true ? 'drwxr-xr-x' : '-rw-r--r--'} ${size} ${modified} ${name}`
|
||||
}
|
||||
|
||||
const ls: ShellProgram = async (argv, io, state, fs) => {
|
||||
const options = parseOptions(argv)
|
||||
const operands = options.operands.length > 0 ? options.operands : ['.']
|
||||
let status = 0
|
||||
for (const [index, operand] of operands.entries()) {
|
||||
const path = resolveIn(state.cwd, operand)
|
||||
const stats = await fs.stat(path)
|
||||
if (stats === undefined) {
|
||||
io.err(`ls: ${operand}: No such file or directory\n`)
|
||||
status = 2
|
||||
continue
|
||||
}
|
||||
if (operands.length > 1) io.out(`${index > 0 ? '\n' : ''}${operand}:\n`)
|
||||
if (!stats.directory) {
|
||||
io.out(`${options.flags.has('l') ? longEntry(stats, operand) : operand}\n`)
|
||||
continue
|
||||
}
|
||||
const entries = (await fs.list(path)).filter(entry => options.flags.has('a') || !entry.name.startsWith('.'))
|
||||
for (const entry of entries) {
|
||||
const shown = options.flags.has('l')
|
||||
? longEntry(await fs.stat(resolve(path, entry.name)), entry.name)
|
||||
: entry.name
|
||||
io.out(`${shown}\n`)
|
||||
}
|
||||
}
|
||||
return status
|
||||
}
|
||||
|
||||
const find: ShellProgram = async (argv, io, state, fs) => {
|
||||
// `find` spells multi-letter predicates with one dash, which the shared
|
||||
// option parser would read as bundled short flags; this walk reads them.
|
||||
const roots: string[] = []
|
||||
let namePattern: string | undefined
|
||||
let kind: string | undefined
|
||||
let maxDepth = Number.POSITIVE_INFINITY
|
||||
const words = argv.slice(1)
|
||||
for (let index = 0; index < words.length; index += 1) {
|
||||
const word = words[index] as string
|
||||
if (word === '-name') { index += 1; namePattern = words[index]; continue }
|
||||
if (word === '-type') { index += 1; kind = words[index]; continue }
|
||||
if (word === '-maxdepth') { index += 1; maxDepth = Number.parseInt(words[index] ?? '', 10); continue }
|
||||
if (word.startsWith('-')) {
|
||||
io.err(`find: unsupported predicate ${word}\n`)
|
||||
return 2
|
||||
}
|
||||
roots.push(word)
|
||||
}
|
||||
const matches = namePattern === undefined ? undefined : picomatch(namePattern, { dot: true })
|
||||
let status = 0
|
||||
const visit = async (path: string, display: string, depth: number): Promise<void> => {
|
||||
const stats = await fs.stat(path)
|
||||
if (stats === undefined) {
|
||||
io.err(`find: ${display}: No such file or directory\n`)
|
||||
status = 1
|
||||
return
|
||||
}
|
||||
const selected = (matches === undefined || matches(basename(display)))
|
||||
&& (kind === undefined || (kind === 'd') === stats.directory)
|
||||
if (selected) io.out(`${display}\n`)
|
||||
if (!stats.directory || depth >= maxDepth) return
|
||||
for (const entry of await fs.list(path)) {
|
||||
await visit(resolve(path, entry.name), `${display === '/' ? '' : display}/${entry.name}`, depth + 1)
|
||||
}
|
||||
}
|
||||
for (const root of roots.length > 0 ? roots : ['.']) await visit(resolveIn(state.cwd, root), root, 0)
|
||||
return status
|
||||
}
|
||||
|
||||
const mkdir: ShellProgram = async (argv, io, state, fs) => {
|
||||
const options = parseOptions(argv)
|
||||
let status = 0
|
||||
for (const operand of options.operands) {
|
||||
try {
|
||||
await fs.mkdir(resolveIn(state.cwd, operand), options.flags.has('p'))
|
||||
} catch (error) {
|
||||
io.err(`${describeFailure('mkdir', operand, error)}\n`)
|
||||
status = 1
|
||||
}
|
||||
}
|
||||
return status
|
||||
}
|
||||
|
||||
const rmdir: ShellProgram = async (argv, io, state, fs) => {
|
||||
const options = parseOptions(argv)
|
||||
let status = 0
|
||||
for (const operand of options.operands) {
|
||||
const path = resolveIn(state.cwd, operand)
|
||||
if ((await fs.list(path)).length > 0) {
|
||||
io.err(`rmdir: ${operand}: Directory not empty\n`)
|
||||
status = 1
|
||||
continue
|
||||
}
|
||||
try {
|
||||
await fs.remove(path, { recursive: true, force: false })
|
||||
} catch (error) {
|
||||
io.err(`${describeFailure('rmdir', operand, error)}\n`)
|
||||
status = 1
|
||||
}
|
||||
}
|
||||
return status
|
||||
}
|
||||
|
||||
const rm: ShellProgram = async (argv, io, state, fs) => {
|
||||
const options = parseOptions(argv)
|
||||
const recursive = options.flags.has('r') || options.flags.has('R')
|
||||
const force = options.flags.has('f')
|
||||
let status = 0
|
||||
for (const operand of options.operands) {
|
||||
const path = resolveIn(state.cwd, operand)
|
||||
const stats = await fs.stat(path)
|
||||
if (stats === undefined) {
|
||||
if (force) continue
|
||||
io.err(`rm: ${operand}: No such file or directory\n`)
|
||||
status = 1
|
||||
continue
|
||||
}
|
||||
if (stats.directory && !recursive) {
|
||||
io.err(`rm: ${operand}: Is a directory\n`)
|
||||
status = 1
|
||||
continue
|
||||
}
|
||||
try {
|
||||
await fs.remove(path, { recursive, force })
|
||||
} catch (error) {
|
||||
io.err(`${describeFailure('rm', operand, error)}\n`)
|
||||
status = 1
|
||||
}
|
||||
}
|
||||
return status
|
||||
}
|
||||
|
||||
/** Copy one file or one whole subtree. */
|
||||
async function copyTree(from: string, to: string, fs: ShellFileSystem): Promise<void> {
|
||||
const stats = await fs.stat(from)
|
||||
if (stats?.directory !== true) {
|
||||
await fs.writeText(to, await fs.readText(from))
|
||||
return
|
||||
}
|
||||
await fs.mkdir(to, true)
|
||||
for (const entry of await fs.list(from)) await copyTree(resolve(from, entry.name), resolve(to, entry.name), fs)
|
||||
}
|
||||
|
||||
/** Resolve the real destination of a copy or move: into a directory, or onto a path. */
|
||||
async function destinationFor(target: string, source: string, fs: ShellFileSystem): Promise<string> {
|
||||
return (await fs.stat(target))?.directory === true ? resolve(target, basename(source)) : target
|
||||
}
|
||||
|
||||
const cp: ShellProgram = async (argv, io, state, fs) => {
|
||||
const options = parseOptions(argv)
|
||||
const sources = options.operands.slice(0, -1)
|
||||
const target = options.operands[options.operands.length - 1]
|
||||
if (target === undefined || sources.length === 0) {
|
||||
io.err('cp: expected a source and a destination\n')
|
||||
return 2
|
||||
}
|
||||
const targetPath = resolveIn(state.cwd, target)
|
||||
let status = 0
|
||||
for (const source of sources) {
|
||||
const sourcePath = resolveIn(state.cwd, source)
|
||||
const stats = await fs.stat(sourcePath)
|
||||
if (stats === undefined) {
|
||||
io.err(`cp: ${source}: No such file or directory\n`)
|
||||
status = 1
|
||||
continue
|
||||
}
|
||||
if (stats.directory && !(options.flags.has('r') || options.flags.has('R'))) {
|
||||
io.err(`cp: ${source}: Is a directory\n`)
|
||||
status = 1
|
||||
continue
|
||||
}
|
||||
try {
|
||||
await copyTree(sourcePath, await destinationFor(targetPath, source, fs), fs)
|
||||
} catch (error) {
|
||||
io.err(`${describeFailure('cp', source, error)}\n`)
|
||||
status = 1
|
||||
}
|
||||
}
|
||||
return status
|
||||
}
|
||||
|
||||
const mv: ShellProgram = async (argv, io, state, fs) => {
|
||||
const options = parseOptions(argv)
|
||||
const sources = options.operands.slice(0, -1)
|
||||
const target = options.operands[options.operands.length - 1]
|
||||
if (target === undefined || sources.length === 0) {
|
||||
io.err('mv: expected a source and a destination\n')
|
||||
return 2
|
||||
}
|
||||
const targetPath = resolveIn(state.cwd, target)
|
||||
let status = 0
|
||||
for (const source of sources) {
|
||||
try {
|
||||
await fs.rename(resolveIn(state.cwd, source), await destinationFor(targetPath, source, fs))
|
||||
} catch (error) {
|
||||
io.err(`${describeFailure('mv', source, error)}\n`)
|
||||
status = 1
|
||||
}
|
||||
}
|
||||
return status
|
||||
}
|
||||
|
||||
const touch: ShellProgram = async (argv, io, state, fs) => {
|
||||
const options = parseOptions(argv)
|
||||
let status = 0
|
||||
for (const operand of options.operands) {
|
||||
const path = resolveIn(state.cwd, operand)
|
||||
try {
|
||||
// Rewriting the existing bytes is what advances the VFS timestamp.
|
||||
await fs.writeText(path, await fs.stat(path) === undefined ? '' : await fs.readText(path))
|
||||
} catch (error) {
|
||||
io.err(`${describeFailure('touch', operand, error)}\n`)
|
||||
status = 1
|
||||
}
|
||||
}
|
||||
return status
|
||||
}
|
||||
|
||||
const stat: ShellProgram = async (argv, io, state, fs) => {
|
||||
const options = parseOptions(argv)
|
||||
let status = 0
|
||||
for (const operand of options.operands) {
|
||||
const path = resolveIn(state.cwd, operand)
|
||||
const stats = await fs.stat(path)
|
||||
if (stats === undefined) {
|
||||
io.err(`stat: ${operand}: No such file or directory\n`)
|
||||
status = 1
|
||||
continue
|
||||
}
|
||||
io.out(`${path} ${stats.directory ? 'directory' : 'file'} ${String(stats.size)} ${new Date(stats.mtimeMs).toISOString()}\n`)
|
||||
}
|
||||
return status
|
||||
}
|
||||
|
||||
const dirnameProgram: ShellProgram = (argv, io) => {
|
||||
for (const operand of argv.slice(1)) io.out(`${dirname(operand)}\n`)
|
||||
return argv.length > 1 ? 0 : 2
|
||||
}
|
||||
|
||||
const basenameProgram: ShellProgram = (argv, io) => {
|
||||
const [, path, suffix] = argv
|
||||
if (path === undefined) {
|
||||
io.err('basename: expected a path\n')
|
||||
return 2
|
||||
}
|
||||
io.out(`${basename(path, suffix)}\n`)
|
||||
return 0
|
||||
}
|
||||
|
||||
/** Refuse a utility whose effect the VFS cannot represent at all. */
|
||||
const unavailable = (name: string): ShellProgram => (_argv, io) => {
|
||||
io.err(`${name}: not available in the worker host\n`)
|
||||
return 127
|
||||
}
|
||||
|
||||
/** The file utilities, keyed by the name a command line uses. */
|
||||
export const FILE_PROGRAMS: Readonly<Record<string, ShellProgram>> = {
|
||||
ls,
|
||||
find,
|
||||
mkdir,
|
||||
rmdir,
|
||||
rm,
|
||||
cp,
|
||||
mv,
|
||||
touch,
|
||||
stat,
|
||||
dirname: dirnameProgram,
|
||||
basename: basenameProgram,
|
||||
// Symbolic links have no representation in the VFS; refusing is honest and
|
||||
// keeps a script from believing it created one.
|
||||
ln: unavailable('ln'),
|
||||
readlink: unavailable('readlink'),
|
||||
}
|
||||
@@ -0,0 +1,45 @@
|
||||
/**
|
||||
* The command table: every program name this shell can run. A browser worker
|
||||
* spawns no processes, so this table IS the machine's `/bin` — a name that is
|
||||
* not here reports `command not found`, exactly as a real shell would for a
|
||||
* binary that is not installed.
|
||||
* @module @deepseek-ai/dsh-experimental-webworker-runtime/src/shell/programs
|
||||
*/
|
||||
|
||||
import type { ShellProgram } from '../types.ts'
|
||||
import { BUILTIN_PROGRAMS } from './builtins.ts'
|
||||
import { FILE_PROGRAMS } from './files.ts'
|
||||
import { TEXT_PROGRAMS } from './text.ts'
|
||||
|
||||
let table: Map<string, ShellProgram> | undefined
|
||||
|
||||
/**
|
||||
* The standard command table, built once and shared by every command line.
|
||||
* @returns the program table, keyed by command name.
|
||||
*/
|
||||
export function standardPrograms(): ReadonlyMap<string, ShellProgram> {
|
||||
table ??= new Map<string, ShellProgram>([
|
||||
...Object.entries(BUILTIN_PROGRAMS),
|
||||
...Object.entries(FILE_PROGRAMS),
|
||||
...Object.entries(TEXT_PROGRAMS),
|
||||
['which', which],
|
||||
])
|
||||
return table
|
||||
}
|
||||
|
||||
/** Reports which of the requested names this shell can run. */
|
||||
const which: ShellProgram = (argv, io) => {
|
||||
const known = standardPrograms()
|
||||
let status = 0
|
||||
for (const name of argv.slice(1)) {
|
||||
// Every program is built into the shell, so a known name reports itself
|
||||
// instead of a path that would not exist in the VFS.
|
||||
if (known.has(name)) {
|
||||
io.out(`${name}: shell built-in command\n`)
|
||||
continue
|
||||
}
|
||||
io.err(`which: no ${name} in the worker host command table\n`)
|
||||
status = 1
|
||||
}
|
||||
return status
|
||||
}
|
||||
@@ -0,0 +1,96 @@
|
||||
/**
|
||||
* Argument splitting shared by the command table: short flags (bundled or
|
||||
* separate), long flags, `--`, and the operands that follow.
|
||||
* @module @deepseek-ai/dsh-experimental-webworker-runtime/src/shell/programs/options
|
||||
*/
|
||||
|
||||
/** One parsed argv: which flags were given, and what is left to act on. */
|
||||
export interface ParsedOptions {
|
||||
/** Every short letter and long name seen, without their dashes. */
|
||||
readonly flags: ReadonlySet<string>
|
||||
/** Values of flags that take one (`-n 5` and `--name=x` both land here). */
|
||||
readonly values: ReadonlyMap<string, string>
|
||||
/** Everything that is not a flag, in order. */
|
||||
readonly operands: readonly string[]
|
||||
}
|
||||
|
||||
/**
|
||||
* Split one program's arguments.
|
||||
*
|
||||
* A short letter listed in `valued` consumes the rest of its token (`-n5`) or
|
||||
* the next argument (`-n 5`); every other letter is a plain flag, so `-rn`
|
||||
* sets both `r` and `n`.
|
||||
* @param argv - the program's argv, including its name at index 0.
|
||||
* @param valued - short letters that take a value.
|
||||
* @returns the flags, their values, and the operands.
|
||||
*/
|
||||
export function parseOptions(argv: readonly string[], valued: ReadonlySet<string> = new Set()): ParsedOptions {
|
||||
const flags = new Set<string>()
|
||||
const values = new Map<string, string>()
|
||||
const operands: string[] = []
|
||||
const rest = argv.slice(1)
|
||||
let literal = false
|
||||
for (let index = 0; index < rest.length; index += 1) {
|
||||
const argument = rest[index] as string
|
||||
if (literal || argument === '-' || !argument.startsWith('-')) {
|
||||
operands.push(argument)
|
||||
continue
|
||||
}
|
||||
if (argument === '--') {
|
||||
literal = true
|
||||
continue
|
||||
}
|
||||
if (argument.startsWith('--')) {
|
||||
const [name, value] = splitLong(argument.slice(2))
|
||||
flags.add(name)
|
||||
if (value !== undefined) values.set(name, value)
|
||||
continue
|
||||
}
|
||||
for (let cursor = 1; cursor < argument.length; cursor += 1) {
|
||||
const letter = argument[cursor] as string
|
||||
flags.add(letter)
|
||||
if (!valued.has(letter)) continue
|
||||
const inline = argument.slice(cursor + 1)
|
||||
if (inline !== '') {
|
||||
values.set(letter, inline)
|
||||
} else {
|
||||
index += 1
|
||||
values.set(letter, rest[index] ?? '')
|
||||
}
|
||||
break
|
||||
}
|
||||
}
|
||||
return { flags, values, operands }
|
||||
}
|
||||
|
||||
/** Split `name=value`; a long flag without `=` has no value. */
|
||||
function splitLong(text: string): [string, string | undefined] {
|
||||
const separator = text.indexOf('=')
|
||||
return separator < 0 ? [text, undefined] : [text.slice(0, separator), text.slice(separator + 1)]
|
||||
}
|
||||
|
||||
/**
|
||||
* Read a numeric flag value.
|
||||
* @param options - the parsed options.
|
||||
* @param flag - the short letter to read.
|
||||
* @param fallback - value to use when the flag is absent or unparsable.
|
||||
* @returns the number the caller should use.
|
||||
*/
|
||||
export function numberOption(options: ParsedOptions, flag: string, fallback: number): number {
|
||||
const raw = options.values.get(flag)
|
||||
if (raw === undefined) return fallback
|
||||
const parsed = Number.parseInt(raw, 10)
|
||||
return Number.isFinite(parsed) ? parsed : fallback
|
||||
}
|
||||
|
||||
/**
|
||||
* Split text into lines for the line-oriented utilities.
|
||||
* @param text - the text to split.
|
||||
* @returns its lines, without the trailing empty line a final newline creates.
|
||||
*/
|
||||
export function toLines(text: string): string[] {
|
||||
if (text === '') return []
|
||||
const lines = text.split('\n')
|
||||
if (lines[lines.length - 1] === '') lines.pop()
|
||||
return lines
|
||||
}
|
||||
@@ -0,0 +1,363 @@
|
||||
/**
|
||||
* Text utilities of the command table. Each one reads its operands as files
|
||||
* and falls back to standard input, the way its POSIX counterpart does.
|
||||
* @module @deepseek-ai/dsh-experimental-webworker-runtime/src/shell/programs/text
|
||||
*/
|
||||
|
||||
import { describeFailure, resolveIn } from '../fs-access.ts'
|
||||
import type { ShellFileSystem, ShellIo, ShellProgram, ShellState } from '../types.ts'
|
||||
import { numberOption, parseOptions, toLines } from './options.ts'
|
||||
|
||||
/**
|
||||
* Read every operand as a file, reporting the ones that fail.
|
||||
* @param program - name used in diagnostics.
|
||||
* @param operands - paths to read; empty means standard input.
|
||||
* @param io - source of standard input and sink for diagnostics.
|
||||
* @param state - shell state supplying the working directory.
|
||||
* @param fs - the filesystem to read from.
|
||||
* @returns one entry per readable source and the status the program should report.
|
||||
*/
|
||||
async function readInputs(
|
||||
program: string,
|
||||
operands: readonly string[],
|
||||
io: ShellIo,
|
||||
state: ShellState,
|
||||
fs: ShellFileSystem,
|
||||
): Promise<{ sources: { name: string; text: string }[]; status: number }> {
|
||||
if (operands.length === 0) return { sources: [{ name: '-', text: io.stdin }], status: 0 }
|
||||
const sources: { name: string; text: string }[] = []
|
||||
let status = 0
|
||||
for (const operand of operands) {
|
||||
if (operand === '-') {
|
||||
sources.push({ name: '-', text: io.stdin })
|
||||
continue
|
||||
}
|
||||
const path = resolveIn(state.cwd, operand)
|
||||
try {
|
||||
sources.push({ name: operand, text: await fs.readText(path) })
|
||||
} catch (error) {
|
||||
io.err(`${describeFailure(program, operand, error)}\n`)
|
||||
status = 1
|
||||
}
|
||||
}
|
||||
return { sources, status }
|
||||
}
|
||||
|
||||
/** Append a trailing newline unless the text already ends with one. */
|
||||
function terminated(text: string): string {
|
||||
return text === '' || text.endsWith('\n') ? text : `${text}\n`
|
||||
}
|
||||
|
||||
const echo: ShellProgram = (argv, io) => {
|
||||
const suppressNewline = argv[1] === '-n'
|
||||
const words = argv.slice(suppressNewline ? 2 : 1)
|
||||
io.out(`${words.join(' ')}${suppressNewline ? '' : '\n'}`)
|
||||
return 0
|
||||
}
|
||||
|
||||
const printf: ShellProgram = (argv, io) => {
|
||||
const format = argv[1] ?? ''
|
||||
const operands = argv.slice(2)
|
||||
let cursor = 0
|
||||
// The conversions a shell script realistically uses; anything else is left
|
||||
// verbatim so the output shows what was not understood.
|
||||
const rendered = format.replace(/%[sdi%]/g, (match) => {
|
||||
if (match === '%%') return '%'
|
||||
const value = operands[cursor] ?? ''
|
||||
cursor += 1
|
||||
if (match === '%s') return value
|
||||
const parsed = Number.parseInt(value, 10)
|
||||
return String(Number.isFinite(parsed) ? parsed : 0)
|
||||
})
|
||||
io.out(rendered.replace(/\\n/g, '\n').replace(/\\t/g, '\t'))
|
||||
return 0
|
||||
}
|
||||
|
||||
const cat: ShellProgram = async (argv, io, state, fs) => {
|
||||
const options = parseOptions(argv)
|
||||
const { sources, status } = await readInputs('cat', options.operands, io, state, fs)
|
||||
let line = 1
|
||||
for (const source of sources) {
|
||||
if (!options.flags.has('n')) {
|
||||
io.out(source.text)
|
||||
continue
|
||||
}
|
||||
for (const content of toLines(source.text)) {
|
||||
io.out(`${String(line).padStart(6)}\t${content}\n`)
|
||||
line += 1
|
||||
}
|
||||
}
|
||||
return status
|
||||
}
|
||||
|
||||
const head: ShellProgram = async (argv, io, state, fs) => {
|
||||
const options = parseOptions(argv, new Set(['n']))
|
||||
const count = numberOption(options, 'n', 10)
|
||||
const { sources, status } = await readInputs('head', options.operands, io, state, fs)
|
||||
for (const [index, source] of sources.entries()) {
|
||||
if (sources.length > 1) io.out(`${index > 0 ? '\n' : ''}==> ${source.name} <==\n`)
|
||||
io.out(terminated(toLines(source.text).slice(0, count).join('\n')))
|
||||
}
|
||||
return status
|
||||
}
|
||||
|
||||
const tail: ShellProgram = async (argv, io, state, fs) => {
|
||||
const options = parseOptions(argv, new Set(['n']))
|
||||
const count = numberOption(options, 'n', 10)
|
||||
const { sources, status } = await readInputs('tail', options.operands, io, state, fs)
|
||||
for (const [index, source] of sources.entries()) {
|
||||
if (sources.length > 1) io.out(`${index > 0 ? '\n' : ''}==> ${source.name} <==\n`)
|
||||
io.out(terminated(toLines(source.text).slice(-count).join('\n')))
|
||||
}
|
||||
return status
|
||||
}
|
||||
|
||||
const wc: ShellProgram = async (argv, io, state, fs) => {
|
||||
const options = parseOptions(argv)
|
||||
const { sources, status } = await readInputs('wc', options.operands, io, state, fs)
|
||||
const selected = ['l', 'w', 'c'].filter(flag => options.flags.has(flag))
|
||||
const columns = selected.length > 0 ? selected : ['l', 'w', 'c']
|
||||
for (const source of sources) {
|
||||
const counts: Record<string, number> = {
|
||||
l: toLines(source.text).length,
|
||||
w: source.text.split(/\s+/).filter(word => word !== '').length,
|
||||
c: source.text.length,
|
||||
}
|
||||
const cells = columns.map(column => String(counts[column] ?? 0).padStart(columns.length > 1 ? 8 : 1))
|
||||
io.out(`${cells.join(' ')}${source.name === '-' ? '' : ` ${source.name}`}\n`)
|
||||
}
|
||||
return status
|
||||
}
|
||||
|
||||
/** Collect every file under one directory, for `grep -r`. */
|
||||
async function walkFiles(
|
||||
path: string,
|
||||
display: string,
|
||||
into: { path: string; display: string }[],
|
||||
fs: ShellFileSystem,
|
||||
): Promise<void> {
|
||||
for (const entry of await fs.list(path)) {
|
||||
const child = `${path.endsWith('/') ? path : `${path}/`}${entry.name}`
|
||||
const shown = `${display.endsWith('/') ? display : `${display}/`}${entry.name}`
|
||||
if (entry.directory) await walkFiles(child, shown, into, fs)
|
||||
else into.push({ path: child, display: shown })
|
||||
}
|
||||
}
|
||||
|
||||
const grep: ShellProgram = async (argv, io, state, fs) => {
|
||||
const options = parseOptions(argv, new Set(['e']))
|
||||
const pattern = options.values.get('e') ?? options.operands[0]
|
||||
const targets = options.values.get('e') === undefined ? options.operands.slice(1) : options.operands
|
||||
if (pattern === undefined) {
|
||||
io.err('grep: no pattern given\n')
|
||||
return 2
|
||||
}
|
||||
// Patterns are JavaScript regular expressions; `-F` matches them literally.
|
||||
const source = options.flags.has('F') ? pattern.replace(/[.*+?^${}()|[\]\\]/g, '\\$&') : pattern
|
||||
let matcher: RegExp
|
||||
try {
|
||||
matcher = new RegExp(source, options.flags.has('i') ? 'i' : '')
|
||||
} catch (error) {
|
||||
io.err(`grep: invalid pattern: ${error instanceof Error ? error.message : String(error)}\n`)
|
||||
return 2
|
||||
}
|
||||
|
||||
const sources: { name: string; text: string }[] = []
|
||||
let status = 0
|
||||
if (targets.length === 0) {
|
||||
sources.push({ name: '', text: io.stdin })
|
||||
} else {
|
||||
for (const target of targets) {
|
||||
const path = resolveIn(state.cwd, target)
|
||||
const stats = await fs.stat(path)
|
||||
if (stats?.directory === true) {
|
||||
if (!options.flags.has('r') && !options.flags.has('R')) {
|
||||
io.err(`grep: ${target}: Is a directory\n`)
|
||||
status = Math.max(status, 2)
|
||||
continue
|
||||
}
|
||||
const files: { path: string; display: string }[] = []
|
||||
await walkFiles(path, target, files, fs)
|
||||
for (const file of files) sources.push({ name: file.display, text: await fs.readText(file.path) })
|
||||
continue
|
||||
}
|
||||
try {
|
||||
sources.push({ name: target, text: await fs.readText(path) })
|
||||
} catch (error) {
|
||||
io.err(`${describeFailure('grep', target, error)}\n`)
|
||||
status = Math.max(status, 2)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const label = sources.length > 1 || options.flags.has('H')
|
||||
let matched = false
|
||||
for (const entry of sources) {
|
||||
const hits = toLines(entry.text)
|
||||
.map((text, index) => ({ text, number: index + 1 }))
|
||||
.filter(line => matcher.test(line.text) !== options.flags.has('v'))
|
||||
if (hits.length > 0) matched = true
|
||||
if (options.flags.has('l')) {
|
||||
if (hits.length > 0) io.out(`${entry.name}\n`)
|
||||
continue
|
||||
}
|
||||
if (options.flags.has('c')) {
|
||||
io.out(`${label && entry.name !== '' ? `${entry.name}:` : ''}${String(hits.length)}\n`)
|
||||
continue
|
||||
}
|
||||
for (const hit of hits) {
|
||||
const prefix = `${label && entry.name !== '' ? `${entry.name}:` : ''}${options.flags.has('n') ? `${String(hit.number)}:` : ''}`
|
||||
io.out(`${prefix}${hit.text}\n`)
|
||||
}
|
||||
}
|
||||
// `grep` reports "nothing matched" as status 1, distinct from an error.
|
||||
return status !== 0 ? status : matched ? 0 : 1
|
||||
}
|
||||
|
||||
const sort: ShellProgram = async (argv, io, state, fs) => {
|
||||
const options = parseOptions(argv)
|
||||
const { sources, status } = await readInputs('sort', options.operands, io, state, fs)
|
||||
let lines = sources.flatMap(source => toLines(source.text))
|
||||
lines = options.flags.has('n')
|
||||
? [...lines].sort((left, right) => (Number.parseFloat(left) || 0) - (Number.parseFloat(right) || 0))
|
||||
: [...lines].sort((left, right) => left < right ? -1 : left > right ? 1 : 0)
|
||||
if (options.flags.has('r')) lines.reverse()
|
||||
if (options.flags.has('u')) lines = [...new Set(lines)]
|
||||
io.out(terminated(lines.join('\n')))
|
||||
return status
|
||||
}
|
||||
|
||||
const uniq: ShellProgram = async (argv, io, state, fs) => {
|
||||
const options = parseOptions(argv)
|
||||
const { sources, status } = await readInputs('uniq', options.operands, io, state, fs)
|
||||
const lines = sources.flatMap(source => toLines(source.text))
|
||||
const groups: { text: string; count: number }[] = []
|
||||
for (const line of lines) {
|
||||
const previous = groups[groups.length - 1]
|
||||
if (previous !== undefined && previous.text === line) previous.count += 1
|
||||
else groups.push({ text: line, count: 1 })
|
||||
}
|
||||
const selected = options.flags.has('d')
|
||||
? groups.filter(group => group.count > 1)
|
||||
: options.flags.has('u') ? groups.filter(group => group.count === 1) : groups
|
||||
for (const group of selected) {
|
||||
io.out(`${options.flags.has('c') ? `${String(group.count).padStart(7)} ` : ''}${group.text}\n`)
|
||||
}
|
||||
return status
|
||||
}
|
||||
|
||||
const cut: ShellProgram = async (argv, io, state, fs) => {
|
||||
const options = parseOptions(argv, new Set(['d', 'f', 'c']))
|
||||
const delimiter = options.values.get('d') ?? '\t'
|
||||
const fields = (options.values.get('f') ?? '').split(',').map(field => Number.parseInt(field, 10)).filter(Number.isFinite)
|
||||
const characters = options.values.get('c')
|
||||
const { sources, status } = await readInputs('cut', options.operands, io, state, fs)
|
||||
if (fields.length === 0 && characters === undefined) {
|
||||
io.err('cut: expected -f or -c\n')
|
||||
return 2
|
||||
}
|
||||
for (const source of sources) {
|
||||
for (const line of toLines(source.text)) {
|
||||
if (characters !== undefined) {
|
||||
const [from, to] = characters.split('-')
|
||||
const start = Number.parseInt(from ?? '1', 10) || 1
|
||||
const end = to === undefined || to === '' ? start : Number.parseInt(to, 10)
|
||||
io.out(`${line.slice(start - 1, end)}\n`)
|
||||
continue
|
||||
}
|
||||
const parts = line.split(delimiter)
|
||||
io.out(`${fields.map(field => parts[field - 1] ?? '').join(delimiter)}\n`)
|
||||
}
|
||||
}
|
||||
return status
|
||||
}
|
||||
|
||||
/** Expand one `tr` set: `a-z` becomes every character in that range. */
|
||||
function characterSet(set: string): string[] {
|
||||
// oxlint-disable-next-line typescript/no-misused-spread -- a `tr` set names characters, and code points are that unit.
|
||||
const characters = [...set]
|
||||
const expanded: string[] = []
|
||||
for (let index = 0; index < characters.length; index += 1) {
|
||||
const start = characters[index] as string
|
||||
const end = characters[index + 2]
|
||||
if (characters[index + 1] === '-' && end !== undefined) {
|
||||
for (let code = start.codePointAt(0) as number; code <= (end.codePointAt(0) as number); code += 1) {
|
||||
expanded.push(String.fromCodePoint(code))
|
||||
}
|
||||
index += 2
|
||||
continue
|
||||
}
|
||||
expanded.push(start)
|
||||
}
|
||||
return expanded
|
||||
}
|
||||
|
||||
const tr: ShellProgram = (argv, io) => {
|
||||
const options = parseOptions(argv)
|
||||
const [fromSet, toSet] = options.operands
|
||||
const from = fromSet === undefined ? undefined : characterSet(fromSet).join('')
|
||||
const to = toSet === undefined ? undefined : characterSet(toSet).join('')
|
||||
if (from === undefined) {
|
||||
io.err('tr: expected a source set\n')
|
||||
return 2
|
||||
}
|
||||
if (options.flags.has('d')) {
|
||||
// oxlint-disable-next-line typescript/no-misused-spread -- `tr` deletes per character, and code points are the unit it deletes.
|
||||
io.out([...io.stdin].filter(character => !from.includes(character)).join(''))
|
||||
return 0
|
||||
}
|
||||
if (to === undefined) {
|
||||
io.err('tr: expected a replacement set\n')
|
||||
return 2
|
||||
}
|
||||
// oxlint-disable-next-line typescript/no-misused-spread -- `tr` translates per character, and code points are the unit it maps.
|
||||
io.out([...io.stdin].map((character) => {
|
||||
const index = from.indexOf(character)
|
||||
return index < 0 ? character : to[Math.min(index, to.length - 1)] as string
|
||||
}).join(''))
|
||||
return 0
|
||||
}
|
||||
|
||||
/** `sed` accepts only the substitute command; anything else is reported, not guessed at. */
|
||||
const sed: ShellProgram = async (argv, io, state, fs) => {
|
||||
const options = parseOptions(argv, new Set(['e']))
|
||||
const script = options.values.get('e') ?? options.operands[0]
|
||||
const targets = options.values.get('e') === undefined ? options.operands.slice(1) : options.operands
|
||||
const parsed = /^s(.)(.*?[^\\])?\1(.*?)\1([gi]*)$/.exec(script ?? '')
|
||||
if (parsed === null) {
|
||||
io.err('sed: only substitution scripts (s/pattern/replacement/) run in the worker host\n')
|
||||
return 2
|
||||
}
|
||||
const [, , pattern = '', replacement = '', modifiers = ''] = parsed
|
||||
let matcher: RegExp
|
||||
try {
|
||||
matcher = new RegExp(pattern, modifiers.includes('g') ? `g${modifiers.replace('g', '')}` : modifiers)
|
||||
} catch (error) {
|
||||
io.err(`sed: invalid pattern: ${error instanceof Error ? error.message : String(error)}\n`)
|
||||
return 2
|
||||
}
|
||||
const { sources, status } = await readInputs('sed', targets, io, state, fs)
|
||||
for (const source of sources) {
|
||||
for (const line of toLines(source.text)) io.out(`${line.replace(matcher, replacement.replace(/\\(\d)/g, '$$$1'))}\n`)
|
||||
}
|
||||
return status
|
||||
}
|
||||
|
||||
const tee: ShellProgram = async (argv, io, state, fs) => {
|
||||
const options = parseOptions(argv)
|
||||
io.out(io.stdin)
|
||||
for (const operand of options.operands) {
|
||||
try {
|
||||
await fs.writeText(resolveIn(state.cwd, operand), io.stdin, options.flags.has('a'))
|
||||
} catch (error) {
|
||||
io.err(`${describeFailure('tee', operand, error)}\n`)
|
||||
return 1
|
||||
}
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
/** The text utilities, keyed by the name a command line uses. */
|
||||
export const TEXT_PROGRAMS: Readonly<Record<string, ShellProgram>> = {
|
||||
echo, printf, cat, head, tail, wc, grep, sort, uniq, cut, tr, sed, tee,
|
||||
}
|
||||
@@ -0,0 +1,146 @@
|
||||
/**
|
||||
* Types of the in-worker shell: the state one command line mutates, the byte
|
||||
* face a program reads and writes, and the program signature the command table
|
||||
* stores. A browser worker has no processes, so a "program" is a JavaScript
|
||||
* function over the VFS and the state below is the whole machine.
|
||||
* @module @deepseek-ai/dsh-experimental-webworker-runtime/src/shell/types
|
||||
*/
|
||||
|
||||
/**
|
||||
* Mutable state of one shell instance. A subshell copies it; a group and the
|
||||
* top level share it, which is what makes `cd` visible to later commands of
|
||||
* the same line and invisible outside `( … )`.
|
||||
*/
|
||||
export interface ShellState {
|
||||
/** Absolute working directory every relative path resolves against. */
|
||||
cwd: string
|
||||
/** Exported variables — the environment a program reads. */
|
||||
environment: Record<string, string>
|
||||
/** Shell variables that were assigned but never exported. */
|
||||
variables: Record<string, string>
|
||||
/** Exit status of the last completed command, read back as `$?`. */
|
||||
lastStatus: number
|
||||
/** Set by `exit`; the interpreter stops the line and reports this status. */
|
||||
exitRequested: number | undefined
|
||||
/**
|
||||
* The caller's cancellation, for programs that would otherwise keep the
|
||||
* whole line waiting after a kill. The interpreter checks it between
|
||||
* commands; a program that waits on anything must check it too.
|
||||
*/
|
||||
readonly signal: AbortSignal | undefined
|
||||
}
|
||||
|
||||
/**
|
||||
* The byte face of one running program. Output is collected as text rather
|
||||
* than streamed: every program is a JavaScript function that returns before
|
||||
* the next one starts, so a pipeline is a string handed along, and nothing in
|
||||
* this shell can observe a partially written stream.
|
||||
*/
|
||||
export interface ShellIo {
|
||||
/** Everything on standard input, already complete. */
|
||||
readonly stdin: string
|
||||
/** Append to standard output. Declared as a value: redirections hand it around. */
|
||||
readonly out: (text: string) => void
|
||||
/** Append to standard error. Declared as a value: redirections hand it around. */
|
||||
readonly err: (text: string) => void
|
||||
}
|
||||
|
||||
/** One directory entry, as the listing, glob, and walk programs read it. */
|
||||
export interface ShellDirent {
|
||||
readonly name: string
|
||||
readonly directory: boolean
|
||||
}
|
||||
|
||||
/** What a program can learn about one path. */
|
||||
export interface ShellStats {
|
||||
readonly directory: boolean
|
||||
readonly size: number
|
||||
readonly mtimeMs: number
|
||||
}
|
||||
|
||||
/**
|
||||
* The filesystem a shell run acts on.
|
||||
*
|
||||
* Asynchronous by construction: a command that runs in its own worker reaches
|
||||
* the VFS by message, and the browser offers no way to block on that (a
|
||||
* synchronous face would need `SharedArrayBuffer`, which requires
|
||||
* cross-origin isolation this deployment cannot have). The in-host
|
||||
* implementation answers immediately from the mounted VFS.
|
||||
*/
|
||||
export interface ShellFileSystem {
|
||||
/**
|
||||
* Stat one path.
|
||||
* @param path - absolute VFS path.
|
||||
* @returns the entry's facts, or undefined when nothing is there.
|
||||
*/
|
||||
stat(path: string): Promise<ShellStats | undefined>
|
||||
/**
|
||||
* List one directory.
|
||||
* @param path - absolute VFS path of the directory.
|
||||
* @returns its entries, sorted by name.
|
||||
* @throws a Node-shaped error when the path is absent or is not a directory.
|
||||
*/
|
||||
list(path: string): Promise<ShellDirent[]>
|
||||
/**
|
||||
* Read one file as UTF-8 text.
|
||||
* @param path - absolute VFS path.
|
||||
* @returns the file's contents.
|
||||
* @throws a Node-shaped error when the path is absent or is a directory.
|
||||
*/
|
||||
readText(path: string): Promise<string>
|
||||
/**
|
||||
* Write text to one file.
|
||||
* @param path - absolute VFS path; its parent must exist.
|
||||
* @param text - the text to store.
|
||||
* @param append - true to keep the existing contents and add after them.
|
||||
*/
|
||||
writeText(path: string, text: string, append?: boolean): Promise<void>
|
||||
/**
|
||||
* Create one directory.
|
||||
* @param path - absolute VFS path.
|
||||
* @param recursive - true to create missing parents and tolerate an existing directory.
|
||||
*/
|
||||
mkdir(path: string, recursive: boolean): Promise<void>
|
||||
/**
|
||||
* Remove one path.
|
||||
* @param path - absolute VFS path.
|
||||
* @param options - `recursive` to take a whole subtree, `force` to tolerate absence.
|
||||
*/
|
||||
remove(path: string, options: { recursive: boolean; force: boolean }): Promise<void>
|
||||
/**
|
||||
* Move one file or subtree.
|
||||
* @param from - absolute source path.
|
||||
* @param to - absolute destination path.
|
||||
*/
|
||||
rename(from: string, to: string): Promise<void>
|
||||
}
|
||||
|
||||
/**
|
||||
* One executable of the command table.
|
||||
*
|
||||
* A program reports its exit status like a POSIX process: 0 for success, and a
|
||||
* nonzero status it also explains on {@link ShellIo.err}. Throwing is reserved
|
||||
* for a defect in the program itself — the interpreter turns a throw into
|
||||
* status 1 plus a diagnostic naming the program.
|
||||
* @param argv - the program name at index 0, then its arguments, fully expanded.
|
||||
* @param io - standard input contents and the output sinks.
|
||||
* @param state - shell state; a program that changes it (`cd`, `export`) mutates in place.
|
||||
* @param fs - the filesystem this run acts on.
|
||||
* @returns the exit status.
|
||||
*/
|
||||
export type ShellProgram = (
|
||||
argv: readonly string[],
|
||||
io: ShellIo,
|
||||
state: ShellState,
|
||||
fs: ShellFileSystem,
|
||||
) => number | Promise<number>
|
||||
|
||||
/** Outcome of one interpreted command line. */
|
||||
export interface ShellRunOutcome {
|
||||
/** Exit status of the last command the line ran. */
|
||||
exitCode: number
|
||||
/** Everything written to standard output. */
|
||||
stdout: string
|
||||
/** Everything written to standard error. */
|
||||
stderr: string
|
||||
}
|
||||
@@ -20,16 +20,32 @@ import { installAsyncContextHooks } from './polyfill/async-context/async-context
|
||||
import { createNodeBuiltins, REPLACED_PREFIXES } from './node/builtins.ts'
|
||||
import { whenRequestListener } from './node/builtin_modules/implemented/http.ts'
|
||||
import { installTimerGlobals } from './node/globals/timers.ts'
|
||||
import { installProcessGlobal } from './node/globals/process.ts'
|
||||
import { isShellStartFrame } from './shell/process/protocol.ts'
|
||||
import { runShellProcess } from './shell/process/host.ts'
|
||||
|
||||
// Before the timer globals, so the wrappers close over the patched platform.
|
||||
installAsyncContextHooks()
|
||||
installTimerGlobals()
|
||||
|
||||
let host: { handleMessage(data: unknown): void } | undefined
|
||||
let shellRole = false
|
||||
const pending: unknown[] = []
|
||||
|
||||
self.addEventListener('message', (event: MessageEvent) => {
|
||||
const data = event.data as Record<string, unknown> | null
|
||||
// Role, decided by the first frame: a worker started by the host's shell
|
||||
// runs one command and closes. It mounts no image and boots no tree, so the
|
||||
// whole assembly below never happens in it.
|
||||
if (host === undefined && isShellStartFrame(data)) {
|
||||
shellRole = true
|
||||
// The command's own directory and environment are the only `process` facts
|
||||
// a shell process needs; bundled code that reads the global (picomatch's
|
||||
// platform check) must not find it missing.
|
||||
installProcessGlobal({ cwd: data.cwd, env: data.env })
|
||||
runShellProcess(data, self)
|
||||
return
|
||||
}
|
||||
if (host === undefined && data !== null && typeof data === 'object' && data.t === 'init') {
|
||||
if (typeof data.image !== 'string') {
|
||||
throw new Error('webworker: init frame needs a string image url')
|
||||
@@ -54,6 +70,10 @@ self.addEventListener('message', (event: MessageEvent) => {
|
||||
return
|
||||
}
|
||||
if (host === undefined) {
|
||||
// A shell-role worker's later frames (fs replies, signals) belong to
|
||||
// runShellProcess's own listener; parking them here would hold every
|
||||
// file body until the worker exits.
|
||||
if (shellRole) return
|
||||
pending.push(event.data)
|
||||
return
|
||||
}
|
||||
|
||||
@@ -0,0 +1,152 @@
|
||||
/**
|
||||
* The `node:child_process` face over the in-worker shell, and the ladder above
|
||||
* it: the REAL local subprocess service, running unmodified against this
|
||||
* module instead of a host kernel. That ladder is what the bash tool walks in
|
||||
* the browser, so proving it here is what makes the browser probe a
|
||||
* confirmation rather than the only evidence.
|
||||
*
|
||||
* A Node test host has no DOM `Worker`, so the commands here run through the
|
||||
* inline strategy; the worker strategy and its frames are proven in
|
||||
* `../shell/shell-process.spec.ts`, and both meet again in the preview probe.
|
||||
*
|
||||
* `process.kill` is redirected to the worker's process table for the same
|
||||
* reason the worker does it: the subprocess service polls process-group
|
||||
* liveness through it, and on a test host those pids belong to real processes.
|
||||
*/
|
||||
import { afterEach, beforeEach, expect, it, vi } from 'vitest'
|
||||
import { MemoryVfs } from '@deepseek-ai/dsh-experimental-webworker-runtime/src/storage/memory.ts'
|
||||
import { setActiveVfs } from '@deepseek-ai/dsh-experimental-webworker-runtime/src/storage/active.ts'
|
||||
import { spawn, spawnSync } from '@deepseek-ai/dsh-experimental-webworker-runtime/src/node/builtin_modules/implemented/child_process.ts'
|
||||
import { processAlive, signalProcess } from '@deepseek-ai/dsh-experimental-webworker-runtime/src/node/process-table.ts'
|
||||
import { spawnSubprocess } from '@deepseek-ai/dsh-subprocess-local/src/spawn.ts'
|
||||
|
||||
vi.mock('node:child_process', async () =>
|
||||
await import('@deepseek-ai/dsh-experimental-webworker-runtime/src/node/builtin_modules/implemented/child_process.ts'))
|
||||
|
||||
const WORKSPACE = '/dsh/workspace'
|
||||
|
||||
let vfs: MemoryVfs
|
||||
|
||||
beforeEach(() => {
|
||||
vfs = new MemoryVfs()
|
||||
setActiveVfs(vfs)
|
||||
vfs.mkdirSync(WORKSPACE, { recursive: true })
|
||||
vi.spyOn(process, 'kill').mockImplementation((pid: number, signal?: string | number): true => {
|
||||
if (signal === 0) {
|
||||
if (processAlive(pid)) return true
|
||||
const error = new Error('kill ESRCH') as NodeJS.ErrnoException
|
||||
error.code = 'ESRCH'
|
||||
throw error
|
||||
}
|
||||
signalProcess(pid, (signal ?? 'SIGTERM') as NodeJS.Signals)
|
||||
return true
|
||||
})
|
||||
})
|
||||
|
||||
afterEach(() => {
|
||||
vi.restoreAllMocks()
|
||||
})
|
||||
|
||||
/** Collect one child's stdout, stderr, and settlement. */
|
||||
async function collect(child: ReturnType<typeof spawn>): Promise<{ stdout: string; stderr: string; code: number | null }> {
|
||||
let stdout = ''
|
||||
let stderr = ''
|
||||
child.stdout?.on('data', (chunk: unknown) => { stdout += String(chunk) })
|
||||
child.stderr?.on('data', (chunk: unknown) => { stderr += String(chunk) })
|
||||
const code = await new Promise<number | null>((settle, fail) => {
|
||||
child.on('close', (value: unknown) => { settle(value as number | null) })
|
||||
child.on('error', fail)
|
||||
})
|
||||
return { stdout, stderr, code }
|
||||
}
|
||||
|
||||
it('runs a bash command line and reports its output through the pipes', async () => {
|
||||
const child = spawn('bash', ['-c', 'echo hi; echo oops >&2'], { cwd: WORKSPACE })
|
||||
expect(child.pid).toBeGreaterThan(1)
|
||||
expect(await collect(child)).toEqual({ stdout: 'hi\n', stderr: 'oops\n', code: 0 })
|
||||
})
|
||||
|
||||
it('runs an explicit argv without re-parsing it as a command line', async () => {
|
||||
vfs.writeFileSync(`${WORKSPACE}/spaced name.txt`, 'kept\n')
|
||||
const child = spawn('cat', ['spaced name.txt'], { cwd: WORKSPACE })
|
||||
expect((await collect(child)).stdout).toBe('kept\n')
|
||||
})
|
||||
|
||||
it('fails a program the command table does not hold the way a missing binary does', async () => {
|
||||
const child = spawn('nowhere-binary', [], { cwd: WORKSPACE })
|
||||
// A caller that configures the pipes first (the browser launcher does) must
|
||||
// reach the ENOENT, not a TypeError on the configuration line.
|
||||
child.stdout?.setEncoding()
|
||||
child.stderr?.setEncoding()
|
||||
const error = await new Promise<NodeJS.ErrnoException>((settle) => {
|
||||
child.on('error', (value: unknown) => { settle(value as NodeJS.ErrnoException) })
|
||||
})
|
||||
expect(error.code).toBe('ENOENT')
|
||||
expect(error.syscall).toBe('spawn nowhere-binary')
|
||||
})
|
||||
|
||||
it('refuses a command name that is not a string, as Node does', () => {
|
||||
expect(() => spawn(undefined as unknown as string)).toThrow(/must be a non-empty string/)
|
||||
})
|
||||
|
||||
it('reports that a synchronous run cannot happen, without throwing at the probe', () => {
|
||||
expect(spawnSync('bwrap').error?.code).toBe('ENOENT')
|
||||
expect(spawnSync('echo').error?.message).toContain('commands run asynchronously')
|
||||
})
|
||||
|
||||
it('carries a command through the real local subprocess service', async () => {
|
||||
const handle = spawnSubprocess({
|
||||
argv: ['bash', '-c', 'echo written > note.txt && cat note.txt'],
|
||||
cwd: WORKSPACE,
|
||||
stdio: {
|
||||
stdin: 'ignore',
|
||||
stdout: { maxBytes: 64_000 },
|
||||
stderr: { maxBytes: 64_000 },
|
||||
},
|
||||
graceMs: 3_000,
|
||||
env: {},
|
||||
})
|
||||
const outcome = await handle.done
|
||||
expect(outcome).toEqual({ exitCode: 0, signal: null })
|
||||
expect(handle.collected.stdout?.readFrom(0).text).toBe('written\n')
|
||||
expect(vfs.readFileSync(`${WORKSPACE}/note.txt`, 'utf8')).toBe('written\n')
|
||||
})
|
||||
|
||||
it('writes the caller-supplied standard input into the command', async () => {
|
||||
const handle = spawnSubprocess({
|
||||
argv: ['bash', '-c', 'grep -c ""'],
|
||||
cwd: WORKSPACE,
|
||||
stdio: {
|
||||
stdin: { data: 'one\ntwo\nthree\n' },
|
||||
stdout: { maxBytes: 64_000 },
|
||||
stderr: { maxBytes: 64_000 },
|
||||
},
|
||||
graceMs: 3_000,
|
||||
env: {},
|
||||
})
|
||||
await handle.done
|
||||
expect(handle.collected.stdout?.readFrom(0).text).toBe('3\n')
|
||||
})
|
||||
|
||||
it('kills a running command through the service and reports the signal', async () => {
|
||||
const handle = spawnSubprocess({
|
||||
argv: ['bash', '-c', 'sleep 30; echo never'],
|
||||
cwd: WORKSPACE,
|
||||
stdio: {
|
||||
stdin: 'ignore',
|
||||
stdout: { maxBytes: 64_000 },
|
||||
stderr: { maxBytes: 64_000 },
|
||||
},
|
||||
graceMs: 3_000,
|
||||
env: {},
|
||||
})
|
||||
const started = performance.now()
|
||||
handle.terminate()
|
||||
const outcome = await handle.done
|
||||
expect(outcome.signal).toBe('SIGTERM')
|
||||
expect(outcome.exitCode).toBeNull()
|
||||
expect(handle.collected.stdout?.readFrom(0).text).toBe('')
|
||||
// The command settles on the signal, not on the interval it was waiting out:
|
||||
// a `sleep` that ignored the abort would hold this handle open for 30s.
|
||||
expect(performance.now() - started).toBeLessThan(5_000)
|
||||
})
|
||||
@@ -0,0 +1,165 @@
|
||||
/**
|
||||
* The two run options a command inside a process worker supplies: the
|
||||
* filesystem it acts on, and the callback that reports output before the run
|
||||
* settles. `src/shell/process/child.ts` passes a message-backed filesystem and
|
||||
* posts a frame per write, so both are load-bearing for every backgrounded
|
||||
* command the bash tool starts.
|
||||
*
|
||||
* No VFS is mounted here, deliberately. The in-host filesystem reads the
|
||||
* process-wide slot on first use, so a program that reached it instead of the
|
||||
* injected face fails with `no filesystem is mounted` — a suite that mounted a
|
||||
* VFS as well would pass either way.
|
||||
*/
|
||||
import { describe, expect, it } from 'vitest'
|
||||
import { runShellCommand } from '@deepseek-ai/dsh-experimental-webworker-runtime/src/shell/interpret.ts'
|
||||
import { filesystemError } from '@deepseek-ai/dsh-experimental-webworker-runtime/src/shell/fs-access.ts'
|
||||
import type {
|
||||
ShellDirent, ShellFileSystem, ShellRunOutcome, ShellStats,
|
||||
} from '@deepseek-ai/dsh-experimental-webworker-runtime/src/shell/types.ts'
|
||||
|
||||
const WORKSPACE = '/dsh/workspace'
|
||||
|
||||
/** One call a program made on the injected filesystem. */
|
||||
interface Call {
|
||||
readonly op: string
|
||||
readonly path: string
|
||||
readonly text?: string
|
||||
readonly append?: boolean
|
||||
}
|
||||
|
||||
/**
|
||||
* A filesystem over a flat map of absolute paths, recording every call.
|
||||
*
|
||||
* Directories are the parents of the files it holds, which is all the programs
|
||||
* below ask about; nothing here reaches the mounted VFS.
|
||||
*/
|
||||
function recordingFileSystem(files: Record<string, string>): {
|
||||
fs: ShellFileSystem
|
||||
calls: Call[]
|
||||
contents: Map<string, string>
|
||||
} {
|
||||
const contents = new Map(Object.entries(files))
|
||||
const calls: Call[] = []
|
||||
const directories = (): Set<string> => {
|
||||
const known = new Set<string>()
|
||||
for (const path of contents.keys()) {
|
||||
for (let parent = path.slice(0, path.lastIndexOf('/')); parent !== ''; parent = parent.slice(0, parent.lastIndexOf('/'))) {
|
||||
known.add(parent)
|
||||
}
|
||||
}
|
||||
return known
|
||||
}
|
||||
const fs: ShellFileSystem = {
|
||||
stat: async (path: string): Promise<ShellStats | undefined> => {
|
||||
calls.push({ op: 'stat', path })
|
||||
const text = contents.get(path)
|
||||
if (text !== undefined) return { directory: false, size: text.length, mtimeMs: 1 }
|
||||
return directories().has(path) ? { directory: true, size: 0, mtimeMs: 1 } : undefined
|
||||
},
|
||||
list: async (path: string): Promise<ShellDirent[]> => {
|
||||
calls.push({ op: 'list', path })
|
||||
if (!directories().has(path)) throw filesystemError('ENOENT', 'scandir', path)
|
||||
const prefix = `${path}/`
|
||||
const names = new Set<string>()
|
||||
for (const candidate of [...contents.keys(), ...directories()]) {
|
||||
if (!candidate.startsWith(prefix)) continue
|
||||
names.add(candidate.slice(prefix.length).split('/')[0] as string)
|
||||
}
|
||||
return [...names].sort().map(name => ({ name, directory: directories().has(`${prefix}${name}`) }))
|
||||
},
|
||||
readText: async (path: string): Promise<string> => {
|
||||
calls.push({ op: 'readText', path })
|
||||
const text = contents.get(path)
|
||||
if (text === undefined) throw filesystemError('ENOENT', 'open', path)
|
||||
return text
|
||||
},
|
||||
writeText: async (path: string, text: string, append = false): Promise<void> => {
|
||||
calls.push({ op: 'writeText', path, text, append })
|
||||
contents.set(path, append ? `${contents.get(path) ?? ''}${text}` : text)
|
||||
},
|
||||
mkdir: async (path: string, recursive: boolean): Promise<void> => {
|
||||
calls.push({ op: 'mkdir', path, append: recursive })
|
||||
},
|
||||
remove: async (path: string): Promise<void> => {
|
||||
calls.push({ op: 'remove', path })
|
||||
contents.delete(path)
|
||||
},
|
||||
rename: async (from: string, to: string): Promise<void> => {
|
||||
calls.push({ op: 'rename', path: from, text: to })
|
||||
const text = contents.get(from)
|
||||
if (text === undefined) throw filesystemError('ENOENT', 'rename', from)
|
||||
contents.delete(from)
|
||||
contents.set(to, text)
|
||||
},
|
||||
}
|
||||
return { fs, calls, contents }
|
||||
}
|
||||
|
||||
describe('injected filesystem', () => {
|
||||
it('reads through the injected face, at the path the shell resolved', async () => {
|
||||
const { fs, calls } = recordingFileSystem({ [`${WORKSPACE}/notes.txt`]: 'alpha\nbeta\n' })
|
||||
const result = await runShellCommand('cat notes.txt', { cwd: WORKSPACE, env: {}, fs })
|
||||
expect(result).toEqual({ exitCode: 0, stdout: 'alpha\nbeta\n', stderr: '' })
|
||||
// Programs receive the word as written; the absolute path is the shell's work.
|
||||
expect(calls.filter(call => call.op === 'readText').map(call => call.path)).toEqual([`${WORKSPACE}/notes.txt`])
|
||||
})
|
||||
|
||||
it('performs a redirection as a truncating write followed by appends', async () => {
|
||||
const { fs, calls, contents } = recordingFileSystem({})
|
||||
const result = await runShellCommand('echo one > out.txt; echo two >> out.txt', { cwd: WORKSPACE, env: {}, fs })
|
||||
expect(result.exitCode).toBe(0)
|
||||
expect(contents.get(`${WORKSPACE}/out.txt`)).toBe('one\ntwo\n')
|
||||
expect(calls.filter(call => call.op === 'writeText').map(call => [call.text, call.append])).toEqual([
|
||||
// `> file` empties the file when the redirection is set up, so a command
|
||||
// that writes nothing still leaves it empty.
|
||||
['', false],
|
||||
['one\n', true],
|
||||
['two\n', true],
|
||||
])
|
||||
})
|
||||
|
||||
it('reports an injected failure as the utility does, not as a filesystem error', async () => {
|
||||
const { fs } = recordingFileSystem({})
|
||||
const result = await runShellCommand('cat missing.txt', { cwd: WORKSPACE, env: {}, fs })
|
||||
expect(result.exitCode).toBe(1)
|
||||
expect(result.stderr).toBe('cat: missing.txt: No such file or directory\n')
|
||||
})
|
||||
})
|
||||
|
||||
describe('incremental output', () => {
|
||||
/** Run a line, collecting what the callback saw in order. */
|
||||
async function reported(command: string): Promise<{ seen: [string, string][]; outcome: ShellRunOutcome }> {
|
||||
const { fs } = recordingFileSystem({ [`${WORKSPACE}/notes.txt`]: 'alpha\n' })
|
||||
const seen: [string, string][] = []
|
||||
const outcome = await runShellCommand(command, {
|
||||
cwd: WORKSPACE,
|
||||
env: {},
|
||||
fs,
|
||||
onOutput: (stream, text) => { seen.push([stream, text]) },
|
||||
})
|
||||
return { seen, outcome }
|
||||
}
|
||||
|
||||
it('reports each write as it happens and still returns the complete text', async () => {
|
||||
const { seen, outcome } = await reported('echo one; echo two')
|
||||
expect(seen).toEqual([['stdout', 'one\n'], ['stdout', 'two\n']])
|
||||
expect(outcome.stdout).toBe('one\ntwo\n')
|
||||
})
|
||||
|
||||
it('tags a diagnostic as standard error', async () => {
|
||||
const { seen, outcome } = await reported('definitely-not-a-program')
|
||||
expect(seen).toEqual([['stderr', 'bash: definitely-not-a-program: command not found\n']])
|
||||
expect(outcome).toEqual({ exitCode: 127, stdout: '', stderr: 'bash: definitely-not-a-program: command not found\n' })
|
||||
})
|
||||
|
||||
it('reports only what the line writes out, not what it hands along or captures', async () => {
|
||||
// A pipeline stage writes into the next stage's input and a redirection
|
||||
// writes into a file: neither is output of the line, so a caller polling for
|
||||
// progress must not see it.
|
||||
const piped = await reported('cat notes.txt | cat')
|
||||
expect(piped.seen).toEqual([['stdout', 'alpha\n']])
|
||||
const redirected = await reported('echo captured > out.txt')
|
||||
expect(redirected.seen).toEqual([])
|
||||
expect(redirected.outcome.stdout).toBe('')
|
||||
})
|
||||
})
|
||||
@@ -0,0 +1,199 @@
|
||||
/**
|
||||
* The process model: a command runs in its own worker, reaches the VFS only by
|
||||
* message, and dies when the host says so.
|
||||
*
|
||||
* The `Worker` here is a loopback that runs the REAL child half
|
||||
* (`runShellProcess`) against the REAL host half, so the frames, the
|
||||
* filesystem service, and the termination ladder are the shipped ones — only
|
||||
* the thread boundary is simulated, because a Node test host has no DOM
|
||||
* `Worker` to cross. That a browser worker really can start a nested worker
|
||||
* and terminate it mid-burn is measured separately, in the preview probe.
|
||||
*/
|
||||
import { afterEach, beforeEach, expect, it, vi } from 'vitest'
|
||||
import { MemoryVfs } from '@deepseek-ai/dsh-experimental-webworker-runtime/src/storage/memory.ts'
|
||||
import { setActiveVfs } from '@deepseek-ai/dsh-experimental-webworker-runtime/src/storage/active.ts'
|
||||
import { startProcess } from '@deepseek-ai/dsh-experimental-webworker-runtime/src/shell/process/host.ts'
|
||||
import { runShellProcess } from '@deepseek-ai/dsh-experimental-webworker-runtime/src/shell/process/child.ts'
|
||||
import { isShellStartFrame } from '@deepseek-ai/dsh-experimental-webworker-runtime/src/shell/process/protocol.ts'
|
||||
import type { FromProcessFrame, ToProcessFrame } from '@deepseek-ai/dsh-experimental-webworker-runtime/src/shell/process/protocol.ts'
|
||||
|
||||
const WORKSPACE = '/dsh/workspace'
|
||||
const WORKER_URL = 'https://example.test/assets/worker.js'
|
||||
|
||||
let vfs: MemoryVfs
|
||||
/** Every loopback worker the code under test constructed. */
|
||||
let started: LoopbackWorker[]
|
||||
|
||||
/**
|
||||
* A `Worker` that keeps the child half on this thread. Delivery is deferred so
|
||||
* neither half can observe the other's synchronous progress, which is the one
|
||||
* property of the real boundary that changes behaviour.
|
||||
*/
|
||||
class LoopbackWorker {
|
||||
readonly url: string
|
||||
terminated = false
|
||||
private readonly hostListeners: ((event: MessageEvent) => void)[] = []
|
||||
private childListener: ((event: MessageEvent) => void) | undefined
|
||||
private closed = false
|
||||
|
||||
constructor(url: string | URL, options?: { type?: string }) {
|
||||
this.url = String(url)
|
||||
expect(options?.type).toBe('module')
|
||||
started.push(this)
|
||||
}
|
||||
|
||||
/** Host → child. The first frame starts the real child half. */
|
||||
postMessage(frame: ToProcessFrame): void {
|
||||
if (this.terminated) return
|
||||
queueMicrotask(() => {
|
||||
if (this.terminated) return
|
||||
if (isShellStartFrame(frame)) {
|
||||
runShellProcess(frame, {
|
||||
postMessage: (reply: FromProcessFrame) => { this.toHost(reply) },
|
||||
addEventListener: (_type: 'message', listener: (event: MessageEvent) => void) => { this.childListener = listener },
|
||||
close: () => { this.closed = true },
|
||||
})
|
||||
return
|
||||
}
|
||||
this.childListener?.({ data: frame } as MessageEvent)
|
||||
})
|
||||
}
|
||||
|
||||
/** Child → host. */
|
||||
private toHost(frame: FromProcessFrame): void {
|
||||
if (this.terminated) return
|
||||
queueMicrotask(() => {
|
||||
if (this.terminated) return
|
||||
for (const listener of this.hostListeners) listener({ data: frame } as MessageEvent)
|
||||
})
|
||||
}
|
||||
|
||||
addEventListener(type: 'message' | 'error', listener: (event: MessageEvent) => void): void {
|
||||
if (type === 'message') this.hostListeners.push(listener)
|
||||
}
|
||||
|
||||
terminate(): void {
|
||||
this.terminated = true
|
||||
}
|
||||
|
||||
/** Whether the child closed itself after reporting its status. */
|
||||
get childClosed(): boolean {
|
||||
return this.closed
|
||||
}
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
vfs = new MemoryVfs()
|
||||
setActiveVfs(vfs)
|
||||
vfs.mkdirSync(WORKSPACE, { recursive: true })
|
||||
started = []
|
||||
// The selection in `startProcess` reads exactly these two globals.
|
||||
vi.stubGlobal('Worker', LoopbackWorker)
|
||||
vi.stubGlobal('self', { location: { href: WORKER_URL } })
|
||||
})
|
||||
|
||||
afterEach(() => {
|
||||
vi.unstubAllGlobals()
|
||||
})
|
||||
|
||||
/** Run one command line through the process model and collect everything. */
|
||||
async function run(script: string, stdin = ''): Promise<{ code: number; stdout: string; stderr: string }> {
|
||||
let stdout = ''
|
||||
let stderr = ''
|
||||
const code = await new Promise<number>((settle) => {
|
||||
startProcess({
|
||||
script,
|
||||
argv: ['bash', '-c', script],
|
||||
cwd: WORKSPACE,
|
||||
env: { HOME: '/dsh/home' },
|
||||
stdin,
|
||||
onOutput: (stream, text) => {
|
||||
if (stream === 'stdout') stdout += text
|
||||
else stderr += text
|
||||
},
|
||||
onExit: settle,
|
||||
})
|
||||
})
|
||||
return { code, stdout, stderr }
|
||||
}
|
||||
|
||||
it('starts the command as a worker from this bundle, not on this thread', async () => {
|
||||
await run('echo hi')
|
||||
expect(started).toHaveLength(1)
|
||||
// The child is this very bundle in another role: no second asset to serve.
|
||||
expect(started[0]?.url).toBe(WORKER_URL)
|
||||
})
|
||||
|
||||
it('runs the command in the child and reports its output and status', async () => {
|
||||
expect(await run('echo hi; echo oops >&2; exit 3')).toEqual({ code: 3, stdout: 'hi\n', stderr: 'oops\n' })
|
||||
})
|
||||
|
||||
it('reaches the host filesystem by message', async () => {
|
||||
const written = await run('mkdir -p nested && echo carried > nested/file.txt && cat nested/file.txt')
|
||||
expect(written).toEqual({ code: 0, stdout: 'carried\n', stderr: '' })
|
||||
// The child holds no VFS of its own: the bytes can only have arrived here
|
||||
// through the filesystem frames.
|
||||
expect(vfs.readFileSync(`${WORKSPACE}/nested/file.txt`, 'utf8')).toBe('carried\n')
|
||||
})
|
||||
|
||||
it('carries a filesystem failure back with its code, not as a lost exception', async () => {
|
||||
const missing = await run('cat nowhere.txt')
|
||||
expect(missing.code).toBe(1)
|
||||
expect(missing.stderr).toBe('cat: nowhere.txt: No such file or directory\n')
|
||||
})
|
||||
|
||||
it('delivers standard input to the child', async () => {
|
||||
expect((await run('grep -c ""', 'a\nb\nc\n')).stdout).toBe('3\n')
|
||||
})
|
||||
|
||||
it('closes the child once the command settles', async () => {
|
||||
await run('true')
|
||||
expect(started[0]?.childClosed).toBe(true)
|
||||
})
|
||||
|
||||
it('asks first and terminates second', async () => {
|
||||
const events: number[] = []
|
||||
const running = startProcess({
|
||||
script: 'sleep 30',
|
||||
argv: ['bash', '-c', 'sleep 30'],
|
||||
cwd: WORKSPACE,
|
||||
env: {},
|
||||
stdin: '',
|
||||
onOutput: () => {},
|
||||
onExit: code => events.push(code),
|
||||
})
|
||||
// The first rung asks the command to stop; a `sleep` honours it.
|
||||
running.interrupt()
|
||||
await vi.waitFor(() => { expect(events).toHaveLength(1) })
|
||||
expect(events[0]).toBe(130)
|
||||
|
||||
// The second rung does not ask: the worker is gone whatever it was doing.
|
||||
const stubborn = startProcess({
|
||||
script: 'sleep 30',
|
||||
argv: ['bash', '-c', 'sleep 30'],
|
||||
cwd: WORKSPACE,
|
||||
env: {},
|
||||
stdin: '',
|
||||
onOutput: () => {},
|
||||
onExit: code => events.push(code),
|
||||
})
|
||||
stubborn.destroy()
|
||||
await vi.waitFor(() => { expect(events).toHaveLength(2) })
|
||||
expect(started[1]?.terminated).toBe(true)
|
||||
})
|
||||
|
||||
it('runs an explicit argv without a command line to parse', async () => {
|
||||
vfs.writeFileSync(`${WORKSPACE}/spaced name.txt`, 'kept\n')
|
||||
let stdout = ''
|
||||
const code = await new Promise<number>((settle) => {
|
||||
startProcess({
|
||||
argv: ['cat', 'spaced name.txt'],
|
||||
cwd: WORKSPACE,
|
||||
env: {},
|
||||
stdin: '',
|
||||
onOutput: (_stream, text) => { stdout += text },
|
||||
onExit: settle,
|
||||
})
|
||||
})
|
||||
expect({ code, stdout }).toEqual({ code: 0, stdout: 'kept\n' })
|
||||
})
|
||||
@@ -0,0 +1,225 @@
|
||||
/**
|
||||
* The in-worker shell: structure (pipelines, chaining, subshells, redirections,
|
||||
* expansion) and the command table's effects on a real MemoryVfs.
|
||||
*
|
||||
* ONE module instance, like `../node/fs.spec.ts`: the command table reaches the VFS
|
||||
* through the module-level slot, so the mount here and the programs under test
|
||||
* must be the same copy of `src/storage/memory.ts`.
|
||||
*/
|
||||
import { beforeEach, describe, expect, it } from 'vitest'
|
||||
import { MemoryVfs } from '@deepseek-ai/dsh-experimental-webworker-runtime/src/storage/memory.ts'
|
||||
import { setActiveVfs } from '@deepseek-ai/dsh-experimental-webworker-runtime/src/storage/active.ts'
|
||||
import { runShellCommand } from '@deepseek-ai/dsh-experimental-webworker-runtime/src/shell/interpret.ts'
|
||||
import type { ShellRunOutcome } from '@deepseek-ai/dsh-experimental-webworker-runtime/src/shell/types.ts'
|
||||
|
||||
const WORKSPACE = '/dsh/workspace'
|
||||
|
||||
let vfs: MemoryVfs
|
||||
|
||||
/** Run one command line in a fresh workspace with a fixed environment. */
|
||||
async function run(command: string, options: { stdin?: string; cwd?: string } = {}): Promise<ShellRunOutcome> {
|
||||
return await runShellCommand(command, {
|
||||
cwd: options.cwd ?? WORKSPACE,
|
||||
env: { HOME: '/dsh/home', PWD: WORKSPACE, GREETING: 'hello world' },
|
||||
stdin: options.stdin,
|
||||
})
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
vfs = new MemoryVfs()
|
||||
setActiveVfs(vfs)
|
||||
vfs.mkdirSync(WORKSPACE, { recursive: true })
|
||||
vfs.mkdirSync(`${WORKSPACE}/src`, { recursive: true })
|
||||
vfs.writeFileSync(`${WORKSPACE}/notes.txt`, 'alpha\nbeta\ngamma\n')
|
||||
vfs.writeFileSync(`${WORKSPACE}/src/a.ts`, 'export const a = 1\n')
|
||||
vfs.writeFileSync(`${WORKSPACE}/src/b.ts`, 'export const b = 2\n')
|
||||
})
|
||||
|
||||
describe('command execution', () => {
|
||||
it('runs a program and reports its output and status', async () => {
|
||||
expect(await run('echo hi')).toEqual({ exitCode: 0, stdout: 'hi\n', stderr: '' })
|
||||
})
|
||||
|
||||
it('reports an unknown command the way a shell does', async () => {
|
||||
const result = await run('definitely-not-a-program --help')
|
||||
expect(result.exitCode).toBe(127)
|
||||
expect(result.stderr).toBe('bash: definitely-not-a-program: command not found\n')
|
||||
})
|
||||
|
||||
it('reads a file through the VFS', async () => {
|
||||
expect((await run('cat notes.txt')).stdout).toBe('alpha\nbeta\ngamma\n')
|
||||
})
|
||||
|
||||
it('reports a missing file as the utility does, with a nonzero status', async () => {
|
||||
const result = await run('cat missing.txt')
|
||||
expect(result.exitCode).toBe(1)
|
||||
expect(result.stderr).toBe('cat: missing.txt: No such file or directory\n')
|
||||
})
|
||||
})
|
||||
|
||||
describe('structure', () => {
|
||||
it('pipes standard output into the next stage', async () => {
|
||||
expect((await run('cat notes.txt | grep -n "^[ab]"')).stdout).toBe('1:alpha\n2:beta\n')
|
||||
})
|
||||
|
||||
it('takes the pipeline status from its last stage', async () => {
|
||||
expect((await run('cat notes.txt | grep zeta')).exitCode).toBe(1)
|
||||
})
|
||||
|
||||
it('honours && and || on the previous status', async () => {
|
||||
expect((await run('true && echo yes || echo no')).stdout).toBe('yes\n')
|
||||
expect((await run('false && echo yes || echo no')).stdout).toBe('no\n')
|
||||
})
|
||||
|
||||
it('runs ; separated commands in order', async () => {
|
||||
expect((await run('echo one; echo two')).stdout).toBe('one\ntwo\n')
|
||||
})
|
||||
|
||||
it('keeps a subshell directory change out of the parent', async () => {
|
||||
expect((await run('(cd src && pwd); pwd')).stdout).toBe(`${WORKSPACE}/src\n${WORKSPACE}\n`)
|
||||
})
|
||||
|
||||
it('keeps a directory change made by the line itself', async () => {
|
||||
expect((await run('cd src; pwd')).stdout).toBe(`${WORKSPACE}/src\n`)
|
||||
})
|
||||
|
||||
it('stops the line at exit and reports its status', async () => {
|
||||
const result = await run('echo before; exit 3; echo after')
|
||||
expect(result).toEqual({ exitCode: 3, stdout: 'before\n', stderr: '' })
|
||||
})
|
||||
})
|
||||
|
||||
describe('redirections', () => {
|
||||
it('writes standard output to a file and truncates it first', async () => {
|
||||
await run('echo first > out.txt')
|
||||
await run('echo second > out.txt')
|
||||
expect(vfs.readFileSync(`${WORKSPACE}/out.txt`, 'utf8')).toBe('second\n')
|
||||
})
|
||||
|
||||
it('creates an empty file when the command writes nothing', async () => {
|
||||
await run('true > empty.txt')
|
||||
expect(vfs.readFileSync(`${WORKSPACE}/empty.txt`, 'utf8')).toBe('')
|
||||
})
|
||||
|
||||
it('appends with >>', async () => {
|
||||
await run('echo one > log.txt; echo two >> log.txt')
|
||||
expect(vfs.readFileSync(`${WORKSPACE}/log.txt`, 'utf8')).toBe('one\ntwo\n')
|
||||
})
|
||||
|
||||
it('reads standard input from a file and from a here-string', async () => {
|
||||
expect((await run('grep beta < notes.txt')).stdout).toBe('beta\n')
|
||||
expect((await run('cat <<< inline')).stdout).toBe('inline\n')
|
||||
})
|
||||
|
||||
it('sends standard error to its own file with 2>', async () => {
|
||||
const result = await run('cat missing.txt 2> err.txt')
|
||||
expect(result.stderr).toBe('')
|
||||
expect(vfs.readFileSync(`${WORKSPACE}/err.txt`, 'utf8')).toBe('cat: missing.txt: No such file or directory\n')
|
||||
})
|
||||
|
||||
it('merges standard error into standard output with 2>&1', async () => {
|
||||
const result = await run('cat missing.txt 2>&1')
|
||||
expect(result.stderr).toBe('')
|
||||
expect(result.stdout).toBe('cat: missing.txt: No such file or directory\n')
|
||||
})
|
||||
|
||||
it('reports a missing input file itself and never runs the command', async () => {
|
||||
// Setting up the redirection is the shell's own work, so the diagnostic is
|
||||
// prefixed `bash` on the resolved path rather than by the utility.
|
||||
expect(await run('cat < missing.txt')).toEqual({
|
||||
exitCode: 1,
|
||||
stdout: '',
|
||||
stderr: `bash: ${WORKSPACE}/missing.txt: No such file or directory\n`,
|
||||
})
|
||||
})
|
||||
|
||||
it('refuses a target that expands to more than one word', async () => {
|
||||
const result = await run('cat < src/*.ts')
|
||||
expect(result.exitCode).toBe(1)
|
||||
expect(result.stderr).toBe('bash: ambiguous redirect\n')
|
||||
})
|
||||
|
||||
it('refuses a descriptor duplication other than between stdout and stderr', async () => {
|
||||
const result = await run('echo hi 3>&1')
|
||||
expect(result.exitCode).toBe(1)
|
||||
expect(result.stderr).toBe('bash: 3>&1: unsupported descriptor redirection\n')
|
||||
})
|
||||
})
|
||||
|
||||
describe('expansion', () => {
|
||||
it('expands variables, quoted and unquoted', async () => {
|
||||
expect((await run('echo "$GREETING"')).stdout).toBe('hello world\n')
|
||||
expect((await run('echo ${MISSING:-fallback}')).stdout).toBe('fallback\n')
|
||||
})
|
||||
|
||||
it('reports the previous status as $?', async () => {
|
||||
expect((await run('false; echo $?')).stdout).toBe('1\n')
|
||||
})
|
||||
|
||||
it('substitutes command output', async () => {
|
||||
expect((await run('echo "[$(head -n 1 notes.txt)]"')).stdout).toBe('[alpha]\n')
|
||||
})
|
||||
|
||||
it('evaluates arithmetic', async () => {
|
||||
expect((await run('echo $((1 + 2 * 3))')).stdout).toBe('7\n')
|
||||
})
|
||||
|
||||
it('expands globs against the VFS and keeps an unmatched pattern literal', async () => {
|
||||
expect((await run('echo src/*.ts')).stdout).toBe('src/a.ts src/b.ts\n')
|
||||
expect((await run('echo *.missing')).stdout).toBe('*.missing\n')
|
||||
})
|
||||
|
||||
it('passes an assignment prefix as environment for that command only', async () => {
|
||||
expect((await run('MARK=set printenv MARK; echo "[${MARK}]"')).stdout).toBe('set\n[]\n')
|
||||
})
|
||||
})
|
||||
|
||||
describe('file utilities', () => {
|
||||
it('lists a directory one entry per line', async () => {
|
||||
expect((await run('ls src')).stdout).toBe('a.ts\nb.ts\n')
|
||||
})
|
||||
|
||||
it('creates, copies, moves, and removes trees', async () => {
|
||||
const result = await run('mkdir -p deep/nested && cp -r src deep/nested/copy && mv notes.txt deep/ && rm -r src')
|
||||
expect(result.exitCode).toBe(0)
|
||||
expect(vfs.existsSync(`${WORKSPACE}/deep/nested/copy/a.ts`)).toBe(true)
|
||||
expect(vfs.existsSync(`${WORKSPACE}/deep/notes.txt`)).toBe(true)
|
||||
expect(vfs.existsSync(`${WORKSPACE}/src`)).toBe(false)
|
||||
})
|
||||
|
||||
it('finds by name and type', async () => {
|
||||
expect((await run('find . -name "*.ts"')).stdout).toBe('./src/a.ts\n./src/b.ts\n')
|
||||
expect((await run('find . -type d')).stdout).toBe('.\n./src\n')
|
||||
})
|
||||
|
||||
it('counts, sorts, and deduplicates text', async () => {
|
||||
expect((await run('wc -l notes.txt')).stdout.trim()).toBe('3 notes.txt')
|
||||
expect((await run('sort -r notes.txt | head -n 1')).stdout).toBe('gamma\n')
|
||||
expect((await run('printf "b\\nb\\na\\n" | sort | uniq')).stdout).toBe('a\nb\n')
|
||||
})
|
||||
|
||||
it('translates and deletes characters by range', async () => {
|
||||
expect((await run('echo shell-works | tr a-z A-Z')).stdout).toBe('SHELL-WORKS\n')
|
||||
expect((await run('echo a1b2c3 | tr -d 0-9')).stdout).toBe('abc\n')
|
||||
})
|
||||
|
||||
it('substitutes with sed and refuses any other script', async () => {
|
||||
expect((await run('sed s/alpha/ALPHA/ notes.txt | head -n 1')).stdout).toBe('ALPHA\n')
|
||||
const refused = await run('sed 1d notes.txt')
|
||||
expect(refused.exitCode).toBe(2)
|
||||
expect(refused.stderr).toContain('only substitution scripts')
|
||||
})
|
||||
})
|
||||
|
||||
describe('cancellation', () => {
|
||||
it('stops before the next command once the caller aborts', async () => {
|
||||
const controller = new AbortController()
|
||||
controller.abort()
|
||||
const result = await runShellCommand('echo never', {
|
||||
cwd: WORKSPACE,
|
||||
env: {},
|
||||
signal: controller.signal,
|
||||
})
|
||||
expect(result).toEqual({ exitCode: 130, stdout: '', stderr: '' })
|
||||
})
|
||||
})
|
||||
@@ -1,9 +1,36 @@
|
||||
import { createRequire } from 'node:module'
|
||||
import { dirname, join } from 'node:path'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
import { defineConfig } from 'tsdown'
|
||||
import { MODULE_PROXIES, MODULE_PROXY_PREFIXES } from './src/module-proxies.ts'
|
||||
|
||||
const here = (relative: string): string => fileURLToPath(new URL(relative, import.meta.url))
|
||||
|
||||
const resolveFrom = createRequire(import.meta.url)
|
||||
|
||||
/**
|
||||
* `@yarnpkg/parsers`' shell face, reached at its own file rather than through the
|
||||
* package root. The root barrel also re-exports the Syml parser, which pulls in
|
||||
* that grammar and js-yaml — around 175 kB of this bundle for a format the worker
|
||||
* never parses, plus their module bodies at worker start. The shell entry holds
|
||||
* `parseShell` and the stringifiers, requiring only its own grammar.
|
||||
*
|
||||
* This pins a path inside the package, which its `exports` field does not
|
||||
* publish: upgrading `@yarnpkg/parsers` must re-check that `lib/shell.js` is still
|
||||
* where the shell parser lives and still exports `parseShell`. The path is
|
||||
* derived from the package manifest, the one subpath the package does publish, so
|
||||
* a move fails the build here rather than silently reinstating the barrel.
|
||||
*/
|
||||
const SHELL_PARSER_ENTRY = join(dirname(resolveFrom.resolve('@yarnpkg/parsers/package.json')), 'lib/shell.js')
|
||||
|
||||
/** Redirects the parsers root specifier onto {@link SHELL_PARSER_ENTRY}. */
|
||||
const shellParserOnlyPlugin = {
|
||||
name: 'dsh-shell-parser-only',
|
||||
resolveId(source: string): string | null {
|
||||
return source === '@yarnpkg/parsers' ? SHELL_PARSER_ENTRY : null
|
||||
},
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve the module proxy table at bundle time: every Node builtin or
|
||||
* replaced external the worker graph imports lands on its `./node/` proxy,
|
||||
@@ -61,7 +88,7 @@ export default defineConfig([{
|
||||
dts: false,
|
||||
clean: false,
|
||||
noExternal: [/.*/],
|
||||
plugins: [moduleProxyPlugin],
|
||||
plugins: [moduleProxyPlugin, shellParserOnlyPlugin],
|
||||
outputOptions: { inlineDynamicImports: true },
|
||||
}, {
|
||||
// Page half: an ordinary browser ES module the deployment's page imports. It
|
||||
|
||||
@@ -48,6 +48,12 @@ const windowsUnsupportedTests = process.platform === 'win32'
|
||||
// Windows. The worker always speaks POSIX; the Linux lanes hold the diff.
|
||||
'packages/experimental/webworker-runtime/tests/node/path-diff.spec.ts',
|
||||
'packages/experimental/webworker-runtime/tests/node/shim-diff.spec.ts',
|
||||
// The subprocess ladder over the worker's child_process face: its kill
|
||||
// rung reaches the in-worker process table through `process.kill`,
|
||||
// which the ladder's win32 branch replaces with taskkill-by-real-pid —
|
||||
// undeliverable to a table pid. The worker host always reports 'linux',
|
||||
// so the Linux lanes hold the ladder.
|
||||
'packages/experimental/webworker-runtime/tests/node/child-process.spec.ts',
|
||||
]
|
||||
: []
|
||||
|
||||
|
||||
Reference in New Issue
Block a user