From 9da174c3043aece0f71eb972dc2133d13e5c52a9 Mon Sep 17 00:00:00 2001 From: _Kerman Date: Mon, 7 Sep 2026 15:34:19 +0800 Subject: [PATCH 01/35] fix(web): display local media paths referenced in session prose Assistant prose that references a workspace-contained local media path (e.g. `![](/Users/.../x.png)`) now renders through a same-origin `GET|HEAD /api/file?path=` route instead of inert alt text. - ui-primitives: MarkdownText gains a settled-only MarkdownPathImages vocabulary gate (same posture as file mentions); no vocabulary means byte-identical output. - ui-chat: AssistantMarkdown supplies a page-stable rewrite vocabulary for absolute POSIX paths (local-path-media.ts). - session-controller: SessionMediaReferences plugin contribution mounts the route on the authenticated connection.fetch channel; per-request policy = workspace-root containment after realpath, regular file, allowlisted media extension (images additionally signature-checked), range/HEAD streaming, private no-store + nosniff, fail-closed statuses. - Agent Note added (feature/2026-09-07-session-prose-local-media-display). Closes #3662. --- ...ession-prose-local-media-display.i18n.yaml | 6 + ...09-07-session-prose-local-media-display.md | 39 +++ ...07-session-prose-local-media-display.zh.md | 39 +++ packages/api/session-controller/src/index.ts | 2 + .../src/media-references.ts | 257 +++++++++++++++++ .../tests/media-references.host.spec.ts | 265 ++++++++++++++++++ .../api/session-controller/tsconfig.host.json | 2 + .../src/client/chat/AssistantMarkdown.tsx | 11 +- .../src/client/chat/local-path-media.ts | 24 ++ ...stant-markdown-path-images.client.spec.tsx | 46 +++ .../tests/local-path-media.client.spec.ts | 30 ++ packages/client/ui-primitives/src/index.ts | 2 +- .../src/markdown/MarkdownText.tsx | 21 +- .../ui-primitives/src/markdown/render.tsx | 57 +++- .../markdown-path-images.client.spec.tsx | 83 ++++++ .../markdown-render-units.client.spec.tsx | 1 + 16 files changed, 872 insertions(+), 13 deletions(-) create mode 100644 .agents/notes/implemented/feature/2026-09-07-session-prose-local-media-display.i18n.yaml create mode 100644 .agents/notes/implemented/feature/2026-09-07-session-prose-local-media-display.md create mode 100644 .agents/notes/implemented/feature/2026-09-07-session-prose-local-media-display.zh.md create mode 100644 packages/api/session-controller/src/media-references.ts create mode 100644 packages/api/session-controller/tests/media-references.host.spec.ts create mode 100644 packages/client/ui-chat/src/client/chat/local-path-media.ts create mode 100644 packages/client/ui-chat/tests/assistant-markdown-path-images.client.spec.tsx create mode 100644 packages/client/ui-chat/tests/local-path-media.client.spec.ts create mode 100644 packages/client/ui-primitives/tests/markdown-path-images.client.spec.tsx diff --git a/.agents/notes/implemented/feature/2026-09-07-session-prose-local-media-display.i18n.yaml b/.agents/notes/implemented/feature/2026-09-07-session-prose-local-media-display.i18n.yaml new file mode 100644 index 0000000000..db2a7595b2 --- /dev/null +++ b/.agents/notes/implemented/feature/2026-09-07-session-prose-local-media-display.i18n.yaml @@ -0,0 +1,6 @@ +# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each +# side as of the last confirmed-consistent state. Both languages carry equal authority; +# after editing either side, bring the other along and re-record with: +# pnpm run verify-translation-pairing --write .agents/notes/implemented/feature/2026-09-07-session-prose-local-media-display.md +2026-09-07-session-prose-local-media-display.md: e2693a6192cb78641d8a96937b39a4b06e884f04 +2026-09-07-session-prose-local-media-display.zh.md: bc0c62868e3ffab44fc3effc1030d9caee331f26 diff --git a/.agents/notes/implemented/feature/2026-09-07-session-prose-local-media-display.md b/.agents/notes/implemented/feature/2026-09-07-session-prose-local-media-display.md new file mode 100644 index 0000000000..e2693a6192 --- /dev/null +++ b/.agents/notes/implemented/feature/2026-09-07-session-prose-local-media-display.md @@ -0,0 +1,39 @@ +# Agent Note: Session prose local media paths display through a same-origin file route + +Status: implemented + +English | [中文](2026-09-07-session-prose-local-media-display.zh.md) + +## Problem + +Assistant prose sometimes references an image by its local filesystem path (markdown `![](/Users/.../x.png)`). The Web renderer only allowed absolute HTTP(S) image destinations, so such references fell back to inert alt text: the browser cannot read Host files, and nothing served them. Searches of the formal and external issue trackers found no existing record, and issue #3662 logged the gap (Web cannot display local-path images referenced by agent answers). + +## Decision + +Local media paths in session prose render through one same-origin file route, with the rewrite vocabulary and the serving policy each owned where the repo's seams say they belong. + +- **Renderer seam (`ui-primitives`)**: `MarkdownText` gained a `MarkdownPathImages` vocabulary (`pathImages` prop) with the same settled-only gate as `fileMentions`: while a message streams, frozen cached blocks never bake in a vocabulary handler; the settled pass rewrites image destinations that fail the remote-URL allowlist, and a rewritten destination is emitted only when it is an absolute `http(s)`/`blob`/`data` URL. Without a vocabulary the renderer output is byte-identical to before. +- **Chat wiring (`ui-chat`)**: `AssistantMarkdown` supplies a page-stable vocabulary mapping absolute POSIX paths to same-origin `/api/file?path=…` GETs (`local-path-media.ts`); non-HTTP transports (Electron `file://`) and relative/protocol-relative destinations stay inert. +- **Host route (`session-controller`)**: `SessionMediaReferences`, a plugin contribution registered beside `SessionFileReferences`, mounts `GET|HEAD /api/file` on the shared authenticated `connection.fetch` channel (same trust fence and browser authentication as `/api` RPC). Per request it fail-closes: the path must be absolute, its `realpath` must lie inside a registered workspace root, the file must be regular, and its extension must name an allowlisted media type (PNG/JPEG/GIF/WebP/AVIF, MP4/WebM/MOV/OGG, MP3/WAV/Ogg/M4A/AAC/FLAC); image extensions are additionally checked against their byte signature. Responses stream with HTTP range support (206/416) so video and audio can seek, and carry `private, no-store` and `nosniff`. The contribution activates only where `connection` and `workspaceRegistry` are composed (pending-until-composed, like the package's other optional contributions). + +The route is presentational and stateless: it never writes, follows no redirects, and returns 400/403/404/415/416 instead of approximating another file-serving behavior. + +## Alternatives considered + +- **Register on the Typert gateway**: the gateway owns Remote RPC dispatch (endpoint claims, WebSocket mux, forwarded events), not file serving; placing the route there put HTTP presentation of workspace files into the RPC transport layer. Rejected and fully reverted. +- **Register under `workspace-controller`**: that package owns workspace registry lifecycle (CRUD, ordering, feed), and shares only the registry as a policy data source. Rejected and fully reverted. +- **Fetch bytes over the session RPC and show blob/data URLs**: attachment images already do this, but markdown rewriting needs a deterministic synchronous URL at render time (streaming freeze caches, memoization); an async round trip cannot be the render seam. Rejected. +- **A per-image or image-only route**: media types share one path-and-contain policy; video/audio need range streaming anyway. One `/api/file` route with an extension allowlist plus signature checks for images covers all current and next media types. Rejected as narrower alternatives. +- **Per-request interactive authorization, client-negotiated endpoints, or arbitrary host paths**: the rewrite is presentation; the route re-validates every request and limits readable bytes to registered workspace roots and allowlisted media, and the same-origin endpoint is a fixed channel contract rather than a negotiated capability. Rejected for security and determinism reasons. + +## Consequences + +- Assistant prose that references workspace-contained image files now displays them in Web chat; previously inert alt text disappears only where the host can serve the bytes. +- Policy is enforced host-side per request; the client vocabulary never expands what the route allows. +- Scope is deliberately narrow: only files under registered workspace roots with allowlisted media types are served; anything else keeps the authored fallback. Trajectory and tool-card markdown consumers do not pass a vocabulary yet, and video/audio markdown nodes are not rendered as `