docs(website): serve every page as raw Markdown with an llms.txt index

Append .md to any published route for the page as plain Markdown: the
build emits a raw-Markdown twin of every route (frontmatter dropped,
home bodies kept, images beside pages) plus a manifest-generated
llms.txt, the dev server serves both per request, and the post-build
gate fails when either is missing. Fixes #2846.
This commit is contained in:
Yichen Jiang
2026-08-20 22:10:15 +08:00
parent a67b9a4d31
commit f3ce8218cc
10 changed files with 493 additions and 37 deletions
@@ -0,0 +1,6 @@
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/process/2026-08-20-doc-site-raw-markdown-twins.md
2026-08-20-doc-site-raw-markdown-twins.md: 8174d67b43776d7264710d88d11540288bf7b63f
2026-08-20-doc-site-raw-markdown-twins.zh.md: 1babf957a6bdb1bd0a6ed91d6bbcb44b84aa713c
@@ -0,0 +1,35 @@
# Agent Note: Raw-Markdown twins and llms.txt on the documentation site
Status: implemented
English | [中文](2026-08-20-doc-site-raw-markdown-twins.zh.md)
## Problem
The documentation site serves rendered HTML only, so an agent reading the docs has to scrape VitePress markup or fall back to the repository, where links and image references follow source layout rather than public routes. Claude's platform documentation set the convention this feature adopts: append `.md` to any page URL for the page as raw Markdown, with a root `llms.txt` as the agent-facing index. [The site projection](2026-07-13-documentation-site-projection.md) rewrites every page's links for the public site already, so the gap was serving that projection as plain Markdown.
## Decision
`vitepress build` ends by emitting a raw-Markdown twin of every published route into the build output. `emitRawMarkdownPages` runs the same manifest-and-projector pass that fills `website/.generated/`, but writes `<outDir>/<route>` with raw page content: no `editSource`/`outline` projection frontmatter, no locale-home truncation — frontmatter is VitePress rendering configuration, so the twin drops it and keeps the body — and the same repository-chrome stripping as the rendered site. Referenced images are copied beside the twins.
One projection serves both trees because its site-internal links are relative. `./sibling.md` renders as a clean URL on the HTML site and resolves file-to-file in the raw tree, so the twins need no second link-rewriting mode. Every route is emitted, including the frontmatter-only locale homes, because published pages link to them and the raw tree must stay link-closed; a spec walks every emitted relative link to pin that closure.
`llms.txt` is generated from the publication manifest at the site root: both locale trees in sidebar order, one `- [label](<base><route>): <section>` row per page, links site-absolute under the deploy-time `DOCS_BASE`. Locale homes stay out — the file itself is the agent entry point.
The dev server serves the same surface. A middleware in the doc-projector plugin projects `.md` requests from their canonical sources per hit and generates `llms.txt` on demand, so `docs:dev` matches production without a rebuild.
`verify-doc-site-fragments`, the post-build gate, fails the build when any route's twin or `llms.txt` is missing from the output, so deleting the `buildEnd` wiring cannot pass CI.
## Alternatives considered
**`vitepress-plugin-llms`.** Actively maintained, MIT, used by the Vite, Vue, and Vitest sites; it generates per-page Markdown, `llms.txt`, `llms-full.txt`, and dev-server responses. Two hard-coded behaviors break this site. It flattens `dir/index.md` to `dir.md` with no opt-out while leaving in-page links unrewritten, which 404s every relative link into a section landing page — this site publishes seven index routes per locale. And it rewrites image references to root-absolute hashed bundle paths without the site base, which 404s on this subpath GitHub Pages deployment; its adopters deploy at domain roots and never hit either. Its `llms.txt` also reads sidebars only from the top-level theme config, not this site's per-locale ones. Correcting all that means a fork or a post-processing layer coupled to plugin internals — more owned surface than the small emitter reusing the tested projector.
**Emit through Vite's `publicDir`.** Vite supports one public directory and the site already points it at the tracked `website/public/`; generated twins would land in a tracked tree the layout gate forbids.
**Absolute links inside the twins.** platform.claude.com links absolutely because its host is fixed. This site's base varies between local `/` and the Pages subpath, and the projector's relative links resolve in both trees as they are, so absolute rewriting would add a second link grammar without improving resolution.
**Skip locale homes in the raw tree.** Published pages link to `docs/user/index.md`, so omitting the home routes breaks link closure. Their full bodies (H1 plus guide links) cost nothing, and the redirect frontmatter means nothing outside VitePress.
## Consequences
Agents fetch any page as plain Markdown at `<page URL>.md` and discover the whole set at `/llms.txt`; the rendered site is unchanged. The build output carries one extra Markdown file per route plus image copies beside them — kilobytes against the bundled assets. Twin content keeps GitHub-style heading text while the rendered site slugs punctuation-heavy headings differently; agents resolve headings themselves, so no gate covers raw-tree fragments. Deferred as unneeded for the twins' audience: `llms-full.txt`, and a per-page "view as Markdown" control, which would require a theme directory the stock-theme site deliberately lacks.
@@ -0,0 +1,35 @@
# Agent Note:文档站的纯 Markdown 孪生页与 llms.txt
Status: implemented
[English](2026-08-20-doc-site-raw-markdown-twins.md) | 中文
## Problem
文档站只提供渲染后的 HTML,agent 读文档要么抓取 VitePress 标记,要么退回仓库源文件,而后者的链接和图片引用跟随源码布局而非公开路由。Claude 平台文档确立了本特性采纳的约定:任意页面 URL 加 `.md` 后缀即得到该页的原始 Markdown,站根 `llms.txt` 作为面向 agent 的索引。[站点投影](2026-07-13-documentation-site-projection.md)本就为公开站点重写每页链接,缺的只是把这份投影以纯 Markdown 形式对外提供。
## Decision
`vitepress build` 结束时向构建输出发射每个已发布路由的纯 Markdown 孪生页。`emitRawMarkdownPages` 复用填充 `website/.generated/` 的同一趟 manifest 加投影器流程,但以原始页面内容写入 `<outDir>/<route>`:不带 `editSource`/`outline` 投影 frontmatter,不做 locale 首页截断——frontmatter 是 VitePress 的渲染配置,孪生页丢弃它并保留正文——仓库版式(语言切换行、徽章)的剥离与渲染站一致。页面引用的图片复制到孪生页旁边。
一份投影同时服务两棵树,因为站内链接是相对路径。`./sibling.md` 在 HTML 站渲染为 clean URL,在原始树中按文件对文件解析,孪生页不需要第二套链接改写模式。所有路由都被发射,包括仅有 frontmatter 的 locale 首页:已发布页面链接到它们,原始树必须保持链接封闭;一个 spec 遍历发射树中的每条相对链接来钉住这条闭合性。
`llms.txt` 由发布 manifest 生成于站根:两棵语言树按侧边栏顺序排列,每页一行 `- [label](<base><route>): <section>`,链接为携带部署期 `DOCS_BASE` 的站内绝对路径。locale 首页不列入——这个文件本身就是 agent 的入口。
开发服务器提供同一表面。doc-projector 插件中的 middleware 逐请求从 canonical 源投影 `.md` 请求并按需生成 `llms.txt``docs:dev` 无需重建即与线上一致。
构建后门禁 `verify-doc-site-fragments` 在任一路由的孪生页或 `llms.txt` 缺失时判定构建失败,删掉 `buildEnd` 接线无法通过 CI。
## Alternatives considered
**`vitepress-plugin-llms`。** 维护活跃、MIT 协议,Vite、Vue、Vitest 官方站点在用;能生成每页 Markdown、`llms.txt``llms-full.txt` 和开发服务器响应。但两个硬编码行为在本站产生破坏。它把 `dir/index.md` 展平为 `dir.md` 且无法关闭,同时不改写页内链接,所有指向栏目落地页的相对链接全部 404——本站每个语言树发布七个 index 路由。它还把图片引用改写为不带站点 base 的根绝对哈希资产路径,在本站的子路径 GitHub Pages 部署下必然 404;其采用者都部署在域名根,两个问题都碰不到。其 `llms.txt` 只认顶层 theme 配置的侧边栏,认不出本站按 locale 存放的侧边栏。修正这些意味着 fork 或一层耦合插件内部行为的后处理——比复用已测投影器的小发射器拥有更多自有面积。
**经 Vite `publicDir` 发射。** Vite 只支持一个 public 目录,本站已将其指向被跟踪的 `website/public/`;生成的孪生页会落入布局门禁禁止的跟踪树。
**孪生页内用绝对链接。** platform.claude.com 用绝对链接是因为其主机固定。本站 base 在本地 `/` 与 Pages 子路径之间变化,而投影器的相对链接在两棵树中原样可解析,绝对化改写只会引入第二套链接语法而不改善解析。
**原始树跳过 locale 首页。** 已发布页面链接到 `docs/user/index.md`,省略首页路由会破坏链接封闭。首页正文(H1 加指南链接)成本为零,重定向 frontmatter 在 VitePress 之外没有意义。
## Consequences
Agent 在 `<page URL>.md` 获取任意页面的纯 Markdown,并在 `/llms.txt` 发现全集;渲染站不变。构建输出为每个路由多带一个 Markdown 文件及旁置图片副本——相对打包资产只是千字节级。孪生页保留 GitHub 风格的标题文本,而渲染站对含标点的标题使用不同 slug;agent 自行解析标题,因此没有门禁覆盖原始树的 fragment。因孪生页受众不需要而暂缓:`llms-full.txt`,以及每页的"查看 Markdown"控件——后者需要 stock-theme 站点刻意不设的 theme 目录。
+1 -1
View File
@@ -5,7 +5,7 @@ description: Use when publishing, updating, moving, or removing DeepSeek Harness
# Synchronizing the DeepSeek Harness Documentation Site # Synchronizing the DeepSeek Harness Documentation Site
Keep repository Markdown as the only editable content source. Treat the website as a tested projection: [website/docs.ts](../../../website/docs.ts) selects public pages, [scripts/project-doc-site.ts](../../../scripts/project-doc-site.ts) rewrites them into the disposable `website/.generated/` tree, and VitePress builds that tree. Keep repository Markdown as the only editable content source. Treat the website as a tested projection: [website/docs.ts](../../../website/docs.ts) selects public pages, [scripts/project-doc-site.ts](../../../scripts/project-doc-site.ts) rewrites them into the disposable `website/.generated/` tree, and VitePress builds that tree. The build additionally emits a raw-Markdown twin of every route (`<page URL>.md`) and a root `llms.txt` index; both derive from the same manifest and projector, so publishing, moving, or removing a page updates them automatically and `docs:build` fails when one is missing.
Repository translations follow the sibling pairing contract: English `foo.md`, Chinese `foo.zh.md`, and `foo.i18n.yaml` live together. Never create `zh-CN/` or other locale directories for website content. The site route trees are independent of that source layout: `foo.zh.md` projects to the root route and `foo.md` projects to the matching `/en/` route. Repository translations follow the sibling pairing contract: English `foo.md`, Chinese `foo.zh.md`, and `foo.i18n.yaml` live together. Never create `zh-CN/` or other locale directories for website content. The site route trees are independent of that source layout: `foo.zh.md` projects to the root route and `foo.md` projects to the matching `/en/` route.
+161 -3
View File
@@ -3,11 +3,16 @@
import { execFileSync } from 'node:child_process' import { execFileSync } from 'node:child_process'
import { existsSync, globSync, mkdirSync, mkdtempSync, readFileSync, realpathSync, rmSync, symlinkSync, writeFileSync } from 'node:fs' import { existsSync, globSync, mkdirSync, mkdtempSync, readFileSync, realpathSync, rmSync, symlinkSync, writeFileSync } from 'node:fs'
import { tmpdir } from 'node:os' import { tmpdir } from 'node:os'
import { basename, join, resolve } from 'node:path' import { basename, dirname, join, resolve } from 'node:path'
import { afterEach, describe, expect, it } from 'vitest' import { fromMarkdown } from 'mdast-util-from-markdown'
import { gfmFromMarkdown } from 'mdast-util-gfm'
import { gfm } from 'micromark-extension-gfm'
import type { Nodes } from 'mdast'
import { afterAll, afterEach, beforeAll, describe, expect, it } from 'vitest'
import { docsPages, landingLink, routeLink, sectionSpec, type DocsPage } from '../website/docs.ts' import { docsPages, landingLink, routeLink, sectionSpec, type DocsPage } from '../website/docs.ts'
import { import {
addProjectionFrontmatter, projectedPageContent, publishableImage, resolveRepositoryRef, rewriteMarkdown, addProjectionFrontmatter, emitRawMarkdownPages, llmsTxt, projectedPageContent, publishableImage,
rawMarkdownPageContent, rawMarkdownRoute, resolveRepositoryRef, rewriteMarkdown,
} from './project-doc-site.ts' } from './project-doc-site.ts'
const roots: string[] = [] const roots: string[] = []
@@ -557,3 +562,156 @@ describe('projectedPageContent', () => {
.toThrow('locale home source "docs/index.zh.md" must start with YAML frontmatter') .toThrow('locale home source "docs/index.zh.md" must start with YAML frontmatter')
}) })
}) })
describe('rawMarkdownPageContent', () => {
it('keeps the home body the rendered site omits and drops the VitePress frontmatter', () => {
expect(rawMarkdownPageContent(
'---\nlayout: false\nhead:\n - - meta\n - http-equiv: refresh\n content: 0; url=./guide/quickstart\n---\n\n# Harness\n\nEnglish | [中文](./index.md)\n\nBody.\n',
)).toBe('# Harness\n\nBody.\n')
})
it('drops the language switcher and repository badge like the rendered site', () => {
const badge = '[![](https://img.shields.io/badge/powered_by-dsh-4D6BFE?style=flat-square)](https://github.com/deepseek-ai/deepseek-harness)'
expect(rawMarkdownPageContent(`# Guide\n\nEnglish | [中文](./x)\n\nBody.\n\n${badge}\n`))
.toBe('# Guide\n\nBody.\n')
})
it('rejects unclosed frontmatter', () => {
expect(() => rawMarkdownPageContent('---\nlayout: false\n')).toThrow('unclosed YAML frontmatter')
})
})
describe('emitRawMarkdownPages', () => {
function mirrorDir(): string {
const out = mkdtempSync(join(tmpdir(), 'dsh-doc-mirror-'))
roots.push(out)
return out
}
it('writes every route with rewritten links, placed images, and no projection frontmatter', () => {
const { root, pages } = fixture()
writeFileSync(join(root, 'docs/a.md'), '[B](b.md) ![logo](../packages/logo.svg)\n')
const out = mirrorDir()
// The real path, because image placement proves containment via realpath.
emitRawMarkdownPages(out, { pages, repoRoot: realpathSync(root), repositoryRef: 'abc123' })
expect(readFileSync(join(out, 'a.md'), 'utf8')).toBe('[B](./reference-root/b.md) ![logo](./logo.svg)\n')
expect(readFileSync(join(out, 'en/a.md'), 'utf8')).toBe('[B](./reference/b.md) ![logo](./logo.svg)\n')
expect(readFileSync(join(out, 'reference-root/b.md'), 'utf8')).toBe('# B\n')
expect(existsSync(join(out, 'logo.svg'))).toBe(true)
expect(existsSync(join(out, 'en/logo.svg'))).toBe(true)
})
it('emits the full body of a locale home page', () => {
const { root, pages } = fixture()
writeFileSync(join(root, 'docs/home.md'), '---\nlayout: false\n---\n\n# Home\n\n[A](a.md)\n')
pages.push({
locale: 'root', contentLocale: 'zh-CN', source: 'docs/home.md', route: 'index.md',
label: 'Home', sidebar: null, section: 'Home', order: 0,
})
const out = mirrorDir()
emitRawMarkdownPages(out, { pages, repoRoot: root, repositoryRef: 'abc123' })
expect(readFileSync(join(out, 'index.md'), 'utf8')).toBe('# Home\n\n[A](./a.md)\n')
})
})
describe('raw Markdown projection of the published manifest', () => {
let mirror: string
beforeAll(() => {
mirror = mkdtempSync(join(tmpdir(), 'dsh-doc-mirror-real-'))
emitRawMarkdownPages(mirror, { pages: docsPages, repoRoot: repositoryRoot, repositoryRef: 'master' })
})
afterAll(() => {
rmSync(mirror, { recursive: true, force: true })
})
it('emits every published route', () => {
for (const page of docsPages) {
expect(existsSync(join(mirror, page.route)), page.route).toBe(true)
}
})
it('emits home pages with their bodies instead of the frontmatter stub', () => {
for (const route of ['index.md', 'en/index.md']) {
const home = readFileSync(join(mirror, route), 'utf8')
expect(home.startsWith('---'), route).toBe(false)
expect(home, route).toContain('# DeepSeek Harness')
}
})
it('resolves every relative link inside the emitted tree', () => {
// Raw pages are read outside the site, so a relative target that only the
// rendered site serves would strand every agent following it.
const broken: string[] = []
for (const file of globSync('**/*.md', { cwd: mirror }).sort()) {
for (const target of relativeTargets(readFileSync(join(mirror, file), 'utf8'))) {
if (!existsSync(resolve(mirror, dirname(file), target))) broken.push(`${file}: ${target}`)
}
}
expect(broken).toEqual([])
})
})
function relativeTargets(markdown: string): string[] {
const tree = fromMarkdown(markdown, { extensions: [gfm()], mdastExtensions: [gfmFromMarkdown()] })
const targets: string[] = []
const visit = (node: Nodes): void => {
if ((node.type === 'link' || node.type === 'image' || node.type === 'definition') && 'url' in node) {
const external = node.url.startsWith('#')
|| node.url.startsWith('/')
|| /^[a-zA-Z][a-zA-Z0-9+.-]*:/.test(node.url)
const path = node.url.split(/[?#]/)[0] ?? ''
if (!external && path !== '') targets.push(decodeURIComponent(path))
}
if ('children' in node) {
for (const child of node.children) visit(child)
}
}
visit(tree)
return targets
}
describe('llmsTxt', () => {
const site = { base: '/x/', title: 'DeepSeek Harness', description: '插件化 SDK' }
it('lists every sidebar page as a base-prefixed raw-Markdown link', () => {
const text = llmsTxt(site)
for (const page of docsPages) {
if (page.sidebar === null) expect(text, page.route).not.toContain(`](/x/${page.route})`)
else expect(text, page.route).toContain(`- [${page.label}](/x/${page.route}): ${page.section}`)
}
})
it('groups the two locale trees under their own headings', () => {
const text = llmsTxt(site)
expect(text.indexOf('## 简体中文')).toBeGreaterThan(-1)
expect(text.indexOf('## English')).toBeGreaterThan(text.indexOf('## 简体中文'))
})
it('carries the site identity and the raw-Markdown convention', () => {
const text = llmsTxt(site)
expect(text.startsWith('# DeepSeek Harness\n')).toBe(true)
expect(text).toContain('> 插件化 SDK')
expect(text).toMatch(/`\.md`/)
})
})
describe('rawMarkdownRoute', () => {
it('projects one published route on demand', () => {
const { root, pages } = fixture()
writeFileSync(join(root, 'docs/a.md'), '# A\n\n[B](b.md)\n')
expect(rawMarkdownRoute('en/a.md', { pages, repoRoot: root, repositoryRef: 'abc123' }))
.toBe('# A\n\n[B](./reference/b.md)\n')
})
it('returns undefined for a path the manifest does not publish', () => {
const { root, pages } = fixture()
expect(rawMarkdownRoute('en/missing.md', { pages, repoRoot: root, repositoryRef: 'abc123' })).toBeUndefined()
})
})
+152 -16
View File
@@ -3,6 +3,8 @@
* *
* The generated tree is disposable: sources stay in their owning `docs/` * The generated tree is disposable: sources stay in their owning `docs/`
* tier, while this adapter rewrites cross-source links for the public site. * tier, while this adapter rewrites cross-source links for the public site.
* The same projection also emits a raw-Markdown twin of every route into the
* build output, so `<page URL>.md` serves the page as plain Markdown.
*/ */
import { import {
@@ -13,7 +15,7 @@ import { fromMarkdown } from 'mdast-util-from-markdown'
import { gfmFromMarkdown } from 'mdast-util-gfm' import { gfmFromMarkdown } from 'mdast-util-gfm'
import { gfm } from 'micromark-extension-gfm' import { gfm } from 'micromark-extension-gfm'
import type { Nodes } from 'mdast' import type { Nodes } from 'mdast'
import { docsPages, type DocsLocale, type DocsPage } from '../website/docs.ts' import { docsPages, orderedPages, type DocsLocale, type DocsPage, type DocsSidebar } from '../website/docs.ts'
import { import {
isExternalOrAbsoluteMarkdownUrl, isExternalOrAbsoluteMarkdownUrl,
markdownDestination, markdownDestination,
@@ -317,34 +319,50 @@ export function docsSourceFiles(): string[] {
return [...new Set([...docsPages.map(page => resolve(root, page.source)), ...referencedImages()])] return [...new Set([...docsPages.map(page => resolve(root, page.source)), ...referencedImages()])]
} }
/** Rebuild the disposable VitePress source tree from the publication manifest. */ /** Manifest and repository inputs for one projection pass. */
export function projectDocs(): void { export interface ProjectionContext {
/** Pages to project. */
pages: DocsPage[]
/** Repository root every source and placed image must live under. */
repoRoot: string
/** Public ref used by projected GitHub links. */
repositoryRef: string
}
function defaultProjectionContext(): ProjectionContext {
return { pages: docsPages, repoRoot: root, repositoryRef: resolveRepositoryRef(process.env) }
}
/** Project every page and its images into one target tree. */
function projectPagesInto(
targetRoot: string,
context: ProjectionContext,
pageContent: (markdown: string, page: DocsPage) => string,
): void {
const routes = new Set<string>() const routes = new Set<string>()
/** Projected path to the repository file that claimed it, pages and images alike. */ /** Projected path to the repository file that claimed it, pages and images alike. */
const claimed = new Map<string, string>() const claimed = new Map<string, string>()
const repositoryRef = resolveRepositoryRef(process.env)
rmSync(generatedRoot, { recursive: true, force: true })
/** Reserve one projected path, refusing a second source for it. */ /** Reserve one projected path, refusing a second source for it. */
const claim = (target: string, sourceAbs: string): void => { const claim = (target: string, sourceAbs: string): void => {
const holder = claimed.get(target) const holder = claimed.get(target)
if (holder !== undefined && holder !== sourceAbs) { if (holder !== undefined && holder !== sourceAbs) {
throw new Error( throw new Error(
`project-doc-site: ${repoPath(sourceAbs, root)} and ${repoPath(holder, root)}` `project-doc-site: ${repoPath(sourceAbs, context.repoRoot)} and ${repoPath(holder, context.repoRoot)}`
+ ` both project to ${relative(generatedRoot, target).split(sep).join('/')}.`, + ` both project to ${relative(targetRoot, target).split(sep).join('/')}.`,
) )
} }
claimed.set(target, sourceAbs) claimed.set(target, sourceAbs)
} }
for (const page of docsPages) { for (const page of context.pages) {
if (routes.has(page.route)) throw new Error(`project-doc-site: duplicate route ${JSON.stringify(page.route)}.`) if (routes.has(page.route)) throw new Error(`project-doc-site: duplicate route ${JSON.stringify(page.route)}.`)
routes.add(page.route) routes.add(page.route)
const sourceAbs = resolve(root, page.source) const sourceAbs = resolve(context.repoRoot, page.source)
if (!existsSync(sourceAbs) || !lstatSync(sourceAbs).isFile()) { if (!existsSync(sourceAbs) || !lstatSync(sourceAbs).isFile()) {
throw new Error(`project-doc-site: source ${JSON.stringify(page.source)} does not exist or is not a file.`) throw new Error(`project-doc-site: source ${JSON.stringify(page.source)} does not exist or is not a file.`)
} }
const output = resolve(generatedRoot, page.route) const output = resolve(targetRoot, page.route)
// Claimed before the images are placed: a page and an image landing on one // Claimed before the images are placed: a page and an image landing on one
// path would otherwise overwrite each other in whichever order they ran. // path would otherwise overwrite each other in whichever order they ran.
claim(output, sourceAbs) claim(output, sourceAbs)
@@ -354,14 +372,14 @@ export function projectDocs(): void {
sourcePath: page.source, sourcePath: page.source,
locale: page.locale, locale: page.locale,
route: page.route, route: page.route,
pages: docsPages, pages: context.pages,
repoRoot: root, repoRoot: context.repoRoot,
repositoryRef, repositoryRef: context.repositoryRef,
placeImage: (absPath) => { placeImage: (absPath) => {
const real = publishableImage(absPath, root) const real = publishableImage(absPath, context.repoRoot)
if (real === undefined) { if (real === undefined) {
throw new Error( throw new Error(
`project-doc-site: ${page.source} references image ${repoPath(absPath, root)},` `project-doc-site: ${page.source} references image ${repoPath(absPath, context.repoRoot)},`
+ ' which is not a regular file inside the repository.', + ' which is not a regular file inside the repository.',
) )
} }
@@ -377,6 +395,124 @@ export function projectDocs(): void {
return `./${encodeURI(name)}` return `./${encodeURI(name)}`
}, },
}) })
writeFileSync(output, addProjectionFrontmatter(projectedPageContent(projected, page), page)) writeFileSync(output, pageContent(projected, page))
} }
} }
/** Rebuild the disposable VitePress source tree from the publication manifest. */
export function projectDocs(): void {
rmSync(generatedRoot, { recursive: true, force: true })
projectPagesInto(generatedRoot, defaultProjectionContext(), (markdown, page) =>
addProjectionFrontmatter(projectedPageContent(markdown, page), page))
}
/**
* Strip the leading YAML frontmatter of a projected page.
*
* @param markdown Rewritten canonical Markdown content.
* @returns The content after the frontmatter block, or the input when none opens it.
*/
function withoutFrontmatter(markdown: string): string {
if (!markdown.startsWith('---\n')) return markdown
const closingDelimiter = '\n---\n'
const closing = markdown.indexOf(closingDelimiter, 4)
if (closing === -1) throw new Error('project-doc-site: page has unclosed YAML frontmatter.')
return markdown.slice(closing + closingDelimiter.length).replace(/^\n+/, '')
}
/**
* The raw-Markdown twin of one published page.
*
* Frontmatter is VitePress rendering configuration, so it is dropped rather
* than truncated to: a locale home page keeps its body here even though the
* rendered site replaces it with a redirect.
*
* @param markdown Rewritten canonical Markdown content.
* @returns Plain Markdown without frontmatter or repository chrome.
*/
export function rawMarkdownPageContent(markdown: string): string {
return withoutRepositoryChrome(withoutFrontmatter(markdown))
}
/**
* Emit the raw-Markdown twin of every published route into a built site, so
* static hosting serves `<page URL>.md` beside each rendered page. Referenced
* images are copied beside the pages, keeping the same relative URLs valid in
* both trees. Existing build files stay in place.
*
* @param outDir Build output directory to emit into.
* @param context Manifest and repository inputs, defaulting to this repository.
*/
export function emitRawMarkdownPages(outDir: string, context: ProjectionContext = defaultProjectionContext()): void {
projectPagesInto(outDir, context, markdown => rawMarkdownPageContent(markdown))
}
/**
* Raw Markdown served for one site route.
*
* Dev-server counterpart of {@link emitRawMarkdownPages}: images are not
* copied because the generated tree already serves them beside the page.
*
* @param route Manifest route, including its `.md` suffix.
* @param context Manifest and repository inputs, defaulting to this repository.
* @returns The projected page, or `undefined` when the manifest does not publish the route.
*/
export function rawMarkdownRoute(route: string, context: ProjectionContext = defaultProjectionContext()): string | undefined {
const page = context.pages.find(candidate => candidate.route === route)
if (page === undefined) return undefined
const markdown = readFileSync(resolve(context.repoRoot, page.source), 'utf8')
return rawMarkdownPageContent(rewriteMarkdown(markdown, {
sourcePath: page.source,
locale: page.locale,
route: page.route,
pages: context.pages,
repoRoot: context.repoRoot,
repositoryRef: context.repositoryRef,
placeImage: absPath => `./${encodeURI(basename(absPath))}`,
}))
}
/** Site identity written into llms.txt. */
export interface LlmsTxtSite {
/** Site base path, carrying the leading and trailing slashes VitePress requires. */
base: string
/** Site title. */
title: string
/** Site description. */
description: string
}
/** Locale groups llms.txt lists, in the order the site's navigation presents them. */
const llmsTxtLocales: readonly { heading: string; locale: DocsLocale; collections: readonly DocsSidebar[] }[] = [
{ heading: '简体中文', locale: 'root', collections: ['zh-guide', 'zh-develop', 'zh-reference'] },
{ heading: 'English', locale: 'en', collections: ['en-guide', 'en-develop', 'en-reference'] },
]
/**
* The llms.txt index of every published page's raw-Markdown twin.
*
* Links are site-absolute so an agent resolves them against the host it
* fetched llms.txt from; locale home pages stay out because this file is the
* agent-facing entry point itself.
*
* @param site Site identity and base path.
* @returns llms.txt content listing both locale trees.
*/
export function llmsTxt(site: LlmsTxtSite): string {
const lines = [
`# ${site.title}`,
'',
`> ${site.description}`,
'',
'每个页面的 URL 加 `.md` 后缀即为该页的原始 Markdown。Append `.md` to any page URL for its raw Markdown source.',
]
for (const { heading, locale, collections } of llmsTxtLocales) {
lines.push('', `## ${heading}`, '')
for (const collection of collections) {
for (const page of orderedPages(locale, collection)) {
lines.push(`- [${page.label}](${site.base}${page.route}): ${page.section}`)
}
}
}
return `${lines.join('\n')}\n`
}
+11 -1
View File
@@ -4,7 +4,7 @@ import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs'
import { tmpdir } from 'node:os' import { tmpdir } from 'node:os'
import { join } from 'node:path' import { join } from 'node:path'
import { afterEach, describe, expect, it } from 'vitest' import { afterEach, describe, expect, it } from 'vitest'
import { inspectSiteFragments } from './verify-doc-site-fragments.ts' import { inspectSiteFragments, missingSiteFiles } from './verify-doc-site-fragments.ts'
const roots: string[] = [] const roots: string[] = []
@@ -85,3 +85,13 @@ describe('inspectSiteFragments', () => {
]) ])
}) })
}) })
describe('missingSiteFiles', () => {
it('reports the expected files a build did not emit', () => {
const root = fixture()
writeFileSync(join(root, 'guide/start.md'), '# Ready\n')
expect(missingSiteFiles(root, ['guide/start.md', 'guide/absent.md', 'llms.txt']))
.toEqual(['guide/absent.md', 'llms.txt'])
})
})
+33 -10
View File
@@ -1,15 +1,17 @@
/** /**
* Verify fragment links against the HTML emitted by VitePress. Markdown and * Verify fragment links against the HTML emitted by VitePress, and that the
* VitePress use different heading-slug algorithms, so source-link validation * build carries the raw-Markdown twin of every route plus llms.txt. Markdown
* alone cannot prove that a published fragment exists. * and VitePress use different heading-slug algorithms, so source-link
* validation alone cannot prove that a published fragment exists.
* *
* This runs as part of `docs:build` and can also run directly after a build * This runs as part of `docs:build` and can also run directly after a build
* with `tsx scripts/verify-doc-site-fragments.ts`. * with `tsx scripts/verify-doc-site-fragments.ts`.
*/ */
import { globSync, readFileSync } from 'node:fs' import { existsSync, globSync, readFileSync } from 'node:fs'
import { resolve, sep } from 'node:path' import { resolve, sep } from 'node:path'
import { JSDOM } from 'jsdom' import { JSDOM } from 'jsdom'
import { docsPages } from '../website/docs.ts'
const root = resolve(import.meta.dirname, '..') const root = resolve(import.meta.dirname, '..')
@@ -138,18 +140,39 @@ export function inspectSiteFragments(distRoot: string): SiteFragmentReport {
return { checked, broken } return { checked, broken }
} }
/**
* Expected files a build did not emit.
*
* @param distRoot - Directory containing the built site.
* @param expected - Site-relative files the build must carry.
* @returns The absent files, in the given order.
*/
export function missingSiteFiles(distRoot: string, expected: readonly string[]): string[] {
return expected.filter(file => !existsSync(resolve(distRoot, file)))
}
function main(): number { function main(): number {
const distRoot = resolve(root, 'website/.dist') const distRoot = resolve(root, 'website/.dist')
const report = inspectSiteFragments(distRoot) const report = inspectSiteFragments(distRoot)
if (report.broken.length === 0) { const missing = missingSiteFiles(distRoot, [...docsPages.map(page => page.route), 'llms.txt'])
console.log(`verify-doc-site-fragments: ${report.checked} internal fragment reference(s) resolve.`) if (report.broken.length === 0 && missing.length === 0) {
console.log(
`verify-doc-site-fragments: ${report.checked} internal fragment reference(s) resolve;`
+ ` ${docsPages.length} raw-Markdown route(s) and llms.txt emitted.`,
)
return 0 return 0
} }
console.error(`verify-doc-site-fragments: ${report.broken.length} broken fragment reference(s):`) if (report.broken.length > 0) {
for (const item of report.broken) { console.error(`verify-doc-site-fragments: ${report.broken.length} broken fragment reference(s):`)
const target = item.target === undefined ? 'target route was not built' : `${item.target} has no id ${JSON.stringify(item.fragment)}` for (const item of report.broken) {
console.error(` ${item.source}: ${JSON.stringify(item.href)} (${target})`) const target = item.target === undefined ? 'target route was not built' : `${item.target} has no id ${JSON.stringify(item.fragment)}`
console.error(` ${item.source}: ${JSON.stringify(item.href)} (${target})`)
}
}
if (missing.length > 0) {
console.error(`verify-doc-site-fragments: ${missing.length} expected raw-Markdown file(s) missing from the build:`)
for (const file of missing) console.error(` ${file}`)
} }
return 1 return 1
} }
+57 -6
View File
@@ -1,12 +1,12 @@
/** VitePress configuration for the locally projected documentation site. */ /** VitePress configuration for the locally projected documentation site. */
import { readFileSync } from 'node:fs' import { readFileSync, writeFileSync } from 'node:fs'
import { resolve } from 'node:path' import { resolve } from 'node:path'
import type { DefaultTheme, PageData } from 'vitepress' import type { DefaultTheme, PageData, SiteConfig } from 'vitepress'
import type { ViteDevServer } from 'vite' import type { ViteDevServer } from 'vite'
import { withMermaid } from 'vitepress-plugin-mermaid' import { withMermaid } from 'vitepress-plugin-mermaid'
import { landingLink, orderedPages, routeLink, sectionSpec, type DocsLocale, type DocsPage, type DocsSidebar } from '../docs.ts' import { landingLink, orderedPages, routeLink, sectionSpec, type DocsLocale, type DocsPage, type DocsSidebar } from '../docs.ts'
import { docsSourceFiles, projectDocs } from '../../scripts/project-doc-site.ts' import { docsSourceFiles, emitRawMarkdownPages, llmsTxt, projectDocs, rawMarkdownRoute } from '../../scripts/project-doc-site.ts'
projectDocs() projectDocs()
@@ -111,6 +111,43 @@ function watchCanonicalDocs(server: ViteDevServer): void {
}) })
} }
/**
* Serve the raw-Markdown twin of each route and llms.txt during development,
* matching what `buildEnd` emits into the static build. Pages project from
* their canonical sources per request, so an edit shows without a rebuild.
*/
function serveRawMarkdown(server: ViteDevServer): void {
server.middlewares.use((req, res, next) => {
if (req.url === undefined || (req.method !== 'GET' && req.method !== 'HEAD')) {
next()
return
}
// The dev client imports page modules at these same `.md` URLs, and a
// module script must reach Vite's transform. Browsers declare the purpose:
// `script` for module imports, `document` for address-bar navigation.
// Header-less clients (curl, agents) read the raw twin.
const fetchDest = req.headers['sec-fetch-dest']
if (fetchDest !== undefined && fetchDest !== 'document') {
next()
return
}
const pathname = req.url.split(/[?#]/, 1)[0] ?? ''
const sitePath = pathname.startsWith(base) ? pathname.slice(base.length) : pathname.replace(/^\//, '')
if (sitePath === 'llms.txt') {
res.setHeader('Content-Type', 'text/plain; charset=utf-8')
res.end(llmsTxt({ base, ...siteIdentity }))
return
}
const content = sitePath.endsWith('.md') ? rawMarkdownRoute(sitePath) : undefined
if (content === undefined) {
next()
return
}
res.setHeader('Content-Type', 'text/markdown; charset=utf-8')
res.end(content)
})
}
function escapeVueInterpolation(html: string): string { function escapeVueInterpolation(html: string): string {
return html.replaceAll('{{', '&#123;&#123;').replaceAll('}}', '&#125;&#125;') return html.replaceAll('{{', '&#123;&#123;').replaceAll('}}', '&#125;&#125;')
} }
@@ -163,6 +200,12 @@ const sharedTheme: Pick<DefaultTheme.Config, 'search' | 'socialLinks' | 'editLin
/** Site base path, carrying the leading and trailing slashes VitePress requires. */ /** Site base path, carrying the leading and trailing slashes VitePress requires. */
const base = process.env.DOCS_BASE ?? '/' const base = process.env.DOCS_BASE ?? '/'
/** Site identity shared by the VitePress configuration and the llms.txt index. */
const siteIdentity = {
title: 'DeepSeek Harness',
description: '用于构建 Agent Harness 的插件化 SDK',
}
/** /**
* The DeepSeek wordmark, inlined so its `currentColor` fills follow the active * The DeepSeek wordmark, inlined so its `currentColor` fills follow the active
* theme. An `<img>` would freeze the mark at the colors the file declares. * theme. An `<img>` would freeze the mark at the colors the file declares.
@@ -247,9 +290,14 @@ function siteTitle(previewTag: string): string {
} }
export default withMermaid({ export default withMermaid({
title: 'DeepSeek Harness', title: siteIdentity.title,
description: '用于构建 Agent Harness 的插件化 SDK', description: siteIdentity.description,
base, base,
/** Emit the raw-Markdown twin of every route plus llms.txt beside the rendered site. */
buildEnd(siteConfig: SiteConfig) {
emitRawMarkdownPages(siteConfig.outDir)
writeFileSync(resolve(siteConfig.outDir, 'llms.txt'), llmsTxt({ base, ...siteIdentity }))
},
head: [ head: [
// VitePress leaves head hrefs untouched, so the base belongs here explicitly. // VitePress leaves head hrefs untouched, so the base belongs here explicitly.
['link', { rel: 'icon', type: 'image/svg+xml', href: `${base}favicon.svg` }], ['link', { rel: 'icon', type: 'image/svg+xml', href: `${base}favicon.svg` }],
@@ -322,7 +370,10 @@ export default withMermaid({
plugins: [ plugins: [
{ {
name: 'deepseek-harness-doc-projector', name: 'deepseek-harness-doc-projector',
configureServer: watchCanonicalDocs, configureServer(server) {
watchCanonicalDocs(server)
serveRawMarkdown(server)
},
}, },
], ],
}, },
+2
View File
@@ -10,4 +10,6 @@ Keep canonical prose and generated catalogs in their owning `docs/` tier, then e
The projector writes disposable Markdown to the ignored `website/.generated/` directory. Never edit or commit `.generated/`, `.cache/`, or `.dist/`. The projector writes disposable Markdown to the ignored `website/.generated/` directory. Never edit or commit `.generated/`, `.cache/`, or `.dist/`.
The build also emits each route's raw-Markdown twin and a root `llms.txt` index into `.dist/`, so `<page URL>.md` serves the page as plain Markdown. Both derive from the publication manifest at build time; neither is ever a file in this tree.
Run `pnpm docs:check` after changing this subtree; the gate rejects additional non-ignored Markdown under `website/`. Run `pnpm docs:check` after changing this subtree; the gate rejects additional non-ignored Markdown under `website/`.