diff --git a/docs/i18n/translation-prompt.md b/docs/i18n/translation-prompt.md index 8bc15d6e8e..d59170b00a 100644 --- a/docs/i18n/translation-prompt.md +++ b/docs/i18n/translation-prompt.md @@ -1,6 +1,6 @@ # Translation prompt (pipeline asset) -本文件是自动翻译流水线使用的 prompt 模板；从 `# Translation Prompt` 开始的正文会逐字进入模型请求，因此本文件不参与双语配对（见 [README.md](README.md) 排除清单）。渲染时会把 [translation-rules.md](translation-rules.md) 全文填入 `{{translation_rules}}`，把 [terminology.md](terminology.md) 整表填入 `{{terminology}}`，以免模板另存一份规则而日后失去同步。[style-samples.md](style-samples.md) 定义文体，模板中的 Examples 只用于说明典型问题；术语表、忠实性和结构规则优先于样例，样例只在这些硬性约束内决定文体。修改本文件会改变翻译行为，需正常经过 PR 评审。 +本文件是自动翻译流水线的 prompt 模板；从 `# Translation Prompt` 开始的正文会逐字进入模型请求，因此本文件不参与双语配对（见 [README.md](README.md) 排除清单）。模板正文与内嵌 few-shot 正误例由 jingtingxiang 基于对存量译文的质量评审撰写，是流水线行为的拍板基线。渲染时把 [terminology.md](terminology.md) 整表填入 `{{terminology}}`；除此之外不注入任何其他仓库文件（translation-rules.md 约束人和 agent 的翻译工作，不注入本模板）。[style-samples.md](style-samples.md) 定义文体，模板中的 Examples 只用于说明典型问题，两者冲突时以文体样例为准。修改本文件会改变翻译行为，需正常经过 PR 评审。 ## 占位符契约 @@ -10,18 +10,13 @@ |---|---|---| | `{{source_lang}}` | 源语言名（`English` / `Chinese`） | 由改动侧文件推断：`.zh.md` 被改则为 `Chinese` | | `{{target_lang}}` | 目标语言名（`Chinese` / `English`） | 与 `{{source_lang}}` 相对 | -| `{{translation_rules}}` | [translation-rules.md](translation-rules.md) 全文（Markdown 原文） | 渲染时读取仓库当前版本，不缓存 | | `{{terminology}}` | [terminology.md](terminology.md) 的完整表格（Markdown 原文） | 渲染时读取仓库当前版本，不缓存 | -| `{{source_filename}}` | 源文档的 basename（如 `foo.md` 或 `foo.zh.md`） | 由流水线从待译文件路径取得 | -| `{{source_filename_zh}}` | 中文侧 basename（如 `foo.zh.md`） | 英文源追加 `.zh`；中文源使用自身 basename | -例如，英译中时若源文件是 `foo.md`，`{{source_filename}}` 填 `foo.md`，`{{source_filename_zh}}` 填 `foo.zh.md`；中译英时若源文件是 `foo.zh.md`，两个占位符都填 `foo.zh.md`。 - -流水线只识别上表中的占位符，并且一次翻译整篇文档。它不支持 `{{to}}`、`{{title_prompt}}`、`{{summary_prompt}}`、`{{terms_prompt}}`、`{{imt_style_guide}}` 或 `%%` 分段协议。输出必须是一个以 `` 为根元素的 XML 文档；三个子元素中的 Markdown 内容都放在 CDATA 中。内容出现 `]]>` 时写成 `]]]]>`，XML 解析后仍会还原为原文。 +流水线只识别上表中的占位符，并且一次翻译整篇文档。它不支持 `{{to}}`、`{{title_prompt}}`、`{{summary_prompt}}`、`{{terms_prompt}}`、`{{imt_style_guide}}`、`{{translation_rules}}` 或 `%%` 分段协议；输出采用模板正文规定的三段 XML，流水线解析取 `` 段。 ## Few-shot 金标 -流水线使用**整篇文档**的中英对照作为 few-shot，不是模板内嵌的句子级正误例。以下 5 组配对文档均经过人工评审，并以仓库当前版本为准，随仓库一同更新： +流水线使用**整篇文档**的中英对照作为 few-shot，不是模板内嵌的句子级正误例。以下 5 组配对文档均经过人工评审，以仓库当前版本为准、随仓库更新： - `README.md` ↔ `README.zh.md` - `docs/development.md` ↔ `docs/development.zh.md` @@ -29,58 +24,131 @@ - `docs/i18n/translation-rules.md` ↔ `docs/i18n/translation-rules.zh.md` - `docs/rfc/implemented/process/2026-07-02-bilingual-docs-and-pairing-gate.md` ↔ 对应 `.zh.md` -注入时按当前翻译方向选择每组的源侧与目标侧：user 消息包含源文档全文，assistant 消息采用模板正文规定的 XML 协议；`translation` 与 `final` 都放入目标文档全文，`review` 填 `- [None] No corrections.`。CDATA 遵循上文的 `]]>` 拆分规则。上下文不足时，按上列顺序从后往前删减示例组数。这 5 组也是评审校准锚点；改动任何一组都会改变流水线行为。 +注入方式：在系统消息（本模板）之后、待译文档之前，每组作为一轮示例对话——user 消息为源文档全文，assistant 消息为定稿译文全文（裸文本，不带三段 XML 包装；只有真实请求要求三段输出）。上下文不足时按上列顺序从后往前删减组数。这 5 组也是评审校准锚点（见 [style-samples.md](style-samples.md)），改动任何一组即改变流水线行为。 ## 模板正文 ````text # Translation Prompt -You are a senior technical translator specializing in LLM and agent development documentation. Translate the complete source document from {{source_lang}} to {{target_lang}} as natural, professional technical prose. +You are a senior technical translator specializing in LLM and agent development documentation. Your task is to translate the given source document from {{source_lang}} to {{target_lang}}, producing natural, professional technical prose. -## Binding Translation Rules +## Quality Requirements -The canonical repository rules below are injected verbatim. Apply every direction-appropriate requirement. In those rules, the authored document is the source for this request and the generated document is its counterpart. +### Structure and Format Preservation +- Output a complete translated document that maintains exactly the same structure as the source: heading hierarchy, list shape, table columns, link targets, and code blocks. +- Fenced code blocks must be byte-identical to the source, including ALL comments inside them. Do NOT translate comments inside code blocks. This is a hard rule with no exceptions. +- Inline code spans (commands, flags, paths, API names, version numbers) must be kept verbatim. Never translate or reformat them. +- Every relative link must point to the same target as in the source. Link text is translated; link targets are not. +- Language switcher line: when translating into Chinese, write `[English](source-filename.md) | 中文`. When translating into English, write `English | [中文](source-filename.zh.md)`. Do NOT copy the switcher line from the source file unchanged — you must flip the link direction. +- After a closing bold marker `**`, always insert a space before the next character. -{{translation_rules}} +### Tone and Style +- The translation must read as if originally written in the target language by a native speaker. If an expression sounds like a word-for-word rendering from the source language, rephrase it. +- Write in a professional, formal tone appropriate for developer documentation. Never use colloquial or casual expressions. +- Use polite imperative forms where the text instructs the reader to do something. +- Keep the author's register: concise stays concise, detailed stays detailed. -## Request-Specific Structure +### Sentence Structure +- Break long sentences with commas or semicolons. Avoid run-on sentences. +- Prefer active voice. Convert passive constructions to active if it reads more naturally. +- Translate meaning, not words. Restructure sentences where the target language grammar requires it. +- Do not invent words or expressions that do not exist in natural technical writing of the target language. -- The source basename is `{{source_filename}}`. When translating into Chinese, write `[English]({{source_filename}}) | 中文` immediately after the H1. When translating into English, write `English | [中文]({{source_filename_zh}})` immediately after the H1. -- Emit the switcher for a new pair and flip an existing switcher; never copy it unchanged. +### Word Choice +- Prefer precise, formal vocabulary over casual or colloquial alternatives. +- When multiple synonyms exist, choose the one most commonly used in professional technical documentation of the target language. +- Avoid slang, internal jargon, or overly literal translations that would not be recognized by the general developer audience. +- Do not use the same word to translate two different source-language terms that carry distinct meanings. +- Avoid repeating the same verb in close proximity; vary word choice for readability. -## Binding Terminology +#### When translating into Chinese +- When a number modifies a noun, always include a Chinese classifier or measure word (量词). For example: "three-package seam" → "由三个包构成的 seam", not "三包 seam". -Apply the current table below exactly as required by the injected translation rules. +### Punctuation + +#### When translating into Chinese +- Use full-width Chinese punctuation in prose: `，。：；？！（）「」`. +- Strongly prefer replacing all em-dashes (——) with colons, periods, commas, or parentheses. Keep an em-dash only if no other punctuation works at all. +- Use enumeration commas (、) between parallel items, not regular commas. +- List item endings: use semicolons or no punctuation. Do not end list items with commas. +- Put one half-width space between Chinese text and Latin words/numbers. +- For RFC 2119 keywords (MUST, MUST NOT, SHOULD, MAY), render the corresponding Chinese term in italics: *必须*、*禁止*、*应当*、*可以*. + +#### When translating into English +(To be added.) + +## Terminology + +A terminology table is provided below. Follow it strictly: +- Render every listed term exactly as specified. +- First occurrence: write as shown in the "首次出现" column (with parenthetical gloss). Subsequent occurrences: write only the part before the parentheses. +- If a term has already been glossed as part of a compound term, do not gloss it again when it appears alone later. +- NEVER use translations listed in the "不要译作" column. +- For technical terms not in the table: keep them in the source language. Do not invent a translation. This rule applies to terminology only; for general prose, freely restructure and paraphrase for natural expression. {{terminology}} ## Output Format -Return exactly one well-formed XML document with this root and these three child elements. Do not wrap it in a Markdown code fence. Put all Markdown and review text inside CDATA. If any content contains the CDATA terminator, split it as `]]]]>` so XML parsing reconstructs the original `]]>` sequence. +Produce your output in three XML sections: ```xml - - - - - + +(Complete translation of the source document) + + + +(Self-review notes, one correction per line with category tag, e.g.) +- [Tone] "旁挂记录" → "伴随记录"（生造词） +- [Sentence] 第 3 段补充逗号断句 +- [Punctuation] 两处破折号替换为冒号 +- 无修正 + + + +(Final translation after corrections) + ``` ## Self-Review Instructions -After writing ``, re-read it in the target language without looking at the source. Then apply the injected translation rules as a clause-by-clause comparison against the source and record actual corrections in English inside ``. Apply every recorded correction in ``. If no correction is needed, write only `- [None] No corrections.` in `` and copy `` unchanged into ``. +After writing ``, re-read it in the target language only, without looking at the source. Check by category: + +**Structure** +- Is the heading hierarchy, list shape, and code block content identical to the source? +- Are ALL comments inside code blocks left untranslated (byte-identical to source)? +- Is the language switcher line correctly flipped (not copied from source)? +- Are link targets preserved and bold markers followed by a space? + +**Tone & Style** +- Does every sentence read as if originally written by a native speaker? +- Is there any colloquial, casual, or overly informal phrasing? + +**Sentence Structure** +- Are there run-on sentences that need breaking? +- Are there stiff passive constructions that should be converted to active voice? + +**Word Choice** +- Are there overly literal translations that sound unnatural? +- Is the same target-language word used to translate two distinct source concepts? +- Is any slang or internal jargon present? + +**Terminology** +- Are first-occurrence glosses correctly applied (not missing, not repeated)? +- Are any "不要译作" forbidden translations present? +- Are unlisted terms correctly kept in the source language? + +**Punctuation** (when target is Chinese) +- Are there em-dashes that should be replaced with colons, periods, or commas? +- Are list items ending with commas instead of semicolons? +- Are RFC 2119 keywords rendered in italics? + +Record corrections in `` with category tags. Then output the corrected version in ``. If no corrections are needed, write "无修正" in `` and copy the translation unchanged into ``. ## Examples -Follow the Good versions; these sentence-level examples illustrate error categories, not the assistant-message wire format. +Below are representative examples of common problems and their corrections. Follow the "Good" versions. ### Colloquial verb → Professional verb - Source: `The repo pins pnpm@11.7.0 in package.json` @@ -102,40 +170,40 @@ Follow the Good versions; these sentence-level examples illustrate error categor - Bad: `旁挂记录两侧 blob hash，使一致性可检查` - Good: `伴随记录保存两侧 blob hash，使一致性可检查` +### Em-dash → Colon/period +- Source: `FIXME — an issue that should block a new release. A release should not ship with an open FIXME unless reviewers explicitly agree the change can be merged anyway.` +- Bad: `FIXME——应当阻塞新版本发布的问题。除非评审者明确同意可以照常合入，发布不应带着未解决的 FIXME 出门。` +- Good: `FIXME：应当阻塞新版本发布的问题。除非评审者明确同意该更改可以合并，否则发布版本不应包含未解决的 FIXME。` + ### Overly literal → Meaningful rendering - Source: `awkward phrasing is easier to hear without the source anchoring you` - Bad: `没有源文锚着，别扭的表述更容易被听出来` - Good: `不对照原文时，更容易察觉别扭的表达` -### Terminology — keep the binding English form +### Terminology — do not translate what should be kept in English - Source: `typed service seams, and explicit extension points` - Bad: `类型化的服务 seam（扩展点）与显式扩展点` - Good: `类型化的服务 seam 与显式扩展点` -### Slang → Professional phrasing +### Slang/jargon → Professional phrasing - Source: `The committed agent workflow lives in .agents/skills/dsh-translate-docs` - Bad: `进仓的 agent 工作流见 .agents/skills/dsh-translate-docs` - Good: `仓库内置的 agent 工作流见 .agents/skills/dsh-translate-docs` -### Chinese → English — idiomatic subject and predicate -- Source: `门禁绿并不代表译文内容正确。` -- Bad: `The gate green does not represent that the translation content is correct.` -- Good: `A green gate does not mean the translation is correct.` +### "For humans" — translate the intent, not the word +- Source: `For humans, start with the development guide` +- Bad: `对于人工读者，请先从开发指南开始`（"人工读者"生硬） +- Good: `面向开发者：请先阅读开发指南`（"开发者"自然，且中文里冒号在此处更自然） -### Code block comments — never translate +### Code block comments — NEVER translate - Source code block contains: `# REPL agent demo (needs DEEPSEEK_API_KEY)` - Bad: `# REPL agent 演示（需要 DEEPSEEK_API_KEY）` -- Good: `# REPL agent demo (needs DEEPSEEK_API_KEY)` (byte-identical) +- Good: `# REPL agent demo (needs DEEPSEEK_API_KEY)` (keep exactly as-is, byte-for-byte) -### Language switcher — English to Chinese -- Source: `English | [中文](README.zh.md)` -- Bad: `English | [中文](README.zh.md)` -- Good: `[English](README.md) | 中文` - -### Language switcher — Chinese to English -- Source: `[English](README.md) | 中文` -- Bad: `[English](README.md) | 中文` -- Good: `English | [中文](README.zh.md)` +### Language switcher — flip direction +- Source file (English) has: `English | [中文](README.zh.md)` +- Bad (copying source unchanged): `English | [中文](README.zh.md)` +- Good (flipped for Chinese file): `[English](README.md) | 中文` --- diff --git a/package.json b/package.json index aaec1e62c4..51fd789a2b 100644 --- a/package.json +++ b/package.json @@ -100,7 +100,6 @@ "mermaid": "11.16.0", "micromark-extension-gfm": "^3.0.0", "publint": "^0.3.21", - "saxes": "^6.0.0", "tsdown": "^0.22.2", "tsx": "^4.22.4", "typescript": "^6.0.3", diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index b1a3a78030..0e71da2ea1 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -68,9 +68,6 @@ importers: publint: specifier: ^0.3.21 version: 0.3.21 - saxes: - specifier: ^6.0.0 - version: 6.0.0 tsdown: specifier: ^0.22.2 version: 0.22.2(oxc-resolver@11.20.0)(publint@0.3.21)(tsx@4.22.4)(typescript@6.0.3) @@ -819,7 +816,7 @@ importers: dependencies: '@earendil-works/pi-ai': specifier: ^0.79.1 - version: 0.79.3(ws@8.21.0)(zod@4.4.3) + version: 0.79.3(@modelcontextprotocol/sdk@1.29.0(zod@4.4.3))(ws@8.21.0)(zod@4.4.3) schemastery: specifier: ^3.18.0 version: 3.18.0 @@ -3059,6 +3056,10 @@ packages: cpu: [x64] os: [win32] + '@pkgjs/parseargs@0.11.0': + resolution: {integrity: sha512-+1VkjdD0QBLPodGrJUeqarH8VAIvQODIbwh9XpP5Syisf7YoQgsJKPNFoqqLQlu+VQ/tVSshMR6loPMn8U+dPg==} + engines: {node: '>=14'} + '@protobufjs/aspromise@1.1.2': resolution: {integrity: sha512-j+gKExEuLmKwvz3OgROXtrJ2UG2x8Ch2YZUxahh+s1F2HZ+wAceUNLkvy6zKCPVRkU++ZWQrdxsUeQXmcg4uoQ==} @@ -6137,11 +6138,11 @@ snapshots: '@csstools/css-tokenizer@4.0.0': {} - '@earendil-works/pi-ai@0.79.3(ws@8.21.0)(zod@4.4.3)': + '@earendil-works/pi-ai@0.79.3(@modelcontextprotocol/sdk@1.29.0(zod@4.4.3))(ws@8.21.0)(zod@4.4.3)': dependencies: '@anthropic-ai/sdk': 0.91.1(zod@4.4.3) '@aws-sdk/client-bedrock-runtime': 3.1048.0 - '@google/genai': 1.52.0 + '@google/genai': 1.52.0(@modelcontextprotocol/sdk@1.29.0(zod@4.4.3)) '@mistralai/mistralai': 2.2.1 '@smithy/node-http-handler': 4.7.3 http-proxy-agent: 7.0.2 @@ -6299,12 +6300,14 @@ snapshots: '@exodus/bytes@1.15.1': {} - '@google/genai@1.52.0': + '@google/genai@1.52.0(@modelcontextprotocol/sdk@1.29.0(zod@4.4.3))': dependencies: google-auth-library: 10.7.0 p-retry: 4.6.2 protobufjs: 7.6.4 ws: 8.21.0 + optionalDependencies: + '@modelcontextprotocol/sdk': 1.29.0(zod@4.4.3) transitivePeerDependencies: - bufferutil - supports-color @@ -6563,6 +6566,9 @@ snapshots: '@oxc-resolver/binding-win32-x64-msvc@11.20.0': optional: true + '@pkgjs/parseargs@0.11.0': + optional: true + '@protobufjs/aspromise@1.1.2': {} '@protobufjs/base64@1.1.2': {} @@ -8034,6 +8040,8 @@ snapshots: jackspeak@3.4.3: dependencies: '@isaacs/cliui': 8.0.2 + optionalDependencies: + '@pkgjs/parseargs': 0.11.0 jiti@2.7.0: {} diff --git a/scripts/translation-prompt.spec.ts b/scripts/translation-prompt.spec.ts index 93754a79a1..db3f8b31a0 100644 --- a/scripts/translation-prompt.spec.ts +++ b/scripts/translation-prompt.spec.ts @@ -1,5 +1,7 @@ -/** Regression tests for the executable translation prompt contract. */ +/** Unit tests for the prompt-v4 renderer and three-section response parser. */ +import { readFileSync } from 'node:fs' +import { join, resolve } from 'node:path' import { describe, expect, it } from 'vitest' import { parseTranslationResponse, @@ -7,70 +9,43 @@ import { renderTranslationResponse, } from './translation-prompt.ts' -const document = `# Wrapper - -## 模板正文 - -\`\`\`\`text -{{source_lang}} to {{target_lang}} -{{translation_rules}} -{{terminology}} -[English]({{source_filename}}) | [中文]({{source_filename_zh}}) -\`\`\`\` -` +const root = resolve(import.meta.dirname, '..') +const document = readFileSync(join(root, 'docs/i18n/translation-prompt.md'), 'utf8') +const terminology = '| English | 中文 |\n|---|---|\n| agent | agent |' describe('translation prompt rendering', () => { - it('renders every supported placeholder without recursively rewriting injected rules', () => { - const rendered = renderTranslationPrompt(document, { - sourceLanguage: 'English', - sourceFilename: 'guide.md', - translationRules: 'A literal {{source_lang}} in injected rules.', - terminology: '| English | 中文 |', - }) - expect(rendered).toContain('English to Chinese') - expect(rendered).toContain('A literal {{source_lang}} in injected rules.') - expect(rendered).toContain('[English](guide.md) | [中文](guide.zh.md)') + it('renders both directions with every placeholder resolved', () => { + const en = renderTranslationPrompt(document, { sourceLanguage: 'English', terminology }) + expect(en).toContain('from English to Chinese') + expect(en).toContain(terminology) + expect(en).not.toContain('{{') + const zh = renderTranslationPrompt(document, { sourceLanguage: 'Chinese', terminology }) + expect(zh).toContain('from Chinese to English') }) - it('rejects a filename whose suffix contradicts the source language', () => { - expect(() => renderTranslationPrompt(document, { - sourceLanguage: 'Chinese', - sourceFilename: 'guide.md', - translationRules: 'rules', - terminology: 'terms', - })).toThrow('does not match source language Chinese') - }) - - it('rejects malformed template placeholders before injecting rule contents', () => { - expect(() => renderTranslationPrompt(document.replace('{{source_lang}}', '{{source-lang}}'), { - sourceLanguage: 'English', - sourceFilename: 'guide.md', - translationRules: 'A literal {{source_lang}} in injected rules.', - terminology: '| English | 中文 |', - })).toThrow('template contains malformed placeholder syntax') + it('rejects a template with unknown or missing placeholders', () => { + const alien = document.replaceAll('{{terminology}}', '{{terms_prompt}}') + expect(() => renderTranslationPrompt(alien, { sourceLanguage: 'English', terminology })).toThrow(/unsupported placeholder/) + const missing = document.replaceAll('{{terminology}}', '') + expect(() => renderTranslationPrompt(missing, { sourceLanguage: 'English', terminology })).toThrow(/required placeholder/) }) }) -describe('translation response XML', () => { - it('round-trips Markdown and the CDATA terminator', () => { - const response = { - translation: '# Draft\n\nA ]]> marker.', - review: '- [Tone] Fixed.', - final: '# Final\n\nA ]]> marker.', - } +describe('translation response sections', () => { + it('round-trips Markdown bodies', () => { + const response = { translation: '# 标题\n\n正文 **加粗**。', review: '- [Tone] 修正一处。\n- 无修正', final: '# 标题\n\n定稿。' } expect(parseTranslationResponse(renderTranslationResponse(response))).toEqual(response) }) - it('rejects missing, reordered, nested, attributed, or non-CDATA children', () => { - expect(() => parseTranslationResponse('')).toThrow('translation, review, and final') - expect(() => parseTranslationResponse('')) - .toThrow('expected translation, got review') - expect(() => parseTranslationResponse(renderTranslationResponse({ translation: 'x', review: 'y', final: 'z' }) - .replace('', ''))) - .toThrow('nested element b is not allowed') - expect(() => parseTranslationResponse(renderTranslationResponse({ translation: 'x', review: 'y', final: 'z' }).replace('', ''))) - .toThrow('review must not have attributes') - expect(() => parseTranslationResponse(renderTranslationResponse({ translation: 'x', review: 'y', final: 'z' }).replace('', 'x'))) - .toThrow('all response field content must be inside CDATA') + it('tolerates a fenced xml wrapper around the whole response', () => { + const fenced = '```xml\n\nA\n\n\n\n- 无修正\n\n\n\nA\n\n```' + expect(parseTranslationResponse(fenced).final).toBe('A') + }) + + it('rejects missing, unterminated, or duplicated sections', () => { + expect(() => parseTranslationResponse('\nA\n')).toThrow(/missing /) + expect(() => parseTranslationResponse('\nA')).toThrow(/unterminated /) + const dup = '\nA\n\n\nR\n\n\nF\n\n\nG\n' + expect(() => parseTranslationResponse(dup)).toThrow(/duplicate /) }) }) diff --git a/scripts/translation-prompt.ts b/scripts/translation-prompt.ts index e30c962498..0556ff39fa 100644 --- a/scripts/translation-prompt.ts +++ b/scripts/translation-prompt.ts @@ -1,20 +1,16 @@ /** - * Executable renderer and strict response parser for the committed - * documentation-translation prompt contract. + * Executable renderer and response parser for the committed + * documentation-translation prompt contract (prompt-v4). + * + * The v4 contract: three placeholders (`source_lang`, `target_lang`, + * `terminology`), whole-document translation, and a three-section response + * (``, ``, `` in order, bare XML tags with raw + * Markdown bodies). The switcher filename is spelled out by the model from + * the document itself; the pipeline injects no other repository file. */ -import { basename } from 'node:path' -import { SaxesParser } from 'saxes' - /** Placeholder names supported by the committed translation prompt. */ -export const TRANSLATION_PROMPT_PLACEHOLDERS = [ - 'source_lang', - 'target_lang', - 'translation_rules', - 'terminology', - 'source_filename', - 'source_filename_zh', -] as const +export const TRANSLATION_PROMPT_PLACEHOLDERS = ['source_lang', 'target_lang', 'terminology'] as const type TranslationPromptPlaceholder = (typeof TRANSLATION_PROMPT_PLACEHOLDERS)[number] @@ -24,15 +20,11 @@ type TranslationLanguage = 'English' | 'Chinese' /** Inputs that vary for one rendered translation request. */ export interface TranslationPromptInput { sourceLanguage: TranslationLanguage - /** Source basename, including `.md` or `.zh.md`. */ - sourceFilename: string - /** Complete current `translation-rules.md` contents. */ - translationRules: string /** Complete current `terminology.md` contents. */ terminology: string } -/** Parsed contents of the three-element XML response. */ +/** Parsed contents of the three-section response. */ export interface TranslationResponse { translation: string review: string @@ -42,7 +34,7 @@ export interface TranslationResponse { const PLACEHOLDER = /{{([a-z_]+)}}/g const TEMPLATE_OPEN = '## 模板正文\n\n````text\n' const TEMPLATE_CLOSE = '\n````' -const RESPONSE_CHILDREN = ['translation', 'review', 'final'] as const +const RESPONSE_SECTIONS = ['translation', 'review', 'final'] as const /** Extract the machine-consumed text fence from `translation-prompt.md`. */ function extractTranslationPrompt(document: string): string { @@ -61,31 +53,15 @@ export function documentedTranslationPromptPlaceholders(document: string): strin return [...document.slice(0, preambleEnd).matchAll(/^\| `{{([a-z_]+)}}` \|/gm)].map(match => match[1] ?? '') } -/** Render one system prompt from the checked-in template and canonical rules. */ +/** Render one system prompt from the checked-in template. */ export function renderTranslationPrompt(document: string, input: TranslationPromptInput): string { - if (basename(input.sourceFilename) !== input.sourceFilename) { - throw new Error(`translation prompt: sourceFilename must be a basename; got ${JSON.stringify(input.sourceFilename)}`) - } - const sourceIsChinese = input.sourceFilename.endsWith('.zh.md') - if (input.sourceLanguage === 'Chinese' ? !sourceIsChinese : sourceIsChinese || !input.sourceFilename.endsWith('.md')) { - throw new Error(`translation prompt: ${input.sourceFilename} does not match source language ${input.sourceLanguage}`) - } - const targetLanguage: TranslationLanguage = input.sourceLanguage === 'English' ? 'Chinese' : 'English' - const sourceFilenameZh = sourceIsChinese ? input.sourceFilename : input.sourceFilename.replace(/\.md$/, '.zh.md') const values: Record = { source_lang: input.sourceLanguage, target_lang: targetLanguage, - translation_rules: input.translationRules, terminology: input.terminology, - source_filename: input.sourceFilename, - source_filename_zh: sourceFilenameZh, } const template = extractTranslationPrompt(document) - const placeholderFreeTemplate = template.replace(PLACEHOLDER, '') - if (placeholderFreeTemplate.includes('{{') || placeholderFreeTemplate.includes('}}')) { - throw new Error('translation prompt: template contains malformed placeholder syntax') - } const names = [...template.matchAll(PLACEHOLDER)].map(match => match[1] ?? '') const unknown = names.filter(name => !TRANSLATION_PROMPT_PLACEHOLDERS.includes(name as TranslationPromptPlaceholder)) if (unknown.length > 0) throw new Error(`translation prompt: unsupported placeholder(s): ${[...new Set(unknown)].join(', ')}`) @@ -95,77 +71,37 @@ export function renderTranslationPrompt(document: string, input: TranslationProm return template.replace(PLACEHOLDER, (_token, name: string) => values[name as TranslationPromptPlaceholder]) } -/** Escape one value so it remains byte-identical inside an XML CDATA field. */ -function escapeTranslationCdata(value: string): string { - return value.replaceAll(']]>', ']]]]>') -} - -/** Serialize a response using the exact XML wire contract in the prompt. */ +/** Serialize a response in the exact three-section shape the prompt requests. */ export function renderTranslationResponse(response: TranslationResponse): string { - return [ - '', - ``, - ``, - ``, - '', - ].join('\n') + return RESPONSE_SECTIONS.map(section => `<${section}>\n${response[section]}\n`).join('\n\n') } -/** Parse and validate the exact XML response shape emitted by the model. */ -export function parseTranslationResponse(xml: string): TranslationResponse { - const values: TranslationResponse = { translation: '', review: '', final: '' } - const stack: string[] = [] - const cdataFields = new Set() - let rootSeen = false - let childIndex = 0 - const fail = (message: string): never => { - throw new Error(`translation response: ${message}`) - } - const parser = new SaxesParser({ xmlns: false }) +/** + * Parse the three-section response. Sections must each appear exactly once + * and in order; bodies are raw Markdown taken verbatim between the tags. + * A fenced ```xml wrapper around the whole response is tolerated, matching + * the shape some models echo back from the prompt's own example. + */ +export function parseTranslationResponse(text: string): TranslationResponse { + let body = text.trim() + const fenced = /^```(?:xml)?\n([\s\S]*?)\n```$/.exec(body) + if (fenced?.[1] !== undefined) body = fenced[1].trim() - parser.on('opentag', (tag) => { - if (stack.length === 0) { - if (rootSeen) fail('contains more than one root element') - if (tag.name !== 'dsh-translation-response') fail(`expected dsh-translation-response root, got ${tag.name}`) - const attributes = Object.keys(tag.attributes) - if (attributes.length !== 1 || tag.attributes.version !== '1') fail('root must have only version="1"') - rootSeen = true - } else if (stack.length === 1) { - const expected = RESPONSE_CHILDREN[childIndex] - if (tag.name !== expected) fail(`expected ${expected ?? 'no more children'}, got ${tag.name}`) - if (Object.keys(tag.attributes).length !== 0) fail(`${tag.name} must not have attributes`) - childIndex++ - } else { - fail(`nested element ${tag.name} is not allowed`) - } - stack.push(tag.name) - }) - parser.on('text', (value) => { - if (stack.length <= 1 && value.trim() === '') return - fail('all response field content must be inside CDATA') - }) - parser.on('cdata', (value) => { - const field = stack.at(-1) - if (field === undefined || !RESPONSE_CHILDREN.includes(field as (typeof RESPONSE_CHILDREN)[number])) { - fail('CDATA is allowed only inside translation, review, or final') - } - const key = field as (typeof RESPONSE_CHILDREN)[number] - values[key] += value - cdataFields.add(key) - }) - parser.on('closetag', (tag) => { - const expected = stack.pop() - if (expected !== tag.name) fail(`closing ${tag.name} does not match ${expected ?? 'nothing'}`) - }) - parser.on('comment', () => fail('comments are not allowed')) - parser.on('doctype', () => fail('doctypes are not allowed')) - parser.on('processinginstruction', () => fail('processing instructions are not allowed')) - parser.on('error', error => fail(`invalid XML: ${error.message}`)) - parser.write(xml).close() - - if (childIndex !== RESPONSE_CHILDREN.length) fail('translation, review, and final must each appear exactly once and in order') - for (const field of RESPONSE_CHILDREN) { - if (!cdataFields.has(field)) fail(`${field} must contain a CDATA section`) + const values: Partial> = {} + let cursor = 0 + for (const section of RESPONSE_SECTIONS) { + const open = `<${section}>` + const close = `` + const start = body.indexOf(open, cursor) + if (start === -1) throw new Error(`translation response: missing <${section}> section`) + const end = body.indexOf(close, start + open.length) + if (end === -1) throw new Error(`translation response: unterminated <${section}> section`) + values[section] = body.slice(start + open.length, end).replace(/^\n/, '').replace(/\n$/, '') + cursor = end + close.length } - return values + for (const section of RESPONSE_SECTIONS) { + const again = body.indexOf(`<${section}>`, cursor) + if (again !== -1) throw new Error(`translation response: duplicate <${section}> section`) + } + return values as TranslationResponse } diff --git a/scripts/verify-translation-prompt.ts b/scripts/verify-translation-prompt.ts index 66d83e47ad..df72ac73ce 100644 --- a/scripts/verify-translation-prompt.ts +++ b/scripts/verify-translation-prompt.ts @@ -18,37 +18,27 @@ function read(path: string): string { try { const document = read('docs/i18n/translation-prompt.md') - const translationRules = read('docs/i18n/translation-rules.md') const terminology = read('docs/i18n/terminology.md') const documented = documentedTranslationPromptPlaceholders(document) if (documented.join('\n') !== TRANSLATION_PROMPT_PLACEHOLDERS.join('\n')) { throw new Error(`placeholder table must list exactly: ${TRANSLATION_PROMPT_PLACEHOLDERS.join(', ')}`) } - const englishSource = renderTranslationPrompt(document, { - sourceLanguage: 'English', - sourceFilename: 'example.md', - translationRules, - terminology, - }) - const chineseSource = renderTranslationPrompt(document, { - sourceLanguage: 'Chinese', - sourceFilename: 'example.zh.md', - translationRules, - terminology, - }) - if (!englishSource.includes('[English](example.md) | 中文')) throw new Error('English-source render does not carry the Chinese switcher instruction') - if (!chineseSource.includes('English | [中文](example.zh.md)')) throw new Error('Chinese-source render does not carry the English switcher instruction') + const englishSource = renderTranslationPrompt(document, { sourceLanguage: 'English', terminology }) + const chineseSource = renderTranslationPrompt(document, { sourceLanguage: 'Chinese', terminology }) + if (englishSource.includes('{{') || chineseSource.includes('{{')) throw new Error('rendered prompt contains an unresolved placeholder') + if (!englishSource.includes('from English to Chinese')) throw new Error('English-source render does not translate into Chinese') + if (!chineseSource.includes('from Chinese to English')) throw new Error('Chinese-source render does not translate into English') const example = /```xml\n([\s\S]*?)\n```/.exec(englishSource)?.[1] - if (example === undefined) throw new Error('rendered prompt has no XML response example') + if (example === undefined) throw new Error('rendered prompt has no three-section response example') parseTranslationResponse(example) - const roundTrip = { translation: 'first ]]> pass', review: '- [None] No corrections.', final: 'final ]]> text' } + const roundTrip = { translation: 'first pass\n\nwith **markdown**', review: '- 无修正', final: 'final text' } const parsed = parseTranslationResponse(renderTranslationResponse(roundTrip)) - if (JSON.stringify(parsed) !== JSON.stringify(roundTrip)) throw new Error('CDATA split rule does not round-trip response content') + if (JSON.stringify(parsed) !== JSON.stringify(roundTrip)) throw new Error('three-section response does not round-trip') - console.log('verify-translation-prompt: both directions render and the XML response contract parses.') + console.log('verify-translation-prompt: both directions render and the three-section response contract parses.') } catch (error) { const message = error instanceof Error ? error.message : String(error) console.error(`verify-translation-prompt: ${message}`)