mirror of
https://github.com/deepseek-ai/deepseek-harness.git
synced 2026-08-29 04:26:38 +00:00
Merge pull request #2956 from deepseek-harness/worktree/win32-utf16-nul-truncation-20260823
fix(directory-picker-native): stop truncating Win32 UTF-16 paths at U+XX00
This commit is contained in:
Tianyi Cui
@@ -0,0 +1,6 @@
|
||||
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
|
||||
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
||||
# after editing either side, bring the other along and re-record with:
|
||||
# pnpm run verify-translation-pairing --write .agents/notes/implemented/bug-fix/2026-08-23-win32-utf16-nul-truncation.md
|
||||
2026-08-23-win32-utf16-nul-truncation.md: 3962730c66d72b9927e5ce6dabde0f50101c5787
|
||||
2026-08-23-win32-utf16-nul-truncation.zh.md: 23df80053b23c5e2fbb810c95668bbeef7c91fe9
|
||||
@@ -0,0 +1,29 @@
|
||||
# Agent Note: Win32 folder-picker paths stop truncating at U+XX00 code units
|
||||
|
||||
Status: implemented
|
||||
|
||||
English | [中文](2026-08-23-win32-utf16-nul-truncation.zh.md)
|
||||
|
||||
## Problem
|
||||
|
||||
`readUtf16` in `packages/host/directory-picker-native/src/win32-dialog-bindings.ts` translated the `IFileOpenDialog` result buffer by scanning for a zero byte with `bytes[end] !== 0`. UTF-16LE encodes NUL as two zero bytes, so any BMP code unit whose low byte is zero — U+XX00, such as 开 (U+5F00) — ended the scan early. Selecting a folder like `C:\Users\XIAOPAN\Desktop\安卓开发` returned `C:\Users\XIAOPAN\Desktop\安卓`, and the workspace-creation call failed with `workspace-invalid-path ... ENOENT`.
|
||||
|
||||
## Decision
|
||||
|
||||
The scan ends only when both bytes of a code unit are zero, still advancing two bytes at a time over the same 32KiB `koffi.view` buffer. A regression test drives `readUtf16` through the existing fake koffi COM world with a path containing 安卓开发 (U+5F00), so the termination rule is proven without a real Windows host.
|
||||
|
||||
The fix is adopted verbatim from the community patch series on the `fix/win32-utf16-nul-truncation` branch of the ericcaiwx-star fork — [c8aac14703](https://github.com/ericcaiwx-star/deepseek-harness/commit/c8aac14703a517b8db1573f9ca4ed94dc58e276b) for the scan fix and [e1d6265cb9](https://github.com/ericcaiwx-star/deepseek-harness/commit/e1d6265cb930a0a74cba03c40e73ed872a83575f) for the fixture cleanup — reported in [discussion #580](https://github.com/deepseek-ai/deepseek-harness/discussions/580) (earlier reported in [discussion #563](https://github.com/deepseek-ai/deepseek-harness/discussions/563)). Both cherry-picks retain the original author, ericcaiwx-star; the upstream fork is the source of record for the patch.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
**Reject the community patch and rewrite the scan locally.** Rejected: the patch is minimal, fits the dialog's existing test approach, and a byte-identical cherry-pick preserves provenance and credit.
|
||||
|
||||
**Decode the whole buffer with `toString('utf16le')` and split at `\0`.** Rejected: it copies the entire buffer instead of scanning, and the split would still depend on the same two-zero-byte rule.
|
||||
|
||||
**Ask COM or koffi for a string length.** Rejected: the binding surface provides no length; the double-zero scan is the standard UTF-16LE NUL test.
|
||||
|
||||
## Consequences
|
||||
|
||||
- Any path containing a U+XX00 code unit survives the picker translation; paths with such characters (for example Chinese folder names) can be selected and used to create workspaces.
|
||||
- The fix changes no ABI usage, buffer size, or dialog flow; the COM child-process architecture in the [Win32 folder dialog note](../feature/2026-08-02-win32-in-process-folder-dialog.md) is untouched.
|
||||
- Real-dialog rendering and selection remain a manual Windows check; this change's regression test exercises only the byte-to-string translation against the fake COM world. The fixture path is synthetic (`C:\fixture\安卓开发`) so no real user path appears in the repository.
|
||||
@@ -0,0 +1,29 @@
|
||||
# Agent Note: Win32 目录选择器路径不再在 U+XX00 码元处截断
|
||||
|
||||
Status: implemented
|
||||
|
||||
[English](2026-08-23-win32-utf16-nul-truncation.md) | 中文
|
||||
|
||||
## 问题
|
||||
|
||||
`packages/host/directory-picker-native/src/win32-dialog-bindings.ts` 的 `readUtf16` 用 `bytes[end] !== 0` 扫描 `IFileOpenDialog` 结果缓冲区来寻找零字节。UTF-16LE 真正的 NUL 是两个零字节，因此任何低字节为 0 的 BMP 码元——U+XX00，例如「开」(U+5F00)——都会提前结束扫描。选择 `C:\Users\XIAOPAN\Desktop\安卓开发` 这类目录会得到 `C:\Users\XIAOPAN\Desktop\安卓`，随后创建工作区的调用以 `workspace-invalid-path ... ENOENT` 失败。
|
||||
|
||||
## 决策
|
||||
|
||||
扫描只有在一个码元的两个字节都为零时才结束，仍按每次两个字节在同一个 32KiB `koffi.view` 缓冲区上推进。回归测试通过既有的假 koffi COM 世界驱动 `readUtf16`，路径包含「安卓开发」(U+5F00)，从而不依赖真实 Windows 主机验证终止规则。
|
||||
|
||||
修复逐字采用 ericcaiwx-star fork 的 `fix/win32-utf16-nul-truncation` 分支上的社区补丁系列——[c8aac14703](https://github.com/ericcaiwx-star/deepseek-harness/commit/c8aac14703a517b8db1573f9ca4ed94dc58e276b) 是扫描修复，[e1d6265cb9](https://github.com/ericcaiwx-star/deepseek-harness/commit/e1d6265cb930a0a74cba03c40e73ed872a83575f) 是 fixture 清理——在 [discussion #580](https://github.com/deepseek-ai/deepseek-harness/discussions/580) 报告（更早在 [discussion #563](https://github.com/deepseek-ai/deepseek-harness/discussions/563) 报告）。两次 cherry-pick 均保留原作者 ericcaiwx-star；上游 fork 是补丁的记录来源。
|
||||
|
||||
## 考虑过的替代方案
|
||||
|
||||
**拒绝社区补丁，本地重写扫描。** 拒绝：补丁极小，与目录选择器现有测试方式一致；逐字节一致的 cherry-pick 保留来源与署名。
|
||||
|
||||
**用 `toString('utf16le')` 解码整个缓冲区再按 `\0` 切分。** 拒绝：复制整个缓冲区而非扫描，且切分仍依赖同一「双零字节」规则。
|
||||
|
||||
**向 COM 或 koffi 索取字符串长度。** 拒绝：绑定面不提供长度；双零扫描是标准的 UTF-16LE NUL 判定。
|
||||
|
||||
## 后果
|
||||
|
||||
- 任何含 U+XX00 码元的路径组件都能通过选择器转译；含这类字符的路径（例如中文目录名）可以选中并用于创建工作区。
|
||||
- 修复不改变 ABI 用法、缓冲区大小或对话框流程；[Win32 目录选择器 note](../feature/2026-08-02-win32-in-process-folder-dialog.zh.md) 中的 COM 子进程架构不受影响。
|
||||
- 真实对话框渲染与选择仍是手动 Windows 检查；本次回归测试只针对假 COM 世界中的字节到字符串转译。fixture 路径为合成路径（`C:\fixture\安卓开发`），仓库中不出现真实用户路径。
|
||||
@@ -37,7 +37,9 @@ interface Koffi {
|
||||
function readUtf16(koffi: Koffi, address: unknown): string {
|
||||
const bytes = Buffer.from(koffi.view(address, 32768))
|
||||
let end = 0
|
||||
while (end + 1 < bytes.length && bytes[end] !== 0) end += 2
|
||||
// UTF-16LE NUL is two zero bytes. A single zero low byte is a valid BMP
|
||||
// code unit (U+XX00, e.g. 开 = U+5F00) and must not terminate the scan.
|
||||
while (end + 1 < bytes.length && !(bytes[end] === 0 && bytes[end + 1] === 0)) end += 2
|
||||
return bytes.toString('utf16le', 0, end)
|
||||
}
|
||||
|
||||
|
||||
@@ -180,6 +180,15 @@ describe('loadWin32DialogBindings over the fake COM world', () => {
|
||||
expect(world.uninitialized).toBe(1)
|
||||
})
|
||||
|
||||
it('reads a UTF-16 path whose BMP code unit has a zero low byte (U+5F00 开)', async () => {
|
||||
// 开 = U+5F00 → UTF-16LE bytes 00 5F. A scan that treats any zero low
|
||||
// byte as NUL truncates here and returns the nonexistent ...\安卓.
|
||||
const world = comWorld({ path: 'C:\\fixture\\安卓开发' })
|
||||
installFakeKoffi(world)
|
||||
const bindings = await (await loadBindingsModule()).loadWin32DialogBindings()
|
||||
expect(runFolderDialog(bindings, 'Pick', vi.fn())).toBe('C:\\fixture\\安卓开发')
|
||||
})
|
||||
|
||||
it('maps dismissal and the S_FALSE CoInitializeEx', async () => {
|
||||
const world = comWorld({ showHr: HRESULT_CANCELLED, coInitHr: 1 })
|
||||
installFakeKoffi(world)
|
||||
|
||||
Reference in New Issue
Block a user