feat(skills): record browser GIFs with Playwright video

This commit is contained in:
yudshj
2026-09-08 11:52:37 +08:00
committed by Yudong Han
parent 7263193655
commit e57e7dc57f
9 changed files with 326 additions and 70 deletions
@@ -2,5 +2,5 @@
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/process/2026-08-08-browser-gif-evidence-chain.md
2026-08-08-browser-gif-evidence-chain.md: 8a7c28792bbab178cac6ba6743628882a15fdf6e
2026-08-08-browser-gif-evidence-chain.zh.md: ff9b6cc729c23dd9f3ba170752fa05fe65c7fb53
2026-08-08-browser-gif-evidence-chain.md: 66d4679de6fa462a5287b262cc6df3ed5f75084c
2026-08-08-browser-gif-evidence-chain.zh.md: 1d87768fcfd8460cdf1b7958ee562137f2b8e1ea
@@ -16,7 +16,7 @@ The [`record-browser-gif`](../../../skills/record-browser-gif/SKILL.md) workflow
Browser automation waits for unique, exact semantic states. When the claim concerns a tool call, rejection, or recovery, the storyboard includes a detail or trajectory frame that identifies the tool, shows its status or stable error code, and shows the downstream result. The final encoded GIF remains the verification subject; when a viewer cannot animate it, representative frames are decoded from that GIF instead of treating source screenshots as equivalent evidence.
The available browser-control workflow remains preferred. When it is unavailable, the recorder uses the repository-declared Playwright dependency in an isolated headless browser rather than installing another driver or opening the user's browser. A native production surface may be replaced only through normal application configuration with an official browser-operable production backend, and that override is stated next to the GIF. Fixtures, mock transports, synthetic events, and test-only hooks do not substantiate a real-production claim.
The [Playwright video decision](2026-09-08-playwright-video-gif.md) owns recording-backend selection and timing. The recorder uses the repository-declared browser dependency without installing another driver or opening the user's browser. A native production surface may be replaced only through normal application configuration with an official browser-operable production backend, and that override is stated next to the GIF. Fixtures, mock transports, synthetic events, and test-only hooks do not substantiate a real-production claim.
Publication verifies the boundary again. The preferred path attaches with `gh --attach` (v2.99.0 or later; github.com only; at most 10 MB): the body file references the local GIF path, the command uploads the verified artifact and rewrites that reference in place, and the live body must then show the rewritten uploaded URL, which must respond with `200` and `image/gif`. When attach cannot apply — the GIF exceeds 10 MB, `gh` is older, or the repository is GitHub Enterprise Server — the assets-branch path remains: the branch contains media only, the staged and published bytes match the verified artifact, and a private-repository asset is checked through authenticated API or raw requests for its path, byte size, checksum, response status, and media type, proving only the repository-member review path (the [documentation-site image decision](../../archived/process/2026-08-06-doc-site-carries-its-images.md) owns why a public site cannot depend on a private raw URL). Immediately before the pull-request body changes, the live head must still equal the recorded head. After the edit, the live head is checked again and must remain at that recorded value; GitHub's Markdown renderer separately must produce the expected image.
@@ -16,7 +16,7 @@ Status: implemented
浏览器自动化会等待唯一且精确的语义状态。如果需要证明工具调用、拒绝或恢复,分镜就必须包含详情帧或轨迹帧:标明工具、显示其状态或稳定错误码,并展示后续结果。最终编码出的 GIF 始终是验证对象;如果查看器无法播放动画,应从该 GIF 中解码出代表性帧,而不能将源截图视为等效证据。
仍应优先使用已有的浏览器控制工作流。如果该工作流不可用,录制程序应在隔离的无头浏览器中使用仓库已声明的 Playwright 依赖,而不是安装其他驱动打开用户的浏览器。只有通过正常应用配置选用官方且可由浏览器操作的生产后端,才能替换原生生产界面,并且必须在 GIF 旁注明这一覆盖。fixture(测试前置数据)、mock 传输层、合成事件和测试专用钩子均不能支撑真实生产实现的主张。
[Playwright 视频决策](2026-09-08-playwright-video-gif.zh.md)规定录制后端的选择和时序。录制程序使用仓库已声明的浏览器依赖,不安装其他驱动,也不打开用户的浏览器。只有通过正常应用配置选用官方且可由浏览器操作的生产后端,才能替换原生生产界面,并且必须在 GIF 旁注明这一覆盖。fixture(测试前置数据)、mock 传输层、合成事件和测试专用钩子均不能支撑真实生产实现的主张。
发布环节会再次验证边界。首选路径使用 `gh --attach` 上传(v2.99.0 或更高版本;仅限 github.com;不超过 10 MB):正文文件引用本地 GIF 路径,命令上传已验证的产物并把该引用就地改写,之后在线正文必须显示改写后的上传 URL,且该 URL 必须返回 `200``image/gif`。当 attach 无法使用时——GIF 超过 10 MB、`gh` 版本过旧或仓库位于 GitHub Enterprise Server——仍保留资产分支路径:该分支只包含媒体文件,暂存和发布的字节必须与已验证产物一致,私有仓库中的资产须通过经身份验证的 API 或原始内容请求检查其路径、字节大小、校验和、响应状态和媒体类型,这只能证明仓库成员的评审访问路径([文档站点图片决策](../../archived/process/2026-08-06-doc-site-carries-its-images.md)解释了公共站点为何不能依赖私有的原始内容 URL)。修改 PR 正文之前,必须再次确认在线 head 仍与录制时的 head 相同。编辑后还要再次检查在线 head,且它必须保持为该记录值;GitHub 的 Markdown 渲染器则须单独生成预期图片。
@@ -0,0 +1,6 @@
# Bilingual-pair consistency record (docs/i18n/README.md): the git blob hash of each
# side as of the last confirmed-consistent state. Both languages carry equal authority;
# after editing either side, bring the other along and re-record with:
# pnpm run verify-translation-pairing --write .agents/notes/implemented/process/2026-09-08-playwright-video-gif.md
2026-09-08-playwright-video-gif.md: d3512ebdbfa781f178ee2e891db2bd5e79a5d7e5
2026-09-08-playwright-video-gif.zh.md: 17c082fcfb36dd8d1e099a867b0ef185c770fde2
@@ -0,0 +1,29 @@
# Agent Note: Playwright video captures continuous browser demos
Status: implemented
English | [中文](2026-09-08-playwright-video-gif.zh.md)
## Problem
A screenshot storyboard omits motion between verified UI states and can miss short-lived progress indicators. Browser tooling also varies between agent environments. GIF production needs a repeatable recording path that preserves interactions and keeps text readable without adding a browser driver dependency.
## Decision
The [recording skill](../../../skills/record-browser-gif/SKILL.md) prefers the repository-declared Playwright dependency with `recordVideo` in an isolated context. Viewport and video dimensions match explicitly, avoiding Playwright's default scaling to fit 800×800. The recorder retains the page video, awaits context closure, and saves the completed WebM before encoding. Screenshot storyboards remain available when video recording is unavailable or explicitly unwanted.
One encoder accepts either a video file or a screenshot directory. Video input selects one continuous interval, applies a declared playback multiplier, and extends its final frame. The JSON summary records source duration, selected interval, speed, final hold, and encoded dimensions, duration, frame count, and size. Mode-inappropriate options and invalid intervals fail. The original video remains available for review; trimming and speed never establish model response latency.
This choice replaces only the browser-control-first preference in the [evidence-chain decision](2026-08-08-browser-gif-evidence-chain.md). That note continues to own isolated application state, real model execution, exact commit attribution, and verified publication. Failed recordings cannot contribute frames to a successful run.
## Alternatives considered
**Keep screenshots as the default.** Explicit state holds make small, legible GIFs, but omit scrolling, animation, and interactions between states. They remain useful as a fallback and for a requested storyboard.
**Install a separate recorder or capture the desktop.** The repository already declares Playwright. Another driver adds setup and version management; desktop capture can include unrelated windows and personal state.
## Consequences
Continuous recording preserves intermediate states, so reviewers must inspect the selected interval for sensitive content and readability. Raw video consumes additional scratch storage, and encoding may require trimming or scaling to meet the byte limit. Context closure is part of successful recording, not optional cleanup.
The encoder's local Python unittest suite invokes real ffmpeg and ffprobe to check timing, palette order, screenshot holds, rejected options, overwrite protection, and size limits. It requires the skill's media prerequisites and is run explicitly; repository CI does not provision these media binaries. Product demonstrations additionally exercise the pull request's built server and real model flow.
@@ -0,0 +1,29 @@
# Agent Note: Playwright 视频录制连续浏览器演示
Status: implemented
[English](2026-09-08-playwright-video-gif.md) | 中文
## 问题
截图分镜省略了已验证 UI 状态之间的运动,也可能遗漏短暂的进度指示。不同代理环境提供的浏览器工具也不一致。GIF 制作需要可重复的录制路径,保留交互、保证文字可读,同时不增加浏览器驱动依赖。
## 决策
[录制 skill](../../../skills/record-browser-gif/SKILL.md) 优先使用仓库已声明的 Playwright 依赖,在隔离上下文中启用 `recordVideo`。显式设置相同的 viewport 与视频尺寸,避免 Playwright 默认缩放到 800×800 以内。录制程序保留页面视频对象,等待上下文关闭,再保存完整 WebM 并编码。视频录制不可用或用户明确不需要时,仍可使用截图分镜。
同一编码器接受视频文件或截图目录。视频输入选择一个连续区间,应用明确标注的播放倍率,并延长末帧。JSON 摘要记录源视频时长、所选区间、倍率、末帧停留时间,以及编码后的尺寸、时长、帧数和大小。与输入模式不符的选项及无效区间会失败。保留原视频供检查;裁剪与变速不能证明模型响应延迟。
本决策仅替代[证据链决策](2026-08-08-browser-gif-evidence-chain.zh.md)中优先使用浏览器控制工作流的偏好。该记录继续规定应用状态隔离、真实模型执行、准确提交归属及发布验证。失败录制的画面不得用于成功运行。
## 曾考虑的替代方案
**默认使用截图。**明确的状态停留可以生成小巧、易读的 GIF,但会省略状态之间的滚动、动画和交互。截图仍适合作为备用方案,或用于用户要求的分镜。
**安装独立录制器或录制桌面。**仓库已经声明 Playwright。另一套驱动增加安装与版本管理成本;桌面录制可能包含无关窗口和个人状态。
## 后果
连续录制保留中间状态,因此检查者必须查看所选区间是否包含敏感内容,以及画面是否易读。原始视频占用额外临时空间,编码时可能需要裁剪或缩放以满足字节上限。关闭上下文是成功录制的一部分,不是可选清理。
编码器的本地 Python unittest 测试调用真实 ffmpeg 和 ffprobe,检查时序、调色板顺序、截图停留、选项拒绝、覆盖保护和大小限制。测试依赖 skill 要求的媒体工具,需显式运行;仓库 CI 不预装这些媒体程序。产品演示还会执行 PR 构建后的服务器与真实模型流程。
+63 -32
View File
@@ -1,11 +1,11 @@
---
name: record-browser-gif
description: Record browser or Web UI interaction demos as optimized GIFs using the available built-in browser, state-based frame capture, and deterministic encoding, then attach the GIF to a pull request with `gh --attach`, falling back to a dedicated assets branch where attach cannot apply. Use when asked to make, record, or generate a GIF that demonstrates a browser workflow, and for every pull request that changes product-user-visible GUI behavior, which MUST include a GIF recorded from the pull request's real server and model flow.
description: Record browser or Web UI interaction demos as optimized GIFs using Playwright Videos and deterministic encoding, with screenshot capture when video is unavailable, then attach the GIF to a pull request with `gh --attach`, falling back to a dedicated assets branch where attach cannot apply. Use when asked to make, record, or generate a GIF that demonstrates a browser workflow, and for every pull request that changes product-user-visible GUI behavior, which MUST include a GIF recorded from the pull request's real server and model flow.
---
# Record Browser GIF
Produce a short, truthful UI demonstration as a local GIF, and — only when the task includes attaching it to a pull request — publish it through the attach workflow at the end of this skill. Use the browser-control skill for interaction and the bundled encoder for repeatable timing, dimensions, and size.
Produce a short, truthful UI demonstration as a local GIF, and — only when the task includes attaching it to a pull request — publish it through the attach workflow at the end of this skill. Prefer the repository-declared Playwright dependency with [Videos](https://playwright.dev/docs/videos); use the bundled encoder for trimming, playback speed, final hold, dimensions, and size.
The [evidence-chain decision](../../notes/implemented/process/2026-08-08-browser-gif-evidence-chain.md) owns why one storyboard comes from one isolated run and why publication revalidates both the artifact and the demonstrated pull-request head.
@@ -17,7 +17,7 @@ The recording itself is part of the evidence: use a real server booted from that
## Keep recording separate from publication
- Recording produces frame images and one local `.gif` artifact only; it never mutates remote state.
- Recording produces local video or screenshots and one `.gif` artifact only; it never mutates remote state.
- Publication — attaching the GIF to a pull request body with `gh --attach`, or pushing it to an assets branch and embedding its URL where attach cannot apply — is the separate final step, performed only when the task includes attaching the GIF to a pull request. It never touches the pull request's own branch.
- Preserve the requested recording conditions. A real-server or real-API demo must not use fixture queries, mock transports, synthetic event injection, or test-only hooks. If credentials or the server are unavailable, report that limitation instead of substituting a fixture.
- Never read or expose credential values. Use the application's normal configuration path and a benign demonstration prompt.
@@ -31,51 +31,80 @@ A GIF for a specific pull request demonstrates that pull request's tree, so stag
3. Treat one storyboard as one evidence run: every published frame comes from that server and those state roots, workspace, session, and model-backed scenario run. If capture automation fails, discard its frames and rerun from fresh roots; never splice frames from separate runs.
4. When switching between pull requests, stop the old server by PID or an exact match on its command line. A broad `pkill -f` pattern can match and kill the shell that launched it — including your own.
## Record the flow
## Record with Playwright Videos
1. Invoke the available browser-control skill and follow its setup, interaction, and cleanup instructions. Use the user's existing Chrome state only when requested or required; state that exception in the provenance and do not claim fresh client state. If browser control is unavailable, use the repository-declared Playwright dependency in an isolated headless browser; do not install another driver or launch the user's browser. State that fallback in the provenance.
2. Before recording, identify the exact origin, whether the app is built or in development, the transport, and any fixture or mock mode. Record only claims that the observed setup supports.
3. When a production default opens a native operating-system surface that headless automation cannot drive, select an official browser-operable production backend through the application's normal configuration. State the override in the provenance; a fixture, mock transport, or test-only hook is not an acceptable substitute.
4. Choose three to six states that tell one story, such as typed, running, settled, and detail. Prefer semantic state changes over continuous capture; omit loading churn that does not help the viewer.
5. Keep one viewport and crop for every frame, and name frames lexically: `00-initial.png`, `01-typed.png`, and so on.
6. Store frames under the repository's gitignored `.playwright-mcp/` directory — browser-tool screenshots can only be written under the tool's allowed roots, and relative filenames resolve against the repository root. Create the frame subdirectory first (`mkdir -p .playwright-mcp/gif-frames-<label>`); writing into a missing directory fails with ENOENT at capture time.
7. Before each screenshot, wait for a concrete UI condition such as a unique label, enabled control, changed document title, or completed response. Require the locator to resolve exactly one element; for Playwright accessible-name locators, use `exact: true` when equality is intended because descendant text or a prompt echo can otherwise create a false match. Do not use a fixed delay as proof that the application reached the state.
8. Make completion predicates match an exact-text element — for example, an element whose trimmed text equals the expected reply — never a substring check such as `body.textContent.includes(...)`, which the echo of the user's own prompt also satisfies.
9. When the claim involves a tool call, rejection, or recovery, include a detail or trajectory frame that shows the tool identity, status or stable error code, and the downstream result. A chat-only outcome does not prove why the tool path behaved that way.
10. Capture a transient state (spinner, running row) by driving a slow foreground operation — for example, a `sleep 15` bash command — and polling a concrete DOM marker (a `data-*` attribute) inside one browser-script call that also takes the screenshot. State polled across separate tool calls is lost, because the turn settles between calls.
11. Engineer the prompt so the state you need actually occurs: instruct the model to wait in the foreground when it would otherwise background a slow command, and give it a settle sentinel such as "reply with the single word done" to anchor the completion predicate.
12. Capture no secrets, personal data, unrelated tabs, or transient notifications. Stop any unnecessarily long real-API run after the demonstrated state is visible.
Use the repository-declared Playwright dependency in an isolated Chromium context. In this repository it resolves from `apps/web/package.json`; do not install another driver or open the user's browser. If the available browser-control workflow exposes `recordVideo`, it can own the context instead. When video is unavailable, use [screenshot capture](#screenshot-capture) and state the limitation. Existing user browser state remains an explicit provenance exception.
Use the browser's own screenshot API. When it returns image bytes, save those bytes directly; the encoder detects image content independently of the filename extension.
Before recording, identify the origin, built or development server, transport, and any mode overrides. When a production default opens a native surface that automation cannot drive, select an official browser-operable production backend through normal application configuration and disclose the override.
Store the script, raw video, timing notes, QA frames, and GIF under the repository's gitignored `.playwright-mcp/` directory. Create the run directory first. Match `viewport` and `recordVideo.size` explicitly: Playwright otherwise scales the video down to fit 800×800, which can make UI text unreadable.
```js
const { chromium } = createRequire(join(repo, 'apps/web/package.json'))('playwright')
const browser = await chromium.launch()
const size = { width: 1440, height: 900 }
const context = await browser.newContext({
viewport: size,
recordVideo: { dir: join(runDir, 'videos'), size },
})
try {
const page = await context.newPage()
const video = page.video()
// Navigate and exercise the real application here.
await context.close()
await video.saveAs(join(runDir, 'demo.webm'))
} finally {
await context.close()
await browser.close()
}
```
Import `createRequire` from `node:module` and `join` from `node:path`; set `repo` and a fresh `runDir` to absolute paths in the recording script. Retain the page's video handle before closing it. Await `context.close()` before `video.saveAs()` or encoding; closing only the browser does not guarantee the video's flush. Each page has its own video: choose the demonstrated page explicitly and do not concatenate unrelated pages or runs. Failed runs are diagnostic only.
Choose a short story with three to six meaningful states. Wait for unique semantic locators before acting; use `exact: true` for accessible-name equality and exact-text completion predicates that cannot match a prompt echo. Fixed waits may provide a reading hold after the state is verified, but never establish readiness. Preserve animations and scrolling in the continuous video.
When demonstrating a tool call, rejection, or recovery, open its detail or trajectory so the video shows the tool identity, status or stable error code, and downstream result. If a transient running state matters, prompt for a slow foreground operation and observe its concrete DOM marker; the video records it without screenshot timing races. Give the model a short final sentinel to anchor completion. Stop an unnecessarily long real-API run after the demonstrated state is visible.
Capture no secrets, personal data, unrelated tabs, or notifications. Browser video contains page content, not browser chrome; avoid rendering credential-bearing URLs in the application. Review the whole selected interval, including intermediate states. Keep one viewport throughout.
## Encode the GIF
Require `python3`, `ffmpeg`, and `ffprobe`. If either media binary is missing, report the dependency instead of installing software without authorization.
Export `GIF_SKILL_DIR` as this skill's absolute directory on its own line before the python command — an inline `GIF_SKILL_DIR=... python3 "$GIF_SKILL_DIR/..."` assignment fails, because the argument expands before the assignment takes effect:
Require `python3`, `ffmpeg`, and `ffprobe`. If a media binary is missing, report the dependency instead of installing software without authorization. Export `GIF_SKILL_DIR` on its own line before using it; an inline assignment cannot affect argument expansion in the same command.
```sh
export GIF_SKILL_DIR=/absolute/path/to/this/skill
python3 "$GIF_SKILL_DIR/scripts/encode_gif.py" \
/absolute/path/to/frames \
/absolute/path/to/demo.webm \
/absolute/path/to/demo.gif \
--durations 1.5,1.5,1.5,3.5 \
--fps 10 \
--max-width 1200 \
--colors 128
--start 2 --end 32 --speed 2 --final-hold 3 \
--fps 10 --max-width 1200 --colors 128
```
One duration applies to every frame; otherwise provide one comma-separated positive duration per frame, holding the final settled state longest. The encoder rejects fewer than two frames, mismatched dimensions or durations, invalid limits, accidental overwrite, unexpected duration, and output above `--max-bytes`.
`--start` and `--end` select one continuous source interval in seconds. Defaults retain the full video at 1× speed and add a two-second final hold. `--speed` changes playback speed; disclose it and the selected interval beside the GIF so the demo cannot imply measured response latency. Use observed video times, not guessed wall-clock offsets, and preserve the complete cause and outcome of the demonstrated behavior. The final hold repeats the last selected frame. Keep the original WebM for QA; do not splice separate runs or synthesize missing states.
For a large artifact, reduce `--max-width` first, then `--colors` or `--fps`; retain readable text and the final state long enough to inspect. Use `--force` only after resolving the exact output path.
The encoder probes WebM container duration, applies trim and speed before palette conversion, and checks encoded duration, animation, width, and byte size. It refuses an empty or out-of-range interval, a selection shorter than two output frames, mode-inappropriate flags, and accidental overwrite. Reduce `--max-width`, then `--colors` or `--fps` for a large artifact; preserve readable text. Use `--force` only after resolving the exact output path.
### Screenshot capture
When continuous video is unavailable or the user requests a storyboard, follow the available browser-control workflow. Capture three to six verified states from one isolated run with the browser's screenshot API. Save returned image bytes directly under one run directory as `00-initial.png`, `01-typed.png`, and so on; use identical dimensions and crop. For a transient state, poll its DOM marker and capture within the same browser-script call.
```sh
python3 "$GIF_SKILL_DIR/scripts/encode_gif.py" \
/absolute/path/to/frames /absolute/path/to/demo.gif \
--durations 1.5,1.5,1.5,3.5 --fps 10 --max-width 1200 --colors 128
```
One duration applies to every screenshot; otherwise supply one positive duration per frame and hold the settled state longest. Directory input rejects fewer than two frames and mismatched dimensions or duration counts. Video timing flags apply only to video files; `--durations` and `--pattern` apply only to screenshot directories.
## Verify the artifact
1. Read the encoder's JSON summary and confirm the output path, source and encoded frame counts, dimensions, duration, and byte size.
1. Read the encoder's JSON summary and confirm the output path, source interval and speed (or screenshot count), encoded frame count, dimensions, duration, and byte size.
2. Visually read the encoded GIF itself, not only the source frames. Confirm that the transition is legible, the last state is held long enough, and no sensitive content appears. If the viewer renders only the first frame, decode representative frames from the encoded GIF with `ffmpeg` and inspect those; the pre-encode screenshots do not prove the encoded order, palette, or final hold.
3. Run `git status --short` and confirm frames and the artifact landed only under ignored paths.
3. Run `git status --short` and confirm raw video, QA frames, and the artifact landed only under ignored paths.
4. Return the absolute GIF path, render it when the client supports local media, and state whether the recording used a real API, fixture, or another transport. When the task does not include attaching the GIF to a pull request, stop here.
Encoder maintenance: run `python3 -m unittest discover -s "$GIF_SKILL_DIR/scripts" -p 'test_*.py' -v` with the media prerequisites installed. These local media tests do not run in repository CI.
## Publish the GIF
Perform this step only when the task includes attaching the GIF to a pull request.
@@ -92,14 +121,16 @@ Write the GIF into the body file as an ordinary local-path reference, using the
![<alt text>](<path/to/demo.gif>)
```
Immediately before attaching, re-read the pull request's live head — for a new pull request, the pushed branch tip — and compare it with the commit recorded next to the GIF. Stop and re-record when it moved. Then attach:
The demonstrated pull request is normally the publication target. A tooling pull request may instead embed a clearly labeled example from another pull request; name that source PR and compare its live head in every check below. Never attribute the example to the tooling branch.
Immediately before attaching, re-read the demonstrated pull request's live head — for a new demonstrated pull request, the pushed branch tip — and compare it with the commit recorded next to the GIF. Stop and re-record when it moved. Then attach:
```sh
gh pr create --body-file <body.md> --attach <path/to/demo.gif> # new pull request
gh pr edit <pr> --body-file <body.md> --attach <path/to/demo.gif> # existing pull request
```
`--attach` is repeatable but refuses the same file twice. A GIF the body does not reference is appended at the end, where alt text set on the flag (`--attach '<path>#<alt text>'`) applies; a rewritten reference keeps the body's alt text. After attaching, re-read the live head and require it to remain at that recorded commit. Re-read the live body and confirm the reference now points at the uploaded URL, render the body through GitHub's Markdown API and confirm the expected `<img>`, and fetch the uploaded URL once to confirm `200` and `image/gif`.
`--attach` is repeatable but refuses the same file twice. A GIF the body does not reference is appended at the end, where alt text set on the flag (`--attach '<path>#<alt text>'`) applies; a rewritten reference keeps the body's alt text. After attaching, re-read the demonstrated live head and require it to remain at that recorded commit. Re-read the live body and confirm the reference now points at the uploaded URL, render the body through GitHub's Markdown API and confirm the expected `<img>`, and fetch the uploaded URL once to confirm `200` and `image/gif`.
### Fall back to an assets branch
@@ -122,7 +153,7 @@ For a new series, make a fresh shallow scratch clone (`git clone --depth 1 <repo
After pushing, use authenticated GitHub API or raw requests to confirm the remote path, byte size, checksum, `200` response, and `image/gif` content type. An anonymous `404` does not disprove a private-repository asset; authenticate the verification instead. This proves the repository-member review path, not public availability.
Immediately before editing the pull-request body, re-read its live head and compare it with the commit recorded next to the GIF. Stop and re-record when it moved. After the edit, re-read the live head and require it to remain at that recorded commit. Separately, render the body through GitHub's Markdown API and confirm that the expected `<img>` is present.
Immediately before editing the pull-request body, re-read the demonstrated pull request's live head and compare it with the commit recorded next to the GIF. Stop and re-record when it moved. After the edit, re-read the demonstrated live head and require it to remain at that recorded commit. Separately, render the body through GitHub's Markdown API and confirm that the expected `<img>` is present.
Embed the GIF in the pull request body with the raw blob URL; the `?raw=true` suffix is required, because the plain blob URL renders GitHub's file page instead of the image:
@@ -1,5 +1,5 @@
#!/usr/bin/env python3
"""Encode lexically ordered browser screenshots into a verified GIF."""
"""Encode a Playwright video or ordered browser screenshots into a verified GIF."""
from __future__ import annotations
@@ -43,6 +43,17 @@ def positive_int(value: str) -> int:
return parsed
def nonnegative_float(value: str) -> float:
"""Parse a finite nonnegative timestamp or hold duration."""
try:
parsed = float(value)
except ValueError:
fail(f"expected a number, got {value!r}")
if not math.isfinite(parsed) or parsed < 0:
fail(f"expected a nonnegative finite number, got {value!r}")
return parsed
def parse_durations(value: str, frame_count: int) -> list[float]:
"""Expand one hold duration or validate one duration per source frame."""
parts = [part.strip() for part in value.split(",")]
@@ -102,6 +113,17 @@ def probe_stream(ffprobe: str, path: Path) -> dict[str, object]:
return streams[0]
def video_duration(ffprobe: str, path: Path) -> float:
"""Read container duration because Playwright WebM omits stream duration."""
result = run_json([
ffprobe, "-v", "error", "-show_entries", "format=duration", "-of", "json", str(path),
])
metadata = result.get("format")
if not isinstance(metadata, dict):
fail(f"missing format metadata for {path}")
return positive_float(str(metadata.get("duration")))
def stream_int(stream: dict[str, object], key: str, path: Path) -> int:
"""Read a positive integer stream field."""
try:
@@ -134,14 +156,17 @@ def write_concat_manifest(path: Path, frames: list[Path], durations: list[float]
def build_parser() -> argparse.ArgumentParser:
"""Build the command-line contract."""
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("frames", type=Path, help="directory containing lexically ordered frames")
parser.add_argument("input", type=Path, help="video file or directory of ordered screenshots")
parser.add_argument("output", type=Path, help="output .gif path")
parser.add_argument("--pattern", default="*.png", help="frame glob within the input directory")
parser.add_argument("--pattern", help="screenshot glob (default: *.png; directory input only)")
parser.add_argument(
"--durations",
default="2",
help="one hold duration or one comma-separated value per frame",
help="screenshot holds (default: 2 seconds each; directory input only)",
)
parser.add_argument("--start", type=nonnegative_float, help="video trim start in seconds (default: 0)")
parser.add_argument("--end", type=positive_float, help="video trim end in seconds (default: EOF)")
parser.add_argument("--speed", type=positive_float, help="video playback multiplier (default: 1)")
parser.add_argument("--final-hold", type=nonnegative_float, help="extra final video frame hold (default: 2 seconds)")
parser.add_argument("--fps", type=positive_int, default=10, help="encoded frames per second")
parser.add_argument(
"--max-width",
@@ -168,11 +193,14 @@ def build_parser() -> argparse.ArgumentParser:
def main() -> None:
"""Validate inputs, encode the GIF, verify it, and print a JSON summary."""
args = build_parser().parse_args()
frame_dir = args.frames.resolve()
source = args.input.resolve()
output = args.output.resolve()
if not frame_dir.is_dir():
fail(f"frame directory does not exist: {frame_dir}")
if not source.exists():
fail(f"input does not exist: {source}")
is_video = source.is_file()
if not is_video and not source.is_dir():
fail(f"input must be a video file or frame directory: {source}")
if output.suffix.lower() != ".gif":
fail(f"output must end in .gif: {output}")
if output.exists() and not args.force:
@@ -182,33 +210,68 @@ def main() -> None:
if args.fps > 30:
fail("--fps must not exceed 30")
frames = sorted(path.resolve() for path in frame_dir.glob(args.pattern) if path.is_file())
if len(frames) < 2:
fail(f"expected at least two frames matching {args.pattern!r} in {frame_dir}")
if output in frames:
fail("output path must not match an input frame")
durations = parse_durations(args.durations, len(frames))
expected_duration = sum(durations)
ffmpeg = require_binary("ffmpeg")
ffprobe = require_binary("ffprobe")
dimensions = {
(stream_int(stream, "width", frame), stream_int(stream, "height", frame))
for frame in frames
for stream in [probe_stream(ffprobe, frame)]
}
if len(dimensions) != 1:
fail(f"all frames must have identical dimensions, got {sorted(dimensions)}")
details: dict[str, object]
if is_video:
if args.pattern is not None or args.durations is not None:
fail("--pattern and --durations require a screenshot directory")
if source == output:
fail("output path must not match the input video")
probe_stream(ffprobe, source)
source_duration = video_duration(ffprobe, source)
start = args.start if args.start is not None else 0.0
end = args.end if args.end is not None else source_duration
speed = args.speed if args.speed is not None else 1.0
final_hold = args.final_hold if args.final_hold is not None else 2.0
if not start < end <= source_duration:
fail(f"video trim must satisfy 0 <= start < end <= {source_duration:.3f}")
if (end - start) / speed < 2 / args.fps:
fail("video selection must span at least two output frames before the final hold")
expected_duration = (end - start) / speed + final_hold
details = {
"sourceVideo": str(source), "sourceDurationSeconds": source_duration,
"startSeconds": start, "endSeconds": end, "speed": speed,
"finalHoldSeconds": final_hold,
}
else:
if any(value is not None for value in (args.start, args.end, args.speed, args.final_hold)):
fail("--start, --end, --speed and --final-hold require a video file")
pattern = args.pattern if args.pattern is not None else "*.png"
if not pattern:
fail("--pattern must not be empty")
frames = sorted(path.resolve() for path in source.glob(pattern) if path.is_file())
if len(frames) < 2:
fail(f"expected at least two frames in {source}")
if output in frames:
fail("output path must not match an input frame")
durations = parse_durations(args.durations if args.durations is not None else "2", len(frames))
expected_duration = sum(durations)
dimensions = {
(stream_int(stream, "width", frame), stream_int(stream, "height", frame))
for frame in frames
for stream in [probe_stream(ffprobe, frame)]
}
if len(dimensions) != 1:
fail(f"all frames must have identical dimensions, got {sorted(dimensions)}")
details = {"sourceFrames": len(frames)}
output.parent.mkdir(parents=True, exist_ok=True)
with tempfile.TemporaryDirectory(prefix="record-browser-gif-") as temporary:
manifest = Path(temporary) / "frames.ffconcat"
write_concat_manifest(manifest, frames, durations)
if is_video:
inputs = ["-i", str(source)]
timing = f"trim=start={start}:end={end},setpts=(PTS-STARTPTS)/{speed},"
hold = f",tpad=stop_mode=clone:stop_duration={final_hold}"
else:
manifest = Path(temporary) / "frames.ffconcat"
write_concat_manifest(manifest, frames, durations)
inputs = ["-f", "concat", "-safe", "0", "-i", str(manifest)]
timing = ""
hold = ""
scale = f"scale='min({args.max_width},iw)':-2:flags=lanczos"
palette = f"palettegen=max_colors={args.colors}:stats_mode=full"
filters = (
f"fps={args.fps},{scale},split[base][palette_input];"
f"{timing}fps={args.fps},{scale}{hold},split[base][palette_input];"
f"[palette_input]{palette}[palette];"
"[base][palette]paletteuse=dither=bayer:bayer_scale=3:diff_mode=rectangle"
)
@@ -217,12 +280,7 @@ def main() -> None:
"-hide_banner",
"-loglevel",
"error",
"-f",
"concat",
"-safe",
"0",
"-i",
str(manifest),
*inputs,
"-vf",
filters,
"-loop",
@@ -261,7 +319,7 @@ def main() -> None:
json.dumps(
{
"output": str(output),
"sourceFrames": len(frames),
**details,
"encodedFrames": encoded_frames,
"width": width,
"height": height,
@@ -0,0 +1,103 @@
"""Exercise GIF timing and rejection through real ffmpeg/ffprobe subprocesses."""
import json
import subprocess
import sys
import tempfile
import unittest
from pathlib import Path
SCRIPT = Path(__file__).with_name('encode_gif.py')
class EncodeGifTest(unittest.TestCase):
def setUp(self):
self.temporary = tempfile.TemporaryDirectory(prefix='gif-encoder-test-')
self.addCleanup(self.temporary.cleanup)
self.root = Path(self.temporary.name)
self.video = self.root / 'browser video.webm'
self.output = self.root / 'demo.gif'
subprocess.run([
'ffmpeg', '-v', 'error',
'-f', 'lavfi', '-i', 'color=red:s=64x48:r=10:d=2',
'-f', 'lavfi', '-i', 'color=lime:s=64x48:r=10:d=2',
'-f', 'lavfi', '-i', 'color=blue:s=64x48:r=10:d=2',
'-filter_complex', '[0:v][1:v][2:v]concat=n=3:v=1:a=0',
'-c:v', 'libvpx', str(self.video),
], check=True, capture_output=True)
def encode(self, source=None, *options):
return subprocess.run([
sys.executable, str(SCRIPT), str(source or self.video), str(self.output), *options,
], capture_output=True, text=True)
def assert_color(self, seconds, channel):
result = subprocess.run([
'ffmpeg', '-v', 'error', '-i', str(self.output), '-ss', str(seconds),
'-frames:v', '1', '-vf', 'scale=1:1', '-f', 'rawvideo', '-pix_fmt', 'rgb24', '-',
], check=True, capture_output=True)
pixel = result.stdout
self.assertEqual(len(pixel), 3)
self.assertGreater(pixel[channel], 200)
self.assertTrue(all(value < 40 for index, value in enumerate(pixel) if index != channel))
def test_video_trim_speed_final_hold_and_palette_order(self):
result = self.encode(None, '--start', '1', '--end', '5', '--speed', '2',
'--final-hold', '1', '--max-width', '32')
self.assertEqual(result.returncode, 0, result.stderr)
summary = json.loads(result.stdout)
self.assertAlmostEqual(summary['durationSeconds'], 3, delta=0.2)
self.assertEqual((summary['width'], summary['height']), (32, 24))
self.assert_color(0.1, 0)
self.assert_color(0.8, 1)
self.assert_color(2.7, 2)
def test_video_defaults_keep_full_duration(self):
result = self.encode()
self.assertEqual(result.returncode, 0, result.stderr)
self.assertAlmostEqual(json.loads(result.stdout)['durationSeconds'], 8, delta=0.2)
self.assert_color(7.7, 2)
def test_invalid_video_options_do_not_write_output(self):
for options in [
['--start', '-1'], ['--speed', 'nan'], ['--final-hold', 'inf'],
['--start', '3', '--end', '3'], ['--end', '7'],
['--speed', '1000'], ['--durations', '2'], ['--pattern', '*.png'],
]:
with self.subTest(options=options):
result = self.encode(None, *options)
self.assertNotEqual(result.returncode, 0)
self.assertIn('error:', result.stderr)
self.assertFalse(self.output.exists())
def test_screenshot_durations_and_video_flag_rejection(self):
frames = self.root / 'frames'
frames.mkdir()
for index, color in enumerate([b'\xff\x00\x00', b'\x00\x00\xff']):
(frames / f'{index}.ppm').write_bytes(b'P6\n64 48\n255\n' + color * (64 * 48))
rejected = self.encode(frames, '--speed', '1')
self.assertNotEqual(rejected.returncode, 0)
self.assertIn('require a video file', rejected.stderr)
for options in [['--pattern', ''], ['--pattern', '*.ppm', '--durations', ''],
['--pattern', '*.ppm', '--durations', '1,2,3']]:
with self.subTest(options=options):
self.assertNotEqual(self.encode(frames, *options).returncode, 0)
self.assertFalse(self.output.exists())
result = self.encode(frames, '--pattern', '*.ppm', '--durations', '0.5,1.5')
self.assertEqual(result.returncode, 0, result.stderr)
self.assertEqual(json.loads(result.stdout)['sourceFrames'], 2)
self.assert_color(0.1, 0)
self.assert_color(1.8, 2)
def test_overwrite_and_size_limit(self):
self.output.write_bytes(b'keep')
rejected = self.encode()
self.assertNotEqual(rejected.returncode, 0)
self.assertEqual(self.output.read_bytes(), b'keep')
oversized = self.encode(None, '--force', '--max-bytes', '1', '--final-hold', '0.0')
self.assertNotEqual(oversized.returncode, 0)
self.assertIn('above --max-bytes', oversized.stderr)
if __name__ == '__main__':
unittest.main()