Skip to content

feat(cli): NDJSON render progress event stream for agents (--progress-format) - #3755

Open
mvanhorn wants to merge 1 commit into
heygen-com:mainfrom
mvanhorn:feat/render-progress-ndjson
Open

feat(cli): NDJSON render progress event stream for agents (--progress-format)#3755
mvanhorn wants to merge 1 commit into
heygen-com:mainfrom
mvanhorn:feat/render-progress-ndjson

Conversation

@mvanhorn

@mvanhorn mvanhorn commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

What

Adds --progress-format <tty|ndjson|none> (default tty) to hyperframes render, plus --progress-fd N.

With ndjson, every producer progress tick becomes one newline-terminated JSON object on stdout (or on an inherited file descriptor via --progress-fd): render.progress ticks plus exactly one terminal render.completed / render.failed event per render. The terminal failure event carries the producer's structured failure contract — failedStage and errorDetails (message, elapsed time, free memory, browser console tail, per-stage timings). Batch rows multiplex into the same stream, each event stamped with its row index. none disables progress output without touching the rest of the human output.

{"type":"render.progress","ts":"2026-09-08T00:49:12.599Z","progress":0.25,"status":"rendering","stage":"Starting frame capture","message":"Starting frame capture","framesRendered":0,"totalFrames":90,"failedStage":null}

Why

hyperframes render only drives a TTY progress bar, and --quiet removes even that. Agents and CI pipelines driving renders need machine-readable progress: is the render moving, what stage is it in, and — on failure — which stage died and why, in a form a program can branch on instead of regex-matching human log text. Remotion exposes this via onProgress callbacks; the producer already has a ProgressCallback contract internally — this PR surfaces it at the CLI boundary as an NDJSON stream.

How

  • packages/cli/src/ui/progressNdjson.ts — new ProgressNdjsonWriter, extracted next to progress.ts. It is ProgressCallback-shaped (publish(job, message)), maps the producer's terminal failed/complete status ticks to render.failed/render.completed, and latches after one terminal event per writer so the double-report paths (the producer's own terminal tick via publishRenderFailure + the CLI's handleRenderError) can't duplicate the terminal line. Sink failures (EPIPE when the consumer exits early) disable the stream instead of killing the render — same containment rule as the producer's OrderedRenderEventPublisher.
  • render.ts — flags, resolveRenderProgressCallback (precedence: active NDJSON writer → streams even under --quiet, since quiet governs human output; then --quiet/none disable; then the TTY bar), a guaranteed render.completed after a successful executeRenderJob, and a render.failed emission at the top of handleRenderError so throwOnError paths (batch rows) still close the stream.
  • render/plan.ts — flag parsing/validation: ndjson is rejected with --docker (the container's output is opaque to the host CLI) and with --json unless --progress-fd moves the stream off stdout; --progress-fd requires ndjson.
  • render/execute.ts — one shared sink per command, one writer per render; batch rows each get a writer stamped with row.index. When the stream owns stdout, human output is quiet-suppressed (the rule --batch --json already applies via effectiveQuiet) and stdout-bound console.log/info/debug are rerouted to stderr — caught live: the engine's [BrowserManager] Browser launched line otherwise lands mid-stream.
  • Producer untouched — the existing ProgressCallback/OrderedRenderEventPublisher/publishRenderFailure contract already delivers everything the stream needs.
  • Docsdocs/reference/cli-render.mdx (event schema, flag interactions) and docs/guides/agents.mdx (the agent recipe), both added to the nav.

Demo

HyperFrames-rendered walkthrough — the demo below is itself a HyperFrames composition (walkthrough/index.html, 25s · 1920×1080 · five scenes: title → NDJSON stream with live progress HUD → jq consumption ending in render.completed → the failure path with failedStage extraction → flag end card). It passed npx hyperframes lint and npx hyperframes check (0 errors, 139/139 WCAG contrast checks) and was rendered with:

npx hyperframes render --quality high --output demo.mp4   # hyperframes 0.8.31

The GIF is derived from that HyperFrames-rendered MP4 (ffmpeg palette two-pass, 960px/12fps).

HyperFrames-rendered walkthrough: NDJSON render progress for agents — streaming render.progress ticks with a live progress HUD, jq consuming the stream to render.completed, then the failure path extracting failedStage from render.failed, ending on the --progress-format / --progress-fd flag card

(MP4 version)

Previous demo (terminal screen recording — superseded by the HyperFrames-rendered walkthrough above) hyperframes render --progress-format ndjson demo: jq consuming live NDJSON progress events, then extracting failedStage from render.failed

(MP4 version)

Test plan

  • Unit tests added/updated

  • Manual testing performed

  • Documentation updated (if applicable)

  • packages/cli/src/ui/progressNdjson.test.ts (17 tests): event shape, publish ordering, terminal mapping + exactly-one-terminal dedupe, cancelled latch, batch row stamping, progress clamping, EPIPE containment, console redirect, stdout sink.

  • packages/cli/src/commands/render.progressNdjson.test.ts (16 tests): resolveRenderProgressCallback precedence with a fake ProgressCallback tick sequence (including quiet interaction and a stray post-terminal tick — no browser required), and createRenderPlan flag validation (--docker/--json/--progress-fd interactions, effectiveQuiet).

  • Full CLI suite: 3042 passed | 3 skipped (bunx vitest run in packages/cli).

  • bunx oxlint / bunx oxfmt --check clean on all touched files; tsc --noEmit clean for the CLI package.

  • Live end-to-end on Linux: success stream (pure JSON on stdout — zero non-JSON lines), --quiet keeps the stream flowing, --progress-fd 3 keeps human output on stdout with events on fd 3, a failing render exits 1 with render.failed carrying failedStage: "Extracting video frames" + errorDetails, and a 2-row --batch stamps row: 0/1 with per-row terminal events.

AI assistance disclosure

Add --progress-format <tty|ndjson|none> to hyperframes render. ndjson
streams one JSON object per producer progress tick to stdout (or an
inherited fd via --progress-fd N): render.progress ticks plus exactly
one terminal render.completed / render.failed carrying failedStage and
the producer's structured errorDetails. Batch rows multiplex into the
same stream stamped with their row index.

When the stream owns stdout, human output is quiet-suppressed and
stdout-bound console diagnostics are rerouted to stderr so pipes stay
clean; --quiet keeps silencing human output only, never the stream.

Docs: reference/cli-render + guides/agents (agent recipe).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants