diff --git a/.claude-plugin/marketplace.json b/.claude-plugin/marketplace.json index 91fa63b84..448436238 100644 --- a/.claude-plugin/marketplace.json +++ b/.claude-plugin/marketplace.json @@ -1,7 +1,7 @@ { "$schema": "https://anthropic.com/claude-code/marketplace.schema.json", "name": "diffusers-workflow", - "description": "Skills for driving a diffusers-workflow server from Claude Code: a composition skill per model family, one for cutting a series from them, one camera-and-look vocabulary compiled per family, and storyboards with a timed animatic.", + "description": "Skills for driving a diffusers-workflow server from Claude Code: a composition skill per model family, one for cutting a series from them, one camera-and-look vocabulary compiled per family, storyboards with a timed animatic, and a playbook for pieces longer than one clip.", "owner": { "name": "Don Kackman" }, @@ -9,7 +9,7 @@ { "name": "dw", "source": "./plugins/dw", - "description": "Compose MiniMax H3 video, MiniMax Music 3, LTX-2.5, Kandinsky 6.0 and FLUX 3 Klein workflows over a dw MCP server, cut a multi-episode series from them, direct camera and look in each family's caption format, and storyboard a scene before it is filmed: which template fits which shape, the hard rules, cost, and how to judge the output. Prompt format comes from the vendors' own guides." + "description": "Compose MiniMax H3 video, MiniMax Music 3, LTX-2.5, Kandinsky 6.0 and FLUX 3 Klein workflows over a dw MCP server, cut a multi-episode series from them, direct camera and look in each family's caption format, storyboard a scene before it is filmed, and plan a piece longer than one clip from measured levers: which template fits which shape, the hard rules, cost, and how to judge the output. Prompt format comes from the vendors' own guides." } ] } diff --git a/CLAUDE.md b/CLAUDE.md index 950bceaac..6fcdeeacd 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -74,10 +74,10 @@ The guides in `docs/`, by topic: composition skill per model family (`minimax-h3`, `minimax-music3`, `ltx-2.5`, `kandinsky-6`, `flux3-klein`) that chooses a template for a request's shape and states the family's hard rules, plus cross-cutting composition skills (`script-to-video`, -`series-episodes`, `cinematography`, `storyboard`) - shapes above the families that -orchestrate the decision trees, cast consistency across multiple generations, one -camera-and-look vocabulary compiled per family, and boards and a timed animatic before -video. Every skill the directory holds is named in `plugins/dw/README.md` and here, pinned +`series-episodes`, `cinematography`, `storyboard`, `long-form`) - shapes above the +families that orchestrate the decision trees, cast consistency across multiple +generations, one camera-and-look vocabulary compiled per family, boards and a timed +animatic before video, and the measured playbook for pieces longer than one clip. Every skill the directory holds is named in `plugins/dw/README.md` and here, pinned by `tests/test_plugin_skills.py`; that README says how the rest is pinned and versioned. ## Security Rules diff --git a/README.md b/README.md index 51f1d8fff..257175a74 100644 --- a/README.md +++ b/README.md @@ -18,6 +18,37 @@ variable, run it again, compare takes and keep the one you like. A workspace's overview in the web UI: recent outputs from an agent-made musical trailer, its jobs, and the workflows the agent wrote +### Coming from ComfyUI? + +The job is the same: load models, wire steps together, get media out. What changes is +who builds the graph and how it is checked before it runs. + +| | ComfyUI | diffusers-workflow | +|---|---|---| +| A workflow is | a node graph you draw | a JSON document of named steps an agent writes, and you can edit as a form | +| A new model needs | a custom node pack | nothing, if diffusers has the pipeline: any pipeline class is reachable by name | +| Mistakes are found | links are type-checked on queue; the rest when it runs | at validation: schema, references and every argument against the real pipeline signatures, before any model loads | +| Before a long render | no estimate | `validate_workflow` quotes GPU minutes from this machine's own run history | +| Between runs | models stay loaded | models stay loaded (one persistent worker per GPU) | +| A result | an image with the graph embedded | the same, and a job that keeps the exact workflow it ran, drawn as a data-flow graph, re-runnable with a new seed | + +It is not a ComfyUI replacement for hand-wiring a graph node by node. It is for +describing what you want and letting an agent compose, price, run and judge it, then +taking over in the browser where that is quicker. + +### What it makes + +| | Ready-made templates | +|---|---| +| **Images** | FLUX 3 Klein (text, edits, multi-reference, layout boxes), FLUX.1 / FLUX.2 / Krea, Qwen Image (and edit), Z-Image, SDXL and SD 1.5; ControlNet, IP-Adapter, LoRA, inpaint and outpaint | +| **Video with sound** | MiniMax H3 (dialogue, multi-shot cuts, music videos, reference subjects and voices), LTX-2.5 (image-to-video, keyframes, 2x upscale, restore and colourise, alpha mattes), Kandinsky 6 | +| **Music, speech, sound** | MiniMax Music 3 (sung lyrics or instrumental), Audio8 TTS voice cloning and MMS TTS, MMAudio sound effects from text or under a clip, Whisper transcription | +| **Longer than one clip** | chained segments that hold the look and audio level across seams, repair of one bad segment in place, music and narration of any length, storyboards and timed animatics, cut-and-score assembly, and an OpenTimelineIO export that DaVinci Resolve opens as one clip per shot | + +Anything else diffusers can load runs from a workflow too; the templates are where the +agent starts, not a limit. The full list is in [workflows/](workflows/) and the +[dw plugin](plugins/dw/README.md). + ## Getting started **1. Install.** The script picks the right torch build for your platform, creates a @@ -109,7 +140,8 @@ The [dw plugin](plugins/dw/README.md) adds what a general-purpose agent can't kn on its own: one skill per model family (MiniMax H3 and Music 3, LTX-2.5, Kandinsky 6, FLUX 3 Klein) that picks the template for a request and states the rules that bite. Cross-cutting skills cover script-to-video, series with a consistent cast, -storyboards and animatics, and a shared camera-and-look vocabulary. +storyboards and animatics, pieces longer than one clip, and a shared +camera-and-look vocabulary. ![The same session hours later: shots rendering one at a time, with the agent reporting progress between them](docs/img/claude-generating.png) @@ -182,6 +214,38 @@ you do, the [Workflow Guide](docs/WORKFLOW_GUIDE.md) is the reference, the a command-line client, `python -m dw.run workflow.json prompt="a cat"`, which queues a job on a running server (see [Server & Web UI](docs/SERVER.md#the-command-line)). +Here is the shape, trimmed: a still from FLUX 3 Klein, animated by LTX-2.5. + +```json +{ + "variables": { "prompt": "a lighthouse at dusk, waves breaking", "seed": null }, + "seed": "variable:seed", + "steps": [ + { + "name": "still", + "pipeline": { + "configuration": { "component_type": "ModularPipeline" }, + "from_pretrained_arguments": { "model_name": "black-forest-labs/flux-3-image-klein" }, + "arguments": { "prompt": "variable:prompt", "width": 1280, "height": 720 } + } + }, + { + "name": "clip", + "pipeline": { + "configuration": { "component_type": "LTX2ImageToVideoPipeline" }, + "from_pretrained_arguments": { "model_name": "Lightricks/LTX-2.5-Diffusers" }, + "arguments": { "prompt": "variable:prompt", "image": "previous_result:still" } + }, + "result": { "content_type": "video/mp4" } + } + ] +} +``` + +`component_type` is a diffusers class name and `arguments` are its `__call__` +arguments, so the diffusers docs are the reference for what a step takes. The real +templates add the device placement, quantization and size rules each model needs. + The engine also covers: - **Quantization:** BitsAndBytes, TorchAO, GGUF, SDNQ and optimum-quanto, set per @@ -189,8 +253,8 @@ The engine also covers: - **Acceleration:** FirstBlockCache, FasterCache, MagCache, TaylorSeerCache, `torch.compile` and attention backends. - **Adapters and prompts:** LoRA, IP-Adapter and A1111-style prompt weighting. -- **Long video and caching:** long-video chaining with audio-driven length, and - step-output caching, so re-running a fixed-seed workflow finishes instantly. +- **Step-output caching:** re-running a fixed-seed workflow, or one that changed + only a later step, reuses the steps that did not change. - **Utility tasks:** upscaling, face restoration, segmentation, captioning, frame interpolation, audio slicing, cut-and-score assembly and more. diff --git a/bench/README.md b/bench/README.md new file mode 100644 index 000000000..b0f3703d1 --- /dev/null +++ b/bench/README.md @@ -0,0 +1,36 @@ +# Long-form bench server + +A second `dw.serve` on lem for benching long video takes, so the main server (the automated tester's, port 8765) is never disturbed. + +| | | +|---|---| +| Port | 8766 (`http://lem:8766`) | +| Card | `cuda:1` (`BENCH_DEVICES` overrides); the main server runs on `cuda:0` only (systemd drop-in `~/.config/systemd/user/dw-serve.service.d/devices.conf`) | +| Workspace | `/home/don/bench-workspace` | +| Code | worktree `/home/don/dw-bench`, detached at `origin/`; the main checkout is never touched | +| Token | `BENCH_TOKEN`, else `/home/don/.dw-bench-token` (mode 600) | +| Log | `/home/don/dw-bench.log` | + +## Use + +```bash +# first time (the worktree does not exist yet): from any lem shell +ssh lem 'git -C ~/diffusers-workflow fetch origin feat/long-form-media && git -C ~/diffusers-workflow worktree add --detach ~/dw-bench origin/feat/long-form-media' +# afterwards +ssh lem 'bash ~/dw-bench/scripts/bench_server.sh deploy feat/long-form-media' +ssh lem 'bash ~/dw-bench/scripts/bench_server.sh start' # waits up to 120 s for /api/health +ssh lem 'bash ~/dw-bench/scripts/bench_server.sh status' +ssh lem 'bash ~/dw-bench/scripts/bench_server.sh stop' # SIGTERM, SIGKILL after 30 s +``` + +A code change needs `stop`, `deploy`, `start`: the server runs persistent workers. + +## Host RAM rule + +The two servers do not share the host-RAM hold. lem has 125 GB (122 GB available idle), but no recent H3 chain with a projected `host_ram_need` was on the main server to confirm that both servers' largest jobs fit together. Until that is measured, run the bench only while the main server is idle (`curl localhost:8765/api/jobs?status=running` on lem). + +They share no step cache either: an arms file goes to one server (`bench/driver.py --server`) for its whole life, including measure and rerun, because another server's run is a cache miss. + +## Reaching it from the Mac + +lem's firewall passes 8765 but not 8766 (a rule needs sudo: `sudo ufw allow 8766/tcp`, if ufw is the filter). Until then, tunnel: `ssh -f -N -L 18766:127.0.0.1:8766 lem`, then use `http://127.0.0.1:18766`. diff --git a/bench/arms/audio-long-2.json b/bench/arms/audio-long-2.json new file mode 100644 index 000000000..9844ce7a3 --- /dev/null +++ b/bench/arms/audio-long-2.json @@ -0,0 +1,47 @@ +[ + { + "name": "s6-music-full-335s-s1166", + "scene": "s6-narration", + "workflow": "music-long-s6-full.json", + "arguments": {}, + "seed": 1166 + }, + { + "name": "s6-music-full-335s-s1167", + "scene": "s6-narration", + "workflow": "music-long-s6-full.json", + "arguments": {}, + "seed": 1167 + }, + { + "name": "s6-music-full-335s-xf3000-s1166", + "scene": "s6-narration", + "workflow": "music-long-s6-full.json", + "arguments": { + "crossfade_ms": 3000 + }, + "seed": 1166 + }, + { + "name": "s6-narr-long-t37-wps3-s1166", + "scene": "s6-narration", + "workflow": "narration-long-s6.json", + "arguments": { + "max_seconds": 37, + "words_per_second": 3.0 + }, + "seed": 1166, + "workflow_id_suffix": "-t37" + }, + { + "name": "s6-narr-long-t37-wps3-s1167", + "scene": "s6-narration", + "workflow": "narration-long-s6.json", + "arguments": { + "max_seconds": 37, + "words_per_second": 3.0 + }, + "seed": 1167, + "workflow_id_suffix": "-t37" + } +] diff --git a/bench/arms/audio-long-prep.json b/bench/arms/audio-long-prep.json new file mode 100644 index 000000000..bc61a4a70 --- /dev/null +++ b/bench/arms/audio-long-prep.json @@ -0,0 +1,9 @@ +[ + { + "name": "s6-narrator-ref-s1166", + "scene": "s6-narration", + "workflow": "narrator-reference-s6.json", + "arguments": {}, + "seed": 1166 + } +] diff --git a/bench/arms/audio-long.json b/bench/arms/audio-long.json new file mode 100644 index 000000000..a2e055751 --- /dev/null +++ b/bench/arms/audio-long.json @@ -0,0 +1,57 @@ +[ + { + "name": "s6-narr-long-240s-t60-s1166", + "scene": "s6-narration", + "workflow": "narration-long-s6.json", + "arguments": {}, + "seed": 1166 + }, + { + "name": "s6-narr-long-240s-t60-s1167", + "scene": "s6-narration", + "workflow": "narration-long-s6.json", + "arguments": {}, + "seed": 1167 + }, + { + "name": "s6-narr-long-240s-t45-s1166", + "scene": "s6-narration", + "workflow": "narration-long-s6.json", + "arguments": { + "max_seconds": 45 + }, + "seed": 1166 + }, + { + "name": "s6-narr-long-240s-gap005-s1166", + "scene": "s6-narration", + "workflow": "narration-long-s6.json", + "arguments": { + "gap_seconds": 0.05 + }, + "seed": 1166 + }, + { + "name": "s6-music-long-335s-s1166", + "scene": "s6-narration", + "workflow": "music-long-s6.json", + "arguments": {}, + "seed": 1166 + }, + { + "name": "s6-music-long-335s-s1167", + "scene": "s6-narration", + "workflow": "music-long-s6.json", + "arguments": {}, + "seed": 1167 + }, + { + "name": "s6-music-long-335s-xf3000-s1166", + "scene": "s6-narration", + "workflow": "music-long-s6.json", + "arguments": { + "crossfade_ms": 3000 + }, + "seed": 1166 + } +] diff --git a/bench/arms/audio-pass.json b/bench/arms/audio-pass.json new file mode 100644 index 000000000..197f1117e --- /dev/null +++ b/bench/arms/audio-pass.json @@ -0,0 +1,184 @@ +[ + { + "name": "ap-s4-cold_open-ls1", + "scene": "s4-montage", + "workflow": "add-sound.json", + "arguments": { + "source_video": "output:BenchS4Montage/20261104-040255-9813688b/intermediate/BenchS4Montage-shot@cold_open.2-0.0.mp4", + "prompt": "integrated_multimodal_description: The target video is in the style of a 1990s multi-camera sitcom, shot on studio video with warm, even, shadowless lighting. [Shot 1] A wide shot frames a living room set: a sagging olive-green couch centre frame in front of a brick wall with a window onto a painted city backdrop. A heavyset man in his mid 40s with a shaved head, in an open gray flannel shirt, sits at the right end of the couch reading a magazine and does not look up. A wiry man in his early 30s with short curly dark hair, in a mustard-yellow sweater vest, strides in from frame left holding an empty ice cream carton, stops beside the couch and turns it upside down to show it is empty. The wiry man (S1), a fast, nasal, high-pitched male tenor voice, tight with disbelief, says [English] Okay. Who finished the pistachio? The heavyset man keeps reading. A live studio audience laughs warmly as the carton turns over. The camera holds a static wide shot throughout.\n\noverall_soundscape: A bright, close television-studio room tone. Quick footsteps on a hollow stage floor as the wiry man enters, the papery rattle of the empty carton as it turns over, a soft page turn from the couch, and a warm burst of live studio-audience laughter that begins as the carton flips and settles under the end of the shot.\n\nnon_diegetic_music: None.", + "num_frames": 124, + "width": 960, + "height": 544, + "listen_scale": 1 + }, + "seed": 1166, + "workflow_id_suffix": "-ap", + "moments": [ + "frame:12", + "frame:62", + "frame:118" + ] + }, + { + "name": "ap-s4-deflect-ls1", + "scene": "s4-montage", + "workflow": "add-sound.json", + "arguments": { + "source_video": "output:BenchS4Montage/20261104-040255-9813688b/intermediate/BenchS4Montage-shot@deflect.3-0.0.mp4", + "prompt": "integrated_multimodal_description: The target video is in the style of a 1990s multi-camera sitcom, shot on studio video with warm, even, shadowless lighting. [Shot 1] A medium close-up frames a heavyset man in his mid 40s with a shaved head, calm and heavy-lidded, in an open gray flannel shirt, seated at the end of an olive-green couch in front of a brick wall, reading a magazine held just below the frame line. He does not look up. The man (S1), a flat, low, unhurried male baritone voice, completely unbothered, says [English] Technically? The freezer has a defrost cycle. On the last word he turns a page. A live studio audience laughs as the line lands. The camera holds a static close-up throughout.\n\noverall_soundscape: A bright, close television-studio room tone. A soft page turn, the faint creak of couch springs, and a rolling wave of live studio-audience laughter that starts on the last word of the line and carries under the end of the shot.\n\nnon_diegetic_music: None.", + "num_frames": 124, + "width": 960, + "height": 544, + "listen_scale": 1 + }, + "seed": 1166, + "workflow_id_suffix": "-ap", + "moments": [ + "frame:12", + "frame:62", + "frame:118" + ] + }, + { + "name": "ap-s4-react-ls1", + "scene": "s4-montage", + "workflow": "add-sound.json", + "arguments": { + "source_video": "output:BenchS4Montage/20261104-040255-9813688b/intermediate/BenchS4Montage-shot@react.4-0.0.mp4", + "prompt": "integrated_multimodal_description: The target video is in the style of a 1990s multi-camera sitcom, shot on studio video with warm, even, shadowless lighting. [Shot 1] A medium close-up frames a wiry man in his early 30s with short curly dark hair and expressive eyebrows, in a mustard-yellow sweater vest, standing beside a couch in front of a brick wall, an empty ice cream carton raised in one hand at head height. The man (S1), a fast, nasal, high-pitched male tenor voice, tight with disbelief, cracking upward on the last word, says [English] The defrost cycle used a spoon? He shakes the empty carton once on the word spoon. A live studio audience erupts in a big laugh as the line lands. The camera holds a static close-up throughout.\n\noverall_soundscape: A bright, close television-studio room tone. The hollow papery rattle of the empty carton as it shakes, and a big rising wave of live studio-audience laughter that breaks on the word spoon and rolls under the end of the shot.\n\nnon_diegetic_music: None.", + "num_frames": 124, + "width": 960, + "height": 544, + "listen_scale": 1 + }, + "seed": 1166, + "workflow_id_suffix": "-ap", + "moments": [ + "frame:12", + "frame:62", + "frame:118" + ] + }, + { + "name": "ap-s4-button-ls1", + "scene": "s4-montage", + "workflow": "add-sound.json", + "arguments": { + "source_video": "output:BenchS4Montage/20261104-040255-9813688b/intermediate/BenchS4Montage-shot@button.5-0.0.mp4", + "prompt": "integrated_multimodal_description: The target video is in the style of a 1990s multi-camera sitcom, shot on studio video with warm, even, shadowless lighting. [Shot 1] A medium close-up frames a heavyset man in his mid 40s with a shaved head, calm and heavy-lidded, in an open gray flannel shirt, seated at the end of an olive-green couch in front of a brick wall, a magazine closed in his lap. He looks up, holds a beat of eye contact just past the lens, and gives one slow shrug. The man (S1), a flat, low, unhurried male baritone voice, perfectly deadpan, says [English] I don't know its life. He goes back to the magazine. A live studio audience gives a big laugh with scattered applause. The camera holds a static close-up throughout.\n\noverall_soundscape: A bright, close television-studio room tone. The soft slap of the magazine reopening, a couch-spring creak under the shrug, and a big wave of live studio-audience laughter with a few claps in it that breaks after the line and rolls under the end of the shot.\n\nnon_diegetic_music: None.", + "num_frames": 124, + "width": 960, + "height": 544, + "listen_scale": 1 + }, + "seed": 1166, + "workflow_id_suffix": "-ap", + "moments": [ + "frame:12", + "frame:62", + "frame:118" + ] + }, + { + "name": "ap-s4-cold_open-ls050", + "scene": "s4-montage", + "workflow": "add-sound.json", + "arguments": { + "source_video": "output:BenchS4Montage/20261104-040255-9813688b/intermediate/BenchS4Montage-shot@cold_open.2-0.0.mp4", + "prompt": "integrated_multimodal_description: The target video is in the style of a 1990s multi-camera sitcom, shot on studio video with warm, even, shadowless lighting. [Shot 1] A wide shot frames a living room set: a sagging olive-green couch centre frame in front of a brick wall with a window onto a painted city backdrop. A heavyset man in his mid 40s with a shaved head, in an open gray flannel shirt, sits at the right end of the couch reading a magazine and does not look up. A wiry man in his early 30s with short curly dark hair, in a mustard-yellow sweater vest, strides in from frame left holding an empty ice cream carton, stops beside the couch and turns it upside down to show it is empty. The wiry man (S1), a fast, nasal, high-pitched male tenor voice, tight with disbelief, says [English] Okay. Who finished the pistachio? The heavyset man keeps reading. A live studio audience laughs warmly as the carton turns over. The camera holds a static wide shot throughout.\n\noverall_soundscape: A bright, close television-studio room tone. Quick footsteps on a hollow stage floor as the wiry man enters, the papery rattle of the empty carton as it turns over, a soft page turn from the couch, and a warm burst of live studio-audience laughter that begins as the carton flips and settles under the end of the shot.\n\nnon_diegetic_music: None.", + "num_frames": 124, + "width": 960, + "height": 544, + "listen_scale": 0.5 + }, + "seed": 1166, + "workflow_id_suffix": "-ap", + "moments": [ + "frame:12", + "frame:62", + "frame:118" + ] + }, + { + "name": "ap-s4-cold_open-ls025", + "scene": "s4-montage", + "workflow": "add-sound.json", + "arguments": { + "source_video": "output:BenchS4Montage/20261104-040255-9813688b/intermediate/BenchS4Montage-shot@cold_open.2-0.0.mp4", + "prompt": "integrated_multimodal_description: The target video is in the style of a 1990s multi-camera sitcom, shot on studio video with warm, even, shadowless lighting. [Shot 1] A wide shot frames a living room set: a sagging olive-green couch centre frame in front of a brick wall with a window onto a painted city backdrop. A heavyset man in his mid 40s with a shaved head, in an open gray flannel shirt, sits at the right end of the couch reading a magazine and does not look up. A wiry man in his early 30s with short curly dark hair, in a mustard-yellow sweater vest, strides in from frame left holding an empty ice cream carton, stops beside the couch and turns it upside down to show it is empty. The wiry man (S1), a fast, nasal, high-pitched male tenor voice, tight with disbelief, says [English] Okay. Who finished the pistachio? The heavyset man keeps reading. A live studio audience laughs warmly as the carton turns over. The camera holds a static wide shot throughout.\n\noverall_soundscape: A bright, close television-studio room tone. Quick footsteps on a hollow stage floor as the wiry man enters, the papery rattle of the empty carton as it turns over, a soft page turn from the couch, and a warm burst of live studio-audience laughter that begins as the carton flips and settles under the end of the shot.\n\nnon_diegetic_music: None.", + "num_frames": 124, + "width": 960, + "height": 544, + "listen_scale": 0.25 + }, + "seed": 1166, + "workflow_id_suffix": "-ap", + "moments": [ + "frame:12", + "frame:62", + "frame:118" + ] + }, + { + "name": "ap-s4-cold_open-foley-ls1", + "scene": "s4-montage", + "workflow": "add-sound.json", + "arguments": { + "source_video": "output:BenchS4Montage/20261104-040255-9813688b/intermediate/BenchS4Montage-shot@cold_open.2-0.0.mp4", + "prompt": "integrated_multimodal_description: The target video is in the style of a 1990s multi-camera sitcom, shot on studio video with warm, even, shadowless lighting. [Shot 1] A wide shot frames a living room set: a sagging olive-green couch centre frame in front of a brick wall. A heavyset man with a shaved head sits at the right end of the couch reading a magazine. A wiry man with curly dark hair, in a mustard-yellow sweater vest, strides in from frame left holding an empty ice cream carton, stops beside the couch and turns it upside down. Nobody speaks. The camera holds a static wide shot throughout.\n\noverall_soundscape: A bright, close television-studio room tone. Quick footsteps on a hollow stage floor, the papery rattle of the empty carton as it turns over, a soft page turn from the couch, and a warm burst of live studio-audience laughter that begins as the carton flips and settles under the end of the shot.\n\nnon_diegetic_music: None.", + "num_frames": 124, + "width": 960, + "height": 544, + "listen_scale": 1 + }, + "seed": 1166, + "workflow_id_suffix": "-ap", + "moments": [ + "frame:12", + "frame:62", + "frame:118" + ] + }, + { + "name": "ap-s3-seg1-same-line-ls1", + "scene": "s3-dialogue-walk", + "workflow": "add-sound-span.json", + "arguments": { + "source_video": "output:BenchS3DialogueWalk/20261104-035218-d2733a3a/final/BenchS3DialogueWalk-chained_keyframe_to_video_audio.0-0.0.mp4", + "start_frame": 0, + "prompt": "integrated_multimodal_description: [Shot 1] Cinematic, live-action, one continuous shot. A woman in her early thirties with shoulder-length dark hair, in a long olive coat, walks toward the camera down a quiet cobblestone street at golden hour, slow and steady, glancing up at the lamps. The woman (S1), a warm, low female alto voice, calm and unhurried, says [English] I told myself I would not come back here. The camera holds a steady medium shot at eye level, keeping pace with her as she walks.\n\noverall_soundscape: Soft footsteps on cobblestones, a faint breeze, distant street ambience.\n\nnon_diegetic_music: None.", + "num_frames": 124, + "width": 960, + "height": 544, + "listen_scale": 1 + }, + "seed": 1166, + "workflow_id_suffix": "-ap", + "moments": [ + "frame:12", + "frame:62", + "frame:118" + ] + }, + { + "name": "ap-s3-seg1-new-line-ls1", + "scene": "s3-dialogue-walk", + "workflow": "add-sound-span.json", + "arguments": { + "source_video": "output:BenchS3DialogueWalk/20261104-035218-d2733a3a/final/BenchS3DialogueWalk-chained_keyframe_to_video_audio.0-0.0.mp4", + "start_frame": 0, + "prompt": "integrated_multimodal_description: [Shot 1] Cinematic, live-action, one continuous shot. A woman in her early thirties with shoulder-length dark hair, in a long olive coat, walks toward the camera down a quiet cobblestone street at golden hour, slow and steady, glancing up at the lamps. The woman (S1), a warm, low female alto voice, calm and unhurried, says [English] I swore I would never set foot on this street again. The camera holds a steady medium shot at eye level, keeping pace with her as she walks.\n\noverall_soundscape: Soft footsteps on cobblestones, a faint breeze, distant street ambience.\n\nnon_diegetic_music: None.", + "num_frames": 124, + "width": 960, + "height": 544, + "listen_scale": 1 + }, + "seed": 1166, + "workflow_id_suffix": "-ap", + "moments": [ + "frame:12", + "frame:62", + "frame:118" + ] + } +] diff --git a/bench/arms/baseline-seeds.json b/bench/arms/baseline-seeds.json new file mode 100644 index 000000000..01cf08914 --- /dev/null +++ b/bench/arms/baseline-seeds.json @@ -0,0 +1,82 @@ +[ + { + "name": "control-s1-seed2", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": {}, + "seed": 2 + }, + { + "name": "control-s2-seed2", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": {}, + "seed": 2 + }, + { + "name": "control-s3-seed2", + "scene": "s3-dialogue-walk", + "workflow": "s3-dialogue-walk.json", + "arguments": {}, + "seed": 2 + }, + { + "name": "control-s4-seed2", + "scene": "s4-montage", + "workflow": "s4-montage.json", + "arguments": {}, + "seed": 2 + }, + { + "name": "control-s5-seed2", + "scene": "s5-ltx-action", + "workflow": "s5-ltx-action.json", + "arguments": {}, + "seed": 2 + }, + { + "name": "control-s6-seed2", + "scene": "s6-narration", + "workflow": "s6-narration.json", + "arguments": {}, + "seed": 2 + }, + { + "name": "best-s1-seed1166", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "anchor": "segment_1", + "guide_noise": 0.4 + }, + "seed": 1166 + }, + { + "name": "best-s1-seed2", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "anchor": "segment_1", + "guide_noise": 0.4 + }, + "seed": 2 + }, + { + "name": "best-s2-seed1166", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "guide_noise": 0.4 + }, + "seed": 1166 + }, + { + "name": "best-s2-seed2", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "guide_noise": 0.4 + }, + "seed": 2 + } +] diff --git a/bench/arms/baseline.json b/bench/arms/baseline.json new file mode 100644 index 000000000..8446b959f --- /dev/null +++ b/bench/arms/baseline.json @@ -0,0 +1,44 @@ +[ + { + "name": "baseline-s1", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": {}, + "seed": 1166 + }, + { + "name": "baseline-s2", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": {}, + "seed": 1166 + }, + { + "name": "baseline-s3", + "scene": "s3-dialogue-walk", + "workflow": "s3-dialogue-walk.json", + "arguments": {}, + "seed": 1166 + }, + { + "name": "baseline-s4", + "scene": "s4-montage", + "workflow": "s4-montage.json", + "arguments": {}, + "seed": 1166 + }, + { + "name": "baseline-s5", + "scene": "s5-ltx-action", + "workflow": "s5-ltx-action.json", + "arguments": {}, + "seed": 1166 + }, + { + "name": "baseline-s6", + "scene": "s6-narration", + "workflow": "s6-narration.json", + "arguments": {}, + "seed": 1166 + } +] diff --git a/bench/arms/capstone-controls.json b/bench/arms/capstone-controls.json new file mode 100644 index 000000000..5be0b9551 --- /dev/null +++ b/bench/arms/capstone-controls.json @@ -0,0 +1,10 @@ +[ + { + "name": "ctl10-guide-s2-s2", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": {}, + "seed": 2, + "workflow_id_suffix": "-ctl155" + } +] diff --git a/bench/arms/capstone.json b/bench/arms/capstone.json new file mode 100644 index 000000000..a804691fa --- /dev/null +++ b/bench/arms/capstone.json @@ -0,0 +1,50 @@ +[ + { + "name": "cap90-s2-s1166", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "capstone": { + "enter_below_sigma": 0.9 + } + }, + "seed": 1166, + "workflow_id_suffix": "-cap90" + }, + { + "name": "cap70-s2-s1166", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "capstone": { + "enter_below_sigma": 0.7 + } + }, + "seed": 1166, + "workflow_id_suffix": "-cap70" + }, + { + "name": "cap90-s2-s2", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "capstone": { + "enter_below_sigma": 0.9 + } + }, + "seed": 2, + "workflow_id_suffix": "-cap90" + }, + { + "name": "cap70-s2-s2", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "capstone": { + "enter_below_sigma": 0.7 + } + }, + "seed": 2, + "workflow_id_suffix": "-cap70" + } +] diff --git a/bench/arms/cheap-checks.json b/bench/arms/cheap-checks.json new file mode 100644 index 000000000..9c4b6dbbb --- /dev/null +++ b/bench/arms/cheap-checks.json @@ -0,0 +1,90 @@ +[ + { + "name": "cheap-a-s1", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6 + }, + "seed": 1166, + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "cheap-b-s1", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "guide_noise": 0.4 + }, + "seed": 1166, + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "cheap-c-s1", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "anchor": "segment_1" + }, + "seed": 1166, + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "cheap-a-s2", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "segments": 6 + }, + "seed": 1166, + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "cheap-b-s2", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "segments": 6, + "guide_noise": 0.4 + }, + "seed": 1166, + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "cheap-c-s2", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "segments": 6, + "anchor": "segment_1" + }, + "seed": 1166, + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + } +] \ No newline at end of file diff --git a/bench/arms/check-cut-ratio.json b/bench/arms/check-cut-ratio.json new file mode 100644 index 000000000..a7b27353e --- /dev/null +++ b/bench/arms/check-cut-ratio.json @@ -0,0 +1,44 @@ +[ + { + "name": "chk-s3-seg4-broken-cut", + "scene": "s3-dialogue-walk", + "workflow": "s3-dialogue-walk.json", + "arguments": { + "segments": 6, + "prompts": [ + "For the target video, at 0.00 seconds into the target video, (from [Shot 1]) is fully referenced.\n\nintegrated_multimodal_description: [Shot 1] Cinematic, live-action, one continuous shot. The shot opens exactly on : a woman in her early thirties with shoulder-length dark hair, in a long olive coat, walks toward the camera down a quiet cobblestone street at golden hour. She walks toward the camera down the cobblestone street, slow and steady, glancing up at the lamps. The woman (S1), a warm, low female alto voice, calm and unhurried, says [English] I told myself I would not come back here. The camera holds a steady medium shot at eye level, keeping pace with her as she walks.\n\noverall_soundscape: Soft footsteps on cobblestones, a faint breeze, distant street ambience.\n\nnon_diegetic_music: None.", + "For the target video, at 0.00 seconds into the target video, (from [Shot 1]) is fully referenced.\n\nintegrated_multimodal_description: [Shot 1] Cinematic, live-action, one continuous shot. The shot opens exactly on : a woman in her early thirties with shoulder-length dark hair, in a long olive coat, walks toward the camera down a quiet cobblestone street at golden hour. Her eyes move along the shopfronts as she walks on, the long olive coat swinging with each step. The woman (S1), a warm, low female alto voice, calm and unhurried, says [English] But the street looks exactly the way I left it. The camera holds a steady medium shot at eye level, keeping pace with her as she walks.\n\noverall_soundscape: Soft footsteps on cobblestones, a faint breeze, distant street ambience.\n\nnon_diegetic_music: None.", + "For the target video, at 0.00 seconds into the target video, (from [Shot 1]) is fully referenced.\n\nintegrated_multimodal_description: [Shot 1] Cinematic, live-action, one continuous shot. The shot opens exactly on : a woman in her early thirties with shoulder-length dark hair, in a long olive coat, walks toward the camera down a quiet cobblestone street at golden hour. She slows for a moment, looks down at the stones, then lifts her chin and walks on. The woman (S1), a warm, low female alto voice, calm and unhurried, says [English] Same lamps, same crooked stones, same bakery smell. The camera holds a steady medium shot at eye level, keeping pace with her as she walks.\n\noverall_soundscape: Soft footsteps on cobblestones, a faint breeze, distant street ambience.\n\nnon_diegetic_music: None.", + "For the target video, at 0.00 seconds into the target video, (from [Shot 1]) is fully referenced.\n\nintegrated_multimodal_description: [Shot 1] Cinematic, live-action. The shot opens exactly on , then at 1.00 seconds it cuts hard to a completely different place: a deserted harbour at midnight in pouring rain, lit only by harsh blue-green neon signs reflected in black water. The woman is gone. The camera hangs high above the harbour looking straight down at the dark water and the rain-streaked wooden pier, and swings fast from left to right across moored fishing boats. A distant male voice, rough and loud, shouts [English] Funny how a place waits for you without asking.\n\noverall_soundscape: Heavy rain hammering on water and wood, a ship's horn, buzzing neon.\n\nnon_diegetic_music: None.", + "For the target video, at 0.00 seconds into the target video, (from [Shot 1]) is fully referenced.\n\nintegrated_multimodal_description: [Shot 1] Cinematic, live-action, one continuous shot. The shot opens exactly on : a woman in her early thirties with shoulder-length dark hair, in a long olive coat, walks toward the camera down a quiet cobblestone street at golden hour. Her expression softens and her pace eases, the golden light warm along her shoulder. The woman (S1), a warm, low female alto voice, calm and unhurried, says [English] Maybe that is what I was afraid of. The camera holds a steady medium shot at eye level, keeping pace with her as she walks.\n\noverall_soundscape: Soft footsteps on cobblestones, a faint breeze, distant street ambience.\n\nnon_diegetic_music: None.", + "For the target video, at 0.00 seconds into the target video, (from [Shot 1]) is fully referenced.\n\nintegrated_multimodal_description: [Shot 1] Cinematic, live-action, one continuous shot. The shot opens exactly on : a woman in her early thirties with shoulder-length dark hair, in a long olive coat, walks toward the camera down a quiet cobblestone street at golden hour. She squares her shoulders and walks on, the camera keeping her centered in frame. The woman (S1), a warm, low female alto voice, calm and unhurried, says [English] Well. Let's see if the door still sticks. The camera holds a steady medium shot at eye level, keeping pace with her as she walks.\n\noverall_soundscape: Soft footsteps on cobblestones, a faint breeze, distant street ambience.\n\nnon_diegetic_music: None." + ], + "check": true, + "retries": 1, + "keep_segments": true + }, + "seed": 1166, + "workflow_id_suffix": "-chk2" + }, + { + "name": "chk-s3-seg4-broken-r3", + "scene": "s3-dialogue-walk", + "workflow": "s3-dialogue-walk.json", + "arguments": { + "segments": 6, + "prompts": [ + "For the target video, at 0.00 seconds into the target video, (from [Shot 1]) is fully referenced.\n\nintegrated_multimodal_description: [Shot 1] Cinematic, live-action, one continuous shot. The shot opens exactly on : a woman in her early thirties with shoulder-length dark hair, in a long olive coat, walks toward the camera down a quiet cobblestone street at golden hour. She walks toward the camera down the cobblestone street, slow and steady, glancing up at the lamps. The woman (S1), a warm, low female alto voice, calm and unhurried, says [English] I told myself I would not come back here. The camera holds a steady medium shot at eye level, keeping pace with her as she walks.\n\noverall_soundscape: Soft footsteps on cobblestones, a faint breeze, distant street ambience.\n\nnon_diegetic_music: None.", + "For the target video, at 0.00 seconds into the target video, (from [Shot 1]) is fully referenced.\n\nintegrated_multimodal_description: [Shot 1] Cinematic, live-action, one continuous shot. The shot opens exactly on : a woman in her early thirties with shoulder-length dark hair, in a long olive coat, walks toward the camera down a quiet cobblestone street at golden hour. Her eyes move along the shopfronts as she walks on, the long olive coat swinging with each step. The woman (S1), a warm, low female alto voice, calm and unhurried, says [English] But the street looks exactly the way I left it. The camera holds a steady medium shot at eye level, keeping pace with her as she walks.\n\noverall_soundscape: Soft footsteps on cobblestones, a faint breeze, distant street ambience.\n\nnon_diegetic_music: None.", + "For the target video, at 0.00 seconds into the target video, (from [Shot 1]) is fully referenced.\n\nintegrated_multimodal_description: [Shot 1] Cinematic, live-action, one continuous shot. The shot opens exactly on : a woman in her early thirties with shoulder-length dark hair, in a long olive coat, walks toward the camera down a quiet cobblestone street at golden hour. She slows for a moment, looks down at the stones, then lifts her chin and walks on. The woman (S1), a warm, low female alto voice, calm and unhurried, says [English] Same lamps, same crooked stones, same bakery smell. The camera holds a steady medium shot at eye level, keeping pace with her as she walks.\n\noverall_soundscape: Soft footsteps on cobblestones, a faint breeze, distant street ambience.\n\nnon_diegetic_music: None.", + "For the target video, at 0.00 seconds into the target video, (from [Shot 1]) is fully referenced.\n\nintegrated_multimodal_description: [Shot 1] Cinematic, live-action. The shot opens exactly on , then at 1.00 seconds it cuts hard to a completely different place: a deserted harbour at midnight in pouring rain, lit only by harsh blue-green neon signs reflected in black water. The woman is gone. The camera hangs high above the harbour looking straight down at the dark water and the rain-streaked wooden pier, and swings fast from left to right across moored fishing boats. A distant male voice, rough and loud, shouts [English] Funny how a place waits for you without asking.\n\noverall_soundscape: Heavy rain hammering on water and wood, a ship's horn, buzzing neon.\n\nnon_diegetic_music: None.", + "For the target video, at 0.00 seconds into the target video, (from [Shot 1]) is fully referenced.\n\nintegrated_multimodal_description: [Shot 1] Cinematic, live-action, one continuous shot. The shot opens exactly on : a woman in her early thirties with shoulder-length dark hair, in a long olive coat, walks toward the camera down a quiet cobblestone street at golden hour. Her expression softens and her pace eases, the golden light warm along her shoulder. The woman (S1), a warm, low female alto voice, calm and unhurried, says [English] Maybe that is what I was afraid of. The camera holds a steady medium shot at eye level, keeping pace with her as she walks.\n\noverall_soundscape: Soft footsteps on cobblestones, a faint breeze, distant street ambience.\n\nnon_diegetic_music: None.", + "For the target video, at 0.00 seconds into the target video, (from [Shot 1]) is fully referenced.\n\nintegrated_multimodal_description: [Shot 1] Cinematic, live-action, one continuous shot. The shot opens exactly on : a woman in her early thirties with shoulder-length dark hair, in a long olive coat, walks toward the camera down a quiet cobblestone street at golden hour. She squares her shoulders and walks on, the camera keeping her centered in frame. The woman (S1), a warm, low female alto voice, calm and unhurried, says [English] Well. Let's see if the door still sticks. The camera holds a steady medium shot at eye level, keeping pace with her as she walks.\n\noverall_soundscape: Soft footsteps on cobblestones, a faint breeze, distant street ambience.\n\nnon_diegetic_music: None." + ], + "check": true, + "retries": 3, + "keep_segments": true + }, + "seed": 1166, + "workflow_id_suffix": "-chk3" + } +] diff --git a/bench/arms/check.json b/bench/arms/check.json new file mode 100644 index 000000000..1ac243f6e --- /dev/null +++ b/bench/arms/check.json @@ -0,0 +1,45 @@ +[ + { + "name": "chk-s3-seg4-broken", + "scene": "s3-dialogue-walk", + "workflow": "s3-dialogue-walk.json", + "arguments": { + "segments": 6, + "prompts": [ + "For the target video, at 0.00 seconds into the target video, (from [Shot 1]) is fully referenced.\n\nintegrated_multimodal_description: [Shot 1] Cinematic, live-action, one continuous shot. The shot opens exactly on : a woman in her early thirties with shoulder-length dark hair, in a long olive coat, walks toward the camera down a quiet cobblestone street at golden hour. She walks toward the camera down the cobblestone street, slow and steady, glancing up at the lamps. The woman (S1), a warm, low female alto voice, calm and unhurried, says [English] I told myself I would not come back here. The camera holds a steady medium shot at eye level, keeping pace with her as she walks.\n\noverall_soundscape: Soft footsteps on cobblestones, a faint breeze, distant street ambience.\n\nnon_diegetic_music: None.", + "For the target video, at 0.00 seconds into the target video, (from [Shot 1]) is fully referenced.\n\nintegrated_multimodal_description: [Shot 1] Cinematic, live-action, one continuous shot. The shot opens exactly on : a woman in her early thirties with shoulder-length dark hair, in a long olive coat, walks toward the camera down a quiet cobblestone street at golden hour. Her eyes move along the shopfronts as she walks on, the long olive coat swinging with each step. The woman (S1), a warm, low female alto voice, calm and unhurried, says [English] But the street looks exactly the way I left it. The camera holds a steady medium shot at eye level, keeping pace with her as she walks.\n\noverall_soundscape: Soft footsteps on cobblestones, a faint breeze, distant street ambience.\n\nnon_diegetic_music: None.", + "For the target video, at 0.00 seconds into the target video, (from [Shot 1]) is fully referenced.\n\nintegrated_multimodal_description: [Shot 1] Cinematic, live-action, one continuous shot. The shot opens exactly on : a woman in her early thirties with shoulder-length dark hair, in a long olive coat, walks toward the camera down a quiet cobblestone street at golden hour. She slows for a moment, looks down at the stones, then lifts her chin and walks on. The woman (S1), a warm, low female alto voice, calm and unhurried, says [English] Same lamps, same crooked stones, same bakery smell. The camera holds a steady medium shot at eye level, keeping pace with her as she walks.\n\noverall_soundscape: Soft footsteps on cobblestones, a faint breeze, distant street ambience.\n\nnon_diegetic_music: None.", + "For the target video, at 0.00 seconds into the target video, (from [Shot 1]) is fully referenced.\n\nintegrated_multimodal_description: [Shot 1] Cinematic, live-action. The shot opens exactly on , then at 1.00 seconds it cuts hard to a completely different place: a deserted harbour at midnight in pouring rain, lit only by harsh blue-green neon signs reflected in black water. The woman is gone. The camera hangs high above the harbour looking straight down at the dark water and the rain-streaked wooden pier, and swings fast from left to right across moored fishing boats. A distant male voice, rough and loud, shouts [English] Funny how a place waits for you without asking.\n\noverall_soundscape: Heavy rain hammering on water and wood, a ship's horn, buzzing neon.\n\nnon_diegetic_music: None.", + "For the target video, at 0.00 seconds into the target video, (from [Shot 1]) is fully referenced.\n\nintegrated_multimodal_description: [Shot 1] Cinematic, live-action, one continuous shot. The shot opens exactly on : a woman in her early thirties with shoulder-length dark hair, in a long olive coat, walks toward the camera down a quiet cobblestone street at golden hour. Her expression softens and her pace eases, the golden light warm along her shoulder. The woman (S1), a warm, low female alto voice, calm and unhurried, says [English] Maybe that is what I was afraid of. The camera holds a steady medium shot at eye level, keeping pace with her as she walks.\n\noverall_soundscape: Soft footsteps on cobblestones, a faint breeze, distant street ambience.\n\nnon_diegetic_music: None.", + "For the target video, at 0.00 seconds into the target video, (from [Shot 1]) is fully referenced.\n\nintegrated_multimodal_description: [Shot 1] Cinematic, live-action, one continuous shot. The shot opens exactly on : a woman in her early thirties with shoulder-length dark hair, in a long olive coat, walks toward the camera down a quiet cobblestone street at golden hour. She squares her shoulders and walks on, the camera keeping her centered in frame. The woman (S1), a warm, low female alto voice, calm and unhurried, says [English] Well. Let's see if the door still sticks. The camera holds a steady medium shot at eye level, keeping pace with her as she walks.\n\noverall_soundscape: Soft footsteps on cobblestones, a faint breeze, distant street ambience.\n\nnon_diegetic_music: None." + ], + "check": true, + "retries": 1, + "keep_segments": true + }, + "seed": 1166, + "workflow_id_suffix": "-chk" + }, + { + "name": "chk-s1-fp", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "check": true, + "retries": 0 + }, + "seed": 1166, + "workflow_id_suffix": "-chk" + }, + { + "name": "chk-s2-fp", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "check": true, + "retries": 0 + }, + "seed": 1166, + "workflow_id_suffix": "-chk" + } +] diff --git a/bench/arms/counters-controls.json b/bench/arms/counters-controls.json new file mode 100644 index 000000000..ec2ac3a01 --- /dev/null +++ b/bench/arms/counters-controls.json @@ -0,0 +1,152 @@ +[ + { + "name": "ctl-guide-s1-s1166", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide" + }, + "seed": 1166, + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "ctl-guide-s1-s2", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide" + }, + "seed": 2, + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "ctl-lastframe-s1-s1166", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "last_frame" + }, + "seed": 1166, + "moments": [ + "frame:62", + "frame:307", + "frame:673" + ] + }, + { + "name": "ctl-lastframe-s1-s2", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "last_frame" + }, + "seed": 2, + "moments": [ + "frame:62", + "frame:307", + "frame:673" + ] + }, + { + "name": "ctl-guide-s2-s1166", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "segments": 6, + "continuity": "guide" + }, + "seed": 1166, + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "ctl-guide-s2-s2", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "segments": 6, + "continuity": "guide" + }, + "seed": 2, + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "ctl-lastframe-s2-s1166", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "segments": 6, + "continuity": "last_frame" + }, + "seed": 1166, + "moments": [ + "frame:62", + "frame:307", + "frame:673" + ] + }, + { + "name": "ctl-lastframe-s2-s2", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "segments": 6, + "continuity": "last_frame" + }, + "seed": 2, + "moments": [ + "frame:62", + "frame:307", + "frame:673" + ] + }, + { + "name": "ctl-guide-s3-s1166", + "scene": "s3-dialogue-walk", + "workflow": "s3-dialogue-walk.json", + "arguments": { + "segments": 6, + "continuity": "guide" + }, + "seed": 1166, + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "ctl-guide-s3-s2", + "scene": "s3-dialogue-walk", + "workflow": "s3-dialogue-walk.json", + "arguments": { + "segments": 6, + "continuity": "guide" + }, + "seed": 2, + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + } +] diff --git a/bench/arms/counters-round1.json b/bench/arms/counters-round1.json new file mode 100644 index 000000000..7b1a77aea --- /dev/null +++ b/bench/arms/counters-round1.json @@ -0,0 +1,411 @@ +[ + { + "name": "r1-anchor25-s1", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "anchor": "segment_1", + "anchor_strength": 0.25 + }, + "seed": 1166, + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "r1-anchor50-s1", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "anchor": "segment_1", + "anchor_strength": 0.5 + }, + "seed": 1166, + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "r1-anchor75-s1", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "anchor": "segment_1", + "anchor_strength": 0.75 + }, + "seed": 1166, + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "r1-backoff17-s1", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "guide_backoff": 17 + }, + "seed": 1166, + "moments": [ + "frame:62", + "frame:234", + "frame:498" + ] + }, + { + "name": "r1-keystone-s1", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "keystone": "segment_1" + }, + "seed": 1166, + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "r1-lookhold-s1", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "look_hold": true + }, + "seed": 1166, + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "r1-antiburn-s1", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "antiburn_model_name": "Imagineer13/H3-Anti-Burn-LoRA" + }, + "seed": 1166, + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "r1-lf-carrypick-s1", + "scene": "s1-static", + "workflow": "lf-s1-static.json", + "arguments": { + "segments": 6, + "continuity": "last_frame", + "carry_pick_n": 4 + }, + "seed": 1166, + "moments": [ + "frame:62", + "frame:307", + "frame:673" + ] + }, + { + "name": "r1-lf-lookhold-s1", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "last_frame", + "look_hold": true + }, + "seed": 1166, + "moments": [ + "frame:62", + "frame:307", + "frame:673" + ] + }, + { + "name": "r1-anchor25-s2", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "anchor": "segment_1", + "anchor_strength": 0.25 + }, + "seed": 1166, + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "r1-anchor50-s2", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "anchor": "segment_1", + "anchor_strength": 0.5 + }, + "seed": 1166, + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "r1-anchor75-s2", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "anchor": "segment_1", + "anchor_strength": 0.75 + }, + "seed": 1166, + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "r1-backoff17-s2", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "guide_backoff": 17 + }, + "seed": 1166, + "moments": [ + "frame:62", + "frame:234", + "frame:498" + ] + }, + { + "name": "r1-keystone-s2", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "keystone": "segment_1" + }, + "seed": 1166, + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "r1-lookhold-s2", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "look_hold": true + }, + "seed": 1166, + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "r1-antiburn-s2", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "antiburn_model_name": "Imagineer13/H3-Anti-Burn-LoRA" + }, + "seed": 1166, + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "r1-lf-carrypick-s2", + "scene": "s2-arc", + "workflow": "lf-s2-arc.json", + "arguments": { + "segments": 6, + "continuity": "last_frame", + "carry_pick_n": 4 + }, + "seed": 1166, + "moments": [ + "frame:62", + "frame:307", + "frame:673" + ] + }, + { + "name": "r1-lf-lookhold-s2", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "segments": 6, + "continuity": "last_frame", + "look_hold": true + }, + "seed": 1166, + "moments": [ + "frame:62", + "frame:307", + "frame:673" + ] + }, + { + "name": "r1-anchor25-s3", + "scene": "s3-dialogue-walk", + "workflow": "s3-dialogue-walk.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "anchor": "segment_1", + "anchor_strength": 0.25 + }, + "seed": 1166, + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "r1-anchor50-s3", + "scene": "s3-dialogue-walk", + "workflow": "s3-dialogue-walk.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "anchor": "segment_1", + "anchor_strength": 0.5 + }, + "seed": 1166, + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "r1-anchor75-s3", + "scene": "s3-dialogue-walk", + "workflow": "s3-dialogue-walk.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "anchor": "segment_1", + "anchor_strength": 0.75 + }, + "seed": 1166, + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "r1-backoff17-s3", + "scene": "s3-dialogue-walk", + "workflow": "s3-dialogue-walk.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "guide_backoff": 17 + }, + "seed": 1166, + "moments": [ + "frame:62", + "frame:234", + "frame:498" + ] + }, + { + "name": "r1-keystone-s3", + "scene": "s3-dialogue-walk", + "workflow": "s3-dialogue-walk.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "keystone": "segment_1" + }, + "seed": 1166, + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "r1-lookhold-s3", + "scene": "s3-dialogue-walk", + "workflow": "s3-dialogue-walk.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "look_hold": true + }, + "seed": 1166, + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "r1-antiburn-s3", + "scene": "s3-dialogue-walk", + "workflow": "s3-dialogue-walk.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "antiburn_model_name": "Imagineer13/H3-Anti-Burn-LoRA" + }, + "seed": 1166, + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + } +] diff --git a/bench/arms/counters-round2-combos.json b/bench/arms/counters-round2-combos.json new file mode 100644 index 000000000..1e2fc7331 --- /dev/null +++ b/bench/arms/counters-round2-combos.json @@ -0,0 +1,146 @@ +[ + { + "name": "r2-anchor25-keystone-s1-s1166", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "anchor": "segment_1", + "anchor_strength": 0.25, + "keystone": "segment_1" + }, + "seed": 1166, + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "r2-anchor50-keystone-s1-s1166", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "anchor": "segment_1", + "anchor_strength": 0.5, + "keystone": "segment_1" + }, + "seed": 1166, + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "r2-lf-lookhold-anchor25-s1-s1166", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "last_frame", + "look_hold": true, + "anchor": "segment_1", + "anchor_strength": 0.25 + }, + "seed": 1166, + "moments": [ + "frame:62", + "frame:307", + "frame:673" + ] + }, + { + "name": "r2-lf-lookhold-anchor50-s1-s1166", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "last_frame", + "look_hold": true, + "anchor": "segment_1", + "anchor_strength": 0.5 + }, + "seed": 1166, + "moments": [ + "frame:62", + "frame:307", + "frame:673" + ] + }, + { + "name": "r2-anchor25-keystone-s1-s2", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "anchor": "segment_1", + "anchor_strength": 0.25, + "keystone": "segment_1" + }, + "seed": 2, + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "r2-anchor50-keystone-s1-s2", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "anchor": "segment_1", + "anchor_strength": 0.5, + "keystone": "segment_1" + }, + "seed": 2, + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "r2-lf-lookhold-anchor25-s1-s2", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "last_frame", + "look_hold": true, + "anchor": "segment_1", + "anchor_strength": 0.25 + }, + "seed": 2, + "moments": [ + "frame:62", + "frame:307", + "frame:673" + ] + }, + { + "name": "r2-lf-lookhold-anchor50-s1-s2", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "last_frame", + "look_hold": true, + "anchor": "segment_1", + "anchor_strength": 0.5 + }, + "seed": 2, + "moments": [ + "frame:62", + "frame:307", + "frame:673" + ] + } +] diff --git a/bench/arms/counters-round2-confirm.json b/bench/arms/counters-round2-confirm.json new file mode 100644 index 000000000..eeb85decb --- /dev/null +++ b/bench/arms/counters-round2-confirm.json @@ -0,0 +1,100 @@ +[ + { + "name": "ctl-guide-s1-s1166-rerun", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide" + }, + "seed": 1166, + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ], + "workflow_id_suffix": "-r2" + }, + { + "name": "r2-anchor25-s1-seed2", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "anchor": "segment_1", + "anchor_strength": 0.25 + }, + "seed": 2, + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "r2-anchor50-s1-seed2", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "anchor": "segment_1", + "anchor_strength": 0.5 + }, + "seed": 2, + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "r2-keystone-s1-seed2", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "keystone": "segment_1" + }, + "seed": 2, + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "r2-lf-lookhold-s1-seed2", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "last_frame", + "look_hold": true + }, + "seed": 2, + "moments": [ + "frame:62", + "frame:307", + "frame:673" + ] + }, + { + "name": "r2-lf-lookhold-s2-seed2", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "segments": 6, + "continuity": "last_frame", + "look_hold": true + }, + "seed": 2, + "moments": [ + "frame:62", + "frame:307", + "frame:673" + ] + } +] diff --git a/bench/arms/counters-round2-long.json b/bench/arms/counters-round2-long.json new file mode 100644 index 000000000..177d02b66 --- /dev/null +++ b/bench/arms/counters-round2-long.json @@ -0,0 +1,36 @@ +[ + { + "name": "ctl-guide-s1-120s", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "duration_seconds": 120 + }, + "seed": 1166, + "moments": [ + "frame:62", + "frame:1501", + "frame:2868" + ] + }, + { + "name": "r2-anchor50-s1-120s", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "duration_seconds": 120, + "anchor": "segment_1", + "anchor_strength": 0.5 + }, + "seed": 1166, + "moments": [ + "frame:62", + "frame:1501", + "frame:2868" + ] + } +] diff --git a/bench/arms/counters-round3-long.json b/bench/arms/counters-round3-long.json new file mode 100644 index 000000000..0239ff60d --- /dev/null +++ b/bench/arms/counters-round3-long.json @@ -0,0 +1,39 @@ +[ + { + "name": "r3-anchor25-s1-120s", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "duration_seconds": 120, + "anchor": "segment_1", + "anchor_strength": 0.25 + }, + "seed": 1166, + "moments": [ + "frame:62", + "frame:1501", + "frame:2868" + ] + }, + { + "name": "r3-lf-lookhold-anchor25-s1-120s", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "last_frame", + "look_hold": true, + "anchor": "segment_1", + "anchor_strength": 0.25, + "duration_seconds": 120 + }, + "seed": 1166, + "moments": [ + "frame:62", + "frame:1501", + "frame:2868" + ] + } +] \ No newline at end of file diff --git a/bench/arms/keystone-confirm.json b/bench/arms/keystone-confirm.json new file mode 100644 index 000000000..90cb11709 --- /dev/null +++ b/bench/arms/keystone-confirm.json @@ -0,0 +1,62 @@ +[ + { + "name": "ks-pixel-anchor25-s1-120s-s2", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "duration_seconds": 120, + "anchor": "segment_1", + "anchor_strength": 0.25, + "keystone": "segment_1" + }, + "seed": 2, + "workflow_id_suffix": "-ksp", + "moments": [ + "frame:62", + "frame:1501", + "frame:2868" + ] + }, + { + "name": "ks-lc-anchor25-s1-120s-s2", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "duration_seconds": 120, + "anchor": "segment_1", + "anchor_strength": 0.25, + "latent_carry": true, + "keystone": "segment_1" + }, + "seed": 2, + "workflow_id_suffix": "-ksl", + "moments": [ + "frame:62", + "frame:1501", + "frame:2868" + ] + }, + { + "name": "ks-pixel-noanchor-s1-120s-s1166", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "duration_seconds": 120, + "anchor": "none", + "keystone": "segment_1" + }, + "seed": 1166, + "workflow_id_suffix": "-kspna", + "moments": [ + "frame:62", + "frame:1501", + "frame:2868" + ] + } +] diff --git a/bench/arms/keystone-last-noprompt.json b/bench/arms/keystone-last-noprompt.json new file mode 100644 index 000000000..6f87fa205 --- /dev/null +++ b/bench/arms/keystone-last-noprompt.json @@ -0,0 +1,42 @@ +[ + { + "name": "ksr-last-noprompt-s1166", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "duration_seconds": 120, + "anchor": "none", + "keystone": "segment_1", + "keystone_slot": "last" + }, + "seed": 1166, + "workflow_id_suffix": "-ksrln", + "moments": [ + "frame:62", + "frame:1501", + "frame:2868" + ] + }, + { + "name": "ksr-last-noprompt-s2", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "duration_seconds": 120, + "anchor": "none", + "keystone": "segment_1", + "keystone_slot": "last" + }, + "seed": 2, + "workflow_id_suffix": "-ksrln", + "moments": [ + "frame:62", + "frame:1501", + "frame:2868" + ] + } +] diff --git a/bench/arms/keystone-regular.json b/bench/arms/keystone-regular.json new file mode 100644 index 000000000..aa6967795 --- /dev/null +++ b/bench/arms/keystone-regular.json @@ -0,0 +1,195 @@ +[ + { + "name": "ksr-last-s1166", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "duration_seconds": 120, + "anchor": "none", + "keystone": "segment_1", + "keystone_slot": "last", + "prompts": [ + "For the target video, at 0.00 seconds into the target video, (from [Shot 1]) is fully referenced.\n\nintegrated_multimodal_description: [Shot 1] Cinematic, live-action, one continuous static shot. The shot opens exactly on : a young woman in a plain dark grey sweater stands centered in front of a flat painted wall of bold diagonal rainbow stripes with a black circle at upper left and a teal square at lower right. She breathes calmly, blinks, shifts her weight slightly and turns her head a little left and right, staying in place at the center of the frame. The painted wall stays exactly as it is, evenly lit by soft studio light. The camera is locked off on a tripod and never moves.\n\noverall_soundscape: Quiet studio room tone, the soft rustle of a wool sweater, calm breathing.\n\nnon_diegetic_music: None.", + "For the target video, at 0.00 seconds into the target video, (from [Shot 1]) is fully referenced, and at 5.17 seconds into the target video, (from [Shot 1]) is fully referenced.\n\nintegrated_multimodal_description: [Shot 1] Cinematic, live-action, one continuous static shot. The shot opens exactly on : a young woman in a plain dark grey sweater stands centered in front of a flat painted wall of bold diagonal rainbow stripes with a black circle at upper left and a teal square at lower right. She breathes calmly, blinks, shifts her weight slightly and turns her head a little left and right, staying in place at the center of the frame. The painted wall stays exactly as it is, evenly lit by soft studio light. The camera is locked off on a tripod and never moves. The shot ends exactly on : the same woman, wall and framing.\n\noverall_soundscape: Quiet studio room tone, the soft rustle of a wool sweater, calm breathing.\n\nnon_diegetic_music: None." + ] + }, + "seed": 1166, + "workflow_id_suffix": "-ksrl", + "moments": [ + "frame:62", + "frame:240", + "frame:480", + "frame:720", + "frame:960", + "frame:1200", + "frame:1440", + "frame:1680", + "frame:1920", + "frame:2160", + "frame:2400", + "frame:2640", + "frame:2868" + ] + }, + { + "name": "ksr-last-s2", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "duration_seconds": 120, + "anchor": "none", + "keystone": "segment_1", + "keystone_slot": "last", + "prompts": [ + "For the target video, at 0.00 seconds into the target video, (from [Shot 1]) is fully referenced.\n\nintegrated_multimodal_description: [Shot 1] Cinematic, live-action, one continuous static shot. The shot opens exactly on : a young woman in a plain dark grey sweater stands centered in front of a flat painted wall of bold diagonal rainbow stripes with a black circle at upper left and a teal square at lower right. She breathes calmly, blinks, shifts her weight slightly and turns her head a little left and right, staying in place at the center of the frame. The painted wall stays exactly as it is, evenly lit by soft studio light. The camera is locked off on a tripod and never moves.\n\noverall_soundscape: Quiet studio room tone, the soft rustle of a wool sweater, calm breathing.\n\nnon_diegetic_music: None.", + "For the target video, at 0.00 seconds into the target video, (from [Shot 1]) is fully referenced, and at 5.17 seconds into the target video, (from [Shot 1]) is fully referenced.\n\nintegrated_multimodal_description: [Shot 1] Cinematic, live-action, one continuous static shot. The shot opens exactly on : a young woman in a plain dark grey sweater stands centered in front of a flat painted wall of bold diagonal rainbow stripes with a black circle at upper left and a teal square at lower right. She breathes calmly, blinks, shifts her weight slightly and turns her head a little left and right, staying in place at the center of the frame. The painted wall stays exactly as it is, evenly lit by soft studio light. The camera is locked off on a tripod and never moves. The shot ends exactly on : the same woman, wall and framing.\n\noverall_soundscape: Quiet studio room tone, the soft rustle of a wool sweater, calm breathing.\n\nnon_diegetic_music: None." + ] + }, + "seed": 2, + "workflow_id_suffix": "-ksrl", + "moments": [ + "frame:62", + "frame:240", + "frame:480", + "frame:720", + "frame:960", + "frame:1200", + "frame:1440", + "frame:1680", + "frame:1920", + "frame:2160", + "frame:2400", + "frame:2640", + "frame:2868" + ] + }, + { + "name": "ksr-f34-s1166", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "duration_seconds": 120, + "anchor": "none", + "keystone": "segment_1", + "keystone_slot": { + "frame": 34 + } + }, + "seed": 1166, + "workflow_id_suffix": "-ksrf", + "moments": [ + "frame:62", + "frame:240", + "frame:480", + "frame:720", + "frame:960", + "frame:1200", + "frame:1440", + "frame:1680", + "frame:1920", + "frame:2160", + "frame:2400", + "frame:2640", + "frame:2868" + ] + }, + { + "name": "ksr-f34-gate09-s1166", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "duration_seconds": 120, + "anchor": "none", + "keystone": "segment_1", + "keystone_slot": { + "frame": 34, + "enter_below_sigma": 0.9 + } + }, + "seed": 1166, + "workflow_id_suffix": "-ksrg", + "moments": [ + "frame:62", + "frame:240", + "frame:480", + "frame:720", + "frame:960", + "frame:1200", + "frame:1440", + "frame:1680", + "frame:1920", + "frame:2160", + "frame:2400", + "frame:2640", + "frame:2868" + ] + }, + { + "name": "ksr-noise04-s1166", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "duration_seconds": 120, + "anchor": "none", + "keystone": "segment_1", + "guide_noise": 0.4 + }, + "seed": 1166, + "workflow_id_suffix": "-ksrn", + "moments": [ + "frame:62", + "frame:240", + "frame:480", + "frame:720", + "frame:960", + "frame:1200", + "frame:1440", + "frame:1680", + "frame:1920", + "frame:2160", + "frame:2400", + "frame:2640", + "frame:2868" + ] + }, + { + "name": "ksr-g39-s1166", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "duration_seconds": 120, + "anchor": "none", + "keystone": "segment_1", + "guide_frames": 39 + }, + "seed": 1166, + "workflow_id_suffix": "-ksr39", + "moments": [ + "frame:62", + "frame:240", + "frame:480", + "frame:720", + "frame:960", + "frame:1200", + "frame:1440", + "frame:1680", + "frame:1920", + "frame:2160", + "frame:2400", + "frame:2640", + "frame:2868" + ] + } +] diff --git a/bench/arms/lastframe-ctl-clean-rerun.json b/bench/arms/lastframe-ctl-clean-rerun.json new file mode 100644 index 000000000..cb0957fbf --- /dev/null +++ b/bench/arms/lastframe-ctl-clean-rerun.json @@ -0,0 +1,34 @@ +[ + { + "name": "ctl-lastframe-s1-s1166-clean", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "last_frame" + }, + "seed": 1166, + "workflow_id_suffix": "-lfclean1", + "moments": [ + "frame:62", + "frame:307", + "frame:673" + ] + }, + { + "name": "ctl-lastframe-s1-s2-clean", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "last_frame" + }, + "seed": 2, + "workflow_id_suffix": "-lfclean2", + "moments": [ + "frame:62", + "frame:307", + "frame:673" + ] + } +] diff --git a/bench/arms/latent-carry-controls-r36.json b/bench/arms/latent-carry-controls-r36.json new file mode 100644 index 000000000..7c48b3843 --- /dev/null +++ b/bench/arms/latent-carry-controls-r36.json @@ -0,0 +1,53 @@ +[ + { + "name": "r3-anchor25-s1-120s-s1166-r36", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "duration_seconds": 120, + "anchor": "segment_1", + "anchor_strength": 0.25 + }, + "seed": 1166, + "workflow_id_suffix": "-lcctl", + "moments": [ + "frame:62", + "frame:1501", + "frame:2868" + ] + }, + { + "name": "ctl-guide-s2-s1166-r36", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "segments": 6, + "continuity": "guide" + }, + "seed": 1166, + "workflow_id_suffix": "-lcctl", + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "ctl-guide-s2-s2-r36", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "segments": 6, + "continuity": "guide" + }, + "seed": 2, + "workflow_id_suffix": "-lcctl", + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + } +] diff --git a/bench/arms/latent-carry-controls.json b/bench/arms/latent-carry-controls.json new file mode 100644 index 000000000..221c462f1 --- /dev/null +++ b/bench/arms/latent-carry-controls.json @@ -0,0 +1,53 @@ +[ + { + "name": "r3-anchor25-s1-120s-s2", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "duration_seconds": 120, + "anchor": "segment_1", + "anchor_strength": 0.25 + }, + "seed": 2, + "workflow_id_suffix": "-lcctl", + "moments": [ + "frame:62", + "frame:1501", + "frame:2868" + ] + }, + { + "name": "ctl-guide-s2-s1166-r36", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "segments": 6, + "continuity": "guide" + }, + "seed": 1166, + "workflow_id_suffix": "-lcctl", + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "ctl-guide-s2-s2-r36", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "segments": 6, + "continuity": "guide" + }, + "seed": 2, + "workflow_id_suffix": "-lcctl", + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + } +] diff --git a/bench/arms/latent-carry-noanchor.json b/bench/arms/latent-carry-noanchor.json new file mode 100644 index 000000000..7a3218c00 --- /dev/null +++ b/bench/arms/latent-carry-noanchor.json @@ -0,0 +1,21 @@ +[ + { + "name": "lc-noanchor-s1-120s-s1166", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "duration_seconds": 120, + "anchor": "none", + "latent_carry": true + }, + "seed": 1166, + "workflow_id_suffix": "-lcna", + "moments": [ + "frame:62", + "frame:1501", + "frame:2868" + ] + } +] diff --git a/bench/arms/latent-carry.json b/bench/arms/latent-carry.json new file mode 100644 index 000000000..7d8efcbe8 --- /dev/null +++ b/bench/arms/latent-carry.json @@ -0,0 +1,117 @@ +[ + { + "name": "lc-anchor25-s1-120s-s1166", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "duration_seconds": 120, + "anchor": "segment_1", + "anchor_strength": 0.25, + "latent_carry": true + }, + "seed": 1166, + "workflow_id_suffix": "-lc", + "moments": [ + "frame:62", + "frame:1501", + "frame:2868" + ] + }, + { + "name": "lc-anchor25-s1-120s-s2", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "duration_seconds": 120, + "anchor": "segment_1", + "anchor_strength": 0.25, + "latent_carry": true + }, + "seed": 2, + "workflow_id_suffix": "-lc", + "moments": [ + "frame:62", + "frame:1501", + "frame:2868" + ] + }, + { + "name": "lc-guide-s2-s1166", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "latent_carry": true + }, + "seed": 1166, + "workflow_id_suffix": "-lc", + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "lc-guide-s2-s2", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "latent_carry": true + }, + "seed": 2, + "workflow_id_suffix": "-lc", + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "ks-pixel-anchor25-s1-120s-s1166", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "duration_seconds": 120, + "anchor": "segment_1", + "anchor_strength": 0.25, + "keystone": "segment_1" + }, + "seed": 1166, + "workflow_id_suffix": "-ksp", + "moments": [ + "frame:62", + "frame:1501", + "frame:2868" + ] + }, + { + "name": "ks-lc-anchor25-s1-120s-s1166", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "duration_seconds": 120, + "anchor": "segment_1", + "anchor_strength": 0.25, + "latent_carry": true, + "keystone": "segment_1" + }, + "seed": 1166, + "workflow_id_suffix": "-ksl", + "moments": [ + "frame:62", + "frame:1501", + "frame:2868" + ] + } +] diff --git a/bench/arms/ltx-arc.json b/bench/arms/ltx-arc.json new file mode 100644 index 000000000..b5c3278f6 --- /dev/null +++ b/bench/arms/ltx-arc.json @@ -0,0 +1,58 @@ +[ + { + "name": "ltx-arc-ctl-s1166", + "scene": "s2-arc", + "workflow": "ltx-arc.json", + "arguments": {}, + "seed": 1166, + "workflow_id_suffix": "-mps", + "moments": [ + "frame:60", + "frame:180", + "frame:299" + ] + }, + { + "name": "ltx-arc-lookhold-s1166", + "scene": "s2-arc", + "workflow": "ltx-arc.json", + "arguments": { + "look_hold": true + }, + "seed": 1166, + "workflow_id_suffix": "-mps", + "moments": [ + "frame:60", + "frame:180", + "frame:299" + ] + }, + { + "name": "ltx-arc-ctl-s2", + "scene": "s2-arc", + "workflow": "ltx-arc.json", + "arguments": {}, + "seed": 2, + "workflow_id_suffix": "-mps", + "moments": [ + "frame:60", + "frame:180", + "frame:299" + ] + }, + { + "name": "ltx-arc-lookhold-s2", + "scene": "s2-arc", + "workflow": "ltx-arc.json", + "arguments": { + "look_hold": true + }, + "seed": 2, + "workflow_id_suffix": "-mps", + "moments": [ + "frame:60", + "frame:180", + "frame:299" + ] + } +] diff --git a/bench/arms/ltx-long-probe.json b/bench/arms/ltx-long-probe.json new file mode 100644 index 000000000..48086be2a --- /dev/null +++ b/bench/arms/ltx-long-probe.json @@ -0,0 +1,10 @@ +[ + { + "name": "ltx-ctl-s1166", + "scene": "s5-ltx-action", + "workflow": "s5-ltx-action.json", + "arguments": {}, + "seed": 1166, + "workflow_id_suffix": "-mps" + } +] diff --git a/bench/arms/ltx-long.json b/bench/arms/ltx-long.json new file mode 100644 index 000000000..7868cd842 --- /dev/null +++ b/bench/arms/ltx-long.json @@ -0,0 +1,142 @@ +[ + { + "name": "ltx-ctl-s1166-rerun", + "scene": "s5-ltx-action", + "workflow": "s5-ltx-action.json", + "arguments": {}, + "seed": 1166, + "workflow_id_suffix": "-mps-r" + }, + { + "name": "ltx-ctl-s2", + "scene": "s5-ltx-action", + "workflow": "s5-ltx-action.json", + "arguments": {}, + "seed": 2, + "workflow_id_suffix": "-mps" + }, + { + "name": "ltx-extend", + "scene": "s5-ltx-action", + "workflow": "s5-ltx-extend.json", + "arguments": {}, + "seed": 1166, + "workflow_id_suffix": "-mps" + }, + { + "name": "ltx-lookhold", + "scene": "s5-ltx-action", + "workflow": "s5-ltx-action.json", + "arguments": { + "look_hold": true + }, + "seed": 1166, + "workflow_id_suffix": "-mps" + }, + { + "name": "ltx-carrypick4", + "scene": "s5-ltx-action", + "workflow": "lf-s5-ltx-action.json", + "arguments": { + "carry_pick_n": 4 + }, + "seed": 1166, + "workflow_id_suffix": "-mps" + }, + { + "name": "ltx-check-r1", + "scene": "s5-ltx-action", + "workflow": "s5-ltx-action.json", + "arguments": { + "check": true, + "retries": 1 + }, + "seed": 1166, + "workflow_id_suffix": "-mps-chk" + }, + { + "name": "ltx-ctl-36s", + "scene": "s5-ltx-action", + "workflow": "s5-ltx-action.json", + "arguments": { + "duration_seconds": 36 + }, + "seed": 1166, + "workflow_id_suffix": "-mps", + "moments": [ + "frame:60", + "frame:430", + "frame:850" + ] + }, + { + "name": "ltx-extend-s2", + "scene": "s5-ltx-action", + "workflow": "s5-ltx-extend.json", + "arguments": {}, + "seed": 2, + "workflow_id_suffix": "-mps" + }, + { + "name": "ltx-lookhold-s2", + "scene": "s5-ltx-action", + "workflow": "s5-ltx-action.json", + "arguments": { + "look_hold": true + }, + "seed": 2, + "workflow_id_suffix": "-mps" + }, + { + "name": "ltx-carrypick4-s2", + "scene": "s5-ltx-action", + "workflow": "lf-s5-ltx-action.json", + "arguments": { + "carry_pick_n": 4 + }, + "seed": 2, + "workflow_id_suffix": "-mps" + }, + { + "name": "ltx-check-r1-s2", + "scene": "s5-ltx-action", + "workflow": "s5-ltx-action.json", + "arguments": { + "check": true, + "retries": 1 + }, + "seed": 2, + "workflow_id_suffix": "-mps-chk" + }, + { + "name": "ltx-ctl-36s-s2", + "scene": "s5-ltx-action", + "workflow": "s5-ltx-action.json", + "arguments": { + "duration_seconds": 36 + }, + "seed": 2, + "workflow_id_suffix": "-mps", + "moments": [ + "frame:60", + "frame:430", + "frame:850" + ] + }, + { + "name": "ltx-lookhold-36s", + "scene": "s5-ltx-action", + "workflow": "s5-ltx-action.json", + "arguments": { + "duration_seconds": 36, + "look_hold": true + }, + "seed": 1166, + "workflow_id_suffix": "-mps", + "moments": [ + "frame:60", + "frame:430", + "frame:850" + ] + } +] diff --git a/bench/arms/music-snap.json b/bench/arms/music-snap.json new file mode 100644 index 000000000..289ed6636 --- /dev/null +++ b/bench/arms/music-snap.json @@ -0,0 +1,16 @@ +[ + { + "name": "s6-music-snap8-335s-s1166", + "scene": "s6-narration", + "workflow": "music-long-s6-full.json", + "arguments": {}, + "seed": 1166 + }, + { + "name": "s6-music-snap8-335s-s1167", + "scene": "s6-narration", + "workflow": "music-long-s6-full.json", + "arguments": {}, + "seed": 1167 + } +] diff --git a/bench/arms/narrated-arc-latent.json b/bench/arms/narrated-arc-latent.json new file mode 100644 index 000000000..1b6e7707f --- /dev/null +++ b/bench/arms/narrated-arc-latent.json @@ -0,0 +1,136 @@ +[ + { + "name": "narrated-arc-latent-s1", + "scene": "s3-dialogue-walk", + "workflow": "narrated-take.json", + "arguments": { + "stop_after": 12, + "shot": "walk_keyframed", + "image": "asset:uploads/bench/keyframes-v3/kf00.jpg", + "keyframes": [ + "asset:uploads/bench/keyframes-v3/kf01.jpg", + "asset:uploads/bench/keyframes-v3/kf02.jpg", + "asset:uploads/bench/keyframes-v3/kf03.jpg", + "asset:uploads/bench/keyframes-v3/kf04.jpg", + "asset:uploads/bench/keyframes-v3/kf05.jpg", + "asset:uploads/bench/keyframes-v3/kf06.jpg", + "asset:uploads/bench/keyframes-v3/kf07.jpg", + "asset:uploads/bench/keyframes-v3/kf08.jpg", + "asset:uploads/bench/keyframes-v3/kf09.jpg", + "asset:uploads/bench/keyframes-v3/kf10.jpg", + "asset:uploads/bench/keyframes-v3/kf11.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg" + ], + "experimental_latent_carry": true, + "experimental_capstone": { + "enter_below_sigma": 0.7 + } + }, + "seed": 1, + "workflow_id_suffix": "-narrated-arc-latent", + "moments": [ + "frame:48", + "frame:720", + "frame:1440", + "frame:2040" + ] + }, + { + "name": "narrated-arc-latent-s2", + "scene": "s3-dialogue-walk", + "workflow": "narrated-take.json", + "arguments": { + "stop_after": 12, + "shot": "walk_keyframed", + "image": "asset:uploads/bench/keyframes-v3/kf00.jpg", + "keyframes": [ + "asset:uploads/bench/keyframes-v3/kf01.jpg", + "asset:uploads/bench/keyframes-v3/kf02.jpg", + "asset:uploads/bench/keyframes-v3/kf03.jpg", + "asset:uploads/bench/keyframes-v3/kf04.jpg", + "asset:uploads/bench/keyframes-v3/kf05.jpg", + "asset:uploads/bench/keyframes-v3/kf06.jpg", + "asset:uploads/bench/keyframes-v3/kf07.jpg", + "asset:uploads/bench/keyframes-v3/kf08.jpg", + "asset:uploads/bench/keyframes-v3/kf09.jpg", + "asset:uploads/bench/keyframes-v3/kf10.jpg", + "asset:uploads/bench/keyframes-v3/kf11.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg" + ], + "experimental_latent_carry": true, + "experimental_capstone": { + "enter_below_sigma": 0.7 + } + }, + "seed": 2, + "workflow_id_suffix": "-narrated-arc-latent", + "moments": [ + "frame:48", + "frame:720", + "frame:1440", + "frame:2040" + ] + } +] diff --git a/bench/arms/narrated-arc.json b/bench/arms/narrated-arc.json new file mode 100644 index 000000000..05cda49b1 --- /dev/null +++ b/bench/arms/narrated-arc.json @@ -0,0 +1,128 @@ +[ + { + "name": "narrated-arc-s1", + "scene": "s3-dialogue-walk", + "workflow": "narrated-take.json", + "arguments": { + "stop_after": 12, + "shot": "walk_keyframed", + "image": "asset:uploads/bench/keyframes-v3/kf00.jpg", + "keyframes": [ + "asset:uploads/bench/keyframes-v3/kf01.jpg", + "asset:uploads/bench/keyframes-v3/kf02.jpg", + "asset:uploads/bench/keyframes-v3/kf03.jpg", + "asset:uploads/bench/keyframes-v3/kf04.jpg", + "asset:uploads/bench/keyframes-v3/kf05.jpg", + "asset:uploads/bench/keyframes-v3/kf06.jpg", + "asset:uploads/bench/keyframes-v3/kf07.jpg", + "asset:uploads/bench/keyframes-v3/kf08.jpg", + "asset:uploads/bench/keyframes-v3/kf09.jpg", + "asset:uploads/bench/keyframes-v3/kf10.jpg", + "asset:uploads/bench/keyframes-v3/kf11.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg" + ] + }, + "seed": 1, + "workflow_id_suffix": "-narrated-arc", + "moments": [ + "frame:48", + "frame:720", + "frame:1440", + "frame:2040" + ] + }, + { + "name": "narrated-arc-s2", + "scene": "s3-dialogue-walk", + "workflow": "narrated-take.json", + "arguments": { + "stop_after": 12, + "shot": "walk_keyframed", + "image": "asset:uploads/bench/keyframes-v3/kf00.jpg", + "keyframes": [ + "asset:uploads/bench/keyframes-v3/kf01.jpg", + "asset:uploads/bench/keyframes-v3/kf02.jpg", + "asset:uploads/bench/keyframes-v3/kf03.jpg", + "asset:uploads/bench/keyframes-v3/kf04.jpg", + "asset:uploads/bench/keyframes-v3/kf05.jpg", + "asset:uploads/bench/keyframes-v3/kf06.jpg", + "asset:uploads/bench/keyframes-v3/kf07.jpg", + "asset:uploads/bench/keyframes-v3/kf08.jpg", + "asset:uploads/bench/keyframes-v3/kf09.jpg", + "asset:uploads/bench/keyframes-v3/kf10.jpg", + "asset:uploads/bench/keyframes-v3/kf11.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg", + "asset:uploads/bench/keyframes-v3/kf12.jpg" + ] + }, + "seed": 2, + "workflow_id_suffix": "-narrated-arc", + "moments": [ + "frame:48", + "frame:720", + "frame:1440", + "frame:2040" + ] + } +] diff --git a/bench/arms/narrated-arc4-latent.json b/bench/arms/narrated-arc4-latent.json new file mode 100644 index 000000000..5f1232523 --- /dev/null +++ b/bench/arms/narrated-arc4-latent.json @@ -0,0 +1,136 @@ +[ + { + "name": "narrated-arc4-latent-s1", + "scene": "s3-dialogue-walk", + "workflow": "narrated-take.json", + "arguments": { + "stop_after": 12, + "shot": "walk_keyframed", + "image": "asset:uploads/bench/keyframes-v4/kf00.jpg", + "keyframes": [ + "asset:uploads/bench/keyframes-v4/kf01.jpg", + "asset:uploads/bench/keyframes-v4/kf02.jpg", + "asset:uploads/bench/keyframes-v4/kf03.jpg", + "asset:uploads/bench/keyframes-v4/kf04.jpg", + "asset:uploads/bench/keyframes-v4/kf05.jpg", + "asset:uploads/bench/keyframes-v4/kf06.jpg", + "asset:uploads/bench/keyframes-v4/kf07.jpg", + "asset:uploads/bench/keyframes-v4/kf08.jpg", + "asset:uploads/bench/keyframes-v4/kf09.jpg", + "asset:uploads/bench/keyframes-v4/kf10.jpg", + "asset:uploads/bench/keyframes-v4/kf11.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg" + ], + "experimental_latent_carry": true, + "experimental_capstone": { + "enter_below_sigma": 0.7 + } + }, + "seed": 1, + "workflow_id_suffix": "-narrated-arc4-latent", + "moments": [ + "frame:48", + "frame:720", + "frame:1440", + "frame:2040" + ] + }, + { + "name": "narrated-arc4-latent-s2", + "scene": "s3-dialogue-walk", + "workflow": "narrated-take.json", + "arguments": { + "stop_after": 12, + "shot": "walk_keyframed", + "image": "asset:uploads/bench/keyframes-v4/kf00.jpg", + "keyframes": [ + "asset:uploads/bench/keyframes-v4/kf01.jpg", + "asset:uploads/bench/keyframes-v4/kf02.jpg", + "asset:uploads/bench/keyframes-v4/kf03.jpg", + "asset:uploads/bench/keyframes-v4/kf04.jpg", + "asset:uploads/bench/keyframes-v4/kf05.jpg", + "asset:uploads/bench/keyframes-v4/kf06.jpg", + "asset:uploads/bench/keyframes-v4/kf07.jpg", + "asset:uploads/bench/keyframes-v4/kf08.jpg", + "asset:uploads/bench/keyframes-v4/kf09.jpg", + "asset:uploads/bench/keyframes-v4/kf10.jpg", + "asset:uploads/bench/keyframes-v4/kf11.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg" + ], + "experimental_latent_carry": true, + "experimental_capstone": { + "enter_below_sigma": 0.7 + } + }, + "seed": 2, + "workflow_id_suffix": "-narrated-arc4-latent", + "moments": [ + "frame:48", + "frame:720", + "frame:1440", + "frame:2040" + ] + } +] diff --git a/bench/arms/narrated-arc4.json b/bench/arms/narrated-arc4.json new file mode 100644 index 000000000..4f0913000 --- /dev/null +++ b/bench/arms/narrated-arc4.json @@ -0,0 +1,128 @@ +[ + { + "name": "narrated-arc4-s1", + "scene": "s3-dialogue-walk", + "workflow": "narrated-take.json", + "arguments": { + "stop_after": 12, + "shot": "walk_keyframed", + "image": "asset:uploads/bench/keyframes-v4/kf00.jpg", + "keyframes": [ + "asset:uploads/bench/keyframes-v4/kf01.jpg", + "asset:uploads/bench/keyframes-v4/kf02.jpg", + "asset:uploads/bench/keyframes-v4/kf03.jpg", + "asset:uploads/bench/keyframes-v4/kf04.jpg", + "asset:uploads/bench/keyframes-v4/kf05.jpg", + "asset:uploads/bench/keyframes-v4/kf06.jpg", + "asset:uploads/bench/keyframes-v4/kf07.jpg", + "asset:uploads/bench/keyframes-v4/kf08.jpg", + "asset:uploads/bench/keyframes-v4/kf09.jpg", + "asset:uploads/bench/keyframes-v4/kf10.jpg", + "asset:uploads/bench/keyframes-v4/kf11.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg" + ] + }, + "seed": 1, + "workflow_id_suffix": "-narrated-arc4", + "moments": [ + "frame:48", + "frame:720", + "frame:1440", + "frame:2040" + ] + }, + { + "name": "narrated-arc4-s2", + "scene": "s3-dialogue-walk", + "workflow": "narrated-take.json", + "arguments": { + "stop_after": 12, + "shot": "walk_keyframed", + "image": "asset:uploads/bench/keyframes-v4/kf00.jpg", + "keyframes": [ + "asset:uploads/bench/keyframes-v4/kf01.jpg", + "asset:uploads/bench/keyframes-v4/kf02.jpg", + "asset:uploads/bench/keyframes-v4/kf03.jpg", + "asset:uploads/bench/keyframes-v4/kf04.jpg", + "asset:uploads/bench/keyframes-v4/kf05.jpg", + "asset:uploads/bench/keyframes-v4/kf06.jpg", + "asset:uploads/bench/keyframes-v4/kf07.jpg", + "asset:uploads/bench/keyframes-v4/kf08.jpg", + "asset:uploads/bench/keyframes-v4/kf09.jpg", + "asset:uploads/bench/keyframes-v4/kf10.jpg", + "asset:uploads/bench/keyframes-v4/kf11.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg", + "asset:uploads/bench/keyframes-v4/kf12.jpg" + ] + }, + "seed": 2, + "workflow_id_suffix": "-narrated-arc4", + "moments": [ + "frame:48", + "frame:720", + "frame:1440", + "frame:2040" + ] + } +] diff --git a/bench/arms/narrated-candidate.json b/bench/arms/narrated-candidate.json new file mode 100644 index 000000000..adc805954 --- /dev/null +++ b/bench/arms/narrated-candidate.json @@ -0,0 +1,50 @@ +[ + { + "name": "narrated-candidate-s1", + "scene": "s3-dialogue-walk", + "workflow": "narrated-take.json", + "arguments": { + "stop_after": 12, + "shot": "dialogue", + "experimental_latent_carry": true, + "keystone": "segment_1", + "keystone_slot": "latent", + "experimental_capstone": { + "enter_below_sigma": 0.7 + }, + "anchor_every": 3 + }, + "seed": 1, + "workflow_id_suffix": "-narrated-candidate", + "moments": [ + "frame:48", + "frame:720", + "frame:1440", + "frame:2040" + ] + }, + { + "name": "narrated-candidate-s2", + "scene": "s3-dialogue-walk", + "workflow": "narrated-take.json", + "arguments": { + "stop_after": 12, + "shot": "dialogue", + "experimental_latent_carry": true, + "keystone": "segment_1", + "keystone_slot": "latent", + "experimental_capstone": { + "enter_below_sigma": 0.7 + }, + "anchor_every": 3 + }, + "seed": 2, + "workflow_id_suffix": "-narrated-candidate", + "moments": [ + "frame:48", + "frame:720", + "frame:1440", + "frame:2040" + ] + } +] diff --git a/bench/arms/narrated-control.json b/bench/arms/narrated-control.json new file mode 100644 index 000000000..6d3a5ba76 --- /dev/null +++ b/bench/arms/narrated-control.json @@ -0,0 +1,36 @@ +[ + { + "name": "narrated-control-s1", + "scene": "s3-dialogue-walk", + "workflow": "narrated-take.json", + "arguments": { + "stop_after": 12, + "shot": "dialogue" + }, + "seed": 1, + "workflow_id_suffix": "-narrated-control", + "moments": [ + "frame:48", + "frame:720", + "frame:1440", + "frame:2040" + ] + }, + { + "name": "narrated-control-s2", + "scene": "s3-dialogue-walk", + "workflow": "narrated-take.json", + "arguments": { + "stop_after": 12, + "shot": "dialogue" + }, + "seed": 2, + "workflow_id_suffix": "-narrated-control", + "moments": [ + "frame:48", + "frame:720", + "frame:1440", + "frame:2040" + ] + } +] diff --git a/bench/arms/narrated-moving-keystone.json b/bench/arms/narrated-moving-keystone.json new file mode 100644 index 000000000..c750384d9 --- /dev/null +++ b/bench/arms/narrated-moving-keystone.json @@ -0,0 +1,128 @@ +[ + { + "name": "narrated-moving-keystone-s1", + "scene": "s3-dialogue-walk", + "workflow": "narrated-take.json", + "arguments": { + "stop_after": 12, + "shot": "walk_keyframed", + "image": "asset:uploads/bench/keyframes-v2/kf00.jpg", + "keyframes": [ + "asset:uploads/bench/keyframes-v2/kf01.jpg", + "asset:uploads/bench/keyframes-v2/kf02.jpg", + "asset:uploads/bench/keyframes-v2/kf03.jpg", + "asset:uploads/bench/keyframes-v2/kf04.jpg", + "asset:uploads/bench/keyframes-v2/kf05.jpg", + "asset:uploads/bench/keyframes-v2/kf06.jpg", + "asset:uploads/bench/keyframes-v2/kf07.jpg", + "asset:uploads/bench/keyframes-v2/kf08.jpg", + "asset:uploads/bench/keyframes-v2/kf09.jpg", + "asset:uploads/bench/keyframes-v2/kf10.jpg", + "asset:uploads/bench/keyframes-v2/kf11.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg" + ] + }, + "seed": 1, + "workflow_id_suffix": "-narrated-moving-keystone", + "moments": [ + "frame:48", + "frame:720", + "frame:1440", + "frame:2040" + ] + }, + { + "name": "narrated-moving-keystone-s2", + "scene": "s3-dialogue-walk", + "workflow": "narrated-take.json", + "arguments": { + "stop_after": 12, + "shot": "walk_keyframed", + "image": "asset:uploads/bench/keyframes-v2/kf00.jpg", + "keyframes": [ + "asset:uploads/bench/keyframes-v2/kf01.jpg", + "asset:uploads/bench/keyframes-v2/kf02.jpg", + "asset:uploads/bench/keyframes-v2/kf03.jpg", + "asset:uploads/bench/keyframes-v2/kf04.jpg", + "asset:uploads/bench/keyframes-v2/kf05.jpg", + "asset:uploads/bench/keyframes-v2/kf06.jpg", + "asset:uploads/bench/keyframes-v2/kf07.jpg", + "asset:uploads/bench/keyframes-v2/kf08.jpg", + "asset:uploads/bench/keyframes-v2/kf09.jpg", + "asset:uploads/bench/keyframes-v2/kf10.jpg", + "asset:uploads/bench/keyframes-v2/kf11.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg", + "asset:uploads/bench/keyframes-v2/kf12.jpg" + ] + }, + "seed": 2, + "workflow_id_suffix": "-narrated-moving-keystone", + "moments": [ + "frame:48", + "frame:720", + "frame:1440", + "frame:2040" + ] + } +] diff --git a/bench/arms/narrated-static-long.json b/bench/arms/narrated-static-long.json new file mode 100644 index 000000000..7a7f85067 --- /dev/null +++ b/bench/arms/narrated-static-long.json @@ -0,0 +1,40 @@ +[ + { + "name": "narrated-static-long-s1", + "scene": "s3-dialogue-walk", + "workflow": "narrated-take.json", + "arguments": { + "stop_after": 12, + "shot": "dialogue", + "keystone": "segment_1", + "keystone_slot": "last" + }, + "seed": 1, + "workflow_id_suffix": "-narrated-static-long", + "moments": [ + "frame:48", + "frame:720", + "frame:1440", + "frame:2040" + ] + }, + { + "name": "narrated-static-long-s2", + "scene": "s3-dialogue-walk", + "workflow": "narrated-take.json", + "arguments": { + "stop_after": 12, + "shot": "dialogue", + "keystone": "segment_1", + "keystone_slot": "last" + }, + "seed": 2, + "workflow_id_suffix": "-narrated-static-long", + "moments": [ + "frame:48", + "frame:720", + "frame:1440", + "frame:2040" + ] + } +] diff --git a/bench/arms/noise.json b/bench/arms/noise.json new file mode 100644 index 000000000..2fda626ff --- /dev/null +++ b/bench/arms/noise.json @@ -0,0 +1,93 @@ +[ + { + "name": "noise-a-s1", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": {}, + "seed": 1166 + }, + { + "name": "noise-b-s1", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": {}, + "seed": 1166, + "workflow_id_suffix": "-noise-b" + }, + { + "name": "noise-a-s2", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": {}, + "seed": 1166 + }, + { + "name": "noise-b-s2", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": {}, + "seed": 1166, + "workflow_id_suffix": "-noise-b" + }, + { + "name": "noise-a-s3", + "scene": "s3-dialogue-walk", + "workflow": "s3-dialogue-walk.json", + "arguments": {}, + "seed": 1166 + }, + { + "name": "noise-b-s3", + "scene": "s3-dialogue-walk", + "workflow": "s3-dialogue-walk.json", + "arguments": {}, + "seed": 1166, + "workflow_id_suffix": "-noise-b" + }, + { + "name": "noise-a-s4", + "scene": "s4-montage", + "workflow": "s4-montage.json", + "arguments": {}, + "seed": 1166, + "workflow_id_suffix": "-noise-a2" + }, + { + "name": "noise-b-s4", + "scene": "s4-montage", + "workflow": "s4-montage.json", + "arguments": {}, + "seed": 1166, + "workflow_id_suffix": "-noise-b2" + }, + { + "name": "noise-a-s5", + "scene": "s5-ltx-action", + "workflow": "s5-ltx-action.json", + "arguments": {}, + "seed": 1166 + }, + { + "name": "noise-b-s5", + "scene": "s5-ltx-action", + "workflow": "s5-ltx-action.json", + "arguments": {}, + "seed": 1166, + "workflow_id_suffix": "-noise-b" + }, + { + "name": "noise-a-s6", + "scene": "s6-narration", + "workflow": "s6-narration.json", + "arguments": {}, + "seed": 1166 + }, + { + "name": "noise-b-s6", + "scene": "s6-narration", + "workflow": "s6-narration.json", + "arguments": {}, + "seed": 1166, + "workflow_id_suffix": "-noise-b" + } +] diff --git a/bench/arms/repair.json b/bench/arms/repair.json new file mode 100644 index 000000000..017788f0f --- /dev/null +++ b/bench/arms/repair.json @@ -0,0 +1,13 @@ +[ + { + "name": "repair-s3-seg4", + "scene": "s3-dialogue-walk", + "workflow": "../../workflows/templates/minimax/repair-segment.json", + "arguments": { + "chain_take": "output:BenchS3DialogueWalk-chk/20261105-082441-006f3a57/final/BenchS3DialogueWalk-chk-chained_keyframe_to_video_audio.0-0.0.mp4", + "segment": 4, + "prompt": "For the target video, at 0.00 seconds into the target video, (from [Shot 1]) is fully referenced.\n\nintegrated_multimodal_description: [Shot 1] Cinematic, live-action, one continuous shot. The shot opens exactly on : a woman in her early thirties with shoulder-length dark hair, in a long olive coat, walks toward the camera down a quiet cobblestone street at golden hour. A faint smile crosses her face as she keeps walking toward the camera. The woman (S1), a warm, low female alto voice, calm and unhurried, says [English] Funny how a place waits for you without asking. The camera holds a steady medium shot at eye level, keeping pace with her as she walks.\n\noverall_soundscape: Soft footsteps on cobblestones, a faint breeze, distant street ambience.\n\nnon_diegetic_music: None." + }, + "seed": 4 + } +] diff --git a/bench/arms/selflift.json b/bench/arms/selflift.json new file mode 100644 index 000000000..5316e40b0 --- /dev/null +++ b/bench/arms/selflift.json @@ -0,0 +1,238 @@ +[ + { + "name": "sl-ctl-s1-6seg-s1166", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "width": 1024, + "height": 576 + }, + "seed": 1166, + "workflow_id_suffix": "-sl", + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "sl-k2-s1-6seg-s1166", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "width": 1024, + "height": 576, + "selflift": { + "lift_step": 2, + "start_scale": 0.5 + } + }, + "seed": 1166, + "workflow_id_suffix": "-sl", + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "sl-k1-s1-6seg-s1166", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "width": 1024, + "height": 576, + "selflift": { + "lift_step": 1, + "start_scale": 0.5 + } + }, + "seed": 1166, + "workflow_id_suffix": "-sl", + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "sl-ctl-s1-6seg-s2", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "width": 1024, + "height": 576 + }, + "seed": 2, + "workflow_id_suffix": "-sl", + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "sl-k2-s1-6seg-s2", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "width": 1024, + "height": 576, + "selflift": { + "lift_step": 2, + "start_scale": 0.5 + } + }, + "seed": 2, + "workflow_id_suffix": "-sl", + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "sl-k1-s1-6seg-s2", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "width": 1024, + "height": 576, + "selflift": { + "lift_step": 1, + "start_scale": 0.5 + } + }, + "seed": 2, + "workflow_id_suffix": "-sl", + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "sl-ctl-s2-1seg-s1166", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "segments": 1, + "width": 1024, + "height": 576 + }, + "seed": 1166, + "workflow_id_suffix": "-sl", + "moments": [ + "frame:20", + "frame:62", + "frame:110" + ] + }, + { + "name": "sl-k2-s2-1seg-s1166", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "segments": 1, + "width": 1024, + "height": 576, + "selflift": { + "lift_step": 2, + "start_scale": 0.5 + } + }, + "seed": 1166, + "workflow_id_suffix": "-sl", + "moments": [ + "frame:20", + "frame:62", + "frame:110" + ] + }, + { + "name": "sl-k1-s2-1seg-s1166", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "segments": 1, + "width": 1024, + "height": 576, + "selflift": { + "lift_step": 1, + "start_scale": 0.5 + } + }, + "seed": 1166, + "workflow_id_suffix": "-sl", + "moments": [ + "frame:20", + "frame:62", + "frame:110" + ] + }, + { + "name": "sl-ctl-s2-1seg-s2", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "segments": 1, + "width": 1024, + "height": 576 + }, + "seed": 2, + "workflow_id_suffix": "-sl", + "moments": [ + "frame:20", + "frame:62", + "frame:110" + ] + }, + { + "name": "sl-k2-s2-1seg-s2", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "segments": 1, + "width": 1024, + "height": 576, + "selflift": { + "lift_step": 2, + "start_scale": 0.5 + } + }, + "seed": 2, + "workflow_id_suffix": "-sl", + "moments": [ + "frame:20", + "frame:62", + "frame:110" + ] + }, + { + "name": "sl-k1-s2-1seg-s2", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "segments": 1, + "width": 1024, + "height": 576, + "selflift": { + "lift_step": 1, + "start_scale": 0.5 + } + }, + "seed": 2, + "workflow_id_suffix": "-sl", + "moments": [ + "frame:20", + "frame:62", + "frame:110" + ] + } +] diff --git a/bench/arms/t16-performer-replacement.json b/bench/arms/t16-performer-replacement.json new file mode 100644 index 000000000..ca3393159 --- /dev/null +++ b/bench/arms/t16-performer-replacement.json @@ -0,0 +1,35 @@ +[ + { + "name": "t16-mask-deflect", + "scene": "s4-montage", + "workflow": "t16-person-mask.json", + "arguments": {}, + "seed": 0 + }, + { + "name": "t16-swap-r100-s1194", + "scene": "s4-montage", + "workflow": "t16-performer-swap.json", + "arguments": {}, + "seed": 1194 + }, + { + "name": "t16-swap-r100-s2002", + "scene": "s4-montage", + "workflow": "t16-performer-swap.json", + "arguments": {}, + "seed": 2002 + }, + { + "name": "t16-ctl-fresh-ref2va-s1194", + "scene": "s4-montage", + "workflow": "t16-performer-swap.json", + "arguments": { + "source_video": null, + "mask_video": null, + "mask_grow": null, + "refine_strength": null + }, + "seed": 1194 + } +] diff --git a/bench/arms/t17-refine-120s.json b/bench/arms/t17-refine-120s.json new file mode 100644 index 000000000..0bdc89f80 --- /dev/null +++ b/bench/arms/t17-refine-120s.json @@ -0,0 +1,15 @@ +[ + { + "name": "t17-s1-120s-r030-rebased", + "scene": "s1-static", + "workflow": "refine-windows.json", + "seed": 1166, + "arguments": { + "source": "output:BenchS1Static/20261104-233558-8c6ae697/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "refine_strength": 0.3, + "window_frames": 42, + "window_overlap": 12, + "window_positions": "rebased" + } + } +] diff --git a/bench/arms/t17-refine-r020-r040.json b/bench/arms/t17-refine-r020-r040.json new file mode 100644 index 000000000..5ca88b184 --- /dev/null +++ b/bench/arms/t17-refine-r020-r040.json @@ -0,0 +1,66 @@ +[ + { + "name": "t17-s1-6seg-r020", + "scene": "s1-static", + "workflow": "refine-windows.json", + "seed": 1166, + "arguments": { + "source": "output:BenchS1Static/20261104-054934-fdc9b551/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "refine_strength": 0.2 + } + }, + { + "name": "t17-s2-r020", + "scene": "s2-arc", + "workflow": "refine-windows.json", + "seed": 1166, + "arguments": { + "source": "output:BenchS2Arc/20261104-073022-cd97c6a1/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4", + "refine_strength": 0.2, + "prompt": "integrated_multimodal_description: [Shot 1] Cinematic, live-action, one continuous shot. A bronze statue of a seated man on a stone plinth in a sunlit public garden, a red brick wall with ivy, tall cypress trees and a white gazebo behind it. The camera travels in a slow, steady orbit around the statue, arcing to the right at a constant speed and keeping the statue centered, so the brick wall, the cypresses, the gazebo and the garden behind it sweep past and change with the angle. Warm late-afternoon sunlight, leaves stirring gently in a light breeze.\n\noverall_soundscape: Outdoor garden ambience: light breeze in the leaves, distant birdsong, faint far-off traffic.\n\nnon_diegetic_music: None." + } + }, + { + "name": "t17-s3-r020", + "scene": "s3-dialogue-walk", + "workflow": "refine-windows.json", + "seed": 1166, + "arguments": { + "source": "output:BenchS3DialogueWalk/20261104-084517-6db94fb9/final/BenchS3DialogueWalk-chained_keyframe_to_video_audio.0-0.0.mp4", + "refine_strength": 0.2, + "prompt": "integrated_multimodal_description: [Shot 1] Cinematic, live-action, one continuous shot. A woman in her early thirties with shoulder-length dark hair, in a long olive coat, walks toward the camera down a quiet cobblestone street at golden hour, slow and steady, glancing up at the lamps and along the shopfronts. The woman (S1), a warm, low female alto voice, calm and unhurried, speaks quietly as she walks. The camera holds a steady medium shot at eye level, keeping pace with her as she walks.\n\noverall_soundscape: Soft footsteps on cobblestones, a faint breeze, distant street ambience.\n\nnon_diegetic_music: None." + } + }, + { + "name": "t17-s1-6seg-r040", + "scene": "s1-static", + "workflow": "refine-windows.json", + "seed": 1166, + "arguments": { + "source": "output:BenchS1Static/20261104-054934-fdc9b551/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "refine_strength": 0.4 + } + }, + { + "name": "t17-s2-r040", + "scene": "s2-arc", + "workflow": "refine-windows.json", + "seed": 1166, + "arguments": { + "source": "output:BenchS2Arc/20261104-073022-cd97c6a1/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4", + "refine_strength": 0.4, + "prompt": "integrated_multimodal_description: [Shot 1] Cinematic, live-action, one continuous shot. A bronze statue of a seated man on a stone plinth in a sunlit public garden, a red brick wall with ivy, tall cypress trees and a white gazebo behind it. The camera travels in a slow, steady orbit around the statue, arcing to the right at a constant speed and keeping the statue centered, so the brick wall, the cypresses, the gazebo and the garden behind it sweep past and change with the angle. Warm late-afternoon sunlight, leaves stirring gently in a light breeze.\n\noverall_soundscape: Outdoor garden ambience: light breeze in the leaves, distant birdsong, faint far-off traffic.\n\nnon_diegetic_music: None." + } + }, + { + "name": "t17-s3-r040", + "scene": "s3-dialogue-walk", + "workflow": "refine-windows.json", + "seed": 1166, + "arguments": { + "source": "output:BenchS3DialogueWalk/20261104-084517-6db94fb9/final/BenchS3DialogueWalk-chained_keyframe_to_video_audio.0-0.0.mp4", + "refine_strength": 0.4, + "prompt": "integrated_multimodal_description: [Shot 1] Cinematic, live-action, one continuous shot. A woman in her early thirties with shoulder-length dark hair, in a long olive coat, walks toward the camera down a quiet cobblestone street at golden hour, slow and steady, glancing up at the lamps and along the shopfronts. The woman (S1), a warm, low female alto voice, calm and unhurried, speaks quietly as she walks. The camera holds a steady medium shot at eye level, keeping pace with her as she walks.\n\noverall_soundscape: Soft footsteps on cobblestones, a faint breeze, distant street ambience.\n\nnon_diegetic_music: None." + } + } +] \ No newline at end of file diff --git a/bench/arms/t17-refine-r030.json b/bench/arms/t17-refine-r030.json new file mode 100644 index 000000000..09f93c58d --- /dev/null +++ b/bench/arms/t17-refine-r030.json @@ -0,0 +1,47 @@ +[ + { + "name": "t17-s1-6seg-r030", + "scene": "s1-static", + "workflow": "refine-windows.json", + "seed": 1166, + "arguments": { + "source": "output:BenchS1Static/20261104-054934-fdc9b551/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "refine_strength": 0.3 + } + }, + { + "name": "t17-s1-6seg-r030-rebased", + "scene": "s1-static", + "workflow": "refine-windows.json", + "seed": 1166, + "arguments": { + "source": "output:BenchS1Static/20261104-054934-fdc9b551/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "refine_strength": 0.3, + "window_frames": 42, + "window_overlap": 12, + "window_positions": "rebased" + } + }, + { + "name": "t17-s2-r030", + "scene": "s2-arc", + "workflow": "refine-windows.json", + "seed": 1166, + "arguments": { + "source": "output:BenchS2Arc/20261104-073022-cd97c6a1/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4", + "refine_strength": 0.3, + "prompt": "integrated_multimodal_description: [Shot 1] Cinematic, live-action, one continuous shot. A bronze statue of a seated man on a stone plinth in a sunlit public garden, a red brick wall with ivy, tall cypress trees and a white gazebo behind it. The camera travels in a slow, steady orbit around the statue, arcing to the right at a constant speed and keeping the statue centered, so the brick wall, the cypresses, the gazebo and the garden behind it sweep past and change with the angle. Warm late-afternoon sunlight, leaves stirring gently in a light breeze.\n\noverall_soundscape: Outdoor garden ambience: light breeze in the leaves, distant birdsong, faint far-off traffic.\n\nnon_diegetic_music: None." + } + }, + { + "name": "t17-s3-r030", + "scene": "s3-dialogue-walk", + "workflow": "refine-windows.json", + "seed": 1166, + "arguments": { + "source": "output:BenchS3DialogueWalk/20261104-084517-6db94fb9/final/BenchS3DialogueWalk-chained_keyframe_to_video_audio.0-0.0.mp4", + "refine_strength": 0.3, + "prompt": "integrated_multimodal_description: [Shot 1] Cinematic, live-action, one continuous shot. A woman in her early thirties with shoulder-length dark hair, in a long olive coat, walks toward the camera down a quiet cobblestone street at golden hour, slow and steady, glancing up at the lamps and along the shopfronts. The woman (S1), a warm, low female alto voice, calm and unhurried, speaks quietly as she walks. The camera holds a steady medium shot at eye level, keeping pace with her as she walks.\n\noverall_soundscape: Soft footsteps on cobblestones, a faint breeze, distant street ambience.\n\nnon_diegetic_music: None." + } + } +] \ No newline at end of file diff --git a/bench/arms/t26-claims.json b/bench/arms/t26-claims.json new file mode 100644 index 000000000..5d164aca6 --- /dev/null +++ b/bench/arms/t26-claims.json @@ -0,0 +1,181 @@ +[ + { + "name": "c1-line-plain-s2002", + "scene": "s3-dialogue-walk", + "workflow": "t26-t2va.json", + "arguments": { + "prompt": "summary:\n[text generation] The target video shows a woman alone in a small kitchen, speaking to the camera.\n\ndetailed_description:\nThe target video is in realistic photographic style. [Shot 1] A static medium shot frames a woman in her thirties with short dark hair, in a plain white sleeveless T-shirt, standing alone behind a bare wooden kitchen counter and facing the camera. The counter is empty and the pale wall behind her is plain. She looks at the lens and, in a warm, clear female voice, says [English] My brother Daniel borrowed my red raincoat last winter, and he still hasn't given it back. Then she shrugs and folds her arms. The camera does not move.\n\noverall_soundscape:\nA quiet kitchen room tone.\n\nnon_diegetic_music:\nNone." + }, + "seed": 2002 + }, + { + "name": "c1-line-sentence-s1001", + "scene": "s3-dialogue-walk", + "workflow": "t26-t2va.json", + "arguments": { + "prompt": "summary:\n[text generation] The target video shows a woman alone in a small kitchen, speaking to the camera.\n\ndetailed_description:\nThe target video is in realistic photographic style. [Shot 1] A static medium shot frames a woman in her thirties with short dark hair, in a plain white sleeveless T-shirt, standing alone behind a bare wooden kitchen counter and facing the camera. The counter is empty and the pale wall behind her is plain. She looks at the lens and, in a warm, clear female voice, says [English] My brother Daniel borrowed my red raincoat last winter, and he still hasn't given it back. Then she shrugs and folds her arms. The camera does not move. Spoken words are only heard: the shot shows what it describes, not what anyone talks about.\n\noverall_soundscape:\nA quiet kitchen room tone.\n\nnon_diegetic_music:\nNone." + }, + "seed": 1001 + }, + { + "name": "c1-line-sentence-s2002", + "scene": "s3-dialogue-walk", + "workflow": "t26-t2va.json", + "arguments": { + "prompt": "summary:\n[text generation] The target video shows a woman alone in a small kitchen, speaking to the camera.\n\ndetailed_description:\nThe target video is in realistic photographic style. [Shot 1] A static medium shot frames a woman in her thirties with short dark hair, in a plain white sleeveless T-shirt, standing alone behind a bare wooden kitchen counter and facing the camera. The counter is empty and the pale wall behind her is plain. She looks at the lens and, in a warm, clear female voice, says [English] My brother Daniel borrowed my red raincoat last winter, and he still hasn't given it back. Then she shrugs and folds her arms. The camera does not move. Spoken words are only heard: the shot shows what it describes, not what anyone talks about.\n\noverall_soundscape:\nA quiet kitchen room tone.\n\nnon_diegetic_music:\nNone." + }, + "seed": 2002 + }, + { + "name": "c5-hum-s1001", + "scene": "s3-dialogue-walk", + "workflow": "t26-t2va.json", + "arguments": { + "prompt": "summary:\n[text generation] The target video shows a woman reading at a kitchen table late at night while rain falls outside.\n\ndetailed_description:\nThe target video is in realistic photographic style. [Shot 1] A static medium shot frames a woman in her forties with greying hair tied back, in a navy cardigan, sitting at a wooden kitchen table at night with a paperback and a white mug in front of her. A dark window behind her is streaked with rain. She reads, turns a page, lifts the mug, sips and sets it down again. The camera does not move.\n\noverall_soundscape:\nThe steady low hum of the refrigerator and the buzz of the fluorescent strip light overhead run under everything. Rain pattering steadily against the dark window, a paperback page turning now and then, a ceramic mug set down softly on the wooden table once.\n\nnon_diegetic_music:\nNone." + }, + "seed": 1001 + }, + { + "name": "c5-hum-s2002", + "scene": "s3-dialogue-walk", + "workflow": "t26-t2va.json", + "arguments": { + "prompt": "summary:\n[text generation] The target video shows a woman reading at a kitchen table late at night while rain falls outside.\n\ndetailed_description:\nThe target video is in realistic photographic style. [Shot 1] A static medium shot frames a woman in her forties with greying hair tied back, in a navy cardigan, sitting at a wooden kitchen table at night with a paperback and a white mug in front of her. A dark window behind her is streaked with rain. She reads, turns a page, lifts the mug, sips and sets it down again. The camera does not move.\n\noverall_soundscape:\nThe steady low hum of the refrigerator and the buzz of the fluorescent strip light overhead run under everything. Rain pattering steadily against the dark window, a paperback page turning now and then, a ceramic mug set down softly on the wooden table once.\n\nnon_diegetic_music:\nNone." + }, + "seed": 2002 + }, + { + "name": "c5-nohum-s1001", + "scene": "s3-dialogue-walk", + "workflow": "t26-t2va.json", + "arguments": { + "prompt": "summary:\n[text generation] The target video shows a woman reading at a kitchen table late at night while rain falls outside.\n\ndetailed_description:\nThe target video is in realistic photographic style. [Shot 1] A static medium shot frames a woman in her forties with greying hair tied back, in a navy cardigan, sitting at a wooden kitchen table at night with a paperback and a white mug in front of her. A dark window behind her is streaked with rain. She reads, turns a page, lifts the mug, sips and sets it down again. The camera does not move.\n\noverall_soundscape:\nRain pattering steadily against the dark window, a paperback page turning now and then, a ceramic mug set down softly on the wooden table once.\n\nnon_diegetic_music:\nNone." + }, + "seed": 1001 + }, + { + "name": "c5-nohum-s2002", + "scene": "s3-dialogue-walk", + "workflow": "t26-t2va.json", + "arguments": { + "prompt": "summary:\n[text generation] The target video shows a woman reading at a kitchen table late at night while rain falls outside.\n\ndetailed_description:\nThe target video is in realistic photographic style. [Shot 1] A static medium shot frames a woman in her forties with greying hair tied back, in a navy cardigan, sitting at a wooden kitchen table at night with a paperback and a white mug in front of her. A dark window behind her is streaked with rain. She reads, turns a page, lifts the mug, sips and sets it down again. The camera does not move.\n\noverall_soundscape:\nRain pattering steadily against the dark window, a paperback page turning now and then, a ceramic mug set down softly on the wooden table once.\n\nnon_diegetic_music:\nNone." + }, + "seed": 2002 + }, + { + "name": "c5-nohum-nomusicline-s1001", + "scene": "s3-dialogue-walk", + "workflow": "t26-t2va.json", + "arguments": { + "prompt": "summary:\n[text generation] The target video shows a woman reading at a kitchen table late at night while rain falls outside.\n\ndetailed_description:\nThe target video is in realistic photographic style. [Shot 1] A static medium shot frames a woman in her forties with greying hair tied back, in a navy cardigan, sitting at a wooden kitchen table at night with a paperback and a white mug in front of her. A dark window behind her is streaked with rain. She reads, turns a page, lifts the mug, sips and sets it down again. The camera does not move.\n\noverall_soundscape:\nRain pattering steadily against the dark window, a paperback page turning now and then, a ceramic mug set down softly on the wooden table once." + }, + "seed": 1001 + }, + { + "name": "c5-nohum-nomusicline-s2002", + "scene": "s3-dialogue-walk", + "workflow": "t26-t2va.json", + "arguments": { + "prompt": "summary:\n[text generation] The target video shows a woman reading at a kitchen table late at night while rain falls outside.\n\ndetailed_description:\nThe target video is in realistic photographic style. [Shot 1] A static medium shot frames a woman in her forties with greying hair tied back, in a navy cardigan, sitting at a wooden kitchen table at night with a paperback and a white mug in front of her. A dark window behind her is streaked with rain. She reads, turns a page, lifts the mug, sips and sets it down again. The camera does not move.\n\noverall_soundscape:\nRain pattering steadily against the dark window, a paperback page turning now and then, a ceramic mug set down softly on the wooden table once." + }, + "seed": 2002 + }, + { + "name": "c2-unmentioned-s1001", + "scene": "s3-dialogue-walk", + "workflow": "t26-ref2va.json", + "arguments": { + "prompt": "subject_definitions:\n is the woman in : early thirties, shoulder-length dark hair, a long olive double-breasted coat.\n\nsummary:\n[reference generation] The target video shows sitting on a wooden park bench in autumn, reading a paperback, then looking up and smiling.\n\nretention_analysis:\n (appears in [Shot 1]): fully_preserved - her face, hair, coat and proportions from are kept, newly posed seated on the bench.\n\ndetailed_description:\nThe target video is in realistic photographic style. [Shot 1] A static medium-wide shot at eye level frames sitting on a wooden park bench under trees with yellow and orange leaves, a paperback open in her hands. Soft overcast daylight. She turns a page, then looks up from the book toward the path in front of her and smiles. Leaves drift down behind her. The camera does not move.\n\noverall_soundscape:\nLight wind in the trees, dry leaves skittering across the path, a page turning, distant birds.\n\nnon_diegetic_music:\nNone.", + "picture_1": "asset:bench/walker.png", + "picture_2": "asset:bench/t26-stranger.jpg" + }, + "seed": 1001 + }, + { + "name": "c2-unmentioned-s2002", + "scene": "s3-dialogue-walk", + "workflow": "t26-ref2va.json", + "arguments": { + "prompt": "subject_definitions:\n is the woman in : early thirties, shoulder-length dark hair, a long olive double-breasted coat.\n\nsummary:\n[reference generation] The target video shows sitting on a wooden park bench in autumn, reading a paperback, then looking up and smiling.\n\nretention_analysis:\n (appears in [Shot 1]): fully_preserved - her face, hair, coat and proportions from are kept, newly posed seated on the bench.\n\ndetailed_description:\nThe target video is in realistic photographic style. [Shot 1] A static medium-wide shot at eye level frames sitting on a wooden park bench under trees with yellow and orange leaves, a paperback open in her hands. Soft overcast daylight. She turns a page, then looks up from the book toward the path in front of her and smiles. Leaves drift down behind her. The camera does not move.\n\noverall_soundscape:\nLight wind in the trees, dry leaves skittering across the path, a page turning, distant birds.\n\nnon_diegetic_music:\nNone.", + "picture_1": "asset:bench/walker.png", + "picture_2": "asset:bench/t26-stranger.jpg" + }, + "seed": 2002 + }, + { + "name": "c2-control-s1001", + "scene": "s3-dialogue-walk", + "workflow": "t26-ref2va.json", + "arguments": { + "prompt": "subject_definitions:\n is the woman in : early thirties, shoulder-length dark hair, a long olive double-breasted coat.\n\nsummary:\n[reference generation] The target video shows sitting on a wooden park bench in autumn, reading a paperback, then looking up and smiling.\n\nretention_analysis:\n (appears in [Shot 1]): fully_preserved - her face, hair, coat and proportions from are kept, newly posed seated on the bench.\n\ndetailed_description:\nThe target video is in realistic photographic style. [Shot 1] A static medium-wide shot at eye level frames sitting on a wooden park bench under trees with yellow and orange leaves, a paperback open in her hands. Soft overcast daylight. She turns a page, then looks up from the book toward the path in front of her and smiles. Leaves drift down behind her. The camera does not move.\n\noverall_soundscape:\nLight wind in the trees, dry leaves skittering across the path, a page turning, distant birds.\n\nnon_diegetic_music:\nNone.", + "picture_1": "asset:bench/walker.png", + "picture_2": null + }, + "seed": 1001 + }, + { + "name": "c2-control-s2002", + "scene": "s3-dialogue-walk", + "workflow": "t26-ref2va.json", + "arguments": { + "prompt": "subject_definitions:\n is the woman in : early thirties, shoulder-length dark hair, a long olive double-breasted coat.\n\nsummary:\n[reference generation] The target video shows sitting on a wooden park bench in autumn, reading a paperback, then looking up and smiling.\n\nretention_analysis:\n (appears in [Shot 1]): fully_preserved - her face, hair, coat and proportions from are kept, newly posed seated on the bench.\n\ndetailed_description:\nThe target video is in realistic photographic style. [Shot 1] A static medium-wide shot at eye level frames sitting on a wooden park bench under trees with yellow and orange leaves, a paperback open in her hands. Soft overcast daylight. She turns a page, then looks up from the book toward the path in front of her and smiles. Leaves drift down behind her. The camera does not move.\n\noverall_soundscape:\nLight wind in the trees, dry leaves skittering across the path, a page turning, distant birds.\n\nnon_diegetic_music:\nNone.", + "picture_1": "asset:bench/walker.png", + "picture_2": null + }, + "seed": 2002 + }, + { + "name": "c3-fullbody-s1001", + "scene": "s3-dialogue-walk", + "workflow": "t26-ref2va.json", + "arguments": { + "prompt": "subject_definitions:\n is the woman in : early thirties, shoulder-length dark hair. In this video she wears a bright red knitted turtleneck sweater and light blue jeans, and no coat.\n\nsummary:\n[reference generation] The target video shows in a bright kitchen in the morning, pouring coffee from a glass carafe into a white mug and taking a sip.\n\nretention_analysis:\n (appears in [Shot 1]): partially_preserved - her face and hair from are kept. Her clothing is not taken from : she wears the bright red knitted turtleneck sweater and light blue jeans described here, and no coat.\n\ndetailed_description:\nThe target video is in realistic photographic style. [Shot 1] A static medium shot at eye level frames from the waist up, standing at a white kitchen counter in a bright kitchen with morning sun coming through a window to the left. She wears a bright red knitted turtleneck sweater and light blue jeans. She lifts a glass carafe of coffee, pours it into a white mug, sets the carafe down, and takes a slow sip, smiling. The camera does not move.\n\noverall_soundscape:\nCoffee pouring into the mug, the carafe set down on the counter, a quiet morning kitchen, birds faintly outside.\n\nnon_diegetic_music:\nNone.", + "picture_1": "asset:bench/walker.png" + }, + "seed": 1001 + }, + { + "name": "c3-fullbody-s2002", + "scene": "s3-dialogue-walk", + "workflow": "t26-ref2va.json", + "arguments": { + "prompt": "subject_definitions:\n is the woman in : early thirties, shoulder-length dark hair. In this video she wears a bright red knitted turtleneck sweater and light blue jeans, and no coat.\n\nsummary:\n[reference generation] The target video shows in a bright kitchen in the morning, pouring coffee from a glass carafe into a white mug and taking a sip.\n\nretention_analysis:\n (appears in [Shot 1]): partially_preserved - her face and hair from are kept. Her clothing is not taken from : she wears the bright red knitted turtleneck sweater and light blue jeans described here, and no coat.\n\ndetailed_description:\nThe target video is in realistic photographic style. [Shot 1] A static medium shot at eye level frames from the waist up, standing at a white kitchen counter in a bright kitchen with morning sun coming through a window to the left. She wears a bright red knitted turtleneck sweater and light blue jeans. She lifts a glass carafe of coffee, pours it into a white mug, sets the carafe down, and takes a slow sip, smiling. The camera does not move.\n\noverall_soundscape:\nCoffee pouring into the mug, the carafe set down on the counter, a quiet morning kitchen, birds faintly outside.\n\nnon_diegetic_music:\nNone.", + "picture_1": "asset:bench/walker.png" + }, + "seed": 2002 + }, + { + "name": "c3-facecrop-s1001", + "scene": "s3-dialogue-walk", + "workflow": "t26-ref2va.json", + "arguments": { + "prompt": "subject_definitions:\n is the woman in : early thirties, shoulder-length dark hair. In this video she wears a bright red knitted turtleneck sweater and light blue jeans, and no coat.\n\nsummary:\n[reference generation] The target video shows in a bright kitchen in the morning, pouring coffee from a glass carafe into a white mug and taking a sip.\n\nretention_analysis:\n (appears in [Shot 1]): partially_preserved - her face and hair from are kept. Her clothing is not taken from : she wears the bright red knitted turtleneck sweater and light blue jeans described here, and no coat.\n\ndetailed_description:\nThe target video is in realistic photographic style. [Shot 1] A static medium shot at eye level frames from the waist up, standing at a white kitchen counter in a bright kitchen with morning sun coming through a window to the left. She wears a bright red knitted turtleneck sweater and light blue jeans. She lifts a glass carafe of coffee, pours it into a white mug, sets the carafe down, and takes a slow sip, smiling. The camera does not move.\n\noverall_soundscape:\nCoffee pouring into the mug, the carafe set down on the counter, a quiet morning kitchen, birds faintly outside.\n\nnon_diegetic_music:\nNone.", + "picture_1": "asset:bench/t26-walker-face.png" + }, + "seed": 1001 + }, + { + "name": "c3-facecrop-s2002", + "scene": "s3-dialogue-walk", + "workflow": "t26-ref2va.json", + "arguments": { + "prompt": "subject_definitions:\n is the woman in : early thirties, shoulder-length dark hair. In this video she wears a bright red knitted turtleneck sweater and light blue jeans, and no coat.\n\nsummary:\n[reference generation] The target video shows in a bright kitchen in the morning, pouring coffee from a glass carafe into a white mug and taking a sip.\n\nretention_analysis:\n (appears in [Shot 1]): partially_preserved - her face and hair from are kept. Her clothing is not taken from : she wears the bright red knitted turtleneck sweater and light blue jeans described here, and no coat.\n\ndetailed_description:\nThe target video is in realistic photographic style. [Shot 1] A static medium shot at eye level frames from the waist up, standing at a white kitchen counter in a bright kitchen with morning sun coming through a window to the left. She wears a bright red knitted turtleneck sweater and light blue jeans. She lifts a glass carafe of coffee, pours it into a white mug, sets the carafe down, and takes a slow sip, smiling. The camera does not move.\n\noverall_soundscape:\nCoffee pouring into the mug, the carafe set down on the counter, a quiet morning kitchen, birds faintly outside.\n\nnon_diegetic_music:\nNone.", + "picture_1": "asset:bench/t26-walker-face.png" + }, + "seed": 2002 + }, + { + "name": "c4-continued-s1001", + "scene": "s3-dialogue-walk", + "workflow": "t26-continued.json", + "arguments": {}, + "seed": 1001 + }, + { + "name": "c4-continued-s2002", + "scene": "s3-dialogue-walk", + "workflow": "t26-continued.json", + "arguments": {}, + "seed": 2002 + } +] diff --git a/bench/arms/t26-facecheck.json b/bench/arms/t26-facecheck.json new file mode 100644 index 000000000..5c77ef05c --- /dev/null +++ b/bench/arms/t26-facecheck.json @@ -0,0 +1,42 @@ +[ + { + "name": "facecheck-c2-unmentioned-s1001", + "scene": "s3-dialogue-walk", + "workflow": "t26-face-check.json", + "arguments": { + "video": "output:BenchT26Ref2VA/20261106-033548-3a857c4d/final/BenchT26Ref2VA-clip.0-0.0.mp4", + "reference": "asset:bench/t26-stranger.jpg" + }, + "seed": 0 + }, + { + "name": "facecheck-c2-unmentioned-s2002", + "scene": "s3-dialogue-walk", + "workflow": "t26-face-check.json", + "arguments": { + "video": "output:BenchT26Ref2VA/20261106-034400-d520e7c7/final/BenchT26Ref2VA-clip.0-0.0.mp4", + "reference": "asset:bench/t26-stranger.jpg" + }, + "seed": 0 + }, + { + "name": "facecheck-c2-control-s1001", + "scene": "s3-dialogue-walk", + "workflow": "t26-face-check.json", + "arguments": { + "video": "output:BenchT26Ref2VA/20261106-035011-9246a409/final/BenchT26Ref2VA-clip.0-0.0.mp4", + "reference": "asset:bench/t26-stranger.jpg" + }, + "seed": 0 + }, + { + "name": "facecheck-c2-control-s2002", + "scene": "s3-dialogue-walk", + "workflow": "t26-face-check.json", + "arguments": { + "video": "output:BenchT26Ref2VA/20261106-035727-3d0b72ac/final/BenchT26Ref2VA-clip.0-0.0.mp4", + "reference": "asset:bench/t26-stranger.jpg" + }, + "seed": 0 + } +] diff --git a/bench/arms/t26-prep.json b/bench/arms/t26-prep.json new file mode 100644 index 000000000..2514f25ce --- /dev/null +++ b/bench/arms/t26-prep.json @@ -0,0 +1,18 @@ +[ + { + "name": "t26-portraits", + "scene": "s3-dialogue-walk", + "workflow": "t26-portraits.json", + "arguments": {}, + "seed": 1193 + }, + { + "name": "c1-line-plain-s1001-probe", + "scene": "s3-dialogue-walk", + "workflow": "t26-t2va.json", + "arguments": { + "prompt": "summary:\n[text generation] The target video shows a woman alone in a small kitchen, speaking to the camera.\n\ndetailed_description:\nThe target video is in realistic photographic style. [Shot 1] A static medium shot frames a woman in her thirties with short dark hair, in a plain white sleeveless T-shirt, standing alone behind a bare wooden kitchen counter and facing the camera. The counter is empty and the pale wall behind her is plain. She looks at the lens and, in a warm, clear female voice, says [English] My brother Daniel borrowed my red raincoat last winter, and he still hasn't given it back. Then she shrugs and folds her arms. The camera does not move.\n\noverall_soundscape:\nA quiet kitchen room tone.\n\nnon_diegetic_music:\nNone." + }, + "seed": 1001 + } +] diff --git a/bench/arms/tiers-b.json b/bench/arms/tiers-b.json new file mode 100644 index 000000000..26694286d --- /dev/null +++ b/bench/arms/tiers-b.json @@ -0,0 +1,33 @@ +[ + { + "name": "t20-c1344-int8-s1-s1166", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "width": 1344, + "height": 768, + "weights_dtype": "{int8}" + }, + "seed": 1166, + "workflow_id_suffix": "-t20", + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "t20-c1344-int8-s4-s1166", + "scene": "s4-montage", + "workflow": "s4-montage.json", + "arguments": { + "width": 1344, + "height": 768, + "weights_dtype": "{int8}" + }, + "seed": 1166, + "workflow_id_suffix": "-t20" + } +] diff --git a/bench/arms/tiers-c.json b/bench/arms/tiers-c.json new file mode 100644 index 000000000..4e0c2b002 --- /dev/null +++ b/bench/arms/tiers-c.json @@ -0,0 +1,12 @@ +[ + { + "name": "t20-int8-s4-s1166", + "scene": "s4-montage", + "workflow": "s4-montage.json", + "arguments": { + "weights_dtype": "{int8}" + }, + "seed": 1166, + "workflow_id_suffix": "-t20" + } +] diff --git a/bench/arms/tiers-int8.json b/bench/arms/tiers-int8.json new file mode 100644 index 000000000..9b0807189 --- /dev/null +++ b/bench/arms/tiers-int8.json @@ -0,0 +1,192 @@ +[ + { + "name": "t20-int8-s1-s2", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "weights_dtype": "{int8}" + }, + "seed": 2, + "workflow_id_suffix": "-t20", + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "t20-int8-s2-s1166", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "weights_dtype": "{int8}" + }, + "seed": 1166, + "workflow_id_suffix": "-t20", + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "t20-int8-s3-s1166", + "scene": "s3-dialogue-walk", + "workflow": "s3-dialogue-walk.json", + "arguments": { + "weights_dtype": "{int8}" + }, + "seed": 1166, + "workflow_id_suffix": "-t20" + }, + { + "name": "t20-int8-s2-s2", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "weights_dtype": "{int8}" + }, + "seed": 2, + "workflow_id_suffix": "-t20", + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "t20-ctl-s2-s2", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "segments": 6, + "continuity": "guide" + }, + "seed": 2, + "workflow_id_suffix": "-t20", + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "t20-c1344-int8-s1-s2", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "width": 1344, + "height": 768, + "weights_dtype": "{int8}" + }, + "seed": 2, + "workflow_id_suffix": "-t20", + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "t20-c1344-int8-s2-s1166", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "width": 1344, + "height": 768, + "weights_dtype": "{int8}" + }, + "seed": 1166, + "workflow_id_suffix": "-t20", + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "t20-ctl-s3-s2", + "scene": "s3-dialogue-walk", + "workflow": "s3-dialogue-walk.json", + "arguments": {}, + "seed": 2, + "workflow_id_suffix": "-t20" + }, + { + "name": "t20-int8-s3-s2", + "scene": "s3-dialogue-walk", + "workflow": "s3-dialogue-walk.json", + "arguments": { + "weights_dtype": "{int8}" + }, + "seed": 2, + "workflow_id_suffix": "-t20" + }, + { + "name": "t20-st4int8-s1-s1166", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "weights_dtype": "{int8}", + "num_inference_steps": 4 + }, + "seed": 1166, + "workflow_id_suffix": "-t20", + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "t20-st4int8-s2-s1166", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "weights_dtype": "{int8}", + "num_inference_steps": 4 + }, + "seed": 1166, + "workflow_id_suffix": "-t20", + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "t20-int8-static-long-120s-s1166", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "duration_seconds": 120, + "anchor": "none", + "keystone": "segment_1", + "keystone_slot": "last", + "weights_dtype": "{int8}" + }, + "seed": 1166, + "workflow_id_suffix": "-t20", + "moments": [ + "frame:62", + "frame:1501", + "frame:2868" + ] + } +] diff --git a/bench/arms/tiers-template.json b/bench/arms/tiers-template.json new file mode 100644 index 000000000..601307feb --- /dev/null +++ b/bench/arms/tiers-template.json @@ -0,0 +1,15 @@ +[ + { + "name": "t20-template-static-long-120s-s1166", + "scene": "s1-static", + "workflow": "t20-template-static-long.json", + "arguments": {}, + "seed": 1166, + "workflow_id_suffix": "-t20", + "moments": [ + "frame:62", + "frame:1501", + "frame:2868" + ] + } +] diff --git a/bench/arms/tiers.json b/bench/arms/tiers.json new file mode 100644 index 000000000..3d3b4704b --- /dev/null +++ b/bench/arms/tiers.json @@ -0,0 +1,299 @@ +[ + { + "name": "t20-ctl-s1-s1166", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide" + }, + "seed": 1166, + "workflow_id_suffix": "-t20", + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "t20-c672-s1-s1166", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "width": 672, + "height": 384 + }, + "seed": 1166, + "workflow_id_suffix": "-t20", + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "t20-st4-s1-s1166", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "num_inference_steps": 4 + }, + "seed": 1166, + "workflow_id_suffix": "-t20", + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "t20-c1344-s1-s1166", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "width": 1344, + "height": 768 + }, + "seed": 1166, + "workflow_id_suffix": "-t20", + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "t20-ctl-s2-s1166", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "segments": 6, + "continuity": "guide" + }, + "seed": 1166, + "workflow_id_suffix": "-t20", + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "t20-c672-s2-s1166", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "width": 672, + "height": 384 + }, + "seed": 1166, + "workflow_id_suffix": "-t20", + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "t20-st4-s2-s1166", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "num_inference_steps": 4 + }, + "seed": 1166, + "workflow_id_suffix": "-t20", + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "t20-c1344-s2-s1166", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "width": 1344, + "height": 768 + }, + "seed": 1166, + "workflow_id_suffix": "-t20", + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "t20-c1344-8step-s1-s1166", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "width": 1344, + "height": 768, + "num_inference_steps": 9, + "lora_weight_name": "minimax_h3_fl2v_turbo_8step_v1.0_768p_bf16.safetensors" + }, + "seed": 1166, + "workflow_id_suffix": "-t20", + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "t20-int8-s1-s1166", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "weights_dtype": "{int8}" + }, + "seed": 1166, + "workflow_id_suffix": "-t20", + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "t20-ctl-s1-s2", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide" + }, + "seed": 2, + "workflow_id_suffix": "-t20", + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "t20-c672-s1-s2", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "width": 672, + "height": 384 + }, + "seed": 2, + "workflow_id_suffix": "-t20", + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "t20-c1344-s1-s2", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 6, + "continuity": "guide", + "width": 1344, + "height": 768 + }, + "seed": 2, + "workflow_id_suffix": "-t20", + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ] + }, + { + "name": "t20-ctl-s3-s1166", + "scene": "s3-dialogue-walk", + "workflow": "s3-dialogue-walk.json", + "arguments": {}, + "seed": 1166, + "workflow_id_suffix": "-t20" + }, + { + "name": "t20-c672-s3-s1166", + "scene": "s3-dialogue-walk", + "workflow": "s3-dialogue-walk.json", + "arguments": { + "width": 672, + "height": 384 + }, + "seed": 1166, + "workflow_id_suffix": "-t20" + }, + { + "name": "t20-c1344-s3-s1166", + "scene": "s3-dialogue-walk", + "workflow": "s3-dialogue-walk.json", + "arguments": { + "width": 1344, + "height": 768 + }, + "seed": 1166, + "workflow_id_suffix": "-t20" + }, + { + "name": "t20-ctl-s4-s1166", + "scene": "s4-montage", + "workflow": "s4-montage.json", + "arguments": {}, + "seed": 1166, + "workflow_id_suffix": "-t20" + }, + { + "name": "t20-c672-s4-s1166", + "scene": "s4-montage", + "workflow": "s4-montage.json", + "arguments": { + "width": 672, + "height": 384 + }, + "seed": 1166, + "workflow_id_suffix": "-t20" + }, + { + "name": "t20-st5-s4-s1166", + "scene": "s4-montage", + "workflow": "s4-montage.json", + "arguments": { + "num_inference_steps": 5 + }, + "seed": 1166, + "workflow_id_suffix": "-t20" + }, + { + "name": "t20-c1344-s4-s1166", + "scene": "s4-montage", + "workflow": "s4-montage.json", + "arguments": { + "width": 1344, + "height": 768 + }, + "seed": 1166, + "workflow_id_suffix": "-t20" + } +] diff --git a/bench/arms/tst.json b/bench/arms/tst.json new file mode 100644 index 000000000..b2231bd1a --- /dev/null +++ b/bench/arms/tst.json @@ -0,0 +1,226 @@ +[ + { + "name": "tst-sparse-s1-s1166", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 1 + }, + "seed": 1166, + "workflow_id_suffix": "-tst", + "moments": [ + "frame:20", + "frame:62", + "frame:110" + ] + }, + { + "name": "tst-sparse-s1-s2", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 1 + }, + "seed": 2, + "workflow_id_suffix": "-tst", + "moments": [ + "frame:20", + "frame:62", + "frame:110" + ] + }, + { + "name": "tst-sparse-s2-s1166", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "segments": 1 + }, + "seed": 1166, + "workflow_id_suffix": "-tst", + "moments": [ + "frame:20", + "frame:62", + "frame:110" + ] + }, + { + "name": "tst-sparse-s2-s2", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "segments": 1 + }, + "seed": 2, + "workflow_id_suffix": "-tst", + "moments": [ + "frame:20", + "frame:62", + "frame:110" + ] + }, + { + "name": "tst-dense-s1-s1166", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 1, + "attention_backend": null + }, + "seed": 1166, + "workflow_id_suffix": "-tst", + "moments": [ + "frame:20", + "frame:62", + "frame:110" + ] + }, + { + "name": "tst-dense-s1-s2", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 1, + "attention_backend": null + }, + "seed": 2, + "workflow_id_suffix": "-tst", + "moments": [ + "frame:20", + "frame:62", + "frame:110" + ] + }, + { + "name": "tst-dense-s2-s1166", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "segments": 1, + "attention_backend": null + }, + "seed": 1166, + "workflow_id_suffix": "-tst", + "moments": [ + "frame:20", + "frame:62", + "frame:110" + ] + }, + { + "name": "tst-dense-s2-s2", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "segments": 1, + "attention_backend": null + }, + "seed": 2, + "workflow_id_suffix": "-tst", + "moments": [ + "frame:20", + "frame:62", + "frame:110" + ] + }, + { + "name": "tst-tst20-s1-s1166", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 1, + "attention_backend": null, + "tst": { + "tau": 0.2, + "per_head": true + } + }, + "seed": 1166, + "workflow_id_suffix": "-tst", + "moments": [ + "frame:20", + "frame:62", + "frame:110" + ] + }, + { + "name": "tst-tst20-s1-s2", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 1, + "attention_backend": null, + "tst": { + "tau": 0.2, + "per_head": true + } + }, + "seed": 2, + "workflow_id_suffix": "-tst", + "moments": [ + "frame:20", + "frame:62", + "frame:110" + ] + }, + { + "name": "tst-tst20-s2-s1166", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "segments": 1, + "attention_backend": null, + "tst": { + "tau": 0.2, + "per_head": true + } + }, + "seed": 1166, + "workflow_id_suffix": "-tst", + "moments": [ + "frame:20", + "frame:62", + "frame:110" + ] + }, + { + "name": "tst-tst20-s2-s2", + "scene": "s2-arc", + "workflow": "s2-arc.json", + "arguments": { + "segments": 1, + "attention_backend": null, + "tst": { + "tau": 0.2, + "per_head": true + } + }, + "seed": 2, + "workflow_id_suffix": "-tst", + "moments": [ + "frame:20", + "frame:62", + "frame:110" + ] + }, + { + "name": "tst-tau0-s1-s1166", + "scene": "s1-static", + "workflow": "s1-static.json", + "arguments": { + "segments": 1, + "attention_backend": null, + "tst": { + "tau": 0.0, + "per_head": true + } + }, + "seed": 1166, + "workflow_id_suffix": "-tst", + "moments": [ + "frame:20", + "frame:62", + "frame:110" + ] + } +] diff --git a/bench/cuts.py b/bench/cuts.py new file mode 100644 index 000000000..b8a744a3f --- /dev/null +++ b/bench/cuts.py @@ -0,0 +1,71 @@ +"""Cuts inside a chain's segments (Task 13 Step 5, bench/records/check.md). + +The chain check (dw/pipeline_processors/chain_check.py) measures a segment at +its seam: the one frame step across it and a LOOK_WINDOW of frames either +side. A segment that cuts to another scene inside itself, past those windows, passes +it. This reads a finished take and its shots and reports, per segment, the +largest single-frame step inside the segment against the segment's own typical +step (`analyze_seams`' arithmetic: the 90th percentile, floored at +TYPICAL_DELTA_FLOOR) - the `jump_ratio` of a seam nobody marked - so a cut +ceiling for the check can be set from the bench's own takes. + +Usage (on lem, from a checkout, with the dw venv): + + python bench/cuts.py TAKE.mp4 ... +""" + +import json +import sys + +import numpy + +from dw.runs import shots_beside +from dw.tasks.assess import ( + TYPICAL_DELTA_FLOOR, + TYPICAL_DELTA_PERCENTILE, + read_media, +) + + +def inner_cut(thumbs, start, count): + """(ratio, frame) of the largest frame step inside thumbs[start:start + + count] over the span's floored typical step, or (None, None).""" + span = thumbs[start : start + count].astype(numpy.int16) + if span.shape[0] < 3: + return None, None + deltas = numpy.abs(numpy.diff(span, axis=0)).mean(axis=(1, 2)) + typical = max( + TYPICAL_DELTA_FLOOR, float(numpy.percentile(deltas, TYPICAL_DELTA_PERCENTILE)) + ) + at = int(deltas.argmax()) + return round(float(deltas[at]) / typical, 2), start + at + 1 + + +def take_cuts(path): + media = read_media(path) + shots = shots_beside(path) or [] + rows = [] + for index, shot in enumerate(shots): + ratio, frame = inner_cut( + media.thumbs, int(shot["start_frame"]), int(shot["num_frames"]) + ) + rows.append( + { + "segment": index + 1, + "inner_cut_ratio": ratio, + "at_frame": frame, + "attempts": shot.get("attempts"), + "check_failed": shot.get("check_failed"), + } + ) + return rows + + +def main(argv): + out = {path: take_cuts(path) for path in argv} + print(json.dumps(out, indent=1)) + return 0 + + +if __name__ == "__main__": + sys.exit(main(sys.argv[1:])) diff --git a/bench/driver.py b/bench/driver.py new file mode 100644 index 000000000..8b8168836 --- /dev/null +++ b/bench/driver.py @@ -0,0 +1,379 @@ +"""Long-form bench driver: submit arms to the bench dw.serve, wait, and +measure each output through the server's compare and assess routes (the +same routes the MCP tools call, so the tester can repeat any number). +Stdlib only; never fetches media files.""" + +import argparse +import json +import math +import os +import sys +import time +import urllib.error +import urllib.parse +import urllib.request +from dataclasses import dataclass +from pathlib import Path + +HERE = Path(__file__).resolve().parent +sys.path.insert(0, str(HERE)) + +DEFAULT_SERVER = "http://127.0.0.1:18766" +DEFAULT_TOKEN_FILE = "~/.dw-bench-token" +MAX_COMPARE_BYTES = 4_000_000 +MAX_COMPARE_MOMENTS = 8 +TERMINAL = ("succeeded", "failed", "cancelled") + + +@dataclass +class Arm: + name: str + scene: str + workflow: str # path under bench/workflows/ + arguments: dict + seed: int + workflow_id_suffix: str = "" # distinct inline id forces a step-cache miss + moments: list | None = None # overrides the scene's moments for this arm + + +@dataclass +class Scene: + name: str + crop: list + moments: list # "frame:N" at segments 1, middle and last + face_reference: str | None = None + lines: list | None = None + crop_verified: bool = False + + +def load_arms(path): + data = json.loads(Path(path).read_text()) + return [ + Arm( + name=a["name"], + scene=a["scene"], + workflow=a["workflow"], + arguments=a.get("arguments") or {}, + seed=a["seed"], + workflow_id_suffix=a.get("workflow_id_suffix", ""), + moments=a.get("moments"), + ) + for a in data + ] + + +def load_scenes(path): + data = json.loads(Path(path).read_text()) + return { + s["name"]: Scene( + name=s["name"], + crop=s["crop"], + moments=s["moments"], + face_reference=s.get("face_reference"), + lines=s.get("lines"), + crop_verified=s.get("crop_verified", False), + ) + for s in data + } + + +def http(method, path, server, token, payload=None, params=None): + """One JSON request; the single seam the tests fake.""" + url = server.rstrip("/") + path + if params: + url += "?" + urllib.parse.urlencode(params) + headers = {"Accept": "application/json"} + if token: + headers["Authorization"] = f"Bearer {token}" + data = None + if payload is not None: + data = json.dumps(payload).encode() + headers["Content-Type"] = "application/json" + request = urllib.request.Request(url, data=data, headers=headers, method=method) + try: + with urllib.request.urlopen(request, timeout=120) as response: + return json.loads(response.read() or b"null") + except urllib.error.HTTPError as e: + raise RuntimeError( + f"{method} {path}: {e.code} {e.read().decode()[:500]}" + ) from e + + +def _write_atomic(path, text): + """Write via a temp file and rename, so an interrupted run never leaves a + truncated record.""" + target = Path(path) + target.parent.mkdir(parents=True, exist_ok=True) + tmp = target.with_name(target.name + ".tmp") + tmp.write_text(text) + os.replace(tmp, target) + + +def health(server, token): + return http("GET", "/api/health", server, token) + + +def submit(arm, server, token, workspace=None): + workflow = json.loads((HERE / "workflows" / arm.workflow).read_text()) + if arm.workflow_id_suffix: + workflow["id"] = ( + workflow.get("id", Path(arm.workflow).stem) + arm.workflow_id_suffix + ) + payload = { + "workflow": workflow, + "arguments": {**arm.arguments, "seed": arm.seed}, + "note": f"bench arm {arm.name} (scene {arm.scene}, seed {arm.seed})", + "acknowledged_cost": True, + } + params = {"workspace": workspace} if workspace else None + job = http("POST", "/api/jobs", server, token, payload, params) + return job["id"] + + +def wait(job_id, server, token, timeout_s, poll_s=15): + deadline = time.monotonic() + timeout_s + while True: + job = http("GET", f"/api/jobs/{job_id}", server, token) + if job.get("status") in TERMINAL: + return job + if time.monotonic() >= deadline: + raise TimeoutError( + f"job {job_id} still {job.get('status')} after {timeout_s}s" + ) + time.sleep(poll_s) + + +def output_name(job): + """The gallery name of the job's deliverable video: the last video file + in the manifest, `final` subfolder first.""" + kinds = job.get("output_kinds") or {} + videos = [ + (entry.get("subfolder") == "final", name) + for entry in job.get("manifest") or [] + for name in entry.get("files") or [] + if kinds.get(name, "video") == "video" + ] + finals = [name for final, name in videos if final] + best = (finals or [name for _, name in videos] or [None])[-1] + if best is None: + raise ValueError(f"job {job.get('id')} wrote no video") + return best + + +def _mean(values): + return sum(values) / len(values) if values else None + + +def _stats(out, values, name_mean, name_max): + values = [v for v in values if v is not None] + if values: + out[name_mean] = _mean(values) + out[name_max] = max(values) + + +def measure(job, scene, server, token, workspace=None, moments=None): + name = output_name(job) + params = {"probe": "analyze_seams"} + if workspace: + params["workspace"] = workspace + seams = ( + http( + "GET", + f"/api/gallery/{urllib.parse.quote(name, safe='/')}/assess", + server, + token, + params=params, + ).get("seams") + or [] + ) + out = {} + _stats(out, [s.get("jump_ratio") for s in seams], "seam_jump_mean", "seam_jump_max") + _stats(out, [s.get("look_ratio") for s in seams], "seam_look_mean", "seam_look_max") + _stats( + out, + [abs(s["level_step_db"]) for s in seams if s.get("level_step_db") is not None], + "seam_level_step_db_mean", + "seam_level_step_db_max", + ) + at = moments or scene.moments + cells = [] + # the compare route takes 1 to MAX_COMPARE_MOMENTS moments per call + for i in range(0, len(at), MAX_COMPARE_MOMENTS): + payload = { + "names": [name], + "max_dimension": 384, + "max_total_bytes": MAX_COMPARE_BYTES, + "at": at[i : i + MAX_COMPARE_MOMENTS], + "crop": scene.crop, + "texture_bands": True, + } + compared = http( + "POST", + "/api/gallery/compare", + server, + token, + payload, + {"workspace": workspace} if workspace else None, + ) + row = (compared.get("texture") or [{}])[0] + cells += [c.get("texture") for c in row.get("cells") or []] + if len(cells) >= 2 and cells[0] and cells[-1]: + # Two-sided: blur (ratio < 1) and crust (ratio > 1) both count as + # drift, so the measure is |ln(last / first)| and lower is better. + for band in ("fine", "mid", "coarse"): + luma = cells[0].get("luma") or {}, cells[-1].get("luma") or {} + first, last = luma[0].get(band), luma[1].get(band) + if first and last and first > 0 and last > 0: + out[f"texture_{band}_seg_last"] = abs(math.log(last / first)) + if job.get("started_at") is not None and job.get("finished_at") is not None: + out["wall_minutes"] = (job["finished_at"] - job["started_at"]) / 60.0 + return out + + +def run_arms( + arms, scenes, server, token, do_measure, timeout_s, workspace=None, out_path=None +): + ids = {arm.name: submit(arm, server, token, workspace) for arm in arms} + records = [] + + def flush(): + if out_path: + _write_atomic(out_path, json.dumps(records, indent=2)) + + for arm in arms: + record = { + "arm": arm.name, + "scene": arm.scene, + "seed": arm.seed, + "job_id": ids[arm.name], + } + try: + job = wait(ids[arm.name], server, token, timeout_s) + record["status"] = job.get("status") + record["error"] = job.get("error") + if job.get("status") == "succeeded": + record["output"] = output_name(job) + if do_measure: + record["measures"] = measure( + job, scenes[arm.scene], server, token, workspace, arm.moments + ) + except Exception as exc: # one arm's failure must not lose the others + record.setdefault("status", "error") + record["error"] = f"{type(exc).__name__}: {exc}" + records.append(record) + flush() + return records + + +def measure_records( + records, scenes, arms, server, token, workspace=None, out_path=None +): + """Measure again, without running again, every succeeded record, with the + scene crops and moments as they are now (an arm's own moments, when an + arms file names one, win). Other records pass through untouched.""" + moments = {arm.name: arm.moments for arm in arms} + for record in records: + if record.get("status") != "succeeded": + continue + job = http("GET", f"/api/jobs/{record['job_id']}", server, token) + record["measures"] = measure( + job, + scenes[record["scene"]], + server, + token, + workspace, + moments.get(record["arm"]), + ) + record["error"] = None # a measure error from the run is now stale + if out_path: + _write_atomic(out_path, json.dumps(records, indent=2)) + return records + + +def report(records): + lines = [] + keys = sorted({k for r in records for k in r.get("measures", {})}) + lines.append("| arm | scene | seed | status | " + " | ".join(keys) + " |") + lines.append("|" + "---|" * (4 + len(keys))) + for r in records: + m = r.get("measures", {}) + cells = [f"{m[k]:.3f}" if k in m else "" for k in keys] + lines.append( + f"| {r['arm']} | {r['scene']} | {r['seed']} | {r['status']} | " + + " | ".join(cells) + + " |" + ) + return "\n".join(lines) + + +def _token(args): + if args.token_file: + return Path(args.token_file).expanduser().read_text().strip() + if os.environ.get("BENCH_TOKEN"): + return os.environ["BENCH_TOKEN"] + path = Path(DEFAULT_TOKEN_FILE).expanduser() + if path.is_file(): + return path.read_text().strip() + raise SystemExit( + f"no bench token: pass --token-file, set BENCH_TOKEN, or write {DEFAULT_TOKEN_FILE}" + ) + + +def main(argv=None): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument( + "--server", default=os.environ.get("BENCH_SERVER", DEFAULT_SERVER) + ) + parser.add_argument("--token-file") + parser.add_argument("--workspace") + sub = parser.add_subparsers(dest="command", required=True) + run = sub.add_parser("run", help="submit arms, wait, optionally measure") + run.add_argument("arms") + run.add_argument("--scenes", default=str(HERE / "scenes.json")) + run.add_argument("--measure", action="store_true") + run.add_argument("--out") + run.add_argument("--timeout", type=int, default=7200) + meas = sub.add_parser( + "measure", help="measure a record's succeeded jobs again; rewrites it" + ) + meas.add_argument("record") + meas.add_argument("--scenes", default=str(HERE / "scenes.json")) + meas.add_argument("--arms", help="arms file whose per-arm moments apply") + rep = sub.add_parser("report", help="markdown tables from records") + rep.add_argument("records", nargs="+") + args = parser.parse_args(argv) + if args.command == "report": + for path in args.records: + print(f"### {path}\n") + print(report(json.loads(Path(path).read_text()))) + print() + return 0 + token = _token(args) + if args.command == "measure": + measure_records( + json.loads(Path(args.record).read_text()), + load_scenes(args.scenes), + load_arms(args.arms) if args.arms else [], + args.server, + token, + args.workspace, + args.record, + ) + return 0 + records = run_arms( + load_arms(args.arms), + load_scenes(args.scenes) if args.measure else {}, + args.server, + token, + args.measure, + args.timeout, + args.workspace, + args.out, + ) + if not args.out: + print(json.dumps(records, indent=2)) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/bench/records/audio-long-2.json b/bench/records/audio-long-2.json new file mode 100644 index 000000000..218e0d613 --- /dev/null +++ b/bench/records/audio-long-2.json @@ -0,0 +1,42 @@ +[ + { + "arm": "s6-music-full-335s-s1166", + "scene": "s6-narration", + "seed": 1166, + "job_id": "ae2e1373cff6", + "status": "succeeded", + "error": "ValueError: job ae2e1373cff6 wrote no video" + }, + { + "arm": "s6-music-full-335s-s1167", + "scene": "s6-narration", + "seed": 1167, + "job_id": "f83cf3566727", + "status": "succeeded", + "error": "ValueError: job f83cf3566727 wrote no video" + }, + { + "arm": "s6-music-full-335s-xf3000-s1166", + "scene": "s6-narration", + "seed": 1166, + "job_id": "ef47c7b6684d", + "status": "succeeded", + "error": "ValueError: job ef47c7b6684d wrote no video" + }, + { + "arm": "s6-narr-long-t37-wps3-s1166", + "scene": "s6-narration", + "seed": 1166, + "job_id": "847ccb839875", + "status": "succeeded", + "error": "ValueError: job 847ccb839875 wrote no video" + }, + { + "arm": "s6-narr-long-t37-wps3-s1167", + "scene": "s6-narration", + "seed": 1167, + "job_id": "4467e6380276", + "status": "succeeded", + "error": "ValueError: job 4467e6380276 wrote no video" + } +] \ No newline at end of file diff --git a/bench/records/audio-long-prep.json b/bench/records/audio-long-prep.json new file mode 100644 index 000000000..7a3a4dcd0 --- /dev/null +++ b/bench/records/audio-long-prep.json @@ -0,0 +1,10 @@ +[ + { + "arm": "s6-narrator-ref-s1166", + "scene": "s6-narration", + "seed": 1166, + "job_id": "5623e25a933a", + "status": "succeeded", + "error": "ValueError: job 5623e25a933a wrote no video" + } +] \ No newline at end of file diff --git a/bench/records/audio-long.json b/bench/records/audio-long.json new file mode 100644 index 000000000..6952045b1 --- /dev/null +++ b/bench/records/audio-long.json @@ -0,0 +1,58 @@ +[ + { + "arm": "s6-narr-long-240s-t60-s1166", + "scene": "s6-narration", + "seed": 1166, + "job_id": "3668ebd2e4de", + "status": "succeeded", + "error": "ValueError: job 3668ebd2e4de wrote no video" + }, + { + "arm": "s6-narr-long-240s-t60-s1167", + "scene": "s6-narration", + "seed": 1167, + "job_id": "c9f1f6145c3c", + "status": "succeeded", + "error": "ValueError: job c9f1f6145c3c wrote no video" + }, + { + "arm": "s6-narr-long-240s-t45-s1166", + "scene": "s6-narration", + "seed": 1166, + "job_id": "111a5623cba0", + "status": "succeeded", + "error": "ValueError: job 111a5623cba0 wrote no video" + }, + { + "arm": "s6-narr-long-240s-gap005-s1166", + "scene": "s6-narration", + "seed": 1166, + "job_id": "c221ffe761a9", + "status": "succeeded", + "error": "ValueError: job c221ffe761a9 wrote no video" + }, + { + "arm": "s6-music-long-335s-s1166", + "scene": "s6-narration", + "seed": 1166, + "job_id": "52303a4436d4", + "status": "succeeded", + "error": "ValueError: job 52303a4436d4 wrote no video" + }, + { + "arm": "s6-music-long-335s-s1167", + "scene": "s6-narration", + "seed": 1167, + "job_id": "2fc471b36521", + "status": "succeeded", + "error": "ValueError: job 2fc471b36521 wrote no video" + }, + { + "arm": "s6-music-long-335s-xf3000-s1166", + "scene": "s6-narration", + "seed": 1166, + "job_id": "3bfee8aad7c5", + "status": "succeeded", + "error": "ValueError: job 3bfee8aad7c5 wrote no video" + } +] \ No newline at end of file diff --git a/bench/records/audio-pass.json b/bench/records/audio-pass.json new file mode 100644 index 000000000..4234c0a15 --- /dev/null +++ b/bench/records/audio-pass.json @@ -0,0 +1,83 @@ +[ + { + "arm": "ap-s4-cold_open-ls1", + "scene": "s4-montage", + "seed": 1166, + "job_id": "0e13c36a659b", + "status": "succeeded", + "error": null, + "output": "MiniMaxH3AddSound-ap/20261106-002201-e099b1ba/MiniMaxH3AddSound-ap-sound.0-0.0.mp4" + }, + { + "arm": "ap-s4-deflect-ls1", + "scene": "s4-montage", + "seed": 1166, + "job_id": "adfb95588730", + "status": "succeeded", + "error": null, + "output": "MiniMaxH3AddSound-ap/20261106-002706-a8cc5bf9/MiniMaxH3AddSound-ap-sound.0-0.0.mp4" + }, + { + "arm": "ap-s4-react-ls1", + "scene": "s4-montage", + "seed": 1166, + "job_id": "9f76f275eb8b", + "status": "succeeded", + "error": null, + "output": "MiniMaxH3AddSound-ap/20261106-003015-3fe5ff5d/MiniMaxH3AddSound-ap-sound.0-0.0.mp4" + }, + { + "arm": "ap-s4-button-ls1", + "scene": "s4-montage", + "seed": 1166, + "job_id": "a37bd9bc70d1", + "status": "succeeded", + "error": null, + "output": "MiniMaxH3AddSound-ap/20261106-003323-9a93c662/MiniMaxH3AddSound-ap-sound.0-0.0.mp4" + }, + { + "arm": "ap-s4-cold_open-ls050", + "scene": "s4-montage", + "seed": 1166, + "job_id": "a1d731ddbf29", + "status": "succeeded", + "error": null, + "output": "MiniMaxH3AddSound-ap/20261106-003630-0dd54320/MiniMaxH3AddSound-ap-sound.0-0.0.mp4" + }, + { + "arm": "ap-s4-cold_open-ls025", + "scene": "s4-montage", + "seed": 1166, + "job_id": "c92a26d1c1d3", + "status": "succeeded", + "error": null, + "output": "MiniMaxH3AddSound-ap/20261106-003943-56ccfd42/MiniMaxH3AddSound-ap-sound.0-0.0.mp4" + }, + { + "arm": "ap-s4-cold_open-foley-ls1", + "scene": "s4-montage", + "seed": 1166, + "job_id": "b51cc152524e", + "status": "succeeded", + "error": null, + "output": "MiniMaxH3AddSound-ap/20261106-004224-f12f561f/MiniMaxH3AddSound-ap-sound.0-0.0.mp4" + }, + { + "arm": "ap-s3-seg1-same-line-ls1", + "scene": "s3-dialogue-walk", + "seed": 1166, + "job_id": "864375a2cef2", + "status": "succeeded", + "error": null, + "output": "MiniMaxH3AddSoundSpan-ap/20261106-004532-3a4001fc/final/MiniMaxH3AddSoundSpan-ap-sound.1-0.0.mp4" + }, + { + "arm": "ap-s3-seg1-new-line-ls1", + "scene": "s3-dialogue-walk", + "seed": 1166, + "job_id": "4b6ebb139735", + "status": "succeeded", + "error": null, + "output": "MiniMaxH3AddSoundSpan-ap/20261106-004840-98cb5c66/final/MiniMaxH3AddSoundSpan-ap-sound.1-0.0.mp4" + } +] \ No newline at end of file diff --git a/bench/records/audio.md b/bench/records/audio.md new file mode 100644 index 000000000..7e75147b0 --- /dev/null +++ b/bench/records/audio.md @@ -0,0 +1,553 @@ +# Audio pass, long narration and long music (Task 21 Step 6, Task 22 Step 6) + +2026-11-05. Bench server lem:8766 (worktree at `6a89744a` when measured), cuda:0. +Arms: `bench/arms/audio-pass.json`, `audio-long.json`, `audio-long-prep.json`; +records: `audio-pass.json`, `audio-long.json`, `audio-long-prep.json` (the long +arms' "wrote no video" error text is the driver's, expected for audio-only output). + +How it was measured: outputs copied off lem and read on the Mac. ffmpeg for PSNR, +frame hashes, `ebur128` and `silencedetect`; the server's own `assess` route for the +shot probe; `check_script` (whisper-base) and a per-take whisper-small pass, run +locally on the waveforms; `analyze_beats` run locally on each music section; ECAPA +speaker embeddings (`speechbrain/spkrec-ecapa-voxceleb`) for voice drift. No GPU job +was submitted. Claude cannot judge sound by ear: the listening list is in `for-don.md`. + +## Task 21: audio pass (#1192) + +**Verdict: the mechanism works.** The output plays the source's own frames, and each +track is new audio (correlation with the source track is about 0), so this is not a +copy. Every scripted line is spoken word for word. The levels are sane, with no +clipping. `listen_scale` 0.5 and 0.25 cost less, but this run lost most of the saving +to a first-pass stall. They also put the line 0.7-0.9 s earlier than the line in the +source picture, so lip sync needs Don's eye before either becomes a default. Same seed +throughout, one draw per arm. + +### Don's ear (2026-11-06) + +- **ls1 (A): pass**, "very good lip sync". Shipped at ls1 as + `templates/minimax/add-sound`. +- **ls0.5 (B1): ok**, lip sync acceptable. A speed setting (about 23% faster with a + normal first pass), and the fastest scale that keeps lip sync. +- **ls0.25 (B2): fails lip sync.** Kept as a knob, for a clip with no on-screen speech. +- **S3 re-voice (C), same line and new line: sounds dubbed.** "The c plays do not + have lip sync at all. The character does not appear to be speaking and the dialog + seems like voice over." The picture is the source's own pixels, so a new or + regenerated line has no mouth drawn for it. The pass sounds a silent clip whose + speech, if any, was generated with that picture; it is not a re-voice for an + on-screen speaker, only for an off-screen or voice-over line. + +Note: the S4 "silent" shots are not silent. Each S4 source shot carries its own H3 +track at about -23 LUFS with its line spoken. The pass replaced that track. + +### Picture + +| arm | PSNR vs source (Y / avg / min frame) | decoded frames | +|---|---|---| +| S4 cold_open ls1 / ls0.5 / ls0.25 / foley | 41.6 / 42.9 / 40.7 dB | all four bit-identical to each other | +| S4 deflect ls1 | 42.2 / 43.8 / 42.3 | | +| S4 react ls1 | 42.7 / 44.0 / 42.5 | | +| S4 button ls1 | 42.4 / 43.9 / 42.2 | | +| S3 span same-line / new-line | inf vs the trimmed span (byte-identical stream); 42.8 / 44.2 / 42.5 vs the S3 take | identical to each other | + +About 43 dB is one x264 generation: the S3 trim alone measures the same (42.8 dB) +against the take it was cut from. A VAE round trip would be far lower. For a control, +a wrong-shot pair (cold_open output against the deflect source) measures 14.2 dB. The +picture does not depend on `listen_scale`. + +### Track + +Levels on each 5.17 s track: `ebur128` integrated loudness (I) and true peak; `assess` +dead air (the longest gap below the line's floor); silence share is the 50 ms windows +under -50 dBFS. Clipped samples: 0 everywhere. + +| arm (job) | I LUFS | true peak dBFS | LRA | silent share | dead air (at) | line heard | line span s (source's) | +|---|---|---|---|---|---|---|---| +| cold_open ls1 (`0e13c36a659b`) | -22.8 | -8.0 | 4.4 | 0.50 | 1.2 s (0.0) | exact | 1.22-4.18 (1.14-4.18) | +| cold_open ls0.5 (`a1d731ddbf29`) | -24.1 | -8.1 | 6.3 | 0.44 | 0.9 s (2.0) | exact | 0.46-3.70 | +| cold_open ls0.25 (`c92a26d1c1d3`) | -21.5 | -7.9 | 4.8 | 0.32 | 0.3 s | exact | 0.26-3.20 | +| cold_open foley ls1 (`b51cc152524e`) | -21.5 | -11.1 | 15.2 | 0.59 | 2.9 s (0.0) | no line asked; Whisper heard "Thank you very much." | - | +| deflect ls1 (`adfb95588730`) | -23.9 | -12.1 | 3.6 | 0.34 | 1.05 s (1.0) | exact, then laughter | 0.32-3.42 (0.32-3.56) | +| react ls1 (`9f76f275eb8b`) | -23.8 | -11.9 | 3.1 | 0.19 | 0.8 s (0.0) | exact, then laughter | 0.82-3.34 (0.90-3.38) | +| button ls1 (`a37bd9bc70d1`) | -20.1 | -7.5 | 8.4 | 0.34 | 1.45 s (0.0) | exact, then laughter | 1.48-2.30 (0.56-2.32) | +| S3 seg1 same line (`864375a2cef2`) | -21.5 | -7.6 | 2.8 | 0.51 | none | exact | 1.96-4.46 (1.20-5.14) | +| S3 seg1 new line (`4b6ebb139735`) | -20.2 | -7.3 | 1.5 | 0.43 | none | exact | 1.20-4.12 | +| *sources:* S4 cold_open / deflect / react / button | -24.5 / -23.1 / -22.3 / -23.5 | -9.9 / -5.7 / -11.2 / -9.4 | | | 1.15 / 1.05 / 0.9 / 0.85 s | | | +| *source:* S3 seg1 span | -14.2 | -2.2 | 3.4 | 0.06 | none | | | + +- The new S4 tracks sit within about 2 LU of the source tracks, and their dead air is + the same kind the sources had. The S3 re-voices are 6-7 LU quieter than the S3 take, + whose bed is loud in the low band (-19.8 vs -29.1 dBFS). +- On deflect and react the line starts within 0.1 s of where the source's line starts. + Button starts 0.9 s later and is shorter. The S3 same-line re-voice starts 0.76 s + late and ends 0.68 s early, against a mouth that moves for the source's 3.9 s line. + Whether that reads as lip-sync drift is Don's call. +- `listen_scale` moves the line: ls1 starts 0.08 s after the source's line, ls0.5 + 0.68 s before it, ls0.25 0.88 s before it. That is one seed, but a coarser view of + the face plausibly loosens sync. +- The foley arm asked for no speech. Whisper heard "Thank you very much." (probably a + hallucination over laughter), and the first 2.9 s are dead air, though the prompt + asks for footsteps as he enters. + +### listen_scale cost (cold_open, warm pipeline) + +| | ls1 (mean of 6 warm ls1 runs) | ls0.5 | ls0.25 | +|---|---|---|---| +| job wall s | 188 | 193.5 | 160.5 | +| encode s | 44.5 | 26.8 | 15.9 | +| first denoise pass s | 13.4 | **61.5** (stall) | **45.3** (stall) | +| steady pass s | 10.2 | 8.7 | 8.7 | +| decode s | 20.1-22.2 | 10.2 | 4.7 | +| wall with a normal first pass (est.) | 188 | ~145 (-23%) | ~129 (-31%) | + +Both reduced-scale runs hit the engine's "first denoise pass took 45-62 s" warm-up +warning: a new shape after the ls1 runs. The steady pass falls only 15%, so the +audio tokens and the text dominate a pass. The first ls1 run (`0e13c36a659b`, 312 s) +included a 115.8 s cold load. + +**Recommendation:** promote add-sound to `templates/minimax/add-sound` at ls1 if Don's +ear passes it. Keep 0.5 and 0.25 as knobs, not defaults, until a lip-sync read says +the earlier line is harmless. + +## Task 22: long narration (S6) + +**Verdict: `max_seconds` 60 is not safe. Take 45 is clean, and 45 should be the +default.** *(Superseded 2026-11-06: the shipped default is 37 s at 3.0 words a second, +50e036fe, because the safe limit is about 111 words; it is word-complete on two seeds, +see [S6 rerun](#s6-rerun-fixed-templates-mini-ai). `gap_seconds` stays 0.3 by Don's +ear, see *Joins*.)* At 60, 5 of the 10 takes (two seeds) drop or loop words. At 45, none of the +6 takes does. The join level steps are within the in-take spread. The join pause at +`gap_seconds` 0.3 is at or under a normal sentence pause. At 0.05 the words run +together at two joins. ECAPA finds no voice drift between takes. The narration is +also shorter than designed: the voice speaks at about 3.0 words a second, not the 2.5 +assumed. So the 594-word text runs 198 s at take 45, not 240 s. The t60 runs land +just under the 3-minute gate (178-180 s), because words were dropped. + +### split_takes + +594 words. `split_takes` agrees with the job's step count (5 takes at 60, 6 at 45). It +kept "Dr. Alder" whole, and no sentence was over a limit, so no comma cut was used. +The takes are packed greedily across paragraph breaks. At 60, take 1 ends on the +first sentence of paragraph 2 ("On the night the storm came in, the wind arrived +first."). So a join falls mid-paragraph, while a paragraph break inside a take gets +only a sentence pause. + +| limit | words per take | take 1-n end on | +|---|---|---| +| 60 s (150 words) | 125, 137, 127, 143, 62 | "...wind arrived first." / "...nothing to be done." / "...for a long while." / "...away from the stones." / end | +| 45 s (112 words) | 103, 109, 86, 91, 106, 99 | "...stool to wait." / "...break white." / "...he saw the ship." / "...for a long while." / "...while it turned." / end | + +### Words heard (per-take whisper-small; whisper-base check_script agrees) + +| run (job) | file s | take: words heard / expected (take s) | what is missing | +|---|---|---|---| +| t60 s1166 (`3668ebd2e4de`) | 179.7 | 83/125 (30.8), 137/137, 127/127, 127/144, 62/62 | take 1 says "He trimmed the wick, wiped the salt from the glass" twice (24.2 s and 27.4 s), then stops, dropping the rest of paragraph 1 and paragraph 2's first sentence. Take 4 jumps from "while it turned" to "...every eight seconds the boat turned" (about 153 s): "the beam swept across the water, and every eight seconds it found the boat" is not said | +| t60 s1167 (`c9f1f6145c3c`) | 178.3 | 125/125, 135/137, 98/127 (30.2), 126/144, 62/62 | take 2 stumbles ("does not all wince. This not all wince stands", about 77-80 s) and cuts off at "because there..." (85.6 s, the join). Take 3 stops at "trough of a wa-" (115.7 s), dropping 2 sentences. Take 4 has the same "every eight seconds" skip (about 151 s) | +| t45 s1166 (`111a5623cba0`) | 198.0 | 103/103, 109/109, 86/86, 91/91, 107/107, 99/99 | nothing (numbers transcribed as digits, punctuation only) | +| gap 0.05 s1166 (`c221ffe761a9`) | 178.4 | same speech as t60 s1166, sample for sample | same as t60 s1166 | + +The failures are a take that stops early, or a jump between repeated phrases ("every +eight seconds", three times in one sentence). Neither happens at 45, where the same +sentence is spoken whole. Complete takes run at 2.93-3.10 words a second, so a 60 s +take of 125-144 words is 41-48 s of speech, and a 45 s take is 29-37 s. The reference +is 4.55 s (-24.8 LUFS). The limit that bites is the take's speech length, well short +of the ~95 s context `voice-clone` cites. + +### Joins + +The join is found by its inserted digital silence. The digital gap is 0.241 s at +`gap_seconds` 0.3: the 30 ms crossfade is taken off twice. "Pause" is the whole quiet +run around the join, -45 dBFS at 20 ms. The level step compares 5 s of speech on each +side. + +| run | join at s | pause s | level step dB | +|---|---|---|---| +| t60 s1166 | 30.8 / 76.3 / 117.2 / 159.7 | 0.24 / 0.40 / 0.44 / 0.48 | +1.5 / +1.4 / -1.0 / +2.6 | +| t60 s1167 | 40.6 / 85.6 / 115.8 / 158.1 | 0.50 / 0.24 / 0.26 / 0.46 | +0.4 / +0.2 / -1.2 / +1.2 | +| t45 s1166 | 33.3 / 70.0 / 99.3 / 129.6 / 164.9 | 0.46 / 0.46 / 0.52 / 0.44 / 0.50 | +1.0 / +1.2 / +1.5 / -2.6 / +1.2 | +| gap 0.05 s1166 | 30.8 / 76.1 / 116.7 / 158.9 | 0.08 / 0.17 / 0.20 / 0.23 (5 ms read) | as t60 s1166 | + +- Floor: a sentence pause inside a take measures a median of 0.52-0.56 s (p90 + 0.76-0.79 s). The same 5 s level step taken at in-take pauses measures a median of + 0.6-0.9 dB, p90 1.4-1.8 dB and max 1.7-3.0 dB. Every join step (max 2.6 dB) is + inside that max. No loudness fix is needed: each file is -24.1 to -24.5 LUFS, true + peak -4.4 to -5.6 dBFS, no clipping. +- A join at 0.3 is 0.24-0.52 s, at or under the median sentence pause. The 0.24 s + joins are takes that end and start on speech. At 0.05, joins 1 and 2 have under a + fifth of a second between sentences. **Recommendation: `gap_seconds` 0.45.** That + puts the shortest join near 0.39 s and the typical one near the in-take median. + Don's ear decides. *(Superseded 2026-11-06, Don's ear: keep 0.3.)* +- **Don's ear (2026-11-06).** At 0.3 the joins are fine ("a titch rushed on + occasion") and the voice is fine, so `gap_seconds` stays **0.3**. At 0.05 he heard + the voice get louder at the joins: the +1.5 / +2.6 dB take-to-take steps above, + inside the in-take spread and masked by a 0.3 s pause, become audible once the + pause is gone. So: do not go below about 0.3, and no level fix is needed at the + default gap. +- The 0.05 arm is the t60 s1166 audio with each gap 0.25 s shorter (the takes match + sample for sample). So it isolates the gap: same seed, same takes, same draws. + +### Voice drift (ECAPA cosine) + +| run | take vs reference | adjacent takes | halves within one take (floor) | +|---|---|---|---| +| t60 s1166 | 0.82-0.85 | 0.93-0.95 | 0.91-0.95 | +| t60 s1167 | 0.81-0.84 | 0.92-0.96 | 0.92-0.96 | +| t45 s1166 | 0.79-0.85 | 0.94-0.95 | 0.92-0.96 | + +Takes match each other as closely as two halves of one take do. The lowest score is +the shortest final take each time. Take 2 of the two seeds scores 0.976 against each +other. There is no measured drift, so the ear has the last word. + +## Task 22: long music (S6) + +**Verdict: the gate is not met, and the cause is a template defect.** Each file is +335.5 s (331 s at xf3000), as the arithmetic says. But Music 3 wrote sections of only +26.8-70.1 s for 85 s asked (`audio_duration` 128 is a ceiling, and a 4-line lyric ends +early). `slice_audio` then pads each section to 85 s with digital silence, and the +server warned about it on every run. So the pieces hold 13-57 s silent holes between +sections, and 33-45 s of silence at the end. Music content is 245-246 s at seed 1166 +and 179 s at seed 1167, under one track's 300 s. Only one join per seed-1166 piece is +a real music-to-music crossfade. + +| run (job) | file s | music s | sections generated s (kept 85) | silent holes s (at) | +|---|---|---|---|---| +| s1166 xf1500 (`52303a4436d4`) | 335.5 | 246 | 52.1, 70.1, >=85, 39.8 | 31.5 (52.1), 13.5 (153.6), 45.2 tail (290.3) | +| s1167 xf1500 (`2fc471b36521`) | 335.5 | 179 | 63.2, 26.8, 38.0, 52.1 | 20.6 (63.1), 57.1 (110.3), 45.8 (205.0), 32.9 tail (302.6) | +| s1166 xf3000 (`3bfee8aad7c5`) | 331.0 | 244.5 | same draw as s1166 | 30.0 (52.1), 12.0 (152.1), 45.2 tail (285.8) | + +- Joins: the one real crossfade is s1166 section 3 into 4. Section 3 is cut at 85 s, + mid-music. The level step there is +0.6 dB at xf1500 (250.5 s) and 0.0 dB at xf3000 + (246 s). Every other join is a section's own ending, then silence, then a fresh + start. The step across it (+11.7, -10.0 dB at s1166; -3.7, -0.2, -8.1 dB at s1167) + compares a fade tail with an intro. It is not a seam figure. +- Tempo: `analyze_beats` finds a reliable pulse in only 5 of 12 sections (method + `onset`). Those read 84.5 and 87.0 BPM (s1166 sections 2 and 3, the same in both + crossfades) and 63.2 BPM (s1167 section 2), against the 72 BPM asked. The rest fall + back to loudness peaks (69-105 BPM, unreliable). The beat tracker cannot settle + tempo continuity on this soft folk. That goes to the ear. +- xf3000 against xf1500: the same seed gives the same sections, so the crossfade + changes only the one real join (0.0 vs +0.6 dB) and shortens the file by 4.5 s. + Nothing else can differ. +- Loudness: -18.1 / -18.6 / -18.1 LUFS (silence included). Sample peak -2.1 / -2.8 / + -1.4 dBFS against `normalize_audio`'s -3 dBFS. That is mp3 encode overshoot, true + peak up to -1.3 dBFS, with no clipping. + +**Defect (template `workflows/templates/minimax/music-long.json`):** the `trim` step +slices each section to `item:seconds` with no regard to the generated length. Any +section Music 3 ends early is padded with digital silence, and the crossfade then +joins silence to the next intro. Fix options: +1. Trim each section to `min(generated, seconds)`, cutting its trailing silence before + the fade. +2. Write lyrics long enough for 85 s: about 2-3x these 4-line sections. The skill + should say that the lyric, not `audio_duration`, sets the length. +3. Both, plus a check that refuses (or warns as a finding) a joined piece whose music + is under the requested total. + +The bench result cannot gate S6 music until it is rerun with longer lyrics or the +trim fix. + +## Outputs (lem `~/bench-workspace/outputs/`) + +| arm | job | file | +|---|---|---| +| ap-s4-cold_open-ls1 | `0e13c36a659b` | `MiniMaxH3AddSound-ap/20261106-002201-e099b1ba/MiniMaxH3AddSound-ap-sound.0-0.0.mp4` | +| ap-s4-deflect-ls1 | `adfb95588730` | `MiniMaxH3AddSound-ap/20261106-002706-a8cc5bf9/MiniMaxH3AddSound-ap-sound.0-0.0.mp4` | +| ap-s4-react-ls1 | `9f76f275eb8b` | `MiniMaxH3AddSound-ap/20261106-003015-3fe5ff5d/MiniMaxH3AddSound-ap-sound.0-0.0.mp4` | +| ap-s4-button-ls1 | `a37bd9bc70d1` | `MiniMaxH3AddSound-ap/20261106-003323-9a93c662/MiniMaxH3AddSound-ap-sound.0-0.0.mp4` | +| ap-s4-cold_open-ls050 | `a1d731ddbf29` | `MiniMaxH3AddSound-ap/20261106-003630-0dd54320/MiniMaxH3AddSound-ap-sound.0-0.0.mp4` | +| ap-s4-cold_open-ls025 | `c92a26d1c1d3` | `MiniMaxH3AddSound-ap/20261106-003943-56ccfd42/MiniMaxH3AddSound-ap-sound.0-0.0.mp4` | +| ap-s4-cold_open-foley-ls1 | `b51cc152524e` | `MiniMaxH3AddSound-ap/20261106-004224-f12f561f/MiniMaxH3AddSound-ap-sound.0-0.0.mp4` | +| ap-s3-seg1-same-line-ls1 | `864375a2cef2` | `MiniMaxH3AddSoundSpan-ap/20261106-004532-3a4001fc/final/MiniMaxH3AddSoundSpan-ap-sound.1-0.0.mp4` | +| ap-s3-seg1-new-line-ls1 | `4b6ebb139735` | `MiniMaxH3AddSoundSpan-ap/20261106-004840-98cb5c66/final/MiniMaxH3AddSoundSpan-ap-sound.1-0.0.mp4` | +| s6-narrator-ref-s1166 | `5623e25a933a` | `BenchS6NarratorReference/20261105-233044-a0358f86/final/BenchS6NarratorReference-narrator.0-0.0.wav` (asset `bench/narrator.wav`) | +| s6-narr-long-240s-t60-s1166 | `3668ebd2e4de` | `BenchS6NarrationLong/20261105-233313-a7732021/final/BenchS6NarrationLong-narration.0-0.0.wav` | +| s6-narr-long-240s-t60-s1167 | `c9f1f6145c3c` | `BenchS6NarrationLong/20261105-233643-19bc78a7/final/BenchS6NarrationLong-narration.0-0.0.wav` | +| s6-narr-long-240s-t45-s1166 | `111a5623cba0` | `BenchS6NarrationLong/20261105-234005-77b3c12f/final/BenchS6NarrationLong-narration.0-0.0.wav` | +| s6-narr-long-240s-gap005-s1166 | `c221ffe761a9` | `BenchS6NarrationLong/20261105-234351-b0a08747/final/BenchS6NarrationLong-narration.0-0.0.wav` | +| s6-music-long-335s-s1166 | `52303a4436d4` | `BenchS6MusicLong/20261105-234715-35bcd6ab/final/BenchS6MusicLong-piece.0-0.0.mp3` | +| s6-music-long-335s-s1167 | `2fc471b36521` | `BenchS6MusicLong/20261105-235710-f02aa4dc/final/BenchS6MusicLong-piece.0-0.0.mp3` | +| s6-music-long-335s-xf3000-s1166 | `3bfee8aad7c5` | `BenchS6MusicLong/20261106-000347-934cbc99/final/BenchS6MusicLong-piece.0-0.0.mp3` | + +Sources: S4 shots +`BenchS4Montage/20261104-040255-9813688b/intermediate/BenchS4Montage-shot@{cold_open.2,deflect.3,react.4,button.5}-0.0.mp4`; +S3 take +`BenchS3DialogueWalk/20261104-035218-d2733a3a/final/BenchS3DialogueWalk-chained_keyframe_to_video_audio.0-0.0.mp4` +(frames 0-123). + +## S6 rerun (fixed templates, mini-ai) + +2026-11-05. Bench server mini-ai:8766 (Apple M5 Pro, **MPS**, not lem's CUDA; the +server's worktree at `47a9e025`), after 237ee307 (music-long keeps a short section's +own length, `slice_audio` `past_end` "stop", `joined_short`) and 50e036fe +(narration-long defaults to 37 s takes at 3.0 words a second, paragraph-break +preference). Arms `bench/arms/audio-long-2.json`, record `audio-long-2.json` (the +"wrote no video" text is the driver's, as before). Same method as the first round, +run locally on the Mac on copies of the outputs. One draw per arm. + +**Verdict: Task 22's gate passes on both halves.** + +- **Narration, over 3 minutes and word-complete: passes.** Both seeds come back as + 7 takes, 202.9 s and 201.4 s. All 14 takes are word-complete under whisper-small + and whisper-base, apart from spelling and homophones (pane/pain, harbour/harbor, + toward/towards). Five of the six joins land on a paragraph break. The join level + steps (-1.3 to +1.5 dB) sit inside the in-take spread. ECAPA finds no voice drift. + So 37 s at 3.0 words a second holds on a second seed, where the first round had + only one seed at 45. +- **Music, over 5 minutes and continuous: passes.** The pieces are 319.2 s and + 311.5 s at xf1500, and 314.7 s at xf3000. There is no silence of 0.5 s at -50 dBFS + anywhere. Every join is music into music, and the outro is the song's own ending + (the last lyric line, then a decay), not a cut. +- **Remaining defect (music):** a section that runs *longer* than `seconds` is cut + at 85 s wherever that falls, and nothing warns unless the dropped remainder is + under 5% of the slice. With 10-12 lines, three of the four sections in each draw + ran past 85 s. The pass counts put the verses at about the 128 s ceiling. In + s1166 the cut drops verse_1's last three lines. In s1167 the cut lands on + verse_2's last line. Five of the six joins also drop 7-9 dB, as a section cut + mid-song runs into the next section's quiet intro. That is inside the in-section + maximum (6.3-12.7 dB), but well above its median (1.8-2.4 dB). + +### Narration t37 (`max_seconds` 37, `words_per_second` 3.0) + +`split_takes` gives 7 takes: 103, 109, 77, 88, 72, 83, 62 words. Take 1 ends +mid-paragraph ("...on the wooden stool to wait."). Paragraph 1 is 113 words, over +the 111-word limit, so its last sentence ("Then the light turned...") opens take 2, +and only a sentence pause leads into paragraph 2. The other five joins are paragraph +breaks: the preference worked wherever the limit allowed it. + +| run (job) | file s | take s (words a second, by take length) | words heard / expected, small and base | +|---|---|---|---| +| t37 s1166 (`847ccb839875`) | 202.9 | 34.2, 36.7, 26.5, 29.8, 26.7, 29.1, 20.0 (2.70-3.10) | all 7 complete. small: pane/pain, lantern/"lanterns". base: pane/pain, its/it's, harbour/harbor | +| t37 s1167 (`4467e6380276`) | 201.4 | 34.2, 36.7, 26.6, 29.8, 26.4, 27.6, 20.3 (2.73-3.05) | all 7 complete. small: pane/pain, lantern/"lanterns", toward/towards. base: turned/turn, pane/pain, harbour/harbor (x2) | +| *first round:* t45 s1166 (`111a5623cba0`) | 198.0 | 6 takes | all complete | +| *first round:* t60 s1166 / s1167 | 179.7 / 178.3 | 5 takes | 2 of 5 takes lose words in each | + +The longest take is 36.7 s of file (109 words), and it is complete on both seeds. +The voice still speaks at about 3.0 words a second on MPS. + +**Joins** (found by the inserted digital silence, |sample| <= 2 LSB; pause is the +quiet run at -45 dBFS in 20 ms windows; the level step compares 5 s of speech on each +side): + +| run | join at s | digital gap s | pause s | level step dB | +|---|---|---|---|---| +| t37 s1166 | 34.1 / 70.7 / 97.2 / 127.0 / 153.7 / 182.8 | 0.24-0.28 | 0.48 / 0.46 / 0.44 / 0.42 / 0.44 / 0.36 | +1.3 / -0.3 / -0.5 / +0.5 / -0.8 / +1.5 | +| t37 s1167 | 34.0 / 70.7 / 97.3 / 127.0 / 153.4 / 181.0 | 0.24-0.27 | 0.44 / 0.44 / 0.42 / 0.38 / 0.42 / 0.44 | +0.1 / +0.3 / +0.7 / -1.3 / +0.1 / +1.4 | + +- Floor: in-take sentence pauses have a median of 0.60 s and 0.56 s (p90 0.78 and + 0.84 s). In-take 5 s level steps have a median of 0.7-0.8 dB, p90 1.6-1.8 dB and + max 3.0-3.6 dB. Each file is -24.2 and -24.0 LUFS, true peak -4.5 and -4.9 dBFS, + with no clipping. +- The bench still runs `gap_seconds` 0.3, so every join (0.36-0.48 s) is shorter + than the median sentence pause, as in the first round. The 0.45 recommendation + stands for Don's ear. *(Superseded 2026-11-06: Don kept 0.3; see the first round's + *Joins*.)* + +**Voice drift (ECAPA cosine):** + +| run | take vs reference | adjacent takes | halves within one take (floor) | +|---|---|---|---| +| t37 s1166 | 0.81-0.86 | 0.91-0.96 | 0.91-0.95 | +| t37 s1167 | 0.81-0.86 | 0.90-0.96 | 0.90-0.95 | + +The same take across the two seeds scores 0.97-0.98. The lowest adjacent pair is +the last join both times (0.91 and 0.90), against the shortest take. That is at the +within-take floor, so there is no measured drift. + +### Music, 10-12 lyric lines a section (`bench/workflows/music-long-s6-full.json`) + +| run (job) | file s | sections kept s | I LUFS / LRA | true peak dBFS | +|---|---|---|---|---| +| s1166 xf1500 (`ae2e1373cff6`) | 319.18 | 85, 85, 85, 68.69 (`joined_short`) | -18.2 / 11.9 | -2.0 | +| s1167 xf1500 (`f83cf3566727`) | 311.45 | 85, 85 (of 87.34; `slice_trimmed_tail` warned), 85, 60.95 (`joined_short`) | -17.3 / 10.4 | -2.0 | +| s1166 xf3000 (`ef47c7b6684d`) | 314.68 | as s1166 | -18.2 / 11.8 | -2.1 | + +**Generated lengths:** the server keeps no intermediate file, so the lengths past +85 s are inferred. Each section's denoise passes are a multiple of 30 (30 steps +times about 4.2 s chunks), which fits all three known lengths (60.95, 68.69 and +87.34 s). On that model: verse_1 is 31 chunks in both draws (about 125-128 s, the +ceiling). chorus_1 is 25 chunks at s1166 (about 100-107 s). verse_2 is 26 chunks at +s1166 (about 104-112 s) and 31 at s1167 (about 125-128 s). chorus_2 ended early in +both draws (68.69 and 60.95 s). So the longer lyric overshoots in the verses +(8 verse lines plus a 2-line pre-chorus or bridge) and undershoots in the last +chorus. + +**Silence:** silencedetect at -50 dBFS/0.5 s finds nothing in any piece. At +-40 dBFS/0.2 s it finds only the 0.28-0.33 s lead-in at 0 s, three 0.27-0.33 s dips +inside s1166's verse_1 (43-46 s, between phrases), one 0.27 s dip at 82.0 s, single +0.24 s dips at 196 s and 294 s, and each piece's last 1-2 s of decay. These are not +holes. + +**Joins** (the level step compares the 5 s before the crossfade with the 5 s after +it; in-section floor = |5 s step| every 5 s inside sections): + +| run | crossfade s | level step dB | in-section floor median / p90 / max | +|---|---|---|---| +| s1166 xf1500 | 83.5-85 / 167-168.5 / 250.5-252 | -8.8 / -8.9 / -9.4 | 2.4 / 6.9 / 12.7 | +| s1167 xf1500 | 83.5-85 / 167-168.5 / 250.5-252 | -0.8 / -7.8 / -7.4 | 1.8 / 4.3 / 6.3 | +| s1166 xf3000 | 82-85 / 164-167 / 246-249 | -10.2 / -4.7 / -7.2 | 2.4 / 6.9 / 12.7 | + +The negative steps are a full band cut at 85 s running into the next section's own +sparse intro. Each section opens like a new song. The one flat join (s1167, 85 s) +is verse_1's instrumental tail into chorus_1's opening. + +**What is sung at each cut** (whisper-small with word timestamps over the 20 s +before each crossfade; sung-lyric ASR is lossy, so read it as where the voice is, +not as a lyric check): + +| run | section, cut at | last words heard before the cut | cut verdict | +|---|---|---|---| +| s1166 | verse_1, 85 | "Salt on the glass, I wipe it clear" ends at 80.8 | **lyric lost:** "The only work that matters here" and the 2-line pre-chorus are never sung | +| s1166 | chorus_1, 168.5 | "Every eight seconds, find them there" ends at 157.7 | complete; about 9 s of band after it, then cut mid-music | +| s1166 | verse_2, 252 | bridge sung by 240.2, then a repeat "Somewhere I'll save, somewhere I'll..." at 249.4 | **cut mid-phrase** (a repeated line, level -20 to -25 dB up to the cut) | +| s1167 | verse_1, 85 | pre-chorus "...comes to the door" ends at 77.5 | complete; band tail cut | +| s1167 | chorus_1, 168.5 | post-chorus repeat "...I'm turning for you" ends at 162.0 | complete. The dropped 2.34 s tail is band only, more than 6 s after the last word: **no phrase cut**, but the section's own ending is replaced by the 400 ms fade | +| s1167 | verse_2, 252 | "Somewhere a sail / a hand / a light..." at 241.8-249.8; "they understand" heard inside the crossfade (250.5-252) | **last line lands on the cut** | +| both | chorus_2, end | "One hundred stairs and twelve more still" ends at 315.6 / 308.8; level -33, then -54 dBFS within 1 s | **a real ending:** the lyric's last line, then a decay | + +**Tempo** (`analyze_beats` per section, crossfades excluded; 72 BPM asked): only 4 +of 8 xf1500 sections give an onset pulse. s1166 chorus_1 and verse_2 read 84.5 and +87.0 BPM (the same as the first round's s1166 sections 2 and 3). s1167 chorus_1 and +verse_2 read 92.3 and 77.9 BPM. The rest fall back to loudness peaks (83-140 BPM, +unreliable). Steering to 72 changes nothing. Music 3 does not hold 72, and the tempo +moves between sections by 3-14 BPM where it can be read. + +**Key** (a whole-section chroma against Krumhansl profiles: crude, indicative only; +D minor asked). s1166 reads F major/D minor (the relative pair), Bb major/G minor, +D major, then D major/minor. s1167 reads F major, D major/B minor, C major (0.90), +then A major (0.80). The clearest key change at a seam is s1167's last join: the +10 s before it reads C major (0.95), and the 10 s after it reads A major (0.81). +That goes to the ear. + +**xf3000 against xf1500:** the sections are sample-identical (the first 80 s differ +by 0.0), so only the joins differ. The file is 4.5 s shorter. The steps are -10.2 / +-4.7 / -7.2 dB against -8.8 / -8.9 / -9.4. A longer fade does not hide a band +dropping to an intro, and the intro still arrives. + +**Wall time on MPS:** 40.2, 39.1 and 39.2 min per music job (four sections, encode +174-375 s and denoise 168-345 s per section, 0.4 s a pass). 4.1 and 3.8 min per +narration job. + +**Fix options for the long-section cut** (not made here): +1. Give `audio_duration` less headroom than 1.5x (for example 1.1x), so a long + lyric is squeezed rather than overrun. That is untested, and Music 3 may then + end sections early again. +2. Have `slice_audio` (or music-long) warn when the dropped remainder of a + `past_end` "stop" slice is loud. The current warning only covers remainders + under 5% of the slice and 10 s, so a 40 s overrun is silent. +3. Size the lyric per section: here the 8- and 10-line choruses filled 61-107 s, + while 8 verse lines plus 2 more ran to 104-128 s. + +### Music with the 8 s end snap + +2026-11-06. Arms `bench/arms/music-snap.json`, record `music-snap.json` (the "wrote +no video" text is the driver's). Same workflow and seeds as the no-snap takes +above, after d1d19120 (`slice_audio` `end_snap_seconds`, `slice_dropped_loud_tail`; +music-long `section_end_snap_seconds` 8). Jobs `2179e0c0c9df` (s1166) and +`d51816c55389` (s1167). + +**Verdict: the cut is fixed, the level step is not.** + +- **The cut is fixed.** All six cuts moved 0.6-7.0 s back to a quiet window. None + lands mid-word: each falls after a line's last word or before the next onset. +- **The level step is not fixed.** Four of six joins still drop 7.3-9.7 dB, the same + as the no-snap takes' 7.4-8.9 dB. The step is the next section's sparse intro, and + a cut point cannot change that. The one join that improved (s1166 verse_2, -9.4 to + -1.1 dB) did so because its cut moved 7 s back into a quiet passage. +- **The lyric is still lost.** The snap keeps nothing more. The overrun is lyric + length, as before. s1166 verse_1 still loses "The only work that matters here" and + the pre-chorus. s1167 verse_2 now drops "Somewhere a light they understand" whole, + where the no-snap take cut it mid-line. +- **The warning is right on all six cuts.** Each dropped tail is music still playing, + 0.2-2.1 dB from the kept part's level. + +The generation is the same draw as the no-snap takes: same pass counts per section, +and the first 70 s of each file are sample-identical to `ae2e1373cff6` and +`f83cf3566727`. So the no-snap file is the source up to 83.5 s of each section, and +the two can be compared cut for cut. + +| run (job) | file s | sections kept s (moved back) | I LUFS / LRA | true peak dBFS | +|---|---|---|---|---| +| s1166 (`2179e0c0c9df`) | 308.76 (was 319.18) | 82.17 (2.83), 84.36 (0.64), 78.05 (6.95), 68.69 (`joined_short`) | -18.2 / 11.9 | -2.3 | +| s1167 (`d51816c55389`) | 297.71 (was 311.45) | 78.94 (6.06), 81.00 (4.00), 81.32 (3.68), 60.95 (`joined_short`) | -17.3 / 10.4 | -2.1 | + +**Cuts.** The window level is the quietest 0.2 s RMS window, as the job log +reports it. The section mean is the kept RMS from the warning. Words are from +whisper-small with word timestamps, run on the no-snap file over the 20 s before the +cut (up to 8 s after it where the source reaches that far). + +| run, section | cut s (source) | window dBFS / section mean / delta dB | what is around the cut | mid-word? | +|---|---|---|---|---| +| s1166 verse_1 | 82.17 | -51.1 / -18.6 / -32.5 | "...I wipe it clear" ends at 81.2; the next onset is 0.15 s after the cut | no, a gap between lines | +| s1166 chorus_1 | 84.36 | -43.8 / -18.1 / -25.7 | band only; "...find them there" ends at 74.9 | no, a gap in the band | +| s1166 verse_2 | 78.05 | -30.3 / -20.3 / -10.0 | the bridge is complete by 74.3. The band keeps going at about -30 dBFS, with guitar plucks at -21 every 0.7 s. The cut falls between plucks, 0.85 s before the bridge repeat ("Somewhere...") comes in | no, but it is not a gap either: the band is playing | +| s1167 verse_1 | 78.94 | -35.9 / -21.1 / -14.8 | the pre-chorus "...comes to the door" ends at 77.9 | no, after the line | +| s1167 chorus_1 | 81.00 | -37.9 / -15.0 / -22.9 | "...I'm turning for you" ends at 79.1; band after that | no, a gap in the band | +| s1167 verse_2 | 81.32 | -31.8 / -17.5 / -14.3 | "Somewhere a sail, somewhere a hand" is sung. The last line's "Somewhere" starts 0.1-0.2 s after the cut (onset at -9.5 dBFS) | no, between lines; the last line is dropped | + +Whisper places s1166 verse_2's "Somewhere" across the cut (from 244.25 s in the +no-snap file). A 50 ms level profile puts the voice's onset at 245.9 s, after the cut +at 245.05 s, so whisper's start time is early. Whisper run on the new files' last +8 s before each cut hears no partial word. The one exception is s1166 chorus_1, where +it hallucinates on band-only audio. + +**Joins** (the same method as the no-snap table above: 5 s before the crossfade +against 5 s after it. The same script reproduces that table's no-snap steps +exactly): + +| run | crossfade s (file) | level step dB | no-snap step dB | in-section floor median / p90 / max | +|---|---|---|---|---| +| s1166 join 1 (verse_1 into chorus_1) | 80.67-82.17 | -9.7 | -8.8 | 2.0 / 5.5 / 9.1 | +| s1166 join 2 (chorus_1 into verse_2) | 163.53-165.03 | -7.4 | -8.9 | | +| s1166 join 3 (verse_2 into chorus_2) | 240.08-241.58 | **-1.1** | -9.4 | | +| s1167 join 1 | 77.44-78.94 | -1.5 | -0.8 | 1.3 / 4.6 / 7.3 | +| s1167 join 2 | 156.94-158.44 | -9.4 | -7.8 | | +| s1167 join 3 | 236.76-238.26 | -7.3 | -7.4 | | + +(The floors are from this script, the same on the snap and no-snap files. The table +above used a different window, which gave 2.4/6.9/12.7 and 1.8/4.3/6.3.) Moving the +cut a few seconds earlier moves it from one point in a full band to another. Each +following section still opens like a new song, so the step stays. s1166 join 3 is +flat only because verse_2's band thins to -30 dBFS in its last 7 s. + +**Silence:** silencedetect at -50 dBFS/0.5 s finds nothing in either file. At +-40 dBFS/0.2 s it finds only the lead-in, the same in-section dips as the no-snap +takes (s1166 at 43-46, 192-193 and 284 s; s1167 one 0.83 s dip at 21.3 s, which is +also in `f83cf3566727` and is not a join), and each file's last 1-2 s of decay. No +hole is at a join. + +**Warnings (GET /api/jobs/):** each job has three `slice_dropped_loud_tail` +(one per snapped section) and one `joined_short`. + +| run, section | dropped s | dropped / kept RMS dBFS | right? | +|---|---|---|---| +| s1166 verse_1 | 45.99 | -17.8 / -18.6 | yes: a lyric line and the pre-chorus are lost | +| s1166 chorus_1 | 16.81 | -16.8 / -18.1 | yes: music is playing. Up to 85 s that is band only | +| s1166 verse_2 | 29.60 | -19.2 / -20.3 | yes: a bridge repeat is lost | +| s1167 verse_1 | 49.22 | -21.3 / -21.1 | yes: music is playing. The lyric is complete | +| s1167 chorus_1 | 6.34 | -17.1 / -15.0 | yes: band only, which includes the section's own ending | +| s1167 verse_2 | 46.84 | -15.9 / -17.5 | yes: "Somewhere a light they understand" is lost | + +The server keeps no intermediate files, so whatever lies past 85 s of source cannot +be heard. "Right" means the claim that the tail is still playing. `joined_short` +names the three snapped sections apart from the one that ended early (chorus_2, +68.69 / 60.95 s), and the shortfall is 26.74 / 37.79 s against 335.5. One wording +nit: the warning says the slice "cut it mid-signal at 82.17 s" even when the snap +moved that cut to a -51 dBFS gap. The tail was mid-signal; the cut was not. + +**What is left for the level step** (not made here): the step is in the generation, +not the cut. Only verse_1's caption asks for a sparse opening, yet Music 3 opens +every section like a new song. The levers are a caption that asks the next section +to open at full band (untested), a `match_levels` or a short gain ramp across each +join, or a shorter lyric, so each section ends on its own decay before 85 s. The record's fix option 3 (size the lyric per section) is the one +that also stops the lyric loss. + +### Outputs (mini-ai `~/bench-workspace/outputs/`) + +| arm | job | file | +|---|---|---| +| s6-music-full-335s-s1166 | `ae2e1373cff6` | `BenchS6MusicLongFull/20261106-024659-f8973042/final/BenchS6MusicLongFull-piece.0-0.0.mp3` | +| s6-music-full-335s-s1167 | `f83cf3566727` | `BenchS6MusicLongFull/20261106-032709-655afa0d/final/BenchS6MusicLongFull-piece.0-0.0.mp3` | +| s6-music-full-335s-xf3000-s1166 | `ef47c7b6684d` | `BenchS6MusicLongFull/20261106-040613-0ce22d5b/final/BenchS6MusicLongFull-piece.0-0.0.mp3` | +| s6-narr-long-t37-wps3-s1166 | `847ccb839875` | `BenchS6NarrationLong-t37/20261106-044527-c46db14f/final/BenchS6NarrationLong-t37-narration.0-0.0.wav` | +| s6-narr-long-t37-wps3-s1167 | `4467e6380276` | `BenchS6NarrationLong-t37/20261106-044934-a03a36cc/final/BenchS6NarrationLong-t37-narration.0-0.0.wav` | +| s6-music-snap8-335s-s1166 | `2179e0c0c9df` | `BenchS6MusicLongFull/20261106-081846-f8973042/final/BenchS6MusicLongFull-piece.0-0.0.mp3` | +| s6-music-snap8-335s-s1167 | `d51816c55389` | `BenchS6MusicLongFull/20261106-085848-655afa0d/final/BenchS6MusicLongFull-piece.0-0.0.mp3` | diff --git a/bench/records/baseline.json b/bench/records/baseline.json new file mode 100644 index 000000000..f1c0f2020 --- /dev/null +++ b/bench/records/baseline.json @@ -0,0 +1,212 @@ +[ + { + "arm": "control-s1-seed2", + "scene": "s1-static", + "seed": 2, + "job_id": "4282a9c7f0c8", + "status": "succeeded", + "error": null, + "output": "BenchS1Static/20261104-031311-3ab97814/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.9266666666666666, + "seam_jump_max": 1.45, + "seam_look_mean": 1.3233333333333333, + "seam_look_max": 2.23, + "seam_level_step_db_mean": 11.183333333333334, + "seam_level_step_db_max": 17.74, + "texture_fine_seg_last": 1.0201111779244663, + "texture_mid_seg_last": 0.3860839420115827, + "texture_coarse_seg_last": 0.046858716343750044, + "wall_minutes": 21.279575717449188 + } + }, + { + "arm": "control-s2-seed2", + "scene": "s2-arc", + "seed": 2, + "job_id": "911cf038e1d1", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc/20261104-033426-51d13983/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.1633333333333333, + "seam_jump_max": 1.83, + "seam_look_mean": 1.2111111111111112, + "seam_look_max": 1.77, + "seam_level_step_db_mean": 6.846666666666668, + "seam_level_step_db_max": 24.46, + "texture_fine_seg_last": 3.386474702125032, + "texture_mid_seg_last": 3.378192274816704, + "texture_coarse_seg_last": 2.9058749743617796, + "wall_minutes": 17.863606576124827 + } + }, + { + "arm": "control-s3-seed2", + "scene": "s3-dialogue-walk", + "seed": 2, + "job_id": "f93a17539b2a", + "status": "succeeded", + "error": null, + "output": "BenchS3DialogueWalk/20261104-035218-d2733a3a/final/BenchS3DialogueWalk-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.192, + "seam_jump_max": 1.69, + "seam_look_mean": 1.06, + "seam_look_max": 1.64, + "seam_level_step_db_mean": 2.59, + "seam_level_step_db_max": 6.22, + "texture_fine_seg_last": 0.293776369695831, + "texture_mid_seg_last": 0.22550331446142596, + "texture_coarse_seg_last": 0.29862160095698154, + "wall_minutes": 10.578909420967102 + } + }, + { + "arm": "control-s4-seed2", + "scene": "s4-montage", + "seed": 2, + "job_id": "1a0f702c9e85", + "status": "succeeded", + "error": null, + "output": "BenchS4Montage/20261104-040255-9813688b/final/BenchS4Montage-film.14-0.0.mp4", + "measures": { + "seam_jump_mean": 19.753333333333334, + "seam_jump_max": 22.64, + "seam_look_mean": 17.22, + "seam_look_max": 23.4, + "seam_level_step_db_mean": 1.86, + "seam_level_step_db_max": 2.35, + "texture_fine_seg_last": 1.0392178603891866, + "texture_mid_seg_last": 0.19872986814382518, + "texture_coarse_seg_last": 0.6681090796511284, + "wall_minutes": 20.203961877028146 + } + }, + { + "arm": "control-s5-seed2", + "scene": "s5-ltx-action", + "seed": 2, + "job_id": "958d1ee9d7c7", + "status": "succeeded", + "error": null, + "output": "BenchS5LtxAction/20261104-031441-74587ca7/final/BenchS5LtxAction-chained_image_to_video.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.17, + "seam_jump_max": 1.95, + "seam_look_mean": 1.31, + "seam_look_max": 1.73, + "seam_level_step_db_mean": 3.5650000000000004, + "seam_level_step_db_max": 5.98, + "texture_fine_seg_last": 0.5121222203819994, + "texture_mid_seg_last": 0.8211880233415666, + "texture_coarse_seg_last": 0.34714888810802724, + "wall_minutes": 3.6416345477104186 + } + }, + { + "arm": "control-s6-seed2", + "scene": "s6-narration", + "seed": 2, + "job_id": "c3da32af6ac0", + "status": "succeeded", + "error": null, + "output": "BenchS6Narration/20261104-031331-66be03f2/final/BenchS6Narration-film.9-0.0.mp4", + "measures": { + "seam_jump_mean": 40.2, + "seam_jump_max": 45.33, + "seam_look_mean": 70.635, + "seam_look_max": 82.75, + "seam_level_step_db_mean": 12.485, + "seam_level_step_db_max": 17.59, + "texture_fine_seg_last": 1.4010987900525822, + "texture_mid_seg_last": 1.2212536933691236, + "texture_coarse_seg_last": 0.8530977746815804, + "wall_minutes": 1.17655055920283 + } + }, + { + "arm": "best-s1-seed1166", + "scene": "s1-static", + "seed": 1166, + "job_id": "3603455d0c2a", + "status": "succeeded", + "error": null, + "output": "BenchS1Static/20261104-042305-3bcc7171/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.71, + "seam_jump_max": 0.78, + "seam_look_mean": 0.7322222222222222, + "seam_look_max": 1.67, + "seam_level_step_db_mean": 6.153333333333333, + "seam_level_step_db_max": 17.1, + "texture_fine_seg_last": 0.12281329071298945, + "texture_mid_seg_last": 0.12591968488307506, + "texture_coarse_seg_last": 0.04795116479393186, + "wall_minutes": 25.81393301486969 + } + }, + { + "arm": "best-s1-seed2", + "scene": "s1-static", + "seed": 2, + "job_id": "75f765edb2da", + "status": "succeeded", + "error": null, + "output": "BenchS1Static/20261104-044853-894054fa/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.7077777777777778, + "seam_jump_max": 0.84, + "seam_look_mean": 0.8333333333333334, + "seam_look_max": 1.9, + "seam_level_step_db_mean": 7.248888888888889, + "seam_level_step_db_max": 14.75, + "texture_fine_seg_last": 0.022018064701662118, + "texture_mid_seg_last": 0.08473033239469573, + "texture_coarse_seg_last": 0.044340428064213094, + "wall_minutes": 22.743692163626353 + } + }, + { + "arm": "best-s2-seed1166", + "scene": "s2-arc", + "seed": 1166, + "job_id": "d2a6892e316d", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc/20261104-051138-5de055bd/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.0588888888888888, + "seam_jump_max": 1.65, + "seam_look_mean": 1.2166666666666666, + "seam_look_max": 1.8, + "seam_level_step_db_mean": 2.537777777777778, + "seam_level_step_db_max": 8.5, + "texture_fine_seg_last": 3.2384555405664814, + "texture_mid_seg_last": 2.098823041067278, + "texture_coarse_seg_last": 2.0967858300268203, + "wall_minutes": 17.970901489257812 + } + }, + { + "arm": "best-s2-seed2", + "scene": "s2-arc", + "seed": 2, + "job_id": "1e6617922f42", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc/20261104-052935-e40247b8/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.0577777777777777, + "seam_jump_max": 1.59, + "seam_look_mean": 1.4366666666666665, + "seam_look_max": 2.4, + "seam_level_step_db_mean": 3.7477777777777774, + "seam_level_step_db_max": 7.25, + "texture_fine_seg_last": 0.31886725381461656, + "texture_mid_seg_last": 0.7244061149423758, + "texture_coarse_seg_last": 0.6962845205153961, + "wall_minutes": 17.924632473786673 + } + } +] \ No newline at end of file diff --git a/bench/records/capstone-controls.json b/bench/records/capstone-controls.json new file mode 100644 index 000000000..1a8861aca --- /dev/null +++ b/bench/records/capstone-controls.json @@ -0,0 +1,23 @@ +[ + { + "arm": "ctl10-guide-s2-s2", + "scene": "s2-arc", + "seed": 2, + "job_id": "87038dfc0892", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc-ctl155/20261105-160558-c67e7bc9/final/BenchS2Arc-ctl155-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.1633333333333333, + "seam_jump_max": 1.83, + "seam_look_mean": 1.2111111111111112, + "seam_look_max": 1.77, + "seam_level_step_db_mean": 6.846666666666668, + "seam_level_step_db_max": 24.46, + "texture_fine_seg_last": 3.386474702125032, + "texture_mid_seg_last": 3.378192274816704, + "texture_coarse_seg_last": 2.9058749743617796, + "wall_minutes": 17.828979742527007 + } + } +] \ No newline at end of file diff --git a/bench/records/capstone.json b/bench/records/capstone.json new file mode 100644 index 000000000..561de96c9 --- /dev/null +++ b/bench/records/capstone.json @@ -0,0 +1,86 @@ +[ + { + "arm": "cap90-s2-s1166", + "scene": "s2-arc", + "seed": 1166, + "job_id": "1800676a8213", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc-cap90/20261105-145015-5a6f45eb/final/BenchS2Arc-cap90-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.2911111111111113, + "seam_jump_max": 1.85, + "seam_look_mean": 1.6555555555555557, + "seam_look_max": 4.52, + "seam_level_step_db_mean": 2.531111111111111, + "seam_level_step_db_max": 5.25, + "texture_fine_seg_last": 3.372096702618574, + "texture_mid_seg_last": 2.6640789708026666, + "texture_coarse_seg_last": 2.1929296996989085, + "wall_minutes": 20.58876499334971 + } + }, + { + "arm": "cap70-s2-s1166", + "scene": "s2-arc", + "seed": 1166, + "job_id": "ee132d5ef34e", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc-cap70/20261105-151051-c529d829/final/BenchS2Arc-cap70-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.2066666666666666, + "seam_jump_max": 1.95, + "seam_look_mean": 1.098888888888889, + "seam_look_max": 2.36, + "seam_level_step_db_mean": 2.0311111111111106, + "seam_level_step_db_max": 7.26, + "texture_fine_seg_last": 3.309712953553608, + "texture_mid_seg_last": 2.4683830211657005, + "texture_coarse_seg_last": 2.018008283026852, + "wall_minutes": 18.354071803887685 + } + }, + { + "arm": "cap90-s2-s2", + "scene": "s2-arc", + "seed": 2, + "job_id": "17728f8ef9f6", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc-cap90/20261105-152912-d39c9fc4/final/BenchS2Arc-cap90-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.1966666666666665, + "seam_jump_max": 1.7, + "seam_look_mean": 1.38, + "seam_look_max": 2.16, + "seam_level_step_db_mean": 7.219999999999999, + "seam_level_step_db_max": 35.91, + "texture_fine_seg_last": 3.45178582349353, + "texture_mid_seg_last": 3.319749933418036, + "texture_coarse_seg_last": 2.873726072393926, + "wall_minutes": 18.42096612453461 + } + }, + { + "arm": "cap70-s2-s2", + "scene": "s2-arc", + "seed": 2, + "job_id": "65ef9753bafd", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc-cap70/20261105-154737-1fb2a33e/final/BenchS2Arc-cap70-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.2344444444444445, + "seam_jump_max": 1.72, + "seam_look_mean": 1.4288888888888889, + "seam_look_max": 2.12, + "seam_level_step_db_mean": 6.113333333333333, + "seam_level_step_db_max": 33.0, + "texture_fine_seg_last": 3.6515719339545663, + "texture_mid_seg_last": 3.5260666086740042, + "texture_coarse_seg_last": 3.504043680639653, + "wall_minutes": 18.335200905799866 + } + } +] \ No newline at end of file diff --git a/bench/records/cheap-checks.json b/bench/records/cheap-checks.json new file mode 100644 index 000000000..2a702d49c --- /dev/null +++ b/bench/records/cheap-checks.json @@ -0,0 +1,128 @@ +[ + { + "arm": "cheap-a-s1", + "scene": "s1-static", + "seed": 1166, + "job_id": "a438bffb764f", + "status": "succeeded", + "error": null, + "output": "BenchS1Static/20261104-003017-655187be/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.882, + "seam_jump_max": 1.0, + "seam_look_mean": 1.6219999999999999, + "seam_look_max": 2.2, + "seam_level_step_db_mean": 8.854000000000001, + "seam_level_step_db_max": 17.77, + "texture_fine_seg_last": 0.502764996926132, + "texture_mid_seg_last": 0.17135204694593162, + "texture_coarse_seg_last": 0.007668630503602891, + "wall_minutes": 10.919583070278168 + } + }, + { + "arm": "cheap-b-s1", + "scene": "s1-static", + "seed": 1166, + "job_id": "7e18115259a6", + "status": "succeeded", + "error": null, + "output": "BenchS1Static/20261104-005919-f3588d64/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.8140000000000001, + "seam_jump_max": 0.95, + "seam_look_mean": 1.392, + "seam_look_max": 2.54, + "seam_level_step_db_mean": 16.613999999999997, + "seam_level_step_db_max": 27.02, + "texture_fine_seg_last": 0.8544273526151175, + "texture_mid_seg_last": 0.3867239073129147, + "texture_coarse_seg_last": 0.0022898952157175668, + "wall_minutes": 10.515459883213044 + } + }, + { + "arm": "cheap-c-s1", + "scene": "s1-static", + "seed": 1166, + "job_id": "9cc328702d45", + "status": "succeeded", + "error": null, + "output": "BenchS1Static/20261104-011029-6cf2a1b0/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.162, + "seam_jump_max": 1.63, + "seam_look_mean": 1.404, + "seam_look_max": 2.83, + "seam_level_step_db_mean": 7.5280000000000005, + "seam_level_step_db_max": 16.58, + "texture_fine_seg_last": 0.18974276936522114, + "texture_mid_seg_last": 0.10538200739962529, + "texture_coarse_seg_last": 0.035110393085034976, + "wall_minutes": 13.661045277118683 + } + }, + { + "arm": "cheap-a-s2", + "scene": "s2-arc", + "seed": 1166, + "job_id": "80eb57300971", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc/20261104-014743-a12ce1e7/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.3619999999999999, + "seam_jump_max": 2.09, + "seam_look_mean": 1.286, + "seam_look_max": 1.74, + "seam_level_step_db_mean": 1.4620000000000002, + "seam_level_step_db_max": 3.43, + "texture_fine_seg_last": 3.083963956401898, + "texture_mid_seg_last": 2.2932110623147106, + "texture_coarse_seg_last": 1.4297775601219735, + "wall_minutes": 10.56614822546641 + } + }, + { + "arm": "cheap-b-s2", + "scene": "s2-arc", + "seed": 1166, + "job_id": "ebf09ad9fa12", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc/20261104-020905-75a1b6b3/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.166, + "seam_jump_max": 1.63, + "seam_look_mean": 1.6940000000000002, + "seam_look_max": 3.56, + "seam_level_step_db_mean": 1.59, + "seam_level_step_db_max": 4.06, + "texture_fine_seg_last": 2.883403862056931, + "texture_mid_seg_last": 1.7066904530022595, + "texture_coarse_seg_last": 0.10520650290810232, + "wall_minutes": 10.593446962038676 + } + }, + { + "arm": "cheap-c-s2", + "scene": "s2-arc", + "seed": 1166, + "job_id": "10cbd9b21a72", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc/20261104-023013-fa9a28f1/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.8539999999999999, + "seam_jump_max": 2.87, + "seam_look_mean": 4.27, + "seam_look_max": 6.89, + "seam_level_step_db_mean": 1.6260000000000001, + "seam_level_step_db_max": 3.35, + "texture_fine_seg_last": 2.5632527861842003, + "texture_mid_seg_last": 2.040688435068713, + "texture_coarse_seg_last": 2.0723558586423962, + "wall_minutes": 13.248008183638255 + } + } +] \ No newline at end of file diff --git a/bench/records/check-cut-ratio.json b/bench/records/check-cut-ratio.json new file mode 100644 index 000000000..a68c8d81a --- /dev/null +++ b/bench/records/check-cut-ratio.json @@ -0,0 +1,20 @@ +[ + { + "arm": "chk-s3-seg4-broken-cut", + "scene": "s3-dialogue-walk", + "seed": 1166, + "job_id": "6668a913863c", + "status": "succeeded", + "error": null, + "output": "BenchS3DialogueWalk-chk2/20261105-141039-16409ca0/final/BenchS3DialogueWalk-chk2-chained_keyframe_to_video_audio.0-0.0.mp4" + }, + { + "arm": "chk-s3-seg4-broken-r3", + "scene": "s3-dialogue-walk", + "seed": 1166, + "job_id": "ccce7adc9aaf", + "status": "succeeded", + "error": null, + "output": "BenchS3DialogueWalk-chk3/20261105-142638-f6817b4f/final/BenchS3DialogueWalk-chk3-chained_keyframe_to_video_audio.0-0.0.mp4" + } +] \ No newline at end of file diff --git a/bench/records/check.json b/bench/records/check.json new file mode 100644 index 000000000..caad9fa44 --- /dev/null +++ b/bench/records/check.json @@ -0,0 +1,65 @@ +[ + { + "arm": "chk-s3-seg4-broken", + "scene": "s3-dialogue-walk", + "seed": 1166, + "job_id": "fdea6eba4a33", + "status": "succeeded", + "error": null, + "output": "BenchS3DialogueWalk-chk/20261105-082441-006f3a57/final/BenchS3DialogueWalk-chk-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.208, + "seam_jump_max": 1.93, + "seam_look_mean": 1.528, + "seam_look_max": 2.58, + "seam_level_step_db_mean": 3.658, + "seam_level_step_db_max": 8.81, + "texture_fine_seg_last": 0.7031227143207547, + "texture_mid_seg_last": 0.04337474109546959, + "texture_coarse_seg_last": 0.027532089161735367, + "wall_minutes": 14.436770510673522 + } + }, + { + "arm": "chk-s1-fp", + "scene": "s1-static", + "seed": 1166, + "job_id": "7b76a3796335", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-chk/20261105-083907-e3122505/final/BenchS1Static-chk-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.9300000000000002, + "seam_jump_max": 1.09, + "seam_look_mean": 1.531111111111111, + "seam_look_max": 2.61, + "seam_level_step_db_mean": 9.856666666666667, + "seam_level_step_db_max": 36.56, + "texture_fine_seg_last": 1.1178899576395858, + "texture_mid_seg_last": 0.40150993772889665, + "texture_coarse_seg_last": 0.04875222289931379, + "wall_minutes": 20.60774208307266 + } + }, + { + "arm": "chk-s2-fp", + "scene": "s2-arc", + "seed": 1166, + "job_id": "604e115324b7", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc-chk/20261105-085942-e44e2973/final/BenchS2Arc-chk-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.1944444444444444, + "seam_jump_max": 2.09, + "seam_look_mean": 1.2844444444444445, + "seam_look_max": 2.71, + "seam_level_step_db_mean": 2.035555555555556, + "seam_level_step_db_max": 5.54, + "texture_fine_seg_last": 3.5575967638583803, + "texture_mid_seg_last": 2.8636397627256116, + "texture_coarse_seg_last": 1.9746980403751424, + "wall_minutes": 18.167179222901662 + } + } +] \ No newline at end of file diff --git a/bench/records/check.md b/bench/records/check.md new file mode 100644 index 000000000..e79aa9a70 --- /dev/null +++ b/bench/records/check.md @@ -0,0 +1,149 @@ +# Per-segment check: catch rate and false positives (Task 13 Step 5) + +2026-11-05. Bench server lem:8766 at `082e93ce`, cuda:0/cuda:1, cuDNN +deterministic. Arms: `bench/arms/check.json`; record: `check.json` (copied +from lem's `~/dw-bench-t13`). The bench workflows gained three knobs, all off +by default, so every control is unchanged: `check`, `retries` and +`keep_segments` (chain `save_segments` + `keep_segments`). + +| arm | scene | check | retries | job | wall | +|---|---|---|---|---|---| +| chk-s3-seg4-broken | S3, 6 segments, segment 4's prompt replaced | true | 1 | `fdea6eba4a33` | 14.4 min | +| chk-s1-fp | S1, 10 segments | true | 0 | `7b76a3796335` | 20.6 min | +| chk-s2-fp | S2, 10 segments | true | 0 | `604e115324b7` | 18.2 min | + +Default ceilings: jump 4.0, look 5.5, texture 2.0 (either way). + +## The made-to-fail segment (S3) + +Segment 4's prompt opens on the keyframe and then "at 1.00 seconds cuts hard +to a deserted harbour at midnight in pouring rain ... the woman is gone", +with a stranger's shouted line (the five others are the S3 bench prompts). + +- **Attempt 1 was caught.** `Chain segment 4/6: check failed (look_ratio + 15.1) - re-rolling, attempt 2 of 2 at seed 1001169` (job log 08:33:31). +- **Attempt 2 passed the check, and is broken.** The run record has + segment 4 at `attempts: 2`, `seed: 1001169` and no `check_failed`, so no + warning. By eye (contact sheets of take frames 320-532) the kept attempt + holds the street for its first 26 kept frames, then cuts to the + neon harbour at take frame 354 (render frame 48, about 2 s in; + `bench/cuts.py`'s largest step) and + stays there to the end of the segment. +- **Segment 5 is broken too, and also passed.** Its guide was the harbour + tail, so it opens on the harbour and cuts back to the street at take frame + 449 (19 kept frames in), with a different woman (white top, different + face) for the rest of the take. + +So the check caught 1 of the 3 broken renders: the one whose break sat at +the seam. It missed both breaks that sat inside a segment (26 and 19 frames past +the seam). That +is the check's design, not its thresholds: `jump_ratio` reads the one frame +step across the seam and `look_ratio` a 4-frame window either side +(`dw/tasks/assess.py` `_seam_video`, `LOOK_WINDOW`). analyze_seams on the +kept take sees nothing either: seam jump max 1.93, look max 2.58 (all five +seams under the ceilings; seams 3 and 4 read 0.42 and 0.13 because the +harbour's fast pan raised segment 4's typical step to 20). + +### Fix: `cut_ratio` (commit `d37c3192`, needs a deploy) + +A fourth measure: the segment's largest frame step over its own floored P90 +step - `_seam_video`'s arithmetic on a seam inside the segment. +`bench/cuts.py` measured it on decoded takes: + +| set | segments | P50 | P99 | max | +|---|---|---|---|---| +| 47 bench takes (controls, noise floor, round 1; S1-S6) | 282 | 1.22 | 2.62 | 2.98 | +| this take, segment 4 (cut to the harbour) | 1 | | | 4.16 | +| this take, segment 5 (cut back) | 1 | | | 17.24 | + +`CUT_RATIO_MAX = 4.0`, the seam's own ceiling, sits between the good +segments' 2.98 and the harbour cut's 4.16. The margin on the low side is +thin (the harbour pan inflated segment 4's own typical step); a cut into a +static scene reads far higher, like segment 5's 17.2. Tests: +`tests/test_chain_check.py` (`test_a_cut_inside_the_segment_fails_on_cut_ratio`, +`test_steady_motion_inside_the_segment_passes_cut_ratio`, +`test_cut_ratio_null_skips_it`). To confirm on the GPU after the deploy: +rerun `chk-s3-seg4-broken` - attempt 2 should now fail on `cut_ratio`, be +kept as the closest and flagged with `check_failed` and a warning. + +## False positives (S1, S2 with the check on) + +`retries: 0`, so the check measures and flags but renders nothing extra. Both +takes reproduce the noise-floor runs to the third decimal (S1 jump max 1.09, +look max 2.61, fine texture 1.118; S2 2.09, 2.71, 3.558), so turning the +check on leaves the render alone. + +| scene | checked segments | jump fails | look fails | texture fails | cut_ratio (offline) | +|---|---|---|---|---|---| +| S1 | 9 | 0 | 0 | 3: segments 8, 9, 10 at 2.02, 2.22, 2.43 | 0 (max 0.99) | +| S2 | 9 | 0 | 0 | 3: segments 7, 9, 10 at 0.48, 0.39, 0.32 | 0 (max 2.21) | +| S3 (segments 2, 3, 5, 6) | 4 | 0 | 0 | 0 | segment 5 (a real cut) | + +- **Jump and look: no false positives** in 22 good seams. The highest reads + were 2.09 (jump) and 2.71 (look), about half the ceilings. No retune. +- **Texture: 6 of 18 segments flagged, all late in a 10-segment take.** S1 + crosses 2.0 at segment 8 (30.7 s), the point counters.md already names + ("a static take longer than about 30 s should be cut into shots"); S2's + fine band falls below 0.5 from segment 7, where the eye notes record the + sky blowing out to white. These are the drift the ceiling exists to catch, + not good segments failing, so the ceiling stands at 2.0. +- **But a texture failure cannot be fixed by a re-roll**: the drift is in + the carry, and the re-roll renders from the same carry. At the templates' + `retries: 1` each of those segments renders twice for nothing (S1: 3 extra + segments, about +30% wall time) and is then flagged anyway. Proposal, not + committed: let `texture_ratio` flag and warn without re-rolling (only + jump, look and cut drive a re-roll), or re-roll on texture only when the + previous segment passed it. That is a change of the check's policy, so it + is for the controller to rule on. + +## `cut_ratio` on the GPU (2026-11-05, deploy `155ae56c`) + +Arms `bench/arms/check-cut-ratio.json`, record `check-cut-ratio.json`: the +`chk-s3-seg4-broken` arm again (new workflow id suffix, so no step-cache +hit), and the same arm at `retries: 3`. + +| arm | job | card | retries | wall | segment 1 | +|---|---|---|---|---|---| +| chk-s3-seg4-broken-cut | `6668a913863c` | cuda:1 | 1 | 15.9 min | `24b18bc0/d21b04b0`, identical to the original run `fdea6eba4a33` | +| chk-s3-seg4-broken-r3 | `ccce7adc9aaf` | cuda:0 | 3 | 23.7 min | `806ef3ca/3ba72965`, drifted (the card had just run the 120 s refine) | + +Both differ from the S3 canonical segment 1 (`289012f1`); the original check +arm did too, so the rerun is read against the original, which it matches. + +**`cut_ratio` catches the cut 26 frames in.** On the rerun: + +- attempt 1 of segment 4 fails on look 15.1 *and* cut 5.92; +- attempt 2 (seed 1001169), the one that passed before, now fails on + **`cut_ratio 4.15`** (the offline `bench/cuts.py` read 4.16), is kept as + the closest, and the run record's segment 4 shot carries `attempts: 2`, + `seed: 1001169`, `check_failed: ["cut_ratio"]`; +- segment 5's cut back to the street fails on `cut_ratio 17.25` on both + attempts (kept attempt 1, `check_failed: ["cut_ratio"]`); +- the job carries one `chain_check_failed` warning per segment, naming the + measure and value: "Chain segment 4/6 failed its check on every attempt + (cut_ratio 4.15); kept attempt 2 of 2, the closest" (and 5/6 at 17.25). + +So all three broken renders are now caught, and segments 1-3 and 6 still +pass (no false positive). The margin is thin, as predicted: 4.15 against a +4.0 ceiling, because the harbour's fast pan lifts the segment's own typical +step. + +**The retry budget does not find a clean attempt, here or at `retries: 3`.** +Segment 4's prompt asks for the cut, so every seed renders it: attempts 1-3 +read cut 5.26 (+ look 15.26), 4.48, 5.21 (+ look 13.96), and attempt 4 failed +too (the log names only the measures that trigger a re-roll); kept attempt 2 +at 4.48. Segment 5 opens on the kept harbour tail, so every seed breaks: +attempts 1-3 read cut 7.47, look 6.84, cut 7.91, attempt 4 failed; kept +attempt 1. The budget is +spent (+3 segments, +8 min against `retries: 1`) and the take is flagged, +which is what the check can do: a re-roll fixes a seed's accident, not a +prompt that asks for the break, and not a carry that already holds it +(`repair.md`, Step 3). (The `retries: 3` run is drifted at segment 1, so +its numbers are not comparable to the rerun's; its attempt 2 at the same +seed reads 4.48, not 4.15.) + +## Ceiling moved to 3.5 (#1205) + +The tester's rerun read segment 4's cut under 4.0 on cuda:0 (4.15 on cuda:1), so +a 4.0 ceiling sat on the boundary. `CUT_RATIO_MAX` is 3.5: 0.5 above the good +segments' max (2.98 over 282) and 0.65 below the cut's 4.15. diff --git a/bench/records/counters-controls.json b/bench/records/counters-controls.json new file mode 100644 index 000000000..b6bc2342c --- /dev/null +++ b/bench/records/counters-controls.json @@ -0,0 +1,212 @@ +[ + { + "arm": "ctl-guide-s1-s1166", + "scene": "s1-static", + "seed": 1166, + "job_id": "b40e855eb553", + "status": "succeeded", + "error": null, + "output": "BenchS1Static/20261104-060443-90c37bc2/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.8539999999999999, + "seam_jump_max": 0.97, + "seam_look_mean": 1.464, + "seam_look_max": 2.13, + "seam_level_step_db_mean": 7.653999999999999, + "seam_level_step_db_max": 16.58, + "texture_fine_seg_last": 0.5356445216304937, + "texture_mid_seg_last": 0.17558025571703065, + "texture_coarse_seg_last": 0.016023705618812927, + "wall_minutes": 10.52995136976242 + } + }, + { + "arm": "ctl-guide-s1-s2", + "scene": "s1-static", + "seed": 2, + "job_id": "894911aed6c7", + "status": "succeeded", + "error": null, + "output": "BenchS1Static/20261104-062828-0642442c/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.796, + "seam_jump_max": 0.92, + "seam_look_mean": 1.3439999999999999, + "seam_look_max": 2.27, + "seam_level_step_db_mean": 5.2219999999999995, + "seam_level_step_db_max": 18.88, + "texture_fine_seg_last": 0.4551209802925675, + "texture_mid_seg_last": 0.1437244782673097, + "texture_coarse_seg_last": 0.008246029661432236, + "wall_minutes": 10.541190910339356 + } + }, + { + "arm": "ctl-lastframe-s1-s1166", + "scene": "s1-static", + "seed": 1166, + "job_id": "3a73cb0f5d28", + "status": "succeeded", + "error": null, + "output": "BenchS1Static/20261104-065215-5b43dda0/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.096, + "seam_jump_max": 1.18, + "seam_look_mean": 1.264, + "seam_look_max": 1.79, + "seam_level_step_db_mean": 3.372, + "seam_level_step_db_max": 8.27, + "texture_fine_seg_last": 0.5691057993637598, + "texture_mid_seg_last": 0.3659432265673877, + "texture_coarse_seg_last": 0.008652528830376718, + "wall_minutes": 8.504809057712555 + } + }, + { + "arm": "ctl-lastframe-s1-s2", + "scene": "s1-static", + "seed": 2, + "job_id": "f053abe1ce5f", + "status": "succeeded", + "error": null, + "output": "BenchS1Static/20261104-071117-ae50341d/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.074, + "seam_jump_max": 1.18, + "seam_look_mean": 1.408, + "seam_look_max": 1.8, + "seam_level_step_db_mean": 4.074, + "seam_level_step_db_max": 7.73, + "texture_fine_seg_last": 0.604760302793261, + "texture_mid_seg_last": 0.45312835667160634, + "texture_coarse_seg_last": 0.005912257825027124, + "wall_minutes": 8.497850481669108 + } + }, + { + "arm": "ctl-guide-s2-s1166", + "scene": "s2-arc", + "seed": 1166, + "job_id": "dedd341c9108", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc/20261104-073022-cd97c6a1/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.3619999999999999, + "seam_jump_max": 2.09, + "seam_look_mean": 1.286, + "seam_look_max": 1.74, + "seam_level_step_db_mean": 1.4620000000000002, + "seam_level_step_db_max": 3.43, + "texture_fine_seg_last": 3.083963956401898, + "texture_mid_seg_last": 2.2932110623147106, + "texture_coarse_seg_last": 1.4297775601219735, + "wall_minutes": 12.531881523132324 + } + }, + { + "arm": "ctl-guide-s2-s2", + "scene": "s2-arc", + "seed": 2, + "job_id": "01146e925c94", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc/20261104-075344-d76c369a/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.2719999999999998, + "seam_jump_max": 1.86, + "seam_look_mean": 1.218, + "seam_look_max": 2.01, + "seam_level_step_db_mean": 4.252000000000001, + "seam_level_step_db_max": 8.97, + "texture_fine_seg_last": 3.6891252384161994, + "texture_mid_seg_last": 3.503671468858689, + "texture_coarse_seg_last": 3.090160355237524, + "wall_minutes": 10.456729876995087 + } + }, + { + "arm": "ctl-lastframe-s2-s1166", + "scene": "s2-arc", + "seed": 1166, + "job_id": "b235c35b72c1", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc/20261104-080501-f258d5fa/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.258, + "seam_jump_max": 1.77, + "seam_look_mean": 1.31, + "seam_look_max": 1.93, + "seam_level_step_db_mean": 5.018, + "seam_level_step_db_max": 9.81, + "texture_fine_seg_last": 3.4288768882010605, + "texture_mid_seg_last": 3.6412637689472342, + "texture_coarse_seg_last": 3.683735907100545, + "wall_minutes": 8.763977376619975 + } + }, + { + "arm": "ctl-lastframe-s2-s2", + "scene": "s2-arc", + "seed": 2, + "job_id": "765fc66286bd", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc/20261104-082803-71d81bfc/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.34, + "seam_jump_max": 1.59, + "seam_look_mean": 1.026, + "seam_look_max": 1.86, + "seam_level_step_db_mean": 1.902, + "seam_level_step_db_max": 3.09, + "texture_fine_seg_last": 3.5986793964134756, + "texture_mid_seg_last": 3.7071955343844705, + "texture_coarse_seg_last": 3.4700483083775686, + "wall_minutes": 8.289815723896027 + } + }, + { + "arm": "ctl-guide-s3-s1166", + "scene": "s3-dialogue-walk", + "seed": 1166, + "job_id": "c0b92d0d0cd4", + "status": "succeeded", + "error": null, + "output": "BenchS3DialogueWalk/20261104-084517-6db94fb9/final/BenchS3DialogueWalk-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.524, + "seam_jump_max": 1.83, + "seam_look_mean": 1.47, + "seam_look_max": 1.71, + "seam_level_step_db_mean": 2.968, + "seam_level_step_db_max": 8.81, + "texture_fine_seg_last": 0.0674850955709072, + "texture_mid_seg_last": 0.11366149849198895, + "texture_coarse_seg_last": 0.18802880506763223, + "wall_minutes": 10.579926371574402 + } + }, + { + "arm": "ctl-guide-s3-s2", + "scene": "s3-dialogue-walk", + "seed": 2, + "job_id": "64b4a78c74a2", + "status": "succeeded", + "error": null, + "output": "BenchS3DialogueWalk/20261104-090907-773b20e5/final/BenchS3DialogueWalk-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.192, + "seam_jump_max": 1.69, + "seam_look_mean": 1.06, + "seam_look_max": 1.64, + "seam_level_step_db_mean": 2.59, + "seam_level_step_db_max": 6.22, + "texture_fine_seg_last": 0.293776369695831, + "texture_mid_seg_last": 0.22550331446142596, + "texture_coarse_seg_last": 0.29862160095698154, + "wall_minutes": 10.529952156543732 + } + } +] \ No newline at end of file diff --git a/bench/records/counters-round1.json b/bench/records/counters-round1.json new file mode 100644 index 000000000..3ca5e723c --- /dev/null +++ b/bench/records/counters-round1.json @@ -0,0 +1,527 @@ +[ + { + "arm": "r1-anchor25-s1", + "scene": "s1-static", + "seed": 1166, + "job_id": "0a40c41af97c", + "status": "succeeded", + "error": null, + "output": "BenchS1Static/20261104-054934-fdc9b551/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.9279999999999999, + "seam_jump_max": 1.15, + "seam_look_mean": 1.128, + "seam_look_max": 1.98, + "seam_level_step_db_mean": 12.16, + "seam_level_step_db_max": 23.54, + "texture_fine_seg_last": 0.36683259200135493, + "texture_mid_seg_last": 0.12383734756733206, + "texture_coarse_seg_last": 0.015490472469675862, + "wall_minutes": 15.187270545959473 + } + }, + { + "arm": "r1-anchor50-s1", + "scene": "s1-static", + "seed": 1166, + "job_id": "fa17a2c51e33", + "status": "succeeded", + "error": null, + "output": "BenchS1Static/20261104-061515-5949edcc/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.048, + "seam_jump_max": 1.32, + "seam_look_mean": 1.05, + "seam_look_max": 1.71, + "seam_level_step_db_mean": 13.224, + "seam_level_step_db_max": 20.34, + "texture_fine_seg_last": 0.269789364587522, + "texture_mid_seg_last": 0.11577271085341745, + "texture_coarse_seg_last": 0.027805739291449503, + "wall_minutes": 13.222485268115998 + } + }, + { + "arm": "r1-anchor75-s1", + "scene": "s1-static", + "seed": 1166, + "job_id": "f17bce198a2d", + "status": "succeeded", + "error": null, + "output": "BenchS1Static/20261104-063901-253f19de/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.3, + "seam_jump_max": 1.96, + "seam_look_mean": 1.752, + "seam_look_max": 3.08, + "seam_level_step_db_mean": 15.258, + "seam_level_step_db_max": 23.77, + "texture_fine_seg_last": 0.2596503089388924, + "texture_mid_seg_last": 0.10298178806371444, + "texture_coarse_seg_last": 0.04520919053142924, + "wall_minutes": 13.2272402882576 + } + }, + { + "arm": "r1-backoff17-s1", + "scene": "s1-static", + "seed": 1166, + "job_id": "c594521e59d1", + "status": "succeeded", + "error": null, + "output": "BenchS1Static/20261104-070045-f9d2dc73/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.8859999999999999, + "seam_jump_max": 0.97, + "seam_look_mean": 1.5219999999999998, + "seam_look_max": 2.59, + "seam_level_step_db_mean": 10.744, + "seam_level_step_db_max": 22.35, + "texture_fine_seg_last": 0.4617207756861403, + "texture_mid_seg_last": 0.1669051506362843, + "texture_coarse_seg_last": 0.013911300257081285, + "wall_minutes": 10.537556370099386 + } + }, + { + "arm": "r1-keystone-s1", + "scene": "s1-static", + "seed": 1166, + "job_id": "1e1b6116ce26", + "status": "succeeded", + "error": null, + "output": "BenchS1Static/20261104-071947-19ca76a4/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.85, + "seam_jump_max": 1.04, + "seam_look_mean": 1.55, + "seam_look_max": 1.93, + "seam_level_step_db_mean": 14.556000000000001, + "seam_level_step_db_max": 23.26, + "texture_fine_seg_last": 0.37326355060979177, + "texture_mid_seg_last": 0.11620642238827716, + "texture_coarse_seg_last": 0.02005162817808236, + "wall_minutes": 10.543349421024322 + } + }, + { + "arm": "r1-lookhold-s1", + "scene": "s1-static", + "seed": 1166, + "job_id": "751b43732fb9", + "status": "succeeded", + "error": null, + "output": "BenchS1Static/20261104-074252-80046177/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.8960000000000001, + "seam_jump_max": 1.04, + "seam_look_mean": 1.31, + "seam_look_max": 2.11, + "seam_level_step_db_mean": 11.678, + "seam_level_step_db_max": 23.21, + "texture_fine_seg_last": 0.4637714962831042, + "texture_mid_seg_last": 0.14493266898039445, + "texture_coarse_seg_last": 0.007321449722209709, + "wall_minutes": 10.870044616858165 + } + }, + { + "arm": "r1-antiburn-s1", + "scene": "s1-static", + "seed": 1166, + "job_id": "049d652e1648", + "status": "succeeded", + "error": null, + "output": "BenchS1Static/20261104-080411-0301f0c0/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.724, + "seam_jump_max": 0.88, + "seam_look_mean": 2.646, + "seam_look_max": 3.21, + "seam_level_step_db_mean": 5.458, + "seam_level_step_db_max": 13.26, + "texture_fine_seg_last": 0.04599163189566434, + "texture_mid_seg_last": 0.11351136944410586, + "texture_coarse_seg_last": 0.07415068182164664, + "wall_minutes": 13.260358643531799 + } + }, + { + "arm": "r1-lf-carrypick-s1", + "scene": "s1-static", + "seed": 1166, + "job_id": "ae932366ea0e", + "status": "succeeded", + "error": null, + "output": "BenchLfS1Static/20261104-081730-5241ac41/final/BenchLfS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.994, + "seam_jump_max": 1.12, + "seam_look_mean": 1.252, + "seam_look_max": 1.73, + "seam_level_step_db_mean": 6.795999999999999, + "seam_level_step_db_max": 10.15, + "texture_fine_seg_last": 0.5331471497325624, + "texture_mid_seg_last": 0.33038070340604536, + "texture_coarse_seg_last": 0.0056319814824577295, + "wall_minutes": 10.601044821739197 + } + }, + { + "arm": "r1-lf-lookhold-s1", + "scene": "s1-static", + "seed": 1166, + "job_id": "be3f574df3db", + "status": "succeeded", + "error": null, + "output": "BenchS1Static/20261104-083621-b688e963/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.788, + "seam_jump_max": 0.89, + "seam_look_mean": 0.634, + "seam_look_max": 0.97, + "seam_level_step_db_mean": 4.324, + "seam_level_step_db_max": 6.78, + "texture_fine_seg_last": 0.4913361949092381, + "texture_mid_seg_last": 0.304254005112364, + "texture_coarse_seg_last": 0.005468514740518848, + "wall_minutes": 8.934008312225341 + } + }, + { + "arm": "r1-anchor25-s2", + "scene": "s2-arc", + "seed": 1166, + "job_id": "4350da6ac745", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc/20261104-085551-68fe5c04/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.228, + "seam_jump_max": 1.64, + "seam_look_mean": 2.032, + "seam_look_max": 3.6, + "seam_level_step_db_mean": 1.364, + "seam_level_step_db_max": 3.15, + "texture_fine_seg_last": 0.34010931238341774, + "texture_mid_seg_last": 0.16926133338085134, + "texture_coarse_seg_last": 0.12439291117663875, + "wall_minutes": 13.274113273620605 + } + }, + { + "arm": "r1-anchor50-s2", + "scene": "s2-arc", + "seed": 1166, + "job_id": "3408f7cf9fa9", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc/20261104-091939-682f41d1/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.496, + "seam_jump_max": 2.37, + "seam_look_mean": 3.214, + "seam_look_max": 5.62, + "seam_level_step_db_mean": 0.9879999999999999, + "seam_level_step_db_max": 3.05, + "texture_fine_seg_last": 3.0564484996802395, + "texture_mid_seg_last": 2.424022446085548, + "texture_coarse_seg_last": 2.4009389409679316, + "wall_minutes": 13.273704036076863 + } + }, + { + "arm": "r1-anchor75-s2", + "scene": "s2-arc", + "seed": 1166, + "job_id": "6fb052f46229", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc/20261104-093256-ab1bf1a9/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.666, + "seam_jump_max": 2.73, + "seam_look_mean": 3.5, + "seam_look_max": 5.54, + "seam_level_step_db_mean": 1.1520000000000001, + "seam_level_step_db_max": 3.2, + "texture_fine_seg_last": 2.3128596476539194, + "texture_mid_seg_last": 1.4428776163845796, + "texture_coarse_seg_last": 1.0536375961539397, + "wall_minutes": 13.291542156537373 + } + }, + { + "arm": "r1-backoff17-s2", + "scene": "s2-arc", + "seed": 1166, + "job_id": "ea394296a02d", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc/20261104-094613-88f4b225/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.3699999999999999, + "seam_jump_max": 1.89, + "seam_look_mean": 1.4300000000000002, + "seam_look_max": 2.61, + "seam_level_step_db_mean": 1.126, + "seam_level_step_db_max": 1.99, + "texture_fine_seg_last": 2.872961813106324, + "texture_mid_seg_last": 1.5335407191498742, + "texture_coarse_seg_last": 0.00023034754160976463, + "wall_minutes": 10.46375724474589 + } + }, + { + "arm": "r1-keystone-s2", + "scene": "s2-arc", + "seed": 1166, + "job_id": "f774bdbcc1cc", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc/20261104-095641-03fe99ed/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.356, + "seam_jump_max": 2.0, + "seam_look_mean": 1.404, + "seam_look_max": 2.09, + "seam_level_step_db_mean": 1.718, + "seam_level_step_db_max": 3.85, + "texture_fine_seg_last": 2.8185535247552536, + "texture_mid_seg_last": 1.3903447127123076, + "texture_coarse_seg_last": 0.6923936866449384, + "wall_minutes": 10.462815701961517 + } + }, + { + "arm": "r1-lookhold-s2", + "scene": "s2-arc", + "seed": 1166, + "job_id": "64dc87a2ed3a", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc/20261104-100709-3398e6c0/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.4, + "seam_jump_max": 2.03, + "seam_look_mean": 1.562, + "seam_look_max": 2.97, + "seam_level_step_db_mean": 1.3619999999999999, + "seam_level_step_db_max": 2.76, + "texture_fine_seg_last": 2.978407774332324, + "texture_mid_seg_last": 2.035275636666184, + "texture_coarse_seg_last": 0.7053778346409352, + "wall_minutes": 10.497964894771576 + } + }, + { + "arm": "r1-antiburn-s2", + "scene": "s2-arc", + "seed": 1166, + "job_id": "9124d417577d", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc/20261104-101739-05b1c816/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.248, + "seam_jump_max": 1.69, + "seam_look_mean": 1.364, + "seam_look_max": 2.47, + "seam_level_step_db_mean": 2.6740000000000004, + "seam_level_step_db_max": 5.11, + "texture_fine_seg_last": 3.439930661763011, + "texture_mid_seg_last": 2.9199109838005546, + "texture_coarse_seg_last": 1.3993942728960391, + "wall_minutes": 13.198985393842062 + } + }, + { + "arm": "r1-lf-carrypick-s2", + "scene": "s2-arc", + "seed": 1166, + "job_id": "9713480482bc", + "status": "succeeded", + "error": null, + "output": "BenchLfS2Arc/20261104-103052-7d0e533a/final/BenchLfS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.362, + "seam_jump_max": 1.85, + "seam_look_mean": 0.968, + "seam_look_max": 1.23, + "seam_level_step_db_mean": 2.958, + "seam_level_step_db_max": 4.41, + "texture_fine_seg_last": 2.4214933964458907, + "texture_mid_seg_last": 1.143830085229748, + "texture_coarse_seg_last": 0.6909618928676936, + "wall_minutes": 8.604944415887196 + } + }, + { + "arm": "r1-lf-lookhold-s2", + "scene": "s2-arc", + "seed": 1166, + "job_id": "242358078691", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc/20261104-103927-af5f9b3e/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.34, + "seam_jump_max": 1.84, + "seam_look_mean": 0.40199999999999997, + "seam_look_max": 0.51, + "seam_level_step_db_mean": 3.7840000000000003, + "seam_level_step_db_max": 7.33, + "texture_fine_seg_last": 2.4056601068521393, + "texture_mid_seg_last": 0.971996843480287, + "texture_coarse_seg_last": 0.5949260478474114, + "wall_minutes": 11.123202407360077 + } + }, + { + "arm": "r1-anchor25-s3", + "scene": "s3-dialogue-walk", + "seed": 1166, + "job_id": "7c1aa8f8b38d", + "status": "succeeded", + "error": null, + "output": "BenchS3DialogueWalk/20261104-105035-5a02f9f8/final/BenchS3DialogueWalk-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.6420000000000001, + "seam_jump_max": 2.04, + "seam_look_mean": 1.484, + "seam_look_max": 1.8, + "seam_level_step_db_mean": 3.2300000000000004, + "seam_level_step_db_max": 8.81, + "texture_fine_seg_last": 0.10270507411506632, + "texture_mid_seg_last": 0.16531565183560912, + "texture_coarse_seg_last": 0.49893175017767327, + "wall_minutes": 13.319513722260792 + } + }, + { + "arm": "r1-anchor50-s3", + "scene": "s3-dialogue-walk", + "seed": 1166, + "job_id": "fef6b9aea7c9", + "status": "succeeded", + "error": null, + "output": "BenchS3DialogueWalk/20261104-110353-90e5c4ef/final/BenchS3DialogueWalk-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.5539999999999998, + "seam_jump_max": 2.09, + "seam_look_mean": 1.4040000000000001, + "seam_look_max": 2.2, + "seam_level_step_db_mean": 3.896, + "seam_level_step_db_max": 8.81, + "texture_fine_seg_last": 0.05346154371068419, + "texture_mid_seg_last": 0.01345261829759669, + "texture_coarse_seg_last": 0.3785621742562812, + "wall_minutes": 13.257094446818034 + } + }, + { + "arm": "r1-anchor75-s3", + "scene": "s3-dialogue-walk", + "seed": 1166, + "job_id": "fbf286d9f192", + "status": "succeeded", + "error": null, + "output": "BenchS3DialogueWalk/20261104-111709-bec10029/final/BenchS3DialogueWalk-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.666, + "seam_jump_max": 2.3, + "seam_look_mean": 1.922, + "seam_look_max": 2.82, + "seam_level_step_db_mean": 3.05, + "seam_level_step_db_max": 8.81, + "texture_fine_seg_last": 0.0758930228322501, + "texture_mid_seg_last": 0.12017445555937717, + "texture_coarse_seg_last": 0.45108621546289457, + "wall_minutes": 13.248908603191376 + } + }, + { + "arm": "r1-backoff17-s3", + "scene": "s3-dialogue-walk", + "seed": 1166, + "job_id": "5873332b54fc", + "status": "succeeded", + "error": null, + "output": "BenchS3DialogueWalk/20261104-113024-86593c80/final/BenchS3DialogueWalk-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.496, + "seam_jump_max": 2.04, + "seam_look_mean": 1.124, + "seam_look_max": 1.69, + "seam_level_step_db_mean": 3.646, + "seam_level_step_db_max": 6.47, + "texture_fine_seg_last": 0.3374427639503598, + "texture_mid_seg_last": 0.47185748910332087, + "texture_coarse_seg_last": 0.03580050966538296, + "wall_minutes": 10.53287143309911 + } + }, + { + "arm": "r1-keystone-s3", + "scene": "s3-dialogue-walk", + "seed": 1166, + "job_id": "a2bd5e2545db", + "status": "succeeded", + "error": null, + "output": "BenchS3DialogueWalk/20261104-114056-8aecac36/final/BenchS3DialogueWalk-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.4140000000000001, + "seam_jump_max": 2.08, + "seam_look_mean": 1.182, + "seam_look_max": 2.03, + "seam_level_step_db_mean": 3.66, + "seam_level_step_db_max": 7.15, + "texture_fine_seg_last": 0.10590849383260391, + "texture_mid_seg_last": 0.1867216528836364, + "texture_coarse_seg_last": 0.2221853358516677, + "wall_minutes": 10.542827419439952 + } + }, + { + "arm": "r1-lookhold-s3", + "scene": "s3-dialogue-walk", + "seed": 1166, + "job_id": "a9a134a9acf8", + "status": "succeeded", + "error": null, + "output": "BenchS3DialogueWalk/20261104-115128-bf419738/final/BenchS3DialogueWalk-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.592, + "seam_jump_max": 1.83, + "seam_look_mean": 1.432, + "seam_look_max": 1.61, + "seam_level_step_db_mean": 3.028, + "seam_level_step_db_max": 8.81, + "texture_fine_seg_last": 0.05542212028354263, + "texture_mid_seg_last": 0.10856841199802084, + "texture_coarse_seg_last": 0.24375085860720547, + "wall_minutes": 10.59634923140208 + } + }, + { + "arm": "r1-antiburn-s3", + "scene": "s3-dialogue-walk", + "seed": 1166, + "job_id": "c50ec5961418", + "status": "succeeded", + "error": null, + "output": "BenchS3DialogueWalk/20261104-120204-7fe540b4/final/BenchS3DialogueWalk-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 2.0460000000000003, + "seam_jump_max": 2.39, + "seam_look_mean": 1.7440000000000002, + "seam_look_max": 2.01, + "seam_level_step_db_mean": 4.954, + "seam_level_step_db_max": 6.28, + "texture_fine_seg_last": 0.671210308985029, + "texture_mid_seg_last": 0.8755625223901399, + "texture_coarse_seg_last": 0.5187218760513145, + "wall_minutes": 13.2261123975118 + } + } +] \ No newline at end of file diff --git a/bench/records/counters-round2-combos-batch1.json b/bench/records/counters-round2-combos-batch1.json new file mode 100644 index 000000000..7936e0436 --- /dev/null +++ b/bench/records/counters-round2-combos-batch1.json @@ -0,0 +1,170 @@ +[ + { + "arm": "r2-anchor25-keystone-s1-s1166", + "scene": "s1-static", + "seed": 1166, + "job_id": "7c1c1fb14c8c", + "status": "succeeded", + "error": null, + "output": "BenchS1Static/20261104-143056-8ceecde3/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.992, + "seam_jump_max": 1.59, + "seam_look_mean": 1.5299999999999998, + "seam_look_max": 2.4, + "seam_level_step_db_mean": 4.546, + "seam_level_step_db_max": 16.93, + "texture_fine_seg_last": 0.28338839729447113, + "texture_mid_seg_last": 0.10847843353648676, + "texture_coarse_seg_last": 0.0363608687925832, + "wall_minutes": 13.302251875400543 + } + }, + { + "arm": "r2-anchor50-keystone-s1-s1166", + "scene": "s1-static", + "seed": 1166, + "job_id": "9331f354cc4b", + "status": "succeeded", + "error": null, + "output": "BenchS1Static/20261104-155203-4b68e3c3/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.1720000000000002, + "seam_jump_max": 1.41, + "seam_look_mean": 1.8379999999999999, + "seam_look_max": 2.3, + "seam_level_step_db_mean": 10.940000000000001, + "seam_level_step_db_max": 16.12, + "texture_fine_seg_last": 0.270651578745108, + "texture_mid_seg_last": 0.11900567916362315, + "texture_coarse_seg_last": 0.03707873200773257, + "wall_minutes": 13.285201688607534 + } + }, + { + "arm": "r2-lf-lookhold-anchor25-s1-s1166", + "scene": "s1-static", + "seed": 1166, + "job_id": "8010fb212bb3", + "status": "succeeded", + "error": null, + "output": "BenchS1Static/20261104-160520-ff3d1339/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.78, + "seam_jump_max": 0.99, + "seam_look_mean": 0.628, + "seam_look_max": 0.77, + "seam_level_step_db_mean": 6.811999999999999, + "seam_level_step_db_max": 11.03, + "texture_fine_seg_last": 0.29814702028313184, + "texture_mid_seg_last": 0.2279249174384437, + "texture_coarse_seg_last": 0.002722735827471639, + "wall_minutes": 11.494645738601685 + } + }, + { + "arm": "r2-lf-lookhold-anchor50-s1-s1166", + "scene": "s1-static", + "seed": 1166, + "job_id": "a58834fed81a", + "status": "succeeded", + "error": null, + "output": "BenchS1Static/20261104-161650-fade325a/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.748, + "seam_jump_max": 0.87, + "seam_look_mean": 0.744, + "seam_look_max": 1.2, + "seam_level_step_db_mean": 4.4719999999999995, + "seam_level_step_db_max": 6.15, + "texture_fine_seg_last": 0.2134442858027901, + "texture_mid_seg_last": 0.20782510509447852, + "texture_coarse_seg_last": 0.020010008944102986, + "wall_minutes": 11.496459396680196 + } + }, + { + "arm": "r2-anchor25-keystone-s1-s2", + "scene": "s1-static", + "seed": 2, + "job_id": "36c91319afbc", + "status": "succeeded", + "error": null, + "output": "BenchS1Static/20261104-162820-b9fccaaf/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.772, + "seam_jump_max": 1.01, + "seam_look_mean": 1.23, + "seam_look_max": 1.93, + "seam_level_step_db_mean": 12.206, + "seam_level_step_db_max": 18.36, + "texture_fine_seg_last": 0.3237847876323703, + "texture_mid_seg_last": 0.08936690121277437, + "texture_coarse_seg_last": 0.029174502465705524, + "wall_minutes": 13.293248569965362 + } + }, + { + "arm": "r2-anchor50-keystone-s1-s2", + "scene": "s1-static", + "seed": 2, + "job_id": "5a60b829547e", + "status": "succeeded", + "error": null, + "output": "BenchS1Static/20261104-164137-52a65249/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.944, + "seam_jump_max": 1.4, + "seam_look_mean": 1.666, + "seam_look_max": 2.9, + "seam_level_step_db_mean": 13.508000000000001, + "seam_level_step_db_max": 24.51, + "texture_fine_seg_last": 0.23691695271547183, + "texture_mid_seg_last": 0.08862589080472187, + "texture_coarse_seg_last": 0.03701071880963623, + "wall_minutes": 13.272747123241425 + } + }, + { + "arm": "r2-lf-lookhold-anchor25-s1-s2", + "scene": "s1-static", + "seed": 2, + "job_id": "2059d7bcf69b", + "status": "succeeded", + "error": null, + "output": "BenchS1Static/20261104-165454-347bb719/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.8099999999999999, + "seam_jump_max": 0.98, + "seam_look_mean": 0.766, + "seam_look_max": 1.07, + "seam_level_step_db_mean": 3.2119999999999997, + "seam_level_step_db_max": 6.3, + "texture_fine_seg_last": 0.3645009117263627, + "texture_mid_seg_last": 0.24670841698294105, + "texture_coarse_seg_last": 0.014777550310940635, + "wall_minutes": 11.499499746163686 + } + }, + { + "arm": "r2-lf-lookhold-anchor50-s1-s2", + "scene": "s1-static", + "seed": 2, + "job_id": "daa36f09dc3f", + "status": "succeeded", + "error": null, + "output": "BenchS1Static/20261104-170624-c6d7f38c/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.834, + "seam_jump_max": 0.95, + "seam_look_mean": 0.932, + "seam_look_max": 1.18, + "seam_level_step_db_mean": 2.936, + "seam_level_step_db_max": 3.92, + "texture_fine_seg_last": 0.26128882316970864, + "texture_mid_seg_last": 0.2135885836634359, + "texture_coarse_seg_last": 0.03043084495558615, + "wall_minutes": 11.48792098760605 + } + } +] \ No newline at end of file diff --git a/bench/records/counters-round2-combos.json b/bench/records/counters-round2-combos.json new file mode 100644 index 000000000..cbde3e310 --- /dev/null +++ b/bench/records/counters-round2-combos.json @@ -0,0 +1,170 @@ +[ + { + "arm": "r2-anchor25-keystone-s1-s1166", + "scene": "s1-static", + "seed": 1166, + "job_id": "8accd0c37f91", + "status": "succeeded", + "error": null, + "output": "BenchS1Static/20261104-171753-8ceecde3/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.898, + "seam_jump_max": 1.16, + "seam_look_mean": 1.092, + "seam_look_max": 1.61, + "seam_level_step_db_mean": 10.75, + "seam_level_step_db_max": 15.17, + "texture_fine_seg_last": 0.3091306667237105, + "texture_mid_seg_last": 0.11781661838143291, + "texture_coarse_seg_last": 0.031219723125928647, + "wall_minutes": 13.29447234471639 + } + }, + { + "arm": "r2-anchor50-keystone-s1-s1166", + "scene": "s1-static", + "seed": 1166, + "job_id": "e5c7b013538f", + "status": "succeeded", + "error": null, + "output": "BenchS1Static/20261104-173111-4b68e3c3/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.296, + "seam_jump_max": 1.74, + "seam_look_mean": 2.038, + "seam_look_max": 2.89, + "seam_level_step_db_mean": 5.590000000000001, + "seam_level_step_db_max": 14.46, + "texture_fine_seg_last": 0.2927186701695517, + "texture_mid_seg_last": 0.12763558260519758, + "texture_coarse_seg_last": 0.03874023726477519, + "wall_minutes": 13.27784192164739 + } + }, + { + "arm": "r2-lf-lookhold-anchor25-s1-s1166", + "scene": "s1-static", + "seed": 1166, + "job_id": "835379468ec3", + "status": "succeeded", + "error": null, + "output": "BenchS1Static/20261104-174427-ff3d1339/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.786, + "seam_jump_max": 0.96, + "seam_look_mean": 0.626, + "seam_look_max": 0.85, + "seam_level_step_db_mean": 4.09, + "seam_level_step_db_max": 9.44, + "texture_fine_seg_last": 0.33088128461276445, + "texture_mid_seg_last": 0.2077561367344242, + "texture_coarse_seg_last": 0.00950641334071453, + "wall_minutes": 11.510376906394958 + } + }, + { + "arm": "r2-lf-lookhold-anchor50-s1-s1166", + "scene": "s1-static", + "seed": 1166, + "job_id": "d8b17df7dfb9", + "status": "succeeded", + "error": null, + "output": "BenchS1Static/20261104-175558-fade325a/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.752, + "seam_jump_max": 0.93, + "seam_look_mean": 0.766, + "seam_look_max": 1.19, + "seam_level_step_db_mean": 5.018, + "seam_level_step_db_max": 7.77, + "texture_fine_seg_last": 0.18292503066820404, + "texture_mid_seg_last": 0.17800797159938728, + "texture_coarse_seg_last": 0.011156075845243023, + "wall_minutes": 11.491561122735341 + } + }, + { + "arm": "r2-anchor25-keystone-s1-s2", + "scene": "s1-static", + "seed": 2, + "job_id": "e34a5aebaab4", + "status": "succeeded", + "error": null, + "output": "BenchS1Static/20261104-180728-b9fccaaf/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.8320000000000001, + "seam_jump_max": 1.02, + "seam_look_mean": 1.186, + "seam_look_max": 1.79, + "seam_level_step_db_mean": 14.018, + "seam_level_step_db_max": 27.16, + "texture_fine_seg_last": 0.3113798549487045, + "texture_mid_seg_last": 0.1128822421544756, + "texture_coarse_seg_last": 0.031291867810279345, + "wall_minutes": 13.297784868876139 + } + }, + { + "arm": "r2-anchor50-keystone-s1-s2", + "scene": "s1-static", + "seed": 2, + "job_id": "08e32162985c", + "status": "succeeded", + "error": null, + "output": "BenchS1Static/20261104-224226-52a65249/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.068, + "seam_jump_max": 1.59, + "seam_look_mean": 1.672, + "seam_look_max": 2.71, + "seam_level_step_db_mean": 10.809999999999999, + "seam_level_step_db_max": 24.1, + "texture_fine_seg_last": 0.2463045971257111, + "texture_mid_seg_last": 0.10665196703853218, + "texture_coarse_seg_last": 0.04088462104969011, + "wall_minutes": 15.136626398563385 + } + }, + { + "arm": "r2-lf-lookhold-anchor25-s1-s2", + "scene": "s1-static", + "seed": 2, + "job_id": "41355297a083", + "status": "succeeded", + "error": null, + "output": "BenchS1Static/20261104-225732-347bb719/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.74, + "seam_jump_max": 0.83, + "seam_look_mean": 0.76, + "seam_look_max": 0.9, + "seam_level_step_db_mean": 4.086, + "seam_level_step_db_max": 8.18, + "texture_fine_seg_last": 0.29678905334572925, + "texture_mid_seg_last": 0.23382692820831796, + "texture_coarse_seg_last": 0.014787088108712279, + "wall_minutes": 11.527834463119508 + } + }, + { + "arm": "r2-lf-lookhold-anchor50-s1-s2", + "scene": "s1-static", + "seed": 2, + "job_id": "c8bfadf286ce", + "status": "succeeded", + "error": null, + "output": "BenchS1Static/20261104-230903-c6d7f38c/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.774, + "seam_jump_max": 0.88, + "seam_look_mean": 0.8859999999999999, + "seam_look_max": 1.25, + "seam_level_step_db_mean": 2.46, + "seam_level_step_db_max": 4.34, + "texture_fine_seg_last": 0.23561469742714838, + "texture_mid_seg_last": 0.2448724676437046, + "texture_coarse_seg_last": 0.04016118180555415, + "wall_minutes": 11.522495539983113 + } + } +] \ No newline at end of file diff --git a/bench/records/counters-round2-confirm.json b/bench/records/counters-round2-confirm.json new file mode 100644 index 000000000..4d1ba4bc6 --- /dev/null +++ b/bench/records/counters-round2-confirm.json @@ -0,0 +1,128 @@ +[ + { + "arm": "ctl-guide-s1-s1166-rerun", + "scene": "s1-static", + "seed": 1166, + "job_id": "fd34bc937628", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-r2/20261104-122900-4ae9b3e3/final/BenchS1Static-r2-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.8539999999999999, + "seam_jump_max": 0.96, + "seam_look_mean": 1.656, + "seam_look_max": 2.18, + "seam_level_step_db_mean": 11.616, + "seam_level_step_db_max": 20.23, + "texture_fine_seg_last": 0.5197123813989294, + "texture_mid_seg_last": 0.1819609042238591, + "texture_coarse_seg_last": 0.006720072671439226, + "wall_minutes": 12.758340466022492 + } + }, + { + "arm": "r2-anchor25-s1-seed2", + "scene": "s1-static", + "seed": 2, + "job_id": "93dd359dc11e", + "status": "succeeded", + "error": null, + "output": "BenchS1Static/20261104-124146-e3030154/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.8119999999999999, + "seam_jump_max": 0.95, + "seam_look_mean": 1.092, + "seam_look_max": 1.88, + "seam_level_step_db_mean": 10.425999999999998, + "seam_level_step_db_max": 18.24, + "texture_fine_seg_last": 0.32494732331049675, + "texture_mid_seg_last": 0.12996948109976164, + "texture_coarse_seg_last": 0.030873558857873833, + "wall_minutes": 15.435348010063171 + } + }, + { + "arm": "r2-anchor50-s1-seed2", + "scene": "s1-static", + "seed": 2, + "job_id": "075ed6de83f7", + "status": "succeeded", + "error": null, + "output": "BenchS1Static/20261104-125709-51dda187/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.9039999999999999, + "seam_jump_max": 1.06, + "seam_look_mean": 1.2120000000000002, + "seam_look_max": 1.72, + "seam_level_step_db_mean": 6.55, + "seam_level_step_db_max": 16.23, + "texture_fine_seg_last": 0.27195261694523426, + "texture_mid_seg_last": 0.12106543379097916, + "texture_coarse_seg_last": 0.03032336704713431, + "wall_minutes": 13.273671638965606 + } + }, + { + "arm": "r2-keystone-s1-seed2", + "scene": "s1-static", + "seed": 2, + "job_id": "ab59d503fd4d", + "status": "succeeded", + "error": null, + "output": "BenchS1Static/20261104-131025-68fb6c3e/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.8, + "seam_jump_max": 0.9, + "seam_look_mean": 1.536, + "seam_look_max": 2.71, + "seam_level_step_db_mean": 10.352, + "seam_level_step_db_max": 20.13, + "texture_fine_seg_last": 0.448289310483646, + "texture_mid_seg_last": 0.1230615246516737, + "texture_coarse_seg_last": 0.009572951981541747, + "wall_minutes": 10.456776134173076 + } + }, + { + "arm": "r2-lf-lookhold-s1-seed2", + "scene": "s1-static", + "seed": 2, + "job_id": "b7ca6da7d075", + "status": "succeeded", + "error": null, + "output": "BenchS1Static/20261104-132053-2e49b9ea/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.806, + "seam_jump_max": 1.02, + "seam_look_mean": 0.8380000000000001, + "seam_look_max": 1.06, + "seam_level_step_db_mean": 3.664, + "seam_level_step_db_max": 8.88, + "texture_fine_seg_last": 0.5501537852935141, + "texture_mid_seg_last": 0.41055683061641673, + "texture_coarse_seg_last": 0.0009888595355270686, + "wall_minutes": 8.652199832598368 + } + }, + { + "arm": "r2-lf-lookhold-s2-seed2", + "scene": "s2-arc", + "seed": 2, + "job_id": "a3d6a044fd6b", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc/20261104-132936-10da7ca8/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.31, + "seam_jump_max": 1.64, + "seam_look_mean": 0.386, + "seam_look_max": 0.72, + "seam_level_step_db_mean": 4.252, + "seam_level_step_db_max": 9.04, + "texture_fine_seg_last": 0.8278488882134617, + "texture_mid_seg_last": 0.2718713207179971, + "texture_coarse_seg_last": 0.33871740334294115, + "wall_minutes": 9.195139491558075 + } + } +] \ No newline at end of file diff --git a/bench/records/counters-round2-long-moments.json b/bench/records/counters-round2-long-moments.json new file mode 100644 index 000000000..fbce45651 --- /dev/null +++ b/bench/records/counters-round2-long-moments.json @@ -0,0 +1,410 @@ +[ + { + "arm": "ctl-guide-s1-120s", + "output": "BenchS1Static/20261104-133844-23b1c5b1/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "moments": [ + 62, + 300, + 583, + 1000, + 1501, + 2000, + 2500, + 2868 + ], + "crop": [ + 0, + 0, + 240, + 150 + ], + "texture_drift": { + "62": { + "fine": 0.0, + "mid": 0.0, + "coarse": 0.0 + }, + "300": { + "fine": 0.18025993421208494, + "mid": 0.04490486020150464, + "coarse": 0.000761415263032645 + }, + "583": { + "fine": 0.5356445216304937, + "mid": 0.17558025571703065, + "coarse": 0.016023705618812927 + }, + "1000": { + "fine": 1.1075384076182646, + "mid": 0.406155001202102, + "coarse": 0.056311464238995246 + }, + "1501": { + "fine": 1.3989414462224836, + "mid": 0.7161807400771484, + "coarse": 0.10258601894262132 + }, + "2000": { + "fine": 1.6459796696597704, + "mid": 1.0713307837645822, + "coarse": 0.08019719226533902 + }, + "2500": { + "fine": 1.657883979627354, + "mid": 1.1562509725508072, + "coarse": 0.21290180180563448 + }, + "2868": { + "fine": 1.7060122912259976, + "mid": 1.170543964304953, + "coarse": 0.08828520286094746 + } + }, + "seams": [ + { + "jump_ratio": 0.7, + "look_ratio": 2.13, + "level_step_db": 16.58 + }, + { + "jump_ratio": 0.97, + "look_ratio": 1.75, + "level_step_db": 0.46 + }, + { + "jump_ratio": 0.77, + "look_ratio": 1.68, + "level_step_db": 0.75 + }, + { + "jump_ratio": 0.95, + "look_ratio": 0.73, + "level_step_db": 7.26 + }, + { + "jump_ratio": 0.88, + "look_ratio": 1.03, + "level_step_db": 13.22 + }, + { + "jump_ratio": 1.09, + "look_ratio": 2.61, + "level_step_db": 3.32 + }, + { + "jump_ratio": 0.91, + "look_ratio": 0.99, + "level_step_db": 5.61 + }, + { + "jump_ratio": 1.05, + "look_ratio": 1.01, + "level_step_db": 4.95 + }, + { + "jump_ratio": 1.06, + "look_ratio": 1.83, + "level_step_db": 36.53 + }, + { + "jump_ratio": 1.02, + "look_ratio": 2.35, + "level_step_db": 6.42 + }, + { + "jump_ratio": 1.16, + "look_ratio": 2.47, + "level_step_db": 0.17 + }, + { + "jump_ratio": 1.43, + "look_ratio": 1.9, + "level_step_db": 6.28 + }, + { + "jump_ratio": 1.64, + "look_ratio": 1.88, + "level_step_db": 6.19 + }, + { + "jump_ratio": 1.23, + "look_ratio": 3.26, + "level_step_db": 4.78 + }, + { + "jump_ratio": 1.3, + "look_ratio": 0.9, + "level_step_db": 0.55 + }, + { + "jump_ratio": 0.93, + "look_ratio": 0.72, + "level_step_db": 1.42 + }, + { + "jump_ratio": 0.73, + "look_ratio": 1.54, + "level_step_db": 1.73 + }, + { + "jump_ratio": 0.69, + "look_ratio": 0.43, + "level_step_db": 2.11 + }, + { + "jump_ratio": 1.62, + "look_ratio": 1.31, + "level_step_db": 2.93 + }, + { + "jump_ratio": 1.94, + "look_ratio": 3.75, + "level_step_db": 1.52 + }, + { + "jump_ratio": 1.72, + "look_ratio": 2.19, + "level_step_db": 1.84 + }, + { + "jump_ratio": 1.42, + "look_ratio": 1.33, + "level_step_db": 0.7 + }, + { + "jump_ratio": 0.91, + "look_ratio": 2.83, + "level_step_db": 0.45 + }, + { + "jump_ratio": 1.75, + "look_ratio": 1.93, + "level_step_db": 0.14 + }, + { + "jump_ratio": 0.89, + "look_ratio": 3.5, + "level_step_db": 8.96 + }, + { + "jump_ratio": 6.56, + "look_ratio": 11.51, + "level_step_db": 26.55 + }, + { + "jump_ratio": 1.1, + "look_ratio": 3.2, + "level_step_db": 2.36 + }, + { + "jump_ratio": 1.12, + "look_ratio": 1.76, + "level_step_db": 0.07 + } + ] + }, + { + "arm": "r2-anchor50-s1-120s", + "output": "BenchS1Static/20261104-144414-d9c61bc5/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "moments": [ + 62, + 300, + 583, + 1000, + 1501, + 2000, + 2500, + 2868 + ], + "crop": [ + 0, + 0, + 240, + 150 + ], + "texture_drift": { + "62": { + "fine": 0.0, + "mid": 0.0, + "coarse": 0.0 + }, + "300": { + "fine": 0.1369508249460004, + "mid": 0.03783186367725497, + "coarse": 0.0049794677468551415 + }, + "583": { + "fine": 0.269789364587522, + "mid": 0.11577271085341745, + "coarse": 0.027805739291449503 + }, + "1000": { + "fine": 0.35984393865915915, + "mid": 0.16206212075564824, + "coarse": 0.04574123675256187 + }, + "1501": { + "fine": 0.6024172178552615, + "mid": 0.22709741147372972, + "coarse": 0.028619020258172798 + }, + "2000": { + "fine": 0.7373379159774919, + "mid": 0.3072406728734482, + "coarse": 0.00032004389446539124 + }, + "2500": { + "fine": 0.7589507674521561, + "mid": 0.2725833807242856, + "coarse": 0.09379867156925861 + }, + "2868": { + "fine": 1.0671458806310952, + "mid": 0.4622265049839578, + "coarse": 0.08861803287357502 + } + }, + "seams": [ + { + "jump_ratio": 0.69, + "look_ratio": 1.71, + "level_step_db": 16.58 + }, + { + "jump_ratio": 1.15, + "look_ratio": 1.6, + "level_step_db": 1.54 + }, + { + "jump_ratio": 1.18, + "look_ratio": 0.3, + "level_step_db": 12.23 + }, + { + "jump_ratio": 0.9, + "look_ratio": 0.66, + "level_step_db": 15.43 + }, + { + "jump_ratio": 1.32, + "look_ratio": 0.98, + "level_step_db": 20.34 + }, + { + "jump_ratio": 1.28, + "look_ratio": 2.36, + "level_step_db": 2.91 + }, + { + "jump_ratio": 1.0, + "look_ratio": 1.2, + "level_step_db": 18.02 + }, + { + "jump_ratio": 0.93, + "look_ratio": 0.91, + "level_step_db": 2.99 + }, + { + "jump_ratio": 0.91, + "look_ratio": 0.55, + "level_step_db": 2.57 + }, + { + "jump_ratio": 0.83, + "look_ratio": 0.56, + "level_step_db": 10.1 + }, + { + "jump_ratio": 0.82, + "look_ratio": 1.13, + "level_step_db": 2.54 + }, + { + "jump_ratio": 0.96, + "look_ratio": 1.05, + "level_step_db": 0.15 + }, + { + "jump_ratio": 1.13, + "look_ratio": 1.9, + "level_step_db": 4.84 + }, + { + "jump_ratio": 0.87, + "look_ratio": 1.06, + "level_step_db": 1.06 + }, + { + "jump_ratio": 0.75, + "look_ratio": 1.02, + "level_step_db": 1.94 + }, + { + "jump_ratio": 0.87, + "look_ratio": 1.49, + "level_step_db": 17.5 + }, + { + "jump_ratio": 0.92, + "look_ratio": 1.0, + "level_step_db": 2.07 + }, + { + "jump_ratio": 0.83, + "look_ratio": 1.45, + "level_step_db": 5.33 + }, + { + "jump_ratio": 0.62, + "look_ratio": 1.66, + "level_step_db": 14.88 + }, + { + "jump_ratio": 0.53, + "look_ratio": 1.2, + "level_step_db": 9.76 + }, + { + "jump_ratio": 0.56, + "look_ratio": 1.53, + "level_step_db": 4.33 + }, + { + "jump_ratio": 0.51, + "look_ratio": 1.11, + "level_step_db": 13.9 + }, + { + "jump_ratio": 0.86, + "look_ratio": 0.98, + "level_step_db": 8.67 + }, + { + "jump_ratio": 0.92, + "look_ratio": 0.69, + "level_step_db": 0.05 + }, + { + "jump_ratio": 1.18, + "look_ratio": 0.79, + "level_step_db": 5.08 + }, + { + "jump_ratio": 1.46, + "look_ratio": 1.87, + "level_step_db": 9.73 + }, + { + "jump_ratio": 1.18, + "look_ratio": 2.65, + "level_step_db": 4.54 + }, + { + "jump_ratio": 0.9, + "look_ratio": 1.56, + "level_step_db": 3.42 + } + ] + } +] \ No newline at end of file diff --git a/bench/records/counters-round2-long.json b/bench/records/counters-round2-long.json new file mode 100644 index 000000000..315fe41b0 --- /dev/null +++ b/bench/records/counters-round2-long.json @@ -0,0 +1,44 @@ +[ + { + "arm": "ctl-guide-s1-120s", + "scene": "s1-static", + "seed": 1166, + "job_id": "8d1614ced27a", + "status": "succeeded", + "error": null, + "output": "BenchS1Static/20261104-133844-23b1c5b1/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.3407142857142857, + "seam_jump_max": 6.56, + "seam_look_mean": 2.232857142857143, + "seam_look_max": 11.51, + "seam_level_step_db_mean": 5.851785714285714, + "seam_level_step_db_max": 36.53, + "texture_fine_seg_last": 1.7060122912259976, + "texture_mid_seg_last": 1.170543964304953, + "texture_coarse_seg_last": 0.08828520286094746, + "wall_minutes": 52.203673243522644 + } + }, + { + "arm": "r2-anchor50-s1-120s", + "scene": "s1-static", + "seed": 1166, + "job_id": "107859a31bc1", + "status": "succeeded", + "error": null, + "output": "BenchS1Static/20261104-144414-d9c61bc5/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.9307142857142857, + "seam_jump_max": 1.46, + "seam_look_mean": 1.2489285714285714, + "seam_look_max": 2.65, + "seam_level_step_db_mean": 7.589285714285714, + "seam_level_step_db_max": 20.34, + "texture_fine_seg_last": 1.0671458806310952, + "texture_mid_seg_last": 0.4622265049839578, + "texture_coarse_seg_last": 0.08861803287357502, + "wall_minutes": 67.82185592651368 + } + } +] \ No newline at end of file diff --git a/bench/records/counters-round3-long-moments.json b/bench/records/counters-round3-long-moments.json new file mode 100644 index 000000000..e0ad44eb8 --- /dev/null +++ b/bench/records/counters-round3-long-moments.json @@ -0,0 +1,487 @@ +[ + { + "arm": "r3-anchor25-s1-120s", + "output": "BenchS1Static/20261104-233558-8c6ae697/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "moments": [ + 62, + 300, + 583, + 1000, + 1501, + 2000, + 2500, + 2868 + ], + "crop": [ + 0, + 0, + 240, + 150 + ], + "texture_drift": { + "62": { + "fine": 0.0, + "mid": 0.0, + "coarse": 0.0 + }, + "300": { + "fine": 0.15334517616385027, + "mid": 0.04591805925855212, + "coarse": 0.0018679258900470392 + }, + "583": { + "fine": 0.36683259200135493, + "mid": 0.12383734756733206, + "coarse": 0.015490472469675862 + }, + "1000": { + "fine": 0.49667659932345387, + "mid": 0.1793955659595618, + "coarse": 0.02192424808923237 + }, + "1501": { + "fine": 0.7713517464612992, + "mid": 0.27112590517376556, + "coarse": 0.05175386349388206 + }, + "2000": { + "fine": 0.9314887724824489, + "mid": 0.3186770924337395, + "coarse": 0.012653034895651024 + }, + "2500": { + "fine": 1.0123241014626363, + "mid": 0.3560729875623916, + "coarse": 0.046590245329657345 + }, + "2868": { + "fine": 1.1076040063709027, + "mid": 0.43039061064808104, + "coarse": 0.01932700686817017 + } + }, + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 0.69, + "look_ratio": 1.98, + "level_step_db": 16.58, + "typical_delta": 2.0 + }, + { + "seconds": 9.417, + "jump_ratio": 1.15, + "look_ratio": 1.57, + "level_step_db": 5.73, + "typical_delta": 2.0 + }, + { + "seconds": 13.667, + "jump_ratio": 0.78, + "look_ratio": 1.16, + "level_step_db": 12.1, + "typical_delta": 2.0 + }, + { + "seconds": 17.917, + "jump_ratio": 0.98, + "look_ratio": 0.47, + "level_step_db": 23.54, + "typical_delta": 2.0 + }, + { + "seconds": 22.167, + "jump_ratio": 1.04, + "look_ratio": 0.46, + "level_step_db": 2.85, + "typical_delta": 2.0 + }, + { + "seconds": 26.417, + "jump_ratio": 0.89, + "look_ratio": 0.43, + "level_step_db": 24.17, + "typical_delta": 2.0 + }, + { + "seconds": 30.667, + "jump_ratio": 0.88, + "look_ratio": 0.31, + "level_step_db": 1.96, + "typical_delta": 2.0 + }, + { + "seconds": 34.917, + "jump_ratio": 0.87, + "look_ratio": 0.63, + "level_step_db": 18.52, + "typical_delta": 2.0 + }, + { + "seconds": 39.167, + "jump_ratio": 0.82, + "look_ratio": 0.41, + "level_step_db": 4.08, + "typical_delta": 2.0 + }, + { + "seconds": 43.417, + "jump_ratio": 0.75, + "look_ratio": 0.81, + "level_step_db": 0.37, + "typical_delta": 2.0 + }, + { + "seconds": 47.667, + "jump_ratio": 0.78, + "look_ratio": 0.92, + "level_step_db": 24.34, + "typical_delta": 2.0 + }, + { + "seconds": 51.917, + "jump_ratio": 0.75, + "look_ratio": 0.36, + "level_step_db": 3.78, + "typical_delta": 2.0 + }, + { + "seconds": 56.167, + "jump_ratio": 1.29, + "look_ratio": 2.37, + "level_step_db": 1.93, + "typical_delta": 2.0 + }, + { + "seconds": 60.417, + "jump_ratio": 0.86, + "look_ratio": 0.68, + "level_step_db": 1.49, + "typical_delta": 2.0 + }, + { + "seconds": 64.667, + "jump_ratio": 0.85, + "look_ratio": 0.47, + "level_step_db": 1.32, + "typical_delta": 2.0 + }, + { + "seconds": 68.917, + "jump_ratio": 0.74, + "look_ratio": 0.74, + "level_step_db": 0.05, + "typical_delta": 2.0 + }, + { + "seconds": 73.167, + "jump_ratio": 0.96, + "look_ratio": 1.53, + "level_step_db": 1.71, + "typical_delta": 2.0 + }, + { + "seconds": 77.417, + "jump_ratio": 0.76, + "look_ratio": 0.47, + "level_step_db": 0.27, + "typical_delta": 2.0 + }, + { + "seconds": 81.667, + "jump_ratio": 0.73, + "look_ratio": 0.53, + "level_step_db": 0.59, + "typical_delta": 2.0 + }, + { + "seconds": 85.917, + "jump_ratio": 0.69, + "look_ratio": 0.65, + "level_step_db": 4.57, + "typical_delta": 2.07 + }, + { + "seconds": 90.167, + "jump_ratio": 0.84, + "look_ratio": 0.56, + "level_step_db": 3.56, + "typical_delta": 2.07 + }, + { + "seconds": 94.417, + "jump_ratio": 0.76, + "look_ratio": 0.53, + "level_step_db": 0.15, + "typical_delta": 2.0 + }, + { + "seconds": 98.667, + "jump_ratio": 1.23, + "look_ratio": 1.82, + "level_step_db": 1.87, + "typical_delta": 2.48 + }, + { + "seconds": 102.917, + "jump_ratio": 1.07, + "look_ratio": 1.9, + "level_step_db": 1.55, + "typical_delta": 2.48 + }, + { + "seconds": 107.167, + "jump_ratio": 1.24, + "look_ratio": 1.89, + "level_step_db": 1.44, + "typical_delta": 2.0 + }, + { + "seconds": 111.417, + "jump_ratio": 1.08, + "look_ratio": 1.26, + "level_step_db": 2.04, + "typical_delta": 2.16 + }, + { + "seconds": 115.667, + "jump_ratio": 1.07, + "look_ratio": 1.3, + "level_step_db": 2.51, + "typical_delta": 2.62 + }, + { + "seconds": 119.917, + "jump_ratio": 1.86, + "look_ratio": 2.7, + "level_step_db": 0.29, + "typical_delta": 2.62 + } + ] + }, + { + "arm": "r3-lf-lookhold-anchor25-s1-120s", + "output": "BenchS1Static/20261105-004501-92f5fbde/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "moments": [ + 62, + 300, + 583, + 1000, + 1501, + 2000, + 2500, + 2868 + ], + "crop": [ + 0, + 0, + 240, + 150 + ], + "texture_drift": { + "62": { + "fine": 0.0, + "mid": 0.0, + "coarse": 0.0 + }, + "300": { + "fine": 0.1437505860870709, + "mid": 0.07206262169014278, + "coarse": 0.024209182154661238 + }, + "583": { + "fine": 0.25278929716938614, + "mid": 0.18539856345038577, + "coarse": 0.026384246857596425 + }, + "1000": { + "fine": 0.47825861734320874, + "mid": 0.2509942668353939, + "coarse": 0.0040495054922650145 + }, + "1501": { + "fine": 0.5752855183962793, + "mid": 0.3676001479256144, + "coarse": 0.050833922905556266 + }, + "2000": { + "fine": 0.4082965307947696, + "mid": 0.35065042286939696, + "coarse": 0.024936206661697595 + }, + "2500": { + "fine": 0.696817019162373, + "mid": 0.41072718735028707, + "coarse": 0.02951330154228991 + }, + "2868": { + "fine": 0.9299567713663541, + "mid": 0.6372666412000864, + "coarse": 0.06779884362428489 + } + }, + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 0.72, + "look_ratio": 0.77, + "level_step_db": 5.73, + "typical_delta": 2.0 + }, + { + "seconds": 10.25, + "jump_ratio": 0.67, + "look_ratio": 0.69, + "level_step_db": 7.27, + "typical_delta": 2.0 + }, + { + "seconds": 15.333, + "jump_ratio": 0.72, + "look_ratio": 0.83, + "level_step_db": 6.75, + "typical_delta": 2.0 + }, + { + "seconds": 20.417, + "jump_ratio": 0.78, + "look_ratio": 0.57, + "level_step_db": 1.22, + "typical_delta": 2.0 + }, + { + "seconds": 25.5, + "jump_ratio": 0.77, + "look_ratio": 0.89, + "level_step_db": 5.93, + "typical_delta": 2.0 + }, + { + "seconds": 30.583, + "jump_ratio": 0.64, + "look_ratio": 0.75, + "level_step_db": 15.04, + "typical_delta": 2.0 + }, + { + "seconds": 35.667, + "jump_ratio": 0.99, + "look_ratio": 1.03, + "level_step_db": 5.97, + "typical_delta": 2.0 + }, + { + "seconds": 40.75, + "jump_ratio": 0.73, + "look_ratio": 0.69, + "level_step_db": 2.86, + "typical_delta": 2.0 + }, + { + "seconds": 45.833, + "jump_ratio": 0.99, + "look_ratio": 0.69, + "level_step_db": 0.03, + "typical_delta": 2.0 + }, + { + "seconds": 50.917, + "jump_ratio": 0.82, + "look_ratio": 0.69, + "level_step_db": 3.47, + "typical_delta": 2.0 + }, + { + "seconds": 56.0, + "jump_ratio": 0.85, + "look_ratio": 0.67, + "level_step_db": 10.86, + "typical_delta": 2.0 + }, + { + "seconds": 61.083, + "jump_ratio": 0.83, + "look_ratio": 0.67, + "level_step_db": 7.45, + "typical_delta": 2.0 + }, + { + "seconds": 66.167, + "jump_ratio": 0.76, + "look_ratio": 0.65, + "level_step_db": 0.87, + "typical_delta": 2.0 + }, + { + "seconds": 71.25, + "jump_ratio": 1.02, + "look_ratio": 0.9, + "level_step_db": 8.98, + "typical_delta": 2.0 + }, + { + "seconds": 76.333, + "jump_ratio": 4.24, + "look_ratio": 6.36, + "level_step_db": 8.53, + "typical_delta": 2.0 + }, + { + "seconds": 81.417, + "jump_ratio": 0.91, + "look_ratio": 0.85, + "level_step_db": 2.26, + "typical_delta": 2.0 + }, + { + "seconds": 86.5, + "jump_ratio": 0.8, + "look_ratio": 0.61, + "level_step_db": 6.45, + "typical_delta": 2.0 + }, + { + "seconds": 91.583, + "jump_ratio": 0.78, + "look_ratio": 0.68, + "level_step_db": 4.55, + "typical_delta": 2.0 + }, + { + "seconds": 96.667, + "jump_ratio": 1.14, + "look_ratio": 0.73, + "level_step_db": 1.72, + "typical_delta": 2.0 + }, + { + "seconds": 101.75, + "jump_ratio": 0.91, + "look_ratio": 0.56, + "level_step_db": 0.08, + "typical_delta": 2.09 + }, + { + "seconds": 106.833, + "jump_ratio": 1.25, + "look_ratio": 2.78, + "level_step_db": 3.01, + "typical_delta": 2.09 + }, + { + "seconds": 111.917, + "jump_ratio": 1.27, + "look_ratio": 4.71, + "level_step_db": 9.91, + "typical_delta": 2.52 + }, + { + "seconds": 117.0, + "jump_ratio": 0.86, + "look_ratio": 1.21, + "level_step_db": 7.74, + "typical_delta": 2.52 + } + ] + } +] diff --git a/bench/records/counters-round3-long.json b/bench/records/counters-round3-long.json new file mode 100644 index 000000000..28ac59224 --- /dev/null +++ b/bench/records/counters-round3-long.json @@ -0,0 +1,44 @@ +[ + { + "arm": "r3-anchor25-s1-120s", + "scene": "s1-static", + "seed": 1166, + "job_id": "25f6bd1da507", + "status": "succeeded", + "error": null, + "output": "BenchS1Static/20261104-233558-8c6ae697/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.9432142857142857, + "seam_jump_max": 1.86, + "seam_look_mean": 1.0325, + "seam_look_max": 2.7, + "seam_level_step_db_mean": 5.834285714285714, + "seam_level_step_db_max": 24.34, + "texture_fine_seg_last": 1.1076040063709027, + "texture_mid_seg_last": 0.43039061064808104, + "texture_coarse_seg_last": 0.01932700686817017, + "wall_minutes": 69.08823354641596 + } + }, + { + "arm": "r3-lf-lookhold-anchor25-s1-120s", + "scene": "s1-static", + "seed": 1166, + "job_id": "9877bbfd85a8", + "status": "succeeded", + "error": null, + "output": "BenchS1Static/20261105-004501-92f5fbde/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.0195652173913043, + "seam_jump_max": 4.24, + "seam_look_mean": 1.26, + "seam_look_max": 6.36, + "seam_level_step_db_mean": 5.507826086956522, + "seam_level_step_db_max": 15.04, + "texture_fine_seg_last": 0.9299567713663541, + "texture_mid_seg_last": 0.6372666412000864, + "texture_coarse_seg_last": 0.06779884362428489, + "wall_minutes": 48.023191964626314 + } + } +] \ No newline at end of file diff --git a/bench/records/counters.md b/bench/records/counters.md new file mode 100644 index 000000000..d699d67f0 --- /dev/null +++ b/bench/records/counters.md @@ -0,0 +1,1174 @@ +# Counter round (Task 11) + +Round 1 ran every counter alone against today's chain at 6 segments (ruling +R27), on S1 static, S2 arc and S3 dialogue walk, at seed 1166. Records: +`bench/records/counters-controls.json` (10 controls: guide on S1-S3 and +last_frame on S1-S2, each at seeds 1166 and 2) and +`bench/records/counters-round1.json` (25 arms). All 35 succeeded on the bench +server (lem, port 8766). + +Round 2 (all on S1, plus one S2 diagnostic) is in +`bench/records/counters-round2-confirm.json` (6), +`counters-round2-combos.json` (8, merged from two driver runs split by a power +loss), `counters-round2-long.json` (2 two-minute takes) and +`counters-round2-long-moments.json` (a measure-only pass on the two-minute +takes at eight moments, with every seam). All 16 succeeded. The results and +the **summary table Tasks 15 and 20 build from** are at the end, under +[Round 2 results](#round-2-results) and [Defaults by shot +type](#defaults-by-shot-type). Every verdict was re-scored on 2026-11-05 +under ruling R35 (same-seed reruns are not noise-free, and segment-1 drift +labels): see [Re-scored under R35](#re-scored-under-r35-2026-11-05). +Round 3 (anchor 0.25 at two minutes) is at the end, under [Round 3 (2 +min)](#round-3-2-min-2026-11-05). + +## How the verdicts are read + +- **Control.** Each arm is read against its own scene's control at its own + continuity, at seed 1166 (`ctl-guide-*` or `ctl-lastframe-*`). +- **Floor.** The noise floor is seed variance (ruling R26): + `noise_floor(control@1166, control@2)` for each scene and continuity. It + comes from two seeds only, so it is thin. A measure where the two seeds + happened to agree has a floor near 0, and any difference counts on it + (S1 last_frame `jump max` is 0.000, for example). + *(Superseded 2026-11-05, ruling R35: same-seed reruns are not noise-free, + so the floor is now `combined_floor(seed floor, rerun gap)`, the larger of + the seed floor and the same-seed rerun gap per measure (`bench/verdict.py`). + The floors and the rerun gap are in `noise-floor.md`, *Correction (2026-11-05, + ruling R35)*; the verdicts here were re-scored on it, see* Re-scored under + R35 *below.)* +- **Measures.** `verdict()` reads every measure in the record except + `wall_minutes`. Every measure is lower-is-better. +- **Primary measure per scene.** + - S1: `texture_fine_seg_last`. + - S2: `seam_look_mean` and `seam_jump_mean`, one verdict each. + - S3: `seam_jump_mean`, `seam_look_mean` and `texture_fine_seg_last`, one + verdict each. +- **Two verdict columns.** + - *All measures* is the design's rule as written. + - *Picture only* drops the two `seam_level_step_db` (soundtrack) measures. + S1 and S2 carry room tone and garden ambience, not a held voice. S1's level + step is large and noisy (7.7 vs 5.2 dB between the two control seeds; + 9.9 dB mean and 36.6 dB max at 10 segments in `noise-floor.md`). Almost + every S1 guide arm raises it by 3-7 dB, so under the rule as written no + S1 arm can win on picture. Whether it should count is a question for Don + (`for-don.md`). +- **What `mixed` means here.** `verdict()` says `mixed` both when the primary + wins but another measure is worse past its floor, and when the primary is + inside its floor but another measure is worse. The cells say which: + **bold** is worse than the control past the floor, _italic_ is better past + it, and plain is inside the floor. +- **Backoff arms (R29).** `guide_backoff 17` takes 17 frames off every + hand-over, so the take is 549 frames, not 634. The arm's moments sit at + the same segments as the control's (62/234/498 against 62/277/583). Its + seam means are over the same 5 seams and its texture is at the same last + segment, so it is read segment by segment against the control. +- **last_frame moments.** last_frame takes run 734 frames, with moments at + 62/307/673. +- **The eye check.** Frames came from `GET /api/gallery/{name}/frames?at=` on the bench server, at the moments in each arm file. They + were looked at for every control, every arm that wins or is mixed, and + every arm whose texture number moved a lot. S2's last_frame arms also got + an 8-frame contact sheet (`count=8`). To rerun this, call the same route on + each output named in the tables. + +## S1 static + +Primary: fine-band texture drift at the last moment. Seams second. + +### Guide continuity + +| arm | verdict (all measures) | verdict (picture only) | jump | jump max | look | look max | level dB | level dB max | tex fine | tex mid | tex coarse | job | output | +|---|---|---|---|---|---|---|---|---|---|---|---|---|---| +| floor (seed 1166 vs 2) | | | 0.058 | 0.050 | 0.120 | 0.140 | 2.432 | 2.300 | 0.081 | 0.032 | 0.008 | | | +| ctl-guide-s1-s1166 | control | | 0.854 | 0.970 | 1.464 | 2.130 | 7.654 | 16.580 | 0.536 | 0.176 | 0.016 | `b40e855eb553` | `BenchS1Static/20261104-060443-90c37bc2/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4` | +| ctl-guide-s1-s2 | control | | 0.796 | 0.920 | 1.344 | 2.270 | 5.222 | 18.880 | 0.455 | 0.144 | 0.008 | `894911aed6c7` | `BenchS1Static/20261104-062828-0642442c/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4` | +| r1-anchor25-s1 | tex: mixed | tex: mixed | **0.928** | **1.150** | _1.128_ | _1.980_ | **12.160** | **23.540** | _0.367_ | _0.124_ | 0.015 | `0a40c41af97c` | `BenchS1Static/20261104-054934-fdc9b551/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4` | +| r1-anchor50-s1 | tex: mixed | tex: mixed | **1.048** | **1.320** | _1.050_ | _1.710_ | **13.224** | **20.340** | _0.270_ | _0.116_ | **0.028** | `fa17a2c51e33` | `BenchS1Static/20261104-061515-5949edcc/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4` | +| r1-anchor75-s1 | tex: mixed | tex: mixed | **1.300** | **1.960** | **1.752** | **3.080** | **15.258** | **23.770** | _0.260_ | _0.103_ | **0.045** | `f17bce198a2d` | `BenchS1Static/20261104-063901-253f19de/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4` | +| r1-backoff17-s1 | tex: mixed | tex: mixed | 0.886 | 0.970 | 1.522 | **2.590** | **10.744** | **22.350** | 0.462 | 0.167 | 0.014 | `c594521e59d1` | `BenchS1Static/20261104-070045-f9d2dc73/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4` | +| r1-keystone-s1 | tex: mixed | tex: mixed | 0.850 | **1.040** | 1.550 | _1.930_ | **14.556** | **23.260** | _0.373_ | _0.116_ | 0.020 | `1e1b6116ce26` | `BenchS1Static/20261104-071947-19ca76a4/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4` | +| r1-lookhold-s1 | tex: mixed | tex: mixed | 0.896 | **1.040** | _1.310_ | 2.110 | **11.678** | **23.210** | 0.464 | 0.145 | _0.007_ | `751b43732fb9` | `BenchS1Static/20261104-074252-80046177/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4` | +| r1-antiburn-s1 | tex: mixed | tex: mixed | _0.724_ | _0.880_ | **2.646** | **3.210** | 5.458 | _13.260_ | _0.046_ | _0.114_ | **0.074** | `049d652e1648` | `BenchS1Static/20261104-080411-0301f0c0/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4` | + +### last_frame continuity + +| arm | verdict (all measures) | verdict (picture only) | jump | jump max | look | look max | level dB | level dB max | tex fine | tex mid | tex coarse | job | output | +|---|---|---|---|---|---|---|---|---|---|---|---|---|---| +| floor (seed 1166 vs 2) | | | 0.022 | 0.000 | 0.144 | 0.010 | 0.702 | 0.540 | 0.036 | 0.087 | 0.003 | | | +| ctl-lastframe-s1-s1166 | control | | 1.096 | 1.180 | 1.264 | 1.790 | 3.372 | 8.270 | 0.569 | 0.366 | 0.009 | `3a73cb0f5d28` | `BenchS1Static/20261104-065215-5b43dda0/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4` | +| ctl-lastframe-s1-s2 | control | | 1.074 | 1.180 | 1.408 | 1.800 | 4.074 | 7.730 | 0.605 | 0.453 | 0.006 | `f053abe1ce5f` | `BenchS1Static/20261104-071117-ae50341d/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4` | +| r1-lf-carrypick-s1 | tex: mixed | tex: wins | _0.994_ | _1.120_ | 1.252 | _1.730_ | **6.796** | **10.150** | _0.533_ | 0.330 | _0.006_ | `ae932366ea0e` | `BenchLfS1Static/20261104-081730-5241ac41/final/BenchLfS1Static-chained_keyframe_to_video_audio.0-0.0.mp4` | +| r1-lf-lookhold-s1 | tex: mixed | tex: wins | _0.788_ | _0.890_ | _0.634_ | _0.970_ | **4.324** | _6.780_ | _0.491_ | 0.304 | _0.005_ | `be3f574df3db` | `BenchS1Static/20261104-083621-b688e963/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4` | + +### Eye notes (S1) + +- **Control, guide (both seeds).** Clean at frame 62. By 583 the stripes and + the wall have burned into a crawling, mottled, over-sharpened texture. The + black circle's edge has gone ragged and the face is harder and more + saturated. Seed 2 looks the same. The numbers are measuring this burn. +- **anchor 0.25.** At 583 the stripes are visibly cleaner than the control's, + though the wall is still mottled. The face holds identity. Less burn than + the control. +- **anchor 0.5.** The cleanest of the guide arms at 583. The stripes are + nearly flat paint and the wall's mottling is reduced. Identity holds, with + a slight warm shift. The eye agrees with the texture number. +- **anchor 0.75.** The stripes are clean at 583, but she has drifted: a + narrower face, lighter hair and a different expression. Seams are worse + too. Not carried. +- **guide_backoff 17.** At 498 it burns like the control. No visible gain. +- **keystone.** At 583 it burns a little less than the control (the stripes + are mottled, the wall still crawls). A modest gain that matches its texture + number. +- **look_hold (guide).** At 583 it burns like the control. Its look gain is + colour steadiness at the seams, not the burn. +- **Anti-burn LoRA.** The low texture number is a lie. The background stays + clean, but the face smears to a soft, ghosted blur by 277, and by 583 the + identity is broken (the features dissolve). This is the #1175 trap: the + measure is low because detail is gone. +- **Control, last_frame.** It burns more than guide by 673. The teal square + carries dark blotches. +- **last_frame + carry_pick sharpest 4.** Looks the same as its control: the + same burn and the same blotched square. Its texture gain sits on the floor + (0.0360 against 0.0357). +- **last_frame + look_hold.** Still burns at 673 (wall crawl, a blotch in the + square), at about the control's level. Colour across seams is steadier and + identity holds. The win is in the seams, not the burn. + +## S2 moving arc + +Primary: seam look and seam jump means. Texture second. + +### Guide continuity + +| arm | verdict (all measures) | verdict (picture only) | jump | jump max | look | look max | level dB | level dB max | tex fine | tex mid | tex coarse | job | output | +|---|---|---|---|---|---|---|---|---|---|---|---|---|---| +| floor (seed 1166 vs 2) | | | 0.090 | 0.230 | 0.068 | 0.270 | 2.790 | 5.540 | 0.605 | 1.210 | 1.660 | | | +| ctl-guide-s2-s1166 | control | | 1.362 | 2.090 | 1.286 | 1.740 | 1.462 | 3.430 | 3.084 | 2.293 | 1.430 | `dedd341c9108` | `BenchS2Arc/20261104-073022-cd97c6a1/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4` | +| ctl-guide-s2-s2 | control | | 1.272 | 1.860 | 1.218 | 2.010 | 4.252 | 8.970 | 3.689 | 3.504 | 3.090 | `01146e925c94` | `BenchS2Arc/20261104-075344-d76c369a/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4` | +| r1-anchor25-s2 | look: loses / jump: mixed | look: loses / jump: mixed | _1.228_ | _1.640_ | **2.032** | **3.600** | 1.364 | 3.150 | _0.340_ | _0.169_ | 0.124 | `4350da6ac745` | `BenchS2Arc/20261104-085551-68fe5c04/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4` | +| r1-anchor50-s2 | look: loses / jump: loses | look: loses / jump: loses | **1.496** | **2.370** | **3.214** | **5.620** | 0.988 | 3.050 | 3.056 | 2.424 | 2.401 | `3408f7cf9fa9` | `BenchS2Arc/20261104-091939-682f41d1/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4` | +| r1-anchor75-s2 | look: loses / jump: loses | look: loses / jump: loses | **1.666** | **2.730** | **3.500** | **5.540** | 1.152 | 3.200 | _2.313_ | 1.443 | 1.054 | `6fb052f46229` | `BenchS2Arc/20261104-093256-ab1bf1a9/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4` | +| r1-backoff17-s2 | look: loses / jump: mixed | look: loses / jump: mixed | 1.370 | 1.890 | **1.430** | **2.610** | 1.126 | 1.990 | 2.873 | 1.534 | 0.000 | `ea394296a02d` | `BenchS2Arc/20261104-094613-88f4b225/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4` | +| r1-keystone-s2 | look: loses / jump: mixed | look: loses / jump: mixed | 1.356 | 2.000 | **1.404** | **2.090** | 1.718 | 3.850 | 2.819 | 1.390 | 0.692 | `f774bdbcc1cc` | `BenchS2Arc/20261104-095641-03fe99ed/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4` | +| r1-lookhold-s2 | look: loses / jump: mixed | look: loses / jump: mixed | 1.400 | 2.030 | **1.562** | **2.970** | 1.362 | 2.760 | 2.978 | 2.035 | 0.705 | `64dc87a2ed3a` | `BenchS2Arc/20261104-100709-3398e6c0/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4` | +| r1-antiburn-s2 | look: loses / jump: mixed | look: loses / jump: mixed | _1.248_ | _1.690_ | **1.364** | **2.470** | 2.674 | 5.110 | 3.440 | 2.920 | 1.399 | `9124d417577d` | `BenchS2Arc/20261104-101739-05b1c816/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4` | + +### last_frame continuity + +| arm | verdict (all measures) | verdict (picture only) | jump | jump max | look | look max | level dB | level dB max | tex fine | tex mid | tex coarse | job | output | +|---|---|---|---|---|---|---|---|---|---|---|---|---|---| +| floor (seed 1166 vs 2) | | | 0.082 | 0.180 | 0.284 | 0.070 | 3.116 | 6.720 | 0.170 | 0.066 | 0.214 | | | +| ctl-lastframe-s2-s1166 | control | | 1.258 | 1.770 | 1.310 | 1.930 | 5.018 | 9.810 | 3.429 | 3.641 | 3.684 | `b235c35b72c1` | `BenchS2Arc/20261104-080501-f258d5fa/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4` | +| ctl-lastframe-s2-s2 | control | | 1.340 | 1.590 | 1.026 | 1.860 | 1.902 | 3.090 | 3.599 | 3.707 | 3.470 | `765fc66286bd` | `BenchS2Arc/20261104-082803-71d81bfc/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4` | +| r1-lf-carrypick-s2 | look: mixed / jump: loses | look: mixed / jump: loses | **1.362** | 1.850 | _0.968_ | _1.230_ | 2.958 | 4.410 | _2.421_ | _1.144_ | _0.691_ | `9713480482bc` | `BenchLfS2Arc/20261104-103052-7d0e533a/final/BenchLfS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4` | +| r1-lf-lookhold-s2 | look: wins / jump: tie | look: wins / jump: tie | 1.340 | 1.840 | _0.402_ | _0.510_ | 3.784 | 7.330 | _2.406_ | _0.972_ | _0.595_ | `242358078691` | `BenchS2Arc/20261104-103927-af5f9b3e/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4` | + +### Eye notes (S2) + +- **Control, guide.** The arc completes: it goes round to the statue's back + by about 22 s. By the last segment the sky is blowing out to white and the + scene has invented palms and a tower. Seed 2 is the same, with a white-out + sky at 583. +- **anchor 0.25, 0.5, 0.75.** The arc completes. At 583 the sky is + posterised into blotchy cloud bands (worse at 0.5 and 0.75), and the + statue's coat crawls. The low texture at 0.25 is a flat, banded sky, not a + clean one. The look loss at the seams is real. +- **backoff, keystone, look_hold (guide).** The arc completes, with + end-of-take sky banding similar to the control's. No visible improvement. +- **Anti-burn LoRA.** At 583 the sky blows out to a white patch and the + statue smears into a soft blob. Broken. +- **Control, last_frame.** The arc is slow but real: it reaches a side view + by about 26-30 s. Clean sky. +- **last_frame + carry_pick and last_frame + look_hold: frozen motion.** On + the 8-frame contact sheet both stay on the front view for the whole take. + The statue faces the camera at 30 s as it does at 0 s, and the camera never + goes round. Their look and texture wins are the numbers of a camera that + did not move. **Rejected by eye.** One seed-2 run of last_frame + look_hold + is in round 2, to see whether the freeze repeats. + +## S3 dialogue walk + +Primary: seam jump, seam look, and texture. Guide only (the brief gave S3 no +last_frame arms). + +| arm | verdict (all measures) | verdict (picture only) | jump | jump max | look | look max | level dB | level dB max | tex fine | tex mid | tex coarse | job | output | +|---|---|---|---|---|---|---|---|---|---|---|---|---|---| +| floor (seed 1166 vs 2) | | | 0.332 | 0.140 | 0.410 | 0.070 | 0.378 | 2.590 | 0.226 | 0.112 | 0.111 | | | +| ctl-guide-s3-s1166 | control | | 1.524 | 1.830 | 1.470 | 1.710 | 2.968 | 8.810 | 0.067 | 0.114 | 0.188 | `c0b92d0d0cd4` | `BenchS3DialogueWalk/20261104-084517-6db94fb9/final/BenchS3DialogueWalk-chained_keyframe_to_video_audio.0-0.0.mp4` | +| ctl-guide-s3-s2 | control | | 1.192 | 1.690 | 1.060 | 1.640 | 2.590 | 6.220 | 0.294 | 0.226 | 0.299 | `64b4a78c74a2` | `BenchS3DialogueWalk/20261104-090907-773b20e5/final/BenchS3DialogueWalk-chained_keyframe_to_video_audio.0-0.0.mp4` | +| r1-anchor25-s3 | jump: mixed / look: mixed / tex: mixed | jump: mixed / look: mixed / tex: mixed | 1.642 | **2.040** | 1.484 | **1.800** | 3.230 | 8.810 | 0.103 | 0.165 | **0.499** | `7c1aa8f8b38d` | `BenchS3DialogueWalk/20261104-105035-5a02f9f8/final/BenchS3DialogueWalk-chained_keyframe_to_video_audio.0-0.0.mp4` | +| r1-anchor50-s3 | jump: mixed / look: mixed / tex: mixed | jump: mixed / look: mixed / tex: mixed | 1.554 | **2.090** | 1.404 | **2.200** | **3.896** | 8.810 | 0.053 | 0.013 | **0.379** | `fef6b9aea7c9` | `BenchS3DialogueWalk/20261104-110353-90e5c4ef/final/BenchS3DialogueWalk-chained_keyframe_to_video_audio.0-0.0.mp4` | +| r1-anchor75-s3 | jump: mixed / look: loses / tex: mixed | jump: mixed / look: loses / tex: mixed | 1.666 | **2.300** | **1.922** | **2.820** | 3.050 | 8.810 | 0.076 | 0.120 | **0.451** | `fbf286d9f192` | `BenchS3DialogueWalk/20261104-111709-bec10029/final/BenchS3DialogueWalk-chained_keyframe_to_video_audio.0-0.0.mp4` | +| r1-backoff17-s3 | jump: mixed / look: mixed / tex: loses | jump: mixed / look: mixed / tex: loses | 1.496 | **2.040** | 1.124 | 1.690 | **3.646** | 6.470 | **0.337** | **0.472** | _0.036_ | `5873332b54fc` | `BenchS3DialogueWalk/20261104-113024-86593c80/final/BenchS3DialogueWalk-chained_keyframe_to_video_audio.0-0.0.mp4` | +| r1-keystone-s3 | jump: mixed / look: mixed / tex: mixed | jump: mixed / look: mixed / tex: mixed | 1.414 | **2.080** | 1.182 | **2.030** | **3.660** | 7.150 | 0.106 | 0.187 | 0.222 | `a2bd5e2545db` | `BenchS3DialogueWalk/20261104-114056-8aecac36/final/BenchS3DialogueWalk-chained_keyframe_to_video_audio.0-0.0.mp4` | +| r1-lookhold-s3 | jump: tie / look: tie / tex: tie | jump: tie / look: tie / tex: tie | 1.592 | 1.830 | 1.432 | _1.610_ | 3.028 | 8.810 | 0.055 | 0.109 | 0.244 | `a9a134a9acf8` | `BenchS3DialogueWalk/20261104-115128-bf419738/final/BenchS3DialogueWalk-chained_keyframe_to_video_audio.0-0.0.mp4` | +| r1-antiburn-s3 | jump: loses / look: mixed / tex: loses | jump: loses / look: mixed / tex: loses | **2.046** | **2.390** | 1.744 | **2.010** | **4.954** | 6.280 | **0.671** | **0.876** | **0.519** | `c50ec5961418` | `BenchS3DialogueWalk/20261104-120204-7fe540b4/final/BenchS3DialogueWalk-chained_keyframe_to_video_audio.0-0.0.mp4` | + +### Eye notes (S3) + +- **Control (both seeds).** From 277 on, the sky at the top centre of the + street is a flat white blown-out blob. By 583 the face and coat are + posterised and plasticky. Seed 2 has the same blob, and at 583 the face is + burned as well. +- **anchor 0.25 and 0.75.** At 583 they look the same as the control. +- **anchor 0.5.** Slightly softer, with less burn on the face at 583. Inside + the floor on every primary. +- **backoff, keystone, look_hold.** At 583 the burn and the white blob are at + the control's level. Keystone's face is, if anything, more contrasty. +- **Anti-burn LoRA.** Heavy blur from 277 on: the walker is out of focus and + smeared. Broken. + +## Continuity: last_frame against guide + +| scene | verdict (all) | verdict (picture) | worse past floor | better past floor | +|---|---|---|---|---| +| s1-static | tex: mixed | tex: mixed | jump, jump max, tex mid | look, look max, level dB, level dB max | +| s2-arc | look: mixed / jump: mixed | look: mixed / jump: mixed | level dB, tex mid, tex coarse | jump, jump max | + +| arm | verdict (all) | verdict (picture) | worse past floor | better past floor | +|---|---|---|---|---| +| r1-lf-carrypick-s1 | tex: mixed | tex: mixed | jump, jump max, tex mid | look, look max, level dB max, tex coarse | +| r1-lf-lookhold-s1 | tex: mixed | tex: mixed | tex mid | jump, jump max, look, look max, level dB, level dB max, tex coarse | +| r1-lf-carrypick-s2 | look: wins / jump: tie | look: wins / jump: tie | | jump max, look, look max, tex fine | +| r1-lf-lookhold-s2 | look: wins / jump: tie | look: wins / jump: tie | | jump max, look, look max, tex fine, tex mid | + +**Controls.** As controls, the two continuities are mixed on both scenes: + +- S1: last_frame has the better seam look and the worse seam jump, and burns + a little more by eye. +- S2: last_frame has the better jump, the worse mid and coarse texture, and a + slower arc. + +**last_frame + look_hold.** + +- S1: against the guide control it ties on texture (inside the floor) and is + better on every seam measure. By eye it is about as burned. +- S2: its numbers beat the guide control, but the camera froze. + +**Verdict: no default change from round 1.** last_frame is not shown better +than guide as the default. The S1 last_frame + look_hold result goes to +round 2 for confirmation. + +## Round-1 winners + +**Strict winners (verdict `wins`, confirmed by eye): none on any scene.** + +- **S2.** The only `wins` the rule returned is `r1-lf-lookhold-s2`, and its + arc is frozen. +- **S1.** `r1-lf-lookhold-s1` and `r1-lf-carrypick-s1` win on picture only. + The carry pick wins on the floor exactly, and the eye sees no change. + +The design asks for winners confirmed on a second seed. Round 1 is one seed +against a two-seed floor, so the candidates below go to round 2 as +**provisional winners**: arms whose primary beats the control past the floor, +whose eye check agrees, and whose only losses are on secondary measures. + +| scene | provisional winners for round 2 | why carried | why not a strict win | +|---|---|---|---| +| S1 | `anchor_strength 0.25` (guide) | texture 0.367 vs 0.536 (floor 0.081); look better; the eye sees less burn | seam jump 0.928 vs 0.854 (floor 0.058); level dB | +| S1 | `anchor_strength 0.5` (guide) | texture 0.270; look better; the cleanest by eye | seam jump 1.048 (worse past the floor); coarse texture; level dB | +| S1 | `keystone segment_1` (guide) | texture 0.373; look max better; a modest gain by eye | jump max 1.04 vs 0.97 (floor 0.05); level dB | +| S1 | `look_hold` on last_frame | wins every picture measure against the last_frame control; ties texture and beats every seam against guide | level dB mean 4.32 vs 3.37 (floor 0.70); the eye sees the same burn | +| S2 | none | every guide counter loses seam look; both last_frame counters freeze the arc | | +| S3 | none | every primary is inside S3's wide floor (jump 0.33, look 0.41, texture 0.23); no visible gain | | + +## Status per idea (round 1) + +- **#1190 anchor strength:** mixed. It helps S1 (texture, and the eye sees + less burn at 0.25 and 0.5; 0.75 drifts identity). It loses S2 (seam look) + and is a tie on S3. +- **#1189 hand-over backoff (17):** loses everywhere. S1 texture is inside + the floor, S2 seam look loses, and S3 texture loses. +- **#1187 keystone:** mixed. S1 texture wins with a small jump max cost. It + loses S2 look and ties S3. +- **#1032 sharpest of last 4:** loses everywhere. On S1 it ties exactly on the + floor and the eye sees no change. On S2 it freezes the arc. +- **#1191 look_hold:** mixed. + - Guide: S1 look better with texture tied; S2 loses; S3 ties. + - last_frame: S1 wins on picture, with burn unchanged by eye; S2 freezes + the arc. +- **Anti-burn LoRA (`Imagineer13/H3-Anti-Burn-LoRA`, 0.6, every segment):** + loses everywhere. Its low texture numbers are blur: the face smears on S1, + the sky blows out and the statue smears on S2, and S3 is out of focus. +- **last_frame against guide as the default continuity:** mixed. The + controls are mixed on both scenes. last_frame + look_hold beats guide on + S1 seams but froze S2. Taste: needs Don if seed 2 confirms S1 + (`for-don.md`). + +## Round 2 + +There are three arm files, in priority order. All are S1, since S2 and S3 +have no candidate. Every arm differs from its control only in the counters +named in `tests/test_bench_workflows.py` (`ROUND2`, which checks the +effective variables). + +1. **`bench/arms/counters-round2-confirm.json`** (6 runs): + - `ctl-guide-s1-s1166-rerun`: the round-1 guide control again, past the + step cache (`-r2` id). The workflows gained a null `duration_seconds` + knob, so this checks that the round-1 controls still stand bit for bit. + - anchor 0.25, anchor 0.5, keystone and last_frame + look_hold on S1 at + seed 2, read against `ctl-*-s1-s2`. + - last_frame + look_hold on S2 at seed 2: a diagnostic for the frozen + arc, not a candidate. +2. **`bench/arms/counters-round2-combos.json`** (8 runs): the pairwise + combinations of the S1 candidates that can run together, at seeds 1166 + and 2. Running both seeds now means a combination can be confirmed + without a round 3. + - anchor 0.25 + keystone and anchor 0.5 + keystone (guide). + - last_frame + look_hold + anchor 0.25, and last_frame + look_hold + + anchor 0.5. Anchor accepts last_frame. + - Keystone needs guide and look_hold's keyframe parts need last_frame, so + keystone + look_hold is a different continuity from both controls and + is not run. +3. **`bench/arms/counters-round2-long.json`** (2 runs): two minutes + (`duration_seconds: 120`, 29 segments, 2880 frames, moments + 62/1501/2868) for the guide control and for anchor 0.5, the best single + candidate on S1's primary and by eye. + - The bench workflows gained `duration_seconds` as a variable, null by + default, so a chain keeps `segments`. + - The 2-minute arms are read against each other. A 6-segment control + cannot judge a 29-segment take. + +## Round 2 results + +### The rerun did not reproduce, so the guide floor is wider + +`ctl-guide-s1-s1166-rerun` (job `fd34bc937628`) is the round-1 guide control +again, at the same seed, past the step cache. It is **not** bit-identical to +`ctl-guide-s1-s1166` (`b40e855eb553`): seam look 1.656 against 1.464, level +step 11.62 against 7.65 dB, fine texture 0.520 against 0.536, jump max 0.96 +against 0.97. Seam jump mean is identical (0.854). + +- It is the render, not the measure. Measuring `b40e855eb553` again today + gives its round-1 numbers to the third decimal. +- By eye the two runs look the same at 62/277/583. +- The two-minute control (`8d1614ced27a`, same seed) does reproduce round 1 + over its first six segments: texture 0.536 at frame 583, seams 1-5 jump + 0.85, look 1.46, level 7.7 dB. So only the `-r2` rerun drifted. Its cause + is not known (the distinct workflow id, the null `lora` entry or the card + are candidates). + +The floor no longer has a 0 for same-seed reruns, so from here on: + +- **Guide S1 floor** = the largest pairwise difference among the three guide + controls (1166, its rerun, seed 2): jump 0.058, jump max 0.050, **look + 0.312**, look max 0.140, **level dB 6.394**, level dB max 3.650, tex fine + 0.081, tex mid 0.038, tex coarse 0.009. +- **last_frame S1 floor** = the larger of its two-seed floor and the guide + rerun noise: jump 0.022, jump max 0.010, look 0.192, look max 0.050, level + dB 3.962, level dB max 3.650, tex fine 0.036, tex mid 0.087, tex coarse + 0.009. +- **Across continuities** (an lf arm read against a guide control, or a guide + arm): the larger of the two. + +Two round-1 readings change under this floor. guide `look_hold`'s seam look +gain on S1 (1.31 against 1.46) is now inside the floor: a tie. And the S1 +level step (mean) is inside the floor for every S1 arm in round 1 and round 2, +which answers the level-step question below. + +### Seed-2 confirmation + +Cells: **bold** is worse than the control past the floor, _italic_ is better +past it. Primary is fine texture at the last moment. + +| arm | against | verdict (all) | verdict (picture) | jump | jump max | look | look max | level dB | level dB max | tex fine | tex mid | tex coarse | job | +|---|---|---|---|---|---|---|---|---|---|---|---|---|---| +| r1-anchor25-s1 | ctl-guide-s1-s1166 | mixed | mixed | **0.928** | **1.150** | _1.128_ | _1.980_ | 12.160 | **23.540** | _0.367_ | _0.124_ | 0.015 | `0a40c41af97c` | +| r2-anchor25-s1-seed2 | ctl-guide-s1-s2 | mixed | mixed | 0.812 | 0.950 | 1.092 | _1.880_ | 10.426 | 18.240 | _0.325_ | 0.130 | **0.031** | `93dd359dc11e` | +| r1-anchor50-s1 | ctl-guide-s1-s1166 | mixed | mixed | **1.048** | **1.320** | _1.050_ | _1.710_ | 13.224 | **20.340** | _0.270_ | _0.116_ | **0.028** | `fa17a2c51e33` | +| r2-anchor50-s1-seed2 | ctl-guide-s1-s2 | mixed | mixed | **0.904** | **1.060** | 1.212 | _1.720_ | 6.550 | 16.230 | _0.272_ | 0.121 | **0.030** | `075ed6de83f7` | +| r1-keystone-s1 | ctl-guide-s1-s1166 | mixed | mixed | 0.850 | **1.040** | 1.550 | _1.930_ | **14.556** | **23.260** | _0.373_ | _0.116_ | 0.020 | `1e1b6116ce26` | +| r2-keystone-s1-seed2 | ctl-guide-s1-s2 | mixed | mixed | 0.800 | 0.900 | 1.536 | **2.710** | 10.352 | 20.130 | 0.448 | 0.123 | 0.010 | `ab59d503fd4d` | +| r1-lf-lookhold-s1 | ctl-lastframe-s1-s1166 | wins | wins | _0.788_ | _0.890_ | _0.634_ | _0.970_ | 4.324 | 6.780 | _0.491_ | 0.304 | 0.005 | `be3f574df3db` | +| r2-lf-lookhold-s1-seed2 | ctl-lastframe-s1-s2 | wins | wins | _0.806_ | _1.020_ | _0.838_ | _1.060_ | 3.664 | 8.880 | _0.550_ | 0.411 | 0.001 | `b7ca6da7d075` | +| r1-lf-lookhold-s1 | ctl-guide-s1-s1166 | mixed | mixed | _0.788_ | _0.890_ | _0.634_ | _0.970_ | 4.324 | _6.780_ | 0.491 | **0.304** | _0.005_ | `be3f574df3db` | +| r2-lf-lookhold-s1-seed2 | ctl-guide-s1-s2 | loses | loses | 0.806 | **1.020** | _0.838_ | _1.060_ | 3.664 | _8.880_ | **0.550** | **0.411** | 0.001 | `b7ca6da7d075` | + +Per provisional winner: + +- **anchor 0.25: holds.** Texture beats the control past the floor on both + seeds (0.367 vs 0.536, 0.325 vs 0.455). The secondary costs do not repeat: + the jump cost is seed 1166 only (seed 2: 0.812 vs 0.796, inside), and the + coarse-texture cost is seed 2 only (0.031 vs 0.008; on 1166 it is 0.015 vs + 0.016). No measure is worse past the floor on both seeds. +- **anchor 0.5: holds on texture, and so does its jump cost.** Texture 0.272 + vs 0.455 on seed 2, but jump is again worse past the floor (0.904 vs 0.796, + after 1.048 vs 0.854), and so are jump max and coarse texture, on both + seeds. +- **keystone: does not hold.** Seed-2 texture is 0.448 against 0.455, inside + the floor, and look max is worse (2.71 vs 2.27). **Not a winner.** +- **last_frame + look_hold: holds against its own control, not against + guide.** It is a strict `wins` against the last_frame control on both seeds + (every measure, soundtrack included, under the widened floor). Against the + guide control it loses on seed 2: fine texture 0.550 vs 0.455 and mid + texture 0.411 vs 0.144, both past the floor. +- **S2 last_frame + look_hold, seed 2 (`a3d6a044fd6b`): the camera is held + back again.** It "wins" seam look against the last_frame control (0.386 vs + 1.026) and texture collapses to 0.83 (control 3.60). See the eye notes. + +### Combinations + +Each combination is read against the guide control, against the best single +counter of its own anchor strength, and (for the last_frame sets) against +last_frame + look_hold alone. + +| arm | against | verdict (all) | verdict (picture) | jump | jump max | look | look max | level dB | level dB max | tex fine | tex mid | tex coarse | job | +|---|---|---|---|---|---|---|---|---|---|---|---|---|---| +| r2-anchor25-keystone-s1-s1166 | ctl-guide-s1-s1166 | mixed | mixed | 0.898 | **1.160** | _1.092_ | _1.610_ | 10.750 | 15.170 | _0.309_ | _0.118_ | **0.031** | `8accd0c37f91` | +| r2-anchor25-keystone-s1-s1166 | r1-anchor25-s1 | mixed | mixed | 0.898 | 1.160 | 1.092 | _1.610_ | 10.750 | _15.170_ | 0.309 | 0.118 | **0.031** | `8accd0c37f91` | +| r2-anchor25-keystone-s1-s2 | ctl-guide-s1-s2 | mixed | mixed | 0.832 | **1.020** | 1.186 | _1.790_ | **14.018** | **27.160** | _0.311_ | 0.113 | **0.031** | `e34a5aebaab4` | +| r2-anchor25-keystone-s1-s2 | r2-anchor25-s1-seed2 | mixed | mixed | 0.832 | **1.020** | 1.186 | 1.790 | 14.018 | **27.160** | 0.311 | 0.113 | 0.031 | `e34a5aebaab4` | +| r2-anchor50-keystone-s1-s1166 | ctl-guide-s1-s1166 | mixed | mixed | **1.296** | **1.740** | **2.038** | **2.890** | 5.590 | 14.460 | _0.293_ | _0.128_ | **0.039** | `e5c7b013538f` | +| r2-anchor50-keystone-s1-s1166 | r1-anchor50-s1 | mixed | mixed | **1.296** | **1.740** | **2.038** | **2.890** | _5.590_ | _14.460_ | 0.293 | 0.128 | **0.039** | `e5c7b013538f` | +| r2-anchor50-keystone-s1-s2 | ctl-guide-s1-s2 | mixed | mixed | **1.068** | **1.590** | **1.672** | **2.710** | 10.810 | **24.100** | _0.246_ | 0.107 | **0.041** | `08e32162985c` | +| r2-anchor50-keystone-s1-s2 | r2-anchor50-s1-seed2 | mixed | mixed | **1.068** | **1.590** | **1.672** | **2.710** | 10.810 | **24.100** | 0.246 | 0.107 | **0.041** | `08e32162985c` | +| r2-lf-lookhold-anchor25-s1-s1166 | ctl-lastframe-s1-s1166 | wins | wins | _0.786_ | _0.960_ | _0.626_ | _0.850_ | 4.090 | 9.440 | _0.331_ | _0.208_ | 0.010 | `835379468ec3` | +| r2-lf-lookhold-anchor25-s1-s1166 | ctl-guide-s1-s1166 | wins | wins | _0.786_ | 0.960 | _0.626_ | _0.850_ | 4.090 | _9.440_ | _0.331_ | 0.208 | 0.010 | `835379468ec3` | +| r2-lf-lookhold-anchor25-s1-s1166 | r1-anchor25-s1 | tie | tie | _0.786_ | _0.960_ | _0.626_ | _0.850_ | _4.090_ | _9.440_ | 0.331 | 0.208 | 0.010 | `835379468ec3` | +| r2-lf-lookhold-anchor25-s1-s1166 | r1-lf-lookhold-s1 | mixed | mixed | 0.786 | **0.960** | 0.626 | _0.850_ | 4.090 | 9.440 | _0.331_ | _0.208_ | 0.010 | `835379468ec3` | +| r2-lf-lookhold-anchor25-s1-s2 | ctl-lastframe-s1-s2 | wins | wins | _0.740_ | _0.830_ | _0.760_ | _0.900_ | 4.086 | 8.180 | _0.297_ | _0.234_ | 0.015 | `41355297a083` | +| r2-lf-lookhold-anchor25-s1-s2 | ctl-guide-s1-s2 | mixed | mixed | 0.740 | _0.830_ | _0.760_ | _0.900_ | 4.086 | _8.180_ | _0.297_ | **0.234** | 0.015 | `41355297a083` | +| r2-lf-lookhold-anchor25-s1-s2 | r2-anchor25-s1-seed2 | mixed | mixed | _0.740_ | _0.830_ | _0.760_ | _0.900_ | 4.086 | _8.180_ | 0.297 | **0.234** | _0.015_ | `41355297a083` | +| r2-lf-lookhold-anchor25-s1-s2 | r2-lf-lookhold-s1-seed2 | mixed | mixed | _0.740_ | _0.830_ | 0.760 | _0.900_ | 4.086 | 8.180 | _0.297_ | _0.234_ | **0.015** | `41355297a083` | +| r2-lf-lookhold-anchor50-s1-s1166 | ctl-lastframe-s1-s1166 | wins | wins | _0.752_ | _0.930_ | _0.766_ | _1.190_ | 5.018 | 7.770 | _0.183_ | _0.178_ | 0.011 | `d8b17df7dfb9` | +| r2-lf-lookhold-anchor50-s1-s1166 | ctl-guide-s1-s1166 | wins | wins | _0.752_ | 0.930 | _0.766_ | _1.190_ | 5.018 | _7.770_ | _0.183_ | 0.178 | 0.011 | `d8b17df7dfb9` | +| r2-lf-lookhold-anchor50-s1-s1166 | r1-anchor50-s1 | wins | wins | _0.752_ | _0.930_ | 0.766 | _1.190_ | _5.018_ | _7.770_ | _0.183_ | 0.178 | _0.011_ | `d8b17df7dfb9` | +| r2-lf-lookhold-anchor50-s1-s1166 | r1-lf-lookhold-s1 | mixed | mixed | _0.752_ | **0.930** | 0.766 | **1.190** | 5.018 | 7.770 | _0.183_ | _0.178_ | 0.011 | `d8b17df7dfb9` | +| r2-lf-lookhold-anchor50-s1-s2 | ctl-lastframe-s1-s2 | mixed | mixed | _0.774_ | _0.880_ | _0.886_ | _1.250_ | 2.460 | 4.340 | _0.236_ | _0.245_ | **0.040** | `c8bfadf286ce` | +| r2-lf-lookhold-anchor50-s1-s2 | ctl-guide-s1-s2 | mixed | mixed | 0.774 | 0.880 | _0.886_ | _1.250_ | 2.460 | _4.340_ | _0.236_ | **0.245** | **0.040** | `c8bfadf286ce` | +| r2-lf-lookhold-anchor50-s1-s2 | r2-anchor50-s1-seed2 | mixed | mixed | _0.774_ | _0.880_ | _0.886_ | _1.250_ | 2.460 | _4.340_ | 0.236 | **0.245** | **0.040** | `c8bfadf286ce` | +| r2-lf-lookhold-anchor50-s1-s2 | r2-lf-lookhold-s1-seed2 | mixed | mixed | _0.774_ | _0.880_ | 0.886 | **1.250** | 2.460 | _4.340_ | _0.236_ | _0.245_ | **0.040** | `c8bfadf286ce` | + +What the combinations show: + +- **Keystone adds nothing to anchor 0.25, and hurts anchor 0.5.** With anchor + 0.25, texture is inside the floor of anchor 0.25 alone on both seeds (0.309 + vs 0.367, 0.311 vs 0.325), and jump max and coarse texture get worse. With + anchor 0.5, every seam measure is worse than anchor 0.5 alone on both seeds + (jump 1.296 vs 1.048, 1.068 vs 0.904; look 2.04 vs 1.05, 1.67 vs 1.21). +- **last_frame + look_hold + anchor is the best-measuring set on S1.** It + beats the guide control on texture past the floor on all four runs, and on + seam look on all four. Against anchor alone (guide) its seams are better on + both seeds, and its texture is inside the floor (0.331 vs 0.367, 0.297 vs + 0.325 at 0.25; 0.183 vs 0.270, past the floor, and 0.236 vs 0.272 at 0.5). + Its one repeated cost against guide is mid-band texture (0.208/0.234 at + 0.25, 0.178/0.245 at 0.5, against the guide controls' 0.176/0.144); every + last_frame run carries a higher mid band (the bare last_frame controls are + at 0.37/0.45). +- **The anchor improves last_frame + look_hold.** Against look_hold alone, + both anchor strengths cut the texture past the floor on both seeds (to + 0.18-0.33 from 0.49-0.55), and by eye they take the burn off the face that + look_hold alone leaves (below). +- **Anchor strength under last_frame:** 0.5 measures lower on texture at seed + 1166 (0.183 vs 0.331) and about the same at seed 2 (0.236 vs 0.297, inside + the floor); 0.25 has the better seam look on both seeds (0.626/0.760 vs + 0.766/0.886). + +### Two minutes: the anchor's gain holds on the numbers and fails by eye + +`ctl-guide-s1-120s` (`8d1614ced27a`) against `r2-anchor50-s1-120s` +(`107859a31bc1`): 29 segments, 2880 frames, seed 1166, both guide. There is no +two-minute floor; the 6-segment guide floor is the only scale. + +Texture drift on the S1 crop against frame 62, from the measure-only pass +(`counters-round2-long-moments.json`: `POST /api/gallery/compare` with +`texture_bands`, crop `[0,0,240,150]`, the same route and crop as the +driver, at eight moments; the driver's own measure compares the first and +last moment only): + +| frame (s) | 300 (12.5) | 583 (24.3) | 1000 (41.7) | 1501 (62.5) | 2000 (83.3) | 2500 (104.2) | 2868 (119.5) | +|---|---|---|---|---|---|---|---| +| control fine | 0.180 | 0.536 | 1.108 | 1.399 | 1.646 | 1.658 | 1.706 | +| anchor 0.5 fine | 0.137 | 0.270 | 0.360 | 0.602 | 0.737 | 0.759 | 1.067 | +| control mid | 0.045 | 0.176 | 0.406 | 0.716 | 1.071 | 1.156 | 1.171 | +| anchor 0.5 mid | 0.038 | 0.116 | 0.162 | 0.227 | 0.307 | 0.273 | 0.462 | + +Seams, by stretch of the take (`analyze_seams`, every seam): + +| seams | control jump (max) | anchor jump (max) | control look (max) | anchor look (max) | +|---|---|---|---|---| +| 1-5 | 0.85 (0.97) | 1.05 (1.32) | 1.46 (2.13) | 1.05 (1.71) | +| 6-14 | 1.18 (1.64) | 0.97 (1.28) | 2.03 (3.26) | 1.19 (2.36) | +| 15-23 | 1.25 (1.94) | 0.72 (0.92) | 1.67 (3.75) | 1.27 (1.66) | +| 24-28 | 2.28 (**6.56**) | 1.13 (1.46) | 4.38 (**11.51**) | 1.51 (2.65) | +| all 28 | 1.34 (6.56) | 0.93 (1.46) | 2.23 (11.51) | 1.25 (2.65) | + +On the numbers the anchor's gain **grows** with length: + +- Its texture drift is half the control's at 24 s and a third at 42 s, and + stays 0.6-0.9 lower to the end. +- The seam jump cost seen at 6 segments reverses after seam 5. The control's + seams get worse with length, and at seam 26 it has a 6.56 jump and 11.5 + look, past the chain check's 4.0 ceiling. The anchor's worst seam is 1.46. + +**By eye both takes are broken well before two minutes, in opposite ways** +(face crops at 583/800/1000/1250/1501, full frames at 62/1000/1501/2000/2868): + +- **Control:** the burn of round 1 keeps building. At 33 s the face is + cracking into crust. By 42 s it is a posterised, cut-out face. From 62 s + the whole frame is a mosaic of crusted shards. She stays recognisably the + same person, harsh and mottled. Seam 26 is a visible reset: the mosaic + jumps to a lighter, cleaner backdrop for a segment. +- **Anchor 0.5:** at 24 s it is the cleaner take. At 33 s the face starts to + soften, and by 42 s it is smooth and waxy. At 52 s the features are + dissolving and the eyes have gone. By 62 s the face is a blurred ghost, and + at 83-120 s it is a white mask with dark holes, over a soft, blocky + backdrop. **Identity is lost.** Its low texture number after about 40 s is + blur, the #1175 trap again, as with the anti-burn LoRA. + +So the anchor's gain is real up to about 6-8 segments (25-33 s) and turns +into a different failure past that. **No counter tested makes a two-minute +static take usable.** This is the gap section 5 of the design names as the +gate for the second group (carry without the pixel round trip: #1150, #1044, +#1030) and for the per-segment check and re-roll. Until then a static shot +longer than about 30 s should be cut, not chained. + +### Eye notes (round 2) + +Frames at each arm's moments (62/277/583 guide, 62/307/673 last_frame), plus +crops of the top-left quarter (`crop=0,0,480,272`) at the last moment to see +the circle and stripes at full resolution. + +- **Control rerun:** indistinguishable by eye from round 1's control at all + three moments. +- **Seed-2 control:** the same burn as seed 1166 at 583: a crusted circle and + mottled stripes. +- **anchor 0.25, seed 2:** stripes visibly cleaner and the circle's edge + cleaner than the control's. Identity holds. Agrees with the number. +- **anchor 0.5, seed 2:** about as clean as 0.25 by eye; the circle keeps + some crust. Identity holds at 24 s. +- **keystone, seed 2:** burns like the control. Agrees with the tie. +- **anchor + keystone (both strengths, both seeds):** about as clean as + anchor alone. Keystone adds no visible gain, and identity holds. +- **last_frame + look_hold, seed 2:** still burns at 673 (crusted circle, + heavy stripes), and the **face is blotched and burned**. As in round 1, its + win is in the seams, not the burn. +- **last_frame + look_hold + anchor 0.25 (both seeds):** the stripes are + cleaner than last_frame + look_hold's and the face is clean. The circle + keeps some crust. +- **last_frame + look_hold + anchor 0.5 (both seeds):** the cleanest S1 frames + in the bench at the last moment, at 673, 90 frames later than the guide + arms' 583: the stripes are nearly flat paint and the face is clean. The + circle still has crust, and the wall at the left is mottled at seed 1166. + Identity holds. +- **S2 last_frame + look_hold, seed 2 (8-frame contact sheet, + `count=8`):** not a full freeze this time, but the camera is held back. It + stays on the front view to 13 s and reaches about a three-quarter view by + 30 s. The plain last_frame control at seed 2 goes round to the statue's + back by 26-30 s. Round 1's seed-1166 run never left the front. **The + motion loss repeats on both seeds**, so the "win" on seam look is still a + camera that moved less. + +### The two open questions + +**Anchor 0.5 looks better but measures worse on seam jump.** Seed 2 repeats +the jump cost (0.904 vs 0.796, floor 0.058), and its texture lead over 0.25 +is inside the floor on seed 2 (0.272 vs 0.325), where by eye the two look +alike. At two minutes 0.5 dissolves the face from about 50 s. **Firm +recommendation: anchor 0.25**, not 0.5, as the static default. It keeps the +texture gain on both seeds, its jump cost does not repeat, and it is the +lower identity risk (0.75 drifted identity in round 1; 0.5 loses it at +length). 0.25 has not been run at two minutes. + +**Should the level step count on ambience-only scenes?** The same-seed rerun +moved S1's level step from 7.65 to 11.62 dB, and the spread over the three +guide controls is 6.4 dB. Under that floor no S1 arm's level-step mean is +worse past it. Verdicts with and without it are now the same in every round-2 +row. **Firm recommendation: do not +count `seam_level_step_db_*` on scenes without a held voice or music bed +(S1, S2). Read them on picture.** Count it on S3, S4 and S6. + +### Verdict per idea + +| idea | verdict | numbers | +|---|---|---| +| #1190 anchor strength | **default for static shots, 0.25** (guide, `anchor: segment_1`, `anchor_strength: 0.25`), chains up to about 6-8 segments. Not for moving camera or dialogue walk. | S1 texture 0.367 vs 0.536 (`0a40c41af97c`) and 0.325 vs 0.455 (`93dd359dc11e`), floor 0.081. S2 look loses at every strength (2.03-3.50 vs 1.29, floor 0.07). S3 inside the floor. 0.5 dissolves the face at length (`107859a31bc1`). | +| #1189 hand-over backoff | **cut** | S1 texture 0.462 vs 0.536 (inside 0.081); S2 look 1.43 vs 1.29 (floor 0.07, loses); S3 texture 0.337 vs 0.067 (floor 0.23, loses). Round 1 only; nothing to carry. | +| #1187 keystone as keyframe | **cut** as a short-take counter. *(Superseded 2026-11-06: at two minutes the keystone at slot `last` is the static long-take default, `shot: static_long` on `templates/minimax/chained-segments`, Don's "not a loop", 43a0fb87/d555a373; see `structural.md` *Keystone slot*.)* | Seed 2 texture 0.448 vs 0.455 (tie, `ab59d503fd4d`); with anchor 0.25 no gain over anchor alone (0.311 vs 0.325); with anchor 0.5 every seam worse (jump 1.30/1.07 vs 1.05/0.90, `e5c7b013538f`, `08e32162985c`); S2 look loses (1.40 vs 1.29). | +| #1032 sharpest of last 4 (`carry_pick`) | **cut** | S1 texture 0.533 vs 0.569 on a 0.036 floor (no visible change, `ae932366ea0e`); S2 freezes the arc (`9713480482bc`). | +| #1191 `look_hold` on guide (creep + texture parts) | **cut** | S1 look 1.31 vs 1.46 now inside the 0.31 floor, texture tied; S2 look loses (1.56 vs 1.29); S3 tie. | +| #1191 `look_hold` on last_frame (all four parts) | **available knob, no default** | Wins against the last_frame control on both seeds (`be3f574df3db`, `b7ca6da7d075`) but loses texture to guide on seed 2 (0.550 vs 0.455) and holds the camera back on S2 on both seeds (`242358078691`, `a3d6a044fd6b`). Its value is as part of the static set below (Don's call). | +| #1033 / #1067 seam grade | **available knob, no default** (with #1191 on last_frame) | The seam grade is look_hold's `pickup` part, which only runs on last_frame. It was benched inside the last_frame look_hold arms, not alone; its numbers are that row's. | +| Anti-burn LoRA (0.6, every segment) | **cut** | Low texture is blur: S1 face dissolves by 277 (`049d652e1648`); S2 statue smears (`9124d417577d`); S3 jump 2.05 vs 1.52 and texture 0.671 vs 0.067 lose (`c50ec5961418`). | +| Continuity: guide | **default for every shot type** | last_frame alone is mixed against guide on S1 and S2 (round 1). | +| Continuity: last_frame | **available knob, no default**; the static alternative (last_frame + look_hold + anchor) is **Don's call** | Best-measuring S1 set (`835379468ec3`, `41355297a083`, `d8b17df7dfb9`, `c8bfadf286ce`), but mid texture worse than guide on both seeds, untested at length, and look_hold holds a moving camera back. | + +## Defaults by shot type + +Tasks 15 and 20 build the template defaults from this table. "Build as" is +the exact chain setting. A row marked **Don's call** is built as its "build +as" (the recommendation) unless Don overrules it in `for-don.md`. + +| shot type | build as | evidence (jobs) | status | +|---|---|---|---| +| **static** (locked-off camera, subject in front; S1), up to about 30 s (6-8 H3 segments) | `continuity: guide`, `anchor: segment_1`, `anchor_strength: 0.25`. Nothing else. | anchor 0.25 at seed 1166 `0a40c41af97c` and seed 2 `93dd359dc11e`, against guide controls `b40e855eb553`, `fd34bc937628`, `894911aed6c7` | **Don's call, recommended.** The texture gain holds on two seeds and by eye, but the strict rule says `mixed`: a secondary measure is worse past the floor on each seed (jump on 1166, coarse texture on seed 2), though never the same one twice. | +| static, alternative set | `continuity: last_frame`, `look_hold: true`, `anchor: segment_1`, `anchor_strength: 0.25`. Offered as a knob, not the default. | `835379468ec3`, `41355297a083` (0.25); `d8b17df7dfb9`, `c8bfadf286ce` (0.5); look_hold alone `be3f574df3db`, `b7ca6da7d075` | **Don's call, recommended: not the default.** It measures best on S1 and looks cleanest, but it changes the continuity, carries a mid-texture cost on both seeds, has not run at two minutes, and its look_hold holds a moving camera back. | +| static, longer than about 30 s | Cut it into shots of about 30 s or less. The chain gets no counter beyond the static row. | two-minute takes `8d1614ced27a` (control), `107859a31bc1` (anchor 0.5) | **Settled:** no counter set tested gives a usable two-minute static take (the control burns, anchor 0.5 dissolves the face). This is the drift gap that opens the design's second group. *(Superseded 2026-11-06: a static take past about 30 s is built as `shot: static_long` (guide, keystone `segment_1` at slot `last`, anchor off), which holds 120 s on two seeds; `structural.md` *Keystone slot*.)* | +| **moving camera** (arc, dolly; S2) | `continuity: guide`, no counters. Never `last_frame` with `look_hold` or `carry_pick`. | guide controls `dedd341c9108`, `01146e925c94`; every round-1 S2 arm; look_hold motion loss `242358078691`, `a3d6a044fd6b` | **Settled.** Every guide counter loses seam look; both last_frame counters hold the camera back, on both seeds for look_hold. | +| **dialogue walk** (speaking subject in motion; S3) | `continuity: guide`, no counters. | guide controls `c0b92d0d0cd4`, `64b4a78c74a2`; every round-1 S3 arm | **Settled.** Every counter is inside S3's wide floor or loses, and none looks better. | + +## Re-scored under R35 (2026-11-05) + +Everything above is kept as it was written. This section re-reads every +round-1 and round-2 verdict under ruling R35, which supersedes R26: + +- **The floor.** Same-seed reruns are not noise-free. The floor is + `combined_floor(seed floor, rerun_gap(pairs))` (`bench/verdict.py`): the + larger, per measure, of the two-seed floor above and the same-seed rerun + gap of the 8 round-2 combination pairs that two drivers both ran. A read + across continuities takes the larger of the two continuities' floors. For + the rerun gap and the floors per scene, see `noise-floor.md`, *Correction + (2026-11-05, ruling R35)*. On every scene the seam floors are now jump + 0.124 (S3 0.332), look 0.438, look max 0.790, and level step 6.2 dB mean + and 8.8 dB max. The S1 guide fine-texture floor stays 0.081. +- **The labels.** `bench/segment1.py` hashes segment 1 of each record's + output (decoded video frames 0-59 and AAC frames 0-129) and labels it + `clean` (the canonical fingerprint for its scene and seed) or `drifted`. + The canonical entries are in `bench/records/segment1-canonical.json`, + taken from cold-loaded jobs where one exists (S1 and S2 at seed 1166, S1 + at seed 2), and otherwise from the agreeing majority. The fingerprints + are in `bench/records/segment1-fingerprints.json`. They were taken on lem, + read-only, with lem's ffmpeg 7.0.2. The video halves match the + nondeterminism report's (`d59a1259`, `ed5094c1`). The audio halves differ + from the report's because a Mac ffmpeg decodes AAC differently. +- **Confounded.** A row is confounded when its two sides open on different + segment-1 outputs: one drifted and one clean, or two different + alternates. Under R35 a confounded verdict is re-scored, and re-run only + if the re-score flips it. +- **Marks.** + - *unchanged*: the same verdict in both columns, with every measure on + the same side of the floor. + - *weakened*: the same verdict, but some measure that was past the floor + is now inside it. + - **flipped**: the verdict changes in either column (all measures, or + picture only). + + A flip of a row is not always a flip of the call that row fed. The calls + are under [Calls under R35](#calls-under-r35). + +### Segment-1 labels (rounds 1 and 2) + +`batch 1` is the first, overwritten driver run of the round-2 combinations +(`counters-round2-combos-batch1.json`), measured again for the rerun gap. + +| arm | job | key | segment 1 | label | +|---|---|---|---|---| +| ctl-guide-s1-s1166 | `b40e855eb553` | s1-static@1166 | `d59a1259/b2c2c63e` | clean | +| ctl-guide-s1-s2 | `894911aed6c7` | s1-static@2 | `f47df963/f88334a9` | clean | +| ctl-lastframe-s1-s1166 | `3a73cb0f5d28` | s1-static@1166 | `56e0831e/ac39b399` | drifted | +| ctl-lastframe-s1-s2 | `f053abe1ce5f` | s1-static@2 | `4302cf65/03cb1fa0` | drifted | +| ctl-guide-s2-s1166 | `dedd341c9108` | s2-arc@1166 | `ed5094c1/0a819c94` | clean | +| ctl-guide-s2-s2 | `01146e925c94` | s2-arc@2 | `14a80618/b4d1d486` | clean | +| ctl-lastframe-s2-s1166 | `b235c35b72c1` | s2-arc@1166 | `57cbd211/a1d44644` | drifted | +| ctl-lastframe-s2-s2 | `765fc66286bd` | s2-arc@2 | `130afe0c/f90613fc` | drifted | +| ctl-guide-s3-s1166 | `c0b92d0d0cd4` | s3-dialogue-walk@1166 | `289012f1/d21b04b0` | clean | +| ctl-guide-s3-s2 | `64b4a78c74a2` | s3-dialogue-walk@2 | `31765efa/48363338` | clean | +| r1-anchor25-s1 | `0a40c41af97c` | s1-static@1166 | `d59a1259/b2c2c63e` | clean | +| r1-anchor50-s1 | `fa17a2c51e33` | s1-static@1166 | `d59a1259/b2c2c63e` | clean | +| r1-anchor75-s1 | `f17bce198a2d` | s1-static@1166 | `56e0831e/ac39b399` | drifted | +| r1-backoff17-s1 | `c594521e59d1` | s1-static@1166 | `fe993ff4/fb5cdcbf` | drifted | +| r1-keystone-s1 | `1e1b6116ce26` | s1-static@1166 | `fe993ff4/fb5cdcbf` | drifted | +| r1-lookhold-s1 | `751b43732fb9` | s1-static@1166 | `fe993ff4/fb5cdcbf` | drifted | +| r1-antiburn-s1 | `049d652e1648` | s1-static@1166+lora | `a76a366c/ff5c4416` | clean | +| r1-lf-carrypick-s1 | `ae932366ea0e` | s1-static@1166 | `d59a1259/b2c2c63e` | clean | +| r1-lf-lookhold-s1 | `be3f574df3db` | s1-static@1166 | `d59a1259/b2c2c63e` | clean | +| r1-anchor25-s2 | `4350da6ac745` | s2-arc@1166 | `57cbd211/a1d44644` | drifted | +| r1-anchor50-s2 | `3408f7cf9fa9` | s2-arc@1166 | `57cbd211/a1d44644` | drifted | +| r1-anchor75-s2 | `6fb052f46229` | s2-arc@1166 | `57cbd211/a1d44644` | drifted | +| r1-backoff17-s2 | `ea394296a02d` | s2-arc@1166 | `57cbd211/a1d44644` | drifted | +| r1-keystone-s2 | `f774bdbcc1cc` | s2-arc@1166 | `57cbd211/a1d44644` | drifted | +| r1-lookhold-s2 | `64dc87a2ed3a` | s2-arc@1166 | `57cbd211/a1d44644` | drifted | +| r1-antiburn-s2 | `9124d417577d` | s2-arc@1166+lora | `c7b4068d/33f00579` | clean | +| r1-lf-carrypick-s2 | `9713480482bc` | s2-arc@1166 | `e6e70a03/0a819c94` | drifted | +| r1-lf-lookhold-s2 | `242358078691` | s2-arc@1166 | `ed5094c1/0a819c94` | clean | +| r1-anchor25-s3 | `7c1aa8f8b38d` | s3-dialogue-walk@1166 | `289012f1/d21b04b0` | clean | +| r1-anchor50-s3 | `fef6b9aea7c9` | s3-dialogue-walk@1166 | `289012f1/d21b04b0` | clean | +| r1-anchor75-s3 | `fbf286d9f192` | s3-dialogue-walk@1166 | `289012f1/d21b04b0` | clean | +| r1-backoff17-s3 | `5873332b54fc` | s3-dialogue-walk@1166 | `289012f1/d21b04b0` | clean | +| r1-keystone-s3 | `a2bd5e2545db` | s3-dialogue-walk@1166 | `289012f1/d21b04b0` | clean | +| r1-lookhold-s3 | `a9a134a9acf8` | s3-dialogue-walk@1166 | `289012f1/d21b04b0` | clean | +| r1-antiburn-s3 | `c50ec5961418` | s3-dialogue-walk@1166+lora | `bfcb3f23/949a7e65` | clean | +| ctl-guide-s1-s1166-rerun | `fd34bc937628` | s1-static@1166 | `d59a1259/b2c2c63e` | clean | +| r2-anchor25-s1-seed2 | `93dd359dc11e` | s1-static@2 | `f47df963/f88334a9` | clean | +| r2-anchor50-s1-seed2 | `075ed6de83f7` | s1-static@2 | `f47df963/f88334a9` | clean | +| r2-keystone-s1-seed2 | `ab59d503fd4d` | s1-static@2 | `f47df963/f88334a9` | clean | +| r2-lf-lookhold-s1-seed2 | `b7ca6da7d075` | s1-static@2 | `f47df963/f88334a9` | clean | +| r2-lf-lookhold-s2-seed2 | `a3d6a044fd6b` | s2-arc@2 | `6fbb63c2/f6555387` | drifted | +| r2-anchor25-keystone-s1-s1166 | `8accd0c37f91` | s1-static@1166 | `abd3dabc/d5900069` | drifted | +| r2-anchor50-keystone-s1-s1166 | `e5c7b013538f` | s1-static@1166 | `cd167eee/d5900069` | drifted | +| r2-lf-lookhold-anchor25-s1-s1166 | `835379468ec3` | s1-static@1166 | `cd167eee/d5900069` | drifted | +| r2-lf-lookhold-anchor50-s1-s1166 | `d8b17df7dfb9` | s1-static@1166 | `df2c8907/6b31365e` | drifted | +| r2-anchor25-keystone-s1-s2 | `e34a5aebaab4` | s1-static@2 | `40991534/43283722` | drifted | +| r2-anchor50-keystone-s1-s2 | `08e32162985c` | s1-static@2 | `f47df963/f88334a9` | clean | +| r2-lf-lookhold-anchor25-s1-s2 | `41355297a083` | s1-static@2 | `f47df963/f88334a9` | clean | +| r2-lf-lookhold-anchor50-s1-s2 | `c8bfadf286ce` | s1-static@2 | `f47df963/f88334a9` | clean | +| r2-anchor25-keystone-s1-s1166 | `7c1c1fb14c8c` | s1-static@1166 | `d59a1259/b2c2c63e` | clean | +| r2-anchor50-keystone-s1-s1166 | `9331f354cc4b` | s1-static@1166 | `abd3dabc/d5900069` | drifted | +| r2-lf-lookhold-anchor25-s1-s1166 | `8010fb212bb3` | s1-static@1166 | `abd3dabc/d5900069` | drifted | +| r2-lf-lookhold-anchor50-s1-s1166 | `a58834fed81a` | s1-static@1166 | `abd3dabc/d5900069` | drifted | +| r2-anchor25-keystone-s1-s2 | `36c91319afbc` | s1-static@2 | `86544f72/4c5a2d39` | drifted | +| r2-anchor50-keystone-s1-s2 | `5a60b829547e` | s1-static@2 | `86544f72/4c5a2d39` | drifted | +| r2-lf-lookhold-anchor25-s1-s2 | `2059d7bcf69b` | s1-static@2 | `86544f72/4c5a2d39` | drifted | +| r2-lf-lookhold-anchor50-s1-s2 | `daa36f09dc3f` | s1-static@2 | `86544f72/4c5a2d39` | drifted | +| ctl-guide-s1-120s | `8d1614ced27a` | s1-static@1166 | `d59a1259/b2c2c63e` | clean | +| r2-anchor50-s1-120s | `107859a31bc1` | s1-static@1166 | `d59a1259/b2c2c63e` | clean | + +Summary: + +- **S1, seed 1166.** Clean: the guide control and its rerun, anchor 0.25, + anchor 0.5, both lf arms, both 120 s takes, and batch 1's anchor + 0.25 + keystone. Drifted: + - anchor 0.75 and the last_frame control (`56e0831e`); + - backoff, keystone and guide look_hold (`fe993ff4`); + - all four recorded combinations, and three of batch 1's + (`abd3dabc`, `cd167eee`, `df2c8907`). +- **S1, seed 2.** Clean: the guide control, all four round-2 confirms, and + the three combinations run after the reboot. Drifted: the last_frame + control (`4302cf65`), anchor 0.25 + keystone (`40991534`), and all four + of batch 1's (`86544f72`). +- **S2, seed 1166.** Clean: the guide control, last_frame + look_hold, and + the anti-burn LoRA run. Drifted: the last_frame control and **all six + round-1 guide arms**, which share `57cbd211`; last_frame + carry_pick + (`e6e70a03`). +- **S2, seed 2.** Clean: the guide control. Drifted: the last_frame control + (`130afe0c`) and the round-2 last_frame + look_hold diagnostic + (`6fbb63c2`). +- **S3.** Every round-1 record is clean. +- **Anti-burn LoRA arms.** Each is a cold load, labelled against its own + `+lora` entry (the LoRA changes segment 1 by design). + +### Verdicts re-scored + +Every row of the round-1 tables, the continuity table, the seed-2 +confirmation table and the combinations table, plus the round-2 S2 +diagnostic and batch 1's clean anchor 0.25 + keystone (the only clean +seed-1166 combination). The primary measures are those in *How the +verdicts are read*. Italic names in the last two columns are measures +where the arm is better. + +| round | arm | against | segment 1 (arm / against) | verdict before | verdict under R35 | mark | now inside the floor | past the floor under R35 (worse; _better_) | +|---|---|---|---|---|---|---|---|---| +| 1 | r1-anchor25-s1 | ctl-guide-s1-s1166 | clean / clean | tex fine: mixed | tex fine: wins | **flipped** | jump, jump max, level, level max, _look_, _look max_ | -; _tex fine_, _tex mid_ | +| 1 | r1-anchor50-s1 | ctl-guide-s1-s1166 | clean / clean | tex fine: mixed | tex fine: mixed | weakened | jump max, level, level max, _look_, _look max_ | jump, tex coarse; _tex fine_, _tex mid_ | +| 1 | r1-anchor75-s1 | ctl-guide-s1-s1166 | drifted `56e0831e` / clean (**confounded**) | tex fine: mixed | tex fine: mixed | weakened | look, level max | jump, jump max, look max, level, tex coarse; _tex fine_, _tex mid_ | +| 1 | r1-backoff17-s1 | ctl-guide-s1-s1166 | drifted `fe993ff4` / clean (**confounded**) | tex fine: mixed | tex fine: tie | **flipped** | look max, level, level max | -; - | +| 1 | r1-keystone-s1 | ctl-guide-s1-s1166 | drifted `fe993ff4` / clean (**confounded**) | tex fine: mixed | tex fine: mixed (pic wins) | **flipped** | jump max, level max, _look max_ | level; _tex fine_, _tex mid_ | +| 1 | r1-lookhold-s1 | ctl-guide-s1-s1166 | drifted `fe993ff4` / clean (**confounded**) | tex fine: mixed | tex fine: tie | **flipped** | jump max, level, level max, _look_, _tex coarse_ | -; - | +| 1 | r1-antiburn-s1 | ctl-guide-s1-s1166 | clean / clean | tex fine: mixed | tex fine: mixed | weakened | _jump max_, _level max_ | look, look max, tex coarse; _jump_, _tex fine_, _tex mid_ | +| 1 | r1-lf-carrypick-s1 | ctl-lastframe-s1-s1166 | clean / drifted `56e0831e` (**confounded**) | tex fine: mixed (pic wins) | tex fine: tie | **flipped** | level, level max, _jump_, _jump max_, _look max_, _tex fine_, _tex coarse_ | -; - | +| 1 | r1-lf-lookhold-s1 | ctl-lastframe-s1-s1166 | clean / drifted `56e0831e` (**confounded**) | tex fine: mixed (pic wins) | tex fine: wins | **flipped** | level, _jump max_, _level max_, _tex coarse_ | -; _jump_, _look_, _look max_, _tex fine_ | +| 1 | r1-anchor25-s2 | ctl-guide-s2-s1166 | drifted `57cbd211` / clean (**confounded**) | look: loses / jump: mixed | look: loses / jump: mixed | unchanged | - | look, look max; _jump_, _jump max_, _tex fine_, _tex mid_ | +| 1 | r1-anchor50-s2 | ctl-guide-s2-s1166 | drifted `57cbd211` / clean (**confounded**) | look: loses / jump: loses | look: loses / jump: loses | weakened | jump max | jump, look, look max; - | +| 1 | r1-anchor75-s2 | ctl-guide-s2-s1166 | drifted `57cbd211` / clean (**confounded**) | look: loses / jump: loses | look: loses / jump: loses | unchanged | - | jump, jump max, look, look max; _tex fine_ | +| 1 | r1-backoff17-s2 | ctl-guide-s2-s1166 | drifted `57cbd211` / clean (**confounded**) | look: loses / jump: mixed | look: mixed / jump: mixed | **flipped** | look | look max; - | +| 1 | r1-keystone-s2 | ctl-guide-s2-s1166 | drifted `57cbd211` / clean (**confounded**) | look: loses / jump: mixed | look: tie / jump: tie | **flipped** | look, look max | -; - | +| 1 | r1-lookhold-s2 | ctl-guide-s2-s1166 | drifted `57cbd211` / clean (**confounded**) | look: loses / jump: mixed | look: mixed / jump: mixed | **flipped** | look | look max; - | +| 1 | r1-antiburn-s2 | ctl-guide-s2-s1166 | clean / clean | look: loses / jump: mixed | look: tie / jump: tie | **flipped** | look, look max, _jump_, _jump max_ | -; - | +| 1 | r1-lf-carrypick-s2 | ctl-lastframe-s2-s1166 | drifted `e6e70a03` / drifted `57cbd211` (**confounded**) | look: mixed / jump: loses | look: tie / jump: tie | **flipped** | jump, _look_, _look max_ | -; _tex fine_, _tex mid_, _tex coarse_ | +| 1 | r1-lf-lookhold-s2 | ctl-lastframe-s2-s1166 | clean / drifted `57cbd211` (**confounded**) | look: wins / jump: tie | look: wins / jump: tie | unchanged | - | -; _look_, _look max_, _tex fine_, _tex mid_, _tex coarse_ | +| 1 | r1-anchor25-s3 | ctl-guide-s3-s1166 | clean / clean | jump: mixed / look: mixed / tex fine: mixed | jump: mixed / look: mixed / tex fine: mixed | weakened | jump max, look max | tex coarse; - | +| 1 | r1-anchor50-s3 | ctl-guide-s3-s1166 | clean / clean | jump: mixed / look: mixed / tex fine: mixed | jump: mixed / look: mixed / tex fine: mixed | weakened | jump max, look max, level | tex coarse; - | +| 1 | r1-anchor75-s3 | ctl-guide-s3-s1166 | clean / clean | jump: mixed / look: loses / tex fine: mixed | jump: mixed / look: loses / tex fine: mixed | unchanged | - | jump max, look, look max, tex coarse; - | +| 1 | r1-backoff17-s3 | ctl-guide-s3-s1166 | clean / clean | jump: mixed / look: mixed / tex fine: loses | jump: mixed / look: mixed / tex fine: loses | weakened | jump max, level | tex fine, tex mid; _tex coarse_ | +| 1 | r1-keystone-s3 | ctl-guide-s3-s1166 | clean / clean | jump: mixed / look: mixed / tex fine: mixed | jump: tie / look: tie / tex fine: tie | **flipped** | jump max, look max, level | -; - | +| 1 | r1-lookhold-s3 | ctl-guide-s3-s1166 | clean / clean | jump: tie / look: tie / tex fine: tie | jump: tie / look: tie / tex fine: tie | weakened | _look max_ | -; - | +| 1 | r1-antiburn-s3 | ctl-guide-s3-s1166 | clean / clean | jump: loses / look: mixed / tex fine: loses | jump: loses / look: mixed / tex fine: loses | weakened | look max, level | jump, jump max, tex fine, tex mid, tex coarse; - | +| 1 | ctl-lastframe-s1-s1166 | ctl-guide-s1-s1166 | drifted `56e0831e` / clean (**confounded**) | tex fine: mixed | tex fine: mixed | weakened | jump max, _look_, _look max_, _level_, _level max_ | jump, tex mid; - | +| 1 | ctl-lastframe-s2-s1166 | ctl-guide-s2-s1166 | drifted `57cbd211` / clean (**confounded**) | look: mixed / jump: mixed | look: mixed / jump: mixed | weakened | level, _jump_, _jump max_ | tex mid, tex coarse; - | +| 1 | r1-lf-carrypick-s1 | ctl-guide-s1-s1166 | clean / clean | tex fine: mixed | tex fine: mixed | weakened | jump max, _look_, _look max_, _level max_ | jump, tex mid; _tex coarse_ | +| 1 | r1-lf-lookhold-s1 | ctl-guide-s1-s1166 | clean / clean | tex fine: mixed | tex fine: mixed | weakened | _jump_, _jump max_, _level_ | tex mid; _look_, _look max_, _level max_, _tex coarse_ | +| 1 | r1-lf-carrypick-s2 | ctl-guide-s2-s1166 | drifted `e6e70a03` / clean (**confounded**) | look: wins / jump: tie | look: tie / jump: tie | **flipped** | _jump max_, _look_, _look max_ | -; _tex fine_ | +| 1 | r1-lf-lookhold-s2 | ctl-guide-s2-s1166 | clean / clean | look: wins / jump: tie | look: wins / jump: tie | weakened | _jump max_ | -; _look_, _look max_, _tex fine_, _tex mid_ | +| 2 | r1-anchor25-s1 | ctl-guide-s1-s1166 | clean / clean | tex fine: mixed | tex fine: wins | **flipped** | jump, jump max, level max, _look_, _look max_ | -; _tex fine_, _tex mid_ | +| 2 | r2-anchor25-s1-seed2 | ctl-guide-s1-s2 | clean / clean | tex fine: mixed | tex fine: mixed | weakened | _look max_ | tex coarse; _tex fine_ | +| 2 | r1-anchor50-s1 | ctl-guide-s1-s1166 | clean / clean | tex fine: mixed | tex fine: mixed | weakened | jump max, level max, _look_, _look max_ | jump, tex coarse; _tex fine_, _tex mid_ | +| 2 | r2-anchor50-s1-seed2 | ctl-guide-s1-s2 | clean / clean | tex fine: mixed | tex fine: mixed | weakened | jump, jump max, _look max_ | tex coarse; _tex fine_ | +| 2 | r1-keystone-s1 | ctl-guide-s1-s1166 | drifted `fe993ff4` / clean (**confounded**) | tex fine: mixed | tex fine: mixed (pic wins) | **flipped** | jump max, level max, _look max_ | level; _tex fine_, _tex mid_ | +| 2 | r2-keystone-s1-seed2 | ctl-guide-s1-s2 | clean / clean | tex fine: mixed | tex fine: tie | **flipped** | look max | -; - | +| 2 | r1-lf-lookhold-s1 | ctl-lastframe-s1-s1166 | clean / drifted `56e0831e` (**confounded**) | tex fine: wins | tex fine: wins | weakened | _jump max_ | -; _jump_, _look_, _look max_, _tex fine_ | +| 2 | r2-lf-lookhold-s1-seed2 | ctl-lastframe-s1-s2 | clean / drifted `4302cf65` (**confounded**) | tex fine: wins | tex fine: tie | **flipped** | _jump max_, _look max_, _tex fine_ | -; _jump_, _look_ | +| 2 | r1-lf-lookhold-s1 | ctl-guide-s1-s1166 | clean / clean | tex fine: mixed | tex fine: mixed | weakened | _jump_, _jump max_ | tex mid; _look_, _look max_, _level max_, _tex coarse_ | +| 2 | r2-lf-lookhold-s1-seed2 | ctl-guide-s1-s2 | clean / clean | tex fine: loses | tex fine: loses | weakened | jump max | tex fine, tex mid; _look_, _look max_, _level max_ | +| 2 | r2-anchor25-keystone-s1-s1166 | ctl-guide-s1-s1166 | drifted `abd3dabc` / clean (**confounded**) | tex fine: mixed | tex fine: mixed | weakened | jump max, _look_, _look max_ | tex coarse; _tex fine_, _tex mid_ | +| 2 | r2-anchor25-keystone-s1-s1166 | r1-anchor25-s1 | drifted `abd3dabc` / clean (**confounded**) | tex fine: mixed | tex fine: mixed | weakened | _look max_, _level max_ | tex coarse; - | +| 2 | r2-anchor25-keystone-s1-s2 | ctl-guide-s1-s2 | drifted `40991534` / clean (**confounded**) | tex fine: mixed | tex fine: mixed | weakened | jump max, level max, _look max_ | level, tex coarse; _tex fine_ | +| 2 | r2-anchor25-keystone-s1-s2 | r2-anchor25-s1-seed2 | drifted `40991534` / clean (**confounded**) | tex fine: mixed | tex fine: mixed (pic tie) | **flipped** | jump max | level max; - | +| 2 | r2-anchor50-keystone-s1-s1166 | ctl-guide-s1-s1166 | drifted `cd167eee` / clean (**confounded**) | tex fine: mixed | tex fine: mixed | weakened | look max | jump, jump max, look, tex coarse; _tex fine_, _tex mid_ | +| 2 | r2-anchor50-keystone-s1-s1166 | r1-anchor50-s1 | drifted `cd167eee` / clean (**confounded**) | tex fine: mixed | tex fine: mixed | weakened | jump max, _level max_ | jump, look, look max, tex coarse; _level_ | +| 2 | r2-anchor50-keystone-s1-s2 | ctl-guide-s1-s2 | clean / clean | tex fine: mixed | tex fine: mixed | weakened | look, look max, level max | jump, jump max, tex coarse; _tex fine_, _tex mid_ | +| 2 | r2-anchor50-keystone-s1-s2 | r2-anchor50-s1-seed2 | clean / clean | tex fine: mixed | tex fine: mixed | weakened | level max | jump, jump max, look, look max, tex coarse; - | +| 2 | r2-lf-lookhold-anchor25-s1-s1166 | ctl-lastframe-s1-s1166 | drifted `cd167eee` / drifted `56e0831e` (**confounded**) | tex fine: wins | tex fine: wins | weakened | _jump max_ | -; _jump_, _look_, _look max_, _tex fine_, _tex mid_ | +| 2 | r2-lf-lookhold-anchor25-s1-s1166 | ctl-guide-s1-s1166 | drifted `cd167eee` / clean (**confounded**) | tex fine: wins | tex fine: wins | weakened | _jump_, _level max_ | -; _look_, _look max_, _tex fine_ | +| 2 | r2-lf-lookhold-anchor25-s1-s1166 | r1-anchor25-s1 | drifted `cd167eee` / clean (**confounded**) | tex fine: tie | tex fine: tie | weakened | _jump max_ | -; _jump_, _look_, _look max_, _level_, _level max_ | +| 2 | r2-lf-lookhold-anchor25-s1-s1166 | r1-lf-lookhold-s1 | drifted `cd167eee` / clean (**confounded**) | tex fine: mixed | tex fine: wins | **flipped** | jump max, _look max_ | -; _tex fine_, _tex mid_ | +| 2 | r2-lf-lookhold-anchor25-s1-s2 | ctl-lastframe-s1-s2 | clean / drifted `4302cf65` (**confounded**) | tex fine: wins | tex fine: wins | weakened | _jump max_ | -; _jump_, _look_, _look max_, _tex fine_, _tex mid_ | +| 2 | r2-lf-lookhold-anchor25-s1-s2 | ctl-guide-s1-s2 | clean / clean | tex fine: mixed | tex fine: mixed | weakened | _jump max_ | tex mid; _look_, _look max_, _level max_, _tex fine_ | +| 2 | r2-lf-lookhold-anchor25-s1-s2 | r2-anchor25-s1-seed2 | clean / clean | tex fine: mixed | tex fine: mixed | weakened | _jump_, _jump max_, _look_ | tex mid; _look max_, _level_, _level max_, _tex coarse_ | +| 2 | r2-lf-lookhold-anchor25-s1-s2 | r2-lf-lookhold-s1-seed2 | clean / clean | tex fine: mixed | tex fine: mixed | weakened | _jump_, _jump max_, _look max_ | tex coarse; _tex fine_, _tex mid_ | +| 2 | r2-lf-lookhold-anchor50-s1-s1166 | ctl-lastframe-s1-s1166 | drifted `df2c8907` / drifted `56e0831e` (**confounded**) | tex fine: wins | tex fine: wins | weakened | _jump max_, _look max_ | -; _jump_, _look_, _tex fine_, _tex mid_ | +| 2 | r2-lf-lookhold-anchor50-s1-s1166 | ctl-guide-s1-s1166 | drifted `df2c8907` / clean (**confounded**) | tex fine: wins | tex fine: wins | weakened | _jump_ | -; _look_, _look max_, _level max_, _tex fine_ | +| 2 | r2-lf-lookhold-anchor50-s1-s1166 | r1-anchor50-s1 | drifted `df2c8907` / clean (**confounded**) | tex fine: wins | tex fine: wins | weakened | _jump max_, _look max_ | -; _jump_, _level_, _level max_, _tex fine_, _tex coarse_ | +| 2 | r2-lf-lookhold-anchor50-s1-s1166 | r1-lf-lookhold-s1 | drifted `df2c8907` / clean (**confounded**) | tex fine: mixed | tex fine: wins | **flipped** | jump max, look max, _jump_ | -; _tex fine_, _tex mid_ | +| 2 | r2-lf-lookhold-anchor50-s1-s2 | ctl-lastframe-s1-s2 | clean / drifted `4302cf65` (**confounded**) | tex fine: mixed | tex fine: mixed | weakened | _jump max_, _look max_ | tex coarse; _jump_, _look_, _tex fine_, _tex mid_ | +| 2 | r2-lf-lookhold-anchor50-s1-s2 | ctl-guide-s1-s2 | clean / clean | tex fine: mixed | tex fine: mixed | unchanged | - | tex mid, tex coarse; _look_, _look max_, _level max_, _tex fine_ | +| 2 | r2-lf-lookhold-anchor50-s1-s2 | r2-anchor50-s1-seed2 | clean / clean | tex fine: mixed | tex fine: mixed | weakened | _jump max_, _look_, _look max_ | tex mid, tex coarse; _jump_, _level max_ | +| 2 | r2-lf-lookhold-anchor50-s1-s2 | r2-lf-lookhold-s1-seed2 | clean / clean | tex fine: mixed | tex fine: mixed | weakened | look max, _jump_, _jump max_, _level max_ | tex coarse; _tex fine_, _tex mid_ | +| 2 | r2-anchor25-keystone-s1-s1166, batch 1 (`7c1c1fb14c8c`) | ctl-guide-s1-s1166 | clean / clean | tex fine: mixed | tex fine: mixed | weakened | look max | jump, jump max, tex coarse; _tex fine_, _tex mid_ | +| 2 | r2-anchor25-keystone-s1-s1166, batch 1 (`7c1c1fb14c8c`) | r1-anchor25-s1 | clean / clean | tex fine: mixed | tex fine: mixed | weakened | jump, look, look max, _level max_ | jump max, tex coarse; _level_, _tex fine_ | +| 2 | r2-lf-lookhold-s2-seed2 | ctl-lastframe-s2-s2 | drifted `6fbb63c2` / drifted `130afe0c` (**confounded**) | look: wins / jump: tie | look: wins / jump: tie | unchanged | - | -; _look_, _look max_, _tex fine_, _tex mid_, _tex coarse_ | + +Count: 68 rows. 6 unchanged, 42 weakened, 20 flipped. 15 of the flips are +on confounded rows. The 5 clean flips (anchor 0.25 on both readings, and +keystone at S3, at S1 seed 2, and anti-burn at S2) are final under the +floor: a same-seed rerun cannot settle them. + +### Calls under R35 + +- **S2 has no candidate: weakened, not flipped.** No S2 arm wins a primary + under R35, so there is still no candidate. What changes is the reason. + - Anchor 0.25, 0.5 and 0.75 still lose seam look by 0.75-2.2, past the + 0.438 floor. + - Backoff and guide look_hold move from *loses* to *mixed*. Their + seam-look means are inside the floor, and only look max is worse past + it (2.61 and 2.97 against 1.74, floor 0.79). + - Keystone moves to *tie* on both primaries. + - Anti-burn is clean on both sides and also ties on the numbers; by eye + it is broken. + + So "every guide counter loses seam look" holds for the anchor only. All + six guide arms ran in one drifted worker state (`57cbd211`) against a + clean control, but no flip makes any of them a win, and the eye notes + saw no improvement. No rerun is needed for the call. The guide and + last_frame S2 settings in *Defaults by shot type* stand. +- **Round-1 S1 anchor 0.75: unchanged call (not carried), weakened row.** + This row is confounded: the arm opens on `56e0831e` and the control is + clean. It is still *mixed*, with jump, jump max, look max, level and + coarse texture worse past the floor. Only the seam-look cost is now inside + it. Identity drift by eye stands. No rerun. +- **Round-1 S1 backoff 17: unchanged call (cut).** The row flips from *mixed* + to *tie* (confounded, `fe993ff4`). The costs fall inside the floor, but + the texture gain is still inside it too (0.462 against 0.536, floor + 0.081). S3 is clean and still loses texture (0.337 against 0.067, floor + 0.226). On S2 the row moves from *loses* to *mixed*. Backoff wins nowhere. +- **Round-1 S1 keystone: unchanged call (cut).** The 1166 row flips on + picture only, from *mixed* to *wins* (confounded, `fe993ff4`), because its + jump max cost is now inside the floor. The cut rested on seed 2, and seed + 2 is clean on both sides and now a plain *tie* (0.448 against 0.455). + A winner must hold on both seeds, so the call does not move. S3 (clean) + is now *tie* on every primary, and S2 is *tie* as well. + - **Keystone with anchor 0.25: weakened.** Batch 1's clean 1166 run + (`7c1c1fb14c8c`) reads better than the recorded drifted one. Its + texture beats anchor 0.25 alone by 0.084, past the 0.081 floor by + 0.003, but jump max and coarse texture are worse past the floor + (*mixed*). Its seam look is 1.53, about the control's 1.46, not the 1.09 + the drifted record showed. So "keystone adds nothing to anchor 0.25" + becomes "a texture gain at the edge of the floor, paid for in seams". + The cut stands. +- **Round-1 S1 guide look_hold: unchanged call (cut).** The row flips from + *mixed* to *tie* (confounded). On S2 it moves from *loses* to *mixed* on + look max, and on S3 it is still a tie. +- **The last_frame controls: all four are drifted.** S1 at 1166 is + `56e0831e`, S1 at seed 2 is `4302cf65`, S2 at 1166 is `57cbd211`, and S2 + at seed 2 is `130afe0c`. Every row read against one is confounded: + - **last_frame + look_hold, S1, against its own control: undetermined.** + At 1166 the row goes from *mixed* to *wins*, because the level cost is + now inside the floor. At seed 2 it goes from *wins* to *tie*: texture + is 0.550 against 0.605, inside the new 0.068 floor. So the claim "a + strict win against the last_frame control on both seeds" no longer + holds as measured, and only clean controls can settle it. The call in + *Defaults by shot type* (a knob, not the default) does not rest on this + claim. It rests on the reads against guide, which are clean and + unchanged (*mixed* at 1166 on mid texture, *loses* at seed 2), and on + the S2 motion loss by eye. **The call stands.** + - **last_frame + carry_pick, S1: flipped toward the cut.** It goes from + *mixed* (picture *wins*) to *tie*. The cut stands. + - **Continuity, last_frame against guide (the controls): unchanged.** On + S1, last_frame's seam-look and level advantages are now inside the + floor. It is still worse past the floor on jump and mid texture. On S2, + its jump advantage is now inside the floor, and its mid and coarse + texture costs remain. last_frame is still not better than guide as the + default. + - **S2 last_frame arms.** carry_pick flips to *tie* on both primaries. + look_hold is unchanged (*wins* look), and so is the seed-2 diagnostic. + Both were rejected by eye for holding the camera back, and that stands. + - **The lf + look_hold + anchor set: unchanged against guide.** It beats + the guide control on texture and seam look on all four runs. Against + look_hold alone, both 1166 rows (confounded) flip from *mixed* to + *wins*, while seed 2 (clean) is still *mixed*. It stays Don's call and + not the default. +- **Anchor 0.25 (the static default): strengthened.** Both sides are clean + on both seeds. At 1166 the row flips from *mixed* to a strict *wins*: the + jump cost (0.928 against 0.854) is inside the 0.124 floor. At seed 2 the + row is still *mixed*, on coarse texture only (0.031 against 0.008, floor + 0.010). +- **Anchor 0.5: weakened.** The seed-2 jump cost (0.904 against 0.796) is + now inside the floor, so the jump cost repeats at 1166 only. Coarse + texture is still worse past the floor on both seeds. The recommendation + of 0.25 over 0.5 still stands on three points: + - the face dissolves at two minutes (clean, `107859a31bc1`); + - 0.5's texture lead over 0.25 at seed 2 is inside the floor; + - the coarse-texture cost. +- **Anti-burn LoRA: unchanged call (cut).** On S2 it flips to *tie* (both + sides clean). It still loses S3 (clean), and by eye it blurs every scene. +- **S3: unchanged (no counters).** Every record is clean. Keystone moves + from *mixed* to *tie*, and nothing wins. +- **Two-minute takes: unchanged.** Both are clean (`d59a1259`) and are read + against each other. +- **The level-step question: nearly moot.** With a 6.2/8.8 dB floor, only + two re-scored rows still differ between the all-measures and the + picture-only column: + - r1-keystone-s1 (level 14.6 against 7.7 dB); + - anchor 0.25 + keystone at seed 2 against anchor 0.25 alone (level + max). + + Neither decides a call, so the recommendation to read S1 and S2 on + picture changes nothing. + +### Reruns + +**Minimum set: 2 runs.** These are the only call that is undetermined +(last_frame + look_hold against its own control) and the only control +reads R35 cannot score: + +1. `ctl-lastframe-s1-s1166` (`bench/arms/counters-controls.json`) +2. `ctl-lastframe-s1-s2` + +How to run them: +- Give each a fresh `workflow_id_suffix` (for example `-r35`) so the step + cache cannot serve it. +- Run each as the first job after a model load (a fresh bench server, or a + canary control just before), and confirm `clean` with `bench/segment1.py` + before reading it. +- They take about 9 minutes each, plus about 2 minutes of load. + +These two runs re-score, against a clean control, every row now read +against a drifted one: +- r1-lf-lookhold-s1 and r2-lf-lookhold-s1-seed2 (the undetermined claim); +- r1-lf-carrypick-s1; +- the four lf combinations against the last_frame control; +- the S1 continuity row. + +**Not needed for any call** (a confounded row flipped, but the call it fed +is decided elsewhere): +- r1-keystone-s1 (picture *wins*; the cut rests on the clean seed-2 tie); +- r1-backoff17-s1 and r1-lookhold-s1 (to *tie*; cut); +- r1-backoff17-s2, r1-keystone-s2 and r1-lookhold-s2 (no S2 win either + way); +- r1-lf-carrypick-s2 and the S2 last_frame controls (rejected by eye); +- r2-anchor25-keystone-s1-s2 (picture *tie* against anchor alone; cut). + +A clean rerun of r1-keystone-s1 is the cheapest to add (one run) if the +seed-1166 keystone row should read clean as well. + +### Clean last_frame controls (2026-11-05) + +The minimum set ran (`bench/arms/lastframe-ctl-clean-rerun.json`, +`bench/records/lastframe-ctl-clean-rerun.json`), each the first job after a +bench server restart on cuda:0 alone, with a fresh workflow-id suffix. Both +are **clean** by `bench/segment1.py`: + +| arm | job | segment 1 | jump | jump max | look | look max | level dB | level dB max | tex fine | tex mid | tex coarse | +|---|---|---|---|---|---|---|---|---|---|---|---| +| ctl-lastframe-s1-s1166-clean | `88250351f1fe` | `d59a1259/b2c2c63e` clean | 1.036 | 1.190 | 1.254 | 1.730 | 4.508 | 9.350 | 0.522 | 0.342 | 0.002 | +| ctl-lastframe-s1-s2-clean | `194ee59b559d` | `f47df963/f88334a9` clean | 1.072 | 1.270 | 1.340 | 1.760 | 4.698 | 13.860 | 0.593 | 0.448 | 0.011 | + +Against the drifted originals the clean controls moved fine texture by +0.047 (1166) and 0.012 (seed 2), inside the rerun gap. The S1 last_frame +floor, `combined_floor(noise_floor(clean 1166, clean 2), rerun gap)`, is +jump 0.124, jump max 0.430, look 0.438, look max 0.790, level 6.204 / 8.800 +dB, tex fine **0.071**, tex mid 0.106, tex coarse 0.010. + +| arm | against (clean) | tex fine arm / ctl | verdict (all) | verdict (picture) | past the floor (worse; _better_) | +|---|---|---|---|---|---| +| r1-lf-lookhold-s1 | ctl-lastframe-s1-s1166-clean | 0.491 / 0.522 | tie | tie | -; _jump, look_ | +| r2-lf-lookhold-s1-seed2 | ctl-lastframe-s1-s2-clean | 0.550 / 0.593 | tie | tie | -; _jump, look_ | +| r1-lf-carrypick-s1 | ctl-lastframe-s1-s1166-clean | 0.533 / 0.522 | tie | tie | -; - | +| r2-lf-lookhold-anchor25-s1-s1166 (drifted `cd167eee`) | ctl-lastframe-s1-s1166-clean | 0.331 / 0.522 | wins | wins | -; _jump, look, look max, tex fine, tex mid_ | +| r2-lf-lookhold-anchor50-s1-s1166 (drifted `df2c8907`) | ctl-lastframe-s1-s1166-clean | 0.183 / 0.522 | wins | wins | -; _jump, look, tex fine, tex mid_ | +| r2-lf-lookhold-anchor25-s1-s2 | ctl-lastframe-s1-s2-clean | 0.297 / 0.593 | wins | wins | -; _jump, jump max, look, look max, tex fine, tex mid_ | +| r2-lf-lookhold-anchor50-s1-s2 | ctl-lastframe-s1-s2-clean | 0.236 / 0.593 | mixed | mixed | tex coarse; _jump, look, level max, tex fine, tex mid_ | +| continuity: ctl-lastframe-s1-s1166-clean | ctl-guide-s1-s1166 (cross floor, tex fine 0.081) | 0.522 / 0.536 | mixed | mixed | jump, tex mid; _tex coarse_ | +| continuity: ctl-lastframe-s1-s2-clean | ctl-guide-s1-s2 (cross floor) | 0.593 / 0.455 | loses | loses | jump, tex fine, tex mid; - | + +- **"last_frame + look_hold beats its own control on both seeds": settled, + no.** On the primary (fine texture) it is a **tie on both seeds** (gains + 0.031 and 0.043 against a 0.071 floor). What it does beat, past the floor + on both seeds, is seam jump and seam look mean: look_hold smooths the + last_frame hand-over, not the burn. The call in *Defaults by shot type* + (a knob, not the default) stands, now with clean evidence. +- **last_frame + carry_pick, S1: tie, clean on both sides.** The cut stands. +- **The lf + look_hold + anchor set** beats the clean last_frame control on + texture at both seeds (*wins* three of four; anchor 0.5 at seed 2 is + *mixed* on coarse texture). The anchor carries the gain, as against guide. + Still Don's call, not the default. +- **Continuity (last_frame against guide), clean on both sides:** *mixed* at + 1166 and *loses* at seed 2. Guide stays the default. + +## Round 3 (2 min), 2026-11-05 + +Round 3 asks one question: does Don's static default (guide, `anchor: +segment_1`, `anchor_strength: 0.25`) hold at two minutes? Two S1 runs, both +seed 1166, 120 s, 2880 frames, on cuda:0 +(`bench/arms/counters-round3-long.json`, `bench/records/counters-round3-long.json`): + +- `r3-anchor25-s1-120s` (`25f6bd1da507`): guide + anchor 0.25. 28 seams. +- `r3-lf-lookhold-anchor25-s1-120s` (`9877bbfd85a8`): last_frame + look_hold + + anchor 0.25 (the static alternative set). 23 seams (last_frame segments + are about 5.1 s, not 4.25 s). + +The control is round 2's `ctl-guide-s1-120s` (`8d1614ced27a`), with +`r2-anchor50-s1-120s` (`107859a31bc1`) beside it. The eight-moment texture +and per-seam pass is `bench/records/counters-round3-long-moments.json`. It +was taken in-process on lem with the deployed `dw.media_compare.compare` +and `dw.server.assess.assess` (the code behind the compare and assess +routes), not through the server. It reproduces round 2's moments record +and the driver's measures exactly. + +### Segment-1 labels + +| arm | job | key | segment 1 | label | +|---|---|---|---|---| +| r3-anchor25-s1-120s | `25f6bd1da507` | s1-static@1166 | `d59a1259/b2c2c63e` | clean | +| r3-lf-lookhold-anchor25-s1-120s | `9877bbfd85a8` | s1-static@1166 | `91deb574/008eae4c` | **drifted** (a new alternate) | +| ctl-guide-s1-120s | `8d1614ced27a` | s1-static@1166 | `d59a1259/b2c2c63e` | clean | +| r2-anchor50-s1-120s | `107859a31bc1` | s1-static@1166 | `d59a1259/b2c2c63e` | clean | + +So anchor 0.25 against the control is a clean read. Every read of the +last_frame run is confounded. + +### Floor + +`combined_floor(seed floor, rerun gap)`, the R35 S1 guide row: jump +0.124/0.430, look 0.438/0.790, level 6.2/8.8 dB, texture 0.081/0.032/0.010. +For reads against the last_frame run, the S1 last_frame seed floor is +folded in as well, which only lifts mid texture to 0.087. These are +6-segment floors. **There is no two-minute floor**, and every two-minute +run is a single seed. + +### Texture over time + +Drift against frame 62 on the S1 crop `[0,0,240,150]` (the circle and the +top of the stripes), `|ln(band / band@62)|`: + +| frame (s) | 300 (12.5) | 583 (24.3) | 1000 (41.7) | 1501 (62.5) | 2000 (83.3) | 2500 (104.2) | 2868 (119.5) | +|---|---|---|---|---|---|---|---| +| control fine | 0.180 | 0.536 | 1.108 | 1.399 | 1.646 | 1.658 | 1.706 | +| **anchor 0.25 fine** | 0.153 | 0.367 | 0.497 | 0.771 | 0.931 | 1.012 | 1.108 | +| lf + look_hold + anchor 0.25 fine | 0.144 | 0.253 | 0.478 | 0.575 | 0.408 | 0.697 | 0.930 | +| anchor 0.5 fine | 0.137 | 0.270 | 0.360 | 0.602 | 0.737 | 0.759 | 1.067 | +| control mid | 0.045 | 0.176 | 0.406 | 0.716 | 1.071 | 1.156 | 1.171 | +| **anchor 0.25 mid** | 0.046 | 0.124 | 0.179 | 0.271 | 0.319 | 0.356 | 0.430 | +| lf + look_hold + anchor 0.25 mid | 0.072 | 0.185 | 0.251 | 0.368 | 0.351 | 0.411 | 0.637 | +| anchor 0.5 mid | 0.038 | 0.116 | 0.162 | 0.227 | 0.307 | 0.273 | 0.462 | + +On the numbers, anchor 0.25 both **delays and reduces** the burn: + +- **Delay.** The control reaches 0.54 fine drift at 24 s (segment 6). Anchor + 0.25 reaches the same level between 42 and 62 s (segments 10-14), so it + roughly doubles the clean length. +- **Reduction.** It ends at 1.11 fine and 0.43 mid, against the control's 1.71 + and 1.17. Mid stays at about a third of the control's from 42 s on. +- **Against 0.5.** On fine texture, 0.25 runs 0.1-0.2 above 0.5 from 24 to + 104 s. At the end the two are inside the floor (1.108 against 1.067). + Mid texture is close all the way. + +### Seams + +By stretch of the take (`analyze_seams`, every seam, mean (max)): + +| seconds | control jump | anchor 0.25 jump | lf+lh+a0.25 jump | control look | anchor 0.25 look | lf+lh+a0.25 look | +|---|---|---|---|---|---|---| +| 0-25 | 0.85 (0.97) | 0.93 (1.15) | 0.72 (0.78) | 1.46 (2.13) | 1.13 (1.98) | 0.71 (0.83) | +| 25-60 | 1.17 (1.64) | 0.88 (1.29) | 0.83 (0.99) | 1.88 (2.61) | 0.78 (2.37) | 0.77 (1.03) | +| 60-95 | 1.29 (1.94) | 0.80 (0.96) | 1.33 (**4.24**) | 1.71 (3.75) | 0.68 (1.53) | 1.53 (**6.36**) | +| 95-120 | 2.06 (**6.56**) | 1.26 (1.86) | 1.09 (1.27) | 4.12 (**11.51**) | 1.81 (2.70) | 2.00 (4.71) | + +- Anchor 0.25's worst seam is the very last one, at 119.9 s (jump 1.86, look + 2.70). It never passes the chain check's 4.0. +- The last_frame run is the best set to about 60 s. Then it has one hard + seam at 76.3 s (jump 4.24, look 6.36), and its look degrades through the + last 25 s. +- Level step: anchor 0.25 is 5.8 dB mean (24.3 max) against the control's + 5.9 (36.5); last_frame is 5.5 (15.0). Every level mean is inside the + 6.2 dB floor. Anchor 0.25's loud steps come in the first minute (12.2 and + 9.9 dB means), as on every S1 guide arm. + +### Verdicts (`verdict()`, primary `texture_fine_seg_last`) + +| read | verdict (all / picture) | past the floor | +|---|---|---| +| anchor 0.25 vs control (clean, clean) | **wins / wins** | better on all 8 picture measures and on level max; level mean inside | +| lf+lh+a0.25 vs control (drifted, clean) | wins / wins (confounded) | better on all 8 picture measures and on level max | +| lf+lh+a0.25 vs anchor 0.25 (drifted, clean) | mixed / mixed (confounded) | fine better (0.930 vs 1.108); **worse** jump max (+2.38), look max (+3.66), mid (+0.21), coarse (+0.05) | +| anchor 0.25 vs anchor 0.5 (clean, clean) | tie / tie | only coarse texture better (0.019 vs 0.089) | + +**Motion.** S1 is locked off, and the only motion measure the bench takes is +the within-segment frame delta at each seam (`typical_delta`). It is 2.0-2.4 on +every run, with no stall and no surge. By eye the last_frame run's subject +turns her head at 83 s; the guide runs stay frontal. + +### Eye check: face and backdrop + +Face crops (`crop=240:240:360:30`) at 24/42/52/62/83/104/112/119.5 s, and +full frames at the same moments: + +- **Control:** as in round 2. The backdrop crusts from 24 s and the face + cracks from 42 s. From 83 s she is a mottled mosaic, though still the + same person. +- **Anchor 0.25:** clean at 24 s. At 42 s the backdrop is visibly crusting + (the stripes and the circle). The face is waxy, with heavy blush, but holds. + At 52 s it softens, and at 62 s it is pale and flattened. From 83 s it is + washed out, and by 104-120 s it is a pale mask, with a white patch over the + forehead and faint features. **This is the anchor 0.5 failure, about 20 s + later and milder**: the eyes, nose and mouth are still there at 120 s, + where 0.5's eyes had gone by 62-83 s. The backdrop is a full mosaic by + 62 s. The texture number after about 60 s is part blur (#1175 trap), + as with 0.5. +- **last_frame + look_hold + anchor 0.25:** the best face hold of any + two-minute run. The face is natural to 62 s and still present at 83 s + (head turned). It smears from 104 s and is dissolving into crust at + 120 s. The backdrop burns like anchor 0.25's. It opens on a drifted segment 1 + (the woman at 24 s is visibly a different take from the guide runs'). + +Export for Don (the last 20 s of each, 100-120 s, re-encoded with x264 at +CRF 16): `~/Desktop/dw-bench-r3/` on the Mac: +`r3-guide-anchor25-last20s.mp4`, `r3-lastframe-lookhold-anchor25-last20s.mp4` +and `r2-guide-control-last20s.mp4`. + +### The call + +- **Guide + anchor 0.25 still beats the guide control at two minutes.** This + is a strict *wins* on a clean read, with every picture measure better by + 3-23x its floor, and the margin grows with length. The burn is delayed, not + removed: + - visible backdrop crust from segment 10 (about 42 s); + - the face softens from segment 13 (52 s); + - the face is washed out from segment 20 (83 s). + + The control burns from segment 6 (24 s). +- **The static default stands, and so does the cut rule.** A two-minute + static take is still not usable on any arm. Anchor 0.25 moves the usable + length from about 30 s to about 35-40 s at best, which is not enough to + change "cut static shots at about 30 s (6-8 segments)". The bench's + conclusion is unchanged: the long take needs the second group (#1150, + #1044, #1030) or the per-segment check and re-roll. +- **0.25 is better than 0.5 at length.** The numbers tie (coarse aside), but + by eye 0.25 keeps the face about 20 s longer and never loses the eyes. This + confirms the round-2 recommendation. +- **last_frame + look_hold + anchor 0.25 against guide + anchor 0.25: not + settled, and confounded.** It holds the face longer and ends lower on fine + texture. It is worse past the floor on mid and coarse texture and on the + worst seam (4.24 jump at 76 s, past the 4.0 check). It is also 21 min + faster (48 against 69 min). It opens on a drifted segment 1 (`91deb574`), so + none of these reads is clean, and the mid-texture cost repeats round 2's + on both seeds. The *Defaults by shot type* row (a knob, not the default) + stands. +- **Minimum clean rerun: one run.** Run `r3-lf-lookhold-anchor25-s1-120s` with + a fresh `workflow_id_suffix`, as the first job after a cold model load, and + confirm it `clean` (`d59a1259`) with `bench/segment1.py` before reading it. + This takes about 48 min plus the load. No rerun is needed for the anchor 0.25 + call. Both sides are clean, and every margin is far past the 6-segment + floor. A seed-2 two-minute pair would give the first two-minute floor if + one is wanted. +- **Cost:** anchor 0.25 took 69 min, against the control's 52 min (+32%). + Round 2's anchor 0.5 took 68 min. diff --git a/bench/records/determinism.json b/bench/records/determinism.json new file mode 100644 index 000000000..d4fa04eab --- /dev/null +++ b/bench/records/determinism.json @@ -0,0 +1,561 @@ +[ + { + "batch": "base", + "setup": "one card (cuda:0), defaults; SDNQ_DEBUG_TRITON_AUTOTUNE=1 TRITON_PRINT_AUTOTUNING=1", + "arm": "det-base-01-c1", + "job_id": "dffed107efa7", + "device": "cuda:0", + "wall_s": 345.9, + "frames_video": 226, + "frames_audio": 295, + "segment1": "d59a1259/b2c2c63e", + "segment2": "5c299afd/c6e90bd4", + "first_diff_vs_canonical_control": "None / None" + }, + { + "batch": "base", + "setup": "one card (cuda:0), defaults; SDNQ_DEBUG_TRITON_AUTOTUNE=1 TRITON_PRINT_AUTOTUNING=1", + "arm": "det-base-02-c2", + "job_id": "2f6a9dc818df", + "device": "cuda:0", + "wall_s": 195.6, + "frames_video": 226, + "frames_audio": 295, + "segment1": "d59a1259/b2c2c63e", + "segment2": "5c299afd/c6e90bd4", + "first_diff_vs_canonical_control": "None / None" + }, + { + "batch": "base", + "setup": "one card (cuda:0), defaults; SDNQ_DEBUG_TRITON_AUTOTUNE=1 TRITON_PRINT_AUTOTUNING=1", + "arm": "det-base-03-c3", + "job_id": "dbbc9e059e36", + "device": "cuda:0", + "wall_s": 195.4, + "frames_video": 226, + "frames_audio": 295, + "segment1": "d59a1259/b2c2c63e", + "segment2": "5c299afd/c6e90bd4", + "first_diff_vs_canonical_control": "None / None" + }, + { + "batch": "base", + "setup": "one card (cuda:0), defaults; SDNQ_DEBUG_TRITON_AUTOTUNE=1 TRITON_PRINT_AUTOTUNING=1", + "arm": "det-base-04-c4", + "job_id": "250e23e2cc9c", + "device": "cuda:0", + "wall_s": 195.5, + "frames_video": 226, + "frames_audio": 295, + "segment1": "d59a1259/b2c2c63e", + "segment2": "5c299afd/c6e90bd4", + "first_diff_vs_canonical_control": "None / None" + }, + { + "batch": "base", + "setup": "one card (cuda:0), defaults; SDNQ_DEBUG_TRITON_AUTOTUNE=1 TRITON_PRINT_AUTOTUNING=1", + "arm": "det-base-05-c5", + "job_id": "0f8322609d65", + "device": "cuda:0", + "wall_s": 195.4, + "frames_video": 226, + "frames_audio": 295, + "segment1": "d59a1259/b2c2c63e", + "segment2": "5c299afd/c6e90bd4", + "first_diff_vs_canonical_control": "None / None" + }, + { + "batch": "base", + "setup": "one card (cuda:0), defaults; SDNQ_DEBUG_TRITON_AUTOTUNE=1 TRITON_PRINT_AUTOTUNING=1", + "arm": "det-base-06-a751", + "job_id": "0de6225dc377", + "device": "cuda:0", + "wall_s": 790.9, + "frames_video": 634, + "frames_audio": 826, + "segment1": "d59a1259/b2c2c63e", + "segment2": null, + "first_diff_vs_canonical_control": "(6-seg arm)" + }, + { + "batch": "base", + "setup": "one card (cuda:0), defaults; SDNQ_DEBUG_TRITON_AUTOTUNE=1 TRITON_PRINT_AUTOTUNING=1", + "arm": "det-base-07-c6", + "job_id": "4ac9f71b691a", + "device": "cuda:0", + "wall_s": 195.6, + "frames_video": 226, + "frames_audio": 295, + "segment1": "d59a1259/b2c2c63e", + "segment2": "5c299afd/c6e90bd4", + "first_diff_vs_canonical_control": "None / None" + }, + { + "batch": "base", + "setup": "one card (cuda:0), defaults; SDNQ_DEBUG_TRITON_AUTOTUNE=1 TRITON_PRINT_AUTOTUNING=1", + "arm": "det-base-08-bo171", + "job_id": "9ba8be23ec1d", + "device": "cuda:0", + "wall_s": 630.2, + "frames_video": 549, + "frames_audio": 715, + "segment1": "d59a1259/b2c2c63e", + "segment2": null, + "first_diff_vs_canonical_control": "(6-seg arm)" + }, + { + "batch": "base", + "setup": "one card (cuda:0), defaults; SDNQ_DEBUG_TRITON_AUTOTUNE=1 TRITON_PRINT_AUTOTUNING=1", + "arm": "det-base-09-c7", + "job_id": "dcb0c42ec5ef", + "device": "cuda:0", + "wall_s": 195.5, + "frames_video": 226, + "frames_audio": 295, + "segment1": "d59a1259/b2c2c63e", + "segment2": "5c299afd/c6e90bd4", + "first_diff_vs_canonical_control": "None / None" + }, + { + "batch": "base", + "setup": "one card (cuda:0), defaults; SDNQ_DEBUG_TRITON_AUTOTUNE=1 TRITON_PRINT_AUTOTUNING=1", + "arm": "det-base-10-ks1", + "job_id": "ea13bea33095", + "device": "cuda:0", + "wall_s": 630.8, + "frames_video": 634, + "frames_audio": 826, + "segment1": "d59a1259/b2c2c63e", + "segment2": null, + "first_diff_vs_canonical_control": "(6-seg arm)" + }, + { + "batch": "base", + "setup": "one card (cuda:0), defaults; SDNQ_DEBUG_TRITON_AUTOTUNE=1 TRITON_PRINT_AUTOTUNING=1", + "arm": "det-base-11-c8", + "job_id": "de082dade5df", + "device": "cuda:0", + "wall_s": 195.6, + "frames_video": 226, + "frames_audio": 295, + "segment1": "d59a1259/b2c2c63e", + "segment2": "5c299afd/c6e90bd4", + "first_diff_vs_canonical_control": "None / None" + }, + { + "batch": "long", + "setup": "one card (cuda:0), defaults", + "arm": "det-long-01-c1", + "job_id": "be588a8d846e", + "device": "cuda:0", + "wall_s": 307.2, + "frames_video": 226, + "frames_audio": 295, + "segment1": "d59a1259/b2c2c63e", + "segment2": "5c299afd/c6e90bd4", + "first_diff_vs_canonical_control": "None / None" + }, + { + "batch": "long", + "setup": "one card (cuda:0), defaults", + "arm": "det-long-02-L1", + "job_id": "6bc337a426e1", + "device": "cuda:0", + "wall_s": 4041.0, + "frames_video": 2880, + "frames_audio": 3750, + "segment1": "d59a1259/b2c2c63e", + "segment2": null, + "first_diff_vs_canonical_control": "(6-seg arm)" + }, + { + "batch": "long", + "setup": "one card (cuda:0), defaults", + "arm": "det-long-03-c2", + "job_id": "1650fbc82433", + "device": "cuda:0", + "wall_s": 196.1, + "frames_video": 226, + "frames_audio": 295, + "segment1": "d59a1259/b2c2c63e", + "segment2": "5c299afd/c6e90bd4", + "first_diff_vs_canonical_control": "None / None" + }, + { + "batch": "long", + "setup": "one card (cuda:0), defaults", + "arm": "det-long-04-c3", + "job_id": "ca9fcdc022ac", + "device": "cuda:0", + "wall_s": 195.7, + "frames_video": 226, + "frames_audio": 295, + "segment1": "d59a1259/b2c2c63e", + "segment2": "5c299afd/c6e90bd4", + "first_diff_vs_canonical_control": "None / None" + }, + { + "batch": "dual", + "setup": "two cards (cuda:0,cuda:1), defaults", + "arm": "det-dual-01-c1", + "job_id": "70f6cb3e87e9", + "device": "cuda:0", + "wall_s": 306.3, + "frames_video": 226, + "frames_audio": 295, + "segment1": "d59a1259/b2c2c63e", + "segment2": "5c299afd/c6e90bd4", + "first_diff_vs_canonical_control": "None / None" + }, + { + "batch": "dual", + "setup": "two cards (cuda:0,cuda:1), defaults", + "arm": "det-dual-02-a751", + "job_id": "8ea62d93971f", + "device": "cuda:1", + "wall_s": 931.1, + "frames_video": 634, + "frames_audio": 826, + "segment1": "d59a1259/b2c2c63e", + "segment2": null, + "first_diff_vs_canonical_control": "(6-seg arm)" + }, + { + "batch": "dual", + "setup": "two cards (cuda:0,cuda:1), defaults", + "arm": "det-dual-03-c2", + "job_id": "6913b2a2a3a1", + "device": "cuda:0", + "wall_s": 199.6, + "frames_video": 226, + "frames_audio": 295, + "segment1": "0a81547e/5f61a5f5", + "segment2": "4eb38d29/29cdd6df", + "first_diff_vs_canonical_control": "0 / 0" + }, + { + "batch": "dual", + "setup": "two cards (cuda:0,cuda:1), defaults", + "arm": "det-dual-04-bo171", + "job_id": "a067460bb580", + "device": "cuda:0", + "wall_s": 630.3, + "frames_video": 549, + "frames_audio": 715, + "segment1": "0a81547e/5f61a5f5", + "segment2": null, + "first_diff_vs_canonical_control": "(6-seg arm)" + }, + { + "batch": "dual", + "setup": "two cards (cuda:0,cuda:1), defaults", + "arm": "det-dual-05-c3", + "job_id": "020ee83fd2be", + "device": "cuda:0", + "wall_s": 195.5, + "frames_video": 226, + "frames_audio": 295, + "segment1": "0a81547e/5f61a5f5", + "segment2": "4eb38d29/29cdd6df", + "first_diff_vs_canonical_control": "0 / 0" + }, + { + "batch": "dual", + "setup": "two cards (cuda:0,cuda:1), defaults", + "arm": "det-dual-06-ks1", + "job_id": "63dc35d4ca38", + "device": "cuda:0", + "wall_s": 630.8, + "frames_video": 634, + "frames_audio": 826, + "segment1": "0a81547e/5f61a5f5", + "segment2": null, + "first_diff_vs_canonical_control": "(6-seg arm)" + }, + { + "batch": "dual", + "setup": "two cards (cuda:0,cuda:1), defaults", + "arm": "det-dual-07-c4", + "job_id": "63c5bb0bc8ef", + "device": "cuda:0", + "wall_s": 195.6, + "frames_video": 226, + "frames_audio": 295, + "segment1": "0a81547e/5f61a5f5", + "segment2": "4eb38d29/29cdd6df", + "first_diff_vs_canonical_control": "0 / 0" + }, + { + "batch": "dual", + "setup": "two cards (cuda:0,cuda:1), defaults", + "arm": "det-dual-08-a752", + "job_id": "af0685233af3", + "device": "cuda:0", + "wall_s": 791.0, + "frames_video": 634, + "frames_audio": 826, + "segment1": "0a81547e/5f61a5f5", + "segment2": null, + "first_diff_vs_canonical_control": "(6-seg arm)" + }, + { + "batch": "dual", + "setup": "two cards (cuda:0,cuda:1), defaults", + "arm": "det-dual-09-c5", + "job_id": "89d5442d5f05", + "device": "cuda:0", + "wall_s": 195.6, + "frames_video": 226, + "frames_audio": 295, + "segment1": "0a81547e/5f61a5f5", + "segment2": "4eb38d29/29cdd6df", + "first_diff_vs_canonical_control": "0 / 0" + }, + { + "batch": "dual", + "setup": "two cards (cuda:0,cuda:1), defaults", + "arm": "det-dual-10-c6", + "job_id": "7582ae21237a", + "device": "cuda:0", + "wall_s": 195.5, + "frames_video": 226, + "frames_audio": 295, + "segment1": "0a81547e/5f61a5f5", + "segment2": "4eb38d29/29cdd6df", + "first_diff_vs_canonical_control": "0 / 0" + }, + { + "batch": "nostream", + "setup": "two cards, transformer group_offload use_stream/record_stream/low_cpu_mem_usage false", + "arm": "det-nostream-01-c1", + "job_id": "8654809bd164", + "device": "cuda:0", + "wall_s": 531.7, + "frames_video": 226, + "frames_audio": 295, + "segment1": "d59a1259/b2c2c63e", + "segment2": "8f559b38/eeda7883", + "first_diff_vs_canonical_control": "86 / 160" + }, + { + "batch": "nostream", + "setup": "two cards, transformer group_offload use_stream/record_stream/low_cpu_mem_usage false", + "arm": "det-nostream-02-a751", + "job_id": "9fa08dd91eb8", + "device": "cuda:1", + "wall_s": 1200.7, + "frames_video": 634, + "frames_audio": 826, + "segment1": "ff630564/5dfbe0da", + "segment2": null, + "first_diff_vs_canonical_control": "(6-seg arm)" + }, + { + "batch": "nostream", + "setup": "two cards, transformer group_offload use_stream/record_stream/low_cpu_mem_usage false", + "arm": "det-nostream-03-c2", + "job_id": "0f534c61a027", + "device": "cuda:0", + "wall_s": 310.7, + "frames_video": 226, + "frames_audio": 295, + "segment1": "b56226bd/ffb5374b", + "segment2": "3a341185/3b85b5f9", + "first_diff_vs_canonical_control": "0 / 0" + }, + { + "batch": "nostream", + "setup": "two cards, transformer group_offload use_stream/record_stream/low_cpu_mem_usage false", + "arm": "det-nostream-04-c3", + "job_id": "73b70e140d4e", + "device": "cuda:0", + "wall_s": 310.6, + "frames_video": 226, + "frames_audio": 295, + "segment1": "b56226bd/ffb5374b", + "segment2": "3a341185/3b85b5f9", + "first_diff_vs_canonical_control": "0 / 0" + }, + { + "batch": "cudnn", + "setup": "two cards, settings.json cudnn_benchmark false + cudnn_deterministic true", + "arm": "det-cudnn-01-c1", + "job_id": "4bb77f836864", + "device": "cuda:0", + "wall_s": 314.6, + "frames_video": 226, + "frames_audio": 295, + "segment1": "d59a1259/b2c2c63e", + "segment2": "5c299afd/c6e90bd4", + "first_diff_vs_canonical_control": "None / None" + }, + { + "batch": "cudnn", + "setup": "two cards, settings.json cudnn_benchmark false + cudnn_deterministic true", + "arm": "det-cudnn-02-a751", + "job_id": "13d680942bd3", + "device": "cuda:1", + "wall_s": 920.5, + "frames_video": 634, + "frames_audio": 826, + "segment1": "d59a1259/b2c2c63e", + "segment2": null, + "first_diff_vs_canonical_control": "(6-seg arm)" + }, + { + "batch": "cudnn", + "setup": "two cards, settings.json cudnn_benchmark false + cudnn_deterministic true", + "arm": "det-cudnn-03-c2", + "job_id": "90777e25ab15", + "device": "cuda:0", + "wall_s": 202.0, + "frames_video": 226, + "frames_audio": 295, + "segment1": "d59a1259/b2c2c63e", + "segment2": "5c299afd/c6e90bd4", + "first_diff_vs_canonical_control": "None / None" + }, + { + "batch": "cudnn", + "setup": "two cards, settings.json cudnn_benchmark false + cudnn_deterministic true", + "arm": "det-cudnn-04-c3", + "job_id": "f8aae4c2ec50", + "device": "cuda:0", + "wall_s": 195.1, + "frames_video": 226, + "frames_audio": 295, + "segment1": "d59a1259/b2c2c63e", + "segment2": "5c299afd/c6e90bd4", + "first_diff_vs_canonical_control": "None / None" + }, + { + "batch": "cudnn2", + "setup": "two cards, settings.json cudnn_benchmark false + cudnn_deterministic true", + "arm": "det-cudnn2-01-c1", + "job_id": "b467fc322090", + "device": "cuda:0", + "wall_s": 310.7, + "frames_video": 226, + "frames_audio": 295, + "segment1": "d59a1259/b2c2c63e", + "segment2": "5c299afd/c6e90bd4", + "first_diff_vs_canonical_control": "None / None" + }, + { + "batch": "cudnn2", + "setup": "two cards, settings.json cudnn_benchmark false + cudnn_deterministic true", + "arm": "det-cudnn2-02-a751", + "job_id": "e2a0487c62c2", + "device": "cuda:1", + "wall_s": 908.5, + "frames_video": 634, + "frames_audio": 826, + "segment1": "d59a1259/b2c2c63e", + "segment2": null, + "first_diff_vs_canonical_control": "(6-seg arm)" + }, + { + "batch": "cudnn2", + "setup": "two cards, settings.json cudnn_benchmark false + cudnn_deterministic true", + "arm": "det-cudnn2-03-c2", + "job_id": "1d9356ab7bf1", + "device": "cuda:0", + "wall_s": 218.7, + "frames_video": 226, + "frames_audio": 295, + "segment1": "d59a1259/b2c2c63e", + "segment2": "5c299afd/c6e90bd4", + "first_diff_vs_canonical_control": "None / None" + }, + { + "batch": "cudnn2", + "setup": "two cards, settings.json cudnn_benchmark false + cudnn_deterministic true", + "arm": "det-cudnn2-04-bo171", + "job_id": "5250c077441c", + "device": "cuda:0", + "wall_s": 629.5, + "frames_video": 549, + "frames_audio": 715, + "segment1": "d59a1259/b2c2c63e", + "segment2": null, + "first_diff_vs_canonical_control": "(6-seg arm)" + }, + { + "batch": "cudnn2", + "setup": "two cards, settings.json cudnn_benchmark false + cudnn_deterministic true", + "arm": "det-cudnn2-05-c3", + "job_id": "65de994b65c5", + "device": "cuda:0", + "wall_s": 195.3, + "frames_video": 226, + "frames_audio": 295, + "segment1": "d59a1259/b2c2c63e", + "segment2": "5c299afd/c6e90bd4", + "first_diff_vs_canonical_control": "None / None" + }, + { + "batch": "cudnn2", + "setup": "two cards, settings.json cudnn_benchmark false + cudnn_deterministic true", + "arm": "det-cudnn2-06-ks1", + "job_id": "d6a40a32b801", + "device": "cuda:0", + "wall_s": 630.0, + "frames_video": 634, + "frames_audio": 826, + "segment1": "d59a1259/b2c2c63e", + "segment2": null, + "first_diff_vs_canonical_control": "(6-seg arm)" + }, + { + "batch": "cudnn2", + "setup": "two cards, settings.json cudnn_benchmark false + cudnn_deterministic true", + "arm": "det-cudnn2-07-c4", + "job_id": "8baec9e1feb4", + "device": "cuda:0", + "wall_s": 195.4, + "frames_video": 226, + "frames_audio": 295, + "segment1": "d59a1259/b2c2c63e", + "segment2": "5c299afd/c6e90bd4", + "first_diff_vs_canonical_control": "None / None" + }, + { + "batch": "cudnn2", + "setup": "two cards, settings.json cudnn_benchmark false + cudnn_deterministic true", + "arm": "det-cudnn2-08-a752", + "job_id": "5ac13342ac9f", + "device": "cuda:0", + "wall_s": 790.0, + "frames_video": 634, + "frames_audio": 826, + "segment1": "d59a1259/b2c2c63e", + "segment2": null, + "first_diff_vs_canonical_control": "(6-seg arm)" + }, + { + "batch": "cudnn2", + "setup": "two cards, settings.json cudnn_benchmark false + cudnn_deterministic true", + "arm": "det-cudnn2-09-c5", + "job_id": "b4b2a8c8a42c", + "device": "cuda:0", + "wall_s": 195.3, + "frames_video": 226, + "frames_audio": 295, + "segment1": "d59a1259/b2c2c63e", + "segment2": "5c299afd/c6e90bd4", + "first_diff_vs_canonical_control": "None / None" + }, + { + "batch": "cudnn2", + "setup": "two cards, settings.json cudnn_benchmark false + cudnn_deterministic true", + "arm": "det-cudnn2-10-c6", + "job_id": "b2b3922a924d", + "device": "cuda:0", + "wall_s": 195.2, + "frames_video": 226, + "frames_audio": 295, + "segment1": "d59a1259/b2c2c63e", + "segment2": "5c299afd/c6e90bd4", + "first_diff_vs_canonical_control": "None / None" + } +] \ No newline at end of file diff --git a/bench/records/determinism.md b/bench/records/determinism.md new file mode 100644 index 000000000..afffecb5a --- /dev/null +++ b/bench/records/determinism.md @@ -0,0 +1,155 @@ +# Determinism check (2026-11-05) + +> **Correction (2026-11-07, #1229).** The served worker did not read +> `enable_tf32`, `cudnn_benchmark` or `cudnn_deterministic` when these +> batches ran. Only `startup()` applied them, and `startup()` runs in the +> `dw.serve` parent, which runs no pipeline; each worker is a spawned process +> that never called it. So served jobs ran on PyTorch's own defaults +> (benchmark off, deterministic off, TF32 off) whatever `settings.json` said, +> unless something the worker imports set them, and the `cudnn` and `cudnn2` +> batches below cannot have got their result through those keys as this +> record says (*Answer*, *The suspects*). From #1229 the +> worker applies the keys itself, at start and at each job start, and the +> defaults are `cudnn_benchmark: false`, `cudnn_deterministic: true`, +> `enable_tf32: false`. The first job on a fresh worker narrates the flags it +> actually ran with; what lem reports is recorded on #1238. On 2026-11-07 +> both cards' fresh workers reported `benchmark=false deterministic=true, +> tf32=false` after SD 1.5 and LTX-2.5 loads, so nothing those pipelines +> import sets the flags (`docs/proposals/complete/torch-settings-complete.md`). +> The record below stays as written. + +The question from `.superpowers/sdd/2026-11-03-long-form-media/nondeterminism-report.md`: +why do same-seed, same-arguments runs on the bench server sometimes give a +different output? Records: `bench/records/determinism.json` (one row per job: +batch, setup, card, wall seconds, segment-1 and segment-2 hashes). + +## Answer + +- **Cause: cuDNN autotuning (`cudnn_benchmark: true`, the dw default).** + cuDNN times its candidate algorithms and keeps the fastest. When the other + card's worker is loading or running a job at the same time, the timings + change, cuDNN can pick a different algorithm, and the numbers change with it. + The picks are cached in the worker process, so the output then repeats bit + for bit from job to job (the "persistent alternate" classes) until the + process starts again. +- **The fix: `cudnn_benchmark: false` and `cudnn_deterministic: true`** in + `~/.diffusers_helper/settings.json` (docs/ACCELERATION.md, *TF32 and + cuDNN*). Both batches with this setting stayed on the canonical output, in + the same job order on two cards that drifted with the default. +- **The cost: none that can be measured here.** A warm 2-segment control + takes 195.2-195.4 s against 195.4-195.6 s with the default. The 6-segment + arms take 629.5 s (backoff), 630.0 s (keystone) and 790.0 s (anchor 0.75), + against 630.3, 630.8 and 791.0 s. +- **The trigger reproduced here is a second worker, not the job history on + the first** (other host load fits the earlier cases; see *The earlier + drifts*). On + one card, 16 jobs in a row gave the canonical output (11 jobs including + anchor 0.75, backoff and keystone arms, then a 2-minute anchor 0.25 run). + With two cards, the first control after the cuda:1 worker loaded the model + moved off canonical from frame 0. + +## Method + +- **The probe.** The S1 guide control at `segments: 2`, seed 1166 (226 video + frames, 295 AAC frames). Every job has a fresh workflow-id suffix, so the + step cache never serves one. The 6-segment arms are round 1's + `r1-anchor75-s1`, `r1-backoff17-s1` and `r1-keystone-s1`. The 2-minute run + is `r3-anchor25-s1-120s` (`duration_seconds: 120`, anchor 0.25). +- **The hashes.** + - Segment 1 is `bench/segment1.py`'s fingerprint (video frames 0-59, AAC + frames 0-129), taken with lem's ffmpeg 7.0.2 (the imageio-ffmpeg binary + in the venv). The 2-segment control's segment 1 equals the 6-segment + canonical `d59a1259/b2c2c63e`. + - Segment 2 is the md5 of the decoded-frame framemd5 hashes from video + frame 124 and AAC frame 166 (5.17 s, the seam) to the end. Canonical is + `5c299afd/c6e90bd4`. + - "First diff" is the first decoded video / AAC frame that differs from the + canonical control. +- **Each batch starts on a fresh bench server** (`bench_server.sh stop`, then + `start`), so every batch begins with a cold worker. The driver submits all + of a batch's arms at once, and the server places each one on a free card. + +## Results + +| batch | setup | jobs | segment 1 | segment 2 | result | +|---|---|---|---|---|---| +| base | one card; defaults; SDNQ and Triton autotune logging on | c x5, anchor 0.75, c, backoff 17, c, keystone, c | all `d59a1259/b2c2c63e` | all 8 controls `5c299afd/c6e90bd4` | **reproducible** | +| long | one card; defaults | c, 2-minute anchor 0.25 (67 min), c, c | all canonical | all canonical | **reproducible** | +| dual | **two cards**; defaults | c (cuda:0, cold); anchor 0.75 (**cuda:1, cold load**); then c, backoff, c, keystone, c, anchor 0.75, c, c (all cuda:0) | c1 and the cuda:1 anchor run canonical; **every cuda:0 job after the cuda:1 load is `0a81547e/5f61a5f5`** | the 6 later controls are `4eb38d29/29cdd6df`, first diff at frame 0 / 0 | **drift reproduced**: it repeats bit for bit after it starts | +| nostream | two cards; transformer group_offload `use_stream`, `record_stream`, `low_cpu_mem_usage` all false | c (cuda:0) and anchor 0.75 (cuda:1) **started together**; then c, c | c1 canonical; **the anchor 0.75 cold load is `ff630564/5dfbe0da`**; the later controls are `b56226bd/ffb5374b` | c1 departs at segment 2 (video 86, audio 160, the x264 lookahead before the seam at 124); the later controls differ from frame 0 | **drift persists**, so stream offload is not the cause | +| cudnn | two cards; `cudnn_benchmark: false`, `cudnn_deterministic: true` | c, anchor 0.75 (cuda:1, cold), c, c | all canonical | all canonical | **reproducible** | +| cudnn2 | two cards; `cudnn_benchmark: false`, `cudnn_deterministic: true` | the dual batch's 10 jobs, placed on the same cards | all canonical | all 6 controls canonical | **reproducible** | + +Both kinds of departure in the nondeterminism report showed up, and both had +another worker active at the time: + +- The persistent drift in dual: the cuda:0 worker was idle while cuda:1 loaded + and ran. The next cuda:0 job is about 4 s slower (199.6 s against 195.5 s), + then runs at full speed and repeats its new output. +- In nostream, a control departed at segment 2 when the cuda:1 job started up + next to it, and a cold load drifted from frame 0. A cold load is therefore + not always canonical: it is canonical when nothing else runs while it + loads and tunes. + +The canonical output is the deterministic-algorithm output. With +`cudnn_deterministic: true`, every job reproduces the canonical hashes the +default gives on a quiet host. So on a quiet machine, the autotuner picks the +same algorithms the deterministic setting forces. + +## The suspects + +| suspect | how it is set | result | +|---|---|---| +| cuDNN benchmark (on) and cudnn_deterministic (off) | `~/.diffusers_helper/settings.json` keys, read once at worker start (`dw/settings.py`, `dw/__init__.py`) | **cause.** Turning both over gives canonical output under the same two-card order that drifts with the default | +| stream group offload | workflow: `components.transformer.group_offload.{use_stream, record_stream, low_cpu_mem_usage}` | **ruled out.** Drift persists with all three false, and turning them off costs **+59%** per warm control (310.7 s against 195.5 s) | +| SDNQ / Triton autotune | `cache_results=True`; `SDNQ_DEBUG_TRITON_AUTOTUNE=1` and `TRITON_PRINT_AUTOTUNING=1` in the worker environment | **ruled out for every drift after 2026-11-04 00:15 UTC.** The last autotune entry in `~/.triton/cache` was written then. The logged base batch tuned nothing, because every key was a cache hit, so each config is fixed per key across processes | +| sparse attention (`attention_backend: sparse`) | workflow variable | not run: drift appears and disappears with the cuDNN setting while sparse stays on | +| `CUBLAS_WORKSPACE_CONFIG=:4096:8` | worker environment | not run, for the same reason | +| the job history on one worker (anchor, backoff, keystone, a 2-minute run) | | **ruled out.** 16 jobs on one card are all canonical | +| the cross-worker channel | | each worker sees only its own card (`CUDA_VISIBLE_DEVICES=0` or `1`), and nothing was written to disk while the drift began. What the two workers share is the host: CPU, PCIe, RAM and swap (4 GB of swap was in use after the dual batch). That changes timing, and timing is what the autotuner reads | + +## Recommendation + +- **Bench default: yes, now.** Set + `{"devices": "cuda:0,cuda:1", "cudnn_benchmark": false, "cudnn_deterministic": true}` + in `~/.diffusers_helper/settings.json` on lem, then restart the bench + server. This file is shared with the main dw-serve. It was restored to + `{"devices": "cuda:0,cuda:1"}` after these batches, because changing it + changes the main server too. With the setting, two-card bench runs can be + compared without segment-1 labels, and R35's rerun gap should collapse to + zero for same-seed pairs. A canary control at the start and end of a batch + is the cheap check that it held. +- **Engine default: recommend it, but measure first.** For H3 it costs + nothing measurable: its convolutions are in the VAEs, and the transformer + is matmul and attention. The default also has a real failure mode: any + multi-card `dw.serve` gives timing-dependent output, and so does a single + card next to other GPU work. Before flipping the default in + `dw/settings.py`, time one conv-heavy family (an SD 1.5 or SDXL UNet, and + the LTX VAE decode), where `benchmark` usually pays. If the cost is small + there too, `cudnn_benchmark: false` and `cudnn_deterministic: true` should + be the default, and speed becomes the opt-in. +- **What R35 still needs.** Records made before this setting keep their + `clean` / `drifted` labels. + +## The earlier drifts + +The job list on lem shows when each earlier departure happened and what else +was running: + +- **`cheap-a-s1` (cuda:1, 00:30-00:41 on 2026-11-04)** departed at + segment 2 while cuda:0 ran `noise-b-s4`. That is the nostream case. +- **The round-2 drifts on cuda:1 (from 15:52)** and **round 3's second + 2-minute run** followed long jobs. Nothing ran on the other card at those + times. +- **The first round-1 S1 drift (cuda:0, 06:39)** and the + **`ctl-guide-s1-s1166-rerun` segment-2 departure (12:28)** had no job on + the other card either. + +All of those rounds ran the driver with `--measure`. That measure, the +server's `assess` and `compare` routes, decodes the finished output in the +server process while the next job starts and tunes. These determinism batches +ran without `--measure`. So "a second worker" is the trigger this check +reproduced, but "any other load on the host while a worker meets a new cuDNN +key" fits every case. Each case is consistent with the cause; only the +two-card one was rerun. The fix does not depend on which load it is, because +with the autotuner off, timing no longer selects algorithms. diff --git a/bench/records/drill.md b/bench/records/drill.md new file mode 100644 index 000000000..7ba64d5a2 --- /dev/null +++ b/bench/records/drill.md @@ -0,0 +1,230 @@ +# Cold-session drill (Task 28) + +2026-11-06. Bench server lem:8766 at `6d923095`, MCP mounted at `/mcp`, both +cards (cuda:0, cuda:1). Each ask ran in a fresh headless Claude Code session +(`claude -p --model opus`) in its own empty directory (`~/testing/lf-drill-`), +with only the dw MCP (`--strict-mcp-config`) and the plugin from this branch +(`--plugin-dir plugins/dw`). Turn 1 was the ask, verbatim. Turn 2 was "Looks +good, go ahead and run it. Let me know if anything needs a retry." A later +"Check on the job and finish up." let the session judge and finish. +Transcripts (stream-json) are in the drill agent's scratchpad, not the repo. + +**Setup notes.** +- lem's firewall drops LAN connections to 8766 (and 8765) from this Mac; + only ssh gets through. The sessions reached the server through an ssh + tunnel (`-L 18766:127.0.0.1:8766`), and a cold agent's own `curl` to + `lem:8766` timed out (ask 3, when it tried to download the cut). +- The three first sessions shared the `default` workspace with each other. + The planning reruns after the fixes ran in their own empty workspaces + (`lfdrill1c`, `lfdrill1d`, `lfdrill3c`) through the stdio `dw_mcp` client, + because the first rerun of ask 1 and ask 3 in `default` found the earlier + session's jobs and reused them. +- The user-level Mac skills (`h3-prompt-writing`, `music-caption-rewriter`) + and Bash were available to the cold sessions, as they are in Don's own + drills. + +## Scorecard + +| | Ask 1: 90 s walk-and-talk | Ask 2: 3 min, four scenes, score | Ask 3: 5 min narration over boards | +| --- | --- | --- | --- | +| Read `long-form` first | yes, first call | yes, first call | **no**: went to `storyboard`, then its `references/animatic.md` | +| Shape per scene | chain, `shot: dialogue` (what the playbook says for an unbroken take) | cuts (`dialogue-short`), the playbook's "prefer cuts" | boards + per-board takes + bed + `templates/animatic` | +| Cost quoted with retries | yes: 210 min, worst case about 6.5 h with `retries: 1` | no retries quoted; offered a draft first | guesses only; no retry budget | +| Tier dial | no (the playbook says the tiers are *pending measurement*) | improvised a draft (portraits + 2-3 shots) | no | +| Flagged segment | none: the check passed all 7 segments, and the take drifted anyway | not a chain; the short score and a doubled keeper in shot 4 were caught by eye and metadata and re-rendered alone | take 5 truncated, caught by length and transcript, re-spoken in halves | + +## Ask 1: "Make me a 90-second single take of two friends on a walk-and-talk ..." + +**Call order (turn 1).** `Skill dw:long-form` → `Skill dw:minimax-h3` → +`get_server_info` → `list_workflows(traits="chained")` → +`get_workflow(templates/minimax/chained-segments)` → `h3-prompt-writing` + +the H3 dialogue and chains references → `dw:flux3-klein` → +`validate_workflow` + **`run_workflow`** of the opening frame (FLUX 3 Klein, +`077795ecc576`, 0.4 min, run before asking) → `validate_workflow` on the +chain → `save_workflow(walk-and-talk/chained-dialogue-checked)` with +`check: true, retries: 1` → `validate_workflow` on the saved copy. + +**What it planned.** Seven 345-frame segments, `duration_seconds: 90`, +`shot: dialogue`, `keep_segments: true`, turbo LoRA off at 12 steps (the H3 +skill's voice rule), the check on. 26 scripted lines in `` markup. + +**Cost.** The template's `plan.estimate` was 210.3 min (`derived`, low +confidence, "scaled from the curated 13.5 min: steps 12 vs 5, frames 345 vs +124, segments 7 vs 3"). The saved checked copy came back `basis: unknown`. +The agent scaled the template figure by hand to about 6.5 h worst case. +**The run took 71.0 min** (`ab7f54f73b2f`): the scaled estimate was about 3x +high for this shape. + +**Outcome.** All seven segments passed the check on the first attempt. There +were no warnings. `check-script` (`ad340963c34e`) heard all 26 lines (each +similarity at least 0.6), with one `line_clipped_at_end` at 27.8 s, seam 2. +By eye (frames at 2, 20, 35, 50, 65 and 88 s): clean to about 35 s. From +about 45 s, the woman's face and hair change and the trees go bare. From 60 s +on, the picture is oversaturated and painterly, with speckle on the man's +coat. The coats, props and left/right staging hold all the way. The cold +agent's own judge pass (`analyze_seams`, `get_output_frames` at late +moments, `levers.md`) found the same thing and named it carry drift that a +re-roll cannot fix. It offered: a reset at about 28 s and 55 s, two admitted +cuts, or keeping the first 35 s. + +**Score.** It read the playbook first, picked the shape the playbook gives +an unbroken dialogue take, used the check and `keep_segments`, and judged +well. **The playbook misled it on one thing:** the 30 s rule was stated for +static takes only, so nothing told it that a 90 s dialogue take would not +hold. It quoted 3.5-6.5 h for a take that was unusable past 35 s. + +**Waste.** It ran the FLUX still before asking (cheap, but unasked). It +tried to find the server token in `~/.claude.json` to poll by itself, and +the auto-mode classifier blocked it. + +## Ask 2: "Make a 3-minute piece of four scenes - a lighthouse keeper's day ..." + +**Call order (turn 1).** `Skill dw:long-form` → `get_server_info` → +`list_workflows(shape=sequence|shot|audio)` → `Skill dw:minimax-h3` → +`get_workflow(dialogue-short)`, `get_workflow(assemble-and-score)` → +`dw:minimax-music3`, `h3-prompt-writing`, `music-caption-rewriter` → +`validate_workflow(dialogue-short)` with 16 shots of 277 frames → +`validate_workflow(templates/minimax/music)` with a 13-tag instrumental +body and `audio_duration` 280. + +**Shape.** Four scenes of about 45 s, 16 cut shots on `dialogue-short` +(portrait-referenced keeper and lighthouse, no dialogue, +`non_diegetic_music: N/A`), one score under the cut with `assemble-and-score`. +That is the playbook's *Cuts* row and "one bed" rule. + +**Cost.** Picture 147.5 min (`derived`), score 3.6 min (`catalog`, not +scaled by `audio_duration`), assembly 0.2 min. No retry budget. It offered +"render just the portraits and two or three shots (about 30 min)" first. + +**Run.** Score `353751db0aac` (3.5 min) came back **91 s for 280 s asked**. +The agent caught it from `get_gallery_metadata`, read the playbook's +`music-long` row and re-ran it as four 52 s sections (`6ea608439d09`, +8.9 min, 195 s joined; `slice_audio` warned that three sections were still +playing at their cut). Picture `41692e04da2e`: 130.9 min, against +147.5 quoted. On "finish up", the agent looked at every shot +(`get_output_frames`) and found shot 4 with the keeper doubled on both sides +of the lens. It re-rendered that one shot (`dd8c149d3314`, 10.1 min). Then it +read `assemble-and-score` but wrote its own inline cut, to drop 7 run-up +frames per shot. Two attempts failed at run time: +`2cc8e119e2bf` ("Too many iterations generated: 7.9e38", a cartesian +product of `previous_result` lists) and `620dc103b35c` ("'Image' object is +not subscriptable"). The third, `8e75f5a1a33d`, cut the film: 3:00.3 at +960x544, peak -3.4 dBFS, with up to 8 dB level steps at exterior/interior +cuts, left in on purpose. By eye (a frame every 15.5 s): the keeper and the +red-banded tower hold across all 16 shots, and the night shots are dark but +read. The agent reported that the score's movements land 4-11 s after their +scene changes. + +**Score.** It read the playbook first, picked cuts, and recovered the score +correctly. **The playbook misled it on the score:** "music longer than one +Music 3 track (300 s practical)" read as "a 180 s score is one track", and +a tag-only instrumental ends when its tags run out, not at the ceiling. + +## Ask 3: "Make a 5-minute narration over storyboard boards: ..." + +**Call order (turn 1).** `Skill dw:storyboard` → `animatic.md` (Bash `cat`) +→ `get_server_info` → `get_workflow(templates/animatic)`, +`get_workflow(templates/generate-speech)` → `list_workflows(image|audio|utility)` +→ `get_workflow(narration-from-takes)` → `list_assets`, `list_tasks`, +`get_task(crossfade_audio|mix_audio|generate_speech)` → +`get_workflow(templates/minimax/music)` → `get_guide(workflows, "Authoring a +workflow from an agent")` → `validate_workflow` of a hand-written +`CableNarration` (one Audio8 take per board, **no reference voice**), the +music template (**empty lyrics**, 300 s) and a hand-written `CableBoards`. + +**It never loaded `long-form`.** `storyboard` and `animatic.md` did not point +a long read at it. It found `narration-from-takes`, read "There is no default +voice: `reference_audio` ... required", would not clone `bench/narrator.wav` +because it did not know whose voice that was, and so re-wrote the template +without the reference. That made each take Audio8's un-referenced voice, +with nothing tying the takes together. `animatic.md` also told it +"instrumental if the lyrics are empty", which the run refused. + +**Run.** Boards `078e54d8f3c7` (1.6 min, 12 boards; the contact sheet is +on-style, no stray text). Narration `205ca638ed02` (5.5 min, 303.8 s). Take 5 +came back 12.5 s for 62 words, and the agent caught it by length, confirmed it +by transcript (`8d4140f455d0`), and re-spoke it in halves (`36b5a74093df`; +transcript `45bb96d77fa1` has every word). Music `b176507ac918` **failed: +`lyrics` must be a non-empty string** (validation had passed). The retry with +a 9-tag body (`d00994e81284`) came back **62 s for 300 s asked**, and the +agent looped it with 3 s crossfades. The first animatic attempt (the rerun +session's `f4ff0c77d018`) **failed: stills_to_video would hold 7290 frames +of 1344x768 (21.0 GiB), over the 4 GiB limit**. Validation had passed. The +original session read `get_task(stills_to_video)` and built the animatic at +4 fps: `a5408f2f87d8`, 5:18, every board within 1/8 s of its line. + +**Score.** No playbook read, so no shape from it. Per-board takes are a +reasonable shape for an animatic. Each take was 61-71 words, under the +111-word take limit. The voice, the empty lyrics and the frame budget were +all misled or missing in `storyboard`'s `animatic.md`. Cost was guessed +("not measured") with no retry budget. It caught and repaired its own +flagged take. + +## Fixes (this commit) + +| Where | Fix | Evidence | +| --- | --- | --- | +| `long-form/SKILL.md` §3 | A take of any shot type drifts past about 35 s. Say so and offer chains of at most 35 s joined by cuts before quoting a longer single take. | ask 1 | +| `long-form/SKILL.md` *Run and judge* | A saved copy has no cost row (`basis: unknown`). Validate the template with the same arguments and scale by the worst case. | ask 1 | +| `long-form/SKILL.md` §5 | The lyric, not the ceiling, sets a track's length. A score past about 90 s is `music-long` sections (91 s and 62 s back for 280 s and 300 s asked). | asks 2, 3 | +| `minimax-music3/SKILL.md` | The score bullet points a cut past about 90 s at `music-long`. | asks 2, 3 | +| `long-form/references/audio.md` | "There is no default voice" now reads as "no default reference". With no voice from the user, speak a 5-10 s Audio8 line, keep it, and clone every take from it (how `bench/narrator.wav` was made). Never speak takes without a reference. | ask 3 | +| `storyboard/references/animatic.md` | A read over about 40 s goes to `long-form`'s narration recipe, one take per board through `narration-from-takes`. Empty lyrics are refused, so use a tag-only body, and a bed past about 90 s is `music-long`. The 4 GiB frame budget is checked only at run time: about 58 s at 1344x768 and 24 fps, so lower `fps` for a long animatic. | ask 3 | + +## Reruns after the fixes (turn 1 only, cold, own workspace) + +- **Ask 1** (`lfdrill1c`, before the drift fix): validated the template + itself, quoted 200 min, and quoted the checked worst case as about 6 h from + the same figure. The cost fix holds. It still planned one 90 s chain. +- **Ask 1** (`lfdrill1d`, after the drift fix): `long-form` → + `get_server_info` → `list_workflows(traits="chained")`, then stopped to + advise. A 90 s single take will drift past about 35 s, so it recommended + three continuous 30 s takes joined by two hard cuts on conversation beats, + with one shared park bed. It offered the 90 s take only as "useful to see + the drift". It asked about the cast before validating, so it gave no + figure yet. +- **Ask 2** (`2b`, `default` workspace, after the music fix): `long-form` + first, then cuts on `shots-batch` (four batches of four, after validation + projected 270 GB of host RAM for one 16-shot batch), and **`music-long` + in four sections from the start**. It ordered the score and scene 1 first + as a look check. `music-long` has no estimate (`basis: unknown`), so the + agent guessed about 15 min. +- **Ask 3** (`lfdrill3c`, after the narration fixes): `storyboard` → + `animatic.md` → **`long-form/references/audio.md`** → a 6 s synthetic Audio8 + reference line, then 10 takes of 75-87 words cloned from it through + `narration-from-takes`, a `music-long` bed in four 85 s sections, and the + animatic. Every point the first session got wrong was right in the plan. + It did not yet budget retries. + +## For Don + +- **Validation passes two run-time refusals:** `templates/minimax/music` + with `lyrics: ""`, and `templates/animatic` (`stills_to_video`) over the + 4 GiB frame budget. When the boards are assets, their size can be read at + validation. Each refusal cost a queued slot and a retry. +- **Estimates:** the scaled H3 chain estimate was 210 min against 71 min + real (`ab7f54f73b2f`, 12 steps, 345-frame segments), and `music-long` has + no cost row. `templates/minimax/music` quotes 3.6 min whatever the + `audio_duration`. +- **The tier dial is not usable cold.** No session used it, because + `quality-and-speed.md` says the tiers are *pending measurement* (Tasks + 19-20). Two sessions improvised a draft pass instead. Worth filling once + the speed tiers land. +- **Bench firewall:** lem drops LAN traffic to 8766 from the Mac, so a + session on the Mac cannot reach the server except over an ssh tunnel. +- **Voice without a reference** (ask 3's first animatic) has not been + checked for drift between takes. The ECAPA probe in `bench/records/audio.md` + could measure it on `205ca638ed02`. +- **Hand-rolled cuts fail at run time.** Ask 2's agent wrote its own cut + instead of `assemble-and-score`, to trim H3 run-up frames, and two of its + attempts failed at run time after validation passed: a `previous_result` + cartesian product, and `'Image' object is not subscriptable` (`620dc103b35c`). + The second looks like an engine error message worth a look. + +## Job ids + +| Ask | Jobs | +| --- | --- | +| 1 | `077795ecc576` (still), `ab7f54f73b2f` (90 s chain, 71.0 min), `ad340963c34e` (check-script) | +| 2 | `353751db0aac` (91 s score), `6ea608439d09` (music-long, 195 s), `41692e04da2e` (16 shots, 130.9 min), `dd8c149d3314` (shot 4 retry), `2cc8e119e2bf` / `620dc103b35c` (failed cuts), `8e75f5a1a33d` (final cut, 3:00.3) | +| 3 | `078e54d8f3c7` (boards), `205ca638ed02` (narration), `8d4140f455d0` / `45bb96d77fa1` (transcripts), `36b5a74093df` (take 5 retake), `b176507ac918` (music, failed), `d00994e81284` (62 s music), `f4ff0c77d018` (rerun session's animatic, failed), `a5408f2f87d8` (animatic, 5:18) | diff --git a/bench/records/for-don.md b/bench/records/for-don.md new file mode 100644 index 000000000..e6e618f23 --- /dev/null +++ b/bench/records/for-don.md @@ -0,0 +1,103 @@ +# For Don + +Things only you can check. + +**Start here:** `~/Desktop/dw-review/REVIEW.md` (on Don's Mac) is the step-by-step for every open item below, with every file copied local into numbered folders. + +**Answered by Don (2026-11-06), REVIEW.md steps 1-6:** +1. Pose loop: **not a loop.** Slot "last" is the static long-take default, as `shot: static_long` on `templates/minimax/chained-segments` (43a0fb87, d555a373). +2. LTX: **cap ok** (about 15-20 s, then cut to a fresh still); **look_hold on** (no answer given; Don followed the recommendation, c8fe45bb). +3. Audio pass: **A pass** ("very good lip sync"), shipped as `templates/minimax/add-sound` at listen_scale 1 (483f4487). B: **0.5 ok, 0.25 fails lip sync.** C: a re-voice reads as **voice-over** (dubbed, no mouth movement), so never re-voice an on-screen speaker. +4. Narration: **joins fine, keep `gap_seconds` 0.3** ("a titch rushed on occasion"). At 0.05 the voice gets louder at the joins (the +1.5/+2.6 dB take-to-take step is audible without a pause), so do not go below about 0.3. +5. Music: **drops ok, the ending is good, the key change is not jarring; keep the 1.5 s crossfade** (D better than E). The gain-ramp and beat-aligned-join follow-ups are dropped. +6. Hum: **no drone, no music** (light rain in all; A and B indistinguishable). The claim stays "probed, not a rule". + +Nothing is open: the Resolve import (#1197) passed on 2026-11-06. + +- [x] Import one exported `.otioz` (`export_timeline`, #1197) in DaVinci Resolve 18.5+: File > Import > Timeline, then check one clip per shot, the rate (23.976 as 24000/1001) and that each clip plays its span. Nothing here can run Resolve. *(2026-11-06: imports in Resolve 21, both the `.otioz` and the plain `.otio`, once the file sits in `~/Movies/DaVinci Resolve/`; the App Store build's sandbox can't read `~/Desktop`.)* +- Compact catalog listing budget raised to 13_250 (tests) for the new split-takes, narration-long, narration-from-takes and music-long templates. Recommendation: keep (discovery). *(Superseded 2026-11-06: `COMPACT_BUDGET` is now 13_900, `tests/test_catalog_structure.py`, last raised for `minimax/add-sound`, 483f4487.)* +- Arch ratchet module count grows on feat/long-form-media (presets, chain_pick, chain_look, chain_check, timeline_export, text_takes, ...). Re-baseline at merge needs your ok. Recommendation: approve. +- Compact catalog listing budget raised again to 13_450 (tests) for `minimax/repair-segment` (#1169, task 14), measured 13_432. Recommendation: keep (discovery). *(Superseded: now 13_900, see above.)* +- `minimax/repair-segment` declares its own VRAM formula (folded guide frames, 42.84 bytes/voxel = 28.71 * (124 + 61) / 124, an upper bound with head_frames held at 22 and tail_frames at most 39) and is listed in `TEMPLATE_SPECIFIC` in `tests/test_vram_inheritance.py`; it is derived, not measured, and over-projects a long render (about 0.9 GB at 192 frames on 1344x768). The Step 3 bench run should record its real peak. Recommendation: replace with the measured figure after the bench. +- Compact catalog listing budget raised to 13_500 (tests) for the #1169 review fixes (repair-segment's `chain_take`, `keep_segments` on four H3 chain templates), measured 13_476. Recommendation: keep (discovery). *(Superseded: now 13_900, see above.)* +- `save_segments`/`keep_segments` in the workflow schema now accept a `variable:` reference (they were boolean only). Recommendation: keep. +- Counter round 1, S1: **anchor 0.5 looks better, but measures worse on seam jump.** By eye it removes much of the burn that builds up by the end of the take. At frame 583 the stripes and wall are cleaner, and identity holds. Seam jump is 1.05 against the control's 0.85 (floor 0.06). The run is `r1-anchor50-s1`, job `fa17a2c51e33`; the control is `ctl-guide-s1-s1166`, job `b40e855eb553`. Frames are in `bench/records/counters.md`. Recommendation: take the anchor if seed 2 and the two-minute run (round 2) hold the gain. A 1.05 jump is far under the chain check's 4.0 ceiling, and the burn is what a viewer sees. **Round 2 (seed 2 and two minutes):** seed 2 repeats the jump cost (0.904 vs 0.796, `075ed6de83f7` against `894911aed6c7`), and anchor 0.5's texture lead over 0.25 is inside the floor there (0.272 vs 0.325); by eye the two look alike at 24 s. At two minutes (`107859a31bc1`) the anchor's texture and seam numbers beat the control's all the way, but by eye 0.5 softens the face from about 33 s and dissolves it into a white mask by 62 s. **Revised recommendation: anchor 0.25, not 0.5, as the static default** (`0a40c41af97c`, `93dd359dc11e`), for shots up to about 30 s. The strict rule says `mixed` for 0.25 too (jump worse on seed 1166 only, coarse texture on seed 2 only, never the same measure twice), so it needs your ok. +- Counter round 1, verdict rule: **should the soundtrack level step count on S1/S2?** Those scenes carry room tone and garden ambience, not a held voice. S1's level step is noisy: 7.7 vs 5.2 dB between the two control seeds. Almost every S1 guide arm raises it by 3-7 dB, so as written, no S1 arm can win on picture. `counters.md` gives both verdicts. Recommendation: count the level step only on scenes with a held voice or a music bed (S3, S4, S6), and read S1/S2 on picture. **Round 2:** the same-seed rerun of the guide control (`fd34bc937628`) moved the level step from 7.65 to 11.62 dB with the picture unchanged by eye, and the spread over three guide controls is 6.4 dB. No S1 arm's level-step mean is worse past that floor, and every round-2 verdict is the same with and without it. **Recommendation stands, now firm: drop the level step on S1/S2 (no held voice or bed); count it on S3, S4 and S6.** +- Counter round 1, continuity: **last_frame + look_hold on S1 has much cleaner seams (look 0.63 vs guide's 1.46), but the same burn by eye.** Recommendation: keep guide as the default unless round 2 also shows less burn. Seam steadiness alone does not justify the switch. **Round 2:** last_frame + look_hold alone still burns on seed 2 (the face is blotched at 28 s, `b7ca6da7d075`) and loses texture to guide (0.550 vs 0.455). Adding the anchor changes the picture: last_frame + look_hold + anchor beats the guide control on texture and seam look on all four runs and gives the cleanest S1 frames in the bench (`835379468ec3`, `41355297a083` at 0.25; `d8b17df7dfb9`, `c8bfadf286ce` at 0.5). Its costs are higher mid-band texture than guide on both seeds, a continuity change, no two-minute run, and look_hold holding a moving camera back. **Recommendation: guide stays the default for every shot type; offer last_frame + look_hold + anchor 0.25 as a static-shot knob, not a default, until it has a two-minute run.** +- Counter round 1, finding: **on the S2 arc, last_frame with carry_pick or look_hold froze the camera.** The statue faces front for the whole take, while the plain last_frame control does go round. Their winning numbers are a camera that did not move. Round 2 reruns look_hold at seed 2 to see whether it repeats. Recommendation: the playbook should not pair those counters with a moving camera until it is understood. **Round 2:** the seed-2 S2 run (`a3d6a044fd6b`) does not freeze fully, but the camera is held back: front view to 13 s, about three-quarter by 30 s, while the plain last_frame control at seed 2 reaches the statue's back. Its seam-look "win" (0.386 vs 1.026) is again a camera that moved less. **Recommendation, now firm: never pair look_hold or carry_pick with last_frame on a moving camera** (the template default table in `counters.md` says so). +- Counter round 2, finding: **the S1 guide control rerun at the same seed was not bit-identical** (`fd34bc937628` against `b40e855eb553`): seam look 1.656 vs 1.464, level step 11.62 vs 7.65 dB, same by eye. The two-minute control at the same seed does reproduce round 1 over its first six segments, so only the `-r2` rerun moved; the cause is unknown. `counters.md` widens the floors to cover it. Recommendation: accept the wider floors; no picture finding depends on it except guide look_hold's S1 seam-look gain, which becomes a tie; the all-measures verdicts move only through the level step. +- Counter round 2, finding: **no counter makes a two-minute static take usable.** The guide control burns into a mosaic from about 40 s and anchor 0.5 dissolves the face from about 50 s (`8d1614ced27a`, `107859a31bc1`). Recommendation: the playbook cuts static shots at about 30 s, and this opens the design's second group (carry without the pixel round trip) and the per-segment check. Worth one more pair of two-minute runs before that: anchor 0.25 and last_frame + look_hold + anchor 0.25. + +**Answered by Don (2026-11-04 17:47):** static default. Both takes degrade, anchor 0.25 less so. Decision: guide + anchor 0.25 is the static default for shots up to ~30 s, pending r3-anchor25-s1-120s; the remaining degradation goes to the structural levers (Task 17: windowed refine, latent carry) and the per-segment check. + +**Answered by Don (2026-11-05 14:30), latent carry first look** (`lc-anchor25-s1-120s-s1166`, `fc0f91663277`, against `r3-anchor25-s1-120s`, `25f6bd1da507`; clips in `~/Desktop/dw-bench-t17/lc-first-look/`): +- Picture: cleaner by the end, but quite soft and masky. The texture lead (0.64 vs 1.11 last segment) is softness, not detail: the anchor-0.5 failure mode, not a fix. +- Audio: no clicks. The take lost the music: only breaths, where the control's bed comes in later. Segment 1 is identical in both (-60.7 dB); afterwards the control rises to -45 dB by 60 s, while latent carry stays at -70 dB to the end. The 16/54 dB seam level steps are near-silence differences, which the S1 rule already leaves uncounted. The real finding is that the carry suppresses the late-arriving bed. +- Windowed refine 120 s (60-80 s clip): no shimmer and no lost movement visible, so the -17% motion is not a viewer-visible cost. The blotchiness is the source take's burn, which the refine does not touch (already recorded). + +## 2026-11-06 round: latent carry, SelfLift, TST, audio (records: `structural.md`, `speed.md`, `audio.md`) + +**Video. Clips are in `~/Desktop/dw-bench-t17/`.** +- [x] **Keystone at two minutes, new.** *(Superseded 2026-11-06: the keystone slot round replaced this question; slot "last" is the default, see below.)* Guide + anchor 0.25 + keystone `segment_1` crusts from about 42 s, then the backdrop comes back clean: around 80 s on the pixel take (`05a3a55b043c`), by around 104 s on the latent-carry take (`4766787e793d`). The carry take's face at 104-120 s is the clearest of any two-minute take. Watch `t17-s1-120s-keystone_LEFT-pixel_RIGHT-latentcarry-75-120s.mp4`. Is the reset near 80 s a visible pop? Is the face on the right better? A seed-2 pair and a keystone run without the anchor are on the bench now (`e4c7c19a99cd`, `4cae30688ed4`, `c4ec5a055464`). Recommendation: hold #1188 Step 5 (hybrid windows) until they read. +- **Latent carry, Task 17 Step 3: not a default.** Without the anchor the carry still burns, crusty and never soft (`20ce09d60040`; compare `t17-s1-120s-noanchor_LEFT-pixel-control_RIGHT-latentcarry-40-70s.mp4`). So the "soft and masky" look you saw on `fc0f91663277` comes from anchor 0.25. On S2 the carry makes the seams worse on both seeds, though the ambience is kept. The music bed you heard missing grows only in the seed-1166 pixel takes (seed 2 has none), so the carry is not suppressing it. Recommendation: no feature; keep the scratch flag until the keystone run decides. +- **SelfLift #1195: close.** Lift step 2 paints brush-stroke artifacts on every take. Lift step 1 re-stages the shot (PSNR 18-22 dB) to save about 6% of wall time. Recommendation: close with `speed.md`. +- **TST #1196: close.** At tau 0.2 per head it has no measured or visible effect, and it costs 5% over dense attention (dense is already 22-28% slower than the sparse default). Recommendation: close with `speed.md`, unless you want a tau sweep. +- **Host-RAM gate: confirmed.** Two H3 jobs ran together, one per card, for 65 min. Each committed 42-51 GiB, about 91 GiB of 125.7 GiB together. Recommendation: keep both cards (R41). +- **Two-minute floors now exist.** The face crop moves 0.64 on a same-seed rerun alone, so two-minute face reads must be by eye. Recommendation: accept. + +**Audio. Files are under lem `~/bench-workspace/outputs/`.** +- [x] **Audio pass #1192: listen to these.** Answered 2026-11-06 (review step 3): A pass ("very good lip sync"), shipped as `templates/minimax/add-sound` at scale 1 (483f4487); 0.5 ok, 0.25 fails lip sync; the re-voice reads as voice-over. The picture is the source's own pixels (PSNR 42 dB, one re-encode), and every line is spoken word for word. + - S4 at listen_scale 1, one take per shot: `0e13c36a659b` cold_open, `adfb95588730` deflect, `9f76f275eb8b` react, `a37bd9bc70d1` button (its line starts 0.9 s after the source's). Check lip sync, whether the laugh lands, and the room tone. Recommendation: promote to `templates/minimax/add-sound` at listen_scale 1 if these pass. + - listen_scale on cold_open: 0.5 is `a1d731ddbf29`, 0.25 is `c92a26d1c1d3`. They start the line 0.68 s and 0.88 s early, and would save about 23% and 31% of wall time. Recommendation: knobs only, unless the sync holds. + - S3 re-voice: same line `864375a2cef2` (spoken 1.96-4.46 s against the mouth's 1.20-5.14 s), new line `4b6ebb139735`. Both are 6-7 LU under the take's bed. Does either read as dubbed? + - Foley `b51cc152524e`: no speech was asked for, but Whisper heard "Thank you very much", and the first 2.9 s are dead air. Diagnostic only. +- [x] **Long narration: listen.** Answered 2026-11-06 (review step 4): joins fine, keep 0.3; at 0.05 the voice gets louder at the joins. The default is 37 s takes, not 45 (50e036fe). Each file is `BenchS6NarrationLong//final/...narration.0-0.0.wav`. + - Take 45 `111a5623cba0` (198 s, every word present). Listen at the joins (33.3, 70.0, 99.3, 129.6 and 164.9 s) for pause length and any voice change. + - Take 60 `3668ebd2e4de` and `c9f1f6145c3c`: words lost. There is a loop at 24-30 s, a skipped clause near 153 s, and stops mid-word. The template default is moving to take 45 and 3.0 words a second, both measured (being fixed now). + - Gap 0.05 `c221ffe761a9`: the same speech as take 60 with shorter joins (30.8, 76.1, 116.7 and 158.9 s). Do 0.08-0.23 s joins run sentences together? Do the 0.24-0.5 s joins at gap 0.3 feel rushed? Recommendation: `gap_seconds` 0.45, subject to your ear; it stays 0.3 until you say. *(Superseded 2026-11-06: Don kept 0.3.)* +- **Long music: don't judge yet.** music-long padded short sections with silence (12-57 s holes and a 45 s silent tail), because Music 3 wrote 27-70 s where 85 s was asked. A fix is in progress, and S6 music reruns after it. The one real crossfade, near 250 s on `52303a4436d4` and `3bfee8aad7c5` (xf3000), is the only join worth hearing now. + +**Keystone confirmation (`structural.md` *Keystone confirmation*, be21f33c)** +- **The burn reset comes from the keystone and repeats on seed 2.** + - Resets: seed 2 pixel `e4c7c19a99cd` at 58.7 s and 111 s; seed 2 latent carry `4cae30688ed4` at 91.7 s; keystone without the anchor `c4ec5a055464` (seed 1166) at 39.5 s and 81.5 s. + - No take without a keystone ever comes back. + - Fine texture at the end: 0.11 against 1.14 (seed 2); 0.53 against 1.71 (no anchor). + - The keystone costs no wall time. It also removes seed 1166's late music bed. +- **It is a wipe, not a hold.** The crust grows back between resets. Each reset is a visible pop mid-segment, at times that vary with seed and arm (39-111 s). The seam measure misses it. Light, colour and framing do not jump; only the backdrop texture does. +- **The anchor decides the face, not the backdrop.** Without it the face burns (painted, high contrast). With it the pixel face turns into a soft white mask from about 100 s. The latent-carry face stays sharp on both seeds, slightly glossy. +- [x] **Watch the pop.** *(Superseded 2026-11-06: slot "last" removes the pop and is the default; plain keystone is a knob.)* + - `~/Desktop/dw-bench-t17/t17-s1-120s-seed2-keystone_LEFT-pixel-keystone-anchor25_RIGHT-anchor25-control-50-66s.mp4`: the pop is 8.7 s in. + - `t17-s1-120s-seed2-keystone_LEFT-pixel_RIGHT-latentcarry-88-120s.mp4`: the carry pop is 3.7 s in, the pixel's second reset is about 23 s in, and the faces are at the end. + - Is a mid-shot pop to a clean backdrop better than steady crust? Recommendation: yes, make the keystone the default for static chained takes past about 30 s. Anchor 0.25 stays a knob (+33% wall time, soft face). Never use it on moving shots: round 1's keystone lost on look on S2. +- **#1188 hybrid windows, Steps 5-6: not built in this plan.** The chain keystone answers Step 6's anchor question. What is left is the blended finish, at 36+ min per minute of take, and Step 4's refine already covers seams. The cheaper next lever is to make the reset regular (every segment) inside the chain; that spike is running now. The #1188 commenter is yours to answer. +- **#1044 latent carry: keep the scratch flag, for static shots only.** It ties on numbers, holds the face better by eye on both seeds, and is about 4 min faster. Your read of the 88-120 s clip decides whether it becomes a knob. + +**#1193 skill claims (Task 26; `docs/proposals/audits/2026-11-longvideos-claims.md`)** +- None of the five claims held on both seeds, so none is a rule. Each is recorded under *Probed and not a rule* in its reference. A `` line acting out what it mentions: refuted. An unnamed reference read as a second person: refuted. A full-body portrait copying its clothes back on: refuted. Leaving the portrait out on a continued shot: refuted; leaving it out costs identity, which backs chains.md's advice to keep it. H3 adding music when the music line is left out: refuted. +- [x] **Listen, claim 5 (bench `BenchT26T2VA`).** Answered 2026-11-06 (review step 6): no drone, no music (light rain in all; hum and no-hum indistinguishable), so it stays "probed, not a rule". Hum: `313fe586f422`, `412db5c1ed0e`. No hum: `133cae62bfbd`, `c1b9be136407`. No music line: `3e449fbebbca`, `e0f10599f3d4`. Can you hear a drone in the hum clips, and any music in the no-music-line clips? By measure, the hum adds a floor below 125 Hz, 30-45 dB under the clip, with no tone. Neither no-music-line clip has tonal content. Recommendation: if you hear no drone, keep it as probed. If you do, add "don't name a steady hum the scene doesn't need" to prompt-lint. +- Side note: an unnamed second reference picture lowered the named character's identity similarity (0.39/0.36 against 0.45/0.47). It is recorded, not ruled on. + +**S6 rerun on mini-ai (fixed templates; `audio.md` *S6 rerun*, 54ad3036). Files are under mini-ai `~/bench-workspace/outputs/`.** +- **Task 22 gate passes.** Narration at the new defaults (37 s takes at 3.0 words/s) runs about 200 s on both seeds, with every word and no voice drift. Five of six joins fall on paragraph breaks. Music runs 311-319 s with no silence and a real ending. +- **Music with the 8 s end snap (`audio.md` *Music with the 8 s end snap*): the cut is fixed, the level step is not.** All six section cuts moved 0.6-7.0 s back to a quiet point, and none is mid-word. `slice_dropped_loud_tail` fired on all six, and it was right each time. But four of six joins still drop 7.3-9.7 dB, as before (7.4-8.9). The step comes from the next section's own quiet opening, not from where the cut falls. Lyrics are still lost when a section overruns, because the snap keeps nothing more: s1166 verse_1 still loses its last line and the pre-chorus, and s1167 verse_2 drops its last bridge line whole instead of mid-line. The pieces are 308.8 s and 297.7 s (11-14 s shorter), with no silence holes. What is left is a lyric sized per section, or a gain ramp at each join. Neither is made yet. *(2026-11-06: Don heard the drops as ok; the gain ramp is dropped.)* +- [x] **Listen, narration.** Answered 2026-11-06 (review step 4): joins fine, keep 0.3. `847ccb839875` (`BenchS6NarrationLong-t37/20261106-044527-c46db14f/`), joins at 34.1, 70.7, 97.2, 127.0, 153.7 and 182.8 s. `4467e6380276` (`.../20261106-044934-a03a36cc/`), joins at 34.0, 70.7, 97.3, 127.0, 153.4 and 181.0 s. Each join pause is 0.36-0.48 s against 0.6 s sentence pauses. Too short? Recommendation: raise `gap_seconds` from 0.3 to 0.45 if the joins feel rushed. *(Superseded 2026-11-06: not raised.)* +- [x] **Listen, music with the snap** (answered 2026-11-06, review step 5: drops ok, the ending is good; the gain ramp is dropped) (mini-ai `~/bench-workspace/outputs/BenchS6MusicLongFull/`). `2179e0c0c9df`, `20261106-081846-f8973042/final/BenchS6MusicLongFull-piece.0-0.0.mp3` (s1166): crossfades at 80.7-82.2 s (-9.7 dB), 163.5-165.0 s (-7.4 dB) and 240.1-241.6 s (-1.1 dB, the one the snap flattened), and the ending at 295-309 s. `d51816c55389`, `20261106-085848-655afa0d/final/BenchS6MusicLongFull-piece.0-0.0.mp3` (s1167): 77.4-78.9 s (-1.5 dB), 156.9-158.4 s (-9.4 dB) and 236.8-238.3 s (-7.3 dB; the last bridge line is now dropped cleanly before it). Compare them with the same seeds without the snap below, at 85, 168.5 and 252 s. Does a clean cut followed by a quiet intro still sound like a drop? Recommendation: keep the 8 s snap as the default (it only ever moves a cut to a quieter point), and size each section's lyric so it ends inside 85 s. +- [x] **Listen, music** (to judge the joins, ignoring the cut defect). Answered 2026-11-06 (review step 5): the key change is not jarring, and 1.5 s stays (D better than E); beat-aligned joins are dropped. `ae2e1373cff6` (`BenchS6MusicLongFull/20261106-024659-f8973042/`): 85 s, 168.5 s and 252 s, and the ending at 305-319 s. `f83cf3566727` (`.../20261106-032709-655afa0d/`): 252 s, a possible C major into A major key change. Is that jarring? `ef47c7b6684d` (`.../20261106-040613-0ce22d5b/`): the same draw with 3 s crossfades. Recommendation: keep 1.5 s unless 3 s sounds better to you. Tempo reads 78-92 BPM against the 72 asked; key and tempo continuity across sections is your ear's call. + +**Keystone slot round (`structural.md` *Keystone slot*, dec8d93d)** +- **Slot "last" holds the burn for two minutes with no pop.** In this mode segment 1's opening frame becomes the last image of every later segment. Crust stays at 1.1-1.5x of segment 1 on seeds 1166 and 2, where plain keystone saw-tooths between 1x and 6x. The face stays sharp and natural to 120 s, the best two-minute face so far. Cost: about +9 min wall. +- [x] **The catch, your call.** Answered 2026-11-06 (review step 1): "not a loop". Slot "last" is the static long-take default, as `shot: static_long` on `templates/minimax/chained-segments` (stable chain key `keystone_slot`); plain keystone and no keystone stay knobs. The subject eases back to the opening pose at the end of every 4.25 s segment: one glance or turn, then a return, 28 times in two minutes. There is no snap (the motion step at the seam is inside the take's normal range). Watch `~/Desktop/dw-bench-t17/t17-s1-120s-s1166-keystone_LEFT-slot-last_RIGHT-plain-keystone-30-50s.mp4`. The left take returns to the pose just before 0.7, 4.9, 9.2, 13.4 and 17.7 s; the right take pops at 9.7 s. Does the rhythm read as a loop? Recommendation: if it doesn't, slot "last" replaces plain keystone as the static long-take default (static only; it can't serve a moving camera). If it does read as a loop, plain keystone stays and the slot work closes. +- **Prompt confound settled: the slot alone holds.** Without the `` prompt clause (`f55bd445faa5` seed 1166, `00ed8fb067ca` seed 2), crust stays at 1.1-1.5x with no pop, the face matches the prompted takes, and the pose loop is unchanged (the last frame returns to segment 1's pose on both seeds). So the clause caused neither the hold nor the loop, and the pose-loop call above stands as the only open question. +- **Closed:** + - f34: the reset comes regularly, but the pose snaps to segment 1 each time. + - f34 gated at 0.9: 3-frame flashes, and the worst burn yet. + - guide_noise 0.4: the threshold rose instead of falling. + - guide_frames 39: +18 min, still a sawtooth. + +**LTX long takes (Task 23; `ltx.md`). Run on mini-ai, outputs under mini-ai `~/bench-workspace/outputs/`.** +- **Default stays the last_frame chain, about 15 s.** Nothing beat it on picture on both seeds at no cost. +- [x] **36 s LTX takes are not usable as one shot.** Answered 2026-11-06 (review step 2): cap ok. On both seeds the rider changes by about 30 s, and pale vertical streaks grow from about 20 s. With look_hold, segment 7 swaps the courier for a man on a green moped at 29.8 s. Watch `~/Desktop/dw-bench-t23/t23-s5-36s-s1166_LEFT-control_RIGHT-lookhold-18-36s.mp4` (the swap is 11.8 s in). Recommendation: the playbook caps an LTX long take at about 15-20 s and cuts to a fresh still past that. +- [x] **look_hold as the LTX chain default?** Answered 2026-11-06 (review step 2): on, by the recommendation (c8fe45bb). It wins seam look on both S5 seeds (1.72→1.17, 1.68→1.22), costs no wall time warm, and unlike H3 it does not hold an orbit back: both orbit seeds reach the statue's back with it (`9331cd20fa39`, `64641b316eed`). But no difference is visible by eye. Recommendation: turn it on in `ltx2/chained-segments`; the template is unchanged until you say. *(Done: on.)* +- **extend (whole previous segment as condition): a knob, not a default.** It wins seam jump on both seeds at 2.1x wall time. On seed 2 the courier rides out of frame in segment 1 and extend keeps the road empty for 10 s, where last_frame brings a courier back. Optional look: `~/Desktop/dw-bench-t23/t23-s5-s1166_LEFT-lastframe_RIGHT-extend-0-15s.mp4` (sound is the right take's; it drops out 5-10 s, see next). +- **Audio dropout, follow-up issue candidate.** LTX on mini-ai renders some segments near-silent (-89 dBFS): segment 3 of every seed-2 run, on S5 and on the orbit, whatever the prompt or picture; and segment 2 of extend at seed 1166. The lem CUDA S5 runs had none. The chain check reads picture only and misses it. Recommendation: file an issue (MPS only? audio VAE or vocoder?) and add a segment silence check. +- carry_pick 4 loses on S5 (seed 2 jump 1.25→1.81) and shortens the take. check + 1 re-roll never fired on S5 (every segment passed), so it ties at no cost and the round cannot judge it on LTX. diff --git a/bench/records/keystone-confirm-moments.json b/bench/records/keystone-confirm-moments.json new file mode 100644 index 000000000..8fe058b44 --- /dev/null +++ b/bench/records/keystone-confirm-moments.json @@ -0,0 +1,3808 @@ +{ + "note": "Analyst moments pass 2026-11-06 for the keystone confirmation (in-process on lem from the bench checkout at 47a9e025, /tmp/analyst-t19/measure_all.py with ffmpeg on PATH). 'moments': the t17-t19 moments (frames 62..2868) for the three new takes, plus arm-vs-control PSNR/SSIM. 'dense': scene and face texture bands every 5 s (frame 62, 240..2760 step 120, 2868) for the new takes and their controls, to time the resets.", + "moments": { + "takes": { + "e4c7c19a99cd": { + "scene": "s1-static", + "moments": [ + 62, + 300, + 583, + 1000, + 1501, + 2000, + 2500, + 2868 + ], + "scene_bands": [ + { + "fine": 6.5502, + "mid": 13.9675, + "coarse": 44.0694 + }, + { + "fine": 7.9298, + "mid": 15.1004, + "coarse": 44.8247 + }, + { + "fine": 8.7939, + "mid": 15.2992, + "coarse": 45.2235 + }, + { + "fine": 9.8773, + "mid": 15.3696, + "coarse": 45.8404 + }, + { + "fine": 7.9355, + "mid": 15.3907, + "coarse": 45.4232 + }, + { + "fine": 8.1414, + "mid": 15.0356, + "coarse": 43.0559 + }, + { + "fine": 8.8112, + "mid": 14.8987, + "coarse": 40.7183 + }, + { + "fine": 7.2855, + "mid": 14.7481, + "coarse": 44.0945 + } + ], + "face_bands": [ + { + "fine": 11.3836, + "mid": 27.014, + "coarse": 43.5128 + }, + { + "fine": 12.4566, + "mid": 24.3126, + "coarse": 45.7903 + }, + { + "fine": 12.7699, + "mid": 27.757, + "coarse": 49.2866 + }, + { + "fine": 11.874, + "mid": 26.575, + "coarse": 48.5944 + }, + { + "fine": 9.1253, + "mid": 26.5738, + "coarse": 53.0206 + }, + { + "fine": 9.6347, + "mid": 25.6776, + "coarse": 59.1211 + }, + { + "fine": 6.0633, + "mid": 23.8986, + "coarse": 64.7058 + }, + { + "fine": 5.8601, + "mid": 26.3229, + "coarse": 68.4634 + } + ], + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 0.69, + "look_ratio": 1.83, + "level_step_db": 3.93, + "typical_delta": 2.0 + }, + { + "seconds": 9.417, + "jump_ratio": 0.71, + "look_ratio": 1.04, + "level_step_db": 20.05, + "typical_delta": 2.0 + }, + { + "seconds": 13.667, + "jump_ratio": 0.83, + "look_ratio": 1.24, + "level_step_db": 2.35, + "typical_delta": 2.0 + }, + { + "seconds": 17.917, + "jump_ratio": 0.93, + "look_ratio": 1.45, + "level_step_db": 6.0, + "typical_delta": 2.0 + }, + { + "seconds": 22.167, + "jump_ratio": 1.09, + "look_ratio": 1.74, + "level_step_db": 9.43, + "typical_delta": 2.0 + }, + { + "seconds": 26.417, + "jump_ratio": 0.83, + "look_ratio": 0.38, + "level_step_db": 22.67, + "typical_delta": 2.0 + }, + { + "seconds": 30.667, + "jump_ratio": 0.9, + "look_ratio": 1.41, + "level_step_db": 4.45, + "typical_delta": 2.0 + }, + { + "seconds": 34.917, + "jump_ratio": 0.83, + "look_ratio": 1.09, + "level_step_db": 1.77, + "typical_delta": 2.0 + }, + { + "seconds": 39.167, + "jump_ratio": 1.01, + "look_ratio": 1.21, + "level_step_db": 12.54, + "typical_delta": 2.0 + }, + { + "seconds": 43.417, + "jump_ratio": 0.76, + "look_ratio": 0.77, + "level_step_db": 15.22, + "typical_delta": 2.0 + }, + { + "seconds": 47.667, + "jump_ratio": 0.88, + "look_ratio": 1.22, + "level_step_db": 6.48, + "typical_delta": 2.0 + }, + { + "seconds": 51.917, + "jump_ratio": 0.88, + "look_ratio": 1.25, + "level_step_db": 2.57, + "typical_delta": 2.0 + }, + { + "seconds": 56.167, + "jump_ratio": 0.84, + "look_ratio": 0.68, + "level_step_db": 3.74, + "typical_delta": 2.0 + }, + { + "seconds": 60.417, + "jump_ratio": 0.74, + "look_ratio": 0.32, + "level_step_db": 9.08, + "typical_delta": 2.0 + }, + { + "seconds": 64.667, + "jump_ratio": 0.81, + "look_ratio": 0.62, + "level_step_db": 27.03, + "typical_delta": 2.0 + }, + { + "seconds": 68.917, + "jump_ratio": 0.75, + "look_ratio": 0.71, + "level_step_db": 18.79, + "typical_delta": 2.0 + }, + { + "seconds": 73.167, + "jump_ratio": 0.76, + "look_ratio": 0.52, + "level_step_db": 0.85, + "typical_delta": 2.0 + }, + { + "seconds": 77.417, + "jump_ratio": 0.76, + "look_ratio": 0.51, + "level_step_db": 9.4, + "typical_delta": 2.0 + }, + { + "seconds": 81.667, + "jump_ratio": 0.9, + "look_ratio": 0.72, + "level_step_db": 19.41, + "typical_delta": 2.0 + }, + { + "seconds": 85.917, + "jump_ratio": 0.81, + "look_ratio": 0.53, + "level_step_db": 22.0, + "typical_delta": 2.0 + }, + { + "seconds": 90.167, + "jump_ratio": 0.86, + "look_ratio": 1.32, + "level_step_db": 8.06, + "typical_delta": 2.0 + }, + { + "seconds": 94.417, + "jump_ratio": 1.24, + "look_ratio": 2.28, + "level_step_db": 4.66, + "typical_delta": 2.0 + }, + { + "seconds": 98.667, + "jump_ratio": 0.75, + "look_ratio": 0.86, + "level_step_db": 22.05, + "typical_delta": 2.0 + }, + { + "seconds": 102.917, + "jump_ratio": 0.83, + "look_ratio": 0.91, + "level_step_db": 18.42, + "typical_delta": 2.0 + }, + { + "seconds": 107.167, + "jump_ratio": 0.68, + "look_ratio": 0.62, + "level_step_db": 22.02, + "typical_delta": 2.0 + }, + { + "seconds": 111.417, + "jump_ratio": 1.25, + "look_ratio": 1.57, + "level_step_db": 19.64, + "typical_delta": 2.0 + }, + { + "seconds": 115.667, + "jump_ratio": 0.79, + "look_ratio": 1.43, + "level_step_db": 0.17, + "typical_delta": 2.0 + }, + { + "seconds": 119.917, + "jump_ratio": 0.66, + "look_ratio": 0.17, + "level_step_db": 2.1, + "typical_delta": 2.0 + } + ], + "motion_mean": 0.5527063012123108, + "audio": { + "duration_s": 120.0, + "segments": [ + { + "start": 0.0, + "end": 5.17, + "rms_db": -60.41, + "hf_share_6k": 0.03143653646111488 + }, + { + "start": 5.17, + "end": 9.42, + "rms_db": -56.61, + "hf_share_6k": 0.005284994840621948 + }, + { + "start": 9.42, + "end": 13.67, + "rms_db": -76.96, + "hf_share_6k": 0.08032844215631485 + }, + { + "start": 13.67, + "end": 17.92, + "rms_db": -74.64, + "hf_share_6k": 0.08538959175348282 + }, + { + "start": 17.92, + "end": 22.17, + "rms_db": -68.57, + "hf_share_6k": 0.04784107953310013 + }, + { + "start": 22.17, + "end": 26.42, + "rms_db": -78.27, + "hf_share_6k": 0.09931812435388565 + }, + { + "start": 26.42, + "end": 30.67, + "rms_db": -54.99, + "hf_share_6k": 0.13985933363437653 + }, + { + "start": 30.67, + "end": 34.92, + "rms_db": -50.59, + "hf_share_6k": 0.05248640105128288 + }, + { + "start": 34.92, + "end": 39.17, + "rms_db": -48.53, + "hf_share_6k": 0.2343096137046814 + }, + { + "start": 39.17, + "end": 43.42, + "rms_db": -61.14, + "hf_share_6k": 0.03310305252671242 + }, + { + "start": 43.42, + "end": 47.67, + "rms_db": -76.59, + "hf_share_6k": 0.018122941255569458 + }, + { + "start": 47.67, + "end": 51.92, + "rms_db": -83.4, + "hf_share_6k": 0.057683490216732025 + }, + { + "start": 51.92, + "end": 56.17, + "rms_db": -80.48, + "hf_share_6k": 0.04542357474565506 + }, + { + "start": 56.17, + "end": 60.42, + "rms_db": -77.64, + "hf_share_6k": 0.026044005528092384 + }, + { + "start": 60.42, + "end": 64.67, + "rms_db": -85.96, + "hf_share_6k": 0.07043717056512833 + }, + { + "start": 64.67, + "end": 68.92, + "rms_db": -58.67, + "hf_share_6k": 0.01167861558496952 + }, + { + "start": 68.92, + "end": 73.17, + "rms_db": -77.28, + "hf_share_6k": 0.015405846759676933 + }, + { + "start": 73.17, + "end": 77.42, + "rms_db": -76.52, + "hf_share_6k": 0.008367175236344337 + }, + { + "start": 77.42, + "end": 81.67, + "rms_db": -86.31, + "hf_share_6k": 0.07868078351020813 + }, + { + "start": 81.67, + "end": 85.92, + "rms_db": -66.43, + "hf_share_6k": 0.008301696740090847 + }, + { + "start": 85.92, + "end": 90.17, + "rms_db": -88.84, + "hf_share_6k": 0.10431674867868423 + }, + { + "start": 90.17, + "end": 94.42, + "rms_db": -80.7, + "hf_share_6k": 0.037525475025177 + }, + { + "start": 94.42, + "end": 98.67, + "rms_db": -85.36, + "hf_share_6k": 0.057973504066467285 + }, + { + "start": 98.67, + "end": 102.92, + "rms_db": -62.85, + "hf_share_6k": 0.46497291326522827 + }, + { + "start": 102.92, + "end": 107.17, + "rms_db": -81.4, + "hf_share_6k": 0.01212142501026392 + }, + { + "start": 107.17, + "end": 111.42, + "rms_db": -60.3, + "hf_share_6k": 0.04536306858062744 + }, + { + "start": 111.42, + "end": 115.67, + "rms_db": -79.31, + "hf_share_6k": 0.06805173307657242 + }, + { + "start": 115.67, + "end": 119.92, + "rms_db": -79.01, + "hf_share_6k": 0.0363612063229084 + } + ], + "rms_db_5s": [ + -60.33, + -57.28, + -76.89, + -72.74, + -69.98, + -55.69, + -51.16, + -49.32, + -61.78, + -79.59, + -81.37, + -78.17, + -86.4, + -59.33, + -76.94, + -88.67, + -67.12, + -89.01, + -81.14, + -72.28, + -64.12, + -61.02, + -78.59, + -79.16 + ] + } + }, + "4cae30688ed4": { + "scene": "s1-static", + "moments": [ + 62, + 300, + 583, + 1000, + 1501, + 2000, + 2500, + 2868 + ], + "scene_bands": [ + { + "fine": 6.5502, + "mid": 13.9675, + "coarse": 44.0694 + }, + { + "fine": 7.2255, + "mid": 14.5276, + "coarse": 44.0961 + }, + { + "fine": 7.6465, + "mid": 14.7647, + "coarse": 43.4514 + }, + { + "fine": 7.3266, + "mid": 14.393, + "coarse": 41.9964 + }, + { + "fine": 7.0504, + "mid": 13.6247, + "coarse": 40.0478 + }, + { + "fine": 7.1118, + "mid": 13.1102, + "coarse": 38.7717 + }, + { + "fine": 6.5685, + "mid": 13.0764, + "coarse": 42.0848 + }, + { + "fine": 6.7194, + "mid": 13.2187, + "coarse": 41.0519 + } + ], + "face_bands": [ + { + "fine": 11.3836, + "mid": 27.014, + "coarse": 43.5128 + }, + { + "fine": 11.9338, + "mid": 26.1987, + "coarse": 45.0567 + }, + { + "fine": 11.1925, + "mid": 25.5437, + "coarse": 49.6614 + }, + { + "fine": 13.5201, + "mid": 29.0456, + "coarse": 52.3962 + }, + { + "fine": 11.687, + "mid": 27.923, + "coarse": 47.8788 + }, + { + "fine": 10.8649, + "mid": 28.3977, + "coarse": 52.4111 + }, + { + "fine": 7.1445, + "mid": 27.627, + "coarse": 59.9862 + }, + { + "fine": 7.867, + "mid": 26.3211, + "coarse": 62.4211 + } + ], + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 0.9, + "look_ratio": 1.85, + "level_step_db": 9.62, + "typical_delta": 2.0 + }, + { + "seconds": 9.417, + "jump_ratio": 0.71, + "look_ratio": 1.07, + "level_step_db": 18.8, + "typical_delta": 2.0 + }, + { + "seconds": 13.667, + "jump_ratio": 0.63, + "look_ratio": 0.34, + "level_step_db": 6.38, + "typical_delta": 2.0 + }, + { + "seconds": 17.917, + "jump_ratio": 0.82, + "look_ratio": 1.54, + "level_step_db": 8.61, + "typical_delta": 2.0 + }, + { + "seconds": 22.167, + "jump_ratio": 0.98, + "look_ratio": 1.61, + "level_step_db": 6.9, + "typical_delta": 2.0 + }, + { + "seconds": 26.417, + "jump_ratio": 0.9, + "look_ratio": 0.83, + "level_step_db": 2.41, + "typical_delta": 2.0 + }, + { + "seconds": 30.667, + "jump_ratio": 1.1, + "look_ratio": 1.75, + "level_step_db": 9.32, + "typical_delta": 2.0 + }, + { + "seconds": 34.917, + "jump_ratio": 0.89, + "look_ratio": 1.32, + "level_step_db": 20.43, + "typical_delta": 2.0 + }, + { + "seconds": 39.167, + "jump_ratio": 0.88, + "look_ratio": 1.5, + "level_step_db": 18.83, + "typical_delta": 2.0 + }, + { + "seconds": 43.417, + "jump_ratio": 1.15, + "look_ratio": 2.26, + "level_step_db": 19.78, + "typical_delta": 2.0 + }, + { + "seconds": 47.667, + "jump_ratio": 1.08, + "look_ratio": 2.02, + "level_step_db": 23.51, + "typical_delta": 2.0 + }, + { + "seconds": 51.917, + "jump_ratio": 0.8, + "look_ratio": 1.31, + "level_step_db": 13.52, + "typical_delta": 2.0 + }, + { + "seconds": 56.167, + "jump_ratio": 0.98, + "look_ratio": 2.01, + "level_step_db": 28.05, + "typical_delta": 2.0 + }, + { + "seconds": 60.417, + "jump_ratio": 0.79, + "look_ratio": 1.19, + "level_step_db": 1.28, + "typical_delta": 2.0 + }, + { + "seconds": 64.667, + "jump_ratio": 0.97, + "look_ratio": 1.7, + "level_step_db": 8.61, + "typical_delta": 2.0 + }, + { + "seconds": 68.917, + "jump_ratio": 0.77, + "look_ratio": 1.11, + "level_step_db": 7.78, + "typical_delta": 2.0 + }, + { + "seconds": 73.167, + "jump_ratio": 0.9, + "look_ratio": 1.23, + "level_step_db": 14.63, + "typical_delta": 2.0 + }, + { + "seconds": 77.417, + "jump_ratio": 0.89, + "look_ratio": 1.15, + "level_step_db": 10.33, + "typical_delta": 2.0 + }, + { + "seconds": 81.667, + "jump_ratio": 0.83, + "look_ratio": 1.26, + "level_step_db": 9.03, + "typical_delta": 2.02 + }, + { + "seconds": 85.917, + "jump_ratio": 0.81, + "look_ratio": 1.3, + "level_step_db": 25.97, + "typical_delta": 2.04 + }, + { + "seconds": 90.167, + "jump_ratio": 0.67, + "look_ratio": 0.72, + "level_step_db": 4.72, + "typical_delta": 2.04 + }, + { + "seconds": 94.417, + "jump_ratio": 0.57, + "look_ratio": 0.31, + "level_step_db": 28.66, + "typical_delta": 2.0 + }, + { + "seconds": 98.667, + "jump_ratio": 0.56, + "look_ratio": 0.42, + "level_step_db": 11.48, + "typical_delta": 2.0 + }, + { + "seconds": 102.917, + "jump_ratio": 0.67, + "look_ratio": 0.97, + "level_step_db": 22.69, + "typical_delta": 2.0 + }, + { + "seconds": 107.167, + "jump_ratio": 0.57, + "look_ratio": 0.8, + "level_step_db": 39.21, + "typical_delta": 2.0 + }, + { + "seconds": 111.417, + "jump_ratio": 0.59, + "look_ratio": 0.51, + "level_step_db": 11.07, + "typical_delta": 2.0 + }, + { + "seconds": 115.667, + "jump_ratio": 0.59, + "look_ratio": 0.54, + "level_step_db": 17.8, + "typical_delta": 2.0 + }, + { + "seconds": 119.917, + "jump_ratio": 0.54, + "look_ratio": 0.91, + "level_step_db": 21.95, + "typical_delta": 2.0 + } + ], + "motion_mean": 0.6036450266838074, + "audio": { + "duration_s": 120.0, + "segments": [ + { + "start": 0.0, + "end": 5.17, + "rms_db": -60.41, + "hf_share_6k": 0.03143810108304024 + }, + { + "start": 5.17, + "end": 9.42, + "rms_db": -50.99, + "hf_share_6k": 0.010038517415523529 + }, + { + "start": 9.42, + "end": 13.67, + "rms_db": -69.46, + "hf_share_6k": 0.02182365581393242 + }, + { + "start": 13.67, + "end": 17.92, + "rms_db": -76.14, + "hf_share_6k": 0.08023586869239807 + }, + { + "start": 17.92, + "end": 22.17, + "rms_db": -67.44, + "hf_share_6k": 0.019222022965550423 + }, + { + "start": 22.17, + "end": 26.42, + "rms_db": -74.33, + "hf_share_6k": 0.012460981495678425 + }, + { + "start": 26.42, + "end": 30.67, + "rms_db": -76.65, + "hf_share_6k": 0.025053881108760834 + }, + { + "start": 30.67, + "end": 34.92, + "rms_db": -67.32, + "hf_share_6k": 0.008600120432674885 + }, + { + "start": 34.92, + "end": 39.17, + "rms_db": -46.66, + "hf_share_6k": 0.01605166494846344 + }, + { + "start": 39.17, + "end": 43.42, + "rms_db": -65.5, + "hf_share_6k": 0.007052841130644083 + }, + { + "start": 43.42, + "end": 47.67, + "rms_db": -45.69, + "hf_share_6k": 0.004788658116012812 + }, + { + "start": 47.67, + "end": 51.92, + "rms_db": -69.65, + "hf_share_6k": 0.019207347184419632 + }, + { + "start": 51.92, + "end": 56.17, + "rms_db": -55.71, + "hf_share_6k": 0.09800471365451813 + }, + { + "start": 56.17, + "end": 60.42, + "rms_db": -84.08, + "hf_share_6k": 0.04447990655899048 + }, + { + "start": 60.42, + "end": 64.67, + "rms_db": -85.24, + "hf_share_6k": 0.03790244832634926 + }, + { + "start": 64.67, + "end": 68.92, + "rms_db": -76.71, + "hf_share_6k": 0.012253332883119583 + }, + { + "start": 68.92, + "end": 73.17, + "rms_db": -68.79, + "hf_share_6k": 0.017621036618947983 + }, + { + "start": 73.17, + "end": 77.42, + "rms_db": -83.45, + "hf_share_6k": 0.016794631257653236 + }, + { + "start": 77.42, + "end": 81.67, + "rms_db": -73.22, + "hf_share_6k": 0.015801746398210526 + }, + { + "start": 81.67, + "end": 85.92, + "rms_db": -64.03, + "hf_share_6k": 0.008089600130915642 + }, + { + "start": 85.92, + "end": 90.17, + "rms_db": -90.93, + "hf_share_6k": 0.06846284866333008 + }, + { + "start": 90.17, + "end": 94.42, + "rms_db": -85.41, + "hf_share_6k": 0.05914141982793808 + }, + { + "start": 94.42, + "end": 98.67, + "rms_db": -57.28, + "hf_share_6k": 0.07722102105617523 + }, + { + "start": 98.67, + "end": 102.92, + "rms_db": -67.91, + "hf_share_6k": 0.47484126687049866 + }, + { + "start": 102.92, + "end": 107.17, + "rms_db": -91.51, + "hf_share_6k": 0.1642824113368988 + }, + { + "start": 107.17, + "end": 111.42, + "rms_db": -51.84, + "hf_share_6k": 0.07700885087251663 + }, + { + "start": 111.42, + "end": 115.67, + "rms_db": -63.34, + "hf_share_6k": 0.06398054212331772 + }, + { + "start": 115.67, + "end": 119.92, + "rms_db": -80.62, + "hf_share_6k": 0.036052774637937546 + } + ], + "rms_db_5s": [ + -60.33, + -51.68, + -71.04, + -73.29, + -68.12, + -77.29, + -68.02, + -47.36, + -63.89, + -46.41, + -56.39, + -83.78, + -85.24, + -77.17, + -69.4, + -77.38, + -64.46, + -91.47, + -82.44, + -58.0, + -68.61, + -57.23, + -53.91, + -75.31 + ] + } + }, + "c4ec5a055464": { + "scene": "s1-static", + "moments": [ + 62, + 300, + 583, + 1000, + 1501, + 2000, + 2500, + 2868 + ], + "scene_bands": [ + { + "fine": 6.7328, + "mid": 14.2717, + "coarse": 44.0582 + }, + { + "fine": 7.8611, + "mid": 15.0121, + "coarse": 44.0138 + }, + { + "fine": 9.4592, + "mid": 15.7361, + "coarse": 44.32 + }, + { + "fine": 8.231, + "mid": 15.4934, + "coarse": 45.6257 + }, + { + "fine": 10.7158, + "mid": 16.2661, + "coarse": 44.8371 + }, + { + "fine": 8.3228, + "mid": 16.0835, + "coarse": 44.8176 + }, + { + "fine": 9.7468, + "mid": 16.6517, + "coarse": 43.697 + }, + { + "fine": 11.4002, + "mid": 17.4228, + "coarse": 42.8519 + } + ], + "face_bands": [ + { + "fine": 11.2739, + "mid": 24.8727, + "coarse": 43.6879 + }, + { + "fine": 10.4153, + "mid": 29.1986, + "coarse": 46.3803 + }, + { + "fine": 13.9289, + "mid": 26.8038, + "coarse": 43.9732 + }, + { + "fine": 15.9681, + "mid": 32.1123, + "coarse": 52.3831 + }, + { + "fine": 19.2415, + "mid": 37.3128, + "coarse": 49.6226 + }, + { + "fine": 21.2007, + "mid": 39.306, + "coarse": 52.7688 + }, + { + "fine": 19.1521, + "mid": 39.919, + "coarse": 61.2013 + }, + { + "fine": 16.0677, + "mid": 39.3492, + "coarse": 58.2799 + } + ], + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 0.66, + "look_ratio": 1.83, + "level_step_db": 19.31, + "typical_delta": 2.0 + }, + { + "seconds": 9.417, + "jump_ratio": 0.74, + "look_ratio": 1.66, + "level_step_db": 22.24, + "typical_delta": 2.0 + }, + { + "seconds": 13.667, + "jump_ratio": 0.74, + "look_ratio": 0.93, + "level_step_db": 16.13, + "typical_delta": 2.0 + }, + { + "seconds": 17.917, + "jump_ratio": 0.86, + "look_ratio": 0.84, + "level_step_db": 5.6, + "typical_delta": 2.0 + }, + { + "seconds": 22.167, + "jump_ratio": 0.84, + "look_ratio": 0.71, + "level_step_db": 14.16, + "typical_delta": 2.0 + }, + { + "seconds": 26.417, + "jump_ratio": 0.89, + "look_ratio": 1.43, + "level_step_db": 14.88, + "typical_delta": 2.0 + }, + { + "seconds": 30.667, + "jump_ratio": 0.96, + "look_ratio": 1.54, + "level_step_db": 1.24, + "typical_delta": 2.0 + }, + { + "seconds": 34.917, + "jump_ratio": 0.88, + "look_ratio": 1.29, + "level_step_db": 12.18, + "typical_delta": 2.0 + }, + { + "seconds": 39.167, + "jump_ratio": 0.99, + "look_ratio": 0.69, + "level_step_db": 22.29, + "typical_delta": 2.0 + }, + { + "seconds": 43.417, + "jump_ratio": 0.83, + "look_ratio": 0.65, + "level_step_db": 0.45, + "typical_delta": 2.0 + }, + { + "seconds": 47.667, + "jump_ratio": 0.73, + "look_ratio": 1.38, + "level_step_db": 24.27, + "typical_delta": 2.0 + }, + { + "seconds": 51.917, + "jump_ratio": 0.87, + "look_ratio": 0.48, + "level_step_db": 7.64, + "typical_delta": 2.0 + }, + { + "seconds": 56.167, + "jump_ratio": 0.9, + "look_ratio": 0.62, + "level_step_db": 10.84, + "typical_delta": 2.0 + }, + { + "seconds": 60.417, + "jump_ratio": 0.97, + "look_ratio": 1.86, + "level_step_db": 18.26, + "typical_delta": 2.0 + }, + { + "seconds": 64.667, + "jump_ratio": 0.88, + "look_ratio": 1.16, + "level_step_db": 17.14, + "typical_delta": 2.0 + }, + { + "seconds": 68.917, + "jump_ratio": 0.9, + "look_ratio": 1.55, + "level_step_db": 6.67, + "typical_delta": 2.0 + }, + { + "seconds": 73.167, + "jump_ratio": 0.86, + "look_ratio": 1.75, + "level_step_db": 1.67, + "typical_delta": 2.0 + }, + { + "seconds": 77.417, + "jump_ratio": 0.9, + "look_ratio": 1.02, + "level_step_db": 2.5, + "typical_delta": 2.0 + }, + { + "seconds": 81.667, + "jump_ratio": 0.86, + "look_ratio": 1.55, + "level_step_db": 0.44, + "typical_delta": 2.0 + }, + { + "seconds": 85.917, + "jump_ratio": 0.73, + "look_ratio": 0.9, + "level_step_db": 18.67, + "typical_delta": 2.0 + }, + { + "seconds": 90.167, + "jump_ratio": 0.85, + "look_ratio": 0.62, + "level_step_db": 1.0, + "typical_delta": 2.0 + }, + { + "seconds": 94.417, + "jump_ratio": 0.77, + "look_ratio": 1.15, + "level_step_db": 21.78, + "typical_delta": 2.0 + }, + { + "seconds": 98.667, + "jump_ratio": 0.77, + "look_ratio": 1.04, + "level_step_db": 0.3, + "typical_delta": 2.0 + }, + { + "seconds": 102.917, + "jump_ratio": 0.84, + "look_ratio": 0.53, + "level_step_db": 3.17, + "typical_delta": 2.0 + }, + { + "seconds": 107.167, + "jump_ratio": 0.79, + "look_ratio": 1.23, + "level_step_db": 4.63, + "typical_delta": 2.0 + }, + { + "seconds": 111.417, + "jump_ratio": 0.83, + "look_ratio": 1.72, + "level_step_db": 7.82, + "typical_delta": 2.0 + }, + { + "seconds": 115.667, + "jump_ratio": 1.06, + "look_ratio": 2.04, + "level_step_db": 23.3, + "typical_delta": 2.0 + }, + { + "seconds": 119.917, + "jump_ratio": 0.72, + "look_ratio": 0.21, + "level_step_db": 35.89, + "typical_delta": 2.0 + } + ], + "motion_mean": 0.5144919157028198, + "audio": { + "duration_s": 120.0, + "segments": [ + { + "start": 0.0, + "end": 5.17, + "rms_db": -57.55, + "hf_share_6k": 0.011843456886708736 + }, + { + "start": 5.17, + "end": 9.42, + "rms_db": -77.1, + "hf_share_6k": 0.12243984639644623 + }, + { + "start": 9.42, + "end": 13.67, + "rms_db": -54.94, + "hf_share_6k": 0.00036300168721936643 + }, + { + "start": 13.67, + "end": 17.92, + "rms_db": -70.55, + "hf_share_6k": 0.12426760047674179 + }, + { + "start": 17.92, + "end": 22.17, + "rms_db": -65.18, + "hf_share_6k": 0.007630344945937395 + }, + { + "start": 22.17, + "end": 26.42, + "rms_db": -79.22, + "hf_share_6k": 0.011168069206178188 + }, + { + "start": 26.42, + "end": 30.67, + "rms_db": -64.46, + "hf_share_6k": 0.05832013487815857 + }, + { + "start": 30.67, + "end": 34.92, + "rms_db": -65.79, + "hf_share_6k": 0.016904214397072792 + }, + { + "start": 34.92, + "end": 39.17, + "rms_db": -78.03, + "hf_share_6k": 0.057504527270793915 + }, + { + "start": 39.17, + "end": 43.42, + "rms_db": -55.54, + "hf_share_6k": 0.10523857921361923 + }, + { + "start": 43.42, + "end": 47.67, + "rms_db": -56.23, + "hf_share_6k": 0.0032371755223721266 + }, + { + "start": 47.67, + "end": 51.92, + "rms_db": -80.12, + "hf_share_6k": 0.041847750544548035 + }, + { + "start": 51.92, + "end": 56.17, + "rms_db": -72.98, + "hf_share_6k": 0.04042164608836174 + }, + { + "start": 56.17, + "end": 60.42, + "rms_db": -83.72, + "hf_share_6k": 0.062484245747327805 + }, + { + "start": 60.42, + "end": 64.67, + "rms_db": -66.3, + "hf_share_6k": 0.032981060445308685 + }, + { + "start": 64.67, + "end": 68.92, + "rms_db": -82.42, + "hf_share_6k": 0.03354005515575409 + }, + { + "start": 68.92, + "end": 73.17, + "rms_db": -75.53, + "hf_share_6k": 0.011786597780883312 + }, + { + "start": 73.17, + "end": 77.42, + "rms_db": -77.21, + "hf_share_6k": 0.010976981371641159 + }, + { + "start": 77.42, + "end": 81.67, + "rms_db": -79.82, + "hf_share_6k": 0.02563386969268322 + }, + { + "start": 81.67, + "end": 85.92, + "rms_db": -80.27, + "hf_share_6k": 0.01281445100903511 + }, + { + "start": 85.92, + "end": 90.17, + "rms_db": -62.1, + "hf_share_6k": 0.02495243027806282 + }, + { + "start": 90.17, + "end": 94.42, + "rms_db": -62.46, + "hf_share_6k": 0.05347543954849243 + }, + { + "start": 94.42, + "end": 98.67, + "rms_db": -84.64, + "hf_share_6k": 0.05301510915160179 + }, + { + "start": 98.67, + "end": 102.92, + "rms_db": -84.41, + "hf_share_6k": 0.06531209498643875 + }, + { + "start": 102.92, + "end": 107.17, + "rms_db": -81.04, + "hf_share_6k": 0.027657784521579742 + }, + { + "start": 107.17, + "end": 111.42, + "rms_db": -76.96, + "hf_share_6k": 0.022183725610375404 + }, + { + "start": 111.42, + "end": 115.67, + "rms_db": -71.03, + "hf_share_6k": 0.009625994600355625 + }, + { + "start": 115.67, + "end": 119.92, + "rms_db": -45.04, + "hf_share_6k": 0.31903576850891113 + } + ], + "rms_db_5s": [ + -57.41, + -77.31, + -55.61, + -65.19, + -81.14, + -65.31, + -66.19, + -78.29, + -56.19, + -57.0, + -73.53, + -83.7, + -66.99, + -81.09, + -75.43, + -78.51, + -80.59, + -62.79, + -63.16, + -84.46, + -85.53, + -76.53, + -71.7, + -45.74 + ] + } + } + }, + "pairs": [ + { + "arm": "e4c7c19a99cd", + "control": "bf9ee32e040f", + "moments": [ + 62, + 300, + 583, + 1000, + 1501, + 2000, + 2500, + 2868 + ], + "psnr_db": [ + "identical", + 22.6, + 17.55, + 16.93, + 15.82, + 14.26, + 12.64, + 12.13 + ], + "ssim": [ + 1.0, + 0.8058, + 0.7111, + 0.6121, + 0.431, + 0.3865, + 0.3288, + 0.2959 + ] + }, + { + "arm": "4cae30688ed4", + "control": "e4c7c19a99cd", + "moments": [ + 62, + 300, + 583, + 1000, + 1501, + 2000, + 2500, + 2868 + ], + "psnr_db": [ + "identical", + 25.48, + 20.18, + 18.86, + 16.66, + 16.83, + 14.08, + 15.69 + ], + "ssim": [ + 1.0, + 0.8231, + 0.6841, + 0.5747, + 0.5726, + 0.4849, + 0.4204, + 0.6922 + ] + }, + { + "arm": "c4ec5a055464", + "control": "8d1614ced27a", + "moments": [ + 62, + 300, + 583, + 1000, + 1501, + 2000, + 2500, + 2868 + ], + "psnr_db": [ + 28.17, + 21.27, + 20.78, + 14.38, + 12.54, + 11.38, + 9.31, + 9.76 + ], + "ssim": [ + 0.8704, + 0.7744, + 0.6881, + 0.2976, + 0.3378, + 0.2415, + 0.2562, + 0.2314 + ] + }, + { + "arm": "c4ec5a055464", + "control": "05a3a55b043c", + "moments": [ + 62, + 300, + 583, + 1000, + 1501, + 2000, + 2500, + 2868 + ], + "psnr_db": [ + 28.17, + 23.83, + 21.54, + 15.26, + 15.46, + 16.59, + 15.27, + 14.29 + ], + "ssim": [ + 0.8704, + 0.8034, + 0.7168, + 0.4948, + 0.4563, + 0.6838, + 0.5446, + 0.4954 + ] + } + ] + }, + "dense": { + "e4c7c19a99cd": { + "moments": [ + 62, + 240, + 360, + 480, + 600, + 720, + 840, + 960, + 1080, + 1200, + 1320, + 1440, + 1560, + 1680, + 1800, + 1920, + 2040, + 2160, + 2280, + 2400, + 2520, + 2640, + 2760, + 2868 + ], + "scene_bands": [ + { + "fine": 6.5502, + "mid": 13.9675, + "coarse": 44.0694 + }, + { + "fine": 7.5176, + "mid": 14.9085, + "coarse": 45.0311 + }, + { + "fine": 8.1561, + "mid": 15.3537, + "coarse": 45.0059 + }, + { + "fine": 8.3431, + "mid": 15.5212, + "coarse": 45.2766 + }, + { + "fine": 8.8406, + "mid": 15.4932, + "coarse": 45.3327 + }, + { + "fine": 9.2666, + "mid": 15.4909, + "coarse": 45.8266 + }, + { + "fine": 9.8821, + "mid": 15.5408, + "coarse": 46.5934 + }, + { + "fine": 10.1789, + "mid": 15.4705, + "coarse": 46.2325 + }, + { + "fine": 10.1469, + "mid": 15.3322, + "coarse": 45.8931 + }, + { + "fine": 10.0842, + "mid": 15.2428, + "coarse": 45.6128 + }, + { + "fine": 9.9543, + "mid": 14.9578, + "coarse": 45.2709 + }, + { + "fine": 7.8758, + "mid": 15.6767, + "coarse": 46.0234 + }, + { + "fine": 7.7335, + "mid": 15.0279, + "coarse": 44.8971 + }, + { + "fine": 7.9231, + "mid": 15.0329, + "coarse": 44.0377 + }, + { + "fine": 7.941, + "mid": 15.055, + "coarse": 43.4936 + }, + { + "fine": 8.0302, + "mid": 14.7967, + "coarse": 43.0802 + }, + { + "fine": 8.1097, + "mid": 14.9345, + "coarse": 42.7809 + }, + { + "fine": 8.1862, + "mid": 14.823, + "coarse": 42.2179 + }, + { + "fine": 8.6545, + "mid": 14.9467, + "coarse": 41.4625 + }, + { + "fine": 8.8834, + "mid": 14.9356, + "coarse": 41.1825 + }, + { + "fine": 8.9672, + "mid": 14.8745, + "coarse": 40.7818 + }, + { + "fine": 9.3523, + "mid": 14.9542, + "coarse": 40.1413 + }, + { + "fine": 7.3531, + "mid": 14.8641, + "coarse": 44.8164 + }, + { + "fine": 7.2855, + "mid": 14.7481, + "coarse": 44.0945 + } + ], + "face_bands": [ + { + "fine": 11.3836, + "mid": 27.014, + "coarse": 43.5128 + }, + { + "fine": 11.7684, + "mid": 23.6838, + "coarse": 47.6845 + }, + { + "fine": 11.5215, + "mid": 25.4834, + "coarse": 48.888 + }, + { + "fine": 12.5074, + "mid": 25.2618, + "coarse": 43.7131 + }, + { + "fine": 12.3944, + "mid": 25.8781, + "coarse": 42.3659 + }, + { + "fine": 11.4556, + "mid": 25.6992, + "coarse": 52.4713 + }, + { + "fine": 13.2095, + "mid": 25.982, + "coarse": 45.9752 + }, + { + "fine": 10.3345, + "mid": 25.3329, + "coarse": 58.8331 + }, + { + "fine": 10.6546, + "mid": 24.7213, + "coarse": 50.7597 + }, + { + "fine": 12.0901, + "mid": 26.1623, + "coarse": 49.1485 + }, + { + "fine": 9.7753, + "mid": 26.9509, + "coarse": 59.557 + }, + { + "fine": 9.2799, + "mid": 23.7854, + "coarse": 53.1063 + }, + { + "fine": 8.0695, + "mid": 23.7598, + "coarse": 61.9621 + }, + { + "fine": 8.7619, + "mid": 25.6284, + "coarse": 56.9348 + }, + { + "fine": 10.6841, + "mid": 27.3201, + "coarse": 53.4946 + }, + { + "fine": 8.0949, + "mid": 25.317, + "coarse": 63.3813 + }, + { + "fine": 8.5697, + "mid": 26.4469, + "coarse": 59.3116 + }, + { + "fine": 7.288, + "mid": 26.0909, + "coarse": 65.2396 + }, + { + "fine": 6.766, + "mid": 22.8205, + "coarse": 65.1102 + }, + { + "fine": 5.2657, + "mid": 20.2214, + "coarse": 67.1702 + }, + { + "fine": 6.9821, + "mid": 24.4499, + "coarse": 60.928 + }, + { + "fine": 6.6089, + "mid": 27.2526, + "coarse": 63.8103 + }, + { + "fine": 5.7107, + "mid": 24.6003, + "coarse": 70.7693 + }, + { + "fine": 5.8601, + "mid": 26.3229, + "coarse": 68.4634 + } + ] + }, + "4cae30688ed4": { + "moments": [ + 62, + 240, + 360, + 480, + 600, + 720, + 840, + 960, + 1080, + 1200, + 1320, + 1440, + 1560, + 1680, + 1800, + 1920, + 2040, + 2160, + 2280, + 2400, + 2520, + 2640, + 2760, + 2868 + ], + "scene_bands": [ + { + "fine": 6.5502, + "mid": 13.9675, + "coarse": 44.0694 + }, + { + "fine": 7.0605, + "mid": 14.6608, + "coarse": 44.504 + }, + { + "fine": 7.412, + "mid": 14.8492, + "coarse": 44.1227 + }, + { + "fine": 7.718, + "mid": 15.1274, + "coarse": 44.2013 + }, + { + "fine": 7.6272, + "mid": 14.934, + "coarse": 43.6354 + }, + { + "fine": 7.6047, + "mid": 14.9157, + "coarse": 43.2498 + }, + { + "fine": 7.7169, + "mid": 14.8002, + "coarse": 43.1584 + }, + { + "fine": 7.4711, + "mid": 14.6094, + "coarse": 42.2928 + }, + { + "fine": 7.1414, + "mid": 14.3131, + "coarse": 41.9097 + }, + { + "fine": 7.2657, + "mid": 14.1037, + "coarse": 41.0839 + }, + { + "fine": 7.1883, + "mid": 13.9163, + "coarse": 40.791 + }, + { + "fine": 7.2176, + "mid": 13.7384, + "coarse": 40.4011 + }, + { + "fine": 7.0138, + "mid": 13.4048, + "coarse": 39.8893 + }, + { + "fine": 6.9865, + "mid": 13.2355, + "coarse": 39.306 + }, + { + "fine": 7.1025, + "mid": 13.2523, + "coarse": 39.1059 + }, + { + "fine": 7.0232, + "mid": 12.8886, + "coarse": 38.766 + }, + { + "fine": 7.2229, + "mid": 12.6795, + "coarse": 38.2596 + }, + { + "fine": 7.3158, + "mid": 12.6136, + "coarse": 38.3276 + }, + { + "fine": 6.9016, + "mid": 13.6664, + "coarse": 43.2267 + }, + { + "fine": 6.6719, + "mid": 13.4021, + "coarse": 42.6694 + }, + { + "fine": 6.651, + "mid": 13.231, + "coarse": 42.3299 + }, + { + "fine": 6.6113, + "mid": 13.1917, + "coarse": 42.027 + }, + { + "fine": 6.7485, + "mid": 13.1049, + "coarse": 41.7824 + }, + { + "fine": 6.7194, + "mid": 13.2187, + "coarse": 41.0519 + } + ], + "face_bands": [ + { + "fine": 11.3836, + "mid": 27.014, + "coarse": 43.5128 + }, + { + "fine": 11.5317, + "mid": 24.4071, + "coarse": 45.8399 + }, + { + "fine": 10.2347, + "mid": 25.8782, + "coarse": 46.9143 + }, + { + "fine": 12.7293, + "mid": 26.4919, + "coarse": 47.6941 + }, + { + "fine": 11.612, + "mid": 25.7776, + "coarse": 49.5274 + }, + { + "fine": 12.247, + "mid": 25.3229, + "coarse": 49.4578 + }, + { + "fine": 12.0181, + "mid": 23.808, + "coarse": 47.0459 + }, + { + "fine": 11.4387, + "mid": 29.0439, + "coarse": 45.0118 + }, + { + "fine": 10.0516, + "mid": 28.93, + "coarse": 48.9923 + }, + { + "fine": 11.2132, + "mid": 29.8768, + "coarse": 49.5136 + }, + { + "fine": 13.1179, + "mid": 26.9514, + "coarse": 47.5462 + }, + { + "fine": 9.9183, + "mid": 26.5146, + "coarse": 57.9327 + }, + { + "fine": 9.7195, + "mid": 23.1096, + "coarse": 51.5417 + }, + { + "fine": 8.4492, + "mid": 25.8812, + "coarse": 59.4646 + }, + { + "fine": 11.1024, + "mid": 29.1461, + "coarse": 50.5056 + }, + { + "fine": 11.7825, + "mid": 29.4336, + "coarse": 46.7748 + }, + { + "fine": 8.4145, + "mid": 27.6814, + "coarse": 58.4879 + }, + { + "fine": 10.3089, + "mid": 26.2558, + "coarse": 54.0709 + }, + { + "fine": 6.765, + "mid": 26.0881, + "coarse": 58.4888 + }, + { + "fine": 8.9433, + "mid": 27.7936, + "coarse": 56.1947 + }, + { + "fine": 8.7307, + "mid": 28.1045, + "coarse": 58.3537 + }, + { + "fine": 8.8495, + "mid": 27.9322, + "coarse": 61.0159 + }, + { + "fine": 10.1609, + "mid": 28.6951, + "coarse": 56.1896 + }, + { + "fine": 7.867, + "mid": 26.3211, + "coarse": 62.4211 + } + ] + }, + "c4ec5a055464": { + "moments": [ + 62, + 240, + 360, + 480, + 600, + 720, + 840, + 960, + 1080, + 1200, + 1320, + 1440, + 1560, + 1680, + 1800, + 1920, + 2040, + 2160, + 2280, + 2400, + 2520, + 2640, + 2760, + 2868 + ], + "scene_bands": [ + { + "fine": 6.7328, + "mid": 14.2717, + "coarse": 44.0582 + }, + { + "fine": 7.7361, + "mid": 15.0128, + "coarse": 44.1882 + }, + { + "fine": 8.27, + "mid": 15.482, + "coarse": 44.3163 + }, + { + "fine": 8.7253, + "mid": 15.7091, + "coarse": 44.4866 + }, + { + "fine": 9.4508, + "mid": 15.6594, + "coarse": 44.1681 + }, + { + "fine": 10.4052, + "mid": 16.0421, + "coarse": 43.9055 + }, + { + "fine": 12.9096, + "mid": 16.3959, + "coarse": 42.9772 + }, + { + "fine": 8.3728, + "mid": 15.5028, + "coarse": 45.9355 + }, + { + "fine": 8.4377, + "mid": 15.5626, + "coarse": 45.8275 + }, + { + "fine": 8.7333, + "mid": 15.575, + "coarse": 45.7748 + }, + { + "fine": 9.1666, + "mid": 15.8834, + "coarse": 45.6098 + }, + { + "fine": 9.9405, + "mid": 16.0812, + "coarse": 45.1711 + }, + { + "fine": 11.5562, + "mid": 16.4364, + "coarse": 43.7525 + }, + { + "fine": 13.0407, + "mid": 17.1073, + "coarse": 43.2897 + }, + { + "fine": 14.2079, + "mid": 17.6026, + "coarse": 42.7049 + }, + { + "fine": 12.5293, + "mid": 17.8783, + "coarse": 45.7655 + }, + { + "fine": 8.069, + "mid": 15.791, + "coarse": 44.5332 + }, + { + "fine": 8.2719, + "mid": 16.0954, + "coarse": 44.5068 + }, + { + "fine": 9.543, + "mid": 16.9544, + "coarse": 43.9977 + }, + { + "fine": 9.0154, + "mid": 17.1288, + "coarse": 44.1443 + }, + { + "fine": 9.9048, + "mid": 17.1625, + "coarse": 43.8327 + }, + { + "fine": 9.8744, + "mid": 16.7441, + "coarse": 43.973 + }, + { + "fine": 10.3861, + "mid": 16.8091, + "coarse": 43.1913 + }, + { + "fine": 11.4002, + "mid": 17.4228, + "coarse": 42.8519 + } + ], + "face_bands": [ + { + "fine": 11.2739, + "mid": 24.8727, + "coarse": 43.6879 + }, + { + "fine": 12.4732, + "mid": 23.6703, + "coarse": 46.1187 + }, + { + "fine": 13.2088, + "mid": 25.8428, + "coarse": 47.3251 + }, + { + "fine": 13.5228, + "mid": 26.5596, + "coarse": 45.9634 + }, + { + "fine": 14.8553, + "mid": 28.3702, + "coarse": 46.8623 + }, + { + "fine": 17.3743, + "mid": 28.9181, + "coarse": 44.3247 + }, + { + "fine": 15.7039, + "mid": 30.4025, + "coarse": 53.0534 + }, + { + "fine": 18.8063, + "mid": 30.4869, + "coarse": 46.9238 + }, + { + "fine": 17.8046, + "mid": 32.7017, + "coarse": 45.3888 + }, + { + "fine": 17.7997, + "mid": 33.1643, + "coarse": 51.3079 + }, + { + "fine": 19.7637, + "mid": 33.4957, + "coarse": 50.6966 + }, + { + "fine": 17.4186, + "mid": 33.821, + "coarse": 56.3268 + }, + { + "fine": 21.4021, + "mid": 37.4117, + "coarse": 53.225 + }, + { + "fine": 19.2913, + "mid": 38.8955, + "coarse": 56.2381 + }, + { + "fine": 23.2469, + "mid": 40.5477, + "coarse": 52.0582 + }, + { + "fine": 20.9877, + "mid": 37.828, + "coarse": 55.1533 + }, + { + "fine": 19.0764, + "mid": 36.9065, + "coarse": 61.9935 + }, + { + "fine": 21.7049, + "mid": 37.0745, + "coarse": 59.5012 + }, + { + "fine": 18.9588, + "mid": 38.6212, + "coarse": 63.3387 + }, + { + "fine": 21.3107, + "mid": 41.2983, + "coarse": 58.0093 + }, + { + "fine": 20.5249, + "mid": 43.6752, + "coarse": 54.9704 + }, + { + "fine": 18.3695, + "mid": 41.4003, + "coarse": 61.3097 + }, + { + "fine": 19.267, + "mid": 42.0894, + "coarse": 57.709 + }, + { + "fine": 16.0677, + "mid": 39.3492, + "coarse": 58.2799 + } + ] + }, + "bf9ee32e040f": { + "moments": [ + 62, + 240, + 360, + 480, + 600, + 720, + 840, + 960, + 1080, + 1200, + 1320, + 1440, + 1560, + 1680, + 1800, + 1920, + 2040, + 2160, + 2280, + 2400, + 2520, + 2640, + 2760, + 2868 + ], + "scene_bands": [ + { + "fine": 6.5502, + "mid": 13.9675, + "coarse": 44.0694 + }, + { + "fine": 7.5475, + "mid": 14.9478, + "coarse": 44.7523 + }, + { + "fine": 8.066, + "mid": 15.3368, + "coarse": 44.9085 + }, + { + "fine": 8.4556, + "mid": 15.7182, + "coarse": 45.3898 + }, + { + "fine": 8.9711, + "mid": 15.9095, + "coarse": 45.2771 + }, + { + "fine": 9.6379, + "mid": 16.204, + "coarse": 45.8238 + }, + { + "fine": 11.1114, + "mid": 16.8634, + "coarse": 46.0208 + }, + { + "fine": 12.7484, + "mid": 17.4859, + "coarse": 45.5968 + }, + { + "fine": 12.8148, + "mid": 17.4817, + "coarse": 45.2231 + }, + { + "fine": 14.3712, + "mid": 18.2583, + "coarse": 45.3144 + }, + { + "fine": 15.0543, + "mid": 18.5654, + "coarse": 45.6019 + }, + { + "fine": 15.4314, + "mid": 18.8278, + "coarse": 45.5175 + }, + { + "fine": 15.4088, + "mid": 18.7654, + "coarse": 44.3662 + }, + { + "fine": 15.7877, + "mid": 19.1763, + "coarse": 43.9725 + }, + { + "fine": 15.846, + "mid": 19.7149, + "coarse": 43.8351 + }, + { + "fine": 15.3492, + "mid": 19.4747, + "coarse": 43.2873 + }, + { + "fine": 16.116, + "mid": 19.628, + "coarse": 43.0213 + }, + { + "fine": 16.5023, + "mid": 20.3351, + "coarse": 42.8898 + }, + { + "fine": 16.5616, + "mid": 20.1812, + "coarse": 42.7933 + }, + { + "fine": 17.6009, + "mid": 21.2611, + "coarse": 43.2708 + }, + { + "fine": 19.9584, + "mid": 23.4282, + "coarse": 44.307 + }, + { + "fine": 22.2336, + "mid": 26.2038, + "coarse": 45.8946 + }, + { + "fine": 21.2953, + "mid": 25.5661, + "coarse": 44.9508 + }, + { + "fine": 20.5237, + "mid": 25.1906, + "coarse": 43.3242 + } + ], + "face_bands": [ + { + "fine": 11.3836, + "mid": 27.014, + "coarse": 43.5128 + }, + { + "fine": 11.5075, + "mid": 23.0052, + "coarse": 44.468 + }, + { + "fine": 12.1512, + "mid": 24.0232, + "coarse": 46.2269 + }, + { + "fine": 11.2591, + "mid": 24.5988, + "coarse": 49.2384 + }, + { + "fine": 12.8054, + "mid": 25.8933, + "coarse": 49.7988 + }, + { + "fine": 12.8727, + "mid": 23.9062, + "coarse": 46.5163 + }, + { + "fine": 14.3343, + "mid": 22.6444, + "coarse": 48.0484 + }, + { + "fine": 14.2461, + "mid": 22.7616, + "coarse": 49.2448 + }, + { + "fine": 14.5339, + "mid": 24.1368, + "coarse": 50.3755 + }, + { + "fine": 13.2678, + "mid": 25.3695, + "coarse": 53.4817 + }, + { + "fine": 11.1456, + "mid": 24.9735, + "coarse": 54.6966 + }, + { + "fine": 12.8409, + "mid": 25.5902, + "coarse": 56.673 + }, + { + "fine": 11.4322, + "mid": 26.8681, + "coarse": 53.5904 + }, + { + "fine": 12.1086, + "mid": 25.7345, + "coarse": 55.267 + }, + { + "fine": 13.7351, + "mid": 30.0419, + "coarse": 58.3379 + }, + { + "fine": 12.6066, + "mid": 29.0419, + "coarse": 57.4811 + }, + { + "fine": 13.7691, + "mid": 29.6277, + "coarse": 59.8788 + }, + { + "fine": 11.7957, + "mid": 31.8179, + "coarse": 68.7599 + }, + { + "fine": 10.0098, + "mid": 28.7816, + "coarse": 69.577 + }, + { + "fine": 10.9605, + "mid": 29.3041, + "coarse": 69.223 + }, + { + "fine": 9.255, + "mid": 29.6671, + "coarse": 69.4257 + }, + { + "fine": 7.3116, + "mid": 30.4881, + "coarse": 65.1744 + }, + { + "fine": 7.8411, + "mid": 28.3085, + "coarse": 61.3059 + }, + { + "fine": 7.1348, + "mid": 26.8163, + "coarse": 59.662 + } + ] + }, + "8d1614ced27a": { + "moments": [ + 62, + 240, + 360, + 480, + 600, + 720, + 840, + 960, + 1080, + 1200, + 1320, + 1440, + 1560, + 1680, + 1800, + 1920, + 2040, + 2160, + 2280, + 2400, + 2520, + 2640, + 2760, + 2868 + ], + "scene_bands": [ + { + "fine": 6.5469, + "mid": 13.9587, + "coarse": 43.751 + }, + { + "fine": 7.7343, + "mid": 14.8711, + "coarse": 44.0688 + }, + { + "fine": 8.5727, + "mid": 15.4568, + "coarse": 43.8455 + }, + { + "fine": 9.5824, + "mid": 16.1362, + "coarse": 44.0429 + }, + { + "fine": 11.3943, + "mid": 16.7359, + "coarse": 44.4676 + }, + { + "fine": 12.7076, + "mid": 17.1773, + "coarse": 43.7178 + }, + { + "fine": 17.0776, + "mid": 18.9218, + "coarse": 42.4921 + }, + { + "fine": 20.3964, + "mid": 20.6577, + "coarse": 41.4736 + }, + { + "fine": 22.4625, + "mid": 22.4448, + "coarse": 40.085 + }, + { + "fine": 24.7308, + "mid": 23.6637, + "coarse": 38.6683 + }, + { + "fine": 26.8329, + "mid": 25.1948, + "coarse": 36.8926 + }, + { + "fine": 28.7427, + "mid": 26.749, + "coarse": 35.3158 + }, + { + "fine": 25.728, + "mid": 28.0623, + "coarse": 39.5888 + }, + { + "fine": 27.925, + "mid": 29.662, + "coarse": 38.3007 + }, + { + "fine": 29.7112, + "mid": 30.6813, + "coarse": 36.809 + }, + { + "fine": 32.372, + "mid": 38.7849, + "coarse": 41.1708 + }, + { + "fine": 37.1066, + "mid": 41.0246, + "coarse": 40.3094 + }, + { + "fine": 34.4438, + "mid": 40.9687, + "coarse": 39.6262 + }, + { + "fine": 30.5368, + "mid": 44.9254, + "coarse": 56.7366 + }, + { + "fine": 34.1628, + "mid": 44.3136, + "coarse": 54.7608 + }, + { + "fine": 34.5597, + "mid": 44.3238, + "coarse": 54.0475 + }, + { + "fine": 35.1729, + "mid": 44.2578, + "coarse": 52.9529 + }, + { + "fine": 35.9122, + "mid": 45.0812, + "coarse": 49.4709 + }, + { + "fine": 36.0535, + "mid": 44.9993, + "coarse": 47.7892 + } + ], + "face_bands": [ + { + "fine": 10.8423, + "mid": 24.3037, + "coarse": 43.8552 + }, + { + "fine": 11.7051, + "mid": 25.2251, + "coarse": 46.1623 + }, + { + "fine": 12.8605, + "mid": 27.6211, + "coarse": 48.1243 + }, + { + "fine": 13.5113, + "mid": 28.7635, + "coarse": 47.4964 + }, + { + "fine": 16.6941, + "mid": 33.1914, + "coarse": 49.3747 + }, + { + "fine": 15.8103, + "mid": 34.1358, + "coarse": 49.6035 + }, + { + "fine": 19.0185, + "mid": 32.5549, + "coarse": 47.3272 + }, + { + "fine": 18.7315, + "mid": 39.5319, + "coarse": 53.3768 + }, + { + "fine": 20.9601, + "mid": 40.2329, + "coarse": 48.7309 + }, + { + "fine": 20.5699, + "mid": 40.3404, + "coarse": 51.5157 + }, + { + "fine": 23.147, + "mid": 36.8748, + "coarse": 47.0343 + }, + { + "fine": 21.4884, + "mid": 35.0008, + "coarse": 45.9122 + }, + { + "fine": 21.5509, + "mid": 34.7261, + "coarse": 47.6889 + }, + { + "fine": 22.9084, + "mid": 36.3593, + "coarse": 47.0018 + }, + { + "fine": 19.1408, + "mid": 36.3692, + "coarse": 48.2718 + }, + { + "fine": 24.5821, + "mid": 38.4961, + "coarse": 61.0493 + }, + { + "fine": 32.4521, + "mid": 37.9105, + "coarse": 58.7558 + }, + { + "fine": 25.8567, + "mid": 38.2651, + "coarse": 55.8234 + }, + { + "fine": 23.3104, + "mid": 37.8212, + "coarse": 52.8372 + }, + { + "fine": 24.2356, + "mid": 35.4008, + "coarse": 60.5882 + }, + { + "fine": 22.0316, + "mid": 34.0562, + "coarse": 62.5328 + }, + { + "fine": 25.8394, + "mid": 35.5154, + "coarse": 51.7633 + }, + { + "fine": 21.5742, + "mid": 34.8215, + "coarse": 48.562 + }, + { + "fine": 22.0642, + "mid": 36.4137, + "coarse": 49.9593 + } + ] + }, + "05a3a55b043c": { + "moments": [ + 62, + 240, + 360, + 480, + 600, + 720, + 840, + 960, + 1080, + 1200, + 1320, + 1440, + 1560, + 1680, + 1800, + 1920, + 2040, + 2160, + 2280, + 2400, + 2520, + 2640, + 2760, + 2868 + ], + "scene_bands": [ + { + "fine": 6.5469, + "mid": 13.9587, + "coarse": 43.751 + }, + { + "fine": 7.4871, + "mid": 14.7773, + "coarse": 44.2124 + }, + { + "fine": 8.1119, + "mid": 15.3099, + "coarse": 44.331 + }, + { + "fine": 8.496, + "mid": 15.6041, + "coarse": 44.8306 + }, + { + "fine": 8.7149, + "mid": 15.6141, + "coarse": 45.4282 + }, + { + "fine": 9.4531, + "mid": 15.8436, + "coarse": 45.7554 + }, + { + "fine": 9.873, + "mid": 15.9943, + "coarse": 46.0213 + }, + { + "fine": 10.4594, + "mid": 16.1541, + "coarse": 46.0448 + }, + { + "fine": 9.9743, + "mid": 16.5434, + "coarse": 47.7794 + }, + { + "fine": 9.8643, + "mid": 16.0281, + "coarse": 47.2669 + }, + { + "fine": 10.0189, + "mid": 15.7649, + "coarse": 46.7288 + }, + { + "fine": 10.2897, + "mid": 15.5604, + "coarse": 46.2562 + }, + { + "fine": 10.0725, + "mid": 14.8972, + "coarse": 45.1487 + }, + { + "fine": 10.8271, + "mid": 15.0655, + "coarse": 45.2381 + }, + { + "fine": 11.4073, + "mid": 15.0734, + "coarse": 44.784 + }, + { + "fine": 9.4945, + "mid": 15.3379, + "coarse": 46.9439 + }, + { + "fine": 7.4269, + "mid": 15.0966, + "coarse": 45.1705 + }, + { + "fine": 7.2878, + "mid": 14.989, + "coarse": 44.9404 + }, + { + "fine": 7.1801, + "mid": 15.1647, + "coarse": 44.3627 + }, + { + "fine": 7.2722, + "mid": 15.0757, + "coarse": 43.749 + }, + { + "fine": 7.3911, + "mid": 14.8629, + "coarse": 43.4037 + }, + { + "fine": 7.5295, + "mid": 14.8676, + "coarse": 43.1817 + }, + { + "fine": 6.7523, + "mid": 14.1691, + "coarse": 43.304 + }, + { + "fine": 6.7397, + "mid": 14.1825, + "coarse": 43.066 + } + ], + "face_bands": [ + { + "fine": 10.8423, + "mid": 24.3037, + "coarse": 43.8552 + }, + { + "fine": 10.8504, + "mid": 23.9379, + "coarse": 45.4638 + }, + { + "fine": 12.2824, + "mid": 23.1937, + "coarse": 44.8839 + }, + { + "fine": 12.4147, + "mid": 23.3812, + "coarse": 44.1829 + }, + { + "fine": 12.9791, + "mid": 25.6452, + "coarse": 43.5174 + }, + { + "fine": 12.9004, + "mid": 25.743, + "coarse": 51.717 + }, + { + "fine": 12.2154, + "mid": 23.229, + "coarse": 43.4166 + }, + { + "fine": 9.7149, + "mid": 27.089, + "coarse": 53.4961 + }, + { + "fine": 10.4636, + "mid": 24.9364, + "coarse": 47.6209 + }, + { + "fine": 11.0993, + "mid": 27.6097, + "coarse": 48.1455 + }, + { + "fine": 9.7015, + "mid": 27.5456, + "coarse": 57.1423 + }, + { + "fine": 10.9769, + "mid": 25.314, + "coarse": 51.1569 + }, + { + "fine": 8.835, + "mid": 26.7386, + "coarse": 57.9696 + }, + { + "fine": 10.1043, + "mid": 28.8074, + "coarse": 54.8326 + }, + { + "fine": 10.5659, + "mid": 32.4696, + "coarse": 53.3784 + }, + { + "fine": 8.1327, + "mid": 31.9391, + "coarse": 57.7116 + }, + { + "fine": 9.6382, + "mid": 33.3699, + "coarse": 55.0068 + }, + { + "fine": 8.804, + "mid": 31.9185, + "coarse": 59.1838 + }, + { + "fine": 9.5333, + "mid": 32.8071, + "coarse": 54.3404 + }, + { + "fine": 8.2948, + "mid": 33.7, + "coarse": 54.4019 + }, + { + "fine": 7.8015, + "mid": 33.7692, + "coarse": 55.4998 + }, + { + "fine": 8.8921, + "mid": 36.8769, + "coarse": 53.7667 + }, + { + "fine": 7.9878, + "mid": 32.3186, + "coarse": 55.16 + }, + { + "fine": 8.6457, + "mid": 36.9478, + "coarse": 53.5859 + } + ] + }, + "4766787e793d": { + "moments": [ + 62, + 240, + 360, + 480, + 600, + 720, + 840, + 960, + 1080, + 1200, + 1320, + 1440, + 1560, + 1680, + 1800, + 1920, + 2040, + 2160, + 2280, + 2400, + 2520, + 2640, + 2760, + 2868 + ], + "scene_bands": [ + { + "fine": 6.5469, + "mid": 13.9587, + "coarse": 43.751 + }, + { + "fine": 7.1534, + "mid": 14.6632, + "coarse": 43.8825 + }, + { + "fine": 7.2997, + "mid": 14.743, + "coarse": 43.3354 + }, + { + "fine": 7.5439, + "mid": 14.825, + "coarse": 43.1798 + }, + { + "fine": 7.3138, + "mid": 14.4905, + "coarse": 42.6521 + }, + { + "fine": 7.2501, + "mid": 14.3231, + "coarse": 42.1662 + }, + { + "fine": 7.0359, + "mid": 14.0075, + "coarse": 41.8366 + }, + { + "fine": 7.0338, + "mid": 13.6389, + "coarse": 40.8614 + }, + { + "fine": 6.9526, + "mid": 13.6167, + "coarse": 40.4696 + }, + { + "fine": 6.7264, + "mid": 13.1874, + "coarse": 39.8538 + }, + { + "fine": 6.7437, + "mid": 13.0927, + "coarse": 39.2148 + }, + { + "fine": 6.7478, + "mid": 12.805, + "coarse": 38.8495 + }, + { + "fine": 6.7903, + "mid": 12.2281, + "coarse": 38.383 + }, + { + "fine": 6.7997, + "mid": 12.1002, + "coarse": 38.1414 + }, + { + "fine": 6.8404, + "mid": 12.0137, + "coarse": 37.8266 + }, + { + "fine": 6.8778, + "mid": 11.7923, + "coarse": 37.6237 + }, + { + "fine": 6.8777, + "mid": 11.5699, + "coarse": 37.1558 + }, + { + "fine": 6.8138, + "mid": 11.4542, + "coarse": 37.1967 + }, + { + "fine": 6.9815, + "mid": 11.7738, + "coarse": 37.1276 + }, + { + "fine": 7.0982, + "mid": 11.5515, + "coarse": 36.3553 + }, + { + "fine": 7.0193, + "mid": 13.8188, + "coarse": 43.4527 + }, + { + "fine": 7.0326, + "mid": 13.9037, + "coarse": 43.4521 + }, + { + "fine": 7.0906, + "mid": 13.7776, + "coarse": 42.9143 + }, + { + "fine": 6.9428, + "mid": 13.8349, + "coarse": 42.5374 + } + ], + "face_bands": [ + { + "fine": 10.8423, + "mid": 24.3037, + "coarse": 43.8552 + }, + { + "fine": 11.1618, + "mid": 24.8639, + "coarse": 45.4462 + }, + { + "fine": 12.6839, + "mid": 25.0636, + "coarse": 47.0915 + }, + { + "fine": 11.2919, + "mid": 23.2273, + "coarse": 45.6214 + }, + { + "fine": 11.4721, + "mid": 25.9158, + "coarse": 49.509 + }, + { + "fine": 13.0763, + "mid": 25.7345, + "coarse": 48.8711 + }, + { + "fine": 13.3302, + "mid": 25.4542, + "coarse": 47.3817 + }, + { + "fine": 11.3164, + "mid": 28.3949, + "coarse": 48.0944 + }, + { + "fine": 13.0116, + "mid": 25.9044, + "coarse": 45.5366 + }, + { + "fine": 12.9802, + "mid": 27.9434, + "coarse": 45.1127 + }, + { + "fine": 10.9308, + "mid": 26.3457, + "coarse": 55.449 + }, + { + "fine": 12.0412, + "mid": 24.8917, + "coarse": 48.7506 + }, + { + "fine": 9.8383, + "mid": 25.8198, + "coarse": 56.7525 + }, + { + "fine": 11.0903, + "mid": 26.1148, + "coarse": 51.6231 + }, + { + "fine": 12.1826, + "mid": 30.4792, + "coarse": 49.5976 + }, + { + "fine": 9.0705, + "mid": 28.0606, + "coarse": 60.0891 + }, + { + "fine": 11.2841, + "mid": 30.4994, + "coarse": 55.1425 + }, + { + "fine": 8.9295, + "mid": 28.2138, + "coarse": 61.1357 + }, + { + "fine": 10.2177, + "mid": 30.1422, + "coarse": 54.5577 + }, + { + "fine": 10.1574, + "mid": 30.377, + "coarse": 62.1976 + }, + { + "fine": 9.8789, + "mid": 30.4183, + "coarse": 61.2313 + }, + { + "fine": 12.0424, + "mid": 32.4042, + "coarse": 57.3253 + }, + { + "fine": 8.3942, + "mid": 28.8518, + "coarse": 65.1108 + }, + { + "fine": 9.7504, + "mid": 31.5128, + "coarse": 61.9121 + } + ] + }, + "25f6bd1da507": { + "moments": [ + 62, + 240, + 360, + 480, + 600, + 720, + 840, + 960, + 1080, + 1200, + 1320, + 1440, + 1560, + 1680, + 1800, + 1920, + 2040, + 2160, + 2280, + 2400, + 2520, + 2640, + 2760, + 2868 + ], + "scene_bands": [ + { + "fine": 6.5469, + "mid": 13.9587, + "coarse": 43.751 + }, + { + "fine": 7.6174, + "mid": 14.6499, + "coarse": 44.1026 + }, + { + "fine": 8.1312, + "mid": 15.019, + "coarse": 43.7938 + }, + { + "fine": 8.7302, + "mid": 15.6022, + "coarse": 44.3051 + }, + { + "fine": 9.4909, + "mid": 15.6902, + "coarse": 44.2618 + }, + { + "fine": 9.7332, + "mid": 15.7129, + "coarse": 44.4359 + }, + { + "fine": 10.5127, + "mid": 16.1814, + "coarse": 45.0037 + }, + { + "fine": 10.6985, + "mid": 16.5291, + "coarse": 45.1119 + }, + { + "fine": 10.3671, + "mid": 16.6097, + "coarse": 44.7391 + }, + { + "fine": 11.2333, + "mid": 17.1372, + "coarse": 45.3173 + }, + { + "fine": 11.7053, + "mid": 17.1056, + "coarse": 44.8181 + }, + { + "fine": 12.3467, + "mid": 17.3386, + "coarse": 44.8228 + }, + { + "fine": 14.2703, + "mid": 17.9135, + "coarse": 45.8715 + }, + { + "fine": 15.9362, + "mid": 18.571, + "coarse": 45.3768 + }, + { + "fine": 15.8735, + "mid": 18.4899, + "coarse": 44.8672 + }, + { + "fine": 16.2972, + "mid": 18.5563, + "coarse": 44.5837 + }, + { + "fine": 16.4066, + "mid": 18.621, + "coarse": 43.9071 + }, + { + "fine": 16.2749, + "mid": 18.6942, + "coarse": 44.0047 + }, + { + "fine": 16.4257, + "mid": 19.137, + "coarse": 43.521 + }, + { + "fine": 18.2016, + "mid": 20.8703, + "coarse": 47.4986 + }, + { + "fine": 18.7417, + "mid": 20.2637, + "coarse": 46.026 + }, + { + "fine": 18.2581, + "mid": 20.0157, + "coarse": 44.9314 + }, + { + "fine": 19.8345, + "mid": 21.2924, + "coarse": 45.9072 + }, + { + "fine": 19.8181, + "mid": 21.4665, + "coarse": 44.6048 + } + ], + "face_bands": [ + { + "fine": 10.8423, + "mid": 24.3037, + "coarse": 43.8552 + }, + { + "fine": 11.0227, + "mid": 24.6281, + "coarse": 46.1902 + }, + { + "fine": 11.3286, + "mid": 26.2407, + "coarse": 46.373 + }, + { + "fine": 11.5394, + "mid": 29.0044, + "coarse": 48.6427 + }, + { + "fine": 12.2565, + "mid": 28.8801, + "coarse": 49.2048 + }, + { + "fine": 12.7971, + "mid": 29.8934, + "coarse": 50.2215 + }, + { + "fine": 12.5649, + "mid": 25.5327, + "coarse": 47.9233 + }, + { + "fine": 12.7268, + "mid": 29.5386, + "coarse": 56.0975 + }, + { + "fine": 11.8913, + "mid": 28.1964, + "coarse": 58.8573 + }, + { + "fine": 11.4527, + "mid": 25.9471, + "coarse": 59.6998 + }, + { + "fine": 12.3099, + "mid": 32.3662, + "coarse": 56.7448 + }, + { + "fine": 12.2035, + "mid": 28.9148, + "coarse": 59.8213 + }, + { + "fine": 10.9836, + "mid": 23.7886, + "coarse": 56.3939 + }, + { + "fine": 14.5048, + "mid": 31.8515, + "coarse": 59.7472 + }, + { + "fine": 11.923, + "mid": 27.0301, + "coarse": 63.6184 + }, + { + "fine": 10.7609, + "mid": 25.9312, + "coarse": 63.7148 + }, + { + "fine": 10.4319, + "mid": 23.8213, + "coarse": 62.7373 + }, + { + "fine": 9.2147, + "mid": 24.0869, + "coarse": 61.8828 + }, + { + "fine": 9.766, + "mid": 23.7743, + "coarse": 64.6702 + }, + { + "fine": 11.8476, + "mid": 26.5174, + "coarse": 65.661 + }, + { + "fine": 11.0725, + "mid": 25.4834, + "coarse": 65.144 + }, + { + "fine": 11.5228, + "mid": 24.5725, + "coarse": 62.1102 + }, + { + "fine": 11.6503, + "mid": 24.6299, + "coarse": 61.004 + }, + { + "fine": 10.4146, + "mid": 24.0581, + "coarse": 61.1591 + } + ] + } + } +} \ No newline at end of file diff --git a/bench/records/keystone-confirm.json b/bench/records/keystone-confirm.json new file mode 100644 index 000000000..d16478fff --- /dev/null +++ b/bench/records/keystone-confirm.json @@ -0,0 +1,65 @@ +[ + { + "arm": "ks-pixel-anchor25-s1-120s-s2", + "scene": "s1-static", + "seed": 2, + "job_id": "e4c7c19a99cd", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-ksp/20261106-011016-6b78f214/final/BenchS1Static-ksp-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.8489285714285714, + "seam_jump_max": 1.25, + "seam_look_mean": 1.0142857142857142, + "seam_look_max": 2.28, + "seam_level_step_db_mean": 11.245714285714286, + "seam_level_step_db_max": 27.03, + "texture_fine_seg_last": 0.10639048793963041, + "texture_mid_seg_last": 0.054381058578161, + "texture_coarse_seg_last": 0.0005693940644215875, + "wall_minutes": 72.0299093166987 + } + }, + { + "arm": "ks-lc-anchor25-s1-120s-s2", + "scene": "s1-static", + "seed": 2, + "job_id": "4cae30688ed4", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-ksl/20261106-011016-879ed22c/final/BenchS1Static-ksl-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.8049999999999999, + "seam_jump_max": 1.15, + "seam_look_mean": 1.1975, + "seam_look_max": 2.26, + "seam_level_step_db_mean": 15.048928571428572, + "seam_level_step_db_max": 39.21, + "texture_fine_seg_last": 0.025503281302423943, + "texture_mid_seg_last": 0.0551007086176792, + "texture_coarse_seg_last": 0.07092854421174521, + "wall_minutes": 67.91400047540665 + } + }, + { + "arm": "ks-pixel-noanchor-s1-120s-s1166", + "scene": "s1-static", + "seed": 1166, + "job_id": "c4ec5a055464", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-kspna/20261106-021809-6b827245/final/BenchS1Static-kspna-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.8435714285714286, + "seam_jump_max": 1.06, + "seam_look_mean": 1.1564285714285716, + "seam_look_max": 2.04, + "seam_level_step_db_mean": 11.945357142857144, + "seam_level_step_db_max": 35.89, + "texture_fine_seg_last": 0.5266397944250629, + "texture_mid_seg_last": 0.19950113785815105, + "texture_coarse_seg_last": 0.027761502373700876, + "wall_minutes": 53.02445907592774 + } + } +] \ No newline at end of file diff --git a/bench/records/keystone-last-noprompt.json b/bench/records/keystone-last-noprompt.json new file mode 100644 index 000000000..90f7fddeb --- /dev/null +++ b/bench/records/keystone-last-noprompt.json @@ -0,0 +1,44 @@ +[ + { + "arm": "ksr-last-noprompt-s1166", + "scene": "s1-static", + "seed": 1166, + "job_id": "f55bd445faa5", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-ksrln/20261106-082208-559feb76/final/BenchS1Static-ksrln-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.6889285714285714, + "seam_jump_max": 1.17, + "seam_look_mean": 1.4964285714285714, + "seam_look_max": 2.3, + "seam_level_step_db_mean": 5.934642857142856, + "seam_level_step_db_max": 29.53, + "texture_fine_seg_last": 0.08531495387235152, + "texture_mid_seg_last": 0.004894687442859967, + "texture_coarse_seg_last": 0.005889746054885112, + "wall_minutes": 57.38123092651367 + } + }, + { + "arm": "ksr-last-noprompt-s2", + "scene": "s1-static", + "seed": 2, + "job_id": "00ed8fb067ca", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-ksrln/20261106-082208-22e14837/final/BenchS1Static-ksrln-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.6317857142857143, + "seam_jump_max": 0.84, + "seam_look_mean": 1.3571428571428572, + "seam_look_max": 2.02, + "seam_level_step_db_mean": 12.421428571428573, + "seam_level_step_db_max": 38.22, + "texture_fine_seg_last": 0.12951372030777983, + "texture_mid_seg_last": 0.00814771176519055, + "texture_coarse_seg_last": 0.0030088196501386055, + "wall_minutes": 56.87556756337484 + } + } +] \ No newline at end of file diff --git a/bench/records/keystone-regular.json b/bench/records/keystone-regular.json new file mode 100644 index 000000000..07b5ffc0c --- /dev/null +++ b/bench/records/keystone-regular.json @@ -0,0 +1,128 @@ +[ + { + "arm": "ksr-last-s1166", + "scene": "s1-static", + "seed": 1166, + "job_id": "3232cabd16b1", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-ksrl/20261106-045908-fb21ec84/final/BenchS1Static-ksrl-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.7371428571428572, + "seam_jump_max": 1.5, + "seam_look_mean": 1.6085714285714285, + "seam_look_max": 2.78, + "seam_level_step_db_mean": 14.6675, + "seam_level_step_db_max": 32.75, + "texture_fine_seg_last": 0.09825103418866163, + "texture_mid_seg_last": 0.017033355435932963, + "texture_coarse_seg_last": 0.005261442298246376, + "wall_minutes": 61.90312643448512 + } + }, + { + "arm": "ksr-last-s2", + "scene": "s1-static", + "seed": 2, + "job_id": "94c5c0b219b3", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-ksrl/20261106-045908-a76d22f9/final/BenchS1Static-ksrl-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.6760714285714285, + "seam_jump_max": 1.13, + "seam_look_mean": 1.5896428571428571, + "seam_look_max": 2.53, + "seam_level_step_db_mean": 10.567499999999999, + "seam_level_step_db_max": 30.49, + "texture_fine_seg_last": 0.18335915454329518, + "texture_mid_seg_last": 0.036670341967276074, + "texture_coarse_seg_last": 0.0027926790472022974, + "wall_minutes": 62.096540264288585 + } + }, + { + "arm": "ksr-f34-s1166", + "scene": "s1-static", + "seed": 1166, + "job_id": "726bb72c3908", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-ksrf/20261106-060102-49ecd840/final/BenchS1Static-ksrf-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.3410714285714285, + "seam_jump_max": 4.04, + "seam_look_mean": 1.1060714285714286, + "seam_look_max": 2.49, + "seam_level_step_db_mean": 5.195, + "seam_level_step_db_max": 39.77, + "texture_fine_seg_last": 0.15642989180921119, + "texture_mid_seg_last": 0.029045127372551213, + "texture_coarse_seg_last": 0.01580511574675087, + "wall_minutes": 56.47018893957138 + } + }, + { + "arm": "ksr-f34-gate09-s1166", + "scene": "s1-static", + "seed": 1166, + "job_id": "16f116c27981", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-ksrg/20261106-060113-d4d9ece2/final/BenchS1Static-ksrg-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.0553571428571429, + "seam_jump_max": 1.72, + "seam_look_mean": 1.6867857142857143, + "seam_look_max": 4.53, + "seam_level_step_db_mean": 5.384285714285714, + "seam_level_step_db_max": 23.74, + "texture_fine_seg_last": 1.5415180375236015, + "texture_mid_seg_last": 1.0474774693112396, + "texture_coarse_seg_last": 0.06769496830321924, + "wall_minutes": 55.14977457523346 + } + }, + { + "arm": "ksr-noise04-s1166", + "scene": "s1-static", + "seed": 1166, + "job_id": "60d9f28c6865", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-ksrn/20261106-065622-39638821/final/BenchS1Static-ksrn-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.8428571428571429, + "seam_jump_max": 0.99, + "seam_look_mean": 0.8835714285714286, + "seam_look_max": 2.9, + "seam_level_step_db_mean": 10.959999999999999, + "seam_level_step_db_max": 33.86, + "texture_fine_seg_last": 1.3142873484334245, + "texture_mid_seg_last": 0.6716405081220737, + "texture_coarse_seg_last": 0.009515507937057575, + "wall_minutes": 53.75233529806137 + } + }, + { + "arm": "ksr-g39-s1166", + "scene": "s1-static", + "seed": 1166, + "job_id": "d5cb657fb9e8", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-ksr39/20261106-065730-b866c088/final/BenchS1Static-ksr39-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.9021212121212121, + "seam_jump_max": 1.17, + "seam_look_mean": 1.2169696969696968, + "seam_look_max": 2.33, + "seam_level_step_db_mean": 8.746060606060606, + "seam_level_step_db_max": 35.71, + "texture_fine_seg_last": 0.5294808319304988, + "texture_mid_seg_last": 0.2098924536868174, + "texture_coarse_seg_last": 0.006509621113932019, + "wall_minutes": 71.25989698568979 + } + } +] diff --git a/bench/records/lastframe-ctl-clean-rerun.json b/bench/records/lastframe-ctl-clean-rerun.json new file mode 100644 index 000000000..9c34c01ea --- /dev/null +++ b/bench/records/lastframe-ctl-clean-rerun.json @@ -0,0 +1,50 @@ +[ + { + "arm": "ctl-lastframe-s1-s1166-clean", + "scene": "s1-static", + "seed": 1166, + "job_id": "88250351f1fe", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-lfclean1/20261105-013634-712693ee/final/BenchS1Static-lfclean1-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.036, + "seam_jump_max": 1.19, + "seam_look_mean": 1.254, + "seam_look_max": 1.73, + "seam_level_step_db_mean": 4.508, + "seam_level_step_db_max": 9.35, + "texture_fine_seg_last": 0.5218991266072421, + "texture_mid_seg_last": 0.34220334994179036, + "texture_coarse_seg_last": 0.0017378778291200147, + "wall_minutes": 10.367353157202404 + }, + "segment1": "d59a1259/b2c2c63e", + "segment1_label": "clean", + "history": "first job after a bench server restart on cuda:0 alone (cold model load)" + }, + { + "arm": "ctl-lastframe-s1-s2-clean", + "scene": "s1-static", + "seed": 2, + "job_id": "194ee59b559d", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-lfclean2/20261105-014715-28da8eef/final/BenchS1Static-lfclean2-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.072, + "seam_jump_max": 1.27, + "seam_look_mean": 1.34, + "seam_look_max": 1.76, + "seam_level_step_db_mean": 4.6979999999999995, + "seam_level_step_db_max": 13.86, + "texture_fine_seg_last": 0.5928744772516797, + "texture_mid_seg_last": 0.448291178681586, + "texture_coarse_seg_last": 0.010577093306130311, + "wall_minutes": 10.676378142833709 + }, + "segment1": "f47df963/f88334a9", + "segment1_label": "clean", + "history": "first job after a bench server restart on cuda:0 alone (cold model load)" + } +] \ No newline at end of file diff --git a/bench/records/latent-carry-controls-r36.json b/bench/records/latent-carry-controls-r36.json new file mode 100644 index 000000000..958a1048a --- /dev/null +++ b/bench/records/latent-carry-controls-r36.json @@ -0,0 +1,65 @@ +[ + { + "arm": "r3-anchor25-s1-120s-s1166-r36", + "scene": "s1-static", + "seed": 1166, + "job_id": "2c1fd3ab7869", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-lcctl/20261105-215048-32074799/final/BenchS1Static-lcctl-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.8992857142857142, + "seam_jump_max": 1.51, + "seam_look_mean": 0.8707142857142857, + "seam_look_max": 2.06, + "seam_level_step_db_mean": 4.351785714285714, + "seam_level_step_db_max": 26.07, + "texture_fine_seg_last": 1.1821581751513983, + "texture_mid_seg_last": 0.49590867078968065, + "texture_coarse_seg_last": 0.13771116873686765, + "wall_minutes": 69.92647580305736 + } + }, + { + "arm": "ctl-guide-s2-s1166-r36", + "scene": "s2-arc", + "seed": 1166, + "job_id": "536548e14dd5", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc-lcctl/20261105-215337-2cd709a7/final/BenchS2Arc-lcctl-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.3619999999999999, + "seam_jump_max": 2.09, + "seam_look_mean": 1.286, + "seam_look_max": 1.74, + "seam_level_step_db_mean": 1.4620000000000002, + "seam_level_step_db_max": 3.43, + "texture_fine_seg_last": 3.083963956401898, + "texture_mid_seg_last": 2.2932110623147106, + "texture_coarse_seg_last": 1.4297775601219735, + "wall_minutes": 10.86510852575302 + } + }, + { + "arm": "ctl-guide-s2-s2-r36", + "scene": "s2-arc", + "seed": 2, + "job_id": "1b8edfdba405", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc-lcctl/20261105-220427-e6552f17/final/BenchS2Arc-lcctl-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.306, + "seam_jump_max": 1.83, + "seam_look_mean": 1.418, + "seam_look_max": 1.77, + "seam_level_step_db_mean": 6.542, + "seam_level_step_db_max": 12.38, + "texture_fine_seg_last": 3.5178485966095336, + "texture_mid_seg_last": 2.994312479430393, + "texture_coarse_seg_last": 2.545146643212234, + "wall_minutes": 10.813770214716593 + } + } +] diff --git a/bench/records/latent-carry-controls.json b/bench/records/latent-carry-controls.json new file mode 100644 index 000000000..caecceae6 --- /dev/null +++ b/bench/records/latent-carry-controls.json @@ -0,0 +1,23 @@ +[ + { + "arm": "r3-anchor25-s1-120s-s2", + "scene": "s1-static", + "seed": 2, + "job_id": "bf9ee32e040f", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-lcctl/20261105-163748-138173d5/final/BenchS1Static-lcctl-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.9871428571428572, + "seam_jump_max": 1.43, + "seam_look_mean": 1.2953571428571429, + "seam_look_max": 2.66, + "seam_level_step_db_mean": 10.45142857142857, + "seam_level_step_db_max": 31.99, + "texture_fine_seg_last": 1.14208473240703, + "texture_mid_seg_last": 0.5897377067054688, + "texture_coarse_seg_last": 0.017054293715587, + "wall_minutes": 70.66124661763509 + } + } +] diff --git a/bench/records/latent-carry-noanchor.json b/bench/records/latent-carry-noanchor.json new file mode 100644 index 000000000..af0921df7 --- /dev/null +++ b/bench/records/latent-carry-noanchor.json @@ -0,0 +1,23 @@ +[ + { + "arm": "lc-noanchor-s1-120s-s1166", + "scene": "s1-static", + "seed": 1166, + "job_id": "20ce09d60040", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-lcna/20261105-233057-1721b816/final/BenchS1Static-lcna-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.9325, + "seam_jump_max": 1.49, + "seam_look_mean": 1.0164285714285715, + "seam_look_max": 2.49, + "seam_level_step_db_mean": 12.099285714285715, + "seam_level_step_db_max": 37.6, + "texture_fine_seg_last": 1.104576939882156, + "texture_mid_seg_last": 0.7404191324660745, + "texture_coarse_seg_last": 0.14688019788142942, + "wall_minutes": 50.96668136914571 + } + } +] diff --git a/bench/records/latent-carry.json b/bench/records/latent-carry.json new file mode 100644 index 000000000..9a18ec428 --- /dev/null +++ b/bench/records/latent-carry.json @@ -0,0 +1,128 @@ +[ + { + "arm": "lc-anchor25-s1-120s-s1166", + "scene": "s1-static", + "seed": 1166, + "job_id": "fc0f91663277", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-lc/20261105-182155-84610860/final/BenchS1Static-lc-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.8421428571428572, + "seam_jump_max": 1.17, + "seam_look_mean": 1.1753571428571428, + "seam_look_max": 2.13, + "seam_level_step_db_mean": 15.967142857142857, + "seam_level_step_db_max": 54.07, + "texture_fine_seg_last": 0.6429699025680846, + "texture_mid_seg_last": 0.3341832824336939, + "texture_coarse_seg_last": 0.2164461824377109, + "wall_minutes": 65.77779295841853 + } + }, + { + "arm": "lc-anchor25-s1-120s-s2", + "scene": "s1-static", + "seed": 2, + "job_id": "40bbb40c14ec", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-lc/20261105-192740-39a184be/final/BenchS1Static-lc-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.8314285714285715, + "seam_jump_max": 1.03, + "seam_look_mean": 1.0214285714285716, + "seam_look_max": 1.82, + "seam_level_step_db_mean": 9.819642857142856, + "seam_level_step_db_max": 34.44, + "texture_fine_seg_last": 0.4530011024025377, + "texture_mid_seg_last": 0.21645040446429997, + "texture_coarse_seg_last": 0.2242836883850693, + "wall_minutes": 63.68270232280096 + } + }, + { + "arm": "lc-guide-s2-s1166", + "scene": "s2-arc", + "seed": 1166, + "job_id": "d93adb33249e", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc-lc/20261105-203209-6557d14a/final/BenchS2Arc-lc-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.6640000000000001, + "seam_jump_max": 2.45, + "seam_look_mean": 1.9600000000000002, + "seam_look_max": 2.75, + "seam_level_step_db_mean": 1.2739999999999998, + "seam_level_step_db_max": 2.76, + "texture_fine_seg_last": 3.625164250869989, + "texture_mid_seg_last": 3.647414960745375, + "texture_coarse_seg_last": 3.5359376248824543, + "wall_minutes": 12.798741567134858 + } + }, + { + "arm": "lc-guide-s2-s2", + "scene": "s2-arc", + "seed": 2, + "job_id": "d901c8c82784", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc-lc/20261105-203209-a3c7f4f3/final/BenchS2Arc-lc-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.48, + "seam_jump_max": 2.16, + "seam_look_mean": 2.1399999999999997, + "seam_look_max": 3.54, + "seam_level_step_db_mean": 4.5280000000000005, + "seam_level_step_db_max": 11.89, + "texture_fine_seg_last": 3.525265194264583, + "texture_mid_seg_last": 3.5179398536064244, + "texture_coarse_seg_last": 3.186367710948495, + "wall_minutes": 13.252541661262512 + } + }, + { + "arm": "ks-pixel-anchor25-s1-120s-s1166", + "scene": "s1-static", + "seed": 1166, + "job_id": "05a3a55b043c", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-ksp/20261105-204455-ae6d8604/final/BenchS1Static-ksp-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.9225, + "seam_jump_max": 1.59, + "seam_look_mean": 1.1335714285714285, + "seam_look_max": 2.5, + "seam_level_step_db_mean": 9.71642857142857, + "seam_level_step_db_max": 28.75, + "texture_fine_seg_last": 0.02902375837691561, + "texture_mid_seg_last": 0.015905840429762928, + "texture_coarse_seg_last": 0.01578064700084365, + "wall_minutes": 68.68451871871949 + } + }, + { + "arm": "ks-lc-anchor25-s1-120s-s1166", + "scene": "s1-static", + "seed": 1166, + "job_id": "4766787e793d", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-ksl/20261105-204523-7e0e833b/final/BenchS1Static-ksl-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.8128571428571428, + "seam_jump_max": 1.13, + "seam_look_mean": 1.3135714285714286, + "seam_look_max": 2.08, + "seam_level_step_db_mean": 14.042142857142858, + "seam_level_step_db_max": 38.86, + "texture_fine_seg_last": 0.05871349619366279, + "texture_mid_seg_last": 0.008908584620869322, + "texture_coarse_seg_last": 0.028130780727881395, + "wall_minutes": 65.42316360870997 + } + } +] \ No newline at end of file diff --git a/bench/records/ltx-arc.json b/bench/records/ltx-arc.json new file mode 100644 index 000000000..49635b440 --- /dev/null +++ b/bench/records/ltx-arc.json @@ -0,0 +1,86 @@ +[ + { + "arm": "ltx-arc-ctl-s1166", + "scene": "s2-arc", + "seed": 1166, + "job_id": "a3e99c3df84b", + "status": "succeeded", + "error": null, + "output": "BenchLtxArc-mps/20261106-093752-4c1c7110/final/BenchLtxArc-mps-chained_image_to_video.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.89, + "seam_jump_max": 0.92, + "seam_look_mean": 0.445, + "seam_look_max": 0.49, + "seam_level_step_db_mean": 7.77, + "seam_level_step_db_max": 11.36, + "texture_fine_seg_last": 0.2051936540925779, + "texture_mid_seg_last": 0.6812761184507167, + "texture_coarse_seg_last": 0.01895610828751189, + "wall_minutes": 11.222849468390146 + } + }, + { + "arm": "ltx-arc-lookhold-s1166", + "scene": "s2-arc", + "seed": 1166, + "job_id": "9331cd20fa39", + "status": "succeeded", + "error": null, + "output": "BenchLtxArc-mps/20261106-094905-7510a362/final/BenchLtxArc-mps-chained_image_to_video.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.8600000000000001, + "seam_jump_max": 0.91, + "seam_look_mean": 0.42000000000000004, + "seam_look_max": 0.5, + "seam_level_step_db_mean": 7.675, + "seam_level_step_db_max": 11.17, + "texture_fine_seg_last": 0.23068869709411274, + "texture_mid_seg_last": 0.6995427904613333, + "texture_coarse_seg_last": 0.03987465117137339, + "wall_minutes": 8.910393353303274 + } + }, + { + "arm": "ltx-arc-ctl-s2", + "scene": "s2-arc", + "seed": 2, + "job_id": "84328fe3ad81", + "status": "succeeded", + "error": null, + "output": "BenchLtxArc-mps/20261106-095800-b034643e/final/BenchLtxArc-mps-chained_image_to_video.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.31, + "seam_jump_max": 1.66, + "seam_look_mean": 0.8, + "seam_look_max": 0.88, + "seam_level_step_db_mean": 22.525, + "seam_level_step_db_max": 40.15, + "texture_fine_seg_last": 0.7756513801535185, + "texture_mid_seg_last": 0.856923967469388, + "texture_coarse_seg_last": 0.6523259024877984, + "wall_minutes": 8.793856235345205 + } + }, + { + "arm": "ltx-arc-lookhold-s2", + "scene": "s2-arc", + "seed": 2, + "job_id": "64641b316eed", + "status": "succeeded", + "error": null, + "output": "BenchLtxArc-mps/20261106-100648-3202f70a/final/BenchLtxArc-mps-chained_image_to_video.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.235, + "seam_jump_max": 1.61, + "seam_look_mean": 0.685, + "seam_look_max": 0.78, + "seam_level_step_db_mean": 22.61, + "seam_level_step_db_max": 40.32, + "texture_fine_seg_last": 0.6916616296373697, + "texture_mid_seg_last": 0.7287762921047032, + "texture_coarse_seg_last": 0.64938863454449, + "wall_minutes": 8.948012399673463 + } + } +] \ No newline at end of file diff --git a/bench/records/ltx-cuda-s2.json b/bench/records/ltx-cuda-s2.json new file mode 100644 index 000000000..6efac7b35 --- /dev/null +++ b/bench/records/ltx-cuda-s2.json @@ -0,0 +1,23 @@ +[ + { + "arm": "ltx-ctl-s2", + "scene": "s5-ltx-action", + "seed": 2, + "job_id": "645d8600ac92", + "status": "succeeded", + "error": null, + "output": "BenchS5LtxAction-mps/20261106-121701-6ad858be/final/BenchS5LtxAction-mps-chained_image_to_video.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.17, + "seam_jump_max": 1.95, + "seam_look_mean": 1.31, + "seam_look_max": 1.73, + "seam_level_step_db_mean": 3.715, + "seam_level_step_db_max": 6.14, + "texture_fine_seg_last": 0.5121222203819994, + "texture_mid_seg_last": 0.8211880233415666, + "texture_coarse_seg_last": 0.34714888810802724, + "wall_minutes": 3.236419689655304 + } + } +] \ No newline at end of file diff --git a/bench/records/ltx-long-probe.json b/bench/records/ltx-long-probe.json new file mode 100644 index 000000000..83d4547b6 --- /dev/null +++ b/bench/records/ltx-long-probe.json @@ -0,0 +1,23 @@ +[ + { + "arm": "ltx-ctl-s1166", + "scene": "s5-ltx-action", + "seed": 1166, + "job_id": "29884b7da674", + "status": "succeeded", + "error": null, + "output": "BenchS5LtxAction-mps/20261106-045323-7052dd67/final/BenchS5LtxAction-mps-chained_image_to_video.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.29, + "seam_jump_max": 1.33, + "seam_look_mean": 1.72, + "seam_look_max": 1.88, + "seam_level_step_db_mean": 9.25, + "seam_level_step_db_max": 16.96, + "texture_fine_seg_last": 0.4997510124977312, + "texture_mid_seg_last": 0.5598330559121848, + "texture_coarse_seg_last": 0.6279770765432813, + "wall_minutes": 11.476390695571899 + } + } +] \ No newline at end of file diff --git a/bench/records/ltx-long.json b/bench/records/ltx-long.json new file mode 100644 index 000000000..fc48d3e01 --- /dev/null +++ b/bench/records/ltx-long.json @@ -0,0 +1,275 @@ +[ + { + "arm": "ltx-ctl-s1166-rerun", + "scene": "s5-ltx-action", + "seed": 1166, + "job_id": "3364de6cb5c1", + "status": "succeeded", + "error": null, + "output": "BenchS5LtxAction-mps-r/20261106-050944-8e2742d8/final/BenchS5LtxAction-mps-r-chained_image_to_video.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.29, + "seam_jump_max": 1.33, + "seam_look_mean": 1.72, + "seam_look_max": 1.88, + "seam_level_step_db_mean": 9.25, + "seam_level_step_db_max": 16.96, + "texture_fine_seg_last": 0.4997510124977312, + "texture_mid_seg_last": 0.5598330559121848, + "texture_coarse_seg_last": 0.6279770765432813, + "wall_minutes": 8.860001949469249 + } + }, + { + "arm": "ltx-ctl-s2", + "scene": "s5-ltx-action", + "seed": 2, + "job_id": "7fbfc3451e90", + "status": "succeeded", + "error": null, + "output": "BenchS5LtxAction-mps/20261106-051835-3455784d/final/BenchS5LtxAction-mps-chained_image_to_video.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.245, + "seam_jump_max": 1.52, + "seam_look_mean": 1.6749999999999998, + "seam_look_max": 1.72, + "seam_level_step_db_mean": 23.31, + "seam_level_step_db_max": 42.72, + "texture_fine_seg_last": 0.10367842947484152, + "texture_mid_seg_last": 0.04838758144098727, + "texture_coarse_seg_last": 0.18834826466081508, + "wall_minutes": 8.853334128856659 + } + }, + { + "arm": "ltx-extend", + "scene": "s5-ltx-action", + "seed": 1166, + "job_id": "516802e1acff", + "status": "succeeded", + "error": null, + "output": "BenchS5LtxExtend-mps/20261106-052726-b154648f/final/BenchS5LtxExtend-mps-take.9-0.0.mp4", + "measures": { + "seam_jump_mean": 0.9299999999999999, + "seam_jump_max": 1.02, + "seam_look_mean": 2.0, + "seam_look_max": 2.26, + "seam_level_step_db_mean": 36.565, + "seam_level_step_db_max": 37.44, + "texture_fine_seg_last": 0.1528959745183618, + "texture_mid_seg_last": 0.04141970138470852, + "texture_coarse_seg_last": 0.1751749921437893, + "wall_minutes": 18.61569066842397 + } + }, + { + "arm": "ltx-lookhold", + "scene": "s5-ltx-action", + "seed": 1166, + "job_id": "aaa32cab8384", + "status": "succeeded", + "error": null, + "output": "BenchS5LtxAction-mps/20261106-054604-a8f0dd44/final/BenchS5LtxAction-mps-chained_image_to_video.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.13, + "seam_jump_max": 1.17, + "seam_look_mean": 1.165, + "seam_look_max": 1.19, + "seam_level_step_db_mean": 8.985, + "seam_level_step_db_max": 16.98, + "texture_fine_seg_last": 0.4367280465686361, + "texture_mid_seg_last": 0.42559795776627, + "texture_coarse_seg_last": 0.4554041897703063, + "wall_minutes": 11.396001048882802 + } + }, + { + "arm": "ltx-carrypick4", + "scene": "s5-ltx-action", + "seed": 1166, + "job_id": "d563c0e5d4ab", + "status": "succeeded", + "error": null, + "output": "BenchS5LtxActionCarryPick-mps/20261106-055727-2ceea0c6/final/BenchS5LtxActionCarryPick-mps-chained_image_to_video.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.115, + "seam_jump_max": 1.23, + "seam_look_mean": 1.335, + "seam_look_max": 1.4, + "seam_level_step_db_mean": 5.17, + "seam_level_step_db_max": 6.19, + "texture_fine_seg_last": 0.34440401717906943, + "texture_mid_seg_last": 0.43417415106825014, + "texture_coarse_seg_last": 0.9613865887762229, + "wall_minutes": 8.942894033590953 + } + }, + { + "arm": "ltx-check-r1", + "scene": "s5-ltx-action", + "seed": 1166, + "job_id": "ef9b8abfa067", + "status": "succeeded", + "error": null, + "output": "BenchS5LtxAction-mps-chk/20261106-060624-959b7b19/final/BenchS5LtxAction-mps-chk-chained_image_to_video.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.29, + "seam_jump_max": 1.33, + "seam_look_mean": 1.72, + "seam_look_max": 1.88, + "seam_level_step_db_mean": 9.25, + "seam_level_step_db_max": 16.96, + "texture_fine_seg_last": 0.4997510124977312, + "texture_mid_seg_last": 0.5598330559121848, + "texture_coarse_seg_last": 0.6279770765432813, + "wall_minutes": 8.876843329270681 + } + }, + { + "arm": "ltx-ctl-36s", + "scene": "s5-ltx-action", + "seed": 1166, + "job_id": "cae2ee120f6b", + "status": "succeeded", + "error": null, + "output": "BenchS5LtxAction-mps/20261106-061516-b895f136/final/BenchS5LtxAction-mps-chained_image_to_video.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.27, + "seam_jump_max": 1.81, + "seam_look_mean": 1.4057142857142857, + "seam_look_max": 1.92, + "seam_level_step_db_mean": 12.031428571428572, + "seam_level_step_db_max": 19.1, + "texture_fine_seg_last": 0.3058438471310951, + "texture_mid_seg_last": 0.13082365700499737, + "texture_coarse_seg_last": 0.6478831737993822, + "wall_minutes": 23.478669412930806 + } + }, + { + "arm": "ltx-extend-s2", + "scene": "s5-ltx-action", + "seed": 2, + "job_id": "f032311a7f24", + "status": "succeeded", + "error": null, + "output": "BenchS5LtxExtend-mps/20261106-063845-3d72964a/final/BenchS5LtxExtend-mps-take.9-0.0.mp4", + "measures": { + "seam_jump_mean": 0.765, + "seam_jump_max": 0.91, + "seam_look_mean": 1.15, + "seam_look_max": 1.92, + "seam_level_step_db_mean": 24.825, + "seam_level_step_db_max": 43.23, + "texture_fine_seg_last": 0.14476662428335424, + "texture_mid_seg_last": 0.2241919525984265, + "texture_coarse_seg_last": 0.2147353229123727, + "wall_minutes": 18.321442369620005 + } + }, + { + "arm": "ltx-lookhold-s2", + "scene": "s5-ltx-action", + "seed": 2, + "job_id": "5b471b1428a1", + "status": "succeeded", + "error": null, + "output": "BenchS5LtxAction-mps/20261106-065706-8cf5be16/final/BenchS5LtxAction-mps-chained_image_to_video.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.875, + "seam_jump_max": 0.9, + "seam_look_mean": 1.22, + "seam_look_max": 1.57, + "seam_level_step_db_mean": 23.314999999999998, + "seam_level_step_db_max": 42.73, + "texture_fine_seg_last": 0.1864646230941289, + "texture_mid_seg_last": 0.14680056291059207, + "texture_coarse_seg_last": 0.004205307634811139, + "wall_minutes": 11.412480767567953 + } + }, + { + "arm": "ltx-carrypick4-s2", + "scene": "s5-ltx-action", + "seed": 2, + "job_id": "d0c1b0a56820", + "status": "succeeded", + "error": null, + "output": "BenchS5LtxActionCarryPick-mps/20261106-070829-d31a3b9e/final/BenchS5LtxActionCarryPick-mps-chained_image_to_video.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.8050000000000002, + "seam_jump_max": 2.6, + "seam_look_mean": 1.3599999999999999, + "seam_look_max": 1.58, + "seam_level_step_db_mean": 23.3, + "seam_level_step_db_max": 39.22, + "texture_fine_seg_last": 0.09270253824098439, + "texture_mid_seg_last": 0.25213336570009937, + "texture_coarse_seg_last": 0.17347424433527625, + "wall_minutes": 8.910253715515136 + } + }, + { + "arm": "ltx-check-r1-s2", + "scene": "s5-ltx-action", + "seed": 2, + "job_id": "61b7fc4d4fd9", + "status": "succeeded", + "error": null, + "output": "BenchS5LtxAction-mps-chk/20261106-071724-db7bf6e7/final/BenchS5LtxAction-mps-chk-chained_image_to_video.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.245, + "seam_jump_max": 1.52, + "seam_look_mean": 1.6749999999999998, + "seam_look_max": 1.72, + "seam_level_step_db_mean": 23.31, + "seam_level_step_db_max": 42.72, + "texture_fine_seg_last": 0.10367842947484152, + "texture_mid_seg_last": 0.04838758144098727, + "texture_coarse_seg_last": 0.18834826466081508, + "wall_minutes": 8.823316617806752 + } + }, + { + "arm": "ltx-ctl-36s-s2", + "scene": "s5-ltx-action", + "seed": 2, + "job_id": "f721d3694acb", + "status": "succeeded", + "error": null, + "output": "BenchS5LtxAction-mps/20261106-072613-40a88b3c/final/BenchS5LtxAction-mps-chained_image_to_video.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.3357142857142856, + "seam_jump_max": 1.65, + "seam_look_mean": 1.3814285714285715, + "seam_look_max": 1.72, + "seam_level_step_db_mean": 8.227142857142857, + "seam_level_step_db_max": 23.26, + "texture_fine_seg_last": 1.057669405155067, + "texture_mid_seg_last": 1.5597165420219774, + "texture_coarse_seg_last": 2.15686623786492, + "wall_minutes": 23.498440798123678 + } + }, + { + "arm": "ltx-lookhold-36s", + "scene": "s5-ltx-action", + "seed": 1166, + "job_id": "3bcebd98295e", + "status": "succeeded", + "error": null, + "output": "BenchS5LtxAction-mps/20261106-074943-d5a8b131/final/BenchS5LtxAction-mps-chained_image_to_video.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.7985714285714286, + "seam_jump_max": 1.34, + "seam_look_mean": 0.9971428571428571, + "seam_look_max": 2.84, + "seam_level_step_db_mean": 9.41, + "seam_level_step_db_max": 22.12, + "texture_fine_seg_last": 0.3712595485934952, + "texture_mid_seg_last": 0.5611379869608464, + "texture_coarse_seg_last": 0.26564261288548957, + "wall_minutes": 23.842151780923206 + } + } +] \ No newline at end of file diff --git a/bench/records/ltx.md b/bench/records/ltx.md new file mode 100644 index 000000000..8af17254a --- /dev/null +++ b/bench/records/ltx.md @@ -0,0 +1,271 @@ +# LTX long takes (Task 23) + +## Verdicts + +- **last_frame chain (`ltx2/chained-segments`): stays the default.** Nothing + in the round beats it on picture at no cost and on both seeds. +- **extend (`ltx2/extend-clip` shape, each segment conditioned on the whole + previous one): not a default.** It wins seam jump on both seeds and texture + on seed 1166. But it costs 2.1x the wall time, and on seed 2 the win is an + empty road: the courier rides out of frame in segment 1, and extend keeps + him gone for the last 10 s. It continues what is there, faithfully. Offer it + as a knob for a shot whose subject stays in frame. +- **look_hold: wins on numbers, invisible by eye, safe on a moving camera.** + On S5 it wins seam look on both seeds (1.72→1.17, 1.68→1.22); seed 2 is + strictly mixed by 0.001 on fine texture. By eye the 15 s takes are the + control's shot, same framing at every sampled time. On the LTX orbit it + ties and does not hold the camera back (below). Warm, it costs no wall + time. Recommendation: make it the LTX chain default, with Don's ok, since + the gain is not visible. +- **carry_pick 4: loses.** Mixed on seed 1166 (coarse texture worse), loses + on seed 2 (jump 1.25→1.81). It also shortens the take by 0.2-0.3 s. +- **check + 1 re-roll: tie, free.** No segment failed on either seed (every + shot `attempts: 1`), so the take is bit-identical to the control and wall + time is the same. S5 cannot tell whether the check helps on LTX. It does + not see the audio dropout below; the check measures picture only. +- **duration_seconds 36 (eight segments): not usable as one take.** On both + seeds the rider changes by about 30 s (helmet, box, shirt), and pale + vertical streaks grow in the sky and sea from about 20 s. With look_hold + the courier shrinks to a dot down the road by 22-27 s, and segment 7 + (29.8 s) replaces him with a different man on a green moped. The look_hold + seam numbers at 36 s (look 1.41→1.00) are measured on that swap. + +## Recommended default + +- `ltx2/chained-segments`: last_frame continuity, 3 segments of 121 frames + (about 15 s), no carry_pick, no check. look_hold on is the one candidate + change (Don's call, above); until he rules, the template is unchanged. +- **Cap an LTX long take at about 15-20 s (3-4 segments).** Past that, cut + and start the next shot from a fresh still of the subject. The round gives + no counter that holds the subject longer. +- `ltx2/extend-clip`: no default change. It is the right template when the + shot must continue exactly what the previous clip shows, at 2x the cost. +- No template default changes: the round supports none (plan Step 2). + +## Orbit check (look_hold on a moving camera) + +**look_hold does not hold an LTX orbit back.** The H3 ruling ("never pair +look_hold with last_frame on a moving camera", `counters.md`) is H3's, not +LTX's. + +- **Setup.** `bench/workflows/ltx-arc.json`: the S5 chain (last_frame, three + 121-frame segments) on S2's statue still with an orbit caption, measured + under S2's crop. Arms `bench/arms/ltx-arc.json`, record `ltx-arc.json`. +- **By eye.** On both seeds, with and without look_hold, the camera goes from + the statue's front (0.5 s) to three-quarter (5 s), side-back (10 s) and the + statue's back (14.9 s). The look_hold takes match the controls frame for + frame in framing. +- **Measures.** Every measure is inside the floor on both seeds: a tie. + +| arm | seed | job | jump | look | tex fine | tex mid | tex coarse | wall min | +|---|---|---|---|---|---|---|---|---| +| control | 1166 | `a3e99c3df84b` | 0.89 | 0.45 | 0.205 | 0.681 | 0.019 | 11.2 (cold) | +| look_hold | 1166 | `9331cd20fa39` | 0.86 | 0.42 | 0.231 | 0.700 | 0.040 | 8.9 | +| control | 2 | `84328fe3ad81` | 1.31 | 0.80 | 0.776 | 0.857 | 0.652 | 8.8 | +| look_hold | 2 | `64641b316eed` | 1.24 | 0.69 | 0.692 | 0.729 | 0.649 | 8.9 | + +- **Cost.** Warm, look_hold costs nothing measurable (8.9 against 8.8 min). + The 11.4 min of the S5 look_hold runs is a model reload after the extend + arm, not look_hold. +- **Dropout again.** Seed 2's segment 3 is silent here too (-89 dBFS), with a + different prompt and picture (below). + +## Audio dropout (follow-up candidate) + +LTX sometimes renders a near-silent segment, about -89 dBFS against -23 to +-50 dBFS for its neighbours. Segment RMS per take (dBFS): + +| arm | seed | seg 1 | seg 2 | seg 3 | +|---|---|---|---|---| +| control | 1166 | -40.5 | -23.0 | -24.2 | +| control | 2 | -42.4 | -46.5 | **-89.2** | +| look_hold | 2 | -42.4 | -46.5 | **-89.2** | +| carry_pick 4 | 2 | -42.4 | -50.3 | **-89.2** | +| check r1 | 2 | -42.4 | -46.5 | **-89.2** | +| extend | 1166 | -49.7 | **-87.8** | -52.9 | +| extend | 2 | -39.0 | -46.7 | **-89.0** | + +- On seed 2, segment 3 is silent in every 3-segment arm, though look_hold + and carry_pick change its start image, and in both orbit takes, which have + another prompt and still. The silence follows the segment's seed, not the + prompt or the picture. +- The 36 s seed-2 control's segment 3 is quiet (-57 dBFS) but not silent. +- The lem S5 noise-floor runs (CUDA) had level steps of 2.1-2.9 dB, so no + dropout there. Every dropout so far is on mini-ai (MPS). +- **CUDA check: no dropout.** The seed-2 control rerun on lem (`645d8600ac92`, + cuda:0, same workflow and arguments) has 5 s means of -50.0, -51.1 and + -45.1 dB, so segment 3 is not silent. It is a different draw (CUDA and MPS + noise differ, so the picture differs too), so this does not rule out the + seed, but every silent segment so far is MPS. The CUDA take ran 3.2 min + against 8.9 min warm on mini-ai. +- So on S5 the seam level step measures dropouts, not seams. Knobs here are + judged on picture, as the S1/S2 ruling does for scenes with no held voice. +- The chain check does not catch it. A segment-level silence check (RMS under + about -70 dBFS where the prompt asks for sound) would. Worth an issue: is + it MPS-only, and is it the audio VAE or the vocoder? (#1221; the chain + check's `audio_dbfs` floor and `chain_segment_silent` warning now catch it.) + +### Cause + +**The conditioning, not the audio decode** (#1221 stage B2, #1267; mini-ai, +MPS, 2026-11-08). Two runs of the S5 seed-2 control, the realized workflow +of `7fbfc3451e90`, differing only in `torch_dtype`: + +| run | job | `torch_dtype` | seg 1 | seg 2 | seg 3 | seg 3, muxed | +|---|---|---|---|---|---|---| +| B, control | `cb3697f4179c` | `"torch.bfloat16"` | -46.3 | -56.8 | **-88.7** | -91.7 | +| A, fp32 decode | `b860e3bc8dd6` | `{"default": "torch.bfloat16", "audio_vae": "torch.float32", "vocoder": "torch.float32"}` | -46.3 | -56.8 | **-103.5** | -128.8 | + +Levels in dBFS. The segment columns are the shot's `audio_dbfs`: the RMS of +the generated audio over the kept span, which is what the chain check +measures. "Muxed" is `assess_output`'s shot RMS of the final file. + +- **The probe stands.** Run B's segment 3 reads -88.7, within 0.5 dB of the + -89.2 in the table above, though only just. Segments 1 and 2 don't + match that table's control row (-42.4, -46.5). This server has rendered + seed 2 as -46.3 / -56.8 since at least `1deff016d727`, so the take has + moved since the 2026-11-06 bench runs, and the dropout moved with it. + Run B is the control for run A because the two share their latents. + Segments 1 and 2 agree to 0.1 dB, and the decode runs after denoising. +- **The reading.** In run A, segment 3 is still far under -70. It gets + quieter, close to digital silence (-128.8 muxed), and is not restored. + The fp32 decode turns these latents into silence. bf16 only added + rounding noise on top (the -89). So the audio decode doesn't cause the + dropout. The cause is the **conditioning**: what segment 3 is generated + from, which is segment 2's last frame plus prompt 3. +- **Re-roll.** This record doesn't settle whether a re-roll can bring sound + back. A fresh seed draws fresh audio noise against the same conditioning. + The one re-roll on record, `1deff016d727` (segment 3 re-rolled at seed + 1000005, -88.6), stayed silent. Whether `audio_dbfs` stays re-rollable + or becomes warn-only is Don's call, asked on #1221. +- **No vocoder-only run.** Neither the vocoder nor the audio VAE was + tested alone, and the cause doesn't depend on telling them apart. There + is no vocoder-only run because LTX-2's decode casts the latents to + `audio_vae.dtype` but passes the mel to the vocoder uncast + (`pipeline_ltx2_image2video.py:1766-1768`). A dict with only one of them + in fp32 fails with a dtype mismatch. Both load in fp32 together, or + neither does. +- **Open by construction: MPS-only?** CUDA and MPS draw different noise + from the same seed, so the lem take (`645d8600ac92`) is a different + draw. It can't show whether these conditions would also go silent on + CUDA. +- **No stage C.** The fp32 decode doesn't remove the dropout, so the LTX + templates keep their bf16 `audio_vae` and `vocoder`. + +## Setup + +- **Scene.** S5 (`bench/workflows/s5-ltx-action.json`): a courier cycling a + coastal road, tracking camera, wind and surf, no speech. Three 121-frame + segments at 960x544, 24 fps, about 15 s. +- **Arms.** `bench/arms/ltx-long.json` (13 runs, seeds 1166 and 2) and + `ltx-long-probe.json` (the first control). extend runs + `bench/workflows/s5-ltx-extend.json`: a 241-frame `LTX2ConditionPipeline` + pass conditioned on the whole previous 121-frame segment, keeping its new + 120 frames. carry_pick runs `lf-s5-ltx-action.json`. +- **Server.** mini-ai (MPS), bench server port 8766, LTX-2.5 with uint4 SDNQ + transformer. +- **Records.** `ltx-long.json`, `ltx-long-probe.json`, `ltx-arc.json`. +- **Floor.** jump 0.124, look 0.438, fine 0.081, mid 0.159, coarse 0.118 + (`combined_floor` of the bench's rerun gap and seed floors). On mini-ai the + S5 control reran bit-identical (`29884b7da674`, `3364de6cb5c1`), so this + floor is conservative. + +## Measures + +Each arm is read against the control at its own seed. Cells give the control +value, then the arm. **Bold** is better past the floor; _italic_ is worse past +it. Level step is shown but not counted (dropouts, above). + +| arm | seed | job | jump | look | tex fine | tex mid | tex coarse | level step dB | wall min | +|---|---|---|---|---|---|---|---|---|---| +| control | 1166 | `29884b7da674` | 1.29 | 1.72 | 0.500 | 0.560 | 0.628 | 9.3 | 8.9 (11.5 cold) | +| control | 2 | `7fbfc3451e90` | 1.25 | 1.68 | 0.104 | 0.048 | 0.188 | 23.3 | 8.9 | +| extend | 1166 | `516802e1acff` | **0.93** | 2.00 | **0.153** | **0.041** | **0.175** | 36.6 | 18.6 | +| extend | 2 | `f032311a7f24` | **0.77** | **1.15** | 0.145 | _0.224_ | 0.215 | 24.8 | 18.3 | +| look_hold | 1166 | `aaa32cab8384` | **1.13** | **1.17** | 0.437 | 0.426 | **0.455** | 9.0 | 11.4 (reload) | +| look_hold | 2 | `5b471b1428a1` | **0.88** | **1.22** | _0.186_ | 0.147 | **0.004** | 23.3 | 11.4 (reload) | +| carry_pick 4 | 1166 | `d563c0e5d4ab` | **1.12** | 1.34 | **0.344** | 0.434 | _0.961_ | 5.2 | 8.9 | +| carry_pick 4 | 2 | `d0c1b0a56820` | _1.81_ | 1.36 | 0.093 | _0.252_ | 0.173 | 23.3 | 8.9 | +| check r1 | 1166 | `ef9b8abfa067` | 1.29 | 1.72 | 0.500 | 0.560 | 0.628 | 9.3 | 8.9 | +| check r1 | 2 | `61b7fc4d4fd9` | 1.25 | 1.68 | 0.104 | 0.048 | 0.188 | 23.3 | 8.8 | + +Strict verdicts (`bench/verdict.py`, all five picture measures): extend wins +at 1166 and is mixed at 2 (mid 0.176 over, floor 0.159); look_hold wins at +1166 and is mixed at 2 (fine +0.082 against a 0.081 floor, a hair); carry_pick +is mixed at 1166 and loses at 2; check ties at both. + +36 s takes (eight segments, 864 frames). Seam means span seven seams, so read +them against each other, not against the 15 s rows: + +| arm | seed | job | jump | look | tex fine | tex mid | tex coarse | wall min | +|---|---|---|---|---|---|---|---|---| +| control 36 s | 1166 | `cae2ee120f6b` | 1.27 | 1.41 | 0.306 | 0.131 | 0.648 | 23.5 | +| control 36 s | 2 | `f721d3694acb` | 1.34 | 1.38 | 1.058 | 1.560 | 2.157 | 23.5 | +| look_hold 36 s | 1166 | `3bcebd98295e` | **0.80** | 1.00 | 0.371 | _0.561_ | **0.266** | 23.8 | + +Seed 2's 36 s texture at the last segment is 3-10x seed 1166's: the streaks. + +Cost, wall minutes per output minute on mini-ai (warm): control 35, +look_hold 35 (the 11.4 min rows paid a reload after extend), extend 74, +36 s chain 39. + +## By eye + +Contact sheets at 0.5, 4.9, 5.2, 9.9, 10.2 and 14.9 s (the seams sit at +5.04 and 10.0 s), and at 1, 10, 20, 30 and 35.9 s for 36 s takes. + +- **Seams are clean on every arm.** No arm shows a visible jump at either + seam; the frames either side of 5.04 and 10.0 s match in framing and light. +- **The camera does not track.** The prompt asks for a tracking shot at the + rider's speed. Seed 1166 drifts from a side view to behind him by 10 s, + and he recedes down the road by 15 s. Seed 2 lets him ride out of frame at + the end of segment 1; last_frame brings a courier back into frame in + segment 2. extend does not, and shows an empty wall to 15 s. +- **look_hold at 15 s** is the control's shot on both seeds, with the same + framing at every sampled time; the seam-look gain is not visible at this + size. +- **36 s.** From about 20 s pale vertical streaks run through the sky and + sea; by 30 s they cross the whole frame. The rider changes: seed 1166's + loses the black helmet and the box changes shape; look_hold 36 s swaps him + for a man in a hat on a green moped at 29.8 s. + +## Second reading: S5's own seed floor (2026-11-06) + +The verdicts above use the bench-wide floor (S1-S3 seed floors and the lem +rerun gap). The counter round's rule is the scene's own floor, +`combined_floor(noise_floor(control@1166, control@2), rerun_gap)`, picture +measures only. On mini-ai that is, for S5 at 15 s (rerun gap 0, bit-identical +rerun): + +| jump | jump max | look | look max | tex fine | tex mid | tex coarse | +|---|---|---|---|---|---|---| +| 0.045 | 0.190 | 0.045 | 0.160 | 0.396 | 0.511 | 0.440 | + +and for the 36 s pair: jump 0.066, jump max 0.160, look 0.024, look max 0.200, +fine 0.752, mid 1.429, coarse 1.509. + +Under it (primaries seam jump mean and seam look mean, as for S2): + +| arm | seed 1166 | seed 2 | +|---|---|---| +| look_hold | wins both, nothing worse | wins both, nothing worse | +| extend | jump mixed, look loses (look mean and max worse) | mixed (look max 1.92 vs 1.72) | +| carry_pick 4 | wins both | jump loses (mean and max) | +| check r1 | tie (bit-identical) | tie (bit-identical) | +| look_hold 36 s | mixed: look max 2.84 vs 1.92, the moped swap seam | - | + +So look_hold passes the strict rule on both seeds under the scene floor; the +seed-2 fine-texture "mixed" above comes from S1's 0.081 floor on a crop that +is sky (`noise-floor.md`: S5's fine band is a weak signal). The two-seed +floor is thin (seam look 0.045). Nothing else changes: extend and carry_pick +still fail on picture by eye, and the gain is still not visible at 15 s. + +A ready change for Don's yes: `ltx2/chained-segments` gains a `look_hold` +variable (default true) wired to `chain.look_hold`, with two tests in +`tests/test_catalog_structure.py` (`..._holds_the_look_by_default`, +`..._look_hold_false_is_the_plain_chain`). It needs one more edit before it +lands: with `look_hold` declared as a variable anywhere in the catalog, +`test_a_description_names_only_variables_the_workflow_declares` fails on +`minimax/chained-segments`, whose description names `look_hold` without +declaring it. Reword that sentence (or declare the variable there) in the +same commit; the rest of the suite (18,787 tests) passed. diff --git a/bench/records/music-snap.json b/bench/records/music-snap.json new file mode 100644 index 000000000..f35e0e085 --- /dev/null +++ b/bench/records/music-snap.json @@ -0,0 +1,18 @@ +[ + { + "arm": "s6-music-snap8-335s-s1166", + "scene": "s6-narration", + "seed": 1166, + "job_id": "2179e0c0c9df", + "status": "succeeded", + "error": "ValueError: job 2179e0c0c9df wrote no video" + }, + { + "arm": "s6-music-snap8-335s-s1167", + "scene": "s6-narration", + "seed": 1167, + "job_id": "d51816c55389", + "status": "succeeded", + "error": "ValueError: job d51816c55389 wrote no video" + } +] \ No newline at end of file diff --git a/bench/records/narrated.md b/bench/records/narrated.md new file mode 100644 index 000000000..576d3375b --- /dev/null +++ b/bench/records/narrated.md @@ -0,0 +1,480 @@ +# Narrated take: drift counters and sync, measured (narrated-long-take plan) + +Measured 2026-11-09. Can the lip-sync measure (`dw/tasks/lip_sync.py`) tell +a segment whose mouth moves with its own audio from one whose audio is wrong, +well enough to be a per-segment chain check? The plan's rule: if at least 90% +of good segments score above every bad one at the same segment, the measure +stays in the chain check as built; otherwise it leaves the chain check, the +task stays, and sync is judged by eye. This used no new GPU time. + +## Sync calibration + +### Method + +- **Footage.** The 70 kept segments of lem workspace `constitution-walk-2`: + MiniMax H3 dialogue, one speaker to camera, 1344x768, int8 base, 20 steps, + 24 fps, 187 or 209 frames each. +- **Measure.** `lip_sync._mouth_signal` (the mean absolute grey change inside + the mouth box cut from YuNet's mouth corners) against `lip_sync._envelope` + (the audio's RMS per frame's span of samples), Pearson r at every + whole-frame lag within ±24 frames (±1 s), on CPU. +- **Good.** Each segment with its own audio. +- **Bad.** Each segment with (a) its own audio 0.5 s late (12 frames) and + (b) another segment's audio (seven segments on in the list, padded or cut + to length). +- Raw per-segment numbers: the controller's scratchpad `calib/calib.json` + (`r0`, `near_max` for |lag| <= 3, `best_r`, `best_lag`, per case). + +### Separation + +A segment separates when its good score is above its bad score for that same +segment. The rule needs 90% (63 of 70). + +| Score | Good above late | Good above other audio | +|---|---|---| +| Best r within ±0.4 s (as built) | 40/70 | 60/70 | +| Best r within ±3 frames of lag 0 | 61/70 | 64/70 | +| r at lag 0 | 56/70 | 62/70 | + +Across segments, no threshold separates one segment on its own: the 10th +percentile of good (best r within ±3 frames) is 0.138, the 90th of late is +0.27. Medians: good 0.288, late 0.145, other audio 0.100. + +### Where the best lag sits + +The best lag within ±1 s is the clearer sign. Good segments cluster at 0 to ++2 frames (34/70 at 0..+2, 40/70 at -2..+4). With the audio 0.5 s late, the +best lag moves to +12 to +14 frames (39/70 there), and only 5/70 land within +-2..+4. + +### Verdict + +No score reaches the 90% rule against the late audio, and no threshold +separates a single segment, so lip sync is not a per-segment chain check +(ruling R9): `lip_sync_min` is gone from the chain's `check`, and the chain +records no sync value. `measure_lip_sync` stays as a coarse take-level +indicator: `correlation` is the best r within ±3 frames of lag 0 (the best +row above), and `lag_seconds` is the best lag within `max_lag_seconds` +(default 1 s), which read over a whole take shows a consistent offset. One +segment's correlation is not a verdict. Judge sync by eye. + +## CP1: control vs candidate, seed 1, 12 segments (~86 s) + +Bench `bench/workflows/narrated-take.json` on lem (develop 036cb279, +`DW_EXPERIMENTAL_REFINE_WINDOWS=1`), workspace `constitution-walk-3`, scene +`s3-dialogue-walk`, speaker still `asset:bench/constitution-speaker.jpg`, +held narration `asset:bench/constitution-narration-retimed.wav` (298.45 s, +43 planned segments), `stop_after: 12`, tier `best` (1344x768, int8, 20 +steps, no LoRA), 192/170 frames per segment, guide 22. + +### Control (guide carry, no counters) - job d5584b91816d + +Run `BenchNarratedTake/20261109-225944-af94be0b`. Wall 233 min, 19.4 min +per segment. No re-rolls; `texture_ratio` (warn-only) flagged segments 9-12 +(2.12, 2.25, 2.51, 2.39). + +| Segment | starts (s) | face_drop | +| --- | --- | --- | +| 2 | 8.0 | 0.003 | +| 3 | 15.1 | 0.032 | +| 4 | 22.2 | 0.043 | +| 5 | 29.3 | 0.057 | +| 6 | 36.3 | 0.146 | +| 7 | 43.4 | 0.114 | +| 8 | 50.5 | 0.111 | +| 9 | 57.6 | 0.140 | +| 10 | 64.7 | 0.148 | +| 11 | 71.8 | 0.154 | +| 12 | 78.9 | 0.207 | + +Frame saturation (HSV mean): 68.3 at 0 s, 70.7 at 15 s, 75.6 at 30 s, +77.8 at 45 s, 78.8 at 60 s, 81.4 at 75 s, 81.0 at 85 s. Brightness flat +(111.6-115.5). + +Eye: no resets, no change of size in frame, speaking throughout; the +narration's cadence carries (the halting pauses of constitution-walk-2 are +gone). The face is etched by 30 s, glossy and caricatured by 60 s, crusted +by 85 s; the backdrop turns painterly with repeating trees from about 60 s. +Defect 1 (the smoothing/waxing drift) reproduces at the same lengths as +`bench/records/drill.md` ask 1; `face_drop` steps up at segment 6 (36 s), +the first point an eye calls it. + +### Candidate (latent carry, latent keystone every 3, capstone 0.7) - job 576bad7a2138 + +Run `BenchNarratedTake/20261110-025240-db6a16a5`, same arguments as the +control plus `experimental_latent_carry: true`, `keystone: segment_1`, +`keystone_slot: latent`, `experimental_capstone: {enter_below_sigma: 0.7}`, +`anchor_every: 3`. Wall 224 min, 18.6 min per segment (4% under the +control). No re-rolls; `texture_ratio` flagged nothing. + +| Segment | control face_drop | candidate face_drop | +| --- | --- | --- | +| 2 | 0.003 | 0.001 | +| 3 | 0.032 | 0.038 | +| 4 | 0.043 | 0.023 | +| 5 | 0.057 | 0.037 | +| 6 | 0.146 | 0.094 | +| 7 | 0.114 | 0.105 | +| 8 | 0.111 | 0.102 | +| 9 | 0.140 | 0.113 | +| 10 | 0.148 | 0.141 | +| 11 | 0.154 | 0.195 | +| 12 | 0.207 | 0.205 | + +| Saturation at | 0 s | 15 s | 30 s | 45 s | 60 s | 75 s | 85 s | +| --- | --- | --- | --- | --- | --- | --- | --- | +| control | 68.3 | 70.7 | 75.6 | 77.8 | 78.8 | 81.4 | 81.0 | +| candidate | 68.4 | 67.6 | 68.0 | 66.3 | 67.7 | 66.4 | 65.7 | + +Seams (`analyze_seams`, no findings on either): candidate jump_ratio +0.63-1.53 (seams 10 and 11 at 1.53/1.51), lightness steps up to 6.1; +control 0.49-1.12, lightness steps up to 3.6. By eye the candidate's worst +seams hold framing and pose with a slight brightness lift, no pop. + +Eye: the candidate removes the colour burn outright and holds the face +natural to about 60 s, where the control is already glossy. It does not +hold to 85 s: by segment 11 the face glazes as the control's did at 60 s, +and `face_drop` meets the control's at the end (0.205 against 0.207). +The backdrop drifts in both (the colonnade gives way to rows of trees). + +### CP1 reading + +The candidate moves the face drift from about 30-36 s to about 70 s and +removes the saturation burn, at no extra wall time; neither arm holds a +face to 85 s. Seed 1 only. + +### Static long (keystone segment_1, slot last) - job 7f52bc5b09b6 + +Seed 1, 12 segments, 282 min wall. `face_drop` stays within ±0.015 of +segment 1 throughout; saturation 68 to 75 to 73; `cut_ratio` re-rolled +segments 2 and 3. Eye: face and backdrop hold to 85 s, but she stands: +every segment eases back to the opening frame, so there is no walk. + +### Face repair on the candidate - jobs 44be41a64140, d56817f8fa5d + +`templates/minimax/face-repair` on candidate segments 6 and 11 (about 7.4 +min each). It softens the glaze but has no reference to aim at: the face +comes back blurrier and, on segment 11, older. Dropped as a fix. + +### Don's CP1 verdict (2026-11-10) + +The candidate's glaze is distracting by 40 s; the control degrades +markedly by 40 s; the static long take holds the face throughout. A +walk with the static take's face quality wins, at a higher budget. Arms +D (keystone `last` every 3 segments: pulls her back to the walk's start) +and A (candidate anchors every segment: its anchors barely moved +`face_drop`) were dropped before running. Next: the moving keystone +(`docs/proposals/complete/narrated-long-take-complete.md`, Amendment 1). + +## Moving keystone: keyframes + +### v1: Klein edit per keyframe (dropped) + +Twelve `templates/flux3/image-edit` runs on the speaker still (seed 1110), +one per segment end, each asking for the background "d metres further down +the avenue" (d = 7 to 84). About 1 min each on cuda:0. + +- Identity holds: SFace similarity to the still 0.907-0.931. +- Framing is consistent between keyframes but pulled back from the still: + face width 274 px against the still's 361 (0.76x), so segment 1 would + read as a dolly-out. +- The recession stalls: keyframes 1-3 move the colonnade back, 4-12 are + near-identical. Distance asked for in words stops working past ~28 m. + +### v2: one plate, widening crops, the matted speaker (approved) + +1. **Plate.** `templates/flux3/image-edit` at 2048x1168 on the still + shrunk to half size in a grey border, asked to outpaint and remove her + (seeds 1110, 1111; jobs 39f1ab8bdc05, d9d5d922b1cf). Seed 1111 kept: a + centred colonnade at the end of a tree-lined avenue. +2. **Crops.** Keyframe k (0-12) is a 1.75:1 crop of the plate whose width + grows geometrically from 1155 px to the plate's 2048 (1.77x over the + walk, ~4.9% per segment), the building's base held at 56% of the + height and its centre easing from 72% of the width to the middle, + resized to 1344x768 with a 3 px Gaussian blur for depth of field. +3. **Speaker.** `remove_background` (RMBG-2.0) on the still (job + 7ed5e7747fdb), pasted onto every crop at the same pixels. + +Her face is the still's own pixels in every keyframe: size, position and +identity cannot drift. The backdrop recedes steadily from keyframe 1 to +12. Matte edges (hair, earrings, collar) hold at full size. + +Tried first and dropped: `templates/flux3/multi-reference` composites of +the still onto crops 1, 6 and 12 (seed 1110; jobs 302d42a0f24c, +95514085cb39, f0a4fd947702). The backdrop followed the crops and identity +held (0.913, 0.926, 0.930), but face scale wandered at 0.95x, 0.77x and +0.90x: a zoom pump at every segment end. + +The chain starts from keyframe 0 (her on the tightest crop), not the +still, whose closer, different building would swap partway through +segment 1. Assets: `asset:uploads/bench/keyframes-v2/kf00.jpg` to +`kf12.jpg`. Don approved the series 2026-11-10. + +Watch at CP1: every keyframe holds the same expression and pose, so +each segment lands on the same face. If that reads as a tic, vary the +expression per keyframe. + +## CP1: moving keystone, seed 1 - job 514b463a855f + +`shot: walk_keyframed`, opening on keyframe 0, keyframes 1-12 as each +segment's last image, otherwise the control's arguments. 12 segments, +219 min wall, no re-rolls. + +| Segment | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | +| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | +| `face_drop` | 0.006 | 0.007 | 0.020 | 0.014 | 0.008 | 0.010 | 0.013 | 0.020 | 0.010 | 0.032 | 0.041 | + +Against CP1's arms at segment 12: control 0.207, candidate 0.205. + +| Saturation at | 0 s | 15 s | 30 s | 45 s | 60 s | 75 s | 85 s | +| --- | --- | --- | --- | --- | --- | --- | --- | +| moving keystone | 75.5 | 74.8 | 78.9 | 78.7 | 79.2 | 78.5 | 80.3 | + +(Keyframe 0's plate starts warmer than the still, 75.5 against 68.) + +Seams: jump_ratio mean 0.60, max 0.83; look_ratio max 0.89; audio level +step max 2.1 dB - the cleanest seams of any CP1 arm. + +Eye: the backdrop recedes as the keyframes do (colonnade large at 0 s, +behind her head by 60 s, an avenue of trees at 85 s), with no pull-back. +Face crops at 30/60/85 s match the static long take's at the same +times (both carry the same etched skin texture from about 30 s), and +show none of the candidate's glaze. Whether it reads as a walk is +Don's call on the take. + +## CP1: moving keystone, seed 2 - job e7917ab0cab4 + +Same arm, seed 2. 210 min wall, no re-rolls. + +| Segment | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | +| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | +| `face_drop` | -0.002 | 0.023 | 0.006 | 0.012 | 0.023 | 0.018 | 0.003 | 0.003 | 0.014 | 0.014 | 0.032 | + +| Saturation at | 1 s | 15 s | 30 s | 45 s | 60 s | 75 s | 85 s | +| --- | --- | --- | --- | --- | --- | --- | --- | +| moving keystone s2 | 75.0 | 75.1 | 77.5 | 79.7 | 74.7 | 76.6 | 78.5 | + +Seams: jump_ratio mean 0.37, max 0.60; look_ratio max 0.63. + +Eye: face crops at 1/30/85 s match seed 1's. One sway inside segment 9 +(about 58-61 s): she drifts right of centre and the colonnade comes out +from behind her, sharper, then she is back on her mark by 62 s as the +segment lands on its keyframe. Both seeds replicate the face hold. + +### Don's verdict on the moving keystone (2026-11-11) + +Faces good on both seeds, "maybe some slight burn". No walking, only sway. Seed 2 drops +lip sync on occasion; seed 1 is better. Don chose a slow lateral camera arc over a walk. + +Mouth stillness (YuNet mouth box, grey diff per frame against audio RMS, half-second +bins, flagged when speech > 0.7x median and mouth < 0.45x median): seed 2 had 12 flagged +bins, 7 of them within 0.8 s of a segment end - the closed-mouth keyframes pull the +mouth shut as each segment lands. + +## Keyframes v3: camera arc, mid-word mouths, graded plate + +One plate, graded to the still's background (Reinhard LAB), cropped along a lateral arc +(left 105 -> 505 px over 12 segments), the matted speaker pasted with a 65 px drift. +Keyframes 1-12 cycle four Klein open-mouth edits of the still, aligned on the YuNet +landmarks. Arms: `bench/arms/narrated-arc.json`, `narrated-arc-latent.json`. + +## Arc, seed 1 - jobs f2775de46a93 (arc), 7f9f6f106524 (arc + latent carry + capstone 0.7) + +| | moving keystone s1 | arc | arc + latent | +|---|---|---|---| +| face_drop, segments 2-12 max | 0.041 | 0.068 | 0.049 | +| saturation 0/15/30/45/60/75/85 s | 76.0/75.3/79.4/79.2/79.7/79.0/80.8 | 73.8/74.8/78.2/80.9/79.3/77.5/77.2 | 73.8/74.7/76.0/78.4/77.8/78.5/76.5 | +| still-mouth bins | 7 | 3 | 2 | +| seam_jump max / mean | 0.83 / 0.60 | 1.09 / 0.65 | 1.22 / 0.88 | +| texture fine, last vs first segment | 0.148 | 0.002 | 0.009 | + +Reading: the arc reads (building and trees cross behind her), no reset, lip sync better. +The face is **more** burned than the moving keystone from about 30 s: deeper, sharper +folds and warmer skin (`arc-faces.jpg`). The texture measure is whole-frame and missed +it. Cause: the Klein open-mouth edits aged the whole face (`kf-faces-v2-v3.jpg`), and +the chain follows its keyframes. Seam scores include the camera move. + +## Keyframes v4: the original face, only the mouth blended in + +The v3 series with each variant reduced to a feathered mouth ellipse (centre the YuNet +mouth midpoint, radii 0.95 x 0.62 mouth widths, blur 0.14) over the original matte, +colour-shifted to the original on a ring around the ellipse. Arm: +`bench/arms/narrated-arc4-latent.json`. + +## Path assessment (2026-11-12) + +At 86 s the chain holds narration and pose; burn is not fixed, and identity now rests on +a hand-built keyframe per segment that no agent can build through MCP. Nothing is +tested past 86 s. diffusers 0.43.0 ships Kandinsky S2V 1.0 (reference frame + speech +track, 5 s chunks with FramePack memory, MIT). Don approved: spike S2V on the same still +and narration, run v4 once on H3, then stop keyframe iteration either way. + +## Kandinsky S2V 1.0 spike (2026-11-12) - Lite distilled, 86 s, seed 1 + +Standalone script on lem (`~/spikes/ks2v`, diffusers 0.43.0 beside the server's 0.42.1), +`kandinskylab/Kandinsky-S2V-1.0-Lite-distill-Diffusers`, 16 steps, keyframes-v2 kf00 + +the retimed narration, auto CPU offload, cuda:0. 768x512, 25 fps, peak 17.0 GiB, about +2.6 min per 5 s chunk (86 s in about 50 min). + +| at 0.5/15/30/45/60/75/85 s | moving keystone s1 | arc + latent s1 | S2V Lite distilled | +|---|---|---|---| +| SFace similarity to first frame | 1/.95/.90/.93/.92/.91/.91 | 1/.91/.90/.87/.91/.90/.90 | 1/.90/.80/.76/.87/.79/.71 | +| saturation | 75.4 -> 80.8 | 73.5 -> 76.5 | 70.9 -> 91.0 | +| face-crop Laplacian variance | 554-826 | 552-995 | 60-238 | +| still-mouth bins | 7 | 2 | 5 | + +Reading: fails. Identity drifts to a different, darker face by 85 s, the earrings go, +saturation climbs by 20, and the face is soft throughout (`ks2v-vs-h3-faces.jpg`). The +narration is unbroken and there is no reset, but H3 with the moving keystone holds the +face far better. Lite base (50 steps, guidance 3.0) and Pro (21B, would need 4-bit on +24 GB) are untested. + +## Arc v4 + latent carry, seed 1 - job 3147f5720689 + +| at 0.5/15/30/45/60/75/85 s | moving keystone s1 | arc v3 + latent | arc v4 + latent | +|---|---|---|---| +| saturation | 75.4/75.3/79.4/79.2/79.7/79.0/80.8 | 73.5/74.7/76.0/78.4/77.8/78.5/76.5 | 73.4/73.5/76.0/76.3/75.4/77.0/76.3 | +| SFace similarity to first frame | 1/.95/.90/.93/.92/.91/.91 | 1/.91/.90/.87/.91/.90/.90 | 1/.93/.71/.92/.91/.79/.85 | +| face-crop Laplacian variance | 603/610/700/595/826/554/696 | 614/552/773/955/789/969/995 | 621/558/708/756/944/986/1118 | + +face_drop per segment 0.010-0.074 (max at segment 11); still-mouth bins 2, neither near +a segment end; seam_jump max 1.08, mean 0.79; texture fine 0.052. + +Reading: the mouth-only keyframes removed the orange, aged cast of v3 and saturation is +the flattest of any arm; lip sync is the best of any arm. The face still sharpens +through the take (crop sharpness nearly doubles by 85 s, folds deepen from about 45 s; +`arc4-faces.jpg`). Moving keystone s1 (v2 keyframes, no latent carry) does not sharpen +this way. Which of latent carry, the graded and blurred plate, or the arc causes it is +not isolated. Per R20, keyframe iteration stops here. + +## Arc v4 without latent carry, seed 1 - job ddbac3a46144 + +The isolating run for #1312: the arc v4 arm with `experimental_latent_carry` off +(`bench/arms/narrated-arc4.json`), everything else as job 3147f5720689. 246 min, two +re-rolls (segments 2 and 6). + +Face crop, every sixth frame, per 10 s window (median Laplacian variance of the crop at +256 px / mean HSV saturation of the crop / median SFace similarity to the first frame). +The crop is sized differently from the table above, so compare within this table only. + +| window | moving keystone s1 | arc v4 + latent | arc v4, no latent | +|---|---|---|---| +| 0-10 s | 452 / 99.8 / .92 | 477 / 101.3 / .92 | 472 / 101.8 / .92 | +| 10-20 s | 508 / 102.3 / .91 | 489 / 104.4 / .90 | 439 / 104.7 / .90 | +| 20-30 s | 536 / 104.1 / .90 | 533 / 103.7 / .88 | 410 / 107.6 / .88 | +| 30-40 s | 512 / 104.5 / .89 | 482 / 109.2 / .85 | 435 / 106.0 / .89 | +| 40-50 s | 492 / 103.8 / .90 | 604 / 105.0 / .88 | 468 / 104.4 / .89 | +| 50-60 s | 626 / 104.0 / .90 | 680 / 106.3 / .89 | 519 / 107.0 / .89 | +| 60-70 s | 566 / 102.7 / .90 | 808 / 106.7 / .90 | 462 / 106.0 / .89 | +| 70-80 s | 451 / 101.7 / .89 | 710 / 107.4 / .89 | 407 / 104.7 / .89 | +| 80-90 s | 469 / 103.1 / .87 | 856 / 104.7 / .87 | 390 / 102.8 / .89 | + +Whole-frame saturation at 0.5/15/30/45/60/75/85 s: 73.3/73.7/77.3/76.6/76.6/75.7/75.0. +face_drop per segment 0.026-0.075 (max at segment 7). Still-mouth bins 3 (moving +keystone s1: 7; v4 + latent: 2). seam_jump max 0.89, mean 0.59 (v4 + latent: 1.08 / +0.79); texture fine 0.050. Lowest single-frame SFace similarity 0.51 (60-70 s window); +v4 + latent dips to 0.21, moving keystone s1 to 0.75. + +Reading: latent carry is what over-sharpens the face. With it off, crop sharpness stays +between 390 and 519 for the whole take, where the same keyframes with latent carry climb +from 477 to 856. The plate, the arc and the v4 keyframes are cleared. The v4 mouth-only +keyframes keep their lip-sync gain without latent carry (3 still-mouth bins against 7 +for the v2 keyframes). On these measures this is the cleanest 86 s arm, and it is the +shipped `walk_keyframed` configuration with v4-style keyframes. One seed; Don has not +viewed it (`cp1-arc4-s1.mp4`, `arc4-nolatent-faces.jpg`). + +## Masked handoff probe, seed 101-112 - job 64a392dcebe1 (refs #1322) + +A test of the handoff ComfyUI-Continuity calls "masked": the previous segment's tail is +written into the new segment's own latent and held under a mask, where the guide chain +passes it as extra condition rows. No engine change - twelve unchained fl2va steps in one +workflow, each with `source_video` (the previous tail, 22 frames, padded to 192), +`mask_video` (22 black frames, then white), `mask_grow` 0, `refine_strength` 1.0, the +tail's first frame as `image`, and the narration slice as `hold_audio`. Plain `dialogue` +conditions otherwise: no keystone, no keyframes, no latent carry. The tail goes through a +VAE encode, not as raw latents. Compared with the `dialogue` control, seed 1 +(`20261109-225944-af94be0b`); the two differ in seed as well as handoff. + +Drift split into the step across each seam (1 s after minus 1 s before, summed over 11 +seams) and the slope inside each segment: + +| measure | control: total / seams / in-segment | masked: total / seams / in-segment | +|---|---|---| +| luma | +0.3 / -0.6 / +0.9 | +2.8 / +11.6 / -8.7 | +| saturation | +12.5 / +18.4 / -5.9 | +17.6 / +20.5 / -2.8 | +| face saturation | +21.3 / +33.8 / -12.5 | +24.1 / +39.4 / -15.3 | +| face sharpness | +648 / +624 / +24 | -109 / -145 / +36 | + +Face crop per 10 s window (median sharpness / mean saturation / median SFace similarity +to the first frame), same script as the table above: + +| window | control | masked | +|---|---|---| +| 0-10 s | 204 / 101.3 / .92 | 169 / 97.1 / .92 | +| 20-30 s | 280 / 109.2 / .89 | 207 / 105.9 / .84 | +| 40-50 s | 546 / 110.6 / .78 | 219 / 110.6 / .71 | +| 60-70 s | 830 / 117.2 / .79 | 203 / 119.6 / .71 | +| 80-90 s | 986 / 118.8 / .74 | 190 / 118.2 / .60 | + +Reading. The masked handoff removes the over-sharpening: face-crop sharpness stays +between 156 and 262 for 86 s where the control climbs from 204 to 986, and the crops show +smooth skin at 85 s where the control is etched. It does not remove the colour drift: +saturation still steps up at every seam by the same amount, and brightness now steps up +too. Identity holds less well (similarity .60 at the end against .74), with nothing but +the tail to anchor it. Still-mouth bins 4, all in the first 10 s. + +Seam defects of this construction. The mask rounds to whole latent cells, so 18 of the 22 +tail frames were held and frames 18-21 were regenerated; the workflow's own join (trim +22) therefore shows a one-frame jump of 3-4x the median frame difference at every seam. +A local join at 18 brings the seam peak to 0.9-3.1x (control 0.9-1.8x), still a visible +bump at about half the seams, and the first seam is followed by ~0.4 s of reframing. +14 min per segment against 18-19 for the guide chain; 165 min for the take. + +One seed, different from the control's; Don has not viewed it +(`masked-handoff-s1.mp4`, `masked-vs-control-faces.jpg`). + +## Levelled latent carry, seed 1 - job b800065fb772 (refs #1322) + +`experimental_latent_carry: "levelled"` (d2967163) on the plain `dialogue` take: each +carried tail's video latents are pulled back, per channel, to the mean and spread of +segment 1's tail before the next segment reads them. No keystone, no keyframes. Same +seed and conditions as the control (`20261109-225944-af94be0b`), so segment 1 is the +same render. 192 min for 12 segments, no re-rolls. + +| measure | control: total / seams / in-segment | levelled: total / seams / in-segment | +|---|---|---| +| luma | +0.3 / -0.6 / +0.9 | +4.1 / -4.3 / +8.4 | +| saturation | +12.5 / +18.4 / -5.9 | -2.6 / +6.6 / -9.2 | +| face saturation | +21.3 / +33.8 / -12.5 | +9.2 / +13.7 / -4.5 | +| face sharpness | +648 / +624 / +24 | -170 / -252 / +83 | + +| window | control | levelled | +|---|---|---| +| 0-10 s | 204 / 101.3 / .92 | 219 / 100.5 / .93 | +| 20-30 s | 280 / 109.2 / .89 | 239 / 105.2 / .89 | +| 40-50 s | 546 / 110.6 / .78 | 225 / 103.5 / .83 | +| 60-70 s | 830 / 117.2 / .79 | 174 / 107.7 / .81 | +| 80-90 s | 986 / 118.8 / .74 | 107 / 105.5 / .76 | + +What the level pulled back grew through the take: the tail's channel means sat 0.061 +from the level at segment 2 and 0.191 at segment 12, its spread 1.03x and 1.07x. Each +segment drifts further inside itself even though it starts from a levelled tail. + +Reading. The burn is gone: whole-frame saturation ends where it began, face saturation +rises less than half as far, and the face is never etched. The cost is the other +direction: from about 60 s the picture softens (face sharpness 219 to 107) and by 85 s +it is slightly hazy and pale. Identity drifts as in the control (similarity .76 against +.74; face_drop reaches 0.195 at segment 12) - levelling does not hold the face, only the +tone. Still-mouth bins 1. Seam peak 0.7-1.7x the median frame difference, as smooth as +the control. + +One seed; Don has not viewed it (`levelled-carry-s1.mp4`, +`levelled-vs-control-faces.jpg`). Untried: a partial pull (the spread left alone, or +half strength) against the late softening, and levelled with the keystone or v4 +keyframes for identity. + +### Don's verdict (2026-11-13) + +On `levelled-carry-s1.mp4`: "she becomes progressively glossy". Rejected; the stills and +the face-sharpness measure read the gloss as softening and missed it. Don closed the +experiment here. The untried combinations above were not run. diff --git a/bench/records/noise-floor.json b/bench/records/noise-floor.json new file mode 100644 index 000000000..2c7df3999 --- /dev/null +++ b/bench/records/noise-floor.json @@ -0,0 +1,254 @@ +[ + { + "arm": "noise-a-s1", + "scene": "s1-static", + "seed": 1166, + "job_id": "00e4000a3ffb", + "status": "succeeded", + "error": null, + "output": "BenchS1Static/20261104-000916-390b57bc/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.9311111111111112, + "seam_jump_max": 1.09, + "seam_look_mean": 1.528888888888889, + "seam_look_max": 2.61, + "seam_level_step_db_mean": 9.856666666666667, + "seam_level_step_db_max": 36.56, + "texture_fine_seg_last": 1.1178899576395858, + "texture_mid_seg_last": 0.40150993772889665, + "texture_coarse_seg_last": 0.04875222289931379, + "wall_minutes": 21.064961115519207 + } + }, + { + "arm": "noise-b-s1", + "scene": "s1-static", + "seed": 1166, + "job_id": "b4e31bb17593", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-noise-b/20261104-004113-f663789a/final/BenchS1Static-noise-b-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.9311111111111112, + "seam_jump_max": 1.09, + "seam_look_mean": 1.528888888888889, + "seam_look_max": 2.61, + "seam_level_step_db_mean": 9.856666666666667, + "seam_level_step_db_max": 36.56, + "texture_fine_seg_last": 1.1178899576395858, + "texture_mid_seg_last": 0.40150993772889665, + "texture_coarse_seg_last": 0.04875222289931379, + "wall_minutes": 18.098664323488872 + } + }, + { + "arm": "noise-a-s2", + "scene": "s2-arc", + "seed": 1166, + "job_id": "04978dc6f338", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc/20261104-010950-2f5ecfe0/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.1944444444444444, + "seam_jump_max": 2.09, + "seam_look_mean": 1.2844444444444445, + "seam_look_max": 2.71, + "seam_level_step_db_mean": 2.035555555555556, + "seam_level_step_db_max": 5.54, + "texture_fine_seg_last": 3.5575967638583803, + "texture_mid_seg_last": 2.8636397627256116, + "texture_coarse_seg_last": 1.9746980403751424, + "wall_minutes": 20.045715339978535 + } + }, + { + "arm": "noise-b-s2", + "scene": "s2-arc", + "seed": 1166, + "job_id": "51d7b09236f6", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc-noise-b/20261104-012953-430a6737/final/BenchS2Arc-noise-b-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.1944444444444444, + "seam_jump_max": 2.09, + "seam_look_mean": 1.2844444444444445, + "seam_look_max": 2.71, + "seam_level_step_db_mean": 2.035555555555556, + "seam_level_step_db_max": 5.54, + "texture_fine_seg_last": 3.5575967638583803, + "texture_mid_seg_last": 2.8636397627256116, + "texture_coarse_seg_last": 1.9746980403751424, + "wall_minutes": 17.843897787729897 + } + }, + { + "arm": "noise-a-s3", + "scene": "s3-dialogue-walk", + "seed": 1166, + "job_id": "3c76d1966485", + "status": "succeeded", + "error": null, + "output": "BenchS3DialogueWalk/20261104-015817-5942f052/final/BenchS3DialogueWalk-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.478, + "seam_jump_max": 1.83, + "seam_look_mean": 1.3780000000000001, + "seam_look_max": 1.61, + "seam_level_step_db_mean": 2.7700000000000005, + "seam_level_step_db_max": 8.81, + "texture_fine_seg_last": 0.17510029217190348, + "texture_mid_seg_last": 0.22613031907347164, + "texture_coarse_seg_last": 0.2075222005860499, + "wall_minutes": 10.790832726160685 + } + }, + { + "arm": "noise-b-s3", + "scene": "s3-dialogue-walk", + "seed": 1166, + "job_id": "d2095670cd89", + "status": "succeeded", + "error": null, + "output": "BenchS3DialogueWalk-noise-b/20261104-021940-da114475/final/BenchS3DialogueWalk-noise-b-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.2120000000000002, + "seam_jump_max": 1.6, + "seam_look_mean": 1.324, + "seam_look_max": 1.87, + "seam_level_step_db_mean": 3.0, + "seam_level_step_db_max": 8.65, + "texture_fine_seg_last": 0.17818822030424697, + "texture_mid_seg_last": 0.20762310554096494, + "texture_coarse_seg_last": 0.43956359806218753, + "wall_minutes": 10.548045885562896 + } + }, + { + "arm": "noise-a-s4", + "scene": "s4-montage", + "seed": 1166, + "job_id": "568a3da301c4", + "status": "succeeded", + "error": null, + "output": "BenchS4Montage-noise-a2/20261104-024801-94372b51/final/BenchS4Montage-noise-a2-film.14-0.0.mp4", + "measures": { + "seam_jump_mean": 20.476666666666667, + "seam_jump_max": 22.69, + "seam_look_mean": 13.473333333333331, + "seam_look_max": 18.24, + "seam_level_step_db_mean": 3.4966666666666666, + "seam_level_step_db_max": 4.42, + "texture_fine_seg_last": 1.7140760391646614, + "texture_mid_seg_last": 0.5798479202075943, + "texture_coarse_seg_last": 1.214317730531653, + "wall_minutes": 19.715835420290627 + } + }, + { + "arm": "noise-b-s4", + "scene": "s4-montage", + "seed": 1166, + "job_id": "511b37c9bd70", + "status": "succeeded", + "error": null, + "output": "BenchS4Montage-noise-b2/20261104-024801-6c692da9/final/BenchS4Montage-noise-b2-film.14-0.0.mp4", + "measures": { + "seam_jump_mean": 20.476666666666667, + "seam_jump_max": 22.69, + "seam_look_mean": 13.473333333333331, + "seam_look_max": 18.24, + "seam_level_step_db_mean": 3.4966666666666666, + "seam_level_step_db_max": 4.42, + "texture_fine_seg_last": 1.7140760391646614, + "texture_mid_seg_last": 0.5798479202075943, + "texture_coarse_seg_last": 1.214317730531653, + "wall_minutes": 20.75935315291087 + } + }, + { + "arm": "noise-a-s5", + "scene": "s5-ltx-action", + "seed": 1166, + "job_id": "a91df8d3684c", + "status": "succeeded", + "error": null, + "output": "BenchS5LtxAction/20261104-005131-8fc124d3/final/BenchS5LtxAction-chained_image_to_video.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.245, + "seam_jump_max": 1.36, + "seam_look_mean": 2.085, + "seam_look_max": 2.43, + "seam_level_step_db_mean": 2.145, + "seam_level_step_db_max": 2.85, + "texture_fine_seg_last": 0.545653955553654, + "texture_mid_seg_last": 0.48674276499700386, + "texture_coarse_seg_last": 1.5416578973723583, + "wall_minutes": 3.1932440121968586 + } + }, + { + "arm": "noise-b-s5", + "scene": "s5-ltx-action", + "seed": 1166, + "job_id": "813a4ec70859", + "status": "succeeded", + "error": null, + "output": "BenchS5LtxAction-noise-b/20261104-005443-a7939f5d/final/BenchS5LtxAction-noise-b-chained_image_to_video.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.245, + "seam_jump_max": 1.36, + "seam_look_mean": 2.085, + "seam_look_max": 2.43, + "seam_level_step_db_mean": 2.1399999999999997, + "seam_level_step_db_max": 2.84, + "texture_fine_seg_last": 0.545653955553654, + "texture_mid_seg_last": 0.48674276499700386, + "texture_coarse_seg_last": 1.5416578973723583, + "wall_minutes": 1.9625712672869364 + } + }, + { + "arm": "noise-a-s6", + "scene": "s6-narration", + "seed": 1166, + "job_id": "54240901f86a", + "status": "succeeded", + "error": null, + "output": "BenchS6Narration/20261104-005645-a4669737/final/BenchS6Narration-film.9-0.0.mp4", + "measures": { + "seam_jump_mean": 40.2, + "seam_jump_max": 45.33, + "seam_look_mean": 70.635, + "seam_look_max": 82.75, + "seam_level_step_db_mean": 7.79, + "seam_level_step_db_max": 13.26, + "texture_fine_seg_last": 1.4010987900525822, + "texture_mid_seg_last": 1.2212536933691236, + "texture_coarse_seg_last": 0.8530977746815804, + "wall_minutes": 1.5750840504964192 + } + }, + { + "arm": "noise-b-s6", + "scene": "s6-narration", + "seed": 1166, + "job_id": "5670fac7bc34", + "status": "succeeded", + "error": null, + "output": "BenchS6Narration-noise-b/20261104-005815-12f0dc21/final/BenchS6Narration-noise-b-film.9-0.0.mp4", + "measures": { + "seam_jump_mean": 40.2, + "seam_jump_max": 45.33, + "seam_look_mean": 70.635, + "seam_look_max": 82.75, + "seam_level_step_db_mean": 7.8, + "seam_level_step_db_max": 13.28, + "texture_fine_seg_last": 1.4010987900525822, + "texture_mid_seg_last": 1.2212536933691236, + "texture_coarse_seg_last": 0.8530977746815804, + "wall_minutes": 1.314908997217814 + } + } +] \ No newline at end of file diff --git a/bench/records/noise-floor.md b/bench/records/noise-floor.md new file mode 100644 index 000000000..0660072e9 --- /dev/null +++ b/bench/records/noise-floor.md @@ -0,0 +1,345 @@ +# Noise floor and cheap-setting checks + +Task 4 of the long-form media plan. Every scene's baseline arm was run twice on +lem (bench server, both cards) at seed 1166; the second run carries a distinct +inline workflow id (`-noise-b`) so the step cache cannot serve it. The floor +for a measure is the absolute difference of the two runs (`bench/verdict.py` +`noise_floor`). Records: `bench/records/noise-floor.json` (12 runs), +`bench/records/cheap-checks.json` (6 runs). Every measure is lower-is-better; +`texture_*_seg_last` is `|ln(last/first)|` of the crop's band energy. + +## Correction (2026-11-05, ruling R35): the floor is not zero + +The zero same-seed floor below is **retracted**. Each scene's pair here +happened to agree, but same-seed runs on the bench server do not always +reproduce (`.superpowers/sdd/2026-11-03-long-form-media/nondeterminism-report.md`): + +- A worker that has run several jobs can move segment 1 into an alternate + output, which repeats bit for bit until the next model load. Every cold + load gave the canonical segment 1. `bench/segment1.py` labels a record + `clean` or `drifted` by hashing its segment 1 against + `bench/records/segment1-canonical.json`. +- One-off departures also start at segment 2 inside a single call + (`cheap-a-s1`, `ctl-guide-s1-s1166-rerun`). +- S3's two runs here differ for the same reason (noise-b-s3 opens on a + drifted segment 1), not because of the spoken lines. + +Ruling R35 sets the verdict floor to `combined_floor(seed floor, +rerun_gap(pairs))` (`bench/verdict.py`): the larger, per measure, of the +seed floor (`noise_floor(control@1166, control@2)`) and the same-seed rerun +gap. + +**Same-seed rerun gap.** These are the 8 round-2 combination pairs that +two drivers both ran (same arguments, same seed, S1). For each measure the +gap is the largest `|a - b|` over the pairs. The first batch's records were +overwritten, so its outputs were measured again with the driver's own +`measure` (`bench/records/counters-round2-combos-batch1.json`); the second +batch is `counters-round2-combos.json`. By `bench/segment1.py`, all 8 pairs +already differ in segment 1: 4 are a drifted run against a clean one, and +4 are two different alternates. The gap is therefore the size of a +worker-state drift. The one-off segment-2 departure is the guide control's +rerun, below. + +| pair (S1) | jump | jump max | look | look max | level dB | level dB max | tex fine | tex mid | tex coarse | +|---|---|---|---|---|---|---|---|---|---| +| anchor25+keystone @1166 | 0.094 | 0.430 | 0.438 | 0.790 | 6.204 | 1.760 | 0.026 | 0.009 | 0.005 | +| anchor50+keystone @1166 | 0.124 | 0.330 | 0.200 | 0.590 | 5.350 | 1.660 | 0.022 | 0.009 | 0.002 | +| lf+look_hold+anchor25 @1166 | 0.006 | 0.030 | 0.002 | 0.080 | 2.722 | 1.590 | 0.033 | 0.020 | 0.007 | +| lf+look_hold+anchor50 @1166 | 0.004 | 0.060 | 0.022 | 0.010 | 0.546 | 1.620 | 0.031 | 0.030 | 0.009 | +| anchor25+keystone @2 | 0.060 | 0.010 | 0.044 | 0.140 | 1.812 | 8.800 | 0.012 | 0.024 | 0.002 | +| anchor50+keystone @2 | 0.124 | 0.190 | 0.006 | 0.190 | 2.698 | 0.410 | 0.009 | 0.018 | 0.004 | +| lf+look_hold+anchor25 @2 | 0.070 | 0.150 | 0.006 | 0.170 | 0.874 | 1.880 | 0.068 | 0.013 | 0.000 | +| lf+look_hold+anchor50 @2 | 0.060 | 0.070 | 0.046 | 0.070 | 0.476 | 0.420 | 0.026 | 0.031 | 0.010 | +| **rerun gap (max)** | **0.124** | **0.430** | **0.438** | **0.790** | **6.204** | **8.800** | **0.068** | **0.031** | **0.010** | + +The guide control's own rerun (`b40e855eb553` against `fd34bc937628`: +look 0.192, level 3.96/3.65 dB, fine texture 0.016) is inside every +column, so a 9th pair would change nothing. R35's working values (seam look +under 0.2 and level step under 4 dB are noise) came from that one pair. The +8 pairs put the gap at about twice that: **seam look under 0.44 and level +step under 6.2 dB (mean) or 8.8 dB (max) are noise.** Texture is the +exception: its rerun gap (0.068 fine) stays under the S1 guide seed floor, +which stays **0.081**. + +**The floors verdicts now read** (`combined_floor(seed floor, rerun gap)`; +the seed floor is from `counters-controls.json`; a read across continuities +takes the larger of the two rows): + +| scene, continuity | jump | jump max | look | look max | level dB | level dB max | tex fine | tex mid | tex coarse | +|---|---|---|---|---|---|---|---|---|---| +| S1 guide | 0.124 | 0.430 | 0.438 | 0.790 | 6.204 | 8.800 | 0.081 | 0.032 | 0.010 | +| S1 last_frame | 0.124 | 0.430 | 0.438 | 0.790 | 6.204 | 8.800 | 0.068 | 0.087 | 0.010 | +| S2 guide | 0.124 | 0.430 | 0.438 | 0.790 | 6.204 | 8.800 | 0.605 | 1.210 | 1.660 | +| S2 last_frame | 0.124 | 0.430 | 0.438 | 0.790 | 6.204 | 8.800 | 0.170 | 0.066 | 0.214 | +| S3 guide | 0.332 | 0.430 | 0.438 | 0.790 | 6.204 | 8.800 | 0.226 | 0.112 | 0.111 | + +The gap was measured on S1 only. It is applied to every scene because it +is the only same-seed measurement there is. The re-scored counter verdicts +are in `counters.md`, under *Re-scored under R35*. + +## Reading the floor + +- *(Retracted 2026-11-05, see the correction above: same-seed runs are not + always reproducible, and a matching pair does not show that they are.)* + **The engine is deterministic for a seeded run.** S1, S2, S4, S5 and S6 give + bit-identical picture measures across the two runs, so their floor is exactly + 0 on every seam and texture measure (S5 and S6 differ by 0.005-0.02 dB in the + seam level step, an audio-encode wobble). Only S3, whose picture follows a + spoken line per segment, moves between runs. +- **A floor of 0 means the verdict has no tie zone.** `verdict()` calls any + difference past 0 a win or a loss. That is right for "did this arm change the + result", and says nothing about how much another seed would move the same + measure; the floor measures reproducibility of a run, not seed-to-seed + variance. A bench arm that must hold across seeds needs a second seed, which + this task did not run. +- **S3 is the only non-zero floor**: `texture_coarse_seg_last` 0.23 (0.21 vs + 0.44) and `seam_jump_mean` 0.27 are the large ones; use them for S3 verdicts. +- The two S4 runs ran side by side on the two cards and agree exactly. + +## S4 rerun + +The first S4 pair (jobs `0d08eccff9dc`, `14a8b42f73f0`) failed at step 11 of 15, +the `soundtrack` `slice_audio` step, with "slice_audio needs 'sample_rate' with +a raw waveform". Cause: the bench workflow (not the engine). `music_bed`'s +result hands over a raw Music 3 waveform that carries no rate of its own (the +result's `sample_rate: 44100` only labels the file at save time, and +`save: false` here), and `slice_audio` was given no `sample_rate`. The +`templates/minimax/music.json` and `music-video.json` templates pass +`sample_rate` beside a raw waveform for the same reason. Fix: `"sample_rate": +"variable:sample_rate"` on the `soundtrack` step (commit e716fd2c). The pair was +rerun with fresh ids (`-noise-a2`, `-noise-b2`) so the first pair's cached shots +did not serve it. Both succeeded in about 20 minutes each, not the 70 the +launch estimated. + +## Crop check (ruling R10) + +Frames were read through `GET /api/gallery/{name}/frames` at each scene's +moments, on a job of that scene, with the crop drawn over them. A crop must +stay clear of every subject at each moment, keep a 128 px short side (the +coarse band's longest wavelength; a smaller crop returns no coarse value), and +stay off the frame's border. + +| scene | crop before | decision | why | +|---|---|---|---| +| S1, S2 | `[0,0,240,150]` | unchanged (already verified under #1175) | | +| S3 | `[0,0,240,150]` | unchanged, now verified | shop fronts, top left; the walker stays in the middle third in all three moments and a 12-frame contact sheet | +| S4 | `[0,0,240,150]` | `[500,16,160,130]` | the seated man's vest and the standing man's shoulder reach the old crop's lower right corner at frame 62; the new crop is wall in all three shots (a sconce at frame 62, brick at 186 and 434) | +| S5 | `[0,0,240,150]` | unchanged, now verified | sky above the horizon; the cyclist never leaves the lower half over a 12-frame contact sheet. Plain sky holds little fine texture, so S5's fine-band drift is a weak signal | +| S6 | `[0,0,240,150]` | `[690,340,190,135]` | the old crop held the page border and, at frame 240, the lighthouse finial. A smaller interior crop was tried and returned no coarse band; the new one is lower right, clear of the lighthouse and the climber in all three frames (at frame 240 the beam's faint haze touches its top edge, which is lighting, not subject) | + +All six scenes are `crop_verified: true`. S3, S4, S5 and S6 were measured again +with `driver.py measure` (S4's runs were measured with the new crop once +they existed); S1 and S2 crops did not move, so their records stand. + +## Tables + +One table per scene: both runs, the floor (`noise_floor(a, b)`), job ids, +output names, and wall minutes per output minute. S1/S2 are 10 segments +(1042 frames), S3 6 segments (634), S4 four shots under one bed (496), +S5 LTX (359), S6 narration (288). Wall time includes model load. + +### S1 static (`s1-static`) + +| measure | run a | run b | floor | +|---|---|---|---| +| seam_jump_mean | 0.9311 | 0.9311 | 0.0000 | +| seam_jump_max | 1.0900 | 1.0900 | 0.0000 | +| seam_look_mean | 1.5289 | 1.5289 | 0.0000 | +| seam_look_max | 2.6100 | 2.6100 | 0.0000 | +| seam_level_step_db_mean | 9.8567 | 9.8567 | 0.0000 | +| seam_level_step_db_max | 36.5600 | 36.5600 | 0.0000 | +| texture_fine_seg_last | 1.1179 | 1.1179 | 0.0000 | +| texture_mid_seg_last | 0.4015 | 0.4015 | 0.0000 | +| texture_coarse_seg_last | 0.0488 | 0.0488 | 0.0000 | + +| | run a | run b | +|---|---|---| +| job | `00e4000a3ffb` | `b4e31bb17593` | +| output | `BenchS1Static/20261104-000916-390b57bc/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4` | `BenchS1Static-noise-b/20261104-004113-f663789a/final/BenchS1Static-noise-b-chained_keyframe_to_video_audio.0-0.0.mp4` | +| wall min | 21.1 | 18.1 | +| output min | 0.72 | 0.72 | +| wall min per output min | 29.1 | 25.0 | + +### S2 arc (`s2-arc`) + +| measure | run a | run b | floor | +|---|---|---|---| +| seam_jump_mean | 1.1944 | 1.1944 | 0.0000 | +| seam_jump_max | 2.0900 | 2.0900 | 0.0000 | +| seam_look_mean | 1.2844 | 1.2844 | 0.0000 | +| seam_look_max | 2.7100 | 2.7100 | 0.0000 | +| seam_level_step_db_mean | 2.0356 | 2.0356 | 0.0000 | +| seam_level_step_db_max | 5.5400 | 5.5400 | 0.0000 | +| texture_fine_seg_last | 3.5576 | 3.5576 | 0.0000 | +| texture_mid_seg_last | 2.8636 | 2.8636 | 0.0000 | +| texture_coarse_seg_last | 1.9747 | 1.9747 | 0.0000 | + +| | run a | run b | +|---|---|---| +| job | `04978dc6f338` | `51d7b09236f6` | +| output | `BenchS2Arc/20261104-010950-2f5ecfe0/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4` | `BenchS2Arc-noise-b/20261104-012953-430a6737/final/BenchS2Arc-noise-b-chained_keyframe_to_video_audio.0-0.0.mp4` | +| wall min | 20.0 | 17.8 | +| output min | 0.72 | 0.72 | +| wall min per output min | 27.7 | 24.7 | + +### S3 dialogue walk (`s3-dialogue-walk`) + +| measure | run a | run b | floor | +|---|---|---|---| +| seam_jump_mean | 1.4780 | 1.2120 | 0.2660 | +| seam_jump_max | 1.8300 | 1.6000 | 0.2300 | +| seam_look_mean | 1.3780 | 1.3240 | 0.0540 | +| seam_look_max | 1.6100 | 1.8700 | 0.2600 | +| seam_level_step_db_mean | 2.7700 | 3.0000 | 0.2300 | +| seam_level_step_db_max | 8.8100 | 8.6500 | 0.1600 | +| texture_fine_seg_last | 0.1751 | 0.1782 | 0.0031 | +| texture_mid_seg_last | 0.2261 | 0.2076 | 0.0185 | +| texture_coarse_seg_last | 0.2075 | 0.4396 | 0.2320 | + +| | run a | run b | +|---|---|---| +| job | `3c76d1966485` | `d2095670cd89` | +| output | `BenchS3DialogueWalk/20261104-015817-5942f052/final/BenchS3DialogueWalk-chained_keyframe_to_video_audio.0-0.0.mp4` | `BenchS3DialogueWalk-noise-b/20261104-021940-da114475/final/BenchS3DialogueWalk-noise-b-chained_keyframe_to_video_audio.0-0.0.mp4` | +| wall min | 10.8 | 10.5 | +| output min | 0.44 | 0.44 | +| wall min per output min | 24.5 | 24.0 | + +### S4 montage (`s4-montage`) + +| measure | run a | run b | floor | +|---|---|---|---| +| seam_jump_mean | 20.4767 | 20.4767 | 0.0000 | +| seam_jump_max | 22.6900 | 22.6900 | 0.0000 | +| seam_look_mean | 13.4733 | 13.4733 | 0.0000 | +| seam_look_max | 18.2400 | 18.2400 | 0.0000 | +| seam_level_step_db_mean | 3.4967 | 3.4967 | 0.0000 | +| seam_level_step_db_max | 4.4200 | 4.4200 | 0.0000 | +| texture_fine_seg_last | 1.7141 | 1.7141 | 0.0000 | +| texture_mid_seg_last | 0.5798 | 0.5798 | 0.0000 | +| texture_coarse_seg_last | 1.2143 | 1.2143 | 0.0000 | + +| | run a | run b | +|---|---|---| +| job | `568a3da301c4` | `511b37c9bd70` | +| output | `BenchS4Montage-noise-a2/20261104-024801-94372b51/final/BenchS4Montage-noise-a2-film.14-0.0.mp4` | `BenchS4Montage-noise-b2/20261104-024801-6c692da9/final/BenchS4Montage-noise-b2-film.14-0.0.mp4` | +| wall min | 19.7 | 20.8 | +| output min | 0.34 | 0.34 | +| wall min per output min | 57.2 | 60.3 | + +### S5 LTX action (`s5-ltx-action`) + +| measure | run a | run b | floor | +|---|---|---|---| +| seam_jump_mean | 1.2450 | 1.2450 | 0.0000 | +| seam_jump_max | 1.3600 | 1.3600 | 0.0000 | +| seam_look_mean | 2.0850 | 2.0850 | 0.0000 | +| seam_look_max | 2.4300 | 2.4300 | 0.0000 | +| seam_level_step_db_mean | 2.1450 | 2.1400 | 0.0050 | +| seam_level_step_db_max | 2.8500 | 2.8400 | 0.0100 | +| texture_fine_seg_last | 0.5457 | 0.5457 | 0.0000 | +| texture_mid_seg_last | 0.4867 | 0.4867 | 0.0000 | +| texture_coarse_seg_last | 1.5417 | 1.5417 | 0.0000 | + +| | run a | run b | +|---|---|---| +| job | `a91df8d3684c` | `813a4ec70859` | +| output | `BenchS5LtxAction/20261104-005131-8fc124d3/final/BenchS5LtxAction-chained_image_to_video.0-0.0.mp4` | `BenchS5LtxAction-noise-b/20261104-005443-a7939f5d/final/BenchS5LtxAction-noise-b-chained_image_to_video.0-0.0.mp4` | +| wall min | 3.2 | 2.0 | +| output min | 0.25 | 0.25 | +| wall min per output min | 12.8 | 7.9 | + +### S6 narration (`s6-narration`) + +| measure | run a | run b | floor | +|---|---|---|---| +| seam_jump_mean | 40.2000 | 40.2000 | 0.0000 | +| seam_jump_max | 45.3300 | 45.3300 | 0.0000 | +| seam_look_mean | 70.6350 | 70.6350 | 0.0000 | +| seam_look_max | 82.7500 | 82.7500 | 0.0000 | +| seam_level_step_db_mean | 7.7900 | 7.8000 | 0.0100 | +| seam_level_step_db_max | 13.2600 | 13.2800 | 0.0200 | +| texture_fine_seg_last | 1.4011 | 1.4011 | 0.0000 | +| texture_mid_seg_last | 1.2213 | 1.2213 | 0.0000 | +| texture_coarse_seg_last | 0.8531 | 0.8531 | 0.0000 | + +| | run a | run b | +|---|---|---| +| job | `54240901f86a` | `5670fac7bc34` | +| output | `BenchS6Narration/20261104-005645-a4669737/final/BenchS6Narration-film.9-0.0.mp4` | `BenchS6Narration-noise-b/20261104-005815-12f0dc21/final/BenchS6Narration-noise-b-film.9-0.0.mp4` | +| wall min | 1.6 | 1.3 | +| output min | 0.20 | 0.20 | +| wall min per output min | 7.9 | 6.6 | + +Wall minutes per output minute, across the scenes: S1 25-29, S2 25-28, S3 24, +S4 57-60 (four H3 shots plus a music bed), S5 8-13, S6 7-8. The first run of a +pair in S5 and S6 paid model load; the second did not. + +## Cheap-setting check (6 segments, #1175 ordering) + +Arms A (control), B (`guide_noise` 0.4) and C (`chain.anchor: "segment_1"`) at +6 segments, moments frame 62 / 277 / 583, against #1175's 10-segment ordering +(`docs/proposals/complete/h3-guide-chain-drift-complete.md`, Stage 3). Jobs in +`bench/records/cheap-checks.json`. + +| scene | measure | A | B | C | #1175 (10 segments) | +|---|---|---|---|---|---| +| S1 static | `texture_fine_seg_last` | 0.503 | 0.854 | 0.190 | A 1.00, B 1.47 (rougher), C 0.40 | +| S1 static | `seam_look_mean` | 1.622 | 1.392 | 1.404 | A 1.67, B 1.11, C 1.27 | +| S2 arc | `seam_jump_mean` | 1.362 | 1.166 | 1.854 | A 1.13, B 1.09, C 2.20 | +| S2 arc | `seam_look_mean` | 1.286 | 1.694 | 4.270 | A 0.98, B 1.07, C 3.62 | + +`verdict(A, arm, floor, primary)` with the S1 and S2 floors (all 0, so any +difference counts): + +| scene | arm | primary | verdict | reading | +|---|---|---|---|---| +| S1 | B | `texture_fine_seg_last` | loses | B is rougher, as in #1175 | +| S1 | B | `seam_look_mean` | mixed | wins the look, loses texture | +| S1 | C | `texture_fine_seg_last` | mixed | C beats A on texture, worse elsewhere (seam jump 1.16 vs 0.88) | +| S1 | C | `seam_look_mean` | mixed | | +| S2 | B | `seam_jump_mean` | mixed | best jump of the three, look slightly worse than A | +| S2 | B | `seam_look_mean` | loses | 1.69 vs 1.29, as B 1.07 vs A 0.98 was slightly worse at 10 segments | +| S2 | C | `seam_jump_mean` | loses | worst | +| S2 | C | `seam_look_mean` | loses | worst by a wide margin | + +**Verdict: 6 segments reproduce #1175's ordering on every row**: on the static +scene C beats A on texture and B is rougher than both, and B and C both +improve on A's look; on the arc, C is worst on both seam measures and B has the +best jump, with a look slightly worse than A. The bench may use 6 segments. +The values are not the 10-segment values (the texture drift is smaller at 6 +segments, as it has fewer segments to compound over), so a 6-segment bench +result is a ranking, not a number to compare with #1175's tables. The 960x544 +size and the draft tier were not separately checked; the 6-segment arms ran at +the bench workflows' 960x544 and draft settings, and the ordering held there. + +## Chain check thresholds (ruling R19) + +`dw/pipeline_processors/chain_check.py` measures each segment with +`check_segment`: the seam's `jump_ratio` and `look_ratio` against the segment +before, and the whole-frame fine-band luma against segment 1's. The 28 seams of +the unchecked S1, S2 and S3 noise-run chains (S1 and S2 run a, S3 runs a and b; +S1 and S2 run b are bit-identical to run a) were measured with that function on +the decoded output, each segment's kept frames against the one before: + +| measure | P50 | P99 | max | set | +|---|---|---|---|---| +| `jump_ratio` | 1.10 | 2.02 | 2.08 | `JUMP_RATIO_MAX` 25 -> 4.0 | +| `look_ratio` | 1.28 | 2.67 | 2.71 | `LOOK_RATIO_MAX` 12 -> 5.5 | +| `texture_ratio` (either way) | | 2.99 | 3.13 | kept 2.0, provisional | + +The margin is about a factor of 2 on the P99 (jump 2.02 -> 4.0; look 2.67 -> 5.5, rounded up to half a point). 4.0 stays +below an intended cut's 9.9-20.3 and above a dissolve's 1.8; 5.5 stays well +under the 19.2 a saturation-1.6 grade reads. + +The texture ceiling is not set from these runs, for two reasons. The two +same-seed runs of S1 and S2 are identical, so they show no spread to take a +floor from. And the texture ratio of an unchecked chain is the drift the check +exists to catch (1.07 rising to 2.43 over S1's ten segments, 1.04 falling to +0.32 over S2's), so its percentile measures the defect and a ceiling set at it +would pass the defect. It needs runs of a checked chain, or a repeat over +several seeds, to set. + +The three scenes are one subject type each and one seed; the P99 of 28 values +is the maximum, in effect, so a faster-moving or darker scene may read above +these ceilings and be re-rolled on a good segment. `check` is opt-in per chain, +and a workflow can name its own ceilings. diff --git a/bench/records/performer-replacement.md b/bench/records/performer-replacement.md new file mode 100644 index 000000000..b0030403a --- /dev/null +++ b/bench/records/performer-replacement.md @@ -0,0 +1,134 @@ +# Performer replacement gate (Task 16, #1194) + +**Verdict: works.** A ref2va regional edit at `refine_strength` 1.0 inside a per-frame person +mask replaces the performer on an S4 cut. The new face matches the replacement portrait +better than a fresh ref2va render of the same portrait does. Outside the mask the source +holds, and the soundtrack is the source's. The one flaw is a background seam: the edit +regenerates a band of about 64-80 px around the person, and that band can redraw nearby +set geometry (here the couch back) so it no longer lines up with the kept source. No +engine change was needed. Recommendation: file the follow-up feature below. + +Run 2026-11-07 on lem's bench (H3 on `:8766` behind the int8 queue, masking and face +checks on `:8767`). + +## Setup + +- **Cut:** S4 `deflect` (the close-up of the shaved-head man in the grey flannel shirt + reading a magazine; 960x544, 124 frames, 24 fps, dialogue). Source run + `BenchS4Montage/20261104-040255-9813688b`, file + `intermediate/BenchS4Montage-shot@deflect.3-0.0.mp4`. +- **Replacement:** `asset:bench/t26-stranger.jpg`, a bearded man in round glasses, a + mustard corduroy jacket and a green t-shirt, on a grey studio backdrop. +- **Mask** (`bench/workflows/t16-person-mask.json`): `video_frames`, then `segment` + (GroundingDINO base + SAM2 hiera-large, both Apache-2.0; prompt `person`, threshold 0.3). + `segment` already runs per frame on a video (`per_frame`), so the catalog gives a + per-frame person mask today with no new code. Job `9fb7e66a2211`, 53 s on cuda:1. + - Every frame has a person: coverage 0.27-0.35 of the frame (mean 0.31), union 0.38. + - Mean frame-to-frame flip is 0.55% of pixels (max 6.1%, at the page turn). + - The mask holds the whole body and leaves out the magazine. RMBG-2.0 (`remove_background`) + was not used: it is CC BY-NC. +- **Edit** (`bench/workflows/t16-performer-swap.json`): S4's own ref2va shot pipeline + (int4, ref2v turbo 8-step LoRA, 9 steps, `video_shift` 12, sparse attention). + - Inputs: the cut as `source_video` and `hold_audio`, the mask as `mask_video`, + `mask_grow` 1, `refine_strength` 1.0, and the portrait as ``. + - Prompt: the cut's own prompt with `` rewritten as the portrait's man, plus + "the grey studio backdrop of `` is not used". +- **Control:** the same step with the source, mask and `refine_strength` dropped. This is + a fresh ref2va render of the portrait in the same scene, and it is the ceiling a plain + generation reaches. + +Arms: `bench/arms/t16-performer-replacement.json`. + +## Numbers + +| Run | Job | Wall | face_drift vs portrait, median (min) | +|---|---|---|---| +| Source cut (baseline) | face check `d1e61cba18ef` | - | 0.056 (0.013) | +| Swap, seed 1194 | `0aa481a58b58` (face `70f69429f43a`) | 6.2 min (cold load) | **0.759** (0.700) | +| Swap, seed 2002 | `a78e8dbf981d` (face `70e0e3e71303`) | 4.0 min | **0.728** (0.662) | +| Fresh ref2va control, seed 1194 | `aec28b177ec6` (face `27b86eecde21`) | 3.4 min | 0.683 (0.579) | + +The face_drift threshold is 0.363. All 16 sampled frames found a face on every run. + +Source against edit by region (a scratch numpy script, not committed: PSNR, 7 px windowed luma SSIM and mean abs RGB diff, with the regions taken per frame from that frame's mask): + +| Region | PSNR s1194 / s2002 | SSIM (7 px window) | Mean abs diff | +|---|---|---|---| +| Inside the mask | 10.8 / 10.9 dB | 0.22 / 0.23 | 60 / 59 | +| 1-32 px outside | 12.7 / 13.3 dB | 0.34 / 0.35 | 41 / 38 | +| 33-64 px outside | 18.0 / 18.6 dB | 0.61 / 0.64 | 20 / 19 | +| More than 96 px outside | **37.5 / 37.5 dB** | 0.964 / 0.964 | 2.8 / 2.8 | + +- **Where the change stops.** Mean abs diff by distance from the union mask, in 16 px bands: + 29, 26, 19, 11, 5, 3, then 3. The edit stops about 64-80 px past the person's + silhouette, which is the patch cell plus `mask_grow` 1. Past that the residual of about + 3 is the VAE round trip. +- **Audio.** The swap's track correlates 0.9988 with the source at -0.03 dB, so + `hold_audio` held it. The control's track does not match the source (corr -0.06). +- **Temporal stability.** Mean frame-to-frame luma change in the 1-64 px band is 1.5-1.6, + against 0.52 in the source. More than 64 px out it is 0.12-0.20, against 0.16-0.26 in + the source. The band moves more because the new body moves there, not because it + flickers (frames 40-47 by eye show no shimmer). + +## By eye (frames 0, 62 and 123; full-resolution crops at the shoulder and the head) + +- **Identity.** On both seeds the man is the portrait's man: round wire glasses, a grey and + dark beard, greying swept hair, the mustard corduroy jacket and the green t-shirt. + Nothing of the bald source performer is left. Pose, framing, the page turn and the + magazine position follow the source. The mouth is open on the line at frame 62 and + closed at 0 and 123, as in the source, so lip timing to the held dialogue looks right at + these three moments. This was not checked frame by frame. +- **Outside the mask.** The brick wall, curtain, window and couch arm away from him are the + source's. The control instead renders a different set, with a stair railing and a + different couch, which shows how much the mask is holding. +- **Seam.** The brick wall continues through the regenerated band behind the head with no + visible line: bricks and mortar line up on both seeds. The couch back does not. Beside + the right shoulder the regenerated band redraws the back cushion lower, or as a + different cushion, and its top edge steps where it meets the kept couch. The step is + steady across frames, not flicker, but a viewer would see it on a still. +- **Inside the grown cells.** The new man is slimmer than the source performer. The space + he no longer fills, which the mask still covers, is filled with plausible couch and wall. + The magazine cover lies inside the grown cells and is redrawn: new garbled masthead text + on both seeds. + +## What this answers for #1194 + +- The edit itself needs nothing new. ref2va with `source_video` and `mask_video` already + works, and outside the mask is kept by the engine, so Directors-Cut's `composite_part` + step is built in. +- The identity holds at strength 1.0 because it comes from the reference, as the issue + predicted. It scores above a fresh render, since the source's pose and framing keep the + face frontal. +- The per-frame mask is already in the catalog for a single-person cut (`segment` over + `video_frames`). +- What is missing is the multi-person and multi-cut machinery, not the swap: + - A per-person track. `segment` detects every frame independently and merges all + matches, so with two people it cannot say which person a mask belongs to. + - Cut detection, to split a multi-shot source into cuts. + +## Follow-up to file (owner: Don) + +**feature: performer replacement, the per-cut loop around a masked ref2va edit (#1194).** +Stages: + +1. **Cut-detection task.** Find hard cuts in a source clip and return shot boundaries in + dw's `shots` form, so `assess_output`, `for_each` and the chain tools can read them. A + content or adaptive detector in the PySceneDetect style is BSD-3; check its licence + first. +2. **Person-track task.** Per-person, per-frame masks across a cut: SAM2 video propagation + from a first-frame `segment` box per person, under the same Apache-2.0 weights. It + returns one mask video per tracked person, plus a contact sheet so a caller can say + which person is "Subject N". +3. **Template** `templates/minimax/replace-performer`. `for_each` over cuts: trim the cut + to the `17n + 5` grid, track, run the ref2va regional edit (`refine_strength` 1.0, the + portrait as ``, `hold_audio` the cut), then `concat_videos` and `pair_audio` + with the source track. Each cut gets its own seed, so a bad cut can be re-rolled alone. +4. **The seam (bench it in the template's stage).** Try a tighter edit and see whether it + stops the band redrawing set geometry (the couch-back step here), at the cost of a + harder edge: + - `mask_grow` 0; + - a mask eroded by part of a cell before it is pooled. + Measure the seam with the 16 px distance bands above and the crops at the shoulder. + +Out of scope here, and still open in #1194: the VLM cast-listing prompt, two-person cuts, +the community swap LoRA (unsourced) and FastH3 Omni (#980). diff --git a/bench/records/repair.json b/bench/records/repair.json new file mode 100644 index 000000000..69a88b9d5 --- /dev/null +++ b/bench/records/repair.json @@ -0,0 +1,23 @@ +[ + { + "arm": "repair-s3-seg4", + "scene": "s3-dialogue-walk", + "seed": 4, + "job_id": "46d9c4f4b1d7", + "status": "succeeded", + "error": null, + "output": "MiniMaxH3RepairSegment/20261105-092930-629e7ec0/final/MiniMaxH3RepairSegment-splice.2-0.0.mp4", + "measures": { + "seam_jump_mean": 1.466, + "seam_jump_max": 1.94, + "seam_look_mean": 1.59, + "seam_look_max": 2.57, + "seam_level_step_db_mean": 2.314, + "seam_level_step_db_max": 8.81, + "texture_fine_seg_last": 0.7177642846266591, + "texture_mid_seg_last": 0.054633719270975885, + "texture_coarse_seg_last": 0.022286401911626865, + "wall_minutes": 8.123658215999603 + } + } +] \ No newline at end of file diff --git a/bench/records/repair.md b/bench/records/repair.md new file mode 100644 index 000000000..e4a425280 --- /dev/null +++ b/bench/records/repair.md @@ -0,0 +1,188 @@ +# Repair in place and the capstone gate (Task 14 Steps 3-4) + +2026-11-05. Bench server lem:8766 at `082e93ce`. Arms: `bench/arms/repair.json`; +record: `repair.json`. + +## Step 3: repair segment 4 of the check bench's S3 take + +The take is Task 13's `chk-s3-seg4-broken` (`fdea6eba4a33`, `check.md`), run +with `keep_segments`. Its segment 4 is broken - a cut to a night harbour 26 +frames in - though the check passed the re-roll, so the shot carries no +`check_failed`; it was repaired as the segment the eye flags. +`templates/minimax/repair-segment` with `chain_take` the take, `segment: 4`, +segment 4's own S3 prompt (the street, the smile, its line), seed 4 and the +template's defaults, which are the chain's settings (960x544, 5 steps, int4, +turbo LoRA, sparse attention). Job `46d9c4f4b1d7`, 8.1 min on cuda:0: +`render 175 frames, head guide 22 at 0, tail guide 39 at 136; take frames +328..441 of 634 replaced`. + +### Seams + +`analyze_seams` on the take before and after. The repaired shots are recorded +as chain seams (`segment 4 (repaired)`, trim 22 and 12, `crossfade_ms` 3), so +the probe measures them like the chain's own. + +| seam | before: jump / look / level dB | after: jump / look / level dB | +|---|---|---| +| 1 (seg 1-2) | 1.75 / 1.57 / 8.81 | 1.74 / 1.51 / 8.81 | +| 2 (seg 2-3) | 1.81 / 1.54 / 1.50 | 1.79 / 1.49 / 1.50 | +| **3 (seg 3 - repaired 4)** | 0.42 / 1.36 / 2.89 | **1.57 / 1.11 / 0.79** | +| **4 (repaired 4 - seg 5)** | 0.13 / 0.59 / 4.43 | **0.29 / 1.27 / 0.35** | +| 5 (seg 5-6) | 1.94 / 2.58 / 0.66 | 1.94 / 2.57 / 0.12 | + +The chain's other seams (1, 2, 5) average jump 1.82, look 1.86, level 3.48 dB. +Against them, under `combined_floor()` at the S3 guide row (jump 0.332, look +0.438, level 6.204 dB): + +- seam 3: jump 1.57 (tie, inside 0.332), look 1.11 (better by 0.75), level + 0.79 dB (tie); +- seam 4: jump 0.29 (better by 1.53), look 1.27 (better by 0.59), level + 0.35 dB (tie). Its frame step is 1.54 grey levels against 4.4-8.0 at the + chain's seams: the tail guide is segment 5's own frames, so the rejoin is + nearly invisible. + +Both repaired seams are as good as or better than the chain's own. The +"before" seams 3 and 4 read low only because the harbour's fast pan raised +segment 4's typical step to 20; they were not good seams. Nothing outside +the span moved: seams 1, 2 and 5 are unchanged apart from encode noise +(level step at seam 5 0.66 -> 0.12 dB, inside the floor). + +### By eye: the repair cannot undo a break that has already propagated + +Take frames 328-~400 are now the street, the woman smiling and walking, as +asked. From ~404 the render dissolves to the harbour, and the take stays on +it until segment 5's own cut back to the street at 449. The cause is the +geometry, not the render: the tail guide is segment 5's frames 12-50 (take +442-480), and segment 5 opened on the harbour, because its guide was broken +segment 4's tail; its first 19 kept frames are harbour. The repair is held to +end on that guide, so it has to reach the harbour. `bench/cuts.py` on the +repaired take: segment 4 14.5 (at 404), segment 5 17.0 (at 449). + +So a broken segment in a guide chain breaks the next segment's opening too, +and repairing segment k alone then cannot clean the take. Two consequences: + +- The check has to stop the break at segment k, before its tail becomes a + carry - which is what `cut_ratio` (`d37c3192`, `check.md`) adds. +- A repair of a break that reached segment k+1 needs segments k and k+1 + re-rendered together, between k-1 and k+2 (22 + 204 + 39 = 265 frames + plus the grid overrun, 11-12 s, inside H3's 345-frame ceiling, so one + render - but the template takes one segment), or the take re-run from + segment k. Not built. + +### Peak VRAM + +`nvidia-smi` sampled every 0.5 s (`~/dw-bench-t13/vram.csv` on lem). The +repair ran alone on cuda:0, starting from an emptied card (637 MiB after the +worker's full cleanup). + +| phase | peak (nvidia-smi) | +|---|---| +| load (SDNQ quantizes on cuda) 09:29:30-09:31:20 | 19,377 MiB = 18.92 GiB | +| render, 175 frames + 61 guide frames, 960x544 | 15,319 MiB = 14.96 GiB | +| for reference, the S3 chain's segments (124 + 22) | 13,479 MiB = 13.16 GiB | + +The template's estimate is `16.3 + 42.84 x 960 x 544 x 175 / 2^30` = 19.95 GiB. +It holds: 1.0 GiB over the whole job's peak (the load transient, which the +base term carries for every H3 template) and 5.0 GiB over the render's. The +derived voxel rate is borne out at the margin: the repair's render peak sits +1.80 GiB over a chain segment's for 90 more frames of rows on the same +canvas, 41 bytes a voxel against the 42.84 the template folds the guides +into. No change to the estimate. + +## Step 4: the capstone gate + +**The gate is open.** Plan: spike #1187's late capstone "if Task 11 left S2 +(moving camera) without a winning counter". It did, and the R35 re-score kept +it (`counters.md`: S2 "none - every guide counter loses seam look; both +last_frame counters freeze the arc"; weakened, not flipped). The S2 failure +the capstone targets is real by eye (the sky blows out to white and the scene +invents palms by the last segment), and the check bench shows it as texture +(segments 7-10 fall to 0.48-0.32 of segment 1's fine band, `check.md`). + +**No existing condition-row feature expresses it.** A guide must start on the +17-frame grid and end by `num_frames` (`guide_end_problem`), and every +condition row is in the forward from the first step; the keystone replaces the +keyframe; the latent anchor matches statistics after the loop. A capstone +needs a row past the render's end and a step gate. + +**The smallest engine change, built as a spike** (commit `45115d6a`, behind +`DW_EXPERIMENTAL_REFINE_WINDOWS`, which the bench server runs with; refused +without it; no new module): + +- a guide may carry `past_end: true` (one frame, no audio, timed one latent + past the render's last) and `enter_below_sigma` (`h3_guides`, + `h3_guide_steps`, which records the gated rows); +- the windowed denoiser (`h3_windows`, already installed under the flag) + leaves the gated rows out of the forward while the step's sigma (1 - t) is + at or above the gate - the sequence less those rows, as a window is - and + lets them in below it; +- chain key `experimental_capstone: {enter_below_sigma, from_end: 5}`: segment + 1's frame `num_frames - from_end` (119 at 124 frames) is appended to every + later segment's guides. + +Tests: `tests/test_h3_capstone.py` (27) - the guide keys and refusals, the +row's time one latent past the end, a gated forward equal to the forward of +the sequence without the rows (on the tiny transformer), the rows back in +below the gate, the chain feeding frame 119 to segments 2+, and an end-to-end +tiny-graph render where the gate changes the result. + +**Not yet run: it needs a deploy.** Arms ready in `bench/arms/capstone.json` +(`s2-arc.json` gained a `capstone` variable, null by default): S2 at seeds 1166 +and 2, gates 0.9 (the last two of H3's four turbo forwards, sigma 0.857 and +0.667) and 0.7 (the last only), about 18 min each, read against +`ctl-guide-s2-*` (`counters-controls.json`) under the S2 guide floor, by +eye for the arc. Build it only if it wins S2. + +### The capstone gate: run (2026-11-05, deploy `155ae56c`) + +Arms `bench/arms/capstone.json` (S2, 10 segments, 41 s, guide) plus one fresh +seed-2 control, `bench/arms/capstone-controls.json`; records `capstone.json` +and `capstone-controls.json`. The 10-segment controls are the guide runs at +the same shape: seed 1166 `chk-s2-fp` (`604e115324b7`, deploy `082e93ce`, +equal to the noise-floor runs to the third decimal), seed 2 +`ctl10-guide-s2-s2` (`87038dfc0892`, this deploy), which reproduces round +1's `control-s2-seed2` exactly. (`ctl-guide-s2-*` in `counters-controls.json` +are 6-segment takes and do not match the arms' shape.) + +**Labels** (`bench/segment1.py`): all four capstone arms and both controls +are **clean** (S2@1166 `ed5094c1/0a819c94`, S2@2 `14a80618/b4d1d486`), so +every read below is clean against clean. + +**Floor:** `combined_floor()` S2 guide row (jump 0.124/0.430, look +0.438/0.790, level 6.2/8.8 dB, texture 0.605/1.210/1.660). + +| arm | jump mean / max | look mean / max | level dB mean / max | tex fine / mid / coarse | wall | verdict | +|---|---|---|---|---|---|---| +| control @1166 | 1.194 / 2.09 | 1.284 / 2.71 | 2.04 / 5.54 | 3.558 / 2.864 / 1.975 | 18.2 | | +| cap 0.9 @1166 | 1.291 / 1.85 | 1.656 / **4.52** | 2.53 / 5.25 | 3.372 / 2.664 / 2.193 | 20.6 | no gain; look max **worse** by 1.81 | +| cap 0.7 @1166 | 1.207 / 1.95 | 1.099 / 2.36 | 2.03 / 7.26 | 3.310 / 2.468 / 2.018 | 18.4 | **tie** on every measure | +| control @2 | 1.163 / 1.83 | 1.211 / 1.77 | 6.85 / 24.46 | 3.386 / 3.378 / 2.906 | 17.8 | | +| cap 0.9 @2 | 1.197 / 1.70 | 1.380 / 2.16 | 7.22 / **35.91** | 3.452 / 3.320 / 2.874 | 18.4 | no gain; level max **worse** by 11.5 dB | +| cap 0.7 @2 | 1.234 / 1.72 | 1.429 / 2.12 | 6.11 / 33.00 | 3.652 / 3.526 / 3.504 | 18.3 | **tie** on every measure | + +(`verdict()` with primary fine texture, look or jump: 0.9 reads `mixed` +at both seeds, a tie on the primary with one measure worse past the floor; +0.7 reads `tie`.) + +**By eye** (control | 0.9 | 0.7 at 2.5, 20 and 41 s): segment 1 is identical, +as it must be. At 41 s, seed 1166, the control's sky is blown to white while +both capstone takes keep a blue sky with clouds, which is the failure the +capstone targets, partly held. It does not hold the scene: in all three takes +the seated statue has become a standing figure seen from behind, and the 0.7 +take still invents palms. At seed 2 the three end on three different gardens, +and 0.7's sky is pale. The texture numbers agree: no arm moves fine/mid/coarse +drift past the S2 floor, which is wide (0.6/1.2/1.7) because S2's seeds +disagree so much. + +**Cost:** one gated row past the end, in the last one or two of four +forwards. Wall time +0.2 to +2.4 min on about 18 min, inside the S2 runs' +own spread (17.8-20.0 min): unmeasurable. + +**Verdict: the late capstone does not win S2 at either gate.** Neither gate +beats the control past the floor on drift or seams at either seed. 0.9 (the +last two forwards) is worse past the floor on one seam measure at each seed +(look max at 1166, level max at 2); 0.7 (the last forward) ties everything. +The one visible gain, a sky held blue at seed 1166, does not reach the +numbers and does not repeat at seed 2. Under the plan's rule ("build it only +if it wins S2"), it is not built: the spike (`45115d6a`) stays behind the +experimental flag, and S2 keeps "no counter" in the defaults. diff --git a/bench/records/segment1-canonical.json b/bench/records/segment1-canonical.json new file mode 100644 index 000000000..ab7ad5712 --- /dev/null +++ b/bench/records/segment1-canonical.json @@ -0,0 +1,47 @@ +{ + "s1-static@1166": { + "fingerprint": "d59a1259/b2c2c63e", + "source": "cold load", + "jobs": ["0a40c41af97c", "ae932366ea0e", "fd34bc937628"] + }, + "s1-static@2": { + "fingerprint": "f47df963/f88334a9", + "source": "cold load", + "jobs": ["08e32162985c"] + }, + "s2-arc@1166": { + "fingerprint": "ed5094c1/0a819c94", + "source": "cold load", + "jobs": ["dedd341c9108", "242358078691"] + }, + "s2-arc@2": { + "fingerprint": "14a80618/b4d1d486", + "source": "consensus, no cold load recorded: two workers agree", + "jobs": ["911cf038e1d1", "01146e925c94"] + }, + "s3-dialogue-walk@1166": { + "fingerprint": "289012f1/d21b04b0", + "source": "consensus, no cold load recorded: 8 of 9 runs agree", + "jobs": ["3c76d1966485", "c0b92d0d0cd4"] + }, + "s3-dialogue-walk@2": { + "fingerprint": "31765efa/48363338", + "source": "consensus, no cold load recorded: two runs agree", + "jobs": ["f93a17539b2a", "64b4a78c74a2"] + }, + "s1-static@1166+lora": { + "fingerprint": "a76a366c/ff5c4416", + "source": "cold load (a LoRA change reloads the pipeline); one run", + "jobs": ["049d652e1648"] + }, + "s2-arc@1166+lora": { + "fingerprint": "c7b4068d/33f00579", + "source": "cold load (a LoRA change reloads the pipeline); one run", + "jobs": ["9124d417577d"] + }, + "s3-dialogue-walk@1166+lora": { + "fingerprint": "bfcb3f23/949a7e65", + "source": "cold load (a LoRA change reloads the pipeline); one run", + "jobs": ["c50ec5961418"] + } +} diff --git a/bench/records/segment1-fingerprints.json b/bench/records/segment1-fingerprints.json new file mode 100644 index 000000000..1740a8c3a --- /dev/null +++ b/bench/records/segment1-fingerprints.json @@ -0,0 +1,91 @@ +{ + "BenchS1Static/20261104-000916-390b57bc/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4": "d59a1259/b2c2c63e", + "BenchS1Static-noise-b/20261104-004113-f663789a/final/BenchS1Static-noise-b-chained_keyframe_to_video_audio.0-0.0.mp4": "d59a1259/b2c2c63e", + "BenchS2Arc/20261104-010950-2f5ecfe0/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4": "ed5094c1/0a819c94", + "BenchS2Arc-noise-b/20261104-012953-430a6737/final/BenchS2Arc-noise-b-chained_keyframe_to_video_audio.0-0.0.mp4": "ed5094c1/0a819c94", + "BenchS3DialogueWalk/20261104-015817-5942f052/final/BenchS3DialogueWalk-chained_keyframe_to_video_audio.0-0.0.mp4": "289012f1/d21b04b0", + "BenchS3DialogueWalk-noise-b/20261104-021940-da114475/final/BenchS3DialogueWalk-noise-b-chained_keyframe_to_video_audio.0-0.0.mp4": "73d59e33/5af0ad9e", + "BenchS4Montage-noise-a2/20261104-024801-94372b51/final/BenchS4Montage-noise-a2-film.14-0.0.mp4": "bbeb3a21/060fc3ea", + "BenchS4Montage-noise-b2/20261104-024801-6c692da9/final/BenchS4Montage-noise-b2-film.14-0.0.mp4": "bbeb3a21/060fc3ea", + "BenchS5LtxAction/20261104-005131-8fc124d3/final/BenchS5LtxAction-chained_image_to_video.0-0.0.mp4": "077a2194/9e7bfd50", + "BenchS5LtxAction-noise-b/20261104-005443-a7939f5d/final/BenchS5LtxAction-noise-b-chained_image_to_video.0-0.0.mp4": "077a2194/d4f401f1", + "BenchS6Narration/20261104-005645-a4669737/final/BenchS6Narration-film.9-0.0.mp4": "1fb2ed5e/6c32c248", + "BenchS6Narration-noise-b/20261104-005815-12f0dc21/final/BenchS6Narration-noise-b-film.9-0.0.mp4": "1fb2ed5e/071b7207", + "BenchS1Static/20261104-003017-655187be/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4": "d59a1259/b2c2c63e", + "BenchS1Static/20261104-005919-f3588d64/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4": "d59a1259/b2c2c63e", + "BenchS1Static/20261104-011029-6cf2a1b0/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4": "d59a1259/b2c2c63e", + "BenchS2Arc/20261104-014743-a12ce1e7/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4": "ed5094c1/0a819c94", + "BenchS2Arc/20261104-020905-75a1b6b3/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4": "ed5094c1/0a819c94", + "BenchS2Arc/20261104-023013-fa9a28f1/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4": "d695100d/68551c3e", + "BenchS1Static/20261104-031311-3ab97814/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4": "f47df963/f88334a9", + "BenchS2Arc/20261104-033426-51d13983/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4": "14a80618/b4d1d486", + "BenchS3DialogueWalk/20261104-035218-d2733a3a/final/BenchS3DialogueWalk-chained_keyframe_to_video_audio.0-0.0.mp4": "31765efa/48363338", + "BenchS4Montage/20261104-040255-9813688b/final/BenchS4Montage-film.14-0.0.mp4": "743ade3b/23e30cc1", + "BenchS5LtxAction/20261104-031441-74587ca7/final/BenchS5LtxAction-chained_image_to_video.0-0.0.mp4": "979ad93c/6843ac34", + "BenchS6Narration/20261104-031331-66be03f2/final/BenchS6Narration-film.9-0.0.mp4": "1fb2ed5e/85557817", + "BenchS1Static/20261104-042305-3bcc7171/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4": "052a7db3/e69f649f", + "BenchS1Static/20261104-044853-894054fa/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4": "bd8d3b3c/7330711f", + "BenchS2Arc/20261104-051138-5de055bd/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4": "72d37749/9872e60d", + "BenchS2Arc/20261104-052935-e40247b8/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4": "fbfddccc/5171f78c", + "BenchS1Static/20261104-060443-90c37bc2/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4": "d59a1259/b2c2c63e", + "BenchS1Static/20261104-062828-0642442c/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4": "f47df963/f88334a9", + "BenchS1Static/20261104-065215-5b43dda0/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4": "56e0831e/ac39b399", + "BenchS1Static/20261104-071117-ae50341d/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4": "4302cf65/03cb1fa0", + "BenchS2Arc/20261104-073022-cd97c6a1/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4": "ed5094c1/0a819c94", + "BenchS2Arc/20261104-075344-d76c369a/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4": "14a80618/b4d1d486", + "BenchS2Arc/20261104-080501-f258d5fa/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4": "57cbd211/a1d44644", + "BenchS2Arc/20261104-082803-71d81bfc/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4": "130afe0c/f90613fc", + "BenchS3DialogueWalk/20261104-084517-6db94fb9/final/BenchS3DialogueWalk-chained_keyframe_to_video_audio.0-0.0.mp4": "289012f1/d21b04b0", + "BenchS3DialogueWalk/20261104-090907-773b20e5/final/BenchS3DialogueWalk-chained_keyframe_to_video_audio.0-0.0.mp4": "31765efa/48363338", + "BenchS1Static/20261104-054934-fdc9b551/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4": "d59a1259/b2c2c63e", + "BenchS1Static/20261104-061515-5949edcc/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4": "d59a1259/b2c2c63e", + "BenchS1Static/20261104-063901-253f19de/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4": "56e0831e/ac39b399", + "BenchS1Static/20261104-070045-f9d2dc73/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4": "fe993ff4/fb5cdcbf", + "BenchS1Static/20261104-071947-19ca76a4/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4": "fe993ff4/fb5cdcbf", + "BenchS1Static/20261104-074252-80046177/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4": "fe993ff4/fb5cdcbf", + "BenchS1Static/20261104-080411-0301f0c0/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4": "a76a366c/ff5c4416", + "BenchLfS1Static/20261104-081730-5241ac41/final/BenchLfS1Static-chained_keyframe_to_video_audio.0-0.0.mp4": "d59a1259/b2c2c63e", + "BenchS1Static/20261104-083621-b688e963/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4": "d59a1259/b2c2c63e", + "BenchS2Arc/20261104-085551-68fe5c04/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4": "57cbd211/a1d44644", + "BenchS2Arc/20261104-091939-682f41d1/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4": "57cbd211/a1d44644", + "BenchS2Arc/20261104-093256-ab1bf1a9/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4": "57cbd211/a1d44644", + "BenchS2Arc/20261104-094613-88f4b225/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4": "57cbd211/a1d44644", + "BenchS2Arc/20261104-095641-03fe99ed/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4": "57cbd211/a1d44644", + "BenchS2Arc/20261104-100709-3398e6c0/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4": "57cbd211/a1d44644", + "BenchS2Arc/20261104-101739-05b1c816/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4": "c7b4068d/33f00579", + "BenchLfS2Arc/20261104-103052-7d0e533a/final/BenchLfS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4": "e6e70a03/0a819c94", + "BenchS2Arc/20261104-103927-af5f9b3e/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4": "ed5094c1/0a819c94", + "BenchS3DialogueWalk/20261104-105035-5a02f9f8/final/BenchS3DialogueWalk-chained_keyframe_to_video_audio.0-0.0.mp4": "289012f1/d21b04b0", + "BenchS3DialogueWalk/20261104-110353-90e5c4ef/final/BenchS3DialogueWalk-chained_keyframe_to_video_audio.0-0.0.mp4": "289012f1/d21b04b0", + "BenchS3DialogueWalk/20261104-111709-bec10029/final/BenchS3DialogueWalk-chained_keyframe_to_video_audio.0-0.0.mp4": "289012f1/d21b04b0", + "BenchS3DialogueWalk/20261104-113024-86593c80/final/BenchS3DialogueWalk-chained_keyframe_to_video_audio.0-0.0.mp4": "289012f1/d21b04b0", + "BenchS3DialogueWalk/20261104-114056-8aecac36/final/BenchS3DialogueWalk-chained_keyframe_to_video_audio.0-0.0.mp4": "289012f1/d21b04b0", + "BenchS3DialogueWalk/20261104-115128-bf419738/final/BenchS3DialogueWalk-chained_keyframe_to_video_audio.0-0.0.mp4": "289012f1/d21b04b0", + "BenchS3DialogueWalk/20261104-120204-7fe540b4/final/BenchS3DialogueWalk-chained_keyframe_to_video_audio.0-0.0.mp4": "bfcb3f23/949a7e65", + "BenchS1Static-r2/20261104-122900-4ae9b3e3/final/BenchS1Static-r2-chained_keyframe_to_video_audio.0-0.0.mp4": "d59a1259/b2c2c63e", + "BenchS1Static/20261104-124146-e3030154/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4": "f47df963/f88334a9", + "BenchS1Static/20261104-125709-51dda187/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4": "f47df963/f88334a9", + "BenchS1Static/20261104-131025-68fb6c3e/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4": "f47df963/f88334a9", + "BenchS1Static/20261104-132053-2e49b9ea/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4": "f47df963/f88334a9", + "BenchS2Arc/20261104-132936-10da7ca8/final/BenchS2Arc-chained_keyframe_to_video_audio.0-0.0.mp4": "6fbb63c2/f6555387", + "BenchS1Static/20261104-171753-8ceecde3/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4": "abd3dabc/d5900069", + "BenchS1Static/20261104-173111-4b68e3c3/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4": "cd167eee/d5900069", + "BenchS1Static/20261104-174427-ff3d1339/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4": "cd167eee/d5900069", + "BenchS1Static/20261104-175558-fade325a/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4": "df2c8907/6b31365e", + "BenchS1Static/20261104-180728-b9fccaaf/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4": "40991534/43283722", + "BenchS1Static/20261104-224226-52a65249/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4": "f47df963/f88334a9", + "BenchS1Static/20261104-225732-347bb719/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4": "f47df963/f88334a9", + "BenchS1Static/20261104-230903-c6d7f38c/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4": "f47df963/f88334a9", + "BenchS1Static/20261104-133844-23b1c5b1/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4": "d59a1259/b2c2c63e", + "BenchS1Static/20261104-144414-d9c61bc5/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4": "d59a1259/b2c2c63e", + "BenchS1Static/20261104-143056-8ceecde3/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4": "d59a1259/b2c2c63e", + "BenchS1Static/20261104-155203-4b68e3c3/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4": "abd3dabc/d5900069", + "BenchS1Static/20261104-160520-ff3d1339/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4": "abd3dabc/d5900069", + "BenchS1Static/20261104-161650-fade325a/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4": "abd3dabc/d5900069", + "BenchS1Static/20261104-162820-b9fccaaf/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4": "86544f72/4c5a2d39", + "BenchS1Static/20261104-164137-52a65249/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4": "86544f72/4c5a2d39", + "BenchS1Static/20261104-165454-347bb719/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4": "86544f72/4c5a2d39", + "BenchS1Static/20261104-170624-c6d7f38c/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4": "86544f72/4c5a2d39", + "BenchS1Static/20261104-233558-8c6ae697/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4": "d59a1259/b2c2c63e", + "BenchS1Static/20261105-004501-92f5fbde/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4": "91deb574/008eae4c" +} diff --git a/bench/records/selflift.json b/bench/records/selflift.json new file mode 100644 index 000000000..8105b69e4 --- /dev/null +++ b/bench/records/selflift.json @@ -0,0 +1,218 @@ +[ + { + "arm": "sl-ctl-s1-6seg-s1166", + "scene": "s1-static", + "seed": 1166, + "job_id": "8b21a85f52e8", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-sl/20261105-221517-76d90e96/final/BenchS1Static-sl-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.7939999999999999, + "seam_jump_max": 0.91, + "seam_look_mean": 1.3, + "seam_look_max": 2.09, + "seam_level_step_db_mean": 8.39, + "seam_level_step_db_max": 17.6, + "texture_fine_seg_last": 0.5803930193261041, + "texture_mid_seg_last": 0.18399718380054902, + "texture_coarse_seg_last": 0.01431419264199045, + "wall_minutes": 15.900850892066956 + } + }, + { + "arm": "sl-k2-s1-6seg-s1166", + "scene": "s1-static", + "seed": 1166, + "job_id": "98dc7b8c3a12", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-sl/20261105-223110-23d7267d/final/BenchS1Static-sl-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.072, + "seam_jump_max": 1.3, + "seam_look_mean": 1.734, + "seam_look_max": 1.97, + "seam_level_step_db_mean": 6.044, + "seam_level_step_db_max": 13.54, + "texture_fine_seg_last": 0.07813257191550643, + "texture_mid_seg_last": 0.10032255760998642, + "texture_coarse_seg_last": 0.26772095847791255, + "wall_minutes": 12.11432607571284 + } + }, + { + "arm": "sl-k1-s1-6seg-s1166", + "scene": "s1-static", + "seed": 1166, + "job_id": "874f0e4ac0e8", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-sl/20261105-224317-782da0c7/final/BenchS1Static-sl-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.812, + "seam_jump_max": 1.04, + "seam_look_mean": 1.754, + "seam_look_max": 2.46, + "seam_level_step_db_mean": 11.698, + "seam_level_step_db_max": 29.0, + "texture_fine_seg_last": 0.7019375492742985, + "texture_mid_seg_last": 0.48992214557186076, + "texture_coarse_seg_last": 0.3455606515178659, + "wall_minutes": 10.762526631355286 + } + }, + { + "arm": "sl-ctl-s1-6seg-s2", + "scene": "s1-static", + "seed": 2, + "job_id": "09e78427dacf", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-sl/20261105-225403-2bf14d93/final/BenchS1Static-sl-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.898, + "seam_jump_max": 1.07, + "seam_look_mean": 1.9380000000000002, + "seam_look_max": 2.95, + "seam_level_step_db_mean": 13.288, + "seam_level_step_db_max": 23.83, + "texture_fine_seg_last": 0.6293028501477302, + "texture_mid_seg_last": 0.16851039608370974, + "texture_coarse_seg_last": 0.009001234730441402, + "wall_minutes": 11.504750871658326 + } + }, + { + "arm": "sl-k2-s1-6seg-s2", + "scene": "s1-static", + "seed": 2, + "job_id": "2a77f4abc476", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-sl/20261105-230044-6fdba789/final/BenchS1Static-sl-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.168, + "seam_jump_max": 1.37, + "seam_look_mean": 2.168, + "seam_look_max": 3.22, + "seam_level_step_db_mean": 7.5200000000000005, + "seam_level_step_db_max": 24.57, + "texture_fine_seg_last": 0.08605618114745024, + "texture_mid_seg_last": 0.17345065676887259, + "texture_coarse_seg_last": 0.12379521510253247, + "wall_minutes": 9.033478784561158 + } + }, + { + "arm": "sl-k1-s1-6seg-s2", + "scene": "s1-static", + "seed": 2, + "job_id": "96e48beb2dd3", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-sl/20261105-230533-8137a30c/final/BenchS1Static-sl-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.7939999999999999, + "seam_jump_max": 1.02, + "seam_look_mean": 1.468, + "seam_look_max": 1.96, + "seam_level_step_db_mean": 5.279999999999999, + "seam_level_step_db_max": 9.6, + "texture_fine_seg_last": 0.7343194218780498, + "texture_mid_seg_last": 0.15818195728637685, + "texture_coarse_seg_last": 0.020192068331222567, + "wall_minutes": 10.837625169754029 + } + }, + { + "arm": "sl-ctl-s2-1seg-s1166", + "scene": "s2-arc", + "seed": 1166, + "job_id": "4b79f1280670", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc-sl/20261105-230946-0757abc7/final/BenchS2Arc-sl-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "texture_fine_seg_last": 1.0123283836297114, + "texture_mid_seg_last": 1.0059118596226497, + "texture_coarse_seg_last": 0.2579960396580523, + "wall_minutes": 1.6857365012168883 + } + }, + { + "arm": "sl-k2-s2-1seg-s1166", + "scene": "s2-arc", + "seed": 1166, + "job_id": "9925b2488c98", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc-sl/20261105-231126-a60cb7fc/final/BenchS2Arc-sl-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "texture_fine_seg_last": 1.1942241917598473, + "texture_mid_seg_last": 0.9490860932181293, + "texture_coarse_seg_last": 0.10997933350534662, + "wall_minutes": 1.310364015897115 + } + }, + { + "arm": "sl-k1-s2-1seg-s1166", + "scene": "s2-arc", + "seed": 1166, + "job_id": "b35afd39ed00", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc-sl/20261105-231245-d22c2e9b/final/BenchS2Arc-sl-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "texture_fine_seg_last": 1.3399643915524273, + "texture_mid_seg_last": 0.8919809908619589, + "texture_coarse_seg_last": 0.1810916517428937, + "wall_minutes": 1.4562986811002097 + } + }, + { + "arm": "sl-ctl-s2-1seg-s2", + "scene": "s2-arc", + "seed": 2, + "job_id": "6cbd7727f378", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc-sl/20261105-231412-782e3a09/final/BenchS2Arc-sl-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "texture_fine_seg_last": 0.5411638987263364, + "texture_mid_seg_last": 0.3605480144505551, + "texture_coarse_seg_last": 0.013441371472672365, + "wall_minutes": 1.578862464427948 + } + }, + { + "arm": "sl-k2-s2-1seg-s2", + "scene": "s2-arc", + "seed": 2, + "job_id": "6edd44546d16", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc-sl/20261105-231547-2220e7c5/final/BenchS2Arc-sl-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "texture_fine_seg_last": 2.222332306802891, + "texture_mid_seg_last": 1.696994783655018, + "texture_coarse_seg_last": 0.7599950567380114, + "wall_minutes": 1.2773949742317199 + } + }, + { + "arm": "sl-k1-s2-1seg-s2", + "scene": "s2-arc", + "seed": 2, + "job_id": "41d6746a43dd", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc-sl/20261105-231625-0b510433/final/BenchS2Arc-sl-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "texture_fine_seg_last": 0.8557232371560083, + "texture_mid_seg_last": 0.745666365227127, + "texture_coarse_seg_last": 0.11556987899411937, + "wall_minutes": 1.576944617430369 + } + } +] diff --git a/bench/records/speed.md b/bench/records/speed.md new file mode 100644 index 000000000..78a73df3c --- /dev/null +++ b/bench/records/speed.md @@ -0,0 +1,181 @@ +# Speed levers (Task 19) + +Run 2026-11-05 on the bench server (lem, two RTX 3090s), deployed at +`a2034533`, then `6a89744a` (bench JSON only). Arms: +`bench/arms/selflift.json` and `bench/arms/tst.json`. Records: +`bench/records/selflift.json`, `tst.json`, and the moments pass +`t17-t19-moments.json` (PSNR/SSIM, raw texture, motion and audio, taken +in-process on lem). + +**Short of Step 2's protocol.** Every arm ran 2 seeds, not 3. SelfLift ran at +1024x576 (544 does not halve to a 64 px grid), so the S1 6-segment floor +(measured at 960x544) is the nearest floor there is, not an exact one. + +PSNR bands (workflow guide): about 44 dB is rerun noise, 23-27 dB is changed +numerics with the same shot, and about 19 dB is a re-staged shot. On the +moving S2 arc, even sparse against dense attention, a change that is +lossless by eye, reads 9-13 dB. So PSNR is read on S1 only. + +## SelfLift (#1195) + +### Verdict + +- **Not lossless, and lift step 2 is broken.** + - `lift_step 2` paints brush-stroke artifacts on every take, both scenes and + both seeds. There are smeared stripes, ragged edges on the circle and the + square, white feather streaks over the S2 trees, and streaks across the + face and sweater from the first second. Its "texture win" on S1 is that + damage. Frame-to-frame motion on the static shot doubles (0.47 to 1.20, + and 0.64 to 1.15). + - `lift_step 1` looks clean, but it re-stages the shot (a different head + pose). S1 PSNR is 18.0 and 21.7 dB at frame 62, below the 23-27 dB band. + Fine texture drift is worse past the floor on both seeds (0.58 to 0.70, + and 0.63 to 0.73; floor 0.081). +- **The saving is below the 15-25% expected.** + - Lift step 1 saves 11% of chain denoise and about 6% of the job's wall. + - Lift step 2 saves 22% of denoise and about 14% of wall, but it is unusable. + - On a single segment the saving is larger: lift step 1 saves 13-19% of + denoise, and lift step 2 saves 33%. A chain's later segments carry + full-canvas guide rows, so they gain less than segment 1. +- **As a dial step:** lift step 1 buys about 6% of wall for a re-staged shot + and +0.1 fine drift. That is too small to be worth a step. The draft tier has + bigger levers (steps, canvas). Recommendation: close #1195 with these + numbers. + +### Wall + +Denoise per segment, in s, read from the job event log (`denoise.denoise` to +`decoding`). Steady state is segments 3-6, where no autotune stall remains. + +| arm | card | seg 1 | seg 2 | seg 3-6 (each) | denoise total | job wall | +|---|---|---|---|---|---|---| +| ctl, seed 2 | cuda:0 | 53.5 | 69.8 | 69.7 | 402.8 | 11.50 min | +| lift 1, seed 2 | cuda:0 | 50.3 | 62.3 | 62.0 (**-11%**) | 360.8 | 10.84 min (**-6%**) | +| lift 2, seed 2 | cuda:1 | 39.4 | 50.7 | 50.3 | 291.1 | 9.03 min (cuda:1) | +| ctl, seed 1166 | cuda:0 | 176.9 | 211.0 | 69.7 | 666.5 | 15.90 min (about 11.5 without its stall) | +| lift 1, seed 1166 | cuda:0 | 50.3 | 61.8 | 61.9 | 359.8 | 10.76 min | +| lift 2, seed 1166 | cuda:0 | 105.0 | 117.0 | 54.5 (**-22%**) | 440.0 | 12.11 min (about 9.9 without its stall, **-14%**) | + +- The first S1 job at the new canvas, and the first lift-2 job, each paid a + first-call autotune stall of about 130-260 s in segments 1-2. Half-canvas + shapes autotune too. +- On a single S2 segment, denoise was 63.1 and 58.4 s for the control, 50.9 + and 50.6 s for lift 1, and 41.7 and 39.4 s for lift 2. Job wall was + 1.69/1.58, 1.46/1.58 and 1.31/1.28 min. + +### Quality + +Verdicts are on the picture, primary `texture_fine_seg_last`, floor S1 guide +(the level step is not counted on S1/S2). + +| arm | job | control | PSNR @62/277/583 | jump | look | tex fine | tex mid | tex coarse | motion | verdict | +|---|---|---|---|---|---|---|---|---|---|---| +| S1 lift 1 @1166 | `874f0e4ac0e8` | `8b21a85f52e8` | 18.0 / 22.2 / 13.8 | 0.794→0.812 | 1.300→_1.754_ | 0.580→_0.702_ | 0.184→_0.490_ | 0.014→_0.346_ | 0.47→0.58 | loses | +| S1 lift 1 @2 | `96e48beb2dd3` | `09e78427dacf` | 21.7 / 17.4 / 18.4 | 0.898→0.794 | 1.938→**1.468** | 0.629→_0.734_ | 0.169→0.158 | 0.009→_0.020_ | 0.64→0.55 | loses | +| S1 lift 2 @1166 | `98dc7b8c3a12` | `8b21a85f52e8` | 16.7 / 16.8 / 14.9 | 0.794→_1.072_ | 1.300→1.734 | (0.580→0.078: artifacts) | 0.184→0.100 | 0.014→_0.268_ | 0.47→_1.20_ | broken by eye | +| S1 lift 2 @2 | `2a77f4abc476` | `09e78427dacf` | 17.3 / 14.7 / 14.2 | 0.898→_1.168_ | 1.938→2.168 | (0.629→0.086: artifacts) | 0.169→0.173 | 0.009→_0.124_ | 0.64→_1.15_ | broken by eye | +| S2 1 seg lift 1 @1166 / @2 | `b35afd39ed00` / `41d6746a43dd` | `4b79f1280670` / `6cbd7727f378` | (S2: not read) | | | inside floor | inside floor | inside floor | | tie | +| S2 1 seg lift 2 @1166 / @2 | `9925b2488c98` / `6edd44546d16` | `4b79f1280670` / `6cbd7727f378` | (S2: not read) | | | 1.012→1.194 / 0.541→_2.222_ | | | | broken by eye | + +- **Audio.** S1/S2 tracks are room tone and ambience at -50 to -87 dB, so no + timbre read is possible on these scenes. Nothing goes to `for-don.md`. +- **Segment-1 labels** do not apply. The canvas has no canonical entry, and + the lift changes segment 1 by design. + +## TST (#1196) + +### Verdict + +- **No quality effect, measured or visible, and it costs dense attention.** + - tau 0.2 per head against dense attention: S1 PSNR is 31-35 dB, above the + same-shot band and nearly a rerun. Every texture measure is inside the + floor on both scenes and seeds. + - By eye, S1 is indistinguishable from dense. S2 differs only in the clouds + and far trees, as any numerics change on a moving shot does. +- **Cost.** + - TST adds 5-6% denoise over dense. + - Dense attention adds 22-28% over the sparse default. + - So TST costs about +32% denoise over today's default, for nothing seen. +- **tau 0 is a no-op.** Its output is bit-identical to dense (same frames, + same segment-1 fingerprint `e1959ec1/d4b8f719`), and it still costs 3% (the + processor's overhead). +- `per_head: false` and other tau values were not in the arms. +- Recommendation: close #1196 with these numbers. Reopen only if a tau sweep is + wanted, and that is a Don call: the spike found no gain at tau 0.2. + +### Table + +Denoise in s for one segment (the event log). Seed 1166 ran on cuda:1 and +seed 2 on cuda:0, so compare within a seed. + +| scene, seed | sparse | dense | tst tau 0.2 | tau 0 | tst vs dense | dense vs sparse | PSNR tst vs dense @20/62/110 | PSNR dense vs sparse | +|---|---|---|---|---|---|---|---|---| +| S1 @1166 | 53.0 | 67.8 | 70.9 | 69.8 | +4.6% | +28% | 32.0 / 34.8 / 34.7 | 28.3 / 28.4 / 28.6 | +| S1 @2 | 49.4 | 60.5 | 64.1 | | +6.0% | +22% | 31.2 / 33.6 / 31.5 | 24.0 / 20.8 / 24.0 | +| S2 @1166 | 53.1 | 66.8 | 70.6 | | +5.7% | +26% | 23.3 / 19.1 / 18.5 | 13.2 / 11.5 / 11.9 | +| S2 @2 | 49.2 | 60.4 | 63.9 | | +5.8% | +23% | 23.4 / 20.6 / 18.4 | 11.4 / 9.6 / 9.3 | + +Jobs (sparse, dense, tst, in that order): + +- S1 @1166: `a32149365b7c`, `db14f86eb1bd`, `6dba38159cc7`; tau 0 is + `20db977af359`. +- S1 @2: `dd38d2050097`, `1a172450268f`, `87b8dd26a67a`. +- S2 @1166: `7c3a90fc4c90`, `d0e2aad67720`, `80ed710df33f`. +- S2 @2: `6c644ea1d6c2`, `19e189043889`, `f7a673dae443`. + +The two dense S1 jobs include a cold load of 121-135 s, which is why their job +wall reads 4.1-4.3 min. + +Texture against dense is inside the floor on all four pairs: S1 fine +0.008→0.013 and 0.016→0.006; S2 fine 0.519→0.473 and 0.408→0.537. Motion is +unchanged (0.42→0.42, 0.50→0.51, 3.09→3.04, 4.11→4.23). + +## Host-RAM gate (deployed `c441f92e..6495878e`) + +### Verdict + +**Confirmed: two H3 jobs ran at once, one per card, with no failure.** Each +H3 job commits 42-51 GiB. Two at once commit about 91 GiB of lem's 125.7 GiB, +and the sampled free host memory never fell below 31 GiB. + +### Concurrent windows (UTC, 2026-11-05, from the job records) + +| job | arm | card | start | finish | committed peak | +|---|---|---|---|---|---| +| `05a3a55b043c` | ks-pixel, 120 s | cuda:0 | 20:44:54 | 21:53:36 | 45.6 GiB | +| `4766787e793d` | ks-lc, 120 s | cuda:1 | 20:45:22 | 21:50:47 | 45.5 GiB | +| `d93adb33249e` | lc S2 @1166 | cuda:0 | 20:32:07 | 20:44:54 | 43.4 GiB | +| `d901c8c82784` | lc S2 @2 | cuda:1 | 20:32:07 | 20:45:22 | 43.3 GiB | +| `2c1fd3ab7869` | anchor 0.25 rerun, 120 s | cuda:1 | 21:50:47 | 23:00:43 | 46.0 GiB | +| `536548e14dd5`, `1b8edfdba405`, then SelfLift | S2 controls | cuda:0 | 21:53:36 | ... | 42.8, 41.7 GiB | + +- The two 120 s takes overlapped for 65 min. From 20:32 to 00:51 both cards + were busy, with 40 jobs on two cards. Every job succeeded. +- **Committed figure:** `host_memory_job_peak_committed_mb` in the bench + server's job history (`~/.diffusers_helper/jobs.sqlite`, read-only). Other + H3 jobs: `20ce09d60040` 50.5 GiB, `40bbb40c14ec` 50.7 GiB, `fc0f91663277` + 47.0 GiB, `bf9ee32e040f` 47.3 GiB. +- The old RSS basis put the same jobs at 49-120 GiB (`fc0f91663277`: 120.3 + GiB job-peak RSS). That is the mmapped checkpoint page cache, and it is what + serialised the queue before. +- The lowest `host_memory_available_mb` in the memory events during the + overlap is 31.2-31.9 GiB. Those events are sampled at phase changes, so the + true peaks may be a little higher. +- **Card speed, not contention:** cuda:1 runs about 10% slower per pass than + cuda:0 on every job, alone or paired (17.4 s against 15.7 s for a 120 s + pass). Round 2's `8d1614ced27a`, on cuda:1 a day earlier, reads 17.4 s too. + So the overlap costs nothing measurable. Compare wall times within one card. + +## Promote (Step 3) and the verdicts (Step 4) + +**Nothing new is promoted.** This round benched only the two spikes, and both lost: +SelfLift is lossy (close #1195) and TST has no effect (close #1196). The other +Step 1 levers were not re-benched on the long-form scenes. They keep the verdicts +they already shipped with: sparse top-k 0.15 attention and the 4-step schedule are +the H3 defaults (#931/#932, H3 speed A/B), `sage_hub`, the `first_block` and +MagCache caches and the quantized-weight cache stay knobs, and the LTX levers are +as its skill lists them. The tiers (Task 20) bundle these as dial steps. + +**#980 (FastH3 Omni):** out of long-form scope. It is a 68 GB Ref2VA transformer +finetune, and the long-form chains are t2va/fl2va, so it cannot speed a chain. It +stays parked as its own onboarding. diff --git a/bench/records/structural.md b/bench/records/structural.md new file mode 100644 index 000000000..3bf1cb72c --- /dev/null +++ b/bench/records/structural.md @@ -0,0 +1,872 @@ +# Structural round (Task 17) + +## Step 4: windowed refine + +Task 17 Step 4 asks one question. Does a low-sigma refine of a finished chain, +run in overlapping time windows and fused at every step, smooth the seams and +drift? If it does, is it the seam-smoothing pass for every chain? + +### Setup + +- **Workflow.** `bench/workflows/refine-windows.json` has four steps: + - `encode_h3_latents` reads the take one 17-frame clip at a time. + - An H3 t2va `refine_strength` pass runs with `experimental_refine_windows` + and `hold_audio` set to the take's own mp4. + - `decode_h3_latents` trims the take back to its own length. + - `pair_audio` puts the take's own track back. +- **Settings.** The refine runs 5 sigma points (4 evaluations) at shift 6, with + the fl2v turbo 4-step LoRA, int4 SDNQ and sparse attention: the settings the + chains were made with. The encode and decode steps set `release_models` + (see *Failures* below). +- **Windows.** `h3_windows` has no default window, so the windows are the + spike's protocol value: `{frames: 40, overlap: 12}` latents (about 6.6 s per + window, 2 s overlap), pyramid blend, `positions: "true"`. A 26 s take is 187 + latents in 7 windows. One `positions: "rebased"` arm used `{frames: 42, + overlap: 12}`. Rebasing needs a stride on the 5-latent clip grid, and + `frames ≡ 2 (mod 5)` puts the pulled-back last window on it for any 5m + 2 + take. +- **Sources.** All sources are clean by `bench/segment1.py`, seed 1166, 6 + segments, 634 frames (26.4 s): + + | scene | source | job | + |---|---|---| + | S1 static | guide + anchor 0.25 (the static default) | `0a40c41af97c` | + | S2 arc | guide control (no counters, per *Defaults by shot type*) | `dedd341c9108` | + | S3 dialogue walk | guide control (no counters) | `c0b92d0d0cd4` | + +- **Measures** (`bench/refine_pairs.py`, in-process on lem). A refined take is + decoded fresh, so its run records no shots. The script reads both files and + measures the refined take at the source's own seams: + - `analyze_seams` jump, look and level step; + - texture drift on the scene crop against frame 62, at frames 62, 150, 277, + 400, 583 and 630 ("seg_last" is 630); + - thumbnail frame-delta motion; + - PSNR and SSIM of the refined take against its source. +- **Records.** + - Driver records: `t17-refine-r030.json` and `t17-refine-r020-r040.json`. + - Pair measures: `t17-pairs-r030.json` and `t17-pairs-r020-r040.json`. +- **Floor.** `combined_floor(seed floor, rerun gap)` uses each scene's guide + row from `noise-floor.md`. The refine runs on a fixed source, so this floor + is conservative. + +### Results + +Each cell gives the source value, then the refined value. **Bold** is better +past the floor. Nothing was worse past the floor on any arm. + +| arm | verdict | jump | jump max | look | look max | tex fine | tex mid | motion | PSNR @630 | +|---|---|---|---|---|---|---|---|---|---| +| S1 0.2 | tex tie / jump **wins** / look tie | 0.928→**0.566** | 1.15→0.72 | 1.128→1.114 | 1.98→1.41 | 0.346→0.351 | 0.116→0.107 | −17% | 29.6 | +| S1 0.3 | tex tie / jump **wins** / look tie | 0.928→**0.586** | 1.15→0.87 | 1.128→1.238 | 1.98→1.64 | 0.346→0.366 | 0.116→0.115 | −18% | 28.2 | +| S1 0.3 rebased | tex tie / jump **wins** / look tie | 0.928→**0.516** | 1.15→**0.71** | 1.128→1.014 | 1.98→1.21 | 0.346→0.381 | 0.116→0.115 | −19% | 28.2 | +| S1 0.4 | tex tie / jump **wins** / look tie | 0.928→**0.602** | 1.15→1.00 | 1.128→1.372 | 1.98→1.95 | 0.346→0.368 | 0.116→0.127 | −17% | 26.8 | +| S2 0.2 | look **wins** / jump **wins** | 1.362→**1.128** | 2.09→1.67 | 1.286→**0.808** | 1.74→1.19 | 4.43→4.14 | 4.40→4.03 | +1% | 30.1 | +| S2 0.3 | look **wins** / jump **wins** | 1.362→**1.114** | 2.09→**1.64** | 1.286→**0.766** | 1.74→1.14 | 4.43→4.26 | 4.40→4.19 | +3% | 27.8 | +| S2 0.4 | look **wins** / jump **wins** | 1.362→**1.102** | 2.09→**1.65** | 1.286→**0.720** | 1.74→1.12 | 4.43→4.10 | 4.40→4.02 | +3% | 26.7 | +| S3 0.2 | jump **wins** / look tie / tex tie | 1.524→**1.192** | 1.83→1.45 | 1.470→1.242 | 1.71→1.57 | 0.031→0.145 | 0.511→0.490 | 0% | 29.6 | +| S3 0.3 | jump **wins** / look tie / tex tie | 1.524→**1.100** | 1.83→**1.36** | 1.470→1.324 | 1.71→1.62 | 0.031→0.129 | 0.511→0.495 | +2% | 28.0 | +| S3 0.4 | jump **wins** / look tie / tex tie | 1.524→**1.000** | 1.83→**1.20** | 1.470→1.342 | 1.71→1.70 | 0.031→0.109 | 0.511→0.491 | +4% | 26.8 | + +- **Level step.** The level step does not change (S1 12.2→12.6 dB, S2 and S3 + identical). The track is held, and the change of about 0.4 dB is the + re-encode. +- **Drift against segment 1.** Every texture series is inside the floor at + every moment. On S1 the fine drift is 0.367→0.352 at 583 and 0.346→0.366 at + 630, and the face crop is likewise unchanged. By eye at frame 583, the 0.3 + refine keeps the source's crusted stripes, circle and wall. **The refine + smooths the seams. It does not undo the burn.** +- **Motion.** + - S2 and S3 keep their motion (0 to +4%). The S2 arc completes, with the + same framing at 62, 300 and 583. + - S1's frame-to-frame delta falls by 17-19% at every strength. S1 is locked + off, so this delta is breathing, blinks and the burn's crawl. That the drop + does not depend on strength suggests the refine removes shimmer rather + than motion. This has not been confirmed by eye in motion: it is in the + S1 export. +- **Positions.** `rebased` reads a little better than `true` on S1 (jump max + and look past or near the floor). Both are within noise of each other. For + a take past 15 s, `rebased` keeps every window's media inside H3's trained + span, so it is the setting for the long run. +- **Strength.** 0.2-0.4 all give the same seam verdicts. The seam gains grow + slightly with strength (S3 jump 1.19→1.10→1.00), and fidelity falls + (PSNR ~29.6→28.0→26.8). On S1, 0.4 starts to give the look back (1.37). **0.3 + is the pick**, with 0.2 as the conservative setting. + +### Cost and VRAM (lem, RTX 3090, cuda:0) + +- **Wall time.** A 26.4 s take takes 17-20 min: + - load: 110-115 s; + - encode: about 14 s; + - denoise: 330 s, which is 4 passes of 83 s over 7 windows, plus a first-pass + sparse/autotune stall of up to 115 s on the first job after a load; + - decode: 2 × about 97 s, the step's own decode and `decode_h3_latents`. + That is **about 40 min per minute of take** at this length. +- **Per window.** The denoise is about 2.9 s per window per pass. +- **At 120 s, projected.** 852 latents is 28 windows (rebased 42/12). That + projects to about 22 min of denoise, about 15 min of decode and the 2 min + load: **about 20 min per minute of take** once the fixed costs spread. + *(Superseded: measured at 120 s it is 71.5 min for 2 min of take, **about 36 + min per minute**, half of it decode; see* 120 s *below.)* +- **Peak VRAM.** nvidia-smi, sampled at 1 s, reads 22.9-23.5 GB on a 24 GB + card for every arm. Most of that is the allocator's cache around the two + float32 decodes. The windowed denoise itself is small (one 40-latent window + at a time). + +### 120 s (S1, `r3-anchor25-s1-120s`) + +Run 2026-11-05 on the bench server at `155ae56c` (streamed decode deployed): +`t17-s1-120s-r030-rebased` (`2f6c0073d2f0`, cuda:0), 0.3, `{42, 12}` +rebased, over `r3-anchor25-s1-120s` (`25f6bd1da507`, clean). Arms +`bench/arms/t17-refine-120s.json`; records `t17-refine-120s.json` and +`t17-pairs-120s.json` (`refine_pairs.py` at the round-3 moments 62, 300, +583, 1000, 1501, 2000, 2500, 2868). The refined take is a fresh decode, so it +has no segment-1 label of its own; its source is clean. Floor: the S1 guide +row, as above (no two-minute floor exists). + +| measure | source | refined | verdict | +|---|---|---|---| +| seam jump mean / max | 0.943 / 1.86 | **0.734** / 1.93 | mean **wins** (0.21 > 0.124); max tie | +| seam look mean / max | 1.032 / 2.70 | 1.156 / 2.96 | tie (inside 0.438 / 0.790) | +| level dB mean / max | 5.83 / 24.3 | 5.96 / 24.9 | tie (held track, re-encode) | +| tex fine / mid / coarse @2868 | 1.108 / 0.430 / 0.019 | 1.093 / 0.406 / 0.004 | tie | +| face fine @2868 | 0.040 | 0.086 | tie | +| motion | 0.636 | 0.528 | -17% | +| PSNR vs source | | 31.1 @62 -> 26.0 @1000 -> 23.5-24.2 from 2000 | | + +- **Does it move the burn from about 42 s on? No.** The fine drift series + against frame 62 runs 0.497 / 0.771 / 0.931 / 1.012 / 1.108 (source) and + 0.539 / 0.838 / 0.949 / 1.003 / 1.093 (refined) at 42 / 62 / 83 / 104 / + 119.5 s: inside the floor at every moment, mid likewise. By eye (stills at + 65, 70, 75 and 110 s, and the export) the two sides are the same picture: + the same mosaic crust on the stripes and circle, the same pale, flattened + face and white forehead patch at 110 s. The PSNR falls from 31 to about + 24 dB as the burn grows, so the refine re-renders the burned crust more + freely, but it re-renders it as crust. +- **Do the seams win as at 26 s? Partly.** Jump mean wins past the floor + over the whole take (0.94 -> 0.73), and by stretch it holds through 95 s + (0-25 s 0.93 -> 0.53, 25-60 s 0.88 -> 0.55, 60-95 s 0.80 -> 0.56, max + 0.96 -> 0.75 there). In the last 25 s, where the source is most burned, it + no longer helps: jump 1.26 -> 1.41 and look 1.81 -> 2.10 mean, 2.70 -> 2.96 + max, both inside the floor. Look ties overall, where at 26 s it tied too + (S1). Nothing is worse past the floor. +- **Motion** falls 17%, as at 26 s (-17 to -19%), so the drop does not grow + with length: it is the same shimmer removal, still owed an eye check. +- **Cost.** 71.5 min for 2 min of take, **about 36 min per minute**, not + the 20 projected: encode 10.7 min; refine 33.3 min (load 2.0, encode 0.7, + denoise 23.3 = 4 passes of 5.8 min over 28 windows, as projected, and its + own decode 7.3); `decode_h3_latents` 27.1 min; mux 0.3. The two decodes + are 34 min, half the job: dropping the step's redundant internal decode + saves 7, and the float32 streamed decode is the next cost to look at. +- **Peak VRAM** (nvidia-smi at 1 s, cuda:0, `~/t17-120s-vram.csv` on lem): + **16,519 MiB (16.1 GiB)**, during the refine's load; denoise 12.1 GB, + decode 14.5 GB. The streamed decode takes the 120 s job well under the + 22.9-23.5 GB the 26 s jobs read with the whole-take decode. +- **Export** for Don: `~/Desktop/dw-bench-t17/t17-s1-120s-anchor25_LEFT-source_RIGHT-refine030-rebased-60-80s.mp4` + (60-80 s, the burn region, x264 CRF 16, refined track). + +**Verdict at 120 s:** the windowed refine is a seam pass at length too (jump +wins to 95 s), it does not touch the burn, and it costs about 36 min per +minute of take, half of it decode. + +### Failures on the way, and their causes + +- The first 4 arms failed with "Adapter name(s) {'turbo'} not in the list of + present adapters". This hides the real cause. The worker log shows a **CUDA + OOM while loading the transformer or VAE**, which diffusers' + `load_components` swallows: it logs the OOM and goes on without the + component. The LoRA then found no transformer. +- The OOM came from the `encode_h3_latents` float32 VAE. The task model cache + kept it on cuda:0 into the refine's load. +- `release_models: true` on the encode and decode steps fixed it. +- An engine follow-up is worth filing: fail the load when `load_components` + dropped a component the step needs, so the error names the OOM. + +### Verdict + +| scene | strength | wins? | +|---|---|---| +| S1 static | 0.2 / 0.3 / 0.4 | **Seams yes** (jump past the floor), drift no (tie), nothing worse. The burn stays. | +| S2 arc | 0.2 / 0.3 / 0.4 | **Yes**: look and jump both win past the floor, motion kept, nothing worse. | +| S3 dialogue walk | 0.2 / 0.3 / 0.4 | **Seams yes** (jump past the floor), look and texture tie, motion and the held voice kept. | + +**It is the seam-smoothing pass for every chain shot type measured:** + +- the default is `refine_strength` 0.3, with `{frames: 40-42, overlap: 12}` + windows and `rebased` positions past 15 s; +- it never lost a measure, and it costs about 40 min per minute of take at + 26 s and about 36 min per minute at 120 s (half of it decode); +- **it is not a burn fix**: drift against segment 1 is unchanged on every + scene and strength. + +Promoting it out of the experimental flag needs four things, as Step 4's "if +it wins" says: + +- the 120 s confirmation (done: seams yes to 95 s, burn untouched; see *120 s*); +- an eye check of S1's motion drop; +- dropping the step's redundant internal decode when `videos` is not in the + output; +- the TDD feature stages. + +### Exports for Don + +The exports are in `~/Desktop/dw-bench-t17/` on the Mac: 20 s side by side, +4-24 s, x264 CRF 16, source on the left and refined on the right, with the +refined (held) track. + +- `t17-s2-guide_LEFT-source_RIGHT-refine030-20s.mp4`: S2 at 0.3. +- `t17-s1-6seg-anchor25_LEFT-source_RIGHT-refine030-rebased-20s.mp4`: S1 at + 6 segments, 0.3 rebased. +- `t17-s1-120s-anchor25_LEFT-source_RIGHT-refine030-rebased-60-80s.mp4`: S1 + at 120 s, 0.3 rebased, 60-80 s (the burn region). + +## Steps 2-3: latent carry (#1150, #1044, #1030) + +### Verdict + +- **Latent carry is not a default, and not a burn fix.** On the static scene it + lowers the backdrop crust, but the take still burns. On the moving scene it + makes the seams worse on both seeds. +- **The softness is the anchor's, not the carry's.** Without the anchor, the + latent-carry take is crusty and harsh, like the pixel control. It is not soft. + Don's "soft and masky" on `fc0f91663277` comes from anchor 0.25. The pixel + anchor-0.25 take is just as soft and masky at the same moments. +- **The carry alone (keystone pair, R39) ties on every picture measure.** By + eye its face holds better at 104-120 s. That needs Don's eye (export below). +- **The carry drops the late music bed on seed 1166, and only there.** It keeps + S2's garden ambience on both seeds. The bed is a property of the seed-1166 + pixel chain, not of the carry. Seed 2's pixel take never grows one. +- **New finding: keystone + anchor 0.25 resets the burn at two minutes.** Both + keystone takes crust from about 40 s. Then the backdrop snaps back clean: + pixel between 79 and 81 s, latent carry by about 104 s. Fine drift at 120 s + is 0.03-0.06, against anchor 0.25's 1.11. This is one seed. Round 2's + keystone verdict ("does not hold") was read at 24 s, before any reset. +- **Cost.** The carry skips the pixel re-encode. A two-minute take encodes in + 4.7-7.7 min less (465 s against 757 s, 281 s against 776 s with the + keystone), so it is about 5% faster overall (65.8 against 69.1 min). + +### What the A/B removes + +As R39 says, the keystone pair is #1044's latent-carry arm, not #1150's +VRAM-saving arm 2. Against the pixel guide it removes the VAE encode of the +guide frames, the posterior sampling, and the fp16 rounding of the round trip. +On fl2va every later segment's keyframe image is still a pixel round trip +(`carry.frames[0]`). The keystone pair pins that keyframe to the same fixed +pixels in both arms, so the carry is the only difference. + +### Floors + +These are the first two-minute floors. Both pairs are anchor 0.25, guide, S1, +120 s: + +- seed floor: `25f6bd1da507` (seed 1166) against `bf9ee32e040f` (seed 2), both + clean; +- rerun gap: `25f6bd1da507` against its same-seed rerun `2c1fd3ab7869`. + +| floor | jump | jump max | look | look max | tex fine | tex mid | tex coarse | face fine | +|---|---|---|---|---|---|---|---|---| +| 120 s seed floor | 0.044 | 0.430 | 0.263 | 0.040 | 0.034 | 0.159 | 0.002 | 0.427 | +| 120 s rerun gap | 0.044 | 0.350 | 0.162 | 0.640 | 0.075 | 0.066 | 0.118 | 0.638 | +| S1 guide (6 seg) | 0.124 | 0.430 | 0.438 | 0.790 | 0.081 | 0.032 | 0.010 | | +| **combined (used)** | **0.124** | **0.430** | **0.438** | **0.790** | **0.081** | **0.159** | **0.118** | **0.638** | + +- The 120 s floors fit inside the 6-segment row for seams and fine texture. +- Mid and coarse texture need the wider 120 s values. +- The face crop is too noisy to read at two minutes. The rerun alone moves it by + 0.64 (`2c1fd3ab7869`'s face blurs out, while `25f6bd1da507`'s does not), so + the face is read by eye. +- S2 uses the S2 guide row (fine 0.605, mid 1.21, coarse 1.66), widened by + the r36 reruns (`536548e14dd5` against `dedd341c9108`, which is bit-identical + in motion, and `1b8edfdba405` against `01146e925c94`). +- The level step is shown but not counted on S1/S2 (Don's ruling). + +### Segment-1 labels (`bench/segment1.py`) + +Clean: `fc0f91663277`, `05a3a55b043c`, `4766787e793d`, `20ce09d60040`, +`bf9ee32e040f`, `d93adb33249e`, `d901c8c82784`, `536548e14dd5`, `1b8edfdba405`. +Drifted: + +- `40bbb40c14ec` (lc anchor 0.25, seed 2): `5e870262/8698ff4d`. Its read is + confounded. +- `2c1fd3ab7869` (the anchor-0.25 rerun, cuda:1): `c65cc0b6/b2c2c63e`, video + half only. It is kept in the rerun gap, as R35 does. + +### S1, 120 s (static) + +Measures at the end of the take, and seams over all 28. **Bold** is better +than its control past the floor; _italic_ is worse. + +| arm | job | control | jump | jump max | look | look max | tex fine | tex mid | tex coarse | face fine | level dB (not counted) | verdict (picture) | +|---|---|---|---|---|---|---|---|---|---|---|---|---| +| guide control | `8d1614ced27a` | | 1.341 | 6.56 | 2.233 | 11.51 | 1.706 | 1.171 | 0.088 | 0.711 | 5.9 | | +| lc, no anchor | `20ce09d60040` | `8d1614ced27a` | **0.932** | **1.49** | **1.016** | **2.49** | **1.105** | **0.740** | 0.147 | 0.746 | 12.1 | **wins** | +| anchor 0.25 | `25f6bd1da507` | | 0.943 | 1.86 | 1.032 | 2.70 | 1.108 | 0.430 | 0.019 | 0.040 | 5.8 | | +| lc + anchor 0.25 | `fc0f91663277` | `25f6bd1da507` | 0.842 | **1.17** | 1.175 | 2.13 | **0.643** | 0.334 | _0.216_ | 0.214 | 16.0 | mixed | +| anchor 0.25, seed 2 | `bf9ee32e040f` | | 0.987 | 1.43 | 1.295 | 2.66 | 1.142 | 0.590 | 0.017 | 0.467 | 10.5 | | +| lc + anchor 0.25, seed 2 (drifted) | `40bbb40c14ec` | `bf9ee32e040f` | **0.831** | 1.03 | 1.021 | **1.82** | **0.453** | **0.216** | _0.224_ | 0.048 | 9.8 | mixed (confounded) | +| keystone + anchor 0.25, pixel | `05a3a55b043c` | `25f6bd1da507` | 0.922 | 1.59 | 1.134 | 2.50 | **0.029** | **0.016** | 0.016 | 0.226 | 9.7 | **wins** | +| keystone + anchor 0.25, lc | `4766787e793d` | `05a3a55b043c` | 0.813 | **1.13** | 1.314 | 2.08 | 0.059 | 0.009 | 0.028 | 0.106 | 14.0 | tie | + +Fine texture drift against frame 62, over the take: + +| s | 12.5 | 24 | 42 | 62 | 83 | 104 | 119.5 | +|---|---|---|---|---|---|---|---| +| guide control | 0.180 | 0.536 | 1.108 | 1.399 | 1.646 | 1.658 | 1.706 | +| lc, no anchor | 0.132 | 0.225 | 0.547 | 0.997 | 1.207 | 0.930 | 1.105 | +| anchor 0.25 | 0.153 | 0.367 | 0.497 | 0.771 | 0.931 | 1.012 | 1.108 | +| lc + anchor 0.25 | 0.114 | 0.107 | 0.151 | 0.330 | 0.442 | 0.572 | 0.643 | +| keystone + anchor 0.25, pixel | 0.159 | 0.283 | 0.425 | 0.458 | 0.157 | 0.089 | 0.029 | +| keystone + anchor 0.25, lc | 0.085 | 0.118 | 0.065 | 0.044 | 0.076 | 0.053 | 0.059 | + +The keystone-lc row is low throughout on the circle crop, yet the full frame +crusts from 42 to about 100 s. Read that row with the stills, not alone. + +### By eye + +Face crops and full frames were taken at 24, 42, 62, 83, 104 and 119.5 s, with +extra frames at 52-100 s for the keystone pair. + +- **lc, no anchor (`20ce09d60040`).** It burns like the control: a painted, + cracked face from 62 s, a high-contrast posterised face with white patches + from 104 s, and a crusted backdrop. It is never soft. Its backdrop mosaic at + 120 s is finer than the control's shards, which matches the texture win. +- **lc + anchor 0.25 against anchor 0.25 (seed 1166).** Both are soft. The + carry's backdrop is visibly less crusted at 42 s. From 83 s both faces are + pale and washed out. At 104-120 s the carry's face is ghosted (a doubled, + smeared outline), and the pixel take's is a white mask. Neither is usable. + The carry adds no softness of its own: the pixel take is as soft. +- **Seed 2 (confounded).** The carry's face is present to 83 s and smeared + after. The pixel control is a white mask with dark eye holes from 83 s. +- **Keystone pair.** + - Both crust from about 42 s. The pixel take's crust is coarser. + - Pixel: the backdrop is clean again from 81 s (it is still crusted at 79 s). + No seam scores high there (look 0.93 at 81.7 s). From 104 s the face has + streaks and the hair has turned dark and glossy. + - Latent carry: crusted, though milder, until about 100 s, then clean by + 104 s. Its face at 104-120 s is the clearest of any two-minute take. The + eyes, nose and mouth are sharp, and the face is a little plastic. The hair + darkens in this take too. + - Identity drifts in both: darker hair, a slightly different face from + segment 1. + +### Audio + +Mean level per 20 s window (dBFS, from 5 s RMS windows): + +| take | job | 0-20 | 20-40 | 40-60 | 60-80 | 80-100 | 100-120 | +|---|---|---|---|---|---|---|---| +| guide control | `8d1614ced27a` | -64 | -63 | -32 | -20 | -11 | -14 | +| lc, no anchor | `20ce09d60040` | -53 | -52 | -60 | -63 | -67 | -81 | +| anchor 0.25 | `25f6bd1da507` | -61 | -58 | -44 | -42 | -45 | -49 | +| anchor 0.25 rerun | `2c1fd3ab7869` | -63 | -52 | -47 | -45 | -36 | -20 | +| lc + anchor 0.25 | `fc0f91663277` | -52 | -63 | -55 | -68 | -64 | -67 | +| anchor 0.25, seed 2 | `bf9ee32e040f` | -63 | -62 | -58 | -72 | -62 | -74 | +| lc + anchor 0.25, seed 2 | `40bbb40c14ec` | -59 | -54 | -44 | -55 | -63 | -55 | +| keystone, pixel | `05a3a55b043c` | -60 | -62 | -64 | -79 | -72 | -56 | +| keystone, lc | `4766787e793d` | -62 | -62 | -70 | -79 | -55 | -58 | + +- At seed 1166, without the keystone, all three pixel takes grow a late bed: + -11 to -49 dB. Both carry takes stay at -52 to -81 dB. The bed has almost no + energy above 6 kHz (0-1%), which fits a tonal music bed rather than hiss. +- Seed 2's pixel take has no bed at all, and its carry take is louder than it + is. With the keystone, neither side has one. +- So the bed is something the pixel chain grows at one seed. The carry avoids + it there, but "the carry suppresses the bed" does not hold in general. +- **S2 keeps its ambience.** Per segment, lc against control: + - seed 1166: -50.4/-52.0/-54.1/-54.0/-53.9/-56.8 against + -50.4/-52.1/-55.4/-55.6/-55.2/-57.1; + - seed 2: -45.1/-43.9/-40.3/-45.6/-57.5/-56.9 against + -45.1/-44.2/-40.2/-48.6/-61.3/-54.3. + + That is within 1-4 dB, segment by segment, on both seeds. + +### S2, 6 segments (arc) + +| arm | job | control | jump | jump max | look | look max | tex fine | tex mid | tex coarse | motion | verdict | +|---|---|---|---|---|---|---|---|---|---|---|---| +| guide control @1166 | `536548e14dd5` | | 1.362 | 2.09 | 1.286 | 1.74 | 3.084 | 2.293 | 1.430 | 4.16 | | +| lc @1166 | `d93adb33249e` | `536548e14dd5` | _1.664_ | 2.45 | _1.960_ | _2.75_ | 3.625 | _3.647_ | _3.536_ | 4.07 | mixed (seams worse) | +| guide control @2 | `1b8edfdba405` | | 1.306 | 1.83 | 1.418 | 1.77 | 3.518 | 2.994 | 2.545 | 4.44 | | +| lc @2 | `d901c8c82784` | `1b8edfdba405` | _1.480_ | 2.16 | _2.140_ | _3.54_ | 3.525 | 3.518 | 3.186 | 4.35 | mixed (seams worse) | + +- The rule says *mixed*: the primary (fine texture) ties inside S2's wide floor. + Plainly read, the carry **loses on the seams on both seeds**: look mean + +0.67 and +0.72 against a 0.438 floor, and jump mean past 0.124 on both. +- Motion is kept (4.07 against 4.16, and 4.35 against 4.44), so the camera + still moves. +- The S2 carry takes are 12.8-13.3 min, against 10.8-10.9 min for the + controls. Most of the gap is a fresh model load (160-167 s) in both carry + jobs. Without it they take 10.1-10.5 min, about the same as the controls. + +### The call (Step 3) + +- **#1044 (latent carry, extra rows): not built as a default.** On the static + scene it wins texture at the cost of coarse texture. It does not stop the + burn, and it ties when isolated. On the moving scene it loses the seams. Keep + the scratch flag for the keystone round below; the second ksl seed decides it. + If Don's eye does not prefer the carry's face there, close it with these + numbers. +- **#1150 (the VRAM-saving arm 2) and #1030 (decode-once):** not measured by + this A/B. The only saving seen is the skipped re-encode, about 5 min per + two-minute take. That is not worth a feature on its own. +- **The keystone reset is the lead to chase.** It is one seed, and it is the + first sign of a two-minute static take that comes back clean. + +### Next round for Steps 5-7 (#1188), recommended + +- **Run the keystone confirmation first. It is the smallest round that can + change the plan.** Two arms at seed 2, S1, 120 s, on two cards at once: + - `ks-pixel-anchor25-s1-120s-s2`; + - `ks-lc-anchor25-s1-120s-s2`. + + That is about 2 x 68 min, 2.3 GPU-hours, or 70 min of wall. Optionally add + a third arm, keystone without the anchor at seed 1166 (about 52 min, 0.9 + GPU-hours), to separate the keystone's reset from anchor 0.25's softness. + In total that is about 3.2 GPU-hours, or about 2 h on two cards. +- **Hold the Step 5 build (hybrid windows) until that round reads.** + - Step 4's windowed refine already gives the seam pass, and it does not touch + the burn. + - If the chain keystone resets the burn at seed 2 as well, Step 6's + keystone/capstone question is answered in chain form. Step 5's remaining + value is then the seamless blended finish: a new block, plus at least + 36 min per minute of take. That is a weak case. + - If the reset does not repeat, hybrid windows with a finished window-1 + keystone (Steps 5-6) is the remaining structural burn lever, and it is worth + the build. +- **Latent carry:** do not open a follow-up feature. The ksl seed-2 arm above + decides whether the scratch flag stays (face hold by eye) or #1044 closes. + +### Exports for Don + +Exports are in `~/Desktop/dw-bench-t17/`, side by side, x264 CRF 16: + +- `t17-s1-120s-keystone_LEFT-pixel_RIGHT-latentcarry-75-120s.mp4` + (`05a3a55b043c` | `4766787e793d`, the right take's track). It shows the reset + at about 80 s on the left, and the carry's face at the end. +- `t17-s1-120s-noanchor_LEFT-pixel-control_RIGHT-latentcarry-40-70s.mp4` + (`8d1614ced27a` | `20ce09d60040`, the right take's track). It shows that + without the anchor the carry burns rather than softens. + +Records: `latent-carry.json` (the a and b drivers merged), +`latent-carry-controls.json`, `latent-carry-controls-r36.json` (arms +`bench/arms/latent-carry-controls-r36.json`), `latent-carry-noanchor.json`, +and the moments pass `t17-t19-moments.json`. The moments pass was taken +in-process on lem from the bench checkout at `6a89744a`: raw texture bands, +face crop, seams, motion, per-segment audio, and PSNR. + +## Keystone confirmation (seed 2, and keystone without the anchor) + +Arms `bench/arms/keystone-confirm.json`, record `keystone-confirm.json`, +moments pass `keystone-confirm-moments.json`. S1, 120 s, guide, 6 segments +(28 recorded seams, every 4.25 s). + +### Verdict + +- **The reset repeats at seed 2, in both arms. It is the keystone's, not the + anchor's.** Every keystone take at both seeds comes back clean at least once. + No take without a keystone ever does. + - Seed 2, pixel (`e4c7c19a99cd`): it resets twice, at 58.7 s and 110.5-111.2 s. + - Seed 2, latent carry (`4cae30688ed4`): it resets once, at 91.7 s, and stays + clean to the end. + - Keystone without the anchor, seed 1166 (`c4ec5a055464`): it resets twice, + at 39.5 s and 81.5 s. +- **The reset is a wipe, not a hold.** Between resets the crust grows back. A + take is a sawtooth: clean, then crust from about 35 s, a reset, and crust + again. How much crust a take ends on depends on where its last reset fell. + The timing is not regular: 39-111 s, and it moves with the seed and the arm. +- **Each reset is a visible pop.** The crust is gone within 4 frames, in the + middle of a segment. It is not at a recorded seam, so `analyze_seams` does + not see it. The light, colours, framing and face stay put. Only the backdrop + texture changes. Don's clip below shows it. +- **Against the controls it wins clearly, on both seeds and without the + anchor.** Fine texture at the end is 0.11 against 1.14 (seed 2), and 0.53 + against 1.71 (no anchor). The no-anchor keystone take also wins every seam + measure against the guide control. +- **The anchor decides the face, not the backdrop.** Without the anchor the + face burns: high contrast and painted, but never a mask. With the anchor, + the pixel face goes soft and pale from about 100 s. At seed 2, after the + 111 s reset, it is a soft white mask with faint eyes. The carry face stays + sharp on both seeds. +- **Cost: the keystone is free.** 53.0 min against 52.2 min without the anchor, + and 72.0 against 70.7 with it. The anchor costs about 17 min per two-minute + take (+33%). +- **The keystone also stops seed 1166's late music bed without the anchor.** + The 20 s means are -64/-73/-68/-76/-73/-70 dB, against the guide control's + -64 to -11. Seed 2 grows no bed in any arm. The level step is not counted. + +### Measures + +End of take, and seams over all 28. The floors are the two-minute floors above: +jump 0.124, jump max 0.430, look 0.438, look max 0.790, fine 0.081, mid 0.159, +coarse 0.118. **Bold** is better than its control past the floor; _italic_ is +worse. + +| arm | job | control | jump | jump max | look | look max | tex fine | tex mid | tex coarse | wall min | verdict (picture) | +|---|---|---|---|---|---|---|---|---|---|---|---| +| anchor 0.25, seed 2 (control) | `bf9ee32e040f` | | 0.987 | 1.43 | 1.295 | 2.66 | 1.142 | 0.590 | 0.017 | 70.7 | | +| keystone + anchor 0.25, pixel, seed 2 | `e4c7c19a99cd` | `bf9ee32e040f` | **0.849** | 1.25 | 1.014 | 2.28 | **0.106** | **0.054** | 0.001 | 72.0 | **wins** | +| keystone + anchor 0.25, lc, seed 2 | `4cae30688ed4` | `e4c7c19a99cd` | 0.805 | 1.15 | 1.198 | 2.26 | 0.026 | 0.055 | 0.071 | 67.9 | tie | +| (same, against the seed-2 control) | `4cae30688ed4` | `bf9ee32e040f` | **0.805** | 1.15 | 1.198 | 2.26 | **0.026** | **0.055** | 0.071 | | **wins** | +| guide control, seed 1166 | `8d1614ced27a` | | 1.341 | 6.56 | 2.233 | 11.51 | 1.706 | 1.171 | 0.088 | 52.2 | | +| keystone, no anchor, seed 1166 | `c4ec5a055464` | `8d1614ced27a` | **0.844** | **1.06** | **1.156** | **2.04** | **0.527** | **0.200** | 0.028 | 53.0 | **wins** | +| (same, against keystone + anchor) | `c4ec5a055464` | `05a3a55b043c` | 0.844 | **1.06** | 1.156 | 2.04 | _0.527_ | _0.200_ | 0.028 | | loses (by phase, see below) | + +- The seed-2 carry against pixel ties: its fine gain of 0.080 sits on the + 0.081 floor. +- The no-anchor take "loses" to `05a3a55b043c` only because of where its last + reset fell. Its last reset was at 81.5 s, so it has 38 s of new crust at + 119.5 s. `05a3a55b043c`'s was at 80-81 s, yet it stays clean. On the 5 s + pass the no-anchor take sits at 0.21-0.66 between resets. The keystone + + anchor pixel takes sit at 0.11-0.50. + +Fine texture drift against frame 62, top-left scene crop, every 10 s (5 s pass): + +| s | 20 | 30 | 40 | 50 | 60 | 70 | 80 | 90 | 100 | 110 | 115 | 119.5 | +|---|---|---|---|---|---|---|---|---|---|---|---|---| +| anchor 0.25, seed 2 `bf9ee32e040f` | 0.26 | 0.39 | 0.67 | 0.79 | 0.86 | 0.88 | 0.85 | 0.92 | 0.99 | 1.22 | 1.18 | 1.14 | +| ks + anchor, pixel, seed 2 `e4c7c19a99cd` | 0.24 | 0.35 | 0.44 | 0.43 | **0.18** | 0.19 | 0.20 | 0.22 | 0.30 | 0.36 | **0.12** | 0.11 | +| ks + anchor, lc, seed 2 `4cae30688ed4` | 0.16 | 0.15 | 0.13 | 0.10 | 0.10 | 0.06 | 0.07 | 0.11 | 0.02 | 0.01 | 0.03 | 0.03 | +| guide control, 1166 `8d1614ced27a` | 0.38 | 0.66 | 1.14 | 1.33 | 1.48 | 1.45 | 1.60 | 1.66 | 1.65 | 1.68 | 1.70 | 1.71 | +| ks, no anchor, 1166 `c4ec5a055464` | 0.26 | 0.44 | **0.22** | 0.26 | 0.39 | 0.66 | 0.62 | **0.21** | 0.29 | 0.38 | 0.43 | 0.53 | +| ks + anchor, pixel, 1166 `05a3a55b043c` | 0.26 | 0.37 | 0.47 | 0.41 | 0.45 | 0.50 | 0.37 | **0.11** | 0.11 | 0.14 | 0.03 | 0.03 | +| ks + anchor, lc, 1166 `4766787e793d` | 0.14 | 0.10 | 0.07 | 0.03 | 0.03 | 0.04 | 0.05 | 0.04 | 0.08 | 0.07 | 0.08 | 0.06 | + +**Bold** marks the first moment after a reset. As before, the carry rows stay +low on this crop while the full frame crusts (seed 2: 35-91 s). Read them with +the stills. + +### By eye + +Full frames every 10 s, 1 s strips around each reset, and 4-frame strips across +the pops. + +- **Seed 2, pixel (`e4c7c19a99cd`).** + - Crusts from about 35 s. + - The backdrop pops clean between frames 1408 and 1412 (58.7 s). The seams + either side, 56.2 s and 60.4 s, score look 0.68 and 0.32. + - It is clean at 59-65 s, then crusts again from about 70 s. + - The second reset, at 110.5-111.2 s, is a quicker wipe: one patch of crust + lingers for about 15 frames, up to the 111.4 s seam (look 1.57). + - The face is sharp and natural to 80 s. From 100 s it is soft and smeared. + At 112-119.5 s it is a pale, soft mask with faint eyes and dark glossy + hair. That is anchor 0.25's softness, as in `25f6bd1da507`. + - So the low end-of-take texture is a clean backdrop with a soft face: real + detail on the wall, blur on the face. +- **Seed 2, latent carry (`4cae30688ed4`).** + - Mild crust from about 30 s, full crust at 40-91 s. + - It pops clean between 91.5 and 92.0 s, mid-segment (the seams at 90.2 s + and 94.4 s score look 0.72 and 0.31). It is clean to the end, with a faint + speckle on the right-hand wall at 119.5 s. + - The face is sharp the whole way: clear eyes, nose and mouth. It is a little + glossy and plastic, as at seed 1166. +- **Seed-2 control (`bf9ee32e040f`).** It crusts from 35 s and never recovers. + At 119.5 s the frame has shattered, and the face has been a white mask with + dark eye holes since about 60 s. +- **Keystone, no anchor (`c4ec5a055464`).** + - Crusts from about 25 s, then pops clean at 39-40 s. + - It crusts again from about 60 s and pops clean at 81-82 s. + - It is crusted again by 110 s. + - The face is never soft and never a mask. From 60 s it is high contrast and + painted, with glossy dark hair, blonde streaks and a coarse sweater weave: + the burn, without the backdrop's shards. At 119.5 s the eyes are sharp. + - Against the guide control (`8d1614ced27a`), whose face is cracked and + posterised from 62 s, it is much better. Against `05a3a55b043c` the face is + harsher but not masky, and the backdrop is crusted for longer. +- **Staging holds in every take.** The camera is static, with the circle top + left, the square bottom right and the subject centred, as in segment 1. No + pop moves the camera or the light. +- **Identity drifts in every take**, with or without the keystone: darker hair + with lighter streaks, a heavier brow and fuller lips. The keystone resets the + backdrop, not the face. + +### Answers + +1. **Seed 2.** Yes, in both arms. + - Pixel: 58.7 s, then again at 110.5-111.2 s. + - Carry: 91.7 s. + - Each reset is a pop, mid-segment, within a few frames. Light and framing + do not jump. +2. **Without the anchor.** The keystone alone resets the backdrop, at 39.5 s + and 81.5 s. The reset needs the keystone. The anchor is not needed for it. + The anchor only trades the face's burn (painted) for softness (a soft mask + late in the pixel takes). +3. **Detail or softness.** + - On the backdrop the low texture is detail: the post-reset wall is + segment 1's flat painted wall, with sharp edges on the circle and square. + - On the face, the anchored pixel take is soft and masky from about 100 s. + - The carry face is sharp. +4. **Latent carry under the keystone.** + - It ties on numbers at both seeds. + - By eye its face holds better on both: the clearest face at 104-120 s at + seed 1166, and sharp to the end at seed 2 while the pixel face goes to a + soft mask. + - It resets later on both seeds (104 against 80 s, and 92 against 59 s). + - It is about 4 min faster. + - **Keep the scratch flag.** This was the condition set above (face hold by + eye). It stays scoped to static shots: on S2 it lost the seams on both + seeds. It is not a default, and not a new feature until Don has seen the + 88-120 s clip. +5. **Steps 5-6 and the default.** See *The call* below. + +### The call (Steps 5-7) + +- **Do not build Steps 5-6 (#1188 hybrid windows) in this plan.** Step 7's + verdict is not built, by the rule written above: + - the chain keystone resets the burn on two seeds and without the anchor, so + Step 6's keystone question is answered in chain form; + - Step 5's remaining value is the blended finish. Step 4's windowed refine + already gives the seam pass, and the blended finish costs at least 36 min + per minute of take. + + Don answers the #1188 commenter. If he reopens the build later, the open + problem is the sawtooth: the burn regrows between resets. The cheaper next + lever is to find what triggers the reset and make it regular, per segment, + in the chain. +- **Make the keystone the default for static chained takes past about 30 s, + and keep anchor 0.25 a knob.** + - The keystone is free and wins at both seeds. It never hurts before the + first reset: it matches the control's drift up to about 35 s. + - It is measured on static shots only. In round 1 it lost the look on S2's + arc, so it is not a default for moving shots. + - Anchor 0.25 costs +33% wall. It trades a painted face for a soft or masky + one late in the take, and Don called it "soft and masky". It stays a knob. + - Caveat for Don's eye: the default brings a visible mid-shot pop at an + unpredictable time. The take is still crusted for 40-60% of its length + between resets. It is a large improvement on the burn, not a clean take. + +### Exports for Don + +In `~/Desktop/dw-bench-t17/`, side by side, x264 CRF 16: + +- `t17-s1-120s-seed2-keystone_LEFT-pixel-keystone-anchor25_RIGHT-anchor25-control-50-66s.mp4` + (`e4c7c19a99cd` | `bf9ee32e040f`, the left take's track). The pop is at + 8.7 s into the clip (58.7 s); the control stays crusted, with a white-mask + face. +- `t17-s1-120s-seed2-keystone_LEFT-pixel_RIGHT-latentcarry-88-120s.mp4` + (`e4c7c19a99cd` | `4cae30688ed4`, the right take's track). The carry's pop + is at 3.7 s (91.7 s), and the pixel's second reset at about 23 s + (111 s). Compare the faces at the end: soft mask on the left, sharp on the + right. This clip decides #1044's flag. + +## Keystone slot (#1187, #1188) + +Arms `bench/arms/keystone-regular.json` (engine `experimental_keystone_slot`, +4ef3cd03), record `keystone-regular.json`, re-measured on the standard three +moments (frames 62, 1501, 2868): the run's `--measure` asked for 13 and the +compare route takes 8, so the run left a 400 in every row's `error`. The copy +here clears it. S1, 120 s, guide, anchor none, keystone segment_1. The +control is plain keystone `c4ec5a055464` (seed 1166). + +### Verdict + +- **Don's call (2026-11-06): not a loop; slot last is the static default.** + `keystone_slot` is a stable chain key (`experimental_keystone_slot` is + refused naming it), and `templates/minimax/chained-segments` carries it as + `shot: static_long` (guide, keystone `segment_1`, slot `last`) for static + takes past about 30 s. Plain keystone (`keystone_slot: first`) and no + keystone stay knobs; moving shots are unchanged. +- **Slot "last" holds the crust down for the whole two minutes, with no pop, + on both seeds.** Each segment's crust stays at 1.1-1.5x segment 1 (find.py's + backdrop measure), against plain keystone's sawtooth of 1.0-6.3x. The face + holds too: sharp, natural, segment 1's identity to 120 s, with none of the + painted burn or hair drift. + - Seed 1166 (`3232cabd16b1`): segment means 1.11-1.46x, 1.23x at the end. + - Seed 2 (`94c5c0b219b3`): 1.16-1.40x, 1.37x at the end. +- **Its price is a 4.25 s pose loop.** The subject goes back to segment 1's + exact opening pose at the end of every segment, and nearly stops there. + Each segment is one excursion (a glance, a blink, a head turn) and a + return to front-on. It eases in and out with no snap: the seam's motion + step is no larger than the take's own (2.4 against a p99 of 3.1). But the + rhythm repeats 28 times. In a strip it reads as natural, glance and return. + Played through, it may read as a loop. That is Don's call (clip below). +- **The rest fail.** + - **f34** (`726bb72c3908`) does make the reset regular. The crust drops + from 1.4-1.9x to 1.05x at render frame 32-34 in nearly every segment. + But the reset comes with a visible snap: the pose jumps to segment 1's + in one frame (eyes down to front-on), at a motion step of 12-25 against + a p99 of 11.8. From about 64 s the drop moves to the seam (render 22). + The hair frizzes from about 80 s. + - **f34 gated at sigma 0.9** (`16f116c27981`) is the worst take of the + round. The keystone never takes the guide over. It shows as a + three-frame flash of segment 1's clean frame at render 34-36 in every + segment. The crust climbs to 13.6x by 77 s, past the guide control's + peak of 10.6x, before one partial reset. Close it. + - **guide_noise 0.4** (`60d9f28c6865`): the threshold did not drop. It + rose. Pops are at 45.5 s (from 5.9x) and 85.7-87.0 s (from 6.4x), + against the control's 4.8-5.2x and 6.3x. As predicted, crust grows + faster: segment 2-5 means are 1.26-2.41 against 1.16-2.09. It ends at + 6.4x. It is still a sawtooth. + - **guide_frames 39** (`d5cb657fb9e8`): the first threshold rose as + predicted. The first pop is at 58.5 s, from 9.8x, in one frame (the + hardest pop yet). The later two are lower: 94.9 s (6.5 to 3.9x) and + 101.3-102 s (4.4 to 1.5x). The 39-frame guide means 34 segments and + 71.3 min wall (+18 min). Close it. +- **Recommendation.** Slot "last" is the winner. It should replace plain + keystone as the static long-take default if Don accepts the pose loop. + If he does not, plain keystone stays, and the slot work closes. It is not + promotable until two things are settled: + - **The prompt confound: settled.** The slot alone holds. Without the + clause naming `` at 5.17 s, both seeds hold the crust, the + face and the pose return the same way. See *Confirm without the Picture 2 + clause* below. + - **A moving scene.** The slot pins every segment's last frame to segment + 1's, so it cannot work on S2's arc as it stands. It is static only. + +### Confirm without the Picture 2 clause + +Arms `bench/arms/keystone-last-noprompt.json`, record +`keystone-last-noprompt.json`. Same as slot last, but the later-segment +prompt drops the clause that names `` at the last frame. + +- **It holds.** The slot alone keeps the crust down for two minutes, with no + pop, on both seeds. The clause was not doing the work. + - Seed 1166 (`f55bd445faa5`): segment means 1.10-1.46x, 1.25x at the end. + The prompted take was 1.11-1.46x, 1.23x. + - Seed 2 (`00ed8fb067ca`): 1.12-1.40x, 1.31x at the end. The prompted + take was 1.16-1.40x, 1.37x. +- **The pose loop stays too.** The clause did not cause it. The slot does. + The last frame of each segment sits 4.0 (s1166) and 4.1 (s2) from + segment 1's frame 0, against 11.3 and 13.2 mid-segment. The prompted takes + were 4.1 and 4.1, against 13.4 and 14.4. Motion falls to 0.12 and 0.11 at + the last frame, from 1.6 and 2.0 mid-segment. The seam step is 2.3 (max + 2.7 and 3.4) against a p99 of 2.8 and 4.3. No snap. +- **The face** matches the prompted take at 60 s and 120 s on seed 1166: + sharp, natural, the same person. The eyes go bluer and the skin warmer + than segment 1, as in the prompted takes. No burn, no hair drift. +- **The backdrop** is segment 1's flat painted wall at every 10 s mark on + both seeds, with sharp circle and square edges. +- The look ratio is a little lower than the prompted takes (1.50 and 1.36 + against 1.61 and 1.59). Neither is worse than its control past the + floor. Wall is about 5 min shorter; that is run-to-run, not the clause. + +| arm | job | control | jump | jump max | look | look max | tex fine | tex mid | tex coarse | wall min | verdict (picture) | +|---|---|---|---|---|---|---|---|---|---|---|---| +| slot last, no clause, 1166 | `f55bd445faa5` | `c4ec5a055464` | **0.689** | 1.17 | 1.496 | 2.30 | **0.085** | **0.005** | 0.006 | 57.4 | **wins** | +| slot last, no clause, seed 2 | `00ed8fb067ca` | `e4c7c19a99cd`* | **0.632** | 0.84 | 1.357 | 2.02 | 0.130 | 0.008 | 0.003 | 56.9 | wins by eye | + +\* As above, seed 2 is read against keystone + anchor 0.25. + +| take | 0 | 10 | 20 | 30 | 40 | 50 | 60 | 70 | 80 | 90 | 100 | 110 | end | pops (s, render frame, crust before -> after) | +|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---| +| slot last, no clause, 1166 `f55bd445faa5` | 0.91 | 1.28 | 1.27 | 1.21 | 1.41 | 1.40 | 1.27 | 1.38 | 1.37 | 1.27 | 1.38 | 1.46 | 1.25 | none | +| slot last, no clause, s2 `00ed8fb067ca` | 0.91 | 1.31 | 1.29 | 1.15 | 1.33 | 1.29 | 1.26 | 1.30 | 1.40 | 1.33 | 1.34 | 1.35 | 1.31 | none | + +- find.py flags three dips of 0.12-0.17x per take at render 107-121. The + prompted takes have one to three of the same. Each is a segment's own + ease into the return pose, not a pop: the level falls by 0.1-0.2x, not + back to 1x. + +### Measures + +Seams over all 28 (33 for g39), and the end-of-take texture (frame 2868 +against 62). Floors: jump 0.124, look 0.438, fine 0.081, mid 0.159, coarse +0.118. **Bold** is better than the control past the floor; _italic_ is +worse. Wall minutes are two arms at a time, one per card, as the control was. + +| arm | job | control | jump | jump max | look | look max | tex fine | tex mid | tex coarse | wall min | verdict (picture) | +|---|---|---|---|---|---|---|---|---|---|---|---| +| plain keystone, 1166 (control) | `c4ec5a055464` | | 0.844 | 1.06 | 1.156 | 2.04 | 0.527 | 0.200 | 0.028 | 53.0 | | +| slot last, 1166 | `3232cabd16b1` | `c4ec5a055464` | 0.737 | 1.50 | _1.609_ | 2.78 | **0.098** | **0.017** | 0.005 | 61.9 | **wins** (look by 0.015 past the floor) | +| slot last, seed 2 | `94c5c0b219b3` | `e4c7c19a99cd`* | **0.676** | 1.13 | _1.590_ | 2.53 | 0.183 | 0.037 | 0.003 | 62.1 | wins by eye | +| f34 | `726bb72c3908` | `c4ec5a055464` | _1.341_ | 4.04 | 1.106 | 2.49 | **0.156** | **0.029** | 0.016 | 56.5 | loses by eye (pose snap) | +| f34, gate 0.9 | `16f116c27981` | `c4ec5a055464` | _1.055_ | 1.72 | _1.687_ | 4.53 | _1.542_ | _1.047_ | 0.068 | 55.2 | loses | +| guide_noise 0.4 | `60d9f28c6865` | `c4ec5a055464` | 0.843 | 0.99 | 0.884 | 2.90 | _1.314_ | _0.672_ | 0.010 | 53.8 | loses | +| guide_frames 39 | `d5cb657fb9e8` | `c4ec5a055464` | 0.902 | 1.17 | 1.217 | 2.33 | 0.529 | 0.210 | 0.007 | 71.3 | tie, +18 min | + +\* There is no seed-2 plain keystone take without the anchor. The seed-2 +row is read against keystone + anchor 0.25 (`e4c7c19a99cd`), so read it by +eye. + +- The "last" arms' worse look comes from how they end. Each segment slows to + near stillness on the return pose (motion 0.13 in the last frame, against + about 1 mid-segment). `look_ratio` divides by the local rate, so a small + step scores high. At the seams the eye sees a slight tone step: the eyes + go a little bluer and brighter. It is not a jump. +- End-of-take texture depends on where a take sits on its sawtooth. The + crust table below is the read. + +Backdrop crust (find.py: high-frequency energy of three backdrop crops, in +multiples of segment 1), every 10 s, then the pops (a fall of more than 0.12x +in 4 frames that holds): + +| take | 0 | 10 | 20 | 30 | 40 | 50 | 60 | 70 | 80 | 90 | 100 | 110 | end | pops (s, render frame, crust before -> after) | +|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---| +| plain keystone `c4ec5a055464` | 0.91 | 1.43 | 2.11 | 3.47 | 1.65 | 1.88 | 2.83 | 4.76 | 2.70 | 2.19 | 2.25 | 2.29 | 3.02 | 39.7 (r34, 4.8->1.6); 77.9-80.7 gradual (6.3->1.75); 99.3 (3.2->2.3); 107.7 (2.9->2.3) | +| slot last, 1166 `3232cabd16b1` | 0.91 | 1.29 | 1.30 | 1.24 | 1.47 | 1.42 | 1.25 | 1.49 | 1.39 | 1.23 | 1.37 | 1.34 | 1.23 | none | +| slot last, s2 `94c5c0b219b3` | 0.91 | 1.32 | 1.33 | 1.14 | 1.32 | 1.38 | 1.23 | 1.33 | 1.33 | 1.26 | 1.34 | 1.37 | 1.37 | none | +| f34 `726bb72c3908` | 0.91 | 1.06 | 1.55 | 1.41 | 1.20 | 1.50 | 1.16 | 1.12 | 1.16 | 1.09 | 1.58 | 1.77 | 1.35 | every segment from 31 s: r32-34 (1.4-1.9->1.05) to 57 s, then r22 at 65-95 s, r32 at 103-112 s | +| f34 gate 0.9 `16f116c27981` | 0.91 | 1.29 | 2.39 | 4.15 | 7.13 | 9.97 | 11.55 | 13.08 | 2.41 | 3.98 | 5.97 | 7.03 | 8.65 | 3-frame clean flash at r34 every segment; one reset 77.4 s (13.6->4.5) | +| guide_noise 0.4 `60d9f28c6865` | 0.91 | 1.61 | 2.42 | 3.52 | 5.53 | 1.75 | 2.64 | 4.27 | 5.72 | 1.86 | 3.28 | 4.78 | 6.41 | 44.8-45.7 (r56-77, 5.9->1.5); 84.7-87.0 (r94-r47 of the next, 6.4->1.9) | +| guide_frames 39 `d5cb657fb9e8` | 0.91 | 1.45 | 2.32 | 3.34 | 4.67 | 7.23 | 1.47 | 2.10 | 3.16 | 5.39 | 4.21 | 1.82 | 3.07 | 58.5 (r43, 9.8->1.5, one frame); 94.9 (r68, 6.5->3.9); 101.3-102.0 (r52-68, 4.4->1.5) | +| ks + anchor, s2 `e4c7c19a99cd` | 0.91 | 1.29 | 1.90 | 2.48 | 3.27 | 3.99 | 1.52 | 1.99 | 2.66 | 3.40 | 4.31 | 3.99 | 1.39 | 58.6-58.8 (r81-85, 4.4->1.6); 110.5-111.2 (r102-119, 4.5->1.4) | +| guide control `8d1614ced27a` | 0.91 | 1.37 | 2.32 | 4.24 | 7.49 | 9.29 | 8.96 | 5.89 | 7.09 | 8.73 | 4.95 | 6.55 | 6.92 | partial only: 60.3-63.0 (10.6->4.7); 94.4 (r22, 8.7->4.6) | + +- Plain keystone's first pop is at 39.7 s, render 34, as in the spike's table. + The spike called its second "80.5 s". Here it is a 3 s slide from 77.9 s. +- Thresholds just before each full pop: plain keystone 4.8x, then 6.3x; noise + 0.4 5.9x, then 6.4x; g39 9.8x, then 6.5x, then 4.4x; keystone + anchor (s2) + 4.4x, then 4.5x. The threshold moves with the take, not cleanly with the + knob. +- Single-frame dips on the 17-frame grid (r35, 52, 69, 86, 103, 120) show in + every take without the slot fix, the guide control included. They are the + latent chunk boundaries, not pops, and the table leaves them out. + +### By eye + +Full frames every 10 s for all nine takes, strips across seams 12-14 (slot +last), and frames 1150-1161 (render 28-39 of segment 12, f34 and gate 0.9). +Pose is measured as the subject crop's mean difference from segment 1's +frame 0. + +- **Slot last, both seeds.** + - The backdrop is segment 1's flat painted wall at every 10 s mark, with + sharp circle and square edges. There is a faint speckle at most. + - The face is sharp, natural and the same person to 120 s. Light brown + hair with blonde streaks, a plain sweater weave. It is the best face of + any two-minute take so far, anchor or carry included. + - Pose: the last frame of every segment sits 4.1 from segment 1's frame 0. + The take's own first 20 frames sit 4.05 from it, and mid-segment sits + 13.5. So every segment returns to the opening pose. Plain keystone sits + at 31 throughout. The return eases: motion falls from about 1.4 + mid-segment to 0.13 at the last frame. There is no snap. + - At the seam there is a slight tone step (eyes bluer, skin a touch + warmer), as in every chained take. +- **f34.** + - The backdrop crust stays low, but you can see it breathe. The texture + builds over about 1.5 s, then wipes at render 34. + - At the wipe the pose snaps to segment 1's in one frame (segment 12: + eyes down at 1155, front-on and eyes open at 1156). + - The hair goes frizzy and curly from about 80 s, and stringy at the + ends by 110 s. +- **f34 gate 0.9.** It burns like the guide control, and harder. Shattered + confetti backdrop from 40 s, a metallic face from 50 s. Every segment + flashes segment 1's clean frame for 3 frames (1156-1158). Close it. +- **guide_noise 0.4.** A sawtooth, as plain keystone. The pops at 45 and + 86 s are full wipes. The face stays natural, with mild freckling. +- **guide_frames 39.** A sawtooth. By 100 s the hair is tight corkscrew + curls and the face is cracked. The 58.5 s pop is a one-frame wipe of + heavy crust, the most visible pop in the round. +- **Staging** (camera, circle, square, light) holds in every take. + +### Audio + +Per-segment RMS, then the 20 s means: + +- Slot last: room tone only, -48 to -84 dB per segment. The 20 s means are + -54 to -66 dB (s1166) and -53 to -79 dB (s2). No bed, no break. +- noise 0.4 and plain keystone: room tone, -50 to -87 dB. +- f34 grows a steady bed from about 47 s (-33 to -40 dB per segment). + Gate 0.9 grows one from about 63 s (-27 to -32 dB). g39 rises at the end + (-37 dB). This is like the seed-1166 guide control's late bed (-10 dB), + but quieter. Not checked by ear. +- The soundtrack is unbroken in every take: 120.0 s, with no silent segment. + +### Exports for Don + +In `~/Desktop/dw-bench-t17/`, side by side, x264 CRF 16: + +- `t17-s1-120s-s1166-keystone_LEFT-slot-last_RIGHT-plain-keystone-30-50s.mp4` + (`3232cabd16b1` | `c4ec5a055464`, the left take's track). The right take + pops at 9.7 s (39.7 s). The left take stays clean, with no pop, through + the stretch. Watch the left subject's return to the front-on pose at + seam, just before 0.7, 4.9, 9.2, 13.4 and 17.7 s into the clip. That rhythm is + the call. diff --git a/bench/records/t17-pairs-120s-input.json b/bench/records/t17-pairs-120s-input.json new file mode 100644 index 000000000..8f0f66c08 --- /dev/null +++ b/bench/records/t17-pairs-120s-input.json @@ -0,0 +1 @@ +[{"arm":"t17-s1-120s-r030-rebased","scene":"s1-static","source":"BenchS1Static/20261104-233558-8c6ae697/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4","refined":"BenchRefineWindows/20261105-125907-13420dda/final/BenchRefineWindows-mux.3-0.0.mp4","moments":["frame:62","frame:300","frame:583","frame:1000","frame:1501","frame:2000","frame:2500","frame:2868"]}] diff --git a/bench/records/t17-pairs-120s.json b/bench/records/t17-pairs-120s.json new file mode 100644 index 000000000..b720fe41a --- /dev/null +++ b/bench/records/t17-pairs-120s.json @@ -0,0 +1,594 @@ +[ + { + "arm": "t17-s1-120s-r030-rebased", + "scene": "s1-static", + "moments": [ + "frame:62", + "frame:300", + "frame:583", + "frame:1000", + "frame:1501", + "frame:2000", + "frame:2500", + "frame:2868" + ], + "source": { + "measures": { + "seam_jump_mean": 0.9432142857142857, + "seam_jump_max": 1.86, + "seam_look_mean": 1.0325, + "seam_look_max": 2.7, + "seam_level_step_db_mean": 5.834285714285714, + "seam_level_step_db_max": 24.34, + "typical_delta_mean": 2.0892857142857144, + "texture_fine_seg_last": 1.1076040063709027, + "texture_mid_seg_last": 0.43039061064808104, + "texture_coarse_seg_last": 0.01932700686817017, + "face_fine_seg_last": 0.04024648279647207 + }, + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 0.69, + "look_ratio": 1.98, + "level_step_db": 16.58, + "typical_delta": 2.0 + }, + { + "seconds": 9.417, + "jump_ratio": 1.15, + "look_ratio": 1.57, + "level_step_db": 5.73, + "typical_delta": 2.0 + }, + { + "seconds": 13.667, + "jump_ratio": 0.78, + "look_ratio": 1.16, + "level_step_db": 12.1, + "typical_delta": 2.0 + }, + { + "seconds": 17.917, + "jump_ratio": 0.98, + "look_ratio": 0.47, + "level_step_db": 23.54, + "typical_delta": 2.0 + }, + { + "seconds": 22.167, + "jump_ratio": 1.04, + "look_ratio": 0.46, + "level_step_db": 2.85, + "typical_delta": 2.0 + }, + { + "seconds": 26.417, + "jump_ratio": 0.89, + "look_ratio": 0.43, + "level_step_db": 24.17, + "typical_delta": 2.0 + }, + { + "seconds": 30.667, + "jump_ratio": 0.88, + "look_ratio": 0.31, + "level_step_db": 1.96, + "typical_delta": 2.0 + }, + { + "seconds": 34.917, + "jump_ratio": 0.87, + "look_ratio": 0.63, + "level_step_db": 18.52, + "typical_delta": 2.0 + }, + { + "seconds": 39.167, + "jump_ratio": 0.82, + "look_ratio": 0.41, + "level_step_db": 4.08, + "typical_delta": 2.0 + }, + { + "seconds": 43.417, + "jump_ratio": 0.75, + "look_ratio": 0.81, + "level_step_db": 0.37, + "typical_delta": 2.0 + }, + { + "seconds": 47.667, + "jump_ratio": 0.78, + "look_ratio": 0.92, + "level_step_db": 24.34, + "typical_delta": 2.0 + }, + { + "seconds": 51.917, + "jump_ratio": 0.75, + "look_ratio": 0.36, + "level_step_db": 3.78, + "typical_delta": 2.0 + }, + { + "seconds": 56.167, + "jump_ratio": 1.29, + "look_ratio": 2.37, + "level_step_db": 1.93, + "typical_delta": 2.0 + }, + { + "seconds": 60.417, + "jump_ratio": 0.86, + "look_ratio": 0.68, + "level_step_db": 1.49, + "typical_delta": 2.0 + }, + { + "seconds": 64.667, + "jump_ratio": 0.85, + "look_ratio": 0.47, + "level_step_db": 1.32, + "typical_delta": 2.0 + }, + { + "seconds": 68.917, + "jump_ratio": 0.74, + "look_ratio": 0.74, + "level_step_db": 0.05, + "typical_delta": 2.0 + }, + { + "seconds": 73.167, + "jump_ratio": 0.96, + "look_ratio": 1.53, + "level_step_db": 1.71, + "typical_delta": 2.0 + }, + { + "seconds": 77.417, + "jump_ratio": 0.76, + "look_ratio": 0.47, + "level_step_db": 0.27, + "typical_delta": 2.0 + }, + { + "seconds": 81.667, + "jump_ratio": 0.73, + "look_ratio": 0.53, + "level_step_db": 0.59, + "typical_delta": 2.0 + }, + { + "seconds": 85.917, + "jump_ratio": 0.69, + "look_ratio": 0.65, + "level_step_db": 4.57, + "typical_delta": 2.07 + }, + { + "seconds": 90.167, + "jump_ratio": 0.84, + "look_ratio": 0.56, + "level_step_db": 3.56, + "typical_delta": 2.07 + }, + { + "seconds": 94.417, + "jump_ratio": 0.76, + "look_ratio": 0.53, + "level_step_db": 0.15, + "typical_delta": 2.0 + }, + { + "seconds": 98.667, + "jump_ratio": 1.23, + "look_ratio": 1.82, + "level_step_db": 1.87, + "typical_delta": 2.48 + }, + { + "seconds": 102.917, + "jump_ratio": 1.07, + "look_ratio": 1.9, + "level_step_db": 1.55, + "typical_delta": 2.48 + }, + { + "seconds": 107.167, + "jump_ratio": 1.24, + "look_ratio": 1.89, + "level_step_db": 1.44, + "typical_delta": 2.0 + }, + { + "seconds": 111.417, + "jump_ratio": 1.08, + "look_ratio": 1.26, + "level_step_db": 2.04, + "typical_delta": 2.16 + }, + { + "seconds": 115.667, + "jump_ratio": 1.07, + "look_ratio": 1.3, + "level_step_db": 2.51, + "typical_delta": 2.62 + }, + { + "seconds": 119.917, + "jump_ratio": 1.86, + "look_ratio": 2.7, + "level_step_db": 0.29, + "typical_delta": 2.62 + } + ], + "texture": { + "fine": [ + 0.0, + 0.15334517616385027, + 0.36683259200135493, + 0.49667659932345387, + 0.7713517464612992, + 0.9314887724824489, + 1.0123241014626363, + 1.1076040063709027 + ], + "mid": [ + 0.0, + 0.04591805925855212, + 0.12383734756733206, + 0.1793955659595618, + 0.27112590517376556, + 0.3186770924337395, + 0.3560729875623916, + 0.43039061064808104 + ], + "coarse": [ + 0.0, + 0.0018679258900470392, + 0.015490472469675862, + 0.02192424808923237, + 0.05175386349388206, + 0.012653034895651024, + 0.046590245329657345, + 0.01932700686817017 + ] + }, + "face_texture": { + "fine": [ + 0.0, + 0.015167130564098437, + 0.12309067426459207, + 0.12689992361469846, + 0.15003529283126704, + 0.01129735428088317, + 0.1233189866295138, + 0.04024648279647207 + ], + "mid": [ + 0.0, + 0.09213803415701627, + 0.14758993036246842, + 0.10169097434619166, + 0.10143820315481389, + 0.018883519785137505, + 0.10940684489191058, + 0.010156863945357199 + ], + "coarse": [ + 0.0, + 0.0013690755403704359, + 0.11575207550746598, + 0.23291343411926282, + 0.1845605158279526, + 0.37441778849122875, + 0.3569688225676066, + 0.3325853675759032 + ] + }, + "motion_mean": 0.6359326839447021 + }, + "refined": { + "measures": { + "seam_jump_mean": 0.7339285714285715, + "seam_jump_max": 1.93, + "seam_look_mean": 1.1560714285714284, + "seam_look_max": 2.96, + "seam_level_step_db_mean": 5.962142857142857, + "seam_level_step_db_max": 24.94, + "typical_delta_mean": 2.0410714285714286, + "texture_fine_seg_last": 1.0932291171246382, + "texture_mid_seg_last": 0.4061496299894634, + "texture_coarse_seg_last": 0.0035929966129105866, + "face_fine_seg_last": 0.085910805999141 + }, + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 0.4, + "look_ratio": 1.07, + "level_step_db": 17.11, + "typical_delta": 2.0 + }, + { + "seconds": 9.417, + "jump_ratio": 0.68, + "look_ratio": 1.14, + "level_step_db": 6.19, + "typical_delta": 2.0 + }, + { + "seconds": 13.667, + "jump_ratio": 0.54, + "look_ratio": 1.1, + "level_step_db": 12.25, + "typical_delta": 2.0 + }, + { + "seconds": 17.917, + "jump_ratio": 0.61, + "look_ratio": 1.06, + "level_step_db": 24.46, + "typical_delta": 2.0 + }, + { + "seconds": 22.167, + "jump_ratio": 0.41, + "look_ratio": 0.62, + "level_step_db": 2.97, + "typical_delta": 2.0 + }, + { + "seconds": 26.417, + "jump_ratio": 0.4, + "look_ratio": 0.54, + "level_step_db": 24.94, + "typical_delta": 2.0 + }, + { + "seconds": 30.667, + "jump_ratio": 0.4, + "look_ratio": 0.22, + "level_step_db": 2.01, + "typical_delta": 2.0 + }, + { + "seconds": 34.917, + "jump_ratio": 0.35, + "look_ratio": 0.25, + "level_step_db": 18.8, + "typical_delta": 2.0 + }, + { + "seconds": 39.167, + "jump_ratio": 0.35, + "look_ratio": 0.37, + "level_step_db": 4.16, + "typical_delta": 2.0 + }, + { + "seconds": 43.417, + "jump_ratio": 0.46, + "look_ratio": 0.61, + "level_step_db": 0.26, + "typical_delta": 2.0 + }, + { + "seconds": 47.667, + "jump_ratio": 0.81, + "look_ratio": 1.22, + "level_step_db": 24.59, + "typical_delta": 2.0 + }, + { + "seconds": 51.917, + "jump_ratio": 0.63, + "look_ratio": 1.11, + "level_step_db": 3.79, + "typical_delta": 2.0 + }, + { + "seconds": 56.167, + "jump_ratio": 0.99, + "look_ratio": 2.75, + "level_step_db": 1.93, + "typical_delta": 2.0 + }, + { + "seconds": 60.417, + "jump_ratio": 0.45, + "look_ratio": 0.06, + "level_step_db": 1.49, + "typical_delta": 2.0 + }, + { + "seconds": 64.667, + "jump_ratio": 0.59, + "look_ratio": 0.33, + "level_step_db": 1.33, + "typical_delta": 2.0 + }, + { + "seconds": 68.917, + "jump_ratio": 0.54, + "look_ratio": 1.11, + "level_step_db": 0.05, + "typical_delta": 2.0 + }, + { + "seconds": 73.167, + "jump_ratio": 0.75, + "look_ratio": 1.64, + "level_step_db": 1.71, + "typical_delta": 2.0 + }, + { + "seconds": 77.417, + "jump_ratio": 0.54, + "look_ratio": 1.03, + "level_step_db": 0.27, + "typical_delta": 2.0 + }, + { + "seconds": 81.667, + "jump_ratio": 0.5, + "look_ratio": 0.9, + "level_step_db": 0.59, + "typical_delta": 2.0 + }, + { + "seconds": 85.917, + "jump_ratio": 0.59, + "look_ratio": 1.2, + "level_step_db": 4.58, + "typical_delta": 2.0 + }, + { + "seconds": 90.167, + "jump_ratio": 0.57, + "look_ratio": 0.88, + "level_step_db": 3.58, + "typical_delta": 2.0 + }, + { + "seconds": 94.417, + "jump_ratio": 0.52, + "look_ratio": 0.59, + "level_step_db": 0.16, + "typical_delta": 2.0 + }, + { + "seconds": 98.667, + "jump_ratio": 0.58, + "look_ratio": 0.62, + "level_step_db": 1.88, + "typical_delta": 2.0 + }, + { + "seconds": 102.917, + "jump_ratio": 1.77, + "look_ratio": 2.85, + "level_step_db": 1.55, + "typical_delta": 2.0 + }, + { + "seconds": 107.167, + "jump_ratio": 1.57, + "look_ratio": 2.77, + "level_step_db": 1.44, + "typical_delta": 2.0 + }, + { + "seconds": 111.417, + "jump_ratio": 1.09, + "look_ratio": 1.06, + "level_step_db": 2.05, + "typical_delta": 2.03 + }, + { + "seconds": 115.667, + "jump_ratio": 1.53, + "look_ratio": 2.31, + "level_step_db": 2.52, + "typical_delta": 2.56 + }, + { + "seconds": 119.917, + "jump_ratio": 1.93, + "look_ratio": 2.96, + "level_step_db": 0.28, + "typical_delta": 2.56 + } + ], + "texture": { + "fine": [ + 0.0, + 0.1370009950914923, + 0.39077931460384646, + 0.5385901045640593, + 0.8384000231862799, + 0.9491490388061438, + 1.0028427347809536, + 1.0932291171246382 + ], + "mid": [ + 0.0, + 0.03397233036929515, + 0.1031412430904608, + 0.15496158451434483, + 0.24467156606991705, + 0.2761338770236644, + 0.3589307099640712, + 0.4061496299894634 + ], + "coarse": [ + 0.0, + 0.0010607630715225192, + 0.00129836602440019, + 0.004033584385524984, + 0.02669515306480456, + 0.009685606260966047, + 0.03178443736746201, + 0.0035929966129105866 + ] + }, + "face_texture": { + "fine": [ + 0.0, + 0.04398417837780762, + 0.1547125311548257, + 0.1832527190643538, + 0.17692661868671514, + 0.071654757971895, + 0.16902806785841507, + 0.085910805999141 + ], + "mid": [ + 0.0, + 0.10754319038342584, + 0.17875576171266588, + 0.10968900394252303, + 0.1568015854940254, + 0.005439656500560437, + 0.09736684176884565, + 0.0028736296164729408 + ], + "coarse": [ + 0.0, + 0.004009678284475416, + 0.1000805483273801, + 0.20730275269289963, + 0.15697542553148014, + 0.3526050332096982, + 0.3369300072861077, + 0.30872034432115203 + ] + }, + "motion_mean": 0.5277568101882935 + }, + "fidelity": { + "psnr_db": [ + 31.14, + 30.68, + 28.1, + 26.01, + 24.06, + 23.45, + 24.17, + 23.74 + ], + "ssim": [ + 0.9113, + 0.9082, + 0.8746, + 0.8344, + 0.809, + 0.8118, + 0.8261, + 0.8274 + ] + } + } +] \ No newline at end of file diff --git a/bench/records/t17-pairs-r020-r040.json b/bench/records/t17-pairs-r020-r040.json new file mode 100644 index 000000000..82bfeaf54 --- /dev/null +++ b/bench/records/t17-pairs-r020-r040.json @@ -0,0 +1,1234 @@ +[ + { + "arm": "t17-s1-6seg-r020", + "scene": "s1-static", + "moments": [ + "frame:62", + "frame:150", + "frame:277", + "frame:400", + "frame:583", + "frame:630" + ], + "source": { + "measures": { + "seam_jump_mean": 0.9279999999999999, + "seam_jump_max": 1.15, + "seam_look_mean": 1.128, + "seam_look_max": 1.98, + "seam_level_step_db_mean": 12.16, + "seam_level_step_db_max": 23.54, + "typical_delta_mean": 2.0, + "texture_fine_seg_last": 0.34562108548415904, + "texture_mid_seg_last": 0.11561955976957423, + "texture_coarse_seg_last": 0.009578898374631207, + "face_fine_seg_last": 0.08889803308949114 + }, + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 0.69, + "look_ratio": 1.98, + "level_step_db": 16.58, + "typical_delta": 2.0 + }, + { + "seconds": 9.417, + "jump_ratio": 1.15, + "look_ratio": 1.57, + "level_step_db": 5.73, + "typical_delta": 2.0 + }, + { + "seconds": 13.667, + "jump_ratio": 0.78, + "look_ratio": 1.16, + "level_step_db": 12.1, + "typical_delta": 2.0 + }, + { + "seconds": 17.917, + "jump_ratio": 0.98, + "look_ratio": 0.47, + "level_step_db": 23.54, + "typical_delta": 2.0 + }, + { + "seconds": 22.167, + "jump_ratio": 1.04, + "look_ratio": 0.46, + "level_step_db": 2.85, + "typical_delta": 2.0 + } + ], + "texture": { + "fine": [ + 0.0, + 0.0682604502009544, + 0.15402629479650914, + 0.21795821725171582, + 0.36683259200135493, + 0.34562108548415904 + ], + "mid": [ + 0.0, + 0.018229948807400177, + 0.04317731781727424, + 0.06745046935036213, + 0.12383734756733206, + 0.11561955976957423 + ], + "coarse": [ + 0.0, + 0.004915785068522211, + 0.004406177591744155, + 0.0037528632503382392, + 0.015490472469675862, + 0.009578898374631207 + ] + }, + "face_texture": { + "fine": [ + 0.0, + 0.0911822841364561, + 0.06895136823931643, + 0.10508655101415533, + 0.12309067426459207, + 0.08889803308949114 + ], + "mid": [ + 0.0, + 0.010738957920073698, + 0.04890830498578146, + 0.09406492537405498, + 0.14758993036246842, + 0.11416433457127233 + ], + "coarse": [ + 0.0, + 0.0011189672940962487, + 0.0066611233360087145, + 0.14310966049619053, + 0.11575207550746598, + 0.12307535818924704 + ] + }, + "motion_mean": 0.4278505742549896 + }, + "refined": { + "measures": { + "seam_jump_mean": 0.5660000000000001, + "seam_jump_max": 0.72, + "seam_look_mean": 1.114, + "seam_look_max": 1.41, + "seam_level_step_db_mean": 12.596, + "seam_level_step_db_max": 24.46, + "typical_delta_mean": 2.0, + "texture_fine_seg_last": 0.3505946654778319, + "texture_mid_seg_last": 0.10712199215061319, + "texture_coarse_seg_last": 0.004372988515923356, + "face_fine_seg_last": 0.09807011676333466 + }, + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 0.46, + "look_ratio": 1.18, + "level_step_db": 17.11, + "typical_delta": 2.0 + }, + { + "seconds": 9.417, + "jump_ratio": 0.72, + "look_ratio": 1.41, + "level_step_db": 6.19, + "typical_delta": 2.0 + }, + { + "seconds": 13.667, + "jump_ratio": 0.63, + "look_ratio": 1.41, + "level_step_db": 12.25, + "typical_delta": 2.0 + }, + { + "seconds": 17.917, + "jump_ratio": 0.57, + "look_ratio": 0.93, + "level_step_db": 24.46, + "typical_delta": 2.0 + }, + { + "seconds": 22.167, + "jump_ratio": 0.45, + "look_ratio": 0.64, + "level_step_db": 2.97, + "typical_delta": 2.0 + } + ], + "texture": { + "fine": [ + 0.0, + 0.03360807347780704, + 0.15551640170802516, + 0.23149543236010564, + 0.35256440378144854, + 0.3505946654778319 + ], + "mid": [ + 0.0, + 0.012499641932254408, + 0.036548063575358745, + 0.07036247898873542, + 0.09604934572356864, + 0.10712199215061319 + ], + "coarse": [ + 0.0, + 0.0009062013622760139, + 0.003773860764196791, + 0.0014217210244853627, + 0.0059431404832987625, + 0.004372988515923356 + ] + }, + "face_texture": { + "fine": [ + 0.0, + 0.04495477510583923, + 0.0836033363496605, + 0.12244175358462087, + 0.10645812703319281, + 0.09807011676333466 + ], + "mid": [ + 0.0, + 0.011266933390854423, + 0.07649304909027776, + 0.10337619730601896, + 0.168979048206681, + 0.146010279381115 + ], + "coarse": [ + 0.0, + 0.018745018764372186, + 0.0016182049253476102, + 0.13981800144171416, + 0.10749889766983212, + 0.12454553551694032 + ] + }, + "motion_mean": 0.35453760623931885 + }, + "fidelity": { + "psnr_db": [ + 32.07, + 32.4, + 31.33, + 30.86, + 29.35, + 29.57 + ], + "ssim": [ + 0.9233, + 0.933, + 0.9229, + 0.9197, + 0.9003, + 0.9017 + ] + } + }, + { + "arm": "t17-s2-r020", + "scene": "s2-arc", + "moments": [ + "frame:62", + "frame:150", + "frame:277", + "frame:400", + "frame:583", + "frame:630" + ], + "source": { + "measures": { + "seam_jump_mean": 1.3619999999999999, + "seam_jump_max": 2.09, + "seam_look_mean": 1.286, + "seam_look_max": 1.74, + "seam_level_step_db_mean": 1.4620000000000002, + "seam_level_step_db_max": 3.43, + "typical_delta_mean": 5.272, + "texture_fine_seg_last": 4.43180733926406, + "texture_mid_seg_last": 4.400673061097142, + "texture_coarse_seg_last": 4.575016765752062 + }, + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 2.09, + "look_ratio": 1.09, + "level_step_db": 1.46, + "typical_delta": 2.83 + }, + { + "seconds": 9.417, + "jump_ratio": 1.26, + "look_ratio": 1.74, + "level_step_db": 3.43, + "typical_delta": 3.46 + }, + { + "seconds": 13.667, + "jump_ratio": 1.17, + "look_ratio": 1.4, + "level_step_db": 0.23, + "typical_delta": 4.86 + }, + { + "seconds": 17.917, + "jump_ratio": 1.17, + "look_ratio": 1.33, + "level_step_db": 0.34, + "typical_delta": 6.43 + }, + { + "seconds": 22.167, + "jump_ratio": 1.12, + "look_ratio": 0.87, + "level_step_db": 1.85, + "typical_delta": 8.78 + } + ], + "texture": { + "fine": [ + 0.0, + 2.1200036818408092, + 0.9740211359406922, + 3.043823459255047, + 3.083963956401898, + 4.43180733926406 + ], + "mid": [ + 0.0, + 1.3167841633087687, + 0.32113990928962494, + 1.7593157783836975, + 2.2932110623147106, + 4.400673061097142 + ], + "coarse": [ + 0.0, + 0.6534541539488553, + 0.5154910691119463, + 0.9291353110536407, + 1.4297775601219735, + 4.575016765752062 + ] + }, + "face_texture": null, + "motion_mean": 4.160738945007324 + }, + "refined": { + "measures": { + "seam_jump_mean": 1.128, + "seam_jump_max": 1.67, + "seam_look_mean": 0.808, + "seam_look_max": 1.19, + "seam_level_step_db_mean": 1.462, + "seam_level_step_db_max": 3.42, + "typical_delta_mean": 5.41, + "texture_fine_seg_last": 4.143313309779288, + "texture_mid_seg_last": 4.032639236740412, + "texture_coarse_seg_last": 4.213076814845817 + }, + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 1.67, + "look_ratio": 0.62, + "level_step_db": 1.44, + "typical_delta": 2.89 + }, + { + "seconds": 9.417, + "jump_ratio": 0.98, + "look_ratio": 0.77, + "level_step_db": 3.42, + "typical_delta": 3.54 + }, + { + "seconds": 13.667, + "jump_ratio": 0.93, + "look_ratio": 0.62, + "level_step_db": 0.26, + "typical_delta": 4.93 + }, + { + "seconds": 17.917, + "jump_ratio": 1.03, + "look_ratio": 1.19, + "level_step_db": 0.33, + "typical_delta": 6.54 + }, + { + "seconds": 22.167, + "jump_ratio": 1.03, + "look_ratio": 0.84, + "level_step_db": 1.86, + "typical_delta": 9.15 + } + ], + "texture": { + "fine": [ + 0.0, + 2.0164890636979, + 0.9132300128907689, + 2.791478393937411, + 3.0162002858937327, + 4.143313309779288 + ], + "mid": [ + 0.0, + 1.2908667199213912, + 0.3041254199369465, + 1.6233942140517905, + 2.2264456849461434, + 4.032639236740412 + ], + "coarse": [ + 0.0, + 0.6742899323361063, + 0.4892133324357321, + 0.9079029880162403, + 1.426674661119028, + 4.213076814845817 + ] + }, + "face_texture": null, + "motion_mean": 4.21510124206543 + }, + "fidelity": { + "psnr_db": [ + 24.91, + 25.36, + 26.44, + 28.11, + 29.29, + 30.14 + ], + "ssim": [ + 0.7733, + 0.8017, + 0.8157, + 0.8416, + 0.9033, + 0.9077 + ] + } + }, + { + "arm": "t17-s3-r020", + "scene": "s3-dialogue-walk", + "moments": [ + "frame:62", + "frame:150", + "frame:277", + "frame:400", + "frame:583", + "frame:630" + ], + "source": { + "measures": { + "seam_jump_mean": 1.524, + "seam_jump_max": 1.83, + "seam_look_mean": 1.47, + "seam_look_max": 1.71, + "seam_level_step_db_mean": 2.968, + "seam_level_step_db_max": 8.81, + "typical_delta_mean": 5.206, + "texture_fine_seg_last": 0.030631958826816894, + "texture_mid_seg_last": 0.5106428593471353, + "texture_coarse_seg_last": 0.14468891120755345 + }, + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 1.83, + "look_ratio": 1.61, + "level_step_db": 8.81, + "typical_delta": 2.58 + }, + { + "seconds": 9.417, + "jump_ratio": 1.76, + "look_ratio": 1.52, + "level_step_db": 1.5, + "typical_delta": 3.65 + }, + { + "seconds": 13.667, + "jump_ratio": 1.75, + "look_ratio": 1.34, + "level_step_db": 1.3, + "typical_delta": 5.21 + }, + { + "seconds": 17.917, + "jump_ratio": 1.08, + "look_ratio": 1.71, + "level_step_db": 2.57, + "typical_delta": 6.54 + }, + { + "seconds": 22.167, + "jump_ratio": 1.2, + "look_ratio": 1.17, + "level_step_db": 0.66, + "typical_delta": 8.05 + } + ], + "texture": { + "fine": [ + 0.0, + 0.3798186863416516, + 0.5806663339162876, + 0.231486413508367, + 0.0674850955709072, + 0.030631958826816894 + ], + "mid": [ + 0.0, + 0.2448889387525672, + 0.3457022337825501, + 0.3758202724628086, + 0.11366149849198895, + 0.5106428593471353 + ], + "coarse": [ + 0.0, + 0.5099387711191702, + 0.6808654251173802, + 0.469307126866651, + 0.18802880506763223, + 0.14468891120755345 + ] + }, + "face_texture": null, + "motion_mean": 3.551892042160034 + }, + "refined": { + "measures": { + "seam_jump_mean": 1.192, + "seam_jump_max": 1.45, + "seam_look_mean": 1.242, + "seam_look_max": 1.57, + "seam_level_step_db_mean": 2.9720000000000004, + "seam_level_step_db_max": 8.81, + "typical_delta_mean": 5.21, + "texture_fine_seg_last": 0.1452697229695994, + "texture_mid_seg_last": 0.49015195638052206, + "texture_coarse_seg_last": 0.1491366830947877 + }, + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 1.14, + "look_ratio": 1.25, + "level_step_db": 8.81, + "typical_delta": 2.63 + }, + { + "seconds": 9.417, + "jump_ratio": 1.38, + "look_ratio": 1.41, + "level_step_db": 1.5, + "typical_delta": 3.72 + }, + { + "seconds": 13.667, + "jump_ratio": 1.45, + "look_ratio": 0.9, + "level_step_db": 1.31, + "typical_delta": 5.19 + }, + { + "seconds": 17.917, + "jump_ratio": 0.93, + "look_ratio": 1.57, + "level_step_db": 2.57, + "typical_delta": 6.52 + }, + { + "seconds": 22.167, + "jump_ratio": 1.06, + "look_ratio": 1.08, + "level_step_db": 0.67, + "typical_delta": 7.99 + } + ], + "texture": { + "fine": [ + 0.0, + 0.5174756648227867, + 0.47677831201633847, + 0.1921605336080509, + 0.04447145213208555, + 0.1452697229695994 + ], + "mid": [ + 0.0, + 0.26410064655154186, + 0.3501948337291438, + 0.3802398462213232, + 0.0812754012368564, + 0.49015195638052206 + ], + "coarse": [ + 0.0, + 0.49108208494089683, + 0.6708379878283188, + 0.45436070128589995, + 0.20292912897627677, + 0.1491366830947877 + ] + }, + "face_texture": null, + "motion_mean": 3.5468573570251465 + }, + "fidelity": { + "psnr_db": [ + 32.61, + 32.29, + 31.92, + 30.01, + 29.22, + 29.62 + ], + "ssim": [ + 0.955, + 0.9521, + 0.9493, + 0.9378, + 0.922, + 0.9252 + ] + } + }, + { + "arm": "t17-s1-6seg-r040", + "scene": "s1-static", + "moments": [ + "frame:62", + "frame:150", + "frame:277", + "frame:400", + "frame:583", + "frame:630" + ], + "source": { + "measures": { + "seam_jump_mean": 0.9279999999999999, + "seam_jump_max": 1.15, + "seam_look_mean": 1.128, + "seam_look_max": 1.98, + "seam_level_step_db_mean": 12.16, + "seam_level_step_db_max": 23.54, + "typical_delta_mean": 2.0, + "texture_fine_seg_last": 0.34562108548415904, + "texture_mid_seg_last": 0.11561955976957423, + "texture_coarse_seg_last": 0.009578898374631207, + "face_fine_seg_last": 0.08889803308949114 + }, + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 0.69, + "look_ratio": 1.98, + "level_step_db": 16.58, + "typical_delta": 2.0 + }, + { + "seconds": 9.417, + "jump_ratio": 1.15, + "look_ratio": 1.57, + "level_step_db": 5.73, + "typical_delta": 2.0 + }, + { + "seconds": 13.667, + "jump_ratio": 0.78, + "look_ratio": 1.16, + "level_step_db": 12.1, + "typical_delta": 2.0 + }, + { + "seconds": 17.917, + "jump_ratio": 0.98, + "look_ratio": 0.47, + "level_step_db": 23.54, + "typical_delta": 2.0 + }, + { + "seconds": 22.167, + "jump_ratio": 1.04, + "look_ratio": 0.46, + "level_step_db": 2.85, + "typical_delta": 2.0 + } + ], + "texture": { + "fine": [ + 0.0, + 0.0682604502009544, + 0.15402629479650914, + 0.21795821725171582, + 0.36683259200135493, + 0.34562108548415904 + ], + "mid": [ + 0.0, + 0.018229948807400177, + 0.04317731781727424, + 0.06745046935036213, + 0.12383734756733206, + 0.11561955976957423 + ], + "coarse": [ + 0.0, + 0.004915785068522211, + 0.004406177591744155, + 0.0037528632503382392, + 0.015490472469675862, + 0.009578898374631207 + ] + }, + "face_texture": { + "fine": [ + 0.0, + 0.0911822841364561, + 0.06895136823931643, + 0.10508655101415533, + 0.12309067426459207, + 0.08889803308949114 + ], + "mid": [ + 0.0, + 0.010738957920073698, + 0.04890830498578146, + 0.09406492537405498, + 0.14758993036246842, + 0.11416433457127233 + ], + "coarse": [ + 0.0, + 0.0011189672940962487, + 0.0066611233360087145, + 0.14310966049619053, + 0.11575207550746598, + 0.12307535818924704 + ] + }, + "motion_mean": 0.4278505742549896 + }, + "refined": { + "measures": { + "seam_jump_mean": 0.6020000000000001, + "seam_jump_max": 1.0, + "seam_look_mean": 1.372, + "seam_look_max": 1.95, + "seam_level_step_db_mean": 12.596, + "seam_level_step_db_max": 24.46, + "typical_delta_mean": 2.0, + "texture_fine_seg_last": 0.3676934381983235, + "texture_mid_seg_last": 0.1270435175315469, + "texture_coarse_seg_last": 0.006030007294151427, + "face_fine_seg_last": 0.11991260509297391 + }, + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 0.46, + "look_ratio": 1.74, + "level_step_db": 17.11, + "typical_delta": 2.0 + }, + { + "seconds": 9.417, + "jump_ratio": 1.0, + "look_ratio": 1.95, + "level_step_db": 6.19, + "typical_delta": 2.0 + }, + { + "seconds": 13.667, + "jump_ratio": 0.4, + "look_ratio": 1.05, + "level_step_db": 12.25, + "typical_delta": 2.0 + }, + { + "seconds": 17.917, + "jump_ratio": 0.77, + "look_ratio": 1.45, + "level_step_db": 24.46, + "typical_delta": 2.0 + }, + { + "seconds": 22.167, + "jump_ratio": 0.38, + "look_ratio": 0.67, + "level_step_db": 2.97, + "typical_delta": 2.0 + } + ], + "texture": { + "fine": [ + 0.0, + 0.013772707888498938, + 0.14603978004849835, + 0.22798035522068777, + 0.37655974036298967, + 0.3676934381983235 + ], + "mid": [ + 0.0, + 0.0011628305971872817, + 0.04026390301436445, + 0.06817854142466896, + 0.1152078400086177, + 0.1270435175315469 + ], + "coarse": [ + 0.0, + 0.008724552300970129, + 0.0039024930748447276, + 0.0025059794002034, + 0.005066629404988947, + 0.006030007294151427 + ] + }, + "face_texture": { + "fine": [ + 0.0, + 0.047948038554131996, + 0.08137167350815208, + 0.15165268980926655, + 0.14438622317462196, + 0.11991260509297391 + ], + "mid": [ + 0.0, + 0.0144130991714707, + 0.08808082289137661, + 0.11583325754047724, + 0.16026715151340595, + 0.1341581598150767 + ], + "coarse": [ + 0.0, + 0.017385680541063944, + 0.0034413402681442553, + 0.1274063380072501, + 0.09792415000318012, + 0.10893422107705895 + ] + }, + "motion_mean": 0.3532533645629883 + }, + "fidelity": { + "psnr_db": [ + 30.07, + 30.48, + 29.26, + 28.69, + 26.78, + 26.79 + ], + "ssim": [ + 0.8941, + 0.9051, + 0.8838, + 0.8797, + 0.8377, + 0.8353 + ] + } + }, + { + "arm": "t17-s2-r040", + "scene": "s2-arc", + "moments": [ + "frame:62", + "frame:150", + "frame:277", + "frame:400", + "frame:583", + "frame:630" + ], + "source": { + "measures": { + "seam_jump_mean": 1.3619999999999999, + "seam_jump_max": 2.09, + "seam_look_mean": 1.286, + "seam_look_max": 1.74, + "seam_level_step_db_mean": 1.4620000000000002, + "seam_level_step_db_max": 3.43, + "typical_delta_mean": 5.272, + "texture_fine_seg_last": 4.43180733926406, + "texture_mid_seg_last": 4.400673061097142, + "texture_coarse_seg_last": 4.575016765752062 + }, + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 2.09, + "look_ratio": 1.09, + "level_step_db": 1.46, + "typical_delta": 2.83 + }, + { + "seconds": 9.417, + "jump_ratio": 1.26, + "look_ratio": 1.74, + "level_step_db": 3.43, + "typical_delta": 3.46 + }, + { + "seconds": 13.667, + "jump_ratio": 1.17, + "look_ratio": 1.4, + "level_step_db": 0.23, + "typical_delta": 4.86 + }, + { + "seconds": 17.917, + "jump_ratio": 1.17, + "look_ratio": 1.33, + "level_step_db": 0.34, + "typical_delta": 6.43 + }, + { + "seconds": 22.167, + "jump_ratio": 1.12, + "look_ratio": 0.87, + "level_step_db": 1.85, + "typical_delta": 8.78 + } + ], + "texture": { + "fine": [ + 0.0, + 2.1200036818408092, + 0.9740211359406922, + 3.043823459255047, + 3.083963956401898, + 4.43180733926406 + ], + "mid": [ + 0.0, + 1.3167841633087687, + 0.32113990928962494, + 1.7593157783836975, + 2.2932110623147106, + 4.400673061097142 + ], + "coarse": [ + 0.0, + 0.6534541539488553, + 0.5154910691119463, + 0.9291353110536407, + 1.4297775601219735, + 4.575016765752062 + ] + }, + "face_texture": null, + "motion_mean": 4.160738945007324 + }, + "refined": { + "measures": { + "seam_jump_mean": 1.1019999999999999, + "seam_jump_max": 1.65, + "seam_look_mean": 0.72, + "seam_look_max": 1.12, + "seam_level_step_db_mean": 1.462, + "seam_level_step_db_max": 3.42, + "typical_delta_mean": 5.576, + "texture_fine_seg_last": 4.095390286997235, + "texture_mid_seg_last": 4.015510668414532, + "texture_coarse_seg_last": 4.204155570843362 + }, + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 1.65, + "look_ratio": 0.56, + "level_step_db": 1.44, + "typical_delta": 3.01 + }, + { + "seconds": 9.417, + "jump_ratio": 0.96, + "look_ratio": 0.7, + "level_step_db": 3.42, + "typical_delta": 3.74 + }, + { + "seconds": 13.667, + "jump_ratio": 0.9, + "look_ratio": 0.38, + "level_step_db": 0.26, + "typical_delta": 5.12 + }, + { + "seconds": 17.917, + "jump_ratio": 0.97, + "look_ratio": 1.12, + "level_step_db": 0.33, + "typical_delta": 6.81 + }, + { + "seconds": 22.167, + "jump_ratio": 1.03, + "look_ratio": 0.84, + "level_step_db": 1.86, + "typical_delta": 9.2 + } + ], + "texture": { + "fine": [ + 0.0, + 1.9784745234060015, + 0.9677885890984657, + 2.6650959625807427, + 2.9799733360526983, + 4.095390286997235 + ], + "mid": [ + 0.0, + 1.2696655960269991, + 0.3438070282397562, + 1.634759090252645, + 2.302248377445693, + 4.015510668414532 + ], + "coarse": [ + 0.0, + 0.6665339856599362, + 0.4723184800067063, + 0.9048353737891391, + 1.451450607854744, + 4.204155570843362 + ] + }, + "face_texture": null, + "motion_mean": 4.298374176025391 + }, + "fidelity": { + "psnr_db": [ + 23.24, + 23.68, + 24.8, + 26.57, + 25.98, + 26.67 + ], + "ssim": [ + 0.7093, + 0.7424, + 0.7612, + 0.8064, + 0.8191, + 0.8265 + ] + } + }, + { + "arm": "t17-s3-r040", + "scene": "s3-dialogue-walk", + "moments": [ + "frame:62", + "frame:150", + "frame:277", + "frame:400", + "frame:583", + "frame:630" + ], + "source": { + "measures": { + "seam_jump_mean": 1.524, + "seam_jump_max": 1.83, + "seam_look_mean": 1.47, + "seam_look_max": 1.71, + "seam_level_step_db_mean": 2.968, + "seam_level_step_db_max": 8.81, + "typical_delta_mean": 5.206, + "texture_fine_seg_last": 0.030631958826816894, + "texture_mid_seg_last": 0.5106428593471353, + "texture_coarse_seg_last": 0.14468891120755345 + }, + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 1.83, + "look_ratio": 1.61, + "level_step_db": 8.81, + "typical_delta": 2.58 + }, + { + "seconds": 9.417, + "jump_ratio": 1.76, + "look_ratio": 1.52, + "level_step_db": 1.5, + "typical_delta": 3.65 + }, + { + "seconds": 13.667, + "jump_ratio": 1.75, + "look_ratio": 1.34, + "level_step_db": 1.3, + "typical_delta": 5.21 + }, + { + "seconds": 17.917, + "jump_ratio": 1.08, + "look_ratio": 1.71, + "level_step_db": 2.57, + "typical_delta": 6.54 + }, + { + "seconds": 22.167, + "jump_ratio": 1.2, + "look_ratio": 1.17, + "level_step_db": 0.66, + "typical_delta": 8.05 + } + ], + "texture": { + "fine": [ + 0.0, + 0.3798186863416516, + 0.5806663339162876, + 0.231486413508367, + 0.0674850955709072, + 0.030631958826816894 + ], + "mid": [ + 0.0, + 0.2448889387525672, + 0.3457022337825501, + 0.3758202724628086, + 0.11366149849198895, + 0.5106428593471353 + ], + "coarse": [ + 0.0, + 0.5099387711191702, + 0.6808654251173802, + 0.469307126866651, + 0.18802880506763223, + 0.14468891120755345 + ] + }, + "face_texture": null, + "motion_mean": 3.551892042160034 + }, + "refined": { + "measures": { + "seam_jump_mean": 1.0, + "seam_jump_max": 1.2, + "seam_look_mean": 1.342, + "seam_look_max": 1.7, + "seam_level_step_db_mean": 2.9720000000000004, + "seam_level_step_db_max": 8.81, + "typical_delta_mean": 5.481999999999999, + "texture_fine_seg_last": 0.10888236173608194, + "texture_mid_seg_last": 0.49068868517426795, + "texture_coarse_seg_last": 0.16440292063116896 + }, + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 0.74, + "look_ratio": 1.7, + "level_step_db": 8.81, + "typical_delta": 2.78 + }, + { + "seconds": 9.417, + "jump_ratio": 1.04, + "look_ratio": 1.36, + "level_step_db": 1.5, + "typical_delta": 4.14 + }, + { + "seconds": 13.667, + "jump_ratio": 1.2, + "look_ratio": 0.9, + "level_step_db": 1.31, + "typical_delta": 5.56 + }, + { + "seconds": 17.917, + "jump_ratio": 0.94, + "look_ratio": 1.51, + "level_step_db": 2.57, + "typical_delta": 6.56 + }, + { + "seconds": 22.167, + "jump_ratio": 1.08, + "look_ratio": 1.24, + "level_step_db": 0.67, + "typical_delta": 8.37 + } + ], + "texture": { + "fine": [ + 0.0, + 0.46194392057690653, + 0.49427073497981777, + 0.21389699812171992, + 0.08554147933952622, + 0.10888236173608194 + ], + "mid": [ + 0.0, + 0.26741453489726086, + 0.357855748989471, + 0.39007396813966494, + 0.10679690984897616, + 0.49068868517426795 + ], + "coarse": [ + 0.0, + 0.4842478598841573, + 0.66330449466287, + 0.4538780445233992, + 0.1989830034604991, + 0.16440292063116896 + ] + }, + "face_texture": null, + "motion_mean": 3.682527780532837 + }, + "fidelity": { + "psnr_db": [ + 30.24, + 29.85, + 29.33, + 27.52, + 27.13, + 26.81 + ], + "ssim": [ + 0.9326, + 0.9236, + 0.912, + 0.9074, + 0.8683, + 0.8666 + ] + } + } +] \ No newline at end of file diff --git a/bench/records/t17-pairs-r030.json b/bench/records/t17-pairs-r030.json new file mode 100644 index 000000000..9d70745a2 --- /dev/null +++ b/bench/records/t17-pairs-r030.json @@ -0,0 +1,858 @@ +[ + { + "arm": "t17-s1-6seg-r030", + "scene": "s1-static", + "moments": [ + "frame:62", + "frame:150", + "frame:277", + "frame:400", + "frame:583", + "frame:630" + ], + "source": { + "measures": { + "seam_jump_mean": 0.9279999999999999, + "seam_jump_max": 1.15, + "seam_look_mean": 1.128, + "seam_look_max": 1.98, + "seam_level_step_db_mean": 12.16, + "seam_level_step_db_max": 23.54, + "typical_delta_mean": 2.0, + "texture_fine_seg_last": 0.34562108548415904, + "texture_mid_seg_last": 0.11561955976957423, + "texture_coarse_seg_last": 0.009578898374631207, + "face_fine_seg_last": 0.08889803308949114 + }, + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 0.69, + "look_ratio": 1.98, + "level_step_db": 16.58, + "typical_delta": 2.0 + }, + { + "seconds": 9.417, + "jump_ratio": 1.15, + "look_ratio": 1.57, + "level_step_db": 5.73, + "typical_delta": 2.0 + }, + { + "seconds": 13.667, + "jump_ratio": 0.78, + "look_ratio": 1.16, + "level_step_db": 12.1, + "typical_delta": 2.0 + }, + { + "seconds": 17.917, + "jump_ratio": 0.98, + "look_ratio": 0.47, + "level_step_db": 23.54, + "typical_delta": 2.0 + }, + { + "seconds": 22.167, + "jump_ratio": 1.04, + "look_ratio": 0.46, + "level_step_db": 2.85, + "typical_delta": 2.0 + } + ], + "texture": { + "fine": [ + 0.0, + 0.0682604502009544, + 0.15402629479650914, + 0.21795821725171582, + 0.36683259200135493, + 0.34562108548415904 + ], + "mid": [ + 0.0, + 0.018229948807400177, + 0.04317731781727424, + 0.06745046935036213, + 0.12383734756733206, + 0.11561955976957423 + ], + "coarse": [ + 0.0, + 0.004915785068522211, + 0.004406177591744155, + 0.0037528632503382392, + 0.015490472469675862, + 0.009578898374631207 + ] + }, + "face_texture": { + "fine": [ + 0.0, + 0.0911822841364561, + 0.06895136823931643, + 0.10508655101415533, + 0.12309067426459207, + 0.08889803308949114 + ], + "mid": [ + 0.0, + 0.010738957920073698, + 0.04890830498578146, + 0.09406492537405498, + 0.14758993036246842, + 0.11416433457127233 + ], + "coarse": [ + 0.0, + 0.0011189672940962487, + 0.0066611233360087145, + 0.14310966049619053, + 0.11575207550746598, + 0.12307535818924704 + ] + }, + "motion_mean": 0.4278505742549896 + }, + "refined": { + "measures": { + "seam_jump_mean": 0.5860000000000001, + "seam_jump_max": 0.87, + "seam_look_mean": 1.238, + "seam_look_max": 1.64, + "seam_level_step_db_mean": 12.596, + "seam_level_step_db_max": 24.46, + "typical_delta_mean": 2.0, + "texture_fine_seg_last": 0.3662316014232925, + "texture_mid_seg_last": 0.11495953395175076, + "texture_coarse_seg_last": 0.0002472652023458986, + "face_fine_seg_last": 0.13271563016166085 + }, + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 0.45, + "look_ratio": 1.53, + "level_step_db": 17.11, + "typical_delta": 2.0 + }, + { + "seconds": 9.417, + "jump_ratio": 0.87, + "look_ratio": 1.64, + "level_step_db": 6.19, + "typical_delta": 2.0 + }, + { + "seconds": 13.667, + "jump_ratio": 0.53, + "look_ratio": 1.32, + "level_step_db": 12.25, + "typical_delta": 2.0 + }, + { + "seconds": 17.917, + "jump_ratio": 0.7, + "look_ratio": 1.23, + "level_step_db": 24.46, + "typical_delta": 2.0 + }, + { + "seconds": 22.167, + "jump_ratio": 0.38, + "look_ratio": 0.47, + "level_step_db": 2.97, + "typical_delta": 2.0 + } + ], + "texture": { + "fine": [ + 0.0, + 0.017689476956483752, + 0.14538825302987898, + 0.23305251939709512, + 0.3519029365255072, + 0.3662316014232925 + ], + "mid": [ + 0.0, + 0.003500847836085017, + 0.03587907682150172, + 0.06181587280512738, + 0.10411610628999082, + 0.11495953395175076 + ], + "coarse": [ + 0.0, + 0.004414167352820047, + 0.0010662030685258948, + 0.0029207141431290768, + 0.0009038292080737304, + 0.0002472652023458986 + ] + }, + "face_texture": { + "fine": [ + 0.0, + 0.07862961129494148, + 0.04638182036647024, + 0.15125445773632895, + 0.1404832530751724, + 0.13271563016166085 + ], + "mid": [ + 0.0, + 0.021879543417044695, + 0.094729130579652, + 0.11965771074108487, + 0.17597651267227812, + 0.14890594685344472 + ], + "coarse": [ + 0.0, + 0.019418958550583624, + 0.0004922527011668811, + 0.13574207707879732, + 0.1021244503847628, + 0.11521374639618397 + ] + }, + "motion_mean": 0.349222332239151 + }, + "fidelity": { + "psnr_db": [ + 31.11, + 31.41, + 30.31, + 29.72, + 28.05, + 28.18 + ], + "ssim": [ + 0.9109, + 0.9205, + 0.906, + 0.9012, + 0.8737, + 0.8736 + ] + } + }, + { + "arm": "t17-s1-6seg-r030-rebased", + "scene": "s1-static", + "moments": [ + "frame:62", + "frame:150", + "frame:277", + "frame:400", + "frame:583", + "frame:630" + ], + "source": { + "measures": { + "seam_jump_mean": 0.9279999999999999, + "seam_jump_max": 1.15, + "seam_look_mean": 1.128, + "seam_look_max": 1.98, + "seam_level_step_db_mean": 12.16, + "seam_level_step_db_max": 23.54, + "typical_delta_mean": 2.0, + "texture_fine_seg_last": 0.34562108548415904, + "texture_mid_seg_last": 0.11561955976957423, + "texture_coarse_seg_last": 0.009578898374631207, + "face_fine_seg_last": 0.08889803308949114 + }, + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 0.69, + "look_ratio": 1.98, + "level_step_db": 16.58, + "typical_delta": 2.0 + }, + { + "seconds": 9.417, + "jump_ratio": 1.15, + "look_ratio": 1.57, + "level_step_db": 5.73, + "typical_delta": 2.0 + }, + { + "seconds": 13.667, + "jump_ratio": 0.78, + "look_ratio": 1.16, + "level_step_db": 12.1, + "typical_delta": 2.0 + }, + { + "seconds": 17.917, + "jump_ratio": 0.98, + "look_ratio": 0.47, + "level_step_db": 23.54, + "typical_delta": 2.0 + }, + { + "seconds": 22.167, + "jump_ratio": 1.04, + "look_ratio": 0.46, + "level_step_db": 2.85, + "typical_delta": 2.0 + } + ], + "texture": { + "fine": [ + 0.0, + 0.0682604502009544, + 0.15402629479650914, + 0.21795821725171582, + 0.36683259200135493, + 0.34562108548415904 + ], + "mid": [ + 0.0, + 0.018229948807400177, + 0.04317731781727424, + 0.06745046935036213, + 0.12383734756733206, + 0.11561955976957423 + ], + "coarse": [ + 0.0, + 0.004915785068522211, + 0.004406177591744155, + 0.0037528632503382392, + 0.015490472469675862, + 0.009578898374631207 + ] + }, + "face_texture": { + "fine": [ + 0.0, + 0.0911822841364561, + 0.06895136823931643, + 0.10508655101415533, + 0.12309067426459207, + 0.08889803308949114 + ], + "mid": [ + 0.0, + 0.010738957920073698, + 0.04890830498578146, + 0.09406492537405498, + 0.14758993036246842, + 0.11416433457127233 + ], + "coarse": [ + 0.0, + 0.0011189672940962487, + 0.0066611233360087145, + 0.14310966049619053, + 0.11575207550746598, + 0.12307535818924704 + ] + }, + "motion_mean": 0.4278505742549896 + }, + "refined": { + "measures": { + "seam_jump_mean": 0.516, + "seam_jump_max": 0.71, + "seam_look_mean": 1.014, + "seam_look_max": 1.21, + "seam_level_step_db_mean": 12.596, + "seam_level_step_db_max": 24.46, + "typical_delta_mean": 2.0, + "texture_fine_seg_last": 0.3805143616067333, + "texture_mid_seg_last": 0.11453825175529164, + "texture_coarse_seg_last": 0.00273464621758138, + "face_fine_seg_last": 0.12620476484549062 + }, + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 0.37, + "look_ratio": 0.94, + "level_step_db": 17.11, + "typical_delta": 2.0 + }, + { + "seconds": 9.417, + "jump_ratio": 0.71, + "look_ratio": 1.21, + "level_step_db": 6.19, + "typical_delta": 2.0 + }, + { + "seconds": 13.667, + "jump_ratio": 0.52, + "look_ratio": 1.05, + "level_step_db": 12.25, + "typical_delta": 2.0 + }, + { + "seconds": 17.917, + "jump_ratio": 0.62, + "look_ratio": 1.07, + "level_step_db": 24.46, + "typical_delta": 2.0 + }, + { + "seconds": 22.167, + "jump_ratio": 0.36, + "look_ratio": 0.8, + "level_step_db": 2.97, + "typical_delta": 2.0 + } + ], + "texture": { + "fine": [ + 0.0, + 0.029147453835544037, + 0.16030195585375323, + 0.23426174933987226, + 0.39562236699887765, + 0.3805143616067333 + ], + "mid": [ + 0.0, + 0.009345444668279789, + 0.028490364407945046, + 0.05732728478456865, + 0.11508029391233984, + 0.11453825175529164 + ], + "coarse": [ + 0.0, + 0.0031495007665896303, + 0.001926919238704877, + 0.0019260600424962787, + 0.0028889882627052277, + 0.00273464621758138 + ] + }, + "face_texture": { + "fine": [ + 0.0, + 0.06297208911536252, + 0.08362875183330958, + 0.15761950852121812, + 0.1542686579862127, + 0.12620476484549062 + ], + "mid": [ + 0.0, + 0.016301240634308264, + 0.0824879628592204, + 0.11677265472203373, + 0.17867653691458124, + 0.14821230853372808 + ], + "coarse": [ + 0.0, + 0.01200630955332596, + 0.010201298621587843, + 0.14029169296725805, + 0.10442561374421856, + 0.11763289738278225 + ] + }, + "motion_mean": 0.3484481871128082 + }, + "fidelity": { + "psnr_db": [ + 31.2, + 31.39, + 30.27, + 29.8, + 28.19, + 28.21 + ], + "ssim": [ + 0.9113, + 0.9209, + 0.9053, + 0.8993, + 0.8724, + 0.8744 + ] + } + }, + { + "arm": "t17-s2-r030", + "scene": "s2-arc", + "moments": [ + "frame:62", + "frame:150", + "frame:277", + "frame:400", + "frame:583", + "frame:630" + ], + "source": { + "measures": { + "seam_jump_mean": 1.3619999999999999, + "seam_jump_max": 2.09, + "seam_look_mean": 1.286, + "seam_look_max": 1.74, + "seam_level_step_db_mean": 1.4620000000000002, + "seam_level_step_db_max": 3.43, + "typical_delta_mean": 5.272, + "texture_fine_seg_last": 4.43180733926406, + "texture_mid_seg_last": 4.400673061097142, + "texture_coarse_seg_last": 4.575016765752062 + }, + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 2.09, + "look_ratio": 1.09, + "level_step_db": 1.46, + "typical_delta": 2.83 + }, + { + "seconds": 9.417, + "jump_ratio": 1.26, + "look_ratio": 1.74, + "level_step_db": 3.43, + "typical_delta": 3.46 + }, + { + "seconds": 13.667, + "jump_ratio": 1.17, + "look_ratio": 1.4, + "level_step_db": 0.23, + "typical_delta": 4.86 + }, + { + "seconds": 17.917, + "jump_ratio": 1.17, + "look_ratio": 1.33, + "level_step_db": 0.34, + "typical_delta": 6.43 + }, + { + "seconds": 22.167, + "jump_ratio": 1.12, + "look_ratio": 0.87, + "level_step_db": 1.85, + "typical_delta": 8.78 + } + ], + "texture": { + "fine": [ + 0.0, + 2.1200036818408092, + 0.9740211359406922, + 3.043823459255047, + 3.083963956401898, + 4.43180733926406 + ], + "mid": [ + 0.0, + 1.3167841633087687, + 0.32113990928962494, + 1.7593157783836975, + 2.2932110623147106, + 4.400673061097142 + ], + "coarse": [ + 0.0, + 0.6534541539488553, + 0.5154910691119463, + 0.9291353110536407, + 1.4297775601219735, + 4.575016765752062 + ] + }, + "face_texture": null, + "motion_mean": 4.160738945007324 + }, + "refined": { + "measures": { + "seam_jump_mean": 1.114, + "seam_jump_max": 1.64, + "seam_look_mean": 0.766, + "seam_look_max": 1.14, + "seam_level_step_db_mean": 1.462, + "seam_level_step_db_max": 3.42, + "typical_delta_mean": 5.5120000000000005, + "texture_fine_seg_last": 4.258676215859724, + "texture_mid_seg_last": 4.193007824412465, + "texture_coarse_seg_last": 4.211248608873569 + }, + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 1.64, + "look_ratio": 0.54, + "level_step_db": 1.44, + "typical_delta": 2.94 + }, + { + "seconds": 9.417, + "jump_ratio": 1.0, + "look_ratio": 0.77, + "level_step_db": 3.42, + "typical_delta": 3.66 + }, + { + "seconds": 13.667, + "jump_ratio": 0.88, + "look_ratio": 0.51, + "level_step_db": 0.26, + "typical_delta": 5.08 + }, + { + "seconds": 17.917, + "jump_ratio": 1.01, + "look_ratio": 1.14, + "level_step_db": 0.33, + "typical_delta": 6.66 + }, + { + "seconds": 22.167, + "jump_ratio": 1.04, + "look_ratio": 0.87, + "level_step_db": 1.86, + "typical_delta": 9.22 + } + ], + "texture": { + "fine": [ + 0.0, + 1.959873832010016, + 0.9667702337213963, + 2.7793147378942535, + 2.971672152371171, + 4.258676215859724 + ], + "mid": [ + 0.0, + 1.2910720984570978, + 0.3281613432424549, + 1.577067310745994, + 2.2584607637888654, + 4.193007824412465 + ], + "coarse": [ + 0.0, + 0.6773494939917842, + 0.47426807366662005, + 0.9191689261844561, + 1.409297712685815, + 4.211248608873569 + ] + }, + "face_texture": null, + "motion_mean": 4.26358699798584 + }, + "fidelity": { + "psnr_db": [ + 24.07, + 24.54, + 25.65, + 27.35, + 27.11, + 27.83 + ], + "ssim": [ + 0.7425, + 0.7732, + 0.7908, + 0.8236, + 0.8504, + 0.8512 + ] + } + }, + { + "arm": "t17-s3-r030", + "scene": "s3-dialogue-walk", + "moments": [ + "frame:62", + "frame:150", + "frame:277", + "frame:400", + "frame:583", + "frame:630" + ], + "source": { + "measures": { + "seam_jump_mean": 1.524, + "seam_jump_max": 1.83, + "seam_look_mean": 1.47, + "seam_look_max": 1.71, + "seam_level_step_db_mean": 2.968, + "seam_level_step_db_max": 8.81, + "typical_delta_mean": 5.206, + "texture_fine_seg_last": 0.030631958826816894, + "texture_mid_seg_last": 0.5106428593471353, + "texture_coarse_seg_last": 0.14468891120755345 + }, + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 1.83, + "look_ratio": 1.61, + "level_step_db": 8.81, + "typical_delta": 2.58 + }, + { + "seconds": 9.417, + "jump_ratio": 1.76, + "look_ratio": 1.52, + "level_step_db": 1.5, + "typical_delta": 3.65 + }, + { + "seconds": 13.667, + "jump_ratio": 1.75, + "look_ratio": 1.34, + "level_step_db": 1.3, + "typical_delta": 5.21 + }, + { + "seconds": 17.917, + "jump_ratio": 1.08, + "look_ratio": 1.71, + "level_step_db": 2.57, + "typical_delta": 6.54 + }, + { + "seconds": 22.167, + "jump_ratio": 1.2, + "look_ratio": 1.17, + "level_step_db": 0.66, + "typical_delta": 8.05 + } + ], + "texture": { + "fine": [ + 0.0, + 0.3798186863416516, + 0.5806663339162876, + 0.231486413508367, + 0.0674850955709072, + 0.030631958826816894 + ], + "mid": [ + 0.0, + 0.2448889387525672, + 0.3457022337825501, + 0.3758202724628086, + 0.11366149849198895, + 0.5106428593471353 + ], + "coarse": [ + 0.0, + 0.5099387711191702, + 0.6808654251173802, + 0.469307126866651, + 0.18802880506763223, + 0.14468891120755345 + ] + }, + "face_texture": null, + "motion_mean": 3.551892042160034 + }, + "refined": { + "measures": { + "seam_jump_mean": 1.1, + "seam_jump_max": 1.36, + "seam_look_mean": 1.324, + "seam_look_max": 1.62, + "seam_level_step_db_mean": 2.9720000000000004, + "seam_level_step_db_max": 8.81, + "typical_delta_mean": 5.328, + "texture_fine_seg_last": 0.12875677061184004, + "texture_mid_seg_last": 0.4948345734065851, + "texture_coarse_seg_last": 0.1536373447715356 + }, + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 0.87, + "look_ratio": 1.47, + "level_step_db": 8.81, + "typical_delta": 2.85 + }, + { + "seconds": 9.417, + "jump_ratio": 1.26, + "look_ratio": 1.38, + "level_step_db": 1.5, + "typical_delta": 3.79 + }, + { + "seconds": 13.667, + "jump_ratio": 1.36, + "look_ratio": 0.95, + "level_step_db": 1.31, + "typical_delta": 5.28 + }, + { + "seconds": 17.917, + "jump_ratio": 0.92, + "look_ratio": 1.62, + "level_step_db": 2.57, + "typical_delta": 6.58 + }, + { + "seconds": 22.167, + "jump_ratio": 1.09, + "look_ratio": 1.2, + "level_step_db": 0.67, + "typical_delta": 8.14 + } + ], + "texture": { + "fine": [ + 0.0, + 0.48679982979087044, + 0.48082360147528797, + 0.20331127731242826, + 0.005404809914179108, + 0.12875677061184004 + ], + "mid": [ + 0.0, + 0.26507495901022093, + 0.3546340433307082, + 0.38450685089334935, + 0.08959540131581598, + 0.4948345734065851 + ], + "coarse": [ + 0.0, + 0.4849980713966975, + 0.666584768181334, + 0.4540427109385927, + 0.20342060732265638, + 0.1536373447715356 + ] + }, + "face_texture": null, + "motion_mean": 3.618138551712036 + }, + "fidelity": { + "psnr_db": [ + 31.55, + 31.05, + 30.62, + 28.67, + 28.06, + 28.04 + ], + "ssim": [ + 0.9468, + 0.9394, + 0.9319, + 0.9228, + 0.8954, + 0.8955 + ] + } + } +] \ No newline at end of file diff --git a/bench/records/t17-refine-120s.json b/bench/records/t17-refine-120s.json new file mode 100644 index 000000000..6ee5ec212 --- /dev/null +++ b/bench/records/t17-refine-120s.json @@ -0,0 +1,11 @@ +[ + { + "arm": "t17-s1-120s-r030-rebased", + "scene": "s1-static", + "seed": 1166, + "job_id": "2f6c0073d2f0", + "status": "succeeded", + "error": null, + "output": "BenchRefineWindows/20261105-125907-13420dda/final/BenchRefineWindows-mux.3-0.0.mp4" + } +] \ No newline at end of file diff --git a/bench/records/t17-refine-r020-r040.json b/bench/records/t17-refine-r020-r040.json new file mode 100644 index 000000000..06b525ff6 --- /dev/null +++ b/bench/records/t17-refine-r020-r040.json @@ -0,0 +1,56 @@ +[ + { + "arm": "t17-s1-6seg-r020", + "scene": "s1-static", + "seed": 1166, + "job_id": "2bf137970ed6", + "status": "succeeded", + "error": null, + "output": "BenchRefineWindows/20261105-105022-10306151/final/BenchRefineWindows-mux.3-0.0.mp4" + }, + { + "arm": "t17-s2-r020", + "scene": "s2-arc", + "seed": 1166, + "job_id": "f632dc4da8b0", + "status": "succeeded", + "error": null, + "output": "BenchRefineWindows/20261105-110808-51bf7745/final/BenchRefineWindows-mux.3-0.0.mp4" + }, + { + "arm": "t17-s3-r020", + "scene": "s3-dialogue-walk", + "seed": 1166, + "job_id": "7e896973847a", + "status": "succeeded", + "error": null, + "output": "BenchRefineWindows/20261105-112555-69142e2e/final/BenchRefineWindows-mux.3-0.0.mp4" + }, + { + "arm": "t17-s1-6seg-r040", + "scene": "s1-static", + "seed": 1166, + "job_id": "54465529d877", + "status": "succeeded", + "error": null, + "output": "BenchRefineWindows/20261105-114345-ccefdb91/final/BenchRefineWindows-mux.3-0.0.mp4" + }, + { + "arm": "t17-s2-r040", + "scene": "s2-arc", + "seed": 1166, + "job_id": "c5885e47a386", + "status": "succeeded", + "error": null, + "output": "BenchRefineWindows/20261105-120133-e1907f67/final/BenchRefineWindows-mux.3-0.0.mp4" + }, + { + "arm": "t17-s3-r040", + "scene": "s3-dialogue-walk", + "seed": 1166, + "job_id": "8327447967bb", + "status": "succeeded", + "error": null, + "output": "BenchRefineWindows/20261105-121921-ce582f81/final/BenchRefineWindows-mux.3-0.0.mp4" + } +] \ No newline at end of file diff --git a/bench/records/t17-refine-r030.json b/bench/records/t17-refine-r030.json new file mode 100644 index 000000000..57b46c5f8 --- /dev/null +++ b/bench/records/t17-refine-r030.json @@ -0,0 +1,38 @@ +[ + { + "arm": "t17-s1-6seg-r030", + "scene": "s1-static", + "seed": 1166, + "job_id": "c9bf009ba8ab", + "status": "succeeded", + "error": null, + "output": "BenchRefineWindows/20261105-093739-08736f1c/final/BenchRefineWindows-mux.3-0.0.mp4" + }, + { + "arm": "t17-s1-6seg-r030-rebased", + "scene": "s1-static", + "seed": 1166, + "job_id": "c0caed31ae06", + "status": "succeeded", + "error": null, + "output": "BenchRefineWindows/20261105-095727-35da065f/final/BenchRefineWindows-mux.3-0.0.mp4" + }, + { + "arm": "t17-s2-r030", + "scene": "s2-arc", + "seed": 1166, + "job_id": "7fb00bb6d34e", + "status": "succeeded", + "error": null, + "output": "BenchRefineWindows/20261105-101422-4c02b956/final/BenchRefineWindows-mux.3-0.0.mp4" + }, + { + "arm": "t17-s3-r030", + "scene": "s3-dialogue-walk", + "seed": 1166, + "job_id": "a55b3c6f83a2", + "status": "succeeded", + "error": null, + "output": "BenchRefineWindows/20261105-103223-22014f33/final/BenchRefineWindows-mux.3-0.0.mp4" + } +] \ No newline at end of file diff --git a/bench/records/t17-t19-moments.json b/bench/records/t17-t19-moments.json new file mode 100644 index 000000000..a215e9eae --- /dev/null +++ b/bench/records/t17-t19-moments.json @@ -0,0 +1,7409 @@ +{ + "takes": { + "8d1614ced27a": { + "scene": "s1-static", + "moments": [ + 62, + 300, + 583, + 1000, + 1501, + 2000, + 2500, + 2868 + ], + "scene_bands": [ + { + "fine": 6.5469, + "mid": 13.9587, + "coarse": 43.751 + }, + { + "fine": 7.8401, + "mid": 14.5998, + "coarse": 43.7177 + }, + { + "fine": 11.1857, + "mid": 16.6379, + "coarse": 44.4577 + }, + { + "fine": 19.8168, + "mid": 20.9525, + "coarse": 41.3554 + }, + { + "fine": 26.5209, + "mid": 28.5679, + "coarse": 39.4853 + }, + { + "fine": 33.9528, + "mid": 40.7491, + "coarse": 40.3793 + }, + { + "fine": 34.3594, + "mid": 44.3607, + "coarse": 54.1315 + }, + { + "fine": 36.0535, + "mid": 44.9993, + "coarse": 47.7892 + } + ], + "face_bands": [ + { + "fine": 10.8423, + "mid": 24.3037, + "coarse": 43.8552 + }, + { + "fine": 11.8656, + "mid": 27.9083, + "coarse": 43.7488 + }, + { + "fine": 16.0266, + "mid": 33.0963, + "coarse": 49.0015 + }, + { + "fine": 20.6323, + "mid": 37.8384, + "coarse": 49.573 + }, + { + "fine": 22.1962, + "mid": 36.3777, + "coarse": 48.5192 + }, + { + "fine": 23.5208, + "mid": 39.1921, + "coarse": 52.2755 + }, + { + "fine": 21.8608, + "mid": 34.2892, + "coarse": 56.4156 + }, + { + "fine": 22.0642, + "mid": 36.4137, + "coarse": 49.9593 + } + ], + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 0.7, + "look_ratio": 2.13, + "level_step_db": 16.58, + "typical_delta": 2.0 + }, + { + "seconds": 9.417, + "jump_ratio": 0.97, + "look_ratio": 1.75, + "level_step_db": 0.46, + "typical_delta": 2.0 + }, + { + "seconds": 13.667, + "jump_ratio": 0.77, + "look_ratio": 1.68, + "level_step_db": 0.75, + "typical_delta": 2.0 + }, + { + "seconds": 17.917, + "jump_ratio": 0.95, + "look_ratio": 0.73, + "level_step_db": 7.26, + "typical_delta": 2.0 + }, + { + "seconds": 22.167, + "jump_ratio": 0.88, + "look_ratio": 1.03, + "level_step_db": 13.22, + "typical_delta": 2.0 + }, + { + "seconds": 26.417, + "jump_ratio": 1.09, + "look_ratio": 2.61, + "level_step_db": 3.32, + "typical_delta": 2.0 + }, + { + "seconds": 30.667, + "jump_ratio": 0.91, + "look_ratio": 0.99, + "level_step_db": 5.61, + "typical_delta": 2.0 + }, + { + "seconds": 34.917, + "jump_ratio": 1.05, + "look_ratio": 1.01, + "level_step_db": 4.95, + "typical_delta": 2.0 + }, + { + "seconds": 39.167, + "jump_ratio": 1.06, + "look_ratio": 1.83, + "level_step_db": 36.53, + "typical_delta": 2.0 + }, + { + "seconds": 43.417, + "jump_ratio": 1.02, + "look_ratio": 2.35, + "level_step_db": 6.42, + "typical_delta": 2.0 + }, + { + "seconds": 47.667, + "jump_ratio": 1.16, + "look_ratio": 2.47, + "level_step_db": 0.17, + "typical_delta": 2.0 + }, + { + "seconds": 51.917, + "jump_ratio": 1.43, + "look_ratio": 1.9, + "level_step_db": 6.28, + "typical_delta": 2.0 + }, + { + "seconds": 56.167, + "jump_ratio": 1.64, + "look_ratio": 1.88, + "level_step_db": 6.19, + "typical_delta": 2.0 + }, + { + "seconds": 60.417, + "jump_ratio": 1.23, + "look_ratio": 3.26, + "level_step_db": 4.78, + "typical_delta": 2.0 + }, + { + "seconds": 64.667, + "jump_ratio": 1.3, + "look_ratio": 0.9, + "level_step_db": 0.55, + "typical_delta": 2.0 + }, + { + "seconds": 68.917, + "jump_ratio": 0.93, + "look_ratio": 0.72, + "level_step_db": 1.42, + "typical_delta": 2.41 + }, + { + "seconds": 73.167, + "jump_ratio": 0.73, + "look_ratio": 1.54, + "level_step_db": 1.73, + "typical_delta": 3.02 + }, + { + "seconds": 77.417, + "jump_ratio": 0.69, + "look_ratio": 0.43, + "level_step_db": 2.11, + "typical_delta": 3.02 + }, + { + "seconds": 81.667, + "jump_ratio": 1.62, + "look_ratio": 1.31, + "level_step_db": 2.93, + "typical_delta": 2.31 + }, + { + "seconds": 85.917, + "jump_ratio": 1.94, + "look_ratio": 3.75, + "level_step_db": 1.52, + "typical_delta": 2.14 + }, + { + "seconds": 90.167, + "jump_ratio": 1.72, + "look_ratio": 2.19, + "level_step_db": 1.84, + "typical_delta": 2.1 + }, + { + "seconds": 94.417, + "jump_ratio": 1.42, + "look_ratio": 1.33, + "level_step_db": 0.7, + "typical_delta": 2.8 + }, + { + "seconds": 98.667, + "jump_ratio": 0.91, + "look_ratio": 2.83, + "level_step_db": 0.45, + "typical_delta": 2.8 + }, + { + "seconds": 102.917, + "jump_ratio": 1.75, + "look_ratio": 1.93, + "level_step_db": 0.14, + "typical_delta": 2.65 + }, + { + "seconds": 107.167, + "jump_ratio": 0.89, + "look_ratio": 3.5, + "level_step_db": 8.96, + "typical_delta": 2.65 + }, + { + "seconds": 111.417, + "jump_ratio": 6.56, + "look_ratio": 11.51, + "level_step_db": 26.55, + "typical_delta": 2.0 + }, + { + "seconds": 115.667, + "jump_ratio": 1.1, + "look_ratio": 3.2, + "level_step_db": 2.36, + "typical_delta": 2.0 + }, + { + "seconds": 119.917, + "jump_ratio": 1.12, + "look_ratio": 1.76, + "level_step_db": 0.07, + "typical_delta": 2.0 + } + ], + "motion_mean": 0.9227964282035828, + "audio": { + "duration_s": 120.0, + "segments": [ + { + "start": 0.0, + "end": 5.17, + "rms_db": -58.14, + "hf_share_6k": 0.01400089729577303 + }, + { + "start": 5.17, + "end": 9.42, + "rms_db": -74.89, + "hf_share_6k": 0.07208538055419922 + }, + { + "start": 9.42, + "end": 13.67, + "rms_db": -74.36, + "hf_share_6k": 0.023473380133509636 + }, + { + "start": 13.67, + "end": 17.92, + "rms_db": -73.43, + "hf_share_6k": 0.08978622406721115 + }, + { + "start": 17.92, + "end": 22.17, + "rms_db": -80.92, + "hf_share_6k": 0.02124708890914917 + }, + { + "start": 22.17, + "end": 26.42, + "rms_db": -67.9, + "hf_share_6k": 0.0009163153008557856 + }, + { + "start": 26.42, + "end": 30.67, + "rms_db": -70.81, + "hf_share_6k": 0.0189506895840168 + }, + { + "start": 30.67, + "end": 34.92, + "rms_db": -76.63, + "hf_share_6k": 0.014043614268302917 + }, + { + "start": 34.92, + "end": 39.17, + "rms_db": -81.44, + "hf_share_6k": 0.016327768564224243 + }, + { + "start": 39.17, + "end": 43.42, + "rms_db": -44.65, + "hf_share_6k": 6.067055892344797e-06 + }, + { + "start": 43.42, + "end": 47.67, + "rms_db": -38.26, + "hf_share_6k": 1.4210884273779811e-06 + }, + { + "start": 47.67, + "end": 51.92, + "rms_db": -38.44, + "hf_share_6k": 1.1749993973353412e-06 + }, + { + "start": 51.92, + "end": 56.17, + "rms_db": -32.13, + "hf_share_6k": 1.267172820007545e-06 + }, + { + "start": 56.17, + "end": 60.42, + "rms_db": -25.95, + "hf_share_6k": 4.451892365864296e-08 + }, + { + "start": 60.42, + "end": 64.67, + "rms_db": -21.2, + "hf_share_6k": 3.320639407888848e-08 + }, + { + "start": 64.67, + "end": 68.92, + "rms_db": -21.76, + "hf_share_6k": 2.6143469611383807e-08 + }, + { + "start": 68.92, + "end": 73.17, + "rms_db": -20.37, + "hf_share_6k": 1.864024881115256e-07 + }, + { + "start": 73.17, + "end": 77.42, + "rms_db": -18.64, + "hf_share_6k": 1.0675707429186332e-08 + }, + { + "start": 77.42, + "end": 81.67, + "rms_db": -16.59, + "hf_share_6k": 4.8939504893041885e-08 + }, + { + "start": 81.67, + "end": 85.92, + "rms_db": -13.65, + "hf_share_6k": 6.008720188077632e-09 + }, + { + "start": 85.92, + "end": 90.17, + "rms_db": -12.12, + "hf_share_6k": 8.874179684426053e-07 + }, + { + "start": 90.17, + "end": 94.42, + "rms_db": -10.24, + "hf_share_6k": 2.8896979529235978e-06 + }, + { + "start": 94.42, + "end": 98.67, + "rms_db": -9.55, + "hf_share_6k": 4.4199649096299254e-07 + }, + { + "start": 98.67, + "end": 102.92, + "rms_db": -10.15, + "hf_share_6k": 1.870854475782835e-07 + }, + { + "start": 102.92, + "end": 107.17, + "rms_db": -10.06, + "hf_share_6k": 1.239533958141692e-06 + }, + { + "start": 107.17, + "end": 111.42, + "rms_db": -19.08, + "hf_share_6k": 1.6408406736445613e-05 + }, + { + "start": 111.42, + "end": 115.67, + "rms_db": -45.4, + "hf_share_6k": 6.113595190981869e-06 + }, + { + "start": 115.67, + "end": 119.92, + "rms_db": -47.89, + "hf_share_6k": 9.789177056518383e-06 + } + ], + "rms_db_5s": [ + -58.0, + -75.15, + -74.03, + -75.15, + -68.49, + -71.47, + -77.1, + -57.96, + -41.35, + -38.21, + -35.16, + -27.17, + -21.23, + -22.05, + -19.73, + -17.88, + -14.24, + -12.31, + -10.23, + -9.76, + -10.02, + -12.75, + -35.57, + -47.44 + ] + } + }, + "25f6bd1da507": { + "scene": "s1-static", + "moments": [ + 62, + 300, + 583, + 1000, + 1501, + 2000, + 2500, + 2868 + ], + "scene_bands": [ + { + "fine": 6.5469, + "mid": 13.9587, + "coarse": 43.751 + }, + { + "fine": 7.6319, + "mid": 14.6146, + "coarse": 43.8328 + }, + { + "fine": 9.4482, + "mid": 15.7989, + "coarse": 44.434 + }, + { + "fine": 10.7582, + "mid": 16.7015, + "coarse": 44.7208 + }, + { + "fine": 14.1589, + "mid": 18.306, + "coarse": 46.0749 + }, + { + "fine": 16.6179, + "mid": 19.1975, + "coarse": 44.3081 + }, + { + "fine": 18.017, + "mid": 19.929, + "coarse": 45.8376 + }, + { + "fine": 19.8181, + "mid": 21.4665, + "coarse": 44.6048 + } + ], + "face_bands": [ + { + "fine": 10.8423, + "mid": 24.3037, + "coarse": 43.8552 + }, + { + "fine": 11.008, + "mid": 26.6494, + "coarse": 43.7952 + }, + { + "fine": 12.2625, + "mid": 28.1689, + "coarse": 49.237 + }, + { + "fine": 12.3093, + "mid": 26.9052, + "coarse": 55.3572 + }, + { + "fine": 12.5974, + "mid": 26.8984, + "coarse": 52.7442 + }, + { + "fine": 10.7205, + "mid": 24.767, + "coarse": 63.7718 + }, + { + "fine": 12.2653, + "mid": 27.1136, + "coarse": 62.6687 + }, + { + "fine": 10.4146, + "mid": 24.0581, + "coarse": 61.1591 + } + ], + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 0.69, + "look_ratio": 1.98, + "level_step_db": 16.58, + "typical_delta": 2.0 + }, + { + "seconds": 9.417, + "jump_ratio": 1.15, + "look_ratio": 1.57, + "level_step_db": 5.73, + "typical_delta": 2.0 + }, + { + "seconds": 13.667, + "jump_ratio": 0.78, + "look_ratio": 1.16, + "level_step_db": 12.1, + "typical_delta": 2.0 + }, + { + "seconds": 17.917, + "jump_ratio": 0.98, + "look_ratio": 0.47, + "level_step_db": 23.54, + "typical_delta": 2.0 + }, + { + "seconds": 22.167, + "jump_ratio": 1.04, + "look_ratio": 0.46, + "level_step_db": 2.85, + "typical_delta": 2.0 + }, + { + "seconds": 26.417, + "jump_ratio": 0.89, + "look_ratio": 0.43, + "level_step_db": 24.17, + "typical_delta": 2.0 + }, + { + "seconds": 30.667, + "jump_ratio": 0.88, + "look_ratio": 0.31, + "level_step_db": 1.96, + "typical_delta": 2.0 + }, + { + "seconds": 34.917, + "jump_ratio": 0.87, + "look_ratio": 0.63, + "level_step_db": 18.52, + "typical_delta": 2.0 + }, + { + "seconds": 39.167, + "jump_ratio": 0.82, + "look_ratio": 0.41, + "level_step_db": 4.08, + "typical_delta": 2.0 + }, + { + "seconds": 43.417, + "jump_ratio": 0.75, + "look_ratio": 0.81, + "level_step_db": 0.37, + "typical_delta": 2.0 + }, + { + "seconds": 47.667, + "jump_ratio": 0.78, + "look_ratio": 0.92, + "level_step_db": 24.34, + "typical_delta": 2.0 + }, + { + "seconds": 51.917, + "jump_ratio": 0.75, + "look_ratio": 0.36, + "level_step_db": 3.78, + "typical_delta": 2.0 + }, + { + "seconds": 56.167, + "jump_ratio": 1.29, + "look_ratio": 2.37, + "level_step_db": 1.93, + "typical_delta": 2.0 + }, + { + "seconds": 60.417, + "jump_ratio": 0.86, + "look_ratio": 0.68, + "level_step_db": 1.49, + "typical_delta": 2.0 + }, + { + "seconds": 64.667, + "jump_ratio": 0.85, + "look_ratio": 0.47, + "level_step_db": 1.32, + "typical_delta": 2.0 + }, + { + "seconds": 68.917, + "jump_ratio": 0.74, + "look_ratio": 0.74, + "level_step_db": 0.05, + "typical_delta": 2.0 + }, + { + "seconds": 73.167, + "jump_ratio": 0.96, + "look_ratio": 1.53, + "level_step_db": 1.71, + "typical_delta": 2.0 + }, + { + "seconds": 77.417, + "jump_ratio": 0.76, + "look_ratio": 0.47, + "level_step_db": 0.27, + "typical_delta": 2.0 + }, + { + "seconds": 81.667, + "jump_ratio": 0.73, + "look_ratio": 0.53, + "level_step_db": 0.59, + "typical_delta": 2.0 + }, + { + "seconds": 85.917, + "jump_ratio": 0.69, + "look_ratio": 0.65, + "level_step_db": 4.57, + "typical_delta": 2.07 + }, + { + "seconds": 90.167, + "jump_ratio": 0.84, + "look_ratio": 0.56, + "level_step_db": 3.56, + "typical_delta": 2.07 + }, + { + "seconds": 94.417, + "jump_ratio": 0.76, + "look_ratio": 0.53, + "level_step_db": 0.15, + "typical_delta": 2.0 + }, + { + "seconds": 98.667, + "jump_ratio": 1.23, + "look_ratio": 1.82, + "level_step_db": 1.87, + "typical_delta": 2.48 + }, + { + "seconds": 102.917, + "jump_ratio": 1.07, + "look_ratio": 1.9, + "level_step_db": 1.55, + "typical_delta": 2.48 + }, + { + "seconds": 107.167, + "jump_ratio": 1.24, + "look_ratio": 1.89, + "level_step_db": 1.44, + "typical_delta": 2.0 + }, + { + "seconds": 111.417, + "jump_ratio": 1.08, + "look_ratio": 1.26, + "level_step_db": 2.04, + "typical_delta": 2.16 + }, + { + "seconds": 115.667, + "jump_ratio": 1.07, + "look_ratio": 1.3, + "level_step_db": 2.51, + "typical_delta": 2.62 + }, + { + "seconds": 119.917, + "jump_ratio": 1.86, + "look_ratio": 2.7, + "level_step_db": 0.29, + "typical_delta": 2.62 + } + ], + "motion_mean": 0.6359326839447021, + "audio": { + "duration_s": 120.0, + "segments": [ + { + "start": 0.0, + "end": 5.17, + "rms_db": -58.14, + "hf_share_6k": 0.01400089729577303 + }, + { + "start": 5.17, + "end": 9.42, + "rms_db": -74.89, + "hf_share_6k": 0.0720653310418129 + }, + { + "start": 9.42, + "end": 13.67, + "rms_db": -70.83, + "hf_share_6k": 0.023430997505784035 + }, + { + "start": 13.67, + "end": 17.92, + "rms_db": -56.86, + "hf_share_6k": 0.0014793450245633721 + }, + { + "start": 17.92, + "end": 22.17, + "rms_db": -80.91, + "hf_share_6k": 0.023885471746325493 + }, + { + "start": 22.17, + "end": 26.42, + "rms_db": -77.77, + "hf_share_6k": 0.007944004610180855 + }, + { + "start": 26.42, + "end": 30.67, + "rms_db": -53.67, + "hf_share_6k": 0.002972153015434742 + }, + { + "start": 30.67, + "end": 34.92, + "rms_db": -55.84, + "hf_share_6k": 0.035119205713272095 + }, + { + "start": 34.92, + "end": 39.17, + "rms_db": -74.19, + "hf_share_6k": 0.008978131227195263 + }, + { + "start": 39.17, + "end": 43.42, + "rms_db": -70.36, + "hf_share_6k": 0.02529214881360531 + }, + { + "start": 43.42, + "end": 47.67, + "rms_db": -70.76, + "hf_share_6k": 0.0014990228228271008 + }, + { + "start": 47.67, + "end": 51.92, + "rms_db": -46.35, + "hf_share_6k": 5.021100605517859e-06 + }, + { + "start": 51.92, + "end": 56.17, + "rms_db": -42.05, + "hf_share_6k": 3.7556912957370514e-06 + }, + { + "start": 56.17, + "end": 60.42, + "rms_db": -40.04, + "hf_share_6k": 2.3007476102065993e-06 + }, + { + "start": 60.42, + "end": 64.67, + "rms_db": -41.56, + "hf_share_6k": 1.9763733689615037e-06 + }, + { + "start": 64.67, + "end": 68.92, + "rms_db": -42.85, + "hf_share_6k": 4.893918685411336e-06 + }, + { + "start": 68.92, + "end": 73.17, + "rms_db": -42.81, + "hf_share_6k": 3.03548586089164e-06 + }, + { + "start": 73.17, + "end": 77.42, + "rms_db": -41.14, + "hf_share_6k": 1.434754949514172e-06 + }, + { + "start": 77.42, + "end": 81.67, + "rms_db": -41.5, + "hf_share_6k": 1.434263253941026e-06 + }, + { + "start": 81.67, + "end": 85.92, + "rms_db": -40.96, + "hf_share_6k": 1.1347430017849547e-06 + }, + { + "start": 85.92, + "end": 90.17, + "rms_db": -45.52, + "hf_share_6k": 3.200544369974523e-06 + }, + { + "start": 90.17, + "end": 94.42, + "rms_db": -49.0, + "hf_share_6k": 7.618869403813733e-06 + }, + { + "start": 94.42, + "end": 98.67, + "rms_db": -49.12, + "hf_share_6k": 8.449503184237983e-06 + }, + { + "start": 98.67, + "end": 102.92, + "rms_db": -51.05, + "hf_share_6k": 1.7455708075431176e-05 + }, + { + "start": 102.92, + "end": 107.17, + "rms_db": -49.43, + "hf_share_6k": 1.2286018318263814e-05 + }, + { + "start": 107.17, + "end": 111.42, + "rms_db": -50.94, + "hf_share_6k": 1.538179822091479e-05 + }, + { + "start": 111.42, + "end": 115.67, + "rms_db": -48.79, + "hf_share_6k": 2.0485975255724043e-05 + }, + { + "start": 115.67, + "end": 119.92, + "rms_db": -46.35, + "hf_share_6k": 5.847371085110353e-06 + } + ], + "rms_db_5s": [ + -58.0, + -75.17, + -71.2, + -57.57, + -78.38, + -54.38, + -56.54, + -74.19, + -71.26, + -54.15, + -43.01, + -40.27, + -41.33, + -43.31, + -41.75, + -41.51, + -41.14, + -44.07, + -48.87, + -49.67, + -50.68, + -49.53, + -49.89, + -46.46 + ] + } + }, + "2c1fd3ab7869": { + "scene": "s1-static", + "moments": [ + 62, + 300, + 583, + 1000, + 1501, + 2000, + 2500, + 2868 + ], + "scene_bands": [ + { + "fine": 6.5737, + "mid": 14.0049, + "coarse": 43.7505 + }, + { + "fine": 7.5302, + "mid": 14.711, + "coarse": 43.993 + }, + { + "fine": 9.0879, + "mid": 15.9518, + "coarse": 44.7847 + }, + { + "fine": 11.2597, + "mid": 17.3604, + "coarse": 44.6966 + }, + { + "fine": 15.0259, + "mid": 18.615, + "coarse": 43.924 + }, + { + "fine": 18.6378, + "mid": 20.7975, + "coarse": 43.6961 + }, + { + "fine": 19.5395, + "mid": 23.2805, + "coarse": 53.276 + }, + { + "fine": 21.4395, + "mid": 22.9959, + "coarse": 50.21 + } + ], + "face_bands": [ + { + "fine": 10.8347, + "mid": 24.293, + "coarse": 43.8952 + }, + { + "fine": 10.9399, + "mid": 26.7791, + "coarse": 43.8957 + }, + { + "fine": 12.4812, + "mid": 26.597, + "coarse": 46.2632 + }, + { + "fine": 10.7307, + "mid": 22.5301, + "coarse": 47.4545 + }, + { + "fine": 7.6622, + "mid": 17.8458, + "coarse": 47.2241 + }, + { + "fine": 8.9539, + "mid": 23.0617, + "coarse": 53.9051 + }, + { + "fine": 6.2124, + "mid": 22.9185, + "coarse": 58.1832 + }, + { + "fine": 5.4962, + "mid": 18.4771, + "coarse": 57.6644 + } + ], + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 0.67, + "look_ratio": 2.0, + "level_step_db": 10.36, + "typical_delta": 2.0 + }, + { + "seconds": 9.417, + "jump_ratio": 0.97, + "look_ratio": 1.47, + "level_step_db": 5.71, + "typical_delta": 2.0 + }, + { + "seconds": 13.667, + "jump_ratio": 0.83, + "look_ratio": 1.66, + "level_step_db": 3.33, + "typical_delta": 2.0 + }, + { + "seconds": 17.917, + "jump_ratio": 1.03, + "look_ratio": 0.19, + "level_step_db": 12.74, + "typical_delta": 2.0 + }, + { + "seconds": 22.167, + "jump_ratio": 1.07, + "look_ratio": 0.67, + "level_step_db": 26.07, + "typical_delta": 2.0 + }, + { + "seconds": 26.417, + "jump_ratio": 0.92, + "look_ratio": 0.8, + "level_step_db": 0.2, + "typical_delta": 2.0 + }, + { + "seconds": 30.667, + "jump_ratio": 0.83, + "look_ratio": 0.54, + "level_step_db": 5.63, + "typical_delta": 2.0 + }, + { + "seconds": 34.917, + "jump_ratio": 0.85, + "look_ratio": 0.51, + "level_step_db": 4.23, + "typical_delta": 2.0 + }, + { + "seconds": 39.167, + "jump_ratio": 0.75, + "look_ratio": 0.29, + "level_step_db": 0.14, + "typical_delta": 2.0 + }, + { + "seconds": 43.417, + "jump_ratio": 0.75, + "look_ratio": 0.49, + "level_step_db": 3.24, + "typical_delta": 2.0 + }, + { + "seconds": 47.667, + "jump_ratio": 0.73, + "look_ratio": 0.6, + "level_step_db": 3.08, + "typical_delta": 2.0 + }, + { + "seconds": 51.917, + "jump_ratio": 0.8, + "look_ratio": 0.53, + "level_step_db": 0.62, + "typical_delta": 2.0 + }, + { + "seconds": 56.167, + "jump_ratio": 0.91, + "look_ratio": 1.31, + "level_step_db": 1.77, + "typical_delta": 2.0 + }, + { + "seconds": 60.417, + "jump_ratio": 0.78, + "look_ratio": 0.47, + "level_step_db": 3.15, + "typical_delta": 2.0 + }, + { + "seconds": 64.667, + "jump_ratio": 0.96, + "look_ratio": 1.39, + "level_step_db": 6.2, + "typical_delta": 2.0 + }, + { + "seconds": 68.917, + "jump_ratio": 1.05, + "look_ratio": 1.51, + "level_step_db": 2.69, + "typical_delta": 2.0 + }, + { + "seconds": 73.167, + "jump_ratio": 0.79, + "look_ratio": 0.48, + "level_step_db": 3.0, + "typical_delta": 2.0 + }, + { + "seconds": 77.417, + "jump_ratio": 0.76, + "look_ratio": 0.76, + "level_step_db": 0.34, + "typical_delta": 2.0 + }, + { + "seconds": 81.667, + "jump_ratio": 0.84, + "look_ratio": 0.63, + "level_step_db": 0.94, + "typical_delta": 2.0 + }, + { + "seconds": 85.917, + "jump_ratio": 0.81, + "look_ratio": 0.53, + "level_step_db": 0.73, + "typical_delta": 2.0 + }, + { + "seconds": 90.167, + "jump_ratio": 0.75, + "look_ratio": 0.77, + "level_step_db": 3.12, + "typical_delta": 2.11 + }, + { + "seconds": 94.417, + "jump_ratio": 0.96, + "look_ratio": 0.9, + "level_step_db": 4.9, + "typical_delta": 2.11 + }, + { + "seconds": 98.667, + "jump_ratio": 1.51, + "look_ratio": 0.75, + "level_step_db": 4.25, + "typical_delta": 2.04 + }, + { + "seconds": 102.917, + "jump_ratio": 1.22, + "look_ratio": 0.86, + "level_step_db": 3.34, + "typical_delta": 2.04 + }, + { + "seconds": 107.167, + "jump_ratio": 1.04, + "look_ratio": 2.06, + "level_step_db": 3.38, + "typical_delta": 2.0 + }, + { + "seconds": 111.417, + "jump_ratio": 0.96, + "look_ratio": 1.08, + "level_step_db": 2.77, + "typical_delta": 2.0 + }, + { + "seconds": 115.667, + "jump_ratio": 0.84, + "look_ratio": 0.62, + "level_step_db": 5.55, + "typical_delta": 2.0 + }, + { + "seconds": 119.917, + "jump_ratio": 0.8, + "look_ratio": 0.51, + "level_step_db": 0.37, + "typical_delta": 2.0 + } + ], + "motion_mean": 0.5547353029251099, + "audio": { + "duration_s": 120.0, + "segments": [ + { + "start": 0.0, + "end": 5.17, + "rms_db": -58.14, + "hf_share_6k": 0.01400093175470829 + }, + { + "start": 5.17, + "end": 9.42, + "rms_db": -68.52, + "hf_share_6k": 0.019539538770914078 + }, + { + "start": 9.42, + "end": 13.67, + "rms_db": -74.15, + "hf_share_6k": 0.02664010412991047 + }, + { + "start": 13.67, + "end": 17.92, + "rms_db": -71.09, + "hf_share_6k": 0.06111638620495796 + }, + { + "start": 17.92, + "end": 22.17, + "rms_db": -83.6, + "hf_share_6k": 0.031826067715883255 + }, + { + "start": 22.17, + "end": 26.42, + "rms_db": -57.56, + "hf_share_6k": 0.000841731030959636 + }, + { + "start": 26.42, + "end": 30.67, + "rms_db": -57.89, + "hf_share_6k": 0.006909847259521484 + }, + { + "start": 30.67, + "end": 34.92, + "rms_db": -52.04, + "hf_share_6k": 0.001127829309552908 + }, + { + "start": 34.92, + "end": 39.17, + "rms_db": -47.73, + "hf_share_6k": 0.0001161541003966704 + }, + { + "start": 39.17, + "end": 43.42, + "rms_db": -47.86, + "hf_share_6k": 1.3094735550112091e-05 + }, + { + "start": 43.42, + "end": 47.67, + "rms_db": -44.6, + "hf_share_6k": 5.387328201322816e-06 + }, + { + "start": 47.67, + "end": 51.92, + "rms_db": -47.7, + "hf_share_6k": 7.809984708728734e-06 + }, + { + "start": 51.92, + "end": 56.17, + "rms_db": -48.34, + "hf_share_6k": 1.0875507541641127e-05 + }, + { + "start": 56.17, + "end": 60.42, + "rms_db": -50.16, + "hf_share_6k": 9.675738510850351e-06 + }, + { + "start": 60.42, + "end": 64.67, + "rms_db": -53.41, + "hf_share_6k": 2.3165943275671452e-05 + }, + { + "start": 64.67, + "end": 68.92, + "rms_db": -47.15, + "hf_share_6k": 8.56040969665628e-06 + }, + { + "start": 68.92, + "end": 73.17, + "rms_db": -44.74, + "hf_share_6k": 2.9866293971281266e-06 + }, + { + "start": 73.17, + "end": 77.42, + "rms_db": -41.76, + "hf_share_6k": 1.5971459106367547e-06 + }, + { + "start": 77.42, + "end": 81.67, + "rms_db": -42.38, + "hf_share_6k": 1.972456402654643e-06 + }, + { + "start": 81.67, + "end": 85.92, + "rms_db": -41.31, + "hf_share_6k": 1.8624259610078298e-06 + }, + { + "start": 85.92, + "end": 90.17, + "rms_db": -41.03, + "hf_share_6k": 1.925631067933864e-06 + }, + { + "start": 90.17, + "end": 94.42, + "rms_db": -37.92, + "hf_share_6k": 1.6864574945429922e-06 + }, + { + "start": 94.42, + "end": 98.67, + "rms_db": -33.36, + "hf_share_6k": 7.618439781253983e-07 + }, + { + "start": 98.67, + "end": 102.92, + "rms_db": -29.51, + "hf_share_6k": 5.33551940407051e-07 + }, + { + "start": 102.92, + "end": 107.17, + "rms_db": -26.53, + "hf_share_6k": 3.4154433592448186e-07 + }, + { + "start": 107.17, + "end": 111.42, + "rms_db": -23.56, + "hf_share_6k": 2.2134300081688707e-07 + }, + { + "start": 111.42, + "end": 115.67, + "rms_db": -21.06, + "hf_share_6k": 1.1413817446737085e-07 + }, + { + "start": 115.67, + "end": 119.92, + "rms_db": -15.62, + "hf_share_6k": 2.0453031268630184e-08 + } + ], + "rms_db_5s": [ + -58.0, + -69.13, + -74.0, + -72.25, + -58.27, + -58.68, + -52.51, + -47.54, + -46.29, + -46.29, + -47.98, + -49.52, + -51.85, + -46.94, + -43.1, + -42.27, + -41.48, + -41.17, + -37.42, + -32.31, + -28.05, + -24.64, + -21.63, + -16.03 + ] + } + }, + "bf9ee32e040f": { + "scene": "s1-static", + "moments": [ + 62, + 300, + 583, + 1000, + 1501, + 2000, + 2500, + 2868 + ], + "scene_bands": [ + { + "fine": 6.5502, + "mid": 13.9675, + "coarse": 44.0694 + }, + { + "fine": 7.8425, + "mid": 15.1246, + "coarse": 44.6907 + }, + { + "fine": 9.0652, + "mid": 15.9061, + "coarse": 45.4512 + }, + { + "fine": 11.9209, + "mid": 16.9578, + "coarse": 45.3353 + }, + { + "fine": 15.4658, + "mid": 18.7809, + "coarse": 44.7544 + }, + { + "fine": 16.6307, + "mid": 20.456, + "coarse": 43.4291 + }, + { + "fine": 19.2206, + "mid": 22.896, + "coarse": 44.089 + }, + { + "fine": 20.5237, + "mid": 25.1906, + "coarse": 43.3242 + } + ], + "face_bands": [ + { + "fine": 11.3836, + "mid": 27.014, + "coarse": 43.5128 + }, + { + "fine": 10.8727, + "mid": 24.2582, + "coarse": 49.8561 + }, + { + "fine": 12.18, + "mid": 25.5643, + "coarse": 48.5315 + }, + { + "fine": 15.4232, + "mid": 28.2115, + "coarse": 51.9024 + }, + { + "fine": 12.0653, + "mid": 25.5961, + "coarse": 57.9319 + }, + { + "fine": 13.2956, + "mid": 29.579, + "coarse": 60.846 + }, + { + "fine": 9.031, + "mid": 29.3365, + "coarse": 69.9845 + }, + { + "fine": 7.1348, + "mid": 26.8163, + "coarse": 59.662 + } + ], + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 0.7, + "look_ratio": 1.88, + "level_step_db": 0.3, + "typical_delta": 2.0 + }, + { + "seconds": 9.417, + "jump_ratio": 0.76, + "look_ratio": 1.67, + "level_step_db": 16.11, + "typical_delta": 2.0 + }, + { + "seconds": 13.667, + "jump_ratio": 0.82, + "look_ratio": 0.59, + "level_step_db": 2.67, + "typical_delta": 2.0 + }, + { + "seconds": 17.917, + "jump_ratio": 0.95, + "look_ratio": 0.86, + "level_step_db": 14.81, + "typical_delta": 2.0 + }, + { + "seconds": 22.167, + "jump_ratio": 0.83, + "look_ratio": 0.46, + "level_step_db": 18.24, + "typical_delta": 2.0 + }, + { + "seconds": 26.417, + "jump_ratio": 1.24, + "look_ratio": 2.04, + "level_step_db": 25.34, + "typical_delta": 2.0 + }, + { + "seconds": 30.667, + "jump_ratio": 1.04, + "look_ratio": 1.41, + "level_step_db": 7.33, + "typical_delta": 2.0 + }, + { + "seconds": 34.917, + "jump_ratio": 0.75, + "look_ratio": 0.7, + "level_step_db": 2.18, + "typical_delta": 2.0 + }, + { + "seconds": 39.167, + "jump_ratio": 0.99, + "look_ratio": 1.53, + "level_step_db": 8.81, + "typical_delta": 2.0 + }, + { + "seconds": 43.417, + "jump_ratio": 0.7, + "look_ratio": 0.44, + "level_step_db": 4.07, + "typical_delta": 2.0 + }, + { + "seconds": 47.667, + "jump_ratio": 1.28, + "look_ratio": 2.37, + "level_step_db": 16.86, + "typical_delta": 2.0 + }, + { + "seconds": 51.917, + "jump_ratio": 1.15, + "look_ratio": 1.57, + "level_step_db": 2.82, + "typical_delta": 2.0 + }, + { + "seconds": 56.167, + "jump_ratio": 0.91, + "look_ratio": 1.29, + "level_step_db": 7.94, + "typical_delta": 2.0 + }, + { + "seconds": 60.417, + "jump_ratio": 0.81, + "look_ratio": 0.9, + "level_step_db": 0.73, + "typical_delta": 2.0 + }, + { + "seconds": 64.667, + "jump_ratio": 0.99, + "look_ratio": 1.61, + "level_step_db": 10.49, + "typical_delta": 2.0 + }, + { + "seconds": 68.917, + "jump_ratio": 1.05, + "look_ratio": 1.45, + "level_step_db": 7.71, + "typical_delta": 2.0 + }, + { + "seconds": 73.167, + "jump_ratio": 0.99, + "look_ratio": 0.93, + "level_step_db": 14.96, + "typical_delta": 2.0 + }, + { + "seconds": 77.417, + "jump_ratio": 1.03, + "look_ratio": 1.51, + "level_step_db": 20.85, + "typical_delta": 2.0 + }, + { + "seconds": 81.667, + "jump_ratio": 0.96, + "look_ratio": 0.46, + "level_step_db": 31.99, + "typical_delta": 2.0 + }, + { + "seconds": 85.917, + "jump_ratio": 0.88, + "look_ratio": 1.09, + "level_step_db": 25.28, + "typical_delta": 2.0 + }, + { + "seconds": 90.167, + "jump_ratio": 1.02, + "look_ratio": 1.56, + "level_step_db": 4.61, + "typical_delta": 2.0 + }, + { + "seconds": 94.417, + "jump_ratio": 0.94, + "look_ratio": 1.07, + "level_step_db": 6.98, + "typical_delta": 2.0 + }, + { + "seconds": 98.667, + "jump_ratio": 0.99, + "look_ratio": 0.93, + "level_step_db": 8.34, + "typical_delta": 2.0 + }, + { + "seconds": 102.917, + "jump_ratio": 1.19, + "look_ratio": 0.33, + "level_step_db": 6.31, + "typical_delta": 2.23 + }, + { + "seconds": 107.167, + "jump_ratio": 1.37, + "look_ratio": 1.87, + "level_step_db": 13.78, + "typical_delta": 2.23 + }, + { + "seconds": 111.417, + "jump_ratio": 1.43, + "look_ratio": 2.16, + "level_step_db": 8.09, + "typical_delta": 2.49 + }, + { + "seconds": 115.667, + "jump_ratio": 1.21, + "look_ratio": 2.66, + "level_step_db": 0.17, + "typical_delta": 2.67 + }, + { + "seconds": 119.917, + "jump_ratio": 0.66, + "look_ratio": 0.93, + "level_step_db": 4.87, + "typical_delta": 2.67 + } + ], + "motion_mean": 0.5758509635925293, + "audio": { + "duration_s": 120.0, + "segments": [ + { + "start": 0.0, + "end": 5.17, + "rms_db": -60.41, + "hf_share_6k": 0.03143714368343353 + }, + { + "start": 5.17, + "end": 9.42, + "rms_db": -60.45, + "hf_share_6k": 0.00739676458761096 + }, + { + "start": 9.42, + "end": 13.67, + "rms_db": -76.77, + "hf_share_6k": 0.1023726686835289 + }, + { + "start": 13.67, + "end": 17.92, + "rms_db": -79.51, + "hf_share_6k": 0.1576482355594635 + }, + { + "start": 17.92, + "end": 22.17, + "rms_db": -64.33, + "hf_share_6k": 0.09698781371116638 + }, + { + "start": 22.17, + "end": 26.42, + "rms_db": -83.24, + "hf_share_6k": 0.11469386518001556 + }, + { + "start": 26.42, + "end": 30.67, + "rms_db": -56.96, + "hf_share_6k": 0.007821295410394669 + }, + { + "start": 30.67, + "end": 34.92, + "rms_db": -64.76, + "hf_share_6k": 0.0010758801363408566 + }, + { + "start": 34.92, + "end": 39.17, + "rms_db": -63.09, + "hf_share_6k": 0.06075051426887512 + }, + { + "start": 39.17, + "end": 43.42, + "rms_db": -53.13, + "hf_share_6k": 0.0032093615736812353 + }, + { + "start": 43.42, + "end": 47.67, + "rms_db": -57.2, + "hf_share_6k": 0.0026547389570623636 + }, + { + "start": 47.67, + "end": 51.92, + "rms_db": -74.17, + "hf_share_6k": 0.001943756826221943 + }, + { + "start": 51.92, + "end": 56.17, + "rms_db": -77.01, + "hf_share_6k": 0.011327325366437435 + }, + { + "start": 56.17, + "end": 60.42, + "rms_db": -85.2, + "hf_share_6k": 0.039697982370853424 + }, + { + "start": 60.42, + "end": 64.67, + "rms_db": -84.43, + "hf_share_6k": 0.032235000282526016 + }, + { + "start": 64.67, + "end": 68.92, + "rms_db": -74.1, + "hf_share_6k": 0.004242587834596634 + }, + { + "start": 68.92, + "end": 73.17, + "rms_db": -81.4, + "hf_share_6k": 0.014463082887232304 + }, + { + "start": 73.17, + "end": 77.42, + "rms_db": -66.44, + "hf_share_6k": 0.0002006677386816591 + }, + { + "start": 77.42, + "end": 81.67, + "rms_db": -87.43, + "hf_share_6k": 0.02537946030497551 + }, + { + "start": 81.67, + "end": 85.92, + "rms_db": -55.27, + "hf_share_6k": 0.0077349296770989895 + }, + { + "start": 85.92, + "end": 90.17, + "rms_db": -80.5, + "hf_share_6k": 0.010744323953986168 + }, + { + "start": 90.17, + "end": 94.42, + "rms_db": -75.84, + "hf_share_6k": 0.005934776272624731 + }, + { + "start": 94.42, + "end": 98.67, + "rms_db": -82.89, + "hf_share_6k": 0.012528957799077034 + }, + { + "start": 98.67, + "end": 102.92, + "rms_db": -74.65, + "hf_share_6k": 0.02569250762462616 + }, + { + "start": 102.92, + "end": 107.17, + "rms_db": -80.85, + "hf_share_6k": 0.008330151438713074 + }, + { + "start": 107.17, + "end": 111.42, + "rms_db": -69.23, + "hf_share_6k": 0.038965560495853424 + }, + { + "start": 111.42, + "end": 115.67, + "rms_db": -75.45, + "hf_share_6k": 0.012251093052327633 + }, + { + "start": 115.67, + "end": 119.92, + "rms_db": -75.41, + "hf_share_6k": 0.008680704981088638 + } + ], + "rms_db_5s": [ + -60.33, + -61.07, + -77.52, + -68.96, + -67.04, + -57.81, + -64.68, + -63.8, + -53.83, + -57.83, + -78.15, + -83.59, + -84.87, + -74.29, + -67.11, + -88.02, + -55.97, + -81.1, + -76.3, + -75.67, + -80.18, + -70.31, + -75.05, + -75.21 + ] + } + }, + "fc0f91663277": { + "scene": "s1-static", + "moments": [ + 62, + 300, + 583, + 1000, + 1501, + 2000, + 2500, + 2868 + ], + "scene_bands": [ + { + "fine": 6.5469, + "mid": 13.9587, + "coarse": 43.751 + }, + { + "fine": 7.3348, + "mid": 14.5587, + "coarse": 43.7379 + }, + { + "fine": 7.2828, + "mid": 15.2194, + "coarse": 43.9725 + }, + { + "fine": 7.6167, + "mid": 15.3259, + "coarse": 42.272 + }, + { + "fine": 9.1084, + "mid": 16.1315, + "coarse": 41.7091 + }, + { + "fine": 10.189, + "mid": 16.4457, + "coarse": 38.9021 + }, + { + "fine": 11.6048, + "mid": 18.0786, + "coarse": 36.4009 + }, + { + "fine": 12.453, + "mid": 19.4975, + "coarse": 35.236 + } + ], + "face_bands": [ + { + "fine": 10.8423, + "mid": 24.3037, + "coarse": 43.8552 + }, + { + "fine": 11.0631, + "mid": 26.2229, + "coarse": 45.4461 + }, + { + "fine": 12.093, + "mid": 26.1114, + "coarse": 45.5244 + }, + { + "fine": 12.5753, + "mid": 27.52, + "coarse": 48.6259 + }, + { + "fine": 12.1186, + "mid": 26.8818, + "coarse": 52.2386 + }, + { + "fine": 12.4333, + "mid": 28.6988, + "coarse": 49.2497 + }, + { + "fine": 10.0506, + "mid": 29.7248, + "coarse": 45.3373 + }, + { + "fine": 8.7533, + "mid": 27.7454, + "coarse": 51.4721 + } + ], + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 0.74, + "look_ratio": 2.05, + "level_step_db": 14.97, + "typical_delta": 2.0 + }, + { + "seconds": 9.417, + "jump_ratio": 0.68, + "look_ratio": 1.23, + "level_step_db": 19.09, + "typical_delta": 2.0 + }, + { + "seconds": 13.667, + "jump_ratio": 0.71, + "look_ratio": 1.07, + "level_step_db": 13.2, + "typical_delta": 2.0 + }, + { + "seconds": 17.917, + "jump_ratio": 0.88, + "look_ratio": 1.24, + "level_step_db": 21.23, + "typical_delta": 2.0 + }, + { + "seconds": 22.167, + "jump_ratio": 1.08, + "look_ratio": 1.69, + "level_step_db": 33.11, + "typical_delta": 2.0 + }, + { + "seconds": 26.417, + "jump_ratio": 0.98, + "look_ratio": 1.34, + "level_step_db": 22.55, + "typical_delta": 2.0 + }, + { + "seconds": 30.667, + "jump_ratio": 1.04, + "look_ratio": 1.3, + "level_step_db": 12.88, + "typical_delta": 2.0 + }, + { + "seconds": 34.917, + "jump_ratio": 1.02, + "look_ratio": 1.49, + "level_step_db": 2.14, + "typical_delta": 2.0 + }, + { + "seconds": 39.167, + "jump_ratio": 0.81, + "look_ratio": 1.22, + "level_step_db": 19.83, + "typical_delta": 2.0 + }, + { + "seconds": 43.417, + "jump_ratio": 1.05, + "look_ratio": 1.61, + "level_step_db": 12.3, + "typical_delta": 2.0 + }, + { + "seconds": 47.667, + "jump_ratio": 0.66, + "look_ratio": 0.42, + "level_step_db": 6.34, + "typical_delta": 2.0 + }, + { + "seconds": 51.917, + "jump_ratio": 1.04, + "look_ratio": 1.59, + "level_step_db": 32.77, + "typical_delta": 2.0 + }, + { + "seconds": 56.167, + "jump_ratio": 1.17, + "look_ratio": 1.91, + "level_step_db": 54.07, + "typical_delta": 2.0 + }, + { + "seconds": 60.417, + "jump_ratio": 1.06, + "look_ratio": 1.82, + "level_step_db": 35.63, + "typical_delta": 2.0 + }, + { + "seconds": 64.667, + "jump_ratio": 1.04, + "look_ratio": 1.48, + "level_step_db": 20.25, + "typical_delta": 2.0 + }, + { + "seconds": 68.917, + "jump_ratio": 1.13, + "look_ratio": 1.85, + "level_step_db": 17.84, + "typical_delta": 2.04 + }, + { + "seconds": 73.167, + "jump_ratio": 0.6, + "look_ratio": 0.26, + "level_step_db": 19.13, + "typical_delta": 2.23 + }, + { + "seconds": 77.417, + "jump_ratio": 0.77, + "look_ratio": 1.06, + "level_step_db": 24.07, + "typical_delta": 2.27 + }, + { + "seconds": 81.667, + "jump_ratio": 0.66, + "look_ratio": 0.7, + "level_step_db": 4.18, + "typical_delta": 2.27 + }, + { + "seconds": 85.917, + "jump_ratio": 0.65, + "look_ratio": 0.84, + "level_step_db": 8.69, + "typical_delta": 2.3 + }, + { + "seconds": 90.167, + "jump_ratio": 1.02, + "look_ratio": 2.13, + "level_step_db": 2.43, + "typical_delta": 2.3 + }, + { + "seconds": 94.417, + "jump_ratio": 0.74, + "look_ratio": 0.87, + "level_step_db": 13.24, + "typical_delta": 2.33 + }, + { + "seconds": 98.667, + "jump_ratio": 0.77, + "look_ratio": 1.14, + "level_step_db": 0.3, + "typical_delta": 2.38 + }, + { + "seconds": 102.917, + "jump_ratio": 0.61, + "look_ratio": 0.24, + "level_step_db": 2.24, + "typical_delta": 2.38 + }, + { + "seconds": 107.167, + "jump_ratio": 0.61, + "look_ratio": 0.46, + "level_step_db": 12.26, + "typical_delta": 2.51 + }, + { + "seconds": 111.417, + "jump_ratio": 0.67, + "look_ratio": 0.92, + "level_step_db": 8.9, + "typical_delta": 2.51 + }, + { + "seconds": 115.667, + "jump_ratio": 0.62, + "look_ratio": 0.56, + "level_step_db": 10.95, + "typical_delta": 2.52 + }, + { + "seconds": 119.917, + "jump_ratio": 0.77, + "look_ratio": 0.42, + "level_step_db": 2.49, + "typical_delta": 2.52 + } + ], + "motion_mean": 0.6193464994430542, + "audio": { + "duration_s": 120.0, + "segments": [ + { + "start": 0.0, + "end": 5.17, + "rms_db": -58.14, + "hf_share_6k": 0.014000915922224522 + }, + { + "start": 5.17, + "end": 9.42, + "rms_db": -73.11, + "hf_share_6k": 0.04992169886827469 + }, + { + "start": 9.42, + "end": 13.67, + "rms_db": -54.11, + "hf_share_6k": 0.0023527522571384907 + }, + { + "start": 13.67, + "end": 17.92, + "rms_db": -67.46, + "hf_share_6k": 0.0019092172151431441 + }, + { + "start": 17.92, + "end": 22.17, + "rms_db": -46.12, + "hf_share_6k": 0.0021373238414525986 + }, + { + "start": 22.17, + "end": 26.42, + "rms_db": -79.52, + "hf_share_6k": 0.009233733639121056 + }, + { + "start": 26.42, + "end": 30.67, + "rms_db": -56.72, + "hf_share_6k": 0.03675738349556923 + }, + { + "start": 30.67, + "end": 34.92, + "rms_db": -69.68, + "hf_share_6k": 0.002184038981795311 + }, + { + "start": 34.92, + "end": 39.17, + "rms_db": -67.49, + "hf_share_6k": 0.004658800549805164 + }, + { + "start": 39.17, + "end": 43.42, + "rms_db": -87.53, + "hf_share_6k": 0.06074976921081543 + }, + { + "start": 43.42, + "end": 47.67, + "rms_db": -75.03, + "hf_share_6k": 0.006242981180548668 + }, + { + "start": 47.67, + "end": 51.92, + "rms_db": -81.69, + "hf_share_6k": 0.026957785710692406 + }, + { + "start": 51.92, + "end": 56.17, + "rms_db": -48.7, + "hf_share_6k": 0.06376396864652634 + }, + { + "start": 56.17, + "end": 60.42, + "rms_db": -103.37, + "hf_share_6k": 0.06517573446035385 + }, + { + "start": 60.42, + "end": 64.67, + "rms_db": -66.82, + "hf_share_6k": 0.07340915501117706 + }, + { + "start": 64.67, + "end": 68.92, + "rms_db": -87.4, + "hf_share_6k": 0.048387739807367325 + }, + { + "start": 68.92, + "end": 73.17, + "rms_db": -69.14, + "hf_share_6k": 0.03323311358690262 + }, + { + "start": 73.17, + "end": 77.42, + "rms_db": -88.83, + "hf_share_6k": 0.051027584820985794 + }, + { + "start": 77.42, + "end": 81.67, + "rms_db": -64.45, + "hf_share_6k": 0.019322048872709274 + }, + { + "start": 81.67, + "end": 85.92, + "rms_db": -68.62, + "hf_share_6k": 0.0005157851846888661 + }, + { + "start": 85.92, + "end": 90.17, + "rms_db": -59.72, + "hf_share_6k": 0.01083828043192625 + }, + { + "start": 90.17, + "end": 94.42, + "rms_db": -62.12, + "hf_share_6k": 0.1411079317331314 + }, + { + "start": 94.42, + "end": 98.67, + "rms_db": -75.38, + "hf_share_6k": 0.011888228356838226 + }, + { + "start": 98.67, + "end": 102.92, + "rms_db": -75.76, + "hf_share_6k": 0.009531489573419094 + }, + { + "start": 102.92, + "end": 107.17, + "rms_db": -73.5, + "hf_share_6k": 0.024284925311803818 + }, + { + "start": 107.17, + "end": 111.42, + "rms_db": -61.35, + "hf_share_6k": 0.010785995051264763 + }, + { + "start": 111.42, + "end": 115.67, + "rms_db": -70.97, + "hf_share_6k": 0.004948402754962444 + }, + { + "start": 115.67, + "end": 119.92, + "rms_db": -81.51, + "hf_share_6k": 0.020108859986066818 + } + ], + "rms_db_5s": [ + -58.0, + -73.51, + -54.81, + -46.83, + -67.78, + -57.5, + -69.15, + -68.19, + -75.66, + -81.99, + -50.41, + -56.27, + -67.5, + -72.24, + -73.46, + -65.15, + -69.31, + -60.43, + -62.79, + -75.68, + -74.13, + -63.75, + -65.55, + -79.6 + ] + } + }, + "40bbb40c14ec": { + "scene": "s1-static", + "moments": [ + 62, + 300, + 583, + 1000, + 1501, + 2000, + 2500, + 2868 + ], + "scene_bands": [ + { + "fine": 6.4929, + "mid": 14.0403, + "coarse": 43.9452 + }, + { + "fine": 7.1952, + "mid": 14.8287, + "coarse": 44.5319 + }, + { + "fine": 7.6919, + "mid": 15.4196, + "coarse": 44.195 + }, + { + "fine": 7.7377, + "mid": 15.5227, + "coarse": 43.4151 + }, + { + "fine": 8.1943, + "mid": 15.4002, + "coarse": 41.9275 + }, + { + "fine": 8.7349, + "mid": 15.4845, + "coarse": 39.1683 + }, + { + "fine": 9.9225, + "mid": 16.3864, + "coarse": 36.3063 + }, + { + "fine": 10.2135, + "mid": 17.4333, + "coarse": 35.1161 + } + ], + "face_bands": [ + { + "fine": 11.1487, + "mid": 26.5163, + "coarse": 45.7453 + }, + { + "fine": 10.6245, + "mid": 29.4613, + "coarse": 44.8798 + }, + { + "fine": 11.1126, + "mid": 24.5809, + "coarse": 50.7815 + }, + { + "fine": 12.8515, + "mid": 26.4174, + "coarse": 49.6519 + }, + { + "fine": 12.3536, + "mid": 23.5034, + "coarse": 49.6168 + }, + { + "fine": 11.662, + "mid": 23.2221, + "coarse": 46.787 + }, + { + "fine": 9.3558, + "mid": 16.9744, + "coarse": 32.0361 + }, + { + "fine": 11.6992, + "mid": 23.7714, + "coarse": 36.944 + } + ], + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 0.73, + "look_ratio": 1.82, + "level_step_db": 7.21, + "typical_delta": 2.0 + }, + { + "seconds": 9.417, + "jump_ratio": 0.72, + "look_ratio": 1.37, + "level_step_db": 15.47, + "typical_delta": 2.0 + }, + { + "seconds": 13.667, + "jump_ratio": 0.78, + "look_ratio": 0.67, + "level_step_db": 6.74, + "typical_delta": 2.0 + }, + { + "seconds": 17.917, + "jump_ratio": 0.9, + "look_ratio": 1.59, + "level_step_db": 10.91, + "typical_delta": 2.0 + }, + { + "seconds": 22.167, + "jump_ratio": 0.74, + "look_ratio": 0.97, + "level_step_db": 2.56, + "typical_delta": 2.0 + }, + { + "seconds": 26.417, + "jump_ratio": 0.8, + "look_ratio": 1.23, + "level_step_db": 7.57, + "typical_delta": 2.0 + }, + { + "seconds": 30.667, + "jump_ratio": 0.92, + "look_ratio": 1.59, + "level_step_db": 19.19, + "typical_delta": 2.0 + }, + { + "seconds": 34.917, + "jump_ratio": 0.92, + "look_ratio": 1.27, + "level_step_db": 2.69, + "typical_delta": 2.0 + }, + { + "seconds": 39.167, + "jump_ratio": 0.93, + "look_ratio": 1.25, + "level_step_db": 6.12, + "typical_delta": 2.0 + }, + { + "seconds": 43.417, + "jump_ratio": 0.83, + "look_ratio": 0.61, + "level_step_db": 20.0, + "typical_delta": 2.0 + }, + { + "seconds": 47.667, + "jump_ratio": 0.95, + "look_ratio": 0.96, + "level_step_db": 34.44, + "typical_delta": 2.0 + }, + { + "seconds": 51.917, + "jump_ratio": 0.82, + "look_ratio": 0.8, + "level_step_db": 3.68, + "typical_delta": 2.0 + }, + { + "seconds": 56.167, + "jump_ratio": 0.87, + "look_ratio": 0.87, + "level_step_db": 12.14, + "typical_delta": 2.0 + }, + { + "seconds": 60.417, + "jump_ratio": 0.71, + "look_ratio": 0.65, + "level_step_db": 0.67, + "typical_delta": 2.0 + }, + { + "seconds": 64.667, + "jump_ratio": 0.82, + "look_ratio": 0.86, + "level_step_db": 23.81, + "typical_delta": 2.17 + }, + { + "seconds": 68.917, + "jump_ratio": 0.79, + "look_ratio": 0.72, + "level_step_db": 20.91, + "typical_delta": 2.17 + }, + { + "seconds": 73.167, + "jump_ratio": 0.69, + "look_ratio": 0.54, + "level_step_db": 27.89, + "typical_delta": 2.16 + }, + { + "seconds": 77.417, + "jump_ratio": 0.78, + "look_ratio": 0.91, + "level_step_db": 22.48, + "typical_delta": 2.16 + }, + { + "seconds": 81.667, + "jump_ratio": 0.94, + "look_ratio": 0.76, + "level_step_db": 3.55, + "typical_delta": 2.02 + }, + { + "seconds": 85.917, + "jump_ratio": 1.03, + "look_ratio": 1.41, + "level_step_db": 4.08, + "typical_delta": 2.07 + }, + { + "seconds": 90.167, + "jump_ratio": 0.69, + "look_ratio": 0.6, + "level_step_db": 3.46, + "typical_delta": 2.2 + }, + { + "seconds": 94.417, + "jump_ratio": 0.81, + "look_ratio": 0.97, + "level_step_db": 4.09, + "typical_delta": 2.2 + }, + { + "seconds": 98.667, + "jump_ratio": 0.9, + "look_ratio": 0.98, + "level_step_db": 2.36, + "typical_delta": 2.26 + }, + { + "seconds": 102.917, + "jump_ratio": 0.96, + "look_ratio": 1.22, + "level_step_db": 3.07, + "typical_delta": 2.26 + }, + { + "seconds": 107.167, + "jump_ratio": 0.82, + "look_ratio": 0.84, + "level_step_db": 1.02, + "typical_delta": 2.4 + }, + { + "seconds": 111.417, + "jump_ratio": 0.88, + "look_ratio": 1.54, + "level_step_db": 3.0, + "typical_delta": 2.4 + }, + { + "seconds": 115.667, + "jump_ratio": 0.84, + "look_ratio": 0.58, + "level_step_db": 2.84, + "typical_delta": 2.32 + }, + { + "seconds": 119.917, + "jump_ratio": 0.71, + "look_ratio": 1.02, + "level_step_db": 3.0, + "typical_delta": 2.32 + } + ], + "motion_mean": 0.6840914487838745, + "audio": { + "duration_s": 120.0, + "segments": [ + { + "start": 0.0, + "end": 5.17, + "rms_db": -60.67, + "hf_share_6k": 0.03016573004424572 + }, + { + "start": 5.17, + "end": 9.42, + "rms_db": -53.68, + "hf_share_6k": 0.009892716072499752 + }, + { + "start": 9.42, + "end": 13.67, + "rms_db": -68.84, + "hf_share_6k": 0.027880342677235603 + }, + { + "start": 13.67, + "end": 17.92, + "rms_db": -62.27, + "hf_share_6k": 0.004456851631402969 + }, + { + "start": 17.92, + "end": 22.17, + "rms_db": -73.27, + "hf_share_6k": 0.01570272259414196 + }, + { + "start": 22.17, + "end": 26.42, + "rms_db": -75.7, + "hf_share_6k": 0.014662536792457104 + }, + { + "start": 26.42, + "end": 30.67, + "rms_db": -67.99, + "hf_share_6k": 0.24210244417190552 + }, + { + "start": 30.67, + "end": 34.92, + "rms_db": -48.85, + "hf_share_6k": 0.022705992683768272 + }, + { + "start": 34.92, + "end": 39.17, + "rms_db": -51.28, + "hf_share_6k": 0.00047922125668264925 + }, + { + "start": 39.17, + "end": 43.42, + "rms_db": -57.53, + "hf_share_6k": 0.008671157993376255 + }, + { + "start": 43.42, + "end": 47.67, + "rms_db": -37.49, + "hf_share_6k": 0.013206729665398598 + }, + { + "start": 47.67, + "end": 51.92, + "rms_db": -72.5, + "hf_share_6k": 0.004529087338596582 + }, + { + "start": 51.92, + "end": 56.17, + "rms_db": -68.55, + "hf_share_6k": 0.02082059159874916 + }, + { + "start": 56.17, + "end": 60.42, + "rms_db": -82.67, + "hf_share_6k": 0.020475823432207108 + }, + { + "start": 60.42, + "end": 64.67, + "rms_db": -82.08, + "hf_share_6k": 0.015159304253757 + }, + { + "start": 64.67, + "end": 68.92, + "rms_db": -56.54, + "hf_share_6k": 0.01848381757736206 + }, + { + "start": 68.92, + "end": 73.17, + "rms_db": -79.63, + "hf_share_6k": 0.007748011499643326 + }, + { + "start": 73.17, + "end": 77.42, + "rms_db": -49.42, + "hf_share_6k": 0.020682286471128464 + }, + { + "start": 77.42, + "end": 81.67, + "rms_db": -74.23, + "hf_share_6k": 0.002325694542378187 + }, + { + "start": 81.67, + "end": 85.92, + "rms_db": -70.75, + "hf_share_6k": 0.001076534390449524 + }, + { + "start": 85.92, + "end": 90.17, + "rms_db": -66.66, + "hf_share_6k": 0.00042442206176929176 + }, + { + "start": 90.17, + "end": 94.42, + "rms_db": -63.18, + "hf_share_6k": 0.00021736354392487556 + }, + { + "start": 94.42, + "end": 98.67, + "rms_db": -59.13, + "hf_share_6k": 0.0001344819029327482 + }, + { + "start": 98.67, + "end": 102.92, + "rms_db": -56.62, + "hf_share_6k": 0.001763185835443437 + }, + { + "start": 102.92, + "end": 107.17, + "rms_db": -53.65, + "hf_share_6k": 0.0343216136097908 + }, + { + "start": 107.17, + "end": 111.42, + "rms_db": -54.67, + "hf_share_6k": 0.08567378669977188 + }, + { + "start": 111.42, + "end": 115.67, + "rms_db": -57.67, + "hf_share_6k": 0.36601194739341736 + }, + { + "start": 115.67, + "end": 119.92, + "rms_db": -54.82, + "hf_share_6k": 0.35388660430908203 + } + ], + "rms_db_5s": [ + -60.58, + -54.35, + -69.59, + -63.11, + -73.08, + -68.54, + -47.62, + -69.61, + -55.39, + -38.23, + -69.25, + -83.04, + -82.02, + -57.24, + -50.13, + -75.61, + -71.66, + -67.19, + -62.27, + -58.57, + -55.22, + -53.27, + -58.43, + -55.08 + ] + } + }, + "05a3a55b043c": { + "scene": "s1-static", + "moments": [ + 62, + 300, + 583, + 1000, + 1501, + 2000, + 2500, + 2868 + ], + "scene_bands": [ + { + "fine": 6.5469, + "mid": 13.9587, + "coarse": 43.751 + }, + { + "fine": 7.6743, + "mid": 14.8046, + "coarse": 43.9463 + }, + { + "fine": 8.6918, + "mid": 15.5581, + "coarse": 45.3711 + }, + { + "fine": 10.0112, + "mid": 16.0067, + "coarse": 45.443 + }, + { + "fine": 10.352, + "mid": 15.2055, + "coarse": 45.8283 + }, + { + "fine": 7.6588, + "mid": 15.4112, + "coarse": 45.7376 + }, + { + "fine": 7.154, + "mid": 14.6413, + "coarse": 43.0275 + }, + { + "fine": 6.7397, + "mid": 14.1825, + "coarse": 43.066 + } + ], + "face_bands": [ + { + "fine": 10.8423, + "mid": 24.3037, + "coarse": 43.8552 + }, + { + "fine": 11.1948, + "mid": 27.5142, + "coarse": 45.7543 + }, + { + "fine": 12.6925, + "mid": 25.5163, + "coarse": 45.355 + }, + { + "fine": 10.898, + "mid": 25.1701, + "coarse": 45.3931 + }, + { + "fine": 10.9868, + "mid": 29.3139, + "coarse": 51.2824 + }, + { + "fine": 8.9637, + "mid": 33.2102, + "coarse": 55.2199 + }, + { + "fine": 8.3189, + "mid": 34.8274, + "coarse": 54.1738 + }, + { + "fine": 8.6457, + "mid": 36.9478, + "coarse": 53.5859 + } + ], + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 0.71, + "look_ratio": 1.88, + "level_step_db": 0.58, + "typical_delta": 2.0 + }, + { + "seconds": 9.417, + "jump_ratio": 1.01, + "look_ratio": 1.73, + "level_step_db": 0.81, + "typical_delta": 2.0 + }, + { + "seconds": 13.667, + "jump_ratio": 0.83, + "look_ratio": 0.84, + "level_step_db": 16.93, + "typical_delta": 2.0 + }, + { + "seconds": 17.917, + "jump_ratio": 0.82, + "look_ratio": 0.8, + "level_step_db": 2.79, + "typical_delta": 2.0 + }, + { + "seconds": 22.167, + "jump_ratio": 1.59, + "look_ratio": 2.4, + "level_step_db": 1.62, + "typical_delta": 2.0 + }, + { + "seconds": 26.417, + "jump_ratio": 1.04, + "look_ratio": 0.45, + "level_step_db": 18.9, + "typical_delta": 2.0 + }, + { + "seconds": 30.667, + "jump_ratio": 1.11, + "look_ratio": 0.8, + "level_step_db": 17.46, + "typical_delta": 2.0 + }, + { + "seconds": 34.917, + "jump_ratio": 0.83, + "look_ratio": 0.47, + "level_step_db": 6.08, + "typical_delta": 2.0 + }, + { + "seconds": 39.167, + "jump_ratio": 1.19, + "look_ratio": 1.54, + "level_step_db": 21.25, + "typical_delta": 2.0 + }, + { + "seconds": 43.417, + "jump_ratio": 0.81, + "look_ratio": 1.2, + "level_step_db": 24.8, + "typical_delta": 2.0 + }, + { + "seconds": 47.667, + "jump_ratio": 0.66, + "look_ratio": 1.23, + "level_step_db": 0.91, + "typical_delta": 2.0 + }, + { + "seconds": 51.917, + "jump_ratio": 1.09, + "look_ratio": 1.75, + "level_step_db": 15.77, + "typical_delta": 2.0 + }, + { + "seconds": 56.167, + "jump_ratio": 0.97, + "look_ratio": 1.68, + "level_step_db": 18.08, + "typical_delta": 2.0 + }, + { + "seconds": 60.417, + "jump_ratio": 0.8, + "look_ratio": 0.66, + "level_step_db": 4.02, + "typical_delta": 2.0 + }, + { + "seconds": 64.667, + "jump_ratio": 0.85, + "look_ratio": 1.11, + "level_step_db": 3.81, + "typical_delta": 2.0 + }, + { + "seconds": 68.917, + "jump_ratio": 1.32, + "look_ratio": 2.5, + "level_step_db": 6.87, + "typical_delta": 2.0 + }, + { + "seconds": 73.167, + "jump_ratio": 0.87, + "look_ratio": 1.32, + "level_step_db": 3.39, + "typical_delta": 2.0 + }, + { + "seconds": 77.417, + "jump_ratio": 0.69, + "look_ratio": 1.07, + "level_step_db": 0.13, + "typical_delta": 2.0 + }, + { + "seconds": 81.667, + "jump_ratio": 0.83, + "look_ratio": 0.93, + "level_step_db": 2.36, + "typical_delta": 2.0 + }, + { + "seconds": 85.917, + "jump_ratio": 0.71, + "look_ratio": 1.28, + "level_step_db": 0.11, + "typical_delta": 2.0 + }, + { + "seconds": 90.167, + "jump_ratio": 0.81, + "look_ratio": 0.35, + "level_step_db": 12.31, + "typical_delta": 2.0 + }, + { + "seconds": 94.417, + "jump_ratio": 0.82, + "look_ratio": 0.59, + "level_step_db": 14.89, + "typical_delta": 2.0 + }, + { + "seconds": 98.667, + "jump_ratio": 0.87, + "look_ratio": 0.28, + "level_step_db": 0.26, + "typical_delta": 2.0 + }, + { + "seconds": 102.917, + "jump_ratio": 0.99, + "look_ratio": 0.43, + "level_step_db": 1.78, + "typical_delta": 2.0 + }, + { + "seconds": 107.167, + "jump_ratio": 0.74, + "look_ratio": 0.49, + "level_step_db": 16.36, + "typical_delta": 2.0 + }, + { + "seconds": 111.417, + "jump_ratio": 1.21, + "look_ratio": 1.2, + "level_step_db": 6.97, + "typical_delta": 2.0 + }, + { + "seconds": 115.667, + "jump_ratio": 0.91, + "look_ratio": 2.39, + "level_step_db": 24.07, + "typical_delta": 2.0 + }, + { + "seconds": 119.917, + "jump_ratio": 0.75, + "look_ratio": 0.37, + "level_step_db": 28.75, + "typical_delta": 2.0 + } + ], + "motion_mean": 0.5307310819625854, + "audio": { + "duration_s": 120.0, + "segments": [ + { + "start": 0.0, + "end": 5.17, + "rms_db": -58.14, + "hf_share_6k": 0.014001276344060898 + }, + { + "start": 5.17, + "end": 9.42, + "rms_db": -57.83, + "hf_share_6k": 0.016166076064109802 + }, + { + "start": 9.42, + "end": 13.67, + "rms_db": -58.42, + "hf_share_6k": 0.01244423259049654 + }, + { + "start": 13.67, + "end": 17.92, + "rms_db": -75.82, + "hf_share_6k": 0.03693404793739319 + }, + { + "start": 17.92, + "end": 22.17, + "rms_db": -73.01, + "hf_share_6k": 0.009645718149840832 + }, + { + "start": 22.17, + "end": 26.42, + "rms_db": -74.19, + "hf_share_6k": 0.006302128080278635 + }, + { + "start": 26.42, + "end": 30.67, + "rms_db": -55.44, + "hf_share_6k": 0.0038116038776934147 + }, + { + "start": 30.67, + "end": 34.92, + "rms_db": -72.87, + "hf_share_6k": 0.01809520274400711 + }, + { + "start": 34.92, + "end": 39.17, + "rms_db": -79.02, + "hf_share_6k": 0.043241601437330246 + }, + { + "start": 39.17, + "end": 43.42, + "rms_db": -57.61, + "hf_share_6k": 0.15712276101112366 + }, + { + "start": 43.42, + "end": 47.67, + "rms_db": -82.44, + "hf_share_6k": 0.06108441576361656 + }, + { + "start": 47.67, + "end": 51.92, + "rms_db": -81.41, + "hf_share_6k": 0.05192231014370918 + }, + { + "start": 51.92, + "end": 56.17, + "rms_db": -65.52, + "hf_share_6k": 0.08772318065166473 + }, + { + "start": 56.17, + "end": 60.42, + "rms_db": -83.94, + "hf_share_6k": 0.06634976714849472 + }, + { + "start": 60.42, + "end": 64.67, + "rms_db": -79.68, + "hf_share_6k": 0.03733133524656296 + }, + { + "start": 64.67, + "end": 68.92, + "rms_db": -83.7, + "hf_share_6k": 0.056715451180934906 + }, + { + "start": 68.92, + "end": 73.17, + "rms_db": -76.54, + "hf_share_6k": 0.013981024734675884 + }, + { + "start": 73.17, + "end": 77.42, + "rms_db": -80.03, + "hf_share_6k": 0.016401944682002068 + }, + { + "start": 77.42, + "end": 81.67, + "rms_db": -80.17, + "hf_share_6k": 0.01712898351252079 + }, + { + "start": 81.67, + "end": 85.92, + "rms_db": -77.85, + "hf_share_6k": 0.014040352776646614 + }, + { + "start": 85.92, + "end": 90.17, + "rms_db": -78.12, + "hf_share_6k": 0.009912832640111446 + }, + { + "start": 90.17, + "end": 94.42, + "rms_db": -65.66, + "hf_share_6k": 0.005588446743786335 + }, + { + "start": 94.42, + "end": 98.67, + "rms_db": -80.76, + "hf_share_6k": 0.02648787945508957 + }, + { + "start": 98.67, + "end": 102.92, + "rms_db": -80.96, + "hf_share_6k": 0.042571429163217545 + }, + { + "start": 102.92, + "end": 107.17, + "rms_db": -82.89, + "hf_share_6k": 0.06979598104953766 + }, + { + "start": 107.17, + "end": 111.42, + "rms_db": -66.15, + "hf_share_6k": 0.029536336660385132 + }, + { + "start": 111.42, + "end": 115.67, + "rms_db": -75.42, + "hf_share_6k": 0.044955238699913025 + }, + { + "start": 115.67, + "end": 119.92, + "rms_db": -49.04, + "hf_share_6k": 0.3939625322818756 + } + ], + "rms_db_5s": [ + -58.0, + -58.53, + -59.13, + -72.08, + -74.86, + -57.01, + -63.12, + -79.26, + -58.31, + -81.37, + -66.2, + -83.82, + -80.11, + -81.98, + -76.99, + -80.02, + -78.77, + -78.03, + -66.34, + -80.35, + -82.87, + -66.88, + -75.57, + -49.74 + ] + } + }, + "4766787e793d": { + "scene": "s1-static", + "moments": [ + 62, + 300, + 583, + 1000, + 1501, + 2000, + 2500, + 2868 + ], + "scene_bands": [ + { + "fine": 6.5469, + "mid": 13.9587, + "coarse": 43.751 + }, + { + "fine": 7.1253, + "mid": 14.5221, + "coarse": 43.7193 + }, + { + "fine": 7.3659, + "mid": 14.623, + "coarse": 42.8774 + }, + { + "fine": 6.9834, + "mid": 13.6851, + "coarse": 40.584 + }, + { + "fine": 6.8385, + "mid": 12.5109, + "coarse": 38.597 + }, + { + "fine": 7.0607, + "mid": 11.9064, + "coarse": 37.7641 + }, + { + "fine": 6.902, + "mid": 13.5115, + "coarse": 42.8368 + }, + { + "fine": 6.9428, + "mid": 13.8349, + "coarse": 42.5374 + } + ], + "face_bands": [ + { + "fine": 10.8423, + "mid": 24.3037, + "coarse": 43.8552 + }, + { + "fine": 10.9668, + "mid": 27.197, + "coarse": 46.0975 + }, + { + "fine": 13.0707, + "mid": 25.0944, + "coarse": 44.764 + }, + { + "fine": 14.6709, + "mid": 27.5565, + "coarse": 43.5186 + }, + { + "fine": 11.4423, + "mid": 28.4181, + "coarse": 51.1237 + }, + { + "fine": 9.9368, + "mid": 30.5456, + "coarse": 56.9304 + }, + { + "fine": 10.0452, + "mid": 29.7036, + "coarse": 58.3582 + }, + { + "fine": 9.7504, + "mid": 31.5128, + "coarse": 61.9121 + } + ], + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 0.85, + "look_ratio": 2.08, + "level_step_db": 8.7, + "typical_delta": 2.0 + }, + { + "seconds": 9.417, + "jump_ratio": 0.82, + "look_ratio": 1.53, + "level_step_db": 7.22, + "typical_delta": 2.0 + }, + { + "seconds": 13.667, + "jump_ratio": 0.74, + "look_ratio": 1.16, + "level_step_db": 27.51, + "typical_delta": 2.0 + }, + { + "seconds": 17.917, + "jump_ratio": 0.78, + "look_ratio": 1.27, + "level_step_db": 5.9, + "typical_delta": 2.0 + }, + { + "seconds": 22.167, + "jump_ratio": 1.13, + "look_ratio": 1.73, + "level_step_db": 7.31, + "typical_delta": 2.0 + }, + { + "seconds": 26.417, + "jump_ratio": 1.0, + "look_ratio": 1.47, + "level_step_db": 32.03, + "typical_delta": 2.0 + }, + { + "seconds": 30.667, + "jump_ratio": 0.81, + "look_ratio": 1.45, + "level_step_db": 15.12, + "typical_delta": 2.0 + }, + { + "seconds": 34.917, + "jump_ratio": 1.03, + "look_ratio": 1.49, + "level_step_db": 8.94, + "typical_delta": 2.0 + }, + { + "seconds": 39.167, + "jump_ratio": 0.75, + "look_ratio": 1.19, + "level_step_db": 5.67, + "typical_delta": 2.0 + }, + { + "seconds": 43.417, + "jump_ratio": 0.86, + "look_ratio": 1.62, + "level_step_db": 22.76, + "typical_delta": 2.0 + }, + { + "seconds": 47.667, + "jump_ratio": 0.7, + "look_ratio": 0.88, + "level_step_db": 20.97, + "typical_delta": 2.0 + }, + { + "seconds": 51.917, + "jump_ratio": 0.89, + "look_ratio": 1.76, + "level_step_db": 3.57, + "typical_delta": 2.0 + }, + { + "seconds": 56.167, + "jump_ratio": 0.85, + "look_ratio": 1.5, + "level_step_db": 6.94, + "typical_delta": 2.0 + }, + { + "seconds": 60.417, + "jump_ratio": 0.69, + "look_ratio": 1.04, + "level_step_db": 7.44, + "typical_delta": 2.0 + }, + { + "seconds": 64.667, + "jump_ratio": 1.04, + "look_ratio": 1.98, + "level_step_db": 14.39, + "typical_delta": 2.0 + }, + { + "seconds": 68.917, + "jump_ratio": 0.87, + "look_ratio": 1.64, + "level_step_db": 0.57, + "typical_delta": 2.0 + }, + { + "seconds": 73.167, + "jump_ratio": 0.82, + "look_ratio": 1.15, + "level_step_db": 10.43, + "typical_delta": 2.0 + }, + { + "seconds": 77.417, + "jump_ratio": 0.91, + "look_ratio": 1.56, + "level_step_db": 2.58, + "typical_delta": 2.0 + }, + { + "seconds": 81.667, + "jump_ratio": 0.85, + "look_ratio": 1.52, + "level_step_db": 1.9, + "typical_delta": 2.0 + }, + { + "seconds": 85.917, + "jump_ratio": 0.85, + "look_ratio": 1.53, + "level_step_db": 16.29, + "typical_delta": 2.0 + }, + { + "seconds": 90.167, + "jump_ratio": 1.0, + "look_ratio": 1.83, + "level_step_db": 13.85, + "typical_delta": 2.0 + }, + { + "seconds": 94.417, + "jump_ratio": 1.07, + "look_ratio": 2.02, + "level_step_db": 33.66, + "typical_delta": 2.05 + }, + { + "seconds": 98.667, + "jump_ratio": 0.59, + "look_ratio": 0.21, + "level_step_db": 24.15, + "typical_delta": 2.05 + }, + { + "seconds": 102.917, + "jump_ratio": 0.61, + "look_ratio": 0.85, + "level_step_db": 26.02, + "typical_delta": 2.0 + }, + { + "seconds": 107.167, + "jump_ratio": 0.61, + "look_ratio": 0.78, + "level_step_db": 18.24, + "typical_delta": 2.0 + }, + { + "seconds": 111.417, + "jump_ratio": 0.5, + "look_ratio": 0.41, + "level_step_db": 6.31, + "typical_delta": 2.0 + }, + { + "seconds": 115.667, + "jump_ratio": 0.63, + "look_ratio": 0.68, + "level_step_db": 5.85, + "typical_delta": 2.0 + }, + { + "seconds": 119.917, + "jump_ratio": 0.51, + "look_ratio": 0.45, + "level_step_db": 38.86, + "typical_delta": 2.0 + } + ], + "motion_mean": 0.5881861448287964, + "audio": { + "duration_s": 120.0, + "segments": [ + { + "start": 0.0, + "end": 5.17, + "rms_db": -58.14, + "hf_share_6k": 0.014001724310219288 + }, + { + "start": 5.17, + "end": 9.42, + "rms_db": -67.12, + "hf_share_6k": 0.014924993738532066 + }, + { + "start": 9.42, + "end": 13.67, + "rms_db": -59.89, + "hf_share_6k": 0.0014293893473222852 + }, + { + "start": 13.67, + "end": 17.92, + "rms_db": -87.77, + "hf_share_6k": 0.19439171254634857 + }, + { + "start": 17.92, + "end": 22.17, + "rms_db": -81.7, + "hf_share_6k": 0.038062985986471176 + }, + { + "start": 22.17, + "end": 26.42, + "rms_db": -89.07, + "hf_share_6k": 0.11328215897083282 + }, + { + "start": 26.42, + "end": 30.67, + "rms_db": -56.55, + "hf_share_6k": 0.015088650397956371 + }, + { + "start": 30.67, + "end": 34.92, + "rms_db": -71.92, + "hf_share_6k": 0.027334386482834816 + }, + { + "start": 34.92, + "end": 39.17, + "rms_db": -62.59, + "hf_share_6k": 0.0016415027203038335 + }, + { + "start": 39.17, + "end": 43.42, + "rms_db": -68.61, + "hf_share_6k": 0.03283584862947464 + }, + { + "start": 43.42, + "end": 47.67, + "rms_db": -91.58, + "hf_share_6k": 0.12183188647031784 + }, + { + "start": 47.67, + "end": 51.92, + "rms_db": -70.6, + "hf_share_6k": 0.0053401212207973 + }, + { + "start": 51.92, + "end": 56.17, + "rms_db": -73.64, + "hf_share_6k": 0.10200407356023788 + }, + { + "start": 56.17, + "end": 60.42, + "rms_db": -66.76, + "hf_share_6k": 0.049309100955724716 + }, + { + "start": 60.42, + "end": 64.67, + "rms_db": -74.3, + "hf_share_6k": 0.02561621367931366 + }, + { + "start": 64.67, + "end": 68.92, + "rms_db": -89.21, + "hf_share_6k": 0.10272756963968277 + }, + { + "start": 68.92, + "end": 73.17, + "rms_db": -88.93, + "hf_share_6k": 0.0959198921918869 + }, + { + "start": 73.17, + "end": 77.42, + "rms_db": -78.11, + "hf_share_6k": 0.029228532686829567 + }, + { + "start": 77.42, + "end": 81.67, + "rms_db": -80.98, + "hf_share_6k": 0.016334734857082367 + }, + { + "start": 81.67, + "end": 85.92, + "rms_db": -78.65, + "hf_share_6k": 0.02340671420097351 + }, + { + "start": 85.92, + "end": 90.17, + "rms_db": -62.69, + "hf_share_6k": 0.06464733183383942 + }, + { + "start": 90.17, + "end": 94.42, + "rms_db": -48.42, + "hf_share_6k": 0.019401414319872856 + }, + { + "start": 94.42, + "end": 98.67, + "rms_db": -82.04, + "hf_share_6k": 0.11317678540945053 + }, + { + "start": 98.67, + "end": 102.92, + "rms_db": -58.18, + "hf_share_6k": 0.12825606763362885 + }, + { + "start": 102.92, + "end": 107.17, + "rms_db": -84.26, + "hf_share_6k": 0.10631091892719269 + }, + { + "start": 107.17, + "end": 111.42, + "rms_db": -66.24, + "hf_share_6k": 0.008670108392834663 + }, + { + "start": 111.42, + "end": 115.67, + "rms_db": -59.63, + "hf_share_6k": 0.1768902838230133 + }, + { + "start": 115.67, + "end": 119.92, + "rms_db": -53.5, + "hf_share_6k": 0.33101049065589905 + } + ], + "rms_db_5s": [ + -58.0, + -67.75, + -60.59, + -84.43, + -84.05, + -57.26, + -72.58, + -63.29, + -69.3, + -71.31, + -74.43, + -67.44, + -74.93, + -91.33, + -88.73, + -77.19, + -79.29, + -63.39, + -49.12, + -68.46, + -59.36, + -66.92, + -60.36, + -54.2 + ] + } + }, + "20ce09d60040": { + "scene": "s1-static", + "moments": [ + 62, + 300, + 583, + 1000, + 1501, + 2000, + 2500, + 2868 + ], + "scene_bands": [ + { + "fine": 6.5469, + "mid": 13.9587, + "coarse": 43.751 + }, + { + "fine": 7.4715, + "mid": 14.7611, + "coarse": 43.9759 + }, + { + "fine": 8.1991, + "mid": 16.077, + "coarse": 44.1284 + }, + { + "fine": 11.3121, + "mid": 18.8467, + "coarse": 40.6958 + }, + { + "fine": 17.737, + "mid": 23.828, + "coarse": 33.6703 + }, + { + "fine": 21.8832, + "mid": 28.8235, + "coarse": 26.9845 + }, + { + "fine": 16.5973, + "mid": 26.6741, + "coarse": 40.0963 + }, + { + "fine": 19.7582, + "mid": 29.2688, + "coarse": 37.7745 + } + ], + "face_bands": [ + { + "fine": 10.8423, + "mid": 24.3037, + "coarse": 43.8552 + }, + { + "fine": 11.4082, + "mid": 27.3329, + "coarse": 45.1862 + }, + { + "fine": 15.0773, + "mid": 27.8434, + "coarse": 44.9644 + }, + { + "fine": 18.5019, + "mid": 35.8774, + "coarse": 50.133 + }, + { + "fine": 21.3871, + "mid": 41.0272, + "coarse": 52.4473 + }, + { + "fine": 20.0571, + "mid": 36.0468, + "coarse": 51.0695 + }, + { + "fine": 21.6477, + "mid": 50.9089, + "coarse": 50.034 + }, + { + "fine": 22.8502, + "mid": 48.2119, + "coarse": 55.4402 + } + ], + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 0.79, + "look_ratio": 2.49, + "level_step_db": 14.97, + "typical_delta": 2.0 + }, + { + "seconds": 9.417, + "jump_ratio": 0.86, + "look_ratio": 1.65, + "level_step_db": 17.35, + "typical_delta": 2.0 + }, + { + "seconds": 13.667, + "jump_ratio": 0.85, + "look_ratio": 1.17, + "level_step_db": 8.62, + "typical_delta": 2.0 + }, + { + "seconds": 17.917, + "jump_ratio": 0.88, + "look_ratio": 1.16, + "level_step_db": 17.02, + "typical_delta": 2.0 + }, + { + "seconds": 22.167, + "jump_ratio": 1.21, + "look_ratio": 1.77, + "level_step_db": 17.43, + "typical_delta": 2.0 + }, + { + "seconds": 26.417, + "jump_ratio": 0.87, + "look_ratio": 0.51, + "level_step_db": 19.6, + "typical_delta": 2.0 + }, + { + "seconds": 30.667, + "jump_ratio": 0.73, + "look_ratio": 0.45, + "level_step_db": 19.6, + "typical_delta": 2.0 + }, + { + "seconds": 34.917, + "jump_ratio": 0.79, + "look_ratio": 0.57, + "level_step_db": 7.09, + "typical_delta": 2.0 + }, + { + "seconds": 39.167, + "jump_ratio": 0.76, + "look_ratio": 0.7, + "level_step_db": 11.57, + "typical_delta": 2.0 + }, + { + "seconds": 43.417, + "jump_ratio": 0.82, + "look_ratio": 0.71, + "level_step_db": 17.05, + "typical_delta": 2.0 + }, + { + "seconds": 47.667, + "jump_ratio": 1.09, + "look_ratio": 1.8, + "level_step_db": 6.64, + "typical_delta": 2.0 + }, + { + "seconds": 51.917, + "jump_ratio": 0.78, + "look_ratio": 0.9, + "level_step_db": 29.86, + "typical_delta": 2.0 + }, + { + "seconds": 56.167, + "jump_ratio": 0.95, + "look_ratio": 0.76, + "level_step_db": 37.6, + "typical_delta": 2.0 + }, + { + "seconds": 60.417, + "jump_ratio": 0.92, + "look_ratio": 1.09, + "level_step_db": 13.4, + "typical_delta": 2.0 + }, + { + "seconds": 64.667, + "jump_ratio": 0.89, + "look_ratio": 1.12, + "level_step_db": 6.13, + "typical_delta": 2.0 + }, + { + "seconds": 68.917, + "jump_ratio": 0.97, + "look_ratio": 0.8, + "level_step_db": 6.18, + "typical_delta": 2.0 + }, + { + "seconds": 73.167, + "jump_ratio": 1.16, + "look_ratio": 1.55, + "level_step_db": 0.55, + "typical_delta": 2.0 + }, + { + "seconds": 77.417, + "jump_ratio": 0.88, + "look_ratio": 0.76, + "level_step_db": 22.41, + "typical_delta": 2.0 + }, + { + "seconds": 81.667, + "jump_ratio": 0.97, + "look_ratio": 0.61, + "level_step_db": 10.12, + "typical_delta": 2.0 + }, + { + "seconds": 85.917, + "jump_ratio": 0.86, + "look_ratio": 0.67, + "level_step_db": 0.13, + "typical_delta": 2.0 + }, + { + "seconds": 90.167, + "jump_ratio": 0.96, + "look_ratio": 0.75, + "level_step_db": 4.44, + "typical_delta": 2.0 + }, + { + "seconds": 94.417, + "jump_ratio": 1.49, + "look_ratio": 2.36, + "level_step_db": 19.7, + "typical_delta": 2.0 + }, + { + "seconds": 98.667, + "jump_ratio": 0.91, + "look_ratio": 0.48, + "level_step_db": 7.75, + "typical_delta": 2.0 + }, + { + "seconds": 102.917, + "jump_ratio": 0.97, + "look_ratio": 0.39, + "level_step_db": 8.5, + "typical_delta": 2.0 + }, + { + "seconds": 107.167, + "jump_ratio": 1.02, + "look_ratio": 0.49, + "level_step_db": 1.98, + "typical_delta": 2.0 + }, + { + "seconds": 111.417, + "jump_ratio": 0.93, + "look_ratio": 1.09, + "level_step_db": 1.51, + "typical_delta": 2.0 + }, + { + "seconds": 115.667, + "jump_ratio": 1.04, + "look_ratio": 1.05, + "level_step_db": 5.42, + "typical_delta": 2.0 + }, + { + "seconds": 119.917, + "jump_ratio": 0.76, + "look_ratio": 0.61, + "level_step_db": 6.16, + "typical_delta": 2.0 + } + ], + "motion_mean": 0.48988500237464905, + "audio": { + "duration_s": 120.0, + "segments": [ + { + "start": 0.0, + "end": 5.17, + "rms_db": -58.14, + "hf_share_6k": 0.014000915922224522 + }, + { + "start": 5.17, + "end": 9.42, + "rms_db": -73.11, + "hf_share_6k": 0.04992246255278587 + }, + { + "start": 9.42, + "end": 13.67, + "rms_db": -55.81, + "hf_share_6k": 0.0023110518231987953 + }, + { + "start": 13.67, + "end": 17.92, + "rms_db": -64.64, + "hf_share_6k": 0.001181543106213212 + }, + { + "start": 17.92, + "end": 22.17, + "rms_db": -47.39, + "hf_share_6k": 0.0020101997070014477 + }, + { + "start": 22.17, + "end": 26.42, + "rms_db": -64.91, + "hf_share_6k": 0.0005280057666823268 + }, + { + "start": 26.42, + "end": 30.67, + "rms_db": -45.24, + "hf_share_6k": 0.0031668690498918295 + }, + { + "start": 30.67, + "end": 34.92, + "rms_db": -64.93, + "hf_share_6k": 0.007594342343509197 + }, + { + "start": 34.92, + "end": 39.17, + "rms_db": -72.1, + "hf_share_6k": 0.004199942573904991 + }, + { + "start": 39.17, + "end": 43.42, + "rms_db": -60.71, + "hf_share_6k": 0.031726378947496414 + }, + { + "start": 43.42, + "end": 47.67, + "rms_db": -77.45, + "hf_share_6k": 0.012090765871107578 + }, + { + "start": 47.67, + "end": 51.92, + "rms_db": -84.29, + "hf_share_6k": 0.0561225451529026 + }, + { + "start": 51.92, + "end": 56.17, + "rms_db": -54.68, + "hf_share_6k": 0.09900829941034317 + }, + { + "start": 56.17, + "end": 60.42, + "rms_db": -92.24, + "hf_share_6k": 0.11317753791809082 + }, + { + "start": 60.42, + "end": 64.67, + "rms_db": -78.34, + "hf_share_6k": 0.020349256694316864 + }, + { + "start": 64.67, + "end": 68.92, + "rms_db": -72.19, + "hf_share_6k": 0.003009866690263152 + }, + { + "start": 68.92, + "end": 73.17, + "rms_db": -78.61, + "hf_share_6k": 0.014448754489421844 + }, + { + "start": 73.17, + "end": 77.42, + "rms_db": -79.11, + "hf_share_6k": 0.013767893426120281 + }, + { + "start": 77.42, + "end": 81.67, + "rms_db": -56.73, + "hf_share_6k": 0.043877195566892624 + }, + { + "start": 81.67, + "end": 85.92, + "rms_db": -66.62, + "hf_share_6k": 0.0002610864758025855 + }, + { + "start": 85.92, + "end": 90.17, + "rms_db": -66.71, + "hf_share_6k": 0.024603787809610367 + }, + { + "start": 90.17, + "end": 94.42, + "rms_db": -62.66, + "hf_share_6k": 0.04343166574835777 + }, + { + "start": 94.42, + "end": 98.67, + "rms_db": -82.29, + "hf_share_6k": 0.04759487137198448 + }, + { + "start": 98.67, + "end": 102.92, + "rms_db": -74.34, + "hf_share_6k": 0.03825734928250313 + }, + { + "start": 102.92, + "end": 107.17, + "rms_db": -82.95, + "hf_share_6k": 0.04010206088423729 + }, + { + "start": 107.17, + "end": 111.42, + "rms_db": -81.18, + "hf_share_6k": 0.024821022525429726 + }, + { + "start": 111.42, + "end": 115.67, + "rms_db": -79.86, + "hf_share_6k": 0.013077710755169392 + }, + { + "start": 115.67, + "end": 119.92, + "rms_db": -85.14, + "hf_share_6k": 0.012541242875158787 + } + ], + "rms_db_5s": [ + -58.0, + -73.51, + -56.52, + -48.05, + -64.26, + -45.94, + -65.63, + -72.8, + -61.34, + -83.53, + -55.61, + -68.46, + -79.03, + -72.88, + -79.07, + -57.41, + -67.32, + -67.41, + -63.36, + -76.3, + -78.65, + -80.86, + -80.0, + -85.6 + ] + } + }, + "dedd341c9108": { + "scene": "s2-arc", + "moments": [ + 62, + 277, + 583 + ], + "scene_bands": [ + { + "fine": 16.1608, + "mid": 24.2724, + "coarse": 33.6887 + }, + { + "fine": 6.1017, + "mid": 17.6053, + "coarse": 56.4104 + }, + { + "fine": 0.7398, + "mid": 2.4501, + "coarse": 8.0638 + } + ], + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 2.09, + "look_ratio": 1.09, + "level_step_db": 1.46, + "typical_delta": 2.83 + }, + { + "seconds": 9.417, + "jump_ratio": 1.26, + "look_ratio": 1.74, + "level_step_db": 3.43, + "typical_delta": 3.46 + }, + { + "seconds": 13.667, + "jump_ratio": 1.17, + "look_ratio": 1.4, + "level_step_db": 0.23, + "typical_delta": 4.86 + }, + { + "seconds": 17.917, + "jump_ratio": 1.17, + "look_ratio": 1.33, + "level_step_db": 0.34, + "typical_delta": 6.43 + }, + { + "seconds": 22.167, + "jump_ratio": 1.12, + "look_ratio": 0.87, + "level_step_db": 1.85, + "typical_delta": 8.78 + } + ], + "motion_mean": 4.160738945007324, + "audio": { + "duration_s": 26.432, + "segments": [ + { + "start": 0.0, + "end": 5.17, + "rms_db": -50.43, + "hf_share_6k": 0.007636647671461105 + }, + { + "start": 5.17, + "end": 9.42, + "rms_db": -52.07, + "hf_share_6k": 0.0048231338150799274 + }, + { + "start": 9.42, + "end": 13.67, + "rms_db": -55.43, + "hf_share_6k": 0.0007750485674478114 + }, + { + "start": 13.67, + "end": 17.92, + "rms_db": -55.57, + "hf_share_6k": 0.0002021346299443394 + }, + { + "start": 17.92, + "end": 22.17, + "rms_db": -55.19, + "hf_share_6k": 0.0001378381421091035 + }, + { + "start": 22.17, + "end": 26.43, + "rms_db": -57.15, + "hf_share_6k": 9.783898713067174e-05 + } + ], + "rms_db_5s": [ + -50.55, + -52.05, + -55.7, + -55.71, + -55.77, + -56.99 + ] + } + }, + "01146e925c94": { + "scene": "s2-arc", + "moments": [ + 62, + 277, + 583 + ], + "scene_bands": [ + { + "fine": 15.0557, + "mid": 20.411, + "coarse": 22.5499 + }, + { + "fine": 5.4029, + "mid": 18.3162, + "coarse": 60.9712 + }, + { + "fine": 0.3763, + "mid": 0.6141, + "coarse": 1.0259 + } + ], + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 1.86, + "look_ratio": 2.01, + "level_step_db": 0.9, + "typical_delta": 3.1 + }, + { + "seconds": 9.417, + "jump_ratio": 1.25, + "look_ratio": 1.12, + "level_step_db": 3.33, + "typical_delta": 3.86 + }, + { + "seconds": 13.667, + "jump_ratio": 1.13, + "look_ratio": 0.58, + "level_step_db": 6.54, + "typical_delta": 5.9 + }, + { + "seconds": 17.917, + "jump_ratio": 0.94, + "look_ratio": 1.24, + "level_step_db": 8.97, + "typical_delta": 8.94 + }, + { + "seconds": 22.167, + "jump_ratio": 1.18, + "look_ratio": 1.14, + "level_step_db": 1.52, + "typical_delta": 8.94 + } + ], + "motion_mean": 4.396059989929199, + "audio": { + "duration_s": 26.432, + "segments": [ + { + "start": 0.0, + "end": 5.17, + "rms_db": -45.1, + "hf_share_6k": 0.00044890973367728293 + }, + { + "start": 5.17, + "end": 9.42, + "rms_db": -44.38, + "hf_share_6k": 8.141966100083664e-05 + }, + { + "start": 9.42, + "end": 13.67, + "rms_db": -40.87, + "hf_share_6k": 1.0958671737171244e-05 + }, + { + "start": 13.67, + "end": 17.92, + "rms_db": -47.45, + "hf_share_6k": 2.7156278520124033e-05 + }, + { + "start": 17.92, + "end": 22.17, + "rms_db": -56.49, + "hf_share_6k": 0.00016959257482085377 + }, + { + "start": 22.17, + "end": 26.43, + "rms_db": -58.1, + "hf_share_6k": 0.00022172830358613282 + } + ], + "rms_db_5s": [ + -45.16, + -43.73, + -41.6, + -51.45, + -58.84, + -56.54 + ] + } + }, + "536548e14dd5": { + "scene": "s2-arc", + "moments": [ + 62, + 277, + 583 + ], + "scene_bands": [ + { + "fine": 16.1608, + "mid": 24.2724, + "coarse": 33.6887 + }, + { + "fine": 6.1017, + "mid": 17.6053, + "coarse": 56.4104 + }, + { + "fine": 0.7398, + "mid": 2.4501, + "coarse": 8.0638 + } + ], + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 2.09, + "look_ratio": 1.09, + "level_step_db": 1.46, + "typical_delta": 2.83 + }, + { + "seconds": 9.417, + "jump_ratio": 1.26, + "look_ratio": 1.74, + "level_step_db": 3.43, + "typical_delta": 3.46 + }, + { + "seconds": 13.667, + "jump_ratio": 1.17, + "look_ratio": 1.4, + "level_step_db": 0.23, + "typical_delta": 4.86 + }, + { + "seconds": 17.917, + "jump_ratio": 1.17, + "look_ratio": 1.33, + "level_step_db": 0.34, + "typical_delta": 6.43 + }, + { + "seconds": 22.167, + "jump_ratio": 1.12, + "look_ratio": 0.87, + "level_step_db": 1.85, + "typical_delta": 8.78 + } + ], + "motion_mean": 4.160738945007324, + "audio": { + "duration_s": 26.432, + "segments": [ + { + "start": 0.0, + "end": 5.17, + "rms_db": -50.43, + "hf_share_6k": 0.007636647671461105 + }, + { + "start": 5.17, + "end": 9.42, + "rms_db": -52.07, + "hf_share_6k": 0.0048231338150799274 + }, + { + "start": 9.42, + "end": 13.67, + "rms_db": -55.43, + "hf_share_6k": 0.0007750485674478114 + }, + { + "start": 13.67, + "end": 17.92, + "rms_db": -55.57, + "hf_share_6k": 0.0002021346299443394 + }, + { + "start": 17.92, + "end": 22.17, + "rms_db": -55.19, + "hf_share_6k": 0.0001378381421091035 + }, + { + "start": 22.17, + "end": 26.43, + "rms_db": -57.15, + "hf_share_6k": 9.783898713067174e-05 + } + ], + "rms_db_5s": [ + -50.55, + -52.05, + -55.7, + -55.71, + -55.77, + -56.99 + ] + } + }, + "1b8edfdba405": { + "scene": "s2-arc", + "moments": [ + 62, + 277, + 583 + ], + "scene_bands": [ + { + "fine": 15.0557, + "mid": 20.411, + "coarse": 22.5499 + }, + { + "fine": 6.0813, + "mid": 17.3078, + "coarse": 57.7195 + }, + { + "fine": 0.4466, + "mid": 1.022, + "coarse": 1.7693 + } + ], + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 1.83, + "look_ratio": 1.74, + "level_step_db": 1.04, + "typical_delta": 3.14 + }, + { + "seconds": 9.417, + "jump_ratio": 1.2, + "look_ratio": 1.43, + "level_step_db": 3.89, + "typical_delta": 4.16 + }, + { + "seconds": 13.667, + "jump_ratio": 1.07, + "look_ratio": 0.79, + "level_step_db": 8.43, + "typical_delta": 5.71 + }, + { + "seconds": 17.917, + "jump_ratio": 0.97, + "look_ratio": 1.36, + "level_step_db": 12.38, + "typical_delta": 8.53 + }, + { + "seconds": 22.167, + "jump_ratio": 1.46, + "look_ratio": 1.77, + "level_step_db": 6.97, + "typical_delta": 8.53 + } + ], + "motion_mean": 4.443963527679443, + "audio": { + "duration_s": 26.432, + "segments": [ + { + "start": 0.0, + "end": 5.17, + "rms_db": -45.1, + "hf_share_6k": 0.00044880103087052703 + }, + { + "start": 5.17, + "end": 9.42, + "rms_db": -44.24, + "hf_share_6k": 8.664214692544192e-05 + }, + { + "start": 9.42, + "end": 13.67, + "rms_db": -40.17, + "hf_share_6k": 1.2566936675284524e-05 + }, + { + "start": 13.67, + "end": 17.92, + "rms_db": -48.64, + "hf_share_6k": 3.770463081309572e-05 + }, + { + "start": 17.92, + "end": 22.17, + "rms_db": -61.29, + "hf_share_6k": 0.0005221444298513234 + }, + { + "start": 22.17, + "end": 26.43, + "rms_db": -54.26, + "hf_share_6k": 9.285019768867642e-05 + } + ], + "rms_db_5s": [ + -45.16, + -43.46, + -41.07, + -54.07, + -59.38, + -50.84 + ] + } + }, + "d93adb33249e": { + "scene": "s2-arc", + "moments": [ + 62, + 277, + 583 + ], + "scene_bands": [ + { + "fine": 16.1608, + "mid": 24.2724, + "coarse": 33.6887 + }, + { + "fine": 8.3841, + "mid": 17.3943, + "coarse": 55.0524 + }, + { + "fine": 0.4306, + "mid": 0.6325, + "coarse": 0.9814 + } + ], + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 2.45, + "look_ratio": 2.43, + "level_step_db": 1.43, + "typical_delta": 2.82 + }, + { + "seconds": 9.417, + "jump_ratio": 1.88, + "look_ratio": 2.75, + "level_step_db": 2.05, + "typical_delta": 3.36 + }, + { + "seconds": 13.667, + "jump_ratio": 1.19, + "look_ratio": 1.44, + "level_step_db": 0.09, + "typical_delta": 5.23 + }, + { + "seconds": 17.917, + "jump_ratio": 1.49, + "look_ratio": 2.21, + "level_step_db": 0.04, + "typical_delta": 6.34 + }, + { + "seconds": 22.167, + "jump_ratio": 1.31, + "look_ratio": 0.97, + "level_step_db": 2.76, + "typical_delta": 7.08 + } + ], + "motion_mean": 4.067596435546875, + "audio": { + "duration_s": 26.432, + "segments": [ + { + "start": 0.0, + "end": 5.17, + "rms_db": -50.43, + "hf_share_6k": 0.007643754128366709 + }, + { + "start": 5.17, + "end": 9.42, + "rms_db": -51.96, + "hf_share_6k": 0.007466106675565243 + }, + { + "start": 9.42, + "end": 13.67, + "rms_db": -54.06, + "hf_share_6k": 0.0044075981713831425 + }, + { + "start": 13.67, + "end": 17.92, + "rms_db": -53.96, + "hf_share_6k": 0.0019184263655915856 + }, + { + "start": 17.92, + "end": 22.17, + "rms_db": -53.86, + "hf_share_6k": 0.0006010688957758248 + }, + { + "start": 22.17, + "end": 26.43, + "rms_db": -56.8, + "hf_share_6k": 0.0006540807662531734 + } + ], + "rms_db_5s": [ + -50.55, + -51.9, + -53.77, + -54.51, + -54.9, + -57.18 + ] + } + }, + "d901c8c82784": { + "scene": "s2-arc", + "moments": [ + 62, + 277, + 583 + ], + "scene_bands": [ + { + "fine": 15.0557, + "mid": 20.411, + "coarse": 22.5499 + }, + { + "fine": 8.1257, + "mid": 19.4493, + "coarse": 63.4189 + }, + { + "fine": 0.4433, + "mid": 0.6054, + "coarse": 0.9318 + } + ], + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 2.16, + "look_ratio": 3.54, + "level_step_db": 1.23, + "typical_delta": 3.08 + }, + { + "seconds": 9.417, + "jump_ratio": 1.41, + "look_ratio": 2.88, + "level_step_db": 3.53, + "typical_delta": 3.94 + }, + { + "seconds": 13.667, + "jump_ratio": 1.2, + "look_ratio": 1.3, + "level_step_db": 5.31, + "typical_delta": 5.91 + }, + { + "seconds": 17.917, + "jump_ratio": 1.32, + "look_ratio": 1.57, + "level_step_db": 11.89, + "typical_delta": 6.43 + }, + { + "seconds": 22.167, + "jump_ratio": 1.31, + "look_ratio": 1.41, + "level_step_db": 0.68, + "typical_delta": 7.78 + } + ], + "motion_mean": 4.34854793548584, + "audio": { + "duration_s": 26.432, + "segments": [ + { + "start": 0.0, + "end": 5.17, + "rms_db": -45.1, + "hf_share_6k": 0.0004489357816055417 + }, + { + "start": 5.17, + "end": 9.42, + "rms_db": -43.93, + "hf_share_6k": 7.097611523931846e-05 + }, + { + "start": 9.42, + "end": 13.67, + "rms_db": -40.31, + "hf_share_6k": 9.771635632205289e-06 + }, + { + "start": 13.67, + "end": 17.92, + "rms_db": -45.65, + "hf_share_6k": 1.3554002180171665e-05 + }, + { + "start": 17.92, + "end": 22.17, + "rms_db": -57.54, + "hf_share_6k": 0.00017924397252500057 + }, + { + "start": 22.17, + "end": 26.43, + "rms_db": -56.89, + "hf_share_6k": 0.00019973925373051316 + } + ], + "rms_db_5s": [ + -45.16, + -43.25, + -40.88, + -49.86, + -62.81, + -52.93 + ] + } + }, + "8b21a85f52e8": { + "scene": "s1-static", + "moments": [ + 62, + 277, + 583 + ], + "scene_bands": [ + { + "fine": 5.3803, + "mid": 11.4488, + "coarse": 39.46 + }, + { + "fine": 6.4522, + "mid": 12.2048, + "coarse": 39.8778 + }, + { + "fine": 9.6132, + "mid": 13.7616, + "coarse": 40.0289 + } + ], + "face_bands": [ + { + "fine": 10.2749, + "mid": 27.6882, + "coarse": 46.0399 + }, + { + "fine": 12.5447, + "mid": 28.5639, + "coarse": 47.4219 + }, + { + "fine": 16.1705, + "mid": 29.9177, + "coarse": 45.6158 + } + ], + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 0.65, + "look_ratio": 1.93, + "level_step_db": 17.26, + "typical_delta": 2.0 + }, + { + "seconds": 9.417, + "jump_ratio": 0.91, + "look_ratio": 2.09, + "level_step_db": 2.83, + "typical_delta": 2.0 + }, + { + "seconds": 13.667, + "jump_ratio": 0.69, + "look_ratio": 1.07, + "level_step_db": 3.67, + "typical_delta": 2.0 + }, + { + "seconds": 17.917, + "jump_ratio": 0.88, + "look_ratio": 0.83, + "level_step_db": 0.59, + "typical_delta": 2.0 + }, + { + "seconds": 22.167, + "jump_ratio": 0.84, + "look_ratio": 0.58, + "level_step_db": 17.6, + "typical_delta": 2.0 + } + ], + "motion_mean": 0.47436562180519104, + "audio": { + "duration_s": 26.432, + "segments": [ + { + "start": 0.0, + "end": 5.17, + "rms_db": -57.85, + "hf_share_6k": 0.06652561575174332 + }, + { + "start": 5.17, + "end": 9.42, + "rms_db": -74.73, + "hf_share_6k": 0.07423253357410431 + }, + { + "start": 9.42, + "end": 13.67, + "rms_db": -77.67, + "hf_share_6k": 0.049163125455379486 + }, + { + "start": 13.67, + "end": 17.92, + "rms_db": -73.68, + "hf_share_6k": 0.014758037403225899 + }, + { + "start": 17.92, + "end": 22.17, + "rms_db": -73.22, + "hf_share_6k": 0.01301735732704401 + }, + { + "start": 22.17, + "end": 26.43, + "rms_db": -56.19, + "hf_share_6k": 0.08902986347675323 + } + ], + "rms_db_5s": [ + -57.72, + -74.58, + -76.74, + -72.85, + -56.86, + -75.52 + ] + } + }, + "98dc7b8c3a12": { + "scene": "s1-static", + "moments": [ + 62, + 277, + 583 + ], + "scene_bands": [ + { + "fine": 7.6588, + "mid": 16.6279, + "coarse": 43.6323 + }, + { + "fine": 6.4221, + "mid": 15.9071, + "coarse": 38.6762 + }, + { + "fine": 8.2812, + "mid": 18.3826, + "coarse": 33.384 + } + ], + "face_bands": [ + { + "fine": 9.3892, + "mid": 21.3577, + "coarse": 36.1462 + }, + { + "fine": 10.5684, + "mid": 27.5703, + "coarse": 38.7494 + }, + { + "fine": 10.5703, + "mid": 26.3934, + "coarse": 28.3776 + } + ], + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 0.84, + "look_ratio": 1.81, + "level_step_db": 1.71, + "typical_delta": 2.0 + }, + { + "seconds": 9.417, + "jump_ratio": 0.97, + "look_ratio": 1.9, + "level_step_db": 13.54, + "typical_delta": 2.0 + }, + { + "seconds": 13.667, + "jump_ratio": 1.04, + "look_ratio": 1.97, + "level_step_db": 0.69, + "typical_delta": 2.0 + }, + { + "seconds": 17.917, + "jump_ratio": 1.3, + "look_ratio": 1.4, + "level_step_db": 8.08, + "typical_delta": 2.0 + }, + { + "seconds": 22.167, + "jump_ratio": 1.21, + "look_ratio": 1.59, + "level_step_db": 6.2, + "typical_delta": 2.02 + } + ], + "motion_mean": 1.2047109603881836, + "audio": { + "duration_s": 26.432, + "segments": [ + { + "start": 0.0, + "end": 5.17, + "rms_db": -62.85, + "hf_share_6k": 0.12151610106229782 + }, + { + "start": 5.17, + "end": 9.42, + "rms_db": -64.6, + "hf_share_6k": 0.06541916728019714 + }, + { + "start": 9.42, + "end": 13.67, + "rms_db": -51.4, + "hf_share_6k": 0.16495545208454132 + }, + { + "start": 13.67, + "end": 17.92, + "rms_db": -50.95, + "hf_share_6k": 0.032117825001478195 + }, + { + "start": 17.92, + "end": 22.17, + "rms_db": -42.71, + "hf_share_6k": 0.018856626003980637 + }, + { + "start": 22.17, + "end": 26.43, + "rms_db": -49.25, + "hf_share_6k": 0.025400394573807716 + } + ], + "rms_db_5s": [ + -62.74, + -52.76, + -59.32, + -45.05, + -45.1, + -58.35 + ] + } + }, + "874f0e4ac0e8": { + "scene": "s1-static", + "moments": [ + 62, + 277, + 583 + ], + "scene_bands": [ + { + "fine": 5.3686, + "mid": 11.42, + "coarse": 39.4906 + }, + { + "fine": 6.7974, + "mid": 12.3522, + "coarse": 39.56 + }, + { + "fine": 10.832, + "mid": 18.6396, + "coarse": 55.7916 + } + ], + "face_bands": [ + { + "fine": 9.5425, + "mid": 24.0418, + "coarse": 38.7728 + }, + { + "fine": 8.659, + "mid": 26.5285, + "coarse": 47.053 + }, + { + "fine": 11.5971, + "mid": 24.2652, + "coarse": 37.8971 + } + ], + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 0.67, + "look_ratio": 2.03, + "level_step_db": 3.2, + "typical_delta": 2.0 + }, + { + "seconds": 9.417, + "jump_ratio": 0.81, + "look_ratio": 2.46, + "level_step_db": 7.28, + "typical_delta": 2.0 + }, + { + "seconds": 13.667, + "jump_ratio": 0.7, + "look_ratio": 1.11, + "level_step_db": 18.25, + "typical_delta": 2.0 + }, + { + "seconds": 17.917, + "jump_ratio": 1.04, + "look_ratio": 1.89, + "level_step_db": 0.76, + "typical_delta": 2.0 + }, + { + "seconds": 22.167, + "jump_ratio": 0.84, + "look_ratio": 1.28, + "level_step_db": 29.0, + "typical_delta": 2.0 + } + ], + "motion_mean": 0.5805790424346924, + "audio": { + "duration_s": 26.432, + "segments": [ + { + "start": 0.0, + "end": 5.17, + "rms_db": -63.6, + "hf_share_6k": 0.12788529694080353 + }, + { + "start": 5.17, + "end": 9.42, + "rms_db": -66.66, + "hf_share_6k": 0.038347575813531876 + }, + { + "start": 9.42, + "end": 13.67, + "rms_db": -59.69, + "hf_share_6k": 0.014412923716008663 + }, + { + "start": 13.67, + "end": 17.92, + "rms_db": -77.69, + "hf_share_6k": 0.09054634720087051 + }, + { + "start": 17.92, + "end": 22.17, + "rms_db": -78.7, + "hf_share_6k": 0.09443619847297668 + }, + { + "start": 22.17, + "end": 26.43, + "rms_db": -49.7, + "hf_share_6k": 0.13183683156967163 + } + ], + "rms_db_5s": [ + -63.49, + -65.77, + -60.77, + -78.1, + -50.48, + -61.58 + ] + } + }, + "09e78427dacf": { + "scene": "s1-static", + "moments": [ + 62, + 277, + 583 + ], + "scene_bands": [ + { + "fine": 5.4625, + "mid": 11.5751, + "coarse": 39.6168 + }, + { + "fine": 6.5248, + "mid": 12.5455, + "coarse": 39.9444 + }, + { + "fine": 10.2493, + "mid": 13.6996, + "coarse": 39.2618 + } + ], + "face_bands": [ + { + "fine": 10.1538, + "mid": 25.5857, + "coarse": 43.7332 + }, + { + "fine": 12.7904, + "mid": 24.4098, + "coarse": 43.4707 + }, + { + "fine": 15.7289, + "mid": 30.7221, + "coarse": 46.0372 + } + ], + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 0.65, + "look_ratio": 1.61, + "level_step_db": 9.63, + "typical_delta": 2.0 + }, + { + "seconds": 9.417, + "jump_ratio": 1.01, + "look_ratio": 2.23, + "level_step_db": 5.26, + "typical_delta": 2.0 + }, + { + "seconds": 13.667, + "jump_ratio": 1.07, + "look_ratio": 2.95, + "level_step_db": 23.83, + "typical_delta": 2.0 + }, + { + "seconds": 17.917, + "jump_ratio": 0.85, + "look_ratio": 2.24, + "level_step_db": 16.3, + "typical_delta": 2.0 + }, + { + "seconds": 22.167, + "jump_ratio": 0.91, + "look_ratio": 0.66, + "level_step_db": 11.42, + "typical_delta": 2.0 + } + ], + "motion_mean": 0.6370080709457397, + "audio": { + "duration_s": 26.432, + "segments": [ + { + "start": 0.0, + "end": 5.17, + "rms_db": -67.48, + "hf_share_6k": 0.02645031549036503 + }, + { + "start": 5.17, + "end": 9.42, + "rms_db": -77.43, + "hf_share_6k": 0.027723316103219986 + }, + { + "start": 9.42, + "end": 13.67, + "rms_db": -82.8, + "hf_share_6k": 0.05570268630981445 + }, + { + "start": 13.67, + "end": 17.92, + "rms_db": -58.48, + "hf_share_6k": 0.00029159357654862106 + }, + { + "start": 17.92, + "end": 22.17, + "rms_db": -74.96, + "hf_share_6k": 0.008846098557114601 + }, + { + "start": 22.17, + "end": 26.43, + "rms_db": -86.51, + "hf_share_6k": 0.04454170539975166 + } + ], + "rms_db_5s": [ + -67.39, + -77.3, + -59.2, + -75.59, + -80.8, + -87.44 + ] + } + }, + "2a77f4abc476": { + "scene": "s1-static", + "moments": [ + 62, + 277, + 583 + ], + "scene_bands": [ + { + "fine": 10.5177, + "mid": 26.3276, + "coarse": 41.1777 + }, + { + "fine": 11.7319, + "mid": 31.4867, + "coarse": 40.0819 + }, + { + "fine": 11.4629, + "mid": 31.3141, + "coarse": 36.383 + } + ], + "face_bands": [ + { + "fine": 11.7633, + "mid": 28.0686, + "coarse": 39.6155 + }, + { + "fine": 12.4526, + "mid": 25.4237, + "coarse": 43.6093 + }, + { + "fine": 13.3239, + "mid": 28.7927, + "coarse": 36.5484 + } + ], + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 0.86, + "look_ratio": 1.79, + "level_step_db": 6.5, + "typical_delta": 2.0 + }, + { + "seconds": 9.417, + "jump_ratio": 1.02, + "look_ratio": 1.99, + "level_step_db": 3.18, + "typical_delta": 2.0 + }, + { + "seconds": 13.667, + "jump_ratio": 1.32, + "look_ratio": 2.36, + "level_step_db": 3.12, + "typical_delta": 2.0 + }, + { + "seconds": 17.917, + "jump_ratio": 1.37, + "look_ratio": 3.22, + "level_step_db": 0.23, + "typical_delta": 2.0 + }, + { + "seconds": 22.167, + "jump_ratio": 1.27, + "look_ratio": 1.48, + "level_step_db": 24.57, + "typical_delta": 2.0 + } + ], + "motion_mean": 1.146178126335144, + "audio": { + "duration_s": 26.432, + "segments": [ + { + "start": 0.0, + "end": 5.17, + "rms_db": -68.46, + "hf_share_6k": 0.08272543549537659 + }, + { + "start": 5.17, + "end": 9.42, + "rms_db": -64.08, + "hf_share_6k": 0.1496630758047104 + }, + { + "start": 9.42, + "end": 13.67, + "rms_db": -60.26, + "hf_share_6k": 0.172471821308136 + }, + { + "start": 13.67, + "end": 17.92, + "rms_db": -62.04, + "hf_share_6k": 0.0033664025831967592 + }, + { + "start": 17.92, + "end": 22.17, + "rms_db": -62.27, + "hf_share_6k": 0.02474142797291279 + }, + { + "start": 22.17, + "end": 26.43, + "rms_db": -86.99, + "hf_share_6k": 0.08839025348424911 + } + ], + "rms_db_5s": [ + -68.39, + -64.74, + -58.86, + -62.81, + -79.24, + -87.13 + ] + } + }, + "96e48beb2dd3": { + "scene": "s1-static", + "moments": [ + 62, + 277, + 583 + ], + "scene_bands": [ + { + "fine": 5.5494, + "mid": 11.5434, + "coarse": 39.5156 + }, + { + "fine": 6.9518, + "mid": 12.2206, + "coarse": 39.8273 + }, + { + "fine": 11.5653, + "mid": 13.5217, + "coarse": 38.7257 + } + ], + "face_bands": [ + { + "fine": 11.3119, + "mid": 27.0439, + "coarse": 48.1411 + }, + { + "fine": 11.8301, + "mid": 26.1865, + "coarse": 49.4934 + }, + { + "fine": 13.2364, + "mid": 27.7194, + "coarse": 42.9335 + } + ], + "seams": [ + { + "seconds": 5.167, + "jump_ratio": 0.61, + "look_ratio": 1.89, + "level_step_db": 6.98, + "typical_delta": 2.0 + }, + { + "seconds": 9.417, + "jump_ratio": 0.85, + "look_ratio": 1.58, + "level_step_db": 3.47, + "typical_delta": 2.0 + }, + { + "seconds": 13.667, + "jump_ratio": 0.72, + "look_ratio": 1.0, + "level_step_db": 9.6, + "typical_delta": 2.0 + }, + { + "seconds": 17.917, + "jump_ratio": 1.02, + "look_ratio": 1.96, + "level_step_db": 5.2, + "typical_delta": 2.0 + }, + { + "seconds": 22.167, + "jump_ratio": 0.77, + "look_ratio": 0.91, + "level_step_db": 1.15, + "typical_delta": 2.0 + } + ], + "motion_mean": 0.5481722354888916, + "audio": { + "duration_s": 26.432, + "segments": [ + { + "start": 0.0, + "end": 5.17, + "rms_db": -65.22, + "hf_share_6k": 0.0404362790286541 + }, + { + "start": 5.17, + "end": 9.42, + "rms_db": -58.69, + "hf_share_6k": 0.017587054520845413 + }, + { + "start": 9.42, + "end": 13.67, + "rms_db": -62.76, + "hf_share_6k": 0.015188822522759438 + }, + { + "start": 13.67, + "end": 17.92, + "rms_db": -71.37, + "hf_share_6k": 0.021209632977843285 + }, + { + "start": 17.92, + "end": 22.17, + "rms_db": -66.2, + "hf_share_6k": 0.044519759714603424 + }, + { + "start": 22.17, + "end": 26.43, + "rms_db": -64.98, + "hf_share_6k": 0.01466421689838171 + } + ], + "rms_db_5s": [ + -65.13, + -59.38, + -63.46, + -65.91, + -66.09, + -69.43 + ] + } + }, + "4b79f1280670": { + "scene": "s2-arc", + "moments": [ + 20, + 62, + 110 + ], + "scene_bands": [ + { + "fine": 17.2578, + "mid": 25.7832, + "coarse": 28.5807 + }, + { + "fine": 14.1066, + "mid": 25.035, + "coarse": 39.6467 + }, + { + "fine": 6.271, + "mid": 9.4292, + "coarse": 22.0814 + } + ], + "seams": [], + "motion_mean": 2.7160439491271973, + "audio": { + "duration_s": 5.184, + "segments": [ + { + "start": 0.0, + "end": 5.18, + "rms_db": -54.53, + "hf_share_6k": 0.033201515674591064 + } + ], + "rms_db_5s": [ + -54.49 + ] + } + }, + "9925b2488c98": { + "scene": "s2-arc", + "moments": [ + 20, + 62, + 110 + ], + "scene_bands": [ + { + "fine": 16.1762, + "mid": 25.5785, + "coarse": 24.9147 + }, + { + "fine": 13.3926, + "mid": 22.3207, + "coarse": 27.4454 + }, + { + "fine": 4.9004, + "mid": 9.9013, + "coarse": 22.3199 + } + ], + "seams": [], + "motion_mean": 2.8598549365997314, + "audio": { + "duration_s": 5.184, + "segments": [ + { + "start": 0.0, + "end": 5.18, + "rms_db": -56.23, + "hf_share_6k": 0.014550630003213882 + } + ], + "rms_db_5s": [ + -56.21 + ] + } + }, + "b35afd39ed00": { + "scene": "s2-arc", + "moments": [ + 20, + 62, + 110 + ], + "scene_bands": [ + { + "fine": 17.6529, + "mid": 23.9316, + "coarse": 26.8961 + }, + { + "fine": 13.3169, + "mid": 21.6137, + "coarse": 37.5291 + }, + { + "fine": 4.6225, + "mid": 9.8082, + "coarse": 22.441 + } + ], + "seams": [], + "motion_mean": 2.9902145862579346, + "audio": { + "duration_s": 5.184, + "segments": [ + { + "start": 0.0, + "end": 5.18, + "rms_db": -52.71, + "hf_share_6k": 0.03610703721642494 + } + ], + "rms_db_5s": [ + -52.63 + ] + } + }, + "6cbd7727f378": { + "scene": "s2-arc", + "moments": [ + 20, + 62, + 110 + ], + "scene_bands": [ + { + "fine": 18.1454, + "mid": 25.9876, + "coarse": 33.7124 + }, + { + "fine": 16.7312, + "mid": 23.0112, + "coarse": 23.8963 + }, + { + "fine": 10.5619, + "mid": 18.121, + "coarse": 34.1686 + } + ], + "seams": [], + "motion_mean": 1.8615168333053589, + "audio": { + "duration_s": 5.184, + "segments": [ + { + "start": 0.0, + "end": 5.18, + "rms_db": -54.1, + "hf_share_6k": 0.010429994203150272 + } + ], + "rms_db_5s": [ + -54.01 + ] + } + }, + "6edd44546d16": { + "scene": "s2-arc", + "moments": [ + 20, + 62, + 110 + ], + "scene_bands": [ + { + "fine": 16.4024, + "mid": 23.338, + "coarse": 25.9795 + }, + { + "fine": 10.8222, + "mid": 13.6875, + "coarse": 24.9915 + }, + { + "fine": 1.7773, + "mid": 4.2763, + "coarse": 12.1498 + } + ], + "seams": [], + "motion_mean": 2.820767641067505, + "audio": { + "duration_s": 5.184, + "segments": [ + { + "start": 0.0, + "end": 5.18, + "rms_db": -60.61, + "hf_share_6k": 0.0006138643948361278 + } + ], + "rms_db_5s": [ + -60.53 + ] + } + }, + "41d6746a43dd": { + "scene": "s2-arc", + "moments": [ + 20, + 62, + 110 + ], + "scene_bands": [ + { + "fine": 17.6083, + "mid": 24.7147, + "coarse": 27.4304 + }, + { + "fine": 13.9799, + "mid": 22.1103, + "coarse": 27.7198 + }, + { + "fine": 7.4831, + "mid": 11.7251, + "coarse": 24.4366 + } + ], + "seams": [], + "motion_mean": 2.970733165740967, + "audio": { + "duration_s": 5.184, + "segments": [ + { + "start": 0.0, + "end": 5.18, + "rms_db": -57.67, + "hf_share_6k": 0.009171543642878532 + } + ], + "rms_db_5s": [ + -57.58 + ] + } + }, + "a32149365b7c": { + "scene": "s1-static", + "moments": [ + 20, + 62, + 110 + ], + "scene_bands": [ + { + "fine": 6.5374, + "mid": 14.0587, + "coarse": 43.9985 + }, + { + "fine": 6.7292, + "mid": 14.09, + "coarse": 43.8151 + }, + { + "fine": 6.541, + "mid": 14.1284, + "coarse": 43.8697 + } + ], + "face_bands": [ + { + "fine": 9.866, + "mid": 23.8816, + "coarse": 44.7249 + }, + { + "fine": 11.1341, + "mid": 23.9533, + "coarse": 43.4909 + }, + { + "fine": 11.4354, + "mid": 24.5686, + "coarse": 43.7801 + } + ], + "seams": [], + "motion_mean": 0.451201856136322, + "audio": { + "duration_s": 5.184, + "segments": [ + { + "start": 0.0, + "end": 5.18, + "rms_db": -58.05, + "hf_share_6k": 0.01146689523011446 + } + ], + "rms_db_5s": [ + -57.9 + ] + } + }, + "dd38d2050097": { + "scene": "s1-static", + "moments": [ + 20, + 62, + 110 + ], + "scene_bands": [ + { + "fine": 6.479, + "mid": 13.9687, + "coarse": 44.0237 + }, + { + "fine": 6.5502, + "mid": 13.9675, + "coarse": 44.0694 + }, + { + "fine": 6.4352, + "mid": 13.9791, + "coarse": 44.0058 + } + ], + "face_bands": [ + { + "fine": 10.6897, + "mid": 23.4093, + "coarse": 44.8191 + }, + { + "fine": 11.3836, + "mid": 27.014, + "coarse": 43.5128 + }, + { + "fine": 11.1008, + "mid": 23.2286, + "coarse": 43.8034 + } + ], + "seams": [], + "motion_mean": 0.7497071027755737, + "audio": { + "duration_s": 5.184, + "segments": [ + { + "start": 0.0, + "end": 5.18, + "rms_db": -60.43, + "hf_share_6k": 0.031436920166015625 + } + ], + "rms_db_5s": [ + -60.33 + ] + } + }, + "7c3a90fc4c90": { + "scene": "s2-arc", + "moments": [ + 20, + 62, + 110 + ], + "scene_bands": [ + { + "fine": 18.7846, + "mid": 25.9552, + "coarse": 32.1733 + }, + { + "fine": 15.6602, + "mid": 24.4612, + "coarse": 31.7723 + }, + { + "fine": 6.504, + "mid": 10.8008, + "coarse": 25.9965 + } + ], + "seams": [], + "motion_mean": 2.4851231575012207, + "audio": { + "duration_s": 5.184, + "segments": [ + { + "start": 0.0, + "end": 5.18, + "rms_db": -52.99, + "hf_share_6k": 0.007285800762474537 + } + ], + "rms_db_5s": [ + -53.09 + ] + } + }, + "6c644ea1d6c2": { + "scene": "s2-arc", + "moments": [ + 20, + 62, + 110 + ], + "scene_bands": [ + { + "fine": 18.9965, + "mid": 25.5218, + "coarse": 34.2072 + }, + { + "fine": 15.0557, + "mid": 20.411, + "coarse": 22.5499 + }, + { + "fine": 6.854, + "mid": 10.5101, + "coarse": 23.4723 + } + ], + "seams": [], + "motion_mean": 2.4122097492218018, + "audio": { + "duration_s": 5.184, + "segments": [ + { + "start": 0.0, + "end": 5.18, + "rms_db": -45.11, + "hf_share_6k": 0.00044984917622059584 + } + ], + "rms_db_5s": [ + -45.16 + ] + } + }, + "db14f86eb1bd": { + "scene": "s1-static", + "moments": [ + 20, + 62, + 110 + ], + "scene_bands": [ + { + "fine": 6.8167, + "mid": 14.4838, + "coarse": 44.874 + }, + { + "fine": 6.7543, + "mid": 14.2877, + "coarse": 44.943 + }, + { + "fine": 6.7647, + "mid": 14.4999, + "coarse": 44.9713 + } + ], + "face_bands": [ + { + "fine": 10.0641, + "mid": 24.0926, + "coarse": 46.781 + }, + { + "fine": 11.6442, + "mid": 24.997, + "coarse": 45.2081 + }, + { + "fine": 11.7576, + "mid": 24.3473, + "coarse": 44.7149 + } + ], + "seams": [], + "motion_mean": 0.42361462116241455, + "audio": { + "duration_s": 5.184, + "segments": [ + { + "start": 0.0, + "end": 5.18, + "rms_db": -51.85, + "hf_share_6k": 0.006914800964295864 + } + ], + "rms_db_5s": [ + -51.69 + ] + } + }, + "1a172450268f": { + "scene": "s1-static", + "moments": [ + 20, + 62, + 110 + ], + "scene_bands": [ + { + "fine": 6.5805, + "mid": 14.1606, + "coarse": 44.8004 + }, + { + "fine": 6.6889, + "mid": 14.4043, + "coarse": 44.7938 + }, + { + "fine": 6.6891, + "mid": 14.5959, + "coarse": 44.9432 + } + ], + "face_bands": [ + { + "fine": 11.2527, + "mid": 24.2233, + "coarse": 46.2878 + }, + { + "fine": 11.3464, + "mid": 24.6479, + "coarse": 44.9722 + }, + { + "fine": 11.7056, + "mid": 24.6187, + "coarse": 44.8759 + } + ], + "seams": [], + "motion_mean": 0.5010091662406921, + "audio": { + "duration_s": 5.184, + "segments": [ + { + "start": 0.0, + "end": 5.18, + "rms_db": -56.29, + "hf_share_6k": 0.02098509669303894 + } + ], + "rms_db_5s": [ + -56.14 + ] + } + }, + "d0e2aad67720": { + "scene": "s2-arc", + "moments": [ + 20, + 62, + 110 + ], + "scene_bands": [ + { + "fine": 19.0268, + "mid": 25.4124, + "coarse": 28.856 + }, + { + "fine": 11.3761, + "mid": 17.8071, + "coarse": 39.6688 + }, + { + "fine": 11.3176, + "mid": 16.599, + "coarse": 27.3388 + } + ], + "seams": [], + "motion_mean": 3.088986396789551, + "audio": { + "duration_s": 5.184, + "segments": [ + { + "start": 0.0, + "end": 5.18, + "rms_db": -59.14, + "hf_share_6k": 0.00230212789028883 + } + ], + "rms_db_5s": [ + -59.16 + ] + } + }, + "19e189043889": { + "scene": "s2-arc", + "moments": [ + 20, + 62, + 110 + ], + "scene_bands": [ + { + "fine": 6.5896, + "mid": 11.5339, + "coarse": 24.5058 + }, + { + "fine": 7.637, + "mid": 9.3064, + "coarse": 19.5006 + }, + { + "fine": 4.3809, + "mid": 12.2031, + "coarse": 27.738 + } + ], + "seams": [], + "motion_mean": 4.110535621643066, + "audio": { + "duration_s": 5.184, + "segments": [ + { + "start": 0.0, + "end": 5.18, + "rms_db": -55.54, + "hf_share_6k": 0.0031649761367589235 + } + ], + "rms_db_5s": [ + -55.59 + ] + } + }, + "6dba38159cc7": { + "scene": "s1-static", + "moments": [ + 20, + 62, + 110 + ], + "scene_bands": [ + { + "fine": 6.7885, + "mid": 14.5601, + "coarse": 45.3965 + }, + { + "fine": 6.881, + "mid": 14.4933, + "coarse": 45.3711 + }, + { + "fine": 6.6995, + "mid": 14.4544, + "coarse": 45.3826 + } + ], + "face_bands": [ + { + "fine": 10.0141, + "mid": 24.3373, + "coarse": 47.111 + }, + { + "fine": 11.476, + "mid": 25.263, + "coarse": 45.7194 + }, + { + "fine": 12.0054, + "mid": 24.9005, + "coarse": 45.6829 + } + ], + "seams": [], + "motion_mean": 0.4212363660335541, + "audio": { + "duration_s": 5.184, + "segments": [ + { + "start": 0.0, + "end": 5.18, + "rms_db": -52.36, + "hf_share_6k": 0.008557300083339214 + } + ], + "rms_db_5s": [ + -52.2 + ] + } + }, + "87b8dd26a67a": { + "scene": "s1-static", + "moments": [ + 20, + 62, + 110 + ], + "scene_bands": [ + { + "fine": 6.7107, + "mid": 14.4458, + "coarse": 45.5994 + }, + { + "fine": 6.7448, + "mid": 14.6209, + "coarse": 45.5127 + }, + { + "fine": 6.6674, + "mid": 14.412, + "coarse": 45.4024 + } + ], + "face_bands": [ + { + "fine": 11.051, + "mid": 24.1113, + "coarse": 46.8116 + }, + { + "fine": 11.9141, + "mid": 24.6301, + "coarse": 45.5862 + }, + { + "fine": 11.8135, + "mid": 24.529, + "coarse": 45.7425 + } + ], + "seams": [], + "motion_mean": 0.5140370726585388, + "audio": { + "duration_s": 5.184, + "segments": [ + { + "start": 0.0, + "end": 5.18, + "rms_db": -58.34, + "hf_share_6k": 0.027739616110920906 + } + ], + "rms_db_5s": [ + -58.2 + ] + } + }, + "80ed710df33f": { + "scene": "s2-arc", + "moments": [ + 20, + 62, + 110 + ], + "scene_bands": [ + { + "fine": 19.1037, + "mid": 24.9577, + "coarse": 28.0698 + }, + { + "fine": 11.2628, + "mid": 18.2411, + "coarse": 36.8634 + }, + { + "fine": 11.8999, + "mid": 17.03, + "coarse": 29.0262 + } + ], + "seams": [], + "motion_mean": 3.036754846572876, + "audio": { + "duration_s": 5.184, + "segments": [ + { + "start": 0.0, + "end": 5.18, + "rms_db": -58.15, + "hf_share_6k": 0.0034541708882898092 + } + ], + "rms_db_5s": [ + -58.18 + ] + } + }, + "f7a673dae443": { + "scene": "s2-arc", + "moments": [ + 20, + 62, + 110 + ], + "scene_bands": [ + { + "fine": 6.8371, + "mid": 11.5326, + "coarse": 24.4408 + }, + { + "fine": 7.5461, + "mid": 9.4854, + "coarse": 19.7105 + }, + { + "fine": 3.9959, + "mid": 11.0146, + "coarse": 25.4153 + } + ], + "seams": [], + "motion_mean": 4.227589130401611, + "audio": { + "duration_s": 5.184, + "segments": [ + { + "start": 0.0, + "end": 5.18, + "rms_db": -52.56, + "hf_share_6k": 0.004017747938632965 + } + ], + "rms_db_5s": [ + -52.59 + ] + } + }, + "20db977af359": { + "scene": "s1-static", + "moments": [ + 20, + 62, + 110 + ], + "scene_bands": [ + { + "fine": 6.8167, + "mid": 14.4838, + "coarse": 44.874 + }, + { + "fine": 6.7543, + "mid": 14.2877, + "coarse": 44.943 + }, + { + "fine": 6.7647, + "mid": 14.4999, + "coarse": 44.9713 + } + ], + "face_bands": [ + { + "fine": 10.0641, + "mid": 24.0926, + "coarse": 46.781 + }, + { + "fine": 11.6442, + "mid": 24.997, + "coarse": 45.2081 + }, + { + "fine": 11.7576, + "mid": 24.3473, + "coarse": 44.7149 + } + ], + "seams": [], + "motion_mean": 0.42361462116241455, + "audio": { + "duration_s": 5.184, + "segments": [ + { + "start": 0.0, + "end": 5.18, + "rms_db": -51.85, + "hf_share_6k": 0.006914800964295864 + } + ], + "rms_db_5s": [ + -51.69 + ] + } + } + }, + "pairs": [ + { + "arm": "fc0f91663277", + "control": "25f6bd1da507", + "moments": [ + 62, + 300, + 583, + 1000, + 1501, + 2000, + 2500, + 2868 + ], + "psnr_db": [ + "identical", + 20.97, + 22.36, + 18.53, + 17.07, + 14.88, + 14.04, + 11.29 + ], + "ssim": [ + 1.0, + 0.778, + 0.6972, + 0.5459, + 0.4303, + 0.3486, + 0.3371, + 0.2742 + ] + }, + { + "arm": "40bbb40c14ec", + "control": "bf9ee32e040f", + "moments": [ + 62, + 300, + 583, + 1000, + 1501, + 2000, + 2500, + 2868 + ], + "psnr_db": [ + 29.67, + 24.62, + 23.86, + 19.38, + 16.69, + 15.17, + 13.53, + 12.99 + ], + "ssim": [ + 0.9087, + 0.8119, + 0.7448, + 0.565, + 0.4306, + 0.3612, + 0.2906, + 0.2763 + ] + }, + { + "arm": "4766787e793d", + "control": "05a3a55b043c", + "moments": [ + 62, + 300, + 583, + 1000, + 1501, + 2000, + 2500, + 2868 + ], + "psnr_db": [ + "identical", + 26.83, + 21.99, + 17.61, + 17.39, + 16.48, + 16.29, + 15.28 + ], + "ssim": [ + 1.0, + 0.8323, + 0.7045, + 0.5449, + 0.4954, + 0.5402, + 0.5694, + 0.701 + ] + }, + { + "arm": "20ce09d60040", + "control": "8d1614ced27a", + "moments": [ + 62, + 300, + 583, + 1000, + 1501, + 2000, + 2500, + 2868 + ], + "psnr_db": [ + "identical", + 21.14, + 20.34, + 16.13, + 12.13, + 10.28, + 9.81, + 10.04 + ], + "ssim": [ + 1.0, + 0.7715, + 0.6112, + 0.3907, + 0.2713, + 0.2095, + 0.2487, + 0.2246 + ] + }, + { + "arm": "2c1fd3ab7869", + "control": "25f6bd1da507", + "moments": [ + 62, + 300, + 583, + 1000, + 1501, + 2000, + 2500, + 2868 + ], + "psnr_db": [ + 39.74, + 28.66, + 24.07, + 18.38, + 15.14, + 15.89, + 13.13, + 12.56 + ], + "ssim": [ + 0.9713, + 0.8819, + 0.8167, + 0.6622, + 0.5223, + 0.4722, + 0.4258, + 0.3805 + ] + }, + { + "arm": "25f6bd1da507", + "control": "8d1614ced27a", + "moments": [ + 62, + 300, + 583, + 1000, + 1501, + 2000, + 2500, + 2868 + ], + "psnr_db": [ + "identical", + 26.96, + 24.34, + 17.51, + 12.9, + 12.08, + 9.45, + 11.22 + ], + "ssim": [ + 1.0, + 0.8881, + 0.7898, + 0.4991, + 0.3761, + 0.302, + 0.2651, + 0.2497 + ] + }, + { + "arm": "d93adb33249e", + "control": "536548e14dd5", + "moments": [ + 62, + 277, + 583 + ], + "psnr_db": [ + "identical", + 13.98, + 10.68 + ], + "ssim": [ + 1.0, + 0.2981, + 0.5413 + ] + }, + { + "arm": "d901c8c82784", + "control": "1b8edfdba405", + "moments": [ + 62, + 277, + 583 + ], + "psnr_db": [ + "identical", + 14.71, + 10.75 + ], + "ssim": [ + 1.0, + 0.2979, + 0.4989 + ] + }, + { + "arm": "536548e14dd5", + "control": "dedd341c9108", + "moments": [ + 62, + 277, + 583 + ], + "psnr_db": [ + "identical", + "identical", + "identical" + ], + "ssim": [ + 1.0, + 1.0, + 1.0 + ] + }, + { + "arm": "1b8edfdba405", + "control": "01146e925c94", + "moments": [ + 62, + 277, + 583 + ], + "psnr_db": [ + "identical", + 15.56, + 11.75 + ], + "ssim": [ + 1.0, + 0.4194, + 0.523 + ] + }, + { + "arm": "874f0e4ac0e8", + "control": "8b21a85f52e8", + "moments": [ + 62, + 277, + 583 + ], + "psnr_db": [ + 18.01, + 22.15, + 13.82 + ], + "ssim": [ + 0.7676, + 0.7649, + 0.3439 + ] + }, + { + "arm": "98dc7b8c3a12", + "control": "8b21a85f52e8", + "moments": [ + 62, + 277, + 583 + ], + "psnr_db": [ + 16.68, + 16.81, + 14.89 + ], + "ssim": [ + 0.6463, + 0.5816, + 0.4085 + ] + }, + { + "arm": "b35afd39ed00", + "control": "4b79f1280670", + "moments": [ + 20, + 62, + 110 + ], + "psnr_db": [ + 15.49, + 11.96, + 12.21 + ], + "ssim": [ + 0.367, + 0.3126, + 0.2939 + ] + }, + { + "arm": "9925b2488c98", + "control": "4b79f1280670", + "moments": [ + 20, + 62, + 110 + ], + "psnr_db": [ + 14.59, + 12.62, + 12.2 + ], + "ssim": [ + 0.3111, + 0.2913, + 0.2907 + ] + }, + { + "arm": "6dba38159cc7", + "control": "db14f86eb1bd", + "moments": [ + 20, + 62, + 110 + ], + "psnr_db": [ + 31.96, + 34.82, + 34.7 + ], + "ssim": [ + 0.9171, + 0.939, + 0.9462 + ] + }, + { + "arm": "db14f86eb1bd", + "control": "a32149365b7c", + "moments": [ + 20, + 62, + 110 + ], + "psnr_db": [ + 28.33, + 28.41, + 28.59 + ], + "ssim": [ + 0.8637, + 0.8647, + 0.8726 + ] + }, + { + "arm": "80ed710df33f", + "control": "d0e2aad67720", + "moments": [ + 20, + 62, + 110 + ], + "psnr_db": [ + 23.26, + 19.13, + 18.51 + ], + "ssim": [ + 0.7797, + 0.6338, + 0.6049 + ] + }, + { + "arm": "d0e2aad67720", + "control": "7c3a90fc4c90", + "moments": [ + 20, + 62, + 110 + ], + "psnr_db": [ + 13.21, + 11.5, + 11.91 + ], + "ssim": [ + 0.3653, + 0.2958, + 0.3027 + ] + }, + { + "arm": "96e48beb2dd3", + "control": "09e78427dacf", + "moments": [ + 62, + 277, + 583 + ], + "psnr_db": [ + 21.65, + 17.42, + 18.35 + ], + "ssim": [ + 0.7974, + 0.7186, + 0.5529 + ] + }, + { + "arm": "2a77f4abc476", + "control": "09e78427dacf", + "moments": [ + 62, + 277, + 583 + ], + "psnr_db": [ + 17.27, + 14.69, + 14.16 + ], + "ssim": [ + 0.6457, + 0.525, + 0.3399 + ] + }, + { + "arm": "41d6746a43dd", + "control": "6cbd7727f378", + "moments": [ + 20, + 62, + 110 + ], + "psnr_db": [ + 11.72, + 10.76, + 10.24 + ], + "ssim": [ + 0.2737, + 0.2579, + 0.2551 + ] + }, + { + "arm": "6edd44546d16", + "control": "6cbd7727f378", + "moments": [ + 20, + 62, + 110 + ], + "psnr_db": [ + 11.6, + 10.72, + 10.39 + ], + "ssim": [ + 0.2499, + 0.2495, + 0.2542 + ] + }, + { + "arm": "87b8dd26a67a", + "control": "1a172450268f", + "moments": [ + 20, + 62, + 110 + ], + "psnr_db": [ + 31.21, + 33.58, + 31.48 + ], + "ssim": [ + 0.9062, + 0.9243, + 0.9066 + ] + }, + { + "arm": "1a172450268f", + "control": "dd38d2050097", + "moments": [ + 20, + 62, + 110 + ], + "psnr_db": [ + 24.0, + 20.77, + 24.04 + ], + "ssim": [ + 0.8182, + 0.797, + 0.8117 + ] + }, + { + "arm": "f7a673dae443", + "control": "19e189043889", + "moments": [ + 20, + 62, + 110 + ], + "psnr_db": [ + 23.36, + 20.64, + 18.42 + ], + "ssim": [ + 0.8404, + 0.7367, + 0.6829 + ] + }, + { + "arm": "19e189043889", + "control": "6c644ea1d6c2", + "moments": [ + 20, + 62, + 110 + ], + "psnr_db": [ + 11.42, + 9.55, + 9.31 + ], + "ssim": [ + 0.2864, + 0.2522, + 0.2455 + ] + }, + { + "arm": "20db977af359", + "control": "db14f86eb1bd", + "moments": [ + 20, + 62, + 110 + ], + "psnr_db": [ + "identical", + "identical", + "identical" + ], + "ssim": [ + 1.0, + 1.0, + 1.0 + ] + } + ], + "note": "Analyst moments pass 2026-11-06 (in-process on lem, bench checkout 6a89744a): raw texture bands (luma fine/mid/coarse) on the scene crop and the S1 face crop at each moment, analyze_seams at recorded shots, thumbnail motion, per-segment audio RMS dBFS and >=6 kHz energy share (mono 32 kHz), and arm-vs-control PSNR/SSIM." +} \ No newline at end of file diff --git a/bench/records/t26-claims.json b/bench/records/t26-claims.json new file mode 100644 index 000000000..632334015 --- /dev/null +++ b/bench/records/t26-claims.json @@ -0,0 +1,173 @@ +[ + { + "arm": "c1-line-plain-s2002", + "scene": "t26", + "seed": 2002, + "job_id": "2b93dfe5eb65", + "status": "succeeded", + "error": null, + "output": "BenchT26T2VA/20261106-032735-6ed7056e/final/BenchT26T2VA-clip.0-0.0.mp4" + }, + { + "arm": "c1-line-sentence-s1001", + "scene": "t26", + "seed": 1001, + "job_id": "d45357868b54", + "status": "succeeded", + "error": null, + "output": "BenchT26T2VA/20261106-032735-8f2cb391/final/BenchT26T2VA-clip.0-0.0.mp4" + }, + { + "arm": "c1-line-sentence-s2002", + "scene": "t26", + "seed": 2002, + "job_id": "79a9969093de", + "status": "succeeded", + "error": null, + "output": "BenchT26T2VA/20261106-032902-702aee3d/final/BenchT26T2VA-clip.0-0.0.mp4" + }, + { + "arm": "c5-hum-s1001", + "scene": "t26", + "seed": 1001, + "job_id": "313fe586f422", + "status": "succeeded", + "error": null, + "output": "BenchT26T2VA/20261106-033029-b8ab76a0/final/BenchT26T2VA-clip.0-0.0.mp4" + }, + { + "arm": "c5-hum-s2002", + "scene": "t26", + "seed": 2002, + "job_id": "412db5c1ed0e", + "status": "succeeded", + "error": null, + "output": "BenchT26T2VA/20261106-033135-8c9f6ca7/final/BenchT26T2VA-clip.0-0.0.mp4" + }, + { + "arm": "c5-nohum-s1001", + "scene": "t26", + "seed": 1001, + "job_id": "133cae62bfbd", + "status": "succeeded", + "error": null, + "output": "BenchT26T2VA/20261106-033200-6aff45ee/final/BenchT26T2VA-clip.0-0.0.mp4" + }, + { + "arm": "c5-nohum-s2002", + "scene": "t26", + "seed": 2002, + "job_id": "c1b9be136407", + "status": "succeeded", + "error": null, + "output": "BenchT26T2VA/20261106-033259-dfd811ea/final/BenchT26T2VA-clip.0-0.0.mp4" + }, + { + "arm": "c5-nohum-nomusicline-s1001", + "scene": "t26", + "seed": 1001, + "job_id": "3e449fbebbca", + "status": "succeeded", + "error": null, + "output": "BenchT26T2VA/20261106-033326-b212f0a9/final/BenchT26T2VA-clip.0-0.0.mp4" + }, + { + "arm": "c5-nohum-nomusicline-s2002", + "scene": "t26", + "seed": 2002, + "job_id": "e0f10599f3d4", + "status": "succeeded", + "error": null, + "output": "BenchT26T2VA/20261106-033422-5c604a64/final/BenchT26T2VA-clip.0-0.0.mp4" + }, + { + "arm": "c2-unmentioned-s1001", + "scene": "t26", + "seed": 1001, + "job_id": "6c6ebdc54886", + "status": "succeeded", + "error": null, + "output": "BenchT26Ref2VA/20261106-033548-3a857c4d/final/BenchT26Ref2VA-clip.0-0.0.mp4" + }, + { + "arm": "c2-unmentioned-s2002", + "scene": "t26", + "seed": 2002, + "job_id": "ae1d84995e95", + "status": "succeeded", + "error": null, + "output": "BenchT26Ref2VA/20261106-034400-d520e7c7/final/BenchT26Ref2VA-clip.0-0.0.mp4" + }, + { + "arm": "c2-control-s1001", + "scene": "t26", + "seed": 1001, + "job_id": "e1d5e3dcc86b", + "status": "succeeded", + "error": null, + "output": "BenchT26Ref2VA/20261106-035011-9246a409/final/BenchT26Ref2VA-clip.0-0.0.mp4" + }, + { + "arm": "c2-control-s2002", + "scene": "t26", + "seed": 2002, + "job_id": "e05c79b91eda", + "status": "succeeded", + "error": null, + "output": "BenchT26Ref2VA/20261106-035727-3d0b72ac/final/BenchT26Ref2VA-clip.0-0.0.mp4" + }, + { + "arm": "c3-fullbody-s1001", + "scene": "t26", + "seed": 1001, + "job_id": "aea9c681dc2f", + "status": "succeeded", + "error": null, + "output": "BenchT26Ref2VA/20261106-040156-d5acb2d8/final/BenchT26Ref2VA-clip.0-0.0.mp4" + }, + { + "arm": "c3-fullbody-s2002", + "scene": "t26", + "seed": 2002, + "job_id": "0d5f4cafb7f2", + "status": "succeeded", + "error": null, + "output": "BenchT26Ref2VA/20261106-040627-576c609c/final/BenchT26Ref2VA-clip.0-0.0.mp4" + }, + { + "arm": "c3-facecrop-s1001", + "scene": "t26", + "seed": 1001, + "job_id": "07e3855ea57c", + "status": "succeeded", + "error": null, + "output": "BenchT26Ref2VA/20261106-041057-03669648/final/BenchT26Ref2VA-clip.0-0.0.mp4" + }, + { + "arm": "c3-facecrop-s2002", + "scene": "t26", + "seed": 2002, + "job_id": "fa445bdc6ba1", + "status": "succeeded", + "error": null, + "output": "BenchT26Ref2VA/20261106-041426-8cd99e99/final/BenchT26Ref2VA-clip.0-0.0.mp4" + }, + { + "arm": "c4-continued-s1001", + "scene": "t26", + "seed": 1001, + "job_id": "1b4869e2676a", + "status": "succeeded", + "error": null, + "output": "BenchT26Continued/20261106-041753-5dc78d09/final/seg2_carry_only-0.0.mp4" + }, + { + "arm": "c4-continued-s2002", + "scene": "t26", + "seed": 2002, + "job_id": "a3930b5f3fb4", + "status": "succeeded", + "error": null, + "output": "BenchT26Continued/20261106-043836-bf81ac7b/final/seg2_carry_only-0.0.mp4" + } +] \ No newline at end of file diff --git a/bench/records/t26-facecheck.json b/bench/records/t26-facecheck.json new file mode 100644 index 000000000..79219b0d7 --- /dev/null +++ b/bench/records/t26-facecheck.json @@ -0,0 +1,34 @@ +[ + { + "arm": "facecheck-c2-unmentioned-s1001", + "scene": "t26", + "seed": 0, + "job_id": "43d225620b94", + "status": "succeeded", + "error": "ValueError: job 43d225620b94 wrote no video" + }, + { + "arm": "facecheck-c2-unmentioned-s2002", + "scene": "t26", + "seed": 0, + "job_id": "4fb8ff3ee82b", + "status": "succeeded", + "error": "ValueError: job 4fb8ff3ee82b wrote no video" + }, + { + "arm": "facecheck-c2-control-s1001", + "scene": "t26", + "seed": 0, + "job_id": "bf8c275d35d3", + "status": "succeeded", + "error": "ValueError: job bf8c275d35d3 wrote no video" + }, + { + "arm": "facecheck-c2-control-s2002", + "scene": "t26", + "seed": 0, + "job_id": "c00bec92b585", + "status": "succeeded", + "error": "ValueError: job c00bec92b585 wrote no video" + } +] \ No newline at end of file diff --git a/bench/records/t26-prep.json b/bench/records/t26-prep.json new file mode 100644 index 000000000..1e4fa4491 --- /dev/null +++ b/bench/records/t26-prep.json @@ -0,0 +1,19 @@ +[ + { + "arm": "t26-portraits", + "scene": "t26", + "seed": 1193, + "job_id": "af6dd086993b", + "status": "succeeded", + "error": "ValueError: job af6dd086993b wrote no video" + }, + { + "arm": "c1-line-plain-s1001-probe", + "scene": "t26", + "seed": 1001, + "job_id": "26c02d9ea5a4", + "status": "succeeded", + "error": null, + "output": "BenchT26T2VA/20261106-032334-b777890e/final/BenchT26T2VA-clip.0-0.0.mp4" + } +] \ No newline at end of file diff --git a/bench/records/tiers.json b/bench/records/tiers.json new file mode 100644 index 000000000..0422f2199 --- /dev/null +++ b/bench/records/tiers.json @@ -0,0 +1,1265 @@ +{ + "records": [ + { + "arm": "t20-ctl-s1-s1166", + "scene": "s1-static", + "seed": 1166, + "job_id": "f5bb2add1911", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-t20/20261106-152816-82fa821f/final/BenchS1Static-t20-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.8539999999999999, + "seam_jump_max": 0.97, + "seam_look_mean": 1.464, + "seam_look_max": 2.13, + "seam_level_step_db_mean": 7.653999999999999, + "seam_level_step_db_max": 16.58, + "texture_fine_seg_last": 0.5356445216304937, + "texture_mid_seg_last": 0.17558025571703065, + "texture_coarse_seg_last": 0.016023705618812927, + "wall_minutes": 14.115122592449188 + } + }, + { + "arm": "t20-c672-s1-s1166", + "scene": "s1-static", + "seed": 1166, + "job_id": "f9d87fbc0d2f", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-t20/20261106-164514-18367151/final/BenchS1Static-t20-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.9120000000000001, + "seam_jump_max": 1.1, + "seam_look_mean": 1.562, + "seam_look_max": 2.29, + "seam_level_step_db_mean": 10.912, + "seam_level_step_db_max": 21.1, + "texture_fine_seg_last": 0.3302821862207051, + "texture_mid_seg_last": 0.13450270795942615, + "texture_coarse_seg_last": 0.00801344397042457, + "wall_minutes": 11.130547018845876 + } + }, + { + "arm": "t20-st4-s1-s1166", + "scene": "s1-static", + "seed": 1166, + "job_id": "cc6e23eb555a", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-t20/20261106-165618-af800782/final/BenchS1Static-t20-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.96, + "seam_jump_max": 1.52, + "seam_look_mean": 1.6940000000000002, + "seam_look_max": 2.21, + "seam_level_step_db_mean": 10.82, + "seam_level_step_db_max": 14.72, + "texture_fine_seg_last": 0.5192449889208433, + "texture_mid_seg_last": 0.19545688686485782, + "texture_coarse_seg_last": 0.016648633996337862, + "wall_minutes": 9.340407319863637 + } + }, + { + "arm": "t20-c1344-s1-s1166", + "scene": "s1-static", + "seed": 1166, + "job_id": "1c5014a737ab", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-t20/20261106-170539-5194793c/final/BenchS1Static-t20-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.73, + "seam_jump_max": 0.78, + "seam_look_mean": 1.436, + "seam_look_max": 1.91, + "seam_level_step_db_mean": 5.2, + "seam_level_step_db_max": 9.77, + "texture_fine_seg_last": 1.0077467783553395, + "texture_mid_seg_last": 0.4865680854664115, + "texture_coarse_seg_last": 0.04847271702323955, + "wall_minutes": 26.626994999249778 + } + }, + { + "arm": "t20-c1344-int8-s1-s1166", + "scene": "s1-static", + "seed": 1166, + "job_id": "6810b23c2e35", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-t20/20261106-173216-1c1c9dcf/final/BenchS1Static-t20-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.628, + "seam_jump_max": 0.67, + "seam_look_mean": 0.442, + "seam_look_max": 0.59, + "seam_level_step_db_mean": 13.318000000000001, + "seam_level_step_db_max": 24.33, + "texture_fine_seg_last": 0.3588784874488774, + "texture_mid_seg_last": 0.2721636854662333, + "texture_coarse_seg_last": 0.000765596706577355, + "wall_minutes": 23.740568236509958 + } + }, + { + "arm": "t20-ctl-s2-s1166", + "scene": "s2-arc", + "seed": 1166, + "job_id": "e649195caf6c", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc-t20/20261106-175606-0dbca112/final/BenchS2Arc-t20-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.3619999999999999, + "seam_jump_max": 2.09, + "seam_look_mean": 1.286, + "seam_look_max": 1.74, + "seam_level_step_db_mean": 1.4620000000000002, + "seam_level_step_db_max": 3.43, + "texture_fine_seg_last": 3.083963956401898, + "texture_mid_seg_last": 2.2932110623147106, + "texture_coarse_seg_last": 1.4297775601219735, + "wall_minutes": 13.092115044593811 + } + }, + { + "arm": "t20-c672-s2-s1166", + "scene": "s2-arc", + "seed": 1166, + "job_id": "12bc51e5dbe7", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc-t20/20261106-180906-d0311b05/final/BenchS2Arc-t20-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.6019999999999999, + "seam_jump_max": 1.81, + "seam_look_mean": 2.222, + "seam_look_max": 3.48, + "seam_level_step_db_mean": 3.8899999999999997, + "seam_level_step_db_max": 7.35, + "texture_fine_seg_last": 3.4683960165186796, + "texture_mid_seg_last": 2.5565626934756747, + "texture_coarse_seg_last": 1.5009749573336681, + "wall_minutes": 9.260554714997609 + } + }, + { + "arm": "t20-c1344-s2-s1166", + "scene": "s2-arc", + "seed": 1166, + "job_id": "4b3d50baadbf", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc-t20/20261106-181822-83df4dce/final/BenchS2Arc-t20-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.1280000000000001, + "seam_jump_max": 1.5, + "seam_look_mean": 1.598, + "seam_look_max": 2.48, + "seam_level_step_db_mean": 3.724, + "seam_level_step_db_max": 8.98, + "texture_fine_seg_last": 2.601751299478124, + "texture_mid_seg_last": 1.3160584576920404, + "texture_coarse_seg_last": 0.21399790856555714, + "wall_minutes": 23.19484119017919 + } + }, + { + "arm": "t20-ctl-s3-s1166", + "scene": "s3-dialogue-walk", + "seed": 1166, + "job_id": "1b406282af1a", + "status": "succeeded", + "error": null, + "output": "BenchS3DialogueWalk-t20/20261106-184135-b01f8f1b/final/BenchS3DialogueWalk-t20-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.524, + "seam_jump_max": 1.83, + "seam_look_mean": 1.47, + "seam_look_max": 1.71, + "seam_level_step_db_mean": 2.968, + "seam_level_step_db_max": 8.81, + "texture_fine_seg_last": 0.0674850955709072, + "texture_mid_seg_last": 0.11366149849198895, + "texture_coarse_seg_last": 0.18802880506763223, + "wall_minutes": 10.877164657910665 + } + }, + { + "arm": "t20-c672-s3-s1166", + "scene": "s3-dialogue-walk", + "seed": 1166, + "job_id": "f566f7ff5971", + "status": "succeeded", + "error": null, + "output": "BenchS3DialogueWalk-t20/20261106-185226-fb0d3ce5/final/BenchS3DialogueWalk-t20-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.568, + "seam_jump_max": 1.97, + "seam_look_mean": 1.8399999999999999, + "seam_look_max": 2.73, + "seam_level_step_db_mean": 3.9839999999999995, + "seam_level_step_db_max": 11.1, + "texture_fine_seg_last": 0.8372636436964732, + "texture_mid_seg_last": 0.7210462856578476, + "texture_coarse_seg_last": 0.49963813638055593, + "wall_minutes": 7.85509439309438 + } + }, + { + "arm": "t20-c1344-s3-s1166", + "scene": "s3-dialogue-walk", + "seed": 1166, + "job_id": "27cdc5df3a40", + "status": "succeeded", + "error": null, + "output": "BenchS3DialogueWalk-t20/20261106-190018-f6fc4249/final/BenchS3DialogueWalk-t20-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.244, + "seam_jump_max": 1.52, + "seam_look_mean": 1.402, + "seam_look_max": 1.77, + "seam_level_step_db_mean": 3.6420000000000003, + "seam_level_step_db_max": 12.13, + "texture_fine_seg_last": 0.4617625025287106, + "texture_mid_seg_last": 0.19037516225523277, + "texture_coarse_seg_last": 0.08028926449034897, + "wall_minutes": 20.173055589199066 + } + }, + { + "arm": "t20-ctl-s4-s1166", + "scene": "s4-montage", + "seed": 1166, + "job_id": "6d33a88d24de", + "status": "succeeded", + "error": null, + "output": "BenchS4Montage-t20/20261106-192033-718e41fa/final/BenchS4Montage-t20-film.14-0.0.mp4", + "measures": { + "seam_jump_mean": 20.476666666666667, + "seam_jump_max": 22.69, + "seam_look_mean": 13.473333333333331, + "seam_look_max": 18.24, + "seam_level_step_db_mean": 3.4966666666666666, + "seam_level_step_db_max": 4.42, + "texture_fine_seg_last": 1.7140760391646614, + "texture_mid_seg_last": 0.5798479202075943, + "texture_coarse_seg_last": 1.214317730531653, + "wall_minutes": 20.300911211967467 + } + }, + { + "arm": "t20-c672-s4-s1166", + "scene": "s4-montage", + "seed": 1166, + "job_id": "217149738709", + "status": "succeeded", + "error": null, + "output": "BenchS4Montage-t20/20261106-194046-a0ba9681/final/BenchS4Montage-t20-film.14-0.0.mp4", + "measures": { + "seam_jump_mean": 18.566666666666666, + "seam_jump_max": 23.38, + "seam_look_mean": 14.053333333333333, + "seam_look_max": 17.91, + "seam_level_step_db_mean": 2.8433333333333333, + "seam_level_step_db_max": 4.14, + "texture_fine_seg_last": 1.8119346587536003, + "texture_mid_seg_last": 0.6186453830740996, + "texture_coarse_seg_last": 1.0778464584666931, + "wall_minutes": 14.526031410694122 + } + }, + { + "arm": "t20-st5-s4-s1166", + "scene": "s4-montage", + "seed": 1166, + "job_id": "cad009e3c50b", + "status": "succeeded", + "error": null, + "output": "BenchS4Montage-t20/20261106-195518-e89bd4fd/final/BenchS4Montage-t20-film.14-0.0.mp4", + "measures": { + "seam_jump_mean": 19.81, + "seam_jump_max": 22.29, + "seam_look_mean": 12.546666666666667, + "seam_look_max": 17.3, + "seam_level_step_db_mean": 3.5133333333333336, + "seam_level_step_db_max": 4.61, + "texture_fine_seg_last": 2.183647378873617, + "texture_mid_seg_last": 1.1447336981334733, + "texture_coarse_seg_last": 1.7710712942778766, + "wall_minutes": 11.425593213240306 + } + }, + { + "arm": "t20-c1344-s4-s1166", + "scene": "s4-montage", + "seed": 1166, + "job_id": "518d49656426", + "status": "succeeded", + "error": null, + "output": "BenchS4Montage-t20/20261106-200643-cd42805f/final/BenchS4Montage-t20-film.14-0.0.mp4", + "measures": { + "seam_jump_mean": 20.87, + "seam_jump_max": 23.45, + "seam_look_mean": 14.37, + "seam_look_max": 20.03, + "seam_level_step_db_mean": 3.52, + "seam_level_step_db_max": 5.15, + "texture_fine_seg_last": 0.09549922703287453, + "texture_mid_seg_last": 0.4102656307713152, + "texture_coarse_seg_last": 0.07494789720569656, + "wall_minutes": 25.79624469280243 + } + }, + { + "arm": "t20-c1344-int8-s4-s1166", + "scene": "s4-montage", + "seed": 1166, + "job_id": "413a48627655", + "status": "succeeded", + "error": null, + "output": "BenchS4Montage-t20/20261106-203231-c7342114/final/BenchS4Montage-t20-film.14-0.0.mp4", + "measures": { + "seam_jump_mean": 19.493333333333332, + "seam_jump_max": 21.99, + "seam_look_mean": 13.236666666666666, + "seam_look_max": 18.7, + "seam_level_step_db_mean": 3.7466666666666666, + "seam_level_step_db_max": 5.7, + "texture_fine_seg_last": 0.008836387561767543, + "texture_mid_seg_last": 0.3077449866933474, + "texture_coarse_seg_last": 0.14507158447689864, + "wall_minutes": 26.764348049958546 + } + }, + { + "arm": "t20-int8-s4-s1166", + "scene": "s4-montage", + "seed": 1166, + "job_id": "26e3fef72e97", + "status": "succeeded", + "error": null, + "output": "BenchS4Montage-t20/20261106-205917-bf029714/final/BenchS4Montage-t20-film.14-0.0.mp4", + "measures": { + "seam_jump_mean": 19.076666666666668, + "seam_jump_max": 21.64, + "seam_look_mean": 12.29, + "seam_look_max": 16.07, + "seam_level_step_db_mean": 3.25, + "seam_level_step_db_max": 4.15, + "texture_fine_seg_last": 1.868558872097837, + "texture_mid_seg_last": 0.6838765454977062, + "texture_coarse_seg_last": 0.9729203094041952, + "wall_minutes": 18.277531015872956 + } + }, + { + "arm": "t20-ctl-s1-s2", + "scene": "s1-static", + "seed": 2, + "job_id": "9badf299b28e", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-t20/20261106-211735-cbf6edfc/final/BenchS1Static-t20-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.792, + "seam_jump_max": 0.9, + "seam_look_mean": 1.3539999999999999, + "seam_look_max": 2.23, + "seam_level_step_db_mean": 10.552, + "seam_level_step_db_max": 17.74, + "texture_fine_seg_last": 0.4626851523202583, + "texture_mid_seg_last": 0.1453849701047438, + "texture_coarse_seg_last": 0.015569326422847992, + "wall_minutes": 12.918285779158275 + } + }, + { + "arm": "t20-c672-s1-s2", + "scene": "s1-static", + "seed": 2, + "job_id": "170650f97b54", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-t20/20261106-213029-4f1f7eb9/final/BenchS1Static-t20-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.8560000000000001, + "seam_jump_max": 1.07, + "seam_look_mean": 1.5939999999999999, + "seam_look_max": 1.98, + "seam_level_step_db_mean": 5.816, + "seam_level_step_db_max": 10.57, + "texture_fine_seg_last": 0.19864889339724406, + "texture_mid_seg_last": 0.11873915090731876, + "texture_coarse_seg_last": 0.04338757551795929, + "wall_minutes": 7.4917115012804665 + } + }, + { + "arm": "t20-c1344-s1-s2", + "scene": "s1-static", + "seed": 2, + "job_id": "bca819290484", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-t20/20261106-213758-9c4f1e57/final/BenchS1Static-t20-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.746, + "seam_jump_max": 0.83, + "seam_look_mean": 1.312, + "seam_look_max": 1.55, + "seam_level_step_db_mean": 5.906000000000001, + "seam_level_step_db_max": 17.28, + "texture_fine_seg_last": 1.0855251302327158, + "texture_mid_seg_last": 0.48133571915438367, + "texture_coarse_seg_last": 0.05941665279178097, + "wall_minutes": 19.61352721055349 + } + }, + { + "arm": "t20-st4-s2-s1166", + "scene": "s2-arc", + "seed": 1166, + "job_id": "d14a8877f205", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc-t20/20261106-215736-ad087337/final/BenchS2Arc-t20-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.362, + "seam_jump_max": 1.74, + "seam_look_mean": 2.1439999999999997, + "seam_look_max": 3.01, + "seam_level_step_db_mean": 3.254, + "seam_level_step_db_max": 5.88, + "texture_fine_seg_last": 3.050167480109008, + "texture_mid_seg_last": 1.889405389435909, + "texture_coarse_seg_last": 1.0218429284317911, + "wall_minutes": 9.388901472091675 + } + }, + { + "arm": "t20-int8-s1-s1166", + "scene": "s1-static", + "seed": 1166, + "job_id": "74c4feddb02b", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-t20/20261106-220702-d5598518/final/BenchS1Static-t20-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.8880000000000001, + "seam_jump_max": 1.11, + "seam_look_mean": 0.5900000000000001, + "seam_look_max": 0.76, + "seam_level_step_db_mean": 4.364, + "seam_level_step_db_max": 10.24, + "texture_fine_seg_last": 0.2694783614967982, + "texture_mid_seg_last": 0.04737409136129679, + "texture_coarse_seg_last": 0.017274928157711537, + "wall_minutes": 14.593006312847137 + } + }, + { + "arm": "t20-c1344-8step-s1-s1166", + "scene": "s1-static", + "seed": 1166, + "job_id": "856b243ef5fd", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-t20/20261106-173201-ff72add7/final/BenchS1Static-t20-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.012, + "seam_jump_max": 1.3, + "seam_look_mean": 2.48, + "seam_look_max": 2.88, + "seam_level_step_db_mean": 1.454, + "seam_level_step_db_max": 2.01, + "texture_fine_seg_last": 1.0260782072230006, + "texture_mid_seg_last": 0.36233581792874164, + "texture_coarse_seg_last": 0.10089041475455637, + "wall_minutes": 38.77653369108836 + } + }, + { + "arm": "t20-template-static-long-120s-s1166", + "scene": "s1-static", + "seed": 1166, + "job_id": "aa80458ab4f7", + "status": "succeeded", + "error": null, + "output": "BenchT20TemplateStaticLong-t20/20261106-154544-e8f987c4/final/BenchT20TemplateStaticLong-t20-take.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.7217857142857144, + "seam_jump_max": 1.44, + "seam_look_mean": 1.4985714285714287, + "seam_look_max": 2.67, + "seam_level_step_db_mean": 7.053571428571429, + "seam_level_step_db_max": 30.94, + "texture_fine_seg_last": 0.12073004337567397, + "texture_mid_seg_last": 0.015038198140201632, + "texture_coarse_seg_last": 0.007180848303426303, + "wall_minutes": 59.15389323631923 + } + }, + { + "arm": "t20-int8-s1-s2", + "scene": "s1-static", + "seed": 2, + "job_id": "d9e376286817", + "status": "failed", + "error": "Worker process died: killed by SIGKILL (typically the out-of-memory killer)" + }, + { + "arm": "t20-int8-s3-s1166", + "scene": "s3-dialogue-walk", + "seed": 1166, + "job_id": "bc828c7ffe51", + "status": "failed", + "error": "Worker process died: killed by SIGKILL (typically the out-of-memory killer)" + }, + { + "arm": "t20-ctl-s2-s2", + "scene": "s2-arc", + "seed": 2, + "job_id": "96a443e3aa38", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc-t20/20261106-224105-825983d4/final/BenchS2Arc-t20-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.394, + "seam_jump_max": 1.93, + "seam_look_mean": 1.452, + "seam_look_max": 2.0, + "seam_level_step_db_mean": 2.008, + "seam_level_step_db_max": 4.23, + "texture_fine_seg_last": 2.8618473366443644, + "texture_mid_seg_last": 1.6058030699028314, + "texture_coarse_seg_last": 0.5231217148768786, + "wall_minutes": 13.681074039141338 + } + }, + { + "arm": "t20-c1344-int8-s2-s1166", + "scene": "s2-arc", + "seed": 1166, + "job_id": "9fb01b149f00", + "status": "failed", + "error": "Worker process died: killed by SIGKILL (typically the out-of-memory killer)" + }, + { + "arm": "t20-int8-s3-s2", + "scene": "s3-dialogue-walk", + "seed": 2, + "job_id": "78440455b4f2", + "status": "failed", + "error": "Worker process died: killed by SIGKILL (typically the out-of-memory killer)" + }, + { + "arm": "t20-int8-s2-s1166", + "scene": "s2-arc", + "seed": 1166, + "job_id": "6c72b6242412", + "status": "failed", + "error": "Worker process died: killed by SIGKILL (typically the out-of-memory killer)" + }, + { + "arm": "t20-int8-s2-s2", + "scene": "s2-arc", + "seed": 2, + "job_id": "a6c8c306fe92", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc-t20/20261106-223833-ad8a3416/final/BenchS2Arc-t20-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.392, + "seam_jump_max": 1.64, + "seam_look_mean": 1.06, + "seam_look_max": 1.21, + "seam_level_step_db_mean": 6.1259999999999994, + "seam_level_step_db_max": 12.9, + "texture_fine_seg_last": 2.826671859024718, + "texture_mid_seg_last": 2.551120883179456, + "texture_coarse_seg_last": 3.654787829600797, + "wall_minutes": 14.923994016647338 + } + }, + { + "arm": "t20-c1344-int8-s1-s2", + "scene": "s1-static", + "seed": 2, + "job_id": "fe9149eb0028", + "status": "failed", + "error": "Worker process died: killed by SIGKILL (typically the out-of-memory killer)" + }, + { + "arm": "t20-ctl-s3-s2", + "scene": "s3-dialogue-walk", + "seed": 2, + "job_id": "81006f364dae", + "status": "succeeded", + "error": null, + "output": "BenchS3DialogueWalk-t20/20261106-225651-47a3f9b0/final/BenchS3DialogueWalk-t20-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.192, + "seam_jump_max": 1.69, + "seam_look_mean": 1.06, + "seam_look_max": 1.64, + "seam_level_step_db_mean": 2.59, + "seam_level_step_db_max": 6.22, + "texture_fine_seg_last": 0.293776369695831, + "texture_mid_seg_last": 0.22550331446142596, + "texture_coarse_seg_last": 0.29862160095698154, + "wall_minutes": 13.717829513549805 + } + }, + { + "arm": "t20-template-static-long-int8-120s-s1166", + "scene": "s1-static", + "seed": 1166, + "job_id": "1fbbc89cf0eb", + "status": "failed", + "error": "Workflow execution error: module diffusers has no attribute int8" + }, + { + "arm": "t20-st4int8-s1-s1166", + "scene": "s1-static", + "seed": 1166, + "job_id": "d79a0ff70ccf", + "status": "failed", + "error": "Worker process died: killed by SIGKILL (typically the out-of-memory killer)" + }, + { + "arm": "t20-st4int8-s2-s1166", + "scene": "s2-arc", + "seed": 1166, + "job_id": "3c57f30e4091", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc-t20/20261106-231554-44628425/final/BenchS2Arc-t20-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.44, + "seam_jump_max": 2.13, + "seam_look_mean": 1.238, + "seam_look_max": 2.22, + "seam_level_step_db_mean": 2.0, + "seam_level_step_db_max": 5.35, + "texture_fine_seg_last": 1.4032595731119095, + "texture_mid_seg_last": 0.4193340024882476, + "texture_coarse_seg_last": 0.8541313803980615, + "wall_minutes": 12.584421586990356 + } + }, + { + "arm": "t20-c1344-int8-s1-s2", + "scene": "s1-static", + "seed": 2, + "job_id": "00fddd52dd59", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-t20/20261106-233920-cabc3fa7/final/BenchS1Static-t20-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.732, + "seam_jump_max": 0.99, + "seam_look_mean": 0.484, + "seam_look_max": 0.86, + "seam_level_step_db_mean": 3.976, + "seam_level_step_db_max": 7.58, + "texture_fine_seg_last": 0.40556984537482693, + "texture_mid_seg_last": 0.2335645245862428, + "texture_coarse_seg_last": 0.03630281953473958, + "wall_minutes": 22.75272980531057 + } + }, + { + "arm": "t20-c1344-int8-s2-s1166", + "scene": "s2-arc", + "seed": 1166, + "job_id": "5c3dd03000a1", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc-t20/20261107-000206-1a0ad59a/final/BenchS2Arc-t20-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.468, + "seam_jump_max": 2.27, + "seam_look_mean": 0.998, + "seam_look_max": 1.16, + "seam_level_step_db_mean": 3.8, + "seam_level_step_db_max": 9.38, + "texture_fine_seg_last": 3.4182624590546697, + "texture_mid_seg_last": 3.552189517400435, + "texture_coarse_seg_last": 3.3721355044930474, + "wall_minutes": 20.505803497632346 + } + }, + { + "arm": "t20-int8-s1-s2", + "scene": "s1-static", + "seed": 2, + "job_id": "ebfd064fa9de", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-t20/20261107-002236-6a782614/final/BenchS1Static-t20-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.738, + "seam_jump_max": 1.03, + "seam_look_mean": 0.906, + "seam_look_max": 1.14, + "seam_level_step_db_mean": 3.522, + "seam_level_step_db_max": 7.96, + "texture_fine_seg_last": 0.3359574873437288, + "texture_mid_seg_last": 0.010950511859812048, + "texture_coarse_seg_last": 0.013805754629401607, + "wall_minutes": 12.337140166759491 + } + }, + { + "arm": "t20-int8-s2-s1166", + "scene": "s2-arc", + "seed": 1166, + "job_id": "f2dc3661076c", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc-t20/20261107-003456-5b0c71ab/final/BenchS2Arc-t20-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.77, + "seam_jump_max": 2.63, + "seam_look_mean": 1.574, + "seam_look_max": 2.15, + "seam_level_step_db_mean": 3.4219999999999997, + "seam_level_step_db_max": 10.52, + "texture_fine_seg_last": 2.894320793553489, + "texture_mid_seg_last": 2.316768463042273, + "texture_coarse_seg_last": 1.1275438764750392, + "wall_minutes": 12.34909850358963 + } + }, + { + "arm": "t20-int8-s3-s1166", + "scene": "s3-dialogue-walk", + "seed": 1166, + "job_id": "167656fcf86d", + "status": "succeeded", + "error": null, + "output": "BenchS3DialogueWalk-t20/20261107-004717-65723f22/final/BenchS3DialogueWalk-t20-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.8599999999999999, + "seam_jump_max": 2.57, + "seam_look_mean": 0.99, + "seam_look_max": 1.57, + "seam_level_step_db_mean": 2.348, + "seam_level_step_db_max": 5.05, + "texture_fine_seg_last": 0.2617250401485604, + "texture_mid_seg_last": 0.36709936333704496, + "texture_coarse_seg_last": 0.012196454827989601, + "wall_minutes": 12.366925501823426 + } + }, + { + "arm": "t20-int8-s3-s2", + "scene": "s3-dialogue-walk", + "seed": 2, + "job_id": "3242ffebdc08", + "status": "succeeded", + "error": null, + "output": "BenchS3DialogueWalk-t20/20261107-005939-bdc1b5f3/final/BenchS3DialogueWalk-t20-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 1.474, + "seam_jump_max": 1.92, + "seam_look_mean": 1.01, + "seam_look_max": 1.65, + "seam_level_step_db_mean": 1.492, + "seam_level_step_db_max": 1.82, + "texture_fine_seg_last": 0.2439627342744519, + "texture_mid_seg_last": 0.31917995286478856, + "texture_coarse_seg_last": 0.0028218977153204035, + "wall_minutes": 12.337414741516113 + } + }, + { + "arm": "t20-int8-static-long-120s-s1166", + "scene": "s1-static", + "seed": 1166, + "job_id": "513f5ddd8d76", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-t20/20261107-011200-aba34e1d/final/BenchS1Static-t20-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "seam_jump_mean": 0.5442857142857143, + "seam_jump_max": 0.87, + "seam_look_mean": 0.6582142857142858, + "seam_look_max": 1.38, + "seam_level_step_db_mean": 3.9846428571428567, + "seam_level_step_db_max": 16.11, + "texture_fine_seg_last": 0.05279142833081548, + "texture_mid_seg_last": 0.025078282074567238, + "texture_coarse_seg_last": 0.018855506674797972, + "wall_minutes": 62.18286972840627 + } + } + ], + "staging_and_crust": { + "t20-ctl-s1-s1166": { + "sharp": { + "62": 194.9, + "277": 359.4, + "583": 878.7 + } + }, + "t20-c672-s1-s1166": { + "sharp": { + "62": 284.6, + "277": 392.2, + "583": 942.3 + }, + "vs": "t20-ctl-s1-s1166", + "carry": { + "62": { + "psnr": 18.97, + "ssim": 0.7385 + }, + "277": { + "psnr": 19.14, + "ssim": 0.6434 + }, + "583": { + "psnr": 17.29, + "ssim": 0.3422 + } + } + }, + "t20-st4-s1-s1166": { + "sharp": { + "62": 202.6, + "277": 308.7, + "583": 793.2 + }, + "vs": "t20-ctl-s1-s1166", + "carry": { + "62": { + "psnr": 32.29, + "ssim": 0.9387 + }, + "277": { + "psnr": 27.11, + "ssim": 0.8697 + }, + "583": { + "psnr": 21.04, + "ssim": 0.7151 + } + } + }, + "t20-c1344-s1-s1166": { + "sharp": { + "62": 65.9, + "277": 97.6, + "583": 361.8 + }, + "vs": "t20-ctl-s1-s1166", + "carry": { + "62": { + "psnr": 19.28, + "ssim": 0.7267 + }, + "277": { + "psnr": 19.07, + "ssim": 0.5842 + }, + "583": { + "psnr": 18.25, + "ssim": 0.3246 + } + } + }, + "t20-c1344-int8-s1-s1166": { + "sharp": { + "62": 53.8, + "277": 50.3, + "583": 84.0 + }, + "vs": "t20-ctl-s1-s1166", + "carry": { + "62": { + "psnr": 17.28, + "ssim": 0.7222 + }, + "277": { + "psnr": 18.71, + "ssim": 0.6253 + }, + "583": { + "psnr": 15.0, + "ssim": 0.3469 + } + } + }, + "t20-ctl-s2-s1166": { + "sharp": { + "62": 1040.9, + "277": 667.8, + "583": 159.8 + } + }, + "t20-c672-s2-s1166": { + "sharp": { + "62": 1143.6, + "277": 769.0, + "583": 204.4 + }, + "vs": "t20-ctl-s2-s1166", + "carry": { + "62": { + "psnr": 12.69, + "ssim": 0.3141 + }, + "277": { + "psnr": 11.7, + "ssim": 0.2153 + }, + "583": { + "psnr": 9.69, + "ssim": 0.4627 + } + } + }, + "t20-c1344-s2-s1166": { + "sharp": { + "62": 410.1, + "277": 320.4, + "583": 126.8 + }, + "vs": "t20-ctl-s2-s1166", + "carry": { + "62": { + "psnr": 16.2, + "ssim": 0.3646 + }, + "277": { + "psnr": 11.56, + "ssim": 0.1456 + }, + "583": { + "psnr": 9.72, + "ssim": 0.3589 + } + } + }, + "t20-ctl-s3-s1166": { + "sharp": { + "62": 80.1, + "277": 88.2, + "583": 107.2 + } + }, + "t20-c672-s3-s1166": { + "sharp": { + "62": 139.5, + "277": 155.0, + "583": 193.0 + }, + "vs": "t20-ctl-s3-s1166", + "carry": { + "62": { + "psnr": 15.25, + "ssim": 0.3878 + }, + "277": { + "psnr": 13.37, + "ssim": 0.3652 + }, + "583": { + "psnr": 10.37, + "ssim": 0.2678 + } + } + }, + "t20-c1344-s3-s1166": { + "sharp": { + "62": 40.7, + "277": 50.9, + "583": 72.7 + }, + "vs": "t20-ctl-s3-s1166", + "carry": { + "62": { + "psnr": 15.46, + "ssim": 0.4041 + }, + "277": { + "psnr": 12.34, + "ssim": 0.3082 + }, + "583": { + "psnr": 10.4, + "ssim": 0.209 + } + } + }, + "t20-ctl-s1-s2": { + "sharp": { + "62": 164.8, + "277": 317.4, + "583": 803.9 + } + }, + "t20-c672-s1-s2": { + "sharp": { + "62": 266.7, + "277": 400.0, + "583": 1082.3 + }, + "vs": "t20-ctl-s1-s2", + "carry": { + "62": { + "psnr": 19.78, + "ssim": 0.7496 + }, + "277": { + "psnr": 21.72, + "ssim": 0.6694 + }, + "583": { + "psnr": 17.44, + "ssim": 0.3743 + } + } + }, + "t20-c1344-s1-s2": { + "sharp": { + "62": 76.6, + "277": 111.4, + "583": 501.5 + }, + "vs": "t20-ctl-s1-s2", + "carry": { + "62": { + "psnr": 20.69, + "ssim": 0.7372 + }, + "277": { + "psnr": 18.49, + "ssim": 0.568 + }, + "583": { + "psnr": 17.34, + "ssim": 0.2643 + } + } + }, + "t20-st4-s2-s1166": { + "sharp": { + "62": 1083.7, + "277": 787.2, + "583": 445.4 + }, + "vs": "t20-ctl-s2-s1166", + "carry": { + "62": { + "psnr": 21.6, + "ssim": 0.6602 + }, + "277": { + "psnr": 12.58, + "ssim": 0.2053 + }, + "583": { + "psnr": 10.53, + "ssim": 0.3715 + } + } + }, + "t20-int8-s1-s1166": { + "sharp": { + "62": 140.7, + "277": 136.9, + "583": 232.2 + }, + "vs": "t20-ctl-s1-s1166", + "carry": { + "62": { + "psnr": 16.97, + "ssim": 0.7312 + }, + "277": { + "psnr": 17.85, + "ssim": 0.6812 + }, + "583": { + "psnr": 15.44, + "ssim": 0.4336 + } + } + }, + "t20-c1344-8step-s1-s1166": { + "sharp": { + "62": 73.1, + "277": 149.7, + "583": 636.4 + }, + "vs": "t20-ctl-s1-s1166", + "carry": { + "62": { + "psnr": 18.35, + "ssim": 0.714 + }, + "277": { + "psnr": 18.18, + "ssim": 0.504 + }, + "583": { + "psnr": 15.22, + "ssim": 0.2295 + } + } + }, + "t20-ctl-s4-s1166": { + "sharp": { + "62": 258.7, + "186": 159.8, + "434": 119.4 + } + }, + "t20-c672-s4-s1166": { + "sharp": { + "62": 373.9, + "186": 236.7, + "434": 215.2 + }, + "vs": "t20-ctl-s4-s1166", + "carry": { + "62": { + "psnr": 13.02, + "ssim": 0.2292 + }, + "186": { + "psnr": 15.12, + "ssim": 0.3453 + }, + "434": { + "psnr": 15.34, + "ssim": 0.3847 + } + } + }, + "t20-st5-s4-s1166": { + "sharp": { + "62": 270.9, + "186": 169.9, + "434": 148.3 + }, + "vs": "t20-ctl-s4-s1166", + "carry": { + "62": { + "psnr": 14.15, + "ssim": 0.252 + }, + "186": { + "psnr": 17.48, + "ssim": 0.4403 + }, + "434": { + "psnr": 15.48, + "ssim": 0.3909 + } + } + }, + "t20-c1344-s4-s1166": { + "sharp": { + "62": 140.5, + "186": 78.5, + "434": 66.2 + }, + "vs": "t20-ctl-s4-s1166", + "carry": { + "62": { + "psnr": 12.42, + "ssim": 0.1968 + }, + "186": { + "psnr": 15.4, + "ssim": 0.3274 + }, + "434": { + "psnr": 15.19, + "ssim": 0.3219 + } + } + }, + "t20-c1344-int8-s4-s1166": { + "sharp": { + "62": 167.5, + "186": 74.1, + "434": 61.5 + }, + "vs": "t20-ctl-s4-s1166", + "carry": { + "62": { + "psnr": 12.71, + "ssim": 0.1889 + }, + "186": { + "psnr": 14.16, + "ssim": 0.2938 + }, + "434": { + "psnr": 14.36, + "ssim": 0.2874 + } + } + }, + "t20-int8-s4-s1166": { + "sharp": { + "62": 326.5, + "186": 138.6, + "434": 141.8 + }, + "vs": "t20-ctl-s4-s1166", + "carry": { + "62": { + "psnr": 12.26, + "ssim": 0.2027 + }, + "186": { + "psnr": 14.37, + "ssim": 0.3827 + }, + "434": { + "psnr": 14.73, + "ssim": 0.3578 + } + } + }, + "t20-ctl-long-s1166": { + "sharp": { + "62": 194.9, + "1501": 351.1, + "2868": 393.5 + } + }, + "t20-int8-long-s1166": { + "sharp": { + "62": 146.8, + "1501": 185.8, + "2868": 197.7 + }, + "vs": "t20-ctl-long-s1166", + "carry": { + "62": { + "psnr": 16.83, + "ssim": 0.7276 + }, + "1501": { + "psnr": 22.64, + "ssim": 0.7774 + }, + "2868": { + "psnr": 28.83, + "ssim": 0.8701 + } + } + } + } +} diff --git a/bench/records/tiers.md b/bench/records/tiers.md new file mode 100644 index 000000000..b2a0a965a --- /dev/null +++ b/bench/records/tiers.md @@ -0,0 +1,250 @@ +# Tiers on the long-form templates (Task 20) + +Run 2026-11-06 on the bench server (lem, RTX 3090s) at `279086b1`. From 15:45 +UTC it ran on cuda:0 only, at Don's request. From 17:35 a second server on +:8767 (cuda:1, same workspace) took half of the arms. Arms: +`bench/arms/tiers.json`, `tiers-b.json`, `tiers-c.json`, `tiers-int8.json` and +`tiers-template.json`. Wall times are cuda:0 unless a row says cuda:1, which +runs about 10% slower. Every arm is seed 1166 unless named `-s2`. S1-S3 run 6 +segments (634 frames, 26.4 s). S4 is the 4-shot montage. + +## What the tiers are + +The `tier` variable (`draft`, `standard`, `best`; default `standard`) moves +canvas, steps and weights together through `presets` +(`dw/presets.py`). Any variable the caller passes still wins. `tier` is +independent of `shot` on `minimax/chained-segments`. `shot` picks the +drift counters, `tier` the render settings, and the two bundles share no +variable. + +| template | draft | standard (unchanged default) | best | +|---|---|---|---| +| `minimax/chained-segments` (FL2VA, 4-step v1.2 LoRA) | 960x544, **4 steps**, int4 | 960x544, 5 steps, int4 | **1344x768**, 5 steps, **int8** | +| `minimax/chain-video-continuity`, `minimax/chain-matched-to-audio`, `minimax/chain-matched-and-aligned`, `minimax/dialogue-short`, `minimax/music-video` (Ref2VA, 8-step v1.0 LoRA) | 960x544, **5 steps**, int4 | 960x544, 9 steps, int4 | **1344x768**, 9 steps, **int8** | + +Wall time against standard, from the 26 s scenes, as a guide for a take +of any length: + +| template family | draft | standard | best | +|---|---|---|---| +| FL2VA chain (S1, S2) | 0.66-0.72x | 1x (S1 14.1 min for 26 s; 2-minute `static_long` take 59.2 min) | 1.68x on S1 (about 100 min for a 2-minute take, estimated, not run) | +| Ref2VA shots (S4) | 0.56x | 1x (S4 20.3 min) | 1.32x | + +**Cost rule (Step 2): passes.** `standard` is today's default, and a 2-minute +take at `standard` with `shot: static_long` ran **59.2 min** on one card +(`aa80458ab4f7`, below), well under the 3 h line. `best` is about 100 min +for the same take. It needs about 65 GB of host RAM per job, so a 128 GB +box runs one such job at a time (*Host RAM*). + +Not tiered: + +- **`ltx2/chained-segments`, `ltx2/extend-clip`: skipped.** LTX-2.5's + distilled transformer runs a fixed 8-sigma schedule with no step knob + (`docs/RECIPES_24GB.md`). Only `uint4` fits with the transformer resident + on a 24 GB card, so no weights step exists either. Canvas is the only dial + left. It is the caller's aspect choice (768x768 and 768x448 by default), and + the recipe spends headroom on the two-stage refine rather than on base + resolution. The quality step past the default is a separate pass + (`ltx2/upscale-clip`, `two-stage`), not a setting of the chain. Takes are + capped at about 15-20 s (`ltx.md`), so a draft would save minutes, not + hours. +- **Kandinsky 6:** out of long-form scope (ruling). +- **Attention, cache, check and retries** are not in any bundle. Sparse + top-k 0.15 is already the default at every tier (`speed.md`). The + `first_block` and MagCache caches are measured only on the base model's + 20-30-step schedule (#862), not on these 4- and 8-step LoRAs. The chain + check's re-roll cannot fix texture drift (`check.md`). These stay knobs. + +## Step 1: quality per minute + +### FL2VA chain (`minimax/chained-segments` settings, bench S1/S2/S3) + +Records `bench/records/tiers.json` (the driver records, failures included, and the staging/crust pass), copied from `run1/tiers.out.json`, `run2.out.json`, `cuda1.out.json`, +`i8*.out.json` (lem `~/bench-t20`). Measures are the driver's (lower is +better). Staging carry is PSNR/SSIM against the same scene and seed's +standard control at frames 62, 277 and 583, both frames resized to 480x272. +PSNR bands from `speed.md`: about 44 dB is rerun noise, 23-27 dB the same +shot with changed numerics, and 19 dB or less a re-staged shot. On the S2 arc, +even a lossless change reads 9-13 dB. Crust is the Laplacian variance of the +whole frame at native size. It grows with the burn, so read its rise across +the take within one canvas only. + +| arm | job | jump | look | level dB | tex fine | tex mid | tex coarse | crust f62 / f583 | carry PSNR f62 / f277 / f583 | wall min | +|---|---|---|---|---|---|---|---|---|---|---| +| S1 standard (ctl) | `f5bb2add1911` | 0.854 | 1.464 | 7.65 | 0.536 | 0.176 | 0.016 | 195 / 879 | - | 14.1 | +| S1 4 steps (draft) | `cc6e23eb555a` | 0.960 | 1.694 | 10.82 | 0.519 | 0.195 | 0.017 | 203 / 793 | **32.3 / 27.1 / 21.0** | **9.3** | +| S1 672x384 | `f9d87fbc0d2f` | 0.912 | 1.562 | 10.91 | 0.330 | 0.135 | 0.008 | (canvas) | 19.0 / 19.1 / 17.3 | 11.1 | +| S1 int8 | `74c4feddb02b` | 0.888 | **0.590** | 4.36 | **0.269** | 0.047 | 0.017 | **141 / 232** | 17.0 / 17.9 / 15.4 | 14.6 | +| S1 1344x768 | `1c5014a737ab` | 0.730 | 1.436 | 5.20 | 1.008 | 0.487 | 0.048 | (canvas) 66 / 362 | 19.3 / 19.1 / 18.3 | 26.6 | +| S1 1344x768 int8 (best) | `6810b23c2e35` | **0.628** | **0.442** | 13.32 | 0.359 | 0.272 | 0.001 | (canvas) **54 / 84** | 17.3 / 18.7 / 15.0 | 23.7 | +| S1 1344x768 8-step v1.0 @9 | `856b243ef5fd` (cuda:1) | 1.012 | 2.480 | 1.45 | 1.026 | 0.362 | 0.101 | (canvas) 73 / 636 | 18.4 / 18.2 / 15.2 | 38.8 | +| S1 standard, seed 2 | `9badf299b28e` | 0.792 | 1.354 | 10.55 | 0.463 | 0.145 | 0.016 | 165 / 804 | - | 12.9 | +| S1 672x384, seed 2 | `170650f97b54` | 0.856 | 1.594 | 5.82 | 0.199 | 0.119 | 0.043 | (canvas) | 19.8 / 21.7 / 17.4 | 7.5 | +| S1 1344x768, seed 2 | `bca819290484` | 0.746 | 1.312 | 5.91 | 1.086 | 0.481 | 0.059 | (canvas) 77 / 502 | 20.7 / 18.5 / 17.3 | 19.6 | +| S2 standard (ctl) | `e649195caf6c` | 1.362 | 1.286 | 1.46 | 3.084 | 2.293 | 1.430 | - | - | 13.1 | +| S2 4 steps (draft) | `d14a8877f205` | 1.362 | 2.144 | 3.25 | 3.050 | 1.889 | 1.022 | - | **21.6** / 12.6 / 10.5 | **9.4** | +| S2 672x384 | `12bc51e5dbe7` | 1.602 | 2.222 | 3.89 | 3.468 | 2.557 | 1.501 | - | 12.7 / 11.7 / 9.7 | 9.3 | +| S2 1344x768 | `4b3d50baadbf` | 1.128 | 1.598 | 3.72 | 2.602 | 1.316 | 0.214 | - | 16.2 / 11.6 / 9.7 | 23.2 | +| S2 standard, seed 2 | `96a443e3aa38` | 1.394 | 1.452 | 2.01 | 2.862 | 1.606 | 0.523 | - | - | 13.7 | +| S2 int8, seed 2 | `a6c8c306fe92` (cuda:1) | 1.392 | **1.060** | 6.13 | 2.827 | 2.551 | 3.655 | - | - | 14.9 | +| S2 4 steps int8 | `3c57f30e4091` (cuda:1) | 1.440 | 1.238 | 2.00 | 1.403 | 0.419 | 0.854 | - | - | 12.6 | +| S3 standard (ctl) | `1b406282af1a` | 1.524 | 1.470 | 2.97 | 0.067 | 0.114 | 0.188 | 80 / 107 | - | 10.9 | +| S3 672x384 | `f566f7ff5971` | 1.568 | 1.840 | 3.98 | 0.837 | 0.721 | 0.500 | (canvas) | 15.3 / 13.4 / 10.4 | 7.9 | +| S3 1344x768 | `27cdc5df3a40` | 1.244 | 1.402 | 3.64 | 0.462 | 0.190 | 0.080 | (canvas) | 15.5 / 12.3 / 10.4 | 20.2 | +| S3 standard, seed 2 | `81006f364dae` | 1.192 | 1.060 | 2.59 | 0.294 | 0.226 | 0.299 | - | - | 13.7 | + +Int8 confirmation (`bench/arms/tiers-int8.json`). Run one at a time on :8766 +(`i8e`), after the two-server OOM (*Host RAM*). Controls are the rows above. + +| arm | job | jump | look | level dB | tex fine | tex mid | tex coarse | wall min | against its control | +|---|---|---|---|---|---|---|---|---|---| +| S1 int8, seed 2 | `ebfd064fa9de` | 0.738 | **0.906** | 3.52 | **0.336** | 0.011 | 0.014 | 12.3 | wins look (1.354) and fine texture (0.463) | +| S1 1344x768 int8, seed 2 | `00fddd52dd59` | 0.732 | **0.484** | 3.98 | 0.406 | 0.234 | 0.036 | 22.8 | wins look; best of seed 2 again | +| S2 int8 | `f2dc3661076c` | _1.770_ | 1.574 | 3.42 | 2.894 | 2.317 | 1.128 | 12.3 | **jump worse** (1.362), look inside the gap | +| S2 int8, seed 2 | `a6c8c306fe92` (cuda:1) | 1.392 | **1.060** | 6.13 | 2.827 | 2.551 | _3.655_ | 14.9 | wins look, coarse worse: mixed | +| S2 1344x768 int8 | `5c3dd03000a1` | _1.468_ | **0.998** | 3.80 | 3.418 | 3.552 | 3.372 | 20.5 | wins look, jump and mid/coarse worse: mixed | +| S3 int8 | `167656fcf86d` | _1.860_ | **0.990** | 2.35 | _0.262_ | 0.367 | 0.012 | 12.4 | jump and fine worse, look wins: mixed | +| S3 int8, seed 2 | `3242ffebdc08` | _1.474_ | 1.010 | 1.49 | 0.244 | 0.319 | 0.003 | 12.3 | jump worse (1.192), rest inside: mixed | + +**Two minutes, `static_long` (keystone slot last), int8** on the S1 bench +copy, the same chain as the template run below (`t20-int8-static-long-120s-s1166`): + +| arm | job | jump | look | level dB | tex fine | tex mid | tex coarse | crust f62 / f1501 / f2868 | wall min | +|---|---|---|---|---|---|---|---|---|---| +| int4 (the template run, below) | `aa80458ab4f7` | 0.722 | 1.499 | 7.05 | 0.121 | 0.015 | 0.007 | 195 / 351 / 394 | 59.2 | +| int8 | `513f5ddd8d76` | **0.544** | **0.658** | 3.99 | **0.053** | 0.025 | 0.019 | 147 / 186 / 198 | 62.2 (+5%) | + +By eye at 62, 1501 and 2868 frames both two-minute takes hold the wall and +the face, since the keystone slot does most of the work. The int8 take is +the cleaner of the two at every moment: smoother stripes, less grain on the +white ground and the circle. + +**Int8 verdict.** On the static scene it wins on both seeds, at 26 s and at +two minutes, for 0-5% more wall time. On the arc and the dialogue walk it is +mixed on every arm: seam look improves, and seam jump gets worse on three of +four arms, past the 0.124 rerun gap. So it is a static-shot lever, not a +general one. + +By eye (frames 62 and 583, S1 contact sheet; frame 583 on S2 and S3): + +- **Draft (4 steps)** is the standard take: same framing, pose and face at + frame 62 (PSNR 32 dB, closer than the "same shot" band), and the same crust + by 24 s. It diverges later in the chain, as any change does once seams + compound (21 dB at 24 s). On the S2 arc segment 1 carries (21.6 dB against + a 9-13 dB lossless floor) and the later segments go their own way. It + saves 28-34%. **This is the draft:** the only cheap point whose staging + carries to `standard`. +- **672x384** re-stages (19 dB S1, 12.7 dB S2) and looks worse: a blotchy + face and a crusted black circle on S1 at 24 s, a washed-out sky on S2, a + gaunter face that loses identity on S3. It saves 21-42%. Rejected: it is + neither a preview of the standard take nor a usable take. +- **int8** removes most of the S1 burn. At 24 s the wall's stripes are + smooth, the white ground is clean and the face is natural, where int4 has + crusted everything. The measures agree: seam look 1.46 to 0.59, fine + texture drift 0.54 to 0.27, and crust rises 1.6x across the take + against 4.5x. Same wall time (14.6 against 14.1 min). It re-stages + (17 dB), so it is a different take, not a final of an int4 draft. +- **1344x768** alone keeps the int4 crust (crust 66 to 362, 5.5x) at 1.9x + the wall time. With int8 it is the cleanest S1 take measured: crust flat + (54 to 84), seam look 0.44, jump 0.63, at 1.68x the wall time (int8 runs + faster than int4 at this canvas: 23.7 against 26.6 min). On S2 and S3 + 1344x768 sharpens the scene and holds S3's face. **This is best.** +- **The 8-step v1.0 768p file at 9 steps** loses to the 4-step file at + the same canvas on every seam measure. Its crust reaches 636 by 24 s, and it + costs 2.7x standard. Rejected. + +### Ref2VA shots (`dialogue-short`/`music-video` settings, bench S4) + +S4 is four `dialogue-short` shots cut together, so its seams are cuts and +its seam measures are not drift (all arms read jump 19-21 and look 12-14). Read +the wall time and the eye check. PSNR against the control is 12-15 dB for +every arm, including the 5-step one. Talking close-ups move too much for a +frame match to show staging. + +| arm | job | tex fine | tex mid | tex coarse | wall min | +|---|---|---|---|---|---| +| S4 standard (ctl, 9 steps int4) | `6d33a88d24de` | 1.714 | 0.580 | 1.214 | 20.3 | +| S4 5 steps (draft) | `cad009e3c50b` | 2.184 | 1.145 | 1.771 | **11.4** | +| S4 672x384 | `217149738709` | 1.812 | 0.619 | 1.078 | 14.5 | +| S4 int8 | `26e3fef72e97` | 1.869 | 0.684 | 0.973 | 18.3 | +| S4 1344x768 | `518d49656426` | 0.095 | 0.410 | 0.075 | 25.8 | +| S4 1344x768 int8 (best) | `413a48627655` | 0.009 | 0.308 | 0.145 | 26.8 | + +By eye (frames 62 and 186): every arm gives the same close-up of the same +man with the same book and couch. Framing, pose and the read hold at 5 steps, +so the draft keeps the shot. 672x384 frames him tighter, a re-staged shot +for a 29% saving: rejected. The 1344x768 arms are the sharpest. Their +crop texture reads 0.01-0.10 against 1.7, though across cuts that is a +shot-to-shot difference, not drift. 1344x768 is the Ref2VA +LoRA's own canvas, and the H3 ramp's TV final (L7) is 1344x768 int8 too +(`plugins/dw/skills/minimax-h3/references/quality-and-speed.md`). +**best = 1344x768 int8, draft = 5 steps.** + +## The template run (`templates/minimax/chained-segments`, `shot: static_long`, 120 s) + +`bench/workflows/t20-template-static-long.json` composes the catalog +template by its catalog name, with S1's still and prompt, `shot: +static_long`, `duration_seconds: 120` and seed 1166. Everything else is the +template's own default. + +| job | jump | jump max | look | look max | level dB | tex fine | tex mid | tex coarse | wall min | +|---|---|---|---|---|---|---|---|---|---| +| `aa80458ab4f7` | 0.722 | 1.44 | 1.499 | 2.67 | 7.05 | 0.121 | 0.015 | 0.007 | **59.2** | + +The template path works end to end, inside the expected 57-62 min +(`structural.md`, *Keystone slot*). Fine-texture drift over two minutes is +0.12, against 0.54 for a 26 s standard chain with no keystone. A +step-cache-hit duplicate of this arm is `1f782a579fbb` (0.3 min). + +## Host RAM + +An int8 H3 worker holds about 63-69 GB of anonymous memory (the kernel's +OOM records, 22:35-23:18 UTC). An int4 one holds about 42-51 GB +(`speed.md`). On lem (125 GB), two int8 jobs on two servers at once +OOM-killed each other seven times: `d9e376286817`, `bc828c7ffe51`, +`9fb01b149f00`, `78440455b4f2`, `6c72b6242412`, `fe9149eb0028` and +`d79a0ff70ccf`. Those arms reran one at a time (`i8e`). One dw.serve +with both cards has a single RAM gate, which waits instead of failing. The two +bench servers each had their own gate. So on a 128 GB two-card box, `best` +runs one job at a time. + +The engine now enforces this (#1227): the H3 templates declare `host_ram_gb` +(51 for draft and standard, 69 for best); admission refuses a run whose +declared need plus margin exceeds the host's total RAM, so a 64 GB host +refuses `best` up front; and each server publishes its running jobs' claims +so a second server on the host holds a job instead of OOM-killing the first. + +## Choices + +- **standard stays today's default** on every template: int4, 960x544, + the template's LoRA and steps. The cost rows stand: `chained-segments` + 13.5 min, `dialogue-short` 21.2, `music-video` 35. The `tier` variable + joins `cost_drivers` (the observed-cost buckets), and the curated cost + counts as measured at the default bundle (`docs/WORKFLOW_GUIDE.md`, + *Presets*). +- **int8 at standard: Don's call, not made here.** On static shots it is the + largest quality gain measured, on both seeds and at two minutes, at 0-5% + more wall time. On moving and dialogue shots it is mixed: seam jump gets + worse. It costs about 40% more host RAM (63-69 GB against 42-51), which + halves lem's H3 throughput and may not fit a 64 GB host at all. It also + re-stages the shot. The recommendation, if Don wants it, is int8 on + `shot: static` and `static_long` only. That needs a rule for a bundle + value both selectors set: `tier` sets `weights_dtype` too, and today the + two bundles share no variable. Until then `tier: best` or + `weights_dtype: "{int8}"` is the knob. +- **draft = 4 steps (FL2VA) / 5 steps (Ref2VA)**, same canvas and weights: + 28-44% off, staging carries. +- **best = 1344x768 int8**: the cleanest static take on both seeds, and the + sharpest frame on every scene (S2 S3 S4). On the arc it is mixed by + measure: look improves and jump worsens, the int8 pattern above. It costs + 1.3-1.7x standard's wall time and runs one at a time on a 128 GB host. + +## Known issue found on the way + +Composing a template with a braced `*_dtype` argument loses the braces: +the wrapper passed `weights_dtype: "{int8}"` to +`templates/minimax/chained-segments`, and the run failed at load with "module +diffusers has no attribute int8" (`1fbbc89cf0eb`) after validation passed. +`EscapedString` (`dw/arguments.py`) does not survive the hop into a child +workflow's variables. The int8 long take ran on the bench copy of S1 instead +(`t20-int8-static-long-120s-s1166`). diff --git a/bench/records/tst.json b/bench/records/tst.json new file mode 100644 index 000000000..3bb783db7 --- /dev/null +++ b/bench/records/tst.json @@ -0,0 +1,197 @@ +[ + { + "arm": "tst-sparse-s1-s1166", + "scene": "s1-static", + "seed": 1166, + "job_id": "a32149365b7c", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-tst/20261105-231704-3d07c0cf/final/BenchS1Static-tst-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "texture_fine_seg_last": 0.0005505260721379298, + "texture_mid_seg_last": 0.00494553480577879, + "texture_coarse_seg_last": 0.002931665659462092, + "wall_minutes": 1.5192577242851257 + } + }, + { + "arm": "tst-sparse-s1-s2", + "scene": "s1-static", + "seed": 2, + "job_id": "dd38d2050097", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-tst/20261105-231759-fd9c74d7/final/BenchS1Static-tst-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "texture_fine_seg_last": 0.00678325687162438, + "texture_mid_seg_last": 0.0007442446618372398, + "texture_coarse_seg_last": 0.0004066818565754152, + "wall_minutes": 1.54106920560201 + } + }, + { + "arm": "tst-sparse-s2-s1166", + "scene": "s2-arc", + "seed": 1166, + "job_id": "7c3a90fc4c90", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc-tst/20261105-231836-d6581ddb/final/BenchS2Arc-tst-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "texture_fine_seg_last": 1.0606200129684038, + "texture_mid_seg_last": 0.8767517694329892, + "texture_coarse_seg_last": 0.21317500245807816, + "wall_minutes": 1.520261243979136 + } + }, + { + "arm": "tst-sparse-s2-s2", + "scene": "s2-arc", + "seed": 2, + "job_id": "6c644ea1d6c2", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc-tst/20261105-231932-2f62cb6b/final/BenchS2Arc-tst-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "texture_fine_seg_last": 1.0194223282188646, + "texture_mid_seg_last": 0.8871962893131665, + "texture_coarse_seg_last": 0.3766151457217526, + "wall_minutes": 1.5938232024510701 + } + }, + { + "arm": "tst-dense-s1-s1166", + "scene": "s1-static", + "seed": 1166, + "job_id": "db14f86eb1bd", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-tst/20261105-232009-1576a96a/final/BenchS1Static-tst-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "texture_fine_seg_last": 0.007657569042856663, + "texture_mid_seg_last": 0.0011109693832020246, + "texture_coarse_seg_last": 0.0021659460881937793, + "wall_minutes": 4.267374646663666 + } + }, + { + "arm": "tst-dense-s1-s2", + "scene": "s1-static", + "seed": 2, + "job_id": "1a172450268f", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-tst/20261105-232110-4eaebcef/final/BenchS1Static-tst-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "texture_fine_seg_last": 0.016368605650056264, + "texture_mid_seg_last": 0.030277207109399844, + "texture_coarse_seg_last": 0.0031824023221582193, + "wall_minutes": 4.130329004923502 + } + }, + { + "arm": "tst-dense-s2-s1166", + "scene": "s2-arc", + "seed": 1166, + "job_id": "d0e2aad67720", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc-tst/20261105-232423-17032543/final/BenchS2Arc-tst-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "texture_fine_seg_last": 0.5194894754618639, + "texture_mid_seg_last": 0.425894791291317, + "texture_coarse_seg_last": 0.05401100500242832, + "wall_minutes": 1.7693364063898722 + } + }, + { + "arm": "tst-dense-s2-s2", + "scene": "s2-arc", + "seed": 2, + "job_id": "19e189043889", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc-tst/20261105-232515-d7d79b61/final/BenchS2Arc-tst-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "texture_fine_seg_last": 0.4082384659125036, + "texture_mid_seg_last": 0.05639949268578385, + "texture_coarse_seg_last": 0.12389349000682051, + "wall_minutes": 1.7509605169296265 + } + }, + { + "arm": "tst-tst20-s1-s1166", + "scene": "s1-static", + "seed": 1166, + "job_id": "6dba38159cc7", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-tst/20261105-232609-5862027e/final/BenchS1Static-tst-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "texture_fine_seg_last": 0.01319710731167332, + "texture_mid_seg_last": 0.007286044398674605, + "texture_coarse_seg_last": 0.0003062378919516865, + "wall_minutes": 1.8292582035064697 + } + }, + { + "arm": "tst-tst20-s1-s2", + "scene": "s1-static", + "seed": 2, + "job_id": "87b8dd26a67a", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-tst/20261105-232700-ab83dbf7/final/BenchS1Static-tst-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "texture_fine_seg_last": 0.006473288613714353, + "texture_mid_seg_last": 0.0023425219840875733, + "texture_coarse_seg_last": 0.004329591452834346, + "wall_minutes": 1.8794138431549072 + } + }, + { + "arm": "tst-tst20-s2-s1166", + "scene": "s2-arc", + "seed": 1166, + "job_id": "80ed710df33f", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc-tst/20261105-232759-88a96d07/final/BenchS2Arc-tst-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "texture_fine_seg_last": 0.4733520368484713, + "texture_mid_seg_last": 0.38220589714489667, + "texture_coarse_seg_last": 0.033504605094703536, + "wall_minutes": 1.8417776425679524 + } + }, + { + "arm": "tst-tst20-s2-s2", + "scene": "s2-arc", + "seed": 2, + "job_id": "f7a673dae443", + "status": "succeeded", + "error": null, + "output": "BenchS2Arc-tst/20261105-232853-5699e89f/final/BenchS2Arc-tst-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "texture_fine_seg_last": 0.5370948296746267, + "texture_mid_seg_last": 0.04595614208118365, + "texture_coarse_seg_last": 0.03909748789002631, + "wall_minutes": 1.8133281071980794 + } + }, + { + "arm": "tst-tau0-s1-s1166", + "scene": "s1-static", + "seed": 1166, + "job_id": "20db977af359", + "status": "succeeded", + "error": null, + "output": "BenchS1Static-tst/20261105-232949-231a8a56/final/BenchS1Static-tst-chained_keyframe_to_video_audio.0-0.0.mp4", + "measures": { + "texture_fine_seg_last": 0.007657569042856663, + "texture_mid_seg_last": 0.0011109693832020246, + "texture_coarse_seg_last": 0.0021659460881937793, + "wall_minutes": 1.854285232226054 + } + } +] diff --git a/bench/records/words.md b/bench/records/words.md new file mode 100644 index 000000000..f1eeeb4a3 --- /dev/null +++ b/bench/records/words.md @@ -0,0 +1,69 @@ +# Words check: similarity spread and Whisper CPU cost (#1262 stage A, #1297) + +Measured 2026-11-08. Does a dialogue segment's similarity to its line (the +`check_script` measure that `chain_check.check_words` uses on one line) +separate the takes Don kept from the ones he re-rendered? And what does one +Whisper pass on CPU cost? This used no new GPU time. The similarities come +from the existing `templates/check-script` outputs in workspace +`constitution-walk-2` (openai/whisper-large-v3-turbo, threshold 0.8), +re-scored offline with the #1297 normalisation (number words folded to +digits). + +## Labels + +A take is **bad** when Don re-rendered it for its words, as the job notes +record: am5c ("seg3 … Nor"), am6a ("orig last segment misspoke") and am9 +("conscrued + dropped to"). A take is **good** when it was kept. am7a seg3 +dropped its head word "and", and nobody noted it; it is marked bad? because, +by the script's words, it is. + +## Spread + +| Label | Take, line | Heard (the difference) | Before folding | With folding | +|---|---|---|---|---| +| good | about 25 lines (am1, am2, am5a, am5b, am5c r4/r5, am6a re-render, am6b, am7a, am7b, am10, …) | exact | 1.0 | 1.0 | +| good | am6a re-render seg1 | `Amendment 6` | 0.9000 | 1.0000 | +| good | am7b seg2 | `Amendment 8` | 0.9167 | 1.0000 | +| good | am10 seg1 | `Amendment 10` | 0.9231 | 1.0000 | +| good | am7a seg1 | `Amendment 7`, `$20` for "twenty dollars" | 0.8387 | 0.9677 | +| good | am6b seg4 | `defense` for "defence" | 0.9000 | 0.9000 | +| good | am5b seg2 | `offense` for "offence" | 0.9000 | 0.9000 | +| good? | am9 seg1 | `Amendment 9`, `conscrued` (a Whisper mishearing) | 0.8571 | 0.9286 | +| bad | am5c seg3, five renders | "Nor" dropped from the line's head | 0.9565 | 0.9565 | +| bad? | am7a seg3 | "and" dropped from the line's head | 0.9565 | 0.9565 | +| bad | am9 seg2 | "to" dropped from the line's head | 0.9412 | 0.9412 | +| bad | am6a seg4 | `asserturned` (misspoke "ascertained") | 0.8889 | 0.8889 | + +- **Good:** 0.90 to 1.0 with folding. The lowest are spelling variants + (British "defence"/"offence" heard as American), not wrong words. +- **Bad:** 0.889 to 0.957. A dropped word costs a long line 1 - 2/(2n+1), + where n is the line's word count: about 0.04 for a 12-word line. + +## Verdict: no clean separation + +The ranges overlap. Three bad takes (0.9412 and 0.9565) score above two good +ones (0.90), so no single minimum passes every good take and fails every bad +one. What the bad takes share is a dropped word at a segment's **head**, the +seam the chain cuts. Similarity can't see that; an alignment can. **Gate +fired:** stage B ships `words_similarity` with a **null default**, so a chain +doesn't re-roll on similarity unless the caller sets a minimum. + +**Ceiling:** `WORDS_SIMILARITY_MIN = 0.80`. That's the most a minimum can be +and still pass every good take seen, with margin under the lowest (0.90). It +catches only gross failures: a wrong or missing line, or a garbled segment. + +## Whisper CPU cost + +On lem, 16 torch threads (`CUDA_VISIBLE_DEVICES=` empty), whisper-large-v3-turbo +through `transcribe_audio(device="cpu", timestamps="word")`, on 8.0 s of a +constitution-walk-2 segment (am3, "Amendment 3. No soldier shall in time of +peace be quartered in any"): + +| Pass | Seconds | +|---|---| +| first (pipeline built in the process, weights on disk) | 3.00 | +| second | 1.28 | +| third | 1.27 | + +That's about 1.3 s for an 8 s segment once the model is loaded. **Gate not +fired:** that's well under the 20 s re-plan gate, so stage B builds as planned. diff --git a/bench/refine_pairs.py b/bench/refine_pairs.py new file mode 100644 index 000000000..f9bf9ee2c --- /dev/null +++ b/bench/refine_pairs.py @@ -0,0 +1,202 @@ +"""Measure a refined take against its unrefined source (Task 17 Step 4). + +A refine's output is decoded fresh, so its run records no shots and the +server's assess route finds no seams in it. This reads both files in-process +(the code behind the compare and assess routes, as round 3's moments pass +did) and measures the refined take at the source's own seams: + +- `analyze_seams` at every seam the source's run recorded: jump, look and + level step, and `typical_delta` (the within-shot frame delta beside each + seam, the bench's motion measure); +- texture drift against the first moment, `|ln(band / band@first)|`, on the + scene crop at every moment (and a face crop where the scene has one); +- motion: the mean absolute frame-to-frame delta of the grey thumbnails, + over the whole take; +- the refined take against its source at each moment: PSNR and SSIM. + +Run on lem from the bench checkout, with the deployed dw on the path: + + PYTHONPATH=~/dw-bench ~/diffusers-workflow/venv/bin/python \\ + bench/refine_pairs.py ~/bench-workspace/outputs pairs.json --out out.json + +`pairs.json` is a list of {arm, scene, source, refined, moments}: output +names relative to the outputs root, and moments as "frame:N". +""" + +import argparse +import json +import math +import sys +from pathlib import Path + +# Face crops (x, y, w, h) for the scenes with a face to watch +FACE_CROPS = {"s1-static": [360, 30, 240, 240]} +SCENE_CROPS = { + "s1-static": [0, 0, 240, 150], + "s2-arc": [0, 0, 240, 150], + "s3-dialogue-walk": [0, 0, 240, 150], +} + + +def _mean(values): + values = [v for v in values if v is not None] + return sum(values) / len(values) if values else None + + +def seam_measures(path, shots): + from dw.server.assess import assess + + seams = assess(path, "video", shots, probe="analyze_seams").get("seams") or [] + out = {"seams": [], "measures": {}} + for seam in seams: + out["seams"].append( + { + key: seam.get(key) + for key in ( + "seconds", + "jump_ratio", + "look_ratio", + "level_step_db", + "typical_delta", + ) + } + ) + m = out["measures"] + for key, name in (("jump_ratio", "seam_jump"), ("look_ratio", "seam_look")): + values = [s.get(key) for s in seams if s.get(key) is not None] + if values: + m[f"{name}_mean"], m[f"{name}_max"] = _mean(values), max(values) + levels = [ + abs(s["level_step_db"]) for s in seams if s.get("level_step_db") is not None + ] + if levels: + m["seam_level_step_db_mean"], m["seam_level_step_db_max"] = ( + _mean(levels), + max(levels), + ) + deltas = [s.get("typical_delta") for s in seams if s.get("typical_delta")] + if deltas: + m["typical_delta_mean"] = _mean(deltas) + return out + + +def motion(path): + import numpy + + from dw.tasks.assess import read_media + + media = read_media(path) + thumbs = media.thumbs.astype(numpy.float32) + deltas = numpy.abs(numpy.diff(thumbs, axis=0)).mean(axis=(1, 2)) + return float(deltas.mean()), [float(d) for d in deltas] + + +def _source(name, path): + from dw.media_readers import video_shape + + return {"name": name, "kind": "video", "path": path, "shape": video_shape(path)} + + +def texture_series(path, name, moments, crop): + from dw import media_compare + + answer = media_compare.compare( + [_source(name, path)], moments, crop, 0, "shared", 384, texture_bands=True + ) + cells = answer["texture"][0]["cells"] + first = (cells[0].get("texture") or {}).get("luma") or {} + series = {} + for band in ("fine", "mid", "coarse"): + row = [] + for cell in cells: + luma = (cell.get("texture") or {}).get("luma") or {} + a, b = first.get(band), luma.get(band) + row.append(abs(math.log(b / a)) if a and b and a > 0 and b > 0 else None) + series[band] = row + return series + + +def fidelity(source_path, refined_path, moments): + from dw import media_compare + + answer = media_compare.compare( + [_source("source", source_path), _source("refined", refined_path)], + moments, + None, + 0, + None, + 384, + ) + cells = answer["metrics"][0]["cells"] + return { + "psnr_db": [c.get("psnr_db") for c in cells], + "ssim": [c.get("ssim") for c in cells], + } + + +def measure_take(path, name, shots, scene, moments): + seams = seam_measures(path, shots) + m = dict(seams["measures"]) + series = texture_series(path, name, moments, SCENE_CROPS[scene]) + for band, row in series.items(): + if row and row[-1] is not None: + m[f"texture_{band}_seg_last"] = row[-1] + face = None + if scene in FACE_CROPS: + face = texture_series(path, name, moments, FACE_CROPS[scene]) + if face["fine"][-1] is not None: + m["face_fine_seg_last"] = face["fine"][-1] + motion_mean, _ = motion(path) + return { + "measures": m, + "seams": seams["seams"], + "texture": series, + "face_texture": face, + "motion_mean": motion_mean, + } + + +def measure_pair(root, pair): + from dw.runs import recorded_shots + + shots = recorded_shots(str(root), pair["source"]) + if not shots: + raise ValueError(f"{pair['source']} recorded no shots") + source_path = str(Path(root) / pair["source"]) + refined_path = str(Path(root) / pair["refined"]) + moments = pair["moments"] + return { + "arm": pair["arm"], + "scene": pair["scene"], + "moments": moments, + "source": measure_take( + source_path, pair["source"], shots, pair["scene"], moments + ), + "refined": measure_take( + refined_path, pair["refined"], shots, pair["scene"], moments + ), + "fidelity": fidelity(source_path, refined_path, moments), + } + + +def main(argv=None): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("outputs", help="the bench workspace's outputs root") + parser.add_argument("pairs") + parser.add_argument("--out", required=True) + args = parser.parse_args(argv) + root = Path(args.outputs).expanduser() + results = [] + for pair in json.loads(Path(args.pairs).read_text()): + try: + results.append(measure_pair(root, pair)) + except Exception as exc: # one pair's failure must not lose the others + results.append( + {"arm": pair["arm"], "error": f"{type(exc).__name__}: {exc}"} + ) + Path(args.out).write_text(json.dumps(results, indent=2)) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/bench/scenes.json b/bench/scenes.json new file mode 100644 index 000000000..88a46beca --- /dev/null +++ b/bench/scenes.json @@ -0,0 +1,107 @@ +[ + { + "name": "s1-static", + "crop": [ + 0, + 0, + 240, + 150 + ], + "moments": [ + "frame:62", + "frame:481", + "frame:990" + ], + "face_reference": "asset:bench/static-stripes.jpg", + "crop_verified": true + }, + { + "name": "s2-arc", + "crop": [ + 0, + 0, + 240, + 150 + ], + "moments": [ + "frame:62", + "frame:481", + "frame:990" + ], + "crop_verified": true + }, + { + "name": "s3-dialogue-walk", + "crop": [ + 0, + 0, + 240, + 150 + ], + "moments": [ + "frame:62", + "frame:277", + "frame:583" + ], + "face_reference": "asset:bench/walker.png", + "lines": [ + "I told myself I would not come back here.", + "But the street looks exactly the way I left it.", + "Same lamps, same crooked stones, same bakery smell.", + "Funny how a place waits for you without asking.", + "Maybe that is what I was afraid of.", + "Well. Let's see if the door still sticks." + ], + "crop_verified": true + }, + { + "name": "s4-montage", + "crop": [ + 500, + 16, + 160, + 130 + ], + "moments": [ + "frame:62", + "frame:186", + "frame:434" + ], + "crop_verified": true + }, + { + "name": "s5-ltx-action", + "crop": [ + 0, + 0, + 240, + 150 + ], + "moments": [ + "frame:60", + "frame:180", + "frame:299" + ], + "crop_verified": true + }, + { + "name": "s6-narration", + "crop": [ + 690, + 340, + 190, + 135 + ], + "moments": [ + "frame:48", + "frame:144", + "frame:240" + ], + "lines": [ + "The keeper climbed the stairs every evening, long before the storm.", + "By the time he reached the lamp, the sea had already gone dark.", + "Then the light turned, and the whole coast could see him." + ], + "crop_verified": true + } +] diff --git a/bench/segment1.py b/bench/segment1.py new file mode 100644 index 000000000..a64f7a812 --- /dev/null +++ b/bench/segment1.py @@ -0,0 +1,189 @@ +"""Segment-1 drift labels (ruling R35). + +No counter the bench runs touches segment 1 of a chain, so every arm of one +scene at one seed should open on the same segment 1. A cold-loaded worker +gives that canonical segment 1; a worker that has run some jobs can move it +into an alternate output (`nondeterminism-report.md`). This hashes segment 1 +of a record's output and labels the record `clean` (the cold-load canonical +fingerprint for its scene and seed) or `drifted`. + +The fingerprint is the md5 of the decoded-frame framemd5 hashes of video +frames 0-59 (2.5 s) and of AAC frames 0-129 (4.16 s at 32 kHz), written +`vvvvvvvv/aaaaaaaa`. Both sit inside segment 1 and before any seam, including +a `guide_backoff 17` arm's shorter segment 1 (107 kept frames, 4.46 s; at 140 +AAC frames the window would cross that seam). Decoded H.264 is bit-exact +across ffmpeg builds, but decoded AAC is not (lem's ffmpeg 7.0.2 and a Mac +build give different audio halves for the same file), so take every +fingerprint of one table, canonical entries included, with one ffmpeg. + +An arm whose arguments name a LoRA that runs on every segment +(`antiburn_model_name`) changes segment 1 by design; its key carries +`+lora`, and it is labelled against its own canonical entry when there is one. + +Usage (on the host that holds the outputs, stdlib only): + + python3 segment1.py --root ~/bench-workspace/outputs RECORD.json ... + python3 segment1.py --fingerprints fp.json --canonical C.json \ + --arms ARMS.json ... RECORD.json ... +""" + +import argparse +import hashlib +import json +import subprocess +import sys +from pathlib import Path + +VIDEO_FRAMES = 60 +AUDIO_FRAMES = 130 +SEGMENT1_ARGUMENTS = ("antiburn_model_name",) + + +def framemd5_hashes(text): + """The per-frame hashes of one stream's framemd5 output, in order.""" + return [ + line.split(",")[-1].strip() + for line in text.splitlines() + if line.strip() and not line.startswith("#") + ] + + +def _digest(hashes, count, what): + if len(hashes) < count: + raise ValueError(f"{what} has {len(hashes)} frames, segment 1 needs {count}") + return hashlib.md5("".join(hashes[:count]).encode()).hexdigest()[:8] + + +def fingerprint_from_framemd5( + video_text, audio_text, video_frames=VIDEO_FRAMES, audio_frames=AUDIO_FRAMES +): + """`vvvvvvvv/aaaaaaaa` from the video and audio framemd5 outputs.""" + return "/".join( + ( + _digest(framemd5_hashes(video_text), video_frames, "video"), + _digest(framemd5_hashes(audio_text), audio_frames, "audio"), + ) + ) + + +def _framemd5(path, ffmpeg, stream, frames): + # One stream per call: ffmpeg ends every output stream when the first + # one reaches its -frames cap. + command = [ + ffmpeg, + "-v", + "error", + "-i", + str(path), + "-map", + f"0:{stream}:0", + f"-frames:{stream}", + str(frames), + "-f", + "framemd5", + "-", + ] + return subprocess.run(command, capture_output=True, text=True, check=True).stdout + + +def fingerprint_file(path, ffmpeg="ffmpeg"): + return fingerprint_from_framemd5( + _framemd5(path, ffmpeg, "v", VIDEO_FRAMES), + _framemd5(path, ffmpeg, "a", AUDIO_FRAMES), + ) + + +def segment1_key(scene, seed, arguments=None): + key = f"{scene}@{seed}" + if any((arguments or {}).get(name) for name in SEGMENT1_ARGUMENTS): + key += "+lora" + return key + + +def label(fingerprint, canonical): + """`clean`, `drifted`, or `no canonical` against one canonical entry.""" + if not canonical: + return "no canonical" + return "clean" if fingerprint == canonical["fingerprint"] else "drifted" + + +def classify(records, fingerprints, canonical, arguments_by_arm=None): + """One row per succeeded record: arm, job, key, fingerprint, label.""" + arguments_by_arm = arguments_by_arm or {} + rows = [] + for record in records: + if record.get("status", "succeeded") != "succeeded": + continue + key = segment1_key( + record["scene"], record["seed"], arguments_by_arm.get(record["arm"]) + ) + fingerprint = fingerprints.get(record["output"]) + rows.append( + { + "arm": record["arm"], + "job_id": record.get("job_id"), + "key": key, + "fingerprint": fingerprint, + "label": ( + label(fingerprint, canonical.get(key)) + if fingerprint + else "no output" + ), + } + ) + return rows + + +def _records(paths): + records = [] + for path in paths: + records.extend(json.loads(Path(path).read_text())) + return records + + +def main(argv=None): + parser = argparse.ArgumentParser(description=__doc__.split("\n\n")[0]) + parser.add_argument("records", nargs="+", help="bench record files") + parser.add_argument("--root", help="outputs folder; fingerprints each output") + parser.add_argument("--ffmpeg", default="ffmpeg") + parser.add_argument("--fingerprints", help="JSON {output: fingerprint}") + parser.add_argument("--canonical", help="JSON {scene@seed: {fingerprint}}") + parser.add_argument("--arms", nargs="*", default=[], help="arms files") + args = parser.parse_args(argv) + records = _records(args.records) + if args.root: + fingerprints = {} + for record in records: + if record.get("status", "succeeded") != "succeeded": + continue + try: + fingerprints[record["output"]] = fingerprint_file( + Path(args.root).expanduser() / record["output"], args.ffmpeg + ) + except (OSError, ValueError, subprocess.CalledProcessError) as exc: + print(f"{record['arm']}: {exc}", file=sys.stderr) + if not args.canonical: + print(json.dumps(fingerprints, indent=1)) + return 0 + elif args.fingerprints: + fingerprints = json.loads(Path(args.fingerprints).read_text()) + else: + parser.error("one of --root or --fingerprints is needed") + canonical = json.loads(Path(args.canonical).read_text()) if args.canonical else {} + arguments = { + arm["name"]: arm.get("arguments") or {} + for path in args.arms + for arm in json.loads(Path(path).read_text()) + } + print("| arm | job | key | segment 1 | label |") + print("|---|---|---|---|---|") + for row in classify(records, fingerprints, canonical, arguments): + print( + f"| {row['arm']} | `{row['job_id']}` | {row['key']} | " + f"`{row['fingerprint']}` | {row['label']} |" + ) + return 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/bench/verdict.py b/bench/verdict.py new file mode 100644 index 000000000..cd4f3b9a2 --- /dev/null +++ b/bench/verdict.py @@ -0,0 +1,59 @@ +"""The bench's verdict rule. Every measure is lower-is-better (the driver +negates the ones where higher is better before it gets here). + +The floor a verdict reads is the larger, per measure, of the seed floor +(`noise_floor` of a control at two seeds) and the same-seed rerun gap +(`rerun_gap` over pairs of runs with the same arguments and seed): ruling +R35, after same-seed reruns on the bench server were found not to +reproduce (`bench/records/noise-floor.md`).""" + + +def noise_floor(a, b): + """Per-measure absolute difference of two runs.""" + return {key: abs(a[key] - b[key]) for key in a if key in b} + + +def rerun_gap(pairs): + """Per-measure largest `noise_floor` over same-seed, same-arguments + pairs of runs.""" + gap = {} + for a, b in pairs: + for key, value in noise_floor(a, b).items(): + gap[key] = max(gap.get(key, 0.0), value) + return gap + + +def combined_floor(*floors): + """Per-measure maximum of several floors (seed floor, rerun gap, or the + floors of two continuities a cross-continuity read spans).""" + out = {} + for floor in floors: + for key, value in floor.items(): + out[key] = max(out.get(key, 0.0), value) + return out + + +def verdict(control, arm, noise, primary): + """`wins` when the primary beats control by more than its noise and no + other measure is worse by more than its noise; `loses` when the primary + is worse by more than noise; `mixed` when the primary wins but another + measure is worse past noise; `tie` otherwise.""" + missing = [ + name for name, run in (("control", control), ("arm", arm)) if primary not in run + ] + if missing: + raise ValueError( + f"primary measure {primary!r} missing from {' and '.join(missing)}" + ) + gain = control[primary] - arm[primary] + floor = noise.get(primary, 0.0) + worse_elsewhere = any( + arm[key] - control[key] > noise.get(key, 0.0) + for key in control + if key != primary and key in arm + ) + if gain > floor: + return "mixed" if worse_elsewhere else "wins" + if -gain > floor: + return "loses" + return "mixed" if worse_elsewhere else "tie" diff --git a/bench/workflows/add-sound-span.json b/bench/workflows/add-sound-span.json new file mode 100644 index 000000000..2e544dc54 --- /dev/null +++ b/bench/workflows/add-sound-span.json @@ -0,0 +1,230 @@ +{ + "id": "MiniMaxH3AddSoundSpan", + "description": "DRAFT bench workflow (#1192, plan Task 21 Step 6): bench/workflows/add-sound.json with a trim_video step in front, so the audio pass can run over one span of a longer take - one segment of a chain, whose whole take is past H3's 345-frame ceiling. 'source_video' is the take, 'start_frame' the span's first frame and 'num_frames' its length on the 17n + 5 grid; the 'sound' step is add-sound's audio pass over the trimmed span (its picture, its rate). Everything else is add-sound's: no distillation LoRA, 12 steps, shifts 12/3, 'listen_scale' 1, 0.5 or 0.25. Not in the catalog.", + "summary": "A new H3 soundtrack over one span of a finished take's own picture.", + "cost_drivers": [ + "num_frames", + "num_inference_steps", + "width", + "height", + "listen_scale", + "weights_dtype", + "lora_model_name", + "attention_backend", + "cache" + ], + "vram_estimate": { + "base_gb": 16.3, + "bytes_per_voxel": 28.71, + "bytes_per_guide_voxel": 28.71, + "voxel_variables": [ + "width", + "height", + "num_frames" + ], + "reason": "VAE decode's conv3d activation scales with the voxel count width * height * num_frames; base_gb and bytes_per_voxel are calibrated from two real runs: image-to-video's family baseline at 960x544x124 (on-demand VAE) succeeded at 18.03 GiB peak reserved (docs/WORKFLOW_GUIDE.md), and video-with-audio-768p at 1344x768x345 (the family's own num_frames ceiling) OOM'd mid-denoise needing >=25.82 GiB - 20.25 GiB already allocated plus a refused 5.57 GiB allocation, on a 23.56 GiB card (job 282172105da1, 2026-09-22) - closing the gap #265/#266 left the H3 family with nothing declared to check the engine's vram_estimate machinery against (#324)" + }, + "variables": { + "source_video": "asset:take.mp4", + "start_frame": 0, + "prompt": "prompt:minimax/fox_dawn_context_ir", + "num_inference_steps": 12, + "num_frames": 124, + "width": 960, + "height": 544, + "video_shift": 12.0, + "audio_shift": 3.0, + "weights_dtype": "{int4}", + "lora_scale": 1.0, + "lora_alpha": null, + "lora_model_name": null, + "lora_weight_name": "minimax_h3_fl2v_turbo_4step_v1.2_768p_bf16.safetensors", + "lora_adapter_name": "turbo", + "seed": 42, + "attention_backend": "sparse", + "sparse_tau": 0.5, + "sparse_topk": 0.15, + "cache": null, + "listen_scale": 1.0 + }, + "variable_constraints": { + "num_frames": { + "modulus": 17, + "remainder": 5, + "min_frames": 124, + "max_frames": 345, + "snap": "up", + "reason": "the video VAE encodes 17 * n + 5 frames, and MiniMax-H3 generates between 5 and 15 seconds at 24 fps" + }, + "width": { + "modulus": 32, + "remainder": 0, + "reason": "MiniMax-H3's pipeline refuses a `width` or `height` that is not a multiple of 32, and only after the weights have loaded; refused here instead, with no 'snap', since rounding would be a silent change to the size asked for" + }, + "height": { + "modulus": 32, + "remainder": 0, + "reason": "MiniMax-H3's pipeline refuses a `width` or `height` that is not a multiple of 32, and only after the weights have loaded; refused here instead, with no 'snap', since rounding would be a silent change to the size asked for" + } + }, + "seed": "variable:seed", + "steps": [ + { + "name": "span", + "task": { + "command": "trim_video", + "arguments": { + "video": "variable:source_video", + "start_frame": "variable:start_frame", + "num_frames": "variable:num_frames" + } + }, + "result": { + "content_type": "video/mp4", + "fps": 24, + "subfolder": "intermediate" + } + }, + { + "name": "sound", + "pipeline": { + "configuration": { + "component_type": "ModularPipeline", + "pre_load_modules": [ + "sdnq" + ], + "load_components": { + "dtype": "torch.bfloat16", + "quantization_config": { + "transformer": { + "configuration": { + "config_type": "sdnq.SDNQConfig" + }, + "arguments": { + "weights_dtype": "variable:weights_dtype", + "quantization_device": "cuda", + "return_device": "cpu", + "use_quantized_matmul": true, + "dequantize_fp32": false, + "modules_to_not_convert": [ + "proj_in", + "audio_proj_in", + "context_embedder", + "time_embedder", + "time_proj", + "token_refiner", + "norm_out", + "proj_out", + "audio_proj_out" + ] + } + }, + "text_encoder": { + "configuration": { + "config_type": "sdnq.SDNQConfig" + }, + "arguments": { + "weights_dtype": "variable:weights_dtype", + "quantization_device": "cuda", + "return_device": "cpu", + "dequantize_fp32": false, + "modules_to_not_convert": [ + ".model.visual", + "lm_head" + ] + } + }, + "vae": { + "configuration": { + "config_type": "sdnq.SDNQConfig" + }, + "arguments": { + "weights_dtype": "{int8}", + "quant_conv": true, + "use_quantized_matmul_conv": true, + "quantization_device": "cuda", + "return_device": "cpu", + "dequantize_fp32": false + } + } + } + }, + "components": { + "transformer": { + "attention_backend": "variable:attention_backend", + "sparse_tau": "variable:sparse_tau", + "sparse_topk": "variable:sparse_topk", + "group_offload": { + "offload_type": "block_level", + "num_blocks_per_group": 2, + "use_stream": true, + "record_stream": true, + "low_cpu_mem_usage": true + } + }, + "text_encoder": { + "remove_modules": [ + "lm_head" + ] + }, + "text_encoder.model": { + "truncate_layers": { + "language_model.layers": 51 + }, + "group_offload": { + "offload_type": "leaf_level" + } + }, + "vae": { + "device": "cuda", + "residency": "on_demand" + }, + "audio_vae": { + "device": "cuda", + "residency": "on_demand" + } + }, + "cache": "variable:cache" + }, + "from_pretrained_arguments": { + "model_name": "MiniMaxAI/MiniMax-H3", + "workflow": "t2va" + }, + "loras": [ + { + "model_name": "variable:lora_model_name", + "weight_name": "variable:lora_weight_name", + "adapter_name": "variable:lora_adapter_name", + "scale": "variable:lora_scale", + "alpha": "variable:lora_alpha" + } + ], + "scheduler": { + "shift": "variable:video_shift" + }, + "audio_scheduler": { + "shift": "variable:audio_shift" + }, + "arguments": { + "prompt": "variable:prompt", + "source_video": "previous_result:span", + "listen_scale": "variable:listen_scale", + "num_frames": "variable:num_frames", + "width": "variable:width", + "height": "variable:height", + "num_inference_steps": "variable:num_inference_steps", + "output": [ + "videos", + "audio", + "sampling_rate" + ] + } + }, + "result": { + "content_type": "video/mp4", + "fps": 24, + "subfolder": "final" + } + } + ] +} diff --git a/bench/workflows/add-sound.json b/bench/workflows/add-sound.json new file mode 100644 index 000000000..c5a219b07 --- /dev/null +++ b/bench/workflows/add-sound.json @@ -0,0 +1,212 @@ +{ + "id": "MiniMaxH3AddSound", + "description": "DRAFT, not in the catalog (#1192): H3 audio over a finished picture. 'source_video' is the finished take (an 'asset:' or 'output:' video, a silent one is the usual case) and 'prompt' the take's own prompt with the sound written in: its overall_soundscape, its music and its dialogue lines. The one 'sound' step is the baseline t2va pipeline given the take as 'source_video' with no mask and no hold_audio, which makes it an audio pass: every video row is the take's own clean latent at every step and only the audio is generated, from full noise, attending to the real picture. The result plays the take's own frames, not a VAE round trip, under the new track. It runs with no distillation LoRA on the dialogue schedule (12 steps, shifts 12/3), since the turbo LoRA is what digitizes H3 voices (#984 B0). 'listen_scale' 0.5 or 0.25 encodes the take at that fraction of its canvas, snapped to 32 px, which is cheaper; the track still runs the take's length and the picture returned is full size. 'width', 'height' and 'num_frames' must be the take's, on the 17n + 5 grid. Kept under bench/workflows until Don has listened to it.", + "summary": "A new H3 soundtrack - sound, music, dialogue - over a finished take's own picture.", + "cost_drivers": [ + "num_frames", + "num_inference_steps", + "width", + "height", + "listen_scale", + "weights_dtype", + "lora_model_name", + "attention_backend", + "cache" + ], + "vram_estimate": { + "base_gb": 16.3, + "bytes_per_voxel": 28.71, + "bytes_per_guide_voxel": 28.71, + "voxel_variables": [ + "width", + "height", + "num_frames" + ], + "reason": "VAE decode's conv3d activation scales with the voxel count width * height * num_frames; base_gb and bytes_per_voxel are calibrated from two real runs: image-to-video's family baseline at 960x544x124 (on-demand VAE) succeeded at 18.03 GiB peak reserved (docs/WORKFLOW_GUIDE.md), and video-with-audio-768p at 1344x768x345 (the family's own num_frames ceiling) OOM'd mid-denoise needing >=25.82 GiB - 20.25 GiB already allocated plus a refused 5.57 GiB allocation, on a 23.56 GiB card (job 282172105da1, 2026-09-22) - closing the gap #265/#266 left the H3 family with nothing declared to check the engine's vram_estimate machinery against (#324)" + }, + "variables": { + "source_video": "asset:take.mp4", + "prompt": "prompt:minimax/fox_dawn_context_ir", + "num_inference_steps": 12, + "num_frames": 124, + "width": 960, + "height": 544, + "video_shift": 12.0, + "audio_shift": 3.0, + "weights_dtype": "{int4}", + "lora_scale": 1.0, + "lora_alpha": null, + "lora_model_name": null, + "lora_weight_name": "minimax_h3_fl2v_turbo_4step_v1.2_768p_bf16.safetensors", + "lora_adapter_name": "turbo", + "seed": 42, + "attention_backend": "sparse", + "sparse_tau": 0.5, + "sparse_topk": 0.15, + "cache": null, + "listen_scale": 1.0 + }, + "variable_constraints": { + "num_frames": { + "modulus": 17, + "remainder": 5, + "min_frames": 124, + "max_frames": 345, + "snap": "up", + "reason": "the video VAE encodes 17 * n + 5 frames, and MiniMax-H3 generates between 5 and 15 seconds at 24 fps" + }, + "width": { + "modulus": 32, + "remainder": 0, + "reason": "MiniMax-H3's pipeline refuses a `width` or `height` that is not a multiple of 32, and only after the weights have loaded; refused here instead, with no 'snap', since rounding would be a silent change to the size asked for" + }, + "height": { + "modulus": 32, + "remainder": 0, + "reason": "MiniMax-H3's pipeline refuses a `width` or `height` that is not a multiple of 32, and only after the weights have loaded; refused here instead, with no 'snap', since rounding would be a silent change to the size asked for" + } + }, + "seed": "variable:seed", + "steps": [ + { + "name": "sound", + "pipeline": { + "configuration": { + "component_type": "ModularPipeline", + "pre_load_modules": [ + "sdnq" + ], + "load_components": { + "dtype": "torch.bfloat16", + "quantization_config": { + "transformer": { + "configuration": { + "config_type": "sdnq.SDNQConfig" + }, + "arguments": { + "weights_dtype": "variable:weights_dtype", + "quantization_device": "cuda", + "return_device": "cpu", + "use_quantized_matmul": true, + "dequantize_fp32": false, + "modules_to_not_convert": [ + "proj_in", + "audio_proj_in", + "context_embedder", + "time_embedder", + "time_proj", + "token_refiner", + "norm_out", + "proj_out", + "audio_proj_out" + ] + } + }, + "text_encoder": { + "configuration": { + "config_type": "sdnq.SDNQConfig" + }, + "arguments": { + "weights_dtype": "variable:weights_dtype", + "quantization_device": "cuda", + "return_device": "cpu", + "dequantize_fp32": false, + "modules_to_not_convert": [ + ".model.visual", + "lm_head" + ] + } + }, + "vae": { + "configuration": { + "config_type": "sdnq.SDNQConfig" + }, + "arguments": { + "weights_dtype": "{int8}", + "quant_conv": true, + "use_quantized_matmul_conv": true, + "quantization_device": "cuda", + "return_device": "cpu", + "dequantize_fp32": false + } + } + } + }, + "components": { + "transformer": { + "attention_backend": "variable:attention_backend", + "sparse_tau": "variable:sparse_tau", + "sparse_topk": "variable:sparse_topk", + "group_offload": { + "offload_type": "block_level", + "num_blocks_per_group": 2, + "use_stream": true, + "record_stream": true, + "low_cpu_mem_usage": true + } + }, + "text_encoder": { + "remove_modules": [ + "lm_head" + ] + }, + "text_encoder.model": { + "truncate_layers": { + "language_model.layers": 51 + }, + "group_offload": { + "offload_type": "leaf_level" + } + }, + "vae": { + "device": "cuda", + "residency": "on_demand" + }, + "audio_vae": { + "device": "cuda", + "residency": "on_demand" + } + }, + "cache": "variable:cache" + }, + "from_pretrained_arguments": { + "model_name": "MiniMaxAI/MiniMax-H3", + "workflow": "t2va" + }, + "loras": [ + { + "model_name": "variable:lora_model_name", + "weight_name": "variable:lora_weight_name", + "adapter_name": "variable:lora_adapter_name", + "scale": "variable:lora_scale", + "alpha": "variable:lora_alpha" + } + ], + "scheduler": { + "shift": "variable:video_shift" + }, + "audio_scheduler": { + "shift": "variable:audio_shift" + }, + "arguments": { + "prompt": "variable:prompt", + "source_video": "variable:source_video", + "listen_scale": "variable:listen_scale", + "num_frames": "variable:num_frames", + "width": "variable:width", + "height": "variable:height", + "num_inference_steps": "variable:num_inference_steps", + "output": [ + "videos", + "audio", + "sampling_rate" + ] + } + }, + "result": { + "content_type": "video/mp4", + "fps": 24 + } + } + ] +} diff --git a/bench/workflows/lf-s1-static.json b/bench/workflows/lf-s1-static.json new file mode 100644 index 000000000..8c2a9697a --- /dev/null +++ b/bench/workflows/lf-s1-static.json @@ -0,0 +1,214 @@ +{ + "id": "BenchLfS1Static", + "description": "Long-form bench S1 with last_frame continuity and carry_pick: the s1-static workflow with the guide-only keys (guide_frames, guide_backoff, keystone) removed and chain.carry_pick = {sharpest: carry_pick_n} (default 4). The counter-round last_frame carry_pick arms run here; every other last_frame arm runs the plain s1-static workflow with continuity=last_frame. `duration_seconds` (null by default, so the chain keeps `segments`) asks for a length in seconds instead, for the two-minute confirmations.", + "variables": { + "prompt": "For the target video, at 0.00 seconds into the target video, (from [Shot 1]) is fully referenced.\n\nintegrated_multimodal_description: [Shot 1] Cinematic, live-action, one continuous static shot. The shot opens exactly on : a young woman in a plain dark grey sweater stands centered in front of a flat painted wall of bold diagonal rainbow stripes with a black circle at upper left and a teal square at lower right. She breathes calmly, blinks, shifts her weight slightly and turns her head a little left and right, staying in place at the center of the frame. The painted wall stays exactly as it is, evenly lit by soft studio light. The camera is locked off on a tripod and never moves.\n\noverall_soundscape: Quiet studio room tone, the soft rustle of a wool sweater, calm breathing.\n\nnon_diegetic_music: None.", + "image": "asset:bench/static-stripes.jpg", + "num_frames": 124, + "segments": 10, + "continuity": "last_frame", + "width": 960, + "height": 544, + "num_inference_steps": 5, + "video_shift": 6, + "audio_shift": 3, + "weights_dtype": "{int4}", + "seed": 1166, + "anchor": "none", + "guide_noise": null, + "lora_model_name": "lightx2v/Minimax-h3-Turbo", + "lora_weight_name": "minimax_h3_fl2v_turbo_4step_v1.2_768p_bf16.safetensors", + "attention_backend": "sparse", + "anchor_strength": null, + "look_hold": null, + "antiburn_model_name": null, + "antiburn_weight_name": "h3_antiburn_universal_v1.safetensors", + "antiburn_scale": 0.6, + "carry_pick_n": 4, + "duration_seconds": null + }, + "variable_constraints": { + "num_frames": { + "modulus": 17, + "remainder": 5, + "min_frames": 124, + "max_frames": 345, + "snap": "up", + "reason": "the video VAE encodes 17 * n + 5 frames, and MiniMax-H3 generates between 5 and 15 seconds at 24 fps" + }, + "width": { + "modulus": 32, + "remainder": 0, + "reason": "H3 refuses width/height not a multiple of 32" + }, + "height": { + "modulus": 32, + "remainder": 0, + "reason": "H3 refuses width/height not a multiple of 32" + } + }, + "seed": "variable:seed", + "steps": [ + { + "name": "chained_keyframe_to_video_audio", + "pipeline": { + "configuration": { + "component_type": "ModularPipeline", + "pre_load_modules": [ + "sdnq" + ], + "load_components": { + "dtype": "torch.bfloat16", + "quantization_config": { + "transformer": { + "configuration": { + "config_type": "sdnq.SDNQConfig" + }, + "arguments": { + "weights_dtype": "variable:weights_dtype", + "quantization_device": "cuda", + "return_device": "cpu", + "use_quantized_matmul": true, + "dequantize_fp32": false, + "modules_to_not_convert": [ + "proj_in", + "audio_proj_in", + "context_embedder", + "time_embedder", + "time_proj", + "token_refiner", + "norm_out", + "proj_out", + "audio_proj_out" + ] + } + }, + "text_encoder": { + "configuration": { + "config_type": "sdnq.SDNQConfig" + }, + "arguments": { + "weights_dtype": "variable:weights_dtype", + "quantization_device": "cuda", + "return_device": "cpu", + "dequantize_fp32": false, + "modules_to_not_convert": [ + ".model.visual", + "lm_head" + ] + } + }, + "vae": { + "configuration": { + "config_type": "sdnq.SDNQConfig" + }, + "arguments": { + "weights_dtype": "{int8}", + "quant_conv": true, + "use_quantized_matmul_conv": true, + "quantization_device": "cuda", + "return_device": "cpu", + "dequantize_fp32": false + } + } + } + }, + "components": { + "transformer": { + "attention_backend": "variable:attention_backend", + "sparse_tau": 0.5, + "sparse_topk": 0.15, + "group_offload": { + "offload_type": "block_level", + "num_blocks_per_group": 2, + "use_stream": true, + "record_stream": true, + "low_cpu_mem_usage": true + } + }, + "text_encoder": { + "remove_modules": [ + "lm_head" + ] + }, + "text_encoder.model": { + "truncate_layers": { + "language_model.layers": 51 + }, + "group_offload": { + "offload_type": "leaf_level" + } + }, + "vae": { + "device": "cuda", + "residency": "on_demand" + }, + "audio_vae": { + "device": "cuda", + "residency": "on_demand" + } + } + }, + "from_pretrained_arguments": { + "model_name": "MiniMaxAI/MiniMax-H3", + "workflow": "fl2va" + }, + "loras": [ + { + "model_name": "variable:lora_model_name", + "weight_name": "variable:lora_weight_name", + "adapter_name": "turbo", + "scale": 1 + }, + { + "model_name": "variable:antiburn_model_name", + "weight_name": "variable:antiburn_weight_name", + "adapter_name": "antiburn", + "scale": "variable:antiburn_scale" + } + ], + "chain": { + "segments": "variable:segments", + "duration_seconds": "variable:duration_seconds", + "continuity": "variable:continuity", + "anchor": "variable:anchor", + "segment_argument": "image", + "trim_frames": 2, + "crossfade_ms": 80, + "fps": 24, + "frame_snap": "constraint:num_frames", + "anchor_strength": "variable:anchor_strength", + "look_hold": "variable:look_hold", + "carry_pick": { + "sharpest": "variable:carry_pick_n" + } + }, + "scheduler": { + "shift": "variable:video_shift" + }, + "audio_scheduler": { + "shift": "variable:audio_shift" + }, + "arguments": { + "prompt": "variable:prompt", + "image": "variable:image", + "num_frames": "variable:num_frames", + "width": "variable:width", + "height": "variable:height", + "num_inference_steps": "variable:num_inference_steps", + "guide_noise": "variable:guide_noise", + "output": [ + "videos", + "audio", + "sampling_rate" + ] + } + }, + "result": { + "content_type": "video/mp4", + "fps": 24, + "subfolder": "final" + } + } + ] +} diff --git a/bench/workflows/lf-s2-arc.json b/bench/workflows/lf-s2-arc.json new file mode 100644 index 000000000..42909ae11 --- /dev/null +++ b/bench/workflows/lf-s2-arc.json @@ -0,0 +1,214 @@ +{ + "id": "BenchLfS2Arc", + "description": "Long-form bench S2 with last_frame continuity and carry_pick: the s2-arc workflow with the guide-only keys (guide_frames, guide_backoff, keystone) removed and chain.carry_pick = {sharpest: carry_pick_n} (default 4). The counter-round last_frame carry_pick arms run here; every other last_frame arm runs the plain s2-arc workflow with continuity=last_frame. `duration_seconds` (null by default, so the chain keeps `segments`) asks for a length in seconds instead, for the two-minute confirmations.", + "variables": { + "prompt": "For the target video, at 0.00 seconds into the target video, (from [Shot 1]) is fully referenced.\n\nintegrated_multimodal_description: [Shot 1] Cinematic, live-action, one continuous shot. The shot opens exactly on : a bronze statue of a seated man on a stone plinth in a sunlit public garden, a red brick wall with ivy, tall cypress trees and a white gazebo behind it. The camera travels in a slow, steady orbit around the statue, arcing to the right at a constant speed and keeping the statue centered, so the brick wall, the cypresses, the gazebo and the garden behind it sweep past and change with the angle. Warm late-afternoon sunlight, leaves stirring gently in a light breeze.\n\noverall_soundscape: Outdoor garden ambience: light breeze in the leaves, distant birdsong, faint far-off traffic.\n\nnon_diegetic_music: None.", + "image": "asset:bench/arc-statue.jpg", + "num_frames": 124, + "segments": 10, + "continuity": "last_frame", + "width": 960, + "height": 544, + "num_inference_steps": 5, + "video_shift": 6, + "audio_shift": 3, + "weights_dtype": "{int4}", + "seed": 1166, + "anchor": "none", + "guide_noise": null, + "lora_model_name": "lightx2v/Minimax-h3-Turbo", + "lora_weight_name": "minimax_h3_fl2v_turbo_4step_v1.2_768p_bf16.safetensors", + "attention_backend": "sparse", + "anchor_strength": null, + "look_hold": null, + "antiburn_model_name": null, + "antiburn_weight_name": "h3_antiburn_universal_v1.safetensors", + "antiburn_scale": 0.6, + "carry_pick_n": 4, + "duration_seconds": null + }, + "variable_constraints": { + "num_frames": { + "modulus": 17, + "remainder": 5, + "min_frames": 124, + "max_frames": 345, + "snap": "up", + "reason": "the video VAE encodes 17 * n + 5 frames, and MiniMax-H3 generates between 5 and 15 seconds at 24 fps" + }, + "width": { + "modulus": 32, + "remainder": 0, + "reason": "H3 refuses width/height not a multiple of 32" + }, + "height": { + "modulus": 32, + "remainder": 0, + "reason": "H3 refuses width/height not a multiple of 32" + } + }, + "seed": "variable:seed", + "steps": [ + { + "name": "chained_keyframe_to_video_audio", + "pipeline": { + "configuration": { + "component_type": "ModularPipeline", + "pre_load_modules": [ + "sdnq" + ], + "load_components": { + "dtype": "torch.bfloat16", + "quantization_config": { + "transformer": { + "configuration": { + "config_type": "sdnq.SDNQConfig" + }, + "arguments": { + "weights_dtype": "variable:weights_dtype", + "quantization_device": "cuda", + "return_device": "cpu", + "use_quantized_matmul": true, + "dequantize_fp32": false, + "modules_to_not_convert": [ + "proj_in", + "audio_proj_in", + "context_embedder", + "time_embedder", + "time_proj", + "token_refiner", + "norm_out", + "proj_out", + "audio_proj_out" + ] + } + }, + "text_encoder": { + "configuration": { + "config_type": "sdnq.SDNQConfig" + }, + "arguments": { + "weights_dtype": "variable:weights_dtype", + "quantization_device": "cuda", + "return_device": "cpu", + "dequantize_fp32": false, + "modules_to_not_convert": [ + ".model.visual", + "lm_head" + ] + } + }, + "vae": { + "configuration": { + "config_type": "sdnq.SDNQConfig" + }, + "arguments": { + "weights_dtype": "{int8}", + "quant_conv": true, + "use_quantized_matmul_conv": true, + "quantization_device": "cuda", + "return_device": "cpu", + "dequantize_fp32": false + } + } + } + }, + "components": { + "transformer": { + "attention_backend": "variable:attention_backend", + "sparse_tau": 0.5, + "sparse_topk": 0.15, + "group_offload": { + "offload_type": "block_level", + "num_blocks_per_group": 2, + "use_stream": true, + "record_stream": true, + "low_cpu_mem_usage": true + } + }, + "text_encoder": { + "remove_modules": [ + "lm_head" + ] + }, + "text_encoder.model": { + "truncate_layers": { + "language_model.layers": 51 + }, + "group_offload": { + "offload_type": "leaf_level" + } + }, + "vae": { + "device": "cuda", + "residency": "on_demand" + }, + "audio_vae": { + "device": "cuda", + "residency": "on_demand" + } + } + }, + "from_pretrained_arguments": { + "model_name": "MiniMaxAI/MiniMax-H3", + "workflow": "fl2va" + }, + "loras": [ + { + "model_name": "variable:lora_model_name", + "weight_name": "variable:lora_weight_name", + "adapter_name": "turbo", + "scale": 1 + }, + { + "model_name": "variable:antiburn_model_name", + "weight_name": "variable:antiburn_weight_name", + "adapter_name": "antiburn", + "scale": "variable:antiburn_scale" + } + ], + "chain": { + "segments": "variable:segments", + "duration_seconds": "variable:duration_seconds", + "continuity": "variable:continuity", + "anchor": "variable:anchor", + "segment_argument": "image", + "trim_frames": 2, + "crossfade_ms": 80, + "fps": 24, + "frame_snap": "constraint:num_frames", + "anchor_strength": "variable:anchor_strength", + "look_hold": "variable:look_hold", + "carry_pick": { + "sharpest": "variable:carry_pick_n" + } + }, + "scheduler": { + "shift": "variable:video_shift" + }, + "audio_scheduler": { + "shift": "variable:audio_shift" + }, + "arguments": { + "prompt": "variable:prompt", + "image": "variable:image", + "num_frames": "variable:num_frames", + "width": "variable:width", + "height": "variable:height", + "num_inference_steps": "variable:num_inference_steps", + "guide_noise": "variable:guide_noise", + "output": [ + "videos", + "audio", + "sampling_rate" + ] + } + }, + "result": { + "content_type": "video/mp4", + "fps": 24, + "subfolder": "final" + } + } + ] +} diff --git a/bench/workflows/lf-s5-ltx-action.json b/bench/workflows/lf-s5-ltx-action.json new file mode 100644 index 000000000..7704a8b81 --- /dev/null +++ b/bench/workflows/lf-s5-ltx-action.json @@ -0,0 +1,191 @@ +{ + "id": "BenchS5LtxActionCarryPick", + "description": "Long-form bench S5 with last_frame continuity and carry_pick: the s5-ltx-action workflow with chain.carry_pick = {sharpest: carry_pick_n} (default 4). duration_seconds is removed (the chain refuses it beside carry_pick).", + "summary": "A long LTX-2.5 clip with audio, chained from short segments that each start on the previous one's last frame.", + "cost_drivers": [ + "segments", + "num_frames", + "width", + "height" + ], + "variables": { + "transformer_weights_dtype": "{uint4}", + "text_encoder_weights_dtype": "{int8}", + "prompt": "The courier, a man in a blue shirt and black helmet with a yellow delivery box on the back, is pedaling steadily along the coastal road toward the right of frame, leaning slightly into the wind, the sea glittering beyond the low stone wall behind him as the camera tracks alongside at his speed. The only sounds are the whir of the chain, the hum of tires on asphalt and the steady rush of the wind and surf, with no speech and no music.", + "prompts": [ + "The courier, a man in a blue shirt and black helmet with a yellow delivery box on the back, is pedaling steadily along the coastal road toward the right of frame, leaning slightly into the wind, the sea glittering beyond the low stone wall behind him as the camera tracks alongside at his speed. The only sounds are the whir of the chain, the hum of tires on asphalt and the steady rush of the wind and surf, with no speech and no music.", + "The courier keeps pedaling at the same pace, the road running on along the shoreline, waves rolling in beyond the stone wall as the camera continues to track alongside at his speed. The only sounds are the whir of the chain, tires on asphalt and the wind and surf, with no speech and no music.", + "The courier pedals on without slowing, the road running straight ahead along the water, gulls wheeling over the waves as the camera continues to track alongside at his speed. The only sounds are the whir of the chain, tires on asphalt and the wind and surf, with no speech and no music." + ], + "negative_prompt": "constant:diffusers.pipelines.ltx2.utils.DEFAULT_NEGATIVE_PROMPT", + "width": 960, + "height": 544, + "frame_rate": 24.0, + "num_frames": 121, + "segments": 3, + "continuity": "last_frame", + "image": "asset:bench/runner.png", + "seed": 1166, + "look_hold": null, + "check": false, + "retries": 0, + "carry_pick_n": 4 + }, + "variable_constraints": { + "width": { + "modulus": 32, + "remainder": 0, + "reason": "LTX-2.5's pipelines refuse a width not divisible by 32 outright (ValueError, not a silent floor), so it is checked here before the chain's first segment load rather than after it" + }, + "height": { + "modulus": 32, + "remainder": 0, + "reason": "LTX-2.5's pipelines refuse a height not divisible by 32 outright (ValueError, not a silent floor), so it is checked here before the chain's first segment load rather than after it" + }, + "num_frames": { + "modulus": 8, + "remainder": 1, + "min_frames": 9, + "reason": "LTX-2.5's video VAE encodes 8 * n + 1 frames, so an off-grid count is not refused by the pipeline but floored to the grid below - a shorter clip than the one asked for. Refused here instead: no 'snap', because rounding a length the other way from the pipeline's own would be a second silent change" + } + }, + "seed": "variable:seed", + "steps": [ + { + "name": "chained_image_to_video", + "pipeline": { + "configuration": { + "component_type": "LTX2ImageToVideoPipeline", + "pre_load_modules": [ + "sdnq" + ], + "components": { + "transformer": { + "device": "cuda" + }, + "text_encoder": { + "group_offload": { + "offload_type": "leaf_level", + "use_stream": true + } + }, + "connectors": { + "group_offload": { + "offload_type": "leaf_level", + "use_stream": true + } + }, + "vae": { + "device": "cuda" + }, + "audio_vae": { + "device": "cuda" + }, + "vocoder": { + "device": "cuda" + }, + "duration_head": { + "device": "cuda" + } + }, + "vae": { + "enable_tiling": true + } + }, + "from_pretrained_arguments": { + "model_name": "Lightricks/LTX-2.5-Diffusers", + "torch_dtype": "torch.bfloat16" + }, + "transformer": { + "configuration": { + "component_type": "LTX2VideoTransformer3DModel", + "preserve_device_placement": true + }, + "quantization_config": { + "configuration": { + "config_type": "sdnq.SDNQConfig" + }, + "arguments": { + "weights_dtype": "variable:transformer_weights_dtype", + "quantization_device": "cuda", + "return_device": "cuda", + "use_quantized_matmul": true, + "dequantize_fp32": false + } + }, + "from_pretrained_arguments": { + "model_name": "Lightricks/LTX-2.5-Diffusers", + "subfolder": "transformer", + "torch_dtype": "torch.bfloat16" + } + }, + "text_encoder": { + "configuration": { + "component_type": "transformers.Gemma4UnifiedForConditionalGeneration", + "preserve_device_placement": true + }, + "quantization_config": { + "configuration": { + "config_type": "sdnq.SDNQConfig" + }, + "arguments": { + "weights_dtype": "variable:text_encoder_weights_dtype", + "quantization_device": "cuda", + "return_device": "cpu", + "dequantize_fp32": false, + "modules_to_not_convert": [ + "embed_vision", + "embed_audio", + "lm_head" + ] + } + }, + "from_pretrained_arguments": { + "model_name": "Lightricks/LTX-2.5-Diffusers", + "subfolder": "text_encoder", + "torch_dtype": "torch.bfloat16" + } + }, + "chain": { + "segments": "variable:segments", + "prompts": "variable:prompts", + "continuity": "variable:continuity", + "segment_argument": "image", + "trim_frames": 2, + "crossfade_ms": 80, + "save_segments": true, + "fps": 24, + "frame_snap": "constraint:num_frames", + "look_hold": "variable:look_hold", + "check": "variable:check", + "retries": "variable:retries", + "carry_pick": { + "sharpest": "variable:carry_pick_n" + } + }, + "arguments": { + "prompt": "variable:prompt", + "negative_prompt": "variable:negative_prompt", + "image": "variable:image", + "width": "variable:width", + "height": "variable:height", + "num_frames": "variable:num_frames", + "frame_rate": "variable:frame_rate", + "sigmas": "constant:diffusers.pipelines.ltx2.utils.DISTILLED_SIGMA_VALUES", + "guidance_scale": 1.0, + "audio_guidance_scale": 1.0, + "stg_scale": 0.0, + "audio_stg_scale": 0.0, + "modality_scale": 1.0, + "audio_modality_scale": 1.0, + "output_type": "{np}" + } + }, + "result": { + "content_type": "video/mp4", + "fps": "variable:frame_rate", + "subfolder": "final" + } + } + ] +} diff --git a/bench/workflows/ltx-arc.json b/bench/workflows/ltx-arc.json new file mode 100644 index 000000000..c9b0ae7cf --- /dev/null +++ b/bench/workflows/ltx-arc.json @@ -0,0 +1,190 @@ +{ + "id": "BenchLtxArc", + "description": "Long-form bench, LTX orbit check for Task 23: the s5-ltx-action workflow (last_frame chain, three 121-frame segments) on S2's still (asset:bench/arc-statue.jpg) with an orbit caption, to see whether look_hold holds an LTX camera move back the way it held H3's S2 orbit back (bench/records/counters.md). Measured under scene s2-arc's crop with per-arm moments; the crop was verified on H3 frames, not these, so its texture reads are not used.", + "summary": "A long LTX-2.5 clip with audio, chained from short segments that each start on the previous one's last frame.", + "cost_drivers": [ + "segments", + "duration_seconds", + "num_frames", + "width", + "height" + ], + "variables": { + "transformer_weights_dtype": "{uint4}", + "text_encoder_weights_dtype": "{int8}", + "prompt": "A wide shot frames a bronze statue of a seated man on a weathered stone plinth in a sunlit public garden, a red brick wall covered in ivy, tall dark-green cypress trees and a white wooden gazebo behind it, as the camera travels in a slow, steady orbit around the statue, arcing to the right at a constant speed and keeping the statue centered, so the brick wall, the cypresses and the gazebo sweep past behind it and the statue is seen from a steadily changing side angle. Warm late-afternoon sunlight falls across the bronze and the gravel path, and leaves stir gently in a light breeze. A light breeze rustles the leaves, distant birdsong carries across the garden and faint traffic hums far off, with no speech and no music.", + "prompts": [ + "A wide shot frames a bronze statue of a seated man on a weathered stone plinth in a sunlit public garden, a red brick wall covered in ivy, tall dark-green cypress trees and a white wooden gazebo behind it, as the camera travels in a slow, steady orbit around the statue, arcing to the right at a constant speed and keeping the statue centered, so the brick wall, the cypresses and the gazebo sweep past behind it and the statue is seen from a steadily changing side angle. Warm late-afternoon sunlight falls across the bronze and the gravel path, and leaves stir gently in a light breeze. A light breeze rustles the leaves, distant birdsong carries across the garden and faint traffic hums far off, with no speech and no music.", + "The camera keeps orbiting to the right around the bronze statue of the seated man at the same slow, steady speed, keeping the statue centered in a wide shot, so the garden behind it keeps sweeping past and the statue turns from a side view toward its back. Warm late-afternoon sunlight, leaves stirring in a light breeze. A light breeze in the leaves, distant birdsong and faint far-off traffic, with no speech and no music.", + "The camera keeps orbiting to the right around the bronze statue of the seated man at the same slow, steady speed, keeping the statue centered in a wide shot, so the garden behind it keeps sweeping past and the statue turns from a side view toward its back. Warm late-afternoon sunlight, leaves stirring in a light breeze. A light breeze in the leaves, distant birdsong and faint far-off traffic, with no speech and no music." + ], + "negative_prompt": "constant:diffusers.pipelines.ltx2.utils.DEFAULT_NEGATIVE_PROMPT", + "width": 960, + "height": 544, + "frame_rate": 24.0, + "num_frames": 121, + "segments": 3, + "continuity": "last_frame", + "image": "asset:bench/arc-statue.jpg", + "seed": 1166, + "duration_seconds": null, + "look_hold": null, + "check": false, + "retries": 0 + }, + "variable_constraints": { + "width": { + "modulus": 32, + "remainder": 0, + "reason": "LTX-2.5's pipelines refuse a width not divisible by 32 outright (ValueError, not a silent floor), so it is checked here before the chain's first segment load rather than after it" + }, + "height": { + "modulus": 32, + "remainder": 0, + "reason": "LTX-2.5's pipelines refuse a height not divisible by 32 outright (ValueError, not a silent floor), so it is checked here before the chain's first segment load rather than after it" + }, + "num_frames": { + "modulus": 8, + "remainder": 1, + "min_frames": 9, + "reason": "LTX-2.5's video VAE encodes 8 * n + 1 frames, so an off-grid count is not refused by the pipeline but floored to the grid below - a shorter clip than the one asked for. Refused here instead: no 'snap', because rounding a length the other way from the pipeline's own would be a second silent change" + } + }, + "seed": "variable:seed", + "steps": [ + { + "name": "chained_image_to_video", + "pipeline": { + "configuration": { + "component_type": "LTX2ImageToVideoPipeline", + "pre_load_modules": [ + "sdnq" + ], + "components": { + "transformer": { + "device": "cuda" + }, + "text_encoder": { + "group_offload": { + "offload_type": "leaf_level", + "use_stream": true + } + }, + "connectors": { + "group_offload": { + "offload_type": "leaf_level", + "use_stream": true + } + }, + "vae": { + "device": "cuda" + }, + "audio_vae": { + "device": "cuda" + }, + "vocoder": { + "device": "cuda" + }, + "duration_head": { + "device": "cuda" + } + }, + "vae": { + "enable_tiling": true + } + }, + "from_pretrained_arguments": { + "model_name": "Lightricks/LTX-2.5-Diffusers", + "torch_dtype": "torch.bfloat16" + }, + "transformer": { + "configuration": { + "component_type": "LTX2VideoTransformer3DModel", + "preserve_device_placement": true + }, + "quantization_config": { + "configuration": { + "config_type": "sdnq.SDNQConfig" + }, + "arguments": { + "weights_dtype": "variable:transformer_weights_dtype", + "quantization_device": "cuda", + "return_device": "cuda", + "use_quantized_matmul": true, + "dequantize_fp32": false + } + }, + "from_pretrained_arguments": { + "model_name": "Lightricks/LTX-2.5-Diffusers", + "subfolder": "transformer", + "torch_dtype": "torch.bfloat16" + } + }, + "text_encoder": { + "configuration": { + "component_type": "transformers.Gemma4UnifiedForConditionalGeneration", + "preserve_device_placement": true + }, + "quantization_config": { + "configuration": { + "config_type": "sdnq.SDNQConfig" + }, + "arguments": { + "weights_dtype": "variable:text_encoder_weights_dtype", + "quantization_device": "cuda", + "return_device": "cpu", + "dequantize_fp32": false, + "modules_to_not_convert": [ + "embed_vision", + "embed_audio", + "lm_head" + ] + } + }, + "from_pretrained_arguments": { + "model_name": "Lightricks/LTX-2.5-Diffusers", + "subfolder": "text_encoder", + "torch_dtype": "torch.bfloat16" + } + }, + "chain": { + "segments": "variable:segments", + "duration_seconds": "variable:duration_seconds", + "prompts": "variable:prompts", + "continuity": "variable:continuity", + "segment_argument": "image", + "trim_frames": 2, + "crossfade_ms": 80, + "save_segments": true, + "fps": 24, + "frame_snap": "constraint:num_frames", + "look_hold": "variable:look_hold", + "check": "variable:check", + "retries": "variable:retries" + }, + "arguments": { + "prompt": "variable:prompt", + "negative_prompt": "variable:negative_prompt", + "image": "variable:image", + "width": "variable:width", + "height": "variable:height", + "num_frames": "variable:num_frames", + "frame_rate": "variable:frame_rate", + "sigmas": "constant:diffusers.pipelines.ltx2.utils.DISTILLED_SIGMA_VALUES", + "guidance_scale": 1.0, + "audio_guidance_scale": 1.0, + "stg_scale": 0.0, + "audio_stg_scale": 0.0, + "modality_scale": 1.0, + "audio_modality_scale": 1.0, + "output_type": "{np}" + } + }, + "result": { + "content_type": "video/mp4", + "fps": "variable:frame_rate", + "subfolder": "final" + } + } + ] +} diff --git a/bench/workflows/music-long-s6-full.json b/bench/workflows/music-long-s6-full.json new file mode 100644 index 000000000..d68234169 --- /dev/null +++ b/bench/workflows/music-long-s6-full.json @@ -0,0 +1,60 @@ +{ + "id": "BenchS6MusicLongFull", + "summary": "About 335 s of music in four sections with full-length lyrics, through templates/minimax/music-long.", + "description": "Long-form bench S6 rerun after the music-long trim fix (237ee307), with about 10-12 lyric lines per 85 s section so each section can fill its length (the 4-line lyrics of music-long-s6.json gave 27-70 s): composes templates/minimax/music-long by its catalog name with four 85 s sections (each generated at a 128 s ceiling, 1.5 x seconds per the Music 3 skill), so the joined piece is about 335 s, past one Music 3 track's 300 s. Every caption repeats the same style tags, key and tempo. What to read: key, tempo, instrumentation and mix at each of the three crossfades (music-long's description says continuity is not guaranteed), and the level step at each join. A joined_short warning, or none, says whether every section reached 85 s.", + "variables": { + "sections": [ + { + "name": "verse_1", + "caption": "slow cinematic folk, solo fingerpicked acoustic guitar, low cello, soft brushed snare, warm intimate male baritone vocal, D minor, 72 BPM, close dry mix, a sparse opening, guitar and voice only, the cello entering late", + "lyrics": "[intro]\n[verse]\nOne hundred stairs and twelve more still\nIron worn smooth beneath my heel\nMy father climbed them, so will I\nI light the lamp against the sky\nThe oil is low, the wick is new\nI trim it like he taught me to\nSalt on the glass, I wipe it clear\nThe only work that matters here\n[pre-chorus]\nAnd the dark comes down the water\nAnd the dark comes to the door", + "seconds": 85, + "audio_duration": 128 + }, + { + "name": "chorus_1", + "caption": "slow cinematic folk, solo fingerpicked acoustic guitar, low cello, soft brushed snare, warm intimate male baritone vocal, D minor, 72 BPM, close dry mix, the band fills in, brushed snare steady, the cello carrying the melody under the voice", + "lyrics": "[chorus]\nTurn, light, turn across the water\nEvery eight seconds, find them there\nTurn, light, turn for someone's daughter\nSomeone's son out in the air\n[post-chorus]\nTurn, turn, the long night through\nTurn, turn, I'm turning for you\n[chorus]\nTurn, light, turn across the water\nEvery eight seconds, find them there", + "seconds": 85, + "audio_duration": 128 + }, + { + "name": "verse_2", + "caption": "slow cinematic folk, solo fingerpicked acoustic guitar, low cello, soft brushed snare, warm intimate male baritone vocal, D minor, 72 BPM, close dry mix, the storm verse, the snare busier, the cello low and urgent, the voice strained", + "lyrics": "[verse]\nThe wind came first across the point\nIt shook the bolt in every joint\nA lantern swinging, small and low\nI counted forty, let it go\nThe rain came sideways off the sea\nIt found the cracks it left for me\nI held the rail and climbed again\nAnd kept the beam on all the men\n[bridge]\nSomewhere a sail, somewhere a hand\nSomewhere a light they understand", + "seconds": 85, + "audio_duration": 128 + }, + { + "name": "chorus_2", + "caption": "slow cinematic folk, solo fingerpicked acoustic guitar, low cello, soft brushed snare, warm intimate male baritone vocal, D minor, 72 BPM, close dry mix, the last chorus then a quiet outro, the band drops away to guitar and cello, a soft ending", + "lyrics": "[chorus]\nTurn, light, turn across the water\nEvery eight seconds, bring them home\nTurn, light, turn for someone's daughter\nNo one out there sails alone\n[chorus]\nTurn, light, turn across the water\nEvery eight seconds, bring them home\n[outro]\nBread and honey on the stair\nI never asked who left them there\nThe morning grey, the water still\nOne hundred stairs and twelve more still", + "seconds": 85, + "audio_duration": 128 + } + ], + "crossfade_ms": 1500, + "section_fade_out_ms": 400, + "seed": 1166 + }, + "seed": "variable:seed", + "steps": [ + { + "name": "piece", + "workflow": { + "path": "templates/minimax/music-long", + "arguments": { + "sections": "variable:sections", + "crossfade_ms": "variable:crossfade_ms", + "section_fade_out_ms": "variable:section_fade_out_ms", + "seed": "variable:seed" + } + }, + "result": { + "content_type": "audio/mp3", + "sample_rate": 44100, + "subfolder": "final" + } + } + ] +} diff --git a/bench/workflows/music-long-s6.json b/bench/workflows/music-long-s6.json new file mode 100644 index 000000000..c4d99f7f0 --- /dev/null +++ b/bench/workflows/music-long-s6.json @@ -0,0 +1,60 @@ +{ + "id": "BenchS6MusicLong", + "summary": "About 335 s of music in four sections, through templates/minimax/music-long.", + "description": "Long-form bench S6 (long music over 5 minutes): composes templates/minimax/music-long by its catalog name with four 85 s sections (each generated at a 128 s ceiling, 1.5 x seconds per the Music 3 skill), so the joined piece is about 335 s, past one Music 3 track's 300 s. Every caption repeats the same style tags, key and tempo. What to read: key, tempo, instrumentation and mix at each of the three crossfades (music-long's description says continuity is not guaranteed), and the level step at each join.", + "variables": { + "sections": [ + { + "name": "verse_1", + "caption": "slow cinematic folk, solo fingerpicked acoustic guitar, low cello, soft brushed snare, warm intimate male baritone vocal, D minor, 72 BPM, close dry mix, a sparse opening, guitar and voice only, the cello entering late", + "lyrics": "[intro]\n[verse]\nOne hundred stairs and twelve more still\nIron worn smooth beneath my heel\nMy father climbed them, so will I\nI light the lamp against the sky", + "seconds": 85, + "audio_duration": 128 + }, + { + "name": "chorus_1", + "caption": "slow cinematic folk, solo fingerpicked acoustic guitar, low cello, soft brushed snare, warm intimate male baritone vocal, D minor, 72 BPM, close dry mix, the band fills in, brushed snare steady, the cello carrying the melody under the voice", + "lyrics": "[chorus]\nTurn, light, turn across the water\nEvery eight seconds, find them there\nTurn, light, turn for someone's daughter\nSomeone's son out in the air", + "seconds": 85, + "audio_duration": 128 + }, + { + "name": "verse_2", + "caption": "slow cinematic folk, solo fingerpicked acoustic guitar, low cello, soft brushed snare, warm intimate male baritone vocal, D minor, 72 BPM, close dry mix, the storm verse, the snare busier, the cello low and urgent, the voice strained", + "lyrics": "[verse]\nThe wind came first across the point\nIt shook the bolt in every joint\nA lantern swinging, small and low\nI counted forty, let it go", + "seconds": 85, + "audio_duration": 128 + }, + { + "name": "chorus_2", + "caption": "slow cinematic folk, solo fingerpicked acoustic guitar, low cello, soft brushed snare, warm intimate male baritone vocal, D minor, 72 BPM, close dry mix, the last chorus then a quiet outro, the band drops away to guitar and cello, a soft ending", + "lyrics": "[chorus]\nTurn, light, turn across the water\nEvery eight seconds, bring them home\n[outro]\nBread and honey on the stair\nI never asked who left them there", + "seconds": 85, + "audio_duration": 128 + } + ], + "crossfade_ms": 1500, + "section_fade_out_ms": 400, + "seed": 1166 + }, + "seed": "variable:seed", + "steps": [ + { + "name": "piece", + "workflow": { + "path": "templates/minimax/music-long", + "arguments": { + "sections": "variable:sections", + "crossfade_ms": "variable:crossfade_ms", + "section_fade_out_ms": "variable:section_fade_out_ms", + "seed": "variable:seed" + } + }, + "result": { + "content_type": "audio/mp3", + "sample_rate": 44100, + "subfolder": "final" + } + } + ] +} diff --git a/bench/workflows/narrated-take.json b/bench/workflows/narrated-take.json new file mode 100644 index 000000000..a6b057146 --- /dev/null +++ b/bench/workflows/narrated-take.json @@ -0,0 +1,347 @@ +{ + "id": "BenchNarratedTake", + "summary": "templates/minimax/narrated-take with the experimental chain keys exposed, on the CP0 Constitution narration.", + "description": "Narrated long take bench (#1308 plan Task 9 step 5, CP1/CP2): a copy of templates/minimax/narrated-take - the template composed by its catalog name could not take them - whose chain block also reads experimental_latent_carry, experimental_capstone and anchor_every from variables of those names (keystone, keystone_slot and keyframes are the template's own variables now). Their defaults (false, null, null) are the template's own chain, so an arm passing none of them runs the template as shipped; tests/test_bench_workflows.py pins the copy to the template. The defaults are the CP0 inputs: 'narration' is the retimed track approved at CP0 (run templates/narration-retime/20261109-200509-4083f66c, final/narration-retimed-0.0-audio.wav, 302.57 s), uploaded as asset:bench/constitution-narration-retimed.wav; 'image' is the Constitution walk's speaker still (constitution-walk-2, output:templates/flux3/image-edit/20261108-011458-22568496/final/Flux3KleinImageEdit-edit.1-0.0.jpg), uploaded as asset:bench/constitution-speaker.jpg; 'script' is the Preamble and the ten amendments the narration speaks. The end of Amendment 3 (85.11 s) falls in segment 12 (ends 85.92 s), so a probe to it is stop_after 12.", + "shape": "sequence", + "vram_estimate": { + "base_gb": 16.3, + "bytes_per_voxel": 28.71, + "bytes_per_guide_voxel": 28.71, + "voxel_variables": [ + "width", + "height", + "num_frames" + ], + "reason": "each segment is one H3 run, so the ceiling is a segment's: the t2va calibration (#324) - peak activation memory scales with width * height * num_frames - plus, under 'guide' continuity, the guide each segment after the first is given, which is encoded on the target canvas and costs about as much per voxel as the clip itself (docs/proposals/h3-guide-vram.md, #694)" + }, + "host_ram_gb": "variable:host_ram_gb", + "cost_drivers": [ + "tier", + "continuity", + "shot", + "num_frames", + "num_inference_steps", + "width", + "height", + "weights_dtype", + "lora_model_name", + "attention_backend", + "cache", + "check", + "retries" + ], + "variables": { + "tier": "best", + "host_ram_gb": 69, + "image": "asset:bench/constitution-speaker.jpg", + "narration": "asset:bench/constitution-narration-retimed.wav", + "script": "We the People of the United States, in Order to form a more perfect Union, establish Justice, insure domestic Tranquility, provide for the common defence, promote the general Welfare, and secure the Blessings of Liberty to ourselves and our Posterity, do ordain and establish this Constitution for the United States of America.\n\nAmendment One. Congress shall make no law respecting an establishment of religion, or prohibiting the free exercise thereof; or abridging the freedom of speech, or of the press; or the right of the people peaceably to assemble, and to petition the Government for a redress of grievances.\n\nAmendment Two. A well regulated Militia, being necessary to the security of a free State, the right of the people to keep and bear Arms, shall not be infringed.\n\nAmendment Three. No Soldier shall, in time of peace be quartered in any house, without the consent of the Owner, nor in time of war, but in a manner to be prescribed by law.\n\nAmendment Four. The right of the people to be secure in their persons, houses, papers, and effects, against unreasonable searches and seizures, shall not be violated, and no Warrants shall issue, but upon probable cause, supported by Oath or affirmation, and particularly describing the place to be searched, and the persons or things to be seized.\n\nAmendment Five. No person shall be held to answer for a capital, or otherwise infamous crime, unless on a presentment or indictment of a Grand Jury, except in cases arising in the land or naval forces, or in the Militia, when in actual service in time of War or public danger; nor shall any person be subject for the same offence to be twice put in jeopardy of life or limb; nor shall be compelled in any criminal case to be a witness against himself, nor be deprived of life, liberty, or property, without due process of law; nor shall private property be taken for public use, without just compensation.\n\nAmendment Six. In all criminal prosecutions, the accused shall enjoy the right to a speedy and public trial, by an impartial jury of the State and district wherein the crime shall have been committed, which district shall have been previously ascertained by law, and to be informed of the nature and cause of the accusation; to be confronted with the witnesses against him; to have compulsory process for obtaining witnesses in his favor, and to have the Assistance of Counsel for his defence.\n\nAmendment Seven. In Suits at common law, where the value in controversy shall exceed twenty dollars, the right of trial by jury shall be preserved, and no fact tried by a jury, shall be otherwise re-examined in any Court of the United States, than according to the rules of the common law.\n\nAmendment Eight. Excessive bail shall not be required, nor excessive fines imposed, nor cruel and unusual punishments inflicted.\n\nAmendment Nine. The enumeration in the Constitution, of certain rights, shall not be construed to deny or disparage others retained by the people.\n\nAmendment Ten. The powers not delegated to the United States by the Constitution, nor prohibited by it to the States, are reserved to the States respectively, or to the people.", + "scene": "a close-up from the top of the shoulders up of the woman in , centred, with close-cropped silver natural hair, small gold stud earrings, a charcoal grey wool overcoat over a cream turtleneck and a deep burgundy wool scarf, the bare elm-lined gravel path and the pale neoclassical stone building softly out of focus behind her, in soft overcast late-afternoon light with a muted earthy grade. She walks slowly and steadily toward the camera, which travels backward on a smooth dolly at exactly her walking pace, her eyes level on the lens, her face composed and grave.", + "speaker": "a Black American woman in her mid-sixties with a low, warm alto voice that carries the settled weight of age, a faint natural huskiness, a slow, measured, serious delivery with clear diction and quiet resolve, never theatrical", + "soundscape": "Quiet open-air park ambience under her voice: a soft steady breeze through the bare elm branches, the slow, even crunch of her shoes on the gravel, faint distant birdsong.", + "prompt_template": "For the target video, at 0.00 seconds into the target video, (from [Shot 1]) is fully referenced.\n\nintegrated_multimodal_description: [Shot 1] Live-action, cinematic, photorealistic, one continuous shot that carries over without a break from the previous shot. The shot opens exactly on : {scene} Preserving the speaker's face, clothing, framing and the setting exactly, the camera holds its framing for the whole shot: it never pushes in, and the speaker stays the same size in frame from the first frame to the last. (S1), {speaker}, at the same quiet level and unhurried pace as before, is already speaking and continues mid-flow, lips and jaw shaping every word clearly and in time with the sound: [English] {line} The delivery carries straight on to the end of the shot, with no pause to start over.\n\noverall_soundscape: {soundscape} The ambience runs under the voice throughout and never drops out.\n\nnon_diegetic_music: N/A", + "shot": "dialogue", + "continuity": "guide", + "num_frames": 192, + "guide_frames": 22, + "stop_after": null, + "resume_from": null, + "keep_segments": true, + "check": { + "jump_ratio_max": 4.0, + "look_ratio_max": 5.5, + "cut_ratio_max": 3.5, + "words_similarity_min": null, + "face_drop_max": null + }, + "retries": 1, + "width": 1344, + "height": 768, + "num_inference_steps": 20, + "video_shift": 12.0, + "audio_shift": 3.0, + "weights_dtype": "{int8}", + "lora_scale": 1.0, + "lora_alpha": null, + "lora_model_name": null, + "lora_weight_name": "minimax_h3_fl2v_turbo_4step_v1.2_768p_bf16.safetensors", + "lora_adapter_name": "turbo", + "seed": null, + "attention_backend": "sparse", + "sparse_tau": 0.5, + "sparse_topk": 0.15, + "cache": null, + "experimental_latent_carry": false, + "keystone": "none", + "keystone_slot": null, + "keyframes": null, + "experimental_capstone": null, + "anchor_every": null + }, + "presets": { + "shot": { + "dialogue": { + "continuity": "guide" + }, + "standing": { + "continuity": "guide", + "keystone": "segment_1", + "keystone_slot": "last" + }, + "walk_keyframed": { + "continuity": "guide", + "keystone": "segment_1", + "keystone_slot": "last" + } + }, + "tier": { + "standard": { + "width": 960, + "height": 544, + "num_inference_steps": 20, + "weights_dtype": "{int4}", + "host_ram_gb": 51 + }, + "best": { + "width": 1344, + "height": 768, + "num_inference_steps": 20, + "weights_dtype": "{int8}", + "host_ram_gb": 69 + } + } + }, + "variable_constraints": { + "image": { + "required": true, + "reason": "the speaker's first frame, of every segment prompt; a placeholder would film the wrong person" + }, + "narration": { + "required": true, + "reason": "the retimed narration from templates/narration-retime, the take's held soundtrack and the source of its segment count; there is no sensible default track" + }, + "script": { + "required": true, + "min_length": 1, + "reason": "the text the narration speaks, so each segment's line carries its spelling and punctuation" + }, + "scene": { + "required": true, + "min_length": 1, + "reason": "the shot's framing, the speaker's look and the setting, written for this take; a placeholder would describe another scene" + }, + "speaker": { + "required": true, + "min_length": 1, + "reason": "the speaker's voice and delivery for the (S1) tag; a placeholder would describe another voice" + }, + "num_frames": { + "modulus": 17, + "remainder": 5, + "min_frames": 124, + "max_frames": 345, + "snap": "up", + "reason": "the video VAE encodes 17 * n + 5 frames, and MiniMax-H3 generates between 5 and 15 seconds at 24 fps" + }, + "width": { + "modulus": 32, + "remainder": 0, + "reason": "MiniMax-H3's pipeline refuses a `width` or `height` that is not a multiple of 32, and only after the weights have loaded; refused here instead, with no 'snap', since rounding would be a silent change to the size asked for" + }, + "height": { + "modulus": 32, + "remainder": 0, + "reason": "MiniMax-H3's pipeline refuses a `width` or `height` that is not a multiple of 32, and only after the weights have loaded; refused here instead, with no 'snap', since rounding would be a silent change to the size asked for" + } + }, + "seed": "variable:seed", + "steps": [ + { + "name": "lines", + "task": { + "command": "narration_lines", + "arguments": { + "audio": "variable:narration", + "text": "variable:script", + "segment_frames": "variable:num_frames", + "guide_frames": "variable:guide_frames", + "fps": 24, + "prompt_template": "variable:prompt_template", + "fill": { + "scene": "variable:scene", + "speaker": "variable:speaker", + "soundscape": "variable:soundscape" + } + } + }, + "result": { + "content_type": "application/json", + "file_base_name": "narration-lines", + "subfolder": "intermediate" + } + }, + { + "name": "take", + "pipeline": { + "configuration": { + "component_type": "ModularPipeline", + "pre_load_modules": [ + "sdnq" + ], + "load_components": { + "dtype": "torch.bfloat16", + "quantization_config": { + "transformer": { + "configuration": { + "config_type": "sdnq.SDNQConfig" + }, + "arguments": { + "weights_dtype": "variable:weights_dtype", + "quantization_device": "cuda", + "return_device": "cpu", + "use_quantized_matmul": true, + "dequantize_fp32": false, + "modules_to_not_convert": [ + "proj_in", + "audio_proj_in", + "context_embedder", + "time_embedder", + "time_proj", + "token_refiner", + "norm_out", + "proj_out", + "audio_proj_out" + ] + } + }, + "text_encoder": { + "configuration": { + "config_type": "sdnq.SDNQConfig" + }, + "arguments": { + "weights_dtype": "variable:weights_dtype", + "quantization_device": "cuda", + "return_device": "cpu", + "dequantize_fp32": false, + "modules_to_not_convert": [ + ".model.visual", + "lm_head" + ] + } + }, + "vae": { + "configuration": { + "config_type": "sdnq.SDNQConfig" + }, + "arguments": { + "weights_dtype": "{int8}", + "quant_conv": true, + "use_quantized_matmul_conv": true, + "quantization_device": "cuda", + "return_device": "cpu", + "dequantize_fp32": false + } + } + } + }, + "components": { + "transformer": { + "attention_backend": "variable:attention_backend", + "sparse_tau": "variable:sparse_tau", + "sparse_topk": "variable:sparse_topk", + "group_offload": { + "offload_type": "block_level", + "num_blocks_per_group": 2, + "use_stream": true, + "record_stream": true, + "low_cpu_mem_usage": true + } + }, + "text_encoder": { + "remove_modules": [ + "lm_head" + ] + }, + "text_encoder.model": { + "truncate_layers": { + "language_model.layers": 51 + }, + "group_offload": { + "offload_type": "leaf_level" + } + }, + "vae": { + "device": "cuda", + "residency": "on_demand" + }, + "audio_vae": { + "device": "cuda", + "residency": "on_demand" + } + }, + "cache": "variable:cache" + }, + "from_pretrained_arguments": { + "model_name": "MiniMaxAI/MiniMax-H3", + "workflow": "fl2va" + }, + "loras": [ + { + "model_name": "variable:lora_model_name", + "weight_name": "variable:lora_weight_name", + "adapter_name": "variable:lora_adapter_name", + "scale": "variable:lora_scale", + "alpha": "variable:lora_alpha" + } + ], + "chain": { + "match_audio": true, + "prompts": "previous_result:lines.prompts", + "continuity": "variable:continuity", + "guide_frames": "variable:guide_frames", + "check": "variable:check", + "retries": "variable:retries", + "stop_after": "variable:stop_after", + "segment_argument": "image", + "trim_frames": 2, + "crossfade_ms": 80, + "fps": 24, + "frame_snap": "constraint:num_frames", + "save_segments": true, + "keep_segments": "variable:keep_segments", + "resume_from": "variable:resume_from", + "experimental_latent_carry": "variable:experimental_latent_carry", + "keystone": "variable:keystone", + "keystone_slot": "variable:keystone_slot", + "keystone_images": "variable:keyframes", + "experimental_capstone": "variable:experimental_capstone", + "anchor_every": "variable:anchor_every" + }, + "scheduler": { + "shift": "variable:video_shift" + }, + "audio_scheduler": { + "shift": "variable:audio_shift" + }, + "arguments": { + "image": "variable:image", + "hold_audio": "variable:narration", + "num_frames": "variable:num_frames", + "width": "variable:width", + "height": "variable:height", + "num_inference_steps": "variable:num_inference_steps", + "output": [ + "videos", + "audio", + "sampling_rate" + ] + } + }, + "result": { + "content_type": "video/mp4", + "fps": 24, + "subfolder": "final" + } + } + ] +} diff --git a/bench/workflows/narration-long-s6.json b/bench/workflows/narration-long-s6.json new file mode 100644 index 000000000..64519766e --- /dev/null +++ b/bench/workflows/narration-long-s6.json @@ -0,0 +1,38 @@ +{ + "id": "BenchS6NarrationLong", + "summary": "About four minutes of narration from one text, through templates/narration-long.", + "description": "Long-form bench S6 (long narration over 3 minutes): composes templates/narration-long by its catalog name, so the bench runs the template as shipped. The text is about 590 words (about 240 s at the template's 2.5 words a second), with an abbreviation (Dr.) and long comma-heavy sentences so split_takes' sentence and comma rules are exercised. The voice is the synthetic narrator clip narrator-reference-s6.json writes, kept as asset:bench/narrator.wav. What to read: the level step and the pause at each take join, voice drift between takes, and check-script on the joined file against 'text'.", + "variables": { + "text": "The keeper climbed the stairs every evening, long before the storm. There were one hundred and twelve of them, iron and worn smooth in the middle, and he knew each one by the sound it made under his boots. His father had climbed them before him, and his grandfather before that, and none of them had ever missed a night. By the time he reached the lamp, the sea had already gone dark. He trimmed the wick, wiped the salt from the glass, and wound the clockwork that turned the great lens, and then he sat down on the wooden stool to wait. Then the light turned, and the whole coast could see him.\n\nOn the night the storm came in, the wind arrived first. It came across the water low and fast, flattening the grass on the headland and rattling the door at the foot of the tower until the bolt shook in its housing. The keeper heard it before he felt it. He set down his tea, buttoned his coat to the collar, and went to the window that faced the open sea. Out past the point, where the shelf of rock ran under the surface like the back of some sleeping animal, the waves had begun to break white.\n\nDr. Alder had warned him about the glass. It was old, the doctor said, older than the town, and a pane that has stood a hundred winters does not always stand the hundred and first. The keeper had nodded politely and done nothing, because there was nothing to be done. There was no new glass to be had, not on that coast, not that year, and the light would turn whether he worried about it or not.\n\nAt a little after midnight he saw the ship. It was a fishing boat, by the shape of it, small and low in the water, and it was running for the harbour with the wind behind it. He watched it the way you watch a child on a high wall, without breathing, as if your attention alone could hold it up. Twice the boat vanished into the trough of a wave and twice it came back, a little closer each time, its single lantern swinging like a pendulum.\n\nThe third time it did not come back for a long while. The keeper stood with both hands flat against the cold glass and counted, the way his father had taught him, one, and two, and three, all the way to forty. At forty-one the lantern rose again out of the dark, further north than before, and he understood that the boat had lost its line and was drifting toward the rocks.\n\nSo he did the only thing a keeper can do. He went to the lamp and turned up the flame as high as it would go, until the room was so bright he had to shield his eyes, and he stood beside it while it turned. Every eight seconds the beam swept across the water, and every eight seconds it found the boat, and every eight seconds the boat turned a little further south, toward the harbour mouth and away from the stones.\n\nIn the morning the storm was gone and the sea was flat and grey and ordinary. A man came up from the harbour with a loaf of bread and a jar of honey and left them on the bottom step without knocking. The keeper found them when he came down to sleep. He did not need to ask who they were from.", + "max_seconds": 60, + "words_per_second": 2.5, + "reference_audio": "asset:bench/narrator.wav", + "reference_text": "The tide comes in twice a day, and the lamp has to be ready for both.", + "gap_seconds": 0.3, + "crossfade_ms": 30, + "seed": 1166 + }, + "seed": "variable:seed", + "steps": [ + { + "name": "narration", + "workflow": { + "path": "templates/narration-long", + "arguments": { + "text": "variable:text", + "max_seconds": "variable:max_seconds", + "words_per_second": "variable:words_per_second", + "reference_audio": "variable:reference_audio", + "reference_text": "variable:reference_text", + "seed": "variable:seed", + "gap_seconds": "variable:gap_seconds", + "crossfade_ms": "variable:crossfade_ms" + } + }, + "result": { + "content_type": "audio/wav", + "subfolder": "final" + } + } + ] +} diff --git a/bench/workflows/narrator-reference-s6.json b/bench/workflows/narrator-reference-s6.json new file mode 100644 index 000000000..c563e51df --- /dev/null +++ b/bench/workflows/narrator-reference-s6.json @@ -0,0 +1,26 @@ +{ + "id": "BenchS6NarratorReference", + "summary": "A synthetic narrator clip for the S6 long-narration arms.", + "description": "Long-form bench S6 (long narration), preparation: Audio8-TTS speaks 'reference_text' in its own default voice, about 5 s, so the narration-long arms have a reference voice to clone. The voice is synthetic, so no person's consent is at issue; no cast voice clip is on the bench host. Keep the output as the asset 'bench/narrator.wav' (keep_output) before running bench/arms/audio-long.json (this one is bench/arms/audio-long-prep.json, run first: the driver submits a file's arms together, and a missing asset is refused at submit), which read it as asset:bench/narrator.wav with this same 'reference_text'.", + "variables": { + "reference_text": "The tide comes in twice a day, and the lamp has to be ready for both.", + "seed": 1166 + }, + "seed": "variable:seed", + "steps": [ + { + "name": "narrator", + "task": { + "command": "generate_speech", + "arguments": { + "text": "variable:reference_text", + "model_name": "Edge0/Audio8-TTS-Realistic-0.6b" + } + }, + "result": { + "content_type": "audio/wav", + "subfolder": "final" + } + } + ] +} diff --git a/bench/workflows/refine-windows.json b/bench/workflows/refine-windows.json new file mode 100644 index 000000000..b3b37c6f7 --- /dev/null +++ b/bench/workflows/refine-windows.json @@ -0,0 +1,216 @@ +{ + "id": "BenchRefineWindows", + "description": "Task 17 Step 4 bench: a temporal windowed refine over a finished chain. encode_h3_latents reads the take (an output: or asset: mp4) a 17-frame clip at a time; an H3 t2va refine_strength pass re-denoises it in overlapping time windows fused every step (experimental_refine_windows, needs DW_EXPERIMENTAL_REFINE_WINDOWS=1 on the server), holding the take's own soundtrack; decode_h3_latents trims the padding back to the take's length and pair_audio puts the take's own track back. Windows are in latent frames (a latent is about 4 pixel frames).", + "variables": { + "source": "output:BenchS1Static/20261104-054934-fdc9b551/final/BenchS1Static-chained_keyframe_to_video_audio.0-0.0.mp4", + "prompt": "integrated_multimodal_description: [Shot 1] Cinematic, live-action, one continuous static shot. A young woman in a plain dark grey sweater stands centered in front of a flat painted wall of bold diagonal rainbow stripes with a black circle at upper left and a teal square at lower right. She breathes calmly, blinks, shifts her weight slightly and turns her head a little left and right, staying in place at the center of the frame. The painted wall stays exactly as it is, evenly lit by soft studio light. The camera is locked off on a tripod and never moves.\n\noverall_soundscape: Quiet studio room tone, the soft rustle of a wool sweater, calm breathing.\n\nnon_diegetic_music: None.", + "width": 960, + "height": 544, + "refine_strength": 0.3, + "num_inference_steps": 5, + "window_frames": 40, + "window_overlap": 12, + "window_blend": "pyramid", + "window_positions": "true", + "video_shift": 6, + "audio_shift": 3, + "weights_dtype": "{int4}", + "lora_model_name": "lightx2v/Minimax-h3-Turbo", + "lora_weight_name": "minimax_h3_fl2v_turbo_4step_v1.2_768p_bf16.safetensors", + "attention_backend": "sparse", + "seed": 1166 + }, + "variable_constraints": { + "width": { + "modulus": 32, + "remainder": 0, + "reason": "H3 refuses width/height not a multiple of 32" + }, + "height": { + "modulus": 32, + "remainder": 0, + "reason": "H3 refuses width/height not a multiple of 32" + } + }, + "seed": "variable:seed", + "steps": [ + { + "name": "encode", + "task": { + "command": "encode_h3_latents", + "arguments": { + "source": "variable:source" + } + }, + "release_models": true + }, + { + "name": "refine", + "release_pipeline": true, + "pipeline": { + "configuration": { + "component_type": "ModularPipeline", + "pre_load_modules": [ + "sdnq" + ], + "load_components": { + "dtype": "torch.bfloat16", + "quantization_config": { + "transformer": { + "configuration": { + "config_type": "sdnq.SDNQConfig" + }, + "arguments": { + "weights_dtype": "variable:weights_dtype", + "quantization_device": "cuda", + "return_device": "cpu", + "use_quantized_matmul": true, + "dequantize_fp32": false, + "modules_to_not_convert": [ + "proj_in", + "audio_proj_in", + "context_embedder", + "time_embedder", + "time_proj", + "token_refiner", + "norm_out", + "proj_out", + "audio_proj_out" + ] + } + }, + "text_encoder": { + "configuration": { + "config_type": "sdnq.SDNQConfig" + }, + "arguments": { + "weights_dtype": "variable:weights_dtype", + "quantization_device": "cuda", + "return_device": "cpu", + "dequantize_fp32": false, + "modules_to_not_convert": [ + ".model.visual", + "lm_head" + ] + } + }, + "vae": { + "configuration": { + "config_type": "sdnq.SDNQConfig" + }, + "arguments": { + "weights_dtype": "{int8}", + "quant_conv": true, + "use_quantized_matmul_conv": true, + "quantization_device": "cuda", + "return_device": "cpu", + "dequantize_fp32": false + } + } + } + }, + "components": { + "transformer": { + "attention_backend": "variable:attention_backend", + "sparse_tau": 0.5, + "sparse_topk": 0.15, + "group_offload": { + "offload_type": "block_level", + "num_blocks_per_group": 2, + "use_stream": true, + "record_stream": true, + "low_cpu_mem_usage": true + } + }, + "text_encoder": { + "remove_modules": [ + "lm_head" + ] + }, + "text_encoder.model": { + "truncate_layers": { + "language_model.layers": 51 + }, + "group_offload": { + "offload_type": "leaf_level" + } + }, + "vae": { + "device": "cuda", + "residency": "on_demand" + }, + "audio_vae": { + "device": "cuda", + "residency": "on_demand" + } + } + }, + "from_pretrained_arguments": { + "model_name": "MiniMaxAI/MiniMax-H3", + "workflow": "t2va" + }, + "loras": [ + { + "model_name": "variable:lora_model_name", + "weight_name": "variable:lora_weight_name", + "adapter_name": "turbo", + "scale": 1 + } + ], + "scheduler": { + "shift": "variable:video_shift" + }, + "audio_scheduler": { + "shift": "variable:audio_shift" + }, + "arguments": { + "prompt": "variable:prompt", + "latents": "previous_result:encode.latents", + "num_frames": "previous_result:encode.num_frames", + "width": "variable:width", + "height": "variable:height", + "hold_audio": "variable:source", + "refine_strength": "variable:refine_strength", + "num_inference_steps": "variable:num_inference_steps", + "experimental_refine_windows": { + "frames": "variable:window_frames", + "overlap": "variable:window_overlap", + "blend": "variable:window_blend", + "positions": "variable:window_positions" + }, + "output": [ + "latents" + ] + } + } + }, + { + "name": "decode", + "task": { + "command": "decode_h3_latents", + "arguments": { + "latents": "previous_result:refine.latents", + "num_frames": "previous_result:encode.source_frames" + } + }, + "release_models": true + }, + { + "name": "mux", + "task": { + "command": "pair_audio", + "arguments": { + "video": "previous_result:decode", + "audio": "variable:source", + "fps": 24, + "fit": "video" + } + }, + "result": { + "content_type": "video/mp4", + "fps": 24, + "subfolder": "final" + } + } + ] +} diff --git a/bench/workflows/s1-static.json b/bench/workflows/s1-static.json new file mode 100644 index 000000000..9959d9cec --- /dev/null +++ b/bench/workflows/s1-static.json @@ -0,0 +1,233 @@ +{ + "id": "BenchS1Static", + "description": "Long-form bench S1 (static): the #1175 control (arm A), reproduced exactly - same workflow, prompt, still, seed 1166 and arguments as probe-1166's drift-chain. #1175 drift probe: templates/minimax/chained-segments as shipped (turbo-4step-keyframe, fl2va) with guide continuity, plus two probe variables: `anchor` (chain.anchor, arm C) and `guide_noise` (H3 call argument, arm B; null is dropped before the call, so the default is today's call). Defaults are the control (arm A). Counter-round knobs (all null/off by default, so the defaults are still the control): anchor_strength, guide_backoff, keystone, look_hold (chain keys) and antiburn_model_name (set to minimax-h3's Imagineer13/H3-Anti-Burn-LoRA to load it at antiburn_scale on every segment). `duration_seconds` (null by default, so the chain keeps `segments`) asks for a length in seconds instead, for the two-minute confirmations. Task 13/14 knobs (off by default, so the defaults are still the control): `check` and `retries` (the per-segment check and its re-roll budget, chain keys) and `keep_segments` (save_segments + keep_segments, the segment files repair-segment needs). `latent_carry` (null by default) is the experimental latent carry (#1150, chain key experimental_latent_carry; needs DW_EXPERIMENTAL_REFINE_WINDOWS=1 on the server): each guide takes the previous segment's own tail latents instead of re-encoding its decoded frames. `tst` (null by default) is the experimental TST temporal-attention correction (#1196, H3 call argument experimental_tst {tau, per_head}; needs DW_EXPERIMENTAL_REFINE_WINDOWS=1 and attention_backend null - it runs on dense attention). `selflift` (null by default) is the experimental SelfLift (#1195, H3 call argument experimental_selflift {lift_step, start_scale 0.5, upsample?}; needs DW_EXPERIMENTAL_REFINE_WINDOWS=1 and a width and height that are multiples of 64, e.g. 1024x576): the first lift_step evaluations at half canvas, then lifted to full size inside the same schedule. `keystone_slot` (null by default, plain keystone) is the chain key keystone_slot (needs keystone segment_1; a gated frame slot needs DW_EXPERIMENTAL_REFINE_WINDOWS=1): 'last' makes segment 1's frame 0 every later segment's fl2va last_image, {frame, enter_below_sigma?} a one-frame guide at that frame. `prompts` (null by default, every segment on `prompt`) is the chain's per-segment prompt list, for the 'last' arm's later-segment prompt that names .", + "variables": { + "prompt": "For the target video, at 0.00 seconds into the target video, (from [Shot 1]) is fully referenced.\n\nintegrated_multimodal_description: [Shot 1] Cinematic, live-action, one continuous static shot. The shot opens exactly on : a young woman in a plain dark grey sweater stands centered in front of a flat painted wall of bold diagonal rainbow stripes with a black circle at upper left and a teal square at lower right. She breathes calmly, blinks, shifts her weight slightly and turns her head a little left and right, staying in place at the center of the frame. The painted wall stays exactly as it is, evenly lit by soft studio light. The camera is locked off on a tripod and never moves.\n\noverall_soundscape: Quiet studio room tone, the soft rustle of a wool sweater, calm breathing.\n\nnon_diegetic_music: None.", + "image": "asset:bench/static-stripes.jpg", + "num_frames": 124, + "segments": 10, + "continuity": "guide", + "guide_frames": 22, + "width": 960, + "height": 544, + "num_inference_steps": 5, + "video_shift": 6, + "audio_shift": 3, + "weights_dtype": "{int4}", + "seed": 1166, + "anchor": "none", + "guide_noise": null, + "lora_model_name": "lightx2v/Minimax-h3-Turbo", + "lora_weight_name": "minimax_h3_fl2v_turbo_4step_v1.2_768p_bf16.safetensors", + "attention_backend": "sparse", + "anchor_strength": null, + "guide_backoff": null, + "keystone": null, + "look_hold": null, + "antiburn_model_name": null, + "antiburn_weight_name": "h3_antiburn_universal_v1.safetensors", + "antiburn_scale": 0.6, + "duration_seconds": null, + "check": false, + "retries": 0, + "keep_segments": false, + "latent_carry": null, + "tst": null, + "selflift": null, + "keystone_slot": null, + "prompts": null + }, + "variable_constraints": { + "num_frames": { + "modulus": 17, + "remainder": 5, + "min_frames": 124, + "max_frames": 345, + "snap": "up", + "reason": "the video VAE encodes 17 * n + 5 frames, and MiniMax-H3 generates between 5 and 15 seconds at 24 fps" + }, + "width": { + "modulus": 32, + "remainder": 0, + "reason": "H3 refuses width/height not a multiple of 32" + }, + "height": { + "modulus": 32, + "remainder": 0, + "reason": "H3 refuses width/height not a multiple of 32" + } + }, + "seed": "variable:seed", + "steps": [ + { + "name": "chained_keyframe_to_video_audio", + "pipeline": { + "configuration": { + "component_type": "ModularPipeline", + "pre_load_modules": [ + "sdnq" + ], + "load_components": { + "dtype": "torch.bfloat16", + "quantization_config": { + "transformer": { + "configuration": { + "config_type": "sdnq.SDNQConfig" + }, + "arguments": { + "weights_dtype": "variable:weights_dtype", + "quantization_device": "cuda", + "return_device": "cpu", + "use_quantized_matmul": true, + "dequantize_fp32": false, + "modules_to_not_convert": [ + "proj_in", + "audio_proj_in", + "context_embedder", + "time_embedder", + "time_proj", + "token_refiner", + "norm_out", + "proj_out", + "audio_proj_out" + ] + } + }, + "text_encoder": { + "configuration": { + "config_type": "sdnq.SDNQConfig" + }, + "arguments": { + "weights_dtype": "variable:weights_dtype", + "quantization_device": "cuda", + "return_device": "cpu", + "dequantize_fp32": false, + "modules_to_not_convert": [ + ".model.visual", + "lm_head" + ] + } + }, + "vae": { + "configuration": { + "config_type": "sdnq.SDNQConfig" + }, + "arguments": { + "weights_dtype": "{int8}", + "quant_conv": true, + "use_quantized_matmul_conv": true, + "quantization_device": "cuda", + "return_device": "cpu", + "dequantize_fp32": false + } + } + } + }, + "components": { + "transformer": { + "attention_backend": "variable:attention_backend", + "sparse_tau": 0.5, + "sparse_topk": 0.15, + "group_offload": { + "offload_type": "block_level", + "num_blocks_per_group": 2, + "use_stream": true, + "record_stream": true, + "low_cpu_mem_usage": true + } + }, + "text_encoder": { + "remove_modules": [ + "lm_head" + ] + }, + "text_encoder.model": { + "truncate_layers": { + "language_model.layers": 51 + }, + "group_offload": { + "offload_type": "leaf_level" + } + }, + "vae": { + "device": "cuda", + "residency": "on_demand" + }, + "audio_vae": { + "device": "cuda", + "residency": "on_demand" + } + } + }, + "from_pretrained_arguments": { + "model_name": "MiniMaxAI/MiniMax-H3", + "workflow": "fl2va" + }, + "loras": [ + { + "model_name": "variable:lora_model_name", + "weight_name": "variable:lora_weight_name", + "adapter_name": "turbo", + "scale": 1 + }, + { + "model_name": "variable:antiburn_model_name", + "weight_name": "variable:antiburn_weight_name", + "adapter_name": "antiburn", + "scale": "variable:antiburn_scale" + } + ], + "chain": { + "segments": "variable:segments", + "duration_seconds": "variable:duration_seconds", + "continuity": "variable:continuity", + "guide_frames": "variable:guide_frames", + "anchor": "variable:anchor", + "segment_argument": "image", + "trim_frames": 2, + "crossfade_ms": 80, + "fps": 24, + "frame_snap": "constraint:num_frames", + "anchor_strength": "variable:anchor_strength", + "guide_backoff": "variable:guide_backoff", + "keystone": "variable:keystone", + "look_hold": "variable:look_hold", + "check": "variable:check", + "retries": "variable:retries", + "save_segments": "variable:keep_segments", + "keep_segments": "variable:keep_segments", + "experimental_latent_carry": "variable:latent_carry", + "keystone_slot": "variable:keystone_slot", + "prompts": "variable:prompts" + }, + "scheduler": { + "shift": "variable:video_shift" + }, + "audio_scheduler": { + "shift": "variable:audio_shift" + }, + "arguments": { + "prompt": "variable:prompt", + "image": "variable:image", + "num_frames": "variable:num_frames", + "width": "variable:width", + "height": "variable:height", + "num_inference_steps": "variable:num_inference_steps", + "guide_noise": "variable:guide_noise", + "experimental_tst": "variable:tst", + "experimental_selflift": "variable:selflift", + "output": [ + "videos", + "audio", + "sampling_rate" + ] + } + }, + "result": { + "content_type": "video/mp4", + "fps": 24, + "subfolder": "final" + } + } + ] +} diff --git a/bench/workflows/s2-arc.json b/bench/workflows/s2-arc.json new file mode 100644 index 000000000..a10ea3556 --- /dev/null +++ b/bench/workflows/s2-arc.json @@ -0,0 +1,231 @@ +{ + "id": "BenchS2Arc", + "description": "Long-form bench S2 (arc): the #1175 control (arm A), reproduced exactly - same workflow, prompt, still, seed 1166 and arguments as probe-1166's drift-chain. #1175 drift probe: templates/minimax/chained-segments as shipped (turbo-4step-keyframe, fl2va) with guide continuity, plus two probe variables: `anchor` (chain.anchor, arm C) and `guide_noise` (H3 call argument, arm B; null is dropped before the call, so the default is today's call). Defaults are the control (arm A). Counter-round knobs (all null/off by default, so the defaults are still the control): anchor_strength, guide_backoff, keystone, look_hold (chain keys) and antiburn_model_name (set to minimax-h3's Imagineer13/H3-Anti-Burn-LoRA to load it at antiburn_scale on every segment). `duration_seconds` (null by default, so the chain keeps `segments`) asks for a length in seconds instead, for the two-minute confirmations. Task 13/14 knobs (off by default, so the defaults are still the control): `check` and `retries` (the per-segment check and its re-roll budget, chain keys) and `keep_segments` (save_segments + keep_segments, the segment files repair-segment needs). `capstone` (null by default) is the experimental late capstone (#1187, chain key experimental_capstone; needs DW_EXPERIMENTAL_REFINE_WINDOWS=1 on the server). `latent_carry` (null by default) is the experimental latent carry (#1150, chain key experimental_latent_carry; needs DW_EXPERIMENTAL_REFINE_WINDOWS=1 on the server): each guide takes the previous segment's own tail latents instead of re-encoding its decoded frames. `tst` (null by default) is the experimental TST temporal-attention correction (#1196, H3 call argument experimental_tst {tau, per_head}; needs DW_EXPERIMENTAL_REFINE_WINDOWS=1 and attention_backend null - it runs on dense attention). `selflift` (null by default) is the experimental SelfLift (#1195, H3 call argument experimental_selflift {lift_step, start_scale 0.5, upsample?}; needs DW_EXPERIMENTAL_REFINE_WINDOWS=1 and a width and height that are multiples of 64, e.g. 1024x576): the first lift_step evaluations at half canvas, then lifted to full size inside the same schedule.", + "variables": { + "prompt": "For the target video, at 0.00 seconds into the target video, (from [Shot 1]) is fully referenced.\n\nintegrated_multimodal_description: [Shot 1] Cinematic, live-action, one continuous shot. The shot opens exactly on : a bronze statue of a seated man on a stone plinth in a sunlit public garden, a red brick wall with ivy, tall cypress trees and a white gazebo behind it. The camera travels in a slow, steady orbit around the statue, arcing to the right at a constant speed and keeping the statue centered, so the brick wall, the cypresses, the gazebo and the garden behind it sweep past and change with the angle. Warm late-afternoon sunlight, leaves stirring gently in a light breeze.\n\noverall_soundscape: Outdoor garden ambience: light breeze in the leaves, distant birdsong, faint far-off traffic.\n\nnon_diegetic_music: None.", + "image": "asset:bench/arc-statue.jpg", + "num_frames": 124, + "segments": 10, + "continuity": "guide", + "guide_frames": 22, + "width": 960, + "height": 544, + "num_inference_steps": 5, + "video_shift": 6, + "audio_shift": 3, + "weights_dtype": "{int4}", + "seed": 1166, + "anchor": "none", + "guide_noise": null, + "lora_model_name": "lightx2v/Minimax-h3-Turbo", + "lora_weight_name": "minimax_h3_fl2v_turbo_4step_v1.2_768p_bf16.safetensors", + "attention_backend": "sparse", + "anchor_strength": null, + "guide_backoff": null, + "keystone": null, + "look_hold": null, + "antiburn_model_name": null, + "antiburn_weight_name": "h3_antiburn_universal_v1.safetensors", + "antiburn_scale": 0.6, + "duration_seconds": null, + "check": false, + "retries": 0, + "keep_segments": false, + "capstone": null, + "latent_carry": null, + "tst": null, + "selflift": null + }, + "variable_constraints": { + "num_frames": { + "modulus": 17, + "remainder": 5, + "min_frames": 124, + "max_frames": 345, + "snap": "up", + "reason": "the video VAE encodes 17 * n + 5 frames, and MiniMax-H3 generates between 5 and 15 seconds at 24 fps" + }, + "width": { + "modulus": 32, + "remainder": 0, + "reason": "H3 refuses width/height not a multiple of 32" + }, + "height": { + "modulus": 32, + "remainder": 0, + "reason": "H3 refuses width/height not a multiple of 32" + } + }, + "seed": "variable:seed", + "steps": [ + { + "name": "chained_keyframe_to_video_audio", + "pipeline": { + "configuration": { + "component_type": "ModularPipeline", + "pre_load_modules": [ + "sdnq" + ], + "load_components": { + "dtype": "torch.bfloat16", + "quantization_config": { + "transformer": { + "configuration": { + "config_type": "sdnq.SDNQConfig" + }, + "arguments": { + "weights_dtype": "variable:weights_dtype", + "quantization_device": "cuda", + "return_device": "cpu", + "use_quantized_matmul": true, + "dequantize_fp32": false, + "modules_to_not_convert": [ + "proj_in", + "audio_proj_in", + "context_embedder", + "time_embedder", + "time_proj", + "token_refiner", + "norm_out", + "proj_out", + "audio_proj_out" + ] + } + }, + "text_encoder": { + "configuration": { + "config_type": "sdnq.SDNQConfig" + }, + "arguments": { + "weights_dtype": "variable:weights_dtype", + "quantization_device": "cuda", + "return_device": "cpu", + "dequantize_fp32": false, + "modules_to_not_convert": [ + ".model.visual", + "lm_head" + ] + } + }, + "vae": { + "configuration": { + "config_type": "sdnq.SDNQConfig" + }, + "arguments": { + "weights_dtype": "{int8}", + "quant_conv": true, + "use_quantized_matmul_conv": true, + "quantization_device": "cuda", + "return_device": "cpu", + "dequantize_fp32": false + } + } + } + }, + "components": { + "transformer": { + "attention_backend": "variable:attention_backend", + "sparse_tau": 0.5, + "sparse_topk": 0.15, + "group_offload": { + "offload_type": "block_level", + "num_blocks_per_group": 2, + "use_stream": true, + "record_stream": true, + "low_cpu_mem_usage": true + } + }, + "text_encoder": { + "remove_modules": [ + "lm_head" + ] + }, + "text_encoder.model": { + "truncate_layers": { + "language_model.layers": 51 + }, + "group_offload": { + "offload_type": "leaf_level" + } + }, + "vae": { + "device": "cuda", + "residency": "on_demand" + }, + "audio_vae": { + "device": "cuda", + "residency": "on_demand" + } + } + }, + "from_pretrained_arguments": { + "model_name": "MiniMaxAI/MiniMax-H3", + "workflow": "fl2va" + }, + "loras": [ + { + "model_name": "variable:lora_model_name", + "weight_name": "variable:lora_weight_name", + "adapter_name": "turbo", + "scale": 1 + }, + { + "model_name": "variable:antiburn_model_name", + "weight_name": "variable:antiburn_weight_name", + "adapter_name": "antiburn", + "scale": "variable:antiburn_scale" + } + ], + "chain": { + "segments": "variable:segments", + "duration_seconds": "variable:duration_seconds", + "continuity": "variable:continuity", + "guide_frames": "variable:guide_frames", + "anchor": "variable:anchor", + "segment_argument": "image", + "trim_frames": 2, + "crossfade_ms": 80, + "fps": 24, + "frame_snap": "constraint:num_frames", + "anchor_strength": "variable:anchor_strength", + "guide_backoff": "variable:guide_backoff", + "keystone": "variable:keystone", + "look_hold": "variable:look_hold", + "check": "variable:check", + "retries": "variable:retries", + "save_segments": "variable:keep_segments", + "keep_segments": "variable:keep_segments", + "experimental_latent_carry": "variable:latent_carry", + "experimental_capstone": "variable:capstone" + }, + "scheduler": { + "shift": "variable:video_shift" + }, + "audio_scheduler": { + "shift": "variable:audio_shift" + }, + "arguments": { + "prompt": "variable:prompt", + "image": "variable:image", + "num_frames": "variable:num_frames", + "width": "variable:width", + "height": "variable:height", + "num_inference_steps": "variable:num_inference_steps", + "guide_noise": "variable:guide_noise", + "experimental_tst": "variable:tst", + "experimental_selflift": "variable:selflift", + "output": [ + "videos", + "audio", + "sampling_rate" + ] + } + }, + "result": { + "content_type": "video/mp4", + "fps": 24, + "subfolder": "final" + } + } + ] +} diff --git a/bench/workflows/s3-dialogue-walk.json b/bench/workflows/s3-dialogue-walk.json new file mode 100644 index 000000000..a0c3b35cc --- /dev/null +++ b/bench/workflows/s3-dialogue-walk.json @@ -0,0 +1,233 @@ +{ + "id": "BenchS3DialogueWalk", + "description": "Long-form bench S3 (dialogue walk): templates/minimax/chained-segments with continuity 'guide' and carry_audio, six segments, a cast portrait (asset:bench/walker.png) and one short spoken line per segment through `prompts`. Each line is sized to a segment (see plugins/dw/skills/minimax-h3/references/dialogue.md). `anchor` and `guide_noise` are the drift-probe knobs; their defaults are today's call. Counter-round knobs (all null/off by default, so the defaults are still the control): anchor_strength, guide_backoff, keystone, look_hold (chain keys) and antiburn_model_name (set to minimax-h3's Imagineer13/H3-Anti-Burn-LoRA to load it at antiburn_scale on every segment). `duration_seconds` (null by default, so the chain keeps `segments`) asks for a length in seconds instead, for the two-minute confirmations. Task 13/14 knobs (off by default, so the defaults are still the control): `check` and `retries` (the per-segment check and its re-roll budget, chain keys) and `keep_segments` (save_segments + keep_segments, the segment files repair-segment needs).", + "variables": { + "prompt": "For the target video, at 0.00 seconds into the target video, (from [Shot 1]) is fully referenced.\n\nintegrated_multimodal_description: [Shot 1] Cinematic, live-action, one continuous shot. The shot opens exactly on : a woman in her early thirties with shoulder-length dark hair, in a long olive coat, walks toward the camera down a quiet cobblestone street at golden hour. She walks toward the camera down the cobblestone street, slow and steady, glancing up at the lamps. The woman (S1), a warm, low female alto voice, calm and unhurried, says [English] I told myself I would not come back here. The camera holds a steady medium shot at eye level, keeping pace with her as she walks.\n\noverall_soundscape: Soft footsteps on cobblestones, a faint breeze, distant street ambience.\n\nnon_diegetic_music: None.", + "image": "asset:bench/walker.png", + "num_frames": 124, + "segments": 6, + "continuity": "guide", + "guide_frames": 22, + "width": 960, + "height": 544, + "num_inference_steps": 5, + "video_shift": 6, + "audio_shift": 3, + "weights_dtype": "{int4}", + "seed": 1166, + "anchor": "none", + "guide_noise": null, + "lora_model_name": "lightx2v/Minimax-h3-Turbo", + "lora_weight_name": "minimax_h3_fl2v_turbo_4step_v1.2_768p_bf16.safetensors", + "attention_backend": "sparse", + "prompts": [ + "For the target video, at 0.00 seconds into the target video, (from [Shot 1]) is fully referenced.\n\nintegrated_multimodal_description: [Shot 1] Cinematic, live-action, one continuous shot. The shot opens exactly on : a woman in her early thirties with shoulder-length dark hair, in a long olive coat, walks toward the camera down a quiet cobblestone street at golden hour. She walks toward the camera down the cobblestone street, slow and steady, glancing up at the lamps. The woman (S1), a warm, low female alto voice, calm and unhurried, says [English] I told myself I would not come back here. The camera holds a steady medium shot at eye level, keeping pace with her as she walks.\n\noverall_soundscape: Soft footsteps on cobblestones, a faint breeze, distant street ambience.\n\nnon_diegetic_music: None.", + "For the target video, at 0.00 seconds into the target video, (from [Shot 1]) is fully referenced.\n\nintegrated_multimodal_description: [Shot 1] Cinematic, live-action, one continuous shot. The shot opens exactly on : a woman in her early thirties with shoulder-length dark hair, in a long olive coat, walks toward the camera down a quiet cobblestone street at golden hour. Her eyes move along the shopfronts as she walks on, the long olive coat swinging with each step. The woman (S1), a warm, low female alto voice, calm and unhurried, says [English] But the street looks exactly the way I left it. The camera holds a steady medium shot at eye level, keeping pace with her as she walks.\n\noverall_soundscape: Soft footsteps on cobblestones, a faint breeze, distant street ambience.\n\nnon_diegetic_music: None.", + "For the target video, at 0.00 seconds into the target video, (from [Shot 1]) is fully referenced.\n\nintegrated_multimodal_description: [Shot 1] Cinematic, live-action, one continuous shot. The shot opens exactly on : a woman in her early thirties with shoulder-length dark hair, in a long olive coat, walks toward the camera down a quiet cobblestone street at golden hour. She slows for a moment, looks down at the stones, then lifts her chin and walks on. The woman (S1), a warm, low female alto voice, calm and unhurried, says [English] Same lamps, same crooked stones, same bakery smell. The camera holds a steady medium shot at eye level, keeping pace with her as she walks.\n\noverall_soundscape: Soft footsteps on cobblestones, a faint breeze, distant street ambience.\n\nnon_diegetic_music: None.", + "For the target video, at 0.00 seconds into the target video, (from [Shot 1]) is fully referenced.\n\nintegrated_multimodal_description: [Shot 1] Cinematic, live-action, one continuous shot. The shot opens exactly on : a woman in her early thirties with shoulder-length dark hair, in a long olive coat, walks toward the camera down a quiet cobblestone street at golden hour. A faint smile crosses her face as she keeps walking toward the camera. The woman (S1), a warm, low female alto voice, calm and unhurried, says [English] Funny how a place waits for you without asking. The camera holds a steady medium shot at eye level, keeping pace with her as she walks.\n\noverall_soundscape: Soft footsteps on cobblestones, a faint breeze, distant street ambience.\n\nnon_diegetic_music: None.", + "For the target video, at 0.00 seconds into the target video, (from [Shot 1]) is fully referenced.\n\nintegrated_multimodal_description: [Shot 1] Cinematic, live-action, one continuous shot. The shot opens exactly on : a woman in her early thirties with shoulder-length dark hair, in a long olive coat, walks toward the camera down a quiet cobblestone street at golden hour. Her expression softens and her pace eases, the golden light warm along her shoulder. The woman (S1), a warm, low female alto voice, calm and unhurried, says [English] Maybe that is what I was afraid of. The camera holds a steady medium shot at eye level, keeping pace with her as she walks.\n\noverall_soundscape: Soft footsteps on cobblestones, a faint breeze, distant street ambience.\n\nnon_diegetic_music: None.", + "For the target video, at 0.00 seconds into the target video, (from [Shot 1]) is fully referenced.\n\nintegrated_multimodal_description: [Shot 1] Cinematic, live-action, one continuous shot. The shot opens exactly on : a woman in her early thirties with shoulder-length dark hair, in a long olive coat, walks toward the camera down a quiet cobblestone street at golden hour. She squares her shoulders and walks on, the camera keeping her centered in frame. The woman (S1), a warm, low female alto voice, calm and unhurried, says [English] Well. Let's see if the door still sticks. The camera holds a steady medium shot at eye level, keeping pace with her as she walks.\n\noverall_soundscape: Soft footsteps on cobblestones, a faint breeze, distant street ambience.\n\nnon_diegetic_music: None." + ], + "anchor_strength": null, + "guide_backoff": null, + "keystone": null, + "look_hold": null, + "antiburn_model_name": null, + "antiburn_weight_name": "h3_antiburn_universal_v1.safetensors", + "antiburn_scale": 0.6, + "duration_seconds": null, + "check": false, + "retries": 0, + "keep_segments": false + }, + "variable_constraints": { + "num_frames": { + "modulus": 17, + "remainder": 5, + "min_frames": 124, + "max_frames": 345, + "snap": "up", + "reason": "the video VAE encodes 17 * n + 5 frames, and MiniMax-H3 generates between 5 and 15 seconds at 24 fps" + }, + "width": { + "modulus": 32, + "remainder": 0, + "reason": "H3 refuses width/height not a multiple of 32" + }, + "height": { + "modulus": 32, + "remainder": 0, + "reason": "H3 refuses width/height not a multiple of 32" + } + }, + "seed": "variable:seed", + "steps": [ + { + "name": "chained_keyframe_to_video_audio", + "pipeline": { + "configuration": { + "component_type": "ModularPipeline", + "pre_load_modules": [ + "sdnq" + ], + "load_components": { + "dtype": "torch.bfloat16", + "quantization_config": { + "transformer": { + "configuration": { + "config_type": "sdnq.SDNQConfig" + }, + "arguments": { + "weights_dtype": "variable:weights_dtype", + "quantization_device": "cuda", + "return_device": "cpu", + "use_quantized_matmul": true, + "dequantize_fp32": false, + "modules_to_not_convert": [ + "proj_in", + "audio_proj_in", + "context_embedder", + "time_embedder", + "time_proj", + "token_refiner", + "norm_out", + "proj_out", + "audio_proj_out" + ] + } + }, + "text_encoder": { + "configuration": { + "config_type": "sdnq.SDNQConfig" + }, + "arguments": { + "weights_dtype": "variable:weights_dtype", + "quantization_device": "cuda", + "return_device": "cpu", + "dequantize_fp32": false, + "modules_to_not_convert": [ + ".model.visual", + "lm_head" + ] + } + }, + "vae": { + "configuration": { + "config_type": "sdnq.SDNQConfig" + }, + "arguments": { + "weights_dtype": "{int8}", + "quant_conv": true, + "use_quantized_matmul_conv": true, + "quantization_device": "cuda", + "return_device": "cpu", + "dequantize_fp32": false + } + } + } + }, + "components": { + "transformer": { + "attention_backend": "variable:attention_backend", + "sparse_tau": 0.5, + "sparse_topk": 0.15, + "group_offload": { + "offload_type": "block_level", + "num_blocks_per_group": 2, + "use_stream": true, + "record_stream": true, + "low_cpu_mem_usage": true + } + }, + "text_encoder": { + "remove_modules": [ + "lm_head" + ] + }, + "text_encoder.model": { + "truncate_layers": { + "language_model.layers": 51 + }, + "group_offload": { + "offload_type": "leaf_level" + } + }, + "vae": { + "device": "cuda", + "residency": "on_demand" + }, + "audio_vae": { + "device": "cuda", + "residency": "on_demand" + } + } + }, + "from_pretrained_arguments": { + "model_name": "MiniMaxAI/MiniMax-H3", + "workflow": "fl2va" + }, + "loras": [ + { + "model_name": "variable:lora_model_name", + "weight_name": "variable:lora_weight_name", + "adapter_name": "turbo", + "scale": 1 + }, + { + "model_name": "variable:antiburn_model_name", + "weight_name": "variable:antiburn_weight_name", + "adapter_name": "antiburn", + "scale": "variable:antiburn_scale" + } + ], + "chain": { + "prompts": "variable:prompts", + "segments": "variable:segments", + "duration_seconds": "variable:duration_seconds", + "continuity": "variable:continuity", + "guide_frames": "variable:guide_frames", + "anchor": "variable:anchor", + "segment_argument": "image", + "trim_frames": 2, + "crossfade_ms": 80, + "fps": 24, + "frame_snap": "constraint:num_frames", + "carry_audio": true, + "anchor_strength": "variable:anchor_strength", + "guide_backoff": "variable:guide_backoff", + "keystone": "variable:keystone", + "look_hold": "variable:look_hold", + "check": "variable:check", + "retries": "variable:retries", + "save_segments": "variable:keep_segments", + "keep_segments": "variable:keep_segments" + }, + "scheduler": { + "shift": "variable:video_shift" + }, + "audio_scheduler": { + "shift": "variable:audio_shift" + }, + "arguments": { + "prompt": "variable:prompt", + "image": "variable:image", + "num_frames": "variable:num_frames", + "width": "variable:width", + "height": "variable:height", + "num_inference_steps": "variable:num_inference_steps", + "guide_noise": "variable:guide_noise", + "output": [ + "videos", + "audio", + "sampling_rate" + ] + } + }, + "result": { + "content_type": "video/mp4", + "fps": 24, + "subfolder": "final" + } + } + ] +} diff --git a/bench/workflows/s4-montage.json b/bench/workflows/s4-montage.json new file mode 100644 index 000000000..8fae1bc3d --- /dev/null +++ b/bench/workflows/s4-montage.json @@ -0,0 +1,502 @@ +{ + "id": "BenchS4Montage", + "description": "Long-form bench S4 (montage): templates/minimax/dialogue-short's cast and four of its shots, cut and scored under one templates/minimax/music instrumental bed the way templates/assemble-and-score does it. Each shot is generated fresh from the same portraits, so it carries no chain drift; the bed is the one thing the cuts share.", + "summary": "Four cut H3 scenes under one Music 3 bed, cut and scored.", + "cost": [ + { + "device": "cuda", + "name": "RTX 3090", + "vram_gb": 24, + "minutes": 21.2 + } + ], + "cost_drivers": [ + "shots", + "num_inference_steps", + "width", + "height", + "weights_dtype", + "lora_model_name", + "attention_backend", + "cache", + "audio_duration" + ], + "vram_estimate": { + "base_gb": 16.0, + "bytes_per_voxel": 28.71, + "gb_per_reference": 1.0, + "voxel_variables": [ + "width", + "height", + "num_frames" + ], + "reason": "Ref2VA holds every reference's encoding resident beside the video latents, so the ceiling is a fixed base, a per-voxel term for the VAE decode's conv3d activation (width * height * num_frames), and a per-reference term. Classified against the 24 GB entry from the #479 field report (lem, RTX 3090 24 GB, int4 + 768p turbo LoRA, 9 steps, 2026-09-25/26) at 1344x768: 175 frames x 2 refs, 209 x 2 and 175 x 3 ran (projected 22.83, 23.77 and 23.83 GB); 209 x 3 OOM'd in reference encode needing a further 5.1 GB and 260 x 1 OOM'd at denoise step 1 needing a further 4.8 GB (projected 24.77 and 24.18 GB). The refused allocations are lower bounds, not peaks, so this is a classification with about 0.2 GB of margin either side, not a fitted curve. bytes_per_voxel is unchanged from #324's calibration (job 282172105da1); base_gb drops 0.3 GB now the references are counted separately. The cost entry's 42 min was that field run at 1344x768; re-priced to these 960x544 defaults it is 42 x 960*544 / (1344*768) = 21.2 (#866)" + }, + "variables": { + "character_a_portrait_prompt": "a 3/4-length studio portrait of a wiry man in his early 30s with short curly dark hair and expressive eyebrows, wearing a mustard-yellow sweater vest over a white button-down shirt, standing in a 1990s multi-camera sitcom apartment set with a sagging olive-green couch and a brick wall behind him, warm even studio lighting, sharp focus, photographic", + "character_b_portrait_prompt": "a 3/4-length studio portrait of a heavyset man in his mid 40s with a shaved head and a calm, heavy-lidded expression, wearing an open gray flannel shirt over a dark t-shirt, standing in a 1990s multi-camera sitcom apartment set with a sagging olive-green couch and a brick wall behind him, warm even studio lighting, sharp focus, photographic", + "shots": [ + { + "name": "cold_open", + "num_frames": 124, + "references": [ + { + "reference_type": "variable:subject_reference_type", + "from_previous_result": "draw_character_a" + }, + { + "reference_type": "variable:subject_reference_type", + "from_previous_result": "draw_character_b" + }, + { + "reference_type": "variable:voice_reference_type", + "from_file": "variable:character_a_voice" + }, + { + "reference_type": "variable:voice_reference_type", + "from_file": "variable:character_b_voice" + } + ], + "prompt": "subject_definitions:\n is the wiry man in , early 30s, short curly dark hair, mustard-yellow sweater vest over a white button-down shirt.\n is the heavyset man in , mid 40s, shaved head, open gray flannel shirt over a dark t-shirt.\n\nsummary:\n[reference generation] The target video is one shot of a 1990s multi-camera sitcom: reads on the couch while enters holding an empty ice cream carton and confronts him, in front of a live studio audience.\n\nretention_analysis:\n (appears in [Shot 1]): fully_preserved - the face, hair, sweater vest and proportions of are kept exactly, with the body newly animated to walk and speak.\n (appears in [Shot 1]): fully_preserved - the face, head, flannel shirt and proportions of are kept exactly, with the body newly posed seated and reading.\n\ndetailed_description:\nThe target video is in the style of a 1990s multi-camera sitcom, shot on studio video with warm, even, shadowless lighting. [Shot 1] A wide shot frames a living room set: a sagging olive-green couch centre frame in front of a brick wall with a window onto a painted city backdrop. sits at the right end of the couch, ankles crossed, reading a magazine and not looking up. strides in from frame left holding an empty pint-sized ice cream carton at chest height, stops beside the couch, holds the carton out toward and turns it upside down to show it is empty. (S1), a fast, nasal, high-pitched male tenor voice, tight with disbelief, says [English] Okay. Who finished the pistachio? keeps reading and does not react. A live studio audience laughs warmly as the carton turns over. The camera holds a static wide shot throughout, level with the set, the way a sitcom master camera holds a scene.\n\noverall_soundscape:\nA bright, close television-studio room tone. Quick footsteps on a hollow stage floor as enters, the papery rattle of the empty carton as it turns over, a soft page turn from the couch, and a warm burst of live studio-audience laughter that begins as the carton flips and settles under the end of the shot.\n\nnon_diegetic_music:\nN/A" + }, + { + "name": "deflect", + "num_frames": 124, + "references": [ + { + "reference_type": "variable:subject_reference_type", + "from_previous_result": "draw_character_b" + }, + { + "reference_type": "variable:voice_reference_type", + "from_file": "variable:character_b_voice" + } + ], + "prompt": "subject_definitions:\n is the heavyset man in , mid 40s, shaved head, calm and heavy-lidded, open gray flannel shirt over a dark t-shirt.\n\nsummary:\n[reference generation] The target video is one close-up shot of a 1990s multi-camera sitcom: answers an accusation without looking up from his magazine, in front of a live studio audience.\n\nretention_analysis:\n (appears in [Shot 1]): fully_preserved - the face, head, flannel shirt and proportions of are kept exactly, with the head and jaw newly animated to speak.\n\ndetailed_description:\nThe target video is in the style of a 1990s multi-camera sitcom, shot on studio video with warm, even, shadowless lighting. [Shot 1] A medium close-up frames from the chest up, seated at the end of an olive-green couch in front of a brick wall, holding an open magazine just below the frame line and reading it. He does not look up. (S1), a flat, low, unhurried male baritone voice, completely unbothered, says [English] Technically? The freezer has a defrost cycle. On the last word he turns a page. A live studio audience laughs as the line lands. The camera holds a static close-up throughout, the way a sitcom cross-shot camera holds a speaker.\n\noverall_soundscape:\nA bright, close television-studio room tone. A soft page turn, the faint creak of couch springs as settles, and a rolling wave of live studio-audience laughter that starts on the last word of the line and carries under the end of the shot.\n\nnon_diegetic_music:\nN/A" + }, + { + "name": "react", + "num_frames": 124, + "references": [ + { + "reference_type": "variable:subject_reference_type", + "from_previous_result": "draw_character_a" + }, + { + "reference_type": "variable:voice_reference_type", + "from_file": "variable:character_a_voice" + } + ], + "prompt": "subject_definitions:\n is the wiry man in , early 30s, short curly dark hair, expressive eyebrows, mustard-yellow sweater vest over a white button-down shirt.\n\nsummary:\n[reference generation] The target video is one close-up shot of a 1990s multi-camera sitcom: repeats an absurd excuse back in disbelief, holding up an empty ice cream carton, in front of a live studio audience.\n\nretention_analysis:\n (appears in [Shot 1]): fully_preserved - the face, hair, sweater vest and proportions of are kept exactly, with the head, jaw and arm newly animated to speak and gesture.\n\ndetailed_description:\nThe target video is in the style of a 1990s multi-camera sitcom, shot on studio video with warm, even, shadowless lighting. [Shot 1] A medium close-up frames from the chest up, standing beside a couch in front of a brick wall, an empty pint-sized ice cream carton raised in one hand at head height. His eyebrows are high and his mouth is half open before he speaks. (S1), a fast, nasal, high-pitched male tenor voice, tight with disbelief, cracking upward on the last word, says [English] The defrost cycle used a spoon? He shakes the empty carton once on the word spoon. A live studio audience erupts in a big laugh as the line lands. The camera holds a static close-up throughout, the way a sitcom cross-shot camera holds a speaker.\n\noverall_soundscape:\nA bright, close television-studio room tone. The hollow papery rattle of the empty carton as it shakes, and a big rising wave of live studio-audience laughter that breaks on the word spoon and rolls under the end of the shot.\n\nnon_diegetic_music:\nN/A" + }, + { + "name": "button", + "num_frames": 124, + "references": [ + { + "reference_type": "variable:subject_reference_type", + "from_previous_result": "draw_character_b" + }, + { + "reference_type": "variable:voice_reference_type", + "from_file": "variable:character_b_voice" + } + ], + "prompt": "subject_definitions:\n is the heavyset man in , mid 40s, shaved head, calm and heavy-lidded, open gray flannel shirt over a dark t-shirt.\n\nsummary:\n[reference generation] The target video is one close-up shot of a 1990s multi-camera sitcom: shrugs off the accusation with a deadpan line, in front of a live studio audience.\n\nretention_analysis:\n (appears in [Shot 1]): fully_preserved - the face, head, flannel shirt and proportions of are kept exactly, with the head, jaw and shoulders newly animated to speak and shrug.\n\ndetailed_description:\nThe target video is in the style of a 1990s multi-camera sitcom, shot on studio video with warm, even, shadowless lighting. [Shot 1] A medium close-up frames from the chest up, seated at the end of an olive-green couch in front of a brick wall, the magazine now closed in his lap. He looks up for the first time, holds a beat of eye contact just past the lens, and gives one slow shrug. (S1), a flat, low, unhurried male baritone voice, perfectly deadpan, says [English] I don't know its life. He goes back to the magazine. A live studio audience gives a big laugh with scattered applause as he looks back down. The camera holds a static close-up throughout, the way a sitcom cross-shot camera holds a speaker.\n\noverall_soundscape:\nA bright, close television-studio room tone. The soft slap of the magazine reopening, a couch-spring creak under the shrug, and a big wave of live studio-audience laughter with a few claps in it that breaks after the line and rolls under the end of the shot.\n\nnon_diegetic_music:\nN/A" + } + ], + "character_a_voice": null, + "character_b_voice": null, + "voice_reference_type": "diffusers.modular_pipelines.minimax_h3.MiniMaxH3AudioReference", + "subject_reference_type": "diffusers.modular_pipelines.minimax_h3.MiniMaxH3ImageReference", + "audio_bleed_ms": 1800, + "audio_bleed_gain_db": 0.0, + "seam_fade_ms": null, + "match_levels": null, + "match_levels_dbfs": null, + "width": 960, + "height": 544, + "num_inference_steps": 9, + "video_shift": 12.0, + "audio_shift": 3.0, + "weights_dtype": "{int4}", + "lora_scale": 1.0, + "lora_alpha": null, + "lora_model_name": "lightx2v/Minimax-h3-Turbo", + "lora_weight_name": "minimax_h3_ref2v_turbo_8step_v1.0_768p_bf16.safetensors", + "lora_adapter_name": "turbo", + "seed": 1166, + "attention_backend": "sparse", + "sparse_tau": 0.5, + "sparse_topk": 0.15, + "cache": null, + "music_prompt": "A slow, tense instrumental score: low bowed cello and a soft pulsing synth under sparse piano notes, restrained, cinematic, no vocals.", + "music_lyrics": "[intro]\n[instrumental]\n[solo]\n[outro]", + "audio_duration": 30, + "sample_rate": 44100, + "fps": 24, + "score_start_frame": 0, + "total_frames": 496, + "score_gain": 1.0, + "world_gain": 1.8, + "world_fade_out_ms": 0, + "target_lufs": null, + "limit": false + }, + "variable_constraints": { + "num_frames": { + "modulus": 17, + "remainder": 5, + "min_frames": 124, + "max_frames": 345, + "snap": "up", + "reason": "the video VAE encodes 17 * n + 5 frames, and MiniMax-H3 generates between 5 and 15 seconds at 24 fps" + }, + "width": { + "modulus": 32, + "remainder": 0, + "reason": "MiniMax-H3's pipeline refuses a `width` or `height` that is not a multiple of 32, and only after the weights have loaded; refused here instead, with no 'snap', since rounding would be a silent change to the size asked for" + }, + "height": { + "modulus": 32, + "remainder": 0, + "reason": "MiniMax-H3's pipeline refuses a `width` or `height` that is not a multiple of 32, and only after the weights have loaded; refused here instead, with no 'snap', since rounding would be a silent change to the size asked for" + } + }, + "seed": "variable:seed", + "steps": [ + { + "name": "draw_character_a", + "pipeline": { + "configuration": { + "component_type": "ZImagePipeline", + "offload": "model" + }, + "from_pretrained_arguments": { + "model_name": "Tongyi-MAI/Z-Image-Turbo", + "torch_dtype": "torch.bfloat16", + "low_cpu_mem_usage": true + }, + "arguments": { + "prompt": "variable:character_a_portrait_prompt", + "num_inference_steps": 9, + "guidance_scale": 0.0, + "width": 768, + "height": 768 + } + }, + "result": { + "content_type": "image/jpeg", + "embed_metadata": true, + "subfolder": "intermediate", + "save": false + } + }, + { + "name": "draw_character_b", + "release_pipeline": true, + "pipeline": { + "configuration": { + "component_type": "ZImagePipeline", + "offload": "model" + }, + "from_pretrained_arguments": { + "model_name": "Tongyi-MAI/Z-Image-Turbo", + "torch_dtype": "torch.bfloat16", + "low_cpu_mem_usage": true + }, + "arguments": { + "prompt": "variable:character_b_portrait_prompt", + "num_inference_steps": 9, + "guidance_scale": 0.0, + "width": 768, + "height": 768 + } + }, + "result": { + "content_type": "image/jpeg", + "embed_metadata": true, + "subfolder": "intermediate", + "save": false + } + }, + { + "name": "shot", + "for_each": "variable:shots", + "release_pipeline": true, + "pipeline": { + "configuration": { + "component_type": "ModularPipeline", + "pre_load_modules": [ + "sdnq" + ], + "load_components": { + "dtype": "torch.bfloat16", + "quantization_config": { + "transformer_ref": { + "configuration": { + "config_type": "sdnq.SDNQConfig" + }, + "arguments": { + "weights_dtype": "variable:weights_dtype", + "quantization_device": "cuda", + "return_device": "cpu", + "use_quantized_matmul": true, + "dequantize_fp32": false, + "modules_to_not_convert": [ + "proj_in", + "audio_proj_in", + "context_embedder", + "time_embedder", + "time_proj", + "token_refiner", + "norm_out", + "proj_out", + "audio_proj_out" + ] + } + }, + "text_encoder": { + "configuration": { + "config_type": "sdnq.SDNQConfig" + }, + "arguments": { + "weights_dtype": "variable:weights_dtype", + "quantization_device": "cuda", + "return_device": "cpu", + "dequantize_fp32": false, + "modules_to_not_convert": [ + ".model.visual", + "lm_head" + ] + } + }, + "vae": { + "configuration": { + "config_type": "sdnq.SDNQConfig" + }, + "arguments": { + "weights_dtype": "{int8}", + "quant_conv": true, + "use_quantized_matmul_conv": true, + "quantization_device": "cuda", + "return_device": "cpu", + "dequantize_fp32": false + } + } + } + }, + "components": { + "transformer_ref": { + "attention_backend": "variable:attention_backend", + "sparse_tau": "variable:sparse_tau", + "sparse_topk": "variable:sparse_topk", + "group_offload": { + "offload_type": "block_level", + "num_blocks_per_group": 1, + "use_stream": true, + "record_stream": true, + "low_cpu_mem_usage": true + } + }, + "text_encoder": { + "remove_modules": [ + "lm_head" + ] + }, + "text_encoder.model": { + "truncate_layers": { + "language_model.layers": 51 + }, + "group_offload": { + "offload_type": "leaf_level" + } + }, + "vae": { + "device": "cuda", + "residency": "on_demand" + }, + "audio_vae": { + "device": "cuda", + "residency": "on_demand" + } + }, + "cache": "variable:cache" + }, + "from_pretrained_arguments": { + "model_name": "MiniMaxAI/MiniMax-H3", + "workflow": "ref2va" + }, + "loras": [ + { + "model_name": "variable:lora_model_name", + "weight_name": "variable:lora_weight_name", + "adapter_name": "variable:lora_adapter_name", + "scale": "variable:lora_scale", + "alpha": "variable:lora_alpha" + } + ], + "scheduler": { + "shift": "variable:video_shift" + }, + "audio_scheduler": { + "shift": "variable:audio_shift" + }, + "arguments": { + "prompt": "item:prompt", + "references": "item:references", + "num_frames": "item:num_frames", + "width": "variable:width", + "height": "variable:height", + "num_inference_steps": "variable:num_inference_steps", + "output": [ + "videos", + "audio", + "sampling_rate" + ] + } + }, + "result": { + "content_type": "video/mp4", + "fps": 24, + "subfolder": "intermediate" + } + }, + { + "name": "music_bed", + "pipeline": { + "configuration": { + "component_type": "ModularPipeline", + "load_components": { + "dtype": "torch.bfloat16" + }, + "components_manager": { + "enable_auto_cpu_offload": true + } + }, + "from_pretrained_arguments": { + "model_name": "MiniMaxAI/MiniMax-Music3" + }, + "arguments": { + "prompt": "variable:music_prompt", + "lyrics": "variable:music_lyrics", + "audio_duration": "variable:audio_duration", + "output": "audios" + } + }, + "result": { + "content_type": "audio/mp3", + "sample_rate": 44100, + "save": false + }, + "release_pipeline": true + }, + { + "name": "edit", + "task": { + "command": "concat_videos", + "arguments": { + "videos": "gather:shot", + "fps": "variable:fps", + "sample_rate": "variable:sample_rate", + "audio_bleed_ms": "variable:audio_bleed_ms", + "audio_bleed_gain_db": "variable:audio_bleed_gain_db", + "seam_fade_ms": "variable:seam_fade_ms", + "match_levels": "variable:match_levels", + "match_levels_dbfs": "variable:match_levels_dbfs" + } + } + }, + { + "name": "world", + "task": { + "command": "resample_audio", + "arguments": { + "audio": "previous_result:edit", + "target_sample_rate": "variable:sample_rate" + } + } + }, + { + "name": "world_faded", + "task": { + "command": "fade_audio", + "arguments": { + "audio": "previous_result:world", + "fade_out_ms": "variable:world_fade_out_ms", + "sample_rate": "variable:sample_rate" + } + } + }, + { + "name": "soundtrack", + "task": { + "command": "slice_audio", + "arguments": { + "audio": "previous_result:music_bed", + "sample_rate": "variable:sample_rate", + "start_frame": "variable:score_start_frame", + "num_frames": "variable:total_frames", + "fps": "variable:fps" + } + } + }, + { + "name": "soundtrack_resampled", + "task": { + "command": "resample_audio", + "arguments": { + "audio": "previous_result:soundtrack", + "target_sample_rate": "variable:sample_rate" + } + } + }, + { + "name": "mixed", + "task": { + "command": "mix_audio", + "arguments": { + "audios": [ + "previous_result:soundtrack_resampled", + "previous_result:world_faded" + ], + "gains": [ + "variable:score_gain", + "variable:world_gain" + ], + "sample_rate": "variable:sample_rate" + } + } + }, + { + "name": "balanced", + "task": { + "command": "normalize_audio", + "arguments": { + "audio": "previous_result:mixed", + "peak_dbfs": -3.0, + "target_lufs": "variable:target_lufs", + "limit": "variable:limit", + "sample_rate": "variable:sample_rate" + } + } + }, + { + "name": "film", + "task": { + "command": "pair_audio", + "arguments": { + "video": "previous_result:edit", + "audio": "previous_result:balanced", + "sample_rate": "variable:sample_rate", + "fit": "video" + } + }, + "result": { + "content_type": "video/mp4", + "fps": "variable:fps", + "subfolder": "final" + } + } + ] +} diff --git a/bench/workflows/s5-ltx-action.json b/bench/workflows/s5-ltx-action.json new file mode 100644 index 000000000..0ebd3b535 --- /dev/null +++ b/bench/workflows/s5-ltx-action.json @@ -0,0 +1,190 @@ +{ + "id": "BenchS5LtxAction", + "description": "Long-form bench S5 (LTX action): templates/ltx2/chained-segments, three 121-frame segments of one continuous silent action (a courier cycling along a coastal road, from asset:bench/runner.png), chained by last frame. No speech anywhere; the sound is wind, sea and the bicycle. Task 23 (LTX long takes) added `duration_seconds`, `look_hold`, `check` and `retries`, all off by default, so the control is unchanged.", + "summary": "A long LTX-2.5 clip with audio, chained from short segments that each start on the previous one's last frame.", + "cost_drivers": [ + "segments", + "duration_seconds", + "num_frames", + "width", + "height" + ], + "variables": { + "transformer_weights_dtype": "{uint4}", + "text_encoder_weights_dtype": "{int8}", + "prompt": "The courier, a man in a blue shirt and black helmet with a yellow delivery box on the back, is pedaling steadily along the coastal road toward the right of frame, leaning slightly into the wind, the sea glittering beyond the low stone wall behind him as the camera tracks alongside at his speed. The only sounds are the whir of the chain, the hum of tires on asphalt and the steady rush of the wind and surf, with no speech and no music.", + "prompts": [ + "The courier, a man in a blue shirt and black helmet with a yellow delivery box on the back, is pedaling steadily along the coastal road toward the right of frame, leaning slightly into the wind, the sea glittering beyond the low stone wall behind him as the camera tracks alongside at his speed. The only sounds are the whir of the chain, the hum of tires on asphalt and the steady rush of the wind and surf, with no speech and no music.", + "The courier keeps pedaling at the same pace, the road running on along the shoreline, waves rolling in beyond the stone wall as the camera continues to track alongside at his speed. The only sounds are the whir of the chain, tires on asphalt and the wind and surf, with no speech and no music.", + "The courier pedals on without slowing, the road running straight ahead along the water, gulls wheeling over the waves as the camera continues to track alongside at his speed. The only sounds are the whir of the chain, tires on asphalt and the wind and surf, with no speech and no music." + ], + "negative_prompt": "constant:diffusers.pipelines.ltx2.utils.DEFAULT_NEGATIVE_PROMPT", + "width": 960, + "height": 544, + "frame_rate": 24.0, + "num_frames": 121, + "segments": 3, + "continuity": "last_frame", + "image": "asset:bench/runner.png", + "seed": 1166, + "duration_seconds": null, + "look_hold": null, + "check": false, + "retries": 0 + }, + "variable_constraints": { + "width": { + "modulus": 32, + "remainder": 0, + "reason": "LTX-2.5's pipelines refuse a width not divisible by 32 outright (ValueError, not a silent floor), so it is checked here before the chain's first segment load rather than after it" + }, + "height": { + "modulus": 32, + "remainder": 0, + "reason": "LTX-2.5's pipelines refuse a height not divisible by 32 outright (ValueError, not a silent floor), so it is checked here before the chain's first segment load rather than after it" + }, + "num_frames": { + "modulus": 8, + "remainder": 1, + "min_frames": 9, + "reason": "LTX-2.5's video VAE encodes 8 * n + 1 frames, so an off-grid count is not refused by the pipeline but floored to the grid below - a shorter clip than the one asked for. Refused here instead: no 'snap', because rounding a length the other way from the pipeline's own would be a second silent change" + } + }, + "seed": "variable:seed", + "steps": [ + { + "name": "chained_image_to_video", + "pipeline": { + "configuration": { + "component_type": "LTX2ImageToVideoPipeline", + "pre_load_modules": [ + "sdnq" + ], + "components": { + "transformer": { + "device": "cuda" + }, + "text_encoder": { + "group_offload": { + "offload_type": "leaf_level", + "use_stream": true + } + }, + "connectors": { + "group_offload": { + "offload_type": "leaf_level", + "use_stream": true + } + }, + "vae": { + "device": "cuda" + }, + "audio_vae": { + "device": "cuda" + }, + "vocoder": { + "device": "cuda" + }, + "duration_head": { + "device": "cuda" + } + }, + "vae": { + "enable_tiling": true + } + }, + "from_pretrained_arguments": { + "model_name": "Lightricks/LTX-2.5-Diffusers", + "torch_dtype": "torch.bfloat16" + }, + "transformer": { + "configuration": { + "component_type": "LTX2VideoTransformer3DModel", + "preserve_device_placement": true + }, + "quantization_config": { + "configuration": { + "config_type": "sdnq.SDNQConfig" + }, + "arguments": { + "weights_dtype": "variable:transformer_weights_dtype", + "quantization_device": "cuda", + "return_device": "cuda", + "use_quantized_matmul": true, + "dequantize_fp32": false + } + }, + "from_pretrained_arguments": { + "model_name": "Lightricks/LTX-2.5-Diffusers", + "subfolder": "transformer", + "torch_dtype": "torch.bfloat16" + } + }, + "text_encoder": { + "configuration": { + "component_type": "transformers.Gemma4UnifiedForConditionalGeneration", + "preserve_device_placement": true + }, + "quantization_config": { + "configuration": { + "config_type": "sdnq.SDNQConfig" + }, + "arguments": { + "weights_dtype": "variable:text_encoder_weights_dtype", + "quantization_device": "cuda", + "return_device": "cpu", + "dequantize_fp32": false, + "modules_to_not_convert": [ + "embed_vision", + "embed_audio", + "lm_head" + ] + } + }, + "from_pretrained_arguments": { + "model_name": "Lightricks/LTX-2.5-Diffusers", + "subfolder": "text_encoder", + "torch_dtype": "torch.bfloat16" + } + }, + "chain": { + "segments": "variable:segments", + "duration_seconds": "variable:duration_seconds", + "prompts": "variable:prompts", + "continuity": "variable:continuity", + "segment_argument": "image", + "trim_frames": 2, + "crossfade_ms": 80, + "save_segments": true, + "fps": 24, + "frame_snap": "constraint:num_frames", + "look_hold": "variable:look_hold", + "check": "variable:check", + "retries": "variable:retries" + }, + "arguments": { + "prompt": "variable:prompt", + "negative_prompt": "variable:negative_prompt", + "image": "variable:image", + "width": "variable:width", + "height": "variable:height", + "num_frames": "variable:num_frames", + "frame_rate": "variable:frame_rate", + "sigmas": "constant:diffusers.pipelines.ltx2.utils.DISTILLED_SIGMA_VALUES", + "guidance_scale": 1.0, + "audio_guidance_scale": 1.0, + "stg_scale": 0.0, + "audio_stg_scale": 0.0, + "modality_scale": 1.0, + "audio_modality_scale": 1.0, + "output_type": "{np}" + } + }, + "result": { + "content_type": "video/mp4", + "fps": "variable:frame_rate", + "subfolder": "final" + } + } + ] +} diff --git a/bench/workflows/s5-ltx-extend.json b/bench/workflows/s5-ltx-extend.json new file mode 100644 index 000000000..9b0b8273f --- /dev/null +++ b/bench/workflows/s5-ltx-extend.json @@ -0,0 +1,449 @@ +{ + "id": "BenchS5LtxExtend", + "description": "Long-form bench S5 extended rather than chained by last frame (Task 23): segment 1 is the control's own image-to-video segment (same pipeline, prompt, seed); each later segment is one LTX2ConditionPipeline pass of 241 frames conditioned at index 0 on the whole previous 121-frame segment (extend-clip's shape), and only its new 120 frames are kept. The take is the three 121/120/120-frame pieces joined with continuous_joins (no frames trimmed, no crossfade: the pieces tile one timeline), so analyze_seams reads seams at the same places as the chain's. Each kept piece's audio is the pass that rendered its picture. Prompts: prompt_1..prompt_3, the S5 chain prompts.", + "summary": "S5 as extend-clip passes, each conditioned on the whole previous segment.", + "cost_drivers": [ + "num_frames", + "extend_frames", + "width", + "height" + ], + "variables": { + "transformer_weights_dtype": "{uint4}", + "text_encoder_weights_dtype": "{int8}", + "prompt_1": "The courier, a man in a blue shirt and black helmet with a yellow delivery box on the back, is pedaling steadily along the coastal road toward the right of frame, leaning slightly into the wind, the sea glittering beyond the low stone wall behind him as the camera tracks alongside at his speed. The only sounds are the whir of the chain, the hum of tires on asphalt and the steady rush of the wind and surf, with no speech and no music.", + "prompt_2": "The courier keeps pedaling at the same pace, the road running on along the shoreline, waves rolling in beyond the stone wall as the camera continues to track alongside at his speed. The only sounds are the whir of the chain, tires on asphalt and the wind and surf, with no speech and no music.", + "prompt_3": "The courier pedals on without slowing, the road running straight ahead along the water, gulls wheeling over the waves as the camera continues to track alongside at his speed. The only sounds are the whir of the chain, tires on asphalt and the wind and surf, with no speech and no music.", + "negative_prompt": "constant:diffusers.pipelines.ltx2.utils.DEFAULT_NEGATIVE_PROMPT", + "width": 960, + "height": 544, + "frame_rate": 24.0, + "num_frames": 121, + "extend_frames": 241, + "image": "asset:bench/runner.png", + "seed": 1166 + }, + "variable_constraints": { + "width": { + "modulus": 32, + "remainder": 0, + "reason": "LTX-2.5's pipelines refuse a width not divisible by 32 outright (ValueError, not a silent floor), so it is checked here before the chain's first segment load rather than after it" + }, + "height": { + "modulus": 32, + "remainder": 0, + "reason": "LTX-2.5's pipelines refuse a height not divisible by 32 outright (ValueError, not a silent floor), so it is checked here before the chain's first segment load rather than after it" + }, + "num_frames": { + "modulus": 8, + "remainder": 1, + "min_frames": 9, + "reason": "LTX-2.5's video VAE encodes 8 * n + 1 frames, so an off-grid count is not refused by the pipeline but floored to the grid below - a shorter clip than the one asked for. Refused here instead: no 'snap', because rounding a length the other way from the pipeline's own would be a second silent change" + } + }, + "seed": "variable:seed", + "steps": [ + { + "name": "seg1", + "release_pipeline": true, + "pipeline": { + "configuration": { + "component_type": "LTX2ImageToVideoPipeline", + "pre_load_modules": [ + "sdnq" + ], + "components": { + "transformer": { + "device": "cuda" + }, + "text_encoder": { + "group_offload": { + "offload_type": "leaf_level", + "use_stream": true + } + }, + "connectors": { + "group_offload": { + "offload_type": "leaf_level", + "use_stream": true + } + }, + "vae": { + "device": "cuda" + }, + "audio_vae": { + "device": "cuda" + }, + "vocoder": { + "device": "cuda" + }, + "duration_head": { + "device": "cuda" + } + }, + "vae": { + "enable_tiling": true + } + }, + "from_pretrained_arguments": { + "model_name": "Lightricks/LTX-2.5-Diffusers", + "torch_dtype": "torch.bfloat16" + }, + "transformer": { + "configuration": { + "component_type": "LTX2VideoTransformer3DModel", + "preserve_device_placement": true + }, + "quantization_config": { + "configuration": { + "config_type": "sdnq.SDNQConfig" + }, + "arguments": { + "weights_dtype": "variable:transformer_weights_dtype", + "quantization_device": "cuda", + "return_device": "cuda", + "use_quantized_matmul": true, + "dequantize_fp32": false + } + }, + "from_pretrained_arguments": { + "model_name": "Lightricks/LTX-2.5-Diffusers", + "subfolder": "transformer", + "torch_dtype": "torch.bfloat16" + } + }, + "text_encoder": { + "configuration": { + "component_type": "transformers.Gemma4UnifiedForConditionalGeneration", + "preserve_device_placement": true + }, + "quantization_config": { + "configuration": { + "config_type": "sdnq.SDNQConfig" + }, + "arguments": { + "weights_dtype": "variable:text_encoder_weights_dtype", + "quantization_device": "cuda", + "return_device": "cpu", + "dequantize_fp32": false, + "modules_to_not_convert": [ + "embed_vision", + "embed_audio", + "lm_head" + ] + } + }, + "from_pretrained_arguments": { + "model_name": "Lightricks/LTX-2.5-Diffusers", + "subfolder": "text_encoder", + "torch_dtype": "torch.bfloat16" + } + }, + "arguments": { + "prompt": "variable:prompt_1", + "negative_prompt": "variable:negative_prompt", + "image": "variable:image", + "width": "variable:width", + "height": "variable:height", + "num_frames": "variable:num_frames", + "frame_rate": "variable:frame_rate", + "sigmas": "constant:diffusers.pipelines.ltx2.utils.DISTILLED_SIGMA_VALUES", + "guidance_scale": 1.0, + "audio_guidance_scale": 1.0, + "stg_scale": 0.0, + "audio_stg_scale": 0.0, + "modality_scale": 1.0, + "audio_modality_scale": 1.0, + "output_type": "{np}" + } + }, + "result": { + "content_type": "video/mp4", + "fps": "variable:frame_rate", + "save": false + } + }, + { + "name": "seg1_frames", + "task": { + "command": "video_frames", + "arguments": { + "video": "previous_result:seg1" + } + }, + "result": { + "content_type": "video/mp4", + "fps": "variable:frame_rate", + "save": false + } + }, + { + "name": "seg2", + "pipeline": { + "configuration": { + "component_type": "LTX2ConditionPipeline", + "pre_load_modules": [ + "sdnq" + ], + "components": { + "transformer": { + "device": "cuda" + }, + "text_encoder": { + "group_offload": { + "offload_type": "leaf_level", + "use_stream": true + } + }, + "connectors": { + "group_offload": { + "offload_type": "leaf_level", + "use_stream": true + } + }, + "vae": { + "device": "cuda" + }, + "audio_vae": { + "device": "cuda" + }, + "vocoder": { + "device": "cuda" + }, + "duration_head": { + "device": "cuda" + } + }, + "vae": { + "enable_tiling": true + } + }, + "from_pretrained_arguments": { + "model_name": "Lightricks/LTX-2.5-Diffusers", + "torch_dtype": "torch.bfloat16" + }, + "transformer": { + "configuration": { + "component_type": "LTX2VideoTransformer3DModel", + "preserve_device_placement": true + }, + "quantization_config": { + "configuration": { + "config_type": "sdnq.SDNQConfig" + }, + "arguments": { + "weights_dtype": "variable:transformer_weights_dtype", + "quantization_device": "cuda", + "return_device": "cuda", + "use_quantized_matmul": true, + "dequantize_fp32": false + } + }, + "from_pretrained_arguments": { + "model_name": "Lightricks/LTX-2.5-Diffusers", + "subfolder": "transformer", + "torch_dtype": "torch.bfloat16" + } + }, + "text_encoder": { + "configuration": { + "component_type": "transformers.Gemma4UnifiedForConditionalGeneration", + "preserve_device_placement": true + }, + "quantization_config": { + "configuration": { + "config_type": "sdnq.SDNQConfig" + }, + "arguments": { + "weights_dtype": "variable:text_encoder_weights_dtype", + "quantization_device": "cuda", + "return_device": "cpu", + "dequantize_fp32": false, + "modules_to_not_convert": [ + "embed_vision", + "embed_audio", + "lm_head" + ] + } + }, + "from_pretrained_arguments": { + "model_name": "Lightricks/LTX-2.5-Diffusers", + "subfolder": "text_encoder", + "torch_dtype": "torch.bfloat16" + } + }, + "arguments": { + "prompt": "variable:prompt_2", + "negative_prompt": "variable:negative_prompt", + "width": "variable:width", + "height": "variable:height", + "num_frames": "variable:extend_frames", + "frame_rate": "variable:frame_rate", + "sigmas": "constant:diffusers.pipelines.ltx2.utils.DISTILLED_SIGMA_VALUES", + "guidance_scale": 1.0, + "audio_guidance_scale": 1.0, + "stg_scale": 0.0, + "audio_stg_scale": 0.0, + "modality_scale": 1.0, + "audio_modality_scale": 1.0, + "output_type": "{np}", + "conditions": [ + { + "condition_type": "diffusers.pipelines.ltx2.pipeline_ltx2_condition.LTX2VideoCondition", + "from_arguments": { + "frames": "previous_result:seg1_frames", + "index": 0, + "strength": 1.0 + } + } + ] + } + }, + "result": { + "content_type": "video/mp4", + "fps": "variable:frame_rate", + "save": false + } + }, + { + "name": "seg2_tail", + "task": { + "command": "trim_video", + "arguments": { + "video": "previous_result:seg2", + "start_frame": 120, + "num_frames": 121, + "fps": "variable:frame_rate" + } + }, + "result": { + "content_type": "video/mp4", + "fps": "variable:frame_rate", + "save": false + } + }, + { + "name": "seg2_tail_frames", + "task": { + "command": "video_frames", + "arguments": { + "video": "previous_result:seg2_tail" + } + }, + "result": { + "content_type": "video/mp4", + "fps": "variable:frame_rate", + "save": false + } + }, + { + "name": "seg3", + "pipeline_reference": { + "reference_name": "seg2", + "arguments": { + "prompt": "variable:prompt_3", + "negative_prompt": "variable:negative_prompt", + "width": "variable:width", + "height": "variable:height", + "num_frames": "variable:extend_frames", + "frame_rate": "variable:frame_rate", + "sigmas": "constant:diffusers.pipelines.ltx2.utils.DISTILLED_SIGMA_VALUES", + "guidance_scale": 1.0, + "audio_guidance_scale": 1.0, + "stg_scale": 0.0, + "audio_stg_scale": 0.0, + "modality_scale": 1.0, + "audio_modality_scale": 1.0, + "output_type": "{np}", + "conditions": [ + { + "condition_type": "diffusers.pipelines.ltx2.pipeline_ltx2_condition.LTX2VideoCondition", + "from_arguments": { + "frames": "previous_result:seg2_tail_frames", + "index": 0, + "strength": 1.0 + } + } + ] + } + }, + "result": { + "content_type": "video/mp4", + "fps": "variable:frame_rate", + "save": false + } + }, + { + "name": "piece_1", + "task": { + "command": "trim_video", + "arguments": { + "video": "previous_result:seg2", + "start_frame": 0, + "num_frames": 121, + "fps": "variable:frame_rate" + } + }, + "result": { + "content_type": "video/mp4", + "fps": "variable:frame_rate", + "save": false + } + }, + { + "name": "piece_2", + "task": { + "command": "trim_video", + "arguments": { + "video": "previous_result:seg2", + "start_frame": 121, + "num_frames": 120, + "fps": "variable:frame_rate" + } + }, + "result": { + "content_type": "video/mp4", + "fps": "variable:frame_rate", + "save": false + } + }, + { + "name": "piece_3", + "task": { + "command": "trim_video", + "arguments": { + "video": "previous_result:seg3", + "start_frame": 121, + "num_frames": 120, + "fps": "variable:frame_rate" + } + }, + "result": { + "content_type": "video/mp4", + "fps": "variable:frame_rate", + "save": false + } + }, + { + "name": "take", + "task": { + "command": "concat_videos", + "arguments": { + "videos": [ + "previous_result:piece_1", + "previous_result:piece_2", + "previous_result:piece_3" + ], + "trim_frames": 0, + "fps": "variable:frame_rate", + "continuous_joins": true + } + }, + "result": { + "content_type": "video/mp4", + "fps": "variable:frame_rate", + "subfolder": "final" + } + } + ] +} diff --git a/bench/workflows/s6-narration.json b/bench/workflows/s6-narration.json new file mode 100644 index 000000000..4c78bc033 --- /dev/null +++ b/bench/workflows/s6-narration.json @@ -0,0 +1,180 @@ +{ + "id": "BenchS6Narration", + "summary": "Three narration takes and a Music 3 bed under three animatic boards.", + "description": "Long-form bench S6 (narration): three templates/generate-speech narration takes in the default voice (no consented cast voice clip is on the bench host, so templates/voice-clone is not used), a templates/minimax/music instrumental bed, and three storyboard stills held by templates/animatic. Narration and bed are mixed and paired under the boards.", + "cost_drivers": [ + "audio_duration" + ], + "variables": { + "take_1": "The keeper climbed the stairs every evening, long before the storm.", + "take_2": "By the time he reached the lamp, the sea had already gone dark.", + "take_3": "Then the light turned, and the whole coast could see him.", + "speech_model": "facebook/mms-tts-eng", + "music_prompt": "A slow, spare instrumental score: soft piano and a low sustained string pad, patient and melancholy, no vocals.", + "music_lyrics": "[intro]\n[instrumental]\n[outro]", + "audio_duration": 30, + "sample_rate": 44100, + "boards": [ + "asset:bench/board_1.png", + "asset:bench/board_2.png", + "asset:bench/board_3.png" + ], + "durations": [ + 4.0, + 4.0, + 4.0 + ], + "fps": 24.0, + "narration_gain": 1.0, + "bed_gain": 0.35, + "seed": 1166 + }, + "seed": "variable:seed", + "steps": [ + { + "name": "narrate_1", + "task": { + "command": "generate_speech", + "arguments": { + "text": "variable:take_1", + "model_name": "variable:speech_model" + } + } + }, + { + "name": "narrate_2", + "task": { + "command": "generate_speech", + "arguments": { + "text": "variable:take_2", + "model_name": "variable:speech_model" + } + } + }, + { + "name": "narrate_3", + "task": { + "command": "generate_speech", + "arguments": { + "text": "variable:take_3", + "model_name": "variable:speech_model" + } + } + }, + { + "name": "narration", + "task": { + "command": "crossfade_audio", + "arguments": { + "audios": [ + "previous_result:narrate_1", + "previous_result:narrate_2", + "previous_result:narrate_3" + ], + "crossfade_ms": 75, + "sample_rate": "variable:sample_rate" + } + } + }, + { + "name": "narration_resampled", + "task": { + "command": "resample_audio", + "arguments": { + "audio": "previous_result:narration", + "target_sample_rate": "variable:sample_rate" + } + } + }, + { + "name": "generate_music", + "pipeline": { + "configuration": { + "component_type": "ModularPipeline", + "load_components": { + "dtype": "torch.bfloat16" + }, + "components_manager": { + "enable_auto_cpu_offload": true + } + }, + "from_pretrained_arguments": { + "model_name": "MiniMaxAI/MiniMax-Music3" + }, + "arguments": { + "prompt": "variable:music_prompt", + "lyrics": "variable:music_lyrics", + "audio_duration": "variable:audio_duration", + "output": "audios" + } + }, + "result": { + "content_type": "audio/mp3", + "sample_rate": 44100, + "save": false + }, + "release_pipeline": true + }, + { + "name": "balanced", + "task": { + "command": "normalize_audio", + "arguments": { + "audio": "previous_result:generate_music", + "peak_dbfs": -3.0, + "sample_rate": 44100 + } + }, + "result": { + "content_type": "audio/mp3", + "sample_rate": 44100, + "save": false + } + }, + { + "name": "soundtrack", + "task": { + "command": "mix_audio", + "arguments": { + "audios": [ + "previous_result:narration_resampled", + "previous_result:balanced" + ], + "gains": [ + "variable:narration_gain", + "variable:bed_gain" + ], + "sample_rate": "variable:sample_rate" + } + } + }, + { + "name": "stills", + "task": { + "command": "stills_to_video", + "arguments": { + "images": "variable:boards", + "durations": "variable:durations", + "fps": "variable:fps" + } + } + }, + { + "name": "film", + "task": { + "command": "pair_audio", + "arguments": { + "video": "previous_result:stills", + "audio": "previous_result:soundtrack", + "fps": "variable:fps", + "fit": "video" + } + }, + "result": { + "content_type": "video/mp4", + "fps": "variable:fps", + "subfolder": "final" + } + } + ] +} diff --git a/bench/workflows/t16-performer-swap.json b/bench/workflows/t16-performer-swap.json new file mode 100644 index 000000000..b509b7805 --- /dev/null +++ b/bench/workflows/t16-performer-swap.json @@ -0,0 +1,184 @@ +{ + "id": "BenchT16PerformerSwap", + "summary": "One S4 cut, its performer replaced by a masked ref2va edit.", + "description": "Task 16 (#1194) spike, step 1: performer replacement on one cut of S4 (the 'deflect' close-up of the shaved-head man). The step is S4's own ref2va shot pipeline (int4, ref2v turbo 8-step LoRA, 9 steps, video_shift 12, 960x544x124) given the cut as 'source_video', a per-frame person mask from BenchT16PersonMask as 'mask_video' and the cut itself as 'hold_audio', at refine_strength 1.0: inside the mask is a fresh generation, and the identity comes from the replacement 'portrait' as . The prompt is the cut's own prompt with the subject rewritten to the portrait's man. Outside the mask the engine writes the source latent back every step.", + "variables": { + "source_video": "output:BenchS4Montage/20261104-040255-9813688b/intermediate/BenchS4Montage-shot@deflect.3-0.0.mp4", + "mask_video": "output:BenchT16PersonMask/20261107-001508-24196385/final/mask-0.0.mp4", + "portrait": "asset:bench/t26-stranger.jpg", + "prompt": "subject_definitions:\n is the man in , early 60s, short greying dark hair, round wire-rimmed glasses, a full grey-and-dark beard, wearing a mustard corduroy jacket over a dark green crew-neck t-shirt.\n\nsummary:\n[reference generation] The target video is one close-up shot of a 1990s multi-camera sitcom: answers an accusation without looking up from his magazine, in front of a live studio audience.\n\nretention_analysis:\n (appears in [Shot 1]): fully_preserved - the face, glasses, beard, hair, corduroy jacket and proportions of are kept exactly, with the head and jaw newly animated to speak; the grey studio backdrop of is not used.\n\ndetailed_description:\nThe target video is in the style of a 1990s multi-camera sitcom, shot on studio video with warm, even, shadowless lighting. [Shot 1] A medium close-up frames from the chest up, seated at the end of an olive-green couch in front of a brick wall, holding an open magazine just below the frame line and reading it, his glasses low on his nose. He does not look up. (S1), a flat, low, unhurried male baritone voice, completely unbothered, says [English] Technically? The freezer has a defrost cycle. On the last word he turns a page. A live studio audience laughs as the line lands. The camera holds a static close-up throughout, the way a sitcom cross-shot camera holds a speaker.\n\noverall_soundscape:\nA bright, close television-studio room tone. A soft page turn, the faint creak of couch springs as settles, and a rolling wave of live studio-audience laughter that starts on the last word of the line and carries under the end of the shot.\n\nnon_diegetic_music:\nN/A", + "refine_strength": 1.0, + "mask_grow": 1, + "num_frames": 124, + "width": 960, + "height": 544, + "num_inference_steps": 9, + "video_shift": 12.0, + "audio_shift": 3.0, + "weights_dtype": "{int4}", + "lora_scale": 1.0, + "lora_alpha": null, + "lora_model_name": "lightx2v/Minimax-h3-Turbo", + "lora_weight_name": "minimax_h3_ref2v_turbo_8step_v1.0_768p_bf16.safetensors", + "lora_adapter_name": "turbo", + "attention_backend": "sparse", + "sparse_tau": 0.5, + "sparse_topk": 0.15, + "cache": null, + "seed": 1194 + }, + "seed": "variable:seed", + "steps": [ + { + "name": "swap", + "pipeline": { + "configuration": { + "component_type": "ModularPipeline", + "pre_load_modules": [ + "sdnq" + ], + "load_components": { + "dtype": "torch.bfloat16", + "quantization_config": { + "transformer_ref": { + "configuration": { + "config_type": "sdnq.SDNQConfig" + }, + "arguments": { + "weights_dtype": "variable:weights_dtype", + "quantization_device": "cuda", + "return_device": "cpu", + "use_quantized_matmul": true, + "dequantize_fp32": false, + "modules_to_not_convert": [ + "proj_in", + "audio_proj_in", + "context_embedder", + "time_embedder", + "time_proj", + "token_refiner", + "norm_out", + "proj_out", + "audio_proj_out" + ] + } + }, + "text_encoder": { + "configuration": { + "config_type": "sdnq.SDNQConfig" + }, + "arguments": { + "weights_dtype": "variable:weights_dtype", + "quantization_device": "cuda", + "return_device": "cpu", + "dequantize_fp32": false, + "modules_to_not_convert": [ + ".model.visual", + "lm_head" + ] + } + }, + "vae": { + "configuration": { + "config_type": "sdnq.SDNQConfig" + }, + "arguments": { + "weights_dtype": "{int8}", + "quant_conv": true, + "use_quantized_matmul_conv": true, + "quantization_device": "cuda", + "return_device": "cpu", + "dequantize_fp32": false + } + } + } + }, + "components": { + "transformer_ref": { + "attention_backend": "variable:attention_backend", + "sparse_tau": "variable:sparse_tau", + "sparse_topk": "variable:sparse_topk", + "group_offload": { + "offload_type": "block_level", + "num_blocks_per_group": 1, + "use_stream": true, + "record_stream": true, + "low_cpu_mem_usage": true + } + }, + "text_encoder": { + "remove_modules": [ + "lm_head" + ] + }, + "text_encoder.model": { + "truncate_layers": { + "language_model.layers": 51 + }, + "group_offload": { + "offload_type": "leaf_level" + } + }, + "vae": { + "device": "cuda", + "residency": "on_demand" + }, + "audio_vae": { + "device": "cuda", + "residency": "on_demand" + } + }, + "cache": "variable:cache" + }, + "from_pretrained_arguments": { + "model_name": "MiniMaxAI/MiniMax-H3", + "workflow": "ref2va" + }, + "loras": [ + { + "model_name": "variable:lora_model_name", + "weight_name": "variable:lora_weight_name", + "adapter_name": "variable:lora_adapter_name", + "scale": "variable:lora_scale", + "alpha": "variable:lora_alpha" + } + ], + "scheduler": { + "shift": "variable:video_shift" + }, + "audio_scheduler": { + "shift": "variable:audio_shift" + }, + "arguments": { + "prompt": "variable:prompt", + "references": [ + { + "reference_type": "diffusers.modular_pipelines.minimax_h3.MiniMaxH3ImageReference", + "from_file": "variable:portrait" + } + ], + "source_video": "variable:source_video", + "mask_video": "variable:mask_video", + "mask_grow": "variable:mask_grow", + "hold_audio": "variable:source_video", + "refine_strength": "variable:refine_strength", + "num_frames": "variable:num_frames", + "width": "variable:width", + "height": "variable:height", + "num_inference_steps": "variable:num_inference_steps", + "output": [ + "videos", + "audio", + "sampling_rate" + ] + } + }, + "result": { + "content_type": "video/mp4", + "fps": 24, + "subfolder": "final", + "file_base_name": "swap" + } + } + ] +} \ No newline at end of file diff --git a/bench/workflows/t16-person-mask.json b/bench/workflows/t16-person-mask.json new file mode 100644 index 000000000..e65ff7286 --- /dev/null +++ b/bench/workflows/t16-person-mask.json @@ -0,0 +1,39 @@ +{ + "id": "BenchT16PersonMask", + "summary": "A per-frame person mask of one cut: segment (GroundingDINO + SAM2) on every frame.", + "description": "Task 16 (#1194) spike, step 1: the per-frame person mask a performer replacement needs. 'video' (an output: reference to one cut) is read as frames, and segment runs on each frame with the text prompt 'person', giving one mask frame per source frame (white is the person) at the source's size and rate, for a regional edit's mask_video. GroundingDINO base and SAM2 hiera-large are both Apache-2.0. Detection runs per frame with no tracking, so a frame where the detector misses comes back black.", + "variables": { + "video": "output:BenchS4Montage/20261104-040255-9813688b/intermediate/BenchS4Montage-shot@deflect.3-0.0.mp4", + "prompt": "person", + "threshold": 0.3, + "seed": 0 + }, + "steps": [ + { + "name": "frames", + "task": { + "command": "video_frames", + "arguments": { + "video": "variable:video" + } + } + }, + { + "name": "mask", + "task": { + "command": "segment", + "arguments": { + "image": "previous_result:frames", + "prompt": "variable:prompt", + "threshold": "variable:threshold" + } + }, + "result": { + "content_type": "video/mp4", + "fps": 24, + "subfolder": "final", + "file_base_name": "mask" + } + } + ] +} diff --git a/bench/workflows/t20-template-static-long.json b/bench/workflows/t20-template-static-long.json new file mode 100644 index 000000000..65d4d4076 --- /dev/null +++ b/bench/workflows/t20-template-static-long.json @@ -0,0 +1,33 @@ +{ + "id": "BenchT20TemplateStaticLong", + "summary": "templates/minimax/chained-segments as shipped, shot static_long, on S1's still and prompt.", + "description": "Task 20 (#1199): the catalog template templates/minimax/chained-segments composed by its catalog name, not a bench copy, with S1's still and prompt, 'shot' (default static_long: guide + keystone segment_1 at keystone_slot last, anchor off), 'duration_seconds' (default 120) and 'seed' set and everything else the template's own defaults, so the GPU run confirms the template path end to end and records its wall time.", + "variables": { + "prompt": "For the target video, at 0.00 seconds into the target video, (from [Shot 1]) is fully referenced.\n\nintegrated_multimodal_description: [Shot 1] Cinematic, live-action, one continuous static shot. The shot opens exactly on : a young woman in a plain dark grey sweater stands centered in front of a flat painted wall of bold diagonal rainbow stripes with a black circle at upper left and a teal square at lower right. She breathes calmly, blinks, shifts her weight slightly and turns her head a little left and right, staying in place at the center of the frame. The painted wall stays exactly as it is, evenly lit by soft studio light. The camera is locked off on a tripod and never moves.\n\noverall_soundscape: Quiet studio room tone, the soft rustle of a wool sweater, calm breathing.\n\nnon_diegetic_music: None.", + "image": "asset:bench/static-stripes.jpg", + "shot": "static_long", + "duration_seconds": 120, + "seed": 1166 + }, + "seed": "variable:seed", + "steps": [ + { + "name": "take", + "workflow": { + "path": "templates/minimax/chained-segments", + "arguments": { + "prompt": "variable:prompt", + "image": "variable:image", + "shot": "variable:shot", + "duration_seconds": "variable:duration_seconds", + "seed": "variable:seed" + } + }, + "result": { + "content_type": "video/mp4", + "fps": 24, + "subfolder": "final" + } + } + ] +} diff --git a/bench/workflows/t26-continued.json b/bench/workflows/t26-continued.json new file mode 100644 index 000000000..3859ec517 --- /dev/null +++ b/bench/workflows/t26-continued.json @@ -0,0 +1,532 @@ +{ + "id": "BenchT26Continued", + "summary": "A two-segment Ref2VA take: segment 2 with and without the portrait, for #1193.", + "description": "Task 26 (#1193) claim 4: on a continued shot, leave the portrait out when the carried tail already shows the person. templates/minimax/chain-video-continuity cannot drop a reference after segment 1 (its references are the same on every segment), so this builds its seam by hand from the same ref2va step: segment 1 from the portrait, its last 48 frames (the template's carry_frames) (frames and audio, trim_video) as a video reference, and segment 2 twice at the same seed - 'seg2_portrait' with the portrait and the tail (the template's behaviour), 'seg2_carry_only' with the tail alone. analyze_face_drift scores each segment against the portrait. No audio reference: the claim is about the picture.", + "variables": { + "seg1_prompt": "subject_definitions:\n is the woman in : early thirties, shoulder-length dark hair, a long olive double-breasted coat.\n\nsummary:\n[reference generation] The target video shows walking alone toward the camera down a quiet cobblestone street at golden hour.\n\nretention_analysis:\n (appears in [Shot 1]): fully_preserved - her face, hair, coat and proportions from are kept, newly animated to walk.\n\ndetailed_description:\nThe target video is in realistic photographic style. [Shot 1] A medium shot at eye level frames from the knees up, centred, as she walks slowly toward the camera down an empty cobblestone street lined with pale shopfronts, her hands in her coat pockets, glancing up at the street lamps as she goes. Low golden sunlight comes from behind her and rims her hair. The camera tracks backward at her pace, keeping her the same size in frame, as one continuous take.\n\noverall_soundscape:\nHer footsteps on the cobblestones, a light breeze, faint distant street ambience.\n\nnon_diegetic_music:\nNone.", + "seg2_portrait_prompt": "subject_definitions:\n is the woman in : early thirties, shoulder-length dark hair, a long olive double-breasted coat.\n