Repository navigation
Fix black preview for pictures an agent places on the Editor timeline - #14
Open
anessbelbati wants to merge 3 commits into
Open
anessbelbati wants to merge 3 commits into
anessbelbati wants to merge 3 commits into
Conversation
When editor_edit inserts a Canvas file that the edit document's media map does not list yet, ensureArtifactEditorMedia writes a placeholder media id (videofs-<uuid>) on the element and records the file's source key under it. The Editor draws nothing for a media id its bin does not hold, so the element is black in the preview and in an exported MP4, while the agent is told the insert succeeded. The placeholder was adopted only right after the browser imported that file for the first time, and only on the project loaded at that moment. Two common orders never reach that point: - the Editor already holds the file, so hydration skips it before any remap; - the Editor imports the file when it opens, before the sync tick loads the agent's edit, so the element is not there yet when the remap runs. The browser's next push then replaces the server's media map with its own, and the placeholder's entry is gone. Adoption now runs at the end of every hydration pass, for every host file the bin holds, and right after the sync tick loads an agent edit, in the queue that loads it. An element is pointed at the bin's copy of the file its placeholder entry names; with no entry left, at the file its artifact identity names: the exact version file for a version placement, the record's current file for a record placement. The old remap matched on the artifact id alone, so it could pick another version of the same artifact. tests/smoke/editor-agent-media.spec.ts places a picture through the editor_edit route in both orders and checks that the element ends up on media the bin holds, in the browser and in the server copy, and that the preview shows the picture.
Loading the project empties the Editor's media bin and refills it from storage. The sync tick loads the project to apply an agent edit, and its first run comes as the Editor opens, while hydration may be importing a file. When the two overlap, the import can drop out of the bin in memory while storage keeps it: the timeline points at a file the bin no longer holds, and the preview stays black until the next load. The browser test that opens the Editor after an agent placed a picture shows this on main too: the block is pointed at the bin's copy of the picture and the preview stays black (8 of 10 runs on Linux). The agent edit is now applied in the hydration queue, so a reload never overlaps an import. The browser test now turns the setup guide and the project tour off (the tour dims the preview), inserts as soon as the import lands, and skips when the Editor pushed its media map before the insert, since the server then writes no placeholder.
An Editor push between reading the edit document and the insert makes the insert's base revision stale. The test now reads the document again and retries, as the conflict answer tells an agent to; when that push carried the picture in its media map, the try is skipped, since the server then writes no placeholder. The Editor's first appearance gets 30 s: in a run of the whole suite it compiles on that visit.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this fix, and what changes for the user?
A picture an agent places with
editor_editinsertcan come out black, in the Editor's preview and in an exported MP4, while the agent is told the insert worked. I hit it while driving the app through its MCP tools: 3 of 3 pictures came out black.There are two causes:
videofs-…) on the element (editor-timeline-commands.ts). The Editor swaps it for its own id only right after a first import, so a file the bin already holds keeps the placeholder (editor-mount.tsx) and nothing is drawn.loadProject, which empties the media bin and refills it from storage. When that overlaps hydration importing the picture, the picture drops out of the bin in memory and the preview stays black.This branch swaps placeholders at the end of every hydration pass and right after an agent edit is applied (a pure rule in
opencut/host/placeholder-media.ts, 10 unit tests), and applies agent edits in the hydration queue, so a reload never overlaps an import.How was it tested?
A new browser test,
tests/smoke/editor-agent-media.spec.ts, places a one-colour picture through/api/paper/toolsin both orders, then checks that both copies of the project point at a file the bin holds and that the preview shows the colour. Ten runs of each order in one dev-server start on a Linux runner, against main at 2e9b99e:videofs-id, which the bin doesn't holdRun: https://github.com/anessbelbati/impractical-video/actions/runs/38230505835
The test retries an insert refused as stale (
EDITOR_REVISION_CONFLICT), as the conflict answer tells an agent to. It never retries a check. It skips itself when the Editor pushed its media map, picture included, before the insert, since the server then writes no placeholder: 3 runs on main did, none on this branch.CI (
verify), unchanged, passes on the branch: https://github.com/anessbelbati/impractical-video/actions/runs/38230505574Any remaining limitations or migration steps?