Skip to content

fix: Switch simulcast layers on a keyframe - #80

Merged
kimsun721 merged 3 commits into
mainfrom
fix/78-switch-on-keyframe
Oct 6, 2026
Merged

kimsun721 merged 3 commits into
mainfrom
fix/78-switch-on-keyframe

Conversation

@kimsun721

@kimsun721 kimsun721 commented Oct 6, 2026 •

Copy link
Copy Markdown
Owner

Switch simulcast layers on a keyframe

Why

A layer change took effect on the next frame, which is almost always a delta
frame of the new layer. The viewer's decoder has no reference for it, so the
picture stayed broken until that layer's next keyframe. With OBS that is up to
a full keyframe interval.

Changes

Switching

A viewer's track now holds the layer it is being sent and a target layer. A
layer request records the target and requests its keyframe, and the current
layer keeps flowing. The switch happens when a keyframe of the target arrives,
and that keyframe is the first frame forwarded from the new layer.

  • layer_changed is sent at the switch, not at the request.
  • A new viewer starts the same way, so nothing is forwarded before a keyframe
    of its starting layer.
  • Asking for the current layer again cancels a pending switch.
  • Automatic mode decides from the pending target when there is one.
  • A keyframe request from the viewer is passed on for the pending target.

Viewer protocol

layer_status carries current_layer and target_layer in place of
chosen_layer. current_layer is null until playback starts.

Watch page

The page marks the pending layer and says it is waiting for a keyframe.

A debug mode, from a button or ?debug, logs each switch with how long it
waited, each freeze the browser counts with the packets lost and frames
dropped around it, and any jump in the RTP timestamp.

Keyframes are the publisher's job

The issue listed requesting the keyframe again until it arrives. That is left
out: the server asks once and then waits for a keyframe of the target layer.
OBS sends keyframes on its own interval whether or not it is asked, so a lost
request costs at most one interval. A publisher that sends them only on
request, as a browser does, has to keep an interval of its own.

The README said publishing works from a browser as it does from OBS. It now
names OBS as what is tested, and says that any other WHIP client can publish
but has to send keyframes on an interval.

Tested

Unit tests cover the three cases from the issue: a delta frame of the target
does not switch, a keyframe of another layer does not switch, a keyframe of
the target does. Others cover a new viewer starting on a keyframe, cancelling
a switch, and a repeated request asking once. The integration publisher now
sends media on every layer, since nothing is forwarded before a keyframe.

With OBS sending four layers from 1080p to 270p at 60 fps:

Switches by hand more than 30, none with a broken picture or a freeze
Wait for a switch 0.5 to 4.1 s, on a 4.17 s grid per layer, which is OBS's keyframe interval of 250 frames
RTP timestamp across a switch no jump

close #78

@kimsun721
kimsun721 force-pushed the fix/78-switch-on-keyframe branch from 39a198b to d5a5dc8 Compare October 6, 2026 10:34
@kimsun721
kimsun721 merged commit d2cdf9f into main Oct 6, 2026
1 check passed
@kimsun721
kimsun721 deleted the fix/78-switch-on-keyframe branch October 6, 2026 10:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FIX] Switch simulcast layers on a keyframe

1 participant