Repository navigation
Resolve received media by mid, not didAddTrack - #1188
xianshijing-lk wants to merge 1 commit into
Conversation
In single PC mode received media lands on the publisher's `recvonly` sections. libwebrtc fires `didAddTrack` only when a transceiver is first created, never when the server binds a track to a section that was already negotiated — which is what it does to a section the client offered before there was media for it. That subscription then stays pending with no error. Browsers do not hit this because blink diffs receiver streams on every `setRemoteDescription` (`RTCPeerConnection::DidModifyTransceivers`); rust-sdks reimplements that diff in `process_remote_track_addition`. This is the same diff, driven from the server's `mid` → track SID mapping and keyed by `(mid, trackSid)` because the ObjC `RTCRtpReceiver` does not expose `streams`. Scanning the transceivers rather than diffing the mapping is what makes it self-healing: a section negotiated before it carries a track, or named before its publication arrives, is picked up by a later scan. A room move is handled explicitly — it rebuilds every publication on the same peer connection, so the dispatch record has to be dropped or a section the server carries over unchanged never re-attaches. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Devin Review found 3 potential issues.
1 flag not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)
| if publisher.singlePCMode { | ||
| await attachReceivedMedia(on: publisher, midToTrackSid: midToTrackSid) |
There was a problem hiding this comment.
🔴 Newly negotiated tracks attach twice
When a new receive section has a stream ID, attachReceivedMedia attaches its track alongside the existing didAddTrack callback. Both call addSubscribedMediaTrack, replacing the publication's track and emitting duplicate subscription events.
Learn more
Single-PC peer connections still forward native receiver callbacks through didAddTrack. When a receive section first gains a transceiver and a real stream ID, that callback queues an attachment. The answer path also reports that same section, and addSubscribedMediaTrack always creates a new track and sends subscription events. The two paths therefore replace each other's track and announce the same subscription twice.
Example: A publisher joins with an audio track. The subscriber negotiates a new audio section; its receiver callback queues a subscribe, while the answer scan attaches the same SID. The queued callback attaches it again, leaving two subscribe events for one publication.
Recommended fix: In single-PC mode, route received media exclusively through the MID scan, or share one atomic deduplication gate across the callback and scan before either calls addSubscribedMediaTrack. Keep the native callback path for dual-PC mode.
Was this helpful? React with 👍 or 👎 to provide feedback.
| /// | ||
| /// - Returns: Bound sections not yet dispatched, for the caller to attach. | ||
| func receivedMedia(applying mapping: [String: Track.Sid]) -> [ReceivedMedia] { | ||
| _midToTrackSid.merge(mapping) { _, new in new } |
There was a problem hiding this comment.
🔴 Resubscriptions stay detached on reused sections
After set(subscribed: false) clears a track, receivedMedia retains its dispatch record across an unbound answer. The old MID mapping and receiver track keep that section present, so resubscribing the same SID cannot reattach it.
Learn more
The mapping is cumulative, but an answer can leave a previously used MID unbound. set(subscribed:) proactively clears the publication when unsubscribing because the transceiver is reused. This scan counts any non-nil receiver track as still bound by the cached MID, even if the new answer no longer binds that SID; _dispatchedSections therefore keeps its prior entry. If the same SID is subscribed again on that section, the scan sees it as already dispatched and the native callback does not fire for an existing transceiver.
Example: Audio SID TR_A arrives on MID 2. Unsubscribe clears its publication; the next answer has no 2 binding, while the receiver object remains. Resubscribe TR_A on MID 2: the cached (2, TR_A) entry prevents attachment.
Recommended fix: Derive current bindings from the negotiated answer's actual media sections, including which MIDs have no a=msid, and remove dispatch records when a binding disappears. Preserve mappings only where the protocol guarantees they remain valid.
Was this helpful? React with 👍 or 👎 to provide feedback.
| log("No publication yet for \(media.trackSid), deferring attach", .debug) | ||
| await publisher.undispatch(mid: media.mid, trackSid: media.trackSid) | ||
| continue |
There was a problem hiding this comment.
🔴 Late publications leave subscriptions pending
If an answer precedes its publication update, attachReceivedMedia defers the track and clears its dispatch record. didUpdateParticipants never rescans, so the publication remains unsubscribed until another answer arrives.
Learn more
A received answer supplies a MID-to-SID binding, but the matching publication is created from a separate participant update. When the answer arrives first, this branch undoes the dispatch and waits for another scan. didUpdateParticipants creates the publication but never calls receivedMedia; only subsequent answers do. Thus the track remains absent if no later negotiation occurs.
Example: An answer binds MID 3 to TR_B, followed by a participant update announcing TR_B. The first scan defers TR_B; the update creates its publication, but no other answer occurs, so it never subscribes.
Recommended fix: Retry pending MID attachments when participant updates add publications, using the last applied answer's mapping and current transceivers. Ensure retry dispatch remains deduplicated against normal answer scans.
Was this helpful? React with 👍 or 👎 to provide feedback.
Extracted from #1112, which bundled this with preallocated receive sections. This is the correctness half only — no preallocation, no behaviour change outside single peer connection mode.
Problem
In single PC mode received media is assigned to the publisher's
recvonlym-sections, and the SDK identifies it from libwebrtc'sdidAddTrack. That callback fires when a transceiver is created. It does not fire when the server binds a track to a section that was already negotiated, changing only thea=msid— which is what the SFU does to a section the client offered before there was media for it, viaMediaSectionsRequirement.When that happens the track is never attached. There is no error and no retry: the subscription simply stays pending.
This is the same defect webrtc-sdk/webrtc#321 patches in libwebrtc, and the reason
livekit-clientskips up-front recvonly sections on React Native (livekit/client-sdk-js#1984) and rust-sdks synthesizes the event itself (livekit/rust-sdks#1151).Fix
Browsers do not hit this because blink diffs receiver streams on every
setRemoteDescription(RTCPeerConnection::DidModifyTransceivers). This is that diff, at the SDK layer:SignalClientpasses the answer'smid_to_track_idthroughdidReceiveAnswer.Transport.receivedMedia(applying:)merges it into a per-connection mapping, then scans_pc.transceiversand reports bound sections the room has not been given yet, deduped by(mid, trackSid).Room.attachReceivedMediaattaches them, callingundispatchwhen the publication has not arrived yet so a later scan retries.Keyed by
(mid, trackSid)rather than receiver streams because the ObjCRTCRtpReceiverdoes not exposestreams.The trigger is the scan, not a change in the mapping. That is what makes it self-healing — a section negotiated before it carries a track, or named before its publication arrives, is picked up by a later scan with no bookkeeping at the point of failure.
room_movedis handled explicitly: it rebuilds every publication on the same peer connection, so the dispatch record is dropped or a carried-over section would still read as dispatched and never re-attach.Relationship to #1112 and webrtc-sdk#321
didAddTrackwill start firing a second time for these sections and reachaddSubscribedMediaTrackthroughRoom+TransportDelegateas well —TransportMode.subscriberis the publisher in single PC mode, so that guard does not exclude it.addSubscribedMediaTrackis not idempotent, so that would double-attach. Worth a follow-up guard; deliberately not in this PR to keep it minimal.Testing
xcodebuild build -scheme LiveKit -destination 'platform=macOS'— succeeds, no new warnings.swiftlint Sources Tests— clean.LiveKitCoreTests, 588 tests — one failure,droppingTheBinderDetaches, a known pre-existing flake in VideoView code this PR does not touch (passes in isolation in 0.008s).Not covered: a room move in single PC mode is reasoned about but not exercised by a test. Adding one needs a harness that can drive
room_moved.🤖 Generated with Claude Code