Capture: device source by ladvoc · Pull Request #1354 · livekit/rust-sdks

ladvoc · 2026-08-24T23:50:34Z

Add a capture source for camera devices.

This PR defines the API for device source (which is the same across platforms); backend implementations (e.g., AVFoundation for macOS, V4L2 for Linux, etc.) are left to subsequent PRs.

Closes BOT-528

Stack history

This PR starts the active continuation of the original capture stack after its first six layers merged. The merged layers remain in GitHub Stack 1371, and the active layers are in GitHub Stack 1523.

Full sequence: #1348 → #1349 → #1350 → #1351 → #1352 → #1353 → #1354 → #1355 → #1356 → #1357 → #1358 → #1370.

devin-ai-integration

Note

This report is out of date. Scroll down for Devin Review's latest report on this PR.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 1 additional finding.

github-actions · 2026-08-25T21:35:32Z

Changeset ✓

This PR includes a changeset covering all affected packages:

Package	Bump
`livekit-capture`	`minor`
`livekit-ffi`	`minor`

alan-george-lk

Approved with some minor stuff

alan-george-lk · 2026-08-26T17:16:39Z

+    /// Packed BGRA.
+    Bgra,
+    /// Packed RGB24.
+    Rgb24,


Had this feedback with @stephen-derosa on the C++ side, any reason to not make these all-caps matching the comment/acronym?

Mainly due to the pascal case convention Rust usually follows (e.g., acronyms don't have any special rules).

alan-george-lk · 2026-08-26T17:16:53Z

+            Self::Nv12 => "nv12",
+            Self::Bgra => "bgra",
+            Self::Rgb24 => "rgb24",
+            Self::Bgr24 => "bgr24",


Do these need to be lowercase?

Not strictly, but lowercase is a fairly standard convention for stable string representations.

alan-george-lk · 2026-08-26T17:23:19Z

+#[allow(dead_code)]
+fn capture_frame_metadata(
+    capture_wall_time_us: u64,
+) -> livekit::webrtc::video_frame::FrameMetadata {
+    livekit::webrtc::video_frame::FrameMetadata {
+        user_timestamp: Some(capture_wall_time_us),
+        frame_id: None,
+        user_data: None,
+    }
+}
+
+/// Validates the platform-neutral parts of a configuration; `supported`
+/// reports whether the backend can deliver a frame format.
+#[allow(dead_code)]


Similar to other PRs, why allow dead code on new stuff?

This is due to this PR being in a stack; some code is implemented here but not used until the subsequent PRs. Since we don't treat dead code as an error in CI, I removed the allows.

alan-george-lk · 2026-08-28T21:29:24Z

+//! platform-native buffers without a CPU copy. Otherwise they are converted
+//! to I420.
+
+use unsupported as backend;


I wasn't sure on this so I asked GPT, so take with a grain of salt:

Device capture is not implemented on any platform. mod.rs:28 unconditionally aliases the backend to unsupported, and source-device = [] adds no platform backend or dependencies. Consequently, devices(), devices_blocking(), DeviceVideoSource::new(), and the new FFI requests will always return UnsupportedPlatform, including on macOS/Linux/Windows.
use unsupported as backend; is not a runtime fallback decision—it selects that module at compile time. The selected module’s Session::open() always returns an error. So this PR exposes a polished public API, but no code path can ever open a camera.

Is there a way to make a unit test or some sort of test that exercises this? Is this actually an issue for this PR?

No, this is also a placeholder that is overridden by the follow-up PRs in this stack.

devin-ai-integration

Note

This report is out of date. Scroll down for Devin Review's latest report on this PR.

Devin Review found 1 new potential issue.

2 flags not posted on this PR by your GitHub settings — view them in Devin Review. (Configure)

devin-ai-integration · 2026-08-31T18:48:25Z

+// Format selection requested from a capture device. The device negotiates
+// the delivered format; CaptureSourceInfo reports the outcome.
+message DeviceFormatRequest {


🟡 Negotiated camera format goes missing

When DeviceFormatRequest permits negotiation, the response omits the selected frame rate and pixel format. FFI clients cannot discover the delivered format promised by this API.

Prompt for agents

Extend the FFI result for device source creation to expose the complete negotiated DeviceFormat, not only CaptureSourceInfo.resolution. Update livekit-ffi/protocol/capture.proto, populate the value from DeviceVideoSource::format in livekit-ffi/src/server/capture.rs before erasing the source behind PixelVideoSource, and regenerate all protocol bindings. Preserve behavior for non-device capture sources, likely with an optional device-specific field.

Was this helpful? React with 👍 or 👎 to provide feedback.

devin-ai-integration

Devin Review found 1 new potential issue.

4 flags not posted on this PR by your GitHub settings — view them in Devin Review. (Configure)

devin-ai-integration · 2026-09-09T23:59:26Z

devices

+    server.async_runtime.spawn(async move {
+        let message = match device::devices().await {


🟡 Enumeration panics lose callbacks

A backend panic makes on_list_capture_devices drop its callback. devices resumes blocking-task panics, while the detached task remains unobserved.

Learn more

devices deliberately resumes a panic from its blocking worker. That panic then terminates this detached Tokio task before it constructs ListCaptureDevicesCallback. The returned async ID has no terminal callback, and the discarded JoinHandle also hides the failure from the FFI panic reporting path.

Example: A platform backend panics while querying a malformed native device descriptor. The request immediately returns async ID 42, but callback 42 never arrives.

Recommended fix: Keep and supervise the task handle through the server's task-watching mechanism, and convert enumeration panics into the callback's error branch so every accepted async ID completes exactly once.

Was this helpful? React with 👍 or 👎 to provide feedback.

devin-ai-integration Bot reviewed Aug 24, 2026

View reviewed changes

ladvoc force-pushed the ladvoc/capture-source-device branch from f590f27 to 3f9fa7d Compare August 25, 2026 21:35

ladvoc force-pushed the ladvoc/capture-source-device branch 2 times, most recently from 01e3fbd to c7580d8 Compare August 28, 2026 03:09

ladvoc changed the base branch from ladvoc/capture-source-gstreamer to ladvoc/capture-source-rtsp August 28, 2026 03:14

alan-george-lk approved these changes Aug 28, 2026

View reviewed changes

ladvoc force-pushed the ladvoc/capture-source-device branch from 5beef13 to 9b4398b Compare August 28, 2026 23:01

ladvoc force-pushed the ladvoc/capture-source-device branch from 9b4398b to fd4c331 Compare August 31, 2026 18:43

devin-ai-integration Bot reviewed Aug 31, 2026

View reviewed changes

ladvoc force-pushed the ladvoc/capture-source-device branch 2 times, most recently from e0da75a to ea9bd68 Compare September 9, 2026 23:55

devin-ai-integration Bot reviewed Sep 9, 2026

View reviewed changes

ladvoc force-pushed the ladvoc/capture-source-device branch from ea9bd68 to 8b19169 Compare September 10, 2026 18:49

ladvoc force-pushed the ladvoc/capture-source-device branch from 8b19169 to 929dc5f Compare September 17, 2026 18:07

ladvoc force-pushed the ladvoc/capture-source-device branch from 5326fba to 7fc151f Compare September 17, 2026 20:37

ladvoc force-pushed the ladvoc/capture-source-device branch from 7fc151f to be05ba8 Compare September 17, 2026 21:59

ladvoc force-pushed the ladvoc/capture-source-device branch from be05ba8 to cf3f721 Compare September 18, 2026 18:37

ladvoc force-pushed the ladvoc/capture-source-device branch from 65cefb3 to 0a00ec5 Compare September 22, 2026 16:42

ladvoc force-pushed the ladvoc/capture-source-device branch from 0a00ec5 to 88f8840 Compare September 22, 2026 16:43

ladvoc force-pushed the ladvoc/capture-source-device branch from 88f8840 to f37d277 Compare September 25, 2026 18:39

ladvoc force-pushed the ladvoc/capture-source-device branch from f37d277 to 8a8c45c Compare October 5, 2026 19:53

ladvoc force-pushed the ladvoc/capture-source-device branch 2 times, most recently from 68eec12 to af333c5 Compare October 8, 2026 19:55

ladvoc and others added 8 commits October 14, 2026 13:21

Add source feature

e87aab5

Implement source

75eef11

Document new source

2ea61a9

Expose source over FFI

265f797

Generate protobuf

b0a0234

Changeset

d4c6b4a

generated protobuf

9368361

Use spawn_blocking directly

81049c5

ladvoc removed this pull request from stack #1371 October 14, 2026 20:26

ladvoc force-pushed the ladvoc/capture-source-device branch from af333c5 to 81049c5 Compare October 14, 2026 20:27

ladvoc requested review from 1egoman, cloudwebrtc and lukasIO as code owners October 14, 2026 20:27

ladvoc changed the base branch from ladvoc/capture-source-rtsp to main October 14, 2026 20:27

ladvoc added this pull request to stack #1523 October 14, 2026 20:28

ladvoc mentioned this pull request Oct 14, 2026

Capture: GStreamer source #1353

Merged

Remove allow dead code

24b27d8

cloudwebrtc approved these changes Oct 15, 2026

View reviewed changes

ladvoc added 2 commits October 15, 2026 12:29

Fix enumeration panic loses the FFI callback

170bb50

Negotiated format in FFI result

032de97

		server.async_runtime.spawn(async move {
		let message = match device::devices().await {

Conversation

ladvoc commented Aug 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Stack history

Uh oh!

devin-ai-integration Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

✅ Devin Review: No Issues Found

Uh oh!

github-actions Bot commented Aug 25, 2026

Changeset ✓

Uh oh!

alan-george-lk left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

devin-ai-integration Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

devin-ai-integration Bot Aug 31, 2026

Choose a reason for hiding this comment

Uh oh!

devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

Uh oh!

devin-ai-integration Bot Sep 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

ladvoc commented Aug 24, 2026 •

edited

Loading

devin-ai-integration Bot left a comment •

edited

Loading

devin-ai-integration Bot left a comment •

edited

Loading

devin-ai-integration Bot Sep 9, 2026 •

edited

Loading