Skip to content

Audio: support separate master and headphone output devices - #10

Merged
beneaze merged 3 commits into
beneaze:mainfrom
timz:feature/separate-master-headphones-output
Nov 5, 2026
Merged

beneaze merged 3 commits into
beneaze:mainfrom
timz:feature/separate-master-headphones-output

Conversation

@timz

@timz timz commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Allow speakers and headphone cue/PFL to use two independent audio output devices. The existing single-device channel routing modes remain available.

Refs #9.

Changes

  • Select master/headphone devices by ID, with legacy name resolution. SeparateDevice negotiates a strict common stereo F32 format; single-device modes retain preferred-rate, device-default-rate and app startup fallbacks. Temporary system-default startup never overwrites the saved audio configuration.
  • Drive one engine core from the master callback and transport cue through a preallocated SPSC queue with adaptive sinc resampling for clock drift.
  • Calibrate fixed output delays after two seconds of smoothed cue timing, with manual correction of +/-500 ms. Continuous correction stays in the cue resampler; master delay never follows queue jitter. Apply recalibrates.
  • Without driver timestamps, compensate the known queued cue frames and sinc lookahead while reporting that device timing is unavailable.
  • Count recoverable stream events without muting output. Callback stalls clear when callbacks return; terminal device/stream failures require reopening. Defer coalesced error-kind logging to non-audio diagnostic polling.
  • Stage settings behind Apply/Discard, preserve deck state on reopen failure, and keep recording before output alignment delay.
  • Add output documentation, synchronization/recovery regression tests and a silent probe that applies minimum gains before either stream starts, using Engine::start_with_commands.

Validation

Windows checks for the review fixes:

  • cargo build -p clubcraft -j 4 — passed.
  • cargo clippy --workspace --all-targets -j 4 -- -D warnings — passed.
  • Engine tests — 44 unit tests, 31 integration tests and 1 doctest passed, including virtual-hour drift simulation at +/-1000 ppm.
  • Four application startup fallback and saved-settings regression tests passed.
  • cargo test --workspace --no-fail-fast -j 4 — all changed-area tests passed; the same 8 previously reported failures remain: 3 DJ.Studio importer tests on Windows and 5 separation tests requiring missing ONNX models. The workspace run is not green.
  • At commit 2950371, the updated silent probe opened and reopened Senary Audio, rejected a missing device and preserved minimum gain. Only one output device was available. No additional hardware or listening test was performed for the persistence/calibration follow-up.

Before the review fixes, silent probes opened and reopened Senary Audio and AKG N9 Hybrid in both directions. The maintainer also reported passing macOS Clippy and engine tests for the original commit. These results do not establish cross-platform hardware or acoustic alignment for the revised code.

Limitations

Driver timestamps do not cover the complete wireless/analog path. Bluetooth listening tests showed approximately 140 ms of additional headphone lag; use manual correction to delay master accordingly. Bluetooth is unsuitable for low-latency cueing.

Fixed calibration avoids master pitch modulation but can leave residual offset if device latency or cue buffering changes; Apply recalibrates. The resampler still regulates queue fill, rather than steering on timestamp error. Analog alignment, physical unplug/replug and revised two-device behavior on macOS/Linux need hardware validation.

Developed with AI assistance and tested locally.

@beneaze beneaze left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the careful write-up and the thorough tests. Handing the core back by destroying the callback, rather than waiting for one more callback, is a real improvement over the current reopen logic.

Checked on macOS: cargo clippy --workspace --all-targets -- -D warnings passes, and cargo test -p clubcraft-engine passes (38 unit, 31 integration, 1 doctest). No listening test here.

Blocker

Any stream error now mutes the master until Apply, in every routing mode. err_cb sets health.master.failed, which is sticky, and the master callback then outputs silence. cpal 0.18.2 reports ErrorKind::Xrun through the error callback on every backend (CoreAudio, WASAPI, ALSA, PipeWire, JACK, ASIO). CoreAudio also reports DeviceChanged for a default-device stream whose route changed, while the stream keeps running. So the first underrun in a set silences the speakers. The log::warn! is gone too, so the error isn't logged anywhere. Please mark an endpoint failed only for errors that really end the stream (DeviceNotAvailable, StreamInvalidated, …), and count xruns as diagnostics.

Significant

  1. The stall detector never recovers. check_stalled runs whenever the UI reads stream info. A gap of more than 2 s (waking from sleep, another app taking exclusive mode) marks the output failed for good, even after callbacks resume. It should clear once callbacks are back.
  2. Single-device setups lose their fallbacks. LiveEngine::start no longer falls back to AudioConfig::default(). choose_format no longer falls back to the device's default rate, a saved sample_rate is now strict, and supports_buffer rejects buffer sizes outside the reported range off Windows. So a saved setup that no longer matches the hardware now starts with no audio at all, where today it plays on the default output. Strict negotiation makes sense for SeparateDevice; please keep the existing modes as forgiving as they are on main.
  3. The master runs through a moving delay. master_delay follows a smoothed offset that includes the wandering fill of the cue queue. The tap slews at up to 0.25 frames per sample, which is a 25 % speed change while it moves; settling about 25 ms sweeps the master's pitch for around 100 ms. I'd keep the master delay fixed (set on open or when the correction changes) and do all continuous correction on the cue side, for example by steering the servo on timing error rather than on queue fill.

On the headphone lag

  • With Bluetooth, the driver estimates don't cover the codec delay (often 100–300 ms), and compensating for it means delaying the master by that much. I think it's fine to document Bluetooth as unsuitable for cue rather than keep the PR open on it.
  • Something the code can fix: without driver timestamps, render_at uses alignment(0.0, …) (output.rs:282). That ignores the latency the cue path adds itself (target fill of about 20–30 ms plus the 15-frame sinc look-ahead), which the code knows exactly. Compensating for it would remove part of the lag on drivers without timestamps.

Minor

  • output.rs has unexplained constants (0.94 cutoff, 1e-7/2e-6 servo gains, the 0.25 slew) and one-letter names. AGENTS.md asks for comments that explain why.
  • output_probe starts the streams before setting the gains to −60 dB. That's harmless with no tracks loaded, but setting gains before starting would follow the house rule exactly.

Next steps

Please fix the blocker and items 1–2 before marking this ready. Item 3 is a design suggestion; happy to discuss.

@timz
timz marked this pull request as ready for review November 3, 2026 17:41
@timz

timz commented Nov 3, 2026

Copy link
Copy Markdown
Contributor Author

This is my first major contribution to an external project, so please forgive me if I’ve done anything wrong. 😅

@beneaze

beneaze commented Nov 3, 2026

Copy link
Copy Markdown
Owner

No it's perfectly fine, and thanks for the contribution! I just had Claude checking on the PR.

@timz

timz commented Nov 3, 2026

Copy link
Copy Markdown
Contributor Author

I tested it with Bluetooth headphones, and there's a delay of around 140 milliseconds. However, there's a field to manually compensate for this delay. Codex couldn't find a better way to detect the latency. I didn't have this issue in VirtualDJ, but I couldn't figure out how they implemented it.

@beneaze

beneaze commented Nov 3, 2026

Copy link
Copy Markdown
Owner

Thanks, that matches what I'd expect: on Windows the Bluetooth codec delay isn't part of the timing cpal gets from the driver, so the manual correction is the right tool, and documenting Bluetooth as unsuitable for low-latency cueing is fine.

On VirtualDJ: ClubCraft follows a clean-room rule (docs/clean-room.md). We can compare outputs but not look into how it's implemented. If you want to compare, record the speakers and the headphones together with one microphone while VirtualDJ plays. That shows whether it delays its master or just leaves the headphones late.

Since the PR is now marked ready: there are no new commits yet, so the points from the review still apply. Most important is the blocker. Any xrun reported to the error callback mutes the master until Apply, in every routing mode. Please fix that and items 1–2 before we merge.

@timz

timz commented Nov 4, 2026

Copy link
Copy Markdown
Contributor Author

Addressed the review in commit 2950371.

  • Blocker: both stream error callbacks now classify CPAL error kinds. Xrun, DeviceChanged and other recoverable events increment diagnostics without setting the terminal failure flag. DeviceNotAvailable, StreamInvalidated and other terminal kinds still require reopening. BackendError uses callback liveness instead of permanently muting a stream whose callbacks may continue. Coalesced error kinds are logged outside audio callbacks when stream diagnostics are polled.
  • Item 1: a stall is computed from heartbeat age without setting the terminal flag. Both callbacks update their heartbeat before output guards, so resumed callbacks restore status automatically. Tests cover stall/recovery and terminal failures remaining latched.
  • Item 2: single-device modes again try the saved rate, 44.1/48 kHz and the device default, and let the backend validate fixed buffer requests. LiveEngine restores the system-default startup retry, with a visible warning. SeparateDevice keeps strict format negotiation and does not substitute the selected master. Three mocked startup tests cover fallback, strict separation and double failure.
  • Item 3: output delay targets latch at the first usable estimate after opening and stay fixed through jitter/rebuffering. Apply recalibrates. Tap changes crossfade fixed positions over 128 frames instead of slewing a read head, so they do not create the 25% speed sweep. Continuous clock correction remains in cue resampling. This deliberately retains queue-fill servo control; it does not introduce a timestamp-error servo, and fixed calibration may leave residual offset if buffering changes.
  • Missing timestamps: alignment now includes the known cue backlog and sinc history, instead of assuming zero added cue latency. Unknown device/Bluetooth latency is still not inferred.
  • Minor: documented/named filter and servo constants, clarified DSP locals, and added Engine::start_with_commands so the probe sets minimum gains before starting either output. The probe can also exercise a single output.

Validation: Windows app build and workspace Clippy pass; 42 engine unit tests, 31 integration tests, 1 doctest and 3 app startup tests pass. The full workspace run still reports the same 3 importer failures and 5 missing-model failures. A silent Senary Audio reopen probe passes; AKG is not currently connected, so I could not repeat the revised two-device hardware probe. No new listening or macOS/Linux hardware result is claimed.

The PR description and output documentation now describe the revised behavior and Bluetooth limitation.

@beneaze

beneaze commented Nov 4, 2026

Copy link
Copy Markdown
Owner

Thanks, this addresses everything from the review. The error classification, stall recovery, single-device fallbacks and fixed master delay all look right, and clippy plus the engine and app tests pass on macOS.

One regression from combining the restored fallback with sync_audio: when start_audio falls back to AudioConfig::default(), app.rs:192 syncs the settings to the engine's default config, and app.rs:1472 saves that on exit. A single launch with the interface unplugged therefore permanently replaces the saved device. On main the saved config survived the fallback. Please skip sync_audio (or keep the saved config) when the fallback was used.

Smaller, not blocking: set_alignment latches the first raw estimate, before the one-second smoothing has had any effect, and the first callbacks after opening are often the most irregular. Latching once the estimate has settled (for example after about 2 s) would make the fixed delay more reliable.

@timz

timz commented Nov 4, 2026

Copy link
Copy Markdown
Contributor Author

Addressed both follow-up points in e696117.

  • Saved configuration: start_audio now reports used_fallback explicitly. LiveEngine only exposes a canonical startup_audio_config when the requested setup opened successfully. Both the initial app startup and the Settings restart skip sync_audio during system-default recovery, so exiting preserves the original saved device, rate and buffer. Successful explicit Apply continues using the existing save path. The regression test simulates two fallback/save/load cycles, then reconnects the original device and verifies that successful startup can still persist its resolved ID.
  • Calibration: delay targets now latch after two seconds of rendered cue audio, rather than the first estimate. The first second uses sample-weighted averaging, then a one-second low pass, reducing the influence of irregular initial callbacks. Rebuffering or a driver-timestamp availability change restarts warmup before calibration completes. After calibration, the master delay remains fixed through these events. Timestamp-free queue compensation uses the same warmup. Tests cover a startup outlier, the two-second threshold, warmup reset, timing-source changes and fixed delay after locking.

Windows validation: app build and workspace Clippy pass; 44 engine unit tests, 31 integration tests, 1 doctest and all 4 startup/persistence tests pass. The full workspace run still has the same 3 importer failures and 5 missing-model failures. No additional hardware or listening result is claimed for this follow-up.

Updated the output documentation, UI hint and PR description accordingly.

@beneaze
beneaze merged commit 9743262 into beneaze:main Nov 5, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants