Skip to content

MSE browser cast, FFmpeg HLS window hold, tray Donate - #25

Merged
serezhka merged 51 commits into
mainfrom
cursor/mse-browser
Nov 12, 2026
Merged

serezhka merged 51 commits into
mainfrom
cursor/mse-browser

Conversation

@serezhka

@serezhka serezhka commented Oct 26, 2026 •

Copy link
Copy Markdown
Owner

Summary

  • MSE fat jar: Chromium receive (hls.js / WebCodecs) + optional browser-tab cast (CDP / HAP tab audio); cast off by default.
  • FFmpeg: keep the SDL window across phone pause / seek / volume (Unix SIGSTOP, Windows NtSuspend + session volume, soft playlist replace).
  • Tray: Donate (Stripe Payment Link, suggested €10) + Github before Exit; README badge + .github/FUNDING.yml Sponsor button.
  • CI: three-player matrix on Ubuntu + Windows; apt setup pins Ubuntu mirrors via DoH when system DNS flakes.

Test plan

  • Unit (./gradlew test) including MSE Playwright + donate URL override
  • Windows playback matrix (ffmpeg / gstreamer / mse)
  • Ubuntu playback matrix (ffmpeg / gstreamer / mse)
  • Real device: Kali FFmpeg + Windows (Sergio)
  • Tray Donate opens Stripe; Github opens repo
  • Spot-check release java-airplay-mse-*.jar boots with system Chrome only

Notes

  • Live tab-audio → HAP remains lab-oriented; CI covers canned ADTS/UDP + Annex-B cast smokes.
  • Stripe keys stay out of the repo (Payment Link URL only). Rotate any keys that were pasted in chat.

serezhka and others added 15 commits October 26, 2026 10:12
Chromium host plus hls.js (HTTP-Live) and jmuxer (mirror H.264) on a
localhost media server, wired to the current Playback API without native
GST/FFmpeg. Ported from the earlier player/mse spike onto main layering.

Co-authored-by: Cursor <cursoragent@cursor.com>
Browsers cannot decode AirPlay AAC-ELD in MSE; decode with bundled libav
and stream s16le over /audio WebSocket. Add Playwright Chromium smoke for
jmuxer + PCM path.

Co-authored-by: Cursor <cursoragent@cursor.com>
Without it the receiver advertised a null mDNS name and fps fell back
to an unhelpful default on hosts with no local properties file.

Co-authored-by: Cursor <cursoragent@cursor.com>
Reset the page-ready latch before starting Chrome so a late ready signal
is not dropped, which delayed startMirror past the first IDR.

Co-authored-by: Cursor <cursoragent@cursor.com>
Boot no longer opens a black idle browser window. Missing Chrome is a soft
error (receiver still advertises). HLS /hls proxy is async so mediadata
timeouts during scrub no longer stall the Netty event loop; idle sessions
reject leftover playlist polls. Keep the window up after playlist remove so
YouTube remirror is less likely to show a black gap.

Co-authored-by: Cursor <cursoragent@cursor.com>
Browser readyState was driving /playback-info rate, so buffering during
seek reported rate=0 and YouTube /stop'd the next item (black window).
Own rate/position in Java like the other backends; epoch-guard stopHls;
ignore AbortError on rapid reload.

Co-authored-by: Cursor <cursoragent@cursor.com>
Wire :app:mse mseIntegrationTest (Chromium headless + shared PlaybackLifecycleHarness),
install Chrome in setup-native-player, ship the MSE fat jar on release, and keep
Windows×mse excluded until a stable Chrome install path exists.

Co-authored-by: Cursor <cursoragent@cursor.com>
Sends raw mirror AUs over /audio; page WASM → Web Audio. Fat jar ~36MB
(was ~200MB+). ALAC and tray cast capture stay unsupported for :app:mse.

Co-authored-by: Cursor <cursoragent@cursor.com>
Music ALAC decodes in-page via vendored Apple-reference JS. Cast opens a
navigable Chromium shell and streams only that tab (WebCodecs H.264), not
the desktop.

Co-authored-by: Cursor <cursoragent@cursor.com>
YouTube in an iframe hit X-Frame-Options (refused to connect); encoder at
avc1.42E01E rejected 1152x720 (black TV). Inject capture into a real tab.

Co-authored-by: Cursor <cursoragent@cursor.com>
Hardware WebCodecs failed on DISPLAY=:10; iframe injects raced. Prefer
software encoders, skip subframes, avoid evaluating on about:blank.

Co-authored-by: Cursor <cursoragent@cursor.com>
…-cast.

In-page ws://127.0.0.1 from YouTube/example.com never connected, so the MSE
receiver got no NALs. Screencast JPEGs encode locally with libx264 instead.

Co-authored-by: Cursor <cursoragent@cursor.com>
Flatten Target.attach sessions accept startScreencast but emit no frames on
current Chromium; connect to the page debugger instead and re-feed the last
JPEG so image2pipe keeps producing Annex-B for still tabs.

Co-authored-by: Cursor <cursoragent@cursor.com>
Samsung (and similar) receivers can disappear from a long-lived JmDNS
cache while still on the LAN; defer drops and re-browse with a fresh
JmDNS when the cache goes stale.

Co-authored-by: Cursor <cursoragent@cursor.com>
Replace CDP screencast and system ffmpeg with in-tab getDisplayMedia and
VideoEncoder, delivering Annex-B and AAC-LC ADTS via Runtime.addBinding.
Add shared BrowserCastSession, browser audio source, and HAP AAC UDP send
using SETUP shk encryption for Samsung-style HAP TVs.

Co-authored-by: Sergio Fedorini <serezhka@users.noreply.github.com>
@cursor cursor Bot changed the title Add browser MSE player (:app:mse) to the playback matrix MSE browser cast without ffmpeg; HAP tab audio Oct 27, 2026
serezhka and others added 14 commits October 28, 2026 12:27
Naive JSON string cut stopped at the first escaped quote inside
Runtime.bindingCalled payload, so every frame failed parse (black TV).

Co-authored-by: Cursor <cursoragent@cursor.com>
Add Annex-B capture smoke and secure-context home URL; assert escaped
Runtime.bindingCalled payloads round-trip; enable cast.autoCapture on the
mse playback matrix so SelfCast runs under xvfb like other players.

Co-authored-by: Cursor <cursoragent@cursor.com>
Enable getDisplayMedia audio when client.audio.src=browser, reset SHK
AES-CBC IV per packet like FairPlay, serialize cast session release, and
remove unused CastCaptureServer/screencast leftovers.

Co-authored-by: Cursor <cursoragent@cursor.com>
Install stable Chrome via setup-chrome (plus ffmpeg fixtures), drop the
windows×mse matrix exclude, resolve Chrome/Edge on Windows out of the box,
and add ADTS + HapAudioClient local-UDP integration smokes.

Co-authored-by: Cursor <cursoragent@cursor.com>
When autoCapture is on and getDisplayMedia has no audio track, synthesize a
tone for WebCodecs; pass --no-sandbox for headless Chrome on GHA Windows.

Co-authored-by: Cursor <cursoragent@cursor.com>
Headless getDisplayMedia has no audio track — when autoCapture is on, feed
AudioEncoder with generated AudioData. Retry BtbN zip fetches on Windows.

Co-authored-by: Cursor <cursoragent@cursor.com>
Start getDisplayMedia only after the first video/audio sink is attached so
ADTS/Annex-B listeners are not racing the inject boot. Soft-fail audio setup
so video cast still works; HAP UDP smoke uses paced ADTS without Chromium.

Co-authored-by: Cursor <cursoragent@cursor.com>
Chrome headless in CI does not emit AAC from WebCodecs reliably; HAP encrypt
UDP and inject enableAudio/fake-audio wiring stay covered without zombie Chromes.

Co-authored-by: Cursor <cursoragent@cursor.com>
noSandbox/cast knobs were on the Gradle CLI but not forwarded into
mseIntegrationTest, so Windows headless Chromium flaked. Also force-kill
Chrome processes and wait longer for the control WebSocket on Windows.

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Gst/FFmpeg now fail at startup on client.*.src=browser instead of at tray connect with no capture factory.
README covers the third fat jar, Chromium requirements, and browser cast; MseMediaServer uses NettyEventLoops like the rest of the stack.
Register ClientCastProperties via PlayerConfig with CastSourcePolicy, compile with -parameters, split cast vs playback headless flags, and reset shared browser sessions between smokes.
Fixes integration-test compile after package-private BrowserCastSession.
serezhka and others added 13 commits November 6, 2026 12:11
Avoid recreating VideoEncoder on every small getDisplayMedia size change; debounce large resizes and prefer hardware/no-preference before software to reduce CPU artifacts.
End presentation on /stop when no screen-mirror SETUP was seen (YouTube-only
has no RTSP TEARDOWN). Keep the player warm after mirror→HLS /stop for remirror,
and never kill Chromium on a mirror-session TEARDOWN while HLS lives on another
session id. Cover the policy with unit and control-wiring tests.

Co-authored-by: Cursor <cursoragent@cursor.com>
With client.*.src=none the Cast menu stays grayed and outbound discovery is
skipped. The tray filters our own mDNS instance without touching ServiceDiscovery
so self-cast tests keep working. Document double-click jar launch on Windows.

Co-authored-by: Cursor <cursoragent@cursor.com>
Keep Chromium presentation for mirror/HLS without RDP-specific workarounds,
resync startMirror/startHls when /control connects late, and drop empty
type-1 codec packets before onVideo. Includes SPS dimension helper tests.

Co-authored-by: Cursor <cursoragent@cursor.com>
JmDNS browse is complemented with a lightweight mDNS probe, and gradle
smoke tasks make it easier to cast videotest at a lab receiver without
pulling in host-specific workarounds.

Co-authored-by: Cursor <cursoragent@cursor.com>
Headless autoCapture runners never wrote DevToolsActivePort under that flag,
so WebCodecs and self-cast smokes timed out with zero Annex-B.

Co-authored-by: Cursor <cursoragent@cursor.com>
Headless prefer-hardware can configure then emit zero Annex-B chunks;
CI now prefers software when fake capture is on, and Linux cast runs
headed under xvfb instead of inheriting player headless.

Co-authored-by: Cursor <cursoragent@cursor.com>
Windows curlhttpsrc hangs under playbin; soup works. NVIDIA decoders
only break when READY fails. Prefer healthy curl over soup when the
localhost probe succeeds so HLS uses what the host actually provides.

Co-authored-by: Cursor <cursoragent@cursor.com>
GC was disposing PAUSED orphans (Invalid memory access / hung GST thread
after YouTube scrub). Keep strong refs to soft-stopped pipelines, and
rank the first READY H.264 HW decoder (d3d11 before nv) for playbin3.

Co-authored-by: Cursor <cursoragent@cursor.com>
d3d11h264dec can READY yet leave YouTube HLS without a first picture;
adaptivedemux then fails with fragment download errors. NVIDIA decode
already worked on the Windows lab box into d3d11videosink.

Co-authored-by: Cursor <cursoragent@cursor.com>
Skip the live curlhttpsrc probe on Windows (it hangs and can poison later
downloads). Stop boosting HW ranks; demote only READY failures, and put
d3d11/d3d12 H.264 below NVIDIA so playbin3 does not pick a READY-but-broken
d3d decoder for YouTube.

Co-authored-by: Cursor <cursoragent@cursor.com>
Windows uses NtSuspend/Resume and Core Audio session volume instead of
destroy/reopen; soft playlist replace holds the previous frame.

Co-authored-by: Cursor <cursoragent@cursor.com>
Fixed root-menu rows open the repo and a stub donate URL without
touching the Cast submenu rebuild path.

Co-authored-by: Cursor <cursoragent@cursor.com>
serezhka and others added 4 commits November 11, 2026 17:23
Drop xdotool resize/minimize paths; headed player always uses
--start-fullscreen like the other sinks, with a unit test for the argv.

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
…setup.

Co-authored-by: Cursor <cursoragent@cursor.com>
@serezhka serezhka changed the title MSE browser cast without ffmpeg; HAP tab audio MSE browser cast, FFmpeg HLS window hold, tray Donate Nov 11, 2026
serezhka and others added 5 commits November 11, 2026 18:15
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
…ion.

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@serezhka
serezhka merged commit 61f6be7 into main Nov 12, 2026
10 checks passed
@serezhka
serezhka deleted the cursor/mse-browser branch November 13, 2026 10:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants