Skip to content

Player: one state machine rescues a play refused before its first picture - #1060

Merged
GLinnik21 merged 23 commits into
mainfrom
series/open-recovery
Oct 26, 2026
Merged

GLinnik21 merged 23 commits into
mainfrom
series/open-recovery

Conversation

@GLinnik21

Copy link
Copy Markdown
Owner

Second PR of the playback-refusal series (#918). It is stacked on #1054 (S0); the diff shown is S1a only.

A play the TV refuses before its first picture is now rescued by one registered state machine, OpenRecovery, which replaces the open_failure_action decision function. A manual-quality DTS play refused with num=601 (#695) now retries with the server converting the DTS track, instead of ending on the error screen.

Closes #1045

The machine

rust-modules/media/src/player/recovery.rs, registered in ci/machines.ini. player/pump.rs drives it (drive_open_recovery), and engine.rs steps LoadStarted. Each play request owns one machine on PlaybackSession. It holds the request's capability value from #1054 (start, and attempt derived from it). It reads no global and performs no IO; the driver runs its effects through the existing flight dispatch.

  • States: Opening, Confirming, Flight{rung}, Playing, Failed.
  • Events: Refused{refusal, facts}, RefusalConfirmed{newly}, FlightLanded, FlightRefused, FirstPicture, LoadStarted.
  • Effects: ConfirmRefusal, Rollback, ConvertFacet{facet, flight}, StartAutoHls, ShowError{refusal}.
  • LoadRefusal (Refused{num}, TimedOut, Stalled, SourceFailed) is the event payload and the read-out's cause.

The ladder, for a refusal before the refusing Engine's first picture:

  1. Roll back a pending HLS→Original trial. This is main's first arm. It is not counted as a flight, and it narrows the attempt by a native-audio refusal it just learned.
  2. Forced Direct Play goes to the error read-out.
  3. Audio facet rung: when the refused stream's audio is not AAC, AC3 or E-AC3, re-plan with that audio excluded (start.without(audio)), and the server converts it (dispatch_refusal_conversion). It runs once, and only if excluding the audio actually narrows start. Audio: per-TV Opus limits, and formats the player learns to avoid #1034's Opus conversion is this rung's Opus case.
  4. Both-facets rung: Auto starts the retained HLS bootstrap (main's arm); manual waits for A play refused before its first picture because of its video is retried with the video converted #1059.
  5. Error read-out.

At most MAX_RECOVERY_FLIGHTS = 2 flights per request. SourceFailed and TimedOut keep main's arms. RefusalConfirmed{newly} only decides whether the install preference is written; it never picks the rung.

Behaviour changes from main

  • A manual-quality play refused with num=601 (or any refusal) whose audio is DTS, TrueHD, Opus or similar retries once with that audio converted. On main it ended on the error screen.
  • An Auto play whose Original is watched, with such audio, converts the audio before falling back to full HLS.
  • Fixed: a native-audio stall of a combination the install had already learned left the viewer on "Buffering…" forever. It now ends on the read-out. Found on the webOS 4.5 dev set.
  • Fixed: a refused foreground-resume Load was handled twice: the pump raised the error read-out, then the lifecycle retried it (Playback: a foreground Load refused while the plaintext grants return is claimed again #1018). A Load call's own synchronous refusal now belongs to the foreground lifecycle alone (Engine.foreground_follows_load), and the ladder owns refusals observed after the call returned.
  • After Home and a restore, a refused restored Load no longer spins on "Connecting". LoadStarted reopens a machine left in Flight or Failed. As a backstop, a refusal the machine does not handle fails the Engine, as main always did.
  • A track-switch reload refused before its picture still converts (the rung is per Engine), as on main.
  • After a DTS rescue, the ABR climb re-grades the Original candidate's audio against the narrowed value (regrade_original_audio), so it does not offer DTS direct play again.
  • New delivery reason refusal_recovery (telemetry, PRIVACY.md).
  • Debug only: plxnative-refuse601 makes the pipeline refuse the first Load of a request with num=601. It is a runtime devtrig flag and compiles out without devtriggers.

Known deviation: the conversion keeps the refused track, so #695's files play with DTS converted rather than on their AC3 sibling, which is what 0.8.0 picked. Picking the sibling on the first retry needs the item's stream list on the session; #1059 decides whether to add it. The stored refusal (S2) makes the next play pick the sibling.

Tests

  • recovery.rs: pure transition tests over the machine, including a pinned outcome for every state × event pair and a fixpoint walk proving the two-flight cap.
  • pump.rs hostsim rig, including:
    • a_dts_601_before_the_first_picture_converts_dts_in_one_flight (Feature Request: Support DTS/TrueHD local Transcoding #695);
    • an_auto_original_refused_on_dts_flies_the_conversion_not_hls;
    • a_refused_conversion_ends_on_the_read_out_and_keeps_the_engine;
    • the_climb_after_a_dts_rescue_plans_no_dts_direct_play;
    • a_native_audio_stall_of_a_known_combination_ends_on_the_read_out_not_the_spinner;
    • a_native_audio_refusal_on_a_reloaded_engine_after_the_first_picture_still_converts;
    • the Home/restore cases;
    • refuse601_publishes_once_per_request_after_the_load.
  • app/run.rs hostsim: a_sync_refusal_of_a_foreground_resume_load_is_settled_by_the_lifecycle_alone, plus the async companion.
  • plx_telemetry: delivery_reason_codes_are_closed_and_listed.

Written red-first, with the failure observed: the machine's new ladder rows, the stall, the resume-Load split, the Home/restore cases, the reloaded-Engine case, the ABR climb and the 601 trigger. The other rig tests were written after the code; a mutation of suspect() makes each of them fail for its own reason.

Crate suites on the committed tree:

  • make test-crate C=plx_media: test result: ok. 1793 passed; 0 failed; 64 ignored
  • make test-hostsim C=plx_media: test result: ok. 1973 passed; 0 failed; 64 ignored
  • make test-crate C=app: test result: ok. 1059 passed; 0 failed
  • make test-hostsim: test result: ok. 1174 passed; 0 failed
  • make test-crate C=plx_platform: test result: ok. 307 passed; 0 failed
  • make test-crate C=plx_telemetry: test result: ok. 329 passed; 0 failed; 1 ignored
  • make check-nodef: clean.
  • ci/check-deps.sh, check-module-layers, check-machine-shape (and its self-test) and check-list-caps are green.

ci/allow/* is unchanged; recovery.rs joins the workers/globals machine scope at zero sites. make check and the ARM build are left to CI.

Docs

  • player/CLAUDE.md's failed-open section now describes the ladder, the per-Engine rung, LoadStarted, and who owns a refusal.
  • docs/sans-io.md and the check-deps.sh header name recovery.rs.
  • docs/parity-gaps.md's cold Auto Original note is updated.
  • The pump and decision docs that described the old terminal arms are corrected.

🤖 Generated with Claude Code

GLinnik21 and others added 20 commits October 26, 2026 04:05
… media follows)

Refs #1044

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… recovery flight read it (wip, tests next)

PlaybackSession holds an Arc<Caps> read once where an attempt starts
(request_play_with_queue, and the native-audio conversion's dispatch);
ResolveEnv, the rebase/recovery/retranscode/ABR captures and TranscodeSpec
carry it, so the route's gates and the profile string PMS reads come from the
same value instead of two reads of the process-wide answer.

Refs #1044

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…d refusal reaching its flight)

Caps A (a DTS row) and B (= A.without(Audio dts)) pick differently and profile
differently with no global installed; the native-audio recovery flight's
replacement /decision carries the ceiling learned mid-attempt.

Refs #1044

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ost-lesson caps

Refs #1044

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
… none

The app's play start (request_movie / request_episode / the content and
queue paths) reads devcaps::caps() ONCE and hands it to the route's
request_play* entry points, which begin the attempt on it. The planner,
the resolve worker and every mid-film rebuild plan from that value
(PlaybackSession::caps, ResolveEnv::caps); route no longer reads the
process-wide answer to plan, and plan.rs holds no devcaps::caps( and no
zero-argument is_dp_audio( / is_dp_audio_track( (the cfg(test) pick
wrappers take the &Caps too).

The pump's ConvertNativeAudio no longer re-reads the global after the
lesson: dispatch_native_audio_conversion narrows the playing attempt by
what the demuxer fed (Caps::without), so the replacement /decision
carries the learned ceiling from the attempt's own value. The #1034
pin test (the refusal reaching its recovery flight) is unchanged.

ci/check-deps.sh: the `globals` rule is widened to devcaps::caps( and the
zero-argument is_dp_audio( / is_dp_audio_track( (qualified, or bare after
an import; the _with forms do not match). It is a coverage expansion, not
a shrink: no machine reads them, so zero ledger lines were seeded and
PINNED_LEDGER_TOTALS is unchanged. A self-test covers the shapes.

Refs #1044

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…, not only the conversion's

The failed-open recovery chose the Original trial's rollback over the conversion when both
applied, and only the conversion narrowed the attempt's Caps. The refusal was learned (the
process-wide answer lost the audio), but the attempt kept the value the play started on, so the
rollback's rebase profile, a later retry and a quality change offered native Opus again. The
pipeline refused it again, `learn` returned false (known combination), and a manual Original ended
on the read-out where main converted.

recover_failed_source_route now narrows the attempt (PlaybackSession::narrow_attempt) before it
picks the recovery, once per learned refusal; dispatch_native_audio_conversion no longer repeats it.
The pump rig test a_native_audio_refusal_learned_during_the_original_trial_reaches_the_rollbacks_profile
failed on the rollback's /decision profile before this and passes after.

Also: the docs the change made false (learn_native_audio_refusal, devcaps::learn_refusal, the Caps
struct, plex/mod.rs's allow list, webos5-port.md's codec-capability bullet, sans-io.md), and
FlightRig now begins its attempt on the thread's devcaps::caps() so rig tests plan from the table
they install rather than the pre-probe fallback.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… does without HEVC

The previous wording said "H.264 alone" for a set whose table lists no HEVC decoder, which drops
the VP9 / AV1 copy lanes a table may still list. The direct-play list omits hevc
(direct_play_video_codecs) and the transcode target's encode head is h264 (Caps::encode_vcodec).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…very lands

Step 0 of the playback-refusal S1 lane. Two hostsim rig tests drive the real foreground actuator
and the pump -> poll_foreground_load order of playback_tick:

- an asynchronous type=18 after the Load call returned (route Started, then load_failed): the
  lifecycle reads Started and lets go (Idle); the pump's failed-open ladder fails the Engine.
  This is the case OpenRecovery will own.
- a synchronous refusal (StartFailed together with load_failed, as threads.rs publishes it): the
  pump fails the Engine (read-out) AND the lifecycle then settles the same refusal as #1018's
  retry (Prepared, Engine retired). Two machines act on one refusal today; the test names say so,
  and the split that fixes it (the lifecycle keeps the Load call's own result) follows.

launch_restore is the first half of refuse_restore, extracted unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ng main's four arms

`open_failure_action` was a pure function over four booleans the pump gathered after it
had already run the install's learner and narrowed the attempt. It is now `OpenRecovery`
(`player/recovery.rs`): a tested `State + Event -> Effects` machine owned per request by
`PlaybackSession::recovery`, driven by `pump.rs` (`drive_open_recovery`) the way
`ForegroundLifecycle` is driven by `foreground.rs`.

Characterization only: the ladder is main's, in main's order (pending HLS->Original rollback,
the conversion of native audio the install just learned the pipeline refuses, a cold Auto
Original's HLS contingency, the read-out), and a native-audio STALL of a combination the
install already knew still only logs (that is the next change's bug, pinned by a test).

- Events `Refused{refusal, facts}`, `RefusalConfirmed{newly}`, `FlightLanded`, `FlightRefused`,
  `FirstPicture`; effects `ConfirmRefusal`, `Rollback`, `StartFlight`, `StartAutoHls`,
  `ShowError{refusal}`. The learner is IO, so the machine asks for it (`ConfirmRefusal`) and the
  pump answers with whether the combination was new, in the same drain; the rung is picked on
  that answer. `LoadRefusal` is the typed payload: `Refused{num}`, `TimedOut`, plus `Stalled` and
  `SourceFailed` for the watchdog and the demuxer/transport branches that share the ladder.
- The attempt's capabilities moved into the machine (`start` plus the audio facets excluded, the
  attempt always re-derived from `start`): `PlaybackSession::caps` and `narrow_attempt` are gone,
  `attempt_caps()` delegates, and the narrowing happens inside `step` on a newly learned refusal,
  on every arm, as `recover_failed_source_route` did.
- `FirstPicture` is stepped from the pump's decoded-frame branch; the machine is idempotent there.
- `ci/machines.ini` registers the machine; `player/recovery.rs` joins the workers/globals scope
  of `ci/check-deps.sh` with no ledger line. `open_failure_action`'s unit test moved into the
  machine's table tests with the same seven expectations.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…w ends on the read-out, not on "Buffering..."

TV run G (2026-10-26): a stored {opus, 6ch} refusal and a transcode fixture serving 1600 B
6-channel Opus again. `native_audio_stall_fired` returned false for a combination the install
already knew ("only logs"), so the pump neither failed the Engine nor showed a read-out, the
watchdog was spent, and the viewer stayed on the buffering spinner for good.

The stall is a refusal of the same class as `type=18` (`LoadRefusal::Stalled`). OpenRecovery
now runs the ladder on the learner's answer whatever the refusal: a known combination is a
refusal with no conversion left, so it takes the next rung (a pending Original trial's rollback,
a cold Auto Original's HLS contingency) and else `ShowError`.

Red first: the rig test (a stored {opus,6ch}, both AU queues holding data, nothing accepted for
NATIVE_AUDIO_STALL_MS, the watchdog's STALL line asserted as its precondition) failed with
"a stall of a combination the install already knew left the viewer on Connecting" before the
change and passes after it. player/CLAUDE.md's "a known refusal fails as before" held only for
the `type=18` path; it now says what both paths do.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…, owned by OpenRecovery

Comment-only. `PlaybackSession::caps` no longer exists (the value lives in the request's
`OpenRecovery`); three doc comments in plan.rs, devcaps.rs and session.rs still named the
field or the route as the narrower.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ecycle's alone; the ladder owns refusals observed after the call returned

The pump's failed-open ladder and the foreground lifecycle both acted on a synchronous
refusal of a foreground-resume Load (StartFailed + load_failed): the pump set Error, then the
lifecycle settled the same refusal as #1018's retry and retired the Engine.

Split by who observes the refusal. The foreground actuator marks the Engine whose exact Load
attempt it follows (Engine::foreground_follows_load, set where it starts or takes over following
an attempt). The pump decides before stepping the machine: a failed Load on a followed Engine
whose route_start_status is Failed is the Load call's own result, so it returns and the
lifecycle's Failed settlement owns it. A refusal seen after the call returned, and a fresh play's
synchronous refusal (no lifecycle behind it), stay the ladder's.

The step-0 SYNC test is flipped to the desired behaviour (red first: the pump left Error), the
ASYNC test is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…-facets rung and a 2-flight cap

A pipeline refusal before the first picture now converts the suspect audio (a codec outside
aac/ac3/eac3, named by recovery::suspect from the stream the pipeline was fed) in one recovery
flight, then takes the HLS contingency of a cold Auto Original, then the read-out;
MAX_RECOVERY_FLIGHTS bounds the flights of one request. The learner's answer (newly) no longer
selects a rung: a facet the attempt's start already lacks narrows nothing and gets no flight.
Forced Direct Play ends on the read-out after a pending rollback. StartFlight is ConvertFacet and
dispatch_native_audio_conversion is dispatch_refusal_conversion.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…efusalRecovery

The Retranscode landing of a recovery flight notes the delivery (and the app's own recovery as the
open's origin) through report::note_refusal_recovery_delivery; the reason joins the closed list,
the consent legend and PRIVACY.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…y classes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… landing re-grades the Original candidate's audio; the #695 rig tests

plxnative-refuse601 (dev builds) publishes type=18 num=601 through sf_on_event once the Load
returned and before a picture, once per playback trace generation, so the converted reload of
the same request plays. The flag is read only inside that open window (a process-wide latch
cannot be armed per test).

The refusal-recovery landing calls regrade_original_audio, so the stored Original candidate's
audio is graded against the narrowed attempt and the ABR climb plans the remux, not the
refused audio's Part.

Hostsim tests: a DTS 601 converts in one flight (#695, with and without a sibling), an Auto
Original flies the conversion not HLS, a refused conversion ends on the read-out, the climb
after a rescue plans no DTS direct play, the landing notes its delivery reason and counts the
attempt once, refuse601 publishes once per request.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…1a ladder made false is rewritten

ConvertFacet carries the facet and the flight ordinal, so the driver logs 'converting {facet}
(flight n/2)'. The learner's doc, the pump's refusal comments, dispatch_refusal_conversion's doc,
the type=18 callback comment and the player guide describe the ladder as it is: the facet rung is
chosen by suspect() and the loop guard, not by the learner's answer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… the ladder did not ask for re-opens it; an unhandled refusal fails the Engine

The ladder lives per request but a picture is per Engine: the facet rung is gated on
the refusal's own Engine having no decoded frame (facts.no_frames), not on the
request's history, so a native-audio refusal on a reloaded Engine converts again.
RecoveryEvent::LoadStarted, raised by engine::start_bufferfeed_for (the funnel every
Engine start passes), takes a Flight or Failed machine back to Opening when a park and
restore re-Loaded the route without a landing; the recovery landing's own reload finds
the machine already in Opening and changes nothing. recover_failed_open fails the
Engine on a refusal the machine does not handle, as before the machine.

The stall branch gets the preview check the other failure branches have; the DTS rig
tests are renamed and tightened.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ladder change made false is corrected

The pure table now walks the full cap as a fixpoint over (state, flights, excluded) with
FirstPicture, a free RefusalConfirmed, FlightRefused, LoadStarted and a Refused of an Engine that
did show a picture, and pins every (state, event) pair once. The CLAUDE.md ladder paragraph, the
Sans-IO scope notes, the check-deps header, the pump and decision docs and parity-gaps item 5
now describe the per-Engine facet rung, the LoadStarted re-entry, and the driver failing the
Engine for a refusal the machine does not handle.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@GLinnik21 GLinnik21 added bug Something isn't working playback Playback, the player and its controls labels Oct 26, 2026
…fusal of their Load before the next frame

The rig's Engine is never built by a real sf_load (the rig forces the host object ready), so its teardown cannot prove
the release and quarantines the host sink; every Load a test then starts (the foreground restore, the facet landing's
reload) is refused by the stub from the MEDIA thread (ok=0, no type=18 callback, so a refusal with num=0). The tests
assert "not Error" after a frame that follows the start; they pass only while that frame beats the media thread. When
the media thread wins, the frame reads a second refusal of an open whose facet is spent and the ladder ends on the
read-out ("no rung left for type=18 num=0"). The CI hostsim job failed on S3 (#1063) in two of these tests with no
S3 code involved; a loop of only the pump tests failed the same way (1 of 60 and 4 of 250 runs under load).

pipeline_takes_the_load awaits the stub's refusal of the Load and takes it down, and is called after each Load these
tests go on to grade: park_and_restore (Scenario B), Scenario A's restore, and refuse601's reload. The refusal is now
always absorbed before the frame, so every run takes the ordering that used to lose; the test's own
pipeline_refuses_601 still ends the open. No production change: on a set the restore's Load returns ok=1 and the
pipeline then refuses it with 601.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Base automatically changed from series/attempt-caps to main October 26, 2026 09:37
@GLinnik21
GLinnik21 merged commit f732396 into main Oct 26, 2026
19 checks passed
@GLinnik21
GLinnik21 deleted the series/open-recovery branch October 26, 2026 09:47
Dr-Amp pushed a commit to Dr-Amp/plx-native that referenced this pull request Oct 26, 2026
Brings in main's GLinnik21#1054 (one capability value per playback request), GLinnik21#1060
(OpenRecovery state machine for refusals before the first picture) and GLinnik21#1053
(tools/pr-land.py).

Live TV adapts to the capability value: request_live takes the Arc<Caps> the
way request_play does, and the app passes it at play start. ResolveEnv no
longer derives Default, so the channel tests build their environment from
ResolveEnv::snapshot and set the live channel on it. The LiveTune type moves
above the ResolveEnv doc comment, which it had split.

Signed-off-by: Dr-Amp <dr_amp@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working playback Playback, the player and its controls

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A play refused before its first picture is retried once per facet with the refused stream converted

1 participant