Skip to content

WP5: reliability and latency - #9

Merged
broslife merged 14 commits into
mainfrom
wp5-reliability
Nov 2, 2026
Merged

broslife merged 14 commits into
mainfrom
wp5-reliability

Conversation

@broslife

@broslife broslife commented Nov 1, 2026 •

Copy link
Copy Markdown
Collaborator

WP5: measurements, a faster receive path, smoother pacing, and two dial slots that did nothing.

  • Picture copy. It goes through libavutil's frame and a copy again, as on main. The straight VIC
    transfer into the staging buffer saved nothing on the console: decode fell by about 0.6 ms and
    the transfer rose by the same. An empty file named vic-direto in /switch/punktfunk/ turns the
    straight route back on at start, so one build compares both routes. The nvdec[10s] line names the
    route and times the shim's copy and the staging copy on the GPU.
  • Receive. quinn-udp's Horizon leg reads up to 32 datagrams per bsd call through libnx's
    recvmmsg, where it read one. If the first batch is refused, it drops back to one per call.
    At 16-20 Mb/s the console was near its receive ceiling: the 1 MiB socket buffer filled to about
    450 ms behind, docked as well as wired, and the stream then lost packets. At 10 Mb/s the dock ran clean.
  • Smoothness. A one-frame budget is now core's adaptive cushion alone. Before, every frame
    of the budget was a whole stream period added on top of the cushion, so one frame held each
    picture a refresh longer than the jitter needed. Handheld, the display stage ran 21–30 ms
    there, against 13 ms in the latency mode. Two and three frames now add one and two periods.
    A burst on the wire also left one due picture standing behind every take, so no due time ever
    held a picture. After 30 takes in a row with a second due, the slot now skips one. On the
    console, decode to present fell from 20 to 17 ms at one frame and from 21 to 15 ms at two, for
    about one skipped picture in 150.
  • Automatic bitrate. Where only NVDEC decodes, Automatic now tops out at 20 Mb/s, the rate
    a pinned stream ran clean at, handheld at stock clocks. It still cuts below that for a dock or
    a weak Wi-Fi. Core finds the link's edge by crossing it, and its climbs to 22-25 Mb/s preceded
    every 60-110 ms delay spike. A title that decodes PyroWave keeps the 650 Mb/s ceiling.
  • Hosts. A paired host follows its advert to a new address, matched by its pinned
    fingerprint. A router change had moved the tester's PC from .137 to .10. The saved record then
    showed offline, and adding the host again saved the same pin twice. If a record at the new
    address already pins the certificate, the stale copies are dropped. An address that still
    advertises stays put.
  • GPU. The stats: and menu lines carry the CPU, GPU and memory clocks. The stats: line
    also gives the composition's GPU time per frame. Handheld that is about 6 of 16.7 ms.
  • Audio sync. Every packet is placed against the picture on the glass. A new audio[10s] line
    logs the offset (audio plays 40-90 ms late, steady), splits underruns into silent and audible
    ones, and counts the host's pauses. This only measures; it does not steer playback.
  • Dial: Keyboard. Opens the system keyboard (libnx swkbd, through a small C shim) and types
    the confirmed text on the host as committed characters, when the host can type text and
    grants the keyboard. Tested on the console.
  • Dial: shortcuts. A shortcut from Settings > Input > Quick actions presses its keys on the
    host (F1-F12, Escape, Ctrl/Alt/Shift/Win chords), down in order and up in reverse. F8 tested on
    the console.

Not in this PR: undocking mid-stream leaves the old session's demux thread spinning on a dead
socket. That is in punktfunk-core's transport/shared.rs. Until it is fixed, end the stream before
undocking.

On the console, the batched receive held 20 Mb/s handheld with no loss. Docked 1080p still queues
at 16–17 Mb/s, so docked play wants 10–12 Mb/s. A handheld clock sweep found that the CPU clock
is the one that counts: at 1785 MHz the RTT falls from 19 to 7 ms and e2e from about 49 to
43 ms. GPU and memory on their own change little. docs/improvements.md now carries the
contract for a Switch platform in the shared shell.

The smoothness, backlog and host changes are tested on the console, handheld at 20 Mb/s. The
Automatic ceiling is not tested yet; the next builds will show it.

docs/plan.md closes WP5. The ten-hour soak, memory in applet mode and a trace of the menu's
long frames leave the plan: public use answers the first, the title does not launch in applet
mode, and the CPU clock explains the third. Clock control and PyroWave on NVK are their own tasks
after the baseline.

broslife and others added 14 commits November 1, 2026 12:17
The VIC now writes each picture straight into the presenter's staging
buffer: the shim hands av_hwframe_transfer_data the caller's planes, which
the nvtegra hwcontext maps per transfer. Through libavutil's own frame the
picture was copied again, and that frame's 16-byte alignment likely sent
the hwcontext to its CPU detiling instead of the VIC. A failed straight
transfer falls back to the old route for the rest of the decoder's life.
The nvdec line names the route and adds the shim's copy and the staging
copy's time on the GPU.

The stats and menu lines carry the CPU, GPU and memory clocks, read from
clkrst every ten seconds, and the stats line the composition's GPU time
per frame from timestamp queries, which is the margin the clock leaves.

The audio route places every packet against the picture on the glass with
core's AvSync and logs the offset, smoothed and its spread, every ten
seconds; it only measures and does not steer the ring. The same line
splits underruns into silent and audible ones and counts the host's pauses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The dial's Keyboard slot only logged that it does nothing here. It now
shows libnx's swkbd applet over the stream and types the confirmed text
on the host as committed characters (InputKind::TextInput), when the
host can type text and grants the keyboard. The pads stay off the wire
while the keyboard has them, as under the dial.

swkbd's config block is versioned per firmware, so it stays in a small
C shim (csrc/pf_swkbd.c) that pf-nx-sys now always builds on the
console. The app reaches it through Lifecycle::keyboard, which reads
None everywhere else.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A shortcut slot (Settings > Input > Quick actions: a key from the
editor's grid, F1-F12 and Escape among them, with Ctrl, Alt, Shift or
Win) only logged that it does nothing here. It now presses every key
down in order and lets them up in reverse, as the desktop client does.
A key name this build does not know fires nothing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The straight VIC route saved nothing on the console: decode fell by about
0.6 ms and the transfer rose by about the same. It is off by default now,
and a file named vic-direto in the data directory turns it back on at start,
so one build compares both routes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The demux read one datagram per bsd call, and at 16 Mbit/s that kept the
console near its receive ceiling: the 1 MiB socket buffer filled to about
450 ms behind. quinn-udp's Horizon receive now batches through libnx's
recvmmsg. A first refusal other than an empty socket drops back to one
datagram per call.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Saved hosts were matched by address alone. When the router moved the
PC from .137 to .10, the record stayed on the old address and showed
offline, and adding the host again saved the same pin a second time.

An advert whose fingerprint pins a saved record now moves that record
to its address. If a record there already pins it, the stale copies
are dropped. An address that still advertises is a second interface
and stays put.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Smoothness added the budget as whole stream periods on top of core's
adaptive cushion, so one frame held every picture a refresh longer than
the jitter asked for. Handheld at one frame, the display stage ran
21-30 ms; under the latency intent it ran 13 ms. Pictures still go out
in order, so none are replaced as they are in the latency mode.

The adaptive cushion is now the budget's first frame. Two and three
frames add one and two periods.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The client fronts the shell as the PS5, so About > Stream controls
shows a DualSense and its touchpad chords, which do nothing on the
Switch. Improvements now lists what the monorepo would add:
- Platform::Switch and Client::Switch;
- the Joy-Con pad group with Minus + A and the escape chord;
- confirm on east;
- a decision for every PS5 branch.

It also lists the lines this client drops once a release carries them.

The smoothness section now says that one frame no longer adds a period
on top of core's cushion.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
NVK gives the console Vulkan 1.3, so PyroWave may yet run here. The
contract keeps it behind pyrowave_ok, as on the PS5, rather than
dropping it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Controllers card is display-only on every platform but Android and
Apple. The Switch pads rumble in every stream, so the contract offers
the two rows there, and this client answers them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Smooth at one frame and at two showed the same on the console: e2e
42.5 against 42.7 ms, display 20.5 against 22.5. The extra stream
period of the second frame should have added about 17 ms, so the due
times may not hold the pictures at all.

The nvdec[10s] line now says how many pictures came out with no due
time, and how far ahead of it the rest came out. A new slot[10s] line
counts, under smoothness:
- the vblanks a due time held a picture back;
- the takes with no due, or taken early;
- how late the takes ran;
- how many pictures stood behind each.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A burst on the wire left one due picture waiting behind every take for
the rest of the session. No due time ever held a picture, and Smooth 1
and 2 waited the same ~37 ms from decode to take. After 30 takes in a
row with a second due picture, the slot now skips one, and the
slot[10s] line counts it as caught up.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Core finds the link's edge by crossing it. Handheld, its climbs to
22-25 Mb/s preceded every 60-110 ms delay spike, while a pinned 20 Mb/s
ran clean at stock clocks. Where only NVDEC decodes, Automatic now stops
at 20 Mb/s and still cuts below it for a dock or a weak Wi-Fi. A title
that decodes PyroWave keeps the 650 Mb/s ceiling its quality floor
needs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The ten-hour soak goes to public use, applet mode does not launch, and
the CPU clock explains the menu's long frames. Automatic keeps a 20 Mb/s
ceiling where only NVDEC decodes. Clock control and PyroWave on NVK
become their own tasks after the baseline.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@broslife broslife changed the title WP5: measure the picture copy, the GPU and the audio sync; the dial's keyboard and shortcuts WP5: reliability and latency Nov 2, 2026
@broslife
broslife marked this pull request as ready for review November 2, 2026 18:04
@broslife
broslife merged commit de0b771 into main Nov 2, 2026
2 checks passed
@broslife
broslife deleted the wp5-reliability branch November 5, 2026 22:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant