Add a low latency decoder option and actually apply the MediaCodec keys

[iflyhere]

Note

Compile tested only (arm64-v8a + armeabi-v7a). Not yet flown. A glass-to-glass
comparison with the switch on/off on real hardware would be very welcome,
especially on non-Qualcomm SoCs.

The problem

writeAndroidPerformanceParams() in app/videonative/src/main/cpp/helper/AndroidMediaFormatHelper.h
sets the two decoder keys that matter for a live stream:

AMediaFormat_setInt32(format, "low-latency", 1);
AMediaFormat_setInt32(format, "priority", 0);   // 0 = realtime, 1 = best effort

It is never called. Both call sites are commented out:

• h264_configureAMediaFormat() — // writeAndroidPerformanceParams(format);
• h265_configureAMediaFormat() — // writeAndroidPerformanceParams(format);

and the same keys sit commented out a second time in
VideoDecoder::configureStartDecoder(), including the vendor variants:

// AMediaFormat_setInt32(format, "low-latency", 1);
// AMediaFormat_setInt32(format, "vendor.qti-ext-dec-low-latency.enable", 1);
// ...
// AMediaFormat_setInt32(format, "priority", 0);

So every decoder is configured with the stock MediaCodec pipeline. Without
KEY_LOW_LATENCY the codec may hold output frames back for reordering, which
for a majestic stream (no B-frames, low slice count) buys nothing and only
costs frames of latency. Without priority = 0 the codec runs as best effort
instead of realtime, so it competes with everything else on the device.

The change

The keys are written now, but behind a switch. "Change how the decoder
behaves" is exactly the kind of thing that should be escapable on a device
whose vendor codec does not like it, so:

Settings → Video → Low latency, persisted as low_latency_decoder,
default on.

Plumbing:

• VideoPlayer.setLowLatency(boolean) → nativeSetLowLatency() →
  VideoDecoder::setLowLatency()
• VideoDecoder::configureStartDecoder() calls
  writeAndroidPerformanceParams() only when the flag is set
• VideoActivity pushes the persisted value down as soon as the player exists,
  so the setting survives a restart

Because the decoder is created lazily once SPS/PPS arrive, toggling the switch
takes effect the next time the decoder is configured (next video start /
channel change), not on a decoder that is already running. The menu says so.

writeAndroidPerformanceParams() also picked up the vendor keys that were
commented out in VideoDecoder.cpp (Qualcomm, HiSilicon, rtc-ext) and is now
static, like the two functions next to it. Unknown AMediaFormat keys are
ignored by MediaCodec, so writing all variants is safe across vendors and
Android versions.

Not in this PR

VideoDecoder::feedDecoder() blocks the receive thread for up to
BUFFER_TIMEOUT_US (17 ms) per NALU and retries for up to a second, and
BufferedPacketQueue has no time bound on how long it holds a reorder buffer.
Both also cost latency but need their own discussion.

---

Part of a series of independent fixes found while building an immersive (OpenXR) mode on a
Quest 3, each standalone and mergeable in any order:

• Add a low latency decoder option and actually apply the MediaCodec keys #113 — apply the MediaCodec low-latency keys, behind a new switch
• Render the main video on a SurfaceView unless object detection needs the TextureView #114 — SurfaceView for the main video unless object detection needs the TextureView
• Fix: DVR recording leaks one heap buffer per frame #115 — DVR leaked one heap buffer per recorded frame
• Harden the USB adapter lifecycle #116 — USB adapter lifecycle, including an ANR from an unbounded join on the main thread
• Keep the activity alive across configuration changes #117 — keep the activity alive across configuration changes
• Log the ids of attached usb devices, and make wirelessInfo() safe #118 — log the ids of attached usb devices, and make wirelessInfo() safe
• VideoPlayer and WfbNgLink take a Context, not an AppCompatActivity #119 — VideoPlayer / WfbNgLink take a Context
• Bound how long the packet queue holds a frame back #120 — bound how long the packet queue holds a frame back

#113 and #116 are now confirmed on hardware (Quest 3, Horizon OS, Android 14).

[qodo-free-for-open-source-projects]

PR Summary by Qodo

Apply MediaCodec low-latency keys behind a user setting

🐞 Bug fix ✨ Enhancement ⚙️ Configuration changes 🕐 20-40 Minutes

AI Description

• Adds a persisted, default-on low-latency setting with a vendor compatibility escape hatch.
• Propagates the preference through Java and JNI into native decoder configuration.
• Applies AOSP and vendor low-latency keys plus realtime priority on decoder startup.

Diagram

graph TD
  UI["Video Settings"] -->|persists| PREFS[("Shared Preferences")] -->|loads| JAVA["Java Player"] -->|native call| JNI["JNI Bridge"] -->|forwards| NATIVE["Native Player"] -->|configures| DECODER["Video Decoder"] -->|sets keys| CODEC["MediaCodec"]
  UI -->|toggles| JAVA

Loading

High-Level Assessment

The following are alternative approaches to this PR:

1. Reconfigure the active decoder immediately

• ➕ Makes setting changes effective without restarting video or changing channels.
• ➖ Interrupts playback and adds synchronization risk around surfaces, codec threads, and buffered NALUs.

2. Select keys by codec vendor

• ➕ Avoids sending irrelevant vendor keys and could reduce codec-specific failures.
• ➖ Requires brittle device or codec detection and ongoing maintenance across fragmented Android implementations.

Recommendation: Keep the current persisted opt-out and apply changes at the next natural decoder configuration. It minimizes playback disruption and compatibility risk; hardware testing should validate the default-on behavior before considering immediate reconfiguration or vendor-specific selection.

Files changed (7) +83 / -30

Enhancement (5) +63 / -0

VideoActivity.javaAdd persisted low-latency video setting +35/-0

Add persisted low-latency video setting

• Adds a default-on low-latency preference and a checkable Video submenu option. The saved value is pushed into each newly created player, while toggles notify users that changes apply on the next video start.

app/src/main/java/com/openipc/pixelpilot/VideoActivity.java

VideoDecoder.hStore thread-safe low-latency decoder state +6/-0

Store thread-safe low-latency decoder state

• Adds a setter and default-enabled atomic flag controlling whether performance keys are added during future decoder configurations.

app/videonative/src/main/cpp/VideoDecoder.h

VideoPlayer.cppBridge low-latency setting through JNI +10/-0

Bridge low-latency setting through JNI

• Adds nativeSetLowLatency to resolve the native player instance and forward the Java option safely when the instance exists.

app/videonative/src/main/cpp/VideoPlayer.cpp

VideoPlayer.hForward low-latency configuration to the decoder +2/-0

Forward low-latency configuration to the decoder

• Exposes a native player setter that delegates the option to its VideoDecoder instance.

app/videonative/src/main/cpp/VideoPlayer.h

VideoPlayer.javaExpose managed low-latency player API +10/-0

Expose managed low-latency player API

• Declares the native JNI method and adds a documented Java setter for controlling decoder tuning before the next configuration.

app/videonative/src/main/java/com/openipc/videonative/VideoPlayer.java

Bug fix (2) +20 / -30

VideoDecoder.cppApply performance keys during decoder configuration +5/-9

Apply performance keys during decoder configuration

• Removes duplicated commented tuning keys and conditionally invokes the shared MediaFormat helper after H.264 or H.265 format setup. This ensures enabled tuning reaches MediaCodec when a decoder is created.

app/videonative/src/main/cpp/VideoDecoder.cpp

AndroidMediaFormatHelper.hConsolidate AOSP and vendor decoder tuning keys +15/-21

Consolidate AOSP and vendor decoder tuning keys

• Makes the performance helper file-local and adds generic, Qualcomm, HiSilicon, and RTC low-latency keys alongside realtime priority. Removes dead commented helper calls from codec-specific format builders.

app/videonative/src/main/cpp/helper/AndroidMediaFormatHelper.h

[qodo-free-for-open-source-projects]

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0)

Great, no issues found!

Qodo reviewed your code and found no material issues that require review

Tip of the day

💡 Did you know, you can ask Qodo to dismiss a finding you disagree with, with your reason on record

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗