Skip to content

Render the main video on a SurfaceView unless object detection needs the TextureView #114

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
vertexodessa merged 2 commits into from
Sep 2, 2026
Merged

Render the main video on a SurfaceView unless object detection needs the TextureView #114

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
8ecdea0
Render the main video on a SurfaceView unless object detection needs …
iflyhere Aug 27, 2026
22e3015
Pick the renderer on the same condition that decides whether detectio…
iflyhere Sep 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 43 additions & 2 deletions app/src/main/java/com/openipc/pixelpilot/VideoActivity.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -134,6 +134,9 @@ public void run() {
private Timer recordTimer = null;
private int seconds = 0;
private boolean isVRMode = false;
// Which view the main video is rendered into. SurfaceView is the low latency
// default; the TextureView is only used when object detection needs getBitmap().
private boolean videoUsesTextureView = false;
private ConstraintLayout constraintLayout;
private ConstraintSet constraintSet;
private WfbNgLink wfbLink;
Expand Down Expand Up @@ -395,6 +398,7 @@ private void initializeVideoPlayers() {
*/
private void setupVRVideoPlayers() {
binding.mainVideo.setVisibility(View.GONE);
binding.mainVideoSurface.setVisibility(View.GONE);
binding.surfaceViewLeft.getHolder().addCallback(videoPlayer.configure1(0));
binding.surfaceViewRight.getHolder().addCallback(videoPlayer.configure1(1));
}
Expand All @@ -405,7 +409,30 @@ private void setupVRVideoPlayers() {
private void setupStandardVideoPlayer() {
binding.surfaceViewRight.setVisibility(View.GONE);
binding.surfaceViewLeft.setVisibility(View.GONE);
binding.mainVideo.setSurfaceTextureListener(videoPlayer.configureTextureView(0));

// Object detection reads frames back with TextureView.getBitmap(), which forces
// the video through the view hierarchy's GPU composition. Without it a
// SurfaceView is used so the video stays on a hardware overlay plane.
//
// The preference alone is not enough: setObjectDetectionEnabled() turns detection
// back off in onResume when the runtime or the selected model is missing, and returns
// before the renderer swap - which would leave the session on the TextureView with
// nothing reading from it. Both checks are cheap when od_enabled is false, and when
// it is true the runtime check only loads a library that is about to be used anyway.
videoUsesTextureView = getSharedPreferences("general", MODE_PRIVATE)
.getBoolean("od_enabled", false)
&& isObjectDetectionRuntimeSupported()
&& isSelectedObjectDetectionModelAvailable();

if (videoUsesTextureView) {
binding.mainVideoSurface.setVisibility(View.GONE);
binding.mainVideo.setVisibility(View.VISIBLE);
binding.mainVideo.setSurfaceTextureListener(videoPlayer.configureTextureView(0));
} else {
binding.mainVideo.setVisibility(View.GONE);
binding.mainVideoSurface.setVisibility(View.VISIBLE);
binding.mainVideoSurface.getHolder().addCallback(videoPlayer.configure1(0));
}
}

// ----------------------------------------------------------------------------
Expand Down Expand Up @@ -1602,6 +1629,7 @@ public void onVideoRatioChanged(final int videoW, final int videoH) {
Log.d(TAG, "Set resolution: " + videoW + "x" + videoH);

updateViewRatio(R.id.mainVideo, lastVideoW, lastVideoH);
updateViewRatio(R.id.mainVideoSurface, lastVideoW, lastVideoH);
updateViewRatio(R.id.surfaceViewLeft, lastVideoW, lastVideoH);
updateViewRatio(R.id.surfaceViewRight, lastVideoW, lastVideoH);
}
Expand Down Expand Up @@ -2043,7 +2071,19 @@ private void setObjectDetectionEnabled(boolean enabled) {
}

isObjectDetectionEnabled = enabled;
prefs.edit().putBoolean("od_enabled", enabled).apply();
// commit(), not apply(): the restart below ends the process with System.exit()
// before an asynchronous write would be flushed, and the renderer picked on the
// next launch is read from exactly this value.
prefs.edit().putBoolean("od_enabled", enabled).commit();

// Enabling / disabling detection swaps the main video renderer. Handing the
// decoder a different surface at runtime would need the receiver lifecycle in
// VideoPlayer reworked, so restart instead - same as the VR mode toggle does.
if (!isVRMode && enabled != videoUsesTextureView) {
Toast.makeText(this, "Restarting to switch video renderer...", Toast.LENGTH_SHORT).show();
resetApp();
Comment on lines +2082 to +2084

@qodo-free-for-open-source-projects qodo-free-for-open-source-projects Bot Aug 27, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Action required

1. Restart can lose toggle 🐞 Bug ☼ Reliability

The new renderer-switch branch calls resetApp() immediately after od_enabled was saved with
asynchronous apply(), and resetApp() terminates the process with System.exit(0). If the disk
write has not completed, the relaunched activity reads the old value and selects the old renderer,
so the user's toggle appears to fail.
Agent Prompt 
## Issue description
The renderer restart can terminate the process before the asynchronous `od_enabled` preference write reaches disk, causing the app to restart with the old renderer.

## Issue Context
`resetApp()` immediately starts a fresh task and calls `System.exit(0)`. The existing VR restart path uses synchronous `commit()` before invoking the same reset helper.

## Fix Focus Areas
- app/src/main/java/com/openipc/pixelpilot/VideoActivity.java[2004-2013]
- app/src/main/java/com/openipc/pixelpilot/VideoActivity.java[202-210]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools

return;
}

if (enabled) {
binding.detectionOverlay.setVisibility(View.VISIBLE);
Expand All @@ -2064,6 +2104,7 @@ private void restartObjectDetector() {

private void startObjectDetectionLoop() {
if (isVRMode) return; // Standard mode only
if (!videoUsesTextureView) return; // getBitmap() needs the TextureView renderer
if (objectDetectionExecutor == null) {
objectDetectionExecutor = Executors.newSingleThreadExecutor();
}
Expand Down
14 changes: 14 additions & 0 deletions app/src/main/res/layout/activity_video.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,24 @@
android:layout_width="match_parent"
android:layout_height="match_parent">

<SurfaceView
android:id="@+id/mainVideoSurface"
android:layout_width="0dp"
android:layout_height="0dp"
app:layout_constraintBottom_toBottomOf="parent"
app:layout_constraintDimensionRatio="16:9"
app:layout_constraintEnd_toEndOf="parent"
app:layout_constraintStart_toStartOf="parent"
app:layout_constraintTop_toTopOf="parent" />

<!-- Only used when object detection is enabled: TextureView.getBitmap() is how
frames are handed to MediaPipe. It costs a GPU composition pass and an extra
frame of latency compared to mainVideoSurface, so it stays hidden otherwise. -->
<TextureView
android:id="@+id/mainVideo"
android:layout_width="0dp"
android:layout_height="0dp"
android:visibility="gone"
app:layout_constraintBottom_toBottomOf="parent"
app:layout_constraintDimensionRatio="16:9"
app:layout_constraintEnd_toEndOf="parent"
Expand Down