Skip to content

Harden background sync against stalls and add diagnostics - #186

Open
anod wants to merge 6 commits into
masterfrom
fix-stalled-background-sync
Open

anod wants to merge 6 commits into
masterfrom
fix-stalled-background-sync

Conversation

@anod

@anod anod commented Oct 26, 2026 •

Copy link
Copy Markdown
Owner

Problem

Beta users (and the maintainer) stop receiving update notifications until the app is opened. On-device refresh history shows every multi-day gap starting with a scheduled sync row that started but never finished.

Likely cause (not yet confirmed): a scheduled sync blocks forever on AccountManager getAuthToken(...).result (no timeout, not cancellable) while holding the PlaySessionCoordinator mutex, so every later sync waits on the lock.

Fixes

  • Auth token requests are bounded (60s) and interruptible via runInterruptible; OperationCanceledException caused by cancellation is surfaced as cancellation.
  • Google Drive silent sign-in (Tasks.await with no timeout) runs interruptibly so the sync cap can stop it.
  • OkHttp callTimeout of 2 minutes (DFE calls are enqueued and also cancelled with the coroutine).
  • Single shared session gate (cancellable wait, no acquisition timeout, explicit owner); active sync capped at 6 minutes, under WorkManager's 10-minute limit. The cap is passed explicitly via Koin.

Diagnostics

  • New history status: "Interrupted before finishing", recorded on in-process cancellation/timeout without overwriting an already recorded result.
  • Interrupted-sync Crashlytics non-fatals include duration, WorkManager stop reason, session holder/stage timings, and a failureKind label.
  • Stage, session and notification-decision logging under PlaySession, UpdateCheck, SyncWorker, and Notification.

Opt-in on-device diagnostic logs moved to #187.

Release

  • versionCode 17205; release baseline profiles regenerated on a physical device.

Tests

  • Unit coverage for token timeout/cancellation, serialized sessions, and interruption recording.
  • :app:testDebugUnitTest 235/235 passing; :app:assembleDebug and main ktlint clean.

Copilot AI balanced review requested due to automatic review settings October 26, 2026 20:05

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Cancellation races can leak the session lock or duplicate history rows, while blocking Drive maintenance can outlast the sync deadline.

3 open issues
What changed in this PR

This PR hardens AppWatcher’s background update checks against stalls and adds diagnostics to investigate gaps in update notifications.

Changes:

  • Adds bounded token requests, HTTP calls, session-lock waits, and sync execution, with cancellation handling.
  • Records interrupted and abandoned syncs, exposes new history statuses, and logs session stages and worker stop reasons.
  • Adds regression tests, increments versionCode to 17205, and refreshes the release startup profile.
File Description
app/​src/​test/​java/​com/​anod/​appwatcher/​sync/​UpdateCheckRegistrationTest.kt Tests timeout, cancellation, busy-session, and abandoned-history handling.
app/​src/​test/​java/​com/​anod/​appwatcher/​accounts/​AuthTokenBlockingTest.kt Tests token interruption and session-wait timeout recovery.
app/​src/​release/​generated/​baselineProfiles/​startup-prof.txt Refreshes the generated release startup profile.
app/​src/​main/​java/​com/​anod/​appwatcher/​sync/​UpdateCheck.kt Adds sync deadlines, interruption recording, and stage diagnostics.
app/​src/​main/​java/​com/​anod/​appwatcher/​sync/​SyncWorker.kt Logs worker lifecycle and supplies stop reasons.
app/​src/​main/​java/​com/​anod/​appwatcher/​sync/​SchedulesHistoryScreen.kt Displays interrupted and busy-session statuses.
app/​src/​main/​java/​com/​anod/​appwatcher/​database/​SchedulesTable.kt Adds unfinished-history lookup and conditional interruption updates.
app/​src/​main/​java/​com/​anod/​appwatcher/​database/​entities/​Schedule.kt Defines the new failure statuses.
app/​src/​main/​java/​com/​anod/​appwatcher/​CrashlyticsExceptionFilter.kt Keeps interruption diagnostics reportable.
app/​src/​main/​java/​com/​anod/​appwatcher/​AppModule.kt Configures a two-minute OkHttp call timeout.
app/​src/​main/​java/​com/​anod/​appwatcher/​accounts/​AuthTokenBlocking.kt Bounds token waits and makes blocking requests interruptible.
app/​src/​main/​java/​com/​anod/​appwatcher/​accounts/​AuthAccountInitializer.kt Adds timed session acquisition and holder-stage diagnostics.
app/​build.gradle.kts Increments the release build number.

Review effort: Balanced

Comment on lines +160 to +162
withTimeoutOrNull(timeouts.syncMillis) {
performSerialized(manualSync, stopReason)
} ?: -1
Comment on lines +228 to +230
val recorded = if (schedule.id == -1L) {
SchedulesTable.Queries.save(interrupted, database)
true
Copilot AI balanced review requested due to automatic review settings October 27, 2026 20:38

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved concurrency and cancellation issues can leave callers suspended indefinitely, duplicate sync history, and block UI work, while a missing test dependency causes a timeout.

6 open issues
1 resolved since last review

Review effort: Balanced

Copilot AI balanced review requested due to automatic review settings October 27, 2026 21:30

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment on lines +163 to +165
withTimeoutOrNull(timeouts.syncMillis) {
performSerialized(manualSync, stopReason)
} ?: -1
Copilot AI balanced review requested due to automatic review settings October 28, 2026 17:37

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot AI balanced review requested due to automatic review settings October 28, 2026 19:19

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment on lines +157 to +158
withTimeoutOrNull(timeouts.syncMillis) {
performSerialized(manualSync, stopReason)
AppLog.i("Perform Google Drive sync", "UpdateCheck")
val googleAccount = signIn.signInLocked()
// Silent sign-in blocks on a Play services task without a timeout; let the sync cap interrupt it.
val googleAccount = runInterruptible(Dispatchers.IO) { signIn.signInLocked() }
Copilot AI added 5 commits October 30, 2026 16:17
- Bound AccountManager token waits and make them interruptible
- Add OkHttp call timeout, session wait timeout, and overall sync timeout
- Record interrupted/busy syncs in history and report them to Crashlytics
- Log sync stages, session holder, and WorkManager stop reason
- Bump versionCode to 17205 and regenerate baseline profiles

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- Remove timed session acquisition and busy status; keep single gate,
  bounded token/HTTP calls and active sync cap
- Add account-allowlisted persistent diagnostic recorder (30 days, 10 MiB)
  with Settings toggle, share and delete
- Record scheduler, worker, session, sync stage and startup state events
- Pass diagnostic recorder explicitly to sync and settings components

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- Bind diagnostics consent to the account hash in Preferences
- Make recorder operations non-throwing and self-refreshing
- Reduce SettingsViewModel diagnostics handling and reuse StartActivity
- Fix missing diagnostics dependency in account selection test

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Diagnostics now tee AppLog output to a local file while the allowlisted
account has opted in, instead of recording structured events from sync,
session and notification code. The tee is installed only when enabled,
so disabled or disallowed builds keep the plain logger.

Removes abandoned-sync detection; interrupted runs are still recorded.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- Remove opt-in diagnostics (now an independent PR against master)
- Require explicit session owner, sync timeouts, and log release before unlock
- Make stage updates non-suspend and label chunks by position
- Run Drive silent sign-in interruptibly so the sync cap can stop it

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 30, 2026 14:26
@anod
anod force-pushed the fix-stalled-background-sync branch from 6b92c69 to 7c5d900 Compare October 30, 2026 14:26

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Wrap the sync body in withTimeout inside performSerialized so a real timeout is detected through TimeoutCancellationException instead of a duration heuristic. The heuristic compared against a clock that starts after the timer, so a timeout could be labelled a cancellation. The interruption is still recorded under NonCancellable and a timeout still returns -1.

In the stalled-sync test, use the 300 ms capped UpdateCheck only for the stalling sync. The recovery sync now runs under the regular cap and no longer times out on slow CI runners.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 30, 2026 19:38

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Cancellation handling remains unsafe: silent sign-in can turn interruption into an ordinary failure, history insertion races can create duplicate and unfinished rows, and blocking Drive transfers can exceed the six-minute cap.

6 open issues

Review effort: Balanced

Comment on lines +165 to +167
withTimeout(timeouts.syncMillis) {
syncSerialized(schedule, manualSync)
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants