Skip to content

test(#1611): Android emulator instrumented tests for the Kotlin exact-ref model host (Spec 138 0.13.1) - #1647

Merged
enricopiovesan merged 2 commits into
mainfrom
claude/issue-1611-android-emulator-tests
Oct 9, 2026
Merged

enricopiovesan merged 2 commits into
mainfrom
claude/issue-1611-android-emulator-tests

Conversation

@enricopiovesan

@enricopiovesan enricopiovesan commented Oct 9, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Closes the gap Decision 108 left open: the Kotlin exact-ref model host now runs on a real Android runtime in CI, not only on the host-JVM build of traverse-android-host.

  • New android-emulator-tests CI job (ubuntu-latest, KVM):

    • cross-builds traverse-android-host with cargo-ndk (the AAR's arm64-v8a/x86_64 libraries, export and 16 KiB-page checks included);
    • boots an x86_64 API 34 google_apis emulator with reactivecircus/android-emulator-runner;
    • runs connectedDebugAndroidTest.

    Flakiness is mitigated with a cached AVD and boot snapshot (actions/cache), -no-snapshot-save for a clean state, a headless emulator with animations disabled, the Orchestrator, and test reports uploaded on failure. The timeout is 45 minutes.

  • ExactModelHostInstrumentedTest (androidTest) loads traverse_android_host through System.loadLibrary from jniLibs, as an app does. It covers:

    • the signed vectors byte-for-byte: classifier, digits-mlp, digits-onnx (+simd128), and prepared-v3 over 3 rounds (fresh, then snapshot);
    • the shared 21-case rights suite, with JSON equality on codes, reasons, details and evidence;
    • missingNativeLibraryFailsClosed: pointing the loader at a nonexistent library, before ExactModelNative initialises, fails the load as a stripped .so would, so every model call gives model_unavailable / engine_unavailable.

    The fixtures are fixtures/models, bundled as test-APK assets.

  • Android Test Orchestrator with clearPackageData, so every test runs in its own process and the forced load failure can't leak into the others.

  • Docs: docs/android-emulator-tests.md covers what runs, the runtime, the flakiness mitigations, and how to run it locally.

  • Spec 138 0.13.0 → 0.13.1 amends FR-048, whose text said the host-JVM build was used "until Android-emulator coverage lands (Android emulator instrumented tests for Kotlin exact-ref model execution (Decision 108 follow-up) #1611)". Testing only; no behaviour change.

Governing Spec

  • 138-governed-exact-model-execution
  • 068-public-platform-embedder-packages
  • 031-supply-chain-hardening
  • 060-wasm-resource-limits
  • 004-spec-alignment-gate

Project Item

Validation

  • This machine has no JDK or Android SDK, so the instrumented tests and the emulator job are verified by this PR's CI (android-emulator-tests). The test logic is a port of the passing host-JVM ExactModelHostTest (same vectors, same suite assertions); only the fixture source (APK assets) and the library loading differ.
  • The workflow YAML parses. spec_alignment_check.sh and pr_body_check.sh pass.
  • No Rust or runtime code changed. Every new fixture referenced (conformance/signed-prepared-v3.json, rights-conformance/) is already on main.

🤖 Generated with Claude Code

enricopiovesan and others added 2 commits October 9, 2026 11:14
…-ref model host

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…mented tests

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@enricopiovesan
enricopiovesan force-pushed the claude/issue-1611-android-emulator-tests branch from 803a576 to 145c418 Compare October 9, 2026 17:15
@enricopiovesan enricopiovesan changed the title test(#1611): Android emulator instrumented tests for the Kotlin exact-ref model host (Spec 138 0.12.2) test(#1611): Android emulator instrumented tests for the Kotlin exact-ref model host (Spec 138 0.13.1) Oct 9, 2026
@enricopiovesan
enricopiovesan enabled auto-merge (squash) October 9, 2026 17:35
@enricopiovesan
enricopiovesan merged commit 34c7a59 into main Oct 9, 2026
44 checks passed
@enricopiovesan
enricopiovesan deleted the claude/issue-1611-android-emulator-tests branch October 9, 2026 17:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Android emulator instrumented tests for Kotlin exact-ref model execution (Decision 108 follow-up)

1 participant