Skip to content

EXP: passive GitHub-hosted runner capability census - #16

Merged
mark-e-deyoung merged 47 commits into
mainfrom
experiment/github-runner-capability-census
Sep 20, 2026
Merged

mark-e-deyoung merged 47 commits into
mainfrom
experiment/github-runner-capability-census

Conversation

@mark-e-deyoung

Copy link
Copy Markdown
Contributor

Public-safe execution half of SemperSupra/agent-dispatch-private#223.

Adds:

  • dependency-free passive runner observer;
  • contract tests preserving presence != exercised/oracle-proven;
  • bounded Actions matrix across current distinct Linux/macOS public runner labels;
  • max-parallel 2 and 10-minute job bounds;
  • 7-day public-safe receipt artifacts;
  • explicit public boundary documentation.

The probe is read-only: no package installation, device opening, service startup, scanning, bandwidth testing, metadata probing, or environment-variable dump. It records only ephemeral runner facts and selected capability presence.

This PR is intentionally draft while the first matrix evidence is collected. It does not contain private placement policy, target metadata, evaluator logic, or provider/model information.

Copy link
Copy Markdown
Contributor Author

Wave 1 passive matrix PASS: Actions run 35462448214 completed the contract test plus all eight runner targets successfully. Receipt artifacts were ingested into private experiment authority SemperSupra/agent-dispatch-private#223. Presence remains passive evidence only; no primitive has been promoted to callable/exercised/oracle-proven yet. Next public increment is bounded capacity characterization in fresh jobs.

Copy link
Copy Markdown
Contributor Author

Wave 2 capacity characterization passed on all eight targets (run 35462691076). Passive/capacity workflows are now frozen to explicit workflow_dispatch reruns so subsequent experiment commits do not replay completed waves. Next increment is a fresh-job primitive oracle matrix: x64 KVM API open/ioctl, Linux Docker daemon callability, ARM macOS Rosetta execution, and Apple toolchain/Simulator-runtime query.

Copy link
Copy Markdown
Contributor Author

Wave 3 primitive oracles complete (run 35462904846). Direct-user KVM API open is permission-denied on Ubuntu 24/26 x64 despite /dev/kvm presence; full Ubuntu x64/ARM Docker daemon APIs are proven; ubuntu-slim has CLI but no daemon; Rosetta is proven on ARM macOS; Xcode/clang/Simulator-runtime queries are proven on macOS26 ARM/Intel and xcode-27. Next bounded increment: privileged KVM control only, tiny Docker container nonce+bind mount on full Ubuntu, and disposable Simulator create/boot/query/delete.

Copy link
Copy Markdown
Contributor Author

Frontier workload run 35480722264 complete. Apple iPhoneSimulator SDK compile passed on macos-26 ARM, macos-26-intel, and xcode-27 ARM. Ubuntu 24/26 x64 Android-emulator qualification stopped at a clean setup boundary: SDK root exists and KVM VCPU is already proven, but the emulator binary is not preinstalled. No large downloads were introduced. Private placement evidence has been reconciled in agent-dispatch-private#223.

Copy link
Copy Markdown
Contributor Author

Android setup/accelerated-emulator qualification is now complete on both x64 Ubuntu generations. Ubuntu 26.04 clean rerun 35480993301 and Ubuntu 24.04 run 35481166743 both installed exactly emulator + system-images;android-35;google_apis;x86_64, confirmed KVM v12 acceleration, and booted an x86_64 AVD to sys.boot_completed=1. Observed setup growth was ~4.62 GB. The earlier Ubuntu 26 red run was a post-success cleanup defect from sudo-owned AVD state and has been corrected. Private placement evidence is updated under agent-dispatch-private#223.

Copy link
Copy Markdown
Contributor Author

iOS Simulator app-lifecycle qualification is closed at a bounded negative frontier. Final run 35481468685 on macos-26-intel passed app compile, plist/signature validation, device Booted state, getenv, and guest-spawn, but simctl install of the validated synthetic app bundle timed out at 30 s. Lower Simulator primitives remain proven; the synthetic install/launch path is not promoted. The workflow is now frozen to manual reruns and private placement evidence is updated in agent-dispatch-private#223.

Copy link
Copy Markdown
Contributor Author

Runner capability experiment execution is complete to the review/merge boundary. All 11 experiment workflows are now manual-only (workflow_dispatch) so later branch work cannot replay completed census waves. Public PR #16 contains only public-safe probes/workflows/tests/docs; accepted placement policy remains private. Current private authority and final disposition are recorded in agent-dispatch-private#223 / draft PR #224. No merge/release authority is inferred.

@mark-e-deyoung mark-e-deyoung left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review complete at head 674852a.

No blocking findings.

Reviewed against the experiment acceptance criteria:

  • public/private value boundary preserved;
  • public files are limited to public-safe probes, tests, workflows, and methodology;
  • completed experiment workflows are manual-only (workflow_dispatch);
  • negative capability observations are separated from harness failures;
  • bounded resource controls and fresh-job isolation are retained;
  • no scheduler, ranking service, credential handling, or private placement policy was introduced;
  • empirical workload/oracle evidence is durably referenced in agent-dispatch-private#223.

This PR is ready for merge subject to repository-native merge checks.

@mark-e-deyoung
mark-e-deyoung marked this pull request as ready for review September 20, 2026 07:12
@mark-e-deyoung
mark-e-deyoung merged commit 3fb029a into main Sep 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant