Skip to content

[build-tools] Let serve-sim start the iOS Simulator for agent-device - #4544

Merged
gwdp merged 4 commits into
mainfrom
gwdp/serve-sim-agent-device-boot
Oct 16, 2026
Merged

gwdp merged 4 commits into
mainfrom
gwdp/serve-sim-agent-device-boot

Conversation

@gwdp

@gwdp gwdp commented Oct 6, 2026 •

Copy link
Copy Markdown
Contributor

Why

Boot the Simulator while the app downloads, and let serve-sim install and launch it for agent-device sessions.

How

  • Start serve-sim alongside download and daemon startup, using the selected Simulator.
  • Check the egress guard before install/launch. Finish Simulator setup before launching; keep launch arguments and the deep link.

Release serve-sim with the app-control API before deploying this worker change. Android keeps its existing path.

Test Plan

  • Required CI, Node 24 and fresh Devin analysis passed on af09740b. Codex ran 406 focused tests after the main rebase; typecheck, changed-file lint and formatting passed.
  • Earlier local launch E2E passed: Universe-generated steps through build-tools download/install/start to pinned serve-sim on a real iOS Simulator. Verified one app start, arguments and deep link. Cloud calls used loopback adapters; hosted and current-head device runs remain pending.
  • Earlier local CLI/Universe worker acceptance used serve-sim 0.7.0-canary-20261009-782fe97; the app reached its HTTP origin. Guarded acceptance on this head is pending.

Cold startup benchmark

Session Main median Stack median Median paired saving
agent-device 34.80s 33.87s 0.37s (1.1%)

Agent-device already overlaps boot, download and daemon startup on main, so there was less new parallel work to unlock than for Argent/Appium. In these trials, the fixture downloaded in milliseconds and native installation took longer (12.29s → 15.34s median), offsetting the earlier install start. The cause of that extra installer work was not established.

Codex measured four alternating pairs per type on an initialized iPhone 17 Pro Simulator (iOS 26.4, Tart 6 CPU / 8 GiB), starting shut down with the app absent and packages cached. Timing runs from worker entry to local session reporting, using real controllers and native boot/install/launch with loopback cloud adapters.

This compares the completed startup stack e0b7462 against main 4aca8b1, with the same pinned local serve-sim #260 build; it does not isolate this PR's contribution. Hosted dispatch and first browser frame were not measured.

Savings are the median of paired differences, not the difference between the two arm medians. Individual pairs ranged from 1.51s slower to 5.29s faster. This sample does not establish a consistent speedup or a no-slowdown guarantee.

Authorship and LLM use

Gabe wrote the initial implementation and guided the design. Codex implemented and split the changes. AI review led to fixes for startup order and cleanup. Gabe reviewed the final changes.

@gwdp

This comment has been minimized.

@codecov

codecov Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 74.81%. Comparing base (60a0b48) to head (84a2d58).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4544      +/-   ##
==========================================
+ Coverage   74.79%   74.81%   +0.02%     
==========================================
  Files        1039     1039              
  Lines       51104    51135      +31     
  Branches    11130    11140      +10     
==========================================
+ Hits        38217    38250      +33     
+ Misses      11896    11894       -2     
  Partials      991      991              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@gwdp gwdp changed the title [build-tools] Let serve-sim boot agent-device iOS sessions [build-tools] Let serve-sim boot and launch agent-device iOS sessions Oct 6, 2026
@gwdp

This comment has been minimized.

@gwdp
gwdp force-pushed the gwdp/serve-sim-agent-device-boot branch from aca69f5 to e589453 Compare October 9, 2026 09:12
@gwdp gwdp changed the title [build-tools] Let serve-sim boot and launch agent-device iOS sessions [build-tools] Let serve-sim boot, install and launch agent-device iOS sessions Oct 9, 2026
@gwdp
gwdp changed the base branch from gwdp/serve-sim-server-log-artifact to gwdp/serve-sim-argent-logs October 9, 2026 09:12
@gwdp

This comment has been minimized.

@gwdp
gwdp force-pushed the gwdp/serve-sim-agent-device-boot branch from e589453 to 9f038b4 Compare October 9, 2026 19:49
@gwdp
gwdp force-pushed the gwdp/serve-sim-agent-device-boot branch from 9f038b4 to 5cad245 Compare October 9, 2026 23:46
@gwdp
gwdp force-pushed the gwdp/serve-sim-agent-device-boot branch from 5cad245 to d982e75 Compare October 10, 2026 00:25
@gwdp gwdp changed the title [build-tools] Let serve-sim boot, install and launch agent-device iOS sessions [build-tools] Let serve-sim start agent-device iOS sessions Oct 10, 2026
@gwdp
gwdp changed the base branch from gwdp/serve-sim-argent-logs to gwdp/serve-sim-capture-egress-handoff October 10, 2026 00:25
@gwdp
gwdp added this pull request to stack #4599 October 10, 2026 00:26
@gwdp
gwdp force-pushed the gwdp/serve-sim-agent-device-boot branch from d982e75 to 238427d Compare October 10, 2026 08:37
@gwdp
gwdp removed this pull request from stack #4599 October 10, 2026 08:38
@gwdp
gwdp changed the base branch from gwdp/serve-sim-capture-egress-handoff to gwdp/serve-sim-web-preview-startup October 10, 2026 08:39
@gwdp
gwdp force-pushed the gwdp/serve-sim-agent-device-boot branch from 238427d to 7e309fd Compare October 10, 2026 08:39
@gwdp
gwdp added this pull request to stack #4604 October 10, 2026 08:40
@gwdp

This comment has been minimized.

@gwdp gwdp changed the title [build-tools] Let serve-sim start agent-device iOS sessions [build-tools] Let serve-sim start the iOS Simulator for agent-device Oct 10, 2026
@gwdp
gwdp force-pushed the gwdp/serve-sim-agent-device-boot branch from 7e309fd to 60f377f Compare October 10, 2026 16:25
@gwdp
gwdp force-pushed the gwdp/serve-sim-agent-device-boot branch from 6aa2150 to 2adc5e8 Compare October 10, 2026 20:06
@gwdp

This comment has been minimized.

@gwdp
gwdp force-pushed the gwdp/serve-sim-agent-device-boot branch 3 times, most recently from 266aece to c8e7f58 Compare October 15, 2026 05:09
@gwdp
gwdp force-pushed the gwdp/serve-sim-agent-device-boot branch from c8e7f58 to 472e5f5 Compare October 15, 2026 16:20
@gwdp

gwdp commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor Author

Codex rebased this stack on 4ec2a5fe, preserving the worker-token download API and explicit ngrok credentials. The Argent/Appium follow-ups now stop their new parallel setup on Simulator failure; this layer retains its own patch. Local focused tests, typecheck, changed-file lint and formatting passed. Fresh review/CI is separate from deployed QA of 3e00e86e.

Clarified the existing benchmark paragraph: agent-device already overlaps startup tasks, and slower native installation offset the earlier start in that measured batch. Existing timings and evidence boundaries are unchanged.

Gabe confirmed review of the final changes at af09740b. Codex updated the review record and list formatting, hid completed automated-review comments, marked the PR ready, and requested @szdziedzic as reviewer.

— Codex

Base automatically changed from gwdp/serve-sim-web-preview-startup to main October 15, 2026 16:28
@gwdp
gwdp force-pushed the gwdp/serve-sim-agent-device-boot branch from 472e5f5 to b462637 Compare October 15, 2026 16:28
@gwdp

This comment has been minimized.

@devin-ai-integration

This comment has been minimized.

devin-ai-integration[bot]

This comment was marked as outdated.

@gwdp
gwdp force-pushed the gwdp/serve-sim-agent-device-boot branch from b462637 to af09740 Compare October 15, 2026 18:00
@gwdp

This comment has been minimized.

@devin-ai-integration

This comment has been minimized.

@gwdp
gwdp marked this pull request as ready for review October 15, 2026 19:07
@gwdp
gwdp requested a review from szdziedzic October 15, 2026 19:07
@gwdp
gwdp force-pushed the gwdp/serve-sim-agent-device-boot branch from af09740 to 84a2d58 Compare October 16, 2026 21:38
@gwdp

gwdp commented Oct 16, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto main 60a0b483e; updated head: 84a2d58.

Codex resolved the overlaps with main's private-preview, iframe and CORS changes, preserving those settings alongside serve-sim startup. Both regression test sets remain; no unrelated source changes.

434 focused tests, typecheck across all 13 packages, full-repository lint and formatting pass. Independent Codex and Opus reviews found no rebase regression. Fresh CI is pending; no new hosted/device run was performed for this rebase.

— Codex

@github-actions

Copy link
Copy Markdown

✅ Thank you for adding a changeset!

@gwdp
gwdp merged commit 904859b into main Oct 16, 2026
10 checks passed
@gwdp
gwdp deleted the gwdp/serve-sim-agent-device-boot branch October 16, 2026 21:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants