@testingbot/mobilewright-driver

Run Mobilewright mobile tests on TestingBot's device cloud: real iOS/Android devices, iOS simulators and Android emulators.

Requires mobilewright >= 0.0.53 (the first release that accepts driver instances) and Node >= 18.

Quickstart

One command:

npx @testingbot/mobilewright-driver init

answers a few questions and writes mobilewright.config.ts plus a sample test (add --yes for defaults). Or set up manually:

1. Install (in your test project):

npm install --save-dev mobilewright @mobilewright/test @testingbot/mobilewright-driver

2. Get your credentials from testingbot.com/members/user/edit:

export TESTINGBOT_KEY=...
export TESTINGBOT_SECRET=...

3. Create mobilewright.config.ts:

import { defineConfig } from 'mobilewright';
import { TestingBotDriver } from '@testingbot/mobilewright-driver';

export default defineConfig({
  testDir: '.',
  bundleId: 'com.example.MyApp',
  driver: new TestingBotDriver({
    // TestingBot sessions start with the app pre-installed — declare it here.
    // Local paths (.apk / .ipa / simulator .zip) upload automatically.
    apps: {
      // A single path, or an array whose first entry is the app under test
      // and the rest install alongside it (e.g. a mock server or test harness).
      android: './build/app.apk',
      'ios-simulator': './build/app-sim.zip',
      'ios-real': './build/app.ipa', // must be test-signed for real devices
    },
  }),
  projects: [
    { name: 'android', use: { platform: 'android', deviceType: 'emulator' } },
    { name: 'ios', use: { platform: 'ios', deviceType: 'real', osVersion: '>=17' } },
  ],
});

4. Write a test (app.test.ts):

import { test, expect } from '@mobilewright/test';

test('shows the welcome screen', async ({ device, screen, bundleId }) => {
  await device.launchApp(bundleId);
  await expect(screen.getByText('Welcome')).toBeVisible();
});

5. Run:

npx mobilewright test

Each device slot maps to a TestingBot session on your dashboard, with video and pass/fail reported automatically. Prefix with DEBUG=testingbot:* to see everything the driver does.

Selecting devices

Device selection comes from the mobilewright project config (use block):

Key	Example	Notes
platform	'ios' | 'android'	required
deviceType	'real' | 'simulator' | 'emulator'	default: virtual device
deviceName	/iPhone 1[45]/, /pixel/i	regex, matched against TestingBot's device catalogs
osVersion	'17', '17.4', '>=16 <18'	exact, prefix, or range expression
deviceId	'2241'	pin one specific physical device (id from GET /v1/devices)

Regexes and version ranges work on both real and virtual devices: real devices resolve against the live physical catalog (/v1/devices — idle devices preferred; if the chosen device is busy, TestingBot queues the session until it frees up), virtual ones against the simulator/emulator catalog (/v1/browsers, newest matching OS version wins).

Sessions, naming, and reporting

• By default a device slot's TestingBot session is reused across tests (fast: you pay device startup once). Sessions that hosted exactly one test are named after that test with its own pass/fail; sessions that hosted several get a run summary (e.g. 12/12 tests passed).
• Set sessionPerTest: true for one session, video, and named verdict per test — the clearest dashboard, at the cost of device startup on every test.
• Git metadata is reported automatically when mobilewright captures it: the commit hash, branch, author and subject land in the session's extra field (alongside your own extra, never replacing it), and the branch is added as a tag so the dashboard can filter by it. Enable it with captureGitInfo: { commit: true } in mobilewright.config.ts — Playwright turns it on by itself in CI.
• Failure messages are reported to the session's status message; build groups sessions on the dashboard and defaults automatically from CI environments (GitHub Actions, GitLab, CircleCI, Buildkite, Bitrise, Travis, Azure DevOps, Jenkins, TeamCity → e.g. owner/repo #123), or from a TESTINGBOT_BUILD variable.

All options

new TestingBotDriver({
  apps: { android: './build/app.apk' }, // app under test, most specific key wins
                                        // ('ios-real' beats 'ios'); tb:// URLs allowed.
                                        // Array = app under test first, helper apps after:
                                        //   android: ['./app.apk', './mock-server.apk']
  key: '...',                 // default: TESTINGBOT_KEY env
  secret: '...',              // default: TESTINGBOT_SECRET env
  sessionPerTest: false,      // true = fresh session (and video) per test
  name: 'checkout flow',      // session name (default 'mobilewright'; per-test names override)
  build: 'ci-1234',           // default: auto-detected from CI env / TESTINGBOT_BUILD
  testResults: 'on',          // 'off' disables pass/fail reporting

  allocationTimeout: 395_000, // ms to wait for a device (TestingBot queues busy devices)
  idleTimeout: 230,           // seconds before TestingBot reaps an idle session
  maxDuration: 1800,          // max session length in seconds
  appiumVersion: 'latest',    // pin the Appium version TestingBot uses

  timeZone: 'Europe/Brussels',   // device timezone (tz database name)
  geoCountryCode: 'DE',          // route device traffic via a proxy in this country
  throttleNetwork: '3G',         // or { downloadSpeed, uploadSpeed, latency, loss }
  autoGrantPermissions: true,    // Android: auto-grant app permission dialogs
  autoAcceptAlerts: true,        // iOS: auto-accept system permission alerts
  screenshots: true,             // screenshot at every step (default false)
  video: true,                   // session video (default true)
  recordLogs: 'strip-parameters',// command logs: true | false | 'strip-parameters'
  public: false,                 // make results publicly accessible
  extra: 'commit=abc123',        // custom metadata on the test detail page
  tabletOnly: false,             // restrict allocation to tablets
  phoneOnly: false,              // ...or to phones

  capabilities: {},           // extra Appium capabilities (escape hatch)
  tbOptions: {},              // extra tb:options entries (escape hatch)
});

TestingBot commands inside tests

Some TestingBot features are runtime commands rather than session capabilities. Import the testingbot helper and call them from a test body — it targets the device session mobilewright connected for that test:

import { test, expect } from '@mobilewright/test';
import { testingbot } from '@testingbot/mobilewright-driver';

test('checkout survives a slow network', async ({ device, screen, bundleId }) => {
  await device.launchApp(bundleId);

  await testingbot.annotate('starting checkout');   // shows in the session timeline
  await testingbot.throttle('3G');
  await expect(screen.getByText('Order placed')).toBeVisible();

  console.log(testingbot.dashboardUrl());
});

Available commands

Command	TestingBot command	What it does
throttle(conditions)	tb:throttle	Network conditions mid-test: 'Edge', '3G', '4G', 'airplane', 'disable', or { downloadSpeed, uploadSpeed, latency, loss } (kb/s, ms). Reset automatically at the end of the test.
annotate(text)	tb:test-context	Logs a step into the session's command list, so the timeline shows what the test was doing.
setName(name)	tb:test-name	Names the session on the dashboard.
setBuild(build)	tb:test-build	Groups the session under a build.
setTags([...])	tb:test-tags	Tags the session.
setResult(passed)	tb:test-result	Overrides the pass/fail verdict (the driver reports it automatically — use this only for outcomes mobilewright cannot see).
updateInfo({...})	tb:test-info	Bulk update: name, build, public, statusMessage, extra.
breakpoint()	tb:break	Pauses the session for manual inspection with a live view. Debugging only — never leave it in CI.
shell(command, args)	mobile: shell	ADB shell on Android (whitelisted subset on physical devices).
execute(script, ...args)	any	Escape hatch for any Appium or TestingBot execute-script command.
sessionId() / dashboardUrl()	—	This test's TestingBot session id and dashboard link.

Notes:

• Throttling is reset automatically when the test's device session is released, so slow-network conditions never leak into the next test that reuses the device.
• Calling a command outside a test body raises an explanatory error rather than a null-reference.

Recipe: name every session after its test

test.beforeEach(async ({ device }, testInfo) => {
  await testingbot.setName(testInfo.title);
  await testingbot.setTags([testInfo.project.name]);
});

Requesting the device fixture is what makes the session exist before the hook runs. With sessionPerTest: true each session gets exactly one name; in pooled mode the last test to run on a session wins (the driver still reports per-test verdicts afterwards — see above).

Testing localhost / staging servers (TestingBot Tunnel)

Either run a TestingBot Tunnel yourself and pass its identifier:

driver: new TestingBotDriver({ tunnelIdentifier: 'my-tunnel' })

or let the driver manage one around the run (requires Java and npm install --save-dev testingbot-tunnel-launcher):

driver: new TestingBotDriver({ tunnel: true })                    // anonymous tunnel
driver: new TestingBotDriver({ tunnel: { identifier: 'ci-42' } }) // named tunnel

The driver starts the tunnel before any device is allocated and closes it after the run.

Running in CI (GitHub Actions example)

jobs:
  mobile-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 20, cache: npm }
      - run: npm ci
      - run: npx mobilewright test
        env:
          TESTINGBOT_KEY: ${{ secrets.TESTINGBOT_KEY }}
          TESTINGBOT_SECRET: ${{ secrets.TESTINGBOT_SECRET }}
          # build name auto-detects as "owner/repo #run" — no config needed

Troubleshooting

Symptom	Cause / fix
TestingBot credentials are missing	Set TESTINGBOT_KEY / TESTINGBOT_SECRET, or pass { key, secret }.
TestingBot sessions must start with an app	Declare the app in the driver's apps option (per platform / device type).
No TestingBot real device matches ... / No TestingBot simulator matches ...	The criteria matched nothing in the catalog — check available devices, loosen osVersion/deviceName.
Allocation waits for minutes	The matching device is busy; TestingBot queues the session until it frees up (bounded by allocationTimeout).
Test timeout ... while setting up "device"	More workers than your plan's parallel limit: TestingBot queues up to 2× the limit server-side and the driver waits out the rest, all inside the test-scoped device fixture. The driver warns with your plan's number — set workers to at most that, or raise the mobilewright timeout.
... is a simulator-only build and cannot be installed on a real device	Build a test-signed .ipa for deviceType: 'real' projects.
"..." is not installed on this device	The config's bundleId is not the package id of the build in apps (check with aapt2 dump packagename app.apk), or a stale bundleId/env override is in play.
cannot install "..." mid-session	TestingBot has no mid-session installs — every app in installApps must also be listed in the driver's apps option (first entry = app under test, the rest install as helper apps).
Modifier chords fail on iOS	XCUITest cannot hold modifier keys; chords are Android-only (clearText() handles iOS differently).
Sessions show the run summary instead of a test name	The session hosted several tests; use sessionPerTest: true for guaranteed per-test naming.
DEBUG=testingbot:*	Prints every allocation, hub command, upload, and report the driver performs.

How it maps to TestingBot

mobilewright concept	TestingBot implementation
device allocation	Appium session on hub.testingbot.com (the session UUID is the device id); busy devices queue server-side
app install	uploaded to TestingBot storage at allocation time and passed as appium:app, helper apps as appium:otherApps (re-signed automatically for real iOS) — no mid-session installs
taps/swipes/gestures	W3C pointer actions
view hierarchy	Appium page source, mapped to mobilewright's ViewNode tree
webviews	Appium contexts (webViewBridge)
pass/fail reporting	PUT /v1/tests/:session via the TestObserver hooks
git metadata	commit/branch/author/subject from the run report → session extra + branch tag
video	recorded by TestingBot; startRecording({ output }) downloads the MP4 locally at run end

Current limitations

• One TestingBot session may host several mobilewright tests unless sessionPerTest: true; multi-test sessions report the run-level verdict.
• Modifier key chords (pressKeys(['ctrl+a'])) work on Android only — XCUITest cannot hold modifier keys.
• pressButton on iOS supports HOME, VOLUME_UP, VOLUME_DOWN; listApps() reports the foreground app only.
• Screenshots are always PNG. applyDeviceSettings turns Android animations off best-effort via mobile: shell (a no-op on iOS).
• Real iOS devices need a test-signed .ipa; simulator builds are rejected on real devices with a clear error.
• While attached to a webview, the session's single Appium context is the web layer.

Development

npm install
npm test          # unit + FakeHub integration tests (no credentials needed)
npm run build

Live e2e tests against a real TestingBot account live in e2e/ (see e2e/mobilewright.config.ts).

Releasing

npm version minor -m "Release v%s"      # patch | minor | major
git push --follow-tags
gh release create "v$(node -p "require('./package.json').version")" --generate-notes

License

Apache-2.0