Skip to content

Migrate the lt dev test DB before the compiled API starts, and stop claiming a reset that never happens - #110

Merged
DKoenig9 merged 2 commits into
mainfrom
fix/dev-3289-migrate-test-db
Sep 21, 2026
Merged

DKoenig9 merged 2 commits into
mainfrom
fix/dev-3289-migrate-test-db

Conversation

@DKoenig9

Copy link
Copy Markdown
Contributor

Fixes DEV-3289.

lt dev test starts the compiled API bundle directly. That bypasses the project's start script, and with it the migration lt projects chain in front of the server:

"start": "pnpm run migrate:up && pnpm run start:local"

The <…>-test database outlives every run, because the CLI never drops it. So it fell behind by one more migration each time. In GEQ a migration made the unique index {email, owner} partial; the test DB never got it, MongoDB refused the differing index Mongoose asked for (IndexOptionsConflict, swallowed at boot), and the Playwright suite failed with a 409 that looked like a bug in the feature under test. A data migration is worse: nothing fails, and the suite tests a data state no environment has. CI and deployments were never affected (fresh DB per job; docker-entrypoint.sh migrates), only the local run right before a push.

The bring-up log made it harder to find: it printed db: … (reset before the suite by Playwright global-setup) for every project. The starter's Playwright setup does no such reset, and neither did GEQ's.

What changed

1. One migration step for every compiled start. applyPendingMigrations() in dev-api-launch.ts runs <pm> run migrate:up and reports applied / skipped (no such script, e.g. nest-base) / failed (non-zero or signal). startCompiledApi (lt dev up --api-compiled) already had this inline and now uses the helper, with unchanged behaviour.

2. The test stack migrates before the API starts. The API half of bringUpTestSession moves into startTestApi(): build → migrate → spawn. The migration runs with the API's own env, because the migration store resolves its database from the same config.env.ts + NSC__MONGOOSE__URI merge, and sharing the env is what makes both hit the same DB. It runs before the spawn, since the API syncs its indexes on boot. Sibling shards (skipBuild) still migrate, because each has its own DB. The start-script fallback does not migrate twice; that script owns it.

3. A failed migration aborts the run, via migrationFailedError, for the same reason DEV-3208 made readiness abort: a suite on an un-migrated DB reports on the setup, not on the feature. lt dev test catches it, tears the stack down and exits 1 before the App is even built.

4. The log states only what the CLI does. The reset claim is gone, and so are the docblocks that called the DB "fresh + discarded per run". docs/commands.md described lt dev test as it worked before the isolated stack existed (it "invokes lt dev up", uses .lt-dev/.env, --teardown) and now describes it as it works today.

Verification

  • Behaviour tests for startTestApi / applyPendingMigrations with a mocked process layer (__tests__/dev-test-session-api.test.ts). Mutation-checked: dropping the migration turns 4 tests red, migrating with process.env instead of the API env turns 2 red.
  • Run end-to-end from this branch against GEQ, whose test DB was one migration behind:
    • Running 1 pending migration(s)… 1789647000000-company-organizations.js after the API build and before the spawn; afterwards gequma-test had it, and gequma-local, which already had it, was untouched. firmendetails.spec.ts: 10 passed.
    • Immediate re-run of the same migrate:up: No pending migrations, exit 0. Follow-up runs do not abort.
    • With a deliberately throwing migration: migrate:up failed (exit 1) against the test DB "gequma-test" — test API not started …, exit 1, no App build, no Playwright, teardown residue-free, failed migration not recorded, lock released.
  • npm run check green: 0 vulnerabilities, lint clean, 1164 tests, compile + CLI start smoke.

Also fixed (pre-existing): waitForHttp › keeps polling while abort stays false blocked the pre-push of this branch on unchanged code. It probed the black-holed TEST-NET address, where each curl probe burns its full --max-time 2. Only two probes fit into the 5s budget, with ~0.4s to spare, and retry skips abort once the budget is spent. It now probes a refusing port instead: measured at load 133, that gives 10 abort calls in 5s against 2 before. Separate commit.

Out of scope, filed as DEV-3310: on Postgres projects the test database and role do not exist at all (P1000). nest-base has no migrate:up script, so this change skips it there and leaves that path as it was.

🤖 Generated with Claude Code

DKoenig9 and others added 2 commits September 21, 2026 12:04
`lt dev test` starts the compiled API bundle directly, which bypasses the
project's `start` script and with it the `migrate:up` that script chains in
front of the server. The `<…>-test` DB outlives every run — the CLI never
drops it — so it fell behind every migration. In GEQ a unique index made
partial by a migration never reached it, and the Playwright suite failed with
a 409 that looked like a bug in the feature (DEV-3289).

- dev-api-launch.ts: `applyPendingMigrations` is the one migration step for
  every compiled start; `startCompiledApi` (`lt dev up --api-compiled`) now
  uses it, unchanged in behaviour
- dev-test-session.ts: the API half of the bring-up moves into `startTestApi`
  — build → migrate with the API's own env (so both resolve the same DB) →
  start. A failed migration aborts via `migrationFailedError`; a project
  without `migrate:up` (nest-base) starts unmigrated and says so; sibling
  shards migrate their own DB
- the bring-up log no longer claims "reset before the suite by Playwright
  global-setup" — the starter's global-setup does no such thing, and the claim
  pointed the diagnosis away from the un-migrated DB; same for the docblocks
- docs/commands.md: `lt dev test` described as it behaves today (isolated
  stack, migration, abort rules, `--keep` / `down` / `--shard`)
- tests: behaviour of `startTestApi` + `applyPendingMigrations` with a mocked
  process layer; mutation-checked (dropping the migration → 4 red, migrating
  with the wrong env → 2 red)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…s depending on load

`keeps polling while abort stays false` probed the black-holed TEST-NET
address, where every curl probe burns its full `--max-time 2`. Two probes
fit into the 5s budget with ~0.4s to spare, and `retry` skips `abort` once
the budget is spent — so under load the test failed with `calls === 1` on
unchanged code (it blocked a pre-push at load average ~130 on 8 cores).

A refused connection returns in milliseconds, so the 500ms poll interval
decides the count: measured at load 133, 10 abort calls in 5s against
127.0.0.1:1 versus 2 against 192.0.2.1. Windows retries a refused connect
for up to ~2s, which is no slower than the black hole was.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@DKoenig9
DKoenig9 merged commit 148c1ff into main Sep 21, 2026
2 checks passed
@DKoenig9
DKoenig9 deleted the fix/dev-3289-migrate-test-db branch September 21, 2026 10:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant