[eterceros]

At a glance: before vs. after

The FHIR Server Benchmark comparison report named its load rows differently from the product (the web UI calls $bulk-submit "Import"/"Bulk Import", and the POST / Bundle screen "Batch / Transaction"), and a run that did not ask for crud/search showed no rows for them, which read as lost metrics.

Row in the comparison report	Before	After
POST / Bundles (suite import)	import, POST / transaction Bundles	Batch/Transaction Import, POST / Bundles
$bulk-submit (suite bulk-import)	bulk-import, $bulk-submit	Bulk Import, $bulk-submit
Load phase of $bulk-submit	bulk-import ingest	Bulk Import: ingest
Search-index rebuild after the load	bulk-import initial index	Bulk Import: index rebuild
crud / search in a run that did not request them (e.g. tests=bulk-import)	row missing	n/a (not requested)
Capacity gate failure	no record of which containers held the Docker host's memory	top holders in the job log, the step summary and bench-results/capacity-gate-holders.txt

Suite ids (import, bulk-import, indexing, ...) and the tests input are unchanged, so runs stay comparable with the public HealthSamurai report.

What changed

• compare_legs.py, summary_backends.py: visible row labels, notes, legend and per-leg summaries use the names above. A new note says the two load rows are not comparable (committed Bundle entries per second of the k6 scenario vs. resources until the search index covers them).
• compare_legs.py: crud and search are always listed; when the run did not request them the cells say n/a (not requested) instead of n/a (not run), so a trimmed run is not mistaken for a report that lost them.
• fhir-benchmark.yml: header comments, the inline results table and the tests input description follow the new names. No executable logic changed.
• capacity-gate.sh: on every wait poll and on the final failure, appends docker stats (sorted by memory) and docker ps (image, age, hfs-ci / hfs-bench-run / testcontainers labels) to bench-results/capacity-gate-holders.txt, and prints the top 10 to the log and the step summary. Best effort and never fatal; thresholds, margin and timeouts are unchanged.

No measurement, backend setting or threshold changed: the figures in the report are the same as before.

Tests

• python3 -m unittest discover -p 'test_*.py' in .github/scripts/fhir-bench: 81 tests pass (updated label constants, a changed insert-only assertion, and a new test for a bulk-import-only run that still lists crud and search).
• bash -n capacity-gate.sh passes and the capture was exercised with a stubbed docker; shellcheck is not installed here, so it was not run.
• The workflow YAML parses.
• Not verified: a full benchmark run with these changes. The gate itself was checked on run 39630573597 (cancelled on purpose after the gate): Docker host MemTotal: 20000MB, MemAvailable=10431MB, capacity OK.

Not in this PR

Deliberately deferred to keep this reviewable: pinning k6 and the upstream benchmark ref (#1914), vus/duration overriding fixed scenarios (#1915), failing a leg when its database is unreachable during setup(), enabling validation for published runs, per-container CPU/memory limits, a full-text suite, and the "Leg health" row showing Patient 0 · Observation 0 on bulk-import-only runs.

Refs #190

🤖 Generated with Claude Code

[claude]

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.