Repository navigation
[M2-AC-01] AC-4.x acceptance suite (projection-mode independence, 60 FPS sort gate) - #99
Merged
Merged
Conversation
…FPS sort gate)
AC-4.1: one engine build, config-only difference. The golden tool now
renders multi-pass frames; two new fixtures (ac4_iso, ac4_side_view)
carry the SAME 96-sprite world + 3-widget UI under the two projection
configs (Iso at the ADR 0005 preset, SideView via planeOrtho). The
GL-free GoldenFixtures.Ac4* tests pin the shared scene shape and the
config-only difference. Documented geometry note: SpriteItem carries
the 2D ground-plane position (no elevation field), so the side-view
world pass is degenerate by construction; the side-view frame's
visible content is the screen-space UI pass (the AC-4.1 assertion is
on the scene, not the world raster).
AC-4.2: tools/laige-surface-lint (Python stdlib) proves no
laige-sim / laige-net public surface mentions ProjectionMode
(RENDER-006): exit 0 OK / 1 violation / 2 structural, with the
ac4_surface* CTest entries (fixture trees for each exit path + the
real tree, the laige-include-lint pattern).
AC-4.3 + AC-4.4: the laige-bench ac4 suite (alias render-sort-10k):
10 000 overlapping sprites (100 x 10 grid, 10 stacked per cell -
equal-key ties + overdraw), 4 (atlas, material, blend) groups, one
60 FPS frame (sprites_10k_frame budget, mean <= 16.7 ms) with the
batcher zero-allocation assertion; setup asserts the AC-4.4 engine
default (ProjectionView{} is Iso). Budgets.json gains
sprites_10k_frame (the BudgetHarnessTable pin moves 17 -> 18); the
laige_bench_ac4_fpx16/_fp32 CTest entries join the ci:slow:bench-50k
label-gated jobs (both workflows).
The two ac4 golden references are committed in this PR's second
commit (the label-gated golden-references job prints them; the first
PR run is expected to fail golden on the missing references - the
M2-GOLD-01 bootstrap contract). Local 4-tree gate green: 121/121
gcc, 121/121 clang, 118/118 asan, 118/118 tsan (14 GL-creating entries
excluded, the documented GPU-restricted local gate).
offdev
added
ci:windows
ci:macos
ci:slow:bench-50k
The runFixture local Ac4SceneConfig cfg shadowed the function's Config& cfg parameter (warning-as-error on the Windows lane, the only MSVC-compiled tree). Local 4-tree gate green after the fix (121/121 gcc, 121/121 clang, 118/118 asan, 118/118 tsan, the 14 GL-creating entries excluded).
Diagnostic only (removed once root-caused): laige_bench_ac4_fpx16_gdb runs the ac4 suite under gdb (reduced frame count) and prints the backtrace of the exit-time segfault seen on the CI Linux lanes. Local 4-tree gate green (the entry is in the GL-creating exclusion list, the documented GPU-restricted local gate).
The generator expression in the test COMMAND was mangled by CMake; use the bare binary name (ctest's PATH includes build/bin). Also add the same reduced ac4 run under AddressSanitizer (heap errors report with a stack). Both entries are removed once the exit-time segfault is root-caused. Local 4-tree gate green (121/121 gcc, 121/121 clang, 118/118 asan, 118/118 tsan, GL-creating entries excluded).
CTest passes each quoted CMake literal as a separate argv element and bash only takes the first as its -c script; the multi-string form broke the command line (verified locally with a two-test experiment). Local 4-tree gate green (121/121, 121/121, 118/118, 118/118).
CTest resolves bare command names itself, but the PATH the bash -c child inherits does not include build/bin. Local 4-tree gate green (121/121, 121/121, 118/118, 118/118).
The 20-frame run exits cleanly even on CI: the segfault only reproduces at the full frame count, so the diagnostic must use the same --runs=3000 --warmup=1000 as the crashing entries. Local 4-tree gate green (121/121, 121/121, 118/118, 118/118).
The crash is a Heisenbug: the 3000-frame ac4 entries segfault at exit on the CI Linux lanes but exit cleanly under gdb, under ASan, and locally. Capture the crashing stack in the plain process instead: a SIGSEGV/SIGBUS handler prints the crashed thread id (vs the main tid printed at startup) plus a backtrace (frame-pointer based, -O0 Debug) to the CI job log, then re-raises. -Wl,--export-dynamic on the bench target gives the symbols names. Removed with the root-cause fix. Local 4-tree gate green.
The in-process crash capture showed the exit segfault hits THREE non-main worker threads (tids 4209/4211/4212 vs main 4208) with no usable backtrace — the crash is inside library workers racing with the exit path. Dump the process's thread names (/proc/self/task comm) at startup and right after the GL context is created, to identify which component spawns them. Removed with the root-cause fix. Local 4-tree gate green.
ctest job logs only show output of failing tests, and the segfault does not reproduce on every lane (run 9: 3 of 4 failed). Make the ac4 and render-50k entries fail with the thread census (startup + post-GL, with the render-50k entry as the clean-exit control) whenever worker threads are present, so the census always reaches the job log. Removed with the root-cause fix. Local 4-tree gate green (the diagnostic entries are GL-gated and excluded locally).
Root cause (CI evidence, runs 38543888405 + 38548562965): the exit segfault hits llvmpipe's worker pool threads (the /proc census named them llvmpipe-0..3; the crash tids 4209/4211/4212 are pool threads, main is 4208). The software rasterizer executes submitted GL work on that pool; the ac4 process destroys the EGL context — and dlclose's the driver — while the pool may still be inside it, so the pool threads segfault. The crash is timing-dependent (Heisenbug): it does not reproduce under gdb, under ASan, at 20 frames, or on every lane. render-50k escapes by luck of its longer per-frame drain tail. Fix: GlContext::Impl::destroy() now makes the context current and glFinish()es (draining all pending work, per the GL spec) before eglDestroyContext / glfwDestroyWindow, on both backends. One-shot teardown cost, never on the frame loop. The laige-bench crash handler (TID + backtrace, -Wl,--export-dynamic) stays TEMPORARILY for one verification run and is removed with it. Local 4-tree gate green (GL entries CI-only).
The glFinish drain alone did not fix the exit segfault (run 38550914045: a llvmpipe worker still segfaulted after the drain). The remaining hazard is the dlclose itself: the driver runs a worker pool it never joins, and dlclose'ing a library with live threads inside it is undefined behavior (its fini may wake the workers into unmapped pages). The dlopened libEGL handle is now kept for the process's remaining lifetime (one per created context; reclaimed at exit) — the glFinish drain before eglDestroyContext stays. Same one-time crash-handler diagnostic still in laige-bench for verification. Local 4-tree gate green.
…s, final cleanup - docs/benchmarks/baselines/m2-ac4.md (the thirteenth baseline): the AC-4.x acceptance checklist, the first CI measurement (run 38526109942: 11.1168 / 12.6959 / 11.5452 / 12.1453 ms mean on the four lanes, draw_calls 4, zero allocation violations, 37.5% margin on the reference lane), and the root-cause record of the ac4 exit-time segfault (llvmpipe worker pool vs the context teardown/dlclose). - budgets.json: sprites_10k_frame measured 0 -> 12.1453 (the worse backend on the CI reference (Clang) lane, the M2-PERF-01 convention). - The two ac4 golden references (ac4_iso.png, ac4_side_view.png, 1280x720) from the label-gated golden-references job: the two-run bootstrap completes, the golden entries go green. - roadmap: M2-AC-01 checked + Shipped (incl. the segfault fix summary), changelog row, Progress Board M2 31/33 (total 78/194; the board's stale Total row corrected). - Final cleanup: the temporary crash handler, thread census, main-tid print, linux include block, and the --export-dynamic link option are removed with the fix. Local 4-tree gate green: gcc 121/121, clang 121/121, asan 118/118, tsan 118/118 (GL-creating entries CI-only per the GPU policy).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The M2-AC-01 step of roadmap/M2-rendering-2.5d.md: the AC-4.x acceptance suite, wired into CI as
laige-bench --suite=ac4+ctest -R ac4.AC-4.1 — one engine build, config-only difference. The golden tool now renders multi-pass frames; two new fixtures (
ac4_iso,ac4_side_view) carry the SAME 96-sprite world (12x8 grid, four (atlas, material, blend) groups of 24, engine depth keys) + the SAME 3-widget UI pass under the two projection configs. The GL-freeGoldenFixtures.Ac4*tests pin the shared scene shape and that the configs differ only in the projection. Documented geometry note:SpriteItemcarries the 2D ground-plane position (no elevation field, no new engine API), so the side-view frame's world pass is degenerate by construction; the side-view frame's visible content is the screen-space UI pass, identical in both configs.AC-4.2 — no projection dependency in sim/net surfaces. New
tools/laige-surface-lint(Python stdlib): the public surface oflaige-sim/laige-netmust not mentionProjectionMode(exit 0 OK / 1 violation / 2 structural). CTest entriesac4_surface*: fixture trees for each exit path + the real tree (thelaige-include-lintpattern), in every PR and merge run.AC-4.3 — 10k overlapping sprites at 60 FPS. The new
ac4bench suite (aliasrender-sort-10k, the AC-4.3 scope's dedicated name): 10 000 sprites stacked 10 deep over a 100 x 10 cell grid (equal-key ties + overdraw), four draw groups, one whole frame measured (batch + submit,sprites_10k_framebudget, mean <= 16.7 ms) with the batcher zero-allocation assertion. New CTest entrieslaige_bench_ac4_fpx16/_fp32(Linux, canonical 3000/1000), joining the existingci:slow:bench-50klabel-gated jobs in both workflows;budgets.jsongainssprites_10k_frame(theBudgetHarnessTablepin moves 17 -> 18).AC-4.4 — isometric defaults, asserted here. The suite's setup asserts the engine default
ProjectionView{}.mode == Iso(ADR 0005) as a loud failure; the picking/depth-rebuild half of AC-4.4 is the existing M2-ISO-03 / M2-ISO-02 test suites, both green in this PR's runs.Expected first run:
goldenfails on the two missing ac4 references (every fixture is compared by the entry). Theci:golden-update-labeledgolden-referencesjob in this same run prints both PNGs; they are committed in a follow-up commit (plus the CI-measuredsprites_10k_framebaseline indocs/benchmarks/baselines/m2-ac4.md), per the M2-GOLD-01 bootstrap contract.Local 4-tree gate green before push: 121/121 gcc, 121/121 clang, 118/118 asan, 118/118 tsan (the 14 GL-creating entries excluded, the GPU-restricted local gate).