build: increase random_benchmarks step timeout - #14326
Merged
kgryte merged 1 commit intoAug 17, 2026
Merged
Conversation
The job `Run random benchmarks` on workflow `random_benchmarks` failed on develop with `The action 'Run random benchmarks' has timed out after 60 minutes.`, reproduced on 22 of the last 30 scheduled runs on unrelated develop SHAs, and has not recovered on its own. Root cause: the "Run JavaScript and native add-on benchmarks" step (`BUILD_ADDONS=1`) runs `install-node-addons`/`clean-node-addons` for each of the 50 randomly sampled packages; both targets pattern-match against the full package tree and rebuild/clean a native add-on per package, so the aggregate step time grows with total repository size even though `RANDOM_SELECTION_SIZE` itself stays fixed at 50. Every observed failure hit the 60-minute wall exactly (the data is right-censored), so the true time required is unknown; this commit raises the step timeout from 60 to 90 minutes as a pragmatic interim margin, not a precisely sized fix. A structurally similar timeout issue is separately proposed (unmerged, draft) in PR #12513 for the sibling `run_affected_benchmarks` workflow. If 90 minutes also proves insufficient, the underlying per-package addon clean/rebuild + glob overhead should be addressed directly (e.g., caching or parallelizing add-on builds) rather than raising the timeout again. Ref: https://github.com/stdlib-js/stdlib/actions/runs/31916609719
kgryte
added
the
CI
kgryte
marked this pull request as ready for review
August 17, 2026 00:57
kgryte
approved these changes
Aug 17, 2026
kgryte
changed the title
random_benchmarks step timeoutrandom_benchmarks step timeout
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This pull request:
timeout-minutesfrom60to90on the "Run JavaScript and native add-on benchmarks" step in.github/workflows/random_benchmarks.yml.Failing run: https://github.com/stdlib-js/stdlib/actions/runs/31916609719
Symptom:
The action 'Run random benchmarks' has timed out after 60 minutes.— chronic, hitting the majority of recent nightlyrandom_benchmarksruns on unrelated develop SHAs, not a one-off.Root cause: with
BUILD_ADDONS=1, each of the 50 randomly sampled packages triggers a native add-on clean+rebuild cycle (install-node-addons/clean-node-addons), and both targets pattern-match against the full package tree. That per-package overhead grows with total repository size even thoughRANDOM_SELECTION_SIZEitself stays fixed, so the aggregate step time creeps past the 60-minute budget over time.Every observed failure hit the 60-minute wall exactly (the data is right-censored), so this is a pragmatic interim margin rather than a precisely sized fix — noted directly in the commit message. If 90 minutes also proves insufficient, the underlying per-package addon clean/rebuild + glob overhead should be addressed directly (e.g. caching or parallelizing add-on builds).
Related Issues
This pull request has the following related issues:
Questions
No.
Other
Reviewer notes: one reviewer noted the observed failure count in the commit message ("22 of the last 30 runs") likely includes 1-2 runs that time out on the separate "Run C benchmarks" step rather than the JS step this PR targets; the qualitative conclusion (chronic, non-flaky, majority of runs) holds either way. Validated via three independent automated reviewers (correctness, regression scope, style/conventions) across two review cycles; round 1 caught a factual error in the root-cause citation (a reference to PR #12513 as "already applied" when it is actually still open/draft), which was corrected before this PR was opened. All three reviewers approved on round 2.
Checklist
Please ensure the following tasks are completed before submitting this pull request.
AI Assistance
Yes
No
Code generation (e.g., when writing an implementation or fixing a bug)
Test/benchmark generation
Documentation (including examples)
Research and understanding
Disclosure
This PR was proposed by Claude Code as part of an automated CI-failure investigation routine. The failure enumeration/clustering, root-cause analysis, the fix, and three-reviewer validation (two rounds, correctness/regression-scope/style) were produced by Claude Code. Final review and merge decision rest with the maintainers.
@stdlib-js/reviewers
Generated by Claude Code