Skip to content

e2e: diff PRs against today's main; wait for every replica (5161) - #688

Merged
jacobecox merged 2 commits into
mainfrom
jacob/e2e-runner-fixes
Oct 29, 2026
Merged

jacobecox merged 2 commits into
mainfrom
jacob/e2e-runner-fixes

Conversation

@jacobecox

Copy link
Copy Markdown
Contributor

Two fixes to the e2e CI (work item 5161), found on #664, the first PR that wasn't one of the onboarding PRs.

1. PRs were tested against an old main. plan and the coverage gate compared the PR with pull_request.base.sha, which is main as of when the PR was opened. Every test merged since then counted as part of the PR. #664 changed only timescaledb-highly-available, but CI ran 12 other templates for 35 minutes and went red on redis-cluster. Both workflows now compare against the first parent of the merge commit they check out, which is today's main.

2. Wait for every replica before running checks. After --wait, the runner polls get-deployments until every location has its desiredScale replicas, all of them ready, on two polls in a row. Locations scaled to zero and cron workloads are skipped. On #664, redis-cluster was reported ready while node 5 was still starting, the cluster formed after the checks gave up, and every cluster check failed. elasticsearch #678 had the same problem and worked around it with long retries.

  • Local run, redis-cluster 1.7.0 fresh install in test-gvc-3: all checks pass and teardown is clean. A side-by-side poll showed the 6 replicas becoming ready one at a time over about 4 minutes, with desiredScale: 6 reported from the start.

This PR changes only .github/ and the guide, so its own e2e run tests nothing. The proof is a manual run of redis-cluster and elasticsearch from this branch, linked below.

🤖 Generated with Claude Code

https://claude.ai/code/session_01FZrfgWEQM4Kb8nMRmMPHEC

- plan and the coverage gate compared a PR against pull_request.base.sha, which is main as of when the PR
  was opened, so every test merged since counted as the PR's own change: #664 (timescaledb-ha only) ran
  12 unrelated templates for 35 min and went red on one of them. The checkout is the PR merged onto
  today's main, so diff against its first parent.
- after helm --wait, poll get-deployments until every location has its desiredScale replicas, all ready,
  on two polls in a row. On #664 the redis-cluster install was reported ready while node 5 was still
  starting, and every cluster check timed out (same as elasticsearch #678).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FZrfgWEQM4Kb8nMRmMPHEC
@jacobecox
jacobecox marked this pull request as ready for review October 27, 2026 22:36
@jacobecox

Copy link
Copy Markdown
Contributor Author

Proof run from this branch: https://github.com/controlplane-com/templates/actions/runs/40593636101. redis-cluster 1.7.0 (fresh install, and upgrade from 1.4.3) and elasticsearch 1.0.1 (fresh install, and upgrade from 1.0.0) pass every check. On the elasticsearch fresh install, cluster-green passed 5 s after the install was reported ready. Before this change it needed about 4 minutes of retries. This PR's own run selected nothing to test, which is correct.

github-actions Bot added a commit that referenced this pull request Oct 27, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
github-actions Bot added a commit that referenced this pull request Oct 27, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
github-actions Bot added a commit that referenced this pull request Oct 27, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
github-actions Bot added a commit that referenced this pull request Oct 27, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
github-actions Bot added a commit that referenced this pull request Oct 27, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
github-actions Bot added a commit that referenced this pull request Oct 27, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
github-actions Bot added a commit that referenced this pull request Oct 27, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
github-actions Bot added a commit that referenced this pull request Oct 27, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
github-actions Bot added a commit that referenced this pull request Oct 27, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
github-actions Bot added a commit that referenced this pull request Oct 27, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
github-actions Bot added a commit that referenced this pull request Oct 27, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
github-actions Bot added a commit that referenced this pull request Oct 28, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
github-actions Bot added a commit that referenced this pull request Oct 28, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
github-actions Bot added a commit that referenced this pull request Oct 28, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
github-actions Bot added a commit that referenced this pull request Oct 28, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
github-actions Bot added a commit that referenced this pull request Oct 28, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
github-actions Bot added a commit that referenced this pull request Oct 28, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
github-actions Bot added a commit that referenced this pull request Oct 28, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
github-actions Bot added a commit that referenced this pull request Oct 28, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
github-actions Bot added a commit that referenced this pull request Oct 28, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
github-actions Bot added a commit that referenced this pull request Oct 28, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
github-actions Bot added a commit that referenced this pull request Oct 28, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
github-actions Bot added a commit that referenced this pull request Oct 28, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
github-actions Bot added a commit that referenced this pull request Oct 28, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
github-actions Bot added a commit that referenced this pull request Oct 28, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
github-actions Bot added a commit that referenced this pull request Oct 28, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
github-actions Bot added a commit that referenced this pull request Oct 28, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
github-actions Bot added a commit that referenced this pull request Oct 28, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
github-actions Bot added a commit that referenced this pull request Oct 28, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
github-actions Bot added a commit that referenced this pull request Oct 28, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
github-actions Bot added a commit that referenced this pull request Oct 28, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
github-actions Bot added a commit that referenced this pull request Oct 28, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
github-actions Bot added a commit that referenced this pull request Oct 28, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
github-actions Bot added a commit that referenced this pull request Oct 28, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
github-actions Bot added a commit that referenced this pull request Oct 28, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
github-actions Bot added a commit that referenced this pull request Oct 28, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
github-actions Bot added a commit that referenced this pull request Oct 28, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
github-actions Bot added a commit that referenced this pull request Oct 28, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
github-actions Bot added a commit that referenced this pull request Oct 28, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
github-actions Bot added a commit that referenced this pull request Oct 28, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
github-actions Bot added a commit that referenced this pull request Oct 28, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
github-actions Bot added a commit that referenced this pull request Oct 28, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
github-actions Bot added a commit that referenced this pull request Oct 28, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
github-actions Bot added a commit that referenced this pull request Oct 28, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
github-actions Bot added a commit that referenced this pull request Oct 28, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
github-actions Bot added a commit that referenced this pull request Oct 29, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
github-actions Bot added a commit that referenced this pull request Oct 29, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
github-actions Bot added a commit that referenced this pull request Oct 29, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
@github-actions github-actions Bot added the not-on-test-branch Conflicts with main or another open PR, so it is not on the test branch label Oct 29, 2026
…ly for standard workloads

- Workflows: take main's merge-base diff (#691), which supersedes this branch's HEAD^1 fix.
- wait_all_replicas: a standard workload reports ONE status.versions entry per version with the replica
  counts in containers.*.resources (3 ready = 1 entry, replicasReady 3), while a stateful one reports one
  entry per replica (measured on throwaway workloads in test-gvc-2). len(versions) >= desiredScale never
  held for a multi-replica standard workload, so its suite would time out. Sum replicasReady over the
  newest version's entries instead. Also drops a duplicate 'import time' from the merge.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WZ5WWhTn9mEkPY7cLt1cr4
@github-actions github-actions Bot removed the not-on-test-branch Conflicts with main or another open PR, so it is not on the test branch label Oct 29, 2026
github-actions Bot added a commit that referenced this pull request Oct 29, 2026
e2e: diff PRs against today's main; wait for every replica (5161)
@jacobecox
jacobecox merged commit 1a2336f into main Oct 29, 2026
14 checks passed
@jacobecox
jacobecox deleted the jacob/e2e-runner-fixes branch October 29, 2026 21:31

This branch was successfully deployed

1 active deployment
e2e-nifah — 9535c93e Deployed Oct 29, 2026 by jacobecox via test (keycloak/1.5.1) #173
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant