Skip to content

fix(helm): serialize repository state during parallel setup - #8113

Merged
devantler merged 1 commit into
mainfrom
codex/helm-repository-state-8112-r72
Nov 1, 2026
Merged

devantler merged 1 commit into
mainfrom
codex/helm-repository-state-8112-r72

Conversation

@devantler

Copy link
Copy Markdown
Contributor

🤖 Generated by the Agentic Engineer

Why

Parallel component setup can lose or corrupt Helm repository information, failing cluster creation even when the charts are available.

What

Protect repository changes and chart reads as one shared operation, while letting canceled requests stop promptly. Component installation remains parallel after chart loading.

Fixes #8112

Share a cancelable acquisition gate across repository registration and chart reads, preserving every entry and parallel installation after acquisition. Part of #8112.

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

Validation at 219d6f0f576c1490beae04dbb234faa8cb40e87e: the old implementation reproduced a lost completed registration, reader/writer overlap in both directions, and canceled waiters accessing state. The real HTTP/filesystem/chart fixtures now preserve both repository entries, exclude overlapping access, return cancellation promptly and allow later progress.

The complete Helm race suite passed in 47.831s, the installer race suites passed, vet passed, focused configured lint reported zero issues, and independent static review found no remaining correctness/security findings. The three source hashes were stable during validation. Native cluster CI and current-head provider review remain pending; this is still a draft. The gate protects clients in one process, while unrelated external Helm processes remain outside its scope. No retry budget, cache format, timeout or installation assertion was changed.

@github-actions

github-actions Bot commented Nov 1, 2026

Copy link
Copy Markdown
Contributor

✅⚠️MegaLinter analysis: Success with warnings

⚠️ BASH / bash-exec - 1 error
Results of bash-exec linter (version 5.3.9)
See documentation on https://megalinter.io/10.1.0/descriptors/bash_bash_exec/
-----------------------------------------------

✅ [SUCCESS] .github/actions/cleanup-hetzner/cleanup-hetzner.sh
✅ [SUCCESS] .github/actions/cleanup-hetzner/cleanup-hetzner.test.sh
✅ [SUCCESS] .github/actions/free-disk-space/free-disk-space.sh
✅ [SUCCESS] .github/actions/ksail-cluster/hetzner-inventory-config.sh
✅ [SUCCESS] .github/actions/ksail-system-test/autoscaler-audit-trial.sh
✅ [SUCCESS] .github/actions/ksail-system-test/calico-migration.sh
✅ [SUCCESS] .github/actions/ksail-system-test/calico-migration.test.sh
✅ [SUCCESS] .github/actions/ksail-system-test/chart-values-drift.sh
✅ [SUCCESS] .github/actions/ksail-system-test/helm-values-precedence.sh
✅ [SUCCESS] .github/actions/ksail-system-test/offline-ecr-upstream.sh
✅ [SUCCESS] .github/actions/ksail-system-test/offline-ecr-upstream.test.sh
✅ [SUCCESS] .github/actions/ksail-system-test/talos-kubernetes-upgrade.sh
✅ [SUCCESS] .github/actions/ksail-system-test/verify-offline-ecr-pull.sh
✅ [SUCCESS] .github/actions/ksail-system-test/verify-offline-ecr-pull.test.sh
✅ [SUCCESS] .github/actions/ksail-test-workload/install-with-recovery.sh
✅ [SUCCESS] .github/actions/ksail-test-workload/install-with-recovery.test.sh
✅ [SUCCESS] .github/actions/ksail-test-workload/reject-remote-kustomize-resources.sh
✅ [SUCCESS] .github/actions/ksail-test-workload/remote-resource-guard.test.sh
✅ [SUCCESS] .github/actions/pull-registry-image/pull-registry-image.sh
✅ [SUCCESS] .github/actions/rate-limit-gate/wait-for-rate-limit.sh
✅ [SUCCESS] .github/actions/rate-limit-gate/wait-for-rate-limit.test.sh
✅ [SUCCESS] .github/actions/warm-mirror-cache/cache-validation.test.sh
✅ [SUCCESS] .github/actions/warm-mirror-cache/mirror-artifact.sh
✅ [SUCCESS] .github/actions/warm-mirror-cache/pull-helper-image.sh
✅ [SUCCESS] .github/actions/warm-mirror-cache/pull-helper-image.test.sh
✅ [SUCCESS] .github/scripts/assert-gitops-deployed.sh
✅ [SUCCESS] .github/scripts/assert-gitops-deployed.test.sh
✅ [SUCCESS] .github/scripts/assert-k3k-server-creator.sh
✅ [SUCCESS] .github/scripts/assert-k3k-server-creator.test.sh
✅ [SUCCESS] .github/scripts/build-calico-migration-baseline.sh
✅ [SUCCESS] .github/scripts/cask-pr-handoff-workflow.test.sh
✅ [SUCCESS] .github/scripts/cask-rendering.test.sh
✅ [SUCCESS] .github/scripts/check-eks-capacity.sh
✅ [SUCCESS] .github/scripts/check-eks-capacity.test.sh
✅ [SUCCESS] .github/scripts/codeql-autobuild-prepare.sh
✅ [SUCCESS] .github/scripts/codeql-autobuild-prepare.test.sh
✅ [SUCCESS] .github/scripts/codeql-source.sh
✅ [SUCCESS] .github/scripts/collect-cask-pr-handoff.sh
✅ [SUCCESS] .github/scripts/collect-cask-pr-handoff.test.sh
✅ [SUCCESS] .github/scripts/delete-eks-smoke-cluster.sh
✅ [SUCCESS] .github/scripts/delete-eks-smoke-cluster.test.sh
✅ [SUCCESS] .github/scripts/delete-old-workflow-runs.sh
✅ [SUCCESS] .github/scripts/delete-old-workflow-runs.test.sh
✅ [SUCCESS] .github/scripts/download-go-modules.sh
✅ [SUCCESS] .github/scripts/eks-report-test-lib.sh
✅ [SUCCESS] .github/scripts/find-merged-cask-pr-handoff.sh
✅ [SUCCESS] .github/scripts/find-merged-cask-pr-handoff.test.sh
✅ [SUCCESS] .github/scripts/finish-cask-pr-handoff.sh
✅ [SUCCESS] .github/scripts/finish-cask-pr-handoff.test.sh
✅ [SUCCESS] .github/scripts/install-release.test.sh
✅ [SUCCESS] .github/scripts/list-link-check-files.sh
✅ [SUCCESS] .github/scripts/profile-codeql-cohort.sh
✅ [SUCCESS] .github/scripts/profile-codeql-compare.sh
✅ [SUCCESS] .github/scripts/profile-codeql-compare.test.sh
✅ [SUCCESS] .github/scripts/profile-codeql-files.sh
✅ [SUCCESS] .github/scripts/profile-codeql-files.test.sh
✅ [SUCCESS] .github/scripts/profile-codeql-go.sh
✅ [SUCCESS] .github/scripts/profile-codeql-go.test.sh
✅ [SUCCESS] .github/scripts/profile-codeql-swap.sh
✅ [SUCCESS] .github/scripts/profile-codeql-swap.test.sh
✅ [SUCCESS] .github/scripts/publish-mcp-registry.sh
✅ [SUCCESS] .github/scripts/publish-mcp-registry.test.sh
✅ [SUCCESS] .github/scripts/publish-operator-chart.sh
✅ [SUCCESS] .github/scripts/redraft-evergreen-cask-prs.sh
✅ [SUCCESS] .github/scripts/redraft-evergreen-cask-prs.test.sh
✅ [SUCCESS] .github/scripts/report-eks-account-capacity.sh
✅ [SUCCESS] .github/scripts/report-eks-account-capacity.test.sh
✅ [SUCCESS] .github/scripts/report-eks-create-failure.sh
✅ [SUCCESS] .github/scripts/report-eks-create-failure.test.sh
✅ [SUCCESS] .github/scripts/resolve-cask-publication.sh
✅ [SUCCESS] .github/scripts/resolve-cask-publication.test.sh
✅ [SUCCESS] .github/scripts/resolve-release-publish-state.sh
✅ [SUCCESS] .github/scripts/resolve-release-publish-state.test.sh
❌ [ERROR] .github/scripts/resolve-release-replay.sh
    Error: File:[.github/scripts/resolve-release-replay.sh] is not executable

✅ [SUCCESS] .github/scripts/strict-link-transport.sh
✅ [SUCCESS] .github/scripts/strict-link-transport.test.sh
✅ [SUCCESS] .github/scripts/style-clean-cask-branch.sh
✅ [SUCCESS] .github/scripts/style-clean-cask-branch.test.sh
✅ [SUCCESS] .github/scripts/upload-release-assets.sh
✅ [SUCCESS] .github/scripts/upload-release-assets.test.sh
✅ [SUCCESS] .github/scripts/validate-cask-pr-handoff.sh
✅ [SUCCESS] .github/scripts/validate-cask-pr-handoff.test.sh
✅ [SUCCESS] .github/scripts/validate-release-ref.sh
✅ [SUCCESS] .github/scripts/validate-release-ref.test.sh
✅ [SUCCESS] .github/scripts/verify-desktop-codeql.sh
✅ [SUCCESS] .github/scripts/verify-desktop-codeql.test.sh
✅ [SUCCESS] .github/scripts/verify-go-archive-parity.sh
✅ [SUCCESS] .github/scripts/verify-go-archive-parity.test.sh
✅ [SUCCESS] .github/scripts/verify-otelzap-parity.sh
✅ [SUCCESS] .github/scripts/verify-otelzap-parity.test.sh
✅ [SUCCESS] .github/scripts/verify-vendored-comment-scan.sh
✅ [SUCCESS] .github/scripts/wait-for-github-api-budget.sh
✅ [SUCCESS] .github/scripts/wait-for-github-api-budget.test.sh
✅ [SUCCESS] desktop/scripts/make-icns.sh
✅ [SUCCESS] desktop/scripts/make-info-plist.sh
✅ [SUCCESS] desktop/scripts/make-macos-app.sh
✅ [SUCCESS] docs/demos/render.sh
✅ [SUCCESS] images/ksail-analysis-runner/smoke.sh
✅ [SUCCESS] install.sh
✅ [SUCCESS] internal/ciharness/testdata/schematic_fake_ksail.sh
✅ [SUCCESS] pkg/svc/provisioner/cluster/vcluster/containerd_registry_hosts.sh
✅ [SUCCESS] scripts/stage-webui.sh

✅ Linters with no issues

actionlint, git_diff, hadolint, jscpd, jsonlint, lychee, markdown-table-formatter, markdownlint, prettier, prettier, shellcheck, shfmt, stylelint, syft, trivy-sbom, trufflehog, v8r, v8r, yamllint

See detailed reports in MegaLinter artifacts

MegaLinter is provided by OX Security
Show us your support by starring ⭐ the repository

@github-code-quality

Copy link
Copy Markdown
Contributor

Code coverage overview

Languages: Go

Go / code-coverage/go

main 47c8e1f codex/helm-repositor... 219d6f0 +/-
71% 71% 0%
View coverage of the 6 most impacted files.
File main 47c8e1f codex/helm-repositor... 219d6f0 +/-
pkg/webui/api/wsmultiplexer.go 81% 74% -7%
pkg/svc/provisi...etzner/token.go 75% 71% -4%
pkg/client/helm...art_location.go 72% 72% 0%
pkg/client/helm/client.go 61% 62% +1%
pkg/svc/provisi...pgrader_wait.go 92% 94% +2%
pkg/client/helm/repository.go 21% 73% +52%

ℹ️ Calculations represent line coverage. Results may change as calculations are updated.

@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Nov 1, 2026 •

Copy link
Copy Markdown
Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

Native validation at 219d6f0f576c1490beae04dbb234faa8cb40e87e is now complete: all 95 reported checks settled successfully or intentionally skipped. The Vanilla/Docker full-stack Kyverno job that failed Helm acquisition on the blocked CI-recovery branch passed in 12m19s. Its logs bind the real binary to tested merge 2076fca76f04ec5649b10a0d7a5f9cf0554cf172; that merge and this head have identical full source trees. Concurrent component installation, the persisted admission proof and owned cluster/registry cleanup succeeded. This is one integration observation, not a sustained reliability claim or a whole-pipeline 15-minute result.

The earlier deterministic real HTTP/filesystem tests establish the lost-registration and overlapping-access failures and their correction; complete Helm and installer race suites remain green. The published commit is verified signed, independently rechecked with the existing verification backend.

MegaLinter's non-blocking executable-bit warning concerns an unchanged baseline helper: identical mode and blob at base/head, with both production callers explicitly invoking Bash. It is outside this three-file Helm change and is not a failed native gate. Current-head provider review has been requested; the PR remains a draft until that gate is satisfied.

@coderabbitai

coderabbitai Bot commented Nov 1, 2026

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review.

Recent review info
Run configuration
Commits

Reviewing files that changed from the base of the PR and between 47c8e1f and 219d6f0.

Files selected for processing: 3
  • pkg/client/helm/chart_location.go
  • pkg/client/helm/repository.go
  • pkg/client/helm/repository_state_test.go

Included review availability: This review used your included allowance. Your plan provides up to 2 included reviews per hour; 1 remain after this review.

Recent review details
Additional context used
Code guidelines: 1
CONTRIBUTING.md — auto-discovered
Path-based instructions: 2
Applicability reference: AGENTS.md: # KSail - Kubernetes SDK for Local GitOps Development **Recommended local validation before any PR** (matches `CONTRIBUTING.md`; CI re-runs equivalents via the org-wide `validate-go-proj...

CodeRabbit inference engine (CONTRIBUTING.md)

Files:

  • pkg/client/helm/repository.go
  • pkg/client/helm/chart_location.go
  • pkg/client/helm/repository_state_test.go
Applicability reference: AGENTS.md: # KSail - Kubernetes SDK for Local GitOps Development **Recommended local validation before any PR** (matches `CONTRIBUTING.md`; CI re-runs equivalents via the org-wide `validate-go-project` reusable work...

CodeRabbit inference engine (CONTRIBUTING.md)

Files:

  • pkg/client/helm/repository_state_test.go
Additional comments: 2
pkg/client/helm/chart_location.go: 1

11-11: LGTM!

Also applies to: 77-82, 98-126, 142-147

pkg/client/helm/repository_state_test.go: 1

1-282: LGTM!

Walkthrough

A shared, context-aware semaphore now protects Helm chart acquisition and repository registration. AddRepository holds the gate through repository configuration and cache setup, index download, and repository-file update. New tests cover overlapping registrations, registration and chart acquisition, and canceled waiters.

Severity of issue fixed: Medium

Merge Risk: Minimal · up to 219d6

No actionable issue remains from this review. Registration and chart loading may wait for a slow repository by design, but an index request has a timeout.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 60.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documenta… Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Title check Passed The title clearly and concisely describes serializing Helm repository state during parallel setup, which is the main change.
Description check Passed The description directly explains the repository-state corruption problem and the change that serializes repository updates and chart reads while preserving cancellation and parallel installation.
Linked Issues check Passed Directly linked issue #8112 is the only active target. Its coding requirements are covered at head 219d6f0:
- Shared gate across clients: AddRepository (pkg/client/helm/repository.go) calls acquireCh…
Out of Scope Changes check Passed All three changed files serve #8112:
- pkg/client/helm/chart_location.go: changes chartAcquireMu from sync.Mutex to a weighted semaphore. This is needed for context-aware acquisition. It also adds acq…
Full details: Linked Issues check

Explanation

Directly linked issue #8112 is the only active target. Its coding requirements are covered at head 219d6f0:

  • Shared gate across clients: AddRepository (pkg/client/helm/repository.go) calls acquireChartState after request validation and holds chartAcquireMu through config load, cache setup, index download, and repositories.yaml write. chartAcquireMu is a package-level semaphore, so it covers distinct clients in one process.
  • Registration and chart acquisition exclude each other in both directions. loadChartAndValues (pkg/client/helm/chart_location.go) takes the same gate for locate, load, and values reads.
  • No nested locking: AddRepository does not call loadChartAndValues or acquireChartState again. The gate is released by defer when loadChartAndValues returns, so install and render run outside it and stay parallel.
  • Cancellation: acquireChartState returns a wrapped context error when the context is already canceled or while waiting. It releases the token when cancellation races a successful acquisition.
  • Regression tests (pkg/client/helm/repository_state_test.go): TestRepositoryRegistrationsPreserveEntries, TestRepositoryRegistrationAndChartAcquisitionExcludeEachOther, and TestRepositoryStateWaitCancellation. They use isolated HELM_* paths, an httptest server, and a generated local chart. They cover lost updates, reader/writer exclusion, and waiter cancellation without shared-state access.
  • Preserved behavior: validateRepositoryRequest and its error text are unchanged. Retry constants and downloadRepositoryIndex retry logic are unchanged at head. The author reports that retry, cache, timeout, and installation assertions did not change. I could not diff against the base because the diff operation failed, so this relies on head content and the author's report.
  • Non-coding items (full CI, native full-stack leg, current-head review, publication) were excluded.
    Uncertainty: test pass results are author-reported. I did not execute tests. The new cancellation path returns 'acquire Helm chart state: ...' wrapping context.Canceled. Earlier validation keeps 'add repository context cancelled'. Both wrap with %w, so errors.Is still matches.
Full details: Out of Scope Changes check

Explanation

All three changed files serve #8112:

  • pkg/client/helm/chart_location.go: changes chartAcquireMu from sync.Mutex to a weighted semaphore. This is needed for context-aware acquisition. It also adds acquireChartState and updates lock documentation to include registration.
  • pkg/client/helm/repository.go: adds the shared gate to AddRepository. This is the core fix.
  • pkg/client/helm/repository_state_test.go: adds regression tests for the required behaviors.
    The reviewed summary shows no unrelated refactoring, dependency, or public API change. Exported declarations are unchanged.
Full details: Docstring Coverage

Explanation

Docstring coverage is 60.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 5 functions across 3 files.

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

The docstring-percentage warning is refuted on its merits. The public AddRepository contract remains documented; the shared semaphore's adjacent contract explains its complete registration/acquisition scope, continued parallel installation and cancellable waiting. acquireChartState also documents why a cancellation racing successful acquisition must return the token before refusing access. The fixture comments explain the real lost-update and reader/writer interleavings, and the cancellation test asserts both prompt refusal and later progress. There is no missing public API documentation or unexplained synchronization boundary established by the aggregate 60% figure. Repeating those contracts above private helpers solely to meet that percentage would add redundant prose. No source change is needed.

CodeRabbit's actual review reports no actionable comments and no inline findings. This disclosed resolution starts a new ordered review round at the same head 219d6f0f576c1490beae04dbb234faa8cb40e87e.

@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Nov 1, 2026 •

Copy link
Copy Markdown
Action not completed

Already reviewed the last commit. Use @coderabbitai full review to rerun a review of the entire changeset.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

cr:no-gate@219d6f0f576c1490beae04dbb234faa8cb40e87e for the restarted round beginning 2026-11-01T18:46:56Z: CodeRabbit declined request 6562294745 as already reviewed. The fresh no-gate guard admits this round; the earlier aggregate docstring warning is resolved in comment 6562292006. Advancing to Codex in the required order.

@devantler

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 1, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-01T18:49:46.327651Z 219d6f0 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Another round soon, please!

Reviewed commit: 219d6f0f57

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copy link
Copy Markdown
Contributor Author

🤖 Generated by the Agentic Engineer

Ready at 219d6f0f576c1490beae04dbb234faa8cb40e87e, freshly bound to base 47c8e1fc02aa91885fde0bdf836052c5f16f67f2: all 95 reported checks have settled successfully or intentionally skipped, the required aggregate and immutable shared workflows passed, native merge state is CLEAN, and there are no review threads or unresolved findings. Codex's clean receipt 6562324323 follows the disclosed documentation-warning resolution and the ordered CodeRabbit refusal. No further provider is requested.

The real repository/chart HTTP and filesystem fixtures reproduced the original races before the fix and now pass with cancellation and later-progress controls. Complete Helm and installer race suites, vet and focused lint passed. The actual native full-stack Vanilla/Kyverno journey at this complete source tree installed components, persisted its admission probe and cleaned up successfully. The signed head was verified locally and by GitHub. Scope remains shared Helm state within one process; installation execution stays parallel outside chart acquisition. Promoting for normal protected merge.

@devantler
devantler marked this pull request as ready for review November 1, 2026 18:52
@devantler
devantler merged commit 5554495 into main Nov 1, 2026
95 checks passed
@devantler
devantler deleted the codex/helm-repository-state-8112-r72 branch November 1, 2026 18:53
@github-project-automation github-project-automation Bot moved this from 🫴 Ready to ✅ Done in 🌊 Project Board Nov 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: ✅ Done

Development

Successfully merging this pull request may close these issues.

Serialize Helm repository registration with chart acquisition

1 participant