Skip to content

OCPBUGS-113639: move empty CRIOCredentialProviderConfig CR to run-level 0000_10 - #3010

Merged
openshift-merge-bot[bot] merged 1 commit into
openshift:release-4.22from
sdodson:OCPBUGS-99266-crio-crd-runlevel
Aug 31, 2026
Merged

OCPBUGS-113639: move empty CRIOCredentialProviderConfig CR to run-level 0000_10#3010
openshift-merge-bot[bot] merged 1 commit into
openshift:release-4.22from
sdodson:OCPBUGS-99266-crio-crd-runlevel

Conversation

@sdodson

@sdodson sdodson commented Aug 24, 2026

Copy link
Copy Markdown
Member

Backport of OCPBUGS-99266 to release-4.22.

Problem

The empty CRIOCredentialProviderConfig CR shipped at run-level 0000_05
(0000_05_config-operator_02_criocredentialproviderconfig.cr.yaml, gated on the
CRIOCredentialProviderConfig feature gate), while its CRD is generated at
run-level 0000_10
(0000_10_config-operator_01_criocredentialproviderconfigs.crd.yaml).

When the feature gate is enabled mid-update (e.g. setting TechPreviewNoUpgrade),
the update-mode CVO cannot complete run-level 0000_05 because the CR's CRD does
not yet exist, and that failure blocks it from ever reaching run-level 0000_10
to create the CRD — deadlocking the update with
UpdatePayloadResourceTypeMissing: ... the server does not recognize this resource.

Fix

Move the empty CR to run-level 0000_10 (operatorOrdering 02) so it sorts and
applies after the CRD at 0000_10_config-operator_01 within the same
run-level and component. This is a pure rename of the static empty-resource
manifest — no code generation, no schema change.

This matches the existing precedent for the other non-bootstrap config resource,
0000_10_config-operator_02_node.cr.yaml, whose CRD (nodes) is likewise
generated at run-level 0000_10.

Why not the master fix

Master was fixed by #2844 (promote the feature gate to Default + delete the
manual-override manifests). That is a feature promotion and not appropriate to
backport to a released z-stream, so this PR applies the minimal, low-risk
run-level reorder instead.

Verification

LC_ALL=C sort of the pair now yields the CRD (_01) before the CR (_02):

0000_10_config-operator_01_criocredentialproviderconfigs.crd.yaml
0000_10_config-operator_02_criocredentialproviderconfig.cr.yaml

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: LGTM mode

@openshift-ci-robot openshift-ci-robot added jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Aug 24, 2026
@openshift-ci-robot

Copy link
Copy Markdown

@sdodson: This pull request references Jira Issue OCPBUGS-113639, which is invalid:

  • release note text must be set and not match the template OR release note type must be set to "Release Note Not Required". For more information you can reference the OpenShift Bug Process.
  • expected Jira Issue OCPBUGS-113639 to depend on a bug targeting a version in 5.0.0 and in one of the following states: MODIFIED, ON_QA, VERIFIED, but no dependents were found

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

Backport of OCPBUGS-99266 to release-4.22.

Problem

The empty CRIOCredentialProviderConfig CR ships at run-level 0000_05
(0000_05_config-operator_02_criocredentialproviderconfig.cr.yaml, gated on the
CRIOCredentialProviderConfig feature gate), but its CRD is generated at
run-level 0000_10
(0000_10_config-operator_01_criocredentialproviderconfigs.crd.yaml,
feature-set CustomNoUpgrade,DevPreviewNoUpgrade,TechPreviewNoUpgrade).

When the feature gate is enabled mid-update (e.g. setting TechPreviewNoUpgrade),
the update-mode CVO cannot complete run-level 0000_05 because the CR's CRD does
not yet exist, and that failure blocks it from ever reaching run-level 0000_10
to create the CRD — deadlocking the update with
UpdatePayloadResourceTypeMissing: ... the server does not recognize this resource.

Fix

Change the +openshift:file-pattern marker from cvoRunLevel=0000_10 to
cvoRunLevel=0000_05 (keeping operatorOrdering=01) and regenerate. The CRD now
sorts as 0000_05_config-operator_01_criocredentialproviderconfigs.crd.yaml,
which — in the same run-level and same component (config-operator) as the CR at
_02 — is applied and established before the CR.

The CRD schema content is unchanged; only the run-level-encoding filename and its
two generated references (zz_generated.featuregated-crd-manifests*) change.

Why not the master fix

Master was fixed by #2844 (promote the feature gate to Default + delete the
manual-override manifests). That is a feature promotion and not appropriate to
backport to a released z-stream, so this PR applies the minimal, low-risk
run-level reorder instead.

Verification

  • LC_ALL=C sort of the pair now yields the CRD (_01) before the CR (_02).
  • Re-running hack/update-codegen-crds.sh is idempotent (no drift).

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci

openshift-ci Bot commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Hello @sdodson! Some important instructions when contributing to openshift/api:
API design plays an important part in the user experience of OpenShift and as such API PRs are subject to a high level of scrutiny to ensure they follow our best practices. If you haven't already done so, please review the OpenShift API Conventions and ensure that your proposed changes are compliant. Following these conventions will help expedite the api review process for your PR.

@openshift-ci openshift-ci Bot added the size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. label Aug 24, 2026
@coderabbitai

coderabbitai Bot commented Aug 24, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 8002db2c-70fa-405b-8409-6130422fd4f8

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Comment @coderabbitai help to get the list of available commands.

…el 0000_10

Backport of OCPBUGS-99266.

The empty CRIOCredentialProviderConfig CR shipped at run-level 0000_05
(0000_05_config-operator_02_criocredentialproviderconfig.cr.yaml, gated on the
CRIOCredentialProviderConfig feature gate) while its CRD is generated at
run-level 0000_10. When the feature gate is enabled mid-update (e.g. setting
TechPreviewNoUpgrade), the update-mode CVO cannot complete run-level 0000_05
because the CR's CRD does not yet exist, and that run-level 0000_05 failure
blocks it from ever reaching run-level 0000_10 to create the CRD, deadlocking
the update.

Move the empty CR to run-level 0000_10 (operatorOrdering 02) so it sorts (and
applies) after the CRD at 0000_10_config-operator_01 within the same run-level
and component. This matches the existing precedent for the non-bootstrap Node
config resource (0000_10_config-operator_02_node.cr.yaml).

Master was fixed differently (openshift#2844, promote feature gate to
Default + delete manual overrides); that is a feature promotion unsuitable for
a z-stream, so this backport applies the minimal run-level reorder only.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

rh-pre-commit.version: 2.4.0
rh-pre-commit.check-secrets: ENABLED
@sdodson
sdodson force-pushed the OCPBUGS-99266-crio-crd-runlevel branch from 216dfc7 to eb17e6c Compare August 24, 2026 21:22
@sdodson sdodson changed the title OCPBUGS-113639: move CRIOCredentialProviderConfig CRD to run-level 0000_05 OCPBUGS-113639: move empty CRIOCredentialProviderConfig CR to run-level 0000_10 Aug 24, 2026

@everettraven everettraven left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Aug 24, 2026
@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Scheduling tests matching the pipeline_run_if_changed or not excluded by pipeline_skip_if_only_changed parameters:
/test e2e-aws-ovn
/test e2e-aws-ovn-hypershift
/test e2e-aws-ovn-hypershift-conformance
/test e2e-aws-ovn-techpreview
/test e2e-aws-serial-1of2
/test e2e-aws-serial-2of2
/test e2e-aws-serial-techpreview-1of2
/test e2e-aws-serial-techpreview-2of2
/test e2e-azure
/test e2e-gcp
/test e2e-upgrade
/test e2e-upgrade-out-of-change

@everettraven

Copy link
Copy Markdown
Contributor

/label backport-risk-assessed

@openshift-ci openshift-ci Bot added the backport-risk-assessed Indicates a PR to a release branch has been evaluated and considered safe to accept. label Aug 24, 2026
@everettraven

Copy link
Copy Markdown
Contributor

/approve

@openshift-ci

openshift-ci Bot commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: everettraven

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 24, 2026
@openshift-ci

openshift-ci Bot commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: everettraven

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci

openshift-ci Bot commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

@sdodson: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/verify-hypershift-integration eb17e6c link true /test verify-hypershift-integration

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@sdodson

sdodson commented Aug 25, 2026

Copy link
Copy Markdown
Member Author

/hold
Claude tells me that the ordering issue still exists on main branch so we'll need to fix it there, release-5.0 as well then here I guess. I'll add a test for the run level problem on main and I guess backport to release-5.0. On 4.22 I'll just correct the ordering issue once things have merged into release-5.0.

@openshift-ci openshift-ci Bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Aug 25, 2026
@sdodson

sdodson commented Aug 31, 2026

Copy link
Copy Markdown
Member Author

/jira refresh
/hold cancel
/verified by CI

@openshift-ci openshift-ci Bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Aug 31, 2026
@openshift-ci-robot openshift-ci-robot added the jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. label Aug 31, 2026
@openshift-ci-robot openshift-ci-robot removed the jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. label Aug 31, 2026
@openshift-ci-robot

Copy link
Copy Markdown

@sdodson: This pull request references Jira Issue OCPBUGS-113639, which is valid. The bug has been moved to the POST state.

7 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.22.0) matches configured target version for branch (4.22.0)
  • bug is in the state New, which is one of the valid states (NEW, ASSIGNED, POST)
  • release note type set to "Release Note Not Required"
  • dependent bug Jira Issue OCPBUGS-114877 is in the state Verified, which is one of the valid states (MODIFIED, ON_QA, VERIFIED)
  • dependent Jira Issue OCPBUGS-114877 targets the "5.0.0" version, which is one of the valid target versions: 5.0.0
  • bug has dependents

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

/jira refresh
/hold cancel
/verified by CI

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot openshift-ci-robot added the verified Signifies that the PR passed pre-merge verification criteria label Aug 31, 2026
@openshift-ci-robot

Copy link
Copy Markdown

@sdodson: This PR has been marked as verified by CI.

Details

In response to this:

/jira refresh
/hold cancel
/verified by CI

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

/retest-required

Remaining retests: 0 against base HEAD 2c17c59 and 2 for PR HEAD eb17e6c in total

@openshift-merge-bot
openshift-merge-bot Bot merged commit 160d28e into openshift:release-4.22 Aug 31, 2026
27 checks passed
@openshift-ci-robot

Copy link
Copy Markdown

@sdodson: Jira Issue Verification Checks: Jira Issue OCPBUGS-113639
✔️ This pull request was pre-merge verified.
✔️ All associated pull requests have merged.
✔️ All associated, merged pull requests were pre-merge verified.

Jira Issue OCPBUGS-113639 has been moved to the MODIFIED state and will move to the VERIFIED state when the change is available in an accepted nightly payload. 🕓

Details

In response to this:

Backport of OCPBUGS-99266 to release-4.22.

Problem

The empty CRIOCredentialProviderConfig CR shipped at run-level 0000_05
(0000_05_config-operator_02_criocredentialproviderconfig.cr.yaml, gated on the
CRIOCredentialProviderConfig feature gate), while its CRD is generated at
run-level 0000_10
(0000_10_config-operator_01_criocredentialproviderconfigs.crd.yaml).

When the feature gate is enabled mid-update (e.g. setting TechPreviewNoUpgrade),
the update-mode CVO cannot complete run-level 0000_05 because the CR's CRD does
not yet exist, and that failure blocks it from ever reaching run-level 0000_10
to create the CRD — deadlocking the update with
UpdatePayloadResourceTypeMissing: ... the server does not recognize this resource.

Fix

Move the empty CR to run-level 0000_10 (operatorOrdering 02) so it sorts and
applies after the CRD at 0000_10_config-operator_01 within the same
run-level and component. This is a pure rename of the static empty-resource
manifest — no code generation, no schema change.

This matches the existing precedent for the other non-bootstrap config resource,
0000_10_config-operator_02_node.cr.yaml, whose CRD (nodes) is likewise
generated at run-level 0000_10.

Why not the master fix

Master was fixed by #2844 (promote the feature gate to Default + delete the
manual-override manifests). That is a feature promotion and not appropriate to
backport to a released z-stream, so this PR applies the minimal, low-risk
run-level reorder instead.

Verification

LC_ALL=C sort of the pair now yields the CRD (_01) before the CR (_02):

0000_10_config-operator_01_criocredentialproviderconfigs.crd.yaml
0000_10_config-operator_02_criocredentialproviderconfig.cr.yaml

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. backport-risk-assessed Indicates a PR to a release branch has been evaluated and considered safe to accept. jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. verified Signifies that the PR passed pre-merge verification criteria

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants