Skip to content

Onboarding: wait for a live compute service and recover from a stale identity - #380

Open
fwiesel wants to merge 1 commit into
mainfrom
fix-786-stale-service-id
Open

fwiesel wants to merge 1 commit into
mainfrom
fix-786-stale-service-id

Conversation

@fwiesel

@fwiesel fwiesel commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

After a node replacement, onboarding can adopt the leftover compute record of the dead agent and store its service ID. When the replacement agent registers, the stored ID points at an orphan: the enable step runs against the wrong service, the current service stays disabled, and the smoke test loops on No valid host until someone clears status.serviceId and enables the service by hand.

Onboarding now adopts only a record whose service is up, and waits for one to register otherwise. When the stored service vanishes (the enable step returns 404) or the smoke test cannot be scheduled, the stored identity is cleared and the next resolve starts over with the current service. Nova is not polled otherwise.

@fwiesel
fwiesel force-pushed the fix-786-stale-service-id branch from b2f3e90 to e1655f4 Compare October 30, 2026 17:20
@fwiesel fwiesel changed the title Onboarding: re-validate the Nova identity and repair stale service IDs (nova-issues#786) Onboarding: re-validate the Nova identity and repair stale service IDs Nov 9, 2026
@fwiesel
fwiesel force-pushed the fix-786-stale-service-id branch 11 times, most recently from cd0b3b9 to 1013379 Compare November 16, 2026 09:29
@fwiesel fwiesel changed the title Onboarding: re-validate the Nova identity and repair stale service IDs Re-validate the Nova identity during onboarding Nov 16, 2026
@fwiesel
fwiesel force-pushed the fix-786-stale-service-id branch from 1013379 to 6af0982 Compare November 16, 2026 09:49
@fwiesel fwiesel changed the title Re-validate the Nova identity during onboarding Onboarding: re-validate the Nova identity and repair stale service IDs Nov 16, 2026
@fwiesel
fwiesel force-pushed the fix-786-stale-service-id branch 2 times, most recently from e71ec0c to 4658af8 Compare November 16, 2026 10:19
@fwiesel fwiesel changed the title Onboarding: re-validate the Nova identity and repair stale service IDs Onboarding: refuse stale compute records and re-resolve the identity Nov 16, 2026
@fwiesel
fwiesel force-pushed the fix-786-stale-service-id branch 8 times, most recently from 3ea3cb8 to 2153a33 Compare November 16, 2026 11:51
@fwiesel fwiesel changed the title Onboarding: refuse stale compute records and re-resolve the identity Onboarding: wait for a live compute service and recover from a stale identity Nov 16, 2026
@fwiesel
fwiesel force-pushed the fix-786-stale-service-id branch from 2153a33 to 4862faf Compare November 16, 2026 12:00
@fwiesel
fwiesel force-pushed the fix-786-stale-service-id branch from 4862faf to 180ad30 Compare November 16, 2026 12:26
…identity

Onboarding looked up the hypervisor/service pair only once, while the
fields were empty, and adopted any record matching the hostname,
including the leftover of a dead agent. After a node replacement the
stored service ID can therefore point at an orphan: the enable step
runs against the wrong service and the smoke test loops on
'No valid host' until someone clears status.serviceId by hand.

Onboarding now adopts only a record whose service is up, and waits for
one to register otherwise. When the stored service vanishes (the enable
step returns 404) or the smoke test cannot be scheduled, the stored
identity is cleared and the next resolve starts over with the current
service.

Signed-off-by: Fabian Wiesel <fabian.wiesel@sap.com>
@fwiesel
fwiesel force-pushed the fix-786-stale-service-id branch from 180ad30 to 92d1527 Compare November 16, 2026 12:59
@github-actions

Copy link
Copy Markdown

Merging this branch will increase overall coverage

Impacted Packages Coverage Δ 🤖
github.com/cobaltcore-dev/openstack-hypervisor-operator/internal/controller 76.11% (+0.68%) 👍
Coverage details

Compared to commit cfee313 (run #43438290245)

Changed File Coverage Δ Total Covered Missed 🤖
github.com/cobaltcore-dev/openstack-hypervisor-operator/internal/controller/onboarding_controller.go 68.29% (+3.01%) 287 (+22) 196 (+23) 91 (-1) 👍

Please note that the "Total", "Covered", and "Missed" counts above refer to code statements instead of lines of code. The value in brackets refers to the test coverage of that file in the old version of the code.

Changed unit test files:

  • github.com/cobaltcore-dev/openstack-hypervisor-operator/internal/controller/onboarding_controller_test.go

@fwiesel
fwiesel marked this pull request as ready for review November 16, 2026 13:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant