Skip to content

[WIP] fix(azure): validate metadata IPv4 against guest networking - #80

Draft
Garrett Settles (gsettles01) wants to merge 4 commits into
aclmainfrom
gsettles/18833-azure-metadata-ip-fix
Draft

Garrett Settles (gsettles01) wants to merge 4 commits into
aclmainfrom
gsettles/18833-azure-metadata-ip-fix

Conversation

@gsettles01

@gsettles01 Garrett Settles (gsettles01) commented Sep 24, 2026 •

Copy link
Copy Markdown

Summary

On Azure Arm64 VMs, WireServer SharedConfig can report an IPv4 address that is not assigned to the VM (seen: 10.10.10.10). Afterburn writes it to COREOS_AZURE_IPV4_DYNAMIC, etcd binds to an address the VM does not have, and acl.etcd-member.discovery fails on Azure aarch64.

After Afterburn writes metadata, a new helper checks the route-selected source IPv4 against the interface's assigned addresses and IMDS (matching MAC and IP), and only then corrects COREOS_AZURE_IPV4_DYNAMIC. The first WireServer result is kept in /run/metadata/flatcar.wireserver. If any check fails, the metadata is left unchanged and coreos-metadata.service fails.

To keep CI working, merge in this order:

  1. acl-pipelines !29414, which adds the testAzureMetadataIpv4 gate parameter. Until it merges, this PR's validation fails on that unknown parameter.
  2. This PR.
  3. [WIP] test(metadata): check Azure IPv4 against the assigned NIC azure-container-linux-mantle#45, which makes cl.metadata.azure check the address.

Change Log

  • Add azure-metadata-ipv4 and a coreos-metadata.service drop-in that runs it as ExecStartPost; manglefs_rpm.sh installs both in RPM mode only.
  • Add build_library/rpm/tests/test_azure_metadata_ipv4.sh (22 offline cases; needs only Bash and jq) and run it in PR validation via .pipelines/github-pr-validation.yml.
  • Enforce cl.metadata.azure in acl/tests/kola_enforcing.yaml.

Type of Change

  • Image build change (base image, sysexts, OEM images)
  • Package/SPEC update
  • CI/automation change
  • SDK/toolchain update
  • Configuration change
  • Documentation update
  • Bug fix

Does this affect the image build?

  • Yes
  • No

Adds the helper and drop-in to the Azure OEM sysext in RPM images. Other platforms and Portage builds are unchanged.

Associated Issues

Test Methodology

  • Original failure: [ARM64]-Prod-BuildACL-Nightly 1204420.
  • Offline: bash build_library/rpm/tests/test_azure_metadata_ipv4.sh passes 22/22 in the SDK container with no network, on a plain Ubuntu host, and in Azure Linux 3 using the pipeline step. Replacing the helper with exit 0 makes it fail.
  • Azure Arm64, images built from this branch: two fresh 3-node clusters passed acl.etcd-member.discovery. A third failed on an unrelated DNS error while pulling the etcd image (systemd retried it successfully); its metadata was correct on all 3 nodes.
  • Latest Azure run: on all 3 nodes the corrected IPv4 matched the NIC and IMDS, the WireServer value was preserved, and coreos-metadata.service succeeded with no restarts.
  • The correction held through metadata regeneration and reboots on 3/3 nodes in each of the last two runs. An unfixed image reproduced the wrong address on 3/3 nodes.
  • Pending: an ACL-Dev aarch64 run of this branch with the Mantle branch (cl.metadata.azure, acl.etcd-member.discovery), plus an aclmain baseline where cl.metadata.azure should fail.
  • Pending: required CI checks, after the acl-pipelines prerequisite merges.

Merge Checklist

All applicable boxes should be checked before merging.

  • Prerequisite PR merged - acl-pipelines !29414
  • Image builds successfully with this change
  • Relevant kola tests pass (ACL-Dev aarch64 run)
  • Required CI checks pass
  • Ready to merge

Azure WireServer SharedConfig can report an IPv4 address not assigned to
the VM, causing etcd to bind to the wrong address.

Validate the route-selected IPv4 against the interface and IMDS before
metadata consumers start. Keep this in Azure RPM images, preserve the
first raw result, and add offline regression coverage.

Validation: 22 offline cases and one fresh ARM64 discovery run passed.
All three nodes passed regeneration and reboot with SELinux enforcing.

Refs: ACL Bug 18833
The helper runs as ExecStartPost after Afterburn has already reached
WireServer and IMDS, so the drop-in needs no network-online ordering.

The assigned-address check already rejects impossible IPv4 values, so
drop the octet range loop. The pattern check still guards the sed
rewrite; the invalid-ipv4 test now uses a value it rejects.
Enable the acl-pipelines testAzureMetadataIpv4 step so PR validation
runs the offline test on the build agent. The test only needs Bash and
jq because ip and curl are fixtures.

Enforce cl.metadata.azure so Azure kola runs catch a wrong
COREOS_AZURE_IPV4_DYNAMIC. It is registered for Azure only, so QEMU
runs never select it.

Requires the acl-pipelines testAzureMetadataIpv4 parameter on main.

Refs: ACL Bug 18833

This branch was successfully deployed

1 active deployment
development — 38ea0e04 Deployed Sep 24, 2026 by gsettles01 via Check if we need to update the SDK #47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant