Repository navigation
fix(esp32): use the published toolchain slice - #1953
Merged
Merged
Conversation
Two bugs made the per-chip toolchain slices inert in real builds: 1. `slice_shard` matched only the manifest name `toolchain-xtensa-esp-elf`, but `Esp32Toolchain` names its package `esp32-xtensa-gcc`, so the shard was never fetched and every cold ESP32 build downloaded the whole 176 MB archive. 2. After the slice download, `install` re-verified the archive against the package checksum, which pins the WHOLE archive -- so every slice install died with `expected da31f36d..., got 36a9f0e7...`. Both were hidden by tests that force the slice and pass `checksum: None`. Fixed by aliasing the toolchain package name in `slice_shard` and by having `acquire_archive` report whether the archive is a slice, so the whole-archive pin applies only to the whole archive. Real ESP32-S3 cold-cache build: 106 MB slice, chip-scoped 603 MB install dir, no whole archive, build succeeds.
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the
 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1952. Makes the published per-chip toolchain slices
actually get used. Two client bugs,
both found by running a real ESP32-S3 build against the shard that #1951
published — neither is reachable from the existing tests, which is why the
feature shipped inert in #1949.
Bug 1: the toolchain package's name never reached the shard
PackageBase::published_slicekeys the shard byself.name, andslice_shardknew onlytoolchain-xtensa-esp-elf— the manifest/slicer name.Esp32Toolchain::new/from_resolvedname the same artifactesp32-xtensa-gcc,so
slice_shardreturnedNoneand the client never even fetched the@esp-toolchainshard.Measured before the fix (fresh cache, real build): 168 MB
…tar.xzprogress(
118/168 MB), 46 s download, and NO@esp-toolchainshard in<cache>/mirror-manifest/shards/— onlyesp-libs. Every ESP32-S3 build on acold cache downloaded the whole unified toolchain.
Bug 2: the slice was then verified against the whole archive's checksum
PackageBase::installre-verified the acquired archive againstself.checksum, which the metadata sets to the WHOLE archive's digest. A sliceis already verified against its own published
sha256insidedownload_slice,so the second check always failed:
acquire_archivenow returns whether the archive came from a slice, and thewhole-archive pin is applied only to the whole archive.
Reproduced live against the published shard before the fix (bug 1 fixed, bug 2
still present):
fbuild buildfetched the slice, then died with exactly thatchecksum mismatch— so the failure is real, not inferred.Tests
every_name_the_esp_orchestrator_builds_resolves_to_the_toolchain_shard—both names →
@esp-toolchain, and both RISC-V names →None(that toolchainis never sliced). RED when the alias is removed.
a_slice_is_not_checked_against_the_whole_archive_checksum— installs aslice under a package pinned to a different whole-archive sha. RED when the
from_slicefilter is removed (exact live error text).a_whole_archive_is_still_checked_against_its_pin— the opposite direction:dropping the check entirely must not pass.
270 lib testsforfbuild-packages-fetchpass (soldr cargo test -p fbuild-packages-fetch --lib),soldr cargo fmt --all --checkclean.Post-fix end-to-end
Rebuilt
fbuild+fbuild-daemonfrom this branch, isolated empty cache,FBUILD_MIRROR_MANIFEST_URLpointed at the published manifest, minimalesp32-s3-devkitc-1project:…tar.xz(progress118/168 MB)mirror-manifest/shards/esp-libsesp-libs+@esp-toolchain…-linux-gnu(1.2 GB)…-linux-gnu+esp32s3(603 MB)checksum mismatch/ whole downloadbuild succeeded in 65.1s,flash: 355899 bytesFirmware is byte-comparable to the pre-fix build (same flash/RAM numbers), so the
slice is a real toolchain, not a degraded one.