Skip to content

bug(esp32): published toolchain slices are never used (name mismatch + wrong checksum) #1952

Closed

Description

@zackees

Two client bugs make the published per-chip toolchain slices inert

Found while running a real ESP32-S3 build against the @esp-toolchain shard that
#1951 published. Before this, every cold-cache ESP32 build downloaded the whole
176 MB unified Xtensa archive even though a 106 MB slice was published and
listed.

1. slice_shard does not know the toolchain's real package name

PackageBase::published_slice asks variants::published_slice(&self.name, …),
and slice_shard matched only toolchain-xtensa-esp-elf (the slicer/manifest
name). Esp32Toolchain::new / from_resolved
(crates/fbuild-toolchain/src/toolchain/esp32.rs) name the same artifact
esp32-xtensa-gcc, so the map returned None and the @esp-toolchain shard was
never fetched.

Evidence (fresh isolated cache, minimal esp32-s3-devkitc-1 project):

esp32-xtensa-gcc …: downloading xtensa-esp-elf-14.2.0_20260121-x86_64-linux-gnu.tar.xz: 118/168 MB (70%)
$ ls <cache>/mirror-manifest/shards/     # only the libs shard, no toolchain one
shard-833986bbdbc5481a.json

2. The slice is then verified against the whole archive's checksum

PackageBase::install re-verifies the acquired archive against self.checksum,
which the platform metadata pins to the WHOLE archive. download_slice already
verified the slice against its own published sha256, so the second check fails
every time:

build error: package error: checksum mismatch for …+esp32s3_staging/slice.tar.zst:
expected da31f36d79d4e99f24e55a90a71e65d5694714f16199960bf7885724b706a48c,
got 36a9f0e7d232506fca8411de1171249c7e38f9a2d05ec69b6a8e0425230f0858

Both were invisible to the unit tests: they construct the package with
name = "esp32-xtensa-gcc" but look the slice up through forced_slice, which
skips the name lookup, and they pass checksum: None, which skips the
whole-archive check. The tests suppressed exactly the two fields that fail.

Fix

  • slice_shard accepts both names of the Xtensa toolchain (and neither RISC-V
    name: that toolchain is never sliced).
  • acquire_archive reports whether the archive came from a slice; the
    whole-archive pin is applied only to the whole archive.

With both fixed, a cold-cache build installs …14.2.0_20260121-x86_64-linux-gnu+esp32s3
(603 MB extracted instead of 1.2 GB), fetches TWO shards (esp-libs and
@esp-toolchain), and no …tar.xz whole archive is ever downloaded.

Acceptance: on a cold cache the ESP32-S3 build must log the slice download, keep
mirror-manifest/shards/ with a toolchain shard, and never progress past 106 MB
for the toolchain.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions