diff --git a/docs/src/user_manual/explanation/netcdf_io.rst b/docs/src/user_manual/explanation/netcdf_io.rst index 4702b81d2d..a643d61135 100644 --- a/docs/src/user_manual/explanation/netcdf_io.rst +++ b/docs/src/user_manual/explanation/netcdf_io.rst @@ -67,6 +67,11 @@ calculated based on a number of factors, including: >>> print(cube.core_data().chunksize) (60, 37, 49) +A file chunk is read and decompressed whole, so a Dask chunk smaller than one +costs a re-read per division. Iris does not adjust its chunking to avoid +that, but it warns when a file chunk is larger than Dask's default chunksize, +which is where the cost starts to be worth your attention. + For more user control, functionality was updated in :pull:`5588`, with the creation of the :data:`iris.fileformats.cf.loader.CHUNK_CONTROL` class. diff --git a/docs/src/user_manual/explanation/zarr_io.rst b/docs/src/user_manual/explanation/zarr_io.rst new file mode 100644 index 0000000000..76efe4374f --- /dev/null +++ b/docs/src/user_manual/explanation/zarr_io.rst @@ -0,0 +1,190 @@ +.. explanation:: Zarr I/O Handling in Iris + :tags: topic_load_save + + Read about how Iris loads Zarr stores. + +.. include:: ../../common_links.inc + +.. _zarr_io: + +========================= +Zarr I/O Handling in Iris +========================= + +.. readingtime:: + +`Zarr `_ stores an array as many independently compressed +chunks under a prefix, rather than as one file. Iris reads Zarr version 2 and +version 3 stores natively, through +`zarr-python `_. + +``zarr`` is an **optional dependency**, and Iris requires at least version +3.0.8. Install it to load Zarr stores:: + + conda install -c conda-forge "zarr>=3.0.8" + +Iris without it installs and runs normally, raising an ``ImportError`` that +names the package and its minimum required version if you ask it to load a +store. + +.. _zarr_io_loading: + +Loading +------- + +A store is a directory, or a URL, so pass it where you would pass a file:: + + cubes = iris.load("/data/forecast.zarr") + +Iris recognises a local store by the metadata file at its root -- ``zarr.json`` +for version 3, ``.zgroup`` for version 2 -- not by the ``.zarr`` suffix, so a +directory that merely has the suffix is not mistaken for one. + +A store holding UGRID mesh data loads as a mesh cube, the same as the +equivalent netCDF file would. :func:`iris.mesh.load_meshes` reads a store +too. + +Saving +------ + +Iris does not save to a native Zarr store. ``iris.save(cube, "out.zarr")`` +raises ``ValueError: Cannot save; no saver``, naming neither Zarr nor an +alternative. Save to netCDF instead, or write Zarr through netCDF-c's NCZarr +driver, covered below in :ref:`zarr_io_nczarr`. + +A cube loaded from Zarr may carry an attribute netCDF cannot hold, which +fails the save. See :ref:`zarr_io_attributes`. + +Remote stores +------------- + +A URL with one of the ``s3``, ``gs``, ``az`` or ``abfs`` schemes, ending in +``.zarr``, is passed to zarr-python unaltered:: + + cubes = iris.load("s3://noaa-gfs-bdp-pds/forecast.zarr") + +Iris adds no store library of its own and handles no credentials. Install the +one your scheme needs -- ``s3fs``, ``gcsfs``, ``adlfs`` -- and configure it as +that package documents. zarr-python raises an ``ImportError`` naming the +package if it is missing. + +Groups +------ + +A Zarr store may hold several groups, and they need not describe the same +thing. Iris loads **one** group per call, like xarray, rather than merging +them: ``iris.load`` reads the root group, and a store whose root holds no data +variables raises a message naming the groups that do. + +To read another group, call the Zarr loader directly:: + + from iris.cube import CubeList + import iris.fileformats.zarr + + cubes = CubeList( + iris.fileformats.zarr.load_cubes( + "/data/sentinel.zarr", group="measurements/reflectance/r10m" + ) + ) + +Dimension names +--------------- + +A netCDF dimension has a name, one length, and a meaning shared across the +whole file. A Zarr dimension name promises less. An array need not name its +dimensions at all, and the same name in two arrays guarantees nothing -- not +even the same length. Both are permitted by the specification. + +So Iris treats two arrays as sharing a dimension only when they name it +identically **and** it is the same length in both. Anything else loads as an +**anonymous** dimension, which is an ordinary Iris cube dimension with no +coordinate on it. That covers both a dimension the store never named, and a +name the store used at two different lengths. + +A coordinate left with nothing to attach to is not discarded. It goes to +:data:`iris.loading.LOAD_PROBLEMS`, and the load warns that not every object +was parsed. + +Version 2 stores have no dimension-name field. They carry names in an +``_ARRAY_DIMENSIONS`` attribute instead -- the convention xarray established +-- which Iris reads, so dimensions are named on both versions. The attribute +describes the store rather than the data, so it does not itself reach +``cube.attributes``. + +Missing data +------------ + +Zarr returns a plain array and applies no masking, so Iris does it, from the +CF attributes: ``_FillValue`` and ``missing_value`` for sentinels, +``valid_min``/``valid_max``/``valid_range`` for a range, and +``scale_factor``/``add_offset`` for packed data. + +A Zarr array also carries a ``fill_value`` in its metadata. That says what +unwritten storage reads back as, not which values are missing, and +zarr-python defaults it to zero. Iris reads it as missing data only on a +version 2 store whose array declares no CF attribute -- which is what version +2 writers meant by it, and what xarray does. + +That one case can mask a genuine zero, so Iris warns when it happens, naming +the variable. Setting a ``_FillValue`` attribute on the array prevents both +the masking and the warning. + +Chunking +-------- + +Iris chooses Dask chunks from the store's own chunking, as it does for +netCDF, and :data:`~iris.fileformats.cf.loader.CHUNK_CONTROL` works the same +way for both formats. + +A Zarr chunk is the atomic unit of storage: reading any part of one fetches +and decompresses the whole object, so a Dask chunk smaller than a store chunk +costs a re-read per division. Iris does not adjust its chunking to avoid +that, any more than it does for netCDF, but it warns when a store chunk is +larger than Dask's target chunk size -- where the cost starts to be worth +your attention. Use ``CHUNK_CONTROL`` to choose the chunking yourself. + +A **sharded** array is chunked on its inner chunks, not its shards. Zarr's +sharding codec is indexed, so a reader fetches only the inner chunks it +needs. + +zarr-python fetches chunks concurrently, and Dask's worker count multiplies +with that concurrency. If a remote store rate-limits you, lower either Dask's +worker count or zarr-python's own ``zarr.config`` setting +``async.concurrency``. + +.. _zarr_io_attributes: + +Attributes +---------- + +A Zarr attribute value may be any JSON literal, including many things CF +cannot express, such as multidimensional arrays, dictionaries or lists of +lists. Iris carries such a value onto the cube unchanged and does not +interpret it. See :issue:`7288` for the wider question of how the CF +attribute model should map onto JSON. + +.. note:: + + A cube carrying such an attribute **cannot be saved as netCDF**, because + netCDF attribute values are scalars, strings and one-dimensional arrays. + The netCDF library raises ``TypeError: illegal data type for attribute + b'stac_discovery'``, which does not name the cube. Delete the attribute + before saving:: + + del cube.attributes["stac_discovery"] + + See :issue:`7329`. + +.. _zarr_io_nczarr: + +NCZarr +------ + +Iris also reads and writes Zarr through netCDF-c's NCZarr driver, reached by +way of a URL fragment:: + + iris.load("file:///data/forecast.zarr#mode=nczarr,file") + +NCZarr implements Zarr version 2, and writes scalar variables as length-one +arrays. The native reader described above handles both versions, and is +what a plain path or URL -- one with no ``#mode=`` fragment -- reaches. diff --git a/docs/src/user_manual/section_indexes/general.rst b/docs/src/user_manual/section_indexes/general.rst index f3d28824a1..219cab679b 100644 --- a/docs/src/user_manual/section_indexes/general.rst +++ b/docs/src/user_manual/section_indexes/general.rst @@ -26,3 +26,4 @@ Below are any pages not belonging to any other User Manual section. ../explanation/um_files_loading ../explanation/ux_guide ../explanation/which_regridder_to_use + ../explanation/zarr_io diff --git a/docs/src/user_manual/tutorial/s3_io.rst b/docs/src/user_manual/tutorial/s3_io.rst index ed5e8dca5f..c03d78de7e 100644 --- a/docs/src/user_manual/tutorial/s3_io.rst +++ b/docs/src/user_manual/tutorial/s3_io.rst @@ -1,7 +1,8 @@ .. tutorial:: Loading From and Saving To S3 Buckets :tags: topic_load_save - A lesson on using s3-fuse with Iris to load/save data from/to S3 buckets. + A lesson on using s3-fuse with Iris to load/save netCDF, PP and GRIB + data from/to S3 buckets, and on reading a Zarr store from S3 directly. .. _s3_io: @@ -19,14 +20,34 @@ objects. S3 access URLs can also contain a nested `'prefix string' `_ structure, which naturally mirrors sub-directories in a file-system. -While it would be possible for Iris to support S3 access directly, as it does the -"OpenDAP" protocol for netCDF data, this approach has some serious limitations : most -notably, each supported file format would have to be separately extended to support S3 -URLs in the place of file paths for loading and saving. +Zarr stores are accessed directly +--------------------------------- + +Everything below concerns **files** -- netCDF, PP, GRIB -- which Iris reaches +through a file-system path. A :ref:`Zarr store ` is +different: Iris passes an ``s3://`` URL straight to zarr-python, which reads +the objects it needs over the network:: + + cubes = iris.load("s3://my-bucket/data.zarr") -Instead, we have found that it is most practical to perform this access using a virtual -file system approach. However, one drawback is that this is best controlled *outside* -the Python code -- see details below. +That needs ``s3fs`` installed and configured, and no mount. If your data is in +Zarr, prefer this to the FUSE approach described below. + +Other filetypes via a virtual file system +----------------------------------------- + +While it would be possible for Iris to support S3 access directly, as it does the +"OpenDAP" protocol for netCDF data, this approach has some serious limitations for +netCDF, PP and GRIB : most notably, each supported file format would have to be +separately extended to support S3 URLs in the place of file paths for loading and +saving. The Zarr route above is that direct approach, made practical because +zarr-python is already a URL-aware store library; netCDF, PP and GRIB have no +equivalent, so that limitation still applies to them. + +Instead, for netCDF, PP and GRIB, we have found that it is most practical to +perform this access using a virtual file system approach. However, one +drawback is that this is best controlled *outside* the Python code -- see +details below. TL;DR diff --git a/docs/superpowers/plans/2026-10-13-zarr-io-pr4.md b/docs/superpowers/plans/2026-10-13-zarr-io-pr4.md new file mode 100644 index 0000000000..395922e229 --- /dev/null +++ b/docs/superpowers/plans/2026-10-13-zarr-io-pr4.md @@ -0,0 +1,6723 @@ +# PR 4 — Zarr loading + +> **For agentic workers:** REQUIRED SUB-SKILL: Use +> `superpowers:subagent-driven-development` (recommended) or +> `superpowers:executing-plans` to implement this plan task-by-task. Steps use +> checkbox (`- [ ]`) syntax for tracking. + +> **This plan ships with its implementation.** Plan and code are delivered on +> the same branch and in the same pull request. If implementation proves a step +> wrong, correct the step here on the branch — the document is frozen when the +> work merges, not while it is under way. Programme status lives in §12 of the +> [design spec](../specs/2026-09-21-zarr-io-design.md); nothing here duplicates +> it. + +| | | +|---|---| +| **Spec** | [`../specs/2026-09-21-zarr-io-design.md`](../specs/2026-09-21-zarr-io-design.md) §2.4, §3, §4.1, §4.2, §4.4, §4.6, §4.7, §5 (PR 4), §6 | +| **Implements** | The fourth of the eight pull requests in spec §5. The subject of [#6979](https://github.com/SciTools/iris/issues/6979) | +| **Branch** | `zarr-io-pr-4`, cut from `zarr-io-pr-3`. A **preview** pull request opens on the fork, `trexfeathers/iris`, with `zarr-io-pr-3` as its base, so that its diff is PR 4's changes and nothing else. The real pull request opens against `SciTools/iris:brownfield` once PR 3 merges (Task 15, Step 6) | +| **Builds on** | **PR 3** ([#7316](https://github.com/SciTools/iris/pull/7316)), which is this branch's **base**, not a prerequisite waited on: this plan is written against `iris.fileformats.cf.loader`, `CFDatasetVariable.read_data` and the `cache_key=` keyword on `as_lazy_data`, none of which are on `brownfield` yet | +| **Baseline** | `zarr-io-pr-3` at `231cd019e`, which carries `brownfield` at `40d377d26` — where the spec review of [#7330](https://github.com/SciTools/iris/pull/7330) merged — plus PR 3's own ten commits; Iris `3.17.0.dev`, zarr-python `3.4.1`, dask `2025.x`, NumPy `2.5.2` | +| **Delivers** | `iris.fileformats.zarr` — `_dataset.py`, `_decode.py`, `loader.py` — the `"Zarr"` format specification, `iris.io.load_stores`, and `iris.load("store.zarr")` working for version 2 and version 3 stores, local and remote | +| **Behaviour change** | Three, all previously errors: a local `.zarr` directory and an `s3`/`gs`/`az`/`abfs` URL now load instead of raising; `CFVariable.spans` matches on `(name, size)` pairs rather than bare names, which is a widening no netCDF file can observe | +| **Public API change** | Additive: the new public module `iris.fileformats.zarr` (`load_cubes`), `iris.io.load_stores`, `CFDataset.format`, `CFDataset.encoding` and `CFDatasetVariable.encoding` | + +**Created:** 2026-10-13 + +Claims marked **[verified]** were checked by running code in the `iris-dev` +environment against zarr-python 3.4.1 while this plan was written; everything +else is read from source or from the spec. + +**Goal:** Make `iris.load("store.zarr")` and +`iris.load("s3://bucket/store.zarr")` return cubes from a native Zarr version 2 +or version 3 store, by implementing the `CFDataset` interface over zarr-python +and teaching Iris's load chain that a target can be a directory or a URL rather +than a seekable file. + +**Architecture:** `iris/fileformats/zarr/_dataset.py` implements `CFDataset` and +`CFDatasetVariable` over a `zarr.Group` and its `zarr.Array`s, which is all the +CF layer from PR 2 and PR 3 needs in order to classify variables and build +cubes. There is no data proxy: `zarr.Array` is picklable, so `read_data` hands +it straight to `as_lazy_data` and adds CF decoding — masking, `valid_range`, +`scale_factor`/`add_offset` — as explicit `map_blocks` layers over the raw +array, implemented as pure functions in `_decode.py`. `iris/fileformats/zarr/ +loader.py` is a thin `load_cubes` that opens a `ZarrDataset`, hands it to +`CFReader`, and runs the same `_load_cube` loop the netCDF loader runs. +`iris.io.load_stores` is the new entry point for a target that is not a file, +and a `ZarrStore` file element lets `FORMAT_AGENT` recognise one. + +**Tech Stack:** Python 3.12–3.14, zarr-python ≥3.0.8, NumPy, Dask, pytest + +pytest-xdist, pre-commit (ruff, numpydoc, mypy, sort-all), `abc.ABC`, +`collections.abc.MutableMapping`. + +**Spec:** [`../specs/2026-09-21-zarr-io-design.md`](../specs/2026-09-21-zarr-io-design.md) +— §2.4 lists the Zarr facts that shape the design, §3 the constraints agreed +before design, §4.1 the module layout, §4.2 the `CFDataset` interface, §4.4 the +whole of reading, §4.6 groups, §4.7 format detection and the load chain, §5's +fourth entry is this pull request, §6 the testing rules. The plan argues from +the spec; read both. + +--- + +## Global Constraints + +Project-wide requirements every task below implicitly carries. Values are +copied from the spec and the repository's agent rules, not restated from +memory. + +| Constraint | Value | Source | +|---|---|---| +| Target branch | `brownfield`. **Not** `main`. Reached by rebase once PR 3 merges; until then the branch sits on `zarr-io-pr-3` and the preview pull request is based there | spec §5 | +| Labels | `Agentic` and `Type: Feature Branch` | spec §5 | +| Issue linkage | `Part of #6979` — never a closing keyword; those live on the merge-back | spec §5 | +| Changelog | Fragments land with the merge-back; the pull request body must say the omission is deliberate | spec §5 | +| Dependency | `zarr` is **optional** and lazily imported. "Absent, Zarr tests skip and Zarr use raises a clear `ImportError`" | spec §3 | +| Version pin | `zarr >=3.0.8`, "which is above the known data-loss bug" | spec §8 | +| Read support | **Version 2 and version 3** | spec §3 | +| Write support | Not here. The `ZarrDataset` write surface is PR 6 | spec §5 | +| Remote stores | "URLs are passed through to zarr-python. Iris adds no dependency and handles no credentials" — no fsspec, s3fs or gcsfs in Iris's requirements | spec §3, §9 | +| Groups | "Root group by default; a `group=` keyword selects another"; Iris loads **one** group per call and does not walk nested groups | spec §3, §4.6 | +| No implementation-type tests | "No code above `cf/dataset.py` tests the implementation type. No `isinstance`, no checking a class name, no `hasattr` probe standing in for one" | spec §4.2 | +| `format` is not a branch key | "Where the generic layer needs a capability, it asks for the capability, not the format. `format` is for messages, diagnostics and the user" | spec §4.2 | +| Dimension contract | A dimension name may be absent, null, repeated within an array, or shared across arrays at different lengths. "All four are conforming Zarr. None may raise" | spec §4.4 | +| Masking precedence | Version 3: CF attributes only. Version 2: `Array.fill_value` when no CF attribute is present | spec §4.4 | +| Decode layers | "Decoding is applied as **explicit dask graph layers** over the raw array … not hidden inside an opaque `__getitem__`" | spec §4.4 | +| No `ZarrDataProxy` | "Do not mirror `NetCDFDataProxy` on the Zarr side" | spec §4.4 | +| Cache key | `(store location, array path, json.dumps(Array.metadata.to_dict(), sort_keys=True))`, treated as opaque | spec §4.4 | +| Read unit | `chunking` returns `Array.chunks`, never `Array.shards`; alignment aggregates **upward** only | spec §4.4 | +| Format detection | Local: a directory holding `zarr.json` or `.zgroup`. Remote: scheme in `s3`, `gs`, `az`, `abfs` **and** the path ends `.zarr`. "**Do not add a third rule matching any path that ends in `.zarr`**" | spec §4.7 | +| `load_files` | "is not taught about directories" — a separate `iris.io.load_stores` answers "this target is not a file", once, for local and remote alike | spec §4.7 | +| NCZarr | Unaffected. `_uri_is_nczarr` is left exactly as it is | spec §4.7, §9 | +| Attribute policy | A nested JSON object or array is carried onto the cube unchanged and not interpreted | spec §4.4 | +| Tests | "pytest style, no network, no unittest classes in new files" | spec §6 | +| Skipping | "A single `pytest.importorskip(\"zarr\")` fixture in the Zarr test package. Iris without zarr installed must show no new failures" | spec §6 | +| Remote tests | "covered by an in-memory store and a local directory store only. No test touches the network" | spec §6 | +| Every commit | `pre-commit run --files ` then `pytest`, both **before** the commit. Never `--no-verify` | `AGENTS.md` | +| Test tiers | `pytest lib/iris/tests/unit//` per edit (no `-n auto` on small runs); `-n auto` only for parent-directory and full-suite runs | `lib/iris/tests/AGENTS.md` | +| Test data | `OVERRIDE_TEST_DATA_REPOSITORY` must be set **and verified** before trusting any run; read the skip count, not just the failures | `lib/iris/tests/AGENTS.md` | +| Line length | 88 characters, Ruff default | `AGENTS.md` | +| Docstrings | NumPy style, numpydoc-validated. Module docstring mandatory outside tests (D100); a public module needs a `z_reference` item in its first 25 lines | `AGENTS.md`, `docs/AGENTS.md` | +| New `.py` files | Four-line copyright header, checked by `check-license-headers` | `AGENTS.md` | +| netCDF access | No direct `import netCDF4` — always `iris.fileformats.netcdf._thread_safe_nc` | `AGENTS.md` | +| New modules | Aim under ~1000 lines; cohesion wins over line count | `lib/iris/AGENTS.md` | +| Warning classes | Every `UserWarning` subclass descends from `IrisUserWarning`; load-time CF complaints use `IrisCfLoadWarning`, chunking complaints `IrisLoadWarning` | `AGENTS.md`, `iris.warnings` | +| Prose | No citation of a plan number, task number or phase in code, tests or docs. Cite an issue, a pull request, the design document, or the convention's section | `lib/iris/tests/AGENTS.md`, `docs/AGENTS.md` | +| Attribution | Commits end `Co-Authored-By: Claude Opus 5 `; every GitHub post opens `(posted by Claude Opus)` | `AGENTS.md` | + +--- + +## Review Focus + +Five failure modes the spec implies that **no task's own tests would otherwise +exercise**, most likely first. Each names the input or condition and the +behaviour a reasonable person expects; each has a test added to the task that +owns the code, written out in that task's own step style. + +1. **A version 2 store's `_ARRAY_DIMENSIONS` attribute landing on the loaded + cube.** It is how a version 2 store records dimension names, so under the + §4.2 split it is a *property*, not a CF attribute — and every attribute a + `CFVariable` does not consume reaches `cube.attributes` + (`cf/loader.py::_add_unused_attributes`). Nothing in `_CF_ATTRS` excludes + it, so unless `ZarrDatasetVariable.attributes` withholds it, every cube + loaded from a version 2 store gains a stray `_ARRAY_DIMENSIONS` attribute + and fails to compare equal to the same cube loaded from version 3. The user + expects the two to be indistinguishable. → **Task 3, Step 9**, and + **Task 13, Step 4** end to end. + +2. **A store whose root group holds no arrays.** Spec §4.6 requires "a message + that names the groups that do", and §7 records a real store in that shape — + the ESA EOPF Sentinel-2 product, whose root holds only subgroups. The + failure to defend against is not the message but its absence: an empty + `CubeList` returned silently from `iris.load` of a store that plainly + contains data is the worst available answer, and the one that falls out of + writing the loop without thinking about it. → **Task 10, Step 2**. + +3. **`iris.load("data/*")` matching a mix of `.nc` files and `.zarr` + directories.** Spec §4.7 states the requirement and names the consequence — + the file-or-store decision has to happen after glob expansion and before + format dispatch. The quiet failure is partial: `_generate_cubes` partitions + what `expand_filespecs` returns, so a bug that mis-partitions raises + `IsADirectoryError` from `load_files` for the stores while the netCDF files + load fine, and a user who did not expect the glob to match a store reads it + as an Iris bug. → **Task 12, Step 2**. + +4. **A `scale_factor` or `add_offset` that changes the result dtype.** + `cf/loader.py::_get_actual_dtype` computes the post-unpacking dtype and the + netCDF path gets the arithmetic itself from netCDF4. On the Zarr path Iris + does the arithmetic, in `_decode.py`, so the dask array's declared dtype and + what the decode layer actually produces are two separately-written + statements that must agree. When they disagree dask does not complain: the + graph is built, and the mismatch surfaces later as a wrong dtype on the + cube, or as a `ValueError` from a downstream operation. A `uint16` band with + `scale_factor = 0.0001` — the EOPF reflectance case in spec §7 — must give a + `float64` cube whose lazy and realised dtypes match. → **Task 7, Step 11**. + +5. **A `zarr` that is installed but too old, or not installed at all.** Spec §3 + requires "a clear `ImportError`" and §6 requires that "Iris without zarr + installed must show no new failures". Two distinct paths reach a user here — + `iris.load` of a `.zarr` target, and `import iris.fileformats.zarr` — and the + second is the one that breaks the first rule by accident, because + `iris.fileformats.__init__` imports every format module it registers. If the + Zarr import is not deferred, installing Iris without zarr stops `import iris` + working at all. → **Task 1, Steps 5–6**, and §6.3. + +--- + +## 1. Scope boundary + +**In.** + +- `lib/iris/fileformats/zarr/` — `__init__.py`, `_dataset.py`, `_decode.py`, + `loader.py`. +- `CFDataset.format` / `CFDatasetVariable.format`, and `CFDataset.encoding` / + `CFDatasetVariable.encoding`, on the interface and on both implementations. +- `CFVariable.spans` and its three overrides matching on `(name, size)` pairs. +- `CFReader` accepting an already-open `CFDataset`. +- The `"Zarr"` `FormatSpecification` and its `ZarrStore` file element. +- The four load-chain changes of spec §4.7, including `iris.io.load_stores`. +- `zarr` added to the optional-dependency section of + `requirements/py3{12,13,14}.yml`, and the lock files regenerated once. +- User-guide and API-reference documentation for Zarr loading, including the + remote URL forms and `group=`; and the S3 page rewritten. +- Unit and integration tests for all of the above. + +**Out.** + +- **Saving.** No `zarr/saver.py`, no `ZarrDataset` write path, no + `consolidate_metadata`, no registration in `_savers`. `ZarrDataset`'s + write-surface members exist because the ABC declares them and raise + `NotImplementedError`. That is PR 6. +- **The write half of the shared `CFDataset` contract.** It cannot run against + a dataset that refuses to create variables; §2 F10 explains the split. +- **Real-world fixtures and benchmarks.** The NOAA GFS and ESA EOPF samples, + and the ASV coverage, are PR 7. Every fixture here is synthetic, built in + `tmp_path` or in memory. +- **xarray as a test dependency.** Spec §6 lands it in PR 6, with the saver it + validates. Nothing here needs a second reader. +- **The netCDF read path's own sub-chunk re-read trap** (spec §12.3 Q4). The + alignment guard is added on the Zarr path only. +- **UGRID meshes from a Zarr store.** `iris.mesh.load_meshes` gains the ability + to *reach* a Zarr store (spec §4.7), because its scheme dispatch is a copy of + `loading.py`'s and the two move together. Nothing here builds or tests a mesh + from Zarr. +- **Changing the NCZarr path**, beyond leaving it alone. + +--- + +## 2. Findings that refine the spec + +Checked while writing this plan. Each is either a correction the implementer +would otherwise hit as a failing test, or an answer the spec asked for. + +**F1 — `zarr.Array` has no `dimension_names` property, and the error spec §2.4 +quotes is a *write* error.** Version 3 names live at +`Array.metadata.dimension_names`. `create_array(..., dimension_names=…)` on a +version 2 group raises `ValueError: Zarr format 2 arrays do not support +dimension names.`, which is the message the spec cites — but **reading** +`ArrayV2Metadata.dimension_names` raises `AttributeError`, because the +attribute simply is not there **[verified]**. So `ZarrDatasetVariable.dimensions` +must branch on `metadata.zarr_format` and read `attrs["_ARRAY_DIMENSIONS"]` on +version 2. A `try: … except ValueError` would not catch anything. + +**F2 — zarr-python rejects a wrong-length `dimension_names` on write and on +read, so the case is reachable only through a version 2 store.** +`create_array(..., shape=(2, 2), dimension_names=("y",))` raises +``ValueError: `dimension_names` and `shape` need to have the same number of +dimensions`` **[verified]** — and the identical check runs on **read**: a +version 3 store whose `zarr.json` is hand-edited to the wrong length raises +that same `ValueError` from `open_group(...)["v"]` itself, before Iris ever +holds an array **[verified]**. Spec §4.4's warn-and-load-anonymously rule is +therefore untestable, and unreachable, on version 3. The fixture for that +test is a version 2 store instead, whose `_ARRAY_DIMENSIONS` carries no such +validation. The length check in the reader stays format-agnostic and correct +as defensive code; it simply never fires for a version 3 array that reached +Iris at all. + +**F3 — spec §12.3 Q2 is answered: `=3.0.8` in the optional-dependency section | +| `requirements/locks/*.lock` | Regenerated, once, by `make lockfiles` | +| `docs/src/user_manual/tutorial/s3_io.rst` | Replace "S3 is unsupported, use a FUSE mount" with the Zarr route (spec §4.7) | +| `docs/src/user_manual/section_indexes/general.rst` | List the new page in the toctree, beside `netcdf_io` | +| `docs/superpowers/specs/2026-09-21-zarr-io-design.md` | §12.1 state, §12.3 Q2 closed, §12.4 entries for F10 and F11, §5's PR 4/PR 5 split amended | + +### Sequencing, and why it is this way + +The order is: make a Zarr store answerable as a `CFDataset`, then make the CF +layer able to hold one, then make Iris able to find one. + +1. **Tasks 1–7 build the Zarr dataset bottom-up** — dependency, interface, + variable, group, contract, decoding, reading — and are testable without any + of Iris's loading machinery. Each one ends with a store built in `tmp_path` + and questions asked of it directly. +2. **Tasks 8–9 are the two changes to generic code.** They are separated from + the Zarr work because they are the only parts a netCDF-only reviewer has to + check, and because `spans` is the one place in this pull request where a + mistake silently changes what loads from an existing netCDF file. +3. **Task 10 is the loader**, which can only be written once 1–9 exist. +4. **Tasks 11–12 are the load chain**, which is independent of everything above + and could be reviewed by someone who knows nothing about Zarr. They come + last among the code tasks because until Task 10 exists there is nothing for + the chain to dispatch *to*. +5. **Tasks 13–15** are integration, documentation and the pull request. + +The dependency that matters in the other direction is **PR 3**. This plan calls +`CFDatasetVariable.read_data`, `iris.fileformats.cf.loader._get_actual_dtype`, +`_LAZYVAR_MIN_BYTES` and `as_lazy_data(..., cache_key=...)`, none of which are +on `brownfield` until [#7316](https://github.com/SciTools/iris/pull/7316) +merges. So this branch is cut from `zarr-io-pr-3` rather than waiting, and +lands on `brownfield` later: + +```bash +git switch -c zarr-io-pr-4 zarr-io-pr-3 +``` + +Work proceeds on that base, and the preview pull request described in Task 15 +is raised against it on the fork. When #7316 merges, Task 15's Step 6 moves +this branch's own commits across: + +```bash +git fetch upstream +git rebase --onto upstream/brownfield zarr-io-pr-3 zarr-io-pr-4 +``` + +and re-runs Task 6 and Task 9's tests — the two that read PR 3's code most +closely — before anything is trusted again. Nothing between here and there +assumes `brownfield`; every instruction that names a base names +`zarr-io-pr-3`. + +--- + +## 4. The Zarr implementation, as designed + +Signatures the tasks below fill in. Read this before Task 1; it is the single +place the names and types agree. + +```python +# lib/iris/fileformats/cf/dataset.py (additions) + +class CFDatasetVariable(ABC): + @property + @abstractmethod + def format(self) -> str: + """The concrete format of the dataset holding this variable.""" + + @property + @abstractmethod + def encoding(self) -> Mapping[str, Any]: + """The storage settings this variable actually holds.""" + + +class CFDataset(ABC): + @property + @abstractmethod + def format(self) -> str: + """The concrete format: "NETCDF4", "NETCDF3_CLASSIC", "ZARR_V2", "ZARR_V3".""" + + @property + @abstractmethod + def encoding(self) -> Mapping[str, Any]: + """The store-wide settings this store actually holds.""" +``` + +```python +# lib/iris/fileformats/zarr/_dataset.py + +#: What CFDataset.format answers, by the store's Zarr version. +_FORMAT_BY_ZARR_VERSION = {2: "ZARR_V2", 3: "ZARR_V3"} + +#: The version 2 attribute that carries dimension names. It is storage, not a +#: CF attribute, so it is withheld from `attributes` and read by `dimensions`. +_ARRAY_DIMENSIONS = "_ARRAY_DIMENSIONS" + + +def _dimension_names(array: "zarr.Array") -> tuple[str | None, ...]: ... +def _assemble_dimensions(variables: Mapping) -> dict[str, int]: ... +def _fixed_flags(dims_fixed, ndim: int) -> tuple[bool, ...]: ... +def _store_chunk_exceeds_dask_target(store_chunks, dtype) -> bool: ... + + +def _align_chunks( + chunks: tuple[int, ...], + store_chunks: tuple[int, ...] | None, + shape: tuple[int, ...], + dtype: np.dtype, + dims_fixed, + name: str, +) -> tuple[int, ...]: + """Round `chunks` up to whole multiples of the store's own chunk shape.""" + + +class _ZarrAttributes(MutableMapping): + """A zarr node's attributes as a mapping, withholding the storage ones.""" + + def __init__(self, node, *, withhold: frozenset[str] = frozenset()): ... + + +class ZarrDatasetVariable(CFDatasetVariable): + def __init__(self, array: "zarr.Array", location: str, format: str): ... + + name: str # array.basename + location: str # the dataset's location, verbatim + format: str # "ZARR_V2" | "ZARR_V3" + dimensions: tuple[str | None, ...] # see the table in spec 4.4 + shape: tuple[int, ...] + dtype: np.dtype + size: int + fill_value: Any | None # array.fill_value: STORAGE, not a mask + chunking: tuple[int, ...] | None # array.chunks, never array.shards + encoding: Mapping[str, Any] # chunks, shards, codecs, zarr_format + attributes: MutableMapping # array.attrs, less _ARRAY_DIMENSIONS + + def __getitem__(self, keys) -> np.ndarray: ... + def __setitem__(self, keys, values) -> None: ... + def read_data(self, chunking_policy: Callable[[], tuple]) -> Any: ... + def write_handle(self) -> "ZarrDatasetVariable": ... # returns self + + @property + def cache_key(self) -> str: + """`(location, array path, the metadata document)`, as one string.""" + + @property + def zarr_format(self) -> int: + """2 or 3. Zarr-only; the generic layer never asks which version.""" + + @property + def array(self) -> "zarr.Array": + """The backing zarr array. Zarr-only; nothing generic may read it.""" + + +class ZarrDataset(CFDataset): + def __init__( + self, + location: str, + mode: str = "r", + *, + group: str | None = None, + ): ... + + location: str # the target as given, with the group path appended + format: str # "ZARR_V2" | "ZARR_V3" + mode: str + closed: bool + variables: Mapping[str, ZarrDatasetVariable] + dimensions: Mapping[str, int] # names used at one consistent size + encoding: Mapping[str, Any] # zarr_format, consolidated + attributes: MutableMapping # group.attrs + + def create_dimension(self, name, size) -> None: ... # NotImplementedError + def create_variable(self, name, dtype, dimensions=(), *, + fill_value=None, **encoding): ... # NotImplementedError + def sync(self) -> None: ... # no-op; a Zarr write is not buffered + def finalise(self) -> None: ... # NotImplementedError until PR 6 + def close(self) -> None: ... # records closure; a store is not a handle + + @property + def subgroup_names(self) -> tuple[str, ...]: + """The names of this group's subgroups. Zarr-only; for the no-data message.""" +``` + +```python +# lib/iris/fileformats/zarr/_decode.py + +def decode_fill_value(value: Any, dtype: np.dtype, variable_name: str) -> Any | None: + """Return a CF `_FillValue`/`missing_value` as a value of `dtype`, or None. + + Accepts a JSON number, and the base64 of a little-endian float64 that + released xarray writes for every floating dtype. Anything else warns and + returns None. + """ + + +def mask_and_scale( + block: np.ndarray, + *, + sentinels: tuple, + valid_min: Any | None, + valid_max: Any | None, + scale_factor: Any | None, + add_offset: Any | None, + dtype: np.dtype, +) -> np.ndarray: + """Apply the CF decode rules netCDF4 applies for free, in CF order.""" + + +def mask_by_sentinels(block: np.ndarray, sentinels: tuple) -> np.ndarray: ... +def mask_by_valid_range(block, valid_min, valid_max) -> np.ndarray: ... +def unpack(block, scale_factor, add_offset, dtype) -> np.ndarray: ... + + +def decode_parameters( + attributes: Mapping, + *, + dtype: np.dtype, + storage_fill_value: Any, + zarr_format: int, + variable_name: str, +) -> dict: + """Read the CF decode parameters for one variable. + + Plain arguments, not a variable, so that spec section 4.4's version-aware + masking precedence is testable without building a store for each case: + on version 3 only the CF attributes are a mask source; on version 2 + `Array.fill_value` is one as well, when no CF attribute is present. + + Always returns the same keys - the keyword arguments `mask_and_scale` + takes. `decoding_required` says whether they do anything. + """ + + +def decoding_required(parameters: dict) -> bool: ... +def _unpacked_dtype(dtype, scale_factor, add_offset) -> np.dtype: ... +``` + +```python +# lib/iris/fileformats/zarr/loader.py + +def load_cubes(file_sources, callback=None, constraints=None, group=None): + """Generate cubes from Zarr stores.""" +``` + +```python +# lib/iris/io/__init__.py (additions) + +#: Modules whose `save` takes a CubeList directly, rather than one cube at a +#: time with `append=True`. One line per format. +_CUBELIST_SAVER_MODULES = frozenset({"iris.fileformats.netcdf.saver"}) + +#: URI schemes Iris passes through to a store library rather than resolving as +#: a local path. +REMOTE_STORE_SCHEMES = frozenset({"s3", "gs", "az", "abfs"}) + + +def _is_remote_store_uri(spec) -> bool: ... + + +def load_stores(targets, callback, constraints=None): + """Create a generator of Cubes from targets that are stores, not files.""" +``` + +```python +# lib/iris/fileformats/__init__.py (additions) + +#: What a version 3 and a version 2 Zarr store put at the root of a store. +_ZARR_MARKERS = ("zarr.json", ".zgroup") + + +def _is_zarr_store(target) -> bool: + """Whether `target` names a native Zarr store.""" + + +def _load_zarr(*args, **kwargs): + """Load from Zarr stores, reporting clearly when zarr-python is absent.""" +``` + +```python +# lib/iris/io/format_picker.py (addition) + +class ZarrStore(FileElement): + """A :class:`FileElement` that returns a target that may be a Zarr store. + + Carries the whole target, not just its basename, because recognising a + local store means reading a marker file inside it. + """ + + def __init__(self): + super().__init__(requires_fh=False) + + def get_element(self, basename, file_handle): ... # returns basename +``` + +Three points about this shape are worth stating once, here, rather than +repeating in every task. + +**`fill_value` means different things on the two backends, deliberately.** +`NetCDFDatasetVariable.fill_value` returns the `_FillValue` *attribute*; +`ZarrDatasetVariable.fill_value` returns `Array.fill_value`, the *storage* +field. On netCDF the two coincide. On Zarr they are different things (spec +§4.4), and conflating them masks every valid zero in an array that never +declared a sentinel. Nothing generic reads `fill_value` to decide masking — +`zarr/_decode.py::decode_parameters` does, and it reads the CF attributes +first. + +**`encoding` is readable and never interpreted.** Generic code may print it, +record it, or hand it back to a user; it must not branch on a key. Its keys are +the backend's own spelling: `chunks`, `shards`, `codecs`, `zarr_format` for +Zarr; `chunksizes`, `zlib`, `complevel` for netCDF. `chunking` is the typed, +guaranteed member that the chunk calculation reads (spec §4.2). + +**`cache_key` is a member of `ZarrDatasetVariable`, not of the interface.** +`read_data` is the only caller and both implementations build their own key — +netCDF from `repr(proxy)`, Zarr from the metadata document. Putting it on the +ABC would add a member no generic code reads, which is the indirection +`lib/iris/AGENTS.md` bans for a single caller. + +--- +## 5. Tasks + +### 5.0 The green check every task ends with + +Every task's last step is a commit, and every commit is preceded by this, +unchanged: + +```bash +# 1. Confirm the test-data path took effect. A missing repository SKIPS tests +# silently, so an unset variable removes coverage without failing anything. +echo "$OVERRIDE_TEST_DATA_REPOSITORY" +ls "$OVERRIDE_TEST_DATA_REPOSITORY" > /dev/null + +# 2. Hooks rewrite files in place: re-stage and re-run until clean. +pre-commit run --files +git add +pre-commit run --files + +# 3. The tier the task's own steps name. Never --no-verify on the commit. +``` + +Read the **skip count** as well as the failure count. A Zarr test that skips +because `zarr` is missing from the environment is a test that did not run, and +`pytest -q` reports it in the same line as the passes. + +--- + +### Task 1: `zarr` as an optional dependency, and a package that does not need it + +**Files:** +- Modify: `requirements/py312.yml`, `requirements/py313.yml`, + `requirements/py314.yml` +- Modify: `requirements/locks/py312-linux-64.lock`, `py313-linux-64.lock`, + `py314-linux-64.lock` +- Create: `lib/iris/fileformats/zarr/__init__.py` +- Create: `lib/iris/tests/unit/fileformats/zarr/__init__.py` +- Create: `lib/iris/tests/unit/fileformats/zarr/stores.py` +- Create: `lib/iris/tests/unit/fileformats/zarr/test_optional_dependency.py` + +**Interfaces:** +- Consumes: nothing. +- Produces: the importable package `iris.fileformats.zarr`; the test package + whose `__init__.py` skips everything below it when `zarr` is absent; and + `stores.py`'s builders — `make_store(path, *, zarr_format=3, chunks="auto", + shards=None)`, `make_array(group, name, shape, dtype, zarr_format, *, + dimensions=None, chunks="auto", shards=None, fill_value=None, + attributes=None)`, and the constants `DIMENSIONS`, `DATA`, `VARIABLE_ATTRS`, + `GLOBALS` — which every later Zarr test builds its fixtures from. + +The package's `__init__.py` re-exports nothing yet; `loader.py` does not exist +until Task 10, and a re-export of a name that is not there is an import error. +Task 10 adds the `from .loader import load_cubes` line. The registration that +gives a user a clear `ImportError` is Task 11, beside the format +specification, which is the only place that needs it. + +- [ ] **Step 1: Add the dependency to all three requirement files** + +In each of `requirements/py312.yml`, `py313.yml` and `py314.yml`, under the +`# Optional dependencies.` heading, after ` - affine`: + +```yaml + - zarr >=3.0.8 +``` + +The floor is spec §8: "Pin `>=3.0.8`, which is above the known data-loss bug". + +- [ ] **Step 2: Regenerate the lock files** + +```bash +make lockfiles +git status --short requirements/locks/ +``` + +Expected: the three `py3{12,13,14}-linux-64.lock` files change, each gaining +`zarr`, `numcodecs`, `donfig` and their own dependencies. + +This needs network access and takes minutes. If it cannot run here, say so in +the pull request body and ask a maintainer to regenerate — do **not** hand-edit +a lock file. + +- [ ] **Step 3: Create the package** + +Create `lib/iris/fileformats/zarr/__init__.py`: + +```python +# Copyright Iris contributors +# +# This file is part of Iris and is released under the BSD license. +# See LICENSE in the root of the repository for full licensing details. +"""Native Zarr support, built on zarr-python. + +.. z_reference:: iris.fileformats.zarr + :tags: topic_load_save + + API reference + +Reads Zarr version 2 and version 3 stores, local or remote. Not NCZarr: that +is netCDF-c's own Zarr driver, reached through a URL fragment such as +``file:///path/store.zarr#mode=nczarr,file``, and it stays where it is in +:mod:`iris.fileformats.netcdf`. + +``zarr`` is an optional dependency. Nothing in :mod:`iris.fileformats` imports +this package until a Zarr target is actually loaded, so Iris installs and +imports without it. + +This package shadows the third-party ``zarr`` distribution by name. Python 3 +resolves ``import zarr`` absolutely, so inside these modules it still finds the +third-party package; xarray's ``xarray.backends.zarr`` has the same +arrangement. + +See sections 4.1 and 4.4 of +``docs/superpowers/specs/2026-09-21-zarr-io-design.md``. + +""" +``` + +- [ ] **Step 4: Create the test package, with the skip that guards it** + +Create `lib/iris/tests/unit/fileformats/zarr/__init__.py`: + +```python +# Copyright Iris contributors +# +# This file is part of Iris and is released under the BSD license. +# See LICENSE in the root of the repository for full licensing details. +"""Unit tests for :mod:`iris.fileformats.zarr`. + +``zarr`` is optional, so every module below this one is skipped when it is +absent. The skip lives here rather than in a ``conftest.py``: the same call +there raises during collection and is reported as an **error**, not a skip. +""" + +import pytest + +pytest.importorskip("zarr", minversion="3.0.8") +``` + +- [ ] **Step 5: Write the failing test for the import guard** + +Create `lib/iris/tests/unit/fileformats/zarr/test_optional_dependency.py`: + +```python +# Copyright Iris contributors +# +# This file is part of Iris and is released under the BSD license. +# See LICENSE in the root of the repository for full licensing details. +"""Iris must import, and load netCDF, without zarr-python installed.""" + +import subprocess +import sys + + +def _imports_zarr(statement): + """Run `statement` in a fresh interpreter; report whether zarr was imported.""" + probe = f"{statement}\nimport sys\nprint('zarr' in sys.modules)\n" + completed = subprocess.run( + [sys.executable, "-c", probe], + capture_output=True, + text=True, + check=True, + ) + return completed.stdout.strip() == "True" + + +def test_importing_iris_does_not_import_zarr(): + assert not _imports_zarr("import iris") + + +def test_importing_the_format_registry_does_not_import_zarr(): + # iris.fileformats imports every format module it registers a handler + # from. If the Zarr loader joined that list, Iris would stop installing + # without zarr-python. + assert not _imports_zarr("import iris.fileformats") + + +def test_importing_the_zarr_package_does_not_import_zarr(): + # The package docstring is reachable without the dependency; only the + # modules under it need zarr itself. + assert not _imports_zarr("import iris.fileformats.zarr") +``` + +- [ ] **Step 6: Run the tests to see them pass, and know why** + +```bash +pytest lib/iris/tests/unit/fileformats/zarr/test_optional_dependency.py -v +``` + +Expected: 3 passed. These pass from the start — that is the point. They are a +*regression* guard, not a red-green cycle: the way they fail is by someone +later adding `from . import zarr` to `iris/fileformats/__init__.py`, or an +`import zarr` to `iris/fileformats/zarr/__init__.py`. Prove they can fail +before trusting them, by temporarily adding `import zarr` to the package +`__init__.py` and re-running: + +```bash +pytest lib/iris/tests/unit/fileformats/zarr/test_optional_dependency.py -v +``` + +Expected with the temporary line: `test_importing_the_zarr_package_does_not_import_zarr` +FAILS. Remove the line. + +- [ ] **Step 7: Write the shared store builders** + +Create `lib/iris/tests/unit/fileformats/zarr/stores.py`: + +```python +# Copyright Iris contributors +# +# This file is part of Iris and is released under the BSD license. +# See LICENSE in the root of the repository for full licensing details. +"""Builders for the small Zarr stores these tests ask questions of. + +One canonical store, deliberately the same contents as the shared CFDataset +contract writes, so that a Zarr answer and a netCDF answer can be compared +without translating between two sets of fixtures. +""" + +import numpy as np +import zarr + +#: The canonical store's dimensions, in order. +DIMENSIONS = {"y": 3, "x": 2} +DATA = np.arange(6, dtype="f8").reshape(3, 2) +VARIABLE_ATTRS = {"units": "K", "long_name": "surface temperature"} +GLOBALS = {"Conventions": "CF-1.7", "title": "contract"} + +#: The version 2 attribute carrying dimension names. +ARRAY_DIMENSIONS = "_ARRAY_DIMENSIONS" + + +def make_array( + group, + name, + shape, + dtype, + zarr_format, + *, + dimensions=None, + chunks="auto", + shards=None, + fill_value=None, + attributes=None, +): + """Create one array, recording dimension names the way its version does. + + zarr-python refuses `dimension_names` on a version 2 array, so version 2 + names are written to the `_ARRAY_DIMENSIONS` attribute by hand - which is + what xarray and NCZarr both do. + """ + extra = {} + if zarr_format == 3 and dimensions is not None: + extra["dimension_names"] = dimensions + array = group.create_array( + name, + shape=shape, + dtype=dtype, + chunks=chunks, + shards=shards, + fill_value=fill_value, + **extra, + ) + if zarr_format == 2 and dimensions is not None: + array.attrs[ARRAY_DIMENSIONS] = list(dimensions) + for key, value in (attributes or {}).items(): + array.attrs[key] = value + return array + + +def make_store(path, *, zarr_format=3, chunks="auto", shards=None): + """Write the canonical store at `path`, and return `path` as a string.""" + root = zarr.open_group(store=path, mode="w", zarr_format=zarr_format) + for name, value in GLOBALS.items(): + root.attrs[name] = value + + array = make_array( + root, + "air_temperature", + DATA.shape, + "f8", + zarr_format, + dimensions=tuple(DIMENSIONS), + chunks=chunks, + shards=shards, + fill_value=-1.0, + attributes=VARIABLE_ATTRS, + ) + array[:] = DATA + + # A dimensionless variable: what a grid mapping is, and the one shape a + # CFDatasetVariable must handle without a leading dimension. + scalar = make_array( + root, + "grid", + (), + "i4", + zarr_format, + dimensions=(), + attributes={"grid_mapping_name": "latitude_longitude"}, + ) + scalar[()] = 0 + + return str(path) +``` + +- [ ] **Step 8: Prove the builders build both versions** + +Append to `test_optional_dependency.py`: + +```python +import zarr + +from . import stores + + +def test_make_store_version_3(tmp_path): + location = stores.make_store(tmp_path / "v3.zarr") + group = zarr.open_group(store=location, mode="r") + assert group["air_temperature"].metadata.zarr_format == 3 + assert group["air_temperature"].metadata.dimension_names == ("y", "x") + + +def test_make_store_version_2(tmp_path): + location = stores.make_store(tmp_path / "v2.zarr", zarr_format=2) + group = zarr.open_group(store=location, mode="r") + array = group["air_temperature"] + assert array.metadata.zarr_format == 2 + assert array.attrs[stores.ARRAY_DIMENSIONS] == ["y", "x"] +``` + +- [ ] **Step 9: Run them** + +```bash +pytest lib/iris/tests/unit/fileformats/zarr/ -v +``` + +Expected: 5 passed. + +- [ ] **Step 10: Commit** + +Run §5.0, then: + +```bash +git add requirements lib/iris/fileformats/zarr lib/iris/tests/unit/fileformats/zarr +git commit -m "Add zarr as an optional dependency, and the package that uses it + +The package holds only its docstring for now: iris.fileformats does not +import it, so Iris still installs and imports without zarr-python. The +test package skips itself when the dependency is absent." +``` + +--- + +### Task 2: `format` and `encoding` on the interface, and on netCDF + +**Files:** +- Modify: `lib/iris/fileformats/cf/dataset.py` +- Modify: `lib/iris/fileformats/netcdf/_dataset.py` +- Modify: `lib/iris/tests/unit/fileformats/cf/dataset/contract.py` +- Modify: `lib/iris/tests/unit/fileformats/netcdf/dataset/test_NetCDFDataset__contract.py` + — this is where `TestNetCDFDatasetContract` lives, not in + `test_NetCDFDataset.py` + +**Interfaces:** +- Consumes: the existing `CFDataset` / `CFDatasetVariable` from PR 2. +- Produces: `CFDataset.format`, `CFDataset.encoding`, + `CFDatasetVariable.format`, `CFDatasetVariable.encoding`; and the contract + split into `CFDatasetReadContract` and `CFDatasetWriteContract` (F10), which + Task 5 subclasses. + +Spec §5 gives `format` to PR 5. It has to be here instead, because PR 4 merges +first and a `ZarrDataset` that cannot say which Zarr version it is cannot +implement §4.4's masking rule or name itself in a warning (F11). +`supports_dtype` is a saver question and stays in PR 5. + +- [ ] **Step 1: Write the failing contract tests** + +In `lib/iris/tests/unit/fileformats/cf/dataset/contract.py`, add to +`CFDatasetContract`, under the `# -- The dataset itself ---` heading: + +```python + def test_format_is_a_non_empty_string(self, readable): + assert isinstance(readable.format, str) + assert readable.format + + def test_encoding_is_a_readable_mapping(self, readable): + # Its keys are the backend's own spelling and generic code must never + # branch on one, but it must be possible to read them: otherwise a + # caller that wants to know how a store is laid out has to reach + # through to the netCDF4 or zarr object, which is the abstraction + # leaking at the point it exists to hold. + assert isinstance(readable.encoding, Mapping) + dict(readable.encoding) +``` + +and under `# -- A variable ---`: + +```python + def test_variable_format_matches_its_dataset(self, variable, readable): + assert variable.format == readable.format + + def test_variable_encoding_reports_the_chunking(self, variable): + encoding = variable.encoding + assert isinstance(encoding, Mapping) + # Whatever a backend spells it, the chunk shape it reports through + # `chunking` has to be somewhere in `encoding` too, or `encoding` is + # not reporting what the store holds. + assert variable.chunking in (None, *encoding.values()) +``` + +Add `from collections.abc import Mapping` to the module's imports. + +- [ ] **Step 2: Run them to verify they fail** + +```bash +pytest lib/iris/tests/unit/fileformats/netcdf/dataset/ -v -k "format or encoding" +``` + +Expected: FAIL, `AttributeError: 'NetCDFDataset' object has no attribute 'format'`. + +- [ ] **Step 3: Add the two members to the interface** + +In `lib/iris/fileformats/cf/dataset.py`, in `CFDatasetVariable`, after +`location`: + +```python + @property + @abstractmethod + def format(self) -> str: + """The concrete format of the dataset holding this variable. + + Repeated here from the dataset so that a variable can answer for + itself, as :attr:`location` is. For messages and diagnostics only: + generic code asks for a capability, never for a format. + + """ +``` + +and after `chunking`: + +```python + @property + @abstractmethod + def encoding(self) -> Mapping: + """The storage settings this variable holds, in the backend's spelling. + + ``zlib``, ``complevel``, ``chunksizes`` for netCDF; ``chunks``, + ``shards``, ``codecs``, ``zarr_format`` for Zarr. Readable, never + interpreted: no generic code may branch on a key. The typed, + guaranteed-present member is :attr:`chunking`. + + """ +``` + +In `CFDataset`, after `mode`: + +```python + @property + @abstractmethod + def format(self) -> str: + """The concrete format: ``NETCDF4``, ``NETCDF3_CLASSIC``, ``ZARR_V2``, ``ZARR_V3``. + + What the loader, the saver and ``LOAD_PROBLEMS`` *say* when they have + to name the format. Reconstructing it from the implementation class + would be the type test this interface exists to forbid. + + """ + + @property + @abstractmethod + def encoding(self) -> Mapping: + """The store-wide settings this store holds, in the backend's spelling.""" +``` + +`Mapping` is already imported in that module. + +- [ ] **Step 4: Implement them for netCDF** + +In `lib/iris/fileformats/netcdf/_dataset.py`, in `NetCDFDatasetVariable`, after +`location`: + +```python + @property + def format(self) -> str: + """The netCDF format of the file holding this variable.""" + return self._format +``` + +taking a `format` argument in `__init__` — add it to the signature after +`location`, store it as `self._format`, and pass +`self._dataset.file_format` from both construction sites in `NetCDFDataset` +(`_materialised_variables` and `create_variable`). + +After `chunking`: + +```python + @property + def encoding(self) -> Mapping: + """The netCDF storage settings this variable holds. + + Read back off the file, so a variable created with `zlib=True` reports + the compression netCDF4 actually applied rather than the keyword it + was given. + + """ + filters = self._variable.filters() or {} + return MappingProxyType({**filters, "chunksizes": self.chunking}) +``` + +Add `from types import MappingProxyType` to the module's imports. + +In `NetCDFDataset`, after `mode`: + +```python + @property + def format(self) -> str: + """The netCDF format of this file, as netCDF4 reports it.""" + assert self._dataset is not None + return self._dataset.file_format + + @property + def encoding(self) -> Mapping: + """The store-wide netCDF settings. Only the format is one.""" + return MappingProxyType({"format": self.format}) +``` + +- [ ] **Step 5: Run the contract** + +```bash +pytest lib/iris/tests/unit/fileformats/netcdf/dataset/ -v +``` + +Expected: all pass, including the four new cases. + +- [ ] **Step 6: Split the contract into its read and write halves** + +Rename `CFDatasetContract` to `CFDatasetReadContract`, keeping everything +above the `# -- Writing ---` heading and the `readable`, `variable` and +`scalar` fixtures. Move everything below that heading, plus the `writable` and +`reopen` fixtures and `populate()`, into a new class +`CFDatasetWriteContract` in the same module. Update the module docstring: + +```python +"""One test body, run against every :class:`CFDataset` implementation. + +Subclass :class:`CFDatasetReadContract` in a module named for the +implementation, supply the fixtures it declares, and every test here runs +against it. Both classes deliberately lack a ``Test`` prefix, so pytest does +not collect them where they are written, only where they are subclassed. + +The split is read from write because an implementation may arrive in two +pieces: the Zarr loader can answer every question in the read half before a +Zarr saver exists. Skipping half of one class instead would hide an +incomplete implementation behind a skip, and the point of a shared body is +that divergence fails loudly. + +``docs/superpowers/specs/2026-09-21-zarr-io-design.md`` section 6: the +interface is only worth having if both sides of it agree, and agreement is +cheapest to check by asking the same questions twice. + +Deliberately not exercised here: :meth:`CFDatasetVariable.deprecated_netcdf_member`. +It is an explicitly netCDF4-specific compatibility escape hatch, so asserting +either its presence or its absence in a shared body would itself be +backend-specific - the omission is intentional, not a gap. +""" +``` + +In `lib/iris/tests/unit/fileformats/netcdf/dataset/test_NetCDFDataset__contract.py`, +change the test class to inherit both, and update its import: + +```python +from iris.tests.unit.fileformats.cf.dataset.contract import ( + CFDatasetReadContract, + CFDatasetWriteContract, +) + + +class TestNetCDFDatasetContract(CFDatasetReadContract, CFDatasetWriteContract): +``` + +- [ ] **Step 7: Run the netCDF contract again and compare the count** + +```bash +pytest lib/iris/tests/unit/fileformats/netcdf/dataset/ -q +``` + +Expected: the same number of tests as before the split, plus the four added in +Step 1. The split must move tests, not lose them — count before and after. + +- [ ] **Step 8: Commit** + +Run §5.0, then: + +```bash +git add lib/iris/fileformats/cf/dataset.py lib/iris/fileformats/netcdf/_dataset.py lib/iris/tests/unit/fileformats +git commit -m "Add format and encoding to the CF dataset interface + +format names the concrete format for a message or a diagnostic; encoding +reports the storage settings a store actually holds, readable but never +interpreted. Splits the shared contract into its read and write halves, so +that an implementation arriving in two pieces can be held to the first." +``` + +--- + +### Task 3: `ZarrDatasetVariable` — the properties, and the dimension contract + +**Files:** +- Create: `lib/iris/fileformats/zarr/_dataset.py` +- Create: `lib/iris/tests/unit/fileformats/zarr/dataset/__init__.py` +- Create: `lib/iris/tests/unit/fileformats/zarr/dataset/test_ZarrDatasetVariable.py` + +**Interfaces:** +- Consumes: `CFDatasetVariable` with `format` and `encoding` (Task 2); + `stores.make_array`, `stores.make_store` (Task 1). +- Produces: `ZarrDatasetVariable(array, location, format)` with every read + member of the interface except `read_data`, which is Task 7; the constants + `_FORMAT_BY_ZARR_VERSION` and `_ARRAY_DIMENSIONS`; and the module docstring + that later tasks add to. + +Spec §4.4's dimension table is the whole of this task: + +| store | `dimensions` | +|---|---| +| version 3, `dimension_names` present | its entries, null becoming `None` | +| version 3, `dimension_names` absent | `(None,) * ndim` | +| version 2, `_ARRAY_DIMENSIONS` present | its entries | +| version 2, `_ARRAY_DIMENSIONS` absent | `(None,) * ndim` | +| any version, scalar array | `()` | +| any version, wrong length | `(None,) * ndim`, with an `IrisCfLoadWarning` | + +- [ ] **Step 1: Write the failing tests for the dimension table** + +Create `lib/iris/tests/unit/fileformats/zarr/dataset/__init__.py` with only the +copyright header, then +`lib/iris/tests/unit/fileformats/zarr/dataset/test_ZarrDatasetVariable.py`: + +```python +# Copyright Iris contributors +# +# This file is part of Iris and is released under the BSD license. +# See LICENSE in the root of the repository for full licensing details. +"""Unit tests for :class:`iris.fileformats.zarr._dataset.ZarrDatasetVariable`.""" + +import json + +import numpy as np +import pytest +import zarr + +from iris.fileformats.zarr._dataset import ZarrDatasetVariable +from iris.warnings import IrisCfLoadWarning + +from .. import stores + + +def wrap(array, location="store.zarr"): + """Return the ZarrDatasetVariable over `array`, naming its own format.""" + zarr_format = array.metadata.zarr_format + return ZarrDatasetVariable(array, location, f"ZARR_V{zarr_format}") + + +@pytest.fixture(params=[2, 3], ids=["v2", "v3"]) +def zarr_format(request): + return request.param + + +@pytest.fixture +def group(tmp_path, zarr_format): + return zarr.open_group( + store=tmp_path / "probe.zarr", mode="w", zarr_format=zarr_format + ) + + +def test_named_dimensions(group, zarr_format): + array = stores.make_array( + group, "v", (3, 2), "f8", zarr_format, dimensions=("y", "x") + ) + assert wrap(array).dimensions == ("y", "x") + + +def test_unnamed_dimensions_are_anonymous(group, zarr_format): + array = stores.make_array(group, "v", (3, 2), "f8", zarr_format) + assert wrap(array).dimensions == (None, None) + + +def test_scalar_has_no_dimensions(group, zarr_format): + array = stores.make_array(group, "v", (), "i4", zarr_format) + assert wrap(array).dimensions == () + + +def test_partially_named_dimensions_version_3(tmp_path): + # The version 3 core specification: "an unnamed dimension is indicated by + # the null object". A store may name some dimensions and not others. + group = zarr.open_group(store=tmp_path / "p.zarr", mode="w", zarr_format=3) + array = group.create_array( + "v", shape=(3, 2), dtype="f8", dimension_names=("y", None) + ) + assert wrap(array).dimensions == ("y", None) + + +def test_repeated_name_within_one_array(tmp_path): + # "It is recommended but not required that all non-null dimension names + # are distinct." Recommended is not required, so this must not raise. + group = zarr.open_group(store=tmp_path / "p.zarr", mode="w", zarr_format=3) + array = group.create_array( + "v", shape=(2, 2), dtype="f8", dimension_names=("d", "d") + ) + assert wrap(array).dimensions == ("d", "d") +``` + +- [ ] **Step 2: Add the malformed-store test** + +zarr-python rejects a wrong-length `dimension_names` at creation (F2), so this +one writes the metadata by hand. Append: + +```python +def test_wrong_length_dimension_names_warns_and_loads_anonymous(tmp_path): + """A `dimension_names` whose length is not the rank is a malformation. + + The version 3 core specification says the field "must be an array of + strings or null objects with the same length as `shape`". zarr-python + refuses to write one, so the metadata is edited directly. + """ + group = zarr.open_group(store=tmp_path / "bad.zarr", mode="w", zarr_format=3) + group.create_array("v", shape=(3, 2), dtype="f8", dimension_names=("y", "x")) + + metadata_path = tmp_path / "bad.zarr" / "v" / "zarr.json" + document = json.loads(metadata_path.read_text()) + document["dimension_names"] = ["y"] + metadata_path.write_text(json.dumps(document)) + + array = zarr.open_group(store=tmp_path / "bad.zarr", mode="r")["v"] + with pytest.warns(IrisCfLoadWarning, match="dimension_names"): + dimensions = wrap(array).dimensions + assert dimensions == (None, None) +``` + +- [ ] **Step 3: Run them to verify they fail** + +```bash +pytest lib/iris/tests/unit/fileformats/zarr/dataset/ -v +``` + +Expected: collection error, +`ModuleNotFoundError: No module named 'iris.fileformats.zarr._dataset'`. + +- [ ] **Step 4: Write the module, and the variable's dimension logic** + +Create `lib/iris/fileformats/zarr/_dataset.py`: + +```python +# Copyright Iris contributors +# +# This file is part of Iris and is released under the BSD license. +# See LICENSE in the root of the repository for full licensing details. +"""The Zarr implementation of the :mod:`iris.fileformats.cf.dataset` interface. + +This is the only module that knows both the CF interface and the zarr-python +API. Everything Zarr-specific the CF layer could want - where dimension names +are recorded, what a chunk is, what the store's own fill value is - either has +a named member on the interface or lives here as a Zarr-only member. + +Three things about Zarr shape this module, and each is a trap if forgotten. + +**A dimension name guarantees less than a netCDF one.** The version 3 core +specification makes ``dimension_names`` optional, allows a null entry, only +*recommends* that names are distinct within an array, and places no +restriction at all on reusing a name across arrays. All of that is conforming +Zarr, so none of it may raise. A dimension with no name is **anonymous**, which +Iris has always had: a cube dimension carrying no coordinate. + +**``Array.fill_value`` is not ``_FillValue``.** It says what unwritten storage +reads back as; it does not say the value is missing. zarr-python defaults it to +zero, so reading it as a missing-value sentinel masks every valid zero in an +array that never declared one. :attr:`ZarrDatasetVariable.fill_value` reports +it as the storage fact it is, and the masking decision is made in +:mod:`iris.fileformats.zarr._decode` from the CF attributes. + +**The read unit is the inner chunk, even when the array is sharded.** Zarr's +sharding codec is indexed, so a reader fetches only the inner chunks it needs. +:attr:`ZarrDatasetVariable.chunking` therefore returns ``Array.chunks``, never +``Array.shards``, and :func:`_align_chunks` aggregates upward into whole +multiples of it. + +See sections 4.2, 4.4 and 4.6 of +``docs/superpowers/specs/2026-09-21-zarr-io-design.md``. + +""" + +from collections.abc import Iterator, Mapping, MutableMapping +from types import MappingProxyType +from typing import Any +import warnings + +import numpy as np +import zarr + +from iris.fileformats.cf.dataset import CFDataset, CFDatasetVariable +import iris.warnings + +#: What :attr:`CFDataset.format` answers, by the store's Zarr version. +_FORMAT_BY_ZARR_VERSION = {2: "ZARR_V2", 3: "ZARR_V3"} + +#: The version 2 attribute carrying dimension names, as xarray and NCZarr +#: write it. Storage, not a CF attribute, so it is withheld from `attributes`. +_ARRAY_DIMENSIONS = "_ARRAY_DIMENSIONS" + + +def _dimension_names(array) -> tuple: + """Return one name or None per dimension of `array`. + + Parameters + ---------- + array : :class:`zarr.Array` + The array to read names from. Version 3 carries them in its metadata; + version 2 carries them in the ``_ARRAY_DIMENSIONS`` attribute, which + is a de-facto convention rather than part of the Zarr specification. + + Returns + ------- + tuple + Of length ``array.ndim``. ``None`` where the store names no dimension. + + Warns + ----- + IrisCfLoadWarning + If the store names a number of dimensions other than ``array.ndim``. + The version 3 specification says the lengths "must" match, so such a + store is malformed; its names are ignored and the array loads with + anonymous dimensions. + + """ + anonymous = (None,) * array.ndim + if not array.ndim: + return () + + if array.metadata.zarr_format == 3: + names = array.metadata.dimension_names + else: + names = array.attrs.get(_ARRAY_DIMENSIONS) + + if names is None: + return anonymous + + if len(names) != array.ndim: + message = ( + "Ignoring malformed dimension_names, which names " + f"{len(names)} dimensions of a {array.ndim}-dimensional array: " + f"{array.basename}" + ) + warnings.warn(message, category=iris.warnings.IrisCfLoadWarning) + return anonymous + + return tuple(names) +``` + +- [ ] **Step 5: Write the class** + +Append to `_dataset.py`: + +```python +class ZarrDatasetVariable(CFDatasetVariable): + """One array of a Zarr store, presented through the CF interface.""" + + def __init__(self, array, location: str, format: str): + """Wrap `array`. + + Parameters + ---------- + array : :class:`zarr.Array` + The array this variable presents. + location : str + The path or URL of the dataset this array belongs to. + format : str + ``ZARR_V2`` or ``ZARR_V3``, as its dataset reports. + + """ + self._array = array + self._location = location + self._format = format + self._dimensions = _dimension_names(array) + self._attributes = _ZarrAttributes(array, withhold={_ARRAY_DIMENSIONS}) + + @property + def name(self) -> str: + """The array's name within its group.""" + return self._array.basename + + @property + def location(self) -> str: + """The path or URL of the dataset holding this variable.""" + return self._location + + @property + def format(self) -> str: + """``ZARR_V2`` or ``ZARR_V3``.""" + return self._format + + @property + def dimensions(self) -> tuple: + """One name per dimension, ``None`` where the store names none.""" + return self._dimensions + + @property + def shape(self) -> tuple: + """The array's shape.""" + return tuple(self._array.shape) + + @property + def dtype(self) -> np.dtype: + """The array's stored data type.""" + return self._array.dtype + + @property + def size(self) -> int: + """The total number of elements in the array.""" + return int(self._array.size) + + @property + def fill_value(self) -> Any: + """What unwritten storage reads back as. **Not** a missing-value flag. + + Version 3 requires this field, and zarr-python defaults it to zero, so + it says nothing about whether a value is missing. The CF + ``_FillValue`` and ``missing_value`` attributes say that, and + :mod:`iris.fileformats.zarr._decode` reads them. + + """ + return self._array.fill_value + + @property + def chunking(self) -> tuple | None: + """The array's inner chunk shape, never its shard shape.""" + chunks = self._array.chunks + return tuple(chunks) if chunks else None + + @property + def encoding(self) -> Mapping: + """The Zarr storage settings this array holds.""" + return MappingProxyType( + { + "chunks": self._array.chunks, + "shards": self._array.shards, + "codecs": self._array.metadata.to_dict().get("codecs"), + "zarr_format": self._array.metadata.zarr_format, + } + ) + + @property + def attributes(self) -> MutableMapping: + """The array's CF attributes, less the ones that are really storage.""" + return self._attributes + + def __getitem__(self, keys) -> np.ndarray: + """Return the indexed portion of the array's data.""" + return self._array[keys] + + def __setitem__(self, keys, values) -> None: + """Write `values` into the indexed portion of the array.""" + self._array[keys] = values + + def write_handle(self) -> "ZarrDatasetVariable": + """Return self: a Zarr write target needs no proxy. + + ``zarr.Array`` is picklable and a chunk is an independent object in a + key-value store, so there is no open file handle to avoid holding and + no lock to acquire. + + """ + return self + + def __repr__(self) -> str: + """Return a string representation.""" + return f"{self.__class__.__name__}({self.name!r}, {self.location!r})" + + # Zarr-only members below. + + @property + def array(self): + """The backing :class:`zarr.Array`. Nothing generic may read this.""" + return self._array +``` + +Note `chunking` returns `None` for a scalar array, because `Array.chunks` is +`()` there and an empty tuple is not a chunk shape. + +- [ ] **Step 6: Write the attributes mapping** + +Append to `_dataset.py`, above `ZarrDatasetVariable`: + +```python +class _ZarrAttributes(MutableMapping): + """A Zarr node's attributes as a mapping, less the ones that are storage. + + ``_ARRAY_DIMENSIONS`` is how a version 2 store records dimension names. + Under the interface's split that is a property, reported by + :attr:`ZarrDatasetVariable.dimensions`, not a CF attribute - so it is + withheld here. Leaving it in would put it on every cube loaded from a + version 2 store, since every attribute the loading rules do not consume + reaches ``cube.attributes``. + """ + + def __init__(self, node, *, withhold: frozenset = frozenset()): + """Materialise the attributes of `node`, a zarr array or group.""" + self._node = node + self._withhold = frozenset(withhold) + self._values = { + key: value + for key, value in node.attrs.items() + if key not in self._withhold + } + + def __getitem__(self, key: str) -> Any: + """Return `key`'s value.""" + return self._values[key] + + def __setitem__(self, key: str, value: Any) -> None: + """Set `key`'s value, here and in the store.""" + if key in self._withhold: + msg = f"Cannot set a storage attribute through `attributes`: {key}" + raise KeyError(msg) + self._node.attrs[key] = value + self._values[key] = value + + def __delitem__(self, key: str) -> None: + """Remove `key`, here and from the store.""" + del self._node.attrs[key] + del self._values[key] + + def __iter__(self) -> Iterator[str]: + """Return an iterator over the attribute names, in store order.""" + return iter(self._values) + + def __len__(self) -> int: + """Return the number of attributes.""" + return len(self._values) + + def __repr__(self) -> str: + """Return a string representation.""" + return f"{self.__class__.__name__}({self._values!r})" +``` + +- [ ] **Step 7: Run the dimension tests** + +```bash +pytest lib/iris/tests/unit/fileformats/zarr/dataset/ -v +``` + +Expected: all pass. + +- [ ] **Step 8: Write the tests for the remaining properties** + +Append to `test_ZarrDatasetVariable.py`: + +```python +def test_plain_properties(tmp_path, zarr_format): + location = stores.make_store(tmp_path / "s.zarr", zarr_format=zarr_format) + array = zarr.open_group(store=location, mode="r")["air_temperature"] + variable = wrap(array, location) + + assert variable.name == "air_temperature" + assert variable.location == location + assert variable.format == f"ZARR_V{zarr_format}" + assert variable.shape == stores.DATA.shape + assert variable.dtype == np.dtype("f8") + assert variable.size == stores.DATA.size + assert variable.ndim == stores.DATA.ndim + assert len(variable) == stores.DATA.shape[0] + np.testing.assert_array_equal(variable[:], stores.DATA) + np.testing.assert_array_equal(variable[1], stores.DATA[1]) + + +def test_fill_value_is_the_storage_field(tmp_path, zarr_format): + """`fill_value` is what unwritten storage reads as, not `_FillValue`. + + zarr-python defaults the field to zero, so an array that declares no CF + sentinel still reports one here. Reading it as a mask source is what + masks every valid zero. + """ + group = zarr.open_group( + store=tmp_path / "f.zarr", mode="w", zarr_format=zarr_format + ) + array = stores.make_array(group, "v", (3,), "i2", zarr_format) + assert wrap(array).fill_value == 0 + + declared = stores.make_array( + group, "w", (3,), "i2", zarr_format, fill_value=-999 + ) + assert wrap(declared).fill_value == -999 + + +def test_chunking_is_the_inner_chunk_not_the_shard(tmp_path): + """A sharded array reports `Array.chunks`. + + Zarr's sharding codec is indexed: a reader fetches the shard index, then + byte ranges for only the inner chunks it needs. The inner chunk is the + independently decodable unit. + """ + group = zarr.open_group(store=tmp_path / "sh.zarr", mode="w", zarr_format=3) + array = group.create_array( + "v", shape=(1024,), dtype="i4", chunks=(8,), shards=(1024,) + ) + variable = wrap(array) + assert variable.chunking == (8,) + assert variable.encoding["shards"] == (1024,) + + +def test_encoding_reports_the_store(tmp_path, zarr_format): + location = stores.make_store(tmp_path / "e.zarr", zarr_format=zarr_format) + array = zarr.open_group(store=location, mode="r")["air_temperature"] + encoding = wrap(array, location).encoding + assert encoding["chunks"] == array.chunks + assert encoding["zarr_format"] == zarr_format +``` + +- [ ] **Step 9: Write the test for Review Focus 1 — `_ARRAY_DIMENSIONS`** + +Append: + +```python +def test_array_dimensions_is_not_a_cf_attribute(tmp_path): + """`_ARRAY_DIMENSIONS` is storage, so it must not reach `attributes`. + + It is how a version 2 store records dimension names. Every attribute the + loading rules do not consume reaches `cube.attributes`, so leaving it in + would put it on every cube loaded from a version 2 store, and make that + cube unequal to the same data loaded from version 3. + """ + group = zarr.open_group(store=tmp_path / "v2.zarr", mode="w", zarr_format=2) + array = stores.make_array( + group, + "v", + (3, 2), + "f8", + 2, + dimensions=("y", "x"), + attributes={"units": "K"}, + ) + variable = wrap(array) + + assert variable.dimensions == ("y", "x") + assert stores.ARRAY_DIMENSIONS in array.attrs + assert stores.ARRAY_DIMENSIONS not in variable.attributes + assert dict(variable.attributes) == {"units": "K"} + + +def test_version_2_and_version_3_present_the_same_attributes(tmp_path): + v2 = stores.make_store(tmp_path / "a2.zarr", zarr_format=2) + v3 = stores.make_store(tmp_path / "a3.zarr", zarr_format=3) + from_2 = zarr.open_group(store=v2, mode="r")["air_temperature"] + from_3 = zarr.open_group(store=v3, mode="r")["air_temperature"] + assert dict(wrap(from_2, v2).attributes) == dict(wrap(from_3, v3).attributes) +``` + +- [ ] **Step 10: Write the test for nested and vector attributes (F14)** + +Append: + +```python +def test_structured_attribute_values_are_carried_unchanged(tmp_path): + """A JSON object or array attribute is carried, not interpreted. + + The version 3 specification allows an attribute to be any JSON literal, + and CF cannot express a nested one at any rank. Two independent producers + write them - NOAA GFS gives `valid_time` a `statistics_approximate` + object, and the ESA EOPF product root carries `stac_discovery`. + A flat numeric array is an ordinary CF vector attribute and is not special. + """ + group = zarr.open_group(store=tmp_path / "n.zarr", mode="w", zarr_format=3) + array = stores.make_array( + group, + "v", + (2,), + "f8", + 3, + attributes={ + "statistics_approximate": {"min": 0.0, "max": 1.0}, + "valid_range": [0.0, 1.0], + "nested_list": [[1, 2], [3, 4]], + }, + ) + attributes = wrap(array).attributes + assert attributes["statistics_approximate"] == {"min": 0.0, "max": 1.0} + assert attributes["valid_range"] == [0.0, 1.0] + assert attributes["nested_list"] == [[1, 2], [3, 4]] +``` + +- [ ] **Step 11: Run the module** + +```bash +pytest lib/iris/tests/unit/fileformats/zarr/ -v +``` + +Expected: all pass. + +- [ ] **Step 12: Commit** + +Run §5.0, then: + +```bash +git add lib/iris/fileformats/zarr lib/iris/tests/unit/fileformats/zarr +git commit -m "Present a Zarr array through the CF dataset interface + +The dimension contract is the substance: a Zarr dimension may be unnamed, +and an unnamed one is anonymous rather than an error. _ARRAY_DIMENSIONS is +storage, so it is withheld from the CF attributes, and fill_value is +reported as the storage fact it is rather than as a missing-value flag." +``` + +--- +### Task 4: `ZarrDataset` — the group, its variables and its dimensions + +**Files:** +- Modify: `lib/iris/fileformats/zarr/_dataset.py` +- Create: `lib/iris/tests/unit/fileformats/zarr/dataset/test_ZarrDataset.py` + +**Interfaces:** +- Consumes: `ZarrDatasetVariable`, `_ZarrAttributes`, + `_FORMAT_BY_ZARR_VERSION` (Task 3). +- Produces: `ZarrDataset(location, mode="r", *, group=None)` with + `location`, `format`, `mode`, `closed`, `variables`, `dimensions`, + `encoding`, `attributes`, `close`, `sync`, and the Zarr-only + `subgroup_names`. The write members exist and raise. + +Spec §4.4's second rule is the substance here: **`CFDataset.dimensions` is the +names that are used consistently.** A name that appears at two different sizes +is not an error and is not silently merged — it is omitted from the mapping, +with an `IrisCfLoadWarning` naming the dimension and the sizes. The variables +that use it keep it in their own `dimensions`, where the `(name, size)` rule of +Task 8 stops it linking them. + +- [ ] **Step 1: Write the failing tests** + +Create `lib/iris/tests/unit/fileformats/zarr/dataset/test_ZarrDataset.py`: + +```python +# Copyright Iris contributors +# +# This file is part of Iris and is released under the BSD license. +# See LICENSE in the root of the repository for full licensing details. +"""Unit tests for :class:`iris.fileformats.zarr._dataset.ZarrDataset`.""" + +import numpy as np +import pytest +import zarr + +from iris.fileformats.zarr._dataset import ZarrDataset +from iris.warnings import IrisCfLoadWarning + +from .. import stores + + +@pytest.fixture(params=[2, 3], ids=["v2", "v3"]) +def zarr_format(request): + return request.param + + +def test_opens_and_reports_itself(tmp_path, zarr_format): + location = stores.make_store(tmp_path / "s.zarr", zarr_format=zarr_format) + dataset = ZarrDataset(location) + + assert dataset.location == location + assert dataset.format == f"ZARR_V{zarr_format}" + assert dataset.mode == "r" + assert dataset.closed is False + assert sorted(dataset.variables) == ["air_temperature", "grid"] + assert dict(dataset.attributes) == stores.GLOBALS + assert dataset.encoding["zarr_format"] == zarr_format + + +def test_dimensions_are_assembled_from_the_arrays(tmp_path, zarr_format): + location = stores.make_store(tmp_path / "s.zarr", zarr_format=zarr_format) + assert dict(ZarrDataset(location).dimensions) == stores.DIMENSIONS + + +def test_a_name_used_at_two_sizes_is_omitted_and_warns(tmp_path): + """Two arrays may share a dimension name at different lengths. + + The version 3 specification "does not place any restrictions on the use of + the same dimension name across multiple arrays", so this is conforming + Zarr and must not raise. Merging the two would produce a wrong cube. + """ + group = zarr.open_group(store=tmp_path / "c.zarr", mode="w", zarr_format=3) + stores.make_array(group, "short", (2,), "f8", 3, dimensions=("t",)) + stores.make_array(group, "long", (5,), "f8", 3, dimensions=("t",)) + + with pytest.warns(IrisCfLoadWarning, match="t"): + dimensions = dict(ZarrDataset(str(tmp_path / "c.zarr")).dimensions) + + assert "t" not in dimensions + + +def test_anonymous_dimensions_are_not_in_the_mapping(tmp_path): + group = zarr.open_group(store=tmp_path / "a.zarr", mode="w", zarr_format=3) + stores.make_array(group, "named", (2,), "f8", 3, dimensions=("t",)) + stores.make_array(group, "unnamed", (4,), "f8", 3) + + dataset = ZarrDataset(str(tmp_path / "a.zarr")) + assert dict(dataset.dimensions) == {"t": 2} + + +def test_close_is_idempotent(tmp_path): + location = stores.make_store(tmp_path / "s.zarr") + dataset = ZarrDataset(location) + dataset.close() + dataset.close() + assert dataset.closed is True + + +def test_context_manager_closes(tmp_path): + location = stores.make_store(tmp_path / "s.zarr") + dataset = ZarrDataset(location) + with dataset as entered: + assert entered is dataset + assert dataset.closed is True + + +def test_group_keyword_selects_a_subgroup(tmp_path): + root = zarr.open_group(store=tmp_path / "n.zarr", mode="w", zarr_format=3) + sub = root.create_group("measurements/r10m") + stores.make_array(sub, "b02", (2,), "u2", 3, dimensions=("p",)) + + dataset = ZarrDataset(str(tmp_path / "n.zarr"), group="measurements/r10m") + assert list(dataset.variables) == ["b02"] + assert dataset.location.endswith("measurements/r10m") + + +def test_subgroup_names_lists_only_the_groups(tmp_path): + root = zarr.open_group(store=tmp_path / "n.zarr", mode="w", zarr_format=3) + root.create_group("conditions") + root.create_group("measurements") + stores.make_array(root, "v", (2,), "f8", 3) + + dataset = ZarrDataset(str(tmp_path / "n.zarr")) + assert sorted(dataset.subgroup_names) == ["conditions", "measurements"] + + +def test_nested_groups_are_not_walked(tmp_path): + """Iris loads one group per call, like xarray. + + A Zarr store can legitimately hold unrelated datasets, and merging them + into one CubeList would be wrong. + """ + root = zarr.open_group(store=tmp_path / "n.zarr", mode="w", zarr_format=3) + stores.make_array(root, "top", (2,), "f8", 3) + stores.make_array(root.create_group("sub"), "inner", (2,), "f8", 3) + + assert list(ZarrDataset(str(tmp_path / "n.zarr")).variables) == ["top"] + + +def test_the_write_surface_refuses(tmp_path): + # Saving to Zarr is not implemented. The members exist because the + # interface declares them; refusing is the honest answer. + dataset = ZarrDataset(stores.make_store(tmp_path / "s.zarr")) + with pytest.raises(NotImplementedError, match="Saving"): + dataset.create_dimension("t", 3) + with pytest.raises(NotImplementedError, match="Saving"): + dataset.create_variable("v", np.dtype("f4"), ("t",)) + with pytest.raises(NotImplementedError, match="Saving"): + dataset.finalise() + + +def test_a_write_mode_refuses(tmp_path): + stores.make_store(tmp_path / "s.zarr") + with pytest.raises(NotImplementedError, match="Saving"): + ZarrDataset(str(tmp_path / "s.zarr"), mode="w") +``` + +- [ ] **Step 2: Run them to verify they fail** + +```bash +pytest lib/iris/tests/unit/fileformats/zarr/dataset/test_ZarrDataset.py -v +``` + +Expected: collection error, `ImportError: cannot import name 'ZarrDataset'`. + +- [ ] **Step 3: Write the dimension assembly** + +Append to `lib/iris/fileformats/zarr/_dataset.py`, after `_dimension_names`: + +```python +def _assemble_dimensions(variables) -> dict: + """Return the dimension sizes the store's arrays agree on. + + Parameters + ---------- + variables : mapping of str to :class:`ZarrDatasetVariable` + Every array in one group. + + Returns + ------- + dict + Name to size, for every name used at exactly one size. Anonymous + dimensions have no name and so are absent. + + Warns + ----- + IrisCfLoadWarning + For each name used at more than one size. Zarr places no restriction + on reusing a name across arrays, so this is conforming and must not + raise; merging the two would produce a wrong cube, so the name is + omitted instead and the arrays that use it keep it in their own + `dimensions`. + + """ + sizes: dict[str, int] = {} + conflicts: dict[str, set[int]] = {} + for variable in variables.values(): + for name, size in zip(variable.dimensions, variable.shape): + if name is None: + continue + if name not in sizes: + sizes[name] = size + elif sizes[name] != size: + conflicts.setdefault(name, {sizes[name]}).add(size) + + for name, found in conflicts.items(): + message = ( + "Ignoring a dimension name used at more than one size, " + f"{sorted(found)}: {name}" + ) + warnings.warn(message, category=iris.warnings.IrisCfLoadWarning) + del sizes[name] + + return sizes +``` + +- [ ] **Step 4: Write the class** + +Append to `_dataset.py`: + +```python +#: What `mode` a dataset may be opened in. Writing is not implemented. +_READ_MODES = ("r",) + +#: The one message every unimplemented write member raises. +_NO_SAVING = "Saving to Zarr is not supported." + + +class ZarrDataset(CFDataset): + """A Zarr store, or one group of one, presented through the CF interface.""" + + def __init__(self, location, mode: str = "r", *, group: str | None = None): + """Open a Zarr store. + + Parameters + ---------- + location : str or :class:`pathlib.Path` + The store's path or URL. A URL is handed to zarr-python + unaltered; Iris adds no store library and handles no credentials, + so an unreachable scheme fails with zarr-python's own message. + mode : str, default="r" + Only ``"r"`` is supported. + group : str, optional + A ``/``-separated path to a group within the store. The root group + by default. Iris reads one group per call and does not walk + nested groups. + + Raises + ------ + NotImplementedError + For any `mode` other than ``"r"``. + FileNotFoundError + If `location` does not exist. + zarr.errors.GroupNotFoundError + If `location` is not a Zarr store, or holds no such `group`. + + """ + # Set first, so that close() is safe if opening fails. + self._group = None + self._closed = False + self._variables: dict[str, ZarrDatasetVariable] | None = None + self._attributes: _ZarrAttributes | None = None + self._dimensions: dict[str, int] | None = None + + if mode not in _READ_MODES: + raise NotImplementedError(f"{_NO_SAVING} Cannot open with mode={mode!r}.") + + self._mode = mode + self._group_path = group + target = str(location) + self._location = target if group is None else f"{target.rstrip('/')}/{group}" + + self._group = zarr.open_group(store=target, mode=mode, path=group) + self._format = _FORMAT_BY_ZARR_VERSION[self._group.metadata.zarr_format] + + @property + def location(self) -> str: + """The store's path or URL, with the group path when there is one.""" + return self._location + + @property + def format(self) -> str: + """``ZARR_V2`` or ``ZARR_V3``.""" + return self._format + + @property + def mode(self) -> str: + """The mode the store was opened in.""" + return self._mode + + @property + def closed(self) -> bool: + """Whether :meth:`close` has been called. + + A Zarr store is a namespace rather than an open file handle, so there + is nothing to release and nothing else to ask. + + """ + return self._closed + + @property + def variables(self) -> Mapping: + """The group's arrays, by name. Arrays in subgroups are not included.""" + if self._variables is None: + self._variables = { + name: ZarrDatasetVariable(array, self._location, self._format) + for name, array in self._group.arrays() + } + return self._variables + + @property + def dimensions(self) -> Mapping: + """The dimension sizes this group's arrays agree on.""" + if self._dimensions is None: + self._dimensions = _assemble_dimensions(self.variables) + return self._dimensions + + @property + def encoding(self) -> Mapping: + """The store-wide Zarr settings.""" + consolidated = self._group.metadata.consolidated_metadata + return MappingProxyType( + { + "zarr_format": self._group.metadata.zarr_format, + "consolidated": consolidated is not None, + } + ) + + @property + def attributes(self) -> MutableMapping: + """The group's attributes.""" + if self._attributes is None: + self._attributes = _ZarrAttributes(self._group) + return self._attributes + + def create_dimension(self, name: str, size: int | None) -> None: + """Raise: saving to Zarr is not supported.""" + raise NotImplementedError(_NO_SAVING) + + def create_variable( + self, name: str, dtype, dimensions=(), *, fill_value=None, **encoding + ): + """Raise: saving to Zarr is not supported.""" + raise NotImplementedError(_NO_SAVING) + + def sync(self) -> None: + """Do nothing: a Zarr write is not buffered.""" + + def finalise(self) -> None: + """Raise: saving to Zarr is not supported. + + A Zarr store's completion step is consolidating its metadata, which + only a writer performs. + + """ + raise NotImplementedError(_NO_SAVING) + + def close(self) -> None: + """Record that this dataset is finished with. + + A store is a namespace, not an open handle, so nothing is released. + The flag exists because callers - :class:`iris.fileformats.cf.CFReader` + among them - must be able to ask. + + """ + self._closed = True + + def __repr__(self) -> str: + """Return a string representation.""" + return f"{self.__class__.__name__}({self.location!r}, mode={self.mode!r})" + + # Zarr-only members below. + + @property + def subgroup_names(self) -> tuple: + """The names of this group's subgroups. + + What the loader names when a group holds no data variables but the + store plainly holds data elsewhere. + + """ + return tuple(self._group.group_keys()) +``` + +- [ ] **Step 5: Run the tests** + +```bash +pytest lib/iris/tests/unit/fileformats/zarr/dataset/test_ZarrDataset.py -v +``` + +Expected: all pass. + +- [ ] **Step 6: Add the remote and error-path tests** + +Spec §6: "Remote stores are covered by an in-memory store and a local +directory store only. No test touches the network." Append to +`test_ZarrDataset.py`: + +```python +def test_an_in_memory_store_is_just_another_location(tmp_path): + """A store need not be a directory. zarr-python decides what it is.""" + memory = zarr.storage.MemoryStore() + group = zarr.open_group(store=memory, mode="w", zarr_format=3) + stores.make_array(group, "v", (2,), "f8", 3, dimensions=("t",)) + + dataset = ZarrDataset(memory) + assert list(dataset.variables) == ["v"] + + +def test_a_missing_store_raises_zarrs_own_error(tmp_path): + with pytest.raises(FileNotFoundError): + ZarrDataset(str(tmp_path / "nothing.zarr")) + + +def test_a_directory_that_is_not_a_store_raises(tmp_path): + (tmp_path / "plain").mkdir() + with pytest.raises(zarr.errors.GroupNotFoundError): + ZarrDataset(str(tmp_path / "plain")) + + +def test_a_missing_group_raises(tmp_path): + location = stores.make_store(tmp_path / "s.zarr") + with pytest.raises(zarr.errors.GroupNotFoundError): + ZarrDataset(location, group="nope") +``` + +`ZarrDataset(memory)` passes a store object where the signature says "str or +Path". That is deliberate and documented in the docstring's `location` +paragraph as "handed to zarr-python": `str(store)` of a `MemoryStore` is not a +path, so `location` becomes zarr-python's own rendering of it, which is the +best name available for a store with no address. Add that sentence to the +`location` docstring when this test is written. + +- [ ] **Step 7: Run and commit** + +```bash +pytest lib/iris/tests/unit/fileformats/zarr/ -v +``` + +Expected: all pass. Then run §5.0 and: + +```bash +git add lib/iris/fileformats/zarr lib/iris/tests/unit/fileformats/zarr +git commit -m "Present a Zarr group through the CF dataset interface + +Dimensions are assembled from the arrays, and a name used at two sizes is +omitted with a warning rather than merged: Zarr places no restriction on +reusing a name across arrays, so a store meaning two things by one name is +conforming, and merging them would produce a wrong cube." +``` + +--- + +### Task 5: the shared read contract, run against Zarr + +**Files:** +- Create: `lib/iris/tests/unit/fileformats/zarr/dataset/test_ZarrDataset_contract.py` + +**Interfaces:** +- Consumes: `CFDatasetReadContract` (Task 2); `ZarrDataset` (Task 4); + `stores.make_store` (Task 1). +- Produces: nothing other than coverage. This task exists so that a + netCDF/Zarr divergence fails loudly, which is spec §6's whole argument for a + shared body. + +The contract's canonical store is the one `stores.make_store` writes, which is +why Task 1 wrote it to match `contract.populate()` exactly — same dimensions, +same data, same attributes, same scalar `grid` variable. + +- [ ] **Step 1: Write the test module** + +Create +`lib/iris/tests/unit/fileformats/zarr/dataset/test_ZarrDataset_contract.py`: + +```python +# Copyright Iris contributors +# +# This file is part of Iris and is released under the BSD license. +# See LICENSE in the root of the repository for full licensing details. +"""The shared CFDataset read contract, asked of the Zarr implementation. + +Only the read half: the Zarr write path does not exist, and `ZarrDataset` +refuses every member of the write surface. The write half of the contract +arrives with the saver. +""" + +import pytest + +from iris.fileformats.zarr._dataset import ZarrDataset +from iris.tests.unit.fileformats.cf.dataset import contract + +from .. import stores + + +class TestZarrDatasetVersion3(contract.CFDatasetReadContract): + @pytest.fixture + def readable(self, tmp_path): + return ZarrDataset(stores.make_store(tmp_path / "contract.zarr")) + + +class TestZarrDatasetVersion2(contract.CFDatasetReadContract): + @pytest.fixture + def readable(self, tmp_path): + location = stores.make_store(tmp_path / "contract.zarr", zarr_format=2) + return ZarrDataset(location) +``` + +- [ ] **Step 2: Run it, and expect disagreements** + +```bash +pytest lib/iris/tests/unit/fileformats/zarr/dataset/test_ZarrDataset_contract.py -v +``` + +This is the step that earns the task. Expect failures, and resolve each one +by deciding **which side is wrong**, not by loosening the contract: + +- `test_variable_fill_value` asserts `-1.0`, which `stores.make_store` sets as + the array's `fill_value`. Zarr answers from storage and netCDF from the + `_FillValue` attribute (§4, "`fill_value` means different things"), so both + give `-1.0` here and the test should pass. If it does not, the Zarr store + builder is not setting what it claims to. +- `test_variable_chunking_is_none_or_a_shape` already admits `np.integer`, + which was written for Zarr. +- `test_mode_is_a_read_mode` asserts `mode in ("r", "r+", "a")`; `ZarrDataset` + answers `"r"`. +- `test_scalar_variable` expects `len(scalar)` to raise `TypeError` matching + "unsized". `CFDatasetVariable.__len__` supplies that for any implementation + with an empty `shape`, and `ZarrDatasetVariable` does not override it. + +Record each resolution in the commit message. A contract test that needs a +Zarr-shaped exception is a design signal, not a test problem. + +- [ ] **Step 3: Confirm the netCDF side is untouched** + +```bash +pytest lib/iris/tests/unit/fileformats/netcdf/dataset/ -q +``` + +Expected: the same count as at the end of Task 2. + +- [ ] **Step 4: Add the no-isinstance check** + +Spec §6: "A separate check greps the `cf` package for `isinstance` against an +implementation class and asserts there are none — the rule in §4.2 is only +worth writing down if something notices when it is broken." + +Create `lib/iris/tests/unit/fileformats/cf/test_no_implementation_coupling.py`: + +```python +# Copyright Iris contributors +# +# This file is part of Iris and is released under the BSD license. +# See LICENSE in the root of the repository for full licensing details. +"""The CF layer must not branch on which storage implementation it holds. + +`isinstance(thing, CFDataset)` tests the *interface*, which is how generic +code is supposed to accept any backend, and `CFReader` does exactly that. +What is banned is testing an *implementation*: that is what makes a third +backend a rewrite rather than a new module. Constructing one is a third +thing again, and legitimate - `CFReader` opens a `NetCDFDataset` for a bare +path, which is a default, not a branch on something it was handed. +""" + +from pathlib import Path + +import iris.fileformats.cf + +#: Implementation class names that may not appear in a type test. +IMPLEMENTATION_NAMES = ( + "NetCDFDataset", + "NetCDFDatasetVariable", + "ZarrDataset", + "ZarrDatasetVariable", +) + +#: The spellings of a type test. `__class__` catches the hand-rolled form. +TYPE_TESTS = ("isinstance", "issubclass", "__class__", "type(") + + +def cf_package_sources(): + """Yield `(path, text)` for every module in `iris.fileformats.cf`.""" + package = Path(iris.fileformats.cf.__file__).parent + for path in sorted(package.glob("*.py")): + yield path, path.read_text(encoding="utf-8") + + +def test_no_module_type_tests_an_implementation_class(): + offenders = [] + for path, text in cf_package_sources(): + for line_number, line in enumerate(text.splitlines(), start=1): + if line.lstrip().startswith("#"): + continue + names_one = any(name in line for name in IMPLEMENTATION_NAMES) + tests_type = any(spelling in line for spelling in TYPE_TESTS) + if names_one and tests_type: + offenders.append(f"{path.name}:{line_number}: {line.strip()}") + assert offenders == [] +``` + +This passes as written, and the way it fails is by someone adding the branch +it forbids. Prove it can fail before trusting it: temporarily add +`if isinstance(self._dataset, NetCDFDataset): pass` to `cf/_reader.py`, re-run, +see one offender, and remove it. + +- [ ] **Step 5: Commit** + +Run §5.0, then: + +```bash +git add lib/iris/tests/unit/fileformats +git commit -m "Ask the shared CFDataset read contract of the Zarr store + +Both Zarr versions answer the same questions the netCDF implementation +does, so a divergence between the two fails here rather than somewhere in +the loader. Adds the check that no module in the cf package names an +implementation class." +``` + +--- +### Task 6: `_decode.py` — the CF rules netCDF4 applies for free + +**Files:** +- Create: `lib/iris/fileformats/zarr/_decode.py` +- Create: `lib/iris/tests/unit/fileformats/zarr/test__decode.py` + +**Interfaces:** +- Consumes: nothing. Every function here is pure and takes plain values, which + is the point: spec §6 wants decoding "unit-testable without a store at all". +- Produces: `decode_fill_value(value, dtype, variable_name)`, + `decode_parameters(attributes, *, dtype, storage_fill_value, zarr_format, + variable_name)`, `decoding_required(parameters)`, + `mask_and_scale(block, *, sentinels, valid_min, valid_max, scale_factor, + add_offset, dtype)`, and the three rules it composes. + +Two rules from spec §4.4 are the substance: + +- **The CF attribute governs masking and the storage field does not** — except + on version 2, where `Array.fill_value` is the mask source when no CF + attribute is present, "because version 2 stores predate the convention and + this is what their writers meant". This matches released xarray's + `use_zarr_fill_value_as_mask`. +- **A floating `_FillValue` is base64 on version 3, and that is a convention, + not a malformation.** `base64(struct.pack(" Any: + """Return a CF missing-value sentinel as a value of `dtype`. + + Parameters + ---------- + value : Any + The attribute's value, as the store holds it: a JSON number, or - for + a floating variable - the base64 of a little-endian ``float64``. + dtype : numpy.dtype + The variable's own dtype. The result is narrowed to it, because a + sentinel of a different width never equals anything in the array. + variable_name : str + Named in the warning, so the user can find the variable. + + Returns + ------- + numpy.generic or None + ``None`` when `value` is neither form, which is a producer error. + + Warns + ----- + IrisCfLoadWarning + When `value` cannot be read as a sentinel of `dtype`. + + """ + if isinstance(value, (bool, np.bool_)): + # A JSON true/false is a number to Python and nothing to CF. + decoded = None + elif isinstance(value, (int, float, np.number)): + decoded = value + elif isinstance(value, str) and dtype.kind == "f": + decoded = _decode_packed_double(value) + else: + decoded = None + + if decoded is None: + message = ( + "Ignoring a missing-value attribute that is neither a number nor " + f"the base64 of a float64, on variable {variable_name}: {value!r}" + ) + warnings.warn(message, category=iris.warnings.IrisCfLoadWarning) + return None + + return dtype.type(decoded) + + +def _decode_packed_double(value: str) -> float | None: + """Return the float in a base64 little-endian float64, or None.""" + try: + raw = base64.b64decode(value, validate=True) + except (binascii.Error, ValueError): + return None + if len(raw) != _PACKED_DOUBLE_BYTES: + return None + return struct.unpack(_PACKED_DOUBLE, raw)[0] +``` + +- [ ] **Step 4: Run the first group** + +```bash +pytest lib/iris/tests/unit/fileformats/zarr/test__decode.py -v +``` + +Expected: the seven `decode_fill_value` tests pass. + +- [ ] **Step 5: Write the failing tests for the masking precedence** + +Append to `test__decode.py`: + +```python +def parameters(attributes, *, dtype="i2", storage_fill_value=0, zarr_format=3): + return _decode.decode_parameters( + attributes, + dtype=np.dtype(dtype), + storage_fill_value=storage_fill_value, + zarr_format=zarr_format, + variable_name="v", + ) + + +def test_version_3_ignores_the_storage_fill_value(): + """An int16 [0, 1, 2] with no CF attributes loads with nothing masked. + + zarr-python defaults `fill_value` to zero, so reading it as a sentinel + masks valid data in the ordinary case, not the exotic one. + """ + assert parameters({}, storage_fill_value=0)["sentinels"] == () + + +def test_version_2_uses_the_storage_fill_value_when_there_is_no_attribute(): + result = parameters({}, storage_fill_value=0, zarr_format=2) + assert result["sentinels"] == (0,) + + +def test_a_cf_attribute_beats_the_storage_fill_value_on_version_2(): + result = parameters( + {"_FillValue": -999}, storage_fill_value=0, zarr_format=2 + ) + assert result["sentinels"] == (np.int16(-999),) + + +def test_both_cf_missing_value_attributes_are_sentinels(): + result = parameters({"_FillValue": -999, "missing_value": -888}) + assert set(result["sentinels"]) == {np.int16(-999), np.int16(-888)} + + +def test_valid_range_sets_both_bounds(): + result = parameters({"valid_range": [0, 10]}) + assert result["valid_min"] == 0 + assert result["valid_max"] == 10 + + +def test_valid_min_and_valid_max_are_read_separately(): + result = parameters({"valid_min": 1, "valid_max": 9}) + assert (result["valid_min"], result["valid_max"]) == (1, 9) + + +def test_packing_parameters_and_the_resulting_dtype(): + result = parameters({"scale_factor": 0.0001, "add_offset": 1.0}, dtype="u2") + assert result["scale_factor"] == 0.0001 + assert result["add_offset"] == 1.0 + assert result["dtype"] == np.dtype("f8") + + +def test_decoding_required_is_false_for_a_plain_array(): + assert _decode.decoding_required(parameters({})) is False + + +def test_decoding_required_is_true_for_each_rule(): + assert _decode.decoding_required(parameters({"_FillValue": -999})) is True + assert _decode.decoding_required(parameters({"valid_min": 1})) is True + assert _decode.decoding_required(parameters({"scale_factor": 2})) is True +``` + +- [ ] **Step 6: Write `decode_parameters` and `decoding_required`** + +Append to `_decode.py`: + +```python +def decode_parameters( + attributes, + *, + dtype: np.dtype, + storage_fill_value: Any, + zarr_format: int, + variable_name: str, +) -> dict: + """Return the CF decode parameters for one variable. + + Parameters + ---------- + attributes : mapping + The variable's CF attributes. + dtype : numpy.dtype + The variable's stored dtype, before any unpacking. + storage_fill_value : Any + ``zarr.Array.fill_value`` - what unwritten storage reads back as. A + mask source only on version 2, and only when the variable declares no + CF missing-value attribute. + zarr_format : int + 2 or 3. The one place the version changes what is decoded. + variable_name : str + Named in any warning. + + Returns + ------- + dict + The keyword arguments :func:`mask_and_scale` takes, always with the + same keys. :func:`decoding_required` says whether they do anything. + + """ + sentinels = [] + for name in ("_FillValue", "missing_value"): + if name in attributes: + decoded = decode_fill_value(attributes[name], dtype, variable_name) + if decoded is not None: + sentinels.append(decoded) + + if ( + not sentinels + and zarr_format < _FILL_VALUE_IS_A_MASK_BEFORE + and storage_fill_value is not None + ): + sentinels.append(storage_fill_value) + + valid_min = attributes.get("valid_min") + valid_max = attributes.get("valid_max") + valid_range = attributes.get("valid_range") + if valid_range is not None and len(valid_range) == 2: + valid_min, valid_max = valid_range + + scale_factor = attributes.get("scale_factor") + add_offset = attributes.get("add_offset") + + return { + "sentinels": tuple(sentinels), + "valid_min": valid_min, + "valid_max": valid_max, + "scale_factor": scale_factor, + "add_offset": add_offset, + "dtype": _unpacked_dtype(dtype, scale_factor, add_offset), + } + + +def _unpacked_dtype(dtype, scale_factor, add_offset) -> np.dtype: + """Return the dtype the unpacked values will have. + + The same calculation as + :func:`iris.fileformats.cf.loader._get_actual_dtype`, done here on plain + values so that the dtype the Dask graph declares and the dtype the decode + layer produces are one statement rather than two. + """ + probe = np.zeros(1, dtype=dtype) + if scale_factor is not None: + probe = scale_factor * probe + if add_offset is not None: + probe = add_offset + probe + return probe.dtype + + +def decoding_required(parameters: dict) -> bool: + """Whether these parameters would change anything. + + A variable with nothing to decode gets no ``map_blocks`` layer and an + unmasked Dask meta. Adding a masked meta without a layer beneath it would + be a meta that lies about its own graph. + """ + if parameters["sentinels"]: + return True + return any( + parameters[name] is not None + for name in ("valid_min", "valid_max", "scale_factor", "add_offset") + ) +``` + +- [ ] **Step 7: Run, then write the failing tests for `mask_and_scale`** + +```bash +pytest lib/iris/tests/unit/fileformats/zarr/test__decode.py -v +``` + +Expected: all pass so far. Then append: + +```python +def test_mask_by_sentinels(): + block = np.array([0, 1, -999], dtype="i2") + result = _decode.mask_by_sentinels(block, (np.int16(-999),)) + assert result.tolist() == [False, False, True] + + +def test_mask_by_a_nan_sentinel(): + """NaN never equals itself, so a NaN sentinel needs `isnan`, not `==`. + + The base64 `_FillValue` nearly every xarray-written store carries is NaN, + so this is the common case rather than the exotic one. + """ + block = np.array([1.0, np.nan], dtype="f4") + result = _decode.mask_by_sentinels(block, (np.float32("nan"),)) + assert result.tolist() == [False, True] + + +def test_mask_by_valid_range(): + block = np.array([-1, 5, 11], dtype="i2") + result = _decode.mask_by_valid_range(block, 0, 10) + assert result.tolist() == [True, False, True] + + +def test_unpack(): + block = np.array([1, 2], dtype="u2") + result = _decode.unpack(block, 0.5, 1.0, np.dtype("f8")) + np.testing.assert_allclose(result, [1.5, 2.0]) + assert result.dtype == np.dtype("f8") + + +def test_mask_and_scale_masks_the_packed_values(): + """Masking is applied before unpacking, as netCDF4 does it. + + CF section 2.5.1: `_FillValue` is "of the same type as its variable", so + it is a packed value and comparing it with an unpacked one matches + nothing. + """ + block = np.array([-999, 1, 2], dtype="i2") + result = _decode.mask_and_scale( + block, + sentinels=(np.int16(-999),), + valid_min=None, + valid_max=None, + scale_factor=0.5, + add_offset=0.0, + dtype=np.dtype("f8"), + ) + assert result.mask.tolist() == [True, False, False] + np.testing.assert_allclose(result.compressed(), [0.5, 1.0]) + + +def test_mask_and_scale_always_returns_a_masked_array(): + # The Dask meta says masked, so every block must be, mask or no mask. + block = np.array([1, 2], dtype="i2") + result = _decode.mask_and_scale( + block, + sentinels=(np.int16(-999),), + valid_min=None, + valid_max=None, + scale_factor=None, + add_offset=None, + dtype=np.dtype("i2"), + ) + assert isinstance(result, np.ma.MaskedArray) + assert not result.mask.any() +``` + +- [ ] **Step 8: Write the masking and unpacking functions** + +Append to `_decode.py`: + +```python +def mask_by_sentinels(block: np.ndarray, sentinels: tuple) -> np.ndarray: + """Return a boolean mask of the elements equal to any sentinel. + + A NaN sentinel is matched with :func:`numpy.isnan`, because NaN does not + equal itself and NaN is what nearly every version 3 store's base64 + ``_FillValue`` decodes to. + """ + mask = np.zeros(block.shape, dtype=bool) + for sentinel in sentinels: + if isinstance(sentinel, (float, np.floating)) and np.isnan(sentinel): + mask |= np.isnan(block) + else: + mask |= block == sentinel + return mask + + +def mask_by_valid_range(block: np.ndarray, valid_min, valid_max) -> np.ndarray: + """Return a boolean mask of the elements outside the valid range.""" + mask = np.zeros(block.shape, dtype=bool) + if valid_min is not None: + mask |= block < valid_min + if valid_max is not None: + mask |= block > valid_max + return mask + + +def unpack(block: np.ndarray, scale_factor, add_offset, dtype: np.dtype) -> np.ndarray: + """Return `block` unpacked by CF's scale and offset, as `dtype`.""" + result = block + if scale_factor is not None: + result = scale_factor * result + if add_offset is not None: + result = add_offset + result + return result.astype(dtype, copy=False) + + +def mask_and_scale( + block: np.ndarray, + *, + sentinels: tuple, + valid_min, + valid_max, + scale_factor, + add_offset, + dtype: np.dtype, +) -> np.ma.MaskedArray: + """Apply the CF decode rules to one block, in CF's order. + + Masking first, on the **packed** values, because CF declares a sentinel to + be "of the same type as its variable"; unpacking second. The result is + always a masked array, so that every block matches the Dask meta the + loader declares for the layer. + + Parameters + ---------- + block : numpy.ndarray + One Dask block of the raw stored values. + sentinels : tuple + Values meaning "missing", already narrowed to the stored dtype. + valid_min, valid_max : optional + CF's valid range. Values outside it are masked. + scale_factor, add_offset : optional + CF's packing. ``value * scale_factor + add_offset``. + dtype : numpy.dtype + The dtype of the result, which the Dask graph has already declared. + + Returns + ------- + numpy.ma.MaskedArray + + """ + mask = mask_by_sentinels(block, sentinels) + mask |= mask_by_valid_range(block, valid_min, valid_max) + return np.ma.masked_array( + unpack(block, scale_factor, add_offset, dtype), mask=mask + ) +``` + +- [ ] **Step 9: Run the module** + +```bash +pytest lib/iris/tests/unit/fileformats/zarr/test__decode.py -v +``` + +Expected: all pass. + +- [ ] **Step 10: Commit** + +Run §5.0, then: + +```bash +git add lib/iris/fileformats/zarr/_decode.py lib/iris/tests/unit/fileformats/zarr/test__decode.py +git commit -m "Implement the CF decoding Zarr does not do for itself + +Masking, valid-range masking and scale/offset unpacking, as pure functions +of values. The two rules worth reading twice: a floating _FillValue is +base64 on version 3, which is xarray's convention rather than a defect; +and Array.fill_value is a mask source on version 2 only, and only when the +variable declares no CF attribute of its own." +``` + +--- + +### Task 7: `read_data` — laziness, chunk alignment and the cache key + +> The chunk-alignment half of this task was reversed on review. See "Task 7, +> reversed after review" under *What changed during implementation*. The task +> text below is left as it was executed. + +**Files:** +- Modify: `lib/iris/fileformats/zarr/_dataset.py` +- Create: `lib/iris/tests/unit/fileformats/zarr/dataset/test__align_chunks.py` +- Create: `lib/iris/tests/unit/fileformats/zarr/dataset/test_read_data.py` + +**Interfaces:** +- Consumes: `_decode.decode_parameters`, `decoding_required`, `mask_and_scale` + (Task 6); `ZarrDatasetVariable` (Task 3); + `iris.fileformats.cf.loader._LAZYVAR_MIN_BYTES` and `_get_actual_dtype`, and + `iris._lazy_data.as_lazy_data(..., cache_key=...)` (**PR 3**). +- Produces: `ZarrDatasetVariable.read_data(chunking_policy)`, + `ZarrDatasetVariable.cache_key`, `ZarrDatasetVariable.zarr_format`, and the + module function `_align_chunks(chunks, store_chunks, shape, dtype, + dims_fixed, name)`. + +Three spec rules meet here. + +**No proxy.** `zarr.Array` is picklable, a chunk is an independent object in a +key-value store, and a store is a namespace rather than an open handle, so +every reason `NetCDFDataProxy` exists is absent. The array goes to +`as_lazy_data` directly. + +**Alignment aggregates upward only.** `_optimum_chunksize` expands a small +store chunk to a whole multiple, which is right; but when the store chunk is +*larger* than dask's target it shrinks below it, and subdividing a Zarr chunk +buys nothing — a read of any part of one fetches and decompresses the whole +object (F7 has the two reproductions). So the computed chunking is rounded +**up** to a whole multiple of the store chunk, never below, and the variable +warns when the store's own chunking exceeds the dask target. + +**The key carries the metadata, not just the address.** `CACHE` is a +process-wide `LRUCache(100)`, so an address goes stale the moment a store is +rewritten in place. + +- [ ] **Step 1: Write the failing tests for `_align_chunks`** + +Create `lib/iris/tests/unit/fileformats/zarr/dataset/test__align_chunks.py`: + +```python +# Copyright Iris contributors +# +# This file is part of Iris and is released under the BSD license. +# See LICENSE in the root of the repository for full licensing details. +"""Unit tests for the Zarr read-side chunk alignment. + +No store is built here: the guard is a function of chunk shapes, which is +most of the point of having it as one. +""" + +import numpy as np +import pytest + +from iris.fileformats.zarr._dataset import _align_chunks +from iris.warnings import IrisLoadWarning + +F8 = np.dtype("f8") + + +def test_a_whole_multiple_is_left_alone(): + assert _align_chunks((4096, 4096), (512, 512), (4096, 4096), F8, None, "v") == ( + 4096, + 4096, + ) + + +def test_a_sub_multiple_is_rounded_up(): + """A dask chunk that subdivides a store chunk costs N re-reads. + + A read of any part of a Zarr chunk fetches and decompresses the whole + object, so N dask blocks over one store chunk is N times the bytes and N + times the CPU - and, over a remote store, N HTTP GETs for one object. + """ + with pytest.warns(IrisLoadWarning, match="v"): + result = _align_chunks((2000, 8000), (8000, 8000), (16000, 16000), F8, None, "v") + assert result == (8000, 8000) + + +def test_rounding_up_is_capped_at_the_shape(): + assert _align_chunks((2000, 6000), (6000, 6000), (6000, 6000), F8, None, "v") == ( + 6000, + 6000, + ) + + +def test_an_unchunked_store_is_left_alone(): + assert _align_chunks((10,), None, (100,), F8, None, "v") == (10,) + + +def test_an_explicit_chunk_control_setting_wins_but_warns(): + """A deliberate instruction is honoured, on the same terms. + + CHUNK_CONTROL marks the dimensions it set as fixed. Overriding them would + make the keyword a suggestion. + """ + with pytest.warns(IrisLoadWarning, match="v"): + result = _align_chunks( + (100, 8000), (8000, 8000), (16000, 16000), F8, (True, False), "v" + ) + assert result[0] == 100 + + +def test_a_small_store_chunk_does_not_warn(): + # Dask's target is the binding constraint here, which is the ordinary + # case and nothing for the user to act on. + with warnings.catch_warnings(): + warnings.simplefilter("error") + result = _align_chunks((1024, 1024), (512, 512), (4096, 4096), F8, None, "v") + assert result == (1024, 1024) + + +@pytest.mark.parametrize( + "chunks, store_chunks, shape", + [ + ((2000, 8000), (8000, 8000), (16000, 16000)), + ((1024, 1024), (512, 512), (4096, 4096)), + ((3000, 10000), (1000, 1000), (10000, 10000)), + ], +) +def test_the_result_tiles_the_store_grid_exactly(chunks, store_chunks, shape): + with warnings.catch_warnings(): + warnings.simplefilter("ignore", IrisLoadWarning) + result = _align_chunks(chunks, store_chunks, shape, F8, None, "v") + for size, store_size, extent in zip(result, store_chunks, shape): + assert size % store_size == 0 or size == extent +``` + +`import warnings` goes at the top of the module with the others. + +- [ ] **Step 2: Run them to verify they fail** + +```bash +pytest lib/iris/tests/unit/fileformats/zarr/dataset/test__align_chunks.py -v +``` + +Expected: `ImportError: cannot import name '_align_chunks'`. + +- [ ] **Step 3: Write `_align_chunks`** + +Append to `lib/iris/fileformats/zarr/_dataset.py`, after `_assemble_dimensions`: + +```python +def _fixed_flags(dims_fixed, ndim: int) -> tuple: + """Return one "do not change this" flag per dimension. + + `iris.fileformats.cf.loader._chunks_from_chunk_control` answers ``(None,)`` + when nothing is fixed, whatever the rank, so a length that does not match + means nothing is fixed. + """ + if dims_fixed is None or len(dims_fixed) != ndim: + return (False,) * ndim + return tuple(bool(flag) for flag in dims_fixed) + + +def _align_chunks( + chunks: tuple, + store_chunks: tuple | None, + shape: tuple, + dtype: np.dtype, + dims_fixed, + name: str, +) -> tuple: + """Round `chunks` up to whole multiples of the store's own chunk shape. + + A Zarr chunk is the atomic unit of storage: reading any part of one + fetches and decompresses the whole object. So a Dask chunking that + subdivides the store's costs a re-read per subdivision - and over a remote + store, one HTTP GET per subdivision for the same object. Aggregating + upward costs nothing, because peak memory per task is a whole decompressed + store chunk either way. + + Parameters + ---------- + chunks : tuple of int + The chunking computed for this variable, already through + :func:`iris._lazy_data._optimum_chunksize`. + store_chunks : tuple of int, optional + The store's own chunk shape. ``None`` for an unchunked store, which + constrains nothing. + shape : tuple of int + The variable's full shape. A result is never larger than this. + dtype : numpy.dtype + The stored dtype, for deciding whether one store chunk already exceeds + Dask's target chunk size. + dims_fixed : sequence of bool, optional + Dimensions an explicit ``CHUNK_CONTROL`` setting pinned. These are + left exactly as given: the keyword is an instruction, not a + suggestion. + name : str + The variable's name, for the warning. + + Returns + ------- + tuple of int + + Warns + ----- + IrisLoadWarning + When the result cannot tile the store grid without subdividing it - + either because one store chunk already exceeds Dask's target size, so + the store's layout is the binding constraint, or because an explicit + setting subdivides it. + + """ + if store_chunks is None: + return tuple(chunks) + + fixed = _fixed_flags(dims_fixed, len(shape)) + aligned = [] + subdivided = False + for size, store_size, extent, is_fixed in zip(chunks, store_chunks, shape, fixed): + if is_fixed: + aligned.append(size) + subdivided |= size < store_size + continue + whole = -(-size // store_size) * store_size + aligned.append(min(whole, extent)) + subdivided |= whole > extent and extent % store_size != 0 + + if _store_chunk_exceeds_dask_target(store_chunks, dtype): + subdivided = True + + if subdivided: + message = ( + "A Zarr chunk is read whole, so this store's chunking is the " + "binding constraint on memory per task rather than Dask's target " + f"chunk size: {name}" + ) + warnings.warn(message, category=iris.warnings.IrisLoadWarning) + + return tuple(aligned) + + +def _store_chunk_exceeds_dask_target(store_chunks: tuple, dtype: np.dtype) -> bool: + """Whether one whole store chunk is larger than Dask's target chunk size.""" + target = dask.utils.parse_bytes(dask.config.get("array.chunk-size")) + return int(np.prod(store_chunks)) * dtype.itemsize > target +``` + +Add `import dask`, `import dask.config` and `import dask.utils` to the module +imports. + +- [ ] **Step 4: Run the alignment tests** + +```bash +pytest lib/iris/tests/unit/fileformats/zarr/dataset/test__align_chunks.py -v +``` + +Expected: all pass. The two regression cases are F7's: `(8000, 8000)` over +`(16000, 16000)` and `(6000, 6000)` over `(6000, 6000)`, both of which +`_optimum_chunksize` subdivides on its own. + +- [ ] **Step 5: Write the failing tests for `read_data`** + +Create `lib/iris/tests/unit/fileformats/zarr/dataset/test_read_data.py`: + +```python +# Copyright Iris contributors +# +# This file is part of Iris and is released under the BSD license. +# See LICENSE in the root of the repository for full licensing details. +"""Unit tests for :meth:`ZarrDatasetVariable.read_data`.""" + +import dask.array as da +import numpy as np +import pytest +import zarr + +from iris.fileformats.cf import loader as cf_loader +from iris.fileformats.zarr._dataset import ZarrDataset + +from .. import stores + + +def variable(location, name="air_temperature"): + return ZarrDataset(location).variables[name] + + +def policy(): + """The chunking policy a loader outside any CHUNK_CONTROL supplies.""" + return None, None + + +def test_a_small_variable_is_read_whole(tmp_path): + location = stores.make_store(tmp_path / "s.zarr") + result = variable(location).read_data(policy) + assert not isinstance(result, da.Array) + np.testing.assert_array_equal(np.asarray(result), stores.DATA) + + +def test_a_small_variable_never_consults_the_policy(tmp_path): + """Under CHUNK_CONTROL.from_file() the policy raises for an unchunked store. + + A variable small enough to read whole must reach its data without asking, + or `from_file` starts rejecting small cubes that load today. + """ + + def raising_policy(): + raise AssertionError("chunking policy consulted for a small variable") + + location = stores.make_store(tmp_path / "s.zarr") + variable(location).read_data(raising_policy) + + +def test_a_large_variable_is_lazy(tmp_path, monkeypatch): + monkeypatch.setattr(cf_loader, "_LAZYVAR_MIN_BYTES", 1) + location = stores.make_store(tmp_path / "s.zarr") + result = variable(location).read_data(policy) + assert isinstance(result, da.Array) + np.testing.assert_array_equal(result.compute(), stores.DATA) + + +def test_a_plain_array_gets_an_unmasked_meta(tmp_path, monkeypatch): + # No decode layer, so no masked meta: a masked meta with nothing beneath + # it is a meta that lies about its own graph. + monkeypatch.setattr(cf_loader, "_LAZYVAR_MIN_BYTES", 1) + location = stores.make_store(tmp_path / "s.zarr") + result = variable(location).read_data(policy) + assert not isinstance(result._meta, np.ma.MaskedArray) + + +def test_a_decoded_array_gets_a_masked_meta(tmp_path, monkeypatch): + monkeypatch.setattr(cf_loader, "_LAZYVAR_MIN_BYTES", 1) + group = zarr.open_group(store=tmp_path / "m.zarr", mode="w", zarr_format=3) + array = stores.make_array( + group, "v", (4,), "i2", 3, dimensions=("t",), + attributes={"_FillValue": -999}, + ) + array[:] = [-999, 1, 2, 3] + + result = ZarrDataset(str(tmp_path / "m.zarr")).variables["v"].read_data(policy) + assert isinstance(result._meta, np.ma.MaskedArray) + realised = result.compute() + assert realised.mask.tolist() == [True, False, False, False] +``` + +- [ ] **Step 6: Run them to verify they fail** + +```bash +pytest lib/iris/tests/unit/fileformats/zarr/dataset/test_read_data.py -v +``` + +Expected: FAIL, +`TypeError: Can't instantiate abstract class ZarrDatasetVariable with abstract method read_data` +— `read_data` is abstract on the interface, so until it is written nothing in +the Zarr package instantiates at all. If Tasks 3–5 passed, the class already +has a stub; delete it and watch this failure appear, so that the test is known +to be able to fail. + +- [ ] **Step 7: Write `read_data`, `cache_key` and `zarr_format`** + +In `ZarrDatasetVariable`, add to the Zarr-only members: + +```python + @property + def zarr_format(self) -> int: + """2 or 3. The one Zarr fact that changes how a value is read.""" + return self._array.metadata.zarr_format + + @property + def cache_key(self) -> str: + """An identity for this array that goes stale when the array does. + + `iris._lazy_data.CACHE` is process-wide, so an address alone is not + enough: a store rewritten in place keeps its address and changes its + contents. The metadata document covers shape, chunks, shards, dtype, + fill value, codecs, dimension names and attributes in one value, and + is treated as opaque - listing the parts Iris cares about would go + stale, silently, the first time zarr-python added a field that + mattered. + """ + return json.dumps( + [self._location, self._array.path, self._array.metadata.to_dict()], + sort_keys=True, + ) +``` + +and, among the interface members, after `__setitem__`: + +```python + def read_data(self, chunking_policy): + """Return this array's data, lazily where a lazy array is worth it. + + Parameters + ---------- + chunking_policy : callable + Called with no arguments, returning the ``(chunks, dims_fixed)`` + pair to build a lazy array with. Called only once the result is + known to be lazy. + + Returns + ------- + numpy.ndarray, numpy.ma.MaskedArray or dask.array.Array + Real data for a small array, lazy data otherwise. Masked whenever + the variable declares anything to decode. + + Notes + ----- + There is no data proxy. ``zarr.Array`` is picklable, a chunk is an + independent object in a key-value store so concurrent reads need no + lock, and a store is a namespace rather than an open file handle. CF + decoding is a ``map_blocks`` layer over the raw array rather than + something hidden inside ``__getitem__``, so it is visible in the + graph and testable without a store. + + """ + # Deferred, and the module rather than its members: the CF package + # reaches this one through iris.fileformats.zarr.loader, so a + # module-level import would be a cycle; and the tests patch + # _LAZYVAR_MIN_BYTES on the module object. + from iris.fileformats.cf import loader + + parameters = _decode.decode_parameters( + self._attributes, + dtype=self.dtype, + storage_fill_value=self.fill_value, + zarr_format=self.zarr_format, + variable_name=self.name, + ) + decoding = _decode.decoding_required(parameters) + + if self.size * self.dtype.itemsize < loader._LAZYVAR_MIN_BYTES: + # A lazy array would cost more memory AND more time to access. + block = np.asanyarray(self._array[...]) + if decoding: + return _decode.mask_and_scale(block, **parameters) + return block + + chunks, dims_fixed = chunking_policy() + raw_meta = np.empty((0,) * self.ndim, dtype=self.dtype) + + if chunks == "auto": + # CHUNK_CONTROL.as_dask(): the user asked for Dask's own policy, + # so the store's layout is deliberately not consulted. + lazy = as_lazy_data( + self._array, chunks="auto", meta=raw_meta, cache_key=self.cache_key + ) + else: + chunks = _optimum_chunksize( + chunks, shape=self.shape, dtype=self.dtype, dims_fixed=dims_fixed + ) + chunks = _align_chunks( + chunks, self.chunking, self.shape, self.dtype, dims_fixed, self.name + ) + # Fixed in every dimension, because the alignment above is the + # final word: as_lazy_data would otherwise re-run the + # optimisation and undo it. + lazy = as_lazy_data( + self._array, + chunks=chunks, + meta=raw_meta, + dims_fixed=(True,) * self.ndim, + cache_key=self.cache_key, + ) + + if not decoding: + return lazy + + return da.map_blocks( + _decode.mask_and_scale, + lazy, + dtype=parameters["dtype"], + meta=np.ma.masked_array(np.empty((0,) * self.ndim, parameters["dtype"])), + **parameters, + ) +``` + +Add to the module imports: `import json`, `import dask.array as da`, +`from iris._lazy_data import _optimum_chunksize, as_lazy_data`, and +`from . import _decode`. + +Note `chunks` arrives as a list from `_chunks_from_chunk_control`; +`_optimum_chunksize` returns a tuple, and `_align_chunks` returns one, so what +reaches `as_lazy_data` is hashable — which matters, because it goes into the +cache key. + +- [ ] **Step 8: Run, then write the cache-key tests** + +```bash +pytest lib/iris/tests/unit/fileformats/zarr/dataset/test_read_data.py -v +``` + +Expected: all pass. Then append the staleness and sensitivity cases from +spec §6: + +```python +def key_of(location, name="v"): + return ZarrDataset(location).variables[name].cache_key + + +def test_the_key_is_stable_across_a_reopen(tmp_path): + group = zarr.open_group(store=tmp_path / "k.zarr", mode="w", zarr_format=3) + stores.make_array(group, "v", (4,), "i4", 3, dimensions=("t",)) + location = str(tmp_path / "k.zarr") + assert key_of(location) == key_of(location) + + +def test_the_key_changes_when_the_shape_does(tmp_path): + """A store rewritten in place keeps its address and changes its contents. + + Keyed on the address alone, an array reopened after its shape changed + from (4,) to (6,) returns the cached four values, silently truncating - + and that is weaker than the netCDF key it replaces, since + repr(NetCDFDataProxy) already carries shape and dtype. + """ + location = str(tmp_path / "k.zarr") + group = zarr.open_group(store=location, mode="w", zarr_format=3) + stores.make_array(group, "v", (4,), "i4", 3, dimensions=("t",), chunks=(2,)) + before = key_of(location) + + group = zarr.open_group(store=location, mode="a", zarr_format=3) + group.create_array( + "v", shape=(6,), dtype="i4", chunks=(2,), dimension_names=("t",), + overwrite=True, + ) + assert key_of(location) != before + + +def test_the_key_changes_when_the_fill_value_does(tmp_path): + location = str(tmp_path / "k.zarr") + group = zarr.open_group(store=location, mode="w", zarr_format=3) + group.create_array("v", shape=(4,), dtype="i4", fill_value=-1) + before = key_of(location) + + group = zarr.open_group(store=location, mode="a", zarr_format=3) + group.create_array("v", shape=(4,), dtype="i4", fill_value=-2, overwrite=True) + assert key_of(location) != before + + +@pytest.mark.parametrize( + "change", + [ + {"shape": (8,)}, + {"chunks": (4,)}, + {"shards": (8,)}, + {"dtype": "f8"}, + {"fill_value": -7}, + {"dimension_names": ("other",)}, + ], +) +def test_the_key_is_sensitive_to_every_metadata_field(tmp_path, change): + """`Array.metadata.to_dict()` is zarr-python's rendering, not a guarantee. + + Nothing in the Zarr specification defines what that method returns, so + these are what fail if a release stops carrying a field the key needs. + """ + base = dict( + shape=(4,), dtype="i4", chunks=(2,), fill_value=-1, + dimension_names=("t",), shards=None, + ) + location = str(tmp_path / "k.zarr") + group = zarr.open_group(store=location, mode="w", zarr_format=3) + group.create_array("v", **base) + before = key_of(location) + + group = zarr.open_group(store=location, mode="a", zarr_format=3) + group.create_array("v", **{**base, **change}, overwrite=True) + assert key_of(location) != before + + +def test_the_key_is_sensitive_to_attributes(tmp_path): + location = str(tmp_path / "k.zarr") + group = zarr.open_group(store=location, mode="w", zarr_format=3) + array = group.create_array("v", shape=(4,), dtype="i4") + before = key_of(location) + array.attrs["units"] = "K" + assert key_of(location) != before + + +def test_two_cubes_sharing_an_array_share_one_dask_array(tmp_path, monkeypatch): + # What the cache is for: many cubes in one store sharing a coordinate. + monkeypatch.setattr(cf_loader, "_LAZYVAR_MIN_BYTES", 1) + location = stores.make_store(tmp_path / "s.zarr") + first = variable(location).read_data(policy) + second = variable(location).read_data(policy) + assert first.name == second.name +``` + +The sensitivity test's `shards` case needs `chunks` to divide `shards`; keep +`chunks=(2,)` and `shards=(8,)` together when writing it, and drop the case +with a comment if zarr-python refuses the combination rather than loosening the +assertion. + +- [ ] **Step 9: Write the sharded read-unit test** + +Append to `test_read_data.py`: + +```python +class CountingStore(zarr.storage.WrapperStore): + """Wrap a store, adding every byte it returns to a caller-owned tally. + + zarr-python copies a store when it opens it read-only, through + `with_read_only` and `_with_store`, so a tally kept on the instance is + never the one the live store writes to, and the test passes vacuously. + """ + + def __init__(self, store, tally): + super().__init__(store=store) + self.tally = tally + + def _with_store(self, store): + return type(self)(store, self.tally) + + async def get(self, key, prototype, byte_range=None): + buffer = await self._store.get(key, prototype, byte_range) + if buffer is not None: + self.tally.append(len(buffer)) + return buffer + + async def get_partial_values(self, prototype, key_ranges): + buffers = await self._store.get_partial_values(prototype, key_ranges) + self.tally.extend(len(b) for b in buffers if b is not None) + return buffers + + +def bytes_to_read_first_chunk(location, dask_chunks): + """Return the bytes fetched reading the first eight values of "v".""" + tally = [] + store = CountingStore(zarr.storage.LocalStore(location), tally) + array = zarr.open_group(store=store, mode="r")["v"] + tally.clear() + lazy = da.from_array(array, chunks=dask_chunks, meta=np.empty((0,), "i4")) + lazy = da.map_blocks( + lambda block: np.ma.masked_array(block), lazy, dtype="i4" + ) + lazy[:8].compute() + return sum(tally) + + +def test_a_sharded_array_is_read_by_the_inner_chunk(tmp_path): + """Shard-sized Dask blocks fetch the whole shard, because of the layer. + + Bare Dask slicing can push a slice down into the array's own __getitem__, + but a map_blocks layer in between cannot, so the block is materialised in + full. The absolute byte counts are zarr-version-dependent; the comparison + is not. + """ + location = str(tmp_path / "sh.zarr") + group = zarr.open_group(store=location, mode="w", zarr_format=3) + array = group.create_array( + "v", shape=(1024,), dtype="i4", chunks=(8,), shards=(1024,) + ) + array[:] = np.arange(1024, dtype="i4") + + inner = bytes_to_read_first_chunk(location, 8) + whole_shard = bytes_to_read_first_chunk(location, 1024) + assert inner < whole_shard + + # And the variable chooses the inner chunk for exactly this reason. + assert ZarrDataset(location).variables["v"].chunking == (8,) +``` + +- [ ] **Step 10: Run the module** + +```bash +pytest lib/iris/tests/unit/fileformats/zarr/ -v +``` + +Expected: all pass. + +- [ ] **Step 11: Write the test for Review Focus 4 — the unpacked dtype** + +Append to `test_read_data.py`: + +```python +def test_the_declared_dtype_matches_what_the_decode_layer_produces(tmp_path, monkeypatch): + """A packed variable's lazy dtype must equal its realised dtype. + + On the netCDF path netCDF4 does the arithmetic and reports the dtype. On + the Zarr path Iris does both, in two separately written statements, and + Dask does not check them against each other: a mismatch builds a graph + and surfaces later as a wrong dtype on the cube. + """ + monkeypatch.setattr(cf_loader, "_LAZYVAR_MIN_BYTES", 1) + group = zarr.open_group(store=tmp_path / "p.zarr", mode="w", zarr_format=3) + array = stores.make_array( + group, + "b02", + (4,), + "u2", + 3, + dimensions=("p",), + attributes={"scale_factor": 0.0001, "add_offset": -0.1}, + ) + array[:] = [1000, 2000, 3000, 4000] + + lazy = ZarrDataset(str(tmp_path / "p.zarr")).variables["b02"].read_data(policy) + realised = lazy.compute() + + assert lazy.dtype == realised.dtype + assert lazy.dtype == np.dtype("f8") + np.testing.assert_allclose(realised, [0.0, 0.1, 0.2, 0.3]) + + +def test_the_declared_dtype_matches_the_cf_loader_calculation(tmp_path): + """And it is the same answer `_get_actual_dtype` gives. + + The cube's data variable and its coordinates must agree about what a + packed value becomes, and they reach that answer through different code. + """ + group = zarr.open_group(store=tmp_path / "p.zarr", mode="w", zarr_format=3) + stores.make_array( + group, "v", (4,), "u2", 3, + attributes={"scale_factor": np.float32(0.01)}, + ) + variable = ZarrDataset(str(tmp_path / "p.zarr")).variables["v"] + from iris.fileformats.zarr import _decode + + parameters = _decode.decode_parameters( + variable.attributes, + dtype=variable.dtype, + storage_fill_value=variable.fill_value, + zarr_format=variable.zarr_format, + variable_name=variable.name, + ) + assert parameters["dtype"] == cf_loader._get_actual_dtype(variable) +``` + +- [ ] **Step 12: Run everything built so far, and commit** + +```bash +pytest lib/iris/tests/unit/fileformats/zarr/ lib/iris/tests/unit/fileformats/cf/ lib/iris/tests/unit/lazy_data/ -q +``` + +Expected: all pass. Then run §5.0 and: + +```bash +git add lib/iris/fileformats/zarr lib/iris/tests/unit/fileformats/zarr +git commit -m "Read Zarr data lazily, aligned to the store's own chunks + +No proxy: a zarr.Array is picklable and a store is a namespace, so the +array goes to as_lazy_data directly and CF decoding is a visible map_blocks +layer. Chunking is rounded up to whole multiples of the store's own, never +below, because reading any part of a Zarr chunk fetches the whole object. +The Dask cache key carries the array's metadata document rather than its +address, which alone goes stale when a store is rewritten in place." +``` + +--- +### Task 8: `spans` matches on `(name, size)` pairs + +**Files:** +- Modify: `lib/iris/fileformats/cf/_variables.py` +- Create: `lib/iris/tests/unit/fileformats/cf/test_spans.py` + +**Interfaces:** +- Consumes: `ZarrDataset` (Task 4), `NetCDFDataset`, `stores.make_array`. +- Produces: `CFVariable._dimension_identities()` and the module-private + `_AnonymousDimension`; `CFVariable.spans` and the three overrides in + `CFBoundaryVariable`, `CFClimatologyVariable` and `CFLabelVariable` matching + on pairs. + +This is the one change in this pull request that can alter what loads from an +existing netCDF file, so it is on its own and its tests run on both backends. +Spec §4.4: + +> **The identity that links two variables is the pair `(name, size)`, and an +> anonymous dimension has no identity.** +> +> - Two variables share a dimension when they name it identically *and* it is +> the same length in both. A name reused at two lengths links nothing. +> - An anonymous dimension never matches anything — not a named dimension, and +> not another anonymous dimension, including within the same variable. +> - A variable may name a dimension the dataset does not list, and may name +> some of its dimensions and not others. Neither is an error. + +For netCDF this changes nothing by construction: dimension names are unique +within a group, so `(name, size)` and `name` are the same key, and no netCDF +variable has an anonymous dimension. + +- [ ] **Step 1: Write the failing shared suite** + +Create `lib/iris/tests/unit/fileformats/cf/test_spans.py`: + +```python +# Copyright Iris contributors +# +# This file is part of Iris and is released under the BSD license. +# See LICENSE in the root of the repository for full licensing details. +"""`CFVariable.spans`, asked of both storage backends. + +Dimension matching is a pair, `(name, size)`, not a bare name - which is a +widening to admit what Zarr allows, not a change to what netCDF does. These +tests run against both so that the second half of that sentence is checked +rather than asserted. +""" + +import numpy as np +import pytest + +from iris.fileformats.cf import ( + CFBoundaryVariable, + CFDataVariable, + CFLabelVariable, +) + +zarr = pytest.importorskip("zarr") + +from iris.fileformats.netcdf._dataset import NetCDFDataset # noqa: E402 +from iris.fileformats.zarr._dataset import ZarrDataset # noqa: E402 + + +def build_netcdf(path, described): + """Write `described` as a netCDF file and return its open dataset.""" + dataset = NetCDFDataset(path, mode="w") + sizes = {} + for dimensions, shape in described.values(): + for name, size in zip(dimensions, shape): + sizes.setdefault(name, size) + for name, size in sizes.items(): + dataset.create_dimension(name, size) + for name, (dimensions, shape) in described.items(): + dataset.create_variable(name, np.dtype("f4"), dimensions) + return dataset + + +def build_zarr(path, described): + """Write `described` as a Zarr store and return its open dataset.""" + group = zarr.open_group(store=path, mode="w", zarr_format=3) + for name, (dimensions, shape) in described.items(): + group.create_array( + name, shape=shape, dtype="f4", dimension_names=dimensions + ) + return ZarrDataset(str(path)) + + +@pytest.fixture(params=[build_netcdf, build_zarr], ids=["netcdf", "zarr"]) +def build(request, tmp_path): + """Return a callable giving `{name: CFDataVariable}` for a description.""" + suffix = ".nc" if request.param is build_netcdf else ".zarr" + + def make(described): + dataset = request.param(tmp_path / f"spans{suffix}", described) + return { + name: CFDataVariable(name, dataset.variables[name]) + for name in described + } + + return make + + +def test_identical_dimensions_span(build): + variables = build({"data": (("y", "x"), (3, 2)), "aux": (("y", "x"), (3, 2))}) + assert variables["aux"].spans(variables["data"]) is True + + +def test_a_subset_spans(build): + variables = build({"data": (("y", "x"), (3, 2)), "aux": (("y",), (3,))}) + assert variables["aux"].spans(variables["data"]) is True + + +def test_a_different_name_does_not_span(build): + variables = build({"data": (("y", "x"), (3, 2)), "aux": (("t",), (3,))}) + assert variables["aux"].spans(variables["data"]) is False + + +def test_a_scalar_always_spans(build): + variables = build({"data": (("y", "x"), (3, 2)), "grid": ((), ())}) + assert variables["grid"].spans(variables["data"]) is True + + +def test_a_bounds_variable_ignores_its_extent_dimension(build): + variables = build( + {"data": (("y",), (3,)), "bounds": (("y", "bnds"), (3, 2))} + ) + bounds = CFBoundaryVariable("bounds", variables["bounds"].cf_data) + assert bounds.spans(variables["data"]) is True + + +def test_a_label_variable_ignores_its_string_length_dimension(build): + variables = build( + {"data": (("y",), (3,)), "label": (("y", "strlen"), (3, 8))} + ) + label = CFLabelVariable("label", variables["label"].cf_data) + assert label.spans(variables["data"]) is True +``` + +- [ ] **Step 2: Run it against today's code** + +```bash +pytest lib/iris/tests/unit/fileformats/cf/test_spans.py -v +``` + +Expected: all pass on netCDF, all pass on Zarr. These are the cases that must +be **unchanged** by this task — run them before the change so that the +"netCDF behaviour is unchanged" claim has a before as well as an after. + +- [ ] **Step 3: Write the failing tests for the new rules** + +Append to `test_spans.py`: + +```python +def test_a_name_at_two_sizes_does_not_link(tmp_path): + """Zarr-only: a name guarantees nothing across arrays. + + The version 3 core specification "does not place any restrictions on the + use of the same dimension name across multiple arrays within the same Zarr + hierarchy", so two arrays may mean different things by one name. Matching + on the name alone would produce a cube whose coordinate is the wrong + length. netCDF cannot express this: a dimension name has one size per + group. + """ + group = zarr.open_group(store=tmp_path / "s.zarr", mode="w", zarr_format=3) + group.create_array("data", shape=(3,), dtype="f4", dimension_names=("t",)) + group.create_array("aux", shape=(5,), dtype="f4", dimension_names=("t",)) + + dataset = ZarrDataset(str(tmp_path / "s.zarr")) + data = CFDataVariable("data", dataset.variables["data"]) + aux = CFDataVariable("aux", dataset.variables["aux"]) + assert aux.spans(data) is False + + +def test_an_anonymous_dimension_matches_nothing(tmp_path): + group = zarr.open_group(store=tmp_path / "s.zarr", mode="w", zarr_format=3) + group.create_array("data", shape=(3,), dtype="f4", dimension_names=("t",)) + group.create_array("aux", shape=(3,), dtype="f4") + + dataset = ZarrDataset(str(tmp_path / "s.zarr")) + data = CFDataVariable("data", dataset.variables["data"]) + aux = CFDataVariable("aux", dataset.variables["aux"]) + assert aux.spans(data) is False + + +def test_two_anonymous_dimensions_of_one_size_do_not_match(tmp_path): + """Not even each other, and not within one variable. + + There is no information to match on, so claiming a match would be + invention. + """ + group = zarr.open_group(store=tmp_path / "s.zarr", mode="w", zarr_format=3) + group.create_array("data", shape=(3, 3), dtype="f4") + group.create_array("aux", shape=(3,), dtype="f4") + + dataset = ZarrDataset(str(tmp_path / "s.zarr")) + data = CFDataVariable("data", dataset.variables["data"]) + aux = CFDataVariable("aux", dataset.variables["aux"]) + assert aux.spans(data) is False + + +def test_a_partly_named_variable_matches_on_the_named_part(tmp_path): + # "A variable may name some of its dimensions and not others." The named + # ones still have to be present in the target, and the anonymous one + # still matches nothing - so this does not span. + group = zarr.open_group(store=tmp_path / "s.zarr", mode="w", zarr_format=3) + group.create_array( + "data", shape=(3, 2), dtype="f4", dimension_names=("y", "x") + ) + group.create_array( + "aux", shape=(3, 2), dtype="f4", dimension_names=("y", None) + ) + + dataset = ZarrDataset(str(tmp_path / "s.zarr")) + data = CFDataVariable("data", dataset.variables["data"]) + aux = CFDataVariable("aux", dataset.variables["aux"]) + assert aux.spans(data) is False +``` + +- [ ] **Step 4: Run them to verify they fail** + +```bash +pytest lib/iris/tests/unit/fileformats/cf/test_spans.py -v -k "two_sizes or anonymous or partly" +``` + +Expected: four FAILs — today's `set(self.dimensions).issubset(...)` matches a +bare name, so a name at two sizes spans, and `None` matches `None`. + +- [ ] **Step 5: Add the identity helper** + +In `lib/iris/fileformats/cf/_variables.py`, after `_NCZARR_SCALAR_DIMENSION`: + +```python +class _AnonymousDimension: + """One dimension the store gives no name. + + A distinct instance per dimension, relying on object identity for + equality, so that an anonymous dimension matches nothing: not a named + dimension, not another anonymous one, and not itself as seen from a + second variable. There is nothing to match on, so claiming a match would + be invention. + + Zarr's version 3 core specification makes ``dimension_names`` optional and + allows a null entry, so a conforming store can present this and it must + not be an error. No netCDF variable ever has one. + """ + + def __repr__(self) -> str: + return "" +``` + +and on `CFVariable`, beside `spans`: + +```python + def _dimension_identities(self) -> tuple: + """Return one hashable identity per dimension this variable spans. + + A named dimension's identity is the pair ``(name, size)``: two + variables share a dimension when they name it identically *and* it is + the same length in both. A name reused at two lengths links nothing, + which is the only reading that cannot produce a wrong cube. + + An unnamed dimension gets a fresh :class:`_AnonymousDimension`, which + equals nothing but itself. + + Relies on the :class:`~iris.fileformats.cf.dataset.CFDatasetVariable` + invariant that ``dimensions`` and ``shape`` have the same length. + """ + return tuple( + (name, size) if name is not None else _AnonymousDimension() + for name, size in zip(self.dimensions, self.shape) + ) +``` + +- [ ] **Step 6: Rewrite the four `spans` methods** + +`CFVariable.spans` (keep the docstring, replace the comparison): + +```python + # Scalar variables always span the target variable. + if self._is_scalar(): + return True + + source = self._dimension_identities() + target = cf_variable._dimension_identities() + return set(source).issubset(target) +``` + +`CFBoundaryVariable.spans`, `CFClimatologyVariable.spans` and +`CFLabelVariable.spans` keep their own comments about which extent dimension +they ignore, and become: + +```python + # Scalar variables always span the target variable. + result = True + if not self._is_scalar(): + source = self._dimension_identities() + target = cf_variable._dimension_identities() + # Ignore the bounds extent dimension. + result = set(source[:-1]).issubset(target) or set(source[1:]).issubset( + target + ) + return result +``` + +with "bounds" replaced by "climatology" and "label string length" in the other +two, exactly as they read today. + +- [ ] **Step 7: Run the whole suite** + +```bash +pytest lib/iris/tests/unit/fileformats/cf/test_spans.py -v +pytest -n auto lib/iris/tests/unit/fileformats/ lib/iris/tests/integration/netcdf/ -q +``` + +Expected: the new tests pass, and **nothing else changes**. This is the claim +the task exists to support: a netCDF file cannot tell the difference. + +- [ ] **Step 8: Pin the two claims that keep this change contained (F16)** + +Append to `test_spans.py`: + +```python +def test_a_non_spanning_variable_is_dropped_and_promoted(tmp_path): + """An anonymous-dimension variable becomes a cube of its own. + + This is what keeps `None` out of `iris.fileformats._nc_load_rules.helpers`, + which matches dimensions by name and would read `None in (None, "x")` as a + match. `CFReader._build_cf_groups` drops any variable whose `spans` is + False from the referrer's group and promotes it. + """ + from iris.fileformats.cf import CFReader + + group = zarr.open_group(store=tmp_path / "s.zarr", mode="w", zarr_format=3) + data = group.create_array( + "data", shape=(3,), dtype="f4", dimension_names=("t",) + ) + data.attrs["coordinates"] = "aux" + group.create_array("aux", shape=(3,), dtype="f4") + + with CFReader(ZarrDataset(str(tmp_path / "s.zarr"))) as reader: + assert "aux" not in reader.cf_group["data"].cf_group + assert "aux" in reader.cf_group.promoted + + +def test_an_anonymous_dimension_is_not_a_coordinate_variable(tmp_path): + """A one-dimensional array is a coordinate variable by name coincidence. + + `CFCoordinateVariable.identify` requires the variable's own name to be + among its dimension names, which no anonymous dimension satisfies - so + the second route into the name-matching helpers is closed too. + """ + from iris.fileformats.cf import CFCoordinateVariable + + group = zarr.open_group(store=tmp_path / "s.zarr", mode="w", zarr_format=3) + group.create_array("t", shape=(3,), dtype="f4") + + dataset = ZarrDataset(str(tmp_path / "s.zarr")) + assert CFCoordinateVariable.identify(dataset.variables) == {} +``` + +These two depend on Task 9, which teaches `CFReader` to take an open +`CFDataset`. Write them here and run them at the end of Task 9. + +- [ ] **Step 9: Commit** + +Run §5.0, then: + +```bash +git add lib/iris/fileformats/cf/_variables.py lib/iris/tests/unit/fileformats/cf/test_spans.py +git commit -m "Match CF dimensions on (name, size), not on the name alone + +Zarr places no restriction on reusing a dimension name across arrays, so a +name alone can link two variables that mean different things by it. An +unnamed dimension is anonymous and matches nothing, including another +anonymous dimension. netCDF names are unique within a group, so this is a +widening that no netCDF file can observe." +``` + +--- + +### Task 9: `CFReader` accepts an already-open `CFDataset` + +**Files:** +- Modify: `lib/iris/fileformats/cf/_reader.py` +- Modify: `lib/iris/tests/unit/fileformats/cf/test_CFReader__dataset.py` + +**Interfaces:** +- Consumes: `CFDataset` (the ABC), `ZarrDataset` (Task 4). +- Produces: `CFReader(file_source)` where `file_source` may be a `CFDataset`, + which is how Task 10's loader hands a Zarr store to the CF layer. + +`CFReader.__init__` has two branches today: a `str` opens a `NetCDFDataset`, +and anything else is passed to `NetCDFDataset.from_existing`. A `ZarrDataset` +falls into the second and is wrapped as a netCDF dataset (F12). The new branch +goes first and borrows rather than owns, because whoever opened the dataset is +responsible for closing it. + +- [ ] **Step 1: Write the failing test** + +Append to `lib/iris/tests/unit/fileformats/cf/test_CFReader__dataset.py`: + +```python +def test_an_already_open_cf_dataset_is_used_as_is(tmp_path): + """A loader that has already opened a store hands it straight over. + + The alternative - a path and a format keyword - would put the choice of + backend inside CFReader, which is the branch the dataset interface exists + to remove. + """ + from iris.fileformats.netcdf._dataset import NetCDFDataset + + path = str(tmp_path / "source.nc") + with NetCDFDataset(path, mode="w") as writer: + writer.create_dimension("x", 2) + writer.create_variable("v", np.dtype("f4"), ("x",)) + + dataset = NetCDFDataset(path, mode="r") + with CFReader(dataset) as reader: + assert reader.filename == dataset.location + assert "v" in reader.cf_group + # Borrowed, not owned: leaving the context must not close it. + assert dataset.closed is False + dataset.close() +``` + +and, in the Zarr unit tests, a matching one — +`lib/iris/tests/unit/fileformats/zarr/test_CFReader.py`: + +```python +# Copyright Iris contributors +# +# This file is part of Iris and is released under the BSD license. +# See LICENSE in the root of the repository for full licensing details. +"""The CF layer reads a Zarr store through the same reader netCDF uses.""" + +from iris.fileformats.cf import CFReader +from iris.fileformats.zarr._dataset import ZarrDataset + +from . import stores + + +def test_the_cf_reader_classifies_a_zarr_store(tmp_path): + location = stores.make_store(tmp_path / "s.zarr") + with CFReader(ZarrDataset(location)) as reader: + assert "air_temperature" in reader.cf_group.data_variables + assert "grid" in reader.cf_group.grid_mappings + assert reader.cf_group.global_attributes["title"] == "contract" + assert reader.filename == location +``` + +- [ ] **Step 2: Run them to verify they fail** + +```bash +pytest lib/iris/tests/unit/fileformats/zarr/test_CFReader.py -v +``` + +Expected: FAIL. `NetCDFDataset.from_existing` is handed a `ZarrDataset`, which +has no `THREAD_SAFE_FLAG`, so `EncodedDataset.from_existing` raises on it. + +- [ ] **Step 3: Add the branch** + +In `lib/iris/fileformats/cf/_reader.py`, add `from .dataset import CFDataset` +to the imports, and make `__init__`'s dispatch read: + +```python + # Ensure safe operation for destructor, should init fail. + self._own_file = False + if isinstance(file_source, CFDataset): + # Already open, in whatever format: use it, don't own it. The + # caller that opened it is responsible for closing it. + self._dataset = file_source + self._filename = file_source.location + elif isinstance(file_source, str): + # Create from filepath : open it + own it (=close when we die). + if not urlparse(file_source).scheme: + self._filename = Path(file_source).expanduser() + else: + self._filename = file_source + + self._dataset = NetCDFDataset( + self._filename, mode="r", warn_legacy_format=warn + ) + self._own_file = True + else: + # We have been passed an open dataset. + # We use it but don't own it (don't close it). + self._dataset = NetCDFDataset.from_existing( + file_source, warn_legacy_format=warn + ) + self._filename = self._dataset.location +``` + +- [ ] **Step 4: Update the module docstring** + +The paragraph beginning "This is the one module in :mod:`iris.fileformats.cf` +still coupled to :mod:`iris.fileformats.netcdf`" is now only half true, and a +stale docstring is worse than none. Replace it with: + +```python +``CFReader`` takes one of three things. A path opens a netCDF file, because +netCDF is the default a bare filename means. An open netCDF4 dataset, or an +emulator of one such as ncdata passes, is wrapped. And a +:class:`~iris.fileformats.cf.dataset.CFDataset` of any format is used as it +stands, which is how a Zarr store reaches this machinery: its loader opens +the store and hands it over, so nothing here has to know that Zarr exists. +Only the first case mentions netCDF, and only because something has to be +the default. +``` + +- [ ] **Step 5: Run the tests** + +```bash +pytest lib/iris/tests/unit/fileformats/cf/ lib/iris/tests/unit/fileformats/zarr/ -v +``` + +Expected: all pass, including Task 8's two deferred tests and Task 5's +no-type-test check. + +- [ ] **Step 6: Commit** + +Run §5.0, then: + +```bash +git add lib/iris/fileformats/cf/_reader.py lib/iris/tests/unit/fileformats +git commit -m "Let CFReader read any already-open CF dataset + +A loader that has opened a store hands it straight over, so the choice of +backend stays with the loader rather than becoming a branch inside the +reader. A borrowed dataset is not closed by the reader that borrowed it." +``` + +--- + +### Task 10: `zarr/loader.py` — `load_cubes` + +**Files:** +- Create: `lib/iris/fileformats/zarr/loader.py` +- Modify: `lib/iris/fileformats/zarr/__init__.py` +- Create: `lib/iris/tests/unit/fileformats/zarr/loader/__init__.py` +- Create: `lib/iris/tests/unit/fileformats/zarr/loader/test_load_cubes.py` + +**Interfaces:** +- Consumes: `ZarrDataset` (Task 4); `CFReader` taking one (Task 9); + `iris.fileformats.cf.loader._actions_engine`, `_load_cube`, + `_load_aux_factory`, `_translate_constraints_to_var_callback` (**PR 3**); + `iris.io.run_callback`. +- Produces: `iris.fileformats.zarr.load_cubes(file_sources, callback=None, + constraints=None, group=None)`, re-exported from the package. + +The shape is `netcdf/loader.py::load_cubes` with the mesh handling removed: +UGRID from Zarr is out of scope here (§1), and `_meshes_from_cf` lives in +`netcdf/ugrid_load.py`, which this package must not import. + +Spec §4.6's rule is the one piece that is not a copy: "a store whose root +group holds no data variables raises a message that names the groups that do". + +- [ ] **Step 1: Write the failing tests** + +Create `lib/iris/tests/unit/fileformats/zarr/loader/__init__.py` with only the +copyright header, then `.../loader/test_load_cubes.py`: + +```python +# Copyright Iris contributors +# +# This file is part of Iris and is released under the BSD license. +# See LICENSE in the root of the repository for full licensing details. +"""Unit tests for :func:`iris.fileformats.zarr.load_cubes`.""" + +import numpy as np +import pytest +import zarr + +from iris.cube import Cube +from iris.exceptions import TranslationError +from iris.fileformats.zarr import load_cubes + +from .. import stores + + +def test_one_cube_per_data_variable(tmp_path): + location = stores.make_store(tmp_path / "s.zarr") + cubes = list(load_cubes(location)) + assert len(cubes) == 1 + cube = cubes[0] + assert isinstance(cube, Cube) + assert cube.var_name == "air_temperature" + assert cube.units == "K" + np.testing.assert_array_equal(cube.data, stores.DATA) + + +def test_a_version_2_store_loads_the_same_cube(tmp_path): + v3 = list(load_cubes(stores.make_store(tmp_path / "v3.zarr")))[0] + v2 = list(load_cubes(stores.make_store(tmp_path / "v2.zarr", zarr_format=2)))[0] + assert v2 == v3 + + +def test_a_callback_can_reject_a_cube(tmp_path): + from iris.exceptions import IgnoreCubeException + + def reject(cube, field, filename): + raise IgnoreCubeException + + location = stores.make_store(tmp_path / "s.zarr") + assert list(load_cubes(location, callback=reject)) == [] + + +def test_the_group_keyword_selects_a_group(tmp_path): + root = zarr.open_group(store=tmp_path / "n.zarr", mode="w", zarr_format=3) + sub = root.create_group("measurements") + band = stores.make_array( + sub, "b02", (3,), "u2", 3, dimensions=("p",), attributes={"units": "1"} + ) + band[:] = [1, 2, 3] + + cubes = list(load_cubes(str(tmp_path / "n.zarr"), group="measurements")) + assert [cube.var_name for cube in cubes] == ["b02"] + + +def test_several_stores_in_one_call(tmp_path): + first = stores.make_store(tmp_path / "a.zarr") + second = stores.make_store(tmp_path / "b.zarr") + assert len(list(load_cubes([first, second]))) == 2 +``` + +- [ ] **Step 2: Write the failing tests for Review Focus 2 — an empty root** + +Append: + +```python +def test_a_group_with_no_arrays_names_the_groups_that_have_them(tmp_path): + """A real store in this shape exists, and silence is the worst answer. + + The ESA EOPF Sentinel-2 products hold no arrays in the root group, only + subgroups - `conditions/geometry`, `measurements/reflectance/r10m` and + others. Iris loads one group per call, so loading the default root yields + nothing; returning an empty CubeList from a store that plainly holds data + leaves the user with no way to find out why. + """ + root = zarr.open_group(store=tmp_path / "e.zarr", mode="w", zarr_format=3) + root.create_group("conditions") + measurements = root.create_group("measurements") + stores.make_array(measurements, "b02", (2,), "u2", 3, dimensions=("p",)) + + with pytest.raises(TranslationError, match="measurements"): + list(load_cubes(str(tmp_path / "e.zarr"))) + + +def test_the_message_says_so_when_there_are_no_groups_either(tmp_path): + zarr.open_group(store=tmp_path / "empty.zarr", mode="w", zarr_format=3) + with pytest.raises(TranslationError, match="no data variables"): + list(load_cubes(str(tmp_path / "empty.zarr"))) + + +def test_nested_groups_are_not_walked(tmp_path): + """Iris loads one group per call, like xarray. + + A Zarr store can legitimately hold unrelated datasets, and merging them + into one CubeList would be wrong. + """ + root = zarr.open_group(store=tmp_path / "n.zarr", mode="w", zarr_format=3) + top = stores.make_array( + root, "top", (2,), "f8", 3, dimensions=("p",), attributes={"units": "1"} + ) + top[:] = [1.0, 2.0] + inner = stores.make_array( + root.create_group("sub"), "inner", (2,), "f8", 3, dimensions=("p",) + ) + inner[:] = [3.0, 4.0] + + cubes = list(load_cubes(str(tmp_path / "n.zarr"))) + assert [cube.var_name for cube in cubes] == ["top"] +``` + +- [ ] **Step 3: Run them to verify they fail** + +```bash +pytest lib/iris/tests/unit/fileformats/zarr/loader/ -v +``` + +Expected: `ImportError: cannot import name 'load_cubes'`. + +- [ ] **Step 4: Write the loader** + +Create `lib/iris/fileformats/zarr/loader.py`: + +```python +# Copyright Iris contributors +# +# This file is part of Iris and is released under the BSD license. +# See LICENSE in the root of the repository for full licensing details. +"""Load Iris cubes from Zarr stores. + +.. z_reference:: iris.fileformats.zarr.loader + :tags: topic_load_save + + API reference + +A thin loop: open the store as a +:class:`~iris.fileformats.zarr._dataset.ZarrDataset`, hand it to +:class:`~iris.fileformats.cf.CFReader`, and build one cube per CF data +variable with :mod:`iris.fileformats.cf.loader` - the same machinery the +netCDF loader runs. Everything Zarr-specific happens below this module, in +:mod:`iris.fileformats.zarr._dataset`. + +Iris loads **one** group per call. A Zarr store can legitimately hold +unrelated datasets, and merging them into one ``CubeList`` would be wrong, so +nested groups are not walked and ``group=`` selects which one to read. + +UGRID meshes are not built here. :func:`iris.mesh.load_meshes` reaches a Zarr +store through the same format dispatch, but the mesh machinery lives in +:mod:`iris.fileformats.netcdf.ugrid_load` and this package does not import it. + +See sections 4.6 and 4.7 of +``docs/superpowers/specs/2026-09-21-zarr-io-design.md``. + +""" + +from collections.abc import Iterable +import warnings + +from iris.exceptions import TranslationError +from iris.fileformats.cf import CFReader +from iris.fileformats.cf.loader import ( + _actions_engine, + _load_aux_factory, + _load_cube, + _translate_constraints_to_var_callback, +) +import iris.warnings + +from ._dataset import ZarrDataset + + +def _check_has_data_variables(reader, dataset) -> None: + """Raise if this group holds nothing to make a cube from. + + Returning an empty ``CubeList`` from a store that plainly holds data + elsewhere leaves the user nothing to act on, and a store whose root group + holds only subgroups is a real shape - the ESA EOPF Sentinel products are + built that way. + """ + if reader.cf_group.data_variables or reader.cf_group.promoted: + return + + subgroups = dataset.subgroup_names + if subgroups: + message = ( + f"No data variables in Zarr group {dataset.location!r}. Load one " + "of its groups instead, with the 'group' keyword: " + f"{', '.join(sorted(subgroups))}" + ) + else: + message = f"No data variables in Zarr store {dataset.location!r}." + raise TranslationError(message) + + +def load_cubes(file_sources, callback=None, constraints=None, group=None): + """Load cubes from one or more Zarr stores. + + Parameters + ---------- + file_sources : str or iterable of str + The stores to load. A local directory path, or a URL that + zarr-python can open - ``s3://``, ``gs://``, ``az://`` and ``abfs://`` + need the matching store library installed, which Iris does not + depend on. + callback : callable, optional + Passed to :func:`iris.io.run_callback`. + constraints : optional + Used to skip data variables that cannot match, before they are built. + group : str, optional + A ``/``-separated path to a group within each store. The root group by + default. + + Yields + ------ + iris.cube.Cube + + Raises + ------ + iris.exceptions.TranslationError + If the selected group holds no data variables. The message names the + store's other groups, when it has any. + + Notes + ----- + This function maintains laziness; it does not realise data. + + """ + from iris.io import run_callback + + var_callback = _translate_constraints_to_var_callback(constraints) + engine = _actions_engine() + + if isinstance(file_sources, str) or not isinstance(file_sources, Iterable): + file_sources = [file_sources] + + for file_source in file_sources: + with ZarrDataset(file_source, group=group) as dataset: + with CFReader(dataset) as cf: + _check_has_data_variables(cf, dataset) + + data_variables = list(cf.cf_group.data_variables.values()) + list( + cf.cf_group.promoted.values() + ) + for cf_var in data_variables: + if var_callback and not var_callback(cf_var): + continue + + cube = _load_cube(engine, cf, cf_var, cf.filename) + + try: + _load_aux_factory(engine, cube) + except ValueError as exception: + warnings.warn( + str(exception), + category=iris.warnings.IrisLoadWarning, + ) + + cube = run_callback(callback, cube, cf_var, file_source) + if cube is None: + # The callback rejected it. + continue + + yield cube +``` + +- [ ] **Step 5: Re-export it** + +In `lib/iris/fileformats/zarr/__init__.py`, after the docstring: + +```python +from .loader import load_cubes + +__all__ = ["load_cubes"] +``` + +This is the line Task 1 deferred. Re-run Task 1's import-guard tests +immediately: `iris.fileformats.zarr` now imports `zarr` through `loader`, so +`test_importing_the_zarr_package_does_not_import_zarr` **must now fail**, and +that is correct — the test was about the package before it had a loader. +Replace it with the one that still matters: + +```python +def test_importing_the_zarr_package_imports_zarr(): + # Once the package exports a loader it needs the dependency. What must + # stay true is that nothing in `iris` or `iris.fileformats` imports this + # package to find out that it exists. + assert _imports_zarr("import iris.fileformats.zarr") +``` + +- [ ] **Step 6: Run the loader tests** + +```bash +pytest lib/iris/tests/unit/fileformats/zarr/ -v +``` + +Expected: all pass. + +- [ ] **Step 7: Check what a cube from a Zarr store actually carries** + +Append to `test_load_cubes.py`: + +```python +def test_the_cube_is_lazy(tmp_path, monkeypatch): + from iris.fileformats.cf import loader as cf_loader + + monkeypatch.setattr(cf_loader, "_LAZYVAR_MIN_BYTES", 1) + cube = list(load_cubes(stores.make_store(tmp_path / "s.zarr")))[0] + assert cube.has_lazy_data() + + +def test_the_grid_mapping_variable_does_not_become_a_cube(tmp_path): + cubes = list(load_cubes(stores.make_store(tmp_path / "s.zarr"))) + assert "grid" not in [cube.var_name for cube in cubes] + + +def test_cf_reserved_attributes_do_not_reach_the_cube(tmp_path): + """`units` becomes a property, so it must not also be an attribute. + + Which attributes survive is decided by which ones the loading rules did + not read, so an attribute Iris consumes but fails to record as read + arrives on the cube as well. + """ + cube = list(load_cubes(stores.make_store(tmp_path / "s.zarr")))[0] + assert "units" not in cube.attributes + assert "long_name" not in cube.attributes + assert cube.attributes["Conventions"] == "CF-1.7" + + +def test_a_structured_attribute_reaches_the_cube_unchanged(tmp_path): + """CF cannot express a nested attribute; Zarr allows one. Carry it. + + Two independent producers write them - NOAA GFS gives `valid_time` a + `statistics_approximate` object, and the ESA EOPF product root carries + `stac_discovery`. Interpreting them needs the attribute-model discussion + in SciTools/iris#7288; dropping them loses the publisher's metadata. + """ + group = zarr.open_group(store=tmp_path / "n.zarr", mode="w", zarr_format=3) + array = stores.make_array( + group, + "v", + (2,), + "f8", + 3, + dimensions=("p",), + attributes={"stac_discovery": {"id": "abc"}}, + ) + array[:] = [1.0, 2.0] + + cube = list(load_cubes(str(tmp_path / "n.zarr")))[0] + assert cube.attributes["stac_discovery"] == {"id": "abc"} + + +def test_a_structured_attribute_cannot_then_be_saved_to_netcdf(tmp_path): + """Known limitation, characterised so that it is not a surprise. + + Zarr allows any JSON literal as an attribute value and netCDF does not, + so a store carrying one produces a cube Iris cannot save as netCDF. + Checking attribute content before saving is SciTools/iris#7329, and + fixing it means changing the netCDF saver. Strip the attribute, or save + to Zarr once that is supported. + """ + import iris + + group = zarr.open_group(store=tmp_path / "n.zarr", mode="w", zarr_format=3) + array = stores.make_array( + group, + "v", + (2,), + "f8", + 3, + dimensions=("p",), + attributes={"stac_discovery": {"id": "abc"}}, + ) + array[:] = [1.0, 2.0] + cube = list(load_cubes(str(tmp_path / "n.zarr")))[0] + + with pytest.raises(Exception): + iris.save(cube, str(tmp_path / "out.nc")) + + # And the documented workaround works. + del cube.attributes["stac_discovery"] + iris.save(cube, str(tmp_path / "out.nc")) + + +def test_version_3_zeros_are_not_masked(tmp_path): + """An int16 [0, 1, 2] with no CF attributes loads with nothing masked. + + zarr-python defaults `fill_value` to zero, so reading the storage field + as a sentinel masks valid data in the ordinary case. + """ + group = zarr.open_group(store=tmp_path / "z3.zarr", mode="w", zarr_format=3) + array = stores.make_array( + group, "v", (3,), "i2", 3, dimensions=("p",), attributes={"units": "1"} + ) + array[:] = [0, 1, 2] + + cube = list(load_cubes(str(tmp_path / "z3.zarr")))[0] + assert not np.ma.is_masked(cube.data) + np.testing.assert_array_equal(cube.data, [0, 1, 2]) + + +def test_version_2_zeros_are_masked(tmp_path): + """The same array as version 2 does use `fill_value` as the mask source. + + Version 2 stores predate the CF `_FillValue` convention, and this is what + their writers meant - which is what released xarray's + `use_zarr_fill_value_as_mask` does. + """ + group = zarr.open_group(store=tmp_path / "z2.zarr", mode="w", zarr_format=2) + array = stores.make_array( + group, "v", (3,), "i2", 2, dimensions=("p",), attributes={"units": "1"} + ) + array[:] = [0, 1, 2] + + cube = list(load_cubes(str(tmp_path / "z2.zarr")))[0] + assert cube.data.mask.tolist() == [True, False, False] + + +def test_a_cf_fill_value_beats_the_storage_one(tmp_path): + group = zarr.open_group(store=tmp_path / "z.zarr", mode="w", zarr_format=3) + array = stores.make_array( + group, + "v", + (3,), + "i2", + 3, + dimensions=("p",), + fill_value=0, + attributes={"units": "1", "_FillValue": -999}, + ) + array[:] = [0, -999, 2] + + cube = list(load_cubes(str(tmp_path / "z.zarr")))[0] + assert cube.data.mask.tolist() == [False, True, False] +``` + +- [ ] **Step 8: Run and commit** + +```bash +pytest lib/iris/tests/unit/fileformats/zarr/ -v +``` + +Expected: all pass. Then run §5.0 and: + +```bash +git add lib/iris/fileformats/zarr lib/iris/tests/unit/fileformats/zarr +git commit -m "Build cubes from a Zarr store + +Open the store, hand it to CFReader, and run the same cube-building loop +the netCDF loader runs. One group per call: a store can hold unrelated +datasets, so a group with no data variables names the groups that have +some rather than returning nothing." +``` + +--- +### Task 11: format detection — the `ZarrStore` element and the `"Zarr"` specification + +**Files:** +- Modify: `lib/iris/io/format_picker.py` +- Modify: `lib/iris/fileformats/__init__.py` +- Create: `lib/iris/tests/unit/io/format_picker/test_ZarrStore.py` + +**Interfaces:** +- Consumes: `iris.io.decode_uri`, `iris.io._is_nczarr_fragment`; + `iris.fileformats.zarr.load_cubes` (Task 10). +- Produces: `ZarrStore(FileElement)`; `iris.io.REMOTE_STORE_SCHEMES`; + `iris.fileformats._is_zarr_store(target)` and `_load_zarr(*args, **kwargs)`; + and the registered `"Zarr"` `FormatSpecification`, which Task 12 dispatches + to. + +Spec §4.7 sets the matching rule and forbids the obvious simplification: + +> - **locally**, the target is a directory containing `zarr.json` (version 3) +> or `.zgroup` (version 2); +> - **remotely**, the URI scheme is one of `s3`, `gs`, `az`, `abfs` and the +> path ends in `.zarr`. +> +> **Do not add a third rule matching any path that ends in `.zarr`.** Joined by +> "or" it swallows the remote rule whole, leaving the scheme test unreachable. + +One case the spec's prose does not spell out and the code must handle: the +format agent is asked about NCZarr URLs too, because `load_http` calls +`get_spec(url, None)` with the fragment still attached. A local NCZarr target +— `file:///path/store.zarr#mode=nczarr,file` — is a real Zarr store on disk, so +the marker-file test says yes unless the fragment is checked first. Spec §4.7's +claim that "the native reader and the NCZarr one are never candidates for the +same target" is true of what the *user* asked for, and this is what makes the +code agree with it. + +- [ ] **Step 1: Write the failing tests** + +Create `lib/iris/tests/unit/io/format_picker/test_ZarrStore.py`: + +```python +# Copyright Iris contributors +# +# This file is part of Iris and is released under the BSD license. +# See LICENSE in the root of the repository for full licensing details. +"""Recognising a Zarr store, locally by its marker file and remotely by its URL.""" + +import pytest + +from iris.fileformats import _is_zarr_store +from iris.io.format_picker import ZarrStore + + +def test_the_element_carries_the_whole_target(): + # Not the basename: recognising a local store means reading a file inside + # it, so the element needs the path, not the last component of it. + element = ZarrStore() + assert element.requires_fh is False + assert element.get_element("/data/store.zarr", None) == "/data/store.zarr" + + +def test_a_version_3_store_is_recognised(tmp_path): + store = tmp_path / "v3.zarr" + store.mkdir() + (store / "zarr.json").write_text('{"zarr_format": 3, "node_type": "group"}') + assert _is_zarr_store(str(store)) is True + + +def test_a_version_2_store_is_recognised(tmp_path): + store = tmp_path / "v2.zarr" + store.mkdir() + (store / ".zgroup").write_text('{"zarr_format": 2}') + assert _is_zarr_store(str(store)) is True + + +def test_a_directory_named_zarr_with_no_marker_is_not_a_store(tmp_path): + """The suffix alone is not the local test, and must not become one. + + Reading the marker file is what distinguishes a real store from a + directory someone happened to name `.zarr`, and locally that costs one + stat. + """ + store = tmp_path / "pretend.zarr" + store.mkdir() + assert _is_zarr_store(str(store)) is False + + +def test_a_plain_file_is_not_a_store(tmp_path): + path = tmp_path / "data.nc" + path.write_bytes(b"CDF\x01") + assert _is_zarr_store(str(path)) is False + + +@pytest.mark.parametrize("scheme", ["s3", "gs", "az", "abfs"]) +def test_a_remote_store_is_recognised_by_scheme_and_suffix(scheme): + assert _is_zarr_store(f"{scheme}://bucket/path/data.zarr") is True + assert _is_zarr_store(f"{scheme}://bucket/path/data.zarr/") is True + + +def test_a_remote_url_without_the_suffix_is_not_a_store(): + # Remotely the suffix is all there is to go on: reading a marker file + # would cost a network round trip before Iris knows it wants the target. + assert _is_zarr_store("s3://bucket/path/data") is False + + +def test_an_http_url_is_not_a_store(): + # OPeNDAP, which the netCDF loader already claims. + assert _is_zarr_store("https://example.invalid/data.zarr") is False + + +def test_an_nczarr_url_is_not_a_native_store(tmp_path): + """A user who writes `#mode=nczarr` has asked for NCZarr by name. + + The store on disk is a real one, so the marker-file test says yes unless + the fragment is checked first - and `load_http` passes the fragment + through to the format agent. + """ + store = tmp_path / "s.zarr" + store.mkdir() + (store / "zarr.json").write_text('{"zarr_format": 3, "node_type": "group"}') + assert _is_zarr_store(f"file://{store}#mode=nczarr,file") is False + + +def test_a_non_string_target_is_not_a_store(): + # `load_data_objects` passes open datasets, not paths. + assert _is_zarr_store(object()) is False +``` + +- [ ] **Step 2: Run them to verify they fail** + +```bash +pytest lib/iris/tests/unit/io/format_picker/test_ZarrStore.py -v +``` + +Expected: `ImportError: cannot import name 'ZarrStore'`. + +- [ ] **Step 3: Add the file element** + +In `lib/iris/io/format_picker.py`, after `UriProtocol`: + +```python +class ZarrStore(FileElement): + """A :class:`FileElement` that returns a target that may be a Zarr store. + + Carries the whole target rather than its basename, because recognising a + local store means reading a marker file inside it, and recognising a + remote one means reading its scheme. + + """ + + def __init__(self): + FileElement.__init__(self, requires_fh=False) + + def get_element(self, basename, file_handle): + """Return `basename`, which here is the whole path or URL.""" + return basename +``` + +- [ ] **Step 4: Add the schemes and the matching rule** + +In `lib/iris/io/__init__.py`, beside `_is_nczarr_fragment`: + +```python +#: URI schemes Iris hands to a store library rather than resolving as a local +#: path. Iris depends on none of them: zarr-python raises its own ImportError +#: naming what to install. +REMOTE_STORE_SCHEMES = frozenset({"s3", "gs", "az", "abfs"}) +``` + +In `lib/iris/fileformats/__init__.py`, after `_uri_is_nczarr`: + +```python +#: What a version 3 and a version 2 Zarr store put at the root of a store. +_ZARR_MARKERS = ("zarr.json", ".zgroup") + + +def _is_zarr_store(target): + """Whether `target` names a native Zarr store. + + Locally the test is a marker file, which costs one stat and is what tells + a real store from a directory someone named ``.zarr``. Remotely it is the + scheme and the suffix, because reading a marker file would cost a network + round trip before Iris knows it wants the target at all; a remote + ``.zarr`` that turns out not to be a store fails when zarr-python opens + it, with zarr-python's message. + + These are two tests for two kinds of target, not two chances for one. A + third rule matching any path ending in ``.zarr`` would subsume the remote + one and leave the scheme test unreachable. + """ + if not isinstance(target, str): + # iris.io.load_data_objects passes open datasets, not paths. + return False + + scheme, part, fragment = decode_uri(target) + if _is_nczarr_fragment(fragment): + # The user asked for netCDF-c's Zarr driver by name. The store on + # disk is a real one, so the local test below would otherwise claim it. + return False + + if scheme in REMOTE_STORE_SCHEMES: + return part.rstrip("/").endswith(".zarr") + + directory = Path(part) + return any((directory / marker).is_file() for marker in _ZARR_MARKERS) + + +def _load_zarr(*args, **kwargs): + """Load from Zarr stores, reporting clearly when zarr-python is absent.""" + try: + from iris.fileformats.zarr import load_cubes + except ImportError as exception: + message = ( + "Unable to load a Zarr store - the 'zarr' package is not " + "installed. Install zarr >=3.0.8, for example with " + "'conda install -c conda-forge zarr'." + ) + raise ImportError(message) from exception + + return load_cubes(*args, **kwargs) +``` + +Add `from pathlib import Path` and extend the existing +`from iris.io import _is_nczarr_fragment` to +`from iris.io import REMOTE_STORE_SCHEMES, _is_nczarr_fragment, decode_uri`, +and add `ZarrStore` to the `iris.io.format_picker` import list. + +`_load_zarr` follows `_load_grib`, three functions above it: a module-level +shim is how Iris already defers an optional format dependency, and it keeps +`iris.fileformats` importable without zarr. + +- [ ] **Step 5: Register the specification** + +In `lib/iris/fileformats/__init__.py`, under the `# Zarr files.` heading, +**before** the `NcZarr` specification so that the two read in version order: + +```python +FORMAT_AGENT.add_spec( + FormatSpecification( + "Zarr", + ZarrStore(), + _is_zarr_store, + _load_zarr, + priority=5, + constraint_aware_handler=True, + ) +) +``` + +- [ ] **Step 6: Run the tests** + +```bash +pytest lib/iris/tests/unit/io/ -v +``` + +Expected: all pass. + +- [ ] **Step 7: Prove the registration does not disturb the other formats** + +Append to `test_ZarrStore.py`: + +```python +def test_the_agent_still_picks_netcdf_for_a_netcdf_file(tmp_path): + """A `requires_fh=False` spec is consulted for every target with no buffer. + + Adding one risks claiming a target another format already handles, and + the format agent returns the first match rather than the best. + """ + import iris.tests as tests + from iris.fileformats import FORMAT_AGENT + + path = tests.get_data_path(("NetCDF", "global", "xyt", "SMALL_hires.nc")) + with open(path, "rb") as handle: + spec = FORMAT_AGENT.get_spec("SMALL_hires.nc", handle) + assert spec.name.startswith("NetCDF") + + +def test_the_agent_picks_zarr_for_a_store(tmp_path): + from iris.fileformats import FORMAT_AGENT + + store = tmp_path / "s.zarr" + store.mkdir() + (store / "zarr.json").write_text('{"zarr_format": 3, "node_type": "group"}') + assert FORMAT_AGENT.get_spec(str(store), None).name == "Zarr" + + +def test_the_agent_still_picks_nczarr_for_an_nczarr_url(tmp_path): + from iris.fileformats import FORMAT_AGENT + + store = tmp_path / "s.zarr" + store.mkdir() + (store / "zarr.json").write_text('{"zarr_format": 3, "node_type": "group"}') + uri = f"file://{store}#mode=nczarr,file" + assert FORMAT_AGENT.get_spec(uri, None).name == "NcZarr" +``` + +The first needs `iris-test-data`; it is a skip if the repository is missing, +which is why §5.0 checks the path. + +- [ ] **Step 8: Run, then commit** + +```bash +pytest lib/iris/tests/unit/io/ lib/iris/tests/unit/fileformats/ -q +``` + +Expected: all pass. Then run §5.0 and: + +```bash +git add lib/iris/io/format_picker.py lib/iris/fileformats/__init__.py lib/iris/tests/unit/io +git commit -m "Teach the format agent to recognise a Zarr store + +Locally by the marker file a store puts at its root, remotely by the URI +scheme and the .zarr suffix. Not by the suffix alone: that would claim any +directory someone named .zarr, and joined with the remote rule it would +leave the scheme test unreachable. An NCZarr URL is still NCZarr's, which +needs saying in code because the store it names is a real one." +``` + +--- + +### Task 12: the load chain — a target that is not a file + +**Files:** +- Modify: `lib/iris/io/__init__.py` +- Modify: `lib/iris/loading.py` +- Modify: `lib/iris/fileformats/netcdf/ugrid_load.py` +- Create: `lib/iris/tests/unit/io/test_load_stores.py` +- Modify: `lib/iris/tests/unit/io/test_expand_filespecs.py` (or the module + holding those tests today) + +**Interfaces:** +- Consumes: `REMOTE_STORE_SCHEMES`, the `"Zarr"` specification (Task 11). +- Produces: `iris.io.load_stores(targets, callback, constraints=None)`; + `iris.io._is_remote_store_uri(spec)`; `iris.io._CUBELIST_SAVER_MODULES`; and + `iris.load("store.zarr")` and `iris.load("s3://bucket/store.zarr")` working. + +Spec §4.7's four changes, and the one consequence it names: + +> The one thing this costs is that the file-or-store decision has to happen +> *after* glob expansion and *before* format dispatch, because `iris.load` +> must still accept `"data/*"` matching a mix of `.nc` files and `.zarr` +> stores. + +- [ ] **Step 1: Write the failing tests for `load_stores`** + +Create `lib/iris/tests/unit/io/test_load_stores.py`: + +```python +# Copyright Iris contributors +# +# This file is part of Iris and is released under the BSD license. +# See LICENSE in the root of the repository for full licensing details. +"""Unit tests for :func:`iris.io.load_stores`.""" + +import numpy as np +import pytest + +import iris +from iris.io import _is_remote_store_uri, load_stores + +zarr = pytest.importorskip("zarr", minversion="3.0.8") + +from iris.tests.unit.fileformats.zarr import stores # noqa: E402 + + +def test_a_local_store_loads(tmp_path): + location = stores.make_store(tmp_path / "s.zarr") + cubes = list(load_stores([location], None)) + assert [cube.var_name for cube in cubes] == ["air_temperature"] + + +def test_several_stores_in_one_call(tmp_path): + first = stores.make_store(tmp_path / "a.zarr") + second = stores.make_store(tmp_path / "b.zarr") + assert len(list(load_stores([first, second], None))) == 2 + + +def test_constraints_reach_the_handler(tmp_path): + location = stores.make_store(tmp_path / "s.zarr") + constraint = iris.Constraint(cube_func=lambda cube: False) + assert list(load_stores([location], None, constraint)) == [] + + +@pytest.mark.parametrize( + "uri, expected", + [ + ("s3://bucket/data.zarr", True), + ("gs://bucket/data.zarr", True), + ("az://container/data.zarr", True), + ("abfs://container/data.zarr", True), + ("https://example.invalid/data.nc", False), + ("/local/path/data.zarr", False), + ("relative/data.zarr", False), + ], +) +def test_which_uris_are_remote_stores(uri, expected): + assert _is_remote_store_uri(uri) is expected +``` + +- [ ] **Step 2: Write the failing tests for the chain** + +Append: + +```python +def test_iris_load_accepts_a_local_store(tmp_path): + location = stores.make_store(tmp_path / "s.zarr") + cubes = iris.load(location) + assert len(cubes) == 1 + np.testing.assert_array_equal(cubes[0].data, stores.DATA) + + +def test_iris_load_accepts_a_remote_url_and_does_not_absolutise_it(): + """A remote URL must reach zarr-python intact. + + `expand_filespecs` would otherwise turn `s3://bucket/data.zarr` into + `//s3:/bucket/data.zarr`. No network is touched: opening the store + fails, and what matters is that the message still names the URL. + """ + with pytest.raises(Exception) as caught: + iris.load("s3://bucket/data.zarr") + assert "s3://bucket/data.zarr" in str(caught.value) or isinstance( + caught.value, ImportError + ) + + +def test_a_glob_matching_both_files_and_stores(tmp_path): + """`iris.load("data/*")` must accept a mix. + + The file-or-store decision happens after glob expansion and before format + dispatch. Mis-partitioning raises IsADirectoryError from `load_files` for + the stores while the netCDF files load fine, which reads as an Iris bug + to a user who did not expect the glob to match a store. + """ + import iris.tests as tests + + source = iris.load_cube( + tests.get_data_path(("NetCDF", "global", "xyt", "SMALL_hires.nc")) + ) + iris.save(source, str(tmp_path / "a.nc")) + stores.make_store(tmp_path / "b.zarr") + + cubes = iris.load(str(tmp_path / "*")) + names = sorted(cube.var_name or cube.name() for cube in cubes) + assert "air_temperature" in names + assert len(cubes) == 2 +``` + +- [ ] **Step 3: Run them to verify they fail** + +```bash +pytest lib/iris/tests/unit/io/test_load_stores.py -v +``` + +Expected: `ImportError: cannot import name 'load_stores'`, and once that is +written, `IsADirectoryError` from `iris.load`. + +- [ ] **Step 4: Teach `expand_filespecs` about remote schemes** + +In `lib/iris/io/__init__.py`, beside `REMOTE_STORE_SCHEMES`: + +```python +def _is_remote_store_uri(spec): + """Whether `spec` names a store Iris hands to a store library untouched. + + Such a URI must not be resolved as a local path: `.absolute()` turns + ``s3://bucket/data.zarr`` into ``//s3:/bucket/data.zarr``. + """ + if not isinstance(spec, str): + return False + scheme, _part, _fragment = decode_uri(spec) + return scheme in REMOTE_STORE_SCHEMES +``` + +and in `expand_filespecs`, replace the two statements that build `filenames` +and `glob_expanded`: + +```python + # Remove any hostname component - currently unused. A remote store URI is + # passed through: it is not a path, and resolving it mangles it. + filenames = [ + fn + if _is_remote_store_uri(fn) + else str(pathlib.Path(fn.removeprefix("//")).expanduser().absolute()) + for fn in file_specs + ] + + if files_expected: + # Try to expand all filenames as globs. A remote URI stands for + # itself; Iris does not list a remote store's neighbours. + glob_expanded = OrderedDict( + [ + [fn, [fn] if _is_remote_store_uri(fn) else sorted(glob.glob(fn))] + for fn in filenames + ] + ) +``` + +- [ ] **Step 5: Write `load_stores`** + +In `lib/iris/io/__init__.py`, after `load_files`: + +```python +def load_stores(targets, callback, constraints=None): + """Create a generator of Cubes from targets that are stores, not files. + + A store is a directory or a URL: something with no single seekable file + to open, so :func:`load_files` cannot reach it and the format agent is + asked without a buffer. + + Parameters + ---------- + targets : iterable of str + Local directory paths or remote URLs, already expanded. + callback : callable + Passed to the format handler. + constraints : optional + Passed to a constraint-aware format handler. + + Yields + ------ + iris.cube.Cube + + Notes + ----- + Typically, this function should not be called directly; instead, the + intended interface for loading is :func:`iris.load`. + + """ + from iris.fileformats import FORMAT_AGENT + + handler_map = collections.defaultdict(list) + for target in targets: + handling_format_spec = FORMAT_AGENT.get_spec(target, None) + handler_map[handling_format_spec].append(target) + + for handling_format_spec in sorted(handler_map): + names = handler_map[handling_format_spec] + if handling_format_spec.constraint_aware_handler: + for cube in handling_format_spec.handler(names, callback, constraints): + yield cube + else: + for cube in handling_format_spec.handler(names, callback): + yield cube +``` + +- [ ] **Step 6: Partition in `_generate_cubes`** + +In `lib/iris/loading.py`, replace the `file` branch and the scheme list: + +```python + if category == "file": + part_names = [part for _cat, _scheme, part in groups] + # The file-or-store decision has to happen after glob expansion + # and before format dispatch, because a glob may match both. + # Neither routine below learns about the other's kind of target. + expanded = iris.io.expand_filespecs(part_names) + files = [name for name in expanded if not Path(name).is_dir()] + stores = [name for name in expanded if Path(name).is_dir()] + if files: + for cube in iris.io.load_files(files, callback, constraints): + yield cube + if stores: + for cube in iris.io.load_stores(stores, callback, constraints): + yield cube + elif category in ["http", "https", "nczarr"]: + urls = [f"{scheme}:{part}" for _cat, scheme, part in groups] + for cube in iris.io.load_http(urls, callback): + yield cube + elif category in iris.io.REMOTE_STORE_SCHEMES: + urls = [f"{scheme}:{part}" for _cat, scheme, part in groups] + for cube in iris.io.load_stores(urls, callback, constraints): + yield cube + elif category == "data": +``` + +Add `from pathlib import Path` to `loading.py`'s imports. + +- [ ] **Step 7: Run the chain tests** + +```bash +pytest lib/iris/tests/unit/io/ -v +``` + +Expected: all pass. + +- [ ] **Step 8: Update the saver-side pair** + +Still in `lib/iris/io/__init__.py`. Beside `_savers`: + +```python +#: Modules whose `save` takes a CubeList directly, rather than one cube at a +#: time with `append=True`. One line per format. +_CUBELIST_SAVER_MODULES = frozenset({"iris.fileformats.netcdf.saver"}) +``` + +In `find_saver`: + +```python + _check_init_savers() + # A store is a directory, so "out.zarr/" is a natural thing to type. + candidate = filespec.rstrip("/\\").lower() + matches = [ + ext for ext in _savers if candidate.endswith("." + ext) or candidate == ext + ] +``` + +In `save`, replace the substring test: + +```python + # Only allow cubelist saving for those fileformats that are capable. + if saver.__module__ not in _CUBELIST_SAVER_MODULES: +``` + +- [ ] **Step 9: Test the saver-side pair** + +Append to `test_load_stores.py`: + +```python +def test_find_saver_tolerates_a_trailing_separator(): + from iris.io import find_saver + + assert find_saver("out.nc") is find_saver("out.nc/") + + +def test_the_cubelist_saver_set_names_the_netcdf_saver(): + """The substring test it replaces matched by accident of module naming. + + `"iris.fileformats.netcdf" not in saver.__module__` is true of any module + whose path happens to contain that string, and says nothing a reader can + check. The set is one line per format. + """ + from iris.fileformats.netcdf import save as netcdf_save + from iris.io import _CUBELIST_SAVER_MODULES + + assert netcdf_save.__module__ in _CUBELIST_SAVER_MODULES + + +def test_saving_a_cubelist_to_netcdf_still_takes_one_call(tmp_path): + import iris.tests as tests + + cubes = iris.load(tests.get_data_path(("NetCDF", "global", "xyt", "SMALL_hires.nc"))) + iris.save(cubes, str(tmp_path / "out.nc")) + assert len(iris.load(str(tmp_path / "out.nc"))) == len(cubes) +``` + +- [ ] **Step 10: Update the mesh loader's copy of the dispatch** + +In `lib/iris/fileformats/netcdf/ugrid_load.py`, the `load_meshes` scheme +dispatch is a second copy of `_generate_cubes`'s and moves with it: + +```python + for category, groups in groupby(uri_tuples, key=lambda x: x[0]): + # Call each scheme handler with the appropriate URIs + if category == "file": + filenames = [part for _cat, _scheme, part in groups] + sources = expand_filespecs(filenames) + elif category in ["http", "https", "nczarr"]: + sources = [f"{scheme}:{part}" for _cat, scheme, part in groups] + elif category in REMOTE_STORE_SCHEMES: + sources = [f"{scheme}:{part}" for _cat, scheme, part in groups] + else: + message = f"Iris cannot handle the URI scheme: {category}" + raise ValueError(message) + + for source in sources: + if category == "file" and not Path(source).is_dir(): + with open(source, "rb") as fh: + handling_format_spec = FORMAT_AGENT.get_spec(Path(source).name, fh) + else: + handling_format_spec = FORMAT_AGENT.get_spec(source, None) + + if handling_format_spec.handler == nc_loader.load_cubes: + valid_sources.append(source) + else: + message = f"Ignoring non-NetCDF file: {source}" + logger.info(msg=message, extra=dict(cls=None)) +``` + +Add `REMOTE_STORE_SCHEMES` to its `iris.io` import. + +A Zarr store reaches `CFReader(source)` as a path, which opens a +`NetCDFDataset` and fails — so `load_meshes` does not yet build a mesh from +Zarr, and must not claim to. The handler check above rejects it: the `"Zarr"` +specification's handler is `_load_zarr`, not `nc_loader.load_cubes`, so a Zarr +store is logged and skipped exactly as a PP file is. That is the honest +behaviour for this pull request, and it is what the test below pins. + +- [ ] **Step 11: Test the mesh dispatch** + +Append to `test_load_stores.py`: + +```python +def test_load_meshes_skips_a_zarr_store_rather_than_raising(tmp_path): + """Reaching a store is this pull request's change; reading a mesh from + one is not. What must not happen is the ValueError the scheme dispatch + raised for anything that was not a file. + """ + from iris.mesh import load_meshes + + stores.make_store(tmp_path / "s.zarr") + assert load_meshes(str(tmp_path / "s.zarr")) == {} +``` + +- [ ] **Step 12: Run the neighbourhood** + +```bash +pytest -n auto lib/iris/tests/unit/io/ lib/iris/tests/unit/fileformats/ lib/iris/tests/unit/test_load*.py -q +pytest -n auto lib/iris/tests/integration/netcdf/ -q +``` + +Expected: all pass, with no change to the netCDF counts. + +- [ ] **Step 13: Commit** + +Run §5.0, then: + +```bash +git add lib/iris/io lib/iris/loading.py lib/iris/fileformats/netcdf/ugrid_load.py lib/iris/tests/unit +git commit -m "Let the load chain reach a target that is not a file + +A new iris.io.load_stores answers 'this target is not a file', once, for a +local directory and a remote URL alike, rather than teaching load_files +about directories in one place and the scheme dispatch in another. The +partition happens after glob expansion, so iris.load('data/*') still +accepts a mix of netCDF files and Zarr stores." +``` + +--- +### Task 13: integration — a realistic cube, both Zarr versions + +**Files:** +- Create: `lib/iris/tests/integration/zarr/__init__.py` +- Create: `lib/iris/tests/integration/zarr/fixtures.py` +- Create: `lib/iris/tests/integration/zarr/test_load.py` + +**Interfaces:** +- Consumes: everything above; `iris.tests.stock.realistic_4d_w_everything`; + the existing NCZarr save path. +- Produces: coverage only. + +PR 4 has no saver, so there is no Iris round trip to make. Two substitutes do +the work, and between them they are better evidence than a round trip would +be, because neither is Iris marking its own homework: + +- **Version 2** comes from Iris's own NCZarr `mode=xarray` save path, which + already writes a version 2 store with `_ARRAY_DIMENSIONS` (spec §6). Save + `realistic_4d_w_everything()` through it, load it back natively, and compare + cubes. That is a direct check that the native reader and the netCDF-c writer + agree. +- **Version 3** comes from a fixture written with zarr-python, carrying the CF + features that matter: dimension coordinates, a two-dimensional auxiliary + coordinate, bounds, a scalar grid mapping, packing, a base64 `_FillValue` + and sharding. Those are the properties spec §7 says make the real NOAA GFS + fixture worth having; PR 7 replaces this with the real thing. + +- [ ] **Step 1: Create the package and the version 3 fixture builder** + +Create `lib/iris/tests/integration/zarr/__init__.py`: + +```python +# Copyright Iris contributors +# +# This file is part of Iris and is released under the BSD license. +# See LICENSE in the root of the repository for full licensing details. +"""Integration tests for native Zarr loading. + +Skipped when zarr-python is absent, which is also how the unit tests do it. +""" + +import pytest + +pytest.importorskip("zarr", minversion="3.0.8") +``` + +Create `lib/iris/tests/integration/zarr/fixtures.py`: + +```python +# Copyright Iris contributors +# +# This file is part of Iris and is released under the BSD license. +# See LICENSE in the root of the repository for full licensing details. +"""A version 3 store carrying the CF features a real one does. + +Modelled on the NOAA GFS archive at dynamical.org, whose properties the Zarr +I/O design names as the ones worth exercising: version 3 `dimension_names`, a +sharded codec, a scalar grid-mapping variable, a two-dimensional auxiliary +coordinate, and the base64 `_FillValue` that released xarray writes for every +floating dtype. A cut-down copy of the real store replaces this. +""" + +import base64 +import struct + +import numpy as np +import zarr + +SHAPE = (2, 4, 6, 5) +DIMENSIONS = ("time", "level", "latitude", "longitude") + + +def float_fill_value(value): + """Return `value` in the encoding released xarray writes for a float.""" + return base64.b64encode(struct.pack("`_ stores an array as many independently compressed +chunks under a prefix, rather than as one file. Iris reads Zarr version 2 and +version 3 stores natively, through +`zarr-python `_. + +``zarr`` is an **optional dependency**. Install it to load Zarr stores:: + + conda install -c conda-forge zarr + +Iris without it behaves exactly as before, and raises an ``ImportError`` +naming the package if you ask it to load a store. + +Loading +------- + +A store is a directory, or a URL, so pass it where you would pass a file:: + + cubes = iris.load("/data/forecast.zarr") + +Iris recognises a local store by the metadata file at its root - ``zarr.json`` +for version 3, ``.zgroup`` for version 2 - not by the ``.zarr`` suffix, so a +directory that merely has the suffix is not mistaken for one. + +Remote stores +------------- + +A URL with one of the ``s3``, ``gs``, ``az`` or ``abfs`` schemes, ending in +``.zarr``, is passed to zarr-python unaltered:: + + cubes = iris.load("s3://noaa-gfs-bdp-pds/forecast.zarr") + +Iris adds no store library of its own and handles no credentials. Install the +one your scheme needs -- ``s3fs``, ``gcsfs``, ``adlfs`` -- and configure it as +that package documents. zarr-python raises an ``ImportError`` naming the +package if it is missing. + +Groups +------ + +A Zarr store may hold several groups, and they need not describe the same +thing. Iris loads **one** group per call, like xarray, rather than merging +them: ``iris.load`` reads the root group, and a store whose root holds no data +variables raises a message naming the groups that do. + +To read another group, call the Zarr loader directly:: + + from iris.cube import CubeList + import iris.fileformats.zarr + + cubes = CubeList( + iris.fileformats.zarr.load_cubes( + "/data/sentinel.zarr", group="measurements/reflectance/r10m" + ) + ) + +Dimension names +--------------- + +A netCDF dimension has a name, one length, and a meaning shared across the +whole file. A Zarr dimension name is weaker in three ways, and all three are +permitted by the version 3 specification: + +* an array need not name its dimensions at all; +* it may name some and not others; +* the same name in two arrays guarantees nothing -- not even the same length. + +So Iris matches dimensions on the pair *(name, length)*. Two arrays share a +dimension when they name it identically **and** it is the same length in both. +A dimension with no name is **anonymous**, and never matches anything; the +cube simply has a dimension with no coordinate on it, which is an ordinary +Iris cube. A dimension name used at two different lengths is reported with a +warning and links nothing. + +Version 2 stores have no dimension-name field, and carry names in an +``_ARRAY_DIMENSIONS`` attribute instead -- the convention xarray established. +Iris reads it, and does not pass it on to the cube. + +Missing data +------------ + +Zarr applies no masking of its own, so Iris does it, from the CF attributes: +``_FillValue`` and ``missing_value`` for sentinels, +``valid_min``/``valid_max``/``valid_range`` for a range, and +``scale_factor``/``add_offset`` for packed data. + +A Zarr array also has a ``fill_value`` in its metadata. This is **not** a +missing-value declaration: it says what a chunk that was never written reads +back as, and zarr-python defaults it to zero. Iris therefore ignores it as a +mask source on version 3, where it is a required field, and uses it on version +2 only when the array declares no CF attribute -- which is what version 2 +writers meant by it, and what xarray does. + +Chunking +-------- + +Iris chooses Dask chunks from the store's own chunking, as it does for netCDF, +and :data:`~iris.fileformats.cf.loader.CHUNK_CONTROL` works the same way for +both formats. + +One rule is specific to Zarr. A Zarr chunk is the atomic unit of storage: +reading any part of one fetches and decompresses the whole object. So Iris +rounds the computed chunking **up** to a whole multiple of the store's +chunking, never below it. When a single store chunk is already larger than +Dask's target chunk size, the store's layout is the binding constraint on +memory per task, and Iris warns to say so. + +A **sharded** array is chunked on its inner chunks, not its shards. Zarr's +sharding codec is indexed, so a reader fetches only the inner chunks it needs. + +zarr-python fetches chunks concurrently, under its own +``zarr.config`` setting ``async.concurrency``. Iris does not change it. Dask's +worker count and that concurrency multiply, so a large Dask pool over a remote +store can open far more connections than either number suggests; if you are +rate-limited, lower one of them. + +Attributes +---------- + +A Zarr attribute value may be any JSON literal, including a nested object or +an array of arrays, which CF cannot express at any rank. Iris carries such a +value onto the cube unchanged and does not interpret it. See +:issue:`7288` for the wider question of how the CF attribute model should map +onto JSON. + +.. note:: + + A cube carrying such an attribute **cannot be saved as netCDF**, because + netCDF attribute values are scalars, strings and one-dimensional arrays. + The failure comes from the netCDF library and does not name the attribute; + see :issue:`7329`. Delete the attribute before saving:: + + del cube.attributes["stac_discovery"] + +NCZarr +------ + +Iris has long been able to read and write Zarr through netCDF-c's NCZarr +driver, by way of a URL fragment:: + + iris.load("file:///data/forecast.zarr#mode=nczarr,file") + +That is unchanged, and is still how you reach NCZarr. NCZarr implements Zarr +version 2 and writes scalar variables as length-one arrays; the native reader +described above handles both versions and is what a plain path or URL now +uses. +``` + +Check `:issue:` is the role `common_links.inc` provides; if it is not, write +the full URL. + +- [ ] **Step 2: List it** + +In `docs/src/user_manual/section_indexes/general.rst`, add to the toctree, +keeping the alphabetical order the list is in: + +```rst + ../explanation/zarr_io +``` + +- [ ] **Step 3: Rewrite the S3 page's premise** + +`docs/src/user_manual/tutorial/s3_io.rst` says S3 is unsupported and +recommends a FUSE mount. That is still true for netCDF and no longer true for +Zarr. Change the sphinx-needs summary and add a section near the top, before +"TL;DR": + +```rst +Zarr stores are accessed directly +--------------------------------- + +Everything below concerns **files** -- netCDF, PP, GRIB -- which Iris reaches +through a file-system path. A :ref:`Zarr store ` is different: Iris +passes an ``s3://`` URL straight to zarr-python, which reads the objects it +needs over the network:: + + cubes = iris.load("s3://my-bucket/data.zarr") + +That needs ``s3fs`` installed and configured, and no mount. If your data is in +Zarr, prefer this to the FUSE approach described below. +``` + +and amend the paragraph beginning "While it would be possible for Iris to +support S3 access directly" to say that this is what the Zarr path now does, +and that the limitation it describes applies to the file formats. + +- [ ] **Step 4: Check the metadata and build** + +```bash +pre-commit run check-docs-page-metadata +cd docs && make html-noplot +``` + +Expected: the hook passes, and the build is warning-free. Warnings are errors +in the standard build, so a bad cross-reference fails CI. + +- [ ] **Step 5: Read the generated API pages** + +`sphinxcontrib-apidoc` writes one page per non-private module from its +docstring, so `iris.fileformats.zarr` and `iris.fileformats.zarr.loader` get +pages from what Tasks 1 and 10 wrote. Open them and read them: + +```bash +cd docs && make html-noplot +python -m webbrowser src/_build/html/generated/api/iris/fileformats/zarr.html +``` + +`docs/AGENTS.md` gotcha 5 is the reason this is a step rather than an +assumption: a re-exporting `__init__.py` has three failure modes, two of them +silent. Check that `load_cubes` appears on the package page — it is in +`__all__`, which is what autodoc needs — and that it is described once, not +twice. + +- [ ] **Step 6: Run the doctests** + +The page has no `>>>` examples — the blocks above are `::` literals, not +doctests, deliberately, because none of them can run without a store or a +network. If you add a `>>>` example, run: + +```bash +cd docs/src && make doctest +``` + +- [ ] **Step 7: Commit** + +Run §5.0, then: + +```bash +git add docs/src +git commit -m "Document Zarr loading + +The page a user comparing formats will look for, beside the netCDF one: +what a store is, how a remote one is reached, what a Zarr dimension name +does and does not guarantee, why the array fill value is not a +missing-value flag, and why chunking rounds up. The S3 page no longer +begins by saying this is impossible." +``` + +--- + +### Task 15: the spec, the full-suite diff, and the pull requests + +**Files:** +- Modify: `docs/superpowers/specs/2026-09-21-zarr-io-design.md` +- Modify: `docs/superpowers/plans/2026-10-13-zarr-io-pr4.md` (this file) + +**Interfaces:** +- Consumes: everything. +- Produces: a preview pull request on the fork now, the real one against + `brownfield` when PR 3 merges, and a spec that tells the next reader the + truth. + +Steps 1–5 run now, on the `zarr-io-pr-3` base. **Step 6 runs later**, when +#7316 merges, and is the only step that touches `brownfield`. + +- [ ] **Step 1: Run the full suite, twice** + +```bash +echo "$OVERRIDE_TEST_DATA_REPOSITORY" +git stash list # expect empty + +git switch --detach zarr-io-pr-3 +pytest -n auto lib/iris/tests -q 2>&1 | tail -5 > /tmp/baseline.txt + +git switch - +pytest -n auto lib/iris/tests -q 2>&1 | tail -5 > /tmp/branch.txt +diff /tmp/baseline.txt /tmp/branch.txt +``` + +The baseline is `zarr-io-pr-3`, **not** `brownfield`: this branch's changes are +the ones made on top of PR 3, and measuring against `brownfield` would fold +PR 3's ten commits into the comparison and attribute their effects here. Step 6 +repeats the comparison against `brownfield` after the rebase, which is where +that question gets its own answer. + +Expected: the branch adds passes and adds no failures. Compare **like with +like** — a tier run says nothing about a full-suite one — and read the **skip** +count as well as the failures. A skip that is new on the branch is a test that +never ran. + +- [ ] **Step 2: Update the spec's progress record** + +In `docs/superpowers/specs/2026-09-21-zarr-io-design.md`: + +- §12.1: set PR 4's state to `Built, awaiting PR 3` and say that the work is + complete on a branch based on `zarr-io-pr-3`. The state becomes `In review` + and gains a number in Step 6, when the real pull request opens — a fork + preview has no number worth recording in a document that outlives it. +- The header table's **Progress** and **Next action** rows. +- §12.3: move **Q2** to §12.4 with today's date and the answer from F3 — + fixed-length unicode arrays read as themselves on both versions, verified + against zarr-python 3.4.1, so no special handling is needed. +- §12.4: append two entries, dated, for the decisions this pull request took + that change what the spec says: + + ```markdown + **2026-10-13 — during PR 4** + + - `CFDataset.format` lands in PR 4, not PR 5. `ZarrDataset` cannot answer + `ZARR_V2` versus `ZARR_V3` without it, and PR 4 merges first. + `supports_dtype` is a saver question and stays in PR 5. §5's PR 4 and PR 5 + entries are amended to match. + - The shared `CFDataset` contract is split into a read half and a write + half. PR 4 implements the Zarr read path only, so the write half is + unrunnable against it; one class with half of it skipped would hide an + incomplete implementation behind a skip, which is the opposite of what §6 + asks a shared body to do. PR 6 adds the write half for Zarr. + ``` + +- §5, PR 4 and PR 5 entries: move `CFDataset.format` from PR 5's sentence to + PR 4's list of generic-code changes. + +- [ ] **Step 3: Record what the implementation taught you** + +Add a section to this plan, below §2, titled **"What changed during +implementation"**, listing every step whose code differed from what is written +here and why. The plan ships with the pull request, so a reader comparing the +two must not have to guess which one is true. + +- [ ] **Step 4: Open the preview pull request, on the fork** + +The base branch of a pull request has to live in the repository the pull +request is raised against, and `zarr-io-pr-3` lives on the fork. Raising this +against `SciTools/iris:brownfield` now would therefore show PR 3's ten commits +as part of PR 4. Instead the preview is raised fork-internally, where +`zarr-io-pr-3` *is* available as a base and the diff is exactly this pull +request's work: + +```bash +git push -u origin zarr-io-pr-4 +gh pr create --repo trexfeathers/iris --base zarr-io-pr-3 --head zarr-io-pr-4 \ + --draft --title "Zarr I/O PR 4: Zarr loading" --body-file /tmp/pr4-body.md +``` + +No labels: `Agentic` and `Type: Feature Branch` are SciTools labels and may not +exist on the fork. They go on in Step 6. + +Know what this preview is and is not. It **is** a rehearsal — the real diff, +the real body, somewhere a reviewer can be sent by link, and a place to +self-review before anyone else reads it. It is **not** the pull request: it +does not appear in SciTools' list, a `Part of #6979` in it does not reach the +issue, its CI runs under the fork without the upstream's secrets and may not +run at all, and any review comment left on it stays behind when Step 6 opens +the real one. Do not solicit review on it that you would mind re-soliciting. + +Write the body now and keep the file: Step 6 reuses it unchanged, which is the +point of writing it this early. + +The body opens `(posted by Claude Opus)` before anything else, per +`AGENTS.md`, which **replaces** any trailing attribution — except the +`🤖 Generated with Claude Code` line the harness asks for, which stays at the +end. Never hard-wrap it. It must cover: + +- **What this does.** `iris.load("store.zarr")` and + `iris.load("s3://bucket/store.zarr")` return cubes, for version 2 and + version 3 stores. +- **Part of #6979.** Not `Closes` — a closing keyword only fires on a merge + into the default branch, and the real pull request targets `brownfield`. The + line is written the same way in the preview, where it is inert: a reference + from a fork does not reach the parent repository's issue. +- **The changelog fragment is deferred to the merge-back**, deliberately, per + spec §5. Say so, so the omission does not read as an oversight. +- **Behaviour changes**, all three, each with what used to happen: a local + `.zarr` directory raised `IsADirectoryError`; an `s3://` URL raised + `ValueError("Iris cannot handle the URI scheme")`; and `CFVariable.spans` + now matches `(name, size)` pairs, which no netCDF file can observe and which + the test suite asserts on both backends. +- **Public API added**: `iris.fileformats.zarr`, `iris.io.load_stores`, + `CFDataset.format`, `CFDataset.encoding`, `CFDatasetVariable.encoding`. +- **The spec amendments** from Step 2, each in one line, so a reviewer who + knows the spec is not surprised. +- **What is not here**: saving, real-world fixtures, benchmarks, xarray as a + test dependency, UGRID meshes from Zarr. +- **How to review it**: the task order in §3 of this plan is the reading + order, and Tasks 8 and 12 are the two a netCDF-only reviewer should check. +- **The dependency on #7316.** This branch is cut from `zarr-io-pr-3` and + calls four names that #7316 introduces. Say, in the preview, that it will be + rebased onto `brownfield` before the real pull request opens; in the real + one, say that it was. + +- [ ] **Step 5: Commit the spec and plan updates** + +Run §5.0, then: + +```bash +git add docs/superpowers +git commit -m "Record PR 4 in the spec, and what implementation changed + +Closes open question Q2: fixed-length unicode arrays read as themselves on +both Zarr versions, so #6961's note describes the specification rather than +zarr-python. Logs the two decisions that move work between pull requests: +format lands here rather than with the saver, and the shared dataset +contract splits into read and write halves." +``` + +- [ ] **Step 6: When #7316 merges — rebase, re-verify, raise the real pull + request** + +This step is deferred by definition. Everything above is done and pushed; this +is what turns a rehearsal into a pull request, and none of it can happen until +PR 3 is on `brownfield`. + +```bash +git fetch upstream +git log --oneline -1 upstream/brownfield # confirm #7316 is actually in +git rebase --onto upstream/brownfield zarr-io-pr-3 zarr-io-pr-4 +``` + +The three-argument form moves **only** this branch's own commits: everything +after `zarr-io-pr-3` is replayed onto `brownfield`, and PR 3's commits are left +behind because they are already there. A plain `git rebase upstream/brownfield` +would try to replay them too. + +Expect conflicts in `docs/superpowers/specs/2026-09-21-zarr-io-design.md` +alone, and only if PR 3 changed §12 on its way through review. The code should +not conflict: PR 3 moved the CF loader, and this branch calls it rather than +touching it. + +Then re-verify, because a rebase is a rewrite and nothing on the old commits +was tested in this arrangement: + +```bash +# The two tasks that read PR 3's code most closely. +pytest lib/iris/tests/unit/fileformats/zarr/test__decode.py \ + lib/iris/tests/unit/fileformats/zarr/dataset/test_read_data.py -q + +# Then the whole suite, now against brownfield - the comparison Step 1 +# deliberately could not make. +git switch --detach upstream/brownfield +pytest -n auto lib/iris/tests -q 2>&1 | tail -5 > /tmp/baseline-brownfield.txt +git switch - +pytest -n auto lib/iris/tests -q 2>&1 | tail -5 > /tmp/branch-final.txt +diff /tmp/baseline-brownfield.txt /tmp/branch-final.txt +``` + +Update the spec's §12.1 state from `Built, awaiting PR 3` to `In review` with +the number, once there is one. Then: + +```bash +git push --force-with-lease origin zarr-io-pr-4 +gh pr create --repo SciTools/iris --base brownfield \ + --title "Zarr I/O PR 4: Zarr loading" --body-file /tmp/pr4-body.md +gh pr edit --add-label "Agentic" --add-label "Type: Feature Branch" +gh pr close --repo trexfeathers/iris \ + --comment "(posted by Claude Opus) + +Superseded by SciTools/iris#, now that #7316 has merged and this branch sits on brownfield." +``` + +The body is Step 4's file, with its #7316 paragraph changed from *will be +rebased* to *was rebased*. Close the preview rather than leaving it: two pull +requests showing the same work, one of them stale, is how a reviewer ends up +commenting on the wrong one. + +--- + +## 6. Verification + +What "done" is measured against, and how to measure it. + +### 6.1 The green check, per task + +Every task ends with §5.0: the test-data path confirmed, `pre-commit` clean on +the paths touched, and the task's own test tier passing. No task's commit is +made with `--no-verify`. + +### 6.2 The full suite, against the base + +Run once at the start of implementation and once at the end, on the same +machine, with `OVERRIDE_TEST_DATA_REPOSITORY` set and verified: + +```bash +pytest -n auto lib/iris/tests -q +``` + +The base is `zarr-io-pr-3` while the branch sits there, and `brownfield` after +Task 15's Step 6 rebases it. Both comparisons get made, in that order: the +first is the only one that isolates this pull request's effect, and the second +is the one the eventual reviewer sees. + +Record `passed`, `skipped`, `failed` and `errors` for both. The branch must +add passes, add no failures, and **add no skips**. A new skip is a test that +never ran, and the usual causes — a missing optional dependency, a missing +`iris-test-data` — are exactly the ones this pull request introduces a new +instance of. + +`zarr` must be installed in the environment the final run uses, or every test +this pull request adds is a skip and the comparison is meaningless. + +### 6.3 Without zarr + +The other half of spec §6's "Iris without zarr installed must show no new +failures". In an environment with zarr removed: + +```bash +python -c "import iris; print(iris.__version__)" +pytest -n auto lib/iris/tests -q +``` + +Expected: Iris imports; the Zarr unit and integration packages skip as whole +directories; nothing fails. If a clean environment is not available, state +that in the pull request rather than claiming the check was made. + +### 6.4 The documentation build + +```bash +pre-commit run check-docs-page-metadata --all-files +cd docs && make html +``` + +Expected: no warnings. The standard build uses `-W --keep-going`, so any new +warning fails CI. + +### 6.5 The claims this pull request makes, and what checks each + +| Claim | Checked by | +|---|---| +| A netCDF file loads exactly as it did | §6.2's diff, plus `test_spans.py` running the same suite on both backends (Task 8) | +| `_ARRAY_DIMENSIONS` does not reach a cube | Task 3 Step 9, and Task 13 Step 4 end to end | +| A version 3 array's zeros are not masked | Task 6 Step 5 (the rule), Task 10 Step 7 (a loaded cube) | +| A version 2 array's fill value *is* a mask source | Task 6 Step 5, Task 10 Step 7 | +| A CF `_FillValue` beats the storage one | Task 6 Step 5, Task 10 Step 7 | +| A nested attribute reaches the cube, and saving it as netCDF fails loudly | Task 10 Step 7, documented in Task 14 | +| Both `_FillValue` encodings read | Task 6 Steps 1–2, Task 13 Step 2 | +| A sharded array is read by the inner chunk | Task 7 Step 9, on bytes fetched through an instrumented store | +| The chunking tiles the store grid | Task 7 Steps 1–4, Task 13 Step 3 | +| The dask cache key goes stale when the array does | Task 7 Step 8 | +| An anonymous dimension matches nothing | Task 8 Steps 3–4 | +| A group with no data variables says so | Task 10 Step 2 | +| A `.zarr`-suffixed directory that is not a store is not claimed | Task 11 Step 1 | +| NCZarr still reaches NCZarr | Task 11 Steps 1 and 7 | +| A glob matching files and stores works | Task 12 Step 2 | +| Iris imports without zarr | Task 1 Steps 5–6, §6.3 | +| The native reader agrees with a second writer | Task 13 Step 4, against NCZarr | + +### 6.6 What is deliberately not checked here + +- **Against xarray.** Spec §6 adds xarray as a test dependency in PR 6, with + the saver it validates. Nothing in PR 4 produces a store for another reader + to open. +- **Against a real published store.** PR 7. +- **Over a network.** Spec §6: "No test touches the network. The URL path is + exercised by checking that the right store is constructed, not by fetching." +- **Performance.** The ASV benchmarks are PR 7. +- **A masked cube round-tripping its mask.** Spec §6 lists it among the fill + value cases; it is a save test, and it catches a saver that sets the storage + `fill_value` without also writing the `_FillValue` attribute. PR 6. +- **Saving a nested attribute to netCDF.** Task 10 pins that it fails, not + what the message says: fixing the message means changing the netCDF saver + ([#7329](https://github.com/SciTools/iris/issues/7329)), and PR 5 relocates + that code unchanged. + +--- + +## 7. Definition of done + +- [ ] Every task's checkboxes are ticked, and every commit was made with + `pre-commit` clean and its tier of tests passing. +- [ ] `pytest -n auto lib/iris/tests` adds passes, adds no failures and adds + no skips against `zarr-io-pr-3`, with `zarr` installed and + `OVERRIDE_TEST_DATA_REPOSITORY` set (§6.2) — and again against + `brownfield` after the rebase. +- [ ] Iris imports and its suite runs with `zarr` absent (§6.3), or the pull + request says the check could not be made. +- [ ] `cd docs && make html` is warning-free (§6.4). +- [ ] `iris.load("store.zarr")` works for a version 2 and a version 3 store, + and `iris.fileformats.zarr.load_cubes(store, group=...)` reads a + subgroup. +- [ ] `requirements/py3{12,13,14}.yml` carry `zarr >=3.0.8` and the lock files + are regenerated — or the pull request asks a maintainer to regenerate + them. +- [ ] The spec's §12.1 records PR 4 as `Built, awaiting PR 3`; §12.3 and §12.4 + are updated, and §5's PR 4 and PR 5 entries reflect where `format` + landed (Task 15 Step 2). +- [ ] This plan carries a "What changed during implementation" section, or + states that nothing did. +- [ ] The preview pull request is open on `trexfeathers/iris`, based on + `zarr-io-pr-3`, in draft, with the body Step 6 will reuse. + +Then, once #7316 has merged (Task 15, Step 6 — the rest of this list can be +complete without it): + +- [ ] The branch is rebased onto `brownfield` with + `git rebase --onto upstream/brownfield zarr-io-pr-3`, and Task 6's and + Task 9's tests pass on the rebased commits. +- [ ] The pull request is open against `SciTools/iris:brownfield`, labelled + `Agentic` and `Type: Feature Branch`, says `Part of #6979`, opens + `(posted by Claude Opus)`, and states that its changelog fragment is + deferred to the merge-back. +- [ ] The preview pull request is closed, pointing at the real one. +- [ ] The spec's §12.1 records PR 4 as `In review`, with its number. + diff --git a/docs/superpowers/specs/2026-09-21-zarr-io-design.md b/docs/superpowers/specs/2026-09-21-zarr-io-design.md index 9545fe1da1..298e80e62c 100644 --- a/docs/superpowers/specs/2026-09-21-zarr-io-design.md +++ b/docs/superpowers/specs/2026-09-21-zarr-io-design.md @@ -7,14 +7,14 @@ | | | |---|---| | **Phase** | In delivery | -| **Progress** | PRs 1 and 2 merged into `brownfield` ([#7298](https://github.com/SciTools/iris/pull/7298), [#7303](https://github.com/SciTools/iris/pull/7303)); PR 3 in review ([#7316](https://github.com/SciTools/iris/pull/7316)); merge-back not started — see §12.1 | -| **Next action** | PR 2.5 and PR 4 — see §12.1 | +| **Progress** | PRs 1, 2, 2.5, 3 and 5 merged into `brownfield`; PR 4 in review ([#7336](https://github.com/SciTools/iris/pull/7336)); merge-back not started — see §12.1 | +| **Next action** | PR 4's review, then PR 6 — see §12.1 | | **Blocked on** | Nothing | | **Branch** | `zarr-io-design` on `bjlittle/iris`, targeting `SciTools/iris:brownfield` | | **Delivery** | Nine pull requests into `brownfield`, then one merge-back into `main` — §5 | | **Resolves** | SciTools/iris#6977, #6979, #6980 — at merge-back, not before | -**Created:** 2026-09-21 · **Last updated:** 2026-10-16 (see §12.6) +**Created:** 2026-09-21 · **Last updated:** 2026-10-20 (see §12.6) **Baseline:** `brownfield` at `b0ab90b98`, Iris `3.17.0.dev4` **Verified against:** zarr-python 3.4.0, numcodecs 0.16.5, Python 3.13, dask 2026.7.1, xarray 2026.7.0 @@ -303,7 +303,7 @@ class CFDatasetVariable(ABC): size: int fill_value: Any | None chunking: tuple[int, ...] | None # None when the store is unchunked - encoding: Mapping[str, Any] # backend storage settings; see below + storage_settings: Mapping[str, Any] # backend storage settings; see below attributes: MutableMapping[str, Any] # materialised once def __getitem__(self, keys) -> np.ndarray: ... @@ -323,12 +323,12 @@ class CFDataset(ABC): closed: bool # F3 variables: Mapping[str, CFDatasetVariable] dimensions: Mapping[str, int] - encoding: Mapping[str, Any] # backend store settings; see below + storage_settings: Mapping[str, Any] # backend store settings; see below attributes: MutableMapping[str, Any] def create_dimension(self, name: str, size: int | None) -> None: ... def create_variable(self, name, dtype, dimensions=(), *, - fill_value=None, **encoding) -> CFDatasetVariable: ... + fill_value=None, **storage_settings) -> CFDatasetVariable: ... def supports_dtype(self, dtype: np.dtype) -> bool: ... # see below def sync(self) -> None: ... def finalise(self) -> None: ... # one-shot; NOT part of close() @@ -367,7 +367,7 @@ this store hold this dtype?", so that is what the interface offers. for every dtype Zarr has a data type for. The error message that follows names `format`, so the user still reads `NETCDF3_CLASSIC`. -#### `format` and `encoding` +#### `format` and `storage_settings` `format` names the concrete format an implementation provides — `"NETCDF4"`, `"NETCDF3_CLASSIC"`, `"ZARR_V2"`, `"ZARR_V3"`. Some parts of the loader and @@ -377,7 +377,7 @@ the class is exactly the type test the rule above forbids. On a variable it repeats its dataset's value, for the same reason `location` does: so a variable can answer for itself without a back-reference. -`encoding` is the open-ended bag of backend storage settings: what +`storage_settings` is the open-ended bag of backend storage settings: what `create_variable` was given on write, and what the store actually holds on read — `zlib`, `complevel`, `chunksizes` for netCDF; `compressors`, `chunks`, `shards`, `zarr_format` for Zarr. Its keys are backend-specific by nature and @@ -389,7 +389,7 @@ that is the abstraction leaking at exactly the point it is supposed to hold. It overlaps `chunking` deliberately and they are not duplicates. `chunking` is load-bearing: it is typed, it is guaranteed present on every implementation, -and §4.4's chunk calculation reads it. `encoding["chunks"]` is a backend's own +and §4.4's chunk calculation reads it. `storage_settings["chunks"]` is a backend's own spelling of its own settings, which generic code must never depend on. Three members exist only to keep multi-process writing reachable later, and @@ -428,12 +428,12 @@ The split that matters is `attributes` versus everything else. Today cannot tell. After this change, CF attributes are only ever reached through `.attributes`, and properties are named fields. -`create_variable` takes `**encoding` against the general rule about keyword +`create_variable` takes `**storage_settings` against the general rule about keyword passthrough, because the accepted keys are backend-specific by nature (`zlib`/`complevel`/`chunksizes` for netCDF, `compressors`/`chunks`/`shards` for Zarr). Each implementation documents its own accepted keys, and `cf/saver.py` never constructs them. What it is given is retained and surfaced -on `CFDatasetVariable.encoding`, so create-time configuration is readable +on `CFDatasetVariable.storage_settings`, so create-time configuration is readable afterwards rather than write-only. ### 4.3 `CFVariable.__getattr__` and backwards compatibility @@ -571,7 +571,30 @@ whose entries are names or `None`: A `dimension_names` whose length is not `ndim` *is* a malformation — the specification says "must" — so it is ignored with an `IrisCfLoadWarning` -naming the array, and the array loads with anonymous dimensions. +naming the array, and the array loads with anonymous dimensions. zarr-python +enforces the length match on **read** as well as on write, so this case +reaches Iris only through a version 2 store: `_ARRAY_DIMENSIONS` is a plain +attribute with no such check, where a version 3 array in the same shape is +rejected before `open_group` ever returns it. The check in the reader is +written for both versions regardless, as defensive code that costs nothing +and happens to be exercised by only one of them. + +**Two attributes are storage conventions rather than CF metadata, and are +withheld from `attributes` for that reason.** `_ARRAY_DIMENSIONS` is version +2's own carrier for the dimension names above — `dimensions` already reads +it, so leaving it in `attributes` too would put it on every cube loaded from +a version 2 store. `_NCProperties` is netCDF-c's own bookkeeping stamp, +written onto the root group of every store its NCZarr driver produces; +netCDF4-python already hides it from `Dataset.ncattrs()`, so it has never +reached a cube loaded through netCDF, and Zarr has no equivalent convention +to hide it by. Left in, it would surface only on cubes loaded from an +NCZarr-written store, so the same data would produce cubes that differ by a +provenance string depending on which writer made the store. `_ARRAY_DIMENSIONS` +is withheld from an array's own `attributes`, where it lives; `_NCProperties` +is withheld from the dataset's, where it lives. Neither is withheld from the +other kind of node, because neither ever appears there: `_ARRAY_DIMENSIONS` is +not a group attribute and `_NCProperties` is not an array or subgroup +attribute. **The identity that links two variables is the pair `(name, size)`, and an anonymous dimension has no identity.** One rule covers every case above: @@ -595,6 +618,21 @@ pairs, and must treat `None` as unmatchable. PR 4 makes that change and tests it on both backends, since it is reachable only from Zarr but must not disturb netCDF. +**The same rule decides which cube dimensions a coordinate is attached to.** +`spans` is only the first half of the question. Once a coordinate is found to +span the data variable, the loading rules have to say *which* of the data +variable's dimensions it covers, and they did that by name alone — +`cf_var.dimensions.index(name)`, in `_add_dimension_coordinate`, +`_add_auxiliary_coordinate`, `_add_cell_measure` and `_add_ancil_var`. Given a +data variable whose dimensions are `('d', 'd')` at sizes `(2, 3)` and a +three-point coordinate on `d`, `spans` says yes and `.index('d')` answers 0, +so the coordinate is rejected as the wrong length and lands in +`LOAD_PROBLEMS` — with the axis it belongs on uniquely identifiable. The +identity rule is therefore a function, `_dimension_identities(cf_variable)` in +`cf/_variables.py`, used by `spans` and by one shared +`helpers._common_data_dims`: one rule, not two that can drift. netCDF cannot +observe the difference, its dimension names being unique within a group. + **`CFDataset.dimensions` is the names that are used consistently.** It maps name to size, as it does for netCDF, and on Zarr it is assembled from the arrays. A name that appears at two different sizes is not an error and is not @@ -705,9 +743,14 @@ carries `stac_discovery`, `other_metadata` and `processing_history` as nested objects **[verified, two independent producers]**. Iris carries such values onto the cube unchanged, does not interpret them, and -does not write them back through the netCDF saver. This is the provisional -answer; the durable one needs the attribute-model discussion in -SciTools/iris#7288 (§10). +does not write them back through the netCDF saver. A cube carrying one +therefore cannot be saved as netCDF: the attempt fails inside netCDF4 with a +`TypeError` naming the attribute — `illegal data type for attribute +b'stac_discovery', ..., got O` — but not the cube, since checking attribute +content before saving and raising a message that does both is +[#7329](https://github.com/SciTools/iris/issues/7329), out of scope here +(§4.5, §9). This is the provisional answer; the durable one needs the +attribute-model discussion in SciTools/iris#7288 (§10). #### Masking and unpacking @@ -718,6 +761,16 @@ netCDF4 applies `_FillValue`/`missing_value` masking, shared because the netCDF path gets the behaviour from the C library and has nothing to reuse. +`missing_value` may be a **vector**. CF section 2.5.1 makes it "a scalar or +vector containing values indicating missing data", and netCDF4 masks on every +element of one, so a store and a file carrying the same attribute have to mask +the same values. `_FillValue` is not read that way: the same section makes it +a scalar, and a vector there warns and is ignored. Read as a scalar, a vector +`missing_value` fails twice over — the declared sentinels do not mask, and on +version 2 the variable then looks as though it declared no CF attribute at +all, so the storage fill value becomes the sentinel and every valid zero is +masked. + Decoding is applied as **explicit dask graph layers** over the raw array — `da.map_blocks(_mask_and_scale, raw, ...)` — not hidden inside an opaque `__getitem__`. Each rule is a small pure function of `(block, parameters)`, @@ -862,20 +915,66 @@ target (128 MiB by default) it shrinks below it **[verified]**: | `(6000, 6000)` | 275 MiB | `(6000, 6000)` | `(2000, 6000)` | 3 × re-read | Subdividing buys nothing here. Peak memory per task is still a whole -decompressed chunk, because that is what the store hands back; only the -I/O and decompression multiply. So on the Zarr path the loader rounds the -computed chunking **up** to a whole multiple of the store chunk, never below -it, and emits an `IrisLoadWarning` naming the variable when the store's own -chunking exceeds the dask target — the store's layout is then the binding -constraint and the user should know. An explicit `CHUNK_CONTROL` setting still -wins, since that is a deliberate instruction, but it warns on the same terms. - -This trap is not new and not Zarr-specific: `NetCDFDataProxy` reopens the -`Dataset` on every `__getitem__`, so HDF5's chunk cache is discarded between -tasks and the netCDF path re-reads sub-chunks too. It simply costs far more -over a network than over a page cache. Changing the netCDF path is out of scope -here — the relocation PRs must not alter behaviour — but it is recorded as an -open question in §12.3. +decompressed chunk, because that is what the store hands back; only the I/O +and decompression multiply. + +**Iris does not correct for this, and warns instead.** The loader sizes +chunks with `_optimum_chunksize` and stops there, for both storage backends. +When a stored chunk is larger than the dask target it emits an +`IrisLoadWarning` naming the variable, and `CHUNK_CONTROL` is how a caller +who wants the other trade takes it. + +An earlier revision of this section had the Zarr path round the computed +chunking **up** to a whole multiple of the store chunk. That was removed on +review (#7336), on grounds of proportionality rather than mechanism: + +- The mechanism was never in doubt, and measurement sharpened it. Against a + 33.6 MiB store chunk subdivided into quarters, aligned chunking held lower + peak RSS at every worker count above one (305 MiB against 446 MiB at four + workers) and ran in roughly half the time, with 1.0 fetches per store chunk + against 3.2 **[verified]**. +- But alignment only ever changes the answer in the shrink case above. + Wherever `_optimum_chunksize` expands, it already lands on a whole multiple + and alignment is a no-op **[verified]**. So the whole of it existed to + override one decision. +- And that decision is one the netCDF path makes too. Both backends pass the + file's own chunking into the same `_optimum_chunksize`, and netCDF has + always accepted the subdivision — `NetCDFDataProxy` additionally reopens the + `Dataset` on every `__getitem__`, discarding HDF5's chunk cache between + tasks. Correcting it on one path only would make two formats behave + differently over physics they share, and would make `CHUNK_CONTROL` mean + different things depending on what was loaded. + +The house preference decides it: give the user the tools to adapt their own +case rather than second-guess it. The warning is what makes that fair — it +says the situation exists and names the tool — and it now covers both +backends, from `iris.fileformats.cf.loader`. The remaining asymmetry is only +that this costs far more over a network than over a page cache, which is an +argument for the user reaching for `CHUNK_CONTROL` sooner, not for Iris +choosing on their behalf. + +**Chunks are sized on the decoded dtype, not the stored one.** A packed +variable's values are unpacked block by block, on the way out of the graph, so +what a task holds is the decoded block: an `int8` array with a `scale_factor` +that unpacks to `float64` is eight times its stored size the moment the decode +layer runs. Sized on the stored dtype, a `(512, 512, 512)` `int8` array gets +one chunk spanning the array — 128 MiB stored, 1 GiB decoded, against a 128 +MiB target. The netCDF path reaches the same answer from the other end: its +data proxy declares the decoded dtype, and `as_lazy_data` sizes on the array +it is given. `_optimum_chunksize` therefore takes the decoded dtype, which for +an unpacked variable is the stored one. + +`CHUNK_CONTROL.as_dask()` is not exempt. That setting hands sizing to Dask, +and `da.from_array` sizes on the array it is given — which is the stored one, +so the same multiplier survives there. The Zarr path therefore calls +`dask.array.core.normalize_chunks("auto", shape, dtype=decoded_dtype)` +itself and passes the result on with every dimension fixed. The algorithm and +the configured target are still Dask's; only the dtype it budgets against is +supplied rather than inferred. Fixing the dimensions is what stops +`as_lazy_data` re-sizing the result on the stored dtype and undoing it — and +is the thing to watch in that branch, since Iris's own sizing or the store +alignment creeping back in on top would make `as_dask()` mean something other +than what it says. `CFDatasetVariable.chunking` returns the store's own chunk shape, which `cf/loader.py` uses exactly as it uses `chunking()` today. @@ -904,9 +1003,10 @@ between cannot, so the block is materialised in full. On a `shape=(1024,)`, | shard `(1024,)` | **yes** | **6,148** — the whole shard | 2,052 of those bytes are the shard index and 32 are the data actually wanted. -So `chunking` returns `Array.chunks`, and the alignment rule above aggregates -*upward* into whole multiples of it — never down, and never to a shard -boundary. This is not academic: the NOAA GFS fixture is sharded (§7). +So `chunking` returns `Array.chunks`, never a shard boundary — the read unit +is the inner chunk, and that is what the chunking policy and the warning above +are both sized against. This is not academic: the NOAA GFS fixture is sharded +(§7). Shard-sized blocks remain a *write* requirement, for a different reason and in one direction only. §4.5 keeps them there. @@ -1386,6 +1486,17 @@ so it is consulted when no file handle is available. It matches when: The split is local-versus-remote, and the two tests are not alternatives for the same target. +**The element answers `None` when a file handle is supplied**, so the +specification cannot match a target that opened as a file. `requires_fh = +False` means "consult me without a handle", not "consult me with one". Two +things go wrong otherwise. A store is a directory or a URL, so a target with a +handle is settled already. And the caller that opened the file — +`iris.io._load_files` — passes `pathlib.Path(fn).name`, a basename rather than +a path, which resolves against the working directory: a NIMROD file named +`weather` was classified as Zarr whenever the working directory happened to +contain a `weather/zarr.json`, the priority-5 Zarr specification being +consulted before priority-3 NIMROD. + **Do not add a third rule matching any path that ends in `.zarr`.** Joined by "or" it swallows the remote rule whole, leaving the scheme test unreachable, and it is the wrong rule locally in any case. Reading the marker file is what @@ -1419,14 +1530,24 @@ rather than once in `load_files` and again in the scheme dispatch. The one thing this costs is that the file-or-store decision has to happen *after* glob expansion and *before* format dispatch, because `iris.load` must -still accept `"data/*"` matching a mix of `.nc` files and `.zarr` stores. -`expand_filespecs` already runs first and already returns concrete paths, so -`_generate_cubes` partitions what comes back and calls `load_files` and -`load_stores` with their own share. Neither routine learns about the other's -kind of target. +still accept `"data/*"` matching a mix of `.nc` files and `.zarr` stores. And +it must happen only **once**: `expand_filespecs` keys its own glob cache by +the spec string, so expanding an already-expanded list collapses two +identical paths matched by two different patterns into one. `load_files` +cannot simply receive the partitioned, already-expanded list, because its +public signature promises a glob-accepting `filenames` to every other caller. +So the expansion stays behind that signature — `load_files` expands once and +delegates to a private `_load_files`, which expects a list already expanded +— and `_generate_cubes` calls `_load_files` directly, with the list it has +already expanded for itself and partitioned between `_load_files` and +`load_stores`. Neither routine learns about the other's kind of target. The duplicated dispatch in `netcdf/ugrid_load.py:156-183` is updated in step -with `loading.py`, so `iris.mesh.load_meshes` accepts Zarr stores too. +with `loading.py`, so `iris.mesh.load_meshes` accepts Zarr stores too. It also +has to accept the Zarr *handler*, not only the Zarr scheme: the format spec it +gets back for a store names `iris.fileformats._load_zarr`, and a test against +`nc_loader.load_cubes` alone sends every store down the "ignore this" branch +while the scheme dispatch above says the store was fine. **NCZarr is unaffected, and native Zarr is already the default.** No precedence rule is needed to make it so, because the native reader and the @@ -1596,7 +1717,7 @@ Carries the §6 read tests for the rules that are easiest to get wrong: the version-aware masking split, both `_FillValue` encodings, the inner-chunk read unit on a sharded array, and the cache-key staleness and sensitivity cases. -Also carries two changes to generic code that only Zarr can exercise: +Also carries three changes to generic code that only Zarr can exercise: - `dimensions` becomes `tuple[str | None, ...]`, and `CFVariable.spans` and its three overrides match on `(name, size)` pairs with an anonymous @@ -1604,9 +1725,28 @@ Also carries two changes to generic code that only Zarr can exercise: construction, and the tests say so on both backends. - `encoding` on `CFDataset` and `CFDatasetVariable` (§4.2), reporting what the opened store actually holds. +- `format` on `CFDatasetVariable` (§4.2), naming the concrete Zarr version so + the version-aware masking split (§4.4) can tell `ZARR_V2` from `ZARR_V3`. + `CFDataset.format` arrived with PR 5, which merged first; this pull request + adds the variable's half and `ZarrDataset`'s answer to both it and + `supports_dtype` (§12.4). + +**UGRID meshes load too**, through +`iris.fileformats.netcdf.ugrid_load`. Nothing in that module is netCDF-specific +— it reads a `CFReader`, and `CFReader` over a `ZarrDataset` answers the same +questions — so the Zarr loader calls into it rather than growing a second copy. +The mesh block that was inline in `netcdf/loader.py` becomes +`_prepare_mesh_coords` and `_attach_mesh_coords`, two functions because the +MeshCoords have to be built *before* `_load_cube` and attached *after* it: built +after, `helpers._add_unused_attributes` has already taken the data variable's +`mesh` and `location` attributes and left them on the cube as ordinary +attributes. One netCDF detail survives the move: the "mesh could not be found" +message keeps being raised on the `iris.fileformats.netcdf` logger rather than +`ugrid_load`'s own, which sits at `INFO` and would drop it. After this pull request, `iris.load("store.zarr")` and -`iris.load("s3://bucket/store.zarr")` work for version 2 and version 3 stores. +`iris.load("s3://bucket/store.zarr")` work for version 2 and version 3 stores, +mesh data included. ### PR 5 — Relocate the CF saver @@ -1715,6 +1855,14 @@ Every fragment credits `` :user:`claude` `` per `changelog/AGENTS.md`. and becomes visible to users only here. The merge-back checks it builds clean against `main` with `-W`, since the two branches can drift. +**Issues to raise, not to act on.** One deliberately deferred follow-up, +raised *here* rather than when it was found, so that nobody starts on it while +the programme's pull requests are still in flight and depending on each other: + +| What | Why it waits | +|---|---| +| Unify the cube-building loops in `netcdf/loader.py` and `zarr/loader.py`, and with them the format dispatch `load_meshes` duplicates (§2.3) | Raised by @pp-mo reviewing PR 4. Since PR 4 gave the Zarr loader the same mesh handling, the two bodies differ only in how the dataset is opened. The duplicated dispatch is the same defect wearing a different hat, and is the only reason `netcdf/ugrid_load.py` names Zarr: the mesh code it shares with the Zarr loader names no format at all. A `TODO` in that function points here. The refactor is internal — neither `load_cubes` signature moves — so it is safe to do later and disruptive to do now | + **Ordering rationale.** Refactor and feature alternate so that no feature pull request is large, and every relocation is a separate, skimmable diff. Tests land before the rewrite they protect. Loading lands before saving because it is @@ -2059,8 +2207,8 @@ nothing else, and can land beside any of the others. | 2 | `CFDataset`, and the CF variable classes rewritten against it | Merged | [#7303](https://github.com/SciTools/iris/pull/7303) | | 2.5 | Deprecate `CFVariable.__getattr__` | Merged | [#7333](https://github.com/SciTools/iris/pull/7333) | | 3 | Relocate the CF loader | Merged | [#7316](https://github.com/SciTools/iris/pull/7316) | -| 4 | Zarr loading | Not started | — | -| 5 | Relocate the CF saver | In review | [#7338](https://github.com/SciTools/iris/pull/7338) | +| 4 | Zarr loading | In review | [#7336](https://github.com/SciTools/iris/pull/7336) | +| 5 | Relocate the CF saver | Merged | [#7338](https://github.com/SciTools/iris/pull/7338) | | 6 | Zarr saving | Not started | — | | 7 | Real-world test data and benchmarks | Not started | — | | 8 | The terms-of-art sweep | Not started | — | @@ -2072,9 +2220,11 @@ Companion work outside `SciTools/iris`: |---|---|---|---| | `SciTools/iris-test-data` | Cut-down NOAA GFS (v3) and ESA EOPF (v2) fixtures for PR 7 | Not started | — | -States are `Not started`, `Drafted`, `In review`, `Changes requested`, -`Merged` or `Abandoned`. Record the pull request number and its state as soon -as it is opened, not when it merges. +States are `Not started`, `Drafted`, `Built, awaiting PR N` (work complete on +a branch based on another pull request not yet merged, with no number of its +own worth recording), `In review`, `Changes requested`, `Merged` or +`Abandoned`. Record the pull request number and its state as soon as it is +opened, not when it merges. There is deliberately no `Closes` column and no changelog-fragment column: both obligations belong to the merge-back, not to the feature-branch pull @@ -2106,7 +2256,6 @@ closing keywords live (§5). | # | Question | Provisional answer | Tracked | Blocks | |---|---|---|---|---| | Q1 | How should the CF attribute model map onto JSON, on write and on read? | JSON-native both ways (§4.4, §4.5) | #7288, §10 | Nothing. Changing it touches only the attribute conversion in `zarr/_dataset.py` | -| Q2 | Are ` Optional[list[int]]: + """Return the dimensions of `cf_data_var` that `cf_var` shares with it. + + Two variables share a dimension when they name it identically *and* it is + the same length in both - the ``(name, size)`` rule + :meth:`iris.fileformats.cf.CFVariable.spans` decides spanning by. The bare + name is not enough: a Zarr store may give one variable two dimensions of + the same name at two different lengths, and ``dimensions.index(name)`` + would then answer with the first of them whichever one was meant. No + netCDF file can tell the difference, a netCDF dimension name being unique + within its group and carrying one size. + + Returns ``None`` rather than an empty list when nothing is shared, because + that is what the ``Cube.add_*`` methods take for "no dimensions". + """ + # `_dimension_identities` rather than a pairing written out here, so that + # the rule deciding a variable spans the data variable and the rule + # attaching it to its dimensions are one rule, not two that can drift. + data_identities = _dimension_identities(cf_data_var) + data_dims = [ + data_identities.index(identity) + for identity in _dimension_identities(cf_var) + if identity in data_identities + ] + return data_dims or None + + def _add_dimension_coordinate( engine: Engine, cf_coord_var: cf.CFCoordinateVariable, @@ -1480,15 +1510,9 @@ def _add_dimension_coordinate( assert engine.cube is not None assert engine.cube_parts is not None - # Determine the name of the dimension/s shared between the CF-netCDF - # data variable and the coordinate being built. - common_dims = [ - dim for dim in cf_coord_var.dimensions if dim in engine.cf_var.dimensions - ] - data_dims = None - if common_dims: - # Calculate the offset of each common dimension. - data_dims = [int(engine.cf_var.dimensions.index(dim)) for dim in common_dims] + # Determine which dimension/s of the CF-netCDF data variable the + # coordinate being built covers. + data_dims = _common_data_dims(cf_coord_var, engine.cf_var) if hasattr(coord, "circular") and data_dims is not None: # Appease MyPy. The check itself uses duck typing to avoid any @@ -1639,15 +1663,9 @@ def _add_auxiliary_coordinate( assert engine.cube is not None assert engine.cube_parts is not None - # Determine the name of the dimension/s shared between the CF-netCDF data variable - # and the coordinate being built. - coord_dims = cf_coord_var.dimensions - datavar_dims = engine.cf_var.dimensions - common_dims = [dim for dim in coord_dims if dim in datavar_dims] - data_dims = None - if common_dims: - # Calculate the offset of each common dimension. - data_dims = [engine.cf_var.dimensions.index(dim) for dim in common_dims] + # Determine which dimension/s of the CF-netCDF data variable the + # coordinate being built covers. + data_dims = _common_data_dims(cf_coord_var, engine.cf_var) engine.cube.add_aux_coord(coord, data_dims) @@ -1723,13 +1741,9 @@ def _add_cell_measure( cf_var = engine.cf_var cube = engine.cube - # Determine the name of the dimension/s shared between the CF-netCDF data - # variable and the coordinate being built. - common_dims = [dim for dim in cf_cm_var.dimensions if dim in cf_var.dimensions] - data_dims = None - if common_dims: - # Calculate the offset of each common dimension. - data_dims = [cf_var.dimensions.index(dim) for dim in common_dims] + # Determine which dimension/s of the CF-netCDF data variable the cell + # measure being built covers. + data_dims = _common_data_dims(cf_cm_var, cf_var) # Add it to the cube cube.add_cell_measure(cell_measure, data_dims) @@ -1795,13 +1809,9 @@ def _add_ancil_var( cf_var = engine.cf_var cube = engine.cube - # Determine the name of the dimension/s shared between the CF-netCDF data variable - # and the AV being built. - common_dims = [dim for dim in cf_av_var.dimensions if dim in cf_var.dimensions] - data_dims = None - if common_dims: - # Calculate the offset of each common dimension. - data_dims = [cf_var.dimensions.index(dim) for dim in common_dims] + # Determine which dimension/s of the CF-netCDF data variable the ancillary + # variable being built covers. + data_dims = _common_data_dims(cf_av_var, cf_var) # Add it to the cube cube.add_ancillary_variable(av, data_dims) diff --git a/lib/iris/fileformats/cf/_reader.py b/lib/iris/fileformats/cf/_reader.py index bfe9ba5f85..2b9caceb43 100644 --- a/lib/iris/fileformats/cf/_reader.py +++ b/lib/iris/fileformats/cf/_reader.py @@ -39,15 +39,14 @@ before anything that can raise, so the destructor is safe even if ``__init__`` fails part way. -This module is coupled to :mod:`iris.fileformats.netcdf`: opening a file means -building a :class:`~iris.fileformats.netcdf._dataset.NetCDFDataset`. The rest -of the load path works against the format-agnostic -:class:`~iris.fileformats.cf.dataset.CFDataset` / -:class:`~iris.fileformats.cf.dataset.CFDatasetVariable` interface, so a Zarr -store can be read by the same machinery once a -:class:`~iris.fileformats.cf.dataset.CFDataset` implementation exists for it --- see §4.2 of the native Zarr I/O design, -``docs/superpowers/specs/2026-09-21-zarr-io-design.md``. +``CFReader`` takes one of three things. A path opens a netCDF file, because +netCDF is the default a bare filename means. An open netCDF4 dataset, or an +emulator of one such as ncdata passes, is wrapped. And a +:class:`~iris.fileformats.cf.dataset.CFDataset` of any format is used as it +stands, which is how a Zarr store reaches this machinery: its loader opens +the store and hands it over, so nothing here has to know that Zarr exists. +Only the first case mentions netCDF, and only because something has to be +the default. """ @@ -78,6 +77,7 @@ CFUGridMeshVariable, _CFFormulaTermsVariable, ) +from .dataset import CFDataset #: Supported dimensionless vertical coordinate reference surface/phemomenon #: formula terms. Ref: [CF] Appendix D. @@ -133,7 +133,12 @@ class CFReader: def __init__(self, file_source, warn=False, monotonic=False): # Ensure safe operation for destructor, should init fail. self._own_file = False - if isinstance(file_source, str): + if isinstance(file_source, CFDataset): + # Already open, in whatever format: use it, don't own it. The + # caller that opened it is responsible for closing it. + self._dataset = file_source + self._filename = file_source.location + elif isinstance(file_source, str): # Create from filepath : open it + own it (=close when we die). if not urlparse(file_source).scheme: self._filename = Path(file_source).expanduser() diff --git a/lib/iris/fileformats/cf/_variables.py b/lib/iris/fileformats/cf/_variables.py index 1b0135cc38..91f229bd21 100644 --- a/lib/iris/fileformats/cf/_variables.py +++ b/lib/iris/fileformats/cf/_variables.py @@ -76,6 +76,65 @@ # NCZarr stores scalar variables as size-1 arrays on this pseudo-dimension. _NCZARR_SCALAR_DIMENSION = "_scalar_" + +class _DimensionIdentity: + """One dimension of a CF variable, as something to match on. + + A named dimension compares on ``(name, size)``: two variables share it + when they name it identically *and* it is the same length in both. A name + reused at two lengths links nothing, which is the only reading that + cannot produce a wrong cube. + + An unnamed dimension is **anonymous**, and matches nothing: not a named + dimension, not another anonymous one, and not itself as seen from a + second variable. There is nothing to match on, so claiming a match would + be invention. + + Zarr's version 3 core specification makes ``dimension_names`` optional and + allows a null entry, so a conforming store can present this and it must + not be an error. No netCDF variable ever has one. + """ + + def __init__(self, name: str | None, size: int): + self.name = name + self.size = size + + def __eq__(self, other) -> bool: + if not isinstance(other, _DimensionIdentity): + return NotImplemented + if self.name is None or other.name is None: + return self is other + return self.name == other.name and self.size == other.size + + def __hash__(self) -> int: + if self.name is None: + return object.__hash__(self) + return hash((self.name, self.size)) + + def __repr__(self) -> str: + if self.name is None: + return f"" + return f"" + + +def _dimension_identities(cf_variable) -> tuple[_DimensionIdentity, ...]: + """Return one hashable identity per dimension `cf_variable` spans. + + A function rather than a method, because the netCDF loading rules decide + which cube dimensions to attach a coordinate to, and have to decide it by + the rule that let the coordinate through :meth:`CFVariable.spans` in the + first place - see + :func:`iris.fileformats._nc_load_rules.helpers._common_data_dims`. + + Relies on the :class:`~iris.fileformats.cf.dataset.CFDatasetVariable` + invariant that ``dimensions`` and ``shape`` have the same length. + """ + return tuple( + _DimensionIdentity(name, size) + for name, size in zip(cf_variable.dimensions, cf_variable.shape) + ) + + # NetCDF variable attributes handled by the netCDF4 module and # therefore automatically classed as "used" attributes. _CF_ATTRS_IGNORE = set(["_FillValue", "add_offset", "missing_value", "scale_factor"]) @@ -202,6 +261,11 @@ def spans(self, cf_variable): Determine whether the dimensionality of this variable is a subset of the specified target variable. + A dimension matches on the pair *(name, size)*, not on the bare + name: see :func:`_dimension_identities`. A name that is the same in + both variables but a different size links nothing, and a Zarr + store's anonymous dimensions never match anything, even each other. + Note that, by default scalar variables always span the dimensionality of the target variable. @@ -219,8 +283,9 @@ def spans(self, cf_variable): if self._is_scalar(): return True - result = set(self.dimensions).issubset(cf_variable.dimensions) - return result + source = _dimension_identities(self) + target = _dimension_identities(cf_variable) + return set(source).issubset(target) def __eq__(self, other): # CF variable names are unique. @@ -236,7 +301,11 @@ def __hash__(self): @property def dimensions(self) -> tuple: - """The names of the dimensions this variable spans, in order.""" + """The names of the dimensions this variable spans, in order. + + An entry is ``None`` where the store names no dimension - only a + Zarr store does this; no netCDF variable has an anonymous dimension. + """ return tuple(self.cf_data.dimensions) @property @@ -546,8 +615,8 @@ def spans(self, cf_variable): # Scalar variables always span the target variable. result = True if not self._is_scalar(): - source = self.dimensions - target = cf_variable.dimensions + source = _dimension_identities(self) + target = _dimension_identities(cf_variable) # Ignore the bounds extent dimension. result = set(source[:-1]).issubset(target) or set(source[1:]).issubset( target @@ -622,8 +691,8 @@ def spans(self, cf_variable): # Scalar variables always span the target variable. result = True if not self._is_scalar(): - source = self.dimensions - target = cf_variable.dimensions + source = _dimension_identities(self) + target = _dimension_identities(cf_variable) # Ignore the climatology extent dimension. result = set(source[:-1]).issubset(target) or set(source[1:]).issubset( target @@ -945,8 +1014,8 @@ def spans(self, cf_variable): # Scalar variables always span the target variable. result = True if not self._is_scalar(): - source = self.dimensions - target = cf_variable.dimensions + source = _dimension_identities(self) + target = _dimension_identities(cf_variable) # Ignore label string length dimension. result = set(source[:-1]).issubset(target) or set(source[1:]).issubset( target diff --git a/lib/iris/fileformats/cf/dataset.py b/lib/iris/fileformats/cf/dataset.py index 75b017700f..2982708a2e 100644 --- a/lib/iris/fileformats/cf/dataset.py +++ b/lib/iris/fileformats/cf/dataset.py @@ -148,6 +148,17 @@ def name(self) -> str: def location(self) -> str: """The path or URL of the dataset holding this variable.""" + @property + @abstractmethod + def format(self) -> str: + """The concrete format of the dataset holding this variable. + + Repeated here from the dataset so that a variable can answer for + itself, as :attr:`location` is. For messages and diagnostics only: + generic code asks for a capability, never for a format. + + """ + @property @abstractmethod def dimensions(self) -> tuple: @@ -178,6 +189,18 @@ def fill_value(self) -> Any: def chunking(self) -> tuple | None: """The variable's storage chunk shape, or ``None`` when unchunked.""" + @property + @abstractmethod + def storage_settings(self) -> Mapping: + """The storage settings this variable holds, in the backend's spelling. + + ``zlib``, ``complevel``, ``chunksizes`` for netCDF; ``chunks``, + ``shards``, ``codecs``, ``zarr_format`` for Zarr. Readable, never + interpreted: no generic code may branch on a key. The typed, + guaranteed-present member is :attr:`chunking`. + + """ + @property @abstractmethod def attributes(self) -> MutableMapping: @@ -276,6 +299,11 @@ def format(self) -> str: """ + @property + @abstractmethod + def storage_settings(self) -> Mapping: + """The store-wide settings this store holds, in the backend's spelling.""" + @property @abstractmethod def closed(self) -> bool: @@ -307,11 +335,11 @@ def create_dimension(self, name: str, size: int | None) -> None: @abstractmethod def create_variable( - self, name: str, dtype, dimensions=(), *, fill_value=None, **encoding + self, name: str, dtype, dimensions=(), *, fill_value=None, **storage_settings ) -> "CFDatasetVariable": """Create and return a new variable. - ``**encoding`` is storage-specific: ``zlib``, ``complevel`` and + ``**storage_settings`` is storage-specific: ``zlib``, ``complevel`` and ``chunksizes`` for netCDF; ``compressors``, ``chunks`` and ``shards`` for Zarr. diff --git a/lib/iris/fileformats/cf/loader.py b/lib/iris/fileformats/cf/loader.py index c342a04869..7133f10503 100644 --- a/lib/iris/fileformats/cf/loader.py +++ b/lib/iris/fileformats/cf/loader.py @@ -38,6 +38,8 @@ import threading import warnings +import dask.config +import dask.utils import numpy as np from iris.aux_factory import ( @@ -219,6 +221,38 @@ def _get_actual_dtype(cf_var): _LAZYVAR_MIN_BYTES = 5000 +def _warn_if_storage_chunk_binds(cf_var) -> None: + """Warn when one stored chunk is larger than Dask's target chunk size. + + Storage reads a chunk whole and decompresses it whole, for netCDF and for + Zarr alike. So when Dask's target is the smaller of the two, the stored + chunking is what sets peak memory per task, and the chunking Iris picks + divides stored chunks rather than tiling them - fetching each one once per + division. + + Iris does not correct for this. Which trade is right depends on the case, + and :data:`CHUNK_CONTROL` is how a caller says so; this warning exists so + that they know there is something to say. See section 4.6 of + ``docs/superpowers/specs/2026-09-21-zarr-io-design.md``. + """ + storage_chunks = cf_var.cf_data.chunking + if storage_chunks is None: + return + + target = dask.utils.parse_bytes(dask.config.get("array.chunk-size")) + stored_bytes = int(np.prod(storage_chunks)) * _get_actual_dtype(cf_var).itemsize + if stored_bytes <= target: + return + + message = ( + "One stored chunk is larger than Dask's target chunk size, so the " + "stored chunking sets peak memory per task and Iris's chunking will " + "divide it. Use iris.fileformats.cf.loader.CHUNK_CONTROL to choose " + f"the chunking yourself: {cf_var.cf_name}" + ) + warnings.warn(message, category=iris.warnings.IrisLoadWarning) + + def _chunks_from_chunk_control(cf_var): """Return the ``(chunks, dims_fixed)`` pair for a lazy array of this variable. @@ -292,6 +326,11 @@ def _chunks_from_chunk_control(cf_var): dims_fixed[i_dim] = True if dims_fixed is None: dims_fixed = [dims_fixed] + if not any(dims_fixed): + # Nothing pinned, so Iris is choosing and the caller may want to know + # that the store's own layout has a say. A caller who has pinned + # anything has already taken that decision. + _warn_if_storage_chunk_binds(cf_var) return chunks, tuple(dims_fixed) diff --git a/lib/iris/fileformats/netcdf/_dataset.py b/lib/iris/fileformats/netcdf/_dataset.py index ebaa08888c..925223aad7 100644 --- a/lib/iris/fileformats/netcdf/_dataset.py +++ b/lib/iris/fileformats/netcdf/_dataset.py @@ -23,6 +23,7 @@ """ from collections.abc import Iterator, Mapping, MutableMapping +from types import MappingProxyType from typing import Any import warnings @@ -114,7 +115,9 @@ def __repr__(self) -> str: class NetCDFDatasetVariable(CFDatasetVariable): """One variable of a netCDF file, presented through the CF interface.""" - def __init__(self, variable, location: str, *, write_lock_factory=None): + def __init__( + self, variable, location: str, format: str, *, write_lock_factory=None + ): """Wrap ``variable``, a thread-safe netCDF variable wrapper. Parameters @@ -126,6 +129,8 @@ def __init__(self, variable, location: str, *, write_lock_factory=None): file's own. This class reports whatever the wrapper reports. location : str The path or URL of the dataset this variable belongs to. + format : str + The netCDF format of the dataset this variable belongs to. write_lock_factory : callable, optional Returns the lock shared by every variable of one dataset. Called by :meth:`write_handle`. A callable rather than a lock, so that a @@ -134,6 +139,7 @@ def __init__(self, variable, location: str, *, write_lock_factory=None): """ self._variable = variable self._location = location + self._format = format self._write_lock_factory = write_lock_factory self._attributes = _NetCDFAttributes(variable) @@ -147,6 +153,11 @@ def location(self) -> str: """The path or URL of the dataset holding this variable.""" return self._location + @property + def format(self) -> str: + """The netCDF format of the file holding this variable.""" + return self._format + @property def dimensions(self) -> tuple: """The names of the dimensions this variable spans, in order.""" @@ -180,6 +191,18 @@ def chunking(self) -> tuple | None: return None return tuple(chunks) + @property + def storage_settings(self) -> Mapping: + """The netCDF storage settings this variable holds. + + Read back off the file, so a variable created with `zlib=True` reports + the compression netCDF4 actually applied rather than the keyword it + was given. + + """ + filters = self._variable.filters() or {} + return MappingProxyType({**filters, "chunksizes": self.chunking}) + @property def attributes(self) -> MutableMapping: """The variable's CF attributes.""" @@ -551,6 +574,11 @@ def format(self) -> str: assert self._dataset is not None return self._dataset.file_format + @property + def storage_settings(self) -> Mapping: + """The store-wide netCDF settings. Only the format is one.""" + return MappingProxyType({"format": self.format}) + @property def closed(self) -> bool: """Whether the file has been released, by this object or by its owner. @@ -575,6 +603,7 @@ def _materialised_variables(self) -> dict[str, "NetCDFDatasetVariable"]: name: NetCDFDatasetVariable( variable, self._location, + self._dataset.file_format, write_lock_factory=self._write_lock_factory, ) for name, variable in self._dataset.variables.items() @@ -611,21 +640,24 @@ def create_dimension(self, name: str, size: int | None) -> None: self._dataset.createDimension(name, size) def create_variable( - self, name: str, dtype, dimensions=(), *, fill_value=None, **encoding + self, name: str, dtype, dimensions=(), *, fill_value=None, **storage_settings ) -> NetCDFDatasetVariable: """Create and return a new variable. - ``**encoding`` reaches :meth:`netCDF4.Dataset.createVariable` + ``**storage_settings`` reaches :meth:`netCDF4.Dataset.createVariable` unaltered: ``compression``, ``zlib``, ``complevel``, ``shuffle``, ``chunksizes``, ``least_significant_digit`` and the rest. """ assert self._dataset is not None variable = self._dataset.createVariable( - name, dtype, tuple(dimensions), fill_value=fill_value, **encoding + name, dtype, tuple(dimensions), fill_value=fill_value, **storage_settings ) wrapped = NetCDFDatasetVariable( - variable, self._location, write_lock_factory=self._write_lock_factory + variable, + self._location, + self._dataset.file_format, + write_lock_factory=self._write_lock_factory, ) # Register this wrapper, so that a later lookup by name returns it # rather than a second wrapper with its own attribute cache. diff --git a/lib/iris/fileformats/netcdf/loader.py b/lib/iris/fileformats/netcdf/loader.py index 9a8c40b2ad..2eb780d7ec 100644 --- a/lib/iris/fileformats/netcdf/loader.py +++ b/lib/iris/fileformats/netcdf/loader.py @@ -27,7 +27,6 @@ """ from collections.abc import Iterable -from functools import partial import warnings from iris._deprecation import warn_deprecated @@ -44,9 +43,8 @@ # module rather than by value, so that assigning to it here still works. DEBUG = False -# Get the logger (shared logger for all in 'iris.fileformats.netcdf') and -# _bytecoding_datasets, needed by NetCDFDataProxy just below. -from . import _bytecoding_datasets, logger +# Get _bytecoding_datasets, needed by NetCDFDataProxy just below. +from . import _bytecoding_datasets # An expected part of the public loader API, but includes thread safety # concerns so is housed in _thread_safe_nc. @@ -152,15 +150,13 @@ def load_cubes(file_sources, callback=None, constraints=None): """ # Deferred import to avoid circular imports. - from iris.cube import Cube - from iris.fileformats._nc_load_rules.helpers import _add_or_capture from iris.fileformats.cf import CFReader from iris.io import run_callback - from iris.loading import LoadProblems from .ugrid_load import ( - _build_mesh_coords, + _attach_mesh_coords, _meshes_from_cf, + _prepare_mesh_coords, ) # Create a low-level data-var filter from the original load constraints, if they are suitable. @@ -186,65 +182,14 @@ def load_cubes(file_sources, callback=None, constraints=None): # Deliver only selected results. continue - # cf_var-specific mesh handling, if a mesh is present. - # Build the mesh_coords *before* loading the cube - avoids - # mesh-related attributes being picked up by - # _add_unused_attributes(). - mesh_name = None - mesh = None - mesh_coords, mesh_dim = [], None - mesh_name = cf_var.attributes.get("mesh") - if mesh_name is not None: - try: - mesh = meshes[mesh_name] - except KeyError: - message = ( - f"Mesh '{mesh_name}' - " - f"referenced by variable: '{cf_var.cf_name}' - " - "could not be found in file." - ) - logger.debug(message) - - if mesh is not None: - # Unconventional 'split' usage of _add_or_capture - - # attribute handling means MeshCoords need to be built - # BEFORE loading the Cube. - capture_kwargs = dict( - cf_var=cf.cf_group.meshes[mesh_name], - # MeshCoords are an Iris concept; the best fallback we - # have is to capture the CF Mesh. - destination=LoadProblems.Problem.Destination( - iris_class=Cube, - identifier=cf_var.cf_name, - ), - ) - - def _build_mesh_coords_inner(): - nonlocal mesh_coords - nonlocal mesh_dim - mesh_coords, mesh_dim = _build_mesh_coords(mesh, cf_var) - - def _add_mesh_coords(coords_and_dim): - coords, dim = coords_and_dim - for coord in coords: - cube.add_aux_coord(coord, dim) - - # MeshCoords part 1. - _ = _add_or_capture( - build_func=partial(_build_mesh_coords_inner), - add_method=partial(lambda built: None), - **capture_kwargs, - ) + # MeshCoords are built before the cube and attached after it: + # see _prepare_mesh_coords for why the two cannot be one step. + pending_mesh_coords = _prepare_mesh_coords(cf, meshes, cf_var) cube = _load_cube(engine, cf, cf_var, cf.filename) - if mesh is not None: - # MeshCoords part 2. - _ = _add_or_capture( - build_func=partial(lambda: (mesh_coords, mesh_dim)), - add_method=partial(_add_mesh_coords), - **capture_kwargs, - ) + if pending_mesh_coords is not None: + _attach_mesh_coords(pending_mesh_coords, cube) # Process any associated formula terms and attach # the corresponding AuxCoordFactory. diff --git a/lib/iris/fileformats/netcdf/ugrid_load.py b/lib/iris/fileformats/netcdf/ugrid_load.py index bbeb033360..d2f46bb779 100644 --- a/lib/iris/fileformats/netcdf/ugrid_load.py +++ b/lib/iris/fileformats/netcdf/ugrid_load.py @@ -20,6 +20,8 @@ """ +from contextlib import ExitStack +from dataclasses import dataclass from functools import partial from itertools import groupby from pathlib import Path @@ -29,7 +31,11 @@ from iris.common.mixin import CFVariableMixin from iris.config import get_logger from iris.coords import AuxCoord -from iris.io import _is_nczarr_fragment, decode_uri, expand_filespecs +from iris.io import ( + _REMOTE_STORE_SCHEMES, + _categorise_uri, + expand_filespecs, +) from iris.loading import LoadProblems from iris.mesh.components import Connectivity, MeshXY from iris.util import guess_coord_axis @@ -87,8 +93,119 @@ def _meshes_from_cf(cf_reader): return meshes +@dataclass +class _PendingMeshCoords: + """MeshCoords built ahead of their cube, waiting to be attached to it.""" + + coords: list + dim: int | None + capture_kwargs: dict + + +def _prepare_mesh_coords(cf_reader, meshes, cf_var): + """Build the MeshCoords a data variable needs, before its cube exists. + + Unconventional 'split' usage of ``_add_or_capture``: this half builds and + captures, :func:`_attach_mesh_coords` adds. The split is forced by + attribute handling - built after the cube, + ``helpers._add_unused_attributes`` has already taken the variable's + ``mesh`` and ``location`` attributes for cube attributes. + + Parameters + ---------- + cf_reader : iris.fileformats.cf.CFReader + The reader the mesh variables were found in. + meshes : dict + Meshes by name, as returned by :func:`_meshes_from_cf`. + cf_var : iris.fileformats.cf.CFDataVariable + The data variable about to be built into a cube. + + Returns + ------- + _PendingMeshCoords or None + None when `cf_var` names no mesh, or names one this dataset does not + hold. + + """ + # NOTE: dynamic imports avoid circularity : see note with module imports + from iris.cube import Cube + from iris.fileformats._nc_load_rules.helpers import _add_or_capture + + # Not this module's `logger`, which sits at INFO and would drop a debug + # message at source. `TestsMesh.test_missing_mesh` asserts the package + # logger, where this message has always been raised. + from iris.fileformats.netcdf import logger as netcdf_logger + + mesh_name = cf_var.attributes.get("mesh") + if mesh_name is None: + return None + + mesh = meshes.get(mesh_name) + if mesh is None: + message = ( + f"Mesh '{mesh_name}' - " + f"referenced by variable: '{cf_var.cf_name}' - " + "could not be found in file." + ) + netcdf_logger.debug(message) + return None + + pending = _PendingMeshCoords( + coords=[], + dim=None, + capture_kwargs=dict( + cf_var=cf_reader.cf_group.meshes[mesh_name], + # MeshCoords are an Iris concept; the best fallback we have is to + # capture the CF Mesh. + destination=LoadProblems.Problem.Destination( + iris_class=Cube, + identifier=cf_var.cf_name, + ), + ), + ) + + def _stash(coords_and_dim): + pending.coords, pending.dim = coords_and_dim + + _ = _add_or_capture( + build_func=partial(_build_mesh_coords, mesh, cf_var), + add_method=partial(_stash), + **pending.capture_kwargs, + ) + return pending + + +def _attach_mesh_coords(pending, cube): + """Attach MeshCoords prepared by :func:`_prepare_mesh_coords` to their cube. + + Parameters + ---------- + pending : _PendingMeshCoords + The return of :func:`_prepare_mesh_coords`. Its `coords` are empty if + the build failed, in which case the failure is already recorded and + this call adds nothing. + cube : iris.cube.Cube + The cube built from the data variable the MeshCoords were prepared + for. + + """ + # NOTE: dynamic imports avoid circularity : see note with module imports + from iris.fileformats._nc_load_rules.helpers import _add_or_capture + + def _add(coords_and_dim): + coords, dim = coords_and_dim + for coord in coords: + cube.add_aux_coord(coord, dim) + + _ = _add_or_capture( + build_func=partial(lambda: (pending.coords, pending.dim)), + add_method=partial(_add), + **pending.capture_kwargs, + ) + + def load_mesh(uris, var_name=None): - """Load a single :class:`~iris.mesh.MeshXY` object from one or more NetCDF files. + """Load a single :class:`~iris.mesh.MeshXY` from one or more files or stores. Raises an error if more/less than one :class:`~iris.mesh.MeshXY` is found. @@ -96,8 +213,9 @@ def load_mesh(uris, var_name=None): Parameters ---------- uris : str or iterable of str - One or more filenames/URI's. Filenames can include wildcards. Any URI's - must support OpenDAP or NcZarr. + One or more filenames/URI's. Filenames can include wildcards. NetCDF + files and Zarr stores are read; anything else is ignored. Any URI's + must support OpenDAP, NcZarr, or a Zarr remote store scheme. var_name : str, optional Only return a :class:`~iris.mesh.MeshXY` if its var_name matches this value. @@ -117,13 +235,14 @@ def load_mesh(uris, var_name=None): def load_meshes(uris, var_name=None): - r"""Load :class:`~iris.mesh.MeshXY` objects from one or more NetCDF files. + r"""Load :class:`~iris.mesh.MeshXY` objects from files or Zarr stores. Parameters ---------- uris : str or iterable of str - One or more filenames/URI's. Filenames can include wildcards. Any URI's - must support OpenDAP nor NcZarr. + One or more filenames/URI's. Filenames can include wildcards. NetCDF + files and Zarr stores are read; anything else is ignored. Any URI's + must support OpenDAP, NcZarr, or a Zarr remote store scheme. var_name : str, optional Only return :class:`~iris.mesh.MeshXY` that have var_names matching this value. @@ -139,7 +258,7 @@ def load_meshes(uris, var_name=None): # NOTE: no constraints or callbacks supported - these assume they are operating # on a Cube. # NOTE: dynamic imports avoid circularity : see note with module imports - from iris.fileformats import FORMAT_AGENT + from iris.fileformats import FORMAT_AGENT, _load_zarr from iris.fileformats.cf import CFReader import iris.fileformats.netcdf.loader as nc_loader @@ -151,44 +270,61 @@ def load_meshes(uris, var_name=None): # Make a string, Dataset, or other single item, into an iterable. uris = [uris] - def _categorise(decoded: tuple[str, str, str | None]) -> tuple[str, str, str]: - scheme, part, fragment = decoded - category = "nczarr" if _is_nczarr_fragment(fragment) else scheme - if fragment: - part = f"{part}#{fragment}" - return category, scheme, part - # Group collections of uris by their iris handler # Create list of tuples relating schemes to part names. - uri_tuples = sorted(_categorise(decode_uri(uri)) for uri in uris) + uri_tuples = sorted(_categorise_uri(uri) for uri in uris) valid_sources = [] for category, groups in groupby(uri_tuples, key=lambda x: x[0]): # Call each scheme handler with the appropriate URIs if category == "file": filenames = [part for _cat, _scheme, part in groups] sources = expand_filespecs(filenames) - elif category in ["http", "https", "nczarr"]: + elif category in {"http", "https", "nczarr"} | _REMOTE_STORE_SCHEMES: sources = [f"{scheme}:{part}" for _cat, scheme, part in groups] else: message = f"Iris cannot handle the URI scheme: {category}" raise ValueError(message) for source in sources: - if category == "file": + # iris.loading._generate_cubes makes this same file-or-store + # distinction independently, for general cube loading - keep + # both in step. + if category == "file" and not Path(source).is_dir(): with open(source, "rb") as fh: handling_format_spec = FORMAT_AGENT.get_spec(Path(source).name, fh) else: handling_format_spec = FORMAT_AGENT.get_spec(source, None) + # TODO: drop this format dispatch once the netCDF and Zarr + # cube-building loops are unified. This copy, not the mesh code + # below, is the only reason a netCDF-named module names Zarr at + # all. See the merge-back section of + # `docs/superpowers/specs/2026-09-21-zarr-io-design.md`. + # + # `_load_zarr` is compared, never `zarr.load_cubes`: a store's + # presence is what should reach zarr-python, not this comparison. if handling_format_spec.handler == nc_loader.load_cubes: - valid_sources.append(source) + valid_sources.append((source, False)) + elif handling_format_spec.handler == _load_zarr: + valid_sources.append((source, True)) else: - message = f"Ignoring non-NetCDF file: {source}" + message = f"Ignoring file of a format that holds no mesh: {source}" logger.info(msg=message, extra=dict(cls=None)) result = {} - for source in valid_sources: - with CFReader(source) as cf_reader: + for source, is_zarr in valid_sources: + with ExitStack() as stack: + if is_zarr: + # Deferred: a caller that named no Zarr store must not reach + # zarr-python at all. + from iris.fileformats.zarr._dataset import ZarrDataset + + # CFReader takes an open CFDataset or a filename, so the mesh + # machinery below never learns which it was given. + target = stack.enter_context(ZarrDataset(source)) + else: + target = source + cf_reader = stack.enter_context(CFReader(target)) meshes_dict = _meshes_from_cf(cf_reader) meshes = list(meshes_dict.values()) if var_name is not None: diff --git a/lib/iris/fileformats/zarr/__init__.py b/lib/iris/fileformats/zarr/__init__.py new file mode 100644 index 0000000000..3b6306653e --- /dev/null +++ b/lib/iris/fileformats/zarr/__init__.py @@ -0,0 +1,121 @@ +# Copyright Iris contributors +# +# This file is part of Iris and is released under the BSD license. +# See LICENSE in the root of the repository for full licensing details. +"""Native Zarr support, built on zarr-python. + +.. z_reference:: iris.fileformats.zarr + :tags: topic_load_save + + API reference + +Reads Zarr version 2 and version 3 stores, local or remote. Not NCZarr: that +is netCDF-c's own Zarr driver, reached through a URL fragment such as +``file:///path/store.zarr#mode=nczarr,file``, and it stays where it is in +:mod:`iris.fileformats.netcdf`. + +``zarr`` is an optional dependency. Nothing in :mod:`iris.fileformats` imports +this package until a Zarr target is actually loaded, so Iris installs and +imports without it. + +This package shadows the third-party ``zarr`` distribution by name. Python 3 +resolves ``import zarr`` absolutely, so inside these modules it still finds the +third-party package; xarray's ``xarray.backends.zarr`` has the same +arrangement. + +See sections 4.1 and 4.4 of +``docs/superpowers/specs/2026-09-21-zarr-io-design.md``. + +""" + +import importlib.metadata + +from packaging.version import Version + +__all__ = ["load_cubes"] + +#: The version floor ``docs/src/user_manual/explanation/zarr_io.rst`` and +#: ``requirements/py3*.yml`` both promise. +_MINIMUM_VERSION = "3.0.8" + +#: Raised whether ``zarr`` is missing entirely or present below +#: :data:`_MINIMUM_VERSION`. Defined once here, not in +#: :func:`iris.fileformats._load_zarr`, so that the two routes a caller can +#: reach a Zarr load through - ``iris.load`` and calling +#: :func:`load_cubes` directly - produce the same text rather than each +#: carrying its own copy. +_MISSING_MESSAGE = ( + "Unable to load a Zarr store - the 'zarr' package is not " + "installed. Install zarr >=3.0.8, for example with " + "'conda install -c conda-forge zarr'." +) + + +def _check_zarr_version() -> None: + """Raise a clear ``ImportError`` if ``zarr`` is installed below the floor. + + Reads installed-distribution metadata rather than importing ``zarr`` + itself: a zarr-python 2.x ``Group`` has no ``.metadata``, so importing it + only to compare `__version__` would load far enough to reach + :class:`~iris.fileformats.zarr._dataset.ZarrDataset` and fail there + instead, with a bare ``AttributeError``. + + Does nothing if ``zarr`` is not installed at all - :func:`__getattr__` + below reports that case itself, from the import it goes on to attempt. + + Raises + ------ + ImportError + If ``zarr`` is installed below :data:`_MINIMUM_VERSION`. + + """ + try: + installed = importlib.metadata.version("zarr") + except importlib.metadata.PackageNotFoundError: + return + + if Version(installed) < Version(_MINIMUM_VERSION): + raise ImportError(_MISSING_MESSAGE) + + +def __getattr__(name): + """Forward ``load_cubes`` to :mod:`iris.fileformats.zarr.loader`. + + PEP 562, not a plain ``from .loader import load_cubes``: ``loader`` + imports ``._dataset``, which imports ``zarr`` at module level, so an + eager import here would make the dependency load with the package rather + than with a Zarr load - breaking the promise this module's own docstring + makes, that nothing imports ``zarr`` until a Zarr target is actually + loaded. Resist "simplifying" this back to a plain import. + + Parameters + ---------- + name : str + The attribute being looked up. Python calls this only for names that + are not already module attributes. + + Returns + ------- + Any + ``load_cubes``, the only name this forwards. + + Raises + ------ + AttributeError + For any other name. + ImportError + If ``zarr`` is not installed, or is installed below + :data:`_MINIMUM_VERSION`. + + """ + if name != "load_cubes": + raise AttributeError(f"module {__name__!r} has no attribute {name!r}") + + _check_zarr_version() + + try: + from .loader import load_cubes + except ImportError as exception: + raise ImportError(_MISSING_MESSAGE) from exception + + return load_cubes diff --git a/lib/iris/fileformats/zarr/_dataset.py b/lib/iris/fileformats/zarr/_dataset.py new file mode 100644 index 0000000000..8c9d5a5683 --- /dev/null +++ b/lib/iris/fileformats/zarr/_dataset.py @@ -0,0 +1,701 @@ +# Copyright Iris contributors +# +# This file is part of Iris and is released under the BSD license. +# See LICENSE in the root of the repository for full licensing details. +"""The Zarr implementation of the :mod:`iris.fileformats.cf.dataset` interface. + +This is the only module that knows both the CF interface and the zarr-python +API. Everything Zarr-specific the CF layer could want - where dimension names +are recorded, what a chunk is, what the store's own fill value is - either has +a named member on the interface or lives here as a Zarr-only member. + +Three things about Zarr shape this module, and each is a trap if forgotten. + +**A dimension name guarantees less than a netCDF one.** The version 3 core +specification makes ``dimension_names`` optional, allows a null entry, only +*recommends* that names are distinct within an array, and places no +restriction at all on reusing a name across arrays. All of that is conforming +Zarr, so none of it may raise. A dimension with no name is **anonymous**, which +Iris has always had: a cube dimension carrying no coordinate. + +**``Array.fill_value`` is not ``_FillValue``.** It says what unwritten storage +reads back as; it does not say the value is missing. zarr-python defaults it to +zero, so reading it as a missing-value sentinel masks every valid zero in an +array that never declared one. :attr:`ZarrDatasetVariable.fill_value` reports +it as the storage fact it is, and the masking decision is made in +:mod:`iris.fileformats.zarr._decode` from the CF attributes. + +**The read unit is the inner chunk, even when the array is sharded.** Zarr's +sharding codec is indexed, so a reader fetches only the inner chunks it needs. +:attr:`ZarrDatasetVariable.chunking` therefore returns ``Array.chunks``, never +``Array.shards``. + +Chunking is sized by :func:`iris._lazy_data._optimum_chunksize`, the same way +the netCDF path sizes it, and is not then adjusted to the store's own +chunking. Iris warns when a stored chunk is larger than Dask's target, from +:func:`iris.fileformats.cf.loader._chunks_from_chunk_control`, and leaves the +choice to ``CHUNK_CONTROL``. + +See sections 4.2, 4.4 and 4.6 of +``docs/superpowers/specs/2026-09-21-zarr-io-design.md``. + +""" + +from collections.abc import Callable, Iterator, Mapping, MutableMapping +from functools import partial +import json +from types import MappingProxyType +from typing import Any +import warnings + +import dask +import dask.array as da +from dask.array.core import normalize_chunks +import dask.config +import dask.utils +import numpy as np +import zarr + +from iris._lazy_data import _optimum_chunksize, as_lazy_data +from iris.exceptions import TranslationError +from iris.fileformats.cf import loader as cf_loader +from iris.fileformats.cf.dataset import CFDataset, CFDatasetVariable +import iris.warnings + +from . import _decode + +#: What :attr:`CFDataset.format` answers, by the store's Zarr version. +_FORMAT_BY_ZARR_VERSION = {2: "ZARR_V2", 3: "ZARR_V3"} + +#: The version 2 attribute carrying dimension names, as xarray and NCZarr +#: write it. Storage, not a CF attribute, so it is withheld from `attributes`. +_ARRAY_DIMENSIONS = "_ARRAY_DIMENSIONS" + +#: netCDF-c's own bookkeeping attribute, stamped onto the root group of every +#: store it writes (confirmed on NCZarr output in both its `nczarr` and +#: `xarray` URL modes, and never on a subgroup or an array). netCDF4-python +#: hides it from `Dataset.ncattrs()` by convention, so it has never reached a +#: cube loaded through netCDF; Zarr has no equivalent convention, so it must +#: be withheld here for the same reason as `_ARRAY_DIMENSIONS` above - +#: provenance the writer left behind, not CF metadata the user wrote. +_NC_PROPERTIES = "_NCProperties" + + +def _dimension_names(array) -> tuple: + """Return one name or None per dimension of `array`. + + Parameters + ---------- + array : :class:`zarr.Array` + The array to read names from. Version 3 carries them in its metadata; + version 2 carries them in the ``_ARRAY_DIMENSIONS`` attribute, which + is a de-facto convention rather than part of the Zarr specification. + + Returns + ------- + tuple + Of length ``array.ndim``. ``None`` where the store names no dimension. + + Raises + ------ + iris.exceptions.TranslationError + If `array` declares any Zarr version but 2 or 3. zarr-python parses + ``zarr_format`` as a ``Literal`` and rejects such an array before + Iris sees it, so this guards against a future widening there. + + Warns + ----- + IrisCfLoadWarning + If the store names a number of dimensions other than ``array.ndim``. + The version 3 specification says the lengths "must" match, so such a + store is malformed; its names are ignored and the array loads with + anonymous dimensions. + + """ + anonymous = (None,) * array.ndim + if not array.ndim: + return () + + zarr_format = array.metadata.zarr_format + if zarr_format == 3: + names = array.metadata.dimension_names + elif zarr_format == 2: + names = array.attrs.get(_ARRAY_DIMENSIONS) + else: + message = ( + "Iris reads Zarr version 2 and version 3 stores. This array " + f"declares Zarr version {zarr_format!r}: {array.basename}" + ) + raise TranslationError(message) + + if names is None: + return anonymous + + if len(names) != array.ndim: + message = ( + "Ignoring malformed dimension_names, which names " + f"{len(names)} dimensions of a {array.ndim}-dimensional array: " + f"{array.basename}" + ) + warnings.warn(message, category=iris.warnings.IrisCfLoadWarning) + return anonymous + + return tuple(names) + + +def _assemble_dimensions(variables) -> dict: + """Return the dimension sizes the store's arrays agree on. + + Parameters + ---------- + variables : mapping of str to :class:`ZarrDatasetVariable` + Every array in one group. + + Returns + ------- + dict + Name to size, for every name used at exactly one size. Anonymous + dimensions have no name and so are absent. + + Warns + ----- + IrisCfLoadWarning + For each name used at more than one size. Zarr places no restriction + on reusing a name across arrays, so this is conforming and must not + raise; merging the two would produce a wrong cube, so the name is + omitted instead and the arrays that use it keep it in their own + `dimensions`. + + """ + sizes: dict[str, int] = {} + conflicts: dict[str, set[int]] = {} + for variable in variables.values(): + for name, size in zip(variable.dimensions, variable.shape): + if name is None: + continue + if name not in sizes: + sizes[name] = size + elif sizes[name] != size: + conflicts.setdefault(name, {sizes[name]}).add(size) + + for name, found in conflicts.items(): + message = ( + "Ignoring a dimension name used at more than one size, " + f"{sorted(found)}: {name}" + ) + warnings.warn(message, category=iris.warnings.IrisCfLoadWarning) + del sizes[name] + + return sizes + + +class _ZarrAttributes(MutableMapping): + """A Zarr node's attributes as a mapping, less the ones `withhold` names. + + Which attributes those are is the caller's decision, taken where the + mapping is built. Both the constants named there - `_ARRAY_DIMENSIONS` + and `_NC_PROPERTIES` - carry the reason with them. + """ + + def __init__( + self, + node: "zarr.Group | zarr.Array", + *, + withhold: frozenset = frozenset(), + ): + """Materialise the attributes of `node`, a zarr array or group. + + Writing through this mapping writes to `node`, which is the store. + """ + self._node = node + self._withhold = frozenset(withhold) + self._values = { + key: value for key, value in node.attrs.items() if key not in self._withhold + } + + def __getitem__(self, key: str) -> Any: + """Return `key`'s value.""" + return self._values[key] + + def __setitem__(self, key: str, value: Any) -> None: + """Set `key`'s value, here and in the store.""" + if key in self._withhold: + msg = f"Cannot set a storage attribute through `attributes`: {key}" + raise KeyError(msg) + self._node.attrs[key] = value + self._values[key] = value + + def __delitem__(self, key: str) -> None: + """Remove `key`, here and from the store.""" + del self._node.attrs[key] + del self._values[key] + + def __iter__(self) -> Iterator[str]: + """Return an iterator over the attribute names, in store order.""" + return iter(self._values) + + def __len__(self) -> int: + """Return the number of attributes.""" + return len(self._values) + + def __repr__(self) -> str: + """Return a string representation.""" + return f"{self.__class__.__name__}({self._values!r})" + + +class ZarrDatasetVariable(CFDatasetVariable): + """One array of a Zarr store, presented through the CF interface.""" + + def __init__(self, array, location: str, format: str): + """Wrap `array`. + + Parameters + ---------- + array : :class:`zarr.Array` + The array this variable presents. + location : str + The path or URL of the dataset this array belongs to. + format : str + ``ZARR_V2`` or ``ZARR_V3``, as its dataset reports. + + """ + self._array = array + self._location = location + self._format = format + self._dimensions = _dimension_names(array) + self._attributes = _ZarrAttributes( + array, withhold=frozenset({_ARRAY_DIMENSIONS}) + ) + + @property + def name(self) -> str: + """The array's name within its group.""" + return self._array.basename + + @property + def location(self) -> str: + """The path or URL of the dataset holding this variable.""" + return self._location + + @property + def format(self) -> str: + """``ZARR_V2`` or ``ZARR_V3``.""" + return self._format + + @property + def dimensions(self) -> tuple: + """One name per dimension, ``None`` where the store names none.""" + return self._dimensions + + @property + def shape(self) -> tuple: + """The array's shape.""" + return tuple(self._array.shape) + + @property + def dtype(self) -> np.dtype: + """The array's stored data type.""" + return self._array.dtype + + @property + def size(self) -> int: + """The total number of elements in the array.""" + return int(self._array.size) + + @property + def fill_value(self) -> Any: + """What unwritten storage reads back as. **Not** a missing-value flag. + + Version 3 requires this field, and zarr-python defaults it to zero, so + it says nothing about whether a value is missing. The CF + ``_FillValue`` and ``missing_value`` attributes say that, and + :mod:`iris.fileformats.zarr._decode` reads them. + + """ + return self._array.fill_value + + @property + def chunking(self) -> tuple | None: + """The array's inner chunk shape, never its shard shape.""" + chunks = self._array.chunks + return tuple(chunks) if chunks else None + + @property + def storage_settings(self) -> Mapping: + """The Zarr storage settings this array holds.""" + return MappingProxyType( + { + "chunks": self._array.chunks, + "shards": self._array.shards, + "codecs": self._array.metadata.to_dict().get("codecs"), + "zarr_format": self._array.metadata.zarr_format, + } + ) + + @property + def attributes(self) -> MutableMapping: + """The array's CF attributes, less the ones that are really storage.""" + return self._attributes + + def __getitem__(self, keys) -> np.ndarray: + """Return the indexed portion of the array's data.""" + return self._array[keys] + + def __setitem__(self, keys, values) -> None: + """Write `values` into the indexed portion of the array.""" + self._array[keys] = values + + def read_data(self, chunking_policy: Callable[[], tuple]) -> Any: + """Return this array's data, lazily where a lazy array is worth it. + + Parameters + ---------- + chunking_policy : callable + Called with no arguments, returning the ``(chunks, dims_fixed)`` + pair to build a lazy array with. Called only once the result is + known to be lazy. + + Returns + ------- + numpy.ndarray, numpy.ma.MaskedArray or dask.array.Array + Real data for a small array, lazy data otherwise. Masked whenever + the variable declares anything to decode. + + Notes + ----- + There is no data proxy. ``zarr.Array`` is picklable, a chunk is an + independent object in a key-value store so concurrent reads need no + lock, and a store is a namespace rather than an open file handle. CF + decoding is a ``map_blocks`` layer over the raw array rather than + something hidden inside ``__getitem__``, so it is visible in the + graph and testable without a store. + + """ + parameters = _decode.decode_parameters( + self._attributes, + dtype=self.dtype, + storage_fill_value=self.fill_value, + zarr_format=self.zarr_format, + variable_name=self.name, + ) + decoding = _decode.decoding_required(parameters) + + if self.size * self.dtype.itemsize < cf_loader._LAZYVAR_MIN_BYTES: + # A lazy array would cost more memory AND more time to access. + result = np.asanyarray(self._array[...]) + if decoding: + result = _decode.mask_and_scale(result, **parameters) + else: + chunks, dims_fixed = chunking_policy() + # Chunks are sized on the dtype the values will have once decoded, + # not on the one the store holds: a packed int8 that unpacks to + # float64 costs eight times its stored size the moment the decode + # layer runs, and Dask's target chunk size is a target for what one + # task holds. The netCDF path arrives at the same place from the + # other end - its data proxy declares the decoded dtype, and + # `as_lazy_data` sizes on the array it is given. `_unpacked_dtype` + # returns the stored dtype for a variable with no packing, so + # nothing moves in the usual case. + decoded_dtype = parameters["dtype"] + + if chunks == "auto": + # CHUNK_CONTROL.as_dask(): the user asked for Dask's own + # policy, so the store's layout is deliberately not consulted. + # The policy is still Dask's - its algorithm, its configured + # target - but it is called here rather than left to + # `from_array`, which is the only way to hand it the decoded + # dtype. `from_array` would size on the array it is given, and + # the array it is given is the stored one. + chunks = tuple( + sizes[0] + for sizes in normalize_chunks( + "auto", shape=self.shape, dtype=decoded_dtype + ) + ) + else: + if chunks is None: + # What `as_lazy_data` makes of None, done here because this + # path reaches `_optimum_chunksize` first and that does not + # accept it: one chunk spanning the array, divided down + # below. + chunks = self.shape + chunks = _optimum_chunksize( + chunks, shape=self.shape, dtype=decoded_dtype, dims_fixed=dims_fixed + ) + + # Fixed in every dimension, because the sizing above is the final + # word whichever branch produced it: `as_lazy_data` would otherwise + # re-run its own sizing on the stored dtype and undo it. + result = as_lazy_data( + self._array, + chunks=chunks, + meta=np.empty((0,) * self.ndim, dtype=self.dtype), + dims_fixed=(True,) * self.ndim, + cache_key=self.cache_key, + ) + + if decoding: + # `partial` rather than `**parameters` on the `map_blocks` + # call: `dtype` is a keyword `map_blocks` consumes for itself + # rather than forwarding, and `mask_and_scale` needs it too. + # Binding it once here also makes the dtype Dask declares and + # the dtype the layer produces one value rather than two + # statements that could disagree. + result = da.map_blocks( + partial(_decode.mask_and_scale, **parameters), + result, + dtype=decoded_dtype, + meta=np.ma.masked_array(np.empty((0,) * self.ndim, decoded_dtype)), + ) + + return result + + def write_handle(self) -> "ZarrDatasetVariable": + """Return self: a Zarr write target needs no proxy. + + ``zarr.Array`` is picklable and a chunk is an independent object in a + key-value store, so there is no open file handle to avoid holding and + no lock to acquire. + + """ + return self + + def __repr__(self) -> str: + """Return a string representation.""" + return f"{self.__class__.__name__}({self.name!r}, {self.location!r})" + + # Zarr-only members below. + + @property + def array(self): + """The backing :class:`zarr.Array`. Nothing generic may read this.""" + return self._array + + @property + def zarr_format(self) -> int: + """2 or 3. The one Zarr fact that changes how a value is read.""" + return self._array.metadata.zarr_format + + @property + def cache_key(self) -> str: + """An identity for this array that goes stale when the array does. + + `iris._lazy_data.CACHE` is process-wide, so an address alone is not + enough: a store rewritten in place keeps its address and changes its + contents. The metadata document covers shape, chunks, shards, dtype, + fill value, codecs, dimension names and attributes in one value, and + is treated as opaque - listing the parts Iris cares about would go + stale, silently, the first time zarr-python added a field that + mattered. + """ + return json.dumps( + [self._location, self._array.path, self._array.metadata.to_dict()], + sort_keys=True, + ) + + +#: What `mode` a dataset may be opened in. Writing is not implemented. +_READ_MODES = ("r",) + +#: The one message every unimplemented write member raises. +_NO_SAVING = "Saving to Zarr is not supported." + + +class ZarrDataset(CFDataset): + """A Zarr store, or one group of one, presented through the CF interface.""" + + def __init__(self, location, mode: str = "r", *, group: str | None = None): + """Open a Zarr store. + + Parameters + ---------- + location : str or :class:`pathlib.Path` + The store's path or URL. A URL is handed to zarr-python + unaltered; Iris adds no store library and handles no credentials, + so an unreachable scheme fails with zarr-python's own message. A + store object - such as a :class:`zarr.storage.MemoryStore`, which + has no path - is also accepted and handed through unaltered; + `location` then becomes zarr-python's own rendering of it, the + best name available for a store with no address. + mode : str, default="r" + Only ``"r"`` is supported. + group : str, optional + A ``/``-separated path to a group within the store. The root group + by default. Iris reads one group per call and does not walk + nested groups. + + Raises + ------ + NotImplementedError + For any `mode` other than ``"r"``. + FileNotFoundError + If `location` does not exist. + zarr.errors.GroupNotFoundError + If `location` is not a Zarr store, or holds no such `group`. + + """ + # Set first, so that close() is safe if opening fails. + self._group: zarr.Group | None = None + self._closed = False + self._variables: dict[str, ZarrDatasetVariable] | None = None + self._attributes: _ZarrAttributes | None = None + self._dimensions: dict[str, int] | None = None + + if mode not in _READ_MODES: + raise NotImplementedError(f"{_NO_SAVING} Cannot open with mode={mode!r}.") + + self._mode = mode + target = str(location) + self._location = target if group is None else f"{target.rstrip('/')}/{group}" + + # `location` itself, not its stringified `target`, is handed to + # zarr-python: stringifying a store object such as a `MemoryStore` + # loses the store, leaving only a name zarr-python cannot reopen. + self._group = zarr.open_group(store=location, mode=mode, path=group) + self._format = _FORMAT_BY_ZARR_VERSION[self._group.metadata.zarr_format] + + # Assembled here rather than on first access, because assembling is + # what warns about a dimension name used at two different lengths and + # a load never asks for the mapping: the CF layer reads each + # variable's own `dimensions`. Left to first access, that warning + # reached the unit tests and no user. It costs no extra reads - + # `variables` is materialised by everything that opens a store. + self._dimensions = _assemble_dimensions(self.variables) + + @property + def location(self) -> str: + """The store's path or URL, with the group path when there is one.""" + return self._location + + @property + def format(self) -> str: + """``ZARR_V2`` or ``ZARR_V3``.""" + return self._format + + @property + def mode(self) -> str: + """The mode the store was opened in.""" + return self._mode + + @property + def closed(self) -> bool: + """Whether :meth:`close` has been called. + + A Zarr store is a namespace rather than an open file handle, so there + is nothing to release and nothing else to ask. + + """ + return self._closed + + @property + def variables(self) -> Mapping: + """The group's arrays, by name. Arrays in subgroups are not included.""" + if self._variables is None: + assert self._group is not None + self._variables = { + name: ZarrDatasetVariable(array, self._location, self._format) + for name, array in self._group.arrays() + } + return self._variables + + @property + def dimensions(self) -> Mapping: + """The dimension sizes this group's arrays agree on. + + Assembled when the store is opened, not on first access here. + """ + assert self._dimensions is not None + return self._dimensions + + @property + def storage_settings(self) -> Mapping: + """The store-wide Zarr settings.""" + assert self._group is not None + consolidated = self._group.metadata.consolidated_metadata + return MappingProxyType( + { + "zarr_format": self._group.metadata.zarr_format, + "consolidated": consolidated is not None, + } + ) + + @property + def attributes(self) -> MutableMapping: + """The group's attributes, less netCDF-c's own bookkeeping stamp.""" + if self._attributes is None: + assert self._group is not None + self._attributes = _ZarrAttributes( + self._group, withhold=frozenset({_NC_PROPERTIES}) + ) + return self._attributes + + def create_dimension(self, name: str, size: int | None) -> None: + """Raise: saving to Zarr is not supported.""" + raise NotImplementedError(_NO_SAVING) + + def create_variable( + self, name: str, dtype, dimensions=(), *, fill_value=None, **storage_settings + ): + """Raise: saving to Zarr is not supported.""" + raise NotImplementedError(_NO_SAVING) + + def supports_dtype(self, dtype) -> bool: + """Whether this store can hold a variable of this dtype. + + Always True. Zarr's data types cover the whole of NumPy's fixed-width + numeric range, in both versions, so there is no dtype a caller could + reach here with that the store would have to narrow. Contrast + :meth:`~iris.fileformats.netcdf._dataset.NetCDFDataset.supports_dtype`, + which answers False for the integer types the netCDF 3 formats have no + type for. + + Parameters + ---------- + dtype : :class:`numpy.dtype` + The dtype a variable would be created with. + + Returns + ------- + bool + + """ + return True + + def sync(self) -> None: + """Do nothing: a Zarr write is not buffered.""" + + def finalise(self) -> None: + """Raise: saving to Zarr is not supported. + + A Zarr store's completion step is consolidating its metadata, which + only a writer performs. + + """ + raise NotImplementedError(_NO_SAVING) + + def close(self) -> None: + """Record that this dataset is finished with. + + A store is a namespace, not an open handle, so nothing is released. + The flag exists because callers - :class:`iris.fileformats.cf.CFReader` + among them - must be able to ask. + + """ + self._closed = True + + def __repr__(self) -> str: + """Return a string representation.""" + return f"{self.__class__.__name__}({self.location!r}, mode={self.mode!r})" + + # Zarr-only members below. + + @property + def subgroup_names(self) -> tuple: + """The names of this group's subgroups. + + What the loader names when a group holds no data variables but the + store plainly holds data elsewhere. + + """ + assert self._group is not None + return tuple(self._group.group_keys()) diff --git a/lib/iris/fileformats/zarr/_decode.py b/lib/iris/fileformats/zarr/_decode.py new file mode 100644 index 0000000000..e2064a430a --- /dev/null +++ b/lib/iris/fileformats/zarr/_decode.py @@ -0,0 +1,387 @@ +# Copyright Iris contributors +# +# This file is part of Iris and is released under the BSD license. +# See LICENSE in the root of the repository for full licensing details. +"""The CF decoding netCDF4 does for free and Zarr does not. + +netCDF4 applies ``_FillValue``/``missing_value`` masking, +``valid_min``/``valid_max``/``valid_range`` masking and +``scale_factor``/``add_offset`` unpacking on every read. Zarr returns a plain +``numpy.ndarray`` and applies none of it, so these rules are implemented here. +They live on the Zarr side rather than in the shared CF layer because the +netCDF path gets the behaviour from the C library and has nothing to reuse. + +Everything here is a pure function of values, with no store and no Dask in +sight. :func:`mask_and_scale` is what the loader lays over a raw array as a +``map_blocks`` layer - visible in the graph, fusable with its neighbours, and +testable on a plain ndarray. + +Three names are this module's surface: :func:`decode_parameters`, +:func:`decoding_required` and :func:`mask_and_scale`. Everything else is a +step of one of those, and carries a leading underscore to say so - the module +being private already does not make the distinction worthless, since +``_dataset.py`` should not be reaching for a decode step on its own. + +Two traps, both from the Zarr specification rather than from CF. + +**A floating ``_FillValue`` is base64 on version 3.** Released xarray encodes +it as base64 of a little-endian ``float64`` for *every* floating dtype, +including ``float32``, and its reader requires that form. It is the de-facto +encoding, not a producer's mistake, so Iris reads both it and a plain JSON +number. CF section 2.5.1 asks for "the scalar attribute with the name +``_FillValue`` and of the same type as its variable"; this is a deliberate, +narrow deviation, taken because interoperability with the other Zarr readers +is the reason for native Zarr support. + +**``Array.fill_value`` is not a missing-value declaration.** It says what +unwritten storage reads back as, and zarr-python defaults it to zero. On +version 3 it is never a mask source: an ``int16`` array of ``[0, 1, 2]`` with +no CF attributes would otherwise load with its valid zero masked. On version 2 +it is one, but only when no CF attribute is present, because version 2 stores +predate the convention - which is what released xarray's +``use_zarr_fill_value_as_mask`` does. zarr-python 3 defaults the field to zero +for a version 2 store too, so a store written today can lose a genuine zero +this way; :func:`decode_parameters` warns for exactly that case - the storage +fill value is zero - and not for a non-zero one, which is the writer's own +deliberate, pre-convention sentinel and not something to warn about. + +See section 4.4 of ``docs/superpowers/specs/2026-09-21-zarr-io-design.md``. + +""" + +import base64 +import binascii +import struct +from typing import Any +import warnings + +import numpy as np + +import iris.warnings + +#: The width of the base64 `_FillValue` encoding: one little-endian float64. +_PACKED_DOUBLE = " Any: + """Return a CF missing-value sentinel as a value of `dtype`. + + Parameters + ---------- + value : Any + The attribute's value, as the store holds it: a JSON number, or - for + a floating variable - the base64 of a little-endian ``float64``. + dtype : numpy.dtype + The variable's own dtype. The result is narrowed to it, because a + sentinel of a different width never equals anything in the array. + variable_name : str + Named in the warning, so the user can find the variable. + + Returns + ------- + numpy.generic or None + ``None`` when `value` is neither form, or does not fit `dtype` - + both are producer errors. + + Warns + ----- + IrisCfLoadWarning + When `value` cannot be read as a sentinel of `dtype`. + + """ + if isinstance(value, (bool, np.bool_)): + # A JSON true/false is a number to Python and nothing to CF. + decoded = None + elif isinstance(value, (int, float, np.number)): + decoded = value + elif isinstance(value, str) and dtype.kind == "f": + decoded = _decode_packed_double(value) + else: + decoded = None + + if decoded is None: + message = ( + "Ignoring a missing-value attribute that is neither a number nor " + f"the base64 of a float64, on variable {variable_name}: {value!r}" + ) + warnings.warn(message, category=iris.warnings.IrisCfLoadWarning) + return None + + try: + return dtype.type(decoded) + except (OverflowError, ValueError): + # Out of range for dtype: a producer error, not a reason to fail + # the whole load. A float dtype never raises here - an overflowing + # float narrows to +/-inf instead, which is ordinary numpy + # behaviour and not something to guard against. + message = ( + f"Ignoring a missing-value attribute that does not fit {dtype}, " + f"on variable {variable_name}: {value!r}" + ) + warnings.warn(message, category=iris.warnings.IrisCfLoadWarning) + return None + + +def _decode_missing_values(value: Any, dtype: np.dtype, variable_name: str) -> list: + """Return every sentinel a ``missing_value`` attribute declares. + + CF section 2.5.1 makes ``missing_value`` "a scalar or vector containing + values indicating missing data", so a JSON array there is as valid as a + number and every element of it masks - which is what netCDF4 does with a + vector ``missing_value``, and what Iris has always done through it. + ``_FillValue`` is not read this way: section 2.5.1 makes that one a + scalar. + + Parameters + ---------- + value : Any + The attribute's value: one sentinel, or a sequence of them. + dtype : numpy.dtype + The variable's own dtype, which every sentinel is narrowed to. + variable_name : str + Named in any warning. + + Returns + ------- + list + One entry per element :func:`_decode_fill_value` could read. An + element it could not is warned about and dropped, leaving the rest: + one bad sentinel in a vector is no reason to ignore the good ones. + + """ + if isinstance(value, (list, tuple)) or ( + isinstance(value, np.ndarray) and value.ndim > 0 + ): + elements = list(value) + else: + elements = [value] + decoded = ( + _decode_fill_value(element, dtype, variable_name) for element in elements + ) + return [sentinel for sentinel in decoded if sentinel is not None] + + +def _decode_packed_double(value: str) -> float | None: + """Return the float in a base64 little-endian float64, or None.""" + try: + raw = base64.b64decode(value, validate=True) + except (binascii.Error, ValueError): + return None + if len(raw) != _PACKED_DOUBLE_BYTES: + return None + return struct.unpack(_PACKED_DOUBLE, raw)[0] + + +def decode_parameters( + attributes, + *, + dtype: np.dtype, + storage_fill_value: Any, + zarr_format: int, + variable_name: str, +) -> dict: + """Return the CF decode parameters for one variable. + + Parameters + ---------- + attributes : mapping + The variable's CF attributes. + dtype : numpy.dtype + The variable's stored dtype, before any unpacking. + storage_fill_value : Any + ``zarr.Array.fill_value`` - what unwritten storage reads back as. A + mask source only on version 2, and only when the variable declares no + CF missing-value attribute. + zarr_format : int + 2 or 3. The one place the version changes what is decoded. + variable_name : str + Named in any warning. + + Returns + ------- + dict + The keyword arguments :func:`mask_and_scale` takes, always with the + same keys. :func:`decoding_required` says whether they do anything. + + Warns + ----- + IrisCfLoadWarning + When a version 2 variable declares no CF missing-value attribute, and + the storage fill value about to be used as a sentinel is zero. Zero + is zarr-python's own default for a storage fill value a writer never + set, so it is the case design document section 4.4 calls "the + ordinary case, not the exotic one" when it rules the field out as a + mask source on version 3 - a plausible real value, not a sentinel a + writer chose. A non-zero storage fill value - the classic netCDF + sentinel ``9.96921e+36``, say - is the case section 4.4 keeps this + rule for, and does not warn: a writer old enough to predate the CF + convention chose it on purpose, which is the rule's own premise. + + """ + # The two attributes are read differently because CF section 2.5.1 + # defines them differently: `_FillValue` is a scalar, `missing_value` is + # "a scalar or vector". + sentinels = [] + if "_FillValue" in attributes: + decoded = _decode_fill_value(attributes["_FillValue"], dtype, variable_name) + if decoded is not None: + sentinels.append(decoded) + if "missing_value" in attributes: + sentinels.extend( + _decode_missing_values(attributes["missing_value"], dtype, variable_name) + ) + + if ( + not sentinels + and zarr_format < _FILL_VALUE_IS_A_MASK_BEFORE + and storage_fill_value is not None + ): + # Warn only when the sentinel about to be adopted is zero - the + # value this field defaults to when a writer never set it (design + # document section 4.4). A non-zero storage fill value is a writer's + # deliberate choice, old enough to predate `_FillValue`, which is + # exactly the case this rule exists for; masking on it is + # unremarkable and does not warn. Masking itself is unchanged either + # way - only whether this warns. + if storage_fill_value == 0: + message = ( + f"Variable {variable_name} declares no `_FillValue` or " + f"`missing_value` on a version 2 store, so its storage fill " + f"value {storage_fill_value!r} will be read as missing: values " + "equal to it are masked. Set a `_FillValue` attribute to " + "control this." + ) + warnings.warn(message, category=iris.warnings.IrisCfLoadWarning) + sentinels.append(storage_fill_value) + + valid_min = attributes.get("valid_min") + valid_max = attributes.get("valid_max") + valid_range = attributes.get("valid_range") + if valid_range is not None and len(valid_range) == 2: + valid_min, valid_max = valid_range + + scale_factor = attributes.get("scale_factor") + add_offset = attributes.get("add_offset") + + return { + "sentinels": tuple(sentinels), + "valid_min": valid_min, + "valid_max": valid_max, + "scale_factor": scale_factor, + "add_offset": add_offset, + "dtype": _unpacked_dtype(dtype, scale_factor, add_offset), + } + + +def _unpacked_dtype(dtype, scale_factor, add_offset) -> np.dtype: + """Return the dtype the unpacked values will have. + + The same calculation as + :func:`iris.fileformats.cf.loader._get_actual_dtype`, done here on plain + values so that the dtype the Dask graph declares and the dtype the decode + layer produces are one statement rather than two. + """ + probe = np.zeros(1, dtype=dtype) + if scale_factor is not None: + probe = scale_factor * probe + if add_offset is not None: + probe = add_offset + probe + return probe.dtype + + +def decoding_required(parameters: dict) -> bool: + """Whether these parameters would change anything. + + A variable with nothing to decode gets no ``map_blocks`` layer and an + unmasked Dask meta. Adding a masked meta without a layer beneath it would + be a meta that lies about its own graph. + """ + if parameters["sentinels"]: + return True + return any( + parameters[name] is not None + for name in ("valid_min", "valid_max", "scale_factor", "add_offset") + ) + + +def _mask_by_sentinels(block: np.ndarray, sentinels: tuple) -> np.ndarray: + """Return a boolean mask of the elements equal to any sentinel. + + A NaN sentinel is matched with :func:`numpy.isnan`, because NaN does not + equal itself and NaN is what nearly every version 3 store's base64 + ``_FillValue`` decodes to. + """ + mask = np.zeros(block.shape, dtype=bool) + for sentinel in sentinels: + if isinstance(sentinel, (float, np.floating)) and np.isnan(sentinel): + mask |= np.isnan(block) + else: + mask |= block == sentinel + return mask + + +def _mask_by_valid_range(block: np.ndarray, valid_min, valid_max) -> np.ndarray: + """Return a boolean mask of the elements outside the valid range.""" + mask = np.zeros(block.shape, dtype=bool) + if valid_min is not None: + mask |= block < valid_min + if valid_max is not None: + mask |= block > valid_max + return mask + + +def _unpack(block: np.ndarray, scale_factor, add_offset, dtype: np.dtype) -> np.ndarray: + """Return `block` unpacked by CF's scale and offset, as `dtype`.""" + result = block + if scale_factor is not None: + result = scale_factor * result + if add_offset is not None: + result = add_offset + result + return result.astype(dtype, copy=False) + + +def mask_and_scale( + block: np.ndarray, + *, + sentinels: tuple, + valid_min, + valid_max, + scale_factor, + add_offset, + dtype: np.dtype, +) -> np.ma.MaskedArray: + """Apply the CF decode rules to one block, in CF's order. + + Masking first, on the **packed** values, because CF declares a sentinel to + be "of the same type as its variable"; unpacking second. The result is + always a masked array, so that every block matches the Dask meta the + loader declares for the layer. + + Parameters + ---------- + block : numpy.ndarray + One Dask block of the raw stored values. + sentinels : tuple + Values meaning "missing", already narrowed to the stored dtype. + valid_min, valid_max : optional + CF's valid range. Values outside it are masked. + scale_factor, add_offset : optional + CF's packing. ``value * scale_factor + add_offset``. + dtype : numpy.dtype + The dtype of the result, which the Dask graph has already declared. + + Returns + ------- + numpy.ma.MaskedArray + + """ + mask = _mask_by_sentinels(block, sentinels) + mask |= _mask_by_valid_range(block, valid_min, valid_max) + return np.ma.masked_array( + _unpack(block, scale_factor, add_offset, dtype), mask=mask + ) diff --git a/lib/iris/fileformats/zarr/loader.py b/lib/iris/fileformats/zarr/loader.py new file mode 100644 index 0000000000..e6178e96dc --- /dev/null +++ b/lib/iris/fileformats/zarr/loader.py @@ -0,0 +1,158 @@ +# Copyright Iris contributors +# +# This file is part of Iris and is released under the BSD license. +# See LICENSE in the root of the repository for full licensing details. +"""Load Iris cubes from Zarr stores. + +.. z_reference:: iris.fileformats.zarr.loader + :tags: topic_load_save + + API reference + +A thin loop: open the store as a +:class:`~iris.fileformats.zarr._dataset.ZarrDataset`, hand it to +:class:`~iris.fileformats.cf.CFReader`, and build one cube per CF data +variable with :mod:`iris.fileformats.cf.loader` - the same machinery the +netCDF loader runs. Everything Zarr-specific happens below this module, in +:mod:`iris.fileformats.zarr._dataset`. + +Iris loads **one** group per call. A Zarr store can legitimately hold +unrelated datasets, and merging them into one ``CubeList`` would be wrong, so +nested groups are not walked and ``group=`` selects which one to read. + +UGRID meshes load from a Zarr store exactly as they do from a netCDF file. +Nothing in the mesh machinery is netCDF-specific - it reads a +:class:`~iris.fileformats.cf.CFReader`, and a `CFReader` over a `ZarrDataset` +answers the same questions - so this module calls into +:mod:`iris.fileformats.netcdf.ugrid_load` rather than growing a second copy. +That module's name is the historical home of the code, not a statement about +what it can read. + +See sections 4.6 and 4.7 of +``docs/superpowers/specs/2026-09-21-zarr-io-design.md``. + +""" + +from collections.abc import Iterable +import warnings + +from iris.exceptions import TranslationError +from iris.fileformats.cf import CFReader +from iris.fileformats.cf.loader import ( + _actions_engine, + _load_aux_factory, + _load_cube, + _translate_constraints_to_var_callback, +) +import iris.warnings + +from ._dataset import ZarrDataset + + +def _check_has_data_variables(reader, dataset) -> None: + """Raise if this group holds nothing to make a cube from. + + Returning an empty ``CubeList`` from a store that plainly holds data + elsewhere leaves the user nothing to act on, and a store whose root group + holds only subgroups is a real shape - the ESA EOPF Sentinel products are + built that way. + """ + if reader.cf_group.data_variables or reader.cf_group.promoted: + return + + subgroups = dataset.subgroup_names + if subgroups: + message = ( + f"Zarr group {dataset.location!r} has no data variables. Load " + "one of its groups instead, with the 'group' keyword: " + f"{', '.join(sorted(subgroups))}" + ) + else: + message = f"Zarr store {dataset.location!r} has no data variables." + raise TranslationError(message) + + +def load_cubes(file_sources, callback=None, constraints=None, group=None): + """Load cubes from one or more Zarr stores. + + Parameters + ---------- + file_sources : str or iterable of str + The stores to load. A local directory path, or a URL that + zarr-python can open - ``s3://``, ``gs://``, ``az://`` and ``abfs://`` + need the matching store library installed, which Iris does not + depend on. + callback : callable, optional + Passed to :func:`iris.io.run_callback`. + constraints : optional + Used to skip data variables that cannot match, before they are built. + group : str, optional + A ``/``-separated path to a group within each store. The root group by + default. + + Yields + ------ + iris.cube.Cube + + Raises + ------ + iris.exceptions.TranslationError + If the selected group holds no data variables. The message names the + store's other groups, when it has any. + + Notes + ----- + This function maintains laziness; it does not realise data. + + """ + from iris.fileformats.netcdf.ugrid_load import ( + _attach_mesh_coords, + _meshes_from_cf, + _prepare_mesh_coords, + ) + from iris.io import run_callback + + var_callback = _translate_constraints_to_var_callback(constraints) + engine = _actions_engine() + + if isinstance(file_sources, str) or not isinstance(file_sources, Iterable): + file_sources = [file_sources] + + for file_source in file_sources: + with ZarrDataset(file_source, group=group) as dataset: + with CFReader(dataset) as cf: + _check_has_data_variables(cf, dataset) + + meshes = _meshes_from_cf(cf) + + data_variables = list(cf.cf_group.data_variables.values()) + list( + cf.cf_group.promoted.values() + ) + for cf_var in data_variables: + if var_callback and not var_callback(cf_var): + continue + + # MeshCoords are built before the cube and attached after + # it: see _prepare_mesh_coords for why the two cannot be + # one step. + pending_mesh_coords = _prepare_mesh_coords(cf, meshes, cf_var) + + cube = _load_cube(engine, cf, cf_var, cf.filename) + + if pending_mesh_coords is not None: + _attach_mesh_coords(pending_mesh_coords, cube) + + try: + _load_aux_factory(engine, cube) + except ValueError as exception: + warnings.warn( + str(exception), + category=iris.warnings.IrisLoadWarning, + ) + + cube = run_callback(callback, cube, cf_var, file_source) + if cube is None: + # The callback rejected it. + continue + + yield cube diff --git a/lib/iris/io/__init__.py b/lib/iris/io/__init__.py index ddeb6a7157..9889aaf31b 100644 --- a/lib/iris/io/__init__.py +++ b/lib/iris/io/__init__.py @@ -38,9 +38,38 @@ def _is_nczarr_fragment(fragment): return False +# URI schemes Iris hands to a store library rather than resolving as a local +# path. Iris depends on none of them: zarr-python raises its own ImportError +# naming what to install. +_REMOTE_STORE_SCHEMES = frozenset({"s3", "gs", "az", "abfs"}) + + +def _is_remote_store_uri(spec): + """Whether `spec` names a store Iris hands to a store library untouched. + + Such a URI must not be resolved as a local path: `.absolute()` turns + ``s3://bucket/data.zarr`` into ``//s3:/bucket/data.zarr``. + """ + if not isinstance(spec, str): + return False + scheme, _part, _fragment = decode_uri(spec) + return scheme in _REMOTE_STORE_SCHEMES + + # Saving routines, indexed by file extension. class _SaversDict(dict): - """A dictionary that can only have string keys with no overlap.""" + """A dictionary that can only have string keys with no overlap. + + :attr:`cubelist_savers` records which of the registered savers take a + ``CubeList`` directly, rather than one cube at a time with + ``append=True``. It holds the saver functions themselves, not extensions, + because :func:`save` is also reached with a saver it never looked up here + - passed by the caller, or chosen directly for an NCZarr target. + """ + + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + self.cubelist_savers = set() def __setitem__(self, key, value): if not isinstance(key, str): @@ -170,6 +199,29 @@ def decode_uri(uri, default="file"): return scheme, part, fragment +def _categorise_uri(uri) -> tuple[str, str, str]: + """Return `uri` as ``(category, scheme, part)``, for grouping by handler. + + The category is the scheme, except that an NCZarr fragment makes it + ``"nczarr"``. netCDF-c's Zarr driver is asked for by ``#mode=nczarr,...`` + rather than by scheme, so ``file:///s.zarr`` and + ``file:///s.zarr#mode=nczarr,file`` need different handlers despite + sharing one scheme. The fragment is folded back into `part`, because a + handler is handed a URI to reopen rather than the decoded pieces. + + A native Zarr store has no category of its own. Remotely it is known by + its scheme, and locally it is a directory that only a marker file + distinguishes from any other - see + :func:`iris.fileformats._is_zarr_store`. Deciding it here would put a + filesystem test inside a function that otherwise only reads a string. + """ + scheme, part, fragment = decode_uri(uri) + category = "nczarr" if _is_nczarr_fragment(fragment) else scheme + if fragment: + part = f"{part}#{fragment}" + return category, scheme, part + + def expand_filespecs(file_specs, files_expected=True): """Find all matching file paths from a list of file-specs. @@ -190,15 +242,24 @@ def expand_filespecs(file_specs, files_expected=True): If files_expected is ``False``: A list of expanded file paths. """ - # Remove any hostname component - currently unused + # Remove any hostname component - currently unused. A remote store URI is + # passed through: it is not a path, and resolving it mangles it. filenames = [ - str(pathlib.Path(fn.removeprefix("//")).expanduser().absolute()) + fn + if _is_remote_store_uri(fn) + else str(pathlib.Path(fn.removeprefix("//")).expanduser().absolute()) for fn in file_specs ] if files_expected: - # Try to expand all filenames as globs - glob_expanded = OrderedDict([[fn, sorted(glob.glob(fn))] for fn in filenames]) + # Try to expand all filenames as globs. A remote URI stands for + # itself; Iris does not list a remote store's neighbours. + glob_expanded = OrderedDict( + [ + [fn, [fn] if _is_remote_store_uri(fn) else sorted(glob.glob(fn))] + for fn in filenames + ] + ) # If any of the specs expanded to an empty list then raise an error all_expanded = glob_expanded.values() @@ -231,13 +292,27 @@ def load_files(filenames, callback, constraints=None): intended interface for loading is :func:`iris.load`. """ - from iris.fileformats import FORMAT_AGENT + yield from _load_files(expand_filespecs(filenames), callback, constraints) + - all_file_paths = expand_filespecs(filenames) +def _load_files(filenames, callback, constraints=None): + """Create a generator of Cubes from files already expanded by `expand_filespecs`. + + `filenames` is not expanded here: a glob matching the same file twice, by + two different patterns, must reach the format handler as two entries, and + :func:`expand_filespecs` keys its own glob cache by the spec string, so + expanding an already-expanded list collapses two identical paths into + one. :func:`load_files` is the public entry point that expands once and + delegates here; :func:`iris.loading._generate_cubes` is the other caller, + expanding once for itself so it can partition files from stores before + either sees the list. + + """ + from iris.fileformats import FORMAT_AGENT # Create default dict mapping iris format handler to its associated filenames handler_map = collections.defaultdict(list) - for fn in all_file_paths: + for fn in filenames: with open(fn, "rb") as fh: handling_format_spec = FORMAT_AGENT.get_spec(pathlib.Path(fn).name, fh) handler_map[handling_format_spec].append(fn) @@ -253,6 +328,49 @@ def load_files(filenames, callback, constraints=None): yield cube +def load_stores(targets, callback, constraints=None): + """Create a generator of Cubes from targets that are stores, not files. + + A store is a directory or a URL: something with no single seekable file + to open, so :func:`load_files` cannot reach it and the format agent is + asked without a buffer. + + Parameters + ---------- + targets : iterable of str + Local directory paths or remote URLs, already expanded. + callback : callable + Passed to the format handler. + constraints : optional + Passed to a constraint-aware format handler. + + Yields + ------ + iris.cube.Cube + + Notes + ----- + Typically, this function should not be called directly; instead, the + intended interface for loading is :func:`iris.load`. + + """ + from iris.fileformats import FORMAT_AGENT + + handler_map = collections.defaultdict(list) + for target in targets: + handling_format_spec = FORMAT_AGENT.get_spec(target, None) + handler_map[handling_format_spec].append(target) + + for handling_format_spec in sorted(handler_map): + names = handler_map[handling_format_spec] + if handling_format_spec.constraint_aware_handler: + for cube in handling_format_spec.handler(names, callback, constraints): + yield cube + else: + for cube in handling_format_spec.handler(names, callback): + yield cube + + def load_http(urls, callback): """Create generator of Cubes from the given OPeNDAP or NcZarr URLs. @@ -350,6 +468,8 @@ def _check_init_savers(): "grib2": _grib_save, } ) + # One line per format that takes a CubeList whole. + _savers.cubelist_savers.add(netcdf.save) def add_saver(file_extension, new_saver): @@ -392,10 +512,10 @@ def find_saver(filespec): """ _check_init_savers() + # A store is a directory, so "out.zarr/" is a natural thing to type. + candidate = filespec.rstrip("/\\").lower() matches = [ - ext - for ext in _savers - if filespec.lower().endswith("." + ext) or filespec.lower() == ext + ext for ext in _savers if candidate.endswith("." + ext) or candidate == ext ] # Multiple matches could occur if one of the savers included a '.': # e.g. _savers = {'.dot.png': dot_png_saver, '.png': png_saver} @@ -527,7 +647,8 @@ def save(source, target, saver=None, **kwargs): isinstance(source, (list, tuple)) and all([isinstance(i, Cube) for i in source]) ): # Only allow cubelist saving for those fileformats that are capable. - if "iris.fileformats.netcdf" not in saver.__module__: + _check_init_savers() + if saver not in _savers.cubelist_savers: # Make sure the saver accepts an append keyword if "append" not in saver.__code__.co_varnames: raise ValueError( diff --git a/lib/iris/io/format_picker.py b/lib/iris/io/format_picker.py index ac80e03611..dc04fd549e 100644 --- a/lib/iris/io/format_picker.py +++ b/lib/iris/io/format_picker.py @@ -377,6 +377,33 @@ def get_element(self, basename, file_handle): return decode_uri(basename) +class ZarrStore(FileElement): + """A :class:`FileElement` that returns a target that may be a Zarr store. + + Carries the whole target rather than its basename, because recognising a + local store means reading a marker file inside it, and recognising a + remote one means reading its scheme. + + """ + + def __init__(self): + FileElement.__init__(self, requires_fh=False) + + def get_element(self, basename, file_handle): + """Return the target, which here is the whole path or URL. + + ``None`` when a file handle is supplied, which no store ever has: a + store is a directory or a URL, so the caller that opened a single + seekable file has already settled the question. That caller also + passes a basename rather than a path, which resolved against the + working directory could name an unrelated store that happens to share + the file's name. + """ + if file_handle is not None: + return None + return basename + + class DataSourceObjectProtocol(FileElement): """A :class:`FileElement` that simply returns the URI entry itself. diff --git a/lib/iris/loading.py b/lib/iris/loading.py index 89e2e43059..e4bb5e0b09 100644 --- a/lib/iris/loading.py +++ b/lib/iris/loading.py @@ -13,6 +13,7 @@ from contextlib import contextmanager from dataclasses import dataclass import itertools +from pathlib import Path import threading from traceback import TracebackException from typing import Any, Iterable, Type @@ -83,26 +84,38 @@ def _generate_cubes(uris, callback, constraints): # Make a string, Dataset, or other single item, into an iterable. uris = [uris] - def _categorise(decoded: tuple[str, str, str | None]) -> tuple[str, str, str]: - scheme, part, fragment = decoded - category = "nczarr" if iris.io._is_nczarr_fragment(fragment) else scheme - if fragment: - part = f"{part}#{fragment}" - return category, scheme, part - # Group collections of uris by their iris handler # Create list of tuples relating schemes to part names - uri_tuples = sorted(_categorise(iris.io.decode_uri(uri)) for uri in uris) + uri_tuples = sorted(iris.io._categorise_uri(uri) for uri in uris) for category, groups in itertools.groupby(uri_tuples, key=lambda x: x[0]): # Call each scheme handler with the appropriate URIs if category == "file": part_names = [part for _cat, _scheme, part in groups] - for cube in iris.io.load_files(part_names, callback, constraints): - yield cube + # The file-or-store decision has to happen after glob expansion + # and before format dispatch, because a glob may match both. + # Neither routine below learns about the other's kind of target. + # iris.fileformats.netcdf.ugrid_load.load_meshes makes this same + # distinction independently, for mesh loading - keep both in step. + expanded = iris.io.expand_filespecs(part_names) + files = [name for name in expanded if not Path(name).is_dir()] + stores = [name for name in expanded if Path(name).is_dir()] + if files: + # `expanded` is already the result of `expand_filespecs`, so + # the private, non-expanding `_load_files` is used here - see + # its docstring for why expanding twice would be wrong. + for cube in iris.io._load_files(files, callback, constraints): + yield cube + if stores: + for cube in iris.io.load_stores(stores, callback, constraints): + yield cube elif category in ["http", "https", "nczarr"]: urls = [f"{scheme}:{part}" for _cat, scheme, part in groups] for cube in iris.io.load_http(urls, callback): yield cube + elif category in iris.io._REMOTE_STORE_SCHEMES: + urls = [f"{scheme}:{part}" for _cat, scheme, part in groups] + for cube in iris.io.load_stores(urls, callback, constraints): + yield cube elif category == "data": data_objects = [part for _cat, _scheme, part in groups] for cube in iris.io.load_data_objects(data_objects, callback): diff --git a/lib/iris/tests/integration/zarr/__init__.py b/lib/iris/tests/integration/zarr/__init__.py new file mode 100644 index 0000000000..b7761f385a --- /dev/null +++ b/lib/iris/tests/integration/zarr/__init__.py @@ -0,0 +1,12 @@ +# Copyright Iris contributors +# +# This file is part of Iris and is released under the BSD license. +# See LICENSE in the root of the repository for full licensing details. +"""Integration tests for native Zarr loading. + +Skipped when zarr-python is absent, which is also how the unit tests do it. +""" + +import pytest + +pytest.importorskip("zarr", minversion="3.0.8") diff --git a/lib/iris/tests/integration/zarr/fixtures.py b/lib/iris/tests/integration/zarr/fixtures.py new file mode 100644 index 0000000000..ee300200b0 --- /dev/null +++ b/lib/iris/tests/integration/zarr/fixtures.py @@ -0,0 +1,140 @@ +# Copyright Iris contributors +# +# This file is part of Iris and is released under the BSD license. +# See LICENSE in the root of the repository for full licensing details. +"""A version 3 store carrying the CF features a real one does. + +Modelled on the NOAA GFS archive at dynamical.org, whose properties the Zarr +I/O design names as the ones worth exercising: version 3 `dimension_names`, a +sharded codec, a scalar grid-mapping variable, a two-dimensional auxiliary +coordinate, and the base64 `_FillValue` that released xarray writes for every +floating dtype. A cut-down copy of the real store replaces this. + +Bounds on `latitude` are not part of that real store, but earn their place +anyway: `CFBoundaryVariable.spans` is one of the overrides this pull request +rewrote against the dimension contract, so a bounded coordinate exercises +code this branch actually changed. +""" + +import base64 +import struct + +import numpy as np +import zarr + +SHAPE = (2, 4, 6, 5) +DIMENSIONS = ("time", "level", "latitude", "longitude") + +#: Contiguous bounds for `latitude`, half a cell either side of each centre. +LATITUDE_BOUNDS = np.stack( + [np.arange(SHAPE[2], dtype="f8") - 0.5, np.arange(SHAPE[2], dtype="f8") + 0.5], + axis=1, +) + + +def float_fill_value(value): + """Return `value` in the encoding released xarray writes for a float.""" + return base64.b64encode(struct.pack(" int: + """Return `name`'s fixed size, claiming the next unused one if new.""" + if name not in _STUB_DIMENSION_SIZES: + _STUB_DIMENSION_SIZES[name] = len(_STUB_DIMENSION_SIZES) + 1 + return _STUB_DIMENSION_SIZES[name] + + class _NetCDFVarWithDimensions(_NetCDFVar): """Stub NetCDF variable with dimensions for spans() tests.""" @@ -50,6 +69,17 @@ def __init__(self, name, dimensions, dtype=int): super().__init__(name, dtype=dtype) self.dimensions = dimensions + @property + def shape(self) -> tuple: + """One size per dimension, agreeing with every other stub naming it. + + A property, not an instance attribute set in ``__init__``: `ncattrs` + reads `self.__dict__`, so a stored value would leak onto + `attributes` exactly as an unguarded `dimensions` would - a property + is never in `self.__dict__` and cannot leak that way. + """ + return tuple(_stub_dimension_size(name) for name in self.dimensions) + def assert_warning_gated(operation, warning_category, warning_regex): """Shared convenience for warning asserts.""" diff --git a/lib/iris/tests/unit/fileformats/cf/test_CFReader__dataset.py b/lib/iris/tests/unit/fileformats/cf/test_CFReader__dataset.py index 97a185290b..2d91747616 100644 --- a/lib/iris/tests/unit/fileformats/cf/test_CFReader__dataset.py +++ b/lib/iris/tests/unit/fileformats/cf/test_CFReader__dataset.py @@ -205,3 +205,26 @@ def test_an_unread_attribute_reaches_the_cube(self, sample_path): # This does not pin the reverse: that no attribute Iris *read* reaches # the cube. Every attribute read here is in saver.py's _CF_ATTRS, and # _add_unused_attributes drops those before tracking is consulted. + + +def test_an_already_open_cf_dataset_is_used_as_is(tmp_path): + """A loader that has already opened a store hands it straight over. + + The alternative - a path and a format keyword - would put the choice of + backend inside CFReader, which is the branch the dataset interface exists + to remove. + """ + from iris.fileformats.netcdf._dataset import NetCDFDataset + + path = str(tmp_path / "source.nc") + with NetCDFDataset(path, mode="w") as writer: + writer.create_dimension("x", 2) + writer.create_variable("v", np.dtype("f4"), ("x",)) + + dataset = NetCDFDataset(path, mode="r") + with CFReader(dataset) as reader: + assert reader.filename == dataset.location + assert "v" in reader.cf_group + # Borrowed, not owned: leaving the context must not close it. + assert dataset.closed is False + dataset.close() diff --git a/lib/iris/tests/unit/fileformats/cf/test_CFVariable.py b/lib/iris/tests/unit/fileformats/cf/test_CFVariable.py index cbce96bbfb..c830c1cc67 100644 --- a/lib/iris/tests/unit/fileformats/cf/test_CFVariable.py +++ b/lib/iris/tests/unit/fileformats/cf/test_CFVariable.py @@ -15,6 +15,8 @@ from iris.fileformats.cf import _variables from iris.fileformats.cf.dataset import TrackedAttributes +from .identify_mixins import _NetCDFVarWithDimensions + class CFVariableSub(cf.CFVariable): """A subclass of CFVariable for testing purposes.""" @@ -106,28 +108,32 @@ def test_scalar_dimension_always_true(self, mocker, nc_var): assert cf_var.spans(other) - def test_is_subset_check(self, mocker, nc_vars): - lhs_nc_var, other_nc_var, _ = nc_vars - lhs_nc_var.dimensions = ("time",) - lhs = CFVariableSub("lhs", lhs_nc_var) - - rhs = mocker.MagicMock() - rhs.dimensions = ("time", "lat") - + def test_is_subset_check(self): + # `spans()` now reads `shape` as well as `dimensions` on both sides of + # the comparison, so a bare `mocker.MagicMock()` collaborator will not + # do here: `_dimension_identities()` reads both off an unconfigured + # mock and gets something that iterates as empty, making `spans()` + # trivially "true" instead of exercising the real comparison - see + # the Zarr I/O design doc, section 4.4, and this task's own history. + lhs = CFVariableSub("lhs", _NetCDFVarWithDimensions("lhs_var", ("time",))) + rhs = CFVariableSub("rhs", _NetCDFVarWithDimensions("rhs_var", ("time", "lat"))) assert lhs.spans(rhs) - other_nc_var.dimensions = ("height",) - other = CFVariableSub("other", other_nc_var) + other = CFVariableSub( + "other", _NetCDFVarWithDimensions("other_var", ("height",)) + ) assert not other.spans(rhs) - def test_scalar_dimension_with_others_is_not_scalar(self, mocker, nc_var): + def test_scalar_dimension_with_others_is_not_scalar(self): # _scalar_ means "this variable is scalar", which a variable with a # second dimension is not. It is not a wildcard. - nc_var.dimensions = (_variables._NCZARR_SCALAR_DIMENSION, "bnds") - cf_var = CFVariableSub("not_scalar", nc_var) - - other = mocker.MagicMock() - other.dimensions = ("time",) + cf_var = CFVariableSub( + "not_scalar", + _NetCDFVarWithDimensions( + "not_scalar_var", (_variables._NCZARR_SCALAR_DIMENSION, "bnds") + ), + ) + other = CFVariableSub("other", _NetCDFVarWithDimensions("other_var", ("time",))) assert not cf_var.spans(other) diff --git a/lib/iris/tests/unit/fileformats/cf/test__DimensionIdentity.py b/lib/iris/tests/unit/fileformats/cf/test__DimensionIdentity.py new file mode 100644 index 0000000000..4ee34c1878 --- /dev/null +++ b/lib/iris/tests/unit/fileformats/cf/test__DimensionIdentity.py @@ -0,0 +1,69 @@ +# Copyright Iris contributors +# +# This file is part of Iris and is released under the BSD license. +# See LICENSE in the root of the repository for full licensing details. +"""Tests for `_DimensionIdentity`, which decides what is the same dimension. + +Equality and hashing are hand-written, and `CFVariable.spans` and +`iris.fileformats._nc_load_rules.helpers._common_data_dims` both lean on +them through `set.issubset`, `in` and `tuple.index`. The pairs below are the +whole contract. +""" + +from iris.fileformats.cf._variables import _DimensionIdentity + + +class Test___eq__: + def test_same_name_and_size_are_equal(self): + assert _DimensionIdentity("time", 4) == _DimensionIdentity("time", 4) + + def test_same_name_at_two_sizes_is_not_equal(self): + # The reason matching is a pair: Zarr lets one name mean two lengths. + assert _DimensionIdentity("time", 4) != _DimensionIdentity("time", 5) + + def test_different_names_at_one_size_are_not_equal(self): + assert _DimensionIdentity("time", 4) != _DimensionIdentity("level", 4) + + def test_an_anonymous_dimension_equals_itself(self): + identity = _DimensionIdentity(None, 4) + assert identity == identity + + def test_two_anonymous_dimensions_are_not_equal(self): + # Not even at the same size. There is nothing to match on. + assert _DimensionIdentity(None, 4) != _DimensionIdentity(None, 4) + + def test_an_anonymous_dimension_does_not_equal_a_named_one(self): + assert _DimensionIdentity(None, 4) != _DimensionIdentity("time", 4) + assert _DimensionIdentity("time", 4) != _DimensionIdentity(None, 4) + + def test_a_foreign_object_is_not_equal(self): + # `__eq__` returns NotImplemented, so Python falls back to identity. + assert _DimensionIdentity("time", 4) != ("time", 4) + + +class Test___hash__: + def test_same_name_and_size_hash_alike(self): + # Equal members must land in the same bucket, or `set.issubset` misses. + assert hash(_DimensionIdentity("time", 4)) == hash( + _DimensionIdentity("time", 4) + ) + + +class Test_set_membership: + """How `spans` and `_common_data_dims` actually consume the class. + + `__eq__` and `__hash__` imply these, so they are not independent + behaviour. They are here because set membership - not the dunders - is + the contract those two callers depend on, and an identity that stopped + satisfying it would break loading rather than fail a dunder test. + """ + + def test_named_identities_work_as_a_subset(self): + source = {_DimensionIdentity("time", 4)} + target = {_DimensionIdentity("time", 4), _DimensionIdentity("level", 3)} + assert source.issubset(target) + + def test_an_anonymous_dimension_is_never_a_subset(self): + source = {_DimensionIdentity(None, 4)} + target = {_DimensionIdentity(None, 4), _DimensionIdentity("level", 3)} + assert not source.issubset(target) diff --git a/lib/iris/tests/unit/fileformats/cf/test_no_implementation_coupling.py b/lib/iris/tests/unit/fileformats/cf/test_no_implementation_coupling.py new file mode 100644 index 0000000000..65eaa1c7e9 --- /dev/null +++ b/lib/iris/tests/unit/fileformats/cf/test_no_implementation_coupling.py @@ -0,0 +1,48 @@ +# Copyright Iris contributors +# +# This file is part of Iris and is released under the BSD license. +# See LICENSE in the root of the repository for full licensing details. +"""The CF layer must not branch on which storage implementation it holds. + +`isinstance(thing, CFDataset)` tests the *interface*, which is how generic +code is supposed to accept any backend, and `CFReader` does exactly that. +What is banned is testing an *implementation*: that is what makes a third +backend a rewrite rather than a new module. Constructing one is a third +thing again, and legitimate - `CFReader` opens a `NetCDFDataset` for a bare +path, which is a default, not a branch on something it was handed. +""" + +from pathlib import Path + +import iris.fileformats.cf + +#: Implementation class names that may not appear in a type test. +IMPLEMENTATION_NAMES = ( + "NetCDFDataset", + "NetCDFDatasetVariable", + "ZarrDataset", + "ZarrDatasetVariable", +) + +#: The spellings of a type test. `__class__` catches the hand-rolled form. +TYPE_TESTS = ("isinstance", "issubclass", "__class__", "type(") + + +def cf_package_sources(): + """Yield `(path, text)` for every module in `iris.fileformats.cf`.""" + package = Path(iris.fileformats.cf.__file__).parent + for path in sorted(package.glob("*.py")): + yield path, path.read_text(encoding="utf-8") + + +def test_no_module_type_tests_an_implementation_class(): + offenders = [] + for path, text in cf_package_sources(): + for line_number, line in enumerate(text.splitlines(), start=1): + if line.lstrip().startswith("#"): + continue + names_one = any(name in line for name in IMPLEMENTATION_NAMES) + tests_type = any(spelling in line for spelling in TYPE_TESTS) + if names_one and tests_type: + offenders.append(f"{path.name}:{line_number}: {line.strip()}") + assert offenders == [] diff --git a/lib/iris/tests/unit/fileformats/cf/test_spans.py b/lib/iris/tests/unit/fileformats/cf/test_spans.py new file mode 100644 index 0000000000..2bcfcc93d0 --- /dev/null +++ b/lib/iris/tests/unit/fileformats/cf/test_spans.py @@ -0,0 +1,203 @@ +# Copyright Iris contributors +# +# This file is part of Iris and is released under the BSD license. +# See LICENSE in the root of the repository for full licensing details. +"""`CFVariable.spans`, asked of both storage backends. + +Dimension matching is a pair, `(name, size)`, not a bare name - which is a +widening to admit what Zarr allows, not a change to what netCDF does. These +tests run against both so that the second half of that sentence is checked +rather than asserted. +""" + +import numpy as np +import pytest + +from iris.fileformats.cf import ( + CFBoundaryVariable, + CFDataVariable, + CFLabelVariable, +) + +zarr = pytest.importorskip("zarr") + +from iris.fileformats.netcdf._dataset import NetCDFDataset # noqa: E402 +from iris.fileformats.zarr._dataset import ZarrDataset # noqa: E402 + + +def build_netcdf(path, described): + """Write `described` as a netCDF file and return its open dataset.""" + dataset = NetCDFDataset(path, mode="w") + sizes = {} + for dimensions, shape in described.values(): + for name, size in zip(dimensions, shape): + sizes.setdefault(name, size) + for name, size in sizes.items(): + dataset.create_dimension(name, size) + for name, (dimensions, shape) in described.items(): + dataset.create_variable(name, np.dtype("f4"), dimensions) + return dataset + + +def build_zarr(path, described): + """Write `described` as a Zarr store and return its open dataset.""" + group = zarr.open_group(store=path, mode="w", zarr_format=3) + for name, (dimensions, shape) in described.items(): + group.create_array(name, shape=shape, dtype="f4", dimension_names=dimensions) + return ZarrDataset(str(path)) + + +@pytest.fixture(params=[build_netcdf, build_zarr], ids=["netcdf", "zarr"]) +def build(request, tmp_path): + """Return a callable giving `{name: CFDataVariable}` for a description.""" + suffix = ".nc" if request.param is build_netcdf else ".zarr" + + def make(described): + dataset = request.param(tmp_path / f"spans{suffix}", described) + return { + name: CFDataVariable(name, dataset.variables[name]) for name in described + } + + return make + + +def test_identical_dimensions_span(build): + variables = build({"data": (("y", "x"), (3, 2)), "aux": (("y", "x"), (3, 2))}) + assert variables["aux"].spans(variables["data"]) is True + + +def test_a_subset_spans(build): + variables = build({"data": (("y", "x"), (3, 2)), "aux": (("y",), (3,))}) + assert variables["aux"].spans(variables["data"]) is True + + +def test_a_different_name_does_not_span(build): + variables = build({"data": (("y", "x"), (3, 2)), "aux": (("t",), (3,))}) + assert variables["aux"].spans(variables["data"]) is False + + +def test_a_scalar_always_spans(build): + variables = build({"data": (("y", "x"), (3, 2)), "grid": ((), ())}) + assert variables["grid"].spans(variables["data"]) is True + + +def test_a_bounds_variable_ignores_its_extent_dimension(build): + variables = build({"data": (("y",), (3,)), "bounds": (("y", "bnds"), (3, 2))}) + bounds = CFBoundaryVariable("bounds", variables["bounds"].cf_data) + assert bounds.spans(variables["data"]) is True + + +def test_a_label_variable_ignores_its_string_length_dimension(build): + variables = build({"data": (("y",), (3,)), "label": (("y", "strlen"), (3, 8))}) + label = CFLabelVariable("label", variables["label"].cf_data) + assert label.spans(variables["data"]) is True + + +def test_a_name_at_two_sizes_does_not_link(tmp_path): + """Zarr-only: a name guarantees nothing across arrays. + + The version 3 core specification "does not place any restrictions on the + use of the same dimension name across multiple arrays within the same Zarr + hierarchy", so two arrays may mean different things by one name. Matching + on the name alone would produce a cube whose coordinate is the wrong + length. netCDF cannot express this: a dimension name has one size per + group. + """ + group = zarr.open_group(store=tmp_path / "s.zarr", mode="w", zarr_format=3) + group.create_array("data", shape=(3,), dtype="f4", dimension_names=("t",)) + group.create_array("aux", shape=(5,), dtype="f4", dimension_names=("t",)) + + dataset = ZarrDataset(str(tmp_path / "s.zarr")) + data = CFDataVariable("data", dataset.variables["data"]) + aux = CFDataVariable("aux", dataset.variables["aux"]) + assert aux.spans(data) is False + + +def test_an_anonymous_dimension_matches_nothing(tmp_path): + """Not even another anonymous dimension, of the same size, elsewhere. + + Bare-name matching treats `None == None` as a match: `aux` would span + `data` under the old rule, for a dimension neither variable actually + named. There is nothing shared between two differently-sourced "no + name"s, so claiming a match would be invention. + """ + group = zarr.open_group(store=tmp_path / "s.zarr", mode="w", zarr_format=3) + group.create_array("data", shape=(3,), dtype="f4") + group.create_array("aux", shape=(3,), dtype="f4") + + dataset = ZarrDataset(str(tmp_path / "s.zarr")) + data = CFDataVariable("data", dataset.variables["data"]) + aux = CFDataVariable("aux", dataset.variables["aux"]) + assert aux.spans(data) is False + + +def test_two_anonymous_dimensions_of_one_size_do_not_match(tmp_path): + """Not even each other, and not within one variable. + + There is no information to match on, so claiming a match would be + invention. + """ + group = zarr.open_group(store=tmp_path / "s.zarr", mode="w", zarr_format=3) + group.create_array("data", shape=(3, 3), dtype="f4") + group.create_array("aux", shape=(3,), dtype="f4") + + dataset = ZarrDataset(str(tmp_path / "s.zarr")) + data = CFDataVariable("data", dataset.variables["data"]) + aux = CFDataVariable("aux", dataset.variables["aux"]) + assert aux.spans(data) is False + + +def test_an_anonymous_dimension_is_not_a_coordinate_variable(tmp_path): + """A one-dimensional array is a coordinate variable by name coincidence. + + `CFCoordinateVariable.identify` requires the variable's own name to be + among its dimension names, which no anonymous dimension satisfies - so an + anonymous one-dimensional array is never mistaken for a coordinate + variable, whatever it is named. + """ + from iris.fileformats.cf import CFCoordinateVariable + + group = zarr.open_group(store=tmp_path / "s.zarr", mode="w", zarr_format=3) + group.create_array("t", shape=(3,), dtype="f4") + + dataset = ZarrDataset(str(tmp_path / "s.zarr")) + assert CFCoordinateVariable.identify(dataset.variables) == {} + + +def test_a_partly_named_variable_matches_on_the_named_part(tmp_path): + # "A variable may name some of its dimensions and not others." The named + # part ("y") is present in the target and would be enough to span under + # the old rule once the anonymous part is set aside - so `data` carries + # an anonymous dimension of its own, of the same size as `aux`'s, to show + # that one does not rescue the match either: bare-name matching would + # read both `None`s as the same dimension and wrongly call this a span. + group = zarr.open_group(store=tmp_path / "s.zarr", mode="w", zarr_format=3) + group.create_array( + "data", shape=(3, 2, 2), dtype="f4", dimension_names=("y", "x", None) + ) + group.create_array("aux", shape=(3, 2), dtype="f4", dimension_names=("y", None)) + + dataset = ZarrDataset(str(tmp_path / "s.zarr")) + data = CFDataVariable("data", dataset.variables["data"]) + aux = CFDataVariable("aux", dataset.variables["aux"]) + assert aux.spans(data) is False + + +def test_a_non_spanning_variable_is_dropped_and_promoted(tmp_path): + """An anonymous-dimension variable becomes a cube of its own. + + This is what keeps `None` out of `iris.fileformats._nc_load_rules.helpers`, + which matches dimensions by name and would read `None in (None, "x")` as a + match. `CFReader._build_cf_groups` drops any variable whose `spans` is + False from the referrer's group and promotes it. + """ + from iris.fileformats.cf import CFReader + + group = zarr.open_group(store=tmp_path / "s.zarr", mode="w", zarr_format=3) + data = group.create_array("data", shape=(3,), dtype="f4", dimension_names=("t",)) + data.attrs["coordinates"] = "aux" + group.create_array("aux", shape=(3,), dtype="f4") + + with CFReader(ZarrDataset(str(tmp_path / "s.zarr"))) as reader: + assert "aux" not in reader.cf_group["data"].cf_group + assert "aux" in reader.cf_group.promoted diff --git a/lib/iris/tests/unit/fileformats/cf/test_storage_chunk_warning.py b/lib/iris/tests/unit/fileformats/cf/test_storage_chunk_warning.py new file mode 100644 index 0000000000..c6cf546d55 --- /dev/null +++ b/lib/iris/tests/unit/fileformats/cf/test_storage_chunk_warning.py @@ -0,0 +1,96 @@ +# Copyright Iris contributors +# +# This file is part of Iris and is released under the BSD license. +# See LICENSE in the root of the repository for full licensing details. +"""The warning raised when a stored chunk is larger than Dask's target. + +Iris sizes Dask chunks to Dask's target and does not round back up to the +store's own chunking, so a large stored chunk is divided and then fetched once +per division. The warning is what tells a caller there is a `CHUNK_CONTROL` +decision to make. It has to reach both storage backends, or netCDF and Zarr +would behave differently over physics they share. +""" + +import warnings + +import dask.config +import numpy as np +import pytest + +from iris.fileformats.cf import CFDataVariable +from iris.fileformats.cf.loader import CHUNK_CONTROL, _chunks_from_chunk_control +from iris.warnings import IrisLoadWarning + +zarr = pytest.importorskip("zarr") + +from iris.fileformats.netcdf._dataset import NetCDFDataset # noqa: E402 +from iris.fileformats.zarr._dataset import ZarrDataset # noqa: E402 + +SHAPE = (64,) +#: Stored chunk of 32 float64 values, which is 256 bytes. +BIG_CHUNK = (32,) +#: Stored chunk of 4 float64 values, which is 32 bytes. +SMALL_CHUNK = (4,) +#: Sits between the two, so one choice of stored chunk crosses it and the +#: other does not. +TARGET = "128B" + + +def build_netcdf(path, chunks): + dataset = NetCDFDataset(str(path), mode="w") + dataset.create_dimension("t", SHAPE[0]) + dataset.create_variable("v", np.dtype("f8"), ("t",), chunksizes=chunks) + dataset.close() + return NetCDFDataset(str(path)) + + +def build_zarr(path, chunks): + group = zarr.open_group(store=path, mode="w", zarr_format=3) + group.create_array( + "v", shape=SHAPE, dtype="f8", chunks=chunks, dimension_names=("t",) + ) + return ZarrDataset(str(path)) + + +@pytest.fixture(params=[build_netcdf, build_zarr], ids=["netcdf", "zarr"]) +def build(request, tmp_path): + """Return a callable giving the `CFDataVariable` for a stored chunking.""" + suffix = ".nc" if request.param is build_netcdf else ".zarr" + + def make(chunks): + dataset = request.param(tmp_path / f"chunked{suffix}", chunks) + return CFDataVariable("v", dataset.variables["v"]) + + return make + + +def test_a_stored_chunk_over_the_target_warns(build): + cf_var = build(BIG_CHUNK) + with dask.config.set({"array.chunk-size": TARGET}): + with pytest.warns(IrisLoadWarning, match="CHUNK_CONTROL"): + _chunks_from_chunk_control(cf_var) + + +def test_the_warning_names_the_variable(build): + cf_var = build(BIG_CHUNK) + with dask.config.set({"array.chunk-size": TARGET}): + with pytest.warns(IrisLoadWarning, match="v$"): + _chunks_from_chunk_control(cf_var) + + +def test_a_stored_chunk_under_the_target_is_quiet(build): + cf_var = build(SMALL_CHUNK) + with dask.config.set({"array.chunk-size": TARGET}): + with warnings.catch_warnings(): + warnings.simplefilter("error", IrisLoadWarning) + _chunks_from_chunk_control(cf_var) + + +def test_a_pinned_chunking_is_not_second_guessed(build): + """A caller who has set the chunking has already made this decision.""" + cf_var = build(BIG_CHUNK) + with dask.config.set({"array.chunk-size": TARGET}): + with warnings.catch_warnings(): + warnings.simplefilter("error", IrisLoadWarning) + with CHUNK_CONTROL.set("v", t=8): + _chunks_from_chunk_control(cf_var) diff --git a/lib/iris/tests/unit/fileformats/nc_load_rules/helpers/test__common_data_dims.py b/lib/iris/tests/unit/fileformats/nc_load_rules/helpers/test__common_data_dims.py new file mode 100644 index 0000000000..5ba346b721 --- /dev/null +++ b/lib/iris/tests/unit/fileformats/nc_load_rules/helpers/test__common_data_dims.py @@ -0,0 +1,75 @@ +# Copyright Iris contributors +# +# This file is part of Iris and is released under the BSD license. +# See LICENSE in the root of the repository for full licensing details. +"""Test function :func:`iris.fileformats._nc_load_rules.helpers._common_data_dims`.""" + +from dataclasses import dataclass + +from iris.fileformats._nc_load_rules.helpers import _common_data_dims + + +@dataclass +class Variable: + """The two members the dimension rule reads off a CF variable.""" + + dimensions: tuple + shape: tuple + + +def test_the_one_shared_dimension(): + coord = Variable(("x",), (3,)) + data = Variable(("y", "x"), (2, 3)) + assert _common_data_dims(coord, data) == [1] + + +def test_several_shared_dimensions_keep_the_coordinates_order(): + coord = Variable(("x", "y"), (3, 2)) + data = Variable(("y", "x"), (2, 3)) + assert _common_data_dims(coord, data) == [1, 0] + + +def test_a_scalar_variable_shares_nothing(): + # `None`, not `[]`: it is what `Cube.add_aux_coord` takes for "no + # dimensions", and an empty list is not. + assert _common_data_dims(Variable((), ()), Variable(("x",), (3,))) is None + + +def test_a_dimension_the_data_variable_does_not_have(): + assert _common_data_dims(Variable(("t",), (4,)), Variable(("x",), (3,))) is None + + +def test_the_same_name_at_a_different_length_shares_nothing(): + """A name is not an identity once a Zarr store is in scope. + + `CFVariable.spans` decided this the same way before the coordinate got + this far, so answering differently here would attach a coordinate that + was never found to span the data variable. + """ + assert _common_data_dims(Variable(("x",), (3,)), Variable(("x",), (4,))) is None + + +def test_a_repeated_dimension_name_is_resolved_by_length(): + """The case a Zarr store can present and no netCDF file can. + + A netCDF dimension name is unique within its group and carries one size; + a Zarr dimension name is a label with no such guarantee, so one variable + may name both of its dimensions alike at two lengths. Matched on the bare + name, the coordinate lands on dimension 0 and is rejected as the wrong + length. + """ + coord = Variable(("d",), (3,)) + data = Variable(("d", "d"), (2, 3)) + assert _common_data_dims(coord, data) == [1] + + +def test_an_anonymous_dimension_matches_nothing(): + """Not even another anonymous dimension: there is nothing to match on. + + A Zarr version 3 store may leave a dimension unnamed, and + `CFVariable.spans` rejects a coordinate that has one, so this is the same + answer arrived at twice rather than two rules disagreeing. + """ + coord = Variable((None,), (3,)) + data = Variable((None,), (3,)) + assert _common_data_dims(coord, data) is None diff --git a/lib/iris/tests/unit/fileformats/nc_load_rules/helpers/test_build_and_add_ancil_var.py b/lib/iris/tests/unit/fileformats/nc_load_rules/helpers/test_build_and_add_ancil_var.py index b5b6ddc300..0fd7f3fb89 100644 --- a/lib/iris/tests/unit/fileformats/nc_load_rules/helpers/test_build_and_add_ancil_var.py +++ b/lib/iris/tests/unit/fileformats/nc_load_rules/helpers/test_build_and_add_ancil_var.py @@ -22,7 +22,10 @@ def mock_engine(mocker): return mocker.Mock( cube=mocker.Mock(), - cf_var=mocker.Mock(dimensions=("foo", "bar")), + # A data variable has a shape as well as dimension names, because an + # ancillary variable is attached to the dimensions it matches on + # `(name, size)` - see `helpers._common_data_dims`. + cf_var=mocker.Mock(dimensions=("foo", "bar"), shape=(6, 2)), filename="DUMMY", cube_parts=dict(ancillary_variables=[]), ) diff --git a/lib/iris/tests/unit/fileformats/nc_load_rules/helpers/test_build_and_add_auxiliary_coordinate.py b/lib/iris/tests/unit/fileformats/nc_load_rules/helpers/test_build_and_add_auxiliary_coordinate.py index 83d8b4a9f9..1d33aed985 100644 --- a/lib/iris/tests/unit/fileformats/nc_load_rules/helpers/test_build_and_add_auxiliary_coordinate.py +++ b/lib/iris/tests/unit/fileformats/nc_load_rules/helpers/test_build_and_add_auxiliary_coordinate.py @@ -46,8 +46,14 @@ def _setup(self, mocker): self.engine = mocker.Mock( cube=mocker.Mock(), + # A data variable has a shape as well as dimension names, because + # a coordinate is attached to the dimensions it matches on + # `(name, size)` - see `helpers._common_data_dims`. cf_var=mocker.Mock( - dimensions=("foo", "bar"), cf_data=cf_data, dtype=np.int32 + dimensions=("foo", "bar"), + shape=(self.dim_names_lens["foo"], self.dim_names_lens["bar"]), + cf_data=cf_data, + dtype=np.int32, ), filename="DUMMY", cube_parts=dict(coordinates=[]), @@ -185,7 +191,9 @@ def _setup(self, mocker): self.engine = mocker.Mock( cube=mocker.Mock(), - cf_var=mocker.Mock(dimensions=("foo", "bar"), dtype=np.int32), + cf_var=mocker.Mock( + dimensions=("foo", "bar"), shape=points.shape, dtype=np.int32 + ), filename="DUMMY", cube_parts=dict(coordinates=[]), ) @@ -260,7 +268,9 @@ def _setup(self, mocker): # Create dummy pyke engine. self.engine = mocker.Mock( cube=mocker.Mock(), - cf_var=mocker.Mock(dimensions=("foo", "bar"), dtype=np.float32), + cf_var=mocker.Mock( + dimensions=("foo", "bar"), shape=(6, 2), dtype=np.float32 + ), filename="DUMMY", cube_parts=dict(coordinates=[]), ) diff --git a/lib/iris/tests/unit/fileformats/nc_load_rules/helpers/test_build_and_add_cell_measure.py b/lib/iris/tests/unit/fileformats/nc_load_rules/helpers/test_build_and_add_cell_measure.py index f350c44a0d..2252eaf7e4 100644 --- a/lib/iris/tests/unit/fileformats/nc_load_rules/helpers/test_build_and_add_cell_measure.py +++ b/lib/iris/tests/unit/fileformats/nc_load_rules/helpers/test_build_and_add_cell_measure.py @@ -22,7 +22,10 @@ def mock_engine(mocker): return mocker.Mock( cube=mocker.Mock(), - cf_var=mocker.Mock(dimensions=("foo", "bar")), + # A data variable has a shape as well as dimension names, because a + # cell measure is attached to the dimensions it matches on + # `(name, size)` - see `helpers._common_data_dims`. + cf_var=mocker.Mock(dimensions=("foo", "bar"), shape=(6, 2)), filename="DUMMY", cube_parts=dict(cell_measures=[]), ) diff --git a/lib/iris/tests/unit/fileformats/nc_load_rules/helpers/test_build_and_add_dimension_coordinate.py b/lib/iris/tests/unit/fileformats/nc_load_rules/helpers/test_build_and_add_dimension_coordinate.py index dd87b22e3a..a032a78b5b 100644 --- a/lib/iris/tests/unit/fileformats/nc_load_rules/helpers/test_build_and_add_dimension_coordinate.py +++ b/lib/iris/tests/unit/fileformats/nc_load_rules/helpers/test_build_and_add_dimension_coordinate.py @@ -28,7 +28,7 @@ def _mixin_setup(self, mocker): # Create dummy pyke engine. self.engine = mocker.Mock( cube=mocker.Mock(), - cf_var=mocker.Mock(dimensions=("foo", "bar"), dtype=np.int32), + cf_var=mocker.Mock(dimensions=("foo", "bar"), shape=(6, 2), dtype=np.int32), filename="DUMMY", cube_parts=dict(coordinates=[]), ) @@ -59,6 +59,16 @@ def get_cf_bounds_var(coord_var): new=get_cf_bounds_var, ) + def _set_data_var_length(self, length): + """Keep the mock data variable's "foo" dimension the coordinate's length. + + A coordinate is attached to the dimensions it matches on + `(name, size)` - see `helpers._common_data_dims` - so a data variable + naming "foo" at some other length shares nothing with the coordinate + and the coordinate reaches the cube with no dimensions at all. + """ + self.engine.cf_var.shape = (length, 2) + def _make_bounds_var(self, bounds, dimensions, units): bounds = np.array(bounds) # No flag_values/flag_masks/flag_meanings: their absence from @@ -103,6 +113,7 @@ def _set_cf_coord_var(self, points): self.cf_coord_var.size = np.prod(points.shape) self.cf_coord_var.dtype = points.dtype self.cf_coord_var.filename = "DUMMY" + self._set_data_var_length(points.shape[0]) def check_case_dim_coord_construction(self, climatology=False): # Test a generic dimension coordinate, with or without @@ -348,6 +359,7 @@ def _setup(self, mocker): self.cf_coord_var.cf_data = RealArrayCfData(points) self.cf_coord_var.shape = points.shape self.cf_coord_var.dtype = points.dtype + self._set_data_var_length(points.shape[0]) def test_slowest_varying_vertex_dim__normalise_bounds(self): # Create the bounds cf variable. @@ -441,6 +453,7 @@ def _make_vars(self, points, bounds=None, units="degrees"): self.cf_coord_var.cf_data = RealArrayCfData(points) self.cf_coord_var.shape = points.shape self.cf_coord_var.dtype = points.dtype + self._set_data_var_length(points.shape[0]) if bounds: bounds = np.array(bounds).reshape(self.cf_coord_var.shape + (2,)) dimensions = ("x", "nv") diff --git a/lib/iris/tests/unit/fileformats/netcdf/dataset/test_NetCDFDatasetVariable.py b/lib/iris/tests/unit/fileformats/netcdf/dataset/test_NetCDFDatasetVariable.py index 1d6018e2f2..131714e278 100644 --- a/lib/iris/tests/unit/fileformats/netcdf/dataset/test_NetCDFDatasetVariable.py +++ b/lib/iris/tests/unit/fileformats/netcdf/dataset/test_NetCDFDatasetVariable.py @@ -35,7 +35,10 @@ def encoded(sample_path): def wrap(dataset, name, location, write_lock_factory=None): return _dataset.NetCDFDatasetVariable( - dataset.variables[name], location, write_lock_factory=write_lock_factory + dataset.variables[name], + location, + dataset.file_format, + write_lock_factory=write_lock_factory, ) @@ -131,7 +134,7 @@ def test_values_are_read_once(self, mocker, sample_location): variable = mocker.Mock() variable.ncattrs.return_value = ["units"] variable.getncattr.return_value = "K" - wrapped = _dataset.NetCDFDatasetVariable(variable, sample_location) + wrapped = _dataset.NetCDFDatasetVariable(variable, sample_location, "NETCDF4") _ = wrapped.attributes["units"] _ = wrapped.attributes["units"] assert variable.ncattrs.call_count == 1 @@ -156,7 +159,7 @@ def test_unreadable_attribute_becomes_empty_string(self, mocker, sample_location variable.getncattr.side_effect = lambda name: ( "m" if name == "units" else _raise(AttributeError(name)) ) - wrapped = _dataset.NetCDFDatasetVariable(variable, sample_location) + wrapped = _dataset.NetCDFDatasetVariable(variable, sample_location, "NETCDF4") assert wrapped.attributes["broken"] == "" @@ -183,7 +186,7 @@ def test_variable_exposes_the_backing_wrapper(self, raw, sample_location): # so the check has to compare against the one instance actually # passed in, not a second, independently fetched wrapper. variable = raw.variables["air_temperature"] - wrapped = _dataset.NetCDFDatasetVariable(variable, sample_location) + wrapped = _dataset.NetCDFDatasetVariable(variable, sample_location, "NETCDF4") assert wrapped.variable is variable def test_not_variable_length(self, air): @@ -209,7 +212,7 @@ def test_emulated_round_trip(self, sample_location): # The emulation hook ncdata uses (issue #4994): an emulating variable # carries its own array instead of file storage. variable = _EmulatedVariable() - wrapped = _dataset.NetCDFDatasetVariable(variable, sample_location) + wrapped = _dataset.NetCDFDatasetVariable(variable, sample_location, "NETCDF4") variable._data_array = np.zeros(3) assert wrapped.is_emulated is True np.testing.assert_array_equal(wrapped.emulated_data_array, np.zeros(3)) @@ -248,7 +251,7 @@ def test_emulated_array_is_returned_as_it_stands(self, sample_location, mocker): mocker.patch("iris.fileformats.cf.loader._LAZYVAR_MIN_BYTES", 0) variable = _EmulatedVariable() variable._data_array = np.arange(6, dtype="f4").reshape(2, 3) - wrapped = _dataset.NetCDFDatasetVariable(variable, sample_location) + wrapped = _dataset.NetCDFDatasetVariable(variable, sample_location, "NETCDF4") result = wrapped.read_data(_unreachable_policy) assert result is variable._data_array @@ -259,7 +262,7 @@ def test_emulated_array_builds_no_proxy(self, sample_location, mocker): proxy_class = mocker.patch.object(_thread_safe_nc, "NetCDFDataProxy") variable = _EmulatedVariable() variable._data_array = np.arange(6, dtype="f4").reshape(2, 3) - wrapped = _dataset.NetCDFDatasetVariable(variable, sample_location) + wrapped = _dataset.NetCDFDatasetVariable(variable, sample_location, "NETCDF4") wrapped.read_data(_unreachable_policy) assert proxy_class.call_count == 0 diff --git a/lib/iris/tests/unit/fileformats/netcdf/dataset/test_NetCDFDataset__contract.py b/lib/iris/tests/unit/fileformats/netcdf/dataset/test_NetCDFDataset__contract.py index 0836f8c404..e72c15fa45 100644 --- a/lib/iris/tests/unit/fileformats/netcdf/dataset/test_NetCDFDataset__contract.py +++ b/lib/iris/tests/unit/fileformats/netcdf/dataset/test_NetCDFDataset__contract.py @@ -8,19 +8,21 @@ from iris.fileformats.netcdf import _dataset from iris.tests.unit.fileformats.cf.dataset.contract import ( - CFDatasetContract, - populate, + CFDatasetReadContract, + CFDatasetWriteContract, ) -class TestNetCDFDatasetContract(CFDatasetContract): +class TestNetCDFDatasetContract(CFDatasetReadContract, CFDatasetWriteContract): """The netCDF side of the contract.""" + chunking_key = "chunksizes" + @pytest.fixture def readable(self, tmp_path): path = tmp_path / "contract.nc" with _dataset.NetCDFDataset(path, mode="w", netcdf_format="NETCDF4") as dataset: - populate(dataset) + self.populate(dataset) with _dataset.NetCDFDataset(path) as dataset: yield dataset diff --git a/lib/iris/tests/unit/fileformats/netcdf/dataset/test_NetCDFDataset__write.py b/lib/iris/tests/unit/fileformats/netcdf/dataset/test_NetCDFDataset__write.py index e8eca7d9dc..458efdfde7 100644 --- a/lib/iris/tests/unit/fileformats/netcdf/dataset/test_NetCDFDataset__write.py +++ b/lib/iris/tests/unit/fileformats/netcdf/dataset/test_NetCDFDataset__write.py @@ -145,7 +145,10 @@ def test_write_handle_of_an_unencoded_variable_still_encodes(self, path): raw.createDimension("x", 2) raw.createVariable("air", "f4", ("x",)) variable = _dataset.NetCDFDatasetVariable( - raw.variables["air"], str(path), write_lock_factory=None + raw.variables["air"], + str(path), + raw.file_format, + write_lock_factory=None, ) assert not isinstance( variable.variable, _bytecoding_datasets.EncodedVariable diff --git a/lib/iris/tests/unit/fileformats/netcdf/loader/ugrid_load/test_load_meshes.py b/lib/iris/tests/unit/fileformats/netcdf/loader/ugrid_load/test_load_meshes.py index 538554a428..1111034e53 100644 --- a/lib/iris/tests/unit/fileformats/netcdf/loader/ugrid_load/test_load_meshes.py +++ b/lib/iris/tests/unit/fileformats/netcdf/loader/ugrid_load/test_load_meshes.py @@ -168,7 +168,7 @@ def test_invalid_scheme(self): @_shared_utils.skip_data def test_non_nc(self, caplog): - log_regex = r"Ignoring non-NetCDF file:.*" + log_regex = r"Ignoring file of a format that holds no mesh:.*" with _shared_utils.assert_logs( caplog, logger, level="INFO", msg_regex=log_regex ): diff --git a/lib/iris/tests/unit/fileformats/zarr/__init__.py b/lib/iris/tests/unit/fileformats/zarr/__init__.py new file mode 100644 index 0000000000..2016f9f175 --- /dev/null +++ b/lib/iris/tests/unit/fileformats/zarr/__init__.py @@ -0,0 +1,14 @@ +# Copyright Iris contributors +# +# This file is part of Iris and is released under the BSD license. +# See LICENSE in the root of the repository for full licensing details. +"""Unit tests for :mod:`iris.fileformats.zarr`. + +``zarr`` is optional, so every module below this one is skipped when it is +absent. The skip lives here rather than in a ``conftest.py``: the same call +there raises during collection and is reported as an **error**, not a skip. +""" + +import pytest + +pytest.importorskip("zarr", minversion="3.0.8") diff --git a/lib/iris/tests/unit/fileformats/zarr/dataset/__init__.py b/lib/iris/tests/unit/fileformats/zarr/dataset/__init__.py new file mode 100644 index 0000000000..66f11b11ae --- /dev/null +++ b/lib/iris/tests/unit/fileformats/zarr/dataset/__init__.py @@ -0,0 +1,5 @@ +# Copyright Iris contributors +# +# This file is part of Iris and is released under the BSD license. +# See LICENSE in the root of the repository for full licensing details. +"""Unit tests for :class:`iris.fileformats.zarr._dataset.ZarrDatasetVariable`.""" diff --git a/lib/iris/tests/unit/fileformats/zarr/dataset/test_ZarrDataset.py b/lib/iris/tests/unit/fileformats/zarr/dataset/test_ZarrDataset.py new file mode 100644 index 0000000000..a742e45e84 --- /dev/null +++ b/lib/iris/tests/unit/fileformats/zarr/dataset/test_ZarrDataset.py @@ -0,0 +1,199 @@ +# Copyright Iris contributors +# +# This file is part of Iris and is released under the BSD license. +# See LICENSE in the root of the repository for full licensing details. +"""Unit tests for :class:`iris.fileformats.zarr._dataset.ZarrDataset`.""" + +import numpy as np +import pytest +import zarr + +from iris.fileformats.zarr._dataset import ZarrDataset +from iris.warnings import IrisCfLoadWarning + +from .. import stores + + +@pytest.fixture(params=[2, 3], ids=["v2", "v3"]) +def zarr_format(request): + return request.param + + +class Test_opening: + def test_opens_and_reports_itself(self, tmp_path, zarr_format): + location = stores.make_store(tmp_path / "s.zarr", zarr_format=zarr_format) + dataset = ZarrDataset(location) + + assert dataset.location == location + assert dataset.format == f"ZARR_V{zarr_format}" + assert dataset.mode == "r" + assert dataset.closed is False + assert sorted(dataset.variables) == ["air_temperature", "grid"] + assert dict(dataset.attributes) == stores.GLOBALS + assert dataset.storage_settings["zarr_format"] == zarr_format + + +class Test_attributes: + def test_nc_properties_is_not_a_cf_attribute(self, tmp_path): + """`_NCProperties` is netCDF-c's own bookkeeping, not CF metadata. + + It is the stamp netCDF-c writes onto the root group of every store it + creates (confirmed on NCZarr output), reading `version=...,netcdf=...`. + netCDF4-python hides it from `Dataset.ncattrs()` by convention, so no + cube loaded through netCDF has ever carried it; a cube loaded from the + same store through the native Zarr reader must not gain it either, or + the two would stop comparing equal over nothing more than provenance. + """ + group = zarr.open_group(store=tmp_path / "nc.zarr", mode="w", zarr_format=3) + group.attrs["Conventions"] = "CF-1.7" + group.attrs["_NCProperties"] = "version=2,netcdf=4.10.1,nczarr=2.0.0" + stores.make_array(group, "v", (2,), "f8", 3, dimensions=("t",)) + + dataset = ZarrDataset(str(tmp_path / "nc.zarr")) + assert "_NCProperties" in group.attrs + assert "_NCProperties" not in dataset.attributes + assert dict(dataset.attributes) == {"Conventions": "CF-1.7"} + + +class Test_dimensions: + def test_dimensions_are_assembled_from_the_arrays(self, tmp_path, zarr_format): + location = stores.make_store(tmp_path / "s.zarr", zarr_format=zarr_format) + assert dict(ZarrDataset(location).dimensions) == stores.DIMENSIONS + + def test_a_name_used_at_two_sizes_is_omitted_and_warns(self, tmp_path): + """Two arrays may share a dimension name at different lengths. + + The version 3 specification "does not place any restrictions on the use of + the same dimension name across multiple arrays", so this is conforming + Zarr and must not raise. Merging the two would produce a wrong cube. + + Opening the store is what reports it. The mapping is assembled in + `__init__` so that the warning reaches a load, which asks each variable + for its own `dimensions` and never asks the dataset for this mapping. + """ + group = zarr.open_group(store=tmp_path / "c.zarr", mode="w", zarr_format=3) + stores.make_array(group, "short", (2,), "f8", 3, dimensions=("t",)) + stores.make_array(group, "long", (5,), "f8", 3, dimensions=("t",)) + + with pytest.warns(IrisCfLoadWarning, match="t"): + dataset = ZarrDataset(str(tmp_path / "c.zarr")) + + dimensions = dict(dataset.dimensions) + assert "t" not in dimensions + # The arrays keep the name in their own dimensions: nothing is renamed or + # dropped, only the dataset-wide mapping declines to link them. + assert dataset.variables["short"].dimensions == ("t",) + assert dataset.variables["long"].dimensions == ("t",) + + def test_anonymous_dimensions_are_not_in_the_mapping(self, tmp_path): + group = zarr.open_group(store=tmp_path / "a.zarr", mode="w", zarr_format=3) + stores.make_array(group, "named", (2,), "f8", 3, dimensions=("t",)) + stores.make_array(group, "unnamed", (4,), "f8", 3) + + dataset = ZarrDataset(str(tmp_path / "a.zarr")) + assert dict(dataset.dimensions) == {"t": 2} + + +class Test_close: + def test_close_is_idempotent(self, tmp_path): + location = stores.make_store(tmp_path / "s.zarr") + dataset = ZarrDataset(location) + dataset.close() + dataset.close() + assert dataset.closed is True + + def test_context_manager_closes(self, tmp_path): + location = stores.make_store(tmp_path / "s.zarr") + dataset = ZarrDataset(location) + with dataset as entered: + assert entered is dataset + assert dataset.closed is True + + +class Test_groups: + def test_group_keyword_selects_a_subgroup(self, tmp_path): + root = zarr.open_group(store=tmp_path / "n.zarr", mode="w", zarr_format=3) + sub = root.create_group("measurements/r10m") + stores.make_array(sub, "b02", (2,), "u2", 3, dimensions=("p",)) + + dataset = ZarrDataset(str(tmp_path / "n.zarr"), group="measurements/r10m") + assert list(dataset.variables) == ["b02"] + assert dataset.location.endswith("measurements/r10m") + + def test_subgroup_names_lists_only_the_groups(self, tmp_path): + root = zarr.open_group(store=tmp_path / "n.zarr", mode="w", zarr_format=3) + root.create_group("conditions") + root.create_group("measurements") + stores.make_array(root, "v", (2,), "f8", 3) + + dataset = ZarrDataset(str(tmp_path / "n.zarr")) + assert sorted(dataset.subgroup_names) == ["conditions", "measurements"] + + def test_nested_groups_are_not_walked(self, tmp_path): + """Iris loads one group per call, like xarray. + + A Zarr store can legitimately hold unrelated datasets, and merging them + into one CubeList would be wrong. + """ + root = zarr.open_group(store=tmp_path / "n.zarr", mode="w", zarr_format=3) + stores.make_array(root, "top", (2,), "f8", 3) + stores.make_array(root.create_group("sub"), "inner", (2,), "f8", 3) + + assert list(ZarrDataset(str(tmp_path / "n.zarr")).variables) == ["top"] + + def test_a_missing_group_raises(self, tmp_path): + location = stores.make_store(tmp_path / "s.zarr") + with pytest.raises(zarr.errors.GroupNotFoundError): + ZarrDataset(location, group="nope") + + +class Test_the_write_surface: + def test_the_write_surface_refuses(self, tmp_path): + # Saving to Zarr is not implemented. The members exist because the + # interface declares them; refusing is the honest answer. + dataset = ZarrDataset(stores.make_store(tmp_path / "s.zarr")) + with pytest.raises(NotImplementedError, match="Saving"): + dataset.create_dimension("t", 3) + with pytest.raises(NotImplementedError, match="Saving"): + dataset.create_variable("v", np.dtype("f4"), ("t",)) + with pytest.raises(NotImplementedError, match="Saving"): + dataset.finalise() + + def test_a_write_mode_refuses(self, tmp_path): + stores.make_store(tmp_path / "s.zarr") + with pytest.raises(NotImplementedError, match="Saving"): + ZarrDataset(str(tmp_path / "s.zarr"), mode="w") + + +class Test_supports_dtype: + @pytest.mark.parametrize( + "dtype", ["i1", "i2", "i4", "i8", "u1", "u2", "u4", "u8", "f4", "f8", "c8"] + ) + def test_supports_every_dtype(self, tmp_path, zarr_format, dtype): + # The netCDF 3 formats have no 64-bit or unsigned integer type, which is + # what makes supports_dtype worth asking. Zarr has every one of them, in + # both versions, so the Zarr answer carries no format test. + dataset = ZarrDataset( + stores.make_store(tmp_path / "s.zarr", zarr_format=zarr_format) + ) + assert dataset.supports_dtype(np.dtype(dtype)) is True + + +class Test_locations: + def test_an_in_memory_store_is_just_another_location(self, tmp_path): + """A store need not be a directory. zarr-python decides what it is.""" + memory = zarr.storage.MemoryStore() + group = zarr.open_group(store=memory, mode="w", zarr_format=3) + stores.make_array(group, "v", (2,), "f8", 3, dimensions=("t",)) + + dataset = ZarrDataset(memory) + assert list(dataset.variables) == ["v"] + + def test_a_missing_store_raises_zarrs_own_error(self, tmp_path): + with pytest.raises(FileNotFoundError): + ZarrDataset(str(tmp_path / "nothing.zarr")) + + def test_a_directory_that_is_not_a_store_raises(self, tmp_path): + (tmp_path / "plain").mkdir() + with pytest.raises(zarr.errors.GroupNotFoundError): + ZarrDataset(str(tmp_path / "plain")) diff --git a/lib/iris/tests/unit/fileformats/zarr/dataset/test_ZarrDatasetVariable.py b/lib/iris/tests/unit/fileformats/zarr/dataset/test_ZarrDatasetVariable.py new file mode 100644 index 0000000000..268273bb28 --- /dev/null +++ b/lib/iris/tests/unit/fileformats/zarr/dataset/test_ZarrDatasetVariable.py @@ -0,0 +1,247 @@ +# Copyright Iris contributors +# +# This file is part of Iris and is released under the BSD license. +# See LICENSE in the root of the repository for full licensing details. +"""Unit tests for :class:`iris.fileformats.zarr._dataset.ZarrDatasetVariable`.""" + +import json +from types import SimpleNamespace + +import numpy as np +import pytest +import zarr + +from iris.exceptions import TranslationError +from iris.fileformats.zarr._dataset import ZarrDatasetVariable +from iris.warnings import IrisCfLoadWarning + +from .. import stores + + +def wrap(array, location="store.zarr"): + """Return the ZarrDatasetVariable over `array`, naming its own format.""" + zarr_format = array.metadata.zarr_format + return ZarrDatasetVariable(array, location, f"ZARR_V{zarr_format}") + + +@pytest.fixture(params=[2, 3], ids=["v2", "v3"]) +def zarr_format(request): + return request.param + + +@pytest.fixture +def group(tmp_path, zarr_format): + return zarr.open_group( + store=tmp_path / "probe.zarr", mode="w", zarr_format=zarr_format + ) + + +def test_named_dimensions(group, zarr_format): + array = stores.make_array( + group, "v", (3, 2), "f8", zarr_format, dimensions=("y", "x") + ) + assert wrap(array).dimensions == ("y", "x") + + +def test_unnamed_dimensions_are_anonymous(group, zarr_format): + array = stores.make_array(group, "v", (3, 2), "f8", zarr_format) + assert wrap(array).dimensions == (None, None) + + +def test_scalar_has_no_dimensions(group, zarr_format): + array = stores.make_array(group, "v", (), "i4", zarr_format) + assert wrap(array).dimensions == () + + +def test_partially_named_dimensions_version_3(tmp_path): + # The version 3 core specification: "an unnamed dimension is indicated by + # the null object". A store may name some dimensions and not others. + group = zarr.open_group(store=tmp_path / "p.zarr", mode="w", zarr_format=3) + array = group.create_array( + "v", shape=(3, 2), dtype="f8", dimension_names=("y", None) + ) + assert wrap(array).dimensions == ("y", None) + + +def test_repeated_name_within_one_array(tmp_path): + # "It is recommended but not required that all non-null dimension names + # are distinct." Recommended is not required, so this must not raise. + group = zarr.open_group(store=tmp_path / "p.zarr", mode="w", zarr_format=3) + array = group.create_array( + "v", shape=(2, 2), dtype="f8", dimension_names=("d", "d") + ) + assert wrap(array).dimensions == ("d", "d") + + +def test_an_unsupported_zarr_format_raises(): + """Iris names the Zarr versions it reads, and refuses others. + + The array is a stand-in because zarr-python cannot produce the input, + which is the point of the companion test below. + """ + array = SimpleNamespace( + ndim=1, + basename="v", + attrs={}, + metadata=SimpleNamespace(zarr_format=4, dimension_names=("t",)), + ) + with pytest.raises(TranslationError, match="declares Zarr version 4"): + wrap(array) + + +def test_zarr_python_rejects_an_unsupported_zarr_format(tmp_path): + """zarr-python is what makes the branch above unreachable in practice. + + It parses `zarr_format` as a `Literal` and raises on anything else - + `expected Literal[3], got 4` for an array in a version 3 group. If it + ever widens that, the branch above starts earning its keep. + """ + group = zarr.open_group(store=tmp_path / "v4.zarr", mode="w", zarr_format=3) + stores.make_array(group, "v", (2,), "f8", 3, dimensions=("t",)) + + metadata_path = tmp_path / "v4.zarr" / "v" / "zarr.json" + metadata = json.loads(metadata_path.read_text()) + metadata["zarr_format"] = 4 + metadata_path.write_text(json.dumps(metadata)) + + with pytest.raises(Exception, match="zarr_format"): + zarr.open_group(store=tmp_path / "v4.zarr", mode="r")["v"] + + +def test_wrong_length_dimension_names_warns_and_loads_anonymous(tmp_path): + """A dimension-name count that disagrees with `ndim` is a malformation. + + Version 2 only: the version 3 core specification says `dimension_names` + "must" be the same length as `shape`, and zarr-python enforces that on + every read, not just on write - a hand-edited `zarr.json` with the wrong + length raises `ValueError` from `Array.open` itself, before Iris sees an + array at all. `_ARRAY_DIMENSIONS` carries no such validation, so this is + reachable only through a version 2 store. + """ + group = zarr.open_group(store=tmp_path / "bad.zarr", mode="w", zarr_format=2) + array = stores.make_array(group, "v", (3, 2), "f8", 2, dimensions=("y", "x")) + array.attrs[stores.ARRAY_DIMENSIONS] = ["y"] + + with pytest.warns(IrisCfLoadWarning, match="dimension_names"): + dimensions = wrap(array).dimensions + assert dimensions == (None, None) + + +def test_plain_properties(tmp_path, zarr_format): + location = stores.make_store(tmp_path / "s.zarr", zarr_format=zarr_format) + array = zarr.open_group(store=location, mode="r")["air_temperature"] + variable = wrap(array, location) + + assert variable.name == "air_temperature" + assert variable.location == location + assert variable.format == f"ZARR_V{zarr_format}" + assert variable.shape == stores.DATA.shape + assert variable.dtype == np.dtype("f8") + assert variable.size == stores.DATA.size + assert variable.ndim == stores.DATA.ndim + assert len(variable) == stores.DATA.shape[0] + np.testing.assert_array_equal(variable[:], stores.DATA) + np.testing.assert_array_equal(variable[1], stores.DATA[1]) + + +def test_fill_value_is_the_storage_field(tmp_path, zarr_format): + """`fill_value` is what unwritten storage reads as, not `_FillValue`. + + zarr-python defaults the field to zero, so an array that declares no CF + sentinel still reports one here. Reading it as a mask source is what + masks every valid zero. + """ + group = zarr.open_group( + store=tmp_path / "f.zarr", mode="w", zarr_format=zarr_format + ) + array = stores.make_array(group, "v", (3,), "i2", zarr_format) + assert wrap(array).fill_value == 0 + + declared = stores.make_array(group, "w", (3,), "i2", zarr_format, fill_value=-999) + assert wrap(declared).fill_value == -999 + + +def test_chunking_is_the_inner_chunk_not_the_shard(tmp_path): + """A sharded array reports `Array.chunks`. + + Zarr's sharding codec is indexed: a reader fetches the shard index, then + byte ranges for only the inner chunks it needs. The inner chunk is the + independently decodable unit. + """ + group = zarr.open_group(store=tmp_path / "sh.zarr", mode="w", zarr_format=3) + array = group.create_array( + "v", shape=(1024,), dtype="i4", chunks=(8,), shards=(1024,) + ) + variable = wrap(array) + assert variable.chunking == (8,) + assert variable.storage_settings["shards"] == (1024,) + + +def test_storage_settings_reports_the_store(tmp_path, zarr_format): + location = stores.make_store(tmp_path / "e.zarr", zarr_format=zarr_format) + array = zarr.open_group(store=location, mode="r")["air_temperature"] + storage_settings = wrap(array, location).storage_settings + assert storage_settings["chunks"] == array.chunks + assert storage_settings["zarr_format"] == zarr_format + + +def test_array_dimensions_is_not_a_cf_attribute(tmp_path): + """`_ARRAY_DIMENSIONS` is storage, so it must not reach `attributes`. + + It is how a version 2 store records dimension names. Every attribute the + loading rules do not consume reaches `cube.attributes`, so leaving it in + would put it on every cube loaded from a version 2 store, and make that + cube unequal to the same data loaded from version 3. + """ + group = zarr.open_group(store=tmp_path / "v2.zarr", mode="w", zarr_format=2) + array = stores.make_array( + group, + "v", + (3, 2), + "f8", + 2, + dimensions=("y", "x"), + attributes={"units": "K"}, + ) + variable = wrap(array) + + assert variable.dimensions == ("y", "x") + assert stores.ARRAY_DIMENSIONS in array.attrs + assert stores.ARRAY_DIMENSIONS not in variable.attributes + assert dict(variable.attributes) == {"units": "K"} + + +def test_version_2_and_version_3_present_the_same_attributes(tmp_path): + v2 = stores.make_store(tmp_path / "a2.zarr", zarr_format=2) + v3 = stores.make_store(tmp_path / "a3.zarr", zarr_format=3) + from_2 = zarr.open_group(store=v2, mode="r")["air_temperature"] + from_3 = zarr.open_group(store=v3, mode="r")["air_temperature"] + assert dict(wrap(from_2, v2).attributes) == dict(wrap(from_3, v3).attributes) + + +def test_structured_attribute_values_are_carried_unchanged(tmp_path): + """A JSON object or array attribute is carried, not interpreted. + + The version 3 specification allows an attribute to be any JSON literal, + and CF cannot express a nested one at any rank. Two independent producers + write them - NOAA GFS gives `valid_time` a `statistics_approximate` + object, and the ESA EOPF product root carries `stac_discovery`. + A flat numeric array is an ordinary CF vector attribute and is not special. + """ + group = zarr.open_group(store=tmp_path / "n.zarr", mode="w", zarr_format=3) + array = stores.make_array( + group, + "v", + (2,), + "f8", + 3, + attributes={ + "statistics_approximate": {"min": 0.0, "max": 1.0}, + "valid_range": [0.0, 1.0], + "nested_list": [[1, 2], [3, 4]], + }, + ) + attributes = wrap(array).attributes + assert attributes["statistics_approximate"] == {"min": 0.0, "max": 1.0} + assert attributes["valid_range"] == [0.0, 1.0] + assert attributes["nested_list"] == [[1, 2], [3, 4]] diff --git a/lib/iris/tests/unit/fileformats/zarr/dataset/test_ZarrDataset_contract.py b/lib/iris/tests/unit/fileformats/zarr/dataset/test_ZarrDataset_contract.py new file mode 100644 index 0000000000..54a9ae8493 --- /dev/null +++ b/lib/iris/tests/unit/fileformats/zarr/dataset/test_ZarrDataset_contract.py @@ -0,0 +1,34 @@ +# Copyright Iris contributors +# +# This file is part of Iris and is released under the BSD license. +# See LICENSE in the root of the repository for full licensing details. +"""The shared CFDataset read contract, asked of the Zarr implementation. + +Only the read half: the Zarr write path does not exist, and `ZarrDataset` +refuses every member of the write surface. The write half of the contract +arrives with the saver. +""" + +import pytest + +from iris.fileformats.zarr._dataset import ZarrDataset +from iris.tests.unit.fileformats.cf.dataset import contract + +from .. import stores + + +class TestZarrDatasetVersion3(contract.CFDatasetReadContract): + chunking_key = "chunks" + + @pytest.fixture + def readable(self, tmp_path): + return ZarrDataset(stores.make_store(tmp_path / "contract.zarr")) + + +class TestZarrDatasetVersion2(contract.CFDatasetReadContract): + chunking_key = "chunks" + + @pytest.fixture + def readable(self, tmp_path): + location = stores.make_store(tmp_path / "contract.zarr", zarr_format=2) + return ZarrDataset(location) diff --git a/lib/iris/tests/unit/fileformats/zarr/dataset/test_read_data.py b/lib/iris/tests/unit/fileformats/zarr/dataset/test_read_data.py new file mode 100644 index 0000000000..d3c9f6990a --- /dev/null +++ b/lib/iris/tests/unit/fileformats/zarr/dataset/test_read_data.py @@ -0,0 +1,424 @@ +# Copyright Iris contributors +# +# This file is part of Iris and is released under the BSD license. +# See LICENSE in the root of the repository for full licensing details. +"""Unit tests for :meth:`ZarrDatasetVariable.read_data`.""" + +import dask +import dask.array as da +from dask.array.core import normalize_chunks +import dask.config +import numpy as np +import pytest +import zarr + +from iris import _lazy_data +from iris.fileformats.cf import loader as cf_loader +from iris.fileformats.zarr._dataset import ZarrDataset + +from .. import stores + + +def variable(location, name="air_temperature"): + return ZarrDataset(location).variables[name] + + +def policy(): + """The chunking policy a loader outside any CHUNK_CONTROL supplies.""" + return None, None + + +def dask_policy(): + """The chunking policy CHUNK_CONTROL.as_dask() supplies.""" + return "auto", None + + +def key_of(location, name="v"): + return ZarrDataset(location).variables[name].cache_key + + +class CountingStore(zarr.storage.WrapperStore): + """Wrap a store, adding every byte it returns to a caller-owned tally. + + zarr-python copies a store when it opens it read-only, through + `with_read_only` and `_with_store`, so a tally kept on the instance is + never the one the live store writes to, and the test passes vacuously. + """ + + def __init__(self, store, tally): + super().__init__(store=store) + self.tally = tally + + def _with_store(self, store): + return type(self)(store, self.tally) + + async def get(self, key, prototype, byte_range=None): + buffer = await self._store.get(key, prototype, byte_range) + if buffer is not None: + self.tally.append(len(buffer)) + return buffer + + async def get_partial_values(self, prototype, key_ranges): + buffers = await self._store.get_partial_values(prototype, key_ranges) + self.tally.extend(len(b) for b in buffers if b is not None) + return buffers + + +def bytes_to_read_first_chunk(location, dask_chunks): + """Return the bytes fetched reading the first eight values of "v".""" + tally = [] + store = CountingStore(zarr.storage.LocalStore(location), tally) + array = zarr.open_group(store=store, mode="r")["v"] + tally.clear() + lazy = da.from_array(array, chunks=dask_chunks, meta=np.empty((0,), "i4")) + lazy = da.map_blocks(lambda block: np.ma.masked_array(block), lazy, dtype="i4") + lazy[:8].compute() + return sum(tally) + + +def packed_store(tmp_path, shape, chunks): + """Write an int8 array that unpacks to float64, and return its location.""" + location = str(tmp_path / "p.zarr") + group = zarr.open_group(store=location, mode="w", zarr_format=3) + array = stores.make_array( + group, + "v", + shape, + "i1", + 3, + dimensions=tuple("yx"[: len(shape)]), + chunks=chunks, + attributes={"scale_factor": 0.5}, + ) + array[:] = 1 + return location + + +class Test_laziness: + def test_a_small_variable_is_read_whole(self, tmp_path): + location = stores.make_store(tmp_path / "s.zarr") + result = variable(location).read_data(policy) + assert not isinstance(result, da.Array) + np.testing.assert_array_equal(np.asarray(result), stores.DATA) + + def test_a_small_variable_never_consults_the_policy(self, tmp_path): + """Under CHUNK_CONTROL.from_file() the policy raises for an unchunked store. + + A variable small enough to read whole must reach its data without asking, + or `from_file` starts rejecting small cubes that load today. + """ + + def raising_policy(): + raise AssertionError("chunking policy consulted for a small variable") + + location = stores.make_store(tmp_path / "s.zarr") + variable(location).read_data(raising_policy) + + def test_a_large_variable_is_lazy(self, tmp_path, mocker): + mocker.patch.object(cf_loader, "_LAZYVAR_MIN_BYTES", 1) + location = stores.make_store(tmp_path / "s.zarr") + result = variable(location).read_data(policy) + assert isinstance(result, da.Array) + np.testing.assert_array_equal(result.compute(), stores.DATA) + + def test_a_plain_array_gets_an_unmasked_meta(self, tmp_path, mocker): + # No decode layer, so no masked meta: a masked meta with nothing beneath + # it is a meta that lies about its own graph. + mocker.patch.object(cf_loader, "_LAZYVAR_MIN_BYTES", 1) + location = stores.make_store(tmp_path / "s.zarr") + result = variable(location).read_data(policy) + assert not isinstance(result._meta, np.ma.MaskedArray) + + def test_a_decoded_array_gets_a_masked_meta(self, tmp_path, mocker): + mocker.patch.object(cf_loader, "_LAZYVAR_MIN_BYTES", 1) + group = zarr.open_group(store=tmp_path / "m.zarr", mode="w", zarr_format=3) + array = stores.make_array( + group, + "v", + (4,), + "i2", + 3, + dimensions=("t",), + attributes={"_FillValue": -999}, + ) + array[:] = [-999, 1, 2, 3] + + result = ZarrDataset(str(tmp_path / "m.zarr")).variables["v"].read_data(policy) + assert isinstance(result._meta, np.ma.MaskedArray) + realised = result.compute() + assert realised.mask.tolist() == [True, False, False, False] + + +class Test_cache_key: + def test_the_key_is_stable_across_a_reopen(self, tmp_path): + group = zarr.open_group(store=tmp_path / "k.zarr", mode="w", zarr_format=3) + stores.make_array(group, "v", (4,), "i4", 3, dimensions=("t",)) + location = str(tmp_path / "k.zarr") + assert key_of(location) == key_of(location) + + def test_the_key_changes_when_the_shape_does(self, tmp_path): + """A store rewritten in place keeps its address and changes its contents. + + Keyed on the address alone, an array reopened after its shape changed + from (4,) to (6,) returns the cached four values, silently truncating - + and that is weaker than the netCDF key it replaces, since + repr(NetCDFDataProxy) already carries shape and dtype. + """ + location = str(tmp_path / "k.zarr") + group = zarr.open_group(store=location, mode="w", zarr_format=3) + stores.make_array(group, "v", (4,), "i4", 3, dimensions=("t",), chunks=(2,)) + before = key_of(location) + + group = zarr.open_group(store=location, mode="a", zarr_format=3) + group.create_array( + "v", + shape=(6,), + dtype="i4", + chunks=(2,), + dimension_names=("t",), + overwrite=True, + ) + assert key_of(location) != before + + def test_the_key_changes_when_the_fill_value_does(self, tmp_path): + location = str(tmp_path / "k.zarr") + group = zarr.open_group(store=location, mode="w", zarr_format=3) + group.create_array("v", shape=(4,), dtype="i4", fill_value=-1) + before = key_of(location) + + group = zarr.open_group(store=location, mode="a", zarr_format=3) + group.create_array("v", shape=(4,), dtype="i4", fill_value=-2, overwrite=True) + assert key_of(location) != before + + @pytest.mark.parametrize( + "change", + [ + {"shape": (8,)}, + {"chunks": (4,)}, + {"shards": (8,)}, + {"dtype": "f8"}, + {"fill_value": -7}, + {"dimension_names": ("other",)}, + ], + ) + def test_the_key_is_sensitive_to_every_metadata_field(self, tmp_path, change): + """`Array.metadata.to_dict()` is zarr-python's rendering, not a guarantee. + + Nothing in the Zarr specification defines what that method returns, so + these are what fail if a release stops carrying a field the key needs. + """ + base = dict( + shape=(4,), + dtype="i4", + chunks=(2,), + fill_value=-1, + dimension_names=("t",), + shards=None, + ) + location = str(tmp_path / "k.zarr") + group = zarr.open_group(store=location, mode="w", zarr_format=3) + group.create_array("v", **base) + before = key_of(location) + + group = zarr.open_group(store=location, mode="a", zarr_format=3) + group.create_array("v", **{**base, **change}, overwrite=True) + assert key_of(location) != before + + def test_the_key_is_sensitive_to_attributes(self, tmp_path): + location = str(tmp_path / "k.zarr") + group = zarr.open_group(store=location, mode="w", zarr_format=3) + array = group.create_array("v", shape=(4,), dtype="i4") + before = key_of(location) + array.attrs["units"] = "K" + assert key_of(location) != before + + def test_two_cubes_sharing_an_array_share_one_dask_array(self, tmp_path, mocker): + """What the cache is for: many cubes in one store sharing a coordinate. + + Comparing the two Dask names proves nothing on its own: Dask tokenises a + zarr.Array deterministically, so two independently opened arrays get the + same name whether or not `read_data` passes a cache key at all. What has + to be shown is that the first read puts an entry in the process-wide + cache and the second read comes back out of it. + + The cache is replaced for the test rather than measured in place, because + it is process-wide and holds only its last hundred entries - a test that + counted the real one would depend on what ran before it. + """ + mocker.patch.object(cf_loader, "_LAZYVAR_MIN_BYTES", 1) + cache = _lazy_data.LRUCache(100) + mocker.patch.object(_lazy_data, "CACHE", cache) + location = stores.make_store(tmp_path / "s.zarr") + + first = variable(location).read_data(policy) + assert len(cache._cache) == 1, "the first read did not populate the cache" + + second = variable(location).read_data(policy) + assert len(cache._cache) == 1, "the second read built a second array" + assert first.name == second.name + + +class Test_sharding: + def test_a_sharded_array_is_read_by_the_inner_chunk(self, tmp_path): + """Shard-sized Dask blocks fetch the whole shard, because of the layer. + + Bare Dask slicing can push a slice down into the array's own __getitem__, + but a map_blocks layer in between cannot, so the block is materialised in + full. The absolute byte counts are zarr-version-dependent; the comparison + is not. + """ + location = str(tmp_path / "sh.zarr") + group = zarr.open_group(store=location, mode="w", zarr_format=3) + array = group.create_array( + "v", shape=(1024,), dtype="i4", chunks=(8,), shards=(1024,) + ) + array[:] = np.arange(1024, dtype="i4") + + inner = bytes_to_read_first_chunk(location, 8) + whole_shard = bytes_to_read_first_chunk(location, 1024) + assert inner < whole_shard + + # And the variable chooses the inner chunk for exactly this reason. + assert ZarrDataset(location).variables["v"].chunking == (8,) + + +class Test_declared_dtype: + def test_the_declared_dtype_matches_what_the_decode_layer_produces( + self, tmp_path, mocker + ): + """A packed variable's lazy dtype must equal its realised dtype. + + On the netCDF path netCDF4 does the arithmetic and reports the dtype. On + the Zarr path Iris does both, in two separately written statements, and + Dask does not check them against each other: a mismatch builds a graph + and surfaces later as a wrong dtype on the cube. + """ + mocker.patch.object(cf_loader, "_LAZYVAR_MIN_BYTES", 1) + group = zarr.open_group(store=tmp_path / "p.zarr", mode="w", zarr_format=3) + array = stores.make_array( + group, + "b02", + (4,), + "u2", + 3, + dimensions=("p",), + attributes={"scale_factor": 0.0001, "add_offset": -0.1}, + ) + array[:] = [1000, 2000, 3000, 4000] + + lazy = ZarrDataset(str(tmp_path / "p.zarr")).variables["b02"].read_data(policy) + realised = lazy.compute() + + assert lazy.dtype == realised.dtype + assert lazy.dtype == np.dtype("f8") + np.testing.assert_allclose(realised, [0.0, 0.1, 0.2, 0.3]) + + @pytest.mark.parametrize( + ("scale_factor", "expected"), + # A Zarr attribute is JSON, so a scale factor reaches Iris as a Python + # float or a Python int and never as a NumPy scalar - zarr-python refuses + # to write one. Both forms are here because they promote differently: + # a weak Python int leaves a uint16 variable uint16. + [(0.01, "f8"), (1, "u2")], + ) + def test_the_declared_dtype_matches_the_cf_loader_calculation( + self, tmp_path, scale_factor, expected + ): + """And it is the same answer `_get_actual_dtype` gives. + + The cube's data variable and its coordinates must agree about what a + packed value becomes, and they reach that answer through different code. + """ + group = zarr.open_group(store=tmp_path / "p.zarr", mode="w", zarr_format=3) + stores.make_array( + group, + "v", + (4,), + "u2", + 3, + attributes={"scale_factor": scale_factor}, + ) + variable = ZarrDataset(str(tmp_path / "p.zarr")).variables["v"] + from iris.fileformats.zarr import _decode + + parameters = _decode.decode_parameters( + variable.attributes, + dtype=variable.dtype, + storage_fill_value=variable.fill_value, + zarr_format=variable.zarr_format, + variable_name=variable.name, + ) + assert parameters["dtype"] == cf_loader._get_actual_dtype(variable) + assert parameters["dtype"] == np.dtype(expected) + + +class Test_chunking: + @pytest.mark.parametrize("chunking", ["default", "as_dask"]) + def test_chunks_are_sized_on_the_decoded_dtype(self, tmp_path, mocker, chunking): + """What a task holds is the unpacked block, not the stored one. + + An int8 variable unpacking to float64 is eight times its stored size the + moment the decode layer runs, so chunks sized on the stored dtype hand a + task eight times the memory Dask was asked to target. The netCDF path + sizes on the decoded dtype, its data proxy declaring it, and this one has + to agree - under `CHUNK_CONTROL.as_dask()` as well as by default, since + `da.from_array` would otherwise size on the array it is given, which is + the stored one. + """ + mocker.patch.object(cf_loader, "_LAZYVAR_MIN_BYTES", 1) + location = packed_store(tmp_path, (64, 64), (4, 64)) + chunking_policy = {"default": policy, "as_dask": dask_policy}[chunking] + + target = 2048 + with dask.config.set({"array.chunk-size": f"{target}B"}): + lazy = ZarrDataset(location).variables["v"].read_data(chunking_policy) + + assert lazy.dtype == np.dtype("f8") + block_bytes = int(np.prod(lazy.chunksize)) * lazy.dtype.itemsize + assert block_bytes <= target + + def test_as_dask_is_still_dasks_own_answer(self, tmp_path, mocker): + """Calling Dask's policy by hand must not turn into replacing it. + + Sizing on the decoded dtype means calling `normalize_chunks` here rather + than leaving it to `from_array`, and the risk in doing that is Iris's own + sizing or store alignment creeping back in on top. So: the result is what + Dask's policy gives for the decoded dtype, and the store's chunking of 7 + leaves no mark on it. + """ + mocker.patch.object(cf_loader, "_LAZYVAR_MIN_BYTES", 1) + location = packed_store(tmp_path, (64, 64), (7, 64)) + + with dask.config.set({"array.chunk-size": "2048B"}): + lazy = ZarrDataset(location).variables["v"].read_data(dask_policy) + # Inside the context: Dask's policy reads the configured target. + expected = normalize_chunks("auto", shape=(64, 64), dtype=np.dtype("f8")) + + assert lazy.chunksize == tuple(sizes[0] for sizes in expected) + assert lazy.chunksize[0] % 7 != 0 + + def test_a_lazy_read_divides_a_store_chunk_to_meet_dasks_target( + self, tmp_path, mocker + ): + """Chunking follows Dask's target, even where that divides a store chunk. + + Iris used to round back up to whole store chunks here. Dividing one costs + a re-read per division, because a chunk is fetched and decompressed + whole - but netCDF has always made the same trade, and which side of it a + caller wants depends on their case. `CHUNK_CONTROL` is how they say, and + `iris.fileformats.cf.loader` warns so that they know to. + """ + mocker.patch.object(cf_loader, "_LAZYVAR_MIN_BYTES", 1) + location = str(tmp_path / "b.zarr") + group = zarr.open_group(store=location, mode="w", zarr_format=3) + array = group.create_array( + "v", shape=(64,), dtype="i4", chunks=(16,), dimension_names=("t",) + ) + array[:] = np.arange(64, dtype="i4") + + with dask.config.set({"array.chunk-size": "32B"}): + lazy = ZarrDataset(location).variables["v"].read_data(policy) + + # 32 bytes of int32 is 8 elements, half the store's chunk of 16. + assert lazy.chunksize[0] == 8 + np.testing.assert_array_equal(lazy.compute(), np.arange(64)) diff --git a/lib/iris/tests/unit/fileformats/zarr/loader/__init__.py b/lib/iris/tests/unit/fileformats/zarr/loader/__init__.py new file mode 100644 index 0000000000..dc37340f8b --- /dev/null +++ b/lib/iris/tests/unit/fileformats/zarr/loader/__init__.py @@ -0,0 +1,5 @@ +# Copyright Iris contributors +# +# This file is part of Iris and is released under the BSD license. +# See LICENSE in the root of the repository for full licensing details. +"""Unit tests for :func:`iris.fileformats.zarr.load_cubes`.""" diff --git a/lib/iris/tests/unit/fileformats/zarr/loader/test_load_cubes.py b/lib/iris/tests/unit/fileformats/zarr/loader/test_load_cubes.py new file mode 100644 index 0000000000..09a56862cf --- /dev/null +++ b/lib/iris/tests/unit/fileformats/zarr/loader/test_load_cubes.py @@ -0,0 +1,407 @@ +# Copyright Iris contributors +# +# This file is part of Iris and is released under the BSD license. +# See LICENSE in the root of the repository for full licensing details. +"""Unit tests for :func:`iris.fileformats.zarr.load_cubes`.""" + +import numpy as np +import pytest +import zarr + +import iris +from iris.cube import Cube +from iris.exceptions import TranslationError +from iris.fileformats.zarr import load_cubes +from iris.loading import LOAD_PROBLEMS +from iris.warnings import IrisCfLoadWarning + +from .. import stores + + +@pytest.fixture(autouse=True) +def _assert_clean_load(): + """Every load in this file must leave `LOAD_PROBLEMS` empty. + + A malformed grid mapping on the shared `stores.make_store` fixture once + made every test that loaded through it record a load problem and raise + the standing `IrisLoadWarning` that announces one - silently covering the + one signal that would reveal a genuine Zarr load problem. This guards + against that recurring, for every test in this file, not only the ones + that use the canonical fixture. + """ + LOAD_PROBLEMS.reset() + yield + assert LOAD_PROBLEMS.problems == [] + + +def test_the_canonical_store_loads_its_one_data_variable(tmp_path): + location = stores.make_store(tmp_path / "s.zarr") + cubes = list(load_cubes(location)) + assert len(cubes) == 1 + cube = cubes[0] + assert isinstance(cube, Cube) + assert cube.var_name == "air_temperature" + assert cube.units == "K" + np.testing.assert_array_equal(cube.data, stores.DATA) + + +def test_one_cube_per_data_variable(tmp_path): + root = zarr.open_group(store=tmp_path / "two.zarr", mode="w", zarr_format=3) + for name in ("first", "second", "third"): + array = stores.make_array( + root, name, (2,), "f8", 3, dimensions=("p",), attributes={"units": "1"} + ) + array[:] = [1.0, 2.0] + + cubes = list(load_cubes(str(tmp_path / "two.zarr"))) + assert sorted(cube.var_name for cube in cubes) == ["first", "second", "third"] + + +def test_a_version_2_store_loads_the_same_cube(tmp_path): + v3 = list(load_cubes(stores.make_store(tmp_path / "v3.zarr")))[0] + v2 = list(load_cubes(stores.make_store(tmp_path / "v2.zarr", zarr_format=2)))[0] + assert v2 == v3 + + +def test_a_name_constraint_skips_variables_before_they_are_built(tmp_path, mocker): + """`constraints` is a pre-build filter, not a post-build one. + + `_translate_constraints_to_var_callback` only answers for a bare + `NameConstraint`; any other constraint returns None and every variable is + built. The spy is what distinguishes "filtered early" from "built then + discarded" - the cube list alone cannot. + """ + from iris.fileformats.zarr import loader as zarr_loader + + root = zarr.open_group(store=tmp_path / "two.zarr", mode="w", zarr_format=3) + for name in ("wanted", "unwanted"): + array = stores.make_array( + root, name, (2,), "f8", 3, dimensions=("p",), attributes={"units": "1"} + ) + array[:] = [1.0, 2.0] + + # The loader binds `_load_cube` at import, so the spy must go on its own + # module reference rather than on `cf.loader`'s. + spy = mocker.spy(zarr_loader, "_load_cube") + constraint = iris.NameConstraint(var_name="wanted") + cubes = list(load_cubes(str(tmp_path / "two.zarr"), constraints=constraint)) + + assert [cube.var_name for cube in cubes] == ["wanted"] + assert spy.call_count == 1 + + +def test_a_callback_can_reject_a_cube(tmp_path): + from iris.exceptions import IgnoreCubeException + + def reject(cube, field, filename): + raise IgnoreCubeException + + location = stores.make_store(tmp_path / "s.zarr") + assert list(load_cubes(location, callback=reject)) == [] + + +def test_the_group_keyword_selects_a_group(tmp_path): + root = zarr.open_group(store=tmp_path / "n.zarr", mode="w", zarr_format=3) + sub = root.create_group("measurements") + band = stores.make_array( + sub, "b02", (3,), "u2", 3, dimensions=("p",), attributes={"units": "1"} + ) + band[:] = [1, 2, 3] + + cubes = list(load_cubes(str(tmp_path / "n.zarr"), group="measurements")) + assert [cube.var_name for cube in cubes] == ["b02"] + + +def test_several_stores_in_one_call(tmp_path): + first = stores.make_store(tmp_path / "a.zarr") + second = stores.make_store(tmp_path / "b.zarr") + assert len(list(load_cubes([first, second]))) == 2 + + +def test_a_group_with_no_arrays_names_the_groups_that_have_them(tmp_path): + """A real store in this shape exists, and silence is the worst answer. + + The ESA EOPF Sentinel-2 products hold no arrays in the root group, only + subgroups - `conditions/geometry`, `measurements/reflectance/r10m` and + others. Iris loads one group per call, so loading the default root yields + nothing; returning an empty CubeList from a store that plainly holds data + leaves the user with no way to find out why. + """ + root = zarr.open_group(store=tmp_path / "e.zarr", mode="w", zarr_format=3) + root.create_group("conditions") + measurements = root.create_group("measurements") + stores.make_array(measurements, "b02", (2,), "u2", 3, dimensions=("p",)) + + with pytest.raises(TranslationError, match="measurements"): + list(load_cubes(str(tmp_path / "e.zarr"))) + + +def test_the_message_says_so_when_there_are_no_groups_either(tmp_path): + zarr.open_group(store=tmp_path / "empty.zarr", mode="w", zarr_format=3) + with pytest.raises(TranslationError, match="no data variables"): + list(load_cubes(str(tmp_path / "empty.zarr"))) + + +def test_nested_groups_are_not_walked(tmp_path): + """Iris loads one group per call, like xarray. + + A Zarr store can legitimately hold unrelated datasets, and merging them + into one CubeList would be wrong. + """ + root = zarr.open_group(store=tmp_path / "n.zarr", mode="w", zarr_format=3) + top = stores.make_array( + root, "top", (2,), "f8", 3, dimensions=("p",), attributes={"units": "1"} + ) + top[:] = [1.0, 2.0] + inner = stores.make_array( + root.create_group("sub"), "inner", (2,), "f8", 3, dimensions=("p",) + ) + inner[:] = [3.0, 4.0] + + cubes = list(load_cubes(str(tmp_path / "n.zarr"))) + assert [cube.var_name for cube in cubes] == ["top"] + + +def test_the_cube_is_lazy(tmp_path, mocker): + from iris.fileformats.cf import loader as cf_loader + + mocker.patch.object(cf_loader, "_LAZYVAR_MIN_BYTES", 1) + cube = list(load_cubes(stores.make_store(tmp_path / "s.zarr")))[0] + assert cube.has_lazy_data() + + +def test_the_grid_mapping_variable_does_not_become_a_cube(tmp_path): + cubes = list(load_cubes(stores.make_store(tmp_path / "s.zarr"))) + assert "grid" not in [cube.var_name for cube in cubes] + + +def test_cf_reserved_attributes_do_not_reach_the_cube(tmp_path): + """`units` becomes a property, so it must not also be an attribute. + + Which attributes survive is decided by which ones the loading rules did + not read, so an attribute Iris consumes but fails to record as read + arrives on the cube as well. + """ + cube = list(load_cubes(stores.make_store(tmp_path / "s.zarr")))[0] + assert "units" not in cube.attributes + assert "long_name" not in cube.attributes + assert cube.attributes["Conventions"] == "CF-1.7" + + +def test_a_structured_attribute_reaches_the_cube_unchanged(tmp_path): + """CF cannot express a nested attribute; Zarr allows one. Carry it. + + Two independent producers write them - NOAA GFS gives `valid_time` a + `statistics_approximate` object, and the ESA EOPF product root carries + `stac_discovery`. Interpreting them needs the attribute-model discussion + in SciTools/iris#7288; dropping them loses the publisher's metadata. + """ + group = zarr.open_group(store=tmp_path / "n.zarr", mode="w", zarr_format=3) + array = stores.make_array( + group, + "v", + (2,), + "f8", + 3, + dimensions=("p",), + attributes={"stac_discovery": {"id": "abc"}}, + ) + array[:] = [1.0, 2.0] + + cube = list(load_cubes(str(tmp_path / "n.zarr")))[0] + assert cube.attributes["stac_discovery"] == {"id": "abc"} + + +def test_a_structured_attribute_cannot_then_be_saved_to_netcdf(tmp_path): + """Known limitation, characterised so that it is not a surprise. + + Zarr allows any JSON literal as an attribute value and netCDF does not, + so a store carrying one produces a cube Iris cannot save as netCDF. + Checking attribute content before saving is SciTools/iris#7329, and + fixing it means changing the netCDF saver. Strip the attribute, or save + to Zarr once that is supported. + """ + import iris + + group = zarr.open_group(store=tmp_path / "n.zarr", mode="w", zarr_format=3) + array = stores.make_array( + group, + "v", + (2,), + "f8", + 3, + dimensions=("p",), + attributes={"stac_discovery": {"id": "abc"}}, + ) + array[:] = [1.0, 2.0] + cube = list(load_cubes(str(tmp_path / "n.zarr")))[0] + + with pytest.raises(TypeError, match="stac_discovery"): + iris.save(cube, str(tmp_path / "out.nc")) + + # And the documented workaround works. + del cube.attributes["stac_discovery"] + iris.save(cube, str(tmp_path / "out.nc")) + + +def test_a_dimension_name_reused_at_two_lengths_still_places_a_coordinate(tmp_path): + """Zarr dimension names carry none of netCDF's uniqueness guarantee. + + A data variable may name both of its dimensions `d` at two lengths, which + makes the name alone useless for choosing an axis. `(name, size)` names + one axis here, and the coordinate belongs on it; matching on the bare name + picks the first `d` and the coordinate is dropped as the wrong length. + """ + group = zarr.open_group(store=tmp_path / "r.zarr", mode="w", zarr_format=3) + data = stores.make_array( + group, + "v", + (2, 3), + "f8", + 3, + dimensions=("d", "d"), + attributes={"units": "1", "coordinates": "lat"}, + ) + data[:] = np.zeros((2, 3)) + lat = stores.make_array( + group, + "lat", + (3,), + "f8", + 3, + dimensions=("d",), + attributes={"units": "degrees_north", "standard_name": "latitude"}, + ) + lat[:] = [10.0, 20.0, 30.0] + + cube = list(load_cubes(str(tmp_path / "r.zarr")))[0] + assert cube.coord_dims("latitude") == (1,) + + +def test_version_3_zeros_are_not_masked(tmp_path): + """An int16 [0, 1, 2] with no CF attributes loads with nothing masked. + + zarr-python defaults `fill_value` to zero, so reading the storage field + as a sentinel masks valid data in the ordinary case. + """ + group = zarr.open_group(store=tmp_path / "z3.zarr", mode="w", zarr_format=3) + array = stores.make_array( + group, "v", (3,), "i2", 3, dimensions=("p",), attributes={"units": "1"} + ) + array[:] = [0, 1, 2] + + cube = list(load_cubes(str(tmp_path / "z3.zarr")))[0] + assert not np.ma.is_masked(cube.data) + np.testing.assert_array_equal(cube.data, [0, 1, 2]) + + +def test_version_2_zeros_are_masked(tmp_path): + """The same array as version 2 does use `fill_value` as the mask source. + + Version 2 stores predate the CF `_FillValue` convention, and this is what + their writers meant - which is what released xarray's + `use_zarr_fill_value_as_mask` does. Iris warns about exactly this, since + zarr-python 3 defaults a version 2 store's `fill_value` to zero too, and a + store written today can lose a genuine zero this way. + """ + group = zarr.open_group(store=tmp_path / "z2.zarr", mode="w", zarr_format=2) + array = stores.make_array( + group, "v", (3,), "i2", 2, dimensions=("p",), attributes={"units": "1"} + ) + array[:] = [0, 1, 2] + + with pytest.warns(IrisCfLoadWarning, match="v"): + cube = list(load_cubes(str(tmp_path / "z2.zarr")))[0] + assert cube.data.mask.tolist() == [True, False, False] + + +def test_a_cf_fill_value_beats_the_storage_one(tmp_path): + group = zarr.open_group(store=tmp_path / "z.zarr", mode="w", zarr_format=3) + array = stores.make_array( + group, + "v", + (3,), + "i2", + 3, + dimensions=("p",), + fill_value=0, + attributes={"units": "1", "_FillValue": -999}, + ) + array[:] = [0, -999, 2] + + cube = list(load_cubes(str(tmp_path / "z.zarr")))[0] + assert cube.data.mask.tolist() == [False, True, False] + + +class TestMesh: + """UGRID data, which a Zarr store holds exactly as a netCDF file does. + + Nothing in `iris.fileformats.netcdf.ugrid_load` is netCDF-specific: it + reads a `CFReader`, and a `CFReader` over a `ZarrDataset` answers the + same questions. These assert that, rather than re-testing the mesh + machinery itself. + """ + + @pytest.fixture + def cube(self, tmp_path): + location = stores.make_mesh_store(tmp_path / "mesh.zarr") + return list(load_cubes(location))[0] + + def test_the_cube_carries_a_mesh(self, cube): + assert cube.mesh is not None + assert cube.mesh.var_name == "mesh" + assert cube.mesh.topology_dimension == 2 + + def test_the_mesh_is_on_the_right_dimension(self, cube): + assert cube.mesh_dim() == cube.coord_dims("time")[0] + 1 + + def test_the_mesh_coords_are_attached(self, cube): + names = sorted(coord.name() for coord in cube.coords(mesh_coords=True)) + assert names == ["latitude", "longitude"] + + def test_the_connectivity_is_read(self, cube): + connectivity = cube.mesh.face_node_connectivity + np.testing.assert_array_equal( + connectivity.indices, stores.MESH_ARRAYS["face_nodes"][0] + ) + + def test_mesh_attributes_do_not_also_reach_the_cube(self, cube): + """`mesh` and `location` are consumed by the mesh build, not carried. + + They reach the cube as stray attributes if the MeshCoords are built + after it rather than before - which is why `_prepare_mesh_coords` and + `_attach_mesh_coords` are two calls with `_load_cube` between them. + """ + assert "mesh" not in cube.attributes + assert "location" not in cube.attributes + + def test_a_version_2_store_gives_the_same_mesh(self, tmp_path): + v3 = list(load_cubes(stores.make_mesh_store(tmp_path / "v3.zarr")))[0] + v2 = list( + load_cubes(stores.make_mesh_store(tmp_path / "v2.zarr", zarr_format=2)) + )[0] + assert v2.mesh == v3.mesh + assert v2 == v3 + + def test_load_meshes_reads_a_zarr_store(self, tmp_path): + """`iris.mesh.load_meshes` dispatches on format, so it reaches Zarr too. + + Its dispatch is a second copy of the main loader's; the two are meant + to move together. + """ + location = stores.make_mesh_store(tmp_path / "mesh.zarr") + meshes = iris.mesh.load_meshes(location) + assert list(meshes) == [location] + assert [mesh.var_name for mesh in meshes[location]] == ["mesh"] + + def test_a_store_naming_an_absent_mesh_still_loads(self, tmp_path): + """A dangling `mesh` reference is logged, not raised. + + Same treatment as netCDF: the cube is worth having without its mesh. + """ + location = stores.make_mesh_store(tmp_path / "mesh.zarr") + group = zarr.open_group(store=location, mode="r+") + group["temp"].attrs["mesh"] = "absent" + + cube = list(load_cubes(location))[0] + assert cube.mesh is None diff --git a/lib/iris/tests/unit/fileformats/zarr/stores.py b/lib/iris/tests/unit/fileformats/zarr/stores.py new file mode 100644 index 0000000000..7986a06821 --- /dev/null +++ b/lib/iris/tests/unit/fileformats/zarr/stores.py @@ -0,0 +1,224 @@ +# Copyright Iris contributors +# +# This file is part of Iris and is released under the BSD license. +# See LICENSE in the root of the repository for full licensing details. +"""Builders for the small Zarr stores these tests ask questions of. + +One canonical store, built from the constants the shared CFDataset contract +writes, so that a Zarr answer and a netCDF answer can be compared without +translating between two sets of fixtures - and so that the two cannot drift +apart. +""" + +import numpy as np +import zarr + +from iris.tests.unit.fileformats.cf.dataset.contract import ( + CONTRACT_DATA as DATA, +) +from iris.tests.unit.fileformats.cf.dataset.contract import ( + CONTRACT_DIMENSIONS as DIMENSIONS, +) +from iris.tests.unit.fileformats.cf.dataset.contract import ( + CONTRACT_GLOBALS as GLOBALS, +) +from iris.tests.unit.fileformats.cf.dataset.contract import ( + CONTRACT_VARIABLE_ATTRS, +) + +#: The canonical store's variable attributes. +#: +#: The contract's own, plus the `grid_mapping` that makes the `grid` array +#: below mean something. The contract checks its attributes are present +#: rather than that they are all there is, so the extra is legitimate. +VARIABLE_ATTRS = {**CONTRACT_VARIABLE_ATTRS, "grid_mapping": "grid"} + +#: The version 2 attribute carrying dimension names. +ARRAY_DIMENSIONS = "_ARRAY_DIMENSIONS" + +#: `make_array`'s own default for `fill_value`, distinct from `None`. +#: +#: `Group.create_array` treats an explicit `fill_value=None` as "this array +#: has no fill value" on a version 2 store, which is not the same thing as +#: "the caller did not ask for one". Passing this sentinel through instead +#: means `make_array`'s own default lets zarr-python choose its own default +#: fill value, on both versions, the way omitting the argument would. +_DEFAULT_FILL_VALUE = object() + + +def make_array( + group, + name, + shape, + dtype, + zarr_format, + *, + dimensions=None, + chunks="auto", + shards=None, + fill_value=_DEFAULT_FILL_VALUE, + attributes=None, +): + """Create one array, recording dimension names the way its version does. + + zarr-python refuses `dimension_names` on a version 2 array, so version 2 + names are written to the `_ARRAY_DIMENSIONS` attribute by hand - which is + what xarray and NCZarr both do. + """ + extra = {} + if zarr_format == 3 and dimensions is not None: + extra["dimension_names"] = dimensions + if fill_value is not _DEFAULT_FILL_VALUE: + extra["fill_value"] = fill_value + array = group.create_array( + name, + shape=shape, + dtype=dtype, + chunks=chunks, + shards=shards, + **extra, + ) + if zarr_format == 2 and dimensions is not None: + array.attrs[ARRAY_DIMENSIONS] = list(dimensions) + for key, value in (attributes or {}).items(): + array.attrs[key] = value + return array + + +def make_store(path, *, zarr_format=3, chunks="auto", shards=None): + """Write the canonical store at `path`, and return `path` as a string.""" + root = zarr.open_group(store=path, mode="w", zarr_format=zarr_format) + for name, value in GLOBALS.items(): + root.attrs[name] = value + + array = make_array( + root, + "air_temperature", + DATA.shape, + "f8", + zarr_format, + dimensions=tuple(DIMENSIONS), + chunks=chunks, + shards=shards, + fill_value=-1.0, + attributes=VARIABLE_ATTRS, + ) + array[:] = DATA + + # A dimensionless variable: what a grid mapping is, and the one shape a + # CFDatasetVariable must handle without a leading dimension. + scalar = make_array( + root, + "grid", + (), + "i4", + zarr_format, + dimensions=(), + attributes={ + "grid_mapping_name": "latitude_longitude", + "earth_radius": 6371229.0, + }, + ) + scalar[()] = 0 + + return str(path) + + +#: The UGRID store's mesh topology variable, as CF attributes. +MESH_ATTRS = { + "cf_role": "mesh_topology", + "topology_dimension": 2, + "node_coordinates": "node_x node_y", + "face_node_connectivity": "face_nodes", + "face_coordinates": "face_x face_y", + "long_name": "topology of a 2D mesh", +} + +#: The UGRID store's arrays: name -> (values, dimension names, attributes). +#: +#: Two quadrilateral faces over six nodes, with a time dimension on the data +#: so that the mesh dimension is not the only one. Every dimension is a +#: different size, so a coordinate cannot be attached to the right axis by +#: accident - see `_DimensionIdentity`. +MESH_ARRAYS = { + "node_x": ( + np.array([0.0, 1.0, 2.0, 0.0, 1.0, 2.0]), + ("node",), + {"standard_name": "longitude", "units": "degrees_east"}, + ), + "node_y": ( + np.array([0.0, 0.0, 0.0, 1.0, 1.0, 1.0]), + ("node",), + {"standard_name": "latitude", "units": "degrees_north"}, + ), + "face_x": ( + np.array([0.5, 1.5]), + ("face",), + {"standard_name": "longitude", "units": "degrees_east"}, + ), + "face_y": ( + np.array([0.5, 0.5]), + ("face",), + {"standard_name": "latitude", "units": "degrees_north"}, + ), + "face_nodes": ( + np.array([[0, 1, 4, 3], [1, 2, 5, 4]], dtype="i4"), + ("face", "vertex"), + {"cf_role": "face_node_connectivity", "start_index": 0}, + ), + "time": ( + np.array([0.0, 1.0, 2.0]), + ("time",), + {"standard_name": "time", "units": "days since 2000-01-01"}, + ), + "temp": ( + np.arange(6, dtype="f8").reshape(3, 2), + ("time", "face"), + { + "standard_name": "air_temperature", + "units": "K", + "mesh": "mesh", + "location": "face", + "coordinates": "face_x face_y", + }, + ), +} + + +def make_mesh_store(path, *, zarr_format=3): + """Write a UGRID store at `path`, and return `path` as a string. + + The same topology this module's netCDF counterpart writes, so that the + two backends' meshes can be compared directly. + + Every array is written with no fill value. A mesh is full of legitimate + zeros - node coordinates on an axis, and connectivity indices under + `start_index = 0` - and on a version 2 store zarr-python's default fill + value is zero, which Iris reads as "mask the zeros". That rule has its + own test; a mesh fixture that tripped it would be testing it twice and + reading its own topology back full of masks. + """ + root = zarr.open_group(store=path, mode="w", zarr_format=zarr_format) + root.attrs["Conventions"] = "CF-1.7, UGRID-1.0" + + mesh = make_array( + root, "mesh", (), "i4", zarr_format, dimensions=(), fill_value=None + ) + for key, value in MESH_ATTRS.items(): + mesh.attrs[key] = value + mesh[()] = 0 + + for name, (values, dimensions, attributes) in MESH_ARRAYS.items(): + array = make_array( + root, + name, + values.shape, + values.dtype, + zarr_format, + dimensions=dimensions, + fill_value=None, + attributes=attributes, + ) + array[:] = values + + return str(path) diff --git a/lib/iris/tests/unit/fileformats/zarr/test_CFReader.py b/lib/iris/tests/unit/fileformats/zarr/test_CFReader.py new file mode 100644 index 0000000000..e318d9ec05 --- /dev/null +++ b/lib/iris/tests/unit/fileformats/zarr/test_CFReader.py @@ -0,0 +1,19 @@ +# Copyright Iris contributors +# +# This file is part of Iris and is released under the BSD license. +# See LICENSE in the root of the repository for full licensing details. +"""The CF layer reads a Zarr store through the same reader netCDF uses.""" + +from iris.fileformats.cf import CFReader +from iris.fileformats.zarr._dataset import ZarrDataset + +from . import stores + + +def test_the_cf_reader_classifies_a_zarr_store(tmp_path): + location = stores.make_store(tmp_path / "s.zarr") + with CFReader(ZarrDataset(location)) as reader: + assert "air_temperature" in reader.cf_group.data_variables + assert "grid" in reader.cf_group.grid_mappings + assert reader.cf_group.global_attributes["title"] == "contract" + assert reader.filename == location diff --git a/lib/iris/tests/unit/fileformats/zarr/test__decode.py b/lib/iris/tests/unit/fileformats/zarr/test__decode.py new file mode 100644 index 0000000000..30dcd3608f --- /dev/null +++ b/lib/iris/tests/unit/fileformats/zarr/test__decode.py @@ -0,0 +1,266 @@ +# Copyright Iris contributors +# +# This file is part of Iris and is released under the BSD license. +# See LICENSE in the root of the repository for full licensing details. +"""Unit tests for :mod:`iris.fileformats.zarr._decode`.""" + +import base64 +import struct + +import numpy as np +import pytest + +from iris.fileformats.zarr import _decode +from iris.warnings import IrisCfLoadWarning + + +def base64_double(value): + """Return `value` in the encoding released xarray writes for a float.""" + return base64.b64encode(struct.pack("l", 512) + b"\x00" * 600) + + with open(path, "rb") as handle: + spec = FORMAT_AGENT.get_spec(path.name, handle) + assert spec.name == "NIMROD" + + +def test_a_version_3_store_is_recognised(tmp_path): + store = tmp_path / "v3.zarr" + store.mkdir() + (store / "zarr.json").write_text('{"zarr_format": 3, "node_type": "group"}') + assert _is_zarr_store(str(store)) is True + + +def test_a_version_2_store_is_recognised(tmp_path): + store = tmp_path / "v2.zarr" + store.mkdir() + (store / ".zgroup").write_text('{"zarr_format": 2}') + assert _is_zarr_store(str(store)) is True + + +def test_a_directory_named_zarr_with_no_marker_is_not_a_store(tmp_path): + """The suffix alone is not the local test, and must not become one. + + Reading the marker file is what distinguishes a real store from a + directory someone happened to name `.zarr`, and locally that costs one + stat. + """ + store = tmp_path / "pretend.zarr" + store.mkdir() + assert _is_zarr_store(str(store)) is False + + +def test_a_plain_file_is_not_a_store(tmp_path): + path = tmp_path / "data.nc" + path.write_bytes(b"CDF\x01") + assert _is_zarr_store(str(path)) is False + + +@pytest.mark.parametrize("scheme", ["s3", "gs", "az", "abfs"]) +def test_a_remote_store_is_recognised_by_scheme_and_suffix(scheme): + assert _is_zarr_store(f"{scheme}://bucket/path/data.zarr") is True + assert _is_zarr_store(f"{scheme}://bucket/path/data.zarr/") is True + + +def test_a_remote_url_without_the_suffix_is_not_a_store(): + # Remotely the suffix is all there is to go on: reading a marker file + # would cost a network round trip before Iris knows it wants the target. + assert _is_zarr_store("s3://bucket/path/data") is False + + +def test_an_http_url_is_not_a_store(): + # OPeNDAP, which the netCDF loader already claims. + assert _is_zarr_store("https://example.invalid/data.zarr") is False + + +def test_an_nczarr_url_is_not_a_native_store(tmp_path): + """A user who writes `#mode=nczarr` has asked for NCZarr by name. + + The store on disk is a real one, so the marker-file test says yes unless + the fragment is checked first - and `load_http` passes the fragment + through to the format agent. + """ + store = tmp_path / "s.zarr" + store.mkdir() + (store / "zarr.json").write_text('{"zarr_format": 3, "node_type": "group"}') + assert _is_zarr_store(f"file://{store}#mode=nczarr,file") is False + + +def test_a_non_string_target_is_not_a_store(): + # `load_data_objects` passes open datasets, not paths. + assert _is_zarr_store(object()) is False + + +@_shared_utils.skip_data +def test_the_agent_still_picks_netcdf_for_a_netcdf_file(): + """A `requires_fh=False` spec is consulted for every target with no buffer. + + Adding one risks claiming a target another format already handles, and + the format agent returns the first match rather than the best. + """ + from iris.fileformats import FORMAT_AGENT + + path = _shared_utils.get_data_path( + ("NetCDF", "global", "xyt", "SMALL_hires_wind_u_for_ipcc4.nc") + ) + with open(path, "rb") as handle: + spec = FORMAT_AGENT.get_spec("SMALL_hires_wind_u_for_ipcc4.nc", handle) + assert spec.name.startswith("NetCDF") + + +def test_the_agent_picks_zarr_for_a_store(tmp_path): + from iris.fileformats import FORMAT_AGENT + + store = tmp_path / "s.zarr" + store.mkdir() + (store / "zarr.json").write_text('{"zarr_format": 3, "node_type": "group"}') + assert FORMAT_AGENT.get_spec(str(store), None).name == "Zarr" + + +def test_the_agent_still_picks_nczarr_for_an_nczarr_url(tmp_path): + from iris.fileformats import FORMAT_AGENT + + store = tmp_path / "s.zarr" + store.mkdir() + (store / "zarr.json").write_text('{"zarr_format": 3, "node_type": "group"}') + uri = f"file://{store}#mode=nczarr,file" + assert FORMAT_AGENT.get_spec(uri, None).name == "NcZarr" diff --git a/lib/iris/tests/unit/io/test__generate_cubes.py b/lib/iris/tests/unit/io/test__generate_cubes.py index 406ea2fa32..b207955a83 100644 --- a/lib/iris/tests/unit/io/test__generate_cubes.py +++ b/lib/iris/tests/unit/io/test__generate_cubes.py @@ -22,7 +22,10 @@ def test_pathlib_paths(self, mocker): decode_uri_mock = mocker.patch( "iris.iris.io.decode_uri", return_value=("file", None, None) ) - mocker.patch("iris.iris.io.load_files") + # `decode_uri` always reporting a `None` part is not a real path, so + # the file-or-store partition has nothing to resolve or classify. + mocker.patch("iris.iris.io.expand_filespecs", return_value=[]) + mocker.patch("iris.iris.io._load_files") for gc_arg, du_arg in test_variants: decode_uri_mock.reset_mock() @@ -38,7 +41,7 @@ def test_pathlib_paths(self, mocker): ], ) def test_nczarr_uri(self, uri, mocker): - load_files_mock = mocker.patch("iris.iris.io.load_files", return_value=[]) + load_files_mock = mocker.patch("iris.iris.io._load_files", return_value=[]) load_http_mock = mocker.patch("iris.iris.io.load_http", return_value=[]) list(_generate_cubes(uri, None, None)) @@ -48,7 +51,12 @@ def test_nczarr_uri(self, uri, mocker): def test_non_nczarr_uri(self, mocker): uri = "file:///foo#bar=baz" - load_files_mock = mocker.patch("iris.iris.io.load_files", return_value=[]) + # The partition needs a real-looking part name to classify, not a + # file-system lookup against a path that was never written. + mocker.patch("iris.iris.io.expand_filespecs", side_effect=lambda names: names) + # The "file" branch calls the private, non-expanding `_load_files` + # directly, since it has already expanded - see its docstring. + load_files_mock = mocker.patch("iris.iris.io._load_files", return_value=[]) load_http_mock = mocker.patch("iris.iris.io.load_http", return_value=[]) list(_generate_cubes(uri, None, None)) diff --git a/lib/iris/tests/unit/io/test_expand_filespecs.py b/lib/iris/tests/unit/io/test_expand_filespecs.py index 0a33f87c15..9ff879867e 100644 --- a/lib/iris/tests/unit/io/test_expand_filespecs.py +++ b/lib/iris/tests/unit/io/test_expand_filespecs.py @@ -105,3 +105,14 @@ def test_false_bool_relative(self): assert item_out == item_in finally: os.chdir(cwd) + + def test_remote_store_uri_is_passed_through_unresolved(self): + # `.absolute()` would otherwise turn this into + # "//s3:/bucket/data.zarr", which zarr-python cannot open. + uri = "s3://bucket/data.zarr" + assert iio.expand_filespecs([uri]) == [uri] + + def test_remote_store_uri_is_not_globbed(self): + # A remote URI stands for itself: Iris does not list its neighbours. + uri = "s3://bucket/data.zarr" + assert iio.expand_filespecs([uri], False) == [uri] diff --git a/lib/iris/tests/unit/io/test_load_stores.py b/lib/iris/tests/unit/io/test_load_stores.py new file mode 100644 index 0000000000..8d2196e07d --- /dev/null +++ b/lib/iris/tests/unit/io/test_load_stores.py @@ -0,0 +1,155 @@ +# Copyright Iris contributors +# +# This file is part of Iris and is released under the BSD license. +# See LICENSE in the root of the repository for full licensing details. +"""Unit tests for :func:`iris.io.load_stores`, and the load chain it joins.""" + +import numpy as np +import pytest + +import iris +from iris.io import _is_remote_store_uri, load_stores +from iris.tests import _shared_utils + +zarr = pytest.importorskip("zarr", minversion="3.0.8") + +from iris.tests.unit.fileformats.zarr import stores # noqa: E402 + + +class Test_load_stores: + def test_a_local_store_loads(self, tmp_path): + location = stores.make_store(tmp_path / "s.zarr") + cubes = list(load_stores([location], None)) + assert [cube.var_name for cube in cubes] == ["air_temperature"] + + def test_several_stores_in_one_call(self, tmp_path): + first = stores.make_store(tmp_path / "a.zarr") + second = stores.make_store(tmp_path / "b.zarr") + assert len(list(load_stores([first, second], None))) == 2 + + def test_constraints_reach_the_handler(self, tmp_path): + """A constraint the handler can pre-filter on rules out a data variable. + + `load_stores` forwards `constraints` to a constraint-aware handler + rather than applying it itself - the general `cube_func` form is only + enforced later, by `_CubeFilter.add` - so the constraint used here has + to be one `_translate_constraints_to_var_callback` can act on: a + `NameConstraint` with no `STASH` component. + """ + location = stores.make_store(tmp_path / "s.zarr") + constraint = iris.NameConstraint(var_name="not_air_temperature") + assert list(load_stores([location], None, constraint)) == [] + + +class Test__is_remote_store_uri: + @pytest.mark.parametrize( + ("uri", "expected"), + [ + ("s3://bucket/data.zarr", True), + ("gs://bucket/data.zarr", True), + ("az://container/data.zarr", True), + ("abfs://container/data.zarr", True), + ("https://example.invalid/data.nc", False), + ("/local/path/data.zarr", False), + ("relative/data.zarr", False), + ], + ) + def test_which_uris_are_remote_stores(self, uri, expected): + assert _is_remote_store_uri(uri) is expected + + +class Test_iris_load: + def test_accepts_a_local_store(self, tmp_path): + location = stores.make_store(tmp_path / "s.zarr") + cubes = iris.load(location) + assert len(cubes) == 1 + np.testing.assert_array_equal(cubes[0].data, stores.DATA) + + def test_accepts_a_remote_url_and_does_not_absolutise_it(self, mocker): + """A remote URL must reach `load_stores` unmangled, not as a local path. + + `expand_filespecs` would otherwise turn `s3://bucket/data.zarr` into + `//s3:/bucket/data.zarr`. The dispatch is what is under test, so + `load_stores` itself is mocked out: no store library is invoked, and + no network is touched, whatever is installed alongside zarr-python. + """ + mock_load_stores = mocker.patch("iris.io.load_stores", return_value=iter(())) + iris.load("s3://bucket/data.zarr") + mock_load_stores.assert_called_once_with(["s3://bucket/data.zarr"], None, None) + + @_shared_utils.skip_data + def test_a_glob_matching_both_files_and_stores(self, tmp_path): + """`iris.load("data/*")` must accept a mix. + + The file-or-store decision happens after glob expansion and before + format dispatch. Mis-partitioning raises IsADirectoryError from + `load_files` for the stores while the netCDF files load fine, which + reads as an Iris bug to a user who did not expect the glob to match a + store. + """ + source = iris.load_cube( + _shared_utils.get_data_path( + ("NetCDF", "global", "xyt", "SMALL_hires_wind_u_for_ipcc4.nc") + ) + ) + iris.save(source, str(tmp_path / "a.nc")) + stores.make_store(tmp_path / "b.zarr") + + cubes = iris.load(str(tmp_path / "*")) + names = sorted(cube.var_name or cube.name() for cube in cubes) + assert "air_temperature" in names + assert len(cubes) == 2 + + +class Test_load_files: + def test_still_expands_a_glob(self, tmp_path, mocker): + """Public `load_files` keeps its promise: filenames may be globs. + + The duplicate-collapsing bug fixed alongside `load_stores` lives in + calling `expand_filespecs` twice, once in a caller and again inside + `load_files` - not in `load_files` expanding at all. Pinning this + guards against quietly narrowing the public contract while fixing + that. + """ + from iris.io import load_files + + (tmp_path / "a.nc").write_bytes(b"") + (tmp_path / "b.nc").write_bytes(b"") + mock_load = mocker.patch("iris.io._load_files", return_value=iter(())) + + list(load_files([str(tmp_path / "*.nc")], None)) + + mock_load.assert_called_once_with( + [str(tmp_path / "a.nc"), str(tmp_path / "b.nc")], None, None + ) + + +class Test_saving: + def test_find_saver_tolerates_a_trailing_separator(self): + from iris.io import find_saver + + assert find_saver("out.nc") is find_saver("out.nc/") + + @_shared_utils.skip_data + def test_saving_a_cubelist_to_netcdf_still_takes_one_call(self, tmp_path): + cubes = iris.load( + _shared_utils.get_data_path( + ("NetCDF", "global", "xyt", "SMALL_hires_wind_u_for_ipcc4.nc") + ) + ) + iris.save(cubes, str(tmp_path / "out.nc")) + assert len(iris.load(str(tmp_path / "out.nc"))) == len(cubes) + + +class Test_load_meshes: + def test_a_store_holding_no_mesh_yields_no_meshes(self, tmp_path): + """Reaching a Zarr store is what was new here. + + What must not happen is the `ValueError` the scheme dispatch raised + for anything that was not a file. Reading a mesh *out* of a store is + covered by `unit/fileformats/zarr/loader/test_load_cubes.py::TestMesh`. + """ + from iris.mesh import load_meshes + + stores.make_store(tmp_path / "s.zarr") + assert load_meshes(str(tmp_path / "s.zarr")) == {} diff --git a/lib/iris/tests/unit/io/test_save.py b/lib/iris/tests/unit/io/test_save.py index 7010d4b67f..3662c8d3c6 100644 --- a/lib/iris/tests/unit/io/test_save.py +++ b/lib/iris/tests/unit/io/test_save.py @@ -6,8 +6,11 @@ from pathlib import Path +import pytest + import iris from iris.cube import Cube +import iris.fileformats.netcdf.saver class TestSave: @@ -39,3 +42,49 @@ def replace_expand(file_specs, files_expected=True): print("ValueError") pass find_saver_mock.assert_called_with(fs_val) + + +class TestCubeListSavers: + """Which savers take a `CubeList` whole, rather than one cube at a time. + + `iris.io._savers.cubelist_savers` holds saver functions, not extensions, + because `save` is reached with savers it never looked up by extension: + one passed by the caller, and the one chosen directly for an NCZarr + target. Keyed by extension, both would fall to the `append=True` path, + and the netCDF saver has no `append` parameter to fall to. + """ + + def test_the_netcdf_saver_is_registered(self): + """Registered by identity, so no name has to be matched. + + Two earlier forms matched on the module name: a substring test, true + of any module whose path happened to contain `iris.fileformats.netcdf`, + and then an exact set of module strings. Neither could be checked by a + reader against the function actually being called. + """ + from iris.fileformats.netcdf.saver import save as netcdf_save + + iris.io._check_init_savers() + assert netcdf_save in iris.io._savers.cubelist_savers + + def test_a_saver_passed_directly_is_recognised(self, tmp_path): + """The path that never reaches `find_saver`. + + `netcdf.save` has no `append` parameter, so if the registry were + keyed by extension and missed this saver, the fallback would raise + rather than quietly saving one cube at a time. + """ + from iris.fileformats.netcdf.saver import save as netcdf_save + + cubes = iris.cube.CubeList([Cube([1], long_name="a"), Cube([2], long_name="b")]) + target = tmp_path / "out.nc" + iris.save(cubes, str(target), saver=netcdf_save) + assert len(iris.load(str(target))) == 2 + + def test_an_unregistered_saver_still_needs_append(self, tmp_path): + def saver_without_append(cube, target): + pass + + cubes = iris.cube.CubeList([Cube([1], long_name="a"), Cube([2], long_name="b")]) + with pytest.raises(ValueError, match="Cannot append cubes"): + iris.save(cubes, str(tmp_path / "out.xyz"), saver=saver_without_append) diff --git a/requirements/locks/py312-linux-64.lock b/requirements/locks/py312-linux-64.lock index 541a4e0716..72edbabce9 100644 --- a/requirements/locks/py312-linux-64.lock +++ b/requirements/locks/py312-linux-64.lock @@ -1,21 +1,21 @@ # Generated by conda-lock. # platform: linux-64 -# input_hash: ec70083cb9ed06616b8d4368678123e62003bbdc710c1a3214f19caf1ef913ce +# input_hash: 882352f1d7a8b3e3b415f119530f26065dfc71c10354593ad1ecc6a92f1282cf @EXPLICIT -https://conda.anaconda.org/conda-forge/linux-64/eigen-abi-5.0.1.80-hf414acd_0.conda#a85c1768af7780d67c6446c17848b76f +https://conda.anaconda.org/conda-forge/linux-64/eigen-abi-5.0.1.80-hf414acd_3.conda#56a1cca9a4a7bed2999630e5644c8d0e https://conda.anaconda.org/conda-forge/noarch/font-ttf-dejavu-sans-mono-2.37-hab24e00_0.tar.bz2#0c96522c6bdaed4b1566d11387caaf45 https://conda.anaconda.org/conda-forge/noarch/font-ttf-inconsolata-3.000-h77eed37_0.tar.bz2#34893075a5c9e55cdafac56607368fc6 https://conda.anaconda.org/conda-forge/noarch/font-ttf-source-code-pro-2.038-h77eed37_0.tar.bz2#4d59c254e01d9cde7957100457e2d5fb https://conda.anaconda.org/conda-forge/noarch/font-ttf-ubuntu-0.83-h77eed37_3.conda#49023d73832ef61042f6a237cb2687e7 https://conda.anaconda.org/conda-forge/linux-64/hicolor-icon-theme-0.17-ha770c72_3.conda#129e404c5b001f3ef5581316971e3ea0 https://conda.anaconda.org/conda-forge/linux-64/nlohmann_json-3.12.0-h54a6638_2.conda#fe93be7dd9209e6ae673962e3031ff51 -https://conda.anaconda.org/conda-forge/noarch/python_abi-3.12-8_cp312.conda#c3efd25ac4d74b1584d2f7a57195ddf1 +https://conda.anaconda.org/conda-forge/noarch/python_abi-3.12-9_cp312.conda#4c32076993e6270825441d059ab5c18b https://conda.anaconda.org/conda-forge/noarch/tzdata-2026c-h151e31d_0.conda#fcb489df604d100968b737f2cb6076c6 https://conda.anaconda.org/conda-forge/linux-64/utfcpp-4.09-ha770c72_0.conda#99884244028fe76046e3914f90d4ad05 https://conda.anaconda.org/conda-forge/noarch/ca-certificates-2026.7.22-hbd8a1cb_0.conda#0f51e2391ade309db462a55611263e9c https://conda.anaconda.org/conda-forge/noarch/fonts-conda-forge-1-hc364b38_1.conda#a7970cd949a077b7cb9696379d338681 https://conda.anaconda.org/conda-forge/linux-64/libglvnd-1.7.0-ha4b6fd6_5.conda#cfcf11edcfd4acf0094771a9c3b8587f -https://conda.anaconda.org/conda-forge/linux-64/libgomp-16.2.0-he0feb66_4.conda#89d2c1231f47bd818f5d624b9411459d +https://conda.anaconda.org/conda-forge/linux-64/libgomp-16.2.0-he0feb66_7.conda#d904c7ddd95e026254b4b80808ee9e12 https://conda.anaconda.org/conda-forge/linux-64/libzlib-1.3.2-h25fd6f3_3.conda#0de0122d9570a8ab637c6b73db268389 https://conda.anaconda.org/conda-forge/linux-64/_openmp_mutex-4.5-20_gnu.conda#a9f577daf3de00bca7c3c76c0ecbd1de https://conda.anaconda.org/conda-forge/noarch/fonts-conda-ecosystem-1-0.tar.bz2#fee5683a3f04bd15cbd8318b096a27ab @@ -23,23 +23,23 @@ https://conda.anaconda.org/conda-forge/linux-64/libegl-1.7.0-ha4b6fd6_5.conda#3a https://conda.anaconda.org/conda-forge/linux-64/libopengl-1.7.0-ha4b6fd6_5.conda#d207a2e9bae9da476bc0a603215d5167 https://conda.anaconda.org/conda-forge/linux-64/zlib-1.3.2-h25fd6f3_3.conda#6acb86426229f96f93e5468d1df3a5e8 https://conda.anaconda.org/conda-forge/linux-64/zstd-1.5.7-hb78ec9c_7.conda#aa459086047c0e5e27023ab19f8cb86a -https://conda.anaconda.org/conda-forge/linux-64/ld_impl_linux-64-2.46.1-default_hbd61a6d_102.conda#449500f2c089da11c40f5c21312e3e07 -https://conda.anaconda.org/conda-forge/linux-64/libgcc-16.2.0-ha9f2e26_4.conda#cba14d01083fc62ffd32c24d7d390633 +https://conda.anaconda.org/conda-forge/linux-64/ld_impl_linux-64-2.46.1-default_hbd61a6d_103.conda#2362c70e7e79827c32db35547d9c179b +https://conda.anaconda.org/conda-forge/linux-64/libgcc-16.2.0-ha9f2e26_7.conda#735629b400b70eeab21a2ba8f5b1c2ce https://conda.anaconda.org/conda-forge/linux-64/alsa-lib-1.2.16.1-h7cc23a3_1.conda#7094e0d8d14de0eff6d83f0d2f1f661e -https://conda.anaconda.org/conda-forge/linux-64/aws-c-common-0.14.3-hb03c661_0.conda#ce8664d7c01d4a598ad38107b289b5ec +https://conda.anaconda.org/conda-forge/linux-64/aws-c-common-1.0.3-hebe6cf0_0.conda#cdb5797968bfe1737f1b71f525fc6518 https://conda.anaconda.org/conda-forge/linux-64/bzip2-1.0.8-hda65f42_10.conda#e675fabcf81499adc7edf58124fb1e01 https://conda.anaconda.org/conda-forge/linux-64/c-ares-1.34.8-hebe6cf0_2.conda#3ad2b403be8fc5612b9159e2ce621f69 -https://conda.anaconda.org/conda-forge/linux-64/fribidi-1.0.16-hb03c661_1.conda#1cd10eda5692519d01bb20e086e214c9 -https://conda.anaconda.org/conda-forge/linux-64/giflib-6.1.3-h280c20c_1.conda#6d9a90cfee2c7343d6c05a1b37ef75fb -https://conda.anaconda.org/conda-forge/linux-64/json-c-0.18-h6688a6e_0.conda#38f5dbc9ac808e31c00650f7be1db93f +https://conda.anaconda.org/conda-forge/linux-64/fribidi-1.0.17-h7cc23a3_0.conda#f9d5302bca196ee87e62f393f2626562 +https://conda.anaconda.org/conda-forge/linux-64/giflib-6.1.3-hebe6cf0_2.conda#07bdd65afe03df3fb5dbcaa8257c64a3 +https://conda.anaconda.org/conda-forge/linux-64/json-c-0.18-hebe6cf0_2.conda#9aa77c4e7f9a988df94da18a7b9b90fe https://conda.anaconda.org/conda-forge/linux-64/keyutils-1.6.3-h7cc23a3_1.conda#ba55d1b89fd7775e67de8291029b4059 -https://conda.anaconda.org/conda-forge/linux-64/libbrotlicommon-1.2.0-h39a168f_3.conda#7a2499a177753582fb7ae7e9dc4a908a +https://conda.anaconda.org/conda-forge/linux-64/libbrotlicommon-1.2.0-h39a168f_4.conda#df210655e30a05e3b069c91b7aaddd2d https://conda.anaconda.org/conda-forge/linux-64/libdeflate-1.25-hd45a770_1.conda#40f9b31aa9cf007789867df0decd0492 https://conda.anaconda.org/conda-forge/linux-64/libev-4.33-h280c20c_3.conda#7bc31538d6e3fb349e897353969237ec -https://conda.anaconda.org/conda-forge/linux-64/libexpat-2.8.1-hecca717_1.conda#b24d3c612f71e7aa74158d92106318b2 +https://conda.anaconda.org/conda-forge/linux-64/libexpat-2.8.5-hd2095e1_0.conda#72cfd2b95b0540247220b42f58a148d7 https://conda.anaconda.org/conda-forge/linux-64/libffi-3.7.0-h81df57d_1.conda#6525a0b06aa4fd390795f0740636a9dd -https://conda.anaconda.org/conda-forge/linux-64/libgcc-ng-16.2.0-h69a702a_4.conda#b3e52878163a841f6fb951989cc0b217 -https://conda.anaconda.org/conda-forge/linux-64/libgfortran5-16.2.0-h6b99dfc_4.conda#c22348a769bb072b6184eb6f7f05e4f2 +https://conda.anaconda.org/conda-forge/linux-64/libgcc-ng-16.2.0-h69a702a_7.conda#2005515ac9340f816fb2ac7e83afe553 +https://conda.anaconda.org/conda-forge/linux-64/libgfortran5-16.2.0-h6b99dfc_7.conda#7719ed7be38ad8c492c4bd5cc2bce0c4 https://conda.anaconda.org/conda-forge/linux-64/libiconv-1.18-h0cb94f2_3.conda#f92233bf33e24a25668bb2119e2c51f9 https://conda.anaconda.org/conda-forge/linux-64/libjpeg-turbo-3.2.0-hb03c661_1.conda#898d1c9793eaa52efc4727bd84d2e39a https://conda.anaconda.org/conda-forge/linux-64/liblzma-5.8.3-hb03c661_1.conda#1390b7c5ac0b1d8e447bc5efa6d3c8c2 @@ -47,80 +47,82 @@ https://conda.anaconda.org/conda-forge/linux-64/libnsl-2.0.1-hb9d3cd8_1.conda#d8 https://conda.anaconda.org/conda-forge/linux-64/libntlm-1.8-hb9d3cd8_0.conda#7c7927b404672409d9917d49bff5f2d6 https://conda.anaconda.org/conda-forge/linux-64/libogg-1.3.5-hd0c01bc_1.conda#68e52064ed3897463c0e958ab5c8f91b https://conda.anaconda.org/conda-forge/linux-64/libpciaccess-0.19-hb03c661_1.conda#35fa2b34bbced424e6976d30f5fde576 -https://conda.anaconda.org/conda-forge/linux-64/libpng-1.6.58-h922cc85_1.conda#fcf71c8d979148873f6f8ad4cfc73d86 -https://conda.anaconda.org/conda-forge/linux-64/libstdcxx-16.2.0-h934c35e_4.conda#2f2ef0d96de5bdd8c1270ff22fdf9352 -https://conda.anaconda.org/conda-forge/linux-64/libuuid-2.42.2-h5347b49_0.conda#01bb81d12c957de066ea7362007df642 +https://conda.anaconda.org/conda-forge/linux-64/libpng-1.6.59-h922cc85_0.conda#8c9caf742a485ac688ffa88bbadbf0be +https://conda.anaconda.org/conda-forge/linux-64/libpython-3.12.15-ha57cdbe_2_cpython.conda#fe256863f652549334a81a8e49fdf097 +https://conda.anaconda.org/conda-forge/linux-64/libstdcxx-16.2.0-h934c35e_7.conda#9b6707e0caca917fa6ea9ee473f37a5d +https://conda.anaconda.org/conda-forge/linux-64/libuuid-2.42.4-hcfc3c73_0.conda#54a948337e66706380ff24d6b5fde63f https://conda.anaconda.org/conda-forge/linux-64/libwebp-base-1.6.0-hd42ef1d_1.conda#9332b53d0ea93c5d39e33be03a0c611a https://conda.anaconda.org/conda-forge/linux-64/libxcrypt-4.4.38-h280c20c_0.conda#f7a7ff5a6ab331e037abd34f379a631d https://conda.anaconda.org/conda-forge/linux-64/lzo-2.10-hebe6cf0_1003.conda#10e0aa58ed390ba598b728badf9eb1b7 https://conda.anaconda.org/conda-forge/linux-64/ncurses-6.6-hdb14827_1.conda#ee6c0cd80a60961a1f48aa3e0b91f986 -https://conda.anaconda.org/conda-forge/linux-64/openssl-3.6.4-h781a0a9_0.conda#16e3034a330cc625f2c685edec60cf4e -https://conda.anaconda.org/conda-forge/linux-64/pthread-stubs-0.4-hb03c661_1003.conda#df2c27f36bdb0dde779f55b5df76a352 +https://conda.anaconda.org/conda-forge/linux-64/openssl-3.6.5-h781a0a9_0.conda#5a51a2ea653f120240b1a9bf76e0aae9 +https://conda.anaconda.org/conda-forge/linux-64/pthread-stubs-0.4-h7cc23a3_1004.conda#bb66b610707b811e3c65181a6431e7a8 https://conda.anaconda.org/conda-forge/linux-64/tk-8.6.13-noxft_h1df4ec4_4.conda#676a2f9b1c4fcf57b70aa5c65f6c60d0 https://conda.anaconda.org/conda-forge/linux-64/xorg-libice-1.1.2-h280c20c_0.conda#85c9442aec283b4e464fa9ecc484a2f3 -https://conda.anaconda.org/conda-forge/linux-64/xorg-libxau-1.0.12-hb03c661_2.conda#f06ef439c280a5f90b8bf62355008dbc -https://conda.anaconda.org/conda-forge/linux-64/xorg-libxdmcp-1.1.5-hb03c661_2.conda#2e66c929f3d879708335b6ea4557c838 -https://conda.anaconda.org/conda-forge/linux-64/xorg-xorgproto-2025.1-h280c20c_1.conda#3b51576511038b50fdbd05245e22e4b1 +https://conda.anaconda.org/conda-forge/linux-64/xorg-libxau-1.0.12-h7cc23a3_2.conda#da33efee1a93603d92e06d4a2b68def6 +https://conda.anaconda.org/conda-forge/linux-64/xorg-libxdmcp-1.1.5-h7cc23a3_2.conda#eabfc45786244de252090cf8d55128a8 +https://conda.anaconda.org/conda-forge/linux-64/xorg-xorgproto-2025.1-hebe6cf0_3.conda#c3d0a6f03ba3aa312777b41b20a71ec0 https://conda.anaconda.org/conda-forge/linux-64/xxhash-0.8.3-hec8ed23_1.conda#72ab4ea51529ec63ff010ae9fb62b02f https://conda.anaconda.org/conda-forge/linux-64/yaml-0.2.5-hebe6cf0_3.conda#e741576fb8f89821ac7c1c537322a33d -https://conda.anaconda.org/conda-forge/linux-64/aws-c-cal-0.9.15-h6bbde05_1.conda#8f2b374c79908f271be3d0e9f1c462a8 -https://conda.anaconda.org/conda-forge/linux-64/aws-c-compression-0.3.2-h01ee8f8_5.conda#ae862abdbb3160f97478d3d4749164c0 -https://conda.anaconda.org/conda-forge/linux-64/aws-c-sdkutils-0.2.7-h01ee8f8_3.conda#412f0cf18cc4c4945655dba954ca2d76 -https://conda.anaconda.org/conda-forge/linux-64/aws-checksums-0.2.10-h01ee8f8_5.conda#4d70caf5260e0787675c3bf61ae5b64c +https://conda.anaconda.org/conda-forge/linux-64/aws-c-cal-1.0.0-hcf7459e_1.conda#2009b71b358163738f339c76640b512c +https://conda.anaconda.org/conda-forge/linux-64/aws-c-compression-1.0.0-ha404f16_1.conda#da0c7b86d5379eb44acfa084897e8b98 +https://conda.anaconda.org/conda-forge/linux-64/aws-c-sdkutils-1.0.0-ha404f16_1.conda#cbbd999c74a030951365225909e376c7 +https://conda.anaconda.org/conda-forge/linux-64/aws-checksums-1.0.1-ha404f16_0.conda#10e6e2f11271e66dfad43eb002359f22 https://conda.anaconda.org/conda-forge/linux-64/cli11-2.7.2-h54a6638_1.conda#42761b55062088f6f7fb58d0633e0769 https://conda.anaconda.org/conda-forge/linux-64/double-conversion-3.4.0-hecca717_0.conda#dbe3ec0f120af456b3477743ffd99b74 https://conda.anaconda.org/conda-forge/linux-64/fmt-12.1.0-h76c4fd7_1.conda#a70bb6d42ad121aef456e0007e92bed6 https://conda.anaconda.org/conda-forge/linux-64/geos-3.14.1-h55b0958_0.conda#5f856b72ca9cd5b5b4a1c43551804622 https://conda.anaconda.org/conda-forge/linux-64/graphite2-1.3.15-h54a6638_1.conda#f9fe2984587fa8235a6af6004760cd18 https://conda.anaconda.org/conda-forge/linux-64/icu-78.3-py310h44b86e0_2.conda#72a381cbad04f24b1c2a43ef707f45b4 -https://conda.anaconda.org/conda-forge/linux-64/jsoncpp-1.9.8-h4dbf13b_1.conda#16e15ba8628a2cd3c752f871791056f9 +https://conda.anaconda.org/conda-forge/linux-64/jsoncpp-1.9.8-h4dbf13b_2.conda#c8985329431780d56025998322650199 https://conda.anaconda.org/conda-forge/linux-64/lerc-4.2.0-hdb68285_0.conda#fb9d356b1a57d6d54768be7ebd5fce09 https://conda.anaconda.org/conda-forge/linux-64/libaec-1.1.5-h088129d_0.conda#86f7414544ae606282352fa1e116b41f -https://conda.anaconda.org/conda-forge/linux-64/libbrotlidec-1.2.0-ha411449_3.conda#6ab3315dc56618d652c1da42a648a129 -https://conda.anaconda.org/conda-forge/linux-64/libbrotlienc-1.2.0-h018ffa1_3.conda#2ac965638d4c6b2b38383bb1aebaf543 +https://conda.anaconda.org/conda-forge/linux-64/libbrotlidec-1.2.0-ha411449_4.conda#0cfd601eb03cca403dcc248844787486 +https://conda.anaconda.org/conda-forge/linux-64/libbrotlienc-1.2.0-h018ffa1_4.conda#4136f6e4ef4835cc7df39a707cbd511c https://conda.anaconda.org/conda-forge/linux-64/libdrm-2.4.129-h7cc23a3_0.conda#64cc91512b6278c315349dc13c53f680 -https://conda.anaconda.org/conda-forge/linux-64/libedit-3.1.20250104-pl5321h373387f_1.conda#50708d3b951d0f8e2d7f2df5b5edc040 +https://conda.anaconda.org/conda-forge/linux-64/libedit-3.1.20260512-pl5321h373387f_0.conda#27dbd7e2984b8fe48e069379636ccfdd https://conda.anaconda.org/conda-forge/linux-64/libfreetype6-2.14.3-h5e6c136_2.conda#b72a266a9317036fd0464cb98b027cd0 -https://conda.anaconda.org/conda-forge/linux-64/libgfortran-16.2.0-h69a702a_4.conda#5e92b8413fd1c8f8f3006f4661b12def +https://conda.anaconda.org/conda-forge/linux-64/libgfortran-16.2.0-h69a702a_7.conda#3a8d285f5f22138ef826ad54cf267e73 https://conda.anaconda.org/conda-forge/linux-64/libglu-9.0.3-h5888daf_1.conda#8422fcc9e5e172c91e99aef703b3ce65 -https://conda.anaconda.org/conda-forge/linux-64/libhwy-1.4.0-h57c4cff_1.conda#d58bfbaaf51ed85771eae92c2e7f5816 +https://conda.anaconda.org/conda-forge/linux-64/libhwy-1.4.0-h57c4cff_2.conda#145497b38fe581a7760dd88b7377c96b https://conda.anaconda.org/conda-forge/linux-64/libnghttp2-1.68.1-h74cf4be_1.conda#75d211f04ac87506f1f43420712a69fe https://conda.anaconda.org/conda-forge/linux-64/libssh2-1.11.1-h6154650_1.conda#5c526561e1d2a2e071941174eae84557 -https://conda.anaconda.org/conda-forge/linux-64/libstdcxx-ng-16.2.0-hdf11a46_4.conda#de0dceacf3e33c5fc88e167885dc8274 +https://conda.anaconda.org/conda-forge/linux-64/libstdcxx-ng-16.2.0-hdf11a46_7.conda#25a23bde5e26d548374f3e02b4669670 https://conda.anaconda.org/conda-forge/linux-64/libudunits2-2.2.28-h40f5838_3.conda#4bdace082e911a3e1f1f0b721bed5b56 https://conda.anaconda.org/conda-forge/linux-64/libvorbis-1.3.7-h54a6638_2.conda#b4ecbefe517ed0157c37f8182768271c https://conda.anaconda.org/conda-forge/linux-64/libxcb-1.17.0-hb83e432_2.conda#61f0ffba9161cbb3a77722869b21e4bb -https://conda.anaconda.org/conda-forge/linux-64/libzip-1.11.2-h6991a6a_0.conda#a7b27c075c9b7f459f1c022090697cba +https://conda.anaconda.org/conda-forge/linux-64/libzip-1.11.4-h6008cf6_0.conda#05adf41813469e50f2103db6eba38edb https://conda.anaconda.org/conda-forge/linux-64/lz4-c-1.10.0-hee9eb32_2.conda#e38a3253d72ab07d471483f24947e2a2 -https://conda.anaconda.org/conda-forge/linux-64/minizip-4.2.2-hb71707f_0.conda#88450ba743694055e218d03a2fdd561e -https://conda.anaconda.org/conda-forge/linux-64/muparser-2.3.5-h5888daf_0.conda#ab3e3db511033340e75e7002e80ce8c0 +https://conda.anaconda.org/conda-forge/linux-64/minizip-4.2.2-haac24f3_1.conda#3fee8d11b0d8a54fb899749052accb4f +https://conda.anaconda.org/conda-forge/linux-64/muparser-2.3.5-hee9eb32_1.conda#7de84fd76ec2bdb7ac617507de3d3052 https://conda.anaconda.org/conda-forge/linux-64/pcre2-10.47-h8b3dc9c_1.conda#06f6113cf1ff4a54b65f87ead132a5f1 https://conda.anaconda.org/conda-forge/linux-64/pixman-0.46.4-h54a6638_3.conda#0ee5bb30034b081a1386c1e2c98ab0a7 https://conda.anaconda.org/conda-forge/linux-64/pugixml-1.15-h3f63f65_0.conda#b11a4c6bf6f6f44e5e143f759ffa2087 https://conda.anaconda.org/conda-forge/linux-64/readline-8.3-hd6e31c0_1.conda#69c01c781e7c8190bbdaf79e3848c5ca -https://conda.anaconda.org/conda-forge/linux-64/s2n-1.7.6-h7e3ee7f_0.conda#e112cadcd79412d81496e784884bddf0 -https://conda.anaconda.org/conda-forge/linux-64/snappy-1.2.2-h03e3b7b_1.conda#98b6c9dc80eb87b2519b97bcf7e578dd +https://conda.anaconda.org/conda-forge/linux-64/s2n-1.7.7-ha5dc4ae_0.conda#4928f1e1964a6f38f0da349b9d0e1c6a +https://conda.anaconda.org/conda-forge/linux-64/snappy-1.2.2-h34e00bb_2.conda#2b7e31ffbd476eecf33b3b833692339c https://conda.anaconda.org/conda-forge/linux-64/wayland-1.26.0-hc1c935e_2.conda#88a47e22b53e50865b1cabe2eb648352 https://conda.anaconda.org/conda-forge/linux-64/xorg-libsm-1.2.6-h0d788c3_1.conda#aa7459ed9ad086ba11dda843d764b33d -https://conda.anaconda.org/conda-forge/linux-64/zfp-1.0.1-h909a3a2_5.conda#6a0eb48e58684cca4d7acc8b7a0fd3c7 +https://conda.anaconda.org/conda-forge/linux-64/zfp-1.0.1-h79e284d_6.conda#9d7141ec747a9ede5da83e864fcc8213 https://conda.anaconda.org/conda-forge/linux-64/zlib-ng-2.3.3-hce19668_1.conda#1726acec19beeed1c764385cd8622405 -https://conda.anaconda.org/conda-forge/linux-64/aws-c-io-0.27.5-h4f381ba_1.conda#948f5a4fddac779b7942626e5807caba +https://conda.anaconda.org/conda-forge/linux-64/aws-c-io-1.0.0-h07ba609_0.conda#72593f68772d8c7e72c8a09681b0fe42 https://conda.anaconda.org/conda-forge/linux-64/blosc-1.21.6-he440d0b_1.conda#2c2fae981fd2afd00812c92ac47d023d -https://conda.anaconda.org/conda-forge/linux-64/brotli-bin-1.2.0-h9908984_3.conda#8929291d9efc59715df1cfa7daf5e3ed -https://conda.anaconda.org/conda-forge/linux-64/freexl-2.0.0-h9dce30a_2.conda#ecb5d11305b8ba1801543002e69d2f2f +https://conda.anaconda.org/conda-forge/linux-64/brotli-bin-1.2.0-h9908984_4.conda#c48c09444f3736800ea0b9550c365baf +https://conda.anaconda.org/conda-forge/linux-64/freexl-2.0.0-h1f12c17_3.conda#4a0a8fd87ee4767db791047546f56501 https://conda.anaconda.org/conda-forge/linux-64/hdf4-4.2.15-h2a13503_7.conda#bd77f8da987968ec3927990495dc22e4 https://conda.anaconda.org/conda-forge/linux-64/krb5-1.22.2-hbc21106_2.conda#53318d715316929a574f83591308b1f8 +https://conda.anaconda.org/conda-forge/linux-64/libcrc32c-1.1.2-h9c3ff4c_0.tar.bz2#c965a5aa0d5c1c37ffc62dff36e28400 https://conda.anaconda.org/conda-forge/linux-64/libfreetype-2.14.3-ha770c72_2.conda#f8054e759d0ddddaf34a5c8fedc900a1 -https://conda.anaconda.org/conda-forge/linux-64/libglib-2.88.3-he503a2a_2.conda#533d342dedadf134c67760ce6fc5b748 -https://conda.anaconda.org/conda-forge/linux-64/libjxl-0.12.0-heb2dce7_2.conda#485788d339785bac87fe86315b4a0627 +https://conda.anaconda.org/conda-forge/linux-64/libglib-2.90.0-h569388d_0.conda#c85e9539551f1fd21104e518d940c244 +https://conda.anaconda.org/conda-forge/linux-64/libjxl-0.12.0-heb2dce7_4.conda#8770a39fd4c5559357efc7cdb8f0c742 https://conda.anaconda.org/conda-forge/linux-64/libmo_unpack-3.1.2-hf484d3e_1001.tar.bz2#95f32a6a5a666d33886ca5627239f03d -https://conda.anaconda.org/conda-forge/linux-64/libopenblas-0.3.34-pthreads_h94d23a6_0.conda#c282d68f272927612462b5d626838ef1 +https://conda.anaconda.org/conda-forge/linux-64/libopenblas-0.3.34-pthreads_hf13c14d_2.conda#7e19fcd73d7f0997d3cb3326d2586a86 https://conda.anaconda.org/conda-forge/linux-64/libpsl-0.23.1-hd9e3e90_1.conda#77be60417aa572afcb48cbe0f967401d -https://conda.anaconda.org/conda-forge/linux-64/librttopo-1.1.0-h46dd2a8_20.conda#df81fd57eacf341588d728c97920e86d -https://conda.anaconda.org/conda-forge/linux-64/libsqlite-3.53.4-h13e7031_1.conda#e72bbec309c2b0f37823ee7d4fabfcd3 +https://conda.anaconda.org/conda-forge/linux-64/librttopo-1.1.0-h55b0958_21.conda#6c639697369e9669032ca08d0a566282 +https://conda.anaconda.org/conda-forge/linux-64/libsqlite-3.54.0-h13e7031_1.conda#bb145547395d3329f3a1d930de9848b2 https://conda.anaconda.org/conda-forge/linux-64/libtheora-1.2.0-h85c0a6d_0.conda#177ef038f292390abe69137711fc0cbf https://conda.anaconda.org/conda-forge/linux-64/libtiff-4.7.2-hcc2c06a_1.conda#0907d876f460ebc941ae590338b6102f -https://conda.anaconda.org/conda-forge/linux-64/libxml2-16-2.15.3-hca6bf5a_1.conda#20e4d67ec908f0405124f11612c48305 +https://conda.anaconda.org/conda-forge/linux-64/libxml2-16-2.15.4-hf3af7cc_1.conda#046d9eada488e1a1657c80b55c23da52 https://conda.anaconda.org/conda-forge/linux-64/qhull-2020.2-h434a139_5.conda#353823361b1d27eb3960efb076dfcaf6 https://conda.anaconda.org/conda-forge/linux-64/udunits2-2.2.28-h40f5838_3.conda#6bb8deb138f87c9d48320ac21b87e7a1 https://conda.anaconda.org/conda-forge/linux-64/uriparser-0.9.8-hac33072_0.conda#d71d3a66528853c0a1ac2c02d79a0284 @@ -128,29 +130,29 @@ https://conda.anaconda.org/conda-forge/linux-64/xcb-util-0.4.1-h4f16b4b_2.conda# https://conda.anaconda.org/conda-forge/linux-64/xcb-util-keysyms-0.4.1-hb711507_0.conda#ad748ccca349aec3e91743e08b5e2b50 https://conda.anaconda.org/conda-forge/linux-64/xcb-util-renderutil-0.3.10-hb711507_0.conda#0e0cbe0564d03a99afd5fd7b362feecd https://conda.anaconda.org/conda-forge/linux-64/xcb-util-wm-0.4.2-hb711507_0.conda#608e0ef8256b81d04456e8d211eee3e8 -https://conda.anaconda.org/conda-forge/linux-64/xerces-c-3.3.0-hd9031aa_1.conda#66a1db55ecdb7377d2b91f54cd56eafa +https://conda.anaconda.org/conda-forge/linux-64/xerces-c-3.3.0-h78fed68_2.conda#f2ff0de6c209e90c53b3a195c2708f5f https://conda.anaconda.org/conda-forge/linux-64/xorg-libx11-1.8.13-he1eb515_1.conda#8c282bbe4808a3cc80a5c98e9aec1cfc https://conda.anaconda.org/conda-forge/linux-64/atk-1.0-2.38.0-h04ea711_2.conda#f730d54ba9cd543666d7220c9f7ed563 -https://conda.anaconda.org/conda-forge/linux-64/aws-c-http-0.11.0-hbe094ff_5.conda#c09f7ec7327b8bd52fc2d6eacaa18d3b -https://conda.anaconda.org/conda-forge/linux-64/brotli-1.2.0-h505cf86_3.conda#6ff307b78fbfb7dcafb7e25ff8bc9c10 +https://conda.anaconda.org/conda-forge/linux-64/aws-c-http-1.0.0-h392d1a3_1.conda#6701f6e712f9513e722ba57cde99f023 +https://conda.anaconda.org/conda-forge/linux-64/brotli-1.2.0-h505cf86_4.conda#746aa619a1bcaf4da541101179d7c60e https://conda.anaconda.org/conda-forge/linux-64/cyrus-sasl-2.1.28-hac629b4_1.conda#af491aae930edc096b58466c51c4126c -https://conda.anaconda.org/conda-forge/linux-64/dbus-1.16.2-h24cb091_1.conda#ce96f2f470d39bd96ce03945af92e280 +https://conda.anaconda.org/conda-forge/linux-64/dbus-1.16.2-he8c428d_2.conda#a6eb37b51be1a9a654dc3a8aca039b1a https://conda.anaconda.org/conda-forge/linux-64/fontconfig-2.18.3-h4db4eae_1.conda#922776b528a470ab5afa81fd42abfa1d https://conda.anaconda.org/conda-forge/linux-64/freetype-2.14.3-ha770c72_2.conda#2d0ea23b23603e07ca47f23f949f67b6 -https://conda.anaconda.org/conda-forge/linux-64/gdk-pixbuf-2.44.8-h68053f1_0.conda#10dab6a745f32ceeac6c9e00d9979797 -https://conda.anaconda.org/conda-forge/linux-64/glib-tools-2.88.3-h67ed8a3_2.conda#5c35ace78b7d630e77efcb37dacb62ae +https://conda.anaconda.org/conda-forge/linux-64/gdk-pixbuf-2.44.8-h68053f1_2.conda#d7bd6ccfc58eb6796bd6bba85878fc0e +https://conda.anaconda.org/conda-forge/linux-64/glib-tools-2.90.0-h3d503d9_0.conda#d4adaebf8a2c9040ada6549c52a6b8d8 https://conda.anaconda.org/conda-forge/linux-64/gts-0.7.6-h977cf35_4.conda#4d8df0b0db060d33c9a702ada998a8fe -https://conda.anaconda.org/conda-forge/linux-64/lcms2-2.19.1-h0c24ade_1.conda#8b3ce45e929cd8e8e5f4d18586b56d8b -https://conda.anaconda.org/conda-forge/linux-64/libblas-3.11.0-9_h4a7cf45_openblas.conda#f5c4b041925dea221dc4bad2e50569d9 +https://conda.anaconda.org/conda-forge/linux-64/lcms2-2.19.1-h9073bf1_3.conda#596464faa06e0b656b35416188ef9bba +https://conda.anaconda.org/conda-forge/linux-64/libblas-3.11.0-11_h4a7cf45_openblas.conda#b8cce1486f3c62f33291d6318c193d0d https://conda.anaconda.org/conda-forge/linux-64/libcups-2.3.3-h7a8fb5f_6.conda#49c553b47ff679a6a1e9fc80b9c5a2d4 -https://conda.anaconda.org/conda-forge/linux-64/libcurl-8.21.0-ha042cf0_5.conda#0ed167049513943d078a6c997df2b4e0 +https://conda.anaconda.org/conda-forge/linux-64/libcurl-8.22.0-ha042cf0_0.conda#e617deee7af8eb0c04f6296d12b54157 https://conda.anaconda.org/conda-forge/linux-64/libglx-1.7.0-ha4b6fd6_5.conda#614e10aae180f87b84f0d1ca8ceb7d9e https://conda.anaconda.org/conda-forge/linux-64/libkml-1.3.0-haa4a5bd_1023.conda#953b7cca897e21215302dbfe2af5cd0c https://conda.anaconda.org/conda-forge/linux-64/libxkbfile-1.2.0-hb03c661_0.conda#f330a1b107cb5861f26dbb72a26f33e8 -https://conda.anaconda.org/conda-forge/linux-64/libxml2-2.15.3-h49c6c72_1.conda#2d34cdb31014cbc8f1e9384c89ede0a5 -https://conda.anaconda.org/conda-forge/linux-64/openjpeg-2.5.4-h55fea9a_0.conda#11b3379b191f63139e29c0d19dee24cd -https://conda.anaconda.org/conda-forge/linux-64/python-3.12.14-h8ab3286_0_cpython.conda#e9dcdd23a1c68738eb3257d23d5f7285 -https://conda.anaconda.org/conda-forge/linux-64/sqlite-3.53.4-h9ffa6c4_1.conda#d42b24574925bfa3167efb3571c905ad +https://conda.anaconda.org/conda-forge/linux-64/libxml2-2.15.4-h7df9aa5_1.conda#51e5162f95bd6840f6b2b3169125fff5 +https://conda.anaconda.org/conda-forge/linux-64/openjpeg-2.5.4-heb1ab33_2.conda#a3f3ac7a68d01c198e276dbb99e62032 +https://conda.anaconda.org/conda-forge/linux-64/python-3.12.15-h5f976f7_2_cpython.conda#13cef25ee4055b1fec19d35d20aadda2 +https://conda.anaconda.org/conda-forge/linux-64/sqlite-3.54.0-h9ffa6c4_1.conda#1060a8b69c6774ec85d81878eb299038 https://conda.anaconda.org/conda-forge/linux-64/xcb-util-image-0.4.0-hb711507_2.conda#a0901183f08b6c7107aab109733a3c91 https://conda.anaconda.org/conda-forge/linux-64/xkeyboard-config-2.48-h280c20c_0.conda#b233b41be0bf210989d57160ed39b394 https://conda.anaconda.org/conda-forge/linux-64/xorg-libxext-1.3.7-h7cc23a3_1.conda#e5b6b28536b81b3f4cb20db4668a4642 @@ -159,81 +161,82 @@ https://conda.anaconda.org/conda-forge/linux-64/xorg-libxrender-0.9.12-hb03c661_ https://conda.anaconda.org/conda-forge/noarch/aiohappyeyeballs-2.7.1-pyhd8ed1ab_0.conda#e51e09bf3b91c62b7461a9f94e92c2c9 https://conda.anaconda.org/conda-forge/noarch/alabaster-1.0.0-pyhd8ed1ab_1.conda#1fd9696649f65fd6611fcdb4ffec738a https://conda.anaconda.org/conda-forge/noarch/attrs-26.1.0-pyhcf101f3_0.conda#c6b0543676ecb1fb2d7643941fe375f2 -https://conda.anaconda.org/conda-forge/linux-64/aws-c-auth-0.10.4-h4610da3_2.conda#cefa84dbf94066e7a6902af288a2fedd +https://conda.anaconda.org/conda-forge/linux-64/aws-c-auth-1.0.0-h7648d90_2.conda#f4f169a23884de1bdb9c25ded8f2aec1 https://conda.anaconda.org/conda-forge/noarch/babel-2.18.0-pyhcf101f3_1.conda#f1976ce927373500cc19d3c0b2c85177 -https://conda.anaconda.org/conda-forge/linux-64/backports.zstd-1.7.0-py312h3f22e6b_0.conda#e7eb25765bdf21397cc6e30828871625 -https://conda.anaconda.org/conda-forge/linux-64/brotli-python-1.2.0-py312he9c40d5_3.conda#d176f3ed2824f930b524c45eb8f158bb -https://conda.anaconda.org/conda-forge/linux-64/cairo-1.18.4-h3c89d7e_2.conda#e83331a940393006789fedddc3b492f3 +https://conda.anaconda.org/conda-forge/linux-64/backports.zstd-1.8.0-py312h3f22e6b_0.conda#0f5fe32a47138697efdd6d8ee40a00d0 +https://conda.anaconda.org/conda-forge/linux-64/brotli-python-1.2.0-py312he9c40d5_4.conda#edb667e7ce56106424e8afab2dc0f0cb +https://conda.anaconda.org/conda-forge/linux-64/cairo-1.18.6-h3c89d7e_0.conda#7920701f9abcb488411495742cf365a7 https://conda.anaconda.org/conda-forge/noarch/certifi-2026.7.22-pyhd8ed1ab_0.conda#37e13edbe3b48f1095a9d085ef9cd83b https://conda.anaconda.org/conda-forge/noarch/cfgv-3.5.0-pyhd8ed1ab_0.conda#381bd45fb7aa032691f3063aff47e3a1 -https://conda.anaconda.org/conda-forge/noarch/charset-normalizer-3.5.1-pyhd8ed1ab_0.conda#e0ac3accc64e23e40969d660e5f58ac8 -https://conda.anaconda.org/conda-forge/noarch/click-8.4.2-pyhc90fa1f_0.conda#2c4bd6aeb90bb157456841c3270a0d92 +https://conda.anaconda.org/conda-forge/noarch/click-8.5.0-pyh5ded981_0.conda#dea5b13a211bbf99876deb980b408a66 https://conda.anaconda.org/conda-forge/noarch/cloudpickle-3.1.2-pyhcf101f3_1.conda#61b8078a0905b12529abc622406cb62c https://conda.anaconda.org/conda-forge/noarch/colorama-0.4.6-pyhd8ed1ab_1.conda#962b9857ee8e7018c22f2776ffa0b2d7 https://conda.anaconda.org/conda-forge/noarch/colorcet-3.2.1-pyhd8ed1ab_0.conda#0b6419b9a27c540bd96e66e12a44c379 -https://conda.anaconda.org/conda-forge/linux-64/coverage-7.16.0-py312h8b7ec3f_0.conda#385233507bd4b374469cc6563734c22b -https://conda.anaconda.org/conda-forge/noarch/cpython-3.12.14-py312hd8ed1ab_0.conda#c602a908e40a3ae4431b162695a8ba8b +https://conda.anaconda.org/conda-forge/linux-64/coverage-7.16.2-py312h8b7ec3f_0.conda#1d1ef312ddf97e5f5218d3a3d56dd7a0 +https://conda.anaconda.org/conda-forge/noarch/cpython-3.12.15-py312hd8ed1ab_2.conda#34ff082ed834c6a62f4b11f877820c1d https://conda.anaconda.org/conda-forge/noarch/cycler-0.12.1-pyhcf101f3_2.conda#4c2a8fef270f6c69591889b93f9f55c1 https://conda.anaconda.org/conda-forge/linux-64/cython-3.3.0-py312h03570ca_0.conda#b2498aa25d1654210448beb48fdce604 https://conda.anaconda.org/conda-forge/noarch/distlib-0.4.3-pyhcf101f3_0.conda#58638f77697c4f6726753eb8be34818b https://conda.anaconda.org/conda-forge/noarch/docstring_parser-0.18.0-pyhd8ed1ab_0.conda#b0c7c29a60c82d57c5b4c4c38f0642c8 https://conda.anaconda.org/conda-forge/noarch/docutils-0.21.2-pyhd8ed1ab_1.conda#24c1ca34138ee57de72a943237cde4cc https://conda.anaconda.org/conda-forge/noarch/execnet-2.1.2-pyhd8ed1ab_0.conda#a57b4be42619213a94f31d2c69c5dda7 -https://conda.anaconda.org/conda-forge/noarch/filelock-3.32.4-pyhd8ed1ab_0.conda#0171dc5d54fdbb3f6e55f285f805e0fe +https://conda.anaconda.org/conda-forge/noarch/filelock-4.1.1-pyhd8ed1ab_0.conda#fbff0cf6c30bdd9b38d0448fdadfe72d https://conda.anaconda.org/conda-forge/linux-64/frozenlist-1.8.0-py312h447239a_0.conda#6a42923f35087cc88a9fac31ef096ce6 -https://conda.anaconda.org/conda-forge/noarch/fsspec-2026.7.0-pyhd8ed1ab_0.conda#b9176565976c773a0739bd83deaf06cc +https://conda.anaconda.org/conda-forge/noarch/fsspec-2026.9.0-pyhd8ed1ab_0.conda#42de5cd170647599a84f468a42fabb89 +https://conda.anaconda.org/conda-forge/linux-64/google-crc32c-1.8.0-py312hda0ef95_2.conda#c5d2e1c00832cf3d97018829efd3d480 https://conda.anaconda.org/conda-forge/noarch/hpack-4.2.0-pyhd8ed1ab_0.conda#b395909221b9bd1df066e5930e18855b https://conda.anaconda.org/conda-forge/noarch/hyperframe-6.1.0-pyhd8ed1ab_0.conda#8e6923fc12f1fe8f8c4e5c9f343256ac -https://conda.anaconda.org/conda-forge/noarch/idna-3.19-pyhcf101f3_0.conda#a39ae05027e9b707742e41b30d296b75 +https://conda.anaconda.org/conda-forge/noarch/idna-3.20-pyh5ded981_0.conda#ee73be987f9de9ea5c2fdaeefa7b6491 https://conda.anaconda.org/conda-forge/noarch/imagesize-2.0.1-pyhd8ed1ab_0.conda#ebc56022e4ef7e74a829be94c53797ca -https://conda.anaconda.org/conda-forge/noarch/iniconfig-2.3.0-pyhd8ed1ab_0.conda#9614359868482abba1bd15ce465e3c42 +https://conda.anaconda.org/conda-forge/noarch/iniconfig-2.3.1-pyhd8ed1ab_0.conda#c0a803d97dd09da7e2c8b0fdfaa5739b https://conda.anaconda.org/conda-forge/noarch/iris-sample-data-2.5.2-pyhd8ed1ab_0.conda#895f6625dd8a246fece9279fcc12c1de https://conda.anaconda.org/conda-forge/linux-64/jsonschema-rs-0.37.4-py312h121d7ae_0.conda#1a2a2624770f712a536b8c5758c3387d -https://conda.anaconda.org/conda-forge/linux-64/kiwisolver-1.5.1-py312h9be0db6_0.conda#6ec45525bac89c279b6e5f4c47eec6bf +https://conda.anaconda.org/conda-forge/linux-64/kiwisolver-1.5.1-py312h9be0db6_3.conda#52258f682691a6838dd40316265d9cee https://conda.anaconda.org/conda-forge/linux-64/libarchive-3.8.9-gpl_h3152399_101.conda#680520cff78a974d564cd99cdf5ec0b5 -https://conda.anaconda.org/conda-forge/linux-64/libcblas-3.11.0-9_h0358290_openblas.conda#092c5649f3727af436ab0f67f48c3811 +https://conda.anaconda.org/conda-forge/linux-64/libcblas-3.11.0-11_h0358290_openblas.conda#73a248c30811075059b5aeda766c2624 https://conda.anaconda.org/conda-forge/linux-64/libgd-2.3.3-h5fbf134_12.conda#88c1c66987cd52a712eea89c27104be6 https://conda.anaconda.org/conda-forge/linux-64/libgl-1.7.0-ha4b6fd6_5.conda#81141db127a106eb5a91df69a29c9918 https://conda.anaconda.org/conda-forge/linux-64/libglx-devel-1.7.0-ha4b6fd6_5.conda#fb00b4fb800f2d431303a5c1625ef9d0 -https://conda.anaconda.org/conda-forge/linux-64/libhwloc-2.13.0-default_he001693_1000.conda#c197985b58bc813d26b42881f0021c82 -https://conda.anaconda.org/conda-forge/linux-64/liblapack-3.11.0-9_h47877c9_openblas.conda#e51473c2b7e1f9cb61daafccfd912abf +https://conda.anaconda.org/conda-forge/linux-64/libhwloc-2.15.0-default_hd7c4d0c_1001.conda#66c755b3bd9ec82308fa6ec7674f473f +https://conda.anaconda.org/conda-forge/linux-64/liblapack-3.11.0-11_h47877c9_openblas.conda#5622a13917855ce4b953b9e655cf9262 https://conda.anaconda.org/conda-forge/linux-64/libxkbcommon-1.13.2-h51789e4_1.conda#f888787e0eab7a8076a67d197e155ffc -https://conda.anaconda.org/conda-forge/linux-64/libxml2-devel-2.15.3-h49c6c72_1.conda#f986d9c9de80813d076d235f753889ff +https://conda.anaconda.org/conda-forge/linux-64/libxml2-devel-2.15.4-h7df9aa5_1.conda#c6f14f767cbfe20fa03a324beb6cd7eb https://conda.anaconda.org/conda-forge/noarch/locket-1.0.0-pyhd8ed1ab_0.tar.bz2#91e27ef3d05cc772ce627e51cff111c4 -https://conda.anaconda.org/conda-forge/linux-64/markupsafe-3.0.3-py312h8a5da7c_1.conda#93a4752d42b12943a355b682ee43285b +https://conda.anaconda.org/conda-forge/linux-64/markupsafe-3.0.4-py312h8b7ec3f_1.conda#ee4135fc1ce77066d2be65475bcfbbff https://conda.anaconda.org/conda-forge/noarch/mdurl-0.1.2-pyhd8ed1ab_1.conda#592132998493b3ff25fd7479396e8351 -https://conda.anaconda.org/conda-forge/linux-64/msgpack-python-1.2.2-py312h9be0db6_0.conda#881c695332c119ffd74b0a856f2c8fba +https://conda.anaconda.org/conda-forge/linux-64/msgpack-python-1.2.3-py312h9be0db6_0.conda#9f6cd0c3ce48c45d9a9c1645ecb186d1 +https://conda.anaconda.org/conda-forge/linux-64/msgspec-0.22.0-py312h5cc1888_0.conda#c762212af9018465d4bd165bd2c9938d https://conda.anaconda.org/conda-forge/linux-64/multidict-6.7.1-py312h8a5da7c_0.conda#17c77acc59407701b54404cfd3639cac https://conda.anaconda.org/conda-forge/noarch/munkres-1.1.4-pyhd8ed1ab_1.conda#37293a85a0f4f77bbd9cf7aaefc62609 https://conda.anaconda.org/conda-forge/linux-64/openldap-2.6.13-hbde042b_0.conda#680608784722880fbfe1745067570b00 https://conda.anaconda.org/conda-forge/noarch/packaging-26.3-pyhc364b38_0.conda#936687ed80f295a1f5dbcf8bd34c252c -https://conda.anaconda.org/conda-forge/linux-64/pillow-12.3.0-py312h50c33e8_0.conda#d749d04e1965315078f29320565c595a -https://conda.anaconda.org/conda-forge/noarch/platformdirs-4.11.5-pyhcf101f3_0.conda#152ec36401f710145a7db03475594410 +https://conda.anaconda.org/conda-forge/linux-64/pillow-12.3.0-py312h38079b3_4.conda#87d8e4053a968e8b20937e6cdd945a18 +https://conda.anaconda.org/conda-forge/noarch/platformdirs-4.13.0-pyh5ded981_0.conda#35cc2a176fc6f886d6efa41ecb5f1de0 https://conda.anaconda.org/conda-forge/noarch/pluggy-1.6.0-pyhf9edf01_1.conda#d7585b6550ad04c8c5e21097ada2888e https://conda.anaconda.org/conda-forge/linux-64/proj-9.8.1-he0df7b0_0.conda#b23619e5e9009eaa070ead0342034027 -https://conda.anaconda.org/conda-forge/linux-64/propcache-0.5.2-py312h8a5da7c_0.conda#d681d6ad9fa2ca3c8cacb7f3b23d54f3 -https://conda.anaconda.org/conda-forge/linux-64/psutil-7.2.2-py312h1b36aeb_1.conda#1653ce584b7859810f116e592eba9c0c -https://conda.anaconda.org/conda-forge/noarch/pycparser-3.0-pyhcf101f3_0.conda#9c5491066224083c41b6d5635ed7107b +https://conda.anaconda.org/conda-forge/linux-64/propcache-0.5.4-py312hfd18d64_0.conda#a1010a8328530ad72302e24a99104fe8 +https://conda.anaconda.org/conda-forge/linux-64/psutil-7.2.2-py312h1b36aeb_3.conda#2f8faed6c111c9ed0c1d217bc1be75f2 +https://conda.anaconda.org/conda-forge/noarch/pycparser-3.11-pyh5ded981_0.conda#3cd164745122ab557cc30879390089e8 https://conda.anaconda.org/conda-forge/noarch/pygments-2.21.0-pyhcf101f3_0.conda#2882dee445dfa45b0c5afce3ffb7730a -https://conda.anaconda.org/conda-forge/noarch/pyparsing-3.3.2-pyhcf101f3_0.conda#3687cc0b82a8b4c17e1f0eb7e47163d5 +https://conda.anaconda.org/conda-forge/noarch/pyparsing-3.3.3-pyh5ded981_0.conda#8bf43ae1312e9b966063459e04c20fa7 https://conda.anaconda.org/conda-forge/noarch/pyshp-3.1.6-pyhcf101f3_0.conda#6778d276eb7b4230d622bdd2a732c75e https://conda.anaconda.org/conda-forge/noarch/pysocks-1.7.1-pyha55dd90_7.conda#461219d1a5bd61342293efa2c0c90eac -https://conda.anaconda.org/conda-forge/linux-64/python-xxhash-3.7.0-py312h0d868a3_0.conda#5a2d6c150e20e46919f3810dfeb45e4b +https://conda.anaconda.org/conda-forge/linux-64/python-xxhash-3.7.0-py312h4b96c75_1.conda#037fc1dd0cb3ce8fb4310b0aac975b35 https://conda.anaconda.org/conda-forge/linux-64/pyyaml-6.0.3-py312h8a5da7c_1.conda#15878599a87992e44c059731771591cb -https://conda.anaconda.org/conda-forge/noarch/scooby-0.11.2-pyhd8ed1ab_0.conda#c5b13fecbbd3984f12a70599973c6551 +https://conda.anaconda.org/conda-forge/noarch/scooby-0.12.0-pyhd8ed1ab_0.conda#820b402968e73e94ae47ff322c3206a5 https://conda.anaconda.org/conda-forge/noarch/setuptools-84.0.0-pyh332efcf_0.conda#62ac906f1cd582c6c264c95625cb9d6f https://conda.anaconda.org/conda-forge/noarch/six-1.17.0-pyhe01879c_1.conda#3339e3b65d58accf4ca4fb8748ab16b3 https://conda.anaconda.org/conda-forge/noarch/snowballstemmer-3.1.1-pyhd8ed1ab_0.conda#46b6abe31482f6bca064b965696ae807 https://conda.anaconda.org/conda-forge/noarch/sortedcontainers-2.4.0-pyhd8ed1ab_1.conda#0401a17ae845fa72c7210e206ec5647d -https://conda.anaconda.org/conda-forge/noarch/soupsieve-2.9.2-pyhd8ed1ab_0.conda#ba33e6c8a46ee373fdf6dd8665212778 +https://conda.anaconda.org/conda-forge/noarch/soupsieve-3.0.3-pyhd8ed1ab_0.conda#bd2ea090567b9d0a53240bd1367c87ad https://conda.anaconda.org/conda-forge/noarch/sphinxcontrib-jsmath-1.0.1-pyhd8ed1ab_1.conda#fa839b5ff59e192f411ccc7dae6588bb https://conda.anaconda.org/conda-forge/noarch/tabulate-0.10.0-pyhcf101f3_0.conda#3b887b7b3468b0f494b4fad40178b043 https://conda.anaconda.org/conda-forge/noarch/tblib-3.2.2-pyhcf101f3_0.conda#f88bb644823094f436792f80fba3207e -https://conda.anaconda.org/conda-forge/noarch/tomli-2.4.1-pyhcf101f3_0.conda#b5325cf06a000c5b14970462ff5e4d58 -https://conda.anaconda.org/conda-forge/noarch/toolz-1.1.0-pyhd8ed1ab_1.conda#c07a6153f8306e45794774cf9b13bd32 -https://conda.anaconda.org/conda-forge/linux-64/tornado-6.5.8-py312h4c3975b_0.conda#5423c2b8f82d60320e65b9ff30babbdf +https://conda.anaconda.org/conda-forge/noarch/tomli-2.5.0-pyh5ded981_0.conda#36d6a162326c52d6339c8b606f43932d +https://conda.anaconda.org/conda-forge/noarch/toolz-1.2.0-pyh5ded981_0.conda#17f8db9088267a3cb38d56cc22d268ae +https://conda.anaconda.org/conda-forge/linux-64/tornado-6.5.10-py312h5cc1888_0.conda#b0414832500d750321eddce3822bea7b https://conda.anaconda.org/conda-forge/noarch/typing_extensions-4.16.0-pyhcf101f3_0.conda#c70ad746c22219b9700931707482992c -https://conda.anaconda.org/conda-forge/linux-64/unicodedata2-17.0.1-py312h4c3975b_0.conda#0b6c506ec1f272b685240e70a29261b8 +https://conda.anaconda.org/conda-forge/linux-64/unicodedata2-18.0.0-py312h5cc1888_0.conda#070e15c0fbb80b2f2fe405419b763f68 https://conda.anaconda.org/conda-forge/linux-64/xcb-util-cursor-0.1.6-hb03c661_0.conda#4d1fc190b99912ed557a8236e958c559 https://conda.anaconda.org/conda-forge/linux-64/xorg-libxcomposite-0.4.7-hb03c661_0.conda#f2ba4192d38b6cef2bb2c25029071d90 https://conda.anaconda.org/conda-forge/linux-64/xorg-libxcursor-1.2.3-hb9d3cd8_0.conda#2ccd714aa2242315acaf0a67faea780b @@ -243,124 +246,128 @@ https://conda.anaconda.org/conda-forge/linux-64/xorg-libxinerama-1.1.6-hecca717_ https://conda.anaconda.org/conda-forge/linux-64/xorg-libxrandr-1.5.5-h7cc23a3_1.conda#798a8c9d171859a022e1cb89e7d0eb10 https://conda.anaconda.org/conda-forge/linux-64/xorg-libxxf86vm-1.1.7-hb03c661_0.conda#665d152b9c6e78da404086088077c844 https://conda.anaconda.org/conda-forge/noarch/zict-3.0.0-pyhd8ed1ab_1.conda#e52c2ef711ccf31bb7f70ca87d144b9e -https://conda.anaconda.org/conda-forge/noarch/zipp-4.1.0-pyhcf101f3_0.conda#ba3dcdc8584155c97c648ae9c044b7a3 +https://conda.anaconda.org/conda-forge/noarch/zipp-4.1.1-pyh5ded981_0.conda#6cad4d2cec9b7b64538c225ac91d2e8a https://conda.anaconda.org/conda-forge/noarch/accessible-pygments-0.0.5-pyhd8ed1ab_1.conda#74ac5069774cdbc53910ec4d631a3999 -https://conda.anaconda.org/conda-forge/noarch/affine-3.0.0-pyhd8ed1ab_0.conda#71dd867a02739013ae65894ca32d67b7 +https://conda.anaconda.org/conda-forge/noarch/affine-3.0.1-pyhecae5ae_0.conda#aeb404cab9e719a49264ab40928f80c8 https://conda.anaconda.org/conda-forge/noarch/aiosignal-1.4.0-pyhd8ed1ab_0.conda#421a865222cd0c9d83ff08bc78bf3a61 -https://conda.anaconda.org/conda-forge/linux-64/aws-c-s3-0.13.1-h7508075_1.conda#2daf54d3eef87b757f68d7597534dd2f -https://conda.anaconda.org/conda-forge/linux-64/cffi-2.1.1-py312h703531f_2.conda#3101547f7c22db267bd40988f67d7ab1 +https://conda.anaconda.org/conda-forge/linux-64/aws-c-s3-1.0.0-hfe03eda_1.conda#7a8a507033f2e478670a617bb4ffb0b8 +https://conda.anaconda.org/conda-forge/linux-64/cffi-2.1.1-py312h703531f_3.conda#41f019d067f8c52c6d9283d8afb28889 https://conda.anaconda.org/conda-forge/noarch/click-default-group-1.2.4-pyhd8ed1ab_1.conda#7cd83dd6831b61ad9624a694e4afd7dc https://conda.anaconda.org/conda-forge/noarch/click-plugins-1.1.1.2-pyhd8ed1ab_0.conda#e9b05deb91c013e5224672a4ba9cf8d1 https://conda.anaconda.org/conda-forge/noarch/cligj-0.7.2-pyhd8ed1ab_2.conda#55c7804f428719241a90b152016085a1 -https://conda.anaconda.org/conda-forge/linux-64/cytoolz-1.1.0-py312h4c3975b_2.conda#29fd0bdf551881ab3d2801f7deaba528 +https://conda.anaconda.org/conda-forge/linux-64/cytoolz-1.2.0-py312h1b36aeb_0.conda#20f3a25d978a201f5dc592380781a7e6 +https://conda.anaconda.org/conda-forge/noarch/donfig-0.8.1.post1-pyhd8ed1ab_1.conda#c56a7fa5597ad78b62e1f5d21f7f8b8f https://conda.anaconda.org/conda-forge/noarch/exceptiongroup-1.3.1-pyhd8ed1ab_0.conda#8e662bd460bda79b1ea39194e3c4c9ab -https://conda.anaconda.org/conda-forge/linux-64/fonttools-4.63.0-py312h8a5da7c_0.conda#294fb524171e2a2748cb7fe708aba826 +https://conda.anaconda.org/conda-forge/linux-64/fonttools-4.66.1-py312hfd18d64_0.conda#3a646243ead357a2184be86b64be5203 https://conda.anaconda.org/conda-forge/linux-64/gl2ps-1.4.2-h36e74d4_2.conda#787c780ff43f9f79d78d01e476b81a7c https://conda.anaconda.org/conda-forge/linux-64/glew-2.3.0-h71661d4_0.conda#7eefecda1c71c380bfc406d16e78bbee https://conda.anaconda.org/conda-forge/noarch/h2-4.4.1-pyhcf101f3_0.conda#e652ac7756069c456d0da2a922cd7df5 https://conda.anaconda.org/conda-forge/noarch/importlib-metadata-9.0.1-pyhcf101f3_0.conda#77ec68e0aa61c3fff9ecbf840f93d64e -https://conda.anaconda.org/conda-forge/noarch/importlib_resources-7.1.0-pyhd8ed1ab_0.conda#0ba6225c279baf7ea9473a62ea0ec9ae https://conda.anaconda.org/conda-forge/noarch/jinja2-3.1.6-pyhcf101f3_1.conda#04558c96691bed63104678757beb4f8d -https://conda.anaconda.org/conda-forge/noarch/lazy-loader-0.5-pyhd8ed1ab_0.conda#75932da6f03a6bef32b70a51e991f6eb +https://conda.anaconda.org/conda-forge/noarch/lazy-loader-0.6-pyhd8ed1ab_0.conda#4ed11ec126e6da490282edfd0673fea0 https://conda.anaconda.org/conda-forge/linux-64/libgl-devel-1.7.0-ha4b6fd6_5.conda#f5f7fc038c611a25b22758c0a40d25c9 -https://conda.anaconda.org/conda-forge/linux-64/libharfbuzz-14.4.0-h23af247_0.conda#4463210c2a16ff5d3bf7f502af23f1f4 -https://conda.anaconda.org/conda-forge/linux-64/libpq-18.6-h9d76c99_0.conda#72e019c04ed02a179da38c56965802d1 -https://conda.anaconda.org/conda-forge/linux-64/libspatialite-5.1.0-gpl_hab3fe16_120.conda#5947bdda89ced07ab0d8a5d6503082c3 +https://conda.anaconda.org/conda-forge/linux-64/libharfbuzz-14.6.0-h23af247_2.conda#6a93147cab011f36d1659745c3f579e2 +https://conda.anaconda.org/conda-forge/linux-64/libpq-18.6-h3883810_3.conda#cc998047391971a452fed680158329f3 +https://conda.anaconda.org/conda-forge/linux-64/libspatialite-5.1.0-gpl_h03b4e02_121.conda#affb2721df7014b4abcd75cc539c6cef https://conda.anaconda.org/conda-forge/linux-64/libvulkan-loader-1.4.357.0-h0e34353_2.conda#c5f5f159b66332152197893427071695 https://conda.anaconda.org/conda-forge/noarch/markdown-it-py-4.2.0-pyhd8ed1ab_0.conda#6d03368f2b2b0a5fb6839df53b2eb5e0 https://conda.anaconda.org/conda-forge/noarch/nodeenv-1.10.0-pyhd8ed1ab_0.conda#eb52d14a901e23c39e9e7b4a1a5c015f -https://conda.anaconda.org/conda-forge/linux-64/numpy-2.5.2-py312h33ff503_0.conda#3e15aca2584b6c69dc09ba2aa5ae1503 +https://conda.anaconda.org/conda-forge/linux-64/numpy-2.5.4-py312he827f4e_0.conda#759b701f4b548ff6d06794d99788e6bb https://conda.anaconda.org/conda-forge/noarch/partd-1.4.2-pyhd8ed1ab_0.conda#0badf9c54e24cecfb0ad2f99d680c163 -https://conda.anaconda.org/conda-forge/linux-64/pyproj-3.7.2-py312hb99b6cc_5.conda#42e2d36478a1fcc0dbc6358ffdb46786 +https://conda.anaconda.org/conda-forge/linux-64/pyproj-3.8.0-py312hb99b6cc_1.conda#838b75fbf68ce1cf79970f30835ab334 https://conda.anaconda.org/conda-forge/noarch/python-dateutil-2.9.0.post0-pyhe01879c_2.conda#5b8d21249ff20967101ffa321cab24e8 -https://conda.anaconda.org/conda-forge/noarch/python-discovery-1.6.0-pyhcf101f3_0.conda#e87738e32eaf5dfa98a5d49f611d6312 -https://conda.anaconda.org/conda-forge/noarch/python-gil-3.12.14-hd8ed1ab_0.conda#3b084ff55bd64d09178194042f50f6ba -https://conda.anaconda.org/conda-forge/linux-64/tbb-2023.0.0-hab88423_2.conda#7073b15f9364ebc118998601ac6ca6a6 +https://conda.anaconda.org/conda-forge/noarch/python-discovery-1.6.2-pyh5ded981_0.conda#17e6c8b2fc2d021a217d317fe4f89fc4 +https://conda.anaconda.org/conda-forge/noarch/python-gil-3.12.15-hd8ed1ab_2.conda#deec3cfe011e962fe2eb872f2eae716c +https://conda.anaconda.org/conda-forge/linux-64/tbb-2023.1.0-haa8c3ee_2.conda#f805909f478c179e230baf7bd5878e26 https://conda.anaconda.org/conda-forge/noarch/typing-extensions-4.16.0-h69aa097_0.conda#c680b5747e8c4c8f23dca0bb7042a8fc -https://conda.anaconda.org/conda-forge/noarch/vcs_versioning-2.3.1-pyhcf101f3_0.conda#6043a03a3733302373b30e16ecc408b7 +https://conda.anaconda.org/conda-forge/noarch/vcs_versioning-2.6.0-pyh5ded981_0.conda#958f3e46d5025cdd9aa9c2417adceec8 https://conda.anaconda.org/conda-forge/noarch/wheel-0.48.0-pyhd8ed1ab_0.conda#9a2124517bfd5d792e0f7b86eefdfeb8 https://conda.anaconda.org/conda-forge/linux-64/xorg-libxtst-1.2.5-h7cc23a3_4.conda#752a5ac9322e0fbbc24146c1ce3ae44e -https://conda.anaconda.org/conda-forge/linux-64/yarl-1.24.5-py312h8a5da7c_0.conda#ef4788bdfaa84f3ae3a9c1390eaf3e73 +https://conda.anaconda.org/conda-forge/linux-64/yarl-1.25.1-py312hfd18d64_0.conda#91bb35918653d559dbd514c9064e512b https://conda.anaconda.org/conda-forge/noarch/_python_abi3_support-1.0-hd8ed1ab_3.conda#3845f3d75991bae0fb90884662f4327c -https://conda.anaconda.org/conda-forge/linux-64/aiohttp-3.14.3-py312h5d8c7f2_0.conda#fe9eb34a1c0e51dfc5d98f41c5ddc8b8 +https://conda.anaconda.org/conda-forge/linux-64/aiohttp-3.14.5-py312h8b7ec3f_0.conda#e5e9629c0a1c75dc79ec1ac35b0560da https://conda.anaconda.org/conda-forge/noarch/asv_runner-0.3.1-pyhd8ed1ab_0.conda#960a0f64467ce624941ad7c67b80a2e2 https://conda.anaconda.org/conda-forge/linux-64/at-spi2-core-2.40.3-h0630a04_0.tar.bz2#8cb2fc4cd6cc63f1369cfa318f581cc3 https://conda.anaconda.org/conda-forge/noarch/beautifulsoup4-4.15.0-pyha770c72_0.conda#3b261da3fe9b4168738712832410b022 -https://conda.anaconda.org/conda-forge/linux-64/cftime-1.6.5-py312h4f23490_1.conda#84bf349fad55056ed326fc550671b65c +https://conda.anaconda.org/conda-forge/linux-64/cftime-1.6.5-py312h0ec8046_2.conda#d2a80aab39dfab9fe3285cb2bb658259 https://conda.anaconda.org/conda-forge/noarch/colorspacious-1.1.2-pyhecae5ae_1.conda#04151bb8e351c6209caad045e4b1f4bd -https://conda.anaconda.org/conda-forge/linux-64/contourpy-1.3.3-py312h0a2e395_4.conda#43c2bc96af3ae5ed9e8a10ded942aa50 +https://conda.anaconda.org/conda-forge/linux-64/contourpy-1.4.0-py312h9be0db6_1.conda#64e62dfa49dd4688301b02d8cfea3687 https://conda.anaconda.org/conda-forge/noarch/dask-core-2026.8.0-pyhc364b38_0.conda#2ccdf45d2372e15c7edcb7a72d573e76 -https://conda.anaconda.org/conda-forge/linux-64/hdf5-2.2.0-nompi_hc529623_100.conda#336f7fb9af0c49cc246e2cff0992c3c4 -https://conda.anaconda.org/conda-forge/noarch/importlib-resources-7.1.0-pyhd8ed1ab_0.conda#e3bffa82b874f8b9a2631bddb3869529 -https://conda.anaconda.org/conda-forge/noarch/lazy_loader-0.5-pyhd8ed1ab_0.conda#4c8327180586e7b1cd8b6815fc8827f1 +https://conda.anaconda.org/conda-forge/linux-64/hdf5-2.2.0-nompi_hdbb200c_101.conda#66762e6067bf7d52586ff21c78c970db +https://conda.anaconda.org/conda-forge/noarch/lazy_loader-0.6-pyhd8ed1ab_0.conda#baa2be54811ef381364ae730a4613d0f https://conda.anaconda.org/conda-forge/linux-64/libegl-devel-1.7.0-ha4b6fd6_5.conda#577800fc8c9d8b7d9727246bbfde704e -https://conda.anaconda.org/conda-forge/linux-64/libgdal-core-3.13.3-h5fdb907_0.conda#c0723084e1315a5247d5a033d9eb5522 -https://conda.anaconda.org/conda-forge/linux-64/libharfbuzz-devel-14.4.0-h23af247_0.conda#26e37e05324d7bb723350d50b33057fc +https://conda.anaconda.org/conda-forge/linux-64/libgdal-core-3.13.3-h31aba09_1.conda#44c6780e3082468d8676aebdcd5068a6 +https://conda.anaconda.org/conda-forge/linux-64/libharfbuzz-devel-14.6.0-h23af247_2.conda#3cf459e930e6ea2e57bbf04be7fcb7c6 https://conda.anaconda.org/conda-forge/linux-64/libraqm-0.11.0-h6406941_0.conda#3ac89a48d224409739dbf6200e524373 -https://conda.anaconda.org/conda-forge/linux-64/mo_pack-0.3.1-py312h4f23490_2.conda#cec5bc5f7d374f8f8095f8e28e31f6cb -https://conda.anaconda.org/conda-forge/linux-64/pandas-3.0.5-py312h8ecdadd_1.conda#85eb29ade84d1bd97be5ec55607efb00 -https://conda.anaconda.org/conda-forge/linux-64/pango-1.58.2-hda50119_0.conda#6a2822aaf9a34ac3708904a47ff3dd7e +https://conda.anaconda.org/conda-forge/linux-64/mo_pack-0.3.1-py312h0ec8046_3.conda#eb270473659371f1291b774377eef4b5 +https://conda.anaconda.org/conda-forge/linux-64/numcodecs-0.17.0-py312h0ec8046_1.conda#f21c0e5e878142ca05450f56019f99da +https://conda.anaconda.org/conda-forge/linux-64/pandas-3.0.6-np2py312h91ec553_0.conda#2143c2cef115b49a934d526f7da91696 +https://conda.anaconda.org/conda-forge/linux-64/pango-1.58.2-h7bb47b9_1.conda#0b889b1c6b472d215d6332c6b195589c https://conda.anaconda.org/conda-forge/noarch/pip-26.2.1-pyh8b19718_0.conda#77b6cf3b0689d9fc68561df0db9b856d -https://conda.anaconda.org/conda-forge/linux-64/pykdtree-1.4.3-py312h4f23490_2.conda#1af24b0eb99b1158c0d8c64a4d6c5d79 +https://conda.anaconda.org/conda-forge/linux-64/pykdtree-1.4.3-py312h0ec8046_3.conda#653c74a0c65777b3aa69ed36106424d3 https://conda.anaconda.org/conda-forge/noarch/pytest-9.0.3-pyhc364b38_1.conda#6a991452eadf2771952f39d43615bb3e -https://conda.anaconda.org/conda-forge/linux-64/pywavelets-1.9.0-py312h4f23490_2.conda#ab856c36638ab1acf90e70349c525cf9 +https://conda.anaconda.org/conda-forge/noarch/pyvista-validation-0.2.2-pyh5ded981_0.conda#e06843d2fd3ed2b0472c7f11b52e7f3e +https://conda.anaconda.org/conda-forge/linux-64/pywavelets-1.10.0-py312h1b36aeb_1.conda#00e0ab1af3f50cf4007f268a14a226d3 https://conda.anaconda.org/conda-forge/noarch/rich-15.0.0-pyhcf101f3_0.conda#0242025a3c804966bf71aa04eee82f66 -https://conda.anaconda.org/conda-forge/linux-64/scipy-1.18.0-py312h54fa4ab_0.conda#f8d242c552b0f7f682451ce95879af5e -https://conda.anaconda.org/conda-forge/noarch/setuptools-scm-10.2.1-pyhcf101f3_0.conda#eefa3bc61c9224107d3a9afeb37552a9 -https://conda.anaconda.org/conda-forge/linux-64/shapely-2.1.2-py312h383787d_2.conda#69e400d3deca12ee7afd4b73a5596905 +https://conda.anaconda.org/conda-forge/linux-64/scipy-1.18.1-py312h106c528_1.conda#73cc9629383ee4e1f2166b314527b324 +https://conda.anaconda.org/conda-forge/noarch/setuptools-scm-10.3.4-pyh5ded981_0.conda#437557778d99cc6f311f3c0ce023ad85 +https://conda.anaconda.org/conda-forge/linux-64/shapely-2.2.0-np2py312h224e744_2.conda#c89c76fe2aa0b35631ea0255902f27fd https://conda.anaconda.org/conda-forge/noarch/snuggs-1.4.7-pyhd8ed1ab_2.conda#9aa358575bbd4be126eaa5e0039f835c +https://conda.anaconda.org/conda-forge/noarch/towncrier-26.9.0-pyhd8ed1ab_0.conda#d4167f473b71694bbca708347abee99e https://conda.anaconda.org/conda-forge/linux-64/ukkonen-1.1.0-py312hd9148b4_0.conda#55fd03988b1b1bc6faabbfb5b481ecd7 -https://conda.anaconda.org/conda-forge/noarch/urllib3-2.7.0-pyhd8ed1ab_0.conda#cbb88288f74dbe6ada1c6c7d0a97223e -https://conda.anaconda.org/conda-forge/noarch/virtualenv-21.7.7-pyhcf101f3_0.conda#768a861e1491dc8ee2a93fa8b7ee4317 +https://conda.anaconda.org/conda-forge/noarch/urllib3-2.8.0-pyhd8ed1ab_0.conda#407a3d3778570bae8e8fcf554e803e43 +https://conda.anaconda.org/conda-forge/noarch/virtualenv-21.14.7-pyh5ded981_0.conda#1635534aa129eb9ab6ce5e01929d86ff https://conda.anaconda.org/conda-forge/linux-64/at-spi2-atk-2.38.0-h0630a04_3.tar.bz2#6b889f174df1e0f816276ae69281af4d -https://conda.anaconda.org/conda-forge/linux-64/cf-units-3.3.1-py312h4f23490_0.conda#6aef45ba3c0123547eb7b0f15852cac9 +https://conda.anaconda.org/conda-forge/linux-64/cf-units-3.3.1-np2hc8c11d4_2.conda#9b777f085506bff796833be0ae292db1 +https://conda.anaconda.org/conda-forge/linux-64/charset-normalizer-3.5.2-h1114479_0.conda#78d5c6112c9320a1a797935cecfec7c2 https://conda.anaconda.org/conda-forge/noarch/distributed-2026.8.0-pyhc364b38_0.conda#f382c4eed5376ee372a6e67a0f6cd4d9 https://conda.anaconda.org/conda-forge/linux-64/epoxy-1.5.10-hb03c661_2.conda#057083b06ccf1c2778344b6dabace38b -https://conda.anaconda.org/conda-forge/linux-64/harfbuzz-14.4.0-ha770c72_0.conda#c2d4a4fe3216b26ef30e91d917c1b718 -https://conda.anaconda.org/conda-forge/noarch/identify-2.6.19-pyhd8ed1ab_0.conda#84a3233b709a289a4ddd7a2fd27dd988 +https://conda.anaconda.org/conda-forge/linux-64/harfbuzz-14.6.0-ha770c72_2.conda#0709eacc38906e8a237d9ef172af6688 +https://conda.anaconda.org/conda-forge/noarch/identify-2.6.20-pyhd8ed1ab_0.conda#be9ea4f030b4b2bc121b2a548b066c0b https://conda.anaconda.org/conda-forge/noarch/imagehash-4.3.2-pyhd8ed1ab_0.conda#efbc812363856906a80344b496389d2e https://conda.anaconda.org/conda-forge/linux-64/libnetcdf-4.10.1-nompi_he3e3c8e_201.conda#912aefa694a66897afeb7a2330b17d9f -https://conda.anaconda.org/conda-forge/linux-64/matplotlib-base-3.11.1-py312h4c94fcb_2.conda#97eb87f2704fc5212a05b5fb6202ace0 -https://conda.anaconda.org/conda-forge/linux-64/minijinja-2.22.0-py310hd8a072f_0.conda#ad9070e3f9613ecdd611e93bbb49c7e8 -https://conda.anaconda.org/conda-forge/noarch/pbr-7.1.1-pyhd8ed1ab_0.conda#51b000c0a6d35e244019f9fc8ab1c9f6 +https://conda.anaconda.org/conda-forge/linux-64/librsvg-2.62.4-ha427ee3_0.conda#f9bda2728debbde4c641bd1af103c623 +https://conda.anaconda.org/conda-forge/linux-64/matplotlib-base-3.11.2-py312hfe9aac7_1.conda#0745f1af620efcb1095d8ddabcb2ed8b +https://conda.anaconda.org/conda-forge/noarch/pbr-7.1.3-pyhd8ed1ab_0.conda#66415d5d2894f1d10928005a264c2717 https://conda.anaconda.org/conda-forge/noarch/pytest-cov-7.1.0-pyhcf101f3_0.conda#67d1790eefa81ed305b89d8e314c7923 -https://conda.anaconda.org/conda-forge/noarch/pytest-mock-3.15.1-pyhd8ed1ab_0.conda#0511afbe860b1a653125d77c719ece53 +https://conda.anaconda.org/conda-forge/noarch/pytest-mock-3.16.0-pyhd8ed1ab_0.conda#b038267c191ebae9d6374caf30cc2cca https://conda.anaconda.org/conda-forge/noarch/pytest-xdist-3.8.0-pyhd8ed1ab_0.conda#8375cfbda7c57fbceeda18229be10417 -https://conda.anaconda.org/conda-forge/linux-64/python-stratify-0.4.0-py312h4c3975b_3.conda#b0610b4174af97290f5f466a72583071 -https://conda.anaconda.org/conda-forge/linux-64/qt6-main-6.11.2-pl5321h9df5c37_0.conda#2ff98660281d1c6bfebda5bffa52cc89 -https://conda.anaconda.org/conda-forge/linux-64/rasterio-1.5.1-py312h5d657d5_1.conda#c0d4e6e0b2f6a3e73ab81630deab839d -https://conda.anaconda.org/conda-forge/noarch/requests-2.34.2-pyhcf101f3_0.conda#4a85203c1d80c1059086ae860836ffb9 -https://conda.anaconda.org/conda-forge/noarch/rich-rst-2.1.0-pyhd8ed1ab_0.conda#0633059139afdc156bc3feaf1ff9e734 -https://conda.anaconda.org/conda-forge/noarch/towncrier-25.8.0-pyhd8ed1ab_0.conda#3e0e8e44292bdac62f7bcbf0450b5cc7 +https://conda.anaconda.org/conda-forge/noarch/python-abi3-3.12-he1d15d4_3.conda#ea0c63b97585ce9587b3a33316b2995b +https://conda.anaconda.org/conda-forge/linux-64/python-stratify-0.4.0-py312h5cc1888_4.conda#bb80e017cd2afba2950e328ee2709e82 +https://conda.anaconda.org/conda-forge/linux-64/qt6-main-6.12.0-pl5321hcf69127_1.conda#db3e81b8f192e5c9ee8e1e8c215c40b0 +https://conda.anaconda.org/conda-forge/linux-64/rasterio-1.5.2-py312h5613827_1.conda#7af5ba4cf7007c202f4a15bd273047fc +https://conda.anaconda.org/conda-forge/noarch/rich-rst-2.2.0-pyhd8ed1ab_0.conda#d91900e108041fa4d06ff329970027a1 https://conda.anaconda.org/conda-forge/linux-64/viskores-1.1.1-cpu_ha53dc59_1.conda#228549cd2f01bb53664bd2b5b78d7389 -https://conda.anaconda.org/conda-forge/noarch/wslink-2.5.7-pyhd8ed1ab_0.conda#410dee7cbee308f33b01645f16f11efc +https://conda.anaconda.org/conda-forge/noarch/wslink-2.6.0-pyhd8ed1ab_0.conda#c3bcfd3bb22061d48cb833925795105f +https://conda.anaconda.org/conda-forge/noarch/zarr-3.4.1-pyhc364b38_0.conda#052cab2fa318f6ffeefc149addd9e6fb +https://conda.anaconda.org/conda-forge/noarch/adwaita-icon-theme-51.0-unix_0.conda#6f62266fb2bfc35a5071cfa1d6ffdcc9 https://conda.anaconda.org/conda-forge/linux-64/cartopy-0.25.0-np2py312h0f77346_3.conda#612c6dd5655f10e16f5d693c57259131 https://conda.anaconda.org/conda-forge/noarch/cmocean-4.0.3-pyhd8ed1ab_1.conda#dd71e4ec2fbffe38c0359976505f816e -https://conda.anaconda.org/conda-forge/noarch/cyclopts-4.23.1-pyhcf101f3_0.conda#252497d51f0133ecc9e68fc6eaa01a79 +https://conda.anaconda.org/conda-forge/noarch/cyclopts-5.2.0-pyh5ded981_0.conda#fae753a13c1df14b3aeaf2f8e5b5b314 https://conda.anaconda.org/conda-forge/linux-64/gtk3-3.24.52-ha5ea40c_0.conda#bcaea22d85999a4f17918acfab877e61 -https://conda.anaconda.org/conda-forge/linux-64/librsvg-2.62.3-h4c96295_0.conda#492c8d9b1c564c2e948b6cb4ba0f8261 +https://conda.anaconda.org/conda-forge/linux-64/minijinja-3.0.0-py311h1180b86_0.conda#ebba9c2cc80a43f736c13772bee155d8 https://conda.anaconda.org/conda-forge/noarch/nc-time-axis-1.4.1-pyhd8ed1ab_1.conda#9a2be7d0089f5934b550933ca0d9fe85 -https://conda.anaconda.org/conda-forge/linux-64/netcdf-fortran-4.6.3-nompi_hd5f62d4_101.conda#32619150ef344fce7cb11a593c82b96f -https://conda.anaconda.org/conda-forge/linux-64/netcdf4-1.7.4-nompi_py311hb115678_109.conda#d1f58fad9a5369f80109b81122a1af0d -https://conda.anaconda.org/conda-forge/noarch/pooch-1.9.0-pyhd8ed1ab_0.conda#dd4b6337bf8886855db6905b336db3c8 -https://conda.anaconda.org/conda-forge/noarch/pre-commit-4.6.2-pyha770c72_0.conda#c36b88db560b4e74f294380613f1a239 -https://conda.anaconda.org/conda-forge/noarch/requests-file-2.1.0-pyhd8ed1ab_1.conda#69be0472744969b97324d5d3b21845a9 +https://conda.anaconda.org/conda-forge/linux-64/netcdf-fortran-4.6.4-nompi_h52816da_101.conda#2f18bb969e8acd8a1f45293e9782374e +https://conda.anaconda.org/conda-forge/linux-64/netcdf4-1.7.4-nompi_py311hd34d2e2_111.conda#d697ebf773f33945402fe3e1fda13635 +https://conda.anaconda.org/conda-forge/noarch/pre-commit-4.7.0-pyha770c72_0.conda#167169a52ede77767f9fa66b52983a20 +https://conda.anaconda.org/conda-forge/noarch/requests-2.34.2-pyhcf101f3_0.conda#4a85203c1d80c1059086ae860836ffb9 https://conda.anaconda.org/conda-forge/noarch/sphinxcontrib-apidoc-0.3.0-py_1.tar.bz2#855b087883443abb10f5faf6eef40860 -https://conda.anaconda.org/conda-forge/linux-64/vtk-base-9.6.2-py312ha5f6713_7.conda#6b658f4d57ebbee46d7c67bc62a8be6d -https://conda.anaconda.org/conda-forge/noarch/adwaita-icon-theme-49.0-unix_0.conda#b3f0179590f3c0637b7eb5309898f79e +https://conda.anaconda.org/conda-forge/linux-64/vtk-base-9.7.0-py312hdc8235f_0.conda#f71b016e14a4df7e22c04b564afc21ec https://conda.anaconda.org/conda-forge/linux-64/esmf-8.9.1-nompi_h39d0119_4.conda#074ca9356f9e916ec51ad6ab47fd55ba -https://conda.anaconda.org/conda-forge/noarch/pyvista-0.48.4-pyhd8ed1ab_1.conda#7688fc6ae896d8fc848776f9ba60d716 +https://conda.anaconda.org/conda-forge/linux-64/graphviz-14.1.2-h8b86629_0.conda#341fc61cfe8efa5c72d24db56c776f44 +https://conda.anaconda.org/conda-forge/noarch/pooch-1.9.0-pyhd8ed1ab_0.conda#dd4b6337bf8886855db6905b336db3c8 +https://conda.anaconda.org/conda-forge/noarch/requests-file-2.1.0-pyhd8ed1ab_1.conda#69be0472744969b97324d5d3b21845a9 https://conda.anaconda.org/conda-forge/noarch/esmpy-8.9.1-pyhdfbf58e_0.conda#0155d4a55415cda7f7f09bc0b7760f51 +https://conda.anaconda.org/conda-forge/noarch/pyvista-0.49.1-pyha804496_0.conda#ea32ac5d548e14030818673f68e6a756 https://conda.anaconda.org/conda-forge/noarch/geovista-0.5.3-pyhd8ed1ab_1.conda#64348d05eedb1b1b5676f63101d004f2 -https://conda.anaconda.org/conda-forge/linux-64/graphviz-14.1.2-h8b86629_0.conda#341fc61cfe8efa5c72d24db56c776f44 https://conda.anaconda.org/conda-forge/noarch/pydata-sphinx-theme-0.19.0-pyhcf101f3_0.conda#87921f66a4dc56ce92e4ff13be5f63dc https://conda.anaconda.org/conda-forge/noarch/sphinx-copybutton-0.5.2-pyhd8ed1ab_1.conda#bf22cb9c439572760316ce0748af3713 https://conda.anaconda.org/conda-forge/noarch/sphinx-data-viewer-0.1.5-pyhd8ed1ab_1.conda#fbe6437c6899c54e52f9258518035aca https://conda.anaconda.org/conda-forge/noarch/sphinx-design-0.7.0-pyhd8ed1ab_0.conda#28eddfb8b9ecdd044a6f609f985398a7 -https://conda.anaconda.org/conda-forge/noarch/sphinx-gallery-0.21.0-pyhd8ed1ab_0.conda#9b783047bd5bef0998f129bef8fad477 +https://conda.anaconda.org/conda-forge/noarch/sphinx-gallery-0.22.1-pyhd8ed1ab_0.conda#b7ec27cd4e50d3c48e34965448f5e65e https://conda.anaconda.org/conda-forge/noarch/sphinx-last-updated-by-git-0.3.8-pyhe01879c_0.conda#405a232fb900fc631d2f1b5cdf01dea9 https://conda.anaconda.org/conda-forge/noarch/sphinx-sitemap-2.9.0-pyhcf101f3_0.conda#3baeff262222dc87e978a68702bc5797 -https://conda.anaconda.org/conda-forge/noarch/sphinx-markdown-builder-0.6.11-pyhd8ed1ab_0.conda#8c2c507ff41616cf325c1dddf11fbccd +https://conda.anaconda.org/conda-forge/noarch/sphinx-markdown-builder-0.6.11-pyh5ded981_1.conda#bde76da7789d94a15b7d570b41535f93 https://conda.anaconda.org/conda-forge/noarch/sphinx-llm-1.0.0-pyhcf101f3_0.conda#6afdccbc5bdf7a00ba1d4b7606e9fd9f https://conda.anaconda.org/conda-forge/noarch/sphinx-reredirects-1.1.0-pyhd8ed1ab_0.conda#8392493ce6fbb8ab72a101af8e8a5d03 https://conda.anaconda.org/conda-forge/noarch/sphinx_changelog-1.6.0-pyhd8ed1ab_1.conda#3c4521d251abd353af49c3293030f674 @@ -368,7 +375,7 @@ https://conda.anaconda.org/conda-forge/noarch/sphinxcontrib-applehelp-2.0.0-pyhd https://conda.anaconda.org/conda-forge/noarch/sphinxcontrib-devhelp-2.0.0-pyhd8ed1ab_1.conda#910f28a05c178feba832f842155cbfff https://conda.anaconda.org/conda-forge/noarch/sphinxcontrib-htmlhelp-2.1.0-pyhd8ed1ab_1.conda#e9fb3fe8a5b758b4aff187d434f94f03 https://conda.anaconda.org/conda-forge/noarch/sphinxcontrib-jquery-4.1-pyhd8ed1ab_1.conda#403185829255321ea427333f7773dd1f -https://conda.anaconda.org/conda-forge/noarch/sphinx-needs-8.4.0-pyhcf101f3_0.conda#a1b92a7251c7cbb12e48e1caaa73558b +https://conda.anaconda.org/conda-forge/noarch/sphinx-needs-8.5.0-pyh5ded981_0.conda#b977a530aee976c189a7629b963f7516 https://conda.anaconda.org/conda-forge/noarch/sphinxcontrib-qthelp-2.0.0-pyhd8ed1ab_1.conda#00534ebcc0375929b45c3039b5ba7636 https://conda.anaconda.org/conda-forge/noarch/sphinx-8.1.3-pyhd8ed1ab_1.conda#1a3281a0dc355c02b5506d87db2d78ac https://conda.anaconda.org/conda-forge/noarch/sphinxcontrib-serializinghtml-2.0.0-pyhd8ed1ab_0.conda#f77df1fcf9af03b7287342638befca77 diff --git a/requirements/locks/py313-linux-64.lock b/requirements/locks/py313-linux-64.lock index 464c78499a..cfee319c17 100644 --- a/requirements/locks/py313-linux-64.lock +++ b/requirements/locks/py313-linux-64.lock @@ -1,21 +1,21 @@ # Generated by conda-lock. # platform: linux-64 -# input_hash: 744ae3daa77a848be733ec9f02b1ea2eaa61374e7ccd2b0443f3814db0f06f9a +# input_hash: d36df080c40dbcaada47fdd3f5ee101a9b9ed5a3261d5620e155496bacf93c7c @EXPLICIT -https://conda.anaconda.org/conda-forge/linux-64/eigen-abi-5.0.1.80-hf414acd_0.conda#a85c1768af7780d67c6446c17848b76f +https://conda.anaconda.org/conda-forge/linux-64/eigen-abi-5.0.1.80-hf414acd_3.conda#56a1cca9a4a7bed2999630e5644c8d0e https://conda.anaconda.org/conda-forge/noarch/font-ttf-dejavu-sans-mono-2.37-hab24e00_0.tar.bz2#0c96522c6bdaed4b1566d11387caaf45 https://conda.anaconda.org/conda-forge/noarch/font-ttf-inconsolata-3.000-h77eed37_0.tar.bz2#34893075a5c9e55cdafac56607368fc6 https://conda.anaconda.org/conda-forge/noarch/font-ttf-source-code-pro-2.038-h77eed37_0.tar.bz2#4d59c254e01d9cde7957100457e2d5fb https://conda.anaconda.org/conda-forge/noarch/font-ttf-ubuntu-0.83-h77eed37_3.conda#49023d73832ef61042f6a237cb2687e7 https://conda.anaconda.org/conda-forge/linux-64/hicolor-icon-theme-0.17-ha770c72_3.conda#129e404c5b001f3ef5581316971e3ea0 https://conda.anaconda.org/conda-forge/linux-64/nlohmann_json-3.12.0-h54a6638_2.conda#fe93be7dd9209e6ae673962e3031ff51 -https://conda.anaconda.org/conda-forge/noarch/python_abi-3.13-8_cp313.conda#94305520c52a4aa3f6c2b1ff6008d9f8 +https://conda.anaconda.org/conda-forge/noarch/python_abi-3.13-9_cp313.conda#c5abc97af551bc12d71305852392f75c https://conda.anaconda.org/conda-forge/noarch/tzdata-2026c-h151e31d_0.conda#fcb489df604d100968b737f2cb6076c6 https://conda.anaconda.org/conda-forge/linux-64/utfcpp-4.09-ha770c72_0.conda#99884244028fe76046e3914f90d4ad05 https://conda.anaconda.org/conda-forge/noarch/ca-certificates-2026.7.22-hbd8a1cb_0.conda#0f51e2391ade309db462a55611263e9c https://conda.anaconda.org/conda-forge/noarch/fonts-conda-forge-1-hc364b38_1.conda#a7970cd949a077b7cb9696379d338681 https://conda.anaconda.org/conda-forge/linux-64/libglvnd-1.7.0-ha4b6fd6_5.conda#cfcf11edcfd4acf0094771a9c3b8587f -https://conda.anaconda.org/conda-forge/linux-64/libgomp-16.2.0-he0feb66_4.conda#89d2c1231f47bd818f5d624b9411459d +https://conda.anaconda.org/conda-forge/linux-64/libgomp-16.2.0-he0feb66_7.conda#d904c7ddd95e026254b4b80808ee9e12 https://conda.anaconda.org/conda-forge/linux-64/libzlib-1.3.2-h25fd6f3_3.conda#0de0122d9570a8ab637c6b73db268389 https://conda.anaconda.org/conda-forge/linux-64/_openmp_mutex-4.5-20_gnu.conda#a9f577daf3de00bca7c3c76c0ecbd1de https://conda.anaconda.org/conda-forge/noarch/fonts-conda-ecosystem-1-0.tar.bz2#fee5683a3f04bd15cbd8318b096a27ab @@ -23,23 +23,23 @@ https://conda.anaconda.org/conda-forge/linux-64/libegl-1.7.0-ha4b6fd6_5.conda#3a https://conda.anaconda.org/conda-forge/linux-64/libopengl-1.7.0-ha4b6fd6_5.conda#d207a2e9bae9da476bc0a603215d5167 https://conda.anaconda.org/conda-forge/linux-64/zlib-1.3.2-h25fd6f3_3.conda#6acb86426229f96f93e5468d1df3a5e8 https://conda.anaconda.org/conda-forge/linux-64/zstd-1.5.7-hb78ec9c_7.conda#aa459086047c0e5e27023ab19f8cb86a -https://conda.anaconda.org/conda-forge/linux-64/ld_impl_linux-64-2.46.1-default_hbd61a6d_102.conda#449500f2c089da11c40f5c21312e3e07 -https://conda.anaconda.org/conda-forge/linux-64/libgcc-16.2.0-ha9f2e26_4.conda#cba14d01083fc62ffd32c24d7d390633 +https://conda.anaconda.org/conda-forge/linux-64/ld_impl_linux-64-2.46.1-default_hbd61a6d_103.conda#2362c70e7e79827c32db35547d9c179b +https://conda.anaconda.org/conda-forge/linux-64/libgcc-16.2.0-ha9f2e26_7.conda#735629b400b70eeab21a2ba8f5b1c2ce https://conda.anaconda.org/conda-forge/linux-64/alsa-lib-1.2.16.1-h7cc23a3_1.conda#7094e0d8d14de0eff6d83f0d2f1f661e -https://conda.anaconda.org/conda-forge/linux-64/aws-c-common-0.14.3-hb03c661_0.conda#ce8664d7c01d4a598ad38107b289b5ec +https://conda.anaconda.org/conda-forge/linux-64/aws-c-common-1.0.3-hebe6cf0_0.conda#cdb5797968bfe1737f1b71f525fc6518 https://conda.anaconda.org/conda-forge/linux-64/bzip2-1.0.8-hda65f42_10.conda#e675fabcf81499adc7edf58124fb1e01 https://conda.anaconda.org/conda-forge/linux-64/c-ares-1.34.8-hebe6cf0_2.conda#3ad2b403be8fc5612b9159e2ce621f69 -https://conda.anaconda.org/conda-forge/linux-64/fribidi-1.0.16-hb03c661_1.conda#1cd10eda5692519d01bb20e086e214c9 -https://conda.anaconda.org/conda-forge/linux-64/giflib-6.1.3-h280c20c_1.conda#6d9a90cfee2c7343d6c05a1b37ef75fb -https://conda.anaconda.org/conda-forge/linux-64/json-c-0.18-h6688a6e_0.conda#38f5dbc9ac808e31c00650f7be1db93f +https://conda.anaconda.org/conda-forge/linux-64/fribidi-1.0.17-h7cc23a3_0.conda#f9d5302bca196ee87e62f393f2626562 +https://conda.anaconda.org/conda-forge/linux-64/giflib-6.1.3-hebe6cf0_2.conda#07bdd65afe03df3fb5dbcaa8257c64a3 +https://conda.anaconda.org/conda-forge/linux-64/json-c-0.18-hebe6cf0_2.conda#9aa77c4e7f9a988df94da18a7b9b90fe https://conda.anaconda.org/conda-forge/linux-64/keyutils-1.6.3-h7cc23a3_1.conda#ba55d1b89fd7775e67de8291029b4059 -https://conda.anaconda.org/conda-forge/linux-64/libbrotlicommon-1.2.0-h39a168f_3.conda#7a2499a177753582fb7ae7e9dc4a908a +https://conda.anaconda.org/conda-forge/linux-64/libbrotlicommon-1.2.0-h39a168f_4.conda#df210655e30a05e3b069c91b7aaddd2d https://conda.anaconda.org/conda-forge/linux-64/libdeflate-1.25-hd45a770_1.conda#40f9b31aa9cf007789867df0decd0492 https://conda.anaconda.org/conda-forge/linux-64/libev-4.33-h280c20c_3.conda#7bc31538d6e3fb349e897353969237ec -https://conda.anaconda.org/conda-forge/linux-64/libexpat-2.8.1-hecca717_1.conda#b24d3c612f71e7aa74158d92106318b2 +https://conda.anaconda.org/conda-forge/linux-64/libexpat-2.8.5-hd2095e1_0.conda#72cfd2b95b0540247220b42f58a148d7 https://conda.anaconda.org/conda-forge/linux-64/libffi-3.7.0-h81df57d_1.conda#6525a0b06aa4fd390795f0740636a9dd -https://conda.anaconda.org/conda-forge/linux-64/libgcc-ng-16.2.0-h69a702a_4.conda#b3e52878163a841f6fb951989cc0b217 -https://conda.anaconda.org/conda-forge/linux-64/libgfortran5-16.2.0-h6b99dfc_4.conda#c22348a769bb072b6184eb6f7f05e4f2 +https://conda.anaconda.org/conda-forge/linux-64/libgcc-ng-16.2.0-h69a702a_7.conda#2005515ac9340f816fb2ac7e83afe553 +https://conda.anaconda.org/conda-forge/linux-64/libgfortran5-16.2.0-h6b99dfc_7.conda#7719ed7be38ad8c492c4bd5cc2bce0c4 https://conda.anaconda.org/conda-forge/linux-64/libiconv-1.18-h0cb94f2_3.conda#f92233bf33e24a25668bb2119e2c51f9 https://conda.anaconda.org/conda-forge/linux-64/libjpeg-turbo-3.2.0-hb03c661_1.conda#898d1c9793eaa52efc4727bd84d2e39a https://conda.anaconda.org/conda-forge/linux-64/liblzma-5.8.3-hb03c661_1.conda#1390b7c5ac0b1d8e447bc5efa6d3c8c2 @@ -48,80 +48,82 @@ https://conda.anaconda.org/conda-forge/linux-64/libnsl-2.0.1-hb9d3cd8_1.conda#d8 https://conda.anaconda.org/conda-forge/linux-64/libntlm-1.8-hb9d3cd8_0.conda#7c7927b404672409d9917d49bff5f2d6 https://conda.anaconda.org/conda-forge/linux-64/libogg-1.3.5-hd0c01bc_1.conda#68e52064ed3897463c0e958ab5c8f91b https://conda.anaconda.org/conda-forge/linux-64/libpciaccess-0.19-hb03c661_1.conda#35fa2b34bbced424e6976d30f5fde576 -https://conda.anaconda.org/conda-forge/linux-64/libpng-1.6.58-h922cc85_1.conda#fcf71c8d979148873f6f8ad4cfc73d86 -https://conda.anaconda.org/conda-forge/linux-64/libstdcxx-16.2.0-h934c35e_4.conda#2f2ef0d96de5bdd8c1270ff22fdf9352 -https://conda.anaconda.org/conda-forge/linux-64/libuuid-2.42.2-h5347b49_0.conda#01bb81d12c957de066ea7362007df642 +https://conda.anaconda.org/conda-forge/linux-64/libpng-1.6.59-h922cc85_0.conda#8c9caf742a485ac688ffa88bbadbf0be +https://conda.anaconda.org/conda-forge/linux-64/libpython-3.13.16-hd51c5da_102_cp313.conda#de5467b4cae23d95c711f84b2033341d +https://conda.anaconda.org/conda-forge/linux-64/libstdcxx-16.2.0-h934c35e_7.conda#9b6707e0caca917fa6ea9ee473f37a5d +https://conda.anaconda.org/conda-forge/linux-64/libuuid-2.42.4-hcfc3c73_0.conda#54a948337e66706380ff24d6b5fde63f https://conda.anaconda.org/conda-forge/linux-64/libwebp-base-1.6.0-hd42ef1d_1.conda#9332b53d0ea93c5d39e33be03a0c611a https://conda.anaconda.org/conda-forge/linux-64/libxcrypt-4.4.38-h280c20c_0.conda#f7a7ff5a6ab331e037abd34f379a631d https://conda.anaconda.org/conda-forge/linux-64/lzo-2.10-hebe6cf0_1003.conda#10e0aa58ed390ba598b728badf9eb1b7 https://conda.anaconda.org/conda-forge/linux-64/ncurses-6.6-hdb14827_1.conda#ee6c0cd80a60961a1f48aa3e0b91f986 -https://conda.anaconda.org/conda-forge/linux-64/openssl-3.6.4-h781a0a9_0.conda#16e3034a330cc625f2c685edec60cf4e -https://conda.anaconda.org/conda-forge/linux-64/pthread-stubs-0.4-hb03c661_1003.conda#df2c27f36bdb0dde779f55b5df76a352 +https://conda.anaconda.org/conda-forge/linux-64/openssl-3.6.5-h781a0a9_0.conda#5a51a2ea653f120240b1a9bf76e0aae9 +https://conda.anaconda.org/conda-forge/linux-64/pthread-stubs-0.4-h7cc23a3_1004.conda#bb66b610707b811e3c65181a6431e7a8 https://conda.anaconda.org/conda-forge/linux-64/tk-8.6.13-noxft_h1df4ec4_4.conda#676a2f9b1c4fcf57b70aa5c65f6c60d0 https://conda.anaconda.org/conda-forge/linux-64/xorg-libice-1.1.2-h280c20c_0.conda#85c9442aec283b4e464fa9ecc484a2f3 -https://conda.anaconda.org/conda-forge/linux-64/xorg-libxau-1.0.12-hb03c661_2.conda#f06ef439c280a5f90b8bf62355008dbc -https://conda.anaconda.org/conda-forge/linux-64/xorg-libxdmcp-1.1.5-hb03c661_2.conda#2e66c929f3d879708335b6ea4557c838 -https://conda.anaconda.org/conda-forge/linux-64/xorg-xorgproto-2025.1-h280c20c_1.conda#3b51576511038b50fdbd05245e22e4b1 +https://conda.anaconda.org/conda-forge/linux-64/xorg-libxau-1.0.12-h7cc23a3_2.conda#da33efee1a93603d92e06d4a2b68def6 +https://conda.anaconda.org/conda-forge/linux-64/xorg-libxdmcp-1.1.5-h7cc23a3_2.conda#eabfc45786244de252090cf8d55128a8 +https://conda.anaconda.org/conda-forge/linux-64/xorg-xorgproto-2025.1-hebe6cf0_3.conda#c3d0a6f03ba3aa312777b41b20a71ec0 https://conda.anaconda.org/conda-forge/linux-64/xxhash-0.8.3-hec8ed23_1.conda#72ab4ea51529ec63ff010ae9fb62b02f https://conda.anaconda.org/conda-forge/linux-64/yaml-0.2.5-hebe6cf0_3.conda#e741576fb8f89821ac7c1c537322a33d -https://conda.anaconda.org/conda-forge/linux-64/aws-c-cal-0.9.15-h6bbde05_1.conda#8f2b374c79908f271be3d0e9f1c462a8 -https://conda.anaconda.org/conda-forge/linux-64/aws-c-compression-0.3.2-h01ee8f8_5.conda#ae862abdbb3160f97478d3d4749164c0 -https://conda.anaconda.org/conda-forge/linux-64/aws-c-sdkutils-0.2.7-h01ee8f8_3.conda#412f0cf18cc4c4945655dba954ca2d76 -https://conda.anaconda.org/conda-forge/linux-64/aws-checksums-0.2.10-h01ee8f8_5.conda#4d70caf5260e0787675c3bf61ae5b64c +https://conda.anaconda.org/conda-forge/linux-64/aws-c-cal-1.0.0-hcf7459e_1.conda#2009b71b358163738f339c76640b512c +https://conda.anaconda.org/conda-forge/linux-64/aws-c-compression-1.0.0-ha404f16_1.conda#da0c7b86d5379eb44acfa084897e8b98 +https://conda.anaconda.org/conda-forge/linux-64/aws-c-sdkutils-1.0.0-ha404f16_1.conda#cbbd999c74a030951365225909e376c7 +https://conda.anaconda.org/conda-forge/linux-64/aws-checksums-1.0.1-ha404f16_0.conda#10e6e2f11271e66dfad43eb002359f22 https://conda.anaconda.org/conda-forge/linux-64/cli11-2.7.2-h54a6638_1.conda#42761b55062088f6f7fb58d0633e0769 https://conda.anaconda.org/conda-forge/linux-64/double-conversion-3.4.0-hecca717_0.conda#dbe3ec0f120af456b3477743ffd99b74 https://conda.anaconda.org/conda-forge/linux-64/fmt-12.1.0-h76c4fd7_1.conda#a70bb6d42ad121aef456e0007e92bed6 https://conda.anaconda.org/conda-forge/linux-64/geos-3.14.1-h55b0958_0.conda#5f856b72ca9cd5b5b4a1c43551804622 https://conda.anaconda.org/conda-forge/linux-64/graphite2-1.3.15-h54a6638_1.conda#f9fe2984587fa8235a6af6004760cd18 https://conda.anaconda.org/conda-forge/linux-64/icu-78.3-py310h44b86e0_2.conda#72a381cbad04f24b1c2a43ef707f45b4 -https://conda.anaconda.org/conda-forge/linux-64/jsoncpp-1.9.8-h4dbf13b_1.conda#16e15ba8628a2cd3c752f871791056f9 +https://conda.anaconda.org/conda-forge/linux-64/jsoncpp-1.9.8-h4dbf13b_2.conda#c8985329431780d56025998322650199 https://conda.anaconda.org/conda-forge/linux-64/lerc-4.2.0-hdb68285_0.conda#fb9d356b1a57d6d54768be7ebd5fce09 https://conda.anaconda.org/conda-forge/linux-64/libaec-1.1.5-h088129d_0.conda#86f7414544ae606282352fa1e116b41f -https://conda.anaconda.org/conda-forge/linux-64/libbrotlidec-1.2.0-ha411449_3.conda#6ab3315dc56618d652c1da42a648a129 -https://conda.anaconda.org/conda-forge/linux-64/libbrotlienc-1.2.0-h018ffa1_3.conda#2ac965638d4c6b2b38383bb1aebaf543 +https://conda.anaconda.org/conda-forge/linux-64/libbrotlidec-1.2.0-ha411449_4.conda#0cfd601eb03cca403dcc248844787486 +https://conda.anaconda.org/conda-forge/linux-64/libbrotlienc-1.2.0-h018ffa1_4.conda#4136f6e4ef4835cc7df39a707cbd511c https://conda.anaconda.org/conda-forge/linux-64/libdrm-2.4.129-h7cc23a3_0.conda#64cc91512b6278c315349dc13c53f680 -https://conda.anaconda.org/conda-forge/linux-64/libedit-3.1.20250104-pl5321h373387f_1.conda#50708d3b951d0f8e2d7f2df5b5edc040 +https://conda.anaconda.org/conda-forge/linux-64/libedit-3.1.20260512-pl5321h373387f_0.conda#27dbd7e2984b8fe48e069379636ccfdd https://conda.anaconda.org/conda-forge/linux-64/libfreetype6-2.14.3-h5e6c136_2.conda#b72a266a9317036fd0464cb98b027cd0 -https://conda.anaconda.org/conda-forge/linux-64/libgfortran-16.2.0-h69a702a_4.conda#5e92b8413fd1c8f8f3006f4661b12def +https://conda.anaconda.org/conda-forge/linux-64/libgfortran-16.2.0-h69a702a_7.conda#3a8d285f5f22138ef826ad54cf267e73 https://conda.anaconda.org/conda-forge/linux-64/libglu-9.0.3-h5888daf_1.conda#8422fcc9e5e172c91e99aef703b3ce65 -https://conda.anaconda.org/conda-forge/linux-64/libhwy-1.4.0-h57c4cff_1.conda#d58bfbaaf51ed85771eae92c2e7f5816 +https://conda.anaconda.org/conda-forge/linux-64/libhwy-1.4.0-h57c4cff_2.conda#145497b38fe581a7760dd88b7377c96b https://conda.anaconda.org/conda-forge/linux-64/libnghttp2-1.68.1-h74cf4be_1.conda#75d211f04ac87506f1f43420712a69fe https://conda.anaconda.org/conda-forge/linux-64/libssh2-1.11.1-h6154650_1.conda#5c526561e1d2a2e071941174eae84557 -https://conda.anaconda.org/conda-forge/linux-64/libstdcxx-ng-16.2.0-hdf11a46_4.conda#de0dceacf3e33c5fc88e167885dc8274 +https://conda.anaconda.org/conda-forge/linux-64/libstdcxx-ng-16.2.0-hdf11a46_7.conda#25a23bde5e26d548374f3e02b4669670 https://conda.anaconda.org/conda-forge/linux-64/libudunits2-2.2.28-h40f5838_3.conda#4bdace082e911a3e1f1f0b721bed5b56 https://conda.anaconda.org/conda-forge/linux-64/libvorbis-1.3.7-h54a6638_2.conda#b4ecbefe517ed0157c37f8182768271c https://conda.anaconda.org/conda-forge/linux-64/libxcb-1.17.0-hb83e432_2.conda#61f0ffba9161cbb3a77722869b21e4bb -https://conda.anaconda.org/conda-forge/linux-64/libzip-1.11.2-h6991a6a_0.conda#a7b27c075c9b7f459f1c022090697cba +https://conda.anaconda.org/conda-forge/linux-64/libzip-1.11.4-h6008cf6_0.conda#05adf41813469e50f2103db6eba38edb https://conda.anaconda.org/conda-forge/linux-64/lz4-c-1.10.0-hee9eb32_2.conda#e38a3253d72ab07d471483f24947e2a2 -https://conda.anaconda.org/conda-forge/linux-64/minizip-4.2.2-hb71707f_0.conda#88450ba743694055e218d03a2fdd561e -https://conda.anaconda.org/conda-forge/linux-64/muparser-2.3.5-h5888daf_0.conda#ab3e3db511033340e75e7002e80ce8c0 +https://conda.anaconda.org/conda-forge/linux-64/minizip-4.2.2-haac24f3_1.conda#3fee8d11b0d8a54fb899749052accb4f +https://conda.anaconda.org/conda-forge/linux-64/muparser-2.3.5-hee9eb32_1.conda#7de84fd76ec2bdb7ac617507de3d3052 https://conda.anaconda.org/conda-forge/linux-64/pcre2-10.47-h8b3dc9c_1.conda#06f6113cf1ff4a54b65f87ead132a5f1 https://conda.anaconda.org/conda-forge/linux-64/pixman-0.46.4-h54a6638_3.conda#0ee5bb30034b081a1386c1e2c98ab0a7 https://conda.anaconda.org/conda-forge/linux-64/pugixml-1.15-h3f63f65_0.conda#b11a4c6bf6f6f44e5e143f759ffa2087 https://conda.anaconda.org/conda-forge/linux-64/readline-8.3-hd6e31c0_1.conda#69c01c781e7c8190bbdaf79e3848c5ca -https://conda.anaconda.org/conda-forge/linux-64/s2n-1.7.6-h7e3ee7f_0.conda#e112cadcd79412d81496e784884bddf0 -https://conda.anaconda.org/conda-forge/linux-64/snappy-1.2.2-h03e3b7b_1.conda#98b6c9dc80eb87b2519b97bcf7e578dd +https://conda.anaconda.org/conda-forge/linux-64/s2n-1.7.7-ha5dc4ae_0.conda#4928f1e1964a6f38f0da349b9d0e1c6a +https://conda.anaconda.org/conda-forge/linux-64/snappy-1.2.2-h34e00bb_2.conda#2b7e31ffbd476eecf33b3b833692339c https://conda.anaconda.org/conda-forge/linux-64/wayland-1.26.0-hc1c935e_2.conda#88a47e22b53e50865b1cabe2eb648352 https://conda.anaconda.org/conda-forge/linux-64/xorg-libsm-1.2.6-h0d788c3_1.conda#aa7459ed9ad086ba11dda843d764b33d -https://conda.anaconda.org/conda-forge/linux-64/zfp-1.0.1-h909a3a2_5.conda#6a0eb48e58684cca4d7acc8b7a0fd3c7 +https://conda.anaconda.org/conda-forge/linux-64/zfp-1.0.1-h79e284d_6.conda#9d7141ec747a9ede5da83e864fcc8213 https://conda.anaconda.org/conda-forge/linux-64/zlib-ng-2.3.3-hce19668_1.conda#1726acec19beeed1c764385cd8622405 -https://conda.anaconda.org/conda-forge/linux-64/aws-c-io-0.27.5-h4f381ba_1.conda#948f5a4fddac779b7942626e5807caba +https://conda.anaconda.org/conda-forge/linux-64/aws-c-io-1.0.0-h07ba609_0.conda#72593f68772d8c7e72c8a09681b0fe42 https://conda.anaconda.org/conda-forge/linux-64/blosc-1.21.6-he440d0b_1.conda#2c2fae981fd2afd00812c92ac47d023d -https://conda.anaconda.org/conda-forge/linux-64/brotli-bin-1.2.0-h9908984_3.conda#8929291d9efc59715df1cfa7daf5e3ed -https://conda.anaconda.org/conda-forge/linux-64/freexl-2.0.0-h9dce30a_2.conda#ecb5d11305b8ba1801543002e69d2f2f +https://conda.anaconda.org/conda-forge/linux-64/brotli-bin-1.2.0-h9908984_4.conda#c48c09444f3736800ea0b9550c365baf +https://conda.anaconda.org/conda-forge/linux-64/freexl-2.0.0-h1f12c17_3.conda#4a0a8fd87ee4767db791047546f56501 https://conda.anaconda.org/conda-forge/linux-64/hdf4-4.2.15-h2a13503_7.conda#bd77f8da987968ec3927990495dc22e4 https://conda.anaconda.org/conda-forge/linux-64/krb5-1.22.2-hbc21106_2.conda#53318d715316929a574f83591308b1f8 +https://conda.anaconda.org/conda-forge/linux-64/libcrc32c-1.1.2-h9c3ff4c_0.tar.bz2#c965a5aa0d5c1c37ffc62dff36e28400 https://conda.anaconda.org/conda-forge/linux-64/libfreetype-2.14.3-ha770c72_2.conda#f8054e759d0ddddaf34a5c8fedc900a1 -https://conda.anaconda.org/conda-forge/linux-64/libglib-2.88.3-he503a2a_2.conda#533d342dedadf134c67760ce6fc5b748 -https://conda.anaconda.org/conda-forge/linux-64/libjxl-0.12.0-heb2dce7_2.conda#485788d339785bac87fe86315b4a0627 +https://conda.anaconda.org/conda-forge/linux-64/libglib-2.90.0-h569388d_0.conda#c85e9539551f1fd21104e518d940c244 +https://conda.anaconda.org/conda-forge/linux-64/libjxl-0.12.0-heb2dce7_4.conda#8770a39fd4c5559357efc7cdb8f0c742 https://conda.anaconda.org/conda-forge/linux-64/libmo_unpack-3.1.2-hf484d3e_1001.tar.bz2#95f32a6a5a666d33886ca5627239f03d -https://conda.anaconda.org/conda-forge/linux-64/libopenblas-0.3.34-pthreads_h94d23a6_0.conda#c282d68f272927612462b5d626838ef1 +https://conda.anaconda.org/conda-forge/linux-64/libopenblas-0.3.34-pthreads_hf13c14d_2.conda#7e19fcd73d7f0997d3cb3326d2586a86 https://conda.anaconda.org/conda-forge/linux-64/libpsl-0.23.1-hd9e3e90_1.conda#77be60417aa572afcb48cbe0f967401d -https://conda.anaconda.org/conda-forge/linux-64/librttopo-1.1.0-h46dd2a8_20.conda#df81fd57eacf341588d728c97920e86d -https://conda.anaconda.org/conda-forge/linux-64/libsqlite-3.53.4-h13e7031_1.conda#e72bbec309c2b0f37823ee7d4fabfcd3 +https://conda.anaconda.org/conda-forge/linux-64/librttopo-1.1.0-h55b0958_21.conda#6c639697369e9669032ca08d0a566282 +https://conda.anaconda.org/conda-forge/linux-64/libsqlite-3.54.0-h13e7031_1.conda#bb145547395d3329f3a1d930de9848b2 https://conda.anaconda.org/conda-forge/linux-64/libtheora-1.2.0-h85c0a6d_0.conda#177ef038f292390abe69137711fc0cbf https://conda.anaconda.org/conda-forge/linux-64/libtiff-4.7.2-hcc2c06a_1.conda#0907d876f460ebc941ae590338b6102f -https://conda.anaconda.org/conda-forge/linux-64/libxml2-16-2.15.3-hca6bf5a_1.conda#20e4d67ec908f0405124f11612c48305 +https://conda.anaconda.org/conda-forge/linux-64/libxml2-16-2.15.4-hf3af7cc_1.conda#046d9eada488e1a1657c80b55c23da52 https://conda.anaconda.org/conda-forge/linux-64/qhull-2020.2-h434a139_5.conda#353823361b1d27eb3960efb076dfcaf6 https://conda.anaconda.org/conda-forge/linux-64/udunits2-2.2.28-h40f5838_3.conda#6bb8deb138f87c9d48320ac21b87e7a1 https://conda.anaconda.org/conda-forge/linux-64/uriparser-0.9.8-hac33072_0.conda#d71d3a66528853c0a1ac2c02d79a0284 @@ -129,29 +131,29 @@ https://conda.anaconda.org/conda-forge/linux-64/xcb-util-0.4.1-h4f16b4b_2.conda# https://conda.anaconda.org/conda-forge/linux-64/xcb-util-keysyms-0.4.1-hb711507_0.conda#ad748ccca349aec3e91743e08b5e2b50 https://conda.anaconda.org/conda-forge/linux-64/xcb-util-renderutil-0.3.10-hb711507_0.conda#0e0cbe0564d03a99afd5fd7b362feecd https://conda.anaconda.org/conda-forge/linux-64/xcb-util-wm-0.4.2-hb711507_0.conda#608e0ef8256b81d04456e8d211eee3e8 -https://conda.anaconda.org/conda-forge/linux-64/xerces-c-3.3.0-hd9031aa_1.conda#66a1db55ecdb7377d2b91f54cd56eafa +https://conda.anaconda.org/conda-forge/linux-64/xerces-c-3.3.0-h78fed68_2.conda#f2ff0de6c209e90c53b3a195c2708f5f https://conda.anaconda.org/conda-forge/linux-64/xorg-libx11-1.8.13-he1eb515_1.conda#8c282bbe4808a3cc80a5c98e9aec1cfc https://conda.anaconda.org/conda-forge/linux-64/atk-1.0-2.38.0-h04ea711_2.conda#f730d54ba9cd543666d7220c9f7ed563 -https://conda.anaconda.org/conda-forge/linux-64/aws-c-http-0.11.0-hbe094ff_5.conda#c09f7ec7327b8bd52fc2d6eacaa18d3b -https://conda.anaconda.org/conda-forge/linux-64/brotli-1.2.0-h505cf86_3.conda#6ff307b78fbfb7dcafb7e25ff8bc9c10 +https://conda.anaconda.org/conda-forge/linux-64/aws-c-http-1.0.0-h392d1a3_1.conda#6701f6e712f9513e722ba57cde99f023 +https://conda.anaconda.org/conda-forge/linux-64/brotli-1.2.0-h505cf86_4.conda#746aa619a1bcaf4da541101179d7c60e https://conda.anaconda.org/conda-forge/linux-64/cyrus-sasl-2.1.28-hac629b4_1.conda#af491aae930edc096b58466c51c4126c -https://conda.anaconda.org/conda-forge/linux-64/dbus-1.16.2-h24cb091_1.conda#ce96f2f470d39bd96ce03945af92e280 +https://conda.anaconda.org/conda-forge/linux-64/dbus-1.16.2-he8c428d_2.conda#a6eb37b51be1a9a654dc3a8aca039b1a https://conda.anaconda.org/conda-forge/linux-64/fontconfig-2.18.3-h4db4eae_1.conda#922776b528a470ab5afa81fd42abfa1d https://conda.anaconda.org/conda-forge/linux-64/freetype-2.14.3-ha770c72_2.conda#2d0ea23b23603e07ca47f23f949f67b6 -https://conda.anaconda.org/conda-forge/linux-64/gdk-pixbuf-2.44.8-h68053f1_0.conda#10dab6a745f32ceeac6c9e00d9979797 -https://conda.anaconda.org/conda-forge/linux-64/glib-tools-2.88.3-h67ed8a3_2.conda#5c35ace78b7d630e77efcb37dacb62ae +https://conda.anaconda.org/conda-forge/linux-64/gdk-pixbuf-2.44.8-h68053f1_2.conda#d7bd6ccfc58eb6796bd6bba85878fc0e +https://conda.anaconda.org/conda-forge/linux-64/glib-tools-2.90.0-h3d503d9_0.conda#d4adaebf8a2c9040ada6549c52a6b8d8 https://conda.anaconda.org/conda-forge/linux-64/gts-0.7.6-h977cf35_4.conda#4d8df0b0db060d33c9a702ada998a8fe -https://conda.anaconda.org/conda-forge/linux-64/lcms2-2.19.1-h0c24ade_1.conda#8b3ce45e929cd8e8e5f4d18586b56d8b -https://conda.anaconda.org/conda-forge/linux-64/libblas-3.11.0-9_h4a7cf45_openblas.conda#f5c4b041925dea221dc4bad2e50569d9 +https://conda.anaconda.org/conda-forge/linux-64/lcms2-2.19.1-h9073bf1_3.conda#596464faa06e0b656b35416188ef9bba +https://conda.anaconda.org/conda-forge/linux-64/libblas-3.11.0-11_h4a7cf45_openblas.conda#b8cce1486f3c62f33291d6318c193d0d https://conda.anaconda.org/conda-forge/linux-64/libcups-2.3.3-h7a8fb5f_6.conda#49c553b47ff679a6a1e9fc80b9c5a2d4 -https://conda.anaconda.org/conda-forge/linux-64/libcurl-8.21.0-ha042cf0_5.conda#0ed167049513943d078a6c997df2b4e0 +https://conda.anaconda.org/conda-forge/linux-64/libcurl-8.22.0-ha042cf0_0.conda#e617deee7af8eb0c04f6296d12b54157 https://conda.anaconda.org/conda-forge/linux-64/libglx-1.7.0-ha4b6fd6_5.conda#614e10aae180f87b84f0d1ca8ceb7d9e https://conda.anaconda.org/conda-forge/linux-64/libkml-1.3.0-haa4a5bd_1023.conda#953b7cca897e21215302dbfe2af5cd0c https://conda.anaconda.org/conda-forge/linux-64/libxkbfile-1.2.0-hb03c661_0.conda#f330a1b107cb5861f26dbb72a26f33e8 -https://conda.anaconda.org/conda-forge/linux-64/libxml2-2.15.3-h49c6c72_1.conda#2d34cdb31014cbc8f1e9384c89ede0a5 -https://conda.anaconda.org/conda-forge/linux-64/openjpeg-2.5.4-h55fea9a_0.conda#11b3379b191f63139e29c0d19dee24cd -https://conda.anaconda.org/conda-forge/linux-64/python-3.13.15-hb101c97_101_cp313.conda#ad58731521aa42f9e70f3fc6c898e134 -https://conda.anaconda.org/conda-forge/linux-64/sqlite-3.53.4-h9ffa6c4_1.conda#d42b24574925bfa3167efb3571c905ad +https://conda.anaconda.org/conda-forge/linux-64/libxml2-2.15.4-h7df9aa5_1.conda#51e5162f95bd6840f6b2b3169125fff5 +https://conda.anaconda.org/conda-forge/linux-64/openjpeg-2.5.4-heb1ab33_2.conda#a3f3ac7a68d01c198e276dbb99e62032 +https://conda.anaconda.org/conda-forge/linux-64/python-3.13.16-hf47f18c_102_cp313.conda#d406e245f44390c9c3adcd86cda21899 +https://conda.anaconda.org/conda-forge/linux-64/sqlite-3.54.0-h9ffa6c4_1.conda#1060a8b69c6774ec85d81878eb299038 https://conda.anaconda.org/conda-forge/linux-64/xcb-util-image-0.4.0-hb711507_2.conda#a0901183f08b6c7107aab109733a3c91 https://conda.anaconda.org/conda-forge/linux-64/xkeyboard-config-2.48-h280c20c_0.conda#b233b41be0bf210989d57160ed39b394 https://conda.anaconda.org/conda-forge/linux-64/xorg-libxext-1.3.7-h7cc23a3_1.conda#e5b6b28536b81b3f4cb20db4668a4642 @@ -160,80 +162,81 @@ https://conda.anaconda.org/conda-forge/linux-64/xorg-libxrender-0.9.12-hb03c661_ https://conda.anaconda.org/conda-forge/noarch/aiohappyeyeballs-2.7.1-pyhd8ed1ab_0.conda#e51e09bf3b91c62b7461a9f94e92c2c9 https://conda.anaconda.org/conda-forge/noarch/alabaster-1.0.0-pyhd8ed1ab_1.conda#1fd9696649f65fd6611fcdb4ffec738a https://conda.anaconda.org/conda-forge/noarch/attrs-26.1.0-pyhcf101f3_0.conda#c6b0543676ecb1fb2d7643941fe375f2 -https://conda.anaconda.org/conda-forge/linux-64/aws-c-auth-0.10.4-h4610da3_2.conda#cefa84dbf94066e7a6902af288a2fedd +https://conda.anaconda.org/conda-forge/linux-64/aws-c-auth-1.0.0-h7648d90_2.conda#f4f169a23884de1bdb9c25ded8f2aec1 https://conda.anaconda.org/conda-forge/noarch/babel-2.18.0-pyhcf101f3_1.conda#f1976ce927373500cc19d3c0b2c85177 -https://conda.anaconda.org/conda-forge/linux-64/backports.zstd-1.7.0-py313h8f72f4d_0.conda#40454ef16bb96ae63f063b50c8b68603 -https://conda.anaconda.org/conda-forge/linux-64/brotli-python-1.2.0-py313h2fc2bef_3.conda#3bc8e37c6272e48b6eb6d15267b8a377 -https://conda.anaconda.org/conda-forge/linux-64/cairo-1.18.4-h3c89d7e_2.conda#e83331a940393006789fedddc3b492f3 +https://conda.anaconda.org/conda-forge/linux-64/backports.zstd-1.8.0-py313h8f72f4d_0.conda#b2b55be09675e65d54fb1709208bd250 +https://conda.anaconda.org/conda-forge/linux-64/brotli-python-1.2.0-py313h2fc2bef_4.conda#8beb0b96d9c701006ff5511a3a92329f +https://conda.anaconda.org/conda-forge/linux-64/cairo-1.18.6-h3c89d7e_0.conda#7920701f9abcb488411495742cf365a7 https://conda.anaconda.org/conda-forge/noarch/certifi-2026.7.22-pyhd8ed1ab_0.conda#37e13edbe3b48f1095a9d085ef9cd83b https://conda.anaconda.org/conda-forge/noarch/cfgv-3.5.0-pyhd8ed1ab_0.conda#381bd45fb7aa032691f3063aff47e3a1 -https://conda.anaconda.org/conda-forge/noarch/charset-normalizer-3.5.1-pyhd8ed1ab_0.conda#e0ac3accc64e23e40969d660e5f58ac8 -https://conda.anaconda.org/conda-forge/noarch/click-8.4.2-pyhc90fa1f_0.conda#2c4bd6aeb90bb157456841c3270a0d92 +https://conda.anaconda.org/conda-forge/noarch/click-8.5.0-pyh5ded981_0.conda#dea5b13a211bbf99876deb980b408a66 https://conda.anaconda.org/conda-forge/noarch/cloudpickle-3.1.2-pyhcf101f3_1.conda#61b8078a0905b12529abc622406cb62c https://conda.anaconda.org/conda-forge/noarch/colorama-0.4.6-pyhd8ed1ab_1.conda#962b9857ee8e7018c22f2776ffa0b2d7 https://conda.anaconda.org/conda-forge/noarch/colorcet-3.2.1-pyhd8ed1ab_0.conda#0b6419b9a27c540bd96e66e12a44c379 -https://conda.anaconda.org/conda-forge/linux-64/coverage-7.16.0-py313h2af15c8_0.conda#e53c02d7a82ec851a22628cdd241f182 -https://conda.anaconda.org/conda-forge/noarch/cpython-3.13.15-py313hd8ed1ab_101.conda#128ab71e26d605b5d93e058dc7d563cc +https://conda.anaconda.org/conda-forge/linux-64/coverage-7.16.2-py313h2af15c8_0.conda#97d81e36eb4c8fff723a1ff8a6c7d6fb +https://conda.anaconda.org/conda-forge/noarch/cpython-3.13.16-py313hd8ed1ab_102.conda#f06b4e55916af895989a48ea4c2732c1 https://conda.anaconda.org/conda-forge/noarch/cycler-0.12.1-pyhcf101f3_2.conda#4c2a8fef270f6c69591889b93f9f55c1 https://conda.anaconda.org/conda-forge/linux-64/cython-3.3.0-py313hb646302_0.conda#b7ddbe402c405d950a56bcb4ddb992a1 https://conda.anaconda.org/conda-forge/noarch/distlib-0.4.3-pyhcf101f3_0.conda#58638f77697c4f6726753eb8be34818b https://conda.anaconda.org/conda-forge/noarch/docstring_parser-0.18.0-pyhd8ed1ab_0.conda#b0c7c29a60c82d57c5b4c4c38f0642c8 https://conda.anaconda.org/conda-forge/noarch/docutils-0.21.2-pyhd8ed1ab_1.conda#24c1ca34138ee57de72a943237cde4cc https://conda.anaconda.org/conda-forge/noarch/execnet-2.1.2-pyhd8ed1ab_0.conda#a57b4be42619213a94f31d2c69c5dda7 -https://conda.anaconda.org/conda-forge/noarch/filelock-3.32.4-pyhd8ed1ab_0.conda#0171dc5d54fdbb3f6e55f285f805e0fe +https://conda.anaconda.org/conda-forge/noarch/filelock-4.1.1-pyhd8ed1ab_0.conda#fbff0cf6c30bdd9b38d0448fdadfe72d https://conda.anaconda.org/conda-forge/linux-64/frozenlist-1.8.0-py313h6b9daa2_0.conda#a51a4bc5d02deee4354781783c132b7f -https://conda.anaconda.org/conda-forge/noarch/fsspec-2026.7.0-pyhd8ed1ab_0.conda#b9176565976c773a0739bd83deaf06cc +https://conda.anaconda.org/conda-forge/noarch/fsspec-2026.9.0-pyhd8ed1ab_0.conda#42de5cd170647599a84f468a42fabb89 +https://conda.anaconda.org/conda-forge/linux-64/google-crc32c-1.8.0-py313h5cbdff7_2.conda#c70fc04cb55616e53e2a2a835ffd1c69 https://conda.anaconda.org/conda-forge/noarch/hpack-4.2.0-pyhd8ed1ab_0.conda#b395909221b9bd1df066e5930e18855b https://conda.anaconda.org/conda-forge/noarch/hyperframe-6.1.0-pyhd8ed1ab_0.conda#8e6923fc12f1fe8f8c4e5c9f343256ac -https://conda.anaconda.org/conda-forge/noarch/idna-3.19-pyhcf101f3_0.conda#a39ae05027e9b707742e41b30d296b75 +https://conda.anaconda.org/conda-forge/noarch/idna-3.20-pyh5ded981_0.conda#ee73be987f9de9ea5c2fdaeefa7b6491 https://conda.anaconda.org/conda-forge/noarch/imagesize-2.0.1-pyhd8ed1ab_0.conda#ebc56022e4ef7e74a829be94c53797ca -https://conda.anaconda.org/conda-forge/noarch/iniconfig-2.3.0-pyhd8ed1ab_0.conda#9614359868482abba1bd15ce465e3c42 +https://conda.anaconda.org/conda-forge/noarch/iniconfig-2.3.1-pyhd8ed1ab_0.conda#c0a803d97dd09da7e2c8b0fdfaa5739b https://conda.anaconda.org/conda-forge/noarch/iris-sample-data-2.5.2-pyhd8ed1ab_0.conda#895f6625dd8a246fece9279fcc12c1de https://conda.anaconda.org/conda-forge/linux-64/jsonschema-rs-0.37.4-py313hdeb11d6_0.conda#e736b02c4a0905ad7636d798d48f3bed -https://conda.anaconda.org/conda-forge/linux-64/kiwisolver-1.5.1-py313h2551ef2_0.conda#cb89d589c15417938ec7f4fe67bc4088 +https://conda.anaconda.org/conda-forge/linux-64/kiwisolver-1.5.1-py313h2551ef2_3.conda#45c8698dd577ba504513001219cfd0fd https://conda.anaconda.org/conda-forge/linux-64/libarchive-3.8.9-gpl_h3152399_101.conda#680520cff78a974d564cd99cdf5ec0b5 -https://conda.anaconda.org/conda-forge/linux-64/libcblas-3.11.0-9_h0358290_openblas.conda#092c5649f3727af436ab0f67f48c3811 +https://conda.anaconda.org/conda-forge/linux-64/libcblas-3.11.0-11_h0358290_openblas.conda#73a248c30811075059b5aeda766c2624 https://conda.anaconda.org/conda-forge/linux-64/libgd-2.3.3-h5fbf134_12.conda#88c1c66987cd52a712eea89c27104be6 https://conda.anaconda.org/conda-forge/linux-64/libgl-1.7.0-ha4b6fd6_5.conda#81141db127a106eb5a91df69a29c9918 https://conda.anaconda.org/conda-forge/linux-64/libglx-devel-1.7.0-ha4b6fd6_5.conda#fb00b4fb800f2d431303a5c1625ef9d0 -https://conda.anaconda.org/conda-forge/linux-64/libhwloc-2.13.0-default_he001693_1000.conda#c197985b58bc813d26b42881f0021c82 -https://conda.anaconda.org/conda-forge/linux-64/liblapack-3.11.0-9_h47877c9_openblas.conda#e51473c2b7e1f9cb61daafccfd912abf +https://conda.anaconda.org/conda-forge/linux-64/libhwloc-2.15.0-default_hd7c4d0c_1001.conda#66c755b3bd9ec82308fa6ec7674f473f +https://conda.anaconda.org/conda-forge/linux-64/liblapack-3.11.0-11_h47877c9_openblas.conda#5622a13917855ce4b953b9e655cf9262 https://conda.anaconda.org/conda-forge/linux-64/libxkbcommon-1.13.2-h51789e4_1.conda#f888787e0eab7a8076a67d197e155ffc -https://conda.anaconda.org/conda-forge/linux-64/libxml2-devel-2.15.3-h49c6c72_1.conda#f986d9c9de80813d076d235f753889ff +https://conda.anaconda.org/conda-forge/linux-64/libxml2-devel-2.15.4-h7df9aa5_1.conda#c6f14f767cbfe20fa03a324beb6cd7eb https://conda.anaconda.org/conda-forge/noarch/locket-1.0.0-pyhd8ed1ab_0.tar.bz2#91e27ef3d05cc772ce627e51cff111c4 -https://conda.anaconda.org/conda-forge/linux-64/markupsafe-3.0.3-py313h3dea7bd_1.conda#aeb9b9da79fd0258b3db091d1fefcd71 +https://conda.anaconda.org/conda-forge/linux-64/markupsafe-3.0.4-py313h2af15c8_1.conda#d15255d7f81d9188edecde1e024a1649 https://conda.anaconda.org/conda-forge/noarch/mdurl-0.1.2-pyhd8ed1ab_1.conda#592132998493b3ff25fd7479396e8351 -https://conda.anaconda.org/conda-forge/linux-64/msgpack-python-1.2.2-py313h2551ef2_0.conda#d987c68afd77b8584d4a98a5d1f1801b +https://conda.anaconda.org/conda-forge/linux-64/msgpack-python-1.2.3-py313h2551ef2_0.conda#276a99d974564361ffeef499749393e0 +https://conda.anaconda.org/conda-forge/linux-64/msgspec-0.22.0-py313h995f894_0.conda#fe6b5c45d2e8fff6cb46cbfa5cd8cff2 https://conda.anaconda.org/conda-forge/linux-64/multidict-6.7.1-py313h3dea7bd_0.conda#4f3e7bf5a9fc60a7d39047ba9e84c84c https://conda.anaconda.org/conda-forge/noarch/munkres-1.1.4-pyhd8ed1ab_1.conda#37293a85a0f4f77bbd9cf7aaefc62609 https://conda.anaconda.org/conda-forge/linux-64/openldap-2.6.13-hbde042b_0.conda#680608784722880fbfe1745067570b00 https://conda.anaconda.org/conda-forge/noarch/packaging-26.3-pyhc364b38_0.conda#936687ed80f295a1f5dbcf8bd34c252c -https://conda.anaconda.org/conda-forge/linux-64/pillow-12.3.0-py313h80991f8_0.conda#730e462e7e141813192b606cf07ebdd0 +https://conda.anaconda.org/conda-forge/linux-64/pillow-12.3.0-py313h3b26573_4.conda#6739325f3022c709753d26fff83ae20e https://conda.anaconda.org/conda-forge/noarch/pip-26.2.1-pyh145f28c_0.conda#573cb3ed111004230ed3de650653cd4d -https://conda.anaconda.org/conda-forge/noarch/platformdirs-4.11.5-pyhcf101f3_0.conda#152ec36401f710145a7db03475594410 +https://conda.anaconda.org/conda-forge/noarch/platformdirs-4.13.0-pyh5ded981_0.conda#35cc2a176fc6f886d6efa41ecb5f1de0 https://conda.anaconda.org/conda-forge/noarch/pluggy-1.6.0-pyhf9edf01_1.conda#d7585b6550ad04c8c5e21097ada2888e https://conda.anaconda.org/conda-forge/linux-64/proj-9.8.1-he0df7b0_0.conda#b23619e5e9009eaa070ead0342034027 -https://conda.anaconda.org/conda-forge/linux-64/propcache-0.5.2-py313h3dea7bd_0.conda#9f8899a9482c1737bf402d8218780df1 -https://conda.anaconda.org/conda-forge/linux-64/psutil-7.2.2-py313hd42f317_1.conda#04b0a2e271f49b0463be6d57ae4c2b56 -https://conda.anaconda.org/conda-forge/noarch/pycparser-3.0-pyhcf101f3_0.conda#9c5491066224083c41b6d5635ed7107b +https://conda.anaconda.org/conda-forge/linux-64/propcache-0.5.4-py313h6cd9899_0.conda#c38cae05bdbe3d317baf350ebd9de3bb +https://conda.anaconda.org/conda-forge/linux-64/psutil-7.2.2-py313hd42f317_3.conda#0bc97f7c2f4a6993798c004fd557e3a1 +https://conda.anaconda.org/conda-forge/noarch/pycparser-3.11-pyh5ded981_0.conda#3cd164745122ab557cc30879390089e8 https://conda.anaconda.org/conda-forge/noarch/pygments-2.21.0-pyhcf101f3_0.conda#2882dee445dfa45b0c5afce3ffb7730a -https://conda.anaconda.org/conda-forge/noarch/pyparsing-3.3.2-pyhcf101f3_0.conda#3687cc0b82a8b4c17e1f0eb7e47163d5 +https://conda.anaconda.org/conda-forge/noarch/pyparsing-3.3.3-pyh5ded981_0.conda#8bf43ae1312e9b966063459e04c20fa7 https://conda.anaconda.org/conda-forge/noarch/pyshp-3.1.6-pyhcf101f3_0.conda#6778d276eb7b4230d622bdd2a732c75e https://conda.anaconda.org/conda-forge/noarch/pysocks-1.7.1-pyha55dd90_7.conda#461219d1a5bd61342293efa2c0c90eac -https://conda.anaconda.org/conda-forge/linux-64/python-xxhash-3.7.0-py313heab5758_0.conda#3e14deae910b2de18153b78a372a4a0e +https://conda.anaconda.org/conda-forge/linux-64/python-xxhash-3.7.0-py313h554d1e1_1.conda#b137c626d5fd1febe9186636f38e1e41 https://conda.anaconda.org/conda-forge/linux-64/pyyaml-6.0.3-py313h3dea7bd_1.conda#f256753e840c3cd3766488c9437a8f8b -https://conda.anaconda.org/conda-forge/noarch/scooby-0.11.2-pyhd8ed1ab_0.conda#c5b13fecbbd3984f12a70599973c6551 +https://conda.anaconda.org/conda-forge/noarch/scooby-0.12.0-pyhd8ed1ab_0.conda#820b402968e73e94ae47ff322c3206a5 https://conda.anaconda.org/conda-forge/noarch/setuptools-84.0.0-pyh332efcf_0.conda#62ac906f1cd582c6c264c95625cb9d6f https://conda.anaconda.org/conda-forge/noarch/six-1.17.0-pyhe01879c_1.conda#3339e3b65d58accf4ca4fb8748ab16b3 https://conda.anaconda.org/conda-forge/noarch/snowballstemmer-3.1.1-pyhd8ed1ab_0.conda#46b6abe31482f6bca064b965696ae807 https://conda.anaconda.org/conda-forge/noarch/sortedcontainers-2.4.0-pyhd8ed1ab_1.conda#0401a17ae845fa72c7210e206ec5647d -https://conda.anaconda.org/conda-forge/noarch/soupsieve-2.9.2-pyhd8ed1ab_0.conda#ba33e6c8a46ee373fdf6dd8665212778 +https://conda.anaconda.org/conda-forge/noarch/soupsieve-3.0.3-pyhd8ed1ab_0.conda#bd2ea090567b9d0a53240bd1367c87ad https://conda.anaconda.org/conda-forge/noarch/sphinxcontrib-jsmath-1.0.1-pyhd8ed1ab_1.conda#fa839b5ff59e192f411ccc7dae6588bb https://conda.anaconda.org/conda-forge/noarch/tabulate-0.10.0-pyhcf101f3_0.conda#3b887b7b3468b0f494b4fad40178b043 https://conda.anaconda.org/conda-forge/noarch/tblib-3.2.2-pyhcf101f3_0.conda#f88bb644823094f436792f80fba3207e -https://conda.anaconda.org/conda-forge/noarch/tomli-2.4.1-pyhcf101f3_0.conda#b5325cf06a000c5b14970462ff5e4d58 -https://conda.anaconda.org/conda-forge/noarch/toolz-1.1.0-pyhd8ed1ab_1.conda#c07a6153f8306e45794774cf9b13bd32 -https://conda.anaconda.org/conda-forge/linux-64/tornado-6.5.8-py313h07c4f96_0.conda#0c8fd5b50c36b6da9b5c57189f65a869 +https://conda.anaconda.org/conda-forge/noarch/tomli-2.5.0-pyh5ded981_0.conda#36d6a162326c52d6339c8b606f43932d +https://conda.anaconda.org/conda-forge/noarch/toolz-1.2.0-pyh5ded981_0.conda#17f8db9088267a3cb38d56cc22d268ae +https://conda.anaconda.org/conda-forge/linux-64/tornado-6.5.10-py313h995f894_0.conda#94985bd39e52fedc2270adb00e4779e6 https://conda.anaconda.org/conda-forge/noarch/typing_extensions-4.16.0-pyhcf101f3_0.conda#c70ad746c22219b9700931707482992c https://conda.anaconda.org/conda-forge/linux-64/xcb-util-cursor-0.1.6-hb03c661_0.conda#4d1fc190b99912ed557a8236e958c559 https://conda.anaconda.org/conda-forge/linux-64/xorg-libxcomposite-0.4.7-hb03c661_0.conda#f2ba4192d38b6cef2bb2c25029071d90 @@ -244,122 +247,126 @@ https://conda.anaconda.org/conda-forge/linux-64/xorg-libxinerama-1.1.6-hecca717_ https://conda.anaconda.org/conda-forge/linux-64/xorg-libxrandr-1.5.5-h7cc23a3_1.conda#798a8c9d171859a022e1cb89e7d0eb10 https://conda.anaconda.org/conda-forge/linux-64/xorg-libxxf86vm-1.1.7-hb03c661_0.conda#665d152b9c6e78da404086088077c844 https://conda.anaconda.org/conda-forge/noarch/zict-3.0.0-pyhd8ed1ab_1.conda#e52c2ef711ccf31bb7f70ca87d144b9e -https://conda.anaconda.org/conda-forge/noarch/zipp-4.1.0-pyhcf101f3_0.conda#ba3dcdc8584155c97c648ae9c044b7a3 +https://conda.anaconda.org/conda-forge/noarch/zipp-4.1.1-pyh5ded981_0.conda#6cad4d2cec9b7b64538c225ac91d2e8a https://conda.anaconda.org/conda-forge/noarch/accessible-pygments-0.0.5-pyhd8ed1ab_1.conda#74ac5069774cdbc53910ec4d631a3999 -https://conda.anaconda.org/conda-forge/noarch/affine-3.0.0-pyhd8ed1ab_0.conda#71dd867a02739013ae65894ca32d67b7 +https://conda.anaconda.org/conda-forge/noarch/affine-3.0.1-pyhecae5ae_0.conda#aeb404cab9e719a49264ab40928f80c8 https://conda.anaconda.org/conda-forge/noarch/aiosignal-1.4.0-pyhd8ed1ab_0.conda#421a865222cd0c9d83ff08bc78bf3a61 -https://conda.anaconda.org/conda-forge/linux-64/aws-c-s3-0.13.1-h7508075_1.conda#2daf54d3eef87b757f68d7597534dd2f -https://conda.anaconda.org/conda-forge/linux-64/cffi-2.1.1-py313ha52865f_2.conda#3c23b9907fd858c635a29ec77cf43301 +https://conda.anaconda.org/conda-forge/linux-64/aws-c-s3-1.0.0-hfe03eda_1.conda#7a8a507033f2e478670a617bb4ffb0b8 +https://conda.anaconda.org/conda-forge/linux-64/cffi-2.1.1-py313ha52865f_3.conda#2b17c17965f71b59158030dce83832e1 https://conda.anaconda.org/conda-forge/noarch/click-default-group-1.2.4-pyhd8ed1ab_1.conda#7cd83dd6831b61ad9624a694e4afd7dc https://conda.anaconda.org/conda-forge/noarch/click-plugins-1.1.1.2-pyhd8ed1ab_0.conda#e9b05deb91c013e5224672a4ba9cf8d1 https://conda.anaconda.org/conda-forge/noarch/cligj-0.7.2-pyhd8ed1ab_2.conda#55c7804f428719241a90b152016085a1 -https://conda.anaconda.org/conda-forge/linux-64/cytoolz-1.1.0-py313h07c4f96_2.conda#7e7e3c5a8a28c6b8eb430183e0554adf +https://conda.anaconda.org/conda-forge/linux-64/cytoolz-1.2.0-py313hd42f317_0.conda#4667e0f1ee7d0f9dba663c070323f9a1 +https://conda.anaconda.org/conda-forge/noarch/donfig-0.8.1.post1-pyhd8ed1ab_1.conda#c56a7fa5597ad78b62e1f5d21f7f8b8f https://conda.anaconda.org/conda-forge/noarch/exceptiongroup-1.3.1-pyhd8ed1ab_0.conda#8e662bd460bda79b1ea39194e3c4c9ab -https://conda.anaconda.org/conda-forge/linux-64/fonttools-4.63.0-py313h3dea7bd_0.conda#ae83c999b4cfc4c171ce88b99c8b43cc +https://conda.anaconda.org/conda-forge/linux-64/fonttools-4.66.1-py313h6cd9899_0.conda#bc5e99327ffd810f4e3574315ed2d954 https://conda.anaconda.org/conda-forge/linux-64/gl2ps-1.4.2-h36e74d4_2.conda#787c780ff43f9f79d78d01e476b81a7c https://conda.anaconda.org/conda-forge/linux-64/glew-2.3.0-h71661d4_0.conda#7eefecda1c71c380bfc406d16e78bbee https://conda.anaconda.org/conda-forge/noarch/h2-4.4.1-pyhcf101f3_0.conda#e652ac7756069c456d0da2a922cd7df5 https://conda.anaconda.org/conda-forge/noarch/importlib-metadata-9.0.1-pyhcf101f3_0.conda#77ec68e0aa61c3fff9ecbf840f93d64e -https://conda.anaconda.org/conda-forge/noarch/importlib_resources-7.1.0-pyhd8ed1ab_0.conda#0ba6225c279baf7ea9473a62ea0ec9ae https://conda.anaconda.org/conda-forge/noarch/jinja2-3.1.6-pyhcf101f3_1.conda#04558c96691bed63104678757beb4f8d -https://conda.anaconda.org/conda-forge/noarch/lazy-loader-0.5-pyhd8ed1ab_0.conda#75932da6f03a6bef32b70a51e991f6eb +https://conda.anaconda.org/conda-forge/noarch/lazy-loader-0.6-pyhd8ed1ab_0.conda#4ed11ec126e6da490282edfd0673fea0 https://conda.anaconda.org/conda-forge/linux-64/libgl-devel-1.7.0-ha4b6fd6_5.conda#f5f7fc038c611a25b22758c0a40d25c9 -https://conda.anaconda.org/conda-forge/linux-64/libharfbuzz-14.4.0-h23af247_0.conda#4463210c2a16ff5d3bf7f502af23f1f4 -https://conda.anaconda.org/conda-forge/linux-64/libpq-18.6-h9d76c99_0.conda#72e019c04ed02a179da38c56965802d1 -https://conda.anaconda.org/conda-forge/linux-64/libspatialite-5.1.0-gpl_hab3fe16_120.conda#5947bdda89ced07ab0d8a5d6503082c3 +https://conda.anaconda.org/conda-forge/linux-64/libharfbuzz-14.6.0-h23af247_2.conda#6a93147cab011f36d1659745c3f579e2 +https://conda.anaconda.org/conda-forge/linux-64/libpq-18.6-h3883810_3.conda#cc998047391971a452fed680158329f3 +https://conda.anaconda.org/conda-forge/linux-64/libspatialite-5.1.0-gpl_h03b4e02_121.conda#affb2721df7014b4abcd75cc539c6cef https://conda.anaconda.org/conda-forge/linux-64/libvulkan-loader-1.4.357.0-h0e34353_2.conda#c5f5f159b66332152197893427071695 https://conda.anaconda.org/conda-forge/noarch/markdown-it-py-4.2.0-pyhd8ed1ab_0.conda#6d03368f2b2b0a5fb6839df53b2eb5e0 https://conda.anaconda.org/conda-forge/noarch/nodeenv-1.10.0-pyhd8ed1ab_0.conda#eb52d14a901e23c39e9e7b4a1a5c015f -https://conda.anaconda.org/conda-forge/linux-64/numpy-2.5.2-py313hf6604e3_0.conda#29a27a75bfa2f40221f1557dd5e14e65 +https://conda.anaconda.org/conda-forge/linux-64/numpy-2.5.4-py313hb5f73ae_0.conda#1ddee4b1dc2f784b4137a43789be3816 https://conda.anaconda.org/conda-forge/noarch/partd-1.4.2-pyhd8ed1ab_0.conda#0badf9c54e24cecfb0ad2f99d680c163 -https://conda.anaconda.org/conda-forge/noarch/pbr-7.1.1-pyhd8ed1ab_0.conda#51b000c0a6d35e244019f9fc8ab1c9f6 -https://conda.anaconda.org/conda-forge/linux-64/pyproj-3.7.2-py313hd11bf8e_5.conda#bf6564747d6b5c3a8462ded5b8e7012c +https://conda.anaconda.org/conda-forge/noarch/pbr-7.1.3-pyhd8ed1ab_0.conda#66415d5d2894f1d10928005a264c2717 +https://conda.anaconda.org/conda-forge/linux-64/pyproj-3.8.0-py313hd11bf8e_1.conda#4fcc7972635bb83fcc9376559c338e4f https://conda.anaconda.org/conda-forge/noarch/python-dateutil-2.9.0.post0-pyhe01879c_2.conda#5b8d21249ff20967101ffa321cab24e8 -https://conda.anaconda.org/conda-forge/noarch/python-discovery-1.6.0-pyhcf101f3_0.conda#e87738e32eaf5dfa98a5d49f611d6312 -https://conda.anaconda.org/conda-forge/noarch/python-gil-3.13.15-h4df99d1_101.conda#c5e565a020c31fd7aba0a87dc2fc29d0 -https://conda.anaconda.org/conda-forge/linux-64/tbb-2023.0.0-hab88423_2.conda#7073b15f9364ebc118998601ac6ca6a6 +https://conda.anaconda.org/conda-forge/noarch/python-discovery-1.6.2-pyh5ded981_0.conda#17e6c8b2fc2d021a217d317fe4f89fc4 +https://conda.anaconda.org/conda-forge/noarch/python-gil-3.13.16-h4df99d1_102.conda#6577480ed084a42b5f000087df16b894 +https://conda.anaconda.org/conda-forge/linux-64/tbb-2023.1.0-haa8c3ee_2.conda#f805909f478c179e230baf7bd5878e26 https://conda.anaconda.org/conda-forge/noarch/typing-extensions-4.16.0-h69aa097_0.conda#c680b5747e8c4c8f23dca0bb7042a8fc -https://conda.anaconda.org/conda-forge/noarch/vcs_versioning-2.3.1-pyhcf101f3_0.conda#6043a03a3733302373b30e16ecc408b7 +https://conda.anaconda.org/conda-forge/noarch/vcs_versioning-2.6.0-pyh5ded981_0.conda#958f3e46d5025cdd9aa9c2417adceec8 https://conda.anaconda.org/conda-forge/linux-64/xorg-libxtst-1.2.5-h7cc23a3_4.conda#752a5ac9322e0fbbc24146c1ce3ae44e -https://conda.anaconda.org/conda-forge/linux-64/yarl-1.24.5-py313h3dea7bd_0.conda#abf83ec0fc49a445929114ed51133d7b +https://conda.anaconda.org/conda-forge/linux-64/yarl-1.25.1-py313h6cd9899_0.conda#3e0ea10bd102b49c5e5b9d61f450673c https://conda.anaconda.org/conda-forge/noarch/_python_abi3_support-1.0-hd8ed1ab_3.conda#3845f3d75991bae0fb90884662f4327c -https://conda.anaconda.org/conda-forge/linux-64/aiohttp-3.14.3-py313hd6074c6_0.conda#9a084d4234ccd2e19fee2579414d773b +https://conda.anaconda.org/conda-forge/linux-64/aiohttp-3.14.5-py313h2af15c8_0.conda#76e62dcb31698f72375b91a2c0882ae1 https://conda.anaconda.org/conda-forge/noarch/asv_runner-0.3.1-pyhd8ed1ab_0.conda#960a0f64467ce624941ad7c67b80a2e2 https://conda.anaconda.org/conda-forge/linux-64/at-spi2-core-2.40.3-h0630a04_0.tar.bz2#8cb2fc4cd6cc63f1369cfa318f581cc3 https://conda.anaconda.org/conda-forge/noarch/beautifulsoup4-4.15.0-pyha770c72_0.conda#3b261da3fe9b4168738712832410b022 -https://conda.anaconda.org/conda-forge/linux-64/cftime-1.6.5-py313h29aa505_1.conda#c63d5f9d63fe2f48b0ad75005fcae7ba +https://conda.anaconda.org/conda-forge/linux-64/cftime-1.6.5-py313h12b3e8d_2.conda#e86d6e3b6b59cb813986534a7426e0db https://conda.anaconda.org/conda-forge/noarch/colorspacious-1.1.2-pyhecae5ae_1.conda#04151bb8e351c6209caad045e4b1f4bd -https://conda.anaconda.org/conda-forge/linux-64/contourpy-1.3.3-py313hc8edb43_4.conda#33639459bc29437315d4bff9ed5bc7a7 +https://conda.anaconda.org/conda-forge/linux-64/contourpy-1.4.0-py313h2551ef2_1.conda#1883f23b947ffaa13f9511ea817fc4e6 https://conda.anaconda.org/conda-forge/noarch/dask-core-2026.8.0-pyhc364b38_0.conda#2ccdf45d2372e15c7edcb7a72d573e76 -https://conda.anaconda.org/conda-forge/linux-64/hdf5-2.2.0-nompi_hc529623_100.conda#336f7fb9af0c49cc246e2cff0992c3c4 -https://conda.anaconda.org/conda-forge/noarch/importlib-resources-7.1.0-pyhd8ed1ab_0.conda#e3bffa82b874f8b9a2631bddb3869529 -https://conda.anaconda.org/conda-forge/noarch/lazy_loader-0.5-pyhd8ed1ab_0.conda#4c8327180586e7b1cd8b6815fc8827f1 +https://conda.anaconda.org/conda-forge/linux-64/hdf5-2.2.0-nompi_hdbb200c_101.conda#66762e6067bf7d52586ff21c78c970db +https://conda.anaconda.org/conda-forge/noarch/lazy_loader-0.6-pyhd8ed1ab_0.conda#baa2be54811ef381364ae730a4613d0f https://conda.anaconda.org/conda-forge/linux-64/libegl-devel-1.7.0-ha4b6fd6_5.conda#577800fc8c9d8b7d9727246bbfde704e -https://conda.anaconda.org/conda-forge/linux-64/libgdal-core-3.13.3-h5fdb907_0.conda#c0723084e1315a5247d5a033d9eb5522 -https://conda.anaconda.org/conda-forge/linux-64/libharfbuzz-devel-14.4.0-h23af247_0.conda#26e37e05324d7bb723350d50b33057fc +https://conda.anaconda.org/conda-forge/linux-64/libgdal-core-3.13.3-h31aba09_1.conda#44c6780e3082468d8676aebdcd5068a6 +https://conda.anaconda.org/conda-forge/linux-64/libharfbuzz-devel-14.6.0-h23af247_2.conda#3cf459e930e6ea2e57bbf04be7fcb7c6 https://conda.anaconda.org/conda-forge/linux-64/libraqm-0.11.0-h6406941_0.conda#3ac89a48d224409739dbf6200e524373 -https://conda.anaconda.org/conda-forge/linux-64/mo_pack-0.3.1-py313h29aa505_2.conda#ad53894d278895bf15c8fc324727d224 -https://conda.anaconda.org/conda-forge/linux-64/pandas-3.0.5-py313hbfd7664_1.conda#a78866632a871f274cd2015bccdd2ca0 -https://conda.anaconda.org/conda-forge/linux-64/pango-1.58.2-hda50119_0.conda#6a2822aaf9a34ac3708904a47ff3dd7e -https://conda.anaconda.org/conda-forge/linux-64/pykdtree-1.4.3-py313h29aa505_2.conda#e3a598d20bf2fa7b03a771e9e4471be9 +https://conda.anaconda.org/conda-forge/linux-64/mo_pack-0.3.1-py313h12b3e8d_3.conda#5a7f561522fe020e1ff4b31522ec37b0 +https://conda.anaconda.org/conda-forge/linux-64/numcodecs-0.17.0-py313h12b3e8d_1.conda#a9a5095b73124de28d231040ea871de1 +https://conda.anaconda.org/conda-forge/linux-64/pandas-3.0.6-np2py313h4900a6c_0.conda#871596bf7cdadf8ccff03437e9b92173 +https://conda.anaconda.org/conda-forge/linux-64/pango-1.58.2-h7bb47b9_1.conda#0b889b1c6b472d215d6332c6b195589c +https://conda.anaconda.org/conda-forge/linux-64/pykdtree-1.4.3-py313h12b3e8d_3.conda#3b2d1978d69a506ef1487feddd12f7b9 https://conda.anaconda.org/conda-forge/noarch/pytest-9.0.3-pyhc364b38_1.conda#6a991452eadf2771952f39d43615bb3e -https://conda.anaconda.org/conda-forge/linux-64/pywavelets-1.9.0-py313h29aa505_2.conda#60f5d1c039da10fe89a530cc93ea50ac +https://conda.anaconda.org/conda-forge/noarch/pyvista-validation-0.2.2-pyh5ded981_0.conda#e06843d2fd3ed2b0472c7f11b52e7f3e +https://conda.anaconda.org/conda-forge/linux-64/pywavelets-1.10.0-py313hd42f317_1.conda#e8e0591c0fe1590fd521a16c74a60870 https://conda.anaconda.org/conda-forge/noarch/rich-15.0.0-pyhcf101f3_0.conda#0242025a3c804966bf71aa04eee82f66 -https://conda.anaconda.org/conda-forge/linux-64/scipy-1.18.0-py313h4b8bb8b_0.conda#a32f88181ff9a3451c71be1f17a7c799 -https://conda.anaconda.org/conda-forge/noarch/setuptools-scm-10.2.1-pyhcf101f3_0.conda#eefa3bc61c9224107d3a9afeb37552a9 -https://conda.anaconda.org/conda-forge/linux-64/shapely-2.1.2-py313had47c43_2.conda#6e550dd748e9ac9b2925411684e076a1 +https://conda.anaconda.org/conda-forge/linux-64/scipy-1.18.1-py313he2a5f1a_1.conda#70c6a7ea500089cf1543378c69f94fac +https://conda.anaconda.org/conda-forge/noarch/setuptools-scm-10.3.4-pyh5ded981_0.conda#437557778d99cc6f311f3c0ce023ad85 +https://conda.anaconda.org/conda-forge/linux-64/shapely-2.2.0-np2py313ha973fa7_2.conda#9f38bb516b77ea8c97348d514b2abdbe https://conda.anaconda.org/conda-forge/noarch/snuggs-1.4.7-pyhd8ed1ab_2.conda#9aa358575bbd4be126eaa5e0039f835c https://conda.anaconda.org/conda-forge/noarch/sphinxcontrib-apidoc-0.3.0-py_1.tar.bz2#855b087883443abb10f5faf6eef40860 +https://conda.anaconda.org/conda-forge/noarch/towncrier-26.9.0-pyhd8ed1ab_0.conda#d4167f473b71694bbca708347abee99e https://conda.anaconda.org/conda-forge/linux-64/ukkonen-1.1.0-py313h7037e92_0.conda#cb423e0853b3dde2b3738db4dedf5ba2 -https://conda.anaconda.org/conda-forge/noarch/urllib3-2.7.0-pyhd8ed1ab_0.conda#cbb88288f74dbe6ada1c6c7d0a97223e -https://conda.anaconda.org/conda-forge/noarch/virtualenv-21.7.7-pyhcf101f3_0.conda#768a861e1491dc8ee2a93fa8b7ee4317 +https://conda.anaconda.org/conda-forge/noarch/urllib3-2.8.0-pyhd8ed1ab_0.conda#407a3d3778570bae8e8fcf554e803e43 +https://conda.anaconda.org/conda-forge/noarch/virtualenv-21.14.7-pyh5ded981_0.conda#1635534aa129eb9ab6ce5e01929d86ff https://conda.anaconda.org/conda-forge/linux-64/at-spi2-atk-2.38.0-h0630a04_3.tar.bz2#6b889f174df1e0f816276ae69281af4d -https://conda.anaconda.org/conda-forge/linux-64/cf-units-3.3.1-py313h29aa505_0.conda#3942b6a86fe92d0888b3373f2c1e1676 +https://conda.anaconda.org/conda-forge/linux-64/cf-units-3.3.1-np2hc8c11d4_2.conda#9b777f085506bff796833be0ae292db1 +https://conda.anaconda.org/conda-forge/linux-64/charset-normalizer-3.5.2-h1114479_0.conda#78d5c6112c9320a1a797935cecfec7c2 https://conda.anaconda.org/conda-forge/noarch/distributed-2026.8.0-pyhc364b38_0.conda#f382c4eed5376ee372a6e67a0f6cd4d9 https://conda.anaconda.org/conda-forge/linux-64/epoxy-1.5.10-hb03c661_2.conda#057083b06ccf1c2778344b6dabace38b -https://conda.anaconda.org/conda-forge/linux-64/harfbuzz-14.4.0-ha770c72_0.conda#c2d4a4fe3216b26ef30e91d917c1b718 -https://conda.anaconda.org/conda-forge/noarch/identify-2.6.19-pyhd8ed1ab_0.conda#84a3233b709a289a4ddd7a2fd27dd988 +https://conda.anaconda.org/conda-forge/linux-64/harfbuzz-14.6.0-ha770c72_2.conda#0709eacc38906e8a237d9ef172af6688 +https://conda.anaconda.org/conda-forge/noarch/identify-2.6.20-pyhd8ed1ab_0.conda#be9ea4f030b4b2bc121b2a548b066c0b https://conda.anaconda.org/conda-forge/noarch/imagehash-4.3.2-pyhd8ed1ab_0.conda#efbc812363856906a80344b496389d2e https://conda.anaconda.org/conda-forge/linux-64/libnetcdf-4.10.1-nompi_he3e3c8e_201.conda#912aefa694a66897afeb7a2330b17d9f -https://conda.anaconda.org/conda-forge/linux-64/matplotlib-base-3.11.1-py313h134213a_2.conda#2c99937357c9476a1f5dc5ecd08d0c75 -https://conda.anaconda.org/conda-forge/linux-64/minijinja-2.22.0-py310hd8a072f_0.conda#ad9070e3f9613ecdd611e93bbb49c7e8 +https://conda.anaconda.org/conda-forge/linux-64/librsvg-2.62.4-ha427ee3_0.conda#f9bda2728debbde4c641bd1af103c623 +https://conda.anaconda.org/conda-forge/linux-64/matplotlib-base-3.11.2-py313he85fb2c_1.conda#85e795dd9e44b84961fafb4e7dd589aa https://conda.anaconda.org/conda-forge/noarch/pytest-cov-7.1.0-pyhcf101f3_0.conda#67d1790eefa81ed305b89d8e314c7923 -https://conda.anaconda.org/conda-forge/noarch/pytest-mock-3.15.1-pyhd8ed1ab_0.conda#0511afbe860b1a653125d77c719ece53 +https://conda.anaconda.org/conda-forge/noarch/pytest-mock-3.16.0-pyhd8ed1ab_0.conda#b038267c191ebae9d6374caf30cc2cca https://conda.anaconda.org/conda-forge/noarch/pytest-xdist-3.8.0-pyhd8ed1ab_0.conda#8375cfbda7c57fbceeda18229be10417 -https://conda.anaconda.org/conda-forge/linux-64/python-stratify-0.4.0-py313h07c4f96_3.conda#b7810803a3481e22968022a94107ed93 -https://conda.anaconda.org/conda-forge/linux-64/qt6-main-6.11.2-pl5321h9df5c37_0.conda#2ff98660281d1c6bfebda5bffa52cc89 -https://conda.anaconda.org/conda-forge/linux-64/rasterio-1.5.1-py313ha8e1118_1.conda#5d82c0250bf4a2cdb02ebf9b7d3cf5f9 -https://conda.anaconda.org/conda-forge/noarch/requests-2.34.2-pyhcf101f3_0.conda#4a85203c1d80c1059086ae860836ffb9 -https://conda.anaconda.org/conda-forge/noarch/rich-rst-2.1.0-pyhd8ed1ab_0.conda#0633059139afdc156bc3feaf1ff9e734 -https://conda.anaconda.org/conda-forge/noarch/towncrier-25.8.0-pyhd8ed1ab_0.conda#3e0e8e44292bdac62f7bcbf0450b5cc7 +https://conda.anaconda.org/conda-forge/noarch/python-abi3-3.13-h9bebd56_3.conda#ba55acfc336780a55ee658d8608aa2d4 +https://conda.anaconda.org/conda-forge/linux-64/python-stratify-0.4.0-py313h995f894_4.conda#7082bb91cc3fe449f0ba89bff1fc8ea4 +https://conda.anaconda.org/conda-forge/linux-64/qt6-main-6.12.0-pl5321hcf69127_1.conda#db3e81b8f192e5c9ee8e1e8c215c40b0 +https://conda.anaconda.org/conda-forge/linux-64/rasterio-1.5.2-py313h548c9e1_1.conda#697e246bb0b7eab5d417561af1dd291b +https://conda.anaconda.org/conda-forge/noarch/rich-rst-2.2.0-pyhd8ed1ab_0.conda#d91900e108041fa4d06ff329970027a1 https://conda.anaconda.org/conda-forge/linux-64/viskores-1.1.1-cpu_ha53dc59_1.conda#228549cd2f01bb53664bd2b5b78d7389 -https://conda.anaconda.org/conda-forge/noarch/wslink-2.5.7-pyhd8ed1ab_0.conda#410dee7cbee308f33b01645f16f11efc +https://conda.anaconda.org/conda-forge/noarch/wslink-2.6.0-pyhd8ed1ab_0.conda#c3bcfd3bb22061d48cb833925795105f +https://conda.anaconda.org/conda-forge/noarch/zarr-3.4.1-pyhc364b38_0.conda#052cab2fa318f6ffeefc149addd9e6fb +https://conda.anaconda.org/conda-forge/noarch/adwaita-icon-theme-51.0-unix_0.conda#6f62266fb2bfc35a5071cfa1d6ffdcc9 https://conda.anaconda.org/conda-forge/linux-64/cartopy-0.25.0-np2py313h73dcb5b_3.conda#85793522f0ed1e2d31e7dfc53fd66035 https://conda.anaconda.org/conda-forge/noarch/cmocean-4.0.3-pyhd8ed1ab_1.conda#dd71e4ec2fbffe38c0359976505f816e -https://conda.anaconda.org/conda-forge/noarch/cyclopts-4.23.1-pyhcf101f3_0.conda#252497d51f0133ecc9e68fc6eaa01a79 +https://conda.anaconda.org/conda-forge/noarch/cyclopts-5.2.0-pyh5ded981_0.conda#fae753a13c1df14b3aeaf2f8e5b5b314 https://conda.anaconda.org/conda-forge/linux-64/gtk3-3.24.52-ha5ea40c_0.conda#bcaea22d85999a4f17918acfab877e61 -https://conda.anaconda.org/conda-forge/linux-64/librsvg-2.62.3-h4c96295_0.conda#492c8d9b1c564c2e948b6cb4ba0f8261 +https://conda.anaconda.org/conda-forge/linux-64/minijinja-3.0.0-py311h1180b86_0.conda#ebba9c2cc80a43f736c13772bee155d8 https://conda.anaconda.org/conda-forge/noarch/nc-time-axis-1.4.1-pyhd8ed1ab_1.conda#9a2be7d0089f5934b550933ca0d9fe85 -https://conda.anaconda.org/conda-forge/linux-64/netcdf-fortran-4.6.3-nompi_hd5f62d4_101.conda#32619150ef344fce7cb11a593c82b96f -https://conda.anaconda.org/conda-forge/linux-64/netcdf4-1.7.4-nompi_py311hb115678_109.conda#d1f58fad9a5369f80109b81122a1af0d +https://conda.anaconda.org/conda-forge/linux-64/netcdf-fortran-4.6.4-nompi_h52816da_101.conda#2f18bb969e8acd8a1f45293e9782374e +https://conda.anaconda.org/conda-forge/linux-64/netcdf4-1.7.4-nompi_py311hd34d2e2_111.conda#d697ebf773f33945402fe3e1fda13635 +https://conda.anaconda.org/conda-forge/noarch/pre-commit-4.7.0-pyha770c72_0.conda#167169a52ede77767f9fa66b52983a20 +https://conda.anaconda.org/conda-forge/noarch/requests-2.34.2-pyhcf101f3_0.conda#4a85203c1d80c1059086ae860836ffb9 +https://conda.anaconda.org/conda-forge/linux-64/vtk-base-9.7.0-py313h6adec9e_0.conda#f8db7b1c571bc85ccacb28a1a59ef355 +https://conda.anaconda.org/conda-forge/linux-64/esmf-8.9.1-nompi_h39d0119_4.conda#074ca9356f9e916ec51ad6ab47fd55ba +https://conda.anaconda.org/conda-forge/linux-64/graphviz-14.1.2-h8b86629_0.conda#341fc61cfe8efa5c72d24db56c776f44 https://conda.anaconda.org/conda-forge/noarch/pooch-1.9.0-pyhd8ed1ab_0.conda#dd4b6337bf8886855db6905b336db3c8 -https://conda.anaconda.org/conda-forge/noarch/pre-commit-4.6.2-pyha770c72_0.conda#c36b88db560b4e74f294380613f1a239 https://conda.anaconda.org/conda-forge/noarch/requests-file-2.1.0-pyhd8ed1ab_1.conda#69be0472744969b97324d5d3b21845a9 -https://conda.anaconda.org/conda-forge/linux-64/vtk-base-9.6.2-py313h2d83dc5_7.conda#53910c5150cbfaace5e3fb3cad62606f -https://conda.anaconda.org/conda-forge/noarch/adwaita-icon-theme-49.0-unix_0.conda#b3f0179590f3c0637b7eb5309898f79e -https://conda.anaconda.org/conda-forge/linux-64/esmf-8.9.1-nompi_h39d0119_4.conda#074ca9356f9e916ec51ad6ab47fd55ba -https://conda.anaconda.org/conda-forge/noarch/pyvista-0.48.4-pyhd8ed1ab_1.conda#7688fc6ae896d8fc848776f9ba60d716 https://conda.anaconda.org/conda-forge/noarch/esmpy-8.9.1-pyhdfbf58e_0.conda#0155d4a55415cda7f7f09bc0b7760f51 +https://conda.anaconda.org/conda-forge/noarch/pyvista-0.49.1-pyha804496_0.conda#ea32ac5d548e14030818673f68e6a756 https://conda.anaconda.org/conda-forge/noarch/geovista-0.5.3-pyhd8ed1ab_1.conda#64348d05eedb1b1b5676f63101d004f2 -https://conda.anaconda.org/conda-forge/linux-64/graphviz-14.1.2-h8b86629_0.conda#341fc61cfe8efa5c72d24db56c776f44 https://conda.anaconda.org/conda-forge/noarch/pydata-sphinx-theme-0.19.0-pyhcf101f3_0.conda#87921f66a4dc56ce92e4ff13be5f63dc https://conda.anaconda.org/conda-forge/noarch/sphinx-copybutton-0.5.2-pyhd8ed1ab_1.conda#bf22cb9c439572760316ce0748af3713 https://conda.anaconda.org/conda-forge/noarch/sphinx-data-viewer-0.1.5-pyhd8ed1ab_1.conda#fbe6437c6899c54e52f9258518035aca https://conda.anaconda.org/conda-forge/noarch/sphinx-design-0.7.0-pyhd8ed1ab_0.conda#28eddfb8b9ecdd044a6f609f985398a7 -https://conda.anaconda.org/conda-forge/noarch/sphinx-gallery-0.21.0-pyhd8ed1ab_0.conda#9b783047bd5bef0998f129bef8fad477 +https://conda.anaconda.org/conda-forge/noarch/sphinx-gallery-0.22.1-pyhd8ed1ab_0.conda#b7ec27cd4e50d3c48e34965448f5e65e https://conda.anaconda.org/conda-forge/noarch/sphinx-last-updated-by-git-0.3.8-pyhe01879c_0.conda#405a232fb900fc631d2f1b5cdf01dea9 https://conda.anaconda.org/conda-forge/noarch/sphinx-sitemap-2.9.0-pyhcf101f3_0.conda#3baeff262222dc87e978a68702bc5797 -https://conda.anaconda.org/conda-forge/noarch/sphinx-markdown-builder-0.6.11-pyhd8ed1ab_0.conda#8c2c507ff41616cf325c1dddf11fbccd +https://conda.anaconda.org/conda-forge/noarch/sphinx-markdown-builder-0.6.11-pyh5ded981_1.conda#bde76da7789d94a15b7d570b41535f93 https://conda.anaconda.org/conda-forge/noarch/sphinx-llm-1.0.0-pyhcf101f3_0.conda#6afdccbc5bdf7a00ba1d4b7606e9fd9f https://conda.anaconda.org/conda-forge/noarch/sphinx-reredirects-1.1.0-pyhd8ed1ab_0.conda#8392493ce6fbb8ab72a101af8e8a5d03 https://conda.anaconda.org/conda-forge/noarch/sphinx_changelog-1.6.0-pyhd8ed1ab_1.conda#3c4521d251abd353af49c3293030f674 @@ -367,7 +374,7 @@ https://conda.anaconda.org/conda-forge/noarch/sphinxcontrib-applehelp-2.0.0-pyhd https://conda.anaconda.org/conda-forge/noarch/sphinxcontrib-devhelp-2.0.0-pyhd8ed1ab_1.conda#910f28a05c178feba832f842155cbfff https://conda.anaconda.org/conda-forge/noarch/sphinxcontrib-htmlhelp-2.1.0-pyhd8ed1ab_1.conda#e9fb3fe8a5b758b4aff187d434f94f03 https://conda.anaconda.org/conda-forge/noarch/sphinxcontrib-jquery-4.1-pyhd8ed1ab_1.conda#403185829255321ea427333f7773dd1f -https://conda.anaconda.org/conda-forge/noarch/sphinx-needs-8.4.0-pyhcf101f3_0.conda#a1b92a7251c7cbb12e48e1caaa73558b +https://conda.anaconda.org/conda-forge/noarch/sphinx-needs-8.5.0-pyh5ded981_0.conda#b977a530aee976c189a7629b963f7516 https://conda.anaconda.org/conda-forge/noarch/sphinxcontrib-qthelp-2.0.0-pyhd8ed1ab_1.conda#00534ebcc0375929b45c3039b5ba7636 https://conda.anaconda.org/conda-forge/noarch/sphinx-8.1.3-pyhd8ed1ab_1.conda#1a3281a0dc355c02b5506d87db2d78ac https://conda.anaconda.org/conda-forge/noarch/sphinxcontrib-serializinghtml-2.0.0-pyhd8ed1ab_0.conda#f77df1fcf9af03b7287342638befca77 diff --git a/requirements/locks/py314-linux-64.lock b/requirements/locks/py314-linux-64.lock index bf0fca9583..2b99ea97de 100644 --- a/requirements/locks/py314-linux-64.lock +++ b/requirements/locks/py314-linux-64.lock @@ -1,208 +1,209 @@ # Generated by conda-lock. # platform: linux-64 -# input_hash: 44e0f500cd52311fec1d8114cd6dcb464ded7a96eedaaf6a9daaf7fbb548c949 +# input_hash: 5229ccb2d47adc9bc4e1cf97d02301218610f0857e80d366efae609fed73833c @EXPLICIT https://conda.anaconda.org/conda-forge/noarch/font-ttf-dejavu-sans-mono-2.37-hab24e00_0.tar.bz2#0c96522c6bdaed4b1566d11387caaf45 https://conda.anaconda.org/conda-forge/noarch/font-ttf-inconsolata-3.000-h77eed37_0.tar.bz2#34893075a5c9e55cdafac56607368fc6 https://conda.anaconda.org/conda-forge/noarch/font-ttf-source-code-pro-2.038-h77eed37_0.tar.bz2#4d59c254e01d9cde7957100457e2d5fb https://conda.anaconda.org/conda-forge/noarch/font-ttf-ubuntu-0.83-h77eed37_3.conda#49023d73832ef61042f6a237cb2687e7 https://conda.anaconda.org/conda-forge/linux-64/hicolor-icon-theme-0.17-ha770c72_3.conda#129e404c5b001f3ef5581316971e3ea0 -https://conda.anaconda.org/conda-forge/noarch/python_abi-3.14-8_cp314.conda#0539938c55b6b1a59b560e843ad864a4 +https://conda.anaconda.org/conda-forge/noarch/python_abi-3.14-9_cp314.conda#350d89b50d7de805abf2e63e29dee451 https://conda.anaconda.org/conda-forge/noarch/tzdata-2026c-h151e31d_0.conda#fcb489df604d100968b737f2cb6076c6 https://conda.anaconda.org/conda-forge/noarch/ca-certificates-2026.7.22-hbd8a1cb_0.conda#0f51e2391ade309db462a55611263e9c https://conda.anaconda.org/conda-forge/noarch/fonts-conda-forge-1-hc364b38_1.conda#a7970cd949a077b7cb9696379d338681 https://conda.anaconda.org/conda-forge/linux-64/libglvnd-1.7.0-ha4b6fd6_5.conda#cfcf11edcfd4acf0094771a9c3b8587f -https://conda.anaconda.org/conda-forge/linux-64/libgomp-16.2.0-he0feb66_4.conda#89d2c1231f47bd818f5d624b9411459d +https://conda.anaconda.org/conda-forge/linux-64/libgomp-16.2.0-he0feb66_7.conda#d904c7ddd95e026254b4b80808ee9e12 https://conda.anaconda.org/conda-forge/linux-64/libzlib-1.3.2-h25fd6f3_3.conda#0de0122d9570a8ab637c6b73db268389 https://conda.anaconda.org/conda-forge/linux-64/_openmp_mutex-4.5-20_gnu.conda#a9f577daf3de00bca7c3c76c0ecbd1de https://conda.anaconda.org/conda-forge/noarch/fonts-conda-ecosystem-1-0.tar.bz2#fee5683a3f04bd15cbd8318b096a27ab https://conda.anaconda.org/conda-forge/linux-64/libegl-1.7.0-ha4b6fd6_5.conda#3a1b41c4591a0a1734382af3e2b8bc08 https://conda.anaconda.org/conda-forge/linux-64/zlib-1.3.2-h25fd6f3_3.conda#6acb86426229f96f93e5468d1df3a5e8 https://conda.anaconda.org/conda-forge/linux-64/zstd-1.5.7-hb78ec9c_7.conda#aa459086047c0e5e27023ab19f8cb86a -https://conda.anaconda.org/conda-forge/linux-64/ld_impl_linux-64-2.46.1-default_hbd61a6d_102.conda#449500f2c089da11c40f5c21312e3e07 -https://conda.anaconda.org/conda-forge/linux-64/libgcc-16.2.0-ha9f2e26_4.conda#cba14d01083fc62ffd32c24d7d390633 -https://conda.anaconda.org/conda-forge/linux-64/aws-c-common-0.14.3-hb03c661_0.conda#ce8664d7c01d4a598ad38107b289b5ec +https://conda.anaconda.org/conda-forge/linux-64/ld_impl_linux-64-2.46.1-default_hbd61a6d_103.conda#2362c70e7e79827c32db35547d9c179b +https://conda.anaconda.org/conda-forge/linux-64/libgcc-16.2.0-ha9f2e26_7.conda#735629b400b70eeab21a2ba8f5b1c2ce +https://conda.anaconda.org/conda-forge/linux-64/aws-c-common-1.0.3-hebe6cf0_0.conda#cdb5797968bfe1737f1b71f525fc6518 https://conda.anaconda.org/conda-forge/linux-64/bzip2-1.0.8-hda65f42_10.conda#e675fabcf81499adc7edf58124fb1e01 https://conda.anaconda.org/conda-forge/linux-64/c-ares-1.34.8-hebe6cf0_2.conda#3ad2b403be8fc5612b9159e2ce621f69 -https://conda.anaconda.org/conda-forge/linux-64/fribidi-1.0.16-hb03c661_1.conda#1cd10eda5692519d01bb20e086e214c9 -https://conda.anaconda.org/conda-forge/linux-64/giflib-6.1.3-h280c20c_1.conda#6d9a90cfee2c7343d6c05a1b37ef75fb -https://conda.anaconda.org/conda-forge/linux-64/json-c-0.18-h6688a6e_0.conda#38f5dbc9ac808e31c00650f7be1db93f +https://conda.anaconda.org/conda-forge/linux-64/fribidi-1.0.17-h7cc23a3_0.conda#f9d5302bca196ee87e62f393f2626562 +https://conda.anaconda.org/conda-forge/linux-64/giflib-6.1.3-hebe6cf0_2.conda#07bdd65afe03df3fb5dbcaa8257c64a3 +https://conda.anaconda.org/conda-forge/linux-64/json-c-0.18-hebe6cf0_2.conda#9aa77c4e7f9a988df94da18a7b9b90fe https://conda.anaconda.org/conda-forge/linux-64/keyutils-1.6.3-h7cc23a3_1.conda#ba55d1b89fd7775e67de8291029b4059 -https://conda.anaconda.org/conda-forge/linux-64/libbrotlicommon-1.2.0-h39a168f_3.conda#7a2499a177753582fb7ae7e9dc4a908a +https://conda.anaconda.org/conda-forge/linux-64/libbrotlicommon-1.2.0-h39a168f_4.conda#df210655e30a05e3b069c91b7aaddd2d https://conda.anaconda.org/conda-forge/linux-64/libdeflate-1.25-hd45a770_1.conda#40f9b31aa9cf007789867df0decd0492 https://conda.anaconda.org/conda-forge/linux-64/libev-4.33-h280c20c_3.conda#7bc31538d6e3fb349e897353969237ec -https://conda.anaconda.org/conda-forge/linux-64/libexpat-2.8.1-hecca717_1.conda#b24d3c612f71e7aa74158d92106318b2 +https://conda.anaconda.org/conda-forge/linux-64/libexpat-2.8.5-hd2095e1_0.conda#72cfd2b95b0540247220b42f58a148d7 https://conda.anaconda.org/conda-forge/linux-64/libffi-3.7.0-h81df57d_1.conda#6525a0b06aa4fd390795f0740636a9dd -https://conda.anaconda.org/conda-forge/linux-64/libgcc-ng-16.2.0-h69a702a_4.conda#b3e52878163a841f6fb951989cc0b217 -https://conda.anaconda.org/conda-forge/linux-64/libgfortran5-16.2.0-h6b99dfc_4.conda#c22348a769bb072b6184eb6f7f05e4f2 +https://conda.anaconda.org/conda-forge/linux-64/libgcc-ng-16.2.0-h69a702a_7.conda#2005515ac9340f816fb2ac7e83afe553 +https://conda.anaconda.org/conda-forge/linux-64/libgfortran5-16.2.0-h6b99dfc_7.conda#7719ed7be38ad8c492c4bd5cc2bce0c4 https://conda.anaconda.org/conda-forge/linux-64/libiconv-1.18-h0cb94f2_3.conda#f92233bf33e24a25668bb2119e2c51f9 https://conda.anaconda.org/conda-forge/linux-64/libjpeg-turbo-3.2.0-hb03c661_1.conda#898d1c9793eaa52efc4727bd84d2e39a https://conda.anaconda.org/conda-forge/linux-64/liblzma-5.8.3-hb03c661_1.conda#1390b7c5ac0b1d8e447bc5efa6d3c8c2 https://conda.anaconda.org/conda-forge/linux-64/libmpdec-4.0.0-hb03c661_2.conda#fcfed1dc5053eb1901b66e7b1fc32588 https://conda.anaconda.org/conda-forge/linux-64/libnsl-2.0.1-hb9d3cd8_1.conda#d864d34357c3b65a4b731f78c0801dc4 https://conda.anaconda.org/conda-forge/linux-64/libpciaccess-0.19-hb03c661_1.conda#35fa2b34bbced424e6976d30f5fde576 -https://conda.anaconda.org/conda-forge/linux-64/libpng-1.6.58-h922cc85_1.conda#fcf71c8d979148873f6f8ad4cfc73d86 -https://conda.anaconda.org/conda-forge/linux-64/libstdcxx-16.2.0-h934c35e_4.conda#2f2ef0d96de5bdd8c1270ff22fdf9352 -https://conda.anaconda.org/conda-forge/linux-64/libuuid-2.42.2-h5347b49_0.conda#01bb81d12c957de066ea7362007df642 +https://conda.anaconda.org/conda-forge/linux-64/libpng-1.6.59-h922cc85_0.conda#8c9caf742a485ac688ffa88bbadbf0be +https://conda.anaconda.org/conda-forge/linux-64/libpython-3.14.8-hd51c5da_102_cp314.conda#5bc9fb686950e9ef2543adc230476238 +https://conda.anaconda.org/conda-forge/linux-64/libstdcxx-16.2.0-h934c35e_7.conda#9b6707e0caca917fa6ea9ee473f37a5d +https://conda.anaconda.org/conda-forge/linux-64/libuuid-2.42.4-hcfc3c73_0.conda#54a948337e66706380ff24d6b5fde63f https://conda.anaconda.org/conda-forge/linux-64/libwebp-base-1.6.0-hd42ef1d_1.conda#9332b53d0ea93c5d39e33be03a0c611a https://conda.anaconda.org/conda-forge/linux-64/lzo-2.10-hebe6cf0_1003.conda#10e0aa58ed390ba598b728badf9eb1b7 https://conda.anaconda.org/conda-forge/linux-64/ncurses-6.6-hdb14827_1.conda#ee6c0cd80a60961a1f48aa3e0b91f986 -https://conda.anaconda.org/conda-forge/linux-64/openssl-3.6.4-h781a0a9_0.conda#16e3034a330cc625f2c685edec60cf4e -https://conda.anaconda.org/conda-forge/linux-64/pthread-stubs-0.4-hb03c661_1003.conda#df2c27f36bdb0dde779f55b5df76a352 +https://conda.anaconda.org/conda-forge/linux-64/openssl-3.6.5-h781a0a9_0.conda#5a51a2ea653f120240b1a9bf76e0aae9 +https://conda.anaconda.org/conda-forge/linux-64/pthread-stubs-0.4-h7cc23a3_1004.conda#bb66b610707b811e3c65181a6431e7a8 https://conda.anaconda.org/conda-forge/linux-64/tk-8.6.13-noxft_h1df4ec4_4.conda#676a2f9b1c4fcf57b70aa5c65f6c60d0 https://conda.anaconda.org/conda-forge/linux-64/xorg-libice-1.1.2-h280c20c_0.conda#85c9442aec283b4e464fa9ecc484a2f3 -https://conda.anaconda.org/conda-forge/linux-64/xorg-libxau-1.0.12-hb03c661_2.conda#f06ef439c280a5f90b8bf62355008dbc -https://conda.anaconda.org/conda-forge/linux-64/xorg-libxdmcp-1.1.5-hb03c661_2.conda#2e66c929f3d879708335b6ea4557c838 -https://conda.anaconda.org/conda-forge/linux-64/xorg-xorgproto-2025.1-h280c20c_1.conda#3b51576511038b50fdbd05245e22e4b1 +https://conda.anaconda.org/conda-forge/linux-64/xorg-libxau-1.0.12-h7cc23a3_2.conda#da33efee1a93603d92e06d4a2b68def6 +https://conda.anaconda.org/conda-forge/linux-64/xorg-libxdmcp-1.1.5-h7cc23a3_2.conda#eabfc45786244de252090cf8d55128a8 +https://conda.anaconda.org/conda-forge/linux-64/xorg-xorgproto-2025.1-hebe6cf0_3.conda#c3d0a6f03ba3aa312777b41b20a71ec0 https://conda.anaconda.org/conda-forge/linux-64/xxhash-0.8.3-hec8ed23_1.conda#72ab4ea51529ec63ff010ae9fb62b02f https://conda.anaconda.org/conda-forge/linux-64/yaml-0.2.5-hebe6cf0_3.conda#e741576fb8f89821ac7c1c537322a33d -https://conda.anaconda.org/conda-forge/linux-64/aws-c-cal-0.9.15-h6bbde05_1.conda#8f2b374c79908f271be3d0e9f1c462a8 -https://conda.anaconda.org/conda-forge/linux-64/aws-c-compression-0.3.2-h01ee8f8_5.conda#ae862abdbb3160f97478d3d4749164c0 -https://conda.anaconda.org/conda-forge/linux-64/aws-c-sdkutils-0.2.7-h01ee8f8_3.conda#412f0cf18cc4c4945655dba954ca2d76 -https://conda.anaconda.org/conda-forge/linux-64/aws-checksums-0.2.10-h01ee8f8_5.conda#4d70caf5260e0787675c3bf61ae5b64c +https://conda.anaconda.org/conda-forge/linux-64/aws-c-cal-1.0.0-hcf7459e_1.conda#2009b71b358163738f339c76640b512c +https://conda.anaconda.org/conda-forge/linux-64/aws-c-compression-1.0.0-ha404f16_1.conda#da0c7b86d5379eb44acfa084897e8b98 +https://conda.anaconda.org/conda-forge/linux-64/aws-c-sdkutils-1.0.0-ha404f16_1.conda#cbbd999c74a030951365225909e376c7 +https://conda.anaconda.org/conda-forge/linux-64/aws-checksums-1.0.1-ha404f16_0.conda#10e6e2f11271e66dfad43eb002359f22 https://conda.anaconda.org/conda-forge/linux-64/geos-3.14.1-h55b0958_0.conda#5f856b72ca9cd5b5b4a1c43551804622 https://conda.anaconda.org/conda-forge/linux-64/graphite2-1.3.15-h54a6638_1.conda#f9fe2984587fa8235a6af6004760cd18 https://conda.anaconda.org/conda-forge/linux-64/icu-78.3-py310h44b86e0_2.conda#72a381cbad04f24b1c2a43ef707f45b4 https://conda.anaconda.org/conda-forge/linux-64/lerc-4.2.0-hdb68285_0.conda#fb9d356b1a57d6d54768be7ebd5fce09 https://conda.anaconda.org/conda-forge/linux-64/libaec-1.1.5-h088129d_0.conda#86f7414544ae606282352fa1e116b41f -https://conda.anaconda.org/conda-forge/linux-64/libbrotlidec-1.2.0-ha411449_3.conda#6ab3315dc56618d652c1da42a648a129 -https://conda.anaconda.org/conda-forge/linux-64/libbrotlienc-1.2.0-h018ffa1_3.conda#2ac965638d4c6b2b38383bb1aebaf543 +https://conda.anaconda.org/conda-forge/linux-64/libbrotlidec-1.2.0-ha411449_4.conda#0cfd601eb03cca403dcc248844787486 +https://conda.anaconda.org/conda-forge/linux-64/libbrotlienc-1.2.0-h018ffa1_4.conda#4136f6e4ef4835cc7df39a707cbd511c https://conda.anaconda.org/conda-forge/linux-64/libdrm-2.4.129-h7cc23a3_0.conda#64cc91512b6278c315349dc13c53f680 -https://conda.anaconda.org/conda-forge/linux-64/libedit-3.1.20250104-pl5321h373387f_1.conda#50708d3b951d0f8e2d7f2df5b5edc040 +https://conda.anaconda.org/conda-forge/linux-64/libedit-3.1.20260512-pl5321h373387f_0.conda#27dbd7e2984b8fe48e069379636ccfdd https://conda.anaconda.org/conda-forge/linux-64/libfreetype6-2.14.3-h5e6c136_2.conda#b72a266a9317036fd0464cb98b027cd0 -https://conda.anaconda.org/conda-forge/linux-64/libgfortran-16.2.0-h69a702a_4.conda#5e92b8413fd1c8f8f3006f4661b12def -https://conda.anaconda.org/conda-forge/linux-64/libhwy-1.4.0-h57c4cff_1.conda#d58bfbaaf51ed85771eae92c2e7f5816 +https://conda.anaconda.org/conda-forge/linux-64/libgfortran-16.2.0-h69a702a_7.conda#3a8d285f5f22138ef826ad54cf267e73 +https://conda.anaconda.org/conda-forge/linux-64/libhwy-1.4.0-h57c4cff_2.conda#145497b38fe581a7760dd88b7377c96b https://conda.anaconda.org/conda-forge/linux-64/libnghttp2-1.68.1-h74cf4be_1.conda#75d211f04ac87506f1f43420712a69fe -https://conda.anaconda.org/conda-forge/linux-64/libpython-3.14.7-hdc7f604_101_cp314.conda#1bb97845e1174b789edc36283b349dd8 https://conda.anaconda.org/conda-forge/linux-64/libssh2-1.11.1-h6154650_1.conda#5c526561e1d2a2e071941174eae84557 -https://conda.anaconda.org/conda-forge/linux-64/libstdcxx-ng-16.2.0-hdf11a46_4.conda#de0dceacf3e33c5fc88e167885dc8274 +https://conda.anaconda.org/conda-forge/linux-64/libstdcxx-ng-16.2.0-hdf11a46_7.conda#25a23bde5e26d548374f3e02b4669670 https://conda.anaconda.org/conda-forge/linux-64/libudunits2-2.2.28-h40f5838_3.conda#4bdace082e911a3e1f1f0b721bed5b56 https://conda.anaconda.org/conda-forge/linux-64/libxcb-1.17.0-hb83e432_2.conda#61f0ffba9161cbb3a77722869b21e4bb -https://conda.anaconda.org/conda-forge/linux-64/libzip-1.11.2-h6991a6a_0.conda#a7b27c075c9b7f459f1c022090697cba +https://conda.anaconda.org/conda-forge/linux-64/libzip-1.11.4-h6008cf6_0.conda#05adf41813469e50f2103db6eba38edb https://conda.anaconda.org/conda-forge/linux-64/lz4-c-1.10.0-hee9eb32_2.conda#e38a3253d72ab07d471483f24947e2a2 -https://conda.anaconda.org/conda-forge/linux-64/minizip-4.2.2-hb71707f_0.conda#88450ba743694055e218d03a2fdd561e -https://conda.anaconda.org/conda-forge/linux-64/muparser-2.3.5-h5888daf_0.conda#ab3e3db511033340e75e7002e80ce8c0 +https://conda.anaconda.org/conda-forge/linux-64/minizip-4.2.2-haac24f3_1.conda#3fee8d11b0d8a54fb899749052accb4f +https://conda.anaconda.org/conda-forge/linux-64/muparser-2.3.5-hee9eb32_1.conda#7de84fd76ec2bdb7ac617507de3d3052 https://conda.anaconda.org/conda-forge/linux-64/pcre2-10.47-h8b3dc9c_1.conda#06f6113cf1ff4a54b65f87ead132a5f1 https://conda.anaconda.org/conda-forge/linux-64/pixman-0.46.4-h54a6638_3.conda#0ee5bb30034b081a1386c1e2c98ab0a7 https://conda.anaconda.org/conda-forge/linux-64/readline-8.3-hd6e31c0_1.conda#69c01c781e7c8190bbdaf79e3848c5ca -https://conda.anaconda.org/conda-forge/linux-64/s2n-1.7.6-h7e3ee7f_0.conda#e112cadcd79412d81496e784884bddf0 -https://conda.anaconda.org/conda-forge/linux-64/snappy-1.2.2-h03e3b7b_1.conda#98b6c9dc80eb87b2519b97bcf7e578dd +https://conda.anaconda.org/conda-forge/linux-64/s2n-1.7.7-ha5dc4ae_0.conda#4928f1e1964a6f38f0da349b9d0e1c6a +https://conda.anaconda.org/conda-forge/linux-64/snappy-1.2.2-h34e00bb_2.conda#2b7e31ffbd476eecf33b3b833692339c https://conda.anaconda.org/conda-forge/linux-64/wayland-1.26.0-hc1c935e_2.conda#88a47e22b53e50865b1cabe2eb648352 https://conda.anaconda.org/conda-forge/linux-64/xorg-libsm-1.2.6-h0d788c3_1.conda#aa7459ed9ad086ba11dda843d764b33d https://conda.anaconda.org/conda-forge/linux-64/zlib-ng-2.3.3-hce19668_1.conda#1726acec19beeed1c764385cd8622405 -https://conda.anaconda.org/conda-forge/linux-64/aws-c-io-0.27.5-h4f381ba_1.conda#948f5a4fddac779b7942626e5807caba +https://conda.anaconda.org/conda-forge/linux-64/aws-c-io-1.0.0-h07ba609_0.conda#72593f68772d8c7e72c8a09681b0fe42 https://conda.anaconda.org/conda-forge/linux-64/blosc-1.21.6-he440d0b_1.conda#2c2fae981fd2afd00812c92ac47d023d -https://conda.anaconda.org/conda-forge/linux-64/brotli-bin-1.2.0-h9908984_3.conda#8929291d9efc59715df1cfa7daf5e3ed -https://conda.anaconda.org/conda-forge/linux-64/freexl-2.0.0-h9dce30a_2.conda#ecb5d11305b8ba1801543002e69d2f2f +https://conda.anaconda.org/conda-forge/linux-64/brotli-bin-1.2.0-h9908984_4.conda#c48c09444f3736800ea0b9550c365baf +https://conda.anaconda.org/conda-forge/linux-64/freexl-2.0.0-h1f12c17_3.conda#4a0a8fd87ee4767db791047546f56501 https://conda.anaconda.org/conda-forge/linux-64/hdf4-4.2.15-h2a13503_7.conda#bd77f8da987968ec3927990495dc22e4 https://conda.anaconda.org/conda-forge/linux-64/krb5-1.22.2-hbc21106_2.conda#53318d715316929a574f83591308b1f8 +https://conda.anaconda.org/conda-forge/linux-64/libcrc32c-1.1.2-h9c3ff4c_0.tar.bz2#c965a5aa0d5c1c37ffc62dff36e28400 https://conda.anaconda.org/conda-forge/linux-64/libfreetype-2.14.3-ha770c72_2.conda#f8054e759d0ddddaf34a5c8fedc900a1 -https://conda.anaconda.org/conda-forge/linux-64/libglib-2.88.3-he503a2a_2.conda#533d342dedadf134c67760ce6fc5b748 -https://conda.anaconda.org/conda-forge/linux-64/libjxl-0.12.0-heb2dce7_2.conda#485788d339785bac87fe86315b4a0627 +https://conda.anaconda.org/conda-forge/linux-64/libglib-2.90.0-h569388d_0.conda#c85e9539551f1fd21104e518d940c244 +https://conda.anaconda.org/conda-forge/linux-64/libjxl-0.12.0-heb2dce7_4.conda#8770a39fd4c5559357efc7cdb8f0c742 https://conda.anaconda.org/conda-forge/linux-64/libmo_unpack-3.1.2-hf484d3e_1001.tar.bz2#95f32a6a5a666d33886ca5627239f03d -https://conda.anaconda.org/conda-forge/linux-64/libopenblas-0.3.34-pthreads_h94d23a6_0.conda#c282d68f272927612462b5d626838ef1 +https://conda.anaconda.org/conda-forge/linux-64/libopenblas-0.3.34-pthreads_hf13c14d_2.conda#7e19fcd73d7f0997d3cb3326d2586a86 https://conda.anaconda.org/conda-forge/linux-64/libpsl-0.23.1-hd9e3e90_1.conda#77be60417aa572afcb48cbe0f967401d -https://conda.anaconda.org/conda-forge/linux-64/librttopo-1.1.0-h46dd2a8_20.conda#df81fd57eacf341588d728c97920e86d -https://conda.anaconda.org/conda-forge/linux-64/libsqlite-3.53.4-h13e7031_1.conda#e72bbec309c2b0f37823ee7d4fabfcd3 +https://conda.anaconda.org/conda-forge/linux-64/librttopo-1.1.0-h55b0958_21.conda#6c639697369e9669032ca08d0a566282 +https://conda.anaconda.org/conda-forge/linux-64/libsqlite-3.54.0-h13e7031_1.conda#bb145547395d3329f3a1d930de9848b2 https://conda.anaconda.org/conda-forge/linux-64/libtiff-4.7.2-hcc2c06a_1.conda#0907d876f460ebc941ae590338b6102f -https://conda.anaconda.org/conda-forge/linux-64/libxml2-16-2.15.3-hca6bf5a_1.conda#20e4d67ec908f0405124f11612c48305 +https://conda.anaconda.org/conda-forge/linux-64/libxml2-16-2.15.4-hf3af7cc_1.conda#046d9eada488e1a1657c80b55c23da52 https://conda.anaconda.org/conda-forge/linux-64/qhull-2020.2-h434a139_5.conda#353823361b1d27eb3960efb076dfcaf6 https://conda.anaconda.org/conda-forge/linux-64/udunits2-2.2.28-h40f5838_3.conda#6bb8deb138f87c9d48320ac21b87e7a1 https://conda.anaconda.org/conda-forge/linux-64/uriparser-0.9.8-hac33072_0.conda#d71d3a66528853c0a1ac2c02d79a0284 -https://conda.anaconda.org/conda-forge/linux-64/xerces-c-3.3.0-hd9031aa_1.conda#66a1db55ecdb7377d2b91f54cd56eafa +https://conda.anaconda.org/conda-forge/linux-64/xerces-c-3.3.0-h78fed68_2.conda#f2ff0de6c209e90c53b3a195c2708f5f https://conda.anaconda.org/conda-forge/linux-64/xorg-libx11-1.8.13-he1eb515_1.conda#8c282bbe4808a3cc80a5c98e9aec1cfc https://conda.anaconda.org/conda-forge/linux-64/atk-1.0-2.38.0-h04ea711_2.conda#f730d54ba9cd543666d7220c9f7ed563 -https://conda.anaconda.org/conda-forge/linux-64/aws-c-http-0.11.0-hbe094ff_5.conda#c09f7ec7327b8bd52fc2d6eacaa18d3b -https://conda.anaconda.org/conda-forge/linux-64/brotli-1.2.0-h505cf86_3.conda#6ff307b78fbfb7dcafb7e25ff8bc9c10 -https://conda.anaconda.org/conda-forge/linux-64/dbus-1.16.2-h24cb091_1.conda#ce96f2f470d39bd96ce03945af92e280 +https://conda.anaconda.org/conda-forge/linux-64/aws-c-http-1.0.0-h392d1a3_1.conda#6701f6e712f9513e722ba57cde99f023 +https://conda.anaconda.org/conda-forge/linux-64/brotli-1.2.0-h505cf86_4.conda#746aa619a1bcaf4da541101179d7c60e +https://conda.anaconda.org/conda-forge/linux-64/dbus-1.16.2-he8c428d_2.conda#a6eb37b51be1a9a654dc3a8aca039b1a https://conda.anaconda.org/conda-forge/linux-64/fontconfig-2.18.3-h4db4eae_1.conda#922776b528a470ab5afa81fd42abfa1d https://conda.anaconda.org/conda-forge/linux-64/freetype-2.14.3-ha770c72_2.conda#2d0ea23b23603e07ca47f23f949f67b6 -https://conda.anaconda.org/conda-forge/linux-64/gdk-pixbuf-2.44.8-h68053f1_0.conda#10dab6a745f32ceeac6c9e00d9979797 -https://conda.anaconda.org/conda-forge/linux-64/glib-tools-2.88.3-h67ed8a3_2.conda#5c35ace78b7d630e77efcb37dacb62ae +https://conda.anaconda.org/conda-forge/linux-64/gdk-pixbuf-2.44.8-h68053f1_2.conda#d7bd6ccfc58eb6796bd6bba85878fc0e +https://conda.anaconda.org/conda-forge/linux-64/glib-tools-2.90.0-h3d503d9_0.conda#d4adaebf8a2c9040ada6549c52a6b8d8 https://conda.anaconda.org/conda-forge/linux-64/gts-0.7.6-h977cf35_4.conda#4d8df0b0db060d33c9a702ada998a8fe -https://conda.anaconda.org/conda-forge/linux-64/lcms2-2.19.1-h0c24ade_1.conda#8b3ce45e929cd8e8e5f4d18586b56d8b -https://conda.anaconda.org/conda-forge/linux-64/libblas-3.11.0-9_h4a7cf45_openblas.conda#f5c4b041925dea221dc4bad2e50569d9 +https://conda.anaconda.org/conda-forge/linux-64/lcms2-2.19.1-h9073bf1_3.conda#596464faa06e0b656b35416188ef9bba +https://conda.anaconda.org/conda-forge/linux-64/libblas-3.11.0-11_h4a7cf45_openblas.conda#b8cce1486f3c62f33291d6318c193d0d https://conda.anaconda.org/conda-forge/linux-64/libcups-2.3.3-h7a8fb5f_6.conda#49c553b47ff679a6a1e9fc80b9c5a2d4 -https://conda.anaconda.org/conda-forge/linux-64/libcurl-8.21.0-ha042cf0_5.conda#0ed167049513943d078a6c997df2b4e0 +https://conda.anaconda.org/conda-forge/linux-64/libcurl-8.22.0-ha042cf0_0.conda#e617deee7af8eb0c04f6296d12b54157 https://conda.anaconda.org/conda-forge/linux-64/libglx-1.7.0-ha4b6fd6_5.conda#614e10aae180f87b84f0d1ca8ceb7d9e https://conda.anaconda.org/conda-forge/linux-64/libkml-1.3.0-haa4a5bd_1023.conda#953b7cca897e21215302dbfe2af5cd0c -https://conda.anaconda.org/conda-forge/linux-64/libxml2-2.15.3-h49c6c72_1.conda#2d34cdb31014cbc8f1e9384c89ede0a5 -https://conda.anaconda.org/conda-forge/linux-64/openjpeg-2.5.4-h55fea9a_0.conda#11b3379b191f63139e29c0d19dee24cd -https://conda.anaconda.org/conda-forge/linux-64/python-3.14.7-hcd007b5_101_cp314.conda#4770984ac1045c080e867da13bf393ed -https://conda.anaconda.org/conda-forge/linux-64/sqlite-3.53.4-h9ffa6c4_1.conda#d42b24574925bfa3167efb3571c905ad +https://conda.anaconda.org/conda-forge/linux-64/libxml2-2.15.4-h7df9aa5_1.conda#51e5162f95bd6840f6b2b3169125fff5 +https://conda.anaconda.org/conda-forge/linux-64/openjpeg-2.5.4-heb1ab33_2.conda#a3f3ac7a68d01c198e276dbb99e62032 +https://conda.anaconda.org/conda-forge/linux-64/python-3.14.8-hcd007b5_102_cp314.conda#bcfadea0ff38555e26872aab0e974dbd +https://conda.anaconda.org/conda-forge/linux-64/sqlite-3.54.0-h9ffa6c4_1.conda#1060a8b69c6774ec85d81878eb299038 https://conda.anaconda.org/conda-forge/linux-64/xkeyboard-config-2.48-h280c20c_0.conda#b233b41be0bf210989d57160ed39b394 https://conda.anaconda.org/conda-forge/linux-64/xorg-libxext-1.3.7-h7cc23a3_1.conda#e5b6b28536b81b3f4cb20db4668a4642 https://conda.anaconda.org/conda-forge/linux-64/xorg-libxfixes-6.0.2-h7cc23a3_1.conda#09132e874fe0e1f5e4492b0b6b904b6e https://conda.anaconda.org/conda-forge/linux-64/xorg-libxrender-0.9.12-hb03c661_1.conda#e470d224a7a5be1b1d021bded7abb536 https://conda.anaconda.org/conda-forge/noarch/alabaster-1.0.0-pyhd8ed1ab_1.conda#1fd9696649f65fd6611fcdb4ffec738a https://conda.anaconda.org/conda-forge/noarch/attrs-26.1.0-pyhcf101f3_0.conda#c6b0543676ecb1fb2d7643941fe375f2 -https://conda.anaconda.org/conda-forge/linux-64/aws-c-auth-0.10.4-h4610da3_2.conda#cefa84dbf94066e7a6902af288a2fedd +https://conda.anaconda.org/conda-forge/linux-64/aws-c-auth-1.0.0-h7648d90_2.conda#f4f169a23884de1bdb9c25ded8f2aec1 https://conda.anaconda.org/conda-forge/noarch/babel-2.18.0-pyhcf101f3_1.conda#f1976ce927373500cc19d3c0b2c85177 -https://conda.anaconda.org/conda-forge/noarch/backports.zstd-1.7.0-py314h680f03e_0.conda#92adf685875ab717f68ad4172ef6de27 -https://conda.anaconda.org/conda-forge/linux-64/brotli-python-1.2.0-py314hcd2bdb6_3.conda#bd1be0851060138e038f6f4e09cc1eb4 -https://conda.anaconda.org/conda-forge/linux-64/cairo-1.18.4-h3c89d7e_2.conda#e83331a940393006789fedddc3b492f3 +https://conda.anaconda.org/conda-forge/noarch/backports.zstd-1.8.0-py314h680f03e_0.conda#5d5df669855f081b1c05f547f77925c6 +https://conda.anaconda.org/conda-forge/linux-64/brotli-python-1.2.0-py314hcd2bdb6_4.conda#d9aabceecc99b3b8b0eb7090c5776af4 +https://conda.anaconda.org/conda-forge/linux-64/cairo-1.18.6-h3c89d7e_0.conda#7920701f9abcb488411495742cf365a7 https://conda.anaconda.org/conda-forge/noarch/certifi-2026.7.22-pyhd8ed1ab_0.conda#37e13edbe3b48f1095a9d085ef9cd83b https://conda.anaconda.org/conda-forge/noarch/cfgv-3.5.0-pyhd8ed1ab_0.conda#381bd45fb7aa032691f3063aff47e3a1 -https://conda.anaconda.org/conda-forge/noarch/charset-normalizer-3.5.1-pyhd8ed1ab_0.conda#e0ac3accc64e23e40969d660e5f58ac8 -https://conda.anaconda.org/conda-forge/noarch/click-8.4.2-pyhc90fa1f_0.conda#2c4bd6aeb90bb157456841c3270a0d92 +https://conda.anaconda.org/conda-forge/noarch/click-8.5.0-pyh5ded981_0.conda#dea5b13a211bbf99876deb980b408a66 https://conda.anaconda.org/conda-forge/noarch/cloudpickle-3.1.2-pyhcf101f3_1.conda#61b8078a0905b12529abc622406cb62c https://conda.anaconda.org/conda-forge/noarch/colorama-0.4.6-pyhd8ed1ab_1.conda#962b9857ee8e7018c22f2776ffa0b2d7 -https://conda.anaconda.org/conda-forge/linux-64/coverage-7.16.0-py314h93e9acf_0.conda#d8dac0bde665abbf0f7847e10665d89d -https://conda.anaconda.org/conda-forge/noarch/cpython-3.14.7-py314hd8ed1ab_101.conda#5656db6c2bbe5923be509b769c843ed9 +https://conda.anaconda.org/conda-forge/linux-64/coverage-7.16.2-py314h93e9acf_0.conda#1101ca33d2a2af94aabe7438f9a7c9e1 +https://conda.anaconda.org/conda-forge/noarch/cpython-3.14.8-py314hd8ed1ab_102.conda#54ecbe8d03ac833af969b3543bf116ef https://conda.anaconda.org/conda-forge/noarch/cycler-0.12.1-pyhcf101f3_2.conda#4c2a8fef270f6c69591889b93f9f55c1 https://conda.anaconda.org/conda-forge/linux-64/cython-3.3.0-py314he5485c0_0.conda#c5d05ce7243daca0ff0ebac26d3d4e19 https://conda.anaconda.org/conda-forge/noarch/distlib-0.4.3-pyhcf101f3_0.conda#58638f77697c4f6726753eb8be34818b https://conda.anaconda.org/conda-forge/noarch/docutils-0.21.2-pyhd8ed1ab_1.conda#24c1ca34138ee57de72a943237cde4cc https://conda.anaconda.org/conda-forge/noarch/execnet-2.1.2-pyhd8ed1ab_0.conda#a57b4be42619213a94f31d2c69c5dda7 -https://conda.anaconda.org/conda-forge/noarch/filelock-3.32.4-pyhd8ed1ab_0.conda#0171dc5d54fdbb3f6e55f285f805e0fe -https://conda.anaconda.org/conda-forge/noarch/fsspec-2026.7.0-pyhd8ed1ab_0.conda#b9176565976c773a0739bd83deaf06cc +https://conda.anaconda.org/conda-forge/noarch/filelock-4.1.1-pyhd8ed1ab_0.conda#fbff0cf6c30bdd9b38d0448fdadfe72d +https://conda.anaconda.org/conda-forge/noarch/fsspec-2026.9.0-pyhd8ed1ab_0.conda#42de5cd170647599a84f468a42fabb89 +https://conda.anaconda.org/conda-forge/linux-64/google-crc32c-1.8.0-py314h34eed7f_2.conda#66938009746e41cdbccfeee9bb8a8db5 https://conda.anaconda.org/conda-forge/noarch/hpack-4.2.0-pyhd8ed1ab_0.conda#b395909221b9bd1df066e5930e18855b https://conda.anaconda.org/conda-forge/noarch/hyperframe-6.1.0-pyhd8ed1ab_0.conda#8e6923fc12f1fe8f8c4e5c9f343256ac -https://conda.anaconda.org/conda-forge/noarch/idna-3.19-pyhcf101f3_0.conda#a39ae05027e9b707742e41b30d296b75 +https://conda.anaconda.org/conda-forge/noarch/idna-3.20-pyh5ded981_0.conda#ee73be987f9de9ea5c2fdaeefa7b6491 https://conda.anaconda.org/conda-forge/noarch/imagesize-2.0.1-pyhd8ed1ab_0.conda#ebc56022e4ef7e74a829be94c53797ca -https://conda.anaconda.org/conda-forge/noarch/iniconfig-2.3.0-pyhd8ed1ab_0.conda#9614359868482abba1bd15ce465e3c42 +https://conda.anaconda.org/conda-forge/noarch/iniconfig-2.3.1-pyhd8ed1ab_0.conda#c0a803d97dd09da7e2c8b0fdfaa5739b https://conda.anaconda.org/conda-forge/noarch/iris-sample-data-2.5.2-pyhd8ed1ab_0.conda#895f6625dd8a246fece9279fcc12c1de https://conda.anaconda.org/conda-forge/linux-64/jsonschema-rs-0.37.4-py314h5059d10_0.conda#bbdf9e1de411fd55c05751bea6e2361c -https://conda.anaconda.org/conda-forge/linux-64/kiwisolver-1.5.1-py314h5383ef5_0.conda#bb0d2d018862f44cfb4f5566cb62540c +https://conda.anaconda.org/conda-forge/linux-64/kiwisolver-1.5.1-py314h5383ef5_3.conda#f8b25504f794b11a671868f7262aeab0 https://conda.anaconda.org/conda-forge/linux-64/libarchive-3.8.9-gpl_h3152399_101.conda#680520cff78a974d564cd99cdf5ec0b5 -https://conda.anaconda.org/conda-forge/linux-64/libcblas-3.11.0-9_h0358290_openblas.conda#092c5649f3727af436ab0f67f48c3811 +https://conda.anaconda.org/conda-forge/linux-64/libcblas-3.11.0-11_h0358290_openblas.conda#73a248c30811075059b5aeda766c2624 https://conda.anaconda.org/conda-forge/linux-64/libgd-2.3.3-h5fbf134_12.conda#88c1c66987cd52a712eea89c27104be6 https://conda.anaconda.org/conda-forge/linux-64/libgl-1.7.0-ha4b6fd6_5.conda#81141db127a106eb5a91df69a29c9918 https://conda.anaconda.org/conda-forge/linux-64/libglx-devel-1.7.0-ha4b6fd6_5.conda#fb00b4fb800f2d431303a5c1625ef9d0 -https://conda.anaconda.org/conda-forge/linux-64/liblapack-3.11.0-9_h47877c9_openblas.conda#e51473c2b7e1f9cb61daafccfd912abf +https://conda.anaconda.org/conda-forge/linux-64/liblapack-3.11.0-11_h47877c9_openblas.conda#5622a13917855ce4b953b9e655cf9262 https://conda.anaconda.org/conda-forge/linux-64/libxkbcommon-1.13.2-h51789e4_1.conda#f888787e0eab7a8076a67d197e155ffc -https://conda.anaconda.org/conda-forge/linux-64/libxml2-devel-2.15.3-h49c6c72_1.conda#f986d9c9de80813d076d235f753889ff +https://conda.anaconda.org/conda-forge/linux-64/libxml2-devel-2.15.4-h7df9aa5_1.conda#c6f14f767cbfe20fa03a324beb6cd7eb https://conda.anaconda.org/conda-forge/noarch/locket-1.0.0-pyhd8ed1ab_0.tar.bz2#91e27ef3d05cc772ce627e51cff111c4 -https://conda.anaconda.org/conda-forge/linux-64/markupsafe-3.0.3-py314h67df5f8_1.conda#9a17c4307d23318476d7fbf0fedc0cde -https://conda.anaconda.org/conda-forge/linux-64/msgpack-python-1.2.2-py314h5383ef5_0.conda#66e5d65a13ac23cd29ca87eb858c7d27 +https://conda.anaconda.org/conda-forge/linux-64/markupsafe-3.0.4-py314h93e9acf_1.conda#2ea393741e0fd221c98087478aeed284 +https://conda.anaconda.org/conda-forge/linux-64/msgpack-python-1.2.3-py314h5383ef5_0.conda#d91ed0c0fe2df67f0a80fdc47a9842e4 +https://conda.anaconda.org/conda-forge/linux-64/msgspec-0.22.0-py314h89acca1_0.conda#d0b947875606aa09cb40ee9900202b3b https://conda.anaconda.org/conda-forge/noarch/munkres-1.1.4-pyhd8ed1ab_1.conda#37293a85a0f4f77bbd9cf7aaefc62609 https://conda.anaconda.org/conda-forge/noarch/packaging-26.3-pyhc364b38_0.conda#936687ed80f295a1f5dbcf8bd34c252c -https://conda.anaconda.org/conda-forge/linux-64/pillow-12.3.0-py314h8ec4b1a_0.conda#233e62a8eb894b79b5c93f4f8dec4dcd +https://conda.anaconda.org/conda-forge/linux-64/pillow-12.3.0-py314h50bfbbb_4.conda#2180323126daa9971a80cc679cf12f46 https://conda.anaconda.org/conda-forge/noarch/pip-26.2.1-pyh145f28c_0.conda#573cb3ed111004230ed3de650653cd4d -https://conda.anaconda.org/conda-forge/noarch/platformdirs-4.11.5-pyhcf101f3_0.conda#152ec36401f710145a7db03475594410 +https://conda.anaconda.org/conda-forge/noarch/platformdirs-4.13.0-pyh5ded981_0.conda#35cc2a176fc6f886d6efa41ecb5f1de0 https://conda.anaconda.org/conda-forge/noarch/pluggy-1.6.0-pyhf9edf01_1.conda#d7585b6550ad04c8c5e21097ada2888e https://conda.anaconda.org/conda-forge/linux-64/proj-9.8.1-he0df7b0_0.conda#b23619e5e9009eaa070ead0342034027 -https://conda.anaconda.org/conda-forge/linux-64/psutil-7.2.2-py314hfe1a184_1.conda#1dadeb3cb86afd90e106395730cd87aa -https://conda.anaconda.org/conda-forge/noarch/pycparser-3.0-pyhcf101f3_0.conda#9c5491066224083c41b6d5635ed7107b +https://conda.anaconda.org/conda-forge/linux-64/psutil-7.2.2-py314hfe1a184_3.conda#23189243badc4f63a0f05c429aba161d +https://conda.anaconda.org/conda-forge/noarch/pycparser-3.11-pyh5ded981_0.conda#3cd164745122ab557cc30879390089e8 https://conda.anaconda.org/conda-forge/noarch/pygments-2.21.0-pyhcf101f3_0.conda#2882dee445dfa45b0c5afce3ffb7730a -https://conda.anaconda.org/conda-forge/noarch/pyparsing-3.3.2-pyhcf101f3_0.conda#3687cc0b82a8b4c17e1f0eb7e47163d5 +https://conda.anaconda.org/conda-forge/noarch/pyparsing-3.3.3-pyh5ded981_0.conda#8bf43ae1312e9b966063459e04c20fa7 https://conda.anaconda.org/conda-forge/noarch/pyshp-3.1.6-pyhcf101f3_0.conda#6778d276eb7b4230d622bdd2a732c75e https://conda.anaconda.org/conda-forge/noarch/pysocks-1.7.1-pyha55dd90_7.conda#461219d1a5bd61342293efa2c0c90eac -https://conda.anaconda.org/conda-forge/linux-64/python-xxhash-3.7.0-py314he82b845_0.conda#b7b8156134373dd419399dfbf255daae +https://conda.anaconda.org/conda-forge/linux-64/python-xxhash-3.7.0-py314hc7863f8_1.conda#4ae5b5cd11200f7e8e1a45839e4a395b https://conda.anaconda.org/conda-forge/linux-64/pyyaml-6.0.3-py314h67df5f8_1.conda#2035f68f96be30dc60a5dfd7452c7941 https://conda.anaconda.org/conda-forge/noarch/setuptools-84.0.0-pyh332efcf_0.conda#62ac906f1cd582c6c264c95625cb9d6f https://conda.anaconda.org/conda-forge/noarch/six-1.17.0-pyhe01879c_1.conda#3339e3b65d58accf4ca4fb8748ab16b3 https://conda.anaconda.org/conda-forge/noarch/snowballstemmer-3.1.1-pyhd8ed1ab_0.conda#46b6abe31482f6bca064b965696ae807 https://conda.anaconda.org/conda-forge/noarch/sortedcontainers-2.4.0-pyhd8ed1ab_1.conda#0401a17ae845fa72c7210e206ec5647d -https://conda.anaconda.org/conda-forge/noarch/soupsieve-2.9.2-pyhd8ed1ab_0.conda#ba33e6c8a46ee373fdf6dd8665212778 +https://conda.anaconda.org/conda-forge/noarch/soupsieve-3.0.3-pyhd8ed1ab_0.conda#bd2ea090567b9d0a53240bd1367c87ad https://conda.anaconda.org/conda-forge/noarch/sphinxcontrib-jsmath-1.0.1-pyhd8ed1ab_1.conda#fa839b5ff59e192f411ccc7dae6588bb https://conda.anaconda.org/conda-forge/noarch/tabulate-0.10.0-pyhcf101f3_0.conda#3b887b7b3468b0f494b4fad40178b043 https://conda.anaconda.org/conda-forge/noarch/tblib-3.2.2-pyhcf101f3_0.conda#f88bb644823094f436792f80fba3207e -https://conda.anaconda.org/conda-forge/noarch/tomli-2.4.1-pyhcf101f3_0.conda#b5325cf06a000c5b14970462ff5e4d58 -https://conda.anaconda.org/conda-forge/noarch/toolz-1.1.0-pyhd8ed1ab_1.conda#c07a6153f8306e45794774cf9b13bd32 -https://conda.anaconda.org/conda-forge/linux-64/tornado-6.5.8-py314h5bd0f2a_0.conda#faea84d2f2ccadf70cf9e55381d75b3e +https://conda.anaconda.org/conda-forge/noarch/tomli-2.5.0-pyh5ded981_0.conda#36d6a162326c52d6339c8b606f43932d +https://conda.anaconda.org/conda-forge/noarch/toolz-1.2.0-pyh5ded981_0.conda#17f8db9088267a3cb38d56cc22d268ae +https://conda.anaconda.org/conda-forge/linux-64/tornado-6.5.10-py314h89acca1_0.conda#d2bd0918d9c10df4fe2355e33fe7bc63 https://conda.anaconda.org/conda-forge/noarch/typing_extensions-4.16.0-pyhcf101f3_0.conda#c70ad746c22219b9700931707482992c -https://conda.anaconda.org/conda-forge/linux-64/unicodedata2-17.0.1-py314h5bd0f2a_0.conda#494fdf358c152f9fdd0673c128c2f3dd https://conda.anaconda.org/conda-forge/linux-64/xorg-libxcomposite-0.4.7-hb03c661_0.conda#f2ba4192d38b6cef2bb2c25029071d90 https://conda.anaconda.org/conda-forge/linux-64/xorg-libxcursor-1.2.3-hb9d3cd8_0.conda#2ccd714aa2242315acaf0a67faea780b https://conda.anaconda.org/conda-forge/linux-64/xorg-libxdamage-1.1.6-hb9d3cd8_0.conda#b5fcc7172d22516e1f965490e65e33a4 @@ -211,97 +212,100 @@ https://conda.anaconda.org/conda-forge/linux-64/xorg-libxinerama-1.1.6-hecca717_ https://conda.anaconda.org/conda-forge/linux-64/xorg-libxrandr-1.5.5-h7cc23a3_1.conda#798a8c9d171859a022e1cb89e7d0eb10 https://conda.anaconda.org/conda-forge/linux-64/xorg-libxxf86vm-1.1.7-hb03c661_0.conda#665d152b9c6e78da404086088077c844 https://conda.anaconda.org/conda-forge/noarch/zict-3.0.0-pyhd8ed1ab_1.conda#e52c2ef711ccf31bb7f70ca87d144b9e -https://conda.anaconda.org/conda-forge/noarch/zipp-4.1.0-pyhcf101f3_0.conda#ba3dcdc8584155c97c648ae9c044b7a3 +https://conda.anaconda.org/conda-forge/noarch/zipp-4.1.1-pyh5ded981_0.conda#6cad4d2cec9b7b64538c225ac91d2e8a https://conda.anaconda.org/conda-forge/noarch/accessible-pygments-0.0.5-pyhd8ed1ab_1.conda#74ac5069774cdbc53910ec4d631a3999 -https://conda.anaconda.org/conda-forge/noarch/affine-3.0.0-pyhd8ed1ab_0.conda#71dd867a02739013ae65894ca32d67b7 -https://conda.anaconda.org/conda-forge/linux-64/aws-c-s3-0.13.1-h7508075_1.conda#2daf54d3eef87b757f68d7597534dd2f -https://conda.anaconda.org/conda-forge/linux-64/cffi-2.1.1-py314h8d76f0c_2.conda#314853abf64fc052dea08bd17df17b65 +https://conda.anaconda.org/conda-forge/noarch/affine-3.0.1-pyhecae5ae_0.conda#aeb404cab9e719a49264ab40928f80c8 +https://conda.anaconda.org/conda-forge/linux-64/aws-c-s3-1.0.0-hfe03eda_1.conda#7a8a507033f2e478670a617bb4ffb0b8 +https://conda.anaconda.org/conda-forge/linux-64/cffi-2.1.1-py314h8d76f0c_3.conda#ee66d46f7693fdcbaa1761fb8a4f6c3d https://conda.anaconda.org/conda-forge/noarch/click-plugins-1.1.1.2-pyhd8ed1ab_0.conda#e9b05deb91c013e5224672a4ba9cf8d1 https://conda.anaconda.org/conda-forge/noarch/cligj-0.7.2-pyhd8ed1ab_2.conda#55c7804f428719241a90b152016085a1 -https://conda.anaconda.org/conda-forge/linux-64/cytoolz-1.1.0-py314h5bd0f2a_2.conda#a6a32cab83d59c7812ddbb03220057e3 +https://conda.anaconda.org/conda-forge/linux-64/cytoolz-1.2.0-py314hfe1a184_0.conda#f5c6a287281329672c700ffde2f0f4a2 +https://conda.anaconda.org/conda-forge/noarch/donfig-0.8.1.post1-pyhd8ed1ab_1.conda#c56a7fa5597ad78b62e1f5d21f7f8b8f https://conda.anaconda.org/conda-forge/noarch/exceptiongroup-1.3.1-pyhd8ed1ab_0.conda#8e662bd460bda79b1ea39194e3c4c9ab -https://conda.anaconda.org/conda-forge/noarch/fonttools-4.63.0-pyh7db6752_0.conda#0509ee74d95e5b98eb6fe2a47760e399 +https://conda.anaconda.org/conda-forge/linux-64/fonttools-4.66.1-py314hd42368d_0.conda#3265ea810d2c81a2b37f0d5047179418 https://conda.anaconda.org/conda-forge/noarch/h2-4.4.1-pyhcf101f3_0.conda#e652ac7756069c456d0da2a922cd7df5 https://conda.anaconda.org/conda-forge/noarch/importlib-metadata-9.0.1-pyhcf101f3_0.conda#77ec68e0aa61c3fff9ecbf840f93d64e -https://conda.anaconda.org/conda-forge/noarch/importlib_resources-7.1.0-pyhd8ed1ab_0.conda#0ba6225c279baf7ea9473a62ea0ec9ae https://conda.anaconda.org/conda-forge/noarch/jinja2-3.1.6-pyhcf101f3_1.conda#04558c96691bed63104678757beb4f8d https://conda.anaconda.org/conda-forge/linux-64/libgl-devel-1.7.0-ha4b6fd6_5.conda#f5f7fc038c611a25b22758c0a40d25c9 -https://conda.anaconda.org/conda-forge/linux-64/libharfbuzz-14.4.0-h23af247_0.conda#4463210c2a16ff5d3bf7f502af23f1f4 -https://conda.anaconda.org/conda-forge/linux-64/libspatialite-5.1.0-gpl_hab3fe16_120.conda#5947bdda89ced07ab0d8a5d6503082c3 +https://conda.anaconda.org/conda-forge/linux-64/libharfbuzz-14.6.0-h23af247_2.conda#6a93147cab011f36d1659745c3f579e2 +https://conda.anaconda.org/conda-forge/linux-64/libspatialite-5.1.0-gpl_h03b4e02_121.conda#affb2721df7014b4abcd75cc539c6cef https://conda.anaconda.org/conda-forge/noarch/nodeenv-1.10.0-pyhd8ed1ab_0.conda#eb52d14a901e23c39e9e7b4a1a5c015f -https://conda.anaconda.org/conda-forge/linux-64/numpy-2.5.2-py314h2b28147_0.conda#d77631addad93399a90157d2c597e7f3 +https://conda.anaconda.org/conda-forge/linux-64/numpy-2.5.4-py314hd3a7d6b_0.conda#533d72030f9dbe5f9a439c7f9673940b https://conda.anaconda.org/conda-forge/noarch/partd-1.4.2-pyhd8ed1ab_0.conda#0badf9c54e24cecfb0ad2f99d680c163 -https://conda.anaconda.org/conda-forge/noarch/pbr-7.1.1-pyhd8ed1ab_0.conda#51b000c0a6d35e244019f9fc8ab1c9f6 -https://conda.anaconda.org/conda-forge/linux-64/pyproj-3.7.2-py314hf1f854a_5.conda#d4b14b2897c0337461a70b86f721c6ea +https://conda.anaconda.org/conda-forge/noarch/pbr-7.1.3-pyhd8ed1ab_0.conda#66415d5d2894f1d10928005a264c2717 +https://conda.anaconda.org/conda-forge/linux-64/pyproj-3.8.0-py314hf1f854a_1.conda#f38427e4851bc63bb993b9f038ee97d7 https://conda.anaconda.org/conda-forge/noarch/python-dateutil-2.9.0.post0-pyhe01879c_2.conda#5b8d21249ff20967101ffa321cab24e8 -https://conda.anaconda.org/conda-forge/noarch/python-discovery-1.6.0-pyhcf101f3_0.conda#e87738e32eaf5dfa98a5d49f611d6312 -https://conda.anaconda.org/conda-forge/noarch/python-gil-3.14.7-h4df99d1_101.conda#35bb169c9d712dfc590b88f28f852157 +https://conda.anaconda.org/conda-forge/noarch/python-discovery-1.6.2-pyh5ded981_0.conda#17e6c8b2fc2d021a217d317fe4f89fc4 +https://conda.anaconda.org/conda-forge/noarch/python-gil-3.14.8-h4df99d1_102.conda#556734568e49dfd7cab8c0341ca1e67b https://conda.anaconda.org/conda-forge/noarch/typing-extensions-4.16.0-h69aa097_0.conda#c680b5747e8c4c8f23dca0bb7042a8fc -https://conda.anaconda.org/conda-forge/noarch/vcs_versioning-2.3.1-pyhcf101f3_0.conda#6043a03a3733302373b30e16ecc408b7 +https://conda.anaconda.org/conda-forge/noarch/vcs_versioning-2.6.0-pyh5ded981_0.conda#958f3e46d5025cdd9aa9c2417adceec8 https://conda.anaconda.org/conda-forge/linux-64/xorg-libxtst-1.2.5-h7cc23a3_4.conda#752a5ac9322e0fbbc24146c1ce3ae44e https://conda.anaconda.org/conda-forge/noarch/_python_abi3_support-1.0-hd8ed1ab_3.conda#3845f3d75991bae0fb90884662f4327c https://conda.anaconda.org/conda-forge/noarch/asv_runner-0.3.1-pyhd8ed1ab_0.conda#960a0f64467ce624941ad7c67b80a2e2 https://conda.anaconda.org/conda-forge/linux-64/at-spi2-core-2.40.3-h0630a04_0.tar.bz2#8cb2fc4cd6cc63f1369cfa318f581cc3 https://conda.anaconda.org/conda-forge/noarch/beautifulsoup4-4.15.0-pyha770c72_0.conda#3b261da3fe9b4168738712832410b022 -https://conda.anaconda.org/conda-forge/linux-64/cftime-1.6.5-py314hc02f841_1.conda#552b5d9d8a2a4be882e1c638953e7281 -https://conda.anaconda.org/conda-forge/linux-64/contourpy-1.3.3-py314h97ea11e_4.conda#95bede9cdb7a30a4b611223d52a01aa4 +https://conda.anaconda.org/conda-forge/linux-64/cftime-1.6.5-py314h12ceea2_2.conda#6b75e66981d66ebc1709356e7c8fd6a7 +https://conda.anaconda.org/conda-forge/linux-64/contourpy-1.4.0-py314h5383ef5_1.conda#2848b95cbf026062b30f3f1f504a2f61 https://conda.anaconda.org/conda-forge/noarch/dask-core-2026.8.0-pyhc364b38_0.conda#2ccdf45d2372e15c7edcb7a72d573e76 -https://conda.anaconda.org/conda-forge/linux-64/hdf5-2.2.0-nompi_hc529623_100.conda#336f7fb9af0c49cc246e2cff0992c3c4 -https://conda.anaconda.org/conda-forge/noarch/importlib-resources-7.1.0-pyhd8ed1ab_0.conda#e3bffa82b874f8b9a2631bddb3869529 +https://conda.anaconda.org/conda-forge/linux-64/hdf5-2.2.0-nompi_hdbb200c_101.conda#66762e6067bf7d52586ff21c78c970db https://conda.anaconda.org/conda-forge/linux-64/libegl-devel-1.7.0-ha4b6fd6_5.conda#577800fc8c9d8b7d9727246bbfde704e -https://conda.anaconda.org/conda-forge/linux-64/libgdal-core-3.13.3-h5fdb907_0.conda#c0723084e1315a5247d5a033d9eb5522 -https://conda.anaconda.org/conda-forge/linux-64/libharfbuzz-devel-14.4.0-h23af247_0.conda#26e37e05324d7bb723350d50b33057fc +https://conda.anaconda.org/conda-forge/linux-64/libgdal-core-3.13.3-h31aba09_1.conda#44c6780e3082468d8676aebdcd5068a6 +https://conda.anaconda.org/conda-forge/linux-64/libharfbuzz-devel-14.6.0-h23af247_2.conda#3cf459e930e6ea2e57bbf04be7fcb7c6 https://conda.anaconda.org/conda-forge/linux-64/libraqm-0.11.0-h6406941_0.conda#3ac89a48d224409739dbf6200e524373 -https://conda.anaconda.org/conda-forge/linux-64/mo_pack-0.3.1-py314hc02f841_2.conda#55ac6d85f5dd8ec5e9919e7762fcb31a -https://conda.anaconda.org/conda-forge/linux-64/pandas-3.0.5-py314hb4ffadd_1.conda#15cd0f375b79b47c4048e44396e6e6f1 -https://conda.anaconda.org/conda-forge/linux-64/pango-1.58.2-hda50119_0.conda#6a2822aaf9a34ac3708904a47ff3dd7e +https://conda.anaconda.org/conda-forge/linux-64/mo_pack-0.3.1-py314h12ceea2_3.conda#06aa38d74d46235f8bdb7743e1921848 +https://conda.anaconda.org/conda-forge/linux-64/numcodecs-0.17.0-py314h12ceea2_1.conda#565a3b1cd503bc0953f6c64fb99cd525 +https://conda.anaconda.org/conda-forge/linux-64/pandas-3.0.6-np2py314h9e1d7c2_0.conda#e0f55ab2a87a7f0483a633ac39557098 +https://conda.anaconda.org/conda-forge/linux-64/pango-1.58.2-h7bb47b9_1.conda#0b889b1c6b472d215d6332c6b195589c https://conda.anaconda.org/conda-forge/noarch/pytest-9.0.3-pyhc364b38_1.conda#6a991452eadf2771952f39d43615bb3e -https://conda.anaconda.org/conda-forge/linux-64/pywavelets-1.9.0-py314hc02f841_2.conda#5be92985870940eac3f3b8cda57002cc -https://conda.anaconda.org/conda-forge/linux-64/scipy-1.18.0-py314hf07bd8e_0.conda#62c390c1f8f51240f1ebc7ba782669ad -https://conda.anaconda.org/conda-forge/noarch/setuptools-scm-10.2.1-pyhcf101f3_0.conda#eefa3bc61c9224107d3a9afeb37552a9 -https://conda.anaconda.org/conda-forge/linux-64/shapely-2.1.2-py314hbe3edd8_2.conda#5963e6ee81772d450a35e6bc95522761 +https://conda.anaconda.org/conda-forge/linux-64/pywavelets-1.10.0-py314hfe1a184_1.conda#4eb9dcea543fa478c7ad2f0ef9f4ee13 +https://conda.anaconda.org/conda-forge/linux-64/scipy-1.18.1-py314h36f0660_1.conda#2c5a3b238546edf4e3ea3bd9c1b9fa03 +https://conda.anaconda.org/conda-forge/noarch/setuptools-scm-10.3.4-pyh5ded981_0.conda#437557778d99cc6f311f3c0ce023ad85 +https://conda.anaconda.org/conda-forge/linux-64/shapely-2.2.0-np2py314hefd98a1_2.conda#43ae47a48fc57bbeae0651227fda3d83 https://conda.anaconda.org/conda-forge/noarch/snuggs-1.4.7-pyhd8ed1ab_2.conda#9aa358575bbd4be126eaa5e0039f835c https://conda.anaconda.org/conda-forge/noarch/sphinxcontrib-apidoc-0.3.0-py_1.tar.bz2#855b087883443abb10f5faf6eef40860 +https://conda.anaconda.org/conda-forge/noarch/towncrier-26.9.0-pyhd8ed1ab_0.conda#d4167f473b71694bbca708347abee99e https://conda.anaconda.org/conda-forge/linux-64/ukkonen-1.1.0-py314h9891dd4_0.conda#5d3c008e54c7f49592fca9c32896a76f -https://conda.anaconda.org/conda-forge/noarch/urllib3-2.7.0-pyhd8ed1ab_0.conda#cbb88288f74dbe6ada1c6c7d0a97223e -https://conda.anaconda.org/conda-forge/noarch/virtualenv-21.7.7-pyhcf101f3_0.conda#768a861e1491dc8ee2a93fa8b7ee4317 +https://conda.anaconda.org/conda-forge/noarch/urllib3-2.8.0-pyhd8ed1ab_0.conda#407a3d3778570bae8e8fcf554e803e43 +https://conda.anaconda.org/conda-forge/noarch/virtualenv-21.14.7-pyh5ded981_0.conda#1635534aa129eb9ab6ce5e01929d86ff https://conda.anaconda.org/conda-forge/linux-64/at-spi2-atk-2.38.0-h0630a04_3.tar.bz2#6b889f174df1e0f816276ae69281af4d -https://conda.anaconda.org/conda-forge/linux-64/cf-units-3.3.1-py314hc02f841_0.conda#de50a60eab348de04809a33e180b4b01 +https://conda.anaconda.org/conda-forge/linux-64/cf-units-3.3.1-np2hc8c11d4_2.conda#9b777f085506bff796833be0ae292db1 +https://conda.anaconda.org/conda-forge/linux-64/charset-normalizer-3.5.2-h1114479_0.conda#78d5c6112c9320a1a797935cecfec7c2 https://conda.anaconda.org/conda-forge/noarch/distributed-2026.8.0-pyhc364b38_0.conda#f382c4eed5376ee372a6e67a0f6cd4d9 https://conda.anaconda.org/conda-forge/linux-64/epoxy-1.5.10-hb03c661_2.conda#057083b06ccf1c2778344b6dabace38b -https://conda.anaconda.org/conda-forge/linux-64/harfbuzz-14.4.0-ha770c72_0.conda#c2d4a4fe3216b26ef30e91d917c1b718 -https://conda.anaconda.org/conda-forge/noarch/identify-2.6.19-pyhd8ed1ab_0.conda#84a3233b709a289a4ddd7a2fd27dd988 +https://conda.anaconda.org/conda-forge/linux-64/harfbuzz-14.6.0-ha770c72_2.conda#0709eacc38906e8a237d9ef172af6688 +https://conda.anaconda.org/conda-forge/noarch/identify-2.6.20-pyhd8ed1ab_0.conda#be9ea4f030b4b2bc121b2a548b066c0b https://conda.anaconda.org/conda-forge/noarch/imagehash-4.3.2-pyhd8ed1ab_0.conda#efbc812363856906a80344b496389d2e https://conda.anaconda.org/conda-forge/linux-64/libnetcdf-4.10.1-nompi_he3e3c8e_201.conda#912aefa694a66897afeb7a2330b17d9f -https://conda.anaconda.org/conda-forge/linux-64/matplotlib-base-3.11.1-py314h9a9c090_2.conda#57f668cbc4b70c11ea9d19ebed67295e -https://conda.anaconda.org/conda-forge/linux-64/minijinja-2.22.0-py310hd8a072f_0.conda#ad9070e3f9613ecdd611e93bbb49c7e8 +https://conda.anaconda.org/conda-forge/linux-64/librsvg-2.62.4-ha427ee3_0.conda#f9bda2728debbde4c641bd1af103c623 +https://conda.anaconda.org/conda-forge/linux-64/matplotlib-base-3.11.2-py314h9124352_1.conda#55d364956c0d3a01f6e39db3f6f6cc9b https://conda.anaconda.org/conda-forge/noarch/pytest-cov-7.1.0-pyhcf101f3_0.conda#67d1790eefa81ed305b89d8e314c7923 -https://conda.anaconda.org/conda-forge/noarch/pytest-mock-3.15.1-pyhd8ed1ab_0.conda#0511afbe860b1a653125d77c719ece53 +https://conda.anaconda.org/conda-forge/noarch/pytest-mock-3.16.0-pyhd8ed1ab_0.conda#b038267c191ebae9d6374caf30cc2cca https://conda.anaconda.org/conda-forge/noarch/pytest-xdist-3.8.0-pyhd8ed1ab_0.conda#8375cfbda7c57fbceeda18229be10417 -https://conda.anaconda.org/conda-forge/linux-64/python-stratify-0.4.0-py314h5bd0f2a_3.conda#b55fcaf9e1ad884241180b9c3c94384e -https://conda.anaconda.org/conda-forge/linux-64/rasterio-1.5.1-py314h523d43f_1.conda#16272efd0bc234eaa6739cc9e446ab24 -https://conda.anaconda.org/conda-forge/noarch/requests-2.34.2-pyhcf101f3_0.conda#4a85203c1d80c1059086ae860836ffb9 -https://conda.anaconda.org/conda-forge/noarch/towncrier-25.8.0-pyhd8ed1ab_0.conda#3e0e8e44292bdac62f7bcbf0450b5cc7 +https://conda.anaconda.org/conda-forge/noarch/python-abi3-3.14-hab9af60_3.conda#ad28909b755a16c4f0d3fc8dfbff6fc7 +https://conda.anaconda.org/conda-forge/linux-64/python-stratify-0.4.0-py314h89acca1_4.conda#6cbce4f0cd522372910ccc8869964abf +https://conda.anaconda.org/conda-forge/linux-64/rasterio-1.5.2-py314h2895243_1.conda#2fb192d4cea99fdaf9d446e475e85cef +https://conda.anaconda.org/conda-forge/noarch/zarr-3.4.1-pyhc364b38_0.conda#052cab2fa318f6ffeefc149addd9e6fb +https://conda.anaconda.org/conda-forge/noarch/adwaita-icon-theme-51.0-unix_0.conda#6f62266fb2bfc35a5071cfa1d6ffdcc9 https://conda.anaconda.org/conda-forge/linux-64/cartopy-0.25.0-np2py314h6477eea_3.conda#1c2bf9dbf9be09fad080f505ffd6314c https://conda.anaconda.org/conda-forge/linux-64/gtk3-3.24.52-ha5ea40c_0.conda#bcaea22d85999a4f17918acfab877e61 -https://conda.anaconda.org/conda-forge/linux-64/librsvg-2.62.3-h4c96295_0.conda#492c8d9b1c564c2e948b6cb4ba0f8261 +https://conda.anaconda.org/conda-forge/linux-64/minijinja-3.0.0-py311h1180b86_0.conda#ebba9c2cc80a43f736c13772bee155d8 https://conda.anaconda.org/conda-forge/noarch/nc-time-axis-1.4.1-pyhd8ed1ab_1.conda#9a2be7d0089f5934b550933ca0d9fe85 -https://conda.anaconda.org/conda-forge/linux-64/netcdf-fortran-4.6.3-nompi_hd5f62d4_101.conda#32619150ef344fce7cb11a593c82b96f -https://conda.anaconda.org/conda-forge/linux-64/netcdf4-1.7.4-nompi_py311hb115678_109.conda#d1f58fad9a5369f80109b81122a1af0d -https://conda.anaconda.org/conda-forge/noarch/pre-commit-4.6.2-pyha770c72_0.conda#c36b88db560b4e74f294380613f1a239 -https://conda.anaconda.org/conda-forge/noarch/requests-file-2.1.0-pyhd8ed1ab_1.conda#69be0472744969b97324d5d3b21845a9 -https://conda.anaconda.org/conda-forge/noarch/adwaita-icon-theme-49.0-unix_0.conda#b3f0179590f3c0637b7eb5309898f79e +https://conda.anaconda.org/conda-forge/linux-64/netcdf-fortran-4.6.4-nompi_h52816da_101.conda#2f18bb969e8acd8a1f45293e9782374e +https://conda.anaconda.org/conda-forge/linux-64/netcdf4-1.7.4-nompi_py311hd34d2e2_111.conda#d697ebf773f33945402fe3e1fda13635 +https://conda.anaconda.org/conda-forge/noarch/pre-commit-4.7.0-pyha770c72_0.conda#167169a52ede77767f9fa66b52983a20 +https://conda.anaconda.org/conda-forge/noarch/requests-2.34.2-pyhcf101f3_0.conda#4a85203c1d80c1059086ae860836ffb9 https://conda.anaconda.org/conda-forge/linux-64/esmf-8.9.1-nompi_h39d0119_4.conda#074ca9356f9e916ec51ad6ab47fd55ba -https://conda.anaconda.org/conda-forge/noarch/esmpy-8.9.1-pyhdfbf58e_0.conda#0155d4a55415cda7f7f09bc0b7760f51 https://conda.anaconda.org/conda-forge/linux-64/graphviz-14.1.2-h8b86629_0.conda#341fc61cfe8efa5c72d24db56c776f44 +https://conda.anaconda.org/conda-forge/noarch/requests-file-2.1.0-pyhd8ed1ab_1.conda#69be0472744969b97324d5d3b21845a9 +https://conda.anaconda.org/conda-forge/noarch/esmpy-8.9.1-pyhdfbf58e_0.conda#0155d4a55415cda7f7f09bc0b7760f51 https://conda.anaconda.org/conda-forge/noarch/pydata-sphinx-theme-0.19.0-pyhcf101f3_0.conda#87921f66a4dc56ce92e4ff13be5f63dc https://conda.anaconda.org/conda-forge/noarch/sphinx-copybutton-0.5.2-pyhd8ed1ab_1.conda#bf22cb9c439572760316ce0748af3713 https://conda.anaconda.org/conda-forge/noarch/sphinx-data-viewer-0.1.5-pyhd8ed1ab_1.conda#fbe6437c6899c54e52f9258518035aca https://conda.anaconda.org/conda-forge/noarch/sphinx-design-0.7.0-pyhd8ed1ab_0.conda#28eddfb8b9ecdd044a6f609f985398a7 -https://conda.anaconda.org/conda-forge/noarch/sphinx-gallery-0.21.0-pyhd8ed1ab_0.conda#9b783047bd5bef0998f129bef8fad477 +https://conda.anaconda.org/conda-forge/noarch/sphinx-gallery-0.22.1-pyhd8ed1ab_0.conda#b7ec27cd4e50d3c48e34965448f5e65e https://conda.anaconda.org/conda-forge/noarch/sphinx-last-updated-by-git-0.3.8-pyhe01879c_0.conda#405a232fb900fc631d2f1b5cdf01dea9 https://conda.anaconda.org/conda-forge/noarch/sphinx-sitemap-2.9.0-pyhcf101f3_0.conda#3baeff262222dc87e978a68702bc5797 -https://conda.anaconda.org/conda-forge/noarch/sphinx-markdown-builder-0.6.11-pyhd8ed1ab_0.conda#8c2c507ff41616cf325c1dddf11fbccd +https://conda.anaconda.org/conda-forge/noarch/sphinx-markdown-builder-0.6.11-pyh5ded981_1.conda#bde76da7789d94a15b7d570b41535f93 https://conda.anaconda.org/conda-forge/noarch/sphinx-llm-1.0.0-pyhcf101f3_0.conda#6afdccbc5bdf7a00ba1d4b7606e9fd9f https://conda.anaconda.org/conda-forge/noarch/sphinx-reredirects-1.1.0-pyhd8ed1ab_0.conda#8392493ce6fbb8ab72a101af8e8a5d03 https://conda.anaconda.org/conda-forge/noarch/sphinx_changelog-1.6.0-pyhd8ed1ab_1.conda#3c4521d251abd353af49c3293030f674 @@ -309,7 +313,7 @@ https://conda.anaconda.org/conda-forge/noarch/sphinxcontrib-applehelp-2.0.0-pyhd https://conda.anaconda.org/conda-forge/noarch/sphinxcontrib-devhelp-2.0.0-pyhd8ed1ab_1.conda#910f28a05c178feba832f842155cbfff https://conda.anaconda.org/conda-forge/noarch/sphinxcontrib-htmlhelp-2.1.0-pyhd8ed1ab_1.conda#e9fb3fe8a5b758b4aff187d434f94f03 https://conda.anaconda.org/conda-forge/noarch/sphinxcontrib-jquery-4.1-pyhd8ed1ab_1.conda#403185829255321ea427333f7773dd1f -https://conda.anaconda.org/conda-forge/noarch/sphinx-needs-8.4.0-pyhcf101f3_0.conda#a1b92a7251c7cbb12e48e1caaa73558b +https://conda.anaconda.org/conda-forge/noarch/sphinx-needs-8.5.0-pyh5ded981_0.conda#b977a530aee976c189a7629b963f7516 https://conda.anaconda.org/conda-forge/noarch/sphinxcontrib-qthelp-2.0.0-pyhd8ed1ab_1.conda#00534ebcc0375929b45c3039b5ba7636 https://conda.anaconda.org/conda-forge/noarch/sphinx-8.1.3-pyhd8ed1ab_1.conda#1a3281a0dc355c02b5506d87db2d78ac https://conda.anaconda.org/conda-forge/noarch/sphinxcontrib-serializinghtml-2.0.0-pyhd8ed1ab_0.conda#f77df1fcf9af03b7287342638befca77 diff --git a/requirements/py312.yml b/requirements/py312.yml index 311a91f39d..d37a734226 100644 --- a/requirements/py312.yml +++ b/requirements/py312.yml @@ -37,6 +37,7 @@ dependencies: - python-stratify - rasterio - affine + - zarr >=3.0.8 # Test dependencies. - asv_runner diff --git a/requirements/py313.yml b/requirements/py313.yml index 4f27ae73e6..dfb47e25b0 100644 --- a/requirements/py313.yml +++ b/requirements/py313.yml @@ -37,6 +37,7 @@ dependencies: - python-stratify - rasterio - affine + - zarr >=3.0.8 # Test dependencies. - asv_runner diff --git a/requirements/py314.yml b/requirements/py314.yml index 79965dcdde..11187c6165 100644 --- a/requirements/py314.yml +++ b/requirements/py314.yml @@ -37,6 +37,7 @@ dependencies: - python-stratify - rasterio - affine + - zarr >=3.0.8 # Test dependencies. - asv_runner