Include the l, ln and ml scenarios and tighten PBS requests by paul-leopardi · Pull Request #167 · ACCESS-NRI/CMIP7-Input

paul-leopardi · 2026-10-09T05:07:54Z

Closes #166 by adding the l, ln and ml ScenarioMIP forcings. Also reduces the requested walltime and memory in all PBS jobs.

The changes are AI-assisted, using Google Antigravity and Google Gemini models.

paul-leopardi · 2026-10-09T05:21:06Z

I successfully ran the workflow twice, as run2 and run3.

For run2, before f6bc441:

Logs are in /scratch/tm70/pcl851/cylc-run/cmip7-esm1p6-scen7-l-ln-ml/run2.
Output is in /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.10.08 and https://github.com/ACCESS-NRI/access-esm1.6-configs/tree/gen-*-87e3d079 .

For run3, after f6bc441:

Logs are in /scratch/tm70/pcl851/cylc-run/cmip7-esm1p6-scen7-l-ln-ml/run3.
Output is in /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.10.09 and https://github.com/ACCESS-NRI/access-esm1.6-configs/tree/gen-piControl-e3f2c65a .

MartinDix · 2026-10-12T03:31:19Z

CO2 emissions look correct

MartinDix · 2026-10-13T00:40:13Z

Ozone looks ok

MartinDix · 2026-10-13T01:16:51Z

Nitrogen look ok

blimlim · 2026-10-13T05:40:36Z

I'd missed this in the last review but it's relevant here too. The GHGs in the namelist all finish in 2150: https://github.com/ACCESS-NRI/access-esm1.6-configs/blob/401b65672c35adbcc9e9060cd0fb4227e8a9e2b9/atmosphere/namelists#L1072-L1074

As @MartinDix noted here, we actually need an additional year of data for the UM's interpolation of the GHGs to work properly in the final year. @paul-leopardi, are you able to include 2151 in the GHG output?

Otherwise, the actual data in the namelists looks good:

blimlim · 2026-10-13T23:44:18Z

+                               'l':  'FZJ-CMIP-ozone-l-2-0',
+                               'ln': 'FZJ-CMIP-ozone-ln-2-0',
+                               'ml': 'FZJ-CMIP-ozone-ml-2-0',}


It looks like only the version 2-0 zmta data is available for the 3 new scenarios - is that correct?

Also just wanted to confirm, does the note here have any implications for our processing:

The meaning is unchanged from the v1.0 files. However values which were set to NaN in the v1.0 files because they were below the Earth's surface have been set to the surface values in v2.0. Please be aware of this when processing the files.

I have just searched https://esgf-node.ornl.gov/search?project=input4MIPs and have confirmed that only the version 2.0 zmta data is available for the 3 new scenarios.

The zmta values are dealt with in the UKMO code as amended for nci-gadi. The code does not explictly tests for NaN values, but Iris itself may propagate them. It is therefore better not to have unmasked NaN values in any NetCDF file read by Iris, especially when Iris cubes obtained from that file are used in interpolation.

I will try running with the 2.0 files for the h, hl, m and vl forcings, to see if that makes a difference to the resulting .anc files.

I have run the workflow again using the 2.0 files for the h, hl, m and vl forcings, and the output differs for those forcings now.

Logs are in /scratch/tm70/pcl851/cylc-run/cmip7-esm1p6-scen7-l-ln-ml/run5/log/.

Output is in /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.10.20

[pcl851@gadi-login-08 ~]$ for f in $files20; do g=${f//10.20/10.09}; ls $f; mule-cumf --stashmaster 7.3 --summary $f $g; done /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.10.20/modern/scen7-h/atmosphere/forcing/global.N96/2026.10.20/ozone_h_2022_2150_cmip7.anc %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% * (CUMF-II) Module Information * %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% mule : /g/data/access/projects/access/apps/pythonlib/mule_python3/2026.03.2/mule_lib_intel_python3/mule/__init__.py (version 2025.10.1) um_utils : /g/data/access/projects/access/apps/pythonlib/mule_python3/2026.03.2/mule_lib_intel_python3/um_utils/__init__.py (version 2025.10.1) um_packing : /g/data/access/projects/access/apps/pythonlib/mule_python3/2026.03.2/mule_lib_intel_python3/um_packing/__init__.py (version 2025.10.1) (packing lib from SHUMlib: 2024111) %%%%%%%%%%%%%%%%%%%%%%%%%%%%% * CUMF-II Comparison Report * %%%%%%%%%%%%%%%%%%%%%%%%%%%%% File 1: /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.10.20/modern/scen7-h/atmosphere/forcing/global.N96/2026.10.20/ozone_h_2022_2150_cmip7.anc File 2: /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.10.09/modern/scen7-h/atmosphere/forcing/global.N96/2026.10.09/ozone_h_2022_2150_cmip7.anc Files DO NOT compare * 0 differences in fixed_length_header (with 7 ignored indices) * 59128 field differences, of which 59128 are in data Compared 59736/59736 fields, with 608 matches Maximum RMS diff as % of data in file 1: 45.846706180299726 (field 17917) Maximum RMS diff as % of data in file 2: 37.970776184645203 (field 17917) /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.10.20/modern/scen7-hl/atmosphere/forcing/global.N96/2026.10.20/ozone_hl_2022_2150_cmip7.anc %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% * (CUMF-II) Module Information * ... %%%%%%%%%%%%%%%%%%%%%%%%%%%%% * CUMF-II Comparison Report * %%%%%%%%%%%%%%%%%%%%%%%%%%%%% File 1: /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.10.20/modern/scen7-hl/atmosphere/forcing/global.N96/2026.10.20/ozone_hl_2022_2150_cmip7.anc File 2: /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.10.09/modern/scen7-hl/atmosphere/forcing/global.N96/2026.10.09/ozone_hl_2022_2150_cmip7.anc Files DO NOT compare * 0 differences in fixed_length_header (with 7 ignored indices) * 59546 field differences, of which 59546 are in data Compared 59736/59736 fields, with 190 matches Maximum RMS diff as % of data in file 1: 48.705590113357935 (field 32054) Maximum RMS diff as % of data in file 2: 39.549670034255882 (field 34488) /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.10.20/modern/scen7-l/atmosphere/forcing/global.N96/2026.10.20/ozone_l_2022_2150_cmip7.anc %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% * (CUMF-II) Module Information * ... %%%%%%%%%%%%%%%%%%%%%%%%%%%%% * CUMF-II Comparison Report * %%%%%%%%%%%%%%%%%%%%%%%%%%%%% File 1: /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.10.20/modern/scen7-l/atmosphere/forcing/global.N96/2026.10.20/ozone_l_2022_2150_cmip7.anc File 2: /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.10.09/modern/scen7-l/atmosphere/forcing/global.N96/2026.10.09/ozone_l_2022_2150_cmip7.anc Files compare * 0 differences in fixed_length_header (with 7 ignored indices) * 0 field differences, of which 0 are in data Compared 59736/59736 fields, with 59736 matches /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.10.20/modern/scen7-ln/atmosphere/forcing/global.N96/2026.10.20/ozone_ln_2022_2150_cmip7.anc %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% * (CUMF-II) Module Information * ... %%%%%%%%%%%%%%%%%%%%%%%%%%%%% * CUMF-II Comparison Report * %%%%%%%%%%%%%%%%%%%%%%%%%%%%% File 1: /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.10.20/modern/scen7-ln/atmosphere/forcing/global.N96/2026.10.20/ozone_ln_2022_2150_cmip7.anc File 2: /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.10.09/modern/scen7-ln/atmosphere/forcing/global.N96/2026.10.09/ozone_ln_2022_2150_cmip7.anc Files compare * 0 differences in fixed_length_header (with 7 ignored indices) * 0 field differences, of which 0 are in data Compared 59736/59736 fields, with 59736 matches /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.10.20/modern/scen7-m/atmosphere/forcing/global.N96/2026.10.20/ozone_m_2022_2150_cmip7.anc %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% * (CUMF-II) Module Information * ... %%%%%%%%%%%%%%%%%%%%%%%%%%%%% * CUMF-II Comparison Report * %%%%%%%%%%%%%%%%%%%%%%%%%%%%% File 1: /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.10.20/modern/scen7-m/atmosphere/forcing/global.N96/2026.10.20/ozone_m_2022_2150_cmip7.anc File 2: /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.10.09/modern/scen7-m/atmosphere/forcing/global.N96/2026.10.09/ozone_m_2022_2150_cmip7.anc Files DO NOT compare * 0 differences in fixed_length_header (with 7 ignored indices) * 57456 field differences, of which 57456 are in data Compared 59736/59736 fields, with 2280 matches Maximum RMS diff as % of data in file 1: 47.001714729616459 (field 24456) Maximum RMS diff as % of data in file 2: 38.683370000147818 (field 24456) /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.10.20/modern/scen7-ml/atmosphere/forcing/global.N96/2026.10.20/ozone_ml_2022_2150_cmip7.anc %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% * (CUMF-II) Module Information * ... %%%%%%%%%%%%%%%%%%%%%%%%%%%%% * CUMF-II Comparison Report * %%%%%%%%%%%%%%%%%%%%%%%%%%%%% File 1: /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.10.20/modern/scen7-ml/atmosphere/forcing/global.N96/2026.10.20/ozone_ml_2022_2150_cmip7.anc File 2: /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.10.09/modern/scen7-ml/atmosphere/forcing/global.N96/2026.10.09/ozone_ml_2022_2150_cmip7.anc Files compare * 0 differences in fixed_length_header (with 7 ignored indices) * 0 field differences, of which 0 are in data Compared 59736/59736 fields, with 59736 matches /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.10.20/modern/scen7-vl/atmosphere/forcing/global.N96/2026.10.20/ozone_vl_2022_2150_cmip7.anc %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% * (CUMF-II) Module Information * ... %%%%%%%%%%%%%%%%%%%%%%%%%%%%% * CUMF-II Comparison Report * %%%%%%%%%%%%%%%%%%%%%%%%%%%%% File 1: /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.10.20/modern/scen7-vl/atmosphere/forcing/global.N96/2026.10.20/ozone_vl_2022_2150_cmip7.anc File 2: /g/data/tm70/pcl851/CMIP7/esm1p6_ancil/2026.10.09/modern/scen7-vl/atmosphere/forcing/global.N96/2026.10.09/ozone_vl_2022_2150_cmip7.anc Files DO NOT compare * 0 differences in fixed_length_header (with 7 ignored indices) * 59736 field differences, of which 59736 are in data Compared 59736/59736 fields, with 0 matches Maximum RMS diff as % of data in file 1: 59.72826856171033 (field 25669) Maximum RMS diff as % of data in file 2: 45.253260318252501 (field 25669)

MartinDix · 2026-10-14T01:00:56Z

Aerosols look ok

SO2

Black carbon

Organic carbon

Biomass burning

blimlim · 2026-10-22T06:22:33Z

Thanks @paul-leopardi for rerunning the h, m, l, and vl ozone with the 2-0 zmta file. I've been looking into the reported differences between the two versions for the h scenario.

Differences in zmta files

As noted on the input4mips page:

The meaning is unchanged from the v1.0 files. However values which were set to NaN in the v1.0 files because they were below the Earth's surface have been set to the surface values in v2.0. Please be aware of this when processing the files.

Other than the differences in missing values, the 2-0 and 1-0 zmta files have different values at some of the points near antarctica, and the differences can be quite large:

Above ~600hpa and north of -50 degrees, the two files are identical. As a result, the differences should only affect the resulting ozone above Antarctica.

Differences in heights for interpolation

The UKMO code uses the zmta data to estimate the altitude of the pressure levels in the ozone dataset, roughly by approximating the Δh of each level, and sequentially adding them up:

https://github.com/ACCESS-NRI/ancillary-file-science/blob/78264a2b739ae3a7c8f9fa540b594583e7261c4d/Projects/CMIP/CMIP7/ozone_ancil/site/nci-gadi/ozone_cmip6_ancillary_for_suite.py#L388-L426

In the first step, it calculates the heights of the level boundaries:

        z[:, i + 1, :] = z[:, i, :] + dlogp[i] * integrant.data[:, i, :]

Where integrant $\propto$ zmta. Because it's accumulating over the levels, I thought that an NaN in at one level would propagate to the rest of the column above it, but from inspecting the output values this is not the case, and z itself is not NaN anywhere:

    print("z")
    print(type(z))
    print(np.isnan(z).sum())

> z
> <class 'numpy.ndarray'>
> 0

It turns out z is a numpy.ndarray, while integrant.data is a np.ma.masked_array and that inserting values from a masked array into a np.ndarray doesn't preserve the masking. Printing out values from the script, z ends up as 0.0 at points where integrant.data is masked.

As a result, the 1-0 zmta data gives an altitude of zero at the surface of Antarctica, while the zmta calculates the altitude using a constant temperature below the surface (I'm not sure what's better here).

The following plots compare the z2_pressure_height from the ozone script for the 2-0 zmta data and the 1-0 data. This is the height calculated for the ozone data's pressure levels, after it has been interpolated to the ESM1.6 grid:

The different treatment of the underground levels leads to offsets up to 2000m over Antarctica.

Differences in resulting ozone forcings

As the calculated pressure level heights are used to interpolate the ozone data, the differences affect the final ozone ancillaries.

The following compares the 2100 total column ozone in the input4MIPs data, the ESM1.6 ancillary produced from the zmta-1-0 data and the ancillary produced from the zmta-2-0 data.

The differences can be quite large over the south pole. Currently working with @MartinDix to figure out how to best process the remaining datasets.

paul-leopardi · 2026-10-22T07:43:39Z

@blimlim @MartinDix It may be best to use the zmta-2-0 forcings for all scenarios, for the sake of accuracy and consistency.

paul-leopardi · 2026-10-26T23:09:01Z

See PCMDI/input4MIPs_CVs#490 for the request for 1.0 zmta datasets.

paul-leopardi self-assigned this Oct 9, 2026

paul-leopardi linked an issue Oct 9, 2026 that may be closed by this pull request

Produce ancillaries for the ACCESS-ESM1.6 ScenarioMIP l, ln and ml experiments #166

Open

paul-leopardi requested review from MartinDix and blimlim October 9, 2026 05:21

penguian added this pull request to stack #168 October 12, 2026 03:27

paul-leopardi mentioned this pull request Oct 12, 2026

Add docstrings to all Python functions in esm1p6_ancil #161

Open

paul-leopardi added 2 commits October 13, 2026 10:05

Add the l, ln and ml scenarios

77ffbd8

Revise the PBS time, cpu and memory allocations

f515703

paul-leopardi force-pushed the 166-scen7-l-ln-and-ml branch from f6bc441 to f515703 Compare October 12, 2026 23:06

blimlim reviewed Oct 13, 2026

View reviewed changes

paul-leopardi added 2 commits October 20, 2026 11:30

For greenhouse gases, extend ScenarioMIP ext to 2151

ffb0adb

Try using the 2-0 zmta files for h, hl, m and vl

f822fa8

penguian removed this pull request from stack #168 October 22, 2026 03:48

paul-leopardi added this pull request to stack #173 October 22, 2026 03:57

paul-leopardi mentioned this pull request Oct 26, 2026

Update ESM1.6 nitrogen deposition to FZJ-CMIP-nitrogen-2-0 #176

Open

This was referenced Oct 27, 2026

ScenarioMIP ozone should use consistent zmta datasets #178

Open

Add the esm-flat10-cdr CO₂ emissions ancillary to the Cylc 8 suite #180

Open

Refactor the ozone stage-1 pipeline, starting with a validation and benchmark notebook #184

Open

Navigation Menu

Repository navigation

Include the l, ln and ml scenarios and tighten PBS requests - #167

Include the l, ln and ml scenarios and tighten PBS requests#167
paul-leopardi wants to merge 4 commits into
mainfrom
166-scen7-l-ln-and-ml

paul-leopardi commented Oct 9, 2026 •

edited

Loading

Uh oh!

paul-leopardi commented Oct 9, 2026

Uh oh!

MartinDix commented Oct 12, 2026

Uh oh!

MartinDix commented Oct 13, 2026

Uh oh!

MartinDix commented Oct 13, 2026

Uh oh!

blimlim commented Oct 13, 2026

Uh oh!

blimlim Oct 13, 2026

Uh oh!

paul-leopardi Oct 20, 2026

Uh oh!

paul-leopardi Oct 20, 2026

Uh oh!

MartinDix commented Oct 14, 2026

Uh oh!

blimlim commented Oct 22, 2026

Uh oh!

paul-leopardi commented Oct 22, 2026

Uh oh!

paul-leopardi commented Oct 26, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

paul-leopardi commented Oct 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

paul-leopardi commented Oct 9, 2026

Uh oh!

MartinDix commented Oct 12, 2026

Uh oh!

MartinDix commented Oct 13, 2026

Uh oh!

MartinDix commented Oct 13, 2026

Uh oh!

blimlim commented Oct 13, 2026

Uh oh!

blimlim Oct 13, 2026

Choose a reason for hiding this comment

Uh oh!

paul-leopardi Oct 20, 2026

Choose a reason for hiding this comment

Uh oh!

paul-leopardi Oct 20, 2026

Choose a reason for hiding this comment

Uh oh!

MartinDix commented Oct 14, 2026

Uh oh!

blimlim commented Oct 22, 2026

Differences in zmta files

Differences in heights for interpolation

Differences in resulting ozone forcings

Uh oh!

paul-leopardi commented Oct 22, 2026

Uh oh!

paul-leopardi commented Oct 26, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

paul-leopardi commented Oct 9, 2026 •

edited

Loading