Skip to content

Allow a relative tolerance as fof rule - #121

Draft
muellch wants to merge 1 commit into
MeteoSwiss:mainfrom
muellch:fof_rule_tolerance
Draft

muellch wants to merge 1 commit into
MeteoSwiss:mainfrom
muellch:fof_rule_tolerance

Conversation

@muellch

@muellch muellch commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

[Claude-generated] Drafted by Claude on Christoph's behalf.

Why. ICON's fof check compares every variable except veri_data exactly. plevel and e_o of PILOT and SYNOP come from the model state, so CPU and GPU runs agree only to the last float32 bit, and SYNOP plevel in mixed precision to about 1e-5. The GPU bug fixed in icon-nwp!2385 shifted them by 3e-4 to 1e-3, so a small tolerance separates noise from a real error.

What. A number in --rules is now the admitted relative difference |ref - cur| / (1 + |ref|) for that column, not scaled by --factor; a list keeps its meaning. Two small fixes come along: values missing in both files pass a rule column, and a rule violation in the reports is no longer overwritten by the observation check.

Test. New unit tests in tests/util/test_fof_rules.py. On the full ICON mch_kenda-ch1 fof files with "plevel": 5e-5, "e_o": 1e-6, the fixed GPU run passes AIREP, DRIBU, PILOT and SYNOP against CPU in double and mixed precision; the GPU run with the bug still fails PILOT and SYNOP.

🤖 Generated with Claude Code

A number in --rules is the admitted relative difference |ref - cur| / (1 + |ref|)
for that column, not scaled by --factor. It is meant for plevel and e_o, which
GPU and CPU runs of ICON reproduce only to the last float32 bit, or to about
1e-5 for SYNOP plevel in mixed precision. Values missing in both files now pass
a rule column, and a rule violation in the reports is no longer overwritten by
the observation check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@huppd

huppd commented Nov 13, 2026

Copy link
Copy Markdown
Collaborator

Thanks for pointing out this short coming of the current implementation. Nevertheless I think it is not very clean to mix tolerances with the rules for labels.
My suggestion would be, to add --fof-tolerance-vars (default veri_data) to tolerance and fof_compare, and write one column per variable to the tolerance CSV. check can then read the variable list from the CSV columns and leave those variables out of the exact comparison, so check and select_members need no new option. QC fields (state/flags/check) still go through --rules, and sort keys (lat/lon/level/…) stay exact.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants