Skip to content

Feat: pixml loader for rfc forecasts - #254

Open
msweier wants to merge 31 commits into
mainfrom
feat/pixml
Open

msweier wants to merge 31 commits into
mainfrom
feat/pixml

Conversation

@msweier

@msweier msweier commented Aug 7, 2026

Copy link
Copy Markdown
Collaborator

Use cwms-cli nws pixml to load an NWS/RFC Delft-FEWS PI-XML forecast product into a CWMS database. Parameter mapping, timeseries group overrides, versioning, and issued-time tracking is driven by a JSON config file or a config blob stored in CDA.

nws pixml supports:

  • loading a single PI-XML file or URL (.gz/.zip auto-unzipped)
  • config-driven NWS to CWMS parameter mapping
  • TSID resolution via timeseries group alias override with optional built fallback
  • run selection by filename pattern (e.g. base / auto / CRF)
  • per-run versioning control (versioned or unversioned)
  • issued-time tracking via a consolidated JSON blob

This was based on MVP and MVM's needs and tested on the MVP and MVM RFC products with the hope that it may be configurable for other district's needs.

Developed with Claude Opus.

@msweier
msweier marked this pull request as draft August 7, 2026 20:08

@krowvin krowvin left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A lot of acronyms. I'm not familiar with pixml. Might consider spelling the first ones out or linking to external docs more too.

Commenting for now, give Eric some time to peak this if he can.

Comment thread docs/cli/nws_pixml.rst Outdated
Comment thread docs/cli/nws_pixml.rst Outdated
Comment thread docs/cli/nws_pixml.rst Outdated
Comment thread docs/cli/nws_pixml.rst
Comment thread docs/cli/nws_pixml.rst Outdated
@msweier
msweier marked this pull request as ready for review August 26, 2026 17:56
@msweier
msweier requested review from krowvin August 26, 2026 19:07

@krowvin krowvin left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only thing I really want to confirm before approval is our use of options, flags, and sub commands.

Typically something like
-i is a short option
--input is a long option
cwms-cli nws is a sub command of cwms-cli

In the source it is reading like
cwms-cli nws input ...etc is a valid option/arg.

I'd say the rest is largely up to you to decide if you wish to make changes, just let me know your thoughts in the replies.

That and if Mike has any opinions on that massively negative number. To me that looks like it's a 128bit signed number. Suggesting if you go to the bottom max of a 128bit signed value that's pretty much zero?

Comment thread cwmscli/nws/load_pixml.py
logger = logging.getLogger(__name__)

# CWMS sentinel for a missing value and its quality code.
CWMS_MISSING_VALUE = -340282346638528859811704183484516925440

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is the java max float, right Mike. Probably would be best to have this in cwms-python or have CDA support?

Comment thread cwmscli/nws/load_pixml.py Outdated
Comment thread cwmscli/nws/load_pixml.py
Comment thread cwmscli/nws/__init__.py Outdated
Comment thread cwmscli/nws/__init__.py Outdated
Comment thread docs/cli/nws_pixml.rst Outdated
Comment thread docs/nws/mvm.example.json
Comment thread docs/nws/mvm.example.json
Comment thread docs/nws/mvp.example.json
Comment thread docs/nws/mvp.example.json
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants