Make GemNet scale-file configuration portable

[Daniel Zügner (danielzuegner)]

Summary

• make scale_file an explicit GemNet parameter with a package-relative default
• honor custom scale-file paths instead of silently discarding them
• preserve compatibility with cached checkpoint configs containing the legacy internal /scratch/amlt_code path
• remove the internal path from repository checkpoint and training configs
• add regression coverage for default, custom, and legacy paths

The corresponding Hugging Face checkpoint configs should be refreshed when this change is released.

Fixes #257

[Daniel Zügner (danielzuegner)]

Additional compatibility checks:

• Training path: constructed GemNet without a scale_file override, loaded the packaged gemnet-dT.json, ran a forward pass on the bundled MP-20 debug batch, backpropagated the energy loss, and completed an Adam optimizer step on CPU (loss=0.275989).
• Inference path: loaded the published mattergen_base checkpoint and its existing config containing scale_file: /scratch/amlt_code/mattergen/common/gemnet/gemnet-dT.json. The legacy-path fallback loaded the packaged scale file and completed generation of one six-site structure on CPU. Sampling used two aligned sampler/D3PM steps to keep the smoke run short.

The repository’s full one-batch trainer test could not start because this checkout does not have datasets/cache/mp_20/train/pos.npy; it failed while loading data, before constructing GemNet. GPU validation was intentionally skipped because the available A100 was already under another workload (96% utilization).

[Daniel Zügner (danielzuegner)]

Follow-up: I materialized and processed the MP-20 release into datasets/cache/mp_20 and reran the full trainer smoke. It completed sanity validation, one training batch with backward/optimizer update, and one validation batch using GemNet’s package-relative default scale file.

The current environment has a newer PyTorch where ReduceLROnPlateau(verbose=...) has been removed, so I deleted only that obsolete scheduler argument from the runtime-composed smoke config; no source changes were needed. This is independent of the scale-file change. Together with the published-checkpoint inference smoke above, both training and inference paths now complete.