Apply Miles CLI overrides after all argument groups are registered

[kevintli]

Summary

Fixes an issue with CLI arg ordering that prevented us from appending Miles args like --qkv_format, which we needed for our gpt-oss-20b Spindle runs.

Details

Due to a quirk with how Miles parses CLI args, the effective ordering for Spindle CLI args is:

• Megatron builds the parser and registers its own args
• It calls add_miles_arguments, which first runs Spindle's add_custom_arguments hook before registering its own args. It's unclear exactly why, but potentially Miles does this because it wants to prevent users' hook from unintentionally overriding its own arg definitions.

However, our add_custom_arguments hook relies on whatever CLI options have been registered by that time in order to perform validation, so it fails if the user tries to pass Miles args which haven't been registered by that time.

The workaround is to change our setup to defer_config_overrides, which applies the configs only after all CLI options are registered.

Testing

Verified inside the actual trainer image (radixark/miles:v0.1.0 plus the pinned Megatron) with the gpt-oss cli_options. Previously this failed on qkv_format. It now parses to qkv_format=thd, attention_backend=fused, moe_permute_fusion=False, recompute_*=full/uniform/1. Added a unit test covering options that are registered after the hook runs.

Stacked on #4.

Link to Devin session: https://modal.devinenterprise.com/sessions/f53cfabb210146de8f0338fe388d7973
Open in Devin Desktop: https://modal.devinenterprise.com/desktop/session/f53cfabb210146de8f0338fe388d7973?variant=devin
Requested by: @kevintli

[devin-ai-integration]

I'll fix CI failures and address comments from users with write access that start with 'Devin'.

• Disable automatic comment, CI, and merge conflict monitoring