Add PCA-like structured latent spaces to autoencoders

[sergioald]

Summary

Adds optional structured latent-space regularization to BlueMath_tk
autoencoders, with the goal of providing a more interpretable,
PCA-like latent geometry while retaining nonlinear autoencoder models.

Three modes are supported:

• latent_structure="none" — preserves existing behavior.
• latent_structure="orthogonal" — encourages an orthonormal latent
  projection and decorrelated latent scores.
• latent_structure="pca_like" — adds ordered-prefix training so earlier
  latent coordinates are encouraged to retain more reconstruction utility.

The feature is integrated into:

• StandardAutoencoder
• LSTMAutoencoder
• CNNAutoencoder
• VisionTransformerAutoencoder
• ConvLSTMAutoencoder
• HybridConvLSTMTransformerAutoencoder
• SpatialTokenConvLSTMTransformerAutoencoder
• VariationalAutoencoder

The existing OrthogonalAutoencoder remains unchanged.

Mathematical formulation

Projection orthogonality is regularized using

mean((W W^T - I)^2).

Latent decorrelation uses the mean squared off-diagonal Pearson
correlations of the unmasked latent scores.

The calculation removes each coordinate's scale before normalization,
making the objective invariant to representable nonzero independent
coordinate rescaling.

For an exactly collapsed coordinate, Pearson correlation is undefined.
Pairs involving such coordinates receive the maximal squared-correlation
penalty of 1.0, preventing collapse from reducing the objective.

pca_like additionally uses stochastic ordered-prefix masking during
training. The regularization terms themselves are calculated from the
unmasked latent representation.

VAE behavior

For VariationalAutoencoder:

• structural penalties are evaluated on the unmasked posterior mean mu;
• KL divergence uses the unmasked mu;
• ordered masking is applied only to sampled decoder codes during training;
• deterministic encoding continues to return the unmasked posterior mean;
• evaluation remains unmasked.

Backward compatibility

• latent_structure="none" preserves the existing behavior.
• New latent options are keyword-only.
• Existing positional device arguments retain their historical position.
• Structured latent projections retain the original nn.Linear
weight/bias state-dict keys and do not add trainable parameters.
• Built checkpoint receivers explicitly reject conflicting saved latent
  configuration rather than silently retaining incompatible settings.

Diagnostics

Adds latent diagnostics including:

• per-coordinate variance;
• variance fractions;
• cumulative variance fractions;
• mean/max absolute off-diagonal correlation;
• variance-ordering violations;
• collapsed-coordinate count and indices.

Correlation diagnostics use the same scale-invariant Pearson semantics as
the training regularizer.

Validation

Local regression suite:

893 passed, 113 warnings

git diff --check: PASS

A fixed-configuration synthetic holdout used fresh seeds 47, 101 and 131,
separate from calibration seeds 2, 11 and 23.

Research configuration:

• latent_orthogonality_weight = 1.0
• latent_decorrelation_weight = 0.1
• latent_ordering_probability = 0.5

All predefined holdout criteria passed.

Mean holdout results:

Model	Test RMSE	Orthogonality error	Mean abs latent correlation	Prefix AUC
none	0.3053	0.3905	0.3311	2.5525
orthogonal	0.3212	0.0187	0.1285	2.2662
pca_like	0.3684	0.0160	0.1789	1.3881
PCA	0.2745	~0	0.2162	1.7621

Interpretation

“PCA-like” here refers to a row-orthogonal latent projection,
decorrelated latent scores, and ordered-prefix reconstruction utility.

It does not imply PCA variance optimality, explained-variance ordering,
principal-component recovery, or equivalence between a nonlinear
autoencoder and PCA.

The calibrated regularization values above are reference research
settings rather than universal defaults; dataset-specific work should
tune them using training/validation data only.