[changwangss]

Description

Follow up on the separation-of-concerns feedback in #2340 and #2404. SignRound should not need to know about diffusion-specific cache configuration, CUDA staging, fallback branches, or cache resource management.

• Add a block-scoped BlockTuningSession, exposed through BaseQuantizer.tuning_session(), for iteration/batch access, prediction/reference forwarding, and best-parameter snapshots.
• Move cache eligibility into DiffusionTuningCache.is_supported() and move cache creation, staged/synchronous fallback, warmup timing, and storage policy into the session.
• Replace SignRound cache-specific branches with the common session interface. A context manager retains exception-safe cleanup across forward, backward, optimizer steps, and early exit.

Scope / behavior preservation

This is a responsibility-only refactor, not an expansion of prefetch support:

• Prefetch remains opt-in through diffusion_tuning_cache_size, diffusion-only, single-CUDA, low_gpu_mem_usage=True, and batch_size=1.
• Keep explicit-budget initialization at iteration 0 and auto-budget initialization after one complete synchronous optimizer iteration.
• Preserve sampler/RNG ordering, dual-buffer staging, fallback behavior, optional GPU best-parameter snapshots, and transient prediction-device routing.
• No new CLI/API settings, LLM/MLLM prefetch enablement, multi-GPU support, or larger-batch prefetch support.

Validation

• 130 targeted tests passed after rebasing onto the current upstream main, including configuration routing, DiffusionMixin, block runner, SignRoundV2 loss, and autocast coverage.
• New CPU-only session tests cover policy, warmup, sampling, staged/synchronous dispatch, snapshots, early exit, and error propagation/cleanup.
• New real-CUDA tests compare sample order, loss, gradients, snapshots, and final parameters against a legacy synchronous loop; cover worker/buffer cleanup and actual SignRound wrapper/optimizer/unwrapper execution without model downloads.
• All pre-commit hooks and git diff --check passed.
• Full tiny-FLUX pipeline testing was attempted offline but blocked during fixture setup by a missing local FLUX model snapshot. No full-pipeline or performance claim is made by this draft.

Type of Change

Refactor

Related Issues

Relates to #2340 and #2404.

Checklist Before Submitting

• My code has been tested locally.
• Documentation has been updated as needed (internal docstrings; no user-facing behavior/configuration change).
• New or updated tests are included where applicable.
• The CUDA CI has passed. Local CUDA tests pass; remote CI has not been triggered by this draft.