Chronos combines recurrent computation, sparse expert dispatch and out-of-core weight residency. These mechanisms interact through explicit tensor and cache contracts. Parameter reuse, cache reuse and weight prefetching solve different resource problems.
The diagrams are schematic. Configuration symbols describe a family of models, rather than a fixed-size checkpoint. SYMBOLS.md defines the notation and tensor axes. Implementation claims link to source; REPRODUCIBILITY.md separates measured results from conditional analysis.
Figure K1. Forward execution and supported block interiors. Input lookup and output vocabulary projection are separate operations. Weight tying can make their matrices identical. MLA, multi-head latent attention; SWA, sliding-window attention; MoE, mixture of experts; FFN, feed-forward network; KV, key–value. The four interior cards are configuration alternatives, not four mandatory serial layer types.
Let the physical backbone contain
flowchart TB
subgraph IN["Input lane"]
IDS["Integer token IDs"] --> EMB["Input embedding W_in"]
end
subgraph FWD["Forward stack"]
PRE["Prelude: P ordinary blocks"] --> CORE["Recurrent region: G physical<br/>blocks"] --> CODA["Coda: C ordinary blocks"]
CORE -. "selected recurrence order" .-> CORE
end
subgraph OUT["Output lane"]
NORM["Final RMSNorm"] --> HEAD["Output matrix / linear LM head"] --> LOGITS["Vocabulary logits"]
end
EMB --> PRE
CODA --> NORM
EMB -. "optional shared matrix" .-> HEAD
PRE -. "post-prelude state, R greater<br/>than 1" .-> LOOK["Future expert predictor"]
LOOK -. "expert IDs / requests" .-> IO["Weight-residency runtime"]
The input operation is a lookup. The output operation is a linear projection:
Here hidden vectors use a column-vector convention. Actual batch code applies the equivalent row-vector multiplication. If tie_word_embeddings=True, the two weights share parameters. Drawing two lanes does not create two independent parameter copies.
The implementation returns logits. Sampling, probability normalization and a training loss consume those logits through separate paths. An encoder–decoder output-token embedding or cross-attention stack is not part of this decoder-only graph. See backbone construction and LM-head construction and forward.
When recurrence is disabled or configured depth is one, the model runs the ordinary physical stack. The recurrent controller becomes a no-op. Lookahead then observes the output of physical block zero. For recurrent execution, lookahead observes the post-prelude state, including the embedding state when
These observation points are implemented in the forward dispatch. They should not be inferred from the name “layer-aware lookahead.”
Figure K2. Full pre-normalization residual circuits. Every card contains attention and an MLP. The attention family depends on physical layer parity; the MLP family depends on a model-wide switch. Orange dashed links carry router gates; solid links carry hidden states. Shared experts contribute their mean output. MLA, multi-head latent attention; SWA, sliding-window attention; MLP, multilayer perceptron; RMSNorm, root mean square normalization.
For a full block
The two residual additions have different input states. Attention updates first_only recurrent reader iterations, the attention sublayer is bypassed and the MLP operates on the current recurrent state. It does not repeatedly reuse a frozen attention output as the entire hidden state.
The block forward implements these paths. No full block type consists only of MoE without an attention module.
flowchart TD
CFG["Model configuration"] --> HYB{"Hybrid attention enabled?"}
HYB -->|"no"| GQA["Global grouped-query attention in every layer"]
HYB -->|"yes"| PAR{"Physical layer index parity"}
PAR -->|"even"| MLA["Global MLA"]
PAR -->|"odd"| SWA["Local SWA"]
CFG --> MOE{"use_moe"}
MOE -->|"false"| DENSE["Dense gated FFN; SiLU default"]
MOE -->|"true"| EXP["Routed plus shared gated FFNs; SiLU default"]
Physical parity includes the prelude offset. Recurrent group
For the default SiLU activation, a bias-free SwiGLU expert with intermediate width
Its three matrices contain
For routed expert probabilities
With norm_topk_prob=True,
The Torch selected-row path and supported MLX grouped sparse path reduce routed expert matrix work. For width
The implemented contracted step is indexed by recurrent group and iteration. It is not a single interpolation applied after the whole recurrent stack.
The learned vectors shared_first,
flowchart TB
H["Current state h"] --> ADD["Add iteration and group embeddings"] --> B["Physical ChronosBlock F"]
H --> DELTA["Subtract unconditioned h"]
B --> DELTA
DELTA --> SCALE["Multiply by learned alpha for<br/>this group"] --> NEXT["Add to h"]
H --> NEXT
KV["Writer iteration KV view"] -. "cache policy" .-> B
For block order, one cycle composes the
If conditioning and cached context are fixed, a single group update has the explicit-Euler form for a vector field
For fixed
A bound below one is sufficient for contraction under these assumptions. Bounding
| Authority | What it determines | Interpretation |
|---|---|---|
ChronosConfig class |
Fallback values for unspecified fields | The inspected defaults include recurrent_order="layer" and recurrent_dt_param="clamp". |
| Planner or tuning recipe | A recommendation for a requested budget | Recommendations can select block order and sigmoid parameters. |
| Saved checkpoint configuration | The effective graph and parameter interpretation after fallback resolution | A missing key inherits the class default; a recommendation does not rewrite old weights. |
This distinction matters for both inference reproducibility and interpretation of benchmark arms. See configuration resolution.
The model distinguishes a cache retained across token forwards from the shared attention view used inside one recurrent forward.
sequenceDiagram
participant W as Writer iteration
participant P as Persistent token cache
participant S as Shared view for current forward
participant R as Reader iterations
W->>P: Append the new token or chunk once
W->>S: Return keys/values visible to this chunk
R->>S: Fresh queries attend to writer view
Note over R,P: shared_first readers do not append again
Note over P: SWA may evict old entries after forming the shared view
For MLA, the persistent tuple is normally (c_kv,). If mla_cache_rope=True, it additionally stores rotated rotary keys with shape
Let mla_cache_rope=False uses
With persisted rotary keys, it uses
SWA retains keys and values. For retained length
in bounded decode. A shared prefill view can be longer than the retained cache. The implementation concatenates and slices tensors; a physical ring-buffer allocator should not be inferred. Int8 caches also require scales and metadata, so equations (K.7–K.8) are payload baselines.
Figure K3. Routing-locality mechanism. The shared branch and selected routed experts receive hidden states independently. The optional loss encourages within-stream cluster concentration and across-stream diversity over a fixed expert-ID partition. Node positions are illustrative, not measured high-dimensional embeddings. An offline coactivation graph supplies a separate optional layout route.
Define expert-to-cluster membership
Here
Low conditional entropy concentrates each stream's page demand. High marginal entropy discourages every stream from using one cluster. Segment mode uses separator-derived document segments and weights them by valid-token count when a token mask is supplied. Without a mask, all positions count. If input IDs are absent, the Torch trainer uses fixed windows. Degenerate cluster counts or insufficient stream units produce no useful locality signal. CLUSTER_LOCALITY.md derives the objective and the weighted segment implementation.
flowchart LR
FIX["Fixed training ID partition"] --> LOSS["Optional routing-locality objective"] --> ROUTE["Learned expert routing"]
FIX --> PACK["Training-aligned inference page layout"]
TRACE["Calibration activation trace"] --> GRAPH["Expert coactivation graph"] --> COMMUNITY["Louvain or greedy partition"] --> ALT["Optional repacked page layout"]
PACK --> STORE["Expert store"]
ALT --> STORE
The fixed partition does not move expert weight vectors in a learned geometric embedding. Offline graph communities depend on observed coactivation rather than measured spatial distances between parameters. Neither mechanism demonstrates spontaneous expert migration. The two layout routes should not be presented as an obligatory sequential pipeline.
Figure K4. Storage and runtime control. The host-to-device lane describes Torch/CUDA. MLX uses evaluated arrays in unified memory. The requested-page filtering guarantee belongs to the Torch reader; MLX can materialize a whole cluster transiently before bounded warm admission. Solid links describe weights; dashed links describe requests and policy. SSD, solid-state drive; RAM, random-access memory; H2D, host-to-device; LRU, least recently used.
A .ctsr file is a safetensors-backed cluster container in the inspected implementation. cluster_manifest.json maps experts and clusters to files and can record layer/group ownership. See manifest fields and reader implementation.
flowchart TD
SEL["Current selected expert"] --> HOT{"Device expert materialized?"}
HOT -->|"yes"| RUN["Execute exact expert"]
HOT -->|"no"| MODE{"Demand loader active?"}
MODE -->|"fallback_diagnostic: no"| DIAG["Use diagnostic shared fallback path"]
MODE -->|"on_demand or sync_on_demand: yes"| RAM{"Page warm in RAM?"}
RAM -->|"yes"| PROMOTE["Promote and wait for exact weights"] --> RUN
RAM -->|"no"| POL{"Cold SSD miss policy"}
POL -->|"on_demand"| QUEUE["Queue exact missing page"]
QUEUE --> FALL["Use shared fallback for this dispatch"]
POL -->|"sync_on_demand"| WAIT["Load and wait for exact expert"] --> RUN
The normal shared residual remains active in all these paths when shared experts exist. Substitution for a missing routed expert is additional to that residual. Its deviation from the full-resident result depends on missing routing mass and the difference between routed and shared outputs; no measured universal quality bound is established by the presence of shared experts.
For a missed set
If
| Operation | Torch/CUDA path | MLX path |
|---|---|---|
| Loading | Safetensors reader filters chosen layer/expert pages before get_tensor |
Cluster loading can evaluate a whole file before admission |
| Warm weights | Host cache, pinned when the backend supports it | Evaluated MLX arrays in unified memory |
| Promotion | Materialization and H2D copies, with per-expert event waits | Bind already evaluated arrays; no separate CUDA H2D copy |
| Queue | Bounded Python FIFO; oldest queued request can be dropped | Separate MLX prefetch implementation; inspect its contracts |
| Residency | LRU and protected selected pages | LRU/window policy or optional next-use planner |
Bounded cache admission does not automatically bound transient materialization or process peak memory. Request count, expert-page count, cluster-file count and bytes read are different measurements. RESIDENCY.md defines their accounting scope.
The lookahead head reads a backbone state and predicts expert distributions for the current and future token positions. With
The group axis indexes recurrent physical groups; the horizon axis indexes future tokens. The head is a two-linear-layer MLP with SiLU, not a separate language model that proposes tokens. Its output feeds an expert-request scheduler. Source: Torch head and MLX head.
sequenceDiagram
participant F as Backbone forward
participant H as Lookahead head
participant Q as Request queue
participant S as Expert store
participant N as Next token dispatch
F->>H: Post-prelude hidden state in recurrent execution
H->>Q: Predicted expert IDs for future token horizons
Q->>S: Warm selected weights asynchronously
N->>S: Request actual selected experts
S-->>N: Resident weights or policy-dependent cold-miss handling
Teacher routing distributions are detached for the prediction loss. Detaching the lookahead input is a different option, with a backend-specific implementation boundary documented in LOOKAHEAD.md. A predicted expert set does not prove that every next-token dispatch is ready.
| Quantity | Shape | Precision/storage contract |
|---|---|---|
| Token IDs | Integer input, backend-specific tensor type | |
| Hidden state | Model compute dtype; not universally bf16 | |
| Torch current-router scores | Softmax explicitly evaluated in FP32 | |
| Lookahead probabilities | Eq. (K.12) | Torch inherits head-logit dtype; MLX explicitly uses FP32 logits |
| MLA persistent latent | Compute dtype or configured quantized representation | |
| Optional persisted MLA rotary keys | Separate cache component, not concatenated scalar width | |
| SWA persistent K and V | Two tensors |
Bounded retained length; absolute position tracked separately |
| Returned LM logits | Torch defaults to FP32 on loss/training paths and compute dtype for inference | |
| Chunked MLX CE working logits | Row chunks by vocabulary | Used on the dedicated training-loss path; full logits remain available on other paths |
The ordinary Torch ChronosForCausalLM.forward materializes returned logits and applies cross-entropy when labels are supplied. Setting ce_chunk_size does not rewrite that forward into the MLX chunked-loss path. The vocabulary matrix also remains a
The implementation supports the mechanisms above. Their benefits require workload-specific measurement. A complete experimental argument needs all three checks:
- Execution: the selected configuration follows the expected tensor, cache and page transitions.
- Equivalence or approximation: compare logits or generated tokens under the relevant exactness assumptions and miss policy.
- Resource benefit: record prefill, decode, materialization, bytes/counters, peak memory and output quality under matched conditions.
The existing records establish selected small-model results, not an unconditional performance or quality guarantee for the entire architecture family. The reproducibility chapter gives the measurement ledger and missing controls. The theory chapter states which mathematical conclusions require additional assumptions.



