An experiment exploring the use of Sparse Predictive Hierarchies (SPH) as a policy/value head for a chess engine (Lc0).
To explore whether local learning rules, potentially capable of online learning, combined with a biologically inspired architecture, could handle chess evaluation and policy prediction without backpropagation. The ultimate goal is to enable online learning and extreme sparsity, allowing larger models to run on modest hardware.
Inspired by AOgmaNeo, the architecture features an ART-based encoder. However, instead of mapping each position to a single category, it maps each local receptive field to a single cell (a.k.a. category) in every column.
Following most of Ogma Corp.'s implementations (see the official whitepaper), the architecture uses a Dendritic Neural Layer as a decoder.
Activation function: Leaky ReLU
A standard SPH model acts as the main feature extractor, providing the heads with a feature representation of what the board should look like after the best move. The policy head must then identify which legal move produces a board state most similar to this "better board representation," while the value head converts the representation into a Win-Draw-Loss (WDL) probability distribution.
The heads are also DNNs, but their architecture is closer to that of standard DNN layers than to that of SPH decoders. However, they inherit much of the decoders' logic, making them very similar to decoders at the implementation level.
Instead of feeding the common body and heads directly with two visible input layers, the inputs first pass through an encoder so that the decoder only has to work with a single visible layer.
Exponential memory was deliberately omitted for several reasons:
- To save computation time and memory.
- To avoid synchronization issues with Lc0's MCTS.
- To simplify the model's architecture and codebase.
Weights are stored as u8 instead of f32 to save memory and increase SIMD throughput. To achieve this in decoders, stochastic rounding is applied to the gradients during training.
A major performance concern with sparse models is their poor cache efficiency. At the cache-line level, memory access and modification resemble random accesses to a very large array that does not fit in the cache. Consequently, the encoder frequently exhibits cache-miss rates of approximately 30%, making it one of the least suitable operations for the von Neumann architecture and the model's primary computational bottleneck.
However, even though cache locality remains unoptimized, the time saved by skipping large numbers of operations makes the trade-off worthwhile.
Flamegraphs and perft measurements show that, during the encoder's forward and learning passes and the decoder's forward pass, Rayon's scheduling overhead exceeds the performance gains from parallelization. Therefore, parallelization is used only for the decoder's learning pass and the heads' learning and forward passes.
The input consists of two layers:
- Board layer (8×8×13): Contains the piece placement on the board.
- Auxiliary layer (1×6×16): Contains additional positional information, including castling rights, side to move, and the halfmove clock.
-
Number of layers: 3 (1 bottom layer + 2 body layers)
-
Pipeline / Grid Dimensions (
PIPELINE_SIZES):(8, 8, 32)across all 3 layers -
Local Receptive Field Size (
radius):- Bottom layer: 1x1 (
radius = 0) - Body layers: 3×3 (
radius = 1)
- Bottom layer: 1x1 (
-
Vigilance: (tuned in
src/bin/tune_hyperparams.rs)- Bottom layer:
0.8703 - Body layers:
0.8
- Bottom layer:
-
Learning Rates:
- Bottom layer: Encoder LR =
0.2, Policy LR =0.02, Value LR =0.02 - Body layers (Layer 0 / Layer 1): Encoder LR =
0.1, Decoder LR =0.02/0.01
- Bottom layer: Encoder LR =
-
Other Training & Validation Constants:
- Choice parameter:
1e-4 - Active ratio:
1.0(tuned insrc/bin/tune_hyperparams.rs) - Lambda:
0.25(meaning that the value head target WDL is a blend 25% game outcome 75% MCTS q-value) - Validation samples:
200(only use a small random part of the dataset at a time for validation) - Validation frequency: Every
1000steps (VALIDATE_EVERY) - Logging frequency: Every
100steps (LOG_EVERY)
- Choice parameter:
See src/hyperparams.rs for the complete list of hyperparameters and settings.
- Parameter count: 44,896,256
Although SPHs thrive in online-learning contexts, the primary goal was to measure their predictive performance in chess. To save computation time, the model was trained using supervised learning on chunks of the Lc0 test80 dataset: 10K games for training and 2K games for validation. Each validation pass sampled 200 games from the validation set.
The same training dataset and methodology were used for the control group.
See the Sparse Chess Control Group Script.
Note: Batch size: 256.
Unfortunately, the results suggest that the hypothesis was incorrect. This type of model underperforms compared with architectures such as ResNets, Transformers and even plain CNNs in terms of prediction accuracy. There are two possible explanations:
- Limitations of local learning rules. Local learning rules may be less effective for feature extraction than backpropagation, which optimizes parameters directly against a shared objective: minimizing the loss on the dataset. Local learning rules instead optimize parameters according to other criteria, such as topological similarity in Fuzzy ART. This limitation might be mitigated in the encoders by using Gaussian ARTMAP.
- Information loss due to strict top-1 WTA and CSDR. This is a trade-off made in favor of speed, preserving approximately 99% sparsity. However, experimenting with top-4 CSDR representations could be worthwhile. This would reduce sparsity to approximately 96%, preserving 4 times more information with just a 4% sparsity reduction, although this estimate is rough.
To replicate the experiment, first download one of the chunks from the Lc0 Training Data Files. Specifically, I used one of the test80 chunks for this experiment.
Just split the .gz files into different directories. You can automatically download and split the dataset with the following script:
bash split_dataset.shIt will create a training-dataset directory with training-dataset/train and training-dataset/val containing the .gz chunks. You can change the script config directly in its source, at the very beginning in the Configuration section.
cargo run --release --bin train -- <train dataset path> <val dataset path> <model path> <mlflow uri>MIT License. See LICENSE for more details.