hdrnn - Hand-written Digit Recognition Neural Network in C

A self-contained, zero-dependency handwritten digit recognition neural network written in clean C11. It implements a complete feedforward Multilayer Perceptron (MLP) trained end-to-end on the MNIST dataset using Stochastic Gradient Descent (SGD) and backpropagation, with native binary IDX dataset decoding and an interactive terminal visualizer.

====================================================
 Demo: Sample #0 from Test Set
====================================================
            --##**++::..                                
            %%@@@@@@@@@@################**::            
            ::==--==**%%@@%%@@@@@@@@%%@@@@++            
                      ..::  ::::::::..%%@@==            
                                    --@@##..            
                                  ..%%@@--              
                                  ++@@%%::              
                                ::@@@@::                
                                ++@@##                  
                                ##@@::                  
                              ==@@**                    
                            --@@%%::                    
                          ..%%@@**                      
                          ##@@%%..                      
                        ..@@@@--                        
                      ..%%@@==                          
                      ++@@@@::                          
                    ::@@@@@@::                          
                    ==@@@@%%..                          
                    ==@@##..                            

Ground Truth: 7
Prediction  : 7 [MATCH]
Loss        : 0.001736

Class Probabilities:
  [0]   0.0% |                              |
  [1]   0.0% |                              |
  [2]   0.0% |                              |
  [3]   0.1% |                              |
  [4]   0.0% |                              |
  [5]   0.0% |                              |
  [6]   0.0% |                              |
  [7]  99.8% |##############################| <-- Correct
  [8]   0.0% |                              |
  [9]   0.1% |                              |
====================================================

---

Features

• Pure C11 & Zero External ML Dependencies: Implemented entirely from scratch using only standard C libraries (libc, libm).
• Network Architecture: 3-layer fully connected MLP:
  • Input: $784$ features ($28 \times 28$ normalized grayscale pixels in $[0, 1]$).
  • Hidden 1: $128$ units with ReLU activation.
  • Hidden 2: $64$ units with ReLU activation.
  • Output: $10$ units with numerically stable Softmax and Categorical Cross-Entropy Loss.
• He (Kaiming) Uniform Initialization: Scales weight initialization by $\sqrt{6 / n_{\text{in}}}$ to prevent gradient vanishing or explosion during ReLU training.
• Analytical SGD Backpropagation: Computes exact layer-by-layer gradients $(\delta^{[l]})$ and updates weights and biases online.
• Native IDX Binary Parser: Direct big-endian parser for MNIST idx3-ubyte (images) and idx1-ubyte (labels) formats with magic byte validation.
• Fisher-Yates Shuffling: In-place sample and label shuffling before each epoch.
• Generalization Evaluation: Computes accuracy and loss against the official 10,000-sample held-out t10k test set.
• ASCII Terminal Visualizer: Interactive terminal rendering of 28x28 digits with ASCII density shading and probability distribution bars.
• Model Persistence: Binary serialization (nn_save / nn_load) to save and reload trained weights.

---

Architecture

  [Input: 28x28 Image]
          │
          ▼
   (784 Inputs)
          │
          ▼   [Dense 784 -> 128] + [ReLU]
   (128 Hidden)
          │
          ▼   [Dense 128 -> 64]  + [ReLU]
   (64 Hidden)
          │
          ▼   [Dense 64 -> 10]   + [Softmax]
   (10 Classes: Digits 0 - 9)

---

Benchmarks

Trained with Stochastic Gradient Descent ($\eta = 0.0005$, batch size 1):

Epoch	Train Loss	Train Accuracy	Test Loss (t10k)	Test Accuracy (t10k)
1	0.4675	86.87%	0.2650	92.20%
2	0.2455	92.99%	0.2180	93.75%
3	0.1973	94.31%	0.1895	94.50%

---

Building and Running

Prerequisites

• C11 compliant compiler (GCC, Clang, or MSVC)
• CMake 3.28+
• Ninja (optional, recommended)

Build with CMake Presets

The project includes presets configured in CMakePresets.json:

# Configure using preset
cmake --preset linux-debug

# Build
cmake --build out/build/linux-debug

(On Windows, use x64-debug or x64-release presets).

---

Dataset

If the dataset is not already unpacked in assets/, run the included download script:

./scripts/download_mnist.sh

This downloads and extracts:

• train-images-idx3-ubyte & train-labels-idx1-ubyte (60,000 training samples)
• t10k-images-idx3-ubyte & t10k-labels-idx1-ubyte (10,000 test samples)

---

CLI Usage

Usage: hdrnn [options] [model_path]

Modes:
  --train              Train the network and save the model
  --test               Evaluate the model on the MNIST test set (t10k)
  --demo [index]       Visualize a sample in ASCII and show class probabilities
                       (if index is omitted, a random sample is chosen)

Options:
  -e, --epochs <num>   Number of training epochs (default: 3)
  -l, --lr <rate>      Learning rate (default: 0.0005)
  -m, --model <path>   Path to model weights file (default: models/model.mdl)
  -h, --help           Show this help message

Examples

Train the network for 3 epochs:

./out/build/linux-debug/hdrnn --train -e 3 -l 0.0005

Evaluate on the 10,000-image test set (using default model):

./out/build/linux-debug/hdrnn --test

Specify a custom model path (via flag or positional argument):

./out/build/linux-debug/hdrnn --test -m path/to/model.mdl
./out/build/linux-debug/hdrnn --test path/to/model.mdl

Run an interactive visual demo on a random test sample:

./out/build/linux-debug/hdrnn --demo

Inspect a specific sample (e.g., sample #42):

./out/build/linux-debug/hdrnn --demo 42

---

License

This project is licensed under the terms of the GNU General Public License v3.0.