tools/paradee: memory-map the F32 GGUF; record the F16 and v1.1 evaluation

tools/paradee/README.md

@@ -83,7 +83,55 @@ front end.
 - **GGUF:** the expected sha256 of the F32 converter output. The converter is byte-reproducible.
   `--check-output` enforces this sha256.
 - **No F16 pin:** `--outtype f16` halves the file (16 MB) but fails the parity gate (end-to-end SNR
-  16–23 dB), so it is not pinned or shipped.
+  16–23 dB), so it is not pinned or shipped. See [F16 and v1.1](#f16-and-v11) for the full evaluation.
+
+## Memory
+
+`paradee_load_model` memory-maps an all-F32 GGUF (the converter's default) read-only and the layers read the
+weights in place. The 32 MB of weights are then clean, reclaimable page cache rather than private memory.
+`PARADEE_USE_MMAP=0` reads the file into private memory instead. Any other file (for example `--outtype f16`)
+is read and widened to F32 at load, so it keeps a private F32 copy of every weight as well as the file data.
+
+On a Lenovo TB336FU (4 GB), synthesizing 19.8 s of speech with `paradee-tts` gives byte-identical audio either
+way. Peak private memory is 170 MB mapped against 202 MB read, and peak file-backed memory is 34 MB against
+3 MB. Activations make up most of the peak.
+
+## F16 and v1.1
+
+Measured October 19, 2026 against the pinned v1.0 GGUF, with this directory's parity test, its reference dump,
+and 40 more sentences (20 assistant-style prompts and 20 Harvard sentences). Those sentences were phonemized
+with `tools/misaki-g2p` and transcribed with Phonon-2 (`tools/parakeet`) and with whisper.cpp `base.en`.
+
+| File | Size (deflate -9) | Parity test | Durations (40 sentences) | WER Phonon-2 / whisper | Log-mel vs F32 | RTF, TB336FU / M4 Max |
+|---|---|---|---|---|---|---|
+| F32 (shipped) | 32.1 MB (29.7) | pass, end to end 58–76 dB | – | 3.90% / 3.90% | – | 0.30 / 0.024 |
+| `--outtype f16` | 16.2 MB (14.9) | fail: end to end 9.8–23 dB, stage errors up to 1.1e-2 × RMS | 1 of 40 shorter by 25 ms | 3.90% / 3.90% | 0.039 mean, 0.56 max | 0.32 / 0.024 |
+| F16 decoder, F32 BERT and predictor (experiment, not in the converter) | 22.1 MB (20.4) | end to end 56–61 dB; fails the 1e-3 × RMS stage gate on `asr` and `dec` only | identical | 3.90% / 3.90% | 0.0026 | 0.30 / 0.024 |
+
+- The end-to-end error of `--outtype f16` comes from the BERT and the duration/F0 predictor. The decoder in F16
+  costs little. F0 errors near 1e-3 accumulate in the harmonic source's phase, which the waveform SNR punishes
+  while the spectrum barely changes.
+- F16 does not speed anything up here, because the runtime computes in F32. It does not save RAM either: an F16
+  file cannot be mapped and is widened at load, so private memory goes up by the size of the file (about 16–22 MB).
+- So F32 stays the shipped and pinned file. F16 would only make sense with layers that read F16 weights from
+  the mapping. The ggml conv1d path already converts its weights to a cached F16 copy (`cached_f16_weight`,
+  about 10 MB for the 3-D weights).
+- **v1.1** ([sahilmahendrakar/Paradee-8M-v1.1](https://huggingface.co/sahilmahendrakar/Paradee-8M-v1.1) at
+  `330617ad`, Apache-2.0) is not a weight update this runtime can load. It keeps the text side (same
+  `text_side.pt`) and the decoder's AdaIN trunk, but it replaces the iSTFTNet generator and the phase-locking
+  filter with a frame-rate source-filter head (`training/scripts/vhead.py` at github.com/sahilmahendrakar/paradee
+  `606e16e1`). That head is ConvNeXt blocks at 80 fps that shape the STFT of a 64-harmonic sine bank and of
+  random noise, plus a 3 ms noise path. Its output is not deterministic, because it draws random noise.
+- Upstream's fp32 ONNX graphs, one onnxruntime thread on the M4 Max, the same 40 sentences:
+
+  | Graph | RTF | WER Phonon-2 / whisper |
+  |---|---|---|
+  | v1.0 | 0.051 | 4.16% / 3.90% |
+  | v1.1 | 0.010 | 3.90% / 3.12% |
+
+  int8 graphs: v1.0 0.054, v1.1 0.011, with WER within one word of fp32.
+- Running v1.1 natively needs a new decoder head (sine bank, ConvNeXt, two STFT/iSTFT sizes, a seeded noise
+  source) and a new parity gate that does not compare the noise.
 
 ## Parity gate
 

tools/paradee/include/paradee.h

@@ -63,7 +63,9 @@ const char * paradee_status_str(paradee_status st) noexcept;
 
 // Load a Paradee GGUF written by convert_paradee_pt_to_gguf.py. The file must
 // declare general.architecture == "paradee"; every hparam the graph reads is
-// required. Returns nullptr and sets `err` on failure.
+// required. An all-F32 file is memory-mapped read-only and used in place
+// (PARADEE_USE_MMAP=0 reads it into memory instead); other files are read and
+// widened to F32. Returns nullptr and sets `err` on failure.
 paradee_model_ptr paradee_load_model(const std::string & gguf_path, std::string & err) noexcept;
 
 // 24000 for Paradee-8M v1.0 (0 for a null model).

tools/paradee/src/paradee.cpp

@@ -37,6 +37,13 @@
 #include <unordered_map>
 #include <vector>
 
+#if !defined(_WIN32)
+#include <fcntl.h>
+#include <sys/mman.h>
+#include <sys/stat.h>
+#include <unistd.h>
+#endif
+
 namespace eliza_paradee {
 
 using eliza_kokoro::DecoderConfig;
@@ -74,6 +81,8 @@ struct paradee_model {
     gguf_context * gguf = nullptr;
     ggml_context * file_ctx = nullptr;   // tensors as stored
     ggml_context * ctx = nullptr;        // all-F32 view the layers read
+    void * map_addr = nullptr;           // read-only mapping of an all-F32 file (else null)
+    size_t map_size = 0;
     std::vector<float> style_text, style_dec;
     std::mutex mu;
 };
@@ -83,11 +92,53 @@ void paradee_model_deleter::operator()(paradee_model * m) const noexcept {
     if (m->ctx && m->ctx != m->file_ctx) ggml_free(m->ctx);
     if (m->file_ctx) ggml_free(m->file_ctx);
     if (m->gguf) gguf_free(m->gguf);
+#if !defined(_WIN32)
+    if (m->map_addr) munmap(m->map_addr, m->map_size);
+#endif
     delete m;
 }
 
 namespace {
 
+// PARADEE_USE_MMAP=0 reads the weights into private memory instead of mapping them.
+bool paradee_use_mmap() {
+    const char * e = std::getenv("PARADEE_USE_MMAP");
+    return !(e && (std::strcmp(e, "0") == 0 || std::strcmp(e, "false") == 0 || std::strcmp(e, "off") == 0));
+}
+
+// Map an all-F32 GGUF read-only and point every tensor of `ctx` (a no_alloc
+// context from gguf_init_from_file) at its bytes in the mapping. The layers
+// only read the weights, so the pages stay clean and the kernel can drop and
+// re-read them under memory pressure instead of swapping a private copy.
+bool map_f32_tensors(paradee_model * m, const std::string & path, std::string & err) {
+#if defined(_WIN32)
+    (void) m; (void) path; (void) err;
+    return false;
+#else
+    const int fd = ::open(path.c_str(), O_RDONLY);
+    if (fd < 0) { err = "cannot open '" + path + "'"; return false; }
+    struct stat st {};
+    if (fstat(fd, &st) != 0 || st.st_size <= 0) { ::close(fd); err = "cannot stat '" + path + "'"; return false; }
+    void * addr = mmap(nullptr, (size_t) st.st_size, PROT_READ, MAP_PRIVATE, fd, 0);
+    ::close(fd);
+    if (addr == MAP_FAILED) { err = "mmap failed for '" + path + "'"; return false; }
+    m->map_addr = addr;
+    m->map_size = (size_t) st.st_size;
+    const size_t base = gguf_get_data_offset(m->gguf);
+    for (int64_t i = 0; i < gguf_get_n_tensors(m->gguf); ++i) {
+        ggml_tensor * t = ggml_get_tensor(m->file_ctx, gguf_get_tensor_name(m->gguf, i));
+        if (!t) continue;
+        const size_t off = base + gguf_get_tensor_offset(m->gguf, i);
+        if (off + ggml_nbytes(t) > m->map_size) {
+            err = std::string("tensor '") + ggml_get_name(t) + "' runs past the end of the file";
+            return false;
+        }
+        t->data = (char *) addr + off;
+    }
+    return true;
+#endif
+}
+
 bool read_i32(gguf_context * g, const char * key, int & out, std::string & err) {
     const int64_t id = gguf_find_key(g, key);
     if (id < 0) { err = std::string("missing GGUF key ") + key; return false; }
@@ -180,12 +231,37 @@ paradee_model_ptr paradee_load_model(const std::string & gguf_path, std::string
     err.clear();
     try {
         paradee_model_ptr m(new paradee_model());
-        gguf_init_params params = { /* no_alloc = */ false, /* ctx = */ &m->file_ctx };
-        m->gguf = gguf_init_from_file(gguf_path.c_str(), params);
+        // Metadata first. An all-F32 file (the converter's default) is then
+        // mapped and used in place; anything else is read into memory and
+        // widened below.
+        gguf_init_params meta = { /* no_alloc = */ true, /* ctx = */ &m->file_ctx };
+        m->gguf = gguf_init_from_file(gguf_path.c_str(), meta);
         if (!m->gguf || !m->file_ctx) {
             err = "gguf_init_from_file failed for '" + gguf_path + "'";
             return nullptr;
         }
+        bool mapped = false;
+        if (paradee_use_mmap()) {
+            bool f32_only = true;
+            for (int64_t i = 0; i < gguf_get_n_tensors(m->gguf); ++i) {
+                f32_only = f32_only && gguf_get_tensor_type(m->gguf, i) == GGML_TYPE_F32;
+            }
+            if (f32_only) {
+                mapped = map_f32_tensors(m.get(), gguf_path, err);
+                if (!mapped && !err.empty()) return nullptr;
+            }
+        }
+        if (!mapped) {
+            ggml_free(m->file_ctx);
+            m->file_ctx = nullptr;
+            gguf_free(m->gguf);
+            gguf_init_params params = { /* no_alloc = */ false, /* ctx = */ &m->file_ctx };
+            m->gguf = gguf_init_from_file(gguf_path.c_str(), params);
+            if (!m->gguf || !m->file_ctx) {
+                err = "gguf_init_from_file failed for '" + gguf_path + "'";
+                return nullptr;
+            }
+        }
         gguf_context * g = m->gguf;
         {
             const int64_t id = gguf_find_key(g, "general.architecture");