Traffic stack: TPEG/TEC, RDS-TMC, DAB packet-mode and announcement FIGs

[Supermagnum]

Summary

• Add crates/traffic as a shared application layer for TPEG/TEC and RDS-TMC GeoJSON (dab-tpeg / fm-rds-tmc), including ALERT-C with CA/encryption-admin detection without decrypting LTN=0.
• Extend dabradio with packet-mode FIG 0/2 (tmid==3), 0/3, 0/8, 0/13 plus MSC packet reassembly (--dump-fic, --dump-packets, --traffic). Validated on NRK RIKS 12D: packet CRC pass rate 91.5% (681/744) on SPI/EPG; FIG 0/13 shows UAtype 0x007 (EPG) only, no 0x004 TPEG — matching welle.io on the same IQ.
• Add dabradio FIG 0/18 / 0/19 announcement monitoring (--announcements, AnnouncementEvent JSON). Cross-checked against welle.io on Innland 13E (EID 0xF501): both agree 0/18 and 0/19 are absent in-FIC; a ~40-minute live watch also saw ext18=0.
• Wire fmradio RDS-TMC via group 3A ODA and 8A ALERT-C (--traffic, --location-tables).
• Add tools/fig018_live_watch.py for live FIG 0/18 monitoring against welle-cli dump.fic, and gitignore IQ-files/ so large captures stay local.
• Rustfmt (and related clippy/docs) fixes so CI Check passes.

Test plan

• cargo test -p traffic
• cargo test -p fmradio (ALERT-C unit paths)
• cargo check -p dabradio -p fmradio -p traffic
• Replay NRK RIKS 12D IQ through dabradio packet/traffic path; confirm CRC pass rate and FIG 0/13 UAtype vs welle.io
• Replay Innland 13E (or live watch) for FIG 0/18/0/19 absence agreement with welle.io
• Confirm tools/fig018_live_watch.py tails dump.fic without rewriting under welle's open FD

Made with Cursor

[Supermagnum]

DAB does not send out traffic information any more here in Norway, that was closed "to save money " .

[xoolive]

Thank you for your contribution! I will have to read more in depth, but here are already a few comments I have

[xoolive]

Thanks for the substantial addition. I found several correctness issues that should be addressed before merge:

1. TPEG malformed-input panic: crates/traffic/src/tpeg/transport.rs:131-172 can accept an overflowing IntUnLoMB value and then evaluate unchecked pos + len. A CRC-valid malformed payload can therefore panic (the release profile enables overflow checks). Please reject unrepresentable values before shifting and use checked_add for slice bounds.

2. Unbounded FM memory use: decoded TMC features are always queued in RdsParser, but main.rs drains them only with --traffic. A normal long-running receive of a TMC station will retain repeated features indefinitely. Always drain the queue; print only when the flag is enabled.

3. Partial 3A ODA registration corrupts state: the new 3A path uses/persists blocks C and D even when they failed CRC. A partial group can overwrite a valid ODA registration or cause handle_system_group(0) to mark TMC as encrypted. Require valid C and D blocks before updating ODA/TMC state, with a regression test.

4. Unsupported traffic mode combinations: --mono --traffic takes the mono path without RDS processing; --raw-out --traffic continues before RDS output. Either support these combinations or reject them explicitly.

5. DG flag handling: a packet component whose FIG 0/3 has DG flag=1 (no_data_groups) still unconditionally goes through application_payload() in dabradio/src/main.rs. That strips a data-group header/CRC from raw application data. Use the raw group bytes for no-data-group components.

6. Late packet FIGs are missed: traffic_initialized is set once after generic ensemble completeness. Since this does not require FIG 0/3, 0/8, or 0/13, later packet/TPEG metadata is never picked up; file traffic mode may also exit early. Re-evaluate targets as packet FIG information arrives without resetting existing MSC handlers.

7. Duplicate packet targets: FIG 0/2 deduplication includes the later-resolved mutable subchannel_id, allowing the same (TMId=3, SCId) component to be inserted twice. Deduplicate/update by stable component identity.

Validation: cargo test -p traffic -p fmradio -p dabradio passes (184 passed, 5 ignored). FM replay decoded normal RDS successfully but contained no TMC ODA. A 300-frame RTL-SDR DAB replay decoded Métropolitain 2 (3,588 FIBs) but contains no packet/TPEG/announcement signalling, so it validates the normal DAB path only.

[xoolive]

Agent found a few issues, but I think the biggest one is that I don't have any way to validate the code is actually meaningful on actual data. With my local recordings (in few European countries), none of them found data going through your implementation...

[Supermagnum]

All 7 issues addressed, each with a regression test:

1. TPEG malformed-input panic (transport.rs:131-172) — overflowing IntUnLoMB values are now rejected before shifting; slice-bound arithmetic uses checked_add/try_from. Regression: CRC-valid frame with an overflowing length field no longer panics.
2. Unbounded FM memory growth — take_traffic_features() is now always called to drain the queue; --traffic only gates printing. Regression: draining without the flag set still clears the queue.
3. Partial 3A ODA corruption — ODA/TMC state now only updates when both blocks C and D pass CRC. Regression: a group with valid A/B but corrupted C/D leaves prior state untouched.
4. DG flag handling — added payload_for_dg_flag(); components with DG flag = 1 now use raw group bytes instead of having a nonexistent data-group header stripped. Regression covers both DG=0 (still strips header) and DG=1 (raw bytes preserved).
5. Late packet FIGs missed — replaced the one-shot traffic_initialized gate with continuous merge_traffic_channels; no early exit, UA metadata refreshes as it arrives without resetting existing MSC handlers. Regression: late FIG 0/13 now arms detection without a handler reset. (Confirmed this does not affect the earlier RIKS/Innland "absent" findings — those went through --dump-fic's accumulate-through-EOF path, which never touched this gate.)
6. Duplicate packet targets — FIG 0/2 packet components now dedupe/update by stable SCId instead of the mutable resolved subchannel_id. Regression: re-resolution no longer duplicates a component.
7. Unsupported mode combinations — --mono --traffic and --raw-out --traffic are now rejected at startup with an explicit error instead of silently skipping RDS processing. Regression: combo-validation unit test.

cargo test -p traffic -p fmradio -p dabradio: 185 passed, 4 ignored.

[Supermagnum]

Agent found a few issues, but I think the biggest one is that I don't have any way to validate the code is actually meaningful on actual data. With my local recordings (in few European countries), none of them found data going through your implementation...

Re: validating against actual data — found something since the last update, and it partially closes this.

Real off-air confirmation of genuine TPEG signaling. A raw IQ sample from a Belgian RTBF DAB ensemble (channel 12B), originally shared by another user on dab-cmdline issue #27 back in 2017 (hosted on Mega.nz, still downloading), lists a service named TPEG_PACKET. Running the full FIG 0/13 sweep against it — same procedure used for the RIKS/Innland "absent" findings, cross-checked against welle.io's dump.fic on the same file — confirms it: UAtype 0x004, SId 0xE0606361, resolved via FIG 0/3 to SubCh 14 / packet address 1 / 16 kbps. welle-cli and this implementation agree on all 12 services/labels in the ensemble. This is the first genuine UAtype 0x004 this project has seen on real broadcast data, as opposed to every capture so far (RIKS, Innland, your Métropolitain 2 replay) which validated the FIC/FIG pipeline correctly but happened to carry no TPEG.

MSC/packet decode on this file is weak but real, and the reason is worth stating plainly. Initial packet CRC pass rate was 2.7% (later 3.4% after a couple of fixes below) — I want to be precise that this is not chance-level: chance for a 16-bit CRC is ~0.0015%, so ~2.7–3.4% is roughly 20–40x above noise. I initially suspected a DC-offset issue specific to this file's cu8/RTL-SDR origin (vs. the cf32/USRP-class hardware used for RIKS/Innland) and fixed a real centering bug in the cu8 path along the way (/128 → (b−127.5)/127.5), plus added a real NCO frequency-mix and switched MSC to viterbi_decode_state0. But direct spectrum measurement ruled out DC as the actual cause here (dc_excess_db ≈ 0.02, no spike). The real explanation: this specific file's own FIC quality peaks around ~80% CRC, versus ~99.9% on the clean captures — it's a decade-old RTL-SDR clip with whatever reception conditions existed at capture time, and that's a ceiling on the recording itself, not something the decoder can fix. MSC tracking below that (3–18% depending on component) is consistent with FIC's much stronger protection absorbing more of the same underlying degradation than MSC's thinner margin can.

Why I think the partial result is itself meaningful, not just "not done yet": a broken or brittle decoder would show chance-level output regardless of input quality — that's exactly what the original gr-dab attempt looked like (~1 pass out of ~97,800, indistinguishable from noise). Getting a real, measurably-above-chance partial decode out of a genuinely degraded, ~9-year-old, low-SNR capture is evidence the implementation degrades gracefully with signal quality rather than failing outright — which is closer to what a robust real-world decoder needs to do than only succeeding on clean, modern captures. Combined with the ~91.5% (RIKS) and ~84% (Innland) pass rates already validated on clean signals, I'd frame this as: works well on good signal, still extracts real signal on bad signal, rather than reading the Belgian result in isolation as a failure.

Test suite: cargo test -p traffic -p fmradio -p dabradio — 185 passed, 4 ignored. Full status trail for the Belgian capture is in IQ-files/be_12b_validation/be_12b_validation_summary.json plus the note in crates/dabradio/readme.md, alongside the RIKS/Innland trails from before.

I know this doesn't fully close the "validate against real TPEG content end-to-end" question — the TPEG transport/TEC layer still hasn't been exercised against a cleanly-decoded real packet stream, since this file's ceiling doesn't get there. If you have a European recording of your own from a currently-active TPEG deployment with better SNR than this one, that would be the fastest way to finish that last piece — happy to run it through the same validation procedure.

[xoolive]

A reproducible positive-signal capture is now available: GQRX wideband Belgian DAB capture.

Format: zstd-compressed GQRX cf32, centre frequency 220.936 MHz, sample rate 16 MS/s. It contains:

• 12A / DAB+ VRT: FIG 0/13 UAtype 0x004 TPEG declaration and live FIG 0/18/0/19 road-traffic announcement events;
• 12B / DAB Bruxelles: packet-mode EPG traffic with target-address packets and strong packet CRC results.

SHA-256: 7515b9243759f3f12931f6acc1b60a1f4049762fa64e88ba9e1c7e62c8f3f9f2

The file is intentionally shared outside Git because it is about 695 MiB compressed. It can be replayed directly after decompression, for example:

zstd -dc capture.raw.zst | dabradio - --channel 12A --format cf32 \
  --sample-rate 16000000 --center-freq 220936000 --no-audio --dump-fic --dump-packets --traffic

[Supermagnum]

A reproducible positive-signal capture is now available: GQRX wideband Belgian DAB capture.

Format: zstd-compressed GQRX cf32, centre frequency 220.936 MHz, sample rate 16 MS/s. It contains:

* **12A / DAB+ VRT:** FIG 0/13 `UAtype 0x004` TPEG declaration and live FIG 0/18/0/19 road-traffic announcement events;

* **12B / DAB Bruxelles:** packet-mode EPG traffic with target-address packets and strong packet CRC results.

SHA-256: 7515b9243759f3f12931f6acc1b60a1f4049762fa64e88ba9e1c7e62c8f3f9f2

The file is intentionally shared outside Git because it is about 695 MiB compressed. It can be replayed directly after decompression, for example:

zstd -dc capture.raw.zst | dabradio - --channel 12A --format cf32 \
  --sample-rate 16000000 --center-freq 220936000 --no-audio --dump-fic --dump-packets --traffic

OK, I can run that trough my code if you want that done ?

[xoolive]

Ok actually if you rebase on main you can read directly the .zst file (no need to pipe it)
Then I tried with your code and it seems to say there could be something in this file (but two different things on the 2 channels) but it doesn't work as is. Agent suggested some changes but then nothing really conclusive either.

If you want to try?

[Supermagnum]

Ok actually if you rebase on main you can read directly the .zst file (no need to pipe it) Then I tried with your code and it seems to say there could be something in this file (but two different things on the 2 channels) but it doesn't work as is. Agent suggested some changes but then nothing really conclusive either.

If you want to try?

I will look at it tomorrow, but it depends on how much tokens I have left this most month.

[xoolive]

No rush, I am not in a hurry to merge, at least not until there's evidence some new information appears

[Supermagnum]

GQRX Belgian capture is validated. The missing piece was packet-mode FEC (addr 1022 / RS(204,188)).

Results
Channel Signal Outcome
12A VRT
FIG 0/13 0x004 TPEG + live FIG 0/18/0/19
Confirmed
12A TPEG packets
After FEC: 16 frames, 100% CRC
All addr 0 padding — no TPEG payload in this clip
12B Bruxelles
EPG/SPI packets
FEC on → ~99.8% CRC, 4 data groups
Earlier “strong CRC but groups_complete=0” was misleading: CRC-ok traffic was mostly padding; real packets need the outer RS layer first.

Code
crates/dabradio/src/msc/packet.rs now auto-enables FEC on addr-1022 packets (same model as dab-cmdline). Readme + IQ-files/gqrx_be_validation/gqrx_be_validation_summary.json updated.

zstd -dc gqrx_….raw.zst | dabradio - --channel 12B --format cf32
--sample-rate 16000000 --center-freq 220936000 --no-audio --dump-packets
So: positive FIC/announcement signal on 12A; positive packet reassembly on 12B; still no TPEG GeoJSON from this file because the TPEG subchannel is idle here.