The Maltrail sensor: packet capture and trail detection, in Rust. Same maltrail.conf, same
trails.csv, same event format — so the Maltrail server and web interface need no changes.
Road to becoming Maltrail's default sensor: docs/ROADMAP.md — gated, with
verifiable exit criteria and the open blockers stated plainly.
Needs no root: CAP_NET_RAW (plus CAP_NET_ADMIN for promiscuous mode and PACKET_FANOUT) is
enough, and -T validates a deployment before you trust it.
# build
cd sensor && cargo build --release
# run (live capture)
sudo ./sensor/target/release/maltrail-sensor
# run (offline replay)
./sensor/target/release/maltrail-sensor -r capture.pcap -c maltrail.conf --offline- Corpus: 42 cases, each declaring the detections it must produce, replayed and asserted by
tests/replay.rs. Five pin behaviour that deliberately differs from the retired Python sensor, where that sensor was demonstrably wrong — seedocs/COMPATIBILITY.md§2. - Tests: unit + integration + shared vectors generated by calling the real Python functions +
deterministic fuzzing of every parser, run in both the debug and release profiles. Run
sh sensor/tools/check.shfor the current count; the suite grows, so it is not quoted here. - Live capture and
PACKET_FANOUTverified on real hardware: 4 workers in one kernel group, 20,000 packets distributed 5018/5025/5047/4910 and not duplicated (4-worker total identical to the 1-worker baseline). - Measured offline full-sensor replay with a real 1.5M-trail set: ~13.8x lower per-packet
cost than
sensor.pyon the reference box (1.2 µs vs 16.7 µs) — the conservative floor of the 14-37x range across the tested systems (docs/REPORT.md§4a) — and 3.4x faster end to end for a 300k-packet run (startup dominates a short replay). Software packet path scales to ~10 Mpps across 16 workers on an 8-core/16-thread laptop. - Where
sensor.pystill wins: startup (0.20 s warm vs 1.18 s) and peak RSS (63 vs 88 MB), becausecore/trailsbin.pymmaps a prebuilt binary trail store while the sensor parses the CSV on every start. Python's cold start, which builds that store, takes 7.63 s. Closing this is the top next step indocs/REPORT.md§7.1.
See docs/REPORT.md for the measurements and how to reproduce them, and
docs/COMPATIBILITY.md for the (short) list of deliberate differences and unimplemented
features.
| document | contents |
|---|---|
docs/INSTALL.md |
prerequisites, build, run, configuration, systemd, migration plan, troubleshooting |
docs/ARCHITECTURE.md |
data flow, threading model, module map, hot-path guarantees, reload, metrics |
docs/COMPATIBILITY.md |
what is fully compatible, deliberate differences, not implemented, edge cases |
docs/PORTING_MAP.md |
Python source area → Rust module, function by function |
docs/REPORT.md |
implementation report: changed files, test/parity/benchmark results, limitations, next steps |
sensor/
src/
main.rs CLI, startup, diagnostics, worker spawn, reload + metrics threads
config.rs maltrail.conf parser (port of read_config)
settings.rs compiled regexes / matchers
settings_gen.rs GENERATED from core/settings.py by tools/gen_settings.py
pyre.rs Python `re` compatibility (re.escape, \Z, literal braces, group rules)
addr.rs native IP type + Maltrail's exact address rendering
smallstr.rs fixed-capacity stack string (allocation-free rendering)
trails/ batched CSV loader (parallel parse), string + native lookup tables, wildcard regex
whitelist.rs WHITELIST / WHITELIST_RANGES
packet/ DLT/VLAN offsets, offset learner, IP/TCP/UDP/ICMP
protocols/ dns.rs http.rs tls.rs quic.rs
heuristics/ scan.rs dns_exhaustion.rs nxdomain.rs
process.rs the port of _process_packet() and _check_domain()
state.rs per-worker state (caches, burst suppression, accumulators)
event.rs event tuple, safe_value(), log-line + repr rendering
output.rs log file, LOG_SERVER, CEF, Logstash, condensing, throttling
ignore.rs IGNORE_EVENTS + IGNORE_EVENTS_REGEX
capture/ libpcap handles, PACKET_FANOUT
worker.rs run-to-completion capture loop
metrics.rs worker-local counters + lock-free aggregation
throttle.rs event-log throttling (summarize / legacy / off)
meta.rs condensed observable store (meta.sqlite), port of core/meta.py
trailupdate.rs drives core/update.py via tools/update_trails.py
selftest.rs -T preflight
stats.rs Prometheus endpoint (STATS_ADDRESS)
colorized.rs port of core/colorized.py (--console output)
fasthash.rs FxHash for integer-keyed maps (SipHash kept for wire-derived keys)
testkit.rs in-process harness for tests, benches and fuzz targets
tests/ 18 integration suites (detection, replay, trails, vectors, meta,
loader_parity, multi_worker_parity, fail_closed, lifecycle, ...)
tests/corpus/ generated replay corpus (39 pcaps + fixture trails + manifest)
tests/vectors/ test vectors generated by calling the real Python functions
benches/hotpath.rs microbench / packet-path / replay / worker-scaling benchmarks
tools/ gen_settings.py gen_vectors.py gen_ja_vectors.py gen_corpus.py
check_trails.py check_version.py update_trails.py dump_trails.py
fanout_check.py adversarial_traffic.py check.sh
sh sensor/tools/check.shwhich regenerates the Python-derived constants/vectors/corpus, then runs cargo fmt --check,
cargo clippy -D warnings, the full test suite in both profiles, and a release build.
Separately, once per deployment (needs privileges):
sudo python3 sensor/tools/fanout_check.py --interface eth0 --workers 4cargo bench --bench hotpath # staged microbenchmarksNumbers are reported in three clearly separated categories — isolated microbenchmarks, full packet path, and pcap replay — plus worker scaling, so a microbenchmark can never be mistaken for sensor throughput.