Skip to content

Stop audit record serialization from crashing on absent nested messages - #76

Open
dreadl0ck wants to merge 1 commit into
masterfrom
fix/bfd-nil-auth-header
Open

dreadl0ck wants to merge 1 commit into
masterfrom
fix/bfd-nil-auth-header

Conversation

@dreadl0ck

Copy link
Copy Markdown
Owner

Summary

A BFD packet without authentication crashed the whole capture during --csv export. types/bfd.go called getString() on a nil *BFDAuthHeader, and because the method had a value receiver the nil pointer was dereferenced immediately.

Optional nested protobuf messages are nil whenever the protocol did not carry them, so this is the common case rather than an edge case. Adding a test that serializes every registered audit record type zero-valued showed the same crash in Dot11, CiscoDiscoveryInfo, LinkLayerDiscovery and LinkLayerDiscoveryInfo, and exposed a second, quieter bug class.

Crash Fix

Every toString() / getString() helper in types/ now uses a pointer receiver with an early nil guard, following the existing GRERouting.getString convention. 59 helpers across 18 files. Output for non-nil values is unchanged — only the guard was prepended.

CSV Columns Were Misaligned

CSVHeader() and CSVRecord() disagreed for six record types, so values were written under the wrong column. types.filter() applies header-derived indices positionally, so the header, the record and Encode() must correspond one-to-one.

Type Root cause Fix
IPv4 record emits Padding, header had no entry added the existing unused fieldPadding
GRE header declared SrcPort/DstPort, absent from the message removed both; no data lost
Modbus record emits Payload, header entry commented out restored fieldPayload
Service header declared Notes, record never emitted it emit a.Notes
QUICClientHello record emits CHLO tag values, header entry missing added the existing constant
TLSClientHello record emits Random and SessionID, header had neither added both

The GRE mismatch additionally made Inc() panic with inconsistent label cardinality: expected 20 label values but got 18.

Metric Label Cardinality

Most types use the whole CSV record as Prometheus labels. Exposing the fields above would have turned per-packet-unique values — Modbus Payload, TLS Random and SessionID, QUIC connection IDs and tag values — into label values, creating a new time series per packet.

Modbus, TLSClientHello and QUICClientHello now use dedicated metricValues() helpers that omit those fields, matching the existing IPv4/Service pattern. The pre-existing high-cardinality labels on other types are untouched and remain a separate concern.

QUICClientHello.tagValuesString() also iterated a map unsorted, so that column varied between runs; it is now sorted.

Test

io/audit_record_nil_test.go walks every type registered in io.InitRecord (118) and asserts that CSVHeader, CSVRecord, Encode, JSON, Time, Src, Dst, NetcapType, SetPacketContext and Inc do not panic on a zero value, and that header and record lengths agree. It reports the panic stack per type and recovers so one broken type does not mask the rest.

Reverting the GRE header fix makes it fail with the Prometheus cardinality panic, so the Inc() assertion is effective. The length assertion catches all six mismatches above.

Verification

pcaps/The Ultimate PCAP v20250325.pcapng, 49,380 packets, full DPI build, all decoders, --csv --entropy=true.

  • Before: capture segfaults in types.(*BFD).CSVRecord.
  • After: completes, 204,897 rows written.

Parsing every emitted CSV file with a strict parser, files whose rows all match their header count went from 44 to 48; GRE.csv, IPv4.csv and Modbus.csv became consistent and nothing regressed.

go build ./...                          OK
go vet ./...                            OK
go test -short -race ./...              OK, except the pre-existing
                                        TestCapturePCAP failure caused by a
                                        gitignored PCAP fixture missing
go test ./io/ -run TestAuditRecord...   118/118 subtests pass

Known Limitation, Not Fixed Here

The remaining CSV inconsistencies are a separate pre-existing bug: types/utils.go sets FieldSeparator = "," for nested composite fields, while io/csv_proto_writer.go builds rows with a plain strings.Join(values, ",") and no quoting. Any composite field or any string containing a comma silently splits into extra columns, affecting roughly 33 record types including DNS, Connection and TLSCertificate.

Master shows exactly the same mismatches, so this PR does not cause or worsen it. Fixing it means writing through encoding/csv (or changing the nested separator), which changes emitted output and needs golden-file regeneration — worth its own change.

Column ordering beyond the six corrected types was not audited: the test compares lengths, so a header and record of equal length but permuted order would still pass. Adding columns to IPv4, Modbus, TLSClientHello, QUICClientHello and Service means golden files for those records need regeneration.

Optional nested protobuf messages are nil whenever the protocol did not
carry them, which is the common case rather than an edge case: a BFD
packet without authentication segfaulted the entire capture during CSV
export, because getString had a value receiver and dereferenced the nil
AuthHeader. Dot11, CiscoDiscoveryInfo and both LinkLayerDiscovery records
crashed the same way.

Give every toString/getString helper a pointer receiver and a nil guard,
following the existing GRERouting convention.

While covering all record types, CSVHeader and CSVRecord turned out to
disagree for six of them, so values were written under the wrong column:
IPv4 emitted Padding with no header entry, GRE declared SrcPort/DstPort
that the message does not have, Modbus and QUICClientHello and
TLSClientHello emitted payload, tag and handshake fields the header
omitted, and Service declared Notes without ever emitting it. The GRE
mismatch also made Inc() panic on inconsistent Prometheus label
cardinality.

Keep the newly exposed per-packet values out of the metric labels via
dedicated metricValues helpers, otherwise every packet would create its
own time series. Sort the QUIC tag values so that column is reproducible.

Add a test that exercises every registered audit record type zero-valued.
@github-actions

github-actions Bot commented Sep 6, 2026

Copy link
Copy Markdown

Thank you for your contribution to NETCAP!

Before we can accept your pull request, you need to sign our Contributor License Agreement (CLA).

To sign the CLA, please read the Individual CLA and comment below with:

I have read and agree to the NETCAP CLA

If you are contributing on behalf of a company or organization, please contact the maintainers to complete the Entity CLA instead.


You can re-trigger this check by commenting recheck on this PR.


I have read and agree to the NETCAP CLA


You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant