Repository navigation
fix stream RX in RX threads on LAG member interfaces - #403
Merged
Merged
Conversation
Stream packets received on a LAG member with RX threads enabled were dropped without being counted if the frame was larger than 4074 bytes, so the stream was reported as never received even though the frames arrived on the interface. This affects, for example, downstream jumbo streams to access sessions on a LAG, which can be configured since jumbo-frames support was added (rtbrick#385). bbl_rx_thread() looked up the network and access interfaces on the receiving member, but these are bound to the LAG interface. The lookup failed, so every stream packet was redirected to the main thread through the TXQ, whose slots hold at most BBL_TXQ_BUFFER_LEN (4074) bytes; larger frames were rejected by redirect() without a counter. Resolve a LAG member to its LAG interface, as bbl_rx_handler() already does on the main thread. Stream packets on LAG members are then handled in the RX threads instead of all going through the main thread. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stream packets received on a LAG member with RX threads enabled are dropped without being counted if the frame is larger than 4074 bytes. The stream is then reported as never received, although the frames arrive on the interface, so it looks like loss in the device under test. This affects, for example, downstream jumbo streams to access sessions on a LAG, which can be configured since jumbo-frames support was added (#385).
Cause
bbl_rx_thread()looks up the network and access interfaces on the receiving member, but these are bound to the LAG interface. The lookup fails, so every stream packet received on the member is redirected to the main thread through the TXQ. TXQ slots hold at mostBBL_TXQ_BUFFER_LEN(4074) bytes;redirect()rejects larger frames withIO_ERROR, which the packet_mmap RX job does not handle, so they are dropped without a counter.Fix
Resolve a LAG member to its LAG interface in
bbl_rx_thread(), asbbl_rx_handler()already does on the main thread. Stream packets on LAG members are then handled in the RX threads instead of all going through the main thread.Testing
Builds without warnings,
ctest4/4 passed (-DBNGBLASTER_TESTS=ON, RelWithDebInfo, gcc 13.3, Ubuntu 24.04).A/B with the same config, only the binary changed: access session (static IPoE, QinQ) behind a LAG with one LACP member,
rx-threads: 2,io-mode: packet_mmap_raw,jumbo-frames: true, one stream per frame size and direction at 1000 pps for 60 s. Received / sent:In both runs the member NIC's
port.rx_size_bigcounter increased by exactly the 2000, 4000 and 9000 byte downstream packets sent, so the 9000 byte frames did arrive before the fix.🤖 Generated with Claude Code