Skip to content

NetMQ: wait for a subscriber before the first publish (slow joiner) - #106

Merged
christopherread merged 1 commit into
masterfrom
fix/netmq-slow-joiner
Oct 25, 2026
Merged

christopherread merged 1 commit into
masterfrom
fix/netmq-slow-joiner

Conversation

@christopherread

Copy link
Copy Markdown
Owner

Fixes #102.

Cause

ZeroMQ publisher sockets drop messages sent before a subscriber has connected (the "slow joiner"). MessagePublisher creates its socket on the first publish and then slept a fixed 1 second "to wait for subscribers". Any subscriber slower than that lost the first messages. That kept happening on CI runners, which is why the NetMQ tests had to stay out of the CI broker step.

Fix

  • XPUB socket: the publisher uses an XPublisherSocket, which receives subscribe messages, so it can tell when a subscriber for its topic has actually joined.
  • First publish:
    • It still gives subscribers at least 1 second to join, as before. Stopping as soon as the first subscriber joined would cut short a second subscriber still connecting, and the service bus test (two clients) caught exactly that when I first tried it.
    • If no subscriber for the topic has joined by then, it keeps waiting, up to 5 seconds, for the first one.
  • Housekeeping: each publish discards any subscription frames that have queued up, so they don't build up.
  • CI: the NetMQ tests are back in the "Test transports against brokers" step.

Trade-off: a publisher with no subscriber at all now waits up to 5 seconds on its first publish, instead of 1. This happens once per publisher.

Testing (all local, no broker needed)

  • New test, written first: FirstMessageReachesASubscriberThatConnectsLate. The subscriber joins 1.5 seconds after the publisher starts.
    • Before the fix it failed every time (3/3); with the fix it passes.
  • Whole NetMQ suite, all 5 tests together:
    • 10 of 10 runs pass normally.
    • 5 of 5 runs pass with all 10 CPU cores busy, to imitate a slow CI runner.
    • Before the fix, the service bus test lost messages in about 1 run in 11 locally, and often on CI.
  • Removed test: I also wrote a test requiring the first publish to take under 900 ms. It turned out to encode the wrong requirement, since a faster first publish loses messages for other subscribers, so I removed it rather than bending it to pass.
  • Unit tests: all 129 pass, and the solution builds with 0 warnings.

🤖 Generated with Claude Code

The publisher slept a fixed second after binding before its first send,
so subscribers slower than that (as on CI runners) missed the first
messages. Use an XPUB socket, which receives subscriptions: still give
subscribers at least the same second to join, and if none has subscribed
to the topic by then, keep waiting for one, up to 5 seconds. Run the
NetMQ tests in the CI broker step again.

Fixes #102

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@christopherread
christopherread merged commit 468a909 into master Oct 25, 2026
1 check passed
@christopherread
christopherread deleted the fix/netmq-slow-joiner branch October 25, 2026 23:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

NetMQ: first messages lost before subscribers connect (slow joiner)

1 participant