Report a refused connection once it has persisted - #183
Merged
Merged
Conversation
The client announced an authentication failure on the very first refusal, at ERROR, and repeated it on every attempt. A platform that refuses a perfectly good token for a moment - as ours did for 2.3 seconds during a node rotation - therefore filled customer logs with errors telling them to check credentials that were fine, for a condition the retry loop resolved unaided. retrieveSettings no longer logs; it maps the error and leaves the decision to connectSafely, which now asks whether a failure is worth reporting at all. A refusal on the very first attempt an application ever makes is said at once, since nothing has ever worked and whoever is starting it is usually watching. After that, a refusal is most likely transient and waits for the periodic report, as connection failures already did. Everything that remains is info rather than warn or error: the client is not essential to the application it runs in, and an outage on our side should not trip a customer's alerting. Ports #181 to the AF5 line.
|
stefanmirkovic
self-requested a review
September 23, 2026 13:21
stefanmirkovic
approved these changes
Sep 23, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



Ports #181 to the AF5 line.
The problem
The client announced an authentication failure on the first refusal, at ERROR, and repeated it on every attempt:
During a GKE node rotation our gateway was replaced and, for 2.3 seconds, Axon Server still routed
ValidateAccessTokento the dead pod. Inspector turned that into a refusal, and every connected customer application logged errors telling operators to check credentials that were perfectly fine — for something the retry loop resolved on its own, in under three seconds.What changes
retrieveSettingsno longer logs. It maps the error and leaves the decision toconnectSafely, which now asks whether a failure is worth reporting:A refusal on the very first attempt an application ever makes is said at once: nothing has ever worked, so a misconfigured token is far likelier than a blip, and whoever is starting the application is watching. Once a connection has been established the same refusal is most likely transient.
isAuthenticationFailurewalks the cause chain (depth-bounded — a self-referential cause would otherwise spin) because RSocket wraps the server's error.Levels
Everything that remains is info. The client is not essential to the application it runs in, and an outage on our side should not trip a customer's alerting.
Differences from #181
mainalready gated atretryCount == 4then% 10; this unifies both lines on% 10, so the first report lands at attempt 10 rather than 4main's registrar already dropped theauthentication_failedWARN in 75093c6, so only the client needed changingOn the server side
Inspector now distinguishes denied from could not ask and, when it cannot reach the gateway, drops the connection instead of sending
authentication_failed(axoniq-platform#1105). So the stronger "check the access token" wording here only ever fires on a genuine denial. The two changes are independent — this one stands alone, and already-deployed clients benefit from the server change without it.Tests
10 new unit tests covering the classifier and the reporting policy, including the exact incident (
refused, hasEverConnected = true, retryCount = 1→ silent) and the startup case.176 of 180 pass. The four failures are
AxoniqConsoleRSocketClientToxiproxyIntegrationTest, which fail identically on unmodifiedmain— verified on a clean checkout. Pre-existing and worth a separate look.