Skip to content

perf(upstream): list tools from servers concurrently during discovery - #1814

Open
poolski wants to merge 1 commit into
smart-mcp-proxy:mainfrom
poolski:perf/parallel-tool-discovery
Open

poolski wants to merge 1 commit into
smart-mcp-proxy:mainfrom
poolski:perf/parallel-tool-discovery

Conversation

@poolski

@poolski poolski commented Nov 2, 2026

Copy link
Copy Markdown

A discovery sweep listed each connected server's tools one after another, so one slow or large upstream delayed every other server's tools/list by up to its own timeout.

The sweep now lists up to 8 servers at once (discoveryListConcurrency). Results are recorded in the original order with the same logging, markSwept and capture handling. A server's concurrent tools/list calls were already coalesced into one request, and each capture carries its own process-global sequence, so the order in which lists finish does not change how their results are applied.

TestDiscoverToolsListsServersConcurrently uses three upstreams whose tools/list waits for another list to be in flight and requires the sweep to overlap them.

Testing

  • go test -race ./internal/upstream/...
  • scripts/test-api-e2e.sh (70/70)
  • golangci-lint with .github/.golangci.yml, bare and --build-tags server: no new issues

A discovery sweep listed each connected server's tools one after
another, so one slow or large upstream delayed every other server's
tools/list by up to its own timeout. The sweep now lists up to 8
servers at once (discoveryListConcurrency) and records the results in
the same order and with the same logging as before. A server's
concurrent tools/list calls were already coalesced into one request,
and each capture carries its own sequence, so the order in which the
lists finish does not affect how their results are applied.
@poolski
poolski marked this pull request as ready for review November 2, 2026 09:36

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant