Skip to content

ALEC-308: One LLM exchange for both features, with time for local models - #178

Open
joseanesONMS wants to merge 1 commit into
ja/alec-308-token-metricsfrom
ja/alec-308-llm-exchange
Open

joseanesONMS wants to merge 1 commit into
ja/alec-308-token-metricsfrom
ja/alec-308-llm-exchange

Conversation

@joseanesONMS

Copy link
Copy Markdown
Contributor

Jira: https://opennms.atlassian.net/browse/ALEC-308

Second slice of ALEC-308, stacked on #177. Not a chat feature: /chat/completions is the name of the HTTP API every hosted or local LLM server exposes, and this is the one place ALEC now talks to it.

  • The problem: root cause analysis and LLM clustering each had their own copy of the request/response code, and both gave up after 30 seconds; a local model prefilling a large alarm set and generating a grouping needs longer, so every clustering pass against LM Studio or Ollama timed out before the answer arrived and nothing was ever recorded.
  • The exchange now lives once in a new features/llm-client bundle (LlmExchange): build the request, force a tool call, run any data-tool calls the caller offers (none yet; the MCP tools come in the next slice), stop at the terminal call, record token usage per round into the ALEC-308: Chart LLM token usage on the Resource Graphs #177 sink.
  • Both features call it in single-shot mode, so the wire format they send is unchanged.
  • Read timeouts rise to 180 s per request, 60 s for the interactive validation probe, and the pending-analysis stale window grows to match.
  • A malformed terminal tool call is now a failure instead of an empty success, tokens billed before a failure stay accounted for in usage rows, budgets and gauges, and a base URL a local server rejects gets a hint to append /v1.
  • Verified against LM Studio (gemma-4-e4b): a clustering pass completes under the new timeout; 30 exchange tests plus the adapted engine and suggestions suites pass and the Karaf feature verification stays green.

Root cause analysis and LLM clustering each carried their own copy of the
same HTTP exchange with the model's /chat/completions endpoint, and both
copies gave up after 30 seconds. A locally hosted model (LM Studio, Ollama)
prefilling a large alarm set and generating a grouping needs far longer, so
every clustering pass against such a model was abandoned before the answer
arrived — nothing was ever recorded, and the operator saw no clustering
activity at all.

The exchange now lives once, in features/llm-client (LlmExchange): build the
request, force a tool call, run any data-tool calls the caller offers (none
yet — the MCP tools come in a later change), stop at the terminal call, and
record token usage per round into the engine/api sink. Both features call it
in single-shot mode, so their wire format is unchanged. Read timeouts rise to
180 s per request (60 s for the interactive validation probe), a malformed
terminal call is a failure instead of an empty success, tokens the provider
billed before a failure stay accounted for (usage rows, budget, gauges), the
pending-analysis stale window matches the longer wait, and a base URL that a
local server rejects gets a hint to append /v1.
@github-actions github-actions Bot added the docs label Sep 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant