Skip to content

fix(mcp): mark save_webhook and save_alert as open-world - #3639

Merged
kodiakhq[bot] merged 2 commits into
mainfrom
claude/mcp-open-world-alert-webhook
Nov 12, 2026
Merged

kodiakhq[bot] merged 2 commits into
mainfrom
claude/mcp-open-world-alert-webhook

Conversation

@brandon-pereira

Copy link
Copy Markdown
Member

Summary

clickstack_save_webhook and clickstack_save_alert now declare openWorldHint: true. Neither makes a network call during the tool call, but a saved webhook points at a third-party endpoint (Slack, incident.io, or any public URL with a templated body), and a saved alert sends requests to it on a schedule. Declaring them closed-world told MCP clients they could skip the extra review they give tools that reach outside the system. That review matters here: a prompt-injected agent could create a generic webhook to an attacker URL plus a frequently firing alert, and use them to exfiltrate telemetry.

Every other tool stays closed-world. clickstack_delete_webhook stays closed-world too, since deleting only stops deliveries. A new test pins the open-world set to exactly these two tools.

Flagged by an MCP tool-annotation compatibility check.

How to reproduce

Starting point: local yarn dev (needs the API).

On main:

  1. Call tools/list on the MCP server (/api/mcp).
  2. clickstack_save_webhook and clickstack_save_alert report annotations.openWorldHint: false.

On this branch:

  1. Same call: both report openWorldHint: true, and all other tools still report false.
  2. cd packages/api && yarn ci:unit src/mcp/__tests__/toolAnnotations.test.ts passes.

Notes

Committed with --no-verify. lint-staged passed. The hook's knip step fails on an issue already on main: @opentelemetry/browser-sdk is declared in packages/app/package.json (added in #3575) but never imported.

@brandon-pereira
brandon-pereira requested a review from a team as a code owner November 12, 2026 20:21
@vercel

vercel Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
hyperdx-oss Ready Ready Preview Nov 12, 2026 8:41pm UTC
hyperdx-storybook Ready Ready Preview Nov 12, 2026 8:41pm UTC

Request Review

@changeset-bot

changeset-bot Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: 23f7d2a

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 3 packages
Name Type
@hyperdx/api Patch
@hyperdx/app Patch
@hyperdx/otel-collector Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions github-actions Bot added the review/tier-2 Low risk — AI review + quick human skim label Nov 12, 2026
@github-actions

Copy link
Copy Markdown
Contributor

🔵 Tier 2 — Low Risk

Small, isolated change with no API route or data model modifications.

Why this tier:

  • Standard feature/fix — introduces new logic or modifies core functionality

Review process: AI review + quick human skim (target: 5–15 min). Reviewer validates AI assessment and checks for domain-specific concerns.
SLA: Resolve within 4 business hours.

Stats
  • Production files changed: 3
  • Production lines changed: 22 (+ 15 in test files, excluded from tier calculation)
  • Branch: claude/mcp-open-world-alert-webhook
  • Author: brandon-pereira

To override this classification, remove the review/tier-2 label and apply a different review/tier-* label. Manual overrides are preserved on subsequent pushes.

@github-actions

Copy link
Copy Markdown
Contributor

PR Review

✅ No issues found.

@github-actions

Copy link
Copy Markdown
Contributor

Duplicate code analysis

No new duplicate code introduced by this PR.


jscpd compared this branch against origin/main
across app, api and common-utils. Only new clones are listed; the codebase
currently holds 108 in total (1.18%).
Run yarn dupes locally to reproduce, or yarn dupes:all for the full picture.

@github-actions

Copy link
Copy Markdown
Contributor

Deep Review

✅ No critical issues found.

This is a focused metadata change: it adds an OPEN_WORLD_UPSERT_TOOL annotation constant ({...UPSERT_TOOL, openWorldHint: true}) and applies it to clickstack_save_webhook and clickstack_save_alert, with a test pinning the open-world set to exactly those two tools. Reviewers verified the change is correct and complete:

  • The spread produces the intended annotation (openWorldHint: true, all other hints unchanged), and OPEN_WORLD_UPSERT_TOOL is the only annotation carrying openWorldHint: true.
  • The open-world boundary is complete: across all MCP mutation tools, only saveWebhook persists an arbitrary destination URL and only saveAlert schedules recurring requests to it. saveSource/saveSavedSearch/saveDashboard/patchDashboard write only MongoDB metadata or query ClickHouse (in-system), and deleteWebhook only stops deliveries — all correctly stay closed-world.
  • The new test uses exact sorted-set equality, so it catches both an omitted open-world tool and an accidentally-added one.
  • The changeset ('@hyperdx/api': patch) and the SCREAMING_SNAKE_CASE constant name follow existing repo conventions.

No P0/P1 or P2 findings. No actionable nitpicks.


Reviewers (5): correctness, testing, maintainability, security, project-standards.

Testing gaps: The open-world pin is only as complete as the tools surfaced in the default test context (client.listTools()); a future open-world tool registered only under a config/feature gate could be absent from that context and slip past the exact-set assertion. Not introduced by this diff.

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

E2E Test Results

✅ All tests passed • 463 passed • 1 skipped • 1153s

Status Count
✅ Passed 463
❌ Failed 0
⚠️ Flaky 3
⏭️ Skipped 1

Tests ran across 4 shards in parallel.

View full report →

@kodiakhq
kodiakhq Bot merged commit 1fba1c3 into main Nov 12, 2026
28 checks passed
@kodiakhq
kodiakhq Bot deleted the claude/mcp-open-world-alert-webhook branch November 12, 2026 20:47

This branch was successfully deployed

2 active deployments
Preview – hyperdx-storybook — 23f7d2a6 Deployed Nov 12, 2026 by vercel[bot]
Preview – hyperdx-oss — 23f7d2a6 Deployed Nov 12, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

automerge review/tier-2 Low risk — AI review + quick human skim

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants