Skip to content

feat: connect agents to the DataSQRL Cloud MCP server - #19

Merged
lavieennoir merged 8 commits into
mainfrom
feat/mcp-server
Nov 17, 2026
Merged

lavieennoir merged 8 commits into
mainfrom
feat/mcp-server

Conversation

@lavieennoir

@lavieennoir lavieennoir commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Merge only once prod serves /api/mcp. DataSQRL/cloud-frontend#1424 is merged, but it is not deployed to prod yet. Users install from main, so merging releases this. This check must answer 401, not 307:

curl -s -o /dev/null -w '%{http_code}\n' -X POST https://cloud.datasqrl.com/api/mcp

What does this change?

Connects the agent to the DataSQRL Cloud MCP server (<base URL>/api/mcp), so every Cloud API operation is a tool, with the token from the existing device-flow sign-in. Fixes #17.

  • Claude Code: HTTP server; headersHelper runs datasqrl-cloud.sh mcp-headers on connect, reconnect and 401.
  • Codex, Cursor: stdio bridge, datasqrl-cloud.sh mcp-bridge, adding a fresh token per request (retry once after a 401). Calls time out after 10s connect / 300s total.
  • Token refresh under a lock: Auth0 rotates refresh tokens, so concurrent refreshes would otherwise spend the same one. The refresh call gives up after 5s, a lock counts as stale by folder age, and a process removes only its own lock.
  • No secrets on command lines: tokens, URLs and request bodies reach curl and jq through stdin, a config file descriptor or the environment, never ps-visible arguments. This also lifts the 128 KiB argument limit on tool calls.
  • New cloud skill for everything beyond deploy, status/logs and promote. Every tool that is not read-only needs the user's yes first, even when named; clients may not prompt (e.g. Claude Code auto mode). Create-deployment tools route to deploy.
  • start routes to cloud and keeps Cloud tools inside the skills, which take precedence over the server's own instructions (now MCP-specific: DataSQRL/cloud-frontend#1454).
  • Routing and guardrail evals for cloud and deploy in tests/claude-plugin-evals.
  • deploy/promote keep the script; check-plugin.sh validates the MCP declarations; README and DATA_HANDLING updated.

Area

  • Skills
  • Launcher (scripts/datasqrl-agent.sh)
  • Cloud client (scripts/datasqrl-cloud.sh)
  • Manifests
  • CI
  • Docs

How did you test it?

  • tests/cloud/test_mcp.py against a fake Cloud: header helper, refresh and rotation, concurrent refresh, slow refresh (no lock takeover), stale lock, no secrets in curl/jq arguments, a 200 KB tool call, bridge relay, notifications, 401 retry, 429, errors, not signed in, unreachable host. Full suite passes.
  • Evals (tests/claude-plugin-evals/skill-specific-tests/{cloud,deploy}), 3 runs each, all 3/3: alerts reach cloud; status and logs stay with deploy; terminate with no tools stops without a curl workaround; an uncommitted change gets commit-and-push, not a content deploy.
  • Staging, live, after the review fixes and with the MCP-specific instructions (DataSQRL/cloud-frontend#1454):
    • Bridge: initialize serves the new instructions, 42 tools, read call.
    • Forced token expiry: 3 concurrent mcp-headers return the same new token, the lock is cleaned up.
    • cloud skill via claude -p: "delete project ship-17-gate" and "create secret …" both ask for a yes and attempt no write tool.
  • Staging, live, before the review fixes: Claude Code deploys two commits and promotes; in default mode Claude Code prompts for a destructive tool and the skill stops when declined; Codex installs the plugin and the bridge serves all 42 tools.
  • Not tested: Cursor (README says so); Codex approval behaviour with a signed-in Codex.

Checklist

  • The PR title starts with a type such as feat:, fix: or docs:
  • .github/check-plugin.sh passes
  • Skills contain no SQRL and no HTML comments
  • A launcher change that needs a new agent image option waits until that image is published
  • README or DATA_HANDLING.md is updated if behavior or data handling changed

- Claude Code: HTTP server with a headersHelper (datasqrl-cloud.sh mcp-headers)
- Codex, Cursor: stdio bridge (datasqrl-cloud.sh mcp-bridge) adding a fresh token per request
- Token refresh under a lock, so concurrent refreshes don't burn the rotated refresh token
- New cloud skill for everything beyond deploy/promote; destructive tools need the user's go-ahead
- check-plugin.sh validates the MCP declarations; docs updated

Fixes #17
@lavieennoir
lavieennoir marked this pull request as ready for review November 12, 2026 08:47
Comment thread scripts/datasqrl-cloud.sh Outdated
local waited=0
until mkdir "$REFRESH_LOCK" 2>/dev/null; do
if [ "$waited" -ge "$REFRESH_LOCK_WAIT_SECONDS" ]; then
rmdir "$REFRESH_LOCK" 2>/dev/null

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After 8 seconds this removes the lock, also when its holder is still refreshing. The refresh request() has no curl timeout, so a slow Auth0 call keeps the lock longer. Then two processes spend the same rotated refresh token, and release_refresh_lock in the first one removes the lock that the second one now holds.

Reproduced with tests/cloud/fake_cloud.py: two mcp-headers calls at once with a 9.5 s refresh give 2 refresh calls, and the second fails after 17.6 s with "Unknown or reused refresh token". With real Auth0 reuse detection, that signs the user out of every agent, and 17.6 s is also past the 10 s header-helper timeout.

Could you add --max-time 5 to the refresh call, treat a lock as stale by the age of the lock folder, remove only a lock this process created, and add a test with cloud.refresh_delay = 9?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 41b68d0. The refresh call has --max-time 5. A lock counts as stale only when its folder is older than 8s, and a live holder gives up by 5s. release_refresh_lock removes the lock only when its owner file matches this process. New tests, both failing on the previous script: test_a_slow_refresh_gives_up_and_its_lock_is_never_taken_over (refresh_delay = 9: the first call fails within 7s, and the second refresh is sent only after the first process exits) and test_a_lock_left_by_a_killed_process_is_taken_over.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow-up in 2b29f9c: CI now runs on Ubuntu 26.04, whose uutils coreutils mkdir (0.10.0) lets concurrent calls both succeed (432 successes in 300 rounds of 3 parallel mkdirs in an ubuntu:26.04 container). Two processes refreshed at once, and the concurrent test failed about 1 run in 15. The lock is now a file created with bash noclobber (O_EXCL, no external tool), dated by the creation time it holds, with the file mtime as fallback. A 26.04 stress run (6 processes, 40 rounds): the old lock double-spent the token, the new one never did. The concurrent test now runs 6 processes.

Comment thread scripts/datasqrl-cloud.sh Outdated
local -a curl_args=(-sS -X POST -w '\n%{http_code}'
-H 'Content-Type: application/json'
-H 'Accept: application/json, text/event-stream'
-H "Authorization: Bearer $ACCESS_TOKEN")

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mcp_post passes the bearer token (-H "Authorization: Bearer $ACCESS_TOKEN") and the whole MCP message (--data-binary "$1") as curl arguments. ps shows both to other local users, and the message is the tool input, so a createSecret call puts the secret value in the command line. Also, Linux limits one argument to 128 KiB, so a larger tool call fails to start curl and the bridge reports "cannot reach".

This keeps both out of the argument list and works on macOS /bin/bash 3.2:

response=$(printf '%s' "$1" | curl "${curl_args[@]}" \
    -H @<(printf 'Authorization: Bearer %s\n' "$ACCESS_TOKEN") \
    --data-binary @- "$BASE_URL/api/mcp" 2>/dev/null) \

Could you make this change here and in request(), and add a test for a body larger than 128 KiB?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 41b68d0, in request() and mcp_post. The body goes on stdin (--data-binary @-). The URL and the Authorization header go through -K <(curl_secrets ...): the URL too, because the device-code poll carries device_code in its query string. jq reads tokens from the environment (env.T) in write_cached, the refresh body and mcp-headers. New tests: test_tokens_never_appear_in_a_command_line (curl and jq shims log their arguments; no token or tool input appears) and test_relays_a_message_larger_than_one_command_line_argument_may_be (200 KB).

Comment thread skills/cloud/SKILL.md Outdated

# Step 2 — Call the tools

Read-only tools (listing, status, logs, metrics, alerts) can run straight away. Report what they

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The server marks six write tools with .destructive(false): createDeployment, createDeploymentFromContent, createProject, createSecret, createAlertRoute and snoozeAlert (see cloud-frontend staging). This skill has a rule for read-only tools and one for destructive tools, so these writes fit neither. createDeploymentFromContent also commits package.json content to the user's repository and deploys it, which skips the deploy skill's commit checks and makes skills/deploy/SKILL.md:240 ("Deploy uncommitted or unpushed code: Not possible") false. README.md:44 ("Tools that change or delete something are marked destructive") is false too.

Could you add a rule for non-destructive writes (say what it creates, then wait for a yes), route both create-deployment tools to the deploy skill, and fix the two sentences?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 41b68d0. Every tool that is not read-only (create, snooze, change, delete) now needs a yes first, and the skill says the server marks only some of them destructive. createDeployment and createDeploymentFromContent route to deploy, which now says never to deploy with an MCP tool and names the one that commits content. The deploy row now reads "Not offered: the user commits and pushes it". The README sentence is rewritten.

Comment thread DATA_HANDLING.md Outdated
- The DataSQRL Code Agent runs in a Docker container on your machine. It sends your prompts and project code to the model provider you choose, and to nothing else unless you opt in.
- Telemetry is off by default. With `DATASQRL_TELEMETRY=1`, the run log is uploaded to DataSQRL for troubleshooting.
- `deploy` and `promote` talk to DataSQRL Cloud, and only when you ask for them.
- `deploy`, `promote` and the DataSQRL Cloud MCP tools talk to DataSQRL Cloud. Once you are signed in, your coding agent also connects to the MCP server when a session starts, to list the tools. The `cloud` skill runs a tool marked destructive only when you named the action and its target, or agreed to it.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sentence reads as a guarantee, but only the skill text backs it. The tools are in the agent's tool list in every session, and the server sends its API description as MCP instructions in every session. That text says to deploy with POST /api/v1/deployments and promote with runDeploymentAction, while the deploy and promote skills say to use the script. runDeploymentAction is one tool for promote, stop, resize and terminate, so "always allow" on one promote in Claude Code also allows terminate.

Could you reword this as guidance (the agent's permission settings decide outside the skill), add a rule to skills/start/SKILL.md that Cloud tools run only through deploy, promote or cloud, and ask cloud-frontend for MCP instructions that do not contradict the skills? Also, issue #17 accepts "Terminate deployment X asks for approval first", while skills/cloud/SKILL.md:53-55 runs it with no question. Is that agreed?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 41b68d0. DATA_HANDLING now reads as guidance: the skills tell the agent to ask; outside them, the agent permission settings decide. start has a new rule: Cloud tools only through the skills, which take precedence over the server instructions. The README notes that "always allow" on runDeploymentAction also allows terminate. MCP instructions: DataSQRL/cloud-frontend#1447. On the #17 acceptance: agreed with the issue. The cloud skill now asks before every write, including "terminate deployment X" by name.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update: DataSQRL/cloud-frontend#1454 (merged, live on staging) replaces the OpenAPI overview with MCP-specific instructions. They name tools instead of HTTP paths, state what the write tools do (one runDeploymentAction for five actions, irreversible terminate, deleteProject terminating deployments, createDeploymentFromContent committing to the repository), and point clients with their own workflow, such as these skills, to it. They set no approval policy: that stays with the client. Checked live on staging: the bridge serves the new text, and the cloud skill asked before deleteProject and createSecret.

Comment thread skills/cloud/SKILL.md Outdated
@@ -0,0 +1,80 @@
---
name: cloud
description: Use when the user wants to look at or change something in DataSQRL Cloud other than deploying a commit or promoting a deployment - alerts, metrics, logs, deployment status, projects, secrets, members, or terminating, stopping, starting or resizing a deployment. Not for creating a deployment from a commit (the deploy skill) or making one the project's main deployment (the promote skill).

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This description claims "logs, deployment status, projects", but deploy also claims status and logs, and skills/start/SKILL.md:254 routes them to deploy. start has no row for cloud, so "show the alerts of deployment X" that enters through start has no route. The code-agent eval case deploy-check-status expects deploy for a status-and-logs request.

Could you give status and logs one owner (I suggest deploy), add a cloud row to the "Deploying to DataSQRL Cloud" table in skills/start/SKILL.md, and add cloud routing cases to code-agent/agent-evals/plugin_routing_evals.yaml?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 41b68d0. deploy owns status and logs: they are gone from the cloud description, and its routing table sends them to deploy. start routes everything else in DataSQRL Cloud to cloud, in its overview table and in the Deploying section. The code-agent eval cases are deferred to a follow-up.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update: the routing evals are now in this PR (6a769af), in the plugin repo own harness (tests/claude-plugin-evals/skill-specific-tests/{cloud,deploy}) next to the skills, instead of code-agent. Cases: alerts reach cloud; status and logs stay with deploy; terminate with no tools stops without a curl workaround; an uncommitted change gets commit-and-push. Each passed 3/3.

Comment thread scripts/datasqrl-cloud.sh Outdated

# Like request(), but a network failure sets HTTP_STATUS=000 instead of exiting.
mcp_post() {
local -a curl_args=(-sS -X POST -w '\n%{http_code}'

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mcp_post has no curl timeout, and the bridge handles one message at a time. One hung call blocks every call after it, and a notifications/cancelled from the host waits behind it, so it cannot stop the call. Could you add --connect-timeout 10 --max-time 300 here? The existing 000 branch then reports it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 41b68d0: --connect-timeout 10 --max-time 300. A timeout lands in the existing 000 branch.

@HasanAlpCaferoglu

Copy link
Copy Markdown
Collaborator

Merge order: this PR needs the MCP server on production first

Users install the plugin from main (.claude-plugin/marketplace.json has "source": "./"), so merging this PR releases it. There is no separate release step after the merge.

Status on 2026-11-12, 12:55 UTC:

  • Production has no /api/mcp route yet:
    $ curl -s -o /dev/null -w '%{http_code} -> %{redirect_url}' -X POST https://cloud.datasqrl.com/api/mcp
    307 -> https://cloud.datasqrl.com/
    
  • The MCP server (DataSQRL/cloud-frontend#1411) is on the cloud-frontend staging branch only.
  • DataSQRL/cloud-frontend#1424 ("Staging", staging -> main) is approved and contains the MCP server files. It is not merged yet.

After a merge now, Claude Code shows a failed datasqrl server for signed-in users, the bridge answers each request with "HTTP 307", and the deploy and promote skills send terminate and resize requests to a cloud skill with no tools.

Could you keep this PR unmerged (or mark it as a draft) until DataSQRL/cloud-frontend#1424 is merged and deployed? This check then answers 401:

curl -s -o /dev/null -w '%{http_code}\n' -X POST https://cloud.datasqrl.com/api/mcp

Could you also change "Release: only after" to "Merge only after" in the description?

- Token, URL and body reach curl through stdin and a config fd, tokens reach jq through env: nothing secret in ps, and tool calls over 128 KiB work
- Refresh gives up after 5s; a lock counts as stale by folder age; release removes only a lock this process owns
- Bridge calls time out (10s connect, 300s total)
- cloud skill asks before every write tool, destructive or not; status and logs stay with deploy; create-deployment tools go to deploy
- start routes to cloud and keeps Cloud tools inside the skills; docs no longer promise a client prompt
# Conflicts:
#	.claude-plugin/plugin.json
#	.codex-plugin/plugin.json
#	.cursor-plugin/plugin.json
#	skills/start/SKILL.md
@lavieennoir

Copy link
Copy Markdown
Contributor Author

Review round 1 — dispositions

Merge order: agreed. The description now starts with "Merge only after DataSQRL/cloud-frontend#1424 is merged and deployed to prod", with your curl check. Prod still answers 307.

# Finding Decision Where
1 Merge before prod has /api/mcp KEEP description gate; not merging until the check answers 401
2 Lock taken over while a slow refresh holds it KEEP 41b68d0: --max-time 5, stale by folder age, release only own lock; 2 tests
3 Token and tool input in curl arguments; 128 KiB limit KEEP 41b68d0: stdin + -K fd for curl, env for jq, also in request(); 2 tests
4 Non-destructive writes fit no rule; createDeploymentFromContent skips deploy KEEP 41b68d0: every write asks first; create-deployment tools go to deploy
5 DATA_HANDLING reads as a guarantee; MCP instructions contradict the skills; #17 acceptance KEEP 41b68d0: guidance wording, start rule, README note; DataSQRL/cloud-frontend#1447; skill now asks before every write
6 Status and logs claimed by deploy and cloud; start has no cloud route KEEP 41b68d0: deploy owns them; start routes to cloud. code-agent eval cases deferred
7 No curl timeout in the bridge KEEP 41b68d0: --connect-timeout 10 --max-time 300

Also merged main (#14) into the branch: one conflict in skills/start/SKILL.md, resolved by keeping #14's table and adding the cloud route. check-plugin.sh and the full test suite (173) pass.

Not verified live: the review fixes on staging. The tests above cover the transport; the always-ask rule is a skill text change.

Owner file and reader use the same clock; the folder mtime stays the fallback for a holder killed before writing it. Warn when taking over a lock; the slow-refresh test reports its timeline.
uutils coreutils mkdir (Ubuntu 26.04, the CI image) lets concurrent calls both succeed, so two processes refreshed at once. bash noclobber creates the lock file with O_EXCL and needs no external tool. Drops the temporary CI traces.
Alerts reach cloud; status and logs stay with deploy; terminate with no tools stops without a curl workaround; an uncommitted change gets commit-and-push, not a content deploy. Each passed 3/3. start no longer quotes the server's old instructions.
@lavieennoir
lavieennoir merged commit cb9a0c5 into main Nov 17, 2026
4 checks passed
@lavieennoir
lavieennoir deleted the feat/mcp-server branch November 17, 2026 12:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Use the DataSQRL Cloud MCP server for the full API

2 participants