Repository navigation
feat: connect agents to the DataSQRL Cloud MCP server - #19
Conversation
- Claude Code: HTTP server with a headersHelper (datasqrl-cloud.sh mcp-headers) - Codex, Cursor: stdio bridge (datasqrl-cloud.sh mcp-bridge) adding a fresh token per request - Token refresh under a lock, so concurrent refreshes don't burn the rotated refresh token - New cloud skill for everything beyond deploy/promote; destructive tools need the user's go-ahead - check-plugin.sh validates the MCP declarations; docs updated Fixes #17
| local waited=0 | ||
| until mkdir "$REFRESH_LOCK" 2>/dev/null; do | ||
| if [ "$waited" -ge "$REFRESH_LOCK_WAIT_SECONDS" ]; then | ||
| rmdir "$REFRESH_LOCK" 2>/dev/null |
There was a problem hiding this comment.
After 8 seconds this removes the lock, also when its holder is still refreshing. The refresh request() has no curl timeout, so a slow Auth0 call keeps the lock longer. Then two processes spend the same rotated refresh token, and release_refresh_lock in the first one removes the lock that the second one now holds.
Reproduced with tests/cloud/fake_cloud.py: two mcp-headers calls at once with a 9.5 s refresh give 2 refresh calls, and the second fails after 17.6 s with "Unknown or reused refresh token". With real Auth0 reuse detection, that signs the user out of every agent, and 17.6 s is also past the 10 s header-helper timeout.
Could you add --max-time 5 to the refresh call, treat a lock as stale by the age of the lock folder, remove only a lock this process created, and add a test with cloud.refresh_delay = 9?
There was a problem hiding this comment.
Fixed in 41b68d0. The refresh call has --max-time 5. A lock counts as stale only when its folder is older than 8s, and a live holder gives up by 5s. release_refresh_lock removes the lock only when its owner file matches this process. New tests, both failing on the previous script: test_a_slow_refresh_gives_up_and_its_lock_is_never_taken_over (refresh_delay = 9: the first call fails within 7s, and the second refresh is sent only after the first process exits) and test_a_lock_left_by_a_killed_process_is_taken_over.
There was a problem hiding this comment.
Follow-up in 2b29f9c: CI now runs on Ubuntu 26.04, whose uutils coreutils mkdir (0.10.0) lets concurrent calls both succeed (432 successes in 300 rounds of 3 parallel mkdirs in an ubuntu:26.04 container). Two processes refreshed at once, and the concurrent test failed about 1 run in 15. The lock is now a file created with bash noclobber (O_EXCL, no external tool), dated by the creation time it holds, with the file mtime as fallback. A 26.04 stress run (6 processes, 40 rounds): the old lock double-spent the token, the new one never did. The concurrent test now runs 6 processes.
| local -a curl_args=(-sS -X POST -w '\n%{http_code}' | ||
| -H 'Content-Type: application/json' | ||
| -H 'Accept: application/json, text/event-stream' | ||
| -H "Authorization: Bearer $ACCESS_TOKEN") |
There was a problem hiding this comment.
mcp_post passes the bearer token (-H "Authorization: Bearer $ACCESS_TOKEN") and the whole MCP message (--data-binary "$1") as curl arguments. ps shows both to other local users, and the message is the tool input, so a createSecret call puts the secret value in the command line. Also, Linux limits one argument to 128 KiB, so a larger tool call fails to start curl and the bridge reports "cannot reach".
This keeps both out of the argument list and works on macOS /bin/bash 3.2:
response=$(printf '%s' "$1" | curl "${curl_args[@]}" \
-H @<(printf 'Authorization: Bearer %s\n' "$ACCESS_TOKEN") \
--data-binary @- "$BASE_URL/api/mcp" 2>/dev/null) \Could you make this change here and in request(), and add a test for a body larger than 128 KiB?
There was a problem hiding this comment.
Fixed in 41b68d0, in request() and mcp_post. The body goes on stdin (--data-binary @-). The URL and the Authorization header go through -K <(curl_secrets ...): the URL too, because the device-code poll carries device_code in its query string. jq reads tokens from the environment (env.T) in write_cached, the refresh body and mcp-headers. New tests: test_tokens_never_appear_in_a_command_line (curl and jq shims log their arguments; no token or tool input appears) and test_relays_a_message_larger_than_one_command_line_argument_may_be (200 KB).
|
|
||
| # Step 2 — Call the tools | ||
|
|
||
| Read-only tools (listing, status, logs, metrics, alerts) can run straight away. Report what they |
There was a problem hiding this comment.
The server marks six write tools with .destructive(false): createDeployment, createDeploymentFromContent, createProject, createSecret, createAlertRoute and snoozeAlert (see cloud-frontend staging). This skill has a rule for read-only tools and one for destructive tools, so these writes fit neither. createDeploymentFromContent also commits package.json content to the user's repository and deploys it, which skips the deploy skill's commit checks and makes skills/deploy/SKILL.md:240 ("Deploy uncommitted or unpushed code: Not possible") false. README.md:44 ("Tools that change or delete something are marked destructive") is false too.
Could you add a rule for non-destructive writes (say what it creates, then wait for a yes), route both create-deployment tools to the deploy skill, and fix the two sentences?
There was a problem hiding this comment.
Fixed in 41b68d0. Every tool that is not read-only (create, snooze, change, delete) now needs a yes first, and the skill says the server marks only some of them destructive. createDeployment and createDeploymentFromContent route to deploy, which now says never to deploy with an MCP tool and names the one that commits content. The deploy row now reads "Not offered: the user commits and pushes it". The README sentence is rewritten.
| - The DataSQRL Code Agent runs in a Docker container on your machine. It sends your prompts and project code to the model provider you choose, and to nothing else unless you opt in. | ||
| - Telemetry is off by default. With `DATASQRL_TELEMETRY=1`, the run log is uploaded to DataSQRL for troubleshooting. | ||
| - `deploy` and `promote` talk to DataSQRL Cloud, and only when you ask for them. | ||
| - `deploy`, `promote` and the DataSQRL Cloud MCP tools talk to DataSQRL Cloud. Once you are signed in, your coding agent also connects to the MCP server when a session starts, to list the tools. The `cloud` skill runs a tool marked destructive only when you named the action and its target, or agreed to it. |
There was a problem hiding this comment.
This sentence reads as a guarantee, but only the skill text backs it. The tools are in the agent's tool list in every session, and the server sends its API description as MCP instructions in every session. That text says to deploy with POST /api/v1/deployments and promote with runDeploymentAction, while the deploy and promote skills say to use the script. runDeploymentAction is one tool for promote, stop, resize and terminate, so "always allow" on one promote in Claude Code also allows terminate.
Could you reword this as guidance (the agent's permission settings decide outside the skill), add a rule to skills/start/SKILL.md that Cloud tools run only through deploy, promote or cloud, and ask cloud-frontend for MCP instructions that do not contradict the skills? Also, issue #17 accepts "Terminate deployment X asks for approval first", while skills/cloud/SKILL.md:53-55 runs it with no question. Is that agreed?
There was a problem hiding this comment.
Fixed in 41b68d0. DATA_HANDLING now reads as guidance: the skills tell the agent to ask; outside them, the agent permission settings decide. start has a new rule: Cloud tools only through the skills, which take precedence over the server instructions. The README notes that "always allow" on runDeploymentAction also allows terminate. MCP instructions: DataSQRL/cloud-frontend#1447. On the #17 acceptance: agreed with the issue. The cloud skill now asks before every write, including "terminate deployment X" by name.
There was a problem hiding this comment.
Update: DataSQRL/cloud-frontend#1454 (merged, live on staging) replaces the OpenAPI overview with MCP-specific instructions. They name tools instead of HTTP paths, state what the write tools do (one runDeploymentAction for five actions, irreversible terminate, deleteProject terminating deployments, createDeploymentFromContent committing to the repository), and point clients with their own workflow, such as these skills, to it. They set no approval policy: that stays with the client. Checked live on staging: the bridge serves the new text, and the cloud skill asked before deleteProject and createSecret.
| @@ -0,0 +1,80 @@ | |||
| --- | |||
| name: cloud | |||
| description: Use when the user wants to look at or change something in DataSQRL Cloud other than deploying a commit or promoting a deployment - alerts, metrics, logs, deployment status, projects, secrets, members, or terminating, stopping, starting or resizing a deployment. Not for creating a deployment from a commit (the deploy skill) or making one the project's main deployment (the promote skill). | |||
There was a problem hiding this comment.
This description claims "logs, deployment status, projects", but deploy also claims status and logs, and skills/start/SKILL.md:254 routes them to deploy. start has no row for cloud, so "show the alerts of deployment X" that enters through start has no route. The code-agent eval case deploy-check-status expects deploy for a status-and-logs request.
Could you give status and logs one owner (I suggest deploy), add a cloud row to the "Deploying to DataSQRL Cloud" table in skills/start/SKILL.md, and add cloud routing cases to code-agent/agent-evals/plugin_routing_evals.yaml?
There was a problem hiding this comment.
Fixed in 41b68d0. deploy owns status and logs: they are gone from the cloud description, and its routing table sends them to deploy. start routes everything else in DataSQRL Cloud to cloud, in its overview table and in the Deploying section. The code-agent eval cases are deferred to a follow-up.
There was a problem hiding this comment.
Update: the routing evals are now in this PR (6a769af), in the plugin repo own harness (tests/claude-plugin-evals/skill-specific-tests/{cloud,deploy}) next to the skills, instead of code-agent. Cases: alerts reach cloud; status and logs stay with deploy; terminate with no tools stops without a curl workaround; an uncommitted change gets commit-and-push. Each passed 3/3.
|
|
||
| # Like request(), but a network failure sets HTTP_STATUS=000 instead of exiting. | ||
| mcp_post() { | ||
| local -a curl_args=(-sS -X POST -w '\n%{http_code}' |
There was a problem hiding this comment.
mcp_post has no curl timeout, and the bridge handles one message at a time. One hung call blocks every call after it, and a notifications/cancelled from the host waits behind it, so it cannot stop the call. Could you add --connect-timeout 10 --max-time 300 here? The existing 000 branch then reports it.
There was a problem hiding this comment.
Fixed in 41b68d0: --connect-timeout 10 --max-time 300. A timeout lands in the existing 000 branch.
|
Merge order: this PR needs the MCP server on production first Users install the plugin from Status on 2026-11-12, 12:55 UTC:
After a merge now, Claude Code shows a failed Could you keep this PR unmerged (or mark it as a draft) until DataSQRL/cloud-frontend#1424 is merged and deployed? This check then answers curl -s -o /dev/null -w '%{http_code}\n' -X POST https://cloud.datasqrl.com/api/mcpCould you also change "Release: only after" to "Merge only after" in the description? |
- Token, URL and body reach curl through stdin and a config fd, tokens reach jq through env: nothing secret in ps, and tool calls over 128 KiB work - Refresh gives up after 5s; a lock counts as stale by folder age; release removes only a lock this process owns - Bridge calls time out (10s connect, 300s total) - cloud skill asks before every write tool, destructive or not; status and logs stay with deploy; create-deployment tools go to deploy - start routes to cloud and keeps Cloud tools inside the skills; docs no longer promise a client prompt
# Conflicts: # .claude-plugin/plugin.json # .codex-plugin/plugin.json # .cursor-plugin/plugin.json # skills/start/SKILL.md
Review round 1 — dispositionsMerge order: agreed. The description now starts with "Merge only after DataSQRL/cloud-frontend#1424 is merged and deployed to prod", with your curl check. Prod still answers
Also merged Not verified live: the review fixes on staging. The tests above cover the transport; the always-ask rule is a skill text change. |
Owner file and reader use the same clock; the folder mtime stays the fallback for a holder killed before writing it. Warn when taking over a lock; the slow-refresh test reports its timeline.
uutils coreutils mkdir (Ubuntu 26.04, the CI image) lets concurrent calls both succeed, so two processes refreshed at once. bash noclobber creates the lock file with O_EXCL and needs no external tool. Drops the temporary CI traces.
Alerts reach cloud; status and logs stay with deploy; terminate with no tools stops without a curl workaround; an uncommitted change gets commit-and-push, not a content deploy. Each passed 3/3. start no longer quotes the server's old instructions.
What does this change?
Connects the agent to the DataSQRL Cloud MCP server (
<base URL>/api/mcp), so every Cloud API operation is a tool, with the token from the existing device-flow sign-in. Fixes #17.headersHelperrunsdatasqrl-cloud.sh mcp-headerson connect, reconnect and 401.datasqrl-cloud.sh mcp-bridge, adding a fresh token per request (retry once after a 401). Calls time out after 10s connect / 300s total.curlandjqthrough stdin, a config file descriptor or the environment, neverps-visible arguments. This also lifts the 128 KiB argument limit on tool calls.cloudskill for everything beyond deploy, status/logs and promote. Every tool that is not read-only needs the user's yes first, even when named; clients may not prompt (e.g. Claude Code auto mode). Create-deployment tools route todeploy.startroutes tocloudand keeps Cloud tools inside the skills, which take precedence over the server's owninstructions(now MCP-specific: DataSQRL/cloud-frontend#1454).cloudanddeployintests/claude-plugin-evals.deploy/promotekeep the script;check-plugin.shvalidates the MCP declarations; README and DATA_HANDLING updated.Area
scripts/datasqrl-agent.sh)scripts/datasqrl-cloud.sh)How did you test it?
tests/cloud/test_mcp.pyagainst a fake Cloud: header helper, refresh and rotation, concurrent refresh, slow refresh (no lock takeover), stale lock, no secrets incurl/jqarguments, a 200 KB tool call, bridge relay, notifications, 401 retry, 429, errors, not signed in, unreachable host. Full suite passes.tests/claude-plugin-evals/skill-specific-tests/{cloud,deploy}), 3 runs each, all 3/3: alerts reachcloud; status and logs stay withdeploy; terminate with no tools stops without acurlworkaround; an uncommitted change gets commit-and-push, not a content deploy.mcp-headersreturn the same new token, the lock is cleaned up.cloudskill viaclaude -p: "delete project ship-17-gate" and "create secret …" both ask for a yes and attempt no write tool.Checklist
feat:,fix:ordocs:.github/check-plugin.shpasses