Skip to content

Add a synth-miner Claude Code skill and a one-go GCP miner deploy - #351

Open
Thykof wants to merge 4 commits into
mainfrom
docs/synth-miner-skill
Open

Thykof wants to merge 4 commits into
mainfrom
docs/synth-miner-skill

Conversation

@Thykof

@Thykof Thykof commented Oct 27, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Adds a Claude Code skill at .claude/skills/synth-miner/. An AI coding agent opened in a synth-subnet checkout, or pointed at it with --add-dir, picks it up and can then set up, deploy, debug and improve a Synth miner.

It also adds a one-go deploy: from a laptop with an authenticated gcloud and a funded wallet to a registered, serving miner on a GCP VM that runs a published autoquant champion (or the stock miner), on mainnet or testnet. For it, miner-setup/terraform gains a static IP, an install-once startup script, a systemd unit and a champion mode.

What's in it

The skill

  • SKILL.md: the job, the three competitions, routing to the references, ground rules, what happens after registration, the response contract, scoring essentials and a quick diagnosis table.
  • references/:
    • setup.md: host and network, install, wallet split between human and agent, registration, immunity, testnet;
    • deployment.md: PM2, Docker, Terraform/Ansible, and synth-lib's ChampionMiner; firewall, health checks, updates;
    • gcp-end-to-end.md: the one-go deploy runbook (below);
    • protocol.md: request and response contract, every validation message, where to plug a model in, anchoring, blacklist, concurrency;
    • scoring.md: CRPS intervals, the volatility term, prompt scores, backfill, softmax, emission split, change timeline;
    • model-and-backtesting.md: market data, generating without lookahead, the backtester, diagnosing vol_crps, rules that keep results honest;
    • api-and-monitoring.md, troubleshooting.md, synth-ultra.md.
  • scripts/ (each runs with uv run --script or plain python3):
    • miner_status.py: standard library only. Shows a UID's latest validation, its standings per competition, its Synth Ultra rows and, with --scores, misses per asset. It paces its requests.
    • check_responses.py: runs a generate_simulations-style function for every competition asset at 1000 paths through the validator's validate_responses, and flags prices ≤ 0 and calls over 1 s.
    • register.py: registers a hotkey with register_limit, a price ceiling the chain enforces, and the coldkey password read from a 0600 file. It never prompts, prints only public data, and never registers twice. --check reports the addresses, registration, price, balance and published axon without the password. A PEP 723 header pins bittensor.
    • pick_champion.py: picks the best eligible autoquant champion from the published 30-day verdicts, or validates a named one, and prints the two Terraform values.

The one-go deploy (references/gcp-end-to-end.md)

  1. Preflight: tools, GCP project, wallet files, password file, balance. Installs uv and Terraform under ~/.synth-miner/bin if missing.
  2. Picks the model.
  3. Creates the VM with miner-setup/terraform/gcp and waits for provisioning.
  4. Copies coldkeypub.txt and the hotkey (never the coldkey), then reboots to prove the miner starts, loads its model and reaches the chain. A champion then pre-warms its market data and event calendar with synth-miner-prewarm.service, so the miner serves almost at once after registration instead of missing prompts while it fetches them. All of this happens before any TAO is spent.
  5. Registers under the ceiling, starts the miner at once, and checks it from the VM, the laptop and the chain, and on mainnet from the public API and the dashboard.

The network, netuid, wallets folder, wallet and hotkey names, password file and GCP project are inputs. On testnet, where the public API does not report, the runbook proves an answer from the miner log against the validators its blacklist lets through.

It runs only in a local Claude Code session (the CLI, or a local session in the Claude Desktop app's Code tab), since it needs the laptop's gcloud, wallet and SSH key.

miner-setup/terraform (GCP, and AWS through the same startup script)

  • Static IP, published as the axon's external IP.
  • Startup script that installs once and records provisioned or FAILED at <step>; every step can run again.
  • synth-miner.service (systemd), started once the hotkey is on the VM, and synth-miner-prewarm.service (oneshot, run by hand before registration).
  • Stock mode (synth-subnet's neurons/miner.py) or champion mode (an autoquant champion unpacked with synth-lib, which runs its contract gate at install).
  • A random benchmark seed per VM, kept for its life.
  • New variables only, all with defaults, and input validation: existing terraform.tfvars keep working. miner-setup/README.md documents them.

Verification

  • Testnet, end to end: steps 1 to 7 on netuid 247 with a published champion. The VM provisioned, the reboot rehearsal passed, registration went through under the ceiling, the axon was on chain at the static IP, the cold warm-up took 5.5 minutes, the first validator request came 13 minutes after the start, and the error filter was clean. The testnet fixes in the last commit came from that run.
  • Pre-warm: the startup script was rendered with terraform console in both modes, bash -n passes, and run.sh --prewarm was exercised with stubs. It has not yet run on a real VM.
  • Not yet run on mainnet.
  • Lint: CI's black and flake8 cover .claude/, and the scripts pass with the repo config.

Notes for reviewers

  • Ground rules:
    • coldkeys, mnemonics and passwords never enter the conversation: the coldkey stays on the laptop, register.py reads the password from a file the human writes in their own terminal, and wallet creation stays with the human;
    • asking for the one-go deploy with a funded wallet and the password file in place is the consent to create the VM and register within the ceiling; other TAO spending and outward-facing actions need explicit confirmation;
    • the public API is paced.
  • Sources: facts come from main of synth-subnet, synth-lib, synth-ultra and autoquant, checked against the code. Where docs and code disagree, the skill follows the code and says so: "at least 1 minute to answer", the p90 rule, 1/3 per competition, "0.25 TAO is enough".
  • Figures that change (registration price and its hyperparameters, the immunity period, btcli behaviour) are dated, next to commands that read them live.
  • Pre-warm and synth-lib: the startup script carries a small prewarm.py against the synth-lib that autoquant pins today. Serving: a champion can prewarm its market data before registration; crypto-1h warms first synth-lib#97 adds entrypoint.sh --prewarm (and warms the crypto-1h assets first); once autoquant pins it, run.sh can call that instead.
  • Outside .claude/, only miner-setup/ changes. No action is needed for running miners.

Doc issues the skill works around

Candidates for follow-ups:

  • miner.test.config.js passes --blacklist.validator.min_stake 0, a misspelled flag that is silently ignored, so the 65000 floor still applies on testnet.
  • The docs' pm2 start miner.config.js -- <flags> examples change nothing: PM2 drops arguments after -- when starting from a config file.
  • --strict rejects valid miner flags such as --blacklist.validator_min_stake.
  • docs/miner_reference.md passes --neuron.events_retention_size 2147483648, which crashes startup.
  • A bare --subtensor.chain_endpoint is overridden by the default network, so miner.dev.config.js reaches finney.
  • The tutorial and provisioning use ufw allow 8091 without a protocol, which chain_sync.py cannot remove.
  • On testnet, the base miner's Serving miner axon … on network: line prints finney's default endpoint, not the chain in use.

🤖 Generated with Claude Code

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.

Tip: disable this comment in your organization's Code Review settings.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Status reporting can misattribute recycled-UID history, and malformed model output can terminate the response checker before the remaining assets are checked.

3 open issues
What changed in this PR

Adds a Claude Code skill for setting up, deploying, debugging and improving Synth miners, without changing the existing miner runtime.

Changes:

  • Defines skill routing, operational safeguards and miner protocol guidance.
  • Adds references covering deployment, scoring, backtesting, monitoring and Synth Ultra.
  • Provides scripts for paced status queries and response validation across competition assets.
File Description
.claude/​skills/​synth-miner/​SKILL.md Defines the skill, safeguards and reference routing.
.claude/​skills/​synth-miner/​scripts/​miner_status.py Reports validation, standings and per-asset scores.
.claude/​skills/​synth-miner/​scripts/​check_responses.py Checks model responses, prices and latency.
.claude/​skills/​synth-miner/​references/​troubleshooting.md Maps operational symptoms to fixes.
.claude/​skills/​synth-miner/​references/​synth-ultra.md Documents Ultra submission, contracts and scoring.
.claude/​skills/​synth-miner/​references/​setup.md Covers installation, wallets and registration.
.claude/​skills/​synth-miner/​references/​scoring.md Explains scoring and reward allocation.
.claude/​skills/​synth-miner/​references/​protocol.md Details response requirements and model integration.
.claude/​skills/​synth-miner/​references/​model-and-backtesting.md Guides data preparation and model evaluation.
.claude/​skills/​synth-miner/​references/​deployment.md Documents deployment options and health checks.
.claude/​skills/​synth-miner/​references/​api-and-monitoring.md Describes public APIs and monitoring workflows.

Review effort: Balanced

time.perf_counter() - began,
)
elapsed = time.perf_counter() - began
verdict = validate_responses(response, simulation_input, f"{elapsed:.3f}")
Comment on lines +192 to +194
misses = sum(1 for row in rows if row["crps"] == -1)
mean = (
sum(row["prompt_score"] for row in rows) / len(rows)
# ISO start times); keep --cadence-minutes, which is still required but then ignored
```

- **No lookahead.** The generator ends each context at `context_end`: `start − 120 s` for 1h
Thykof and others added 2 commits November 13, 2026 12:26
A skill in .claude/skills/synth-miner/ that teaches an AI coding agent to
set up, deploy, debug and improve a Synth miner: wallets and
registration, serving the axon (PM2, Docker, Terraform/Ansible, synth-lib
serving), the response contract, scoring, backtesting with synth-lib, the
public APIs, troubleshooting and Synth Ultra.

Two stdlib/validator-backed helper scripts: miner_status.py reports a
UID's status from the public APIs, and check_responses.py validates a
generate function on every competition shape at 1000 paths.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adds a runbook that takes a laptop with an authenticated gcloud and a
funded wallet to a registered, serving SN50 miner on a GCP VM, followed
until its first leaderboard row, from a Claude Code session (the CLI or a
local session in the Claude Desktop app's Code tab).

- scripts/register.py: coldkey-only register_limit with a chain-enforced
  price ceiling and the password read from a 0600 file; a PEP 723 header
  pins bittensor so it runs with `uv run --script` from any directory.
- scripts/pick_champion.py: picks the best eligible autoquant champion
  from the published 30-day verdicts.
- miner-setup/terraform: static IP, startup-script metadata with an
  install-once guard, a systemd unit, stock or champion mode, a random
  benchmark seed per VM, and input validation.
- The miner is rehearsed with a reboot before any TAO is spent.
- References: v1.15.0 cadence, axon error-line and warm-up patterns,
  and check_responses.py's 1 s budget.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Thykof
Thykof force-pushed the docs/synth-miner-skill branch from 42f2515 to bff2622 Compare November 13, 2026 11:26
Thykof and others added 2 commits November 18, 2026 19:19
A champion fetched its minute history and event calendar only after
registration, so its first prompts failed while it warmed up, and each
failed prompt scores at the field's 95th percentile.

cloud-init writes synth-miner-prewarm.service, a oneshot unit started by
hand that runs `run.sh --prewarm`: /opt/synth-miner/prewarm.py fills
market_data/ with synth-lib's own warm_up and calendar build, with no
wallet and no chain. Runbook step 5 runs it after the reboot rehearsal
and checks every asset warmed up; registration follows within 24 hours,
while the stored calendar is still served at start. A registered hotkey
skips it: that miner must start at once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The runbook hardcoded mainnet and the default wallet layout. It now takes
the network and netuid, the wallets folder (btcli's wallet_path), wallet
and hotkey names, password file and GCP project as inputs, and passes them
to register.py, Terraform and every gcloud command.

On testnet the public API and dashboard are unavailable, and the axon logs
answered requests only at trace level. Step 7.6 gets a testnet check: a
Requesting UID line from a validator the blacklist lets through, listed by
an SDK one-liner, with the error filter empty. Step 8 is mainnet only.

Troubleshooting adds the low real-bar coverage warning and the finney
endpoint echoed by the axon log on testnet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Thykof Thykof changed the title docs: add a synth-miner Claude Code skill Add a synth-miner Claude Code skill and a one-go GCP miner deploy Nov 18, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants