Repository navigation
Conversation
There was a problem hiding this comment.
Claude Code Review
This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.
Tip: disable this comment in your organization's Code Review settings.
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Status reporting can misattribute recycled-UID history, and malformed model output can terminate the response checker before the remaining assets are checked.
3 open issues
What changed in this PR
Adds a Claude Code skill for setting up, deploying, debugging and improving Synth miners, without changing the existing miner runtime.
Changes:
- Defines skill routing, operational safeguards and miner protocol guidance.
- Adds references covering deployment, scoring, backtesting, monitoring and Synth Ultra.
- Provides scripts for paced status queries and response validation across competition assets.
| File | Description |
|---|---|
.claude/skills/synth-miner/SKILL.md |
Defines the skill, safeguards and reference routing. |
.claude/skills/synth-miner/scripts/miner_status.py |
Reports validation, standings and per-asset scores. |
.claude/skills/synth-miner/scripts/check_responses.py |
Checks model responses, prices and latency. |
.claude/skills/synth-miner/references/troubleshooting.md |
Maps operational symptoms to fixes. |
.claude/skills/synth-miner/references/synth-ultra.md |
Documents Ultra submission, contracts and scoring. |
.claude/skills/synth-miner/references/setup.md |
Covers installation, wallets and registration. |
.claude/skills/synth-miner/references/scoring.md |
Explains scoring and reward allocation. |
.claude/skills/synth-miner/references/protocol.md |
Details response requirements and model integration. |
.claude/skills/synth-miner/references/model-and-backtesting.md |
Guides data preparation and model evaluation. |
.claude/skills/synth-miner/references/deployment.md |
Documents deployment options and health checks. |
.claude/skills/synth-miner/references/api-and-monitoring.md |
Describes public APIs and monitoring workflows. |
Review effort: Balanced
| time.perf_counter() - began, | ||
| ) | ||
| elapsed = time.perf_counter() - began | ||
| verdict = validate_responses(response, simulation_input, f"{elapsed:.3f}") |
Comment on lines
+192
to
+194
| misses = sum(1 for row in rows if row["crps"] == -1) | ||
| mean = ( | ||
| sum(row["prompt_score"] for row in rows) / len(rows) |
| # ISO start times); keep --cadence-minutes, which is still required but then ignored | ||
| ``` | ||
|
|
||
| - **No lookahead.** The generator ends each context at `context_end`: `start − 120 s` for 1h |
A skill in .claude/skills/synth-miner/ that teaches an AI coding agent to set up, deploy, debug and improve a Synth miner: wallets and registration, serving the axon (PM2, Docker, Terraform/Ansible, synth-lib serving), the response contract, scoring, backtesting with synth-lib, the public APIs, troubleshooting and Synth Ultra. Two stdlib/validator-backed helper scripts: miner_status.py reports a UID's status from the public APIs, and check_responses.py validates a generate function on every competition shape at 1000 paths. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adds a runbook that takes a laptop with an authenticated gcloud and a funded wallet to a registered, serving SN50 miner on a GCP VM, followed until its first leaderboard row, from a Claude Code session (the CLI or a local session in the Claude Desktop app's Code tab). - scripts/register.py: coldkey-only register_limit with a chain-enforced price ceiling and the password read from a 0600 file; a PEP 723 header pins bittensor so it runs with `uv run --script` from any directory. - scripts/pick_champion.py: picks the best eligible autoquant champion from the published 30-day verdicts. - miner-setup/terraform: static IP, startup-script metadata with an install-once guard, a systemd unit, stock or champion mode, a random benchmark seed per VM, and input validation. - The miner is rehearsed with a reboot before any TAO is spent. - References: v1.15.0 cadence, axon error-line and warm-up patterns, and check_responses.py's 1 s budget. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Thykof
force-pushed
the
docs/synth-miner-skill
branch
from
November 13, 2026 11:26
42f2515 to
bff2622
Compare
A champion fetched its minute history and event calendar only after registration, so its first prompts failed while it warmed up, and each failed prompt scores at the field's 95th percentile. cloud-init writes synth-miner-prewarm.service, a oneshot unit started by hand that runs `run.sh --prewarm`: /opt/synth-miner/prewarm.py fills market_data/ with synth-lib's own warm_up and calendar build, with no wallet and no chain. Runbook step 5 runs it after the reboot rehearsal and checks every asset warmed up; registration follows within 24 hours, while the stored calendar is still served at start. A registered hotkey skips it: that miner must start at once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The runbook hardcoded mainnet and the default wallet layout. It now takes the network and netuid, the wallets folder (btcli's wallet_path), wallet and hotkey names, password file and GCP project as inputs, and passes them to register.py, Terraform and every gcloud command. On testnet the public API and dashboard are unavailable, and the axon logs answered requests only at trace level. Step 7.6 gets a testnet check: a Requesting UID line from a validator the blacklist lets through, listed by an SDK one-liner, with the error filter empty. Step 8 is mainnet only. Troubleshooting adds the low real-bar coverage warning and the finney endpoint echoed by the axon log on testnet. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.


Summary
Adds a Claude Code skill at
.claude/skills/synth-miner/. An AI coding agent opened in a synth-subnet checkout, or pointed at it with--add-dir, picks it up and can then set up, deploy, debug and improve a Synth miner.It also adds a one-go deploy: from a laptop with an authenticated gcloud and a funded wallet to a registered, serving miner on a GCP VM that runs a published autoquant champion (or the stock miner), on mainnet or testnet. For it,
miner-setup/terraformgains a static IP, an install-once startup script, a systemd unit and a champion mode.What's in it
The skill
SKILL.md: the job, the three competitions, routing to the references, ground rules, what happens after registration, the response contract, scoring essentials and a quick diagnosis table.references/:setup.md: host and network, install, wallet split between human and agent, registration, immunity, testnet;deployment.md: PM2, Docker, Terraform/Ansible, and synth-lib'sChampionMiner; firewall, health checks, updates;gcp-end-to-end.md: the one-go deploy runbook (below);protocol.md: request and response contract, every validation message, where to plug a model in, anchoring, blacklist, concurrency;scoring.md: CRPS intervals, the volatility term, prompt scores, backfill, softmax, emission split, change timeline;model-and-backtesting.md: market data, generating without lookahead, the backtester, diagnosingvol_crps, rules that keep results honest;api-and-monitoring.md,troubleshooting.md,synth-ultra.md.scripts/(each runs withuv run --scriptor plainpython3):miner_status.py: standard library only. Shows a UID's latest validation, its standings per competition, its Synth Ultra rows and, with--scores, misses per asset. It paces its requests.check_responses.py: runs agenerate_simulations-style function for every competition asset at 1000 paths through the validator'svalidate_responses, and flags prices ≤ 0 and calls over 1 s.register.py: registers a hotkey withregister_limit, a price ceiling the chain enforces, and the coldkey password read from a 0600 file. It never prompts, prints only public data, and never registers twice.--checkreports the addresses, registration, price, balance and published axon without the password. A PEP 723 header pins bittensor.pick_champion.py: picks the best eligible autoquant champion from the published 30-day verdicts, or validates a named one, and prints the two Terraform values.The one-go deploy (
references/gcp-end-to-end.md)~/.synth-miner/binif missing.miner-setup/terraform/gcpand waits for provisioning.coldkeypub.txtand the hotkey (never the coldkey), then reboots to prove the miner starts, loads its model and reaches the chain. A champion then pre-warms its market data and event calendar withsynth-miner-prewarm.service, so the miner serves almost at once after registration instead of missing prompts while it fetches them. All of this happens before any TAO is spent.The network, netuid, wallets folder, wallet and hotkey names, password file and GCP project are inputs. On testnet, where the public API does not report, the runbook proves an answer from the miner log against the validators its blacklist lets through.
It runs only in a local Claude Code session (the CLI, or a local session in the Claude Desktop app's Code tab), since it needs the laptop's gcloud, wallet and SSH key.
miner-setup/terraform(GCP, and AWS through the same startup script)provisionedorFAILED at <step>; every step can run again.synth-miner.service(systemd), started once the hotkey is on the VM, andsynth-miner-prewarm.service(oneshot, run by hand before registration).neurons/miner.py) or champion mode (an autoquant champion unpacked with synth-lib, which runs its contract gate at install).terraform.tfvarskeep working.miner-setup/README.mddocuments them.Verification
terraform consolein both modes,bash -npasses, andrun.sh --prewarmwas exercised with stubs. It has not yet run on a real VM..claude/, and the scripts pass with the repo config.Notes for reviewers
register.pyreads the password from a file the human writes in their own terminal, and wallet creation stays with the human;mainof synth-subnet, synth-lib, synth-ultra and autoquant, checked against the code. Where docs and code disagree, the skill follows the code and says so: "at least 1 minute to answer", the p90 rule, 1/3 per competition, "0.25 TAO is enough".prewarm.pyagainst the synth-lib that autoquant pins today. Serving: a champion can prewarm its market data before registration; crypto-1h warms first synth-lib#97 addsentrypoint.sh --prewarm(and warms the crypto-1h assets first); once autoquant pins it,run.shcan call that instead..claude/, onlyminer-setup/changes. No action is needed for running miners.Doc issues the skill works around
Candidates for follow-ups:
miner.test.config.jspasses--blacklist.validator.min_stake 0, a misspelled flag that is silently ignored, so the 65000 floor still applies on testnet.pm2 start miner.config.js -- <flags>examples change nothing: PM2 drops arguments after--when starting from a config file.--strictrejects valid miner flags such as--blacklist.validator_min_stake.docs/miner_reference.mdpasses--neuron.events_retention_size 2147483648, which crashes startup.--subtensor.chain_endpointis overridden by the default network, sominer.dev.config.jsreaches finney.ufw allow 8091without a protocol, whichchain_sync.pycannot remove.Serving miner axon … on network:line prints finney's default endpoint, not the chain in use.🤖 Generated with Claude Code