Skip to content

feat(hermes): report setup outcome so catalog installs are counted (v0.2.4) - #64

Merged
Zechereh merged 2 commits into
mainfrom
zacharylyon/pf-6610-track-plugin-installs-outside-tinyfish-connect-hermes-native
Oct 21, 2026
Merged

Zechereh merged 2 commits into
mainfrom
zacharylyon/pf-6610-track-plugin-installs-outside-tinyfish-connect-hermes-native

Conversation

@Zechereh

@Zechereh Zechereh commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Installs are only counted when tinyfish connect runs. Catalog installs (hermes plugins install tinyfish) never do, so the largest group of Hermes users has no install event. hermes tinyfish setup now sends one.

Change Why
rest_client.report_setup() POSTs a harness_plugin_setup event to agent.tinyfish.ai/api/cli/connect-event Same endpoint the TinyFish CLI reports to
Sends a one-way hash of the key, never the key Joins the event to the account without the key leaving the machine
cmd_setup reports completed, key_rejected, no_key, save_failed or backend_failed, plus key_source Shows where catalog installers drop off before their first API call
2s timeout, swallows every error Telemetry can never fail or slow setup
TINYFISH_NO_TELEMETRY opt-out, same semantics as the CLI; README section Disclosure and off switch
scripts/e2e.sh sets TINYFISH_NO_TELEMETRY=1 CI setups would count as real installs
0.2.3 → 0.2.4 plugin.yaml ships in the npm package

Don't merge until the endpoint accepts harness_plugin_setup — until then it rejects the event and the plugin drops it silently.

make lint test: 240 passed, ruff and mypy clean.

🤖 Generated with Claude Code

@coderabbitai

coderabbitai Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Warning

Review limit reached

  • Run on-demand review

This review includes 4 billable files and costs up to $1.00.

  • Ask an admin to make reviews automatic

Open in CodeRabbit

Reviews can continue after your included limit without a manual trigger. An admin must approve usage-based billing.

Or wait 28 minutes for your next included review.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available. Your 30 included PR review attempts over the past 7 days set your current allowance at 4 reviews per hour.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Essentials
  • Run ID: 2136ca30-8cb7-46e3-9835-003de5c18414
📥 Commits

Reviewing files that changed from the base of the PR and between 5a828b0 and 06b1bc4.

📒 Files selected for processing: 4
  • hermes/README.md
  • hermes/tests/test_setup_cli.py
  • hermes/tinyfish_hermes/rest_client.py
  • hermes/tinyfish_hermes/setup_cli.py

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Essentials
  • Run ID: d7cdcc3a-914f-4b55-9fed-0a2ab006d843
📥 Commits

Reviewing files that changed from the base of the PR and between c77a13d and 5a828b0.

📒 Files selected for processing: 3
  • hermes/README.md
  • hermes/tests/test_rest_client.py
  • hermes/tinyfish_hermes/rest_client.py
🚧 Files skipped from review as they are similar to previous changes: 1
  • hermes/README.md

Included review availability: This review used your included allowance. 1 included review remains after this review. Your included PR review attempts over the past 7 days set your current allowance at 5 reviews per hour.

📝 Walkthrough

Hermes setup now reports setup outcomes to TinyFish telemetry. Reports include the outcome, API-key source, Hermes version, platform, and a UUID. When an API key is available, the request includes its SHA-256 hash. The TINYFISH_NO_TELEMETRY variable can disable reporting, and telemetry failures are suppressed. Release versions change to 0.2.4.

Sequence Diagram(s)

sequenceDiagram
  participant cmd_setup
  participant report_setup
  participant TELEMETRY_URL
  cmd_setup->>report_setup: Send setup outcome and key source
  report_setup->>TELEMETRY_URL: POST telemetry event
Loading

Priority: ⬇️ Low

Merge Risk: ⚪ Minimal · up to 5a828

No actionable merge-blocking defect was established. Setup reporting and packaging appear ready for normal release checks; endpoint ingestion has not been confirmed.

Pre-merge checks: Passed 4
Check name Status Explanation
Title check Passed The title clearly and concisely identifies the main change: reporting Hermes setup outcomes so catalog installs are counted.
Description check Passed The description directly explains the telemetry implementation, setup outcomes, opt-out behavior, version bump, testing, and endpoint dependency.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

…0.2.4)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Zechereh
Zechereh force-pushed the zacharylyon/pf-6610-track-plugin-installs-outside-tinyfish-connect-hermes-native branch from c77a13d to 5a828b0 Compare October 21, 2026 21:36
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Zechereh
Zechereh merged commit 6134bed into main Oct 21, 2026
6 checks passed
@Zechereh
Zechereh deleted the zacharylyon/pf-6610-track-plugin-installs-outside-tinyfish-connect-hermes-native branch October 21, 2026 22:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants