Repository navigation
[repo] Port workflows to a python CLI & Resable actions - #440
Merged
Merged
Conversation
The registry automation had grown to roughly 5,000 lines of inline YAML and Bash across the plugin workflows, with the app-token block copy-pasted 6 times, the fork-checkout dance 3 times, and the CodeQL SARIF jq program pasted 3 times. This moves the logic into a testable Python package and leaves the workflows as thin wiring. - Add .github/pluginctl: a stdlib-only Python package (core, validate, publish, integrations) exposing one CLI subcommand per task, with a 113-test pytest suite covering the pure logic and golden PR-comment/manifest/README output. - Add two composite actions (gh-app-token, trusted-checkout) and two reusable workflows (_codeql-scan, _clamav-scan) to remove the biggest duplication. - Thin the five core workflows to composite-action + `python -m pluginctl` steps; validate-plugin.yml drops from 1,370 to 442 lines. - Add an opt-in, best-effort signed webhook emitter (HMAC-SHA256) plus docs/webhooks.md, so a downstream consumer can react to registry events. - Delete the V1 validate/publish shell scripts; keep scripts/keys. - Remove the disabled standalone codeql workflow. - Fix a latent bug: deprecated plugins' version count now derives from the manifest's versions[] instead of the removed zips/ path (which rendered 0). User-facing output is kept byte-identical to the old shell, verified by the golden tests. Final fork end-to-end validation is still pending.
…on it Adds a tests.yml workflow that runs the pluginctl pytest suite, triggered on push to main for any change outside plugins/, and also exposed via workflow_call so validate-plugin.yml can run it on a PR's merge commit. It runs with a read-only token and no secrets. - validate-plugin.yml gains a test-suite job that calls tests.yml for authorized repo-update PRs (outside_files present, not a violation, not closing). - The result flows through report (a "Tooling test suite" comment section on failure, folded into overall_failed) and through the Plugin PR Check gate, which now fails a repo-update PR with red tests even when validation is otherwise skipped. gate.evaluate and report.build_comment take a test_result; cmd_gate/cmd_report read TEST_RESULT. - Branch protection is unchanged: still one required check (Plugin PR Check).
Upstream Plugins#193 fixed a Bash octal-parsing crash in version_greater_than() for versions like 0.2.08. This repo's pluginctl already avoids the bug since Python's int() has no octal ambiguity, but there was no test coverage proving it.
…im dead code Consolidates duplicated git subprocess wrapping and bot-identity setup into core/git.py, adds shared core/hashing.py and core/timeutil.py helpers, drops the unused core/models.Plugin, collapses redundant git-log/API/set_output calls, and removes assorted copy-paste and dead conditionals across validate/ and publish/. Behavior-preserving; pytest suite (130 tests) still passes.
An unsafe-named plugins/ folder was silently dropped from the scan matrix instead of blocking the PR, letting it ride along unscanned to merge and publish alongside a legitimate change. detect.py now closes the PR (unsafe-plugin-name) instead, and zips.py adds the same allowlist check as defense in depth before publishing. Separately, a plugin.json description containing an embedded newline could smuggle in a forged <!--META_ROW:--> line ahead of the genuine one, letting report.parse_fragments render attacker-chosen repo/discord links into the bot's PR comment. parse_fragments now anchors on the last marker line, and plugin.py collapses description to a single line before embedding it.
Port upstream PR Dispatcharr#208 into pluginctl: SARIF results with a recognized inline `codeql[...]` suppression no longer count toward blocking/medium/low CodeQL findings, get their own "suppressed findings" table in the PR comment, and gain a CodeQL Suppression Used label that forces manual review before merge. Also drops the 150-char truncation on finding descriptions.
…tcharr#226) to pluginctl Rebasing onto upstream/main pulled in PR Dispatcharr#225's AlertSuppression fix and PR Dispatcharr#226's follow-up, but both only touched the old inline CodeQL block in validate-plugin.yml, which the pluginctl rewrite (4973a50) had already replaced with _codeql-scan.yml. Port the same fix into that architecture: point CodeQL init at codeql-config.yml (sourced from the trusted base-branch checkout, not the untrusted PR merge tree) and dedupe repeated data-flow sentences in pluginctl's process_message, matching PR Dispatcharr#225's jq logic.
The validate-title job interpolated github.event.pull_request.title directly into a run: shell script instead of passing it through env, letting an attacker-controlled PR title break out of the quoted --title argument and execute arbitrary shell in a job with pull-requests/issues write permissions.
Add custom Python CodeQL queries and a hardcoded feature flag that keeps sandbox-bypass findings inactive until explicitly enabled. When enabled, findings remain informational, apply a maintainer-review label, and block automatic merging without failing PR validation.
Use the CodeQL Action's explicit binary path in the custom query test workflow and grant its required read-only security-events permission.
Restore split metadata and download base URL generation behind a hardcoded disabled rollout flag. Format generated manifest wrappers without changing compact comparison or signing payloads.
Classify fully metadata-only plugin updates during change detection.\nSkip CodeQL for those PRs while retaining ClamAV scans without\nre-downloading external release ZIPs.
Publish ai_assisted metadata from manual plugin declarations or recognized source-history attribution. Add expanded agent detection, rollback cleanup, concise documentation, and coverage.
Mirror Dispatcharr capability and manifest compatibility policy in pluginctl. Add manifest-aware capability-contract findings, focused sandbox-bypass queries, and a single manual-review gate for CodeQL findings and suppressions.
Remove public documentation for the experimental manifest disclosure and leave the feature disabled pending a long-term policy decision.
Add capability and sandbox-bypass queries, manifest-aware validation, tested query baselines, dependency locks, and filesystem bypass coverage.
Add optional author blacklisting, rolling author and plugin limits, a durable Auto Merged label, and serialized merge processing. Centralize required repository labels in pluginctl and document the controls.
Check out the trusted local composite action before jobs invoke it, allowing detect-changes and plugin validation to load the action.
Copy the base-branch CodeQL pack into the sparse analysis workspace before initialization so its configured local pack path resolves.
Select the local Python pack as a query directory, exclude trusted workflow files from analysis, and skip artifact downloads with zero findings.
Use an explicit relative path for the trusted local Python query pack.
Rename the reserved CodeQL result binding in the sys.modules tampering query.
Use CodeQL's getATarget API in the sandbox mutation queries.
Rename the reserved CodeQL binding in the builtins mutation query.
Validate all custom Python queries before running fixture-based query tests.
Keep sandbox and capability-contract findings in their dedicated review counters instead of reporting them as generic low-severity CodeQL results.
Remove redundant auto-merge wording from informational CodeQL review notices.
Detect locally traceable dynamic subprocess and builtins access, add semantic tests for sandbox mutation rules, and use validator-owned fragment status when building PR reports.
Compile and execute the custom Python CodeQL query tests through the reusable repo-update test workflow.
Keep custom query descriptions and findings focused on the required capability.
Assert that capability-contract and sandbox-bypass detection are disabled by default, while preserving explicit enabled-mode coverage. Pass PR inputs through environment variables before invoking pluginctl detect.
Clear GitHub repository variables before asserting the missing-repository path, so the test cannot invoke label creation on a workflow runner.
Route Discord webhook deliveries by event, render logo-backed release and PR cards, upload release ZIP attachments, and notify maintainers after validated PRs require review. Add local preview and send commands with real Dispatcharr/Plugins fixtures.
Resolve plugin metadata from PR heads, improve random fixtures, and add registry-release fallbacks for plugins without an upstream GitHub repository.
Run the post-validation merge decision even when automatic merging is disabled or the GitHub App is unavailable, then request manual review for every block.
Match the Manual Review Required label color to GitHub's default Duplicate gray.
Discover changed plugin metadata before requesting review when automatic merging is disabled.
Classify external plugins by source_type and allow enabled detection flags in the test suite.
Grant reusable CodeQL scans permission to read workflow metadata.
Mint label-management tokens through the shared GitHub App action and render local plugin comparisons using registry commit hashes.
Include the shared GitHub App token action in ClamAV's trusted sparse checkout.
Use the recorded main-branch squash commit instead of a comparison range in local plugin publish notifications.
dispatcharr-plugins-bot
Bot
added
the
Repo Update
Add the repository license and Code of Conduct, clarify contribution boundaries and issue scope, and document AI-assisted development.
Use a numbered Code of Conduct policy list and move manifest-signature verification instructions into the documentation directory.
Allow contributors to skip the template while requiring meaningful PR descriptions. Document auto-merge consequences and automation guidance.
Consolidate and label conduct rules, add sandbox protections, and direct users to Dispatcharr's in-app plugin browser.
Describe required secrets and variables, signing-key rotation, auto-merge controls, webhooks, and maintainer workflows.
Document eligibility, manual-review conditions, rate limits, and maintainer controls for plugin update pull requests.
Warn users about the workflow transition, route workflow reports to the upstream issue form, and correct issue-submission links.
Warn Code of Conduct readers about expected workflow transition hiccups and link to the upstream workflow-issue form.
Rename the contribution guide heading to a concise project-wide label.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replaces the plugin registry’s large inline workflow and shell-script automation with a tested, standard-library Python CLI named
pluginctl. Workflows are reduced to thin orchestration around reusable actions and workflows.Highlights
Compatibility
Existing registry publishing, manifest generation, README output, and PR validation comments are preserved.