A rigor toolkit for Claude Code, for anyone doing real research or R&D — not just people building AI pipelines. If you're writing a thesis, validating a business idea, running competitive/market research, doing due diligence, or building any kind of research or engineering pipeline, this gives Claude a concrete, checkable standard for what "thorough" actually means: read full sources instead of skimming, verify a citation is relevant and not just real, never quietly drop or truncate findings, and never call a research task "done" without an auditable reason.
It's not a curated list of links — it's a working /pipelineengine slash command plus a 27-protocol verification skill you drop into Claude Code and point at whatever you're working on: a literature review, a market-sizing exercise, a pitch deck's claims, a thesis chapter's citations, or an autonomous multi-stage research pipeline.
Where it comes from: the protocols were built from a real audit of a 7-stage AI research pipeline that found it was silently contradicting itself between reruns, marking a fabricated number "ALL CRITICAL CLAIMS VERIFIED" without ever checking the claim that mattered, and re-admitting a previously-killed fraudulent number through an external citation. That audit was then checked against the published state of the art (Sakana AI Scientist, Agent Laboratory, STORM, PaperQA2, GPT Researcher, and three independent 2025-2026 benchmarks — MLReplicate, ResearchClawBench, PaperBench) to confirm these are not one pipeline's bugs — they're the default failure modes of any process, human or automated, that claims to have researched something thoroughly.
- For a thesis or paper: catches citations that exist but don't actually support the claim next to them, literature reviews that stopped too early without a stated reason, and sections that restate sources instead of demonstrating you understood them (the comprehension-depth gate — Protocol 16 — checks for exactly this).
- For a business/market research pass: forces a completeness check before you present — did you actually cover every real competitor and data source, or just the first page of search results? Protocol 12–15 exist specifically to stop research from quietly stopping early.
- For an engineering or AI research pipeline: the original use case — determinism checks, adversarial audits, citation-relevance verification, and a persistent log of failure patterns so the same mistake doesn't recur next run.
- For due diligence or any high-stakes claim-checking: a two-gate rule that a source existing is not the same as a source actually supporting the claim it's attached to — the single most common way "verified" research turns out not to be.
Three independent benchmarks built specifically to test AI research/reproduction systems against real target papers with hidden ground truth all found the same thing: the best systems reproduce only 21-26% of a target paper's actual results, and self-reported "passed peer review" is not a trust signal — MLReplicate found 59% of automated-review-accepted manuscripts still contained fabricated or unsupported claims. Silent truncation is not a bug specific to one tool — it is the default behavior of major agent frameworks (hardcoded output caps, max_results limits, .slice(0, N) truncation with no logging). None of this is unique to AI pipelines either — "I searched a lot" and "I read the literature" have always been claims people make about their own research without a way to verify them. A process that "ran" is not evidence it actually searched, verified, or understood what it found — whether that process is a human researcher, a student writing a thesis, or an autonomous agent.
This toolkit exists so any research effort — a thesis, a pitch deck, a competitive analysis, an autonomous pipeline, or something built next year on a completely different topic — has the gates that catch these failure classes built in from the start, instead of discovered by a professor, an investor, or a failed replication after something already shipped wrong.
.claude/
commands/
pipelineengine.md # the /pipelineengine slash command
skills/
research-pipeline-verification-gates/
SKILL.md # the 27-protocol verification framework
self-improving-agents-and-harness-engineering/
SKILL.md # 2025-2026 RSI, continual learning, harness-engineering research
delegation-boundary-and-verification/
SKILL.md # when an AI should decide/act vs. ask — 2026 human-AI delegation research
semantic-layer-for-agentic-analytics/
SKILL.md # governed metrics for AI agents over a data warehouse — Cube/dbt architecture + a real bespoke precedent
multi-council-decision-coordination/
SKILL.md # making multi-agent debate/board deliberation actually improve decisions — 2026 research on when it doesn't
self-improving-agents-and-harness-engineering — grounded in real 2026 papers (two surveys covering 1,250+ arXiv papers combined, the Darwin/Red Queen Gödel Machine line of work, Sakana AI's Recursive Harness Self-Improvement, a Dagstuhl-seminar continual-learning position paper) plus a real production deployment (Bridgewater Associates' internally-deployed "PAT" analyst tool, hundreds of daily users, LangChain Interrupt 2026). Useful whenever you're designing an agent that needs to keep improving across sessions without catastrophic forgetting, or evaluating whether someone else's "self-improving system" claim is actually grounded or just a demo.
delegation-boundary-and-verification — the other half of rigor: knowing when an AI should decide/act on its own versus when it must stop and ask a human, and why a nominal "can you check this?" often isn't real oversight. Grounded in 2026 delegation-boundary research (Google DeepMind's "Intelligent AI Delegation," a formal "Interposition Problem" model with three named failure modes — under-delegation, over-delegation, and "agency laundering" — and a medical-AI framework for what actually makes oversight meaningful rather than procedural) plus Bridgewater's real per-user security harness as production evidence for graduated authority. Useful for anyone building an agentic system that touches money, credentials, third-party data, or anything else where a wrong autonomous call is expensive.
semantic-layer-for-agentic-analytics — why pointing an LLM agent directly at a data warehouse produces a different "revenue" number every session, and how a governed semantic layer (Cube, dbt MetricFlow, or a bespoke context/tool-shaping layer like Bridgewater's PAT) fixes it: metrics, dimensions, joins, and access rules defined once, enforced before the query runs, reached over MCP. Useful for anyone wiring an agent to financial, product, or business-metrics data where "the same question must return the same, correctly-access-controlled number every time" actually matters.
multi-council-decision-coordination — the uncomfortable 2026 finding this skill is built around: vanilla multi-agent debate (5 personas discussing) often underperforms simple majority vote despite costing more compute, unless two specific mechanisms are present (genuinely diverse reasoning approaches, not just diverse labels; and calibrated confidence that later voices condition on). Covers which decision protocol fits which task (voting vs. consensus), why more discussion rounds before a decision measurably hurts, and a real AWS+HSBC 2026 mechanism (Conformal Social Choice) for knowing when a board's unanimous agreement should trigger autonomous action versus human escalation — because unanimous agreement among agents is not, by itself, evidence of correctness.
Claude Code loads commands and skills from ~/.claude/commands/ and ~/.claude/skills/ (or a project-local .claude/ folder). To install:
# clone this repo
git clone https://github.com/<your-org-or-user>/Awesome-Research-Development.git
cd Awesome-Research-Development
# copy into your global Claude Code config
cp .claude/commands/pipelineengine.md ~/.claude/commands/
cp -r .claude/skills/research-pipeline-verification-gates ~/.claude/skills/Or, for a single project only, copy the same .claude/commands/ and .claude/skills/ folders into that project's root instead of your home directory.
Inside Claude Code, run:
/pipelineengine <pipeline-or-project-or-topic> [optional: which protocols to focus on]
Examples:
/pipelineengine my-research-agent # audit or govern a specific pipeline/project
/pipelineengine "LLM optimizer benchmark" # apply the protocols to a live research task
If you don't pass an argument, it will ask you what to run against rather than guessing or defaulting to a prior project — this is intentional, per Protocol design below.
The skill organizes protocols by what stage of a pipeline they attach to. Full detail, worked examples, and citations are in SKILL.md — this is the short map:
| Group | Protocols | What they catch |
|---|---|---|
| Verification gates | 1–7 | Determinism/reconciliation, adversarial audit, two-gate citation verification (existence ≠ relevance), claim-provenance tagging, reasoning-tree deliberation, trace-log retention, pre-allocation falsification |
| Ontology, scale, memory | 8–11 | Provenance/contradiction tracking, full-utilization manifests, map-reduce scale sharding, persistent failure-pattern library |
| Breadth and honesty | 12–15 | Perspective-discovery before retrieval, statistically-justified stopping rules, observable-truncation mandates, real-benchmark grounding |
| Depth and exhaustiveness | 16–17 | Comprehension-depth gate (translation, near-transfer, evidence-locator, perturbation, lazy-thinking checks), exhaustive idea-generation gate |
| Extended pack | 18–27 | Domain-expansion protocols layered on top of the core 17 — see SKILL.md for the full list |
- Don't run it as a checklist recital. Reading through 27 protocol names without checking anything concrete against your actual pipeline is exactly the "the pipeline ran" vs. "the pipeline verified this" gap this framework exists to close.
- Identify what you're actually doing first: auditing existing output, designing/upgrading a pipeline before it runs, or governing a live research task right now. The protocols apply differently to each.
- If auditing existing output, read the raw artifacts, not a polished summary — Protocol 6 (trace-log retention) exists because contradictions hide in raw intermediate output, not final reports.
- If a protocol doesn't apply to your specific pipeline, say so explicitly and why — don't silently skip it or force a fit.
- Don't call a research task "done" until the Verification Checklist passes, including Protocols 16–17, which are the easiest to skip under time pressure since they don't produce an obvious error the way a missing citation does.
This framework doesn't hardcode or require any specific search/research API — it's model- and tool-agnostic verification logic. But if you're building a research pipeline for it to govern, here's where to get keys for commonly used sources. Never commit real keys to this or any repo — see the Security section below.
| Service | What it's for | Get a key at | Free tier |
|---|---|---|---|
| Exa | Neural/semantic web search, deep-reasoning search mode | dashboard.exa.ai → API Keys |
Yes, limited monthly credits |
| Tavily | AI-optimized search, extract, crawl — clean structured results for RAG | app.tavily.com → API Keys |
1,000 requests/month |
| Firecrawl | Web/paper/GitHub scraping and crawling | firecrawl.dev → dashboard → API Keys |
Limited free requests/month |
| Semantic Scholar | 200M+ papers, citation graphs, TLDR summaries | semanticscholar.org/product/api → request a key |
Free, rate-limited without a key, higher with one |
| SerpAPI | Structured Google search results | serpapi.com → dashboard → API Key |
100 searches/month |
| HuggingFace | Models, datasets, inference API, Papers With Code data | huggingface.co/settings/tokens |
Free tier available |
| OpenAlex | 250M+ works, fully open | No key needed — add your email to the mailto param for a faster "polite pool" |
Free, unlimited with politeness |
| arXiv API | Preprints | No key needed | Free |
| Crossref | DOI metadata | No key needed — add a mailto header for the polite pool |
Free |
| PubMed E-utilities | Biomedical literature | Optional key at ncbi.nlm.nih.gov/account/ for higher rate limits |
Free |
General pattern for wiring a key into your own pipeline once you have it:
# set as an environment variable, never hardcoded in source
export EXA_API_KEY="your-key-here"
export TAVILY_API_KEY="your-key-here"import os
api_key = os.environ["EXA_API_KEY"] # fails loudly if unset, rather than silently using a placeholderThis repo's .gitignore already excludes common secret-bearing filenames. If you fork this or add your own pipeline code alongside it:
- Never paste a real key into a file tracked by git, including this README, example configs, or code comments.
- Use
.envfor local secrets and keep.envin.gitignore(already done here) — commit only.env.examplewith placeholder values. - Before your first commit with real config, run a check for accidentally-included secrets:
git diff --cached | grep -iE "api[_-]?key|secret|token|bearer|sk-[a-zA-Z0-9]{10,}"
- If a real key is ever committed, rotate/revoke it at the provider immediately — removing it from a later commit does not remove it from git history. Use
git filter-repoor BFG Repo-Cleaner to purge history if that happens, then force-push and rotate the key regardless (assume it's compromised the moment it touched a commit, even a private repo). - Prefer read-only, scoped tokens wherever a provider offers them, and set spend/rate limits on any key used by an autonomous pipeline — a runaway loop hitting a paid API is a real cost risk, not just a security one.
Business Source License 1.1 — see LICENSE for the governing legal text. In plain language:
- 🟢 Free — personal use, academic/research use, and any organization under USD 1M in annual revenue, including production use.
- 🔒 Commercial license required — if you resell this (or a competing offering built on it) as a hosted/managed service, or if an organization above USD 1M in revenue builds it into a commercial product or paid service.
- 🔓 Opens fully after 4 years — each version automatically converts to Apache License 2.0 on its own Change Date. Source-available today, unconditionally open tomorrow.
- ✉️ Commercial licensing — vancongvinh157@gmail.com
Every widely-used option trades openness for monetization somewhere. Here's the landscape, and why BUSL was the right fit for a framework meant to be both freely adopted and commercially sustainable:
| License | Can the author charge? | Openness / virality | Notable adopters |
|---|---|---|---|
| MIT / Apache 2.0 | No — anyone, including competitors, may resell it freely (Apache adds a patent grant, otherwise equivalent) | ⭐⭐⭐⭐⭐ Maximum | The default for dev tools optimizing for adoption |
| BUSL 1.1 ← this repo | Yes — free below a revenue threshold, licensed above it, with fully custom terms | ⭐⭐⭐⭐ High — reads as source-available, indexes and stars normally on GitHub | MariaDB, CockroachDB, early HashiCorp |
| FSL | Yes, narrower — only blocks direct competing products; most commercial users stay free | ⭐⭐⭐⭐ High — marketed as the "friendlier" alternative to BUSL | Sentry |
| Elastic License 2.0 | Partial — blocks third-party hosted resale only; everything else stays free | ⭐⭐⭐⭐ High | Elastic, several data-infra projects |
| SSPL | Yes, aggressively — anyone hosting it as a service must open-source their entire stack | ⭐⭐ Low — not OSI-recognized as open source, draws real pushback | MongoDB |
| Proprietary | Yes, fully | ⭐ Minimal — no public source at all | Traditional licensed software |
BUSL won out for three reasons specific to this project: the Additional Use Grant is fully custom, so the free/paid line is drawn exactly where it makes sense here (a revenue threshold, not a vague "non-production" rule); it still reads as source-available to anyone browsing GitHub, unlike SSPL which is widely viewed with suspicion; and it guarantees full openness eventually via the Change Date, so early adopters know this never becomes permanently closed.
Issues and PRs welcome, especially: additional protocols for domains not yet covered, real-world failure cases that should become new gates (per Protocol 11's failure-pattern library), and corrections to any of the cited benchmarks/papers if something here goes stale. Note: by contributing, you agree your contribution is licensed under the same BUSL 1.1 terms above.