orphan-gate

Fail your deploy when a page in your sitemap has no real inbound link.

Search engines index what they can reach by following links. A URL that is in your sitemap but linked from nowhere gets crawled late or never — no matter how many times you submit it through Search Console, IndexNow or the Indexing API. On a site that Googlebot visits a few dozen times a day, one orphaned money page can sit unindexed for months.

orphan-gate crawls your live site from its home page(s), builds the internal link graph, diffs it against the sitemap, and exits non-zero when something is unreachable. Run it right after a deploy, in CI, or as a GitHub Action. Zero dependencies, one file, Python 3.9+.

pip install orphan-gate
orphan-gate https://example.com

sitemap: 419 urls
seeds: / + 5 locale homes (/el, /it, /me, /mk, /sq)
crawled: 647 pages

· ORPHAN: 0
✗ SIBLING_ONLY: 1
    /fr/about  {'inbound': 1, 'depth': 2, 'sibling_sources': ['/about']}
· BROKEN: 0
· SITEWIDE_ONLY: 0
! DEEP: 2
    /guides/old/deep-page  {'depth': 4, 'inbound': 1}
· NOT_IN_SITEMAP: 0

FAIL: 1 blocking link-graph issue(s):
  SIBLING_ONLY  /fr/about

What it catches

Every one of these has shipped to production on a real site and was only noticed weeks later in Search Console.

Finding	Meaning	Blocks?
ORPHAN	In the sitemap, zero inbound links from any crawled page.	yes
SIBLING_ONLY	The only inbound links come from the page's own hreflang alternates — i.e. the language switcher. It looks linked; it isn't. Siblings are read from each page's own <link rel="alternate" hreflang> set, so translated slugs (/sq/punesim ↔ /hire) are handled.	yes
BROKEN	Linked to, but answers 4xx/5xx. Re-checked once before reporting, because a gate that runs seconds after a rollout sees transient errors.	yes
SITEWIDE_ONLY	Reachable only through header/footer furniture (a link present on >40 % of pages), with no contextual link from related content.	advisory
DEEP	Four or more hops from the nearest home page.	advisory
NOT_IN_SITEMAP	An indexable 200 page the crawl found that the sitemap omits. Pages with noindex or a canonical pointing elsewhere are ignored on purpose.	advisory

Adopting it on a site with existing debt

You probably have orphans today. Record them once, then fail only on regressions:

orphan-gate https://example.com --write-baseline          # → .orphan-gate-baseline.json, commit it
orphan-gate https://example.com --fail-on-new             # exit 1 only for NEW blocking findings

Fix an old one? Re-run --write-baseline. The baseline is a plain JSON map of category → URLs, easy to review in a PR.

ISR / stale-while-revalidate sites

Listing pages on Next.js (ISR), Nuxt (SWR), Astro server islands and similar frameworks can prerender empty at build time and only fill on the first request. Audit a fresh rollout naïvely and every blog post is reported as an orphan — because the index genuinely links none of them yet.

--warm requests every sitemap URL before crawling. It does so twice, with a pause, because the first request returns the stale page and merely kicks off regeneration in the background. Set --warm-delay above your longest revalidate window:

orphan-gate https://example.com --fail-on-new --warm --warm-delay 310   # app revalidates every 300 s

Multilingual sites

When the default locale is unprefixed (/about, /fr/about, /de/about), a single-seed crawl makes every other locale exactly one hop deeper than its twin and biases the DEEP cut against them. orphan-gate seeds the crawl from / and every /xx or /xx-yy path it finds in the sitemap, so depth means "hops from your home page". Disable with --no-locale-seeds.

GitHub Action

- uses: square-al/orphan-gate@v0.1.0
  with:
    base: https://example.com
    args: --fail-on-new --warm --warm-delay 0

The JSON report is uploaded as a workflow artifact. Commit .orphan-gate-baseline.json to use --fail-on-new.

As a deploy gate

./deploy && orphan-gate https://example.com --fail-on-new --warm \
  || { echo "link graph regressed — see report"; exit 1; }

Python API

from orphan_gate import audit, new_findings

report = audit("https://example.com", warm=True, warm_delay=310, log=print)
report["findings"]["ORPHAN"]         # [{'url': '/x', 'inbound': 0, 'depth': None, ...}]
new_findings(report, baseline={})    # [('ORPHAN', '/x'), ...]

All options

orphan-gate BASE [--sitemap URL] [--baseline FILE] [--fail-on-new] [--write-baseline]
            [--json FILE] [--warm] [--warm-delay SECONDS] [--max-pages N] [--concurrency N]
            [--sitewide-ratio 0.4] [--deep-depth 4] [--no-locale-seeds] [--ua STRING]
            [--timeout SECONDS] [--quiet]

Exit codes: 0 clean · 1 blocking findings · 2 nothing to audit (sitemap empty or unreachable).

Why not just check Search Console?

Search Console tells you after the fact, with a lag of days to weeks, and its "Discovered – currently not indexed" bucket does not distinguish an orphan from a page Google simply hasn't gotten to. The link graph is knowable the moment you deploy. Gate on it.

Limitations

• Follows <a href> in server-rendered HTML only. Links injected by client-side JavaScript are invisible — which is also how most crawlers see them on first pass.
• Same-host only; subdomains are treated as external.
• A sitemap index is followed; robots.txt Sitemap: lines are not (pass --sitemap).

Background

Extracted from the deploy pipeline of Square Software, where three money pages shipped orphaned in one summer before this became a gate. The ISR double-warm pass, the hreflang-sibling rule and the per-locale seeding each exist because of a specific false report; the story is in the source comments.

MIT © Square Software SHPK