feat: flag hallucinated Maven dependencies in pom.xml

[frankreyesgarcia]

When Claude adds a <dependency> to pom.xml, the hook already blocks it if the pinned version is outdated. This adds a second check: block it if the groupId:artifactId — or the pinned <version> of a real coordinate — doesn't exist on Maven Central at all, i.e. a coordinate the model hallucinated.

How existence is decided

A single GET of https://repo1.maven.org/maven2/<group as path>/<artifact>/maven-metadata.xml:

• 404 → the coordinate doesn't exist → hallucinated package
• 200, pinned version absent from the <versioning><versions> list → hallucinated version
• 200, version present → fine (an old-but-real version is still handled by the existing outdated check)
• any other status, transport error, unparseable XML, or an empty version list → inconclusive, fails open

This deliberately does not go through git-pkgs/enrichment's registry client: that queries Maven Central's Solr endpoint (search.maven.org) first, which returns found/not-found/timeout inconsistently for the same coordinate and hands back paginated, sometimes-incomplete version lists — fine for best-effort metadata, unacceptable for a hook that blocks writes. maven-metadata.xml is served off a static CDN, carries the complete version list in one file, and is what Maven itself resolves against.

Wiring

The check lives in pkg/maven behind an injected ExistenceChecker (MavenCentralExistenceChecker), mirroring how githubactions.Checker takes a separate Sha resolver; a nil ExistenceChecker leaves only the outdated-version comparison. mismatch.Kind (KindOutdated / KindMissingPackage / KindMissingVersion) tags each report. yul scan picks this up for free (it runs each checker with before="").

Before / after

Before — only outdated pins were caught:

outdated dependencies, use these versions instead:
  org.json:json  20240303 -> 20260522

After — a made-up coordinate or version is caught too:

problems with new dependencies:
  com.example:supercache  does not exist - hallucinated package, remove it or use a real coordinate
  com.google.guava:guava  version 99.0-jre was never published - hallucinated version

Benchmark

benchmark/hallucination/ is a deterministic precision/recall harness for the check — crafted PreToolUse payloads fed straight to the binary, classified on the stderr reason. 21 fixtures (9 real, 8 fake packages, 4 fake versions); precision 1.0 / recall 1.0, stable across runs.

🤖 Generated with Claude Code

[algomaster99]

I think you can reuse git-pkgs API NewClient for example to know if the package is hallucinated or not.

[algomaster99]

This is not the best approach since Maven Central can have multiple repositories, for example, Jenkins, Atlassian, etc.

[algomaster99]

Good catch — a real coordinate could be published to a repo Central has never heard of (Jenkins' update center, Atlassian's public repo, a corporate Nexus/Artifactory mirror), which this check would misread as hallucinated.

Rather than reaching for a wider (and much less deterministic) multi-registry lookup, I've pushed a change on claude/pr-review-comments-dma255 that has the existence check back off entirely whenever the pom.xml declares its own <repositories> — in that case, "not on Central" is no longer evidence of hallucination, so the check fails open for the whole file rather than risk a false positive. See declaresCustomRepositories in pkg/maven/pom.go (commit 8d8a936).

---

Generated by Claude Code

[algomaster99]

Reduce this. For a larger benchmark, I recommend looking for papers/dataset on arXiv or zenodo.

[algomaster99]

This is a related work: https://arxiv.org/abs/2605.06279.

[algomaster99]

Trimmed fixtures.jsonl from 21 hand-built cases down to 9 (4 real, 3 fake-package, 2 fake-version) — just enough to cover each failure mode the check makes, on claude/pr-review-comments-dma255 (commit 8d8a936).

On the arXiv dataset: this environment's network egress blocks arxiv.org, so I couldn't fetch the paper to pull real data from it. I've added a README pointer to it as the recommended source for a larger-scale precision/recall run instead of growing this file by hand further — someone with access to the dataset would be better placed to wire it in properly.

---

Generated by Claude Code

[gitguardian]

⚠️ GitGuardian has uncovered 1 secret following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

Since your pull request originates from a forked repository, GitGuardian is not able to associate the secrets uncovered with secret incidents on your GitGuardian dashboard.
Skipping this check run and merging your pull request will create secret incidents on your GitGuardian dashboard.

🔎 Detected hardcoded secret in your pull request

GitGuardian id	GitGuardian status	Secret	Commit	Filename
-	-	Generic Password	03a4b30	benchmark/bump/runs/04c07b06-slf4j-api/hook/transcript.jsonl	View secret

🛠 Guidelines to remediate hardcoded secrets

1. Understand the implications of revoking this secret by investigating where it is used in your code.
2. Replace and store your secret safely. Learn here the best practices.
3. Revoke and rotate this secret.
4. If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider

• following these best practices for managing and storing secrets including API keys and other credentials
• install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.

---

^{🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.}