Skip to content

CAMEL-24172: Add camel-apicurio-registry component - #24844

Open
carlesarnal wants to merge 10 commits into
apache:mainfrom
carlesarnal:camel-apicurio-registry-component
Open

carlesarnal wants to merge 10 commits into
apache:mainfrom
carlesarnal:camel-apicurio-registry-component

Conversation

@carlesarnal

@carlesarnal carlesarnal commented Jul 17, 2026 •

Copy link
Copy Markdown

Summary

JIRA: https://issues.apache.org/jira/browse/CAMEL-24172

Adds a Preview component for Apicurio Registry v3 using io.apicurio:apicurio-registry-java-sdk:3.3.0, targeting Camel 4.23.0.

URI: apicurio-registry:groupId/artifactId[?options]

The producer supports artifact/group CRUD, content and metadata retrieval, search, version listing, and validation/compatibility checks. Both checks use the registry’s dry-run API for proposed schema versions. Rule violations return error details; authentication and connection errors propagate normally.

The consumer polls for artifact versions using an in-memory global-ID watermark and sorts returned versions before delivery. Content retrieval returns byte arrays and closes response streams. Exchanges are released in a finally block, including when content retrieval or processing throws. Failed versions remain eligible for retry.

Authentication supports Basic and OAuth2/OIDC client credentials. The endpoint manages its own Vert.x resources, while supplied clients remain caller-owned.

Includes the generated catalog, BOM, documentation links, main/dependency indexes, and component/endpoint DSL factories.

Review updates

Rebased onto main at b078bbeaf9a, resolving the parent POM conflict while retaining the newer upstream dependency version. Both module parents and first-version metadata use 4.23. SDK dependency management is centralized, credential annotations are corrected, and tests use AssertJ.

The latest follow-up fixes exchange cleanup and empty group-ID parsing, documents compatibility error headers, and adds pooled-exchange failure/retry tests. The documentation also explains textual schema inputs, URI delimiters, replay behavior, and the scope of validation.

The 3.3.0 container image supports Linux amd64, arm64, ppc64le and s390x. Test infrastructure explicitly enables artifact deletion so the lifecycle tests can assert it succeeds.

Validation

../../mvnw install -Pit -q in the component module passed: 40 unit tests and 8 integration tests, with no failures or skips. Coverage includes Basic/OIDC HTTP authentication, client lifecycle, compatibility errors, pooled-exchange cleanup and retries, stream closure, header overrides, missing artifacts, and second-version polling.

Formatting/import sorting and affected metadata/catalog/DSL generation completed. The full all-component build was not run; unrelated stale outputs from partial catalog generation were discarded. GitHub workflow approval requires a repository admin.

Related: Apicurio/apicurio-registry#8671

AI-assisted description prepared with OpenCode on behalf of @carlesarnal.

@github-actions

Copy link
Copy Markdown
Contributor

🌟 Thank you for your contribution to the Apache Camel project! 🌟
🤖 CI automation will test this PR automatically.

🐫 Apache Camel Committers, please review the following items:

  • First-time contributors require MANUAL approval for the GitHub Actions to run
  • You can use the command /component-test (camel-)component-name1 (camel-)component-name2.. to request a test from the test bot although they are normally detected and executed by CI.
  • You can label PRs using skip-tests and test-dependents to fine-tune the checks executed by this PR.
  • Build and test logs are available in the summary page. Only Apache Camel committers have access to the summary.

⚠️ Be careful when sharing logs. Review their contents before sharing them publicly.

@apupier

apupier commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

there are seveal modules to regen:

There are uncommitted changes
HEAD detached at pull/24844/merge
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   bom/camel-bom/pom.xml
	modified:   catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/components.properties
	modified:   core/camel-main/src/generated/resources/org/apache/camel/main/components.properties
	modified:   docs/components/modules/ROOT/nav.adoc
	modified:   dsl/camel-componentdsl/src/generated/java/org/apache/camel/builder/component/ComponentsBuilderFactory.java
	modified:   dsl/camel-endpointdsl/src/generated/java/org/apache/camel/builder/endpoint/EndpointBuilderFactory.java
	modified:   dsl/camel-endpointdsl/src/generated/java/org/apache/camel/builder/endpoint/EndpointBuilders.java
	modified:   dsl/camel-endpointdsl/src/generated/java/org/apache/camel/builder/endpoint/EndpointHeaderBuilders.java
	modified:   dsl/camel-endpointdsl/src/generated/java/org/apache/camel/builder/endpoint/StaticEndpointBuilders.java
	modified:   dsl/camel-kamelet-main/src/generated/resources/camel-component-known-dependencies.properties

Untracked files:
  (use "git add <file>..." to include in what will be committed)
	catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/components/apicurio-registry.json
	docs/components/modules/ROOT/examples/json/apicurio-registry.json
	docs/components/modules/ROOT/pages/apicurio-registry-component.adoc
	dsl/camel-componentdsl/src/generated/java/org/apache/camel/builder/component/dsl/ApicurioRegistryComponentBuilderFactory.java
	dsl/camel-endpointdsl/src/generated/java/org/apache/camel/builder/endpoint/dsl/ApicurioRegistryEndpointBuilderFactory.java

@carlesarnal

Copy link
Copy Markdown
Author

there are seveal modules to regen:

There are uncommitted changes
HEAD detached at pull/24844/merge
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	modified:   bom/camel-bom/pom.xml
	modified:   catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/components.properties
	modified:   core/camel-main/src/generated/resources/org/apache/camel/main/components.properties
	modified:   docs/components/modules/ROOT/nav.adoc
	modified:   dsl/camel-componentdsl/src/generated/java/org/apache/camel/builder/component/ComponentsBuilderFactory.java
	modified:   dsl/camel-endpointdsl/src/generated/java/org/apache/camel/builder/endpoint/EndpointBuilderFactory.java
	modified:   dsl/camel-endpointdsl/src/generated/java/org/apache/camel/builder/endpoint/EndpointBuilders.java
	modified:   dsl/camel-endpointdsl/src/generated/java/org/apache/camel/builder/endpoint/EndpointHeaderBuilders.java
	modified:   dsl/camel-endpointdsl/src/generated/java/org/apache/camel/builder/endpoint/StaticEndpointBuilders.java
	modified:   dsl/camel-kamelet-main/src/generated/resources/camel-component-known-dependencies.properties

Untracked files:
  (use "git add <file>..." to include in what will be committed)
	catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/components/apicurio-registry.json
	docs/components/modules/ROOT/examples/json/apicurio-registry.json
	docs/components/modules/ROOT/pages/apicurio-registry-component.adoc
	dsl/camel-componentdsl/src/generated/java/org/apache/camel/builder/component/dsl/ApicurioRegistryComponentBuilderFactory.java
	dsl/camel-endpointdsl/src/generated/java/org/apache/camel/builder/endpoint/dsl/ApicurioRegistryEndpointBuilderFactory.java

Yes, this is in draft still, I'm still wrapping my head around a few things on how to structure different elements. Thanks!

@carlesarnal
carlesarnal force-pushed the camel-apicurio-registry-component branch from 0780ba9 to 218e2c1 Compare July 20, 2026 06:30
Comment thread components/camel-apicurio-registry/pom.xml
@atiaomar1978-hub

Copy link
Copy Markdown
Contributor

Grok code review

Verdict: Approve with improvements — well-structured new component with appropriate SDK delegation, generated metadata, and meaningful test coverage.


Architecture

ApicurioRegistryComponent (URI parse groupId/artifactId)
    └── ApicurioRegistryEndpoint (RegistryClient lifecycle, auth)
            ├── ApicurioRegistryProducer (HeaderSelectorProducer + @InvokeOnHeader ops)
            └── ApicurioRegistryConsumer (ScheduledPollConsumer, globalId watermark)

Good separation; SDK types stay at the boundary.


Strengths

  • Apicurio v3 SDK integration with basic + OIDC auth paths.
  • Ten producer operations cover common registry workflows.
  • Consumer fetchContent toggle is a nice UX knob.
  • ApicurioRegistryValidationException gives callers a typed failure mode.
  • Dedicated test-infra module with health-check wait strategy on /apis/registry/v3/system/info.
  • AsciiDoc page with Java/XML/YAML examples.

Follow-ups (see inline Grok threads)

# Area Suggestion
1 Client lifecycle Close/shutdown SDK client in doStop() if supported
2 Headers HEADER_DRY_RUN declared but unused
3 Duplication Merge validate / testCompatibility dry-run logic
4 Tests Prefer AssertJ over JUnit assertions in new tests
5 Docs Document consumer first-start replay + watermark semantics
6 URI parsing / in group/artifact IDs not supported — document limit

Test coverage

Class Focus
ApicurioRegistryComponentTest Endpoint creation
ApicurioRegistryProducerTest Mocked SDK ops
ApicurioRegistryConsumerTest Poll / watermark (mocked)
ApicurioRegistryProducerIT / ValidateIT / ConsumerIT Testcontainers end-to-end

Recommendation

Ready for committer review. Suggest reviewers with registry/cloud component experience (e.g. recent schema/registry contributors).

AI-generated Grok review on behalf of atiaomar1978-hub.

@atiaomar1978-hub

Copy link
Copy Markdown
Contributor

Bugbot review

Reviewed camel-apicurio-registry-component vs main (~3k LOC new component + test-infra).

Scope

New Preview component wrapping Apicurio Registry v3 REST API via apicurio-registry-java-sdk 3.3.0. Producer ops (CRUD, search, validate/compatibility dry-run), consumer polls new versions by globalId watermark, basic/OIDC auth, unit tests (mocked SDK), ITs (Testcontainers quay.io/apicurio/apicurio-registry:3.3.0), AsciiDoc docs.

Verified (looks good)

  • Clean Camel layout: Component / Endpoint / Producer / Consumer / Configuration / Constants.
  • HeaderSelectorProducer + @InvokeOnHeader for operation dispatch.
  • Secrets marked security = "secret" on username/password/clientSecret.
  • Test-infra follows Camel conventions (@InfraService, container.properties, quay.io image).
  • Good operation coverage in mocked producer tests + IT suite for create/search/validate/consumer.

Issues flagged (inline comments posted)

Severity Area Finding
Major cacheTtl option Documented URI param never used in code
Major Consumer watermark In-memory lastSeenGlobalId — restart replays all versions
Major Consumer ordering Unsorted version list can skip IDs if API order ≠ globalId order
Major Producer body getBody(String.class) unsuitable for binary AVRO/Protobuf payloads
Medium getArtifactContent Returns raw InputStream — leak risk without documented lifecycle
Medium testCompatibility Broad catch (Exception) masks network/auth errors
Medium Dependency mgmt SDK version local to module pom, not parent BOM
Minor Docs vs reality PR text says container 3.0.6, properties use 3.3.0

Bugbot verdict: Solid first component drop — approve with follow-ups above before Preview promotion. No security blockers (producer is trusted route config; consumer polls configured registry).

AI-generated Bugbot review on behalf of atiaomar1978-hub.

@atiaomar1978-hub

Copy link
Copy Markdown
Contributor

Bugbot — test coverage review

Assessed unit + IT coverage for camel-apicurio-registry (~890 LOC tests across 7 classes + test-infra).

Coverage matrix (producer operations)

Operation Unit (mock SDK) IT (Testcontainers)
createArtifact ✅ ✅ lifecycle IT
updateArtifact ✅ ✅ lifecycle IT
deleteArtifact ✅ ✅ lifecycle IT
getArtifactContent ✅ ✅ lifecycle IT
getArtifactMetadata ✅ ✅ lifecycle IT
listVersions ✅ ✅ lifecycle IT
createGroup ✅ ✅ lifecycle IT
searchArtifacts ❌ route only, no test ✅ dedicated IT
testCompatibility ❌ ⚠️ happy path only (assertNotNull body)
validate ❌ ⚠️ failOnValidation=false, header non-null only

Consumer coverage

Scenario Unit IT
Poll new versions (mock) ✅ —
Skip already-seen globalId ✅ —
Real registry poll — ⚠️ single version, headers only
fetchContent=true ❌ ❌
Second version after initial poll ❌ ❌
Missing groupId/artifactId error ❌ ❌

Config / auth

Area Covered?
URI parsing (group / group+artifact) ✅ ComponentTest
configuration.copy() ✅
EndpointServiceLocation ✅
Basic auth options binding ✅ (no live auth call)
OIDC auth ❌
cacheTtl option ❌ (also unused in prod code)

Verdict

Adequate for Preview merge — strong lifecycle IT and good mocked coverage for core CRUD. Not yet comprehensive for a GA component: three producer ops lack unit tests, validation/compatibility negative paths are thin, consumer watermark/fetchContent gaps, and auth is config-only.

Priority additions before promotion:

  1. Unit tests for searchArtifacts, validate, testCompatibility
  2. Negative validate/compatibility ITs + ApicurioRegistryValidationException when failOnValidation=true
  3. Consumer IT: second version + optional fetchContent=true
  4. Consumer unit test for out-of-order globalId (guards known watermark risk)

AI-generated Bugbot test-coverage review on behalf of atiaomar1978-hub.

@atiaomar1978-hub

Copy link
Copy Markdown
Contributor

Grok — test coverage review

Verdict: Good foundation, gaps on edge cases and half the producer surface at unit level


What is well covered

  • ApicurioRegistryProducerIT.testFullArtifactLifecycle — best test in the PR; exercises createGroup → create → get meta/content → update → listVersions → delete against real Apicurio 3.3.0.
  • Mocked producer tests — verify SDK call graph for 8/10 operations + header-driven operation selection.
  • Mocked consumer tests — latch-based MockEndpoint.assertIsSatisfied with timeout (good Camel convention).
  • Test-infra module — reusable ApicurioRegistryService + health-checked container; ITs are not one-offs.
  • ComponentTest — URI parsing, config copy, service location (often skipped in new components — nice to see).

Gaps by risk

Risk Gap Suggested test
High No unit test for validate / testCompatibility Mock dry-run POST success/failure; assert headers/body/ApicurioRegistryValidationException
High Validate IT never asserts HEADER_VALIDATION_RESULT == true or failure paths Incompatible JSON schema + failOnValidation=true
Medium searchArtifacts route defined, never invoked in unit tests One mock test with name/groupId query params
Medium Consumer watermark edge cases untested Mock versions [globalId=2, globalId=1] — expect both or document sort requirement
Medium fetchContent consumer option Unit + IT: body is stream/content vs SearchedVersion metadata
Low OIDC auth path WireMock token endpoint or skip with @DisabledIf + documented manual QA
Low Header override resolution (resolveGroupId from header vs URI) Single producer test overriding URI path via header

Test quality notes

  • ITs use JUnit assertions throughout; project prefers AssertJ in new tests (non-blocking).
  • ApicurioRegistryConsumerIT uses a private JUnit assertEquals helper — migrate to AssertJ when touching file.
  • Assertions are often existence-only (assertNotNull) where value assertions would catch regressions (validation result, compatibility boolean).

Scorecard

Layer Score Notes
Unit breadth 7/10 8/10 ops + 2 consumer scenarios
Unit depth 5/10 Few header/error-path assertions
Integration 8/10 Strong lifecycle; weak validate/consumer multi-version
Infra 9/10 Proper test-infra split
Overall 7/10 Acceptable for Preview; tighten before stable

AI-generated Grok test-coverage review on behalf of atiaomar1978-hub.

@carlesarnal

Copy link
Copy Markdown
Author

Thanks @davsclaus. All three are addressed in 9ffb472, on top of the rebased branch.

  1. Failed versions. After processing, the consumer now checks exchange.getException(). If it is set, the consumer reports it through getExceptionHandler(), stops the poll and leaves the watermark where it was, so the version is retried next time. There is a new route-based test using the default error handler: version 1 fails once, then versions 1 and 2 arrive in order.
  2. Pagination. The consumer requests versions ordered by globalId, newest first, 100 per page, and stops at the watermark. It deduplicates by globalId in case a new version shifts the pages between requests. listVersions now follows every page. The tests use a fake that behaves like the registry, including its default page of 20.
  3. Vert.x. There is now a single Vert.x instance per component. It is created on first use and closed when the component stops. It can also be supplied through the new vertx option, or autowired from the registry, in which case the component leaves it open.

From the minor points: tests now use camel-test-junit6, and the test infra is registered in camel-test-infra-all (dependency, jandex, metadata and the catalog copy). I left getServiceProtocol() as "http", as you suggested. testCompatibility also sets the result header now, as the bot review asked.

The docs and generated catalog/DSL files are updated. Locally, 42 unit tests and 8 integration tests pass.

AI-assisted response prepared with OpenCode on behalf of @carlesarnal.

@gnodet-bot gnodet-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review of 9ffb47280a07 — all three @davsclaus items addressed, plus the outstanding gnodet-bot finding from the previous round.

Addressed ✅

  1. Failed version retry — Consumer now checks exchange.getException() after process(). When the error handler handles the exception (sets it on the exchange without rethrowing), the consumer reports it via getExceptionHandler(), stops the poll, and leaves the watermark unchanged so the version retries on the next poll. testRouteFailureHandledByErrorHandlerIsRetried confirms version 1 fails once, then both 1 and 2 arrive in order.

  2. Pagination — Consumer pages newest-first (SortOrder.Desc, PAGE_SIZE=100), stops at the watermark. TreeMap<Long, SearchedVersion> deduplicates if a new version shifts pages mid-request. testPollPagesThroughManyVersionsAndStopsAtWatermark covers 250 versions across 3 pages and verifies the watermark cuts off older pages on the next poll. Producer's listVersions also follows all pages (ascending). FakeVersionPages is a well-designed test helper that simulates the registry's sort/offset/limit behavior.

  3. Shared Vert.x — Single Vertx per component, created lazily (getOrCreateVertx()), closed on doStop() with synchronized guard and 30s timeout. Caller-supplied Vert.x via setVertx() is never closed. Tests confirm shared instance across endpoints (testEndpointsShareComponentVertxClosedWithComponent) and that a provided instance is left alone (testProvidedVertxIsNotClosed).

  4. testCompatibility now sets HEADER_VALIDATION_RESULT — our previous finding is addressed; both validate and testCompatibility set the header, with matching test assertions.

  5. camel-test-junit6 migration — all test classes updated.

  6. camel-test-infra-all registration — as @davsclaus requested.

Code is solid. No new issues found.


This review was generated by an AI agent, Hermès on behalf of @gnodet.

@davsclaus davsclaus left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the thorough update. All the earlier points are addressed: a failed version is no longer skipped (the watermark only advances on success, with a route test), version lists are paged in both the consumer and listVersions, the component shares one Vert.x (and never closes one the user supplies), the tests use camel-test-junit6, and the test-infra is registered.

One new issue worth fixing before merge, and a few smaller ones:

  • WebClient leak (inline): each endpoint start creates a Vert.x WebClient that is never closed.
  • setVertx() (inline): supplying a Vert.x after the component created its own leaves managedVertx true, so doStop() closes the user's instance.
  • A version that always fails is retried on every poll and blocks all newer versions. That is the behaviour I asked for, but please document how to skip it (onException(...).handled(true) advances the watermark). The failure is also logged twice (error handler and getExceptionHandler()).
  • Offset paging can skip an entry if a version is deleted between page requests, and the watermark then moves past it. Rare; a docs note is enough.
  • Nit: the lastSeenGlobalId == null || globalId > lastSeenGlobalId check in the consumer is now redundant.

(CI has now been approved to run.)

Claude Code on behalf of davsclaus

This review was generated by an AI agent and may contain inaccuracies. Please verify all suggestions before applying.

@gnodet-bot gnodet-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review of 9ffb47280a0 — all previous findings addressed. Clean APPROVE for this Preview component.

Previous findings status

Finding Status
Side-effect HEADER_VALIDATION_ERRORS in testCompatibility ✅ Addressed — author chose option (b): both testCompatibility and validate now set HEADER_VALIDATION_RESULT, and the error header is documented for both operations. Header description updated to "Cleared before each check."
Exchange pooling leak (createExchange without releaseExchange) ✅ Addressed — consumer uses createExchange(false) paired with releaseExchange(exchange, false) in a finally block. Pooled-exchange tests verify cleanup.
Empty group-ID parsing (split("/") leaving empty string) ✅ Addressed — parts[0].isEmpty() ? null : parts[0] normalizes empty strings. Parsing tests added for /someArtifact case.

Latest changes reviewed (9ffb472)

  • Consumer error handling — exchange.getException() checked after processing; on failure, error reported via getExceptionHandler(), poll stops without advancing watermark (version retried next cycle). Correct pattern.
  • Pagination — fetchNewVersions() requests pages newest-first with SortOrder.Desc, stops at watermark, deduplicates via TreeMap<Long, SearchedVersion>, returns ascending order. listVersions producer follows every page. Both correct.
  • Vert.x lifecycle — single instance per component, synchronized creation, outside-lock close in doStop() with 30s timeout. Caller-supplied Vertx left open. Correct.
  • Test infra — registered in camel-test-infra-all (dependency, jandex, metadata, catalog). camel-test-junit6 used throughout.

Solid Preview component — well structured, good test coverage (42 unit + 8 IT), clean Camel conventions.


This review was generated by an AI agent, Hermès on behalf of @gnodet.

@oscerd oscerd left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the thorough iterations, Carles. Every point from my 2026-09-14 change request is resolved at 9ffb472: both module parents are 4.23.0-SNAPSHOT, the generated catalog/DSL files are committed and match the module sources, CAMEL-24172 is linked, firstVersion/:since: are 4.23, the tests use AssertJ, the quay.io image covers amd64/arm64/ppc64le/s390x, and username/clientId are no longer secret. CI is green, the component is well structured, it has no unsafe deserialization, and the consumer sets only a fixed set of headers. I'm keeping changes requested for one new consumer bug: with fetchContent=true, a DISABLED version permanently stalls delivery. I'd also like searchArtifacts to stop silently truncating at 20 results. davsclaus's WebClient-leak and setVertx threads are still open at this head; the remaining findings are minor.

Claude Code on behalf of oscerd. This review was generated by an AI agent and may contain inaccuracies. Please verify all suggestions before applying. It does not replace specialized review tools or static analysis.

Comment thread components/camel-apicurio-registry/src/main/docs/apicurio-registry-component.adoc Outdated
@carlesarnal

Copy link
Copy Markdown
Author

Thanks both. I pushed 9ef714c and replied on each inline thread.

@davsclaus, for the points outside the inline threads:

  • A version that always fails: the docs now say it blocks newer versions and show how to skip it with onException(...).handled(true). There is a route test where version 1 always fails and is handled, and version 2 is still delivered.
  • Logged twice: when the route's error handler has already dealt with the failure, the consumer now only logs at debug. It still uses the exception handler when no error handler was involved.
  • Deletion between pages: documented as a known limitation of offset paging.
  • Redundant watermark check: removed.

@oscerd, all of your points are addressed: DISABLED and missing versions with fetchContent, searchArtifacts filtering and paging, security = "secret", fail-fast ID checks, the idle-poll probe, TLS/proxy options, and docs for deletion and consumer headers.

Locally, 53 unit tests and 8 integration tests pass. The generated catalog and DSL files are updated.

AI-assisted response prepared with OpenCode on behalf of @carlesarnal.

@oscerd oscerd left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Carles, all eight points from my previous review are fixed at 9ef714c, and I checked each in the code and tests. DISABLED/404 versions no longer stall fetchContent=true (other errors are still retried). searchArtifacts filters by artifact ID/type/labels and has offset/limit/order headers. Secrets use security = "secret". Missing IDs fail fast. An idle poll is a single limit=1 probe. TLS/proxy options exist, and the docs cover artifact deletion and consumer-header precedence. davsclaus's WebClient-leak and setVertx threads look addressed too, and the generated files match the sources. No CI has run on this head yet (the workflows are waiting for approval), so I'll approve once it is green. The three new points below are minor and can be follow-ups.

Claude Code on behalf of oscerd. This review was generated by an AI agent and may contain inaccuracies. Please verify all suggestions before applying. It does not replace specialized review tools or static analysis.

Comment thread components/camel-apicurio-registry/pom.xml

@davsclaus davsclaus left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @carlesarnal, all my points from the last round are addressed: each endpoint now builds and closes its own WebClient (and the OAuth2 provider), setVertx() never closes a user-supplied Vert.x while still closing the one the component created, the docs explain how to skip a version that always fails with onException(...).handled(true) and mention the offset-paging caveat, the double logging is gone (isErrorHandlerHandledSet()), and the redundant watermark check is removed. I also checked oscerd's 2026-10-06 points in the code, and they are all fixed. CI is green on 9ef714c.

Before merge:

  • Please rebase onto main and regenerate. The branch is based on a commit from before CAMEL-25407 (#27474), which changed the UriFactory generator, so ApicurioRegistryEndpointUriFactory is generated differently on current main (inline) and main's uncommitted-changes check would fail after merge. It merges without conflicts, so it should only be a rebase plus mvn install in the module.
  • I agree with oscerd's three comments from today: Basic/OIDC credentials are no longer validated now that the SDK takes the customWebClient path, skips are logged only at DEBUG, and the dependency tree has vertx-core 4.5.34 next to vertx-web-client/vertx-auth-oauth2 4.5.26 (via kiota-http-vertx). Declaring vertx-web-client and vertx-auth-oauth2 with ${vertx-version} would fix the last one.

A few small ones inline. With the rebase and the credential fail-fast, I'm happy to approve this for 4.23 as a Preview component.

Claude Code on behalf of davsclaus. This review was generated by an AI agent and may contain inaccuracies. Please verify all suggestions before applying.

@davsclaus
davsclaus self-requested a review October 7, 2026 12:32

@oscerd oscerd left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed at head 9ef714c5. All eight points from my 2026-10-06 review are genuinely fixed — I verified each in the code rather than taking my own earlier follow-up on trust:

# Item Evidence at head
1 fetchContent=true stalls on a DISABLED version ApicurioRegistryConsumer.java:65-70 skips VersionState.DISABLED and advances lastSeenGlobalId; :121-134 returns null only on HTTP 404 and rethrows otherwise, so transient 5xx still retry
2 searchArtifacts truncates at 20, ignores artifact ID ApicurioRegistryProducer.java:182-212 now honours artifactId, artifactType, labels, offset, limit, order, orderby
3 Deprecated secret = true ApicurioRegistryConfiguration.java:47,56,93 all use security = "secret"; none left
4 Consumer headers override a downstream producer's URI path Documented at apicurio-registry-component.adoc:107-117 with the removeHeaders("CamelApicurioRegistry*") example. Behaviour unchanged — a reasonable call
5 Fail fast on missing group/artifact IDs ApicurioRegistryEndpoint.createConsumer():86-89 throws at route start
6 Idle polls download 100 versions every 500 ms ApicurioRegistryConsumer.java:145-151 does a limit = 1 probe first
7 No TLS / proxy configuration ApicurioRegistryConfiguration.java:79-94 adds sslContextParameters (wired via VertxHelper.setupSSLOptions) plus proxy options. Correctly no trustAll knob, so no security = "insecure:ssl" is owed
8 Document that the registry must allow deletion apicurio-registry-component.adoc:64 covers apicurio.rest.deletion.artifact.enabled and the 405 otherwise

Eight substantive findings turned around in a single iteration, on a PR open since July — that is a good showing, and every one of them checks out.

The new-component checklist is also clean: direct child of components/ so no MojoHelper registration is needed (correctly absent); catalog JSON and docs mirrors are byte-identical to the module copies; camel-bom, camel-allcomponents, parent/pom.xml and ArtifactUtils entries all present; tests are AssertJ-only with no mixing and no Thread.sleep; and the container image quay.io/apicurio/apicurio-registry:3.3.0 publishes amd64, arm64, s390x and ppc64le (I queried the quay manifest list), so no skipITs.* is needed.

I'm keeping the block, but re-scoped to two things

1. The branch is 952 commits behind main, and the green CI is stale in a way that matters. All 7 checks pass on 9ef714c, but that run completed 2026-10-07 08:20Z and CAMEL-25407 (#27474, c9737a20e661) merged at 08:33Z — thirteen minutes later. That commit changed the EndpointUriFactory generator, and I confirmed the drift directly:

this PR:        Map<String, Object> copy = new HashMap<>(properties);
current main:   Map<String, Object> copy = copyParameters(properties);

So the uncommitted-changes gate will fail on rebase. Please rebase onto current main, regenerate, commit, and let CI re-run. I checked the catalog JSON key set against a current-main component and found no further drift, but 952 commits touch several packaging mojos, so a full mvn clean install after the rebase is the only safe answer.

2. @davsclaus' auth findings from 2026-10-07 post-date the last commit and are still open. The one I would most want closed: with authType=basic and no username/password, ApicurioRegistryEndpoint builds Base64("null:null") and sends Authorization: Basic bnVsbDpudWxs on every request — silent 401s rather than a startup failure. An unknown authType (a typo like oauth) likewise falls through to the bare webClient with auth silently disabled. A single validation block in createWebClient fixes both, plus the related WebClient leak if OAuth2Auth.create throws.

Two prior claims I checked and would not pass on

  • @gnodet-bot's re-raised "testCompatibility does not set HEADER_VALIDATION_RESULT" is false at head — ApicurioRegistryProducer.java:290 sets it. Stale thread, just needs resolving.
  • @gnodet-bot's "getServiceProtocol() returns "http" — wrong identifier" is not a defect: 8 components on main return the literal "http", and the EndpointServiceLocation javadoc gives "http, amqp, tcp" as examples.

Merge housekeeping for later

CAMEL-24172 is Open, unassigned, fixVersions empty — it needs assigning to @carlesarnal with fixVersions = 4.23.0 set before closing. There are also ~26 unresolved threads (mostly stale bot ones) that need a sweep, since unresolved conversations block merge.

Happy to approve as soon as the rebase is green and the auth fail-fast is in.


Reviewed with Claude Code (Claude Opus 5) on behalf of @oscerd. This review was generated by an AI agent and may contain inaccuracies; please verify all suggestions before applying. It is a rules-and-conventions review and does not replace CodeRabbit, Sourcery, or SonarCloud.

@carlesarnal
carlesarnal force-pushed the camel-apicurio-registry-component branch from 9ef714c to e828c1e Compare October 14, 2026 12:51

@gnodet-bot gnodet-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Following up on my earlier reviews — all three previously open findings are now addressed.

HEADER_VALIDATION_RESULT not set by testCompatibility — ✅ addressed. Both testCompatibility() and validate() now set the header symmetrically via doDryRun(), with unit tests (testTestCompatibilitySuccess, testTestCompatibilityFailure) verifying the header value for both outcomes.

Unbounded versions().get() — silent data loss on 20+ versions — ✅ addressed. fetchNewVersions() paginates 100 at a time, newest-first, with a cheap limit=1 probe to skip full pagination on idle polls. listVersions() paginates identically. FakeVersionPages simulates real registry default-page behaviour (20 entries) and tests verify correct offsets for 250 versions.

I also reviewed the new changes in this push (e828c1e5f9c): credential validation in doStart(), unknown authType rejection, NoCookieStore for Basic auth sessions, OAuth2WebClient flow on credentials (not the deprecated OAuth2Options), and explicit Vert.x web-client/auth version alignment. No issues found. The resource cleanup path for the OIDC OAuth2Auth instance is correctly handled by closeOwnedResources() (it checks ownedWebClient and oauth2 independently), and notFoundGlobalId is safe without volatile because ScheduledPollConsumer uses a single-thread executor.

LGTM.

This review was generated by an AI agent, Hermès on behalf of @gnodet.

@carlesarnal

Copy link
Copy Markdown
Author

Thanks @oscerd and @davsclaus. I've pushed e828c1e, rebased onto current main.

  • Rebase: ApicurioRegistryEndpointUriFactory is regenerated and now uses copyParameters(properties). The docs and catalog mirrors are regenerated too.
  • Auth fail-fast: missing Basic/OIDC credentials, or an unknown authType such as oauth, now fail the endpoint start with a message naming the option. If OAuth2 setup fails, the WebClient is closed. Each case has a test.
  • Skipped versions: a 404 on content is retried once on the next poll and only then skipped, with a WARN. A skipped DISABLED version is logged at INFO. The docs say skipped versions are not redelivered.
  • Vert.x: vertx-web-client and vertx-auth-oauth2 are declared with ${vertx-version}, so everything resolves to 4.5.35.
  • Smaller items: the deprecated setFlow call is gone, the Basic auth session keeps no cookies (tested over HTTP), and an unknown ifExists value now fails clearly.

Locally, 63 unit tests and 8 integration tests pass. I resolved the 18 older threads you both confirmed as fixed, including the stale bot ones, and left this round's threads open for you.

JIRA: I'll assign CAMEL-24172 to myself and set fixVersions to 4.23.0 before it is closed.

AI-assisted response prepared with OpenCode on behalf of @carlesarnal.

@carlesarnal

Copy link
Copy Markdown
Author

CI on e828c1e is green apart from license-check. That job failed before any check ran: Maven couldn't download the Develocity extension from Central ("Connection reset"). The same check passes locally. Could someone with access re-run the failed job (run)? I can't re-run it from my account.

AI-assisted response prepared with OpenCode on behalf of @carlesarnal.

@davsclaus davsclaus left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code on behalf of davsclaus

Thanks @carlesarnal. The rebase and the last round of fixes are in, and I checked each one against the code at e828c1e:

  • ApicurioRegistryEndpointUriFactory is regenerated on current main (copyParameters(properties)), and the build jobs pass on the rebased head with no uncommitted changes.
  • validateAuthentication() makes the endpoint fail on start when Basic or OIDC credentials are missing or authType is unknown. The plain WebClient is closed if OAuth2Auth.create throws, and the parameterized test covers every missing option.
  • The OAuth2 flow is now set on Oauth2Credentials, so the deprecated OAuth2Options.setFlow call is gone.
  • NoCookieStore keeps response cookies out of the Basic-auth session, and the HTTP test checks that the second request sends no cookie.
  • ifExists goes through parseEnum, so an unknown value fails with a clear message.
  • A content 404 is retried once on the next poll before the version is skipped with a WARN. The version list is ordered by globalId, so the retry hits the same version first. The docs say that skipped versions are not delivered again.
  • vertx-web-client and vertx-auth-oauth2 are declared with ${vertx-version}.

I resolved my five threads. oscerd's three threads are addressed in the same commit too, but I'll leave them for him to resolve.

On CI, the only red check is license-check. It fails while downloading develocity-maven-extension from Maven Central (Connection reset), so it's an infrastructure flake and not caused by this PR. A re-run should clear it.

Before merging, CAMEL-24172 needs an assignee and fixVersions set before the issue is closed.

Approving as a Preview component. Thanks for sticking with this through so many review rounds.

This review was generated by an AI agent and may contain inaccuracies. Please verify all suggestions before applying.

@oscerd oscerd left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TL;DR

All of my earlier points are fixed at e828c1e and the new commit reads well. However, main cut 4.23.0 and moved to Vert.x 5.2.1 on 2026-10-15, the day after this branch was last rebased, so we need one more rebase before merging. Nothing in the PR caused this.

Blockers

  1. Stale 4.23 versions. The module parents (components/camel-apicurio-registry/pom.xml:27, test-infra/camel-test-infra-apicurio-registry/pom.xml:25) and the generated bom/camel-bom/pom.xml:142 entry still say 4.23.0-SNAPSHOT, while main has been 4.24.0-SNAPSHOT since 5686684 (#27901). GitHub shows no conflict because these are added lines, but the merged build cannot resolve those parents: it is the same Non-resolvable parent POM failure as in my first review. 4.23.0 has shipped without this component, so firstVersion (ApicurioRegistryEndpoint.java:53) and :since: (apicurio-registry-component.adoc:6) should become 4.24.0 / 4.24, as camel-test-citrus did. A full regen then updates the catalog JSON, component.properties, the test-infra metadata and the Since: line in the DSL javadoc.
  2. No CI on Vert.x 5 yet. CAMEL-23647 (#27825) moved vertx-version from 4.5.35 to 5.2.1. After the rebase, pom.xml:52,57 and camel-vertx-common resolve to 5.2.1, but the SDK's adapter (kiota-http-vertx 0.0.35, from Apicurio 3.3.0 on Quarkus 3.33.1) is built for Vert.x 4.5. I checked the 5.2.1 sources: the endpoint's own calls and the adapter's customWebClient path (requestAbs, send, sendBuffer, toCompletionStage) all still exist. OAuth2Options.setFlow is gone in 5.2.1, so moving the flow to Oauth2Credentials was the right call. I expect CI to pass, but we need to see it pass. If it fails, the camel-arangodb note in the 4.24 upgrade guide is the precedent.

Other findings: 3 non-blocking (1 suggestion, 1 question, 1 Jira note)

Follow-up on my earlier comments

  • 2026-09-14 (parents, generated files, Jira, AssertJ, image architectures, username/clientId): still fixed, apart from the version numbers above. The quay 3.3.0 manifest still lists amd64, arm64, ppc64le and s390x.
  • 2026-10-06 (eight inline points): still fixed. e828c1e keeps the DISABLED skip and refines the 404 path.
  • 2026-10-07: fixed. Missing credentials now fail the endpoint start (ApicurioRegistryEndpoint.java:154-207, with the six-case test and the WebClient-close test). DISABLED skips log at INFO, a repeated 404 logs at WARN, the docs say skipped versions are not delivered again, and the Vert.x libraries use ${vertx-version}.
  • 2026-10-12: the rebase/regen (copyParameters) and the auth fail-fast are done, and all review threads are resolved.

New

  • [Suggestion] ApicurioRegistryConsumer.java:95-101: with greedy=true, ScheduledPollConsumer polls again at once after a poll that delivered a version. So the "retry on the next poll" can come milliseconds later, and a version on a lagging replica is then skipped for good. A doc line would help, or the consumer could give up only once delay has passed since the first 404, which keeps the wait.
  • [Question] parent/pom.xml:82: is SDK 3.3.0 deliberate? 3.3.3 is the latest patch on Central. If there is no reason to stay on 3.3.0, bumping the SDK and the test image tag together during the rebase would start the component on the latest fixes.
  • CAMEL-24172 is still Open, unassigned and has no fixVersions. The target is now 4.24.0, not the 4.23.0 planned on 10-14.

Happy to approve once the rebased head is green.

Reviewed with Claude Code on behalf of oscerd. This review was generated by an AI agent and may contain inaccuracies; please verify all suggestions before applying.

carlesarnal and others added 10 commits October 23, 2026 11:42
New Camel component wrapping the Apicurio Registry v3 REST API
via the apicurio-registry-java-sdk.

Assisted by Claude, model: claude-opus-4-6
Assisted by Claude, model: claude-opus-4-6
- Remove unused cacheTtl option and HEADER_DRY_RUN constant
- Sort versions by globalId before processing in consumer
- Convert InputStream to byte[] in getArtifactContent to avoid leaks
- Extract shared doDryRun helper for testCompatibility/validate
- Catch only ApiException in dry-run operations, rethrow others
- Move apicurio-registry-sdk-version property to parent/pom.xml
- Document consumer first-start replay behavior in .adoc
- Add unit tests for searchArtifacts, testCompatibility, validate
- Add consumer unit test for out-of-order globalId sorting
- Improve validate IT assertions

Assisted by Claude, model: claude-opus-4-6
Assisted by Claude, model: claude-opus-4-6
Update to Camel 4.23 and regenerate catalog, BOM, documentation and DSL files. Manage SDK resources, preserve injected clients, distinguish rule violations from infrastructure failures, and expand authentication, validation and consumer coverage.

Co-authored-by: GitHub Copilot <copilot@github.com>
Use manual pooled exchange release on success and failure, preserve retries after failed processing, normalize empty group IDs, and document compatibility error headers with regression coverage.

Co-authored-by: GitHub Copilot <copilot@github.com>
The consumer no longer advances its watermark when the route error handler records a failure, version lists are paged instead of truncated at the registry default, and endpoints share one component-level Vert.x instance. testCompatibility also sets the result header. Tests move to camel-test-junit6 and the test infra is registered in camel-test-infra-all.

Co-authored-by: GitHub Copilot <copilot@github.com>
…rches

Endpoints now own and close their Vert.x WebClient (and OAuth2 provider), with TLS and proxy options. setVertx no longer lets the component close a user instance. The consumer skips disabled or missing versions when fetching content, probes before paging, fails fast on missing IDs and avoids logging failures twice. searchArtifacts applies the artifact ID and other filters and exposes paging and sorting headers. Producer operations report missing IDs clearly, and the docs cover deletion, consumer headers and retry behaviour.

Co-authored-by: GitHub Copilot <copilot@github.com>
…gn Vert.x

Rebased on main and regenerated the endpoint URI factory. The endpoint fails to start on missing Basic/OIDC credentials or an unknown authType, closes its WebClient when OAuth2 setup fails, avoids the deprecated OAuth2 flow option and keeps no cookies. A 404 for version content is retried once before the version is skipped with a warning. Unknown ifExists values fail clearly, and the Vert.x client libraries are declared with the Camel Vert.x version.

Co-authored-by: GitHub Copilot <copilot@github.com>
@davsclaus
davsclaus force-pushed the camel-apicurio-registry-component branch from e828c1e to 3aada5b Compare October 23, 2026 09:47

@gnodet-bot gnodet-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Following up on my earlier reviews (2026-09-15 and 2026-10-14) — all four findings are now addressed.

Resolved ✅

  • Empty group ID normalization: remaining.split("/", 2) now sets groupId = null (not "") — confirmed in ApicurioRegistryComponent and tested with a parsing test for apicurio-registry:/someArtifact.
  • Exchange auto-release: createExchange(false) paired with releaseExchange(exchange, false) in a finally block, covering content-fetch failures, processor failures, and normal completion.
  • HEADER_VALIDATION_ERRORS documented for testCompatibility: ApicurioRegistryConstants now carries @Metadata(description = "Rule violation details from validate or testCompatibility. Cleared before each check."), and the adoc explicitly says both operations set this header. Tests testTestCompatibilitySuccess and testTestCompatibilityFailure verify it is cleared before each check and populated on failure.
  • Exchange release on pre-processor failure: confirmed covered in the pooled-exchange tests.

The three new commits (dc31f42, 954c450, 3aada5b) add substantial improvements: shared Vert.x with per-endpoint WebClient lifecycle, credential validation on startup, 404-retry-once logic, pagination probe, and disabled-version skipping. All well-tested.

Two minor observations on the new code

Inline comments below.


This review was generated by an AI agent, Hermès on behalf of @gnodet.

private final ApicurioRegistryEndpoint endpoint;
private final ApicurioRegistryConfiguration configuration;
private volatile Long lastSeenGlobalId;
private Long notFoundGlobalId;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: notFoundGlobalId is not volatile, which is correct — ScheduledPollConsumer guarantees that poll() is called sequentially on a single scheduler thread, so there is no cross-thread visibility concern here. A brief comment would prevent future reviewers from reflexively flagging it:

// only accessed from the single poll thread managed by ScheduledPollConsumer — volatile not needed
private Long notFoundGlobalId;

@Override
protected void doStop() throws Exception {
super.doStop();
if (ownedWebClient != null) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: doStop() only nulls registryClient when ownedWebClient != null. When the client was injected via setRegistryClient(), the field is intentionally left in place so a subsequent doStart() reuses it. The logic is correct, but the asymmetry can be surprising at first glance — a brief comment on the if guard would help:

// only clean up resources this endpoint owns; a caller-injected registryClient survives restart
if (ownedWebClient != null) {
    registryClient = null;
    closeOwnedResources();
}

@davsclaus davsclaus left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @carlesarnal. The code is unchanged since my approval on e828c1e, and this head is only a rebase onto current main. Two things are needed before merge:

  • Bump to 4.24. main moved to 4.24.0-SNAPSHOT, and 4.23.0 shipped without this component. The <parent><version> in components/camel-apicurio-registry/pom.xml:27 and test-infra/camel-test-infra-apicurio-registry/pom.xml:25 is still 4.23.0-SNAPSHOT, so build (21, false) fails at once with Non-resolvable parent POM. Please also change firstVersion in ApicurioRegistryEndpoint.java:53 to 4.24.0 and :since: in the adoc to 4.24, then regenerate. That updates the bom entry, the catalog JSON, component.properties and the "Since" lines in the Endpoint DSL javadoc.
  • Vert.x 5. After that, CI will run against Vert.x 5.2.1 for the first time (oscerd's second point). Statically, everything kiota-http-vertx and the customWebClient path call exists in 5.2.1 with the same signatures, so I expect it to pass, but a green run is what settles it.

While you're at it: oscerd's SDK 3.3.0 → 3.3.3 question and his greedy=true note are worth a look, and the PR description still mentions 4.23. The two gnodet-bot nits only ask for code comments; adding them or replying is fine.

Reviewed with Claude Code on behalf of davsclaus. This review was generated by an AI agent and may contain inaccuracies; please verify all suggestions before applying.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants