Skip to content

CAMEL-24961: Catalog body types for the core components and languages - #28276

Merged
davsclaus merged 41 commits into
mainfrom
feature/CAMEL-24961-catalog-body-types-core
Oct 24, 2026
Merged

davsclaus merged 41 commits into
mainfrom
feature/CAMEL-24961-catalog-body-types-core

Conversation

@davsclaus

@davsclaus davsclaus commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Stacked on #28275. This is the core group of the catalog body types (CAMEL-24961): the core components without body types so far, knative, and the core languages.

What the catalog says now

Component Consumer Producer input Producer output
bean, class (producer only) any: the first parameter of the method gets it, converted to its type unknown: what the method returns, null too; unchanged after a void method
browse the body the producer sent any unchanged
controlbus (producer only) any; the expression as text for the language command unchanged; text (or unchanged) for status and stats; the result of the expression for the language command
dataformat (producer only) any: as the data format named in the path takes it unknown
dataset unknown: what the DataSet bean creates any, compared with the DataSet unchanged
dataset-test (producer only) any, compared with the expected bodies unchanged
dynamic-router (producer only) any unknown: what the recipient ends with; a text when no subscription matches
dynamic-router-control (producer only) any the subscription id (text) for subscribe and update, true for unsubscribe, text for list and statistics
event a CamelEvent (pojo); a List with batchSize (consumer only)
exec (producer only) any: the standard input, when it converts to a stream the ExecResult (pojo)
knative bytes or no body; the parsed data field of a structured event text, bytes, stream or file the response: bytes, or null
language (producer only) any: as the language reads it; the script never comes from the body unknown: the result of the script; unchanged with transform=false; a stream with binary=true
once no body; with the body option, not known: text, a resource, a value or the result of a language (consumer only)
ref unknown: as the endpoint in the registry any unknown
rest-api no body (the GET of the API document) (consumer only)
saga (producer only) any unchanged
stream each line as text; a List with groupLines; bytes with readLine=false text, bytes, stream, file or no body unchanged
stub the body the producer sent any unknown: unchanged for InOnly or without a consumer
validator (producer only) XML: text, bytes, stream, file or a DOM unchanged
wasm (producer only) text, bytes, stream, file or no body, as the standard input bytes: the standard output
xslt-saxon (producer only) as xslt, and JSON text with useJsonBody as xslt
Language Reads Gives
bean (method) any: the first parameter of the method unknown: what the method returns; null for a void method
header, exchangeProperty, variable (not the body) unknown: the value as it is
ref any unknown
file any, as simple unknown: text, a Long for file:size and file:modified
tokenize text, bytes, stream or file an iterator of text
xtokenize XML: text, bytes, stream, file or a DOM an iterator of XML text

The xpath and jq languages read the source option instead of the body when it is set, so their input side now depends on it, as for tokenize and xtokenize.

The bodies a step needs have one word for whatever the body is: any. The build now refuses unknown on those sides; unknown stays for the body after a step.

Checked at runtime. Routes run with camel run logged the class of the body after each step:

  • bean:
    • a void method keeps the body, and a method that returns null makes it null;
    • a Map given to a String parameter is its toString();
    • the method language gives null for a void method.
  • language:
    • transform=false keeps the body;
    • binary=true with a resource: script gives a stream;
    • the script is never taken from the body.
  • controlbus:
    • status gives the text of the state, and stats XML;
    • start and stop keep the body, and so does a route that does not exist;
    • the language command gives the result of the expression.
  • stream:
    • the consumer gives each line as a String, a List with groupLines, and a byte[] with readLine=false;
    • stream:out writes text, bytes, a stream and the content of a file, and a Map as its toString(); a List fails.
  • exec: the body is the standard input (cat), and the result is an ExecResult; a Map is not sent.
  • validator: text, bytes and a stream pass; a Map and no body fail.
  • xslt-saxon: the outputs string, bytes and DOM.
  • browse and stub: the producer keeps the body, and their consumers get it.
  • Also run:
    • once, dataset, event, and dynamic-router with its control endpoint;
    • knative, with binary and structured events over a local environment file;
    • the header, exchangeProperty, variable, file, tokenize and xtokenize languages in setBody.
  • Read from the code only: ref, rest-api, saga, wasm and dataset-test.

Docs fixed. These pages said things about the body that the code does not do:

  • stream:
    • the producer writes bytes, streams and files too, each with a new line (appendNewLine);
    • stream:header reads the stream header, not header.
  • exec:
    • the package of ExecResult was wrong;
    • with outFile, the body is still the ExecResult, which converts to the content of the file.
  • language:
    • a script in the header needs allowTemplateFromHeader=true;
    • the groovy example was not valid Groovy.
  • once: body=file:data.json (it said file.data.json, which is the literal text).
  • browse: the Java example did not compile.
  • dataset: defaultHeaders is the headers, not the body.

Measured.

  • I ran the validator over the 1204 human-written and 1209 model-written route files, and the reports are the same as with CAMEL-24961: Catalog body types for cloud messaging #28275.
  • The share of the endpoint uses in the human-written routes whose component has body types goes from 90.2% to 91.4%.

Tests

  • TextBodyFlowTest.testTheBodyOfCoreComponents:
    • a field read after the stream consumer is reported;
    • parsed data sent to stream:out is reported;
    • the knative consumer and the body after a bean are not reported.
  • These pass:
    • the 283 tests of camel-yaml-dsl-validator;
    • the 265 validator and catalog doc tests of camel-jbang-core;
    • the build of camel-yaml-dsl-deserializers, which reads the catalog.
  • The answers of camel_catalog_doc stay under 18 000 characters: knative, the largest of this group, is 12 555.

Claude Code on behalf of davsclaus

🤖 Generated with Claude Code

davsclaus and others added 29 commits October 21, 2026 13:58
…s, and the body it leaves after a step

A @BodyTypes annotation on the endpoint, data format or language class says the
body per side (consumerOut, consumerReply, producerIn, producerOut, marshal,
unmarshal and expression in and out): its carriers (null, text, bytes, stream,
file, value, map, list, pojo, unchanged, any, unknown), the Java type when it is
always the same, the format, the options it depends on, the body for other
option values, and a description. The package plugin writes it as a bodyTypes
section of the catalog JSON and checks that each side fits the kind and that
the options it names exist. A subclass declares its own.

The first slice: file, direct, seda, timer, cron, kamelet, http, https,
rest-openapi, xslt, sql and the json data format (jackson), read from the code
that sets the body and checked against the body types the message history
records when routes run. camel_catalog_doc returns them.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
…tp, jms, kafka, aws2-sqs, the csv, jacksonXml and yaml data formats, and the languages

The second slice of the catalog body types:
- components: platform-http, rest (and the rest-openapi consumer), ftp, ftps,
  sftp, jms, activemq, activemq6, amqp, kafka, aws2-sqs, log, mock, scheduler,
  quartz;
- data formats: csv, jacksonXml, snakeYaml;
- languages: simple, constant, jsonpath, xpath, groovy, jq.

Each value is read from the code that sets the body, and checked against the
body types the message history recorded when the tests of the modules and
reference routes ran (the ftp integration tests included).

camel_catalog_doc gives the body types a line per side, and its answer is cut
at 18000 characters instead of 16000, so the four sides of the jms components
leave the options a limit asks for room.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
paho-mqtt5: the consumer gives the payload as a byte array, the producer sends
what converts to one and leaves the body unchanged.

mongodb: the consumer gives a Document (or null for a change without its full
document with changeStreams); the producer takes a document or a query, and
leaves a body by operation: a Document or null for a find of one, a List for
findAll, findDistinct and aggregate, a Long for count, the result of the driver
for the other writes.

Read from the code that sets the body, and checked against a mosquitto and a
mongodb container started with camel infra.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
…ts an option value without carriers

From the review of PR 28241:

- An iterator is read once and has no size and no index, so tooling that
  reads list as a List accepts ${body.size()} or ${body[0]} on it, which fail
  at runtime. The new iterator carrier names it: sql with outputType=StreamList
  gives an iterator, not a list.
- A @BodyType.When with no carriers (carrier = {}) wrote "carriers": [] into
  the catalog. The build now rejects it, as it does a side without carriers.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
…MEL-24961-catalog-body-types-2

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
… camel_catalog_doc says unknown for no carriers

From the review of PR 28242:

- The csv data format with lazyLoad=true gives an iterator, with the iterator
  carrier of PR 28241, not a list.
- camel_catalog_doc says unknown for a body without carriers, instead of
  failing. The build rejects it now, but a catalog read from elsewhere may
  have one.
- Sqs2Endpoint imports org.apache.camel.spi.* already: only the import of the
  nested BodyType.Carrier stays.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
…types of the catalog instead of their own tables

The YAML validator's body walks read what a component or a data format does
with the body from the catalog, instead of from tables in their source:

- the consumer a route starts with (consumerOut): the file as read, text,
  bytes or data, instead of the file consumer only;
- the body after a to: (producerOut): unchanged for log and mock and now for a
  file write, the response of the http producer as bytes or a stream;
- the producers that need text (producerIn), instead of file, ftp, ftps and
  sftp only;
- the body after an unmarshal (unmarshalOut), instead of the json, jacksonXml,
  yaml and csv table and the options that make something else;
- the consumers that reply with text (consumerReply), instead of platform-http;
- the consumers that give no body, instead of timer, quartz, scheduler and cron.

A side holds for a step only when the step decides every option it depends
on: an option the component can also be configured with (the valueDeserializer
of kafka) counts only when the endpoint sets it, and a placeholder value is not
known. The catalog of an older Camel version has no body types, so the
validator's own catalog stands in for it. After a step that fails at runtime
the walk stops, so a file write that keeps the body is reported once.

The rules about HTTP methods, REST bindings and EIPs stay in the validator; the
EIPs get body types with CAMEL-25928.

The same reports as before on 1204 human-written route files (the Camel
repository, the examples, kamelets, camel-k and camel-kit) and on 1209
model-written ones (the r17 benchmark series), and the doc examples of the
validator tests still validate.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
…docling, milvus, qdrant, ai-tool and ibm-watson-language

The AI components declare the body they need and the body they leave, per
operation where the operation decides it. The answer of openai and
langchain4j-chat is text, also when it is asked for JSON, so the YAML
validator now reports a field read on it that has no unmarshal before.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
… endpoint count for its body types

The operation of openai is in the path (openai:embeddings), so the validator
reads the path options with the syntax of the component, as the catalog
parses an endpoint. A path the catalog cannot read, such as a toD with an
expression, says nothing about its options.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
…pe for a value its option cannot have

A byOption value must be one of the enum of its option, or true or false for
a boolean, ignoring case as tooling compares it with the endpoint: a
misspelled operation such as embedding for embeddings would never hold.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
The attribute holds an array, and the catalog JSON and BodyTypeModel call
the same field carriers. The Javadoc of BodyTypeModel.isUnknown() says that
a side with no carriers is unknown too.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
…AMEL-24961-catalog-body-types-ai

# Conflicts:
#	tooling/maven/camel-package-maven-plugin/src/main/java/org/apache/camel/maven/packaging/BodyTypesHelper.java
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
The body types of aws2-s3, azure-storage-blob, azure-files, dropbox,
google-storage, ibm-cos and minio, read from the code and checked at
runtime against LocalStack S3 (aws2-s3, and ibm-cos and minio through
their S3 API) and Azurite (azure-storage-blob).

The pages of aws2-s3, google-storage and minio said getObject returns an
S3Object, a Blob or a MinioObject: it returns the content of the object.

The validator calls a body the file as read (a GenericFile) only when the
file is its only carrier, not for a blob that is a stream or a file.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
…e data format and language schemas

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
…n read as a list, as the carriers do

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
The body types of aws2-sns, aws2-kinesis, aws2-kinesis-firehose,
google-pubsub, azure-servicebus, azure-eventhubs and azure-storage-queue,
read from the code and checked at runtime against LocalStack (SNS and
Kinesis), the Pub/Sub emulator and Azurite (queues).

aws2-sqs sends bytes, a stream or a file as text as well, and
sendBatchMessage takes a SendMessageBatchRequest too, which its producer
side now says.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
The body types of bean, class, browse, controlbus, dataformat, dataset,
dataset-test, dynamic-router, dynamic-router-control, event, exec,
knative, language, once, ref, rest-api, saga, stream, stub, validator,
wasm and xslt-saxon, and of the bean, header, exchangeProperty, variable,
ref, file, tokenize and xtokenize languages, read from the code and
checked at runtime with camel run (knative over a local environment
file).

The xpath and jq languages read the source option instead of the body
when it is set, which their expressionIn now says, as tokenize and
xtokenize do.

The component docs of stream, exec, language, once, browse and dataset
said things about the body that the code does not do.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
@davsclaus davsclaus added this to the 4.24.0 milestone Oct 22, 2026
@github-actions

Copy link
Copy Markdown
Contributor

🌟 Thank you for your contribution to the Apache Camel project! 🌟
🤖 CI automation will test this PR automatically.

🐫 Apache Camel Committers, please review the following items:

  • First-time contributors require MANUAL approval for the GitHub Actions to run
  • You can use the command /component-test (camel-)component-name1 (camel-)component-name2.. to request a test from the test bot although they are normally detected and executed by CI.
  • You can label PRs using skip-tests and test-dependents to fine-tune the checks executed by this PR.
  • Build and test logs are available in the summary page. Only Apache Camel committers have access to the summary.

⚠️ Be careful when sharing logs. Review their contents before sharing them publicly.

@gnodet-bot gnodet-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All 6 documentation fixes verified against source code and official documentation:

  1. browse-component.adoc — getBody() → getBody(String.class): correct — the variable is typed String, so getBody(String.class) is the type-safe Camel idiom.
  2. dataset-component.adoc — defaultHeaders described as "default message body" → "default message headers": correct — the property is Map<String,Object> for headers.
  3. language-component.adoc — ${body} * 2} → body * 2: correct — ${body} is Simple language syntax, not Groovy; also fixes the trailing mismatched }.
  4. language-component.adoc — added allowTemplateFromHeader=true: correct — required when providing a script via the CamelLanguageScript header (security default is false).
  5. once-component.adoc — body=file.data.json → body=file:data.json: correct — file: is the documented URI scheme prefix per the body parameter description.
  6. stream-component.adoc — header → stream: correct — StreamProducer.java confirms exchange.getIn().getHeader("stream").

The @BodyTypes annotation additions and JSON catalog changes are consistent metadata enhancements.

This review was generated by an AI agent, Hermès on behalf of @gnodet.

davsclaus and others added 3 commits October 22, 2026 11:55
…as the other components do

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
ref, dataformat, the language component and the ref language said
unknown for the body they take. The bodies a step needs now have one word
for whatever the body is, any, and the build refuses unknown there:
unknown stays for the body after a step.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>

@gnodet-bot gnodet-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Following up on my earlier review — the documentation fixes from the previous SHA are all present and verified in the new commit.

This PR's new commits add @BodyTypes annotations to the core component and language endpoint classes, with matching JSON catalog regeneration. I've verified the following:

  • Carrier semantics: no production code uses Carrier.UNKNOWN on input sides (producerIn, consumerIn, expressionIn) or Carrier.ANY on output sides (producerOut, consumerOut, expressionOut). The one UNKNOWN-on-input occurrence is in SomeEndpointWithBadBodyTypes.java — a negative test confirming the validator rejects it correctly.
  • Annotation ↔ JSON consistency: spot-checked StreamEndpoint (complex byOption/dependsOn / javaType), KnativeEndpoint, ControlBusEndpoint. Java annotations and generated JSON are in sync.
  • dependsOn option names: all validated against component option names (groupLines, readLine, body, batchSize, binding, consumerComponentName, headerName, failOnNullBody, source, useJsonBody, output). All exist.
  • BodyType.java Javadoc: UNKNOWN carrier doc updated identically in both tooling/spi-annotations (source) and core/camel-api (generated copy). The added sentence correctly states the carrier constraint.
  • File pairing: all 20 components/languages have changes in both the component-local generated JSON and the catalog copy — no orphans.

LGTM.

This review was generated by an AI agent, Hermès on behalf of @gnodet.

…atalog-body-types-3

Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
…atalog-body-types-ai

# Conflicts:
#	docs/user-manual/modules/ROOT/pages/camel-jbang-validation.adoc
#	dsl/camel-yaml-dsl/camel-yaml-dsl-validator/src/main/java/org/apache/camel/dsl/yaml/validator/CatalogBodyTypes.java
#	dsl/camel-yaml-dsl/camel-yaml-dsl-validator/src/test/java/org/apache/camel/dsl/yaml/validator/TextBodyFlowTest.java
…atalog-body-types-storage

# Conflicts:
#	dsl/camel-yaml-dsl/camel-yaml-dsl-validator/src/test/java/org/apache/camel/dsl/yaml/validator/TextBodyFlowTest.java
…atalog-body-types-messaging

# Conflicts:
#	dsl/camel-yaml-dsl/camel-yaml-dsl-validator/src/test/java/org/apache/camel/dsl/yaml/validator/TextBodyFlowTest.java
@davsclaus
davsclaus changed the base branch from feature/CAMEL-24961-catalog-body-types-messaging to main October 24, 2026 07:08
…atalog-body-types-core

# Conflicts:
#	dsl/camel-yaml-dsl/camel-yaml-dsl-validator/src/test/java/org/apache/camel/dsl/yaml/validator/TextBodyFlowTest.java
@davsclaus davsclaus added the enhancement New feature or request label Oct 24, 2026
@davsclaus
davsclaus merged commit f2b6dc8 into main Oct 24, 2026
2 of 7 checks passed
@davsclaus
davsclaus deleted the feature/CAMEL-24961-catalog-body-types-core branch October 24, 2026 07:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants