Skip to content

CAMEL-24961: Catalog body types for Bedrock, Polly, Translate and eight more AWS components - #28340

Merged
davsclaus merged 80 commits into
mainfrom
feature/CAMEL-24961-catalog-body-types-aws-3
Oct 24, 2026
Merged

davsclaus merged 80 commits into
mainfrom
feature/CAMEL-24961-catalog-body-types-aws-3

Conversation

@davsclaus

Copy link
Copy Markdown
Contributor

Stacked on #28327. This group covers eleven more AWS components of the catalog body types (CAMEL-24961): aws-bedrock, aws-bedrock-agent, aws-bedrock-agent-runtime, aws2-polly, aws2-translate, aws2-comprehend, aws2-textract, aws2-transcribe, aws2-rekognition, aws-cloudtrail and aws2-timestream.

What the catalog says now

Component Consumer Producer input Producer output
aws-bedrock the request of the model as JSON text for the invoke operations; any for converse, converseStream and applyGuardrail, whose input is in headers invokeTextModel: the JSON response as text for Titan, a Jackson JsonNode for the other models; invokeImageModel: a JsonNode array of the images; invokeEmbeddingsModel: the InvokeModelResponse; converse and applyGuardrail: text; the streaming operations and converseStream: text, or a List of the chunks
aws-bedrock-agent the IngestionJob, one per poll any: the ids are options or headers the job id as text for startIngestionJob, a List for listIngestionJobs, the IngestionJob for getIngestionJob
aws-bedrock-agent-runtime text, bytes, a stream or a file: the question or the input; a String, a FlowInput or a List of them for invokeFlow the answer as text for retrieveAndGenerate, a List of results for retrieve, text or a List of the chunks for invokeAgent and invokeInlineAgent, the document of the flow output for invokeFlow
aws2-polly any; text, bytes, a stream or a file for synthesizeSpeech, the default, and startSpeechSynthesisTask the audio as a stream for synthesizeSpeech; a List for describeVoices, listLexicons and listSpeechSynthesisTasks; the response of the SDK for the others
aws2-translate text, bytes, a stream or a file the translated text
aws2-comprehend text, bytes, a stream or a file, for every operation a List of what is found; the response of the SDK for detectSentiment and classifyDocument
aws2-textract any: the document as bytes, a stream or a file, unless options or headers name an S3 object the response of the SDK
aws2-transcribe any: the audio is in S3, and the inputs are headers the response of the SDK; unchanged for deleteTranscriptionJob and deleteVocabulary
aws2-rekognition any: the image is an Image of the SDK in a header the response of the SDK
aws-cloudtrail the event as JSON bytes, one per event
aws2-timestream any: the records and the query are headers the response of the SDK

Checked at runtime. Routes run with camel run logged the class of the body after each step, with text, bytes, streams, files, Maps, POJOs and no body. The AWS emulator of camel infra run aws answers Comprehend, Textract, Rekognition, Transcribe, CloudTrail and the invoke and converse calls of Bedrock. Polly, Timestream, Bedrock Agent, Bedrock Agent Runtime and the streaming of Bedrock ran against a small fake of the AWS API that logs what Camel sends. Translate ran against the fake too: the AWS SDK 2.56 talks to it in Smithy RPC v2 CBOR, which the emulator does not know. CloudTrail ran with a canned CloudTrailClient bean, as the emulator records no events.

  • aws-bedrock:
    • the invoke operations send the body as its toString(): bytes go as [B@..., a stream as the name of its class, a file as its path, a Map as {inputText=Hello} and no body as null;
    • invokeTextModel gives a String for Titan, an ObjectNode for Claude 3 and Mistral, and a TextNode for Llama; invokeImageModel gives an ArrayNode, and invokeEmbeddingsModel the InvokeModelResponse;
    • streaming gives the text, or an ArrayList of the chunks with streamOutputMode=chunks or its header;
    • converse and applyGuardrail fail without their header, and a Map body is not used.
  • aws-bedrock-agent: the body is not read; startIngestionJob gives the job id as text; the consumer sends the same IngestionJob on every poll.
  • aws-bedrock-agent-runtime: retrieveAndGenerate, retrieve and invokeAgent read text, bytes, a stream and a file as text, and a Map as its toString(); invokeFlow takes a String, and a Map or bytes fail.
  • aws2-polly: synthesizeSpeech reads the text as above and gives the audio as a stream, cached at the next step; putLexicon takes the lexicon from the body when no option or header gives it.
  • aws2-translate and aws2-comprehend: text, bytes, a stream and a file are sent as text, and a Map or a POJO as its toString().
  • aws2-textract: bytes, a stream and a file are the document, and a Map fails; with the S3 headers or options the body is not read.
  • aws2-rekognition: an image body is not used ("Image is a required field"); the CamelAwsRekognitionImage header with an Image works, and with bytes it does not.
  • aws2-transcribe and aws2-timestream: the body is not read, a List of records in the body included; the two transcribe delete operations keep the body.
  • aws-cloudtrail: one message per event, the JSON record as a byte array; the CloudEvent data type keeps the bytes.

Choices

  • The invoke operations of aws-bedrock take text only: bytes, a stream and a file are sent as their toString(), so the description says to convert them to a String first.
  • Where a header can stand in for the body, the input side is any: the S3 object of aws2-textract, for example.
  • pojoRequest is named only where the input side is any, as in the previous AWS group: every AWS option is a component option, so naming it would turn off the check of a text body for a step that leaves it out.
  • Where most operations give the same body, the output side says that body, and an option value covers each operation that differs. This keeps the camel_catalog_doc answers under the cap of 18 000 characters. aws-bedrock now lists 12 of its options in full instead of 18, as its options carry long enums.
  • aws2-timestream: the annotation is on Timestream2AbstractEndpoint, the class with @UriEndpoint, and covers both client types.

Bugs found and filed, one per component:

  • CAMEL-26069 (Major), aws-bedrock: the invoke operations send the body as its toString(); the image and embeddings streaming operations give an empty text; pojoRequest=true without the modelId option fails with a NullPointerException after the call; streaming does not resolve inference profile ids (CAMEL-24224 fixed invokeTextModel only); amazon.titan-embed-text-v2:0 is accepted by invokeTextModel; converseStream never sets the GuardrailTrace header; no operation fails with a NullPointerException.
  • CAMEL-26070, aws-bedrock-agent: the catalog lists the 26 headers of aws-bedrock-agent-runtime and none of its own (wrong headersClass); the page shows the URI of aws-bedrock-agent-runtime; no operation fails with a NullPointerException.
  • CAMEL-26071, aws-bedrock-agent-runtime: invokeFlow does not convert bytes, a stream or a file; invokeAgent picked by the operation header on an endpoint configured for retrieve fails, as the async client is missing; modelId and knowledgeBaseId are required for every operation; no operation fails with a NullPointerException.
  • CAMEL-26072 (Major), aws2-polly: the CamelAwsPollyTextType and CamelAwsPollyOutputFormat headers are never read since 4.18.0, so SSML sent with the header is spoken as text; the list operations drop the next token.
  • CAMEL-26073, aws2-translate: one language as an option and the other as a header sends no language for the header; Croatian is cr, which is Cree (hr).
  • CAMEL-26074, aws2-textract: the FeatureTypes header as a List of Strings fails with a ClassCastException; the start operations give the job id only in the response.
  • CAMEL-26075, aws2-transcribe: no operation fails with a NullPointerException; createVocabulary sends a List of phrases as one phrase "[a, b]"; two of the six delete operations keep the body.
  • CAMEL-26076, aws2-rekognition: no operation fails with a NullPointerException; the DetectionAttributes header as Strings fails with a ClassCastException; the operation option is described with the Lambda operations.
  • CAMEL-26077, aws2-timestream: describeScheduledQuery never sends the ARN of its header; the CamelAwsTimestreamRecord header is sent as the common attributes, not as a record; no operation fails with a NullPointerException.
  • CAMEL-26078, aws-cloudtrail: the CloudEvent data type sets text/plain for a JSON body; the page says a stream must be created first.
  • CAMEL-26079, camel-aws2: 14 trace logs of an AWS error can still throw a NullPointerException (polly, aws-config, textract, rekognition, sns), missed by CAMEL-24262; aws2-s3 reads the status code of the error details twice without a check.

Docs fixed

  • aws-bedrock: the chunks mode gives one message with a List of the chunks, not one message per chunk: the option description and the page. The component and endpoint DSL carry the option description.
  • aws2-comprehend: the feedback example keeps the text in a variable for detectKeyPhrases; it analyzed the toString() of the sentiment response.
  • aws2-timestream: the POJO example builds a CreateDatabaseRequest that compiles, and names the request it builds.

Measured.

Tests

  • TextBodyFlowTest.testTheBodyOfAwsAiServices:
    • reported: a Map sent to aws2-translate, to aws2-comprehend, to synthesizeSpeech, to invokeTextModel and to retrieveAndGenerate; a field read after aws2-translate and after the aws-cloudtrail consumer;
    • not reported: a Map sent to converse, aws2-rekognition, aws2-textract and aws2-timestream, and to aws2-polly without an operation, which the component may set.
  • These pass:
    • the 294 tests of camel-yaml-dsl-validator;
    • the 265 validator and catalog doc tests of camel-jbang-core;
    • the builds of camel-yaml-dsl-maven-plugin and camel-yaml-dsl-deserializers.
  • The answers of camel_catalog_doc stay under 18 000 characters: aws2-rekognition, the largest, is 17 730.

Claude Code on behalf of davsclaus

🤖 Generated with Claude Code

davsclaus and others added 30 commits October 21, 2026 13:58
…s, and the body it leaves after a step

A @BodyTypes annotation on the endpoint, data format or language class says the
body per side (consumerOut, consumerReply, producerIn, producerOut, marshal,
unmarshal and expression in and out): its carriers (null, text, bytes, stream,
file, value, map, list, pojo, unchanged, any, unknown), the Java type when it is
always the same, the format, the options it depends on, the body for other
option values, and a description. The package plugin writes it as a bodyTypes
section of the catalog JSON and checks that each side fits the kind and that
the options it names exist. A subclass declares its own.

The first slice: file, direct, seda, timer, cron, kamelet, http, https,
rest-openapi, xslt, sql and the json data format (jackson), read from the code
that sets the body and checked against the body types the message history
records when routes run. camel_catalog_doc returns them.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
…tp, jms, kafka, aws2-sqs, the csv, jacksonXml and yaml data formats, and the languages

The second slice of the catalog body types:
- components: platform-http, rest (and the rest-openapi consumer), ftp, ftps,
  sftp, jms, activemq, activemq6, amqp, kafka, aws2-sqs, log, mock, scheduler,
  quartz;
- data formats: csv, jacksonXml, snakeYaml;
- languages: simple, constant, jsonpath, xpath, groovy, jq.

Each value is read from the code that sets the body, and checked against the
body types the message history recorded when the tests of the modules and
reference routes ran (the ftp integration tests included).

camel_catalog_doc gives the body types a line per side, and its answer is cut
at 18000 characters instead of 16000, so the four sides of the jms components
leave the options a limit asks for room.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
paho-mqtt5: the consumer gives the payload as a byte array, the producer sends
what converts to one and leaves the body unchanged.

mongodb: the consumer gives a Document (or null for a change without its full
document with changeStreams); the producer takes a document or a query, and
leaves a body by operation: a Document or null for a find of one, a List for
findAll, findDistinct and aggregate, a Long for count, the result of the driver
for the other writes.

Read from the code that sets the body, and checked against a mosquitto and a
mongodb container started with camel infra.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
…ts an option value without carriers

From the review of PR 28241:

- An iterator is read once and has no size and no index, so tooling that
  reads list as a List accepts ${body.size()} or ${body[0]} on it, which fail
  at runtime. The new iterator carrier names it: sql with outputType=StreamList
  gives an iterator, not a list.
- A @BodyType.When with no carriers (carrier = {}) wrote "carriers": [] into
  the catalog. The build now rejects it, as it does a side without carriers.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
…MEL-24961-catalog-body-types-2

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
… camel_catalog_doc says unknown for no carriers

From the review of PR 28242:

- The csv data format with lazyLoad=true gives an iterator, with the iterator
  carrier of PR 28241, not a list.
- camel_catalog_doc says unknown for a body without carriers, instead of
  failing. The build rejects it now, but a catalog read from elsewhere may
  have one.
- Sqs2Endpoint imports org.apache.camel.spi.* already: only the import of the
  nested BodyType.Carrier stays.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
…types of the catalog instead of their own tables

The YAML validator's body walks read what a component or a data format does
with the body from the catalog, instead of from tables in their source:

- the consumer a route starts with (consumerOut): the file as read, text,
  bytes or data, instead of the file consumer only;
- the body after a to: (producerOut): unchanged for log and mock and now for a
  file write, the response of the http producer as bytes or a stream;
- the producers that need text (producerIn), instead of file, ftp, ftps and
  sftp only;
- the body after an unmarshal (unmarshalOut), instead of the json, jacksonXml,
  yaml and csv table and the options that make something else;
- the consumers that reply with text (consumerReply), instead of platform-http;
- the consumers that give no body, instead of timer, quartz, scheduler and cron.

A side holds for a step only when the step decides every option it depends
on: an option the component can also be configured with (the valueDeserializer
of kafka) counts only when the endpoint sets it, and a placeholder value is not
known. The catalog of an older Camel version has no body types, so the
validator's own catalog stands in for it. After a step that fails at runtime
the walk stops, so a file write that keeps the body is reported once.

The rules about HTTP methods, REST bindings and EIPs stay in the validator; the
EIPs get body types with CAMEL-25928.

The same reports as before on 1204 human-written route files (the Camel
repository, the examples, kamelets, camel-k and camel-kit) and on 1209
model-written ones (the r17 benchmark series), and the doc examples of the
validator tests still validate.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
…docling, milvus, qdrant, ai-tool and ibm-watson-language

The AI components declare the body they need and the body they leave, per
operation where the operation decides it. The answer of openai and
langchain4j-chat is text, also when it is asked for JSON, so the YAML
validator now reports a field read on it that has no unmarshal before.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
… endpoint count for its body types

The operation of openai is in the path (openai:embeddings), so the validator
reads the path options with the syntax of the component, as the catalog
parses an endpoint. A path the catalog cannot read, such as a toD with an
expression, says nothing about its options.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
…pe for a value its option cannot have

A byOption value must be one of the enum of its option, or true or false for
a boolean, ignoring case as tooling compares it with the endpoint: a
misspelled operation such as embedding for embeddings would never hold.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
The attribute holds an array, and the catalog JSON and BodyTypeModel call
the same field carriers. The Javadoc of BodyTypeModel.isUnknown() says that
a side with no carriers is unknown too.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
…AMEL-24961-catalog-body-types-ai

# Conflicts:
#	tooling/maven/camel-package-maven-plugin/src/main/java/org/apache/camel/maven/packaging/BodyTypesHelper.java
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
The body types of aws2-s3, azure-storage-blob, azure-files, dropbox,
google-storage, ibm-cos and minio, read from the code and checked at
runtime against LocalStack S3 (aws2-s3, and ibm-cos and minio through
their S3 API) and Azurite (azure-storage-blob).

The pages of aws2-s3, google-storage and minio said getObject returns an
S3Object, a Blob or a MinioObject: it returns the content of the object.

The validator calls a body the file as read (a GenericFile) only when the
file is its only carrier, not for a blob that is a stream or a file.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
…e data format and language schemas

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
…n read as a list, as the carriers do

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
The body types of aws2-sns, aws2-kinesis, aws2-kinesis-firehose,
google-pubsub, azure-servicebus, azure-eventhubs and azure-storage-queue,
read from the code and checked at runtime against LocalStack (SNS and
Kinesis), the Pub/Sub emulator and Azurite (queues).

aws2-sqs sends bytes, a stream or a file as text as well, and
sendBatchMessage takes a SendMessageBatchRequest too, which its producer
side now says.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
The body types of bean, class, browse, controlbus, dataformat, dataset,
dataset-test, dynamic-router, dynamic-router-control, event, exec,
knative, language, once, ref, rest-api, saga, stream, stub, validator,
wasm and xslt-saxon, and of the bean, header, exchangeProperty, variable,
ref, file, tokenize and xtokenize languages, read from the code and
checked at runtime with camel run (knative over a local environment
file).

The xpath and jq languages read the source option instead of the body
when it is set, which their expressionIn now says, as tokenize and
xtokenize do.

The component docs of stream, exec, language, once, browse and dataset
said things about the body that the code does not do.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
… and infinispan

The body types of jira, github2, pqc, dns, caffeine-cache,
caffeine-loadcache, infinispan and infinispan-embedded, the most used
components in the route files that had none yet. pqc, dns, caffeine and
infinispan-embedded were checked at runtime with camel run; jira and
github2 were read from the code.

The validator now takes an option a body depends on as unknown also when
another option of the step names a body, such as xslt with output=bytes
and a resultHandlerFactory, as the BodyType javadoc says.

The component docs of jira, github2, dns and infinispan said things about
the body that the code does not do.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
…atalog-body-types-ai

# Conflicts:
#	docs/user-manual/modules/ROOT/pages/camel-jbang-validation.adoc
#	dsl/camel-yaml-dsl/camel-yaml-dsl-validator/src/main/java/org/apache/camel/dsl/yaml/validator/CatalogBodyTypes.java
#	dsl/camel-yaml-dsl/camel-yaml-dsl-validator/src/test/java/org/apache/camel/dsl/yaml/validator/TextBodyFlowTest.java
…atalog-body-types-storage

# Conflicts:
#	dsl/camel-yaml-dsl/camel-yaml-dsl-validator/src/test/java/org/apache/camel/dsl/yaml/validator/TextBodyFlowTest.java
…atalog-body-types-messaging

# Conflicts:
#	dsl/camel-yaml-dsl/camel-yaml-dsl-validator/src/test/java/org/apache/camel/dsl/yaml/validator/TextBodyFlowTest.java
…atalog-body-types-core

# Conflicts:
#	dsl/camel-yaml-dsl/camel-yaml-dsl-validator/src/test/java/org/apache/camel/dsl/yaml/validator/TextBodyFlowTest.java
…atalog-body-types-transformation

# Conflicts:
#	dsl/camel-yaml-dsl/camel-yaml-dsl-validator/src/test/java/org/apache/camel/dsl/yaml/validator/TextBodyFlowTest.java
…atalog-body-types-dataformats-text

# Conflicts:
#	dsl/camel-yaml-dsl/camel-yaml-dsl-validator/src/test/java/org/apache/camel/dsl/yaml/validator/TextBodyFlowTest.java
…' into feature/CAMEL-24961-catalog-body-types-dataformats-binary
…atalog-body-types-dataformats-binary

# Conflicts:
#	dsl/camel-yaml-dsl/camel-yaml-dsl-validator/src/test/java/org/apache/camel/dsl/yaml/validator/TextBodyFlowTest.java
…atalog-body-types-most-used-2

# Conflicts:
#	dsl/camel-yaml-dsl/camel-yaml-dsl-validator/src/test/java/org/apache/camel/dsl/yaml/validator/TextBodyFlowTest.java
…atalog-body-types-aws-2

# Conflicts:
#	dsl/camel-yaml-dsl/camel-yaml-dsl-validator/src/test/java/org/apache/camel/dsl/yaml/validator/TextBodyFlowTest.java
…atalog-body-types-most-used-3

# Conflicts:
#	dsl/camel-yaml-dsl/camel-yaml-dsl-validator/src/test/java/org/apache/camel/dsl/yaml/validator/TextBodyFlowTest.java
@davsclaus
davsclaus changed the base branch from feature/CAMEL-24961-catalog-body-types-most-used-3 to main October 24, 2026 08:11
…atalog-body-types-aws-3

# Conflicts:
#	dsl/camel-yaml-dsl/camel-yaml-dsl-validator/src/test/java/org/apache/camel/dsl/yaml/validator/TextBodyFlowTest.java
@davsclaus davsclaus added the enhancement New feature or request label Oct 24, 2026
@davsclaus
davsclaus merged commit 0df5802 into main Oct 24, 2026
2 of 7 checks passed
@davsclaus
davsclaus deleted the feature/CAMEL-24961-catalog-body-types-aws-3 branch October 24, 2026 08:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants