Repository navigation
CAMEL-24961: Catalog body types for spring-redis, couchbase, fhir, solr, ssh, slack and telegram - #28309
Conversation
…s, and the body it leaves after a step A @BodyTypes annotation on the endpoint, data format or language class says the body per side (consumerOut, consumerReply, producerIn, producerOut, marshal, unmarshal and expression in and out): its carriers (null, text, bytes, stream, file, value, map, list, pojo, unchanged, any, unknown), the Java type when it is always the same, the format, the options it depends on, the body for other option values, and a description. The package plugin writes it as a bodyTypes section of the catalog JSON and checks that each side fits the kind and that the options it names exist. A subclass declares its own. The first slice: file, direct, seda, timer, cron, kamelet, http, https, rest-openapi, xslt, sql and the json data format (jackson), read from the code that sets the body and checked against the body types the message history records when routes run. camel_catalog_doc returns them. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
…tp, jms, kafka, aws2-sqs, the csv, jacksonXml and yaml data formats, and the languages The second slice of the catalog body types: - components: platform-http, rest (and the rest-openapi consumer), ftp, ftps, sftp, jms, activemq, activemq6, amqp, kafka, aws2-sqs, log, mock, scheduler, quartz; - data formats: csv, jacksonXml, snakeYaml; - languages: simple, constant, jsonpath, xpath, groovy, jq. Each value is read from the code that sets the body, and checked against the body types the message history recorded when the tests of the modules and reference routes ran (the ftp integration tests included). camel_catalog_doc gives the body types a line per side, and its answer is cut at 18000 characters instead of 16000, so the four sides of the jms components leave the options a limit asks for room. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
paho-mqtt5: the consumer gives the payload as a byte array, the producer sends what converts to one and leaves the body unchanged. mongodb: the consumer gives a Document (or null for a change without its full document with changeStreams); the producer takes a document or a query, and leaves a body by operation: a Document or null for a find of one, a List for findAll, findDistinct and aggregate, a Long for count, the result of the driver for the other writes. Read from the code that sets the body, and checked against a mosquitto and a mongodb container started with camel infra. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
…ts an option value without carriers
From the review of PR 28241:
- An iterator is read once and has no size and no index, so tooling that
reads list as a List accepts ${body.size()} or ${body[0]} on it, which fail
at runtime. The new iterator carrier names it: sql with outputType=StreamList
gives an iterator, not a list.
- A @BodyType.When with no carriers (carrier = {}) wrote "carriers": [] into
the catalog. The build now rejects it, as it does a side without carriers.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
…MEL-24961-catalog-body-types-2 Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
… camel_catalog_doc says unknown for no carriers From the review of PR 28242: - The csv data format with lazyLoad=true gives an iterator, with the iterator carrier of PR 28241, not a list. - camel_catalog_doc says unknown for a body without carriers, instead of failing. The build rejects it now, but a catalog read from elsewhere may have one. - Sqs2Endpoint imports org.apache.camel.spi.* already: only the import of the nested BodyType.Carrier stays. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
…types of the catalog instead of their own tables The YAML validator's body walks read what a component or a data format does with the body from the catalog, instead of from tables in their source: - the consumer a route starts with (consumerOut): the file as read, text, bytes or data, instead of the file consumer only; - the body after a to: (producerOut): unchanged for log and mock and now for a file write, the response of the http producer as bytes or a stream; - the producers that need text (producerIn), instead of file, ftp, ftps and sftp only; - the body after an unmarshal (unmarshalOut), instead of the json, jacksonXml, yaml and csv table and the options that make something else; - the consumers that reply with text (consumerReply), instead of platform-http; - the consumers that give no body, instead of timer, quartz, scheduler and cron. A side holds for a step only when the step decides every option it depends on: an option the component can also be configured with (the valueDeserializer of kafka) counts only when the endpoint sets it, and a placeholder value is not known. The catalog of an older Camel version has no body types, so the validator's own catalog stands in for it. After a step that fails at runtime the walk stops, so a file write that keeps the body is reported once. The rules about HTTP methods, REST bindings and EIPs stay in the validator; the EIPs get body types with CAMEL-25928. The same reports as before on 1204 human-written route files (the Camel repository, the examples, kamelets, camel-k and camel-kit) and on 1209 model-written ones (the r17 benchmark series), and the doc examples of the validator tests still validate. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
…docling, milvus, qdrant, ai-tool and ibm-watson-language The AI components declare the body they need and the body they leave, per operation where the operation decides it. The answer of openai and langchain4j-chat is text, also when it is asked for JSON, so the YAML validator now reports a field read on it that has no unmarshal before. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
… endpoint count for its body types The operation of openai is in the path (openai:embeddings), so the validator reads the path options with the syntax of the component, as the catalog parses an endpoint. A path the catalog cannot read, such as a toD with an expression, says nothing about its options. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
…pe for a value its option cannot have A byOption value must be one of the enum of its option, or true or false for a boolean, ignoring case as tooling compares it with the endpoint: a misspelled operation such as embedding for embeddings would never hold. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
The attribute holds an array, and the catalog JSON and BodyTypeModel call the same field carriers. The Javadoc of BodyTypeModel.isUnknown() says that a side with no carriers is unknown too. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
…MEL-24961-catalog-body-types-2
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
…AMEL-24961-catalog-body-types-3
…AMEL-24961-catalog-body-types-ai # Conflicts: # tooling/maven/camel-package-maven-plugin/src/main/java/org/apache/camel/maven/packaging/BodyTypesHelper.java
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
The body types of aws2-s3, azure-storage-blob, azure-files, dropbox, google-storage, ibm-cos and minio, read from the code and checked at runtime against LocalStack S3 (aws2-s3, and ibm-cos and minio through their S3 API) and Azurite (azure-storage-blob). The pages of aws2-s3, google-storage and minio said getObject returns an S3Object, a Blob or a MinioObject: it returns the content of the object. The validator calls a body the file as read (a GenericFile) only when the file is its only carrier, not for a blob that is a stream or a file. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
…e data format and language schemas Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
…-types' into feature/CAMEL-24961-catalog-body-types-2
…AMEL-24961-catalog-body-types-3
…AMEL-24961-catalog-body-types-ai
…AMEL-24961-catalog-body-types-storage
…n read as a list, as the carriers do Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
The body types of aws2-sns, aws2-kinesis, aws2-kinesis-firehose, google-pubsub, azure-servicebus, azure-eventhubs and azure-storage-queue, read from the code and checked at runtime against LocalStack (SNS and Kinesis), the Pub/Sub emulator and Azurite (queues). aws2-sqs sends bytes, a stream or a file as text as well, and sendBatchMessage takes a SendMessageBatchRequest too, which its producer side now says. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
…AMEL-24961-catalog-body-types-3
…ature/CAMEL-24961-catalog-body-types-messaging
…AMEL-24961-catalog-body-types-ai
…AMEL-24961-catalog-body-types-storage
The body types of bean, class, browse, controlbus, dataformat, dataset, dataset-test, dynamic-router, dynamic-router-control, event, exec, knative, language, once, ref, rest-api, saga, stream, stub, validator, wasm and xslt-saxon, and of the bean, header, exchangeProperty, variable, ref, file, tokenize and xtokenize languages, read from the code and checked at runtime with camel run (knative over a local environment file). The xpath and jq languages read the source option instead of the body when it is set, which their expressionIn now says, as tokenize and xtokenize do. The component docs of stream, exec, language, once, browse and dataset said things about the body that the code does not do. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
… and infinispan The body types of jira, github2, pqc, dns, caffeine-cache, caffeine-loadcache, infinispan and infinispan-embedded, the most used components in the route files that had none yet. pqc, dns, caffeine and infinispan-embedded were checked at runtime with camel run; jira and github2 were read from the code. The validator now takes an option a body depends on as unknown also when another option of the step names a body, such as xslt with output=bytes and a resultHandlerFactory, as the BodyType javadoc says. The component docs of jira, github2, dns and infinispan said things about the body that the code does not do. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
gnodet-bot
left a comment
There was a problem hiding this comment.
Both open items from the previous review are fully resolved:
Couchbase producerOut byOption coverage — the three operations are now correctly declared: CCB_PUT → Boolean (default), CCB_GET → GetResult POJO, CCB_DEL → MutationResult as text. The description is accurate and matches the byOption entries.
Spring-redis read-returning commands — the explanation is correct: with dependsOn = "command", a command absent from byOption gives no answer from CatalogBodyTypes.answer, and these commands (GET, HGET, LPOP, RPOP, SPOP, etc.) are correctly documented in the description as "not known". The validator test (lookup route) confirms the right behaviour: no warning is issued when the output is unknown.
The isSameValue normalisation in both CatalogBodyTypes.java and BodyTypesHelper.java is clean — dash-case, camelCase, and UPPER_SNAKE_CASE are all collapsed to the canonical form before comparison, consistent with what the catalog and the enum type converter accept. The two new tests (testAnOptionValueInAnotherStyleOfTheEnum, testEndpointWithBodyTypesForEnumValuesInOtherStyles) cover the round-trip correctly. The new test endpoint SomeEndpointWithEnumStyleBodyTypes exercises the BodyTypesHelper check path.
The remaining first-time additions (fhir, slack, solr, ssh, telegram) are consistent with the series and correctly structured.
This review was generated by an AI agent, Hermès on behalf of @gnodet.
…atalog-body-types-3 Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
The description of the CamelSmppEncoding header changed with the body types of smpp, and the endpoint DSL holds a copy of it. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Signed-off-by: Claus Ibsen <claus.ibsen@gmail.com>
…o feature/CAMEL-24961-catalog-body-types-most-used-2
…AMEL-24961-catalog-body-types-ai
…atalog-body-types-ai # Conflicts: # docs/user-manual/modules/ROOT/pages/camel-jbang-validation.adoc # dsl/camel-yaml-dsl/camel-yaml-dsl-validator/src/main/java/org/apache/camel/dsl/yaml/validator/CatalogBodyTypes.java # dsl/camel-yaml-dsl/camel-yaml-dsl-validator/src/test/java/org/apache/camel/dsl/yaml/validator/TextBodyFlowTest.java
…AMEL-24961-catalog-body-types-storage
…atalog-body-types-storage # Conflicts: # dsl/camel-yaml-dsl/camel-yaml-dsl-validator/src/test/java/org/apache/camel/dsl/yaml/validator/TextBodyFlowTest.java
…ature/CAMEL-24961-catalog-body-types-messaging
…atalog-body-types-messaging # Conflicts: # dsl/camel-yaml-dsl/camel-yaml-dsl-validator/src/test/java/org/apache/camel/dsl/yaml/validator/TextBodyFlowTest.java
…feature/CAMEL-24961-catalog-body-types-core
…atalog-body-types-core # Conflicts: # dsl/camel-yaml-dsl/camel-yaml-dsl-validator/src/test/java/org/apache/camel/dsl/yaml/validator/TextBodyFlowTest.java
CAMEL-24961-catalog-body-types-most-used
…atalog-body-types-most-used
…feature/CAMEL-24961-catalog-body-types-transformation
…atalog-body-types-transformation # Conflicts: # dsl/camel-yaml-dsl/camel-yaml-dsl-validator/src/test/java/org/apache/camel/dsl/yaml/validator/TextBodyFlowTest.java
…into feature/CAMEL-24961-catalog-body-types-dataformats-text
…atalog-body-types-dataformats-text # Conflicts: # dsl/camel-yaml-dsl/camel-yaml-dsl-validator/src/test/java/org/apache/camel/dsl/yaml/validator/TextBodyFlowTest.java
…' into feature/CAMEL-24961-catalog-body-types-dataformats-binary
…atalog-body-types-dataformats-binary # Conflicts: # dsl/camel-yaml-dsl/camel-yaml-dsl-validator/src/test/java/org/apache/camel/dsl/yaml/validator/TextBodyFlowTest.java
…ry' into feature/CAMEL-24961-catalog-body-types-messaging-2
…atalog-body-types-messaging-2
…o feature/CAMEL-24961-catalog-body-types-most-used-2
…atalog-body-types-most-used-2 # Conflicts: # dsl/camel-yaml-dsl/camel-yaml-dsl-validator/src/test/java/org/apache/camel/dsl/yaml/validator/TextBodyFlowTest.java
Stacked on #28306. This group covers seven more of the most used components of the catalog body types (CAMEL-24961): spring-redis, couchbase, fhir, solr, ssh, slack and telegram. fhir is the first API component with body types.
What the catalog says now
unknown: the message as the serializer reads itany: the arguments come from headersunchangedfor SET, the default, and the result of 39 other commands (a Boolean, a Long, text, a Map, a Set or a List); not known for the rest, such as GET and the other commands that read one stored objectGetResultof the SDK; the row as JSON text with fullDocument=falsetruefor CCB_PUT, theGetResultfor CCB_GET, and theMutationResultas text for CCB_DELany: only used with inBodyany; text or a List for DELETEmapstreamtext,bytes,streamorfile: the commandstreamcom.slack.api.model.MessageunchangedIncomingMessageor another update object; no body for other updatesMessageResult, orunchangedwith no bodyChecked at runtime. Routes run with
camel runlogged the class of the body after each step:camel infra run redis,solrandfhir;camel infra run couchbase, with the arm64 image of Couchbase 8.0.3;camel run.What the runs showed:
GetResult, or the row as JSON text with fullDocument=false.SimpleOrderedMap, which is a Map;MessageResult;Option values in any style of an enum constant (second commit). The catalog and the enum type converter take
convert-to-json,convertToJsonandCONVERT_TO_JSONas one value, but the build check of the@BodyTypesoption values and the YAML validator compared them only in any case. So a step withoperation=convert-to-jsondid not match theCONVERT_TO_JSONbody of docling, and the validator took the text of the default operation: a field read after it was reported. Both now compare the values as the catalog does. API components need this for API names such asload-page, which the catalog lists asLOAD_PAGE.Bugs found and filed, one per component:
Docs fixed:
Measured.
Tests
TextBodyFlowTest.testTheBodyOfMoreComponents:TextBodyFlowTest.testAnOptionValueInAnotherStyleOfTheEnum: docling withoperation=convert-to-jsonandoperation=convertToJsongives the document, withoperation=convert-to-markdowntext. Without the fix the first two are reported as well.EndpointSchemaGeneratorMojoTest.testEndpointWithBodyTypesForEnumValuesInOtherStyles: the build takesconvert-to-jsonandchunkHybridfor an option with the valuesCONVERT_TO_JSONandCHUNK_HYBRID.Claude Code on behalf of davsclaus
🤖 Generated with Claude Code