Upgrade DebeziumIO to Debezium 3.5.2.Final

What needs to happen?

DebeziumIO currently pins Debezium 3.1.3.Final (bumped in #37571). Upstream has moved on considerably and the connector should be brought up to the current stable patch line, 3.5.2.Final.

What the upgrade absorbs:

Kafka Connect 4.x alignment. Debezium 3.5 builds against Kafka Connect 4.1.2, so the connect-api pin has to move from 3.9.0. Connect 4.1 also adds an abstract pluginMetrics() to SourceTaskContext, which BeamSourceTaskContext must implement.
debezium-core module split. Debezium 3.5 split debezium-core into smaller modules (debezium-config, debezium-connector-common, and friends). The Gradle dependencies and the expansion-service shadow jar need auditing so the split classes are still bundled.
Config and behaviour changes. The schema_only snapshot mode was removed in 3.3, and connection failures now surface wrapped in a RetriableException.
Postgres JDBC driver. The 3.5 Postgres connector requires driver 42.7.x, while Beam globally pins 42.6.2 for CVE reasons (24f562d).

Notes

3.6.0 is intentionally skipped for now: it shipped a rewritten MySQL DDL parser with several open regressions and there is no 3.6.1 yet. A follow-up bump to the 3.6 line should be trivial once those settle.

Issue Priority

Priority: 2 (default / most bugs should be filed as P2)

Issue Components

Component: Java SDK
Component: IO connector

@tkaymak

hi @tkaymak

I noticed a typo here. May I submit a pull request for the correction?

 * <p>See <a href="https://debezium.io/documentation/reference/3.5/connectors/index.html">Debezium

@ilyasahsan123

@ilyasahsan123 please go ahead

@tkaymak

Hi @tkaymak

We should enable the pos and row attributes within the metadata to support data duplication (MySQL) in the data warehouse.
Additionally, We also need source.ts_ms to identify the database commit time, which is useful for debugging.

Currently, Apache Beam supports only these following attributes, and we could not deduplicate the data correctly.

Apache Beam:

metadata={
  "connector": "mysql",
  "version": "3.5.2.Final",
  "name": "beam-debezium-connector",
  "database": "inventory",
  "schema": "mysql-bin.000003",
  "table": "products_on_hand"
}

However, here is the complete metadata generated by Debezium:

Debezium

"source": {
  "version": "3.5.2.Final",
  "connector": "mysql",
  "name": "tutorial",
  "ts_ms": 1791689147000,
  "snapshot": "false",
  "db": "inventory",
  "sequence": null,
  "ts_us": 1791689147000000,
  "ts_ns": 1791689147000000000,
  "table": "customers",
  "server_id": 223344,
  "gtid": null,
  "file": "mysql-bin.000003",
  "pos": 796,
  "row": 0,
  "thread": 13,
  "query": null
}

Is there any explanation about this?

Thanks

@tkaymak

hi @tkaymak @Abacn,

How are you? I hope you are doing well.

Do you have answer about my question above?
Please tell me if you need assistance to enable that; I am glad to contribute to this issue.

@tkaymak

hi @tkaymak

I am writing to share that I have published a blog post on DebeziumIO within the Debezium Blog page. The link is below:

Thanks

@ilyasahsan123

Hi @ilyasahsan123, sorry for the slow reply, and thanks a lot for the blog post and the example, really great to see!

You're right about the metadata. SourceRecordJson only kept six fields of the Debezium source block and dropped everything else, including pos, row and ts_ms. I opened #40701 for it, and #40702 adds the complete source block under metadata.source (additive, existing fields unchanged). That works for MySQL (file, pos, row, gtid, ts_ms) as well as the other connectors. Would be great if you could take a look and maybe try it with your pipeline.

Since the 3.5.2 upgrade itself landed with #39569 (shipped in 2.76.0), I'll close this issue and we can continue on #40701.

Completed by #39569; follow-up metadata work tracked in #40701.

Navigation Menu

Repository navigation

Upgrade DebeziumIO to Debezium 3.5.2.Final #39568

What needs to happen?

Notes

Issue Priority

Issue Components

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Upgrade DebeziumIO to Debezium 3.5.2.Final #39568

Description

What needs to happen?

Notes

Issue Priority

Issue Components

Activity

ilyasahsan123 commented on Sep 22, 2026

tkaymak commented on Oct 3, 2026

ilyasahsan123 commented on Oct 11, 2026

ilyasahsan123 commented on Oct 20, 2026

ilyasahsan123 commented on Oct 21, 2026

tkaymak commented on Oct 21, 2026

tkaymak commented on Oct 21, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions