Skip to content

[DO NOT REVIEW] [Part 1/2] Add Oracle schema discovery, config, and type mapping in cdc-data-generator - #4500

Open
bharadwaj-aditya wants to merge 7 commits into
GoogleCloudPlatform:mainfrom
bharadwaj-aditya:oracle_cdc_data_generator
Open

bharadwaj-aditya wants to merge 7 commits into
GoogleCloudPlatform:mainfrom
bharadwaj-aditya:oracle_cdc_data_generator

Conversation

@bharadwaj-aditya

@bharadwaj-aditya bharadwaj-aditya commented Oct 28, 2026 •

Copy link
Copy Markdown
Contributor

Part 1 of 2 for adding Oracle support in v2/cdc-data-generator:\n- Enhances OracleInformationSchemaScanner in v2/spanner-common to discover primary keys, unique/non-unique indexes, foreign keys, generated/identity columns, and column bounds.\n- Adds OracleSinkConfig, OracleTypeMapper, and OracleSchemaFetcher (with unit tests) in v2/cdc-data-generator.

@bharadwaj-aditya
bharadwaj-aditya requested a review from a team as a code owner October 28, 2026 10:04
@bharadwaj-aditya bharadwaj-aditya added the addition New feature or request label Oct 28, 2026
@bharadwaj-aditya bharadwaj-aditya changed the title added implementation of oracle in the cdc data generator [DO NOT REVIEW] added implementation of oracle in the cdc data generator Oct 28, 2026
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces full support for Oracle databases as a sink within the CDC data generator. The changes include the implementation of core connector components such as schema fetching, data writing, and type mapping, alongside necessary updates to the factory and configuration options. Comprehensive testing ensures that the new Oracle integration handles data types and batch operations correctly.

Highlights

  • New Oracle Connector: Implemented the OracleSourceConnector to enable Oracle databases as a supported sink in the CDC data generator.
  • Schema and Data Handling: Added OracleSchemaFetcher for metadata discovery and OracleDataWriter for efficient batch upserts using MERGE statements.
  • Configuration and Dependencies: Updated pom.xml with Oracle JDBC drivers and introduced OracleSinkConfig to manage sharded Oracle instances.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds Oracle database support to the CDC Data Generator, introducing Oracle-specific implementations for data writing, schema fetching, type mapping, and source connection, along with comprehensive unit tests. Feedback on these changes highlights critical issues in the shared OracleInformationSchemaScanner utility: first, arbitrary capping of column size and precision limits can corrupt schema metadata for other pipelines utilizing this shared module; second, restricting index scanning to unique indexes by passing true to getIndexInfo may omit non-unique indexes required by other templates. It is recommended to map actual uncapped metadata and scan all indexes to ensure compatibility across all shared components.

@codecov

codecov Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 82.50729% with 60 lines in your changes missing coverage. Please review.
✅ Project coverage is 57.02%. Comparing base (a2d54f2) to head (0ac299c).
⚠️ Report is 6 commits behind head on main.

Files with missing lines Patch % Lines
...teleport/v2/source/oracle/OracleSchemaFetcher.java 76.36% 18 Missing and 21 partials ⚠️
...nner/sourceddl/OracleInformationSchemaScanner.java 86.89% 3 Missing and 16 partials ⚠️
...ud/teleport/v2/source/oracle/OracleSinkConfig.java 86.66% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main    #4500      +/-   ##
============================================
+ Coverage     56.87%   57.02%   +0.15%     
+ Complexity     8088     7676     -412     
============================================
  Files          1164     1167       +3     
  Lines         74114    74429     +315     
  Branches       8735     8803      +68     
============================================
+ Hits          42150    42443     +293     
+ Misses        29164    29146      -18     
- Partials       2800     2840      +40     
Components Coverage Δ
spanner-templates 83.97% <86.89%> (+0.27%) ⬆️
spanner-import-export 72.20% <ø> (-0.15%) ⬇️
spanner-live-forward-migration 87.62% <86.89%> (+0.70%) ⬆️
spanner-live-reverse-replication 79.82% <86.89%> (+0.62%) ⬆️
spanner-bulk-migration 88.25% <86.89%> (+0.37%) ⬆️
gcs-spanner-dv 86.52% <86.89%> (+0.82%) ⬆️
Files with missing lines Coverage Δ
...ud/teleport/v2/source/oracle/OracleTypeMapper.java 100.00% <100.00%> (ø)
...e/cloud/teleport/v2/templates/utils/Constants.java 0.00% <ø> (ø)
...ud/teleport/v2/source/oracle/OracleSinkConfig.java 86.66% <86.66%> (ø)
...nner/sourceddl/OracleInformationSchemaScanner.java 88.82% <86.89%> (+88.82%) ⬆️
...teleport/v2/source/oracle/OracleSchemaFetcher.java 76.36% <76.36%> (ø)

... and 5 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@bharadwaj-aditya bharadwaj-aditya left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Summary

PR #4500 adds Oracle sink support (SinkType.ORACLE) to v2/cdc-data-generator and extends OracleInformationSchemaScanner in v2/spanner-common to scan column bounds, indexes, and foreign keys. All 71 unit tests across v2/spanner-common and v2/cdc-data-generator compile and pass.

See inline comments for detailed findings.

@bharadwaj-aditya
bharadwaj-aditya requested review from dhwanilpatel and shreyakhajanchi and removed request for Vaibhavch9876 and aasthabharill October 29, 2026 06:20
@bharadwaj-aditya bharadwaj-aditya changed the title [DO NOT REVIEW] added implementation of oracle in the cdc data generator added implementation of oracle in the cdc data generator Oct 29, 2026
@bharadwaj-aditya bharadwaj-aditya changed the title added implementation of oracle in the cdc data generator [DO NOT REVIEW] [Part 1/2] Add Oracle schema discovery, config, and type mapping in cdc-data-generator Oct 30, 2026
…SchemaScanner and use Constants.DEFAULT_STRING_LENGTH
…1000 with LOBs, and synthesize negative-scale numerics
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

addition New feature or request size/XXL

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants