Repository navigation
[DO NOT REVIEW] [Part 1/2] Add Oracle schema discovery, config, and type mapping in cdc-data-generator - #4500
Conversation
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request introduces full support for Oracle databases as a sink within the CDC data generator. The changes include the implementation of core connector components such as schema fetching, data writing, and type mapping, alongside necessary updates to the factory and configuration options. Comprehensive testing ensures that the new Oracle integration handles data types and batch operations correctly. Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
There was a problem hiding this comment.
Code Review
This pull request adds Oracle database support to the CDC Data Generator, introducing Oracle-specific implementations for data writing, schema fetching, type mapping, and source connection, along with comprehensive unit tests. Feedback on these changes highlights critical issues in the shared OracleInformationSchemaScanner utility: first, arbitrary capping of column size and precision limits can corrupt schema metadata for other pipelines utilizing this shared module; second, restricting index scanning to unique indexes by passing true to getIndexInfo may omit non-unique indexes required by other templates. It is recommended to map actual uncapped metadata and scan all indexes to ensure compatibility across all shared components.
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #4500 +/- ##
============================================
+ Coverage 56.87% 57.02% +0.15%
+ Complexity 8088 7676 -412
============================================
Files 1164 1167 +3
Lines 74114 74429 +315
Branches 8735 8803 +68
============================================
+ Hits 42150 42443 +293
+ Misses 29164 29146 -18
- Partials 2800 2840 +40
🚀 New features to boost your workflow:
|
bharadwaj-aditya
left a comment
There was a problem hiding this comment.
Review Summary
PR #4500 adds Oracle sink support (SinkType.ORACLE) to v2/cdc-data-generator and extends OracleInformationSchemaScanner in v2/spanner-common to scan column bounds, indexes, and foreign keys. All 71 unit tests across v2/spanner-common and v2/cdc-data-generator compile and pass.
See inline comments for detailed findings.
…SchemaScanner and use Constants.DEFAULT_STRING_LENGTH
…1000 with LOBs, and synthesize negative-scale numerics
Part 1 of 2 for adding Oracle support in
v2/cdc-data-generator:\n- EnhancesOracleInformationSchemaScannerinv2/spanner-commonto discover primary keys, unique/non-unique indexes, foreign keys, generated/identity columns, and column bounds.\n- AddsOracleSinkConfig,OracleTypeMapper, andOracleSchemaFetcher(with unit tests) inv2/cdc-data-generator.