Skip to content

[#13323] test(client): Add Ray Data Iceberg REST contract tests - #13325

Open
jiangxt2 wants to merge 3 commits into
apache:mainfrom
jiangxt2:test-ray-iceberg-contract
Open

jiangxt2 wants to merge 3 commits into
apache:mainfrom
jiangxt2:test-ray-iceberg-contract

Conversation

@jiangxt2

Copy link
Copy Markdown
Collaborator

What changes were proposed in this pull request?

This pull request adds a Gravitino-owned Ray Data Iceberg REST integration test to the Python client test suite.

The change:

  • adds pinned Ray 2.55.1, PyIceberg 0.12.0, and pandas 2.3.3 test dependencies;
  • adds a dedicated rayIcebergIT Gradle task with an isolated Python environment;
  • generates the ignored gravitino/version.ini file before running the client test;
  • configures the Gravitino Iceberg REST auxiliary service with the dynamic catalog provider;
  • uses Gravitino's memory Iceberg catalog and PyIceberg to create a namespace and pre-existing table fixture;
  • verifies empty-table reads, Ray Data writes, subsequent reads, field order, row counts, and filtering;
  • adds setup-failure cleanup for Gravitino configuration, metadata, and temporary data.

The implementation does not modify Java Iceberg REST code, Ray source code, or Ray Catalog.resolve() behavior.

Why are the changes needed?

Gravitino documents Ray Data access through the Iceberg REST endpoint and catalog_kwargs, but the Python client integration suite did not have a real end-to-end regression test for this contract.

The test protects the existing-table read/write path and validates the Gravitino auxiliary service lifecycle using the existing Lance Ray IT as the implementation baseline.

Fix: #13323

Does this PR introduce any user-facing change?

No end-user API or runtime behavior changes are introduced; contributor-facing Python integration-test tooling is added.

How was this patch tested?

Passed:

  • git diff --check
  • ruff check --no-cache clients/client-python/tests/integration/test_ray_iceberg.py
  • ruff format --check --no-cache clients/client-python/tests/integration/test_ray_iceberg.py
  • Gradle task graph for :clients:client-python:rayIcebergIT --dry-run
  • ./gradlew :clients:client-python:rayIcebergIT --no-daemon --console=plain — passed; 1 test ran and completed successfully.

Before running the E2E, build or reuse the Gravitino distribution with ./gradlew compileDistribution -PskipWeb=true -x test.

This E2E is currently opt-in/manual and is not wired into the default CI workflow.

Signed-off-by: jiangxt2 <jiangxt2@vip.qq.com>
Signed-off-by: jiangxt2 <jiangxt2@vip.qq.com>
@jiangxt2 jiangxt2 self-assigned this Sep 18, 2026
@github-actions

github-actions Bot commented Sep 18, 2026

Copy link
Copy Markdown

Code Coverage Report

Overall Project 70.57% -0.25% 🟢
Files changed 50.4% 🔴

Module Coverage
aliyun 19.74% 🔴
api 51.99% 🟢
authorization-common 85.96% 🟢
authorization-ranger 4.38% -1.22% 🔴
aws 53.54% 🟢
aws-bundle 0.0% 🔴
azure 32.1% 🔴
azure-bundle 0.0% 🔴
catalog-common 28.55% 🔴
catalog-fileset 82.28% 🟢
catalog-glue 71.52% 🟢
catalog-hive 83.08% 🟢
catalog-jdbc-common 45.72% 🟢
catalog-jdbc-doris 83.47% 🟢
catalog-jdbc-mysql 81.8% 🟢
catalog-jdbc-postgresql 83.89% 🟢
catalog-jdbc-starrocks 79.16% 🟢
catalog-kafka 75.21% 🟢
catalog-lakehouse-generic 63.27% +5.01% 🟢
catalog-lakehouse-hudi 79.1% 🟢
catalog-lakehouse-iceberg 85.98% 🟢
catalog-lakehouse-paimon 84.31% 🟢
catalog-model 77.99% 🟢
cli 44.62% 🟢
client-java 78.46% 🟢
common 58.2% 🟢
core 84.92% -0.11% 🟢
filesystem-hadoop3 76.48% 🟢
flink 18.07% 🔴
flink-common 53.95% 🟢
flink-runtime 0.0% 🔴
gcp 32.2% 🔴
hadoop-auth 68.0% 🟢
hadoop-common 17.84% 🔴
hive-metastore-common 54.41% 🟢
iceberg-aliyun-bundle 0.0% 🔴
iceberg-common 66.16% 🟢
iceberg-rest-server 77.1% 🟢
idp-basic 86.77% 🟢
integration-test-common 0.0% 🔴
jobs 59.96% 🟢
lance-common 36.14% 🔴
lance-rest-server 69.51% 🟢
lineage 59.39% 🟢
optimizer 83.33% 🟢
optimizer-api 42.56% 🟢
server 90.23% +0.04% 🟢
server-common 82.37% 🟢
spark 60.5% 🟢
tencent 81.78% 🟢
trino-connector 61.99% 🟢
Files
Module File Coverage
authorization-ranger RangerAuthorizationHadoopSQLPlugin.java 0.0% 🔴
catalog-lakehouse-generic LanceTableOperations.java 66.4% 🟢
core TableMetaService.java 100.0% 🟢
TableHookDispatcher.java 90.0% 🟢
TableColumnMetaService.java 75.0% 🟢
AuthorizationUtils.java 66.54% 🟢
server MetadataObjectRoleOperations.java 100.0% 🟢

Signed-off-by: jiangxt2 <jiangxt2@vip.qq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Improvement] Add Ray Data Iceberg REST contract tests

1 participant