Antalya 26.6 Backport of 113505: Added support for inserting data to iceberg s3 tables(AWS)

[subkanthi]

backport of ClickHouse#113505

Changelog category (leave one):

• New Feature

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Add ability to insert data to s3 tables from DataLakeCatalog.

Documentation entry for user-facing changes

Add ability to insert data to s3 tables from DataLakeCatalog.

CI/CD Options

Exclude tests:

• Fast test
• Integration Tests
• Stateless tests
• Stateful tests
• Performance tests
• Aarch64 tests
• All with ASAN
• All with TSAN
• All with MSAN
• All with UBSAN
• All with Coverage
• All Regression
• Disable CI Cache

Regression jobs to run:

• Fast suites (mostly <1h)
• Aggregate Functions (2h)
• Alter (1.5h)
• Benchmark (30m)
• CAS (content-addressed storage; Antalya only)
• ClickHouse Keeper (1h)
• Iceberg (2h)
• LDAP (1h)
• OAuth (5m)
• Parquet (1.5h)
• RBAC (1.5h)
• SSL Server (1h)
• S3 (2h)
• S3 Export (2h)
• Swarms (30m)
• Tiered Storage (2h)

[github-actions]

Workflow [PR], commit [182da61]

[subkanthi]

continuation of ClickHouse#103220

[subkanthi]

@blau-ai

[blau-ai]

CI triage for #2251 (commit 3cea6fa)

Verdict: 0 failures attributable to this PR. Every red check is infra / flaky / pre-existing. Crucially, all the CI that actually exercises this PR's code is green — see the "why I'm confident" section below. Nothing here needs a code change; the reds just need re-runs (and the persistent ones are base-branch issues, not this PR).

Summary of red checks: PR-caused: 0 · flaky/infra: integration shards 1/8, 3/8, 4/8 + SQLLogic · pre-existing/unrelated: Regression CAS/settings/tiered_storage + Grype.

---

1. Integration tests (amd_asan_ubsan, db disk, old analyzer) — 1/8, 3/8, 4/8 → flaky infra

• 3/8 & 1/8 ("interrupted, exit status 2"): the entire test_s3_cluster/test.py module fails at fixture setup —
  E Exception: Timed out while waiting for instance 's0_0_1' ... to start. The node never came up; no test body ran.
• 4/8 ("Failures: 12/1146"): all 12 are test_storage_iceberg_with_spark/test_schema_inference.py::test_schema_inference[...], failing with
  Code: 210. DB::NetException: Connection refused (172.16.1.10:9000) — i.e. the ClickHouse node was not accepting connections at all.
• Why this is infra, not the PR:
  • The failing 4/8 tests include both s3-* and azure-* parametrizations, failing identically. This PR only touches the S3 code path — if it had broken iceberg schema inference, the azure-* variants would still pass. They don't, which means the node itself was down, independent of code path.
  • "Connection refused" / "instance timed out to start" are container/startup failures, not query errors — they happen before the changed code executes.
  • Shards 2/8, 5/8, 6/8, 7/8, 8/8 are fully green (0 failures). A real regression from a 71-line iceberg patch would not be confined to a subset of shards while the same test types pass elsewhere.
• Next step: re-run the three red integration shards. Expected to go green.

2. SQLLogic test → flaky (runner memory pressure), unrelated

• Report: OK: total tests 5,939,581 >= minimum; OK: failed tests 163,715 <= maximum 174,004; FAILED: 230 new failures detected.
• Every one of the "new failures" is random/select/slt_good_*.test hitting
  Code: 241 ... (total) memory limit exceeded ... current RSS: ~26 GiB, maximum: 25.74 GiB.
• That's the server hitting its total-memory limit on random-generated SELECTs — a resource/runner condition on this run, with no connection to iceberg/S3. This PR changes nothing about memory limits or random-select execution.
• Next step: re-run; if it persists it's a base-branch SQLLogic threshold/memory issue to track separately, not a Antalya 26.6 Backport of 113505: Added support for inserting data to iceberg s3 tables(AWS) #2251 blocker.

3. Regression release — CAS (aggregate_functions, alter, atomic_insert, engines, lightweight_delete, selects), Common/settings, TieredStorageCAS → pre-existing / unrelated domain

• These are the Antalya content-addressed-storage (CAS) and settings/tiered-storage testflows suites. Failure summaries show the flaky signature, e.g. CAS modules 35 retries (7 ok, 2 failed, 26 retried) and settings 1706 scenarios (1700 ok, 2 failed, 4 xfail) — a couple of residual failures after heavy retrying, spread across several unrelated suites.
• None of these domains touch iceberg/S3 inserts, which is all this PR changes. The base branch antalya-26.6 MasterCI is itself red on every recent run, consistent with these being pre-existing.
• Next step: compare against the base branch's own regression run; I expect the same modules red there. Not caused by Antalya 26.6 Backport of 113505: Added support for inserting data to iceberg s3 tables(AWS) #2251.

4. Grype Scan (clickhouse-server ...-alpine) → base-image CVE, not source

• Grype Scan Completed with 1 high/critical vulnerabilities. This scans the built container image's OS/dependency packages, not the C++ diff. It's independent of this PR and appears whenever the alpine base image carries an unpatched CVE.
• Next step: address at the image/base level (or waive); nothing to change in Antalya 26.6 Backport of 113505: Added support for inserting data to iceberg s3 tables(AWS) #2251.

---

Why I'm confident this PR is healthy

The suites that directly validate this change all passed:

PR-relevant suite	Result
Regression Iceberg (1) / Iceberg (2)	✅ pass
Regression S3 (aws_s3 ×2, azure ×2, gcs ×2, minio ×3)	✅ pass
Regression S3Export (part, partition)	✅ pass
Regression Parquet / ParquetS3 (aws_s3, minio)	✅ pass
Regression Swarms	✅ pass
Stateless tests, s3 storage (amd_debug, distributed plan)	✅ 0 failed
Fast test, all Builds (amd), Source upload	✅ pass

The diff is small and well-scoped (+71/−16): opt-in suppression of the deprecated Keep-Alive request header (only enabled on the S3 client path in PocoHTTPClient), S3TablesCatalog metadata/endpoint/credential normalization, and an iceberg_delete_data_on_drop-gated dropTable(delete_data). None of it affects server startup or the code paths behind any of the red checks.

Recommendation: re-run integration shards 1/8, 3/8, 4/8 and SQLLogic; treat Regression CAS/settings/tiered_storage and Grype as pre-existing base-branch/infra items. No fix to the PR is required for these failures. Happy to dig into any specific check further if you'd like.

_{— automated CI triage. I can't build/run ClickHouse in this container; this is based on the praktika S3 reports and CI metadata for commit 3cea6fa.}