fix(expr): reject non-string StartsWith operands instead of pruning the file - #3162
Open
jackylee-ch wants to merge 1 commit into
Open
Conversation
…he file The catch-all arm returned Ok(false), which this visitor reads as "this data file cannot match", so a StartsWith predicate on a non-string partition column silently dropped files that contain matching rows. Return the same error the other visitors already return for this input.
There was a problem hiding this comment.
🟢 Approval recommended
The change corrects a pruning bug with consistent error behavior and includes a focused regression test covering the reported failure mode.
Pull request overview
Fixes a correctness issue in partition pruning where StartsWith on a non-string partition column could silently evaluate to false, causing data files that do contain matching rows to be incorrectly pruned during scans.
Changes:
- Update
ExpressionEvaluatorVisitor::starts_withto return anErrorKind::Unexpectederror when either operand is non-string, instead of falling back toOk(false). - Add a regression test covering both
StartsWithandNotStartsWithagainst an identity-partitionedbinarycolumn.
File summaries
| File | Description |
|---|---|
| crates/iceberg/src/expr/visitors/expression_evaluator.rs | Rejects non-string StartsWith operands with an error and adds regression coverage to prevent silent mis-pruning. |
Review details
- Files reviewed: 1/1 changed files
- Comments generated: 0
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
None — filed directly.
What changes are included in this PR?
ExpressionEvaluator::starts_withmatched only(String, String)and fell through toOk(false). In this visitorfalsemeans "this data file cannot match", so aStartsWithpredicate on a non-string partition column silently pruned files that do contain matching rows.
It is reachable:
Reference::new("a").starts_with(Datum::binary(b"ab"))binds without error, andwith an identity-partitioned
binarycolumn a file whose partition value isX'616263'wasdropped from the scan.
Now it returns the same error the other four visitors already return for this input —
inclusive_metrics_evaluator.rs:308,row_group_metrics_evaluator.rs:347,page_index_evaluator.rs:586,manifest_evaluator.rs:331.Are these changes tested?
Yes —
test_expr_starts_with_non_string_errorscoversStartsWithandNotStartsWithon a binarypartition; it returns
falseon the parent commit.cargo test --release -p iceberg --lib expr::→ 287 passed.
AI Disclosure
Written with AI assistance (Claude Code); I reviewed the change and ran the tests above.
Worth a reviewer's attention: this turns
NOT STARTS WITHon a non-string column from "no pruning"into an error. Java rejects the predicate earlier, when binding, so validating in
BinaryExpression::bindwould be a reasonable follow-up.