Skip to content

fix(expr): reject non-string StartsWith operands instead of pruning the file - #3162

Open
jackylee-ch wants to merge 1 commit into
apache:mainfrom
jackylee-ch:fix-expression-evaluator-starts-with-non-string
Open

fix(expr): reject non-string StartsWith operands instead of pruning the file#3162
jackylee-ch wants to merge 1 commit into
apache:mainfrom
jackylee-ch:fix-expression-evaluator-starts-with-non-string

Conversation

@jackylee-ch

Copy link
Copy Markdown

Which issue does this PR close?

None — filed directly.

What changes are included in this PR?

ExpressionEvaluator::starts_with matched only (String, String) and fell through to
Ok(false). In this visitor false means "this data file cannot match", so a StartsWith
predicate on a non-string partition column silently pruned files that do contain matching rows.

It is reachable: Reference::new("a").starts_with(Datum::binary(b"ab")) binds without error, and
with an identity-partitioned binary column a file whose partition value is X'616263' was
dropped from the scan.

Now it returns the same error the other four visitors already return for this input —
inclusive_metrics_evaluator.rs:308, row_group_metrics_evaluator.rs:347,
page_index_evaluator.rs:586, manifest_evaluator.rs:331.

Are these changes tested?

Yes — test_expr_starts_with_non_string_errors covers StartsWith and NotStartsWith on a binary
partition; it returns false on the parent commit. cargo test --release -p iceberg --lib expr::
→ 287 passed.

AI Disclosure

Written with AI assistance (Claude Code); I reviewed the change and ran the tests above.
Worth a reviewer's attention: this turns NOT STARTS WITH on a non-string column from "no pruning"
into an error. Java rejects the predicate earlier, when binding, so validating in
BinaryExpression::bind would be a reasonable follow-up.

…he file

The catch-all arm returned Ok(false), which this visitor reads as "this data
file cannot match", so a StartsWith predicate on a non-string partition column
silently dropped files that contain matching rows. Return the same error the
other visitors already return for this input.
Copilot AI lite review requested due to automatic review settings September 7, 2026 08:01

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The change corrects a pruning bug with consistent error behavior and includes a focused regression test covering the reported failure mode.

Pull request overview

Fixes a correctness issue in partition pruning where StartsWith on a non-string partition column could silently evaluate to false, causing data files that do contain matching rows to be incorrectly pruned during scans.

Changes:

  • Update ExpressionEvaluatorVisitor::starts_with to return an ErrorKind::Unexpected error when either operand is non-string, instead of falling back to Ok(false).
  • Add a regression test covering both StartsWith and NotStartsWith against an identity-partitioned binary column.
File summaries
File Description
crates/iceberg/src/expr/visitors/expression_evaluator.rs Rejects non-string StartsWith operands with an error and adds regression coverage to prevent silent mis-pruning.
Review details
  • Files reviewed: 1/1 changed files
  • Comments generated: 0
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants