Skip to content

perf: reuse evaluated hash join build keys - #27285

Open
Phoenix500526 wants to merge 1 commit into
apache:mainfrom
Phoenix500526:issue/26806
Open

Phoenix500526 wants to merge 1 commit into
apache:mainfrom
Phoenix500526:issue/26806

Conversation

@Phoenix500526

@Phoenix500526 Phoenix500526 commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Closes #26806.

Rationale for this change

After #26793, computed integer build keys can be evaluated twice over the concatenated build batch under NullEqualsNull: once for the join and again just to check for nulls when selecting an ArrayMap.

What changes are included in this PR?

Pass the already evaluated key arrays (&left_values) to array_map_key_range. Keep the single-key and supported-integer-type checks inside the function, using the evaluated arrays. Derive the build row count from key.len() inside the function instead of passing a separate num_rows argument. Remove the duplicate expression evaluation, row-count loop, and unused parameters.

What is the testing strategy for this PR?

The existing 619 hash join execution tests pass. These cover NULL handling, non-integer and multiple keys, empty inputs, and map selection; no new tests are added because join behavior is unchanged. Formatting, cargo clippy --all-targets --all-features -- -D warnings, and the full workspace test suite with extended tests also pass.

A local example using a computed physical join key was also checked under LLDB: after entering build_in_memory, BinaryExpr::evaluate is now called once instead of twice. The example's NULL and non-NULL cases retain their results and map selection.

The existing hash_join_semi_anti/right_semi_d100_h100/1000000 benchmark was compared with a local main baseline; Criterion reported the change within its noise threshold. This is a regression check, not a measurement of the computed-key improvement.

Are there any user-facing changes?

No SQL result or public API changes. Avoids redundant build-key evaluation for eligible null-safe hash joins.

@github-actions github-actions Bot added the physical-plan Changes to the physical-plan crate label Nov 10, 2026
@Phoenix500526
Phoenix500526 marked this pull request as ready for review November 10, 2026 08:35
@Phoenix500526

Copy link
Copy Markdown
Contributor Author

CC @jayzhan211 @dwsmith1983 This PR is ready for review now.

@dwsmith1983

Copy link
Copy Markdown
Contributor

Thanks @Phoenix500526, looks good to me. left_values is already evaluated from the concatenated batch right above the call, and checking the evaluated key's type is the same test is_perfect_hash_join_candidate ran on the expression, so behavior stays the same while the extra key evaluation goes away. Nice to lose the too_many_arguments expect as well.

@dwsmith1983

Copy link
Copy Markdown
Contributor

A couple of small things I missed earlier, none blocking:

  • The title says fix:, but nothing was broken before, so perf: or refactor: probably fits better.
  • num_rows always equals key.len(), so the function could take one fewer argument. The issue suggested passing num_rows, so up to you.

Also, the Rust CI jobs (clippy, cargo test) show as skipped here. I ran clippy and the joins::hash_join tests on 200f3e4 locally and they pass.

Avoid evaluating build keys again just to check for nulls.

CLOSES apache#26806
@Phoenix500526

Copy link
Copy Markdown
Contributor Author

A couple of small things I missed earlier, none blocking:

  • The title says fix:, but nothing was broken before, so perf: or refactor: probably fits better.
  • num_rows always equals key.len(), so the function could take one fewer argument. The issue suggested passing num_rows, so up to you.

Also, the Rust CI jobs (clippy, cargo test) show as skipped here. I ran clippy and the joins::hash_join tests on 200f3e4 locally and they pass.

Thanks @dwsmith1983 I've removed the argument.

@Phoenix500526 Phoenix500526 changed the title fix: reuse evaluated hash join build keys perf: reuse evaluated hash join build keys Nov 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

physical-plan Changes to the physical-plan crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Avoid re-evaluating build keys in array_map_key_range

2 participants