You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Consolidate the Scala "cannot fail" checks: arrays.cannotThrow and QueryPlanSerde.cannotFail #7585
Comet's Scala side has more than one answer to "can evaluating this expression raise an error?":
cannotThrow in arrays.scala admits only a Literal, Attribute or BoundReference. CometArrayAppend uses it to decide whether the item can be evaluated for a row Spark would not evaluate it on:
Each copy has to be widened or tightened on its own, so a fix to one (for example a new cast that cannot fail, or a kernel found to raise where Spark does not) is easy to miss in the others.
Describe the potential solution
Keep one Scala notion of an expression that cannot fail, QueryPlanSerde.cannotFail, and have arrays.scala (and NativeLambdaSerde.mayRaise, if #7322 lands first) call it instead of keeping its own list.
This widens when array_append runs natively, so the change needs its own tests for the newly admitted item expressions, including ones that fail under ANSI.
Activity