Skip to content

Carry an array of fixed-size values as the contiguous C array MEOS reads - #164

Merged
estebanzimanyi merged 1 commit into
MobilityDB:mainfrom
estebanzimanyi:codegen/struct-value-arrays
Oct 9, 2026
Merged

estebanzimanyi merged 1 commit into
MobilityDB:mainfrom
estebanzimanyi:codegen/struct-value-arrays

Conversation

@estebanzimanyi

Copy link
Copy Markdown
Member

A SQL array of a value whose C type is a fixed-size catalog struct reaches MEOS as the contiguous
array a constructor reads, spanset_make(Span *spans, int count) and set(npoint[]) on
npointset_make(const Npoint *values, int count) among them, once the catalog states the array in
shape.inputArrays. _array_arg of tools/codegen_jvm.py decodes each element as it decodes the
pointers of an array of values and MeosSqlRuntime.Inputs.structs copies each into the C array,
each of the size #struct_layout of tools/codegen_spark_udfs.py gives the struct, the layout
SqlModel reads once for both engines. The typed Flink and Spark surfaces register
spanset(intspan[]), spanset(bigintspan[]), spanset(floatspan[]), spanset(datespan[]),
spanset(tstzspan[]), set(npoint[]) and set(cbuffer[]). The Spark UDF arm refuses such a
parameter under its C name, since a UDF decoding one value would hand MEOS the count the caller
gives: spanset_make, stboxarr_round, npointset_make and cbufferset_make have no C-named UDF and
the gaps ledger lists them as array: *.

Witness: over MobilityDB 2a1d52a7ee both typed surfaces refuse the seven signatures as
arity:sql, and the C-named Spark UDF spanset_make(spans, count) decodes one span and passes
count to spanset_make, so a count above 1 reads past the value.

Measured over MobilityDB 2a1d52a7ee with the catalog of MEOS-API
catalog/struct-value-input-arrays: both typed surfaces register 1154 SQL functions and 8754
overloads, against 8747 with main's generator, the seven passing Span, Npoint and Cbuffer
elements of 24, 16 and 32 bytes, the sizes gcc gives the structs of the installed headers; the
Spark UDF surface registers the same functions as main's but those four, and the gaps step
reports none new with the ledger at 764 functions. MobilitySpark passes its 35 tests and
MobilityFlink its 21 binding and 12 benchmark tests with assertions that spanset over two int
spans answers {[1, 3), [5, 7)} and set over three npoints, one repeated, and over two cbuffers
answers 2 values; with main's generator MobilitySpark fails on "function spanset(array)
does not exist". Over the catalog of MEOS-API master e84319b both typed surfaces register the
8747 overloads main's generator does and the Spark UDF surface the same functions, the generated
sources differing by Inputs.structs alone.

Why: a SQL array of values answers on every engine as PostgreSQL answers it, and a binding hands
MEOS the array it reads, never one value standing for several.

A SQL array of a value whose C type is a fixed-size catalog struct reaches MEOS as the contiguous
array a constructor reads, spanset_make(Span *spans, int count) and set(npoint[]) on
npointset_make(const Npoint *values, int count) among them, once the catalog states the array in
shape.inputArrays. _array_arg of tools/codegen_jvm.py decodes each element as it decodes the
pointers of an array of values and MeosSqlRuntime.Inputs.structs copies each into the C array,
each of the size #struct_layout of tools/codegen_spark_udfs.py gives the struct, the layout
SqlModel reads once for both engines. The typed Flink and Spark surfaces register
spanset(intspan[]), spanset(bigintspan[]), spanset(floatspan[]), spanset(datespan[]),
spanset(tstzspan[]), set(npoint[]) and set(cbuffer[]). The Spark UDF arm refuses such a
parameter under its C name, since a UDF decoding one value would hand MEOS the count the caller
gives: spanset_make, stboxarr_round, npointset_make and cbufferset_make have no C-named UDF and
the gaps ledger lists them as array:<struct> *.

Witness: over MobilityDB 2a1d52a7ee both typed surfaces refuse the seven signatures as
arity:sql, and the C-named Spark UDF spanset_make(spans, count) decodes one span and passes
count to spanset_make, so a count above 1 reads past the value.

Measured over MobilityDB 2a1d52a7ee with the catalog of MEOS-API
catalog/struct-value-input-arrays: both typed surfaces register 1154 SQL functions and 8754
overloads, against 8747 with main's generator, the seven passing Span, Npoint and Cbuffer
elements of 24, 16 and 32 bytes, the sizes gcc gives the structs of the installed headers; the
Spark UDF surface registers the same functions as main's but those four, and the gaps step
reports none new with the ledger at 764 functions. MobilitySpark passes its 35 tests and
MobilityFlink its 21 binding and 12 benchmark tests with assertions that spanset over two int
spans answers {[1, 3), [5, 7)} and set over three npoints, one repeated, and over two cbuffers
answers 2 values; with main's generator MobilitySpark fails on "function spanset(array<intspan>)
does not exist". Over the catalog of MEOS-API master e84319b both typed surfaces register the
8747 overloads main's generator does and the Spark UDF surface the same functions, the generated
sources differing by Inputs.structs alone.

Why: a SQL array of values answers on every engine as PostgreSQL answers it, and a binding hands
MEOS the array it reads, never one value standing for several.
@estebanzimanyi
estebanzimanyi merged commit 8869436 into MobilityDB:main Oct 9, 2026
2 checks passed
@estebanzimanyi
estebanzimanyi deleted the codegen/struct-value-arrays branch October 9, 2026 18:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant