[rust][client][server][gateway] Relax partial update nullability rule - #3885
Open
gstamatakis95 wants to merge 5 commits into
Open
[rust][client][server][gateway] Relax partial update nullability rule#3885gstamatakis95 wants to merge 5 commits into
gstamatakis95 wants to merge 5 commits into
Conversation
gstamatakis95
force-pushed
the
fix-3849
branch
2 times, most recently
from
August 6, 2026 16:57
d1d732a to
caa1c14
Compare
gstamatakis95
force-pushed
the
fix-3849
branch
from
August 22, 2026 09:04
caa1c14 to
38fa758
Compare
gstamatakis95
marked this pull request as ready for review
August 28, 2026 07:55
gstamatakis95
force-pushed
the
fix-3849
branch
2 times, most recently
from
August 28, 2026 08:26
87cd787 to
220a7ea
Compare
…al update Partial update required every non-primary-key column to be nullable, even columns explicitly listed in the target columns. A listed column is always supplied by the writer, so the requirement rejected valid usage. Restrict the requirement to omitted columns in the Rust client, the Java client and the server. Fixing only the clients is not enough, the server runs its own copy of the check. Auto increment columns keep the requirement. They are always omitted and only receive their value after the merge, so updateRow writes null into them first. Partial delete keeps the requirement on non-primary-key target columns, since it sets them to null. The check sits after the whole-row-removal short-circuit and only exists on the server, because legality depends on the stored row. Tables using the aggregation merge engine keep the stricter requirement on every non-primary-key column. Its merger returns the new row unchanged on the first write instead of null filling, so the server rejects the relaxed schema. The client checks it too, otherwise the writer is created and every write fails asynchronously instead. MergeMode.OVERWRITE is exempt, since it bypasses the configured merge engine and merges with the default merger.
gstamatakis95
force-pushed
the
fix-3849
branch
from
August 28, 2026 09:46
220a7ea to
f5c44b8
Compare
Document the relaxed rule on Upsert, the interface callers actually use, align the website docs on the target columns term, and cover the whole-row-removal delete exemption through KvTablet.
The gateway now requires only omitted columns to be nullable and keeps the per row presence check. The Java client rejects a null in a NOT NULL target column and rejects first_row and versioned partial update early.
Reword the client guard javadoc for the full coverage case and drop a colon from the aggregation paragraph in pk-table.md.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Partial update validation required every non-primary-key column of a primary key table to be nullable, including
columns explicitly listed in the target columns. A listed column is supplied by the writer on every request, so the
requirement rejected valid schemas. The loop's own comment described the intended behavior ("check the columns not in
targetColumns"), but the loop never consulted the target column set.
The check is duplicated across four layers that validate independently, and all four carried the defect:
fluss-rust/.../client/table/upsert.rs,UpsertWriterFactory::sanity_checkfluss-client/.../writer/UpsertWriterImpl.java,sanityCheckfluss-server/.../kv/partialupdate/PartialUpdater.java,sanityCheckfluss-gateway/src/protocol/rest/records.rs,sparse_targetsThe reported symptom cannot be resolved in the Rust client alone. A client-only fix permits writer construction and
defers the same rejection to the server at first write, so all four layers change together.
Resulting validation rules
updateRowwrites null into it when the row does not yet existNOT NULLdeleteRowsets it to null unless the whole row is removedDivergence from the fix proposed in the issue
The issue proposes retaining the auto-increment exemption
(
!target_column_set[i] && !pk_column_set[i] && !auto_increment_column_set[i]) and this PR removes it. Removing theexemption is the correct direction because of ordering on the write path.
PartialUpdater.updateRownull-fillsomitted columns before
AutoIncrementUpdaterassigns a value (KvWriteProcessor.processUpserttoapplyInserttoupdateAutoIncrementColumns). Relaxing the server instead would allow aNOT NULLauto increment column to reachBinaryWriter.createNotNullValueWriterholding null, producing an NPE surfaced to the client asUnknownServerException.Runtime guards
The previous rule guaranteed that a row reaching
PartialUpdatercannot carry null in a non-nullable slot. Relaxingit needs guards, because the decode path is not defensive:
InternalRow.createFieldGetterandCompactedRowReaderomit the
isNullAtbranch for non-nullable types, and CompactedRow is a sequential variable-length encoding, so anull bit in a non-nullable slot corrupts every later field (IndexedRow, whose reader honors the null bit per column,
corrupts only that column).
updateRowrejects a null value supplied for aNOT NULLtarget column, primary key columns included. The checkruns before any field getter, because the first getter deserializes the whole row with the non-null-checking
readers and would fail first with an error dependent on the bytes that follow.
isNullAtreads only the null-bitheader, so the check itself never deserializes.
deleteRowrejects partial delete when a non-primary-key target column isNOT NULL. The guard is placed afterthe
isFieldsNullshort circuit, so whole-row removal, which nulls nothing, remains legal. The guard is serverside only, since legality depends on the stored row, and a client-side equivalent would be stricter and would make
a server-supported operation unreachable.
would otherwise fail with a bare
NullPointerExceptionbefore the request is sent.NOT NULLtarget column in each upsert entry of a sparse batch,using the decoder's existing per-row required-column enforcement. Delete entries carry only primary key values, so
the server judges them against the stored row.
One path stays outside the server guards: target columns covering every schema column short circuit in
DefaultRowMerger.configureTargetColumnsand never build aPartialUpdater. The exposure equals a plain full-rowupsert, the Java client guard covers the case client side, and the encoders reject a null before anything is stored.
Merge engines
The
first_rowandversionedmergers reject partial update outright, and theaggregationmerger does not fillomitted columns with null on the first write, so aggregation tables keep requiring every column except the primary key to
be nullable. The Java client now checks all three at writer creation with the server's wording. Before,
first_rowand
versionedfailed at the first write asUnknownServerException, and aggregation was rejected at creation withthe generic message.
MergeMode.OVERWRITEis exempt, becauseKvWriteProcessormerges an overwrite with the default merger rather thanthe configured merge engine.
The Rust client reads no merge engine configuration today. The table config is available at writer creation, but the
guard is not part of this PR, to keep the Rust diff small. A Rust user on such a table still sees the rejection at
the first write. The Rust client also has no per-row check for a null in a
NOT NULLtarget column. Such a write failsin the encoder with a type error before it is sent, so nothing bad reaches the server. The gateway performs no merge
engine check either, so a REST partial update on such a table is rejected by the backend writer rather than at
preflight.
Error message
Changed from:
to:
A
NOT NULLauto increment column gets a dedicated message, because the user cannot follow the generic advice oflisting the column, targeting an auto increment column is itself rejected:
The gateway keeps its own message style and now reports the omitted column the same way, including the dedicated
auto increment message.
Flink
A Flink sink writing a partial update (a column subset in the SQL INSERT, or partial update columns on
FlussSinkBuilder) with aNOT NULLtarget column initializes and writes, and the first retract record is rejectedby the server's delete guard, which is not retriable. Insert-only pipelines are unaffected. The sink does not check this at plan time, so the rejection surfaces when the first delete arrives.
Compatibility
No wire format, storage format, or public API signature changes. The change relaxes validation, so previously
accepted writes remain accepted. Three cases move their failure from the first write to writer creation: partial
update on
first_rowandversionedtables, and aNOT NULLauto increment column. The rejection message foraggregation tables changes from
Partial Update requires...toPartial aggregate requires....Fixes #3849
🤖 AI-assisted changes - reviewed by human developer