Blackbird: ClassFile generation of unrolled codec class, optimize record / builder / field - #359
Open
stevenschlansker wants to merge 43 commits into
Open
Blackbird: ClassFile generation of unrolled codec class, optimize record / builder / field#359stevenschlansker wants to merge 43 commits into
stevenschlansker wants to merge 43 commits into
Conversation
added 12 commits
September 6, 2026 22:31
…ecs (ClassFile API) New engine: BBDeserializerModifier wraps eligible stock BeanDeserializers in a placeholder that generates a hidden-class codec at resolve time (matcher from ctxt.tokenStreamFactory(), classData for the matcher and per-property SettableBeanProperty payloads, no ClassOption.STRONG so codecs unload with the mapper). Tier-A scalars (String/int/long/boolean public setters with stock deserializers) inline; everything else rides the stock property from generated dispatch; VALUE_NULL always routes through the stock property; an active view delegates per call to the stock deserializer. Conservative modify-time gates keep stock behavior for creators, builders, object ids, any-setters, aliases, merge, non-public or inner classes. Old LambdaMetafactory deserializer engine removed (SuperSonic*, Optimized*, Settable*Property, CreatorOptimizer). Serializer side unchanged for now. Module compiles at release 25 (java.lang.classfile). Suite: 356/356 on the module path; on the classpath the pre-existing tofix/TestBBClassloaders mode-dependence surfaces (passes there on pristine 3.x too). Known follow-up: hidden-class definition does not trigger loading of its not-yet-loaded superclass under module-path deployment (worked around by deriving the superclass ClassDesc from the class literal; minimal repro owed).
… artifact The NoClassDefFoundError came from GeneratedCodecBase.class being absent from target/test-classes (javac implicit compilation under --patch-module emits only compile-time-referenced main classes, and the test module shadows target/classes under ModuleFinder first-wins), not from hidden-class superclass resolution. Investigated separately with a minimal repro; the JDK follows its specification. The class-literal form stays as the durable fix.
Records collect components into typed locals in canonical-constructor order and build via the constructor MethodHandle (invokeExact); VALUE_NULL and non-tier-A components route through the stock CreatorProperty's deserialize() with a wrapper-to-primitive convert. Gated on one CreatorProperty per component with matching index and type, and on the user Lookup (records have a private canonical constructor). Modifier bails under FAIL_ON_MISSING/NULL_ CREATOR_PROPERTIES and skips the no-arg-ctor check for records.
…or and build method Builder-based beans generate a codec that creates the builder via the stock ValueInstantiator, applies properties with deserializeSetAndReturn semantics (tier-A fluent setters must return void or exactly the builder class), and finishes through the build method unreflected via the user lookup and asType'd to (Object)Object. The build method rides a ThreadLocal from updateBuilder to modifyDeserializer, the only place it is exposed. Abstract value types are allowed for builder beans (the builder instantiates).
…k odd-token semantics CHILD properties (value deserializer is a generated codec) call the child codec directly through a classData constant in all three modes, with a non-START_OBJECT guard falling back to the stock property so coercion and error semantics stay stock. Non-public POJO scalar setters reach their bean through classData MethodHandles from the user lookup; access failure demotes to the stock path. Unexpected tokens where a property name belongs now report through DeserializationContext.handleUnexpectedToken like the stock deserializer (engagement tests switch from exception-type probes to a capture modifier; the builder flow keys modifiers by the builder class).
Generated writers: straight-line writeName/write-value with pre-encoded name constants, tier-A public getters (exact BeanPropertyWriter, no null suppression, stock scalar serializers), child-writer linking through classData constants, the stock PropertyWriter.serializeAsProperty for everything else, and per-call delegation to the stock serializer when a view is active. Interface-typed beans dispatch getters with invokeinterface. Old LambdaMetafactory serializer engine removed.
On the AArch64 C2 backend, inlining small hot write helpers into a looping caller makes the register allocator spill working state across the loop (perfasm: 31% of hot-region samples in stack traffic). Generated writer bodies contain inline list loops, which is exactly the trigger shape. Emit each property write through a per-kind static helper that fuses writeName with the value write and is padded past FreqInlineSize, so every helper compiles as a standalone unit. Paired benchmarks: order-record writes go from -24% to +9% against the previous engine on aarch64, +4% on x86_64; media writes +18%/+9%; run-to-run bimodality is gone on both architectures. OpenJDK report on the underlying C2 behavior is drafted; the padding gets the JDK- number once triaged.
Describe the ClassFile-API engine, the JDK 25 floor, the record and builder acceleration, and the per-bean fallback gates. Remove the metaspace OOM warning: hidden classes are anchored by the codec instances in the mapper's caches and unload with the mapper.
The inject tests asserted the deleted per-property engine internals (OptimizedSettableBeanProperty, OptimizedBeanPropertyWriter) and captured deserializers before Blackbird's modifier ran. The harness now registers the capture module first (modifiers run in reverse registration order) and the tests assert the current contract: eligible classpath beans engage a generated codec (BBCodecPlaceholder / BBSerCodecPlaceholder), values and output match stock databind, and field-backed properties ride the codec's stock-property arms. The CrossLoaderAccess fast-path pin stays and is deleted together with CrossLoaderAccess (modules-base#350).
…#350) After both engines were replaced, CrossLoaderAccess survived only as an unused accessGrant constructor parameter on the modifiers, ReflectionHack had no references at all, and Unchecked, Sneaky, CheckedFunction, and CheckedSupplier only fed each other. FasterXML#350 already scheduled the CrossLoaderAccess removal for 3.3 (deprecated forRemoval since 3.2); this deletes the whole chain, drops the accessGrant plumbing, and removes the CrossLoaderAccess fast-path pin test with it. Suites green: blackbird 361/361, blackbird-tests 8/8.
Field-backed properties were left on the stock path (Kind.STOCK / WKind.STOCK); old Blackbird could not write fields at all through LambdaMetafactory. The generator now stores a public non-final field through putfield and reads a public field through getfield, in the hidden class. A non-public field is reached through a user-lookup-derived setter handle on the deserializer side (the same fallback non-public setters use); non-public fields on the serializer side keep the stock writer, matching the lack of a non-public read path there. Final fields keep stock behavior on the deserializer side. Config gates and the VALUE_NULL / null-to-primitive routing are identical to the setter path. New BBCodecFieldTest covers public-field values/nulls/unknowns/reversed order, null-to-primitive throwing, final-field and mixed field+setter parity with stock, and private-field access with and without a lookup. blackbird-tests FieldAccessNotOptimizedTest becomes FieldAccessTest (fields now accelerate); SerializerInjectionTest's field case updated to the same contract.
…nst exotic beans tofix/TestBBClassloaders failed on the module path because it read the bean class bytes through classloader getResource, which JPMS encapsulation nulls; Class#getResourceAsStream resolves inside the module and works in both modes. With that fixed the test exposed a real gap: the modifier gates call getEnclosingClass, which raises IncompatibleClassChangeError for a child-classloader bean whose InnerClasses metadata disagrees with the parent-loaded enclosing class. Both modifiers now demote to the stock (de)serializer on any gate RuntimeException or LinkageError, and both factories gate generation on the bean class resolving to the identical class through this module's loader, since generated code refers to it by name. The test moves to ser/ChildClassloaderTest as a positive cross-loader regression test (the failure-expected annotation is gone; the child-loader bean takes the stock path and the pinned contract is correct output). Suites pass on the module path and with -Dsurefire.useModulePath=false.
added 16 commits
September 7, 2026 06:02
Demoting to the stock path on RuntimeException or LinkageError keeps an acceleration module from breaking beans stock databind handles, but a silent demotion can also hide a real bug. CodegenFallbacks splits the policy by phase: gate failures (reflective checks throwing on exotic classes) log a warning once per type, and generation failures (the type passed every gate and the generator still failed) log severe once per type - or rethrow when the tools.jackson.module.blackbird.failOnCodegenError system property is set, which both test bases enable so the whole suite runs fail-fast. The report set is keyed by class name, never Class, so it pins no foreign classloader. Suites green in both module-path and classpath modes.
…or checks, unwrapping, and entry shapes
Behavioral parity fixes, each verified against a vanilla mapper:
- Tier-A scalar arms take the inline read only when the value token is the
expected one (VALUE_STRING, VALUE_NUMBER_INT, VALUE_TRUE/VALUE_FALSE);
every other shape routes through the stock property, which owns coercion
and null handling. Quoted scalars ({"i":"123"}) coerce again instead of
throwing InputCoercionException.
- The unknown-name arm calls DeserializationContext.handleUnknownProperty
through a base-class helper, so per-call FAIL_ON_UNKNOWN_PROPERTIES and
DeserializationProblemHandlers work. Beans with ignored or included
property sets stay on the stock deserializer (new modifier gate), keeping
ignoral semantics exact.
- Record codecs track seen components in a bitmask and report missing ones
like PropertyValueBuffer: @JsonProperty(required=true) fails, and per-call
FAIL_ON_MISSING_CREATOR_PROPERTIES is honored. Records with more than 64
components or injectable components stay stock.
- Generated codecs and placeholders forward unwrappingDeserializer/
unwrappingSerializer (and acceptJsonFormatVisitor) to the stock delegate;
@JsonUnwrapped with a codec-eligible child now produces stock output and
reads back correctly, plain and prefixed.
- deserialize entered on any token but START_OBJECT delegates to the stock
deserializer, which accepts PROPERTY_NAME and other entry shapes.
- BBDeserializerModifier recreates its transient ThreadLocal after JDK
deserialization.
New BBCodecCompatibilityTest covers every case against vanilla; suites green
in module-path and classpath modes.
blackbird compiles at release 25 (ClassFile API) while the reactor floor stays JDK 17, so the two blackbird modules move behind a jdk-activated profile and the workflow matrix gains a 25 entry. The release/deploy leg moves from 17 to 25 so the deployed reactor includes blackbird; every other module still builds and tests on 17, 21, and 24.
…with the reference path Non-public classes were the old engine's bread and butter; the rewrite now covers them again. Non-public beans (package-private, protected static nested; private stays stock) define their hidden codec in the bean's package context via privateLookupIn with the user-supplied lookup - on the classpath the module's own lookup suffices, and under JPMS a module that does not open the bean's package demotes to stock (an environment gate, not an error). Member gates relax accordingly: a non-private member of a same-package class is directly accessible to a bean-context codec, decided by one shared predicate (CodecAccess) on both the deserializer and serializer sides. A constructor-accessibility gate replaces the public-only check, closing a latent IllegalAccessError for public beans with package-private constructors. Widening the accelerated set exposed three more parity gaps, now gated or fixed: - Property exceptions carry the reference path: every generated property arm runs under an exception handler that rethrows through the stock wrapAndThrow, so error paths and messages match stock databind. - Beans with injected values stay stock (injection happens outside the property loop). - Serializer-side custom includes (JsonInclude CUSTOM and friends) stay on the stock writer, detected through a BeanPropertyWriter suppression probe. PackagePrivateBeanTest covers engagement and byte parity for package-private beans, members, records (required check included), protected nested beans, and the private-stays-stock contract. OptionalDeser355Test's package-private beans engage codecs again, restoring the original test's intent with new-engine equivalents of its per-property assertions. Suites green in module-path and classpath modes.
A hidden codec defined in a user module's context must resolve its supertype from that module, so GeneratedCodecBase and GeneratedWriterBase move to tools.jackson.module.blackbird.internal, exported as documented-internal API (package-info states the contract). The generators, CodecAccess, and CodegenFallbacks stay unexported: only constant-pool-referenced supertypes need to be resolvable from outside, and the bases' own bodies resolve in blackbird's context. With the export, bean-context defines succeed for any user module that reads blackbird - which supplying a lookup implies - so package-private acceleration works under JPMS; a define still demotes to stock only when the bean's module does not read blackbird at all. The OSGi manifest already exports tools.jackson.module.* by wildcard.
The parent chain configures compiler source/target only, so a build on a newer JDK would link 17-floor modules against newer JDK APIs with 17 syntax. maven.compiler.release=17 in the root pom pins linkage to the JDK 17 API (verified: mrbean built on JDK 26 emits class-file 61, and a JDK 21 API reference fails to compile); blackbird keeps its release=25 override and sets javac.src.version/javac.target.version to 25 so the jar manifest's X-Compile-Source/Target-JDK entries match the class-file-69 bytecode. The inherited source/target properties stay: the compiler ignores them once release is set, and oss-parent feeds them into the manifest entries. jackson-base upstream is the better long-term home for the release property.
…lackbird blackbird.jpms.test is the arrangement a modular application has: a named module that requires blackbird, holds a package-private bean, opens the bean package to databind (as stock reflection needs), and supplies its own lookup. With the lookup, the bean accelerates: the codec is defined in the test module's package context and resolves its supertype through blackbird's exported internal package - notably without the bean package being opened to blackbird, and the accelerated path did not even need the databind opens to construct. With the default lookup the bean demotes to the working stock path. Engagement is asserted through the blackbird.debug.codegen diagnostic stream, since generated and stock output are byte-identical by design.
The module asserts module-path semantics; the reactor-wide -Dsurefire.useModulePath=false toggle must not apply to it (on the classpath the unnamed module is fully open and the default-lookup control correctly accelerates instead of demoting).
With -Dblackbird.debug.dumpDir=<dir>, both generators write each generated class's bytes to <dir>/<bean-class-name>-codec.class or -writer.class immediately before the hidden class is defined, so the emitted code can be inspected with javap or a decompiler. A dump failure logs at FINE and never affects generation. Documented next to the existing debug property in the README.
An active view makes both generated codecs delegate the whole call to the stock fallback, whose own view machinery (filtered writers on the ser side, visibleInView checks on the deser side) then applies - the stock serialize and deserialize entry points route to their view-filtered paths internally, so whole-call delegation is stock-equivalent by construction. The existing view tests use package-private beans, so this pins the contract through codecs that provably engage: split-view properties compared against vanilla in both directions and per view, a record, a field-backed bean, a nested codec-generated child, skipped-value stream consumption, and an engagement probe.
An active view previously delegated the whole call to the stock (de)serializer, losing the generated fast path exactly where old Blackbird kept it. Generated codecs now resolve a per-view visibility bitmask (cached per view on the codec instance, computed from the stock properties so visibleInView / getViews and DEFAULT_VIEW_INCLUSION semantics stay exact) and each property arm tests its bit: a visible property runs the fast arm, a hidden one is consumed with stock skip semantics (deser) or omitted (ser). Beans with more than 64 properties keep the whole-call delegation, and a bean no view can affect emits no view code at all. BBCodecViewTest extends to the mask path, DEFAULT_VIEW_INCLUSION-off, and the >64-property delegation fallback.
MethodParameters and LocalVariableTable entries on every generated method: parameters (p/ctxt, value/g/ctxt, fallback, activeView, the helper g/name/value triple) and the working locals (bean, builder, matcher, ix, prop, e, seen, viewMask, child, and record components under their component names). Both attributes are debug metadata the JIT ignores; slot entries are emitted only when the slot is stored, since LocalVariableTable indexes must stay within max_locals. Dumped codecs (blackbird.debug.dumpDir) now decompile with source-like names.
The first mask rule mirrored stock's blanket _needViewProcesing flag, which is true for every bean when DEFAULT_VIEW_INCLUSION is off - the 3.x default - so every codec carried mask code on the no-view hot path. Worse, the resolved properties cannot even distinguish a declared view from the empty view set that disabled inclusion forces onto unannotated properties, so the modifier now reads the declaration from the property definitions (member and class-level @JSONVIEW through the annotation introspector) and hands the placeholder a declaresViews flag. Beans that declare nothing emit no per-arm view code: with inclusion on, views cannot affect them (NONE); with it off, an active view hides everything, so the degenerate view-active call delegates to stock (DELEGATE) and the no-view hot path stays free of mask tests. The serializer side reads the same signal from BeanPropertyWriter.getViews, where the forced set is empty rather than null. Declared views keep the MASK fast path. Vanilla-compared view tests pass unchanged in both modes.
The JDK classDataAt bootstrap rejects any condy name but "_", which forced positional indexes into the generators and unreadable classDataAt<"_",N> references into dumps. Class data is now a name-addressed map resolved by a classDataEntry bootstrap on the generated-code base classes (owner always resolvable: the internal package is exported for the supertypes), and every condy carries a meaningful name - propertyProp/propertyCodec/propertySetter/ propertyName per property (sanitized to JVM unqualified-name rules and deduped), plus matcher, constructor, instantiator, and buildMethod. A condy still links once and constant-folds, so steady-state code is unchanged; the positional index bookkeeping in both generators is gone.
added 2 commits
September 8, 2026 04:35
Codec stays the umbrella term for a reader/writer pair; single-direction classes now say which they are: GeneratedReaderBase, BeanReaderGenerator, BBReaderFactory, BBReaderPlaceholder, and on the write side BBWriterFactory and BBWriterPlaceholder drop the Ser prefix. Generated reader classes are named BBReader_<Bean> and dump as <bean>-reader.class. Direction-neutral names (CodecAccess, CodegenFallbacks, CodegenDump, the debug properties, the BBCodec*Test umbrella suites) stay.
The entry guard now branches forward to a delegation tail emitted after the main body, so the cold fallback path sits out of the hot fall-through layout and the guard emission loses its label gymnastics. Behavior is identical; decompilers still render the guard as a nested conditional, which the generated-code appendix notes.
stevenschlansker
marked this pull request as ready for review
September 8, 2026 05:34
added 13 commits
September 8, 2026 05:46
The AsProperty polymorphic path hands a subtype deserializer a stream positioned on the property after the type id (id first) or a buffered-replay sequence starting on one (id last), so every polymorphic subtype read entered the stock delegation tail and lost the generated fast path. The entry guard now admits PROPERTY_NAME, and the first match mirrors stock BeanDeserializer's currentNameMatch loop head: a name entry matches the current name and dispatches into the same arms, which advance to their value tokens themselves. Empty-remainder entries (END_OBJECT after an id-only object) and all other shapes keep the stock delegation. New BBReaderEntryTest pins the databind entry contract with a vanilla spy and compares polymorphic reads (both id orders, record subtype with required components, unknown first name, hand-positioned readValue) against vanilla.
…te strict-feature gates The generated record path bypassed PropertyValueBuffer, so per-call FAIL_ON_NULL_CREATOR_PROPERTIES was silently ignored once a codec was cached (the build-time gate only helped when the first use carried the feature). The codec now collects a null mask over reference-typed components at construction - missing and explicit null alike, primitives never - and a cold helper mirrors PropertyValueBuffer's reporting. With required, FAIL_ON_MISSING, and FAIL_ON_NULL all enforced per call, the build-time creator-feature gate is gone, as is the FAIL_ON_UNKNOWN_PROPERTIES gate that the unknown arm's ctxt.handleUnknownProperty already honors, so strict-mode mappers accelerate. New parity tests, all vanilla-compared with engagement asserted: strict features per-call and build-time (BBCodecStrictFeaturesTest), polymorphic writes through engaged writers for PROPERTY and WRAPPER_OBJECT inclusion (BBWriterPolymorphicTest), and readerForUpdating (BBReaderUpdatingTest).
Beans carrying @JsonIgnoreProperties, @JsonIncludeProperties, or @JsonIgnore demoted to stock because the codec's unknown arm implemented only the plain contract. The modifier now carries the ignoral configuration into the codec (a constructor argument on the generated reader), and _handleUnknown consults it in the stock loop's exact order: ignore-all skips silently (even under per-call FAIL_ON_UNKNOWN_PROPERTIES), explicitly ignored and not-included names go through ignored-property handling (honoring FAIL_ON_IGNORED_PROPERTIES with the stock exception), and everything else runs the problem handlers as before. Per-use ignorals attached where a bean is referenced still fall back to the stock contextual instance through the placeholder. BBCodecIgnoralsTest covers each set kind, both strict features, records, and unknown-outside-the-set reporting, vanilla-compared with engagement asserted.
Case-insensitive matching demoted whole mappers and any alias demoted its bean; both now ride the matcher, the way stock BeanPropertyMap builds it. The modifier computes the effective per-class case-insensitivity exactly as the stock builder does (the per-class @jsonformat override wins, the mapper feature is baseline) - a probe showed the class-level override never reaches createContextual, so a config-only check would silently read case-sensitively. The factory then constructs the core CI matcher with the configured locale, and appends alias names after the primaries, each switch case sharing its primary's arm, so aliases hit the same generated code path (and the same seen-bitmask bit for record required tracking). BBCodecMatcherTest covers CI mappers, aliases through every name, alias-satisfies-required, aliases under CI, and the class-level format override, vanilla-compared with engagement asserted.
Vanilla-compared behavior tests assert these beans stay on the stock path (raw stock from the modifier, or a placeholder whose codec never generated for the factory-gated object-id case), so a future change that silently starts generating for them fails a test instead of drifting.
canCreateUsingDefault and canCreateFromObjectWith are true for a no-arg or component-matching @JsonCreator factory too, and databind then constructs through the factory. The generated POJO and record codecs constructed with a direct new / canonical-constructor invokeExact, silently bypassing it. Both paths now gate on the instantiator's selected creator being the constructor itself (for records, an AnnotatedConstructor that passes the per-component index and type match is the canonical constructor); factory-creator beans stay on the stock path. BBCodecCreatorTest pins both cases vanilla-compared. Constructing through the instantiator held in classData is the follow-up that would re-accelerate factory beans.
… instantiator when the creator is not the constructor Two acceleration gaps close. Beans with @JsonAnySetter engaged nothing: the unknown arm now feeds the stock SettableAnyProperty (read off the resolved fallback through a probe subclass, the BeanPropertyWriter probe pattern) in handleUnknownVanilla's exact order - explicit ignorals first, then the any-setter, then ignore-all, then the unknown handling. Records with an any-setter stay stock: their values buffer before construction. POJOs whose default creator is not an accessible no-arg constructor (a no-arg @JsonCreator factory, a custom ValueInstantiator, a non-public constructor) demoted whole; the codec now constructs through the stock instantiator held in class data, exactly like stock, and the property loop stays generated. Records with factory creators still demote (the canonical constructor is the only generated record construction). BBCodecAnySetterTest covers the ordering matrix vanilla-compared; the any-setter demotion pin is widened to an engagement test; BBCodecCreatorTest asserts the factory bean engages and matches vanilla.
…xAccess Databind runs fixAccess on every member before modifiers see it, and unreflection of an accessible-flagged member does no access check, so the module's own lookup reaches exactly what stock databind can invoke. Member stores, loads, and construction now travel as named classData MethodHandle constants with erased descriptors (primitives kept, references widened to Object), and the generated bytecode never names the bean class. Consequences: - Private classes, private members, private record constructors, private builder build methods, and foreign-classloader beans all accelerate; an unreflect failure demotes to the stock path, which fails identically. - Every codec defines in the module's own context; CodecAccess and the privateLookupIn define-context machinery are deleted, along with the visibleToGenerator loader-identity gate and the modifier constructor gates. - No user lookup is required for any acceleration. The BlackbirdModule lookup API remains as released, now-redundant API, and the only JPMS requirement left is the opens-to-tools.jackson.databind stock needs. - Gated beans hand back the raw stock (de)serializer at contextualization, so databind's instanceof-based decisions (Nulls.AS_EMPTY no-creator check) are unchanged. - The generated class name derives from Class.getName, not getSimpleName: the latter reads InnerClasses metadata, which throws for member classes redefined in a foreign classloader. Parity gate (paired probe-gated aarch64, medians of 6 vs the previous commit): order reads 1.045, media reads 1.010, order writes 0.997, media writes 1.001 - no cell regresses beyond noise. Widened pins: ChildClassloaderTest asserts the child-loaded bean generates a writer; PackagePrivateBeanTest asserts private classes and private accessors accelerate; blackbird-jpms-tests asserts acceleration with no user lookup. Suites green in module-path and classpath modes.
… construction Generated writers now override serializeWithType with stock BeanSerializerBase's WritableTypeId flow (typeId with START_OBJECT shape, writeTypePrefix, assignCurrentValue, the property sequence, writeTypeSuffix), so polymorphic writes of accelerated subtypes stay on the generated path for every inclusion mechanism. The props body is emitted into both methods so the plain serialize path keeps its measured shape. Beans with a @JsonTypeId property keep the whole-call forwarding (a probe on the stock serializer reads _typeId); object-id and filter-id beans were already gated. Beans with @JacksonInject values engage codecs: the base class reads the stock deserializer's resolved ValueInjectors through the probe and generated code applies them right after construction, before the property loop - the stock deserializeFromObject placement, so document values override injected ones exactly like stock. Builder codecs inject into the builder. Records with injectables stay stock (no instance exists until the document ends); the modifier gate narrows to records. Widened pins: injectable POJOs engage (with a document-override case), injectable records pinned stock; poly writes vanilla-compared for PROPERTY, WRAPPER_OBJECT, and WRAPPER_ARRAY with a generated-codec assertion, and a @JsonTypeId bean pinned on the forwarding path. Suites green in module-path and classpath modes.
Generated codecs dispatch on the match index and then read the value, so each property paid two out-of-line parser calls. The fused call commits the value token the byte parser already classified during name matching; a non-negative match leaves the current token on the value, and the scalar, child, field, and stock arms consume it directly. Negative results (unknown name, END_OBJECT, odd token) behave exactly as the two-call sequence. Measured through these codecs, fused vs two-call, identical module otherwise: record graph +7.6% (aarch64) / +12.7% (x86_64) median, setter POJO +5.4% / +6.1%. Requires jackson-core with nextNameMatchAndToken (upstream #1688, merged; resolved in the 3.3.0 snapshot). Reapplied onto the follow-up-wave HEAD: the constant-handle rewrite reshaped the arm emission, so the fused change now also drops the standalone advance from the erased stock and view-mask arms.
The generators named most referenced types as strings, so a rename or a typo in a descriptor survived compilation and failed when the generated class was defined. Descs.of(Class) derives them from class literals instead, which the compiler checks and refactoring follows, and the same helper replaces the describeConstable().orElseThrow() incantation at the runtime-type sites. The class literal also keeps each type compile-time-referenced, which the --patch-module test build needs (that requirement was documented on one field; it now holds for all of them by construction). The debug property names were repeated in four places and the trace was written through an if-guard at seventeen. CodegenDebug holds both property names and the resolved flag, and CodegenDump reads the dump directory from it. The JPMS test asserts codec engagement through that trace and set the property from a static initializer; surefire now sets it, which cannot race class initialization.
The generation trace covered the reader factory only, so a bean that never got a generated writer produced no output, and neither modifier reported the whole-bean demotions that account for most of them. Both modifiers now name the bean and the reason, the writer factory traces its gates and its result the way the reader factory does, and each side's per-property demotions are reported from the one stock() helper they already funnel through. Messages carry the side, since the two paths otherwise print the same words for different decisions. CodegenDebug.logSkip leaves out primitives, arrays, enums and JDK types: a modifier sees every type the mapper resolves, and tracing those buried the bean the trace was being read for.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This replaces the Blackbird implementation. The module coordinates, the
BlackbirdModuleAPI, and the registration pattern do not change. The LambdaMetafactory engine is gone; the module now generates one hidden class per bean type with the ClassFile API (JEP 484) and installs it as that bean's deserializer and serializer.Why replace rather than extend: Jackson 3 core ships
PropertyNameMatcherandJsonParser.nextNameMatch(), which match property names from the input buffer and return an index. A generated codec can consume that index in atableswitchand read each property with direct, monomorphic code. The LambdaMetafactory design cannot express this; it only accelerates the accessor calls inside the stock deserializer loop, and the measured gap shows most of the remaining cost sits in that loop, not in the accessors.Against old Blackbird 3.2.2 at this PR head, the rewrite reads records 59% faster (2.2x vanilla databind), builder beans 28% faster, and setter POJOs 25% faster, with writes up 4% to 15%
This PR requires modules-base project to be released with JDK 25, although it maintains JDK 17 support for all modules but Blackbird
Designed with Claude Fable 5. Claude's notes below:
The engine
defineHiddenClassWithClassData. classData carries the name matcher, child codec handles, constructor or build-method handles, and the stock deserializers used as fallbacks. Constant MethodHandles +invokeExactinline like direct calls.ClassOption.STRONG. The codec instance in the mapper's cache anchors the class, so codecs unload with the mapper. This fixes the documented Blackbird classloader-pinning problem (Metaspace/class leak using blackbird #147): the old engine's lambdas stayed alive with the target ClassLoader, so applications that create many mappers ran out of metaspace. The README warning that Blackbird: add memory 'leak' OOM warning #215 added for this comes out.newplusinvokevirtualsetters), records (typed locals in canonical-constructor order, one constructor call), and builder beans (stockValueInstantiatorcreates the builder, fluent setters inline, build method through a constant handle). Afterburner and old Blackbird never accelerated the record or builder paths.SettableBeanProperty, so behavior matches stock databind. Beans the generator cannot cover take the whole-bean bailout and keep the stockBeanDeserializer.What Accelerates
Beyond the base rewrite, the following now accelerate: beans with
@JsonAnySetter(POJO and builder; the unknown arm feeds the stock any-setter inhandleUnknownVanilla's exact order); POJOs whose default creator is a no-arg@JsonCreatorfactory or a custom instantiator (construct through the stockValueInstantiator); beans with@JacksonInject(POJO and builder; injectors apply after construction, before the property loop, so document values override injected ones like stock); and polymorphic writes -serializeWithTypeis generated natively with stockBeanSerializerBase'sWritableTypeIdflow for every inclusion mechanism (PROPERTY, WRAPPER_OBJECT, WRAPPER_ARRAY). Records with an any-setter, injectables, or a factory creator, and beans with a@JsonTypeIdproperty, stay on the stock path (pinned).Measured results
Access
The rewrite no longer needs a user-supplied
MethodHandles.Lookupfor any acceleration. Generated codecs reach members throughMethodHandleconstants unreflected with the module's own lookup after databind has already runfixAccesson those members, and every descriptor is erased (references widened toObject), so the generated bytecode never names the bean class.Consequences: private classes, private members, private record constructors, private builder build methods, and foreign-classloader beans all accelerate; every codec defines in the module's own context; on the module path the only requirement is the same
opens ... to tools.jackson.databindstock databind already needs. TheBlackbirdModulelookup constructors andfindLookup/findLookupSupplieroverrides remain as released API but are no longer required by anything. This deletesCodecAccess, theprivateLookupIndefine-context machinery, and the foreign-classloader gate.Fixes #142 - the affected code path is removed.
Serializer inlining
Generated straight-line writers initially lost ~24% to the old engine on aarch64 number-heavy record shapes. The cause is an AArch64 C2 effect: when small hot jackson-core methods (
NumberOutput.outputInt/outputLong,UTF8JsonGenerator.writeName) inline into a looping writer body, the register allocator spills working state across the loop. The generator now emits per-kind static write helpers sized past the C2 inline threshold, so those copies compile standalone; with that change the generated writers beat the old engine on the regressing shape too (+9% aarch64 record writes, +18% media writes, and the run-to-run bimodality is gone; JDK 25 and 26). The helper-based build was revalidated on x86_64: it wins or ties every cell there as well (media +9% over the old engine, all rounds; order +4% median).Compatibility notes
--release 25; the ClassFile API is final in 24, and 25 is the LTS). Classic Blackbird continues to serve older JDKs, but otherwise is considered retired.Removed dead code
After both engines were replaced,
CrossLoaderAccesssurvived only as an unused constructor parameter and theutilchain (ReflectionHack,Unchecked,Sneaky,CheckedFunction,CheckedSupplier) had no callers.This PR deletes them (-504 lines) and closes #350, which scheduled the CrossLoaderAccess removal for 3.3 (deprecated
forRemovalsince 3.2).Appendix: examples of generated code
Each engaged bean gets one hidden class per direction, generated with the
ClassFile API and defined through
defineHiddenClassWithClassData. The classextends
GeneratedReaderBase(deserializer) orGeneratedWriterBase(serializer) and receives its constants - the property-name matcher, the
stock
SettableBeanProperty/PropertyWriterfallbacks per property, childcodecs, pre-encoded names, and the member-access and construction
MethodHandles - through a name-addressed classData map. The decompilerrenders each entry as the pseudocode
classDataEntry<"idProp">(): in theclass file it is a named dynamic constant (condy), resolved once and
constant-foldable by the JIT, not a per-call lookup.
Member access and construction go through
MethodHandleconstants witherased descriptors: every reference type is widened to
Object, primitiveskept, so the generated bytecode never names the bean class. A setter reads as
classDataEntry<"mediaSet">().invokeExact((Object)bean, (Object)value), aconstructor as
classDataEntry<"constructor">().invokeExact(...), a getter as(int)classDataEntry<"widthGet">().invokeExact((Object)value). The handlesare unreflected with the module's own lookup after databind has run
fixAccesson the members, so this reaches exactly what stock databind caninvoke - private members, non-public classes, records, foreign classloaders -
and every codec defines in the module's own package context regardless of the
bean's module or loader. A condy
MethodHandleplusinvokeExactinlineslike a direct call, so the erased form costs nothing at steady state.
Generated methods carry parameter names and a LocalVariableTable, so
decompiled bodies read with source-like names; reference record components and
the bean receiver appear as
Objectbecause their slots are erased. Views: abean that declares
@JsonViewresolves a cached per-view visibility bitmaskat entry and each property arm tests its bit, so view-active calls stay on the
generated path; the beans below declare none, so they show only the entry
guard, which hands non-
START_OBJECTentries - and, where a view couldmatter, view-active calls - to the stock fallback.
Two decompiler artifacts to read past: the bytecode's entry guard branches
forward to a delegation tail after the body (early-out shape), but the
decompiler renders it as a nested conditional; and the per-arm exception
regions all share one handler (a topology javac never emits), which the
decompiler expresses with synthetic
boolean varNN = falserouting flags andits "$VF: Inserted dummy exception handlers" banner - the class file's
handler is a plain four-instruction rethrow through
_propertyException,with no flags and no dead stores.
The examples below are decompiled with Vineflower 1.10.1 from bytes dumped by
the module itself (
-Dblackbird.debug.dumpDir, see Reproducing below), at thefollow-up-wave HEAD, and are unedited except for the marked elisions. The
shapes come from the benchmark corpus.
Record deserializer -
Order(nested records, child codec, list property; erased construction handle; shown in full)The bean:
The generated reader (
bbnext.model.Order-reader.class): components readinto typed locals under their component names (reference components as
Object, since the constructor handle is erased) and the canonicalconstructor is invoked once through
classDataEntry<"constructor">; eachscalar arm checks the expected token and demotes anything else to the stock
property; the nested
customercalls the child codec only forSTART_OBJECT;the
lineslist rides its stock property;seentracks components forrequired-property and
FAIL_ON_MISSING_CREATOR_PROPERTIESreporting, and anull mask feeds
FAIL_ON_NULL_CREATOR_PROPERTIES; unknown names go through_handleUnknown; a property failure is rethrown with the stock reference pathvia
_propertyException; anything butSTART_OBJECTat entry, or an activeview, delegates to the stock deserializer.
Setter-POJO deserializer -
MediaContent(erased constructor and setter handles, a child codec, a stock list arm; shown in full)The generated reader constructs through the no-arg constructor handle
(
classDataEntry<"constructor">().invokeExact()returningObject), sets thenested
mediathrough an erased setter handle(
classDataEntry<"mediaSet">().invokeExact((Object)bean, (Object)value)) whenthe value is a
START_OBJECTthe child codec can read, and rides the stockproperty for the
imageslist. The bean local isObject; no bean-class nameappears.
Field-backed properties differ only in the store handle: a non-final public,
package-private, or private field is written through
classDataEntry<"...Set">().invokeExact((Object)bean, (Object)value)exactlyas a setter is (the handle is
Lookup.unreflectSetterinstead ofunreflect), so the arm shape is identical and no separate example is shown.Final fields keep the stock property.
Serializer -
Media(scalar-rich writer, erased getter handles and padded helpers; shown in full, plus the bytecode view of one helper)The generated writer: a straight-line sequence of per-kind helper calls with
pre-encoded name constants; each property value loads through an erased getter
handle (
(int)classDataEntry<"bitrateGet">().invokeExact((Object)value)),nested/list properties ride their stock writers. Every helper is padded past
the C2 inline threshold so its jackson-core hot calls compile as standalone
units (the AArch64 register-allocator finding in the PR text); decompilers
elide the padding, so the bytecode view below shows it.
The
$inthelper as bytecode - the 384 leadingnops are the inline-thresholdpadding:
Polymorphic-subtype serializer -
PolyEvent.MediaEvent(nativeserializeWithTypewith the WritableTypeId flow)A subtype of an
@JsonTypeInfohierarchy gets bothserializeand a nativeserializeWithTypethat replicates stockBeanSerializerBase's flow exactly:typeSer.typeId(value, START_OBJECT),writeTypePrefix,assignCurrentValue,the same property sequence as
serialize, thenwriteTypeSuffix. Databindcalls
serializeWithTypeon the subtype's serializer for every inclusionmechanism (PROPERTY, WRAPPER_OBJECT, WRAPPER_ARRAY, ...), so the type wrapper
is written by the
TypeSerializerand the body stays generated. An activeview, or a bean with a
@JsonTypeIdproperty, delegates to the stock path.Only
serializeWithTypeis shown;serializeis the same body without theprefix/suffix pair.
Reproducing
Run any Jackson 3 application with the module registered and
-Dblackbird.debug.dumpDir=/some/dir: every generated reader and writer is written there as<bean-class>-reader.class/<bean-class>-writer.classbefore definition, one file per bean, ready forjavapor any decompiler.-Dblackbird.debug.codegen=trueadditionally traces generation and gate decisions to stderr. The examples above were decompiled with Vineflower 1.10.1.