Skip to content

perf: inject only the schema definitions a compiled schema can reach - #43

Open
vinirib wants to merge 2 commits into
atlassian:masterfrom
vinirib:perf/inject-only-referenced-definitions
Open

vinirib wants to merge 2 commits into
atlassian:masterfrom
vinirib:perf/inject-only-referenced-definitions

Conversation

@vinirib

@vinirib vinirib commented Nov 10, 2026

Copy link
Copy Markdown

Fixes #42.

What

SchemaDefinitionsInjectionTransformer injected a deepCopy of all of components.schemas into every schema the validator compiles. Compiled schemas are cached, so each cache entry kept its own copy of the whole definitions block and the retained heap grew with (schemas validated) x (size of the specification).

It now injects only the definitions the schema can reach: the ones it refers to, and the ones those refer to in turn. A discriminator mapping counts as a reference, with or without the #/components/schemas/ path. Each injected definition is still a deep copy, so nothing shared can be mutated. The transformation context still carries the full set, so DiscriminatorMappingTransformer behaves as before.

Why it is safe

A definition that no reference reaches cannot take part in validating that schema: the JSON schema library only ever looks one up to resolve a $ref. The injected block is listed in declaration order, so it does not depend on traversal order.

Results

A large OpenAPI 3.0 specification (117 operations, 479 component schemas), one request and one response validated per operation, live heap after a full GC:

Live heap, every operation First validation of an operation
master 1917 MB 86 ms
this change 109 MB 14 ms

Generated specification from the issue (400 schemas, 100 operations): 288 MB before, 15 MB after.

Validation reports before and after, on ten real specifications, every operation and every declared response code with four bodies each: 15,192 reports, byte for byte identical, about 97% of them carrying validation errors.

Tests

  • New SchemaDefinitionsInjectionTransformerTest: nothing injected when nothing is referenced, transitive references, a self-referencing definition, discriminator mappings with and without the path, and that injected definitions are copies.
  • mvn clean verify javadoc:javadoc passes.

Not in this change

SchemaValidator.readAndTransformSchemaObject still makes its own deepCopy of all definitions for the transformation context on every compile. That copy is short lived, so it costs time and not retained memory. It could go in a follow-up, since the transformer now copies whatever it injects.

SchemaDefinitionsInjectionTransformer injected a deep copy of all of
components.schemas into every schema the validator compiles. Compiled schemas
are cached, so each cache entry kept its own copy of the whole definitions
block and the retained heap grew with the number of schemas validated times
the size of the specification.

It now injects only the definitions the schema can reach: the ones it refers
to and the ones those refer to in turn. A discriminator mapping counts as a
reference, with or without the #/components/schemas/ path. Each injected
definition is still a deep copy, and the transformation context still carries
the full set.

On a specification with 117 operations and 479 component schemas, validating
every operation once retained 1917 MB before and 109 MB after, and the first
validation of an operation went from 86 ms to 14 ms. Validation reports are
unchanged.

Fixes atlassian#42
@atlassian-cla-bot

atlassian-cla-bot Bot commented Nov 10, 2026 •

Copy link
Copy Markdown

Hooray! All contributors have signed the CLA.

@vinirib

vinirib commented Nov 11, 2026

Copy link
Copy Markdown
Author

@jfnavin could you take a look when you have a moment? The CLA is signed and the check is green.

Short version: SchemaDefinitionsInjectionTransformer copies all of components.schemas into every compiled schema, and compiled schemas are cached. On a spec with 117 operations and 479 schemas, validating each operation once retains 1917 MB on master and 109 MB with this patch. Validation reports are identical on both. Details and a reproducer are in #42.

A reference can point inside a definition
(#/components/schemas/Order/properties/customer) and can percent-encode or
JSON Pointer escape the definition name (My%20Type, a~1b). Both resolved
before and were skipped by the reachability walk, so the definition was not
injected and the reference failed to resolve.

Also iterate the definitions with properties() instead of the deprecated
fields().
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Compiled schemas retain a full copy of components.schemas each: heap grows with (schemas validated) x (spec size)

1 participant