Repository navigation
Conversation
SchemaDefinitionsInjectionTransformer injected a deep copy of all of components.schemas into every schema the validator compiles. Compiled schemas are cached, so each cache entry kept its own copy of the whole definitions block and the retained heap grew with the number of schemas validated times the size of the specification. It now injects only the definitions the schema can reach: the ones it refers to and the ones those refer to in turn. A discriminator mapping counts as a reference, with or without the #/components/schemas/ path. Each injected definition is still a deep copy, and the transformation context still carries the full set. On a specification with 117 operations and 479 component schemas, validating every operation once retained 1917 MB before and 109 MB after, and the first validation of an operation went from 86 ms to 14 ms. Validation reports are unchanged. Fixes atlassian#42
|
Hooray! All contributors have signed the CLA. |
Author
|
@jfnavin could you take a look when you have a moment? The CLA is signed and the check is green. Short version: |
A reference can point inside a definition (#/components/schemas/Order/properties/customer) and can percent-encode or JSON Pointer escape the definition name (My%20Type, a~1b). Both resolved before and were skipped by the reachability walk, so the definition was not injected and the reference failed to resolve. Also iterate the definitions with properties() instead of the deprecated fields().
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #42.
What
SchemaDefinitionsInjectionTransformerinjected adeepCopyof all ofcomponents.schemasinto every schema the validator compiles. Compiled schemas are cached, so each cache entry kept its own copy of the whole definitions block and the retained heap grew with (schemas validated) x (size of the specification).It now injects only the definitions the schema can reach: the ones it refers to, and the ones those refer to in turn. A discriminator
mappingcounts as a reference, with or without the#/components/schemas/path. Each injected definition is still a deep copy, so nothing shared can be mutated. The transformation context still carries the full set, soDiscriminatorMappingTransformerbehaves as before.Why it is safe
A definition that no reference reaches cannot take part in validating that schema: the JSON schema library only ever looks one up to resolve a
$ref. The injected block is listed in declaration order, so it does not depend on traversal order.Results
A large OpenAPI 3.0 specification (117 operations, 479 component schemas), one request and one response validated per operation, live heap after a full GC:
masterGenerated specification from the issue (400 schemas, 100 operations): 288 MB before, 15 MB after.
Validation reports before and after, on ten real specifications, every operation and every declared response code with four bodies each: 15,192 reports, byte for byte identical, about 97% of them carrying validation errors.
Tests
SchemaDefinitionsInjectionTransformerTest: nothing injected when nothing is referenced, transitive references, a self-referencing definition, discriminator mappings with and without the path, and that injected definitions are copies.mvn clean verify javadoc:javadocpasses.Not in this change
SchemaValidator.readAndTransformSchemaObjectstill makes its owndeepCopyof all definitions for the transformation context on every compile. That copy is short lived, so it costs time and not retained memory. It could go in a follow-up, since the transformer now copies whatever it injects.