Conversation
…tation "Compliance with industry provenance standards is ensured by the DataJoint Platform" stood unqualified. Ensured overstates it: what an exported account can carry depends on what was captured, and three cases carry less. Data populated outside the platform runs against library defaults, where JobsSettings ships keep_completed, add_job_metadata and version_method all off, so there is no job metadata and no commit identity. Tables declared before add_job_metadata was enabled carry none either, since enabling it does not backfill. And manually entered rows record no attribution at all — who entered a datum and when is not captured today. Rewritten to say the Platform supports compliance for the computation it manages, and that compliance is a property of the deployment rather than of the format. The surrounding claims are unchanged: lineage lives in DataJoint's schema, maps to OpenLineage or W3C PROV, and integrates in both directions when explicitly configured. Raised in review of platform-roadmap#228, whose readiness section documents the three cases.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Scopes one sentence in
comparison-to-provenance-systems.md. It read:Unqualified, on a public page. Ensured overstates what the Platform can promise, because what an exported account carries depends on what was captured — and three cases capture less.
Why
Raised in review of platform-roadmap#228, a feature brief for provenance export whose readiness section documents the gaps. Verified against
datajoint-pythonorigin/master(v2.3.3):JobsSettings(src/datajoint/settings.py:285, env prefixDJ_JOBS_) shipskeep_completed=False,add_job_metadata=False, andversion_method=None. A laptop run against library defaults produces no job metadata and no commit identity.add_job_metadatadoes not backfill — per its own description, "tables created without this setting will not receive metadata updates during populate."src/datajoint/migrate.py:632exists for that case.All three are things a provenance standard asks for. The claim as written would be read by a regulated buyer as a guarantee across the deployment.
What changed
One sentence becomes a short paragraph: the Platform supports compliance for the computation it manages, what an account carries depends on what was captured, and compliance is a property of the deployment rather than of the format.
Everything around it is unchanged — lineage lives in DataJoint's own schema, maps to OpenLineage or W3C PROV, and integrates in both directions when explicitly configured.
Not in this PR
The three capture gaps themselves. Two have platform-side remedies in flight (
add_job_metadata_columnsfor the migration, platform-roadmap#226 for manual attribution); this PR only stops the page promising more than they currently deliver.