Skip to content

Provenance v2 - #537

Merged
NicoLaval merged 86 commits into
developfrom
feat/provenance
Sep 7, 2026
Merged

Provenance v2#537
NicoLaval merged 86 commits into
developfrom
feat/provenance

Conversation

@hadrienk

Copy link
Copy Markdown
Collaborator

Mostly AI planning and test definition. Need review before going further.

hadrienk and others added 30 commits July 28, 2026 23:32
Extract identity assignment provenance
When ds[…][…] nests clauses, materialize #s{stmt}.{seq} between them so
each clause has a dataset source. Turns chain-filter-calc green.

Co-authored-by: Cursor <cursoragent@cursor.com>
Rename reserved id total to mea_sum and align the golden with the oracle
type of sum(integer) (NUMBER).

Co-authored-by: Cursor <cursoragent@cursor.com>
Emit expression nodes for aggregate functions, pass group-by keys through,
and link aggregated measures with op=aggr. Turns 07 green.

Co-authored-by: Cursor <cursoragent@cursor.com>
Note reserved-id pitfall for the 07 corpus case.

Co-authored-by: Cursor <cursoragent@cursor.com>
OVER belongs inside the analytic call parentheses in the Trevas grammar.

Co-authored-by: Cursor <cursoragent@cursor.com>
Provenance can lead unimplemented engine ops (set/analytic); keep input
bindings and let the visitor derive output structures.

Co-authored-by: Cursor <cursoragent@cursor.com>
Origin-aware component links for identifiers shared across operands;
measures/attributes only where present. Turns 08-join green.

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
union/intersect/symdiff are component-wise; setdiff treats the right
operand as a condition. Turns 09–11 green when the engine cannot eval.

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@NicoLaval NicoLaval changed the title PProvenance v2 Provenance v2 Sep 6, 2026
NicoLaval and others added 24 commits September 6, 2026 10:10
Introduce shared unary PassThrough (hierarchy, time-series, eval) and
align check errorlevel golden with Trevas String default.

Co-authored-by: Cursor <cursoragent@cursor.com>
Permit check, hierarchy, time-series producers, exists_in and eval at
dataset level; keep scalar time ops calc-only via a single dispatch.

Co-authored-by: Cursor <cursoragent@cursor.com>
Emit lineage for check, hierarchy/time-series PassThrough, exists_in,
eval, and kind=scalar assignments with reference-level expression nodes.

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
BPE chains calc/drop on inner_join(…); requireDatasetOrClause now
accepts JoinFunctions as the clause left operand.

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Second view over ProvGraph: value dependsOn → sdth:wasDerivedFrom,
condition → prov:used, with expression and scalar nodes. Leaves
SdthProgramView + RDFUtils unchanged.

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Support-check registers formals + body expr for call-site inlining.

Co-authored-by: Cursor <cursoragent@cursor.com>
Map formals to varID call args from the registered body; drop black-box
op=<operatorName> edges on the call-site expression node.

Co-authored-by: Cursor <cursoragent@cursor.com>
Align 17-udf; add 39-udf-inline (unused formal omitted from lineage).

Co-authored-by: Cursor <cursoragent@cursor.com>
Move fixture InputDataset/InputDirectives into a dependency-light module so harnesses outside provenance can reuse them.

Co-authored-by: Cursor <cursoragent@cursor.com>
Drop the prov2 copies and point extractor, visitor, oracle, and harness tests at fr.insee.vtl.testutils.

Co-authored-by: Cursor <cursoragent@cursor.com>
Document that $input parsing lives in vtl-test-utils; Wave E complete.

Co-authored-by: Cursor <cursoragent@cursor.com>
Add FileInstance, wasDerivedFrom/elaborationOf, hasName/hasVarInstance,
step-only hasSourceCode, and JSON-LD-safe instance ids.

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Drop dead prov.model beans, AntlrUtils, and commented ProvenanceListener stubs.

Co-authored-by: Cursor <cursoragent@cursor.com>
Drop the temporary prov2 package name; keep the public Provenance façade and SDTH beans stable.

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@NicoLaval
NicoLaval marked this pull request as ready for review September 7, 2026 15:46
@NicoLaval
NicoLaval merged commit 2784fd1 into develop Sep 7, 2026
18 checks passed
@NicoLaval
NicoLaval deleted the feat/provenance branch September 7, 2026 15:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants