Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
94 commits
Select commit Hold shift + click to select a range
cfc64cd
docs: correct the wave-2 memo's refs-only OPST claim
bplatz Sep 30, 2026
4188dd7
feat(core): add the RDF 1.2 triple-term value kind
bplatz Sep 30, 2026
27783f8
feat(index): triple-term dictionary, interned at bulk import
bplatz Sep 30, 2026
3d8a85d
feat(index): synthesize triple-term links on rebuild and incremental …
bplatz Sep 30, 2026
d674e49
feat(query): lower reified-triple patterns to the rdf:reifies link
bplatz Sep 30, 2026
26b0d88
feat(query): encoded term components and term-identity constraints on…
bplatz Sep 30, 2026
270984e
fix(index): replay attachment ops per reifier instead of assembling b…
bplatz Sep 30, 2026
9b3980b
perf(query): elide unread reified-edge positions before the fast path…
bplatz Sep 30, 2026
203cc3e
fix(query): term accessors keep the object's datatype on every path
bplatz Sep 30, 2026
8e6daf0
fix(index): fail the build on a base attachment read error; stream li…
bplatz Sep 30, 2026
a2d1f99
fix(query): rdf:reifies takes a triple term, on write and on scan
bplatz Sep 30, 2026
9f38bf2
perf(index): batch base lookups when replaying attachments incrementally
bplatz Sep 30, 2026
5014bf5
chore(core): artifact read accounting behind FLUREE_IO_STATS
bplatz Sep 30, 2026
c308c54
test(index): count the triple-term entry in the built-in o_type table
bplatz Oct 1, 2026
326e455
perf(index): resolve base class Sids in one batched pass on increment…
bplatz Oct 1, 2026
d7760a1
perf(api): read the index root once when loading a ledger
bplatz Oct 1, 2026
b899450
perf(index): stop storing the ledger-wide class table twice in the root
bplatz Oct 1, 2026
af58b71
perf(index): store per-graph class tables in a compact stats tail
bplatz Oct 1, 2026
46e5b15
perf(api): decode the index root once when loading a ledger
bplatz Oct 1, 2026
ef50fb0
perf(index): compact triple-term pack streams on incremental builds
bplatz Oct 1, 2026
e8356ab
perf(index): prewarm triple-term packs with the other forward diction…
bplatz Oct 1, 2026
5f7b284
fix(query): keep a pending bind's target column alive through join steps
bplatz Oct 1, 2026
a4ce766
fix(transact): refuse rdf:reifies with a non-term object after resolv…
bplatz Oct 1, 2026
8711a81
fix(query): term accessors keep the object's datatype for any argument
bplatz Oct 1, 2026
fc67ac3
fix(index): give term pack streams a budget share and a maintenance s…
bplatz Oct 1, 2026
60a2892
fix(index): a dictionary range starting between two leaves reads the …
bplatz Oct 1, 2026
15995e5
feat(query): relate a triple term to its components as a joinable pat…
bplatz Oct 1, 2026
da50cf0
feat(novelty): derive reification links for commits the index has not…
bplatz Oct 1, 2026
82e4d26
feat(query): provisional handles for terms only novelty holds
bplatz Oct 1, 2026
39b9286
perf(query): bracket the novelty walk of a scan bound to a triple term
bplatz Oct 1, 2026
c66dd63
fix(query): a BIND's agreement check decodes arena-keyed numbers
bplatz Oct 1, 2026
99fb4bf
feat(index): link annotations whose object is a decimal, big integer …
bplatz Oct 1, 2026
8c72176
feat(index): live link counts per inner predicate, for the planner
bplatz Oct 2, 2026
2d56587
fix(index): the first terms over an index without a term dictionary
bplatz Oct 2, 2026
bc919ac
perf(query): read an object-bound triple term from an object-first tree
bplatz Oct 2, 2026
7f2a2b4
feat(query): SPARQL 1.2 TRIPLE() and triple-term expressions
bplatz Oct 2, 2026
003d7d8
fix(ledger): a staged view reads the links of its own annotations
bplatz Oct 2, 2026
e978ccf
feat(api): render triple terms in every result format
bplatz Oct 2, 2026
1069d4d
fix(api): keep rdf:reifies out of the Cypher catalog procedures
bplatz Oct 2, 2026
891c398
fix(api): export no longer writes the index's rdf:reifies links
bplatz Oct 2, 2026
f3af076
fix(query): a link is visible only when the triple it names is
bplatz Oct 2, 2026
992d453
feat(query): reified-triple patterns always read the rdf:reifies link
bplatz Oct 2, 2026
ef5ec8d
feat(query): triple-term functions in JSON-LD queries
bplatz Oct 2, 2026
b550342
fix(import): annotations in TriG GRAPH blocks get their links
bplatz Oct 2, 2026
731dad9
fix(query): a novelty-assigned id unifies with the term it names
bplatz Oct 2, 2026
891abee
feat(query): annotation syntax reads the rdf:reifies link
bplatz Oct 2, 2026
988e669
perf(query): an annotated edge is not re-checked against its base triple
bplatz Oct 3, 2026
ec9d5cd
refactor(query): remove the bundle-chain read lanes
bplatz Oct 3, 2026
0a25dd4
fix(import): JSON-LD annotations get their links at bulk import
bplatz Oct 3, 2026
044c7f8
feat(core): commits carry triple terms
bplatz Oct 3, 2026
1558138
feat(transact): write templates take triple terms
bplatz Oct 3, 2026
0fc2c67
fix(index): links written into commits index correctly
bplatz Oct 3, 2026
5e84638
feat: annotations store the rdf:reifies link
bplatz Oct 3, 2026
074c699
refactor: retire the annotation arena
bplatz Oct 3, 2026
be086a1
feat(query): rdf:reifies links are ordinary data in wildcard reads
bplatz Oct 3, 2026
653f66c
feat: reified triples no longer assert their triple (RDF 1.2)
bplatz Oct 3, 2026
8e53cb0
feat(transact): deleting a triple leaves its reifiers outside LPG mode
bplatz Oct 3, 2026
bc25ceb
feat(sparql): UPDATE takes reified triples and named-graph annotations
bplatz Oct 3, 2026
549a5ed
feat: triple terms are values under any predicate (RDF 1.2)
bplatz Oct 3, 2026
c701026
feat(sparql): CONSTRUCT templates take reified triples
bplatz Oct 3, 2026
00f9749
feat: nested triple terms
bplatz Oct 3, 2026
96116f1
fix(transact): INSERT templates write built and graph-name IRIs
bplatz Oct 3, 2026
68509b6
perf(query): an annotated edge checks its triple instead of joining it
bplatz Oct 3, 2026
665f89d
perf(query): link probes bound by a term handle run in bulk
bplatz Oct 4, 2026
0de62a3
perf(query): batch wildcard predicate joins with shared streaming loo…
bplatz Sep 14, 2026
d70ff69
perf(query): a semijoin seeds its build from a small outer side
bplatz Oct 4, 2026
b91e779
perf(query): batched subject probes skip leaflets by their directory …
bplatz Oct 4, 2026
42b5191
perf(query): reuse independent scans and count Cartesian joins
bplatz Sep 14, 2026
088da2c
fix: TriG default-graph blocks, escaped local names and canonical N-T…
bplatz Oct 4, 2026
196d7c4
test(w3c): run the RDF 1.2 N-Triples, N-Quads and TriG suites
bplatz Oct 4, 2026
0ac086e
test(api): serialize the tests that flip the fast-path switch
bplatz Oct 4, 2026
f277ae4
feat: CONSTRUCT templates write triple terms under any predicate
bplatz Oct 4, 2026
10741a9
Merge remote-tracking branch 'origin/main' into feat/annotation-tripl…
bplatz Oct 4, 2026
870d431
test(api): the include-system-facts pragma twin reads the rdf:reifies…
bplatz Oct 5, 2026
9acf941
chore(bench): drop the budgets of the benches the arena retirement re…
bplatz Oct 5, 2026
9f2ea30
Merge remote-tracking branch 'origin/main' into feat/annotation-tripl…
bplatz Oct 6, 2026
54a6857
fix(policy): lanes decline predicates that may hold triple terms
bplatz Oct 6, 2026
c7a5977
fix(api): export and hydration refuse a pre-link index like queries do
bplatz Oct 6, 2026
2b23f9e
perf(query): seed the semijoin per chunk; build first for a large out…
bplatz Oct 6, 2026
441d047
perf(export): read the link set without cloning it twice
bplatz Oct 6, 2026
5cbbeed
perf(query): OPTIONAL annotations keep the batched hash-join lane
bplatz Oct 6, 2026
44b3ac9
feat(query): triple terms in JSON-LD values cells
bplatz Oct 6, 2026
18040ba
bench: annotation link queries, with an outer side past one seeded chunk
bplatz Oct 6, 2026
da463d7
docs: re-asserted edges, lane policy and pre-link refusal; drop EdgeK…
bplatz Oct 6, 2026
74ca745
perf(export): bound annotation memory past a million links
bplatz Oct 6, 2026
1b24482
fix(query): without estimates, seed only an outer side one chunk holds
bplatz Oct 6, 2026
8fa018a
fix(policy): check a triple term before the schema exemption
bplatz Oct 6, 2026
71a2266
fix(query): a triple term is not a literal
bplatz Oct 6, 2026
aaf4903
fix(query): TermComponents streams within batch size and the budget
bplatz Oct 6, 2026
cb78860
fix(export): count novelty's links against the preload bound
bplatz Oct 6, 2026
efce5db
refactor(query): one triple-term object conversion, Binding::term_object
bplatz Oct 6, 2026
ba480bc
refactor(json-ld): share the triple-term shape check across surfaces
bplatz Oct 6, 2026
eeaa3c2
fix(policy): no store, no triple-term decline in the lane gate
bplatz Oct 6, 2026
583c4db
feat: say when an annotated ledger's index predates links
bplatz Oct 6, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
2 changes: 0 additions & 2 deletions BENCHMARKING.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,8 +86,6 @@ this table, so it is the one that rots. Add a row when you add a bench file.
| `fluree-db-api` | `query_hot_property_path.rs` | Hot-cache SPARQL property paths, one scenario per execution mode of the operator (`*` closure, sequence, `?`, …) |
| `fluree-db-api` | `query_hot_whole_graph_agg.rs` | Cypher aggregate folds from `fast_whole_graph_agg` (whole-graph + class scalars, histograms) against a linear-cost pipeline baseline |
| `fluree-db-api` | `query_overlay_matrix.rs` | The same query shapes at four ledger conditions — base / overlay / cached / novelty — so the columnar+novelty merge lane and a cached handle's steady state after background indexing have coverage |
| `fluree-db-api` | `annotation_hydration.rs` | `inject_annotations` hydration cost: index scan vs sealed annotation arena |
| `fluree-db-api` | `annotation_planner.rs` | Planner direction for `f:reifies*` edge-annotation queries: arena-informed row counts vs HLL-only stats |
| `fluree-db-query` | `vector_math.rs` | SIMD vs scalar dot/L2/cosine micro-bench |
| `fluree-db-spatial` | `spatial_bench.rs` | S2 covering build + within/intersects/radius latency |

Expand Down
1 change: 0 additions & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

10 changes: 7 additions & 3 deletions docs/audit/burn-down/sparql12-wave2-triple-terms.md
Original file line number Diff line number Diff line change
Expand Up @@ -349,9 +349,13 @@ schedule the eval epic. Verified against the core:
triple term a single `Sid`/`u64` arena handle inside `FlakeValue`/`Binding` so
the row width and cache footprint of the scan/join path don't grow. Guard with
`query_hot_bsbm` / `query_hot_bsbm_bi`.
- **`Opst` (object-leading index) is documented refs-only**
(`comparator.rs:30,79-83`). Looking up a triple-term object by value needs
index-selection work too.
- **`Opst` (object-leading index) holds every object type**, segmented by
`o_type` (`fluree-db-core/src/comparator.rs`, module doc). `IndexType::for_query`
gates OPST on `o_is_ref` as a conservative default only; `BinaryScanOperator`
already takes OPST for any constant object. A triple-term object therefore
needs its own `o_type` partition and index-selection work, not a refs-only
exception. (An earlier revision of this memo called OPST refs-only; that was
stale.)

**Net:** comparator functions are largely safe (raw-byte compare); the cost and
risk are in the **enum/encoder/decoder/hash/Display arms + a new arena + enum
Expand Down
16 changes: 7 additions & 9 deletions docs/cli/export.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ fluree export [LEDGER] [OPTIONS]
| `--all-graphs` | Export the default graph plus every named graph (dataset export). Requires `--format trig` or `--format nquads`. The ledger's system graphs are excluded — see `--system-graphs`. |
| `--system-graphs` | Also emit the ledger's system graphs (`#txn-meta`, `#config`) under `--all-graphs`. Diagnostic only. |
| `--graph <IRI>` | Export a specific named graph by IRI. Mutually exclusive with `--all-graphs`. |
| `--raw-reifies` | Emit edge annotations as raw `f:reifies*` system triples instead of RDF 1.2 annotation syntax (pre-4.2 output). |
| `--raw-reifies` | Write each edge annotation as its stored link, `r rdf:reifies <<( s p o )>>`, instead of annotation syntax on the base edge. JSON-LD keeps `@annotation`. |
| `--context <JSON>` | JSON-LD context for prefix declarations. Overrides the ledger's default context. |
| `--context-file <FILE>` | Read context from a JSON file. Overrides the ledger's default context. |
| `--at <TIME>` | Export data as of a specific point in time. `t:<N>` (transaction number), `t:latest` or `latest`, `time:<ISO-8601>` (commit event time), `recorded:<ISO-8601>` (the wall-clock time the commit was recorded), or `commit:<hex-prefix>`. A bare transaction number, ISO-8601 timestamp or commit prefix also works; a commit prefix must be at least 6 characters in either spelling; a bare integer is read as a transaction number, so use `commit:<prefix>` to force an all-digit prefix. If omitted, exports at the latest committed time (including data committed but not yet persisted to index). |
Expand Down Expand Up @@ -61,9 +61,9 @@ Every ledger has two system graphs, `urn:fluree:<ledger>:main#txn-meta` (commit

### Edge annotations (RDF 1.2)

An edge annotation attaches a reifier to one specific triple: `ex:alice ex:knows ex:bob ~ ex:claim1 {| ex:confidence 0.8 |}`. Fluree stores that as seven reserved `f:reifies*` system facts plus the reifier's own properties. Export used to emit those system facts verbatim — output no other RDF 1.2 tool understands, and which Fluree's own insert and update surfaces reject as system-controlled predicates.
An edge annotation attaches a reifier to one specific triple: `ex:alice ex:knows ex:bob ~ ex:claim1 {| ex:confidence 0.8 |}`. Fluree stores that as the RDF 1.2 link `ex:claim1 rdf:reifies <<( ex:alice ex:knows ex:bob )>>` plus the reifier's own properties.

Export now emits annotation syntax by default:
Export emits annotation syntax by default:

| Format | Output |
|---|---|
Expand All @@ -80,19 +80,17 @@ ex:claim1
ex:confidence "0.8"^^xsd:decimal .
```

`--raw-reifies` restores the pre-4.2 output. That output only re-imports through `fluree create --from`; the insert and update surfaces reject hand-written `f:reifies*` triples.
`--raw-reifies` writes the links as triples instead — `ex:claim1 rdf:reifies <<( ex:alice ex:knows ex:bob )>> .` in every RDF format — which also re-imports identically.

A ledger that has never carried an annotation pays nothing for any of this: export reads one flag on the snapshot and runs the scan it always ran.
A ledger that has never carried an annotation pays nothing for any of this: export reads one flag on the snapshot and runs the scan it always ran. One that has reads its links once, up front, so the markers can be written as the scan reaches each base edge.

**Known limit.** Annotations written *inside a named graph* are resolved correctly when export reads them from the sealed annotation arena or from the novelty overlay. They are dropped when export falls back to the **base-index scan**, which is blind to them — the fallback taken by a ledger whose index reports annotations but for which no arena was sealed. SPARQL reads them in every case. Export says so rather than dropping them quietly:
Export reports a link it suppressed for annotation syntax without writing the matching `~ <r>` marker, rather than dropping it quietly:

```
warning: 1 edge annotations could not be resolved and are NOT in the output;
re-run with --raw-reifies to emit them as f:reifies* triples
re-run with --raw-reifies to emit them as rdf:reifies triples
```

`FLUREE_EXPORT_ANNOTATION_SCAN=1` forces the base-index scan in place of the sealed annotation arena — how to compare the two sources without rebuilding an index. They agree except on annotations inside a named graph, which the arena resolves and the scan cannot see; forcing the scan on such a ledger produces the warning above.

### Prefixes / Context

Turtle, TriG, and JSON-LD output use prefix compaction to produce compact, readable output. The prefix map is resolved in this order:
Expand Down
1 change: 1 addition & 0 deletions docs/cli/info.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ For ledgers, displays:
- Ledger ID, branch, and type
- Current transaction number (t)
- Commit and index details
- A warning when the ledger holds edge annotations and its index was built by a release before RDF 1.2 triple-term links: those annotations cannot be read until [`fluree reindex`](reindex.md). The `ledger-info` JSON carries `"needs-link-reindex": true` in its `ledger` block for the same case.

For graph sources (Iceberg, R2RML, BM25, etc.), displays:
- Name, branch, and type
Expand Down
6 changes: 3 additions & 3 deletions docs/cli/server-integration.md
Original file line number Diff line number Diff line change
Expand Up @@ -1099,7 +1099,7 @@ non-zero, so a clean export carries none of them.
| Header | Meaning |
|--------|---------|
| `x-fluree-export-named-graphs-omitted` | User-visible named graphs the ledger holds that this export did not cover — set when a dataset format ran without `all_graphs`. |
| `x-fluree-export-annotations-unresolved` | Edge annotations that could not be represented and are **not** in the body. Re-request with `raw_reifies` to get them as `f:reifies*` triples. |
| `x-fluree-export-annotations-unresolved` | Edge annotations that could not be represented and are **not** in the body. Re-request with `raw_reifies` to get them as `rdf:reifies` triples. |
| `x-fluree-export-annotations-out-of-scope` | Reifiers the body names with a `~ <r>` marker whose own properties this export does not contain. The annotation is usable only against a wider export. |
| `x-fluree-export-rows-skipped` | Rows the writer could not represent (unresolvable predicate id, or a value that decoded to null). |

Expand Down Expand Up @@ -2111,14 +2111,14 @@ Content-Type: application/json
| `format` | string | No | `"turtle"` | One of: `turtle`/`ttl`, `ntriples`/`nt`, `nquads`/`n-quads`, `trig`, `jsonld`/`json-ld`/`json`. Case-insensitive. |
| `all_graphs` | bool | No | `false` | Export every user-visible named graph as a dataset. Requires `format` ∈ `trig` / `nquads`. Mutually exclusive with `graph`. The ledger's system graphs (`#txn-meta`, `#config`) are excluded. |
| `system_graphs` | bool | No | `false` | Also emit the system graphs under `all_graphs`. Diagnostic only — the result is named for the source ledger and does not re-import cleanly. |
| `raw_reifies` | bool | No | `false` | Emit edge annotations as raw `f:reifies*` triples instead of RDF 1.2 annotation syntax. The escape hatch the `x-fluree-export-annotations-unresolved` header points at, and the way to keep pre-4.2 bytes. |
| `raw_reifies` | bool | No | `false` | Write each edge annotation as its stored link, `r rdf:reifies <<( s p o )>>`, instead of annotation syntax on the base edge; JSON-LD keeps `@annotation`. The escape hatch the `x-fluree-export-annotations-unresolved` header points at. |
| `graph` | string | No | — | IRI of a single named graph to export. Mutually exclusive with `all_graphs`. |
| `context` | object | No | ledger default | Prefix map for Turtle/TriG/JSON-LD output. Either a bare object (`{ "ex": "..." }`) or `{ "@context": {...} }`. Falls back to the ledger's stored default context when absent. |
| `at` | string | No | latest | Time spec — `t:<N>` (transaction number), `t:latest` or `latest`, `time:<ISO-8601>` (commit event time; `iso:<ISO-8601>` is an alias, and the spelling the CLI sends), `recorded:<ISO-8601>` (the wall-clock time the commit was recorded), or `commit:<hex-prefix>`. A bare transaction number, ISO-8601 timestamp or commit prefix also works; a commit prefix must be at least 6 characters in either spelling; a bare integer is read as a transaction number, so use `commit:<prefix>` to force an all-digit prefix. Identical to the local `--at` flag. |

An empty body is accepted and treated as all-default (Turtle export at HEAD).

**Breaking change in 4.2.** Response bodies now carry RDF 1.2 annotation syntax for edge annotations — `s p o ~ <r>` in Turtle and TriG, a triple term under `rdf:reifies` in N-Triples and N-Quads, `@annotation` in JSON-LD — where previous versions emitted the underlying `f:reifies*` triples. A consumer that parsed those triples directly will not find them. Set `raw_reifies` to keep the old bytes.
**Breaking change in 4.2.** Response bodies now carry RDF 1.2 annotation syntax for edge annotations — `s p o ~ <r>` in Turtle and TriG, a triple term under `rdf:reifies` in N-Triples and N-Quads, `@annotation` in JSON-LD — where previous versions emitted the underlying `f:reifies*` triples. A consumer that parsed those triples directly will not find them; `raw_reifies` writes the `rdf:reifies` links as triples instead.

### Auth

Expand Down
Loading