Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 10 additions & 9 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -1557,17 +1557,18 @@ cd crates/lance-graph-python && maturin develop
# (Rust crate line is 0.33.0 -> 0.37.1 -> 0.38.0; 0.34/0.35/0.36 do not exist as
# Rust crates -- those numbers are the independently-versioned PyPI package.)
#
# The lance family moves in EXACT lockstep -- currently the 13 line. Line of state:
# The lance family moves in EXACT lockstep -- currently the 14 line. Line of state:
# lance 7 (2026-06-14) -> 9 (the lance-9 sweep, b2b08b07 / PR #896 arc) -> 10
# (#1187, D-LNC-1) -> 11 (#1190, D-LNC-3, probe-gated on the D-LNC-2 fragment-id
# probe #1189) -> 12 (2026-09-25, lancedb 0.39) -> 13 (2026-10-10, lancedb 0.40,
# Rust 1.99; lance 13 needs object_store ^0.14.1). arrow/datafusion did NOT move
# Rust 1.99; lance 13 needs object_store ^0.14.1) -> 14 (2026-11-11, lancedb 0.41,
# lance =14.0.2; object_store ^0.14.1 unchanged). arrow/datafusion did NOT move
# with any of them. lance 12 DID move `object_store` 0.13 -> 0.14 while
# datafusion 54 stays on 0.13, so both are in the graph; our direct
# `object_store` follows lance (every call site hands it to lance).
# ⊘ The "=9.0.0 / lancedb =0.33.0" ruling recorded below as
# E-PIN-LANCE9-LANCEDB033-DF541-ARROW58-NO-DF53-1 is SUPERSEDED on its lance
# and lancedb coordinates ONLY (now 13 / 0.40.0); its datafusion half stands
# and lancedb coordinates ONLY (now 14 / 0.41.0); its datafusion half stands
# verbatim -- DF 54.1, never DF 53, and the `delta` feature stays removed.
# (A duplicated copy of this sentence stood here since the lance-9 sweep; both
# copies carried the stale number, so both are replaced by this one.)
Expand Down Expand Up @@ -1596,14 +1597,14 @@ datafusion = "54" # OUR direct pin, in every crate that DEPENDS on it
# 2026-08-18; crates.io releases still DF 53) — as its
# own deliberate PR if a consumer needs Delta.
# Probe: .claude/plans/lance9-datafusion54-upgrade-probe-v1.md
lance = "13.*" # lancedb 0.40 requires `=13.0.0`; we float the patch
lance-linalg = "13.*" # and resolve to whatever it demands (today 13.0.0)
lance-index = "13.*"
lancedb = "0.40.*" # the lance-13 pairing (`default-features = false` in
lance = "14.*" # lancedb 0.41 requires `=14.0.2`; we float the patch
lance-linalg = "14.*" # and resolve to whatever it demands (today 14.0.2)
lance-index = "14.*"
lancedb = "0.41.*" # the lance-14 pairing (`default-features = false` in
# the workspace table; consumers behind `lancedb-sdk`
# add `remote`, which 0.40.0 still needs to compile).
# add `remote`; 0.40.0 needed it to compile, not re-checked for 0.41).
# NOTE: the Rust crate line is 0.33.0 -> 0.37.1 ->
# 0.38.0 -> 0.39.0 -> 0.40.0; 0.34/0.35/0.36 exist only as the
# 0.38.0 -> 0.39.0 -> 0.40.0 -> 0.41.0; 0.34/0.35/0.36 exist only as the
# independently-versioned PyPI package.
rust = "1.99.0" # rust-toolchain.toml is authoritative; see its bump log
ndarray = { path = "../../../ndarray" } # AdaWorldAPI fork, default, optional fallback
Expand Down
12 changes: 6 additions & 6 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -359,9 +359,9 @@ resolver = "2"
# Crate-level pins (canonical source of truth — workspace.dependencies is not
# used; each consumer crate declares its own pin with exact-equals so the
# whole graph resolves to the same semver):
# lance = "13.*" (lance-arrow / lance-index / lance-linalg /
# lance = "14.*" (lance-arrow / lance-index / lance-linalg /
# lance-namespace bump in lock-step)
# lancedb = "0.40.*"
# lancedb = "0.41.*"
# datafusion = "54" (+ datafusion-common / -expr / -sql /
# -functions-aggregate)
# arrow-array = "58" (+ arrow-schema / -buffer / -ipc / arrow)
Expand Down Expand Up @@ -432,7 +432,7 @@ resolver = "2"
# fork of `aws-smithy-json` fixed against `aws-smithy-types 1.7` -- the two
# call sites need `.into()` and one wildcard arm; upstream's own compiler
# hints spell both out.
lance = { version = "13.*", default-features = false, features = [
lance = { version = "14.*", default-features = false, features = [
"azure",
"gcp",
"oss",
Expand All @@ -442,9 +442,9 @@ lance = { version = "13.*", default-features = false, features = [
"goosefs",
"geo",
] }
lance-linalg = "13.*"
lance-index = "13.*"
lancedb = { version = "0.40.*", default-features = false }
lance-linalg = "14.*"
lance-index = "14.*"
lancedb = { version = "0.41.*", default-features = false }
# ⊘ OPERATOR RULING 2026-09-18 — NEVER PIN TO `.0.0`; FLOAT THE PATCH.
# *"never pin to x.00, always float x.*"* / *"so no decimal .0.0"*. The
# `=X.Y.0` exact-equals these replaced are RETIRED. Floating the patch is
Expand Down
6 changes: 3 additions & 3 deletions crates/crepe-plan-probe/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -26,10 +26,10 @@ mork-pathmap-probe = { path = "../mork-pathmap-probe" }
# and the FacetCascade keys.
lance-graph-contract = { path = "../lance-graph-contract" }
# Stage 3 (D-HOLO-H14d): the relations live in a Lance table and are read
# under the address mask (`with_row_addr_prefilter`). lance 13 / lancedb 0.40
# under the address mask (`with_row_addr_prefilter`). lance 14 / lancedb 0.41
# move in lockstep, patch floated, as in the workspace manifest.
lance = { version = "13.*", default-features = false }
lancedb = { version = "0.40.*", default-features = false, features = ["remote"] }
lance = { version = "14.*", default-features = false }
lancedb = { version = "0.41.*", default-features = false, features = ["remote"] }
arrow-array = "58"
arrow-buffer = "58"
arrow-schema = "58"
Expand Down
6 changes: 3 additions & 3 deletions crates/crepe-plan-probe/examples/where_lance_arm.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
//! ClassView -> WideFieldMask (demanded property positions)
//! -> WHERE (logical predicate) / DR selection (identities)
//! -> Lance row-address mask (physical row eligibility)
//! -> Scanner::with_row_addr_prefilter -> Scanner::project (resolved columns only)
//! -> Scanner::with_row_id_prefilter -> Scanner::project (resolved columns only)
//! -> fold
//! ```
//!
Expand All @@ -24,7 +24,7 @@
//! - A FULL: one scan of where-columns + resolved demanded columns, WHERE in Rust.
//! - B MASKED PREFILTER: scan the where-columns with `_rowaddr`, WHERE -> row list,
//! build `RowAddrMask` (MASK INGRESS: this COPIES the addresses into Lance's
//! `RowAddrTreeMap`, it is not zero-copy), re-scan with `with_row_addr_prefilter`
//! `RowAddrTreeMap`, it is not zero-copy), re-scan with `with_row_id_prefilter`
//! projecting the resolved demanded columns only. ALLOW always; BLOCK as well at 50%.
//! - C ATTESTED ADDRESSED: `SealedFacetLane::attest_sorted` once per version (cold),
//! ids -> lane ranges with `seek_from`, `addressed::plan_read` under `EXACT` and
Expand Down Expand Up @@ -338,7 +338,7 @@ async fn scan(
s.with_row_address();
}
if let Some(m) = mask {
s.with_row_addr_prefilter(m);
s.with_row_id_prefilter(m);
}
s.scan_stats_callback(cb);
let b = s.try_into_batch().await.map_err(|e| e.to_string())?;
Expand Down
4 changes: 2 additions & 2 deletions crates/crepe-plan-probe/src/addressed.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
//!
//! The rows of one leading value `a` are then a contiguous range, located by
//! `SealedFacetLane::seek_from` under the lane's `OrderedLaneWitness`. A side
//! is read with `with_row_addr_prefilter` over the union of its ranges, so the
//! is read with `with_row_id_prefilter` over the union of its ranges, so the
//! rows Lance visits are the rows of the demanded keys and nothing else. No
//! value is translated to another coordinate space: the addresses are the
//! lane's own positions, in NodeGuid order.
Expand Down Expand Up @@ -290,7 +290,7 @@ pub fn range_tree(ranges: &[(u32, u32)]) -> RowAddrTreeMap {
m
}

/// [`range_tree`] as an ALLOW mask for `with_row_addr_prefilter`.
/// [`range_tree`] as an ALLOW mask for `with_row_id_prefilter`.
pub fn range_mask(ranges: &[(u32, u32)]) -> RowAddrMask {
RowAddrMask::from_allowed(range_tree(ranges))
}
Expand Down
4 changes: 2 additions & 2 deletions crates/crepe-plan-probe/src/lance_store.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
//! 2. the address mask `R` is built by relalg over that column as a `U32`
//! lane (`eq_u32` for a restriction, `Or` of `eq_u32` for the semijoin
//! centers), lowered by mask-risc onto `ndarray::simd`;
//! 3. the side is read with `with_row_addr_prefilter(R)`;
//! 3. the side is read with `with_row_id_prefilter(R)`;
//! 4. the rows go to `JoinTable::prepare_star`, as on the in-memory path.
//!
//! A side's generation is its table's Lance version.
Expand Down Expand Up @@ -214,7 +214,7 @@ pub(crate) async fn read_rows(
let mut s = ds.scan();
s.project(&["c0", "c1"]).map_err(lance_err)?;
if let Some(m) = mask {
s.with_row_addr_prefilter(m);
s.with_row_id_prefilter(m);
}
s.scan_stats_callback(cb);
let b = s.try_into_batch().await.map_err(lance_err)?;
Expand Down
2 changes: 1 addition & 1 deletion crates/crepe-plan-probe/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
//! planes) as one mask-risc program; terminals read the mask.
//! - [`lance_store`]: the same star over Lance 13 tables (written through
//! lancedb 0.40), each side read under a relalg-built address mask with
//! `with_row_addr_prefilter` (D-HOLO-H14d stage 3).
//! `with_row_id_prefilter` (D-HOLO-H14d stage 3).
//! - [`addressed`]: arm B, the addressed semijoin. Each relation is a lane
//! in key order, attested with `SealedFacetLane`; a side reads only the
//! ranges of its demanded keys.
Expand Down
2 changes: 1 addition & 1 deletion crates/holo-h14d-probe/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ description = "D-HOLO-H14d: C's own operation in four stages: logic, address mas
lance-graph-relalg = { path = "../lance-graph-relalg" }
lance-graph-mask-risc = { path = "../lance-graph-mask-risc" }
lance-graph-r2il = { path = "../lance-graph-r2il" }
lance = { version = "13.*", default-features = false }
lance = { version = "14.*", default-features = false }
arrow-array = "58"
arrow-schema = "58"
futures = "0.3"
Expand Down
4 changes: 2 additions & 2 deletions crates/holo-h14d-probe/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
//! mask-risc program (`relalg::run`, `Measure::Keep`); the COUNT terminal is
//! `Measure::Count`.
//! 3. masked read: `R` prunes a Lance 13 read at the source
//! (`with_row_addr_prefilter`).
//! (`with_row_id_prefilter`).
//! 4. terminal: per survivor `i` of `R`, the presence bit of each requested
//! lane gates a positional load of `T_l[i]`, then the operation runs.
//!
Expand Down Expand Up @@ -487,7 +487,7 @@ async fn read(ds: &Dataset, mask: Option<RowAddrMask>, k: usize) -> (BTreeMap<u3
let mut s = ds.scan();
s.project(&names).unwrap();
if let Some(m) = mask {
s.with_row_addr_prefilter(m);
s.with_row_id_prefilter(m);
}
s.scan_stats_callback(Arc::new(move |c| {
let mut io = sink.lock().unwrap();
Expand Down
2 changes: 1 addition & 1 deletion crates/holo-lance-masked-io-probe/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ description = "D-HOLO-H14c-4/-5: Lance 13 masked I/O and coordinate identity pro
# cargo run --release --manifest-path crates/holo-lance-masked-io-probe/Cargo.toml
# cargo test --release --manifest-path crates/holo-lance-masked-io-probe/Cargo.toml
[dependencies]
lance = { version = "13.*", default-features = false }
lance = { version = "14.*", default-features = false }
arrow-array = "58"
arrow-schema = "58"
futures = "0.3"
Expand Down
2 changes: 1 addition & 1 deletion crates/holo-lance-masked-io-probe/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -231,7 +231,7 @@ async fn read_with(
s.project(&names)
.map_err(|e| ReadError::Lance(e.to_string()))?;
if let Some(m) = mask {
s.with_row_addr_prefilter(m);
s.with_row_id_prefilter(m);
}
s.materialization_style(style);
s.scan_stats_callback(Arc::new(move |c| {
Expand Down
2 changes: 1 addition & 1 deletion crates/holograph/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ arrow-buffer = "58"
datafusion = { version = "54", optional = true }

# Lance columnar format — 100x faster random access than Parquet
lance = { version = "13.*", optional = true, default-features = false }
lance = { version = "14.*", optional = true, default-features = false }

# Async runtime
tokio = { version = "1.49", features = ["rt-multi-thread", "macros"], optional = true }
Expand Down
2 changes: 1 addition & 1 deletion crates/lance-graph-benches/benches/cypher_df_vs_quack.rs
Original file line number Diff line number Diff line change
Expand Up @@ -454,7 +454,7 @@ fn scan_allowed(
let ds = Dataset::open(path).await.unwrap();
let mut sc = ds.scan();
sc.project(columns).unwrap();
sc.with_row_addr_prefilter(allow.clone());
sc.with_row_id_prefilter(allow.clone());
let batches: Vec<RecordBatch> = sc
.try_into_stream()
.await
Expand Down
6 changes: 3 additions & 3 deletions crates/lance-graph-benches/examples/lance_source_probe.rs
Original file line number Diff line number Diff line change
Expand Up @@ -9,14 +9,14 @@
//!
//! and is checked against a row-by-row oracle computed from the generator.
//! The probe adds no op, no lowering and no Lance-side code: it uses the
//! public scanner (`filter`, `project`, `with_row_addr_prefilter`,
//! public scanner (`filter`, `project`, `with_row_id_prefilter`,
//! `scan_stats_callback`) and folds the returned Arrow columns in place
//! through mask-risc.
//!
//! | arm | what Lance does | what mask-risc does |
//! |---|---|---|
//! | A | full scan of `x`, `v` | `x < t` per batch, then COUNT and SUM(v) |
//! | C | `with_row_addr_prefilter` of the row range the sorted fixture puts `x < t` in | SUM(v) only; the predicate is already the range |
//! | C | `with_row_id_prefilter` of the row range the sorted fixture puts `x < t` in | SUM(v) only; the predicate is already the range |
//! | Dz | `filter("xz < t")` through a zone-map index (approximate; Lance rechecks) | SUM(v) only |
//! | Db | `filter("xb < t")` through a B-tree index (exact) | SUM(v) only |
//! | N | `Db` with an empty projection: COUNT only | nothing |
Expand Down Expand Up @@ -376,7 +376,7 @@ async fn run(ds: &Dataset, arm: Arm, t: i32, k_sorted: Option<usize>, f: &mut Fo
let tm = Instant::now();
let mask = sorted_prefix_mask(k_sorted.expect("sorted only"));
mask_ms = tm.elapsed().as_secs_f64() * 1e3;
sc.with_row_addr_prefilter(mask);
sc.with_row_id_prefilter(mask);
sc.project(&["v"]).expect("project");
}
Arm::Dz => {
Expand Down
2 changes: 1 addition & 1 deletion crates/lance-graph-catalog/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ categories = ["database", "data-structures", "science"]
arrow-schema = { workspace = true }
async-trait = "0.1"
datafusion = { workspace = true }
lance-namespace = "13.*"
lance-namespace = "14.*"
reqwest = { version = "0.12", features = ["json"], optional = true }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
Expand Down
2 changes: 1 addition & 1 deletion crates/lance-graph-dir-sim-lance/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ description = "Lance storage for the dir-sim history log: one verified batch per
lance-graph-dir-sim = { path = "../lance-graph-dir-sim" }
# The lance family moves in lockstep with the lance-graph root (13 / arrow
# 58); no cloud object stores, a local or file URI is enough here.
lance = { version = "13.*", default-features = false }
lance = { version = "14.*", default-features = false }
arrow-array = "58"
arrow-schema = "58"

Expand Down
4 changes: 2 additions & 2 deletions crates/lance-graph/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ lance-graph-quack = { path = "../lance-graph-quack" }
lance-graph-r2il = { path = "../lance-graph-r2il" }
lance = { workspace = true }
lance-linalg = { workspace = true }
lance-namespace = "13.*"
lance-namespace = "14.*"
# `remote` is NOT a capability we asked for: lancedb 0.40.0 does not compile
# without it. `src/lib.rs` leaves `pub mod job;` ungated while `src/job.rs`
# builds `Error::Http`, which `src/error.rs` gates on `cfg(feature = "remote")`
Expand Down Expand Up @@ -162,7 +162,7 @@ lite-unified = []
deepnsm = { path = "../deepnsm" }
lance-graph-contract = { path = "../lance-graph-contract" }
futures = "0.3"
lance-arrow = "13.*"
lance-arrow = "14.*"
lance-index = { workspace = true }
tempfile = "3"
# D-MW-P2 probe only: `ShardWriterConfig::shard_id` is a `Uuid`, so the probe
Expand Down
Loading