Skip to content

feat: build and test natively on macOS - #18

Merged
zfarrell merged 16 commits into
mainfrom
feat/native-macos-support
Aug 27, 2026
Merged

zfarrell merged 16 commits into
mainfrom
feat/native-macos-support

Conversation

@zfarrell

Copy link
Copy Markdown

Makes the workspace build, test and run natively on macOS: all 9 t4 mount sites now go through one platform-aware liquid_cache::store::mount (byte-identical to t4::mount on Linux, buffered plus a one-time warning elsewhere), and perf-event2 becomes a Linux-only dependency so the benchmark crate compiles. Adds a macOS CI job, gives src/datafusion/src/utils.rs its first aarch64 test coverage, and documents two things measurement depends on: buffered I/O means memory accounting understates residency off Linux, and FSST picks a different symbol table per architecture so compression ratios do not transfer between arm64 and x86_64.

Closes #12.

Comment thread src/datafusion-local/src/tests/mod.rs Outdated
Comment thread src/datafusion-local/src/tests/mod.rs Outdated
Comment thread src/core/src/store.rs
claude[bot]
claude Bot previously approved these changes Aug 27, 2026

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the full diff. The centralization is clean: t4::mount/mount_with_options now appear only in src/core/src/store.rs, the two crates that dropped the t4 dependency no longer reference t4:: anywhere, and liquid-cache is already a direct dependency of both. The PerfEventCollector stub keeps the call site in inprocess_runner.rs:527-570 unchanged on every target, and the new utils.rs reference oracle is a genuine independent check rather than the existing impl-vs-impl differential.

Three non-blocking comments inline. Note that the macOS job itself is the only real verification of the non-Linux paths and it had not reported when this review started, so I have not confirmed the workspace builds on aarch64-darwin.

@codecov

codecov Bot commented Aug 27, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.92473% with 1 line in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/datafusion-local/src/lib.rs 66.66% 0 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

@github-actions

github-actions Bot commented Aug 27, 2026

Copy link
Copy Markdown

📊 Benchmark Comparison

Current: a8ab766f (Liquid) vs Baseline: a8ab766f (DataFusionDefault)

Query Cold Time Δ Warm Time Δ CPU Time Δ
Q1 3.0ms (3.0ms) +0.0% 0.250ms (0.000ms) +inf% 0.000ms (0.000ms) +0.0%
Q2 11.0ms (6.0ms) +83.3% 3.5ms (5.2ms) -33.3% 5.8ms (7.2ms) -20.7%
Q3 14.0ms (12.0ms) +16.7% 4.8ms (11.8ms) -59.6% 2.0ms (24.0ms) -91.7%
Q4 18.0ms (12.0ms) +50.0% 3.8ms (11.2ms) -66.7% 1.5ms (24.0ms) -93.8%
Q5 53.0ms (56.0ms) -5.4% 43.5ms (57.2ms) -24.0% 3.0ms (27.8ms) -89.2%
Q6 143.0ms (106.0ms) +34.9% 67.5ms (106.8ms) -36.8% 19.0ms (80.2ms) -76.3%
Q7 1.0ms (1.0ms) +0.0% 0.250ms (1.0ms) -75.0% 0.000ms (0.000ms) +0.0%
Q8 8.0ms (9.0ms) -11.1% 4.2ms (5.8ms) -26.1% 7.0ms (7.0ms) +0.0%
Q9 87.0ms (79.0ms) +10.1% 64.8ms (79.5ms) -18.6% 4.0ms (43.8ms) -90.9%
Q10 90.0ms (90.0ms) +0.0% 66.8ms (89.8ms) -25.6% 5.0ms (62.0ms) -91.9%
Q11 38.0ms (26.0ms) +46.2% 18.5ms (26.5ms) -30.2% 34.0ms (37.5ms) -9.3%
Q12 44.0ms (31.0ms) +41.9% 20.2ms (29.5ms) -31.4% 35.5ms (42.8ms) -17.0%
Q13 140.0ms (107.0ms) +30.8% 72.0ms (100.2ms) -28.2% 57.0ms (83.5ms) -31.7%
Q14 312.0ms (132.0ms) +136.4% 112.8ms (135.5ms) -16.8% 83.5ms (111.5ms) -25.1%
Q15 178.0ms (93.0ms) +91.4% 67.5ms (91.5ms) -26.2% 61.8ms (95.8ms) -35.5%
Q16 59.0ms (56.0ms) +5.4% 49.2ms (61.0ms) -19.3% 3.2ms (27.8ms) -88.3%
Q17 345.0ms (152.0ms) +127.0% 151.5ms (151.2ms) +0.2% 75.0ms (103.2ms) -27.4%
Q18 312.0ms (160.0ms) +95.0% 152.2ms (149.2ms) +2.0% 74.8ms (105.8ms) -29.3%
Q19 495.0ms (294.0ms) +68.4% 278.2ms (295.2ms) -5.8% 93.2ms (151.0ms) -38.2%
Q20 14.0ms (13.0ms) +7.7% 3.2ms (11.2ms) -71.1% 6.8ms (24.2ms) -72.2%
Q21 810.0ms (188.0ms) +330.9% 281.5ms (182.0ms) +54.7% 516.0ms (278.8ms) +85.1%
Q22 1.27s (243.0ms) +424.7% 312.8ms (210.8ms) +48.4% 184.8ms (360.2ms) -48.7%
Q23 2.82s (614.0ms) +359.9% 848.5ms (599.2ms) +41.6% 558.2ms (771.5ms) -27.6%
Q24 17.51s (920.0ms) +1803.0% 772.2ms (930.5ms) -17.0% 654.5ms (2.55s) -74.3%
Q25 207.0ms (64.0ms) +223.4% 15.5ms (63.2ms) -75.5% 38.8ms (131.8ms) -70.6%
Q26 96.0ms (52.0ms) +84.6% 18.0ms (46.5ms) -61.3% 48.5ms (86.0ms) -43.6%
Q27 171.0ms (61.0ms) +180.3% 26.2ms (59.8ms) -56.1% 85.8ms (122.0ms) -29.7%
Q28 897.0ms (191.0ms) +369.6% 273.0ms (195.8ms) +39.5% 357.2ms (280.8ms) +27.2%
Q29 1.68s (1.01s) +65.9% 1.10s (1.03s) +6.0% 393.0ms (357.0ms) +10.1%
Q30 35.0ms (28.0ms) +25.0% 25.2ms (29.8ms) -15.1% 6.5ms (21.2ms) -69.4%
Q31 246.0ms (110.0ms) +123.6% 46.8ms (94.5ms) -50.5% 41.2ms (144.5ms) -71.5%
Q32 551.0ms (96.0ms) +474.0% 80.8ms (93.2ms) -13.4% 56.8ms (148.5ms) -61.8%
Q33 190.0ms (188.0ms) +1.1% 158.2ms (171.2ms) -7.6% 8.0ms (70.0ms) -88.6%
Q34 985.0ms (324.0ms) +204.0% 392.8ms (321.5ms) +22.2% 364.0ms (273.5ms) +33.1%
Q35 898.0ms (320.0ms) +180.6% 398.2ms (324.2ms) +22.8% 365.2ms (270.8ms) +34.9%
Q36 59.0ms (56.0ms) +5.4% 49.5ms (55.8ms) -11.2% 3.5ms (24.2ms) -85.6%
Q37 336.0ms (90.0ms) +273.3% 78.5ms (89.2ms) -12.0% 45.5ms (70.8ms) -35.7%
Q38 74.0ms (46.0ms) +60.9% 25.8ms (49.0ms) -47.4% 16.0ms (25.0ms) -36.0%
Q39 249.0ms (45.0ms) +453.3% 13.0ms (46.2ms) -71.9% 12.5ms (69.8ms) -82.1%
Q40 741.0ms (172.0ms) +330.8% 205.5ms (164.5ms) +24.9% 87.8ms (126.8ms) -30.8%
Q41 24.0ms (21.0ms) +14.3% 10.0ms (21.0ms) -52.4% 7.8ms (20.0ms) -61.3%
Q42 23.0ms (20.0ms) +15.0% 10.0ms (18.5ms) -45.9% 7.2ms (16.8ms) -56.7%
Q43 20.0ms (18.0ms) +11.1% 11.5ms (16.8ms) -31.3% 7.5ms (11.8ms) -36.2%

⚠️ LiquidCache is slower on 11 queries (warm)

  • Q1: warm +inf% (0.250ms vs 0.000ms)
  • Q21: warm +54.7% (281.5ms vs 182.0ms)
  • Q22: warm +48.4% (312.8ms vs 210.8ms)
  • Q23: warm +41.6% (848.5ms vs 599.2ms)
  • Q28: warm +39.5% (273.0ms vs 195.8ms)
  • Q40: warm +24.9% (205.5ms vs 164.5ms)
  • Q35: warm +22.8% (398.2ms vs 324.2ms)
  • Q34: warm +22.2% (392.8ms vs 321.5ms)
  • Q29: warm +6.0% (1.10s vs 1.03s)
  • Q18: warm +2.0% (152.2ms vs 149.2ms)
  • Q17: warm +0.2% (151.5ms vs 151.2ms)

Compared Liquid vs DataFusionDefault on the same runner
Cold Time: first iteration; Warm Time: average of remaining iterations.

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both cycle-1 nits are addressed: the memory bound now parses the committed .snap through the same memory_bytes predicate (with the expected.is_empty() guard that keeps the length check from passing vacuously), and the t4::mount/mount_with_options ban is now enforced by clippy in both config files, with src/core::store::mount as the single #[allow]ed call site. Confirmed clippy.toml and src/datafusion/clippy.toml are the only two config files in the tree, and no t4::mount* call sites remain outside src/core/src/store.rs.

No new findings. CI on this push was still queued/in progress when I reviewed, so I am not asserting the macOS job's result.

@zfarrell
zfarrell merged commit 1fc1049 into main Aug 27, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add support for native macOS

1 participant