Skip to content

refactor(runtime): adopt Bycorf storage backend facade - #417

Merged
thweetkomputer merged 6 commits into
eloqdata:mainfrom
thweetkomputer:refactor/storage-backend
Nov 3, 2026
Merged

thweetkomputer merged 6 commits into
eloqdata:mainfrom
thweetkomputer:refactor/storage-backend

Conversation

@thweetkomputer

@thweetkomputer thweetkomputer commented Nov 3, 2026 •

Copy link
Copy Markdown
Collaborator

Context

Adopt the Bycorf refactor that encapsulates SPDK/io_uring storage dispatch in StorageBackend rather than exposing storage through NetBackend and repeating selection in Worker.

Bycorf dependency eloqdata/bycorf#28 is merged. This PR pins its upstream merge commit 4b10a34e8eff7d991018454a682019773e3a112e, whose tree matches the tested refactor revision.

Behavior before and after

Lavik continues to use the same public Worker and storage-awaitable APIs. Each worker still uses one kernel ring for networking, kernel storage, timers and wakeups. Network and storage backend selection is now encapsulated in their respective Bycorf facades.

Implementation

Advance Bycorf from a430b323b65c61467a552d0f908aca0790e1cb99 to 4b10a34e8eff7d991018454a682019773e3a112e and update the system overview. Include bycorf/io/spdk_storage.h directly in server.cpp: replacement-RDB validation calls IsSpdkStoragePath, and the refactored Worker no longer transitively exposes its declaration when bypass is disabled. Also keep Release builds warning-clean by marking the assert-only insertion parameter maybe_unused and explicitly defaulting Lua error counts to empty. These changes preserve runtime behavior.

Design decisions and alternatives

The worker owns the shared ring and its private CRTP network/storage facades. The ring is the first base subobject so it outlives the facades; facade access uses a fixed address adjustment without a stored ring pointer. SPDK dispatch reuses its worker binding as initialized state, without an additional optional wrapper or initialized flag. Details and the shared-ring regression are in the dependent Bycorf PR.

Documentation and comments

Updated docs/architecture/01-overview.md; the Bycorf change updates its I/O/runtime architecture and ownership/drain comments.

Test plan

Commands run from the Lavik repository root:

cmake --build bycorf/build-testcases --parallel 4
ctest --test-dir bycorf/build-testcases --output-on-failure
cmake --build bycorf/build-testcases-bypass --parallel 4
ctest --test-dir bycorf/build-testcases-bypass --output-on-failure
pre-commit run

The test builds use CMAKE_BUILD_TYPE=Debug and BYCORF_BUILD_EXAMPLES=OFF; the bypass build additionally uses BYCORF_KERNEL_BYPASS=ON.

Both Bycorf builds passed: 95 normal tests and 121 bypass-capable tests passed. Bycorf formatting and parent formatting checks passed, including the application build fixes.

Clang 18 syntax checks passed for server.cpp, list_command.cpp and lua_eval.cpp after the build fixes. A complete GCC Release build and LTO link passed with cmake --build build --target lavik --parallel 6, with warnings treated as errors. That local integration check used the same application fixes on the follow-up fixed-buffer branch (Bycorf 3389936); CI validates this PR with the merged refactor pin.

Risk assessment

The runtime ownership/lifecycle refactor is exercised by shared-ring, SPDK stub shutdown/reentry, and DPDK software tests. Physical NIC/NVMe workloads were not benchmarked. The previous missing-declaration and Release warning failures are fixed; the latest CI run validates the exact PR revision.

Rollback plan

Revert this commit to restore the prior Bycorf pin and architecture description.

Reviewer guide

Review the runtime implementation and validation in Bycorf PR #28, then verify this pin and the overview's ownership/lifecycle model.

Follow-up work

The Bycorf dependency is merged and its upstream commit is pinned. Check the latest CI result before merging.

Summary by CodeRabbit

  • Documentation
    • Updated the architecture overview to clarify how workers share an I/O ring across network and storage operations, timers, and wakeups. It also describes how network and storage adapters use the ring and shut down before it.
  • Maintenance
    • Updated the included bycorf project revision.

@coderabbitai

coderabbitai Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review.

Walkthrough

The Bycorf subproject reference now points to a different commit. The architecture overview describes one worker io_uring shared by network and storage I/O, timers, and wakeups, along with adapter initialization and shutdown order.

Changes

io_uring ownership and adapter lifecycle

Layer / File(s) Summary
Worker ring and adapter lifecycle
bycorf, docs/architecture/01-overview.md
The Bycorf reference points to a new commit. The overview describes the worker sharing its io_uring with network and storage adapters, initializing the ring first, and stopping the adapters before ring shutdown.

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Refactor

Possibly related PRs

  • eloqdata/lavik#384: Changes storage reads and the read-buffer model on io_uring and SPDK backends, which share the storage I/O integration described here.
  • eloqdata/lavik#377: Changes storage-buffer allocation and registration for io_uring and SPDK, which use the storage backend integration described here.

Suggested labels: performance-watch

Suggested reviewers: liunyl

Merge Risk: Minimal · up to 54e38

No concrete merge-blocking issue is established for the storage-backend update. Merge readiness remains subject to normal checks and confirmation of the final upstream Bycorf commit.

Pre-merge checks: Passed 4
Check name Status Explanation
Title check Passed The title names the main change: a refactor that adopts the Bycorf storage backend facade. The raw summary covers only a submodule pin bump and an overview doc update, so the title describes a change …
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Description check Passed The description follows the repository template's main sections (Context, Behavior before and after, Implementation, Design decisions and alternatives, Documentation and comments, Test plan, Risk asse…
Full details: Title check

Explanation

The title names the main change: a refactor that adopts the Bycorf storage backend facade. The raw summary covers only a submodule pin bump and an overview doc update, so the title describes a change that the changeset does not show directly. The objectives state the storage dispatch encapsulation as the aim, which matches the title. The title is specific and uses a conventional commit format.

Full details: Description check

Explanation

The description follows the repository template's main sections (Context, Behavior before and after, Implementation, Design decisions and alternatives, Documentation and comments, Test plan, Risk assessment, Rollback plan, Reviewer guide, Follow-up work) and fills each one with specific content. It states the Bycorf pin change from a430b32 to 4b10a34, the overview update, the Bycorf dependency, and the test commands and results. The Test plan lists exact commands and outcomes. Two gaps remain. The description states the Lavik Release build passed with -Werror, but the PR objectives say the Lavik Release build was blocked by existing -Werror diagnostics, and the description's pre-commit claim does not match the objectives' claim that pre-commit skipped all hooks. The description also states the bypass test count and other numbers that the objectives do not confirm. These conflicts need reconciling. The description does not mention the Bycorf pin 3f1577f from the raw summary, so the commit pinned by the PR is inconsistent between the summary and description. The description is mostly complete, so the check passes with these notes.

Finishing Touches
Generate unit tests (beta)
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Tip

🎯 Customize your risk assessments

Guide how CodeRabbit assesses merge risk and security architecture risk. Highlight sensitive changes and explain your security boundaries using Merge Risk Instructions and Security Risk Instructions in your review settings. Security architecture risk assessments require the Advanced plan or above.

✅ Automatic approvals, with your rules

Let CodeRabbit approve pull requests once required threads are resolved, pre-merge checks pass, and your risk limits are met. Enable Automatic approvals in review settings, with overrides for teams or file paths.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

This rabbit watches rings awake,
As adapters join the worker’s wake.
The network borrows, storage too,
Then both stop first when work is through.
The ring rests last beneath the moon.

CodeRabbit Verify

  • Run CodeRabbit Verify to test the changes in this pull request.
Recent review info
Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 09f65894-589b-467a-b40a-4fd6faf37194
Commits

Reviewing files that changed from the base of the PR and between e1efda4 and 54e381b.

Files selected for processing: 2
  • bycorf
  • docs/architecture/01-overview.md

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

The backend refactor stopped transitively exposing spdk_storage.h through Worker when kernel bypass is disabled. Include the declaring header directly for IsSpdkStoragePath in replacement-RDB validation. This fixes the Clang CI failure without changing runtime behavior.
Mark the insertion position used only by assertions as maybe_unused and explicitly default the Lua error-count collection to empty. This preserves behavior while resolving unused-parameter and missing-field-initializers diagnostics under Werror.
@thweetkomputer
thweetkomputer merged commit 8115c94 into eloqdata:main Nov 3, 2026
7 checks passed
@thweetkomputer
thweetkomputer deleted the refactor/storage-backend branch November 3, 2026 05:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant