Skip to content

Background compaction is triggered by bytes written, not dead space #53

Closed

Description

@venkat1701

The maintenance loop starts compaction once per checkpoint_wal_mb (256 MiB) of page writes, and each pass moves at most one segment. Across 16 benchmark runs that was exactly one pass per run. A small database that is mostly dead space can go a long time before anything happens.

It would be better to start from how much can be reclaimed:

  • keep a running count of dead bytes per sealed segment (liveness() walks every tree, too slow to call often)
  • compact when a segment is both mostly dead and big enough to bother
  • pick segments by bytes freed per byte copied

The live ratio barely matters right now: 0.3 and 0.9 picked the same 23 segments in my runs because they were about 95% dead. Part of #47.

Activity

  1. venkat1701 commented on Oct 11, 2026

    @venkat1701
    CollaboratorAuthor

    Fixed by #81. Every page now has a reference count, and each segment keeps a running total of its live bytes, so finding victims no longer needs a walk. The background thread compacts as soon as any sealed segment drops below compaction_live_ratio, emptiest first, one segment per pass, instead of once per 256 MiB written.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions