Skip to content

Add shrink_to_fit to RoaringBitmap and RoaringTreemap - #398

Open
emschwartz wants to merge 1 commit into
RoaringBitmap:mainfrom
emschwartz:shrink-to-fit
Open

emschwartz wants to merge 1 commit into
RoaringBitmap:mainfrom
emschwartz:shrink-to-fit

Conversation

@emschwartz

@emschwartz emschwartz commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Closes #136.

RoaringBitmap::shrink_to_fit shrinks the container vector and every array and run store to its length. Bitmap stores are fixed-size boxes, so there is nothing to shrink there. RoaringTreemap::shrink_to_fit calls it on each bitmap and leaves empty partitions alone, since removing them is optimize's job.

The use case is the one @fjmaco measured on the issue: intersection and difference results keep the capacity of their inputs, so a 55,887-value intersection of two ~970K-value bitmaps holds 1,889 KiB of heap where a clone of it holds 119 KiB. Today that memory only comes back by rebuilding the bitmap, for example with *bitmap = bitmap.clone(). CRoaring has roaring_bitmap_shrink_to_fit for the same reason. This version returns () rather than CRoaring's count of freed bytes, to behave like Vec::shrink_to_fit, which is what a Rust caller will expect from the name. fjmaco offered on the issue to write this too. Happy to close this if fjmaco's version is underway.

Values and store kinds are unchanged. The one visible side effect is that statistics().n_bytes_array_containers goes down, since it is computed from capacity.

Tested with doctests.

Authored with Claude (claude-opus-5-5).

@Kerollmops Kerollmops left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @emschwartz 👋

Thanks for the changes! The code looks good but I would rather prefer removing the ugly "hidden" methods and extensive tests for a simple addition like that and make sure we keep the codebase small and maintanable.

Have a nice day 🌵

Comment on lines +498 to +502
#[cfg(test)]
pub fn capacity(&self) -> usize {
self.0.capacity()
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
#[cfg(test)]
pub fn capacity(&self) -> usize {
self.0.capacity()
}

Comment thread roaring/src/bitmap/inherent.rs Outdated
Comment on lines +960 to +973
#[cfg(test)]
impl RoaringBitmap {
pub(crate) fn has_spare_capacity(&self) -> bool {
use crate::bitmap::store::Store;

self.containers.capacity() > self.containers.len()
|| self.containers.iter().any(|container| match &container.store {
Store::Array(array) => array.capacity() > array.len() as usize,
Store::Bitmap(_) => false,
Store::Run(runs) => runs.capacity() > runs.run_amount() as usize,
})
}
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
#[cfg(test)]
impl RoaringBitmap {
pub(crate) fn has_spare_capacity(&self) -> bool {
use crate::bitmap::store::Store;
self.containers.capacity() > self.containers.len()
|| self.containers.iter().any(|container| match &container.store {
Store::Array(array) => array.capacity() > array.len() as usize,
Store::Bitmap(_) => false,
Store::Run(runs) => runs.capacity() > runs.run_amount() as usize,
})
}
}

Comment thread roaring/src/bitmap/inherent.rs Outdated
Comment on lines +1316 to +1361

#[test]
fn shrink_to_fit() {
// Key 3 is in both bitmaps but not in their intersection
let mut lhs = RoaringBitmap::from_iter(
(0..2000)
.step_by(2)
.chain((0x1_0000..0x2_0000).step_by(2))
.chain(0x2_0000..0x2_1000)
.chain([0x3_0000]),
);
let mut rhs = RoaringBitmap::from_iter(
(1980..4000)
.step_by(2)
.chain((0x1_0000..0x2_0000).step_by(2))
.chain((0x2_0000..0x2_0300).filter(|x| x % 0x100 < 0x10))
.chain([0x3_0001]),
);
lhs.optimize();
rhs.optimize();
let mut bitmap = &lhs & &rhs;

let stores: Vec<_> = bitmap.containers.iter().map(|c| &c.store).collect();
let [Store::Array(array), Store::Bitmap(_), Store::Run(runs)] = stores[..] else {
panic!("expected an array, a bitmap and a run container");
};
assert!(bitmap.containers.capacity() > bitmap.containers.len());
assert!(array.capacity() > array.len() as usize);
assert!(runs.capacity() > runs.run_amount() as usize);

let before = bitmap.clone();
bitmap.shrink_to_fit();
assert!(!bitmap.has_spare_capacity());
assert_eq!(bitmap, before);
}

#[test]
fn shrink_to_fit_empty() {
let mut bitmap = RoaringBitmap::from_iter([1, 0x1_0000]);
bitmap.remove_range(..);
assert!(bitmap.has_spare_capacity());

bitmap.shrink_to_fit();
assert!(!bitmap.has_spare_capacity());
assert!(bitmap.is_empty());
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
#[test]
fn shrink_to_fit() {
// Key 3 is in both bitmaps but not in their intersection
let mut lhs = RoaringBitmap::from_iter(
(0..2000)
.step_by(2)
.chain((0x1_0000..0x2_0000).step_by(2))
.chain(0x2_0000..0x2_1000)
.chain([0x3_0000]),
);
let mut rhs = RoaringBitmap::from_iter(
(1980..4000)
.step_by(2)
.chain((0x1_0000..0x2_0000).step_by(2))
.chain((0x2_0000..0x2_0300).filter(|x| x % 0x100 < 0x10))
.chain([0x3_0001]),
);
lhs.optimize();
rhs.optimize();
let mut bitmap = &lhs & &rhs;
let stores: Vec<_> = bitmap.containers.iter().map(|c| &c.store).collect();
let [Store::Array(array), Store::Bitmap(_), Store::Run(runs)] = stores[..] else {
panic!("expected an array, a bitmap and a run container");
};
assert!(bitmap.containers.capacity() > bitmap.containers.len());
assert!(array.capacity() > array.len() as usize);
assert!(runs.capacity() > runs.run_amount() as usize);
let before = bitmap.clone();
bitmap.shrink_to_fit();
assert!(!bitmap.has_spare_capacity());
assert_eq!(bitmap, before);
}
#[test]
fn shrink_to_fit_empty() {
let mut bitmap = RoaringBitmap::from_iter([1, 0x1_0000]);
bitmap.remove_range(..);
assert!(bitmap.has_spare_capacity());
bitmap.shrink_to_fit();
assert!(!bitmap.has_spare_capacity());
assert!(bitmap.is_empty());
}

Comment thread roaring/src/treemap/inherent.rs Outdated
Comment on lines +613 to +625
#[test]
fn shrink_to_fit() {
let lhs = RoaringTreemap::from_iter((0..2000).chain((1 << 32)..(1 << 32) + 2000));
let rhs = RoaringTreemap::from_iter((1990..4000).chain((1 << 32) + 1990..(1 << 32) + 4000));
let mut treemap = &lhs & &rhs;
assert!(treemap.bitmaps().all(|(_, bitmap)| bitmap.has_spare_capacity()));

let before = treemap.clone();
treemap.shrink_to_fit();
assert!(treemap.bitmaps().all(|(_, bitmap)| !bitmap.has_spare_capacity()));
assert_eq!(treemap, before);
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
#[test]
fn shrink_to_fit() {
let lhs = RoaringTreemap::from_iter((0..2000).chain((1 << 32)..(1 << 32) + 2000));
let rhs = RoaringTreemap::from_iter((1990..4000).chain((1 << 32) + 1990..(1 << 32) + 4000));
let mut treemap = &lhs & &rhs;
assert!(treemap.bitmaps().all(|(_, bitmap)| bitmap.has_spare_capacity()));
let before = treemap.clone();
treemap.shrink_to_fit();
assert!(treemap.bitmaps().all(|(_, bitmap)| !bitmap.has_spare_capacity()));
assert_eq!(treemap, before);
}

Comment thread roaring/src/bitmap/inherent.rs Outdated
use proptest::prelude::*;

use super::*;
use crate::bitmap::store::Store;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
use crate::bitmap::store::Store;

Intersections and differences size their result arrays for the inputs, so
a small result can keep most of its input's capacity. shrink_to_fit frees
that memory by shrinking the container vector and every array and run
store, like CRoaring's roaring_bitmap_shrink_to_fit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature: shrink_to_fit

2 participants