Skip to content

Redis 8.12 BITRANGEINTARRAY (bit enumeration): IDatabase.StringBitPositions - #3315

Merged
mgravell merged 6 commits into
8.12from
marc/bitrangeintarray
Nov 9, 2026
Merged

mgravell merged 6 commits into
8.12from
marc/bitrangeintarray

Conversation

@mgravell

@mgravell mgravell commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

Implements the Redis 8.12 BITRANGEINTARRAY command (redis/redis#15986, part of the native bitmap work in redis/redis#15331). The plan is in notes/bitrangeintarray/plan.md; the implementation follows it.

API

// IDatabase / IDatabaseAsync, [Experimental(Experiments.Server_8_12)]
long[] StringBitPositions(RedisKey key, bool bit, long start = 0, long end = -1,
    StringIndexType indexType = StringIndexType.Byte, long? limit = null,
    CommandFlags flags = CommandFlags.None);

// RedisFeatures
bool StringBitPositions => Version.IsAtLeast(v8_12_0);

The enumerating sibling of StringBitPosition (BITPOS): same key, bit, start, end, indexType shape and defaults, with limit added before flags. bit: false sends ZEROS; StringIndexType.Bit sends BIT, and the default BYTE is omitted exactly as BITPOS/BITCOUNT do. Negative limit throws client-side. The reply is a flat integer array, so ResultProcessor.Int64Array is reused, and the argument tail fits an existing Message.Create overload: no new types, no new processor. Replica-eligible, from the READONLY flag. The XML remarks carry the paging rule (BIT: next start is last + 1; BYTE: the server completes the byte, next start is last / 8 + 1).

Verified

  • dotnet build Build.csproj -c Release /p:CI=true clean.
  • RoundTripUnitTests.BitPositionsRoundTrip: exact wire bytes for every option combination, LIMIT 0, and the two client-side argument checks.
  • BitTests.StringBitPositions_* (ranges, clear bits, limit, paging without gaps in both units, missing key and WRONGTYPE), gated Create(require: RedisFeatures.v8_12_0): green on RESP2 and RESP3 against a local build of unstable at 6a5fee67, which reports redis_version:255.255.255 (and originally skipped on the docker topology).
  • Earlier, the server PR's examples were checked by hand against the same build for both a string bitmap and a native bitmap-roaring one; the plan records them as observed.
  • 8.12 RC2 (2026-11-03): RedisFeatures.v8_12_0 is now pinned to 8.11.241, which is what RC2 reports — the 8.12.0 placeholder would have made the gated tests skip on RCs. The docker topology / CI default image is now redislabs/client-libs-test:8.12-rc2, with cluster-bus-port-protected-mode no added to the test cluster configs (RC2 introduces that check and otherwise refuses to start a plaintext cluster node). All StringBitPositions live and round-trip tests pass against the RC2 topology; the wire syntax (key start end [BYTE|BIT] [LIMIT n] [ZEROS], set bits by default) and the missing-key/WRONGTYPE behaviour are unchanged on RC2.

Shared with #3261 (BLESS)

Both are 8.12, so this adds the same gating pieces with identical text: Experiments.Server_8_12 = "SER011", the SER011 NoWarn entry, RedisFeatures.v8_12_0 (pinned to 8.11.241, the RC2 version), and docs/exp/SER011.md. Whichever lands second merges textually, except the feature list in docs/exp/SER011.md, which needs both bullets.

Also

  • Deletes the BITRANGEINTARRAY row from docs/CommandCoverage.md; the audit script reconciles.
  • Deferred, as in the plan: a Lease<long> variant for dense bitmaps, and an IAsyncEnumerable<long> paging helper.

RC2 ZSCAN duplicates (server behavior change, handled)

The only CI failures on the RC2 image are the 8 ScanTests.SortedSetScanLarge cases (all page sizes, RESP2+RESP3): a full ZSCAN iteration over a stable 2000-element sorted set returns 2001-2003 entries, with a few members duplicated. Reproduced with pure redis-cli against redislabs/client-libs-test:8.12-rc2 (deterministic per dataset; ZCARD is 2000, distinct members 2000), clean on redis:8.2, and HSCAN/SSCAN are unaffected — a ZSCAN behavior change in the 8.12 btree zset encoding, reported upstream as redis/redis#16158. SortedSetScanLarge now asserts the distinct member count, so CI is expected green; revisit if redis/redis#16158 is fixed before GA. Everything else was already green (6700+ tests).

@mgravell mgravell changed the title Redis 8.12 BITRANGEINTARRAY (bit enumeration): implementation plan Redis 8.12 BITRANGEINTARRAY (bit enumeration) support Oct 30, 2026
Enumerates the set (or, with bit: false, the clear) bit positions in a
byte or bit range of a string or native bitmap, ascending, with an
optional LIMIT; the sibling of StringBitPosition (BITPOS). Reuses
ResultProcessor.Int64Array; the argument tail fits Message.Create, with
the default BYTE omitted as BITCOUNT/BITPOS do. Negative limit throws
client-side.

Gated [Experimental(Experiments.Server_8_12)] (SER011), with
RedisFeatures.v8_12_0 and RedisFeatures.StringBitPositions, matching the
BLESS branch so the two merge textually.

Tests: exact wire bytes for every option combination (BitPositionsRoundTrip),
and live tests in BitTests gated on v8_12_0, observed green against a
local build of unstable (6a5fee67) on RESP2 and RESP3.

Removes the BITRANGEINTARRAY row from the command coverage ledger.
@mgravell mgravell changed the title Redis 8.12 BITRANGEINTARRAY (bit enumeration) support Redis 8.12 BITRANGEINTARRAY (bit enumeration): IDatabase.StringBitPositions Oct 30, 2026
… image

- RedisFeatures.v8_12_0 is now 8.11.241, which is what 8.12 RC2 reports
  (the earlier 8.12.0 placeholder made the gated tests skip on RC2)
- the test topology image defaults to redislabs/client-libs-test:8.12-rc2
- the cluster configs opt out of the new cluster-bus-port-protected-mode
  check, which otherwise refuses to start a plaintext cluster node on RC2

Validated locally against the RC2 topology: StringBitPositions live tests
and BitPositionsRoundTrip all pass, and the cluster forms cleanly.
…re-add only the SER011 StringBitPositions entries to Unshipped)
The 8.12 btree zset encoding can return duplicate members even for an
unmodified key (redis/redis#16158; the SCAN contract allows it), so the
test now asserts the distinct member count instead of the raw count.
All 8 cases pass against the RC2 topology.
@mgravell
mgravell changed the base branch from main to 8.12 November 9, 2026 10:59
@mgravell
mgravell merged commit fce8a9e into 8.12 Nov 9, 2026
6 checks passed
@mgravell
mgravell deleted the marc/bitrangeintarray branch November 9, 2026 13:09
mgravell added a commit that referenced this pull request Nov 9, 2026
The hash-field-TTL idea (7.4's HEXPIRE) brought to sets (redis/redis#15938,
8.12-rc2), mirroring the HashField* family:

- SetAddAndSetExpiry (SADDEX): add members and/or (re)set their expiration in
  one atomic call; When.NotExists/Exists map to MNX/MXX, which are
  all-or-nothing across the batch, exactly as HSETEX's FNX/FXX; the bare form
  discards an existing TTL, KEEPTTL keeps it
- SetMemberExpire (SEXPIRE/SPEXPIRE/SEXPIREAT/SPEXPIREAT by precision) ->
  ExpireResult[], per member
- SetMemberGetTimeToLive (SPTTL) and SetMemberGetExpireDateTime (SPEXPIRETIME)
  -> long[]
- SetMemberPersist (SPERSIST) -> PersistResult[]
- KeyNotificationType.SExpired ('sexpired', side-effect block), so the 8.12
  notification - including on the subkey channels, whose payload names the
  lapsed member - parses as a known type; GetSubKeys() already handles the
  payload

Gated [Experimental] behind SER011/Experiments.Server_8_12 and
RedisFeatures.v8_12_0 (8.11.241), both shared with #3261/#3315; the test
docker image moves to client-libs-test:8.12-rc2 as in those PRs.

Round-trip tests pin the exact wire bytes; the live tests (HashFieldTests
mirror) pass against the 8.12-rc2 topology on both protocols. Plan document
in notes/set-member-expiry/.
mgravell added a commit that referenced this pull request Nov 9, 2026
…ST on IDatabase (#3340)

* Initial support for Redis 8.12 set-member expiration

The hash-field-TTL idea (7.4's HEXPIRE) brought to sets (redis/redis#15938,
8.12-rc2), mirroring the HashField* family:

- SetAddAndSetExpiry (SADDEX): add members and/or (re)set their expiration in
  one atomic call; When.NotExists/Exists map to MNX/MXX, which are
  all-or-nothing across the batch, exactly as HSETEX's FNX/FXX; the bare form
  discards an existing TTL, KEEPTTL keeps it
- SetMemberExpire (SEXPIRE/SPEXPIRE/SEXPIREAT/SPEXPIREAT by precision) ->
  ExpireResult[], per member
- SetMemberGetTimeToLive (SPTTL) and SetMemberGetExpireDateTime (SPEXPIRETIME)
  -> long[]
- SetMemberPersist (SPERSIST) -> PersistResult[]
- KeyNotificationType.SExpired ('sexpired', side-effect block), so the 8.12
  notification - including on the subkey channels, whose payload names the
  lapsed member - parses as a known type; GetSubKeys() already handles the
  payload

Gated [Experimental] behind SER011/Experiments.Server_8_12 and
RedisFeatures.v8_12_0 (8.11.241), both shared with #3261/#3315; the test
docker image moves to client-libs-test:8.12-rc2 as in those PRs.

Round-trip tests pin the exact wire bytes; the live tests (HashFieldTests
mirror) pass against the 8.12-rc2 topology on both protocols. Plan document
in notes/set-member-expiry/.

* SADDEX rides SetAdd: two Expiration overloads replace the four-per-arity SetAddAndSetExpiry surface

Review feedback (see the PR): Expiration already encodes EX/PX/EXAT/PXAT/KEEPTTL - exactly SADDEX's
expiration block - so the TimeSpan?/DateTime/keepTtl splits collapse and the surface is
SetAdd(key, value, Expiration, When, flags) and SetAdd(key, values, Expiration, When, flags), both
bool (+async): 8 members become 4, with no new method name. A required Expiration keeps overload
resolution away from the existing SetAdd members.

- Expiration.Persist is rejected (SADDEX has no PERSIST; SetMemberPersist is the way to remove
  member expirations) - previously it would have been sent as a bogus token.
- The overloads always send SADDEX: plain SADD is not an equivalent fallback for the no-expiry
  form, because bare SADDEX discards an existing member's TTL where SADD leaves existing members
  untouched; the XML docs call this out, and that the multi-value form reports only whether
  anything changed, not a count.
- The SetMemberExpire/GetTimeToLive/GetExpireDateTime/Persist family is unchanged (ExpireWhen's
  GT/LT cannot ride Expiration, and it mirrors the HashField shape).
- plan.md carries a revision note; SER011.md updated; all 50 SetMemberExpiry tests pass live
  against 8.12-rc2, 525 KeyPrefixed tests pass.

* Each set-member-expiry test deletes its key up front: SADDEX's changed-anything reply and the MNX/NX conditions make residue from a previous run (or the other protocol) look like a failure
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant