Skip to content

Add BufferCursor, an implementation of Read + Seek for Buffer - #8320

Open
trxcllnt wants to merge 1 commit into
apache:mainfrom
trxcllnt:fea/buffer-cursor
Open

trxcllnt wants to merge 1 commit into
apache:mainfrom
trxcllnt:fea/buffer-cursor

Conversation

@trxcllnt

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Doesn't close #8253, but possibly contributes to it.

Rationale for this change

It is sometimes necessary to interface with APIs that require seekable reads, such as archive or compression libraries.

If a Buffer represents some (contiguous or non-contiguous) data that must be deflated, a user must first materialize the bytes into a contiguous buffer and create a std::io::Cursor in order to call deflate:

use std::io::{Read, Cursor};
use zip::ZipArchive;

let buf = operator.read("key").await?;
// !! User must copy Buffer to a contiguous Vec<u8>
let zip = ZipArchive::new(Cursor::new(buf.to_bytes()))?;

let mut out = vec![];
zip.by_name("data.txt").read_to_end(&mut out)?;
println!("{}", str::from_utf8(&out)?); // "contents of data.txt"

This PR implements a BufferCursor, which allows reading with forwards/backwards seekability. This allows users to call such APIs while maintaining zero-copy semantics:

use std::io::Read;
use zip::ZipArchive;
use opendal::BufferCursor;

let buf = operator.read("key").await?;
// !! BufferCursor reads original Buffer, avoiding an intermediate copy
let zip = ZipArchive::new(BufferCursor::new(buf))?;

let mut out = vec![];
zip.by_name("data.txt").read_to_end(&mut out)?;
println!("{}", str::from_utf8(&out)?); // "contents of data.txt"

Bytes does not support seekable reads, otherwise I would have preferred to rely on that instead.

What changes are included in this PR?

Add a new BufferCursor struct that implements std::io::Read, std::io::BufRead, and std::io::Seek.

Are there any user-facing changes?

Yes, the new BufferCursor struct.

Breaking changes

None

AI Usage Statement

None

@trxcllnt
trxcllnt requested a review from Xuanwo as a code owner September 17, 2026 18:20
@github-actions github-actions Bot added core releases-note/feat The PR implements a new feature or has a title that begins with "feat" size:L This PR changes 100-499 lines, ignoring generated files. labels Sep 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core releases-note/feat The PR implements a new feature or has a title that begins with "feat" size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

new feature: expose seekable and positioned streaming reads in Java

1 participant