Array data imported through the FFI might contain unaligned buffers

Describe the bug

I described the issue in detail on the arrow-adbc bug tracker:

To Reproduce

Executing a query through the Snowflake ADBC driver (Go) that happens to produce array with unaligned buffers. The linked issue above contains a stack trace.

Expected behavior

The Rust code importing the array should check the alignment and copy the buffers to an unaligned position if necessary.

Changing this to an enhancement, as the current behaviour is intentional, even if it could be improved (e.g. returning an error instead of panicking). It would be a bug if buffer misalignment was not being caught (i.e. UB).

See latest attempt

feat(ffi): opt-in buffer alignment in ArrowArrayStreamReader #9951

Still reachable on 60.0.0, shall i take?

Repro (arrow = { version = "=60.0.0", default-features = false, features = ["ffi"] }, cargo run in debug):

use arrow::array::{Array, StringArray, make_array};
use arrow::buffer::Buffer;
use arrow::datatypes::DataType;
use arrow::ffi::{FFI_ArrowArray, FFI_ArrowSchema, from_ffi};

fn main() {
    let strings = StringArray::from(vec!["a", "bb", "ccc"]);
    let offsets = strings.offsets().inner().inner().as_slice();
    // Same offsets, placed one byte past an aligned address.
    let mut bytes = vec![0u8];
    bytes.extend_from_slice(offsets);
    let misaligned = Buffer::from_vec(bytes).slice(1);
    assert_ne!(misaligned.as_ptr() as usize % 4, 0);

    let data = unsafe {
        arrow::array::ArrayData::builder(DataType::Utf8)
            .len(strings.len())
            .add_buffer(misaligned)
            .add_buffer(strings.values().clone())
            .build_unchecked()
    };
    let array = FFI_ArrowArray::new(&data);
    let schema = FFI_ArrowSchema::try_from(&DataType::Utf8).unwrap();
    let imported = unsafe { from_ffi(array, &schema) }.unwrap();
    println!("{:?}", make_array(imported));
}

thread 'main' panicked at arrow-array-60.0.0/src/ffi.rs:495:27:
misaligned pointer dereference: address must be a multiple of 0x4 but is 0x1b16d906b4d
thread caused non-unwinding panic. aborting.

I think there is an API for this: https://docs.rs/arrow/latest/arrow/array/struct.ArrayData.html#method.align_buffers

technically speaking I think this is the bug:

    let data = unsafe {
        arrow::array::ArrayData::builder(DataType::Utf8)
            .len(strings.len())
            .add_buffer(misaligned)
            .add_buffer(strings.values().clone())
            .build_unchecked()
    };

Calling build_unchecked means that the caller guarantees that the buffers are valid arrow arrays, including alignment. I don't think this precondition is upheld

without build unchecked, wdyt?

// Utf8 ["a","bb","ccc"], offsets [0,1,3,6] stored one byte off alignment
let ffi: FFI_ArrowArray = unsafe { ptr::read((&raw_c_array as *const ArrowArray).cast()) };
let imported = unsafe { from_ffi(ffi, &FFI_ArrowSchema::try_from(&DataType::Utf8)?) }?;
// panicked at arrow-array-60.0.0/src/ffi.rs:495:27: misaligned pointer dereference

let imported = unsafe { from_ffi(ffi, &FFI_ArrowSchema::try_from(&DataType::Utf8)?) }?;

https://docs.rs/arrow/latest/arrow/array/ffi/fn.from_ffi.html

This struct assumes that the incoming data agrees with the C data interface.

Since the spec says this data needs to be aligned and you call unsafe you assert that it is (which is is not) I don't think this is a bug

I think the idea is if you need to read data from a source that produces unaligned buffers you should use the align_buffers method (and/or fix the producer)

@alamb

14. > let imported = unsafe { from_ffi(ffi, &FFI_ArrowSchema::try_from(&DataType::Utf8)?) }?;

 https://docs.rs/arrow/latest/arrow/array/ffi/fn.from_ffi.html
 > This struct assumes that the incoming data agrees with the C data interface.
 
 
 Since the spec says this data needs to be aligned and you call `unsafe` you assert that it is (which is is not) I don't think this is a bug
 I think the idea is if you need to read data from a source that produces unaligned buffers you should use the `align_buffers` method (and/or fix the producer)

Quoting the docs:

The C data interface spec says alignment is "recommended, but not required" and consumers "MAY decide not to support unaligned memory", so I read unaligned input as valid but refusable.

And since from_ffi already calls align_buffers (#10030), it seems to intend to support it (?). The only issue is that buffer_len reads the offsets before that runs. A read_unaligned there, or an error instead of the deref, would make it consistent.

thanks for your time and help on this matter @alamb

And since from_ffi already calls align_buffers (#10030), it seems to intend to support it (?). The only issue is that buffer_len reads the offsets before that runs. A read_unaligned there, or an error instead of the deref, would make it consistent.

Sounds reasonable to me

I think the idea is if you need to read data from a source that produces unaligned buffers you should use the align_buffers method (and/or fix the producer)

Fixing the producer is not enough, because you should not trust producers. They might be attackers.

The current panic on unaligned buffers gives us security: we don't trust the producer. Denial-of-service still possible because a panic can crash a process.

We are safe but not robust. The consumer could align (alloc needed) instead of panic'ing. That still means we should fix producers to avoid the allocation on the consumer side. Still a better situation than crashing on bad data that is easy to fix.

I feel I am missing something -- let's just have adbc set align_buffers and then the code will automatically align when needed (by copying).

The consumer could align (alloc needed) instead of panic'ing.

That is what align_buffers does 🤔

i think #10030 and #12010 resolve this issue

i think #10030 and #12010 resolve this issue

They do! I didn't realize these PRs existed.

...let's just have adbc set align_buffers and then the code will automatically align when needed (by copying).

Thanks! I will open a PR for that.

This PR can be closed because #10030 did what I proposed in my (closed) PR that I had suggested as a fix.

I opened a PR on arrow-adbc. We don't even have to merge it, because there is no need to even pass an option to enable the re-aligning behavior. Which is great!

apache/arrow-adbc#5000

fix(ffi): read byte array offsets unaligned in buffer_len (apache#12010)

Navigation Menu

Repository navigation

Array data imported through the FFI might contain unaligned buffers #7136

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Array data imported through the FFI might contain unaligned buffers #7136

Description

Activity

tustvold commented on Feb 15, 2025

alamb commented on May 11, 2026

AndreaBozzo commented on Oct 27, 2026

alamb commented on Oct 27, 2026

alamb commented on Oct 27, 2026

AndreaBozzo commented on Oct 27, 2026

alamb commented on Oct 29, 2026

AndreaBozzo commented on Oct 29, 2026

alamb commented on Oct 29, 2026

felipecrv commented on Nov 4, 2026

alamb commented on Nov 4, 2026

Jefffrey commented on Nov 5, 2026

felipecrv commented on Nov 7, 2026

felipecrv commented on Nov 7, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions