Skip to content

MutableArrayData::new expects a large alignment for IntervalMonthDayNanoType arrays #5917

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
kawadakk opened this issue Jun 19, 2024 · 0 comments
Labels

Comments

@kawadakk
Copy link
Contributor

Describe the bug
MutableArrayData::new panics if you pass an IntervalMonthDayNanoType array not aligned to 16 bytes. IntervalMonthDayNanoType only requires 8-byte alignment as of #5769.

To Reproduce
Add this to arrow/test/array_transform.rs and run cargo test -p arrow test_primitive_interval_month_day_nano:

#[test]
fn test_primitive_interval_month_day_nano() {
    let ty = DataType::Interval(arrow_schema::IntervalUnit::MonthDayNano);

    for i in 0..100 {
        // Construct an `IntervalMonthDayNanoType` array with a varying address
        let mut buffer = arrow_buffer::MutableBuffer::new(0);
        buffer.extend_zeros(i);
        buffer.push(IntervalMonthDayNanoType::make_value(0, 0, 0));
        let buffer = Buffer::from(buffer).slice(i);

        // Convert it into an `ArrayData`, re-aligning the buffer as necessary
        let b = ArrayData::builder(ty.clone())
            .add_buffer(buffer.into())
            .build_aligned()
            .unwrap();

        // Pass it to `MutableArrayData::new`
        MutableArrayData::new(vec![&b], false, 0);
    }
}

The test fails, producing this:

thread 'test_primitive_interval_month_day_nano' panicked at [...]/arrow-rs/arrow-buffer/src/buffer/immutable.rs:243:9:
assertion failed: prefix.is_empty() && suffix.is_empty()
stack backtrace:
   0: rust_begin_unwind
             at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081/library/std/src/panicking.rs:652:5
   1: core::panicking::panic_fmt
             at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081/library/core/src/panicking.rs:72:14
   2: core::panicking::panic
             at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081/library/core/src/panicking.rs:146:5
   3: arrow_buffer::buffer::immutable::Buffer::typed_data
             at [...]/arrow-rs/arrow-buffer/src/buffer/immutable.rs:243:9
   4: arrow_data::data::ArrayData::buffer
             at [...]/arrow-rs/arrow-data/src/data.rs:560:10
   5: arrow_data::transform::primitive::build_extend
             at [...]/arrow-rs/arrow-data/src/transform/primitive.rs:26:18
   6: arrow_data::transform::build_extend
             at [...]/arrow-rs/arrow-data/src/transform/mod.rs:215:59
   7: arrow_data::transform::MutableArrayData::with_capacities::{{closure}}
             at [...]/arrow-rs/arrow-data/src/transform/mod.rs:612:44
   8: core::iter::adapters::map::map_fold::{{closure}}
             at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081/library/core/src/iter/adapters/map.rs:89:28
   9: <core::slice::iter::Iter<T> as core::iter::traits::iterator::Iterator>::fold
             at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081/library/core/src/slice/iter/macros.rs:230:27
  10: <core::iter::adapters::map::Map<I,F> as core::iter::traits::iterator::Iterator>::fold
             at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081/library/core/src/iter/adapters/map.rs:129:9
  11: core::iter::traits::iterator::Iterator::for_each
             at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081/library/core/src/iter/traits/iterator.rs:818:9
  12: alloc::vec::Vec<T,A>::extend_trusted
             at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081/library/alloc/src/vec/mod.rs:3098:17
  13: <alloc::vec::Vec<T,A> as alloc::vec::spec_extend::SpecExtend<T,I>>::spec_extend
             at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081/library/alloc/src/vec/spec_extend.rs:26:9
  14: <alloc::vec::Vec<T> as alloc::vec::spec_from_iter_nested::SpecFromIterNested<T,I>>::from_iter
             at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081/library/alloc/src/vec/spec_from_iter_nested.rs:62:9
  15: <alloc::vec::Vec<T> as alloc::vec::spec_from_iter::SpecFromIter<T,I>>::from_iter
             at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081/library/alloc/src/vec/spec_from_iter.rs:33:9
  16: <alloc::vec::Vec<T> as core::iter::traits::collect::FromIterator<T>>::from_iter
             at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081/library/alloc/src/vec/mod.rs:2972:9
  17: core::iter::traits::iterator::Iterator::collect
             at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081/library/core/src/iter/traits/iterator.rs:2004:9
  18: arrow_data::transform::MutableArrayData::with_capacities
             at [...]/arrow-rs/arrow-data/src/transform/mod.rs:612:18
  19: arrow_data::transform::MutableArrayData::new
             at [...]/arrow-rs/arrow-data/src/transform/mod.rs:342:9
  20: array_transform::test_primitive_interval_month_day_nano
             at ./tests/array_transform.rs:159:9
  21: array_transform::test_primitive_interval_month_day_nano::{{closure}}
             at ./tests/array_transform.rs:143:44
  22: core::ops::function::FnOnce::call_once
             at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081/library/core/src/ops/function.rs:250:5
  23: core::ops::function::FnOnce::call_once
             at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

Expected behavior
The test should pass because we passed a properly-aligned ArrayData to MutableArrayData::new.

Additional context

@kawadakk kawadakk added the bug label Jun 19, 2024
@kawadakk kawadakk changed the title MutableArrayData::new expects a large alignment for IntervalMonthDayNanoType arrays MutableArrayData::new expects a non-standard alignment for IntervalMonthDayNanoType arrays Jun 19, 2024
@kawadakk kawadakk changed the title MutableArrayData::new expects a non-standard alignment for IntervalMonthDayNanoType arrays MutableArrayData::new expects a large alignment for IntervalMonthDayNanoType arrays Jun 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant