arrow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Antoine Pitrou <anto...@python.org>
Subject Re: Help with memory alignment between Rust and C/Python
Date Fri, 18 Sep 2020 16:08:33 GMT

Le 18/09/2020 à 18:03, Jorge Cardoso Leitão a écrit :
> // panics with "memory not aligned"
> Buffer::from_raw_parts(address as *const u8, size, size)
> 
> I get an address such as `4624199872` (i64), but, when converted to `*const
> u8`, our rust implementation does not consider it to be memory aligned (at
> least in x86_64) and panics. Has anyone worked in this problem? Any hints
> on what I am doing wrong?

The Rust implementation should not panic on misaligned buffers.  64-byte
alignment is a recommendation, not a requirement.  Buffers allocated by
foreign systems such as Python / NumPy, or simply sliced on an arbitrary
offset, will not necessarily be aligned.

> AFAIK, our rust implementation requires different alignments, depending on
> the architecture (see memory.rs
> <https://github.com/apache/arrow/blob/master/rust/arrow/src/memory.rs>).

I am not a Rust expert, but I remember noticing the PR where this was
proposed and explaining that this was wrong.  Again: alignment is a
recommendation, not a requirement.  Making it mandatory *and*
platform-specific even sounds a bit perverse...

Regards

Antoine.

Mime
View raw message