arrow-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Lamb <>
Subject Re: does rust API support memory mapped read/write ?
Date Sun, 15 Nov 2020 12:21:40 GMT
I think your conclusion that the Rust API doesn't support using mmap'd
files as a way to read/write arrow files.

In general, I suspect using mmap in Rust is a bit dicey (aka unsafe)  as
the normal Rust rules of ownership are hard to apply to chunks of memory
that can be (potentially) modified by different processes etc. It is
probably fine in most read only cases.

I am curious about your desire to use mmap for reading ipc streams (to
understand more if we should be looking into mmap support). As I understand
it, the ipc interface is designed for streaming reads/writes, and thus the
easy random access of mmap seems less important. Are you concerned about
Reader performance?


On Fri, Nov 13, 2020 at 12:43 AM comic fans <> wrote:

> Hello everyone , I'd like to use rust to read/write feather format
> files (by memory mapped file support) ,but rust API only accept Reader
> as input (arrow::ipc::reader::FileReader) , and I didn't find any
> memory map api/crate usage in arrow , does that mean currently rust
> native API didn't support memory mapped read/write ?  (I can do this
> by arrow C++ API, but I'd like to do this through native rust API ) .

View raw message