arrow-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Nuernberger <ch...@techascent.com>
Subject Re: Plasma store implementation status across client libraries
Date Mon, 04 Jan 2021 15:55:44 GMT
Julia, Python, and R all have some support for mmap operations.

On Mon, Jan 4, 2021 at 8:55 AM Chris Nuernberger <chris@techascent.com>
wrote:

> Could simply saving the arrow file in streaming mode to shared memory and
> then mmap-ing the result in each language solve your problem ?  Plasma
> seems to me to be a layer on top of basic mmap operations; as long as you
> have shared memory and mmap then you can have multiple processes talking to
> the same logical block of memory.
>
> On Mon, Jan 4, 2021 at 8:27 AM Thomas Browne <thomas@crvm.io> wrote:
>
>> I am hoping to use the Apache Arrow project for cross-language numerical
>> computation, and for that the shared-memory idea is very powerful. Am I
>> correct that the Plasma Store is the enabling technology for this,
>> especially for soft real-time computation (ie not moving to parquet or
>> any file-based sharing system)?
>>
>> Is that the case? And if so, then I'm wondering which client libraries,
>> other than Python (and I assume C[++]), implement the Plasma Store. This
>> table doesn't feature a row for Plasma:
>>
>> https://arrow.apache.org/docs/status.html
>>
>> and I can't seem to find any reference to the Plasma store in the Julia,
>> R, or Javascript libraries.
>>
>> https://arrow.apache.org/docs/r/
>>
>> https://arrow.apache.org/docs/js/
>>
>> https://arrow.juliadata.org/stable/
>>
>>
>> Thank you,
>>
>> Thomas
>>
>>
>>

Mime
View raw message