arrow-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Nuernberger <ch...@techascent.com>
Subject Re: Plasma store implementation status across client libraries
Date Mon, 04 Jan 2021 15:55:23 GMT
Could simply saving the arrow file in streaming mode to shared memory and
then mmap-ing the result in each language solve your problem ?  Plasma
seems to me to be a layer on top of basic mmap operations; as long as you
have shared memory and mmap then you can have multiple processes talking to
the same logical block of memory.

On Mon, Jan 4, 2021 at 8:27 AM Thomas Browne <thomas@crvm.io> wrote:

> I am hoping to use the Apache Arrow project for cross-language numerical
> computation, and for that the shared-memory idea is very powerful. Am I
> correct that the Plasma Store is the enabling technology for this,
> especially for soft real-time computation (ie not moving to parquet or
> any file-based sharing system)?
>
> Is that the case? And if so, then I'm wondering which client libraries,
> other than Python (and I assume C[++]), implement the Plasma Store. This
> table doesn't feature a row for Plasma:
>
> https://arrow.apache.org/docs/status.html
>
> and I can't seem to find any reference to the Plasma store in the Julia,
> R, or Javascript libraries.
>
> https://arrow.apache.org/docs/r/
>
> https://arrow.apache.org/docs/js/
>
> https://arrow.juliadata.org/stable/
>
>
> Thank you,
>
> Thomas
>
>
>

Mime
View raw message