arrow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Antoine Pitrou <anto...@python.org>
Subject Re: [DISCUSS] C-level in-process array protocol
Date Tue, 01 Oct 2019 20:28:30 GMT

As currently designed, it's entirely in-process.  Shared memory with
buffer lifetime handling is taking care of by something like Plasma.

Regards

Antoine.


Le 01/10/2019 à 22:22, Jed Brown a écrit :
> I'd just like to chime in with the use case of in-situ data analysis for
> simulations.  This domain tends to be cautious with dependencies and
> there is a lot of C and Fortran, but the in-situ analysis tools will
> preferably reside in separate processes while sharing memory via shared
> memory (/dev/shm or MPI_Win_allocate_shared).  An in-memory protocol
> that holds raw pointers would be problematic because they are typically
> in different virtual address spaces when shared between processes.  I
> think this is a potential application for a C interface with lean
> dependencies, but it wouldn't be useful if it can't be shared
> out-of-process.

Mime
View raw message