arrow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Corey Nolet <cjno...@gmail.com>
Subject Re: Question about mutability
Date Wed, 24 Feb 2016 19:29:43 GMT
Thanks for the fast response Jacques. But that's not to say it would be
impossible to continue to create new data structures and de-allocate others
in order to add and remove data, correct?

On Wed, Feb 24, 2016 at 2:26 PM, Jacques Nadeau <jacques@apache.org> wrote:

> Both are possible. Arrow doesn't force one or the other. You can copy
> between memory spaces and you still benefit from the same representation
> (avoiding ser/deserialization). However, the target is definitely
> leveraging shared memory.
>
> The way memory spaces are shared still needs more definition. My
> expectation is that in most cases the consumer of the structures will
> provide memory for purposes of population by the provider. That being said,
> we'll need to flip this in the case of shared in-memory arrow caching
> layers as those get created.
>
> Back to the original question about mutability:
>
> These structures are designed to be write once, read many. So once they are
> constructed, they are immutable.
>
>
>
>
> On Wed, Feb 24, 2016 at 11:17 AM, Andrew Brust <
> andrew.brust@bluebadgeinsights.com> wrote:
>
> > Hmm...that's not exactly how Jaques described things to me when he
> briefed
> > me on Arrow ahead of the announcement.
> >
> > -----Original Message-----
> > From: Zhe Zhang [mailto:zhz@apache.org]
> > Sent: Wednesday, February 24, 2016 2:08 PM
> > To: dev@arrow.apache.org
> > Subject: Re: Question about mutability
> >
> > I don't think one application/process's memory space will be made
> > available to other applications/processes. It's fundamentally hard for
> > processes to share their address spaces.
> >
> > IIUC, with Arrow, when application A shares data with application B, the
> > data is still duplicated in the memory spaces of A and B. It's just that
> > data serialization/deserialization are much faster with Arrow (compared
> > with Protobuf).
> >
> > On Wed, Feb 24, 2016 at 10:40 AM Corey Nolet <cjnolet@gmail.com> wrote:
> >
> > > Forgive me if this question seems ill-informed. I just started looking
> > > at Arrow yesterday. I looked around the github a tad.
> > >
> > > Are you expecting the memory space held by one application to be
> > > mutable by that application and made available to all applications
> > > trying to read the memory space?
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message