arrow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dimitris Lekkas <dlekk...@gmail.com>
Subject Re: FPGA support for Apache Arrow
Date Mon, 01 Apr 2019 11:01:05 GMT
Alright, I think I understand your suggestion. It's like defining a
SharedMemoryPool for example where we implement our desired
way of memory allocation (with page-alignment + mem mapping).

Thanks!

On Mon, Apr 1, 2019 at 1:34 PM Antoine Pitrou <solipsis@pitrou.net> wrote:

> On Mon, 1 Apr 2019 10:25:26 +0300
> Dimitris Lekkas <dlekkasp@gmail.com> wrote:
> > Hey Antoine,
> >
> > Regarding the C++ side, do you consider variable alignment for buffers
> > a valuable addition? Would you be interested for such a PR?
>
> I think that it you have specific alignment requirements (and perhaps
> other requirements: memory mapping, etc.), the recommended approach is
> that you define your own MemoryPool implementation and pass it around
> to Arrow functions that allocate memory.
>
> Regards
>
> Antoine.
>
>
>
> > Finally, I
> > want to ask if I should be discussing such issues on Jira instead of
> > dev mailing list.
> >
> > Thanks for the replies, despite you being busy with the release
> > of 0.13.0.
> >
> > Dimitris
> >
> > On Mon, Apr 1, 2019 at 12:15 AM Antoine Pitrou <antoine@python.org>
> wrote:
> >
> > >
> > > Hi Dimitris,
> > >
> > > Le 31/03/2019 à 23:09, Dimitris Lekkas a écrit :
> > > >
> > > > Correct me if I'm wrong, but it doesn't seem to me that allocating
> size
> > > of
> > > > 4KB
> > > > multiple would equivalently mean that the starting address of data
> is
> > > also
> > > > a
> > > > multiple of 4KB.
> > >
> > > You're right.
> > >
> > > > For example, in the Java implementation when an ArrowBuf is
> allocated we
> > > > later slice it to allocate both a data buffer and validity buffer.
> > >
> > > That's an interesting approach.  The C++ side allocates separate memory
> > > areas for validity and raw data.
> > >
> > > > Would it help if I
> > > > opened a PR with our workaround for page-alignment in Java?
> > >
> > > The Java maintainers would have to answer.
> > >
> > > Regards
> > >
> > > Antoine.
> > >
> >
>
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message