arrow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Antoine Pitrou <solip...@pitrou.net>
Subject Re: FPGA support for Apache Arrow
Date Mon, 01 Apr 2019 10:26:15 GMT
On Mon, 1 Apr 2019 10:25:26 +0300
Dimitris Lekkas <dlekkasp@gmail.com> wrote:
> Hey Antoine,
> 
> Regarding the C++ side, do you consider variable alignment for buffers
> a valuable addition? Would you be interested for such a PR?

I think that it you have specific alignment requirements (and perhaps
other requirements: memory mapping, etc.), the recommended approach is
that you define your own MemoryPool implementation and pass it around
to Arrow functions that allocate memory.

Regards

Antoine.



> Finally, I
> want to ask if I should be discussing such issues on Jira instead of
> dev mailing list.
> 
> Thanks for the replies, despite you being busy with the release
> of 0.13.0.
> 
> Dimitris
> 
> On Mon, Apr 1, 2019 at 12:15 AM Antoine Pitrou <antoine@python.org> wrote:
> 
> >
> > Hi Dimitris,
> >
> > Le 31/03/2019 à 23:09, Dimitris Lekkas a écrit :  
> > >
> > > Correct me if I'm wrong, but it doesn't seem to me that allocating size  
> > of  
> > > 4KB
> > > multiple would equivalently mean that the starting address of data is  
> > also  
> > > a
> > > multiple of 4KB.  
> >
> > You're right.
> >  
> > > For example, in the Java implementation when an ArrowBuf is allocated we
> > > later slice it to allocate both a data buffer and validity buffer.  
> >
> > That's an interesting approach.  The C++ side allocates separate memory
> > areas for validity and raw data.
> >  
> > > Would it help if I
> > > opened a PR with our workaround for page-alignment in Java?  
> >
> > The Java maintainers would have to answer.
> >
> > Regards
> >
> > Antoine.
> >  
> 




Mime
View raw message