arrow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Edmon Begoli <ebeg...@gmail.com>
Subject Re: Intel CPU architecture
Date Wed, 02 Mar 2016 22:56:44 GMT
Would you guys be interested in perhaps having a Hangout with my team from
JICS/NICS?

We have some major experts and research thrusts in this area (code
optimizations for Intel chipsets, MKL and other kernels, memory/IO
optimizations, etc)

We are a research shop. People just get excited over things like this.


On Wednesday, March 2, 2016, Wes McKinney <wes@cloudera.com> wrote:

> hi Edmon,
>
> Since Arrow arrays are arranged with like-data in contiguous memory
> regions (for example, in an array of strings, the UTF8 bytes are all
> laid out in contiguous memory -- see
> https://github.com/apache/arrow/blob/master/format/Layout.md), it is
> cache-friendly for scan operations and amenable to SIMD computations
> (for example: SIMD-accelerated hash functions). This is especially
> important for nested data, as all the "leaf nodes" in a nested
> structure generally contain contiguous memory.
>
> We have not started doing this yet, but it would be useful to begin
> assembling kernels that use CPU intrinsics (and SSE/AVX) in the Arrow
> codebase, and to make them easily accessible. Having a standard
> benchmark suite and other performance experimentation tools available
> for users to run on their hardware would also be great.
>
> best,
> Wes
>
> On Wed, Mar 2, 2016 at 10:21 AM, Edmon Begoli <ebegoli@gmail.com
> <javascript:;>> wrote:
> > Hey folks,
> >
> > How could I get more details on what and how Arrow uses Intel CPUs for
> > whatever computational advantage?
> >
> > At JICS, we run very large experimental Intel HPC systems, and I would
> like
> > to learn how can we possibly run some interesting Arrow on Intel CPUs
> > experiments.
> >
> > Thank you,
> > Edmon
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message