arrow-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wes McKinney <>
Subject Re: Weld
Date Sat, 26 Nov 2016 20:53:40 GMT
I've always contended that building a sort of "runtime" for in-memory
column expressions was a natural next step after hardening Arrow as a
data structure and memory exchange mechanism. I'm hopeful that we'll
see some cross-pollination between projects like Weld and Arrow to
produce open source solutions that help drive more collaboration and
consolidation around shared low-level needs for building analytical
systems. We still have significant work to do to solidify Arrow as a
standard -- anyone all reading can do to help work against
fragmentation in data structures / memory representations would be
very much appreciated!

- Wes

On Wed, Nov 23, 2016 at 11:35 AM, Donald Foss <> wrote:
> I had that in my queue, and your watching and thoughts about it mean I need
> to watch it today ;)
> Thanks for the link to where you've played with abstraction before. I'm
> going to fork a copy and do some playing myself. I'm very interested in how
> universal it can be, and a library to go from expression to universal
> intermediate, them output the code in the language of one's choice. The
> optimizations could be performed in the intermediate layer, the output
> later, or both if there are specific code optimizations available after
> translation.
> -Donald
> On Nov 22, 2016 2:31 PM, Julien Le Dem
> wrote:

View raw message