drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Julian Hyde <jh...@apache.org>
Subject Re: Drill: Memory Spilling for the Hash Aggregate Operator
Date Sat, 14 Jan 2017 07:00:52 GMT
The attachment didn’t come through. I’m hoping that you settled on a “hybrid” hash
algorithm that can write to disk, or write to memory, and the cost of discovering that is
wrong is not too great. With Goetz Graefe’s hybrid hash join (which can be easily adapted
to hybrid hash aggregate) if the input ALMOST fits in memory you could process most of it
in memory, then revisit the stuff you spilled to disk.

> On Jan 13, 2017, at 7:46 PM, Boaz Ben-Zvi <bben-zvi@mapr.com> wrote:
> 
>  Hi Drill developers,
> 
>      Attached is a document describing the design for memory spilling implementation
for the Hash Aggregate operator.
> 
>      Please send me any comments or questions,
> 
>         -- Boaz


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message