impala-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Armstrong <tarmstr...@cloudera.com>
Subject Re: Impala Sorter just sort small partition?
Date Fri, 04 Aug 2017 16:18:48 GMT
The Sorter does a 3-level hybrid sort with merge sort, quicksort and
insertion sort.

SortHelper implements a 2-level hybrid in-memory sort. It fully sorts an
arbitrarily sized in-memory input. E.g. if 'begin' and 'end' point to the
begin and end of the sorted run, it will sort the full run. It does
quicksort recursively then switches to insertion sort once the partitions
are less than INSERTION_THRESHOLD = 16.

Sorter also supports an external merge sort - if the full input doesn't fit
in memory, it sorts in-memory runs with SortHelper() then does merge sort
with the sorted runs.

On Thu, Aug 3, 2017 at 11:13 PM, 俊杰陈 <cjjnjust@gmail.com> wrote:

> Hi
> I'm looking Sorter.cc and found that Sorter::SortHelper just sort smaller
> partition. Is there anything I missed?
>
> --
> Thanks & Best Regards
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message