drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Paul Rogers (JIRA)" <j...@apache.org>
Subject [jira] [Created] (DRILL-5008) Refactor, document and simplify ExternalSortBatch
Date Mon, 07 Nov 2016 18:01:58 GMT
Paul Rogers created DRILL-5008:

             Summary: Refactor, document and simplify ExternalSortBatch
                 Key: DRILL-5008
                 URL: https://issues.apache.org/jira/browse/DRILL-5008
             Project: Apache Drill
          Issue Type: Improvement
    Affects Versions: 1.8.0
            Reporter: Paul Rogers
            Priority: Minor

ExternalSortBatch provides a spillable sort operator for Drill. The code works fine, but can
be a hard to follow and understand. Make the following changes to improve ease-of-use for

1. Refactor the large methods into bite-sized chunks to aid understanding.
2. Provide additional explanation of the theory and operation of this operator.

The memory limit cases for the spill-needed test seem inconsistent:

For the test for in-memory sort:
    long currentlyAvailable =  popConfig.getMaxAllocation() - oAllocator.getAllocatedMemory();

For reaching the memory limit:
oAllocator.getAllocatedMemory() > .95 * oAllocator.getLimit()

That is, one uses `oAllocator.getLimit` ("the current maximum limit this allocator imposes"),
the other uses `popConfig.getMaxAllocation` (not documented, but the memory limit for the
operator as defined by the Foreman.)

This message was sent by Atlassian JIRA

View raw message