drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steven Phillips (JIRA)" <j...@apache.org>
Subject [jira] [Created] (DRILL-2728) Merge spill files in ExternalSort
Date Thu, 09 Apr 2015 01:11:12 GMT
Steven Phillips created DRILL-2728:
--------------------------------------

             Summary: Merge spill files in ExternalSort
                 Key: DRILL-2728
                 URL: https://issues.apache.org/jira/browse/DRILL-2728
             Project: Apache Drill
          Issue Type: Improvement
            Reporter: Steven Phillips
            Assignee: Steven Phillips


When external sort spills, it first merges all of the in-memory batches, and then keeps one
batch in memory, while spilling the rest to disk. It's possible that we will still run out
of memory, due to the single batch held in memory for each spill. When the memory due to these
batches becomes too large, we should combine the spill files.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message