drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steven Phillips (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (DRILL-2728) Merge spill files in ExternalSort
Date Mon, 13 Apr 2015 23:05:12 GMT

     [ https://issues.apache.org/jira/browse/DRILL-2728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Steven Phillips updated DRILL-2728:
-----------------------------------
    Fix Version/s: 0.9.0

> Merge spill files in ExternalSort
> ---------------------------------
>
>                 Key: DRILL-2728
>                 URL: https://issues.apache.org/jira/browse/DRILL-2728
>             Project: Apache Drill
>          Issue Type: Improvement
>          Components: Execution - Relational Operators
>            Reporter: Steven Phillips
>            Assignee: Venki Korukanti
>             Fix For: 0.9.0
>
>         Attachments: DRILL-2728.patch
>
>
> When external sort spills, it first merges all of the in-memory batches, and then keeps
one batch in memory, while spilling the rest to disk. It's possible that we will still run
out of memory, due to the single batch held in memory for each spill. When the memory due
to these batches becomes too large, we should combine the spill files.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message