drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Deneche A. Hakim (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-4121) External Sort may not spill if above a receiver
Date Mon, 23 Nov 2015 17:12:11 GMT

    [ https://issues.apache.org/jira/browse/DRILL-4121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15022484#comment-15022484

Deneche A. Hakim commented on DRILL-4121:

In one case, multiple drillbits where using +40GB of direct memory (and actually allocated
+90GB from the system) yet the sort operator didn't spill to disk

> External Sort may not spill if above a receiver
> -----------------------------------------------
>                 Key: DRILL-4121
>                 URL: https://issues.apache.org/jira/browse/DRILL-4121
>             Project: Apache Drill
>          Issue Type: Bug
>    Affects Versions: 1.1.0
>            Reporter: Deneche A. Hakim
>            Assignee: Deneche A. Hakim
> If external sort is above a receiver, all received batches will contain non root buffers.
Sort operator doesn't account for non root buffers when estimating how much memory and if
it needs to spill to disk. This may delay the spill and cause the corresponding Drillbit to
use large amounts of memory

This message was sent by Atlassian JIRA

View raw message