drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kunal Khatua (JIRA)" <j...@apache.org>
Subject [jira] [Closed] (DRILL-4121) External Sort may not spill if above a receiver
Date Wed, 22 Jun 2016 23:43:16 GMT

     [ https://issues.apache.org/jira/browse/DRILL-4121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Kunal Khatua closed DRILL-4121.
-------------------------------

Setting direct memory to 10GB and altered the session by setting `planner.memory.max_query_memory_per_node`=21474836
(20MB).
Forcing an external sort, the Drillbits began to spill data to disk with no significant increase
in the Drillbit memory footprint. 
Verified on Drill 1.6.0.

> External Sort may not spill if above a receiver
> -----------------------------------------------
>
>                 Key: DRILL-4121
>                 URL: https://issues.apache.org/jira/browse/DRILL-4121
>             Project: Apache Drill
>          Issue Type: Bug
>    Affects Versions: 1.1.0
>            Reporter: Deneche A. Hakim
>            Assignee: Kunal Khatua
>             Fix For: 1.5.0
>
>
> If external sort is above a receiver, all received batches will contain non root buffers.
Sort operator doesn't account for non root buffers when estimating how much memory and if
it needs to spill to disk. This may delay the spill and cause the corresponding Drillbit to
use large amounts of memory



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message