drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Paul Rogers (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-5503) Disabling exchanges results in "Unable to allocate sv2 buffer" error within the managed external sort code
Date Sat, 17 Jun 2017 01:10:00 GMT

    [ https://issues.apache.org/jira/browse/DRILL-5503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16052613#comment-16052613
] 

Paul Rogers commented on DRILL-5503:
------------------------------------

One change that was made is to log an error message in the case of exceptionally low memory
to help diagnose this problem if it occurs in the field.

{code}
Insufficient memory to merge two batches. \n
Incoming batch size: 269,082,624, \n
available memory: 62,600,000
{code}


> Disabling exchanges results in "Unable to allocate sv2 buffer" error within the managed
external sort code
> ----------------------------------------------------------------------------------------------------------
>
>                 Key: DRILL-5503
>                 URL: https://issues.apache.org/jira/browse/DRILL-5503
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Execution - Flow
>    Affects Versions: 1.10.0
>            Reporter: Rahul Challapalli
>            Assignee: Paul Rogers
>         Attachments: drill5503.log, failure.sys.drill, success.sys.drill
>
>
> Setup :
> {code}
> git.commit.id.abbrev=1e0a14c
> No of drillbits : 1
> DRILL_MAX_DIRECT_MEMORY="32G"
> DRILL_MAX_HEAP="4G"
> {code}
> The below successfully completes
> {code}
> ALTER SESSION SET `exec.sort.disable_managed` = false;
> alter session set `planner.width.max_per_node` = 1;
> alter session set `planner.memory.max_query_memory_per_node` = 62600000;
> alter session set `planner.width.max_per_query` = 17;
> select count(*) from (select * from dfs.`/drill/testdata/resource-manager/5kwidecolumns_500k.tbl`
order by columns[0]) d where d.columns[0] = '4041054511';
> +---------+
> | EXPR$0  |
> +---------+
> | 0       |
> +---------+
> 1 row selected (814.104 seconds)
> {code}
> However if I disable exchanges, I get the following error
> {code}
> alter session set `planner.disable_exchanges` = false;
> select count(*) from (select * from dfs.`/drill/testdata/resource-manager/5kwidecolumns_500k.tbl`
order by columns[0]) d where d.columns[0] = '4041054511';
> +---------+
> | EXPR$0  |
> +---------+
> | 0       |
> +---------+
> 1 row selected (814.104 seconds)
> {code}
> I attached the profile and the log file. The data set used is too large to attach here.




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message