drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Venki Korukanti (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-4165) IllegalStateException in MergeJoin for a query against TPC-DS data
Date Mon, 07 Dec 2015 21:54:11 GMT

    [ https://issues.apache.org/jira/browse/DRILL-4165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15045819#comment-15045819
] 

Venki Korukanti commented on DRILL-4165:
----------------------------------------

[~aah] Seems like this is a regression caused in 1.3. I think it is better to include this
fix in 1.4, otherwise we end up with MergeJoin (when used) generating huge batches which could
potentially cause other issues (memory, rpc delays etc.).

> IllegalStateException in MergeJoin for a query against TPC-DS data
> ------------------------------------------------------------------
>
>                 Key: DRILL-4165
>                 URL: https://issues.apache.org/jira/browse/DRILL-4165
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Execution - Relational Operators
>    Affects Versions: 1.4.0
>            Reporter: Aman Sinha
>            Assignee: amit hadke
>
> I am seeing the following on the 1.4.0 branch. 
> {noformat}
> 0: jdbc:drill:zk=local> alter session set `planner.enable_hashjoin` = false;
> ..
> 0: jdbc:drill:zk=local> select count(*) from dfs.`tpcds/store_sales` ss1, dfs.`tpcds/store_sales`
ss2 where ss1.ss_customer_sk = ss2.ss_customer_sk and ss1.ss_store_sk = 1 and ss2.ss_store_sk
= 2;
> Error: SYSTEM ERROR: IllegalStateException: Incoming batch [#55, MergeJoinBatch] has
size 1984616, which is beyond the limit of 65536
> Fragment 0:0
> [Error Id: 18bf00fe-52d7-4d84-97ec-b04a035afb4e on 192.168.1.103:31010]
>   (java.lang.IllegalStateException) Incoming batch [#55, MergeJoinBatch] has size 1984616,
which is beyond the limit of 65536
>     org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next():305
>     org.apache.drill.exec.record.AbstractRecordBatch.next():119
>     org.apache.drill.exec.record.AbstractRecordBatch.next():109
>     org.apache.drill.exec.record.AbstractSingleRecordBatch.innerNext():51
>     org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext():132
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message