drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Venki Korukanti (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (DRILL-4415) Minimum parallelization width not respected in group scans
Date Wed, 24 Feb 2016 19:30:18 GMT

    [ https://issues.apache.org/jira/browse/DRILL-4415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15163561#comment-15163561
] 

Venki Korukanti edited comment on DRILL-4415 at 2/24/16 7:29 PM:
-----------------------------------------------------------------

The issue here: GroupScan.enforceWidth is not overridden, UnionExchange is removed as part
of the ExcessibleExchangeRemover. So there only one fragment which has GroupScan->Project->Screen.
As the Screen has the max parallelization of 1, it overrides the GroupScan min parallelization.


It seems GroupScan.enforceWidth API is unnecessary, instead ExcessibleExchangeRemover should
look at GroupScan.minParallelization. Also in SimpleParallelizer we should throw exceptions
if encounter a situation where max parallelization of operator overrides min parallelization
of another operator in the same fragment.


was (Author: vkorukanti):
The issue here: GroupScan.enforceWidth is not overridden, UnionExchange is removed as part
of the ExcessibleExchangeRemover. So there only one fragment which has GroupScan->Project->Screen.
As the Screen has the max parallelization of 1, it overrides the GroupScan min parallelization.


It seems GroupScan.enforceWidth API is unnecessary, instead ExcessibleExchangeRemover should
look at GroupScan.minParallelization. Also in SimpleParallelizer we should through exceptions
if encounter a situation where max parallelization of operator overrides min parallelization
of another operator in the same fragment.

> Minimum parallelization width not respected in group scans
> ----------------------------------------------------------
>
>                 Key: DRILL-4415
>                 URL: https://issues.apache.org/jira/browse/DRILL-4415
>             Project: Apache Drill
>          Issue Type: Bug
>          Components:  Server
>            Reporter: MinJi Kim
>
> Even if the minimum parallelization widith is set to > 1 value, this value is not
always respected.
> For example, in AggPruleBase, the decision to do two phase aggregation (create2PhasePlan)
only takes into account the number of rows.  So, if table is small enough, even if the width
is set to > 1 value, the plan will ignore the minimum width request. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message