hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Siddharth Seth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-16690) Configure Tez cartesian product edge based on LLAP cluster size
Date Mon, 17 Jul 2017 21:21:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-16690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16090585#comment-16090585
] 

Siddharth Seth commented on HIVE-16690:
---------------------------------------

+1 for the addendum. If cross product does end up getting invoked, is it possible to log the
configured value for parallelism? This can be in a follow up jira. Also, I think this clusterstate
lookup is very avoidable if cross product is not invoked. Don't need to invoke the ZK registry
for each query (even though it does cache records for 2 minutes)

> Configure Tez cartesian product edge based on LLAP cluster size
> ---------------------------------------------------------------
>
>                 Key: HIVE-16690
>                 URL: https://issues.apache.org/jira/browse/HIVE-16690
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Zhiyuan Yang
>            Assignee: Zhiyuan Yang
>         Attachments: HIVE-16690.1.patch, HIVE-16690.addendum.patch
>
>
> In HIVE-14731 we are using default value for target parallelism of fair cartesian product
edge. Ideally this should be set according to cluster size. In case of LLAP it's pretty easy
to get cluster size, i.e., number of executors.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message