hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chao Sun (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HIVE-16337) HoS: use separate config for mapjoin hash table size limit rather than hive.auto.convert.join.noconditionaltask.size
Date Thu, 30 Mar 2017 17:24:41 GMT
Chao Sun created HIVE-16337:
-------------------------------

             Summary: HoS: use separate config for mapjoin hash table size limit rather than
hive.auto.convert.join.noconditionaltask.size
                 Key: HIVE-16337
                 URL: https://issues.apache.org/jira/browse/HIVE-16337
             Project: Hive
          Issue Type: Improvement
          Components: Spark
            Reporter: Chao Sun


HoS currently share the same config {{hive.auto.convert.join.noconditionaltask.size}} with
MR, however it interprets it differently: in MR this is the file size of the input, while
in Spark it is the data size. This could cause Spark to be over conservative in many mapjoin
optimizations. I suggest we use a separate config for HoS for this purpose.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message