hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sergey Shelukhin <ser...@hortonworks.com>
Subject Re: sql mapjoin very slow
Date Fri, 28 Aug 2015 01:21:36 GMT
Are you using MR and Tez? You could try optimized hash table in case of Tez, although it’s
supposed to improve memory, not necessarily perf.

Can you also share characteristics of the query and data? It is surprising to see so much
time for HashMap.get.

From: "r7raul1984@163.com<mailto:r7raul1984@163.com>" <r7raul1984@163.com<mailto:r7raul1984@163.com>>
Reply-To: user <user@hive.apache.org<mailto:user@hive.apache.org>>
Date: Thursday, August 27, 2015 at 18:03
To: user <user@hive.apache.org<mailto:user@hive.apache.org>>
Subject: sql mapjoin very slow


When I enable mapjoin ,I see Mapjoin task run very slow. My envrioment is hadoop 2.3.0 hive
1.1.0.

My attach is  one map hive log and this map's xprof log.

In map xprof log ,I see
Compiled + native Method
92.3% 643527 + 0 java.util.HashMap.get
2.8% 19856 + 0 java.util.HashMap.put
1.2% 8623 + 0 org.apache.hadoop.hive.ql.exec.persistence.HashMapWrapper$GetAdaptor.setFromRow
0.1% 953 + 0 org.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator._evaluate
0.1% 576 + 0 org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils.copyToStandardObject

________________________________
r7raul1984@163.com<mailto:r7raul1984@163.com>
Mime
View raw message