hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsha HN <99harsha.h....@gmail.com>
Subject Question on Hive Join performance
Date Wed, 22 Apr 2015 07:06:54 GMT
Hi All,



I went through below mentioned Facebook engineering page,

https://www.facebook.com/notes/facebook-engineering/join
-optimization-in-apache-hive/470667928919



I set following for auto conversion of joins,

set hive.auto.convert.join=true;

set hive.mapjoin.smalltable.filesize=1000000000;    (1GB)



I observed some queries performed 2X faster in MAP JOIN as opposed to
Common join

and also instances where MAP JOIN is 3X slower than Common Join.



Any thoughts on what might be slowing down MAP JOIN in some cases ?



I have 40 Node cluster, so I have huge RAM available.



Thanks,

Harsha

Mime
View raw message