hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sumit ghosh <sumi...@yahoo.com>
Subject Skewed Join
Date Fri, 07 Jun 2013 06:42:43 GMT
Hi,
 
I am hit by skewed Join, my last reducer is getting same number of Reduce input groups/records.
Reduce input groups                      432,446,942
Reduce shuffle bytes                  13,012,613,275
Reduce input records                     432,446,942  
 
Why is this happening? I have turned on skew join optimization:
hive.optimize.skewjoin=true;
hive.skewjoin.key=100000;
 
 
Thanks,
Sumit 
Mime
View raw message