hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gaurav Dasgupta <gdsay...@gmail.com>
Subject How to reduce total shuffle time
Date Tue, 28 Aug 2012 07:16:35 GMT
Hi,

I have run some large and small jobs and calculated the Total Shuffle Time
for the jobs. I can see that the Total Shuffle Time is almost half the
Total Time which was taken by the full job to complete.

My question, here, is that how can we decrease the Total Shuffle Time? And
doing so, what will be its effect on the Job?

Thanks,
Gaurav Dasgupta

Mime
View raw message