hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jinyan Xu <Jinyan...@exar.com>
Subject when hadoop report job finished?
Date Wed, 15 Feb 2012 03:39:57 GMT

Hi all,

I run terasort test on pseudo distributed, I  found the one reduce job finished time is shorter
than the four reduces job. I used tool to search why.
Use nmon to monitor the disk read/write operation, I found a phenomena, when one reduce job
report finished but the disk write data operation is still sustained for a long time. For
four reduce jobs, when job report finished the disk read/write operation is done.

In a nutshell, one reduce job finished time add disk write operation time equal to four reduces
job finished time.

So I think this is the reason why one reduce job time is shorter than the four reduces job,
finally, I want to ask why that happened?  What is the reduce job finished mark?

The information and any attached documents contained in this message
may be confidential and/or legally privileged. The message is
intended solely for the addressee(s). If you are not the intended
recipient, you are hereby notified that any use, dissemination, or
reproduction is strictly prohibited and may be unlawful. If you are
not the intended recipient, please contact the sender immediately by
return e-mail and destroy all copies of the original message.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message