hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Iman E <hadoop_...@yahoo.com>
Subject Performance of mappers
Date Fri, 05 Aug 2011 18:02:04 GMT
Hello all,
I have a question regarding the mappers. I can see from the logs that the start time of the
mapper is different from start time of logging. I am having a problem because that time difference
sometimes is few seconds, but other times it is 
 
For example, one mapper that is supposed to read 65 MB. Its start time is 12:30:53 whereis
the logging start time is 12:33:01 and the end time is 12:33:20. All the laoded data are local
to the same rack. 
In a perfect run, these numbers are as follows: the start time is 18:15:45, logging start
time: 18:15:48, and end time: 18:16:02.
 
 
I am running a job of more than 2400 mapper. Because of the above problem, instead of the
job taking 15-20 mins  on 14 machine ( it happened in few runs), other times it is taking
more than 70 mins. Any suggestions how to fix this problem or what could possibly be causing
it.
 
Thanks,
Iman
Mime
View raw message