hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christian Kumpe <christian+Mailingli...@kumpe.de>
Subject Steps in execution time
Date Mon, 11 Apr 2011 07:29:49 GMT
Hi,

I'm doing some measurement on hadoop's execution time for my theses. I
discovered some steps in the jobs execution time when raising the
mappers' execution time continuously.
Here is a plot of the execution times with 1, 30 and 60 parallel
executing mappers:
http://baumbart.drhuim.de/executiontime.gif

The cluster consists of 10 nodes and an overall map task capacity of 60.
I'm using hadoop 0.20.2.

The many small steps about 1 second are probably caused by the polling
interval of the JobClient in monitorAndPrintJob(..) which is about 1
second.

But can anyone give me a hint, what's causing the bigger 3 second steps?
I suppose it's a polling interval, too. But where? In the Jobtracker? In
the Tasktrackers?

Thanks for some help and comments,
  Christian



Mime
View raw message