hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Christian Kumpe <christian+Mailingli...@kumpe.de>
Subject Steps in execution time
Date Mon, 11 Apr 2011 07:29:49 GMT

I'm doing some measurement on hadoop's execution time for my theses. I
discovered some steps in the jobs execution time when raising the
mappers' execution time continuously.
Here is a plot of the execution times with 1, 30 and 60 parallel
executing mappers:

The cluster consists of 10 nodes and an overall map task capacity of 60.
I'm using hadoop 0.20.2.

The many small steps about 1 second are probably caused by the polling
interval of the JobClient in monitorAndPrintJob(..) which is about 1

But can anyone give me a hint, what's causing the bigger 3 second steps?
I suppose it's a polling interval, too. But where? In the Jobtracker? In
the Tasktrackers?

Thanks for some help and comments,

View raw message