hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From abhishek sharma <absha...@usc.edu>
Subject Re: Steps in execution time
Date Mon, 11 Apr 2011 16:38:24 GMT
Christian,

The TaskTrackers send "heartbeat" messages to the JobTracker. The
default interval for these messages is 3 seconds.
This is one reason why you see the 3 second steps.

Abhishek

On Mon, Apr 11, 2011 at 3:19 AM, Christian Kumpe <christian@kumpe.de> wrote:
> Hi,
>
> I'm doing some measurement on hadoop's execution time for my theses. I
> discovered some steps in the jobs execution time when raising the
> mappers' execution time continuously.
> Here is a plot of the execution times with 1, 30 and 60 parallel
> executing mappers:
> http://baumbart.drhuim.de/executiontime.gif
>
> The cluster consists of 10 nodes and an overall map task capacity of 60.
> I'm using hadoop 0.20.2.
>
> The many small steps about 1 second are probably caused by the polling
> interval of the JobClient in monitorAndPrintJob(..) which is about 1
> second.
>
> But can anyone give me a hint, what's causing the bigger 3 second steps?
> I suppose it's a polling interval, too. But where? In the Jobtracker? In
> the Tasktrackers?
>
> Thanks for some help and comments,
>  Christian
>
>

Mime
View raw message