Mailing-List: contact common-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-user@hadoop.apache.org
Received-SPF: neutral (nike.apache.org: local policy)
Subject: Steps in execution time
From: Christian Kumpe <christian+Mailinglists@kumpe.de>
To: common-user@hadoop.apache.org
Content-Type: text/plain; charset="UTF-8"
Date: Mon, 11 Apr 2011 09:29:49 +0200
Message-ID: <1302506989.3434.28.camel@notebook>
Mime-Version: 1.0
Content-Transfer-Encoding: 7bit

Hi,

I'm doing some measurement on hadoop's execution time for my theses. I
discovered some steps in the jobs execution time when raising the
mappers' execution time continuously.
Here is a plot of the execution times with 1, 30 and 60 parallel
executing mappers:
http://baumbart.drhuim.de/executiontime.gif

The cluster consists of 10 nodes and an overall map task capacity of 60.
I'm using hadoop 0.20.2.

The many small steps about 1 second are probably caused by the polling
interval of the JobClient in monitorAndPrintJob(..) which is about 1
second.

But can anyone give me a hint, what's causing the bigger 3 second steps?
I suppose it's a polling interval, too. But where? In the Jobtracker? In
the Tasktrackers?

Thanks for some help and comments,
  Christian