hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hansi Klose" <hansi.kl...@web.de>
Subject Aw: Re: mapreduce.jobtracker.expire.trackers.interval no effect
Date Wed, 04 Dec 2013 08:23:49 GMT
Hi adam,
in our enviroment it does not matter what i insert there it always take over 600 seconds.
I tried 30000 and the resulte was the same.
Regards Hansi

Gesendet: Dienstag, 03. Dezember 2013 um 19:23 Uhr
Von: "Adam Kawa" <kawa.adam@gmail.com>
An: user@hadoop.apache.org
Betreff: Re: mapreduce.jobtracker.expire.trackers.interval no effect

I did a small test, and I a setting mapred.tasktracker.expiry.interval=60000 worked for me
(TT became considered as lost after around 66 seconds).
Can the formula be: mapred.tasktracker.expiry.interval + 2 * some-heartbeat-interval-that-is-3-sec-by-default? 
Otherwise, is the 6 sec some kind of time needed to make a decision to consider TT as lost?
2013/12/3 Hansi Klose <hansi.klose@web.de>I forget to say that we use Cloudera 2.0.0-mr1-cdh4.2.0

> Gesendet: Dienstag, 03. Dezember 2013 um 17:38 Uhr
> Von: "Hansi Klose" <hansi.klose@web.de[hansi.klose@web.de]>
> An: user@hadoop.apache.org[user@hadoop.apache.org]
> Betreff: mapreduce.jobtracker.expire.trackers.interval no effect

> Hi,
> we want to set the heartbeat timout for a tasktracker.
> If the tasktracker does not send heartbeats for 60 seconds he should
> be marked as lost.
> I found the parameter mapreduce.jobtracker.expire.trackers.interval
> which sounds right to me.
> I set
> <property>
> <name>mapreduce.jobtracker.expire.trackers.interval</name>
> <value>60000</value>
> </property>
> in the mapred-site.xml on all servers and restarted the jobtracker and all tasktrackers.
> I started a benchmark "hadoop jar hadoop-examples.jar randomwriter rand" and every tasktracker
gets 2 jobs.
> It is a small test environment.
> On one tasktracker i stopped the network. On the jobtracker i could see the "Seconds
since heartbeat"
> increasing. But after 60 seconds the tasktracker was still in the overview.
> Even in the log of the jobtracker I found nothing.
> After over 600 seconds i found the message
> org.apache.hadoop.mapred.JobTracker: Lost tracker .....
> And the tasktracker wasn't shown any more on the jobtracker.
> Isn't this the right setting?
> Regards Hansi

View raw message