hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hansi Klose" <hansi.kl...@web.de>
Subject Aw: mapreduce.jobtracker.expire.trackers.interval no effect
Date Wed, 04 Dec 2013 09:16:15 GMT
Hi.

i think i found the reason. I looked at the job.xml and found the parameter

mapred.tasktracker.expiry.interval 6000000

and 

mapreduce.jobtracker.expire.trackers.interval 30000

So i tried the deprecated parameter mapred.tasktracker.expiry.interval in my configuration
and voila it works!

Why they write that the parameter is deprecated when the new one is not working and will be
overwritten by the old one
with the default value?

http://archive.cloudera.com/cdh4/cdh/4/hadoop-2.0.0-cdh4.2.0/hadoop-project-dist/hadoop-common/DeprecatedProperties.html

Regards Hansi

> Gesendet: Dienstag, 03. Dezember 2013 um 17:38 Uhr
> Von: "Hansi Klose" <hansi.klose@web.de>
> An: user@hadoop.apache.org
> Betreff: mapreduce.jobtracker.expire.trackers.interval no effect
>
> Hi,
> 
> we want to set the heartbeat timout for a tasktracker.
> 
> If the tasktracker does not send heartbeats for 60 seconds he should
> be marked as lost.
> 
> I found the parameter mapreduce.jobtracker.expire.trackers.interval
> which sounds right to me.
> 
> I set 
> 
> <property>
> <name>mapreduce.jobtracker.expire.trackers.interval</name>
> <value>60000</value>
> </property>
> 
> in the mapred-site.xml on all servers and restarted the jobtracker and all tasktrackers.
> 
> I started a benchmark "hadoop jar hadoop-examples.jar randomwriter rand" and every tasktracker
gets 2 jobs.
> It is a small test environment.
> 
> On one tasktracker i stopped the network. On the jobtracker i could see the "Seconds
since heartbeat"
> increasing. But after 60 seconds the tasktracker was still in the overview. 
> Even in the log of the jobtracker I found nothing.
> 
> After over 600 seconds i found the message
> org.apache.hadoop.mapred.JobTracker: Lost tracker .....
> And the tasktracker wasn't shown any more on the jobtracker.
> 
> Isn't this the right setting?
> 
> Regards Hansi   
> 

Mime
View raw message