hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amar Kamat" <ama...@yahoo-inc.com>
Subject RE: Job Tracker/Name Node redundancy
Date Sat, 10 Jan 2009 04:35:20 GMT
Ryan,
>From the MR (JobTracker) side we have a failover support. 
If a large job is submitted and the JobTracker fails midway then you can start the JobTracker
on the same host and resume
the job. Look at https://issues.apache.org/jira/browse/HADOOP-3245 for more details. Hope
that helps.

Amar


-----Original Message-----
From: Ryan LeCompte [mailto:lecompte@gmail.com]
Sent: Fri 1/9/2009 12:09 PM
To: core-user@hadoop.apache.org
Subject: Job Tracker/Name Node redundancy
 
Are there any plans to build redundancy/failover support for the Job
Tracker and Name Node components in Hadoop? Let's take the current
scenario:

1) A data/cpu intensive job is submitted to a Hadoop cluster of 10 machines.
2) Half-way through the job execution, the Job Tracker or Name Node fails.
3) We bring up a new Job Tracker or Name Node manually.

-- Will the individual task trackers / data nodes "reconnect" to the
new masters? Or will the job have to be resubmitted? If we had
failover support, we could setup essentially 3 Job Tracker masters and
3 NameNode masters so that if one dies the other would gracefully take
over and start handling results from the children nodes.

Thanks!

Ryan


Mime
  • Unnamed multipart/mixed (inline, None, 0 bytes)
View raw message