hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vinod Kumar Vavilapalli <vino...@hortonworks.com>
Subject Re: jobTracker Blacklisted Nodes
Date Tue, 29 Jan 2013 02:18:36 GMT

Overall tasks can fail because of the job itself (bugs or erroneous condition in job's/user's
code) or issues on the node (hardware/system software).

Job level blacklisting accounts for the first type of problems i.e it avoids nodes on which
many tasks of this particular job failed for whatever reason. It is used to make sure that
more tasks of this job don't get launched on the same node.

Cluster level blacklisting accounts for cumulative failures on nodes across all jobs. Used
to heuristically determine generic node issues independent of the jobs.

+Vinod Kumar Vavilapalli
Hortonworks Inc.

On Jan 28, 2013, at 4:16 AM, Dhanasekaran Anbalagan wrote:

> Hi Guys,
> Difference between Job tracker cluster level blacklisted nodes and job level blacklist
> In my case My jobtacker page says no Blacklisted nodes,
> When i open job link it's says Blacklist node entry.
> please guide me how to understand.
> -Dhanasekaran.
> Did I learn something today? If not, I wasted it.

View raw message