hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rohith Sharma K S <rohithsharm...@huawei.com>
Subject RE: Does Hadoop 2.6.0 have job level blacklisting?
Date Mon, 30 Mar 2015 04:10:32 GMT
Hi Chris

Is there still job level blacklisting as there was in earlier versions?
>> yes, job level blacklisting support is there.  Application Master has to identify
the nodes which it wants to blacklists and send those nodes details to ResourceManager via
ApplicationMasterProtocol#allocate request. On blacklisted nodes, containers will not be assigned
thereafter.

   Java Doc
https://hadoop.apache.org/docs/r2.6.0/api/org/apache/hadoop/yarn/api/ApplicationMasterProtocol.html#allocate(org.apache.hadoop.yarn.api.protocolrecords.AllocateRequest)
https://hadoop.apache.org/docs/r2.6.0/api/org/apache/hadoop/yarn/api/protocolrecords/AllocateRequest.html

Thanks & Regards
Rohith Sharma K S
From: Chris Mawata [mailto:chris.mawata@gmail.com]
Sent: 29 March 2015 01:10
To: user@hadoop.apache.org
Subject: Does Hadoop 2.6.0 have job level blacklisting?

At http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/ClusterSetup.html#Monitoring_Health_of_NodeManagers
is a description of how you can have a script check the health of a node and indicate to the
ResourceManager that it is unhealthy. This seems to be at the cluster level. Is there still
job level blacklisting as there was in earlier versions?

Chris Mawata
Mime
View raw message