hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Grandl Robert <rgra...@yahoo.com>
Subject Fw: blacklist additions/removals not propagated to RM
Date Mon, 25 Nov 2013 18:38:18 GMT
I am not sure I posted this question in the right thread. 


Do you have some thoughts with the below mail ? Is there any configuration parameter for RM
to enable blacklisting ? Anyway, it seems my blacklist is sent empty to RM, so I think is
an AM problem though. 


Please let me know about your thoughts, 

robert




On Saturday, November 23, 2013 2:31 PM, Grandl Robert <rgrandl@yahoo.com> wrote:
 
Hi,

I am not 100% sure it's a bug in AM, but it seems so. I am blacklisting some hosts, which
are send through an AllocateRequest in makeRemoteRequest method in RMContainerRequest.java.

However, the blacklist is empty at RM. I traced all over the places where this is lost, and
I found that in ApplicationMasterProtocolPBClientImpl.java, in allocate(AllocateRequest request)
method, if I print the request.getResourceBlacklistRequest().getBlacklistAdditiions() I see
the list, but after it does AllocateRequestProto requestProto = (..).getProto(), 
and I print the requestProto.getBlacklistRequest().getBlacklistAdditions() now is EMPTY. 

So basically, AM send an empty black list to RM. I went even further down and traced till
YarnServiceProtos.java(in hadoop-yarn-api), but at this point all the logs I have, are removed.
it seems this code is regenerated every time I am compiling yarn-api ....

Is there something I am missing guys, or is a bug ? Interesting is that release containers
list is propagated correctly but I think it comes on other path. I am running hadoop-yarn
2.3. 

Thanks,
robert
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message