hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod Kumar Vavilapalli (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-5654) blacklist is not propagated from AM to RM
Date Tue, 26 Nov 2013 00:08:37 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13832102#comment-13832102

Vinod Kumar Vavilapalli commented on MAPREDUCE-5654:

Let's investigate if it turns out to be a bug in YARN before moving this to MR.

[~rgrandl], you are forcing the blacklist by changing MR code and running your own modified
MR App?

Also you are saying that you put logs in ApplicationMasterProtocolPBClientImpl? Can you put
more logs in ResourceBlacklistRequestPBImpl and see where it is breaking?

> blacklist is not propagated from AM to RM
> -----------------------------------------
>                 Key: MAPREDUCE-5654
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5654
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: applicationmaster
>            Reporter: Robert Grandl
> I was trying to blacklist some nodes. I added a set of hosts as strings into blacklistAdditions
list and propagated into RMContainerRequestor#makeRemoteRequest to the RM. 
> However the blacklist is received empty at RM. I logged the path for blacklist in AM
and I found that in ApplicationMasterProtocolPBClientImpl#allocate, this list is lost. 
> I print request.getResourceBlacklistRequest().getBlacklistAdditions().toString() at the
beginning of ApplicationMasterProtocolPBClientImpl#allocate and the blacklisted additions
are there. 
> After AllocateRequestProto requestProto is created based on this request, and I print
again requestProto.getBlacklistRequest().getBlacklistAdditionsList().toString(), the blacklist
additions is empty now.
> I looked even further and log what happened. At some point in yarn-api, I was lost with
my logging as that code was regenerated every time I recompiled yarn-api. 
> Thanks,
> robert

This message was sent by Atlassian JIRA

View raw message