hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arun C Murthy <...@hortonworks.com>
Subject Re: whitelist feature of YARN
Date Wed, 07 Aug 2013 16:53:34 GMT
Go ahead - it's very close, I'll probably create an rc2 in the next couple of days.

On Aug 7, 2013, at 2:06 AM, Krishna Kishore Bonagiri <write2kishore@gmail.com> wrote:

> Sandy, 
>   Thanks again. I found RC1 for 2.1.0-beta available at http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc1/
>    Would this have the fix for YARN-521? and, can I use that?
> 
> -Kishore
> 
> 
> On Wed, Aug 7, 2013 at 12:35 PM, Sandy Ryza <sandy.ryza@cloudera.com> wrote:
> Responses inline:
> 
> 
> On Tue, Aug 6, 2013 at 11:55 PM, Krishna Kishore Bonagiri <write2kishore@gmail.com>
wrote:
> Hi Sandy,
> 
>   Thanks for the reply and it is good to know YARN-521 is done! Please answer my following
questions 
> 
> 1) when is 2.1.0-beta going to be released? is it soon or do you suggest me take it from
the trunk or is there a recent release candidate available?
> 
> We're very close and my guess would be no later than the end of the month (don't hold
me to this).
>  
> 2) I have recently changed my application to use the new Asynchronous interfaces. I am
hoping it works with that too, correct me if I am wrong.
> ContainerRequest is shared by the async interfaces as well so it should work here. 
> 
> 3) Change in interface:
> 
> The old interface for ContainerRequest constructor used to be this:
> 
>  public ContainerRequest(Resource capability, String[] nodes,
>         String[] racks, Priority priority, int containerCount);
> 
> where as now it is changed to 
> 
> a) public ContainerRequest(Resource capability, String[] nodes,
>         String[] racks, Priority priority)
> &
> 
> b) public ContainerRequest(Resource capability, String[] nodes,
>         String[] racks, Priority priority, boolean relaxLocality)
> 
> that means the old argument containerCount is gone! How would I be able to specify how
many containers do I need?
> 
> We now expect that you submit a ContainerRequest for each container you want.
>  
> -Kishore
> 
> 
> 
> 
> On Wed, Aug 7, 2013 at 11:37 AM, Sandy Ryza <sandy.ryza@cloudera.com> wrote:
> YARN-521, which brings whitelisting to the AMRMClient APIs, is now included in 2.1.0-beta.
 Check out the doc for the relaxLocality paramater in ContainerRequest in AMRMClient: https://github.com/apache/hadoop-common/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/main/java/org/apache/hadoop/yarn/client/api/AMRMClient.java
and I can help clarify here if anything's confusing.
> 
> -Sandy
> 
> 
> On Tue, Jul 9, 2013 at 2:54 AM, Krishna Kishore Bonagiri <write2kishore@gmail.com>
wrote:
> Hi Sandy,
> 
>   Yes, I have been using AMRMClient APIs. I am planning to shift to whatever way is this
white list feature is supported with. But am not sure what is meant by submitting ResourceRequests
directly to RM. Can you please elaborate on this or give me a pointer to some example code
on how to do it...
> 
>    Thanks for the reply,
> 
> -Kishore
> 
> 
> On Mon, Jul 8, 2013 at 10:53 PM, Sandy Ryza <sandy.ryza@cloudera.com> wrote:
> Hi Krishna,
> 
> From your previous email, it looks like you are using the AMRMClient APIs.  Support for
whitelisting is not yet supported through them.  I am working on this in YARN-521, which should
be included in the next release after 2.1.0-beta.  If you are submitting ResourceRequests
directly to the RM, you can whitelist a node by
> * setting the relaxLocality flag on the node-level ResourceRequest to true
> * setting the relaxLocality flag on the corresponding rack-level ResourceRequest to false
> * setting the relaxLocality flag on the corresponding any-level ResourceRequest to false
> 
> -Sandy
> 
> 
> On Mon, Jul 8, 2013 at 6:48 AM, Krishna Kishore Bonagiri <write2kishore@gmail.com>
wrote:
> Hi,
> 
>   Can someone please point to some example code of how to use the whitelist feature of
YARN, I have recently got RC1 for hadoop-2.1.0-beta and want to use this feature. 
> 
>   It would be great if you can point me to some description of what this white listing
feature is, I have gone through some JIRA logs related to this but more concrete explanation
would be helpful.
> 
> Thanks,
> Kishore
> 
> 
> 
> 
> 
> 

--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/



Mime
View raw message