hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Weiwei Yang (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-1042) add ability to specify affinity/anti-affinity in container requests
Date Mon, 08 Jun 2015 10:03:03 GMT

     [ https://issues.apache.org/jira/browse/YARN-1042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Weiwei Yang updated YARN-1042:
    Attachment: YARN-1042-001.patch

I have worked out a patch, it is not all complete, but the part I've done works the way I
expected. I tested on my 5 nodes cluster against AFFINITY and ANTI_AFFINITY in NODE scope.
Please kindly help to review the patch, appreciate for comments and suggestions.

Work left to be done are
1. Complete container allocate handlers for RACK scope
2. Write test cases to test rules in RACK scope
3. Add support to argument maxTimeAwaitBeforeCompromise 
4. Complete code changes on other schedulers

> add ability to specify affinity/anti-affinity in container requests
> -------------------------------------------------------------------
>                 Key: YARN-1042
>                 URL: https://issues.apache.org/jira/browse/YARN-1042
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>    Affects Versions: 3.0.0
>            Reporter: Steve Loughran
>            Assignee: Arun C Murthy
>         Attachments: YARN-1042-001.patch, YARN-1042-demo.patch
> container requests to the AM should be able to request anti-affinity to ensure that things
like Region Servers don't come up on the same failure zones. 
> Similarly, you may be able to want to specify affinity to same host or rack without specifying
which specific host/rack. Example: bringing up a small giraph cluster in a large YARN cluster
would benefit from having the processes in the same rack purely for bandwidth reasons.

This message was sent by Atlassian JIRA

View raw message