Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: yarn-issues@hadoop.apache.org
Date: Mon, 8 Jun 2015 10:03:03 +0000 (UTC)
From: "Weiwei Yang (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.12662451.1375899092000.16530.1433757783818@Atlassian.JIRA>
In-Reply-To: <JIRA.12662451.1375899092000@Atlassian.JIRA>
References: <JIRA.12662451.1375899092000@Atlassian.JIRA>
 <JIRA.12662451.1375899092724@arcas>
Subject: [jira] [Updated] (YARN-1042) add ability to specify
 affinity/anti-affinity in container requests
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/YARN-1042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Weiwei Yang updated YARN-1042:
------------------------------
    Attachment: YARN-1042-001.patch

I have worked out a patch, it is not all complete, but the part I've done works the way I expected. I tested on my 5 nodes cluster against AFFINITY and ANTI_AFFINITY in NODE scope. Please kindly help to review the patch, appreciate for comments and suggestions.

Work left to be done are
1. Complete container allocate handlers for RACK scope
2. Write test cases to test rules in RACK scope
3. Add support to argument maxTimeAwaitBeforeCompromise 
4. Complete code changes on other schedulers

> add ability to specify affinity/anti-affinity in container requests
> -------------------------------------------------------------------
>
>                 Key: YARN-1042
>                 URL: https://issues.apache.org/jira/browse/YARN-1042
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>    Affects Versions: 3.0.0
>            Reporter: Steve Loughran
>            Assignee: Arun C Murthy
>         Attachments: YARN-1042-001.patch, YARN-1042-demo.patch
>
>
> container requests to the AM should be able to request anti-affinity to ensure that things like Region Servers don't come up on the same failure zones. 
> Similarly, you may be able to want to specify affinity to same host or rack without specifying which specific host/rack. Example: bringing up a small giraph cluster in a large YARN cluster would benefit from having the processes in the same rack purely for bandwidth reasons.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)