Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Thu, 13 Apr 2017 04:23:41 +0000 (UTC)
From: "Hadoop QA (JIRA)" <jira@apache.org>
To: mapreduce-issues@hadoop.apache.org
Message-ID: <JIRA.13060097.1490810577000.272934.1492057421737@Atlassian.JIRA>
In-Reply-To: <JIRA.13060097.1490810577000@Atlassian.JIRA>
References: <JIRA.13060097.1490810577000@Atlassian.JIRA> <JIRA.13060097.1490810577940@jira-lw-us.apache.org>
Subject: [jira] [Commented] (MAPREDUCE-6871) Allow users to specify racks
 and nodes for strict locality for AMs
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Thu, 13 Apr 2017 04:23:47 -0000


    [ https://issues.apache.org/jira/browse/MAPREDUCE-6871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15967103#comment-15967103 ] 

Hadoop QA commented on MAPREDUCE-6871:
--------------------------------------

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 18s {color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s {color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s {color} | {color:green} The patch appears to include 1 new or modified test files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 45s {color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m 54s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 9s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 35s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 8s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 40s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 46s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 45s {color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 10s {color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 7s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 23s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m 23s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 36s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 38s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 8s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 41s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m 1s {color} | {color:green} hadoop-mapreduce-client-core in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 104m 1s {color} | {color:green} hadoop-mapreduce-client-jobclient in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 29s {color} | {color:green} The patch does not generate ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 141m 24s {color} | {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |  Image:yetus/hadoop:612578f |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12863188/MAPREDUCE-6871.003.patch |
| JIRA Issue | MAPREDUCE-6871 |
| Optional Tests |  asflicense  compile  javac  javadoc  mvninstall  mvnsite  unit  findbugs  checkstyle  |
| uname | Linux 5271f9ca5c19 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 0cab572 |
| Default Java | 1.8.0_121 |
| findbugs | v3.0.0 |
|  Test Results | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6954/testReport/ |
| modules | C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient U: hadoop-mapreduce-project/hadoop-mapreduce-client |
| Console output | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6954/console |
| Powered by | Apache Yetus 0.3.0   http://yetus.apache.org |


This message was automatically generated.


> Allow users to specify racks and nodes for strict locality for AMs
> ------------------------------------------------------------------
>
>                 Key: MAPREDUCE-6871
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6871
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>          Components: client
>            Reporter: Robert Kanter
>            Assignee: Robert Kanter
>         Attachments: MAPREDUCE-6871.001.patch, MAPREDUCE-6871.002.patch, MAPREDUCE-6871.003.patch
>
>
> YARN-6050 fixed the YARN API to allow multiple {{ResourceRequest}}'s when submitting an AM so that you can actually do rack or node locality.  We should allow MapReduce users to take advantage of this by exposing this functionality in some way.  The raw YARN API allows for a lot of flexibility (e.g. different resources per request, etc), but we don't necessarily want to allow the user to do too much here so they don't shoot themselves in the foot and we don't make this overly complicated.  
> I propose we allow users to specify racks and nodes for strict locality.  This would allow users to restrict an MR AM to specific racks and/or nodes.  We could add a new property, {{mapreduce.job.am.resource-request.strict.locality}}, which takes a comma-separated list of entries like:
> - {{/<rack>}}
> - {{/<rack>/<node>}}
> - {{<node>}} (assumes /default-rack)
> MapReduce would then use this information to create the corresponding {{ResourceRequest}}'s.  
> For example, {{mapreduce.job.am.resource-request.strict.locality=/rack1/node1}} would create the following {{ResourceRequest}}'s:
> - resourceName=ANY, relaxLocality=false, capability=<X,Y>
> - resourceName=/rack1, relaxLocality=false, capability=<X,Y>
> - resourceName=node1, relaxLocality=true, capability=<X,Y>
> By default, the property would be unset, and you'd get the normal {{ANY}} {{ResourceRequest}}.


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org