hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-6304) Specifying node labels when submitting MR jobs
Date Fri, 01 May 2015 18:03:06 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14523543#comment-14523543

Wangda Tan commented on MAPREDUCE-6304:

Patch generally LGTM, one minor comment:
- If specify label="" in mapred-default.xml, which means the job will always set label=""
while adding resource-request, but the previously behavior is send "null" instead of empty
string. So I suggest to change the default to be "NOT SPECIFIED" (which is not a valid label
since it contains space) as default value, and we will replace it by null. Does this make
sense to you? Do you have any other ideas on this? 

And could you deploy a few nodes cluster, with 2-3 labels, and run MR job with am.label, mapper.label,
reducer.label, etc. that will be very important to make sure everything works well.

> Specifying node labels when submitting MR jobs
> ----------------------------------------------
>                 Key: MAPREDUCE-6304
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6304
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>            Reporter: Jian Fang
>            Assignee: Naganarasimha G R
>             Fix For: 2.8.0
>         Attachments: MAPREDUCE-6304.20150410-1.patch, MAPREDUCE-6304.20150411-1.patch,
> Per the discussion on YARN-796, we need a mechanism in MAPREDUCE to specify node labels
when submitting MR jobs.

This message was sent by Atlassian JIRA

View raw message