hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Naganarasimha G R (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-6304) Specifying node labels when submitting MR jobs
Date Mon, 27 Apr 2015 23:21:40 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14515840#comment-14515840
] 

Naganarasimha G R commented on MAPREDUCE-6304:
----------------------------------------------

Thanks for the review comments [~wangda],
bq. Beyond AM/Job node-labels setting, should we support Mapper/Reducer setting?
bq. If we need support mapper/reducer's node-label-expression. We may need add some changes
in MR AM side and some tests.
Well my view should be to support them as i feel in some cases we might feel like mapper can
run in any node but may be reducer requires more mem so may be clients might require it to
be run in high mem nodes (may be some constraint on mem for the nodes)  or some other node
constraints.

bq. When job and AM (or mapper/reducer) set together, AM (or mapper/reducer)'s node-label-expression
should overwrite job's setting.
Well, IIUC then {{ApplicationSubmissionContext.getNodeLabelExpression()}} is for  mapper/reducer
and AM's node label expression is set in {{ApplicationSubmissionContext.getAMContainerResourceRequest}}.
I could cross verify the same with code in ApplicationMasterService (line number 495) &
RMAppManager (line number 378) and so did not get what you meant by the above statement, correct
me if my understanding is wrong.

> Specifying node labels when submitting MR jobs
> ----------------------------------------------
>
>                 Key: MAPREDUCE-6304
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6304
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>            Reporter: Jian Fang
>            Assignee: Naganarasimha G R
>             Fix For: 2.8.0
>
>         Attachments: MAPREDUCE-6304.20150410-1.patch, MAPREDUCE-6304.20150411-1.patch
>
>
> Per the discussion on YARN-796, we need a mechanism in MAPREDUCE to specify node labels
when submitting MR jobs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message