hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Naganarasimha G R (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-6304) Specifying node labels when submitting MR jobs
Date Wed, 08 Apr 2015 19:56:13 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14485895#comment-14485895

Naganarasimha G R commented on MAPREDUCE-6304:

Thanks for the comments [~yufeldman] & [~john.jian.fang],
bq. Definitely it depends whether you want separate label for AM and the rest of the tasks.
I have started working on this patch and had already considered the 2 kinds of labels as ApplicationSubmissionContext
was supporting it and REST has already taken care of it .

bq. The reason is that many Hadoop users do not understand or set the labels by themselves
and they simply rely on the Hadoop platform provider (or system admins for on-premise clusters)
to set up the labels for them. I am not sure if this is a general use case, but it is definitely
a feature that we need.
Even though labels should not be dictated by RM, i feel there might be cases as mentioned
by [~john.jian.fang] where users might not be aware what each label stands for! or users might
not want to understand the label as partition (which is more on administrative side in multi
tenant scenarios) and they might just require some auto mapping based on user or group. So
i think we can come up with some configurations of automapping based on user/group (similar
to queues)   for labels too.

> Specifying node labels when submitting MR jobs
> ----------------------------------------------
>                 Key: MAPREDUCE-6304
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6304
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>            Reporter: Jian Fang
>            Assignee: Naganarasimha G R
> Per the discussion on YARN-796, we need a mechanism in MAPREDUCE to specify node labels
when submitting MR jobs.

This message was sent by Atlassian JIRA

View raw message