hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-6304) Specifying node labels when submitting MR jobs
Date Thu, 09 Apr 2015 18:44:12 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14487941#comment-14487941

Wangda Tan commented on MAPREDUCE-6304:

Just found I'm not in the watcher list, missed some discussions.

I think [~john.jian.fang]'s use case is very unique but also interesting, platform provider
has no control and knowledge of user's applications, and user doesn't understand about platform's
details such as which machine is temporary or not.

Maybe a possible way to solve this problem is to add a global-am-resource-request setting,
which can include node-label-expression, node/rack information. Because as YARN RM, it doesn't
know what's the application's running model, it only knows which container is AM or not, which
seems enough for your requirements.

But adding such global-am-resource-request setting can also be dangerous, such as how about
a queue is not accessible node-label-expression of global-am-resource-request?

> Specifying node labels when submitting MR jobs
> ----------------------------------------------
>                 Key: MAPREDUCE-6304
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6304
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>            Reporter: Jian Fang
>            Assignee: Naganarasimha G R
> Per the discussion on YARN-796, we need a mechanism in MAPREDUCE to specify node labels
when submitting MR jobs.

This message was sent by Atlassian JIRA

View raw message