hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-6070) yarn.app.am.resource.mb/cpu-vcores affects uber mode but is not documented
Date Fri, 05 Sep 2014 14:00:36 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14122970#comment-14122970
] 

Jason Lowe commented on MAPREDUCE-6070:
---------------------------------------

Do we want to put these details in the description for mapreduce.job.ubertask.enable instead?
 If users are interested in getting uber jobs to work, they may not even look at these other
properties as often AM properties aren't tweaked for typical jobs.  It also adds noise to
the description of those properties when people aren't interested in uber mode (or may not
even know what uber mode is).  However I don't feel that strongly about it and could be convinced
to leave it in the individual properties if people feel that's better. 

"mapreduce.reduce.cpu.vcore" should be "mapreduce.reduce.cpu.vcores".

> yarn.app.am.resource.mb/cpu-vcores affects uber mode but is not documented
> --------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-6070
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6070
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: documentation
>    Affects Versions: 2.4.0, 2.5.0
>            Reporter: Tsuyoshi OZAWA
>            Assignee: Tsuyoshi OZAWA
>            Priority: Trivial
>         Attachments: YARN-2078.1.patch
>
>
> We should document the condition when uber mode is enabled. Currently, users need to
read following code to understand the condition.
> {code}
>     boolean smallMemory =
>         ( (Math.max(conf.getLong(MRJobConfig.MAP_MEMORY_MB, 0),
>             conf.getLong(MRJobConfig.REDUCE_MEMORY_MB, 0))
>             <= sysMemSizeForUberSlot)
>             || (sysMemSizeForUberSlot == JobConf.DISABLED_MEMORY_LIMIT));
>    boolean smallCpu =
>             Math.max(
>                 conf.getInt(
>                     MRJobConfig.MAP_CPU_VCORES, 
>                     MRJobConfig.DEFAULT_MAP_CPU_VCORES), 
>                 conf.getInt(
>                     MRJobConfig.REDUCE_CPU_VCORES, 
>                     MRJobConfig.DEFAULT_REDUCE_CPU_VCORES)) 
>              <= sysCPUSizeForUberSlot
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message