hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Templeton (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (YARN-6504) Add support for resource profiles in MapReduce
Date Fri, 08 Sep 2017 18:23:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-6504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16159066#comment-16159066
] 

Daniel Templeton edited comment on YARN-6504 at 9/8/17 6:22 PM:
----------------------------------------------------------------

Looks like there's an issue here.  The profile capability has the idea of an override capability
built in, but from the MR side, there will *always* be an override capability for map and
reduce tasks on CPU and memory.  That means that as currently constructed, the CPU and memory
of the profile will always be overridden.  That makes sense in the case that the memory and
CPU were explicitly specified, but even in the case the case where memory and CPU were not
specified, the MR defaults (1024MB and 1 vcore) will be treated as an override.  That's counterintuitive
and should be addressed.


was (Author: templedf):
Looks like there's an issue here.  The profile capability has the idea of an override capability
built in, but from the MR side, there will *always* be an override capability for map and
reduce tasks on CPU.  That means that as currently constructed, the CPU and memory of the
profile will always be overridden.  That makes sense in the case that the memory and CPU were
explicitly specified, but even in the case the case where memory and CPU were not specified,
the MR defaults (1024MB and 1 vcore) will be treated as an override.  That's counterintuitive
and should be addressed.

> Add support for resource profiles in MapReduce
> ----------------------------------------------
>
>                 Key: YARN-6504
>                 URL: https://issues.apache.org/jira/browse/YARN-6504
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager, resourcemanager
>            Reporter: Varun Vasudev
>            Assignee: Varun Vasudev
>         Attachments: YARN-6504-YARN-3926.001.patch, YARN-6504.YARN-3926.002.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message