hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Peter Bacsko (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6504) Add support for resource profiles in MapReduce
Date Mon, 25 Sep 2017 11:07:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-6504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16178851#comment-16178851

Peter Bacsko commented on YARN-6504:

Just my 2 cents: there could be Hadoop users out there who spent significant amount of time
optimizing their MR/Hive/Pig jobs to have the best memory/vcore utilization but they're also
interested in resource profiles. And these users might not want to change their existing configuration
where mem/vcores are defined, because they're fine with it. But they want this feature to
be enabled for new jobs.

We could introduce a flag which tells whether we should go for profiles or apply the mem/vcore
settings that are already in the code - like {{mapreduce.job.profiles.enabled}}. 

So selective override might not be necessary, but being able to fall back to the original
behavior can be useful.

> Add support for resource profiles in MapReduce
> ----------------------------------------------
>                 Key: YARN-6504
>                 URL: https://issues.apache.org/jira/browse/YARN-6504
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager, resourcemanager
>            Reporter: Varun Vasudev
>            Assignee: Varun Vasudev
>         Attachments: YARN-6504-YARN-3926.001.patch, YARN-6504.YARN-3926.002.patch, YARN-6504.YARN-3926.003.patch

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message