hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Templeton (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7292) Revisit Resource Profile Behavior
Date Tue, 13 Feb 2018 04:27:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16361805#comment-16361805

Daniel Templeton commented on YARN-7292:

Hang on.  If I understand that correctly, you're saying that in order to not have the profile
overridden, the app developer has to create their own AMRMClient implementation?  Doesn't
that mean that profiles will always be overridden in all practical cases, e.g. MR, Spark,
Hive, etc.?

> Revisit Resource Profile Behavior
> ---------------------------------
>                 Key: YARN-7292
>                 URL: https://issues.apache.org/jira/browse/YARN-7292
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager, resourcemanager
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>            Priority: Blocker
>         Attachments: YARN-7292.002.patch, YARN-7292.003.patch, YARN-7292.004.patch, YARN-7292.005.patch,
YARN-7292.006.patch, YARN-7292.wip.001.patch
> Had discussions with [~templedf], [~vvasudev], [~sunilg] offline. There're a couple of
resource profile related behaviors might need to be updated:
> 1) Configure resource profile in server side or client side: 
> Currently resource profile can be only configured centrally:
> - Advantages:
> A given resource profile has a the same meaning in the cluster. It won’t change when
we run different apps in different configurations. A job can run under Amazon’s G2.8X can
also run on YARN with G2.8X profile. A side benefit is YARN scheduler can potentially do better
bin packing.
> - Disadvantages: 
> Hard for applications to add their own resource profiles. 
> 2) Do we really need mandatory resource profiles such as minimum/maximum/default? 
> 3) Should we send resource profile name inside ResourceRequest, or should client/AM translate
it to resource and set it to the existing resource fields? 
> 4) Related to above, should we allow resource overrides or client/AM should send final
resource to RM?

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message