hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun Suresh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5982) Simplifying opportunistic container parameters and metrics
Date Thu, 08 Dec 2016 18:41:58 GMT

    [ https://issues.apache.org/jira/browse/YARN-5982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15733021#comment-15733021

Arun Suresh commented on YARN-5982:

Thanks for the patch [~kkaranasos]

Couple of comments:
# You can probably completely remove the OpportunisticContainerAllocatorAMService::createMin/MaxResource,
and replace it with {{rmContext.getScheduler().getMinimumResourceCapability()}} and {{rmContext.getScheduler().getMaximumResourceCapability()}}.
This will ensure the values are in sync with the actual scheduler being used.
# Lets not remove the incr resources from the {{yarn_server_common_service_protos.proto}},
since there exists a *yarn.scheduler.increment-allocation-mb/vcore*. It is used by the FairScheduler,
but not the capacity scheduler yet, and the normalize function actually has an argument for
it. I feel, it would be better if we just add *getIncrementResourceCapability()* in YarnScheduler
interface, and in AbstractYarnScheduler, put in a default implementation that delegates to
*getMinimumResourceCapability()*. FairScheduler already has this, so you just have to put
an *@Override* tag on the method.
# In {{NodeManagerMetrics}}, if you are changing the *opportunisticContainerStarted* method
to *startOpportunisticContainer*, you should also correspondingly change *opportunisticContainerCompleted*
to *completeOpportunisticContainer*. Also, lets no not move the placement of the methods,
since it can cause merge issues during cherry-pick and the diffs become larger and difficult
to make sense of sometimes.

> Simplifying opportunistic container parameters and metrics
> ----------------------------------------------------------
>                 Key: YARN-5982
>                 URL: https://issues.apache.org/jira/browse/YARN-5982
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Konstantinos Karanasos
>            Assignee: Konstantinos Karanasos
>             Fix For: 2.9.0, 3.0.0-alpha2
>         Attachments: YARN-5982.001.patch
> This JIRA removes some of the parameters that are related to opportunistic containers
(e.g., min/max memory/cpu). Instead, we will be using the parameters already used by guaranteed
> The goal is to reduce the number of parameters that need to be used by the user.
> We also fix a small issue related to the container metrics (opportunistic memory reported
in GB in Web UI, although it was in MB).

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message