hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Szilard Nemeth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-9322) Store metrics for custom resource types into FSQueueMetrics and query them in FairSchedulerQueueInfo
Date Wed, 27 Feb 2019 16:28:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-9322?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16779485#comment-16779485

Szilard Nemeth commented on YARN-9322:

Hi [~templedf]!
Thanks for the review! Attached patch005

1. Yep this is true, this had a conflict there, I cleaned that up. Maybe this won't apply
to trunk so I'm adding YARN-9318 as a prerequisite. Pls tell me if YARN-9318 is in and I can
rebase this to trunk again. We have to wait another Jenkins run, though.
2. Fixed the ordering
3. Good point, added assert messages
4. Added simple Resource check (w/o additonal resource type) for every testcase. This led
to some code duplication in the tests but I hope we can live with that for now.


> Store metrics for custom resource types into FSQueueMetrics and query them in FairSchedulerQueueInfo
> ----------------------------------------------------------------------------------------------------
>                 Key: YARN-9322
>                 URL: https://issues.apache.org/jira/browse/YARN-9322
>             Project: Hadoop YARN
>          Issue Type: Improvement
>            Reporter: Szilard Nemeth
>            Assignee: Szilard Nemeth
>            Priority: Major
>         Attachments: Screen Shot 2019-02-21 at 12.06.46.png, YARN-9322.001.patch, YARN-9322.002.patch,
YARN-9322.003.patch, YARN-9322.004.patch, YARN-9322.005.patch
> YARN-8842 implemented storing and exposing of metrics of custom resources.
> FSQueueMetrics should have a similar implementation.
> All metrics stored in this class should have their custom resource counterpart.
> In a consequence of metrics were not stored for custom resource type, FairSchedulerQueueInfo
haven't contained those values therefore the UI v1 could not show them, obviously. 
> See that gpu is missing from the value of  "AM Max Resources" on the attached screenshot.
> Additionally, the callees of the following methods (in class FairSchedulerQueueInfo)
should consider to query values for custom resource types too: 
> getMaxAMShareMB
> getMaxAMShareVCores
> getAMResourceUsageMB
> getAMResourceUsageVCores

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message