hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Manikandan R (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6492) Generate queue metrics for each partition
Date Thu, 07 Dec 2017 18:11:01 GMT

    [ https://issues.apache.org/jira/browse/YARN-6492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16282261#comment-16282261
] 

Manikandan R commented on YARN-6492:
------------------------------------

[~sunilg] Thanks for your comments.

Attached v0 patch. Few notes are

1. Introduced a new PartitionMetrics class similar to QueueMetrics to handle metrics computation
based on above discussed structure. Please refer https://issues.apache.org/jira/browse/YARN-6492?focusedCommentId=16113219&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16113219
for structure.
2. Currently, focus is to compute below metrics: 
allocatedMB,allocatedVCores,allocatedContainers,aggregateContainersAllocated,aggregateContainersReleased,availableMB,availableMB,
pendingMB, pendingVCores, pendingContainers,reservedMB, reservedMB, reservedContainers.
This list has been inherited from YARN-6467.
3. In addtion to above metrics, incorporated metrics for resource vectors as well. These metrics
would be available as AvailableResource.memory-mb, AvailableResource.vcores, PendingResource.memory-mb,
PendingResource.vcores etc in addition to above set.
4. Similar to YARN-6467, focus is on Capacity scheduler metrics, Partition Metrics won't be
computed in FairScheduler for now.
5. Introduced a property in CS configuration to turn this feature ON or OFF similar to user
metrics.
6. User Metrics switch would be honoured with in Partition Metrics as well.
7. As discussed in earlier comments, existing queue metrics would be preserved for backward
compatibility.


> Generate queue metrics for each partition
> -----------------------------------------
>
>                 Key: YARN-6492
>                 URL: https://issues.apache.org/jira/browse/YARN-6492
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: capacity scheduler
>            Reporter: Jonathan Hung
>            Assignee: Manikandan R
>         Attachments: YARN-6492.001.patch, partition_metrics.txt
>
>
> We are interested in having queue metrics for all partitions. Right now each queue has
one QueueMetrics object which captures metrics either in default partition or across all partitions.
(After YARN-6467 it will be in default partition)
> But having the partition metrics would be very useful.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message