hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Manikandan R (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6492) Generate queue metrics for each partition
Date Sun, 21 Jan 2018 16:46:02 GMT

    [ https://issues.apache.org/jira/browse/YARN-6492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16333591#comment-16333591

Manikandan R commented on YARN-6492:

[~Naganarasimha] Thanks for your review and suggestions.

{quote}Though the way we want to get the metrics output is correct i am not happy to have
another class which is just a copy of QueueMetrics and nominal changes to capture the Partition
information. I was thinking PartitionQueueMetrics is just a holder for QueueMetrics and Queue
metrics having the changes to incorporate the Partition information too.{quote}

I've been trying to think through this to see how I can simplify overall changes after our
offline discussions. I did started with couple of metrics in this direction, looks like we
could accommodate this in QueueMetrics class and try to avoid PartitionMetrics class. 

{quote}I think the assumption that queue metrics is created only during the creation of constructor
is wrong as partition can be added dynamically. hence if the given queuemetrics objects are
not present for a given partition we need to create it.{quote}

Ok, As and when any new partition gets added, JMX metrics o/p can show metrics related to
those partitions. Till those partitions being used, metrics (values) would be 0.

{quote}Do we really require "enablePartitionMetrics" in the QueueMatrix ? IMO its sufficient
to have "string partition" class member and by having default value as "DEFAULT". May be we
can further discuss on it.{quote}

Ok. Should be fine as partition is "default" by default. In case there are no partitions,
still JMX o/p can have PartitionQueueMetrics for "default" partition.

Also, In the meantime, We would like to hear thoughts from Community to ensure we are on the
right direction.

> Generate queue metrics for each partition
> -----------------------------------------
>                 Key: YARN-6492
>                 URL: https://issues.apache.org/jira/browse/YARN-6492
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: capacity scheduler
>            Reporter: Jonathan Hung
>            Assignee: Manikandan R
>            Priority: Major
>         Attachments: YARN-6492.001.patch, YARN-6492.002.patch, partition_metrics.txt
> We are interested in having queue metrics for all partitions. Right now each queue has
one QueueMetrics object which captures metrics either in default partition or across all partitions.
(After YARN-6467 it will be in default partition)
> But having the partition metrics would be very useful.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message