hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron T. Myers (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3650) Use MutableQuantiles to provide latency histograms for various operations
Date Wed, 25 Jul 2012 00:17:34 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13421886#comment-13421886

Aaron T. Myers commented on HDFS-3650:

Patch looks great, Andrew. Just a few little comments:

# Instead of doing Configuration#get(...) and then doing the comma separating yourself, you
can use Configuration#getTrimmedStringCollection, which will do the comma handling for you.
For that matter, it might be nice to add a getIntegerCollection method to the Configuration
class, to also handle the integer parsing.
# I find the variable name "splitted" rather unfortunate. How about "splitValues" ?
# There are a few spurious whitespace changes in TestDataNodeMetrics.
# You should add an entry in hdfs-default.xml for the new dfs.metrics.percentiles.intervals.key,
even if it has an empty value, so that you can add a description of what it does, and the
format of what it should be set to.
# I find the loop try/catch of AssertionError in TestDataNodeMetrics#testRoundTripAckPercentilesMetric
kind of unfortunate. How about instead you get the list of DNs involved in the write pipeline
via DFSOutputStream#getPipeline when writing the file, and then always assert the quantile
gauges on the actual appropriate DN?
# If assertQuantileGauges are identical between TestDataNodeMetrics and TestNameNodeMetrics,
how about refactoring those methods? Perhaps as a static method in DFSTestUtil?
# Could also stand to refactor the two new tests in TestNameNodeMetrics, since they appear
identical, save for two values.
> Use MutableQuantiles to provide latency histograms for various operations
> -------------------------------------------------------------------------
>                 Key: HDFS-3650
>                 URL: https://issues.apache.org/jira/browse/HDFS-3650
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>    Affects Versions: 2.0.0-alpha
>            Reporter: Andrew Wang
>            Assignee: Andrew Wang
>         Attachments: hdfs-3650-1.patch
> MutableQuantiles provide accurate estimation of various percentiles for a stream of data.
Many existing metrics reported by a MutableRate would also benefit from having these percentiles;
lets add MutableQuantiles where we think it'd be useful.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message