hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Elliott Clark (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-6261) Better approximate high-percentile percentile latency metrics
Date Thu, 28 Jun 2012 23:24:44 GMT

    [ https://issues.apache.org/jira/browse/HBASE-6261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13403576#comment-13403576
] 

Elliott Clark commented on HBASE-6261:
--------------------------------------

If it only lands in hadoop are we going to be able to use it at all?  Reflection doesn't seem
like it's really viable here where we're trying to call the same method on lots of different
Histogram objects; it would be pretty slow on top of the perf hit we would be taking for the
added accuracy.  

Can it just replace MetricsHistogram ?
                
> Better approximate high-percentile percentile latency metrics
> -------------------------------------------------------------
>
>                 Key: HBASE-6261
>                 URL: https://issues.apache.org/jira/browse/HBASE-6261
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Andrew Wang
>              Labels: metrics
>         Attachments: Latencyestimation.pdf
>
>
> The existing reservoir-sampling based latency metrics in HBase are not well-suited for
providing accurate estimates of high-percentile (e.g. 90th, 95th, or 99th) latency. This is
a well-studied problem in the literature (see [1] and [2]), the question is determining which
methods best suit our needs and then implementing it.
> Ideally, we should be able to estimate these high percentiles with minimal memory and
CPU usage as well as minimal error (e.g. 1% error on 90th, or .1% on 99th). It's also desirable
to provide this over different time-based sliding windows, e.g. last 1 min, 5 mins, 15 mins,
and 1 hour.
> I'll note that this would also be useful in HDFS, or really anywhere latency metrics
are kept.
> [1] http://www.cs.rutgers.edu/~muthu/bquant.pdf
> [2] http://infolab.stanford.edu/~manku/papers/04pods-sliding.pdf

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message