hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Wang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13521) SampleQuantile does not perform well under load
Date Tue, 23 Aug 2016 23:34:21 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15433859#comment-15433859
] 

Andrew Wang commented on HADOOP-13521:
--------------------------------------

We should probably rip out MutableQuantiles and replace it with HdrHistogram, in retrospect
MQ is over-engineered.

> SampleQuantile does not perform well under load
> -----------------------------------------------
>
>                 Key: HADOOP-13521
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13521
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: metrics
>    Affects Versions: 2.3.0
>            Reporter: Mark Wagner
>         Attachments: Screen Shot 2015-07-09 at 12.37.29 PM.png
>
>
> After adding quantile collection to one of our clusters we saw much higher latency for
RPCs. This was traced down to the quantile collection. Samples are being buffered and inserted
in groups of 500. After the buffered samples are inserted, the entire set of samples for this
time period (600 seconds at the longest for us) is "compressed".
> All operations for RPC metrics are synchronized. Usually this isn't an issue but it seems
that this compression operation is taking a significant amount of time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message