hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Enis Soztutar (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15161) Umbrella: Miscellaneous improvements from production usage
Date Wed, 06 Apr 2016 18:48:25 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15228877#comment-15228877

Enis Soztutar commented on HBASE-15161:

In sub tasks of these for new metrics, especially per-table, per-user and hdfs IO metrics,
we discussed briefly about possible perf impact. General agreement is that we should be careful
in adding new metrics to not affect the hot code paths. For example, HBASE-15160 adds the
fs IO latency metrics back which was removed from before. The way it works is the hfile block
reader emits the latency and offers it to a array blocking queue, and later the metrics collector
which runs periodically drains these queues and adds it to the latency histogram metrics.
Instead of this, I was suggesting to increment the metrics directly inline in the hot code
paths directly, because in-theory using {{Counter}}s should be better than using {{ArrayBlockingQueue}}

To quantify whether it is the case or not, I've written a JMH suite for testing Counter vs
AtomicLong vs MetricHistogram vs ArrayBlockingQueue. 

Running these with 1,2,4,8,16,24 threads, some of the results are expected and good confirmation
(Counter is really far better than AtomicLong), it seems that MutableHistogram is very expensive:

> Umbrella: Miscellaneous improvements from production usage
> ----------------------------------------------------------
>                 Key: HBASE-15161
>                 URL: https://issues.apache.org/jira/browse/HBASE-15161
>             Project: HBase
>          Issue Type: Umbrella
>            Reporter: Yu Li
>            Assignee: Yu Li
> We use HBase to (mainly) build index for our search engine in Alibaba. Recently we are
upgrading our online cluster from 0.98.12 to 1.x and I'd like to take the opportunity to contribute
a bunch of our private patches to community (better late than never, I hope :-)). This is
an umbrella to track this effort.

This message was sent by Atlassian JIRA

View raw message