hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vladimir Rodionov <vrodio...@carrieriq.com>
Subject RE: Block cache hit ratio question
Date Sat, 19 Oct 2013 00:04:54 GMT
HBase block cache does not expose any statistics on a per-table basis. This looks like a nice
to have feature in
a next releases. If your load is mostly random reads, than several factors affect the overall

1. The avg read size / block size ratio. The large ratio - the better block cache hit ration
you will have.
2. Hot data set size.
3 Available RAM for block cache

For example,

Block size = 64K
Avg read size = 1K
Hot data set size = 1GB

You will need 64GB of block cache to cache 100% your random reads.
Usually, for random reads the smaller block size - the better, but you can not decrease block
size w/o additional overhead of bookkeeping
larger number of smaller blocks. Your actual limit is around 16KB.

Best regards,
Vladimir Rodionov
Principal Platform Engineer
Carrier IQ, www.carrieriq.com
e-mail: vrodionov@carrieriq.com

From: Vaibhav Puranik [vpuranik@gmail.com]
Sent: Friday, October 18, 2013 3:20 PM
To: user@hbase.apache.org
Subject: Block cache hit ratio question

Hi all,

Our HBase is slowing down because of lots of disk reads. We are trying to
analyze it. We have lots of real time random reads. And We found that HBase
is constantly reading from disk instead of from memory.

Looks like block cache hit ratio is a good place to start.

Is there any way we can get block cache hit ratio by table or by region? I
could only find it per region server in region server UI. I couldn't find
it in API either.

Feel free to suggest any other metrics we should be looking at too.


Confidentiality Notice:  The information contained in this message, including any attachments
hereto, may be confidential and is intended to be read only by the individual or entity to
whom this message is addressed. If the reader of this message is not the intended recipient
or an agent or designee of the intended recipient, please note that any review, use, disclosure
or distribution of this message or its attachments, in any form, is strictly prohibited. 
If you have received this message in error, please immediately notify the sender and/or Notifications@carrieriq.com
and delete or destroy any copy of this message and its attachments.

View raw message