hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dejan Menges <dejan.men...@gmail.com>
Subject Monitor off heap Bucket Cache
Date Mon, 01 Jun 2015 09:24:21 GMT
Hi,

What's the best way to monitor / know how's bucket cache being used, how
much stuff is cached there, etc?

Our RegionServer can use 32G of heap size, so we exported HBASE_OFFHEAPSIZE
to 24G in hbase-env.sh, set hfile.block.cache.size to 0.05, and set couple
of block sizes that we know we are using knowing our usage patterns. And
this is where strange part starts - in web UI we see now, with turning this
off, with those values, that total BlockCache available is 1G - before it
was 10G. What we basically tried to achieve was to double it to 20G.

Documentation we were referring to was
http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/admin_hbase_blockcache_configure.html#concept_cp3_fhy_dr_unique_1__section_m3r_2cz_dr_unique_1
as HBase book is not going into too much details how to properly configure
this and get what you want.

Btw. if we put hfile.block.cache.size to 0.2 we see in web UI that total
available cache is 24G, but then after some time we had region server
crashing. Host server have enough RAM, considering only data node and
region server running there (128G of RAM in total) so we thought we could
increase caching with turning on this functionality.

Do you maybe see what exactly we are doing wrong? How to exactly increase
offheap caching - and is it possible to monitor it anyhow, as in metrics we
don't see anything associating to it?

Thanks,
Dejan

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message