hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Otis Gospodnetic <otis.gospodne...@gmail.com>
Subject Re: Monitor off heap Bucket Cache
Date Tue, 02 Jun 2015 17:19:28 GMT
Hi Dejan,

On Mon, Jun 1, 2015 at 5:24 AM, Dejan Menges <dejan.menges@gmail.com> wrote:

> Hi,
>
> What's the best way to monitor / know how's bucket cache being used, how
> much stuff is cached there, etc?
>

Have a look at these:
https://apps.sematext.com/spm-reports/s/ivzcFizKhG
https://apps.sematext.com/spm-reports/s/UqSDOkO6AX

If this is what you are after, see http://sematext.com/spm

Note the above charts / metrics are for 0.98 HBase.  Not sure if these
metrics changed in 1.x.

Otis
--
Monitoring * Alerting * Anomaly Detection * Centralized Log Management
Solr & Elasticsearch Support * http://sematext.com/



> Our RegionServer can use 32G of heap size, so we exported HBASE_OFFHEAPSIZE
> to 24G in hbase-env.sh, set hfile.block.cache.size to 0.05, and set couple
> of block sizes that we know we are using knowing our usage patterns. And
> this is where strange part starts - in web UI we see now, with turning this
> off, with those values, that total BlockCache available is 1G - before it
> was 10G. What we basically tried to achieve was to double it to 20G.
>
> Documentation we were referring to was
>
> http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/admin_hbase_blockcache_configure.html#concept_cp3_fhy_dr_unique_1__section_m3r_2cz_dr_unique_1
> as HBase book is not going into too much details how to properly configure
> this and get what you want.
>
> Btw. if we put hfile.block.cache.size to 0.2 we see in web UI that total
> available cache is 24G, but then after some time we had region server
> crashing. Host server have enough RAM, considering only data node and
> region server running there (128G of RAM in total) so we thought we could
> increase caching with turning on this functionality.
>
> Do you maybe see what exactly we are doing wrong? How to exactly increase
> offheap caching - and is it possible to monitor it anyhow, as in metrics we
> don't see anything associating to it?
>
> Thanks,
> Dejan
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message