hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: How to validate HBase configuration (for region servers) from HBase admin page?
Date Mon, 28 Jul 2014 22:45:17 GMT
On Mon, Jul 28, 2014 at 1:50 PM, gabriela montiel <
gabriela.montiel@oracle.com> wrote:

> Hi St.Ack,
>
> We are working with HBase 0.96.1.1-cdh5.0.1 using 10 region servers. We
> configured the bucket cache and set the -XX:MaxDirectMemorySize to 32g,
> heap size of 12g, and hbase.regionserver.count=40. The bucket cache
> configuration is the following (added in hbase-site.xml):
>
>
>  <property>
>     <name>hbase.regionserver.handler.count</name>
>     <value>40</value>
>   </property>
>   <property>
>     <name>hbase.bucketcache.ioengine</name>
>     <value>offheap</value>
>   </property>
>   <property>
> <name>hbase.bucketcache.percentage.in.combinedcache</name>
>     <value>0.8</value>
>   </property>
>   <property>
>     <name>hbase.bucketcache.size</name>
>     <value>40000</value>
>   </property>
>
>
> After restarting HBase, we checked out the HBase process from console we
> are able to set the -XX:MaxDirectMemorySize=32, as well as the memory heap
> size we configured.  However, when we go to the http://<hostname>:60010/conf,
> the*hbase.regionserver.count* is still set up to 30 (presumably the default
> value) and the bucketcache parameters added (bucketcache.ioengine,
> bucketcache.percentage.in.combinedcache, bucketcache.size) are not showed
> up, so we are not sure if the parameters are taken into account or not.
>
>
I do not know what hbase.regionserver.count is.  Do you mean
hbase.regionserver.handler.count?


> Is there a way to validate that the bucketcache parameters as well as the
> new value for the hbase.regionserver.count are really taken from the
> hbase-site.xml we changed?
>

It is bit awkward in 0.96 but look in your logs.

Look for '      LOG.info("Allocating LruBlockCache with maximum size " +
StringUtils.humanReadableInt(lruCacheSize));'  Should be 0.2 * 12G.

Also looking for      LOG.info("Started bucket cache");

If you turn on DEBUG even temporarily on the regionserver, you should see
stats about your block cache emitted every 5 minutes or so:

     LOG.debug("BucketCache Stats: " +
         "failedBlockAdditions=" + this.failedBlockAdditions.get() + ", " +
         "total=" + StringUtils.byteDesc(totalSize) + ", " +
         "free=" + StringUtils.byteDesc(freeSize) + ", " +
         "usedSize=" + StringUtils.byteDesc(usedSize) +", " +
         "cacheSize=" + StringUtils.byteDesc(cacheSize) +", " +
         "accesses=" + cacheStats.getRequestCount() + ", " +
         "hits=" + cacheStats.getHitCount() + ", " +
         "IOhitsPerSecond=" + cacheStats.getIOHitsPerSecond() + ", " +
         "IOTimePerHit=" + String.format("%.2f",
cacheStats.getIOTimePerHit())+ ", " +
         "hitRatio=" + (cacheStats.getHitCount() == 0 ? "0," :
           (StringUtils.formatPercent(cacheStats.getHitRatio(), 2)+ ", ")) +
         "cachingAccesses=" + cacheStats.getRequestCachingCount() + ", " +
         "cachingHits=" + cacheStats.getHitCachingCount() + ", " +
         "cachingHitsRatio=" +(cacheStats.getHitCachingCount() == 0 ? "0," :
           (StringUtils.formatPercent(cacheStats.getHitCachingRatio(), 2)+
", ")) +
         "evictions=" + cacheStats.getEvictionCount() + ", " +
         "evicted=" + cacheStats.getEvictedCount() + ", " +
         "evictedPerRun=" + cacheStats.evictedPerEviction());

St.Ack

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message