hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ted Tuttle" <ted.tut...@mentacapital.com>
Subject increasing block cache size
Date Fri, 16 Nov 2012 18:31:11 GMT
Hello All-


We are running 5 node v0.94 cluster on machines w/ 32G of RAM.  I'd like
to utilize more of our memory for block cache.  So I have a couple of


Currently, we are running RSs w/ 16G heap and the following memory






Typically overall stats for RS are:


  numberOfOnlineRegions=228, usedHeapMB=7870, maxHeapMB=16358


Used heap typically goes to ~10G then we get GC back down to about 6-8G.
So by my reckoning we have about 25% of heap unutilized.


More detailed, typical RS info:


numberOfOnlineRegions=245, numberOfStores=245, numberOfStorefiles=249,
storefileIndexSizeMB=17, rootIndexSizeKB=17688,
totalStaticIndexSizeKB=11910, totalStaticBloomSizeKB=36107,
memstoreSizeMB=361, readRequestsCount=-1829306476,
writeRequestsCount=16153099, compactionQueueSize=0, flushQueueSize=0,
usedHeapMB=11407, maxHeapMB=16358, blockCacheSizeMB=3201.95,
blockCacheFreeMB=887.66, blockCacheCount=3243,
blockCacheHitCount=2932634895, blockCacheMissCount=5223350,
blockCacheEvictedCount=678037, blockCacheHitRatio=99%,
blockCacheHitCachingRatio=99%, hdfsBlocksLocalityIndex=95


>From these stats I see my memstores are always <1G or <6% of heap.


So here are my questions:


1) Are memstore settings above relevant in v0.94 w/ off heap memstores
the default?

2) We have good locality and cache hits.  I'd like to take block cache
size to 50% and increase overall heap to 24G.   Any concerns about this
given stats above?

3) Any other memory recommendations?





  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message