hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anthony Nguyen <anthony.an.ngu...@gmail.com>
Subject hbase.bucketcache.percentage.in.combinedcache with ioengine=<FILE ON SSD> on HBase 0.98
Date Tue, 06 Oct 2015 18:36:51 GMT
Hi all,

I'm running HBase 0.98, with the complicated percentage.in.combinedcache
setting. What's the best way to set the percentage.in.combinedcache setting
when using a file on SSD storage as the location for ioengine? The SSD
drive is dedicated for the cache. I don't want to modify the LruBlockCache
settings at all, keeping it available for index + bloom and just add the L2
SSD cache for storage for the data blocks (not the raw L1+L2 method).

My confusion is in how I should best modify the percentage.in.combinedcache
setting. HBASE-11520 makes things much clearer, but unfortunately it isn't
in 0.98. To me, it doesn't make much sense to have L1 be set to
hbase.bucketcache.size (100G) * (1.0 -
hbase.bucketcache.percentage.in.combinedcache (default=0.9) = 1G when I
want the L1 to be the same as it was prior to bucketcache
(hfile.block.cache.size * heap).

Also, I don't want to have any direct addressing of memory since my offheap
is completely on SSD. Does this mean I shouldn't change
XX:MaxDirectMemorySize?

Could someone sanity check these settings for me? Thanks!

30 GB RAM
~120 GB SSD
HBASE_REGIONSERVER_OPTS -Xms17G -Xmx17G
XX:MaxDirectMemorySize <- Don't touch since we're not using direct byte
buffers, right?

<property>
  <name>hbase.bucketcache.ioengine</name>
  <value>file:/mnt/foo/bar</value>
</property>
<property>
  <name>hbase.bucketcache.percentage.in.combinedcache</name>
  <value>0.9</value>
</property>

<property>
  <name>hfile.block.cache.size</name>
  <value>0.6</value>
</property>

<property>
  <name>hbase.bucketcache.size</name>
  <value>102400</value>
</property>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message