hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "chunhui shen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-10403) Simplify offheap cache configuration
Date Sun, 26 Jan 2014 02:25:38 GMT

    [ https://issues.apache.org/jira/browse/HBASE-10403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13882146#comment-13882146

chunhui shen commented on HBASE-10403:

As my personal thought,  I'd like to fix the usage of BlockCache, Two much combination  would
confuse the general users.

In current configuration implementation with patch,  user should config the class name of
"hbase.block.cache.impl". For me, class name is hard to remember. Also setting it as 'CombinedBlockCache'
or 'DoubleBlockCache' seems unfamiliar.
So, should we use the 'type' instead of 'class name', just like Compression algorithm and
DataBlockEncoding algorithm.
For example,
hbase.block.cache.type='LruCache', default value, use LruBlockCache.
hbase.block.cache.type='BucketCache', according to the value of 'hbase.bucketcache.ioengine',
 construct the BlockCache.('heap' or 'offheap' use CombinedBlockCache, 'file' use BucketCache
directly as the L2 cache)
hbase.block.cache.type='SlabCache', use DoubleBlockCache.

Finally,  I think the patch is quite well now. If we could set an familiar type value for
BlockCache instead of detail class name, it will be better if I'm a general user.

> Simplify offheap cache configuration
> ------------------------------------
>                 Key: HBASE-10403
>                 URL: https://issues.apache.org/jira/browse/HBASE-10403
>             Project: HBase
>          Issue Type: Bug
>          Components: io
>            Reporter: Nick Dimiduk
>            Assignee: Nick Dimiduk
>            Priority: Minor
>         Attachments: HBASE-10403.0.patch
> The BucketCache (HBASE-7404) is a very nice piece of functionality which is hidden behind
complex configuration. Enabling it currently requires manual calculation of L1 cache. It'd
be nice to make this easier to use and conform better with the existing heap management tools
we already have.
> Turning it on currently requires explicitly setting LruBlockCache (L1) instance size
and IOEngine (L2) size, making sure that L1 size isn't too big vs global memstore and total
heap. This is further confused by hbase.bucketcache.size accepting a percentage of total heap
OR explicit size in MB. Enabling SlabCache is slightly easier in that it just accepts whatever
LruBlockCache is provided.
> Turning on BucketCache using off-heap mode could look like:
> hbase-env.sh:
>   -Xmx5000m
>   -XX:MaxDirectMemorySize=15000m
> hbase-site.xml:
>  - hbase.regionserver.global.memstore.size = 0.7
>  - hbase.regionserver.onheap.blockcache.size = 0.1
>  - hbase.regionserver.blockcache.impl = BucketCache
>  - hbase.bucketcache.ioengine = offheap
> The result being a CombinedCache instance with 500m LruBlockCache + 15000m ByteBufferIOEngine
running in direct mode.
> This example does a couple things (mostly for the admin):
>  - knows NOT to enable SlabCache
>  - s/hfile.block.cache.size/hbase.regionserver.onheap.blockcache.size/
>  - maintains the validity of HBaseConfiguration's existing check that global MemStore
+ LruBlockCache == 0.8
>  - maps "BucketCache" into meaning "a CombinedCache instance with these implementations
for L1 and L2."
>  - Figures out appropriate values for hbase.bucketcache.size and hbase.bucketcache.percentage.in.combinedcache

This message was sent by Atlassian JIRA

View raw message