hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-10403) Simplify offheap cache configuration
Date Tue, 15 Jul 2014 04:28:04 GMT

    [ https://issues.apache.org/jira/browse/HBASE-10403?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14061642#comment-14061642
] 

stack commented on HBASE-10403:
-------------------------------

bq. Turning it on currently requires explicitly setting LruBlockCache (L1) instance size and
IOEngine (L2) size, making sure that L1 size isn't too big vs global memstore and total heap.
This is further confused by hbase.bucketcache.size accepting a percentage of total heap OR
explicit size in MB. Enabling SlabCache is slightly easier in that it just accepts whatever
LruBlockCache is provided.

SlabCache is gone as is DoubleBlockCache.  That helps this simplification project.

If we removed "hbase.bucketcache.combinedcache.enabled"  that would simplify L1+L2 setup a
good bit.  LruBlockCache is configured the same whether an L2 or not.  An ioengine and a non-zero
bucketcache size enables the bucket cache.  This could serve as a simpler first cut at this
JIRA?  Three-tiers of cache and more work separating caching policy classes from block cache
implementations can also come in later issues?



> Simplify offheap cache configuration
> ------------------------------------
>
>                 Key: HBASE-10403
>                 URL: https://issues.apache.org/jira/browse/HBASE-10403
>             Project: HBase
>          Issue Type: Bug
>          Components: io
>            Reporter: Nick Dimiduk
>            Assignee: Nick Dimiduk
>            Priority: Minor
>             Fix For: 0.99.0
>
>         Attachments: HBASE-10403.0.patch
>
>
> The BucketCache (HBASE-7404) is a very nice piece of functionality which is hidden behind
complex configuration. Enabling it currently requires manual calculation of L1 cache. It'd
be nice to make this easier to use and conform better with the existing heap management tools
we already have.
> Turning it on currently requires explicitly setting LruBlockCache (L1) instance size
and IOEngine (L2) size, making sure that L1 size isn't too big vs global memstore and total
heap. This is further confused by hbase.bucketcache.size accepting a percentage of total heap
OR explicit size in MB. Enabling SlabCache is slightly easier in that it just accepts whatever
LruBlockCache is provided.
> Turning on BucketCache using off-heap mode could look like:
> hbase-env.sh:
>  Set HBASE_REGIONSERVER_OPTS:
>   -Xmx5000m
>   -XX:MaxDirectMemorySize=15000m
> hbase-site.xml:
>  - hbase.regionserver.global.memstore.size = 0.7
>  - hbase.regionserver.onheap.blockcache.size = 0.1
>  - hbase.regionserver.blockcache.impl = BucketCache
>  - hbase.bucketcache.ioengine = offheap
> The result being a CombinedCache instance with 500m LruBlockCache + 15000m ByteBufferIOEngine
running in direct mode.
> This example does a couple things (mostly for the admin):
>  - knows NOT to enable SlabCache
>  - s/hfile.block.cache.size/hbase.regionserver.onheap.blockcache.size/
>  - maintains the validity of HBaseConfiguration's existing check that global MemStore
+ LruBlockCache == 0.8
>  - maps "BucketCache" into meaning "a CombinedCache instance with these implementations
for L1 and L2."
>  - Figures out appropriate values for hbase.bucketcache.size and hbase.bucketcache.percentage.in.combinedcache



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message