hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ramkrishna.s.vasudevan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-17738) BucketCache startup is slow
Date Fri, 26 May 2017 11:05:05 GMT

    [ https://issues.apache.org/jira/browse/HBASE-17738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16026148#comment-16026148
] 

ramkrishna.s.vasudevan commented on HBASE-17738:
------------------------------------------------

-> Patch makes BB creation multi threaded.
-> It avoids BB APIS and instead goes with Unsafe directly. 
The biggest drawback here is that we won't be accountng for the memory that really got created
offheap and check it against the max offheap limit that is configured as DirectMAxMemory.
How ever we have a check in the code to see if Bucket cache is going bigger than the DirectMaxMemory
config.
Do we need unsafe way of creation or just the normal BB way but make it multi threaded? If
so the first patch is fine but the gain startup time may be like 50% better but not more than
that for bigger offheap sizes.
Ping [~saint.ack@gmail.com], [~anoop.hbase]


> BucketCache startup is slow
> ---------------------------
>
>                 Key: HBASE-17738
>                 URL: https://issues.apache.org/jira/browse/HBASE-17738
>             Project: HBase
>          Issue Type: Sub-task
>          Components: BucketCache
>    Affects Versions: 2.0.0
>            Reporter: stack
>            Assignee: ramkrishna.s.vasudevan
>             Fix For: 2.0.0
>
>         Attachments: HBASE-17738_2.patch, HBASE-17738_2.patch, HBASE-17738_3.patch, HBASE-17738_4.patch,
HBASE-17738.patch
>
>
> If you set bucketcache size at 64G say and then start hbase, it takes a long time. Can
we do the allocations in parallel and not inline with the server startup?
> Related, prefetching on a bucketcache is slow. Speed it up.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message