hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ramkrishna.s.vasudevan (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HBASE-17738) BucketCache startup is slow
Date Thu, 20 Jul 2017 10:28:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-17738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16094468#comment-16094468
] 

ramkrishna.s.vasudevan edited comment on HBASE-17738 at 7/20/17 10:27 AM:
--------------------------------------------------------------------------

The Math.round() was causing the issue. In case we need to create 25 buffers and 16 threads
are available, then perthreadCount by doing round off was creating 2 bytebuffers and the last
threads had to create -5 (which is wrong) and also per thread it was creating more than what
is required.
So now we do a Math.floor() and round it off to int. Creating this case with a test case is
difficult so I just reproduced the above case by making ByteBufferarray non final and mocking
the available thread count. So this test will fail based on the number of available threads
in the system and not on any other system if the resulting calculation yields better result
(based on the number of threads). 


was (Author: ram_krish):
The Math.round() was causing the issue. In case we need to create 25 buffers and 16 threads
are available, then perthreadCount by doing round off was creating 2 bytebuffers and the last
threads had to create -5 (which is wrong) and also per thread it was creating more than what
is required.
So now we do a Math.floor() and round it off to int. Creating this case with a test case is
difficult so I just reproduced the above case by making ByteBufferarray non final and mocking
the available thread count. So this test will fail based on the number of available threads
in the system and not on any other system if the resulting calculation yields better result.


> BucketCache startup is slow
> ---------------------------
>
>                 Key: HBASE-17738
>                 URL: https://issues.apache.org/jira/browse/HBASE-17738
>             Project: HBase
>          Issue Type: Sub-task
>          Components: BucketCache
>    Affects Versions: 2.0.0
>            Reporter: stack
>            Assignee: ramkrishna.s.vasudevan
>             Fix For: 2.0.0, 2.0.0-alpha-2
>
>         Attachments: HBASE-17738_10.patch, HBASE-17738_2.patch, HBASE-17738_2.patch,
HBASE-17738_3.patch, HBASE-17738_4.patch, HBASE-17738_5_withoutUnsafe.patch, HBASE-17738_6_withoutUnsafe.patch,
HBASE-17738_8.patch, HBASE-17738_9.patch, HBASE-17738_log.patch, HBASE-17738.patch
>
>
> If you set bucketcache size at 64G say and then start hbase, it takes a long time. Can
we do the allocations in parallel and not inline with the server startup?
> Related, prefetching on a bucketcache is slow. Speed it up.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message