hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anoop Sam John (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HBASE-17073) Increase the max number of buffers in ByteBufferPool
Date Tue, 15 Nov 2016 04:36:58 GMT

    [ https://issues.apache.org/jira/browse/HBASE-17073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15666045#comment-15666045
] 

Anoop Sam John edited comment on HBASE-17073 at 11/15/16 4:36 AM:
------------------------------------------------------------------

Thanks Stack.  Will commit in some time
bq.What we doing to do with direct memory config in 2.0? Have a default that is same as HBASE_HEAPSIZE
or some other such?
{quote}
# Uncomment below if you intend to use off heap cache. For example, to allocate 8G of 
# offheap, set the value to "8G".
# export HBASE_OFFHEAPSIZE=1G
{quote}
This is what we have in hbase-env.sh now.  We need more notes around this and some suggestion?
May be more details in book also.  This depends on whether we want L2 off heap BC as the default
data cache in 2.0.  (I would like to and will open a jira soon)  Also if users use off heap
MSLAB pool, then that also to be added up.. May be I will raise a sub task under the jira
of "memory sizing and tuning" (forgot id. Will check).. Once the off heap write path work
is also over, will work on that.. Thanks for the good Q Stack.  I forgot doing this.  Sounds
ok?


was (Author: anoop.hbase):
Thanks Stack.  Will commit in some time
bq.What we doing to do with direct memory config in 2.0? Have a default that is same as HBASE_HEAPSIZE
or some other such?
# Uncomment below if you intend to use off heap cache. For example, to allocate 8G of 
# offheap, set the value to "8G".
# export HBASE_OFFHEAPSIZE=1G
This is what we have in hbase-env.sh now.  We need more notes around this and some suggestion?
May be more details in book also.  This depends on whether we want L2 off heap BC as the default
data cache in 2.0.  (I would like to and will open a jira soon)  Also if users use off heap
MSLAB pool, then that also to be added up.. May be I will raise a sub task under the jira
of "memory sizing and tuning" (forgot id. Will check).. Once the off heap write path work
is also over, will work on that.. Thanks for the good Q Stack.  I forgot doing this.  Sounds
ok?

> Increase the max number of buffers in ByteBufferPool
> ----------------------------------------------------
>
>                 Key: HBASE-17073
>                 URL: https://issues.apache.org/jira/browse/HBASE-17073
>             Project: HBase
>          Issue Type: Sub-task
>    Affects Versions: 2.0.0
>            Reporter: Anoop Sam John
>            Assignee: Anoop Sam John
>             Fix For: 2.0.0
>
>         Attachments: HBASE-17073.patch
>
>
> Before the HBASE-15525 issue fix, we had variable sized buffers in our buffer pool. The
max size upto which one buffer can grow was 2 MB.  Now we have changed it to be a fixed sized
BBPool. By default 64 KB is the size of each buffer.  But the max number of BBs allowed to
be in the pool was not changed.  ie. twice the number of handlers. May be we should be changing
increasing it now?  To make it equal to the way like 2 MB, we will need 32 * 2 * handlers.
 There is no initial #BBs any way. 2 MB is the default max response size what we have. And
write reqs also, when it is Buffered mutator 2 MB is the default flush limit.  We can make
it to be 32 * #handlers as the def max #BBs I believe.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message