hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anoop Sam John (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15525) OutOfMemory could occur when using BoundedByteBufferPool during RPC bursts
Date Tue, 29 Mar 2016 04:27:25 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15215397#comment-15215397
] 

Anoop Sam John commented on HBASE-15525:
----------------------------------------

Yes Stack,  while reading request into the BB and writing response (Using ByteBufferChain),
 we do read/write from/to socket channel in 64 KB chunks.  So ya while reading also, we can
read into N buffers. This is some thing I discussed with you offline some time back  :-)
Ya the pool will be having all fixed sized BBs and off heap.  The max #BBs in the pool can
be defined by customer. (Default to 2 * handlers)..  We do we have this config already.  The
name of the config is bit odd to me ..  We will discuss that.
There is no initial buffer size and max buffer size.. Only one config using which one can
define the size of the BBs in the pool.  Default of this is set as 64KB same as the socket
chunk size.  If for a use case it is sure that most of the time the response size will be
lesser than this, they can reduce it.  For scan kind of usage advisable not to change this
size.  When the response size is expected to be more than 64KB also, better not change as
this is the chunk size. So even if the buffer size is 256KB we will do 4 writes to socket.
And ya the main another diff is when pool is not able to find a free BB (after reaching the
max capacity), pool will return null..  And the response will allocate new BB in on heap area.

> OutOfMemory could occur when using BoundedByteBufferPool during RPC bursts
> --------------------------------------------------------------------------
>
>                 Key: HBASE-15525
>                 URL: https://issues.apache.org/jira/browse/HBASE-15525
>             Project: HBase
>          Issue Type: Bug
>          Components: IPC/RPC
>            Reporter: deepankar
>            Assignee: Anoop Sam John
>            Priority: Critical
>         Attachments: WIP.patch
>
>
> After HBASE-13819 the system some times run out of direct memory whenever there is some
network congestion or some client side issues.
> This was because of pending RPCs in the RPCServer$Connection.responseQueue and since
all the responses in this queue hold a buffer for cellblock from BoundedByteBufferPool this
could takeup a lot of memory if the BoundedByteBufferPool's moving average settles down towards
a higher value 
> See the discussion here [HBASE-13819-comment|https://issues.apache.org/jira/browse/HBASE-13819?focusedCommentId=15207822&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15207822]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message