hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ramkrishna.s.vasudevan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15525) OutOfMemory could occur when using BoundedByteBufferPool during RPC bursts
Date Mon, 09 May 2016 15:25:13 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15276495#comment-15276495
] 

ramkrishna.s.vasudevan commented on HBASE-15525:
------------------------------------------------

Great work!! Looks great. 
Few questions
{code}
protected List<ByteBuffer> allBufs = new ArrayList<ByteBuffer>();
45	  protected List<ByteBuffer> bufsFromPool = new ArrayList<ByteBuffer>();
{code}
Should we really need 2 lists here?  BB that are direct can be considered to have come from
BB pool?
Ya the flip logic is needed for the Socket to consume the Cellblocks. 
'PoolAwareByteBuffersOutputStream ' should this take the Pool itself?  Other than the queue
from where the BB are polled, can we have the BB Pool to maintain the list of Buffers and
what ever buffer is created can we just use the older ByteBufferOutputStream only ?  So the
entire BB creating and pooling logic will be inside BBPool only. I have not tried  but asking.


> OutOfMemory could occur when using BoundedByteBufferPool during RPC bursts
> --------------------------------------------------------------------------
>
>                 Key: HBASE-15525
>                 URL: https://issues.apache.org/jira/browse/HBASE-15525
>             Project: HBase
>          Issue Type: Sub-task
>          Components: IPC/RPC
>            Reporter: deepankar
>            Assignee: Anoop Sam John
>            Priority: Critical
>             Fix For: 2.0.0
>
>         Attachments: HBASE-15525_V1.patch, HBASE-15525_V2.patch, HBASE-15525_V3.patch,
HBASE-15525_WIP.patch, WIP.patch
>
>
> After HBASE-13819 the system some times run out of direct memory whenever there is some
network congestion or some client side issues.
> This was because of pending RPCs in the RPCServer$Connection.responseQueue and since
all the responses in this queue hold a buffer for cellblock from BoundedByteBufferPool this
could takeup a lot of memory if the BoundedByteBufferPool's moving average settles down towards
a higher value 
> See the discussion here [HBASE-13819-comment|https://issues.apache.org/jira/browse/HBASE-13819?focusedCommentId=15207822&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15207822]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message