hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13819) Make RPC layer CellBlock buffer a DirectByteBuffer
Date Wed, 23 Mar 2016 22:58:25 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15209371#comment-15209371
] 

stack commented on HBASE-13819:
-------------------------------

bq. We recently pulled this patch internally and are seeing some significant side effects
of BoundedByteBufferPool where the system runs out of direct memory whenever there is some
network congestion or some client side issues.

Because we are not returning BB to the pool? The pool is growing w/o bound?

bq. Buffer size settles to 512 kb over time(from debug statements we put in) 

We should add these if not present at TRACE level.

bq. number of rpcs that were queued in the responder was around ~4000 and this leads to exhaustion
of the direct buffer space, 

So 2G instead of 1G? But the pool is bounded?

The responder is not keeping up? It is not moving stuff out of the server fast enough?

Where is pendingCallsQueue?

Did you observe the offheap size used growing? There s a metric IIRC.

I am interested in leaks; how they are happening.

bq. ...and having a fixed size buffers instead

Where would the fixed size be? In BBBP they eventually reach fixed size?

Thanks.




> Make RPC layer CellBlock buffer a DirectByteBuffer
> --------------------------------------------------
>
>                 Key: HBASE-13819
>                 URL: https://issues.apache.org/jira/browse/HBASE-13819
>             Project: HBase
>          Issue Type: Sub-task
>          Components: Scanners
>            Reporter: Anoop Sam John
>            Assignee: Anoop Sam John
>             Fix For: 2.0.0, 1.3.0
>
>         Attachments: HBASE-13819.patch, HBASE-13819_branch-1.patch, HBASE-13819_branch-1.patch,
HBASE-13819_branch-1.patch
>
>
> In RPC layer, when we make a cellBlock to put as RPC payload, we will make an on heap
byte buffer (via BoundedByteBufferPool). The pool will keep upto certain number of buffers.
This jira aims at testing possibility for making this buffers off heap ones. (DBB)  The advantages
> 1. Unsafe based writes to off heap is faster than that to on heap. Now we are not using
unsafe based writes at all. Even if we add, DBB will be better
> 2. When Cells are backed by off heap (HBASE-11425) off heap to off heap writes will be
better
> 3. When checked the code in SocketChannel impl, if we pass a HeapByteBuffer to the socket
channel, it will create a temp DBB and copy data to there and only DBBs will be moved to Sockets.
If we make DBB 1st hand itself, we can  avoid this one more level of copying.
> Will do different perf testing with changed and report back.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message