hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13819) Make RPC layer CellBlock buffer a DirectByteBuffer
Date Thu, 24 Mar 2016 05:24:25 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15209792#comment-15209792

stack commented on HBASE-13819:

bq. Client side can do?

Server has to do it. In this case, we did not write the client (asynchbase).

Thanks for your #1 and #2. Nice summary.

bq. .... its really difficult for the user to predict how much max off heap size he need to

Yeah. Lets fix. We could multiply the max queue size in listener * maximum request size to
get max offheap buffer allocated.... Thats kinda ugly. Need the server throttling.

 bq.  (If we go out of space in off heap area)!!!

Yes. Need to bound offheap usage. What GC does when offheap up against the limit is probably
not pretty.

bq. So that is why my thinking abt changing these temp BB creation when happens, those should
be HBBs.

You still need to do accounting of the spike though.

But yeah, probably easier taking the spike onheap.... Lets discuss in new issue.

> Make RPC layer CellBlock buffer a DirectByteBuffer
> --------------------------------------------------
>                 Key: HBASE-13819
>                 URL: https://issues.apache.org/jira/browse/HBASE-13819
>             Project: HBase
>          Issue Type: Sub-task
>          Components: Scanners
>            Reporter: Anoop Sam John
>            Assignee: Anoop Sam John
>             Fix For: 2.0.0, 1.3.0
>         Attachments: HBASE-13819.patch, HBASE-13819_branch-1.patch, HBASE-13819_branch-1.patch,
HBASE-13819_branch-1.patch, q.png
> In RPC layer, when we make a cellBlock to put as RPC payload, we will make an on heap
byte buffer (via BoundedByteBufferPool). The pool will keep upto certain number of buffers.
This jira aims at testing possibility for making this buffers off heap ones. (DBB)  The advantages
> 1. Unsafe based writes to off heap is faster than that to on heap. Now we are not using
unsafe based writes at all. Even if we add, DBB will be better
> 2. When Cells are backed by off heap (HBASE-11425) off heap to off heap writes will be
> 3. When checked the code in SocketChannel impl, if we pass a HeapByteBuffer to the socket
channel, it will create a temp DBB and copy data to there and only DBBs will be moved to Sockets.
If we make DBB 1st hand itself, we can  avoid this one more level of copying.
> Will do different perf testing with changed and report back.

This message was sent by Atlassian JIRA

View raw message