hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14490) [RpcServer] reuse request read buffer
Date Wed, 04 Nov 2015 05:05:27 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14988891#comment-14988891
] 

stack commented on HBASE-14490:
-------------------------------

Good review [~ikeda]

bq. Is that right usage to throw away a buffer gotten from BoundedByteBufferPool?

Good one [~ikeda] Your patch should go faster if you put the buffer back rather than discard
[~gzh1992n]


bq. Did you check the frequency of GC? I'm interested in how much these patches reduce it,
though it also requires a result of the code without patches.

If it helps, I could run these tests.

bq. To tell the truth, checking the size is required before memory allocation in order to
prevent OutOfMemoryError caused by a malicious client.

True, but beyond the scope of this issue I'd say (we'll happily allocate the buffer whether
pool or not....)

I'm still interested in whether the BBBP allocations were onheap or off. Would be cool if
offheap. I can try it with BBBP offheap too.

> [RpcServer] reuse request read buffer
> -------------------------------------
>
>                 Key: HBASE-14490
>                 URL: https://issues.apache.org/jira/browse/HBASE-14490
>             Project: HBase
>          Issue Type: Improvement
>          Components: IPC/RPC
>    Affects Versions: 2.0.0, 1.0.2
>            Reporter: Zephyr Guo
>            Assignee: Zephyr Guo
>              Labels: performance
>             Fix For: 2.0.0, 1.0.2
>
>         Attachments: ByteBufferPool.java, HBASE-14490-v1.patch, HBASE-14490-v10.patch,
HBASE-14490-v11.patch, HBASE-14490-v12.patch, HBASE-14490-v2.patch, HBASE-14490-v3.patch,
HBASE-14490-v4.patch, HBASE-14490-v5.patch, HBASE-14490-v6.patch, HBASE-14490-v7.patch, HBASE-14490-v8.patch,
HBASE-14490-v9.patch
>
>
> Reuse buffer to read request.It's not necessary to every request free buffer.The idea
of optimization is to reduce the times that allocate ByteBuffer.
> *Modification*
> 1. {{saslReadAndProcess}} ,{{processOneRpc}}, {{processUnwrappedData}}, {{processConnectionHeader}}
accept a ByteBuffer instead of byte[].They can move {{ByteBuffer.position}} correctly when
we have read the data.
> 2. {{processUnwrappedData}} no longer use any extra memory.
> 3. Maintaining a reused ByteBuffer in each {{Connection}}.
> a) Size is dynamic to fit specific client.
> b) If size deviate far from average size, we replace a new one.
> c) Using a SoftReference to reference the buffer.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message