hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hiroshi Ikeda (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14490) [RpcServer] reuse request read buffer
Date Fri, 12 Feb 2016 03:19:18 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15143956#comment-15143956

Hiroshi Ikeda commented on HBASE-14490:

It's been so long and I can remember some things...

I prefer to use a pool of direct buffers whose have a fixed length, NIO_BUFFER_LIMIT, that
is assumed to be larger than the size of the buffer of a native socket. That is not for preventing
GC (and that is still required to use heap buffers), but for reducing the count of native
calls to get chopped data, and that is more efficient rather than worrying about GC. Moreover,
as you know, when you use a heap buffer the Oracle implementation uses an internal direct
buffer pool and copies data between them, and not only that might cause shortage of the off-heap
area, but also that causes overhead of redundant copying. The direct buffer pool can be shared
for writing and BoundedByteBufferPool will be no longer necessary, but I was rejected in some
other JIRA issue if I remember right.

Some operations actually require a byte array, and I think it is meaningless to change signatures
to pass a byte buffer under the condition that the buffer is on-heap. It is better to change
to pass a byte array with a offset and a length, though I think paying a cost to create an
instance of holding them might be subtle. I remember that there is confusion somewhere that
some of methods use them as a start-position and an end-potion.

Also I remember a bug that some byte buffer is not hold when coming data is insufficient.

> [RpcServer] reuse request read buffer
> -------------------------------------
>                 Key: HBASE-14490
>                 URL: https://issues.apache.org/jira/browse/HBASE-14490
>             Project: HBase
>          Issue Type: Improvement
>          Components: IPC/RPC
>    Affects Versions: 2.0.0, 1.0.2
>            Reporter: Zephyr Guo
>            Assignee: Zephyr Guo
>              Labels: performance
>             Fix For: 2.0.0, 1.0.2
>         Attachments: 14490.hack.to.1.2.patch, ByteBufferPool.java, HBASE-14490-v1.patch,
HBASE-14490-v10.patch, HBASE-14490-v11.patch, HBASE-14490-v12.patch, HBASE-14490-v2.patch,
HBASE-14490-v3.patch, HBASE-14490-v4.patch, HBASE-14490-v5.patch, HBASE-14490-v6.patch, HBASE-14490-v7.patch,
HBASE-14490-v8.patch, HBASE-14490-v9.patch, gc.png, hits.png, test-v12-patch
> Reusing buffer to read request.It's not necessary to every request free buffer.The idea
of optimization is to reduce the times that allocate ByteBuffer.
> *Modification*
> 1. {{saslReadAndProcess}} ,{{processOneRpc}}, {{processUnwrappedData}}, {{processConnectionHeader}}
accept a ByteBuffer instead of byte[].They can move {{ByteBuffer.position}} correctly when
we have read the data.
> 2. {{processUnwrappedData}} no longer use any extra memory.
> 3. Maintaining a buffer pool in each {{Connection}}.

This message was sent by Atlassian JIRA

View raw message