hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zephyr Guo (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-14490) [RpcServer] reuse request read buffer
Date Tue, 03 Nov 2015 16:44:28 GMT

     [ https://issues.apache.org/jira/browse/HBASE-14490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Zephyr Guo updated HBASE-14490:
-------------------------------
    Attachment: HBASE-14490-v12.patch

patch-v12:
Using {{BoundedByteBufferPool}} to share buffer in all {{Connection}}.

I did some test contrast between {{BoundedByteBufferPool}} and {{byteBufferPool}} that Hiroshi
Ikeda attached.The former looks better.

*test design*
step1: Using YCSB to put 10000 records.
step2: Dumping information.Each creating and discarding will be recorded in log.
step3: Using YCSB to read 10000 records.
step4: Dumping information

*test result*
BoundedByteBUfferPool test result:
{code}
[OVERALL], RunTime(ms), 43010.0
[OVERALL], Throughput(ops/sec), 232.50406882120438
[INSERT], Operations, 10000.0
[INSERT], AverageLatency(us), 2004.2133

[OVERALL], RunTime(ms), 32663.0
[OVERALL], Throughput(ops/sec), 306.1568135198849
[READ], Operations, 10000.0
[READ], AverageLatency(us), 1057.9065
{code}
{code}
put create times: 15
put discard times: 13
put create total bytes: 1441979
put discard tota bytes: 1282991
read create times: 0
read discard times: 0
read create total bytes: 0
read discard total bytes: 0
{code}
byteBufferPool test result:
{code}
[OVERALL], RunTime(ms), 43470.0
[OVERALL], Throughput(ops/sec), 230.04370830457788
[INSERT], Operations, 10000.0
[INSERT], AverageLatency(us), 1998.8565

[OVERALL], RunTime(ms), 35066.0
[OVERALL], Throughput(ops/sec), 285.1765242685222
[READ], Operations, 10000.0
[READ], AverageLatency(us), 1246.3949
{code}
{code}
put create times: 70
put discard times: 66
put create total bytes: 503152
put discard total bytes: 11632
read create times: 10058
read discard times: 10058
read create total bytes: 1344303
read discard total bytes: 1347255
{code}
Not too much difference between their throughput.But BoundedByteBufferPool do less creating
and discarding obviously.
YCSB properties:
{code}
recordcount=10000
operationcount=10000
workload=com.yahoo.ycsb.workloads.CoreWorkload
threads 25
readproportion=1.0
updateproportion=0
scanproportion=0
insertproportion=0
fieldcount=64
fieldlength=10240
readallfields=false
writeallfields=false
fieldlengthdistribution=zipfian
requestdistribution=zipfian
{code}


> [RpcServer] reuse request read buffer
> -------------------------------------
>
>                 Key: HBASE-14490
>                 URL: https://issues.apache.org/jira/browse/HBASE-14490
>             Project: HBase
>          Issue Type: Improvement
>          Components: IPC/RPC
>    Affects Versions: 2.0.0, 1.0.2
>            Reporter: Zephyr Guo
>            Assignee: Zephyr Guo
>              Labels: performance
>             Fix For: 2.0.0, 1.0.2
>
>         Attachments: ByteBufferPool.java, HBASE-14490-v1.patch, HBASE-14490-v10.patch,
HBASE-14490-v11.patch, HBASE-14490-v12.patch, HBASE-14490-v2.patch, HBASE-14490-v3.patch,
HBASE-14490-v4.patch, HBASE-14490-v5.patch, HBASE-14490-v6.patch, HBASE-14490-v7.patch, HBASE-14490-v8.patch,
HBASE-14490-v9.patch
>
>
> Reuse buffer to read request.It's not necessary to every request free buffer.The idea
of optimization is to reduce the times that allocate ByteBuffer.
> *Modification*
> 1. {{saslReadAndProcess}} ,{{processOneRpc}}, {{processUnwrappedData}}, {{processConnectionHeader}}
accept a ByteBuffer instead of byte[].They can move {{ByteBuffer.position}} correctly when
we have read the data.
> 2. {{processUnwrappedData}} no longer use any extra memory.
> 3. Maintaining a reused ByteBuffer in each {{Connection}}.
> a) Size is dynamic to fit specific client.
> b) If size deviate far from average size, we replace a new one.
> c) Using a SoftReference to reference the buffer.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message