hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "binlijin (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HBASE-15756) Pluggable RpcServer
Date Tue, 13 Dec 2016 02:15:58 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15743861#comment-15743861
] 

binlijin edited comment on HBASE-15756 at 12/13/16 2:15 AM:
------------------------------------------------------------

And i also test the Netty4RpcServer perf on master version, the Netty4RpcServer can get at
 https://github.com/binlijin/hbase/tree/NettyRpcServer.
1 regionserver, 1 client machine, client and rs on different machine.
Write a table with 10M rows, then split into 32 regions, every row have only one cell, keyLength=10B,
valueLength=256B.
Only test random read, and all data cache in LruBlockCache.
Client: YCSB + hbase-1.1.2 client, start 16 YCSB process on client machine, and every process
have 32 threads.

share connections:  use YCSB build with https://github.com/brianfrankcooper/YCSB/commit/57e1ab5a0cae13d1766c8fa6bdbf9d9117ee50d0
No share connections: use YCSB build without https://github.com/brianfrankcooper/YCSB/commit/57e1ab5a0cae13d1766c8fa6bdbf9d9117ee50d0

The result is: 
(Master + Netty)    +  ( RpcClientImpl   + No share connections ) = 500K
(Master + SingleResponder)     +  ( RpcClientImpl   + No share connections ) = 500K

(Master + Netty)    +  ( RpcClientImpl   + share connections ) = 500K
(Master + MultiResponder)    +  ( RpcClientImpl   + share connections ) = 500K
(Master + SingleResponder)     +  ( RpcClientImpl   + share connections ) = 295K

(Master + MultiResponder)  +   ( AsyncRpcClient  + share connections ) = 570K   (Netty client)
(Master + Netty)  +  (AsyncRpcClient  + share connections ) =  620K  (Netty client)

MultiResponder:  see patch MultiResponder.master.patch



was (Author: aoxiang):
And i also test the Netty4RpcServer perf on master version, the Netty4RpcServer can get at
 https://github.com/binlijin/hbase/tree/NettyRpcServer.
1 regionserver, 1 client machine, client and rs on different machine.
Write a table with 10M rows, then split into 32 regions, every row have only one cell, keyLength=10B,
valueLength=256B.
Only test random read, and all data cache in LruBlockCache.
Client: YCSB + hbase-1.1.2 client, start 16 YCSB process on client machine, and every process
have 32 threads.

share connections:  use YCSB build with https://github.com/brianfrankcooper/YCSB/commit/57e1ab5a0cae13d1766c8fa6bdbf9d9117ee50d0
No share connections: use YCSB build without https://github.com/brianfrankcooper/YCSB/commit/57e1ab5a0cae13d1766c8fa6bdbf9d9117ee50d0

The result is: 
(Master + Netty)    +  ( RpcClientImpl   + No share connections ) = 500K
(Master + SingleResponder)     +  ( RpcClientImpl   + No share connections ) = 500K

(Master + Netty)    +  ( RpcClientImpl   + share connections ) = 500K
(Master + MultiResponder)    +  ( RpcClientImpl   + share connections ) = 500K
(Master + SingleResponder)     +  ( RpcClientImpl   + share connections ) = 295K

(Master + MultiResponder)  +   ( AsyncRpcClient  + share connections ) = 570K   (Netty client)
(Master + Netty)  +  (AsyncRpcClient  + share connections ) =  620K  (Netty client)

MultiResponder:  see patch MultiResponder.master.patch/MultiResponder.branch-1.patch


> Pluggable RpcServer
> -------------------
>
>                 Key: HBASE-15756
>                 URL: https://issues.apache.org/jira/browse/HBASE-15756
>             Project: HBase
>          Issue Type: Improvement
>          Components: Performance, rpc
>            Reporter: binlijin
>            Assignee: binlijin
>            Priority: Critical
>             Fix For: 2.0.0
>
>         Attachments: Cluster_total_QPS.png, MultiResponder.branch-1.patch, MultiResponder.master.patch,
Netty4RpcServer_forperf.patch, NettyRpcServer.patch, NettyRpcServer_forperf.patch, PooledByteBufAllocator.patch,
PooledByteBufAllocator2.patch, gc.png, gets.png, gets.png, idle.png, patched.vs.patched_and_cached.vs.no_patch.png,
queue.png
>
>
> Current we use a simple RpcServer, and can not configure and use other implementation.This
issue is to make the RpcServer pluggable, so we can make other implementation for example
netty rpc server. Patch will upload laterly



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message