hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Min Zhou <coderp...@gmail.com>
Subject Re: Questions on HADOOP-6713
Date Mon, 20 Dec 2010 01:27:57 GMT
Anybody can help me with this?

On Fri, Dec 17, 2010 at 12:45 PM, Min Zhou <coderplay@gmail.com> wrote:
> Hi all,
>
> After apply the patch from HADOOP-6713 into our hadoop source tree,  I
> did a benchmark on multi-readers RPC, and didn't found any  throughput
> improvement .
> The attachment of this mail is a patch of my rpc benmark. Test
> scenario listed below.
>
> Network: Gigabit LAN
> CPU: 8 core Intel(R) Xeon(R) CPU           E5420  @ 2.50GHz
> Memory: 8GB
>
> ================== scenario 1 =======================
> server side:
> # export HADOOP_OPS="-Xmx4096m -Xms256m"
> # hadoop/bin/hadoop jar hadoop/hadoop-0.19.1-test.jar rpcbench -server
> -handlers 1 -readers 10 -durationPerRPC 100
>
> client side:
> # hadoop/bin/hadoop jar hadoop/hadoop-0.19.1-test.jar rpcbench -client
> -host hdp38 -p 38668 -req 100 -threads 10 -size 8 -arraySize 200000
> 10/12/17 12:25:07 INFO ipc.RPCBenchmark: Starting 100 rpc call(s).
> 10/12/17 12:26:02 INFO ipc.RPCBenchmark: # operations: 100
> 10/12/17 12:26:02 INFO ipc.RPCBenchmark: Elapsed Time: 55073 ms
> 10/12/17 12:26:02 INFO ipc.RPCBenchmark:  Ops per sec: 1.815771793800955
> 10/12/17 12:26:02 INFO ipc.RPCBenchmark: Average Time: 5249 ms
>
> ================== scenario 2 =======================
> server side:
> # export HADOOP_OPS="-Xmx4096m -Xms256m"
> # hadoop/bin/hadoop jar hadoop/hadoop-0.19.1-test.jar rpcbench -server
> -handlers 10 -readers 10 -durationPerRPC 100
>
> client side:
> # hadoop/bin/hadoop jar hadoop/hadoop-0.19.1-test.jar rpcbench -client
> -host hdp38 -p 38668 -req 100 -threads 10 -size 8 -arraySize 200000
> 10/12/17 12:28:23 INFO ipc.RPCBenchmark: Starting 100 rpc call(s).
> 10/12/17 12:29:11 INFO ipc.RPCBenchmark: # operations: 100
> 10/12/17 12:29:11 INFO ipc.RPCBenchmark: Elapsed Time: 47784 ms
> 10/12/17 12:29:11 INFO ipc.RPCBenchmark:  Ops per sec: 2.092750711535242
> 10/12/17 12:29:11 INFO ipc.RPCBenchmark: Average Time: 4581 ms
>
> ================== scenario 3 =======================
> server side:
> # export HADOOP_OPS="-Xmx4096m -Xms256m"
> # hadoop/bin/hadoop jar hadoop/hadoop-0.19.1-test.jar rpcbench -server
> -handlers 10 -readers 1 -durationPerRPC 100
>
> client side:
> # hadoop/bin/hadoop jar hadoop/hadoop-0.19.1-test.jar rpcbench -client
> -host hdp38 -p 38668 -req 100 -threads 10 -size 8 -arraySize 200000
> 10/12/17 12:31:19 INFO ipc.RPCBenchmark: Starting 100 rpc call(s).
> 10/12/17 12:32:08 INFO ipc.RPCBenchmark: # operations: 100
> 10/12/17 12:32:08 INFO ipc.RPCBenchmark: Elapsed Time: 49155 ms
> 10/12/17 12:32:08 INFO ipc.RPCBenchmark:  Ops per sec: 2.034381039568711
> 10/12/17 12:32:08 INFO ipc.RPCBenchmark: Average Time: 4722 ms
>
> ================== scenario 4 =======================
> server side:
> # export HADOOP_OPS="-Xmx4096m -Xms256m
> # hadoop/bin/hadoop jar hadoop/hadoop-0.19.1-test.jar rpcbench -server
> -handlers 1 -readers 1 -durationPerRPC 100
>
> client side:
> # hadoop/bin/hadoop jar hadoop/hadoop-0.19.1-test.jar rpcbench -client
> -host hdp38 -p 38668 -req 100 -threads 10 -size 8 -arraySize 200000
> 10/12/17 12:36:04 INFO ipc.RPCBenchmark: Starting 100 rpc call(s).
> 10/12/17 12:36:59 INFO ipc.RPCBenchmark: # operations: 100
> 10/12/17 12:36:59 INFO ipc.RPCBenchmark: Elapsed Time: 55054 ms
> 10/12/17 12:36:59 INFO ipc.RPCBenchmark:  Ops per sec: 1.816398445162931
> 10/12/17 12:36:59 INFO ipc.RPCBenchmark: Average Time: 5278 ms
>
> Statistics above indicates that a single reader can consume 200,000
> 8-bytes Strings very fast.  Ten concurrent readers should consume the
> same data more quickly, but reader isn't the bottleneck of RPC at
> least in these scenarios. Can you give an example which scenario
> HADOOP-6713 will take affect?
>
> Thanks,
> Min
> --
> My research interests are distributed systems, parallel computing and
> bytecode based virtual machine.
>
> My profile:
> http://www.linkedin.com/in/coderplay
> My blog:
> http://coderplay.javaeye.com
>



-- 
My research interests are distributed systems, parallel computing and
bytecode based virtual machine.

My profile:
http://www.linkedin.com/in/coderplay
My blog:
http://coderplay.javaeye.com

Mime
View raw message