hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Segel <michael_se...@hotmail.com>
Subject Re: Comparsion between 0.98.0 and 0.98.1RC1 using YCSB
Date Tue, 25 Mar 2014 13:18:25 GMT
How many times did you run this test? (test suite)

On Mar 25, 2014, at 7:29 AM, Andrew Purtell <apurtell@apache.org> wrote:

> Thought I'd look at how 0.98.0 and 0.98.1 RC1 compare using YCSB. It could
> be .1 is just a hair slower than .0, but the results are similar. For some
> workloads either .0 or .1 could be slightly ahead run to run. Results from
> the most recent runs are presented below.
> 
> *Hardware and Versions*
> 
>  Hadoop 2.2.0
>  HBase 0.98.1-hadoop2 RC1
> 
>  5x EC2 cc2.8xlarge
>    32 cores
>    60 GB RAM
>    4 x 840 GB directly attached spinning media, as DN storage volumes
>    NameNode: 4 GB heap
>    DataNode: 1 GB heap
>    Master: 4 GB heap
>    RegionServer: 8 GB heap, 24 GB bucket cache offheap engine
> 
>  1x EC2 cc2.8xlarge
>    YCSB
> 
> *Configuration*
> 
>  Hadoop and HBase default site configuration, except:
>   - dfs.client.read.shortcircuit=true
>   - dfs.client.read.shortcircuit.skip.checksum=true
>   - dfs.client.domain.socket.data.traffic=true
>   - dfs.domain.socket.path=...
>   - dfs.datanode.max.xcievers = 16384
>   - dfs.namenode.handler.count = 100
>   - ipc.server.tcpnodelay=true
>   - hbase.ipc.client.tcpnodelay=true
>   - hbase.bucketcache.ioengine=offheap
>   - hbase.bucketcache.size=0.9
> 
> *Methodology*
> 
> Setup:
> 
>   0. Start cluster
>   1. shell: create "seed", { NAME=>"u", COMPRESSION=>"snappy" }
>   2. YCSB:  Preload 100 million rows into table "seed"
>   3. shell: compact "seed"
>   4. Wait for compaction to complete
>   5. shell: create_snapshot "seed", "seed_snap"
>   6. shell: disable "seed"
> 
>  For each test:
> 
>   7. shell: clone_snapshot "seed_snap", "test"
>   8. YCSB:  Run test -p operationcount=10000000 -threads 32 -target 50000
> (clamp at ~10k ops/server/sec)
>   9. shell: disable "test"
>  10. shell: drop "test"
> 
> *Workload A*
> 
> *  0.98.0*
> 
>  [OVERALL], RunTime(ms), 200313.0
>  [OVERALL], Throughput(ops/sec), 49921.87226989761
>  [READ], Operations, 5002419
>  [READ], AverageLatency(us), 1136.5890318264023
>  [READ], MinLatency(us), 281
>  [READ], MaxLatency(us), 1134224
>  [READ], 95thPercentileLatency(ms), 1
>  [READ], 99thPercentileLatency(ms), 2
>  [UPDATE], Operations, 4997613
>  [UPDATE], AverageLatency(us), 6.608387844356896
>  [UPDATE], MinLatency(us), 0
>  [UPDATE], MaxLatency(us), 724894
>  [UPDATE], 95thPercentileLatency(ms), 0
>  [UPDATE], 99thPercentileLatency(ms), 0
> 
> *  0.98.1*
> 
>  [OVERALL], RunTime(ms), 200323.0
>  [OVERALL], Throughput(ops/sec), 49919.38020097542
>  [READ], Operations, 5001366
>  [READ], AverageLatency(us), 1157.195526182247
>  [READ], MinLatency(us), 311
>  [READ], MaxLatency(us), 1116840
>  [READ], 95thPercentileLatency(ms), 1
>  [READ], 99thPercentileLatency(ms), 2
>  [UPDATE], Operations, 4998666
>  [UPDATE], AverageLatency(us), 6.185563908450774
>  [UPDATE], MinLatency(us), 0
>  [UPDATE], MaxLatency(us), 725483
>  [UPDATE], 95thPercentileLatency(ms), 0
>  [UPDATE], 99thPercentileLatency(ms), 0
> 
> *Workload B*
> 
> *  0.98.0*
> 
>  [OVERALL], RunTime(ms), 264508.0
>  [OVERALL], Throughput(ops/sec), 37806.039892933295
>  [READ], Operations, 9499452
>  [READ], AverageLatency(us), 880.201064545618
>  [READ], MinLatency(us), 290
>  [READ], MaxLatency(us), 1160947
>  [READ], 95thPercentileLatency(ms), 1
>  [READ], 99thPercentileLatency(ms), 1
>  [UPDATE], Operations, 500580
>  [UPDATE], AverageLatency(us), 12.54600063925846
>  [UPDATE], MinLatency(us), 1
>  [UPDATE], MaxLatency(us), 741148
>  [UPDATE], 95thPercentileLatency(ms), 0
>  [UPDATE], 99thPercentileLatency(ms), 0
> 
> *  0.98.1*
> 
>  [OVERALL], RunTime(ms), 265789.0
>  [OVERALL], Throughput(ops/sec), 37623.8294286069
>  [READ], Operations, 9499661
>  [READ], AverageLatency(us), 884.9811973290415
>  [READ], MinLatency(us), 273
>  [READ], MaxLatency(us), 1186243
>  [READ], 95thPercentileLatency(ms), 1
>  [READ], 99thPercentileLatency(ms), 1
>  [UPDATE], Operations, 500371
>  [UPDATE], AverageLatency(us), 16.71922433554303
>  [UPDATE], MinLatency(us), 1
>  [UPDATE], MaxLatency(us), 923659
>  [UPDATE], 95thPercentileLatency(ms), 0
>  [UPDATE], 99thPercentileLatency(ms), 0
> 
> *Workload C*
> 
> *  0.98.0*
> 
>  [OVERALL], RunTime(ms), 275967.0
>  [OVERALL], Throughput(ops/sec), 36236.2166490921
>  [READ], Operations, 10000000
>  [READ], AverageLatency(us), 875.3169807
>  [READ], MinLatency(us), 283
>  [READ], MaxLatency(us), 1123204
>  [READ], 95thPercentileLatency(ms), 1
>  [READ], 99thPercentileLatency(ms), 1
> 
> *  0.98.1*
> 
>  [OVERALL], RunTime(ms), 274182.0
>  [OVERALL], Throughput(ops/sec), 36472.124355355205
>  [READ], Operations, 10000000
>  [READ], AverageLatency(us), 868.9167299
>  [READ], MinLatency(us), 287
>  [READ], MaxLatency(us), 1082448
>  [READ], 95thPercentileLatency(ms), 1
>  [READ], 99thPercentileLatency(ms), 1
> 
> *Workload D*
> 
> *  0.98.0*
> 
>  [OVERALL], RunTime(ms), 430518.0
>  [OVERALL], Throughput(ops/sec), 23227.83251803641
>  [INSERT], Operations, 501258
>  [INSERT], AverageLatency(us), 18.74352529036943
>  [INSERT], MinLatency(us), 5
>  [INSERT], MaxLatency(us), 729485
>  [INSERT], 95thPercentileLatency(ms), 0
>  [INSERT], 99thPercentileLatency(ms), 0
>  [READ], Operations, 9498742
>  [READ], AverageLatency(us), 1435.8229302364462
>  [READ], MinLatency(us), 278
>  [READ], MaxLatency(us), 1086256
>  [READ], 95thPercentileLatency(ms), 3
>  [READ], 99thPercentileLatency(ms), 13
> 
> *  0.98.1*
> 
>  [OVERALL], RunTime(ms), 473086.0
>  [OVERALL], Throughput(ops/sec), 21137.805811205573
>  [INSERT], Operations, 500280
>  [INSERT], AverageLatency(us), 18.910943871431996
>  [INSERT], MinLatency(us), 5
>  [INSERT], MaxLatency(us), 741632
>  [INSERT], 95thPercentileLatency(ms), 0
>  [INSERT], 99thPercentileLatency(ms), 0
>  [READ], Operations, 9499720
>  [READ], AverageLatency(us), 1574.2442979372024
>  [READ], MinLatency(us), 285
>  [READ], MaxLatency(us), 1142568
>  [READ], 95thPercentileLatency(ms), 6
>  [READ], 99thPercentileLatency(ms), 15
> 
> *Workload E*
> 
> *  0.98.0*
> 
>  [OVERALL], RunTime(ms), 1741385.0
>  [OVERALL], Throughput(ops/sec), 5742.555494620661
>  [INSERT], Operations, 498580
>  [INSERT], AverageLatency(us), 12.785394520438045
>  [INSERT], MinLatency(us), 5
>  [INSERT], MaxLatency(us), 47371
>  [INSERT], 95thPercentileLatency(ms), 0
>  [INSERT], 99thPercentileLatency(ms), 0
>  [SCAN], Operations, 9501420
>  [SCAN], AverageLatency(us), 5832.021051169194
>  [SCAN], MinLatency(us), 850
>  [SCAN], MaxLatency(us), 18383187
>  [SCAN], 95thPercentileLatency(ms), 11
>  [SCAN], 99thPercentileLatency(ms), 22
> 
> *  0.98.1*
> 
>  [OVERALL], RunTime(ms), 1626498.0
>  [OVERALL], Throughput(ops/sec), 6148.178479161978
>  [INSERT], Operations, 498248
>  [INSERT], AverageLatency(us), 14.589842809203448
>  [INSERT], MinLatency(us), 5
>  [INSERT], MaxLatency(us), 726310
>  [INSERT], 95thPercentileLatency(ms), 0
>  [INSERT], 99thPercentileLatency(ms), 0
>  [SCAN], Operations, 9501752
>  [SCAN], AverageLatency(us), 5451.874070381968
>  [SCAN], MinLatency(us), 847
>  [SCAN], MaxLatency(us), 1278273
>  [SCAN], 95thPercentileLatency(ms), 12
>  [SCAN], 99thPercentileLatency(ms), 26
> 
> *Workload F*
> 
> *  0.98.0*
> 
>  [OVERALL], RunTime(ms), 376552.0
>  [OVERALL], Throughput(ops/sec), 26556.75710127685
>  [READ], Operations, 10000000
>  [READ], AverageLatency(us), 1192.262719
>  [READ], MinLatency(us), 293
>  [READ], MaxLatency(us), 1108215
>  [READ], 95thPercentileLatency(ms), 1
>  [READ], 99thPercentileLatency(ms), 2
>  [READ-MODIFY-WRITE], Operations, 4999036
>  [READ-MODIFY-WRITE], AverageLatency(us), 1198.5333812359022
>  [READ-MODIFY-WRITE], MinLatency(us), 299
>  [READ-MODIFY-WRITE], MaxLatency(us), 1070294
>  [READ-MODIFY-WRITE], 95thPercentileLatency(ms), 1
>  [READ-MODIFY-WRITE], 99thPercentileLatency(ms), 2
>  [READ-MODIFY-WRITE], 0, 2018024
>  [UPDATE], Operations, 4999068
>  [UPDATE], AverageLatency(us), 3.940626532785711
>  [UPDATE], MinLatency(us), 1
>  [UPDATE], MaxLatency(us), 155147
>  [UPDATE], 95thPercentileLatency(ms), 0
>  [UPDATE], 99thPercentileLatency(ms), 0
> 
> *  0.98.1*
> 
>  [OVERALL], RunTime(ms), 385273.0
>  [OVERALL], Throughput(ops/sec), 25955.621079078992
>  [READ], Operations, 10000000
>  [READ], AverageLatency(us), 1218.9608977
>  [READ], MinLatency(us), 296
>  [READ], MaxLatency(us), 1128785
>  [READ], 95thPercentileLatency(ms), 1
>  [READ], 99thPercentileLatency(ms), 3
>  [READ-MODIFY-WRITE], Operations, 5001281
>  [READ-MODIFY-WRITE], AverageLatency(us), 1226.2466698031965
>  [READ-MODIFY-WRITE], MinLatency(us), 300
>  [READ-MODIFY-WRITE], MaxLatency(us), 1129077
>  [READ-MODIFY-WRITE], 95thPercentileLatency(ms), 1
>  [READ-MODIFY-WRITE], 99thPercentileLatency(ms), 3
>  [UPDATE], Operations, 5001313
>  [UPDATE], AverageLatency(us), 4.26973976633736
>  [UPDATE], MinLatency(us), 1
>  [UPDATE], MaxLatency(us), 120701
>  [UPDATE], 95thPercentileLatency(ms), 0
>  [UPDATE], 99thPercentileLatency(ms), 0
> 
> 
> -- 
> Best regards,
> 
>   - Andy
> 
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)


Mime
View raw message