hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Konstantin Boudnik <...@apache.org>
Subject Re: Comparing the performance of 0.98.2 RC0 and 0.98.0 using YCSB
Date Sun, 11 May 2014 04:29:00 GMT
I think the point wasn't a benchmarking but merely to make sure there's no
regressions.

On Wed, May 07, 2014 at 02:16PM, Vladimir Rodionov wrote:
> *7x EC2 c3.8xlarge: 1 master, 5 slaves, 1 test client*
> 
> Andrew, I think these numbers are far from maximum you can get from this
> set up. Why only 1 test client?
> 
> -Vladimir Rodionov
> 
> 
> On Tue, May 6, 2014 at 6:58 PM, Andrew Purtell <apurtell@apache.org> wrote:
> 
> > Comparing the relative performance of 0.98.2 RC0 and 0.98.0 on Hadoop 2.2.0
> > using YCSB.
> >
> > The hardware used is different than for the previous report comparing
> > 0.98.1 to 0.98.0. However the results are very similar, both in terms of
> > 0.98.2 RC0 numbers with respect to those measured for 0.98.0, and the
> > workload specific deltas observed when testing 0.98.1.
> >
> > *Hardware and Versions*
> >
> > Hadoop 2.2.0
> > HBase 0.98.2-hadoop2 RC0
> >
> > 7x EC2 c3.8xlarge: 1 master, 5 slaves, 1 test client
> >
> >     32 cores
> >
> >     60 GB RAM
> >
> >     2 x 320 GB directly attached SSD
> >
> >     NameNode: 4 GB heap
> >
> >     DataNode: 1 GB heap
> >
> >     Master: 1 GB heap
> >
> >     RegionServer: 8 GB heap, 24 GB bucket cache offheap engine
> >
> >
> > *Methodology*
> >
> >
> > Setup:
> >
> >      0. Start cluster
> >      1. shell: create "seed", { NAME=>"u", COMPRESSION=>"snappy" }
> >      2. YCSB:  Preload 100 million rows into table "seed"
> >       3. shell: flush "seed" ; compact "seed"
> >      4. Wait for compaction to complete
> >      5. shell: create_snapshot "seed", "seed_snap"
> >       6. shell: disable "seed"
> >
> >
> >  For each test:
> >
> >      7. shell: clone_snapshot "seed_snap", "test"
> >       8. YCSB:  Run test -p operationcount=10000000 -threads 32 -target
> > 50000 (clamp at ~10k ops/server/sec)
> >       9. shell: disable "test"
> >     10. shell: drop "test"
> >
> >
> > *Workload A*
> >
> >  *0.98.0*
> >
> >  [OVERALL], RunTime(ms), 2097825
> > [OVERALL], Throughput(ops/sec), 4767
> > [UPDATE], Operations, 4999049
> > [UPDATE], AverageLatency(us), 1.107036384
> > [UPDATE], MinLatency(us), 0
> > [UPDATE], MaxLatency(us), 97865
> > [UPDATE], 95thPercentileLatency(ms), 0
> > [UPDATE], 99thPercentileLatency(ms), 0
> > [READ], Operations, 5000952
> > [READ], AverageLatency(us), 413.9172277
> > [READ], MinLatency(us), 295
> > [READ], MaxLatency(us), 927729
> > [READ], 95thPercentileLatency(ms), 0
> > [READ], 99thPercentileLatency(ms), 0
> >
> >
> >  *0.98.2*
> >
> > ​[OVERALL], RunTime(ms), 2082682
> > [OVERALL], Throughput(ops/sec), 4802
> > [UPDATE], Operations, 5001208
> > [UPDATE], AverageLatency(us), 1.227632714
> > [UPDATE], MinLatency(us), 0
> > [UPDATE], MaxLatency(us), 720423
> > [UPDATE], 95thPercentileLatency(ms), 0
> > [UPDATE], 99thPercentileLatency(ms), 0
> > [READ], Operations, 4998792.667
> > [READ], AverageLatency(us), 411.0522393
> > [READ], MinLatency(us), 288
> > [READ], MaxLatency(us), 977500
> > [READ], 95thPercentileLatency(ms), 0
> > [READ], 99thPercentileLatency(ms), 0
> >
> > ​
> >
> > ​*Workload B*
> >
> >  *0.98.0*
> >
> > [OVERALL], RunTime(ms), 3678408
> > [OVERALL], Throughput(ops/sec), 2719
> > [UPDATE], Operations, 500239
> > [UPDATE], AverageLatency(us), 2.218397098
> > [UPDATE], MinLatency(us), 0
> > [UPDATE], MaxLatency(us), 101523
> > [UPDATE], 95thPercentileLatency(ms), 0
> > [UPDATE], 99thPercentileLatency(ms), 0
> > [READ], Operations, 9499762.333
> > [READ], AverageLatency(us), 384.8231468
> > [READ], MinLatency(us), 283
> > [READ], MaxLatency(us), 922395
> > [READ], 95thPercentileLatency(ms), 0
> > [READ], 99thPercentileLatency(ms), 0
> >
> >  *0.98.2*
> >
> > ​[OVERALL], RunTime(ms), 3643856
> > [OVERALL], Throughput(ops/sec), 2744
> > [UPDATE], Operations, 499256
> > [UPDATE], AverageLatency(us), 2.561636579
> > [UPDATE], MinLatency(us), 0
> > [UPDATE], MaxLatency(us), 713811
> > [UPDATE], 95thPercentileLatency(ms), 0
> > [UPDATE], 99thPercentileLatency(ms), 0
> > [READ], Operations, 9500745
> > [READ], AverageLatency(us), 381.1349225
> > [READ], MinLatency(us), 284
> > [READ], MaxLatency(us), 921680
> > [READ], 95thPercentileLatency(ms), 0
> > [READ], 99thPercentileLatency(ms), 0
> >
> > ​
> > *Workload C*
> >
> >  *0.98.0*
> >
> > [OVERALL], RunTime(ms), 3258845
> > [OVERALL], Throughput(ops/sec), 3069
> > [READ], Operations, 10000000
> > [READ], AverageLatency(us), 323.7287128
> > [READ], MinLatency(us), 276
> > [READ], MaxLatency(us), 928472
> > [READ], 95thPercentileLatency(ms), 0
> > [READ], 99thPercentileLatency(ms), 0
> >
> >  *0.98.2*
> >
> >  [OVERALL], RunTime(ms), 3288822
> > [OVERALL], Throughput(ops/sec), 3041
> > [READ], Operations, 10000000
> > [READ], AverageLatency(us), 326.6214268
> > [READ], MinLatency(us), 284
> > [READ], MaxLatency(us), 924632
> > [READ], 95thPercentileLatency(ms), 0
> > [READ], 99thPercentileLatency(ms), 0
> >
> > ​*Workload D*
> >
> >  *0.98.0*
> >
> >  [OVERALL], RunTime(ms), 3707601
> > [OVERALL], Throughput(ops/sec), 2700
> > [INSERT], Operations, 500774
> > [INSERT], AverageLatency(us), 6.432826519
> > [INSERT], MinLatency(us), 4
> > [INSERT], MaxLatency(us), 40274
> > [INSERT], 95thPercentileLatency(ms), 0
> > [INSERT], 99thPercentileLatency(ms), 0
> > [READ], Operations, 9499225.667
> > [READ], AverageLatency(us), 387.7104498
> > [READ], MinLatency(us), 283
> > [READ], MaxLatency(us), 927377
> > [READ], 95thPercentileLatency(ms), 0
> > [READ], 99thPercentileLatency(ms), 1
> >
> > *0.98.2*
> >
> > [OVERALL], RunTime(ms), 3650724
> > [OVERALL], Throughput(ops/sec), 2740
> > [INSERT], Operations, 499872
> > [INSERT], AverageLatency(us), 6.46417158
> > [INSERT], MinLatency(us), 4
> > [INSERT], MaxLatency(us), 47732
> > [INSERT], 95thPercentileLatency(ms), 0
> > [INSERT], 99thPercentileLatency(ms), 0
> > [READ], Operations, 9500128
> > [READ], AverageLatency(us), 381.6517188
> > [READ], MinLatency(us), 278
> > [READ], MaxLatency(us), 922107
> > [READ], 95thPercentileLatency(ms), 0
> > [READ], 99thPercentileLatency(ms), 1
> >
> > ​
> > ​*Workload E*
> >
> >  *0.98.0*
> >
> >  [OVERALL], RunTime(ms), 15717450.00
> > [OVERALL], Throughput(ops/sec), 636.2355217
> > [INSERT], Operations, 499943
> > [INSERT], AverageLatency(us), 12.57311534
> > [INSERT], MinLatency(us), 6
> > [INSERT], MaxLatency(us), 39539
> > [INSERT], 95thPercentileLatency(ms), 0
> > [INSERT], 99thPercentileLatency(ms), 0
> > [SCAN], Operations, 9500057
> > [SCAN], AverageLatency(us), 1648.836612
> > [SCAN], MinLatency(us), 768
> > [SCAN], MaxLatency(us), 1001461
> > [SCAN], 95thPercentileLatency(ms), 3
> > [SCAN], 99thPercentileLatency(ms), 5
> >
> > *0.98.2*
> >
> > ​[OVERALL], RunTime(ms), 15624378
> > [OVERALL], Throughput(ops/sec), 640
> > [INSERT], Operations, 499679
> > [INSERT], AverageLatency(us), 11.62045822
> > [INSERT], MinLatency(us), 5
> > [INSERT], MaxLatency(us), 40475
> > [INSERT], 95thPercentileLatency(ms), 0
> > [INSERT], 99thPercentileLatency(ms), 0
> > [SCAN], Operations, 9500321
> > [SCAN], AverageLatency(us), 1639.114033
> > [SCAN], MinLatency(us), 753
> > [SCAN], MaxLatency(us), 942908
> > [SCAN], 95thPercentileLatency(ms), 3
> > [SCAN], 99thPercentileLatency(ms), 5
> >
> > ​
> > *Workload F*
> >
> >  *0.98.0*
> >
> >  [OVERALL], RunTime(ms), 4144245
> > [OVERALL], Throughput(ops/sec), 2413
> > [UPDATE], Operations, 4999220
> > [UPDATE], AverageLatency(us), 1.520920945
> > [UPDATE], MinLatency(us), 0
> > [UPDATE], MaxLatency(us), 94874
> > [UPDATE], 95thPercentileLatency(ms), 0
> > [UPDATE], 99thPercentileLatency(ms), 0
> > [READ-MODIFY-WRITE], Operations, 4999219
> > [READ-MODIFY-WRITE], AverageLatency(us), 413.7597696
> > [READ-MODIFY-WRITE], MinLatency(us), 0
> > [READ-MODIFY-WRITE], MaxLatency(us), 107201
> > [READ-MODIFY-WRITE], 95thPercentileLatency(ms), 0
> > [READ-MODIFY-WRITE], 99thPercentileLatency(ms), 0
> > [READ], Operations, 10000000
> > [READ], AverageLatency(us), 410.3120381
> > [READ], MinLatency(us), 1
> > [READ], MaxLatency(us), 921156
> > [READ], 95thPercentileLatency(ms), 0
> > [READ], 99thPercentileLatency(ms), 0
> >
> > *0.98.2*
> >
> > [OVERALL], RunTime(ms), 4169551
> > [OVERALL], Throughput(ops/sec), 2398
> > [UPDATE], Operations, 4999046
> > [UPDATE], AverageLatency(us), 1.524197967
> > [UPDATE], MinLatency(us), 0
> > [UPDATE], MaxLatency(us), 103630
> > [UPDATE], 95thPercentileLatency(ms), 0
> > [UPDATE], 99thPercentileLatency(ms), 0
> > [READ-MODIFY-WRITE], Operations, 4999045
> > [READ-MODIFY-WRITE], AverageLatency(us), 416.3519861
> > [READ-MODIFY-WRITE], MinLatency(us), 0
> > [READ-MODIFY-WRITE], MaxLatency(us), 926671
> > [READ-MODIFY-WRITE], 95thPercentileLatency(ms), 0
> > [READ-MODIFY-WRITE], 99thPercentileLatency(ms), 0
> > [READ], Operations, 10000000
> > [READ], AverageLatency(us), 412.8674851
> > [READ], MinLatency(us), 1
> > [READ], MaxLatency(us), 928123
> > [READ], 95thPercentileLatency(ms), 0
> > [READ], 99thPercentileLatency(ms), 0
> >
> >
> >
> > --
> > Best regards,
> >
> >    - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > (via Tom White)
> >

Mime
View raw message