hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Enis Söztutar <e...@apache.org>
Subject Re: Comparing the performance of 0.98.4 RC0 and 0.98.0 using YCSB - 23% perf regression in workload E
Date Fri, 18 Jul 2014 00:16:15 GMT
This looks indeed concerning. It seems that Workload E is 95% scan, and the
other workloads have no scan, so it seems that we have some regression in
scans.

Should this sink the RC, what do you think?

Enis


On Thu, Jul 17, 2014 at 5:08 PM, Andrew Purtell <apurtell@apache.org> wrote:

> Comparing the relative performance of 0.98.4 RC0 and 0.98.0 on Hadoop 2.2.0
> using YCSB.
>
> This will be the last report of these from me for a while, as I will be
> losing my current access to EC2 resources tomorrow.
>
> 5 concurrent YCSB clients on 5 servers target 100,000 ops/second in
> aggregate. Reported average values are averages of readings from all
> clients over 3 runs. Min values are the minimum reported by any client on
> any run. Max and percentile values are the maximum reported by any client
> on any run. What is interesting is relative differences, because each EC2
> testbed has a varying baseline. 0.98.0 and 0.98.4 tests were run on the
> same instance set.
>
> These tests were run with no security coprocessors installed, using HFile
> V2. The workload E results are a concern. *It appears we have a 23% decline
> in measured scan throughput and an 23% increase in average op time from 27
> ms to 35 ms. *This does not correspond to any active security feature
> (though that could worsen results potentially, untested) so is something
> changed in core code. Other workloads are not affected so this is something
> specific to scanning. Perhaps delete tracking.
>
>
> *Hardware and Versions*
>
>  Hadoop 2.2.0
>
> HBase 0.98.0-hadoop2 + HBASE-11277
>
> HBase 0.98.4-hadoop2 RC0
>
> YCSB 1.0.4
>
>
> 11x EC2 c3.8xlarge: 1 master, 5 slaves, 5 test clients
>
>     32 cores
>
>      60 GB RAM
>
>     2 x 320 GB directly attached SSD
>
>     NameNode: 4 GB heap
>
>     DataNode: 1 GB heap
>
>     Master: 1 GB heap
>
>     RegionServer: 8 GB heap, 24 GB bucket cache offheap engine
>
>
> *Methodology*
>
>
> Setup:
>
>      0. Start cluster
>      1. shell: create "seed", { NAME=>"u", COMPRESSION=>"snappy" }
>      2. YCSB: Preload 100 million rows into table "seed"
>      3. shell: flush "seed" ; compact "seed"
>      4. Wait for compaction to complete
>      5. shell: create_snapshot "seed", "seed_snap"
>      6. shell: disable "seed"
>
>
>  For each test:
>
>      7. shell: clone_snapshot "seed_snap", "test"
>      8. YCSB: On each client (5 clients), run test -p
> operationcount=2000000 -threads 20 -target 20000
>      9. shell: disable "test"
>     10. shell: drop "test"
>
> ​
>
>    *Workload A*
> *0.98.0* *0.98.4*
>
>
>
>  [OVERALL] RunTime(ms) 100743 100693  [OVERALL] Throughput(ops/sec) 99263
> 99312  [UPDATE] Operations 4997918 4999620  [UPDATE] AverageLatency(us) 633
> 647  [UPDATE] MinLatency(us) 269 268  [UPDATE] MaxLatency(us) 1450432
> 713191
> [UPDATE] 95thPercentileLatency(ms) 0 0  [UPDATE] 99thPercentileLatency(ms)
> 5
> 4  [READ] Operations 5002242 5000540  [READ] AverageLatency(us) 151 144
> [READ] MinLatency(us) 0 0  [READ] MaxLatency(us) 1104157 952392  [READ]
> 95thPercentileLatency(ms) 0 0  [READ] 99thPercentileLatency(ms) 0 0
>
>
>
>  *Workload B*
>
>
>
>
>
>
>  [OVERALL] RunTime(ms) 100465 100458  [OVERALL] Throughput(ops/sec) 99537
> 99544  [UPDATE] Operations 9499627 9499891  [UPDATE] AverageLatency(us) 556
> 589  [UPDATE] MinLatency(us) 268 264  [UPDATE] MaxLatency(us) 709604 695863
> [UPDATE] 95thPercentileLatency(ms) 0 0  [UPDATE] 99thPercentileLatency(ms)
> 1
> 2  [READ] Operations 500533 500269  [READ] AverageLatency(us) 147 144
> [READ] MinLatency(us) 0 0  [READ] MaxLatency(us) 571294 495148  [READ]
> 95thPercentileLatency(ms) 0 0  [READ] 99thPercentileLatency(ms) 0 0
>
>
>
>  *Workload C*
>
>
>
>
>
>
>  [OVERALL] RunTime(ms) 100091 100022  [OVERALL] Throughput(ops/sec) 99909
> 99978  [READ] Operations 9916831 10000000  [READ] AverageLatency(us) 524
> 526
> [READ] MinLatency(us) 273 269  [READ] MaxLatency(us) 737108 741634  [READ]
> 95thPercentileLatency(ms) 0 0  [READ] 99thPercentileLatency(ms) 1 2
>
>
>
>  *Workload D*
>
>
>
>
>
>
>  [OVERALL] RunTime(ms) 114244 103308  [OVERALL] Throughput(ops/sec) 89114
> 96809  [INSERT] Operations 9499965 9500306  [INSERT] AverageLatency(us)
> 1145
> 668  [INSERT] MinLatency(us) 270 271  [INSERT] MaxLatency(us) 4598999
> 3291540  [INSERT] 95thPercentileLatency(ms) 6 1  [INSERT]
> 99thPercentileLatency(ms) 13 3  [READ] Operations 500035 499694  [READ]
> AverageLatency(us) 14 15  [READ] MinLatency(us) 4 4  [READ] MaxLatency(us)
> 494730 495198  [READ] 95thPercentileLatency(ms) 0 0  [READ]
> 99thPercentileLatency(ms) 0 0
>
>
>
>  *Workload E*
>
>
>
>
>
>
>  [OVERALL] RunTime(ms) 1600910 2078826  [OVERALL] Throughput(ops/sec) 6308
> 4835  [INSERT] Operations 499131 500322  [INSERT] AverageLatency(us) 14 17
> [INSERT] MinLatency(us) 5 5  [INSERT] MaxLatency(us) 506079 564468
>  [INSERT]
> 95thPercentileLatency(ms) 0 0  [INSERT] 99thPercentileLatency(ms) 0 0
> [SCAN] Operations 9500869 9499678  [SCAN] AverageLatency(us)
> ​​
> ​​
> 26636 34620  [SCAN] MinLatency(us) 746 755  [SCAN] MaxLatency(us) 8067864
> 4615914  [SCAN] 95thPercentileLatency(ms) 117 136  [SCAN]
> 99thPercentileLatency(ms) 169 187
>
>
>
>  *Workload F*
>
>
>
>
>
>
>  [OVERALL] RunTime(ms) 100876 100820  [OVERALL] Throughput(ops/sec) 99133
> 99187  [UPDATE] Operations 10000000 10000000  [UPDATE] AverageLatency(us)
> 737 746  [UPDATE] MinLatency(us) 273 272  [UPDATE] MaxLatency(us) 759812
> 747124  [UPDATE] 95thPercentileLatency(ms) 1 1  [UPDATE]
> 99thPercentileLatency(ms) 5 6  [READ-MODIFY-WRITE] Operations 5000370
> 5000082  [READ-MODIFY-WRITE] AverageLatency(us) 742 750
>  [READ-MODIFY-WRITE]
> MinLatency(us) 280 279  [READ-MODIFY-WRITE] MaxLatency(us) 756180 747197
> [READ-MODIFY-WRITE] 95thPercentileLatency(ms) 1 1  [READ-MODIFY-WRITE]
> 99thPercentileLatency(ms) 5 6  [READ] Operations 5000530 5000242  [READ]
> AverageLatency(us) 22 17  [READ] MinLatency(us) 0 0  [READ] MaxLatency(us)
> 1551953 1097394  [READ] 95thPercentileLatency(ms) 0 0  [READ]
> 99thPercentileLatency(ms) 0 0
> ​
>
> ​​
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message