hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Comparing the performance of 0.98.8 RC1 and 0.98.0 using YCSB
Date Sat, 15 Nov 2014 04:36:31 GMT
Makes sense. 

It is nice to see improvements add up. 

On Nov 14, 2014, at 8:31 PM, Andrew Purtell <andrew.purtell@gmail.com> wrote:

> Ie the color is blue because for 0.98.0 there is 5.88% more time spent there than in
0.98.8. In this case of Bytes#toShort this is due I think to Vladimir's optimizations using
Unsafe we added a few releases back. 
> 
> 
> 
>> On Nov 14, 2014, at 8:27 PM, Andrew Purtell <andrew.purtell@gmail.com> wrote:
>> 
>> No it means the opposite of that. I discuss this in the email. The differential graph
looks at 0.98.0 and shows what _will_ happen in 0.98.8 (if I understand Gregg's blog post
correctly)
>> 
>> 
>> 
>>> On Nov 14, 2014, at 7:56 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>>> 
>>> Andrew:
>>> Looking at http://people.apache.org/~apurtell/results-ycsb-0.98.8/f_diff.svg
>>> , Bytes#toShort shows up with +5.88%
>>> Does this mean that Bytes#toShort in 0.98.8 RC1 has higher impact on
>>> performance compared to that of 0.98.0 ?
>>> 
>>> Thanks
>>> 
>>>> On Fri, Nov 14, 2014 at 7:43 PM, Andrew Purtell <apurtell@apache.org>
wrote:
>>>> 
>>>> Comparing the performance of 0.98.8 RC1 and 0.98.0 using YCSB from
>>>> https://github.com/apurtell/ycsb/tree/new_hbase_client.
>>>> 
>>>> In summary, there are minor differences, generally improvements, and this
>>>> test didn't detect performance regression when looking at operation
>>>> latencies. Note 0.98.8RC2 only has HBASE-11698 and HBASE-12478 in addition
>>>> to earlier changes, neither of which will alter these results.
>>>> 
>>>> Hardware and Versions
>>>> 
>>>>  Hadoop 2.4.1
>>>>  HBase 0.98.8-hadoop2 RC1
>>>> 
>>>>  11x EC2 c3.8xlarge: 1 master, 5 slaves, 5 test clients
>>>>      32 cores
>>>>      60 GB RAM
>>>>      2 x 320 GB directly attached SSD
>>>>      NameNode: 4 GB heap
>>>>      DataNode: 1 GB heap
>>>>      Master: 1 GB heap
>>>>      RegionServer: 8 GB heap, 24 GB bucket cache offheap engine
>>>> 
>>>> Methodology
>>>> 
>>>>  Setup:
>>>> 
>>>>       0. Start cluster
>>>>       1. shell: create "seed", { NAME=>"u", COMPRESSION=>"snappy" }
>>>>       2. YCSB:  Preload 100 million rows into table "seed"
>>>>       3. shell: flush "seed" ; compact "seed"
>>>>       4. Wait for compaction to complete
>>>>       5. shell: create_snapshot "seed", "seed_snap"
>>>>       6. shell: disable "seed"
>>>> 
>>>>   For each test:
>>>> 
>>>>       7. shell: clone_snapshot "seed_snap", "test"
>>>>       8. YCSB:  Run 5x clients each with -p operationcount=4000000
>>>> -threads
>>>> 100 -target 20000 (aggregate 100k/ops/sec)
>>>>       9. shell: disable "test"
>>>>     10. shell: drop "test"
>>>> 
>>>> ​I then ran one more workload for each version with the
>>>> lightweight-java-profiler agent installed on one regionserver and generated
>>>> flame graphs from the resulting traces, including the neat new differential
>>>> type. When looking at the differential flame graphs, you'll note the deltas
>>>> are positive. This is the difference between time sampled in the relevant
>>>> call stack on 0.98.0 versus 0.98.8, a positive number means _less_ time was
>>>> spent by 0.98.8 there than 0.98.0. It's also likely that testing with
>>>> higher loads (this test was only with 100k/ops/sec over a 5 slave cluster)
>>>> will present more significant differences. For more information, see
>>>> http://www.brendangregg.com/blog/2014-11-09/differential-flame-graphs.html
>>>> 
>>>> ​Results
>>>> 
>>>>  ​YCSB summary:
>>>> 
>>>> http://people.apache.org/~apurtell/results-ycsb-0.98.8/ycsb-0.98.0-vs-0.98.8.pdf
>>>> 
>>>>  Flame Graphs
>>>>      0.98.0 A:
>>>> http://people.apache.org/~apurtell/results-ycsb-0.98.8/a_0.98.0.svg
>>>>      0.98.8 A:
>>>> http://people.apache.org/~apurtell/results-ycsb-0.98.8/a_0.98.8.svg
>>>>      Difference:
>>>> http://people.apache.org/~apurtell/results-ycsb-0.98.8/a_diff.svg
>>>> 
>>>>      0.98.0 B:
>>>> http://people.apache.org/~apurtell/results-ycsb-0.98.8/b_0.98.0.svg
>>>>      0.98.8 B:
>>>> http://people.apache.org/~apurtell/results-ycsb-0.98.8/b_0.98.8.svg
>>>>      Difference:
>>>> http://people.apache.org/~apurtell/results-ycsb-0.98.8/b_diff.svg
>>>> 
>>>>      0.98.0 C:
>>>> http://people.apache.org/~apurtell/results-ycsb-0.98.8/c_0.98.0.svg
>>>>      0.98.8 C:
>>>> http://people.apache.org/~apurtell/results-ycsb-0.98.8/c_0.98.8.svg
>>>>      Difference:
>>>> http://people.apache.org/~apurtell/results-ycsb-0.98.8/c_diff.svg
>>>> 
>>>>      0.98.0 D:
>>>> http://people.apache.org/~apurtell/results-ycsb-0.98.8/d_0.98.0.svg
>>>>      0.98.8 D:
>>>> http://people.apache.org/~apurtell/results-ycsb-0.98.8/d_0.98.8.svg
>>>>      Difference:
>>>> http://people.apache.org/~apurtell/results-ycsb-0.98.8/d_diff.svg
>>>> 
>>>>      0.98.0 E:
>>>> http://people.apache.org/~apurtell/results-ycsb-0.98.8/e_0.98.0.svg
>>>>      0.98.8 E:
>>>> http://people.apache.org/~apurtell/results-ycsb-0.98.8/e_0.98.8.svg
>>>>      Difference:
>>>> http://people.apache.org/~apurtell/results-ycsb-0.98.8/e_diff.svg
>>>> 
>>>>      0.98.0 F:
>>>> http://people.apache.org/~apurtell/results-ycsb-0.98.8/f_0.98.0.svg
>>>>      0.98.8 F:
>>>> http://people.apache.org/~apurtell/results-ycsb-0.98.8/f_0.98.8.svg
>>>>      Difference:
>>>> http://people.apache.org/~apurtell/results-ycsb-0.98.8/f_diff.svg
>>>> 
>>>> 
>>>> --
>>>> Best regards,
>>>> 
>>>> - Andy
>>>> 
>>>> Problems worthy of attack prove their worth by hitting back. - Piet Hein
>>>> (via Tom White)
>>>> 

Mime
View raw message