hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon" <edwardy...@apache.org>
Subject Re: PageRank Benchmark Test (0.6.4 vs 0.7.0-SNAPSHOT)
Date Thu, 25 Sep 2014 07:43:54 GMT
Yes, I saw it. I personally want to use the OffHeapVerticesInfo
implementation as a default, and clean up all implementations which
doesn't support dynamic graph operations.

On Thu, Sep 25, 2014 at 4:00 PM, Tommaso Teofili
<tommaso.teofili@gmail.com> wrote:
> 2014-09-24 11:16 GMT+02:00 Edward J. Yoon <edwardyoon@apache.org>:
>
>> Thanks for your benchmarks. BTW, I guess I need to explain more about this.
>>
>> As I described HAMA-919, we have to manage the outgoing messages per
>> vertex (like giraph), and won't use sorted queue for graph job
>> anymore. This is the way to improve the performance and memory
>> efficiency. With this, I think VerticesInfo also can be changed. So
>> now, it's a meaningless testing. I kindly ask you to benchmark again
>> after HAMA-919 is done.
>>
>> However, one thing is clear that Kryo serializer is really useful. :-)
>>
>
> by the way, the OffHeapVerticesInfo (based on Apache DirectMemory) uses
> Kryo for serialization :-)
>
> Tommaso
>
>
>>
>> On Tue, Sep 23, 2014 at 2:32 PM, Victor Lee <vvictor.lee@gmail.com> wrote:
>> > Hello, All ~
>> >
>> > I didn't send attach file. so I send mail again.
>> >
>> > thanks.
>> >
>> >
>> >
>> >
>> > [ Apache Hama Benchmark Test ]
>> > . PageRank Benchmark Test 0.6.4 vs 0.7.0-SNAPSHOT (2014/09/20)
>> > . Environments
>> > - 12 core (2.66Ghz 6core * 2)
>> > - 32G Memory (4G * 8)
>> > - 2 Node Hama Cluster
>> > - 6 tasks (opt : -Xmx1024m, -Xms2048)
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > . Memory : -Xmx1024m
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > vertices Edges
>> > per vertex 0.6.4 0.7.0-SNAPSHOT(2014/09/20) Improvement
>> > rate(%)
>> > 1th 2nd 3rd average 1th 2nd 3rd Average
>> > (seconds)
>> > 40000 500 178.822 178.688 183.85 180.453 147.711 142.232 132.755 140.899
>> > -21.92
>> > 40000 600 231.884 231.964 226.112 229.987 177.764 174.733 171.714 174.737
>> > -24.02
>> > 40000 700 277.061 276.845 273.928 275.945 219.796 222.792 216.817 219.802
>> > -20.35
>> > 40000 800 Job failed. Job failed.
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > . Memory : -Xmx2048m
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > vertices Edges
>> > per vertex 0.6.4 0.7.0-SNAPSHOT(2014/09/20) Improvement
>> > rate(%)
>> > 1th 2nd 3rd average 1th 2nd 3rd Average
>> > (seconds)
>> > 40000 500 168.741 171.706 168.701 169.716 132.737 130.850 126.670 130.086
>> > -23.35
>> > 40000 600 207.749 208.045 195.854 203.883 159.744 153.757 156.681 156.727
>> > -23.13
>> > 40000 700 240.848 240.837 232.015 237.900 181.217 186.806 187.221 185.081
>> > -22.20
>> > 40000 800 280.729 274.51 280.754 278.664 210.918 209.112 217.013 212.348
>> > -23.80
>> > 40000 900 320.428 320.683 314.3 318.470 235.184 242.585 238.244 238.671
>> > -25.06
>> > 40000 1000 355.425 361.342 358.36 358.376 276.705 279.91 281.294 279.303
>> > -22.06
>> > 40000 1100 394.008 401.419 399.351 398.259 307.003 304.691 306.556
>> 306.083
>> > -23.14
>> > 40000 1200 454.358 463.215 454.035 457.203 373.345 366.772 357.682
>> 365.933
>> > -19.96
>> > 40000 1300 509.474 501.059 515.199 508.577 390.705 391.746 377.911
>> 386.787
>> > -23.95
>> > 40000 1400 546.603 544.56 554.895 548.686 445.245 445.604 451.151 447.333
>> > -18.47
>> > 40000 1500 619.412 616.466 610.991 615.623 514.933 496.33 522.703 511.322
>> > -16.94
>> > 40000 1600 Job failed. Job failed.
>> >
>> > 2014-09-23 13:16 GMT+09:00 Victor Lee <vvictor.lee@gmail.com>:
>> >>
>> >> Hello all.
>> >>
>> >> I conducted a perfromance test about latest version 0.7.0 of Apache Hama
>> >> (applied Kryo)
>> >> (Kryo is a fast and efficient object graph serialization framework for
>> >> Java)
>> >>
>> >> The results of the test, the performance has been greatly
>> >> improved(20~25%).
>> >> Kryo's serialization API is speed, efficiency and an easy,
>> >> so useful for hama's network communication.
>> >>
>> >> It seems a very good improvement!. : )
>> >>
>> >> I will attach the test results below.
>> >>
>> >> Thanks.
>> >>
>> >>
>> >>
>> >> ( supported by Byungseok Min. Thanks ! )
>> >>
>> >>
>> >
>>
>>
>>
>> --
>> Best Regards, Edward J. Yoon
>> CEO at DataSayer Co., Ltd.
>>



-- 
Best Regards, Edward J. Yoon
CEO at DataSayer Co., Ltd.

Mime
View raw message