hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thomas Jungblut <thomas.jungb...@googlemail.com>
Subject Re: Awesome bench results after removing Thread.sleep in sync() method.
Date Thu, 22 Sep 2011 11:58:26 GMT
Scripted a fix version:

http://pastebin.com/WbWWxd2R

You can test this as well if you like.

2011/9/22 Thomas Jungblut <thomas.jungblut@googlemail.com>

> And I think we should change this benchmark from random to a stable
> implementation.
> So we should communicate with all of the other peers, not just a random
> peername.
> Then we can precompute (or cache) the tag message and we save the string
> concat operations. We can put them into a directly allocated ByteBuffer as
> well, this will save the serialization.
> But then we cannot compare the results with the versions before.
>
> Additional to the log level change, we can receive a superior performance
> improvement ;)
>
>
> 2011/9/22 Edward J. Yoon <edwardyoon@apache.org>
>
>> Haha, obviously it can't be ignored.
>>
>> Unfortunately, I can't access my test machines now. I'll check tomorrow.
>>
>> On Thu, Sep 22, 2011 at 5:05 PM, Thomas Jungblut
>> <thomas.jungblut@googlemail.com> wrote:
>> > You're going to laugh, but we spend 80% of the time, logging the
>> messages.
>> > Let's change the log level to debug or remove the logging in the bench
>> > example.
>> >
>> > Sadly I still receive
>> >
>> > org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode =
>> >> NoNode for /bsp/job_201109220959_0001/224/ready
>> >>
>> >
>> > and it hangs forever. Current version is after you committed ChiaHung's
>> > patch.
>> > I'm in pseudo-distributed mode with 3 tasks.
>> >
>> > Are you going to bench this without the logging? That would be
>> interesting
>> > though ;D
>> >
>> > 2011/9/22 Thomas Jungblut <thomas.jungblut@googlemail.com>
>> >
>> >> That is great. I think we can push this under 200s.
>> >> I attach a profiler and send you a list of hotspots.
>> >>
>> >> lg.
>> >>
>> >> 2011/9/22 Edward J. Yoon <edwardyoon@apache.org>
>> >>
>> >> By ChiaHung's HAMA-387.patch, hang problem is fixed.
>> >>>
>> >>> And also, on same environment (1 rack, 256 cores), a bench example
>> >>> result is dramatically improved. (184.076 seconds from 307.129
>> >>> seconds)
>> >>>
>> >>> ----
>> >>> # core/bin/hama jar
>> >>> examples/target/hama-examples-0.4.0-incubating-SNAPSHOT.jar bench 16
>> >>> 1000 512
>> >>> ..
>> >>> 11/09/22 10:27:32 INFO bsp.BSPJobClient: Current supersteps number:
>> 504
>> >>> 11/09/22 10:27:35 INFO bsp.BSPJobClient: Current supersteps number:
>> 508
>> >>> 11/09/22 10:27:38 INFO bsp.BSPJobClient: Current supersteps number:
>> 512
>> >>> 11/09/22 10:27:38 INFO bsp.BSPJobClient: The total number of
>> supersteps:
>> >>> 512
>> >>> Job Finished in 184.076 seconds
>> >>>
>> >>> Hama 0.4 (r.1163903) was:
>> >>>
>> >>> 16 bytes | 1000 | 512 | 307.129 seconds
>> >>>
>> >>> --
>> >>> Best Regards, Edward J. Yoon
>> >>> @eddieyoon
>> >>>
>> >>
>> >>
>> >>
>> >> --
>> >> Thomas Jungblut
>> >> Berlin
>> >>
>> >> mobile: 0170-3081070
>> >>
>> >> business: thomas.jungblut@testberichte.de
>> >> private: thomas.jungblut@gmail.com
>> >>
>> >
>> >
>> >
>> > --
>> > Thomas Jungblut
>> > Berlin
>> >
>> > mobile: 0170-3081070
>> >
>> > business: thomas.jungblut@testberichte.de
>> > private: thomas.jungblut@gmail.com
>> >
>>
>>
>>
>> --
>> Best Regards, Edward J. Yoon
>> @eddieyoon
>>
>
>
>
> --
> Thomas Jungblut
> Berlin
>
> mobile: 0170-3081070
>
> business: thomas.jungblut@testberichte.de
> private: thomas.jungblut@gmail.com
>



-- 
Thomas Jungblut
Berlin

mobile: 0170-3081070

business: thomas.jungblut@testberichte.de
private: thomas.jungblut@gmail.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message