hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thomas Jungblut <thomas.jungb...@googlemail.com>
Subject Re: Awesome bench results after removing Thread.sleep in sync() method.
Date Thu, 22 Sep 2011 10:54:08 GMT
And I think we should change this benchmark from random to a stable
implementation.
So we should communicate with all of the other peers, not just a random
peername.
Then we can precompute (or cache) the tag message and we save the string
concat operations. We can put them into a directly allocated ByteBuffer as
well, this will save the serialization.
But then we cannot compare the results with the versions before.

Additional to the log level change, we can receive a superior performance
improvement ;)

2011/9/22 Edward J. Yoon <edwardyoon@apache.org>

> Haha, obviously it can't be ignored.
>
> Unfortunately, I can't access my test machines now. I'll check tomorrow.
>
> On Thu, Sep 22, 2011 at 5:05 PM, Thomas Jungblut
> <thomas.jungblut@googlemail.com> wrote:
> > You're going to laugh, but we spend 80% of the time, logging the
> messages.
> > Let's change the log level to debug or remove the logging in the bench
> > example.
> >
> > Sadly I still receive
> >
> > org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode =
> >> NoNode for /bsp/job_201109220959_0001/224/ready
> >>
> >
> > and it hangs forever. Current version is after you committed ChiaHung's
> > patch.
> > I'm in pseudo-distributed mode with 3 tasks.
> >
> > Are you going to bench this without the logging? That would be
> interesting
> > though ;D
> >
> > 2011/9/22 Thomas Jungblut <thomas.jungblut@googlemail.com>
> >
> >> That is great. I think we can push this under 200s.
> >> I attach a profiler and send you a list of hotspots.
> >>
> >> lg.
> >>
> >> 2011/9/22 Edward J. Yoon <edwardyoon@apache.org>
> >>
> >> By ChiaHung's HAMA-387.patch, hang problem is fixed.
> >>>
> >>> And also, on same environment (1 rack, 256 cores), a bench example
> >>> result is dramatically improved. (184.076 seconds from 307.129
> >>> seconds)
> >>>
> >>> ----
> >>> # core/bin/hama jar
> >>> examples/target/hama-examples-0.4.0-incubating-SNAPSHOT.jar bench 16
> >>> 1000 512
> >>> ..
> >>> 11/09/22 10:27:32 INFO bsp.BSPJobClient: Current supersteps number: 504
> >>> 11/09/22 10:27:35 INFO bsp.BSPJobClient: Current supersteps number: 508
> >>> 11/09/22 10:27:38 INFO bsp.BSPJobClient: Current supersteps number: 512
> >>> 11/09/22 10:27:38 INFO bsp.BSPJobClient: The total number of
> supersteps:
> >>> 512
> >>> Job Finished in 184.076 seconds
> >>>
> >>> Hama 0.4 (r.1163903) was:
> >>>
> >>> 16 bytes | 1000 | 512 | 307.129 seconds
> >>>
> >>> --
> >>> Best Regards, Edward J. Yoon
> >>> @eddieyoon
> >>>
> >>
> >>
> >>
> >> --
> >> Thomas Jungblut
> >> Berlin
> >>
> >> mobile: 0170-3081070
> >>
> >> business: thomas.jungblut@testberichte.de
> >> private: thomas.jungblut@gmail.com
> >>
> >
> >
> >
> > --
> > Thomas Jungblut
> > Berlin
> >
> > mobile: 0170-3081070
> >
> > business: thomas.jungblut@testberichte.de
> > private: thomas.jungblut@gmail.com
> >
>
>
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon
>



-- 
Thomas Jungblut
Berlin

mobile: 0170-3081070

business: thomas.jungblut@testberichte.de
private: thomas.jungblut@gmail.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message