hama-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thomas Jungblut <thomas.jungb...@gmail.com>
Subject Re: Pagerank example doesn't work
Date Thu, 30 Aug 2012 13:44:44 GMT
Great :)
I made a tutorial back then:
http://codingwiththomas.blogspot.de/2011/04/profiling-apache-hama-bsps.html

2012/8/30 Edward J. Yoon <edwardyoon@apache.org>

> Yes, I'll try it.
>
> On Thu, Aug 30, 2012 at 10:38 PM, Thomas Jungblut
> <thomas.jungblut@gmail.com> wrote:
> > Still has some overhead, the wiki dataset I benched were just 1gb.
> > Wonder where our hotspots are, did you get your yourkit license to
> profile?
> >
> > 2012/8/30 Edward J. Yoon <edwardyoon@apache.org>
> >
> >> My settings and envs were like this:
> >>
> >>  * child.java.opts = -Xmx1024m
> >>  * tasks.num per node = 5
> >>  * total tasks = 85
> >>  * network env = infiniband 40G
> >>  * physical nodes = 17
> >>  * hadoop-cdh-0.20
> >>  * zookeeper-cdh
> >>
> >> Finished in 127 secs.
> >>
> >> On Thu, Aug 30, 2012 at 10:27 PM, Thomas Jungblut
> >> <thomas.jungblut@gmail.com> wrote:
> >> > Yeah, lots of ram ;))
> >> >
> >> > 2012/8/30 Edward J. Yoon <edwardyoon@apache.org>
> >> >
> >> >> > It runs locally in under 120s on my computer
> >> >>
> >> >> Haha, single machine?
> >> >>
> >> >> On Thu, Aug 30, 2012 at 10:25 PM, Thomas Jungblut
> >> >> <thomas.jungblut@gmail.com> wrote:
> >> >> > It runs locally in under 120s on my computer so I assume that
this
> is
> >> >> > correct.
> >> >> >
> >> >> > 2012/8/30 Edward J. Yoon <edwardyoon@apache.org>
> >> >> >
> >> >> >> Just two minutes were needed to process wikipedia dataset.
Is this
> >> >> normal?
> >> >> >>
> >> >> >> On Tue, Jul 10, 2012 at 8:17 PM, Thomas Jungblut
> >> >> >> <thomas.jungblut@gmail.com> wrote:
> >> >> >> > http://snap.stanford.edu/data/
> >> >> >> >
> >> >> >> > 2012/7/10 Edward J. Yoon <edwardyoon@apache.org>
> >> >> >> >
> >> >> >> >> Just wondering, do you know more large and downloadable
data
> set?
> >> >> >> >>
> >> >> >> >> On Tue, Jul 10, 2012 at 8:01 PM, Edward J. Yoon <
> >> >> edwardyoon@apache.org>
> >> >> >> >> wrote:
> >> >> >> >> > Oh... thanks. now I can guess what it is.
> >> >> >> >> >
> >> >> >> >> > On Tue, Jul 10, 2012 at 7:57 PM, Thomas Jungblut
> >> >> >> >> > <thomas.jungblut@gmail.com> wrote:
> >> >> >> >> >> Yes, the example graph is not complete thus
resulting in
> NPEs.
> >> >> >> >> >>
> >> >> >> >> >> 2012/7/10 Edward J. Yoon <edwardyoon@apache.org>
> >> >> >> >> >>
> >> >> >> >> >>> Nope, should I set to true?
> >> >> >> >> >>>
> >> >> >> >> >>> On Tue, Jul 10, 2012 at 7:52 PM, Thomas
Jungblut
> >> >> >> >> >>> <thomas.jungblut@gmail.com> wrote:
> >> >> >> >> >>> > Set "hama.graph.repair" to true
in conf?
> >> >> >> >> >>> >
> >> >> >> >> >>> > 2012/7/10 Edward J. Yoon <edwardyoon@apache.org>
> >> >> >> >> >>> >
> >> >> >> >> >>> >> I installed Hama 0.5 on 4 racks
(4 x 18 nodes) w/ 3
> tasks
> >> per
> >> >> >> each
> >> >> >> >> >>> >> node (bsp.child.java.opts =
Xmx3512m), tried to run
> >> PageRank
> >> >> on
> >> >> >> >> >>> >> Wikipedia dataset introduced
at
> >> >> >> >> >>> >> http://wiki.apache.org/hama/WriteHamaGraphFile
> >> >> >> >> >>> >>
> >> >> >> >> >>> >> Doesn't work, the cause of
job fail error is different
> >> every
> >> >> >> time.
> >> >> >> >> >>> >> NullPointerException at loadVertices(),
ERROR
> bsp.BSPTask:
> >> >> >> Started
> >> >> >> >> >>> >> pinging to groom, ..., etc.
> >> >> >> >> >>> >>
> >> >> >> >> >>> >> Has anyone tried this?
> >> >> >> >> >>> >>
> >> >> >> >> >>> >> --
> >> >> >> >> >>> >> Best Regards, Edward J. Yoon
> >> >> >> >> >>> >> @eddieyoon
> >> >> >> >> >>> >>
> >> >> >> >> >>>
> >> >> >> >> >>>
> >> >> >> >> >>>
> >> >> >> >> >>> --
> >> >> >> >> >>> Best Regards, Edward J. Yoon
> >> >> >> >> >>> @eddieyoon
> >> >> >> >> >>>
> >> >> >> >> >
> >> >> >> >> >
> >> >> >> >> >
> >> >> >> >> > --
> >> >> >> >> > Best Regards, Edward J. Yoon
> >> >> >> >> > @eddieyoon
> >> >> >> >>
> >> >> >> >>
> >> >> >> >>
> >> >> >> >> --
> >> >> >> >> Best Regards, Edward J. Yoon
> >> >> >> >> @eddieyoon
> >> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >> --
> >> >> >> Best Regards, Edward J. Yoon
> >> >> >> @eddieyoon
> >> >> >>
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Best Regards, Edward J. Yoon
> >> >> @eddieyoon
> >> >>
> >>
> >>
> >>
> >> --
> >> Best Regards, Edward J. Yoon
> >> @eddieyoon
> >>
>
>
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message