hama-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thomas Jungblut <thomas.jungb...@gmail.com>
Subject Re: Pagerank example doesn't work
Date Thu, 30 Aug 2012 13:38:19 GMT
Still has some overhead, the wiki dataset I benched were just 1gb.
Wonder where our hotspots are, did you get your yourkit license to profile?

2012/8/30 Edward J. Yoon <edwardyoon@apache.org>

> My settings and envs were like this:
>
>  * child.java.opts = -Xmx1024m
>  * tasks.num per node = 5
>  * total tasks = 85
>  * network env = infiniband 40G
>  * physical nodes = 17
>  * hadoop-cdh-0.20
>  * zookeeper-cdh
>
> Finished in 127 secs.
>
> On Thu, Aug 30, 2012 at 10:27 PM, Thomas Jungblut
> <thomas.jungblut@gmail.com> wrote:
> > Yeah, lots of ram ;))
> >
> > 2012/8/30 Edward J. Yoon <edwardyoon@apache.org>
> >
> >> > It runs locally in under 120s on my computer
> >>
> >> Haha, single machine?
> >>
> >> On Thu, Aug 30, 2012 at 10:25 PM, Thomas Jungblut
> >> <thomas.jungblut@gmail.com> wrote:
> >> > It runs locally in under 120s on my computer so I assume that this is
> >> > correct.
> >> >
> >> > 2012/8/30 Edward J. Yoon <edwardyoon@apache.org>
> >> >
> >> >> Just two minutes were needed to process wikipedia dataset. Is this
> >> normal?
> >> >>
> >> >> On Tue, Jul 10, 2012 at 8:17 PM, Thomas Jungblut
> >> >> <thomas.jungblut@gmail.com> wrote:
> >> >> > http://snap.stanford.edu/data/
> >> >> >
> >> >> > 2012/7/10 Edward J. Yoon <edwardyoon@apache.org>
> >> >> >
> >> >> >> Just wondering, do you know more large and downloadable data
set?
> >> >> >>
> >> >> >> On Tue, Jul 10, 2012 at 8:01 PM, Edward J. Yoon <
> >> edwardyoon@apache.org>
> >> >> >> wrote:
> >> >> >> > Oh... thanks. now I can guess what it is.
> >> >> >> >
> >> >> >> > On Tue, Jul 10, 2012 at 7:57 PM, Thomas Jungblut
> >> >> >> > <thomas.jungblut@gmail.com> wrote:
> >> >> >> >> Yes, the example graph is not complete thus resulting
in NPEs.
> >> >> >> >>
> >> >> >> >> 2012/7/10 Edward J. Yoon <edwardyoon@apache.org>
> >> >> >> >>
> >> >> >> >>> Nope, should I set to true?
> >> >> >> >>>
> >> >> >> >>> On Tue, Jul 10, 2012 at 7:52 PM, Thomas Jungblut
> >> >> >> >>> <thomas.jungblut@gmail.com> wrote:
> >> >> >> >>> > Set "hama.graph.repair" to true in conf?
> >> >> >> >>> >
> >> >> >> >>> > 2012/7/10 Edward J. Yoon <edwardyoon@apache.org>
> >> >> >> >>> >
> >> >> >> >>> >> I installed Hama 0.5 on 4 racks (4 x
18 nodes) w/ 3 tasks
> per
> >> >> each
> >> >> >> >>> >> node (bsp.child.java.opts = Xmx3512m),
tried to run
> PageRank
> >> on
> >> >> >> >>> >> Wikipedia dataset introduced at
> >> >> >> >>> >> http://wiki.apache.org/hama/WriteHamaGraphFile
> >> >> >> >>> >>
> >> >> >> >>> >> Doesn't work, the cause of job fail
error is different
> every
> >> >> time.
> >> >> >> >>> >> NullPointerException at loadVertices(),
ERROR bsp.BSPTask:
> >> >> Started
> >> >> >> >>> >> pinging to groom, ..., etc.
> >> >> >> >>> >>
> >> >> >> >>> >> Has anyone tried this?
> >> >> >> >>> >>
> >> >> >> >>> >> --
> >> >> >> >>> >> Best Regards, Edward J. Yoon
> >> >> >> >>> >> @eddieyoon
> >> >> >> >>> >>
> >> >> >> >>>
> >> >> >> >>>
> >> >> >> >>>
> >> >> >> >>> --
> >> >> >> >>> Best Regards, Edward J. Yoon
> >> >> >> >>> @eddieyoon
> >> >> >> >>>
> >> >> >> >
> >> >> >> >
> >> >> >> >
> >> >> >> > --
> >> >> >> > Best Regards, Edward J. Yoon
> >> >> >> > @eddieyoon
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >> --
> >> >> >> Best Regards, Edward J. Yoon
> >> >> >> @eddieyoon
> >> >> >>
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Best Regards, Edward J. Yoon
> >> >> @eddieyoon
> >> >>
> >>
> >>
> >>
> >> --
> >> Best Regards, Edward J. Yoon
> >> @eddieyoon
> >>
>
>
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message