hama-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sandy Ding <sandy.ding...@gmail.com>
Subject Re: How does hama split input file?
Date Wed, 14 Nov 2012 09:13:44 GMT
You are welcome, Edward.
In my case, I run hama on my VM under psudeo-distributed mode.
bsp.local.tasks.maximum = 10
bsp.tasks.maximum = 3
I didn't set task number.
When the file is somehow split into 4(4 tasks), job fails; when it's split
into 3, job succeeds.
And my hama version is 0.5.0.


2012/11/14 Edward J. Yoon <edwardyoon@apache.org>

> Oh, when you set the number of tasks to 3 or more, Job fails.
>
> Right?
>
> Thanks Sandy. It looks like a bug.
>
> On Wed, Nov 14, 2012 at 11:56 AM, Sandy Ding <sandy.dingxin@gmail.com>
> wrote:
> > A B D
> > B C D
> > C A
> > D C
> > won't work.
> >
> > a b d
> > b c d
> > c a
> > d c
> > works under name pr.sim.abcd, but won't work with other names, like
> >  pr.in.xx.
> >
> > a b d
> > b c d
> > c a
> > won't work.
> >
> >
> >
> > 2012/11/13 Edward J. Yoon <edwardyoon@apache.org>
> >
> >> Hi,
> >>
> >> Can you attach your input file here so that I can check what's wrong?
> >>
> >> On Tue, Nov 13, 2012 at 10:28 PM, Sandy Ding <sandy.dingxin@gmail.com>
> >> wrote:
> >> > Hi,
> >> >
> >> > I am recently quite confused about how hama splits input file.
> >> > I ran hama pagerank with a very simple input file (with only 4
> vertices
> >> and
> >> > 6 edges). The file is split into 4 parts(while there are 3 tasks) and
> the
> >> > job failed. The bsp master log shows that :
> >> > Scheduling of job pagerank could not be done successfully, killing it.
> >> > And then zookeeper session timeout.
> >> > The job then succeed dramatically after I change the vertices name of
> the
> >> > input file (call this file1), but fail again when I try to delete one
> >> line
> >> > from the file. Even when I changed the file to be exactly the same as
> >> file1
> >> > but with a different name, the job still failed.
> >> > Can someone tell me how does the splitting part work? I am really
> >> confused.
> >> >
> >> >
> >> > Best,
> >> > Sandy
> >>
> >>
> >>
> >> --
> >> Best Regards, Edward J. Yoon
> >> @eddieyoon
> >>
>
>
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message