hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Suraj Menon <surajsme...@apache.org>
Subject Re: Issue while using DiskVerticesInfo
Date Thu, 18 Jul 2013 09:05:33 GMT
I can take a look at it this weekend.

-Suraj


On Wed, Jul 17, 2013 at 6:06 AM, Tommaso Teofili
<tommaso.teofili@gmail.com>wrote:

> Yes, I assumed the DiskVerticesInfo implementation was sorting vertices to
> accomplish that so I wonder if we have a bug there (or, for example, if
> there's something else to configure).
>
> Maybe @Suraj could help?
>
> Thanks in advance,
> Tommaso
>
> 2013/7/17 Edward J. Yoon <edwardyoon@apache.org>
>
> > Hi,
> >
> > Our graph package is a pregel-like vertex-centric programming model,
> > and it allows to communication between "vertices".
> >
> > Internally, each BSP processor performs computations for all assigned
> > vertices.
> >
> >   /**
> >    * The user-defined function
> >    */
> >   public void compute(Iterable<M> messages) throws IOException;
> >
> > To avoid grouping messages in received queue by vertex ID, we uses
> > Sorted Message Queue, and calls user-defined function for each vertex
> > sequentially. By using this sequential processing approach, we reduce
> > the memory usage.
> >
> > The current problem is vertex loading phase (or partitioner). The
> > loaded vertices in memory of each BSP processor should already be
> > sorted by vertex ID. In ListVerticesInfo case,
> >
> >   @Override
> >   public void finishAdditions() {
> >     Collections.sort(vertices);
> >   }
> >
> > This is quick-fix solution. We have to sort the vertices by vertex ID
> > at partitioning or loading phase.
> >
> > On Wed, Jul 17, 2013 at 4:44 PM, Tommaso Teofili
> > <tommaso.teofili@gmail.com> wrote:
> > > Hi all,
> > >
> > > I was trying to run the TestSubmitGraphJob with DiskVerticesInfo and I
> > got
> > > this :
> > >
> > > 13/07/17 09:21:45 INFO graph.GraphJobRunner: 7 vertices are loaded into
> > > 192.168.1.4:61001
> > >
> > > 13/07/17 09:21:45 ERROR bsp.BSPTask: Error running bsp setup and bsp
> > > function.
> > > java.lang.IllegalArgumentException: Messages must never be behind the
> > > vertex in ID! Current Message ID: facebook.com vs. stackoverflow.com
> > >  at
> org.apache.hama.graph.GraphJobRunner.iterate(GraphJobRunner.java:281)
> > > at
> > org.apache.hama.graph.GraphJobRunner.doSuperstep(GraphJobRunner.java:229)
> > >  at org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:133)
> > > at org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:177)
> > >  at org.apache.hama.bsp.BSPTask.run(BSPTask.java:146)
> > > at
> > org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1262)
> > >
> > > 13/07/17 09:21:45 INFO server.PrepRequestProcessor: Processed session
> > > termination for sessionid: 0x13feb81547f0003
> > >
> > > 13/07/17 09:21:45 INFO server.NIOServerCnxn: Closed socket connection
> for
> > > client /0:0:0:0:0:0:0:1%0:51900 which had sessionid 0x13feb81547f0003
> > >
> > >
> > > Does anyone know what could be the root cause of such a failure?
> > >
> > > Thanks a lot in advance,
> > > Tommaso
> >
> >
> >
> > --
> > Best Regards, Edward J. Yoon
> > @eddieyoon
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message