giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gianmarco De Francisci Morales <g...@apache.org>
Subject Re: Should it be possible to extend BasicVertex<I,V,E,M>?
Date Thu, 21 Jun 2012 13:57:56 GMT
Hi,
Along the same lines, I get this exception for my class:

java.lang.ClassCastException:
org.apache.giraph.examples.RandomWalkWithRestartVertex cannot be cast
to org.apache.giraph.graph.MutableVertex
	at org.apache.giraph.comm.NettyWorkerServer.prepareSuperstep(NettyWorkerServer.java:175)
	at org.apache.giraph.comm.NettyWorkerClientServer.prepareSuperstep(NettyWorkerClientServer.java:123)
	at org.apache.giraph.graph.BspServiceWorker.startSuperstep(BspServiceWorker.java:902)
	at org.apache.giraph.graph.GraphMapper.map(GraphMapper.java:540)
	at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:647)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
	at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:396)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1082)
	at org.apache.hadoop.mapred.Child.main(Child.java:249)


This happens when I use Netty, otherwise it works fine.
The culprit is this line:

      if (vertex != null) {

        ((MutableVertex<I, V, E, M>) vertex).setVertexId(vertexIndex);

        partition.putVertex(vertex);

      } else if (originalVertex != null) {

        partition.removeVertex(originalVertex.getVertexId());

      }


There is a cast to MutableVertex that forces your vertex to extend it.
Is this the desired behavior? Or should one be able to use BasicVertex here?

Cheers,
--
Gianmarco




On Tue, May 29, 2012 at 1:30 PM, Paolo Castagna <
castagna.lists@googlemail.com> wrote:

> Hi Avery,
> yep, I got the message.
> Unfortunately, at the moment, I do not have a solution.
>
> Paolo
>
>
> Avery Ching wrote:
> > Yeah, here-in lies the problem Paolo.  We want developers to be able to
> > extend MutableVertex.  At the same time, we don't want users to be able
> > to call putMessages() and releaseResources() - i.e. shoot themselves in
> > the foot.
> >
> > If you have an thoughts on how to handle this, that would be great.
> >
> > Avery
> >
> > On 5/25/12 10:52 AM, Paolo Castagna wrote:
> >> Hi Avery,
> >> thanks for your reply.
> >>
> >> However, if you expect end users to extend MutableVertex, it should
> >> not have
> >> package private methods inherited from BasicVertex and not implemented
> in
> >> MutableVertex.
> >>
> >> Such as:
> >>
> >>    abstract void putMessages(Iterable<M>  messages);
> >>    abstract void releaseResources();
> >>
> >> I wanted to try to extend MutableVertex because I wanted to see what
> >> it would
> >> take to use indexes (with memory mapped files) instead of keeping
> >> everything in
> >> RAM. I've not done much progress on that, but I needed to put my stuff
> in
> >> org.apache.giraph.graph.* because of the two methods above.
> >>
> >> Please, correct me if I am wrong.
> >>
> >> Cheers,
> >> Paolo
> >>
> >> Avery Ching wrote:
> >>> Sorry for the delayed response, but feel free to extend MutableVertex
> if
> >>> the other Vertex implementations don't meet your needs.  EdgeListVertex
> >>> is probably the most commonly used simple case, but something like
> >>> IntIntNullIntVertex is very optimized for certain primitives (not
> >>> general purposed).  Still, it can be nice to same more memory...
> >>>
> >>> Avery
> >>>
> >>> On 5/21/12 10:08 AM, Paolo Castagna wrote:
> >>>> Benjamin Heitmann wrote:
> >>>>> On 21 May 2012, at 17:15, Paolo Castagna wrote:
> >>>>>> A more direct question would be: are Giraph supposed to extend
> >>>>>> BasicVertex<I,V,E,M>   when they do not find a subclass
of
> >>>>>> BasicVertex which meets
> >>>>>> their needs?
> >>>>> No, they are not. However this is not explicitly documented anywhere.
> >>>>>
> >>>>> Or to say it more clearly: If you look in the javadoc of BasicVertex
> >>>>> and if you search the mailing lists,
> >>>>> then you will find that users are discouraged from using/extending
> >>>>> BasicVertex, but you will not find any suggestion of which
> >>>>> Vertex to extend instead. I even asked basically the same question
> >>>>> once, and got very indirect answers.
> >>>>> But that is okay, I figured it out by trial and error ;)
> >>>>>
> >>>>>
> >>>>> Users are supposed to extend HashMapVertex or EdgeListVertex (both
in
> >>>>> org.apache.giraph.graph).
> >>>> Hi Benjamin,
> >>>> right, I should have seen those (this is a good sign I should stop for
> >>>> today
> >>>> and continue tomorrow morning). I didn't because I was thinking: "I
do
> >>>> not need
> >>>> my vertexes to be mutable" (since, computing PageRank does not need
to
> >>>> change
> >>>> the topology of a graph), so I was not focusing my attention on the
> >>>> MutableVertex hierarchy of classes (my mistake).
> >>>>
> >>>> Now my question would be: why SimplePageRankVertex extends
> >>>> LongDoubleFloatDoubleVertex? (But, I'll look at this tomorrow).
> >>>>
> >>>>> If you try to extend them, you will see that you can basically
> >>>>> plug-in any kind of existing class in the<I,V,E,M>   signature,
> >>>>> as long as the implement the right interfaces, which are all Writable
> >>>>> (and WritableComparable for I).
> >>>>> Then you just need to add your compute() method, and you are ready
to
> >>>>> go.
> >>>>> Both HashMapVertex and EdgeListVertex provide implementations of
all
> >>>>> the housekeeping that giraph needs.
> >>>>>
> >>>>> Trying to work directly by extending BasicVertex will not work,
as a
> >>>>> lot of methods are only accessibly on the same package level.
> >>>> Yep.
> >>>>
> >>>>> It would probably be a good idea to submit a small javadoc patch
> >>>>> which adds documentation to BasicVertex, that users need to look
at
> >>>>> those other two classes.
> >>>>>
> >>>> Yep.
> >>>>
> >>>> Thanks again for your help and for pointing me in the right direction.
> >>>>
> >>>> Paolo
> >
>
>

Mime
View raw message