giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Avery Ching <ach...@apache.org>
Subject Re: Heterogeneous Vertex Types
Date Sat, 28 Jul 2012 07:43:21 GMT
Hi Nick,

Giraph needs to have be able to instantiate the vertex id, value, edges, 
and messages and there can only be a single type for each of them.

That being said, since all the types are user chosen, you can basically 
implement types that support multiple types inside of it.  As a simple, 
stupid example

class MessageA {
...
}

class MessageB {
...
}

class MultiMessage implements Writable {
     private MessageA msgA;
     private MessageB msgB;

     private boolean useMsgA;

     pubic MessageA getMsgA(){
         return msgA;
     }

     pubic MessageB getMsgB(){
         return msgB;
     }

     ...
}

I didn't try and compile it, but hopefully you get the idea.  =)

On 7/27/12 9:52 AM, Jonathan Bishop wrote:
> Nick,
>
> I am guessing it is because Giraph needs to move vertices around and 
> needs to construct them before it reads in their serialization. I am 
> new to Giraph myself so this may be incorrect. Maybe Avery could comment.
>
> Jon
>
> On Fri, Jul 27, 2012 at 6:32 AM, Nick West 
> <nick.west@benchmarksolutions.com 
> <mailto:nick.west@benchmarksolutions.com>> wrote:
>
>     Thanks for the quick reply.
>
>     That (using one uber-class) was my initial thought when I ran into
>     this problem, however I was wondering if there was a solution that
>     would use the type hierarchy (which admittedly is a bit more
>     flexible in scala than java, so this may not be possible).  Is
>     there something in the Giraph architecture that precludes this?
>
>     Thanks,
>     Nick
>
>     On Jul 25, 2012, at 7:11 PM, Jonathan Bishop wrote:
>
>>     Nick,
>>
>>     You may want to reconsider your approach as I don't think Giraph
>>     will be happy with this.
>>
>>     How about using a new vertex value class with BasicVertex which
>>     can do both behaviours for you? Same for edge and message
>>     classes. This should be enough do what you want.
>>
>>     Jon
>>
>>     On Wed, Jul 25, 2012 at 3:29 PM, Nick West
>>     <nick.west@benchmarksolutions.com
>>     <mailto:nick.west@benchmarksolutions.com>> wrote:
>>
>>         Hi,
>>
>>         I'm working on implementing a belief propagation algorithm
>>         over Giraph.  (Do you know if anyone has done this before?)
>>          This requires having (at least) two different types of
>>         vertices implemented (values and factors) and different types
>>         of messages sent between different vertices.
>>
>>         I've been able to set up and run my own vertices (with my own
>>         custom readers and writers), however, whenever I try to
>>         extend this to a more complex  case I run into problems.  For
>>         example, suppose aI have the following two vertex types:
>>
>>         class FooVertex() extends BasicVertex[IntWritable,
>>         IntWritable, Text, IntWritable]
>>         class BarVertex() extends BasicVertex[IntWritable,
>>         IntWritable, Text, IntWritable]
>>
>>         (both of which run fine in the basic set up), and I then
>>         configure the GiraphJob with the following,
>>
>>         job.setVertexClass(classOf[BasicVertex[IntWritable,
>>         IntWritable, Text, IntWritable]])
>>
>>         (as opposed to with FooVertex or BarVertex).
>>
>>         I get a null pointer error in GraphMapper (at line 204 in the
>>         version of the code that I am using - just after the log
>>         statement about the input format vertex index type not being
>>         known, and at the line where it checks if the
>>         vertextIndexTypes are the same).
>>
>>         Do you know what is causing this? (The stack trace is below.)
>>          Have you tried to instantiate different classes for
>>         different vertices in the past?
>>
>>         One last piece of info that might explain things: I'm
>>         developing in scala, which I have seen behave a little bit
>>         poorly with java reflection - could that be a cause?
>>
>>         Thanks,
>>         *Nick West
>>         **
>>         *
>>         Stack Trace:
>>
>>         ./bin/hadoop jar
>>         ~/code/watchdog/modules/inference/target/inference-assembly-1.0.jar
>>         bmrk.inference.giraph.IdiotRunner ising ising_out 4
>>         Input arguments:
>>         [0] = ising
>>         [1] = ising_out
>>         [2] = 4
>>         setting vertex class to: org.apache.giraph.graph.BasicVertex
>>         setting vertex input class to:
>>         bmrk.inference.giraph.IdiotVertexInputFormat
>>         setting vertex output class to:
>>         bmrk.inference.giraph.IdiotVertexOutputFormat
>>         12/07/25 17:48:54 INFO mapred.JobClient: Running job:
>>         job_201207250932_0012
>>         12/07/25 17:48:55 INFO mapred.JobClient:  map 0% reduce 0%
>>         12/07/25 17:52:43 INFO mapred.JobClient: Task Id :
>>         attempt_201207250932_0012_m_000000_0, Status : FAILED
>>         java.lang.NullPointerException
>>         at
>>         org.apache.giraph.graph.GraphMapper.determineClassTypes(GraphMapper.java:204)
>>         at
>>         org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:392)
>>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:655)
>>         at
>>         org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
>>         at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
>>         at java.security.AccessController.doPrivileged(Native Method)
>>         at javax.security.auth.Subject.doAs(Subject.java:396)
>>         at
>>         org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
>>         at org.apache.hadoop.mapred.Child.main(Child.java:249)
>>
>>
>>         Benchmark Solutions
>>         101 Park Avenue - 7th Floor
>>         New York, NY 10178
>>         Tel +1.212.220.4739 <tel:%2B1.212.220.4739> | Mobile
>>         +1.646.267.4324 <tel:%2B1.646.267.4324>
>>         *www.benchmarksolutions.com *
>>         <http://www.benchmarksolutions.com/>
>>         *<image001.png>
>>
>>
>>         *
>>
>>
>
>     *
>     Nick West
>     *
>     *
>     *Benchmark Solutions
>     101 Park Avenue - 7th Floor
>     New York, NY 10178
>     Tel +1.212.220.4739 <tel:%2B1.212.220.4739> | Mobile
>     +1.646.267.4324 <tel:%2B1.646.267.4324>
>     *www.benchmarksolutions.com * <http://www.benchmarksolutions.com/>
>     *
>
>
>     *
>
>


Mime
View raw message