giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nick West <nick.w...@benchmarksolutions.com>
Subject Heterogeneous Vertex Types
Date Wed, 25 Jul 2012 22:29:51 GMT
Hi,

I'm working on implementing a belief propagation algorithm over Giraph.  (Do you know if anyone
has done this before?)  This requires having (at least) two different types of vertices implemented
(values and factors) and different types of messages sent between different vertices.

I've been able to set up and run my own vertices (with my own custom readers and writers),
however, whenever I try to extend this to a more complex  case I run into problems.  For example,
suppose aI have the following two vertex types:

class FooVertex() extends BasicVertex[IntWritable, IntWritable, Text, IntWritable]
class BarVertex() extends BasicVertex[IntWritable, IntWritable, Text, IntWritable]

(both of which run fine in the basic set up), and I then configure the GiraphJob with the
following,

job.setVertexClass(classOf[BasicVertex[IntWritable, IntWritable, Text, IntWritable]])

(as opposed to with FooVertex or BarVertex).

I get a null pointer error in GraphMapper (at line 204 in the version of the code that I am
using - just after the log statement about the input format vertex index type not being known,
and at the line where it checks if the vertextIndexTypes are the same).

Do you know what is causing this? (The stack trace is below.)  Have you tried to instantiate
different classes for different vertices in the past?

One last piece of info that might explain things: I'm developing in scala, which I have seen
behave a little bit poorly with java reflection - could that be a cause?

Thanks,
Nick West

Stack Trace:

./bin/hadoop jar ~/code/watchdog/modules/inference/target/inference-assembly-1.0.jar bmrk.inference.giraph.IdiotRunner
ising ising_out 4
Input arguments:
[0] = ising
[1] = ising_out
[2] = 4
setting vertex class to: org.apache.giraph.graph.BasicVertex
setting vertex input class to: bmrk.inference.giraph.IdiotVertexInputFormat
setting vertex output class to: bmrk.inference.giraph.IdiotVertexOutputFormat
12/07/25 17:48:54 INFO mapred.JobClient: Running job: job_201207250932_0012
12/07/25 17:48:55 INFO mapred.JobClient:  map 0% reduce 0%
12/07/25 17:52:43 INFO mapred.JobClient: Task Id : attempt_201207250932_0012_m_000000_0, Status
: FAILED
java.lang.NullPointerException
at org.apache.giraph.graph.GraphMapper.determineClassTypes(GraphMapper.java:204)
at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:392)
at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:655)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
at org.apache.hadoop.mapred.Child.main(Child.java:249)


Benchmark Solutions
101 Park Avenue - 7th Floor
New York, NY 10178
Tel +1.212.220.4739 | Mobile +1.646.267.4324
www.benchmarksolutions.com <http://www.benchmarksolutions.com/>
[cid:image001.png@01CCA50E.43B4A860]






Mime
View raw message