giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vernon Thommeret <syno...@gmail.com>
Subject Divide by zero error trying to specify input format
Date Mon, 03 Sep 2012 04:44:59 GMT
I'm getting started with Giraph and I have a basic program running
using the PseudoRandomVertexInputFormat. However, when I switch it to
the IntIntNullIntTextInputFormat and specify an input file, I get a
divide by zero error. I'm assuming that (1) I'm not setting this up
properly and (2) there should probably be a length check before the
division happens.

This is the error I'm getting:

java.lang.ArithmeticException: / by zero
        at org.apache.giraph.graph.LocalityInfoSorter.prioritizeLocalInputSplits(LocalityInfoSorter.java:107)
        at org.apache.giraph.graph.LocalityInfoSorter.<init>(LocalityInfoSorter.java:71)
        at org.apache.giraph.graph.BspServiceWorker.reserveInputSplit(BspServiceWorker.java:228)
        at org.apache.giraph.graph.BspServiceWorker.loadVertices(BspServiceWorker.java:317)
        at org.apache.giraph.graph.BspServiceWorker.setup(BspServiceWorker.java:604)
        at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:368)
        at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:569)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
        at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:396)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
        at org.apache.hadoop.mapred.Child.main(Child.java:264)

This is the change that causes the program to no longer work:

------------------------------------

 import org.apache.hadoop.io.LongWritable;

 import org.apache.giraph.graph.Edge;
-import org.apache.giraph.graph.EdgeListVertex;
+import org.apache.giraph.graph.IntIntNullIntVertex;
 import org.apache.giraph.graph.GiraphJob;
-import org.apache.giraph.io.PseudoRandomVertexInputFormat;
+import org.apache.giraph.io.IntIntNullIntTextInputFormat;
 import org.apache.giraph.io.AdjacencyListTextVertexOutputFormat;

 /**
  * Simple function to return the in degree for each vertex.
  */
-public class SimpleInDegreeCountVertex extends EdgeListVertex<
-    LongWritable, DoubleWritable, DoubleWritable, DoubleWritable>
-    implements Tool {
+public class SimpleInDegreeCountVertex extends IntIntNullIntVertex
implements Tool {

   private Configuration conf;

   @Override
-  public void compute(Iterable<DoubleWritable> messages) {
+  public void compute(Iterable<IntWritable> messages) {
     voteToHalt();
   }

@@ -80,15 +78,11 @@ public class SimpleInDegreeCountVertex extends
EdgeListVertex<
     GiraphJob job = new GiraphJob(getConf(), getClass().getName());

     job.setVertexClass(SimpleInDegreeCountVertex.class);
-    job.setVertexInputFormatClass(PseudoRandomVertexInputFormat.class);
+    job.setVertexInputFormatClass(IntIntNullIntTextInputFormat.class);
     job.setVertexOutputFormatClass(AdjacencyListTextVertexOutputFormat.class);
     job.setWorkerConfiguration(10, 10, 100.0f);

-    job.getConfiguration().setLong(
-        PseudoRandomVertexInputFormat.AGGREGATE_VERTICES, 100l);
-    job.getConfiguration().setLong(
-        PseudoRandomVertexInputFormat.EDGES_PER_VERTEX, 2l);
-
+    FileInputFormat.addInputPath(job.getInternalJob(), new
Path("connections/1.txt"));
     FileOutputFormat.setOutputPath(job.getInternalJob(), new
Path("in_degree_output"));

     boolean isVerbose = cmd.hasOption('v');

------------------------------------

"connections/1.txt" and "in_degree_output" are both in my home
directory. /connections/1.txt has the following content:

1       5
2       5       6
3       5       6
4       1       2       3

I've spent some time digging through the source and comparing to some
of the example classes, but I'm having trouble working this out. Any
thoughts?

Thanks!
Vernon

Mime
View raw message