giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vernon Thommeret <syno...@gmail.com>
Subject Re: Divide by zero error trying to specify input format
Date Wed, 05 Sep 2012 05:46:02 GMT
Just want to follow up on this. I was running on the latest version of
Giraph on Github when I was running into this issue. When I switched
back to the Giraph 0.2-SNAPSHOT release, my program was able to run.

Thanks,
Vernon

On Mon, Sep 3, 2012 at 12:44 AM, Vernon Thommeret <synotic@gmail.com> wrote:
> I'm getting started with Giraph and I have a basic program running
> using the PseudoRandomVertexInputFormat. However, when I switch it to
> the IntIntNullIntTextInputFormat and specify an input file, I get a
> divide by zero error. I'm assuming that (1) I'm not setting this up
> properly and (2) there should probably be a length check before the
> division happens.
>
> This is the error I'm getting:
>
> java.lang.ArithmeticException: / by zero
>         at org.apache.giraph.graph.LocalityInfoSorter.prioritizeLocalInputSplits(LocalityInfoSorter.java:107)
>         at org.apache.giraph.graph.LocalityInfoSorter.<init>(LocalityInfoSorter.java:71)
>         at org.apache.giraph.graph.BspServiceWorker.reserveInputSplit(BspServiceWorker.java:228)
>         at org.apache.giraph.graph.BspServiceWorker.loadVertices(BspServiceWorker.java:317)
>         at org.apache.giraph.graph.BspServiceWorker.setup(BspServiceWorker.java:604)
>         at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:368)
>         at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:569)
>         at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647)
>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
>         at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:396)
>         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1157)
>         at org.apache.hadoop.mapred.Child.main(Child.java:264)
>
> This is the change that causes the program to no longer work:
>
> ------------------------------------
>
>  import org.apache.hadoop.io.LongWritable;
>
>  import org.apache.giraph.graph.Edge;
> -import org.apache.giraph.graph.EdgeListVertex;
> +import org.apache.giraph.graph.IntIntNullIntVertex;
>  import org.apache.giraph.graph.GiraphJob;
> -import org.apache.giraph.io.PseudoRandomVertexInputFormat;
> +import org.apache.giraph.io.IntIntNullIntTextInputFormat;
>  import org.apache.giraph.io.AdjacencyListTextVertexOutputFormat;
>
>  /**
>   * Simple function to return the in degree for each vertex.
>   */
> -public class SimpleInDegreeCountVertex extends EdgeListVertex<
> -    LongWritable, DoubleWritable, DoubleWritable, DoubleWritable>
> -    implements Tool {
> +public class SimpleInDegreeCountVertex extends IntIntNullIntVertex
> implements Tool {
>
>    private Configuration conf;
>
>    @Override
> -  public void compute(Iterable<DoubleWritable> messages) {
> +  public void compute(Iterable<IntWritable> messages) {
>      voteToHalt();
>    }
>
> @@ -80,15 +78,11 @@ public class SimpleInDegreeCountVertex extends
> EdgeListVertex<
>      GiraphJob job = new GiraphJob(getConf(), getClass().getName());
>
>      job.setVertexClass(SimpleInDegreeCountVertex.class);
> -    job.setVertexInputFormatClass(PseudoRandomVertexInputFormat.class);
> +    job.setVertexInputFormatClass(IntIntNullIntTextInputFormat.class);
>      job.setVertexOutputFormatClass(AdjacencyListTextVertexOutputFormat.class);
>      job.setWorkerConfiguration(10, 10, 100.0f);
>
> -    job.getConfiguration().setLong(
> -        PseudoRandomVertexInputFormat.AGGREGATE_VERTICES, 100l);
> -    job.getConfiguration().setLong(
> -        PseudoRandomVertexInputFormat.EDGES_PER_VERTEX, 2l);
> -
> +    FileInputFormat.addInputPath(job.getInternalJob(), new
> Path("connections/1.txt"));
>      FileOutputFormat.setOutputPath(job.getInternalJob(), new
> Path("in_degree_output"));
>
>      boolean isVerbose = cmd.hasOption('v');
>
> ------------------------------------
>
> "connections/1.txt" and "in_degree_output" are both in my home
> directory. /connections/1.txt has the following content:
>
> 1       5
> 2       5       6
> 3       5       6
> 4       1       2       3
>
> I've spent some time digging through the source and comparing to some
> of the example classes, but I'm having trouble working this out. Any
> thoughts?
>
> Thanks!
> Vernon

Mime
View raw message