giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paolo Castagna <>
Subject Re: A simple use case: shortest paths on a FOAF (i.e. Friend of a Friend) graph
Date Mon, 23 Apr 2012 14:58:16 GMT
Hi Avery,
first of all, apologies for my delay (I was on holiday) and many thanks for your
help. Further comments inline.

Avery Ching wrote:
> I think we should have some kind of a helper script (similar to
> bin/giraph) to running simple tests in LocalJobRunner.

That would be good, new developer might not have an Hadoop cluster at hand or
may want to debug the stuff they write and test it on their laptop before run it
on a real cluster.

> One thing to remember is that if you rerun it, you'll have to remove the
> _bsp directories that are created, otherwise it will think it has
> already been completed.

This are the program arguments I used to run PageRankBenchmark locally directly
from Eclipse:

  -libjars target/giraph-0.2-SNAPSHOT-jar-with-dependencies.jar
-Dgiraph.SplitMasterWorker=false -Dlocal.test.mode=true  -c 1 -e 2 -s 2 -V 10 -w 1

As you suggested, each time I need to delete the _bsp directories (not ideal,
but necessary).

The -libjars parameter is necessary, otherwise you get a NullPointerException:

12/04/23 15:56:54 WARN mapred.LocalJobRunner: job_local_0001
	at org.apache.giraph.graph.GraphMapper.setup(
	at org.apache.hadoop.mapred.MapTask.runNewMapper(
	at org.apache.hadoop.mapred.LocalJobRunner$

I hope this help others to get started and run/debug their Giraph jobs with Eclipse.

Thanks again,

View raw message