giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthew Laird <lai...@sfu.ca>
Subject LocalJobRunner & examples
Date Sun, 13 Oct 2013 20:57:08 GMT
I've been trying to work through the Quick Start examples and adapting 
it to my local installation, I'm running Hadoop 2.0.5-alpha across 4 
machines.

I've done the checkout from github

sudo git clone https://github.com/apache/giraph.git

Applied the patch to ZooKeeperManager

http://www.mail-archive.com/user@giraph.apache.org/msg00945/check.diff

Fixed the catching of the exception in 
giraph-core/src/main/java/org/apache/giraph/job/HaltApplicationUtils.java

Then building with

mvn clean install -Phadoop_2.0.3 -DskipTests

Then when I go to run the examples, I set -w 4 because I am running a 
split master setup.  Unfortunately this leads to:

lairdm@x007:/opt/giraph$ $HADOOP_HOME/bin/hadoop jar 
$GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.0.3-alpha-jar-with-dependencies.jar

org.apache.giraph.GiraphRunner 
org.apache.giraph.examples.SimpleShortestPathsComputation -vif 
org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat 
-vip /in/tiny_graph.txt -vof 
org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op 
/user/hduser/output/shortestpaths -w 4
13/10/13 13:50:48 WARN util.NativeCodeLoader: Unable to load 
native-hadoop library for your platform... using builtin-java classes 
where applicable
13/10/13 13:50:48 INFO utils.ConfigurationUtils: No edge input format 
specified. Ensure your InputFormat does not require one.
13/10/13 13:50:48 INFO utils.ConfigurationUtils: No edge output format 
specified. Ensure your OutputFormat does not require one.
13/10/13 13:50:49 WARN conf.Configuration: mapreduce.job.counters.limit 
is deprecated. Instead, use mapreduce.job.counters.max
13/10/13 13:50:49 WARN conf.Configuration: mapred.job.map.memory.mb is 
deprecated. Instead, use mapreduce.map.memory.mb
13/10/13 13:50:49 WARN conf.Configuration: mapred.job.reduce.memory.mb 
is deprecated. Instead, use mapreduce.reduce.memory.mb
13/10/13 13:50:49 WARN conf.Configuration: 
mapred.map.tasks.speculative.execution is deprecated. Instead, use 
mapreduce.map.speculative
13/10/13 13:50:49 WARN conf.Configuration: 
mapreduce.user.classpath.first is deprecated. Instead, use 
mapreduce.job.user.classpath.first
13/10/13 13:50:49 WARN conf.Configuration: mapred.map.max.attempts is 
deprecated. Instead, use mapreduce.map.maxattempts
13/10/13 13:50:49 INFO job.GiraphJob: run: Since checkpointing is 
disabled (default), do not allow any task retries (setting 
mapred.map.max.attempts = 0, old value = 4)
13/10/13 13:50:49 WARN conf.Configuration: mapred.job.tracker is 
deprecated. Instead, use mapreduce.jobtracker.address
Exception in thread "main" java.lang.IllegalArgumentException: 
checkLocalJobRunnerConfiguration: When using LocalJobRunner, must have 
only one worker since only 1 task at a time!
         at 
org.apache.giraph.job.GiraphJob.checkLocalJobRunnerConfiguration(GiraphJob.java:159)
         at org.apache.giraph.job.GiraphJob.run(GiraphJob.java:233)
         at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:94)
         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
         at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
         at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
         at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
         at java.lang.reflect.Method.invoke(Method.java:601)
         at org.apache.hadoop.util.RunJar.main(RunJar.java:212)

I'm not sure what parameters I need to give giraph to signal not to use 
LocalJobRunner.  Does anyone have a suggestion?

Thanks.


Mime
View raw message