giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Flavian Jacquot <tki1...@gmail.com>
Subject Run giraph on Hadoop yarn 2.6.0
Date Tue, 02 Dec 2014 10:28:28 GMT
Hello, I'm trying to use Giraph on hadoop 2.6.0 with yarn.

I've managed to build it by removing STATIC_SASL_SYMBOL in <munge.symbols>
in the yarn profile. with the command :
sudo mvn -Phadoop_yarn -Dhadoop.version=2.6.0 -DskipTests package

Then i've setup a single node cluster with hadoop 2.6.0 and I'm able to
perform the basic wordcount example.
I've copied the jar built into the directory share/hadoop/mapreduce to
remove the GiraphMaster not found execption.

Now I'm trying to run the SimpleShortestPaths example with the command :

$HADOOP_PREFIX/bin/hadoop jar
/usr/local/giraph/giraph-examples/target/giraph-examples-1.2.0-SNAPSHOT-for-hadoop-2.6.0-jar-with-dependencies.jar
org.apache.giraph.GiraphRunner
org.apache.giraph.examples.SimpleShortestPathsComputation -vif
org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
-vip input_tiny2.txt -vof
org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
/user/hduser/output/shortestpaths445 -w 1 -ca
giraph.SplitMasterWorker=false -yj
giraph-examples-1.2.0-SNAPSHOT-for-hadoop-2.6.0-jar-with-dependencies.jar

The data used is the tiny graph on the giraph quick start page and the job
should takes minutes to complete but it doesn't.

The full log is at http://pastebin.com/CW1FnWz2

It's possible to make giraph work on hadoop 2.6.0 or do I have to use an
earlier version ? If so which one would you suggest ?

Mime
View raw message