giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexandre Fonseca <alexandrejorgefons...@gmail.com>
Subject Re: Steps for Giraph on hadoop-2.2.0 Cluster
Date Mon, 10 Mar 2014 10:11:57 GMT
Hello Vishal,

I've run it successfully in hadoop-2.2.0 with both MR2 and Yarn computations.

For MR2, you just have to compile Giraph with the new hadoop_2 profile and then 
it should work with no problems using the commands shown in the Giraph 
quickstart.

For Yarn, compile with the hadoop_yarn profile and the application submission 
command follows the following template (replace variables with your particular 
values):

GIRAPH_JAR_FOLDER=/home/alex/Giraph
GIRAPH_JAR_NAME=giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.2.0-jar-with-
dependencies.jar
GIRAPH_JAR=$GIRAPH_JAR_FOLDER/$GIRAPH_JAR_NAME

HADOOP_CLASSPATH=$GIRAPH_JAR_FOLDER hadoop jar $GIRAPH_JAR 
org.apache.giraph.GiraphRunner 
org.apache.giraph.examples.SimpleShortestPathsComputation -vif 
org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip 
/user/alex/input/tiny_graph.txt -vof 
org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op 
/user/alex/output/shortestpaths -w 1 -yj $GIRAPH_JAR_NAME

If you don't want to compile things yourself, the grafos.ml in which I'm 
working at the moment provides, among other things, pre-compiled versions of 
giraph straight from trunk for the major targets integrated with some machine 
learning algorithms. You can check the download page here: 
http://grafos.ml/Download.html

We also have a configuration tool that should make it easy to find the exact 
command to run different algorithms (including most of Giraph examples):
http://grafos.ml/configure.html

Hopefully this should be sufficient for you to successfully run Giraph in 2.2.0.

Best regards,
Alex

On Sunday, March 09, 2014 08:59:32 PM Vishal Mishra wrote:
> Hi Guys,
> I tried giraph on hadoop-2.2.0 cluster a few months back without success
> and moved onto other pressing issues but would really love to try out again.
> 
> Has anyone been able to run giraph successfully on a hadoop-2.2.0 cluster?
> if yes, I'd really appreciate it if you can share the steps to run the
> "SimplePageRankComputation" example.
> If not, please share the steps, I'd give it a try and provide any required
> logs for debugging.
> Our setup (works for our regular hadoop/hive jobs):
> - ubuntu 13.10
> - 1 namenode & resourcemanager. + zookeeper
> - 3 datanode & nodemanager
> 
> Thanx,
> Vishal

Mime
View raw message