incubator-giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Garcia <dgar...@potomacfusion.com>
Subject Re: how to use SimplePageRankVertex
Date Sun, 19 Feb 2012 00:00:31 GMT
Are you submitting this job to a pseudo distributed cluster or a fully distributed cluster?

Sent from my HTC Inspireā„¢ 4G on AT&T

----- Reply message -----
From: "yavuz gokirmak" <ygokirmak@gmail.com>
To: "giraph-user@incubator.apache.org" <giraph-user@incubator.apache.org>
Subject: how to use SimplePageRankVertex
Date: Sat, Feb 18, 2012 2:04 pm



Thank you for advices,

I have a few more questions.

I have created a class named INTPageRankVertex which is similar to SimplePageRankVertex and
generated a jar holding only INTPageRankVertex.java.

Later, try to run with giraph command as below but get classpath errors:

giraph INTPageRankVertex.jar org.test.INTPageRankVertex \
-ip /user/hdfs/pagerankinput/graph.input \
-op /user/hdfs/pagerankoutput/ \
-w 1  \
-if org.test.INTPageRankVertex.INTPageRankVertexInputFormat \
-of org.test.INTPageRankVertex.INTPageRankVertexOutputFormat \


First I get,
Exception in thread "main" java.lang.ClassNotFoundException: org.test.INTPageRankVertex

in bin/giraph user jar is added to classpath on line 58
58. CLASSPATH=${USER_JAR}

but CLASSPATH is overwritten on line 87
87.         CLASSPATH=`mvn dependency:build-classpath | grep -v "[INFO]"`

changing line 87 as below solves my first problem. Does this patch is valid?
87.         CLASSPATH=$CLASSPATH:`mvn dependency:build-classpath | grep -v "[INFO]"`

After changing line 87 I get a different classpath error:
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/giraph/graph/LongDoubleFloatDoubleVertex

And I solved this problem by adding below line
113. CLASSPATH=$CLASSPATH:$JAR

Does these patches are necessary or I am doing something wrong while running my code..

best regards..


On 18 February 2012 18:37, Avery Ching <aching@apache.org<mailto:aching@apache.org>>
wrote:
IntIntNullIntTextInputFormat in the examples package (extending TextVertexInputFormat as David
suggests) is very similar to what you need I think, although the types might be different
for your application.  You can start with that perhaps.

Avery


On 2/18/12 7:48 AM, David Garcia wrote:
The easiest thing to do is to extend text vertex or/and textvertext input format and/or the
record reader.  The record reader will give you the vertices you want.  Look at the record
reader for textvertexinputformat.  It's an innerclass on this format class.

Sent from my HTC Inspireā„¢ 4G on AT&T

----- Reply message -----
From: "yavuz gokirmak" <ygokirmak@gmail.com><mailto:ygokirmak@gmail.com>
To: "giraph-user@incubator.apache.org"<mailto:giraph-user@incubator.apache.org> <giraph-user@incubator.apache.org><mailto:giraph-user@incubator.apache.org>
Subject: how to use SimplePageRankVertex
Date: Sat, Feb 18, 2012 9:08 am



Hi,

I am planning to use giraph for network analysis. First I am trying to fully understand SimplePageRankVertex
implementation and modify in order to serve my needs.

I have a question about example,
What is the expected input format for SimplePageRankVertex, I couldn't understand the input
format although  SimplePageRankVertexReader class has few lines.


My input file is contains of rows such as:
usera, userb
usera, userc
userc, usera
userb, userc
userc, userb
.
.
.
Each row represents a relation between two users,
"usera,userb" means that "usera is clicked userb's profile"

Is it possible to make social network analysis over such kind of data using giraph?
I will be glad if you can give advices..

thanks in advance
best regards
ygokirmak



Mime
View raw message