incubator-giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Benjamin Heitmann <benjamin.heitm...@deri.org>
Subject Re: Problem deploying Giraph job to hadoop cluster: onlineZooKeeperServers connection failure
Date Wed, 21 Mar 2012 20:30:18 GMT

Hi Avery, and thanks for your reply. 

On 21 Mar 2012, at 06:35, Avery Ching wrote:

> Benjamin, my guess is that your jar might not have all the ZooKeeper dependencies.  Can
you look at the log for the process that was supposed to start ZooKeeper?  I'm thinking it
didn't start...

I checked the logs and there is nothing which clearly looks like a ZooKeeper error, or any
other kind of information which might provide 
a clue to why ZooKeeper is not starting.  (It least there is nothing which I can see when
I grep -i zookeeper logs/*.log ) 

In addition, I checked the dependencies (Eclipse allows viewing them like a tree). 
All of the dependencies of zookeeper are part of the jar (in the /lib dir). 
So that should not be the reason for zookeeper to fail in starting. 


In addition, today I tried using bin/giraph (and GiraphRunner ) to run my job. This also was
ultimately unsuccessful. 
As you can see from the patch which I submitted, I found a very annoying bug in bin/giraph.

However, after fixing that, the hadoop class loader complained the following: 

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/giraph/graph/HashMapVertex

That is very puzzling to me, as the giraph-0.2-SNAPSHOT-jar-with-dependencies.jar is provided
two times. 
First as the "jar" argument, and secondly as part of the "-libjars" argument. 


Right now I am freshly out of ideas as to what could be going wrong.

So I checked with my supervisor, and it is okay to put my code on github. 

It is available here: 

---->>>> https://github.com/2nd-metaman/sa-rdf-giraph


A few questions which come to my mind as a sort of checklist: 
* are my assembly instructions in pom.xml and in hadoop-job.xml correct ? 
* is my invocation of the job as described in README correct ? 
* is the construction of the GiraphJob in LoadRDFAdjacencyCSVAndHalt correct ? 

Feel free to run the SimpleRDFVertexTest class, there is a small data file so that you can
see if it works for you. 

I really hope somebody can explain to me why my giraph job is failing in such a "novel" way.

sincerely, Benjamin. 
Mime
View raw message