giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Philipp Nolte <...@daslaboratorium.de>
Subject Re: ClassNotFoundException GiraphYarn Task with Giraph 1.1.0 for Hadoop 2.5.1
Date Mon, 15 Dec 2014 07:21:01 GMT
Maybe its just a configuration thing.

I’ve tried running in giraph.SplitMasterWorker mode and its seems like hadoop is missing
the worker nodes:

Here is my command:
$ hadoop jar giraphs-and-balloons-computation-0.0.1-for-hadoop-2.5.1-and-giraph-1.1.0-RC1-jar-with-dependencies.jar
org.apache.giraph.GiraphRunner\
 de.unipassau.fim.dimis.nolte.computation.FindSupertypeClustersComputation\
 -eif de.unipassau.fim.dimis.nolte.io.NTriplesToDirectedGraphInputFormat\
 -eip /user/hduser/input/equivalence.nt\
 -vof de.unipassau.fim.dimis.nolte.io.SupertypeClustersVertexValueOutputFormat\
 -op /user/hduser/output/equivalence\
 -w 3\
 -ca mapred.job.tracker=master:5431\
 -ca giraph.SplitMasterWorker=true

And it then hangs on:

INFO mapred.LocalJobRunner: MASTER_ZOOKEEPER_ONLY checkWorkers: Only found 0 responses of
3 needed to start superstep -1 > map

If I don’t use SplitMasterWorker, I get

INFO mapred.LocalJobRunner: ALL checkWorkers: Only found 1 responses of 3 needed to start
superstep -1 > map

That is why I assumed, I need to start the task tracker on the worker nodes, but task trackers
are no longer available in hadoop 2.5.1.

My nodes’ NodeManagers are connected to the ResourceManager, so I don’t quite understand
the error message.

Any ideas? Thanks in advance!

Am 15.12.2014 um 03:31 schrieb Roman Shaposhnik <roman@shaposhnik.org>:

> On Sun, Dec 14, 2014 at 4:09 PM, Philipp Nolte <ptn@daslaboratorium.de> wrote:
>> I’ve had a look at the assembled giraph-core jar file and it does not contain any
GiraphYarnTask. How so?
>> 
>> Running my application using the GiraphRunner works fine as long as I only
>> have one worker (local mode). To use the other workers, I need to start MR
>> TaskTrackers on the machines - which aren’t available on hadoop 2.5.1.
>> 
>> Thats why I need the GiraphYarnTask.
> 
> Looks like we're talking past each other. What I am saying is that running
> pure MR-based Giraph job on a fully distributed YARNized cluster is perfectly
> valid and works fine. You don't *have* to use YARN, even though it is
> available on your cluster.
> 
> Thanks,
> Roman.


Mime
View raw message