giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kenrick Fernandes <kenrick....@gmail.com>
Subject Re: Giraph 1.1.0 not running on full cluster with Hadoop 2.6.0
Date Mon, 23 Mar 2015 20:24:13 GMT
Update:

The shortest path example runs now, but only with *1 worker*. Using more
than 1 worker gives the same
*LocalJobRunner *error:

-----------------------------------------
*Command:*
hadoop jar giraph-examples-1.1.0-for-hadoop-2.6.0-jar-with-dependencies.jar
org.apache.giraph.GiraphRunner
org.apache.giraph.examples.SimpleShortestPathsComputation
-vif
org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
-vip /user/kenrick/input/tiny_graph.txt
-vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat
-op /user/kenrick/output/shortestpaths -w 2
-ca giraph.SplitMasterWorker=false

*Error:*
Exception in thread "main" java.lang.IllegalArgumentException:
checkLocalJobRunnerConfiguration: When using LocalJobRunner, must have only
one worker since only 1 task at a time!
at
org.apache.giraph.job.GiraphJob.checkLocalJobRunnerConfiguration(GiraphJob.java:162)
at org.apache.giraph.job.GiraphJob.run(GiraphJob.java:236)
at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:94)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

-----------------------------------------

Suggestions/fixes would be much appreciated.

Thanks,
Kenrick

On Mon, Mar 23, 2015 at 1:30 PM, Kenrick Fernandes <kenrick.f15@gmail.com>
wrote:

> Hi Steve,
>
> Running the Shortest Paths example with that option gives me an
> *IncompatibleClassChange* error, which
> I am still trying to figure out how to fix. However, running the PageRank
> benchmark gives an *Unrecognized option*
> error (both are below) - is there a specific way to pass the param
> depending on whether GiraphRunner is present?
>
>
> -----------------------------------------
> *Command:*
> hadoop jar
> giraph-examples-1.1.0-for-hadoop-2.6.0-jar-with-dependencies.jar
> org.apache.giraph.GiraphRunner
> org.apache.giraph.examples.SimpleShortestPathsComputation
> -vif
> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
> -vip /user/kenrick/input/tiny_graph.txt
> -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat
> -op /user/kenrick/output/shortestpaths -w 1 -ca
> giraph.SplitMasterWorker=false
>
> *Error:*
> Exception in thread "main" java.lang.IncompatibleClassChangeError: Found
> interface org.apache.hadoop.mapreduce.JobContext, but class was expected
> at
> org.apache.giraph.bsp.BspOutputFormat.checkOutputSpecs(BspOutputFormat.java:43)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:562)
> at
> org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:432)
> at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1296)
>
> -----------------------------------------
> *Command:*
> hadoop jar giraph-1.1.0-for-hadoop-2.6.0-jar-with-dependencies.jar
> org.apache.giraph.benchmark.PageRankBenchmark -e 1 -s 3 -v -V 50000000 -w
> 30 -ca giraph.SplitMasterWorker=false
>
> *Error:*
> Exception in thread "main"
> org.apache.commons.cli.UnrecognizedOptionException: Unrecognized option: -ca
> at org.apache.commons.cli.Parser.processOption(Parser.java:363)
> at org.apache.commons.cli.Parser.parse(Parser.java:199)
> at org.apache.commons.cli.Parser.parse(Parser.java:85)
> at org.apache.giraph.benchmark.GiraphBenchmark.run(GiraphBenchmark.java:71)
>
> -----------------------------------------
>
> Thanks,
> Kenrick
>
> On Mon, Mar 23, 2015 at 9:26 AM, Steven Harenberg <sdharenb@ncsu.edu>
> wrote:
>
>> Try adding this option: -ca giraph.SplitMasterWorker=false
>>
>> --Steve
>>
>>
>> On Sun, Mar 22, 2015 at 4:28 PM, Kenrick Fernandes <kenrick.f15@gmail.com
>> > wrote:
>>
>>> Hi,
>>>
>>> I am working with Giraph 1.1.0 and a YARN cluster with Hadoop 2.6.0.
>>> I build Giraph with
>>>      mvn -Phadoop_2 -Dhadoop.version=2.6.0 clean package -DskipTests
>>>
>>> So far, when I run any of the benchmarks or Shortest path examples, I
>>> always get the LocalJobRunner error :
>>>
>>>
>>>
>>> ---------------------------------------------------------------------------------------
>>> Exception in thread "main" java.lang.IllegalArgumentException:
>>> checkLocalJobRunnerConfiguration: When using LocalJobRunner, you cannot run
>>> in split master / worker mode since there is only 1 task at a time!
>>> at
>>> org.apache.giraph.job.GiraphJob.checkLocalJobRunnerConfiguration(GiraphJob.java:168)
>>> at org.apache.giraph.job.GiraphJob.run(GiraphJob.java:236)
>>> at
>>> org.apache.giraph.benchmark.GiraphBenchmark.run(GiraphBenchmark.java:96)
>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>>> at
>>> org.apache.giraph.benchmark.PageRankBenchmark.main(PageRankBenchmark.java:158)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>> at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> at java.lang.reflect.Method.invoke(Method.java:606)
>>> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
>>> -------------------------------------------------------------------
>>>
>>> I have tried some solutions from forums/StackOverflow/lists, but so far
>>> nothing has worked. As far as I can tell, Hadoop is configured right (other
>>> MR benchmarks run fine). I tried changing the above Giraph code file
>>> (making the check function returned what I wanted), but that only starts
>>> the Job and gets it running on a single machine - it never uses more than 1
>>> machine.
>>>
>>> Any help or pointers in the right direction would be much appreciated.
>>>
>>> Thanks,
>>> Kenrick
>>>
>>
>>
>

Mime
View raw message