giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kenrick Fernandes <kenrick....@gmail.com>
Subject Re: Giraph 1.1.0 not running on full cluster with Hadoop 2.6.0
Date Mon, 23 Mar 2015 17:30:48 GMT
Hi Steve,

Running the Shortest Paths example with that option gives me an
*IncompatibleClassChange* error, which
I am still trying to figure out how to fix. However, running the PageRank
benchmark gives an *Unrecognized option*
error (both are below) - is there a specific way to pass the param
depending on whether GiraphRunner is present?


-----------------------------------------
*Command:*
hadoop jar giraph-examples-1.1.0-for-hadoop-2.6.0-jar-with-dependencies.jar
org.apache.giraph.GiraphRunner
org.apache.giraph.examples.SimpleShortestPathsComputation
-vif
org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
-vip /user/kenrick/input/tiny_graph.txt
-vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat
-op /user/kenrick/output/shortestpaths -w 1 -ca
giraph.SplitMasterWorker=false

*Error:*
Exception in thread "main" java.lang.IncompatibleClassChangeError: Found
interface org.apache.hadoop.mapreduce.JobContext, but class was expected
at
org.apache.giraph.bsp.BspOutputFormat.checkOutputSpecs(BspOutputFormat.java:43)
at
org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:562)
at
org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:432)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1296)

-----------------------------------------
*Command:*
hadoop jar giraph-1.1.0-for-hadoop-2.6.0-jar-with-dependencies.jar
org.apache.giraph.benchmark.PageRankBenchmark -e 1 -s 3 -v -V 50000000 -w
30 -ca giraph.SplitMasterWorker=false

*Error:*
Exception in thread "main"
org.apache.commons.cli.UnrecognizedOptionException: Unrecognized option: -ca
at org.apache.commons.cli.Parser.processOption(Parser.java:363)
at org.apache.commons.cli.Parser.parse(Parser.java:199)
at org.apache.commons.cli.Parser.parse(Parser.java:85)
at org.apache.giraph.benchmark.GiraphBenchmark.run(GiraphBenchmark.java:71)

-----------------------------------------

Thanks,
Kenrick

On Mon, Mar 23, 2015 at 9:26 AM, Steven Harenberg <sdharenb@ncsu.edu> wrote:

> Try adding this option: -ca giraph.SplitMasterWorker=false
>
> --Steve
>
>
> On Sun, Mar 22, 2015 at 4:28 PM, Kenrick Fernandes <kenrick.f15@gmail.com>
> wrote:
>
>> Hi,
>>
>> I am working with Giraph 1.1.0 and a YARN cluster with Hadoop 2.6.0.
>> I build Giraph with
>>      mvn -Phadoop_2 -Dhadoop.version=2.6.0 clean package -DskipTests
>>
>> So far, when I run any of the benchmarks or Shortest path examples, I
>> always get the LocalJobRunner error :
>>
>>
>>
>> ---------------------------------------------------------------------------------------
>> Exception in thread "main" java.lang.IllegalArgumentException:
>> checkLocalJobRunnerConfiguration: When using LocalJobRunner, you cannot run
>> in split master / worker mode since there is only 1 task at a time!
>> at
>> org.apache.giraph.job.GiraphJob.checkLocalJobRunnerConfiguration(GiraphJob.java:168)
>> at org.apache.giraph.job.GiraphJob.run(GiraphJob.java:236)
>> at
>> org.apache.giraph.benchmark.GiraphBenchmark.run(GiraphBenchmark.java:96)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>> at
>> org.apache.giraph.benchmark.PageRankBenchmark.main(PageRankBenchmark.java:158)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:606)
>> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
>> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
>> -------------------------------------------------------------------
>>
>> I have tried some solutions from forums/StackOverflow/lists, but so far
>> nothing has worked. As far as I can tell, Hadoop is configured right (other
>> MR benchmarks run fine). I tried changing the above Giraph code file
>> (making the check function returned what I wanted), but that only starts
>> the Job and gets it running on a single machine - it never uses more than 1
>> machine.
>>
>> Any help or pointers in the right direction would be much appreciated.
>>
>> Thanks,
>> Kenrick
>>
>
>

Mime
View raw message