giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steven Harenberg <sdhar...@ncsu.edu>
Subject Re: Giraph 1.1.0 not running on full cluster with Hadoop 2.6.0
Date Tue, 24 Mar 2015 19:06:31 GMT
Hey Kenrick,

For the issue with GiraphApplicationMaster, I followed what Phillip did
here:
http://mail-archives.apache.org/mod_mbox/giraph-user/201503.mbox/%3CCAO3ErG_obGV8mELzX1j%2Be%3DaL6C%3D6%3DtdiSOVRBia2gh0H9tYLZA%40mail.gmail.com%3E

Basically you need the jar for giraph-examples to be in the directory where
you are issuing the command. You can do this by creating a symbolic link. I
have no idea why this worked and you can't use an absolute path, but that
is how it was for me.

Thanks,
Steve

On Mon, Mar 23, 2015 at 7:06 PM, Kenrick Fernandes <kenrick.f15@gmail.com>
wrote:

> Hi Phil,
>
> The build was successful - now running the *ShortestPaths* example gives
> me a different error,
> *"GiraphApplicationMaster* not found" . However, when I run the PageRank
> benchmark, I still
> get the same *LocalJobRunner* error:
>
> -------------------------
> *Command:*
> hadoop jar giraph-1.1.0-for-hadoop-2.6.0-jar-with-dependencies.jar
> org.apache.giraph.benchmark.PageRankBenchmark -e 1 -s 3 -v -V 50000000 -w 30
>
> *Error:*
> Exception in thread "main" java.lang.IllegalArgumentException:
> checkLocalJobRunnerConfiguration: When using LocalJobRunner, must have only
> one worker since only 1 task at a time!
> at
> org.apache.giraph.job.GiraphJob.checkLocalJobRunnerConfiguration(GiraphJob.java:162)
> at org.apache.giraph.job.GiraphJob.run(GiraphJob.java:236)
> at org.apache.giraph.benchmark.GiraphBenchmark.run(GiraphBenchmark.java:96)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> at
> org.apache.giraph.benchmark.PageRankBenchmark.main(PageRankBenchmark.java:158)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> -------------------------
>
> Did PageRank run fine for you ?
>
> Thanks,
> Kenrick
>
> On Mon, Mar 23, 2015 at 4:35 PM, Phillip Rhodes <motley.crue.fan@gmail.com
> > wrote:
>
>> What I had to do to get this to work was:
>>
>> edit the pom.xml and change the hadoop_yarn profile to remove the one
>> munge symbol that was something like _SASL_SOMETHING_OR_OTHER.
>>
>> Build using mvn -Phadoop_yarn -Dhadoop.version=2.5.2 (in my case)
>>
>>
>> Phil
>>
>> This message optimized for indexing by NSA PRISM
>>
>>
>> On Sun, Mar 22, 2015 at 4:28 PM, Kenrick Fernandes
>> <kenrick.f15@gmail.com> wrote:
>> > Hi,
>> >
>> > I am working with Giraph 1.1.0 and a YARN cluster with Hadoop 2.6.0.
>> > I build Giraph with
>> >      mvn -Phadoop_2 -Dhadoop.version=2.6.0 clean package -DskipTests
>> >
>> > So far, when I run any of the benchmarks or Shortest path examples, I
>> always
>> > get the LocalJobRunner error :
>> >
>> >
>> >
>> ---------------------------------------------------------------------------------------
>> > Exception in thread "main" java.lang.IllegalArgumentException:
>> > checkLocalJobRunnerConfiguration: When using LocalJobRunner, you cannot
>> run
>> > in split master / worker mode since there is only 1 task at a time!
>> > at
>> >
>> org.apache.giraph.job.GiraphJob.checkLocalJobRunnerConfiguration(GiraphJob.java:168)
>> > at org.apache.giraph.job.GiraphJob.run(GiraphJob.java:236)
>> > at
>> org.apache.giraph.benchmark.GiraphBenchmark.run(GiraphBenchmark.java:96)
>> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>> > at
>> >
>> org.apache.giraph.benchmark.PageRankBenchmark.main(PageRankBenchmark.java:158)
>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> > at
>> >
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> > at
>> >
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> > at java.lang.reflect.Method.invoke(Method.java:606)
>> > at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
>> > at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
>> > -------------------------------------------------------------------
>> >
>> > I have tried some solutions from forums/StackOverflow/lists, but so far
>> > nothing has worked. As far as I can tell, Hadoop is configured right
>> (other
>> > MR benchmarks run fine). I tried changing the above Giraph code file
>> (making
>> > the check function returned what I wanted), but that only starts the
>> Job and
>> > gets it running on a single machine - it never uses more than 1 machine.
>> >
>> > Any help or pointers in the right direction would be much appreciated.
>> >
>> > Thanks,
>> > Kenrick
>>
>
>

Mime
View raw message