giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kenrick Fernandes <kenrick....@gmail.com>
Subject Re: Giraph 1.1.0 not running on full cluster with Hadoop 2.6.0
Date Wed, 25 Mar 2015 18:56:16 GMT
Hi Steve,

Thanks for the link - there's a different error I get now regarding not
finding some other classes,
but ive seen that before and should be able to find a fix.

Running PageRank however, still gives me the *localJobRunner* error (above)
- did you get that to run
successfully?

Thanks,
Kenrick

On Tue, Mar 24, 2015 at 3:06 PM, Steven Harenberg <sdharenb@ncsu.edu> wrote:

> Hey Kenrick,
>
> For the issue with GiraphApplicationMaster, I followed what Phillip did
> here:
> http://mail-archives.apache.org/mod_mbox/giraph-user/201503.mbox/%3CCAO3ErG_obGV8mELzX1j%2Be%3DaL6C%3D6%3DtdiSOVRBia2gh0H9tYLZA%40mail.gmail.com%3E
>
> Basically you need the jar for giraph-examples to be in the directory
> where you are issuing the command. You can do this by creating a symbolic
> link. I have no idea why this worked and you can't use an absolute path,
> but that is how it was for me.
>
> Thanks,
> Steve
>
> On Mon, Mar 23, 2015 at 7:06 PM, Kenrick Fernandes <kenrick.f15@gmail.com>
> wrote:
>
>> Hi Phil,
>>
>> The build was successful - now running the *ShortestPaths* example gives
>> me a different error,
>> *"GiraphApplicationMaster* not found" . However, when I run the PageRank
>> benchmark, I still
>> get the same *LocalJobRunner* error:
>>
>> -------------------------
>> *Command:*
>> hadoop jar giraph-1.1.0-for-hadoop-2.6.0-jar-with-dependencies.jar
>> org.apache.giraph.benchmark.PageRankBenchmark -e 1 -s 3 -v -V 50000000 -w 30
>>
>> *Error:*
>> Exception in thread "main" java.lang.IllegalArgumentException:
>> checkLocalJobRunnerConfiguration: When using LocalJobRunner, must have only
>> one worker since only 1 task at a time!
>> at
>> org.apache.giraph.job.GiraphJob.checkLocalJobRunnerConfiguration(GiraphJob.java:162)
>> at org.apache.giraph.job.GiraphJob.run(GiraphJob.java:236)
>> at
>> org.apache.giraph.benchmark.GiraphBenchmark.run(GiraphBenchmark.java:96)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>> at
>> org.apache.giraph.benchmark.PageRankBenchmark.main(PageRankBenchmark.java:158)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:606)
>> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
>> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
>> -------------------------
>>
>> Did PageRank run fine for you ?
>>
>> Thanks,
>> Kenrick
>>
>> On Mon, Mar 23, 2015 at 4:35 PM, Phillip Rhodes <
>> motley.crue.fan@gmail.com> wrote:
>>
>>> What I had to do to get this to work was:
>>>
>>> edit the pom.xml and change the hadoop_yarn profile to remove the one
>>> munge symbol that was something like _SASL_SOMETHING_OR_OTHER.
>>>
>>> Build using mvn -Phadoop_yarn -Dhadoop.version=2.5.2 (in my case)
>>>
>>>
>>> Phil
>>>
>>> This message optimized for indexing by NSA PRISM
>>>
>>>
>>> On Sun, Mar 22, 2015 at 4:28 PM, Kenrick Fernandes
>>> <kenrick.f15@gmail.com> wrote:
>>> > Hi,
>>> >
>>> > I am working with Giraph 1.1.0 and a YARN cluster with Hadoop 2.6.0.
>>> > I build Giraph with
>>> >      mvn -Phadoop_2 -Dhadoop.version=2.6.0 clean package -DskipTests
>>> >
>>> > So far, when I run any of the benchmarks or Shortest path examples, I
>>> always
>>> > get the LocalJobRunner error :
>>> >
>>> >
>>> >
>>> ---------------------------------------------------------------------------------------
>>> > Exception in thread "main" java.lang.IllegalArgumentException:
>>> > checkLocalJobRunnerConfiguration: When using LocalJobRunner, you
>>> cannot run
>>> > in split master / worker mode since there is only 1 task at a time!
>>> > at
>>> >
>>> org.apache.giraph.job.GiraphJob.checkLocalJobRunnerConfiguration(GiraphJob.java:168)
>>> > at org.apache.giraph.job.GiraphJob.run(GiraphJob.java:236)
>>> > at
>>> org.apache.giraph.benchmark.GiraphBenchmark.run(GiraphBenchmark.java:96)
>>> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>>> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>>> > at
>>> >
>>> org.apache.giraph.benchmark.PageRankBenchmark.main(PageRankBenchmark.java:158)
>>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> > at
>>> >
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>> > at
>>> >
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> > at java.lang.reflect.Method.invoke(Method.java:606)
>>> > at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
>>> > at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
>>> > -------------------------------------------------------------------
>>> >
>>> > I have tried some solutions from forums/StackOverflow/lists, but so far
>>> > nothing has worked. As far as I can tell, Hadoop is configured right
>>> (other
>>> > MR benchmarks run fine). I tried changing the above Giraph code file
>>> (making
>>> > the check function returned what I wanted), but that only starts the
>>> Job and
>>> > gets it running on a single machine - it never uses more than 1
>>> machine.
>>> >
>>> > Any help or pointers in the right direction would be much appreciated.
>>> >
>>> > Thanks,
>>> > Kenrick
>>>
>>
>>
>

Mime
View raw message