giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steven Harenberg <sdhar...@ncsu.edu>
Subject Re: Giraph 1.1.0 not running on full cluster with Hadoop 2.6.0
Date Fri, 27 Mar 2015 02:08:54 GMT
It doesn't work for me, but it seems to be because I am only on a single
machine while it requires split master/worker mode. If I move onto
distributed soon I will let you know.

--Steve

On Wed, Mar 25, 2015 at 2:56 PM, Kenrick Fernandes <kenrick.f15@gmail.com>
wrote:

> Hi Steve,
>
> Thanks for the link - there's a different error I get now regarding not
> finding some other classes,
> but ive seen that before and should be able to find a fix.
>
> Running PageRank however, still gives me the *localJobRunner* error
> (above) - did you get that to run
> successfully?
>
> Thanks,
> Kenrick
>
> On Tue, Mar 24, 2015 at 3:06 PM, Steven Harenberg <sdharenb@ncsu.edu>
> wrote:
>
>> Hey Kenrick,
>>
>> For the issue with GiraphApplicationMaster, I followed what Phillip did
>> here:
>> http://mail-archives.apache.org/mod_mbox/giraph-user/201503.mbox/%3CCAO3ErG_obGV8mELzX1j%2Be%3DaL6C%3D6%3DtdiSOVRBia2gh0H9tYLZA%40mail.gmail.com%3E
>>
>> Basically you need the jar for giraph-examples to be in the directory
>> where you are issuing the command. You can do this by creating a symbolic
>> link. I have no idea why this worked and you can't use an absolute path,
>> but that is how it was for me.
>>
>> Thanks,
>> Steve
>>
>> On Mon, Mar 23, 2015 at 7:06 PM, Kenrick Fernandes <kenrick.f15@gmail.com
>> > wrote:
>>
>>> Hi Phil,
>>>
>>> The build was successful - now running the *ShortestPaths* example
>>> gives me a different error,
>>> *"GiraphApplicationMaster* not found" . However, when I run the
>>> PageRank benchmark, I still
>>> get the same *LocalJobRunner* error:
>>>
>>> -------------------------
>>> *Command:*
>>> hadoop jar giraph-1.1.0-for-hadoop-2.6.0-jar-with-dependencies.jar
>>> org.apache.giraph.benchmark.PageRankBenchmark -e 1 -s 3 -v -V 50000000 -w 30
>>>
>>> *Error:*
>>> Exception in thread "main" java.lang.IllegalArgumentException:
>>> checkLocalJobRunnerConfiguration: When using LocalJobRunner, must have only
>>> one worker since only 1 task at a time!
>>> at
>>> org.apache.giraph.job.GiraphJob.checkLocalJobRunnerConfiguration(GiraphJob.java:162)
>>> at org.apache.giraph.job.GiraphJob.run(GiraphJob.java:236)
>>> at
>>> org.apache.giraph.benchmark.GiraphBenchmark.run(GiraphBenchmark.java:96)
>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>>> at
>>> org.apache.giraph.benchmark.PageRankBenchmark.main(PageRankBenchmark.java:158)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>> at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> at java.lang.reflect.Method.invoke(Method.java:606)
>>> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
>>> -------------------------
>>>
>>> Did PageRank run fine for you ?
>>>
>>> Thanks,
>>> Kenrick
>>>
>>> On Mon, Mar 23, 2015 at 4:35 PM, Phillip Rhodes <
>>> motley.crue.fan@gmail.com> wrote:
>>>
>>>> What I had to do to get this to work was:
>>>>
>>>> edit the pom.xml and change the hadoop_yarn profile to remove the one
>>>> munge symbol that was something like _SASL_SOMETHING_OR_OTHER.
>>>>
>>>> Build using mvn -Phadoop_yarn -Dhadoop.version=2.5.2 (in my case)
>>>>
>>>>
>>>> Phil
>>>>
>>>> This message optimized for indexing by NSA PRISM
>>>>
>>>>
>>>> On Sun, Mar 22, 2015 at 4:28 PM, Kenrick Fernandes
>>>> <kenrick.f15@gmail.com> wrote:
>>>> > Hi,
>>>> >
>>>> > I am working with Giraph 1.1.0 and a YARN cluster with Hadoop 2.6.0.
>>>> > I build Giraph with
>>>> >      mvn -Phadoop_2 -Dhadoop.version=2.6.0 clean package -DskipTests
>>>> >
>>>> > So far, when I run any of the benchmarks or Shortest path examples,
I
>>>> always
>>>> > get the LocalJobRunner error :
>>>> >
>>>> >
>>>> >
>>>> ---------------------------------------------------------------------------------------
>>>> > Exception in thread "main" java.lang.IllegalArgumentException:
>>>> > checkLocalJobRunnerConfiguration: When using LocalJobRunner, you
>>>> cannot run
>>>> > in split master / worker mode since there is only 1 task at a time!
>>>> > at
>>>> >
>>>> org.apache.giraph.job.GiraphJob.checkLocalJobRunnerConfiguration(GiraphJob.java:168)
>>>> > at org.apache.giraph.job.GiraphJob.run(GiraphJob.java:236)
>>>> > at
>>>> org.apache.giraph.benchmark.GiraphBenchmark.run(GiraphBenchmark.java:96)
>>>> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>>>> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>>>> > at
>>>> >
>>>> org.apache.giraph.benchmark.PageRankBenchmark.main(PageRankBenchmark.java:158)
>>>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>> > at
>>>> >
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>>> > at
>>>> >
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>> > at java.lang.reflect.Method.invoke(Method.java:606)
>>>> > at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
>>>> > at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
>>>> > -------------------------------------------------------------------
>>>> >
>>>> > I have tried some solutions from forums/StackOverflow/lists, but so
>>>> far
>>>> > nothing has worked. As far as I can tell, Hadoop is configured right
>>>> (other
>>>> > MR benchmarks run fine). I tried changing the above Giraph code file
>>>> (making
>>>> > the check function returned what I wanted), but that only starts the
>>>> Job and
>>>> > gets it running on a single machine - it never uses more than 1
>>>> machine.
>>>> >
>>>> > Any help or pointers in the right direction would be much appreciated.
>>>> >
>>>> > Thanks,
>>>> > Kenrick
>>>>
>>>
>>>
>>
>

Mime
View raw message