giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kenrick Fernandes <kenrick....@gmail.com>
Subject Re: Giraph 1.1.0 not running on full cluster with Hadoop 2.6.0
Date Mon, 23 Mar 2015 23:06:13 GMT
Hi Phil,

The build was successful - now running the *ShortestPaths* example gives me
a different error,
*"GiraphApplicationMaster* not found" . However, when I run the PageRank
benchmark, I still
get the same *LocalJobRunner* error:

-------------------------
*Command:*
hadoop jar giraph-1.1.0-for-hadoop-2.6.0-jar-with-dependencies.jar
org.apache.giraph.benchmark.PageRankBenchmark -e 1 -s 3 -v -V 50000000 -w 30

*Error:*
Exception in thread "main" java.lang.IllegalArgumentException:
checkLocalJobRunnerConfiguration: When using LocalJobRunner, must have only
one worker since only 1 task at a time!
at
org.apache.giraph.job.GiraphJob.checkLocalJobRunnerConfiguration(GiraphJob.java:162)
at org.apache.giraph.job.GiraphJob.run(GiraphJob.java:236)
at org.apache.giraph.benchmark.GiraphBenchmark.run(GiraphBenchmark.java:96)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at
org.apache.giraph.benchmark.PageRankBenchmark.main(PageRankBenchmark.java:158)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
-------------------------

Did PageRank run fine for you ?

Thanks,
Kenrick

On Mon, Mar 23, 2015 at 4:35 PM, Phillip Rhodes <motley.crue.fan@gmail.com>
wrote:

> What I had to do to get this to work was:
>
> edit the pom.xml and change the hadoop_yarn profile to remove the one
> munge symbol that was something like _SASL_SOMETHING_OR_OTHER.
>
> Build using mvn -Phadoop_yarn -Dhadoop.version=2.5.2 (in my case)
>
>
> Phil
>
> This message optimized for indexing by NSA PRISM
>
>
> On Sun, Mar 22, 2015 at 4:28 PM, Kenrick Fernandes
> <kenrick.f15@gmail.com> wrote:
> > Hi,
> >
> > I am working with Giraph 1.1.0 and a YARN cluster with Hadoop 2.6.0.
> > I build Giraph with
> >      mvn -Phadoop_2 -Dhadoop.version=2.6.0 clean package -DskipTests
> >
> > So far, when I run any of the benchmarks or Shortest path examples, I
> always
> > get the LocalJobRunner error :
> >
> >
> >
> ---------------------------------------------------------------------------------------
> > Exception in thread "main" java.lang.IllegalArgumentException:
> > checkLocalJobRunnerConfiguration: When using LocalJobRunner, you cannot
> run
> > in split master / worker mode since there is only 1 task at a time!
> > at
> >
> org.apache.giraph.job.GiraphJob.checkLocalJobRunnerConfiguration(GiraphJob.java:168)
> > at org.apache.giraph.job.GiraphJob.run(GiraphJob.java:236)
> > at
> org.apache.giraph.benchmark.GiraphBenchmark.run(GiraphBenchmark.java:96)
> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> > at
> >
> org.apache.giraph.benchmark.PageRankBenchmark.main(PageRankBenchmark.java:158)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> > at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:606)
> > at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> > at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> > -------------------------------------------------------------------
> >
> > I have tried some solutions from forums/StackOverflow/lists, but so far
> > nothing has worked. As far as I can tell, Hadoop is configured right
> (other
> > MR benchmarks run fine). I tried changing the above Giraph code file
> (making
> > the check function returned what I wanted), but that only starts the Job
> and
> > gets it running on a single machine - it never uses more than 1 machine.
> >
> > Any help or pointers in the right direction would be much appreciated.
> >
> > Thanks,
> > Kenrick
>

Mime
View raw message