giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mohammad Islam <misla...@yahoo.com>
Subject Re: GiraphApplicationMaster not found (WAS: Giraph and Hadoop 2.2.0)
Date Tue, 19 Nov 2013 23:16:56 GMT
Rob,
I understand your frustration.
I'm pre-occupied with some other stuffs in next couple of days.
If no one replied by then, I will recreate the situation here and try to give you more concrete
help.

Regards,
Mohammad



On Tuesday, November 19, 2013 2:56 AM, Rob Vesse <rvesse@dotnetrdf.org> wrote:
 
Mohammad

Comments inline:

From:  Mohammad Islam <mislam77@yahoo.com>
Reply-To:  <user@giraph.apache.org>
Date:  Monday, 18 November 2013 22:24
To:  "user@giraph.apache.org" <user@giraph.apache.org>
Subject:  Re: GiraphApplicationMaster not found (WAS: Giraph and Hadoop 2.2.0)


I used the following shell script to run PR where my current working directory contains the
graph*.jar in Hadoop 2.1.1.
>. You can try something similar.

I already am.


>
>
>
>
>
>HAD_VERSION=2.1.1-SNAPSHOT
>hadoop fs -rmr out
>export GIRAPH_HOME=/home/mislam/softwares/giraph
>#export GIRAPH_CORE=$GIRAPH_HOME/giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-$HAD_VERSION-jar-with-dependencies.jar
>#export GIRAPH_EXAMPLE=$GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-$HAD_VERSION-jar-with-dependencies.jar
>
>
>
>
>CLASSPATH=$CLASSPATH:$GIRAPH_HOME/conf

What is the point of putting the conf directory on the CLASSPATH?  I assume this is only
to pick up the giraph configuration but at the same time there is nothing to stop me putting
my Giraph specific config in one of the other Hadoop configuration files e.g. core-site.xml?

#CLASSPATH=$CLASSPATH:$GIRAPH_CORE
>#CLASSPATH=$CLASSPATH:$GIRAPH_EXAMPLE
>
>
>export CLASSPATH=$CLASSPATH
>export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:$CLASSPATH
>
>
>echo $HADOOP_CLASSPATH

I assume the point of the above is to ensure that the Giraph config file is on the class path
Hadoop actually uses but adding this makes no difference to me, I get exactly the same failure
as before.  Modifying the logic to add the examples JAR onto the class path does not help
either.


>
>
>
>hadoop --config $HADOOP_HOME/etc/hadoop jar giraph-examples-1.1.0-SNAPSHOT-for-hadoop-$HAD_VERSION-jar-with-dependencies.jar
 org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimplePageRankComputation -c org.apache.giraph.combiner.DoubleSumCombiner
-w 1 -yh 1024 -yj giraph-examples-1.1.0-SNAPSHOT-for-hadoop-$HAD_VERSION-jar-with-dependencies.jar
-vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat
-mc org.apache.giraph.examples.SimplePageRankComputation\$SimplePageRankMasterCompute -vip
input -op out

This is not the same command that I am running, also DoubleSumCombiner gives a ClassNotFoundException,
you actually want DoubleSumMessageCombiner.  If I adapt my script to run your command I experience
the exact same error as I do trying to run the command I showed so I am still nowhere.

I'm getting a little frustrated that no-one has any idea what the problem is here, either
this is a simple misconfiguration on my part or a bug in Giraph for Hadoop 2.2.0 ?  Given
the thread so far and the fact that I can run MR based Giraph on Hadoop 1.2.1 this is looking
increasingly unlikely I'm starting to lean heavily towards the latter option.

Rob


>
>
>On Monday, November 18, 2013 2:24 AM, Rob Vesse <rvesse@dotnetrdf.org> wrote:
> 
>Hey all
>
>
>Bumping this thread because I am still completely blocked on this and hoping someone can
help me.  As explained earlier in the thread I have Giraph built for pure YARN on stable
Hadoop 2.2.0 release like so:
>
>
>mvn –Phadoop_yarn –Dhadoop.version=2.2.0 –DskipTests=true clean install
>
>
>But when I try and run the simple shortest paths demo like so:
>
>
>$> bin/hadoop jar /Users/rvesse/Documents/Work/Code/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.2.0-jar-with-dependencies.jar
org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleShortestPathsComputation -vif
org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip /user/rvesse/tiny_graph.txt
-vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op /user/rvesse/shortestpaths
-w 1 -yj /Users/rvesse/Documents/Work/Code/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.2.0-jar-with-dependencies.jar
>
>
>The application fails and the YARN logs show the following error:
>
>
>Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/giraph/yarn/GiraphApplicationMaster
Caused by: java.lang.ClassNotFoundException: org.apache.giraph.yarn.GiraphApplicationMaster
at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native
Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>
>
>Any ideas why the GiraphApplicationMaster would not be found?  Have Ijust got something
basic wrong in my configuration or is this a bug in Giraph?
>
>
>Ifound an earlier thread on the same topic (http://mail-archives.apache.org/mod_mbox/giraph-user/201307.mbox/%3c51E5A4D1.40006@googlemail.com%3e)but
it was never resolved hence why Ihave now changed the topic in the hope it will jog someone's
memory
>
>
>Cheers,
>
>
>Rob
>
>
>From:  Rob Vesse <rvesse@dotnetrdf.org>
>Reply-To:  <user@giraph.apache.org>
>Date:  Friday, 15 November 2013 14:14
>To:  <user@giraph.apache.org>
>Subject:  Re: Giraph and Hadoop 2.2.0
>
>
>
>Adding the –yj argument to specify the YARN JARs doesn't make any difference.
>>
>>
>>Here is the complete console output from my run:
>>
>>
>>bin/hadoop jar /Users/rvesse/Documents/Work/Code/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.2.0-jar-with-dependencies.jar
org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleShortestPathsComputation -vif
org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip /user/rvesse/tiny_graph.txt
-vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op /user/rvesse/shortestpaths/3
-w 1 -yj /Users/rvesse/Documents/Work/Code/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.2.0-jar-with-dependencies.jar
>>2013-11-15 14:11:22.897 java[4877:1203] Unable to load realm info from SCDynamicStore
>>13/11/15 14:11:22 WARN util.NativeCodeLoader: Unable to load native-hadoop library
for your platform... using builtin-java classes where applicable
>>13/11/15 14:11:24 INFO utils.ConfigurationUtils: No edge input format specified. Ensure
your InputFormat does not require one.
>>13/11/15 14:11:24 INFO utils.ConfigurationUtils: No edge output format specified.
Ensure your OutputFormat does not require one.
>>13/11/15 14:11:24 INFO yarn.GiraphYarnClient: Final output path is: hdfs://localhost/user/rvesse/shortestpaths/3
>>13/11/15 14:11:24 INFO yarn.GiraphYarnClient: Running Client
>>13/11/15 14:11:24 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
>>13/11/15 14:11:24 INFO yarn.GiraphYarnClient: Got node report from ASM for, nodeId=mbp-rvesse.home:49981,
nodeAddressmbp-rvesse.home:8042, nodeRackName/default-rack, nodeNumContainers0
>>13/11/15 14:11:24 INFO yarn.GiraphYarnClient: Defaulting per-task heap size to 1024MB.
>>13/11/15 14:11:24 INFO yarn.GiraphYarnClient: Obtained new Application ID: application_1384523910015_0004
>>13/11/15 14:11:24 INFO Configuration.deprecation: mapred.job.id is deprecated. Instead,
use mapreduce.job.id
>>13/11/15 14:11:24 INFO yarn.GiraphYarnClient: Set the environment for the application
master
>>13/11/15 14:11:24 INFO yarn.GiraphYarnClient: Environment for AM :{CLASSPATH=${CLASSPATH}:./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/share/hadoop/common/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*:$HADOOP_YARN_HOME/share/hadoop/yarn/*:$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*,$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*}
>>13/11/15 14:11:24 INFO yarn.GiraphYarnClient: buildLocalResourceMap ....
>>13/11/15 14:11:24 INFO Configuration.deprecation: mapred.output.dir is deprecated.
Instead, use mapreduce.output.fileoutputformat.outputdir
>>13/11/15 14:11:25 INFO yarn.YarnUtils: Registered file in LocalResources :: hdfs://localhost/user/rvesse/giraph_yarn_jar_cache/application_1384523910015_0004/giraph-conf.xml
>>13/11/15 14:11:25 INFO yarn.GiraphYarnClient: LIB JARS :/Users/rvesse/Documents/Work/Code/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.2.0-jar-with-dependencies.jar
>>13/11/15 14:11:25 INFO yarn.YarnUtils: Class path name .
>>13/11/15 14:11:25 INFO yarn.YarnUtils: base path checking .
>>13/11/15 14:11:25 INFO yarn.YarnUtils: Class path name null
>>13/11/15 14:11:25 INFO yarn.YarnUtils: base path checking null
>>13/11/15 14:11:25 INFO yarn.YarnUtils: Class path name /Users/rvesse/Documents/Apps/hadoop-2.2.0/etc/hadoop
>>13/11/15 14:11:25 INFO yarn.YarnUtils: base path checking /Users/rvesse/Documents/Apps/hadoop-2.2.0/etc/hadoop
>>13/11/15 14:11:25 INFO yarn.YarnUtils: Class path name /Users/rvesse/Documents/Apps/hadoop-2.2.0/share/hadoop/common/lib/*
>>13/11/15 14:11:25 INFO yarn.YarnUtils: base path checking /Users/rvesse/Documents/Apps/hadoop-2.2.0/share/hadoop/common/lib
>>13/11/15 14:11:25 INFO yarn.YarnUtils: Class path name /Users/rvesse/Documents/Apps/hadoop-2.2.0/share/hadoop/common/*
>>13/11/15 14:11:25 INFO yarn.YarnUtils: base path checking /Users/rvesse/Documents/Apps/hadoop-2.2.0/share/hadoop/common
>>13/11/15 14:11:25 INFO yarn.YarnUtils: Class path name /Users/rvesse/Documents/Apps/hadoop-2.2.0/share/hadoop/hdfs
>>13/11/15 14:11:25 INFO yarn.YarnUtils: base path checking /Users/rvesse/Documents/Apps/hadoop-2.2.0/share/hadoop/hdfs
>>13/11/15 14:11:25 INFO yarn.YarnUtils: Class path name /Users/rvesse/Documents/Apps/hadoop-2.2.0/share/hadoop/hdfs/lib/*
>>13/11/15 14:11:25 INFO yarn.YarnUtils: base path checking /Users/rvesse/Documents/Apps/hadoop-2.2.0/share/hadoop/hdfs/lib
>>13/11/15 14:11:25 INFO yarn.YarnUtils: Class path name /Users/rvesse/Documents/Apps/hadoop-2.2.0/share/hadoop/hdfs/*
>>13/11/15 14:11:25 INFO yarn.YarnUtils: base path checking /Users/rvesse/Documents/Apps/hadoop-2.2.0/share/hadoop/hdfs
>>13/11/15 14:11:25 INFO yarn.YarnUtils: Class path name /Users/rvesse/Documents/Apps/hadoop-2.2.0/share/hadoop/yarn/lib/*
>>13/11/15 14:11:25 INFO yarn.YarnUtils: base path checking /Users/rvesse/Documents/Apps/hadoop-2.2.0/share/hadoop/yarn/lib
>>13/11/15 14:11:25 INFO yarn.YarnUtils: Class path name /Users/rvesse/Documents/Apps/hadoop-2.2.0/share/hadoop/yarn/*
>>13/11/15 14:11:25 INFO yarn.YarnUtils: base path checking /Users/rvesse/Documents/Apps/hadoop-2.2.0/share/hadoop/yarn
>>13/11/15 14:11:25 INFO yarn.YarnUtils: Class path name /Users/rvesse/Documents/Apps/hadoop-2.2.0/share/hadoop/mapreduce/lib/*
>>13/11/15 14:11:25 INFO yarn.YarnUtils: base path checking /Users/rvesse/Documents/Apps/hadoop-2.2.0/share/hadoop/mapreduce/lib
>>13/11/15 14:11:25 INFO yarn.YarnUtils: Class path name /Users/rvesse/Documents/Apps/hadoop-2.2.0/share/hadoop/mapreduce/*
>>13/11/15 14:11:25 INFO yarn.YarnUtils: base path checking /Users/rvesse/Documents/Apps/hadoop-2.2.0/share/hadoop/mapreduce
>>13/11/15 14:11:25 INFO yarn.YarnUtils: Class path name /contrib/capacity-scheduler/*.jar
>>13/11/15 14:11:25 INFO yarn.YarnUtils: base path checking /contrib/capacity-scheduler
>>13/11/15 14:11:25 INFO yarn.GiraphYarnClient: ApplicationSumbissionContext for GiraphApplicationMaster
launch container is populated.
>>13/11/15 14:11:25 INFO yarn.GiraphYarnClient: Submitting application to ASM
>>13/11/15 14:11:25 INFO impl.YarnClientImpl: Submitted application application_1384523910015_0004
to ResourceManager at /0.0.0.0:8032
>>13/11/15 14:11:25 INFO yarn.GiraphYarnClient: Got new appId after submission :application_1384523910015_0004
>>13/11/15 14:11:25 INFO yarn.GiraphYarnClient: GiraphApplicationMaster container request
was submitted to ResourceManager for job: Giraph: org.apache.giraph.examples.SimpleShortestPathsComputation
>>13/11/15 14:11:26 INFO yarn.GiraphYarnClient: Giraph: org.apache.giraph.examples.SimpleShortestPathsComputation,
Elapsed: 0.82 secs
>>13/11/15 14:11:26 INFO yarn.GiraphYarnClient: appattempt_1384523910015_0004_000001,
State: ACCEPTED, Containers used: 1
>>13/11/15 14:11:29 ERROR yarn.GiraphYarnClient: Giraph: org.apache.giraph.examples.SimpleShortestPathsComputation
reports FAILED state, diagnostics show: Application application_1384523910015_0004 failed
2 times due to AM Container for appattempt_1384523910015_0004_000002 exited with  exitCode:
1 due to: Exception from container-launch: 
>>org.apache.hadoop.util.Shell$ExitCodeException: 
>>at org.apache.hadoop.util.Shell.runCommand(Shell.java:464)
>>at org.apache.hadoop.util.Shell.run(Shell.java:379)
>>at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:589)
>>at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
>>at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:283)
>>at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:79)
>>at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>>at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>>at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
>>at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
>>at java.lang.Thread.run(Thread.java:695)
>>
>>
>>
>>
>>.Failing this attempt.. Failing the application.
>>13/11/15 14:11:29 INFO yarn.GiraphYarnClient: Cleaning up HDFS distributed cache directory
for Giraph job.
>>13/11/15 14:11:29 INFO yarn.GiraphYarnClient: Completed Giraph: org.apache.giraph.examples.SimpleShortestPathsComputation:
FAILED, total running time: 0 minutes, 3 seconds.
>>
>>
>>And I see the same error in the logs:
>>
>>
>>Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/giraph/yarn/GiraphApplicationMaster
Caused by: java.lang.ClassNotFoundException: org.apache.giraph.yarn.GiraphApplicationMaster
at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native
Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>>
>>
>>Any more ideas?
>>
>>
>>Rob
>>
>>
>>From:  Mohammad Islam <mislam77@yahoo.com>
>>Reply-To:  <user@giraph.apache.org>
>>Date:  Thursday, 14 November 2013 20:36
>>To:  "user@giraph.apache.org" <user@giraph.apache.org>
>>Subject:  Re: Giraph and Hadoop 2.2.0
>>
>>
>>
>>I tried the following command few months back. I found you are not passing some params.
>>>
>>>
>>>bin/hadoop --config etc/hadoop jar /export/home/clei/giraph/giraph-1.0.0/giraph-examples/target/giraph-examples-1.?.0-for-hadoop-???-jar-with-dependencies.jar
org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimplePageRankVertexOpt -c org.apache.giraph.combiner.DoubleSumCombiner
-w 35 -yh 20480 -yj giraph-examples-1.?.?-for-hadoop-2.??-jar-with-dependencies.jar -vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
-of org.apache.giraph.io.formats.IdWithValueTextOutputFormat -vip /user/me/input -op /user/me/output_35w_20G_80m_1
>>>
>>>
>>>
>>>
>>>On Thursday, November 14, 2013 3:22 AM, Rob Vesse <rvesse@dotnetrdf.org>
wrote:
>>> 
>>>It's the same command line I showed earlier, just trying to run the example application:
>>>
>>>
>>>mbp-rvesse:hadoop-2.2.0 rvesse$ bin/hadoop jar /Users/rvesse/Documents/Work/Code/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.2.0-jar-with-dependencies.jar
org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleShortestPathsComputation -vif
org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip /user/rvesse/tiny_graph.txt
-vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op /user/rvesse/shortestpaths/2
-w 1
>>>
>>>
>>>Rob
>>>
>>>
>>>From:  Mohammad Islam <mislam77@yahoo.com>
>>>Reply-To:  <user@giraph.apache.org>
>>>Date:  Thursday, 14 November 2013 05:57
>>>To:  "user@giraph.apache.org" <user@giraph.apache.org>
>>>Subject:  Re: Giraph and Hadoop 2.2.0
>>>
>>>
>>>
>>>What are you running? PR?
>>>>What is the command line?
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>On Wednesday, November 13, 2013 8:30 AM, Rob Vesse <rvesse@dotnetrdf.org>
wrote:
>>>> 
>>>>So I fixed the previous issue I was having which I reported and submitted
a patch as GIRAPH-795
>>>>
>>>>
>>>>I then went ahead with installing and configuring my external Zookeeper appropriately
and tried running again but now the YARN application fails and I find the following error
in the logs:
>>>>
>>>>
>>>>Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/giraph/yarn/GiraphApplicationMaster
Caused by: java.lang.ClassNotFoundException: org.apache.giraph.yarn.GiraphApplicationMaster
at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native
Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>>>>
>>>>
>>>>Anybody have idea what the cause of this one might be or how to fix it?
>>>>
>>>>
>>>>Cheers,
>>>>
>>>>
>>>>Rob
>>>>
>>>>
>>>>From:  Rob Vesse <rvesse@dotnetrdf.org>
>>>>Reply-To:  <user@giraph.apache.org>
>>>>Date:  Wednesday, 13 November 2013 15:35
>>>>To:  <user@giraph.apache.org>
>>>>Subject:  Re: Giraph and Hadoop 2.2.0
>>>>
>>>>
>>>>
>>>>Mohammad
>>>>>
>>>>>
>>>>>This got me a build that at least attempts to start to run but not I see
the following:
>>>>>
>>>>>
>>>>>mbp-rvesse:hadoop-2.2.0 rvesse$ bin/hadoop jar /Users/rvesse/Documents/Work/Code/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.2.0-jar-with-dependencies.jar
org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleShortestPathsComputation -vif
org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip /user/rvesse/tiny_graph.txt
-vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op /user/rvesse/shortestpaths/2
-w 1
>>>>>2013-11-13 15:13:29.908 java[17402:1203] Unable to load realm info from
SCDynamicStore
>>>>>13/11/13 15:13:29 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
>>>>>13/11/13 15:13:31 INFO utils.ConfigurationUtils: No edge input format
specified. Ensure your InputFormat does not require one.
>>>>>13/11/13 15:13:31 INFO utils.ConfigurationUtils: No edge output format
specified. Ensure your OutputFormat does not require one.
>>>>>13/11/13 15:13:31 INFO yarn.GiraphYarnClient: Final output path is: hdfs://localhost/user/rvesse/shortestpaths/2
>>>>>Exception in thread "main" java.lang.NullPointerException
>>>>>at org.apache.giraph.yarn.GiraphYarnClient.checkJobLocalZooKeeperSupported(GiraphYarnClient.java:460)
>>>>>at org.apache.giraph.yarn.GiraphYarnClient.run(GiraphYarnClient.java:116)
>>>>>at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:96)
>>>>>at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>>>>>at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>>>>>at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:126)
>>>>>at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>>at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>>at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
>>>>>
>>>>>
>>>>>Which indirectly implies that I have some missing configuration to run
Giraph on YARN.  I don't think that should be a NPE and it looks to be a trivial fix so I'll
try and fix and send a pull request for that.
>>>>>
>>>>>
>>>>>The error message I should receive if the NPE wasn't happening is the
following:
>>>>>
>>>>>
>>>>>Giraph on YARN does not currentlysupport Giraph-managed ZK instances:
use a standalone ZooKeeper.
>>>>>
>>>>>
>>>>>What configuration options do I need to have set to point Giraph at an
external Zookeeper?
>>>>>
>>>>>
>>>>>Cheers,
>>>>>
>>>>>
>>>>>Rob
>>>>>
>>>>>
>>>>>From:  Mohammad Islam <mislam77@yahoo.com>
>>>>>Reply-To:  <user@giraph.apache.org>
>>>>>Date:  Wednesday, 13 November 2013 11:15
>>>>>To:  "user@giraph.apache.org" <user@giraph.apache.org>
>>>>>Subject:  Re: Giraph and Hadoop 2.2.0
>>>>>
>>>>>
>>>>>
>>>>>Hi Rob,
>>>>>>Did you try with this command "mvn -Phadoop_yarn -Dhadoop.version=2.2.0
clean package -DskipTests"?
>>>>>>Regards,
>>>>>>Mohammad
>>>>>>
>>>>>>
>>>>>>
>>>>>>On Wednesday, November 13, 2013 2:55 AM, Rob Vesse <rvesse@dotnetrdf.org>
wrote:
>>>>>> 
>>>>>>Hi all
>>>>>>
>>>>>>
>>>>>>I'm having trouble getting Giraph to build and run against a Hadoop
2.2.0 installation.  I tried the hadoop_2.0.3 profile with no luck because I get class cast
errors when trying to run the examples.
>>>>>>
>>>>>>
>>>>>>Is there a specific profile I should be building to support Hadoop
2.2.0 or is this not supported yet?
>>>>>>
>>>>>>
>>>>>>Thanks,
>>>>>>
>>>>>>
>>>>>>Rob
>>>>>>
>>>>>>
>>>>
>>>>
>>>
>>>
>
>
Mime
View raw message