giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mohammad Islam <misla...@yahoo.com>
Subject Re: Giraph and Hadoop 2.2.0
Date Thu, 14 Nov 2013 05:57:47 GMT
What are you running? PR?
What is the command line?




On Wednesday, November 13, 2013 8:30 AM, Rob Vesse <rvesse@dotnetrdf.org> wrote:
 
So I fixed the previous issue I was having which I reported and submitted a patch as GIRAPH-795

I then went ahead with installing and configuring my external Zookeeper appropriately and
tried running again but now the YARN application fails and I find the following error in the
logs:

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/giraph/yarn/GiraphApplicationMaster
Caused by: java.lang.ClassNotFoundException: org.apache.giraph.yarn.GiraphApplicationMaster
at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native
Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:247)

Anybody have idea what the cause of this one might be or how to fix it?

Cheers,

Rob

From:  Rob Vesse <rvesse@dotnetrdf.org>
Reply-To:  <user@giraph.apache.org>
Date:  Wednesday, 13 November 2013 15:35
To:  <user@giraph.apache.org>
Subject:  Re: Giraph and Hadoop 2.2.0


Mohammad
>
>
>This got me a build that at least attempts to start to run but not I see the following:
>
>
>mbp-rvesse:hadoop-2.2.0 rvesse$ bin/hadoop jar /Users/rvesse/Documents/Work/Code/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.2.0-jar-with-dependencies.jar
org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleShortestPathsComputation -vif
org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip /user/rvesse/tiny_graph.txt
-vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op /user/rvesse/shortestpaths/2
-w 1
>2013-11-13 15:13:29.908 java[17402:1203] Unable to load realm info from SCDynamicStore
>13/11/13 15:13:29 WARN util.NativeCodeLoader: Unable to load native-hadoop library for
your platform... using builtin-java classes where applicable
>13/11/13 15:13:31 INFO utils.ConfigurationUtils: No edge input format specified. Ensure
your InputFormat does not require one.
>13/11/13 15:13:31 INFO utils.ConfigurationUtils: No edge output format specified. Ensure
your OutputFormat does not require one.
>13/11/13 15:13:31 INFO yarn.GiraphYarnClient: Final output path is: hdfs://localhost/user/rvesse/shortestpaths/2
>Exception in thread "main" java.lang.NullPointerException
>at org.apache.giraph.yarn.GiraphYarnClient.checkJobLocalZooKeeperSupported(GiraphYarnClient.java:460)
>at org.apache.giraph.yarn.GiraphYarnClient.run(GiraphYarnClient.java:116)
>at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:96)
>at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:126)
>at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>at java.lang.reflect.Method.invoke(Method.java:597)
>at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
>
>
>Which indirectly implies that I have some missing configuration to run Giraph on YARN.
 I don't think that should be a NPE and it looks to be a trivial fix so I'll try and fix
and send a pull request for that.
>
>
>The error message I should receive if the NPE wasn't happening is the following:
>
>
>Giraph on YARN does not currentlysupport Giraph-managed ZK instances: use a standalone
ZooKeeper.
>
>
>What configuration options do I need to have set to point Giraph at an external Zookeeper?
>
>
>Cheers,
>
>
>Rob
>
>
>From:  Mohammad Islam <mislam77@yahoo.com>
>Reply-To:  <user@giraph.apache.org>
>Date:  Wednesday, 13 November 2013 11:15
>To:  "user@giraph.apache.org" <user@giraph.apache.org>
>Subject:  Re: Giraph and Hadoop 2.2.0
>
>
>
>Hi Rob,
>>Did you try with this command "mvn -Phadoop_yarn -Dhadoop.version=2.2.0 clean package
-DskipTests"?
>>Regards,
>>Mohammad
>>
>>
>>
>>On Wednesday, November 13, 2013 2:55 AM, Rob Vesse <rvesse@dotnetrdf.org> wrote:
>> 
>>Hi all
>>
>>
>>I'm having trouble getting Giraph to build and run against a Hadoop 2.2.0 installation.
 I tried the hadoop_2.0.3 profile with no luck because I get class cast errors when trying
to run the examples.
>>
>>
>>Is there a specific profile I should be building to support Hadoop 2.2.0 or is this
not supported yet?
>>
>>
>>Thanks,
>>
>>
>>Rob
>>
>>
Mime
View raw message