giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rob Vesse <rve...@dotnetrdf.org>
Subject Re: Giraph and Hadoop 2.2.0
Date Thu, 14 Nov 2013 11:21:20 GMT
It's the same command line I showed earlier, just trying to run the example
application:

mbp-rvesse:hadoop-2.2.0 rvesse$ bin/hadoop jar
/Users/rvesse/Documents/Work/Code/giraph/giraph-examples/target/giraph-examp
les-1.1.0-SNAPSHOT-for-hadoop-2.2.0-jar-with-dependencies.jar
org.apache.giraph.GiraphRunner
org.apache.giraph.examples.SimpleShortestPathsComputation -vif
org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip
/user/rvesse/tiny_graph.txt -vof
org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
/user/rvesse/shortestpaths/2 -w 1

Rob

From:  Mohammad Islam <mislam77@yahoo.com>
Reply-To:  <user@giraph.apache.org>
Date:  Thursday, 14 November 2013 05:57
To:  "user@giraph.apache.org" <user@giraph.apache.org>
Subject:  Re: Giraph and Hadoop 2.2.0

> What are you running? PR?
> What is the command line?
> 
>  
>  
>  
>  
>  
>   On Wednesday, November 13, 2013 8:30 AM, Rob Vesse <rvesse@dotnetrdf.org>
> wrote:
>   
>   
> So I fixed the previous issue I was having which I reported and submitted a
> patch as GIRAPH-795
> 
> I then went ahead with installing and configuring my external Zookeeper
> appropriately and tried running again but now the YARN application fails and I
> find the following error in the logs:
> 
> Exception in thread "main" java.lang.NoClassDefFoundError:
>  org/apache/giraph/yarn/GiraphApplicationMaster
> Caused by: java.lang.ClassNotFoundException:
> org.apache.giraph.yarn.GiraphApplicationMaster
> 	at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> 	at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> 	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> 	at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> 
> Anybody have idea what the cause of this one might be or how to fix it?
> 
> Cheers,
> 
> Rob
> 
> From:  Rob Vesse <rvesse@dotnetrdf.org>
> Reply-To:  <user@giraph.apache.org>
> Date:  Wednesday, 13 November 2013 15:35
> To:  <user@giraph.apache.org>
> Subject:  Re: Giraph and Hadoop 2.2.0
> 
>> Mohammad
>> 
>> This got me a build that at least attempts to start to run but not I see the
>> following:
>> 
>> mbp-rvesse:hadoop-2.2.0 rvesse$ bin/hadoop jar
>> /Users/rvesse/Documents/Work/Code/giraph/giraph-examples/target/giraph-exampl
>> es-1.1.0-SNAPSHOT-for-hadoop-2.2.0-jar-with-dependencies.jar
>> org.apache.giraph.GiraphRunner
>> org.apache.giraph.examples.SimpleShortestPathsComputation -vif
>> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip
>> /user/rvesse/tiny_graph.txt -vof
>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
>> /user/rvesse/shortestpaths/2 -w 1
>> 2013-11-13 15:13:29.908 java[17402:1203] Unable to load realm info from
>> SCDynamicStore
>> 13/11/13 15:13:29 WARN util.NativeCodeLoader: Unable to load native-hadoop
>> library for your platform... using builtin-java classes where applicable
>> 13/11/13 15:13:31 INFO utils.ConfigurationUtils: No edge input format
>> specified. Ensure your InputFormat does not require one.
>> 13/11/13 15:13:31 INFO utils.ConfigurationUtils: No edge output format
>> specified. Ensure your OutputFormat does not require one.
>> 13/11/13 15:13:31 INFO yarn.GiraphYarnClient: Final output path is:
>> hdfs://localhost/user/rvesse/shortestpaths/2
>> Exception in thread "main" java.lang.NullPointerException
>> at 
>> org.apache.giraph.yarn.GiraphYarnClient.checkJobLocalZooKeeperSupported(Girap
>> hYarnClient.java:460)
>> at org.apache.giraph.yarn.GiraphYarnClient.run(GiraphYarnClient.java:116)
>> at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:96)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>> at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:126)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at 
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>> at 
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.
>> java:25)
>> at java.lang.reflect.Method.invoke(Method.java:597)
>> at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
>> 
>> Which indirectly implies that I have some missing configuration to run Giraph
>> on YARN.  I don't think that should be a NPE and it looks to be a trivial fix
>> so I'll try and fix and send a pull request for that.
>> 
>> The error message I should receive if the NPE wasn't happening is the
>> following:
>> 
>> Giraph on YARN does not currentlysupport Giraph-managed ZK instances: use a
>> standalone ZooKeeper.
>> 
>> What configuration options do I need to have set to point Giraph at an
>> external Zookeeper?
>> 
>> Cheers,
>> 
>> Rob
>> 
>> From:  Mohammad Islam <mislam77@yahoo.com>
>> Reply-To:  <user@giraph.apache.org>
>> Date:  Wednesday, 13 November 2013 11:15
>> To:  "user@giraph.apache.org" <user@giraph.apache.org>
>> Subject:  Re: Giraph and Hadoop 2.2.0
>> 
>>> Hi Rob,
>>> Did you try with this command "mvn -Phadoop_yarn -Dhadoop.version=2.2.0
>>> clean package -DskipTests"?
>>> Regards,
>>> Mohammad
>>>  
>>>  
>>>  
>>>  
>>>  
>>>   On Wednesday, November 13, 2013 2:55 AM, Rob Vesse <rvesse@dotnetrdf.org>
>>> wrote:
>>>   
>>>   
>>> Hi all
>>> 
>>> I'm having trouble getting Giraph to build and run against a Hadoop 2.2.0
>>> installation.  I tried the hadoop_2.0.3 profile with no luck because I get
>>> class cast errors when trying to run the examples.
>>> 
>>> Is there a specific profile I should be building to support Hadoop 2.2.0 or
>>> is this not supported yet?
>>> 
>>> Thanks,
>>> 
>>> Rob
>>> 
>>> 
>>>   
>>>  
>>>   
>>>  
> 
> 
>   
>  
>   
>  



Mime
View raw message