giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nicholas Karkoulias <nicholas...@gmail.com>
Subject GiraphApplicationMaster not found (and other newbie questions)
Date Sun, 07 Jul 2013 14:29:13 GMT
Hello everyone.

This is my first message to an Apache mailing list, so please excuse 
(and correct) possibly incorrect usage or netiquette issues :)

I am a newbie to both Hadoop and Giraph (but not to Linux). After 
overcoming _several_ configuration-related hurdles, I have successfully 
built both (versions Hadoop 2.0.5-alpha and Giraph 1.1.0-SNAPSHOT, using 
the patch for issue #688), and seem to have a properly working 
HDFS/Hadoop installation in pseudo-distributed mode. I can run Hadoop 
code just fine.

However, Giraph fails during the computation (seemingly just before 
returning or writing the result – the time elapsed before the crash 
differs, depending on which example I try to run). See below for the 
error. I don't know whether it is a bug, or me doing something wrong.

I'm using the YARN-enabled version of Giraph (and, thus, an external 
ZooKeeper service), assuming that Giraph will completely move to YARN 
_eventually_. (Is that correct?)

Also, two other (somewhat unrelated) questions:
(1) When is the file conf/giraph-site.xml actually parsed/used? Is it 
read by Hadoop? Should it be copied somewhere? I tried setting the 
ZooKeeper host:port in that file (giraph.zkList), but it was ignored and 
I finally had to add the property in the command line shown below... Any 
relevant documentation? (In general, documentation for certain Hadoop 
features, such as the configuration files, seems to be lacking...)
(2) What's the correct process to submit a patch with a really simple 
typo correction in a string? (perhaps I should just contact the file 
author – it's nothing important)

I'll append the shell commands I used (after $) and the output at the 
end of this message.

Thank you in advance,
Nicholas


$ function giraphrunner(){ hadoop jar 
/tmp/software/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.0.5-alpha-jar-with-dependencies.jar

org.apache.giraph.GiraphRunner -Dgiraph.zkList=localhost:2181 "$@"; }

$ time giraphrunner org.apache.giraph.examples.SimplePageRankComputation 
-vif 
org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat 
-vip /dir/tiny_graph.txt -of 
org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op 
/dir/simplepagerank -w 1
13/07/07 15:06:15 INFO utils.ConfigurationUtils: No edge input format 
specified. Ensure your InputFormat does not require one.
13/07/07 15:06:15 INFO yarn.GiraphYarnClient: Final output path is: 
hdfs://localhost:9000/dir/simplepagerank
13/07/07 15:06:15 INFO service.AbstractService: 
Service:org.apache.hadoop.yarn.client.YarnClientImpl is inited.
13/07/07 15:06:15 INFO service.AbstractService: 
Service:org.apache.hadoop.yarn.client.YarnClientImpl is started.
13/07/07 15:06:15 INFO yarn.GiraphYarnClient: Defaulting per-task heap 
size to 1024MB.
13/07/07 15:06:15 INFO yarn.GiraphYarnClient: Obtained new Application 
ID: application_1372875746593_0018
13/07/07 15:06:15 WARN conf.Configuration: mapred.job.id is deprecated. 
Instead, use mapreduce.job.id
13/07/07 15:06:15 WARN conf.Configuration: mapred.output.dir is 
deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
13/07/07 15:06:15 INFO yarn.YarnUtils: Registered file in 
LocalResources: giraph-conf.xml
13/07/07 15:06:16 INFO yarn.GiraphYarnClient: 
ApplicationSumbissionContext for GiraphApplicationMaster launch 
container is populated.
13/07/07 15:06:16 INFO client.YarnClientImpl: Submitted application 
application_1372875746593_0018 to ResourceManager at 
localhost/127.0.0.1:8032
13/07/07 15:06:16 INFO yarn.GiraphYarnClient: GiraphApplicationMaster 
container request was submitted to ResourceManager for job: Giraph: 
org.apache.giraph.examples.SimplePageRankComputation
13/07/07 15:06:17 INFO yarn.GiraphYarnClient: Giraph: 
org.apache.giraph.examples.SimplePageRankComputation, Elapsed: 0.85 secs
13/07/07 15:06:17 INFO yarn.GiraphYarnClient: 
appattempt_1372875746593_0018_000001, State: ACCEPTED, Containers used: 1
13/07/07 15:06:18 ERROR yarn.GiraphYarnClient: Giraph: 
org.apache.giraph.examples.SimplePageRankComputation reports FAILED 
state, diagnostics show: Application application_1372875746593_0018 
failed 1 times due to AM Container for 
appattempt_1372875746593_0018_000001 exited with  exitCode: 1 due to:
.Failing this attempt.. Failing the application.
13/07/07 15:06:18 INFO yarn.GiraphYarnClient: Cleaning up HDFS 
distributed cache directory for Giraph job.
13/07/07 15:06:18 INFO yarn.GiraphYarnClient: Completed Giraph: 
org.apache.giraph.examples.SimplePageRankComputation: FAILED, total 
running time: 0 minutes, 1 seconds.

real    0m8.392s
user    0m8.825s
sys     0m1.492s

$ cat 
software/hadoop-2.0.5-alpha/logs/userlogs/application_1372875746593_0018/container_1372875746593_0018_01_000001/gam-stderr.log
Exception in thread "main" java.lang.NoClassDefFoundError: 
org/apache/giraph/yarn/GiraphApplicationMaster
Caused by: java.lang.ClassNotFoundException: 
org.apache.giraph.yarn.GiraphApplicationMaster
         at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
         at java.security.AccessController.doPrivileged(Native Method)
         at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
         at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
         at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
Could not find the main class: 
org.apache.giraph.yarn.GiraphApplicationMaster. Program will exit.


Mime
View raw message