Return-Path: X-Original-To: apmail-giraph-user-archive@www.apache.org Delivered-To: apmail-giraph-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 3006BC723 for ; Tue, 12 Aug 2014 07:00:08 +0000 (UTC) Received: (qmail 26135 invoked by uid 500); 12 Aug 2014 07:00:07 -0000 Delivered-To: apmail-giraph-user-archive@giraph.apache.org Received: (qmail 26081 invoked by uid 500); 12 Aug 2014 07:00:07 -0000 Mailing-List: contact user-help@giraph.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@giraph.apache.org Delivered-To: mailing list user@giraph.apache.org Received: (qmail 26071 invoked by uid 99); 12 Aug 2014 07:00:07 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 12 Aug 2014 07:00:07 +0000 X-ASF-Spam-Status: No, hits=2.5 required=5.0 tests=FREEMAIL_ENVFROM_END_DIGIT,HTML_MESSAGE,NORMAL_HTTP_TO_IP,RCVD_IN_DNSWL_NONE,SPF_PASS,WEIRD_PORT X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of sirotin8@web.de designates 212.227.17.11 as permitted sender) Received: from [212.227.17.11] (HELO mout.web.de) (212.227.17.11) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 12 Aug 2014 06:59:41 +0000 Received: from [192.168.178.20] ([37.5.81.102]) by smtp.web.de (mrweb101) with ESMTPSA (Nemesis) id 0MKJ7S-1XIkTy1TZW-001ieK for ; Tue, 12 Aug 2014 08:59:40 +0200 Message-ID: <53E9BB5A.3000503@web.de> Date: Tue, 12 Aug 2014 08:59:38 +0200 From: Alexander Sirotin User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.6.0 MIME-Version: 1.0 To: user@giraph.apache.org Subject: Re: ShortestPath Code execution on Hadoop 2.4.0 Inbox x References: <29481680.367541407822502278.JavaMail.weblogic@epv6ml16> In-Reply-To: Content-Type: multipart/alternative; boundary="------------020806040204010402070704" X-Provags-ID: V03:K0:VKDgxHjQBSxKuac6MKi/g1e1EqgXOlqDZSWq8LQsWi+5rymwrLP AA6pvLxw5ZOGAlEOulu6gxsvBZEv/I6NsyQQ6EWV7nnkKxTFXUBF+NNbNM1AdHJU/CvU+k6 Ub8OnTRE7TuKqowHXxfhSQK27YyWEAc27RbwrFu9c6mmpUzERetL1BV6OT9V5Wa8xAOuXtA 9JZ1N0F5lti6TZthVciCA== X-UI-Out-Filterresults: notjunk:1; X-Virus-Checked: Checked by ClamAV on apache.org This is a multi-part message in MIME format. --------------020806040204010402070704 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Hi, you can increase the Heapsize to 2048 Mb by setting it into etc/hadoop/hadoop-env.sh. Search for "export HADOOP_HEAPSIZE=" But if I fix this, the TaskRunner and GiraphApplicationMaster will not stop to end. This is what I am seeing from the Client everytime repeating: 14/08/12 08:51:06 INFO yarn.GiraphYarnClient: Giraph: org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: 1494,96 secs 14/08/12 08:51:06 INFO yarn.GiraphYarnClient: appattempt_1407762054368_0017_000001, State: RUNNING, Containers used: 2 The gam-stderr.log (GiraphApplicationMaster) showed the following: aphYarnTask 1407762054368 17 2 1 |- 13562 11012 13562 13562 (bash) 0 0 18870272 435 /bin/bash -c java -Xmx1024M -Xms1024M -cp .:${CLASSPATH}:./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/share/hadoop/common/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*:$HADOOP_YARN_HOME/share/hadoop/yarn/*:$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*:./*:/usr/local/hadoop/etc/hadoop:/usr/local/hadoop/share/hadoop/common/*:/usr/local/hadoop/share/hadoop/common/lib/*:/usr/local/hadoop/share/hadoop/hdfs/*:/usr/local/hadoop/share/hadoop/hdfs/lib/*:/usr/local/hadoop/share/hadoop/yarn/*:/usr/local/hadoop/share/hadoop/yarn/lib/*:/usr/local/hadoop/share/hadoop/mapreduce/*:/usr/local/hadoop/share/hadoop/mapreduce/lib/* org.apache.giraph.yarn.GiraphYarnTask 1407762054368 17 2 1 1>/usr/local/hadoop/logs/userlogs/application_1407762054368_0017/container_1407762054368_0017_01_000002/task-2-stdout.log 2>/usr/local/hadoop/logs/userlogs/application_1407762054368_0017/container_1407762054368_0017_01_000002/task-2-stderr.log Container killed on request. Exit code is 143 Container exited with a non-zero exit code 143 14/08/12 08:26:23 INFO yarn.GiraphApplicationMaster: After completion of one conatiner. current status is: completedCount :1 containersToLaunch :2 successfulCount :0 failedCount :1 And the GiraphYarnTask also repeats the following: 14/08/12 08:56:15 WARN zookeeper.ClientCnxn: Session 0x147c8e553a10001 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Verbindungsaufbau abgelehnt at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068) 14/08/12 08:56:16 INFO zookeeper.ClientCnxn: Opening socket connection to server blabla:22181. Will not attempt to authenticate using SASL (unknown error) 14/08/12 08:56:16 WARN zookeeper.ClientCnxn: Session 0x147c8e553a10001 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Verbindungsaufbau abgelehnt at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068) Do you have an idea, how I can fix this? With ZooKeeper, I am also newbie Thx On 12.08.2014 08:14, Vikalp Handa wrote: > Thanks a lot... I have build successfully even without running mvn > clean :P > Now while executing the example code I am facing a different problem > of Cluster Heap Size. > > *How can I update this cluster heap size?* > > *Command: *hadoop jar > /usr/local/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.4.0-jar-with-dependencies.jar > org.apache.giraph.GiraphRunner > org.apache.giraph.examples.SimpleShortestPathsComputation -vif > org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip > /user/hadoop/input/tiny_graph.txt -vof > org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op > /user/hadoop/outputpaths -yh 2048 -w 1 > > *Result:* > * > * > 14/08/12 11:37:15 WARN util.NativeCodeLoader: Unable to load > native-hadoop library for your platform... using builtin-java classes > where applicable > 14/08/12 11:37:15 INFO utils.ConfigurationUtils: No edge input format > specified. Ensure your InputFormat does not require one. > 14/08/12 11:37:15 INFO utils.ConfigurationUtils: No edge output format > specified. Ensure your OutputFormat does not require one. > 14/08/12 11:37:15 INFO yarn.GiraphYarnClient: Final output path is: > hdfs://localhost:9000/user/hadoop/outputpaths > 14/08/12 11:37:15 INFO yarn.GiraphYarnClient: Running Client > 14/08/12 11:37:15 INFO client.RMProxy: Connecting to ResourceManager > at /0.0.0.0:8032 > Exception in thread "main" java.lang.IllegalStateException: Giraph's > estimated cluster heap 4096MB ask is greater than the current > available cluster heap of 0MB. Aborting Job. > at > org.apache.giraph.yarn.GiraphYarnClient.checkPerNodeResourcesAvailable(GiraphYarnClient.java:230) > at > org.apache.giraph.yarn.GiraphYarnClient.run(GiraphYarnClient.java:124) > at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:96) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) > at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:126) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:483) > at org.apache.hadoop.util.RunJar.main(RunJar.java:212) > > > Regards, > Vikalp Handa > > > On Tue, Aug 12, 2014 at 11:18 AM, 이성일 > wrote: > > You just run 'mvn clean' before that command :) > > Best Regards, Seong-il > ------- Original Message ------- > Sender : Vikalp Handa > > Date : 2014-08-12 14:35 (GMT+09:00) > Title : Re: ShortestPath Code execution on Hadoop 2.4.0 Inbox x > > Thanks for your replies. Do I need to uninstall anything before > running the above mvn command ? > > Regards, > Vikalp Handa > > On Tue, Aug 12, 2014 at 10:26 AM, 이성일 > wrote: > Hi, Vikalp > I think you complied as wrong hadoop version. > Try below command > If you use yarn > mvn -Phadoop_yarn -Dhadoop.version=2.4.0 -DskipTests package > or if you use just hadoop2 > mvn -Phadoop_2 -Dhadoop.version=2.4.0 -DskipTests package > Best Regards, Seong-il > > ------- Original Message ------- > Sender : Vikalp Handa > > Date : 2014-08-12 13:40 (GMT+09:00) > Title : ShortestPath Code execution on Hadoop 2.4.0 Inbox x > > Hi everyone, > I am new to Apache Giraph and would like execute Shortestpath and > PageRank example code on Hadoop 2.4.0 single node cluster (my > machine) having Centos 6.5. > I have successfully build Giraph on my machine but unable to > execute ShortestPath code. > Please let me know if there are any dependencies to be resolved > before code execution. > > P.S. : > Command used: > hadoop jar > /usr/local/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-1.2.1-jar-with-dependencies.jar > org.apache.giraph.GiraphRunner > org.apache.giraph.examples.SimpleShortestPathsComputation -vif > org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat > -vip /user/hadoop/input/tiny_graph.txt -vof > org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op > /user/hduser/output/shortestpaths -w 1 -ca > giraph.SplitMasterWorker=false > > Execution Result: > 14/08/11 18:48:37 WARN util.NativeCodeLoader: Unable to load > native-hadoop library for your platform... using builtin-java > classes where applicable > 14/08/11 18:48:40 INFO utils.ConfigurationUtils: No edge input > format specified. Ensure your InputFormat does not require one. > 14/08/11 18:48:40 INFO utils.ConfigurationUtils: No edge output > format specified. Ensure your OutputFormat does not require one. > 14/08/11 18:48:40 INFO utils.ConfigurationUtils: Setting custom > argument [giraph.SplitMasterWorker] to [false] in GiraphConfiguration > 14/08/11 18:48:40 INFO Configuration.deprecation: > mapreduce.job.counters.limit is deprecated. Instead, use > mapreduce.job.counters.max > 14/08/11 18:48:40 INFO Configuration.deprecation: > mapred.job.map.memory.mb is deprecated. Instead, use > mapreduce.map.memory.mb > 14/08/11 18:48:40 INFO Configuration.deprecation: > mapred.job.reduce.memory.mb is deprecated. Instead, use > mapreduce.reduce.memory.mb > 14/08/11 18:48:40 INFO Configuration.deprecation: > mapred.map.tasks.speculative.execution is deprecated. Instead, use > mapreduce.map.speculative > 14/08/11 18:48:40 INFO Configuration.deprecation: > mapreduce.user.classpath.first is deprecated. Instead, use > mapreduce.job.user.classpath.first > 14/08/11 18:48:40 INFO Configuration.deprecation: > mapred.map.max.attempts is deprecated. Instead, use > mapreduce.map.maxattempts > 14/08/11 18:48:40 INFO job.GiraphJob: run: Since checkpointing is > disabled (default), do not allow any task retries (setting > mapred.map.max.attempts = 0, old value = 4) > 14/08/11 18:48:40 INFO Configuration.deprecation: > mapred.job.tracker is deprecated. Instead, use > mapreduce.jobtracker.address > 14/08/11 18:48:41 INFO Configuration.deprecation: session.id > is deprecated. Instead, use dfs.metrics.session-id > 14/08/11 18:48:41 INFO jvm.JvmMetrics: Initializing JVM Metrics > with processName=JobTracker, sessionId= > Exception in thread "main" java.lang.IncompatibleClassChangeError: > Found interface org.apache.hadoop.mapreduce.JobContext, but class > was expected > at > org.apache.giraph.bsp.BspOutputFormat.checkOutputSpecs(BspOutputFormat.java:44) > at > org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:458) > at > org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:343) > at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285) > at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548) > at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282) > at org.apache.giraph.job.GiraphJob.run(GiraphJob.java:250) > at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:94) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) > at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:483) > at org.apache.hadoop.util.RunJar.main(RunJar.java:212) > Regards, > Vikalp Handa > > --------------020806040204010402070704 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit
Hi,

you can increase the Heapsize to 2048 Mb by setting it into etc/hadoop/hadoop-env.sh. Search for "export HADOOP_HEAPSIZE="

But if I fix this, the TaskRunner and GiraphApplicationMaster will not stop to end. This is what I am seeing from the Client everytime repeating:

14/08/12 08:51:06 INFO yarn.GiraphYarnClient: Giraph: org.apache.giraph.examples.SimpleShortestPathsComputation, Elapsed: 1494,96 secs
14/08/12 08:51:06 INFO yarn.GiraphYarnClient: appattempt_1407762054368_0017_000001, State: RUNNING, Containers used: 2

The gam-stderr.log (GiraphApplicationMaster) showed the following:

aphYarnTask 1407762054368 17 2 1
        |- 13562 11012 13562 13562 (bash) 0 0 18870272 435 /bin/bash -c java -Xmx1024M -Xms1024M -cp .:${CLASSPATH}:./*:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/share/hadoop/common/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*:$HADOOP_YARN_HOME/share/hadoop/yarn/*:$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*:./*:/usr/local/hadoop/etc/hadoop:/usr/local/hadoop/share/hadoop/common/*:/usr/local/hadoop/share/hadoop/common/lib/*:/usr/local/hadoop/share/hadoop/hdfs/*:/usr/local/hadoop/share/hadoop/hdfs/lib/*:/usr/local/hadoop/share/hadoop/yarn/*:/usr/local/hadoop/share/hadoop/yarn/lib/*:/usr/local/hadoop/share/hadoop/mapreduce/*:/usr/local/hadoop/share/hadoop/mapreduce/lib/* org.apache.giraph.yarn.GiraphYarnTask 1407762054368 17 2 1 1>/usr/local/hadoop/logs/userlogs/application_1407762054368_0017/container_1407762054368_0017_01_000002/task-2-stdout.log 2>/usr/local/hadoop/logs/userlogs/application_1407762054368_0017/container_1407762054368_0017_01_000002/task-2-stderr.log

Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143

14/08/12 08:26:23 INFO yarn.GiraphApplicationMaster: After completion of one conatiner. current status is: completedCount :1 containersToLaunch :2 successfulCount :0 failedCount :1


And the GiraphYarnTask also repeats the following:

14/08/12 08:56:15 WARN zookeeper.ClientCnxn: Session 0x147c8e553a10001 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Verbindungsaufbau abgelehnt
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
14/08/12 08:56:16 INFO zookeeper.ClientCnxn: Opening socket connection to server blabla:22181. Will not attempt to authenticate using SASL (unknown error)
14/08/12 08:56:16 WARN zookeeper.ClientCnxn: Session 0x147c8e553a10001 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Verbindungsaufbau abgelehnt
        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
        at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)

Do you have an idea, how I can fix this? With ZooKeeper, I am also newbie

Thx




On 12.08.2014 08:14, Vikalp Handa wrote:
Thanks a lot... I have build successfully even without running mvn clean :P
Now while executing the example code I am facing a different problem of Cluster Heap Size.

How can I update this cluster heap size?

Command: hadoop jar /usr/local/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-2.4.0-jar-with-dependencies.jar org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleShortestPathsComputation -vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip /user/hadoop/input/tiny_graph.txt -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op /user/hadoop/outputpaths -yh 2048 -w 1

Result:

14/08/12 11:37:15 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
14/08/12 11:37:15 INFO utils.ConfigurationUtils: No edge input format specified. Ensure your InputFormat does not require one.
14/08/12 11:37:15 INFO utils.ConfigurationUtils: No edge output format specified. Ensure your OutputFormat does not require one.
14/08/12 11:37:15 INFO yarn.GiraphYarnClient: Final output path is: hdfs://localhost:9000/user/hadoop/outputpaths
14/08/12 11:37:15 INFO yarn.GiraphYarnClient: Running Client
14/08/12 11:37:15 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
Exception in thread "main" java.lang.IllegalStateException: Giraph's estimated cluster heap 4096MB ask is greater than the current available cluster heap of 0MB. Aborting Job.
        at org.apache.giraph.yarn.GiraphYarnClient.checkPerNodeResourcesAvailable(GiraphYarnClient.java:230)
        at org.apache.giraph.yarn.GiraphYarnClient.run(GiraphYarnClient.java:124)
        at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:96)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
        at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:126)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:483)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:212)


Regards,
Vikalp Handa


On Tue, Aug 12, 2014 at 11:18 AM, 이성일 <si503.lee@samsung.com> wrote:
You just run 'mvn clean' before that command :)

Best Regards, Seong-il
------- Original Message -------
Sender : Vikalp Handa<handa.vikalp@gmail.com>
Date : 2014-08-12 14:35 (GMT+09:00)
Title : Re: ShortestPath Code execution on Hadoop 2.4.0 Inbox x

Thanks for your replies. Do I need to uninstall anything before running the above mvn command ?

Regards,
Vikalp Handa

On Tue, Aug 12, 2014 at 10:26 AM, 이성일 <si503.lee@samsung.com> wrote:
Hi, Vikalp
I think you complied as wrong hadoop version.
Try below command
If you use yarn
mvn -Phadoop_yarn -Dhadoop.version=2.4.0 -DskipTests package
or if you use just hadoop2
mvn -Phadoop_2  -Dhadoop.version=2.4.0 -DskipTests package
Best Regards, Seong-il

------- Original Message -------
Sender : Vikalp Handa<handa.vikalp@gmail.com>
Date : 2014-08-12 13:40 (GMT+09:00)
Title : ShortestPath Code execution on Hadoop 2.4.0 Inbox x

Hi everyone,
I am new to Apache Giraph and would like execute Shortestpath and PageRank example code on Hadoop 2.4.0 single node cluster (my machine) having Centos 6.5.
I have successfully build Giraph on my machine but unable to execute ShortestPath code.
Please let me know if there are any dependencies to be resolved before code execution.

P.S. :
Command used:
hadoop jar /usr/local/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-1.2.1-jar-with-dependencies.jar org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleShortestPathsComputation -vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip /user/hadoop/input/tiny_graph.txt -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op /user/hduser/output/shortestpaths -w 1  -ca giraph.SplitMasterWorker=false

Execution Result:
14/08/11 18:48:37 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
14/08/11 18:48:40 INFO utils.ConfigurationUtils: No edge input format specified. Ensure your InputFormat does not require one.
14/08/11 18:48:40 INFO utils.ConfigurationUtils: No edge output format specified. Ensure your OutputFormat does not require one.
14/08/11 18:48:40 INFO utils.ConfigurationUtils: Setting custom argument [giraph.SplitMasterWorker] to [false] in GiraphConfiguration
14/08/11 18:48:40 INFO Configuration.deprecation: mapreduce.job.counters.limit is deprecated. Instead, use mapreduce.job.counters.max
14/08/11 18:48:40 INFO Configuration.deprecation: mapred.job.map.memory.mb is deprecated. Instead, use mapreduce.map.memory.mb
14/08/11 18:48:40 INFO Configuration.deprecation: mapred.job.reduce.memory.mb is deprecated. Instead, use mapreduce.reduce.memory.mb
14/08/11 18:48:40 INFO Configuration.deprecation: mapred.map.tasks.speculative.execution is deprecated. Instead, use mapreduce.map.speculative
14/08/11 18:48:40 INFO Configuration.deprecation: mapreduce.user.classpath.first is deprecated. Instead, use mapreduce.job.user.classpath.first
14/08/11 18:48:40 INFO Configuration.deprecation: mapred.map.max.attempts is deprecated. Instead, use mapreduce.map.maxattempts
14/08/11 18:48:40 INFO job.GiraphJob: run: Since checkpointing is disabled (default), do not allow any task retries (setting mapred.map.max.attempts = 0, old value = 4)
14/08/11 18:48:40 INFO Configuration.deprecation: mapred.job.tracker is deprecated. Instead, use mapreduce.jobtracker.address
14/08/11 18:48:41 INFO Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
14/08/11 18:48:41 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
Exception in thread "main" java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.JobContext, but class was expected
        at org.apache.giraph.bsp.BspOutputFormat.checkOutputSpecs(BspOutputFormat.java:44)
        at org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:458)
        at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:343)
        at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)
        at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
        at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282)
        at org.apache.giraph.job.GiraphJob.run(GiraphJob.java:250)
        at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:94)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
        at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:483)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Regards,
Vikalp Handa


--------------020806040204010402070704--