Return-Path: X-Original-To: apmail-giraph-user-archive@www.apache.org Delivered-To: apmail-giraph-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 0DE0310782 for ; Tue, 5 Nov 2013 10:23:08 +0000 (UTC) Received: (qmail 1574 invoked by uid 500); 5 Nov 2013 10:23:06 -0000 Delivered-To: apmail-giraph-user-archive@giraph.apache.org Received: (qmail 1214 invoked by uid 500); 5 Nov 2013 10:23:00 -0000 Mailing-List: contact user-help@giraph.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@giraph.apache.org Delivered-To: mailing list user@giraph.apache.org Received: (qmail 1191 invoked by uid 99); 5 Nov 2013 10:22:58 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 05 Nov 2013 10:22:58 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS,WEIRD_PORT X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of rao.jyoti26yadav@gmail.com designates 74.125.82.50 as permitted sender) Received: from [74.125.82.50] (HELO mail-wg0-f50.google.com) (74.125.82.50) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 05 Nov 2013 10:22:54 +0000 Received: by mail-wg0-f50.google.com with SMTP id n12so3282802wgh.29 for ; Tue, 05 Nov 2013 02:22:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=2DRDnlSvuoTmLzCQ27FtsMa5HIknbRI8aiX8+nHoNFM=; b=gWNV3xtozuJqon5yGi+p1F0YETRSOhgubo4U6Yzg4ugHgFcBaMVcxvcxXIhQnBy1t5 4zVIbskyPugGq0t0xdoJ+j553tttsjDeFi4+COlqSwJ2qTzv1+KcsVFRd5/sMGZm/t7T jV25GueKYJXC/UPYzRx7HQCY6aVV07OOZQPo006N3p7bwUFL3vcKHNNkMQVdydoAcEU5 8yFciJVxterOyJmWCytg4vDZMnRGlAyPEW01Hm6gm039WmScYVH4pltiV/+DaSPsmJwZ W0m4ruK3ze5y3AMIE6wStSBYmV+szuOlnPAI3k62jQwUU2oUMJrL6tcZlIbfyIzeKRkp 0t5w== MIME-Version: 1.0 X-Received: by 10.194.175.66 with SMTP id by2mr717106wjc.59.1383646952749; Tue, 05 Nov 2013 02:22:32 -0800 (PST) Received: by 10.194.234.33 with HTTP; Tue, 5 Nov 2013 02:22:32 -0800 (PST) In-Reply-To: References: Date: Tue, 5 Nov 2013 15:52:32 +0530 Message-ID: Subject: Re: Problem in running giraph program From: Jyoti Yadav To: user@giraph.apache.org Content-Type: multipart/alternative; boundary=089e013d19f81d00a504ea6b69da X-Virus-Checked: Checked by ClamAV on apache.org --089e013d19f81d00a504ea6b69da Content-Type: text/plain; charset=ISO-8859-1 Hi Suman.. In earlier posts,you detailed about how to execute our own giraph job in 4 steps.. I successsfully completed 3 steps. I gave the following command . bin/hadoop jar $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleShortestPathsComputation1 -vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip /user/hduser/graph_input/graph.txt -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op /user/hduser/graph_output1/shortestpaths -w 1 Warning: $HADOOP_HOME is deprecated. 13/11/05 15:28:22 INFO utils.ConfigurationUtils: No edge input format specified. Ensure your InputFormat does not require one. 13/11/05 15:28:22 INFO utils.ConfigurationUtils: No edge output format specified. Ensure your OutputFormat does not require one. 13/11/05 15:28:22 INFO job.GiraphJob: run: Since checkpointing is disabled (default), do not allow any task retries (setting mapred.map.max.attempts = 0, old value = 4) 13/11/05 15:28:33 INFO job.GiraphJob: run: Tracking URL: http://hdfsmaster:50030/jobdetails.jsp?jobid=job_201311050424_0001 13/11/05 15:29:20 INFO job.HaltApplicationUtils$DefaultHaltInstructionsWriter: writeHaltInstructions: To halt after next superstep execute: 'bin/halt-application --zkServer localhost:22181 --zkNode /_hadoopBsp/job_201311050424_0001/_haltComputation' 13/11/05 15:29:20 INFO mapred.JobClient: Running job: job_201311050424_0001 13/11/05 15:29:21 INFO mapred.JobClient: map 50% reduce 0% 13/11/05 15:39:07 INFO mapred.JobClient: Job complete: job_201311050424_0001 13/11/05 15:39:07 INFO mapred.JobClient: Counters: 5 13/11/05 15:39:07 INFO mapred.JobClient: Job Counters 13/11/05 15:39:07 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=614204 13/11/05 15:39:07 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0 13/11/05 15:39:07 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0 13/11/05 15:39:07 INFO mapred.JobClient: Launched map tasks=2 13/11/05 15:39:07 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=0 Here above screenful of output i got.No output file is created...Only log file is created.. Any help is really appreciated... Thanks Jyoti On Sun, Oct 27, 2013 at 8:16 PM, suman bharadwaj wrote: > Hi Jyoti, > > Even I'm new to this and i'm no expert. But this is how i made it work. > Note, This might not be the best way to run the job. I'm not sure :) > > But here you go, > > *Step 1*: I place my own giraph code containing vertex.compute in > *$GIRAPH_HOME/* > *giraph-examples/src/main/java/org/apache/giraph/examples/*. > Note, I added this line to the my vertex giraph java file. > *"package org.apache.giraph.examples;"* > > *Step 2*: Then I went to *$GIRAPH_HOME/**giraph-examples/* directory and > executed the following command *"mvn compile"*. > > *Step 3*: Not sure if this step is needed. But give it a try. I exported > the > *giraph-examples-1.0.0-for-hadoop-0.20.203.0-jar-with-dependencies.jar* which > is present in target directory to hadoop classpath. > > *Step 4*: Ran the below command and job ran successfully. > > *"hadoop jar > giraph-examples-1.0.0-for-hadoop-0.20.203.0-jar-with-dependencies.jar > org.apache.giraph.GiraphRunner org.apache.giraph.examples.MyMaximumValue > -vif > org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat > -vip /user/MaxInput.txt -of > org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op > /user/graphout/test7/ -w 1"* > > > It worked fine for me. Hopefully it will work for you as well. > > > Regards, > Suman Bharadwaj S > > > > On Sun, Oct 27, 2013 at 6:33 PM, Jyoti Yadav wrote: > >> Hi .. >> I want to run my own giraph program. I made some modification to already >> existiing SimpleShortestPathsComputation program. >> >> I compiled it and created a jar file. >> I gave following command. >> >> bin/hadoop jar >> $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar >> org.apache.giraph.GiraphRunner -libjars /home/hduser/sp2.jar >> SimpleShortestPathsComputation*1* -vif >> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat >> -vip /user/hduser/sp_input/tiny_graph.txt -vof >> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op >> /user/hduser/sp_output7/shortestpaths -w 1 >> >> >> I am getting following error.. >> >> >> Exception in thread "main" java.lang.ClassNotFoundException: >> SimpleShortestPathsComputation1 >> at java.net.URLClassLoader$1.run(URLClassLoader.java:366) >> at java.net.URLClassLoader$1.run(URLClassLoader.java:355) >> at java.security.AccessController.doPrivileged(Native Method) >> at java.net.URLClassLoader.findClass(URLClassLoader.java:354) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:424) >> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >> at java.lang.Class.forName0(Native Method) >> at java.lang.Class.forName(Class.java:190) >> at >> org.apache.giraph.utils.ConfigurationUtils.handleComputationClass(ConfigurationUtils.java:470) >> at >> org.apache.giraph.utils.ConfigurationUtils.populateGiraphConfiguration(ConfigurationUtils.java:453) >> at >> org.apache.giraph.utils.ConfigurationUtils.parseArgs(ConfigurationUtils.java:207) >> at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74) >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79) >> at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:606) >> at org.apache.hadoop.util.RunJar.main(RunJar.java:156) >> >> >> Please help me out..I am badly tangled.. >> >> Thanks >> Jyoti >> >> >> > --089e013d19f81d00a504ea6b69da Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Hi Suman..
In = earlier posts,you detailed about how to execute our own giraph job in 4 ste= ps..
I successsfully completed 3 steps.

I gave the fo= llowing command .


bin/hadoop jar $GIRAPH_HOME/giraph-examples/target/giraph-examples-= 1.1.0-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar org.apache.g= iraph.GiraphRunner org.apache.giraph.examples.SimpleShortestPathsComputatio= n1 -vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFo= rmat -vip /user/hduser/graph_input/graph.txt=A0 -vof=A0 org.apache.giraph.i= o.formats.IdWithValueTextOutputFormat -op /user/hduser/graph_output1/shorte= stpaths -w 1

Warning: $HADOOP_HOME is deprecated.

13/11/05 15:28:22 INFO util= s.ConfigurationUtils: No edge input format specified. Ensure your InputForm= at does not require one.
13/11/05 15:28:22 INFO utils.ConfigurationUtils= : No edge output format specified. Ensure your OutputFormat does not requir= e one.
13/11/05 15:28:22 INFO job.GiraphJob: run: Since checkpointing is disabled = (default), do not allow any task retries (setting mapred.map.max.attempts = =3D 0, old value =3D 4)
13/11/05 15:28:33 INFO job.GiraphJob: run: Track= ing URL: http://hdfsmaster:50030/jobdetails.jsp?jobid=3Djob_201311050= 424_0001
13/11/05 15:29:20 INFO job.HaltApplicationUtils$DefaultHaltInstructionsWrit= er: writeHaltInstructions: To halt after next superstep execute: 'bin/h= alt-application --zkServer localhost:22181 --zkNode /_hadoopBsp/job_2013110= 50424_0001/_haltComputation'
13/11/05 15:29:20 INFO mapred.JobClient: Running job: job_201311050424_0001=
13/11/05 15:29:21 INFO mapred.JobClient:=A0 map 50% reduce 0%
13/11/= 05 15:39:07 INFO mapred.JobClient: Job complete: job_201311050424_0001
13/11/05 15:39:07 INFO mapred.JobClient: Counters: 5
13/11/05 15:39:07 I= NFO mapred.JobClient:=A0=A0 Job Counters
13/11/05 15:39:07 INFO mapred.= JobClient:=A0=A0=A0=A0 SLOTS_MILLIS_MAPS=3D614204
13/11/05 15:39:07 INFO= mapred.JobClient:=A0=A0=A0=A0 Total time spent by all reduces waiting afte= r reserving slots (ms)=3D0
13/11/05 15:39:07 INFO mapred.JobClient:=A0=A0=A0=A0 Total time spent by al= l maps waiting after reserving slots (ms)=3D0
13/11/05 15:39:07 INFO map= red.JobClient:=A0=A0=A0=A0 Launched map tasks=3D2
13/11/05 15:39:07 INFO= mapred.JobClient:=A0=A0=A0=A0 SLOTS_MILLIS_REDUCES=3D0



Here above screenful of=A0 output i got.No output file is= created...Only log file is created..

Any help is really appre= ciated...

Thanks

Jyoti


On Sun, Oct 27, 2013 at 8:16 PM, suman bharadwaj <suman= .dna@gmail.com> wrote:
Hi Jyoti,

Even I'm new to this and i'm no expert. But this is how i made it= work. Note, This might not be the best way to run the job. I'm not sur= e :)

But here you go,

Step 1: I place my own gir= aph code containing vertex.compute in $GIRAPH_HOME/giraph-examples/src/main/java/org/apache/giraph/examples= /<yourFile.java>. Note, I added this line to the my vertex giraph= java file.=A0
"package org.apache.giraph.ex= amples;"

Step 2: Then I went = to=A0$GIRAPH_HOME/giraph-= examples/ directory and executed the following command "mvn com= pile".

Step 3: Not sure if this step is needed. But give = it a try. I exported the=A0giraph-examples-1.0.0-for-hadoop-0.20.203.0-j= ar-with-dependencies.jar=A0which is present in target directory to hado= op classpath.=A0

Step 4: = Ran the below command and job ran successfully.

"hadoop jar giraph-examples-1.0.0-for-hadoop-0.20.203.0-jar-with-depe= ndencies.jar org.apache.giraph.GiraphRunner org.apache.giraph.examples.MyMa= ximumValue -vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVerte= xInputFormat -vip /user/MaxInput.txt -of org.apache.giraph.io.formats.IdWit= hValueTextOutputFormat -op /user/graphout/test7/ -w 1"


It worked fine for me. Hopefully it will= work for you as well.


Regards,
Suman Bharadwaj S


On Sun, Oct 27, 2013 at 6:33 PM, Jyoti Yad= av <rao.jyoti26yadav@gmail.com> wrote:
Hi ..
I want to run my own giraph program. I made some mod= ification to already existiing SimpleShortestPathsComputation program.

I compiled it and created a jar file.
I gave following command.

=A0bin/hadoop jar $GIRAPH_HOME/girap= h-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-0.20.203.0-jar-= with-dependencies.jar org.apache.giraph.GiraphRunner -libjars /home/hduser/= sp2.jar SimpleShortestPathsComputation1 -vif org.apache.giraph.io.fo= rmats.JsonLongDoubleFloatDoubleVertexInputFormat -vip /user/hduser/sp_input= /tiny_graph.txt -vof org.apache.giraph.io.formats.IdWithValueTextOutputForm= at -op /user/hduser/sp_output7/shortestpaths -w 1


I am getting following error..


Exception in thread= "main" java.lang.ClassNotFoundException: SimpleShortestPathsComp= utation1
=A0=A0=A0 at java.net.URLClassLoader$1.run(URLClassLoader.java:= 366)
=A0=A0=A0 at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
=A0= =A0=A0 at java.security.AccessController.doPrivileged(Native Method)
=A0= =A0=A0 at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
=A0= =A0=A0 at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
=A0=A0=A0 at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)<= br>=A0=A0=A0 at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
= =A0=A0=A0 at java.lang.Class.forName0(Native Method)
=A0=A0=A0 at java.l= ang.Class.forName(Class.java:190)
=A0=A0=A0 at org.apache.giraph.utils.ConfigurationUtils.handleComputationCl= ass(ConfigurationUtils.java:470)
=A0=A0=A0 at org.apache.giraph.utils.Co= nfigurationUtils.populateGiraphConfiguration(ConfigurationUtils.java:453)=A0=A0=A0 at org.apache.giraph.utils.ConfigurationUtils.parseArgs(Configu= rationUtils.java:207)
=A0=A0=A0 at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74)
= =A0=A0=A0 at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
= =A0=A0=A0 at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
= =A0=A0=A0 at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
=A0=A0=A0 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)=A0=A0=A0 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcces= sorImpl.java:57)
=A0=A0=A0 at sun.reflect.DelegatingMethodAccessorImpl.i= nvoke(DelegatingMethodAccessorImpl.java:43)
=A0=A0=A0 at java.lang.reflect.Method.invoke(Method.java:606)
=A0=A0=A0 = at org.apache.hadoop.util.RunJar.main(RunJar.java:156)


Ple= ase help me out..I am badly tangled..

Thanks
Jyoti




--089e013d19f81d00a504ea6b69da--