Return-Path: X-Original-To: apmail-giraph-user-archive@www.apache.org Delivered-To: apmail-giraph-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 257FC10715 for ; Tue, 30 Jul 2013 15:38:35 +0000 (UTC) Received: (qmail 64793 invoked by uid 500); 30 Jul 2013 15:38:34 -0000 Delivered-To: apmail-giraph-user-archive@giraph.apache.org Received: (qmail 64749 invoked by uid 500); 30 Jul 2013 15:38:33 -0000 Mailing-List: contact user-help@giraph.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@giraph.apache.org Delivered-To: mailing list user@giraph.apache.org Received: (qmail 64741 invoked by uid 99); 30 Jul 2013 15:38:33 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 30 Jul 2013 15:38:33 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of awaagen@gmail.com designates 209.85.215.176 as permitted sender) Received: from [209.85.215.176] (HELO mail-ea0-f176.google.com) (209.85.215.176) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 30 Jul 2013 15:38:27 +0000 Received: by mail-ea0-f176.google.com with SMTP id q16so3648415ead.7 for ; Tue, 30 Jul 2013 08:38:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=Pb2lQSREtfym9h1hkHF1C0VXglbjocBwIptxNLT5Cis=; b=FRJBXMTSE/0/jCHHsOlM2M8BHtbkH2rv8IfKOhDWDR+z67etZTjlVqv8IsU99GKx7k N92kIaWcB7PTEYFclmi1F9OjNhg92XgbfHItWBYfsKM3wikZQ2RdbJHHTnH7bJeFG50u nKkv9tibf7NLEa9CrVOuGU1OY4+shpDOso4yuFDmdBvVveT0HeRNBJperxwwHurUTbhg 1a3I+C2SMA2KEOY9jtYaWm3k0cB11OeHmf9bHMyQ4cqRTXsIl+MyiULWKg5rDVgB+lSN 4G208g+o9e6yzo6PX3YoRs7/ypR7N9HgEq8XZw7RiVJxd4bz51/84Is0hO4X1LrYzDiE Vs+Q== MIME-Version: 1.0 X-Received: by 10.14.241.136 with SMTP id g8mr65042367eer.104.1375198686128; Tue, 30 Jul 2013 08:38:06 -0700 (PDT) Received: by 10.223.94.83 with HTTP; Tue, 30 Jul 2013 08:38:06 -0700 (PDT) In-Reply-To: References: Date: Tue, 30 Jul 2013 08:38:06 -0700 Message-ID: Subject: Re: Shortest paths example in giraf 1.1.0. From: Alex Waagen To: user@giraph.apache.org Content-Type: multipart/alternative; boundary=001a1132ecb62ecefa04e2bc656e X-Virus-Checked: Checked by ClamAV on apache.org --001a1132ecb62ecefa04e2bc656e Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Here is the file. It is almost empty. On Tue, Jul 30, 2013 at 8:33 AM, Kyle Orlando wro= te: > Hmmm, could you post the contents of your mapred-site.xml in > $HADOOP_HOME/conf? You may need to increase the number of map tasks. > > On Tue, Jul 30, 2013 at 11:02 AM, Alex Waagen wrote: > > I am having some trouble getting these examples running. I=92m using gi= raph > > version 1.1.0 and hadoop 0.20.2. I am using the following json file as > > input: > > > > [0,0,[[1,1],[3,3]]] > > [1,0,[[0,1],[2,2],[3,1]]] > > [2,0,[[1,2],[4,4]]] > > [3,0,[[0,3],[1,1],[4,4]]] > > [4,0,[[3,4],[2,4]]] > > > > The command I use is: > > > > hadoop jar > > > /path-to-giraph/giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-0.20.= 203.0-jar-with-dependencies.jar > > org.apache.giraph.GiraphRunner > > org.apache.giraph.examples.SimpleShortestPathsComputation -vif > > org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat > -vip > > /path-to-input/input_file.json -of > > org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op > > /path-to-output/outShortest -w 1 > > > > > > I see the following output. > > > > 13/07/29 14:36:06 INFO utils.ConfigurationUtils: No edge input format > > specified. Ensure your InputFormat does not require one. > > 13/07/29 14:36:06 INFO job.GiraphJob: run: Since checkpointing is > disabled > > (default), do not allow any task retries (setting > mapred.map.max.attempts =3D > > 0, old value =3D 4) > > 13/07/29 14:36:20 INFO mapred.JobClient: Running job: > job_201307232135_0588 > > 13/07/29 14:36:21 INFO mapred.JobClient: map 0% reduce 0% > > 13/07/29 14:36:52 INFO mapred.JobClient: map 50% reduce 0% > > 13/07/29 14:47:24 INFO mapred.JobClient: map 0% reduce 0% > > 13/07/29 14:47:39 INFO mapred.JobClient: Job complete: > job_201307232135_0588 > > 13/07/29 14:47:39 INFO mapred.JobClient: Counters: 6 > > 13/07/29 14:47:39 INFO mapred.JobClient: Job Counters > > 13/07/29 14:47:39 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=3D670508 > > 13/07/29 14:47:39 INFO mapred.JobClient: Total time spent by all reduce= s > > waiting after reserving slots (ms)=3D0 > > 13/07/29 14:47:39 INFO mapred.JobClient: Total time spent by all maps > > waiting after reserving slots (ms)=3D0 > > 13/07/29 14:47:39 INFO mapred.JobClient: Launched map tasks=3D2 > > 13/07/29 14:47:39 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=3D0 > > 13/07/29 14:47:39 INFO mapred.JobClient: Failed map tasks=3D1 > > > > > > When I check the job tracker, I see that two map jobs were killed, with > the > > following errors: > > > > java.lang.Throwable: Child Error > > at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:242) > > Caused by: java.io.IOException: Task process exit with nonzero status o= f > 1. > > at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:229) > > > > java.lang.IllegalStateException: run: Caught an unrecoverable exception > > exists: Failed to check > > > /_hadoopBsp/job_201307232135_0588/_applicationAttemptsDir/0/_superstepDir= /-1/_addressesAndPartitions > > after 3 tries! > > at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:101) > > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647) > > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323) > > at org.apache.hadoop.mapred.Child$4.run(Child.java:270) > > at java.security.AccessController.doPrivileged(Native Method) > > at javax.security.auth.Subject.doAs(Subject.java:396) > > at > > > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation= .java:1157) > > at org.apache.hadoop.mapred.Child.main(Child.java:264) > > Caused by: java.lang.IllegalStateException: exists: Failed to check > > > /_hadoopBsp/job_201307232135_0588/_applicationAttemptsDir/0/_superstepDir= /-1/_addressesAndPartitions > > after 3 tries! > > at org.apache.giraph.zk.ZooKeeperExt.exists(ZooKeeperExt.java:369) > > at org.apache.giraph.worker.BspServiceWorker.s > > Task attempt_201307232135_0588_m_000001_0 failed to report status for 6= 00 > > seconds. Killing! > > > > Any idea what the problem is? > > Thanks in advance. > > > > -- > Kyle Orlando > Computer Engineering Major > University of Maryland > --001a1132ecb62ecefa04e2bc656e Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: quoted-printable
Here is the file. It is almost empty.

<= /div>
<?xml version=3D"1.0"?>
<?xml-styles= heet type=3D"text/xsl" href=3D"configuration.xsl"?><= /div>

<!-- Put site-specific property overrides in this fi= le. -->

<configuration>

</configuration>



On Tue, Jul 30, 2013 at 8:33 AM, Kyle Or= lando <kyle.r.orlando@gmail.com> wrote:
Hmmm, could you post the contents of your mapred-site.xml in
$HADOOP_HOME/conf? =A0You may need to increase the number of map tasks.

On Tue, Jul 30, 2013 at 11:02 AM, Alex Waagen <awaagen@gmail.com> wrote:
> I am having some trouble getting these examples running. I=92m using g= iraph
> version 1.1.0 and hadoop 0.20.2. I am using the following json file as=
> input:
>
> [0,0,[[1,1],[3,3]]]
> [1,0,[[0,1],[2,2],[3,1]]]
> [2,0,[[1,2],[4,4]]]
> [3,0,[[0,3],[1,1],[4,4]]]
> [4,0,[[3,4],[2,4]]]
>
> The command I use is:
>
> hadoop jar
> /path-to-giraph/giraph-core/target/giraph-1.1.0-SNAPSHOT-for-hadoop-0.= 20.203.0-jar-with-dependencies.jar
> org.apache.giraph.GiraphRunner
> org.apache.giraph.examples.SimpleShortestPathsComputation -vif
> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputForma= t -vip
> /path-to-input/input_file.json -of
> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
> /path-to-output/outShortest -w 1
>
>
> I see the following output.
>
> 13/07/29 14:36:06 INFO utils.ConfigurationUtils: No edge input format<= br> > specified. Ensure your InputFormat does not require one.
> 13/07/29 14:36:06 INFO job.GiraphJob: run: Since checkpointing is disa= bled
> (default), do not allow any task retries (setting mapred.map.max.attem= pts =3D
> 0, old value =3D 4)
> 13/07/29 14:36:20 INFO mapred.JobClient: Running job: job_201307232135= _0588
> 13/07/29 14:36:21 INFO mapred.JobClient: map 0% reduce 0%
> 13/07/29 14:36:52 INFO mapred.JobClient: map 50% reduce 0%
> 13/07/29 14:47:24 INFO mapred.JobClient: map 0% reduce 0%
> 13/07/29 14:47:39 INFO mapred.JobClient: Job complete: job_20130723213= 5_0588
> 13/07/29 14:47:39 INFO mapred.JobClient: Counters: 6
> 13/07/29 14:47:39 INFO mapred.JobClient: Job Counters
> 13/07/29 14:47:39 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=3D670508 > 13/07/29 14:47:39 INFO mapred.JobClient: Total time spent by all reduc= es
> waiting after reserving slots (ms)=3D0
> 13/07/29 14:47:39 INFO mapred.JobClient: Total time spent by all maps<= br> > waiting after reserving slots (ms)=3D0
> 13/07/29 14:47:39 INFO mapred.JobClient: Launched map tasks=3D2
> 13/07/29 14:47:39 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=3D0
> 13/07/29 14:47:39 INFO mapred.JobClient: Failed map tasks=3D1
>
>
> When I check the job tracker, I see that two map jobs were killed, wit= h the
> following errors:
>
> java.lang.Throwable: Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:242)
> Caused by: java.io.IOException: Task process exit with nonzero status = of 1.
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:229)
>
> java.lang.IllegalStateException: run: Caught an unrecoverable exceptio= n
> exists: Failed to check
> /_hadoopBsp/job_201307232135_0588/_applicationAttemptsDir/0/_superstep= Dir/-1/_addressesAndPartitions
> after 3 tries!
> at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:101)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformat= ion.java:1157)
> at org.apache.hadoop.mapred.Child.main(Child.java:264)
> Caused by: java.lang.IllegalStateException: exists: Failed to check > /_hadoopBsp/job_201307232135_0588/_applicationAttemptsDir/0/_superstep= Dir/-1/_addressesAndPartitions
> after 3 tries!
> at org.apache.giraph.zk.ZooKeeperExt.exists(ZooKeeperExt.java:369)
> at org.apache.giraph.worker.BspServiceWorker.s
> Task attempt_201307232135_0588_m_000001_0 failed to report status for = 600
> seconds. Killing!
>
> Any idea what the problem is?
> Thanks in advance.



--
Kyle Orlando
Computer Engineering Major
University of Maryland

--001a1132ecb62ecefa04e2bc656e--