Return-Path: X-Original-To: apmail-giraph-user-archive@www.apache.org Delivered-To: apmail-giraph-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id CE059106A5 for ; Mon, 3 Jun 2013 21:23:25 +0000 (UTC) Received: (qmail 43904 invoked by uid 500); 3 Jun 2013 21:23:25 -0000 Delivered-To: apmail-giraph-user-archive@giraph.apache.org Received: (qmail 43868 invoked by uid 500); 3 Jun 2013 21:23:25 -0000 Mailing-List: contact user-help@giraph.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@giraph.apache.org Delivered-To: mailing list user@giraph.apache.org Received: (qmail 43860 invoked by uid 99); 3 Jun 2013 21:23:25 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 03 Jun 2013 21:23:25 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW X-Spam-Check-By: apache.org Received-SPF: error (nike.apache.org: local policy) Received: from [209.85.128.174] (HELO mail-ve0-f174.google.com) (209.85.128.174) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 03 Jun 2013 21:23:19 +0000 Received: by mail-ve0-f174.google.com with SMTP id oz10so3248086veb.33 for ; Mon, 03 Jun 2013 14:22:38 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:x-gm-message-state; bh=JfqKcphARF0P2+CABOmCpfFf25HbaCO2V44dEaeyunM=; b=A478fvuxps5LPQ2/0q7suFX15JZohyVJ2BNYzaXlxTPfnlHFz3a43r/HJ6ldFttFc/ lGTHrml4lSmRv6/kNpWNiTbT2+6u5Ss2F2gST9zbZM17WsQkm404VtJFIpq44xYa9rIG NuMO0wTQHCeZDm0tms6nSmycJsSma1Q4zi9dBndc24x/96wQU3GViNtMvObQEvOtyIIg 5TZWiZmRqUTmPbk8Mgk8lqBdhd9hFqTT4SK3zTWBtIFnjlECKg+vwc0xDmoZaF+9rcRK BV4guTw7jQAm77Qdkdwcn5PrVCjKd9yHQNLz9UqFh7DgtkxalVHvw/mieFROeeLT3Z8w LYWA== MIME-Version: 1.0 X-Received: by 10.59.3.9 with SMTP id bs9mr17734847ved.38.1370294557766; Mon, 03 Jun 2013 14:22:37 -0700 (PDT) Received: by 10.58.252.3 with HTTP; Mon, 3 Jun 2013 14:22:37 -0700 (PDT) In-Reply-To: References: Date: Mon, 3 Jun 2013 22:22:37 +0100 Message-ID: Subject: Re: SimpleShortestPathsComputation with Edge List input file From: Peter Holland To: user@giraph.apache.org Content-Type: multipart/alternative; boundary=e89a8f9219365a8bb704de469006 X-Gm-Message-State: ALoCoQlxSUipKqX++ZueLg6AfYrt9TLpYNsFjYoLQLtxA7RS0R2HwWot4k+GWcCH+q0J/2pZSUIK X-Virus-Checked: Checked by ClamAV on apache.org --e89a8f9219365a8bb704de469006 Content-Type: text/plain; charset=ISO-8859-1 Thank you for your response Claudio. I updated the command with the input class you suggested. *bin/hadoop jar /home/ubuntu/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-1.0.2-jar-with-dependencies.jar * * org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleShortestPathsComputation * *-eif org.apache.giraph.io.formats.IntNullTetxEdgeInputFormat* *-eip /simpleEdgeList/SimpleEdgeList.tsv* *-of org.apache.giraph.io.formats.IdWithValueTextOutputFormat * *-op /outShortestEdgeList01 * *-w 1* Unfortunately I am getting an error message *13/06/03 23:00:08 INFO utils.ConfigurationUtils: No vertex input format specified. Ensure your InputFormat does not require one.* *Exception in thread "main" java.lang.IllegalArgumentException: checkClassTypes: Vertex index types don't match, vertex - class org.apache.hadoop.io.LongWritable, edge input format - class org.apache.hadoop.io.IntWritable* * at org.apache.giraph.job.GiraphConfigurationValidator.verifyEdgeInputFormatGenericTypes(GiraphConfigurationValidator.java:266) * * at org.apache.giraph.job.GiraphConfigurationValidator.validateConfiguration(GiraphConfigurationValidator.java:125) * * at org.apache.giraph.utils.ConfigurationUtils.parseArgs(ConfigurationUtils.java:155) * * at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74)* * at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)* * at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)* * at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)* * at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)* * at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) * * at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) * * at java.lang.reflect.Method.invoke(Method.java:597)* * at org.apache.hadoop.util.RunJar.main(RunJar.java:156)* On 3 June 2013 21:00, Claudio Martella wrote: > with the only problem that you picked an abstract class again... > I advised you to use an inputformat that has the name of the types in the > class name, hence org.apache.giraph.io.formats.IntNullTextEdgeInputFormat > should work for you. > > > On Mon, Jun 3, 2013 at 9:34 PM, Peter Holland wrote: > >> Thank you for the advice Claudio >> >> I updated the run command to use different io classes >> >> *bin/hadoop jar >> /home/ubuntu/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-1.0.2-jar-with-dependencies.jar >> * >> * org.apache.giraph.GiraphRunner >> org.apache.giraph.examples.SimpleShortestPathsComputation * >> *-eif org.apache.giraph.io.EdgeInputFormat* >> * -eip /simpleEdgeList/SimpleEdgeList.tsv* >> *-of org.apache.giraph.io.formats.IdWithValueTextOutputFormat * >> *-op /outShortestEdgeList01 * >> *-w 1* >> * >> * >> This code does start a MapReduce job but progress stays at 0%. The log >> file for the job has the following IOException error; >> >> *MapAttempt TASK_TYPE="MAP" TASKID="task_201306031954_0002_m_000000" >> TASK_ATTEMPT_ID="attempt_201306031954_0002_m_000000_0" TASK_STATUS="FAILED" >> FINISH_TIME="1370282492527" HOSTNAME="ubuntu-VirtualBox" >> ERROR="java\.lang\.Throwable: Child Error* >> * at org\.apache\.hadoop\.mapred\.TaskRunner\.run(TaskRunner\.java:271)* >> *Caused by: java\.io\.IOException: Task process exit with nonzero status >> of 1\.* >> * at org\.apache\.hadoop\.mapred\.TaskRunner\.run(TaskRunner\.java:258)* >> >> So, this leaves 3 questions; >> Is the edge list file format correct? (a tab separated file with a .tsv >> extension) >> Is the input class correct? >> Is the output class correct? >> >> Thank you, >> Peter >> >> >> >> >> On 3 June 2013 01:05, Claudio Martella wrote: >> >>> Hi Peter, >>> >>> shortly, those are abstract classes, that's why you cannot instantiate >>> them. You'll have to use a specific class extending those classes that are >>> aware of the types of the signature of the vertex (I, V, E, M). check out >>> some classes in the format package that have those types in the class name. >>> >>> >>> On Mon, Jun 3, 2013 at 1:25 AM, Peter Holland wrote: >>> >>>> Hello, >>>> I'm new to Giraph and I'm trying to run SimpleShortestPathsComputation >>>> using an edge list input file. I have some questions and and error message >>>> that hopefully I can get some help with. >>>> >>>> Edge List File Format >>>> What is the correct format for an edge list input file? >>>> I have a .tsv file with a vertex represented as an integer. Is this >>>> correct? >>>> >>>> 5 11 >>>> 1 6 >>>> 6 9 >>>> 6 8 >>>> 8 9 >>>> ..... >>>> >>>> Input File Class: >>>> Is org.apache.giraph.io.formats.*TextEdgeInputFormat *the only input >>>> format that can be used for edge lists? >>>> >>>> Output File Class: >>>> Does the output format depend on the job you are running? I have been >>>> using org.apache.giraph.io.formats.*TextVertexOutputFormat* for >>>> SimpleShortestPathsComputation. >>>> >>>> Run Command: >>>> So this is the command I am using to try to run the >>>> SimpleShortestPathsComputation using an edge list input file. >>>> >>>> *bin/hadoop jar >>>> /home/ubuntu/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-1.0.2-jar-with-dependencies.jar >>>> org.apache.giraph.GiraphRunner >>>> org.apache.giraph.examples.SimpleShortestPathsComputation * >>>> *-eif org.apache.giraph.io.formats.TextEdgeInputFormat * >>>> *-eip /simpleEdgeList/SimpleEdgeList.tsv * >>>> *-of org.apache.giraph.io.formats.TextVertexOutputFormat * >>>> *-op /outShortest* >>>> *-w 3* >>>> >>>> Error Message >>>> When I run the above command I get the following error message: >>>> Exception in thread "main" java.lang.IllegalStateException: >>>> newInstance: Couldn't instantiate >>>> org.apache.giraph.io.formats.TextVertexOutputFormat >>>> >>>> Thank you, >>>> Peter >>>> >>> >>> >>> >>> -- >>> Claudio Martella >>> claudio.martella@gmail.com >>> >> >> > > > -- > Claudio Martella > claudio.martella@gmail.com > --e89a8f9219365a8bb704de469006 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Thank you for your response Claudio.=A0

I updated the command with the input class you suggested.=A0

bin/hadoop jar /home/ubuntu/giraph/gi= raph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-1.0.2-jar-wi= th-dependencies.jar=A0
org.apache.girap= h.GiraphRunner org.apach= e.giraph.examples.SimpleShortestPathsComputation=A0
-eif o= rg.apache.giraph.io.formats.IntNullTetxEdgeInputFormat
-eip /simpleEdgeList/SimpleEdgeList.tsv
-of org.ap= ache.giraph.io.formats.IdWithValueTextOutputFormat=A0
-op = /outShortestEdgeList01=A0
-w 1

Unfortunately I am getting an error message

13/06/03 23:00:08 INFO utils.ConfigurationUtils: No vertex in= put format specified. Ensure your InputFormat does not require one.
Exception in thread "main" java.lang.IllegalArgumentExcep= tion: checkClassTypes: Vertex index types don't match, vertex - class o= rg.apache.hadoop.io.LongWritable, edge input format - class org.apache.hado= op.io.IntWritable
at org.apache.gi= raph.job.GiraphConfigurationValidator.verifyEdgeInputFormatGenericTypes(Gir= aphConfigurationValidator.java:266)
at org.apache.giraph.job.GiraphConfigurationVa= lidator.validateConfiguration(GiraphConfigurationValidator.java:125)
at org.apache.gi= raph.utils.ConfigurationUtils.parseArgs(ConfigurationUtils.java:155)
at org.apache= .giraph.GiraphRunner.run(GiraphRunner.java:74)
at org.apache.ha= doop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunne= r.run(ToolRunner.java:79)
at org.apache.gi= raph.GiraphRunner.main(GiraphRunner.java:124)
at sun.reflect.NativeMethodAccessor= Impl.invoke0(Native Method)
at sun.reflect.N= ativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
<= div> at sun.reflect.De= legatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)=
at java.lang.ref= lect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.j= ava:156)


On 3 June 2013 21:00, Claudio Martella <claudio.martella@g= mail.com> wrote:
with the only problem that = you picked an abstract class again...
I advised you to use an inputform= at that has the name of the types in the class name, hence org.apache.girap= h.io.formats.IntNullTextEdgeInputFormat should work for you.


On Mon, Jun 3, 2013 at 9:34 PM, Peter Holland <d99991048= @mydit.ie> wrote:
Thank you for the advice Cl= audio

I updated the run command to use different io clas= ses

bin/hadoop jar /home/ubuntu/giraph/giraph-= examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-1.0.2-jar-with-de= pendencies.jar=A0
org.apache.giraph.Gira= phRunner org.apache.giraph.exa= mples.SimpleShortestPathsComputation=A0
-eif org.apache.giraph.io= .EdgeInputFormat
=A0-eip /simpleEdgeList/SimpleEdgeList.tsv
= -of org.apache.giraph.io.formats.IdWithValueTextOutputFormat=A0<= /div>
-op /outShortestEdgeList01=A0
-w 1
<= div>
This code does start a MapReduce job but progress sta= ys at 0%. The log file for the job has the following IOException error;

MapAttempt TASK_TYPE=3D"MAP" TASKID= =3D"task_201306031954_0002_m_000000" TASK_ATTEMPT_ID=3D"atte= mpt_201306031954_0002_m_000000_0" TASK_STATUS=3D"FAILED" FIN= ISH_TIME=3D"1370282492527" HOSTNAME=3D"ubuntu-VirtualBox&quo= t; ERROR=3D"java\.lang\.Throwable: Child Error
at org\.apache\.hadoop= \.mapred\.TaskRunner\.run(TaskRunner\.java:271)
Caused by:= java\.io\.IOException: Task process exit with nonzero status of 1\.
at org\.apache\.hadoop= \.mapred\.TaskRunner\.run(TaskRunner\.java:258)

So, this leaves 3 questions; =A0
Is the edge list file fo= rmat correct? (a tab separated file with a .tsv extension)=A0
Is the input class correct?=A0
Is the output class correct?= =A0

Thank you,
Peter




On 3 June 2013 01:05, Claudio Martella <= span dir=3D"ltr"><claudio.martella@gmail.com> wrote:
Hi Peter,

shortly, those are abstract c= lasses, that's why you cannot instantiate them. You'll have to use = a specific class extending those classes that are aware of the types of the= signature of the vertex (I, V, E, M). check out some classes in the format= package that have those types in the class name.


On Mon, Jun 3, 2013 at 1:25 AM, Peter Holland <d99991048@mydit.ie&= gt; wrote:
Hello,=A0
I'm new to Giraph and I'm trying to run SimpleShortestPathsComputat= ion using an edge list input file. I have some questions and and error mess= age that hopefully I can get some help with.=A0
=
Edge List File Format
What is the correct format for an edge list input file?=A0
I have a .= tsv file with a vertex represented as an integer. Is this correct?=A0
=
5 11
1 6
6 9
6 8
8 = 9
.....

Input File Class:
Is org.apache.giraph.io.formats.TextEdgeInputForm= at=A0the only input format that can be used for edge lists?=A0
=
Output File Class:=A0
Does the output format depend on the job you are running? I have been using= org.apache.giraph.io.formats.TextVertexOutputFormat=A0for SimpleSho= rtestPathsComputation.

Run Command:=A0
So this is the command I am using to try to = run the SimpleShortestPathsComputation using an edge list input file.=A0
=
bin/hadoop jar /home/ubuntu/giraph/giraph-examples/target/gira= ph-examples-1.1.0-SNAPSHOT-for-hadoop-1.0.2-jar-with-dependencies.jar org.a= pache.giraph.GiraphRunner org.apache.giraph.examples.SimpleShortestPathsCom= putation=A0
= -eif org.apache.giraph.io.formats.TextEdgeInputFormat=A0
-eip /simpleEdgeList/SimpleEdgeList.tsv=A0
-of org.apache.gi= raph.io.formats.TextVertexOutputFormat=A0
-op /outShortest
-w 3

Error Message
When I run the above command I get th= e following error message:=A0
Exception in thread "main" java.lang.IllegalStateException: = newInstance: Couldn't instantiate org.apache.giraph.io.formats.TextVert= exOutputFormat

Thank you,
Peter=A0



<= font color=3D"#888888">--
=A0 =A0Claudio Martella
=A0 =A0claudio.martella@g= mail.com=A0 =A0




<= /div>--
=A0 =A0Claudio = Martella
=A0 =A0claudio.martella@gmail.com=A0 =A0

--e89a8f9219365a8bb704de469006--