Return-Path: X-Original-To: apmail-giraph-user-archive@www.apache.org Delivered-To: apmail-giraph-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 5CA6310286 for ; Sat, 5 Oct 2013 12:36:28 +0000 (UTC) Received: (qmail 55802 invoked by uid 500); 5 Oct 2013 12:36:24 -0000 Delivered-To: apmail-giraph-user-archive@giraph.apache.org Received: (qmail 55750 invoked by uid 500); 5 Oct 2013 12:36:15 -0000 Mailing-List: contact user-help@giraph.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@giraph.apache.org Delivered-To: mailing list user@giraph.apache.org Received: (qmail 55739 invoked by uid 99); 5 Oct 2013 12:36:13 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 05 Oct 2013 12:36:13 +0000 X-ASF-Spam-Status: No, hits=2.5 required=5.0 tests=FREEMAIL_REPLY,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS,WEIRD_PORT X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of silvio.digregorio@gmail.com designates 209.85.223.180 as permitted sender) Received: from [209.85.223.180] (HELO mail-ie0-f180.google.com) (209.85.223.180) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 05 Oct 2013 12:36:06 +0000 Received: by mail-ie0-f180.google.com with SMTP id u16so11486850iet.39 for ; Sat, 05 Oct 2013 05:35:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=oHM6G1DQiF6XrjX2oRUj6NrBHZVoWMXBijX38SeGnLo=; b=wgxh6ihMtZc3mGH/vpW46dw3YdPGnTBoWDTmhKK9tol8EJka8BkcyMpp8inIg+YNJ4 8ZZawfcr1LhpwbobgK44ODeqMGmOsWJV+1btNvhx1jc/vjIDlJmZNDz7EeC12KlaSXxV L1RtkQAI6D0j9u7hI978dn/R1wZZdD+/Hsgg4I4KGXfcVZh/QntHnJ6d3KcO8hrfJAyA QNXbDXETAuQX6s7EnvhwYryuzr4SFZxf3b7su86VzNTDBmpQpgFzZ9QelI/l48vCGhaV 6XEAtfZvMc472zp1OC42FfXfuFr/qR3XOxC/zdl+thbyFYvRxFpgGsNG1yYj7gll27FY U6ng== MIME-Version: 1.0 X-Received: by 10.42.97.71 with SMTP id m7mr11929262icn.33.1380976545204; Sat, 05 Oct 2013 05:35:45 -0700 (PDT) Received: by 10.64.239.226 with HTTP; Sat, 5 Oct 2013 05:35:45 -0700 (PDT) Received: by 10.64.239.226 with HTTP; Sat, 5 Oct 2013 05:35:45 -0700 (PDT) In-Reply-To: References: Date: Sat, 5 Oct 2013 14:35:45 +0200 Message-ID: Subject: Re: connected components example in giraph 1.0 From: Silvio Di gregorio To: user@giraph.apache.org Content-Type: multipart/alternative; boundary=20cf303bf9586b92e304e7fda80b X-Virus-Checked: Checked by ClamAV on apache.org --20cf303bf9586b92e304e7fda80b Content-Type: text/plain; charset=ISO-8859-1 I ha ve build w/ hadoop_cdh4.1.2 parameter. Something is changed, monday i report the result. Now the farm is closed. Il giorno 05/ott/2013 14:06, "Claudio Martella" ha scritto: > Oh, right, -vof is in trunk. Anyway it looks like you built giraph for the > wrong profile. You mentioned you're running on 2.0, but your giraph is > built for 0.20.203. try building with a profile for your hadoop version. > > > On Fri, Oct 4, 2013 at 2:35 PM, Silvio Di gregorio < > silvio.digregorio@gmail.com> wrote: > >> >> org.apache.commons.cli.UnrecognizedOptionException: Unrecognized option: >> -vof >> >> in 1.0 version is >> -of,--outputFormat Vertex output format >> -op,--outputPath Vertex output path >> >> >> >> 2013/10/4 Claudio Martella >> >>> did you try the argument (-vof) i suggested? >>> >>> >>> On Fri, Oct 4, 2013 at 2:13 PM, Silvio Di gregorio < >>> silvio.digregorio@gmail.com> wrote: >>> >>>> >>>> i've specified -of >>>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat >>>> >>>> but the same error was produced >>>> >>>> Exception in thread "main" java.lang.IncompatibleClassChangeError: >>>> Found interface org.apache.hadoop.mapreduce.JobContext, but class was >>>> expected >>>> at >>>> org.apache.giraph.bsp.BspOutputFormat.checkOutputSpecs(BspOutputFormat.java:43) >>>> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:984) >>>> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:945) >>>> at java.security.AccessController.doPrivileged(Native Method) >>>> at javax.security.auth.Subject.doAs(Subject.java:396) >>>> at >>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408) >>>> at >>>> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:945) >>>> at org.apache.hadoop.mapreduce.Job.submit(Job.java:566) >>>> at >>>> org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:596) >>>> at org.apache.giraph.job.GiraphJob.run(GiraphJob.java:237) >>>> at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:94) >>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) >>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) >>>> at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124) >>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>> at >>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >>>> at >>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >>>> at java.lang.reflect.Method.invoke(Method.java:597) >>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:208) >>>> >>>> >>>> >>>> 2013/10/4 Claudio Martella >>>> >>>>> Hi, >>>>> >>>>> you need to specify the vertex outputformat class (-vof option), e.g. >>>>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat. >>>>> >>>>> >>>>> On Fri, Oct 4, 2013 at 1:06 PM, Silvio Di gregorio < >>>>> silvio.digregorio@gmail.com> wrote: >>>>> >>>>>> >>>>>> Hi, >>>>>> >>>>>> I hope I have sent to the right address. >>>>>> >>>>>> i have a graph (directed and unweighted) stored in hdfs like a >>>>>> adjacency list (140Milions of edges 6Milions of vertex) >>>>>> >>>>>> nodeneighbors >>>>>> >>>>>> 23 2 1 3 4 3 >>>>>> >>>>>> 1 999 99923 909 ....... >>>>>> >>>>>> ...... >>>>>> >>>>>> hadoop version Hadoop 2.0.0-cdh4.3.0 - java 1.6 >>>>>> >>>>>> >>>>>> I have executed the giraph-1.0 connected components example, in this >>>>>> fashion >>>>>> >>>>>>> > hadoop jar /usr/local/giraph/giraph-examples/target/giraph- >>>>>>> examples-1.0.0-for-hadoop-0.20.203.0-jar-with-dependencies.jar >>>>>>> org.apache.giraph.GiraphRunner org.apache.giraph.examples.ConnectedComponentsVertex >>>>>>> -vif org.apache.giraph.io.formats.IntIntNullTextInputFormat -vip >>>>>>> /user/hdfs/lista_adj_txt -op connectedgiraph --workers 4 >>>>>> >>>>>> >>>>>> >>>>>> and then fail with: >>>>>> >>>>>> 13/10/04 09:28:29 INFO utils.ConfigurationUtils: No edge input format >>>>>> specified. Ensure your InputFormat does not require one. >>>>>> >>>>>> >>>>>> >>>>>> 13/10/04 09:28:29 INFO utils.ConfigurationUtils: No output format >>>>>> specified. Ensure your OutputFormat does not require one. >>>>>> >>>>>> >>>>>> >>>>>> 13/10/04 09:28:30 INFO job.GiraphJob: run: Since checkpointing is >>>>>> disabled (default), do not allow any task retries (setting >>>>>> mapred.map.max.attempts = 0, old value = 4) >>>>>> >>>>>> >>>>>> >>>>>> 13/10/04 09:28:31 WARN mapred.JobClient: Use GenericOptionsParser for >>>>>> parsing the arguments. Applications should implement Tool for the same. >>>>>> >>>>>> >>>>>> >>>>>> 13/10/04 09:28:31 INFO mapred.JobClient: Cleaning up the staging area >>>>>> hdfs:// >>>>>> srv-bigdata-dev-01.int.sose.it:8020/user/hdfs/.staging/job_201309181636_0535 >>>>>> >>>>>> Exception in thread "main" java.lang.IncompatibleClassChangeError: >>>>>> Found interface org.apache.hadoop.mapreduce.JobContext, but class was >>>>>> expected >>>>>> >>>>>> at >>>>>> org.apache.giraph.bsp.BspOutputFormat.checkOutputSpecs(BspOutputFormat.java:43) >>>>>> >>>>>> ...... >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> Thanks in advance >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Claudio Martella >>>>> claudio.martella@gmail.com >>>>> >>>> >>>> >>> >>> >>> -- >>> Claudio Martella >>> claudio.martella@gmail.com >>> >> >> > > > -- > Claudio Martella > claudio.martella@gmail.com > --20cf303bf9586b92e304e7fda80b Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable

I ha ve build w/ hadoop_cdh4.1.2 parameter.
Something is changed, monday i report the result. Now the farm is closed.

Il giorno 05/ott/2013 14:06, "Claudio Marte= lla" <claudio.martell= a@gmail.com> ha scritto:
Oh, right, -vof is in trunk. Anyway it looks like you buil= t giraph for the wrong profile. You mentioned you're running on 2.0, bu= t your giraph is built for 0.20.203. try building with a profile for your hadoop ve= rsion.


On Fri, Oct 4, 2013 at 2:35 PM, Silvio D= i gregorio <silvio.digregorio@gmail.com> wrote:

org.apache.c= ommons.cli.UnrecognizedOptionException: Unrecognized option: -vof
<= br>
in 1.0 version is
=A0-of,--outputFormat <arg&g= t; =A0 =A0 =A0 =A0 =A0 =A0 =A0 Vertex output format
=A0-op,--outputPath <arg> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 Vertex= output path



2013/10/4 Claudio Martella <claudio.martella@gmail.com>
did you try the argument (-= vof) i suggested?


On Fri, Oct 4, 2013 at 2:13 PM, Silvio Di gregor= io <silvio.digregorio@gmail.com> wrote:

i've spe= cified =A0-of =A0org.apache.giraph.io.formats.IdWithValueTextOutputFormat
but the same error was produced

Exception in thread "main" java.lang.IncompatibleClassChangeErro= r: Found interface org.apache.hadoop.mapreduce.JobContext, but class was ex= pected
=A0 =A0 =A0 =A0 at org.apache.giraph.bsp.BspOutputFormat.checkOutputSp= ecs(BspOutputFormat.java:43)
=A0 =A0 =A0 =A0 at org.apache.= hadoop.mapred.JobClient$2.run(JobClient.java:984)
=A0 =A0 =A0 =A0= at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:945)
=A0 =A0 =A0 =A0 at java.security.AccessController.doPrivileged(Native = Method)
=A0 =A0 =A0 =A0 at javax.security.auth.Subject.doAs(Subje= ct.java:396)
=A0 =A0 =A0 =A0 at org.apache.hadoop.security.UserGr= oupInformation.doAs(UserGroupInformation.java:1408)
=A0 =A0 =A0 =A0 at org.apache.hadoop.mapred.JobClient.submitJobInterna= l(JobClient.java:945)
=A0 =A0 =A0 =A0 at org.apache.hadoop.mapred= uce.Job.submit(Job.java:566)
=A0 =A0 =A0 =A0 at org.apache.hadoop= .mapreduce.Job.waitForCompletion(Job.java:596)
=A0 =A0 =A0 =A0 at org.apache.giraph.job.GiraphJob.run(GiraphJob.java:= 237)
=A0 =A0 =A0 =A0 at org.apache.giraph.GiraphRunner.run(Giraph= Runner.java:94)
=A0 =A0 =A0 =A0 at org.apache.hadoop.util.ToolRun= ner.run(ToolRunner.java:70)
=A0 =A0 =A0 =A0 at org.apache.hadoop.util.ToolRunner.run(ToolRunner.ja= va:84)
=A0 =A0 =A0 =A0 at org.apache.giraph.GiraphRunner.main(Gir= aphRunner.java:124)
=A0 =A0 =A0 =A0 at sun.reflect.NativeMethodAc= cessorImpl.invoke0(Native Method)
=A0 =A0 =A0 =A0 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeM= ethodAccessorImpl.java:39)
=A0 =A0 =A0 =A0 at sun.reflect.Delegat= ingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
=A0 =A0 =A0 =A0 at java.lang.reflect.Method.invoke(Method.java:597)
=A0 =A0 =A0 =A0 at org.apache.hadoop.util.RunJar.main(RunJar.java:208)=



2013/10/4 Claudio Martella <c= laudio.martella@gmail.com>
Hi,

you = need to specify the vertex outputformat class (-vof option), e.g. org.apach= e.giraph.io.formats.IdWithValueTextOutputFormat.


On Fri, Oct 4, 2013 at 1:06 PM, Silvio Di gregorio <= ;silvio.di= gregorio@gmail.com> wrote:

Hi,

I hope I have sent to the right address.

i have a graph (directed and unweighted) stored in hdf= s like a adjacency list (140Milions of edges 6Milions of vertex)

=

node<tab>nei= ghbors

23 =A0 2 =A0 1 =A0= =A03 =A0 =A04 =A0 =A03

1 999 99923 909 .......

......

hadoop v= ersion Hadoop 2.0.0-cdh4.3.0 - java 1.6


I have executed the giraph-1.0 connected components example, in this fashio= n

>=A0hadoop jar /usr/local/giraph/girap= h-examples/target/giraph-examples-1.0.0-for-hadoo= p-= 0.20.203.0-jar-with-dependencies.jar org.apache.gira= ph.GiraphRunner org.apache.giraph.examples.ConnectedComponents= Vertex -vif org.apache.giraph.io.formats.IntIntNullTextInputFo= rmat -vip /user/hdfs/lista_adj_txt -op connectedgiraph --workers 4

=A0

and then fail with:

13/10/= 04 09:28:29 INFO utils.ConfigurationUtils: No edge input format specified. = Ensure your InputFormat does not require one.

=A0

13/10/04 09:28= :29 INFO utils.ConfigurationUtils: No output format specified. Ensure your = OutputFormat does not require one.

=A0

13/10/= 04 09:28:30 INFO job.GiraphJob: run: Since checkpointing is disabled (defau= lt), do not allow any task retries (setting mapred.map.max.attempts =3D 0, = old value =3D 4)

=A0

13/10/= 04 09:28:31 WARN mapred.JobClient: Use GenericOptionsParser for parsing the= arguments. Applications should implement Tool for the same.

=A0

13/10/04 09:28= :31 INFO mapred.JobClient: Cleaning up the staging area hdfs://srv-bigdata-dev-01.int.sose.it:8020/user/hdfs/.s= taging/job_201309181636_0535

Exception in thread &q= uot;main" java.lang.IncompatibleClassChangeError: Found interface org.= apache.hadoop.mapreduce.JobContext, but class was expected

=A0=A0=A0=A0=A0=A0=A0 = at org.apache.giraph.bsp.BspOutputFormat.checkOutputSpecs(BspOutputFormat.j= ava:43)

=A0=A0=A0=A0=A0=A0=A0= =A0......

=A0

=A0

Thanks in ad= vance





<= font color=3D"#888888">--
=A0 =A0Claudio Martella
=A0 =A0claudio.martella@g= mail.com=A0 =A0




<= /div>--
=A0 =A0Claudio Martella
=A0 = =A0claudio.= martella@gmail.com=A0 =A0




--
= =A0 =A0Claudio Martella
=A0 =A0claudio.martella@gmail.com=A0 =A0 --20cf303bf9586b92e304e7fda80b--