Return-Path: X-Original-To: apmail-giraph-user-archive@www.apache.org Delivered-To: apmail-giraph-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id CC87D10E7B for ; Fri, 4 Oct 2013 12:35:51 +0000 (UTC) Received: (qmail 12170 invoked by uid 500); 4 Oct 2013 12:35:50 -0000 Delivered-To: apmail-giraph-user-archive@giraph.apache.org Received: (qmail 12132 invoked by uid 500); 4 Oct 2013 12:35:50 -0000 Mailing-List: contact user-help@giraph.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@giraph.apache.org Delivered-To: mailing list user@giraph.apache.org Received: (qmail 12124 invoked by uid 99); 4 Oct 2013 12:35:49 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 04 Oct 2013 12:35:49 +0000 X-ASF-Spam-Status: No, hits=2.5 required=5.0 tests=FREEMAIL_REPLY,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS,WEIRD_PORT X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of silvio.digregorio@gmail.com designates 209.85.223.174 as permitted sender) Received: from [209.85.223.174] (HELO mail-ie0-f174.google.com) (209.85.223.174) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 04 Oct 2013 12:35:45 +0000 Received: by mail-ie0-f174.google.com with SMTP id u16so8647631iet.5 for ; Fri, 04 Oct 2013 05:35:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=RjLb5j86x8E+ovJGAGjT++LYsUtkLxOh/KgSxr1M24c=; b=FZMVfhp3xGKJb5glIWtJinyCTfiy1GAdnAlNyX0HYqxZiWomgU7WQS7kfV9thHQrjE rlMJBKHh7fSg6GZTmKiXSfMEqPd5kZHIxdWmVCuTvZxez2XJ5q0TNIe65zSvbE9KBkF0 hxsoVGWN8WeqlHj9egzA888FTYkH9JTjcAPKq9Ybx4KxtuIsj+/hiSnjbEsqKNSKv1g0 lY8WvJ8UknEbW12lVE9UaxuKoZIMabE2XTj+FYkUqC4NwkNhyR+Gnj7xOx/yhvxHzHxG 2gwjlEorRVcob52lQnMhDCKZXG8FU6pqpjx97HjldugnmEe9N3X7+wZjc53f9hPRcSBD WgxQ== MIME-Version: 1.0 X-Received: by 10.50.122.40 with SMTP id lp8mr6349081igb.24.1380890124739; Fri, 04 Oct 2013 05:35:24 -0700 (PDT) Received: by 10.64.239.226 with HTTP; Fri, 4 Oct 2013 05:35:24 -0700 (PDT) In-Reply-To: References: Date: Fri, 4 Oct 2013 14:35:24 +0200 Message-ID: Subject: Re: connected components example in giraph 1.0 From: Silvio Di gregorio To: user@giraph.apache.org Content-Type: multipart/alternative; boundary=089e015383585bfa4104e7e989ec X-Virus-Checked: Checked by ClamAV on apache.org --089e015383585bfa4104e7e989ec Content-Type: text/plain; charset=ISO-8859-1 org.apache.commons.cli.UnrecognizedOptionException: Unrecognized option: -vof in 1.0 version is -of,--outputFormat Vertex output format -op,--outputPath Vertex output path 2013/10/4 Claudio Martella > did you try the argument (-vof) i suggested? > > > On Fri, Oct 4, 2013 at 2:13 PM, Silvio Di gregorio < > silvio.digregorio@gmail.com> wrote: > >> >> i've specified -of >> org.apache.giraph.io.formats.IdWithValueTextOutputFormat >> >> but the same error was produced >> >> Exception in thread "main" java.lang.IncompatibleClassChangeError: Found >> interface org.apache.hadoop.mapreduce.JobContext, but class was expected >> at >> org.apache.giraph.bsp.BspOutputFormat.checkOutputSpecs(BspOutputFormat.java:43) >> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:984) >> at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:945) >> at java.security.AccessController.doPrivileged(Native Method) >> at javax.security.auth.Subject.doAs(Subject.java:396) >> at >> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408) >> at >> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:945) >> at org.apache.hadoop.mapreduce.Job.submit(Job.java:566) >> at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:596) >> at org.apache.giraph.job.GiraphJob.run(GiraphJob.java:237) >> at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:94) >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84) >> at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >> at java.lang.reflect.Method.invoke(Method.java:597) >> at org.apache.hadoop.util.RunJar.main(RunJar.java:208) >> >> >> >> 2013/10/4 Claudio Martella >> >>> Hi, >>> >>> you need to specify the vertex outputformat class (-vof option), e.g. >>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat. >>> >>> >>> On Fri, Oct 4, 2013 at 1:06 PM, Silvio Di gregorio < >>> silvio.digregorio@gmail.com> wrote: >>> >>>> >>>> Hi, >>>> >>>> I hope I have sent to the right address. >>>> >>>> i have a graph (directed and unweighted) stored in hdfs like a >>>> adjacency list (140Milions of edges 6Milions of vertex) >>>> >>>> nodeneighbors >>>> >>>> 23 2 1 3 4 3 >>>> >>>> 1 999 99923 909 ....... >>>> >>>> ...... >>>> >>>> hadoop version Hadoop 2.0.0-cdh4.3.0 - java 1.6 >>>> >>>> >>>> I have executed the giraph-1.0 connected components example, in this >>>> fashion >>>> >>>>> > hadoop jar /usr/local/giraph/giraph-examples/target/giraph- >>>>> examples-1.0.0-for-hadoop-0.20.203.0-jar-with-dependencies.jar >>>>> org.apache.giraph.GiraphRunner org.apache.giraph.examples.ConnectedComponentsVertex >>>>> -vif org.apache.giraph.io.formats.IntIntNullTextInputFormat -vip >>>>> /user/hdfs/lista_adj_txt -op connectedgiraph --workers 4 >>>> >>>> >>>> >>>> and then fail with: >>>> >>>> 13/10/04 09:28:29 INFO utils.ConfigurationUtils: No edge input format >>>> specified. Ensure your InputFormat does not require one. >>>> >>>> >>>> >>>> 13/10/04 09:28:29 INFO utils.ConfigurationUtils: No output format >>>> specified. Ensure your OutputFormat does not require one. >>>> >>>> >>>> >>>> 13/10/04 09:28:30 INFO job.GiraphJob: run: Since checkpointing is >>>> disabled (default), do not allow any task retries (setting >>>> mapred.map.max.attempts = 0, old value = 4) >>>> >>>> >>>> >>>> 13/10/04 09:28:31 WARN mapred.JobClient: Use GenericOptionsParser for >>>> parsing the arguments. Applications should implement Tool for the same. >>>> >>>> >>>> >>>> 13/10/04 09:28:31 INFO mapred.JobClient: Cleaning up the staging area >>>> hdfs:// >>>> srv-bigdata-dev-01.int.sose.it:8020/user/hdfs/.staging/job_201309181636_0535 >>>> >>>> Exception in thread "main" java.lang.IncompatibleClassChangeError: >>>> Found interface org.apache.hadoop.mapreduce.JobContext, but class was >>>> expected >>>> >>>> at >>>> org.apache.giraph.bsp.BspOutputFormat.checkOutputSpecs(BspOutputFormat.java:43) >>>> >>>> ...... >>>> >>>> >>>> >>>> >>>> >>>> Thanks in advance >>>> >>>> >>> >>> >>> -- >>> Claudio Martella >>> claudio.martella@gmail.com >>> >> >> > > > -- > Claudio Martella > claudio.martella@gmail.com > --089e015383585bfa4104e7e989ec Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable

org.apache.commons.cli.UnrecognizedOptionEx= ception: Unrecognized option: -vof

in 1.0 version is=
=A0-of,--outputFormat <arg> =A0 =A0 =A0 =A0 =A0 =A0 = =A0 Vertex output format
=A0-op,--outputPath <arg> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 Vertex= output path


2013/10/4 Claudio Martella <claud= io.martella@gmail.com>
did you try the argument (-= vof) i suggested?

On Fri, Oct 4, 2013 at 2:13 PM, Silvio Di gregor= io <silvio.digregorio@gmail.com> wrote:

i've spe= cified =A0-of =A0org.apache.giraph.io.formats.IdWithValueTextOutputFormat
but the same error was produced

Exception in thread "main" java.lang.IncompatibleClassChangeErro= r: Found interface org.apache.hadoop.mapreduce.JobContext, but class was ex= pected
=A0 =A0 =A0 =A0 at org.apache.giraph.bsp.BspOutputFormat.checkOutputSp= ecs(BspOutputFormat.java:43)
=A0 =A0 =A0 =A0 at org.apache.= hadoop.mapred.JobClient$2.run(JobClient.java:984)
=A0 =A0 =A0 =A0= at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:945)
=A0 =A0 =A0 =A0 at java.security.AccessController.doPrivileged(Native = Method)
=A0 =A0 =A0 =A0 at javax.security.auth.Subject.doAs(Subje= ct.java:396)
=A0 =A0 =A0 =A0 at org.apache.hadoop.security.UserGr= oupInformation.doAs(UserGroupInformation.java:1408)
=A0 =A0 =A0 =A0 at org.apache.hadoop.mapred.JobClient.submitJobInterna= l(JobClient.java:945)
=A0 =A0 =A0 =A0 at org.apache.hadoop.mapred= uce.Job.submit(Job.java:566)
=A0 =A0 =A0 =A0 at org.apache.hadoop= .mapreduce.Job.waitForCompletion(Job.java:596)
=A0 =A0 =A0 =A0 at org.apache.giraph.job.GiraphJob.run(GiraphJob.java:= 237)
=A0 =A0 =A0 =A0 at org.apache.giraph.GiraphRunner.run(Giraph= Runner.java:94)
=A0 =A0 =A0 =A0 at org.apache.hadoop.util.ToolRun= ner.run(ToolRunner.java:70)
=A0 =A0 =A0 =A0 at org.apache.hadoop.util.ToolRunner.run(ToolRunner.ja= va:84)
=A0 =A0 =A0 =A0 at org.apache.giraph.GiraphRunner.main(Gir= aphRunner.java:124)
=A0 =A0 =A0 =A0 at sun.reflect.NativeMethodAc= cessorImpl.invoke0(Native Method)
=A0 =A0 =A0 =A0 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeM= ethodAccessorImpl.java:39)
=A0 =A0 =A0 =A0 at sun.reflect.Delegat= ingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
=A0 =A0 =A0 =A0 at java.lang.reflect.Method.invoke(Method.java:597)
=A0 =A0 =A0 =A0 at org.apache.hadoop.util.RunJar.main(RunJar.java:208)=



2013/10/4 Claudio Martella <c= laudio.martella@gmail.com>
Hi,

you = need to specify the vertex outputformat class (-vof option), e.g. org.apach= e.giraph.io.formats.IdWithValueTextOutputFormat.


On Fri, Oct 4, 2013 at 1:06 PM, Silvio Di gregorio <= ;silvio.di= gregorio@gmail.com> wrote:

Hi,

I hope I have sent to the right address.

i have a graph (directed and unweighted) stored in hdf= s like a adjacency list (140Milions of edges 6Milions of vertex)

=

node<tab>nei= ghbors

23 =A0 2 =A0 1 =A0= =A03 =A0 =A04 =A0 =A03

1 999 99923 909 .......

......

hadoop v= ersion Hadoop 2.0.0-cdh4.3.0 - java 1.6


I have executed the giraph-1.0 connected components example, in this fashio= n

>=A0hadoop jar /usr/local/giraph/girap= h-examples/target/giraph-examples-1.0.0-for-hadoo= p-= 0.20.203.0-jar-with-dependencies.jar org.apache.gira= ph.GiraphRunner org.apache.giraph.examples.ConnectedComponents= Vertex -vif org.apache.giraph.io.formats.IntIntNullTextInputFo= rmat -vip /user/hdfs/lista_adj_txt -op connectedgiraph --workers 4

=A0

and then fail with:

13/10/= 04 09:28:29 INFO utils.ConfigurationUtils: No edge input format specified. = Ensure your InputFormat does not require one.

=A0

13/10/04 09:28= :29 INFO utils.ConfigurationUtils: No output format specified. Ensure your = OutputFormat does not require one.

=A0

13/10/= 04 09:28:30 INFO job.GiraphJob: run: Since checkpointing is disabled (defau= lt), do not allow any task retries (setting mapred.map.max.attempts =3D 0, = old value =3D 4)

=A0

13/10/= 04 09:28:31 WARN mapred.JobClient: Use GenericOptionsParser for parsing the= arguments. Applications should implement Tool for the same.

=A0

13/10/04 09:28= :31 INFO mapred.JobClient: Cleaning up the staging area hdfs://srv-bigdata-dev-01.int.sose.it:8020/user/hdfs/.s= taging/job_201309181636_0535

Exception in thread &q= uot;main" java.lang.IncompatibleClassChangeError: Found interface org.= apache.hadoop.mapreduce.JobContext, but class was expected

=A0=A0=A0=A0=A0=A0=A0 = at org.apache.giraph.bsp.BspOutputFormat.checkOutputSpecs(BspOutputFormat.j= ava:43)

=A0=A0=A0=A0=A0=A0=A0= =A0......

=A0

=A0

Thanks in ad= vance





<= font color=3D"#888888">--
=A0 =A0Claudio Martella
=A0 =A0claudio.martella@g= mail.com=A0 =A0




<= /div>--
=A0 =A0Claudio = Martella
=A0 =A0claudio.martella@gmail.com=A0 =A0

--089e015383585bfa4104e7e989ec--