giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthew Laird <lai...@sfu.ca>
Subject Re: connected components example in giraph 1.0
Date Tue, 08 Oct 2013 20:34:42 GMT
Thanks, I appreciate the feedback! It's why open source projects are great.

So just to be clear, I should grab the 1.0 release, and you've 
successfully built it against a Hadoop 2.0.x release?  I add that patch 
from Claudio first, then use -Dhadoop.version=2.0.5 to build?  It will 
still have the MR API in place?

I'll get through that then see what happens with this whole split 
master-worker setup I have on my Hadoop installation which caused the 
examples to fail.  Then on to trying to load my billion vertices and 
clustering the graph (I basically want all the sub-groups of connected 
components, these will be the related genes across all sequenced bacteria).

Thanks again!

On 13-10-07 12:56 PM, Silvio Di gregorio wrote:
> with a working day in my shoulders and my children around  i gave you
> random version,
> tomorrow morning in farm i'll be more precise.
> though i have hadoop 2.0 i use MR1 not YARN.
> I'm sorry again.
>
>
> 2013/10/7 Matthew Laird <lairdm@sfu.ca <mailto:lairdm@sfu.ca>>
>
>     Hmm, it seems there have been some recent pushes that have broken
>     trunk as I discovered yesterday. :)
>
>     But I guess that brings me back to my original question, to build
>     against 2.0.5 using the MR interface rather than YARN as Milinda
>     suggested, what profile would I use?  Would I let it think it's
>     0.20xxx or tell it 2.0.5, or something else?
>
>     Thanks.
>
>
>     On 13-10-07 11:50 AM, Claudio Martella wrote:
>
>         trunk is 1.1.0-SNAPSHOT. I believe the date refers to the last
>         update to
>         the site.
>
>
>         On Mon, Oct 7, 2013 at 8:48 PM, Matthew Laird <lairdm@sfu.ca
>         <mailto:lairdm@sfu.ca>
>         <mailto:lairdm@sfu.ca <mailto:lairdm@sfu.ca>>> wrote:
>
>              Thanks, which is my next curiosity... if I head over to
>         http://giraph.apache.org/ I see at the top is says
>
>         "Version: 1.1.0-SNAPSHOT | Last Published: 2013-08-10"
>
>              However it's not clickable and the only download links I
>         see are for
>              1.0.0.  I'm assuming trunk in git isn't the same as the 1.1.0
>              snapshot since more pushes have occurred since August, and
>         I don't
>              see a tag in git for 1.1.0
>              (https://git-wip-us.apache.____org/repos/asf?p=giraph.git
>         <https://git-wip-us.apache.__org/repos/asf?p=giraph.git
>         <https://git-wip-us.apache.org/repos/asf?p=giraph.git>>). So where
>
>              does one get 1.1.0-SNAPSHOT from?
>
>              Thanks.
>
>
>              On 13-10-07 11:39 AM, Silvio Di gregorio wrote:
>
>                  Hi Matthew
>                  I use giraph 1.1.0 and cdh4 hadoop 0.23
>
>                  the patch of Marco:
>
>         http://www.mail-archive.com/______user@giraph.apache.org/______msg00945/check.diff
>         <http://www.mail-archive.com/____user@giraph.apache.org/____msg00945/check.diff>
>         <http://www.mail-archive.com/____user@giraph.apache.org/____msg00945/check.diff
>         <http://www.mail-archive.com/__user@giraph.apache.org/__msg00945/check.diff>>
>
>
>         <http://www.mail-archive.com/____user@giraph.apache.org/____msg00945/check.diff
>         <http://www.mail-archive.com/__user@giraph.apache.org/__msg00945/check.diff>
>         <http://www.mail-archive.com/__user@giraph.apache.org/__msg00945/check.diff
>         <http://www.mail-archive.com/user@giraph.apache.org/msg00945/check.diff>>>
>
>                  and build
>
>                  mvn clean install -P hadoop_cdh4.1.2
>
>
>
>                  Il giorno 07/ott/2013 20:20, "Matthew Laird"
>         <lairdm@sfu.ca <mailto:lairdm@sfu.ca>
>         <mailto:lairdm@sfu.ca <mailto:lairdm@sfu.ca>>
>         <mailto:lairdm@sfu.ca <mailto:lairdm@sfu.ca>
>         <mailto:lairdm@sfu.ca <mailto:lairdm@sfu.ca>>>> ha scritto:
>
>
>
>                       Hello, so I'm watching this thread with interest
>         as well, since
>                       that's the exact example I'll be trying to run as
>         well, and
>                  I see
>                       Silvio appears to be asking about running Giraph 1.0
>                  against Hadoop
>                       2.0.0.
>
>                       I'm curious, is that a known working combination,
>         1.0 vs
>                  2.0.x?  And
>                       if so, are there patches to apply first and what maven
>                  command line
>                       should I build with? (ie. will a
>         -Dhadoop.version=2.0.5
>                  give the
>                       needed MR based API or will it use YARN?)  I
>         figure it's
>                  better to
>                       ask than aimlessly try all the combinations. :)
>
>                       Thanks!
>
>                       On 13-10-07 11:01 AM, Claudio Martella wrote:
>
>                           try to increase the mappers' heap space through
>                           mapred.child.java.opts.
>
>
>                           On Mon, Oct 7, 2013 at 1:18 PM, Silvio Di gregorio
>         <silvio.digregorio@gmail.com
>         <mailto:silvio.digregorio@gmail.com>
>         <mailto:silvio.digregorio@__gmail.com
>         <mailto:silvio.digregorio@gmail.com>>
>         <mailto:silvio.digregorio@
>         <mailto:silvio.digregorio@>__gm__ail.com <http://gmail.com>
>         <mailto:silvio.digregorio@__gmail.com
>         <mailto:silvio.digregorio@gmail.com>>>
>         <mailto:silvio.digregorio@ <mailto:silvio.digregorio@>
>         <mailto:silvio.digregorio@
>         <mailto:silvio.digregorio@>>__g__m__ail.com <http://gm__ail.com>
>         <http://gmail.com>
>
>
>         <mailto:silvio.digregorio@
>         <mailto:silvio.digregorio@>__gm__ail.com <http://gmail.com>
>         <mailto:silvio.digregorio@__gmail.com
>         <mailto:silvio.digregorio@gmail.com>>>>> wrote:
>
>
>                                I think now my issue is another, thanks
>         to all.
>
>
>
>
>                                java.lang.______IllegalStateException:
>         run: Caught an
>
>
>                           unrecoverable exception waitFor:
>         ExecutionException
>                  occurred
>                           while waiting for
>
>
>         org.apache.giraph.utils.______ProgressableUtils$______FutureWaitable@73a01e8a
>                                    at
>
>
>         org.apache.giraph.graph.______GraphMapper.run(GraphMapper.______java:102)
>
>
>
>                                    ......
>                                Caused by:
>         java.lang.______IllegalStateException:
>
>                  waitFor:
>
>                           ExecutionException occurred while waiting for
>
>
>         org.apache.giraph.utils.______ProgressableUtils$______FutureWaitable@73a01e8a
>                                    at
>
>
>         org.apache.giraph.utils.______ProgressableUtils.waitFor(______ProgressableUtils.java:151)
>
>
>
>
>                                    ... 7 more
>                                Caused by:
>                  java.util.concurrent.______ExecutionException:
>                           java.lang.OutOfMemoryError:*______Java heap space*
>                                    at
>
>
>         org.apache.giraph.utils.______ProgressableUtils.waitFor(______ProgressableUtils.java:143)
>
>
>
>
>
>                                    ... 13 more
>
>
>
>                                2013/10/7 Claudio Martella
>         <claudio.martella@gmail.com <mailto:claudio.martella@gmail.com>
>         <mailto:claudio.martella@__gmail.com
>         <mailto:claudio.martella@gmail.com>>
>         <mailto:claudio.martella@
>         <mailto:claudio.martella@>__gma__il.com <http://gmail.com>
>         <mailto:claudio.martella@__gmail.com
>         <mailto:claudio.martella@gmail.com>>>
>         <mailto:claudio.martella@ <mailto:claudio.martella@>
>         <mailto:claudio.martella@
>         <mailto:claudio.martella@>>__gm__a__il.com <http://gma__il.com>
>         <http://gmail.com>
>
>
>         <mailto:claudio.martella@
>         <mailto:claudio.martella@>__gma__il.com <http://gmail.com>
>         <mailto:claudio.martella@__gmail.com
>         <mailto:claudio.martella@gmail.com>>>>>
>
>                                    OK, thanks.
>
>                                    I really have to push that patch in.
>
>
>                                    On Mon, Oct 7, 2013 at 12:17 PM,
>         Silvio Di
>                  gregorio
>         <silvio.digregorio@gmail.com <mailto:silvio.digregorio@gmail.com>
>         <mailto:silvio.digregorio@__gmail.com
>         <mailto:silvio.digregorio@gmail.com>>
>         <mailto:silvio.digregorio@
>         <mailto:silvio.digregorio@>__gm__ail.com <http://gmail.com>
>         <mailto:silvio.digregorio@__gmail.com
>         <mailto:silvio.digregorio@gmail.com>>>
>         <mailto:silvio.digregorio@ <mailto:silvio.digregorio@>
>         <mailto:silvio.digregorio@
>         <mailto:silvio.digregorio@>>__g__m__ail.com <http://gm__ail.com>
>         <http://gmail.com>
>
>
>         <mailto:silvio.digregorio@
>         <mailto:silvio.digregorio@>__gm__ail.com <http://gmail.com>
>         <mailto:silvio.digregorio@__gmail.com
>         <mailto:silvio.digregorio@gmail.com>>>>> wrote:
>
>                                        yes i do,
>                                        i have seen this in your post in:
>         http://www.mail-archive.com/______user@giraph.apache.org/______msg00957.html
>         <http://www.mail-archive.com/____user@giraph.apache.org/____msg00957.html>
>         <http://www.mail-archive.com/____user@giraph.apache.org/____msg00957.html
>         <http://www.mail-archive.com/__user@giraph.apache.org/__msg00957.html>>
>
>
>         <http://www.mail-archive.com/____user@giraph.apache.org/____msg00957.html
>         <http://www.mail-archive.com/__user@giraph.apache.org/__msg00957.html>
>         <http://www.mail-archive.com/__user@giraph.apache.org/__msg00957.html
>         <http://www.mail-archive.com/user@giraph.apache.org/msg00957.html>>>
>
>                                        excuse me if  i had checked in
>         the mail-achive
>                           first I would
>                                        have avoided the last post.
>
>                                        Now zk issue are resolved.
>
>
>                                        2013/10/7 Claudio Martella
>         <claudio.martella@gmail.com <mailto:claudio.martella@gmail.com>
>         <mailto:claudio.martella@__gmail.com
>         <mailto:claudio.martella@gmail.com>>
>         <mailto:claudio.martella@
>         <mailto:claudio.martella@>__gma__il.com <http://gmail.com>
>         <mailto:claudio.martella@__gmail.com
>         <mailto:claudio.martella@gmail.com>>>
>         <mailto:claudio.martella@ <mailto:claudio.martella@>
>         <mailto:claudio.martella@
>         <mailto:claudio.martella@>>__gm__a__il.com <http://gma__il.com>
>         <http://gmail.com>
>
>
>         <mailto:claudio.martella@
>         <mailto:claudio.martella@>__gma__il.com <http://gmail.com>
>         <mailto:claudio.martella@__gmail.com
>         <mailto:claudio.martella@gmail.com>>>>>
>
>                                            Can you try applying this one
>         first?
>
>         <<< <
>         http://www.mail-archive.com/______user@giraph.apache.org/______msg00945/check.diff
>         <http://www.mail-archive.com/____user@giraph.apache.org/____msg00945/check.diff>
>         <http://www.mail-archive.com/____user@giraph.apache.org/____msg00945/check.diff
>         <http://www.mail-archive.com/__user@giraph.apache.org/__msg00945/check.diff>>
>
>
>         <http://www.mail-archive.com/____user@giraph.apache.org/____msg00945/check.diff
>         <http://www.mail-archive.com/__user@giraph.apache.org/__msg00945/check.diff>
>         <http://www.mail-archive.com/__user@giraph.apache.org/__msg00945/check.diff
>         <http://www.mail-archive.com/user@giraph.apache.org/msg00945/check.diff>>><<<<
>
>
>                                            On Mon, Oct 7, 2013 at 8:40
>         AM, Silvio
>                  Di gregorio
>         <silvio.digregorio@gmail.com <mailto:silvio.digregorio@gmail.com>
>         <mailto:silvio.digregorio@__gmail.com
>         <mailto:silvio.digregorio@gmail.com>>
>         <mailto:silvio.digregorio@
>         <mailto:silvio.digregorio@>__gm__ail.com <http://gmail.com>
>         <mailto:silvio.digregorio@__gmail.com
>         <mailto:silvio.digregorio@gmail.com>>>
>         <mailto:silvio.digregorio@ <mailto:silvio.digregorio@>
>         <mailto:silvio.digregorio@
>         <mailto:silvio.digregorio@>>__g__m__ail.com <http://gm__ail.com>
>         <http://gmail.com>
>
>
>         <mailto:silvio.digregorio@
>         <mailto:silvio.digregorio@>__gm__ail.com <http://gmail.com>
>         <mailto:silvio.digregorio@__gmail.com
>         <mailto:silvio.digregorio@gmail.com>>>>> wrote:
>
>
>
>
>
>
>
>                                                *As i said i have builded*
>
>
>
>
>
>
>
>         /giraph-examples-1.0.0-for-______hadoop-2.0.0-cdh4.1.2-jar-______with-dependencies.jar/
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>                                                *for cdh4, successfully.
>         The job
>                  start to
>                           monitoring the success rate:*
>
>
>
>
>
>
>
>
>
>
>
>
>
>                                                /13/10/07 08:28:45 INFO
>                  mapred.JobClient:
>                             map 0% reduce 0%/
>
>
>
>
>                                                *but then*
>
>
>
>
>                                                /Error running child
>
>                    java.lang.______IllegalStateException: run:
>
>
>                           Caught an unrecoverable exception
>                  java.io.FileNotFoundException:
>                           File
>
>         _bsp/_defaultZkManagerDir/job_______201309181636_0678/___zkServer
>
>
>                           does not exist.
>                                                /
>
>
>
>
>
>
>
>
>
>
>
>
>
>                                                /.../
>
>
>
>
>
>
>
>                                                /Caused by:
>                  java.lang.RuntimeException:
>                           java.io.FileNotFoundException: File
>
>
>         _bsp/_defaultZkManagerDir/job_______201309181636_0678/___zkServer does
>
>
>                           not exist./
>
>
>
>
>
>
>
>
>
>
>                                                2013/10/5 Silvio Di gregorio
>         <silvio.digregorio@gmail.com <mailto:silvio.digregorio@gmail.com>
>         <mailto:silvio.digregorio@__gmail.com
>         <mailto:silvio.digregorio@gmail.com>>
>         <mailto:silvio.digregorio@
>         <mailto:silvio.digregorio@>__gm__ail.com <http://gmail.com>
>         <mailto:silvio.digregorio@__gmail.com
>         <mailto:silvio.digregorio@gmail.com>>>
>         <mailto:silvio.digregorio@ <mailto:silvio.digregorio@>
>         <mailto:silvio.digregorio@
>         <mailto:silvio.digregorio@>>__g__m__ail.com <http://gm__ail.com>
>         <http://gmail.com>
>
>
>         <mailto:silvio.digregorio@
>         <mailto:silvio.digregorio@>__gm__ail.com <http://gmail.com>
>         <mailto:silvio.digregorio@__gmail.com
>         <mailto:silvio.digregorio@gmail.com>>>>>
>
>                                                    I ha ve build w/
>                  hadoop_cdh4.1.2 parameter.
>                                                    Something is changed,
>         monday i
>                  report the
>                                                    result. Now the farm
>         is closed.
>
>                                                    Il giorno 05/ott/2013
>         14:06,
>         "Claudio
>                           Martella"
>         <claudio.martella@gmail.com <mailto:claudio.martella@gmail.com>
>         <mailto:claudio.martella@__gmail.com
>         <mailto:claudio.martella@gmail.com>>
>         <mailto:claudio.martella@
>         <mailto:claudio.martella@>__gma__il.com <http://gmail.com>
>         <mailto:claudio.martella@__gmail.com
>         <mailto:claudio.martella@gmail.com>>>
>         <mailto:claudio.martella@ <mailto:claudio.martella@>
>         <mailto:claudio.martella@
>         <mailto:claudio.martella@>>__gm__a__il.com <http://gma__il.com>
>         <http://gmail.com>
>
>
>         <mailto:claudio.martella@
>         <mailto:claudio.martella@>__gma__il.com <http://gmail.com>
>         <mailto:claudio.martella@__gmail.com
>         <mailto:claudio.martella@gmail.com>>>>> ha scritto:
>
>                                                        Oh, right, -vof is in
>                  trunk. Anyway
>                           it looks
>                                                        like you built
>         giraph for
>                  the wrong
>                           profile.
>                                                        You mentioned you're
>                  running on
>                           2.0, but
>                                                        your giraph is
>         built for
>         0.20.203 <tel:0.20.203> <tel:0.20.203 <tel:0.20.203>>
>         <tel:0.20.203 <tel:0.20.203> <tel:0.20.203 <tel:0.20.203>>>
>         <tel:0.20.203 <tel:0.20.203> <tel:0.20.203 <tel:0.20.203>>
>         <tel:0.20.203 <tel:0.20.203> <tel:0.20.203 <tel:0.20.203>>>>.
try
>
>                  building with a profile
>
>                                                        for your hadoop
>         version.
>
>
>                                                        On Fri, Oct 4,
>         2013 at
>                  2:35 PM,
>                           Silvio Di
>                                                        gregorio
>         <silvio.digregorio@gmail.com <mailto:silvio.digregorio@gmail.com>
>         <mailto:silvio.digregorio@__gmail.com
>         <mailto:silvio.digregorio@gmail.com>>
>         <mailto:silvio.digregorio@
>         <mailto:silvio.digregorio@>__gm__ail.com <http://gmail.com>
>         <mailto:silvio.digregorio@__gmail.com
>         <mailto:silvio.digregorio@gmail.com>>>
>         <mailto:silvio.digregorio@ <mailto:silvio.digregorio@>
>         <mailto:silvio.digregorio@
>         <mailto:silvio.digregorio@>>__g__m__ail.com <http://gm__ail.com>
>         <http://gmail.com>
>
>         <mailto:silvio.digregorio@
>         <mailto:silvio.digregorio@>__gm__ail.com <http://gmail.com>
>         <mailto:silvio.digregorio@__gmail.com
>         <mailto:silvio.digregorio@gmail.com>>>>> wrote:
>
>
>
>
>           org.apache.commons.cli.______UnrecognizedOptionException:
>
>
>                                                            Unrecognized
>         option: -vof
>
>                                                            in 1.0 version is
>
>         -of,--outputFormat <arg>
>                                                            Vertex output
>         format
>
>         -op,--outputPath <arg>
>                                                            Vertex output
>         path
>
>
>
>                                                            2013/10/4
>         Claudio Martella
>         <claudio.martella@gmail.com <mailto:claudio.martella@gmail.com>
>         <mailto:claudio.martella@__gmail.com
>         <mailto:claudio.martella@gmail.com>>
>         <mailto:claudio.martella@
>         <mailto:claudio.martella@>__gma__il.com <http://gmail.com>
>         <mailto:claudio.martella@__gmail.com
>         <mailto:claudio.martella@gmail.com>>>
>         <mailto:claudio.martella@ <mailto:claudio.martella@>
>         <mailto:claudio.martella@
>         <mailto:claudio.martella@>>__gm__a__il.com <http://gma__il.com>
>         <http://gmail.com>
>
>
>         <mailto:claudio.martella@
>         <mailto:claudio.martella@>__gma__il.com <http://gmail.com>
>         <mailto:claudio.martella@__gmail.com
>         <mailto:claudio.martella@gmail.com>>>>>
>
>                                                                did you
>         try the
>                  argument
>                           (-vof) i
>                                                                suggested?
>
>
>                                                                On Fri,
>         Oct 4,
>                  2013 at 2:13 PM,
>                                                                Silvio Di
>         gregorio
>         <silvio.digregorio@gmail.com <mailto:silvio.digregorio@gmail.com>
>         <mailto:silvio.digregorio@__gmail.com
>         <mailto:silvio.digregorio@gmail.com>>
>         <mailto:silvio.digregorio@
>         <mailto:silvio.digregorio@>__gm__ail.com <http://gmail.com>
>         <mailto:silvio.digregorio@__gmail.com
>         <mailto:silvio.digregorio@gmail.com>>>
>         <mailto:silvio.digregorio@ <mailto:silvio.digregorio@>
>         <mailto:silvio.digregorio@
>         <mailto:silvio.digregorio@>>__g__m__ail.com <http://gm__ail.com>
>         <http://gmail.com>
>
>
>         <mailto:silvio.digregorio@
>         <mailto:silvio.digregorio@>__gm__ail.com <http://gmail.com>
>         <mailto:silvio.digregorio@__gmail.com
>         <mailto:silvio.digregorio@gmail.com>>>>> wrote:
>
>
>                                                                    i've
>         specified
>                    -of
>
>
>
>         org.apache.giraph.io.formats.______IdWithValueTextOutputFormat
>
>
>
>                                                                    but
>         the same
>                  error was
>                           produced
>
>
>         Exception in
>                  thread "main"
>
>                           java.lang.______IncompatibleClassChangeError:
>                                                                    Found
>         interface
>
>                           org.apache.hadoop.mapreduce.______JobContext,
>
>
>                                                                    but
>         class was
>                  expected
>
>             at
>
>
>
>         org.apache.giraph.bsp.______BspOutputFormat.______checkOutputSpecs(______BspOutputFormat.java:43)
>
>             at
>
>
>
>         org.apache.hadoop.mapred.______JobClient$2.run(JobClient.______java:984)
>
>             at
>
>
>
>         org.apache.hadoop.mapred.______JobClient$2.run(JobClient.______java:945)
>
>             at
>
>
>           java.security.______AccessController.doPrivileged(______Native
>                                                                    Method)
>
>             at
>
>
>           javax.security.auth.Subject.______doAs(Subject.java:396)
>
>             at
>
>
>
>         org.apache.hadoop.security.______UserGroupInformation.doAs(______UserGroupInformation.java:______1408)
>
>             at
>
>
>
>         org.apache.hadoop.mapred.______JobClient.submitJobInternal(______JobClient.java:945)
>
>             at
>
>
>           org.apache.hadoop.mapreduce.______Job.submit(Job.java:566)
>
>             at
>
>
>
>         org.apache.hadoop.mapreduce.______Job.waitForCompletion(Job.______java:596)
>
>             at
>
>
>
>         org.apache.giraph.job.______GiraphJob.run(GiraphJob.java:______237)
>
>             at
>
>
>
>         org.apache.giraph.______GiraphRunner.run(GiraphRunner.______java:94)
>
>             at
>
>
>
>         org.apache.hadoop.util.______ToolRunner.run(ToolRunner.______java:70)
>
>             at
>
>
>
>         org.apache.hadoop.util.______ToolRunner.run(ToolRunner.______java:84)
>
>             at
>
>
>
>         org.apache.giraph.______GiraphRunner.main(______GiraphRunner.java:124)
>
>             at
>
>
>           sun.reflect.______NativeMethodAccessorImpl.______invoke0(Native
>                                                                    Method)
>
>             at
>
>
>
>         sun.reflect.______NativeMethodAccessorImpl.______invoke(______NativeMethodAccessorImpl.java:______39)
>
>             at
>
>
>
>         sun.reflect.______DelegatingMethodAccessorImpl.______invoke(______DelegatingMethodAccessorImpl.______java:25)
>
>             at
>
>
>           java.lang.reflect.Method.______invoke(Method.java:597)
>
>             at
>
>
>           org.apache.hadoop.util.RunJar.______main(RunJar.java:208)
>
>
>
>
>
>                                                                    2013/10/4
>                  Claudio Martella
>         <claudio.martella@gmail.com <mailto:claudio.martella@gmail.com>
>         <mailto:claudio.martella@__gmail.com
>         <mailto:claudio.martella@gmail.com>>
>         <mailto:claudio.martella@
>         <mailto:claudio.martella@>__gma__il.com <http://gmail.com>
>         <mailto:claudio.martella@__gmail.com
>         <mailto:claudio.martella@gmail.com>>>
>         <mailto:claudio.martella@ <mailto:claudio.martella@>
>         <mailto:claudio.martella@
>         <mailto:claudio.martella@>>__gm__a__il.com <http://gma__il.com>
>         <http://gmail.com>
>
>
>         <mailto:claudio.martella@
>         <mailto:claudio.martella@>__gma__il.com <http://gmail.com>
>         <mailto:claudio.martella@__gmail.com
>         <mailto:claudio.martella@gmail.com>>>>>
>
>                                                                        Hi,
>
>
>         you need
>                  to specify the
>
>         vertex
>                  outputformat
>                           class
>                                                                        (-vof
>                  option), e.g.
>
>
>
>         org.apache.giraph.io.formats.________IdWithValueTextOutputFormat.
>
>
>
>
>
>         On Fri,
>                  Oct 4, 2013
>                           at 1:06
>
>         PM, Silvio
>                  Di gregorio
>         <silvio.digregorio@gmail.com <mailto:silvio.digregorio@gmail.com>
>         <mailto:silvio.digregorio@__gmail.com
>         <mailto:silvio.digregorio@gmail.com>>
>         <mailto:silvio.digregorio@
>         <mailto:silvio.digregorio@>__gm__ail.com <http://gmail.com>
>         <mailto:silvio.digregorio@__gmail.com
>         <mailto:silvio.digregorio@gmail.com>>>
>         <mailto:silvio.digregorio@ <mailto:silvio.digregorio@>
>         <mailto:silvio.digregorio@
>         <mailto:silvio.digregorio@>>__g__m__ail.com <http://gm__ail.com>
>         <http://gmail.com>
>
>
>         <mailto:silvio.digregorio@
>         <mailto:silvio.digregorio@>__gm__ail.com <http://gmail.com>
>         <mailto:silvio.digregorio@__gmail.com
>         <mailto:silvio.digregorio@gmail.com>>>>>
>
>         wrote:
>
>
>
>            Hi,
>
>
>            I hope
>                  I have
>                           sent to
>
>            the
>                  right address.
>
>
>            i have
>                  a graph
>                           (directed
>
>            and
>                  unweighted)
>                           stored
>
>            in
>                  hdfs like a
>                           adjacency
>
>            list
>                  (140Milions of
>
>            edges
>                  6Milions
>                           of vertex)
>
>
>                    node<tab>neighbors
>
>
>            23   2
>                     1    3
>                               4    3
>
>         1 999 99923 <tel:1%20999%2099923> <tel:1%20999%2099923>
>         <tel:1%20999%2099923>
>         <tel:1%20999%2099923>
>
>            909
>                  .......
>
>
>            ......
>
>
>            hadoop
>                  version
>                           Hadoop
>
>                    2.0.0-cdh4.3.0
>                           - java 1.6
>
>
>
>            I have
>                  executed the
>
>            giraph-1.0
>                           connected
>
>            components
>                           example, in
>
>            this
>                  fashion
>
>          >  hadoop jar
>
>
>
>         /usr/local/giraph/giraph-______examples/target/giraph-______examples-1.0.0-for-hadoop-0.20.203.0
>         <tel:0.20.203.0>
>
>         <tel:0.20.203.0 <tel:0.20.203.0>>
>         <tel:0.20.203.0 <tel:0.20.203.0> <tel:0.20.203.0 <tel:0.20.203.0>>>
>         <tel:0.20.203.0 <tel:0.20.203.0> <tel:0.20.203.0
>         <tel:0.20.203.0>> <tel:0.20.203.0 <tel:0.20.203.0>
>         <tel:0.20.203.0 <tel:0.20.203.0>>>>-jar-with-__dependencies.____jar
>
>                           org.apache.giraph.GiraphRunner
>
>
>           org.apache.giraph.examples.______ConnectedComponentsVertex
>
>                -vif
>
>
>           org.apache.giraph.io.formats.______IntIntNullTextInputFormat
>
>
>
>                -vip
>
>                           /user/hdfs/lista_adj_txt
>
>                -op
>                           connectedgiraph
>
>                    --workers 4
>
>
>            and
>                  then fail with:
>
>
>            13/10/04
>                           09:28:29 INFO
>
>                           utils.ConfigurationUtils: No
>
>            edge
>                  input format
>
>            specified.
>                           Ensure your
>
>                    InputFormat
>                           does not
>
>                    require one.
>
>
>            13/10/04
>                           09:28:29 INFO
>
>                           utils.ConfigurationUtils: No
>
>            output
>                  format
>                           specified.
>
>            Ensure
>                  your
>                           OutputFormat
>
>            does not
>                           require one.
>
>
>            13/10/04
>                           09:28:30 INFO
>
>                    job.GiraphJob: run:
>
>            Since
>                           checkpointing is
>
>            disabled
>                           (default), do
>
>            not
>                  allow any task
>
>                    retries (setting
>
>                           mapred.map.max.attempts
>
>            = 0,
>                  old value = 4)
>
>
>            13/10/04
>                           09:28:31 WARN
>
>                           mapred.JobClient: Use
>
>                           GenericOptionsParser for
>
>                    parsing the
>                           arguments.
>
>                    Applications should
>
>                    implement Tool
>                           for the same.
>
>
>            13/10/04
>                           09:28:31 INFO
>
>                    mapred.JobClient:
>
>                    Cleaning up the
>                           staging
>
>            area
>
>                           hdfs://srv-bigdata-dev-01.int
>         <http://srv-bigdata-dev-01.int>
>         <http://srv-bigdata-dev-01.int__>.____sose.it:8020/user/hdfs/.______staging/job_201309181636___0535
>         <http://sose.it:8020/user/hdfs/.____staging/job_201309181636_0535>
>         <http://sose.it:8020/user/__hdfs/.__staging/job___201309181636_0535
>         <http://sose.it:8020/user/hdfs/.__staging/job_201309181636_0535>>
>
>         <http://srv-bigdata-dev-01.__i__nt.sose.it:8020/user/hdfs/.____staging/job_201309181636_0535
>         <http://int.sose.it:8020/user/hdfs/.__staging/job_201309181636_0535>
>         <http://srv-bigdata-dev-01.__int.sose.it:8020/user/hdfs/.__staging/job_201309181636_0535
>         <http://srv-bigdata-dev-01.int.sose.it:8020/user/hdfs/.staging/job_201309181636_0535>>__>
>         <http://srv-bigdata-dev-01.__i____nt.sose.it:8020/user/hdfs/.______staging/job_201309181636___0535
>         <http://i__nt.sose.it:8020/user/hdfs/.____staging/job_201309181636_0535>
>         <http://int.sose.it:8020/user/__hdfs/.__staging/job___201309181636_0535
>         <http://int.sose.it:8020/user/hdfs/.__staging/job_201309181636_0535>>
>
>         <http://srv-bigdata-dev-01.__i__nt.sose.it:8020/user/hdfs/.____staging/job_201309181636_0535
>         <http://int.sose.it:8020/user/hdfs/.__staging/job_201309181636_0535>
>         <http://srv-bigdata-dev-01.__int.sose.it:8020/user/hdfs/.__staging/job_201309181636_0535
>         <http://srv-bigdata-dev-01.int.sose.it:8020/user/hdfs/.staging/job_201309181636_0535>>__>__>
>
>
>                    Exception in thread
>         "main"
>
>                           java.lang.______IncompatibleClassChangeError:
>
>            Found
>                  interface
>
>                           org.apache.hadoop.mapreduce.______JobContext,
>
>
>
>            but
>                  class was
>                           expected
>
>
>                     at
>
>
>
>         org.apache.giraph.bsp.______BspOutputFormat.______checkOutputSpecs(______BspOutputFormat.java:43)
>
>
>
>
>            ......
>
>
>            Thanks
>                  in advance
>
>
>
>
>
>                                                                        --
>
>                    Claudio Martella
>         claudio.martella@gmail.com <mailto:claudio.martella@gmail.com>
>         <mailto:claudio.martella@__gmail.com
>         <mailto:claudio.martella@gmail.com>>
>         <mailto:claudio.martella@
>         <mailto:claudio.martella@>__gma__il.com <http://gmail.com>
>         <mailto:claudio.martella@__gmail.com
>         <mailto:claudio.martella@gmail.com>>>
>         <mailto:claudio.martella@ <mailto:claudio.martella@>
>         <mailto:claudio.martella@
>         <mailto:claudio.martella@>>__gm__a__il.com <http://gma__il.com>
>         <http://gmail.com>
>
>
>         <mailto:claudio.martella@
>         <mailto:claudio.martella@>__gma__il.com <http://gmail.com>
>         <mailto:claudio.martella@__gmail.com
>         <mailto:claudio.martella@gmail.com>>>>
>
>
>
>
>
>
>                                                                --
>
>         Claudio Martella
>         claudio.martella@gmail.com <mailto:claudio.martella@gmail.com>
>         <mailto:claudio.martella@__gmail.com
>         <mailto:claudio.martella@gmail.com>>
>         <mailto:claudio.martella@
>         <mailto:claudio.martella@>__gma__il.com <http://gmail.com>
>         <mailto:claudio.martella@__gmail.com
>         <mailto:claudio.martella@gmail.com>>>
>         <mailto:claudio.martella@ <mailto:claudio.martella@>
>         <mailto:claudio.martella@
>         <mailto:claudio.martella@>>__gm__a__il.com <http://gma__il.com>
>         <http://gmail.com>
>
>
>         <mailto:claudio.martella@
>         <mailto:claudio.martella@>__gma__il.com <http://gmail.com>
>         <mailto:claudio.martella@__gmail.com
>         <mailto:claudio.martella@gmail.com>>>>
>
>
>
>
>
>                                                        --
>                                                            Claudio Martella
>         claudio.martella@gmail.com <mailto:claudio.martella@gmail.com>
>         <mailto:claudio.martella@__gmail.com
>         <mailto:claudio.martella@gmail.com>>
>         <mailto:claudio.martella@
>         <mailto:claudio.martella@>__gma__il.com <http://gmail.com>
>         <mailto:claudio.martella@__gmail.com
>         <mailto:claudio.martella@gmail.com>>>
>         <mailto:claudio.martella@ <mailto:claudio.martella@>
>         <mailto:claudio.martella@
>         <mailto:claudio.martella@>>__gm__a__il.com <http://gma__il.com>
>         <http://gmail.com>
>
>
>         <mailto:claudio.martella@
>         <mailto:claudio.martella@>__gma__il.com <http://gmail.com>
>         <mailto:claudio.martella@__gmail.com
>         <mailto:claudio.martella@gmail.com>>>>
>
>
>
>
>
>                                            --
>                                                Claudio Martella
>         claudio.martella@gmail.com <mailto:claudio.martella@gmail.com>
>         <mailto:claudio.martella@__gmail.com
>         <mailto:claudio.martella@gmail.com>>
>         <mailto:claudio.martella@
>         <mailto:claudio.martella@>__gma__il.com <http://gmail.com>
>         <mailto:claudio.martella@__gmail.com
>         <mailto:claudio.martella@gmail.com>>>
>         <mailto:claudio.martella@ <mailto:claudio.martella@>
>         <mailto:claudio.martella@
>         <mailto:claudio.martella@>>__gm__a__il.com <http://gma__il.com>
>         <http://gmail.com>
>
>
>         <mailto:claudio.martella@
>         <mailto:claudio.martella@>__gma__il.com <http://gmail.com>
>         <mailto:claudio.martella@__gmail.com
>         <mailto:claudio.martella@gmail.com>>>>
>
>
>
>
>
>                                    --
>                                        Claudio Martella
>         claudio.martella@gmail.com <mailto:claudio.martella@gmail.com>
>         <mailto:claudio.martella@__gmail.com
>         <mailto:claudio.martella@gmail.com>>
>         <mailto:claudio.martella@
>         <mailto:claudio.martella@>__gma__il.com <http://gmail.com>
>         <mailto:claudio.martella@__gmail.com
>         <mailto:claudio.martella@gmail.com>>>
>         <mailto:claudio.martella@ <mailto:claudio.martella@>
>         <mailto:claudio.martella@
>         <mailto:claudio.martella@>>__gm__a__il.com <http://gma__il.com>
>         <http://gmail.com>
>
>
>         <mailto:claudio.martella@
>         <mailto:claudio.martella@>__gma__il.com <http://gmail.com>
>         <mailto:claudio.martella@__gmail.com
>         <mailto:claudio.martella@gmail.com>>>>
>
>
>
>
>
>                           --
>                                Claudio Martella
>         claudio.martella@gmail.com <mailto:claudio.martella@gmail.com>
>         <mailto:claudio.martella@__gmail.com
>         <mailto:claudio.martella@gmail.com>>
>         <mailto:claudio.martella@
>         <mailto:claudio.martella@>__gma__il.com <http://gmail.com>
>         <mailto:claudio.martella@__gmail.com
>         <mailto:claudio.martella@gmail.com>>>
>         <mailto:claudio.martella@ <mailto:claudio.martella@>
>         <mailto:claudio.martella@
>         <mailto:claudio.martella@>>__gm__a__il.com <http://gma__il.com>
>         <http://gmail.com>
>
>
>         <mailto:claudio.martella@
>         <mailto:claudio.martella@>__gma__il.com <http://gmail.com>
>         <mailto:claudio.martella@__gmail.com
>         <mailto:claudio.martella@gmail.com>>>>
>
>
>                       --
>                       Matthew Laird
>                       Lead Software Developer, Bioinformatics
>                       Brinkman Laboratory
>                       Simon Fraser University, Burnaby, BC, Canada
>
>
>              --
>              Matthew Laird
>              Lead Software Developer, Bioinformatics
>              Brinkman Laboratory
>              Simon Fraser University, Burnaby, BC, Canada
>
>
>
>
>         --
>              Claudio Martella
>         claudio.martella@gmail.com <mailto:claudio.martella@gmail.com>
>         <mailto:claudio.martella@__gmail.com
>         <mailto:claudio.martella@gmail.com>>
>
>
>     --
>     Matthew Laird
>     Lead Software Developer, Bioinformatics
>     Brinkman Laboratory
>     Simon Fraser University, Burnaby, BC, Canada
>
>

-- 
Matthew Laird
Lead Software Developer, Bioinformatics
Brinkman Laboratory
Simon Fraser University, Burnaby, BC, Canada

Mime
View raw message