giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Claudio Martella <claudio.marte...@gmail.com>
Subject Re: connected components example in giraph 1.0
Date Tue, 08 Oct 2013 14:34:56 GMT
One problem could be related to the fact that with a vertex inputformat if
vertices only occur in the adjacency list of another vertex, they will only
be created when a message is sent to them. I guess this could affect the
execution of the algorithm.
Could this be the case?


On Tue, Oct 8, 2013 at 9:29 AM, Silvio Di gregorio <
silvio.digregorio@gmail.com> wrote:

> I have increased at 4GiB the heap space and the job has been terminated
> successfully;
> In the result file i've seen a strange thing, i have seen, like min vertex
> id, "ZERO", that is:
>
> vertex_id<tab>[min vertex id namely label of connected components]
> 12     0
> 22     2
> 23     2
> but i don't have a vertex w/ named ZERO.
> Has ZERO a special meaning? [for example not assigned etc ....]
>
> PS: i use giraph 1.0, CDH4.3 w/ MP1 0.20.203 not YARN i'm sorry.
>
>
>
> 2013/10/7 Silvio Di gregorio <silvio.digregorio@gmail.com>
>
>> with a working day in my shoulders and my children around  i gave you
>> random version,
>> tomorrow morning in farm i'll be more precise.
>> though i have hadoop 2.0 i use MR1 not YARN.
>> I'm sorry again.
>>
>>
>> 2013/10/7 Matthew Laird <lairdm@sfu.ca>
>>
>>> Hmm, it seems there have been some recent pushes that have broken trunk
>>> as I discovered yesterday. :)
>>>
>>> But I guess that brings me back to my original question, to build
>>> against 2.0.5 using the MR interface rather than YARN as Milinda suggested,
>>> what profile would I use?  Would I let it think it's 0.20xxx or tell it
>>> 2.0.5, or something else?
>>>
>>> Thanks.
>>>
>>>
>>> On 13-10-07 11:50 AM, Claudio Martella wrote:
>>>
>>>> trunk is 1.1.0-SNAPSHOT. I believe the date refers to the last update to
>>>> the site.
>>>>
>>>>
>>>> On Mon, Oct 7, 2013 at 8:48 PM, Matthew Laird <lairdm@sfu.ca
>>>> <mailto:lairdm@sfu.ca>> wrote:
>>>>
>>>>     Thanks, which is my next curiosity... if I head over to
>>>>     http://giraph.apache.org/ I see at the top is says
>>>>
>>>>     "Version: 1.1.0-SNAPSHOT | Last Published: 2013-08-10"
>>>>
>>>>     However it's not clickable and the only download links I see are for
>>>>     1.0.0.  I'm assuming trunk in git isn't the same as the 1.1.0
>>>>     snapshot since more pushes have occurred since August, and I don't
>>>>     see a tag in git for 1.1.0
>>>>     (https://git-wip-us.apache.__**org/repos/asf?p=giraph.git
>>>>     <https://git-wip-us.apache.**org/repos/asf?p=giraph.git<https://git-wip-us.apache.org/repos/asf?p=giraph.git>>).
>>>> So where
>>>>
>>>>     does one get 1.1.0-SNAPSHOT from?
>>>>
>>>>     Thanks.
>>>>
>>>>
>>>>     On 13-10-07 11:39 AM, Silvio Di gregorio wrote:
>>>>
>>>>         Hi Matthew
>>>>         I use giraph 1.1.0 and cdh4 hadoop 0.23
>>>>
>>>>         the patch of Marco:
>>>>
>>>>         http://www.mail-archive.com/__**__user@giraph.apache.org/____**
>>>> msg00945/check.diff<http://www.mail-archive.com/____user@giraph.apache.org/____msg00945/check.diff>
>>>>         <http://www.mail-archive.com/_**_user@giraph.apache.org/__**
>>>> msg00945/check.diff<http://www.mail-archive.com/__user@giraph.apache.org/__msg00945/check.diff>
>>>> >
>>>>
>>>>
>>>>         <http://www.mail-archive.com/_**_user@giraph.apache.org/__**
>>>> msg00945/check.diff<http://www.mail-archive.com/__user@giraph.apache.org/__msg00945/check.diff>
>>>>         <http://www.mail-archive.com/**user@giraph.apache.org/**
>>>> msg00945/check.diff<http://www.mail-archive.com/user@giraph.apache.org/msg00945/check.diff>
>>>> >>
>>>>
>>>>         and build
>>>>
>>>>         mvn clean install -P hadoop_cdh4.1.2
>>>>
>>>>
>>>>
>>>>         Il giorno 07/ott/2013 20:20, "Matthew Laird" <lairdm@sfu.ca
>>>>         <mailto:lairdm@sfu.ca>
>>>>         <mailto:lairdm@sfu.ca <mailto:lairdm@sfu.ca>>> ha scritto:
>>>>
>>>>
>>>>
>>>>              Hello, so I'm watching this thread with interest as well,
>>>> since
>>>>              that's the exact example I'll be trying to run as well, and
>>>>         I see
>>>>              Silvio appears to be asking about running Giraph 1.0
>>>>         against Hadoop
>>>>              2.0.0.
>>>>
>>>>              I'm curious, is that a known working combination, 1.0 vs
>>>>         2.0.x?  And
>>>>              if so, are there patches to apply first and what maven
>>>>         command line
>>>>              should I build with? (ie. will a -Dhadoop.version=2.0.5
>>>>         give the
>>>>              needed MR based API or will it use YARN?)  I figure it's
>>>>         better to
>>>>              ask than aimlessly try all the combinations. :)
>>>>
>>>>              Thanks!
>>>>
>>>>              On 13-10-07 11:01 AM, Claudio Martella wrote:
>>>>
>>>>                  try to increase the mappers' heap space through
>>>>                  mapred.child.java.opts.
>>>>
>>>>
>>>>                  On Mon, Oct 7, 2013 at 1:18 PM, Silvio Di gregorio
>>>>         <silvio.digregorio@gmail.com <mailto:silvio.digregorio@**
>>>> gmail.com <silvio.digregorio@gmail.com>>
>>>>         <mailto:silvio.digregorio@__gm**ail.com <http://gmail.com>
>>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>
>>>> >>
>>>>         <mailto:silvio.digregorio@
>>>>         <mailto:silvio.digregorio@>__g**m__ail.com <http://gm__ail.com><
>>>> http://gmail.com>
>>>>
>>>>
>>>>         <mailto:silvio.digregorio@__gm**ail.com <http://gmail.com>
>>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>>>>>
>>>> wrote:
>>>>
>>>>
>>>>                       I think now my issue is another, thanks to all.
>>>>
>>>>
>>>>
>>>>
>>>>                       java.lang.____**IllegalStateException: run:
>>>> Caught an
>>>>
>>>>
>>>>                  unrecoverable exception waitFor: ExecutionException
>>>>         occurred
>>>>                  while waiting for
>>>>
>>>>         org.apache.giraph.utils.____**ProgressableUtils$____**
>>>> FutureWaitable@73a01e8a
>>>>                           at
>>>>
>>>>         org.apache.giraph.graph.____**GraphMapper.run(GraphMapper.__**
>>>> __java:102)
>>>>
>>>>
>>>>
>>>>                           ......
>>>>                       Caused by: java.lang.____**IllegalStateException:
>>>>
>>>>         waitFor:
>>>>
>>>>                  ExecutionException occurred while waiting for
>>>>
>>>>         org.apache.giraph.utils.____**ProgressableUtils$____**
>>>> FutureWaitable@73a01e8a
>>>>                           at
>>>>
>>>>         org.apache.giraph.utils.____**ProgressableUtils.waitFor(____**
>>>> ProgressableUtils.java:151)
>>>>
>>>>
>>>>
>>>>
>>>>                           ... 7 more
>>>>                       Caused by:
>>>>         java.util.concurrent.____**ExecutionException:
>>>>                  java.lang.OutOfMemoryError:*__**__Java heap space*
>>>>                           at
>>>>
>>>>         org.apache.giraph.utils.____**ProgressableUtils.waitFor(____**
>>>> ProgressableUtils.java:143)
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>                           ... 13 more
>>>>
>>>>
>>>>
>>>>                       2013/10/7 Claudio Martella
>>>>         <claudio.martella@gmail.com <mailto:claudio.martella@**
>>>> gmail.com <claudio.martella@gmail.com>>
>>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>> >>
>>>>         <mailto:claudio.martella@
>>>>         <mailto:claudio.martella@>__gm**a__il.com <http://gma__il.com><
>>>> http://gmail.com>
>>>>
>>>>
>>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>> >>>>
>>>>
>>>>                           OK, thanks.
>>>>
>>>>                           I really have to push that patch in.
>>>>
>>>>
>>>>                           On Mon, Oct 7, 2013 at 12:17 PM, Silvio Di
>>>>         gregorio
>>>>         <silvio.digregorio@gmail.com
>>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>
>>>> >
>>>>         <mailto:silvio.digregorio@__gm**ail.com <http://gmail.com>
>>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>
>>>> >>
>>>>         <mailto:silvio.digregorio@
>>>>         <mailto:silvio.digregorio@>__g**m__ail.com <http://gm__ail.com><
>>>> http://gmail.com>
>>>>
>>>>
>>>>         <mailto:silvio.digregorio@__gm**ail.com <http://gmail.com>
>>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>>>>>
>>>> wrote:
>>>>
>>>>                               yes i do,
>>>>                               i have seen this in your post in:
>>>>         http://www.mail-archive.com/__**__user@giraph.apache.org/____**
>>>> msg00957.html<http://www.mail-archive.com/____user@giraph.apache.org/____msg00957.html>
>>>>         <http://www.mail-archive.com/_**_user@giraph.apache.org/__**
>>>> msg00957.html<http://www.mail-archive.com/__user@giraph.apache.org/__msg00957.html>
>>>> >
>>>>
>>>>
>>>>         <http://www.mail-archive.com/_**_user@giraph.apache.org/__**
>>>> msg00957.html<http://www.mail-archive.com/__user@giraph.apache.org/__msg00957.html>
>>>>         <http://www.mail-archive.com/**user@giraph.apache.org/**
>>>> msg00957.html<http://www.mail-archive.com/user@giraph.apache.org/msg00957.html>
>>>> >>
>>>>
>>>>                               excuse me if  i had checked in the
>>>> mail-achive
>>>>                  first I would
>>>>                               have avoided the last post.
>>>>
>>>>                               Now zk issue are resolved.
>>>>
>>>>
>>>>                               2013/10/7 Claudio Martella
>>>>         <claudio.martella@gmail.com <mailto:claudio.martella@**
>>>> gmail.com <claudio.martella@gmail.com>>
>>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>> >>
>>>>         <mailto:claudio.martella@
>>>>         <mailto:claudio.martella@>__gm**a__il.com <http://gma__il.com><
>>>> http://gmail.com>
>>>>
>>>>
>>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>> >>>>
>>>>
>>>>                                   Can you try applying this one first?
>>>>
>>>>         <<< <
>>>>         http://www.mail-archive.com/__**__user@giraph.apache.org/____**
>>>> msg00945/check.diff<http://www.mail-archive.com/____user@giraph.apache.org/____msg00945/check.diff>
>>>>         <http://www.mail-archive.com/_**_user@giraph.apache.org/__**
>>>> msg00945/check.diff<http://www.mail-archive.com/__user@giraph.apache.org/__msg00945/check.diff>
>>>> >
>>>>
>>>>
>>>>         <http://www.mail-archive.com/_**_user@giraph.apache.org/__**
>>>> msg00945/check.diff<http://www.mail-archive.com/__user@giraph.apache.org/__msg00945/check.diff>
>>>>         <http://www.mail-archive.com/**user@giraph.apache.org/**
>>>> msg00945/check.diff<http://www.mail-archive.com/user@giraph.apache.org/msg00945/check.diff>
>>>> >><<<<
>>>>
>>>>
>>>>                                   On Mon, Oct 7, 2013 at 8:40 AM, Silvio
>>>>         Di gregorio
>>>>         <silvio.digregorio@gmail.com
>>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>
>>>> >
>>>>         <mailto:silvio.digregorio@__gm**ail.com <http://gmail.com>
>>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>
>>>> >>
>>>>         <mailto:silvio.digregorio@
>>>>         <mailto:silvio.digregorio@>__g**m__ail.com <http://gm__ail.com><
>>>> http://gmail.com>
>>>>
>>>>
>>>>         <mailto:silvio.digregorio@__gm**ail.com <http://gmail.com>
>>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>>>>>
>>>> wrote:
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>                                       *As i said i have builded*
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>         /giraph-examples-1.0.0-for-___**_hadoop-2.0.0-cdh4.1.2-jar-___*
>>>> *_with-dependencies.jar/
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>                                       *for cdh4, successfully. The job
>>>>         start to
>>>>                  monitoring the success rate:*
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>                                       /13/10/07 08:28:45 INFO
>>>>         mapred.JobClient:
>>>>                    map 0% reduce 0%/
>>>>
>>>>
>>>>
>>>>
>>>>                                       *but then*
>>>>
>>>>
>>>>
>>>>
>>>>                                       /Error running child
>>>>
>>>>           java.lang.____**IllegalStateException: run:
>>>>
>>>>
>>>>                  Caught an unrecoverable exception
>>>>         java.io.FileNotFoundException:
>>>>                  File
>>>>         _bsp/_defaultZkManagerDir/job_**____201309181636_0678/_**
>>>> zkServer
>>>>
>>>>
>>>>                  does not exist.
>>>>                                       /
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>                                       /.../
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>                                       /Caused by:
>>>>         java.lang.RuntimeException:
>>>>                  java.io.FileNotFoundException: File
>>>>
>>>>         _bsp/_defaultZkManagerDir/job_**____201309181636_0678/_**zkServer
>>>> does
>>>>
>>>>
>>>>                  not exist./
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>                                       2013/10/5 Silvio Di gregorio
>>>>         <silvio.digregorio@gmail.com
>>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>
>>>> >
>>>>         <mailto:silvio.digregorio@__gm**ail.com <http://gmail.com>
>>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>
>>>> >>
>>>>         <mailto:silvio.digregorio@
>>>>         <mailto:silvio.digregorio@>__g**m__ail.com <http://gm__ail.com><
>>>> http://gmail.com>
>>>>
>>>>
>>>>         <mailto:silvio.digregorio@__gm**ail.com <http://gmail.com>
>>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>
>>>> >>>>
>>>>
>>>>                                           I ha ve build w/
>>>>         hadoop_cdh4.1.2 parameter.
>>>>                                           Something is changed, monday i
>>>>         report the
>>>>                                           result. Now the farm is
>>>> closed.
>>>>
>>>>                                           Il giorno 05/ott/2013 14:06,
>>>>         "Claudio
>>>>                  Martella"
>>>>         <claudio.martella@gmail.com <mailto:claudio.martella@**
>>>> gmail.com <claudio.martella@gmail.com>>
>>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>> >>
>>>>         <mailto:claudio.martella@
>>>>         <mailto:claudio.martella@>__gm**a__il.com <http://gma__il.com><
>>>> http://gmail.com>
>>>>
>>>>
>>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>>>>>
>>>> ha scritto:
>>>>
>>>>                                               Oh, right, -vof is in
>>>>         trunk. Anyway
>>>>                  it looks
>>>>                                               like you built giraph for
>>>>         the wrong
>>>>                  profile.
>>>>                                               You mentioned you're
>>>>         running on
>>>>                  2.0, but
>>>>                                               your giraph is built for
>>>>         0.20.203 <tel:0.20.203>
>>>>         <tel:0.20.203 <tel:0.20.203>>
>>>>         <tel:0.20.203 <tel:0.20.203> <tel:0.20.203 <tel:0.20.203>>>.
>>>> try
>>>>
>>>>         building with a profile
>>>>
>>>>                                               for your hadoop version.
>>>>
>>>>
>>>>                                               On Fri, Oct 4, 2013 at
>>>>         2:35 PM,
>>>>                  Silvio Di
>>>>                                               gregorio
>>>>         <silvio.digregorio@gmail.com
>>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>
>>>> >
>>>>         <mailto:silvio.digregorio@__gm**ail.com <http://gmail.com>
>>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>
>>>> >>
>>>>         <mailto:silvio.digregorio@
>>>>         <mailto:silvio.digregorio@>__g**m__ail.com <http://gm__ail.com><
>>>> http://gmail.com>
>>>>
>>>>         <mailto:silvio.digregorio@__gm**ail.com <http://gmail.com>
>>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>>>>>
>>>> wrote:
>>>>
>>>>
>>>>
>>>>                  org.apache.commons.cli.____**
>>>> UnrecognizedOptionException:
>>>>
>>>>
>>>>                                                   Unrecognized option:
>>>> -vof
>>>>
>>>>                                                   in 1.0 version is
>>>>                                                     -of,--outputFormat
>>>> <arg>
>>>>                                                   Vertex output format
>>>>                                                     -op,--outputPath
>>>> <arg>
>>>>                                                   Vertex output path
>>>>
>>>>
>>>>
>>>>                                                   2013/10/4 Claudio
>>>> Martella
>>>>         <claudio.martella@gmail.com <mailto:claudio.martella@**
>>>> gmail.com <claudio.martella@gmail.com>>
>>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>> >>
>>>>         <mailto:claudio.martella@
>>>>         <mailto:claudio.martella@>__gm**a__il.com <http://gma__il.com><
>>>> http://gmail.com>
>>>>
>>>>
>>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>> >>>>
>>>>
>>>>                                                       did you try the
>>>>         argument
>>>>                  (-vof) i
>>>>                                                       suggested?
>>>>
>>>>
>>>>                                                       On Fri, Oct 4,
>>>>         2013 at 2:13 PM,
>>>>                                                       Silvio Di gregorio
>>>>         <silvio.digregorio@gmail.com
>>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>
>>>> >
>>>>         <mailto:silvio.digregorio@__gm**ail.com <http://gmail.com>
>>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>
>>>> >>
>>>>         <mailto:silvio.digregorio@
>>>>         <mailto:silvio.digregorio@>__g**m__ail.com <http://gm__ail.com><
>>>> http://gmail.com>
>>>>
>>>>
>>>>         <mailto:silvio.digregorio@__gm**ail.com <http://gmail.com>
>>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>>>>>
>>>> wrote:
>>>>
>>>>
>>>>                                                           i've specified
>>>>           -of
>>>>
>>>>
>>>>         org.apache.giraph.io.formats._**___IdWithValueTextOutputFormat
>>>>
>>>>
>>>>
>>>>                                                           but the same
>>>>         error was
>>>>                  produced
>>>>
>>>>                                                           Exception in
>>>>         thread "main"
>>>>
>>>>                  java.lang.____**IncompatibleClassChangeError:
>>>>                                                           Found
>>>> interface
>>>>
>>>>                  org.apache.hadoop.mapreduce.__**__JobContext,
>>>>
>>>>
>>>>                                                           but class was
>>>>         expected
>>>>                                                                    at
>>>>
>>>>
>>>>         org.apache.giraph.bsp.____**BspOutputFormat.____**
>>>> checkOutputSpecs(____**BspOutputFormat.java:43)
>>>>                                                                    at
>>>>
>>>>
>>>>         org.apache.hadoop.mapred.____**JobClient$2.run(JobClient.____**
>>>> java:984)
>>>>                                                                    at
>>>>
>>>>
>>>>         org.apache.hadoop.mapred.____**JobClient$2.run(JobClient.____**
>>>> java:945)
>>>>                                                                    at
>>>>
>>>>                  java.security.____**AccessController.doPrivileged(**
>>>> ____Native
>>>>                                                           Method)
>>>>                                                                    at
>>>>
>>>>                  javax.security.auth.Subject.__**
>>>> __doAs(Subject.java:396)
>>>>                                                                    at
>>>>
>>>>
>>>>         org.apache.hadoop.security.___**_UserGroupInformation.doAs(___*
>>>> *_UserGroupInformation.java:___**_1408)
>>>>                                                                    at
>>>>
>>>>
>>>>         org.apache.hadoop.mapred.____**JobClient.submitJobInternal(__**
>>>> __JobClient.java:945)
>>>>                                                                    at
>>>>
>>>>                  org.apache.hadoop.mapreduce.__**
>>>> __Job.submit(Job.java:566)
>>>>                                                                    at
>>>>
>>>>
>>>>         org.apache.hadoop.mapreduce.__**__Job.waitForCompletion(Job.__*
>>>> *__java:596)
>>>>                                                                    at
>>>>
>>>>
>>>>         org.apache.giraph.job.____**GiraphJob.run(GiraphJob.java:_**
>>>> ___237)
>>>>                                                                    at
>>>>
>>>>
>>>>         org.apache.giraph.____**GiraphRunner.run(GiraphRunner.**
>>>> ____java:94)
>>>>                                                                    at
>>>>
>>>>
>>>>         org.apache.hadoop.util.____**ToolRunner.run(ToolRunner.____**
>>>> java:70)
>>>>                                                                    at
>>>>
>>>>
>>>>         org.apache.hadoop.util.____**ToolRunner.run(ToolRunner.____**
>>>> java:84)
>>>>                                                                    at
>>>>
>>>>
>>>>         org.apache.giraph.____**GiraphRunner.main(____**
>>>> GiraphRunner.java:124)
>>>>                                                                    at
>>>>
>>>>                  sun.reflect.____**NativeMethodAccessorImpl.____**
>>>> invoke0(Native
>>>>                                                           Method)
>>>>                                                                    at
>>>>
>>>>
>>>>         sun.reflect.____**NativeMethodAccessorImpl.____**invoke(____**
>>>> NativeMethodAccessorImpl.java:**____39)
>>>>                                                                    at
>>>>
>>>>
>>>>         sun.reflect.____**DelegatingMethodAccessorImpl._**
>>>> ___invoke(____**DelegatingMethodAccessorImpl._**___java:25)
>>>>                                                                    at
>>>>
>>>>                  java.lang.reflect.Method.____**invoke(Method.java:597)
>>>>                                                                    at
>>>>
>>>>                  org.apache.hadoop.util.RunJar.**
>>>> ____main(RunJar.java:208)
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>                                                           2013/10/4
>>>>         Claudio Martella
>>>>         <claudio.martella@gmail.com <mailto:claudio.martella@**
>>>> gmail.com <claudio.martella@gmail.com>>
>>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>> >>
>>>>         <mailto:claudio.martella@
>>>>         <mailto:claudio.martella@>__gm**a__il.com <http://gma__il.com><
>>>> http://gmail.com>
>>>>
>>>>
>>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>> >>>>
>>>>
>>>>                                                               Hi,
>>>>
>>>>                                                               you need
>>>>         to specify the
>>>>                                                               vertex
>>>>         outputformat
>>>>                  class
>>>>                                                               (-vof
>>>>         option), e.g.
>>>>
>>>>
>>>>         org.apache.giraph.io.formats._**___**
>>>> IdWithValueTextOutputFormat.
>>>>
>>>>
>>>>
>>>>
>>>>                                                               On Fri,
>>>>         Oct 4, 2013
>>>>                  at 1:06
>>>>                                                               PM, Silvio
>>>>         Di gregorio
>>>>         <silvio.digregorio@gmail.com
>>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>
>>>> >
>>>>         <mailto:silvio.digregorio@__gm**ail.com <http://gmail.com>
>>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>
>>>> >>
>>>>         <mailto:silvio.digregorio@
>>>>         <mailto:silvio.digregorio@>__g**m__ail.com <http://gm__ail.com><
>>>> http://gmail.com>
>>>>
>>>>
>>>>         <mailto:silvio.digregorio@__gm**ail.com <http://gmail.com>
>>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>
>>>> >>>>
>>>>                                                               wrote:
>>>>
>>>>
>>>>                                                                   Hi,
>>>>
>>>>                                                                   I hope
>>>>         I have
>>>>                  sent to
>>>>                                                                   the
>>>>         right address.
>>>>
>>>>                                                                   i have
>>>>         a graph
>>>>                  (directed
>>>>                                                                   and
>>>>         unweighted)
>>>>                  stored
>>>>                                                                   in
>>>>         hdfs like a
>>>>                  adjacency
>>>>                                                                   list
>>>>         (140Milions of
>>>>                                                                   edges
>>>>         6Milions
>>>>                  of vertex)
>>>>
>>>>
>>>>           node<tab>neighbors
>>>>
>>>>                                                                   23   2
>>>>            1    3
>>>>                      4    3
>>>>
>>>>         1 999 99923 <tel:1%20999%2099923> <tel:1%20999%2099923>
>>>>         <tel:1%20999%2099923>
>>>>                                                                   909
>>>>         .......
>>>>
>>>>                                                                   ......
>>>>
>>>>                                                                   hadoop
>>>>         version
>>>>                  Hadoop
>>>>
>>>>           2.0.0-cdh4.3.0
>>>>                  - java 1.6
>>>>
>>>>
>>>>                                                                   I have
>>>>         executed the
>>>>
>>>> giraph-1.0
>>>>                  connected
>>>>
>>>> components
>>>>                  example, in
>>>>                                                                   this
>>>>         fashion
>>>>
>>>>          >  hadoop jar
>>>>
>>>>
>>>>         /usr/local/giraph/giraph-____**examples/target/giraph-____**
>>>> examples-1.0.0-for-hadoop-0.20.203.0
>>>>
>>>>         <tel:0.20.203.0>
>>>>         <tel:0.20.203.0 <tel:0.20.203.0>>
>>>>         <tel:0.20.203.0 <tel:0.20.203.0> <tel:0.20.203.0
>>>>          <tel:0.20.203.0>>>-jar-with-**dependencies.____jar
>>>>
>>>>                  org.apache.giraph.GiraphRunner
>>>>
>>>>                  org.apache.giraph.examples.___**
>>>> _ConnectedComponentsVertex
>>>>
>>>> -vif
>>>>
>>>>                  org.apache.giraph.io.formats._**
>>>> ___IntIntNullTextInputFormat
>>>>
>>>>
>>>>
>>>> -vip
>>>>
>>>>                  /user/hdfs/lista_adj_txt
>>>>
>>>> -op
>>>>                  connectedgiraph
>>>>
>>>>           --workers 4
>>>>
>>>>                                                                   and
>>>>         then fail with:
>>>>
>>>>
>>>> 13/10/04
>>>>                  09:28:29 INFO
>>>>
>>>>                  utils.ConfigurationUtils: No
>>>>                                                                   edge
>>>>         input format
>>>>
>>>> specified.
>>>>                  Ensure your
>>>>
>>>>           InputFormat
>>>>                  does not
>>>>
>>>>           require one.
>>>>
>>>>
>>>> 13/10/04
>>>>                  09:28:29 INFO
>>>>
>>>>                  utils.ConfigurationUtils: No
>>>>                                                                   output
>>>>         format
>>>>                  specified.
>>>>                                                                   Ensure
>>>>         your
>>>>                  OutputFormat
>>>>                                                                   does
>>>> not
>>>>                  require one.
>>>>
>>>>
>>>> 13/10/04
>>>>                  09:28:30 INFO
>>>>
>>>>           job.GiraphJob: run:
>>>>                                                                   Since
>>>>                  checkpointing is
>>>>
>>>> disabled
>>>>                  (default), do
>>>>                                                                   not
>>>>         allow any task
>>>>
>>>>           retries (setting
>>>>
>>>>                  mapred.map.max.attempts
>>>>                                                                   = 0,
>>>>         old value = 4)
>>>>
>>>>
>>>> 13/10/04
>>>>                  09:28:31 WARN
>>>>
>>>>                  mapred.JobClient: Use
>>>>
>>>>                  GenericOptionsParser for
>>>>
>>>>           parsing the
>>>>                  arguments.
>>>>
>>>>           Applications should
>>>>
>>>>           implement Tool
>>>>                  for the same.
>>>>
>>>>
>>>> 13/10/04
>>>>                  09:28:31 INFO
>>>>
>>>>           mapred.JobClient:
>>>>
>>>>           Cleaning up the
>>>>                  staging
>>>>                                                                   area
>>>>
>>>>                  hdfs://srv-bigdata-dev-01.int
>>>>         <http://srv-bigdata-dev-01.int**>.____sose.it:8020/user/hdfs/.*
>>>> *____staging/job_201309181636_**0535<http://sose.it:8020/user/hdfs/.____staging/job_201309181636_0535>
>>>>         <http://sose.it:8020/user/**hdfs/.__staging/job_**
>>>> 201309181636_0535<http://sose.it:8020/user/hdfs/.__staging/job_201309181636_0535>
>>>> >
>>>>
>>>>         <http://srv-bigdata-dev-01.__i**nt.sose.it:8020/user/hdfs/.__**
>>>> staging/job_201309181636_0535<http://int.sose.it:8020/user/hdfs/.__staging/job_201309181636_0535>
>>>>         <http://srv-bigdata-dev-01.**int.sose.it:8020/user/hdfs/.**
>>>> staging/job_201309181636_0535<http://srv-bigdata-dev-01.int.sose.it:8020/user/hdfs/.staging/job_201309181636_0535>
>>>> >**>
>>>>          <http://srv-bigdata-dev-01.__i**__nt.sose.it:8020/user/hdfs/._
>>>> **___staging/job_201309181636_**0535<http://i__nt.sose.it:8020/user/hdfs/.____staging/job_201309181636_0535>
>>>>         <http://int.sose.it:8020/user/**hdfs/.__staging/job_**
>>>> 201309181636_0535<http://int.sose.it:8020/user/hdfs/.__staging/job_201309181636_0535>
>>>> >
>>>>
>>>>         <http://srv-bigdata-dev-01.__i**nt.sose.it:8020/user/hdfs/.__**
>>>> staging/job_201309181636_0535<http://int.sose.it:8020/user/hdfs/.__staging/job_201309181636_0535>
>>>>         <http://srv-bigdata-dev-01.**int.sose.it:8020/user/hdfs/.**
>>>> staging/job_201309181636_0535<http://srv-bigdata-dev-01.int.sose.it:8020/user/hdfs/.staging/job_201309181636_0535>
>>>> >**>__>
>>>>
>>>>
>>>>           Exception in thread
>>>>         "main"
>>>>
>>>>                  java.lang.____**IncompatibleClassChangeError:
>>>>                                                                   Found
>>>>         interface
>>>>
>>>>                  org.apache.hadoop.mapreduce.__**__JobContext,
>>>>
>>>>
>>>>                                                                   but
>>>>         class was
>>>>                  expected
>>>>
>>>>
>>>>            at
>>>>
>>>>
>>>>         org.apache.giraph.bsp.____**BspOutputFormat.____**
>>>> checkOutputSpecs(____**BspOutputFormat.java:43)
>>>>
>>>>
>>>>
>>>>                                                                   ......
>>>>
>>>>                                                                   Thanks
>>>>         in advance
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>                                                               --
>>>>
>>>>           Claudio Martella
>>>>         claudio.martella@gmail.com <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>> >
>>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>> >>
>>>>         <mailto:claudio.martella@
>>>>         <mailto:claudio.martella@>__gm**a__il.com <http://gma__il.com><
>>>> http://gmail.com>
>>>>
>>>>
>>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>> >>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>                                                       --
>>>>                                                           Claudio
>>>> Martella
>>>>         claudio.martella@gmail.com <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>> >
>>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>> >>
>>>>         <mailto:claudio.martella@
>>>>         <mailto:claudio.martella@>__gm**a__il.com <http://gma__il.com><
>>>> http://gmail.com>
>>>>
>>>>
>>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>> >>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>                                               --
>>>>                                                   Claudio Martella
>>>>         claudio.martella@gmail.com <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>> >
>>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>> >>
>>>>         <mailto:claudio.martella@
>>>>         <mailto:claudio.martella@>__gm**a__il.com <http://gma__il.com><
>>>> http://gmail.com>
>>>>
>>>>
>>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>> >>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>                                   --
>>>>                                       Claudio Martella
>>>>         claudio.martella@gmail.com <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>> >
>>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>> >>
>>>>         <mailto:claudio.martella@
>>>>         <mailto:claudio.martella@>__gm**a__il.com <http://gma__il.com><
>>>> http://gmail.com>
>>>>
>>>>
>>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>> >>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>                           --
>>>>                               Claudio Martella
>>>>         claudio.martella@gmail.com <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>> >
>>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>> >>
>>>>         <mailto:claudio.martella@
>>>>         <mailto:claudio.martella@>__gm**a__il.com <http://gma__il.com><
>>>> http://gmail.com>
>>>>
>>>>
>>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>> >>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>                  --
>>>>                       Claudio Martella
>>>>         claudio.martella@gmail.com <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>> >
>>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>> >>
>>>>         <mailto:claudio.martella@
>>>>         <mailto:claudio.martella@>__gm**a__il.com <http://gma__il.com><
>>>> http://gmail.com>
>>>>
>>>>
>>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>> >>>
>>>>
>>>>
>>>>              --
>>>>              Matthew Laird
>>>>              Lead Software Developer, Bioinformatics
>>>>              Brinkman Laboratory
>>>>              Simon Fraser University, Burnaby, BC, Canada
>>>>
>>>>
>>>>     --
>>>>     Matthew Laird
>>>>     Lead Software Developer, Bioinformatics
>>>>     Brinkman Laboratory
>>>>     Simon Fraser University, Burnaby, BC, Canada
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>>     Claudio Martella
>>>> claudio.martella@gmail.com <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>> >
>>>>
>>>
>>> --
>>> Matthew Laird
>>> Lead Software Developer, Bioinformatics
>>> Brinkman Laboratory
>>> Simon Fraser University, Burnaby, BC, Canada
>>>
>>
>>
>


-- 
   Claudio Martella
   claudio.martella@gmail.com

Mime
View raw message