giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Silvio Di gregorio <silvio.digrego...@gmail.com>
Subject Re: connected components example in giraph 1.0
Date Tue, 08 Oct 2013 16:30:33 GMT
Yes this case occur.
 Il giorno 08/ott/2013 16:35, "Claudio Martella" <claudio.martella@gmail.com>
ha scritto:

> One problem could be related to the fact that with a vertex inputformat if
> vertices only occur in the adjacency list of another vertex, they will only
> be created when a message is sent to them. I guess this could affect the
> execution of the algorithm.
> Could this be the case?
>
>
> On Tue, Oct 8, 2013 at 9:29 AM, Silvio Di gregorio <
> silvio.digregorio@gmail.com> wrote:
>
>> I have increased at 4GiB the heap space and the job has been terminated
>> successfully;
>> In the result file i've seen a strange thing, i have seen, like min
>> vertex id, "ZERO", that is:
>>
>> vertex_id<tab>[min vertex id namely label of connected components]
>> 12     0
>> 22     2
>> 23     2
>> but i don't have a vertex w/ named ZERO.
>> Has ZERO a special meaning? [for example not assigned etc ....]
>>
>> PS: i use giraph 1.0, CDH4.3 w/ MP1 0.20.203 not YARN i'm sorry.
>>
>>
>>
>> 2013/10/7 Silvio Di gregorio <silvio.digregorio@gmail.com>
>>
>>> with a working day in my shoulders and my children around  i gave you
>>> random version,
>>> tomorrow morning in farm i'll be more precise.
>>> though i have hadoop 2.0 i use MR1 not YARN.
>>> I'm sorry again.
>>>
>>>
>>> 2013/10/7 Matthew Laird <lairdm@sfu.ca>
>>>
>>>> Hmm, it seems there have been some recent pushes that have broken trunk
>>>> as I discovered yesterday. :)
>>>>
>>>> But I guess that brings me back to my original question, to build
>>>> against 2.0.5 using the MR interface rather than YARN as Milinda suggested,
>>>> what profile would I use?  Would I let it think it's 0.20xxx or tell it
>>>> 2.0.5, or something else?
>>>>
>>>> Thanks.
>>>>
>>>>
>>>> On 13-10-07 11:50 AM, Claudio Martella wrote:
>>>>
>>>>> trunk is 1.1.0-SNAPSHOT. I believe the date refers to the last update
>>>>> to
>>>>> the site.
>>>>>
>>>>>
>>>>> On Mon, Oct 7, 2013 at 8:48 PM, Matthew Laird <lairdm@sfu.ca
>>>>> <mailto:lairdm@sfu.ca>> wrote:
>>>>>
>>>>>     Thanks, which is my next curiosity... if I head over to
>>>>>     http://giraph.apache.org/ I see at the top is says
>>>>>
>>>>>     "Version: 1.1.0-SNAPSHOT | Last Published: 2013-08-10"
>>>>>
>>>>>     However it's not clickable and the only download links I see are
>>>>> for
>>>>>     1.0.0.  I'm assuming trunk in git isn't the same as the 1.1.0
>>>>>     snapshot since more pushes have occurred since August, and I don't
>>>>>     see a tag in git for 1.1.0
>>>>>     (https://git-wip-us.apache.__**org/repos/asf?p=giraph.git
>>>>>     <https://git-wip-us.apache.**org/repos/asf?p=giraph.git<https://git-wip-us.apache.org/repos/asf?p=giraph.git>>).
>>>>> So where
>>>>>
>>>>>     does one get 1.1.0-SNAPSHOT from?
>>>>>
>>>>>     Thanks.
>>>>>
>>>>>
>>>>>     On 13-10-07 11:39 AM, Silvio Di gregorio wrote:
>>>>>
>>>>>         Hi Matthew
>>>>>         I use giraph 1.1.0 and cdh4 hadoop 0.23
>>>>>
>>>>>         the patch of Marco:
>>>>>
>>>>>         http://www.mail-archive.com/__**__user@giraph.apache.org/____*
>>>>> *msg00945/check.diff<http://www.mail-archive.com/____user@giraph.apache.org/____msg00945/check.diff>
>>>>>         <http://www.mail-archive.com/_**_user@giraph.apache.org/__**
>>>>> msg00945/check.diff<http://www.mail-archive.com/__user@giraph.apache.org/__msg00945/check.diff>
>>>>> >
>>>>>
>>>>>
>>>>>         <http://www.mail-archive.com/_**_user@giraph.apache.org/__**
>>>>> msg00945/check.diff<http://www.mail-archive.com/__user@giraph.apache.org/__msg00945/check.diff>
>>>>>         <http://www.mail-archive.com/**user@giraph.apache.org/**
>>>>> msg00945/check.diff<http://www.mail-archive.com/user@giraph.apache.org/msg00945/check.diff>
>>>>> >>
>>>>>
>>>>>         and build
>>>>>
>>>>>         mvn clean install -P hadoop_cdh4.1.2
>>>>>
>>>>>
>>>>>
>>>>>         Il giorno 07/ott/2013 20:20, "Matthew Laird" <lairdm@sfu.ca
>>>>>         <mailto:lairdm@sfu.ca>
>>>>>         <mailto:lairdm@sfu.ca <mailto:lairdm@sfu.ca>>> ha scritto:
>>>>>
>>>>>
>>>>>
>>>>>              Hello, so I'm watching this thread with interest as well,
>>>>> since
>>>>>              that's the exact example I'll be trying to run as well,
>>>>> and
>>>>>         I see
>>>>>              Silvio appears to be asking about running Giraph 1.0
>>>>>         against Hadoop
>>>>>              2.0.0.
>>>>>
>>>>>              I'm curious, is that a known working combination, 1.0 vs
>>>>>         2.0.x?  And
>>>>>              if so, are there patches to apply first and what maven
>>>>>         command line
>>>>>              should I build with? (ie. will a -Dhadoop.version=2.0.5
>>>>>         give the
>>>>>              needed MR based API or will it use YARN?)  I figure it's
>>>>>         better to
>>>>>              ask than aimlessly try all the combinations. :)
>>>>>
>>>>>              Thanks!
>>>>>
>>>>>              On 13-10-07 11:01 AM, Claudio Martella wrote:
>>>>>
>>>>>                  try to increase the mappers' heap space through
>>>>>                  mapred.child.java.opts.
>>>>>
>>>>>
>>>>>                  On Mon, Oct 7, 2013 at 1:18 PM, Silvio Di gregorio
>>>>>         <silvio.digregorio@gmail.com <mailto:silvio.digregorio@**
>>>>> gmail.com <silvio.digregorio@gmail.com>>
>>>>>         <mailto:silvio.digregorio@__gm**ail.com <http://gmail.com>
>>>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>
>>>>> >>
>>>>>         <mailto:silvio.digregorio@
>>>>>         <mailto:silvio.digregorio@>__g**m__ail.com<http://gm__ail.com><
>>>>> http://gmail.com>
>>>>>
>>>>>
>>>>>         <mailto:silvio.digregorio@__gm**ail.com <http://gmail.com>
>>>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>>>>>
>>>>> wrote:
>>>>>
>>>>>
>>>>>                       I think now my issue is another, thanks to all.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>                       java.lang.____**IllegalStateException: run:
>>>>> Caught an
>>>>>
>>>>>
>>>>>                  unrecoverable exception waitFor: ExecutionException
>>>>>         occurred
>>>>>                  while waiting for
>>>>>
>>>>>         org.apache.giraph.utils.____**ProgressableUtils$____**
>>>>> FutureWaitable@73a01e8a
>>>>>                           at
>>>>>
>>>>>         org.apache.giraph.graph.____**GraphMapper.run(GraphMapper.__**
>>>>> __java:102)
>>>>>
>>>>>
>>>>>
>>>>>                           ......
>>>>>                       Caused by: java.lang.____**
>>>>> IllegalStateException:
>>>>>
>>>>>         waitFor:
>>>>>
>>>>>                  ExecutionException occurred while waiting for
>>>>>
>>>>>         org.apache.giraph.utils.____**ProgressableUtils$____**
>>>>> FutureWaitable@73a01e8a
>>>>>                           at
>>>>>
>>>>>         org.apache.giraph.utils.____**ProgressableUtils.waitFor(____**
>>>>> ProgressableUtils.java:151)
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>                           ... 7 more
>>>>>                       Caused by:
>>>>>         java.util.concurrent.____**ExecutionException:
>>>>>                  java.lang.OutOfMemoryError:*__**__Java heap space*
>>>>>                           at
>>>>>
>>>>>         org.apache.giraph.utils.____**ProgressableUtils.waitFor(____**
>>>>> ProgressableUtils.java:143)
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>                           ... 13 more
>>>>>
>>>>>
>>>>>
>>>>>                       2013/10/7 Claudio Martella
>>>>>         <claudio.martella@gmail.com <mailto:claudio.martella@**
>>>>> gmail.com <claudio.martella@gmail.com>>
>>>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>>> >>
>>>>>         <mailto:claudio.martella@
>>>>>         <mailto:claudio.martella@>__gm**a__il.com <http://gma__il.com><
>>>>> http://gmail.com>
>>>>>
>>>>>
>>>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>>> >>>>
>>>>>
>>>>>                           OK, thanks.
>>>>>
>>>>>                           I really have to push that patch in.
>>>>>
>>>>>
>>>>>                           On Mon, Oct 7, 2013 at 12:17 PM, Silvio Di
>>>>>         gregorio
>>>>>         <silvio.digregorio@gmail.com
>>>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>
>>>>> >
>>>>>         <mailto:silvio.digregorio@__gm**ail.com <http://gmail.com>
>>>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>
>>>>> >>
>>>>>         <mailto:silvio.digregorio@
>>>>>         <mailto:silvio.digregorio@>__g**m__ail.com<http://gm__ail.com><
>>>>> http://gmail.com>
>>>>>
>>>>>
>>>>>         <mailto:silvio.digregorio@__gm**ail.com <http://gmail.com>
>>>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>>>>>
>>>>> wrote:
>>>>>
>>>>>                               yes i do,
>>>>>                               i have seen this in your post in:
>>>>>         http://www.mail-archive.com/__**__user@giraph.apache.org/____*
>>>>> *msg00957.html<http://www.mail-archive.com/____user@giraph.apache.org/____msg00957.html>
>>>>>         <http://www.mail-archive.com/_**_user@giraph.apache.org/__**
>>>>> msg00957.html<http://www.mail-archive.com/__user@giraph.apache.org/__msg00957.html>
>>>>> >
>>>>>
>>>>>
>>>>>         <http://www.mail-archive.com/_**_user@giraph.apache.org/__**
>>>>> msg00957.html<http://www.mail-archive.com/__user@giraph.apache.org/__msg00957.html>
>>>>>         <http://www.mail-archive.com/**user@giraph.apache.org/**
>>>>> msg00957.html<http://www.mail-archive.com/user@giraph.apache.org/msg00957.html>
>>>>> >>
>>>>>
>>>>>                               excuse me if  i had checked in the
>>>>> mail-achive
>>>>>                  first I would
>>>>>                               have avoided the last post.
>>>>>
>>>>>                               Now zk issue are resolved.
>>>>>
>>>>>
>>>>>                               2013/10/7 Claudio Martella
>>>>>         <claudio.martella@gmail.com <mailto:claudio.martella@**
>>>>> gmail.com <claudio.martella@gmail.com>>
>>>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>>> >>
>>>>>         <mailto:claudio.martella@
>>>>>         <mailto:claudio.martella@>__gm**a__il.com <http://gma__il.com><
>>>>> http://gmail.com>
>>>>>
>>>>>
>>>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>>> >>>>
>>>>>
>>>>>                                   Can you try applying this one first?
>>>>>
>>>>>         <<< <
>>>>>         http://www.mail-archive.com/__**__user@giraph.apache.org/____*
>>>>> *msg00945/check.diff<http://www.mail-archive.com/____user@giraph.apache.org/____msg00945/check.diff>
>>>>>         <http://www.mail-archive.com/_**_user@giraph.apache.org/__**
>>>>> msg00945/check.diff<http://www.mail-archive.com/__user@giraph.apache.org/__msg00945/check.diff>
>>>>> >
>>>>>
>>>>>
>>>>>         <http://www.mail-archive.com/_**_user@giraph.apache.org/__**
>>>>> msg00945/check.diff<http://www.mail-archive.com/__user@giraph.apache.org/__msg00945/check.diff>
>>>>>         <http://www.mail-archive.com/**user@giraph.apache.org/**
>>>>> msg00945/check.diff<http://www.mail-archive.com/user@giraph.apache.org/msg00945/check.diff>
>>>>> >><<<<
>>>>>
>>>>>
>>>>>                                   On Mon, Oct 7, 2013 at 8:40 AM,
>>>>> Silvio
>>>>>         Di gregorio
>>>>>         <silvio.digregorio@gmail.com
>>>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>
>>>>> >
>>>>>         <mailto:silvio.digregorio@__gm**ail.com <http://gmail.com>
>>>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>
>>>>> >>
>>>>>         <mailto:silvio.digregorio@
>>>>>         <mailto:silvio.digregorio@>__g**m__ail.com<http://gm__ail.com><
>>>>> http://gmail.com>
>>>>>
>>>>>
>>>>>         <mailto:silvio.digregorio@__gm**ail.com <http://gmail.com>
>>>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>>>>>
>>>>> wrote:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>                                       *As i said i have builded*
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>         /giraph-examples-1.0.0-for-___**_hadoop-2.0.0-cdh4.1.2-jar-___
>>>>> **_with-dependencies.jar/
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>                                       *for cdh4, successfully. The job
>>>>>         start to
>>>>>                  monitoring the success rate:*
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>                                       /13/10/07 08:28:45 INFO
>>>>>         mapred.JobClient:
>>>>>                    map 0% reduce 0%/
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>                                       *but then*
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>                                       /Error running child
>>>>>
>>>>>           java.lang.____**IllegalStateException: run:
>>>>>
>>>>>
>>>>>                  Caught an unrecoverable exception
>>>>>         java.io.FileNotFoundException:
>>>>>                  File
>>>>>         _bsp/_defaultZkManagerDir/job_**____201309181636_0678/_**
>>>>> zkServer
>>>>>
>>>>>
>>>>>                  does not exist.
>>>>>                                       /
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>                                       /.../
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>                                       /Caused by:
>>>>>         java.lang.RuntimeException:
>>>>>                  java.io.FileNotFoundException: File
>>>>>
>>>>>         _bsp/_defaultZkManagerDir/job_**____201309181636_0678/_**zkServer
>>>>> does
>>>>>
>>>>>
>>>>>                  not exist./
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>                                       2013/10/5 Silvio Di gregorio
>>>>>         <silvio.digregorio@gmail.com
>>>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>
>>>>> >
>>>>>         <mailto:silvio.digregorio@__gm**ail.com <http://gmail.com>
>>>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>
>>>>> >>
>>>>>         <mailto:silvio.digregorio@
>>>>>         <mailto:silvio.digregorio@>__g**m__ail.com<http://gm__ail.com><
>>>>> http://gmail.com>
>>>>>
>>>>>
>>>>>         <mailto:silvio.digregorio@__gm**ail.com <http://gmail.com>
>>>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>
>>>>> >>>>
>>>>>
>>>>>                                           I ha ve build w/
>>>>>         hadoop_cdh4.1.2 parameter.
>>>>>                                           Something is changed, monday
>>>>> i
>>>>>         report the
>>>>>                                           result. Now the farm is
>>>>> closed.
>>>>>
>>>>>                                           Il giorno 05/ott/2013 14:06,
>>>>>         "Claudio
>>>>>                  Martella"
>>>>>         <claudio.martella@gmail.com <mailto:claudio.martella@**
>>>>> gmail.com <claudio.martella@gmail.com>>
>>>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>>> >>
>>>>>         <mailto:claudio.martella@
>>>>>         <mailto:claudio.martella@>__gm**a__il.com <http://gma__il.com><
>>>>> http://gmail.com>
>>>>>
>>>>>
>>>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>>>>>
>>>>> ha scritto:
>>>>>
>>>>>                                               Oh, right, -vof is in
>>>>>         trunk. Anyway
>>>>>                  it looks
>>>>>                                               like you built giraph for
>>>>>         the wrong
>>>>>                  profile.
>>>>>                                               You mentioned you're
>>>>>         running on
>>>>>                  2.0, but
>>>>>                                               your giraph is built for
>>>>>         0.20.203 <tel:0.20.203>
>>>>>         <tel:0.20.203 <tel:0.20.203>>
>>>>>         <tel:0.20.203 <tel:0.20.203> <tel:0.20.203 <tel:0.20.203>>>.
>>>>> try
>>>>>
>>>>>         building with a profile
>>>>>
>>>>>                                               for your hadoop version.
>>>>>
>>>>>
>>>>>                                               On Fri, Oct 4, 2013 at
>>>>>         2:35 PM,
>>>>>                  Silvio Di
>>>>>                                               gregorio
>>>>>         <silvio.digregorio@gmail.com
>>>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>
>>>>> >
>>>>>         <mailto:silvio.digregorio@__gm**ail.com <http://gmail.com>
>>>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>
>>>>> >>
>>>>>         <mailto:silvio.digregorio@
>>>>>         <mailto:silvio.digregorio@>__g**m__ail.com<http://gm__ail.com><
>>>>> http://gmail.com>
>>>>>
>>>>>         <mailto:silvio.digregorio@__gm**ail.com <http://gmail.com>
>>>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>>>>>
>>>>> wrote:
>>>>>
>>>>>
>>>>>
>>>>>                  org.apache.commons.cli.____**
>>>>> UnrecognizedOptionException:
>>>>>
>>>>>
>>>>>                                                   Unrecognized option:
>>>>> -vof
>>>>>
>>>>>                                                   in 1.0 version is
>>>>>                                                     -of,--outputFormat
>>>>> <arg>
>>>>>                                                   Vertex output format
>>>>>                                                     -op,--outputPath
>>>>> <arg>
>>>>>                                                   Vertex output path
>>>>>
>>>>>
>>>>>
>>>>>                                                   2013/10/4 Claudio
>>>>> Martella
>>>>>         <claudio.martella@gmail.com <mailto:claudio.martella@**
>>>>> gmail.com <claudio.martella@gmail.com>>
>>>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>>> >>
>>>>>         <mailto:claudio.martella@
>>>>>         <mailto:claudio.martella@>__gm**a__il.com <http://gma__il.com><
>>>>> http://gmail.com>
>>>>>
>>>>>
>>>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>>> >>>>
>>>>>
>>>>>                                                       did you try the
>>>>>         argument
>>>>>                  (-vof) i
>>>>>                                                       suggested?
>>>>>
>>>>>
>>>>>                                                       On Fri, Oct 4,
>>>>>         2013 at 2:13 PM,
>>>>>                                                       Silvio Di
>>>>> gregorio
>>>>>         <silvio.digregorio@gmail.com
>>>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>
>>>>> >
>>>>>         <mailto:silvio.digregorio@__gm**ail.com <http://gmail.com>
>>>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>
>>>>> >>
>>>>>         <mailto:silvio.digregorio@
>>>>>         <mailto:silvio.digregorio@>__g**m__ail.com<http://gm__ail.com><
>>>>> http://gmail.com>
>>>>>
>>>>>
>>>>>         <mailto:silvio.digregorio@__gm**ail.com <http://gmail.com>
>>>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>>>>>
>>>>> wrote:
>>>>>
>>>>>
>>>>>                                                           i've
>>>>> specified
>>>>>           -of
>>>>>
>>>>>
>>>>>         org.apache.giraph.io.formats._**___IdWithValueTextOutputFormat
>>>>>
>>>>>
>>>>>
>>>>>                                                           but the same
>>>>>         error was
>>>>>                  produced
>>>>>
>>>>>                                                           Exception in
>>>>>         thread "main"
>>>>>
>>>>>                  java.lang.____**IncompatibleClassChangeError:
>>>>>                                                           Found
>>>>> interface
>>>>>
>>>>>                  org.apache.hadoop.mapreduce.__**__JobContext,
>>>>>
>>>>>
>>>>>                                                           but class was
>>>>>         expected
>>>>>                                                                    at
>>>>>
>>>>>
>>>>>         org.apache.giraph.bsp.____**BspOutputFormat.____**
>>>>> checkOutputSpecs(____**BspOutputFormat.java:43)
>>>>>                                                                    at
>>>>>
>>>>>
>>>>>         org.apache.hadoop.mapred.____**JobClient$2.run(JobClient.____*
>>>>> *java:984)
>>>>>                                                                    at
>>>>>
>>>>>
>>>>>         org.apache.hadoop.mapred.____**JobClient$2.run(JobClient.____*
>>>>> *java:945)
>>>>>                                                                    at
>>>>>
>>>>>                  java.security.____**AccessController.doPrivileged(**
>>>>> ____Native
>>>>>                                                           Method)
>>>>>                                                                    at
>>>>>
>>>>>                  javax.security.auth.Subject.__**
>>>>> __doAs(Subject.java:396)
>>>>>                                                                    at
>>>>>
>>>>>
>>>>>         org.apache.hadoop.security.___**_UserGroupInformation.doAs(___
>>>>> **_UserGroupInformation.java:___**_1408)
>>>>>                                                                    at
>>>>>
>>>>>
>>>>>         org.apache.hadoop.mapred.____**JobClient.submitJobInternal(__*
>>>>> *__JobClient.java:945)
>>>>>                                                                    at
>>>>>
>>>>>                  org.apache.hadoop.mapreduce.__**
>>>>> __Job.submit(Job.java:566)
>>>>>                                                                    at
>>>>>
>>>>>
>>>>>         org.apache.hadoop.mapreduce.__**__Job.waitForCompletion(Job.__
>>>>> **__java:596)
>>>>>                                                                    at
>>>>>
>>>>>
>>>>>         org.apache.giraph.job.____**GiraphJob.run(GiraphJob.java:_**
>>>>> ___237)
>>>>>                                                                    at
>>>>>
>>>>>
>>>>>         org.apache.giraph.____**GiraphRunner.run(GiraphRunner.**
>>>>> ____java:94)
>>>>>                                                                    at
>>>>>
>>>>>
>>>>>         org.apache.hadoop.util.____**ToolRunner.run(ToolRunner.____**
>>>>> java:70)
>>>>>                                                                    at
>>>>>
>>>>>
>>>>>         org.apache.hadoop.util.____**ToolRunner.run(ToolRunner.____**
>>>>> java:84)
>>>>>                                                                    at
>>>>>
>>>>>
>>>>>         org.apache.giraph.____**GiraphRunner.main(____**
>>>>> GiraphRunner.java:124)
>>>>>                                                                    at
>>>>>
>>>>>                  sun.reflect.____**NativeMethodAccessorImpl.____**
>>>>> invoke0(Native
>>>>>                                                           Method)
>>>>>                                                                    at
>>>>>
>>>>>
>>>>>         sun.reflect.____**NativeMethodAccessorImpl.____**invoke(____**
>>>>> NativeMethodAccessorImpl.java:**____39)
>>>>>                                                                    at
>>>>>
>>>>>
>>>>>         sun.reflect.____**DelegatingMethodAccessorImpl._**
>>>>> ___invoke(____**DelegatingMethodAccessorImpl._**___java:25)
>>>>>                                                                    at
>>>>>
>>>>>                  java.lang.reflect.Method.____**
>>>>> invoke(Method.java:597)
>>>>>                                                                    at
>>>>>
>>>>>                  org.apache.hadoop.util.RunJar.**
>>>>> ____main(RunJar.java:208)
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>                                                           2013/10/4
>>>>>         Claudio Martella
>>>>>         <claudio.martella@gmail.com <mailto:claudio.martella@**
>>>>> gmail.com <claudio.martella@gmail.com>>
>>>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>>> >>
>>>>>         <mailto:claudio.martella@
>>>>>         <mailto:claudio.martella@>__gm**a__il.com <http://gma__il.com><
>>>>> http://gmail.com>
>>>>>
>>>>>
>>>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>>> >>>>
>>>>>
>>>>>                                                               Hi,
>>>>>
>>>>>                                                               you need
>>>>>         to specify the
>>>>>                                                               vertex
>>>>>         outputformat
>>>>>                  class
>>>>>                                                               (-vof
>>>>>         option), e.g.
>>>>>
>>>>>
>>>>>         org.apache.giraph.io.formats._**___**
>>>>> IdWithValueTextOutputFormat.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>                                                               On Fri,
>>>>>         Oct 4, 2013
>>>>>                  at 1:06
>>>>>                                                               PM,
>>>>> Silvio
>>>>>         Di gregorio
>>>>>         <silvio.digregorio@gmail.com
>>>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>
>>>>> >
>>>>>         <mailto:silvio.digregorio@__gm**ail.com <http://gmail.com>
>>>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>
>>>>> >>
>>>>>         <mailto:silvio.digregorio@
>>>>>         <mailto:silvio.digregorio@>__g**m__ail.com<http://gm__ail.com><
>>>>> http://gmail.com>
>>>>>
>>>>>
>>>>>         <mailto:silvio.digregorio@__gm**ail.com <http://gmail.com>
>>>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>
>>>>> >>>>
>>>>>                                                               wrote:
>>>>>
>>>>>
>>>>>                                                                   Hi,
>>>>>
>>>>>                                                                   I
>>>>> hope
>>>>>         I have
>>>>>                  sent to
>>>>>                                                                   the
>>>>>         right address.
>>>>>
>>>>>                                                                   i
>>>>> have
>>>>>         a graph
>>>>>                  (directed
>>>>>                                                                   and
>>>>>         unweighted)
>>>>>                  stored
>>>>>                                                                   in
>>>>>         hdfs like a
>>>>>                  adjacency
>>>>>                                                                   list
>>>>>         (140Milions of
>>>>>                                                                   edges
>>>>>         6Milions
>>>>>                  of vertex)
>>>>>
>>>>>
>>>>>           node<tab>neighbors
>>>>>
>>>>>                                                                   23
>>>>> 2
>>>>>            1    3
>>>>>                      4    3
>>>>>
>>>>>         1 999 99923 <tel:1%20999%2099923> <tel:1%20999%2099923>
>>>>>         <tel:1%20999%2099923>
>>>>>                                                                   909
>>>>>         .......
>>>>>
>>>>>
>>>>> ......
>>>>>
>>>>>
>>>>> hadoop
>>>>>         version
>>>>>                  Hadoop
>>>>>
>>>>>           2.0.0-cdh4.3.0
>>>>>                  - java 1.6
>>>>>
>>>>>
>>>>>                                                                   I
>>>>> have
>>>>>         executed the
>>>>>
>>>>> giraph-1.0
>>>>>                  connected
>>>>>
>>>>> components
>>>>>                  example, in
>>>>>                                                                   this
>>>>>         fashion
>>>>>
>>>>>          >  hadoop jar
>>>>>
>>>>>
>>>>>         /usr/local/giraph/giraph-____**examples/target/giraph-____**
>>>>> examples-1.0.0-for-hadoop-0.20.203.0
>>>>>
>>>>>         <tel:0.20.203.0>
>>>>>         <tel:0.20.203.0 <tel:0.20.203.0>>
>>>>>         <tel:0.20.203.0 <tel:0.20.203.0> <tel:0.20.203.0
>>>>>          <tel:0.20.203.0>>>-jar-with-**dependencies.____jar
>>>>>
>>>>>                  org.apache.giraph.GiraphRunner
>>>>>
>>>>>                  org.apache.giraph.examples.___**
>>>>> _ConnectedComponentsVertex
>>>>>
>>>>> -vif
>>>>>
>>>>>                  org.apache.giraph.io.formats._**
>>>>> ___IntIntNullTextInputFormat
>>>>>
>>>>>
>>>>>
>>>>> -vip
>>>>>
>>>>>                  /user/hdfs/lista_adj_txt
>>>>>
>>>>> -op
>>>>>                  connectedgiraph
>>>>>
>>>>>           --workers 4
>>>>>
>>>>>                                                                   and
>>>>>         then fail with:
>>>>>
>>>>>
>>>>> 13/10/04
>>>>>                  09:28:29 INFO
>>>>>
>>>>>                  utils.ConfigurationUtils: No
>>>>>                                                                   edge
>>>>>         input format
>>>>>
>>>>> specified.
>>>>>                  Ensure your
>>>>>
>>>>>           InputFormat
>>>>>                  does not
>>>>>
>>>>>           require one.
>>>>>
>>>>>
>>>>> 13/10/04
>>>>>                  09:28:29 INFO
>>>>>
>>>>>                  utils.ConfigurationUtils: No
>>>>>
>>>>> output
>>>>>         format
>>>>>                  specified.
>>>>>
>>>>> Ensure
>>>>>         your
>>>>>                  OutputFormat
>>>>>                                                                   does
>>>>> not
>>>>>                  require one.
>>>>>
>>>>>
>>>>> 13/10/04
>>>>>                  09:28:30 INFO
>>>>>
>>>>>           job.GiraphJob: run:
>>>>>                                                                   Since
>>>>>                  checkpointing is
>>>>>
>>>>> disabled
>>>>>                  (default), do
>>>>>                                                                   not
>>>>>         allow any task
>>>>>
>>>>>           retries (setting
>>>>>
>>>>>                  mapred.map.max.attempts
>>>>>                                                                   = 0,
>>>>>         old value = 4)
>>>>>
>>>>>
>>>>> 13/10/04
>>>>>                  09:28:31 WARN
>>>>>
>>>>>                  mapred.JobClient: Use
>>>>>
>>>>>                  GenericOptionsParser for
>>>>>
>>>>>           parsing the
>>>>>                  arguments.
>>>>>
>>>>>           Applications should
>>>>>
>>>>>           implement Tool
>>>>>                  for the same.
>>>>>
>>>>>
>>>>> 13/10/04
>>>>>                  09:28:31 INFO
>>>>>
>>>>>           mapred.JobClient:
>>>>>
>>>>>           Cleaning up the
>>>>>                  staging
>>>>>                                                                   area
>>>>>
>>>>>                  hdfs://srv-bigdata-dev-01.int
>>>>>         <http://srv-bigdata-dev-01.int**>.____sose.it:8020/user/hdfs/.
>>>>> **____staging/job_201309181636_**0535<http://sose.it:8020/user/hdfs/.____staging/job_201309181636_0535>
>>>>>         <http://sose.it:8020/user/**hdfs/.__staging/job_**
>>>>> 201309181636_0535<http://sose.it:8020/user/hdfs/.__staging/job_201309181636_0535>
>>>>> >
>>>>>
>>>>>         <http://srv-bigdata-dev-01.__i**nt.sose.it:8020/user/hdfs/.__*
>>>>> *staging/job_201309181636_0535<http://int.sose.it:8020/user/hdfs/.__staging/job_201309181636_0535>
>>>>>         <http://srv-bigdata-dev-01.**int.sose.it:8020/user/hdfs/.**
>>>>> staging/job_201309181636_0535<http://srv-bigdata-dev-01.int.sose.it:8020/user/hdfs/.staging/job_201309181636_0535>
>>>>> >**>
>>>>>          <http://srv-bigdata-dev-01.__i**
>>>>> __nt.sose.it:8020/user/hdfs/._**___staging/job_201309181636_**0535<http://i__nt.sose.it:8020/user/hdfs/.____staging/job_201309181636_0535>
>>>>>         <http://int.sose.it:8020/user/**hdfs/.__staging/job_**
>>>>> 201309181636_0535<http://int.sose.it:8020/user/hdfs/.__staging/job_201309181636_0535>
>>>>> >
>>>>>
>>>>>         <http://srv-bigdata-dev-01.__i**nt.sose.it:8020/user/hdfs/.__*
>>>>> *staging/job_201309181636_0535<http://int.sose.it:8020/user/hdfs/.__staging/job_201309181636_0535>
>>>>>         <http://srv-bigdata-dev-01.**int.sose.it:8020/user/hdfs/.**
>>>>> staging/job_201309181636_0535<http://srv-bigdata-dev-01.int.sose.it:8020/user/hdfs/.staging/job_201309181636_0535>
>>>>> >**>__>
>>>>>
>>>>>
>>>>>           Exception in thread
>>>>>         "main"
>>>>>
>>>>>                  java.lang.____**IncompatibleClassChangeError:
>>>>>                                                                   Found
>>>>>         interface
>>>>>
>>>>>                  org.apache.hadoop.mapreduce.__**__JobContext,
>>>>>
>>>>>
>>>>>                                                                   but
>>>>>         class was
>>>>>                  expected
>>>>>
>>>>>
>>>>>            at
>>>>>
>>>>>
>>>>>         org.apache.giraph.bsp.____**BspOutputFormat.____**
>>>>> checkOutputSpecs(____**BspOutputFormat.java:43)
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> ......
>>>>>
>>>>>
>>>>> Thanks
>>>>>         in advance
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>                                                               --
>>>>>
>>>>>           Claudio Martella
>>>>>         claudio.martella@gmail.com <mailto:claudio.martella@**
>>>>> gmail.com <claudio.martella@gmail.com>>
>>>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>>> >>
>>>>>         <mailto:claudio.martella@
>>>>>         <mailto:claudio.martella@>__gm**a__il.com <http://gma__il.com><
>>>>> http://gmail.com>
>>>>>
>>>>>
>>>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>>> >>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>                                                       --
>>>>>                                                           Claudio
>>>>> Martella
>>>>>         claudio.martella@gmail.com <mailto:claudio.martella@**
>>>>> gmail.com <claudio.martella@gmail.com>>
>>>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>>> >>
>>>>>         <mailto:claudio.martella@
>>>>>         <mailto:claudio.martella@>__gm**a__il.com <http://gma__il.com><
>>>>> http://gmail.com>
>>>>>
>>>>>
>>>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>>> >>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>                                               --
>>>>>                                                   Claudio Martella
>>>>>         claudio.martella@gmail.com <mailto:claudio.martella@**
>>>>> gmail.com <claudio.martella@gmail.com>>
>>>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>>> >>
>>>>>         <mailto:claudio.martella@
>>>>>         <mailto:claudio.martella@>__gm**a__il.com <http://gma__il.com><
>>>>> http://gmail.com>
>>>>>
>>>>>
>>>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>>> >>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>                                   --
>>>>>                                       Claudio Martella
>>>>>         claudio.martella@gmail.com <mailto:claudio.martella@**
>>>>> gmail.com <claudio.martella@gmail.com>>
>>>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>>> >>
>>>>>         <mailto:claudio.martella@
>>>>>         <mailto:claudio.martella@>__gm**a__il.com <http://gma__il.com><
>>>>> http://gmail.com>
>>>>>
>>>>>
>>>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>>> >>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>                           --
>>>>>                               Claudio Martella
>>>>>         claudio.martella@gmail.com <mailto:claudio.martella@**
>>>>> gmail.com <claudio.martella@gmail.com>>
>>>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>>> >>
>>>>>         <mailto:claudio.martella@
>>>>>         <mailto:claudio.martella@>__gm**a__il.com <http://gma__il.com><
>>>>> http://gmail.com>
>>>>>
>>>>>
>>>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>>> >>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>                  --
>>>>>                       Claudio Martella
>>>>>         claudio.martella@gmail.com <mailto:claudio.martella@**
>>>>> gmail.com <claudio.martella@gmail.com>>
>>>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>>> >>
>>>>>         <mailto:claudio.martella@
>>>>>         <mailto:claudio.martella@>__gm**a__il.com <http://gma__il.com><
>>>>> http://gmail.com>
>>>>>
>>>>>
>>>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>>> >>>
>>>>>
>>>>>
>>>>>              --
>>>>>              Matthew Laird
>>>>>              Lead Software Developer, Bioinformatics
>>>>>              Brinkman Laboratory
>>>>>              Simon Fraser University, Burnaby, BC, Canada
>>>>>
>>>>>
>>>>>     --
>>>>>     Matthew Laird
>>>>>     Lead Software Developer, Bioinformatics
>>>>>     Brinkman Laboratory
>>>>>     Simon Fraser University, Burnaby, BC, Canada
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>     Claudio Martella
>>>>> claudio.martella@gmail.com <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>>>> >
>>>>>
>>>>
>>>> --
>>>> Matthew Laird
>>>> Lead Software Developer, Bioinformatics
>>>> Brinkman Laboratory
>>>> Simon Fraser University, Burnaby, BC, Canada
>>>>
>>>
>>>
>>
>
>
> --
>    Claudio Martella
>    claudio.martella@gmail.com
>

Mime
View raw message