giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Silvio Di gregorio <silvio.digrego...@gmail.com>
Subject Re: connected components example in giraph 1.0
Date Tue, 08 Oct 2013 07:29:16 GMT
I have increased at 4GiB the heap space and the job has been terminated
successfully;
In the result file i've seen a strange thing, i have seen, like min vertex
id, "ZERO", that is:

vertex_id<tab>[min vertex id namely label of connected components]
12     0
22     2
23     2
but i don't have a vertex w/ named ZERO.
Has ZERO a special meaning? [for example not assigned etc ....]

PS: i use giraph 1.0, CDH4.3 w/ MP1 0.20.203 not YARN i'm sorry.



2013/10/7 Silvio Di gregorio <silvio.digregorio@gmail.com>

> with a working day in my shoulders and my children around  i gave you
> random version,
> tomorrow morning in farm i'll be more precise.
> though i have hadoop 2.0 i use MR1 not YARN.
> I'm sorry again.
>
>
> 2013/10/7 Matthew Laird <lairdm@sfu.ca>
>
>> Hmm, it seems there have been some recent pushes that have broken trunk
>> as I discovered yesterday. :)
>>
>> But I guess that brings me back to my original question, to build against
>> 2.0.5 using the MR interface rather than YARN as Milinda suggested, what
>> profile would I use?  Would I let it think it's 0.20xxx or tell it 2.0.5,
>> or something else?
>>
>> Thanks.
>>
>>
>> On 13-10-07 11:50 AM, Claudio Martella wrote:
>>
>>> trunk is 1.1.0-SNAPSHOT. I believe the date refers to the last update to
>>> the site.
>>>
>>>
>>> On Mon, Oct 7, 2013 at 8:48 PM, Matthew Laird <lairdm@sfu.ca
>>> <mailto:lairdm@sfu.ca>> wrote:
>>>
>>>     Thanks, which is my next curiosity... if I head over to
>>>     http://giraph.apache.org/ I see at the top is says
>>>
>>>     "Version: 1.1.0-SNAPSHOT | Last Published: 2013-08-10"
>>>
>>>     However it's not clickable and the only download links I see are for
>>>     1.0.0.  I'm assuming trunk in git isn't the same as the 1.1.0
>>>     snapshot since more pushes have occurred since August, and I don't
>>>     see a tag in git for 1.1.0
>>>     (https://git-wip-us.apache.__**org/repos/asf?p=giraph.git
>>>     <https://git-wip-us.apache.**org/repos/asf?p=giraph.git<https://git-wip-us.apache.org/repos/asf?p=giraph.git>>).
>>> So where
>>>
>>>     does one get 1.1.0-SNAPSHOT from?
>>>
>>>     Thanks.
>>>
>>>
>>>     On 13-10-07 11:39 AM, Silvio Di gregorio wrote:
>>>
>>>         Hi Matthew
>>>         I use giraph 1.1.0 and cdh4 hadoop 0.23
>>>
>>>         the patch of Marco:
>>>
>>>         http://www.mail-archive.com/__**__user@giraph.apache.org/____**
>>> msg00945/check.diff<http://www.mail-archive.com/____user@giraph.apache.org/____msg00945/check.diff>
>>>         <http://www.mail-archive.com/_**_user@giraph.apache.org/__**
>>> msg00945/check.diff<http://www.mail-archive.com/__user@giraph.apache.org/__msg00945/check.diff>
>>> >
>>>
>>>
>>>         <http://www.mail-archive.com/_**_user@giraph.apache.org/__**
>>> msg00945/check.diff<http://www.mail-archive.com/__user@giraph.apache.org/__msg00945/check.diff>
>>>         <http://www.mail-archive.com/**user@giraph.apache.org/**
>>> msg00945/check.diff<http://www.mail-archive.com/user@giraph.apache.org/msg00945/check.diff>
>>> >>
>>>
>>>         and build
>>>
>>>         mvn clean install -P hadoop_cdh4.1.2
>>>
>>>
>>>
>>>         Il giorno 07/ott/2013 20:20, "Matthew Laird" <lairdm@sfu.ca
>>>         <mailto:lairdm@sfu.ca>
>>>         <mailto:lairdm@sfu.ca <mailto:lairdm@sfu.ca>>> ha scritto:
>>>
>>>
>>>
>>>              Hello, so I'm watching this thread with interest as well,
>>> since
>>>              that's the exact example I'll be trying to run as well, and
>>>         I see
>>>              Silvio appears to be asking about running Giraph 1.0
>>>         against Hadoop
>>>              2.0.0.
>>>
>>>              I'm curious, is that a known working combination, 1.0 vs
>>>         2.0.x?  And
>>>              if so, are there patches to apply first and what maven
>>>         command line
>>>              should I build with? (ie. will a -Dhadoop.version=2.0.5
>>>         give the
>>>              needed MR based API or will it use YARN?)  I figure it's
>>>         better to
>>>              ask than aimlessly try all the combinations. :)
>>>
>>>              Thanks!
>>>
>>>              On 13-10-07 11:01 AM, Claudio Martella wrote:
>>>
>>>                  try to increase the mappers' heap space through
>>>                  mapred.child.java.opts.
>>>
>>>
>>>                  On Mon, Oct 7, 2013 at 1:18 PM, Silvio Di gregorio
>>>         <silvio.digregorio@gmail.com <mailto:silvio.digregorio@**
>>> gmail.com <silvio.digregorio@gmail.com>>
>>>         <mailto:silvio.digregorio@__gm**ail.com <http://gmail.com>
>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>
>>> >>
>>>         <mailto:silvio.digregorio@
>>>         <mailto:silvio.digregorio@>__g**m__ail.com <http://gm__ail.com><
>>> http://gmail.com>
>>>
>>>
>>>         <mailto:silvio.digregorio@__gm**ail.com <http://gmail.com>
>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>>>>>
>>> wrote:
>>>
>>>
>>>                       I think now my issue is another, thanks to all.
>>>
>>>
>>>
>>>
>>>                       java.lang.____**IllegalStateException: run:
>>> Caught an
>>>
>>>
>>>                  unrecoverable exception waitFor: ExecutionException
>>>         occurred
>>>                  while waiting for
>>>
>>>         org.apache.giraph.utils.____**ProgressableUtils$____**
>>> FutureWaitable@73a01e8a
>>>                           at
>>>
>>>         org.apache.giraph.graph.____**GraphMapper.run(GraphMapper.__**
>>> __java:102)
>>>
>>>
>>>
>>>                           ......
>>>                       Caused by: java.lang.____**IllegalStateException:
>>>
>>>         waitFor:
>>>
>>>                  ExecutionException occurred while waiting for
>>>
>>>         org.apache.giraph.utils.____**ProgressableUtils$____**
>>> FutureWaitable@73a01e8a
>>>                           at
>>>
>>>         org.apache.giraph.utils.____**ProgressableUtils.waitFor(____**
>>> ProgressableUtils.java:151)
>>>
>>>
>>>
>>>
>>>                           ... 7 more
>>>                       Caused by:
>>>         java.util.concurrent.____**ExecutionException:
>>>                  java.lang.OutOfMemoryError:*__**__Java heap space*
>>>                           at
>>>
>>>         org.apache.giraph.utils.____**ProgressableUtils.waitFor(____**
>>> ProgressableUtils.java:143)
>>>
>>>
>>>
>>>
>>>
>>>                           ... 13 more
>>>
>>>
>>>
>>>                       2013/10/7 Claudio Martella
>>>         <claudio.martella@gmail.com <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>> >
>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>> >>
>>>         <mailto:claudio.martella@
>>>         <mailto:claudio.martella@>__gm**a__il.com <http://gma__il.com>
<
>>> http://gmail.com>
>>>
>>>
>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>> >>>>
>>>
>>>                           OK, thanks.
>>>
>>>                           I really have to push that patch in.
>>>
>>>
>>>                           On Mon, Oct 7, 2013 at 12:17 PM, Silvio Di
>>>         gregorio
>>>         <silvio.digregorio@gmail.com
>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>
>>> >
>>>         <mailto:silvio.digregorio@__gm**ail.com <http://gmail.com>
>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>
>>> >>
>>>         <mailto:silvio.digregorio@
>>>         <mailto:silvio.digregorio@>__g**m__ail.com <http://gm__ail.com><
>>> http://gmail.com>
>>>
>>>
>>>         <mailto:silvio.digregorio@__gm**ail.com <http://gmail.com>
>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>>>>>
>>> wrote:
>>>
>>>                               yes i do,
>>>                               i have seen this in your post in:
>>>         http://www.mail-archive.com/__**__user@giraph.apache.org/____**
>>> msg00957.html<http://www.mail-archive.com/____user@giraph.apache.org/____msg00957.html>
>>>         <http://www.mail-archive.com/_**_user@giraph.apache.org/__**
>>> msg00957.html<http://www.mail-archive.com/__user@giraph.apache.org/__msg00957.html>
>>> >
>>>
>>>
>>>         <http://www.mail-archive.com/_**_user@giraph.apache.org/__**
>>> msg00957.html<http://www.mail-archive.com/__user@giraph.apache.org/__msg00957.html>
>>>         <http://www.mail-archive.com/**user@giraph.apache.org/**
>>> msg00957.html<http://www.mail-archive.com/user@giraph.apache.org/msg00957.html>
>>> >>
>>>
>>>                               excuse me if  i had checked in the
>>> mail-achive
>>>                  first I would
>>>                               have avoided the last post.
>>>
>>>                               Now zk issue are resolved.
>>>
>>>
>>>                               2013/10/7 Claudio Martella
>>>         <claudio.martella@gmail.com <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>> >
>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>> >>
>>>         <mailto:claudio.martella@
>>>         <mailto:claudio.martella@>__gm**a__il.com <http://gma__il.com>
<
>>> http://gmail.com>
>>>
>>>
>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>> >>>>
>>>
>>>                                   Can you try applying this one first?
>>>
>>>         <<< <
>>>         http://www.mail-archive.com/__**__user@giraph.apache.org/____**
>>> msg00945/check.diff<http://www.mail-archive.com/____user@giraph.apache.org/____msg00945/check.diff>
>>>         <http://www.mail-archive.com/_**_user@giraph.apache.org/__**
>>> msg00945/check.diff<http://www.mail-archive.com/__user@giraph.apache.org/__msg00945/check.diff>
>>> >
>>>
>>>
>>>         <http://www.mail-archive.com/_**_user@giraph.apache.org/__**
>>> msg00945/check.diff<http://www.mail-archive.com/__user@giraph.apache.org/__msg00945/check.diff>
>>>         <http://www.mail-archive.com/**user@giraph.apache.org/**
>>> msg00945/check.diff<http://www.mail-archive.com/user@giraph.apache.org/msg00945/check.diff>
>>> >><<<<
>>>
>>>
>>>                                   On Mon, Oct 7, 2013 at 8:40 AM, Silvio
>>>         Di gregorio
>>>         <silvio.digregorio@gmail.com
>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>
>>> >
>>>         <mailto:silvio.digregorio@__gm**ail.com <http://gmail.com>
>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>
>>> >>
>>>         <mailto:silvio.digregorio@
>>>         <mailto:silvio.digregorio@>__g**m__ail.com <http://gm__ail.com><
>>> http://gmail.com>
>>>
>>>
>>>         <mailto:silvio.digregorio@__gm**ail.com <http://gmail.com>
>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>>>>>
>>> wrote:
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>                                       *As i said i have builded*
>>>
>>>
>>>
>>>
>>>
>>>
>>>         /giraph-examples-1.0.0-for-___**_hadoop-2.0.0-cdh4.1.2-jar-___**
>>> _with-dependencies.jar/
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>                                       *for cdh4, successfully. The job
>>>         start to
>>>                  monitoring the success rate:*
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>                                       /13/10/07 08:28:45 INFO
>>>         mapred.JobClient:
>>>                    map 0% reduce 0%/
>>>
>>>
>>>
>>>
>>>                                       *but then*
>>>
>>>
>>>
>>>
>>>                                       /Error running child
>>>
>>>           java.lang.____**IllegalStateException: run:
>>>
>>>
>>>                  Caught an unrecoverable exception
>>>         java.io.FileNotFoundException:
>>>                  File
>>>         _bsp/_defaultZkManagerDir/job_**____201309181636_0678/_**
>>> zkServer
>>>
>>>
>>>                  does not exist.
>>>                                       /
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>                                       /.../
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>                                       /Caused by:
>>>         java.lang.RuntimeException:
>>>                  java.io.FileNotFoundException: File
>>>
>>>         _bsp/_defaultZkManagerDir/job_**____201309181636_0678/_**zkServer
>>> does
>>>
>>>
>>>                  not exist./
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>                                       2013/10/5 Silvio Di gregorio
>>>         <silvio.digregorio@gmail.com
>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>
>>> >
>>>         <mailto:silvio.digregorio@__gm**ail.com <http://gmail.com>
>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>
>>> >>
>>>         <mailto:silvio.digregorio@
>>>         <mailto:silvio.digregorio@>__g**m__ail.com <http://gm__ail.com><
>>> http://gmail.com>
>>>
>>>
>>>         <mailto:silvio.digregorio@__gm**ail.com <http://gmail.com>
>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>
>>> >>>>
>>>
>>>                                           I ha ve build w/
>>>         hadoop_cdh4.1.2 parameter.
>>>                                           Something is changed, monday i
>>>         report the
>>>                                           result. Now the farm is closed.
>>>
>>>                                           Il giorno 05/ott/2013 14:06,
>>>         "Claudio
>>>                  Martella"
>>>         <claudio.martella@gmail.com <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>> >
>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>> >>
>>>         <mailto:claudio.martella@
>>>         <mailto:claudio.martella@>__gm**a__il.com <http://gma__il.com>
<
>>> http://gmail.com>
>>>
>>>
>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>>>>>
>>> ha scritto:
>>>
>>>                                               Oh, right, -vof is in
>>>         trunk. Anyway
>>>                  it looks
>>>                                               like you built giraph for
>>>         the wrong
>>>                  profile.
>>>                                               You mentioned you're
>>>         running on
>>>                  2.0, but
>>>                                               your giraph is built for
>>>         0.20.203 <tel:0.20.203>
>>>         <tel:0.20.203 <tel:0.20.203>>
>>>         <tel:0.20.203 <tel:0.20.203> <tel:0.20.203 <tel:0.20.203>>>.
try
>>>
>>>         building with a profile
>>>
>>>                                               for your hadoop version.
>>>
>>>
>>>                                               On Fri, Oct 4, 2013 at
>>>         2:35 PM,
>>>                  Silvio Di
>>>                                               gregorio
>>>         <silvio.digregorio@gmail.com
>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>
>>> >
>>>         <mailto:silvio.digregorio@__gm**ail.com <http://gmail.com>
>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>
>>> >>
>>>         <mailto:silvio.digregorio@
>>>         <mailto:silvio.digregorio@>__g**m__ail.com <http://gm__ail.com><
>>> http://gmail.com>
>>>
>>>         <mailto:silvio.digregorio@__gm**ail.com <http://gmail.com>
>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>>>>>
>>> wrote:
>>>
>>>
>>>
>>>                  org.apache.commons.cli.____**
>>> UnrecognizedOptionException:
>>>
>>>
>>>                                                   Unrecognized option:
>>> -vof
>>>
>>>                                                   in 1.0 version is
>>>                                                     -of,--outputFormat
>>> <arg>
>>>                                                   Vertex output format
>>>                                                     -op,--outputPath
>>> <arg>
>>>                                                   Vertex output path
>>>
>>>
>>>
>>>                                                   2013/10/4 Claudio
>>> Martella
>>>         <claudio.martella@gmail.com <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>> >
>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>> >>
>>>         <mailto:claudio.martella@
>>>         <mailto:claudio.martella@>__gm**a__il.com <http://gma__il.com>
<
>>> http://gmail.com>
>>>
>>>
>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>> >>>>
>>>
>>>                                                       did you try the
>>>         argument
>>>                  (-vof) i
>>>                                                       suggested?
>>>
>>>
>>>                                                       On Fri, Oct 4,
>>>         2013 at 2:13 PM,
>>>                                                       Silvio Di gregorio
>>>         <silvio.digregorio@gmail.com
>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>
>>> >
>>>         <mailto:silvio.digregorio@__gm**ail.com <http://gmail.com>
>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>
>>> >>
>>>         <mailto:silvio.digregorio@
>>>         <mailto:silvio.digregorio@>__g**m__ail.com <http://gm__ail.com><
>>> http://gmail.com>
>>>
>>>
>>>         <mailto:silvio.digregorio@__gm**ail.com <http://gmail.com>
>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>>>>>
>>> wrote:
>>>
>>>
>>>                                                           i've specified
>>>           -of
>>>
>>>
>>>         org.apache.giraph.io.formats._**___IdWithValueTextOutputFormat
>>>
>>>
>>>
>>>                                                           but the same
>>>         error was
>>>                  produced
>>>
>>>                                                           Exception in
>>>         thread "main"
>>>
>>>                  java.lang.____**IncompatibleClassChangeError:
>>>                                                           Found interface
>>>
>>>                  org.apache.hadoop.mapreduce.__**__JobContext,
>>>
>>>
>>>                                                           but class was
>>>         expected
>>>                                                                    at
>>>
>>>
>>>         org.apache.giraph.bsp.____**BspOutputFormat.____**
>>> checkOutputSpecs(____**BspOutputFormat.java:43)
>>>                                                                    at
>>>
>>>
>>>         org.apache.hadoop.mapred.____**JobClient$2.run(JobClient.____**
>>> java:984)
>>>                                                                    at
>>>
>>>
>>>         org.apache.hadoop.mapred.____**JobClient$2.run(JobClient.____**
>>> java:945)
>>>                                                                    at
>>>
>>>                  java.security.____**AccessController.doPrivileged(**
>>> ____Native
>>>                                                           Method)
>>>                                                                    at
>>>
>>>                  javax.security.auth.Subject.__**
>>> __doAs(Subject.java:396)
>>>                                                                    at
>>>
>>>
>>>         org.apache.hadoop.security.___**_UserGroupInformation.doAs(___**
>>> _UserGroupInformation.java:___**_1408)
>>>                                                                    at
>>>
>>>
>>>         org.apache.hadoop.mapred.____**JobClient.submitJobInternal(__**
>>> __JobClient.java:945)
>>>                                                                    at
>>>
>>>                  org.apache.hadoop.mapreduce.__**
>>> __Job.submit(Job.java:566)
>>>                                                                    at
>>>
>>>
>>>         org.apache.hadoop.mapreduce.__**__Job.waitForCompletion(Job.__**
>>> __java:596)
>>>                                                                    at
>>>
>>>
>>>         org.apache.giraph.job.____**GiraphJob.run(GiraphJob.java:_**
>>> ___237)
>>>                                                                    at
>>>
>>>
>>>         org.apache.giraph.____**GiraphRunner.run(GiraphRunner.**
>>> ____java:94)
>>>                                                                    at
>>>
>>>
>>>         org.apache.hadoop.util.____**ToolRunner.run(ToolRunner.____**
>>> java:70)
>>>                                                                    at
>>>
>>>
>>>         org.apache.hadoop.util.____**ToolRunner.run(ToolRunner.____**
>>> java:84)
>>>                                                                    at
>>>
>>>
>>>         org.apache.giraph.____**GiraphRunner.main(____**
>>> GiraphRunner.java:124)
>>>                                                                    at
>>>
>>>                  sun.reflect.____**NativeMethodAccessorImpl.____**
>>> invoke0(Native
>>>                                                           Method)
>>>                                                                    at
>>>
>>>
>>>         sun.reflect.____**NativeMethodAccessorImpl.____**invoke(____**
>>> NativeMethodAccessorImpl.java:**____39)
>>>                                                                    at
>>>
>>>
>>>         sun.reflect.____**DelegatingMethodAccessorImpl._**___invoke(____
>>> **DelegatingMethodAccessorImpl._**___java:25)
>>>                                                                    at
>>>
>>>                  java.lang.reflect.Method.____**invoke(Method.java:597)
>>>                                                                    at
>>>
>>>                  org.apache.hadoop.util.RunJar.**
>>> ____main(RunJar.java:208)
>>>
>>>
>>>
>>>
>>>
>>>                                                           2013/10/4
>>>         Claudio Martella
>>>         <claudio.martella@gmail.com <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>> >
>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>> >>
>>>         <mailto:claudio.martella@
>>>         <mailto:claudio.martella@>__gm**a__il.com <http://gma__il.com>
<
>>> http://gmail.com>
>>>
>>>
>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>> >>>>
>>>
>>>                                                               Hi,
>>>
>>>                                                               you need
>>>         to specify the
>>>                                                               vertex
>>>         outputformat
>>>                  class
>>>                                                               (-vof
>>>         option), e.g.
>>>
>>>
>>>         org.apache.giraph.io.formats._**___**
>>> IdWithValueTextOutputFormat.
>>>
>>>
>>>
>>>
>>>                                                               On Fri,
>>>         Oct 4, 2013
>>>                  at 1:06
>>>                                                               PM, Silvio
>>>         Di gregorio
>>>         <silvio.digregorio@gmail.com
>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>
>>> >
>>>         <mailto:silvio.digregorio@__gm**ail.com <http://gmail.com>
>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>
>>> >>
>>>         <mailto:silvio.digregorio@
>>>         <mailto:silvio.digregorio@>__g**m__ail.com <http://gm__ail.com><
>>> http://gmail.com>
>>>
>>>
>>>         <mailto:silvio.digregorio@__gm**ail.com <http://gmail.com>
>>>         <mailto:silvio.digregorio@**gmail.com<silvio.digregorio@gmail.com>
>>> >>>>
>>>                                                               wrote:
>>>
>>>
>>>                                                                   Hi,
>>>
>>>                                                                   I hope
>>>         I have
>>>                  sent to
>>>                                                                   the
>>>         right address.
>>>
>>>                                                                   i have
>>>         a graph
>>>                  (directed
>>>                                                                   and
>>>         unweighted)
>>>                  stored
>>>                                                                   in
>>>         hdfs like a
>>>                  adjacency
>>>                                                                   list
>>>         (140Milions of
>>>                                                                   edges
>>>         6Milions
>>>                  of vertex)
>>>
>>>
>>>           node<tab>neighbors
>>>
>>>                                                                   23   2
>>>            1    3
>>>                      4    3
>>>
>>>         1 999 99923 <tel:1%20999%2099923> <tel:1%20999%2099923>
>>>         <tel:1%20999%2099923>
>>>                                                                   909
>>>         .......
>>>
>>>                                                                   ......
>>>
>>>                                                                   hadoop
>>>         version
>>>                  Hadoop
>>>
>>>           2.0.0-cdh4.3.0
>>>                  - java 1.6
>>>
>>>
>>>                                                                   I have
>>>         executed the
>>>
>>> giraph-1.0
>>>                  connected
>>>
>>> components
>>>                  example, in
>>>                                                                   this
>>>         fashion
>>>
>>>          >  hadoop jar
>>>
>>>
>>>         /usr/local/giraph/giraph-____**examples/target/giraph-____**
>>> examples-1.0.0-for-hadoop-0.20.203.0
>>>
>>>         <tel:0.20.203.0>
>>>         <tel:0.20.203.0 <tel:0.20.203.0>>
>>>         <tel:0.20.203.0 <tel:0.20.203.0> <tel:0.20.203.0
>>>          <tel:0.20.203.0>>>-jar-with-**dependencies.____jar
>>>
>>>                  org.apache.giraph.GiraphRunner
>>>
>>>                  org.apache.giraph.examples.___**
>>> _ConnectedComponentsVertex
>>>
>>> -vif
>>>
>>>                  org.apache.giraph.io.formats._**
>>> ___IntIntNullTextInputFormat
>>>
>>>
>>>
>>> -vip
>>>
>>>                  /user/hdfs/lista_adj_txt
>>>                                                                       -op
>>>                  connectedgiraph
>>>
>>>           --workers 4
>>>
>>>                                                                   and
>>>         then fail with:
>>>
>>>
>>> 13/10/04
>>>                  09:28:29 INFO
>>>
>>>                  utils.ConfigurationUtils: No
>>>                                                                   edge
>>>         input format
>>>
>>> specified.
>>>                  Ensure your
>>>
>>>           InputFormat
>>>                  does not
>>>
>>>           require one.
>>>
>>>
>>> 13/10/04
>>>                  09:28:29 INFO
>>>
>>>                  utils.ConfigurationUtils: No
>>>                                                                   output
>>>         format
>>>                  specified.
>>>                                                                   Ensure
>>>         your
>>>                  OutputFormat
>>>                                                                   does
>>> not
>>>                  require one.
>>>
>>>
>>> 13/10/04
>>>                  09:28:30 INFO
>>>
>>>           job.GiraphJob: run:
>>>                                                                   Since
>>>                  checkpointing is
>>>
>>> disabled
>>>                  (default), do
>>>                                                                   not
>>>         allow any task
>>>
>>>           retries (setting
>>>
>>>                  mapred.map.max.attempts
>>>                                                                   = 0,
>>>         old value = 4)
>>>
>>>
>>> 13/10/04
>>>                  09:28:31 WARN
>>>
>>>                  mapred.JobClient: Use
>>>
>>>                  GenericOptionsParser for
>>>
>>>           parsing the
>>>                  arguments.
>>>
>>>           Applications should
>>>
>>>           implement Tool
>>>                  for the same.
>>>
>>>
>>> 13/10/04
>>>                  09:28:31 INFO
>>>
>>>           mapred.JobClient:
>>>
>>>           Cleaning up the
>>>                  staging
>>>                                                                   area
>>>
>>>                  hdfs://srv-bigdata-dev-01.int
>>>         <http://srv-bigdata-dev-01.int**>.____sose.it:8020/user/hdfs/.**
>>> ____staging/job_201309181636_**0535<http://sose.it:8020/user/hdfs/.____staging/job_201309181636_0535>
>>>         <http://sose.it:8020/user/**hdfs/.__staging/job_**
>>> 201309181636_0535<http://sose.it:8020/user/hdfs/.__staging/job_201309181636_0535>
>>> >
>>>
>>>         <http://srv-bigdata-dev-01.__i**nt.sose.it:8020/user/hdfs/.__**
>>> staging/job_201309181636_0535<http://int.sose.it:8020/user/hdfs/.__staging/job_201309181636_0535>
>>>         <http://srv-bigdata-dev-01.**int.sose.it:8020/user/hdfs/.**
>>> staging/job_201309181636_0535<http://srv-bigdata-dev-01.int.sose.it:8020/user/hdfs/.staging/job_201309181636_0535>
>>> >**>
>>>          <http://srv-bigdata-dev-01.__i**__nt.sose.it:8020/user/hdfs/._*
>>> *___staging/job_201309181636_**0535<http://i__nt.sose.it:8020/user/hdfs/.____staging/job_201309181636_0535>
>>>         <http://int.sose.it:8020/user/**hdfs/.__staging/job_**
>>> 201309181636_0535<http://int.sose.it:8020/user/hdfs/.__staging/job_201309181636_0535>
>>> >
>>>
>>>         <http://srv-bigdata-dev-01.__i**nt.sose.it:8020/user/hdfs/.__**
>>> staging/job_201309181636_0535<http://int.sose.it:8020/user/hdfs/.__staging/job_201309181636_0535>
>>>         <http://srv-bigdata-dev-01.**int.sose.it:8020/user/hdfs/.**
>>> staging/job_201309181636_0535<http://srv-bigdata-dev-01.int.sose.it:8020/user/hdfs/.staging/job_201309181636_0535>
>>> >**>__>
>>>
>>>
>>>           Exception in thread
>>>         "main"
>>>
>>>                  java.lang.____**IncompatibleClassChangeError:
>>>                                                                   Found
>>>         interface
>>>
>>>                  org.apache.hadoop.mapreduce.__**__JobContext,
>>>
>>>
>>>                                                                   but
>>>         class was
>>>                  expected
>>>
>>>
>>>            at
>>>
>>>
>>>         org.apache.giraph.bsp.____**BspOutputFormat.____**
>>> checkOutputSpecs(____**BspOutputFormat.java:43)
>>>
>>>
>>>
>>>                                                                   ......
>>>
>>>                                                                   Thanks
>>>         in advance
>>>
>>>
>>>
>>>
>>>
>>>                                                               --
>>>
>>>           Claudio Martella
>>>         claudio.martella@gmail.com <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>> >
>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>> >>
>>>         <mailto:claudio.martella@
>>>         <mailto:claudio.martella@>__gm**a__il.com <http://gma__il.com>
<
>>> http://gmail.com>
>>>
>>>
>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>> >>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>                                                       --
>>>                                                           Claudio
>>> Martella
>>>         claudio.martella@gmail.com <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>> >
>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>> >>
>>>         <mailto:claudio.martella@
>>>         <mailto:claudio.martella@>__gm**a__il.com <http://gma__il.com>
<
>>> http://gmail.com>
>>>
>>>
>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>> >>>
>>>
>>>
>>>
>>>
>>>
>>>                                               --
>>>                                                   Claudio Martella
>>>         claudio.martella@gmail.com <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>> >
>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>> >>
>>>         <mailto:claudio.martella@
>>>         <mailto:claudio.martella@>__gm**a__il.com <http://gma__il.com>
<
>>> http://gmail.com>
>>>
>>>
>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>> >>>
>>>
>>>
>>>
>>>
>>>
>>>                                   --
>>>                                       Claudio Martella
>>>         claudio.martella@gmail.com <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>> >
>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>> >>
>>>         <mailto:claudio.martella@
>>>         <mailto:claudio.martella@>__gm**a__il.com <http://gma__il.com>
<
>>> http://gmail.com>
>>>
>>>
>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>> >>>
>>>
>>>
>>>
>>>
>>>
>>>                           --
>>>                               Claudio Martella
>>>         claudio.martella@gmail.com <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>> >
>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>> >>
>>>         <mailto:claudio.martella@
>>>         <mailto:claudio.martella@>__gm**a__il.com <http://gma__il.com>
<
>>> http://gmail.com>
>>>
>>>
>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>> >>>
>>>
>>>
>>>
>>>
>>>
>>>                  --
>>>                       Claudio Martella
>>>         claudio.martella@gmail.com <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>> >
>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>> >>
>>>         <mailto:claudio.martella@
>>>         <mailto:claudio.martella@>__gm**a__il.com <http://gma__il.com>
<
>>> http://gmail.com>
>>>
>>>
>>>         <mailto:claudio.martella@__gma**il.com <http://gmail.com>
>>>         <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>> >>>
>>>
>>>
>>>              --
>>>              Matthew Laird
>>>              Lead Software Developer, Bioinformatics
>>>              Brinkman Laboratory
>>>              Simon Fraser University, Burnaby, BC, Canada
>>>
>>>
>>>     --
>>>     Matthew Laird
>>>     Lead Software Developer, Bioinformatics
>>>     Brinkman Laboratory
>>>     Simon Fraser University, Burnaby, BC, Canada
>>>
>>>
>>>
>>>
>>> --
>>>     Claudio Martella
>>> claudio.martella@gmail.com <mailto:claudio.martella@**gmail.com<claudio.martella@gmail.com>
>>> >
>>>
>>
>> --
>> Matthew Laird
>> Lead Software Developer, Bioinformatics
>> Brinkman Laboratory
>> Simon Fraser University, Burnaby, BC, Canada
>>
>
>

Mime
View raw message