giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Silvio Di gregorio <silvio.digrego...@gmail.com>
Subject Re: connected components example in giraph 1.0
Date Sat, 05 Oct 2013 12:35:45 GMT
I ha ve build w/ hadoop_cdh4.1.2 parameter.
Something is changed, monday i report the result. Now the farm is closed.
Il giorno 05/ott/2013 14:06, "Claudio Martella" <claudio.martella@gmail.com>
ha scritto:

> Oh, right, -vof is in trunk. Anyway it looks like you built giraph for the
> wrong profile. You mentioned you're running on 2.0, but your giraph is
> built for 0.20.203. try building with a profile for your hadoop version.
>
>
> On Fri, Oct 4, 2013 at 2:35 PM, Silvio Di gregorio <
> silvio.digregorio@gmail.com> wrote:
>
>>
>> org.apache.commons.cli.UnrecognizedOptionException: Unrecognized option:
>> -vof
>>
>> in 1.0 version is
>>  -of,--outputFormat <arg>               Vertex output format
>>  -op,--outputPath <arg>                 Vertex output path
>>
>>
>>
>> 2013/10/4 Claudio Martella <claudio.martella@gmail.com>
>>
>>> did you try the argument (-vof) i suggested?
>>>
>>>
>>> On Fri, Oct 4, 2013 at 2:13 PM, Silvio Di gregorio <
>>> silvio.digregorio@gmail.com> wrote:
>>>
>>>>
>>>> i've specified  -of
>>>>  org.apache.giraph.io.formats.IdWithValueTextOutputFormat
>>>>
>>>> but the same error was produced
>>>>
>>>> Exception in thread "main" java.lang.IncompatibleClassChangeError:
>>>> Found interface org.apache.hadoop.mapreduce.JobContext, but class was
>>>> expected
>>>>         at
>>>> org.apache.giraph.bsp.BspOutputFormat.checkOutputSpecs(BspOutputFormat.java:43)
>>>>         at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:984)
>>>>         at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:945)
>>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>>         at javax.security.auth.Subject.doAs(Subject.java:396)
>>>>         at
>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
>>>>         at
>>>> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:945)
>>>>         at org.apache.hadoop.mapreduce.Job.submit(Job.java:566)
>>>>         at
>>>> org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:596)
>>>>         at org.apache.giraph.job.GiraphJob.run(GiraphJob.java:237)
>>>>         at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:94)
>>>>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>>>>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>>>>         at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
>>>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>         at
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>         at
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>         at java.lang.reflect.Method.invoke(Method.java:597)
>>>>         at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
>>>>
>>>>
>>>>
>>>> 2013/10/4 Claudio Martella <claudio.martella@gmail.com>
>>>>
>>>>> Hi,
>>>>>
>>>>> you need to specify the vertex outputformat class (-vof option), e.g.
>>>>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat.
>>>>>
>>>>>
>>>>> On Fri, Oct 4, 2013 at 1:06 PM, Silvio Di gregorio <
>>>>> silvio.digregorio@gmail.com> wrote:
>>>>>
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I hope I have sent to the right address.
>>>>>>
>>>>>> i have a graph (directed and unweighted) stored in hdfs like a
>>>>>> adjacency list (140Milions of edges 6Milions of vertex)
>>>>>>
>>>>>> node<tab>neighbors
>>>>>>
>>>>>> 23   2   1    3    4    3
>>>>>>
>>>>>> 1 999 99923 909 .......
>>>>>>
>>>>>> ......
>>>>>>
>>>>>> hadoop version Hadoop 2.0.0-cdh4.3.0 - java 1.6
>>>>>>
>>>>>>
>>>>>> I have executed the giraph-1.0 connected components example, in this
>>>>>> fashion
>>>>>>
>>>>>>> > hadoop jar /usr/local/giraph/giraph-examples/target/giraph-
>>>>>>> examples-1.0.0-for-hadoop-0.20.203.0-jar-with-dependencies.jar
>>>>>>> org.apache.giraph.GiraphRunner org.apache.giraph.examples.ConnectedComponentsVertex
>>>>>>> -vif org.apache.giraph.io.formats.IntIntNullTextInputFormat -vip
>>>>>>> /user/hdfs/lista_adj_txt -op connectedgiraph --workers 4
>>>>>>
>>>>>>
>>>>>>
>>>>>> and then fail with:
>>>>>>
>>>>>> 13/10/04 09:28:29 INFO utils.ConfigurationUtils: No edge input format
>>>>>> specified. Ensure your InputFormat does not require one.
>>>>>>
>>>>>>
>>>>>>
>>>>>> 13/10/04 09:28:29 INFO utils.ConfigurationUtils: No output format
>>>>>> specified. Ensure your OutputFormat does not require one.
>>>>>>
>>>>>>
>>>>>>
>>>>>> 13/10/04 09:28:30 INFO job.GiraphJob: run: Since checkpointing is
>>>>>> disabled (default), do not allow any task retries (setting
>>>>>> mapred.map.max.attempts = 0, old value = 4)
>>>>>>
>>>>>>
>>>>>>
>>>>>> 13/10/04 09:28:31 WARN mapred.JobClient: Use GenericOptionsParser
for
>>>>>> parsing the arguments. Applications should implement Tool for the
same.
>>>>>>
>>>>>>
>>>>>>
>>>>>> 13/10/04 09:28:31 INFO mapred.JobClient: Cleaning up the staging
area
>>>>>> hdfs://
>>>>>> srv-bigdata-dev-01.int.sose.it:8020/user/hdfs/.staging/job_201309181636_0535
>>>>>>
>>>>>> Exception in thread "main" java.lang.IncompatibleClassChangeError:
>>>>>> Found interface org.apache.hadoop.mapreduce.JobContext, but class
was
>>>>>> expected
>>>>>>
>>>>>>         at
>>>>>> org.apache.giraph.bsp.BspOutputFormat.checkOutputSpecs(BspOutputFormat.java:43)
>>>>>>
>>>>>>         ......
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> Thanks in advance
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>    Claudio Martella
>>>>>    claudio.martella@gmail.com
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>>    Claudio Martella
>>>    claudio.martella@gmail.com
>>>
>>
>>
>
>
> --
>    Claudio Martella
>    claudio.martella@gmail.com
>

Mime
View raw message