giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Silvio Di gregorio <silvio.digrego...@gmail.com>
Subject Re: connected components example in giraph 1.0
Date Mon, 07 Oct 2013 06:40:18 GMT
*As i said i have builded*

*giraph-examples-1.0.0-for-hadoop-2.0.0-cdh4.1.2-jar-with-dependencies.jar*

*for cdh4, successfully. The job start to monitoring the success rate:*

*13/10/07 08:28:45 INFO mapred.JobClient:  map 0% reduce 0%*

*but then*

*Error running child
java.lang.IllegalStateException: run: Caught an unrecoverable
exception java.io.FileNotFoundException: File
_bsp/_defaultZkManagerDir/job_201309181636_0678/_zkServer does not
exist.
*

*...*

*Caused by: java.lang.RuntimeException: java.io.FileNotFoundException:
File _bsp/_defaultZkManagerDir/job_201309181636_0678/_zkServer does
not exist.*




2013/10/5 Silvio Di gregorio <silvio.digregorio@gmail.com>

> I ha ve build w/ hadoop_cdh4.1.2 parameter.
> Something is changed, monday i report the result. Now the farm is closed.
> Il giorno 05/ott/2013 14:06, "Claudio Martella" <
> claudio.martella@gmail.com> ha scritto:
>
> Oh, right, -vof is in trunk. Anyway it looks like you built giraph for the
>> wrong profile. You mentioned you're running on 2.0, but your giraph is
>> built for 0.20.203. try building with a profile for your hadoop version.
>>
>>
>> On Fri, Oct 4, 2013 at 2:35 PM, Silvio Di gregorio <
>> silvio.digregorio@gmail.com> wrote:
>>
>>>
>>> org.apache.commons.cli.UnrecognizedOptionException: Unrecognized option:
>>> -vof
>>>
>>> in 1.0 version is
>>>  -of,--outputFormat <arg>               Vertex output format
>>>  -op,--outputPath <arg>                 Vertex output path
>>>
>>>
>>>
>>> 2013/10/4 Claudio Martella <claudio.martella@gmail.com>
>>>
>>>> did you try the argument (-vof) i suggested?
>>>>
>>>>
>>>> On Fri, Oct 4, 2013 at 2:13 PM, Silvio Di gregorio <
>>>> silvio.digregorio@gmail.com> wrote:
>>>>
>>>>>
>>>>> i've specified  -of
>>>>>  org.apache.giraph.io.formats.IdWithValueTextOutputFormat
>>>>>
>>>>> but the same error was produced
>>>>>
>>>>> Exception in thread "main" java.lang.IncompatibleClassChangeError:
>>>>> Found interface org.apache.hadoop.mapreduce.JobContext, but class was
>>>>> expected
>>>>>         at
>>>>> org.apache.giraph.bsp.BspOutputFormat.checkOutputSpecs(BspOutputFormat.java:43)
>>>>>         at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:984)
>>>>>         at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:945)
>>>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>>>         at javax.security.auth.Subject.doAs(Subject.java:396)
>>>>>         at
>>>>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1408)
>>>>>         at
>>>>> org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:945)
>>>>>         at org.apache.hadoop.mapreduce.Job.submit(Job.java:566)
>>>>>         at
>>>>> org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:596)
>>>>>         at org.apache.giraph.job.GiraphJob.run(GiraphJob.java:237)
>>>>>         at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:94)
>>>>>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
>>>>>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
>>>>>         at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
>>>>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>         at
>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>>         at
>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>>         at java.lang.reflect.Method.invoke(Method.java:597)
>>>>>         at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
>>>>>
>>>>>
>>>>>
>>>>> 2013/10/4 Claudio Martella <claudio.martella@gmail.com>
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> you need to specify the vertex outputformat class (-vof option),
e.g.
>>>>>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat.
>>>>>>
>>>>>>
>>>>>> On Fri, Oct 4, 2013 at 1:06 PM, Silvio Di gregorio <
>>>>>> silvio.digregorio@gmail.com> wrote:
>>>>>>
>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> I hope I have sent to the right address.
>>>>>>>
>>>>>>> i have a graph (directed and unweighted) stored in hdfs like
a
>>>>>>> adjacency list (140Milions of edges 6Milions of vertex)
>>>>>>>
>>>>>>> node<tab>neighbors
>>>>>>>
>>>>>>> 23   2   1    3    4    3
>>>>>>>
>>>>>>> 1 999 99923 909 .......
>>>>>>>
>>>>>>> ......
>>>>>>>
>>>>>>> hadoop version Hadoop 2.0.0-cdh4.3.0 - java 1.6
>>>>>>>
>>>>>>>
>>>>>>> I have executed the giraph-1.0 connected components example,
in this
>>>>>>> fashion
>>>>>>>
>>>>>>>> > hadoop jar /usr/local/giraph/giraph-examples/target/giraph-
>>>>>>>> examples-1.0.0-for-hadoop-0.20.203.0-jar-with-dependencies.jar
>>>>>>>> org.apache.giraph.GiraphRunner org.apache.giraph.examples.ConnectedComponentsVertex
>>>>>>>> -vif org.apache.giraph.io.formats.IntIntNullTextInputFormat
-vip
>>>>>>>> /user/hdfs/lista_adj_txt -op connectedgiraph --workers 4
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> and then fail with:
>>>>>>>
>>>>>>> 13/10/04 09:28:29 INFO utils.ConfigurationUtils: No edge input
>>>>>>> format specified. Ensure your InputFormat does not require one.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 13/10/04 09:28:29 INFO utils.ConfigurationUtils: No output format
>>>>>>> specified. Ensure your OutputFormat does not require one.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 13/10/04 09:28:30 INFO job.GiraphJob: run: Since checkpointing
is
>>>>>>> disabled (default), do not allow any task retries (setting
>>>>>>> mapred.map.max.attempts = 0, old value = 4)
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 13/10/04 09:28:31 WARN mapred.JobClient: Use GenericOptionsParser
>>>>>>> for parsing the arguments. Applications should implement Tool
for the same.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> 13/10/04 09:28:31 INFO mapred.JobClient: Cleaning up the staging
>>>>>>> area hdfs://
>>>>>>> srv-bigdata-dev-01.int.sose.it:8020/user/hdfs/.staging/job_201309181636_0535
>>>>>>>
>>>>>>> Exception in thread "main" java.lang.IncompatibleClassChangeError:
>>>>>>> Found interface org.apache.hadoop.mapreduce.JobContext, but class
was
>>>>>>> expected
>>>>>>>
>>>>>>>         at
>>>>>>> org.apache.giraph.bsp.BspOutputFormat.checkOutputSpecs(BspOutputFormat.java:43)
>>>>>>>
>>>>>>>         ......
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Thanks in advance
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>>    Claudio Martella
>>>>>>    claudio.martella@gmail.com
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>>    Claudio Martella
>>>>    claudio.martella@gmail.com
>>>>
>>>
>>>
>>
>>
>> --
>>    Claudio Martella
>>    claudio.martella@gmail.com
>>
>

Mime
View raw message