giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Holland <d99991...@mydit.ie>
Subject Re: SimpleShortestPathsComputation with Edge List input file
Date Mon, 03 Jun 2013 21:22:37 GMT
Thank you for your response Claudio.

I updated the command with the input class you suggested.

*bin/hadoop jar
/home/ubuntu/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-1.0.2-jar-with-dependencies.jar
*
* org.apache.giraph.GiraphRunner
org.apache.giraph.examples.SimpleShortestPathsComputation *
*-eif org.apache.giraph.io.formats.IntNullTetxEdgeInputFormat*
*-eip /simpleEdgeList/SimpleEdgeList.tsv*
*-of org.apache.giraph.io.formats.IdWithValueTextOutputFormat *
*-op /outShortestEdgeList01 *
*-w 1*

Unfortunately I am getting an error message

*13/06/03 23:00:08 INFO utils.ConfigurationUtils: No vertex input format
specified. Ensure your InputFormat does not require one.*
*Exception in thread "main" java.lang.IllegalArgumentException:
checkClassTypes: Vertex index types don't match, vertex - class
org.apache.hadoop.io.LongWritable, edge input format - class
org.apache.hadoop.io.IntWritable*
* at
org.apache.giraph.job.GiraphConfigurationValidator.verifyEdgeInputFormatGenericTypes(GiraphConfigurationValidator.java:266)
*
* at
org.apache.giraph.job.GiraphConfigurationValidator.validateConfiguration(GiraphConfigurationValidator.java:125)
*
* at
org.apache.giraph.utils.ConfigurationUtils.parseArgs(ConfigurationUtils.java:155)
*
* at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74)*
* at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)*
* at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)*
* at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)*
* at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)*
* at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
*
* at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
*
* at java.lang.reflect.Method.invoke(Method.java:597)*
* at org.apache.hadoop.util.RunJar.main(RunJar.java:156)*


On 3 June 2013 21:00, Claudio Martella <claudio.martella@gmail.com> wrote:

> with the only problem that you picked an abstract class again...
> I advised you to use an inputformat that has the name of the types in the
> class name, hence org.apache.giraph.io.formats.IntNullTextEdgeInputFormat
> should work for you.
>
>
> On Mon, Jun 3, 2013 at 9:34 PM, Peter Holland <d99991048@mydit.ie> wrote:
>
>> Thank you for the advice Claudio
>>
>> I updated the run command to use different io classes
>>
>> *bin/hadoop jar
>> /home/ubuntu/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-1.0.2-jar-with-dependencies.jar
>> *
>> * org.apache.giraph.GiraphRunner
>> org.apache.giraph.examples.SimpleShortestPathsComputation *
>> *-eif org.apache.giraph.io.EdgeInputFormat*
>> * -eip /simpleEdgeList/SimpleEdgeList.tsv*
>> *-of org.apache.giraph.io.formats.IdWithValueTextOutputFormat *
>> *-op /outShortestEdgeList01 *
>> *-w 1*
>> *
>> *
>> This code does start a MapReduce job but progress stays at 0%. The log
>> file for the job has the following IOException error;
>>
>> *MapAttempt TASK_TYPE="MAP" TASKID="task_201306031954_0002_m_000000"
>> TASK_ATTEMPT_ID="attempt_201306031954_0002_m_000000_0" TASK_STATUS="FAILED"
>> FINISH_TIME="1370282492527" HOSTNAME="ubuntu-VirtualBox"
>> ERROR="java\.lang\.Throwable: Child Error*
>> * at org\.apache\.hadoop\.mapred\.TaskRunner\.run(TaskRunner\.java:271)*
>> *Caused by: java\.io\.IOException: Task process exit with nonzero status
>> of 1\.*
>> * at org\.apache\.hadoop\.mapred\.TaskRunner\.run(TaskRunner\.java:258)*
>>
>> So, this leaves 3 questions;
>> Is the edge list file format correct? (a tab separated file with a .tsv
>> extension)
>> Is the input class correct?
>> Is the output class correct?
>>
>> Thank you,
>> Peter
>>
>>
>>
>>
>> On 3 June 2013 01:05, Claudio Martella <claudio.martella@gmail.com>wrote:
>>
>>> Hi Peter,
>>>
>>> shortly, those are abstract classes, that's why you cannot instantiate
>>> them. You'll have to use a specific class extending those classes that are
>>> aware of the types of the signature of the vertex (I, V, E, M). check out
>>> some classes in the format package that have those types in the class name.
>>>
>>>
>>> On Mon, Jun 3, 2013 at 1:25 AM, Peter Holland <d99991048@mydit.ie>wrote:
>>>
>>>> Hello,
>>>> I'm new to Giraph and I'm trying to run SimpleShortestPathsComputation
>>>> using an edge list input file. I have some questions and and error message
>>>> that hopefully I can get some help with.
>>>>
>>>> Edge List File Format
>>>> What is the correct format for an edge list input file?
>>>> I have a .tsv file with a vertex represented as an integer. Is this
>>>> correct?
>>>>
>>>> 5 11
>>>> 1 6
>>>> 6 9
>>>> 6 8
>>>> 8 9
>>>> .....
>>>>
>>>> Input File Class:
>>>> Is org.apache.giraph.io.formats.*TextEdgeInputFormat *the only input
>>>> format that can be used for edge lists?
>>>>
>>>> Output File Class:
>>>> Does the output format depend on the job you are running? I have been
>>>> using org.apache.giraph.io.formats.*TextVertexOutputFormat* for
>>>> SimpleShortestPathsComputation.
>>>>
>>>> Run Command:
>>>> So this is the command I am using to try to run the
>>>> SimpleShortestPathsComputation using an edge list input file.
>>>>
>>>> *bin/hadoop jar
>>>> /home/ubuntu/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-1.0.2-jar-with-dependencies.jar
>>>> org.apache.giraph.GiraphRunner
>>>> org.apache.giraph.examples.SimpleShortestPathsComputation *
>>>> *-eif org.apache.giraph.io.formats.TextEdgeInputFormat *
>>>> *-eip /simpleEdgeList/SimpleEdgeList.tsv *
>>>> *-of org.apache.giraph.io.formats.TextVertexOutputFormat *
>>>> *-op /outShortest*
>>>> *-w 3*
>>>>
>>>> Error Message
>>>> When I run the above command I get the following error message:
>>>> Exception in thread "main" java.lang.IllegalStateException:
>>>> newInstance: Couldn't instantiate
>>>> org.apache.giraph.io.formats.TextVertexOutputFormat
>>>>
>>>> Thank you,
>>>> Peter
>>>>
>>>
>>>
>>>
>>> --
>>>    Claudio Martella
>>>    claudio.martella@gmail.com
>>>
>>
>>
>
>
> --
>    Claudio Martella
>    claudio.martella@gmail.com
>

Mime
View raw message