giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexander Frolov <alexndr.fro...@gmail.com>
Subject Re: IntIntNullTextInputFormat problem
Date Mon, 03 Feb 2014 17:25:15 GMT
On Mon, Feb 3, 2014 at 9:06 PM, Sebastian Schelter <ssc@apache.org> wrote:

> If you just wanna play with Giraph, I suggest you start with the PageRank
> example:
>
> https://giraph.apache.org/pagerank.html


I have seen it.  I think there is no much difference in SSSP and PageRank
examples. By the way, I have not found LongDoubleNullTextInputFormat in my
sources (I am using release-1.0.0-RC2-152-g585511f) which is mentioned
there.
The same issue with SimplePageRankComputation.

[hduser@localhost giraph]$ $HADOOP_HOME/bin/hadoop jar
$GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
org.apache.giraph.GiraphRunner
org.apache.giraph.examples.SimplePageRankComputation -vif
org.apache.giraph.io.formats.LongLongNullTextInputFormat -vip
/user/hduser/input/test -vof
org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
/user/hduser/output/amazon-shortestpaths -w 1
14/02/03 21:24:16 INFO utils.ConfigurationUtils: No edge input format
specified. Ensure your InputFormat does not require one.
14/02/03 21:24:16 INFO utils.ConfigurationUtils: No edge output format
specified. Ensure your OutputFormat does not require one.
Exception in thread "main" java.lang.IllegalArgumentException:
checkClassTypes: vertex value types not assignable, computation - class
org.apache.hadoop.io.DoubleWritable, VertexInputFormat - class
org.apache.hadoop.io.LongWritable
at
org.apache.giraph.job.GiraphConfigurationValidator.checkAssignable(GiraphConfigurationValidator.java:381)
at
org.apache.giraph.job.GiraphConfigurationValidator.verifyVertexInputFormatGenericTypes(GiraphConfigurationValidator.java:228)
at
org.apache.giraph.job.GiraphConfigurationValidator.validateConfiguration(GiraphConfigurationValidator.java:141)
at
org.apache.giraph.utils.ConfigurationUtils.parseArgs(ConfigurationUtils.java:214)
at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)


>
> --sebastian
>
>
> On 02/03/2014 05:49 PM, Alexander Frolov wrote:
>
>> I've tried .LongLongNullTextInputFormat and got this:
>>
>> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop jar
>> $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-
>> SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
>> org.apache.giraph.GiraphRunner
>> org.apache.giraph.examples.SimpleShortestPathsComputation -vif
>> org.apache.giraph.io.formats.LongLongNullTextInputFormat -vip
>> /user/hduser/input/test -vof
>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
>> /user/hduser/output/amazon-shortestpaths -w 1
>> 14/02/03 20:48:41 INFO utils.ConfigurationUtils: No edge input format
>> specified. Ensure your InputFormat does not require one.
>> 14/02/03 20:48:41 INFO utils.ConfigurationUtils: No edge output format
>> specified. Ensure your OutputFormat does not require one.
>> Exception in thread "main" java.lang.IllegalArgumentException:
>> checkClassTypes: vertex value types not assignable, computation - class
>> org.apache.hadoop.io.DoubleWritable, VertexInputFormat - class
>> org.apache.hadoop.io.LongWritable
>> at
>> org.apache.giraph.job.GiraphConfigurationValidator.checkAssignable(
>> GiraphConfigurationValidator.java:381)
>> at
>> org.apache.giraph.job.GiraphConfigurationValidator.
>> verifyVertexInputFormatGenericTypes(GiraphConfigurationValidator.
>> java:228)
>> at
>> org.apache.giraph.job.GiraphConfigurationValidator.validateConfiguration(
>> GiraphConfigurationValidator.java:141)
>> at
>> org.apache.giraph.utils.ConfigurationUtils.parseArgs(
>> ConfigurationUtils.java:214)
>> at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>> at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(
>> NativeMethodAccessorImpl.java:57)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(
>> DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:606)
>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>
>>
>>
>>
>> On Mon, Feb 3, 2014 at 8:38 PM, Alexander Frolov
>> <alexndr.frolov@gmail.com>wrote:
>>
>>
>>>
>>>
>>> On Mon, Feb 3, 2014 at 8:30 PM, Sebastian Schelter <ssc@apache.org>
>>> wrote
>>>
>>>  What types does the vertex class that you use in your computation
>>>> require?
>>>>
>>>
>>>
>>> I see.
>>>
>>> public class SimpleShortestPathsComputation extends BasicComputation<
>>>      LongWritable, DoubleWritable, FloatWritable, DoubleWritable> {..}
>>>
>>> So that is a mistake.
>>> Thank you very much.
>>>
>>>
>>  Is there any means to import data from file like this (amazon0601.txt
>>> from
>>> SNAP data collection):
>>>
>>> 0   1
>>> 0   2
>>> 0   3
>>> 0   4
>>> 0   5
>>> 0   6
>>> 0   7
>>> 0   8
>>> 0   9
>>> 0   10
>>> ...
>>>
>>> and to perform basic tests from giraph-examples? Most of the tests are
>>> using LongWritable for vertex id.
>>>
>>>
>>>
>>>> On 02/03/2014 05:28 PM, Alexander Frolov wrote:
>>>>
>>>>  On Mon, Feb 3, 2014 at 8:20 PM, Sebastian Schelter <ssc@apache.org>
>>>>> wrote:
>>>>>
>>>>>   Hi Alexander,
>>>>>
>>>>>>
>>>>>> what do you use as type for your vertex ids? It looks that you are
>>>>>> trying
>>>>>> to use longs, while IntIntNullTextInputFormat only provides ints,
that
>>>>>> could be the error.
>>>>>>
>>>>>> --sebastian
>>>>>>
>>>>>>
>>>>>>   I also thought so, but I could not understand why it regard it
as
>>>>>> long.
>>>>>>
>>>>> I just use  -vif org.apache.giraph.io.formats.
>>>>> IntIntNullTextInputFormat
>>>>> -vip /user/hduser/input/test -vof
>>>>>
>>>>> and file consists only of two lines:
>>>>>
>>>>> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop dfs -cat
>>>>> /user/hduser/input/test
>>>>> 0 1
>>>>> 1 0
>>>>>
>>>>> I dont know any other means to specify type of vertex ids.
>>>>>
>>>>>
>>>>>
>>>>>  On 02/03/2014 05:06 PM, Alexander Frolov wrote:
>>>>>>
>>>>>>   Hello,
>>>>>>
>>>>>>>
>>>>>>> I am trying to load data in from file in IntIntNullTextInputFormat.
>>>>>>> But I
>>>>>>> get is the following message:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop jar
>>>>>>> $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-
>>>>>>> SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
>>>>>>> org.apache.giraph.GiraphRunner
>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation -vif
>>>>>>> org.apache.giraph.io.formats.IntIntNullTextInputFormat -vip
>>>>>>> /user/hduser/input/test -vof
>>>>>>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
>>>>>>> /user/hduser/output/amazon-shortestpaths -w 1
>>>>>>>
>>>>>>> 14/02/03 19:46:08 INFO utils.ConfigurationUtils: No edge input
format
>>>>>>> specified. Ensure your InputFormat does not require one.
>>>>>>> 14/02/03 19:46:08 INFO utils.ConfigurationUtils: No edge output
>>>>>>> format
>>>>>>> specified. Ensure your OutputFormat does not require one.
>>>>>>> Exception in thread "main" java.lang.IllegalArgumentException:
>>>>>>> checkClassTypes: vertex index types not assignable, computation
-
>>>>>>> class
>>>>>>> org.apache.hadoop.io.LongWritable, VertexInputFormat - class
>>>>>>> org.apache.hadoop.io.IntWritable
>>>>>>> at
>>>>>>> org.apache.giraph.job.GiraphConfigurationValidator.checkAssignable(
>>>>>>> GiraphConfigurationValidator.java:381)
>>>>>>> at
>>>>>>> org.apache.giraph.job.GiraphConfigurationValidator.
>>>>>>> verifyVertexInputFormatGenericTypes(GiraphConfigurationValidator.
>>>>>>> java:226)
>>>>>>> at
>>>>>>> org.apache.giraph.job.GiraphConfigurationValidator.
>>>>>>> validateConfiguration(
>>>>>>> GiraphConfigurationValidator.java:141)
>>>>>>> at
>>>>>>> org.apache.giraph.utils.ConfigurationUtils.parseArgs(
>>>>>>> ConfigurationUtils.java:214)
>>>>>>> at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74)
>>>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>>>>>>> at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>>> at
>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(
>>>>>>> NativeMethodAccessorImpl.java:57)
>>>>>>> at
>>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(
>>>>>>> DelegatingMethodAccessorImpl.java:43)
>>>>>>> at java.lang.reflect.Method.invoke(Method.java:606)
>>>>>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> The input file is here (just for testing):
>>>>>>>
>>>>>>> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop dfs -cat
>>>>>>> /user/hduser/input/test
>>>>>>> 0 1
>>>>>>> 1 0
>>>>>>>
>>>>>>> Could you please help with this issue?
>>>>>>>
>>>>>>> Best,
>>>>>>>      Alex
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message