giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexander Frolov <alexndr.fro...@gmail.com>
Subject Re: IntIntNullTextInputFormat problem
Date Mon, 03 Feb 2014 17:52:44 GMT
It seems that I understood the problem. PageRank and
SingleSourceShortestPath both need weighted graph to compute.  I just
missed it somehow.
Thank you for help.


On Mon, Feb 3, 2014 at 9:25 PM, Alexander Frolov
<alexndr.frolov@gmail.com>wrote:

>
>
>
> On Mon, Feb 3, 2014 at 9:06 PM, Sebastian Schelter <ssc@apache.org> wrote:
>
>> If you just wanna play with Giraph, I suggest you start with the PageRank
>> example:
>>
>> https://giraph.apache.org/pagerank.html
>
>
> I have seen it.  I think there is no much difference in SSSP and PageRank
> examples. By the way, I have not found LongDoubleNullTextInputFormat in my
> sources (I am using release-1.0.0-RC2-152-g585511f) which is mentioned
> there.
> The same issue with SimplePageRankComputation.
>
> [hduser@localhost giraph]$ $HADOOP_HOME/bin/hadoop jar
> $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
> org.apache.giraph.GiraphRunner
> org.apache.giraph.examples.SimplePageRankComputation -vif
> org.apache.giraph.io.formats.LongLongNullTextInputFormat -vip
> /user/hduser/input/test -vof
> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
> /user/hduser/output/amazon-shortestpaths -w 1
> 14/02/03 21:24:16 INFO utils.ConfigurationUtils: No edge input format
> specified. Ensure your InputFormat does not require one.
> 14/02/03 21:24:16 INFO utils.ConfigurationUtils: No edge output format
> specified. Ensure your OutputFormat does not require one.
> Exception in thread "main" java.lang.IllegalArgumentException:
> checkClassTypes: vertex value types not assignable, computation - class
> org.apache.hadoop.io.DoubleWritable, VertexInputFormat - class
> org.apache.hadoop.io.LongWritable
>  at
> org.apache.giraph.job.GiraphConfigurationValidator.checkAssignable(GiraphConfigurationValidator.java:381)
> at
> org.apache.giraph.job.GiraphConfigurationValidator.verifyVertexInputFormatGenericTypes(GiraphConfigurationValidator.java:228)
>  at
> org.apache.giraph.job.GiraphConfigurationValidator.validateConfiguration(GiraphConfigurationValidator.java:141)
> at
> org.apache.giraph.utils.ConfigurationUtils.parseArgs(ConfigurationUtils.java:214)
>  at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>  at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
>  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>  at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
>  at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>
>
>>
>> --sebastian
>>
>>
>> On 02/03/2014 05:49 PM, Alexander Frolov wrote:
>>
>>> I've tried .LongLongNullTextInputFormat and got this:
>>>
>>> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop jar
>>> $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-
>>> SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
>>> org.apache.giraph.GiraphRunner
>>> org.apache.giraph.examples.SimpleShortestPathsComputation -vif
>>> org.apache.giraph.io.formats.LongLongNullTextInputFormat -vip
>>> /user/hduser/input/test -vof
>>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
>>> /user/hduser/output/amazon-shortestpaths -w 1
>>> 14/02/03 20:48:41 INFO utils.ConfigurationUtils: No edge input format
>>> specified. Ensure your InputFormat does not require one.
>>> 14/02/03 20:48:41 INFO utils.ConfigurationUtils: No edge output format
>>> specified. Ensure your OutputFormat does not require one.
>>> Exception in thread "main" java.lang.IllegalArgumentException:
>>> checkClassTypes: vertex value types not assignable, computation - class
>>> org.apache.hadoop.io.DoubleWritable, VertexInputFormat - class
>>> org.apache.hadoop.io.LongWritable
>>> at
>>> org.apache.giraph.job.GiraphConfigurationValidator.checkAssignable(
>>> GiraphConfigurationValidator.java:381)
>>> at
>>> org.apache.giraph.job.GiraphConfigurationValidator.
>>> verifyVertexInputFormatGenericTypes(GiraphConfigurationValidator.
>>> java:228)
>>> at
>>> org.apache.giraph.job.GiraphConfigurationValidator.
>>> validateConfiguration(GiraphConfigurationValidator.java:141)
>>> at
>>> org.apache.giraph.utils.ConfigurationUtils.parseArgs(
>>> ConfigurationUtils.java:214)
>>> at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74)
>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>>> at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(
>>> NativeMethodAccessorImpl.java:57)
>>> at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(
>>> DelegatingMethodAccessorImpl.java:43)
>>> at java.lang.reflect.Method.invoke(Method.java:606)
>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>>
>>>
>>>
>>>
>>> On Mon, Feb 3, 2014 at 8:38 PM, Alexander Frolov
>>> <alexndr.frolov@gmail.com>wrote:
>>>
>>>
>>>>
>>>>
>>>> On Mon, Feb 3, 2014 at 8:30 PM, Sebastian Schelter <ssc@apache.org>
>>>> wrote
>>>>
>>>>  What types does the vertex class that you use in your computation
>>>>> require?
>>>>>
>>>>
>>>>
>>>> I see.
>>>>
>>>> public class SimpleShortestPathsComputation extends BasicComputation<
>>>>      LongWritable, DoubleWritable, FloatWritable, DoubleWritable> {..}
>>>>
>>>> So that is a mistake.
>>>> Thank you very much.
>>>>
>>>>
>>>  Is there any means to import data from file like this (amazon0601.txt
>>>> from
>>>> SNAP data collection):
>>>>
>>>> 0   1
>>>> 0   2
>>>> 0   3
>>>> 0   4
>>>> 0   5
>>>> 0   6
>>>> 0   7
>>>> 0   8
>>>> 0   9
>>>> 0   10
>>>> ...
>>>>
>>>> and to perform basic tests from giraph-examples? Most of the tests are
>>>> using LongWritable for vertex id.
>>>>
>>>>
>>>>
>>>>> On 02/03/2014 05:28 PM, Alexander Frolov wrote:
>>>>>
>>>>>  On Mon, Feb 3, 2014 at 8:20 PM, Sebastian Schelter <ssc@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>   Hi Alexander,
>>>>>>
>>>>>>>
>>>>>>> what do you use as type for your vertex ids? It looks that you
are
>>>>>>> trying
>>>>>>> to use longs, while IntIntNullTextInputFormat only provides ints,
>>>>>>> that
>>>>>>> could be the error.
>>>>>>>
>>>>>>> --sebastian
>>>>>>>
>>>>>>>
>>>>>>>   I also thought so, but I could not understand why it regard
it as
>>>>>>> long.
>>>>>>>
>>>>>> I just use  -vif org.apache.giraph.io.formats.
>>>>>> IntIntNullTextInputFormat
>>>>>> -vip /user/hduser/input/test -vof
>>>>>>
>>>>>> and file consists only of two lines:
>>>>>>
>>>>>> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop dfs -cat
>>>>>> /user/hduser/input/test
>>>>>> 0 1
>>>>>> 1 0
>>>>>>
>>>>>> I dont know any other means to specify type of vertex ids.
>>>>>>
>>>>>>
>>>>>>
>>>>>>  On 02/03/2014 05:06 PM, Alexander Frolov wrote:
>>>>>>>
>>>>>>>   Hello,
>>>>>>>
>>>>>>>>
>>>>>>>> I am trying to load data in from file in IntIntNullTextInputFormat.
>>>>>>>> But I
>>>>>>>> get is the following message:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop jar
>>>>>>>> $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-
>>>>>>>> SNAPSHOT-for-hadoop-0.20.203.0-jar-with-dependencies.jar
>>>>>>>> org.apache.giraph.GiraphRunner
>>>>>>>> org.apache.giraph.examples.SimpleShortestPathsComputation
-vif
>>>>>>>> org.apache.giraph.io.formats.IntIntNullTextInputFormat -vip
>>>>>>>> /user/hduser/input/test -vof
>>>>>>>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat
-op
>>>>>>>> /user/hduser/output/amazon-shortestpaths -w 1
>>>>>>>>
>>>>>>>> 14/02/03 19:46:08 INFO utils.ConfigurationUtils: No edge
input
>>>>>>>> format
>>>>>>>> specified. Ensure your InputFormat does not require one.
>>>>>>>> 14/02/03 19:46:08 INFO utils.ConfigurationUtils: No edge
output
>>>>>>>> format
>>>>>>>> specified. Ensure your OutputFormat does not require one.
>>>>>>>> Exception in thread "main" java.lang.IllegalArgumentException:
>>>>>>>> checkClassTypes: vertex index types not assignable, computation
-
>>>>>>>> class
>>>>>>>> org.apache.hadoop.io.LongWritable, VertexInputFormat - class
>>>>>>>> org.apache.hadoop.io.IntWritable
>>>>>>>> at
>>>>>>>> org.apache.giraph.job.GiraphConfigurationValidator.checkAssignable(
>>>>>>>> GiraphConfigurationValidator.java:381)
>>>>>>>> at
>>>>>>>> org.apache.giraph.job.GiraphConfigurationValidator.
>>>>>>>> verifyVertexInputFormatGenericTypes(GiraphConfigurationValidator.
>>>>>>>> java:226)
>>>>>>>> at
>>>>>>>> org.apache.giraph.job.GiraphConfigurationValidator.
>>>>>>>> validateConfiguration(
>>>>>>>> GiraphConfigurationValidator.java:141)
>>>>>>>> at
>>>>>>>> org.apache.giraph.utils.ConfigurationUtils.parseArgs(
>>>>>>>> ConfigurationUtils.java:214)
>>>>>>>> at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74)
>>>>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>>>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>>>>>>>> at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
>>>>>>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>>>> at
>>>>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(
>>>>>>>> NativeMethodAccessorImpl.java:57)
>>>>>>>> at
>>>>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(
>>>>>>>> DelegatingMethodAccessorImpl.java:43)
>>>>>>>> at java.lang.reflect.Method.invoke(Method.java:606)
>>>>>>>> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> The input file is here (just for testing):
>>>>>>>>
>>>>>>>> [hduser@localhost ~]$ $HADOOP_HOME/bin/hadoop dfs -cat
>>>>>>>> /user/hduser/input/test
>>>>>>>> 0 1
>>>>>>>> 1 0
>>>>>>>>
>>>>>>>> Could you please help with this issue?
>>>>>>>>
>>>>>>>> Best,
>>>>>>>>      Alex
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message