giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Liannet Reyes <lianne...@gmail.com>
Subject Re: using partition class
Date Wed, 02 Apr 2014 16:24:13 GMT
Thanks again  for your help Lukas,

It works!

Hereby the full command line for future reference.

 hadoop jar $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-
> SNAPSHOT-for-hadoop-1.2.1-jar-with-dependencies.jar
> org.apache.giraph.GiraphRunner
> -Dgiraph.graphPartitionerFactoryClass=org.apache.giraph.partition.
> SimpleLongRangePartitionerFactory -Dgiraph.vertexKeySpaceSize=5
> org.apache.giraph.examples.SimpleShortestPathsComputation -vif
> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
> -vip /user/hduser/input/tiny_graph.txt -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat
> -op /user/hduser/output/shortestpaths  -w 2
>
>
I guest the next step is to implement my custom PartitionerFactoryClass.

Best Regards,

Liannet

2014-04-02 17:32 GMT+02:00 Lukas Nalezenec <lukas.nalezenec@firma.seznam.cz>
:

>  Hi,
>
> 1. It have to work if you set giraph.vertexKeySpaceSize just after or
> before the other property. But it can interfere with the other command line
> options :(.
> 2. Its Number of Vertexes in your graph.
>
> Regards,
> Lukas
>
>
>
> On 2.4.2014 17:21, Liannet Reyes wrote:
>
> Thank you very much Lukas,
>
>  I've notice that using SimpleLongRangePartitionerFactory require the GiraphConstant
> giraph.vertexKeySpaceSize to be set,
>  but I have 2 questions:
>
>  1- Where should I set the giraph constant values? I've tried using
> -Dgiraph.vertexKeySpaceSize=256 after -Dgiraph.graphPartitionerFactoryClass
> but it didn't work
> 2- Which unit is this PARTITION_VERTEX_KEY_SPACE_SIZE? Bytes??
>
>  Regards,
>
>  Liannet
>
>
>
>
>  2014-04-01 20:55 GMT+02:00 Lukas Nalezenec <
> lukas.nalezenec@firma.seznam.cz>:
>
>>  Hi,
>> Partition is for storing vertexes, Partitioner is for distributing
>> vertexes between Partitions.
>>
>> Try this:
>> -Dgiraph.graphPartitionerFactoryClass=
>> org.apache.giraph.partition.SimpleLongRangePartitionerFactory
>>
>> Its good idea to switch Partition to ByteArrayPartition (or better).
>>
>> Lukas
>>
>>
>>
>> On 1.4.2014 19:59, Liannet Reyes wrote:
>>
>>  Hi all,
>>
>>  I´m trying to use the giraph partition class to improve my graph
>> distribution, however I hardly found info on how to use it.
>> I´m testing the SimpleShortestPathsComputation example with the
>> tiny_graph.txt input file and the SimpleLongRangePartitionerFactory
>> out-of-the-box in org.apache.giraph.partition but without success.
>> Apparentely I need a Partition class and the
>> SimpleLongRangePartitionerFactory is not.
>>
>>  Can you kindly please let me know the proper way to do it? See below my
>> command and console output
>>
>>
>> hadoop jar
>> $GIRAPH_HOME/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-1.2.1-jar-with-dependencies.jar
>> org.apache.giraph.GiraphRunner
>> org.apache.giraph.examples.SimpleShortestPathsComputation -vif
>> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
>> -vip /user/hduser/input/tiny_graph.txt -vof
>> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
>> /user/hduser/output/shortestpaths -pc
>> org.apache.giraph.partition.SimpleLongRangePartitionerFactory -w 2
>>
>>  Exception in thread "main" java.lang.RuntimeException: class
>> org.apache.giraph.partition.SimpleLongRangePartitionerFactory not
>> org.apache.giraph.partition.Partition
>>  at org.apache.hadoop.conf.Configuration.setClass(Configuration.java:915)
>>  at org.apache.giraph.conf.ClassConfOption.set(ClassConfOption.java:182)
>>  at
>> org.apache.giraph.conf.GiraphConfiguration.setPartitionClass(GiraphConfiguration.java:594)
>>  at
>> org.apache.giraph.utils.ConfigurationUtils.populateGiraphConfiguration(ConfigurationUtils.java:375)
>>  at
>> org.apache.giraph.utils.ConfigurationUtils.parseArgs(ConfigurationUtils.java:207)
>>  at org.apache.giraph.GiraphRunner.run(GiraphRunner.java:74)
>>  at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>  at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>>  at org.apache.giraph.GiraphRunner.main(GiraphRunner.java:124)
>>  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>  at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>  at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>  at java.lang.reflect.Method.invoke(Method.java:606)
>>  at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>>
>>  Thanks in advance
>>
>>  Liannet
>>
>>
>>
>
>

Mime
View raw message