giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Etienne Dumoulin <etienne.dumou...@idiro.com>
Subject Re: Heterogeneous Vertex Types
Date Fri, 27 Jul 2012 15:10:00 GMT
Hi,

I am not sure to say something stupid or not but if you have two kind of
vertexes,
meaning is there are different in there values... I would looking at doing
something
this way. Have a Writable type which offer options and different
functionalities.
Accessing it from a getVertexValue(). You can load correctly what you need
in the read methods of Writable...

Regards,

√Čtienne

On 27 July 2012 14:32, Nick West <nick.west@benchmarksolutions.com> wrote:

>  Thanks for the quick reply.
>
>  That (using one uber-class) was my initial thought when I ran into this
> problem, however I was wondering if there was a solution that would use the
> type hierarchy (which admittedly is a bit more flexible in scala than java,
> so this may not be possible).  Is there something in the Giraph
> architecture that precludes this?
>
>  Thanks,
> Nick
>
>  On Jul 25, 2012, at 7:11 PM, Jonathan Bishop wrote:
>
>  Nick,
>
> You may want to reconsider your approach as I don't think Giraph will be
> happy with this.
>
> How about using a new vertex value class with BasicVertex which can do
> both behaviours for you? Same for edge and message classes. This should be
> enough do what you want.
>
> Jon
>
> On Wed, Jul 25, 2012 at 3:29 PM, Nick West <
> nick.west@benchmarksolutions.com> wrote:
>
>>  Hi,
>>
>>  I'm working on implementing a belief propagation algorithm over Giraph.
>>  (Do you know if anyone has done this before?)  This requires having (at
>> least) two different types of vertices implemented (values and factors) and
>> different types of messages sent between different vertices.
>>
>>  I've been able to set up and run my own vertices (with my own custom
>> readers and writers), however, whenever I try to extend this to a more
>> complex  case I run into problems.  For example, suppose aI have the
>> following two vertex types:
>>
>>  class FooVertex() extends BasicVertex[IntWritable, IntWritable, Text,
>> IntWritable]
>> class BarVertex() extends BasicVertex[IntWritable, IntWritable, Text,
>> IntWritable]
>>
>>  (both of which run fine in the basic set up), and I then configure the
>> GiraphJob with the following,
>>
>>  job.setVertexClass(classOf[BasicVertex[IntWritable, IntWritable, Text,
>> IntWritable]])
>>
>>  (as opposed to with FooVertex or BarVertex).
>>
>>  I get a null pointer error in GraphMapper (at line 204 in the version
>> of the code that I am using - just after the log statement about the input
>> format vertex index type not being known, and at the line where it checks
>> if the vertextIndexTypes are the same).
>>
>>  Do you know what is causing this? (The stack trace is below.)  Have you
>> tried to instantiate different classes for different vertices in the past?
>>
>>  One last piece of info that might explain things: I'm developing in
>> scala, which I have seen behave a little bit poorly with java reflection -
>> could that be a cause?
>>
>>  Thanks,
>> *Nick West
>> **
>> *
>> Stack Trace:
>>
>>  ./bin/hadoop jar
>> ~/code/watchdog/modules/inference/target/inference-assembly-1.0.jar
>> bmrk.inference.giraph.IdiotRunner ising ising_out 4
>>  Input arguments:
>> [0] = ising
>> [1] = ising_out
>> [2] = 4
>> setting vertex class to: org.apache.giraph.graph.BasicVertex
>> setting vertex input class to:
>> bmrk.inference.giraph.IdiotVertexInputFormat
>> setting vertex output class to:
>> bmrk.inference.giraph.IdiotVertexOutputFormat
>> 12/07/25 17:48:54 INFO mapred.JobClient: Running job:
>> job_201207250932_0012
>> 12/07/25 17:48:55 INFO mapred.JobClient:  map 0% reduce 0%
>> 12/07/25 17:52:43 INFO mapred.JobClient: Task Id :
>> attempt_201207250932_0012_m_000000_0, Status : FAILED
>> java.lang.NullPointerException
>> at
>> org.apache.giraph.graph.GraphMapper.determineClassTypes(GraphMapper.java:204)
>> at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:392)
>> at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:655)
>> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
>> at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at javax.security.auth.Subject.doAs(Subject.java:396)
>> at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
>> at org.apache.hadoop.mapred.Child.main(Child.java:249)
>>
>>
>>  Benchmark Solutions
>> 101 Park Avenue - 7th Floor
>> New York, NY 10178
>> Tel +1.212.220.4739 | Mobile +1.646.267.4324
>> *www.benchmarksolutions.com * <http://www.benchmarksolutions.com/>
>> **
>> *<image001.png>
>>
>>
>>
>>    *
>> **
>>
>
>
> *
> Nick West
> *
> *
> *Benchmark Solutions
> 101 Park Avenue - 7th Floor
> New York, NY 10178
> Tel +1.212.220.4739 | Mobile +1.646.267.4324
> *www.benchmarksolutions.com * <http://www.benchmarksolutions.com/>
> **
> *
>
>
>
>    *
> **
>

Mime
View raw message