giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Avery Ching <ach...@apache.org>
Subject Re: Changing Graph Size
Date Mon, 23 Jan 2012 06:29:55 GMT
Not that updated I guess, but I have some presentations that are kind of 
recent (but still prior to the vertex range changes in GIRAPH-11 
unfortunately).  Here is my most recent one from October 2011 
(http://www.slideshare.net/averyching/20111014hortonworks).  There are 
some folks working on presentations for FOSDEM (Claudio) and Berlin 
Buzzwords (Jakob).  Maybe they have some up-to-date material?

Avery


On 1/22/12 7:23 PM, Deepak Nettem wrote:
> Awesome!
>
> I can't wait to try this out. Is there any other resources / updated 
> documentation where I can get insight into how Giraph works internally?
>
> I am specifically interested in understanding how Mapper-only makes 
> this entire thing possible, what are the key-value pairs (since it's 
> built on top of Hadoop), and where Zookeeper fits in.
>
> Best,
> Deepak
>
> On Sun, Jan 22, 2012 at 5:30 PM, Avery Ching <aching@apache.org 
> <mailto:aching@apache.org>> wrote:
>
>     Hi Deepak,
>
>     Answers inline.
>
>     Happy sunday!
>
>     Avery
>
>
>     On 1/22/12 11:30 AM, Deepak Nettem wrote:
>>     Hi Folks,
>>
>>     I have a Graph processing problem where after each iteration,
>>     some vertices get vanished. That is, they get merged into their
>>     neighbouring nodes based on certain conditions and the graph
>>     keeps getting simplified after each iteration.
>>
>>     I was wondering if Giraph is worth trying for this.
>>
>     Giraph can support this graph mutation at any iteration in the
>     graph.  See
>     https://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/graph/MutableVertex.java,
>     the method r emoveVertexRequest() will remove vertices for you
>     between iterations.
>
>
>>     I was going through the documentation and it says that the input
>>     data has to be sorted.Why is this necessary?
>>
>
>     Ouch, this used to be a requirement, but no longer.  You can load
>     vertices however you like.  The workers will forward them to the
>     appropriate partition.
>
>
>>     Also, how does the  so called 'master' divide vertices in ranges?
>>     Does it use some kind of a range partitioner? If there is range
>>     partitioning, that's a problem for me, because of vanishing
>>     vertices - there will be load imbalance.
>>
>     Again, out of date documentation.  Please see
>     https://issues.apache.org/jira/browse/GIRAPH-11 for the relevant
>     change.  Let us know if you have any other questions.
>
>>     Best,
>>     Deepak
>


Mime
View raw message