Return-Path: X-Original-To: apmail-incubator-giraph-user-archive@minotaur.apache.org Delivered-To: apmail-incubator-giraph-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 5B6D6BE92 for ; Sun, 22 Jan 2012 22:31:00 +0000 (UTC) Received: (qmail 68307 invoked by uid 500); 22 Jan 2012 22:31:00 -0000 Delivered-To: apmail-incubator-giraph-user-archive@incubator.apache.org Received: (qmail 68218 invoked by uid 500); 22 Jan 2012 22:30:59 -0000 Mailing-List: contact giraph-user-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: giraph-user@incubator.apache.org Delivered-To: mailing list giraph-user@incubator.apache.org Received: (qmail 68201 invoked by uid 99); 22 Jan 2012 22:30:59 -0000 Received: from minotaur.apache.org (HELO minotaur.apache.org) (140.211.11.9) by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 22 Jan 2012 22:30:59 +0000 Received: from localhost (HELO achingmbp15.local) (127.0.0.1) (smtp-auth username aching, mechanism plain) by minotaur.apache.org (qpsmtpd/0.29) with ESMTP; Sun, 22 Jan 2012 22:30:58 +0000 Message-ID: <4F1C8E22.8050304@apache.org> Date: Sun, 22 Jan 2012 14:30:58 -0800 From: Avery Ching User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:9.0) Gecko/20111222 Thunderbird/9.0.1 MIME-Version: 1.0 To: giraph-user@incubator.apache.org Subject: Re: Changing Graph Size References: In-Reply-To: Content-Type: multipart/alternative; boundary="------------060605040209090505010402" This is a multi-part message in MIME format. --------------060605040209090505010402 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Hi Deepak, Answers inline. Happy sunday! Avery On 1/22/12 11:30 AM, Deepak Nettem wrote: > Hi Folks, > > I have a Graph processing problem where after each iteration, some > vertices get vanished. That is, they get merged into their > neighbouring nodes based on certain conditions and the graph keeps > getting simplified after each iteration. > > I was wondering if Giraph is worth trying for this. > Giraph can support this graph mutation at any iteration in the graph. See https://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/graph/MutableVertex.java, the method r emoveVertexRequest() will remove vertices for you between iterations. > I was going through the documentation and it says that the input data > has to be sorted.Why is this necessary? > Ouch, this used to be a requirement, but no longer. You can load vertices however you like. The workers will forward them to the appropriate partition. > Also, how does the so called 'master' divide vertices in ranges? Does > it use some kind of a range partitioner? If there is range > partitioning, that's a problem for me, because of vanishing vertices - > there will be load imbalance. > Again, out of date documentation. Please see https://issues.apache.org/jira/browse/GIRAPH-11 for the relevant change. Let us know if you have any other questions. > Best, > Deepak --------------060605040209090505010402 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Hi Deepak,

Answers inline.

Happy sunday!

Avery

On 1/22/12 11:30 AM, Deepak Nettem wrote:

Hi Folks,

I have a Graph processing problem where after each iteration, some vertices get vanished. That is, they get merged into their neighbouring nodes based on certain conditions and the graph keeps getting simplified after each iteration.

I was wondering if Giraph is worth trying for this.

Giraph can support this graph mutation at any iteration in the graph. See https://svn.apache.org/repos/asf/incubator/giraph/trunk/src/main/java/org/apache/giraph/graph/MutableVertex.java, the method r emoveVertexRequest() will remove vertices for you between iterations.

I was going through the documentation and it says that the input data has to be sorted.Why is this necessary?

Ouch, this used to be a requirement, but no longer. You can load vertices however you like. The workers will forward them to the appropriate partition.

Also, how does the so called 'master' divide vertices in ranges? Does it use some kind of a range partitioner? If there is range partitioning, that's a problem for me, because of vanishing vertices - there will be load imbalance.

Again, out of date documentation. Please see https://issues.apache.org/jira/browse/GIRAPH-11 for the relevant change. Let us know if you have any other questions.

Best,
Deepak

--------------060605040209090505010402--