Mailing-List: contact dev-help@spark.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@spark.apache.org
Received-SPF: pass (athena.apache.org: domain of xiaodi@sjtu.edu.cn designates
 202.112.26.52 as permitted sender)
Message-ID: <53D0AECE.2070608@sjtu.edu.cn>
Date: Thu, 24 Jul 2014 14:59:26 +0800
From: Larry Xiao <xiaodi@sjtu.edu.cn>
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.9;
 rv:24.0) Gecko/20100101 Thunderbird/24.6.0
MIME-Version: 1.0
To: dev@spark.apache.org
Subject: GraphX graph partitioning strategy
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

Hi all,

I'm implementing graph partitioning strategy for GraphX, learning from 
researches on graph computing.

I have two questions:

- a specific implement question:
In current design, only vertex ID of src and dst are provided 
(PartitionStrategy.scala).
And some strategies require knowledge about the graph (like degrees) and 
can consist more than one passes to finally produce the partition ID.
So I'm changing the PartitionStrategy.getPartition API to provide more 
info, but I don't want to make it complex. (the current one looks very 
clean)

- an open question:
What advice would you give considering partitioning, considering the 
procedure Spark adopt on graph processing?

Any advice is much appreciated.

Best Regards,
Larry Xiao

Reference

Bipartite-oriented Distributed Graph Partitioning for Big Learning.
PowerLyra : Differentiated Graph Computation and Partitioning on Skewed 
Graphs