giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (GIRAPH-535) Range-partitioning and edge locality benchmark
Date Tue, 26 Feb 2013 05:16:14 GMT

    [ https://issues.apache.org/jira/browse/GIRAPH-535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13586781#comment-13586781
] 

Hudson commented on GIRAPH-535:
-------------------------------

Integrated in Giraph-trunk-Commit #754 (See [https://builds.apache.org/job/Giraph-trunk-Commit/754/])
    Add missing files from GIRAPH-535 (Revision c33ea10c40d0712352171b8f086dd4fcb70a43c3)

     Result = SUCCESS
alessandro : http://git-wip-us.apache.org/repos/asf?p=giraph.git&a=commit&h=c33ea10c40d0712352171b8f086dd4fcb70a43c3
Files : 
* giraph-core/src/main/java/org/apache/giraph/partition/SimpleRangeWorkerPartitioner.java
* giraph-core/src/main/java/org/apache/giraph/partition/SimpleRangeMasterPartitioner.java
* giraph-core/src/main/java/org/apache/giraph/io/formats/PseudoRandomLocalEdgesHelper.java
* giraph-core/src/main/java/org/apache/giraph/partition/SimpleLongRangePartitionerFactory.java
* giraph-core/src/main/java/org/apache/giraph/io/formats/PseudoRandomInputFormatConstants.java
* giraph-core/src/main/java/org/apache/giraph/partition/SimpleIntRangePartitionerFactory.java

                
> Range-partitioning and edge locality benchmark
> ----------------------------------------------
>
>                 Key: GIRAPH-535
>                 URL: https://issues.apache.org/jira/browse/GIRAPH-535
>             Project: Giraph
>          Issue Type: New Feature
>            Reporter: Alessandro Presta
>            Assignee: Alessandro Presta
>         Attachments: GIRAPH-535.patch
>
>
> Range-based partitioning can drastically reduce network communication when using a vertex
id space where close ids correspond to vertices likely to be connected.
> We currently have an incomplete implementation of range-based partitioning that tries
to be very generic (allowing arbitrarily different partition sizes).
> Talking about this with Avery, we thought that for now it's better to add a simpler version
(which tries to split as evenly as possible) and leave the current classes there in case someone
wants to implement a more complex logic.
> A nice-to-have is also extending the pseudo-random formats to generate a required ratio
of partition-local edges, in order to estimate the impact of locality with benchmarks.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message