Return-Path: X-Original-To: apmail-giraph-dev-archive@www.apache.org Delivered-To: apmail-giraph-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 5ED74E58E for ; Fri, 4 Jan 2013 15:44:14 +0000 (UTC) Received: (qmail 92349 invoked by uid 500); 4 Jan 2013 15:44:14 -0000 Delivered-To: apmail-giraph-dev-archive@giraph.apache.org Received: (qmail 91622 invoked by uid 500); 4 Jan 2013 15:44:13 -0000 Mailing-List: contact dev-help@giraph.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@giraph.apache.org Delivered-To: mailing list dev@giraph.apache.org Received: (qmail 91570 invoked by uid 500); 4 Jan 2013 15:44:13 -0000 Delivered-To: apmail-incubator-giraph-dev@incubator.apache.org Received: (qmail 91562 invoked by uid 99); 4 Jan 2013 15:44:12 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 04 Jan 2013 15:44:12 +0000 Date: Fri, 4 Jan 2013 15:44:12 +0000 (UTC) From: "Claudio Martella (JIRA)" To: giraph-dev@incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (GIRAPH-459) Group Vertex Mutations by Partition ID MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/GIRAPH-459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13543959#comment-13543959 ] Claudio Martella commented on GIRAPH-459: ----------------------------------------- Running PageRankBenchmark with 3 workers on my laptop, 100k vertices, 100 edges each, HashMapVertex and PseudoRandomEdgeInputFormat. trunk: Superstep 3 (milliseconds) 6,025 0 6,025 Superstep 4 (milliseconds) 5,605 0 5,605 Superstep 20 (milliseconds) 124 0 124 Setup (milliseconds) 6,025 0 6,025 Shutdown (milliseconds) 154 0 154 Superstep 7 (milliseconds) 5,691 0 5,691 Superstep 16 (milliseconds) 5,605 0 5,605 Superstep 9 (milliseconds) 5,571 0 5,571 Superstep 8 (milliseconds) 10,527 0 10,527 Superstep 14 (milliseconds) 10,538 0 10,538 Input superstep (milliseconds) 15,764 0 15,764 Superstep 6 (milliseconds) 11,103 0 11,103 Superstep 18 (milliseconds) 5,623 0 5,623 Superstep 5 (milliseconds) 10,666 0 10,666 Superstep 1 (milliseconds) 14,376 0 14,376 Total (milliseconds) 186,270 0 186,270 Superstep 15 (milliseconds) 5,609 0 5,609 Superstep 17 (milliseconds) 5,825 0 5,825 Superstep 12 (milliseconds) 5,796 0 5,796 Superstep 10 (milliseconds) 6,037 0 6,037 Superstep 19 (milliseconds) 10,729 0 10,729 Superstep 11 (milliseconds) 10,875 0 10,875 Superstep 0 (milliseconds) 11,098 0 11,098 Superstep 2 (milliseconds) 11,221 0 11,221 Superstep 13 (milliseconds) 5,673 0 5,673 GIRAPH-459: Superstep 3 (milliseconds) 5,845 0 5,845 Superstep 4 (milliseconds) 5,912 0 5,912 Superstep 20 (milliseconds) 145 0 145 Setup (milliseconds) 6,165 0 6,165 Shutdown (milliseconds) 65 0 65 Superstep 7 (milliseconds) 10,625 0 10,625 Superstep 16 (milliseconds) 6,031 0 6,031 Superstep 9 (milliseconds) 5,951 0 5,951 Superstep 8 (milliseconds) 5,626 0 5,626 Superstep 14 (milliseconds) 10,653 0 10,653 Input superstep (milliseconds) 11,213 0 11,213 Superstep 6 (milliseconds) 5,876 0 5,876 Superstep 18 (milliseconds) 10,719 0 10,719 Superstep 5 (milliseconds) 11,108 0 11,108 Superstep 1 (milliseconds) 14,978 0 14,978 Total (milliseconds) 186,255 0 186,255 Superstep 15 (milliseconds) 10,663 0 10,663 Superstep 17 (milliseconds) 10,914 0 10,914 Superstep 12 (milliseconds) 5,636 0 5,636 Superstep 10 (milliseconds) 6,013 0 6,013 Superstep 19 (milliseconds) 11,421 0 11,421 Superstep 11 (milliseconds) 5,559 0 5,559 Superstep 0 (milliseconds) 12,874 0 12,874 Superstep 2 (milliseconds) 6,656 0 6,656 Superstep 13 (milliseconds) 5,602 0 5,602 Looks good. I fixed the rest of the issues. Waiting for a comment on (1). > Group Vertex Mutations by Partition ID > -------------------------------------- > > Key: GIRAPH-459 > URL: https://issues.apache.org/jira/browse/GIRAPH-459 > Project: Giraph > Issue Type: Improvement > Components: graph > Reporter: Claudio Martella > Attachments: GIRAPH-459.patch > > > Currently, vertex mutations, and implicit creations of vertices based on messages to non-existing vertices, are executed randomly partition-wise. The iterated vertices can belong to different partitions. This is bad when we work out-of-core, as we need to load and unload the whole partition for each vertex. We should group these operations per-partition, and batch-execute them. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira