Return-Path: X-Original-To: apmail-giraph-dev-archive@www.apache.org Delivered-To: apmail-giraph-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id BCBDA111C8 for ; Tue, 6 May 2014 21:44:56 +0000 (UTC) Received: (qmail 79294 invoked by uid 500); 6 May 2014 21:43:24 -0000 Delivered-To: apmail-giraph-dev-archive@giraph.apache.org Received: (qmail 79054 invoked by uid 500); 6 May 2014 21:43:09 -0000 Mailing-List: contact dev-help@giraph.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@giraph.apache.org Delivered-To: mailing list dev@giraph.apache.org Received: (qmail 78282 invoked by uid 500); 6 May 2014 21:42:36 -0000 Delivered-To: apmail-incubator-giraph-dev@incubator.apache.org Received: (qmail 77728 invoked by uid 99); 6 May 2014 21:42:16 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 06 May 2014 21:42:16 +0000 Date: Tue, 6 May 2014 21:42:16 +0000 (UTC) From: "Claudio Martella (JIRA)" To: giraph-dev@incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (GIRAPH-895) Trim the edges in Giraph MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/GIRAPH-895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13991173#comment-13991173 ] Claudio Martella commented on GIRAPH-895: ----------------------------------------- this is an interesting memory-footprint improvement. I wonder though whether you trim the vertices in the right place, namely in the computecallable after a vertex is executed. this does indeed manage when the vertex is adding edges (to itself), but it does not manage when an edge is added to a vertex from another vertex via addEdgeRequest, at the end of the superstep. Shouldn't then we just do the trimming at the end of the superstep, and that's it? > Trim the edges in Giraph > ------------------------ > > Key: GIRAPH-895 > URL: https://issues.apache.org/jira/browse/GIRAPH-895 > Project: Giraph > Issue Type: Improvement > Components: graph > Affects Versions: 1.1.0 > Reporter: Sergey Edunov > Fix For: 1.1.0 > > Attachments: GIRAPH-895.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > In many Giraph applications, graphs are immutable, but edges are never trimmed to the proper size, after input phase. This means that on average we often use 1.5x memory for storing them. Considering we are often memory bounded, adding an option to trim the edges after the input phase will help reduce this excess memory usage. For mutable graphs, we can also provide an option for the same method to be called after each superstep. > Review request: https://reviews.apache.org/r/21119/ -- This message was sent by Atlassian JIRA (v6.2#6252)