Return-Path: X-Original-To: apmail-incubator-giraph-user-archive@minotaur.apache.org Delivered-To: apmail-incubator-giraph-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 2A03588F4 for ; Thu, 8 Sep 2011 01:09:57 +0000 (UTC) Received: (qmail 70634 invoked by uid 500); 8 Sep 2011 01:09:57 -0000 Delivered-To: apmail-incubator-giraph-user-archive@incubator.apache.org Received: (qmail 70595 invoked by uid 500); 8 Sep 2011 01:09:56 -0000 Mailing-List: contact giraph-user-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: giraph-user@incubator.apache.org Delivered-To: mailing list giraph-user@incubator.apache.org Received: (qmail 70586 invoked by uid 99); 8 Sep 2011 01:09:56 -0000 Received: from minotaur.apache.org (HELO minotaur.apache.org) (140.211.11.9) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 08 Sep 2011 01:09:56 +0000 Received: from localhost (HELO [0.0.0.0]) (127.0.0.1) (smtp-auth username aching, mechanism plain) by minotaur.apache.org (qpsmtpd/0.29) with ESMTP; Thu, 08 Sep 2011 01:09:55 +0000 Message-ID: <4E6815E3.1050701@apache.org> Date: Wed, 07 Sep 2011 18:09:55 -0700 From: Avery Ching User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:6.0.2) Gecko/20110902 Thunderbird/6.0.2 MIME-Version: 1.0 To: giraph-user@incubator.apache.org Subject: Re: Primitives vs Objects (the Movie!) References: <4E66C2C2.1010808@apache.org> <4E671030.9030103@apache.org> <4E67E19B.1070204@apache.org> <4E67ECDC.8060802@apache.org> <4E67F143.5030501@apache.org> In-Reply-To: Content-Type: multipart/alternative; boundary="------------080003020509020708020400" This is a multi-part message in MIME format. --------------080003020509020708020400 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit On 9/7/11 3:49 PM, Jake Mannix wrote: > On Wed, Sep 7, 2011 at 10:43 PM, Dmitriy Ryaboy > wrote: > > Something else to think about -- in some cases you may want to > implement extreme compression of the edge list. For example, we might > want to calculate n-th degree reach for all nodes in a graph. Given a > graph with a few nodes with lots of incoming edges ("biebernodes", we > call them), anything that jumps across them winds up essentially > replicating large chunks of the graph to each node. Things like RLE > might make sense when you are doing that sort of thing in memory and > only have so many terabytes of that lying around. > > > Right, in some cases, we may want to not have edge values at all. Just > existence in the vertex's target list of a target vertex id denotes > "connection", > and so "SortedMap>" gets replaced with "long[]". > -jake In this type of case, you could also use the vertex value to store a application specific set of edges and edge values (if any). --------------080003020509020708020400 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit On 9/7/11 3:49 PM, Jake Mannix wrote:
On Wed, Sep 7, 2011 at 10:43 PM, Dmitriy Ryaboy <dmitriy@twitter.com> wrote:
Something else to think about -- in some cases you may want to
implement extreme compression of the edge list. For example, we might
want to calculate n-th degree reach for all nodes in a graph. Given a
graph with a few nodes with lots of incoming edges ("biebernodes", we
call them), anything that jumps across them winds up essentially
replicating large chunks of the graph to each node. Things like RLE
might make sense when you are doing that sort of thing in memory and
only have so many terabytes of that lying around.

Right, in some cases, we may want to not have edge values at all.  Just 
existence in the vertex's target list of a target vertex id denotes "connection",
and so "SortedMap<I, Edge<E,I>>" gets replaced with "long[]".
 
  -jake
In this type of case, you could also use the vertex value to store a application specific set of edges and edge values (if any).
--------------080003020509020708020400--