giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Claudio Martella (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (GIRAPH-388) Improve the way we keep outgoing messages
Date Tue, 30 Oct 2012 09:08:15 GMT

    [ https://issues.apache.org/jira/browse/GIRAPH-388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13486756#comment-13486756
] 

Claudio Martella commented on GIRAPH-388:
-----------------------------------------

To be honest I doubt that the "flushing tunings" can have a big impact on social graphs and
in particular on hubs (supernodes). Sure, you can always fine tune, but that's definitely
a different problem, namely load balancing. The only way to actually attack the problem is
at a more fundamental problem, such as by splitting the hubs/neighborhoods across multiple
partitions, like GPS, or maybe trying to balance partitions wrt to edges and not vertices.
                
> Improve the way we keep outgoing messages
> -----------------------------------------
>
>                 Key: GIRAPH-388
>                 URL: https://issues.apache.org/jira/browse/GIRAPH-388
>             Project: Giraph
>          Issue Type: Improvement
>            Reporter: Maja Kabiljo
>            Assignee: Maja Kabiljo
>         Attachments: GIRAPH-388.patch
>
>
> As per discussion on GIRAPH-357, in standard application chances that we get to use client-side
combiner are very low. I experimented with benefits which we can get from not having the client-side
combiner at all. It turns out that having a lot of maps in SendMessageCache, and then collection
inside each of them, really hurts the performance. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message