giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Maja Kabiljo (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (GIRAPH-388) Improve the way we keep outgoing messages
Date Mon, 29 Oct 2012 21:14:14 GMT

    [ https://issues.apache.org/jira/browse/GIRAPH-388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13486367#comment-13486367
] 

Maja Kabiljo commented on GIRAPH-388:
-------------------------------------

My mistake, I meant GIRAPH-322, not 314... The changes I have in mind don't try to improve
the case when we have the same message sent to many vertices, as you are trying to do there
(but they don't hurt it either). That's something we might want to have as an option. Though
in a real social graph application, and running on a large number of workers, the question
is how big is the chance that several destination vertices end up on the same worker. 

I did test this with one of our internal applications also, and the speedup was similar to
the one reported. I agree that benchmarks are not good enough to test all improvements, but
in some cases it's a good guideline. 
                
> Improve the way we keep outgoing messages
> -----------------------------------------
>
>                 Key: GIRAPH-388
>                 URL: https://issues.apache.org/jira/browse/GIRAPH-388
>             Project: Giraph
>          Issue Type: Improvement
>            Reporter: Maja Kabiljo
>            Assignee: Maja Kabiljo
>         Attachments: GIRAPH-388.patch
>
>
> As per discussion on GIRAPH-357, in standard application chances that we get to use client-side
combiner are very low. I experimented with benefits which we can get from not having the client-side
combiner at all. It turns out that having a lot of maps in SendMessageCache, and then collection
inside each of them, really hurts the performance. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message