incubator-hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Miklos Erdelyi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HAMA-380) Send messages in batches to reduce RPC overhead.
Date Tue, 19 Apr 2011 11:40:05 GMT

    [ https://issues.apache.org/jira/browse/HAMA-380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13021537#comment-13021537
] 

Miklos Erdelyi commented on HAMA-380:
-------------------------------------

* You're right that caching constructors minimizes the time needed to instantiate classes.
Thanks for adding this!
* However, in the BSP model there is no guarantee to the order of received messages so you
cannot make assumptions like 'receive 1 Foo class then receive 3 Bar classes'. That is why
I originally didn't deal with the situation where mixed classes might be added to a BSPMessageBundle.
* From a design point of view, you can always embed different message classes into one container
message class so the simpler solution (i.e., to support buffering instances of only one class
in a bundle) still works.

> Send messages in batches to reduce RPC overhead.
> ------------------------------------------------
>
>                 Key: HAMA-380
>                 URL: https://issues.apache.org/jira/browse/HAMA-380
>             Project: Hama
>          Issue Type: Improvement
>          Components: bsp
>    Affects Versions: 0.3.0
>            Reporter: Miklos Erdelyi
>            Assignee: Miklos Erdelyi
>             Fix For: 0.3.0
>
>         Attachments: HAMA-380-v2.patch, bspmsgbundle.patch, bspmsgbundle.v2.patch
>
>   Original Estimate: 4h
>  Remaining Estimate: 4h
>
> When profiling graph computations running on HAMA it turned out that huge amount of time
is spent in RPC calls.
> To reduce this overhead I propose extending the GroomServer's API with a put(BSPMessageBundle)
method so that a group of messages can be transferred from the caller to the callee at once.
Also, the sync() operation should be improved by sending messages in batches rather than one-by-one.
> Using BSPMessageBundle makes it easier to add support for compression of messages later.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message