incubator-hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Miklos Erdelyi (JIRA)" <>
Subject [jira] [Commented] (HAMA-380) Send messages in batches to reduce RPC overhead.
Date Tue, 19 Apr 2011 11:40:05 GMT


Miklos Erdelyi commented on HAMA-380:

* You're right that caching constructors minimizes the time needed to instantiate classes.
Thanks for adding this!
* However, in the BSP model there is no guarantee to the order of received messages so you
cannot make assumptions like 'receive 1 Foo class then receive 3 Bar classes'. That is why
I originally didn't deal with the situation where mixed classes might be added to a BSPMessageBundle.
* From a design point of view, you can always embed different message classes into one container
message class so the simpler solution (i.e., to support buffering instances of only one class
in a bundle) still works.

> Send messages in batches to reduce RPC overhead.
> ------------------------------------------------
>                 Key: HAMA-380
>                 URL:
>             Project: Hama
>          Issue Type: Improvement
>          Components: bsp
>    Affects Versions: 0.3.0
>            Reporter: Miklos Erdelyi
>            Assignee: Miklos Erdelyi
>             Fix For: 0.3.0
>         Attachments: HAMA-380-v2.patch, bspmsgbundle.patch, bspmsgbundle.v2.patch
>   Original Estimate: 4h
>  Remaining Estimate: 4h
> When profiling graph computations running on HAMA it turned out that huge amount of time
is spent in RPC calls.
> To reduce this overhead I propose extending the GroomServer's API with a put(BSPMessageBundle)
method so that a group of messages can be transferred from the caller to the callee at once.
Also, the sync() operation should be improved by sending messages in batches rather than one-by-one.
> Using BSPMessageBundle makes it easier to add support for compression of messages later.

This message is automatically generated by JIRA.
For more information on JIRA, see:

View raw message