hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HAMA-794) When aggregator is enabled, resending message in each message receiving round impacts the iteration performance.
Date Wed, 14 Aug 2013 00:57:48 GMT

    [ https://issues.apache.org/jira/browse/HAMA-794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13739100#comment-13739100
] 

Edward J. Yoon commented on HAMA-794:
-------------------------------------

This is related with HAMA-733 and HAMA-734

Current sync implementation deletes all unconsumed messages in the queues. I'm sure this will
makes overheads (like this case) sometimes.
                
> When aggregator is enabled, resending message in each message receiving round impacts
the iteration performance.
> ----------------------------------------------------------------------------------------------------------------
>
>                 Key: HAMA-794
>                 URL: https://issues.apache.org/jira/browse/HAMA-794
>             Project: Hama
>          Issue Type: Improvement
>          Components: graph
>            Reporter: MaoYuan Xian
>
> When aggregator is enabled, in each message receiving round, all received message have
been resent to put them behind the aggregate message. doAggregationUpdates method of GraphJobRunner
Class:
> {code}
>     if (aggregationRunner.isEnabled() && iteration > 1) {
>       // in case we need to sync, we need to replay the messages that already
>       // are added to the queue. This prevents loosing messages when using
>       // aggregators.
>       if (firstVertexMessage != null) {
>         peer.send(peer.getPeerName(), firstVertexMessage);
>       }
>       GraphJobMessage msg = null;
>       while ((msg = peer.getCurrentMessage()) != null) {
>         peer.send(peer.getPeerName(), msg);
>       }
>       // now sync
>       peer.sync();
>       ...
> {code}
> Should we do some improvement here? Record the original receiveQueue before the sync,
and after sync operation, read the aggregator updated value firstly, then restore the original
receiveQueue such that avoiding the message resending operations.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message