hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apurv Verma (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HAMA-490) Support asynchronous messaging
Date Sat, 22 Sep 2012 15:44:07 GMT

    [ https://issues.apache.org/jira/browse/HAMA-490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13461199#comment-13461199

Apurv Verma commented on HAMA-490:

I am gonna place an AsyncMessageService inside AbstractMessageManager, this will maintain
a fixed pool of threads for sending. For now since we don't have a spilling queue implementation,
I am going to create an extra C.H.M. 
This thread pool will poll from the CHM and send the messages in small batches.

So for any new message to be sent,

Case 1: IT goes in the CHM if the CHM is not heavily populated. From where it will be picked
up by the executor service.

Case 2: If the CHM was already populated, we write the message where the message was being
written earlier. i.e in the DiskQueue / MemoryQueue / future queue implementations

At the end just before leaving the barrier, we shutdown this async service to ensure all pending
tasks have been completed.

In future when a Spilling Disk Queue would be available, then we can do away with the CHM
completely ;)

Any problems in my design, otherwise I am gonna code it up.

> Support asynchronous messaging
> ------------------------------
>                 Key: HAMA-490
>                 URL: https://issues.apache.org/jira/browse/HAMA-490
>             Project: Hama
>          Issue Type: New Feature
>            Reporter: Thomas Jungblut
> We should support asynchronous messaging.
> A new messaging service should send messages during computation time in another thread
to save the time during the barrier sync.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message