hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apurv Verma (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HAMA-490) Support asynchronous messaging
Date Sat, 22 Sep 2012 15:44:07 GMT

    [ https://issues.apache.org/jira/browse/HAMA-490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13461199#comment-13461199
] 

Apurv Verma commented on HAMA-490:
----------------------------------

I am gonna place an AsyncMessageService inside AbstractMessageManager, this will maintain
a fixed pool of threads for sending. For now since we don't have a spilling queue implementation,
I am going to create an extra C.H.M. 
This thread pool will poll from the CHM and send the messages in small batches.

So for any new message to be sent,

Case 1: IT goes in the CHM if the CHM is not heavily populated. From where it will be picked
up by the executor service.

Case 2: If the CHM was already populated, we write the message where the message was being
written earlier. i.e in the DiskQueue / MemoryQueue / future queue implementations

At the end just before leaving the barrier, we shutdown this async service to ensure all pending
tasks have been completed.

In future when a Spilling Disk Queue would be available, then we can do away with the CHM
completely ;)

Any problems in my design, otherwise I am gonna code it up.

                
> Support asynchronous messaging
> ------------------------------
>
>                 Key: HAMA-490
>                 URL: https://issues.apache.org/jira/browse/HAMA-490
>             Project: Hama
>          Issue Type: New Feature
>            Reporter: Thomas Jungblut
>
> We should support asynchronous messaging.
> A new messaging service should send messages during computation time in another thread
to save the time during the barrier sync.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message