hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suraj Menon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HAMA-707) BSPMessageBundle should be able to encapsulate messages serialized in ByteBuffer
Date Tue, 26 Mar 2013 00:19:15 GMT

    [ https://issues.apache.org/jira/browse/HAMA-707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13613334#comment-13613334
] 

Suraj Menon commented on HAMA-707:
----------------------------------

Good point. As an example, SpillingQueue is a byte array message handler after write. I am
working on SpilledDataProcessor to write series of BSPMessageBundle. Each message bundle would
give you its length. Regardless of the communication(sync or async), in one send, a series
of BSPMessageBundle would be sent. The size of this (Series of) byte array would be known
in advance. This is where we can decide whether to compress or not. For netty based solution,
netty itself handles the compression and decompression for us.
                
> BSPMessageBundle should be able to encapsulate messages serialized in ByteBuffer
> --------------------------------------------------------------------------------
>
>                 Key: HAMA-707
>                 URL: https://issues.apache.org/jira/browse/HAMA-707
>             Project: Hama
>          Issue Type: Improvement
>            Reporter: Suraj Menon
>            Assignee: Suraj Menon
>             Fix For: 0.6.1
>
>         Attachments: HAMA-707.patch
>
>
> BSPMessageBundle reads out all the messages from the queue and creates an in memory list
before sending it to peer. This does not work when all messages could not be kept in memory.
In case of spilling queue, where the messages are already serialized, avoid complete read
out of messages.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message