hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon" <edwardy...@apache.org>
Subject [DISCUSS] Disk Queue and Spilling Queue
Date Mon, 12 May 2014 01:01:19 GMT
The old design of outgoing/incoming message queues is readable but it
has some problems, and the most performance and memory issues are
dependent upon this part.

1) To send a messages to destination Peer, we serialize, compress, and
bundle the messages. So, using disk or spilling queue for the outgoing
messages is pointless and cause of degradation. This issue SOLVED by
HAMA-853. We'll need to add disk-based bundle in the future.

2) Receive-side queue is also the same. Instead of unbundling (and
deserializing, decompressing) bundles into {memory, disk, or spilling}
queue, we should use bundles in efficient and asynchronous way.

If you agree with this, I'll start to refactor the whole queue system.

If you have any other ideas e.g., asynchronous message
synchronization, Pls let me know.


Best Regards, Edward J. Yoon
CEO at DataSayer Co., Ltd.

View raw message