hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HAMA-870) Runtime message compression in Bundle
Date Thu, 20 Feb 2014 02:15:20 GMT

    [ https://issues.apache.org/jira/browse/HAMA-870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13906491#comment-13906491
] 

Hudson commented on HAMA-870:
-----------------------------

FAILURE: Integrated in Hama-trunk #277 (See [https://builds.apache.org/job/Hama-trunk/277/])
HAMA-870: Runtime message compression in Bundle (edwardyoon) (edwardyoon: rev 1570039)
* /hama/trunk/CHANGES.txt
* /hama/trunk/conf/hama-default.xml
* /hama/trunk/core/src/main/java/org/apache/hama/bsp/BSPMessageBundle.java
* /hama/trunk/core/src/main/java/org/apache/hama/bsp/LocalBSPRunner.java
* /hama/trunk/core/src/main/java/org/apache/hama/bsp/message/AbstractMessageManager.java
* /hama/trunk/core/src/main/java/org/apache/hama/bsp/message/CompressableMessageManager.java
* /hama/trunk/core/src/main/java/org/apache/hama/bsp/message/HamaMessageManager.java
* /hama/trunk/core/src/main/java/org/apache/hama/bsp/message/HamaMessageManagerImpl.java
* /hama/trunk/core/src/main/java/org/apache/hama/bsp/message/MessageManager.java
* /hama/trunk/core/src/main/java/org/apache/hama/bsp/message/OutgoingMessageManager.java
* /hama/trunk/core/src/main/java/org/apache/hama/bsp/message/OutgoingPOJOMessageBundle.java
* /hama/trunk/core/src/main/java/org/apache/hama/bsp/message/compress/BSPMessageCompressor.java
* /hama/trunk/core/src/main/java/org/apache/hama/bsp/message/compress/Bzip2Compressor.java
* /hama/trunk/core/src/main/java/org/apache/hama/bsp/message/compress/SnappyCompressor.java
* /hama/trunk/core/src/test/java/org/apache/hama/bsp/TestCheckpoint.java
* /hama/trunk/core/src/test/java/org/apache/hama/bsp/message/compress/TestBSPMessageCompressor.java
* /hama/trunk/examples/src/main/java/org/apache/hama/examples/PiEstimator.java
* /hama/trunk/examples/src/main/java/org/apache/hama/examples/SemiClusterJobDriver.java
* /hama/trunk/examples/src/test/java/org/apache/hama/examples/SemiClusterMatchingTest.java
* /hama/trunk/graph/src/main/java/org/apache/hama/graph/OutgoingVertexMessagesManager.java
* /hama/trunk/graph/src/test/java/org/apache/hama/graph/TestSubmitGraphJob.java


> Runtime message compression in Bundle
> -------------------------------------
>
>                 Key: HAMA-870
>                 URL: https://issues.apache.org/jira/browse/HAMA-870
>             Project: Hama
>          Issue Type: Improvement
>          Components: bsp core
>    Affects Versions: 0.6.3
>            Reporter: Edward J. Yoon
>            Assignee: Edward J. Yoon
>             Fix For: 0.7.0
>
>         Attachments: runtimecompression.patch
>
>
> Original compression flow was like below:
> 1) Bundles messages in outgoing queue
> 2) Serializes bundle object to byte array
> 3) Compresses byte array to new byte array
> 4) Creates BSPCompressedBundle object for RPC communication
> So, it was extremely slow!
> Since BSPMessageBundle handles messages in byte array now, we can runtime compress the
messages during bundling. 
> We don't need BSPCompressedBundle class anymore.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message