incubator-hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ashish Agarwal (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HAMA-367) Runtime Compression of BSP Messages to Improve the Performance
Date Tue, 24 May 2011 10:46:47 GMT

    [ https://issues.apache.org/jira/browse/HAMA-367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13038495#comment-13038495
] 

Ashish Agarwal commented on HAMA-367:
-------------------------------------

Hi, I have written some code related to compression and tested against 19 different files
of different types and sizes. 

The best case scenario is a compression rate of 90%. Results (before compression, after compression)
are -
alice29.txt(148.5, 51)
bib (108.7, 33.9)
book1 (750, 299)
book2 (596.5, 197.6)
geo (100, 57.2)
news (368.3, 137.9)
obj1 (21, 8.8)
obj2 (241, 69.3)
paper1 (51.9, 17.8)
paper2 (80.3, 28.5)
paper3 (45.4, 17.3)
paper4 (13, 5.2)
paper5 (11.7, 4.7)
paper6 (37.2, 12.6)
pic (501.2, 42.8)
progc (38.7, 12.8)
progl (70, 15.6)
progp (48.2, 10.7)
trans (91.5, 17.9)


I am asking questions about HAMA-380 on the mailing list.

Thanks
Ashish


> Runtime Compression of BSP Messages to Improve the Performance
> --------------------------------------------------------------
>
>                 Key: HAMA-367
>                 URL: https://issues.apache.org/jira/browse/HAMA-367
>             Project: Hama
>          Issue Type: New Feature
>          Components: bsp, documentation 
>    Affects Versions: 0.3.0
>            Reporter: Edward J. Yoon
>            Assignee: Edward J. Yoon
>              Labels: gsoc, gsoc2011, mentor
>             Fix For: 0.3.0
>
>         Attachments: test_files.tar.gz
>
>   Original Estimate: 2016h
>  Remaining Estimate: 2016h
>
> As you know, the exchanging data between processes, is a core part of whole performance
in Bulk Synchronous Parallel.
> In this research, we investigate BSP message data compression in the context of large-scale
distributed message-passing systems to reduce the communication time of individual messages
and to improve the bandwidth of the overall system. 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message