hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uma Maheswara Rao G (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2115) Transparent compression in HDFS
Date Thu, 30 Jun 2011 04:53:29 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13057615#comment-13057615

Uma Maheswara Rao G commented on HDFS-2115:

Hi Todd,

In our cluster we had implemented the compression support for HDFS (HDFS-1640). 
But here we were not storing the compressed data in DFS. We will decompress and store the
data. Main goal of our compression is to save the network bandwidth. We could achieve ~50-70%
improvements in read and write operations.

bq. Not sure when I'd have time to work on it
We will be happy to coordinate our efforts in implemening this feature.

> Transparent compression in HDFS
> -------------------------------
>                 Key: HDFS-2115
>                 URL: https://issues.apache.org/jira/browse/HDFS-2115
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: data-node, hdfs client
>            Reporter: Todd Lipcon
> In practice, we find that a lot of users store text data in HDFS without using any compression
codec. Improving usability of compressible formats like Avro/RCFile helps with this, but we
could also help many users by providing an option to transparently compress data as it is

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message