hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Harsh J Chouraria (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-1640) Transmission of large files in compressed format will save the network traffic and can improve the data transfer time.
Date Thu, 24 Feb 2011 11:01:38 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12998803#comment-12998803
] 

Harsh J Chouraria commented on HDFS-1640:
-----------------------------------------

Won't this be slightly expensive and thereby useless for blocks that already hold compressed
record(s) data? Compressing compressed data doesn't really help AFAIK. Please correct me if
I'm wrong.

> Transmission of large files in compressed format will save the network traffic and can
improve the data transfer time.
> ----------------------------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-1640
>                 URL: https://issues.apache.org/jira/browse/HDFS-1640
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: data-node, hdfs client
>            Reporter: Uma Maheswara Rao G
>
> *In Write scenario:*
> DFSClient can Compress the data when transmitting it over the network, Data Nodes can
forward the same compressed data to other Data Nodes in pipeline and as well as it can decompress
that data and write on to their local disks. 
> *In Read Scenario:*
>  Data Node can compress the data when transmitting it over the network. DFSClient can
decompress it and write on to the local stream.

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message