hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Runping Qi (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-1194) map output should not do block level compression
Date Tue, 03 Apr 2007 05:06:32 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-1194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12486226
] 

Runping Qi commented on HADOOP-1194:
------------------------------------

The Jira was particularly about the situation where the user sets the map output compression
to be true, and also set 'io.seqfile.compression.type' to be BLOCK (in order to get block
level compression for the reduce outputs). In this case, the map output will also be block-level
compressed, which is not desirble.

So both Owen and Arun's interpretations are correct.



> map output should not do block level compression
> ------------------------------------------------
>
>                 Key: HADOOP-1194
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1194
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.12.2
>            Reporter: Runping Qi
>         Assigned To: Arun C Murthy
>             Fix For: 0.13.0
>
>         Attachments: HADOOP-1194_20070403_1.patch
>
>
> If the user sets to compress the map output, the compression style should be record level,
not block level, since using block level compression for map outputs causes performance degragation
significantly.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message