hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amar Kamat (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-2408) Make Gridmix emulate usage of data compression
Date Sun, 29 May 2011 02:49:47 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13040726#comment-13040726
] 

Amar Kamat commented on MAPREDUCE-2408:
---------------------------------------

Hong,
Thanks a lot for your review. You are right. The compression ratios table will be different
for different codecs. The empirical values table in this patch is computed for the default
codec (i.e Gzip). We have compiled similar table for LZO and it seems LZO too shows some pattern
in that respect. The plan is to add other codecs incrementally. I will open a JIRA to track
LZO compression emulation.

> Make Gridmix emulate usage of data compression
> ----------------------------------------------
>
>                 Key: MAPREDUCE-2408
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2408
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>          Components: contrib/gridmix
>            Reporter: Ravi Gummadi
>            Assignee: Amar Kamat
>             Fix For: 0.23.0
>
>         Attachments: MR-2408-gridmix-compression-emulation-v1.1.patch
>
>
> Currently Gridmix emulates disk IO load only. This JIRA is to make Gridmix emulate load
due to data compression as defined by the job-trace.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message