hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hong Tang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-2408) Make Gridmix emulate usage of data compression
Date Sat, 28 May 2011 18:24:47 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-2408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13040652#comment-13040652
] 

Hong Tang commented on MAPREDUCE-2408:
--------------------------------------

Looks like I missed it before it gets committed. I quickly went through the patch. I like
the approach of using a dictionary and empirically match the compression ratio with the dictionary
size. However, I believe the compression ratio would be different under different compression
codecs (even same codec under different levels). It'd be useful if you could extend CompressionRatioLookupTable
so that it takes as input a compression codec (and you may only support the most common few
codecs lzo, gzip, and bzip2).

> Make Gridmix emulate usage of data compression
> ----------------------------------------------
>
>                 Key: MAPREDUCE-2408
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2408
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>          Components: contrib/gridmix
>            Reporter: Ravi Gummadi
>            Assignee: Amar Kamat
>             Fix For: 0.23.0
>
>         Attachments: MR-2408-gridmix-compression-emulation-v1.1.patch
>
>
> Currently Gridmix emulates disk IO load only. This JIRA is to make Gridmix emulate load
due to data compression as defined by the job-trace.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message