hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Allen Wittenauer (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (MAPREDUCE-2616) [Gridmix] InputStriper should smartly switch between compressed and uncompressed files based on the simulated job's input data characteristics
Date Mon, 09 Mar 2015 23:14:39 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-2616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Allen Wittenauer updated MAPREDUCE-2616:
----------------------------------------
    Fix Version/s:     (was: 0.24.0)

> [Gridmix] InputStriper should smartly switch between compressed and uncompressed files
based on the simulated job's input data characteristics
> ----------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-2616
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2616
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: contrib/gridmix
>    Affects Versions: 0.23.0
>            Reporter: Amar Kamat
>            Assignee: Amar Kamat
>            Priority: Minor
>              Labels: compression-emulation, gridmix
>
> Currently, all the Gridmix input data files are located at <gridmix-io-dir>/input
(<gridmix-io-dir> is expected as a CLI parameter). When compression emulation is enabled,
Gridmix will check for compressed files (based on suffixes) in the input folder. Gridmix will
bail out if there are no compressed input files. If the input folder consists of a mix of
compressed and uncompressed input files, then Gridmix will only use compressed input files
for all the jobs. Gridmix should smartly assign 
> 1. uncompressed input files for jobs the don't need input decompression 
> 2. compressed input files for jobs that need input decompression 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message