hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amar Kamat (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-1042) rumen should be able to output compressed trace files
Date Thu, 24 Nov 2011 01:38:39 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13156448#comment-13156448

Amar Kamat commented on MAPREDUCE-1042:

Rumen supports compressed output files. If the output filename contains a recognized extension
(e.g. .gzip, .zip etc), Rumen will recognize that and generate a compressed output file.
> rumen should be able to output compressed trace files
> -----------------------------------------------------
>                 Key: MAPREDUCE-1042
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1042
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: tools/rumen
>            Reporter: Dick King
>            Assignee: Dick King
>             Fix For: 0.22.0
> rumen is used primarily to create job trace files which are then processed by other tools.
> These trace files can exceed 100 gigabytes.  However, gzip compression normally achieves
15:1 compression on these traces.
> I would like to modify rumen so it can output compressed files directly, rather than
outputting unwieldy uncompressed files and letting me compress it later.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message