incubator-crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Wills (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (CRUNCH-36) Avro Compression settings are ignored by Crunch
Date Wed, 08 Aug 2012 04:51:09 GMT

     [ https://issues.apache.org/jira/browse/CRUNCH-36?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Josh Wills resolved CRUNCH-36.
------------------------------

    Resolution: Fixed

Committed--- thanks Joe!
                
> Avro Compression settings are ignored by Crunch
> -----------------------------------------------
>
>                 Key: CRUNCH-36
>                 URL: https://issues.apache.org/jira/browse/CRUNCH-36
>             Project: Crunch
>          Issue Type: Bug
>          Components: IO
>    Affects Versions: 0.3.0
>            Reporter: Joseph Adler
>             Fix For: 0.3.0
>
>         Attachments: CRUNCH-36.patch
>
>
> Java Avro Map/reduce jobs will compress the output if the configuration setting mapred.output.compress
is true. Avro supports two codecs (deflate and snappy). Users typically select the code by
calling AvroJob.setOutputCodec and specifying either org.apache.avro.file.DataFileConstants.SNAPPY_CODEC
or org.apache.avro.file.DataFileConstants.DEFLATE_CODEC. Users can specify the deflate level
by setting the configuration setting avro.mapred.deflate.level.
> org.apache.crunch.types.avro.AvroOutuptFormat currently ignores these setting.
> I'll follow up with a patch that fixes this problem.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message