Mailing-List: contact crunch-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: crunch-dev@incubator.apache.org
Date: Wed, 8 Aug 2012 00:23:09 +0000 (UTC)
From: "Josh Wills (JIRA)" <jira@apache.org>
To: crunch-dev@incubator.apache.org
Message-ID: <173075165.2209.1344385390398.JavaMail.jiratomcat@issues-vm>
In-Reply-To: <1300120114.1848.1344379930403.JavaMail.jiratomcat@issues-vm>
Subject: [jira] [Commented] (CRUNCH-36) Avro Compression settings are
 ignored by Crunch
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/CRUNCH-36?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13430765#comment-13430765 ] 

Josh Wills commented on CRUNCH-36:
----------------------------------

Patch formats fine. Is there not mapreduce.* API support for this? I would assume not, since this is Avro, but I thought I would check.
                
> Avro Compression settings are ignored by Crunch
> -----------------------------------------------
>
>                 Key: CRUNCH-36
>                 URL: https://issues.apache.org/jira/browse/CRUNCH-36
>             Project: Crunch
>          Issue Type: Bug
>          Components: IO
>    Affects Versions: 0.3.0
>            Reporter: Joseph Adler
>             Fix For: 0.3.0
>
>         Attachments: CRUNCH-36.patch
>
>
> Java Avro Map/reduce jobs will compress the output if the configuration setting mapred.output.compress is true. Avro supports two codecs (deflate and snappy). Users typically select the code by calling AvroJob.setOutputCodec and specifying either org.apache.avro.file.DataFileConstants.SNAPPY_CODEC or org.apache.avro.file.DataFileConstants.DEFLATE_CODEC. Users can specify the deflate level by setting the configuration setting avro.mapred.deflate.level.
> org.apache.crunch.types.avro.AvroOutuptFormat currently ignores these setting.
> I'll follow up with a patch that fixes this problem.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira