avro-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ted Malaska (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (AVRO-1243) Avro support for all compression codecs
Date Fri, 08 Feb 2013 16:51:13 GMT

     [ https://issues.apache.org/jira/browse/AVRO-1243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ted Malaska updated AVRO-1243:
------------------------------

    Attachment: AVRO-1243.not-ready.patch

This is not ready for a submit but I wanted to show progress and invite feedback.

At this point the solution is coded and passes all existing unit tests.  I have to now figure
out how to add unit tests to test what I've added.  

Also I have a list of areas where I need to add comments and possible areas of performance
improvements.
                
> Avro support for all compression codecs
> ---------------------------------------
>
>                 Key: AVRO-1243
>                 URL: https://issues.apache.org/jira/browse/AVRO-1243
>             Project: Avro
>          Issue Type: Improvement
>          Components: java
>    Affects Versions: 1.7.3
>            Reporter: Ted Malaska
>            Priority: Minor
>         Attachments: AVRO-1243.not-ready.patch
>
>
> I may be reading this wrong but at this time org.apache.avro.file.CodecFactory only supports
null, deflate, and snappy compression codecs.
> I would like to change the fromString method to use Class.forName(codec).newInstance();
after the codec was not found in the REGISTERED map but before the AvroRuntimeException is
thrown. 
> Here are some of my supporting thoughts
> 1. This should not interduce much slowness because it will only be called initialize.
> 2. This will allow for support for GZip, BZip2, and LZO with out adding more dependances
to the maven pom file.
> 3. This will allow for a future Jiri I would like to do that would allow AvroOutputFormat
to be able to use the following configs: mapred.output.compress and mapred.output.compression.codec

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message