hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hong Tang (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-3315) New binary file format
Date Thu, 07 May 2009 11:01:30 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-3315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Hong Tang updated HADOOP-3315:

    Fix Version/s: 0.18.3
     Release Note: 
Main changes of this patch since last upload:
    * Solving the missing LzoCodec issue.
    * Enhanced the unit tests.

  was:TFile patch

           Status: Patch Available  (was: In Progress)

The patch is tested to work with trunk head, 0.18.3 and 0.20. 

For 0.18.3 and 0.20, it works as my previous upload and would use the built-in LzoCodec for
"lzo" compression.

For trunk, by default, "lzo" compression is not support. User must specific the class name
for a 3rd-party lzo compression library through property "io.compression.codec.lzo.class",
and set the classpath and java.library.path properly. The code is tested to work with hadoop-gpl-compression:

> New binary file format
> ----------------------
>                 Key: HADOOP-3315
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3315
>             Project: Hadoop Core
>          Issue Type: New Feature
>          Components: io
>            Reporter: Owen O'Malley
>            Assignee: Hong Tang
>             Fix For: 0.21.0, 0.20.0, 0.18.3
>         Attachments: hadoop-3315-0507.patch, HADOOP-3315_20080908_TFILE_PREVIEW_WITH_LZO_TESTS.patch,
HADOOP-3315_20080915_TFILE.patch, hadoop-trunk-tfile.patch, hadoop-trunk-tfile.patch, TFile
Specification 20081217.pdf
> SequenceFile's block compression format is too complex and requires 4 codecs to compress
or decompress. It would be good to have a file format that only needs 

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message