hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amir Youssefi (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3315) New binary file format
Date Fri, 26 Sep 2008 06:19:44 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12634760#action_12634760
] 

Amir Youssefi commented on HADOOP-3315:
---------------------------------------

1) Currently there are two versions one for TFile API and other for BCFile API. Version consists
of a Major and a Minor.

TFile/BCFile API versions are stored when a new file is written. Later when we read a file,
API versions used for reading are compared to versions from file. Currently we only check
Major and throw an exception if there is a mismatch in Major.

We can add more compatibility logic to this code when necessary.

2) There are some compatibility ideas incorporates in the design (see Page 11 of attached
spec for more details).

> New binary file format
> ----------------------
>
>                 Key: HADOOP-3315
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3315
>             Project: Hadoop Core
>          Issue Type: New Feature
>          Components: io
>            Reporter: Owen O'Malley
>            Assignee: Amir Youssefi
>         Attachments: HADOOP-3315_20080908_TFILE_PREVIEW_WITH_LZO_TESTS.patch, HADOOP-3315_20080915_TFILE.patch,
TFile Specification Final.pdf
>
>
> SequenceFile's block compression format is too complex and requires 4 codecs to compress
or decompress. It would be good to have a file format that only needs 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message