hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3315) New binary file format
Date Tue, 13 May 2008 16:27:55 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12596426#action_12596426
] 

Doug Cutting commented on HADOOP-3315:
--------------------------------------

> Map file uses: WritableComparator.get(keyClass) to get the comparator. 

TFile should be independent of Writable, so that it supports other serialization frameworks,
like Thrift.  Our generic Serialization framework does not yet include comparators, but it
should.  We should add a method:

RawComparator Serialization#getComparator();

TFile should use 'new SerializationFactory(conf).getSerialization(keyClass)' to get the serialization,
and then get the serializer, deserializer and comparator from that.

> New binary file format
> ----------------------
>
>                 Key: HADOOP-3315
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3315
>             Project: Hadoop Core
>          Issue Type: New Feature
>          Components: io
>            Reporter: Owen O'Malley
>            Assignee: Srikanth Kakani
>         Attachments: Tfile-1.pdf
>
>
> SequenceFile's block compression format is too complex and requires 4 codecs to compress
or decompress. It would be good to have a file format that only needs 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message