hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hong Tang (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-3315) New binary file format
Date Wed, 24 Jun 2009 04:22:07 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-3315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Hong Tang updated HADOOP-3315:
------------------------------

    Status: Patch Available  (was: Open)

Reviewed the patch hadoop-3315-0619.patch with Chris Douglas. The newly uploaded patch addressed
the following points raised during the review:

- Replaced some inappropriate usage of IllegalStateException with RuntimeException or IOException.
- In ChunkEcoder, write out a new chunk if cached data + user buffer >= chunk buffer.
- Removed constructor ByteArray(String).
- Fixed logical error "256 & 1024" -> "256 * 1024".
- Fixed comments in RawComparable javadoc.
- Renamed various classes and field variables to match with names used in TFile spec.
- Replaced ResettableByteArrayInputStream => DataInputBuffer
- Replaced SimpleByteArrayOutputStream => DataOutputBuffer
- Used IOUtils.cleanup in TFile.Writer.close.
- Removed AssertionException to RuntimeException.
- minBlockSize documentation lacks units (in bytes)
- Document Compression::get(C|D) 0.18 compatibility code
- Unconventional try/catch in finally usage pattern in BCFile
- setting hadoop.native.lib in Compression not Kosher
- Unconventional try/finally pattern in BCFile, converted to try/catch instead of try/finally
- TFile.Location::hashCode incl redundant computation
- double-semicolon @ TFile:801
- Use DefaultCodec instead of GzipCodec to make pooling work for non-native zlib


> New binary file format
> ----------------------
>
>                 Key: HADOOP-3315
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3315
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: io
>            Reporter: Owen O'Malley
>            Assignee: Hong Tang
>             Fix For: 0.21.0
>
>         Attachments: hadoop-3315-0507.patch, hadoop-3315-0509-2.patch, hadoop-3315-0509.patch,
hadoop-3315-0513.patch, hadoop-3315-0514.patch, hadoop-3315-0601.patch, hadoop-3315-0602.patch,
hadoop-3315-0605.patch, hadoop-3315-0612.patch, hadoop-3315-0623-2.patch, HADOOP-3315_20080908_TFILE_PREVIEW_WITH_LZO_TESTS.patch,
HADOOP-3315_20080915_TFILE.patch, hadoop-trunk-tfile.patch, hadoop-trunk-tfile.patch, TFile
Specification 20081217.pdf
>
>
> SequenceFile's block compression format is too complex and requires 4 codecs to compress
or decompress. It would be good to have a file format that only needs 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message