hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Owen O'Malley <omal...@apache.org>
Subject Re: [VOTE] Direction for Hadoop development
Date Tue, 07 Dec 2010 18:25:05 GMT

On Dec 7, 2010, at 9:26 AM, Doug Cutting wrote:

> On 12/07/2010 08:12 AM, Arun C Murthy wrote:
>> Blocking extensions to SequenceFile is unreasonable as has been  
>> noted by
>> several folks, there is no *technical* reason to do that.
> The change to SequenceFile is incompatible with older versions of  
> Hadoop.  It changes the file's version number so that older versions  
> will not be able to read data written by newer versions.  This is a  
> technical issue.

The new code reads the new or old versions of SequenceFile seamlessly  
using auto-detection of the version.  The old code fails with an  
explicit message saying that it can't read this version. This is the  
only mechanism available when upgrading a file format with a single  
version number and is the mechanism that we've used 6 times in the past.

If we'd used ProtocolBuffers for the SequenceFile header, we'd have  
more options for backwards compatibility, but we didn't.

-- Owen

View raw message