hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <apurt...@apache.org>
Subject Re: Add Append-HBase support in upcoming 20.205
Date Fri, 02 Sep 2011 05:56:44 GMT
> From: Arun C Murthy <acm@hortonworks.com>

> +1
> 
> I think it's high time (I thought so too last Dec too: 
> http://s.apache.org/jr) we had an official Hadoop release which supports
> HBase. Thanks for all the effort Andrew - it will be really nice to have
> an Apache 0.20 with security+append!

Thanks to Dhruba, Todd, Hairong, and the other original contributors of append support to
HDFS 0.20.x.

I'm +1 obviously. :-)


Best regards,


    - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)


----- Original Message -----
> From: Arun C Murthy <acm@hortonworks.com>
> To: general@hadoop.apache.org
> Cc: 
> Sent: Friday, September 2, 2011 6:36 AM
> Subject: Re: Add Append-HBase support in upcoming 20.205
> 
> 
> On Aug 31, 2011, at 11:41 AM, sanjay Radia wrote:
> 
>> 
>>  I propose that the 20-append patches (details below)  be included in 20.205 
> which will become the first official Apache
>>  release of Hadoop that supports Append and HBase.
>> 
> 
>>  Background:
>>  There hasn't been a official Apache release that supports HBase. 
>>  The HBase community have instead been using the 20-append branch; the 
> patches were contributed by the HBase community including Facebook. The Cloudera 
> distribution has also included these patches.
>>  Andrew Purtell has ported these patches to 20-security branch.
>> 
> 
> 
> +1
> 
> I think it's high time (I thought so too last Dec too: 
> http://s.apache.org/jr) we had an official Hadoop release which supports HBase. 
> Thanks for all the effort Andrew - it will be really nice to have an Apache 0.20 
> with security+append!
> 
> Arun
> 
>>  Risk Level:
>>  These patches have been used and tested on large HBase clusters by FB , by 
> those who use 20-append branch directly (various users including a 500 node 
> HBase cluster at Yahoo) and by those that use the Cloudera distribution. We have 
> reviewed the patches and have conducted further tests; testing and validation 
> continues.
>> 
>> 
>>  Patches:
>>  HDFS-200. Support append and sync for hadoop 0.20 branch.
>>  HDFS-142. Blocks that are being written by a client are stored in the 
> blocksBeingWritten directory.
>>  HDFS-1057.  Concurrent readers hit ChecksumExceptions if following a writer 
> to very end of file
>>  HDFS-724.  Use a bidirectional heartbeat to detect stuck pipeline.
>>  HDFS-895. Allow hflush/sync to occur in parallel with new writes to the 
> file.
>>  HDFS-1520. Lightweight NameNode operation recoverLease to trigger lease 
> recovery.
>>  HDFS-1555. Disallow pipelien recovery if a file is already being lease 
> recovered.
>>  HDFS-1554. New semantics for recoverLease.
>>  HDFS-988. Fix bug where savenameSpace can corrupt edits log.
>>  HDFS-826. Allow a mechanism for an application to detect that datanode(s) 
> have died in the write pipeline.
>>  HDFS-630. Client can exclude specific nodes in the write pipeline.
>>  HDFS-1141. completeFile does not check lease ownership.
>>  HDFS-1204. Lease expiration should recover single files, not entire lease 
> holder
>>  HDFS-1254. Support append/sync via the default configuration.
>>  HDFS-1346. DFSClient receives out of order packet ack.
>>  HDFS-1054. remove sleep before retry for allocating a block.
>> 
>

Mime
View raw message