hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arun C Murthy <...@hortonworks.com>
Subject Re: Add Append-HBase support in upcoming 20.205
Date Fri, 02 Sep 2011 06:01:19 GMT

On Sep 1, 2011, at 10:56 PM, Andrew Purtell wrote:

>> From: Arun C Murthy <acm@hortonworks.com>
> 
>> +1
>> 
>> I think it's high time (I thought so too last Dec too: 
>> http://s.apache.org/jr) we had an official Hadoop release which supports
>> HBase. Thanks for all the effort Andrew - it will be really nice to have
>> an Apache 0.20 with security+append!
> 
> Thanks to Dhruba, Todd, Hairong, and the other original contributors of append support
to HDFS 0.20.x.
> 

But of course, I really should have qualified my statement by saying: thanks for porting branch-0.20-append
on 0.20.2xx.

Yes, thanks to everyone who contributed to branch-0.20-append of course.

Arun

> I'm +1 obviously. :-)
> 
> 
> Best regards,
> 
> 
>     - Andy
> 
> Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
> 
> 
> ----- Original Message -----
>> From: Arun C Murthy <acm@hortonworks.com>
>> To: general@hadoop.apache.org
>> Cc: 
>> Sent: Friday, September 2, 2011 6:36 AM
>> Subject: Re: Add Append-HBase support in upcoming 20.205
>> 
>> 
>> On Aug 31, 2011, at 11:41 AM, sanjay Radia wrote:
>> 
>>> 
>>> I propose that the 20-append patches (details below)  be included in 20.205 
>> which will become the first official Apache
>>> release of Hadoop that supports Append and HBase.
>>> 
>> 
>>> Background:
>>> There hasn't been a official Apache release that supports HBase. 
>>> The HBase community have instead been using the 20-append branch; the 
>> patches were contributed by the HBase community including Facebook. The Cloudera

>> distribution has also included these patches.
>>> Andrew Purtell has ported these patches to 20-security branch.
>>> 
>> 
>> 
>> +1
>> 
>> I think it's high time (I thought so too last Dec too: 
>> http://s.apache.org/jr) we had an official Hadoop release which supports HBase. 
>> Thanks for all the effort Andrew - it will be really nice to have an Apache 0.20

>> with security+append!
>> 
>> Arun
>> 
>>> Risk Level:
>>> These patches have been used and tested on large HBase clusters by FB , by 
>> those who use 20-append branch directly (various users including a 500 node 
>> HBase cluster at Yahoo) and by those that use the Cloudera distribution. We have

>> reviewed the patches and have conducted further tests; testing and validation 
>> continues.
>>> 
>>> 
>>> Patches:
>>> HDFS-200. Support append and sync for hadoop 0.20 branch.
>>> HDFS-142. Blocks that are being written by a client are stored in the 
>> blocksBeingWritten directory.
>>> HDFS-1057.  Concurrent readers hit ChecksumExceptions if following a writer 
>> to very end of file
>>> HDFS-724.  Use a bidirectional heartbeat to detect stuck pipeline.
>>> HDFS-895. Allow hflush/sync to occur in parallel with new writes to the 
>> file.
>>> HDFS-1520. Lightweight NameNode operation recoverLease to trigger lease 
>> recovery.
>>> HDFS-1555. Disallow pipelien recovery if a file is already being lease 
>> recovered.
>>> HDFS-1554. New semantics for recoverLease.
>>> HDFS-988. Fix bug where savenameSpace can corrupt edits log.
>>> HDFS-826. Allow a mechanism for an application to detect that datanode(s) 
>> have died in the write pipeline.
>>> HDFS-630. Client can exclude specific nodes in the write pipeline.
>>> HDFS-1141. completeFile does not check lease ownership.
>>> HDFS-1204. Lease expiration should recover single files, not entire lease 
>> holder
>>> HDFS-1254. Support append/sync via the default configuration.
>>> HDFS-1346. DFSClient receives out of order packet ack.
>>> HDFS-1054. remove sleep before retry for allocating a block.
>>> 
>> 


Mime
View raw message