Mailing-List: contact general-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: general@hadoop.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
From: Arun C Murthy <acm@hortonworks.com>
Mime-Version: 1.0 (Apple Message framework v1084)
Content-Type: multipart/alternative; boundary=Apple-Mail-30--545547142
Subject: Re: Add Append-HBase support in upcoming 20.205
Date: Thu, 1 Sep 2011 15:36:25 -0700
In-Reply-To: <83DFC074-A9E5-4670-8CBF-80999A86A71C@hortonworks.com>
To: general@hadoop.apache.org
References: <83DFC074-A9E5-4670-8CBF-80999A86A71C@hortonworks.com>
Message-Id: <A5A5E2E2-67F2-485B-A528-B561C021A05C@hortonworks.com>

--Apple-Mail-30--545547142
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii


On Aug 31, 2011, at 11:41 AM, sanjay Radia wrote:

>=20
> I propose that the 20-append patches (details below)  be included in =
20.205 which will become the first official Apache
> release of Hadoop that supports Append and HBase.
>=20

> Background:
> There hasn't been a official Apache release that supports HBase.=20
> The HBase community have instead been using the 20-append branch; the =
patches were contributed by the HBase community including Facebook. The =
Cloudera distribution has also included these patches.
> Andrew Purtell has ported these patches to 20-security branch.
>=20


+1

I think it's high time (I thought so too last Dec too: =
http://s.apache.org/jr) we had an official Hadoop release which supports =
HBase. Thanks for all the effort Andrew - it will be really nice to have =
an Apache 0.20 with security+append!

Arun

> Risk Level:
> These patches have been used and tested on large HBase clusters by FB =
, by those who use 20-append branch directly (various users including a =
500 node HBase cluster at Yahoo) and by those that use the Cloudera =
distribution. We have reviewed the patches and have conducted further =
tests; testing and validation continues.
>=20
>=20
> Patches:
> HDFS-200. Support append and sync for hadoop 0.20 branch.
> HDFS-142. Blocks that are being written by a client are stored in the =
blocksBeingWritten directory.
> HDFS-1057.  Concurrent readers hit ChecksumExceptions if following a =
writer to very end of file
> HDFS-724.  Use a bidirectional heartbeat to detect stuck pipeline.
> HDFS-895. Allow hflush/sync to occur in parallel with new writes to =
the file.
> HDFS-1520. Lightweight NameNode operation recoverLease to trigger =
lease recovery.
> HDFS-1555. Disallow pipelien recovery if a file is already being lease =
recovered.
> HDFS-1554. New semantics for recoverLease.
> HDFS-988. Fix bug where savenameSpace can corrupt edits log.
> HDFS-826. Allow a mechanism for an application to detect that =
datanode(s) have died in the write pipeline.
> HDFS-630. Client can exclude specific nodes in the write pipeline.
> HDFS-1141. completeFile does not check lease ownership.
> HDFS-1204. Lease expiration should recover single files, not entire =
lease holder
> HDFS-1254. Support append/sync via the default configuration.
> HDFS-1346. DFSClient receives out of order packet ack.
> HDFS-1054. remove sleep before retry for allocating a block.
>=20


--Apple-Mail-30--545547142--