From hdfs-dev-return-2602-apmail-hadoop-hdfs-dev-archive=hadoop.apache.org@hadoop.apache.org Tue Mar 29 04:25:11 2011 Return-Path: Delivered-To: apmail-hadoop-hdfs-dev-archive@minotaur.apache.org Received: (qmail 97569 invoked from network); 29 Mar 2011 04:25:10 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 29 Mar 2011 04:25:10 -0000 Received: (qmail 78909 invoked by uid 500); 29 Mar 2011 04:25:10 -0000 Delivered-To: apmail-hadoop-hdfs-dev-archive@hadoop.apache.org Received: (qmail 78869 invoked by uid 500); 29 Mar 2011 04:25:10 -0000 Mailing-List: contact hdfs-dev-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hdfs-dev@hadoop.apache.org Delivered-To: mailing list hdfs-dev@hadoop.apache.org Received: (qmail 78861 invoked by uid 99); 29 Mar 2011 04:25:09 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 29 Mar 2011 04:25:09 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of shv.hadoop@gmail.com designates 209.85.161.176 as permitted sender) Received: from [209.85.161.176] (HELO mail-gx0-f176.google.com) (209.85.161.176) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 29 Mar 2011 04:25:05 +0000 Received: by gxk7 with SMTP id 7so1854165gxk.35 for ; Mon, 28 Mar 2011 21:24:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:in-reply-to:references:date :message-id:subject:from:to:content-type; bh=2QdZOIvWo6Ame1wMseR4jaKuJfmlgGONCAi9bB8xaqY=; b=BJP4RlMMdv2uJ/9ioAfpNPljLMypRLeNN7987soH4l5yWsT2N7PBI/46h9P0p0CPOW T/TWdSBp6ic38yAhJUiRgQfe7aYQmpKZmGvlnXlpeTGQVTANAqyOIxoFMujImuJ7h599 TEanglYzrUVXfy+XnSwPKd1RkmErk4at7TFRM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=iBkyDZs8zyrSKCzoX/H1TleztXpyEEa7vPNhWerqYmqQ+taf5LvVpbE8asPqNHCP1f ChAT7NBClbR/RPnGqnpBrFHCZd6hfIt4Xdc1PTOVPe+Ef/oxwZ5Mn6PP7kqe/ked/4uV vyukHHY9oleeEQP5TXcWfzPWzeutQWa5Aj1ZI= MIME-Version: 1.0 Received: by 10.146.50.6 with SMTP id x6mr4268462yax.17.1301372683709; Mon, 28 Mar 2011 21:24:43 -0700 (PDT) Received: by 10.146.168.19 with HTTP; Mon, 28 Mar 2011 21:24:43 -0700 (PDT) In-Reply-To: References: Date: Mon, 28 Mar 2011 21:24:43 -0700 Message-ID: Subject: Re: Branch for HDFS-1073 and related work From: Konstantin Shvachko To: hdfs-dev@hadoop.apache.org Content-Type: multipart/alternative; boundary=000e0cd53044887263049f9770bf --000e0cd53044887263049f9770bf Content-Type: text/plain; charset=ISO-8859-1 +1 for creating a branch. Agree with Jakob this should not mean less intensive reviewing. --Konstantin On Mon, Mar 28, 2011 at 4:39 PM, Jakob Homan wrote: > Doing this work on a branch makes sense. +1. > > However, the patches that have been committed so far required > extensive review and revision, and in one case, an addendum patch. > Additionally, the patches in related JIRAs such as HDFS-1557 and > HDFS-1572 have required multiple revisions and corrections. While > it's up to each committer which +1 they're willing to accept, I don't > see any harm in keeping our current standards for review, considering > the importance and difficulty of this work. In fact, since moving the > work to a branch will also move it off of many reviewers' radar, it > may even be reasonable to increase the scrutiny. Reviewing giant > branch merges is difficult and, I believe, more error-prone than > reviewing smaller packets of work. So far these patches have received > timely reviews, if this does not turn out to be the case, let other > committers know so we can do our part. > -jg > > > On Mon, Mar 28, 2011 at 1:40 PM, Dhruba Borthakur > wrote: > > +1. I think this will be very helpful in moving the design forward > quickly. > > > > -dhruba > > > > > > On Mon, Mar 28, 2011 at 1:14 PM, Todd Lipcon wrote: > > > >> Hi all, > >> > >> I discussed this with a couple folks over the weekend who are involved > in > >> the project, but wanted to let the dev community at large know: > >> > >> I'm planning on creating a new SVN branch for HDFS-1073 and its > subtasks. > >> For those not aware, HDFS-1073 is a rethinking of how the NN, 2NN, and > >> BackupNode store images and edit logs on disk. This will help make HA > more > >> manageable down the line and has a lot of operational benefits as > described > >> in the JIRA. The "related work" is the addition of transaction IDs to > the > >> persistent storage of the NN, and some refactoring in the edit log > >> subsystem. > >> > >> The reasoning behind creating a branch is that, since this is a fairly > >> large > >> change, it is easier to develop through a number of subtasks. But at > some > >> of > >> the intermediate points, various components will be temporarily broken. > >> Developing on a branch will allow us to make incremental progress > without > >> worrying about keeping all tests green after every change. We will of > >> course > >> make sure all tests pass before merging back into trunk. There will also > be > >> another opportunity to review before the merge into trunk. This is the > same > >> development methodology as was done for the 0.21 Append work and is now > >> being used for Federation. > >> > >> Given that there will be another opportunity to review these changes > before > >> merging into trunk, I would also like to propose that Ivan Kelly be > granted > >> the ability to "+1" patches on this branch despite not being an HDFS > >> committer. Ivan is actively contributing on this project and understands > >> the > >> code well. > >> > >> Unless there are any objections, I will create this branch and an > >> associated > >> Fix Version on JIRA tonight. > >> > >> Thanks! > >> -Todd > >> -- > >> Todd Lipcon > >> Software Engineer, Cloudera > >> > > > > > > > > -- > > Connect to me at http://www.facebook.com/dhruba > > > --000e0cd53044887263049f9770bf--