From general-return-1939-apmail-hadoop-general-archive=hadoop.apache.org@hadoop.apache.org Sat Aug 14 02:41:44 2010 Return-Path: Delivered-To: apmail-hadoop-general-archive@minotaur.apache.org Received: (qmail 6266 invoked from network); 14 Aug 2010 02:41:44 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 14 Aug 2010 02:41:44 -0000 Received: (qmail 52028 invoked by uid 500); 14 Aug 2010 02:41:43 -0000 Delivered-To: apmail-hadoop-general-archive@hadoop.apache.org Received: (qmail 51771 invoked by uid 500); 14 Aug 2010 02:41:41 -0000 Mailing-List: contact general-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: general@hadoop.apache.org Delivered-To: mailing list general@hadoop.apache.org Received: (qmail 51763 invoked by uid 99); 14 Aug 2010 02:41:41 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 14 Aug 2010 02:41:41 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED,T_RP_MATCHES_RCVD X-Spam-Check-By: apache.org Received: from [140.211.11.9] (HELO minotaur.apache.org) (140.211.11.9) by apache.org (qpsmtpd/0.29) with SMTP; Sat, 14 Aug 2010 02:41:40 +0000 Received: (qmail 6239 invoked by uid 99); 14 Aug 2010 02:41:20 -0000 Received: from localhost.apache.org (HELO mail-gx0-f176.google.com) (127.0.0.1) (smtp-auth username cdouglas, mechanism plain) by minotaur.apache.org (qpsmtpd/0.29) with ESMTP; Sat, 14 Aug 2010 02:41:20 +0000 Received: by gxk7 with SMTP id 7so1747313gxk.35 for ; Fri, 13 Aug 2010 19:41:19 -0700 (PDT) MIME-Version: 1.0 Received: by 10.231.157.207 with SMTP id c15mr2242117ibx.143.1281753679003; Fri, 13 Aug 2010 19:41:19 -0700 (PDT) Received: by 10.231.141.11 with HTTP; Fri, 13 Aug 2010 19:41:18 -0700 (PDT) In-Reply-To: <4C62369D.6000703@yahoo-inc.com> References: <4C5C8302.3010008@apache.org> <97E2C026-0C44-4FDC-A569-63E218E03D59@yahoo-inc.com> <4C602C27.6060802@apache.org> <4C62369D.6000703@yahoo-inc.com> Date: Fri, 13 Aug 2010 19:41:18 -0700 Message-ID: Subject: Re: [DISCUSSION] Combine committer lists in Common/HDFS/MapReduce From: Chris Douglas To: general@hadoop.apache.org Content-Type: text/plain; charset=ISO-8859-1 On Tue, Aug 10, 2010 at 10:35 PM, Vinod KV wrote: > > I know of very few to nearly zero number of patches in mapreduce that touch > HDFS also. And vice versa. The common case is of patches that touch both > common and mapreduce or common and hdfs. This is a good point, though changes in Common and HDFS interfaces do break MapReduce, particularly unit tests that take liberties with interface visibility. It would be convenient if HDFS committers could push in the fix with the original issue, to shrink the window where MR is broken, but in practice such changes are usually committed in short order. After all, most committers have commit access to all three projects... though this is one of the reasons why the constraint difficult to justify. > I am one of those who has access to > mapreduce project but not to common project and find more than 50% of the > patches that fall into this category. This *was* an oversight that should be corrected either through this vote or independently. > As for the separation of the repositories, I personally felt separation of > mapreduce from hdfs helped focusing on things a lot better. The last gasp > work done for 0.21, mostly by Tom, did help a lot in decoupling the > projects. Common is the hot point, sure, but as others noted, that is a > separate discussion. +1 ---- The discussion appears to be dying down. Quick summary of comments so far: * That all HDFS and MapReduce committers should have commit rights to Common appears to be undisputed. * The value of splitting the projects at all is disputed. Some have argued that it has complicated work without delivering the benefits to developers it promised, though others have experienced this as discipline rather than inconvenience. Most of these complaints reference patches that touch Common, particularly actively-developed packages like fs. The vote concerns a narrower question than the project split, though it's fair to assert a lack of unanimity on the premise of a split Hadoop, let alone the more limited question of whether to retain a split list of committers. * Not everyone agrees that combining HDFS and MapReduce committers is sound. While there is sufficient overlap today to branch all three together, patch releases could- and likely will- be cut independently. Not everyone thinks the two projects are developing independent communities, but none have difficulty imagining TLP status for both. Please feel free to amend this if I left out an important point. ----- I think we're ready to vote. Though we have no bylaws to amend, this would be a modification to them, I guess. The last-proposed set required a 2/3 majority of the PMC, IIRC. Since adding a committer requires consensus on the PMC, it's probably fair to require a 2/3 majority to cross-pollenate lists instead of a simple majority. Though the vote could be conducted on a huge cross-post to mapreduce-dev@, hdfs-dev@ and common-dev@, it'll be easier to count if it's run on general@; I'll start it here on Monday if nobody minds. -C