Return-Path: X-Original-To: apmail-incubator-general-archive@www.apache.org Delivered-To: apmail-incubator-general-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id CF507EC7E for ; Tue, 26 Feb 2013 18:31:06 +0000 (UTC) Received: (qmail 65486 invoked by uid 500); 26 Feb 2013 18:31:06 -0000 Delivered-To: apmail-incubator-general-archive@incubator.apache.org Received: (qmail 65318 invoked by uid 500); 26 Feb 2013 18:31:05 -0000 Mailing-List: contact general-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: general@incubator.apache.org Delivered-To: mailing list general@incubator.apache.org Received: (qmail 65300 invoked by uid 99); 26 Feb 2013 18:31:05 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 26 Feb 2013 18:31:05 +0000 X-ASF-Spam-Status: No, hits=-2.3 required=5.0 tests=RCVD_IN_DNSWL_MED,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy) Received: from [128.149.139.109] (HELO mail.jpl.nasa.gov) (128.149.139.109) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 26 Feb 2013 18:30:57 +0000 Received: from mail.jpl.nasa.gov (ap-ehub-sp01.jpl.nasa.gov [128.149.137.148]) by smtp.jpl.nasa.gov (Sentrion-MTA-4.2.2/Sentrion-MTA-4.2.2) with ESMTP id r1QIUWPj013037 (using TLSv1/SSLv3 with cipher AES128-SHA (128 bits) verified NO) for ; Tue, 26 Feb 2013 10:30:33 -0800 Received: from AP-EMBX-SP40.RES.AD.JPL ([169.254.7.238]) by ap-ehub-sp01.RES.AD.JPL ([169.254.3.109]) with mapi id 14.02.0342.003; Tue, 26 Feb 2013 10:30:32 -0800 From: "Mattmann, Chris A (388J)" To: "general@incubator.apache.org" Subject: Re: [PROPOSAL] Curator for the Apache Incubator Thread-Topic: [PROPOSAL] Curator for the Apache Incubator Thread-Index: AQHOE766y7/9rAJzW0aw31Ez/kK5JZiL/GAAgABAfoCAAAYQgIAAjn0AgAAK6YCAAA4cgIAABOGA//+H5wA= Date: Tue, 26 Feb 2013 18:30:31 +0000 Message-ID: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: user-agent: Microsoft-MacOutlook/14.3.1.130117 x-originating-ip: [128.149.137.113] Content-Type: text/plain; charset="iso-8859-1" Content-ID: <0204664CA40F23488F3B938B8A68FCC0@ad.jpl> Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Source-Sender: chris.a.mattmann@jpl.nasa.gov X-AUTH: Authorized X-Virus-Checked: Checked by ClamAV on apache.org Hi Guys, Sorry I have to ask the question here. If the mentors consist of PMC members on ZK (at least Pat and Mahadev), what's the problem with creating a branch in ZK and just having the code be there and getting the Curator proposed committers as committers in ZK ville, and hopefully PMC members soon thereafter. I have to agree with Greg here. Seems like Incubation is something that might not be needed. If the end result of this after N months is that Curator "graduates" into another set of flipping bylaw updates, and more legislation that makes these people unofficial=20 PMC members on ZK, then I'm double triple -1 with 50 piles of coal on top. Yes, I'm=20 still remembering the HCatalog/Hive thing here - I still don't get that. Either: (a) define Curator to be its own separate project/community, with a goal of TLP;=20 or (b) nix Incubation and just make these guys part of ZK, on a branch now.=20 Cheers, Chris On 2/26/13 9:40 AM, "Benson Margulies" wrote: >On Tue, Feb 26, 2013 at 12:22 PM, Patrick Hunt wrote: >> On Tue, Feb 26, 2013 at 8:32 AM, Benson Margulies >> wrote: >>> If you think that the right destination for curator is as part of ZK, >>> then it would be good to see substantive participation of the ZK PMC >>> in the incubation. The goal should be to 'graduate' by having the >>> curator community be granted karma at ZK and the code folded in. This >>> would require, I think, the ZK community to supply at least one >>> mentor, and to have a plan in advance for the eventual votes based on >>> success in the incubator. >> >> Hi Benson. What you're suggesting matches my current thinking. >> >> The three Curator mentors are as follows: Mahadev and I (champion) are >> both PMC members on ZK, Enis S=F6ztutar is active on HBase and >> interested in using Curator for that project (which already uses ZK >> heavily). > >OK, that's lovely. > > >> >> Patrick >> >>> On Tue, Feb 26, 2013 at 10:53 AM, Henry Saputra >>> wrote: >>>> Ah no, I was not suggesting about Curator to become subproject of ZK. >>>>I >>>> just afraid that if Curator is going as incubator it will end up as >>>>sub of >>>> ZK as merging process. >>>> >>>> Like Greg has mentioned in another reply, I would prefer Curator to be >>>> merged as a higher level ZK client. Surely project like HBase and >>>>others >>>> that relying on ZK would appreciate simpler client to ZK. >>>> >>>> Thanks, >>>> >>>> Henry >>>> >>>> >>>> On Mon, Feb 25, 2013 at 11:23 PM, Patrick Hunt >>>>wrote: >>>> >>>>> On Mon, Feb 25, 2013 at 11:01 PM, Henry Saputra >>>>> >>>>> wrote: >>>>> > So isnt this similar to HCatalog which relying on Hive metadata >>>>>service >>>>> > that ends up as sub project of Apache Hive? >>>>> > >>>>> >>>>> I was against having Curator as a sub when it came up on the original >>>>> discussion thread, I still am. >>>>> >>>>> Patrick >>>>> >>>>> > >>>>> > >>>>> > On Mon, Feb 25, 2013 at 7:10 PM, Greg Stein >>>>>wrote: >>>>> > >>>>> >> My concern is that we're looking at two "new" committers, rather >>>>>than >>>>> >> a Curator community. Following normal Incubator work, Curator >>>>>would >>>>> >> build a community for itself. But then we'd have a community >>>>> >> *distinct* from that of Zookeeper. And it really looks like this >>>>> >> should be part of Zookeeper itself -- a more capable and >>>>>easier-to-use >>>>> >> client. >>>>> >> >>>>> >> So I question the incubation of this. Why do we want to build a >>>>> >> new/separate project? Why isn't this just part of Zookeeper right >>>>>from >>>>> >> the start? >>>>> >> >>>>> >> I would suggest that this work is placed on a branch within >>>>>Zookeeper. >>>>> >> That Jordan and Jay become committers on that branch (not >>>>>necessarily >>>>> >> Zookeeper trunk). Over time, the branch can be folded into trunk, >>>>> >> along with all the various tests, doc, and other artifacts that I >>>>>see >>>>> >> in the GitHub repository. And hopefully that Jordan and Jay become >>>>> >> regular committers (and PMC members!) of the Zookeeper project >>>>>itself. >>>>> >> >>>>> >> The current Zookeeper client can remain for backwards compat, and >>>>>the >>>>> >> Curator work can become the next-gen client. >>>>> >> >>>>> >> Honestly, in my opnion, incubating this project seems like it >>>>>would >>>>> >> create a distinct community, and really doesn't seem like it would >>>>> >> serve the Zookeeper community. >>>>> >> >>>>> >> All that said, I am not familiar with the Zookeeper or Curator >>>>> >> communities. But from this read, I don't think Incubation is the >>>>>right >>>>> >> approach. I would rather push for a more direct incorporation of >>>>> >> Curator directly into Zookeeper. (use the short-form IP >>>>>clearance) ... >>>>> >> so, unless somebody can help me understand the communities and >>>>> >> situation better, I'm -1 (binding) on this incubation. I'd rather >>>>>see >>>>> >> combined, rather than distinct, communities from the start. >>>>> >> >>>>> >> Thanks, >>>>> >> -g >>>>> >> >>>>> >> On Mon, Feb 25, 2013 at 8:14 PM, Jordan Zimmerman >>>>> >> wrote: >>>>> >> > Hello, >>>>> >> > >>>>> >> > I would like to propose that Curator to be an Apache Incubator >>>>> project. >>>>> >> > >>>>> >> > The proposal can be found here: >>>>> >> http://wiki.apache.org/incubator/CuratorProposal >>>>> >> > >>>>> >> > I have included the contents of the proposal below. >>>>> >> > >>>>> >> > Sincerely, >>>>> >> > >>>>> >> > Jordan Zimmerman >>>>> >> > >>>>> >> > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D >>>>> >> > >>>>> >> > =3D Curator - ZooKeeper client wrapper and rich ZooKeeper >>>>>framework =3D >>>>> >> > >>>>> >> > =3D=3D Abstract =3D=3D >>>>> >> > >>>>> >> > Curator is a set of Java libraries that make using Apache >>>>>ZooKeeper >>>>> much >>>>> >> easier. While ZooKeeper comes bundled with a Java client, using >>>>>the >>>>> client >>>>> >> is non-trivial and error prone. >>>>> >> > >>>>> >> > =3D=3D Proposal =3D=3D >>>>> >> > >>>>> >> > Curator is a set of Java libraries that make using Apache >>>>>ZooKeeper >>>>> much >>>>> >> easier. While ZooKeeper comes bundled with a Java client, using >>>>>the >>>>> client >>>>> >> is non-trivial and error prone. It consists of three components >>>>>that >>>>> build >>>>> >> on each other. Curator Client is a replacement for the bundled >>>>>ZooKeeper >>>>> >> class that takes care of some low-level housekeeping and provides >>>>>some >>>>> >> useful utilities. Curator Framework is a high-level API that >>>>>greatly >>>>> >> simplifies using ZooKeeper. It adds many features that build on >>>>> ZooKeeper >>>>> >> and handles the complexity of managing connections to the >>>>>ZooKeeper >>>>> cluster >>>>> >> and retrying operations. Curator Recipes consists of >>>>>implementations of >>>>> >> some of the common ZooKeeper =B3recipes=B2. Additionally, Curator >>>>>Test is >>>>> >> included which includes utilities to help with unit testing >>>>> ZooKeeper-based >>>>> >> applications. >>>>> >> > >>>>> >> > =3D=3D Background =3D=3D >>>>> >> > >>>>> >> > Curator was initially developed by Netflix to make writing >>>>> >> ZooKeeper-based applications easier and more reliable. Curator was >>>>> >> open-sourced by Netflix on !GitHub as an Apache 2.0 licensed >>>>>project in >>>>> >> July 2011. During this time Curator has been formally released >>>>>many >>>>> times >>>>> >> and has gained widespread adoption. >>>>> >> > >>>>> >> > =3D=3D Rationale =3D=3D >>>>> >> > >>>>> >> > New users of ZooKeeper are surprised to learn that a significant >>>>> amount >>>>> >> of connection management must be done manually. For example, when >>>>>the >>>>> >> ZooKeeper client connects to the ensemble it must negotiate a new >>>>> session, >>>>> >> etc. This takes some time. If you use a ZooKeeper client API >>>>>before the >>>>> >> connection process has completed, ZooKeeper will throw an >>>>>exception. >>>>> These >>>>> >> types of exceptions are referred to as =B3recoverable=B2 errors. >>>>> >> > Curator automatically handles connection management, greatly >>>>> simplifying >>>>> >> client code. Instead of directly using the ZooKeeper APIs you use >>>>> Curator >>>>> >> APIs that internally check for connection completion and wrap each >>>>> >> ZooKeeper API in a retry loop. Curator uses a retry mechanism to >>>>>handle >>>>> >> recoverable errors and automatically retry operations. The method >>>>>of >>>>> retry >>>>> >> is customizable. Curator comes bundled with several >>>>>implementations >>>>> >> (ExponentialBackoffRetry, etc.) or custom implementations can be >>>>> written. >>>>> >> > >>>>> >> > The ZooKeeper documentation describes many possible uses for >>>>>ZooKeeper >>>>> >> calling each a =B3recipe=B2. While the distribution comes bundled >>>>>with a few >>>>> >> implementations of these recipes, most ZooKeeper users will need >>>>>to >>>>> >> manually implement one or more of the recipes. Implementing a >>>>>ZooKeeper >>>>> >> recipe is not trivial. Besides the connection handling issues, >>>>>there are >>>>> >> numerous edge cases that are not well documented that must be >>>>> considered. >>>>> >> For example, many recipes require that an ephemeral-sequential >>>>>node be >>>>> >> created. New users of ZooKeeper will not know that there is an >>>>>edge >>>>> case in >>>>> >> ephemeral-sequential node creation that requires you to put a >>>>>special >>>>> >> =B3marker=B2 in the node=B9s name so that you can search for the >>>>>created node >>>>> if >>>>> >> an I/O failure occurs. This is but one of many edge cases that >>>>>are not >>>>> well >>>>> >> documented but are handled by Curator. >>>>> >> > >>>>> >> > =3D Current Status =3D >>>>> >> > >>>>> >> > =3D=3D Meritocracy =3D=3D >>>>> >> > >>>>> >> > Curator was initially developed by Jordan Zimmerman in 2011 at >>>>> Netflix. >>>>> >> Developers external to Netflix provided feedback, suggested >>>>>features and >>>>> >> fixes and implemented extensions of Curator. Netflix's >>>>>engineering team >>>>> has >>>>> >> since maintained the project and has been dedicated towards its >>>>> >> improvement. Contributors to Curator include developers from >>>>>multiple >>>>> >> organizations around the world. Curator will be a meritocracy as >>>>>it >>>>> enters >>>>> >> the Incubator and beyond. >>>>> >> > >>>>> >> > =3D=3D Community =3D=3D >>>>> >> > >>>>> >> > Curator is currently used by a number of organizations all over >>>>>the >>>>> >> world. Curator has an active and growing user and developer >>>>>community >>>>> with >>>>> >> active participation in the [[ >>>>> http://groups.google.com/group/curator-users]] >>>>> >> mailing list and at its !Github home: [[ >>>>> >> https://github.com/Netflix/curator]]. >>>>> >> > >>>>> >> > Since open sourcing the project, there have been fifteen >>>>>individuals >>>>> >> from various organizations who have contributed code. >>>>> >> > >>>>> >> > =3D=3D Core Developers =3D=3D >>>>> >> > >>>>> >> > The core developers for Curator are: >>>>> >> > * Jordan Zimmerman >>>>> >> > * Jay Zarfoss >>>>> >> > >>>>> >> > Jordan has contributed towards Apache ZooKeeper and both Jordan >>>>>and >>>>> Jay >>>>> >> are familiar with Apache principles and philosophy for community >>>>>driven >>>>> >> software development. >>>>> >> > >>>>> >> > =3D=3D Alignment =3D=3D >>>>> >> > >>>>> >> > Curator is a natural complement for Apache ZooKeeper. Java >>>>>users of >>>>> >> ZooKeeper will naturally want to use Curator. When Curator >>>>>graduates >>>>> from >>>>> >> Incubator it may be useful to distribute Curator artifacts as >>>>>part of >>>>> >> ZooKeeper releases as the preferred/recommended client side >>>>>library. >>>>> >> Further, at graduation a determination can be made as to whether >>>>>Curator >>>>> >> should become a Top Level Project or be merged into ZooKeeper >>>>>itself. >>>>> >> > >>>>> >> > =3D Known Risks =3D >>>>> >> > >>>>> >> > =3D=3D Orphaned Products =3D=3D >>>>> >> > >>>>> >> > Curator is already deployed in production at multiple companies >>>>>and >>>>> they >>>>> >> are actively participating in creating new features. Curator is >>>>>getting >>>>> >> traction with developers and thus the risks of it being orphaned >>>>>are >>>>> >> minimal. >>>>> >> > >>>>> >> > =3D=3D Inexperience with Open Source =3D=3D >>>>> >> > >>>>> >> > All code developed for Curator has been open sourced by Netflix >>>>>under >>>>> >> Apache 2.0 license. All committers to Curator are intimately >>>>>familiar >>>>> with >>>>> >> the Apache model for open-source development and are experienced >>>>>with >>>>> >> working with new contributors. >>>>> >> > >>>>> >> > =3D=3D Homogeneous Developers =3D=3D >>>>> >> > >>>>> >> > The initial committers are from a single organization. However, >>>>>we >>>>> >> expect that once approved for incubation, the project will >>>>>attract new >>>>> >> contributors from diverse organizations and will thus grow >>>>>organically. >>>>> The >>>>> >> submission of patches from developers from several different >>>>> organizations >>>>> >> is a strong indication that Curator will be widely adopted. >>>>> >> > >>>>> >> > =3D=3D Reliance on Salaried Developers =3D=3D >>>>> >> > >>>>> >> > It is expected that Curator will be developed on salaried and >>>>> volunteer >>>>> >> time, although all of the initial developers will work on it >>>>>mainly on >>>>> >> salaried time. >>>>> >> > >>>>> >> > =3D=3D Relationships with Other Apache Products =3D=3D >>>>> >> > >>>>> >> > Curator depends upon other Apache Projects: Apache ZooKeeper, >>>>>Apache >>>>> >> Log4J, and multiple Apache Commons components. Its build depends >>>>>upon >>>>> >> Apache Maven. Notably, there is interest from other Apache >>>>>Projects >>>>> such as >>>>> >> HBase in adopting Curator as the client library for ZooKeeper. >>>>>Apache >>>>> James >>>>> >> Mailbox has already incorporated Curator. >>>>> >> > >>>>> >> > =3D=3D An Excessive Fascination with the Apache Brand =3D=3D >>>>> >> > >>>>> >> > We would like Curator to become an Apache project to further >>>>>foster a >>>>> >> healthy community of contributors and consumers around the >>>>>project. >>>>> Since >>>>> >> Curator directly interacts with Apache ZooKeeper and solves an >>>>>important >>>>> >> problem of many ZooKeeper users, residing in the Apache Software >>>>> Foundation >>>>> >> will increase interaction with the larger community. >>>>> >> > >>>>> >> > =3D Documentation =3D >>>>> >> > >>>>> >> > * Curator wiki at GitHub: >>>>>https://github.com/Netflix/curator/wiki >>>>> >> > * Curator issues at GitHub: >>>>> https://github.com/Netflix/curator/issues >>>>> >> > * Curator javadoc at GitHub: >>>>>http://netflix.github.com/curator/doc/ >>>>> >> > >>>>> >> > =3D Initial Source =3D >>>>> >> > >>>>> >> > * git://github.com/Netflix/curator.git >>>>> >> > >>>>> >> > =3D=3D Source and Intellectual Property Submission Plan =3D=3D >>>>> >> > >>>>> >> > * The initial source is already licensed under the Apache >>>>>License, >>>>> >> Version 2.0. >>>>>https://github.com/Netflix/curator/blob/master/LICENSE.txt >>>>> >> > >>>>> >> > =3D=3D External Dependencies =3D=3D >>>>> >> > >>>>> >> > The required external dependencies are all Apache License or >>>>> compatible >>>>> >> licenses. Following components with non-Apache licenses are >>>>>enumerated: >>>>> >> > >>>>> >> > * org.slf4j: MIT-like License >>>>> >> > * org.mockito: MIT-like License >>>>> >> > >>>>> >> > =3D=3D Cryptography =3D=3D >>>>> >> > >>>>> >> > Curator contains no known cryptography. >>>>> >> > >>>>> >> > =3D Required Resources =3D >>>>> >> > >>>>> >> > =3D=3D Mailing lists =3D=3D >>>>> >> > >>>>> >> > * curator-private (with moderated subscriptions) >>>>> >> > * curator-dev >>>>> >> > * curator-commits >>>>> >> > * curator-user >>>>> >> > >>>>> >> > =3D=3D Github Repositories =3D=3D >>>>> >> > >>>>> >> > http://github.com/apache/curator >>>>>git://git.apache.org/curator.git >>>>> >> > >>>>> >> > =3D=3D Issue Tracking =3D=3D >>>>> >> > >>>>> >> > JIRA Curator (CURATOR) >>>>> >> > >>>>> >> > =3D=3D Other Resources =3D=3D >>>>> >> > >>>>> >> > The existing code already has unit and integration tests so we >>>>>would >>>>> >> like a Jenkins instance to run them whenever a new patch is >>>>>submitted. >>>>> This >>>>> >> can be added after project creation. >>>>> >> > >>>>> >> > =3D Initial Committers =3D >>>>> >> > >>>>> >> > * Jordan Zimmerman (jzimmerman at netflix dot com) >>>>> >> > * Jay Zarfoss (jzarfoss at netflix dot com) >>>>> >> > >>>>> >> > =3D Affiliations =3D >>>>> >> > >>>>> >> > * Jordan Zimmerman, Netflix >>>>> >> > * Jay Zarfoss, Netflix >>>>> >> > >>>>> >> > =3D Sponsors =3D >>>>> >> > >>>>> >> > =3D=3D Champion =3D=3D >>>>> >> > >>>>> >> > * Patrick Hunt >>>>> >> > >>>>> >> > =3D=3D Nominated Mentors =3D=3D >>>>> >> > >>>>> >> > * Patrick Hunt >>>>> >> > * Enis S=F6ztutar >>>>> >> > * Mahadev Konar >>>>> >> > >>>>> >> > =3D=3D Sponsoring Entity =3D=3D >>>>> >> > >>>>> >> > * Apache Incubator PMC >>>>> >> >>>>> >>=20 >>>>>--------------------------------------------------------------------- >>>>> >> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org >>>>> >> For additional commands, e-mail: general-help@incubator.apache.org >>>>> >> >>>>> >> >>>>> >>>>> --------------------------------------------------------------------- >>>>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org >>>>> For additional commands, e-mail: general-help@incubator.apache.org >>>>> >>>>> >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org >>> For additional commands, e-mail: general-help@incubator.apache.org >>> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org >> For additional commands, e-mail: general-help@incubator.apache.org >> > >--------------------------------------------------------------------- >To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org >For additional commands, e-mail: general-help@incubator.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org For additional commands, e-mail: general-help@incubator.apache.org