incubator-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Hunt <ph...@apache.org>
Subject Re: [PROPOSAL] Curator for the Apache Incubator
Date Tue, 26 Feb 2013 07:23:21 GMT
On Mon, Feb 25, 2013 at 11:01 PM, Henry Saputra <henry.saputra@gmail.com> wrote:
> So isnt this similar to HCatalog which relying on Hive metadata service
> that ends up as sub project of Apache Hive?
>

I was against having Curator as a sub when it came up on the original
discussion thread, I still am.

Patrick

>
>
> On Mon, Feb 25, 2013 at 7:10 PM, Greg Stein <gstein@gmail.com> wrote:
>
>> My concern is that we're looking at two "new" committers, rather than
>> a Curator community. Following normal Incubator work, Curator would
>> build a community for itself. But then we'd have a community
>> *distinct* from that of Zookeeper. And it really looks like this
>> should be part of Zookeeper itself -- a more capable and easier-to-use
>> client.
>>
>> So I question the incubation of this. Why do we want to build a
>> new/separate project? Why isn't this just part of Zookeeper right from
>> the start?
>>
>> I would suggest that this work is placed on a branch within Zookeeper.
>> That Jordan and Jay become committers on that branch (not necessarily
>> Zookeeper trunk). Over time, the branch can be folded into trunk,
>> along with all the various tests, doc, and other artifacts that I see
>> in the GitHub repository. And hopefully that Jordan and Jay become
>> regular committers (and PMC members!) of the Zookeeper project itself.
>>
>> The current Zookeeper client can remain for backwards compat, and the
>> Curator work can become the next-gen client.
>>
>> Honestly, in my opnion, incubating this project seems like it would
>> create a distinct community, and really doesn't seem like it would
>> serve the Zookeeper community.
>>
>> All that said, I am not familiar with the Zookeeper or Curator
>> communities. But from this read, I don't think Incubation is the right
>> approach. I would rather push for a more direct incorporation of
>> Curator directly into Zookeeper. (use the short-form IP clearance) ...
>> so, unless somebody can help me understand the communities and
>> situation better, I'm -1 (binding) on this incubation. I'd rather see
>> combined, rather than distinct, communities from the start.
>>
>> Thanks,
>> -g
>>
>> On Mon, Feb 25, 2013 at 8:14 PM, Jordan Zimmerman
>> <jordan@jordanzimmerman.com> wrote:
>> > Hello,
>> >
>> > I would like to propose that Curator to be an Apache Incubator project.
>> >
>> > The proposal can be found here:
>> http://wiki.apache.org/incubator/CuratorProposal
>> >
>> > I have included the contents of the proposal below.
>> >
>> > Sincerely,
>> >
>> > Jordan Zimmerman
>> >
>> > ===================
>> >
>> > = Curator - ZooKeeper client wrapper and rich ZooKeeper framework =
>> >
>> > == Abstract ==
>> >
>> > Curator is a set of Java libraries that make using Apache ZooKeeper much
>> easier. While ZooKeeper comes bundled with a Java client, using the client
>> is non-trivial and error prone.
>> >
>> > == Proposal ==
>> >
>> > Curator is a set of Java libraries that make using Apache ZooKeeper much
>> easier. While ZooKeeper comes bundled with a Java client, using the client
>> is non-trivial and error prone. It consists of three components that build
>> on each other. Curator Client is a replacement for the bundled ZooKeeper
>> class that takes care of some low-level housekeeping and provides some
>> useful utilities. Curator Framework is a high-level API that greatly
>> simplifies using ZooKeeper. It adds many features that build on ZooKeeper
>> and handles the complexity of managing connections to the ZooKeeper cluster
>> and retrying operations. Curator Recipes consists of implementations of
>> some of the common ZooKeeper “recipes”. Additionally, Curator Test is
>> included which includes utilities to help with unit testing ZooKeeper-based
>> applications.
>> >
>> > == Background ==
>> >
>> > Curator was initially developed by Netflix to make writing
>> ZooKeeper-based applications easier and more reliable. Curator was
>> open-sourced by Netflix on !GitHub as an Apache 2.0 licensed project in
>> July 2011. During this time Curator has been formally released many times
>> and has gained widespread adoption.
>> >
>> > == Rationale ==
>> >
>> > New users of ZooKeeper are surprised to learn that a significant amount
>> of connection management must be done manually. For example, when the
>> ZooKeeper client connects to the ensemble it must negotiate a new session,
>> etc. This takes some time. If you use a ZooKeeper client API before the
>> connection process has completed, ZooKeeper will throw an exception. These
>> types of exceptions are referred to as “recoverable” errors.
>> > Curator automatically handles connection management, greatly simplifying
>> client code. Instead of directly using the ZooKeeper APIs you use Curator
>> APIs that internally check for connection completion and wrap each
>> ZooKeeper API in a retry loop. Curator uses a retry mechanism to handle
>> recoverable errors and automatically retry operations. The method of retry
>> is customizable. Curator comes bundled with several implementations
>> (ExponentialBackoffRetry, etc.) or custom implementations can be written.
>> >
>> > The ZooKeeper documentation describes many possible uses for ZooKeeper
>> calling each a “recipe”. While the distribution comes bundled with a few
>> implementations of these recipes, most ZooKeeper users will need to
>> manually implement one or more of the recipes. Implementing a ZooKeeper
>> recipe is not trivial. Besides the connection handling issues, there are
>> numerous edge cases that are not well documented that must be considered.
>> For example, many recipes require that an ephemeral-sequential node be
>> created. New users of ZooKeeper will not know that there is an edge case in
>> ephemeral-sequential node creation that requires you to put a special
>> “marker” in the node’s name so that you can search for the created node if
>> an I/O failure occurs. This is but one of many edge cases that are not well
>> documented but are handled by Curator.
>> >
>> > = Current Status =
>> >
>> > == Meritocracy ==
>> >
>> > Curator was initially developed by Jordan Zimmerman in 2011 at Netflix.
>> Developers external to Netflix provided feedback, suggested features and
>> fixes and implemented extensions of Curator. Netflix's engineering team has
>> since maintained the project and has been dedicated towards its
>> improvement. Contributors to Curator include developers from multiple
>> organizations around the world. Curator will be a meritocracy as it enters
>> the Incubator and beyond.
>> >
>> > == Community ==
>> >
>> > Curator is currently used by a number of organizations all over the
>> world. Curator has an active and growing user and developer community with
>> active participation in the [[http://groups.google.com/group/curator-users]]
>> mailing list and at its !Github home: [[
>> https://github.com/Netflix/curator]].
>> >
>> > Since open sourcing the project, there have been fifteen individuals
>> from various organizations who have contributed code.
>> >
>> > == Core Developers ==
>> >
>> > The core developers for Curator are:
>> >  * Jordan Zimmerman
>> >  * Jay Zarfoss
>> >
>> > Jordan has contributed towards Apache ZooKeeper and both Jordan and Jay
>> are familiar with Apache principles and philosophy for community driven
>> software development.
>> >
>> > == Alignment ==
>> >
>> > Curator is a natural complement for Apache ZooKeeper. Java users of
>> ZooKeeper will naturally want to use Curator. When Curator graduates from
>> Incubator it may be useful to distribute Curator artifacts as part of
>> ZooKeeper releases as the preferred/recommended client side library.
>> Further, at graduation a determination can be made as to whether Curator
>> should become a Top Level Project or be merged into ZooKeeper itself.
>> >
>> > = Known Risks =
>> >
>> > == Orphaned Products ==
>> >
>> > Curator is already deployed in production at multiple companies and they
>> are actively participating in creating new features. Curator is getting
>> traction with developers and thus the risks of it being orphaned are
>> minimal.
>> >
>> > == Inexperience with Open Source ==
>> >
>> > All code developed for Curator has been open sourced by Netflix under
>> Apache 2.0 license.  All committers to Curator are intimately familiar with
>> the Apache model for open-source development and are experienced with
>> working with new contributors.
>> >
>> > == Homogeneous Developers ==
>> >
>> > The initial committers are from a single organization. However, we
>> expect that once approved for incubation, the project will attract new
>> contributors from diverse organizations and will thus grow organically. The
>> submission of patches from developers from several different organizations
>> is a strong indication that Curator will be widely adopted.
>> >
>> > == Reliance on Salaried Developers ==
>> >
>> > It is expected that Curator will be developed on salaried and volunteer
>> time, although all of the initial developers will work on it mainly on
>> salaried time.
>> >
>> > == Relationships with Other Apache Products ==
>> >
>> > Curator depends upon other Apache Projects: Apache ZooKeeper, Apache
>> Log4J, and multiple Apache Commons components. Its build depends upon
>> Apache Maven. Notably, there is interest from other Apache Projects such as
>> HBase in adopting Curator as the client library for ZooKeeper. Apache James
>> Mailbox has already incorporated Curator.
>> >
>> > == An Excessive Fascination with the Apache Brand ==
>> >
>> > We would like Curator to become an Apache project to further foster a
>> healthy community of contributors and consumers around the project.  Since
>> Curator directly interacts with Apache ZooKeeper and solves an important
>> problem of many ZooKeeper users, residing in the Apache Software Foundation
>> will increase interaction with the larger community.
>> >
>> > = Documentation =
>> >
>> >  * Curator wiki at GitHub: https://github.com/Netflix/curator/wiki
>> >  * Curator issues at GitHub: https://github.com/Netflix/curator/issues
>> >  * Curator javadoc at GitHub: http://netflix.github.com/curator/doc/
>> >
>> > = Initial Source =
>> >
>> >  * git://github.com/Netflix/curator.git
>> >
>> > == Source and Intellectual Property Submission Plan ==
>> >
>> >  * The initial source is already licensed under the Apache License,
>> Version 2.0. https://github.com/Netflix/curator/blob/master/LICENSE.txt
>> >
>> > == External Dependencies ==
>> >
>> > The required external dependencies are all Apache License or compatible
>> licenses. Following components with non-Apache licenses are enumerated:
>> >
>> >  * org.slf4j: MIT-like License
>> >  * org.mockito: MIT-like License
>> >
>> > == Cryptography ==
>> >
>> > Curator contains no known cryptography.
>> >
>> > = Required Resources =
>> >
>> > == Mailing lists ==
>> >
>> >  * curator-private (with moderated subscriptions)
>> >  * curator-dev
>> >  * curator-commits
>> >  * curator-user
>> >
>> > == Github Repositories ==
>> >
>> > http://github.com/apache/curator git://git.apache.org/curator.git
>> >
>> > == Issue Tracking ==
>> >
>> > JIRA Curator (CURATOR)
>> >
>> > == Other Resources ==
>> >
>> > The existing code already has unit and integration tests so we would
>> like a Jenkins instance to run them whenever a new patch is submitted. This
>> can be added after project creation.
>> >
>> > = Initial Committers =
>> >
>> >  * Jordan Zimmerman (jzimmerman at netflix dot com)
>> >  * Jay Zarfoss (jzarfoss at netflix dot com)
>> >
>> > = Affiliations =
>> >
>> >  * Jordan Zimmerman, Netflix
>> >  * Jay Zarfoss, Netflix
>> >
>> > = Sponsors =
>> >
>> > == Champion ==
>> >
>> >  * Patrick Hunt
>> >
>> > == Nominated Mentors ==
>> >
>> >  * Patrick Hunt
>> >  * Enis Söztutar
>> >  * Mahadev Konar
>> >
>> > == Sponsoring Entity ==
>> >
>> >  * Apache Incubator PMC
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: general-help@incubator.apache.org
>>
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org


Mime
View raw message