incubator-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Hunt <ph...@apache.org>
Subject Re: [PROPOSAL] Curator for the Apache Incubator
Date Tue, 26 Feb 2013 17:22:53 GMT
On Tue, Feb 26, 2013 at 8:32 AM, Benson Margulies <bimargulies@gmail.com> wrote:
> If you think that the right destination for curator is as part of ZK,
> then it would be good to see substantive participation of the ZK PMC
> in the incubation. The goal should be to 'graduate' by having the
> curator community be granted karma at ZK and the code folded in. This
> would require, I think, the ZK community to supply at least one
> mentor, and to have a plan in advance for the eventual votes based on
> success in the incubator.

Hi Benson. What you're suggesting matches my current thinking.

The three Curator mentors are as follows: Mahadev and I (champion) are
both PMC members on ZK, Enis Söztutar is active on HBase and
interested in using Curator for that project (which already uses ZK
heavily).

Patrick

> On Tue, Feb 26, 2013 at 10:53 AM, Henry Saputra <henry.saputra@gmail.com> wrote:
>> Ah no, I was not suggesting about Curator to become subproject of ZK. I
>> just afraid that if Curator is going as incubator it will end up as sub of
>> ZK as merging process.
>>
>> Like Greg has mentioned in another reply, I would prefer Curator to be
>> merged as a higher level ZK client. Surely project like HBase and others
>> that relying on ZK would appreciate simpler client to ZK.
>>
>> Thanks,
>>
>> Henry
>>
>>
>> On Mon, Feb 25, 2013 at 11:23 PM, Patrick Hunt <phunt@apache.org> wrote:
>>
>>> On Mon, Feb 25, 2013 at 11:01 PM, Henry Saputra <henry.saputra@gmail.com>
>>> wrote:
>>> > So isnt this similar to HCatalog which relying on Hive metadata service
>>> > that ends up as sub project of Apache Hive?
>>> >
>>>
>>> I was against having Curator as a sub when it came up on the original
>>> discussion thread, I still am.
>>>
>>> Patrick
>>>
>>> >
>>> >
>>> > On Mon, Feb 25, 2013 at 7:10 PM, Greg Stein <gstein@gmail.com> wrote:
>>> >
>>> >> My concern is that we're looking at two "new" committers, rather than
>>> >> a Curator community. Following normal Incubator work, Curator would
>>> >> build a community for itself. But then we'd have a community
>>> >> *distinct* from that of Zookeeper. And it really looks like this
>>> >> should be part of Zookeeper itself -- a more capable and easier-to-use
>>> >> client.
>>> >>
>>> >> So I question the incubation of this. Why do we want to build a
>>> >> new/separate project? Why isn't this just part of Zookeeper right from
>>> >> the start?
>>> >>
>>> >> I would suggest that this work is placed on a branch within Zookeeper.
>>> >> That Jordan and Jay become committers on that branch (not necessarily
>>> >> Zookeeper trunk). Over time, the branch can be folded into trunk,
>>> >> along with all the various tests, doc, and other artifacts that I see
>>> >> in the GitHub repository. And hopefully that Jordan and Jay become
>>> >> regular committers (and PMC members!) of the Zookeeper project itself.
>>> >>
>>> >> The current Zookeeper client can remain for backwards compat, and the
>>> >> Curator work can become the next-gen client.
>>> >>
>>> >> Honestly, in my opnion, incubating this project seems like it would
>>> >> create a distinct community, and really doesn't seem like it would
>>> >> serve the Zookeeper community.
>>> >>
>>> >> All that said, I am not familiar with the Zookeeper or Curator
>>> >> communities. But from this read, I don't think Incubation is the right
>>> >> approach. I would rather push for a more direct incorporation of
>>> >> Curator directly into Zookeeper. (use the short-form IP clearance) ...
>>> >> so, unless somebody can help me understand the communities and
>>> >> situation better, I'm -1 (binding) on this incubation. I'd rather see
>>> >> combined, rather than distinct, communities from the start.
>>> >>
>>> >> Thanks,
>>> >> -g
>>> >>
>>> >> On Mon, Feb 25, 2013 at 8:14 PM, Jordan Zimmerman
>>> >> <jordan@jordanzimmerman.com> wrote:
>>> >> > Hello,
>>> >> >
>>> >> > I would like to propose that Curator to be an Apache Incubator
>>> project.
>>> >> >
>>> >> > The proposal can be found here:
>>> >> http://wiki.apache.org/incubator/CuratorProposal
>>> >> >
>>> >> > I have included the contents of the proposal below.
>>> >> >
>>> >> > Sincerely,
>>> >> >
>>> >> > Jordan Zimmerman
>>> >> >
>>> >> > ===================
>>> >> >
>>> >> > = Curator - ZooKeeper client wrapper and rich ZooKeeper framework
=
>>> >> >
>>> >> > == Abstract ==
>>> >> >
>>> >> > Curator is a set of Java libraries that make using Apache ZooKeeper
>>> much
>>> >> easier. While ZooKeeper comes bundled with a Java client, using the
>>> client
>>> >> is non-trivial and error prone.
>>> >> >
>>> >> > == Proposal ==
>>> >> >
>>> >> > Curator is a set of Java libraries that make using Apache ZooKeeper
>>> much
>>> >> easier. While ZooKeeper comes bundled with a Java client, using the
>>> client
>>> >> is non-trivial and error prone. It consists of three components that
>>> build
>>> >> on each other. Curator Client is a replacement for the bundled ZooKeeper
>>> >> class that takes care of some low-level housekeeping and provides some
>>> >> useful utilities. Curator Framework is a high-level API that greatly
>>> >> simplifies using ZooKeeper. It adds many features that build on
>>> ZooKeeper
>>> >> and handles the complexity of managing connections to the ZooKeeper
>>> cluster
>>> >> and retrying operations. Curator Recipes consists of implementations
of
>>> >> some of the common ZooKeeper “recipes”. Additionally, Curator Test
is
>>> >> included which includes utilities to help with unit testing
>>> ZooKeeper-based
>>> >> applications.
>>> >> >
>>> >> > == Background ==
>>> >> >
>>> >> > Curator was initially developed by Netflix to make writing
>>> >> ZooKeeper-based applications easier and more reliable. Curator was
>>> >> open-sourced by Netflix on !GitHub as an Apache 2.0 licensed project
in
>>> >> July 2011. During this time Curator has been formally released many
>>> times
>>> >> and has gained widespread adoption.
>>> >> >
>>> >> > == Rationale ==
>>> >> >
>>> >> > New users of ZooKeeper are surprised to learn that a significant
>>> amount
>>> >> of connection management must be done manually. For example, when the
>>> >> ZooKeeper client connects to the ensemble it must negotiate a new
>>> session,
>>> >> etc. This takes some time. If you use a ZooKeeper client API before
the
>>> >> connection process has completed, ZooKeeper will throw an exception.
>>> These
>>> >> types of exceptions are referred to as “recoverable” errors.
>>> >> > Curator automatically handles connection management, greatly
>>> simplifying
>>> >> client code. Instead of directly using the ZooKeeper APIs you use
>>> Curator
>>> >> APIs that internally check for connection completion and wrap each
>>> >> ZooKeeper API in a retry loop. Curator uses a retry mechanism to handle
>>> >> recoverable errors and automatically retry operations. The method of
>>> retry
>>> >> is customizable. Curator comes bundled with several implementations
>>> >> (ExponentialBackoffRetry, etc.) or custom implementations can be
>>> written.
>>> >> >
>>> >> > The ZooKeeper documentation describes many possible uses for ZooKeeper
>>> >> calling each a “recipe”. While the distribution comes bundled with
a few
>>> >> implementations of these recipes, most ZooKeeper users will need to
>>> >> manually implement one or more of the recipes. Implementing a ZooKeeper
>>> >> recipe is not trivial. Besides the connection handling issues, there
are
>>> >> numerous edge cases that are not well documented that must be
>>> considered.
>>> >> For example, many recipes require that an ephemeral-sequential node
be
>>> >> created. New users of ZooKeeper will not know that there is an edge
>>> case in
>>> >> ephemeral-sequential node creation that requires you to put a special
>>> >> “marker” in the node’s name so that you can search for the created
node
>>> if
>>> >> an I/O failure occurs. This is but one of many edge cases that are not
>>> well
>>> >> documented but are handled by Curator.
>>> >> >
>>> >> > = Current Status =
>>> >> >
>>> >> > == Meritocracy ==
>>> >> >
>>> >> > Curator was initially developed by Jordan Zimmerman in 2011 at
>>> Netflix.
>>> >> Developers external to Netflix provided feedback, suggested features
and
>>> >> fixes and implemented extensions of Curator. Netflix's engineering team
>>> has
>>> >> since maintained the project and has been dedicated towards its
>>> >> improvement. Contributors to Curator include developers from multiple
>>> >> organizations around the world. Curator will be a meritocracy as it
>>> enters
>>> >> the Incubator and beyond.
>>> >> >
>>> >> > == Community ==
>>> >> >
>>> >> > Curator is currently used by a number of organizations all over
the
>>> >> world. Curator has an active and growing user and developer community
>>> with
>>> >> active participation in the [[
>>> http://groups.google.com/group/curator-users]]
>>> >> mailing list and at its !Github home: [[
>>> >> https://github.com/Netflix/curator]].
>>> >> >
>>> >> > Since open sourcing the project, there have been fifteen individuals
>>> >> from various organizations who have contributed code.
>>> >> >
>>> >> > == Core Developers ==
>>> >> >
>>> >> > The core developers for Curator are:
>>> >> >  * Jordan Zimmerman
>>> >> >  * Jay Zarfoss
>>> >> >
>>> >> > Jordan has contributed towards Apache ZooKeeper and both Jordan
and
>>> Jay
>>> >> are familiar with Apache principles and philosophy for community driven
>>> >> software development.
>>> >> >
>>> >> > == Alignment ==
>>> >> >
>>> >> > Curator is a natural complement for Apache ZooKeeper. Java users
of
>>> >> ZooKeeper will naturally want to use Curator. When Curator graduates
>>> from
>>> >> Incubator it may be useful to distribute Curator artifacts as part of
>>> >> ZooKeeper releases as the preferred/recommended client side library.
>>> >> Further, at graduation a determination can be made as to whether Curator
>>> >> should become a Top Level Project or be merged into ZooKeeper itself.
>>> >> >
>>> >> > = Known Risks =
>>> >> >
>>> >> > == Orphaned Products ==
>>> >> >
>>> >> > Curator is already deployed in production at multiple companies
and
>>> they
>>> >> are actively participating in creating new features. Curator is getting
>>> >> traction with developers and thus the risks of it being orphaned are
>>> >> minimal.
>>> >> >
>>> >> > == Inexperience with Open Source ==
>>> >> >
>>> >> > All code developed for Curator has been open sourced by Netflix
under
>>> >> Apache 2.0 license.  All committers to Curator are intimately familiar
>>> with
>>> >> the Apache model for open-source development and are experienced with
>>> >> working with new contributors.
>>> >> >
>>> >> > == Homogeneous Developers ==
>>> >> >
>>> >> > The initial committers are from a single organization. However,
we
>>> >> expect that once approved for incubation, the project will attract new
>>> >> contributors from diverse organizations and will thus grow organically.
>>> The
>>> >> submission of patches from developers from several different
>>> organizations
>>> >> is a strong indication that Curator will be widely adopted.
>>> >> >
>>> >> > == Reliance on Salaried Developers ==
>>> >> >
>>> >> > It is expected that Curator will be developed on salaried and
>>> volunteer
>>> >> time, although all of the initial developers will work on it mainly
on
>>> >> salaried time.
>>> >> >
>>> >> > == Relationships with Other Apache Products ==
>>> >> >
>>> >> > Curator depends upon other Apache Projects: Apache ZooKeeper, Apache
>>> >> Log4J, and multiple Apache Commons components. Its build depends upon
>>> >> Apache Maven. Notably, there is interest from other Apache Projects
>>> such as
>>> >> HBase in adopting Curator as the client library for ZooKeeper. Apache
>>> James
>>> >> Mailbox has already incorporated Curator.
>>> >> >
>>> >> > == An Excessive Fascination with the Apache Brand ==
>>> >> >
>>> >> > We would like Curator to become an Apache project to further foster
a
>>> >> healthy community of contributors and consumers around the project.
>>>  Since
>>> >> Curator directly interacts with Apache ZooKeeper and solves an important
>>> >> problem of many ZooKeeper users, residing in the Apache Software
>>> Foundation
>>> >> will increase interaction with the larger community.
>>> >> >
>>> >> > = Documentation =
>>> >> >
>>> >> >  * Curator wiki at GitHub: https://github.com/Netflix/curator/wiki
>>> >> >  * Curator issues at GitHub:
>>> https://github.com/Netflix/curator/issues
>>> >> >  * Curator javadoc at GitHub: http://netflix.github.com/curator/doc/
>>> >> >
>>> >> > = Initial Source =
>>> >> >
>>> >> >  * git://github.com/Netflix/curator.git
>>> >> >
>>> >> > == Source and Intellectual Property Submission Plan ==
>>> >> >
>>> >> >  * The initial source is already licensed under the Apache License,
>>> >> Version 2.0. https://github.com/Netflix/curator/blob/master/LICENSE.txt
>>> >> >
>>> >> > == External Dependencies ==
>>> >> >
>>> >> > The required external dependencies are all Apache License or
>>> compatible
>>> >> licenses. Following components with non-Apache licenses are enumerated:
>>> >> >
>>> >> >  * org.slf4j: MIT-like License
>>> >> >  * org.mockito: MIT-like License
>>> >> >
>>> >> > == Cryptography ==
>>> >> >
>>> >> > Curator contains no known cryptography.
>>> >> >
>>> >> > = Required Resources =
>>> >> >
>>> >> > == Mailing lists ==
>>> >> >
>>> >> >  * curator-private (with moderated subscriptions)
>>> >> >  * curator-dev
>>> >> >  * curator-commits
>>> >> >  * curator-user
>>> >> >
>>> >> > == Github Repositories ==
>>> >> >
>>> >> > http://github.com/apache/curator git://git.apache.org/curator.git
>>> >> >
>>> >> > == Issue Tracking ==
>>> >> >
>>> >> > JIRA Curator (CURATOR)
>>> >> >
>>> >> > == Other Resources ==
>>> >> >
>>> >> > The existing code already has unit and integration tests so we
would
>>> >> like a Jenkins instance to run them whenever a new patch is submitted.
>>> This
>>> >> can be added after project creation.
>>> >> >
>>> >> > = Initial Committers =
>>> >> >
>>> >> >  * Jordan Zimmerman (jzimmerman at netflix dot com)
>>> >> >  * Jay Zarfoss (jzarfoss at netflix dot com)
>>> >> >
>>> >> > = Affiliations =
>>> >> >
>>> >> >  * Jordan Zimmerman, Netflix
>>> >> >  * Jay Zarfoss, Netflix
>>> >> >
>>> >> > = Sponsors =
>>> >> >
>>> >> > == Champion ==
>>> >> >
>>> >> >  * Patrick Hunt
>>> >> >
>>> >> > == Nominated Mentors ==
>>> >> >
>>> >> >  * Patrick Hunt
>>> >> >  * Enis Söztutar
>>> >> >  * Mahadev Konar
>>> >> >
>>> >> > == Sponsoring Entity ==
>>> >> >
>>> >> >  * Apache Incubator PMC
>>> >>
>>> >> ---------------------------------------------------------------------
>>> >> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>> >> For additional commands, e-mail: general-help@incubator.apache.org
>>> >>
>>> >>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>> For additional commands, e-mail: general-help@incubator.apache.org
>>>
>>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org


Mime
View raw message