incubator-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Benson Margulies <bimargul...@gmail.com>
Subject Re: [PROPOSAL] Curator for the Apache Incubator
Date Tue, 26 Feb 2013 17:40:21 GMT
On Tue, Feb 26, 2013 at 12:22 PM, Patrick Hunt <phunt@apache.org> wrote:
> On Tue, Feb 26, 2013 at 8:32 AM, Benson Margulies <bimargulies@gmail.com> wrote:
>> If you think that the right destination for curator is as part of ZK,
>> then it would be good to see substantive participation of the ZK PMC
>> in the incubation. The goal should be to 'graduate' by having the
>> curator community be granted karma at ZK and the code folded in. This
>> would require, I think, the ZK community to supply at least one
>> mentor, and to have a plan in advance for the eventual votes based on
>> success in the incubator.
>
> Hi Benson. What you're suggesting matches my current thinking.
>
> The three Curator mentors are as follows: Mahadev and I (champion) are
> both PMC members on ZK, Enis Söztutar is active on HBase and
> interested in using Curator for that project (which already uses ZK
> heavily).

OK, that's lovely.


>
> Patrick
>
>> On Tue, Feb 26, 2013 at 10:53 AM, Henry Saputra <henry.saputra@gmail.com> wrote:
>>> Ah no, I was not suggesting about Curator to become subproject of ZK. I
>>> just afraid that if Curator is going as incubator it will end up as sub of
>>> ZK as merging process.
>>>
>>> Like Greg has mentioned in another reply, I would prefer Curator to be
>>> merged as a higher level ZK client. Surely project like HBase and others
>>> that relying on ZK would appreciate simpler client to ZK.
>>>
>>> Thanks,
>>>
>>> Henry
>>>
>>>
>>> On Mon, Feb 25, 2013 at 11:23 PM, Patrick Hunt <phunt@apache.org> wrote:
>>>
>>>> On Mon, Feb 25, 2013 at 11:01 PM, Henry Saputra <henry.saputra@gmail.com>
>>>> wrote:
>>>> > So isnt this similar to HCatalog which relying on Hive metadata service
>>>> > that ends up as sub project of Apache Hive?
>>>> >
>>>>
>>>> I was against having Curator as a sub when it came up on the original
>>>> discussion thread, I still am.
>>>>
>>>> Patrick
>>>>
>>>> >
>>>> >
>>>> > On Mon, Feb 25, 2013 at 7:10 PM, Greg Stein <gstein@gmail.com>
wrote:
>>>> >
>>>> >> My concern is that we're looking at two "new" committers, rather
than
>>>> >> a Curator community. Following normal Incubator work, Curator would
>>>> >> build a community for itself. But then we'd have a community
>>>> >> *distinct* from that of Zookeeper. And it really looks like this
>>>> >> should be part of Zookeeper itself -- a more capable and easier-to-use
>>>> >> client.
>>>> >>
>>>> >> So I question the incubation of this. Why do we want to build a
>>>> >> new/separate project? Why isn't this just part of Zookeeper right
from
>>>> >> the start?
>>>> >>
>>>> >> I would suggest that this work is placed on a branch within Zookeeper.
>>>> >> That Jordan and Jay become committers on that branch (not necessarily
>>>> >> Zookeeper trunk). Over time, the branch can be folded into trunk,
>>>> >> along with all the various tests, doc, and other artifacts that
I see
>>>> >> in the GitHub repository. And hopefully that Jordan and Jay become
>>>> >> regular committers (and PMC members!) of the Zookeeper project itself.
>>>> >>
>>>> >> The current Zookeeper client can remain for backwards compat, and
the
>>>> >> Curator work can become the next-gen client.
>>>> >>
>>>> >> Honestly, in my opnion, incubating this project seems like it would
>>>> >> create a distinct community, and really doesn't seem like it would
>>>> >> serve the Zookeeper community.
>>>> >>
>>>> >> All that said, I am not familiar with the Zookeeper or Curator
>>>> >> communities. But from this read, I don't think Incubation is the
right
>>>> >> approach. I would rather push for a more direct incorporation of
>>>> >> Curator directly into Zookeeper. (use the short-form IP clearance)
...
>>>> >> so, unless somebody can help me understand the communities and
>>>> >> situation better, I'm -1 (binding) on this incubation. I'd rather
see
>>>> >> combined, rather than distinct, communities from the start.
>>>> >>
>>>> >> Thanks,
>>>> >> -g
>>>> >>
>>>> >> On Mon, Feb 25, 2013 at 8:14 PM, Jordan Zimmerman
>>>> >> <jordan@jordanzimmerman.com> wrote:
>>>> >> > Hello,
>>>> >> >
>>>> >> > I would like to propose that Curator to be an Apache Incubator
>>>> project.
>>>> >> >
>>>> >> > The proposal can be found here:
>>>> >> http://wiki.apache.org/incubator/CuratorProposal
>>>> >> >
>>>> >> > I have included the contents of the proposal below.
>>>> >> >
>>>> >> > Sincerely,
>>>> >> >
>>>> >> > Jordan Zimmerman
>>>> >> >
>>>> >> > ===================
>>>> >> >
>>>> >> > = Curator - ZooKeeper client wrapper and rich ZooKeeper framework
=
>>>> >> >
>>>> >> > == Abstract ==
>>>> >> >
>>>> >> > Curator is a set of Java libraries that make using Apache ZooKeeper
>>>> much
>>>> >> easier. While ZooKeeper comes bundled with a Java client, using
the
>>>> client
>>>> >> is non-trivial and error prone.
>>>> >> >
>>>> >> > == Proposal ==
>>>> >> >
>>>> >> > Curator is a set of Java libraries that make using Apache ZooKeeper
>>>> much
>>>> >> easier. While ZooKeeper comes bundled with a Java client, using
the
>>>> client
>>>> >> is non-trivial and error prone. It consists of three components
that
>>>> build
>>>> >> on each other. Curator Client is a replacement for the bundled ZooKeeper
>>>> >> class that takes care of some low-level housekeeping and provides
some
>>>> >> useful utilities. Curator Framework is a high-level API that greatly
>>>> >> simplifies using ZooKeeper. It adds many features that build on
>>>> ZooKeeper
>>>> >> and handles the complexity of managing connections to the ZooKeeper
>>>> cluster
>>>> >> and retrying operations. Curator Recipes consists of implementations
of
>>>> >> some of the common ZooKeeper “recipes”. Additionally, Curator
Test is
>>>> >> included which includes utilities to help with unit testing
>>>> ZooKeeper-based
>>>> >> applications.
>>>> >> >
>>>> >> > == Background ==
>>>> >> >
>>>> >> > Curator was initially developed by Netflix to make writing
>>>> >> ZooKeeper-based applications easier and more reliable. Curator was
>>>> >> open-sourced by Netflix on !GitHub as an Apache 2.0 licensed project
in
>>>> >> July 2011. During this time Curator has been formally released many
>>>> times
>>>> >> and has gained widespread adoption.
>>>> >> >
>>>> >> > == Rationale ==
>>>> >> >
>>>> >> > New users of ZooKeeper are surprised to learn that a significant
>>>> amount
>>>> >> of connection management must be done manually. For example, when
the
>>>> >> ZooKeeper client connects to the ensemble it must negotiate a new
>>>> session,
>>>> >> etc. This takes some time. If you use a ZooKeeper client API before
the
>>>> >> connection process has completed, ZooKeeper will throw an exception.
>>>> These
>>>> >> types of exceptions are referred to as “recoverable” errors.
>>>> >> > Curator automatically handles connection management, greatly
>>>> simplifying
>>>> >> client code. Instead of directly using the ZooKeeper APIs you use
>>>> Curator
>>>> >> APIs that internally check for connection completion and wrap each
>>>> >> ZooKeeper API in a retry loop. Curator uses a retry mechanism to
handle
>>>> >> recoverable errors and automatically retry operations. The method
of
>>>> retry
>>>> >> is customizable. Curator comes bundled with several implementations
>>>> >> (ExponentialBackoffRetry, etc.) or custom implementations can be
>>>> written.
>>>> >> >
>>>> >> > The ZooKeeper documentation describes many possible uses for
ZooKeeper
>>>> >> calling each a “recipe”. While the distribution comes bundled
with a few
>>>> >> implementations of these recipes, most ZooKeeper users will need
to
>>>> >> manually implement one or more of the recipes. Implementing a ZooKeeper
>>>> >> recipe is not trivial. Besides the connection handling issues, there
are
>>>> >> numerous edge cases that are not well documented that must be
>>>> considered.
>>>> >> For example, many recipes require that an ephemeral-sequential node
be
>>>> >> created. New users of ZooKeeper will not know that there is an edge
>>>> case in
>>>> >> ephemeral-sequential node creation that requires you to put a special
>>>> >> “marker” in the node’s name so that you can search for the
created node
>>>> if
>>>> >> an I/O failure occurs. This is but one of many edge cases that are
not
>>>> well
>>>> >> documented but are handled by Curator.
>>>> >> >
>>>> >> > = Current Status =
>>>> >> >
>>>> >> > == Meritocracy ==
>>>> >> >
>>>> >> > Curator was initially developed by Jordan Zimmerman in 2011
at
>>>> Netflix.
>>>> >> Developers external to Netflix provided feedback, suggested features
and
>>>> >> fixes and implemented extensions of Curator. Netflix's engineering
team
>>>> has
>>>> >> since maintained the project and has been dedicated towards its
>>>> >> improvement. Contributors to Curator include developers from multiple
>>>> >> organizations around the world. Curator will be a meritocracy as
it
>>>> enters
>>>> >> the Incubator and beyond.
>>>> >> >
>>>> >> > == Community ==
>>>> >> >
>>>> >> > Curator is currently used by a number of organizations all
over the
>>>> >> world. Curator has an active and growing user and developer community
>>>> with
>>>> >> active participation in the [[
>>>> http://groups.google.com/group/curator-users]]
>>>> >> mailing list and at its !Github home: [[
>>>> >> https://github.com/Netflix/curator]].
>>>> >> >
>>>> >> > Since open sourcing the project, there have been fifteen individuals
>>>> >> from various organizations who have contributed code.
>>>> >> >
>>>> >> > == Core Developers ==
>>>> >> >
>>>> >> > The core developers for Curator are:
>>>> >> >  * Jordan Zimmerman
>>>> >> >  * Jay Zarfoss
>>>> >> >
>>>> >> > Jordan has contributed towards Apache ZooKeeper and both Jordan
and
>>>> Jay
>>>> >> are familiar with Apache principles and philosophy for community
driven
>>>> >> software development.
>>>> >> >
>>>> >> > == Alignment ==
>>>> >> >
>>>> >> > Curator is a natural complement for Apache ZooKeeper. Java
users of
>>>> >> ZooKeeper will naturally want to use Curator. When Curator graduates
>>>> from
>>>> >> Incubator it may be useful to distribute Curator artifacts as part
of
>>>> >> ZooKeeper releases as the preferred/recommended client side library.
>>>> >> Further, at graduation a determination can be made as to whether
Curator
>>>> >> should become a Top Level Project or be merged into ZooKeeper itself.
>>>> >> >
>>>> >> > = Known Risks =
>>>> >> >
>>>> >> > == Orphaned Products ==
>>>> >> >
>>>> >> > Curator is already deployed in production at multiple companies
and
>>>> they
>>>> >> are actively participating in creating new features. Curator is
getting
>>>> >> traction with developers and thus the risks of it being orphaned
are
>>>> >> minimal.
>>>> >> >
>>>> >> > == Inexperience with Open Source ==
>>>> >> >
>>>> >> > All code developed for Curator has been open sourced by Netflix
under
>>>> >> Apache 2.0 license.  All committers to Curator are intimately familiar
>>>> with
>>>> >> the Apache model for open-source development and are experienced
with
>>>> >> working with new contributors.
>>>> >> >
>>>> >> > == Homogeneous Developers ==
>>>> >> >
>>>> >> > The initial committers are from a single organization. However,
we
>>>> >> expect that once approved for incubation, the project will attract
new
>>>> >> contributors from diverse organizations and will thus grow organically.
>>>> The
>>>> >> submission of patches from developers from several different
>>>> organizations
>>>> >> is a strong indication that Curator will be widely adopted.
>>>> >> >
>>>> >> > == Reliance on Salaried Developers ==
>>>> >> >
>>>> >> > It is expected that Curator will be developed on salaried and
>>>> volunteer
>>>> >> time, although all of the initial developers will work on it mainly
on
>>>> >> salaried time.
>>>> >> >
>>>> >> > == Relationships with Other Apache Products ==
>>>> >> >
>>>> >> > Curator depends upon other Apache Projects: Apache ZooKeeper,
Apache
>>>> >> Log4J, and multiple Apache Commons components. Its build depends
upon
>>>> >> Apache Maven. Notably, there is interest from other Apache Projects
>>>> such as
>>>> >> HBase in adopting Curator as the client library for ZooKeeper. Apache
>>>> James
>>>> >> Mailbox has already incorporated Curator.
>>>> >> >
>>>> >> > == An Excessive Fascination with the Apache Brand ==
>>>> >> >
>>>> >> > We would like Curator to become an Apache project to further
foster a
>>>> >> healthy community of contributors and consumers around the project.
>>>>  Since
>>>> >> Curator directly interacts with Apache ZooKeeper and solves an important
>>>> >> problem of many ZooKeeper users, residing in the Apache Software
>>>> Foundation
>>>> >> will increase interaction with the larger community.
>>>> >> >
>>>> >> > = Documentation =
>>>> >> >
>>>> >> >  * Curator wiki at GitHub: https://github.com/Netflix/curator/wiki
>>>> >> >  * Curator issues at GitHub:
>>>> https://github.com/Netflix/curator/issues
>>>> >> >  * Curator javadoc at GitHub: http://netflix.github.com/curator/doc/
>>>> >> >
>>>> >> > = Initial Source =
>>>> >> >
>>>> >> >  * git://github.com/Netflix/curator.git
>>>> >> >
>>>> >> > == Source and Intellectual Property Submission Plan ==
>>>> >> >
>>>> >> >  * The initial source is already licensed under the Apache
License,
>>>> >> Version 2.0. https://github.com/Netflix/curator/blob/master/LICENSE.txt
>>>> >> >
>>>> >> > == External Dependencies ==
>>>> >> >
>>>> >> > The required external dependencies are all Apache License or
>>>> compatible
>>>> >> licenses. Following components with non-Apache licenses are enumerated:
>>>> >> >
>>>> >> >  * org.slf4j: MIT-like License
>>>> >> >  * org.mockito: MIT-like License
>>>> >> >
>>>> >> > == Cryptography ==
>>>> >> >
>>>> >> > Curator contains no known cryptography.
>>>> >> >
>>>> >> > = Required Resources =
>>>> >> >
>>>> >> > == Mailing lists ==
>>>> >> >
>>>> >> >  * curator-private (with moderated subscriptions)
>>>> >> >  * curator-dev
>>>> >> >  * curator-commits
>>>> >> >  * curator-user
>>>> >> >
>>>> >> > == Github Repositories ==
>>>> >> >
>>>> >> > http://github.com/apache/curator git://git.apache.org/curator.git
>>>> >> >
>>>> >> > == Issue Tracking ==
>>>> >> >
>>>> >> > JIRA Curator (CURATOR)
>>>> >> >
>>>> >> > == Other Resources ==
>>>> >> >
>>>> >> > The existing code already has unit and integration tests so
we would
>>>> >> like a Jenkins instance to run them whenever a new patch is submitted.
>>>> This
>>>> >> can be added after project creation.
>>>> >> >
>>>> >> > = Initial Committers =
>>>> >> >
>>>> >> >  * Jordan Zimmerman (jzimmerman at netflix dot com)
>>>> >> >  * Jay Zarfoss (jzarfoss at netflix dot com)
>>>> >> >
>>>> >> > = Affiliations =
>>>> >> >
>>>> >> >  * Jordan Zimmerman, Netflix
>>>> >> >  * Jay Zarfoss, Netflix
>>>> >> >
>>>> >> > = Sponsors =
>>>> >> >
>>>> >> > == Champion ==
>>>> >> >
>>>> >> >  * Patrick Hunt
>>>> >> >
>>>> >> > == Nominated Mentors ==
>>>> >> >
>>>> >> >  * Patrick Hunt
>>>> >> >  * Enis Söztutar
>>>> >> >  * Mahadev Konar
>>>> >> >
>>>> >> > == Sponsoring Entity ==
>>>> >> >
>>>> >> >  * Apache Incubator PMC
>>>> >>
>>>> >> ---------------------------------------------------------------------
>>>> >> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>>> >> For additional commands, e-mail: general-help@incubator.apache.org
>>>> >>
>>>> >>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>>> For additional commands, e-mail: general-help@incubator.apache.org
>>>>
>>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: general-help@incubator.apache.org
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
> For additional commands, e-mail: general-help@incubator.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org


Mime
View raw message