incubator-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mattmann, Chris A (388J)" <chris.a.mattm...@jpl.nasa.gov>
Subject Re: [PROPOSAL] Curator for the Apache Incubator
Date Tue, 26 Feb 2013 18:30:31 GMT
Hi Guys,

Sorry I have to ask the question here. If the mentors consist of PMC
members on ZK
(at least Pat and Mahadev), what's the problem with creating a branch in
ZK and just
having the code be there and getting the Curator proposed committers as
committers in
ZK ville, and hopefully PMC members soon thereafter.

I have to agree with Greg here. Seems like Incubation is something that
might not be
needed. If the end result of this after N months is that Curator
"graduates" into another
set of flipping bylaw updates, and more legislation that makes these
people unofficial 
PMC members on ZK, then I'm double triple -1 with 50 piles of coal on top.
Yes, I'm 
still remembering the HCatalog/Hive thing here - I still don't get that.

Either: (a) define Curator to be its own separate project/community, with
a goal of TLP; 
or (b) nix Incubation and just make these guys part of ZK, on a branch
now. 

Cheers,
Chris

On 2/26/13 9:40 AM, "Benson Margulies" <bimargulies@gmail.com> wrote:

>On Tue, Feb 26, 2013 at 12:22 PM, Patrick Hunt <phunt@apache.org> wrote:
>> On Tue, Feb 26, 2013 at 8:32 AM, Benson Margulies
>><bimargulies@gmail.com> wrote:
>>> If you think that the right destination for curator is as part of ZK,
>>> then it would be good to see substantive participation of the ZK PMC
>>> in the incubation. The goal should be to 'graduate' by having the
>>> curator community be granted karma at ZK and the code folded in. This
>>> would require, I think, the ZK community to supply at least one
>>> mentor, and to have a plan in advance for the eventual votes based on
>>> success in the incubator.
>>
>> Hi Benson. What you're suggesting matches my current thinking.
>>
>> The three Curator mentors are as follows: Mahadev and I (champion) are
>> both PMC members on ZK, Enis Söztutar is active on HBase and
>> interested in using Curator for that project (which already uses ZK
>> heavily).
>
>OK, that's lovely.
>
>
>>
>> Patrick
>>
>>> On Tue, Feb 26, 2013 at 10:53 AM, Henry Saputra
>>><henry.saputra@gmail.com> wrote:
>>>> Ah no, I was not suggesting about Curator to become subproject of ZK.
>>>>I
>>>> just afraid that if Curator is going as incubator it will end up as
>>>>sub of
>>>> ZK as merging process.
>>>>
>>>> Like Greg has mentioned in another reply, I would prefer Curator to be
>>>> merged as a higher level ZK client. Surely project like HBase and
>>>>others
>>>> that relying on ZK would appreciate simpler client to ZK.
>>>>
>>>> Thanks,
>>>>
>>>> Henry
>>>>
>>>>
>>>> On Mon, Feb 25, 2013 at 11:23 PM, Patrick Hunt <phunt@apache.org>
>>>>wrote:
>>>>
>>>>> On Mon, Feb 25, 2013 at 11:01 PM, Henry Saputra
>>>>><henry.saputra@gmail.com>
>>>>> wrote:
>>>>> > So isnt this similar to HCatalog which relying on Hive metadata
>>>>>service
>>>>> > that ends up as sub project of Apache Hive?
>>>>> >
>>>>>
>>>>> I was against having Curator as a sub when it came up on the original
>>>>> discussion thread, I still am.
>>>>>
>>>>> Patrick
>>>>>
>>>>> >
>>>>> >
>>>>> > On Mon, Feb 25, 2013 at 7:10 PM, Greg Stein <gstein@gmail.com>
>>>>>wrote:
>>>>> >
>>>>> >> My concern is that we're looking at two "new" committers, rather
>>>>>than
>>>>> >> a Curator community. Following normal Incubator work, Curator
>>>>>would
>>>>> >> build a community for itself. But then we'd have a community
>>>>> >> *distinct* from that of Zookeeper. And it really looks like
this
>>>>> >> should be part of Zookeeper itself -- a more capable and
>>>>>easier-to-use
>>>>> >> client.
>>>>> >>
>>>>> >> So I question the incubation of this. Why do we want to build
a
>>>>> >> new/separate project? Why isn't this just part of Zookeeper
right
>>>>>from
>>>>> >> the start?
>>>>> >>
>>>>> >> I would suggest that this work is placed on a branch within
>>>>>Zookeeper.
>>>>> >> That Jordan and Jay become committers on that branch (not
>>>>>necessarily
>>>>> >> Zookeeper trunk). Over time, the branch can be folded into trunk,
>>>>> >> along with all the various tests, doc, and other artifacts that
I
>>>>>see
>>>>> >> in the GitHub repository. And hopefully that Jordan and Jay
become
>>>>> >> regular committers (and PMC members!) of the Zookeeper project
>>>>>itself.
>>>>> >>
>>>>> >> The current Zookeeper client can remain for backwards compat,
and
>>>>>the
>>>>> >> Curator work can become the next-gen client.
>>>>> >>
>>>>> >> Honestly, in my opnion, incubating this project seems like it
>>>>>would
>>>>> >> create a distinct community, and really doesn't seem like it
would
>>>>> >> serve the Zookeeper community.
>>>>> >>
>>>>> >> All that said, I am not familiar with the Zookeeper or Curator
>>>>> >> communities. But from this read, I don't think Incubation is
the
>>>>>right
>>>>> >> approach. I would rather push for a more direct incorporation
of
>>>>> >> Curator directly into Zookeeper. (use the short-form IP
>>>>>clearance) ...
>>>>> >> so, unless somebody can help me understand the communities and
>>>>> >> situation better, I'm -1 (binding) on this incubation. I'd rather
>>>>>see
>>>>> >> combined, rather than distinct, communities from the start.
>>>>> >>
>>>>> >> Thanks,
>>>>> >> -g
>>>>> >>
>>>>> >> On Mon, Feb 25, 2013 at 8:14 PM, Jordan Zimmerman
>>>>> >> <jordan@jordanzimmerman.com> wrote:
>>>>> >> > Hello,
>>>>> >> >
>>>>> >> > I would like to propose that Curator to be an Apache Incubator
>>>>> project.
>>>>> >> >
>>>>> >> > The proposal can be found here:
>>>>> >> http://wiki.apache.org/incubator/CuratorProposal
>>>>> >> >
>>>>> >> > I have included the contents of the proposal below.
>>>>> >> >
>>>>> >> > Sincerely,
>>>>> >> >
>>>>> >> > Jordan Zimmerman
>>>>> >> >
>>>>> >> > ===================
>>>>> >> >
>>>>> >> > = Curator - ZooKeeper client wrapper and rich ZooKeeper
>>>>>framework =
>>>>> >> >
>>>>> >> > == Abstract ==
>>>>> >> >
>>>>> >> > Curator is a set of Java libraries that make using Apache
>>>>>ZooKeeper
>>>>> much
>>>>> >> easier. While ZooKeeper comes bundled with a Java client, using
>>>>>the
>>>>> client
>>>>> >> is non-trivial and error prone.
>>>>> >> >
>>>>> >> > == Proposal ==
>>>>> >> >
>>>>> >> > Curator is a set of Java libraries that make using Apache
>>>>>ZooKeeper
>>>>> much
>>>>> >> easier. While ZooKeeper comes bundled with a Java client, using
>>>>>the
>>>>> client
>>>>> >> is non-trivial and error prone. It consists of three components
>>>>>that
>>>>> build
>>>>> >> on each other. Curator Client is a replacement for the bundled
>>>>>ZooKeeper
>>>>> >> class that takes care of some low-level housekeeping and provides
>>>>>some
>>>>> >> useful utilities. Curator Framework is a high-level API that
>>>>>greatly
>>>>> >> simplifies using ZooKeeper. It adds many features that build
on
>>>>> ZooKeeper
>>>>> >> and handles the complexity of managing connections to the
>>>>>ZooKeeper
>>>>> cluster
>>>>> >> and retrying operations. Curator Recipes consists of
>>>>>implementations of
>>>>> >> some of the common ZooKeeper ³recipes². Additionally, Curator
>>>>>Test is
>>>>> >> included which includes utilities to help with unit testing
>>>>> ZooKeeper-based
>>>>> >> applications.
>>>>> >> >
>>>>> >> > == Background ==
>>>>> >> >
>>>>> >> > Curator was initially developed by Netflix to make writing
>>>>> >> ZooKeeper-based applications easier and more reliable. Curator
was
>>>>> >> open-sourced by Netflix on !GitHub as an Apache 2.0 licensed
>>>>>project in
>>>>> >> July 2011. During this time Curator has been formally released
>>>>>many
>>>>> times
>>>>> >> and has gained widespread adoption.
>>>>> >> >
>>>>> >> > == Rationale ==
>>>>> >> >
>>>>> >> > New users of ZooKeeper are surprised to learn that a significant
>>>>> amount
>>>>> >> of connection management must be done manually. For example,
when
>>>>>the
>>>>> >> ZooKeeper client connects to the ensemble it must negotiate
a new
>>>>> session,
>>>>> >> etc. This takes some time. If you use a ZooKeeper client API
>>>>>before the
>>>>> >> connection process has completed, ZooKeeper will throw an
>>>>>exception.
>>>>> These
>>>>> >> types of exceptions are referred to as ³recoverable² errors.
>>>>> >> > Curator automatically handles connection management, greatly
>>>>> simplifying
>>>>> >> client code. Instead of directly using the ZooKeeper APIs you
use
>>>>> Curator
>>>>> >> APIs that internally check for connection completion and wrap
each
>>>>> >> ZooKeeper API in a retry loop. Curator uses a retry mechanism
to
>>>>>handle
>>>>> >> recoverable errors and automatically retry operations. The method
>>>>>of
>>>>> retry
>>>>> >> is customizable. Curator comes bundled with several
>>>>>implementations
>>>>> >> (ExponentialBackoffRetry, etc.) or custom implementations can
be
>>>>> written.
>>>>> >> >
>>>>> >> > The ZooKeeper documentation describes many possible uses
for
>>>>>ZooKeeper
>>>>> >> calling each a ³recipe². While the distribution comes bundled
>>>>>with a few
>>>>> >> implementations of these recipes, most ZooKeeper users will
need
>>>>>to
>>>>> >> manually implement one or more of the recipes. Implementing
a
>>>>>ZooKeeper
>>>>> >> recipe is not trivial. Besides the connection handling issues,
>>>>>there are
>>>>> >> numerous edge cases that are not well documented that must be
>>>>> considered.
>>>>> >> For example, many recipes require that an ephemeral-sequential
>>>>>node be
>>>>> >> created. New users of ZooKeeper will not know that there is
an
>>>>>edge
>>>>> case in
>>>>> >> ephemeral-sequential node creation that requires you to put
a
>>>>>special
>>>>> >> ³marker² in the node¹s name so that you can search for the
>>>>>created node
>>>>> if
>>>>> >> an I/O failure occurs. This is but one of many edge cases that
>>>>>are not
>>>>> well
>>>>> >> documented but are handled by Curator.
>>>>> >> >
>>>>> >> > = Current Status =
>>>>> >> >
>>>>> >> > == Meritocracy ==
>>>>> >> >
>>>>> >> > Curator was initially developed by Jordan Zimmerman in
2011 at
>>>>> Netflix.
>>>>> >> Developers external to Netflix provided feedback, suggested
>>>>>features and
>>>>> >> fixes and implemented extensions of Curator. Netflix's
>>>>>engineering team
>>>>> has
>>>>> >> since maintained the project and has been dedicated towards
its
>>>>> >> improvement. Contributors to Curator include developers from
>>>>>multiple
>>>>> >> organizations around the world. Curator will be a meritocracy
as
>>>>>it
>>>>> enters
>>>>> >> the Incubator and beyond.
>>>>> >> >
>>>>> >> > == Community ==
>>>>> >> >
>>>>> >> > Curator is currently used by a number of organizations
all over
>>>>>the
>>>>> >> world. Curator has an active and growing user and developer
>>>>>community
>>>>> with
>>>>> >> active participation in the [[
>>>>> http://groups.google.com/group/curator-users]]
>>>>> >> mailing list and at its !Github home: [[
>>>>> >> https://github.com/Netflix/curator]].
>>>>> >> >
>>>>> >> > Since open sourcing the project, there have been fifteen
>>>>>individuals
>>>>> >> from various organizations who have contributed code.
>>>>> >> >
>>>>> >> > == Core Developers ==
>>>>> >> >
>>>>> >> > The core developers for Curator are:
>>>>> >> >  * Jordan Zimmerman
>>>>> >> >  * Jay Zarfoss
>>>>> >> >
>>>>> >> > Jordan has contributed towards Apache ZooKeeper and both
Jordan
>>>>>and
>>>>> Jay
>>>>> >> are familiar with Apache principles and philosophy for community
>>>>>driven
>>>>> >> software development.
>>>>> >> >
>>>>> >> > == Alignment ==
>>>>> >> >
>>>>> >> > Curator is a natural complement for Apache ZooKeeper. Java
>>>>>users of
>>>>> >> ZooKeeper will naturally want to use Curator. When Curator
>>>>>graduates
>>>>> from
>>>>> >> Incubator it may be useful to distribute Curator artifacts as
>>>>>part of
>>>>> >> ZooKeeper releases as the preferred/recommended client side
>>>>>library.
>>>>> >> Further, at graduation a determination can be made as to whether
>>>>>Curator
>>>>> >> should become a Top Level Project or be merged into ZooKeeper
>>>>>itself.
>>>>> >> >
>>>>> >> > = Known Risks =
>>>>> >> >
>>>>> >> > == Orphaned Products ==
>>>>> >> >
>>>>> >> > Curator is already deployed in production at multiple companies
>>>>>and
>>>>> they
>>>>> >> are actively participating in creating new features. Curator
is
>>>>>getting
>>>>> >> traction with developers and thus the risks of it being orphaned
>>>>>are
>>>>> >> minimal.
>>>>> >> >
>>>>> >> > == Inexperience with Open Source ==
>>>>> >> >
>>>>> >> > All code developed for Curator has been open sourced by
Netflix
>>>>>under
>>>>> >> Apache 2.0 license.  All committers to Curator are intimately
>>>>>familiar
>>>>> with
>>>>> >> the Apache model for open-source development and are experienced
>>>>>with
>>>>> >> working with new contributors.
>>>>> >> >
>>>>> >> > == Homogeneous Developers ==
>>>>> >> >
>>>>> >> > The initial committers are from a single organization.
However,
>>>>>we
>>>>> >> expect that once approved for incubation, the project will
>>>>>attract new
>>>>> >> contributors from diverse organizations and will thus grow
>>>>>organically.
>>>>> The
>>>>> >> submission of patches from developers from several different
>>>>> organizations
>>>>> >> is a strong indication that Curator will be widely adopted.
>>>>> >> >
>>>>> >> > == Reliance on Salaried Developers ==
>>>>> >> >
>>>>> >> > It is expected that Curator will be developed on salaried
and
>>>>> volunteer
>>>>> >> time, although all of the initial developers will work on it
>>>>>mainly on
>>>>> >> salaried time.
>>>>> >> >
>>>>> >> > == Relationships with Other Apache Products ==
>>>>> >> >
>>>>> >> > Curator depends upon other Apache Projects: Apache ZooKeeper,
>>>>>Apache
>>>>> >> Log4J, and multiple Apache Commons components. Its build depends
>>>>>upon
>>>>> >> Apache Maven. Notably, there is interest from other Apache
>>>>>Projects
>>>>> such as
>>>>> >> HBase in adopting Curator as the client library for ZooKeeper.
>>>>>Apache
>>>>> James
>>>>> >> Mailbox has already incorporated Curator.
>>>>> >> >
>>>>> >> > == An Excessive Fascination with the Apache Brand ==
>>>>> >> >
>>>>> >> > We would like Curator to become an Apache project to further
>>>>>foster a
>>>>> >> healthy community of contributors and consumers around the
>>>>>project.
>>>>>  Since
>>>>> >> Curator directly interacts with Apache ZooKeeper and solves
an
>>>>>important
>>>>> >> problem of many ZooKeeper users, residing in the Apache Software
>>>>> Foundation
>>>>> >> will increase interaction with the larger community.
>>>>> >> >
>>>>> >> > = Documentation =
>>>>> >> >
>>>>> >> >  * Curator wiki at GitHub:
>>>>>https://github.com/Netflix/curator/wiki
>>>>> >> >  * Curator issues at GitHub:
>>>>> https://github.com/Netflix/curator/issues
>>>>> >> >  * Curator javadoc at GitHub:
>>>>>http://netflix.github.com/curator/doc/
>>>>> >> >
>>>>> >> > = Initial Source =
>>>>> >> >
>>>>> >> >  * git://github.com/Netflix/curator.git
>>>>> >> >
>>>>> >> > == Source and Intellectual Property Submission Plan ==
>>>>> >> >
>>>>> >> >  * The initial source is already licensed under the Apache
>>>>>License,
>>>>> >> Version 2.0.
>>>>>https://github.com/Netflix/curator/blob/master/LICENSE.txt
>>>>> >> >
>>>>> >> > == External Dependencies ==
>>>>> >> >
>>>>> >> > The required external dependencies are all Apache License
or
>>>>> compatible
>>>>> >> licenses. Following components with non-Apache licenses are
>>>>>enumerated:
>>>>> >> >
>>>>> >> >  * org.slf4j: MIT-like License
>>>>> >> >  * org.mockito: MIT-like License
>>>>> >> >
>>>>> >> > == Cryptography ==
>>>>> >> >
>>>>> >> > Curator contains no known cryptography.
>>>>> >> >
>>>>> >> > = Required Resources =
>>>>> >> >
>>>>> >> > == Mailing lists ==
>>>>> >> >
>>>>> >> >  * curator-private (with moderated subscriptions)
>>>>> >> >  * curator-dev
>>>>> >> >  * curator-commits
>>>>> >> >  * curator-user
>>>>> >> >
>>>>> >> > == Github Repositories ==
>>>>> >> >
>>>>> >> > http://github.com/apache/curator
>>>>>git://git.apache.org/curator.git
>>>>> >> >
>>>>> >> > == Issue Tracking ==
>>>>> >> >
>>>>> >> > JIRA Curator (CURATOR)
>>>>> >> >
>>>>> >> > == Other Resources ==
>>>>> >> >
>>>>> >> > The existing code already has unit and integration tests
so we
>>>>>would
>>>>> >> like a Jenkins instance to run them whenever a new patch is
>>>>>submitted.
>>>>> This
>>>>> >> can be added after project creation.
>>>>> >> >
>>>>> >> > = Initial Committers =
>>>>> >> >
>>>>> >> >  * Jordan Zimmerman (jzimmerman at netflix dot com)
>>>>> >> >  * Jay Zarfoss (jzarfoss at netflix dot com)
>>>>> >> >
>>>>> >> > = Affiliations =
>>>>> >> >
>>>>> >> >  * Jordan Zimmerman, Netflix
>>>>> >> >  * Jay Zarfoss, Netflix
>>>>> >> >
>>>>> >> > = Sponsors =
>>>>> >> >
>>>>> >> > == Champion ==
>>>>> >> >
>>>>> >> >  * Patrick Hunt
>>>>> >> >
>>>>> >> > == Nominated Mentors ==
>>>>> >> >
>>>>> >> >  * Patrick Hunt
>>>>> >> >  * Enis Söztutar
>>>>> >> >  * Mahadev Konar
>>>>> >> >
>>>>> >> > == Sponsoring Entity ==
>>>>> >> >
>>>>> >> >  * Apache Incubator PMC
>>>>> >>
>>>>> >> 
>>>>>---------------------------------------------------------------------
>>>>> >> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>>>> >> For additional commands, e-mail: general-help@incubator.apache.org
>>>>> >>
>>>>> >>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>>>> For additional commands, e-mail: general-help@incubator.apache.org
>>>>>
>>>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>>> For additional commands, e-mail: general-help@incubator.apache.org
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: general-help@incubator.apache.org
>>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>For additional commands, e-mail: general-help@incubator.apache.org
>


---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org


Mime
View raw message