incubator-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Benson Margulies <bimargul...@gmail.com>
Subject Re: [PROPOSAL] Curator for the Apache Incubator
Date Tue, 26 Feb 2013 16:32:23 GMT
If you think that the right destination for curator is as part of ZK,
then it would be good to see substantive participation of the ZK PMC
in the incubation. The goal should be to 'graduate' by having the
curator community be granted karma at ZK and the code folded in. This
would require, I think, the ZK community to supply at least one
mentor, and to have a plan in advance for the eventual votes based on
success in the incubator.

On Tue, Feb 26, 2013 at 10:53 AM, Henry Saputra <henry.saputra@gmail.com> wrote:
> Ah no, I was not suggesting about Curator to become subproject of ZK. I
> just afraid that if Curator is going as incubator it will end up as sub of
> ZK as merging process.
>
> Like Greg has mentioned in another reply, I would prefer Curator to be
> merged as a higher level ZK client. Surely project like HBase and others
> that relying on ZK would appreciate simpler client to ZK.
>
> Thanks,
>
> Henry
>
>
> On Mon, Feb 25, 2013 at 11:23 PM, Patrick Hunt <phunt@apache.org> wrote:
>
>> On Mon, Feb 25, 2013 at 11:01 PM, Henry Saputra <henry.saputra@gmail.com>
>> wrote:
>> > So isnt this similar to HCatalog which relying on Hive metadata service
>> > that ends up as sub project of Apache Hive?
>> >
>>
>> I was against having Curator as a sub when it came up on the original
>> discussion thread, I still am.
>>
>> Patrick
>>
>> >
>> >
>> > On Mon, Feb 25, 2013 at 7:10 PM, Greg Stein <gstein@gmail.com> wrote:
>> >
>> >> My concern is that we're looking at two "new" committers, rather than
>> >> a Curator community. Following normal Incubator work, Curator would
>> >> build a community for itself. But then we'd have a community
>> >> *distinct* from that of Zookeeper. And it really looks like this
>> >> should be part of Zookeeper itself -- a more capable and easier-to-use
>> >> client.
>> >>
>> >> So I question the incubation of this. Why do we want to build a
>> >> new/separate project? Why isn't this just part of Zookeeper right from
>> >> the start?
>> >>
>> >> I would suggest that this work is placed on a branch within Zookeeper.
>> >> That Jordan and Jay become committers on that branch (not necessarily
>> >> Zookeeper trunk). Over time, the branch can be folded into trunk,
>> >> along with all the various tests, doc, and other artifacts that I see
>> >> in the GitHub repository. And hopefully that Jordan and Jay become
>> >> regular committers (and PMC members!) of the Zookeeper project itself.
>> >>
>> >> The current Zookeeper client can remain for backwards compat, and the
>> >> Curator work can become the next-gen client.
>> >>
>> >> Honestly, in my opnion, incubating this project seems like it would
>> >> create a distinct community, and really doesn't seem like it would
>> >> serve the Zookeeper community.
>> >>
>> >> All that said, I am not familiar with the Zookeeper or Curator
>> >> communities. But from this read, I don't think Incubation is the right
>> >> approach. I would rather push for a more direct incorporation of
>> >> Curator directly into Zookeeper. (use the short-form IP clearance) ...
>> >> so, unless somebody can help me understand the communities and
>> >> situation better, I'm -1 (binding) on this incubation. I'd rather see
>> >> combined, rather than distinct, communities from the start.
>> >>
>> >> Thanks,
>> >> -g
>> >>
>> >> On Mon, Feb 25, 2013 at 8:14 PM, Jordan Zimmerman
>> >> <jordan@jordanzimmerman.com> wrote:
>> >> > Hello,
>> >> >
>> >> > I would like to propose that Curator to be an Apache Incubator
>> project.
>> >> >
>> >> > The proposal can be found here:
>> >> http://wiki.apache.org/incubator/CuratorProposal
>> >> >
>> >> > I have included the contents of the proposal below.
>> >> >
>> >> > Sincerely,
>> >> >
>> >> > Jordan Zimmerman
>> >> >
>> >> > ===================
>> >> >
>> >> > = Curator - ZooKeeper client wrapper and rich ZooKeeper framework =
>> >> >
>> >> > == Abstract ==
>> >> >
>> >> > Curator is a set of Java libraries that make using Apache ZooKeeper
>> much
>> >> easier. While ZooKeeper comes bundled with a Java client, using the
>> client
>> >> is non-trivial and error prone.
>> >> >
>> >> > == Proposal ==
>> >> >
>> >> > Curator is a set of Java libraries that make using Apache ZooKeeper
>> much
>> >> easier. While ZooKeeper comes bundled with a Java client, using the
>> client
>> >> is non-trivial and error prone. It consists of three components that
>> build
>> >> on each other. Curator Client is a replacement for the bundled ZooKeeper
>> >> class that takes care of some low-level housekeeping and provides some
>> >> useful utilities. Curator Framework is a high-level API that greatly
>> >> simplifies using ZooKeeper. It adds many features that build on
>> ZooKeeper
>> >> and handles the complexity of managing connections to the ZooKeeper
>> cluster
>> >> and retrying operations. Curator Recipes consists of implementations of
>> >> some of the common ZooKeeper “recipes”. Additionally, Curator Test is
>> >> included which includes utilities to help with unit testing
>> ZooKeeper-based
>> >> applications.
>> >> >
>> >> > == Background ==
>> >> >
>> >> > Curator was initially developed by Netflix to make writing
>> >> ZooKeeper-based applications easier and more reliable. Curator was
>> >> open-sourced by Netflix on !GitHub as an Apache 2.0 licensed project in
>> >> July 2011. During this time Curator has been formally released many
>> times
>> >> and has gained widespread adoption.
>> >> >
>> >> > == Rationale ==
>> >> >
>> >> > New users of ZooKeeper are surprised to learn that a significant
>> amount
>> >> of connection management must be done manually. For example, when the
>> >> ZooKeeper client connects to the ensemble it must negotiate a new
>> session,
>> >> etc. This takes some time. If you use a ZooKeeper client API before the
>> >> connection process has completed, ZooKeeper will throw an exception.
>> These
>> >> types of exceptions are referred to as “recoverable” errors.
>> >> > Curator automatically handles connection management, greatly
>> simplifying
>> >> client code. Instead of directly using the ZooKeeper APIs you use
>> Curator
>> >> APIs that internally check for connection completion and wrap each
>> >> ZooKeeper API in a retry loop. Curator uses a retry mechanism to handle
>> >> recoverable errors and automatically retry operations. The method of
>> retry
>> >> is customizable. Curator comes bundled with several implementations
>> >> (ExponentialBackoffRetry, etc.) or custom implementations can be
>> written.
>> >> >
>> >> > The ZooKeeper documentation describes many possible uses for ZooKeeper
>> >> calling each a “recipe”. While the distribution comes bundled with a
few
>> >> implementations of these recipes, most ZooKeeper users will need to
>> >> manually implement one or more of the recipes. Implementing a ZooKeeper
>> >> recipe is not trivial. Besides the connection handling issues, there are
>> >> numerous edge cases that are not well documented that must be
>> considered.
>> >> For example, many recipes require that an ephemeral-sequential node be
>> >> created. New users of ZooKeeper will not know that there is an edge
>> case in
>> >> ephemeral-sequential node creation that requires you to put a special
>> >> “marker” in the node’s name so that you can search for the created
node
>> if
>> >> an I/O failure occurs. This is but one of many edge cases that are not
>> well
>> >> documented but are handled by Curator.
>> >> >
>> >> > = Current Status =
>> >> >
>> >> > == Meritocracy ==
>> >> >
>> >> > Curator was initially developed by Jordan Zimmerman in 2011 at
>> Netflix.
>> >> Developers external to Netflix provided feedback, suggested features and
>> >> fixes and implemented extensions of Curator. Netflix's engineering team
>> has
>> >> since maintained the project and has been dedicated towards its
>> >> improvement. Contributors to Curator include developers from multiple
>> >> organizations around the world. Curator will be a meritocracy as it
>> enters
>> >> the Incubator and beyond.
>> >> >
>> >> > == Community ==
>> >> >
>> >> > Curator is currently used by a number of organizations all over the
>> >> world. Curator has an active and growing user and developer community
>> with
>> >> active participation in the [[
>> http://groups.google.com/group/curator-users]]
>> >> mailing list and at its !Github home: [[
>> >> https://github.com/Netflix/curator]].
>> >> >
>> >> > Since open sourcing the project, there have been fifteen individuals
>> >> from various organizations who have contributed code.
>> >> >
>> >> > == Core Developers ==
>> >> >
>> >> > The core developers for Curator are:
>> >> >  * Jordan Zimmerman
>> >> >  * Jay Zarfoss
>> >> >
>> >> > Jordan has contributed towards Apache ZooKeeper and both Jordan and
>> Jay
>> >> are familiar with Apache principles and philosophy for community driven
>> >> software development.
>> >> >
>> >> > == Alignment ==
>> >> >
>> >> > Curator is a natural complement for Apache ZooKeeper. Java users of
>> >> ZooKeeper will naturally want to use Curator. When Curator graduates
>> from
>> >> Incubator it may be useful to distribute Curator artifacts as part of
>> >> ZooKeeper releases as the preferred/recommended client side library.
>> >> Further, at graduation a determination can be made as to whether Curator
>> >> should become a Top Level Project or be merged into ZooKeeper itself.
>> >> >
>> >> > = Known Risks =
>> >> >
>> >> > == Orphaned Products ==
>> >> >
>> >> > Curator is already deployed in production at multiple companies and
>> they
>> >> are actively participating in creating new features. Curator is getting
>> >> traction with developers and thus the risks of it being orphaned are
>> >> minimal.
>> >> >
>> >> > == Inexperience with Open Source ==
>> >> >
>> >> > All code developed for Curator has been open sourced by Netflix under
>> >> Apache 2.0 license.  All committers to Curator are intimately familiar
>> with
>> >> the Apache model for open-source development and are experienced with
>> >> working with new contributors.
>> >> >
>> >> > == Homogeneous Developers ==
>> >> >
>> >> > The initial committers are from a single organization. However, we
>> >> expect that once approved for incubation, the project will attract new
>> >> contributors from diverse organizations and will thus grow organically.
>> The
>> >> submission of patches from developers from several different
>> organizations
>> >> is a strong indication that Curator will be widely adopted.
>> >> >
>> >> > == Reliance on Salaried Developers ==
>> >> >
>> >> > It is expected that Curator will be developed on salaried and
>> volunteer
>> >> time, although all of the initial developers will work on it mainly on
>> >> salaried time.
>> >> >
>> >> > == Relationships with Other Apache Products ==
>> >> >
>> >> > Curator depends upon other Apache Projects: Apache ZooKeeper, Apache
>> >> Log4J, and multiple Apache Commons components. Its build depends upon
>> >> Apache Maven. Notably, there is interest from other Apache Projects
>> such as
>> >> HBase in adopting Curator as the client library for ZooKeeper. Apache
>> James
>> >> Mailbox has already incorporated Curator.
>> >> >
>> >> > == An Excessive Fascination with the Apache Brand ==
>> >> >
>> >> > We would like Curator to become an Apache project to further foster
a
>> >> healthy community of contributors and consumers around the project.
>>  Since
>> >> Curator directly interacts with Apache ZooKeeper and solves an important
>> >> problem of many ZooKeeper users, residing in the Apache Software
>> Foundation
>> >> will increase interaction with the larger community.
>> >> >
>> >> > = Documentation =
>> >> >
>> >> >  * Curator wiki at GitHub: https://github.com/Netflix/curator/wiki
>> >> >  * Curator issues at GitHub:
>> https://github.com/Netflix/curator/issues
>> >> >  * Curator javadoc at GitHub: http://netflix.github.com/curator/doc/
>> >> >
>> >> > = Initial Source =
>> >> >
>> >> >  * git://github.com/Netflix/curator.git
>> >> >
>> >> > == Source and Intellectual Property Submission Plan ==
>> >> >
>> >> >  * The initial source is already licensed under the Apache License,
>> >> Version 2.0. https://github.com/Netflix/curator/blob/master/LICENSE.txt
>> >> >
>> >> > == External Dependencies ==
>> >> >
>> >> > The required external dependencies are all Apache License or
>> compatible
>> >> licenses. Following components with non-Apache licenses are enumerated:
>> >> >
>> >> >  * org.slf4j: MIT-like License
>> >> >  * org.mockito: MIT-like License
>> >> >
>> >> > == Cryptography ==
>> >> >
>> >> > Curator contains no known cryptography.
>> >> >
>> >> > = Required Resources =
>> >> >
>> >> > == Mailing lists ==
>> >> >
>> >> >  * curator-private (with moderated subscriptions)
>> >> >  * curator-dev
>> >> >  * curator-commits
>> >> >  * curator-user
>> >> >
>> >> > == Github Repositories ==
>> >> >
>> >> > http://github.com/apache/curator git://git.apache.org/curator.git
>> >> >
>> >> > == Issue Tracking ==
>> >> >
>> >> > JIRA Curator (CURATOR)
>> >> >
>> >> > == Other Resources ==
>> >> >
>> >> > The existing code already has unit and integration tests so we would
>> >> like a Jenkins instance to run them whenever a new patch is submitted.
>> This
>> >> can be added after project creation.
>> >> >
>> >> > = Initial Committers =
>> >> >
>> >> >  * Jordan Zimmerman (jzimmerman at netflix dot com)
>> >> >  * Jay Zarfoss (jzarfoss at netflix dot com)
>> >> >
>> >> > = Affiliations =
>> >> >
>> >> >  * Jordan Zimmerman, Netflix
>> >> >  * Jay Zarfoss, Netflix
>> >> >
>> >> > = Sponsors =
>> >> >
>> >> > == Champion ==
>> >> >
>> >> >  * Patrick Hunt
>> >> >
>> >> > == Nominated Mentors ==
>> >> >
>> >> >  * Patrick Hunt
>> >> >  * Enis Söztutar
>> >> >  * Mahadev Konar
>> >> >
>> >> > == Sponsoring Entity ==
>> >> >
>> >> >  * Apache Incubator PMC
>> >>
>> >> ---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>> >> For additional commands, e-mail: general-help@incubator.apache.org
>> >>
>> >>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: general-help@incubator.apache.org
>>
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: general-unsubscribe@incubator.apache.org
For additional commands, e-mail: general-help@incubator.apache.org


Mime
View raw message